Compare commits

..
Author SHA1 Message Date
jschoubben 5831eec770 Merge pull request 'R4g+R5 (mesh-controller): the controller asks no build, build-agent and the merge gate go, register by reference (design 47)' (#448) from feat/r4g into main
Reviewed-on: #448
2026-10-23 17:50:22 +00:00
jschoubben c9a7102bc5 Set aside a retired seat's consume the trunk catalogue still holds, and say it, so a group that retires a seat passes its check
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group feat/r4g failed: a member failed
mesh/check checking c9a7102bc53b on 791786d5b87f | novox/mesh-controller#448 in group feat/r4g
2026-10-23 18:05:06 +02:00
jschoubben 67279199e9 Say the trunk rule as ADR 0534 §6 has it in the registration's comment
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-controller/internal/broker.TestTheCataloguesDerivedSubjectsMeet: agreement_catalogue_test.go:130: judge…
2026-10-23 17:19:39 +02:00
jschoubben 9d2a9e62d1 Merge remote-tracking branch 'origin/fix/forge-requires-no-status' into feat/r4g 2026-10-23 17:18:58 +02:00
jschoubben b345d10bec Pin mesh-sdk at feat/r4g's head a2c813a6, its test fixture fix, with go.sum tidied
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/checks/config.TestTheUnmovedListHoldsNoModuleTheTrunksDoesNot: realtree_test.go:14: mesh-cata…
2026-10-23 17:06:08 +02:00
jschoubben 13842a7234 Grant the builder the build seats' streams' information and messages by name, so its take-over at start can tell an ask its key names is on its stream (issue 1910)
mesh/delivery-group group feat/r4g rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#464: checks/health/go.mod names mesh-sdk a9560890ee26, which is neither on mesh-sdk's trunk nor the group's…
2026-10-23 16:59:03 +02:00
jschoubben 4238318601 Pin mesh-sdk at feat/r4g's head a9560890, re-vendor it and capture its conformance events there (R4g) 2026-10-23 16:54:07 +02:00
jschoubben 758c8e84e5 Bound the build seats' workers' deliveries, so an ask no run ends is offered at most five times even when the holder that says it is poison dies too (issue 1907) 2026-10-23 16:42:13 +02:00
jschoubben 9222575ae9 Vendor the dependencies again for R4g's go.mod (mesh-sdk 9205222a, the mesh-host replace as before), so build-agent compiles the controller offline in the group's first tier (issue 1305) 2026-10-23 16:38:31 +02:00
jschoubben c5b8fca8bd Revert "Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6)"
This reverts commit 9c4bf1f08b.
2026-10-23 16:37:40 +02:00
jschoubben ff543d3def Take the watchman's grant on the build seats back: a waiting ask has no pulse, a taken one is read from the steps (issue 1908) 2026-10-23 16:35:28 +02:00
jschoubben ef211faefb Prove the registration by hand reads back as a build and is held by the sweep, and name the store check as the test's limit (issue 1888, C8b) 2026-10-23 16:34:29 +02:00
jschoubben 3c247e80af Fix the view grant test's consumer subjects (issue 1887 A8) 2026-10-23 16:32:43 +02:00
jschoubben bb0d5e17db Grant the watchman the build seats' accept subjects and the view the seat workers' information, each by name, so the backlog and the waiting-ask pulse read live (issue 1887 A8) 2026-10-23 16:32:32 +02:00
jschoubben 24ad859629 Record the checksums of mesh-sdk feat/r4g 9205222a for the group's modules (R4g) 2026-10-23 16:26:11 +02:00
jschoubben 5ccc87f4d0 Capture mesh-catalog at feat/r4g (2246ccc4) with the templates its manifests name and its predefined.json, and lay the copy out as a repository's root
The catalogue's manifests moved their content into template files and fall back to predefined keys, so a copy of
module.json alone no longer passes the catalogue's own check; the bus's Dockerfile is gone from the catalogue.
2026-10-23 16:24:14 +02:00
jschoubben 68c1220e87 Pin mesh-sdk at feat/r4g's head 9205222a, the merge of lanes A, B, S8 and F1, and capture its conformance events there
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:22:07 +02:00
jschoubben 5b5cd6160a Merge branch 'feat/r4g-asks' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben d7d70cd539 Merge branch 'feat/r4g-assigner' into feat/r4g 2026-10-23 16:20:32 +02:00
jschoubben df70ca43a7 Take the controller's publish grants on node-build-agent and mesh-build-machine away, with the cancelled sets and the redelivery of its own asks: it asks no build seat (ADR 0525 §10, C3a) 2026-10-23 16:13:43 +02:00
jschoubben f60c742d93 Prove the grant through the send of the bus's machine (overTheBus.declare), so removing the write from the live path fails the test (issue 1394, review round 1 item 4) 2026-10-23 16:13:42 +02:00
jschoubben 2db5dcdcb4 Write the hand act of register --reference after the registration was made, not before, and test the why and the record (issue 1888, C8b) 2026-10-23 16:12:16 +02:00
jschoubben d9cdb3615d Test that the builder's record names its key and fingerprint after a plan that moves it (issue 1851, C5 criterion) 2026-10-23 16:11:07 +02:00
jschoubben dec3b83e66 Test that the builder, a build module and the controller are taken from the builder's keys and nothing reaches node-build-agent (issue 1851, C2 criterion) 2026-10-23 16:11:07 +02:00
jschoubben 2e36a63096 Pin mesh-sdk at feat/r4g's head c3915ffb, the merge of lane B's layouts, so every r4g module builds against one SDK
go.sum gains its lines from go mod tidy once mesh-sdk feat/r4g is pushed.
2026-10-23 16:01:15 +02:00
jschoubben eb8286fc68 Merge feat/r4g-assigner into feat/r4g: lane B's grants bucket and its writer beside lane C's deletions
The controller's readable states, its bucket lists and the composed golden file keep lane C's removal of the
checks-running and plans-building buckets and the build agent's check subjects, and add lane B's grants bucket.
2026-10-23 15:59:52 +02:00
jschoubben 5540fd4f85 Find the catalogue's repository by its URL or seat source, and fail a plan whose catalogue repository is nowhere, so a take-in from a key no longer waits for ever (issue 1902) 2026-10-23 15:57:30 +02:00
jschoubben e419d0182f Delete the builder, the build asks, the build queue and the merge gate: the builder's keys are the only way a module is built (issues 1851, 1299, 1318; C2b-C5, C10)
The controller asks no build of anybody (ADR 0501 §5, ADR 0525 clause 6, ADR 0503 §3). Every module of a walk is taken from the
builder's done key; a key not done yet waits, named in the plan, and never fails it.

- C2b/C2c: internal/builder and cmd/mesh-controller's asks go (askABuild, onlyTheBuildersOwn, buildOneAsked, takeIn from an
  outcome, buildAgentsOwn, the builder's say and its fallback, the daemon hearing an outcome). The server follows the retired
  seats' built subjects only to let go of them, because the installer's carried user list names them.
- C2d: replay, queue, cancel, clear, kill, pause, resume, delivery-check, check-here and merge-gate are refused by name.
- C3: the check queues (CHECKS_READY, CHECKS_OTHER) and their workers, the checks-running and plans-building buckets and
  their grants, the build queue and its hold (plans_building.go, queue.go, link/holds.go, builds_nats.go, checks_nats.go).
- C4: merge_gate*.go, the pull request's repo-check path, the Checked event and its layers; a pull request asks the
  controller nothing but to tell mesh-config what its head declares.
- C5: the builds bucket is written only by key_takein.
- C10 (help and refusals): the terminal's help lists only what is served.

Integration tests of the plan's tiers, sends, stops and retries now run on a stand-in builder with a done key for every
module (walk_keys_scaffold_integration_test.go); the tests of the asks, the queue, the merge gate and the replays are deleted.
2026-10-23 15:55:15 +02:00
jschoubben 5ca4f14479 Derive a worker per kind for the build seats and grant their users to put asks, so a build node pulls delivery, check and recheck in order and a session cannot ask (issue 1887 A6, ADR 0525 §2) 2026-10-23 15:54:45 +02:00
jschoubben c43883910d Write grants.<module>.<fp> once the bus's user list carrying a module's manifest at that fingerprint is sent, so the assigner's sixth rule reads that its credential exists (issue 1394, subtask C9b; ADR 0528 clause 2, ADR 0429 §6) 2026-10-23 15:39:40 +02:00
jschoubben 548681cc4e Declare the controller's grants bucket, written by the controller alone and readable through mesh-controller.grants, so the assigner can read its sixth rule's key (issue 1394, subtask C9a; ADR 0528 clause 2); the SDK pin moves to the layouts' commit, captured beside 2026-10-23 15:39:40 +02:00
jschoubben 9c4bf1f08b Delete vendor/ and the route proxy's: the build module fetches the modules, the root image builds with -mod=mod (issue 1305, C6) 2026-10-23 15:14:58 +02:00
jschoubben 6300a5b31c Delete merge-check.sh, check-nested-modules.sh, the store-tests list and RELEASE-57: the checker runs the build module's steps, nested modules by convention (issue 1294, C7a and C7b) 2026-10-23 15:14:01 +02:00
jschoubben 505694c424 Register a module at artifacts the store holds, by hand, recorded as a build and a hand act, so the controller can be delivered without the builder (issue 1888, C8a and C8b) 2026-10-23 15:13:17 +02:00
jschoubben 72186815e3 Delete the build-agent module and the builder program: the build module builds, the checker checks (issue 1851, C2a) 2026-10-23 15:07:57 +02:00
jschoubben 526eed99fa Drop the toolchain image from the facts and the dependency edges: no module is compiled in one any more (issue 1851, C1b) 2026-10-23 15:07:29 +02:00
jschoubben b506852388 Move the Go build source reader out of internal/builder so the builder can go (issue 1851, C1a) 2026-10-23 15:06:31 +02:00
mesh-tools 791786d5b8 Merge pull request 'Wait out a resolver that does not answer in a build's registry step, and say it was the resolver (issue 1748)' (#447) from fix/1748-registry-wait-retries-a-lookup into main 2026-10-23 11:20:15 +00:00
jschoubben 0b312218f9 Wait out a failed lookup of the registry's name in a build, and say it was the resolver (issue 1748)
mesh/delivery delivered
mesh/check pass 0b312218f90f on c834e4f82137 | unit 1762 run, 0 kept, 4 skipped | integration 3040 run, 0 kept, 16 skipped | over bound: integration (…
2026-10-23 13:03:17 +02:00
mesh-tools c834e4f821 Merge pull request 'Keep every refused build by its id, so a plan never settles a refused build built and sends the old version (issue 1713)' (#446) from fix/1713-keep-every-refusal into main 2026-10-23 10:58:08 +00:00
jschoubben 149f89e8f9 Keep every refused registration by its build id, so a Retry never settles it built
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792753090890815254 (novox/mesh-controller at c834e4f8 into ma…
mesh/check pass 149f89e8f9bf on 0173d9c70e59 | unit 1759 run, 0 kept, 4 skipped | integration 3037 run, 0 kept, 16 skipped | over bound: integration (…
A build heard and recorded and not registered kept its refusal only when a
plan was open or the plans were held. A tier's first failure fails its plan,
so later refusals were lost; a Retry then settled those builds built from
their records, sent the modules' old versions and reported the plan done.
A build without a readable manifest never reached its plan at all.

Keep the refusal in the store and the process whatever the plans' state, and
fail the asking plan by the build's id when the module is not known.

Issue 1713.
2026-10-23 12:13:33 +02:00
826 changed files with 18268 additions and 47609 deletions
-1
View File
@@ -2,7 +2,6 @@
# A built binary. The lab writes one here when pointed at the repository root by mistake; # A built binary. The lab writes one here when pointed at the repository root by mistake;
# built artifacts belong in build/, which is already ignored. # built artifacts belong in build/, which is already ignored.
/mesh-builder
/mesh-controller /mesh-controller
/postgres-provisioner /postgres-provisioner
/redis-provisioner /redis-provisioner
+1 -33
View File
@@ -46,17 +46,6 @@ image:
@echo @echo
@docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The builder ships as an image too, because it is a module the mesh assigns rather than a program
# somebody starts on a machine by hand.
BUILDER_IMAGE ?= mesh-builder:$(VERSION)
BUILDER_DEV_TAG ?= mesh-builder:development
builder-image:
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) -f cmd/mesh-builder/Dockerfile -t $(BUILDER_IMAGE) -t $(BUILDER_DEV_TAG) .
@echo
@docker image inspect $(BUILDER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The provisioner ships as an image too, because it is the thing that makes a sealed credential # The provisioner ships as an image too, because it is the thing that makes a sealed credential
# true on a machine -- and the mesh cannot, having discarded the plaintext. # true on a machine -- and the mesh cannot, having discarded the plaintext.
PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION) PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION)
@@ -90,26 +79,6 @@ redis-provisioner-image:
@echo @echo
@docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
# The whole check. Raises a database, runs the repository's own check against it -- merge-check.sh,
# the very script `mesh/repo-check` runs -- and takes the database down again, including when the
# check fails, which is why the teardown is not conditional and the script's exit status is the
# target's.
#
# **It calls the script rather than restating it** (novox/hq issue 431): this target used to list its
# own steps, its formatting step walked vendor/ where the script's does not, and it failed on
# third-party code no change could fix -- so a developer's check and the gate disagreed, and the steps
# after formatting never ran through it. The script is the one list of steps; this target only gives it
# a database (MESH_TEST_POSTGRES, exported above).
#
# **Two levels** (issue 895): the unit level, `go test ./...` untagged, timed and held to a bound; and the integration
# level, `-tags integration`, the tests that start a bus, a store or another process. Each is said with its time.
#
# **Packages in parallel, under the race detector, each test on a bus of its own** (internal/testbus),
# as the script runs them: a bus per test, of the release the mesh runs, makes the suite the same in any
# order, and the timeout bounds a hang to a failure with a stack, never a stalled gate.
check: postgres
@sh merge-check.sh ; status=$$? ; docker rm -f -v $(PG_CONTAINER) >/dev/null 2>&1 || true ; exit $$status
# The unit level alone (untagged: no bus, no store, no other process), which a developer runs continuously. The tests # The unit level alone (untagged: no bus, no store, no other process), which a developer runs continuously. The tests
# that need a bus or the store carry `//go:build integration` and run in `check`, not here; without a database the # that need a bus or the store carry `//go:build integration` and run in `check`, not here; without a database the
# store's tests would skip rather than fail anyway. # store's tests would skip rather than fail anyway.
@@ -119,8 +88,7 @@ test:
vet: vet:
go vet ./... go vet ./...
# Quick steps for a developer, not part of `check`. The directories gofmt reads must be the ones # A quick step for a developer: the checker runs the build module's own formatting and vet (ADR 0501 §5, D7).
# merge-check.sh lists, never `.`, which walks vendor/ (novox/hq issue 431).
fmt: fmt:
@unformatted=$$(gofmt -l cmd internal examples) ; \ @unformatted=$$(gofmt -l cmd internal examples) ; \
if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi if [ -n "$$unformatted" ] ; then echo "not gofmt'd:" ; echo "$$unformatted" ; exit 1 ; fi
-6
View File
@@ -1,6 +0,0 @@
# Release 57: config and provisioning
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
-29
View File
@@ -1,29 +0,0 @@
#!/bin/sh
# The unit tests of every Go module of its own below this repository's root, run from the root, for merge-check.sh
# and for the test that tries it on a tree (merge_check_nested_test.go). Its own file so that the test needs no
# particular merge-check.sh: the check that judges a change to merge-check.sh is main's.
set -eu
cgo="${cgo:-0}"
race="${race:-}"
# nested_modules vets and tests every Go module of its own below the root, `modules/<name>/go.mod` (novox/hq ADR 0420
# §5, issue 1025): `./...` stops at a nested go.mod, so a module that lives whole in this repository would have its
# unit tests run by nobody. None is a pass. Each runs in its own directory, outside the vendor directory this
# repository's own packages are built from; a nested module vendors what the forge holds (its own `vendor/`, so the
# check runs offline), as this repository's root module does (issue 1019). A failing module fails the check; the others are still run.
nested_modules() {
nested=0
for gomod in modules/*/go.mod; do
[ -e "$gomod" ] || continue
dir=$(dirname "$gomod")
echo "nested module $dir:"
if ! (cd "$dir" && GOFLAGS= CGO_ENABLED=0 go vet ./... &&
GOFLAGS= CGO_ENABLED="${cgo:-0}" go test ${race:-} -count=1 -timeout 30m ./...); then
echo "nested module $dir FAILED"
nested=1
fi
done
return $nested
}
nested_modules
-22
View File
@@ -1,22 +0,0 @@
ARG ALPINE_BASE=alpine:3
ARG GO_BASE=golang:1.25-alpine
# The builder, as a module ships one.
#
# Not FROM scratch, unlike the control plane: this one runs git and a container client, so it
# needs a filesystem with them on it. That is the honest cost of a machine whose job is to build —
# and it is why building is a MODULE on a machine that has a runtime rather than something the
# control plane does (novox/hq ADR 0005).
FROM ${GO_BASE} AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -buildvcs=false -ldflags '-s -w' -o /mesh-builder ./cmd/mesh-builder
FROM ${ALPINE_BASE}
# git to clone what it is asked to build, and the docker client to build and push it. The daemon
# is the machine's, reached through its socket — a build machine shares the runtime it was given
# rather than running one inside itself.
RUN apk add --no-cache git docker-cli
COPY --from=build /mesh-builder /usr/local/bin/mesh-builder
ENTRYPOINT ["/usr/local/bin/mesh-builder"]
-235
View File
@@ -1,235 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// **The build agent says whether it is busy, and being asked drains it** (novox/hq ADR 0335 §1, §5). Its
// container declares `busy` as `mesh-builder busy`: the node-engine runs it inside the container before it
// replaces it. It exits 0 while a build is in hand and 1 when none is; and from the first ask the agent takes
// no new build request, so its answer ends once the build in hand ends — a replaced builder finishes the check
// it runs, instead of the build queue handing it out again from the start (issue 594).
//
// Two marks in the container's own temporary directory, as the ready mark (ready.go): a new container starts
// without them, and a restarted one removes them before it takes work again.
//
// - in hand: written by the agent when it claims a request, before it reads whether it is drained; removed
// when the build ends.
// - drained: written by the ask, before it reads whether a build is in hand.
//
// Each side writes its own mark before it reads the other's, so an ask racing a fetch sees the build in hand,
// or the agent sees the drain and hands the request back; never neither.
func inHandMark() string { return filepath.Join(os.TempDir(), "mesh-builder.in-hand") }
func drainMark() string { return filepath.Join(os.TempDir(), "mesh-builder.drained") }
// The drain lapses when no ask has come for drainLapse: the node-engine asks every 20 seconds for as long as it
// really waits, so an agent whose replacement was withdrawn — the declaration put back, the walk stopped —
// takes work again rather than staying drained for ever (review of ADR 0335). And an agent drained for longer
// than the node-engine's bound and the slack says it is not ready: the replacement it was drained for never
// came, which is a fault to see.
const (
drainLapse = 2 * time.Minute
drainTooLong = 15*time.Minute + 5*time.Minute
)
// drainedAt is when the drain began and when it was last asked, from its mark; false when there is no mark.
func drainedAt() (began, asked time.Time, ok bool) {
info, err := os.Stat(drainMark())
if err != nil {
return time.Time{}, time.Time{}, false
}
asked = info.ModTime()
began = asked
if raw, err := os.ReadFile(drainMark()); err == nil {
if t, err := time.Parse(time.RFC3339, strings.TrimSpace(string(raw))); err == nil && !t.After(asked) {
began = t
}
}
return began, asked, true
}
// drained says the agent was asked whether it is busy within drainLapse: it takes no new build request.
func drained() bool { return drainedSince(time.Now()) }
func drainedSince(now time.Time) bool {
_, asked, ok := drainedAt()
return ok && now.Sub(asked) < drainLapse
}
// drainedTooLong is the health check's second half: nil unless the agent has been drained, asked all along, for
// longer than its node-engine's bound and the slack.
func drainedTooLong(now time.Time) error {
began, asked, ok := drainedAt()
if !ok || now.Sub(asked) >= drainLapse || now.Sub(began) <= drainTooLong {
return nil
}
return fmt.Errorf("drained since %s and still asked: the replacement its node-engine waits for has not come "+
"within %s (novox/hq ADR 0335)", began.UTC().Format(time.RFC3339), drainTooLong)
}
// inHand is true while this process has claimed a build and not yet released it. The mark in the temporary directory
// is only believed while it is: a mark with no build of this process behind it is stale (a crash, a fault in an
// earlier path) and is removed, because left alone it hands back every ask for ever (issue 1768).
var inHand atomic.Bool
// markRetry is how long an agent that cannot mark a build in hand takes no work, and markFault when it last failed to.
// Taking nothing is the point: an ask fetched and handed back at once, many times a second, spends all its deliveries.
const markRetry = 30 * time.Second
var (
markMu sync.Mutex
markFault time.Time
markSaid string
)
// markFaulty says the agent could not mark a build in hand and has not yet been able to since: it takes no work. Once
// markRetry has passed it tries the mark itself, with no ask in hand, so that an ask is fetched only when marking works.
func markFaulty(now time.Time) bool {
markMu.Lock()
failed := markFault
markMu.Unlock()
if failed.IsZero() {
return false
}
if now.Sub(failed) < markRetry {
return true
}
f, err := os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil && errors.Is(err, os.ErrExist) && !inHand.Load() {
_ = os.Remove(inHandMark())
f, err = os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
}
if err != nil {
markFailed(err)
return true
}
_ = f.Close()
_ = os.Remove(inHandMark())
markMu.Lock()
markFault, markSaid = time.Time{}, ""
markMu.Unlock()
fmt.Fprintf(os.Stderr, "the mark of a build in hand can be made again (%s): taking builds\n", inHandMark())
return false
}
// markFailed notes that a build could not be marked in hand, and says it when it is a new reason.
func markFailed(err error) {
markMu.Lock()
defer markMu.Unlock()
markFault = time.Now()
if said := err.Error(); said != markSaid {
markSaid = said
fmt.Fprintf(os.Stderr, "cannot mark a build in hand (%s), so this agent takes no build for %s and tries again: %v\n",
inHandMark(), markRetry, err)
}
}
// claim marks a request in hand, unless the agent is drained: then nothing is in hand, and the request goes
// back to the queue.
func claim() bool {
if !inHand.Load() {
// Nothing of this process is in hand, so a mark already there is stale.
if err := os.Remove(inHandMark()); err == nil {
fmt.Fprintf(os.Stderr, "removed the mark of a build in hand (%s): no build of this agent is in hand\n", inHandMark())
}
}
// Made, never written over: one build at a time, and a mark already there is a build in hand.
f, err := os.OpenFile(inHandMark(), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil {
// Unmarked, an ask would say idle while this build runs: the build is not taken.
markFailed(err)
return false
}
_, err = f.WriteString(time.Now().UTC().Format(time.RFC3339) + "\n")
if closeErr := f.Close(); err == nil {
err = closeErr
}
if err != nil {
_ = os.Remove(inHandMark())
markFailed(err)
return false
}
if drained() {
_ = os.Remove(inHandMark())
fmt.Fprintln(os.Stderr, "asked whether busy, so this agent takes no new build: handed back")
return false
}
inHand.Store(true)
return true
}
// holding says what is in hand, for the ask to print: written once the request is read.
func holding(what string) {
_ = os.WriteFile(inHandMark(), []byte(time.Now().UTC().Format(time.RFC3339)+" "+what+"\n"), 0o644)
}
// released ends the build in hand.
func released() {
inHand.Store(false)
if err := os.Remove(inHandMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
fmt.Fprintf(os.Stderr, "cannot remove the mark of the build in hand (%s): %v\n", inHandMark(), err)
}
}
// undrained clears both marks at the start of a holder: a container started again is asked afresh.
func undrained() error {
for _, mark := range []string{drainMark(), inHandMark()} {
if err := os.Remove(mark); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove %s: %w", mark, err)
}
}
return nil
}
// busy is the ask: it drains the agent, then says whether a build is in hand. Nil — exit 0 — while one is.
func busy() (string, error) {
// The mark keeps when the drain began while it is asked within drainLapse, and is written again on every
// ask: its time is the last ask.
began := time.Now().UTC()
if was, _, ok := drainedAt(); ok && drained() {
began = was.UTC()
}
if err := os.WriteFile(drainMark(), []byte(began.Format(time.RFC3339)+"\n"), 0o644); err != nil {
// Not drained, it could take the next build: said, and answered busy only for what is in hand now.
fmt.Fprintf(os.Stderr, "cannot mark this agent drained (%s): %v\n", drainMark(), err)
}
held, err := os.ReadFile(inHandMark())
if err != nil {
return "", errors.New("idle: no build in hand, and this agent takes no new one")
}
since, what, _ := strings.Cut(strings.TrimSpace(string(held)), " ")
if what == "" {
what = "a build"
}
return fmt.Sprintf("busy: %s, in hand since %s; taking no new build", what, since), nil
}
var (
roomMu sync.Mutex
roomSaid bool
)
// noRoom says this node is under the floor of free space (novox/hq issue 1834): it takes no work, so another build node
// takes the ask instead of this one taking it and ending it in an error. Said once when it starts and again when it ends.
func noRoom(workspace string) bool {
err := builder.RoomToWork("/", workspace)
roomMu.Lock()
defer roomMu.Unlock()
if err != nil && !roomSaid {
fmt.Fprintf(os.Stderr, "taking no builds: %v\n", err)
} else if err == nil && roomSaid {
fmt.Fprintf(os.Stderr, "there is room again: taking builds\n")
}
roomSaid = err != nil
return err != nil
}
-239
View File
@@ -1,239 +0,0 @@
package main
import (
"github.com/novox/mesh-controller/internal/builder"
"os"
"strings"
"sync"
"testing"
"time"
)
// `mesh-builder busy` answers 0 while a build is in hand and non-zero when idle, and after its first ask the
// agent takes no new build request (novox/hq ADR 0335 §5).
func TestBusyAnswersWhileABuildIsInHandAndDrainsTheAgent(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("an agent never asked is drained")
}
// A build is claimed and runs.
if !claim() {
t.Fatal("an agent never asked refused a build")
}
holding("novox/mesh-controller (check-280)")
said, err := busy()
if err != nil || !strings.Contains(said, "novox/mesh-controller (check-280)") {
t.Fatalf("an agent with a build in hand did not answer busy with what it holds: %q %v", said, err)
}
// From that ask on it takes nothing new: the fetch is not made, and one fetched is handed back.
if !drained() {
t.Fatal("the ask did not drain the agent")
}
if claim() {
t.Fatal("a drained agent took a new build")
}
// The build in hand is still in hand after the refused claim, and busy goes on saying so.
if _, err := busy(); err != nil {
t.Fatalf("a refused claim ended the build in hand: %v", err)
}
// It ends: idle, and still drained.
released()
if _, err := busy(); err == nil {
t.Fatal("an agent with nothing in hand answered busy")
}
if claim() {
t.Fatal("a drained agent took a build after its last one ended")
}
// Started again, it is asked afresh.
if err := undrained(); err != nil {
t.Fatal(err)
}
if drained() || !claim() {
t.Fatal("a holder started again is still drained")
}
released()
}
// An ask racing a fetch sees the build in hand, or the agent sees the drain and hands the request back: never
// neither, which would replace the container under a build just taken.
func TestAnAskRacingAFetchNeverMissesTheBuild(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
for i := 0; i < 200; i++ {
if err := undrained(); err != nil {
t.Fatal(err)
}
var claimed bool
var askErr error
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); claimed = claim() }()
go func() { defer wg.Done(); _, askErr = busy() }()
wg.Wait()
if claimed && askErr != nil {
t.Fatalf("round %d: the agent took a build and the ask said idle", i)
}
released()
}
}
// A drain no longer asked lapses: an agent whose replacement was withdrawn after the first ask (a put-back, a
// revert, a stopped walk) takes work again after drainLapse, rather than staying drained with its health green.
func TestADrainNoLongerAskedLapses(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if !drained() || claim() {
t.Fatal("an agent just asked takes work")
}
// The last ask was three minutes ago.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if drained() {
t.Fatal("a drain no longer asked for three minutes still holds")
}
if !claim() {
t.Fatal("an agent whose drain lapsed takes no work")
}
released()
}
// An agent drained, and asked all along, for longer than its node-engine's bound and the slack is not ready: the
// replacement it was drained for never came. One asked again after a lapse starts a new drain.
func TestAnAgentDrainedTooLongIsNotReady(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := undrained(); err != nil {
t.Fatal(err)
}
taking("mesh-build-machine")
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
now := time.Now()
if err := drainedTooLong(now); err != nil {
t.Fatalf("an agent drained a moment ago is not ready: %v", err)
}
// The drain began 25 minutes ago, and the node-engine asked a moment ago.
began := now.Add(-25 * time.Minute).UTC().Format(time.RFC3339)
if err := os.WriteFile(drainMark(), []byte(began+"\n"), 0o644); err != nil {
t.Fatal(err)
}
err := drainedTooLong(time.Now())
if err == nil || !strings.Contains(err.Error(), "has not come") {
t.Fatalf("an agent drained for 25 minutes reads ready: %v", err)
}
// Each ask keeps when the drain began.
if _, err := busy(); err == nil {
t.Fatal("an idle agent answered busy")
}
if err := drainedTooLong(time.Now()); err == nil {
t.Fatal("an ask started the drain again")
}
// The asks stopped: the drain lapsed, and so did the fault.
old := time.Now().Add(-3 * time.Minute)
if err := os.Chtimes(drainMark(), old, old); err != nil {
t.Fatal(err)
}
if err := drainedTooLong(time.Now()); err != nil {
t.Fatalf("a lapsed drain still reads not ready: %v", err)
}
}
// An ask this build does not know exits non-zero: an older image asked `busy` reads idle, not busy for the
// node-engine's whole bound.
func TestAnUnknownCommandFails(t *testing.T) {
was := os.Args
t.Cleanup(func() { os.Args = was })
os.Args = []string{"mesh-builder", "some-later-word"}
if err := run(); err == nil {
t.Fatal("an unknown command exited 0")
}
os.Args = []string{"mesh-builder", "help"}
if err := run(); err != nil {
t.Fatalf("help failed: %v", err)
}
}
// 1768/AC2: a mark naming no build of this agent (left by a crash or a fault) is stale: the next ask removes it and is
// run, never fetched and handed back in a loop.
func TestAStaleInHandMarkIsRemovedAndTheAskRuns(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
inHand.Store(false)
if err := undrained(); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(inHandMark(), []byte("2026-10-21T18:45:06Z\n"), 0o644); err != nil {
t.Fatal(err)
}
if !claim() {
t.Fatal("a stale in-hand mark made the agent hand back an ask")
}
// A live one — this agent's own build in hand — is never taken over.
if claim() {
t.Fatal("a second build was taken while one is in hand")
}
if _, err := busy(); err != nil {
t.Fatalf("the build in hand was lost: %v", err)
}
released()
}
// 1768/AC3: an agent that cannot mark a build in hand for another reason takes no work, instead of fetching asks and
// handing them back; it takes work again once marking works.
func TestAnAgentThatCannotMarkABuildTakesNoWorkUntilItCan(t *testing.T) {
dir := t.TempDir()
t.Setenv("TMPDIR", dir)
inHand.Store(false)
markFault, markSaid = time.Time{}, ""
t.Cleanup(func() { markFault, markSaid = time.Time{}, "" })
if err := undrained(); err != nil {
t.Fatal(err)
}
// The mark's place is a directory that cannot be removed or made over: a directory named as the mark.
if err := os.MkdirAll(inHandMark()+"/x", 0o755); err != nil {
t.Fatal(err)
}
if claim() {
t.Fatal("an agent that cannot mark a build took it")
}
if !markFaulty(time.Now()) {
t.Fatal("an agent that cannot mark a build still takes work")
}
// Still failing after the retry time: still takes nothing, and no ask was needed to find out.
if !markFaulty(time.Now().Add(2 * markRetry)) {
t.Fatal("an agent whose mark still cannot be made takes work")
}
if err := os.RemoveAll(inHandMark()); err != nil {
t.Fatal(err)
}
if markFaulty(time.Now().Add(4 * markRetry)) {
t.Fatal("an agent whose mark can be made again takes no work")
}
if !claim() {
t.Fatal("an agent that can mark takes no build")
}
released()
}
// 1834/AC4: given a node under the floor of free space, when the taking loop asks, then it takes nothing, and takes again
// once there is room.
func TestANodeUnderTheFloorOfFreeSpaceTakesNoWork(t *testing.T) {
was := builder.FreeBytes
t.Cleanup(func() { builder.FreeBytes = was })
builder.FreeBytes = func(string) (uint64, error) { return 2 << 30, nil }
if !noRoom(t.TempDir()) {
t.Error("a node with 2 GiB free took work")
}
builder.FreeBytes = func(string) (uint64, error) { return 200 << 30, nil }
if noRoom(t.TempDir()) {
t.Error("a node with 200 GiB free refused work")
}
}
-52
View File
@@ -1,52 +0,0 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// The build seat hands the check the judge cache the build agent declares (MESH_JUDGE_CACHE), never a path of its own.
func TestTheBuildSeatHandsTheCheckTheJudgeCacheItDeclares(t *testing.T) {
t.Setenv("MESH_JUDGE_CACHE", "/var/lib/mesh/build-agent/judge-cache")
if got := judgeCache(); got != "/var/lib/mesh/build-agent/judge-cache" {
t.Fatalf("the judge cache the manifest declares is read as %q", got)
}
t.Setenv("MESH_JUDGE_CACHE", "")
if got := judgeCache(); got != "" {
t.Fatalf("with no judge cache declared, %q is used", got)
}
}
// The siblings a check is asked to read reach the build seat's spec and, as read, the outcome (novox/hq ADR 0382).
func TestTheSiblingsAnAskNamesReachTheSpecAndTheOutcome(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Base: "main",
Siblings: []link.CheckSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", Dir: "mesh-controller"}},
SiblingsUnknown: "why"}})
if len(spec.Siblings) != 1 || spec.Siblings[0] != (builder.Sibling{Repository: "novox/mesh-controller", Number: 346,
Head: "a", Base: "main", Dir: "mesh-controller"}) || spec.SiblingsUnknown != "why" {
t.Fatalf("the spec reads %+v", spec)
}
out := siblingsOut([]builder.JudgedSibling{{Repository: "novox/mesh-controller", Number: 346, Head: "a", Base: "main", BaseCommit: "b"}})
if len(out) != 1 || out[0].BaseCommit != "b" || out[0].Number != 346 {
t.Fatalf("the outcome reads %+v", out)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Siblings != nil {
t.Fatalf("a request that names none has siblings %+v", plain.Siblings)
}
}
// 1826/AC2: the held values a check request carries reach the spec; a request that carries none leaves it empty.
func TestTheHeldValuesAnAskCarriesReachTheSpec(t *testing.T) {
spec := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog",
HeldConfig: []link.HeldConfig{{Pull: "novox/mesh-catalog#434", Head: "a", Node: "ace", Module: "n8n", Key: "media-gid",
Value: []byte("2000"), Approved: "the operator"}}}})
if len(spec.Held) != 1 || spec.Held[0].Node != "ace" || spec.Held[0].Key != "media-gid" || string(spec.Held[0].Value) != "2000" ||
spec.Held[0].Approved != "the operator" {
t.Fatalf("the spec reads %+v", spec.Held)
}
if plain := checkSpecOf(link.BuildRequest{Check: &link.CheckRequest{Owner: "o", Repo: "r"}}); plain.Held != nil {
t.Fatalf("a request that carries none has %+v", plain.Held)
}
}
-433
View File
@@ -1,433 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/micro"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// What a holder of the build seat answers for, on its own machine (novox/hq ADR 0219).
//
// **The queue is the controller's; the build running here is this machine's.** The controller can
// see and change what waits in the seat's queue, but an ask a holder already took is a process tree
// on this machine and containers in this machine's runtime, and only this machine can end them. So
// the holder serves four verbs on the seat's subjects for this machine: what it is building, kill
// it, pause, resume.
//
// **One build at a time** (ADR 0190), which is also what builder.Said assumes — a package global
// set per build — so "the build running here" is one or none, and kill names it by id so a call
// that arrives as one build ends and the next begins cannot end the wrong one.
// holder is this machine's state as a holder of the build seat.
type holder struct {
on, seat string
// workspace is where the paused flag is kept, so a holder restarted while paused stays paused
// rather than silently taking work again.
workspace string
// say publishes this machine's state: whether it takes work (link.HolderState).
say func(link.HolderState) error
// remove runs what a kill needs outside the build: the containers left behind.
remove builder.Runner
// scratch is where check builds are kept (novox/hq ADR 0350 §5): what the week's sweep lets go of. Nil where
// this machine publishes to no artifact store.
scratch builder.Scratch
mu sync.Mutex
paused bool
running *running
}
// running is the build this machine is doing.
type running struct {
request link.BuildRequest
step string
started time.Time
cancel context.CancelFunc
killed bool
// returned is the build's own work having ended, before a kill or not; outcome is what was then
// announced, and announced whether it went out.
returned bool
outcome string
announced bool
done chan struct{}
}
// pausedFile is where the flag lives in the workspace.
func pausedFile(workspace string) string { return filepath.Join(workspace, ".mesh-builder-paused") }
// newHolder reads the paused flag the workspace keeps.
func newHolder(on, seat, workspace string, say func(link.HolderState) error) *holder {
h := &holder{on: on, seat: seat, workspace: workspace, say: say, remove: plainRun}
if _, err := os.Stat(pausedFile(workspace)); err == nil {
h.paused = true
}
return h
}
// Paused is asked by the taking loop before every fetch.
func (h *holder) Paused() bool {
h.mu.Lock()
defer h.mu.Unlock()
return h.paused
}
// setPaused records the flag in the workspace first and then in memory, so what this holder says
// it is and what it would be after a restart never differ.
func (h *holder) setPaused(paused bool) error {
path := pausedFile(h.workspace)
if paused {
if err := os.MkdirAll(h.workspace, 0o755); err != nil {
return err
}
if err := os.WriteFile(path, []byte(time.Now().UTC().Format(time.RFC3339)+"\n"), 0o644); err != nil {
return fmt.Errorf("cannot keep the paused flag in %s: %w", path, err)
}
} else if err := os.Remove(path); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the paused flag %s: %w", path, err)
}
h.mu.Lock()
h.paused = paused
h.mu.Unlock()
h.announce()
return nil
}
// announce says whether this machine takes work. Never fatal: the flag is kept either way, and the
// controller reading an older state is a plan read as late rather than a build lost.
func (h *holder) announce() {
if h.say == nil {
return
}
if err := h.say(link.HolderState{On: h.on, Paused: h.Paused(), At: time.Now().UTC().Format(time.RFC3339Nano)}); err != nil {
fmt.Fprintf(os.Stderr, "cannot say whether this machine takes builds: %v\n", err)
}
}
// stateWhilePaused says this machine's state at start, and again every while it stays paused, so
// a pause outlives the events stream's retention.
func (h *holder) stateWhilePaused(ctx context.Context, every time.Duration) {
h.announce()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
if h.Paused() {
h.announce()
}
}
}
}
// expireChecks lets go of every check build older than a week (novox/hq ADR 0350 §5), at start and every while
// after. Every holder sweeps the one store; what one let go of, another finds gone.
func (h *holder) expireChecks(ctx context.Context, every time.Duration, now func() time.Time) {
sweep := func() {
if h.scratch == nil {
return
}
n, err := h.scratch.Expire(ctx, builder.CheckBuildsKept, now())
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d let go of, and not all: %v\n", builder.CheckBuildsKept, n, err)
case n > 0:
fmt.Fprintf(os.Stderr, "check builds older than %s: %d check(s)' builds let go of\n", builder.CheckBuildsKept, n)
}
}
sweep()
tick := time.NewTicker(every)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
sweep()
}
}
}
// begin records the build this machine took, with the cancel that ends it.
func (h *holder) begin(request link.BuildRequest, cancel context.CancelFunc) *running {
r := &running{request: request, started: time.Now(), cancel: cancel, done: make(chan struct{})}
h.mu.Lock()
h.running = r
h.mu.Unlock()
return r
}
// end clears it, and lets a kill waiting on it know it is over.
func (h *holder) end(r *running) {
h.mu.Lock()
if h.running == r {
h.running = nil
}
h.mu.Unlock()
close(r.done)
}
// stepped records the step a build is at, for `current`.
func (h *holder) stepped(r *running, step string) {
h.mu.Lock()
r.step = step
h.mu.Unlock()
}
// currentBuild is what `current` answers.
type currentBuild struct {
On string `json:"on"`
Paused bool `json:"paused"`
Running *struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
} `json:"running,omitempty"`
Said string `json:"said"`
}
func (h *holder) current() currentBuild {
h.mu.Lock()
defer h.mu.Unlock()
out := currentBuild{On: h.on, Paused: h.paused}
taking := "taking builds"
if h.paused {
taking = "paused, taking no new build"
}
if h.running == nil {
out.Said = fmt.Sprintf("%s is building nothing; %s", h.on, taking)
return out
}
r := h.running
elapsed := time.Since(r.started).Round(time.Second)
out.Running = &struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Step string `json:"step,omitempty"`
Started string `json:"started"`
Elapsed string `json:"elapsed"`
}{r.request.ID, r.request.Repository, r.request.Path, r.request.Ref, r.step,
r.started.UTC().Format(time.RFC3339), elapsed.String()}
out.Said = fmt.Sprintf("%s is building %s (%s) at %s for %s; %s",
h.on, r.request.Repository, r.request.ID, orNothing(r.step), elapsed, taking)
return out
}
// The bounds a kill keeps: each pass removing containers, and the wait for the build to end between
// them. The worst case — 15s, 20s, 15s — is inside what the controller waits for the answer
// (killAnswer in the controller's queue.go, 75s).
var (
killRemoves = 15 * time.Second
killWaits = 20 * time.Second
)
// kill ends the build with this id, if it is the one running here and still working: its context
// cancelled — which kills each command's process group — and the containers it started removed by
// their label, once at once and again after the build has ended, so one created while it was being
// killed is not left. The build's own goroutine announces it failed, killed by hand, and settles the
// ask so it is not redelivered; the answer says whether that happened.
func (h *holder) kill(id string) (string, error) {
h.mu.Lock()
r := h.running
if r == nil || r.request.ID != id {
doing := "nothing"
if r != nil {
doing = r.request.ID
}
h.mu.Unlock()
return "", fmt.Errorf("%s is not building %s; it is building %s. `queue` says where an ask is", h.on, id, doing)
}
if r.returned {
h.mu.Unlock()
return "", fmt.Errorf("%s on %s has already ended on its own and is saying how; `builds` shows it", id, h.on)
}
r.killed = true
cancel, done := r.cancel, r.done
h.mu.Unlock()
cancel()
removed, removeErr := h.removeContainers(id)
ended := false
select {
case <-done:
ended = true
case <-time.After(killWaits):
}
again, againErr := h.removeContainers(id)
removed += again
if removeErr == nil {
removeErr = againErr
}
containers := fmt.Sprintf("%d container(s) it started removed", removed)
if removeErr != nil {
containers = "its containers could not all be listed or removed: " + removeErr.Error()
}
if !ended {
return fmt.Sprintf("killed %s on %s: %s; the build has not finished ending yet — `builds` says when "+
"its outcome is in", id, h.on, containers), nil
}
h.mu.Lock()
outcome, announced := r.outcome, r.announced
h.mu.Unlock()
if !announced {
return fmt.Sprintf("killed %s (%s) on %s: its commands ended and %s, and its outcome could not be "+
"announced — the ask is not settled and will be handed out again", id, r.request.Repository, h.on,
containers), nil
}
return fmt.Sprintf("killed %s (%s) on %s: its commands ended, %s, and its outcome announced as failed, %s — "+
"settled, so it is not handed to another machine", id, r.request.Repository, h.on, containers, outcome), nil
}
// removeContainers is one pass of removing what the build left, bounded.
func (h *holder) removeContainers(id string) (int, error) {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
return builder.RemoveContainersOf(cleanup, h.remove, id)
}
// removeLeftBehind removes, once, what earlier holders on this machine left — called before anything is
// taken, so none of it is this holder's — and says what it did. A runtime that cannot be asked is said
// and the holder takes work all the same: what is left is waste, not a reason to stop building.
func (h *holder) removeLeftBehind() int {
cleanup, stop := context.WithTimeout(context.Background(), killRemoves)
defer stop()
n, err := builder.RemoveLeftBehind(cleanup, h.remove)
switch {
case err != nil:
fmt.Fprintf(os.Stderr, "could not look for containers earlier builds left on %s: %v\n", h.on, err)
case n > 0:
fmt.Fprintf(os.Stderr, "removed %d container(s) earlier builds left on %s\n", n, h.on)
}
return n
}
// returned records that the build's work ended, and says whether a kill came first — only then is
// the build killed; an error it ended with on its own is its own outcome.
func (h *holder) returned(r *running) bool {
h.mu.Lock()
defer h.mu.Unlock()
r.returned = true
return r.killed
}
// said records the outcome announced, and whether it went out, for a kill to answer with.
func (h *holder) said(r *running, outcome string, announced bool) {
h.mu.Lock()
r.outcome, r.announced = outcome, announced
h.mu.Unlock()
}
// handlers are the seat's verbs, as this machine answers them.
func (h *holder) handlers() map[string]link.ToolHandler {
return map[string]link.ToolHandler{
"current": func(context.Context, json.RawMessage) (any, error) { return h.current(), nil },
"kill": func(_ context.Context, raw json.RawMessage) (any, error) {
var args struct {
ID string `json:"id"`
}
if err := json.Unmarshal(raw, &args); err != nil || strings.TrimSpace(args.ID) == "" {
return nil, errors.New("kill needs the build's id")
}
said, err := h.kill(strings.TrimSpace(args.ID))
if err != nil {
return nil, err
}
return map[string]any{"said": said}, nil
},
"pause": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(true); err != nil {
return nil, err
}
said := h.on + " is paused: it takes no new build until resumed"
if c := h.current(); c.Running != nil {
said += "; " + c.Running.ID + " runs on and finishes"
}
return map[string]any{"said": said, "paused": true}, nil
},
"resume": func(context.Context, json.RawMessage) (any, error) {
if err := h.setPaused(false); err != nil {
return nil, err
}
return map[string]any{"said": h.on + " takes builds again", "paused": false}, nil
},
}
}
func orNothing(s string) string {
if s == "" {
return "its start"
}
return s
}
// plainRun runs a command outside any build: no line reaches a build's log, because the build whose
// log it would be is the one being ended.
func plainRun(ctx context.Context, dir, name string, args ...string) (string, error) {
cmd := exec.CommandContext(ctx, name, args...)
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
return string(out), fmt.Errorf("%s %s: %w: %s", name, strings.Join(args, " "), err, strings.TrimSpace(string(out)))
}
return string(out), nil
}
// announcement is what this holder answers discovery with (novox/hq ADR 0195, ADR 0197): this
// machine's verbs of the seat, in the shape every tool runtime announces a seat's verb — kind seat,
// the module answering, the seat, scope node, the machine, its description and argument schema — so
// the console finds `<node>/node-build-agent.kill` by searching, as it finds any seat's verb.
//
// One service per machine, named for the seat and identified by the machine, so the answer is this
// machine's four verbs and nothing more. The verbs are the compiled seat row's: a build machine has no
// store, and what it serves is what this binary was built to serve.
func announcement(seat, module, node string) micro.Info {
s, _ := catalogue.SeatNamed(seat)
var endpoints []micro.EndpointInfo
for _, v := range s.Serves {
schema, _ := json.Marshal(v.Input)
endpoints = append(endpoints, micro.EndpointInfo{
Name: seat + "__" + v.Name,
Subject: link.NodeSeatToolSubject(seat, v.Name, node),
// The queue group the verbs are served in, as every runtime announces its own.
QueueGroup: "seat." + seat,
Metadata: map[string]string{
"kind": "seat", "module": module, "tool": v.Name, "seat": seat, "scope": "node",
"node": node, "interchangeable": "false", "description": v.Description, "schema": string(schema),
},
})
}
return micro.Info{
ServiceIdentity: micro.ServiceIdentity{Name: seat, ID: node, Version: "0.1.0",
Metadata: map[string]string{"seat": seat, "scope": "node", "node": node, "module": module}},
Description: "what the build running on " + node + " is, and ending, pausing and resuming it (novox/hq ADR 0219)",
Endpoints: endpoints,
}
}
// moduleOf is the module a credential was issued for: its user is `<node>.<module>`.
func moduleOf(user string) string {
if _, module, ok := strings.Cut(user, "."); ok && module != "" {
return module
}
return "build-agent"
}
-172
View File
@@ -1,172 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A holder paused stays paused across its restart: the flag is kept in its workspace (novox/hq ADR
// 0219), and what it says about itself follows.
func TestAPausedHolderStaysPausedAcrossARestart(t *testing.T) {
workspace := t.TempDir()
var said []link.HolderState
h := newHolder("ace", link.TheBuildMachine, workspace, func(s link.HolderState) error {
said = append(said, s)
return nil
})
if h.Paused() {
t.Fatal("a new holder starts paused")
}
if _, err := h.handlers()["pause"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if !h.Paused() || len(said) != 1 || !said[0].Paused || said[0].On != "ace" {
t.Fatalf("paused: %v, said %+v", h.Paused(), said)
}
again := newHolder("ace", link.TheBuildMachine, workspace, nil)
if !again.Paused() {
t.Fatal("restarted, the holder forgot it was paused")
}
if _, err := again.handlers()["resume"](context.Background(), json.RawMessage(`{}`)); err != nil {
t.Fatal(err)
}
if newHolder("ace", link.TheBuildMachine, workspace, nil).Paused() {
t.Fatal("resumed, the holder came back paused")
}
}
// kill ends the build with that id — its context, which ends its commands — removes what it left by
// label, and refuses an id it is not building.
func TestKillEndsTheBuildRunningHereAndNoOther(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1\n", nil
}
return "", nil
}
kill := h.handlers()["kill"]
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`)); err == nil {
t.Fatal("killed a build while none ran")
}
building, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1", Repository: "novox/a"}, cancel)
h.stepped(r, "image")
if c := h.current(); c.Running == nil || c.Running.ID != "build-1" || c.Running.Step != "image" {
t.Fatalf("current says %+v", c)
}
if _, err := kill(context.Background(), json.RawMessage(`{"id":"build-2"}`)); err == nil ||
!strings.Contains(err.Error(), "build-1") {
t.Fatalf("killed another id: %v", err)
}
// The build's own goroutine: it ends when its context does, as a build's commands do, and
// announces what came of it.
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, true)
}
h.end(r)
}()
answer, err := kill(context.Background(), json.RawMessage(`{"id":"build-1"}`))
if err != nil {
t.Fatal(err)
}
if building.Err() == nil {
t.Fatal("the build's context was not cancelled")
}
if !r.killed {
t.Error("the build is not marked killed, so it would be announced as an ordinary failure")
}
said := answer.(map[string]any)["said"].(string)
if !strings.Contains(said, "2 container(s)") || !strings.Contains(said, "announced as failed") || !strings.Contains(said, link.KilledByHand) {
t.Errorf("kill said %q", said)
}
// Removed at the kill and again once the build had ended: a container made in between is caught.
if len(removed) != 6 || !strings.Contains(removed[0], "label=mesh.build=build-1") || !strings.Contains(removed[3], "label=mesh.build=build-1") {
t.Errorf("removed %v", removed)
}
if c := h.current(); c.Running != nil {
t.Errorf("after the kill current says %+v", c)
}
}
// A holder announces this machine's verbs of the seat as the console reads a seat's verb, and no more.
func TestAHolderAnnouncesItsMachinesVerbsForTheConsole(t *testing.T) {
info := announcement(link.TheBuildMachine, moduleOf("ace.build-agent"), "ace")
if info.Name != "node-build-agent" || info.ID != "ace" || len(info.Endpoints) != 4 {
t.Fatalf("announced %s/%s with %d endpoints", info.Name, info.ID, len(info.Endpoints))
}
kill := info.Endpoints[1]
md := kill.Metadata
if kill.Subject != "mesh.seat.node-build-agent.tool.kill.ace" || kill.QueueGroup != "seat.node-build-agent" || md["kind"] != "seat" || md["seat"] != "node-build-agent" ||
md["scope"] != "node" || md["node"] != "ace" || md["tool"] != "kill" || md["module"] != "build-agent" ||
!strings.Contains(md["description"], "killed by hand") || !strings.Contains(md["schema"], `"id"`) {
t.Fatalf("kill is announced as %+v", kill)
}
body, err := json.Marshal(info)
if err != nil || len(body) > 8*1024 {
t.Fatalf("the answer is %d bytes (%v)", len(body), err)
}
}
// A build that ended on its own as the kill arrived says what it did: the kill is refused, and its
// own error is its outcome. One whose outcome could not be announced is not said to be settled.
func TestAKillArrivingAfterTheBuildEndedIsRefusedAndAnUnannouncedKillSaysSo(t *testing.T) {
h := newHolder("ace", link.TheBuildMachine, t.TempDir(), nil)
h.remove = func(context.Context, string, string, ...string) (string, error) { return "", nil }
_, cancel := context.WithCancel(context.Background())
r := h.begin(link.BuildRequest{ID: "build-1"}, cancel)
if killed := h.returned(r); killed {
t.Fatal("a build nobody killed reads as killed")
}
if _, err := h.kill("build-1"); err == nil || !strings.Contains(err.Error(), "ended on its own") {
t.Fatalf("killed a build that had ended: %v", err)
}
h.end(r)
building, cancel := context.WithCancel(context.Background())
r = h.begin(link.BuildRequest{ID: "build-2"}, cancel)
go func() {
<-building.Done()
if h.returned(r) {
h.said(r, link.KilledByHand, false)
}
h.end(r)
}()
said, err := h.kill("build-2")
if err != nil || strings.Contains(said, "announced as failed") || !strings.Contains(said, "could not be announced") {
t.Fatalf("kill said %q (%v)", said, err)
}
}
// **Issue 306**: a holder starting removes the containers earlier builds left on its machine, through
// the runner a kill uses, and a runtime that cannot be asked does not stop it.
func TestAHolderStartingRemovesWhatEarlierBuildsLeftHere(t *testing.T) {
h := newHolder("novox", link.TheBuildMachine, t.TempDir(), nil)
var removed []string
h.remove = func(_ context.Context, _ string, name string, args ...string) (string, error) {
removed = append(removed, name+" "+strings.Join(args, " "))
if args[0] == "ps" {
return "c1 build-1\nc2 build-1\nc3 check-here-2\n", nil
}
return "", nil
}
if n := h.removeLeftBehind(); n != 2 || len(removed) != 3 || removed[2] != "docker rm -f -v c1 c2" {
t.Fatalf("removed %d: %v", n, removed)
}
h.remove = func(context.Context, string, string, ...string) (string, error) {
return "", errors.New("no runtime")
}
if n := h.removeLeftBehind(); n != 0 {
t.Errorf("a runtime that cannot be asked removed %d", n)
}
}
-946
View File
@@ -1,946 +0,0 @@
// mesh-builder — the thing a build machine runs.
//
// It takes work from the mesh, turns a repository into artifacts, publishes them, and says what
// came out. It is **not** the control plane and it is **not** the host:
//
// - the control plane decides and never touches a machine. Building runs commands on one, and
// what the control plane may send a machine is bounded by the declaration language
// (novox/hq ADR 0005). "Run this build" is not in it, and widening the language so it could
// be would make the control plane able to run anything anywhere.
// - the host applies declarations and holds no opinion about what they contain. A host that
// also built things would need a container runtime and git, on every machine, to do something
// almost none of them will ever do.
//
// So it is a module: a program a machine runs because the mesh told it to, holding its own broker
// credential and nothing else. Compromise of a build machine is compromise of a build machine.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/url"
"os"
"os/signal"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// version is set at build time.
var version = "development"
func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "mesh-builder: %v\n", err)
os.Exit(1)
}
}
const usage = `mesh-builder — builds modules for the mesh
It consumes build requests and answers with what it made. Nothing is listened on and nothing
is dialled except the broker.
MESH_BROKER_FILE a file the mesh sealed to this machine holding the same
MESH_REGISTRY host:port to publish artifacts to, when the mesh has not said
MESH_BINDING a file the mesh wrote saying where the artifact store is
MESH_PACKAGE_BINDING a file the mesh wrote saying where the package registry is
MESH_NPM_TOKEN_FILE a file the mesh sealed holding the token for it
MESH_GENESIS_PACKAGE_REGISTRY the document the installer sealed at genesis, read while no
binding is present (novox/hq ADR 0336)
MESH_NPM_REGISTRY a package registry URL, when the mesh has not said (a person, the bootstrap)
MESH_NPM_TOKEN the token for it, likewise
MESH_NPM_SCOPE the scope it answers for (default: @novox)
MESH_WORKSPACE where to clone and build (default: a temporary directory)
MESH_BUILD_CGROUP_PARENT the slice every container a build or check starts joins, which carries the
machine's limits on builds (novox/hq issue 1798; default: the runtime's own)
It also builds one module and stops, which is how a mesh is raised — before there is a
broker to take work from or a registry to publish into:
mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]
Without --registry the artifacts stay in this machine's container runtime, named by the
digest of their own configuration. The result is printed as JSON.
mesh-builder ready
exits 0 once the agent running in this container is bound to its build work and pulls it, and
1 before: its container's health check.
mesh-builder busy
exits 0 while the agent running in this container has a build in hand, and 1 when it has none;
from the first ask the agent takes no new build: its container's busy command, asked before the
container is replaced (novox/hq ADR 0335).
`
func run() error {
if len(os.Args) > 1 {
switch os.Args[1] {
case "version":
fmt.Println(version)
return nil
case "build":
return buildOnce(context.Background(), os.Args[2:])
case "ready":
// The container's health check (novox/hq issue 544): exits non-zero until the agent takes work, and
// once it has been drained longer than its node-engine would wait to replace it (ADR 0335).
if err := ready(); err != nil {
return err
}
return drainedTooLong(time.Now())
case "busy":
// The container's busy command (novox/hq ADR 0335): drains the agent, and exits non-zero once
// nothing is in hand.
said, err := busy()
if err != nil {
return err
}
fmt.Println(said)
return nil
case "help", "-h", "--help":
fmt.Print(usage)
return nil
default:
// **An ask this build does not know fails** (novox/hq ADR 0335): an image older than `busy`, asked
// it, must read as idle — exiting 0 would hold its container for the node-engine's whole bound.
fmt.Fprint(os.Stderr, usage)
return fmt.Errorf("%q is not a command this mesh-builder knows", os.Args[1])
}
}
if err := notReady(); err != nil {
return err
}
// **The builder's own version is part of every reuse key** (novox/hq ADR 0350 §1): a builder that builds
// differently never takes a build another one made.
builder.Version = builder.SelfVersion(version)
if err := undrained(); err != nil {
return err
}
credential, err := brokerFrom()
if err != nil {
return err
}
registry, err := whereToPublish()
if err != nil {
return err
}
// A build and a check run below the machine's own work (novox/hq issue 1798): the agent itself at the
// lowest priority, which its git and go children inherit, and every container in the build slice.
_ = syscall.Setpriority(syscall.PRIO_PROCESS, 0, 19)
builder.CgroupParent = strings.TrimSpace(os.Getenv("MESH_BUILD_CGROUP_PARENT"))
workspace := os.Getenv("MESH_WORKSPACE")
if workspace == "" {
workspace = os.TempDir() + "/mesh-builder"
}
// **The mesh's name for this machine, not the container's.** A build is reported to the rest
// of the mesh, and a report whose origin reads `104cb10e105b` names something no other module
// can look up. The mesh already knows the answer and has a way to say it — `${machine:name}`
// in the environment file this module is handed — so the hostname is only what is left when
// nobody said.
on := os.Getenv("MESH_NODE")
if on == "" {
hostname, err := os.Hostname()
if err != nil {
return fmt.Errorf("this build machine has no name: nothing said MESH_NODE and the host would not say either: %w", err)
}
on = hostname
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
js, seat, err := dialFor(credential)
if err != nil {
return err
}
defer js.Close()
// **This machine's holder: paused or not, and the build it is running** (novox/hq ADR 0219). The
// paused flag is read from the workspace before anything is taken, so a holder restarted while
// paused takes nothing.
h := newHolder(on, seat, workspace, func(state link.HolderState) error {
body, err := json.Marshal(state)
if err != nil {
return err
}
_, err = js.Context().Publish(link.BuildPausedOf(seat, on), body)
return err
})
if h.Paused() {
fmt.Fprintf(os.Stderr, "paused (kept in %s): taking no build until resumed\n", pausedFile(workspace))
}
// **What an earlier holder here left goes before anything is taken** (novox/hq issue 306): a holder
// recreated mid-check left the check's store and bus running, and the ask went to another machine.
h.removeLeftBehind()
// Paused or drained, it takes nothing new (novox/hq ADR 0219, ADR 0335): a drained agent finishes the build
// in hand and then says idle, and its container is replaced between builds.
machine := link.MachineOverNATSWith(js, on, seat, link.MachineOptions{
Paused: func() bool { return h.Paused() || drained() || markFaulty(time.Now()) || noRoom(workspace) },
Taking: func() { taking(seat) },
Claim: claim,
})
defer machine.Close()
// The seat's verbs, on this machine's subjects. Only the seat that declares them: the retired
// one serves none, and a subscription its holder has no grant for would be refused for ever.
if seat == link.TheBuildMachine {
stopServing, err := link.OverNATS{Conn: js.Conn()}.ServeNodeSeatTools(seat, on, h.handlers(),
log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopServing()
// And says so, for the console to find (novox/hq ADR 0197).
stopAnnouncing, err := link.OverNATS{Conn: js.Conn()}.Announce(
announcement(seat, moduleOf(credential.User), on), log.New(os.Stderr, "", 0))
if err != nil {
return err
}
defer stopAnnouncing()
go h.stateWhilePaused(ctx, time.Hour)
}
fmt.Fprintf(os.Stderr, "building for the mesh, publishing to %s\n", registry)
publisher := builder.Registry{Address: registry, Run: builder.Command}
// Check builds are let go of after a week (novox/hq ADR 0350 §5).
h.scratch = publisher
go h.expireChecks(ctx, time.Hour, time.Now)
return machine.Take(ctx, func(ctx context.Context, work link.Build) {
defer released()
holding(work.Request().Repository + " (" + work.Request().ID + ")")
answer(ctx, publisher, on, workspace, work, h)
})
}
// dialFor opens this machine's link to whichever bus the mesh is on, and says which build seat it
// holds.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5): a build
// machine told about both would take work from one and answer on the other, and every log line would
// say it was fine.
func dialFor(credential Credential) (*broker.JetStream, string, error) {
// **The credential names the bus, and there is one** (novox/hq ADR 0131, design 28 task 5.5).
// A credential for the mesh's bus carries user, password and fingerprint beside the address,
// and that is enough to dial it, pinned.
if !credential.onTheNewBus() {
return nil, "", fmt.Errorf("the credential at hand names %q, which is not the mesh's bus", credential.URL)
}
// Named for what it is, not the controller whose code dials it (novox/hq issue 327).
host, _ := os.Hostname()
js, err := broker.DialPinned(credential.natsURL(), credential.Fingerprint, nats.Name("build agent on "+host))
if err != nil {
return nil, "", err
}
// **The seat this machine serves is the one its credential claims** (novox/hq ADR 0190, the
// handover): the mesh issues a build machine's credential naming the seat its module claims,
// and one binary serves the old role as `builder` and the new as `build-agent` from that alone.
seat := link.BuildSeatClaimed(credential.seatsClaimed())
fmt.Fprintf(os.Stderr, "taking build work as a holder of %s\n", seat)
return js, seat, nil
}
// answer does one build and says what happened, whichever way it went.
func answer(ctx context.Context, publisher builder.Publisher, on, workspace string, work link.Build, h *holder) {
request := work.Request()
// **Its own context, so it can be killed alone** (novox/hq ADR 0219): cancelled by `kill`, it ends
// this build's commands and nothing else; the machine's own context ending — a SIGTERM — still
// reaches it through the parent, and that keeps today's meaning below.
building, cancel := context.WithCancel(ctx)
defer cancel()
var mine *running
if h != nil {
mine = h.begin(request, cancel)
defer h.end(mine)
}
// **First thing, and to stdout.** A build request that arrives and produces no visible line until
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s (%s)\n", request.Repository, request.ID)
// **Everything a build says goes two ways**: to stderr, as always, and onto the bus as the
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
// One line at a time: a check's builds for its merge say their lines while its tests say theirs (novox/hq
// issue 716).
var saying sync.Mutex
say := func(step, message string) {
saying.Lock()
defer saying.Unlock()
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
if mine != nil && step != "run" && step != "output" {
h.stepped(mine, step)
}
}
builder.Said = say
defer func() { builder.Said = nil }()
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
// **A running check watches its pull request's key** (novox/hq ADR 0406 §4, issue 974): when the head or the trunk
// commit in it moves, the check is for a state nobody wants any more, and ends itself — nothing is asked of the forge
// (issue 726). A group's check, a try and a build watch nothing.
var superseded atomic.Bool
if w, ok := work.(interface {
WatchPull(context.Context, func(link.PullKey)) (func(), error)
}); ok && request.Check != nil {
stopWatching, err := w.WatchPull(building, func(k link.PullKey) {
superseded.Store(true)
say("superseded", fmt.Sprintf("the pull request's key moved to %.12s on %.12s while this checked %.12s on %.12s: stopped",
k.Head, k.Trunk, request.Ref, request.Check.Trunk))
cancel()
})
if err != nil {
fmt.Fprintf(os.Stderr, "the key of %s/%s#%d could not be watched, so its check runs on if the key moves: %v\n",
request.Check.Owner, request.Check.Repo, request.Check.Number, err)
} else {
defer stopWatching()
}
}
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Source: request.Source, DryRun: request.DryRun,
}
what := "building " + request.Repository
if request.Path != "" {
what += " at " + request.Path
}
if request.Ref != "" {
what += " on " + request.Ref
}
say("build", what)
npmrc, err := packagesFrom(on)
if err == nil {
say("packages", saysRegistry(npmrc))
}
registryHere, isRegistry := publisher.(builder.Registry)
var built builder.Result
if request.Check != nil {
// **A pull request's merge check, not a build** (novox/hq to-be 45 §9): nothing is built,
// published or registered; the verdict is the outcome.
result.Checked = request.Check
registry := ""
if r, ok := publisher.(builder.Registry); ok {
registry = r.Address
}
spec := checkSpecOf(request)
// The judge cache the build agent declares as its own data, never in the workspace: only judge builds and
// mesh-lab's replays mount it, never a check's containers, so no pull request's script can write what a judge
// is built with or kept as, or what the replays run (novox/hq ADR 0358 §6, issue 709).
spec.JudgeCache = judgeCache()
// What the merge would move, built by the check for the walk to take (novox/hq ADR 0350 §1): into the
// scratch namespace. **Never with the package registry's credential**: this is code nobody has approved
// yet, and a recipe could read the credential it is handed (issue 462). A module that resolves packages
// at build time has no reuse key anyway, so nothing is lost by refusing it here.
if isRegistry && len(spec.Builds) > 0 {
spec.BuildWith = &builder.CheckBuilding{Publish: publisher, Scratch: registryHere, Held: request.Held,
Running: request.Check.Running, Seats: request.Seats, Named: namedOf(request.Check.Named)}
}
var v builder.CheckVerdict
v, err = builder.Check(building, builder.Command, spec, workspace, registry, forgeFrom(), say)
if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside, Built: checkBuiltOf(v.Built),
Validator: v.Validator, Siblings: siblingsOut(v.Siblings), SiblingsUnknown: v.SiblingsUnknown, JudgeFrom: v.JudgeFrom}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
// A branch tried before its pull request reads its outcome from this log, whole (novox/hq issue 686).
if request.Check.Try {
if line, err := json.Marshal(triedOf(v)); err == nil {
say(link.StepTried, string(line))
}
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
// build that could not have resolved its dependencies is refused in front of the reason, not
// after a clone that then fails at npm ci.
// Every container it starts is labelled with its id, so a kill finds what outlived the
// docker client (ADR 0219).
if request.Reuse != nil && isRegistry && !request.DryRun {
// **A walk's build takes the check's build of the same key** (novox/hq ADR 0350 §2).
reuse := builder.Reuse{Scratch: registryHere}
for _, p := range request.Reuse.Pulls {
reuse.Pulls = append(reuse.Pulls, builder.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number})
}
// Only what a verdict names is taken (ADR 0350 §2).
reuse.Named = namedOf(request.Reuse.Builds)
built, err = builder.BuildReusing(building, builder.Labelled(builder.Command, request.ID), publisher, reuse,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
} else {
built, err = builder.Build(building, builder.Labelled(builder.Command, request.ID), publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), say, request.Seats)
}
}
// What it copied into the store, whatever became of the build (novox/hq ADR 0257).
result.Mirrored = built.Mirrored
// Only a build the kill ended: the kill came before its work did. One that finished — built, or
// failed on its own — in the moment the kill arrived says what it did, and the kill is refused.
killed := false
if mine != nil {
killed = h.returned(mine) && err != nil
}
if superseded.Load() && err != nil {
// Ended by its own key moving on (ADR 0406 §4): said as the check's failure, as a kill is.
result.Failed = link.SupersededByKey
say("failed", link.SupersededByKey)
} else if killed {
// **Killed by hand is the outcome, whatever the build was doing** (novox/hq ADR 0219): the
// error it ended with is the kill's consequence, not a fault of the source.
result.Failed = link.KilledByHand
say("failed", link.KilledByHand)
} else if err != nil {
// A failure is a result. A build that fails and says nothing is indistinguishable from a
// builder that is not running, and those want completely different responses.
result.Failed = err.Error()
say("failed", err.Error())
} else if request.Check != nil {
say("checked", result.Check.Verdict+": "+result.Check.Summary)
} else {
manifest, marshalErr := json.Marshal(built.Manifest)
if marshalErr != nil {
result.Failed = marshalErr.Error()
} else {
result.Commit = built.Commit
result.Manifest = manifest
for _, made := range built.Built {
result.Made = append(result.Made, link.MadeArtifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
}
result.Against = built.Against
result.SourceFingerprint = built.Source
result.SourceInputs = built.SourceInputs
// What the walk did instead of building, or why it built (novox/hq ADR 0350 §3, §4).
result.ReuseKey = built.ReuseKey
if built.Reused != nil {
f := built.Reused.From
result.Reused = built.Reused.Words()
result.ReusedFrom = &link.ReusedFrom{Pull: link.PullRef{Owner: f.Pull.Owner, Repo: f.Pull.Repo,
Number: f.Pull.Number}, Head: f.Head, Trunk: f.Trunk, Check: f.Check, Key: f.Key}
} else if request.Reuse != nil {
result.BuiltWhy = built.Why
if result.BuiltWhy == "" {
result.BuiltWhy = "it has no artifact to build"
}
}
result.Trunk, result.OnTrunk, result.Branches = built.Trunk, built.OnTrunk, built.Branches
for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, s := range built.Sources {
result.Sources = append(result.Sources, link.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
// The list of predefined keys the manifest was judged against, for the controller to publish
// (novox/hq ADR 0348 §7).
if p := built.Predefined; p != nil {
result.Predefined = &link.PredefinedList{Keys: p.Keys, Generation: p.Generation}
}
say("built", built.Manifest.Module+" from "+short(built.Commit))
}
}
// A killed build is announced on a context of its own: the build's was the one cancelled, and the
// outcome must go out and the ask be settled — acknowledged, never redelivered to another machine
// to be built again. A machine being stopped is the other case and keeps its meaning: the
// parent's context is gone, nothing is announced or settled, and the ask is redelivered.
announcing := ctx
if killed {
fresh, stop := context.WithTimeout(context.Background(), 30*time.Second)
defer stop()
announcing = fresh
}
announceErr := work.Announce(announcing, result)
if mine != nil {
h.said(mine, result.Failed, announceErr == nil)
}
if err := announceErr; err != nil {
// Said, not fatal: the build happened. A build reported as failed because announcing it
// failed is a lie about work that was done — and the request stays unsettled below only if
// nothing was said at all, so another machine can try.
fmt.Fprintf(os.Stderr, "cannot say what came of a build: %v\n", err)
return
}
// Settled only once the outcome is away, so a machine that dies before answering leaves the work
// for another rather than losing it.
if err := work.Done(); err != nil {
fmt.Fprintf(os.Stderr, "the outcome is away and the request could not be settled: %v\n", err)
}
}
// checkSpecOf is a check request as the builder runs it.
func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
c := request.Check
spec := builder.CheckSpec{ID: request.ID, Repository: request.Repository, Ref: request.Ref,
Owner: c.Owner, Repo: c.Repo, Number: c.Number, Paths: c.Paths, Beside: map[string]builder.Beside{},
Modules: c.Modules, New: c.New, Manifests: c.Manifests, Judge: c.Judge, Base: c.Base,
Toolchain: builder.ToolchainOf(request.Held), Toolchains: builder.ToolchainsOf(request.Held),
Tests: c.Tests, StoreOnly: c.StoreOnly, SiblingsUnknown: c.SiblingsUnknown}
for _, h := range c.HeldConfig {
spec.Held = append(spec.Held, builder.HeldValue{Pull: h.Pull, Head: h.Head, Node: h.Node, Module: h.Module, Key: h.Key,
Value: h.Value, Approved: h.Approved, By: h.By, At: h.At})
}
for _, s := range c.Siblings {
spec.Siblings = append(spec.Siblings, builder.Sibling{Repository: s.Repository, Number: s.Number, Head: s.Head,
Base: s.Base, Dir: s.Dir})
}
for dir, b := range c.Beside {
spec.Beside[dir] = builder.Beside{Repository: b.Repository, Ref: b.Ref}
}
for _, b := range c.Builds {
spec.Builds = append(spec.Builds, builder.CheckBuildSpec{Module: b.Module, Repository: b.Repository, Path: b.Path,
Ref: b.Ref, Merged: b.Merged, Tier: b.Tier, Skip: b.Skip})
}
for _, m := range c.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository,
Ref: m.Ref, Paths: m.Paths, Base: m.Base})
}
return spec
}
// siblingsOut is the siblings a check read as the outcome carries them (novox/hq ADR 0382 §4).
func siblingsOut(read []builder.JudgedSibling) []link.CheckSibling {
var out []link.CheckSibling
for _, s := range read {
out = append(out, link.CheckSibling{Repository: s.Repository, Number: s.Number, Head: s.Head, Base: s.Base,
BaseCommit: s.BaseCommit})
}
return out
}
// judgeCache is the judge cache the build agent's manifest declares (MESH_JUDGE_CACHE, novox/hq ADR 0358 §6): a
// directory mounted at the same path on the host, so a judge build's mounts name what the agent sees. Empty when the
// agent declares none, and every judge is then built cold under its check's root.
func judgeCache() string { return os.Getenv("MESH_JUDGE_CACHE") }
// triedOf is a try's outcome as its log line says it (novox/hq issue 686): the repository's own check's verdict,
// or the check's where that did not run — a head that does not merge — and the lines naming what failed.
func triedOf(v builder.CheckVerdict) link.Tried {
t := link.Tried{Verdict: v.Verdict, Summary: v.Summary, Took: v.Took.Round(time.Second).String(),
Head: v.Head, Trunk: v.Trunk}
if v.Repo != nil && (v.Gate == nil || v.Gate.Verdict == "pass") {
t.Verdict, t.Summary = v.Repo.Verdict, v.Repo.Summary
}
if v.Repo != nil && v.Repo.Failed != "" {
t.Failed = strings.Split(v.Repo.Failed, "\n")
}
if t.Verdict == "" {
t.Verdict = "error"
}
return t
}
// namedOf are the builds the seat's verdicts name, as the builder believes a scratch record by them (novox/hq ADR
// 0350 §2, issue 703): each module's key with every artifact's digest.
func namedOf(builds []link.CheckedBuild) []builder.NamedBuild {
var out []builder.NamedBuild
for _, b := range builds {
named := builder.NamedBuild{Pull: builder.PullRef{Owner: b.Pull.Owner, Repo: b.Pull.Repo,
Number: b.Pull.Number}, Check: b.Check, Module: b.Built.Module, Key: b.Built.Key,
Digests: map[string]string{}}
for _, m := range b.Built.Made {
named.Digests[m.Name] = builder.DigestOf(m.Reference)
}
out = append(out, named)
}
return out
}
// checkBuiltOf is a check's builds for its merge as the outcome carries them (novox/hq ADR 0350 §1).
func checkBuiltOf(built []builder.CheckBuilt) []link.CheckBuilt {
var out []link.CheckBuilt
for _, b := range built {
c := link.CheckBuilt{Module: b.Module, Key: b.Key, NotBuilt: b.NotBuilt, Took: b.Took.Round(time.Second).String(),
Kept: b.Kept, Same: b.Same, SameAs: b.SameAs}
for _, m := range b.Made {
c.Made = append(c.Made, link.MadeArtifact{Name: m.Name, Kind: m.Kind, Reference: m.Reference})
}
out = append(out, c)
}
return out
}
// layerOf is one layer of a check as the outcome carries it.
func layerOf(l *builder.Layer) *link.CheckLayer {
if l == nil {
return nil
}
return &link.CheckLayer{Verdict: l.Verdict, Summary: l.Summary, Modules: l.Modules}
}
// packagesFrom is where a build resolves the mesh's own published packages — the SDK above all
// (novox/hq ADR 0076, issue 053) — on this machine.
//
// **The binding wins, then the genesis document** (novox/hq ADR 0336 §3, §4). The build agent's
// npm-package-registry requirement is optional: on genesis's node, before the gitea module is assigned,
// nothing provides it, and the installer sealed the forge it raised to this agent as one document
// instead. Once the requirement binds, the binding names the endpoint the way the artifact store's
// does and a sealed token file the credential, and the genesis document is left unread. A binding
// file named and absent is a requirement not bound here, never a fault: the environment names the
// path whether or not the mesh wrote it.
//
// The environment variables remain for a builder run by a person. With none of the three the result
// is disabled and says this machine, so a build that needs the registry is refused naming it (§6),
// and a build that needs none builds anyway.
func packagesFrom(machine string) (builder.Npmrc, error) {
scope := strings.TrimSpace(os.Getenv("MESH_NPM_SCOPE"))
if scope == "" {
scope = "@novox"
}
none := builder.Npmrc{Machine: machine}
if path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not bound on this machine: the genesis document, or nothing.
case err != nil:
return none, fmt.Errorf("cannot read what the mesh said about the package registry: %w", err)
default:
return fromBinding(path, raw, scope, machine)
}
}
if path := strings.TrimSpace(os.Getenv("MESH_GENESIS_PACKAGE_REGISTRY")); path != "" {
raw, err := os.ReadFile(path)
switch {
case errors.Is(err, os.ErrNotExist):
// Not this mesh's genesis node, or the installer has not sealed it yet.
case err != nil:
return none, fmt.Errorf("cannot read the genesis document for the package registry: %w", err)
default:
return fromGenesis(path, raw, scope, machine)
}
}
// A builder run by a person.
registry := strings.TrimSpace(os.Getenv("MESH_NPM_REGISTRY"))
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
username := strings.TrimSpace(os.Getenv("MESH_NPM_USER"))
if registry == "" && secret == "" {
return none, nil
}
n := builder.Npmrc{Scope: scope, Registry: registry, From: builder.FromEnvironment, Machine: machine}
if username != "" {
n.Username, n.Password = username, secret
} else {
n.Token = secret
}
return n, nil
}
// fromBinding composes the registry from the npm-package-registry binding the mesh wrote and the
// credential it sealed beside it. The mesh authenticates the ordinary way — a generated password the
// provider only applies (novox/hq ADR 0048) — so with a username the secret is a password (basic
// auth); without one it is a bearer token a provider minted.
func fromBinding(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var told struct {
From string `json:"from"`
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return none, fmt.Errorf("%s is not a binding: %w", path, err)
}
if told.At == "" {
return none, fmt.Errorf(
"%s says the package registry is on %q and gives no address for it", path, told.From)
}
// Composed from what the provider serves, so nothing here knows gitea's URL shape from
// another registry's: it states its port, the path its registry answers on, and the scheme.
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
port, ok := told.Serves["port"]
if !ok {
return none, fmt.Errorf("%s says nothing about which port the package registry answers on", path)
}
npmPath, ok := told.Serves["npm-path"]
if !ok {
return none, fmt.Errorf("%s says nothing about the path the package registry answers on", path)
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
raw, err := os.ReadFile(file)
if err != nil {
return none, fmt.Errorf("cannot read this builder's package-registry credential: %w", err)
}
secret = strings.TrimSpace(string(raw))
}
n := builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%v%v", scheme, told.At, port, npmPath),
From: builder.FromBinding, Machine: machine}
if told.As != "" {
n.Username, n.Password = told.As, secret
} else {
n.Token = secret
}
return n, nil
}
// GenesisRegistry is the genesis document (novox/hq ADR 0336 §3): what the installer sealed to the
// build agent on its node at genesis's packages step, as the `genesis-package-registry` own-secret —
// the forge it raised, the port this node gave the package registry, its npm path, and the account it
// made with that account's password. mesh-host writes it; the field names are the two repositories'
// agreement.
type GenesisRegistry struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port int `json:"port"`
NpmPath string `json:"npm-path"`
As string `json:"as"`
Password string `json:"password"`
}
// fromGenesis composes the registry from the genesis document.
func fromGenesis(path string, raw []byte, scope, machine string) (builder.Npmrc, error) {
none := builder.Npmrc{Machine: machine}
var g GenesisRegistry
if err := json.Unmarshal(raw, &g); err != nil {
return none, fmt.Errorf("%s is not the genesis document for the package registry: %w", path, err)
}
var missing []string
for name, value := range map[string]string{"at": g.At, "npm-path": g.NpmPath, "as": g.As, "password": g.Password} {
if strings.TrimSpace(value) == "" {
missing = append(missing, name)
}
}
if g.Port <= 0 {
missing = append(missing, "port")
}
if len(missing) > 0 {
sort.Strings(missing)
return none, fmt.Errorf("%s, the genesis document for the package registry, says nothing of %s",
path, strings.Join(missing, ", "))
}
scheme := g.Scheme
if scheme == "" {
scheme = "http"
}
return builder.Npmrc{Scope: scope, Registry: fmt.Sprintf("%s://%s:%d%s", scheme, g.At, g.Port, g.NpmPath),
Username: g.As, Password: g.Password, From: builder.FromGenesis, Machine: machine}, nil
}
// saysRegistry is the line each build's log carries about the package registry (novox/hq ADR 0336
// §4): which of the binding and the genesis document it resolved from, or that it has neither here.
func saysRegistry(n builder.Npmrc) string {
if n.Enabled() {
return "the package registry is resolved from " + n.From
}
machine := n.Machine
if machine == "" {
machine = "this machine"
}
return fmt.Sprintf("no package registry: %s is unbound on %s and no genesis document is here; "+
"a build needing it is refused", builder.RegistryRequirement, machine)
}
// forgeFrom is the git credential this builder may offer a clone, composed from the same binding
// and sealed secret its package-registry half already reads: the forge that answers npm is the
// forge that hosts the repositories, and its provisioner applies one password to one user for
// both. Anything missing means no credential, and every clone stays anonymous — which is all a
// mesh of public repositories ever needs.
//
// The URL names the binding's own address — the machine the mesh says the forge is on — so a
// private repository is registered and built by that address, and a clone of anything else is
// never shown this credential (git's credential store matches the whole origin).
func forgeFrom() builder.GitCredential {
path := strings.TrimSpace(os.Getenv("MESH_PACKAGE_BINDING"))
if path == "" {
return builder.GitCredential{}
}
raw, err := os.ReadFile(path)
if err != nil {
return builder.GitCredential{}
}
var told struct {
At string `json:"at"`
As string `json:"as"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil || told.At == "" || told.As == "" {
return builder.GitCredential{}
}
secret := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN"))
if file := strings.TrimSpace(os.Getenv("MESH_NPM_TOKEN_FILE")); file != "" {
if raw, err := os.ReadFile(file); err == nil {
secret = strings.TrimSpace(string(raw))
}
}
if secret == "" {
return builder.GitCredential{}
}
scheme := "https"
if s, ok := told.Serves["scheme"]; ok {
scheme = fmt.Sprintf("%v", s)
}
host := told.At
if port, ok := told.Serves["port"]; ok {
host = fmt.Sprintf("%s:%v", told.At, port)
}
made := url.URL{Scheme: scheme, User: url.UserPassword(told.As, secret), Host: host}
return builder.GitCredential{URL: made.String()}
}
func short(commit string) string {
if len(commit) > 8 {
return commit[:8]
}
return commit
}
// whereToPublish is the artifact store this builder uses.
//
// **Preferably from the mesh.** A builder that is a module requires an artifact store, and the
// mesh writes it a file saying which machine answers that and on what port — the same binding any
// consumer of any provision gets. Reading it means the address is not a setting somebody keeps in
// step by hand, and moving the store is an ordinary reassignment rather than an edit on every
// build machine.
//
// The environment variable remains for a builder run by a person, which is how this started and
// how it is still run while being developed.
func whereToPublish() (string, error) {
binding := strings.TrimSpace(os.Getenv("MESH_BINDING"))
if binding == "" {
registry := strings.TrimSpace(os.Getenv("MESH_REGISTRY"))
if registry == "" {
return "", fmt.Errorf("neither MESH_BINDING nor MESH_REGISTRY: a built artifact " +
"nobody can fetch is not built")
}
return registry, nil
}
raw, err := os.ReadFile(binding)
if err != nil {
return "", fmt.Errorf("cannot read what the mesh said about the artifact store: %w", err)
}
var told struct {
From string `json:"from"`
At string `json:"at"`
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal(raw, &told); err != nil {
return "", fmt.Errorf("%s is not a binding: %w", binding, err)
}
if told.At == "" {
// The provider is not on the private network, so there is no name to reach it by. Said
// rather than falling back to the machine's own name, which would publish to a store on
// the wrong machine and be found out much later.
return "", fmt.Errorf(
"%s says the artifact store is on %q and gives no address for it", binding, told.From)
}
port, ok := told.Serves["port"]
if !ok {
return "", fmt.Errorf("%s says nothing about which port the artifact store answers on",
binding)
}
return fmt.Sprintf("%s:%v", told.At, port), nil
}
// brokerFrom is where this builder connects, and with what.
//
// **Preferably from a file the mesh sealed to this machine.** A builder that is a module is given
// its credential the way every other module is given one: generated or accepted centrally, sealed
// to the machine, written by the host. Putting it in an environment variable instead would mean
// the one copy that matters passing through a terminal and a process listing.
//
// The variable remains for a builder run by a person.
func brokerFrom() (Credential, error) {
if path := strings.TrimSpace(os.Getenv("MESH_BROKER_FILE")); path != "" {
raw, err := os.ReadFile(path)
if err != nil {
return Credential{}, fmt.Errorf("cannot read this builder's credential: %w", err)
}
said := strings.TrimSpace(string(raw))
if said == "" {
// An empty credential file is a machine that will connect as nobody and be refused,
// with the reason three layers away.
return Credential{}, fmt.Errorf("%s is empty, so this builder has no credential", path)
}
var held Credential
if err := json.Unmarshal([]byte(said), &held); err == nil && held.URL != "" {
return held, nil
}
// A file holding only a URL, which is what a person writing one by hand produces. The
// broker is then verified against whatever this machine already trusts.
return Credential{URL: said}, nil
}
return Credential{}, fmt.Errorf(
"no MESH_BROKER_FILE: a build machine with no credential for the bus has nothing to build")
}
// Credential is what a build machine is given so it can reach the broker.
//
// Two things, because reaching a broker over TLS needs both: who to connect as, and what to check
// the certificate against. A mesh's broker presents a certificate of the mesh's own, which is in
// no public trust store, so a URL alone can only connect to a broker somebody else vouches for.
//
// **The same shape a node gets, for the same reason** (novox/hq ADR 0004): the fingerprint travels
// out of band — here, sealed with the credential — and the endpoint is verified once at connect.
type Credential struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
// User and Password ride beside the address on the bus being built (design 25): a credential
// embedded in a URL leaks into every log line that prints a connection, so the mesh seals them
// as two fields and this machine joins them once, here, to dial.
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
// Claims are the seats the module this credential was issued for claims, as the mesh writes
// them beside the credential (novox/hq ADR 0159). The first is the build role this machine
// serves; a credential naming none is from before claims travelled in it.
Claims []struct {
Seat string `json:"seat"`
} `json:"claims,omitempty"`
}
// seatsClaimed is the seats the credential names, in order.
func (c Credential) seatsClaimed() []string {
out := make([]string, 0, len(c.Claims))
for _, claim := range c.Claims {
out = append(out, claim.Seat)
}
return out
}
// onTheNewBus is whether a credential is for the bus being built: its address says so, and the
// mesh only ever seals such a credential with the user and password beside it.
func (c Credential) onTheNewBus() bool { return strings.HasPrefix(strings.TrimSpace(c.URL), "nats://") }
// natsURL is the address with this machine's credential in it, for the one dial that needs it.
func (c Credential) natsURL() string {
rest := strings.TrimPrefix(strings.TrimSpace(c.URL), "nats://")
if c.User == "" {
return "nats://" + rest
}
return "nats://" + c.User + ":" + c.Password + "@" + rest
}
-188
View File
@@ -1,188 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/builder"
)
// buildOnce is the builder doing one build and stopping, with no broker and no mesh.
//
// **This is how a mesh is raised** (novox/hq ADR 0073). The installer carries this program and runs
// it once, before anything else exists, to produce the control plane from the same repository and
// path that every later rebuild of it will use. What raises the mesh is therefore the same thing
// that maintains it — not a second mechanism that has to be kept in step with the first and is
// exercised once per new mesh, which is how often enough to rot.
//
// It takes no work from a queue and answers nobody: there is no broker yet, and the only thing
// waiting for the answer is the installer that started it. So the result goes to standard output as
// JSON, which is what the installer reads.
//
// With no --registry it publishes nowhere and the image stays in this machine's container runtime,
// named by the digest of its own configuration (see builder.Local). That is the genesis case. With
// one, it publishes as it always does — the same command is how an operator builds a module by hand
// on a mesh that already exists.
func buildOnce(ctx context.Context, args []string) error {
set := flag.NewFlagSet("build", flag.ContinueOnError)
path := set.String("path", "", "the module's directory inside the repository (default: its root)")
ref := set.String("ref", "", "the commit to build; a branch is a moving target somebody else controls")
registry := set.String("registry", "",
"host:port to publish to. Without it the artifacts stay in this machine's container runtime, which is the genesis case")
workspace := set.String("workspace", "", "where to clone and build (default: a temporary directory)")
on := multiple(set, "on",
"a base this build stands on, as <module>/<artifact>=<reference>. Repeatable")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 1 {
return errors.New("mesh-builder build <repository> [--path P] [--ref COMMIT] [--registry HOST:PORT]")
}
repository := positionals[0]
where := *workspace
if where == "" {
where = os.TempDir() + "/mesh-builder-once"
}
// What this build may stand on, said on the command line because there is no mesh to ask.
//
// **This is the genesis case and it is meant to be awkward.** On a running mesh the control
// plane answers this, because only it knows what this mesh holds. Here nothing has been built
// yet, so whoever runs this says what to use — and for the control plane, which is the one
// module raised before anything else exists, the answer is ordinarily nothing at all.
bases := map[string]string{}
for _, pair := range *on {
key, reference, found := strings.Cut(pair, "=")
if !found || key == "" || reference == "" {
return fmt.Errorf(
"--on takes <module>/<artifact>=<reference>, and %q is not that", pair)
}
bases[key] = reference
}
// Local unless told otherwise, because the moment this exists for has nowhere to publish. A
// default pointing at a registry would mean genesis failing at a push to something that is not
// there yet, one step away from the thing that could explain it.
var publisher builder.Publisher = builder.Local{Run: builder.Command}
if *registry != "" {
publisher = builder.Registry{Address: *registry, Run: builder.Command}
}
fmt.Fprintf(os.Stderr, "building %s", repository)
if *path != "" {
fmt.Fprintf(os.Stderr, " at %s", *path)
}
if *ref != "" {
fmt.Fprintf(os.Stderr, " at %s", *ref)
}
fmt.Fprintln(os.Stderr)
machine := os.Getenv("MESH_NODE")
if machine == "" {
machine, _ = os.Hostname()
}
npmrc, err := packagesFrom(machine)
if err != nil {
return err
}
fmt.Fprintf(os.Stderr, " [packages] %s\n", saysRegistry(npmrc))
built, buildErr := builder.Build(ctx, builder.Command, publisher, repository, *path, *ref, where, bases, npmrc,
forgeFrom(),
func(step, message string) { fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message) })
if buildErr != nil {
return buildErr
}
// To standard output, and everything else to standard error, so the caller can read this
// without having to separate it from progress.
out := onceResult{
Module: built.Manifest.Module,
Commit: built.Commit,
Repository: repository,
Path: *path,
Ref: *ref,
Manifest: built.Manifest,
Against: built.Against,
Source: built.Source,
}
for _, r := range built.Read {
out.Read = append(out.Read, readRepository{Repository: r.Repository, Ref: r.Ref})
}
for _, made := range built.Built {
out.Made = append(out.Made, madeArtifact{Name: made.Name, Kind: made.Kind, Reference: made.Reference})
}
body, err := json.MarshalIndent(out, "", " ")
if err != nil {
return err
}
fmt.Println(string(body))
fmt.Fprintf(os.Stderr, " built %s from %s\n", built.Manifest.Module, short(built.Commit))
return nil
}
// onceResult is what one build reports to whoever started it.
//
// The same fields the mesh records for a build, so a reader comparing a genesis build against an
// ordinary one is comparing the same thing said the same way.
type onceResult struct {
Module string `json:"module"`
Commit string `json:"commit"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
Manifest any `json:"manifest"`
Made []madeArtifact `json:"made"`
Against []string `json:"against,omitempty"`
Read []readRepository `json:"read,omitempty"`
Source string `json:"source-fingerprint,omitempty"`
}
// readRepository is a repository this build read source from besides the module's own.
type readRepository struct {
Repository string `json:"repository"`
Ref string `json:"ref,omitempty"`
}
type madeArtifact struct {
Name string `json:"name"`
Kind string `json:"kind"`
Reference string `json:"reference"`
}
// parseAround lets flags appear on either side of the repository, because a person writing this by
// hand will put them wherever reads best and the standard parser stops at the first thing that is
// not a flag. The same helper the control plane's commands use, for the same reason.
func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
var positionals []string
rest := args
for {
if err := set.Parse(rest); err != nil {
return nil, err
}
rest = set.Args()
if len(rest) == 0 {
return positionals, nil
}
positionals = append(positionals, rest[0])
rest = rest[1:]
}
}
// multiple is a flag that may be given more than once.
type repeated []string
func (r *repeated) String() string { return strings.Join(*r, ", ") }
func (r *repeated) Set(v string) error { *r = append(*r, v); return nil }
func multiple(set *flag.FlagSet, name, usage string) *[]string {
var values repeated
set.Var(&values, name, usage)
return (*[]string)(&values)
}
-102
View File
@@ -1,102 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/builder"
)
// The build agent's package registry (novox/hq ADR 0336, issue 121): its npm-package-registry
// requirement is optional, so the binding may be absent, and on genesis's node the installer seals
// the forge it raised as one genesis document instead.
// registryFiles names the three paths the build agent's environment always names, writing only those
// given.
func registryFiles(t *testing.T, binding, token, genesis string) {
t.Helper()
dir := t.TempDir()
for env, content := range map[string]string{
"MESH_PACKAGE_BINDING": binding,
"MESH_NPM_TOKEN_FILE": token,
"MESH_GENESIS_PACKAGE_REGISTRY": genesis,
} {
path := filepath.Join(dir, env)
t.Setenv(env, path)
if content != "" {
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
}
for _, env := range []string{"MESH_NPM_REGISTRY", "MESH_NPM_TOKEN", "MESH_NPM_USER", "MESH_NPM_SCOPE"} {
t.Setenv(env, "")
}
}
const (
aBinding = `{"from":"gitea","at":"anchor.mesh","as":"anchor-agent",
"serves":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/"}}`
aGenesisDocument = `{"scheme":"http","at":"127.0.0.1","port":3100,"npm-path":"/api/packages/novox/npm/",
"as":"mesh-builder","password":"genesis-secret"}`
)
func TestUntilTheRequirementBindsTheGenesisDocumentIsRead(t *testing.T) {
registryFiles(t, "", "", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatalf("an absent binding beside a genesis document was refused: %v", err)
}
if n.From != builder.FromGenesis || n.Registry != "http://127.0.0.1:3100/api/packages/novox/npm/" ||
n.Username != "mesh-builder" || n.Password != "genesis-secret" || n.Scope != "@novox" {
t.Fatalf("the genesis document was not what the registry came from: %+v", n)
}
file, err := n.File()
if err != nil || !strings.Contains(file, "//127.0.0.1:3100/api/packages/novox/npm/:username=mesh-builder") {
t.Fatalf("the .npmrc is not written from the genesis document: %v\n%s", err, file)
}
if got := saysRegistry(n); !strings.Contains(got, "genesis document") {
t.Errorf("the log does not say the genesis document: %s", got)
}
}
func TestARealBindingWinsOverTheGenesisDocument(t *testing.T) {
registryFiles(t, aBinding, "bound-secret\n", aGenesisDocument)
n, err := packagesFrom("anchor")
if err != nil {
t.Fatal(err)
}
if n.From != builder.FromBinding || n.Registry != "http://anchor.mesh:3000/api/packages/novox/npm/" ||
n.Username != "anchor-agent" || n.Password != "bound-secret" {
t.Fatalf("the binding did not win over the genesis document: %+v", n)
}
if got := saysRegistry(n); !strings.Contains(got, "binding") {
t.Errorf("the log does not say the binding: %s", got)
}
}
func TestWithNeitherTheAgentHasNoRegistryAndSaysWhere(t *testing.T) {
registryFiles(t, "", "", "")
n, err := packagesFrom("workstation")
if err != nil {
t.Fatalf("an unbound optional requirement was a fault: %v", err)
}
if n.Enabled() || n.Machine != "workstation" {
t.Fatalf("a registry came from nowhere: %+v", n)
}
got := saysRegistry(n)
for _, want := range []string{"npm-package-registry", "unbound", "workstation"} {
if !strings.Contains(got, want) {
t.Errorf("the log does not say %q: %s", want, got)
}
}
}
func TestAGenesisDocumentMissingAFieldIsRefusedByName(t *testing.T) {
registryFiles(t, "", "", `{"at":"127.0.0.1","port":3000,"npm-path":"/api/packages/novox/npm/","as":"mesh-builder"}`)
if _, err := packagesFrom("anchor"); err == nil || !strings.Contains(err.Error(), "password") {
t.Fatalf("a genesis document with no password was taken: %v", err)
}
}
-44
View File
@@ -1,44 +0,0 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
)
// **The build agent says it is ready** (novox/hq issue 544). It listens on nothing, so no http or tcp
// check can see it work; it writes a mark once it is bound to its worker and pulls build work, and its
// container's health check runs `mesh-builder ready`, which passes only while the mark is there. Without
// it the node-engine could judge the agent only alive, and every walk that moved it waited out the whole
// grace on each machine.
//
// The mark lives in the container's own temporary directory, never in the workspace the machine keeps:
// a new container starts without it, and a restarted one removes it before it binds again.
func readyMark() string { return filepath.Join(os.TempDir(), "mesh-builder.ready") }
// notReady removes the mark: a holder starting is not taking work until it says so again.
func notReady() error {
if err := os.Remove(readyMark()); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("cannot remove the mark that this agent takes build work (%s): %w", readyMark(), err)
}
return nil
}
// taking writes the mark. A mark that cannot be written is said, and the agent goes on building: its
// check then says it is not ready, which is the truth the gate should hear.
func taking(seat string) {
if err := os.WriteFile(readyMark(), []byte(seat+"\n"), 0o644); err != nil {
fmt.Fprintf(os.Stderr, "cannot mark that this agent takes build work (%s): %v\n", readyMark(), err)
return
}
fmt.Fprintf(os.Stderr, "taking build work from %s\n", seat)
}
// ready is the health check: nil when this container's agent is bound and pulling build work.
func ready() error {
if _, err := os.Stat(readyMark()); err != nil {
return fmt.Errorf("not taking build work yet: %s is not there", readyMark())
}
return nil
}
-32
View File
@@ -1,32 +0,0 @@
package main
import (
"os"
"testing"
)
// The health check passes only between the agent saying it takes work and its next start (novox/hq
// issue 544).
func TestTheAgentIsReadyOnlyOnceItTakesWork(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
if err := ready(); err == nil {
t.Fatal("an agent that never took work is ready")
}
taking("mesh-build-machine")
if err := ready(); err != nil {
t.Fatalf("an agent taking work is not ready: %v", err)
}
// Started again, it is not ready until it binds again.
if err := notReady(); err != nil {
t.Fatal(err)
}
if err := ready(); err == nil {
t.Fatal("an agent starting again is ready before it takes work")
}
if err := notReady(); err != nil {
t.Fatalf("removing a mark that is not there failed: %v", err)
}
if _, err := os.Stat(readyMark()); err == nil {
t.Fatal("the mark is still there")
}
}
-27
View File
@@ -1,27 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// The seat a build machine serves comes from its credential (novox/hq ADR 0190 handover).
func TestTheCredentialSaysWhichBuildRoleThisMachineServes(t *testing.T) {
var held Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.builder","password":"x",
"claims":[{"seat":"mesh-build-machine","scope":"mesh","serves":[]}]}`), &held); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(held.seatsClaimed()); got != "mesh-build-machine" {
t.Errorf("the old builder's credential serves %q", got)
}
var bare Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.build-agent","password":"x"}`), &bare); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(bare.seatsClaimed()); got != link.TheBuildMachine {
t.Errorf("a credential without claims serves %q, want %s", got, link.TheBuildMachine)
}
}
-45
View File
@@ -1,45 +0,0 @@
package main
import (
"os"
"strings"
"testing"
)
// **The genesis path writes its result to stdout and nothing else there.** The installer captures
// stdout alone and parses it as JSON; one stray log line makes that fail with "invalid character"
// — exactly what a builder fmt.Printf caused, breaking a build that had worked. This keeps
// diagnostics on stderr so a future print cannot repeat it silently.
func TestBuilderDiagnosticsStayOffStdout(t *testing.T) {
allowed := map[string]bool{
"string(body)": true, // once.go: the result JSON, which IS stdout
"version)": true, // --version
`"stopping")`: true, // the loop.s shutdown line
"usage)": true, // --help text, for a human
"said)": true, // busy: the answer its node-engine reads (novox/hq ADR 0335)
}
for _, file := range []string{"once.go", "main.go"} {
src, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
for i, raw := range strings.Split(string(src), "\n") {
line := strings.TrimSpace(raw)
if !strings.HasPrefix(line, "fmt.Printf(") &&
!strings.HasPrefix(line, "fmt.Println(") &&
!strings.HasPrefix(line, "fmt.Print(") {
continue
}
ok := false
for token := range allowed {
if strings.Contains(line, token) {
ok = true
}
}
if !ok {
t.Errorf("%s:%d writes a diagnostic to stdout, which the installer parses as JSON:\n %s",
file, i+1, line)
}
}
}
}
-31
View File
@@ -1,31 +0,0 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/builder"
)
// A try's outcome, as the build seat says it in the build's log (novox/hq issue 686): the repository's own
// check's verdict, how long it took, and each line naming what failed.
func TestATrysOutcomeNamesWhatFailed(t *testing.T) {
tried := triedOf(builder.CheckVerdict{Verdict: "pass", Summary: "the change touches no module", Took: 303 * time.Second,
Head: "c0ffee", Trunk: "ca3f43c", Gate: &builder.Layer{Verdict: "pass"},
Repo: &builder.Layer{Verdict: "fail", Summary: "its merge-check.sh failed: TestX",
Failed: "--- FAIL: TestX (0.01s)\n x_test.go:12: got 1, want 2"}})
if tried.Verdict != "fail" || tried.Took != "5m3s" || len(tried.Failed) != 2 || tried.Trunk != "ca3f43c" ||
tried.Summary != "its merge-check.sh failed: TestX" {
t.Fatalf("a failed try: %+v", tried)
}
// A head that does not merge fails both layers: the check's own word.
conflict := triedOf(builder.CheckVerdict{Verdict: "fail", Summary: "does not merge", Gate: &builder.Layer{Verdict: "fail"},
Repo: &builder.Layer{Verdict: "fail", Summary: "does not merge"}})
if conflict.Verdict != "fail" || conflict.Summary != "does not merge" {
t.Fatalf("a try that does not merge: %+v", conflict)
}
// No verdict is an error, never a pass.
if none := triedOf(builder.CheckVerdict{}); none.Verdict != "error" {
t.Fatalf("a try with no verdict: %+v", none)
}
}
-225
View File
@@ -1,225 +0,0 @@
package main
import (
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/hex"
"math/big"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
// Where a builder publishes.
//
// Preferably from the mesh: a builder that is a module requires an artifact store, and the mesh
// writes it a binding saying which machine answers and on what port. Reading it means the address
// is not a setting somebody keeps in step by hand.
func binding(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "artifact-store.json")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return path
}
func TestTheMeshSaysWhereToPublish(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"provision":"artifact-store",
"from":"anchor","at":"anchor.internal","serves":{"port":5000,"scheme":"http"}}`))
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "anchor.internal:5000" {
t.Fatalf("got %q", where)
}
}
func TestABindingWithNoAddressIsRefused(t *testing.T) {
// The provider is not on the private network, so there is no name to reach it by. Falling
// back to anything would publish to a store on the wrong machine and be found out much later.
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"anchor","serves":{"port":5000}}`))
_, err := whereToPublish()
if err == nil {
t.Fatal("a binding with nowhere to reach was accepted")
}
if !strings.Contains(err.Error(), "anchor") {
t.Fatalf("the failure does not name the machine: %v", err)
}
}
func TestABindingWithNoPortIsRefused(t *testing.T) {
t.Setenv("MESH_BINDING", binding(t, `{"binding":1,"from":"a","at":"a.internal","serves":{}}`))
if _, err := whereToPublish(); err == nil {
t.Fatal("a binding saying nothing about a port was accepted")
}
}
func TestTheVariableStillWorksForABuilderRunByAPerson(t *testing.T) {
// Which is how this started and how it is still run while being developed.
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "127.0.0.1:5000")
where, err := whereToPublish()
if err != nil {
t.Fatal(err)
}
if where != "127.0.0.1:5000" {
t.Fatalf("got %q", where)
}
}
func TestNeitherIsRefusedRatherThanGuessed(t *testing.T) {
t.Setenv("MESH_BINDING", "")
t.Setenv("MESH_REGISTRY", "")
if _, err := whereToPublish(); err == nil {
t.Fatal("a builder with nowhere to publish reported somewhere")
}
}
func TestTheCredentialComesFromAFileTheMeshSealed(t *testing.T) {
// A builder that is a module is given its credential the way every module is: sealed to the
// machine and written by the host. An environment variable instead would put the one copy
// that matters through a terminal and a process listing.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("amqps://a-builder:secret@broker.internal:5671/\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "amqp://should-not-be-used@nowhere/")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("got %q", got.URL)
}
}
// The credential the mesh seals carries what to check the broker's certificate against, because a
// mesh's broker presents a certificate of the mesh's own and no public trust store has it. A URL
// alone can only reach a broker somebody else vouches for.
func TestTheSealedCredentialCarriesWhatVerifiesTheBroker(t *testing.T) {
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte(
`{"url":"amqps://a-builder:secret@broker.internal:5671/","fingerprint":"abc123"}`),
0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
got, err := brokerFrom()
if err != nil {
t.Fatal(err)
}
if got.URL != "amqps://a-builder:secret@broker.internal:5671/" {
t.Fatalf("the url was lost: %q", got.URL)
}
if got.Fingerprint != "abc123" {
t.Fatal("the builder was given nothing to check the broker against, so it can only " +
"connect to a broker some public authority vouches for")
}
}
func TestAnEmptyCredentialFileIsRefused(t *testing.T) {
// Otherwise the builder connects as nobody and is refused, with the reason three layers away.
path := filepath.Join(t.TempDir(), "broker")
if err := os.WriteFile(path, []byte("\n"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_BROKER_FILE", path)
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("an empty credential was accepted")
}
}
func TestABuilderWithNoCredentialAtAllSaysSo(t *testing.T) {
t.Setenv("MESH_BROKER_FILE", "")
t.Setenv("MESH_BROKER_AMQP", "")
if _, err := brokerFrom(); err == nil {
t.Fatal("a builder with no broker reported one")
}
}
// The pin is compared in the spelling the mesh writes it.
//
// A bare digest against a written fingerprint never matches, and the failure is indistinguishable
// from being pointed at the wrong broker — which is the one thing this check exists to report
// truthfully. It cost a lab run.
func TestThePinIsComparedInTheSpellingTheMeshWritesIt(t *testing.T) {
certificate, key := aServerCertificate(t)
der := certificate.Certificate[0]
sum := sha256.Sum256(der)
written := "sha256:" + hex.EncodeToString(sum[:])
listener, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{
Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12,
})
if err != nil {
t.Fatal(err)
}
defer listener.Close()
go func() {
for {
conn, err := listener.Accept()
if err != nil {
return
}
_ = conn.(*tls.Conn).Handshake()
conn.Close()
}
}()
_ = key
// The pin the mesh wrote must be accepted.
if err := handshakeWith(listener.Addr().String(), written); err != nil {
t.Fatalf("the broker this builder was told about was refused: %v", err)
}
// And a different one refused, or the check reports nothing.
other := "sha256:" + strings.Repeat("ab", 32)
if err := handshakeWith(listener.Addr().String(), other); err == nil {
t.Fatal("a broker this builder was not told about was accepted")
}
}
// handshakeWith runs the pin check the builder dials with against an address.
func handshakeWith(address, pin string) error {
conn, err := tls.Dial("tcp", address, broker.PinnedToFingerprint(pin))
if err != nil {
return err
}
return conn.Close()
}
func aServerCertificate(t *testing.T) (tls.Certificate, ed25519.PrivateKey) {
t.Helper()
public, private, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "a broker"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
}
der, err := x509.CreateCertificate(rand.Reader, template, template, public, private)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: private}, private
}
@@ -221,8 +221,4 @@ func TestOnANodeWithNoNetworkTheStoreIsReachedByLoopback(t *testing.T) {
if image != "127.0.0.1:5100/postgres/runtime@"+aDigest { if image != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image) t.Fatalf("on the store's own node, off any network, the image is fetched as %v", image)
} }
held := heldBy(ctx)
if got := held["postgres/runtime"]; got != "127.0.0.1:5100/postgres/runtime@"+aDigest {
t.Fatalf("a builder beside the store is handed the base %q", got)
}
} }
-59
View File
@@ -1,12 +1,9 @@
package main package main
import ( import (
"encoding/json"
"strings" "strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/link"
) )
// An address is read from the node's settings where it is used, never recorded with a port // An address is read from the node's settings where it is used, never recorded with a port
@@ -14,62 +11,6 @@ import (
var aDigest = "sha256:" + strings.Repeat("e", 64) var aDigest = "sha256:" + strings.Repeat("e", 64)
// **A build is recorded by digest and path**, whatever address the builder pushed to — what it
// made and what it stood on both; an image the module runs from elsewhere is left where it says.
func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
manifest, _ := json.Marshal(map[string]any{
"module": "gitea", "version": "1",
"resources": []map[string]any{
{"id": "server", "type": "container", "name": "mesh-gitea",
"image": "anchor.internal:5100/gitea/server@" + aDigest},
{"id": "config", "type": "archive", "path": "/etc/gitea", "digest": aDigest,
"source": "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
{"id": "cache", "type": "container", "name": "mesh-gitea-cache",
"image": "valkey/valkey@" + aDigest},
},
})
kept := buildFrom(link.BuildResult{
ID: "b1", Repository: "https://forge.example/gitea.git", Commit: "abc", On: "laptop",
Manifest: manifest,
Made: []link.MadeArtifact{
{Name: "server", Kind: "image", Reference: "anchor.internal:5100/gitea/server@" + aDigest},
{Name: "config", Kind: "archive", Reference: "http://anchor.internal:5100/v2/gitea/config/blobs/" + aDigest},
},
Against: []string{"anchor.internal:5100/mesh-tools/runtime@" + aDigest},
})
if kept.Module != "gitea" {
t.Fatalf("the module was not read from the recorded manifest: %q", kept.Module)
}
if kept.Made[0].Reference != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the image is recorded as %q, address and all", kept.Made[0].Reference)
}
if kept.Made[1].Reference != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the archive is recorded as %q, address and all", kept.Made[1].Reference)
}
recorded, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
t.Fatal(err)
}
if got := recorded.Resources[0]["image"]; got != catalogue.ArtifactStoreScheme+"gitea/server@"+aDigest {
t.Errorf("the recorded manifest's image is %v", got)
}
if got := recorded.Resources[1]["source"]; got != catalogue.ArtifactStoreScheme+"gitea/config/blobs/"+aDigest {
t.Errorf("the recorded manifest's archive is %v", got)
}
if got := recorded.Resources[2]["image"]; got != "valkey/valkey@"+aDigest {
t.Errorf("an image the build did not make was rewritten: %v", got)
}
if strings.Contains(string(kept.Manifest), "anchor.internal:5100") {
t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest)
}
// What the build stood on is an edge to another module's artifact, and it is recorded the way
// that artifact is: by path in the store, so the edge still names the same thing when the
// store answers at another address.
if kept.Against[0] != catalogue.ArtifactStoreScheme+"mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was recorded by address: %v", kept.Against)
}
}
// aStore is a module offering the artifact store on 5000, published the long way as the // aStore is a module offering the artifact store on 5000, published the long way as the
// distribution module does, so a node may be given another number for it. // distribution module does, so a node may be given another number for it.
func aStore() catalogue.Manifest { func aStore() catalogue.Manifest {
@@ -1,131 +0,0 @@
//go:build integration
package main
import (
"context"
"fmt"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334, the look: when the bus did not answer an ask of any module of a tier, no build is in flight and no event follows,
// so the look itself comes back (it fails, and the looker retries it), asks the modules again, and past the bound fails the plan.
func TestALookComesBackForATierTheBusRefusedWhole(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
for _, name := range []string{"a", "b"} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: name, Version: "1"},
inventory.Source{Repository: "novox/" + name, Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
}
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b"}}, Modules: map[string]*inventory.PlanModule{"a": {}, "b": {}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
bus := false
asksOfEveryModule(t) // the one ask left is the builder's own (ADR 0453 §5): this test is of what a walk does around it
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if !bus {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
look := func() bool {
t.Helper()
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
defer release()
return advanceHeld(ctx, open)
}
read := func() inventory.Plan {
t.Helper()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
return got
}
// Every module refused: the look says it failed, which has the looker come back; the plan is building and says why.
if !look() {
t.Fatal("a look that left the whole tier unasked for the bus did not fail, so nothing would come back to it")
}
if got := read(); got.State != inventory.PlanBuilding || !strings.Contains(got.Note, "bus did not answer") ||
got.Modules["a"].State != "" || got.Modules["b"].State != "" {
t.Fatalf("the plan after a refused tier: %s %q %+v %+v", got.State, got.Note, got.Modules["a"], got.Modules["b"])
}
// Again, still refused: the same fact is not saved again, and the look still comes back.
before := read().Revision
if !look() || read().Revision != before {
t.Fatalf("a second refused look failed to come back or saved the plan again (revision %d, was %d)", read().Revision, before)
}
// The bus answers at the next look: both are asked, and nothing waits any more.
bus = true
if look() {
t.Fatal("a look that asked the whole tier failed")
}
if got := read(); got.Modules["a"].State != "asked" || got.Modules["b"].State != "asked" || got.Modules["a"].BusWaitFrom != nil {
t.Fatalf("the tier was not asked once the bus answered: %+v %+v", got.Modules["a"], got.Modules["b"])
}
}
func TestALookFailsTheTierTheBusRefusedPastTheBound(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "a", Version: "1"},
inventory.Source{Repository: "novox/a", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
long := time.Now().UTC().Add(-busWaitBound - time.Minute)
p := inventory.Plan{ID: "plan-bus", Repository: "novox/a", Branch: "main", Commit: "c1", Created: time.Now().UTC(),
State: inventory.PlanBuilding, Tiers: [][]string{{"a"}}, Modules: map[string]*inventory.PlanModule{"a": {BusWaitFrom: &long}}}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
asksOfEveryModule(t) // the one ask left is the builder's own (ADR 0453 §5): this test is of what a walk does around it
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
release, err := inv.HoldPlans(ctx, true)
if err != nil {
t.Fatal(err)
}
advanceHeld(ctx, open)
release()
got, err := inv.PlanByID(ctx, "plan-bus")
if err != nil {
t.Fatal(err)
}
if got.State != inventory.PlanFailed || !strings.Contains(got.Note, "the bus did not answer an ask for 10m0s (since ") {
t.Fatalf("a bus silent past its bound did not fail the plan in words: %s %q", got.State, got.Note)
}
// And retried by hand: the module is named as waiting, not as asked again.
got, _ = inv.PlanByID(ctx, "plan-bus")
if got.State == inventory.PlanFailed {
got.Modules["a"].State, got.Modules["a"].Why = "failed", "x"
if err := inv.SavePlan(ctx, &got); err != nil {
t.Fatal(err)
}
said, err := retryPlan(ctx, open, "plan-bus")
if err != nil || !strings.Contains(said, "not asked yet: the bus did not answer") {
t.Fatalf("a retry left for the bus is said as %q %v", said, err)
}
}
}
-151
View File
@@ -1,151 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// Issue 1334: a bus that does not answer for a moment (it reloads its user list, and JetStream's API has no responder for a few
// seconds) is not a failed build. The module is left unasked and the plan building, and the next look asks it again; only a
// module's own fault, or a bus silent for ten minutes, fails the plan.
// asksOfEveryModule makes every module a test names the builder's own, the one module a walk still asks build-agent
// for (ADR 0453 §5): what a test of that ask's waits reads.
func asksOfEveryModule(t *testing.T) {
t.Helper()
was := buildersOwn
buildersOwn = func(string) bool { return true }
t.Cleanup(func() { buildersOwn = was })
}
func aTierOfThree() (*inventory.Plan, map[string]inventory.Entry) {
p := &inventory.Plan{ID: "plan-1", State: inventory.PlanBuilding, Tiers: [][]string{{"a", "b", "c"}},
Modules: map[string]*inventory.PlanModule{}}
byName := map[string]inventory.Entry{}
for _, m := range []string{"a", "b", "c"} {
byName[m] = inventory.Entry{Source: inventory.Source{Repository: "novox/" + m, Seat: "git", Ref: "main"}}
}
return p, byName
}
func TestABusThatDoesNotAnswerForAMomentDoesNotFailTheTier(t *testing.T) {
asksOfEveryModule(t)
p, byName := aTierOfThree()
bus := false
was := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
if source.Repository == "novox/b" && !bus {
return "", fmt.Errorf("%w: the bucket mesh-controller_builds is not on the bus: nats: no responders available for request",
inventory.ErrNoChain)
}
return "build-" + source.Repository, nil
}
t.Cleanup(func() { askABuild = was })
for _, m := range []string{"a", "b", "c"} {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding {
t.Fatalf("a bus that did not answer for a moment failed the plan: %s, %q", p.State, p.Note)
}
if p.Modules["a"].State != "asked" || p.Modules["c"].State != "asked" {
t.Fatalf("the modules the bus did answer for were not asked: %+v %+v", p.Modules["a"], p.Modules["c"])
}
if s := p.Modules["b"]; s.State != "" {
t.Fatalf("the module the bus did not answer for is %q, not left unasked", s.State)
}
if got := unaskedOf(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the next look would ask %v, not b", got)
}
if !strings.Contains(p.Note, "bus did not answer") {
t.Fatalf("the plan says nothing of the wait: %q", p.Note)
}
// The next look, the bus answering: b is asked and the plan goes on.
bus = true
for _, m := range unaskedOf(*p) {
askModule(t.Context(), p, m, byName)
}
if p.State != inventory.PlanBuilding || p.Modules["b"].State != "asked" || p.Modules["b"].Build == "" {
t.Fatalf("b was not asked once the bus answered: %s %+v", p.State, p.Modules["b"])
}
}
func TestABusSilentForTenMinutesFailsTheTierAsBefore(t *testing.T) {
asksOfEveryModule(t)
p, byName := aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", fmt.Errorf("%w: nats: no responders available for request", inventory.ErrNoChain)
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding {
t.Fatalf("the first refusal failed the plan: %q", p.Note)
}
long := time.Now().Add(-busWaitBound - time.Minute)
p.Modules["a"].BusWaitFrom = &long
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a bus silent past its bound did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
// Said in plain words: the bus, the wait, and since when.
want := "a could not be asked for: the bus did not answer an ask for 10m0s (since " + long.UTC().Format(time.RFC3339) +
"): this kind of record lives on the bus"
if !strings.HasPrefix(p.Note, want) || p.Modules["a"].Why != strings.TrimPrefix(want, "a could not be asked for: ")+p.Note[len(want):] {
t.Fatalf("the plan says %q, not words starting %q", p.Note, want)
}
// The wait has ended: a Retry made while the bus still reloads waits again, and does not fail at its first refusal.
if p.Modules["a"].BusWaitFrom != nil {
t.Fatalf("a failed module still holds the start of its wait: %v", p.Modules["a"].BusWaitFrom)
}
p.State = inventory.PlanBuilding
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanBuilding || p.Modules["a"].State != "" || p.Modules["a"].BusWaitFrom == nil {
t.Fatalf("a retry made while the bus is silent did not wait afresh: %s %+v", p.State, p.Modules["a"])
}
}
func TestAPlanWithAModuleWaitingForTheBusIsOneTheLookComesBackTo(t *testing.T) {
p, _ := aTierOfThree()
if busWaiting(*p) {
t.Fatal("a plan whose modules are not asked yet waits for the bus")
}
now := time.Now()
p.Modules["b"] = &inventory.PlanModule{BusWaitFrom: &now}
p.Modules["a"] = &inventory.PlanModule{State: "asked"}
if got := busWaitingIn(*p); len(got) != 1 || got[0] != "b" {
t.Fatalf("the modules waiting for the bus are %v, not b", got)
}
p.State = inventory.PlanFailed
if busWaiting(*p) {
t.Fatal("a plan that is not open is waited for")
}
}
func TestAModulesOwnFaultStillFailsThePlan(t *testing.T) {
asksOfEveryModule(t)
p, byName := aTierOfThree()
delete(byName, "a")
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || !strings.Contains(p.Note, "no longer in the catalogue") {
t.Fatalf("a module gone from the catalogue did not fail the plan: %s %q", p.State, p.Note)
}
p, byName = aTierOfThree()
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
return "", errors.New("git.example/novox/a is no repository the catalogue builds a module from")
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if p.State != inventory.PlanFailed || p.Modules["a"].State != "failed" {
t.Fatalf("a module's own refusal did not fail the plan: %s %+v", p.State, p.Modules["a"])
}
}
-35
View File
@@ -1,16 +1,12 @@
package main package main
import ( import (
"errors"
"fmt" "fmt"
"slices" "slices"
"sort" "sort"
"strings" "strings"
"time" "time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
@@ -18,37 +14,6 @@ import (
// (issue 1334). // (issue 1334).
const busWaitBound = 10 * time.Minute const busWaitBound = 10 * time.Minute
// busDidNotAnswer says an ask failed because the bus's JetStream had no responder at that moment, as it has for a few seconds
// when the bus reloads its user list (novox/hq issue 353), and not because of the module: the same ask is made again.
func busDidNotAnswer(err error) bool {
if err == nil {
return false
}
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, nats.ErrJetStreamNotEnabled) ||
errors.Is(err, jetstream.ErrJetStreamNotEnabled) || errors.Is(err, inventory.ErrNoChain) {
return true
}
// An error that crossed a boundary as text only.
text := err.Error()
return strings.Contains(text, "no responders available") || strings.Contains(text, "jetstream not enabled")
}
// waitForTheBus leaves a module whose ask the bus did not answer unasked, the plan building, and says so in the plan's note: the
// next look asks the tier's unasked modules again. It says false, so the caller fails the plan as for any refusal, once the bus
// has been silent for busWaitBound.
func waitForTheBus(p *inventory.Plan, state *inventory.PlanModule, name string, err error, now time.Time) bool {
if state.BusWaitFrom == nil {
state.BusWaitFrom = &now
}
if now.Sub(*state.BusWaitFrom) > busWaitBound {
return false
}
state.State, state.Why = "", ""
p.Note = fmt.Sprintf("tier %d: %s not asked: the bus did not answer (%v); asked again at the next look, failing after %s",
p.Tier, name, firstLine(err.Error()), busWaitBound)
return true
}
// busWaiting says an open plan has a module of its current tier that the bus did not answer an ask of: the look must come back // busWaiting says an open plan has a module of its current tier that the bus did not answer an ask of: the look must come back
// for it, as nothing else will when no build of the tier is in flight (issue 1334). // for it, as nothing else will when no build of the tier is in flight (issue 1334).
func busWaiting(p inventory.Plan) bool { func busWaiting(p inventory.Plan) bool {
@@ -33,7 +33,7 @@ func TestAWarrantForRetryRetriesAFailedPlanInTheStoreAndEndsItsCondition(t *test
Note: "a failed to build in tier 0", Note: "a failed to build in tier 0",
Delivery: &inventory.PlanDelivery{}, Delivery: &inventory.PlanDelivery{},
Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1", Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1",
Why: link.KilledByHand}}, Why: "killed by hand"}},
Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}} Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &failed); err != nil { if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err) t.Fatal(err)
+2 -18
View File
@@ -66,18 +66,8 @@ func TestAPlanGoesOnWithoutAMachineAwayWhenItsTierIsSent(t *testing.T) {
// The build machine and the machines, as far as the plan meets them: an ask is counted, a send is recorded and kept. // The build machine and the machines, as far as the plan meets them: an ask is counted, a send is recorded and kept.
var mu sync.Mutex var mu sync.Mutex
var asks []string
var sends [][]string var sends [][]string
asksOfEveryModule(t) // the one ask left is the builder's own (ADR 0453 §5): this test is of what a walk does around it everyKeyDone(t) // the builds of the tiers: each module is taken from a done key as soon as the walk reaches it
wasAsk := askABuild
askABuild = func(_ context.Context, source buildSource, _, _ string) (string, error) {
mu.Lock()
defer mu.Unlock()
id := link.NewBuildID(time.Now().Add(time.Duration(len(asks)) * time.Millisecond))
asks = append(asks, source.Repository+"#"+id)
return id, nil
}
t.Cleanup(func() { askABuild = wasAsk })
digests := 0 digests := 0
wasSend := sendRollout wasSend := sendRollout
sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) { sendRollout = func(ctx context.Context, open *stores, names []string) ([]string, error) {
@@ -199,13 +189,7 @@ func TestAPlanGoesOnWithoutAMachineAwayWhenItsTierIsSent(t *testing.T) {
if !strings.Contains(moved, "laptop was away: sent when it is heard again") || strings.Contains(moved, "anchor") { if !strings.Contains(moved, "laptop was away: sent when it is heard again") || strings.Contains(moved, "anchor") {
t.Errorf("1161/AC2: the plan says %q; want laptop named away and not anchor", moved) t.Errorf("1161/AC2: the plan says %q; want laptop named away and not anchor", moved)
} }
// The ask for tier 1's build is the next step, and records its build id on the module. // Tier 1's module is taken from its key, which the walk's look does by itself.
eventually(t, "tier 1 asked", func() bool { m := plan().Modules["b"]; return m != nil && m.Build != "" })
b := plan().Modules["b"]
askedAt, _ := link.BuildAskedAt(b.Build)
record(b.Build, "b", "c2", askedAt)
register("b", "c2", askedAt)
planBuilt(ctx, open, "b", "c2", "", askedAt, b.Build)
eventually(t, "tier 1 sent to anchor alone", func() bool { eventually(t, "tier 1 sent to anchor alone", func() bool {
mu.Lock() mu.Lock()
defer mu.Unlock() defer mu.Unlock()
+1 -1
View File
@@ -264,7 +264,7 @@ func TestAPlanLineNamesWhatItMoves(t *testing.T) {
} }
return nil return nil
} }
if got, want := planLineOn(p, now, pauseView{}, tierAtLeast, running), if got, want := planLineOn(p, now, tierAtLeast, running),
"mesh-catalog #175 a tap shows its outcome · messenger, telegram → novox · tier 1 of 1, building for 2m0s"; got != want { "mesh-catalog #175 a tap shows its outcome · messenger, telegram → novox · tier 1 of 1, building for 2m0s"; got != want {
t.Fatalf("the line reads %q, want %q", got, want) t.Fatalf("the line reads %q, want %q", got, want)
} }
-656
View File
@@ -2,24 +2,15 @@ package main
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
"github.com/novox/mesh-controller/internal/conditions" "github.com/novox/mesh-controller/internal/conditions"
"io" "io"
"os" "os"
"slices"
"strings"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
) )
// asking a build machine for a module, and what came back. // asking a build machine for a module, and what came back.
@@ -28,80 +19,14 @@ import (
// cheapest next step. That is how novox/hq ADR 0001 records `hal/sdk` reaching 34,636: // cheapest next step. That is how novox/hq ADR 0001 records `hal/sdk` reaching 34,636:
// nothing in it was wrong, and no one edit was the one that should have been a new file. // nothing in it was wrong, and no one edit was the one that should have been a new file.
// buildFrom turns what a builder said into what the mesh keeps.
//
// **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder
// says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store
// reference and composes the store's address back in where a reference is used. `against` — what
// the build stood on, the catalogue's edge — is recorded the same way, so an edge names a module's
// artifact and not the machine it was pulled from.
func buildFrom(result link.BuildResult) inventory.Build {
kept := inventory.Build{
ID: result.ID, Repository: result.Repository, Ref: result.Ref,
Commit: result.Commit, On: result.On, Failed: result.Failed,
// **What the announcement carries, kept rather than discarded** (novox/hq 04-ISSUES/050).
// The catalogue turns the manifest into requires/provides edges and `against` into build
// edges, and it is not always listening when a build happens — on a fresh mesh it cannot
// be, for exactly the modules it needs most. Keeping them is what makes a replay able to
// rebuild the graph rather than a list of names.
Path: result.Path,
// What it was made from (novox/hq issue 280): two builds with one are one build.
SourceFingerprint: result.SourceFingerprint,
SourceInputs: result.SourceInputs,
// And its reuse key (novox/hq ADR 0350 §1), which the chain names (ADR 0408 §1).
ReuseKey: result.ReuseKey,
}
// When it was asked, which is what orders it against another build of the same module
// (novox/hq 04-ISSUES/219) — not when it was heard.
if asked, ok := link.BuildAskedAt(result.ID); ok {
kept.Asked = asked
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
for _, ref := range result.Mirrored {
kept.Mirrored = append(kept.Mirrored, catalogue.Recorded(ref))
}
for _, r := range result.Read {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
// What it was made from, as files (novox/hq ADR 0267): the planner maps the next merge onto it.
for _, s := range result.Sources {
kept.Sources = append(kept.Sources, inventory.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
var announced []inventory.Artifact
for _, made := range result.Made {
announced = append(announced, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
kept.Made = append(kept.Made, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: catalogue.Recorded(made.Reference),
})
}
kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 {
if list, err := judgedWith(result); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module
}
}
}
return kept
}
// buildsCommand says what has been built lately. // buildsCommand says what has been built lately.
func buildsCommand(ctx context.Context, args []string) error { func buildsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("builds", flag.ContinueOnError) set := flag.NewFlagSet("builds", flag.ContinueOnError)
limit := set.Int("n", 20, "how many to show") limit := set.Int("n", 20, "how many to show")
logOf := set.String("log", "", "a build's id: print what the build machine said, line by line")
positionals, err := parseAround(set, args) positionals, err := parseAround(set, args)
if err != nil { if err != nil {
return err return err
} }
if *logOf != "" {
return buildLog(ctx, *logOf)
}
module := "" module := ""
if len(positionals) == 1 { if len(positionals) == 1 {
module = positionals[0] module = positionals[0]
@@ -182,365 +107,6 @@ func printBuild(w io.Writer, b inventory.Build) {
} }
} }
// builderCommand issues a build machine its own broker credential.
//
// **A build machine is not a node**, and giving it a node's account would let it read another
// machine's declarations. This is narrower and different: read the build queue, write the
// exchange and an asker's reply queue, and nothing else.
//
// Issued rather than assumed, because until this the builder used whatever credential it was
// handed — which in practice meant the broker's own administrative one. A program documented as
// holding its own credential and given somebody else's is worse than one with no story at all.
func builderCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("builder issue", flag.ContinueOnError)
// Which machine will use it. Given, the credential is delivered by the mesh rather than
// printed for somebody to carry — which is the difference between the builder being a module
// and being a program somebody configures.
forNode := set.String("node", "",
"the machine that will run it, so the mesh delivers the credential instead of printing it")
module := set.String("module", "builder", "the module on that machine that will read it")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 2 || positionals[0] != "issue" {
return errors.New("builder issue <name> [--node <machine>]")
}
name := positionals[1]
// **The build machine's credential is a module's credential** (novox/hq ADR 0131, design 28
// task 5.5): minted into the mesh's records and sealed to the machine as the builder module's
// broker secret, usable at the next push — the same act `module issue` performs, and the same
// account the composed user list carries. Nothing is created on a server; the bus reads the list.
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
m, known := shelf[*module]
if !known {
return fmt.Errorf("%s is not in the catalogue; `module add` it first", *module)
}
fmt.Printf("build machine %s: ", name)
node := *forNode
if node == "" {
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
for _, e := range entries {
if e.Manifest.Module == *module && len(e.On) > 0 {
node = e.On[0]
}
}
}
if node == "" {
return fmt.Errorf("%s is assigned nowhere; `assign <machine> %s` first, or say --node", *module, *module)
}
address, err := broker.BusAddress()
if err != nil {
return err
}
return issueOnTheNewBus(ctx, inv, m, node, address)
}
// recordAsked keeps a build request once it is asked (novox/hq issue 325), with who asked it, in a store
// opened for the purpose: the asks reach here from processes that hold none.
func recordAsked(ctx context.Context, r inventory.BuildRequest) {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "build %s is asked and not kept as a build request: %v\n", r.ID, err)
return
}
defer open.Close()
recordBuildRequest(ctx, open.inventory, r)
}
// markWaitFailed says what became of a kept build request whose waited ask failed: never handed over, so
// nothing runs; or handed over and no longer waited for, so asked with its outcome unknown — still read as in
// flight until its outcome or its bound (novox/hq issue 325).
func markWaitFailed(ctx context.Context, id string, why error) {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "build %s could not be marked: %v\n", id, err)
return
}
defer open.Close()
mark := open.inventory.MarkOutcomeUnknown
if errors.Is(why, link.ErrNotHandedOver) {
mark = open.inventory.MarkNotAsked
}
if err := mark(ctx, id, why.Error()); err != nil {
fmt.Fprintf(os.Stderr, "build %s could not be marked: %v\n", id, err)
}
}
// buildOneAsked is buildOne answering the id it asked with — what a plan keeps to match the outcome
// by (novox/hq ADR 0219) — and, for an ask not waited for, optionally a dry run: built and looked
// at, never taken in (issue 240), which is what `replay` asks unless told to register.
//
// asker is who asked, for the build request kept once the ask is made (novox/hq issue 325); empty for an
// asker that keeps the request itself, with its own name, once it knows the ask was made.
func buildOneAsked(ctx context.Context, source buildSource, path, ref string, wait time.Duration,
dryRun bool, asker string) (string, error) {
if dryRun && wait != 0 {
return "", errors.New("a dry run waited for is `build --dry-run`")
}
// Before anything is asked of a builder: a source on a seat nobody holds is refused here, with
// the reason, rather than sent to a machine to fail at `git clone`.
repository, err := cloneFrom(ctx, source)
if err != nil {
return "", err
}
// Through a verb, only a repository the catalogue builds from (novox/hq ADR 0266).
if err := verbMayAsk(ctx, source, repository, path); err != nil {
return "", err
}
ident, err := openIdentity(ctx)
if err != nil {
return "", err
}
defer ident.Close()
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return "", err
}
defer server.Close()
// Correlated by something the control plane makes, not by the module's name: two builds of one
// module can be in flight, and the second answer is not the first one's.
request := link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Path: path,
Ref: ref,
Held: heldBy(ctx),
Seats: seatBases(ctx),
DryRun: dryRun,
}
fmt.Printf("asked for %s", source)
if source.Seat != "" {
fmt.Printf(" (%s)", repository)
}
// The id is how a person follows this build while it runs: `builds --log <id>`.
fmt.Printf(" as %s", request.ID)
if path != "" {
fmt.Printf(" at %s", path)
}
if ref != "" {
fmt.Printf(" on %s", ref)
}
fmt.Println()
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return "", err
}
defer ask.Close()
fmt.Printf(" of %s\n", seat)
// Kept once it is asked, so `assign` knows a module is coming while its build runs (novox/hq issue 325);
// never before, so an ask that failed never reads as a build in flight. A dry run registers nothing, and
// is not kept.
keep := !dryRun && asker != ""
// Asked at the terminal is what lets its outcome register a module from a repository the catalogue does
// not build it from (novox/hq ADR 0266); never through a verb.
asked := inventory.BuildRequest{ID: request.ID, Repository: source.Repository, Seat: source.Seat, Path: path,
Ref: ref, For: asker, AtTerminal: startedAtTheTerminal()}
if wait == 0 {
// Asked and not waited for (novox/hq issue 176): the outcome is the role's event, and the
// controller takes it in — records the build, registers the module — whether or not anybody
// is still here. A tool call cannot hold a connection for the minutes a build takes; it
// follows the build by its id instead.
if err := ask.Ask(ctx, request); err != nil {
return "", err
}
if keep {
recordAsked(ctx, asked)
}
if dryRun {
fmt.Printf("asked as a dry run, not waited for: `builds --log %s` follows it as it runs; "+
"its outcome is not taken in\n", request.ID)
return request.ID, nil
}
fmt.Printf("asked, not waited for: `builds --log %s` follows it as it runs, and `builds` "+
"shows what came of it; the module is registered when the outcome comes\n", request.ID)
return request.ID, nil
}
// Waited for: kept while it is waited for, since that can take minutes, and marked when the hand-over
// failed (not asked) or the wait did (asked, outcome unknown) — an outcome heard later is the last word.
if keep {
recordAsked(ctx, asked)
}
result, err := ask.Submit(ctx, request, wait)
if err != nil {
if keep {
markWaitFailed(ctx, request.ID, err)
}
return request.ID, err
}
open, err := openStores(ctx)
if err != nil {
return request.ID, err
}
defer open.Close()
manifest, kept, err := takeIn(ctx, open.inventory, result)
// The assignments pending on this build, made or ended (novox/hq issue 325).
for _, line := range settlePendingOnBuild(ctx, open, result, manifest.Module, err) {
fmt.Printf(" %s\n", line)
}
if err != nil {
return request.ID, err
}
// Said as recorded: what each artifact is, not where this builder happened to push it.
for _, made := range kept.Made {
fmt.Printf(" %-12s %s %s\n", made.Name, made.Kind, made.Reference)
}
fmt.Printf("\n%s %s, built on %s from %s\n",
manifest.Module, manifest.Version, result.On, short(result.Commit))
saysWhenThePolicyActs(ctx, open.inventory, manifest.Module)
fmt.Printf(" run `assign <node> %s` to put it somewhere\n", manifest.Module)
return request.ID, nil
}
// saysWhenThePolicyActs tells whoever built a module that its upgrade policy will send the
// result on at once (novox/hq issue 126, ADR 0163): a person choreographing a data move must
// know which module will not wait for them.
func saysWhenThePolicyActs(ctx context.Context, inv *inventory.Inventory, module string) {
if u, err := inv.UpgradeOf(ctx, module); err == nil && u.RollOut {
how := "one machine at a time"
if u.Together {
how = "every machine at once"
}
fmt.Printf(" %s rolls out on build: the machines running it are sent this now, %s — "+
"`upgrade %s record` first if something must move before it does\n", module, how, module)
}
}
// takeIn is what the mesh does with a build's outcome, whoever hears it: the waiting command and
// the daemon that follows the role's events both come here (novox/hq issue 176), so a build's
// result reaches the catalogue whether or not the asker was still listening.
//
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something. Then parsed with the same parser a hand-written manifest goes through — a second path
// would be a second thing to disagree about what a manifest is — and registered with where it came
// from: **for a source on a seat, as the path and the seat, never the URL just cloned** (ADR 0111),
// which the request carried and the outcome echoes. A definition naming an installation is refused
// here, where it would enter the catalogue; the build stays recorded and the refusal says which.
//
// Idempotent: the same outcome taken in twice registers the same module twice, which is one row
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
// The list of predefined keys as the store keeps it now: another controller process may have kept a newer one.
if err := loadPredefined(ctx, inv); err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("the list of predefined keys kept cannot be "+
"read: %w", err)
}
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
// Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
// When it was asked, so an older request heard later does not replace a newer one
// (novox/hq 04-ISSUES/219).
Asked: kept.Asked,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
// **A build at a commit does not change the branch a module follows** (novox/hq 04-ISSUES/215):
// the commit is built and recorded as what it was built from, and the module keeps following
// what it followed before — the repository's default branch for one new to the catalogue.
if followedBranch(result.Ref) == "" && result.Ref != "" {
recorded.Ref = ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
}
}
trunk := result.Trunk
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil && followedBranch(was.Ref) != "" {
trunk = followedBranch(was.Ref)
}
if trunk == "" {
trunk = "main"
}
j := takenBuild{kept: kept, manifest: manifest, recorded: recorded, ask: byBuild(result.ID), path: result.Path,
trunk: trunk, by: result.On, what: result.Repository, commit: result.Commit}
// The checks only an outcome of build-agent needs: a key is a trunk build by construction (ADR 0501 §2), in no
// scratch namespace, and carries no list of its own.
j.published = func() error {
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a commit off its repository's
// default branch — a pull request's head, a feature branch built by hand, a `rebuild` or `replay
// --register` of one — is for checking, and is never a module's version; nothing could then send it.
// The branch the module already follows is its trunk — never the branch a build was asked at, or a
// build of a feature branch by name would make that branch its trunk.
follows := ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
follows = followedBranch(was.Ref)
}
if err := publishable(result, follows); err != nil {
return err
}
// **Never a build in the scratch namespace** (novox/hq ADR 0239 §9, ADR 0350 §2): what a check builds is
// registered only as the walk's copy in the module's own namespace.
if scratch := inScratch(result.Made); scratch != "" {
return fmt.Errorf("%s is in the scratch namespace, which only a walk's copy leaves (ADR 0350)", scratch)
}
return nil
}
j.carried = func() error {
// **The newest trunk build's list of predefined keys is published** (novox/hq ADR 0348 §7): a build of the
// catalogue that may be registered keeps its list when its commit is newer than the one kept.
carried, err := carriedList(result)
if err != nil {
return err
}
if carried != nil {
if err := publishPredefined(ctx, inv, result, carried); err != nil {
fmt.Printf("%s: the list of predefined keys it carried is not published: %v\n", result.ID, err)
}
}
return nil
}
manifest, err = registerJudged(ctx, inv, j)
return manifest, kept, err
}
// takenBuild is a build the controller judges and registers, whatever told it of the build: build-agent's outcome (the // takenBuild is a build the controller judges and registers, whatever told it of the build: build-agent's outcome (the
// builder's own module's, until R4g, ADR 0453 §5) or the builder's done key (ADR 0501 §1). **One judging for both**: // builder's own module's, until R4g, ADR 0453 §5) or the builder's done key (ADR 0501 §1). **One judging for both**:
// the rules of who may register, the gate a build failed, the predefined keys and the rule that an unchanged build // the rules of who may register, the gate a build failed, the predefined keys and the rule that an unchanged build
@@ -643,44 +209,6 @@ func registerJudged(ctx context.Context, inv *inventory.Inventory, j takenBuild)
return manifest, nil return manifest, nil
} }
// inScratch is the first artifact a build made into the scratch namespace; empty when none is.
func inScratch(made []link.MadeArtifact) string {
for _, m := range made {
if path, kept := catalogue.InArtifactStore(catalogue.Recorded(m.Reference)); kept && strings.HasPrefix(path, builder.ScratchPrefix) {
return m.Reference
}
}
return ""
}
// errOffTheTrunk is a build of a commit off its repository's trunk, which is never published.
var errOffTheTrunk = errors.New("the commit is not on its repository's trunk: a commit off the trunk is checked, " +
"never published (ADR 0238) — merge it, and the merge builds it")
// publishable says whether a build's outcome may become a module's version: its commit on the module's
// trunk, as the build seat read the forge at the build — the branch the module follows when its source
// names one, else its repository's default branch. A build seat that could not say — one older than the
// rule, building its own successor — is let through and said, so the rule can reach the mesh.
func publishable(result link.BuildResult, follows string) error {
if result.Check != nil || result.Checked != nil || result.DryRun {
return errors.New("a check or a dry run is never published")
}
if result.Trunk == "" {
fmt.Printf("%s: the build seat did not say whether %s is on its repository's trunk (it predates ADR 0238); "+
"registered as before\n", result.ID, short(result.Commit))
return nil
}
trunk := result.Trunk
if follows != "" {
trunk = follows
}
on := slices.Contains(result.Branches, trunk) || (trunk == result.Trunk && result.OnTrunk)
if !on {
return fmt.Errorf("%w (%s is not on %s)", errOffTheTrunk, short(result.Commit), trunk)
}
return nil
}
// answers is what the three questions came back with, read once. // answers is what the three questions came back with, read once.
type answers struct { type answers struct {
// notAnnounced are the owners some module reads the state of that have announced nothing yet (novox/hq ADR 0495, // notAnnounced are the owners some module reads the state of that have announced nothing yet (novox/hq ADR 0495,
@@ -703,8 +231,6 @@ type answers struct {
reported []inventory.Reported reported []inventory.Reported
// plans is what the last merges produced and where each stands (novox/hq ADR 0162). // plans is what the last merges produced and where each stands (novox/hq ADR 0162).
plans []inventory.Plan plans []inventory.Plan
// paused is whether the build seat takes work, which a plan waiting on it says (ADR 0219).
paused pauseView
// tierBounds is how long each repository's plan tier may take before it is late (novox/hq issue 296). // tierBounds is how long each repository's plan tier may take before it is late (novox/hq issue 296).
tierBounds tierBounds tierBounds tierBounds
// refused is why a machine cannot be worked out at all, by name. A different thing from every // refused is why a machine cannot be worked out at all, by name. A different thing from every
@@ -766,185 +292,3 @@ type answers struct {
lastSent map[string]inventory.Send lastSent map[string]inventory.Send
lastSentUnread string lastSentUnread string
} }
// heldBy is every artifact this mesh has built, for a build that may need one as its base.
//
// **A failure here is not a failure to build.** A module that names no base does not need this at
// all, and one that does gets a refusal naming exactly what is missing — which is a better sentence
// than a build command refusing to start because a query did not run. So the store not opening is
// reported and the build goes ahead without it.
//
// Routed through the artifact store as the network reaches it now (novox/hq 04-ISSUES/102): a
// base is recorded by digest and path, and a build machine needs something it can pull.
// runningBy is, keyed "<module>/<artifact>", each artifact of the build the mesh has registered for its module, as a
// builder reaches the artifact store (novox/hq issue 721): what a check compares its builds with. Empty when it cannot
// be read, said, and then nothing is said unchanged.
func runningBy(ctx context.Context) map[string]string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
defer open.Close()
running, err := open.inventory.RegisteredHeld(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh runs, so the check compares nothing: %v\n", err)
// empty-on-error: said above; a check given nothing to compare with says no module unchanged, and all move
return nil
}
address, err := whereABuilderReachesTheStore(ctx, open.inventory)
if err != nil || address == "" {
return running
}
routed := make(map[string]string, len(running))
for key, reference := range running {
routed[key] = catalogue.Rerouted(reference, address)
}
return routed
}
func heldBy(ctx context.Context) map[string]string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh has built, so a module naming a "+
"base will be told that base is missing: %v\n", err)
// empty-on-error: said above; a build that names a base is refused by name for want of it
return nil
}
defer open.Close()
held, err := open.inventory.Held(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what this mesh has built: %v\n", err)
// empty-on-error: said above; a build that names a base is refused by name for want of it
return nil
}
address, err := whereABuilderReachesTheStore(ctx, open.inventory)
if err != nil {
fmt.Fprintf(os.Stderr, "could not find the artifact store on this mesh's network, so a "+
"module naming a base will be handed a reference nothing can fetch: %v\n", err)
return held
}
if address == "" {
return held
}
routed := make(map[string]string, len(held))
for repository, reference := range held {
routed[repository] = catalogue.Rerouted(reference, address)
}
return routed
}
// askOver opens the way a build is asked for, on whichever bus the mesh is on.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5). On the bus
// the mesh runs on today this needs the controller's own connection, so it is handed one; on the bus
// being built it dials, because a build request is a one-shot and holds nothing else.
func askOverOn(seat string) (link.Builders, error) {
// The serving controller asks on its own connection (novox/hq issue 327).
if serving := servingBus.Load(); serving != nil {
return link.BuildsOn(serving, seat), nil
}
address, err := broker.BusAddress()
if err != nil {
return nil, err
}
return link.BuildsOverNATSOn(address, seat)
}
// buildSeatHeld is the build role to ask: the one some assigned module claims (novox/hq ADR 0190,
// the handover). Read from the catalogue at ask time, because the answer changes exactly once, the
// moment the first build-agent is assigned — and a controller that asked the new role before then
// would queue work nothing takes, while the outcome that registers build-agent itself has to come
// from the old builder. When the catalogue cannot be read the current role is asked, said aloud.
func buildSeatHeld(ctx context.Context) string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what is assigned, so the build is asked of %s: %v\n",
link.TheBuildMachine, err)
return link.TheBuildMachine
}
defer open.Close()
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read the catalogue, so the build is asked of %s: %v\n",
link.TheBuildMachine, err)
return link.TheBuildMachine
}
return buildSeatAmong(entries)
}
// buildSeatAmong is the rule, over what the catalogue holds: the current build role when any
// assigned module claims it; else the retired role while an assigned module still claims that; else
// the current role, which is where every ask goes once the handover is done.
func buildSeatAmong(entries []inventory.Entry) string {
heldBefore := false
for _, e := range entries {
if len(e.On) == 0 {
continue
}
if e.Manifest.ClaimsSeat(link.TheBuildMachine) {
return link.TheBuildMachine
}
if e.Manifest.ClaimsSeat(link.TheBuildMachineBefore) {
heldBefore = true
}
}
if heldBefore {
return link.TheBuildMachineBefore
}
return link.TheBuildMachine
}
// buildLog prints everything a build machine said about one build, read back from the bus.
//
// **From the stream, not from a record** (novox/hq ADR 0157). A build's lines are the role's own
// events under the build's id, retained with every other event; the mesh keeps no second copy. Read
// with a consumer of its own that is gone when this returns, so nothing accumulates in the server
// for the reading, and filtered by subject, so one build's lines are all that travel.
func buildLog(ctx context.Context, id string) error {
js, err := aBus()
if err != nil {
return fmt.Errorf("cannot reach the bus to read a build's log: %w", err)
}
defer js.Close()
// Under whichever build role did it: a build asked of the retired role during the handover
// (ADR 0190) said its lines as that role's events, and a reader should not have to know which.
lines := link.BuildLogOf("*", id)
sub, err := js.Context().PullSubscribe(lines, "",
nats.BindStream(broker.EventsStream), nats.DeliverAll(), nats.AckNone())
if err != nil {
return fmt.Errorf("cannot read %s from the bus: %w", lines, err)
}
defer func() { _ = sub.Unsubscribe() }()
printed := 0
for {
batch, err := sub.Fetch(200, nats.MaxWait(2*time.Second))
if err != nil && !errors.Is(err, nats.ErrTimeout) && !errors.Is(err, context.DeadlineExceeded) {
return fmt.Errorf("reading a build's log: %w", err)
}
for _, msg := range batch {
var line link.BuildLine
if err := json.Unmarshal(msg.Data, &line); err != nil {
fmt.Printf(" ? %s\n", string(msg.Data))
continue
}
at := line.At
if t, err := time.Parse(time.RFC3339Nano, line.At); err == nil {
at = t.Local().Format("15:04:05")
}
fmt.Printf("%s %4d [%s] %s\n", at, line.Seq, line.Step, line.Message)
printed++
}
if len(batch) < 200 {
break
}
}
if printed == 0 {
fmt.Printf("nothing on the bus for build %s: no build by that id in the last week, or a build "+
"machine older than this that said nothing while building\n", id)
}
return nil
}
-43
View File
@@ -1,43 +0,0 @@
package main
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func claiming(module, seat string, on ...string) inventory.Entry {
return inventory.Entry{
Manifest: catalogue.Manifest{Module: module, Claims: []catalogue.Claim{{Name: seat}}},
On: on,
}
}
// The controller asks the build role that has a holder (novox/hq ADR 0190 handover): the retired
// one while only the builder is assigned, the current one from the first build-agent on, and the
// current one when nothing holds either — where every ask goes once the handover is done.
func TestTheControllerAsksTheBuildRoleThatHasAHolder(t *testing.T) {
onlyTheBuilder := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine), // registered, assigned nowhere yet
}
if got := buildSeatAmong(onlyTheBuilder); got != link.TheBuildMachineBefore {
t.Errorf("with only the builder assigned, asked %q", got)
}
bothHeld := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine, "home-server"),
}
if got := buildSeatAmong(bothHeld); got != link.TheBuildMachine {
t.Errorf("with a build-agent assigned anywhere, asked %q", got)
}
neither := []inventory.Entry{claiming("builder", link.TheBuildMachineBefore)}
if got := buildSeatAmong(neither); got != link.TheBuildMachine {
t.Errorf("with no holder of either, asked %q, want the current role", got)
}
if got := buildSeatAmong(nil); got != link.TheBuildMachine {
t.Errorf("an empty catalogue asks %q", got)
}
}
+1 -43
View File
@@ -189,19 +189,6 @@ func (f *sourceForms) same(aRepository, aSeat, bRepository, bSeat string) bool {
return f.canonical(aRepository, aSeat) == f.canonical(bRepository, bSeat) return f.canonical(aRepository, aSeat) == f.canonical(bRepository, bSeat)
} }
// buildsFrom says the catalogue builds some module from this repository.
func (f *sourceForms) buildsFrom(entries []inventory.Entry, repository, seat string) bool {
for _, e := range entries {
if e.Provided || e.Source.Repository == "" {
continue
}
if f.same(e.Source.Repository, e.Source.Seat, repository, seat) {
return true
}
}
return false
}
// forgeFacts is what the mesh's forge says of a repository a module is registered from (novox/hq ADR 0266). // forgeFacts is what the mesh's forge says of a repository a module is registered from (novox/hq ADR 0266).
type forgeFacts struct { type forgeFacts struct {
// ID is the forge's own id of the repository: what tells it from one deleted and made again by its name. // ID is the forge's own id of the repository: what tells it from one deleted and made again by its name.
@@ -487,7 +474,7 @@ func (f *sourceForms) onTheForge(repository, seat string) (owner, name string, o
// - from the module's registered repository — the same repository by the forge's own id, not only its name; or, // - from the module's registered repository — the same repository by the forge's own id, not only its name; or,
// for a module new to the catalogue, from a repository the catalogue builds another module from; // for a module new to the catalogue, from a repository the catalogue builds another module from;
// - and from a repository on the mesh's forge whose trunk is protected as a trunk must be: no direct push, at // - and from a repository on the mesh's forge whose trunk is protected as a trunk must be: no direct push, at
// least one required status, no administrator merging past one. // a merge whitelist naming the tools' account and no agent's, no administrator merging past the rule (ADR 0534 §6).
// //
// Anything else only when the build request was kept as asked at the controller's terminal, for this very // Anything else only when the build request was kept as asked at the controller's terminal, for this very
// repository and path; or, for the build of a merge, when mesh-delivery announced it on the operator's Approve of // repository and path; or, for the build of a merge, when mesh-delivery announced it on the operator's Approve of
@@ -512,9 +499,6 @@ type registrationAsk struct {
Terminal bool Terminal bool
} }
// byBuild is the warrant of a build's outcome: read from the request kept under its id.
func byBuild(id string) registrationAsk { return registrationAsk{Build: id} }
// String is the ask as the registration's words name it. // String is the ask as the registration's words name it.
func (a registrationAsk) String() string { func (a registrationAsk) String() string {
switch { switch {
@@ -720,29 +704,3 @@ func sourceWords(repository, seat string) string {
} }
return buildSource{Repository: repository, Seat: seat}.String() return buildSource{Repository: repository, Seat: seat}.String()
} }
// verbMayAsk refuses, for a call through a verb, a build of a repository the catalogue builds no module from
// (novox/hq ADR 0266): the build node would run what an agent wrote, and its outcome could never be
// registered anyway. url is the repository as it is cloned. At the terminal anything may be asked.
func verbMayAsk(ctx context.Context, source buildSource, url, path string) error {
if startedAtTheTerminal() {
return nil
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
return err
}
forms := newSourceForms(ctx, open.inventory)
if forms.buildsFrom(entries, source.Repository, source.Seat) || forms.buildsFrom(entries, url, "") {
return nil
}
return terminalRefusal("%s is no repository the catalogue builds a module from, and a build of any other is "+
"asked at the controller's terminal only, never through a verb: a build node runs what the repository "+
"says, and its outcome would register a module the next push sends — whoever may call a verb includes "+
"agents (novox/hq ADR 0266). Nothing was asked", source)
}
@@ -129,27 +129,6 @@ func TestADeliveryFromTheRegisteredRepositoryIsRegistered(t *testing.T) {
} }
} }
func TestAVerbAsksNoBuildOfARepositoryTheCatalogueDoesNotBuildFrom(t *testing.T) {
theCatalogue(t)
ctx := t.Context()
t.Setenv(verbVar, "build")
t.Setenv(link.CallerVar, "node-tools.anchor, through the mesh-controller seat")
err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "http://forge.internal:20000/agent/sudo.git", "")
var policy *heldAtTheTerminal
if !errors.As(err, &policy) {
t.Fatalf("a verb's build of an agent's repository was asked: %v", err)
}
if err := verbMayAsk(ctx, buildSource{Repository: "novox/mesh-catalog", Seat: "git"},
"http://forge.internal:20000/novox/mesh-catalog.git", ""); err != nil {
t.Fatalf("a verb's build of the catalogue repository was refused: %v", err)
}
t.Setenv(verbVar, "")
t.Setenv(link.CallerVar, "")
if err := verbMayAsk(ctx, buildSource{Repository: "agent/sudo", Seat: "git"}, "", ""); err != nil {
t.Fatalf("the terminal was refused: %v", err)
}
}
func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) { func TestARollbackNeverPutsBackABuildFromAnotherRepository(t *testing.T) {
open := theCatalogue(t) open := theCatalogue(t)
ctx := t.Context() ctx := t.Context()
-11
View File
@@ -2,12 +2,10 @@ package main
import ( import (
"bytes" "bytes"
"reflect"
"strings" "strings"
"testing" "testing"
"time" "time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
@@ -68,12 +66,3 @@ func TestBuildsSaysTheSourceFingerprint(t *testing.T) {
t.Errorf("the other lines changed:\n%s\nwant\n%s", plain.String(), want) t.Errorf("the other lines changed:\n%s\nwant\n%s", plain.String(), want)
} }
} }
// 1215/AC4: what the builder said of the inputs is what the controller keeps of the build.
func TestTheControllerKeepsTheInputsABuilderSaid(t *testing.T) {
in := &sdkdelivery.SourceInputs{Module: "mailu", Tree: "t", Toolchains: map[string]string{"code": "ts sha256:aa 01"}}
kept := buildFrom(link.BuildResult{ID: "build-1", Repository: "r", Commit: "c", SourceFingerprint: "src1:x", SourceInputs: in})
if !reflect.DeepEqual(kept.SourceInputs, in) {
t.Errorf("the controller keeps %+v, the builder said %+v", kept.SourceInputs, in)
}
}
@@ -4,7 +4,6 @@ package main
import ( import (
"context" "context"
"sync"
"testing" "testing"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery" sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
@@ -106,34 +105,5 @@ func (b builderOnTheBus) built(t *testing.T, module, version, fp, manifest, stat
} }
} }
// noBuildAsked fails the test if anything asks build-agent for a build (ADR 0501 §6, ADR 0503 §6).
func noBuildAsked(t *testing.T) {
t.Helper()
was := askABuild
askABuild = func(_ context.Context, _ buildSource, path, _ string) (string, error) {
t.Errorf("a build was asked of build-agent, at %q", path)
return "", context.Canceled
}
t.Cleanup(func() { askABuild = was })
}
// asksWithPaths records every build a merge asks for, as repository, path and ref. The merge window's goroutine and
// the test's own may ask at once, so the record is written under a lock (issue 1543).
func asksWithPaths(t *testing.T) *[][3]string {
t.Helper()
asksOfEveryModule(t) // the one ask left is the builder's own (ADR 0453 §5): these tests are of what a walk does around it
var asked [][3]string
var mu sync.Mutex
was := askABuild
askABuild = func(_ context.Context, source buildSource, path, ref string) (string, error) {
mu.Lock()
defer mu.Unlock()
asked = append(asked, [3]string{source.Repository, path, ref})
return "b-" + path, nil
}
t.Cleanup(func() { askABuild = was })
return &asked
}
// theIndexedModule is the look at the indexer as the controller makes it, kept before any test replaces it. // theIndexedModule is the look at the indexer as the controller makes it, kept before any test replaces it.
var theIndexedModule = indexedModule var theIndexedModule = indexedModule
@@ -1,9 +0,0 @@
package main
import "context"
// A test's builder says it writes references, as R4e's does, unless the test says otherwise (issue 1886): no test reads
// the bus for it.
func init() {
builderSaysReferences = func(context.Context) (bool, error) { return true, nil }
}
-243
View File
@@ -1,243 +0,0 @@
package main
import (
"context"
"errors"
"strings"
"testing"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/inventory"
)
// theBuilderSays sets what the running builder says for one test.
func theBuilderSays(t *testing.T, says bool, err error) {
t.Helper()
was := builderSaysReferences
builderSaysReferences = func(context.Context) (bool, error) { return says, err }
t.Cleanup(func() { builderSaysReferences = was })
}
// askingBuildAgent records the modules build-agent is asked for, by path.
func askingBuildAgent(t *testing.T) *[]string {
t.Helper()
var asked []string
was := askABuild
askABuild = func(_ context.Context, _ buildSource, path, _ string) (string, error) {
asked = append(asked, path)
return "build-9", nil
}
t.Cleanup(func() { askABuild = was })
return &asked
}
// 1886/AC2: while no running builder says it writes references, a walk asks build-agent for a module of its tier, as
// before R4e, and takes nothing from the builder's key.
func TestAWalkAsksBuildAgentWhileTheBuilderDoesNotSayItWritesReferences(t *testing.T) {
theBuilderSays(t, false, nil)
asked := askingBuildAgent(t)
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
s := p.Modules["sudo"]
if s.Key || s.Build != "build-9" || s.State != "asked" || len(*asked) != 1 {
t.Fatalf("the module is not asked of build-agent: %+v, asked %v", s, *asked)
}
}
// 1886/AC3: once a running builder says it writes references, a walk takes the module from the builder's key (ADR
// 0498 §8) and asks build-agent for nothing.
func TestAWalkTakesTheKeyOnceTheBuilderSaysItWritesReferences(t *testing.T) {
theBuilderSays(t, true, nil)
asked := askingBuildAgent(t)
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
if s := p.Modules["sudo"]; !s.Key || s.Build != "" || len(*asked) != 0 {
t.Fatalf("the module is not taken from its key: %+v, asked %v", s, *asked)
}
}
// 1886/AC2: a say that cannot be read leaves the module unasked and says why, so the next look asks it: neither path is
// guessed.
func TestASayNotReadLeavesTheModuleUnasked(t *testing.T) {
theBuilderSays(t, false, errors.New("the bus did not answer"))
asked := askingBuildAgent(t)
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
s := p.Modules["sudo"]
if s.State != "" || s.Key || len(*asked) != 0 || !strings.Contains(s.Why, "the bus did not answer") {
t.Fatalf("a say not read: %+v, asked %v", s, *asked)
}
if got := unaskedOf(*p); len(got) != 1 {
t.Fatalf("the module is not left for the next look to ask: %v", got)
}
}
// 1886/AC2: the one function every ask goes through lets build-agent build any module while the builder does not say
// it writes references, and refuses it again once it does.
func TestBuildAgentIsAskedForAnyModuleWhileTheBuilderDoesNotSay(t *testing.T) {
was := askedModuleOf
t.Cleanup(func() { askedModuleOf = was })
askedModuleOf = func(_ context.Context, _ buildSource, path string) (string, error) {
return strings.TrimPrefix(path, "modules/"), nil
}
theBuilderSays(t, false, nil)
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/mesh-catalog", Seat: "git"}, "modules/sudo"); err != nil {
t.Fatalf("an ask while the builder does not say is refused: %v", err)
}
theBuilderSays(t, true, nil)
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/mesh-catalog", Seat: "git"}, "modules/sudo"); !errors.Is(err, errNotTheBuildersOwn) {
t.Fatalf("an ask once the builder says is not refused: %v", err)
}
}
// 1886/AC4: an open walk that waits on a key the running builder cannot reference (done without a reference, as R4e's
// own walk did in plan-1792707174627079192) is judged again at the next look: the module is handed back unasked, and
// that look's ask goes to build-agent. With the builder saying it writes references, the same walk keeps waiting.
func TestAnOpenWalkWaitingOnAnUnreferencedKeyIsAskedOfBuildAgent(t *testing.T) {
t.Cleanup(func() {
keyTakes.Lock()
keyTakes.running, keyTakes.waits, keyTakes.taken = map[string]bool{}, map[string]string{}, map[string]bool{}
keyTakes.Unlock()
})
fp := "src2:" + strings.Repeat("e", 64)
unreferenced := builderkeys.Key{Module: "sudo", Fingerprint: fp, Found: true, Revision: 1,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildDone, Manifest: strings.Repeat("b", 64),
Artifacts: []sdkdelivery.BuiltArtifact{{Name: "tools", Kind: "bundle", Digest: "sha256:1"}}}}
walk := func() (*inventory.Plan, map[string]inventory.Entry, planKeysOf) {
theBuilderSays(t, true, nil)
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
keys := planKeysOf{values: map[string]sdkdelivery.Module{"sudo@c0ffee0123456789": {FP2: fp}},
keys: map[string]builderkeys.Key{"sudo": unreferenced}}
return p, byName, keys
}
take := func(string, sdkdelivery.Module, string) { t.Fatal("an unreferenced key was taken in") }
records := recordsOf{builds: map[string]inventory.Build{}}
// Said: it waits for the reference, as ADR 0498 §8 has it.
p, _, keys := walk()
if _, err := settleKeys(t.Context(), records, keys, p, p.Tiers[0], take); err != nil {
t.Fatal(err)
}
if s := p.Modules["sudo"]; s.State != "asked" || !s.Key || !strings.Contains(s.Why, "waits for the reference") {
t.Fatalf("with the builder saying, the walk does not wait for the reference: %+v", s)
}
// Not said: handed back, and the next look asks build-agent.
p, byName, keys := walk()
theBuilderSays(t, false, nil)
asked := askingBuildAgent(t)
changed, err := settleKeys(t.Context(), records, keys, p, p.Tiers[0], take)
if err != nil || !changed {
t.Fatalf("the walk was not judged again: %v %v", changed, err)
}
s := p.Modules["sudo"]
if s.Key || s.State != "" || p.State != inventory.PlanBuilding || !strings.Contains(s.Why, "does not say") {
t.Fatalf("the module is not handed back to be asked of build-agent: %+v %s", s, p.State)
}
for _, m := range unaskedOf(*p) {
askModule(t.Context(), p, m, byName)
}
if s := p.Modules["sudo"]; s.Key || s.Build != "build-9" || len(*asked) != 1 {
t.Fatalf("the next look did not ask build-agent: %+v, asked %v", s, *asked)
}
}
// 1886/AC4: a key the running builder failed while it does not say it writes references fails no walk: the module is
// asked of build-agent, whose build is the one delivered.
func TestAFailedKeyOfABuilderThatDoesNotSayFailsNoWalk(t *testing.T) {
theBuilderSays(t, true, nil)
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
_ = byName
fp := "src2:" + strings.Repeat("f", 64)
theBuilderSays(t, false, nil)
keys := planKeysOf{values: map[string]sdkdelivery.Module{"sudo@c0ffee0123456789": {FP2: fp}},
keys: map[string]builderkeys.Key{"sudo": {Module: "sudo", Fingerprint: fp, Found: true,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildFailed, Error: "not stored: tools: no reference"}}}}
if _, err := settleKeys(t.Context(), recordsOf{}, keys, p, p.Tiers[0], func(string, sdkdelivery.Module, string) {}); err != nil {
t.Fatal(err)
}
if s := p.Modules["sudo"]; p.State != inventory.PlanBuilding || s.State != "" || s.Key {
t.Fatalf("a failed key of a builder that does not say failed the walk: %s %q %+v", p.State, p.Note, s)
}
}
// 1886/AC6 (ADR 0522 §6): the parts that write references, the builder and the build modules, are build-agent's own until
// R4g: asked of build-agent and never taken from keys they write themselves, whatever the builder says.
func TestTheReferenceWritersAreAskedOfBuildAgentWhateverTheBuilderSays(t *testing.T) {
theBuilderSays(t, true, nil)
asked := askingBuildAgent(t)
for _, m := range []string{"mesh-build-go", "mesh-build-node", "mesh-build-image", "mesh-delivery-builder"} {
p, byName := aWalkOfOne(m)
askModule(t.Context(), p, m, byName)
if s := p.Modules[m]; s.Key || s.Build != "build-9" {
t.Errorf("%s is not asked of build-agent: %+v", m, s)
}
}
if len(*asked) != 4 {
t.Fatalf("build-agent was asked %v", *asked)
}
}
// 1886 review round 1 (ADR 0522 §2): the guard of the one ask function judges by the say askModule read, never a second
// read: a second read that fails, or that says otherwise, fails no plan.
func TestTheAskGuardUsesTheSayTheWalkRead(t *testing.T) {
was := askedModuleOf
t.Cleanup(func() { askedModuleOf = was })
askedModuleOf = func(_ context.Context, _ buildSource, path string) (string, error) {
return strings.TrimPrefix(path, "modules/"), nil
}
reads := 0
wasSays := builderSaysReferences
t.Cleanup(func() { builderSaysReferences = wasSays })
for _, second := range []struct {
name string
says bool
err error
}{{"a second read that fails", false, errors.New("the bus did not answer")}, {"a say that turned true", true, nil}} {
reads = 0
builderSaysReferences = func(context.Context) (bool, error) {
reads++
if reads == 1 {
return false, nil
}
return second.says, second.err
}
wasAsk := askABuild
askABuild = func(ctx context.Context, source buildSource, path, _ string) (string, error) {
if err := onlyTheBuildersOwn(ctx, source, path); err != nil {
return "", err
}
return "build-9", nil
}
p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName)
askABuild = wasAsk
if s := p.Modules["sudo"]; p.State == inventory.PlanFailed || s.Build != "build-9" || s.Key {
t.Errorf("%s: the walk %s, the module %+v", second.name, p.State, s)
}
}
}
// 1886 merge check hang: the look must judge an ask and a settle by one say. A settle that read the say from another
// source than the ask (the bus's bucket, against the walk's read) handed a module back, the next ask took it from its key
// again, and the look's loop never ended: walk_from_keys_integration_test hung.
func TestTheSettleJudgesByTheSameSayAsTheAsk(t *testing.T) {
theBuilderSays(t, true, nil)
p, _ := aWalkOfOne("sudo")
p.Modules["sudo"] = &inventory.PlanModule{}
askModule(t.Context(), p, "sudo", map[string]inventory.Entry{"sudo": {Source: inventory.Source{Repository: "novox/mesh-catalog", Path: "modules/sudo", Ref: "main"}}})
fp := "src2:" + strings.Repeat("1", 64)
keys := planKeysOf{values: map[string]sdkdelivery.Module{"sudo@c0ffee0123456789": {FP2: fp}},
keys: map[string]builderkeys.Key{"sudo": {Module: "sudo", Fingerprint: fp, Found: true,
Build: sdkdelivery.BuilderBuild{State: sdkdelivery.BuildBuilding, By: "shanks"}}}}
if _, err := settleKeys(t.Context(), recordsOf{}, keys, p, p.Tiers[0], func(string, sdkdelivery.Module, string) {}); err != nil {
t.Fatal(err)
}
if s := p.Modules["sudo"]; !s.Key || s.State != "asked" {
t.Fatalf("the settle judged by another say than the ask's: %+v", s)
}
}
-112
View File
@@ -1,112 +0,0 @@
package main
import (
"context"
"errors"
"strings"
"testing"
)
// A12 (ADR 0453 §5, ADR 0498 §7; issue 1851): the one function every ask goes through refuses any module but the
// builder's own, naming the builder's key it is taken from instead; the builder's own module passes the refusal.
func TestBuildAgentIsAskedForTheBuildersOwnModuleAlone(t *testing.T) {
was := askedModuleOf
t.Cleanup(func() { askedModuleOf = was })
askedModuleOf = func(_ context.Context, _ buildSource, path string) (string, error) {
return strings.TrimPrefix(path, "modules/"), nil
}
_, err := askABuild(t.Context(), buildSource{Repository: "novox/mesh-catalog", Seat: "git"}, "modules/sudo", "main")
if !errors.Is(err, errNotTheBuildersOwn) || !strings.Contains(err.Error(), "sudo is taken from the builder's done key") ||
!strings.Contains(err.Error(), "mesh-delivery-builder_builds") {
t.Fatalf("an ask of another module: %v", err)
}
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/mesh-delivery", Seat: "git"},
"modules/mesh-delivery-builder"); err != nil {
t.Fatalf("the builder's own module is refused: %v", err)
}
askedModuleOf = func(context.Context, buildSource, string) (string, error) { return "", errors.New("no store") }
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/x"}, ""); !errors.Is(err, errNotTheBuildersOwn) {
t.Fatalf("an ask whose module cannot be read went on: %v", err)
}
}
// A12: a walk of the builder's own module and one other asks build-agent for the first alone; the other is taken from
// its key.
func TestAWalkAsksBuildAgentForTheBuildersOwnModuleAlone(t *testing.T) {
var asked []string
was := askABuild
askABuild = func(_ context.Context, _ buildSource, path, _ string) (string, error) {
asked = append(asked, path)
return "build-1", nil
}
t.Cleanup(func() { askABuild = was })
p, byName := aWalkOfOne("sudo")
p.Tiers = [][]string{{"mesh-delivery-builder", "sudo"}}
byName["mesh-delivery-builder"] = byName["sudo"]
e := byName["mesh-delivery-builder"]
e.Source.Repository, e.Source.Path = "novox/mesh-delivery", "modules/mesh-delivery-builder"
byName["mesh-delivery-builder"] = e
for _, m := range p.Tiers[0] {
askModule(t.Context(), p, m, byName)
}
if len(asked) != 1 || asked[0] != "modules/mesh-delivery-builder" {
t.Fatalf("build-agent was asked for %v, want the builder's own module alone", asked)
}
if s := p.Modules["mesh-delivery-builder"]; s.Key || s.Build != "build-1" {
t.Fatalf("the builder's own module is not asked: %+v", s)
}
if s := p.Modules["sudo"]; !s.Key || s.Build != "" {
t.Fatalf("the other module is not taken from its key: %+v", s)
}
}
// ADR 0520 §1 (issue 1851): until R4g build-agent and mesh-tools, the toolchain image its builds stand on, are build-agent's
// own, as the builder's module is: the builder writes no key of them, so a walk asks build-agent for them and reads no key;
// mesh-tools-go (its key from mesh-build-image) and any other module of the same tier are still taken from their keys.
func TestAWalkAsksBuildAgentForItselfAndMeshToolsAndReadsNoKeyOfThem(t *testing.T) {
own := []string{"build-agent", "mesh-tools"}
keyed := []string{"mesh-tools-go", "sudo"}
wasModule := askedModuleOf
askedModuleOf = func(_ context.Context, _ buildSource, path string) (string, error) {
return strings.TrimPrefix(path, "modules/"), nil
}
t.Cleanup(func() { askedModuleOf = wasModule })
for _, m := range own {
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/x", Seat: "git"}, "modules/"+m); err != nil {
t.Fatalf("an ask of %s is refused: %v", m, err)
}
}
if err := onlyTheBuildersOwn(t.Context(), buildSource{Repository: "novox/x", Seat: "git"}, "modules/mesh-tools-go"); !errors.Is(err, errNotTheBuildersOwn) {
t.Fatalf("an ask of mesh-tools-go went on: %v", err)
}
var asked []string
was := askABuild
askABuild = func(_ context.Context, _ buildSource, path, _ string) (string, error) {
asked = append(asked, path)
return "build-" + strings.TrimPrefix(path, "modules/"), nil
}
t.Cleanup(func() { askABuild = was })
p, byName := aWalkOfOne("sudo")
p.Tiers = [][]string{append(append([]string{}, own...), keyed...)}
for _, m := range append(append([]string{}, own...), keyed...) {
e := byName["sudo"]
e.Source.Path = "modules/" + m
byName[m] = e
}
for _, m := range p.Tiers[0] {
askModule(t.Context(), p, m, byName)
}
if strings.Join(asked, " ") != "modules/build-agent modules/mesh-tools" {
t.Fatalf("build-agent was asked for %v", asked)
}
for _, m := range own {
if s := p.Modules[m]; s.Key || s.Build != "build-"+m || s.State != "asked" {
t.Fatalf("%s is not asked of build-agent, or is read from a key: %+v", m, s)
}
}
for _, m := range keyed {
if s := p.Modules[m]; !s.Key || s.Build != "" {
t.Fatalf("%s is not taken from its key: %+v", m, s)
}
}
}
+4 -17
View File
@@ -14,7 +14,6 @@ import (
"github.com/novox/mesh-controller/internal/chain" "github.com/novox/mesh-controller/internal/chain"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/lease" "github.com/novox/mesh-controller/internal/lease"
"github.com/novox/mesh-controller/internal/link"
) )
// **Acceptance 10 of design 47: one read of an assignment names the commit, the build and the rollout that put it there, // **Acceptance 10 of design 47: one read of an assignment names the commit, the build and the rollout that put it there,
@@ -80,7 +79,7 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ { for try := 0; try < 6 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == "" || asking.Modules["notes"].State == "asked"); try++ {
advancePlans(ctx, open) advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil { if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -90,13 +89,6 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if buildID == "" || len(*asked) != 1 || !strings.HasSuffix((*asked)[0], "#"+buildID) { if buildID == "" || len(*asked) != 1 || !strings.HasSuffix((*asked)[0], "#"+buildID) {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID) t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
} }
digest := "sha256:" + strings.Repeat("a", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.ReuseKey = "rk-notes"
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// The walk sends the tier to its first machine. // The walk sends the tier to its first machine.
for try := 0; try < 3 && len(sent) == 0; try++ { for try := 0; try < 3 && len(sent) == 0; try++ {
@@ -116,9 +108,9 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
if wireguard, pulled := assignmentOf(on, "mesh-wireguard"); pulled && (wireguard.Build != "" || wireguard.Rollout != "") { if wireguard, pulled := assignmentOf(on, "mesh-wireguard"); pulled && (wireguard.Build != "" || wireguard.Rollout != "") {
t.Fatalf("a module this rollout did not send names a build or a rollout: %+v", wireguard) t.Fatalf("a module this rollout did not send names a build or a rollout: %+v", wireguard)
} }
if a.Module != "notes" || a.Commit != "c-notes" || a.Build != buildID || a.Rollout != rollout || a.Artifact != digest || if a.Module != "notes" || a.Commit != "c-notes" || a.Build != buildID || a.Rollout != rollout ||
a.Since.IsZero() || a.Assigned.IsZero() || !a.Stands() { a.Since.IsZero() || a.Assigned.IsZero() || !a.Stands() {
t.Fatalf("the assignment names %+v, not commit c-notes, build %s, artifact %s, rollout %s", a, buildID, digest, rollout) t.Fatalf("the assignment names %+v, not commit c-notes, build %s, rollout %s", a, buildID, rollout)
} }
// One read of the rollout. // One read of the rollout.
p, err := inv.PlanByID(ctx, rollout) p, err := inv.PlanByID(ctx, rollout)
@@ -130,16 +122,11 @@ func TestOneReadOfAnAssignmentNamesItsCommitBuildAndRolloutAndOneReadOfTheRollou
p.Steps[0].Node != "anchor" || p.Steps[0].Sent == nil { p.Steps[0].Node != "anchor" || p.Steps[0].Sent == nil {
t.Fatalf("the rollout lists changed %+v, builds %v, steps %+v; want %+v", p.Changed, p.Builds, p.Steps, wantChange) t.Fatalf("the rollout lists changed %+v, builds %v, steps %+v; want %+v", p.Changed, p.Builds, p.Steps, wantChange)
} }
// And one read of the build names the same commit, with what it published and its reuse key. // And one read of the build names the same commit.
built, found, err := inv.BuildByID(ctx, buildID) built, found, err := inv.BuildByID(ctx, buildID)
if err != nil || !found || built.Commit != "c-notes" || built.Module != "notes" { if err != nil || !found || built.Commit != "c-notes" || built.Module != "notes" {
t.Fatalf("the build: %+v %v %v", built, found, err) t.Fatalf("the build: %+v %v %v", built, found, err)
} }
lately, err := inv.Builds(ctx, "notes", 1)
if err != nil || len(lately) != 1 || lately[0].ReuseKey != "rk-notes" || lately[0].Published == nil ||
lately[0].Published.Artifact != digest {
t.Fatalf("what the build published: %+v %v", lately, err)
}
// The verbs say the links. // The verbs say the links.
if lines := changedLines(p); len(lines) != 2 || !strings.Contains(lines[1], "anchor/notes") || if lines := changedLines(p); len(lines) != 2 || !strings.Contains(lines[1], "anchor/notes") ||
!strings.Contains(lines[1], buildID) { !strings.Contains(lines[1], buildID) {
@@ -6,8 +6,6 @@ import (
"strings" "strings"
"testing" "testing"
"time" "time"
"github.com/novox/mesh-controller/internal/link"
) )
// **Acceptance 12 of design 47: no table in the controller's store holds builds, rollouts or assignments** (novox/hq ADR // **Acceptance 12 of design 47: no table in the controller's store holds builds, rollouts or assignments** (novox/hq ADR
@@ -34,7 +32,7 @@ func TestNoTableOfTheControllersStoreHoldsBuildsRolloutsOrAssignments(t *testing
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
for try := 0; try < 3 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == ""); try++ { for try := 0; try < 6 && (asking.Modules["notes"] == nil || asking.Modules["notes"].Build == "" || asking.Modules["notes"].State == "asked"); try++ {
advancePlans(ctx, open) advancePlans(ctx, open)
if asking, err = inv.PlanByID(ctx, rollout); err != nil { if asking, err = inv.PlanByID(ctx, rollout); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -44,12 +42,6 @@ func TestNoTableOfTheControllersStoreHoldsBuildsRolloutsOrAssignments(t *testing
if buildID == "" || len(*asked) != 1 { if buildID == "" || len(*asked) != 1 {
t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID) t.Fatalf("the walk asked %v, and notes' build is %q", *asked, buildID)
} }
digest := "sha256:" + strings.Repeat("b", 64)
result := outcome(buildID, "modules/notes", "c-notes", "notes", "")
result.Made = []link.MadeArtifact{{Name: "notes", Kind: "image", Reference: "artifact-store://notes/notes@" + digest}}
if err := (builds{inv: inv, open: open}).Built(ctx, result); err != nil {
t.Fatal(err)
}
// What the live path made is in the chain. // What the live path made is in the chain.
if built, found, err := inv.BuildByID(ctx, buildID); err != nil || !found || built.Commit != "c-notes" { if built, found, err := inv.BuildByID(ctx, buildID); err != nil || !found || built.Commit != "c-notes" {
-29
View File
@@ -190,32 +190,3 @@ func summaryOf(p link.ChangePlan) string {
} }
return strings.Join(parts, "; ") return strings.Join(parts, "; ")
} }
// planText is a change plan as a person reads it, for the pull request's comment.
func planText(p link.ChangePlan) string {
var b strings.Builder
fmt.Fprintf(&b, "change plan of %s at %.8s into %s: %s\n", p.Repository, p.Head, p.Base, p.Summary)
for i, tier := range p.Tiers {
fmt.Fprintf(&b, " tier %d: %s\n", i, strings.Join(tier, ", "))
}
for _, m := range p.Machines {
line := " " + m.Machine + ": "
if len(m.Receives) > 0 {
line += "receives " + strings.Join(m.Receives, ", ")
}
if len(m.Waits) > 0 {
if len(m.Receives) > 0 {
line += "; "
}
line += "waits for a person: " + strings.Join(m.Waits, ", ")
}
b.WriteString(line + "\n")
}
for _, s := range p.Steps {
b.WriteString(" - " + s + "\n")
}
if len(p.Unread) > 0 {
fmt.Fprintf(&b, " read by no module's build: %s\n", strings.Join(p.Unread, ", "))
}
return b.String()
}
-3
View File
@@ -72,7 +72,4 @@ func TestAChangePlanSaysWhatEachMachineReceives(t *testing.T) {
if !strings.HasPrefix(p.Summary, "builds nothing") || len(p.Machines) != 0 || strings.Join(p.Unread, ",") != "merge-check.sh" { if !strings.HasPrefix(p.Summary, "builds nothing") || len(p.Machines) != 0 || strings.Join(p.Unread, ",") != "merge-check.sh" {
t.Errorf("a root file's plan reads %+v", p) t.Errorf("a root file's plan reads %+v", p)
} }
if !strings.Contains(planText(p), "read by no module's build: merge-check.sh") {
t.Errorf("the plan's text does not say why nothing is built:\n%s", planText(p))
}
} }
-63
View File
@@ -1,63 +0,0 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
)
// A pull request's check builds go when it closes (novox/hq ADR 0350 §5): the build seat keeps them in the artifact
// store's scratch namespace for the walk of its merge to take. **Let go of where the moment is heard**: the
// controller ends the walk answering a merge — after that walk took what it reuses — and the delivery's owner, which
// hears the forge say a pull request closed unmerged, asks the `forget-checks` verb; both let go through the store,
// as the controller's own collection does. The build seat's holders let go of anything a week old on their own,
// so a word lost here leaves a week's builds, never more.
// checkBuildsIn is the scratch namespace as the controller reaches the artifact store.
func checkBuildsIn(ctx context.Context, inv *inventory.Inventory) (builder.Scratch, error) {
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
if address == "" {
return nil, errors.New("the artifact store is not on the mesh's network")
}
return builder.Registry{Address: address}, nil
}
// forgetChecks lets go of a pull request's check builds and answers what it did.
var forgetChecks = func(ctx context.Context, inv *inventory.Inventory, pull builder.PullRef) (string, error) {
scratch, err := checkBuildsIn(ctx, inv)
if err != nil {
return "", err
}
n, err := scratch.Forget(ctx, pull)
if err != nil {
return "", fmt.Errorf("%s: %d check(s)' builds let go of, and not all: %w", pull, n, err)
}
return fmt.Sprintf("%s: %d check(s)' builds let go of", pull, n), nil
}
// forgetChecksOfWalk lets go, on the save that ends a walk, of the check builds of every pull request whose merge
// it answers: the walk took what it reuses. Said, never fatal.
func forgetChecksOfWalk(ctx context.Context, inv *inventory.Inventory, p inventory.Plan) {
if p.WalkSaid != inventory.WalkEnded || p.Delivery == nil {
return
}
for _, pull := range pullsOfWalk(&p) {
said, err := forgetChecks(ctx, inv, builder.PullRef{Owner: pull.Owner, Repo: pull.Repo, Number: pull.Number})
if err != nil {
fmt.Fprintf(os.Stderr, "%s: its check builds stay until the week's sweep: %v\n", p.ID, err)
continue
}
fmt.Printf("%s: %s\n", p.ID, said)
}
}
@@ -1,91 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestAWalkNamesThePullRequestsOfItsMergesOnce(t *testing.T) {
p := &inventory.Plan{Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}, {Repository: "http://forge.invalid/novox/web.git", Number: 3},
{Repository: "novox/shop", Number: 7}, {Repository: "novox/shop"}}}}
if want := []link.PullRef{{Owner: "novox", Repo: "shop", Number: 7}, {Owner: "novox", Repo: "web", Number: 3}}; !reflect.DeepEqual(pullsOfWalk(p), want) {
t.Fatalf("the walk names %+v", pullsOfWalk(p))
}
}
func TestAReusedCheckBuildIsRegisteredByItsDigestAndGatedAsABuiltOne(t *testing.T) {
withTheGate(t) // the first-machine gate's own path (novox/hq ADR 0360 keeps it for earlier walks)
g := aGateMesh(t)
ctx := t.Context()
inv := g.open.inventory
gateEvery = 0
digest := "sha256:" + strings.Repeat("c", 64)
reused := func(id, reference string) link.BuildResult {
manifest := mustJSON(t, catalogue.Manifest{Module: "app", Version: "c3", Resources: []map[string]any{
{"id": "svc", "type": "container", "name": "app", "image": reference}}})
return link.BuildResult{ID: id, Repository: "novox/mesh-catalog", Path: "modules/app", Ref: "main", On: "anchor",
Commit: "c3", Manifest: manifest, Trunk: "main", OnTrunk: true, Branches: []string{"main"},
Made: []link.MadeArtifact{{Name: "server", Kind: "image", Reference: reference}},
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"},
Reused: "reused from the check of novox/mesh-catalog#9 at head abcdefabcdef (merged into trunk 123456123456): server " + digest,
ReusedFrom: &link.ReusedFrom{Pull: link.PullRef{Owner: "novox", Repo: "mesh-catalog", Number: 9},
Head: "abcdefabcdef", Trunk: "123456123456", Key: "r1:" + strings.Repeat("1", 64)}}
}
if _, _, err := takeIn(ctx, inv, reused(link.NewBuildID(time.Now()), "store.internal:5000/scratch/app/server@"+digest)); err == nil ||
!strings.Contains(err.Error(), "scratch") {
t.Fatalf("a build in the scratch namespace was registered: %v", err)
}
id := link.NewBuildID(time.Now().Add(time.Second))
result := reused(id, "store.internal:5000/app/server@"+digest)
if _, _, err := takeIn(ctx, inv, result); err != nil {
t.Fatal(err)
}
current, err := inv.CurrentBuilds(ctx)
if err != nil || current["app"].Commit != "c3" {
t.Fatalf("the reused build is not what app is registered at: %+v %v", current["app"], err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(shelf["app"].Resources)
if !strings.Contains(string(raw), "app/server@"+digest) {
t.Fatalf("app is registered as %s, not by the check's digest in its own namespace", raw)
}
// The walk asked it: the walk keeps what its build did, and its gate judges the registered build.
asked := time.Now().UTC()
p := g.plan(t)
p.Commit = "c3"
p.Modules["app"] = &inventory.PlanModule{State: "asked", AskedAt: &asked, Build: id}
if err := inv.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
planBuiltSaying(ctx, g.open, "app", "c3", "", asked, id, result.MadeWords())
p = g.plan(t)
if m := p.Modules["app"]; m.State != "built" || !strings.HasPrefix(m.Made, "reused from the check of novox/mesh-catalog#9") {
t.Fatalf("the walk keeps %+v", m)
}
for i := 0; i < 6; i++ {
advancePlans(ctx, g.open)
}
p = g.plan(t)
gate := p.Modules["app"].Gate
if gate == nil || gate.To != "c3" || gate.Verdict != inventory.GatePassed {
t.Fatalf("the gate judged %+v, not the reused build", gate)
}
if sent, _, _ := inv.SentBuilds(ctx, "anchor"); sent["app"] != "c3" {
t.Fatalf("the first machine was sent %v, not the reused build", sent)
}
}
-134
View File
@@ -1,134 +0,0 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// One build per change (novox/hq ADR 0350), as the controller asks it and takes it in.
// The check of a change is asked to build what a merge of it would move, in the walk's order: a module of the
// repository checked from the merge, one standing on it from its own repository as the walk builds it, and one
// that reads the repository through a context not at all. A change that moves no module builds nothing.
func TestACheckIsAskedToBuildWhatTheMergeWouldMove(t *testing.T) {
entries := []inventory.Entry{
{Manifest: catalogue.Manifest{Module: "app"}, Source: inventory.Source{Repository: "novox/shop", Seat: "git", Path: "modules/app"}},
{Manifest: catalogue.Manifest{Module: "web"}, Source: inventory.Source{Repository: "novox/web", Seat: "git", Ref: "main"}},
{Manifest: catalogue.Manifest{Module: "docs"}, Source: inventory.Source{Repository: "novox/docs", Seat: "git"}},
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Base: "main", Commit: "abc"}
scope := checkScope{Reach: mergeReach{Packaging: entries[2:], Plan: inventory.Plan{Tiers: [][]string{{"app", "docs"}, {"web"}}}}}
clone := func(s inventory.Source) (string, error) { return "http://forge.invalid/" + s.Repository + ".git", nil }
got := checkBuildsOf(p, scope, entries, clone)
want := []link.CheckBuild{
{Module: "app", Path: "modules/app", Merged: true, Tier: 0},
{Module: "docs", Tier: 0, Skip: "its build reads novox/shop through a context, which a check cannot hand the merge"},
{Module: "web", Repository: "http://forge.invalid/novox/web.git", Ref: "main", Tier: 1},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("the check is asked to build\n%+v\nwant\n%+v", got, want)
}
if got := checkBuildsOf(p, checkScope{}, entries, clone); got != nil {
t.Fatalf("a change that moves no module asks for builds: %+v", got)
}
}
// A walk's ask carries its leave to take a check's build, naming the pull requests of the merges it answers; a dry
// run never does, and neither does an ask that is not a walk's.
// A walk ended lets go of its merges' check builds — only once it ended, after it took what it reuses.
func TestAWalkEndedLetsGoOfItsMergesCheckBuilds(t *testing.T) {
var forgot []builder.PullRef
was := forgetChecks
forgetChecks = func(_ context.Context, _ *inventory.Inventory, pull builder.PullRef) (string, error) {
forgot = append(forgot, pull)
return "", nil
}
t.Cleanup(func() { forgetChecks = was })
p := inventory.Plan{ID: "plan-1", Delivery: &inventory.PlanDelivery{Merges: []inventory.PlanMerge{
{Repository: "novox/shop", Number: 7}}}}
p.WalkSaid = inventory.WalkStarted
forgetChecksOfWalk(t.Context(), nil, p)
p.WalkSaid = ""
forgetChecksOfWalk(t.Context(), nil, p)
if len(forgot) != 0 {
t.Fatalf("a walk not ended let go of %v, which it may still take", forgot)
}
p.WalkSaid = inventory.WalkEnded
forgetChecksOfWalk(t.Context(), nil, p)
if want := []builder.PullRef{{Owner: "novox", Repo: "shop", Number: 7}}; !reflect.DeepEqual(forgot, want) {
t.Fatalf("the walk ended let go of %v", forgot)
}
}
// A build in the scratch namespace is never registered: only the walk's copy in the module's own namespace is.
func TestABuildInTheScratchNamespaceIsNeverRegistered(t *testing.T) {
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: "store.internal:5000/app/server@sha256:aa"}}); got != "" {
t.Fatalf("a module's own build reads as a scratch build: %s", got)
}
scratch := "store.internal:5000/scratch/app/server@sha256:aa"
if got := inScratch([]link.MadeArtifact{{Name: "server", Reference: scratch}}); got != scratch {
t.Fatalf("a scratch build was not found: %q", got)
}
if got := inScratch([]link.MadeArtifact{{Name: "look", Reference: "http://store.internal:5000/v2/scratch/app/look/blobs/sha256:bb"}}); got == "" {
t.Fatal("a scratch archive was not found")
}
}
// The walk says per module what it did: "reused from …" or "built: <why>" — and a build seat from before says
// neither, which is said.
func TestAWalksBuildSaysWhetherItReusedOrBuilt(t *testing.T) {
if got := (link.BuildResult{Reused: "reused from the check of novox/shop#7"}).MadeWords(); got != "reused from the check of novox/shop#7" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{BuiltWhy: "no check built it"}).MadeWords(); got != "built: no check built it" {
t.Fatalf("said %q", got)
}
if got := (link.BuildResult{}).MadeWords(); !strings.Contains(got, "predates ADR 0350") {
t.Fatalf("said %q", got)
}
}
// A walk's module the check's build stood in for is registered by the digest the check built — the copy in the
// module's namespace — and the walk keeps "reused from …" for it; its first-node gate judges that registered
// build as it judges one the walk built. A scratch reference is refused. Needs the store (MESH_TEST_POSTGRES), and
// is named in testdata/store-tests.
// A check is handed only the builds earlier checks of its own pull request at its own head made: another pull
// request's build, or an earlier head's, is never kept (novox/hq issue 703). None when they cannot be read.
func TestACheckIsHandedOnlyItsOwnHeadsBuilds(t *testing.T) {
was := checkedBuildsOf
t.Cleanup(func() { checkedBuildsOf = was })
pull := link.PullRef{Owner: "novox", Repo: "shop", Number: 7}
mine := link.CheckedBuild{Pull: pull, Check: "build-1", Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
earlier := link.CheckedBuild{Pull: pull, Check: "build-0", Head: "aaaaaaaaaaaa", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
another := link.CheckedBuild{Pull: link.PullRef{Owner: "novox", Repo: "shop", Number: 8}, Check: "build-9",
Head: "bbbbbbbbbbbb", Built: link.CheckBuilt{Module: "app", Key: "r1:1"}}
var asked [][]link.PullRef
checkedBuildsOf = func(_ context.Context, pulls []link.PullRef) ([]link.CheckedBuild, error) {
asked = append(asked, pulls)
return []link.CheckedBuild{mine, earlier, another}, nil
}
p := link.PullUpdated{Owner: "novox", Repo: "shop", Number: 7, Commit: "bbbbbbbbbbbb0123456789"}
builds := []link.CheckBuild{{Module: "app"}}
if got := namedForTheCheck(t.Context(), p, builds); !reflect.DeepEqual(got, []link.CheckedBuild{mine}) ||
!reflect.DeepEqual(asked, [][]link.PullRef{{pull}}) {
t.Fatalf("the check is handed %+v, read for %v", got, asked)
}
checkedBuildsOf = func(context.Context, []link.PullRef) ([]link.CheckedBuild, error) {
return nil, errors.New("no bus")
}
if got := namedForTheCheck(t.Context(), p, builds); got != nil {
t.Fatalf("verdicts that could not be read named %+v", got)
}
}
-203
View File
@@ -1,203 +0,0 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
// `check-here` is a pull request's merge check run on the machine at hand **exactly as the build seat
// runs it** (novox/hq issue 286): the same code (builder.Check), the same ask the controller would make of
// the seat — the gate's modules and judge by the planner's own answer over the facts snapshot, the
// repositories beside it at the refs the snapshot says the seat clones them at — in the toolchain image
// the mesh holds, as the user the build seat runs as, against a throwaway store and bus of the versions
// the mesh runs.
//
// **The build seat is the reference.** A merge-check.sh that passed on an agent's machine failed on the
// seat for three reasons that were each the agent's environment, never the change: a newer Go whose gofmt
// lays a file out differently, a sibling checkout at the agent's feature branch where the seat had the
// commit the mesh runs, and a different user. Run here, a check sees what the seat will.
//
// check-here [--tree <checkout>] [--base main] [--registry <artifact store>] [--forge <url of the owner>]
// [--number <n>] [--user uid:gid] [--facts store|<file>] [--keep]
//
// The checkout's HEAD is what is checked, and it must be committed: the seat checks a commit, never a
// working tree.
func checkHereCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("check-here", flag.ContinueOnError)
tree := set.String("tree", ".", "the change's checkout; its HEAD is checked")
base := set.String("base", "main", "the branch the change would merge into")
registry := set.String("registry", os.Getenv("MESH_REGISTRY"), "the artifact store holding the facts and the toolchains")
forge := set.String("forge", "", "where the repositories beside it are cloned from, as <forge>/<repository>.git; "+
"the checkout's origin without its own name when not given")
number := set.Int("number", 0, "the pull request's number, when there is one")
// The build seat's service runs as root, and its check containers run as the builder does.
user := set.String("user", "0:0", "the user the check's containers run as: the build seat's")
keep := set.Bool("keep", false, "keep the workspace afterwards")
factsFrom := set.String("facts", "store", "the facts snapshot: `store`, the one the artifact store holds — "+
"what the seat reads — or a file")
if _, err := parseAround(set, args); err != nil {
return err
}
if *registry == "" {
return errors.New("check-here reads the facts and the toolchains from the artifact store: --registry <host:port> or MESH_REGISTRY")
}
dir, err := filepath.Abs(*tree)
if err != nil {
return err
}
git := func(args ...string) (string, error) {
cmd := exec.CommandContext(ctx, "git", args...)
cmd.Dir = dir
out, err := cmd.Output()
return strings.TrimSpace(string(out)), err
}
if dirty, err := git("status", "--porcelain", "--untracked-files=no"); err != nil {
return fmt.Errorf("%s is not a checkout: %w", dir, err)
} else if dirty != "" {
return errors.New("the checkout has changes not committed: the build seat checks a commit, so commit first")
}
head, err := git("rev-parse", "HEAD")
if err != nil {
return err
}
origin, err := git("remote", "get-url", "origin")
if err != nil {
return fmt.Errorf("the checkout has no origin to say which repository it is: %w", err)
}
owner, repo, prefix := ownerRepoOf(origin)
if *forge == "" {
*forge = prefix
}
if _, err := git("fetch", "--quiet", "origin", *base); err != nil {
return fmt.Errorf("cannot fetch %s to say what the change touches: %w", *base, err)
}
// Without rename detection, so a file moved out of a build source is said under its old name as well:
// its going is a change to the build that held it (novox/hq ADR 0267).
changedText, err := git("diff", "--name-only", "--no-renames", "origin/"+*base+"...HEAD")
if err != nil {
return err
}
var paths, removed []string
for _, p := range strings.Split(changedText, "\n") {
if p = strings.TrimSpace(p); p == "" {
continue
}
paths = append(paths, p)
if _, err := os.Stat(filepath.Join(dir, p)); os.IsNotExist(err) {
removed = append(removed, p)
}
}
_ = os.Setenv("MESH_REGISTRY", *registry)
f, err := readFacts(ctx, *factsFrom)
if err != nil {
return fmt.Errorf("the facts snapshot cannot be read: %w", err)
}
entries, read, edges, err := graphOfFacts(f)
if err != nil {
return err
}
p := link.PullUpdated{Owner: owner, Repo: repo, Number: *number, Base: *base, Commit: head, Paths: paths,
Removed: removed, ModuleDirs: moduleDirsIn(dir, paths), ModuleDirsSaid: true}
scope := pullScope(p, entries, read, edges)
_, scriptErr := os.Stat(filepath.Join(dir, builder.CheckScript))
if !scope.gated() && !scope.Mesh {
fmt.Printf("%s: %s, and the repository is not the mesh's: the build seat runs nothing for it\n",
owner+"/"+repo, noModuleTouched)
return nil
}
if !scope.gated() && scriptErr != nil {
fmt.Printf("%s: %s; %s\n", owner+"/"+repo, noModuleTouched, noMergeCheck)
return nil
}
toolchains := map[string]string{}
for language, reference := range f.Versions.Toolchains {
toolchains[language] = catalogue.Rerouted(reference, *registry)
}
if len(toolchains) == 0 {
return errors.New("the facts snapshot names no toolchain: it was taken by a controller from before " +
"issue 286, and the seat's toolchain cannot be known here")
}
beside := map[string]builder.Beside{}
for d, ref := range f.Beside {
from := d
if d == "mesh-controller-main" {
from = "mesh-controller"
}
beside[d] = builder.Beside{Repository: strings.TrimSuffix(*forge, "/") + "/" + from + ".git", Ref: ref}
}
id := fmt.Sprintf("check-here-%d", time.Now().UnixNano())
spec := builder.CheckSpec{ID: id, Repository: dir, Ref: head, Owner: owner, Repo: repo, Number: *number,
Paths: paths, Beside: beside, Modules: scope.Modules, New: scope.New, Manifests: scope.Manifests,
Base: *base, Judge: scope.Judge, Toolchain: toolchains["go"], Toolchains: toolchains, User: *user}
workspace, err := os.MkdirTemp("", "mesh-check-here-")
if err != nil {
return err
}
if !*keep {
defer removeWorkspace(spec.Toolchain, workspace, *user)
}
fmt.Fprintf(os.Stderr, "checking %s/%s at %.8s as the build seat would, against the facts of %s, in %s\n",
owner, repo, head, f.Taken.Format(time.RFC3339), workspace)
v, err := builder.Check(ctx, builder.Command, spec, workspace, *registry, builder.GitCredential{},
func(step, message string) {
if step != "output" {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
}
})
if err != nil {
return fmt.Errorf("the check could not run — on the seat an error, never a pass: %w", err)
}
fmt.Println(v.Report)
fmt.Println()
fmt.Printf("mesh/merge-gate: %s — %s\n", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary)
if v.Repo != nil {
fmt.Printf("mesh/repo-check: %s — %s\n", strings.ToUpper(v.Repo.Verdict), v.Repo.Summary)
}
for _, l := range []*builder.Layer{v.Gate, v.Repo} {
if l != nil && l.Verdict != "pass" && l.Verdict != "warning" {
return errors.New("the build seat would not pass this change")
}
}
return nil
}
// ownerRepoOf reads owner, repository and the owner's URL from a remote: ssh://git@host:222/novox/mesh-host.git
// → novox, mesh-host, ssh://git@host:222/novox.
func ownerRepoOf(remote string) (string, string, string) {
trimmed := strings.TrimSuffix(strings.TrimSuffix(remote, "/"), ".git")
cut := strings.LastIndexAny(trimmed, "/:")
if cut < 0 {
return "", trimmed, ""
}
repo, prefix := trimmed[cut+1:], trimmed[:cut]
owner := prefix
if at := strings.LastIndexAny(prefix, "/:"); at >= 0 {
owner = prefix[at+1:]
}
return owner, repo, prefix
}
// removeWorkspace removes what the check left, written as the seat's user: by a container of that user
// when it is not this one.
func removeWorkspace(image, workspace, user string) {
if image != "" && user != fmt.Sprintf("%d:%d", os.Getuid(), os.Getgid()) {
// Everything in it — the check's HOME is the workspace, so the toolchain's own files are there too.
_ = exec.Command("docker", "run", "--rm", "--user", user, "--volume", workspace+":/workspace", image,
"sh", "-c", "rm -rf /workspace/* /workspace/.[!.]*").Run()
}
_ = os.RemoveAll(workspace)
}
-44
View File
@@ -1,44 +0,0 @@
package main
import (
"reflect"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// 1212/AC1 and AC2: the check request names the pulls bucket of each module claiming the delivery seat, in the order
// of the claimants' names.
func TestACheckNamesThePullsBucketsOfTheSeatsClaimants(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), namedOnly("mesh-delivery")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery"), namedOnly("gitea")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
"none": {[]inventory.Entry{namedOnly("gitea")}, nil},
} {
if got := pullsBucketsOf(c.entries); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: %v, want %v", name, got, c.want)
}
}
}
// 1212/AC1 and AC2: the check a request carries names those buckets (the wiring, not only the helper).
func TestTheCheckRequestCarriesThePullsBuckets(t *testing.T) {
for name, c := range map[string]struct {
entries []inventory.Entry
want []string
}{
"only the gate": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate")}, []string{"mesh-delivery-gate_pulls"}},
"only today's": {[]inventory.Entry{claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls"}},
"both, name order": {[]inventory.Entry{claimingTheSeat("mesh-delivery-gate"), claimingTheSeat("mesh-delivery")}, []string{"mesh-delivery_pulls", "mesh-delivery-gate_pulls"}},
} {
got := checkOf(link.PullUpdated{Owner: "novox", Repo: "a", Number: 7}, checkScope{}, c.entries, nil, nil, nil, nil)
if !reflect.DeepEqual(got.Pulls, c.want) || got.Owner != "novox" || got.Number != 7 {
t.Errorf("%s: %+v, want pulls %v", name, got, c.want)
}
}
}
+17 -559
View File
@@ -2,9 +2,6 @@ package main
import ( import (
"context" "context"
"encoding/json"
"fmt"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"io" "io"
"os" "os"
"path" "path"
@@ -13,46 +10,18 @@ import (
"slices" "slices"
"sort" "sort"
"strings" "strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
// A pull request's merge check (novox/hq to-be 45 §9, ADR 0237 as amended 2026-10-06): the forge // What a pull request reaches of the mesh's graph (novox/hq to-be 45 §9, issue 280). The controller no longer checks a
// announces every pull request's new head, the controller decides what is checked, asks the build seat to // pull request or says a verdict on it (R4g, ADR 0503 §3, ADR 0501 §5): the checker judges the heads, and mesh-delivery
// check it, and says the verdict as `checked`, which the forge's holder sets as the pull request's // asks the operator's Approve on its verdict. What stays here is the mapping of a pull request's changed paths onto the
// statuses. **Before merge, never after**: every check the mesh had ran after a merge, on a machine. // mesh's module graph by the planner's own answer (reachOfMerge, touchedBy), which the delivery verbs (`delivery order`,
// // the change plan) and the head's declared config (tellPullDeclares) read: **a changed file touches exactly the modules
// **The mesh's module graph decides, not the repository.** The controller holds the graph — every module, // whose build reads it** — a module's own directory (the whole repository for one built from its root), or a repository
// the repository and directory it is built from — and maps the pull request's changed paths onto it by // its recipe packages — and a directory the change adds a module.json in is a new module.
// the planner's own answer (reachOfMerge, touchedBy — the one place a changed file is mapped onto modules,
// for the merge handler, the release planner, the merge gate and this check; issue 280): **a changed file
// touches exactly the modules whose build reads it** — a module's own directory (the whole repository for
// one built from its root), or a repository its recipe packages. A file no build reads — a script at the
// root, a README — touches no module. A directory the change adds a module.json in, which the graph does
// not hold yet (said by the head, issue 278), is a new module and is checked too.
//
// - touches a module: the build seat runs **the gate** — `mesh/merge-gate`, the touched manifests, every
// machine composed with the change, the replays — and the repository's own merge-check.sh beside it;
// - touches none, in a repository that is the mesh's (it sources a module on some branch, or shares the
// core's owner): the gate is a pass that says so — a fact, not a missing check — and the repository's
// own merge-check.sh runs as `mesh/repo-check`, a warning when it has none;
// - touches none, anywhere else: the gate is a pass that says so, and nothing more is said.
//
// What is checked is decided here and run there (internal/builder/check.go).
// checkTimeout is how long one check may run on the build seat. Said here so the ask's watchdog (S6)
// and the builder agree on what late means.
const checkTimeout = 45 * time.Minute
// noModuleTouched is the gate's word for a change that touches nothing of the graph.
const noModuleTouched = "the change touches no module of the mesh's graph"
// noMergeCheck is the repository layer's word for a repository of the mesh with no merge-check.sh.
const noMergeCheck = "the repository declares no merge-check.sh: none of its own tests run before it merges"
// coreModules are the modules whose repositories are the mesh's core, by the directory a check finds // coreModules are the modules whose repositories are the mesh's core, by the directory a check finds
// each beside it — and whose owner is the mesh's own. // each beside it — and whose owner is the mesh's own.
@@ -76,8 +45,6 @@ type checkScope struct {
// Mesh says the repository is the mesh's: modules are built from it on some branch, its owner is the // Mesh says the repository is the mesh's: modules are built from it on some branch, its owner is the
// core's, or the change adds a module to it. // core's, or the change adds a module to it.
Mesh bool Mesh bool
// Judge is who judges the gate (link.JudgeSelf, or the running controller).
Judge string
// From is a module built from the repository, for how the mesh clones it; nil when none is. // From is a module built from the repository, for how the mesh clones it; nil when none is.
From *inventory.Entry From *inventory.Entry
// Reach is the planner's whole answer, which the change plan is made from. // Reach is the planner's whole answer, which the change plan is made from.
@@ -99,10 +66,6 @@ func pullScope(p link.PullUpdated, entries []inventory.Entry, read map[string][]
Width: len(r.Plan.Modules), Tiers: len(r.Plan.Tiers), Reach: r} Width: len(r.Plan.Modules), Tiers: len(r.Plan.Tiers), Reach: r}
for _, e := range append(append([]inventory.Entry{}, r.Touched...), r.Deleted...) { for _, e := range append(append([]inventory.Entry{}, r.Touched...), r.Deleted...) {
s.Manifests = append(s.Manifests, path.Join(strings.Trim(e.Source.Path, "/"), moduleManifestFile)) s.Manifests = append(s.Manifests, path.Join(strings.Trim(e.Source.Path, "/"), moduleManifestFile))
switch e.Manifest.Module {
case "mesh-controller":
s.Judge = link.JudgeSelf
}
} }
for _, d := range r.Added { for _, d := range r.Added {
s.Manifests = append(s.Manifests, path.Join(d, moduleManifestFile)) s.Manifests = append(s.Manifests, path.Join(d, moduleManifestFile))
@@ -138,35 +101,29 @@ func sourceOwner(repository string) string {
return strings.ToLower(parts[len(parts)-2]) return strings.ToLower(parts[len(parts)-2])
} }
// PullUpdated decides a pull request's merge check, and asks for it when there is something to run. // PullUpdated hears a pull request's new head. It asks no check and says no verdict (pullUpdated).
func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error { func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
_, err := f.pullUpdated(ctx, p, os.Stdout) return f.pullUpdated(ctx, p, os.Stdout)
return err
} }
// pullUpdated is PullUpdated saying what it did to w. It returns the id of the check it asked of the build seat, or // pullUpdated is PullUpdated saying what it did to w. It asks nothing of anybody and says no verdict (R4g; ADR 0503 §3, §5,
// "" when it said a verdict itself and asked nothing. A command that answers JSON on standard output (`delivery // ADR 0527): the checker judges every repository whose trunk requires `mesh/check`, hq included. All it still does is tell
// check`) gives it standard error, so its output stays the document alone (novox/hq issue 1311). // mesh-config what the head declares (ADR 0502), whatever judges it.
func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Writer) (string, error) { func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Writer) error {
inv := f.open.inventory inv := f.open.inventory
entries, err := inv.Catalogued(ctx) entries, err := inv.Catalogued(ctx)
if err != nil { if err != nil {
return "", err return err
} }
read, err := readForPlanning(ctx, inv) read, err := readForPlanning(ctx, inv)
if err != nil { if err != nil {
return "", err return err
} }
edges, err := inv.Dependencies(ctx) edges, err := inv.Dependencies(ctx)
if err != nil { if err != nil {
return "", err return err
} }
scope := pullScope(p, entries, read, edges) scope := pullScope(p, entries, read, edges)
// **The merge gate's checks stop** (novox/hq ADR 0503 §3, §5; issue 1299): the checker judges every repository whose
// trunk requires `mesh/check`. A repository whose trunk requires `mesh/repo-check` (hq), whatever else it requires
// (ADR 0513 §4), has its own merge-check.sh asked of build-agent, and said with that layer alone; any other is asked
// nothing and said nothing. What the forge's protection requires is read, never listed (lead decision L11).
// Mesh-config is still told what the head declares (ADR 0502), whatever judges it.
this := pullAtOf(p) this := pullAtOf(p)
if this.Truncated { if this.Truncated {
this.Paths, this.Truncated = truncatedPaths(entries, p.Owner, p.Repo, scope.Modules), false this.Paths, this.Truncated = truncatedPaths(entries, p.Owner, p.Repo, scope.Modules), false
@@ -174,310 +131,7 @@ func (f following) pullUpdated(ctx context.Context, p link.PullUpdated, w io.Wri
if scope.gated() || holdsModules(entries, p.Owner, p.Repo) { if scope.gated() || holdsModules(entries, p.Owner, p.Repo) {
tellPullDeclares(ctx, w, []pullAt{this}) tellPullDeclares(ctx, w, []pullAt{this})
} }
required, err := requiredStatusesOf(ctx, p.Owner, p.Repo, p.Base) return nil
switch {
case err != nil:
fmt.Fprintf(w, "%s/%s#%d (%.8s): what %s's protection requires cannot be read, so nothing is asked and nothing is "+
"said: %v\n", p.Owner, p.Repo, p.Number, p.Commit, p.Base, err)
return "", nil
case !asksRepoCheck(required):
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s does not require %s, and %s; nothing is asked\n", p.Owner, p.Repo, p.Number,
p.Commit, p.Base, sdkdelivery.StatusContextRepoCheck, checkerWords(required))
return "", nil
}
return f.repoCheckAlone(ctx, w, p, scope, entries)
}
// asksRepoCheck says what a trunk's protection has the controller ask: its repository's own merge-check.sh whenever the
// trunk requires `mesh/repo-check`, whatever else it requires (ADR 0513 §4: a trunk requiring both has the checker judge
// `mesh/check` and the repository check asked beside it), and nothing otherwise.
func asksRepoCheck(required []string) bool {
return slices.Contains(required, sdkdelivery.StatusContextRepoCheck)
}
// checkerWords says who judges a trunk that requires no `mesh/repo-check`.
func checkerWords(required []string) string {
if slices.Contains(required, sdkdelivery.StatusContextCheck) {
return "the checker judges its " + sdkdelivery.StatusContextCheck
}
return "does not require " + sdkdelivery.StatusContextCheck + " either, so nothing is said"
}
// repoCheckAlone asks build-agent for a repository's own merge-check.sh alone, for a pull request of a repository whose
// trunk requires `mesh/repo-check`, whether or not it also requires `mesh/check` (ADR 0503 §5, ADR 0513 §4): no gate, no module built, its verdict said with the
// repo-check layer alone.
func (f following) repoCheckAlone(ctx context.Context, w io.Writer, p link.PullUpdated, scope checkScope,
entries []inventory.Entry) (string, error) {
inv := f.open.inventory
plan := changePlanOf(p.Owner+"/"+p.Repo, p.Base, p.Commit, scope.Reach, entries, func(module string) (inventory.Upgrade, bool) {
u, err := inv.UpgradeOf(ctx, module)
return u, err == nil
})
fmt.Fprint(w, planText(plan))
direct := directVerdict(p, plan)
switch {
case !scope.Mesh:
direct.RepoCheck = &link.CheckLayer{Verdict: "warning", Summary: "the repository is not the mesh's: nothing run"}
fmt.Fprintf(w, "%s/%s#%d (%.8s): the repository is not the mesh's: said, nothing run\n", p.Owner, p.Repo, p.Number,
p.Commit)
sayChecked(ctx, w, direct)
return "", nil
case p.MergeCheckSaid && !p.MergeCheck:
direct.RepoCheck = &link.CheckLayer{Verdict: "warning", Summary: noMergeCheck}
fmt.Fprintf(w, "%s/%s#%d (%.8s): %s\n", p.Owner, p.Repo, p.Number, p.Commit, noMergeCheck)
sayChecked(ctx, w, direct)
return "", nil
}
request, err := checkRequestFor(ctx, f.open, p, checkScope{Mesh: true, From: scope.From}, entries)
if err != nil {
return "", err
}
request.Check.Plan = &plan
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil {
return "", err
}
defer ask.Close()
if err := ask.Ask(ctx, request); err != nil {
return "", err
}
fmt.Fprintf(w, "%s/%s#%d (%.8s): asked %s to run its own merge-check.sh alone, as %s\n", p.Owner, p.Repo, p.Number,
p.Commit, seat, request.ID)
return request.ID, nil
}
// requiredStatusesOf is what the forge's protection of a repository's branch requires before a merge, as the forge
// picks the rule it applies (the one named for the branch, else the first glob covering it). A variable so a test needs
// no forge.
var requiredStatusesOf = func(ctx context.Context, owner, repo, branch string) ([]string, error) {
var rules struct {
Rules []protectionRule `json:"rules"`
}
js, err := aBus()
if err != nil {
return nil, err
}
defer js.Close()
raw, _ := json.Marshal(map[string]any{"owner": owner, "repo": repo})
answer, err := link.Ask(ctx, link.OverNATS{Conn: js.Conn()}, "gitea", "gitea_branch_protection_get", raw, 30*time.Second)
if err != nil {
return nil, err
}
if answer.Error != "" {
return nil, fmt.Errorf("gitea.gitea_branch_protection_get: %s", answer.Error)
}
if err := json.Unmarshal(answer.Result, &rules); err != nil {
return nil, err
}
r := appliedRule(rules.Rules, branch)
if r == nil {
// empty-on-error: not an error; no rule covers the branch, so it requires nothing, which the caller says
return nil, nil
}
return r.RequiredStatuses, nil
}
// composedWords is what a group member's own verdict says of its gate layer (novox/hq ADR 0403 clause 1).
const composedWords = "judged by its group's composed check"
// gateComposed is a gate layer's verdict when no gate was run for the member alone: never a pass. Mesh-delivery reads it
// as the composed check's to judge, and the forge's holder sets no mesh/merge-gate from it.
const gateComposed = "composed"
// composedGate says a group member's own verdict has no gate of its own: its gate layer, and the verdict that is
// the gate's, read "composed" in place of a pass.
func composedGate(c *link.Checked) {
c.Verdict, c.Summary = gateComposed, composedWords
c.Gate = &link.CheckLayer{Verdict: gateComposed, Summary: composedWords}
}
// directVerdict is the verdict of a change no build judges — it touches no module, and either the repository is not
// the mesh's or the head holds no merge-check.sh. **It names the trunk commit it holds for** (novox/hq ADR 0323 §3):
// the base branch's head the forge announced with the head, so the merge tool can weigh it against the trunk's head
// and does not refuse it for ever (a verdict naming no trunk is never the trunk's head's). Nothing was built to
// judge: the trunk commit is the one the forge said the change was announced against.
func directVerdict(p link.PullUpdated, plan link.ChangePlan) link.Checked {
trunk := p.BaseSHA
if len(trunk) > 12 {
trunk = trunk[:12]
}
if trunk != "" {
plan.Trunk = trunk
}
return link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan, Trunk: trunk}
}
// checkRequestFor is the ask for one pull request's head: the repository as the mesh clones it, the head,
// and what is read beside it.
func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scope checkScope,
entries []inventory.Entry) (link.BuildRequest, error) {
shelf := map[string]catalogue.Manifest{}
for _, e := range entries {
shelf[e.Manifest.Module] = e.Manifest
}
world, err := restOfTheMesh(ctx, open, shelf, "")
if err != nil {
return link.BuildRequest{}, err
}
clone := func(s inventory.Source) (string, error) {
if s.Seat == "" {
return s.Repository, nil
}
return clonedFromSeat(world, s.Seat, s.Repository)
}
// As the mesh clones a module built from it; a repository no module is built from, from the forge.
source := inventory.Source{Seat: gitSeat, Repository: p.Owner + "/" + p.Repo}
if scope.From != nil {
source = scope.From.Source
}
repository, err := clone(source)
if err != nil {
return link.BuildRequest{}, err
}
current, err := open.inventory.CurrentBuilds(ctx)
if err != nil {
return link.BuildRequest{}, err
}
beside, err := besideOf(entries, func(module string) string { return current[module].Commit }, clone)
if err != nil {
return link.BuildRequest{}, err
}
builds := checkBuildsOf(p, scope, entries, clone)
return link.BuildRequest{
ID: link.NewBuildID(time.Now()),
Repository: repository,
Ref: p.Commit,
Held: heldBy(ctx),
Seats: seatBases(ctx),
Source: sourceOnSeat(source),
Check: checkOf(p, scope, entries, beside, builds, namedForTheCheck(ctx, p, builds), runningBy(ctx)),
}, nil
}
// checkOf is the check a request carries: the pull request, what the change touches, and the buckets of the pull
// requests' keys of the modules claiming the delivery seat (novox/hq ADR 0437 §3, issue 1212).
func checkOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry, beside map[string]link.CheckedOut,
builds []link.CheckBuild, named []link.CheckedBuild, running map[string]string) *link.CheckRequest {
return &link.CheckRequest{Pulls: pullsBucketsOf(entries), Owner: p.Owner, Repo: p.Repo, Number: p.Number, Base: p.Base,
Trunk: p.BaseSHA, Ready: p.Ready, Paths: p.Paths, Beside: beside, Modules: scope.Modules, Dependents: scope.Dependents,
New: scope.New, Manifests: scope.Manifests, Judge: scope.Judge, Builds: builds, Named: named, Running: running}
}
// besideOf is what a check reads beside its own repository, by the directory it is found under: each core repository
// by the module the mesh builds from it, at besideRefs. running is the commit the mesh runs of a module.
func besideOf(entries []inventory.Entry, running func(module string) string,
clone func(inventory.Source) (string, error)) (map[string]link.CheckedOut, error) {
beside := map[string]link.CheckedOut{}
for _, e := range entries {
dir, core := coreModules[e.Manifest.Module]
if !core || e.Provided || e.Source.Repository == "" {
continue
}
url, err := clone(e.Source)
if err != nil {
return nil, err
}
refs := besideRefs(dir, running(e.Manifest.Module))
beside[dir] = link.CheckedOut{Repository: url, Ref: refs[dir]}
if dir == "mesh-controller" {
// The controller's main stays its main: it judges only a check the running controller predates.
beside["mesh-controller-main"] = link.CheckedOut{Repository: url, Ref: refs["mesh-controller-main"]}
if e.Source.Seat != "" {
for _, sibling := range []string{"mesh-lab", "mesh-sdk"} {
if url, err := clone(inventory.Source{Seat: e.Source.Seat, Repository: siblingOf(e.Source.Repository,
sibling)}); err == nil {
beside[sibling] = link.CheckedOut{Repository: url, Ref: refs[sibling]}
}
}
}
}
}
return beside, nil
}
// trunkOf is the branch a module's source follows: main when it records none, or records a commit.
func trunkOf(s inventory.Source) string {
if t := strings.TrimSpace(followedBranch(s.Ref)); t != "" {
return t
}
return "main"
}
// namedForTheCheck are the builds earlier checks of the same pull request at the same head named, read once for the
// check (novox/hq issue 703): asked again because only the trunk moved, the check keeps what they built of a key
// still the same. Another pull request's build, or one of an earlier head, is never kept. None when they cannot be
// read, and the check builds, as before.
func namedForTheCheck(ctx context.Context, p link.PullUpdated, builds []link.CheckBuild) []link.CheckedBuild {
if len(builds) == 0 || p.Commit == "" {
return nil
}
pull := link.PullRef{Owner: p.Owner, Repo: p.Repo, Number: p.Number}
read, err := checkedBuildsOf(ctx, []link.PullRef{pull})
if err != nil {
fmt.Printf(" the earlier checks' builds of %s/%s#%d could not be read, so the check builds: %v\n",
p.Owner, p.Repo, p.Number, err)
// empty-on-error: said above; a check handed no named build builds every module, as before issue 703
return nil
}
var named []link.CheckedBuild
for _, b := range read {
if b.Pull == pull && link.SameHead(b.Head, p.Commit) {
named = append(named, b)
}
}
return named
}
// checkBuildsOf are the modules a merge of the change would build, in the walk's order, as the check builds them
// for the walk to take (novox/hq ADR 0350 §1): a module built from the repository checked, at the branch it merges
// into, from the merge the check makes; any other — a module standing on one — from its own repository at the
// branch it follows, as the walk builds it. A module whose build reads the repository checked through a context,
// or that the merge deletes, cannot be built as the walk would: said, and nothing after it is built. A change that
// moves no module builds nothing.
func checkBuildsOf(p link.PullUpdated, scope checkScope, entries []inventory.Entry,
clone func(inventory.Source) (string, error)) []link.CheckBuild {
if len(scope.Reach.Plan.Tiers) == 0 {
return nil
}
m := link.SourceMoved{Owner: p.Owner, Repo: p.Repo, Base: p.Base, CloneURL: p.CloneURL}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
skip := map[string]string{}
for _, e := range scope.Reach.Packaging {
skip[e.Manifest.Module] = "its build reads " + p.Owner + "/" + p.Repo + " through a context, which a check " +
"cannot hand the merge"
}
for _, e := range scope.Reach.Deleted {
skip[e.Manifest.Module] = "the merge deletes it"
}
var out []link.CheckBuild
for tier, names := range scope.Reach.Plan.Tiers {
for _, name := range names {
e, known := byName[name]
b := link.CheckBuild{Module: name, Tier: tier, Skip: skip[name]}
switch {
case !known:
b.Skip = "the catalogue does not hold it"
case b.Skip != "":
case sourceIs(e.Source, m):
b.Merged, b.Path = true, e.Source.Path
default:
url, err := clone(e.Source)
if err != nil {
b.Skip = "its repository cannot be cloned: " + err.Error()
break
}
b.Repository, b.Path, b.Ref = url, e.Source.Path, followedBranch(e.Source.Ref)
}
out = append(out, b)
}
}
return out
} }
// besideRefs is the ref each repository is cloned at beside a check, by the directory it is found under: // besideRefs is the ref each repository is cloned at beside a check, by the directory it is found under:
@@ -533,201 +187,5 @@ func sdkPinned() string {
return "main" return "main"
} }
// siblingOf is another repository of the same owner: novox/mesh-controller → novox/mesh-lab.
func siblingOf(repository, name string) string {
if cut := strings.LastIndex(repository, "/"); cut >= 0 {
return repository[:cut+1] + name
}
return name
}
// sourceOnSeat is a source's seat form, nil for one on no seat.
func sourceOnSeat(s inventory.Source) *link.SourceOnSeat {
if s.Seat == "" {
return nil
}
return &link.SourceOnSeat{Seat: s.Seat, Repository: s.Repository}
}
// checkEvents is where the serving controller says a check's verdict; nil in a command. // checkEvents is where the serving controller says a check's verdict; nil in a command.
var checkEvents link.Bus var checkEvents link.Bus
// maxCheckReport is how much of a check's report travels in its verdict: enough for the failures and
// the machines, never a log.
const maxCheckReport = 60 << 10
// checked says a merge check's verdict as the controller's `checked`. Nothing is recorded or
// registered: a check builds nothing (issue 240's rule for a dry run, kept for a check).
func checked(ctx context.Context, result link.BuildResult) {
// **A check stopped by its key says nothing** (novox/hq ADR 0406 §4, issue 974, review 2): it stopped because the head or
// the trunk commit moved on, and the newer request will judge. Said as an error it would name the live head — red
// statuses on the forge, a delivery refused — and written to the key it would stand in the newer request's way.
if result.Failed == link.SupersededByKey {
fmt.Printf("%s: stopped, its pull request's key moved on; the newer check will judge\n", result.ID)
return
}
sayChecked(ctx, os.Stdout, checkedOf(result))
}
// checkedOf is what a check's outcome says: each layer, and an error — never a pass — for a check that
// could not run.
func checkedOf(result link.BuildResult) link.Checked {
c := link.Checked{ID: result.ID, On: result.On, Commit: result.Ref}
if result.Checked != nil {
c.Owner, c.Repo, c.Number = result.Checked.Owner, result.Checked.Repo, result.Checked.Number
}
switch {
case result.Check != nil:
c.Verdict, c.Summary, c.Report = result.Check.Verdict, result.Check.Summary, result.Check.Report
c.Gate, c.RepoCheck = result.Check.Gate, result.Check.RepoCheck
// What was judged (novox/hq ADR 0323 §3): the head merged into the trunk commit, and the commits of the
// repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
c.Built = result.Check.Built
c.Validator = result.Check.Validator
c.Siblings, c.SiblingsUnknown, c.JudgeFrom = result.Check.Siblings, result.Check.SiblingsUnknown, result.Check.JudgeFrom
if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
case result.Failed != "":
// The check could not run: an error, never read as a pass — on both layers it was asked for.
c.Verdict, c.Summary = "error", "the check could not run: "+firstLine(result.Failed)
default:
c.Verdict, c.Summary = "error", "the build seat answered the check with no verdict"
}
if c.Verdict == "" {
c.Verdict = "error"
}
if result.Check == nil {
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
c.RepoCheck = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
}
// **A group member's own check has no gate of its own** (novox/hq ADR 0403 clause 1): what ran was the repository's
// check alone, and its gate layer is "composed", never a pass. A failure or an error stays what it was.
if result.Checked != nil && result.Checked.Composed && result.Check != nil && c.Gate != nil &&
(c.Gate.Verdict == "pass" || c.Gate.Verdict == "warning") {
composedGate(&c)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Modules) == 0 {
c.Gate.Modules = append(append([]string{}, result.Checked.Modules...), prefixedAll("new:", result.Checked.New)...)
}
if result.Checked != nil && c.Gate != nil && len(c.Gate.Dependents) == 0 {
c.Gate.Dependents = result.Checked.Dependents
}
if result.Checked != nil {
c.Plan = result.Checked.Plan
if c.Plan != nil && c.Trunk != "" {
// The kept delivery plan records the trunk commit with its other inputs (ADR 0323 §3).
plan := *c.Plan
plan.Trunk = c.Trunk
c.Plan = &plan
}
// A delivery group's composed check (novox/hq ADR 0239): every head it judged, this one first.
if g := result.Checked; g.Group != "" {
c.Group = g.Group
// Every member is named with its trunk commit (novox/hq issue 1106): the seat's word, else the one the forge
// announced with the head, so the forge's merge tool never meets a member whose verdict names none.
c.Members = append(c.Members, link.CheckedMember{Owner: g.Owner, Repo: g.Repo, Number: g.Number,
Commit: result.Ref, Trunk: firstTrunk(c.Trunk, g.Trunk)})
for _, m := range g.Members {
c.Members = append(c.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, Number: m.Number,
Commit: m.Ref, Trunk: firstTrunk(memberTrunk(result.Check, m), m.Trunk)})
}
}
}
for _, l := range []*link.CheckLayer{c.Gate, c.RepoCheck} {
if l != nil && l.Verdict == "" {
l.Verdict = "error"
}
}
// **A check run again says so** (novox/hq issue 594). The summaries are the lines the pull request
// shows, clipped by the forge to a few words, so they get only a short marker after the verdict's own
// words; the report gets the whole sentence. Layers copied, so the build's own outcome is left as it came.
if mark := link.RunMarker(result.Delivery); mark != "" {
c.Summary += " " + mark
for _, l := range []**link.CheckLayer{&c.Gate, &c.RepoCheck} {
if *l != nil {
layer := **l
layer.Summary += " " + mark
*l = &layer
}
}
c.Report = link.Restarted(result.Delivery) + "\n\n" + c.Report
}
if len(c.Report) > maxCheckReport {
c.Report = "…" + c.Report[len(c.Report)-maxCheckReport:]
}
return c
}
// firstTrunk is the first trunk commit named, as twelve characters; empty when none is.
func firstTrunk(named ...string) string {
for _, n := range named {
if n != "" {
return twelve(n)
}
}
return ""
}
// memberTrunk is the trunk commit the build seat merged one of a group's heads into (novox/hq ADR 0323 §6), empty
// when it said none.
func memberTrunk(outcome *link.CheckOutcome, m link.GroupMember) string {
if outcome == nil {
return ""
}
for _, j := range outcome.Members {
if strings.EqualFold(j.Owner, m.Owner) && strings.EqualFold(j.Repo, m.Repo) && strings.HasPrefix(m.Ref, j.Commit) {
return j.Trunk
}
}
return ""
}
func prefixedAll(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
out = append(out, prefix+i)
}
return out
}
// sayChecked says a merge check's verdict on the bus, where the forge's holder hears it.
func sayChecked(ctx context.Context, w io.Writer, c link.Checked) {
// **No merge-gate layer from R4e, for any repository** (ADR 0503 §3, §5; issue 1299): the forge's holder writes
// `mesh/repo-check` from the repo-check layer alone, and the checker owes `mesh/check`.
c.Gate = nil
repo := "none"
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
}
judged := ""
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Fprintf(w, "%s: %s/%s#%d at %.8s%s checked on %s: repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), repo)
if checkEvents == nil {
return
}
body, err := json.Marshal(c)
if err != nil {
return
}
stating, stop := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer stop()
if err := checkEvents.PublishSeatEvent(stating, link.MeshControllerSeat, link.KeyChecked, body); err != nil {
fmt.Fprintf(w, "%s: the verdict could not be said, so the pull request is not told it: %v\n", c.ID, err)
}
}
// pullsBucketsOf are the buckets of the pull requests' keys a check names for the build agent: one per module
// claiming the delivery seat, in the order of their names (novox/hq ADR 0437 §3, issue 1212).
func pullsBucketsOf(entries []inventory.Entry) []string {
var out []string
for _, m := range claimantsOf(entries, catalogue.DeliverySeat) {
out = append(out, broker.BucketName(m, "pulls"))
}
return out
}
@@ -1,56 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
func TestACheckStoppedByItsKeyIsAnnouncedAsNoVerdict(t *testing.T) {
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
legacy, err := js.Conn().JetStream()
if err != nil {
t.Fatal(err)
}
was := checkEvents
checkEvents = link.OverNATS{Conn: js.Conn(), JS: legacy}
t.Cleanup(func() { checkEvents = was })
heard := make(chan *nats.Msg, 4)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeyChecked), heard)
if err != nil {
t.Fatal(err)
}
defer func() { _ = sub.Unsubscribe() }()
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3}
checked(t.Context(), link.BuildResult{ID: "stopped", Ref: "abc", Checked: asked, Failed: link.SupersededByKey})
checked(t.Context(), link.BuildResult{ID: "broken", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
select {
case m := <-heard:
var c link.Checked
_ = json.Unmarshal(m.Data, &c)
if c.ID != "broken" || c.Verdict != "error" {
t.Fatalf("the first verdict said is %+v; want only the broken check's error", c)
}
case <-time.After(5 * time.Second):
t.Fatal("nothing was said")
}
select {
case m := <-heard:
t.Fatalf("a second verdict was said: %s", m.Data)
case <-time.After(300 * time.Millisecond):
}
}
@@ -1,109 +0,0 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// C1 (ADR 0503 §3, §5; issue 1299, lead decision L11), with the store and a real build queue: a pull request of a
// repository whose trunk requires mesh/check asks build-agent nothing and says nothing; one whose trunk requires
// mesh/repo-check alone (hq) has its own merge-check.sh asked, and every verdict said names no gate layer; a protection
// that cannot be read asks and says nothing.
func TestThePullRequestChecksAreAskedOnlyForARepositoryTheCheckerDoesNotJudge(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
js, _ := aBuildQueue(t)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
said := &saidOnTheBus{}
checkEvents = said
t.Cleanup(func() { checkEvents = nil })
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
// hq is cloned as a module built from it is, from its URL: no forge is held in this test.
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "hq-docs", Version: "1"},
inventory.Source{Repository: "http://forge.invalid/novox/hq.git", Ref: "main", Path: "docs", BuiltFrom: "c0",
Head: "c0"}); err != nil {
t.Fatal(err)
}
required := map[string][]string{"novox/mesh-controller": {"mesh/check"}, "novox/hq": {"mesh/repo-check"}}
was := requiredStatusesOf
requiredStatusesOf = func(_ context.Context, owner, repo, _ string) ([]string, error) {
if r, ok := required[owner+"/"+repo]; ok {
return r, nil
}
return nil, errors.New("the forge did not answer")
}
t.Cleanup(func() { requiredStatusesOf = was })
queued := func() int {
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
return len(q.Asks)
}
pull := func(repo string, number int, mergeCheck bool) link.PullUpdated {
p := link.PullUpdated{Owner: "novox", Repo: repo, Number: number, Base: "main", Commit: "abcdefabcdefabcdefabcdefabcdefabcdefabcd",
BaseSHA: "1111111111111111111111111111111111111111", Paths: []string{"main.go"}, ModuleDirsSaid: true}
p.MergeCheck, p.MergeCheckSaid = mergeCheck, true
return p
}
f := following{open: open, oneShot: true}
if err := f.PullUpdated(ctx, pull("mesh-controller", 7, true)); err != nil {
t.Fatal(err)
}
if n := queued(); n != 0 || len(said.events) != 0 {
t.Fatalf("0503 §3: a repository the checker judges asked %d check(s) and said %v", n, said.events)
}
if err := f.PullUpdated(ctx, pull("unread", 3, true)); err != nil {
t.Fatal(err)
}
if n := queued(); n != 0 || len(said.events) != 0 {
t.Fatalf("L11: a protection that cannot be read asked %d check(s) and said %v", n, said.events)
}
if err := f.PullUpdated(ctx, pull("hq", 9, true)); err != nil {
t.Fatal(err)
}
q, err := link.ReadQueue(ctx, js, link.TheBuildMachine)
if err != nil {
t.Fatal(err)
}
if len(q.Asks) != 1 || q.Asks[0].Request.Check == nil || len(q.Asks[0].Request.Check.Modules) != 0 ||
q.Asks[0].Request.Check.Repo != "hq" {
t.Fatalf("0503 §5: hq's repository check is not the one ask: %+v", q.Asks)
}
// A head with no merge-check.sh is said at once, with the repo-check layer alone.
if err := f.PullUpdated(ctx, pull("hq", 10, false)); err != nil {
t.Fatal(err)
}
if len(said.events) != 1 {
t.Fatalf("hq's head with no merge-check.sh was said %d time(s)", len(said.events))
}
var c link.Checked
if err := json.Unmarshal(said.bodies[0], &c); err != nil {
t.Fatal(err)
}
if c.Gate != nil || c.RepoCheck == nil || c.RepoCheck.Verdict != "warning" {
t.Fatalf("0503 §5: the verdict said names a gate layer, or no repo-check layer: %+v", c)
}
// A trunk requiring both (hq, ADR 0513 §4): the checker judges mesh/check, and the repository check is still asked.
required["novox/hq"] = []string{"mesh/check", "mesh/repo-check"}
if err := f.PullUpdated(ctx, pull("hq", 11, true)); err != nil {
t.Fatal(err)
}
if q, err := link.ReadQueue(ctx, js, link.TheBuildMachine); err != nil || len(q.Asks) != 2 ||
q.Asks[1].Request.Check == nil || q.Asks[1].Request.Check.Repo != "hq" {
t.Fatalf("0513 §4: a trunk requiring both had no repository check asked: %+v %v", q.Asks, err)
}
}
+2 -186
View File
@@ -1,7 +1,6 @@
package main package main
import ( import (
"errors"
"strings" "strings"
"testing" "testing"
@@ -76,8 +75,8 @@ func TestThePullRequestIsMappedOntoTheModuleGraph(t *testing.T) {
t.Errorf("%s: %s %q, wanted %q", c.what, what, got[i], want[i]) t.Errorf("%s: %s %q, wanted %q", c.what, what, got[i], want[i])
} }
} }
if s.Mesh != c.mesh || s.Judge != c.judge { if s.Mesh != c.mesh {
t.Errorf("%s: the mesh's %v judged by %q, wanted %v by %q", c.what, s.Mesh, s.Judge, c.mesh, c.judge) t.Errorf("%s: the mesh's %v, wanted %v", c.what, s.Mesh, c.mesh)
} }
if s.gated() != (c.modules != "" || c.new != "") { if s.gated() != (c.modules != "" || c.new != "") {
t.Errorf("%s: gated %v", c.what, s.gated()) t.Errorf("%s: gated %v", c.what, s.gated())
@@ -95,28 +94,6 @@ func TestAPullRequestTouchesWhatPackagesItsRepository(t *testing.T) {
} }
} }
// Each layer is said; a check that could not run is an error on both, never a pass; a build seat from
// before the layers is read as the gate.
func TestAChecksLayersAreEachSaidAndAnErrorIsNeverAPass(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 3, Modules: []string{"gitea"}}
c := checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Failed: "the facts snapshot cannot be read"})
if c.Verdict != "error" || c.Gate == nil || c.Gate.Verdict != "error" || c.RepoCheck == nil || c.RepoCheck.Verdict != "error" {
t.Fatalf("a check that could not run said %+v", c)
}
if strings.Join(c.Gate.Modules, ",") != "gitea" {
t.Errorf("the gate names %v", c.Gate.Modules)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "warning", Summary: "wide"}})
if c.Gate == nil || c.Gate.Verdict != "warning" || c.RepoCheck != nil {
t.Fatalf("an outcome without layers said %+v", c)
}
c = checkedOf(link.BuildResult{ID: "b", Ref: "abc", Checked: asked, Check: &link.CheckOutcome{Verdict: "pass",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "its merge-check.sh failed"}}})
if c.RepoCheck.Verdict != "fail" || c.Gate.Verdict != "pass" {
t.Fatalf("the layers said %+v / %+v", c.Gate, c.RepoCheck)
}
}
// **The check asks the planner, never a mapping of its own** (novox/hq ADR 0238): what a pull request // **The check asks the planner, never a mapping of its own** (novox/hq ADR 0238): what a pull request
// reaches is reachOfMerge's answer — the same that plans a merge — so a file at the root touches no // reaches is reachOfMerge's answer — the same that plans a merge — so a file at the root touches no
// module in both, and what stands on a touched module is named as its dependents in both. // module in both, and what stands on a touched module is named as its dependents in both.
@@ -171,166 +148,5 @@ func TestAPullRequestReachesWhatAMergeOfItWouldPlan(t *testing.T) {
} }
} }
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a build of a commit off its repository's
// default branch — a pull request's head, a branch built by hand, a rebuild or replay of one — is recorded
// and never registered, so nothing can send it; a check or a dry run never is either.
func TestABuildOffTheTrunkIsNeverPublished(t *testing.T) {
on := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main", OnTrunk: true}
if err := publishable(on, ""); err != nil {
t.Errorf("a build on the trunk was refused: %v", err)
}
off := link.BuildResult{ID: "b", Commit: "abc", Trunk: "main"}
if err := publishable(off, ""); !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "main") {
t.Errorf("a build off the trunk was let through: %v", err)
}
for _, r := range []link.BuildResult{
{ID: "c", Trunk: "main", OnTrunk: true, Check: &link.CheckOutcome{Verdict: "pass"}},
{ID: "d", Trunk: "main", OnTrunk: true, Checked: &link.CheckRequest{}},
{ID: "e", Trunk: "main", OnTrunk: true, DryRun: true},
} {
if publishable(r, "") == nil {
t.Errorf("%s, a check or a dry run, was publishable", r.ID)
}
}
// A module that follows a branch other than the default: that branch is its trunk, and the default
// is not.
follows := link.BuildResult{ID: "g", Commit: "abc", Trunk: "master", Branches: []string{"nox-mesh"}}
if err := publishable(follows, "nox-mesh"); err != nil {
t.Errorf("a commit on the branch a module follows was refused: %v", err)
}
if err := publishable(link.BuildResult{ID: "h", Commit: "abc", Trunk: "master", OnTrunk: true,
Branches: []string{"master"}}, "nox-mesh"); err == nil {
t.Error("a commit on the default but not on the branch the module follows was published")
}
if err := publishable(link.BuildResult{ID: "i", Commit: "abc", Trunk: "main", Branches: []string{"feat/x"}}, ""); err == nil {
t.Error("a feature branch's commit was published")
}
// A build seat older than the rule says nothing: let through and said, so the rule can reach the mesh.
if err := publishable(link.BuildResult{ID: "f", Commit: "abc"}, ""); err != nil {
t.Errorf("a build seat that said nothing was refused: %v", err)
}
}
// **A verdict names the trunk commit it judged** (novox/hq ADR 0323 §3, issue 341): the controller's `checked`
// carries the head and the trunk commit the build seat merged it into, the commits of the repositories read beside
// it, how long it took, and the kept delivery plan records the trunk commit with its other inputs. A group's
// composed check names, per member, the trunk commit that head was merged into (§6).
func TestAVerdictNamesTheTrunkCommitItJudged(t *testing.T) {
plan := &link.ChangePlan{Repository: "novox/mesh-host", Base: "main", Head: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
Summary: "builds mesh-host"}
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 66, Base: "main", Plan: plan}
c := checkedOf(link.BuildResult{ID: "b", Ref: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Checked: asked,
Check: &link.CheckOutcome{Verdict: "pass", Took: "4m12s", Head: "dfb1415aaaaa", Trunk: "d28ca95bbbbb",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "pass"},
Beside: map[string]string{"mesh-catalog": "0123456789ab", "mesh-lab": "ba9876543210"}}})
if c.Trunk != "d28ca95bbbbb" || c.Commit != "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Fatalf("the verdict names head %s and trunk %q", c.Commit, c.Trunk)
}
if c.Beside["mesh-catalog"] != "0123456789ab" || c.Beside["mesh-lab"] != "ba9876543210" {
t.Fatalf("the verdict names the repositories read beside it as %v", c.Beside)
}
if c.Took != "4m12s" {
t.Fatalf("the verdict says it took %q", c.Took)
}
if c.Plan == nil || c.Plan.Trunk != "d28ca95bbbbb" || c.Plan.Head != plan.Head {
t.Fatalf("the kept plan records %+v", c.Plan)
}
if plan.Trunk != "" {
t.Fatal("the plan the controller asked with was changed: the verdict's is a copy")
}
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-controller", Number: 7, Base: "main", Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-host", Number: 8, Ref: "cccccccccccc0000", Base: "main"}}}
c = checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "cccccccccccc", Trunk: "222222222222"}}}})
if len(c.Members) != 2 || c.Members[0].Trunk != "111111111111" || c.Members[1].Trunk != "222222222222" ||
c.Members[1].Commit != "cccccccccccc0000" {
t.Fatalf("the composed check names its members %+v", c.Members)
}
}
// **A group's composed verdict names every member's trunk commit** (novox/hq issue 1106): a member the build seat named no
// trunk for (an SDK member whose base the seat did not merge, or a seat from before the rule) is named by the trunk the forge
// announced with its head, and the primary the same, so the merge tool never reads "names no trunk commit" on a member
// whose trunk did not move.
func TestAGroupVerdictNamesEveryMembersTrunkTheSDKMemberToo(t *testing.T) {
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog", Number: 7, Base: "main", Group: "feat/x",
Trunk: "dddddddddddd0000",
Members: []link.GroupMember{
{Owner: "novox", Repo: "mesh-sdk", Number: 8, Ref: "cccccccccccc0000", Base: "main", Trunk: "eeeeeeeeeeee0000"},
{Owner: "novox", Repo: "mesh-host", Number: 9, Ref: "bbbbbbbbbbbb0000", Base: "main", Trunk: "ffffffffffff0000"}}}
c := checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "bbbbbbbbbbbb", Trunk: "222222222222"}}}})
want := []string{"dddddddddddd", "eeeeeeeeeeee", "222222222222"}
if len(c.Members) != 3 {
t.Fatalf("the composed check names its members %+v", c.Members)
}
for i, m := range c.Members {
if m.Trunk != want[i] {
t.Fatalf("member %d (%s/%s) is named with trunk %q, want %q (the seat's word first, else the announced one): %+v",
i, m.Owner, m.Repo, m.Trunk, want[i], c.Members)
}
}
}
// **The merge made for a check is never registered** (novox/hq ADR 0323 §2): a build of it — a commit on no branch
// of the forge — is refused, naming the trunk. The rule of ADR 0238 §1 stands unchanged; this holds it for the
// merge commit a check makes.
func TestABuildOfACheckMergeIsRefusedNamingTheTrunk(t *testing.T) {
merge := link.BuildResult{ID: "b", Commit: "4b5c6d7e8f90aabbccddeeff0011223344556677", Trunk: "main"}
err := publishable(merge, "")
if !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "not on main") {
t.Fatalf("a build of a check's merge was answered %v", err)
}
}
// **A verdict no build judges names its trunk commit too** (novox/hq ADR 0323 §3): a change touching no module of a
// repository with no merge-check.sh (novox/mesh-tools-go) is answered by the controller itself. Naming no trunk, the
// merge tool refused it, asked again, got the same verdict, and refused for ever.
func TestADirectVerdictNamesTheTrunkTheForgeAnnounced(t *testing.T) {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-tools-go", Number: 3, Base: "main", Commit: "1a2b3c4d5e6f7a8b9c0d",
BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
c := directVerdict(p, link.ChangePlan{Repository: "novox/mesh-tools-go", Base: "main", Head: p.Commit})
if c.Trunk != "7a8b9c0d1e2f" || c.Plan == nil || c.Plan.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("the direct verdict names trunk %q, its plan %+v", c.Trunk, c.Plan)
}
if c.Verdict != "pass" || c.Commit != p.Commit {
t.Fatalf("the direct verdict is %+v", c)
}
if c := directVerdict(link.PullUpdated{Owner: "novox", Repo: "x", Commit: "abc"}, link.ChangePlan{}); c.Trunk != "" {
t.Fatalf("a forge that said no base head gave the trunk %q", c.Trunk)
}
}
// A head checked again through mesh-delivery (`delivery check --members`, a recheck) reaches the same direct verdict:
// it names the base head the forge last announced, kept on the member, or the merge tool would refuse it for ever.
func TestARecheckedHeadsDirectVerdictNamesItsTrunk(t *testing.T) {
m := orderMember{ID: "novox/mesh-tools-go@1a2b3c4d5e6f", Repository: "novox/mesh-tools-go", Base: "main",
Head: "1a2b3c4d5e6f7a8b9c0d", Number: 3, BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
if c := directVerdict(m.pull(), link.ChangePlan{}); c.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("a rechecked head's direct verdict names trunk %q", c.Trunk)
}
}
// A check stopped by its pull request's key moving on says no verdict (issue 974, review 2): an error said for it would name the // A check stopped by its pull request's key moving on says no verdict (issue 974, review 2): an error said for it would name the
// live head, and the newer request is the one that judges. A check that failed for any other reason still says its error. // live head, and the newer request is the one that judges. A check that failed for any other reason still says its error.
// C2 (ADR 0513 §4, ADR 0503 §5; issue 1299): a trunk requiring mesh/repo-check has its repository check asked, whether or
// not it also requires mesh/check (hq requires both); a trunk requiring mesh/check alone, or neither, is asked nothing.
func TestARepositoryCheckIsAskedWheneverTheTrunkRequiresIt(t *testing.T) {
for _, c := range []struct {
required []string
asks bool
}{
{[]string{"mesh/check", "mesh/repo-check"}, true},
{[]string{"mesh/repo-check"}, true},
{[]string{"mesh/check"}, false},
{nil, false},
} {
if got := asksRepoCheck(c.required); got != c.asks {
t.Fatalf("a trunk requiring %v asks its repository check: %v, want %v", c.required, got, c.asks)
}
}
}
@@ -1,61 +0,0 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A delivery group's gate is the composed check's, and its members' own checks run their repository's check alone
// (novox/hq ADR 0403 clauses 1 and 7). Pure: no store, no bus.
// The heads of a group's members, by repository.
const (
headHost = "1111111111111111111111111111111111111111"
headController = "2222222222222222222222222222222222222222"
)
// A composed member's verdict never says its gate passed: the gate layer, and the verdict that is the gate's, read
// "composed"; a failure or an error stays what it was.
func TestAComposedMembersVerdictHasNoPassingGate(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Composed: true}
pass := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "no module touched",
Gate: &link.CheckLayer{Verdict: "pass", Summary: "no module touched"}, RepoCheck: &link.CheckLayer{Verdict: "pass", Summary: "ok"}}}
c := checkedOf(pass)
if c.Gate == nil || c.Gate.Verdict != "composed" || c.Verdict != "composed" || c.RepoCheck == nil || c.RepoCheck.Verdict != "pass" {
t.Fatalf("the member's verdict says %+v gate %+v", c, c.Gate)
}
asked.Composed = false
if c := checkedOf(pass); c.Gate.Verdict != "pass" || c.Verdict != "pass" {
t.Fatalf("a member of no composed group has its own gate: %+v", c.Gate)
}
asked.Composed = true
failing := pass
failing.Check = &link.CheckOutcome{Verdict: "fail", Summary: "conflict", Gate: &link.CheckLayer{Verdict: "fail", Summary: "conflict"},
RepoCheck: &link.CheckLayer{Verdict: "fail", Summary: "conflict"}}
if c := checkedOf(failing); c.Gate.Verdict != "fail" || c.Verdict != "fail" {
t.Fatalf("a failure was read as composed: %+v", c.Gate)
}
broken := link.BuildResult{ID: "b", Ref: headHost, Checked: asked, Failed: "the facts snapshot cannot be read"}
if c := checkedOf(broken); c.Verdict != "error" || c.Gate.Verdict != "error" {
t.Fatalf("an error was read as composed: %+v", c)
}
}
// The composed verdict on the bus carries every member, each with its trunk commit, for the forge's holder to set each
// member's mesh/merge-gate from it.
func TestTheComposedVerdictCarriesEveryMember(t *testing.T) {
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 114, Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-controller", Number: 346, Ref: headController}}}
c := checkedOf(link.BuildResult{ID: "g", Ref: headHost, Checked: asked, Check: &link.CheckOutcome{Verdict: "pass", Summary: "ok",
Trunk: "aaaaaaaaaaaa", Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-controller", Commit: headController[:12], Trunk: "bbbbbbbbbbbb"}}}})
body, _ := json.Marshal(c)
for _, want := range []string{`"group":"feat/x"`, `"members":[{"owner":"novox","repo":"mesh-host","number":114,"commit":"` + headHost + `","trunk":"aaaaaaaaaaaa"}`,
`{"owner":"novox","repo":"mesh-controller","number":346,"commit":"` + headController + `","trunk":"bbbbbbbbbbbb"}`} {
if !strings.Contains(string(body), want) {
t.Fatalf("the composed verdict lacks %s:\n%s", want, body)
}
}
}
+5 -58
View File
@@ -815,18 +815,21 @@ func deliveryPlanDocument(m orderMember, entries []inventory.Entry, read map[str
// delivery walks [-n 50] [--plan <id>] // delivery walks [-n 50] [--plan <id>]
func deliveryCommand(ctx context.Context, args []string) error { func deliveryCommand(ctx context.Context, args []string) error {
if len(args) == 0 { if len(args) == 0 {
return errors.New("delivery order|check|go|stop|walks") return errors.New("delivery order|go|stop|walks")
} }
sub, rest := args[0], args[1:] sub, rest := args[0], args[1:]
if sub == "plan" { if sub == "plan" {
// Removed (ADR 0450 §8; issue 1318): refused by name before any flag is read, naming its replacement. // Removed (ADR 0450 §8; issue 1318): refused by name before any flag is read, naming its replacement.
return removedByName("delivery-plan") return removedByName("delivery-plan")
} }
if sub == "check" {
// Removed (ADR 0503 §3, ADR 0527; issue 1318): the checker judges every head, so the controller asks nothing.
return removedByName("delivery-check")
}
set := flag.NewFlagSet("delivery "+sub, flag.ContinueOnError) set := flag.NewFlagSet("delivery "+sub, flag.ContinueOnError)
membersJSON := set.String("members", "", "a group's members, as JSON") membersJSON := set.String("members", "", "a group's members, as JSON")
membersFile := set.String("members-file", "", "a file of a group's members, as JSON: the same as --members, for a list "+ membersFile := set.String("members-file", "", "a file of a group's members, as JSON: the same as --members, for a list "+
"too long for an argument (novox/hq issue 1272)") "too long for an argument (novox/hq issue 1272)")
group := set.String("group", "", "a delivery group's id")
ready := set.Bool("ready", false, "the head's pull request is marked ready: its check is queued before every other check "+ ready := set.Bool("ready", false, "the head's pull request is marked ready: its check is queued before every other check "+
"(novox/hq ADR 0406 §3)") "(novox/hq ADR 0406 §3)")
by := set.String("by", catalogue.DeliverySeat, "who says it") by := set.String("by", catalogue.DeliverySeat, "who says it")
@@ -883,25 +886,6 @@ func deliveryCommand(ctx context.Context, args []string) error {
return fmt.Errorf("what the members say of their manifests cannot be read: %w", err) return fmt.Errorf("what the members say of their manifests cannot be read: %w", err)
} }
return answer(orderOf(members, reachOfMembers(members, entries, read, edges), edges, changes)) return answer(orderOf(members, reachOfMembers(members, entries, read, edges), edges, changes))
case "check":
if *group == "" && len(members) == 1 {
// One head, checked again as the forge's announcement would have it (a recheck): the controller's
// own path for a pull request, run because the delivery's owner asked.
return recheckOneHead(members[0].ID, answer, func() (string, error) {
var asked string
err := sayingOnTheBus(ctx, inv, func() (err error) {
asked, err = (following{open: open, oneShot: true}).pullUpdated(ctx, members[0].pull(), os.Stderr)
return err
})
return asked, err
})
}
if *group == "" || len(members) < 2 {
return errors.New("delivery check --group <id> --members <json> (two heads or more), or --members <one head>")
}
// **The group's composed check is the checker's** (novox/hq ADR 0503 §3; issue 1299, C2): nothing is asked of
// build-agent, and the answer says who judges it.
return answer(groupCheckAnswer(*group))
case "go": case "go":
if len(positionals) != 1 { if len(positionals) != 1 {
return errors.New("delivery go <plan> [--by <who>] [--why <text>]") return errors.New("delivery go <plan> [--by <who>] [--why <text>]")
@@ -1076,14 +1060,6 @@ func theGraph(ctx context.Context, inv *inventory.Inventory) ([]inventory.Entry,
return entries, read, edges, nil return entries, read, edges, nil
} }
// twelve is a commit as twelve characters.
func twelve(c string) string {
if len(c) > 12 {
return c[:12]
}
return c
}
// appendNew appends what is not there yet. // appendNew appends what is not there yet.
func appendNew(to []string, items ...string) []string { func appendNew(to []string, items ...string) []string {
for _, i := range items { for _, i := range items {
@@ -1235,32 +1211,3 @@ func overNATS(conn *nats.Conn) (link.Bus, error) {
} }
return link.OverNATS{Conn: conn, JS: js}, nil return link.OverNATS{Conn: conn, JS: js}, nil
} }
// recheckAnswer is what `delivery check` answers for one head: the id of the check asked of the build seat (empty when
// the controller said the verdict itself and asked nothing), which mesh-delivery reads, and the head rechecked.
func recheckAnswer(asked, member string) map[string]any {
return map[string]any{"asked": asked, "rechecked": member}
}
// recheckOneHead runs a one-head `delivery check` and answers it as data. Standard output is the answer's JSON alone,
// or the seat verb reads no answer from it (novox/hq issue 1311): while the check runs, whatever it prints to standard
// output (the controller's own path prints its prose there, as do the stores it reads) goes to standard error.
func recheckOneHead(member string, answer func(any) error, ask func() (string, error)) error {
real := os.Stdout
os.Stdout = os.Stderr
asked, err := func() (string, error) {
defer func() { os.Stdout = real }()
return ask()
}()
if err != nil {
return err
}
return answer(recheckAnswer(asked, member))
}
// groupCheckAnswer is what `delivery check` of a group answers from R4e: who judges it, and that nothing was asked
// (novox/hq ADR 0503 §3; issue 1299, C2).
func groupCheckAnswer(group string) map[string]any {
return map[string]any{"asked": "", "group": group, "judged-by": checkerModule,
"said": "the checker judges a delivery group's heads merged together (ADR 0503 §3): nothing was asked"}
}
@@ -1,85 +0,0 @@
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// `delivery check` with one head answers the id of the check it asked, as data (novox/hq issue 1311): the prose
// the check says goes to standard error, so standard output is the document alone and answerOf reads it.
func TestDeliveryCheckOfOneHeadAnswersTheRequestID(t *testing.T) {
argv := []string{"delivery", "check", "--members", "[]"}
doc, err := json.Marshal(recheckAnswer("check-42", "novox/x@abc"))
if err != nil {
t.Fatal(err)
}
// Seen failing first: the prose the command used to print on standard output before the document.
polluted := append([]byte("novox/x#7 (abc): asked the build seat as check-42\n"), doc...)
if got := answerOf(argv, polluted, "", true); got.Answer != nil {
t.Fatalf("prose before the document should leave no answer; got %v", got.Answer)
}
got := answerOf(argv, doc, "novox/x#7 (abc): asked the build seat as check-42\n", true)
m, ok := got.Answer.(map[string]any)
if !ok || m["asked"] != "check-42" || m["rechecked"] != "novox/x@abc" {
t.Fatalf("the answer is %v; want asked check-42 and rechecked novox/x@abc", got.Answer)
}
if !strings.Contains(got.Output, "asked the build seat") {
t.Fatalf("the prose is still said in the text: %q", got.Output)
}
}
// The text of the other callers does not change: sayChecked writes the same line to the writer it is given, and
// the controller's own path still gives PullUpdated standard output.
func TestSayCheckedWritesTheSameLineToItsWriter(t *testing.T) {
var w bytes.Buffer
sayChecked(context.Background(), &w, link.Checked{ID: "c1", Owner: "novox", Repo: "x", Number: 7, Commit: "0123456789abcdef",
Verdict: "pass", Summary: "fine"})
want := "c1: novox/x#7 at 01234567 checked on a machine that did not say which: repository none\n"
if w.String() != want {
t.Fatalf("sayChecked said %q; want %q", w.String(), want)
}
}
// Whatever the check prints to standard output while it runs (checks.go namedForTheCheck, the inventory's pointer
// repair) must not reach the answer's standard output: it is exactly one JSON document (novox/hq issue 1311).
func TestOneHeadRecheckLeavesStandardOutputTheAnswerAlone(t *testing.T) {
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
e, ew, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
realOut, realErr := os.Stdout, os.Stderr
os.Stdout, os.Stderr = w, ew
runErr := recheckOneHead("novox/x@abc", func(v any) error { return json.NewEncoder(os.Stdout).Encode(v) },
func() (string, error) {
fmt.Println(" the earlier checks' builds could not be read, so the check builds")
fmt.Fprintln(os.Stdout, "repaired a pointer")
return "check-42", nil
})
os.Stdout, os.Stderr = realOut, realErr
w.Close()
ew.Close()
if runErr != nil {
t.Fatal(runErr)
}
var out, errOut bytes.Buffer
_, _ = out.ReadFrom(r)
_, _ = errOut.ReadFrom(e)
var doc map[string]any
if err := json.Unmarshal(out.Bytes(), &doc); err != nil || doc["asked"] != "check-42" || doc["rechecked"] != "novox/x@abc" {
t.Fatalf("standard output %q is not the answer alone (%v)", out.String(), err)
}
if !strings.Contains(errOut.String(), "repaired a pointer") {
t.Fatalf("the prose should be on standard error: %q", errOut.String())
}
}
+1 -1
View File
@@ -41,7 +41,7 @@ type stalledLine struct {
// waiting for the owner's word and what holds it (novox/hq issues 362, 529). // waiting for the owner's word and what holds it (novox/hq issues 362, 529).
Walk string `json:"walk,omitempty"` Walk string `json:"walk,omitempty"`
// Waiting are the merge checks a line of the state `unanswered` waits on, as mesh-delivery says each: // Waiting are the merge checks a line of the state `unanswered` waits on, as mesh-delivery says each:
// "mesh/merge-gate pending since <UTC time>", "mesh/repo-check never set" (novox/hq issue 438). // "mesh/check pending since <UTC time>", "mesh/check never set" (novox/hq issue 438).
Waiting []string `json:"waiting,omitempty"` Waiting []string `json:"waiting,omitempty"`
// Checks are the same merge checks as data, which the controller words in the operator's own time (novox/hq // Checks are the same merge checks as data, which the controller words in the operator's own time (novox/hq
// issue 443): a time inside a finished sentence cannot be said again in another zone. A mesh-delivery from before // issue 443): a time inside a finished sentence cannot be said again in another zone. A mesh-delivery from before
-2
View File
@@ -136,8 +136,6 @@ func TestTheDeliveryVerbsComposeTheirCommands(t *testing.T) {
{"delivery-walks", map[string]any{}, []string{"delivery", "walks"}}, {"delivery-walks", map[string]any{}, []string{"delivery", "walks"}},
{"delivery-walks", map[string]any{"plan": "plan-1"}, []string{"delivery", "walks", "--plan", "plan-1"}}, {"delivery-walks", map[string]any{"plan": "plan-1"}, []string{"delivery", "walks", "--plan", "plan-1"}},
{"delivery-order", map[string]any{"members": "[]"}, []string{"delivery", "order", "--members", "[]"}}, {"delivery-order", map[string]any{"members": "[]"}, []string{"delivery", "order", "--members", "[]"}},
{"delivery-check", map[string]any{"group": "feat/x", "members": "[]"},
[]string{"delivery", "check", "--group", "feat/x", "--members", "[]"}},
{"plans", map[string]any{"go": "plan-1", "why": "down"}, []string{"plans", "go", "plan-1", "--why", "down"}}, {"plans", map[string]any{"go": "plan-1", "why": "down"}, []string{"plans", "go", "plan-1", "--why", "down"}},
} { } {
got, err := argvFor(c.verb, c.args) got, err := argvFor(c.verb, c.args)
+1 -1
View File
@@ -669,7 +669,7 @@ func localDoctor(ctx context.Context) (*doctor, func(), error) {
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
js, err := dialTheBus() js, err := aBus()
if err != nil { if err != nil {
open.Close() open.Close()
return nil, nil, err return nil, nil, err
@@ -1,20 +0,0 @@
//go:build integration
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
func TestADryRunsOutcomeIsTakenInByNothing(t *testing.T) {
err := builds{}.Built(t.Context(), link.BuildResult{
ID: "build-1", Repository: "ssh://forge/app.git", Ref: "unreviewed", Module: "app", DryRun: true,
Manifest: json.RawMessage(`{"module":"app","version":"1"}`),
})
if err != nil {
t.Fatalf("a dry run's outcome was not simply set aside: %v", err)
}
}
-29
View File
@@ -1,29 +0,0 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A dry run's outcome is looked at, never taken in (novox/hq issue 240). The daemon here holds no
// store at all, so anything that tried to record or register would fail rather than pass quietly.
// The mark survives the wire both ways: asked as a dry run, answered as one.
func TestTheDryRunMarkTravelsWithTheBuild(t *testing.T) {
raw, _ := json.Marshal(link.BuildRequest{ID: "build-1", Repository: "r", DryRun: true})
var asked link.BuildRequest
if err := json.Unmarshal(raw, &asked); err != nil || !asked.DryRun {
t.Fatalf("the request lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-1", DryRun: true})
var answered link.BuildResult
if err := json.Unmarshal(raw, &answered); err != nil || !answered.DryRun {
t.Fatalf("the outcome lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-2"})
if string(raw) != `{"id":"build-2","repository":"","on":""}` {
t.Fatalf("an ordinary outcome carries a dry-run mark: %s", raw)
}
}
-20
View File
@@ -12,7 +12,6 @@ import (
"time" "time"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
) )
// What the core's bounds are set from (novox/hq to-be 45 Phase 0). // What the core's bounds are set from (novox/hq to-be 45 Phase 0).
@@ -24,25 +23,6 @@ import (
// controller as it hears each — a send's first report, a machine's next word, a plan leaving a tier, a // controller as it hears each — a send's first report, a machine's next word, a plan leaving a tier, a
// build's outcome — and summarised here per machine, repository or module. // build's outcome — and summarised here per machine, repository or module.
// recordBuildDuration measures one build from its ask to its outcome heard.
func recordBuildDuration(ctx context.Context, inv *inventory.Inventory, result link.BuildResult, asked time.Time) {
if asked.IsZero() || result.ID == "" {
return
}
subject := result.Module
if subject == "" {
subject = result.Repository
}
detail := "built"
if result.Failed != "" {
detail = "failed: " + firstLine(result.Failed)
}
if err := inv.RecordDuration(ctx, inventory.Duration{Kind: inventory.DurationBuild, Subject: subject,
Node: result.On, Ref: result.ID, Started: asked, Took: time.Since(asked), Detail: detail}); err != nil {
fmt.Fprintf(os.Stderr, "%s: how long it took could not be recorded: %v\n", result.ID, err)
}
}
// durationSummary is one subject's measurements of one kind. // durationSummary is one subject's measurements of one kind.
type durationSummary struct { type durationSummary struct {
Kind string `json:"kind"` Kind string `json:"kind"`
-12
View File
@@ -19,7 +19,6 @@ import (
snapshot "git.novox.be/novox/mesh-sdk/go/facts" snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/artifacts" "github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/broker" "github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
@@ -241,17 +240,6 @@ func gatherFacts(ctx context.Context, open *stores, busVersion string) (snapshot
scrub.Site(o.Site) scrub.Site(o.Site)
} }
// What a merge check runs in and reads beside it, as the build seat would be asked for it.
if held, err := inv.Held(ctx); err == nil {
for language, reference := range builder.ToolchainsOf(held) {
if f.Versions.Toolchains == nil {
f.Versions.Toolchains = map[string]string{}
}
f.Versions.Toolchains[language] = catalogue.Recorded(reference)
}
} else {
return snapshot.Facts{}, err
}
for _, e := range entries { for _, e := range entries {
if dir, core := coreModules[e.Manifest.Module]; core && !e.Provided && e.Source.Repository != "" { if dir, core := coreModules[e.Manifest.Module]; core && !e.Provided && e.Source.Repository != "" {
for d, ref := range besideRefs(dir, current[e.Manifest.Module].Commit) { for d, ref := range besideRefs(dir, current[e.Manifest.Module].Commit) {
+2 -2
View File
@@ -64,7 +64,7 @@ func registerFrom(t *testing.T, forms *sourceForms) (int64, error) {
mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true, mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true,
request: inventory.BuildRequest{ID: "build-1", Repository: aRepoOnTheForge, For: "build"}}, aRepoOnTheForge} request: inventory.BuildRequest{ID: "build-1", Repository: aRepoOnTheForge, For: "build"}}, aRepoOnTheForge}
return mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-view", inventory.Source{Repository: aRepoOnTheForge}, return mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-view", inventory.Source{Repository: aRepoOnTheForge},
byBuild("build-1"), "", "main") registrationAsk{Build: "build-1"}, "", "main")
} }
// 1277/AC2: an address unread twice, then read, is waited out and the forge is asked. // 1277/AC2: an address unread twice, then read, is waited out and the forge is asked.
@@ -112,7 +112,7 @@ func TestAKnownAddressAndAnotherHostIsStillNotOnTheForge(t *testing.T) {
const elsewhere = "http://elsewhere.example/x/y.git" const elsewhere = "http://elsewhere.example/x/y.git"
mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true, request: inventory.BuildRequest{ID: "build-1", For: "build"}}, elsewhere} mesh := &aMeshRegisteredFromTheURL{anEmptyMesh{asked: true, request: inventory.BuildRequest{ID: "build-1", For: "build"}}, elsewhere}
_, err := mayRegisterIn(t.Context(), mesh, sourceFormsRead(context.Background(), time.Now, read), "mesh-delivery-view", _, err := mayRegisterIn(t.Context(), mesh, sourceFormsRead(context.Background(), time.Now, read), "mesh-delivery-view",
inventory.Source{Repository: elsewhere}, byBuild("build-1"), "", "main") inventory.Source{Repository: elsewhere}, registrationAsk{Build: "build-1"}, "", "main")
if !errors.Is(err, errNotItsSource) || !strings.Contains(err.Error(), "is not on the mesh's forge") { if !errors.Is(err, errNotItsSource) || !strings.Contains(err.Error(), "is not on the mesh's forge") {
t.Fatalf("1277/AC4: the refusal changed: %v", err) t.Fatalf("1277/AC4: the refusal changed: %v", err)
} }
+6 -2
View File
@@ -444,8 +444,12 @@ func TestAMergeThatDeletesAModulePlansNothingToBuildForIt(t *testing.T) {
if err := inv.SavePlan(ctx, &plan); err != nil { if err := inv.SavePlan(ctx, &plan); err != nil {
t.Fatal(err) t.Fatal(err)
} }
planBuilt(ctx, open, "kept", "", "http://forge/novox/mesh-catalog.git has no module.json at modules/kept, so "+ if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-k", Module: "kept", Repository: "novox/mesh-catalog",
"there is nothing saying what it is: open …/module.json: no such file or directory", asked0, "build-k") At: time.Now().UTC(), Asked: asked0, Failed: "http://forge/novox/mesh-catalog.git has no module.json at modules/kept, so " +
"there is nothing saying what it is: open …/module.json: no such file or directory"}); err != nil {
t.Fatal(err)
}
advancePlans(ctx, open)
p, _ := inv.PlanByID(ctx, "plan-deleted") p, _ := inv.PlanByID(ctx, "plan-deleted")
if p.State == inventory.PlanFailed || p.Modules["kept"].State != planDeleted { if p.State == inventory.PlanFailed || p.Modules["kept"].State != planDeleted {
t.Fatalf("a module deleted at its source failed the plan: %s %+v", p.State, p.Modules["kept"]) t.Fatalf("a module deleted at its source failed the plan: %s %+v", p.State, p.Modules["kept"])
+168
View File
@@ -0,0 +1,168 @@
package main
import (
"context"
"errors"
"fmt"
"sort"
"strings"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
// The grants issued (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, issue 1394): what the assigner's sixth rule reads,
// that a module's bus credential for a fingerprint exists. A module's permissions on the bus are what the bus's user list
// composes from its registered manifest (inv.BusRecords), and a module's manifest is registered at a fingerprint when the
// controller takes it in from the builder's key; so once the list is sent to the machine that runs the bus, the
// credential of every module it carried is issued at the fingerprint of the build it is registered at. The controller
// writes `<module>.<fingerprint hex>` then, naming the accounts it composed for the module, never removed. A module one
// of whose accounts has no password yet (the list leaves it out) has no grant until a later send carries it.
// grantsFrom is the grant of every module whose accounts all have a password in hashes, at the fingerprint current names
// for it; a module whose current build names no fingerprint of the new form has none. principals are the bus's users
// as broker.Users composes them.
func grantsFrom(principals []broker.Principal, hashes map[string]string, current map[string]string) map[string]sdkdelivery.ControllerGrant {
accounts, missing := map[string][]string{}, map[string]bool{}
for _, p := range principals {
if p.Module == "" || (p.Kind != broker.KindModule && p.Kind != broker.KindProcess) {
continue
}
if hashes[p.Username()] == "" {
missing[p.Module] = true
continue
}
accounts[p.Module] = append(accounts[p.Module], p.Username())
}
out := map[string]sdkdelivery.ControllerGrant{}
for module, fp := range current {
if missing[module] || !strings.HasPrefix(fp, sdkdelivery.SourcePrefix2) {
continue
}
key, err := sdkdelivery.ControllerGrantKey(module, fp)
if err != nil {
continue
}
names := accounts[module]
sort.Strings(names)
out[key] = sdkdelivery.ControllerGrant{Accounts: names}
}
return out
}
// currentFingerprints is, by module, the fingerprint of the build each registered module is registered at: its
// manifest's commit (built_from), that commit's newest successful build, and the fingerprint that build names.
func currentFingerprints(ctx context.Context, inv *inventory.Inventory) (map[string]string, error) {
builds, err := inv.CurrentBuilds(ctx)
if err != nil {
return nil, err
}
out := map[string]string{}
for module, b := range builds {
if b.Commit == "" {
continue
}
id, err := inv.BuildOf(ctx, module, b.Commit)
if err != nil || id == "" {
continue
}
build, found, err := inv.BuildByID(ctx, id)
if err != nil || !found {
continue
}
out[module] = build.SourceFingerprint
}
return out, nil
}
// grantsKV is what writing the grants needs of their bucket.
type grantsKV interface {
Get(ctx context.Context, key string) (jetstream.KeyValueEntry, error)
Create(ctx context.Context, key string, value []byte, opts ...jetstream.KVCreateOpt) (uint64, error)
Update(ctx context.Context, key string, value []byte, revision uint64) (uint64, error)
}
// keepGrants writes every grant the list just sent issued that the bucket does not hold as it is: a create for a new one,
// a compare-and-set for one whose accounts changed. It answers how many it wrote.
func keepGrants(ctx context.Context, kv grantsKV, grants map[string]sdkdelivery.ControllerGrant) (int, error) {
keys := make([]string, 0, len(grants))
for k := range grants {
keys = append(keys, k)
}
sort.Strings(keys)
wrote := 0
var errs []error
for _, key := range keys {
value, err := sdkdelivery.EncodeControllerGrant(grants[key])
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", key, err))
continue
}
e, err := kv.Get(ctx, key)
switch {
case errors.Is(err, jetstream.ErrKeyNotFound):
_, err = kv.Create(ctx, key, value)
case err != nil:
default:
if before, derr := sdkdelivery.DecodeControllerGrant(e.Value()); derr == nil && sdkdelivery.SameControllerGrant(before, withVersion(grants[key])) {
continue
}
_, err = kv.Update(ctx, key, value, e.Revision())
}
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", key, err))
continue
}
wrote++
}
return wrote, errors.Join(errs...)
}
func withVersion(g sdkdelivery.ControllerGrant) sdkdelivery.ControllerGrant {
g.V = sdkdelivery.Version
return g
}
// grantsIssued writes the grants the bus's user list just sent to its machine issued (ADR 0528 clause 2). Said, never
// fatal: a grant not written now is written at the next send of the list, and the assigner's wants wait for it.
func grantsIssued(ctx context.Context, inv *inventory.Inventory, js jetstream.JetStream, indent string) {
kv, err := js.KeyValue(ctx, broker.GrantsBucket)
if err != nil {
fmt.Printf("%sthe grants issued are not written: the bucket %s: %v\n", indent, broker.GrantsBucket, err)
return
}
records, err := inv.BusRecords(ctx)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
principals, err := broker.Users(records)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
kept, err := inv.BusUsers(ctx)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
current, err := currentFingerprints(ctx, inv)
if err != nil {
fmt.Printf("%sthe grants issued are not written: %v\n", indent, err)
return
}
wrote, err := keepGrants(ctx, kv, grantsFrom(principals, hashes, current))
if wrote > 0 {
fmt.Printf("%s%d grant(s) issued written for the assigner\n", indent, wrote)
}
if err != nil {
fmt.Printf("%ssome grants issued are not written: %v\n", indent, err)
}
}
@@ -0,0 +1,91 @@
//go:build integration
package main
import (
"context"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// 1394/AC1 of C9 (ADR 0528 clause 2): when the controller has issued mesh-x's credential for fingerprint F (mesh-x
// registered at a build of F, its account minted, the bus's machine sent its declaration with the user list, through
// overTheBus.declare as a push sends it), then grants.mesh-x.F exists on the bus,
// and a second pass writes it again as nothing new.
func TestTheGrantOfAnIssuedCredentialIsOnTheBus(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
inv := open.inventory
fp := grantFP('d')
m := catalogue.Manifest{Module: "mesh-x", Version: "1", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh-x/broker"}}}
if err := inv.RecordBuild(ctx, inventory.Build{ID: "build-x1", Repository: "novox/mesh-x", Module: "mesh-x", Commit: "c0ffee1",
SourceFingerprint: fp}); err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-x", BuiltFrom: "c0ffee1", Head: "c0ffee1"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-x"); err != nil {
t.Fatal(err)
}
user := broker.Principal{Kind: broker.KindModule, Node: "anchor", Module: "mesh-x"}.Username()
if _, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: user, Kind: inventory.BusModule, Node: "anchor", Module: "mesh-x"}); err != nil {
t.Fatal(err)
}
conn, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(conn.Close)
js, err := jetstream.New(conn)
if err != nil {
t.Fatal(err)
}
if err := broker.OnConn(conn).EnsureControllerBuckets(); err != nil {
t.Fatal(err)
}
// The live path: a send of the bus's machine's declaration carrying the user list (overTheBus.declare), which records
// the list sent and then writes the grants it issued.
legacy, err := conn.JetStream()
if err != nil {
t.Fatal(err)
}
_ = js.DeleteStream(ctx, "GRANTS_DECLARED")
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "GRANTS_DECLARED", Subjects: []string{link.DeclareSubject("anchor")}}); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = js.DeleteStream(context.Background(), "GRANTS_DECLARED") })
send := overTheBus{open: open, server: link.SendingOn(link.OverNATS{Conn: conn, JS: legacy}), signer: aSignerOfZeros{}}
ready := readyNode{node: "anchor", declared: sendable{BusUsers: "the user list the bus's machine is sent"}}
if _, err := send.declare(ctx, ready, []byte(`{}`)); err != nil {
t.Fatalf("the send of the bus's machine: %v", err)
}
kv, err := js.KeyValue(ctx, broker.GrantsBucket)
if err != nil {
t.Fatal(err)
}
key, _ := sdkdelivery.ControllerGrantKey("mesh-x", fp)
e, err := kv.Get(ctx, key)
if err != nil {
t.Fatalf("no grant %s after the list was issued: %v", key, err)
}
g, err := sdkdelivery.DecodeControllerGrant(e.Value())
if err != nil || len(g.Accounts) != 1 || g.Accounts[0] != user {
t.Fatalf("the grant reads %+v, %v; want the account %s", g, err, user)
}
if _, err := send.declare(ctx, ready, []byte(`{}`)); err != nil {
t.Fatalf("the second send: %v", err)
}
if again, _ := kv.Get(ctx, key); again.Revision() != e.Revision() {
t.Errorf("a second pass wrote the grant again (revision %d, was %d)", again.Revision(), e.Revision())
}
}
-34
View File
@@ -9,9 +9,7 @@ import (
"strings" "strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
) )
// The grants reach the bus before the gate (novox/hq issue 490). // The grants reach the bus before the gate (novox/hq issue 490).
@@ -176,35 +174,3 @@ func TestTheGrantsStepKeepsOneMachinesBuilds(t *testing.T) {
t.Error("the grants step keeps the builds of a machine it does not send") t.Error("the grants step keeps the builds of a machine it does not send")
} }
} }
// The delivery plan says the step before the merge.
func TestAChangePlanSaysTheGrantsStep(t *testing.T) {
const catalogue_ = "http://forge.internal:20000/novox/mesh-catalog.git"
entry := func(name string, on ...string) inventory.Entry {
e := fromRepo(name, catalogue_, "modules/"+name)
e.Source.BuiltFrom = "old"
e.On = on
return e
}
nats := entry("nats", "novox")
nats.Manifest.BusUsers = "/etc/nats/users.conf"
nats.Manifest.Claims = []catalogue.Claim{{Name: catalogue.BrokerSeat, Scope: catalogue.ScopeMesh}}
restic := entry("restic", "ace", "novox", "shanks")
only := entry("bus-tools", "novox")
entries := []inventory.Entry{nats, restic, only}
rollsOut := func(string) (inventory.Upgrade, bool) { return inventory.Upgrade{RollOut: true}, true }
plan := func(paths ...string) link.ChangePlan {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: "head", Paths: paths}
return changePlanOf("novox/mesh-catalog", "main", "head", reachOfMerge(m, entries, nil, nil), entries, rollsOut)
}
text := planText(plan("modules/restic/module.json"))
if !strings.Contains(text, "grants step: if this changes what the bus's user list says (a new tool, subject or "+
"user), novox is sent that list alone, every build there kept, before restic is judged on its first machine") {
t.Errorf("the change plan does not say the grants step:\n%s", text)
}
// A module only on the bus's machine goes there with the list in its own send: no step of its own.
if text := planText(plan("modules/bus-tools/module.json")); strings.Contains(text, "grants step") {
t.Errorf("a module on the bus's machine alone reads:\n%s", text)
}
}
+106
View File
@@ -0,0 +1,106 @@
package main
import (
"context"
"strings"
"testing"
"time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
)
func grantFP(c byte) string { return sdkdelivery.SourcePrefix2 + strings.Repeat(string(c), 64) }
// 1394/AC (unit half): the list sent issues the credential of every module whose accounts all have a password, at the
// fingerprint its build names; a module with an account the list leaves out, and one registered at a fingerprint not of
// the new form, get no grant; a module with no account of its own is granted with none.
func TestTheGrantsAListIssues(t *testing.T) {
x := broker.Principal{Kind: broker.KindModule, Node: "novox", Module: "mesh-x"}
xp := broker.Principal{Kind: broker.KindProcess, Node: "novox", Module: "mesh-x", Process: "run"}
y := broker.Principal{Kind: broker.KindModule, Node: "ace", Module: "mesh-y"}
node := broker.Principal{Kind: broker.KindNode, Node: "novox"}
hashes := map[string]string{x.Username(): "h1", xp.Username(): "h2", node.Username(): "h3"}
got := grantsFrom([]broker.Principal{x, xp, y, node}, hashes,
map[string]string{"mesh-x": grantFP('a'), "mesh-y": grantFP('b'), "web": grantFP('c'), "old": "src1:abc"})
xKey, _ := sdkdelivery.ControllerGrantKey("mesh-x", grantFP('a'))
webKey, _ := sdkdelivery.ControllerGrantKey("web", grantFP('c'))
if len(got) != 2 {
t.Fatalf("grants = %v, want mesh-x's and web's", got)
}
if g := got[xKey]; len(g.Accounts) != 2 || g.Accounts[0] != x.Username() && g.Accounts[1] != x.Username() {
t.Errorf("mesh-x's grant = %+v, want both its accounts", g)
}
if g, ok := got[webKey]; !ok || len(g.Accounts) != 0 {
t.Errorf("web, with no account of its own, is granted %+v, %v", g, ok)
}
}
// fakeGrants is the grants bucket in memory.
type fakeGrants struct {
values map[string][]byte
creates int
updates int
}
type fakeEntry struct {
key string
value []byte
rev uint64
}
func (e fakeEntry) Bucket() string { return broker.GrantsBucket }
func (e fakeEntry) Key() string { return e.key }
func (e fakeEntry) Value() []byte { return e.value }
func (e fakeEntry) Revision() uint64 { return e.rev }
func (e fakeEntry) Created() time.Time { return time.Time{} }
func (e fakeEntry) Delta() uint64 { return 0 }
func (e fakeEntry) Operation() jetstream.KeyValueOp { return jetstream.KeyValuePut }
func (f *fakeGrants) Get(_ context.Context, key string) (jetstream.KeyValueEntry, error) {
v, ok := f.values[key]
if !ok {
return nil, jetstream.ErrKeyNotFound
}
return fakeEntry{key: key, value: v, rev: 1}, nil
}
func (f *fakeGrants) Create(_ context.Context, key string, value []byte, _ ...jetstream.KVCreateOpt) (uint64, error) {
if _, ok := f.values[key]; ok {
return 0, jetstream.ErrKeyExists
}
f.values[key] = value
f.creates++
return 1, nil
}
func (f *fakeGrants) Update(_ context.Context, key string, value []byte, _ uint64) (uint64, error) {
f.values[key] = value
f.updates++
return 2, nil
}
// 1394/AC (unit half): a grant is created once; a second send of the same list writes nothing; a reissue for other
// accounts is a new revision of the same key.
func TestAGrantIsWrittenOnceAndMovedOnAReissue(t *testing.T) {
kv := &fakeGrants{values: map[string][]byte{}}
key, _ := sdkdelivery.ControllerGrantKey("mesh-x", grantFP('a'))
grants := map[string]sdkdelivery.ControllerGrant{key: {Accounts: []string{"novox.mesh-x"}}}
for range 2 {
if _, err := keepGrants(t.Context(), kv, grants); err != nil {
t.Fatal(err)
}
}
if kv.creates != 1 || kv.updates != 0 {
t.Errorf("creates %d, updates %d, want one create", kv.creates, kv.updates)
}
grants[key] = sdkdelivery.ControllerGrant{Accounts: []string{"ace.mesh-x"}}
if wrote, err := keepGrants(t.Context(), kv, grants); err != nil || wrote != 1 || kv.updates != 1 {
t.Errorf("a reissue wrote %d (%v), updates %d", wrote, err, kv.updates)
}
if g, err := sdkdelivery.DecodeControllerGrant(kv.values[key]); err != nil || g.Accounts[0] != "ace.mesh-x" {
t.Errorf("the grant reads %+v, %v", g, err)
}
}
-12
View File
@@ -1,12 +0,0 @@
package main
import "testing"
// C2 (ADR 0503 §3; issue 1299): a delivery group's composed check is answered as the checker's, and nothing is asked
// of build-agent: the command answers before any ask is composed.
func TestTheGroupCheckNamesTheCheckerAndAsksNothing(t *testing.T) {
a := groupCheckAnswer("feat/x")
if a["judged-by"] != "mesh-delivery-checker" || a["asked"] != "" || a["group"] != "feat/x" {
t.Fatalf("the group check answered %v", a)
}
}
+3
View File
@@ -64,6 +64,9 @@ var handActVerbs = []handActVerb{
{Verb: "plans stop", Decision: "the operator's answer to an ask is their decision, not a repair (ADR 0259)", {Verb: "plans stop", Decision: "the operator's answer to an ask is their decision, not a repair (ADR 0259)",
DecidedFor: []string{conditions.CauseOperatorAnswer}}, DecidedFor: []string{conditions.CauseOperatorAnswer}},
{Verb: "plans close"}, {Verb: "plans close"},
// A module registered at references by hand, because the builder had no key for it (ADR 0526 clause 3): a repair
// the builder should have made, so it counts.
{Verb: "register"},
// A walk started by a person instead of its delivery's owner (novox/hq ADR 0239): the owner down, or // A walk started by a person instead of its delivery's owner (novox/hq ADR 0239): the owner down, or
// not trusted with it — either is a repair the owner should have made. Unless the operator chose it on // not trusted with it — either is a repair the owner should have made. Unless the operator chose it on
// a warrant (ADR 0259). // a warrant (ADR 0259).
+4 -4
View File
@@ -31,7 +31,7 @@ func TestAWalkWaitingOnAHeldDeliveryNamesTheHold(t *testing.T) {
"check did not pass for the heads that merged — a person decides: `mesh-delivery.release` (id " + "check did not pass for the heads that merged — a person decides: `mesh-delivery.release` (id " +
"novox/mesh-sdk@d41d255f28ad, why) lets it go, `mesh-delivery.stop` ends it" "novox/mesh-sdk@d41d255f28ad, why) lets it go, `mesh-delivery.stop` ends it"
waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@d41d255f28ad", State: "held", Walk: p.ID, Says: held}}} waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@d41d255f28ad", State: "held", Walk: p.ID, Says: held}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits) line := planLineSaid(p, now, tierAtLeast, nil, waits)
for _, want := range []string{"held for a person", "composed check did not pass", "mesh-delivery.release"} { for _, want := range []string{"held for a person", "composed check did not pass", "mesh-delivery.release"} {
if !strings.Contains(line, want) { if !strings.Contains(line, want) {
t.Fatalf("the line does not say %q: %q", want, line) t.Fatalf("the line does not say %q: %q", want, line)
@@ -41,7 +41,7 @@ func TestAWalkWaitingOnAHeldDeliveryNamesTheHold(t *testing.T) {
t.Fatalf("a held walk's line points at a hand start: %q", line) t.Fatalf("a held walk's line points at a hand start: %q", line)
} }
// Told nothing — an older mesh-delivery, or none answering — it reads as before. // Told nothing — an older mesh-delivery, or none answering — it reads as before.
if line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, nil); !strings.Contains(line, "starts it by hand") { if line := planLineSaid(p, now, tierAtLeast, nil, nil); !strings.Contains(line, "starts it by hand") {
t.Fatalf("told nothing, the line reads %q", line) t.Fatalf("told nothing, the line reads %q", line)
} }
} }
@@ -51,7 +51,7 @@ func TestAWalkWaitingOnADeliveryNotAtItsTurnSaysSo(t *testing.T) {
p := waitingWalk(now) p := waitingWalk(now)
waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@9747f3e1b1ac", State: "published", Walk: p.ID, waits := walkWaits{p.ID: {{ID: "novox/mesh-sdk@9747f3e1b1ac", State: "published", Walk: p.ID,
Says: "not let go yet: novox/mesh-tools@3f0c1a6cec18 (published) carried by it not at its turn"}}} Says: "not let go yet: novox/mesh-tools@3f0c1a6cec18 (published) carried by it not at its turn"}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits) line := planLineSaid(p, now, tierAtLeast, nil, waits)
if !strings.Contains(line, "not at its turn") || !strings.Contains(line, "novox/mesh-tools@3f0c1a6cec18") { if !strings.Contains(line, "not at its turn") || !strings.Contains(line, "novox/mesh-tools@3f0c1a6cec18") {
t.Fatalf("the line reads %q", line) t.Fatalf("the line reads %q", line)
} }
@@ -69,7 +69,7 @@ func TestAWalkHeldByNothingSaysWhyAndTheHandStart(t *testing.T) {
waits := walkWaits{p.ID: {{ID: "novox/mesh-media-catalog@cdcdcdcd0000", State: "unheld", Walk: p.ID, waits := walkWaits{p.ID: {{ID: "novox/mesh-media-catalog@cdcdcdcd0000", State: "unheld", Walk: p.ID,
Says: "held by nothing: its merges' deliveries ended as nothing-to-move (novox/mesh-media-catalog@abababab0000), " + Says: "held by nothing: its merges' deliveries ended as nothing-to-move (novox/mesh-media-catalog@abababab0000), " +
"yet the walk moves nzbgeek, nzbget — nothing gives it its word"}}} "yet the walk moves nzbgeek, nzbget — nothing gives it its word"}}}
line := planLineSaid(p, now, pauseView{}, tierAtLeast, nil, waits) line := planLineSaid(p, now, tierAtLeast, nil, waits)
if !strings.Contains(line, "ended as nothing-to-move") || !strings.Contains(line, "plans go "+p.ID) { if !strings.Contains(line, "ended as nothing-to-move") || !strings.Contains(line, "plans go "+p.ID) {
t.Fatalf("the line reads %q", line) t.Fatalf("the line reads %q", line)
} }
+122 -9
View File
@@ -11,6 +11,7 @@ import (
"fmt" "fmt"
"io" "io"
"path" "path"
"sort"
"strings" "strings"
"time" "time"
@@ -91,16 +92,42 @@ var predefinedAtTrunk = func(ctx context.Context, inv *inventory.Inventory) (pre
if err != nil { if err != nil {
return predefinedHead{}, err return predefinedHead{}, err
} }
repository := "" return readPredefinedHead(ctx, entries, formsOfSources(ctx, inv))
}
// formsOfSources reads how the sources are spelled on this mesh; a variable so a test names the forge's address.
var formsOfSources = newSourceForms
// errNoCatalogueRepository says no module of the mesh is built from the catalogue's repository on the forge: not a wait,
// since no read of the forge makes one appear (issue 1902); the plan fails with these words.
var errNoCatalogueRepository = errors.New("no module of the catalogue names the repository " +
catalogue.CatalogueRepository + " on the forge")
// catalogueRepositoryAmong is the catalogue's repository on the forge, as owner/name, found as catch-up finds the trunks
// of the repositories the catalogue builds from: a source in the URL form or the seat form, `.git` dropped (issue 1902).
func catalogueRepositoryAmong(entries []inventory.Entry, forms *sourceForms) (string, error) {
for _, e := range entries { for _, e := range entries {
if e.Source.Seat == gitSeat && path.Base(strings.Trim(e.Source.Repository, "/")) == catalogue.CatalogueRepository { if e.Source.Repository == "" {
repository = strings.Trim(e.Source.Repository, "/") continue
break }
owner, name, onForge := forms.onTheForge(e.Source.Repository, e.Source.Seat)
if onForge && strings.EqualFold(name, catalogue.CatalogueRepository) {
return owner + "/" + name, nil
} }
} }
if repository == "" { return "", errNoCatalogueRepository
return predefinedHead{}, fmt.Errorf("no module of the catalogue names the repository %s on the forge", }
catalogue.CatalogueRepository)
// readPredefinedHead is the list at the trunk head of the catalogue's repository, found among the modules' sources and
// read from the forge.
func readPredefinedHead(ctx context.Context, entries []inventory.Entry, forms *sourceForms) (predefinedHead, error) {
// A URL source is on the forge only once the forge's address is known; an address that cannot be read is a wait.
if _, err := forms.forgeAddress(ctx, forgeAddressWait); err != nil {
return predefinedHead{}, fmt.Errorf("the forge's address cannot be read: %w", err)
}
repository, err := catalogueRepositoryAmong(entries, forms)
if err != nil {
return predefinedHead{}, err
} }
var heads []struct { var heads []struct {
SHA string `json:"sha"` SHA string `json:"sha"`
@@ -142,7 +169,7 @@ var predefinedAtTrunk = func(ctx context.Context, inv *inventory.Inventory) (pre
} }
// askGitea asks the forge's module one tool on the bus and reads its result into into. // askGitea asks the forge's module one tool on the bus and reads its result into into.
func askGitea(ctx context.Context, tool string, args map[string]any, into any) error { var askGitea = func(ctx context.Context, tool string, args map[string]any, into any) error {
js, err := aBus() js, err := aBus()
if err != nil { if err != nil {
return err return err
@@ -171,6 +198,10 @@ type keyTake struct {
Module string Module string
Trunk sdkdelivery.Module Trunk sdkdelivery.Module
Ask registrationAsk Ask registrationAsk
// Hand names, by artifact, the references the controller's terminal registers the module at instead of a key's
// (ADR 0526 clause 3, `register --reference`): the key, when the builder has one, supplies the artifacts not named
// here. Nil for every take-in but that one.
Hand map[string]string
} }
// takeInKey takes a module in from the builder's key for the fingerprint its trunk value names: records the key as a // takeInKey takes a module in from the builder's key for the fingerprint its trunk value names: records the key as a
@@ -178,6 +209,10 @@ type keyTake struct {
func takeInKey(ctx context.Context, inv *inventory.Inventory, keys keyReads, t keyTake) ( func takeInKey(ctx context.Context, inv *inventory.Inventory, keys keyReads, t keyTake) (
catalogue.Manifest, inventory.Build, error) { catalogue.Manifest, inventory.Build, error) {
head, err := predefinedAtTrunk(ctx, inv) head, err := predefinedAtTrunk(ctx, inv)
if errors.Is(err, errNoCatalogueRepository) {
// Fails the plan, loudly, with these words (issue 1902): waiting would never end.
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("%s is not judged: %w", t.Module, err)
}
if err != nil { if err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("%w: the list of predefined keys at the catalogue's "+ return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("%w: the list of predefined keys at the catalogue's "+
"trunk head cannot be read, so %s is not judged: %v", errKeyWaits, t.Module, err) "trunk head cannot be read, so %s is not judged: %v", errKeyWaits, t.Module, err)
@@ -234,7 +269,7 @@ func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
waits := func(format string, a ...any) (inventory.Build, catalogue.Manifest, error) { waits := func(format string, a ...any) (inventory.Build, catalogue.Manifest, error) {
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("%w: "+format, append([]any{errKeyWaits}, a...)...) return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("%w: "+format, append([]any{errKeyWaits}, a...)...)
} }
if t.Trunk.FP2 == "" { if t.Trunk.FP2 == "" && t.Hand == nil {
why := t.Trunk.None2 why := t.Trunk.None2
if why == "" { if why == "" {
why = "the indexer wrote no fingerprint of the new form" why = "the indexer wrote no fingerprint of the new form"
@@ -242,6 +277,12 @@ func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
return waits("%s at %.8s has no fingerprint to take a build by: %s", t.Module, t.Trunk.Commit, why) return waits("%s at %.8s has no fingerprint to take a build by: %s", t.Module, t.Trunk.Commit, why)
} }
k, err := keys.Build(ctx, t.Module, t.Trunk.FP2) k, err := keys.Build(ctx, t.Module, t.Trunk.FP2)
if t.Hand != nil {
// **Registered at a reference, no key needed** (ADR 0526 clause 3): the key the builder holds, if any, is the
// start; the references given replace or add its artifacts, and the key counts as done.
k = handKey(t, k)
err = nil
}
if err != nil { if err != nil {
return waits("%s's key cannot be read: %v", t.Module, err) return waits("%s's key cannot be read: %v", t.Module, err)
} }
@@ -272,6 +313,11 @@ func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the key of %s names the manifest of %s", return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the key of %s names the manifest of %s",
t.Module, manifest.Module) t.Module, manifest.Module)
} }
if t.Hand != nil {
if err := handCoversEvery(manifest, k, t.Hand); err != nil {
return inventory.Build{}, catalogue.Manifest{}, err
}
}
built, err := builtOfKey(ctx, kc.archive, manifest, k) built, err := builtOfKey(ctx, kc.archive, manifest, k)
if err != nil { if err != nil {
return waits("%v", err) return waits("%v", err)
@@ -308,6 +354,10 @@ func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
break break
} }
} }
if t.Hand != nil {
// A build made by hand, said so: the id is its own, so it never reads as the builder's key.
kept.ID, kept.On = "hand-"+keyBuildID(k)+"-"+time.Now().UTC().Format("20060102T150405Z"), "hand"
}
for _, a := range k.Build.Artifacts { for _, a := range k.Build.Artifacts {
kept.Made = append(kept.Made, inventory.Artifact{Name: a.Name, Kind: kindOf(manifest, a), kept.Made = append(kept.Made, inventory.Artifact{Name: a.Name, Kind: kindOf(manifest, a),
Reference: catalogue.Recorded(a.Reference)}) Reference: catalogue.Recorded(a.Reference)})
@@ -458,3 +508,66 @@ func pathsBuildSource(m catalogue.Manifest, dir string) []string {
} }
return out return out
} }
// handKey is the key a registration at references is made from: the builder's key when it has one (its other artifacts
// stand), with each reference given in place of or beside the key's artifact of that name, and counted done.
func handKey(t keyTake, k builderkeys.Key) builderkeys.Key {
if !k.Found {
k = builderkeys.Key{Module: t.Module, Fingerprint: t.Trunk.FP2, Found: true}
}
build := k.Build
build.State, build.By, build.Error = sdkdelivery.BuildDone, "hand", ""
if build.Manifest == "" {
build.Manifest = t.Trunk.Manifest
}
build.Artifacts = nil
for _, a := range k.Build.Artifacts {
if _, replaced := t.Hand[a.Name]; !replaced {
build.Artifacts = append(build.Artifacts, a)
}
}
names := make([]string, 0, len(t.Hand))
for name := range t.Hand {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
reference := t.Hand[name]
build.Artifacts = append(build.Artifacts, sdkdelivery.BuiltArtifact{Name: name,
Digest: handDigestOf.FindString(reference), Reference: reference})
}
k.Build = build
return k
}
// handCoversEvery refuses a registration at references that names an artifact the manifest does not declare, or leaves
// one it declares without a reference (the key's or a given one).
func handCoversEvery(m catalogue.Manifest, k builderkeys.Key, hand map[string]string) error {
declared := map[string]bool{}
if m.Build != nil {
for _, a := range m.Build.Artifacts {
declared[a.Name] = true
}
}
for name := range hand {
if !declared[name] {
return fmt.Errorf("--reference %s: %s declares no artifact of that name. Nothing was registered", name, m.Module)
}
}
have := map[string]bool{}
for _, a := range k.Build.Artifacts {
have[a.Name] = true
}
var missing []string
for name := range declared {
if !have[name] {
missing = append(missing, name)
}
}
sort.Strings(missing)
if len(missing) > 0 {
return fmt.Errorf("%s declares the artifact(s) %s and no --reference names them, nor does a key of the builder. "+
"Nothing was registered", m.Module, strings.Join(missing, ", "))
}
return nil
}
@@ -249,3 +249,23 @@ func TestTheSwitchsListReadsTheHeldBuildsAndTheKeys(t *testing.T) {
t.Fatalf("A13: the switch's list is %+v", lines) t.Fatalf("A13: the switch's list is %+v", lines)
} }
} }
// Issue 1902: the take-in with the lookup of the catalogue's repository NOT stubbed - the catalogue's modules are registered
// with a URL source and no seat, as every live row is - reads the list from the forge's wire alone and registers the module.
func TestATakeInFindsTheCataloguesRepositoryInTheURLForm(t *testing.T) {
open := theCatalogue(t)
ctx := t.Context()
was, wasForms := formsOfSources, askGitea
formsOfSources = func(context.Context, *inventory.Inventory) *sourceForms { return aFormsOnTheForge() }
t.Cleanup(func() { formsOfSources, askGitea = was, wasForms })
aForgeAnsweringTheCatalogue(t)
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "catalogue-anchor", Version: "1"},
inventory.Source{Repository: "http://anchor.internal:20000/novox/mesh-catalog.git", Path: "modules/catalogue-anchor",
Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
keys, trunk := aKeyFor("sudo", "6", fpOf('7'), "novox/mesh-catalog", "modules/sudo", "c0ffee0123456789")
if _, _, err := takeInKey(ctx, open.inventory, keys, keyTake{Module: "sudo", Trunk: trunk, Ask: registrationAsk{Merge: trunk.Commit}}); err != nil {
t.Fatalf("a key was not taken in with the real lookup of the catalogue's repository: %v", err)
}
}
@@ -0,0 +1,317 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"path"
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// What the tests below take a build in with (R4g): the controller itself registers a module only from the builder's done
// key now (takeInKey), but a great many of these tests only need a module registered with a build recorded, in the form
// the old outcome carried, so that is kept here and nowhere else. It is the controller's take-in of a build-agent outcome
// as it was, less the checks only an outcome needed. Nothing in the controller calls any of it.
// buildFrom turns what a builder said into what the mesh keeps.
//
// **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder
// says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store
// reference and composes the store's address back in where a reference is used. `against` — what
// the build stood on, the catalogue's edge — is recorded the same way, so an edge names a module's
// artifact and not the machine it was pulled from.
func buildFrom(result link.BuildResult) inventory.Build {
kept := inventory.Build{
ID: result.ID, Repository: result.Repository, Ref: result.Ref,
Commit: result.Commit, On: result.On, Failed: result.Failed,
// **What the announcement carries, kept rather than discarded** (novox/hq 04-ISSUES/050).
// The catalogue turns the manifest into requires/provides edges and `against` into build
// edges, and it is not always listening when a build happens — on a fresh mesh it cannot
// be, for exactly the modules it needs most. Keeping them is what makes a replay able to
// rebuild the graph rather than a list of names.
Path: result.Path,
// What it was made from (novox/hq issue 280): two builds with one are one build.
SourceFingerprint: result.SourceFingerprint,
SourceInputs: result.SourceInputs,
}
// When it was asked, which is what orders it against another build of the same module
// (novox/hq 04-ISSUES/219) — not when it was heard.
if asked, ok := link.BuildAskedAt(result.ID); ok {
kept.Asked = asked
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
for _, ref := range result.Mirrored {
kept.Mirrored = append(kept.Mirrored, catalogue.Recorded(ref))
}
for _, r := range result.Read {
kept.Read = append(kept.Read, inventory.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
// What it was made from, as files (novox/hq ADR 0267): the planner maps the next merge onto it.
for _, s := range result.Sources {
kept.Sources = append(kept.Sources, inventory.BuildSource{Repository: s.Repository, Ref: s.Ref, Paths: s.Paths})
}
var announced []inventory.Artifact
for _, made := range result.Made {
announced = append(announced, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: made.Reference,
})
kept.Made = append(kept.Made, inventory.Artifact{
Name: made.Name, Kind: made.Kind, Reference: catalogue.Recorded(made.Reference),
})
}
kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 {
if list, err := judgedWith(result); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module
}
}
}
return kept
}
// recordedManifest is a build's manifest with the store's address taken off every reference the
// build itself made. Other references — an image a module runs from a public registry — are what
// they were, which is why this rewrites only what `made` names rather than everything that looks
// like an address.
func recordedManifest(raw json.RawMessage, made []inventory.Artifact) json.RawMessage {
announced := map[string]bool{}
for _, a := range made {
announced[a.Reference] = true
}
return withReferences(raw, func(ref string) (string, bool) {
if !announced[ref] {
return ref, false
}
return catalogue.Recorded(ref), true
})
}
// takeIn is what the mesh does with a build's outcome, whoever hears it: the waiting command and
// the daemon that follows the role's events both come here (novox/hq issue 176), so a build's
// result reaches the catalogue whether or not the asker was still listening.
//
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something. Then parsed with the same parser a hand-written manifest goes through — a second path
// would be a second thing to disagree about what a manifest is — and registered with where it came
// from: **for a source on a seat, as the path and the seat, never the URL just cloned** (ADR 0111),
// which the request carried and the outcome echoes. A definition naming an installation is refused
// here, where it would enter the catalogue; the build stays recorded and the refusal says which.
//
// Idempotent: the same outcome taken in twice registers the same module twice, which is one row
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
// The list of predefined keys as the store keeps it now: another controller process may have kept a newer one.
if err := loadPredefined(ctx, inv); err != nil {
return catalogue.Manifest{}, inventory.Build{}, fmt.Errorf("the list of predefined keys kept cannot be "+
"read: %w", err)
}
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
// Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
// When it was asked, so an older request heard later does not replace a newer one
// (novox/hq 04-ISSUES/219).
Asked: kept.Asked,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
// **A build at a commit does not change the branch a module follows** (novox/hq 04-ISSUES/215):
// the commit is built and recorded as what it was built from, and the module keeps following
// what it followed before — the repository's default branch for one new to the catalogue.
if followedBranch(result.Ref) == "" && result.Ref != "" {
recorded.Ref = ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
}
}
trunk := result.Trunk
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil && followedBranch(was.Ref) != "" {
trunk = followedBranch(was.Ref)
}
if trunk == "" {
trunk = "main"
}
j := takenBuild{kept: kept, manifest: manifest, recorded: recorded, ask: registrationAsk{Build: result.ID}, path: result.Path,
trunk: trunk, by: result.On, what: result.Repository, commit: result.Commit}
// The checks only an outcome of build-agent needs: a key is a trunk build by construction (ADR 0501 §2), in no
// scratch namespace, and carries no list of its own.
j.published = func() error {
// **Only a commit on the trunk is published** (novox/hq ADR 0238): a commit off its repository's
// default branch — a pull request's head, a feature branch built by hand, a `rebuild` or `replay
// --register` of one — is for checking, and is never a module's version; nothing could then send it.
// The branch the module already follows is its trunk — never the branch a build was asked at, or a
// build of a feature branch by name would make that branch its trunk.
follows := ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
follows = followedBranch(was.Ref)
}
if err := publishable(result, follows); err != nil {
return err
}
return nil
}
j.carried = func() error {
// **The newest trunk build's list of predefined keys is published** (novox/hq ADR 0348 §7): a build of the
// catalogue that may be registered keeps its list when its commit is newer than the one kept.
carried, err := carriedList(result)
if err != nil {
return err
}
if carried != nil {
if err := publishPredefined(ctx, inv, result, carried); err != nil {
fmt.Printf("%s: the list of predefined keys it carried is not published: %v\n", result.ID, err)
}
}
return nil
}
manifest, err = registerJudged(ctx, inv, j)
return manifest, kept, err
}
// errOffTheTrunk is a build of a commit off its repository's trunk, which is never published.
var errOffTheTrunk = errors.New("the commit is not on its repository's trunk: a commit off the trunk is checked, " +
"never published (ADR 0238) — merge it, and the merge builds it")
// publishable says whether a build's outcome may become a module's version: its commit on the module's
// trunk, as the build seat read the forge at the build — the branch the module follows when its source
// names one, else its repository's default branch. A build seat that could not say — one older than the
// rule, building its own successor — is let through and said, so the rule can reach the mesh.
func publishable(result link.BuildResult, follows string) error {
if result.DryRun {
return errors.New("a dry run is never published")
}
if result.Trunk == "" {
fmt.Printf("%s: the build seat did not say whether %s is on its repository's trunk (it predates ADR 0238); "+
"registered as before\n", result.ID, short(result.Commit))
return nil
}
trunk := result.Trunk
if follows != "" {
trunk = follows
}
on := slices.Contains(result.Branches, trunk) || (trunk == result.Trunk && result.OnTrunk)
if !on {
return fmt.Errorf("%w (%s is not on %s)", errOffTheTrunk, short(result.Commit), trunk)
}
return nil
}
// fromTheCatalogue is whether a build is of the catalogue's repository, whose list alone the mesh publishes: by
// its name, in the URL form or the seat form.
func fromTheCatalogue(result link.BuildResult) bool {
for _, repository := range []string{result.Repository, sourceRepository(result)} {
name := strings.TrimSuffix(path.Base(strings.TrimRight(repository, "/")), ".git")
if name == catalogue.CatalogueRepository {
return true
}
}
return false
}
func sourceRepository(result link.BuildResult) string {
if result.Source == nil {
return ""
}
return result.Source.Repository
}
// carriedList is the list a build of the catalogue carried, read strictly; nil for a build of another
// repository, or one that carried none.
func carriedList(result link.BuildResult) (catalogue.PredefinedList, error) {
if result.Predefined == nil || !fromTheCatalogue(result) {
return nil, nil
}
list, err := catalogue.ParsePredefined(result.Predefined.Keys)
if err != nil {
return nil, fmt.Errorf("%s carried a list of predefined keys that is refused: %w", result.ID, err)
}
return list, nil
}
// judgedWith is the list a build's manifest is first judged against at registration: the catalogue's own at the
// commit built, which its build judged it against, where it carried one; the list kept otherwise.
func judgedWith(result link.BuildResult) (catalogue.PredefinedList, error) {
list, err := carriedList(result)
if err != nil || list != nil {
return list, err
}
return catalogue.PublishedPredefined(), nil
}
// publishPredefined keeps a trunk build's list when it is newer than the one kept, says so, and loads what the
// store keeps then — this build's, or a newer one another controller process kept.
func publishPredefined(ctx context.Context, inv *inventory.Inventory, result link.BuildResult,
list catalogue.PredefinedList) error {
kept, err := inv.KeepPredefined(ctx, inventory.PredefinedRecord{Keys: list, Repository: result.Repository,
Commit: result.Commit, Generation: result.Predefined.Generation, Build: result.ID})
if err != nil {
return err
}
if kept {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is published: %s\n", result.ID,
short(result.Commit), result.Predefined.Generation, strings.Join(list.Keys(), ", "))
} else {
fmt.Printf("%s: the list of predefined keys at %s (generation %d) is not newer than the one kept, which "+
"stands\n", result.ID, short(result.Commit), result.Predefined.Generation)
}
return loadPredefined(ctx, inv)
}
// builds is what the tests of a pending assignment feed an outcome to: the take-in above, then the tick that settles what
// waited for it (settlePending), as the daemon's hearing of an outcome did before R4g (it also moved the plans, which the
// walk now does from its keys). Nothing in the controller calls it.
type builds struct {
inv *inventory.Inventory
open *stores
}
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
_, _, err := takeIn(ctx, b.inv, result)
if b.open != nil {
settlePending(ctx, b.open, time.Now())
}
if result.Failed != "" {
return nil // the build is recorded; whether it registered is what the tests read
}
return err
}
+4 -23
View File
@@ -34,16 +34,6 @@ func deafObservations(now time.Time, held func(broker.Consumer) (uint64, error))
return obs[0].Said, len(obs) return obs[0].Said, len(obs)
} }
// 1063/AC1: a check in flight on a builder is not the controller's event loop being deaf.
func TestDeafLookIgnoresCheckWorkers(t *testing.T) {
now := time.Now()
for _, w := range broker.CheckWorkers() {
if said, n := deafObservations(now, holdingOne(w.Stream+"/"+w.Name)); n != 0 {
t.Fatalf("%s holding one message raised controller-deaf: %s", w.Name, said)
}
}
}
// 1063/AC2: a message held by one of the controller's own consumers is still deaf, naming its stream. // 1063/AC2: a message held by one of the controller's own consumers is still deaf, naming its stream.
func TestDeafLookStillCountsOwnConsumers(t *testing.T) { func TestDeafLookStillCountsOwnConsumers(t *testing.T) {
now := time.Now() now := time.Now()
@@ -55,25 +45,16 @@ func TestDeafLookStillCountsOwnConsumers(t *testing.T) {
} }
} }
// 1063/AC3: the list is the controller's consumers and none of the check workers. // 1063/AC3: the list is the controller's consumers: the mesh defines no other durable consumer the loop could read (the
// check workers went with the checks' queues, ADR 0525 clause 6).
func TestDeafLookReadsExactlyTheControllersOwnConsumers(t *testing.T) { func TestDeafLookReadsExactlyTheControllersOwnConsumers(t *testing.T) {
workers := map[string]bool{}
for _, w := range broker.CheckWorkers() {
workers[w.Stream+"/"+w.Name] = true
}
want := map[string]bool{} want := map[string]bool{}
for _, c := range broker.MeshConsumers() { for _, c := range broker.MeshConsumers() {
if !workers[c.Stream+"/"+c.Name] { want[c.Stream+"/"+c.Name] = true
want[c.Stream+"/"+c.Name] = true
}
} }
got := map[string]bool{} got := map[string]bool{}
for _, c := range loopConsumers() { for _, c := range loopConsumers() {
k := c.Stream + "/" + c.Name got[c.Stream+"/"+c.Name] = true
if workers[k] {
t.Errorf("the deaf look reads the check worker %s", k)
}
got[k] = true
} }
if len(got) != len(want) { if len(got) != len(want) {
t.Fatalf("deaf look reads %v, want %v", got, want) t.Fatalf("deaf look reads %v, want %v", got, want)
+22 -115
View File
@@ -8,7 +8,6 @@ package main
import ( import (
"context" "context"
"errors"
"flag" "flag"
"fmt" "fmt"
"os" "os"
@@ -19,7 +18,6 @@ import (
"github.com/novox/mesh-controller/internal/identity" "github.com/novox/mesh-controller/internal/identity"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences" "github.com/novox/mesh-controller/internal/licences"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/store" "github.com/novox/mesh-controller/internal/store"
) )
@@ -82,8 +80,6 @@ func run() error {
case "build", "rebuild": case "build", "rebuild":
// **The hand build goes with the asks** (novox/hq ADR 0501 §5): refused by name, naming the replacement. // **The hand build goes with the asks** (novox/hq ADR 0501 §5): refused by name, naming the replacement.
return removedByName(args[0]) return removedByName(args[0])
case "builder":
return builderCommand(ctx, args[1:])
case "board": case "board":
return boardCommand(ctx, args[1:]) return boardCommand(ctx, args[1:])
case "api": case "api":
@@ -98,23 +94,10 @@ func run() error {
return rehearseCommand(ctx, args[1:]) return rehearseCommand(ctx, args[1:])
case "builds": case "builds":
return buildsCommand(ctx, args[1:]) return buildsCommand(ctx, args[1:])
// The build queue, controlled by hand (novox/hq ADR 0219). case "forget-checks", "queue", "cancel", "clear", "kill", "replay", "pause", "resume":
case "queue": return removedByName(args[0])
return queueCommand(ctx, args[1:])
case "cancel":
return cancelCommand(ctx, args[1:])
case "clear":
return clearCommand(ctx, args[1:])
case "replay":
return replayCommand(ctx, args[1:])
case "kill":
return killCommand(ctx, args[1:])
case "forget-checks":
return removedByName("forget-checks")
case "try": case "try":
return tryCommand(ctx, args[1:]) return tryCommand(ctx, args[1:])
case "pause", "resume":
return pauseCommand(ctx, args[0], args[1:])
case "collection": case "collection":
return collectionCommand(ctx, args[1:]) return collectionCommand(ctx, args[1:])
// What the records say the registries may keep, asked on demand (novox/hq ADR 0251). // What the records say the registries may keep, asked on demand (novox/hq ADR 0251).
@@ -157,12 +140,6 @@ func run() error {
// The facts snapshot a merge check is fed (novox/hq to-be 45 §9). // The facts snapshot a merge check is fed (novox/hq to-be 45 §9).
case "facts": case "facts":
return factsCommand(ctx, args[1:]) return factsCommand(ctx, args[1:])
// The merge gate: every machine of the snapshot composed with a change (novox/hq to-be 45 §9).
case "merge-gate":
return mergeGateCommand(ctx, args[1:])
// A pull request's merge check run here exactly as the build seat runs it (novox/hq issue 286).
case "check-here":
return checkHereCommand(ctx, args[1:])
case "upgrade": case "upgrade":
return upgradeCommand(ctx, args[1:]) return upgradeCommand(ctx, args[1:])
// The bus as a planned step (novox/hq to-be 45 §8, ADR 0236). // The bus as a planned step (novox/hq to-be 45 §8, ADR 0236).
@@ -313,15 +290,11 @@ var usageText = `mesh-controller — the control plane
operator key set <public> [--replace] tell the mesh which operator key to seal to operator key set <public> [--replace] tell the mesh which operator key to seal to
operator key show the operator key, and what it can recover operator key show the operator key, and what it can recover
register <module> <owner>/<repository> <path> register a module from a repository the catalogue does not register <module> <owner>/<repository> <path> register a module from a repository the catalogue does not
build, from the builder's done key of its fingerprint (ADR 0501 §4) build, from the builder's done key of its fingerprint (ADR 0501 §4); with
--reference <artifact>=<reference> (repeatable) and --why, at artifacts the store
holds, recorded as a build by hand (ADR 0526 §3)
switch [--json] the switch's list: each module, the build it runs, and what the builder holds switch [--json] the switch's list: each module, the build it runs, and what the builder holds
builds [<module>] what has been built lately, and what came of it builds [<module>] [-n N] what has been built lately: the builder's done keys, recorded as builds
queue [--json] every ask in the build queue: waiting, in flight (where, how long), dead
cancel <id> drop a waiting or dead ask; recorded failed, cancelled by hand
clear [--dead] cancel every waiting ask (and the dead ones); never one in flight
replay <build-id> that build's commit again, a dry run: what a build of it makes
kill <id> end a build where it runs; recorded failed, killed by hand
pause [<node>] / resume [<node>] the build seat's holder there, or every holder, takes nothing new / again
plans retry <id> ask a failed plan's failed builds again, and carry the plan on plans retry <id> ask a failed plan's failed builds again, and carry the plan on
plans stop|close <id> --why <text> end a plan by hand; recorded in the hand-act log plans stop|close <id> --why <text> end a plan by hand; recorded in the hand-act log
hand-act record <what> --why <text> --cause <word> [--condition <key>] hand-act record <what> --why <text> --cause <word> [--condition <key>]
@@ -338,8 +311,6 @@ var usageText = `mesh-controller — the control plane
durations [--kind K] [--days N] [--json] durations [--kind K] [--days N] [--json]
apply, heartbeat, plan-tier, build and walk-phase durations apply, heartbeat, plan-tier, build and walk-phase durations
collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
licence add|list|use|key model access, under the name a person calls it licence add|list|use|key model access, under the name a person calls it
licence manager <name> <node> the node that holds a refreshable licence's refresh token licence manager <name> <node> the node that holds a refreshable licence's refresh token
licence refresh <name> mint a new access token and seal it to every holder licence refresh <name> mint a new access token and seal it to every holder
@@ -390,84 +361,6 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
} }
} }
// Built is the daemon hearing a build's outcome on the bus — its own asking, an announcement's, or
// a tool's that did not wait (novox/hq issue 176) — and taking it in: recorded, and the module
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
// **A merge check builds nothing** (novox/hq to-be 45 §9): its verdict is said, and nothing of it is
// recorded or registered.
if result.Check != nil || result.Checked != nil {
if isTry(result) {
// A branch tried before its pull request (novox/hq issue 686): its asker reads the outcome from the
// build's log. Never said as `checked`, so the forge's holder sets no status.
fmt.Printf("%s: a try of %s at %.8s, read with `try --id`: said to nobody, nothing taken in\n", result.ID,
result.Repository, result.Ref)
return nil
}
checked(ctx, result)
return nil
}
// **A dry run is looked at, never taken in** (novox/hq issue 240). On 2026-10-04 a dry run of an
// unmerged branch was heard here like any build, registered, and its definition reached a machine
// before anyone had reviewed it.
if result.DryRun {
fmt.Printf("%s: a dry run of %s on %s, not taken in\n", result.ID, result.Repository, result.Ref)
return nil
}
manifest, _, err := takeIn(ctx, b.inv, result)
// The assignments pending on this build, made or ended (novox/hq issue 325), said in the daemon's log
// after what became of the build.
if b.open != nil {
defer func() {
for _, line := range settlePendingOnBuild(ctx, b.open, result, manifest.Module, err) {
fmt.Printf("%s: %s\n", result.ID, line)
}
}()
}
// When it was asked, so a plan takes as its outcome only a build asked for it or after it
// (novox/hq 04-ISSUES/219). Zero when the id does not say.
asked, _ := link.BuildAskedAt(result.ID)
// And how long it took, asked to heard, which a build's bound will be set from (novox/hq to-be 45
// Phase 0). Said if lost; never a reason not to take the build in.
recordBuildDuration(ctx, b.inv, result, asked)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
if result.Module != "" {
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed, asked, result.ID)
} else {
planFailedBuild(ctx, b.open, result)
}
return nil
case errors.Is(err, inventory.ErrSuperseded):
// Not a failure: the module is already at what a later request built. A plan that asked
// before that later request is answered by it; one that asked after it ignores this.
fmt.Printf("%s: %v\n", result.ID, err)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked, result.ID)
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
if manifest.Module != "" {
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error(), asked, result.ID)
}
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
// What the walk's build did, kept on the walk whose ask it answers (novox/hq ADR 0350 §4); a build seat from
// before it says nothing, and that is what is kept.
made := result.MadeWords()
if result.Reused != "" || result.BuiltWhy != "" {
fmt.Printf("%s: %s %s\n", result.ID, manifest.Module, made)
}
saysWhenThePolicyActs(ctx, b.inv, manifest.Module)
planBuiltSaying(ctx, b.open, manifest.Module, result.Commit, "", asked, result.ID, made)
// A module registered may be one a machine is now behind: `status` is composed again.
statusFrom.nudge()
return nil
}
// verbVar carries the seat verb a command runs for, from the serving controller to the process it starts. // verbVar carries the seat verb a command runs for, from the serving controller to the process it starts.
const verbVar = "MESH_VERB" const verbVar = "MESH_VERB"
@@ -492,10 +385,24 @@ func removedByName(command string) error {
"catalogue does not build is registered at the controller's terminal with `register`", "catalogue does not build is registered at the controller's terminal with `register`",
"rebuild": "the controller asks no build (ADR 0501 §5): a failed key is built again by the builder as a new " + "rebuild": "the controller asks no build (ADR 0501 §5): a failed key is built again by the builder as a new " +
"revision (ADR 0429 §18), and a walk's Retry waits for it", "revision (ADR 0429 §18), and a walk's Retry waits for it",
"replay --register": "a build is registered only from the builder's done key of a trunk fingerprint (ADR 0501 " + "replay": "the controller asks no build, so there is nothing to replay (ADR 0501 §5, ADR 0525 clause 6); a branch is " +
"§1, §2); `replay` without --register still shows what a build of a recorded commit makes", "tried with `try`",
"queue": "the controller keeps no build queue (ADR 0525 clause 6): the builds and checks are asks on the build modules' " +
"streams, and the delivery view shows their backlog and the oldest ask's age",
"cancel": "the controller keeps no build queue (ADR 0525 clause 6): an ask is on a build module's stream, and the delivery " +
"view shows the backlog",
"clear": "the controller keeps no build queue (ADR 0525 clause 6): an ask is on a build module's stream, and the delivery " +
"view shows the backlog",
"kill": "the controller keeps no build queue and starts no build (ADR 0525 clause 6): a build runs on a build module, " +
"which ends it",
"pause": "the build seat is retired (ADR 0525 clause 7): a node's builds are paused with its build module's own pause, " +
"or a capacity of 0",
"resume": "the build seat is retired (ADR 0525 clause 7): a node's builds are resumed with its build module's own " +
"resume, or its capacity",
"forget-checks": "build-agent makes no check build from R4e (ADR 0503 §3), so there is nothing to let go of", "forget-checks": "build-agent makes no check build from R4e (ADR 0503 §3), so there is nothing to let go of",
"delivery-plan": "a delivery's plan is read with `delivery-order` and the checker's verdict (ADR 0503 §3)", "delivery-plan": "a delivery's plan is read with `delivery-order` and the checker's verdict (ADR 0503 §3)",
"delivery-check": "the checker judges every head and every delivery group (ADR 0503 §3, ADR 0527): the controller asks " +
"nothing and says no verdict of a pull request",
}[command] }[command]
return &removedCommand{msg: fmt.Sprintf("`%s` is removed: %s", command, replacement)} return &removedCommand{msg: fmt.Sprintf("`%s` is removed: %s", command, replacement)}
} }
+2 -14
View File
@@ -47,18 +47,6 @@ func readBackFromTheCommand(t *testing.T, argv []string, members string) {
} }
} }
func TestDeliveryCheckHandsALongMembersListInAFile(t *testing.T) { // 1272/AC1
members := aLongGroup(t)
argv, err := argvFor("delivery-check", map[string]any{"group": "feat/x", "members": members})
if err != nil {
t.Fatal(err)
}
if argv[0] != "delivery" || argv[1] != "check" || !slices.Contains(argv, "--group") {
t.Errorf("the command is %.100v", strings.Join(argv, " "))
}
readBackFromTheCommand(t, argv, members)
}
func TestDeliveryOrderHandsALongMembersListInAFile(t *testing.T) { // 1272/AC2 func TestDeliveryOrderHandsALongMembersListInAFile(t *testing.T) { // 1272/AC2
members := aLongGroup(t) members := aLongGroup(t)
argv, err := argvFor("delivery-order", map[string]any{"members": members}) argv, err := argvFor("delivery-order", map[string]any{"members": members})
@@ -73,8 +61,8 @@ func TestDeliveryOrderHandsALongMembersListInAFile(t *testing.T) { // 1272/AC2
func TestAShortMembersListIsStillTheArgumentItWas(t *testing.T) { // 1272/AC3 func TestAShortMembersListIsStillTheArgumentItWas(t *testing.T) { // 1272/AC3
members := `[{"id":"novox/a#1"},{"id":"novox/b#2"}]` members := `[{"id":"novox/a#1"},{"id":"novox/b#2"}]`
argv, err := argvFor("delivery-check", map[string]any{"group": "feat/x", "members": members, "ready": "true"}) argv, err := argvFor("delivery-order", map[string]any{"members": members})
want := []string{"delivery", "check", "--group", "feat/x", "--members", members, "--ready"} want := []string{"delivery", "order", "--members", members}
if err != nil || !slices.Equal(argv, want) { if err != nil || !slices.Equal(argv, want) {
t.Errorf("a short list gives %v, %v, want %v", argv, err, want) t.Errorf("a short list gives %v, %v, want %v", argv, err, want)
} }
@@ -1,72 +0,0 @@
//go:build integration
package main
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// **A repository's merge check runs every module's unit tests, a Go module of its own below the root included**
// (novox/hq ADR 0420 §5, issue 1025): `./...` stops at a nested go.mod. This runs the script
// check-nested-modules.sh in a tree with a `go` that only records where it was run and fails for a module named bad.
func TestTheMergeCheckVetsAndTestsEveryNestedGoModule(t *testing.T) {
script, err := filepath.Abs("../../check-nested-modules.sh")
if err != nil {
t.Fatal(err)
}
bin, tree := t.TempDir(), t.TempDir()
log := filepath.Join(t.TempDir(), "go.log")
fake := "#!/bin/sh\necho \"$(basename \"$PWD\") $1 GOFLAGS=[$GOFLAGS]\" >>\"$LOG\"\n" +
"[ \"$(basename \"$PWD\")\" = bad ] && [ \"$1\" = test ] && exit 1\nexit 0\n"
if err := os.WriteFile(filepath.Join(bin, "go"), []byte(fake), 0o755); err != nil {
t.Fatal(err)
}
run := func() (string, error) {
cmd := exec.Command("sh", script)
cmd.Dir = tree
cmd.Env = append(os.Environ(), "PATH="+bin+":"+os.Getenv("PATH"), "LOG="+log, "GOFLAGS=-mod=vendor")
out, err := cmd.CombinedOutput()
return string(out), err
}
// No modules directory: a pass, nothing run.
if out, err := run(); err != nil {
t.Fatalf("a repository with no nested module failed: %v\n%s", err, out)
}
if _, err := os.Stat(log); err == nil {
t.Fatal("go ran with no nested module")
}
for _, m := range []string{"good", "bad", "other"} {
dir := filepath.Join(tree, "modules", m)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "go.mod"), []byte("module x\n"), 0o644); err != nil {
t.Fatal(err)
}
}
// A directory with no go.mod is no Go module of its own.
if err := os.MkdirAll(filepath.Join(tree, "modules", "typescript"), 0o755); err != nil {
t.Fatal(err)
}
out, err := run()
if err == nil {
t.Fatalf("a nested module whose tests fail passed the check:\n%s", out)
}
raw, _ := os.ReadFile(log)
ran := string(raw)
for _, want := range []string{"good vet GOFLAGS=[]", "good test GOFLAGS=[]", "other test GOFLAGS=[]", "bad test GOFLAGS=[]"} {
if !strings.Contains(ran, want) {
t.Errorf("%q was not run, nor outside the vendor directory:\n%s", want, ran)
}
}
if strings.Contains(ran, "typescript") {
t.Errorf("a directory with no go.mod was run:\n%s", ran)
}
if !strings.Contains(out, "nested module modules/bad FAILED") {
t.Errorf("the failing module is not named:\n%s", out)
}
}
File diff suppressed because it is too large Load Diff
@@ -1,31 +0,0 @@
package main
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
)
func TestTheGateJudgesAFileOfPathsAsItDoesTheSameListOfThem(t *testing.T) { // 1247/AC3
paths := []string{"modules/a/x.go", "modules/a/y.go", "modules/b/z.go", "README.md"}
file := filepath.Join(t.TempDir(), "changed.txt")
if err := os.WriteFile(file, []byte(strings.Join(paths, "\n")+"\n"), 0o644); err != nil {
t.Fatal(err)
}
fromFile, err := changedPaths("", file)
if err != nil {
t.Fatal(err)
}
fromList, err := changedPaths(strings.Join(paths, ","), "")
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(fromFile, fromList) || len(fromFile) != len(paths) {
t.Errorf("the file gives %v, the list %v", fromFile, fromList)
}
if _, err := changedPaths("", filepath.Join(t.TempDir(), "none")); err == nil {
t.Error("a file that is not there is read as no paths")
}
}
@@ -1,68 +0,0 @@
//go:build integration
package main
import (
"os"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func TestIssue969TheGateRaisesEachMachineAsItsEngineStatedItself(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
said := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
for node, contract := range map[string]int{"anchor": link.HandedContract, "laptop": link.HandedContract - 1} {
if _, err := open.inventory.RecordHealth(ctx, inventory.NodeHealth{Node: node, Contract: contract,
SaidAt: said, HeardAt: said}); err != nil {
t.Fatal(err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
if len(f.Machines) != 2 {
t.Fatalf("the snapshot holds %d machines", len(f.Machines))
}
want := map[string]bool{}
for _, m := range f.Machines {
if m.Contract == 0 {
t.Fatalf("%s: the snapshot carries no contract", m.Name)
}
want[m.Name] = m.Contract >= link.HandedContract
}
handing := 0
for _, hands := range want {
if hands {
handing++
}
}
if handing != 1 {
t.Fatalf("one machine takes handed files in the snapshot, got %v", want)
}
raised, drop, err := throwawayStores(ctx, os.Getenv("MESH_TEST_POSTGRES"))
if err != nil {
t.Fatal(err)
}
defer drop()
if _, err := raiseFromFacts(ctx, raised, f, map[string]catalogue.Manifest{}, map[string]inventory.Source{}); err != nil {
t.Fatal(err)
}
for name, hands := range want {
in, err := readComposeInput(ctx, raised)
if err != nil {
t.Fatal(err)
}
node, _ := nodeOf(in.Input, name)
if got := node.Stated && node.Contract >= compose.HandedContract; got != hands {
t.Errorf("%s: the gate raises it as taking handed files = %v (%v), the mesh says %v", name, got, err, hands)
}
}
}
@@ -1,33 +0,0 @@
package main
import (
"testing"
"time"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/link"
)
// Issue 969: the gate composes a machine as the mesh does, so a moved module is handed where the machine takes
// handed files and composed the old way where it does not.
func TestIssue969TheGateGivesAMachineItsContractSoItTakesHandedFiles(t *testing.T) {
taken := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
stated, ok := standInHealth(snapshot.Machine{Name: "f35", Contract: link.HandedContract}, taken)
if !ok || stated.Contract < link.HandedContract {
t.Fatalf("a machine whose engine takes handed files must be given that in the gate: %+v, %v", stated, ok)
}
old, ok := standInHealth(snapshot.Machine{Name: "e12", Contract: link.HandedContract - 1}, taken)
if !ok || old.Contract >= link.HandedContract {
t.Fatalf("a machine whose engine is older must not take handed files: %+v, %v", old, ok)
}
none, ok := standInHealth(snapshot.Machine{Name: "d01"}, taken)
if ok {
t.Fatalf("a snapshot that carries no contract states nothing: %+v, %v", none, ok)
}
}
// Issue 969, through the store: a machine whose engine stated the handed contract is in the snapshot with it, the
// gate raises it as taking handed files, and one whose statement is below it is raised as not taking them. Run in
// the merge check with the other tests that need a store; the composition itself, handed here and refused there, is
// the catalogue's TestAHandedModuleNamingARunTimeBoundKeyComposesOnlyWhereItIsHanded.
-135
View File
@@ -1,135 +0,0 @@
package main
import (
"encoding/json"
"fmt"
"os"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
)
// Config values held for the keys an open pull request introduces (novox/hq ADR 0502 clause 4, issue 1826): the gate
// lays each approved one over its module's values on its node in the composition with the change, and in that one
// only. In the mesh as it is the key is not declared, so the value would be refused there.
// heldValue is one approved held value as the build agent writes it (--held): the value of Key for Module on Node,
// a machine's real name, which the snapshot holds only as a pseudonym.
type heldValue struct {
Pull string `json:"pull"`
Head string `json:"head,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
Key string `json:"key"`
Value json.RawMessage `json:"value"`
Approved string `json:"approved,omitempty"`
By string `json:"by,omitempty"`
At string `json:"at,omitempty"`
}
// readHeld reads the file --held names; none named, none held.
func readHeld(path string) ([]heldValue, error) {
if path == "" {
return nil, nil
}
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var held []heldValue
if err := json.Unmarshal(raw, &held); err != nil {
return nil, fmt.Errorf("the held values are not readable: %w", err)
}
for _, h := range held {
if h.Node == "" || h.Module == "" || h.Key == "" || len(h.Value) == 0 {
return nil, fmt.Errorf("a held value of %s names no node, module, key or value", h.Pull)
}
}
return held, nil
}
// layHeld is the snapshot with each held value laid as the value of its key in its node's layer of its module, and what
// it said of them. The snapshot given is not changed. **A held value for a machine the snapshot does not hold, or a
// module the change does not define, is said and not laid**, never dropped silently; one for a module not assigned to
// its machine is laid and said. The notes name machines as the snapshot does (roles and pseudonym), not by their names.
func layHeld(f snapshot.Facts, held []heldValue, shelf map[string]catalogue.Manifest) (snapshot.Facts, []string) {
if len(held) == 0 {
return f, nil
}
out := f
out.Machines = make([]snapshot.Machine, len(f.Machines))
copy(out.Machines, f.Machines)
var notes, laid []string
pulls := map[string]bool{}
for _, h := range held {
at := -1
pseudonym := snapshot.Pseudonym("machine", h.Node)
for i, m := range out.Machines {
if m.Name == pseudonym {
at = i
}
}
if at < 0 {
notes = append(notes, fmt.Sprintf("the value held for %s of %s (%s=%s) is not composed: the snapshot holds no machine %s",
h.Module, h.Pull, h.Key, h.Value, pseudonym))
continue
}
if _, defined := shelf[h.Module]; !defined {
notes = append(notes, fmt.Sprintf("the value held for %s of %s (%s=%s) is not composed: the change defines no module %s",
h.Module, h.Pull, h.Key, h.Value, h.Module))
continue
}
var value any
if err := json.Unmarshal(h.Value, &value); err != nil {
notes = append(notes, fmt.Sprintf("the value held for %s of %s (%s) is not composed: it is not a value: %v",
h.Module, h.Pull, h.Key, err))
continue
}
m := &out.Machines[at]
layers := make([]snapshot.Settings, len(m.Settings))
copy(layers, m.Settings)
found := false
for i := range layers {
if layers[i].Module == h.Module {
values := make(map[string]any, len(layers[i].Values)+1)
for k, v := range layers[i].Values {
values[k] = v
}
values[h.Key] = value
layers[i].Values = values
found = true
}
}
if !found {
layers = append(layers, snapshot.Settings{Module: h.Module, Values: map[string]any{h.Key: value}})
sort.Slice(layers, func(a, b int) bool { return layers[a].Module < layers[b].Module })
}
m.Settings = layers
assigned := false
for _, a := range m.Assigned {
assigned = assigned || a == h.Module
}
by := ""
if h.Approved != "" {
by = ", approved by " + h.Approved
}
line := fmt.Sprintf("%s on %s %s=%s%s", h.Module, m.Described(), h.Key, h.Value, by)
if !assigned {
line += " (not assigned there in the snapshot)"
}
laid = append(laid, line)
pulls[h.Pull] = true
}
if len(laid) > 0 {
var names []string
for p := range pulls {
names = append(names, p)
}
sort.Strings(names)
notes = append([]string{fmt.Sprintf("composed with %d value(s) held for %s: %s", len(laid), strings.Join(names, ", "),
strings.Join(laid, "; "))}, notes...)
}
return out, notes
}
@@ -1,85 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
)
func heldGateFacts() snapshot.Facts {
return snapshot.Facts{Machines: []snapshot.Machine{
{Name: snapshot.Pseudonym("machine", "ace"), Roles: []string{"the build node"}, Assigned: []string{"n8n"},
Settings: []snapshot.Settings{{Module: "n8n", Values: map[string]any{"port": 5678.0}}}},
{Name: snapshot.Pseudonym("machine", "g14")},
}}
}
var heldShelf = map[string]catalogue.Manifest{"n8n": {Module: "n8n"}}
// 1826/AC3: a held value is laid in its node's layer of its module, beside what the layer holds, and said.
func TestAHeldValueIsLaidOverItsNodesLayerAndSaid(t *testing.T) {
f := heldGateFacts()
got, notes := layHeld(f, []heldValue{{Pull: "novox/mesh-catalog#434", Node: "ace", Module: "n8n", Key: "media-gid",
Value: []byte("2000"), Approved: "the operator"}}, heldShelf)
layer := got.Machines[0].Settings[0].Values
if layer["media-gid"] != 2000.0 || layer["port"] != 5678.0 {
t.Fatalf("the layer holds %v", layer)
}
if _, has := f.Machines[0].Settings[0].Values["media-gid"]; has {
t.Fatal("the snapshot given was changed")
}
if len(notes) != 1 || !strings.Contains(notes[0], "composed with 1 value(s) held for novox/mesh-catalog#434") ||
!strings.Contains(notes[0], "media-gid=2000") || !strings.Contains(notes[0], "approved by the operator") ||
strings.Contains(notes[0], "ace") {
t.Fatalf("said %q", notes)
}
}
// 1826/AC3: a machine with no layer of the module gets one.
func TestAHeldValueMakesTheLayerAMachineLacks(t *testing.T) {
got, _ := layHeld(heldGateFacts(), []heldValue{{Pull: "p", Node: "g14", Module: "n8n", Key: "k", Value: []byte(`"x"`)}}, heldShelf)
s := got.Machines[1].Settings
if len(s) != 1 || s[0].Module != "n8n" || s[0].Values["k"] != "x" {
t.Fatalf("layers %+v", s)
}
}
// 1826/AC3: a held value for a machine or a module the snapshot or the change does not hold is said, never laid.
func TestAHeldValueForWhatIsNotHeldIsSaidNotLaid(t *testing.T) {
f := heldGateFacts()
got, notes := layHeld(f, []heldValue{
{Pull: "p", Node: "nowhere", Module: "n8n", Key: "k", Value: []byte("1")},
{Pull: "p", Node: "ace", Module: "ghost", Key: "k", Value: []byte("1")},
}, heldShelf)
if len(notes) != 2 || !strings.Contains(notes[0], "holds no machine") || !strings.Contains(notes[1], "defines no module ghost") {
t.Fatalf("said %q", notes)
}
if _, has := got.Machines[0].Settings[0].Values["k"]; has {
t.Fatal("laid")
}
}
// 1826/AC3: none held lays nothing and says nothing; the file is read and refuses a value without a node.
func TestNoHeldValuesLayNothingAndTheFileIsRead(t *testing.T) {
if got, notes := layHeld(heldGateFacts(), nil, heldShelf); len(notes) != 0 || len(got.Machines) != 2 {
t.Fatalf("%v %v", got, notes)
}
dir := t.TempDir()
good := filepath.Join(dir, "good.json")
os.WriteFile(good, []byte(`[{"pull":"p","node":"ace","module":"n8n","key":"k","value":1}]`), 0o644)
if h, err := readHeld(good); err != nil || len(h) != 1 || h[0].Key != "k" {
t.Fatalf("%v %v", h, err)
}
bad := filepath.Join(dir, "bad.json")
os.WriteFile(bad, []byte(`[{"pull":"p","module":"n8n","key":"k","value":1}]`), 0o644)
if _, err := readHeld(bad); err == nil {
t.Fatal("a value without a node was read")
}
if h, err := readHeld(""); h != nil || err != nil {
t.Fatalf("%v %v", h, err)
}
}
@@ -1,582 +0,0 @@
//go:build integration
package main
import (
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
)
func catalogueMesh(t *testing.T) (snapshot.Facts, map[string]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
manifests := map[string]string{
"objects": `{"module":"objects","version":"1",
"provides":[{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"}}],
"receives":{"s3-bucket":"/var/lib/mesh/objects/mesh.json"}}`,
"resolver": `{"module":"resolver","version":"1",
"provides":[{"name":"wildcard-resolution","scope":"mesh","identity":false}]}`,
"networkmanager": `{"module":"networkmanager","version":"1","requires":["wildcard-resolution"]}`,
"album": `{"module":"album","version":"1","requires":["s3-bucket"]}`,
"lemurs": `{"module":"lemurs","version":"1","capabilities":["systemd"],
"resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`,
}
for name, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/" + name, BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
}
for _, a := range [][2]string{{"anchor", "objects"}, {"anchor", "resolver"}, {"laptop", "networkmanager"},
{"laptop", "album"}, {"laptop", "lemurs"}} {
if _, err := assign(ctx, open, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
return f, manifests
}
func gateJudged(t *testing.T, f snapshot.Facts, tree string, changed ...string) mergeVerdict {
t.Helper()
admin := os.Getenv("MESH_TEST_POSTGRES")
in := mergeCheckInput{facts: f, admin: admin, changed: changed}
if tree != "" {
in.repository, in.tree = "novox/mesh-catalog", tree
}
v, err := judgeChange(t.Context(), in)
if err != nil {
t.Fatal(err)
}
return v
}
func TestTheMeshAsItIsPassesTheGate(t *testing.T) {
f, manifests := catalogueMesh(t)
for _, m := range f.Machines {
if !m.Declaration.Composes {
t.Fatalf("the mesh itself does not compose: %+v", m.Declaration)
}
}
v := gateJudged(t, f, aTree(t, manifests))
if v.Verdict != "pass" {
t.Fatalf("an unchanged catalogue does not pass:\n%s", v.Report())
}
for _, m := range v.Machines {
if !m.Base.Composes || !m.Change.Composes {
t.Errorf("%s does not compose in the gate's store as it does on the mesh: %+v / %+v", m.Described, m.Base, m.Change)
}
}
}
func TestIssue263AnIdentityARealMachineNameOverflowsFailsThePullRequest(t *testing.T) {
f, manifests := catalogueMesh(t)
tree := aTree(t, withEdit(manifests, "networkmanager",
`{"module":"networkmanager","version":"1","requires":["wildcard-resolution","s3-bucket"]}`))
v := gateJudged(t, f, tree)
if v.Verdict != "fail" {
t.Fatalf("the overflow passed the gate:\n%s", v.Report())
}
report := v.Report()
if !strings.Contains(report, "networkmanager") || !strings.Contains(report, "s3-bucket") {
t.Errorf("the refusal does not name the module and the provision:\n%s", report)
}
}
func TestIssue236AManifestTheNodeEngineRefusesFailsThePullRequest(t *testing.T) {
f, manifests := catalogueMesh(t)
broken := `{"module":"lemurs","version":"1","capabilities":["systemd"],
"resources":[{"id":"service","type":"service","unit":"lemurs.service","boot":"enabled"}]}`
v := gateJudged(t, f, aTree(t, withEdit(manifests, "lemurs", broken)))
if v.Verdict != "fail" || !strings.Contains(v.Report(), "lemurs") {
t.Fatalf("a service the node-engine refuses passed the gate:\n%s", v.Report())
}
laptop := snapshot.Pseudonym("machine", "laptop")
if !strings.Contains(v.Report(), laptop) {
t.Errorf("the refusal does not name the machine it would be refused on:\n%s", v.Report())
}
// And as a new module, which no machine runs: tried on one that could.
newcomer := strings.ReplaceAll(broken, `"lemurs"`, `"greeter"`)
newcomer = strings.ReplaceAll(newcomer, "lemurs.service", "greeter.service")
v = gateJudged(t, f, aTree(t, withEdit(manifests, "greeter", newcomer)))
if v.Verdict != "fail" || !strings.Contains(v.Report(), "greeter, assigned to") {
t.Fatalf("a new module the node-engine would refuse passed the gate:\n%s", v.Report())
}
}
func TestAManifestTheRunningControllerCannotReadFailsThePullRequest(t *testing.T) {
f, manifests := catalogueMesh(t)
v := gateJudged(t, f, aTree(t, withEdit(manifests, "album",
`{"module":"album","version":"1","requires":["s3-bucket"],"a-field-of-a-newer-controller":true}`)))
if v.Verdict != "fail" || !strings.Contains(v.Report(), "refuses it") {
t.Fatalf("a manifest this controller cannot read passed:\n%s", v.Report())
}
}
func TestAModuleAMachineRunsRemovedFromItsSourceFails(t *testing.T) {
f, manifests := catalogueMesh(t)
v := gateJudged(t, f, aTree(t, withEdit(manifests, "album", "")))
if v.Verdict != "fail" || !strings.Contains(v.Report(), "album is removed") {
t.Fatalf("removing a module a machine runs passed:\n%s", v.Report())
}
}
func TestIssue278AWideRebuildIsSaidBeforeTheMerge(t *testing.T) {
f, manifests := catalogueMesh(t)
was := wideRebuild
wideRebuild = 2
t.Cleanup(func() { wideRebuild = was })
for _, file := range []string{"modules/showcase/index.ts", "merge-check.sh", "README.md"} {
v := gateJudged(t, f, aTree(t, manifests), file)
if v.Width == nil || len(v.Width.Modules) != 0 || len(v.Width.Unread) != 1 || v.Verdict != "pass" {
t.Fatalf("%s, read by no build, reads %+v, %s", file, v.Width, v.Verdict)
}
if !strings.Contains(v.Report(), "read by no module's build") {
t.Errorf("why %s rebuilds nothing is not said:\n%s", file, v.Report())
}
}
v := gateJudged(t, f, aTree(t, manifests), "modules/album/x.ts", "modules/objects/x.go", "modules/resolver/x.go")
if v.Width == nil || len(v.Width.Modules) < 3 || v.Verdict != "warning" {
t.Fatalf("a rebuild wider than the bound is not warned of: %+v, %s", v.Width, v.Verdict)
}
// With the reference module's definition in the tree, its directory is a module, held or not.
withShowcase := withEdit(manifests, "showcase", `{"module":"showcase","version":"1"}`)
v = gateJudged(t, f, aTree(t, withShowcase), "modules/showcase/index.ts")
if v.Width == nil || len(v.Width.Modules) != 0 || len(v.Width.Unread) != 0 {
t.Fatalf("a file of a module nobody holds reads %+v", v.Width)
}
v = gateJudged(t, f, aTree(t, manifests), "modules/album/module.json")
if v.Width == nil || strings.Join(v.Width.Modules, ",") != "album" || v.Verdict != "pass" {
t.Fatalf("a change to one module's directory reads %+v, %s", v.Width, v.Verdict)
}
}
func TestADeliveryGroupsHeadsAreComposedTogether(t *testing.T) {
f, manifests := catalogueMeshWithAnApp(t)
needsTheApp := withEdit(manifests, "album", `{"module":"album","version":"1","requires":["s3-bucket","app-api"]}`)
catTree := aTree(t, needsTheApp)
alone := gateJudged(t, f, catTree, "modules/album/module.json")
if alone.Verdict != "fail" {
t.Fatalf("a requirement nothing provides passed alone:\n%s", alone.Report())
}
app := t.TempDir()
if err := os.WriteFile(filepath.Join(app, "module.json"), []byte(`{"module":"app","version":"1",
"provides":[{"name":"app-api","scope":"mesh","identity":false}]}`), 0o644); err != nil {
t.Fatal(err)
}
in := mergeCheckInput{facts: f, admin: os.Getenv("MESH_TEST_POSTGRES"), repository: "novox/mesh-catalog",
tree: catTree, changed: []string{"modules/album/module.json"},
group: []groupChange{{Repository: "novox/app", Tree: app, Changed: []string{"module.json"}}}}
together, err := judgeChange(t.Context(), in)
if err != nil {
t.Fatal(err)
}
if together.Verdict == "fail" {
t.Fatalf("the group composed together fails:\n%s", together.Report())
}
if together.Modules["app"] != "changed" || together.Modules["album"] != "changed" {
t.Fatalf("the group's heads were not both laid over the mesh: %v", together.Modules)
}
}
func catalogueMeshWithAnApp(t *testing.T) (snapshot.Facts, map[string]string) {
t.Helper()
open := aMesh(t)
ctx := t.Context()
manifests := map[string]string{
"objects": `{"module":"objects","version":"1",
"provides":[{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"}}],
"receives":{"s3-bucket":"/var/lib/mesh/objects/mesh.json"}}`,
"album": `{"module":"album","version":"1","requires":["s3-bucket"]}`,
}
for name, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("%s: %v", name, err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/" + name, BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
}
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "app", Version: "1"},
inventory.Source{Repository: "novox/app", BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
for _, a := range [][2]string{{"anchor", "objects"}, {"anchor", "app"}, {"laptop", "album"}} {
if _, err := assign(ctx, open, a[0], a[1]); err != nil {
t.Fatalf("assign %s %s: %v", a[0], a[1], err)
}
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
return f, manifests
}
func busMesh(t *testing.T) snapshot.Facts {
t.Helper()
open := aMesh(t)
ctx := t.Context()
m, err := catalogue.ParseManifest([]byte(`{"module":"speaker","version":"1",
"own-secrets":{"broker":"/var/lib/speaker/broker"}}`))
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/speaker", BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "speaker"); err != nil {
t.Fatal(err)
}
user := broker.Principal{Kind: broker.KindModule, Node: "laptop", Module: "speaker"}.Username()
password, err := open.inventory.MintBusPassword(ctx, inventory.BusUser{Username: user, Kind: inventory.BusModule,
Node: "laptop", Module: "speaker"})
if err != nil {
t.Fatal(err)
}
if err := open.inventory.AcceptSecretForModule(ctx, "laptop", "speaker", "broker",
`{"user":"`+user+`","password":"`+password+`"}`); err != nil {
t.Fatal(err)
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
for _, mc := range f.Machines {
if !mc.Declaration.Composes {
t.Fatalf("the mesh itself does not compose: %+v", mc.Declaration)
}
}
return f
}
func TestIssue285AModuleOnTheBusComposesInTheGate(t *testing.T) {
f := busMesh(t)
v := gateJudged(t, f, "")
if v.Verdict != "pass" {
t.Fatalf("the mesh as it is does not pass:\n%s", v.Report())
}
for _, m := range v.Machines {
if !m.Base.Composes {
t.Errorf("%s composes on the mesh and not in the gate: %v", m.Described, m.Base.Problems)
}
}
if !strings.Contains(v.Summary, "2 of 2 compose") {
t.Errorf("the summary does not say every machine composes: %s", v.Summary)
}
}
func TestIssue285AMachineTheGateCannotRaiseIsAnErrorNeverAPass(t *testing.T) {
f := busMesh(t)
for i := range f.Machines {
// A fact the snapshot does not carry: the module's credential, gone from the laptop's.
var kept []snapshot.Accepted
for _, a := range f.Machines[i].Accepted {
if a.Name != "broker" {
kept = append(kept, a)
}
}
f.Machines[i].Accepted = kept
}
v := gateJudged(t, f, "")
if v.Verdict != "error" {
t.Fatalf("a gate whose mesh does not compose said %s:\n%s", v.Verdict, v.Report())
}
if len(v.Errors) != 1 || !strings.Contains(v.Errors[0], "speaker") {
t.Errorf("the error does not name what could not be raised: %v", v.Errors)
}
}
func TestAGroupsMoveOfAModuleIsNeitherTwoDefinitionsNorARemoval(t *testing.T) {
const catalogueRepo, controllerRepo = "novox/mesh-catalog", "novox/mesh-controller"
built := func(repo, path string) inventory.Source {
return inventory.Source{Repository: "http://forge.internal:20000/" + repo + ".git", Path: path, BuiltFrom: "c0ffee"}
}
sources := map[string]inventory.Source{"mover": built(catalogueRepo, "modules/mover")}
raw := `{"module":"mover","version":"1"}`
base := map[string]catalogue.Manifest{}
m, err := catalogue.ParseManifest([]byte(raw))
if err != nil {
t.Fatal(err)
}
base["mover"] = m
facts := snapshot.Facts{Machines: []snapshot.Machine{{Name: "anchor", Assigned: []string{"mover"}}}}
catalogueWithout := aTree(t, map[string]string{"other": `{"module":"other","version":"1"}`})
controllerWith := func() string {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "modules", "mover"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "modules", "mover", "module.json"), []byte(raw), 0o644); err != nil {
t.Fatal(err)
}
return dir
}()
removes := groupChange{Repository: catalogueRepo, Tree: catalogueWithout, Changed: []string{"modules/mover/module.json"}}
defines := groupChange{Repository: controllerRepo, Tree: controllerWith, Changed: []string{"modules/mover/module.json"}}
lay := func(heads ...groupChange) (map[string]catalogue.Manifest, []string, map[string]string) {
t.Helper()
moves, err := movesInGroup(heads, sources)
if err != nil {
t.Fatal(err)
}
at := sources
if len(moves) > 0 {
at = map[string]inventory.Source{}
for k, v := range sources {
at[k] = v
}
for name, mv := range moves {
o := at[name]
o.Repository, o.Path = mv.To, mv.Dir
at[name] = o
}
}
shelf, said := base, map[string]string{}
var failures []string
for _, h := range heads {
var f []string
if shelf, f, err = shelfWithChange(shelf, at, facts, h.Repository, h.Tree, said, nil); err != nil {
t.Fatal(err)
}
failures = append(failures, f...)
}
for name := range moves {
said[name] = "moved"
}
return shelf, failures, said
}
for _, order := range [][]groupChange{{removes, defines}, {defines, removes}} {
shelf, failures, said := lay(order...)
if len(failures) > 0 {
t.Fatalf("a move of a module machines run, laid %s first, was refused: %v", order[0].Repository, failures)
}
if said["mover"] != "moved" {
t.Errorf("the move reads %v, not moved", said)
}
if _, held := shelf["mover"]; !held {
t.Errorf("the moved module is not on the shelf after laying %s first", order[0].Repository)
}
}
// Alone, a head is still refused: a second definition (issue 239), and a removal of what runs (ADR 0236).
if _, failures, _ := lay(defines); len(failures) != 1 || !strings.Contains(failures[0], "issue 239") {
t.Errorf("a definition of a module another repository builds, alone: %v", failures)
}
if _, failures, _ := lay(removes); len(failures) != 1 || !strings.Contains(failures[0], "ADR 0236") {
t.Errorf("the removal of a module machines run, alone: %v", failures)
}
}
func TestADeliveryGroupMovingAModuleMachinesRunPassesTheGate(t *testing.T) {
f, manifests := catalogueMeshWithAnApp(t)
catTree := aTree(t, withEdit(manifests, "album", ""))
app := t.TempDir()
// The app module is built from this repository's root, and stays there: only album moves.
if err := os.WriteFile(filepath.Join(app, "module.json"), []byte(`{"module":"app","version":"1"}`), 0o644); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Join(app, "modules", "album"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(app, "modules", "album", "module.json"), []byte(manifests["album"]), 0o644); err != nil {
t.Fatal(err)
}
removes := groupChange{Repository: "novox/mesh-catalog", Tree: catTree, Changed: []string{"modules/album/module.json"}}
defines := groupChange{Repository: "novox/app", Tree: app, Changed: []string{"modules/album/module.json"}}
judge := func(first groupChange, rest ...groupChange) mergeVerdict {
v, err := judgeChange(t.Context(), mergeCheckInput{facts: f, admin: os.Getenv("MESH_TEST_POSTGRES"),
repository: first.Repository, tree: first.Tree, changed: first.Changed, group: rest})
if err != nil {
t.Fatal(err)
}
return v
}
for _, v := range []mergeVerdict{judge(removes, defines), judge(defines, removes)} {
if v.Verdict == "fail" || v.Modules["album"] != "moved" {
t.Fatalf("a group moving a module machines run: %v\n%s", v.Modules, v.Report())
}
}
if v := judge(defines); v.Verdict != "fail" {
t.Errorf("a definition of a module another repository builds passed alone:\n%s", v.Report())
}
if v := judge(removes); v.Verdict != "fail" {
t.Errorf("the removal of a module machines run passed alone:\n%s", v.Report())
}
}
func TestTheGateRefusesAnArtifactContextNamingAnotherRepository(t *testing.T) {
withContext := func(repo string) string {
return `{"module":"proxy","version":"1","build":{"artifacts":[{"name":"server","kind":"image","from":"Dockerfile",` +
`"context":{"seat":"git","repository":"` + repo + `","ref":"main"}}]}}`
}
judge := func(repo string) []string {
t.Helper()
tree := aTree(t, map[string]string{"proxy": withContext(repo)})
_, failures, err := shelfWithChange(map[string]catalogue.Manifest{}, map[string]inventory.Source{},
snapshot.Facts{}, "novox/mesh-controller", tree, map[string]string{}, nil)
if err != nil {
t.Fatal(err)
}
return failures
}
const want = "proxy builds artifact server from novox/mesh-catalog: a module lives whole in one repository (novox/hq ADR 0420 §1)"
failures := judge("novox/mesh-catalog")
if len(failures) != 1 || !strings.Contains(failures[0], want) {
t.Errorf("a context naming another repository: %v, want it to say %q", failures, want)
}
for _, f := range judge("novox/mesh-controller") {
if strings.Contains(f, "ADR 0420 §1") {
t.Errorf("a context naming the module's own repository was refused: %s", f)
}
}
}
func TestAMemberJudgedAloneSeesAMoveThroughTheClonesBeside(t *testing.T) {
const catalogueRepo, controllerRepo = "novox/mesh-catalog", "novox/mesh-controller"
raw := `{"module":"mover","version":"1"}`
sources := map[string]inventory.Source{"mover": {Repository: "http://forge.internal:20000/" + catalogueRepo + ".git",
Path: "modules/mover", BuiltFrom: "c0ffee"}}
// A tree is written into dir: defining mover, or holding only another module.
write := func(dir string, defines bool) string {
name, body := "other", `{"module":"other","version":"1"}`
if defines {
name, body = "mover", raw
}
if err := os.MkdirAll(filepath.Join(dir, "modules", name), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "modules", name, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return dir
}
defining := func() string { return write(t.TempDir(), true) }
without := func() string { return write(t.TempDir(), false) }
// beside lays out a MESH_CHECK_BESIDE directory holding the named repositories' trees, each defining mover or not.
beside := func(trees map[string]bool) string {
root := t.TempDir()
for name, defines := range trees {
write(filepath.Join(root, name), defines)
}
return root
}
moves := func(repository, tree, besideRoot string) map[string]moveOfModule {
t.Helper()
got, err := movesOfChange(mergeCheckInput{repository: repository, tree: tree, beside: besideRoot}, sources)
if err != nil {
t.Fatal(err)
}
return got
}
// The controller's member: defines mover; the catalogue beside has removed it.
if got := moves(controllerRepo, defining(), beside(map[string]bool{"mesh-catalog": false})); got["mover"].To != controllerRepo {
t.Errorf("the controller's change defining mover, the catalogue beside without it: moves %v", got)
}
// The sources the gate then judges with: built from the controller, the definition is not a second one (issue 239);
// unmoved, it still is.
judge := func(at map[string]inventory.Source) []string {
t.Helper()
_, failures, err := shelfWithChange(map[string]catalogue.Manifest{}, at, snapshot.Facts{}, controllerRepo, defining(),
map[string]string{}, nil)
if err != nil {
t.Fatal(err)
}
return failures
}
mv := moves(controllerRepo, defining(), beside(map[string]bool{"mesh-catalog": false}))["mover"]
moved := map[string]inventory.Source{"mover": {Repository: mv.To, Path: mv.Dir, BuiltFrom: "c0ffee"}}
if f := judge(moved); len(f) != 0 {
t.Errorf("a definition of a module the catalogue beside removed was refused: %v", f)
}
if f := judge(sources); len(f) != 1 || !strings.Contains(f[0], "issue 239") {
t.Errorf("with the catalogue beside at main, the definition: %v, want the issue 239 refusal", f)
}
if got := moves(controllerRepo, defining(), beside(map[string]bool{"mesh-catalog": true})); len(got) != 0 {
t.Errorf("the catalogue beside at main still defines mover, yet moves %v", got)
}
// The catalogue's member: removes mover; the controller beside defines it.
if got := moves(catalogueRepo, without(), beside(map[string]bool{"mesh-controller": true})); got["mover"].To != controllerRepo {
t.Errorf("the catalogue's change removing mover, the controller beside defining it: moves %v", got)
}
if got := moves(catalogueRepo, without(), beside(map[string]bool{"mesh-controller": false})); len(got) != 0 {
t.Errorf("the controller beside at main does not define mover, yet moves %v", got)
}
if got := moves(catalogueRepo, without(), ""); len(got) != 0 {
t.Errorf("nothing beside, yet moves %v", got)
}
}
// 1826/AC3 (integration level, a throwaway store): a change whose container says ${config:media-gid} fails the gate for
// want of the value, passes it with the value the operator approved held for its pull request, and the mesh as it is
// composes the same either way.
func TestAChangeNeedingAHeldValueFailsWithoutItAndPassesWithIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
trunk := `{"module":"n8n","version":"1","resources":[{"id":"server","type":"container","name":"n8n",
"image":"n8nio/n8n@sha256:4846eb2f4b874ab04cde7fc1e249d2ddaec66e9aea64439beb2972cfea88e3c0"}]}`
m, err := catalogue.ParseManifest([]byte(trunk))
if err != nil {
t.Fatal(err)
}
if err := open.inventory.RegisterModule(ctx, m, inventory.Source{Repository: "novox/mesh-catalog",
Path: "modules/n8n", BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "laptop", "n8n"); err != nil {
t.Fatal(err)
}
f, err := gatherFacts(ctx, open, "2.11.17")
if err != nil {
t.Fatal(err)
}
for i := range f.Machines {
f.Machines[i].Capabilities = append(f.Machines[i].Capabilities, snapshot.Capability{Name: "container-groups", Present: true})
}
tree := aTree(t, map[string]string{"n8n": n8nWithAGroup})
judge := func(held []heldValue) mergeVerdict {
v, err := judgeChange(t.Context(), mergeCheckInput{facts: f, admin: os.Getenv("MESH_TEST_POSTGRES"),
repository: "novox/mesh-catalog", tree: tree, changed: []string{"modules/n8n/module.json"}, held: held})
if err != nil {
t.Fatal(err)
}
return v
}
without := judge(nil)
if without.Verdict != "fail" || !strings.Contains(without.Report(), "media-gid") {
t.Fatalf("a required value nothing sets passed, or failed for another reason:\n%s", without.Report())
}
with := judge([]heldValue{{Pull: "novox/mesh-catalog#434", Node: "laptop", Module: "n8n", Key: "media-gid",
Value: []byte("2000"), Approved: "the operator"}})
if with.Verdict == "fail" || with.Verdict == "error" {
t.Fatalf("the held value did not make it compose:\n%s", with.Report())
}
if !strings.Contains(with.Report(), "composed with 1 value(s) held for novox/mesh-catalog#434") {
t.Fatalf("the value laid is not said:\n%s", with.Report())
}
for i := range with.Machines {
if with.Machines[i].Base.Composes != without.Machines[i].Base.Composes {
t.Fatalf("the mesh as it is composes differently with a held value: %+v / %+v", with.Machines[i].Base, without.Machines[i].Base)
}
}
}
-167
View File
@@ -1,167 +0,0 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// The merge gate (novox/hq to-be 45 §9): a change judged against every machine of the snapshot, by the
// incidents it is written from. Each case raises a mesh, takes its snapshot, and judges a pull request's
// tree against it in throwaway stores of its own.
// catalogueMesh is two machines and a catalogue repository: a resolver and an object store on the
// anchor, and on the laptop a network manager requiring the resolver, an album requiring the object
// store, and a login manager. Every module is registered from novox/mesh-catalog, as the mesh's are.
// aTree is a checkout of the catalogue repository holding these manifests.
func aTree(t *testing.T, manifests map[string]string) string {
t.Helper()
dir := t.TempDir()
for name, raw := range manifests {
at := filepath.Join(dir, "modules", name)
if err := os.MkdirAll(at, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(at, "module.json"), []byte(raw), 0o644); err != nil {
t.Fatal(err)
}
}
return dir
}
func withEdit(manifests map[string]string, name, raw string) map[string]string {
out := map[string]string{}
for k, v := range manifests {
out[k] = v
}
if raw == "" {
delete(out, name)
} else {
out[name] = raw
}
return out
}
// The mesh as it is passes: every machine composes in the gate's store as the controller composed it.
// **Issue 263**: a change makes the network manager require the object store's provision; on a machine
// whose name is six characters its identity is 26 against a bound of 20. Refused in the pull request,
// naming the module, and not on the anchor after it merged.
// **Issue 236**: a login manager's service that omits its state passed the catalogue check and was
// refused whole by the node-engine on the first machine. A change to a module a machine runs, and a
// module nobody runs yet, are both refused before merge, naming the machine and the module.
// **Version skew**: a manifest the controller judging it cannot read — the one the mesh runs, for a
// catalogue change — fails here, not at registration after the merge.
// A module a machine runs, removed from its source, fails until it is unassigned (ADR 0236).
// **Issues 278 and 280**: a file in no held module's directory read as shared code, and a merge rebuilt the
// catalogue. A file is a module's only by being inside it — no build reads one outside — so it rebuilds
// nothing, and the gate says why; a merge that does rebuild widely is warned of.
// **A delivery group is judged as one future state** (novox/hq ADR 0239): a catalogue change that needs a
// provision only another repository's change adds fails alone and passes composed with it; and a group
// whose other head breaks what the first needs fails, naming it.
// catalogueMeshWithAnApp is catalogueMesh with an application built from a repository of its own, at its
// root, on the anchor.
// busMesh is aMesh with a module on the bus on the laptop, its account issued as `module issue` issues
// one: minted, and its credential held as the module's own secret named broker.
// **Issue 285**: every machine running a module on the bus failed to compose in the gate's store, with
// the change and without — the store held the module's credential and no account for it, which
// composition refuses (issue 203) — and the gate passed every change, "0 of 4 compose". The account the
// mesh issued is raised with its credential, so the machine composes in the gate as on the mesh.
// **Issue 285**: a machine the mesh composes that the gate cannot raise as it is leaves the change judged
// against a machine that is not the mesh's — broken against broken, which passes whatever the change does.
// That is an error, never a pass.
// A path the snapshot withheld is given a path of its own where an access or a place needs one: a bare
// "withheld" is no absolute path, and a machine whose setting composes on the mesh would not in the gate.
func TestAWithheldPathIsStoodInForByAPath(t *testing.T) {
got := standInPaths(map[string]any{
"accesses": map[string]any{"races": "withheld", "films": "/media/films", "shows": "/withheld"},
"places": map[string]any{"config": map[string]any{"path": "withheld", "owner": "1000:1000"}},
"puid": 1000,
})
accesses := got["accesses"].(map[string]any)
if accesses["races"] == accesses["shows"] || !strings.HasPrefix(accesses["races"].(string), "/") ||
!strings.HasPrefix(accesses["shows"].(string), "/") || accesses["films"] != "/media/films" {
t.Errorf("accesses: %v", accesses)
}
place := got["places"].(map[string]any)["config"].(map[string]any)
if !strings.HasPrefix(place["path"].(string), "/") || place["owner"] != "1000:1000" || got["puid"] != 1000 {
t.Errorf("places: %v", got)
}
}
// **Issue 286**: a check run by hand clones beside a change what the seat clones — one rule, read by the
// controller's ask and by the facts — and finds the repository it checks from its origin.
func TestACheckByHandClonesWhatTheSeatClones(t *testing.T) {
for dir, refs := range map[string]map[string]string{
"mesh-catalog": {"mesh-catalog": "main"},
"mesh-host": {"mesh-host": "c0ffee"},
"mesh-controller": {"mesh-controller": "c0ffee", "mesh-controller-main": "main", "mesh-lab": "main",
"mesh-sdk": sdkPinnedByGoMod(t)},
} {
got := besideRefs(dir, "c0ffee")
for d, ref := range refs {
if got[d] != ref {
t.Errorf("beside %s, %s is cloned at %q, not %q", dir, d, got[d], ref)
}
}
}
for owner, want := range map[string][3]string{
"ssh://git@git.example:222/novox/mesh-host.git": {"novox", "mesh-host", "ssh://git@git.example:222/novox"},
"git@git.example:novox/mesh-tools.git": {"novox", "mesh-tools", "git@git.example:novox"},
"http://git.example/novox/hq": {"novox", "hq", "http://git.example/novox"},
} {
o, r, p := ownerRepoOf(owner)
if [3]string{o, r, p} != want {
t.Errorf("%s reads as %s %s %s", owner, o, r, p)
}
}
}
// sdkPinnedByGoMod is the SDK commit this tree's go.mod pins, read from the file rather than the build, so
// sdkPinned is held to what the repository says (novox/hq issue 449).
func sdkPinnedByGoMod(t *testing.T) string {
t.Helper()
raw, err := os.ReadFile(filepath.Join("..", "..", "go.mod"))
if err != nil {
t.Fatal(err)
}
for _, line := range strings.Split(string(raw), "\n") {
fields := strings.Fields(line)
if len(fields) >= 2 && fields[0] == sdkModule {
if m := pseudoCommit.FindStringSubmatch(fields[1]); m != nil {
return m[1]
}
return "go/" + fields[1]
}
}
t.Fatalf("go.mod requires no %s", sdkModule)
return ""
}
// **A module one head of a group removes and another defines is a move** (novox/hq ADR 0420 §6, issue 1025):
// neither a second definition of one module name (issue 239) nor the removal of a module machines run
// (ADR 0236). Alone, each head is still refused. This lays the heads over the shelf as judgeChange does,
// which needs no database.
// A group that moves a module machines run passes the gate whole, in either order of its heads, and says the
// module moved (novox/hq ADR 0420 §6, issue 1025); a head alone is still refused. Needs the test store.
// **A module lives whole in one repository** (novox/hq ADR 0420 §1, issue 1019): an artifact whose build
// context names another repository is refused by the gate, and one naming its own passes it.
// **A member of a delivery group judged alone sees a move** through the core clones beside its check
// (novox/hq ADR 0420 §6, issue 1043): the controller's own gate, with the catalogue beside having removed the
// module, is not refused for a second definition (issue 239); the catalogue's own gate, with the controller
// beside defining it, is not refused for a removal (ADR 0236). A clone beside at main finds no move.
@@ -116,7 +116,7 @@ func TestANewModuleIsRegisteredOnTheOperatorsApproveOfItsMergeAndRefusedOtherwis
if c.built != nil { if c.built != nil {
src = *c.built src = *c.built
} }
id, err := mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-view", src, byBuild("build-1"), "", "main") id, err := mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-view", src, registrationAsk{Build: "build-1"}, "", "main")
switch { switch {
case c.accepted && err != nil: case c.accepted && err != nil:
t.Fatalf("%s: the Approved merge's new module was refused: %v", c.ac, err) t.Fatalf("%s: the Approved merge's new module was refused: %v", c.ac, err)
@@ -157,7 +157,7 @@ func TestASecondNewModuleFromARepositoryIsJudgedAlongsideAndAsksNoWarrant(t *tes
request: inventory.BuildRequest{ID: "build-2", Repository: "novox/mesh-delivery", Seat: "git", For: "build"}}} request: inventory.BuildRequest{ID: "build-2", Repository: "novox/mesh-delivery", Seat: "git", For: "build"}}}
forms := &sourceForms{bases: map[string]string{}, base: func(string) string { return "http://forge.internal:20000" }} forms := &sourceForms{bases: map[string]string{}, base: func(string) string { return "http://forge.internal:20000" }}
built := inventory.Source{Repository: "novox/mesh-delivery", Seat: "git"} built := inventory.Source{Repository: "novox/mesh-delivery", Seat: "git"}
if id, err := mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-watchman", built, byBuild("build-2"), "", "main"); err != nil || id == 0 { if id, err := mayRegisterIn(t.Context(), mesh, forms, "mesh-delivery-watchman", built, registrationAsk{Build: "build-2"}, "", "main"); err != nil || id == 0 {
t.Fatalf("1146/AC5: a second module from a repository the catalogue builds from was refused (id %d): %v", id, err) t.Fatalf("1146/AC5: a second module from a repository the catalogue builds from was refused (id %d): %v", id, err)
} }
} }
+3 -35
View File
@@ -11,7 +11,6 @@ import (
"time" "time"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery" sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/builderkeys" "github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/conditions" "github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
@@ -210,10 +209,7 @@ func watchNewModules(ctx context.Context, open *stores) {
fmt.Printf("new modules: the builder's keys are not watched: %v\n", err) fmt.Printf("new modules: the builder's keys are not watched: %v\n", err)
return return
} }
says := watchingSays{js: keys.JS}
says.watch(ctx)
if err := builderkeys.Watch(ctx, keys.JS, func(module string) { if err := builderkeys.Watch(ctx, keys.JS, func(module string) {
says.watch(ctx)
wakeNewModule(ctx, open, module) wakeNewModule(ctx, open, module)
}); err != nil { }); err != nil {
closeKeys() closeKeys()
@@ -249,8 +245,9 @@ func wakeNewModule(ctx context.Context, open *stores, module string) {
// ends only once the work its mesh started has. // ends only once the work its mesh started has.
var background sync.WaitGroup var background sync.WaitGroup
// inBackground runs work beside the caller, counted in background. // inBackground runs work beside the caller, counted in background. A variable so a test of what a walk does around a
func inBackground(work func()) { // take-in may run it in the caller's turn and need not wait for it.
var inBackground = func(work func()) {
background.Add(1) background.Add(1)
go func() { go func() {
defer background.Done() defer background.Done()
@@ -278,32 +275,3 @@ func newModulesOf(ctx context.Context, repository string) ([]string, error) {
sort.Strings(out) sort.Strings(out)
return out, nil return out, nil
} }
// watchingSays watches the builder's bucket of instances once it is there (issue 1886): a change of what the builder says
// asks the look, which judges every open walk again. A builder from before R4e makes no such bucket; a write of the
// builder's keys asks again whether it is there now, so nothing waits on a timer.
type watchingSays struct {
js jetstream.JetStream
mu *sync.Mutex
on *bool
}
func (w *watchingSays) watch(ctx context.Context) {
if w.mu == nil {
w.mu, w.on = &sync.Mutex{}, new(bool)
}
w.mu.Lock()
defer w.mu.Unlock()
if *w.on {
return
}
on, err := builderkeys.WatchSays(ctx, w.js, askToLook)
if err != nil {
fmt.Printf("plans: what the builder says it writes is not watched now: %v\n", err)
return
}
if on {
*w.on = true
askToLook() // what it says now may differ from what the last look read
}
}
@@ -1,198 +0,0 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// heldElsewhere is the plans held by another act for longer than any test runs: waiting for them ends only with the
// context, and a look that does not wait is told at once that they are busy.
func heldElsewhere(ctx context.Context, wait bool) (func(), error) {
if !wait {
return nil, inventory.ErrPlansBusy
}
<-ctx.Done()
return nil, inventory.ErrPlansBusy
}
// 1708/AC1: given the plans held by another act, when the event loop takes a build's outcome, then it returns at
// once, without waiting the plans' two minutes (on 2026-10-21 at 14:20:45 nodered's outcome held the loop until
// 14:22:45, and the operator was told the controller stopped listening).
func TestAnOutcomeDoesNotWaitForThePlansAnotherActHolds(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second)
defer cancel()
start := time.Now()
release, err := holdPlansForOutcome(ctx, heldElsewhere)
if took := time.Since(start); took > time.Second {
t.Fatalf("taking an outcome waited %s for plans another act holds; the event loop takes nothing meanwhile", took)
}
if err == nil || release != nil {
t.Fatalf("plans another act holds were said to be held: release %v, err %v", release != nil, err)
}
}
// 1708/AC4: given the plans free, when the event loop takes a build's outcome, then it holds them and marks the plan
// from the outcome directly, as before.
func TestAnOutcomeHoldsThePlansWhenTheyAreFree(t *testing.T) {
let := false
release, err := holdPlansForOutcome(t.Context(), func(context.Context, bool) (func(), error) {
return func() { let = true }, nil
})
if err != nil || release == nil {
t.Fatalf("free plans were not held: %v", err)
}
release()
if !let {
t.Fatal("the hold given was not the one taken")
}
}
// withLooker installs a looker that is never run, and answers it, so a test can read whether a look was asked.
func withLooker(t *testing.T) *looker {
t.Helper()
l := newLooker(func(context.Context) bool { return false })
lookerMu.Lock()
before := theLooker
theLooker = l
lookerMu.Unlock()
t.Cleanup(func() {
lookerMu.Lock()
theLooker = before
lookerMu.Unlock()
})
return l
}
// 1708/AC2: given an outcome left to the build records because the plans were held, when it is left, then the look
// at the plans is asked for, so the records settle it once the holder lets go rather than at the next event.
func TestAnOutcomeLeftToTheRecordsAsksForTheLook(t *testing.T) {
l := withLooker(t)
leaveToRecords("nodered", "build-1708-ac2", "", "", inventory.ErrPlansBusy)
select {
case <-l.signal:
default:
t.Fatal("an outcome left to the build records asked for no look: the plan waits for whatever event comes next")
}
}
// 1708/AC3: given an outcome left to the records that said what the build did (ADR 0350 §4), when the look settles
// the plan from the build records by that build's id, then the module keeps those words, once.
func TestTheRecordsSettleAnOutcomeWithWhatTheBuildDid(t *testing.T) {
withLooker(t)
asked := time.Date(2026, 10, 21, 12, 15, 0, 0, time.UTC)
plan := func() inventory.Plan {
return inventory.Plan{ID: "plan-1708", Tiers: [][]string{{"nodered"}},
Modules: map[string]*inventory.PlanModule{"nodered": {State: "asked", AskedAt: &asked, Build: "build-1708-ac3"}}}
}
byID := map[string]inventory.Build{"build-1708-ac3": {ID: "build-1708-ac3", Module: "nodered", Commit: "4efac366",
At: asked.Add(5 * time.Minute)}}
leaveToRecords("nodered", "build-1708-ac3", "reused from build-1708-check", "", inventory.ErrPlansBusy)
p := plan()
if !settleFromRecords(&p, p.Tiers[0], nil, byID) || p.Modules["nodered"].State != "built" {
t.Fatalf("the records did not settle the outcome: %+v", p.Modules["nodered"])
}
if got := p.Modules["nodered"].Made; got != "reused from build-1708-check" {
t.Fatalf("settled from the records, the module says %q of what its build did, not the outcome's words", got)
}
again := plan()
settleFromRecords(&again, again.Tiers[0], nil, byID)
if got := again.Modules["nodered"].Made; got != "" {
t.Fatalf("an outcome's words were kept for a second settle: %q", got)
}
}
// 1708/AC5: given a serving controller (its look at the plans running), when an outcome was taken into the plans,
// then moving them (a release's sends, which may retry for minutes) is left to the look, off the event loop
// (2026-10-21 14:28:40 to 14:32:10: the loop sent okioki's release to g14 after mesh-delivery-builder's outcome).
func TestAnOutcomeLeavesMovingThePlansToTheLook(t *testing.T) {
l := withLooker(t)
moved := false
advanceAfterOutcome(func() { moved = true })
if moved {
t.Fatal("the plans were moved on the event loop after an outcome, where a send that retries holds every message")
}
select {
case <-l.signal:
default:
t.Fatal("the look was not asked to move the plans after an outcome")
}
}
// 1708/AC5: given no look running (a command, which has no event loop), when an outcome was taken, then the plans are
// moved at once, as before.
func TestAnOutcomeMovesThePlansWhereNoLookRuns(t *testing.T) {
lookerMu.Lock()
before := theLooker
theLooker = nil
lookerMu.Unlock()
t.Cleanup(func() {
lookerMu.Lock()
theLooker = before
lookerMu.Unlock()
})
moved := false
advanceAfterOutcome(func() { moved = true })
if !moved {
t.Fatal("with no look running, an outcome left the plans unmoved")
}
}
// 1708/AC6: given a registration the mesh refused (its build recorded as worked, the outcome failed), left to the
// records because the plans were held, when the look settles the plan from the records, then the module and the plan
// are failed, never built: settled as built, the tier would be sent the version from before (reviewer-1708@shanks).
func TestARefusedRegistrationLeftToTheRecordsFailsThePlan(t *testing.T) {
withLooker(t)
asked := time.Date(2026, 10, 21, 12, 15, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1708-ac6", State: inventory.PlanBuilding, Tiers: [][]string{{"nodered"}},
Modules: map[string]*inventory.PlanModule{"nodered": {State: "asked", AskedAt: &asked, Build: "build-1708-ac6"}}}
byID := map[string]inventory.Build{"build-1708-ac6": {ID: "build-1708-ac6", Module: "nodered", Commit: "4efac366",
At: asked.Add(5 * time.Minute)}}
leaveToRecords("nodered", "build-1708-ac6", "", "the mesh does not register it: a port is taken", inventory.ErrPlansBusy)
overlayRefusals(nil, byID, refusalsKeptFor([]string{"build-1708-ac6"}))
settleFromRecords(&p, p.Tiers[0], nil, byID)
if s := p.Modules["nodered"]; s.State != "failed" || p.State != inventory.PlanFailed {
t.Fatalf("a refused registration left to the records settled as %q, the plan %q", s.State, p.State)
}
if !strings.Contains(p.Modules["nodered"].Why, "a port is taken") {
t.Fatalf("the refusal's words were lost: %q", p.Modules["nodered"].Why)
}
}
// 1708/AC6: a refusal kept in the store (a controller replaced before the settle) fails the record as one kept in
// the process does.
func TestARefusalFromTheStoreFailsTheRecord(t *testing.T) {
recorded := map[string][]inventory.Build{"nodered": {{ID: "b-store", Module: "nodered"}, {ID: "b-other", Module: "nodered"}}}
byID := map[string]inventory.Build{"b-store": {ID: "b-store", Module: "nodered"}}
overlayRefusals(recorded, byID, map[string]string{"b-store": "not registered"})
if recorded["nodered"][0].Worked() || byID["b-store"].Worked() {
t.Fatal("a build whose registration was refused still reads as worked")
}
if !recorded["nodered"][1].Worked() {
t.Fatal("a build nobody refused was failed")
}
}
// 1708/AC7: given the records settle a module as deleted at its source, or fail the plan, when the look settles, then
// it does what taking the outcome directly does: the deleted module is named to be forgotten, and the plan's failure
// is said (reviewer-1708@shanks).
func TestSettlingFromTheRecordsDoesWhatTakingTheOutcomeDoes(t *testing.T) {
p := inventory.Plan{State: inventory.PlanFailed, Modules: map[string]*inventory.PlanModule{
"gone": {State: planDeleted}, "x": {State: "failed"}, "y": {State: "built"}}}
before := map[string]string{"gone": "asked", "x": "asked", "y": "built"}
deleted, failed := settledEffects(before, &p, inventory.PlanBuilding)
if len(deleted) != 1 || deleted[0] != "gone" {
t.Fatalf("the module the records found deleted is not forgotten: %v", deleted)
}
if !failed {
t.Fatal("the plan the records failed is not said as failed")
}
if d, f := settledEffects(map[string]string{"gone": planDeleted}, &p, inventory.PlanFailed); len(d) != 0 || f {
t.Fatalf("what was deleted or failed before the settle is acted on again: %v %v", d, f)
}
}
-45
View File
@@ -4,7 +4,6 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"os"
"sort" "sort"
"strings" "strings"
"time" "time"
@@ -14,7 +13,6 @@ import (
"github.com/novox/mesh-controller/internal/builderkeys" "github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/conditions" "github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
) )
// A module not registered yet, and the assignment that waits for it (novox/hq issue 325, ADR 0261). // A module not registered yet, and the assignment that waits for it (novox/hq issue 325, ADR 0261).
@@ -74,15 +72,6 @@ type assignOptions struct {
Build bool Build bool
} }
// recordBuildRequest keeps a build request so `assign` can find it, once it is asked. Said, never fatal: the
// build was asked.
func recordBuildRequest(ctx context.Context, inv *inventory.Inventory, r inventory.BuildRequest) {
if err := inv.RecordBuildRequest(ctx, r); err != nil {
fmt.Fprintf(os.Stderr, "build %s was asked, and could not be kept as a build request, so `assign` "+
"will not know it is coming: %v\n", r.ID, err)
}
}
// The cases of a module the catalogue does not hold (novox/hq issue 325). // The cases of a module the catalogue does not hold (novox/hq issue 325).
const ( const (
unknownModule = iota unknownModule = iota
@@ -474,40 +463,6 @@ func expirePending(ctx context.Context, inv *inventory.Inventory, p inventory.Pe
return note return note
} }
// settlePendingOnBuild is a build's outcome reaching the assignments waiting for it, wherever the outcome is
// taken in (novox/hq issue 325): a module registered makes every assignment pending on it; a build of a
// pending assignment that failed, or was not registered, ends it with why. Returns what was said.
func settlePendingOnBuild(ctx context.Context, open *stores, result link.BuildResult, module string, takeErr error) []string {
inv := open.inventory
waiting, err := inv.Pending(ctx, time.Time{})
if err != nil {
return []string{fmt.Sprintf("the pending assignments could not be read after build %s: %v", result.ID, err)}
}
registered := module != "" && (takeErr == nil || errors.Is(takeErr, inventory.ErrSuperseded))
var said []string
for _, p := range waiting {
switch {
case registered && p.Module == module:
if line := applyPending(ctx, open, p, result.ID); line != "" {
said = append(said, line)
}
case !registered && p.Build == result.ID:
why := firstLine(result.Failed)
if why == "" && takeErr != nil {
why = firstLine(takeErr.Error())
}
what := fmt.Sprintf("build %s of %s failed: %s", result.ID, orRoot(p.Path), why)
if result.Failed == "" {
what = fmt.Sprintf("build %s of %s was recorded and not registered: %s", result.ID, orRoot(p.Path), why)
}
if line := expirePending(ctx, inv, p, what); line != "" {
said = append(said, line)
}
}
}
return said
}
// settlingPending is the controller's tick over the pending assignments (ADR 0261), every settleEvery until // settlingPending is the controller's tick over the pending assignments (ADR 0261), every settleEvery until
// the context ends. What it does is printed to the controller's log, kept in each row's note (which `status` // the context ends. What it does is printed to the controller's log, kept in each row's note (which `status`
// lists for a day), and raised as a condition for an assignment that was not made. // lists for a day), and raised as a condition for an assignment that was not made.
@@ -5,7 +5,6 @@ package main
import ( import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"slices" "slices"
"strings" "strings"
"testing" "testing"
@@ -567,34 +566,6 @@ func TestAStaleClaimIsSettledFromWhatTheMeshHolds(t *testing.T) {
} }
} }
func TestABuildNoLongerWaitedForIsStillInFlight(t *testing.T) {
open := aCatalogueMesh(t)
ctx := t.Context()
inv := open.inventory
for _, id := range []string{"build-timeout", "build-nothandedover"} {
dir := "modules/" + strings.TrimPrefix(id, "build-")
if err := inv.RecordBuildRequest(ctx, inventory.BuildRequest{ID: id, Repository: "novox/mesh-catalog",
Seat: "git", Path: dir, Ref: "main", For: "build"}); err != nil {
t.Fatal(err)
}
}
markWaitFailed(ctx, "build-timeout", errors.New("no build machine answered within 10m0s"))
markWaitFailed(ctx, "build-nothandedover", fmt.Errorf("%w: cannot submit a build: nats: timeout", link.ErrNotHandedOver))
said, err := assign(ctx, open, "laptop", "timeout")
if err != nil || !strings.Contains(said, "being built") || !strings.Contains(said, "kept as pending") {
t.Fatalf("a build no longer waited for: %q %v", said, err)
}
if _, err := assign(ctx, open, "laptop", "nothandedover"); err == nil || !strings.Contains(err.Error(), "was not handed over") {
t.Fatalf("a build never handed over: %v", err)
}
settlePending(ctx, open, time.Now().Add(buildRequestBound+time.Minute))
rows, _ := inv.PendingFor(ctx, "laptop", "timeout")
if len(rows) != 1 || rows[0].State != inventory.PendingExpired || !strings.Contains(rows[0].Note, "its asker stopped waiting") {
t.Fatalf("past the bound: %+v", rows)
}
}
// A10 (ADR 0501 §6; issue 1851): an assignment of a module the indexer holds and the catalogue does not yet waits for // A10 (ADR 0501 §6; issue 1851): an assignment of a module the indexer holds and the catalogue does not yet waits for
// the builder's done key of it, asking nothing, and is made when that key registers the module. // the builder's done key of it, asking nothing, and is made when that key registers the module.
func TestAnAssignmentWaitsForTheKeyAndAsksNothing(t *testing.T) { func TestAnAssignmentWaitsForTheKeyAndAsksNothing(t *testing.T) {
+1 -1
View File
@@ -1084,7 +1084,7 @@ func unansweredWords(l stalledLine, o conditions.Observation) (headline, explana
} }
// uncheckedWaitWords are the merge checks an unanswered line waits on, as the operator reads them: from the checks // uncheckedWaitWords are the merge checks an unanswered line waits on, as the operator reads them: from the checks
// given as data, each time in the controller's local zone, as every other time in a message — "mesh/merge-gate // given as data, each time in the controller's local zone, as every other time in a message — "mesh/check
// pending since 02:11" — never the UTC time inside mesh-delivery's finished words, which cannot be said again in // pending since 02:11" — never the UTC time inside mesh-delivery's finished words, which cannot be said again in
// another zone (novox/hq issue 443). A line from a mesh-delivery that gives no such data is said by its words. // another zone (novox/hq issue 443). A line from a mesh-delivery that gives no such data is said by its words.
func uncheckedWaitWords(l stalledLine) []string { func uncheckedWaitWords(l stalledLine) []string {
+5 -5
View File
@@ -22,11 +22,11 @@ func TestAPlansLineCountsFromItsTierNotFromItsLastSave(t *testing.T) {
if !strings.Contains(line, "building for 40m0s") || !strings.Contains(line, "LATE") { if !strings.Contains(line, "building for 40m0s") || !strings.Contains(line, "LATE") {
t.Fatalf("a plan in its tier for 40m, saved 3s ago, reads %q", line) t.Fatalf("a plan in its tier for 40m, saved 3s ago, reads %q", line)
} }
st := planStatuses([]inventory.Plan{p}, now, pauseView{}, nil)[0] st := planStatuses([]inventory.Plan{p}, now, nil)[0]
if !st.Late || !st.Since.Equal(p.TierEntered) { if !st.Late || !st.Since.Equal(p.TierEntered) {
t.Errorf("status --json says %+v", st) t.Errorf("status --json says %+v", st)
} }
if _, late := openPlans([]inventory.Plan{p}, now, pauseView{}, nil); late != 1 { if _, late := openPlans([]inventory.Plan{p}, now, nil); late != 1 {
t.Errorf("status counted %d late", late) t.Errorf("status counted %d late", late)
} }
@@ -42,7 +42,7 @@ func TestAPlansLineCountsFromItsTierNotFromItsLastSave(t *testing.T) {
q := p q := p
q.TierEntered = now.Add(-in) q.TierEntered = now.Add(-in)
bound := bounds.of(q.Repository) bound := bounds.of(q.Repository)
late := strings.Contains(planLineWith(q, now, pauseView{}, bound), "LATE") late := strings.Contains(planLineWith(q, now, bound), "LATE")
stalled := len(watchPlans(&signalFacts{now: now, plans: []planFacts{{id: q.ID, repository: q.Repository, stalled := len(watchPlans(&signalFacts{now: now, plans: []planFacts{{id: q.ID, repository: q.Repository,
commit: q.Commit, tiers: 1, entered: inTierSince(q), bound: bound}}})) == 1 commit: q.Commit, tiers: 1, entered: inTierSince(q), bound: bound}}})) == 1
if late != stalled || late != (in > bound) { if late != stalled || late != (in > bound) {
@@ -65,7 +65,7 @@ func TestAPlansLineCountsFromItsTierNotFromItsLastSave(t *testing.T) {
if line := planLine(waiting, now); !strings.Contains(line, "for 40m0s") || strings.Contains(line, "LATE") { if line := planLine(waiting, now); !strings.Contains(line, "for 40m0s") || strings.Contains(line, "LATE") {
t.Errorf("a walk waiting 40m reads %q", line) t.Errorf("a walk waiting 40m reads %q", line)
} }
if st := planStatuses([]inventory.Plan{waiting}, now, pauseView{}, nil)[0]; st.Late { if st := planStatuses([]inventory.Plan{waiting}, now, nil)[0]; st.Late {
t.Errorf("a waiting walk is called late: %+v", st) t.Errorf("a waiting walk is called late: %+v", st)
} }
let := now.Add(-5 * time.Minute) let := now.Add(-5 * time.Minute)
@@ -99,7 +99,7 @@ func TestAPlanLineSaysTheRefusalOnceItsTierIsBuilt(t *testing.T) {
if !strings.Contains(line, "a build waits there for its gate: novox") || strings.Contains(line, "building for") { if !strings.Contains(line, "a build waits there for its gate: novox") || strings.Contains(line, "building for") {
t.Errorf("920/AC1: a plan with its tier built and a refusal reads %q", line) t.Errorf("920/AC1: a plan with its tier built and a refusal reads %q", line)
} }
st := planStatuses([]inventory.Plan{p}, now, pauseView{}, nil) st := planStatuses([]inventory.Plan{p}, now, nil)
if len(st) != 1 || !st[0].Late { if len(st) != 1 || !st[0].Late {
t.Errorf("920/AC1: status: %+v", st) t.Errorf("920/AC1: status: %+v", st)
} }
+5 -73
View File
@@ -9,7 +9,6 @@ import (
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery" sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/builderkeys" "github.com/novox/mesh-controller/internal/builderkeys"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
@@ -17,64 +16,19 @@ import (
// A walk takes its modules from the builder's done keys (novox/hq ADR 0503 §6, ADR 0501 §1, ADR 0498 §8; issue 1851, A6 // A walk takes its modules from the builder's done keys (novox/hq ADR 0503 §6, ADR 0501 §1, ADR 0498 §8; issue 1851, A6
// and A7). // and A7).
// //
// From R4e a walk asks build-agent for no module but the builder's own (ADR 0453 §5). A module of its tier is marked // A walk asks no build of anybody (R4g; ADR 0453 §5). A module of its tier is marked
// taken from a key (PlanModule.Key) and settled by the look: its fingerprint is read from the indexer's trunk value // taken from a key (PlanModule.Key) and settled by the look: its fingerprint is read from the indexer's trunk value
// written for the commit the walk carries of its repository; the builder's key of that fingerprint, done with every // written for the commit the walk carries of its repository; the builder's key of that fingerprint, done with every
// artifact referenced, is taken in (recorded and registered, off the plans' hold) and the module is built; a key not done // artifact referenced, is taken in (recorded and registered, off the plans' hold) and the module is built; a key not done
// yet, or waiting for a reference, holds the tier and says so; a failed key fails the walk in the builder's own words. // yet, or waiting for a reference, holds the tier and says so; a failed key fails the walk in the builder's own words.
// **No timer**: the builder's and the indexer's writes wake the look (builderkeys.Watch, watchNewModules). // **No timer**: the builder's and the indexer's writes wake the look (builderkeys.Watch, watchNewModules).
// isBuildersOwn says a module is build-agent's own: asked of build-agent on a merge and taken from its build record, never
// from the builder's key, until R4g (ADR 0453 §5, ADR 0498 §7, ADR 0520 §1).
func isBuildersOwn(module string) bool { return buildersOwn(module) }
// buildAgentsOwn are those modules, named (ADR 0520 §1): the builder's; build-agent itself, whose image's context is another
// repository's; and mesh-tools, the toolchain image build-agent's builds stand on. The builder writes no key of any of them,
// and a walk waiting for one would hold its tier for ever. mesh-tools-go is not among them: it builds from a Dockerfile in
// its own directory on a pinned base, and has its key from mesh-build-image.
//
// **And the parts that write references** (ADR 0522 §6, issue 1886): the build modules push and answer each reference,
// and the builder writes it into the key, so none of them is ever taken from a key they write themselves. Until R4g
// build-agent is the way they are delivered whatever the builder says.
var buildAgentsOwn = map[string]bool{sdkdelivery.BuilderModule: true, builder.AgentModule: true, "mesh-tools": true,
"mesh-build-go": true, "mesh-build-node": true, "mesh-build-image": true}
// buildersOwn is isBuildersOwn's rule: a variable only so a test of the asks left (their bus waits, their plan key) may
// name modules of its own as build-agent's.
var buildersOwn = func(module string) bool { return buildAgentsOwn[module] }
// planKeyReads is what a walk reads of the indexer and the builder (builderkeys.Reader, or a test's). // planKeyReads is what a walk reads of the indexer and the builder (builderkeys.Reader, or a test's).
type planKeyReads interface { type planKeyReads interface {
ModuleAt(ctx context.Context, module, commit string) (sdkdelivery.Module, error) ModuleAt(ctx context.Context, module, commit string) (sdkdelivery.Module, error)
Build(ctx context.Context, module, fp string) (builderkeys.Key, error) Build(ctx context.Context, module, fp string) (builderkeys.Key, error)
} }
// builderSayKey carries, on an ask's context, the builder's say the walk read before asking (withBuilderSay).
type builderSayKey struct{}
// withBuilderSay hands the say a walk read to the ask's guard, so the guard never reads it a second time.
func withBuilderSay(ctx context.Context, says bool) context.Context {
return context.WithValue(ctx, builderSayKey{}, says)
}
// builderSayOf is the say a walk handed the ask, and whether it handed one.
func builderSayOf(ctx context.Context) (says, handed bool) {
says, handed = ctx.Value(builderSayKey{}).(bool)
return says, handed
}
// builderSaysReferences is whether a running builder instance says, in its own bucket of instances, that the keys it
// writes name references (issue 1886). Until one does, a walk asks build-agent for every module, as before R4e: the
// keys the running builder writes could never be delivered from. A variable so a test needs no bus.
var builderSaysReferences = func(ctx context.Context) (bool, error) {
keys, closeKeys, err := openBuilderKeys()
if err != nil {
return false, err
}
defer closeKeys()
return keys.SaysReferences(ctx)
}
// openPlanKeys opens what a walk reads of the indexer and the builder. A variable so a test needs no bus. // openPlanKeys opens what a walk reads of the indexer and the builder. A variable so a test needs no bus.
var openPlanKeys = func() (planKeyReads, func(), error) { var openPlanKeys = func() (planKeyReads, func(), error) {
r, closeKeys, err := openBuilderKeys() r, closeKeys, err := openBuilderKeys()
@@ -112,10 +66,13 @@ var keyTakes = struct {
// the buckets and the records; a done key not recorded yet is taken in off the plans' hold, and its end asks the look. // the buckets and the records; a done key not recorded yet is taken in off the plans' hold, and its end asks the look.
func settleFromKeys(ctx context.Context, open *stores, keys planKeyReads, p *inventory.Plan, tier []string) (bool, error) { func settleFromKeys(ctx context.Context, open *stores, keys planKeyReads, p *inventory.Plan, tier []string) (bool, error) {
return settleKeys(ctx, open.inventory, keys, p, tier, func(module string, trunk sdkdelivery.Module, id string) { return settleKeys(ctx, open.inventory, keys, p, tier, func(module string, trunk sdkdelivery.Module, id string) {
inBackground(func() { takeKeyOffTheHold(context.WithoutCancel(ctx), open, module, trunk, id) }) inBackground(func() { takeKeyIn(context.WithoutCancel(ctx), open, module, trunk, id) })
}) })
} }
// takeKeyIn is how a look takes a done key in; a variable so a test of what a walk does around it needs no builder.
var takeKeyIn = takeKeyOffTheHold
// keyRecords is what settling from keys reads of the records (*inventory.Inventory, or a test's). // keyRecords is what settling from keys reads of the records (*inventory.Inventory, or a test's).
type keyRecords interface { type keyRecords interface {
BuildByID(ctx context.Context, id string) (inventory.Build, bool, error) BuildByID(ctx context.Context, id string) (inventory.Build, bool, error)
@@ -133,20 +90,6 @@ func settleKeys(ctx context.Context, inv keyRecords, keys planKeyReads, p *inven
changed = true changed = true
} }
} }
// One say for the ask and the settle (builderSaysReferences): two sources that differ hand a module back and take it
// from its key again at every turn of the look, which then never ends.
says, err := builderSaysReferences(ctx)
if err != nil {
return changed, fmt.Errorf("whether the builder writes references cannot be read: %w", err)
}
// **A walk never waits on a key the running builder cannot reference** (issue 1886): while no builder says it writes
// references, a module that waits on its key, or whose key failed, is handed back unasked, and the look asks it of
// build-agent. A done key with every reference is still taken.
toAgent := func(s *inventory.PlanModule, why string) {
s.State, s.Key, s.AskedAt, s.KeyAt, s.Build, s.Fingerprint = "", false, nil, "", "", ""
s.Why = "the builder does not say it writes references, so build-agent is asked (" + why + ")"
changed = true
}
for _, m := range tier { for _, m := range tier {
s := p.Modules[m] s := p.Modules[m]
if s == nil || !s.Key || s.State != "asked" { if s == nil || !s.Key || s.State != "asked" {
@@ -154,10 +97,6 @@ func settleKeys(ctx context.Context, inv keyRecords, keys planKeyReads, p *inven
} }
trunk, err := keys.ModuleAt(ctx, m, s.KeyAt) trunk, err := keys.ModuleAt(ctx, m, s.KeyAt)
if errors.Is(err, builderkeys.ErrNoValue) { if errors.Is(err, builderkeys.ErrNoValue) {
if !says {
toAgent(s, "the indexer holds no value of it")
continue
}
set(s, fmt.Sprintf("waits for the indexer's value of %s at %s", m, orNow(s.KeyAt))) set(s, fmt.Sprintf("waits for the indexer's value of %s at %s", m, orNow(s.KeyAt)))
continue continue
} }
@@ -169,10 +108,6 @@ func settleKeys(ctx context.Context, inv keyRecords, keys planKeyReads, p *inven
if why == "" { if why == "" {
why = "the indexer wrote no fingerprint of the new form" why = "the indexer wrote no fingerprint of the new form"
} }
if !says {
toAgent(s, why)
continue
}
set(s, fmt.Sprintf("waits: %s at %s has no fingerprint to take a build by: %s", m, orNow(s.KeyAt), why)) set(s, fmt.Sprintf("waits: %s at %s has no fingerprint to take a build by: %s", m, orNow(s.KeyAt), why))
continue continue
} }
@@ -184,9 +119,6 @@ func settleKeys(ctx context.Context, inv keyRecords, keys planKeyReads, p *inven
return changed, err return changed, err
} }
switch { switch {
case !says && (k.Failed() || !k.Done() || len(k.Unreferenced()) > 0):
toAgent(s, k.Says())
continue
case k.Failed(): case k.Failed():
// **A failed key fails the walk in the builder's own words** (`digests differ:`, `not stored:`, `not pinned:`, // **A failed key fails the walk in the builder's own words** (`digests differ:`, `not stored:`, `not pinned:`,
// `version taken:`), and a rebuild of it is a new revision the next walk takes. // `version taken:`), and a rebuild of it is a new revision the next walk takes.
-15
View File
@@ -73,12 +73,6 @@ func aWalkOfOne(module string) (*inventory.Plan, map[string]inventory.Entry) {
// A6 (ADR 0503 §6, ADR 0498 check §7): a walk whose module has a done key goes asked to built with no build asked: the // A6 (ADR 0503 §6, ADR 0498 check §7): a walk whose module has a done key goes asked to built with no build asked: the
// ask stub is never called, the key is taken in once, and `plans` says it was taken from the builder's key. // ask stub is never called, the key is taken in once, and `plans` says it was taken from the builder's key.
func TestAWalkTakesADoneKeyAndAsksNoBuild(t *testing.T) { func TestAWalkTakesADoneKeyAndAsksNoBuild(t *testing.T) {
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
t.Fatal("a walk asked build-agent for a module that is not the builder's own")
return "", nil
}
t.Cleanup(func() { askABuild = was })
p, byName := aWalkOfOne("sudo") p, byName := aWalkOfOne("sudo")
askModule(t.Context(), p, "sudo", byName) askModule(t.Context(), p, "sudo", byName)
s := p.Modules["sudo"] s := p.Modules["sudo"]
@@ -118,20 +112,11 @@ func TestAWalkTakesADoneKeyAndAsksNoBuild(t *testing.T) {
if s.State != "built" || s.Build != id || !strings.Contains(s.Made, "taken from the builder's key") { if s.State != "built" || s.Build != id || !strings.Contains(s.Made, "taken from the builder's key") {
t.Fatalf("the module is not built from its key: %+v", s) t.Fatalf("the module is not built from its key: %+v", s)
} }
if takesOutcome(*p, &inventory.PlanModule{Key: true, State: "asked", AskedAt: s.AskedAt}, time.Now(), "build-1") {
t.Fatal("a build-agent outcome answers a module taken from a key")
}
} }
// A7: a key building waits and says so, then moves on when done, with no timer and no ask; a failed key fails the walk // A7: a key building waits and says so, then moves on when done, with no timer and no ask; a failed key fails the walk
// in the builder's words; a value the indexer has not written yet waits, named. // in the builder's words; a value the indexer has not written yet waits, named.
func TestAWalkWaitsForAKeyAndFailsWithItsWords(t *testing.T) { func TestAWalkWaitsForAKeyAndFailsWithItsWords(t *testing.T) {
asksOfNone := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
t.Fatal("a walk asked a build")
return "", nil
}
t.Cleanup(func() { askABuild = asksOfNone })
t.Cleanup(func() { t.Cleanup(func() {
keyTakes.Lock() keyTakes.Lock()
keyTakes.running, keyTakes.waits, keyTakes.taken = map[string]bool{}, map[string]string{}, map[string]bool{} keyTakes.running, keyTakes.waits, keyTakes.taken = map[string]bool{}, map[string]string{}, map[string]bool{}
-16
View File
@@ -136,18 +136,6 @@ func askToLook() {
} }
} }
// askedToLook asks the serving controller's look at the plans, and says whether there was one to ask.
func askedToLook() bool {
lookerMu.RLock()
l := theLooker
lookerMu.RUnlock()
if l == nil {
return false
}
l.ask()
return true
}
// startPlanLooks starts the look at the open plans, and asks for the first: the controller took the lease, and a // startPlanLooks starts the look at the open plans, and asks for the first: the controller took the lease, and a
// plan left by a replaced controller resumes from there (ADR 0406 §6). It replaces the rollout's 30 second ticker. // plan left by a replaced controller resumes from there (ADR 0406 §6). It replaces the rollout's 30 second ticker.
// It answers a channel that is closed once the look has returned after the context ended, for a test that restores what // It answers a channel that is closed once the look has returned after the context ended, for a test that restores what
@@ -206,10 +194,6 @@ func servingPlanSaved(ctx context.Context, bus link.Bus, open *stores) func(*inv
return func(_ *inventory.Inventory, p inventory.Plan) { return func(_ *inventory.Inventory, p inventory.Plan) {
sayPlanMoved(ctx, bus, open.inventory, p) sayPlanMoved(ctx, bus, open.inventory, p)
planKept(p) planKept(p)
keepBuildingKey(ctx, p)
if p.WalkSaid == inventory.WalkEnded {
go forgetChecksOfWalk(context.WithoutCancel(ctx), open.inventory, p)
}
} }
} }
+2 -117
View File
@@ -3,14 +3,12 @@ package main
import ( import (
"context" "context"
"fmt" "fmt"
"os"
"slices" "slices"
"sort" "sort"
"strings" "strings"
"time" "time"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
) )
// A plan follows what is done to the build queue (novox/hq ADR 0219). // A plan follows what is done to the build queue (novox/hq ADR 0219).
@@ -26,106 +24,6 @@ import (
// beside it — beside it, the plan would either ask it again or stay failed on an outcome the // beside it — beside it, the plan would either ask it again or stay failed on an outcome the
// rebuild has already replaced. // rebuild has already replaced.
// pauseView is whether the build seat takes work: the holders that said they are paused, and
// whether that is every holder.
type pauseView struct {
Nodes []string
All bool
}
// pausedWaiting is what a plan waits on when the build seat is paused under it, or false: a plan
// building, whose current tier has an ask outstanding, while every holder of the seat is paused.
func pausedWaiting(p inventory.Plan, pause pauseView, now time.Time) (string, bool) {
if p.State != inventory.PlanBuilding || !pause.All || len(pause.Nodes) == 0 || p.Tier >= len(p.Tiers) {
return "", false
}
var asked *time.Time
for _, m := range p.Tiers[p.Tier] {
if s := p.Modules[m]; s != nil && s.State == "asked" && s.AskedAt != nil {
if asked == nil || s.AskedAt.Before(*asked) {
asked = s.AskedAt
}
}
}
if asked == nil {
return "", false
}
waited := now.Sub(*asked)
said := fmt.Sprintf("waiting: the build seat is paused on %s (asked %s ago)",
strings.Join(pause.Nodes, ", "), waited.Round(time.Second))
// Not late — a pause is a decision — but a pause forgotten is a plan that never moves, so one held
// longer than a day is named.
if waited > pausedTooLong {
said += " — PAUSED OVER A DAY: `resume` takes builds again"
}
return said, true
}
// pausedTooLong is how long a plan may wait on a paused build seat before it says the pause is long.
const pausedTooLong = 24 * time.Hour
// awaitsABuild is whether any open plan has an ask outstanding in its current tier — the only case
// where the seat being paused changes what a plan says.
func awaitsABuild(plans []inventory.Plan) bool {
for _, p := range plans {
if p.State != inventory.PlanBuilding || p.Tier >= len(p.Tiers) {
continue
}
for _, m := range p.Tiers[p.Tier] {
if s := p.Modules[m]; s != nil && s.State == "asked" {
return true
}
}
}
return false
}
// buildSeatPause reads whether the build seat's holders take work, from what each last said on the
// bus (link.HolderState) — read only when a plan waits on a build, so a mesh with nothing building
// does not dial the bus to say so. Anything unreadable is said and read as not paused: a plan then
// reads as late, which is what it said before this existed.
func buildSeatPause(ctx context.Context, inv *inventory.Inventory, plans []inventory.Plan) pauseView {
if !awaitsABuild(plans) {
return pauseView{}
}
entries, err := inv.Catalogued(ctx)
if err != nil {
return pauseView{}
}
seat := buildSeatAmong(entries)
holders := holdersAmong(entries, seat)
if len(holders) == 0 {
return pauseView{}
}
// On the serving controller's own connection when this is it: a watchdog tick while a walk waits for a
// build dialled one every 30 seconds (novox/hq issue 327).
js, err := aBus()
if err != nil {
fmt.Fprintf(os.Stderr, "could not reach the bus to read whether the build seat is paused: %v\n", err)
return pauseView{}
}
defer js.Close()
said, err := link.PausedSaid(js, seat, holders)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read whether the build seat is paused: %v\n", err)
return pauseView{}
}
return pauseOf(holders, said)
}
// pauseOf is the view from what each holder said.
func pauseOf(holders []string, said map[string]link.HolderState) pauseView {
var v pauseView
for _, n := range holders {
if said[n].Paused {
v.Nodes = append(v.Nodes, n)
}
}
sort.Strings(v.Nodes)
v.All = len(holders) > 0 && len(v.Nodes) == len(holders)
return v
}
// failedIn is the modules of a plan's current tier that failed to build, sorted. // failedIn is the modules of a plan's current tier that failed to build, sorted.
func failedIn(p inventory.Plan) []string { func failedIn(p inventory.Plan) []string {
if p.Tier >= len(p.Tiers) { if p.Tier >= len(p.Tiers) {
@@ -684,20 +582,10 @@ func holdingABuild(p inventory.Plan) []string {
return out return out
} }
// resumeStopped sets a walk stopped by hand building again at the tier it stopped at, its stop cleared and its asks kept but // resumeStopped sets a walk stopped by hand building again at the tier it stopped at, its stop cleared and its asks kept
// build-agent's own (issue 1886): the next look carries it on as any walk — a build done waits for its reference, and one done on the // (issue 1886): the next look carries it on as any walk — a build done waits for its reference, and one done on the
// builder's key is handed on (novox/hq ADR 0522 §3). Its note keeps who stopped it and that a person resumed it. // builder's key is handed on (novox/hq ADR 0522 §3). Its note keeps who stopped it and that a person resumed it.
func resumeStopped(p *inventory.Plan) string { func resumeStopped(p *inventory.Plan) string {
// **Build-agent's own are asked of build-agent** (novox/hq ADR 0522 §6): one marked taken from a key it writes itself is
// set unasked, and the look asks the rest of a partly asked tier (unaskedOf), of build-agent, whatever the builder says.
var again []string
for _, m := range p.Tiers[p.Tier] {
if s := p.Modules[m]; s != nil && s.State == "asked" && s.Key && isBuildersOwn(m) {
s.State, s.Key, s.AskedAt, s.Build, s.Why = "", false, nil, "", ""
again = append(again, m)
}
}
sort.Strings(again)
asked, unsent, unasked := waitingIn(*p) asked, unsent, unasked := waitingIn(*p)
later := 0 later := 0
for i := p.Tier + 1; i < len(p.Tiers); i++ { for i := p.Tier + 1; i < len(p.Tiers); i++ {
@@ -706,9 +594,6 @@ func resumeStopped(p *inventory.Plan) string {
by, why := p.Delivery.Stopped, p.Delivery.StoppedWhy by, why := p.Delivery.Stopped, p.Delivery.StoppedWhy
waits := fmt.Sprintf("%d asked, %d built and not sent, %d not yet asked in it; %d not yet asked in the tiers after", waits := fmt.Sprintf("%d asked, %d built and not sent, %d not yet asked in it; %d not yet asked in the tiers after",
asked, unsent, unasked, later) asked, unsent, unasked, later)
if len(again) > 0 {
waits += "; " + strings.Join(again, ", ") + " set unasked, for the look to ask of build-agent (its own, ADR 0522 §6)"
}
p.State = inventory.PlanBuilding p.State = inventory.PlanBuilding
goesOnAgain(p) goesOnAgain(p)
p.Delivery.Kept = nil p.Delivery.Kept = nil
+5 -16
View File
@@ -1,7 +1,6 @@
package main package main
import ( import (
"slices"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -27,7 +26,7 @@ func TestRetryResumesAWalkStoppedByHand(t *testing.T) {
p.Modules[m] = &inventory.PlanModule{State: "built", BuiltAt: &sent, SentAt: &sent} p.Modules[m] = &inventory.PlanModule{State: "built", BuiltAt: &sent, SentAt: &sent}
} }
var tier1 []string var tier1 []string
for _, m := range []string{"build-agent", "mesh-tools"} { for _, m := range []string{"mesh-tools-go", "mesh-tools"} {
tier1 = append(tier1, m) tier1 = append(tier1, m)
p.Modules[m] = &inventory.PlanModule{State: "built", AskedAt: &sent, BuiltAt: &sent, Build: "build-" + m} p.Modules[m] = &inventory.PlanModule{State: "built", AskedAt: &sent, BuiltAt: &sent, Build: "build-" + m}
} }
@@ -58,22 +57,12 @@ func TestRetryResumesAWalkStoppedByHand(t *testing.T) {
} }
for _, m := range onKeys { for _, m := range onKeys {
s := p.Modules[m] s := p.Modules[m]
if isBuildersOwn(m) {
// Build-agent's own are asked of build-agent whatever the builder says (ADR 0522 §6): set unasked, for the look.
if s.State != "" || s.Key || s.Build != "" || s.AskedAt != nil {
t.Errorf("%s, build-agent's own, was left on the builder's key: %+v", m, s)
}
if !slices.Contains(unaskedOf(p), m) {
t.Errorf("%s is not among what the look asks next: %v", m, unaskedOf(p))
}
continue
}
if s.State != "asked" || !s.Key || s.Build != "build-"+m { if s.State != "asked" || !s.Key || s.Build != "build-"+m {
t.Errorf("%s's ask was not kept for the look to hand on: %+v", m, s) t.Errorf("%s's ask was not kept for the look to hand on: %+v", m, s)
} }
} }
for _, w := range []string{"plan-r4e", "tier 2 of 3", "5 asked", "2 built and not sent", "3 not yet asked in it", for _, w := range []string{"plan-r4e", "tier 2 of 3", "8 asked", "2 built and not sent", "0 not yet asked in it",
"3 not yet asked in the tiers after", "mesh-build-go, mesh-build-image, mesh-build-node set unasked"} { "3 not yet asked in the tiers after"} {
if !strings.Contains(said, w) { if !strings.Contains(said, w) {
t.Errorf("the answer %q does not say %q", said, w) t.Errorf("the answer %q does not say %q", said, w)
} }
@@ -109,8 +98,8 @@ func TestRetryResumesAWalkStoppedByHand(t *testing.T) {
} }
// A newer plan holding a build of its tier not yet sent refuses it: resuming would send the older build over the newer. // A newer plan holding a build of its tier not yet sent refuses it: resuming would send the older build over the newer.
newerBuilt := inventory.Plan{ID: "plan-agent", Repository: "novox/mesh-controller", Branch: "main", Commit: "d00dfeed", newerBuilt := inventory.Plan{ID: "plan-agent", Repository: "novox/mesh-controller", Branch: "main", Commit: "d00dfeed",
Created: at, State: inventory.PlanDone, Tiers: [][]string{{"build-agent"}}, Created: at, State: inventory.PlanDone, Tiers: [][]string{{"mesh-tools"}},
Modules: map[string]*inventory.PlanModule{"build-agent": {State: "built"}}} Modules: map[string]*inventory.PlanModule{"mesh-tools": {State: "built"}}}
if err := retryRefusal(stopped, []inventory.Plan{stopped, newerBuilt}, ownPathOf(nil)); err == nil || if err := retryRefusal(stopped, []inventory.Plan{stopped, newerBuilt}, ownPathOf(nil)); err == nil ||
!strings.Contains(err.Error(), "older build") { !strings.Contains(err.Error(), "older build") {
t.Errorf("a stopped walk was resumed over a newer plan of a build it has not sent: %v", err) t.Errorf("a stopped walk was resumed over a newer plan of a build it has not sent: %v", err)
-42
View File
@@ -1,7 +1,6 @@
package main package main
import ( import (
"encoding/json"
"fmt" "fmt"
"math/rand/v2" "math/rand/v2"
"sort" "sort"
@@ -9,7 +8,6 @@ import (
"testing" "testing"
"time" "time"
snapshot "git.novox.be/novox/mesh-sdk/go/facts"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
@@ -589,46 +587,6 @@ func describe(entries []inventory.Entry) []string {
return out return out
} }
// **The merge gate reads the build sources the snapshot carries** (novox/hq ADR 0267): the gate's plan of a
// change is the merge handler's, so a snapshot taken by a controller that records build sources narrows the
// gate's plan as it narrows the merge's; one without them reads every module as before.
func TestTheGatePlansFromTheBuildSourcesTheSnapshotCarries(t *testing.T) {
manifest := func(name string) json.RawMessage { return json.RawMessage(`{"module":"` + name + `","version":"1"}`) }
facts := snapshot.Facts{Modules: []snapshot.Module{
{Name: "mesh-controller", Repository: "novox/mesh-controller", Manifest: manifest("mesh-controller"),
Sources: []snapshot.BuildSource{{Own: true, Paths: []string{"module.json", "cmd/mesh-controller/", "internal/broker/"}}}},
{Name: "route-proxy", Repository: "novox/mesh-catalog", Path: "modules/route-proxy", Manifest: manifest("route-proxy"),
Reads: []string{"novox/mesh-controller"},
Sources: []snapshot.BuildSource{{Repository: "novox/mesh-controller", Paths: []string{"modules/route-proxy/cmd/route-proxy/", "internal/broker/"}},
{Own: true, Paths: []string{"modules/route-proxy/module.json"}}}},
}}
for paths, want := range map[string]string{
"cmd/mesh-controller/main.go": "mesh-controller",
"modules/route-proxy/cmd/route-proxy/main.go": "route-proxy",
"internal/broker/broker.go": "mesh-controller,route-proxy",
"README.md": "",
} {
r, err := reachOfChange(facts, "novox/mesh-controller", []string{paths}, "")
if err != nil {
t.Fatal(err)
}
if got := tiered(r.Plan.Tiers); got != want {
t.Errorf("%s: the gate planned %q, wanted %q", paths, got, want)
}
}
// A snapshot without build sources: as before.
for i := range facts.Modules {
facts.Modules[i].Sources = nil
}
r, err := reachOfChange(facts, "novox/mesh-controller", []string{"README.md"}, "")
if err != nil {
t.Fatal(err)
}
if got := tiered(r.Plan.Tiers); got != "mesh-controller,route-proxy" {
t.Errorf("a snapshot without build sources: the gate planned %q for a README", got)
}
}
// **A plan says why each module is in it** (novox/hq ADR 0267, issue 363): the files of its build source the // **A plan says why each module is in it** (novox/hq ADR 0267, issue 363): the files of its build source the
// merge changed, or why it is read whole — never that it packages a repository as though that moved it. // merge changed, or why it is read whole — never that it packages a repository as though that moved it.
func TestAPlanSaysWhyEachModuleIsInIt(t *testing.T) { func TestAPlanSaysWhyEachModuleIsInIt(t *testing.T) {
-238
View File
@@ -1,238 +0,0 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// The plans building (novox/hq ADR 0483 clause 2, issue 1687).
//
// **A key per delivery plan in its build phase, so that the build seat takes no new check while a delivery builds.** The phase
// runs from the ask of the plan's first build until every build of its last tier is built, or the plan fails, stops or waits
// on the operator. It is derived from the plan alone (buildPhaseOf), so a controller that restarts finds the same answer. The
// key is put when the first build is asked (sayBuilding), kept current at every save of the plan (keepBuildingKey) and by a
// look every link.PlansBuildingRefresh while this controller holds the lease (keepingPlansBuilding), and deleted at the phase's
// end; the bucket's age removes the key of a controller that stopped, so checks are held at most that long.
//
// The layout is documented in internal/link/holds.go.
// buildPhaseOf says whether a plan is in its build phase: open, not waiting for the operator's word, with a build asked and a
// build of its tiers still to be built. A tier built and sent leaves the plan rolling while later tiers are still to be asked;
// it is in the phase still, which is what keeps a check from taking a place between two tiers.
func buildPhaseOf(p inventory.Plan) bool {
if !p.Open() || p.Waiting() || p.Release != nil {
return false
}
asked, unbuilt := false, false
for _, tier := range p.Tiers {
for _, m := range tier {
st := p.Modules[m]
if st != nil && st.State != "" {
asked = true
}
if st == nil || (st.State != "built" && st.State != planDeleted) {
unbuilt = true
}
}
}
return asked && unbuilt
}
// buildingSince is when the plan's first build was asked, or fallback when no module says.
func buildingSince(p inventory.Plan, fallback time.Time) time.Time {
since := time.Time{}
for _, st := range p.Modules {
if st != nil && st.AskedAt != nil && (since.IsZero() || st.AskedAt.Before(since)) {
since = *st.AskedAt
}
}
if since.IsZero() {
return fallback
}
return since
}
// buildingStore is the bucket of the plans building.
type buildingStore interface {
Put(ctx context.Context, plan string, value []byte) error
Delete(ctx context.Context, plan string) error
Keys(ctx context.Context) ([]string, error)
}
// syncBuilding makes the bucket say what the plans say: a key for each plan in its build phase, put again, and none for any
// other plan, deleted. It returns the faults, each once.
func syncBuilding(ctx context.Context, store buildingStore, plans []inventory.Plan, now time.Time) []error {
var faults []error
live := map[string]bool{}
for _, p := range plans {
if !buildPhaseOf(p) {
continue
}
live[p.ID] = true
if err := putBuilding(ctx, store, p.ID, buildingSince(p, now), now); err != nil {
faults = append(faults, err)
}
}
keys, err := store.Keys(ctx)
if err != nil {
return append(faults, err)
}
for _, key := range keys {
if live[key] {
continue
}
if err := store.Delete(ctx, key); err != nil {
faults = append(faults, fmt.Errorf("the key of %s could not be taken away: %w", key, err))
}
}
return faults
}
func putBuilding(ctx context.Context, store buildingStore, plan string, since, now time.Time) error {
body, _ := json.Marshal(link.PlanBuilding{Plan: plan, Since: since.UTC(), Seen: now.UTC()})
if err := store.Put(ctx, plan, body); err != nil {
return fmt.Errorf("the key of %s could not be put: %w", plan, err)
}
return nil
}
// busBuilding is the store on the controller's bucket.
type busBuilding struct {
conn *nats.Conn
mu sync.Mutex
kv jetstream.KeyValue
}
func (b *busBuilding) bucket(ctx context.Context) (jetstream.KeyValue, error) {
b.mu.Lock()
defer b.mu.Unlock()
if b.kv != nil {
return b.kv, nil
}
api, err := jetstream.New(b.conn)
if err != nil {
return nil, err
}
kv, err := api.KeyValue(ctx, broker.PlansBuildingBucket)
if err != nil {
return nil, fmt.Errorf("the plans building: %w", err)
}
b.kv = kv
return kv, nil
}
func (b *busBuilding) Put(ctx context.Context, plan string, value []byte) error {
kv, err := b.bucket(ctx)
if err != nil {
return err
}
_, err = kv.Put(ctx, plan, value)
return err
}
func (b *busBuilding) Delete(ctx context.Context, plan string) error {
kv, err := b.bucket(ctx)
if err != nil {
return err
}
if err := kv.Delete(ctx, plan); err != nil && !errors.Is(err, jetstream.ErrKeyNotFound) {
return err
}
return nil
}
func (b *busBuilding) Keys(ctx context.Context) ([]string, error) {
kv, err := b.bucket(ctx)
if err != nil {
return nil, err
}
keys, err := kv.Keys(ctx)
if errors.Is(err, jetstream.ErrNoKeysFound) {
return nil, nil
}
return keys, err
}
// buildingKeys is the store of the serving controller; nil in a process that is not it, which says nothing.
var buildingKeys atomic.Pointer[buildingStore]
func setBuildingStore(s buildingStore) {
if s == nil {
buildingKeys.Store(nil)
return
}
buildingKeys.Store(&s)
}
// sayBuilding puts a plan's key: its first build is being asked (or another of its builds). A key that could not be put is
// said; the look that follows puts it again.
func sayBuilding(ctx context.Context, plan string, since time.Time) {
s := buildingKeys.Load()
if s == nil {
return
}
if err := putBuilding(ctx, *s, plan, since, time.Now()); err != nil {
fmt.Printf("plans: %v; checks may take a place of %s until the next look\n", err, plan)
}
}
// keepBuildingKey is told each plan this controller keeps: its key put again while it builds and taken away once it does not.
func keepBuildingKey(ctx context.Context, p inventory.Plan) {
s := buildingKeys.Load()
if s == nil {
return
}
if buildPhaseOf(p) {
sayBuilding(ctx, p.ID, buildingSince(p, time.Now()))
return
}
if err := (*s).Delete(ctx, p.ID); err != nil {
fmt.Printf("plans: the key of %s could not be taken away; it ages out: %v\n", p.ID, err)
}
}
// keepingPlansBuilding looks at the plans every link.PlansBuildingRefresh for as long as this controller serves and holds the
// lease, so a key is put again before its age and the key of a plan that left the phase unseen is taken away.
func keepingPlansBuilding(ctx context.Context, inv *inventory.Inventory, conn *nats.Conn) {
store := &busBuilding{conn: conn}
setBuildingStore(store)
defer setBuildingStore(nil)
tick := time.NewTicker(broker.PlansBuildingRefresh)
defer tick.Stop()
failing := ""
for {
if epoch, err := theLease.epoch(ctx); err == nil && epoch != 0 {
why := ""
if plans, err := inv.OpenPlans(ctx); err != nil {
why = fmt.Sprintf("the plans cannot be read: %v", err)
} else if faults := syncBuilding(ctx, store, plans, time.Now()); len(faults) > 0 {
why = fmt.Sprint(faults)
}
if why != failing {
if why != "" {
fmt.Printf("plans: the keys of the plans building cannot be kept: %s\n", why)
} else {
fmt.Printf("plans: the keys of the plans building are kept again\n")
}
failing = why
}
}
select {
case <-ctx.Done():
return
case <-tick.C:
}
}
}
-176
View File
@@ -1,176 +0,0 @@
package main
import (
"context"
"encoding/json"
"sort"
"sync"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// Issue 1687 (ADR 0483 clause 2): a delivery plan's key lives from the ask of its first build until every build of its last
// tier is built, or the plan fails, stops or waits on the operator.
type fakeBuildingStore struct {
mu sync.Mutex
keys map[string]link.PlanBuilding
puts map[string]int
}
func newFakeBuildingStore() *fakeBuildingStore {
return &fakeBuildingStore{keys: map[string]link.PlanBuilding{}, puts: map[string]int{}}
}
func (f *fakeBuildingStore) Put(_ context.Context, plan string, value []byte) error {
f.mu.Lock()
defer f.mu.Unlock()
var v link.PlanBuilding
if err := json.Unmarshal(value, &v); err != nil {
return err
}
f.keys[plan] = v
f.puts[plan]++
return nil
}
func (f *fakeBuildingStore) Delete(_ context.Context, plan string) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.keys, plan)
return nil
}
func (f *fakeBuildingStore) Keys(context.Context) ([]string, error) {
f.mu.Lock()
defer f.mu.Unlock()
var out []string
for k := range f.keys {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
func aPlan(id, state string, tiers [][]string, states map[string]string) inventory.Plan {
p := inventory.Plan{ID: id, State: state, Tiers: tiers, Modules: map[string]*inventory.PlanModule{}}
now := time.Now().Add(-time.Minute)
for m, st := range states {
pm := &inventory.PlanModule{State: st}
if st != "" {
pm.AskedAt = &now
}
p.Modules[m] = pm
}
return p
}
func TestAPlanIsInItsBuildPhaseFromTheFirstAskToTheLastBuild(t *testing.T) {
tiers := [][]string{{"a", "b"}, {"c"}}
cases := []struct {
name string
plan inventory.Plan
want bool
}{
{"no build asked yet", aPlan("p", inventory.PlanBuilding, tiers, nil), false},
{"the first tier asked", aPlan("p", inventory.PlanBuilding, tiers, map[string]string{"a": "asked", "b": "asked"}), true},
// Between two tiers: the first built and sent (rolling), the second not asked yet.
{"between two tiers", aPlan("p", inventory.PlanRolling, tiers, map[string]string{"a": "built", "b": "built"}), true},
{"the last tier asked", aPlan("p", inventory.PlanBuilding, tiers, map[string]string{"a": "built", "b": "built", "c": "asked"}), true},
{"every build built", aPlan("p", inventory.PlanRolling, tiers, map[string]string{"a": "built", "b": "built", "c": "built"}), false},
// A module the plan deletes is done as far as its tiers go (release_plan.go), so it holds no key through the rollout.
{"a deleted module is done", aPlan("p", inventory.PlanRolling, tiers, map[string]string{"a": "built", "b": "built", "c": planDeleted}), false},
{"failed", aPlan("p", inventory.PlanFailed, tiers, map[string]string{"a": "failed", "b": "asked"}), false},
{"superseded", aPlan("p", inventory.PlanSuperseded, tiers, map[string]string{"a": "asked"}), false},
{"a batch whose window is open", aPlan("p", inventory.PlanAssembling, tiers, map[string]string{"a": "asked"}), false},
}
for _, c := range cases {
if got := buildPhaseOf(c.plan); got != c.want {
t.Errorf("%s: in the build phase = %v, want %v", c.name, got, c.want)
}
}
waiting := aPlan("p", inventory.PlanBuilding, tiers, map[string]string{"a": "asked"})
waiting.Delivery = &inventory.PlanDelivery{Awaits: "the operator's word"}
if buildPhaseOf(waiting) {
t.Error("a plan that waits on the operator is in its build phase")
}
}
// The key's life: put, refreshed, deleted at the phase's end; a plan that left the phase unseen loses its key at the next look.
func TestAPlanKeyIsPutRefreshedAndDeletedAtThePhasesEnd(t *testing.T) {
ctx := context.Background()
store := newFakeBuildingStore()
tiers := [][]string{{"a"}, {"b"}}
building := aPlan("plan-1", inventory.PlanBuilding, tiers, map[string]string{"a": "asked"})
if faults := syncBuilding(ctx, store, []inventory.Plan{building}, time.Now()); len(faults) > 0 {
t.Fatal(faults)
}
if _, ok := store.keys["plan-1"]; !ok {
t.Fatal("no key while the first build is asked")
}
first := store.keys["plan-1"]
// Refreshed: put again, the phase's start kept.
time.Sleep(2 * time.Millisecond)
syncBuilding(ctx, store, []inventory.Plan{building}, time.Now())
if store.puts["plan-1"] != 2 || !store.keys["plan-1"].Seen.After(first.Seen) || !store.keys["plan-1"].Since.Equal(first.Since) {
t.Errorf("the key was not refreshed with its start kept: %d puts, %+v then %+v", store.puts["plan-1"], first, store.keys["plan-1"])
}
// Between the tiers: still there.
between := aPlan("plan-1", inventory.PlanRolling, tiers, map[string]string{"a": "built"})
syncBuilding(ctx, store, []inventory.Plan{between}, time.Now())
if _, ok := store.keys["plan-1"]; !ok {
t.Error("the key was gone between two tiers")
}
// A key of another plan that is no longer in the phase.
store.keys["plan-0"] = link.PlanBuilding{Plan: "plan-0"}
done := aPlan("plan-1", inventory.PlanRolling, tiers, map[string]string{"a": "built", "b": "built"})
syncBuilding(ctx, store, []inventory.Plan{done}, time.Now())
if len(store.keys) != 0 {
t.Errorf("keys remain after the last build was built: %v", store.keys)
}
}
// keepBuildingKey, told of each save, puts the key while the plan builds and deletes it when the plan fails.
func TestASavedPlanPutsOrDeletesItsKey(t *testing.T) {
ctx := context.Background()
store := newFakeBuildingStore()
var s buildingStore = store
buildingKeys.Store(&s)
defer buildingKeys.Store(nil)
p := aPlan("plan-7", inventory.PlanBuilding, [][]string{{"a"}}, map[string]string{"a": "asked"})
keepBuildingKey(ctx, p)
if _, ok := store.keys["plan-7"]; !ok {
t.Fatal("the key was not put when the plan was saved building")
}
p.State = inventory.PlanFailed
keepBuildingKey(ctx, p)
if _, ok := store.keys["plan-7"]; ok {
t.Fatal("the key stayed after the plan failed")
}
}
// The plan's key is there before its first build is asked, so a check cannot take a place between the ask and the plan's save.
func TestThePlansKeyIsPutBeforeItsFirstBuildIsAsked(t *testing.T) {
asksOfEveryModule(t)
store := newFakeBuildingStore()
var s buildingStore = store
buildingKeys.Store(&s)
defer buildingKeys.Store(nil)
p, byName := aTierOfThree()
keyThere := false
was := askABuild
askABuild = func(context.Context, buildSource, string, string) (string, error) {
_, keyThere = store.keys["plan-1"]
return "build-x", nil
}
t.Cleanup(func() { askABuild = was })
askModule(t.Context(), p, "a", byName)
if !keyThere {
t.Fatal("the plan's key was not there when its first build was asked")
}
}
+3 -3
View File
@@ -149,11 +149,11 @@ func planWaitCommand(ctx context.Context, inv *inventory.Inventory, id, untilAsk
} }
// Said as `plans <id>` says its first line, and as `status` says the plan. // Said as `plans <id>` says its first line, and as `status` says the plan.
now, one := time.Now(), []inventory.Plan{p} now, one := time.Now(), []inventory.Plan{p}
bounds, pause := readTierBounds(ctx, inv, now), buildSeatPause(ctx, inv, one) bounds := readTierBounds(ctx, inv, now)
line := planLineSaid(p, now, pause, bounds.of(p.Repository), func(module string) []string { line := planLineSaid(p, now, bounds.of(p.Repository), func(module string) []string {
on, _ := inv.Running(ctx, module) on, _ := inv.Running(ctx, module)
return on return on
}, waitsOf(ctx, inv, one)) + heldSaid(p, heldNow(ctx, inv)) }, waitsOf(ctx, inv, one)) + heldSaid(p, heldNow(ctx, inv))
doc := planWaitedDoc(p, planStatuses(one, now, pause, bounds)[0], until, came, now.Sub(began), line) doc := planWaitedDoc(p, planStatuses(one, now, bounds)[0], until, came, now.Sub(began), line)
return json.NewEncoder(os.Stdout).Encode(doc) return json.NewEncoder(os.Stdout).Encode(doc)
} }
+1 -1
View File
@@ -94,7 +94,7 @@ func TestAWaitIsRefusedWhatItCannotDo(t *testing.T) {
// the plan and what ended the wait, and its output is the line a person reads — never JSON inside a string. // the plan and what ended the wait, and its output is the line a person reads — never JSON inside a string.
func TestAWaitForAPlanAnswersAsData(t *testing.T) { func TestAWaitForAPlanAnswersAsData(t *testing.T) {
p := followedPlan(7, inventory.PlanRolling, 1) p := followedPlan(7, inventory.PlanRolling, 1)
st := planStatuses([]inventory.Plan{p}, time.Now(), pauseView{}, tierBounds{})[0] st := planStatuses([]inventory.Plan{p}, time.Now(), tierBounds{})[0]
doc := planWaitedDoc(p, st, "tier", "tier", 3*time.Second, "rolling, tier 2 of 2") doc := planWaitedDoc(p, st, "tier", "tier", 3*time.Second, "rolling, tier 2 of 2")
stdout, _ := json.Marshal(doc) stdout, _ := json.Marshal(doc)
a := answerOf([]string{"plans", "wait", "plan-1"}, stdout, "", true) a := answerOf([]string{"plans", "wait", "plan-1"}, stdout, "", true)

Some files were not shown because too many files have changed in this diff Show More