The pipeline was observable from a merge to an artifact and went dark where it touched a machine: a node's report is control traffic only the control plane reads, so nothing said which version a machine runs, or that it refused to (novox/hq ADR 0134). The control plane now states both under the seat it holds — a role's events belong to the role and keep their address when the holder is replaced — and only when the report is news, because a machine reconciles every minute and a fact per report would be a fact per minute per machine. Whether a report is news is the store's answer: it holds the previous one, so the listener returns it and the server states the fact. That also gives the catch-up replay a subject the controller may publish: it was published as a module's event from a module called "control-plane", which does not exist, so the controller's own account refused it and every catalogue that asked what it missed was answered with nothing.
183 lines
8.7 KiB
Go
183 lines
8.7 KiB
Go
package link
|
|
|
|
import (
|
|
"context"
|
|
"crypto/rand"
|
|
"encoding/hex"
|
|
"encoding/json"
|
|
"fmt"
|
|
)
|
|
|
|
// Emitting a module event from Go.
|
|
//
|
|
// **Every event rides one topic exchange** (novox/hq ADR 0042), which is not the direct exchange
|
|
// nodes and the control plane speak over. A module that announces something publishes here, and
|
|
// consumers bind their own durable queue to a pattern over it.
|
|
//
|
|
// This exists because the builder is a module written in Go while every other emitter is
|
|
// TypeScript on the sdk. The envelope is the sdk's, reproduced exactly: the body is the payload
|
|
// alone and everything about the event travels as headers. A second shape would be a second thing
|
|
// for consumers to handle, and they are written against the first.
|
|
const (
|
|
// EventsExchange is where every event rides. Named here rather than imported from the broker
|
|
// package for the same reason BuildQueueName is duplicated there — one direction of dependency.
|
|
EventsExchange = "mesh.events"
|
|
)
|
|
|
|
// EmitEvent publishes one module event, in the envelope the sdk's consumers expect.
|
|
//
|
|
// The envelope is the transport's to write (bus.go) and this is only what goes in it, which is
|
|
// what lets one conformance fixture hold both implementations to the same headers.
|
|
func EmitEvent(ctx context.Context, bus Bus, eventType, source, node string, body any) error {
|
|
payload, err := json.Marshal(body)
|
|
if err != nil {
|
|
return fmt.Errorf("cannot serialise a %s event: %w", eventType, err)
|
|
}
|
|
// The publish is not confirmed by the caller: it has already done the work the event
|
|
// describes, and a build that succeeded must not be reported as failed because saying so
|
|
// failed. Each transport decides what "published" means for it.
|
|
return bus.PublishEvent(ctx, eventType, source, node, payload)
|
|
}
|
|
|
|
// eventID is what a consumer deduplicates on: delivery is at-least-once, so a handler must be able
|
|
// to tell a redelivery from a second event, and only the emitter can say which it is.
|
|
func eventID() (string, error) {
|
|
raw := make([]byte, 16)
|
|
if _, err := rand.Read(raw); err != nil {
|
|
return "", fmt.Errorf("cannot make an event id: %w", err)
|
|
}
|
|
return hex.EncodeToString(raw), nil
|
|
}
|
|
|
|
// MeshControllerSeat is the role the control plane holds, and therefore where its own facts live: a
|
|
// role's events belong to the role, not to whichever container is holding it today (novox/hq ADR 0121,
|
|
// ADR 0129). It is what makes them addressable while the control plane itself is being replaced.
|
|
const MeshControllerSeat = "mesh-controller"
|
|
|
|
// The facts the mesh states about its own work (novox/hq ADR 0134).
|
|
const (
|
|
// KeyApplied: a machine now runs what it was sent.
|
|
KeyApplied = "applied"
|
|
// KeyRefused: a machine did not take what it was sent, and why.
|
|
KeyRefused = "refused"
|
|
// KeyBuiltBefore: a build the mesh already held, for a catalogue that asked what it missed. Not
|
|
// `built` — that is the build machine's, said as it happens, and a replay is neither.
|
|
KeyBuiltBefore = "built-before"
|
|
)
|
|
|
|
// Applied is what a machine now runs, as the mesh states it.
|
|
type Applied struct {
|
|
Node string `json:"node"`
|
|
Declared string `json:"declared,omitempty"`
|
|
// Resources is how many the machine applied, not which: the list is the machine's own account
|
|
// of itself and belongs in the records, not in a fact every listener has to read past.
|
|
Resources int `json:"resources"`
|
|
}
|
|
|
|
// Refused is a machine that would not take what it was sent.
|
|
type Refused struct {
|
|
Node string `json:"node"`
|
|
Declared string `json:"declared,omitempty"`
|
|
Refused string `json:"refused,omitempty"`
|
|
Failed map[string]string `json:"failed,omitempty"`
|
|
}
|
|
|
|
// KeyModuleBuilt is what the builder announces when it has built something. The catalogue places
|
|
// it in the module graph; nothing else need care.
|
|
const KeyModuleBuilt = "module.builder.built"
|
|
|
|
// KeyModuleUpgraded is the catalogue saying a module's current version has moved.
|
|
//
|
|
// **The control plane hooks the meaning, not the build.** The builder says what it built; the
|
|
// catalogue decides whether that was an upgrade — a rebuild producing the commit already current
|
|
// is not one — and only this says anything the control plane can act on. Consuming the build
|
|
// directly would make the control plane re-derive a decision another module already made, and the
|
|
// two would eventually disagree (novox/hq ADR 0072).
|
|
const KeyModuleUpgraded = "module.mesh-catalog.upgraded"
|
|
|
|
// KeyCatchingUp is the catalogue saying it has just started and may have missed things.
|
|
//
|
|
// **A durable queue only keeps what arrived after it existed.** The catalogue's own queue is
|
|
// durable, so nothing is lost once it is running — but the modules built before it first ran were
|
|
// announced to a queue that did not exist yet, and on a fresh mesh those are, necessarily, the
|
|
// shared base, the store the catalogue runs on, and the catalogue itself. The graph's foundation
|
|
// is the part it never hears about (novox/hq 04-ISSUES/050).
|
|
//
|
|
// So it asks, and the control plane answers with what it recorded. Asking rather than being told
|
|
// because only the catalogue knows it has a gap; the control plane cannot tell a fresh catalogue
|
|
// from one that is merely quiet.
|
|
const KeyCatchingUp = "module.mesh-catalog.catching-up"
|
|
|
|
// CatchUpQueue is where that lands. Durable, for the same reason the upgrade queue is: a catalogue
|
|
// that started while the control plane was restarting is exactly the one with a gap to fill.
|
|
const CatchUpQueue = "control.catchup"
|
|
|
|
// UpgradeQueue is where those land. Durable and named, not a temporary queue: an upgrade announced
|
|
// while the control plane is restarting is exactly the one that must not be missed.
|
|
const UpgradeQueue = "control.upgrades"
|
|
|
|
// Replayer answers a catalogue that says it has just started.
|
|
//
|
|
// It is handed every build the mesh recorded, oldest first, and re-announces each. The catalogue
|
|
// registers them as history: a replayed build changed nothing in the world, so announcing it as an
|
|
// upgrade would have the mesh act on news that is years old.
|
|
type Replayer interface {
|
|
// Announceable is every build worth re-announcing, oldest first.
|
|
//
|
|
// It hands them back rather than publishing them: the wire belongs to this package, and a
|
|
// replay that built its own announcements could drift from what the builder emits — which is
|
|
// the one thing it must match exactly, because the catalogue has a single handler for both.
|
|
Announceable(ctx context.Context) ([]Announcement, error)
|
|
}
|
|
|
|
// Announcement is a build, in the shape the builder announces one.
|
|
//
|
|
// The field names are the wire's, not Go's, because a catalogue reads these and a rename here is
|
|
// an event nobody handles.
|
|
type Announcement struct {
|
|
Module string `json:"module"`
|
|
Commit string `json:"commit"`
|
|
Repository string `json:"repository"`
|
|
Path string `json:"path"`
|
|
Ref string `json:"ref"`
|
|
Manifest json.RawMessage `json:"manifest,omitempty"`
|
|
Against []string `json:"against,omitempty"`
|
|
Made []MadeArtifact `json:"made,omitempty"`
|
|
// Replay says this is history rather than news: it was built once, and this is the mesh
|
|
// telling a catalogue that missed it. A consumer registers it and announces nothing — an
|
|
// upgrade that happened months ago is not one anything should act on now.
|
|
Replay bool `json:"replay,omitempty"`
|
|
}
|
|
|
|
// Upgraded is what the catalogue says when a module's current version moves.
|
|
// SourceMoved is what the forge announces when a pull request is merged: which repository, into
|
|
// which branch, producing which commit. The mesh matches it against every module's recorded
|
|
// source and builds what moved, bases first.
|
|
type SourceMoved struct {
|
|
Owner string `json:"owner"`
|
|
Repo string `json:"repo"`
|
|
Base string `json:"base"`
|
|
Head string `json:"head"`
|
|
Commit string `json:"merge_commit_sha"`
|
|
CloneURL string `json:"clone_url"`
|
|
HTMLURL string `json:"html_url"`
|
|
// MergedAt is when the forge merged it, RFC 3339. What decides whether this is news.
|
|
MergedAt string `json:"merged_at"`
|
|
|
|
// Paths are the files the merge changed, from the repository's root. Empty means the forge said
|
|
// nothing about them, and every module built from the repository is treated as affected.
|
|
Paths []string `json:"paths,omitempty"`
|
|
|
|
// PathsTruncated says the merge changed more files than the forge was asked to list, so Paths is
|
|
// a beginning rather than the whole change — and again, everything is treated as affected. Said
|
|
// rather than inferred from a round number, because "this is all of it" and "this is as much as
|
|
// I asked for" are the difference between rebuilding a module and leaving it stale.
|
|
PathsTruncated bool `json:"paths_truncated,omitempty"`
|
|
}
|
|
|
|
type Upgraded struct {
|
|
Module string `json:"module"`
|
|
Commit string `json:"commit"`
|
|
Previous string `json:"previous"`
|
|
}
|