novox/mesh-controller
3
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Merge main: the trunk renamed the seats and made them data

Browse Source 
Both branches changed the seat set from the same starting point, so every number
collided and every `mesh-*` name existed twice. The trunk's numbers and names win:
this branch's records became 0129/0130 and its migrations 0037/0038, and the
hardcoded rename map gave way to the trunk's `seat_alias` table — a rename is a
row now (ADR 0122), not a recompile.

Three of my checks were wrong and the merge is what showed it:

A seat with an empty protocol is a marker, not an incomplete declaration. Most
node-scoped seats are markers — which module is this machine's packet filter —
and refusing one refused most of the set, the showcase module included. A
mistyped field name is already refused by the parser, so an empty protocol was
written as one deliberately.

A claim on a seat this manifest does not declare is not the parser's to judge. A
module may hold a seat another module declared; that is the whole reason ADR 0126
has callers name the seat and not its provider. Whether the seat exists is a fact
about the catalogue, so the refusal is at registration, where every declaration
is in view.

And a seat may share a name with the provision it delivers. `git`, the npm
registry and the artifact store still do, because renaming a delivering seat
cascades to every consumer requiring it, with a window where a holder stops
resolving mid-flight. The trunk deferred exactly those three on purpose.

Full suite green against a real NATS and store.
This commit is contained in:
jschoubben
2026-09-27 18:50:18 +02:00
parent dded086b54 e7b5100324
commit 497f8ea567
71 changed files with 2084 additions and 615 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/broker/agreement_catalogue_test.go
+1 -1
View File
@@ -67,7 +67,7 @@ func TestTheAgreementCheckCatchesASubscriptionThatMatchesNothing(t *testing.T) {
// An event published under a seat's name is real even though no module declares it as its own.
if bad := Disagreements(nil,
[]AConsumer{{Module: "watcher", Consumes: []string{"mesh-artifact-store.image.pushed"}}},
[]DeclaredSeat{{Name: "mesh-artifact-store", Emits: []string{"image.pushed"}}}); len(bad) != 0 {
[]DeclaredSeat{{Name: "the-artifact-store", Emits: []string{"image.pushed"}}}); len(bad) != 0 {
t.Fatalf("an event a seat emits was reported as matching nothing: %v", bad)
}
}
internal/catalogue/artifact_store_seat_test.go
+2 -2
View File
@@ -23,7 +23,7 @@ func TestASecondArtifactStoreAnywhereIsRefusedByName(t *testing.T) {
}
// A second one, on any other machine, is refused — and the refusal names the seat.
elsewhere := World{Held: []Held{{Claim: "mesh-artifact-store", Scope: ScopeMesh,
elsewhere := World{Held: []Held{{Claim: "the-artifact-store", Scope: ScopeMesh,
Node: "anchor", Module: "distribution"}}}
other := workstation()
other.Name = "laptop"
@@ -32,7 +32,7 @@ func TestASecondArtifactStoreAnywhereIsRefusedByName(t *testing.T) {
t.Fatal("a second store was accepted on another machine; it would offer artifact-store a " +
"second time and every consumer elsewhere would refuse to choose")
}
if !strings.Contains(err.Error(), "mesh-artifact-store") || !strings.Contains(err.Error(), "one per mesh") {
if !strings.Contains(err.Error(), "the-artifact-store") || !strings.Contains(err.Error(), "one per mesh") {
t.Fatalf("refused without naming the seat: %v", err)
}
}
internal/catalogue/broker_seat_test.go
+13 -16
View File
@@ -71,9 +71,9 @@ func TestTheAmqpBrokerDoesNotContendForTheSeat(t *testing.T) {
// "the package registry is served on <nil>", which does not say "you renamed an interface".
func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
for _, pair := range []struct{ seat, delivers string }{
{"mesh-git", "git"},
{"mesh-npm-package-registry", "npm-package-registry"},
{"mesh-artifact-store", "artifact-store"},
{"git", "git"},
{"npm-package-registry", "npm-package-registry"},
{"the-artifact-store", "artifact-store"},
{"mesh-store", "postgres-database"},
{"mesh-broker", "mesh-bus"},
} {
@@ -86,19 +86,16 @@ func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
"interface with it, and every consumer requiring it would stop resolving",
pair.seat, s.Delivers, pair.delivers)
}
if _, isSeat := SeatNamed(pair.delivers); isSeat {
t.Errorf("%q is both an interface and a seat name; one of the renames was incomplete",
pair.delivers)
}
// **Three of these deliberately share a name with what they deliver**, and that is not an
// incomplete rename. Renaming a seat that delivers a provision cascades to every consumer
// requiring it, with a mesh-wide window where a holder stops resolving mid-flight — so the
// trunk deferred exactly those three (novox/hq ADR 0121) while renaming the node-scoped ones.
// What this test is for is the other direction: that renaming a seat never moves the
// interface, which once produced "the package registry is served on <nil>".
}
}
// And a manifest written against an old seat name is told what it became, rather than refused as
// unknown — the courtesy the `needs`/`own-secrets` rename already sets.
func TestAnOldSeatNameSaysWhatItBecame(t *testing.T) {
m := Manifest{Module: "old", Claims: []Claim{{Name: "the-catalogue", Scope: ScopeMesh}}}
got := strings.Join(claimProblems(m), "; ")
if !strings.Contains(got, "the-catalogue") || !strings.Contains(got, "mesh-catalog") {
t.Fatalf("the refusal does not name both the old and the new: %q", got)
}
}
// A manifest written against an old seat name is told what it became rather than refused as
// unknown. **That map is the controller's store now, not this package** (novox/hq ADR 0122): a
// rename is a row, so the courtesy survives a rename nobody recompiled for. Checked where the
// table is read, not here, where there is no longer a hardcoded list to check against.
internal/catalogue/declaration.go
+19 -2
View File
@@ -104,6 +104,17 @@ type Rendering struct {
// its mounts (Manifest.BusUsers).
BusUsers string
// MeshRange is the private network's CIDR (the range node addresses are allocated from), for a
// module that must name the whole mesh rather than one machine — an intrusion filter that must
// never ban a tunnel peer, say. A per-mesh value the module cannot know, so it is carried here
// and offered as ${machine:mesh-range}, the same way one machine's address is.
MeshRange string
// Accounts is each machine's operator account, by the same internal name Names uses (novox/hq
// to-be 29). What an ssh Host block's `User` line is composed from; empty for a machine no
// operator account is known on.
Accounts map[string]string
// Kept is every operator-sealed secret in the mesh, for a module that `keeps` them. Nil when
// nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport
@@ -346,6 +357,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
"content": filtering, "mode": "0600",
})
}
// The node's fail2ban jails, composed from every module it runs (novox/hq to-be 31), written
// where the intrusion-prevention holder owns them. Like the rule set above: gathered from all
// modules, written by the one that holds the role.
if j := m.Jailing; j != nil {
first = append(first, jailsInto(r.Modules, j)...)
}
if c := m.Certificate; c != nil {
if with.Certificate == "" {
// Asked for and not issued. Refused rather than skipped: a module that serves TLS
@@ -581,7 +598,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// (novox/hq ADR 0112) — resolved once per module, named by ${dir:…} from any resource.
dirs := dirsFor(m, with)
// And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names)
thisMachine := machineFacts(r, with.Names, with.MeshRange)
// Which of this module's files carry a secret, for the rule that a container may not read
// one of them as its environment without saying so (ADR 0086, issue 041).
@@ -680,7 +697,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// plane's; making a name resolve is the module's software. Emitted as ordinary files under
// this module's name, so they are applied, reported and removed exactly as anything else
// it declares.
given, err := FactsInto(m, r, with.Names, with.Machines, with.Suffix)
given, err := FactsInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix)
if err != nil {
return nil, err
}
internal/catalogue/domain_test.go
+5 -5
View File
@@ -17,7 +17,7 @@ func networkingShelf(extra ...Manifest) map[string]Manifest {
{Module: "networking", Requires: []string{"private-network", "name-resolution"}},
{Module: "mesh-wireguard", Computed: "mesh-wireguard",
Provides: Offers("private-network", "mesh-addressing"),
Claims: []Claim{{Name: "mesh-private-network", Scope: ScopeNode}}},
Claims: []Claim{{Name: "the-private-network", Scope: ScopeNode}}},
{Module: "mesh-names", Computed: "mesh-names",
Provides: Offers("name-resolution"), Requires: []string{"mesh-addressing"}},
}
@@ -44,7 +44,7 @@ func TestASecondVPNTurnsItIntoAChoice(t *testing.T) {
// back under another name.
_, err := Resolve(networkingShelf(
Manifest{Module: "tailscale", Provides: Offers("private-network"),
Claims: []Claim{{Name: "mesh-private-network", Scope: ScopeNode}}},
Claims: []Claim{{Name: "the-private-network", Scope: ScopeNode}}},
), []string{"networking"}, workstation(), World{})
if err == nil {
@@ -62,7 +62,7 @@ func TestChoosingIsAssigning(t *testing.T) {
got, err := Resolve(networkingShelf(
Manifest{Module: "tailscale",
Provides: Offers("private-network", "name-resolution"),
Claims: []Claim{{Name: "mesh-private-network", Scope: ScopeNode}}},
Claims: []Claim{{Name: "the-private-network", Scope: ScopeNode}}},
), []string{"networking", "tailscale"}, workstation(), World{})
if err != nil {
@@ -85,13 +85,13 @@ func TestChoosingOneVPNCannotDragTheOtherBackIn(t *testing.T) {
// machine could run two VPNs for two purposes — but being *the* one the mesh runs over is.
_, err := Resolve(networkingShelf(
Manifest{Module: "tailscale", Provides: Offers("private-network"),
Claims: []Claim{{Name: "mesh-private-network", Scope: ScopeNode}}},
Claims: []Claim{{Name: "the-private-network", Scope: ScopeNode}}},
), []string{"networking", "tailscale"}, workstation(), World{})
if err == nil {
t.Fatal("a machine was given two private networks without being told")
}
if !strings.Contains(err.Error(), "mesh-private-network") {
if !strings.Contains(err.Error(), "the-private-network") {
t.Fatalf("the refusal does not say what collided: %v", err)
}
}
internal/catalogue/facts.go
-184
View File
@@ -1,184 +0,0 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// What only the mesh knows, written where a module asks for it.
//
// **The graph is the control plane's; using it is the module's.** The mesh knows which machines
// exist, what they are called, and where they are. Turning that into a name that resolves is
// somebody's software, and which software is a choice the mesh should not be making.
//
// This replaced three modules — names, a resolver's data, and the private network's own
// configuration — that existed only because computed output needed somewhere to live. They ran no
// software and could not be swapped for anything, which is the test of whether something is a
// module at all (novox/hq ADR 0040).
const (
// FactNodeNames is every machine's name and address, as a hosts file.
//
// Exact names only: `homer` and `homer.internal` resolve to homer. Anything *under* a machine
// is a wildcard, which a hosts file cannot express — that is FactNodeZones.
FactNodeNames = "node-names"
// FactNodeZones is every machine as a wildcard: `*.homer.internal` is homer.
//
// Written in the form a resolver reads. A machine's own name and everything under it are one
// fact — if homer is at an address, so is anything homer serves.
FactNodeZones = "node-zones"
)
// facts is every fact the mesh computes, and what writes it.
//
// **A closed list.** A module asking for a fact the mesh does not have is asking for a file nobody
// will write, and finding that out on a machine — as a daemon that starts, reads nothing, and
// answers no queries — is worse than being told where the manifest is.
// A fact is written from the names it is about. `every` is every name the mesh serves — machines
// and the names it was told to route; `machines` is only the machines. A fact takes the set it is
// true of, and the two must not be confused (novox/hq 04-ISSUES/111).
var facts = map[string]func(r Resolution, every, machines map[string]string, suffix string) string{
FactNodeNames: func(r Resolution, every, _ map[string]string, suffix string) string {
return nodeNames(r, every, suffix)
},
FactNodeZones: func(r Resolution, _, machines map[string]string, suffix string) string {
return nodeZones(r, machines, suffix)
},
}
// FactsInto renders the facts a module asked for, as files it will be given.
//
// The module owns everything after the file exists: loading it, restarting on it, what a resolver
// does with it. This only puts it there.
func FactsInto(m Manifest, r Resolution, addresses, machines map[string]string, suffix string) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
names := make([]string, 0, len(m.Facts))
for name := range m.Facts {
names = append(names, name)
}
sort.Strings(names)
out := make([]map[string]any, 0, len(names))
for _, name := range names {
write, known := facts[name]
if !known {
return nil, fmt.Errorf(
"%s asks the mesh for %q, which it does not compute. It has %s",
m.Module, name, spokenFacts())
}
path := m.Facts[name]
if !strings.HasPrefix(path, "/") {
return nil, fmt.Errorf(
"%s asks for %q at %q, which is not an absolute path", m.Module, name, path)
}
out = append(out, map[string]any{
"id": "fact-" + name, "type": "file", "path": path, "mode": "0644",
"content": write(r, addresses, machines, suffix),
})
}
return out, nil
}
// spokenFacts lists them, so a refusal says what would have worked.
func spokenFacts() string {
names := make([]string, 0, len(facts))
for name := range facts {
names = append(names, name)
}
sort.Strings(names)
return strings.Join(names, ", ")
}
// nodeNames is every machine's name and address, as a hosts file.
//
// **A machine with no address is left out.** The mesh has a record for it — somebody added it —
// and does not yet know where it is, which is the ordinary state between adding a machine and it
// joining. Writing the name anyway would give a name that resolves to nothing, and a connection to
// that hangs; leaving it out fails at once and says the name is unknown.
func nodeNames(r Resolution, addresses map[string]string, suffix string) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever a machine\n")
b.WriteString("# joins or leaves, and an edit would survive until then and vanish.\n\n")
// The floor every Linux expects, and which removing would break things that have nothing to do
// with the mesh.
b.WriteString("127.0.0.1\tlocalhost\n")
b.WriteString("::1\t\tlocalhost ip6-localhost ip6-loopback\n")
if r.Node != "" {
fmt.Fprintf(&b, "127.0.1.1\t%s\n", r.Node)
}
b.WriteString("\n")
for _, name := range sortedNames(addresses) {
at := addresses[name]
internal, bare := meshName(name, suffix)
// Its mesh name resolves to its address on the private network rather than to loopback,
// so a service binding the name it was given stays reachable from everywhere else.
fmt.Fprintf(&b, "%s\t%s\t%s", at, internal, bare)
if bare == r.Node {
b.WriteString("\t# this machine")
}
b.WriteString("\n")
}
return b.String()
}
// nodeZones is every machine as a wildcard, in the form a resolver reads.
//
// `*.homer.internal` is homer, which is the whole rule: if homer is at an address, so is anything
// homer serves. A module wanting this runs the resolver; the mesh only says what is true.
//
// **And the suffix itself, as a local domain.** A resolver that forwards what it cannot answer
// would otherwise send a mesh name it does not know — a machine that left, a typo — to a public
// resolver, which is a leak of the mesh's names for no answer. `local=` keeps everything under the
// suffix here: answered from the lines below or refused. Written in this file rather than in the
// resolver's own configuration because the suffix is the mesh's choice (the operator may have
// picked another) and this file is the one place the mesh writes what it chose.
func nodeZones(_ Resolution, addresses map[string]string, suffix string) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever a machine\n")
b.WriteString("# joins or leaves, and an edit would survive until then and vanish.\n\n")
fmt.Fprintf(&b, "local=/%s/\n", strings.TrimPrefix(suffixOr(suffix), "."))
for _, name := range sortedNames(addresses) {
internal, _ := meshName(name, suffix)
fmt.Fprintf(&b, "address=/%s/%s\n", internal, addresses[name])
}
return b.String()
}
// meshName is a machine's internal name and its bare one, from either. The control plane keys
// the names it hands a resolution by the internal name (`homer.internal`), the same map a
// container gets as its hosts; a caller that keys by the bare name gets the same answer. The
// suffix is the one the control plane composed those names with, handed down rather than written
// here a second time — the alternative was `homer.internal.internal` on every machine.
func meshName(name, suffix string) (internal, bare string) {
dotted := "." + strings.TrimPrefix(suffixOr(suffix), ".")
if strings.HasSuffix(name, dotted) {
return name, strings.TrimSuffix(name, dotted)
}
return name + dotted, name
}
// suffixOr is the suffix given, or the one the mesh composes names with when none was handed down.
// The one place the default is written in this file, so a fact and a name cannot disagree about it.
func suffixOr(suffix string) string {
if suffix == "" {
return "internal"
}
return suffix
}
func sortedNames(addresses map[string]string) []string {
out := make([]string, 0, len(addresses))
for name, at := range addresses {
// See nodeNames: a machine the mesh cannot place is left out rather than named at nothing.
if at == "" {
continue
}
out = append(out, name)
}
sort.Strings(out)
return out
}
internal/catalogue/facts_test.go
-188
View File
@@ -1,188 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// Keyed by the internal name, as the control plane hands them (issue 079).
var threeMachines = map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2", "bart.internal": ""}
// **`*.homer.internal` is homer. That is the whole rule.** And the suffix itself is local: a
// resolver that forwards what it cannot answer must not send a mesh name it does not know — a
// machine that left, a typo — to a public resolver (hal dnsmasq-app conversion, novox/hq
// 08-connectivity).
func TestEveryMachineIsAWildcardUnderItsOwnName(t *testing.T) {
out := nodeZones(Resolution{Node: "homer"}, threeMachines, "")
for _, want := range []string{
"local=/internal/",
"address=/homer.internal/10.42.0.1",
"address=/marge.internal/10.42.0.2",
} {
if !strings.Contains(out, want) {
t.Fatalf("missing %q:\n%s", want, out)
}
}
}
// A machine the mesh has a record for and cannot place is left out of both.
//
// **Not an oversight — the alternative is worse.** A name written with no address resolves to
// nothing, and a connection to that hangs. Leaving it out fails at once and says the name is
// unknown, which is a thing somebody can act on.
func TestAMachineWithNoAddressIsNotNamed(t *testing.T) {
for _, out := range []string{
nodeNames(Resolution{Node: "homer"}, threeMachines, ""),
nodeZones(Resolution{Node: "homer"}, threeMachines, ""),
} {
if strings.Contains(out, "bart") {
t.Fatalf("a machine with no address was named, so its name resolves to nothing:\n%s", out)
}
}
}
// A machine's own mesh name points at its address on the private network, not at loopback — or a
// service binding the name it was given is unreachable from everywhere else.
func TestAMachinesOwnNameIsItsMeshAddress(t *testing.T) {
out := nodeNames(Resolution{Node: "homer"}, threeMachines, "")
var line string
for _, l := range strings.Split(out, "\n") {
if strings.Contains(l, "homer.internal") {
line = l
}
}
if !strings.HasPrefix(line, "10.42.0.1") {
t.Fatalf("a machine's own mesh name is not its mesh address: %q", line)
}
// And the loopback floor is still there, or things with nothing to do with the mesh break.
if !strings.Contains(out, "127.0.0.1\tlocalhost") {
t.Fatalf("the loopback floor was removed:\n%s", out)
}
}
// A module says where it wants a fact, and is given a file.
func TestAModuleIsGivenTheFactsItAskedFor(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{FactNodeZones: "/etc/mesh/zones.conf"}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 1 {
t.Fatalf("expected one file, got %d", len(given))
}
if given[0]["path"] != "/etc/mesh/zones.conf" || given[0]["type"] != "file" {
t.Fatalf("not written where it was asked for: %v", given[0])
}
if !strings.Contains(given[0]["content"].(string), "homer.internal") {
t.Fatalf("the file does not hold the fact: %v", given[0]["content"])
}
}
// **Asking for a fact the mesh does not have is refused here, not on a machine.** A daemon that
// starts, reads a file nobody wrote, and answers no queries is a much worse way to find out.
func TestAskingForAFactTheMeshDoesNotHaveIsRefused(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{"the-weather": "/etc/weather"}}
_, err := FactsInto(m, Resolution{}, nil, nil, "")
if err == nil {
t.Fatal("a module asked for something nobody computes and was given nothing, silently")
}
for _, known := range []string{FactNodeNames, FactNodeZones} {
if !strings.Contains(err.Error(), known) {
t.Fatalf("the refusal does not say what would have worked: %v", err)
}
}
}
// And a relative path is refused, or a module decides where the mesh writes on a machine.
func TestAFactMustBeAskedForAtAnAbsolutePath(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{FactNodeNames: "etc/hosts"}}
if _, err := FactsInto(m, Resolution{}, nil, nil, ""); err == nil {
t.Fatal("a relative path was accepted")
}
}
// **The names the control plane hands a resolution are already internal names** — `homer.internal`,
// the same map every container gets as its hosts. Appending the suffix again wrote
// `homer.internal.internal` into every hosts file and every resolver's zones, and the large mesh
// bed's name test was the first to read it back. Either key gives the same files.
func TestNamesKeyedByInternalNameAreNotSuffixedTwice(t *testing.T) {
internal := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
bare := map[string]string{"homer": "10.42.0.1", "marge": "10.42.0.2"}
if a, b := nodeZones(Resolution{Node: "homer"}, internal, ""), nodeZones(Resolution{Node: "homer"}, bare, ""); a != b {
t.Fatalf("the zones differ by how the names were keyed:\n%s\n---\n%s", a, b)
}
if a, b := nodeNames(Resolution{Node: "homer"}, internal, ""), nodeNames(Resolution{Node: "homer"}, bare, ""); a != b {
t.Fatalf("the hosts differ by how the names were keyed:\n%s\n---\n%s", a, b)
}
zones := nodeZones(Resolution{Node: "homer"}, internal, "")
if strings.Contains(zones, "internal.internal") || !strings.Contains(zones, "address=/homer.internal/10.42.0.1") {
t.Fatalf("the zones carry a doubled suffix or miss the name:\n%s", zones)
}
hosts := nodeNames(Resolution{Node: "homer"}, internal, "")
if !strings.Contains(hosts, "10.42.0.1\thomer.internal\thomer\t# this machine") {
t.Fatalf("the hosts line for the machine itself is not name, bare name and the mark:\n%s", hosts)
}
}
// The suffix the control plane composed the names with is the one the facts write — an operator
// who chose another does not get `.internal` appended to it.
func TestTheFactsWriteTheSuffixTheNamesWereComposedWith(t *testing.T) {
names := map[string]string{"homer.lan": "10.42.0.1"}
zones := nodeZones(Resolution{Node: "homer"}, names, "lan")
if !strings.Contains(zones, "address=/homer.lan/10.42.0.1") || strings.Contains(zones, "internal") {
t.Fatalf("the zones do not carry the operator's suffix as given:\n%s", zones)
}
if !strings.Contains(zones, "local=/lan/") {
t.Fatalf("the local domain is not the operator's suffix, so its names would leak upstream:\n%s", zones)
}
hosts := nodeNames(Resolution{Node: "homer"}, names, "lan")
if !strings.Contains(hosts, "10.42.0.1\thomer.lan\thomer\t# this machine") {
t.Fatalf("the hosts line does not carry the operator's suffix as given:\n%s", hosts)
}
}
// novox/hq 04-ISSUES/111: the map the control plane hands a resolution holds every name the mesh
// serves — the machines, and the names it was told to route to whichever machine serves them. A
// container's hosts wants all of it. A resolver's zones want only the machines: told the mesh's
// suffix is its own, it answers authoritatively for everything under it and forwards nothing, so a
// routed name written there with the suffix appended is a name nobody will ever ask for, standing
// beside the machines and looking as real.
func TestTheResolverIsToldTheMachinesAndNotTheNamesTheMeshMerelyServes(t *testing.T) {
machines := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
every := map[string]string{
"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2",
"drive.example.test": "10.42.0.1", "git.example.test": "10.42.0.2",
}
m := Manifest{Module: "resolver", Facts: map[string]string{
FactNodeZones: "/etc/zones.conf", FactNodeNames: "/etc/hosts",
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, every, machines, "")
if err != nil {
t.Fatal(err)
}
by := map[string]string{}
for _, f := range given {
by[f["path"].(string)] = f["content"].(string)
}
zones := by["/etc/zones.conf"]
for _, machine := range []string{"address=/homer.internal/10.42.0.1", "address=/marge.internal/10.42.0.2"} {
if !strings.Contains(zones, machine) {
t.Fatalf("the resolver was not told %q:\n%s", machine, zones)
}
}
for _, served := range []string{"drive.example.test", "git.example.test"} {
if strings.Contains(zones, served) {
t.Fatalf("the resolver was told %q, a name the mesh serves rather than a machine:\n%s", served, zones)
}
}
// And the hosts file is the other way about: every name, so a container reaching a routed name
// finds the machine serving it.
hosts := by["/etc/hosts"]
for _, name := range []string{"homer.internal", "drive.example.test", "git.example.test"} {
if !strings.Contains(hosts, name) {
t.Fatalf("a container would not resolve %q from its hosts:\n%s", name, hosts)
}
}
}
internal/catalogue/foundation_manifests_test.go
+2 -2
View File
@@ -124,7 +124,7 @@ func TestTheForgesPortIsGivenLikeAnyOtherProvidersPort(t *testing.T) {
// unused.
func TestTheBuilderRequiresTheRegistryTheNpmSeatDelivers(t *testing.T) {
builder := catalogueManifest(t, "builder")
seat, _ := SeatNamed("mesh-npm-package-registry")
seat, _ := SeatNamed("npm-package-registry")
var requires bool
for _, r := range builder.Requires {
requires = requires || r == seat.Delivers
@@ -150,7 +150,7 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
for _, c := range forge.Claims {
holds[c.Name] = true
}
for _, seat := range []string{"mesh-npm-package-registry", "mesh-git"} {
for _, seat := range []string{"npm-package-registry", "git"} {
if !holds[seat] {
t.Errorf("gitea does not claim the %s seat: %+v", seat, forge.Claims)
}
internal/catalogue/hosts_region_test.go
+106
View File
@@ -0,0 +1,106 @@
package catalogue_test
import (
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/overlay"
)
// novox/hq issue 128, held where the host will see it: the shipped networking module's names,
// through the whole composition, and not only through FactsInto.
//
// Composition prefixes a fact's id with the module that asked for it and passes everything else
// through; a step that dropped `into` on the way would send the region as a whole file, and the
// host would write the machine's hosts file over again with every unit test above still green.
// onTheNetwork stands in for the overlay's generator: the node is part of the private network,
// and what the generator writes is not what is under test here.
type onTheNetwork struct{}
func (onTheNetwork) Resources(string) ([]map[string]any, bool, error) {
return []map[string]any{{"id": "overlay-config", "type": "file",
"path": "/etc/wireguard/mesh0.conf", "mode": "0600", "content": "[Interface]\n"}}, true, nil
}
func TestTheHostsRegionArrivesAsTheHostWillReadIt(t *testing.T) {
shelf := provided(t)
// A resolver restarting on the names another module put on the machine, and one resource it
// only runs at start — neither of which composition has any business changing.
resolver, err := catalogue.ParseManifest([]byte(`{
"module": "resolver", "version": "1", "requires": ["mesh-addressing"],
"resources": [
{"id": "seed", "type": "file", "path": "/etc/resolver/seed", "mode": "0644",
"content": "seed\n", "at": "start"},
{"id": "daemon", "type": "service", "unit": "resolver.service", "state": "running",
"restart-on": ["seed", "mesh-wireguard.fact-node-names"]}
]}`))
if err != nil {
t.Fatal(err)
}
shelf[resolver.Module] = resolver
got, err := catalogue.Resolve(shelf, []string{overlay.Domain, "resolver"},
catalogue.Node{Name: "homer", At: "homer.internal"}, catalogue.World{})
if err != nil {
t.Fatal(err)
}
names := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
out, err := got.Declaration(catalogue.Rendering{
Names: names, Machines: names, Suffix: "internal",
Generators: map[string]catalogue.Generator{overlay.Name: onTheNetwork{}},
})
if err != nil {
t.Fatal(err)
}
ids := map[string]map[string]any{}
for _, r := range out {
ids[r["id"].(string)] = r
}
hosts := ids[overlay.Name+".fact-node-names"]
if hosts == nil {
t.Fatalf("no names reached the machine; the declaration has %v", keys(ids))
}
if hosts["path"] != "/etc/hosts" || hosts["into"] != "block" {
t.Fatalf("the hosts file is not written into as a region: %v", hosts)
}
content := hosts["content"].(string)
if !strings.Contains(content, "10.42.0.1\thomer.internal\thomer\t# this machine\n") {
t.Errorf("the region does not name the machine:\n%s", content)
}
for _, floor := range []string{"Generated by the mesh", "localhost", "127.0.1.1"} {
if strings.Contains(content, floor) {
t.Errorf("the region carries %q, which is the machine's:\n%s", floor, content)
}
}
// The resolver's reference to it still names a resource the host will be sent.
daemon := ids["resolver.daemon"]
if daemon == nil {
t.Fatalf("the resolver's service was not composed: %v", keys(ids))
}
for _, named := range daemon["restart-on"].([]any) {
if ids[named.(string)] == nil {
t.Errorf("the resolver restarts on %v, which is nothing the host is sent", named)
}
}
if !reflect.DeepEqual(daemon["restart-on"], []any{"resolver.seed", overlay.Name + ".fact-node-names"}) {
t.Errorf("restart-on is %v", daemon["restart-on"])
}
// And a resource's own `at` passes through as the manifest wrote it.
if seed := ids["resolver.seed"]; seed == nil || seed["at"] != "start" {
t.Errorf("a resource's at did not survive composition: %v", seed)
}
}
func keys(m map[string]map[string]any) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
return out
}
internal/catalogue/jails_into.go
+62
View File
@@ -0,0 +1,62 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// A node's fail2ban jails, composed from the modules it runs (novox/hq to-be 31).
//
// **The same shape as the firewall.** Every module's `listens` become the node's rule set; every
// module's `jails` become the node's fail2ban config. A module that runs an authenticating service
// declares what a break-in on it looks like and how to ban it, naming no node and no path (ADR
// 0112); the intrusion-prevention holder — the one module with `jailing` — gathers them and writes
// them where it owns. A node not running a module has none of its jails.
// jailsInto composes every jail declared by the modules on a node into the files the holder writes:
// one jail file (all stanzas, so the fail2ban service restarts on a single resource) and one filter
// file per jail (its failregex, which fail2ban references by the jail's name).
//
// Owned by the holder, because the directory is: two modules writing into one fail2ban is the
// collision the holder model exists to prevent. Empty when nothing declares a jail — then the file
// is written empty rather than absent, so removing the last jail is an ordinary change the service
// restarts on rather than a file that vanishes.
func jailsInto(modules []Manifest, j *Jailing) []map[string]any {
type declared struct {
module string
jail Jail
}
var jails []declared
for _, m := range modules {
for _, jail := range m.Jails {
jails = append(jails, declared{m.Module, jail})
}
}
// A stable order the host applies as given (ADR 0005), and so the same set composes byte for
// byte every time rather than differing by map iteration.
sort.Slice(jails, func(a, b int) bool { return jails[a].jail.Name < jails[b].jail.Name })
var composed strings.Builder
composed.WriteString("# The mesh's jails, composed from the modules this node runs. Do not edit —\n")
composed.WriteString("# replaced whenever the node's modules change (novox/hq to-be 31).\n")
out := make([]map[string]any, 0, len(jails)+1)
for _, d := range jails {
fmt.Fprintf(&composed, "\n# from %s\n[%s]\nenabled = true\nfilter = %s\n%s\n",
d.module, d.jail.Name, d.jail.Name, strings.TrimRight(d.jail.Jail, "\n"))
// The filter is a file of its own, named as the jail's filter= references it.
out = append(out, map[string]any{
"id": "filter-" + d.jail.Name,
"type": "file", "path": strings.TrimRight(j.FilterInto, "/") + "/" + d.jail.Name + ".conf",
"mode": "0644",
"content": "# Generated by the mesh (from module " + d.module + "). Do not edit.\n" +
"[Definition]\nfailregex = " + d.jail.Failregex + "\n",
})
}
// The one jail file, first, with the fixed id the fail2ban service names in its restart-on.
return append([]map[string]any{{
"id": ComposedJailsID(), "type": "file", "path": j.Into, "mode": "0644",
"content": composed.String(),
}}, out...)
}
internal/catalogue/jails_into_test.go
+46
View File
@@ -0,0 +1,46 @@
package catalogue
import (
"strings"
"testing"
)
// A node's fail2ban jails are composed from the modules it runs (novox/hq to-be 31): the holder
// (jailing) gathers every module's declared jail into one jail file and a filter file per jail.
func TestJailsAreComposedFromTheNodesModules(t *testing.T) {
modules := []Manifest{
{Module: "fail2ban", Jailing: &Jailing{Into: "/etc/fail2ban/jail.d/mesh-composed.conf", FilterInto: "/etc/fail2ban/filter.d"}},
{Module: "postgres", Jails: []Jail{{Name: "postgres-auth", Failregex: "auth failed from <HOST>", Jail: "port = 5432\nmaxretry = 5"}}},
}
files := jailsInto(modules, modules[0].Jailing)
by := map[string]map[string]any{}
for _, f := range files {
by[f["id"].(string)] = f
}
jail := by[ComposedJailsID()]
if jail == nil || jail["path"] != "/etc/fail2ban/jail.d/mesh-composed.conf" {
t.Fatalf("the composed jail file was not written: %v", jail)
}
body := jail["content"].(string)
if !strings.Contains(body, "[postgres-auth]") || !strings.Contains(body, "filter = postgres-auth") ||
!strings.Contains(body, "port = 5432") {
t.Fatalf("the postgres jail stanza was not composed in:\n%s", body)
}
filter := by["filter-postgres-auth"]
if filter == nil || filter["path"] != "/etc/fail2ban/filter.d/postgres-auth.conf" {
t.Fatalf("the jail's filter file was not written: %v", filter)
}
if !strings.Contains(filter["content"].(string), "failregex = auth failed from <HOST>") {
t.Fatalf("the failregex was not written: %v", filter["content"])
}
}
// A holder whose node runs no jail-declaring module still gets the file, empty — so removing the
// last jail is a change the service restarts on, not a file that vanishes.
func TestTheComposedJailFileIsWrittenEvenWhenEmpty(t *testing.T) {
files := jailsInto([]Manifest{{Module: "fail2ban"}}, &Jailing{Into: "/x", FilterInto: "/f"})
if len(files) != 1 || files[0]["id"] != ComposedJailsID() {
t.Fatalf("the empty composed jail file was not written alone: %v", files)
}
}
internal/catalogue/machine_into_files.go
+44 -17
View File
@@ -57,7 +57,7 @@ func machineUsed(content string) []string {
// the hosts file and the resolver's wildcards are written from, so a file naming the machine's
// address and the file every other machine reaches it by cannot disagree. Absent, like `at`, when
// the machine is off the network or the mesh has not placed it.
func machineFacts(r Resolution, names map[string]string) map[string]string {
func machineFacts(r Resolution, names map[string]string, meshRange string) map[string]string {
out := map[string]string{"name": r.Node}
if r.At != "" {
out["at"] = r.At
@@ -65,32 +65,59 @@ func machineFacts(r Resolution, names map[string]string) map[string]string {
out["address"] = address
}
}
// The private network's whole range — a mesh-wide fact, not this machine's, but named here
// because a module cannot know it and sometimes must (an intrusion filter that must never ban a
// tunnel peer). Absent when the mesh has no range to give.
if meshRange != "" {
out["mesh-range"] = meshRange
}
// The operator's login on this machine and where its home is (novox/hq to-be 29), so a module
// that writes operator config names the account and its home rather than a value it cannot know.
// Absent when no operator account is known — a headless box a person never logs into.
if r.Account != "" {
out["account"] = r.Account
out["account-home"] = accountHomeOf(r.Account, r.AccountHome)
}
return out
}
// accountHomeOf is where an account's home is: what was stored, or the derived default — /root for
// root, /home/<account> otherwise. The one place the default is written, so a fact and the store
// cannot disagree about it.
func accountHomeOf(account, home string) string {
if home != "" {
return home
}
if account == "root" {
return "/root"
}
return "/home/" + account
}
// machineInto replaces a file's ${machine:…} placeholders with what the mesh knows about the
// machine the module was assigned to.
//
// A key the mesh does not hold is refused, for the same reason a binding's is: left alone, the
// literal would be written into a configuration file and read as a value.
func machineInto(resource map[string]any, facts map[string]string, module string) error {
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
content, ok := resource["content"].(string)
if !ok {
return nil
}
for _, key := range machineUsed(content) {
value, has := facts[key]
if !has {
return fmt.Errorf(
"%s has a file that says ${machine:%s}, and this machine says %s",
module, key, orNothing(namesOfFacts(facts)))
// Content, and now the path and owner too: a module that writes into a person's home names it
// with ${machine:account-home} and ${machine:account}, which it cannot know until assigned
// (novox/hq to-be 29), the same reason its content names ${machine:address}.
for _, field := range []string{"path", "owner", "content"} {
s, ok := resource[field].(string)
if !ok {
continue
}
for _, key := range machineUsed(s) {
value, has := facts[key]
if !has {
return fmt.Errorf(
"%s has a %s that says ${machine:%s}, and this machine says %s",
module, field, key, orNothing(namesOfFacts(facts)))
}
s = strings.ReplaceAll(s, fmt.Sprintf("${machine:%s}", key), value)
resource[field] = s
}
resource["content"] = strings.ReplaceAll(
content, fmt.Sprintf("${machine:%s}", key), value)
content = resource["content"].(string)
}
return nil
}
internal/catalogue/machine_into_files_test.go
+23
View File
@@ -103,3 +103,26 @@ func TestAModuleNamesTheAddressBehindItsMachinesName(t *testing.T) {
t.Fatalf("a machine off the network was given an address, or refused for another reason: %v", err)
}
}
// The mesh's private range is offered as ${machine:mesh-range}, so a module names it rather than
// hardcoding a value it cannot know (novox/hq ADR 0112) — the fail2ban ignoreip is the case.
func TestAModuleNamesTheMeshRange(t *testing.T) {
facts := machineFacts(Resolution{Node: "anchor", At: "anchor.internal"},
map[string]string{"anchor.internal": "10.10.0.1"}, "10.10.0.0/24")
if facts["mesh-range"] != "10.10.0.0/24" {
t.Fatalf("the mesh range is not a machine fact: %v", facts)
}
res := map[string]any{"type": "file", "id": "jail", "content": "ignoreip = 127.0.0.1/8 ${machine:mesh-range}\n"}
if err := machineInto(res, facts, "fail2ban"); err != nil {
t.Fatal(err)
}
if got := res["content"].(string); !strings.Contains(got, "10.10.0.0/24") || strings.Contains(got, "${machine:") {
t.Fatalf("the mesh range was not written in: %q", got)
}
// A mesh with no range gives no such fact, and a file that names it is refused rather than
// left with a literal placeholder in it.
none := machineFacts(Resolution{Node: "anchor"}, nil, "")
if _, has := none["mesh-range"]; has {
t.Fatal("a mesh with no range still offered one")
}
}
internal/catalogue/manifest.go
+66 -15
View File
@@ -206,10 +206,18 @@ type Manifest struct {
// that every new module would force its predecessors to update.
Claims []Claim `json:"claims,omitempty"`
// Seats this module declares of its own, with their protocols (novox/hq ADR 0118). The set
// of seats a mesh has is the mesh's own plus these, derived from what is registered rather
// than written in the controller — closed, and extensible without changing the mesh.
Seats []SeatDeclaration `json:"seats,omitempty"`
// DefinesSeats are the seats this module defines for itself, with their protocols
// (novox/hq ADR 0121, ADR 0129). The control plane defines the system seats — `mesh-*` and
// `node-*` — and a module may define its own, named outside that namespace, to coordinate its
// own instances: the mesh enforces one-holder-per-scope for it without knowing what it means. A
// module's declared seat is the only non-system name it may then claim; a claim to a name
// neither the mesh nor the module defines is refused.
//
// **Two lines of work built this at once**, one calling it `Seats` with a protocol and one
// `DefinesSeats` without. Same key in the file, so no manifest is affected: this is the trunk's
// name with the richer type, because what a role accepts, emits and serves is what lets the mesh
// check that a holder answers what its seat promises.
DefinesSeats []SeatDeclaration `json:"seats,omitempty"`
// Uses are seats this module sends to. It names the *seat*, never the module holding it, so
// the implementation can be replaced under it and no caller changes. A caller gets publish
@@ -392,6 +400,14 @@ type Manifest struct {
// that could only see its own ports would write a rule set that closed everything else.
Filtering *Filtering `json:"filtering,omitempty"`
// Jails are the fail2ban jails this module declares for its own service (novox/hq to-be 31).
// Written into whichever node runs the module, the same way `listens` become that node's rules.
Jails []Jail `json:"jails,omitempty"`
// Jailing marks the module that composes the node's fail2ban jails — the intrusion-prevention
// holder. Like Filtering: one module per node gathers what every module declared and writes it.
Jailing *Jailing `json:"jailing,omitempty"`
// Guards are ports of this module's the mesh refuses on an adopted node except from the
// private network and from the machine itself (novox/hq ADR 0100) — the store's port and the
// broker's management port. The ports the software uses; the mesh guards where the machine
@@ -400,24 +416,29 @@ type Manifest struct {
// firewall does. Ignored on a converged node, whose derived filter already closes them.
Guards []int `json:"guards,omitempty"`
// Facts are things only the mesh knows, written where this module asks for them.
// Facts are things only the mesh knows, written where this module asks for them — in the
// module's own format.
//
// **The graph is the control plane's; how a machine uses it is the module's.** The mesh knows
// which machines exist, what they are called and where they are. Making a name resolve, or a
// peer reachable, is somebody's software — dnsmasq, a resolver, a VPN — and the mesh has no
// business shipping one, choosing which, or knowing its configuration language.
// **The graph is the control plane's; the format is the module's.** The mesh knows which
// machines exist, what they are called and where they are. Turning that into a name that
// resolves, a peer that is reachable, a host a client trusts, is somebody's software — dnsmasq,
// a resolver, a VPN, ssh — in its own configuration language, and the mesh has no business
// knowing it. So a module gives a path and a template; the mesh renders the roster through it
// and owns nothing of what the file says.
//
// So a module says *put the node names here* and owns everything after that. The same shape as
// `filtering`, generalised: a fact, and a path.
// This used to be a closed list of fact names, each formatted in Go in the control plane, so a
// new consumer meant a new formatter here in the consumer's language. Now the data is the mesh's
// and the format is the module's: the two built-in cases — the network module's `/etc/hosts` and
// dnsmasq's zones — render through the same template path any module uses, and no format lives
// in the control plane at all. See RosterFile for what a template sees.
//
// It replaces three modules that existed only because computed output needed somewhere to
// live — they ran no software, could not be swapped for anything, and appeared in the graph as
// modules while being a data channel wearing a costume.
//
// Keyed by fact name; the names are a closed list, because a module asking for one the mesh
// does not compute is asking for something nobody will write, and finding that out on a machine
// is worse than being told here.
Facts map[string]string `json:"facts,omitempty"`
// Keyed by a name the module chooses, which is the rendered file's id (`fact-<name>`) — what a
// `restart-on` names to restart when the roster changes.
Facts map[string]RosterFile `json:"facts,omitempty"`
// Certificate is where this module wants a certificate for its machine's name inside the
// mesh, and where the key that goes with it can be found.
@@ -650,6 +671,36 @@ func (l Listening) At() string {
return l.Protocol
}
// Jail is a fail2ban jail a module declares for its own service (novox/hq to-be 31).
//
// **The module names no node and no path** (ADR 0112): it says what a break-in on its service looks
// like — the failregex — and the jail's own keys (the port it watches, where it logs, how many
// tries, how long to ban). The mesh writes it into whichever node's fail2ban runs the module, the
// same way a module's `listens` become that node's firewall rules. A node not running the module
// has no such jail.
type Jail struct {
// Name is the jail and its filter, e.g. "postgres-auth". One holder of the name per node.
Name string `json:"name"`
// Failregex is what a failed authentication looks like in the service's log — the filter.
Failregex string `json:"failregex"`
// Jail is the body of the jail's stanza: the keys under [<name>] the module knows and the mesh
// does not — the port it watches, its logpath and backend, maxretry, bantime.
Jail string `json:"jail"`
}
// Jailing says a module composes the node's fail2ban jails — the intrusion-prevention holder. Like
// Filtering for the firewall: one module gathers what every other module declared and writes it
// where it owns. Into is the one jail file the stanzas are composed into (so the fail2ban service
// can restart on a single resource); FilterInto is the directory each jail's filter file goes in.
type Jailing struct {
Into string `json:"into"`
FilterInto string `json:"filter-into"`
}
// ComposedJailsID is the single jail file the mesh composes every declared jail into, so the
// fail2ban service names one resource in its restart-on and a jail added or removed reaches it.
func ComposedJailsID() string { return "composed-jails" }
// Filtering says where a module wants the computed rule set.
type Filtering struct {
// Into is the path to write it to. Whatever loads it is this module's own business — an
internal/catalogue/provided_test.go
+1 -1
View File
@@ -54,7 +54,7 @@ func TestTheShippedNetworkingModulesResolveOnTheirOwn(t *testing.T) {
// network is what gives a machine a name, so the provider asks for the node-names fact and
// there is nothing else to bring in. A module that ran nothing used to be here.
for _, m := range got.Modules {
if m.Module == overlay.Name && m.Facts["node-names"] == "" {
if m.Module == overlay.Name && m.Facts["node-names"].Path == "" {
t.Fatalf("the network's provider does not ask for the names: %+v", m.Facts)
}
}
internal/catalogue/resolve.go
+10
View File
@@ -28,6 +28,10 @@ type Node struct {
// (novox/hq ADR 0066). A route contribution carries only a label — the subdomain — and the mesh
// joins <label>.<public-domain> to make the name it grants, interpreting neither half.
PublicDomain string
// Account is the operator's login on this machine, AccountHome where its home is (novox/hq
// to-be 29). What a home-scoped file is owned by and what ${machine:account} resolves to.
Account string
AccountHome string
}
// World is what the rest of the mesh already has.
@@ -114,6 +118,11 @@ type Resolution struct {
// (novox/hq ADR 0066). Carried from the node so that composing <label>.<public-domain> for a
// route contribution needs no store lookup here — the join is a fact about this one machine.
PublicDomain string
// Account and AccountHome are the operator's login on this machine and where its home is
// (novox/hq to-be 29), carried from the node so a home-scoped file's owner and path resolve
// here without a store lookup.
Account string
AccountHome string
// Modules in the order they were resolved: assigned first, then what they pulled in.
Modules []Manifest
@@ -541,6 +550,7 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
}
resolution := Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome,
Because: because, Needs: needs, Unhostable: unhostable}
for _, n := range providersFirst(order, catalogue) {
resolution.Modules = append(resolution.Modules, catalogue[n])
internal/catalogue/resolver_manifests_test.go
+2 -2
View File
@@ -50,7 +50,7 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
"\nlisten-address=127.0.0.1\n", "\ninterface=mesh0\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts[FactNodeZones] + "\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
if !strings.Contains(config, want) {
t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config)
@@ -170,7 +170,7 @@ func TestTwoThingsDecidingWhatAMachineAsksAreRefused(t *testing.T) {
if err == nil {
t.Fatal("resolv-conf and resolved-split-dns were both assigned to one machine")
}
if !strings.Contains(err.Error(), "mesh-resolver-configuration") {
if !strings.Contains(err.Error(), "node-resolver-config") {
t.Fatalf("the refusal does not say what was claimed: %v", err)
}
}
internal/catalogue/roster.go
+210
View File
@@ -0,0 +1,210 @@
package catalogue
import (
"bytes"
"fmt"
"sort"
"strings"
"text/template"
)
// What only the mesh knows, written where a module asks for it — in the module's own format.
//
// **The graph is the control plane's; the format is the module's.** The mesh knows which machines
// exist, what they are called and where they are. Turning that into a hosts file, a resolver's
// zones, an ssh known_hosts is somebody's configuration language, and the mesh has no business
// knowing it. So the mesh hands the roster to a template the module wrote and renders it; it never
// learns what the file means.
//
// This used to be a closed list of fact names, each with its format written in Go here — a hosts
// file, a resolver's zones. Every new consumer meant a new formatter in the control plane, in the
// consumer's configuration language. Now the data is the mesh's and the format is a template the
// module ships: the two built-in cases (the network module's `/etc/hosts`, dnsmasq's zones) render
// the same way any module's would, and the control plane holds no format at all.
//
// It replaced three modules that existed only because computed output needed somewhere to live —
// they ran no software, could not be swapped for anything, and appeared in the graph as modules
// while being a data channel wearing a costume (novox/hq ADR 0040).
// A RosterFile is a file the mesh renders from the roster of machines, in the format the module
// gives as a Go text/template. The template sees a rosterView: `.Node` (this machine's bare name),
// `.Suffix` (what its mesh name ends in), and two sets of `{Name, FQDN, Address}` — `.Names`, every
// name the mesh serves, and `.Machines`, only the nodes of the mesh. Which set a template ranges is
// how the hq issue 111 distinction is drawn: a container's hosts wants every name; a resolver told
// the suffix is its own wants only the machines.
type RosterFile struct {
// Path is where on the machine the rendered file goes. Absolute, or it is refused here rather
// than discovered as a daemon that reads nothing.
Path string `json:"path"`
// Template is the module's format, a Go text/template over the rosterView. It is the module's,
// not the mesh's: the mesh renders it and does not read it.
Template string `json:"template"`
// Shared is whether the file the fact goes to belongs to the machine rather than the mesh. When
// it does, the mesh owns only a marked region of it and keeps the rest byte for byte (novox/hq
// issue 128) — a hosts file is shared, since the distribution's `localhost`, the operator's own
// lines and other tools' blocks live there too; a resolver's zones file is not, the mesh owns it
// whole. A property of the fact, not of the path: the format determines whether the file is
// wholly the mesh's, not where a module happened to ask for it.
Shared bool `json:"shared,omitempty"`
// Home places the file under this node's operator-account home and chowns it to that account,
// rather than at an absolute system path (novox/hq to-be 29). Then Path is home-relative
// (`.ssh/config.d/mesh`), resolved against the account's home on the node it is composed for; a
// node with no operator account gets no such file. This is how the ssh-client config — every
// other node's Host block — is written into a person's home rather than into /etc.
Home bool `json:"home,omitempty"`
}
// rosterView is what a RosterFile's template sees. A closed shape — a template referencing a field
// the mesh does not compute fails to render here, not on a machine.
type rosterView struct {
Node string
Suffix string
Names []rosterEntry
Machines []rosterEntry
}
// rosterEntry is one machine as a template sees it: its bare name, its full mesh name, its address,
// and the operator account to log into it as (novox/hq to-be 29) — empty when none is known, so an
// ssh Host block template can omit the User line for a machine nobody has an account on.
type rosterEntry struct {
Name string
FQDN string
Address string
Account string
}
// FactsInto renders the roster files a module asked for, as files it will be given.
//
// The module owns everything after the file exists: loading it, restarting on it, what a resolver
// or a client does with it. This only puts it there. `every` is every name the mesh serves;
// `machines` is only the machines — the two must not be confused (novox/hq 04-ISSUES/111), so both
// are given and the template chooses.
func FactsInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
names := make([]string, 0, len(m.Facts))
for name := range m.Facts {
names = append(names, name)
}
sort.Strings(names)
view := rosterView{
Node: r.Node,
Suffix: strings.TrimPrefix(suffixOr(suffix), "."),
Names: entriesFrom(every, accounts, suffix),
Machines: entriesFrom(machines, accounts, suffix),
}
out := make([]map[string]any, 0, len(names))
for _, name := range names {
fact := m.Facts[name]
content, err := renderRoster(fact.Template, view)
if err != nil {
return nil, fmt.Errorf("%s cannot render %q: %w", m.Module, name, err)
}
// Where the file goes: under the operator's home and chowned to it (a home fact), or at the
// absolute system path it names. A home fact on a machine with no operator account cannot be
// placed, and is left out rather than written to nowhere (novox/hq to-be 29).
path := fact.Path
var owner string
if fact.Home {
if r.Account == "" {
continue
}
path = accountHomeOf(r.Account, r.AccountHome) + "/" + strings.TrimLeft(fact.Path, "/")
owner = r.Account
} else if !strings.HasPrefix(fact.Path, "/") {
return nil, fmt.Errorf(
"%s asks for %q at %q, which is not an absolute path", m.Module, name, fact.Path)
}
file := map[string]any{
"id": "fact-" + name, "type": "file", "path": path, "mode": "0644",
"content": content,
}
if owner != "" {
file["owner"] = owner
}
if fact.Shared {
// The host owns only the lines between `# BEGIN mesh <id>` and `# END mesh <id>` and
// keeps the rest of the file byte for byte; undeclared, the region goes and nothing else
// does (novox/hq issue 128). Every node on the private network receives this, so every
// node's host — the controller's own machine included — must be block-aware before a
// controller emitting it is rolled out: the order ADR 0102 set for `into: json`.
file["into"] = "block"
}
out = append(out, file)
}
return out, nil
}
// renderRoster runs a module's template over the roster. A template that will not parse, or reads
// a field the mesh does not have, is an error here — where the manifest is — rather than an empty
// file on a machine.
func renderRoster(tmpl string, view rosterView) (string, error) {
t, err := template.New("roster").Option("missingkey=error").Parse(tmpl)
if err != nil {
return "", err
}
var b bytes.Buffer
if err := t.Execute(&b, view); err != nil {
return "", err
}
return b.String(), nil
}
// entriesFrom is a name→address map as sorted roster entries.
//
// **A machine with no address is left out.** The mesh has a record for it — somebody added it —
// and does not yet know where it is, which is the ordinary state between adding a machine and it
// joining. Writing the name anyway would give a name that resolves to nothing, and a connection to
// that hangs; leaving it out fails at once and says the name is unknown.
func entriesFrom(addresses, accounts map[string]string, suffix string) []rosterEntry {
out := make([]rosterEntry, 0, len(addresses))
for _, name := range sortedNames(addresses) {
internal, bare := meshName(name, suffix)
// The account is looked up by whichever key the caller keys accounts on — the internal name
// or the bare one — so a template gets the right login however the maps were built.
account := accounts[name]
if account == "" {
account = accounts[bare]
}
out = append(out, rosterEntry{Name: bare, FQDN: internal, Address: addresses[name], Account: account})
}
return out
}
// meshName is a machine's internal name and its bare one, from either. The control plane keys
// the names it hands a resolution by the internal name (`homer.internal`), the same map a
// container gets as its hosts; a caller that keys by the bare name gets the same answer. The
// suffix is the one the control plane composed those names with, handed down rather than written
// here a second time — the alternative was `homer.internal.internal` on every machine.
func meshName(name, suffix string) (internal, bare string) {
dotted := "." + strings.TrimPrefix(suffixOr(suffix), ".")
if strings.HasSuffix(name, dotted) {
return name, strings.TrimSuffix(name, dotted)
}
return name + dotted, name
}
// suffixOr is the suffix given, or the one the mesh composes names with when none was handed down.
// The one place the default is written, so a fact and a name cannot disagree about it.
func suffixOr(suffix string) string {
if suffix == "" {
return "internal"
}
return suffix
}
func sortedNames(addresses map[string]string) []string {
out := make([]string, 0, len(addresses))
for name, at := range addresses {
// A machine the mesh cannot place is left out rather than named at nothing.
if at == "" {
continue
}
out = append(out, name)
}
sort.Strings(out)
return out
}
internal/catalogue/roster_test.go
+260
View File
@@ -0,0 +1,260 @@
package catalogue
import (
"strings"
"testing"
)
// Keyed by the internal name, as the control plane hands them (issue 079). bart has no address —
// the ordinary state between adding a machine and it joining.
var threeMachines = map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2", "bart.internal": ""}
// A module says where it wants a roster file and in what format, and is given the rendered file.
func TestAModuleIsGivenTheFileItAskedFor(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/mesh/zones.conf", Template: "{{range .Machines}}address=/{{.FQDN}}/{{.Address}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, nil, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 1 {
t.Fatalf("expected one file, got %d", len(given))
}
if given[0]["path"] != "/etc/mesh/zones.conf" || given[0]["type"] != "file" {
t.Fatalf("not written where it was asked for: %v", given[0])
}
// The id is fact-<name>, which is what a restart-on names when the roster changes.
if given[0]["id"] != "fact-zones" {
t.Fatalf("the file's id is not fact-<name>, so a restart-on cannot find it: %v", given[0]["id"])
}
if !strings.Contains(given[0]["content"].(string), "address=/homer.internal/10.42.0.1") {
t.Fatalf("the file does not hold the fact: %v", given[0]["content"])
}
}
// **A shared fact is written into a region of the machine's file, not over it** (novox/hq issue
// 128). A hosts file is the machine's — its localhost, the operator's lines, other tools' blocks —
// so the mesh owns only a marked region (`into: block`); a resolver's zones file is the mesh's
// whole, and carries no `into`.
func TestASharedFactIsWrittenIntoARegion(t *testing.T) {
roster := map[string]string{"homer.internal": "10.42.0.1"}
m := Manifest{Module: "net", Facts: map[string]RosterFile{
"node-names": {Path: "/etc/hosts", Template: "{{range .Names}}{{.FQDN}}\n{{end}}", Shared: true},
"node-zones": {Path: "/etc/zones", Template: "{{range .Machines}}{{.FQDN}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
by := map[string]map[string]any{}
for _, f := range given {
by[f["path"].(string)] = f
}
if by["/etc/hosts"]["into"] != "block" {
t.Fatalf("a shared fact is not written into a region, so the mesh writes the file whole: %v", by["/etc/hosts"])
}
if _, has := by["/etc/zones"]["into"]; has {
t.Fatalf("an unshared fact was written into a region, so the mesh does not own its own file whole: %v", by["/etc/zones"])
}
}
// **The format is the module's — the mesh renders whatever template it gives.** The same roster
// through two templates is two entirely different files, and the control plane reads neither.
func TestTheFormatIsTheModulesOwn(t *testing.T) {
roster := map[string]string{"homer.internal": "10.42.0.1"}
hostsish := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Names}}{{.Address}}\t{{.Name}}\n{{end}}"}}}
sshish := Manifest{Module: "b", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Names}}Host {{.Name}}\n HostName {{.FQDN}}\n{{end}}"}}}
h, err := FactsInto(hostsish, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
s, err := FactsInto(sshish, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
if h[0]["content"] != "10.42.0.1\thomer\n" {
t.Fatalf("the hosts-shaped template did not render its format: %q", h[0]["content"])
}
if s[0]["content"] != "Host homer\n HostName homer.internal\n" {
t.Fatalf("the ssh-shaped template did not render its format: %q", s[0]["content"])
}
}
// **A template that will not parse is refused here, not on a machine.** A daemon that starts, reads
// a file the mesh could not render, and answers nothing is a much worse way to find out.
func TestABrokenTemplateIsRefusedHere(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{range .Machines}}oops"}}}
_, err := FactsInto(m, Resolution{}, nil, nil, nil, "")
if err == nil {
t.Fatal("a template that does not parse was accepted, so the machine gets an empty file")
}
if !strings.Contains(err.Error(), "resolver") || !strings.Contains(err.Error(), "zones") {
t.Fatalf("the refusal does not say whose template, or which: %v", err)
}
}
// A template reading something the mesh does not compute is refused, not rendered empty. The roster
// is a closed shape; asking it for the weather fails where the manifest is.
func TestATemplateReadingWhatTheMeshDoesNotHaveIsRefused(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{.Weather}}"}}}
if _, err := FactsInto(m, Resolution{}, nil, nil, nil, ""); err == nil {
t.Fatal("a template read a field nobody computes and rendered anyway, silently")
}
}
// A relative path is refused, or a module decides where the mesh writes on a machine.
func TestAFactMustBeAskedForAtAnAbsolutePath(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"hosts": {Path: "etc/hosts", Template: "x"}}}
if _, err := FactsInto(m, Resolution{}, nil, nil, nil, ""); err == nil {
t.Fatal("a relative path was accepted")
}
}
// A machine the mesh has a record for and cannot place is left out of the roster.
//
// **Not an oversight — the alternative is worse.** A name written with no address resolves to
// nothing, and a connection to that hangs. Leaving it out fails at once and says the name is
// unknown, which is a thing somebody can act on.
func TestAMachineWithNoAddressIsNotInTheRoster(t *testing.T) {
m := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Machines}}{{.Name}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, nil, "")
if err != nil {
t.Fatal(err)
}
if strings.Contains(given[0]["content"].(string), "bart") {
t.Fatalf("a machine with no address was in the roster, so its name resolves to nothing:\n%s", given[0]["content"])
}
}
// **The names the control plane hands a resolution are already internal names** — `homer.internal`,
// the same map every container gets as its hosts. A roster entry's FQDN is that name, not it with
// the suffix appended a second time; either key gives the same entries.
func TestNamesAreNotSuffixedTwice(t *testing.T) {
internal := map[string]string{"homer.internal": "10.42.0.1"}
bare := map[string]string{"homer": "10.42.0.1"}
tmpl := RosterFile{Path: "/f", Template: "{{range .Machines}}{{.FQDN}} {{.Name}}\n{{end}}"}
fromInternal, err := FactsInto(Manifest{Module: "a", Facts: map[string]RosterFile{"f": tmpl}}, Resolution{Node: "homer"}, internal, internal, nil, "")
if err != nil {
t.Fatal(err)
}
fromBare, err := FactsInto(Manifest{Module: "a", Facts: map[string]RosterFile{"f": tmpl}}, Resolution{Node: "homer"}, bare, bare, nil, "")
if err != nil {
t.Fatal(err)
}
if fromInternal[0]["content"] != fromBare[0]["content"] {
t.Fatalf("the roster differs by how the names were keyed:\n%q\n%q", fromInternal[0]["content"], fromBare[0]["content"])
}
got := fromInternal[0]["content"].(string)
if strings.Contains(got, "internal.internal") || !strings.Contains(got, "homer.internal homer") {
t.Fatalf("the entry carries a doubled suffix or the wrong bare name:\n%s", got)
}
}
// The suffix the control plane composed the names with is the one a template sees — an operator who
// chose another does not get `.internal`. `.Suffix` is the bare form, and FQDNs carry it.
func TestTheSuffixIsCarriedAsComposed(t *testing.T) {
names := map[string]string{"homer.lan": "10.42.0.1"}
m := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "local=/{{.Suffix}}/\n{{range .Machines}}{{.FQDN}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, names, names, nil, "lan")
if err != nil {
t.Fatal(err)
}
got := given[0]["content"].(string)
if !strings.Contains(got, "local=/lan/") || strings.Contains(got, "internal") {
t.Fatalf("the operator's suffix was not carried, so its names would be wrong:\n%s", got)
}
if !strings.Contains(got, "homer.lan") {
t.Fatalf("the FQDN does not carry the operator's suffix:\n%s", got)
}
}
// novox/hq 04-ISSUES/111: a template is given both sets and chooses. `.Names` is every name the mesh
// serves — the machines and the names it was told to route; `.Machines` is only the machines. A
// container's hosts wants every name so a routed name resolves to the machine serving it; a resolver
// told the suffix is its own wants only the machines, or a routed name written there with the suffix
// is a name nobody will ever ask for, standing beside the machines and looking as real.
func TestATemplateChoosesMachinesOrEveryName(t *testing.T) {
machines := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
every := map[string]string{
"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2",
"drive.example.test": "10.42.0.1", "git.example.test": "10.42.0.2",
}
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{range .Machines}}{{.FQDN}}\n{{end}}"},
"hosts": {Path: "/etc/hosts", Template: "{{range .Names}}{{.FQDN}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, every, machines, nil, "")
if err != nil {
t.Fatal(err)
}
by := map[string]string{}
for _, f := range given {
by[f["path"].(string)] = f["content"].(string)
}
zones := by["/etc/zones"]
for _, served := range []string{"drive.example.test", "git.example.test"} {
if strings.Contains(zones, served) {
t.Fatalf("a template over .Machines saw %q, a name the mesh serves rather than a machine:\n%s", served, zones)
}
}
if !strings.Contains(zones, "homer.internal") {
t.Fatalf("a template over .Machines did not see the machines:\n%s", zones)
}
hosts := by["/etc/hosts"]
for _, name := range []string{"homer.internal", "drive.example.test", "git.example.test"} {
if !strings.Contains(hosts, name) {
t.Fatalf("a template over .Names did not see %q, so a container would not resolve it:\n%s", name, hosts)
}
}
}
// A home fact is placed under the operator account's home and chowned to it, and its template sees
// each node's account (novox/hq to-be 29) — the ssh-client config is the case.
func TestAHomeFactIsPlacedUnderTheAccountsHomeAndOwnedByIt(t *testing.T) {
names := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
accounts := map[string]string{"homer": "jo", "marge": "jo"}
m := Manifest{Module: "ssh-client", Facts: map[string]RosterFile{
"ssh-config": {Path: ".ssh/config.d/mesh", Home: true,
Template: "{{range .Names}}Host {{.Name}}\n HostName {{.FQDN}}\n User {{.Account}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer", Account: "jo"}, names, names, accounts, "")
if err != nil {
t.Fatal(err)
}
f := given[0]
if f["path"] != "/home/jo/.ssh/config.d/mesh" {
t.Fatalf("the home fact was not placed under the account's home: %v", f["path"])
}
if f["owner"] != "jo" {
t.Fatalf("the home fact is not owned by the account: %v", f["owner"])
}
if !strings.Contains(f["content"].(string), "Host marge\n HostName marge.internal\n User jo") {
t.Fatalf("the config does not name the peer's account:\n%s", f["content"])
}
}
// A machine with no operator account gets no home fact — it cannot be placed, so it is left out
// rather than written to nowhere.
func TestAHomeFactIsSkippedWhereThereIsNoAccount(t *testing.T) {
names := map[string]string{"homer.internal": "10.42.0.1"}
m := Manifest{Module: "ssh-client", Facts: map[string]RosterFile{
"ssh-config": {Path: ".ssh/config", Home: true, Template: "x"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, names, names, nil, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 0 {
t.Fatalf("a home fact was placed on a machine with no operator account: %v", given)
}
}
internal/catalogue/seats.go
+136 -83
View File
@@ -42,54 +42,108 @@ type Seat struct {
Decision string
}
// seats is the whole set, in the order a person reads it: the mesh's own, then a node's.
var seats = []Seat{
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
// fallback when it has none (novox/hq ADR 0122). It is the one place the closed set 0110 defines is
// written; the store's table is seeded from it and thereafter is the live, editable copy.
//
// In the order a person reads it: the mesh's own, then a node's.
var defaultSeats = []Seat{
{Name: "mesh-controller", Scope: ScopeMesh, Decision: "novox/hq ADR 0079"},
{Name: "mesh-store", Scope: ScopeMesh, Delivers: "postgres-database", Decision: "novox/hq ADR 0079"},
// What holding this delivers is the mesh's own bus (novox/hq ADR 0120): a module that speaks
// to the mesh requires `mesh-bus` and receives an address, a sealed credential and the trust
// to verify the server. A module that requires nothing gets no account at all — 23 of the
// catalogue's 72 never speak, and an ambient connection would mint a credential for each.
//
// Corrected twice in one day, which is worth the comment. It read `amqp`, which was the old
// broker's interface and not this seat's; ADR 0117 emptied it, reasoning that a bus cannot be
// provisioned; and it is neither. The bus's accounts are composed by the controller rather
// than created by a provisioner, so nothing waits on a bus account in order to make one —
// which is a fact about the *mechanism*, not a reason the connection cannot be required.
//
// `mesh-bus` is the mesh's own; `nats` is a private NATS server a module may provide as a
// backing service, the way `amqp` is provided (ADR 0119). Never the same name.
// **Delivers the mesh's own bus, not `amqp`.** Those were the same word until
// ADR 0127 separated them: `amqp` is a backing service a module may require, and this seat is
// the mesh's own transport. ADR 0128 then made that connection something a module requires
// rather than receives ambiently — 23 of the catalogue's modules never speak, and an ambient
// connection would mint a credential for each.
{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "mesh-bus", Decision: "novox/hq ADR 0079"},
{Name: "mesh-artifact-store", Scope: ScopeMesh, Delivers: "artifact-store", Decision: "novox/hq ADR 0075"},
{Name: "mesh-catalog", Scope: ScopeMesh, Decision: "novox/hq ADR 0110"},
{Name: "mesh-npm-package-registry", Scope: ScopeMesh, Delivers: "npm-package-registry", Decision: "novox/hq ADR 0109"},
{Name: "mesh-git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0111"},
// A build is work submitted to this role, and its outcome is the role's own event (ADR 0121).
{Name: "the-artifact-store", Scope: ScopeMesh, Delivers: "artifact-store", Decision: "novox/hq ADR 0075"},
{Name: "mesh-catalog", Scope: ScopeMesh, Decision: "novox/hq ADR 0121"},
// Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a
// delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight.
// They keep their names until that migration is done deliberately, apart from the node-* pass.
{Name: "npm-package-registry", Scope: ScopeMesh, Delivers: "npm-package-registry", Decision: "novox/hq ADR 0109"},
{Name: "git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0111"},
// A build is work submitted to this role and its outcome is the role's own event (ADR 0129).
// One publish reaches whoever asked, the controller that records it, and the catalogue that
// places it in the module graph — which is what the old bus's shared exchange did for free, and
// what a dedicated build branch was doing a second way.
{Name: "mesh-build-machine", Scope: ScopeNode,
Accepts: []string{"build"}, Emits: []string{"built"}, Decision: "novox/hq ADR 0110"},
{Name: "mesh-dns-port", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "mesh-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "mesh-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "mesh-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "mesh-resolver-configuration", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "mesh-showcase", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
// places it in the graph — what the old bus's shared exchange did for free.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"built"}, Decision: "novox/hq ADR 0121"},
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client
// model change, not a rename, so it stays until that is built.
{Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "node-resolver-config", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// The program that manages the machine's own network. It delivers nothing: its holder only
// keeps the manager and the mesh from contradicting each other — the resolver file left to the
// mesh, the private network's interface left alone — and never declares a link, an address or
// a wireless network, because the link is the only channel a fix could arrive on. A seat
// rather than a condition in the resolver's module, so a machine running two managers is
// refused at assignment instead of found by the resolver being rewritten (novox/hq ADR 0117).
{Name: "node-uplink", Scope: ScopeNode, Decision: "novox/hq ADR 0117"},
}
// A system seat name is the control plane's namespace: `mesh-*` for a mesh-wide role, `node-*` for
// a per-node one (novox/hq ADR 0121). A claim to a system name the mesh does not define is refused;
// any other name is a module's own to define and claim. Some of the mesh's own seats predate this
// convention and are not yet renamed (git, npm-package-registry, the-artifact-store,
// the-private-network) — those are in the set, so they resolve by name, not by prefix.
func isSystemSeatName(name string) bool {
return strings.HasPrefix(name, "mesh-") || strings.HasPrefix(name, "node-")
}
// seats is the working set the lookups read. It starts as the compiled defaults and is replaced by
// what the control plane loaded from its store (novox/hq ADR 0122), so a change to the set is a
// change to data, not to this code.
var seats = defaultSeats
// DefaultSeats is the set the mesh ships with, for seeding the store's seat table.
func DefaultSeats() []Seat { return append([]Seat(nil), defaultSeats...) }
// UseSeats replaces the working set with the one the control plane read from its store.
//
// **Empty is ignored on purpose.** A store that has not been seeded yet — or one that could not be
// read — must leave the compiled defaults in force rather than emptying the set: an empty set would
// refuse every claim and could stop the control plane composing at all, which is a far worse failure
// than running on the set the binary shipped with. So the store can only ever *replace* the set with
// a non-empty one, never erase it.
func UseSeats(s []Seat) {
if len(s) > 0 {
seats = s
}
}
// aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from
// the store alongside the set, so a reference to a name a seat used to have — a manifest's claim, a
// held record — still resolves to it after a rename, and nothing downstream has to change.
var aliases = map[string]string{}
// UseAliases replaces the former-name map with the one the control plane read from its store. Empty
// is fine and ordinary: a mesh whose seats have never been renamed has no aliases.
func UseAliases(m map[string]string) { aliases = m }
// Seats is every seat the mesh defines, in reading order.
func Seats() []Seat {
return append([]Seat(nil), seats...)
}
// SeatNamed is the seat a claim names, if the mesh defines one.
// SeatNamed is the seat a name refers to, whether that is its current name or one it used to have
// (novox/hq ADR 0122). A former name resolves to the seat's canonical row, so a rename breaks no
// reference to the old name.
func SeatNamed(name string) (Seat, bool) {
for _, s := range seats {
if s.Name == name {
return s, true
}
}
if canonical, aliased := aliases[name]; aliased {
for _, s := range seats {
if s.Name == canonical {
return s, true
}
}
}
return Seat{}, false
}
@@ -106,42 +160,63 @@ func SeatDelivering(provision string) (Seat, bool) {
return Seat{}, false
}
// claimProblems is what is wrong with a manifest's claims against the set.
// claimProblems is what is wrong with a manifest's claims and the seats it defines.
//
// Three refusals, each naming the seat: a seat the mesh does not define, a seat claimed at another
// scope, and a seat that delivers a provision claimed by a module that does not provide it — which
// would make the module the mesh's answer for something it cannot answer.
// A claim is one of three things (novox/hq ADR 0121): a **system seat** the control plane defines —
// checked for scope and, if it delivers a provision, that the claimant provides it; a **system name
// the mesh does not define** (`mesh-*`/`node-*`) — refused, because that namespace is the control
// plane's; or a **module-defined seat** — valid only when this manifest also declares it, since a
// module may coordinate its own instances through a seat of its own but may not invent one by
// claiming it. A module's own seat declaration may not sit in the system namespace or shadow a
// system seat.
func claimProblems(m Manifest) []string {
var problems []string
defined := map[string]SeatDeclaration{}
for _, d := range m.DefinesSeats {
if _, isSystem := SeatNamed(d.Name); isSystem || isSystemSeatName(d.Name) {
problems = append(problems, fmt.Sprintf(
"%s defines a seat %q in the mesh's own namespace; a module's seat is named outside "+
"mesh-*/node-* (novox/hq ADR 0121)", m.Module, d.Name))
continue
}
defined[d.Name] = d
}
for _, c := range m.Claims {
if now, was := renamedSeats[c.Name]; was {
// Named rather than refused as unknown: whoever wrote it knew what they meant, and
// the mesh knows what it is called now — the same courtesy the `needs`/`own-secrets`
// rename gets. Without this the refusal would be "not a seat", which sends somebody
// reading code for a name that is one character different.
problems = append(problems, fmt.Sprintf(
"%s claims %q, which is now called %q (novox/hq ADR 0118: the mesh's own seats "+
"are named mesh-*, and the prefix is what reserves them)", m.Module, c.Name, now))
if seat, known := SeatNamed(c.Name); known {
if c.At() != seat.Scope {
problems = append(problems, fmt.Sprintf(
"%s claims %s at scope %q, and %s is a %s seat",
m.Module, c.Name, c.At(), c.Name, seat.Scope))
}
if seat.Delivers != "" && !providesAt(m, seat.Delivers, seat.Scope) {
problems = append(problems, fmt.Sprintf(
"%s claims %s, whose holder answers for %q, and %s does not provide %q at %s scope",
m.Module, c.Name, seat.Delivers, m.Module, seat.Delivers, seat.Scope))
}
continue
}
seat, known := SeatNamed(c.Name)
if !known {
// Not one of the mesh's own, which no longer means it is not a seat: a module may
// declare its own (novox/hq ADR 0118), and whether anybody declared *this* one is a
// fact about the catalogue rather than about this manifest. Deferred to
// CatalogueProblems, which refuses it at registration — the same guarantee ADR 0110
// wanted, at the same moment, from a set nobody maintains by hand.
if isSystemSeatName(c.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims %q, which is a seat in the mesh's own namespace (mesh-*/node-*) that it "+
"does not define (novox/hq ADR 0121) — the seats are: %s", m.Module, c.Name, seatNames()))
continue
}
if c.At() != seat.Scope {
problems = append(problems, fmt.Sprintf(
"%s claims %s at scope %q, and %s is a %s seat",
m.Module, c.Name, c.At(), c.Name, seat.Scope))
d, ours := defined[c.Name]
if !ours {
// **A claim on a seat this manifest does not declare is not the parser's to judge.**
// A module may hold a seat another module declared — that is why ADR 0126 has callers
// name the seat and not its provider, so an implementation can be replaced without
// touching a caller. Whether the seat exists is a fact about the whole catalogue, so
// the refusal is at registration, where every declaration is in view
// (`CatalogueProblems`: "which no module declares and the mesh does not define").
continue
}
if seat.Delivers != "" && !providesAt(m, seat.Delivers, seat.Scope) {
if c.At() != d.At() {
problems = append(problems, fmt.Sprintf(
"%s claims %s, whose holder answers for %q, and %s does not provide %q at %s scope",
m.Module, c.Name, seat.Delivers, m.Module, seat.Delivers, seat.Scope))
"%s claims its own seat %s at scope %q, having declared it at %q",
m.Module, c.Name, c.At(), d.At()))
}
}
return problems
@@ -177,7 +252,10 @@ func HolderAmong(provision string, providers []Provider, held []Held) (Provider,
return Provider{}, false
}
for _, h := range held {
if h.Claim != seat.Name || h.Scope != seat.Scope {
// Resolve the held claim to a seat rather than comparing names, so a record naming a seat's
// former name still matches it after a rename (novox/hq ADR 0122).
hs, ok := SeatNamed(h.Claim)
if !ok || hs.Name != seat.Name || h.Scope != seat.Scope {
continue
}
for _, p := range providers {
@@ -189,31 +267,6 @@ func HolderAmong(provision string, providers []Provider, held []Held) (Provider,
return Provider{}, false
}
// renamedSeats is what the mesh's own seats used to be called (novox/hq ADR 0118).
//
// **A rename here is not a data migration**, which ADR 0118 assumed it was and a progressive
// insight there corrects: a seat's holding is *derived* at resolution from the claims in
// manifests (`resolve.go`), never stored, so there are no recorded old names to rewrite. What
// exists is source — manifests in the catalogue — and this list is how one written against the
// old name is told what it became rather than refused as unknown.
//
// It is kept, not retired after the catalogue is updated: a module lives in its own repository
// ([ADR 0069]) and may be registered from anywhere, so an old name can arrive long after the
// catalogue beside this checkout stopped using one.
var renamedSeats = map[string]string{
"the-artifact-store": "mesh-artifact-store",
"the-catalogue": "mesh-catalog",
"npm-package-registry": "mesh-npm-package-registry",
"git": "mesh-git",
"the-build-machine": "mesh-build-machine",
"the-dns-port": "mesh-dns-port",
"the-intrusion-prevention": "mesh-intrusion-prevention",
"the-packet-filter": "mesh-packet-filter",
"the-private-network": "mesh-private-network",
"the-resolver-configuration": "mesh-resolver-configuration",
"the-showcase": "mesh-showcase",
}
// SeatsWithAProtocol are the mesh's own seats that say something about what may be said to them or by
// them, which is the set the bus derives streams, consumers and permissions from.
//
internal/catalogue/seats_declared.go
+9 -10
View File
@@ -70,7 +70,7 @@ func declaredSeatProblems(m Manifest) []string {
var problems []string
seen := map[string]bool{}
for _, s := range m.Seats {
for _, s := range m.DefinesSeats {
switch {
case s.Name == "":
problems = append(problems, fmt.Sprintf("%s declares a seat with no name", m.Module))
@@ -105,14 +105,13 @@ func declaredSeatProblems(m Manifest) []string {
"%s declares seat %s at scope %q; a seat is held per node, per site or per mesh",
m.Module, s.Name, s.Scope))
}
if len(s.verbs()) == 0 {
// A seat with no protocol is exclusion with nothing on the other side of it. If a
// module only wants "there is one of me", that is a claim, and saying so keeps the
// word "seat" meaning something callers can depend on.
problems = append(problems, fmt.Sprintf(
"%s declares seat %s with no protocol; a seat says what may be sent to it, what "+
"it emits and what it serves", m.Module, s.Name))
}
// A seat with an empty protocol is allowed, and is the mesh saying what a machine is:
// which module is this node's packet filter, or its showcase. Design 26 calls it a seat
// that delivers nothing, and that is most of the node-scoped ones. ADR 0126's "a declared
// seat carries a protocol" governs what a holder must satisfy, not that every seat offers
// something — a marker seat's protocol is satisfied by holding it. Nothing can reach this
// state by accident: a mistyped field name is refused by the parser above, so an empty
// protocol was written as one.
for _, v := range s.verbs() {
if !name.MatchString(v) {
problems = append(problems, fmt.Sprintf(
@@ -143,7 +142,7 @@ func CatalogueProblems(shelf Shelf) []string {
declaredBy := map[string]string{}
declared := map[string]SeatDeclaration{}
for _, module := range shelfOrder(shelf) {
for _, s := range shelf[module].Seats {
for _, s := range shelf[module].DefinesSeats {
if s.Name == "" {
continue
}
internal/catalogue/seats_declared_test.go
+9 -8
View File
@@ -11,7 +11,7 @@ func problemsFor(t *testing.T, shelf Shelf) string {
}
func telegram() Manifest {
return Manifest{Module: "telegram", Tools: []string{"status"}, Seats: []SeatDeclaration{{
return Manifest{Module: "telegram", Tools: []string{"status"}, DefinesSeats: []SeatDeclaration{{
Name: "telegram-sender", Scope: ScopeMesh,
Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}, Serves: []string{"status"},
}}, Claims: []Claim{{Name: "telegram-sender", Scope: ScopeMesh}}}
@@ -28,7 +28,7 @@ func TestAModuleDeclaresItsOwnSeatAndHoldsIt(t *testing.T) {
// The prefix is the reservation rule, so there is no list to maintain and none to drift.
func TestAModuleCannotDeclareAMeshSeat(t *testing.T) {
for _, n := range []string{"mesh-broker", "mesh-anything", "mesh-store"} {
m := Manifest{Module: "impostor", Seats: []SeatDeclaration{{Name: n, Accepts: []string{"x"}}}}
m := Manifest{Module: "impostor", DefinesSeats: []SeatDeclaration{{Name: n, Accepts: []string{"x"}}}}
got := strings.Join(declaredSeatProblems(m), "; ")
if !strings.Contains(got, "reserved to the mesh") {
t.Fatalf("%q was accepted as a module's seat: %q", n, got)
@@ -38,7 +38,7 @@ func TestAModuleCannotDeclareAMeshSeat(t *testing.T) {
// A seat name meaning two protocols is the failure nobody could diagnose afterwards.
func TestTwoModulesCannotDeclareTheSameSeat(t *testing.T) {
other := Manifest{Module: "aardvark", Seats: []SeatDeclaration{{
other := Manifest{Module: "aardvark", DefinesSeats: []SeatDeclaration{{
Name: "telegram-sender", Scope: ScopeMesh, Accepts: []string{"something-else"}}}}
got := problemsFor(t, Shelf{"telegram": telegram(), "aardvark": other})
if !strings.Contains(got, "already declares") {
@@ -70,11 +70,12 @@ func TestAHolderMustServeWhatItsSeatPromises(t *testing.T) {
}
}
// A seat with no protocol is exclusion with nothing on the other side of it.
func TestASeatWithoutAProtocolIsRefused(t *testing.T) {
m := Manifest{Module: "vague", Seats: []SeatDeclaration{{Name: "something", Scope: ScopeMesh}}}
if got := strings.Join(declaredSeatProblems(m), "; "); !strings.Contains(got, "no protocol") {
t.Fatalf("a seat promising nothing was accepted: %s", got)
// A seat with no protocol is a marker: which module is this node's showcase, or its packet filter.
// Most node-scoped seats are markers, so refusing one would refuse the majority of the set.
func TestASeatWithoutAProtocolIsAMarkerNotAMistake(t *testing.T) {
m := Manifest{Module: "vague", DefinesSeats: []SeatDeclaration{{Name: "something", Scope: ScopeNode}}}
if got := strings.Join(declaredSeatProblems(m), "; "); got != "" {
t.Fatalf("a marker seat was refused: %s", got)
}
}
internal/catalogue/seats_test.go
+92 -5
View File
@@ -50,6 +50,26 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
}
}
// novox/hq ADR 0117: a machine's uplink is a seat, held per machine, and delivers nothing.
//
// **Nothing, because nothing may be required of it.** A holder only keeps its network manager from
// contradicting the mesh; a requirement resolving to it would make the manager the mesh's answer
// for something, and the manager's link is the one thing the mesh must never be able to break.
func TestTheUplinkIsANodeSeatThatDeliversNothing(t *testing.T) {
seat, known := SeatNamed("node-uplink")
if !known {
t.Fatalf("the uplink is not a seat; the seats are: %s", seatNames())
}
if seat.Scope != ScopeNode || seat.Delivers != "" || seat.Decision != "novox/hq ADR 0117" {
t.Fatalf("the uplink is %+v, not a node seat delivering nothing by ADR 0117", seat)
}
// And a manager's module can hold it without providing anything.
raw := []byte(`{"module":"networkmanager","version":"1","claims":[{"name":"node-uplink","scope":"node"}]}`)
if _, err := ParseManifest(raw); err != nil {
t.Fatalf("a network manager's module could not hold the uplink: %v", err)
}
}
func claimed(claims string) []byte {
return []byte(`{"module":"thing","version":"1","provides":[{"name":"npm-package-registry","scope":"mesh"}],"claims":` + claims + `}`)
}
@@ -83,7 +103,7 @@ func TestAClaimOnASeatAnotherModuleDeclaresIsAccepted(t *testing.T) {
if err != nil {
t.Fatal(err)
}
declarer := Manifest{Module: "someone", Seats: []SeatDeclaration{
declarer := Manifest{Module: "someone", DefinesSeats: []SeatDeclaration{
{Name: "the-anything", Scope: ScopeNode, Accepts: []string{"work"}},
}}
if problems := CatalogueProblems(Shelf{claimant.Module: claimant, "someone": declarer}); len(problems) != 0 {
@@ -91,8 +111,35 @@ func TestAClaimOnASeatAnotherModuleDeclaresIsAccepted(t *testing.T) {
}
}
// A module may define its own seat and claim it — the mesh enforces exclusivity without knowing
// what it means (novox/hq ADR 0121). But it may not define one in the mesh's own namespace.
func TestAModuleDefinesAndClaimsItsOwnSeat(t *testing.T) {
ok := []byte(`{"module":"showcase","version":"1","seats":[{"name":"the-showcase","scope":"node"}],` +
`"claims":[{"name":"the-showcase","scope":"node"}]}`)
if _, err := ParseManifest(ok); err != nil {
t.Fatalf("a module could not define and claim its own seat: %v", err)
}
// Claiming a name nobody defines is still refused — but **at registration, not here**: with
// seats declared by modules, a claim on a seat *another* module declares is good, and the
// parser cannot tell that from an invented name. See
// TestAClaimOnASeatNobodyDeclaresIsRefusedAtRegistration.
claimant, err := ParseManifest(claimed(`[{"name":"the-anything","scope":"node"}]`))
if err != nil {
t.Fatalf("the parser judged a claim it cannot judge alone: %v", err)
}
if len(CatalogueProblems(Shelf{claimant.Module: claimant})) == 0 {
t.Fatal("a module claimed a seat nobody defines")
}
// A module may not carve its seat out of the mesh's own namespace.
bad := []byte(`{"module":"x","version":"1","seats":[{"name":"node-mine","scope":"node"}],` +
`"claims":[{"name":"node-mine","scope":"node"}]}`)
if _, err := ParseManifest(bad); err == nil || !strings.Contains(err.Error(), "own namespace") {
t.Fatalf("a module defined a seat in the mesh's namespace and was not refused: %v", err)
}
}
func TestASeatClaimedAtAnotherScopeIsRefused(t *testing.T) {
_, err := ParseManifest(claimed(`[{"name":"mesh-npm-package-registry","scope":"node"}]`))
_, err := ParseManifest(claimed(`[{"name":"npm-package-registry","scope":"node"}]`))
if err == nil {
t.Fatal("a mesh seat was held per node")
}
@@ -104,7 +151,7 @@ func TestASeatClaimedAtAnotherScopeIsRefused(t *testing.T) {
func TestADeliveringSeatIsOnlyHeldByAModuleThatProvides(t *testing.T) {
// Holding it makes the module the mesh's answer for the provision. A module that cannot answer
// would be the answer anyway, and every consumer would be sent to it.
raw := []byte(`{"module":"thing","version":"1","claims":[{"name":"mesh-git","scope":"mesh"}]}`)
raw := []byte(`{"module":"thing","version":"1","claims":[{"name":"git","scope":"mesh"}]}`)
_, err := ParseManifest(raw)
if err == nil {
t.Fatal("a module holding the git seat need not provide git")
@@ -163,7 +210,7 @@ func TestEveryManifestInUseClaimsASeatTheMeshDefines(t *testing.T) {
func registryShelf() map[string]Manifest {
return shelf(
Manifest{Module: "gitea", Version: "1", Provides: FromAnywhere("npm-package-registry"),
Claims: []Claim{{Name: "mesh-npm-package-registry", Scope: ScopeMesh}}},
Claims: []Claim{{Name: "npm-package-registry", Scope: ScopeMesh}}},
Manifest{Module: "verdaccio", Version: "1", Provides: FromAnywhere("npm-package-registry")},
Manifest{Module: "builder", Version: "1", Requires: []string{"npm-package-registry"}},
)
@@ -177,7 +224,7 @@ func twoRegistries() map[string][]Provider {
}
func giteaHoldsTheSeat() []Held {
return []Held{{Claim: "mesh-npm-package-registry", Scope: ScopeMesh, Node: "anchor", Module: "gitea"}}
return []Held{{Claim: "npm-package-registry", Scope: ScopeMesh, Node: "anchor", Module: "gitea"}}
}
func TestTheSeatsHolderAnswersWhenSeveralProvide(t *testing.T) {
@@ -229,3 +276,43 @@ func TestTheHolderIsTheModuleNotTheMachine(t *testing.T) {
t.Fatalf("the holder was not told apart from a neighbour: %+v", holder)
}
}
// The working set is loaded from the store, and an empty load never erases it (novox/hq ADR 0122).
func TestUseSeatsReplacesTheSetButNeverEmptiesIt(t *testing.T) {
before := Seats()
defer UseSeats(DefaultSeats()) // restore for other tests, whatever this leaves it as
// An empty load (store not seeded, or unreadable) leaves the compiled defaults in force.
UseSeats(nil)
if len(Seats()) != len(before) {
t.Fatalf("an empty load changed the set from %d to %d seats", len(before), len(Seats()))
}
// A non-empty load replaces it — this is how a rename in the store reaches the lookups.
UseSeats([]Seat{{Name: "node-firewall", Scope: ScopeNode, Decision: "novox/hq ADR 0122"}})
if _, known := SeatNamed("node-firewall"); !known {
t.Fatal("the loaded set did not replace the working set")
}
if len(Seats()) != 1 {
t.Fatalf("the working set is %d seats, not the one that was loaded", len(Seats()))
}
}
// A former name resolves to the seat it was renamed from (novox/hq ADR 0122), so a manifest's claim
// and a held record naming the old name break nothing after a rename.
func TestAFormerNameResolvesAfterARename(t *testing.T) {
defer func() { UseSeats(DefaultSeats()); UseAliases(nil) }()
UseSeats([]Seat{{Name: "git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0121"}})
UseAliases(map[string]string{"git": "git"})
// The old name resolves to the renamed seat.
if s, ok := SeatNamed("git"); !ok || s.Name != "git" {
t.Fatalf("the former name did not resolve to the renamed seat: %+v ok=%v", s, ok)
}
// And a holder recorded under the old name is still found for the provision the seat delivers.
providers := []Provider{{Node: "anchor", At: "anchor.internal", Module: "gitea"}}
held := []Held{{Claim: "git", Scope: ScopeMesh, Node: "anchor", Module: "gitea"}}
holder, found := HolderAmong("git", providers, held)
if !found || holder.Module != "gitea" {
t.Fatalf("the holder recorded under the former name was not matched: %+v found=%v", holder, found)
}
}
internal/catalogue/ssh_client_test.go
+46
View File
@@ -0,0 +1,46 @@
package catalogue
import (
"strings"
"testing"
)
// The ssh-client module, composed as a machine receives it (novox/hq to-be 29): every other node's
// Host block written into a marked region of the operator's ~/.ssh/config, owned by the account,
// with ~/.ssh created 0700 — the operator's own config kept.
func TestSSHClientOwnsTheOperatorsSSHConfig(t *testing.T) {
shelf := shelf(catalogueManifest(t, "ssh-client"))
got, err := Resolve(shelf, []string{"ssh-client"},
Node{Name: "homer", At: "homer.internal", Account: "jo"}, World{})
if err != nil {
t.Fatal(err)
}
names := map[string]string{"homer.internal": "10.10.0.1", "marge.internal": "10.10.0.2"}
out, err := got.Declaration(Rendering{
Names: names, Machines: names, Accounts: map[string]string{"homer": "jo", "marge": "jo"},
Suffix: "internal",
})
if err != nil {
t.Fatal(err)
}
by := map[string]map[string]any{}
for _, r := range out {
by[r["id"].(string)] = r
}
dir := by["ssh-client.ssh-dir"]
if dir == nil || dir["path"] != "/home/jo/.ssh" || dir["owner"] != "jo" || dir["mode"] != "0700" {
t.Fatalf("~/.ssh is not created 0700 owned by the account: %v", dir)
}
cfg := by["ssh-client.fact-ssh-config"]
if cfg == nil || cfg["path"] != "/home/jo/.ssh/config" || cfg["owner"] != "jo" || cfg["into"] != "block" {
t.Fatalf("the ssh config is not written into the operator's ~/.ssh/config as a region: %v", cfg)
}
body := cfg["content"].(string)
if !strings.Contains(body, "Host marge marge.internal") || !strings.Contains(body, "User jo") {
t.Fatalf("the config does not name the peer node and its account:\n%s", body)
}
if strings.Contains(body, "Host homer ") {
t.Fatalf("the config names the machine itself, not only its peers:\n%s", body)
}
}
internal/inventory/addresses.go
+11
View File
@@ -69,6 +69,17 @@ func (i *Inventory) AssignAddress(ctx context.Context, node, cidr string) (strin
if key != nil && p.PublicKey == *key {
// The tunnel already routes to this key: the node keeps that address, and the peer
// notices nothing when its machine enrols.
//
// **Unless the operator named it something else** (novox/hq issue 112). The name is
// what the mesh has been answering for this address in the meantime; a machine
// enrolling under a different one would silently split the two — the name resolving
// here, the node known as that — so it is refused where the operator can read it.
if p.Named != "" && p.Named != name {
return "", fmt.Errorf(
"the carried peer at %s was named %q, and %q is enrolling under its key — "+
"enrol it as %q, or rename the peer first (`overlay name`)",
p.Address, p.Named, name, p.Named)
}
return i.place(ctx, node, p.Address)
}
taken[p.Address] = true
internal/inventory/adoption_test.go
+2 -2
View File
@@ -65,7 +65,7 @@ func TestTakingIsRefusedOnAConvergedNodeAndForAnUnassignedModule(t *testing.T) {
if _, err := inv.AddNode(t.Context(), "converged"); err != nil {
t.Fatal(err)
}
if err := inv.Assign(t.Context(), "converged", "hello-web"); err != nil {
if _, err := inv.Assign(t.Context(), "converged", "hello-web"); err != nil {
t.Fatal(err)
}
if err := inv.Take(t.Context(), "converged", "hello-web"); !errors.Is(err, ErrNotAdopted) {
@@ -91,7 +91,7 @@ func TestATakenModuleOutlivesItsAssignmentAndReturningToAdopted(t *testing.T) {
t.Fatal(err)
}
for _, m := range []string{"hello-web", "postgres"} {
if err := inv.Assign(t.Context(), "anchor", m); err != nil {
if _, err := inv.Assign(t.Context(), "anchor", m); err != nil {
t.Fatal(err)
}
}
internal/inventory/busrecords.go
+1 -1
View File
@@ -35,7 +35,7 @@ func (i *Inventory) BusRecords(ctx context.Context) (broker.Records, error) {
// one module and held by another, which is the whole reason a seat exists (ADR 0118).
seats := map[string]catalogue.SeatDeclaration{}
for _, m := range declared {
for _, s := range m.Seats {
for _, s := range m.DefinesSeats {
seats[s.Name] = s
}
}
internal/inventory/busrecords_test.go
+2 -2
View File
@@ -31,7 +31,7 @@ func aMeshWith(t *testing.T, manifests ...catalogue.Manifest) (*Inventory, conte
func theSeatDeclarer() catalogue.Manifest {
return catalogue.Manifest{
Module: "telegram", Version: "1",
Seats: []catalogue.SeatDeclaration{{
DefinesSeats: []catalogue.SeatDeclaration{{
Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered"},
}},
Claims: []catalogue.Claim{{Name: "telegram-sender", Scope: catalogue.ScopeMesh}},
@@ -50,7 +50,7 @@ func TestAnAssignedModuleBecomesAUserWithWhatItDeclared(t *testing.T) {
if _, err := inv.AddNode(ctx, "one"); err != nil {
t.Fatal(err)
}
if err := inv.Assign(ctx, "one", "shop"); err != nil {
if _, err := inv.Assign(ctx, "one", "shop"); err != nil {
t.Fatal(err)
}
internal/inventory/catalogue.go
+16 -7
View File
@@ -430,27 +430,36 @@ func (i *Inventory) discard(ctx context.Context, name string) error {
return nil
}
// Assign puts a module on a node.
// Assign puts a module on a node, and says whether that is new.
//
// Records the intention and checks nothing. Whether the set of assignments can actually become a
// declaration is resolution's question, asked over the whole set at once — and asking it here,
// one module at a time, would let an assignment look accepted and then refuse when a second
// arrives.
func (i *Inventory) Assign(ctx context.Context, nodeName, module string) error {
//
// **One assignment of a module per node is the rule, not a race lost** (novox/hq ADR 0115). The
// module's name is the assignment's identity — its database user, its broker account, its
// containers and its placed directory are all named by it — so the schema's (node, module) key
// is the decision, and a repeat is absorbed rather than refused. Absorbed audibly: the caller is
// told nothing changed, because "is assigned" printed for a no-op reads as an action.
func (i *Inventory) Assign(ctx context.Context, nodeName, module string) (bool, error) {
node, err := i.NodeByName(ctx, nodeName)
if err != nil {
return err
return false, err
}
if err := i.runsSomewhere(ctx, module); err != nil {
return err
return false, err
}
_, err = i.store.Pool().Exec(ctx,
tag, err := i.store.Pool().Exec(ctx,
`insert into assignment (node, module) values ($1, $2) on conflict do nothing`,
node.ID, module)
if err != nil && strings.Contains(err.Error(), "assignment_module_fkey") {
return fmt.Errorf("%w: %s", ErrNoSuchModule, module)
return false, fmt.Errorf("%w: %s", ErrNoSuchModule, module)
}
return err
if err != nil {
return false, err
}
return tag.RowsAffected() > 0, nil
}
// Unassign takes a module off a node.
internal/inventory/catalogue_test.go
+19 -9
View File
@@ -77,7 +77,7 @@ func TestAModuleAMachineIsRunningCannotBeForgotten(t *testing.T) {
if err := inv.RegisterModule(t.Context(), manifest("thing", nil, nil), Source{}); err != nil {
t.Fatal(err)
}
if err := inv.Assign(t.Context(), "laptop", "thing"); err != nil {
if _, err := inv.Assign(t.Context(), "laptop", "thing"); err != nil {
t.Fatal(err)
}
@@ -104,7 +104,7 @@ func TestRemovingANodeTakesItsAssignments(t *testing.T) {
if err := inv.RegisterModule(t.Context(), manifest("thing", nil, nil), Source{}); err != nil {
t.Fatal(err)
}
if err := inv.Assign(t.Context(), "laptop", "thing"); err != nil {
if _, err := inv.Assign(t.Context(), "laptop", "thing"); err != nil {
t.Fatal(err)
}
if _, err := inv.store.Pool().Exec(t.Context(), `delete from node where id = $1`, node.ID); err != nil {
@@ -136,14 +136,16 @@ func TestAssigningAModuleTheMeshDoesNotKnowIsRefused(t *testing.T) {
if _, err := inv.AddNode(t.Context(), "laptop"); err != nil {
t.Fatal(err)
}
err := inv.Assign(t.Context(), "laptop", "not-a-module")
_, err := inv.Assign(t.Context(), "laptop", "not-a-module")
if !errors.Is(err, ErrNoSuchModule) {
t.Fatalf("assigning an unknown module gave %v", err)
}
}
func TestAssigningTwiceIsNotAnError(t *testing.T) {
// It is a statement of what should be true, and it already is.
func TestAssigningTwiceIsNotAnErrorAndSaysSo(t *testing.T) {
// It is a statement of what should be true, and it already is — one assignment of a module
// per node is the rule (novox/hq ADR 0115), so a repeat is absorbed, audibly: the caller is
// told nothing was new.
inv := fresh(t)
if _, err := inv.AddNode(t.Context(), "laptop"); err != nil {
t.Fatal(err)
@@ -151,10 +153,18 @@ func TestAssigningTwiceIsNotAnError(t *testing.T) {
if err := inv.RegisterModule(t.Context(), manifest("thing", nil, nil), Source{}); err != nil {
t.Fatal(err)
}
for i := 0; i < 3; i++ {
if err := inv.Assign(t.Context(), "laptop", "thing"); err != nil {
first, err := inv.Assign(t.Context(), "laptop", "thing")
if err != nil || !first {
t.Fatalf("the first assignment is the new one; got fresh=%v err=%v", first, err)
}
for i := 0; i < 2; i++ {
again, err := inv.Assign(t.Context(), "laptop", "thing")
if err != nil {
t.Fatalf("assigning again failed: %v", err)
}
if again {
t.Fatal("a repeat must say nothing was new")
}
}
assigned, err := inv.Assigned(t.Context(), "laptop")
if err != nil {
@@ -277,7 +287,7 @@ func TestBeingBehindNamesTheMachinesRunningTheOldOne(t *testing.T) {
t.Fatal(err)
}
for _, n := range []string{"laptop", "workstation"} {
if err := inv.Assign(t.Context(), n, "thing"); err != nil {
if _, err := inv.Assign(t.Context(), n, "thing"); err != nil {
t.Fatal(err)
}
}
@@ -451,7 +461,7 @@ func TestTheCatalogueSaysWhereEachModuleCameFromAndWhoRunsIt(t *testing.T) {
t.Fatal(err)
}
for _, n := range []string{"workstation", "laptop"} {
if err := inv.Assign(ctx, n, "shell"); err != nil {
if _, err := inv.Assign(ctx, n, "shell"); err != nil {
t.Fatal(err)
}
}
internal/inventory/forget_test.go
+1 -1
View File
@@ -212,7 +212,7 @@ func TestAModuleStillAssignedRefusesBeforeAnythingAboutWhatItHolds(t *testing.T)
if err := inv.SetSettings(ctx, "anchor", "step-ca", map[string]any{"a": 1}); err != nil {
t.Fatal(err)
}
if err := inv.Assign(ctx, "anchor", "step-ca"); err != nil {
if _, err := inv.Assign(ctx, "anchor", "step-ca"); err != nil {
t.Fatal(err)
}
for _, forget := range []func() error{
internal/inventory/migrations/0033-a-carried-peer-is-nameable.sql
+10
View File
@@ -0,0 +1,10 @@
-- A carried peer may be named before it enrols (novox/hq issue 112).
--
-- The tunnel the hub took over routes to machines the predecessor knows by name and the mesh
-- knows only by address. A name the predecessor answers for must keep resolving until the
-- machine behind it is a node — so the operator may state which machine a carried address is,
-- and everything derived from "the machines the mesh knows" (a container's hosts, the hosts
-- fact, the resolver) answers for it in the meantime. The mesh records the statement as the
-- operator's, unverified: enrolment is what verifies it, and enrolling under a different name
-- than the one stated is refused rather than silently renamed.
alter table tunnel_peer add column named text;
internal/inventory/migrations/0034-the-seats-are-data.sql
+17
View File
@@ -0,0 +1,17 @@
-- The seats are data the control plane owns, not a slice compiled into it (novox/hq ADR 0122).
--
-- Until this, the closed set 0110 defines lived only as a Go slice, referenced by name everywhere,
-- so renaming a seat or adding one meant a controller rebuild and a mesh-wide, freeze-prone deploy.
-- The set is now a table: one row per seat, seeded from the binary's defaults the first time the
-- control plane comes up, and thereafter the live copy the control plane reads and an operator can
-- change. A rename becomes an update here rather than a release.
--
-- The name is the key for now, because claims and held records still reference a seat by name; the
-- move to a stable id that a rename does not touch is the next step (ADR 0122). `delivers` is empty
-- for a seat that answers for no provision, matching the compiled default.
create table seat (
name text primary key,
scope text not null,
delivers text not null default '',
decided text not null
);
internal/inventory/migrations/0035-a-seat-keeps-its-former-names.sql
+12
View File
@@ -0,0 +1,12 @@
-- A seat keeps its former names, so a rename breaks nothing (novox/hq ADR 0122).
--
-- Phase 1 made the seat set data, but a rename still broke every reference to the old name — a
-- manifest's claim, a held record, the git-seat lookup — because they name the seat and the name
-- had changed. This is the stable identity ADR 0122 asked for, realised the simple way: a seat's
-- canonical name changes, and its old name becomes an alias that resolves to it forever. Nothing
-- downstream has to change — a manifest goes on claiming the old name, the build machine goes on
-- validating it — and a rename is one operation: set the new name, remember the old.
create table seat_alias (
alias text primary key, -- a former name of a seat
seat text not null -- the seat's current canonical name it resolves to
);
internal/inventory/migrations/0036-a-node-has-an-operator-account.sql
+13
View File
@@ -0,0 +1,13 @@
-- A node has an operator account: the human login on it (novox/hq to-be 29).
--
-- The mesh modelled the machine but not the person on it — `jochens` on novox, `ace` on ace,
-- `jochen` on shanks and g14. That name decides who a file under a home is owned by and which
-- account `ssh <node>` logs in as; it was silently lost when the predecessor's per-node `user:`
-- was not carried over, and `ssh ace` failed to `ace` because nothing here said so.
--
-- Empty rather than null and defaulted, because "no operator account known yet" is a real state
-- (a freshly enrolled machine, a headless box). The home is stored too rather than always assumed
-- to be /home/<account>, because root's is /root and a machine may put a home elsewhere; empty
-- means "derive it" (/root for root, /home/<account> otherwise), so the common case needs no entry.
alter table node add column account text not null default '';
alter table node add column account_home text not null default '';
internal/inventory/migrations/0033-the-bus-keeps-its-users-hashes.sql → internal/inventory/migrations/0037-the-bus-keeps-its-users-hashes.sql
View File
internal/inventory/migrations/0034-a-person-may-reach-the-mesh.sql → internal/inventory/migrations/0038-a-person-may-reach-the-mesh.sql
View File
internal/inventory/named_peer_test.go
+92
View File
@@ -0,0 +1,92 @@
package inventory
// A carried peer is nameable (novox/hq issue 112): the operator states which machine a carried
// address is, the mesh answers for the name until the machine enrols, and enrolment verifies the
// statement rather than silently renaming it.
import (
"strings"
"testing"
)
func TestACarriedPeerIsNamedAndTheRegistrySaysSo(t *testing.T) {
inv := fresh(t)
anAdoptedHub(t, inv)
if err := inv.NamePeer(t.Context(), "192.0.2.2", "home-server"); err != nil {
t.Fatal(err)
}
carried, err := inv.CarriedPeers(t.Context())
if err != nil {
t.Fatal(err)
}
byKey := map[string]CarriedPeer{}
for _, c := range carried {
byKey[c.PublicKey] = c
}
if byKey[peerTwo].Named != "home-server" || byKey[peerThree].Named != "" {
t.Fatalf("the statement was not recorded where it was made: %+v", carried)
}
}
func TestNamingRefusesWhatWouldCollide(t *testing.T) {
inv := fresh(t)
anAdoptedHub(t, inv)
if err := inv.NamePeer(t.Context(), "192.0.2.9", "ghost"); err == nil ||
!strings.Contains(err.Error(), "no carried peer") {
t.Fatalf("naming an address nothing carries must refuse; got %v", err)
}
if err := inv.NamePeer(t.Context(), "192.0.2.2", "anchor"); err == nil ||
!strings.Contains(err.Error(), "node of this mesh") {
t.Fatalf("naming a peer after a node must refuse; got %v", err)
}
if err := inv.NamePeer(t.Context(), "192.0.2.2", "home-server"); err != nil {
t.Fatal(err)
}
if err := inv.NamePeer(t.Context(), "192.0.2.3", "home-server"); err == nil ||
!strings.Contains(err.Error(), "already named") {
t.Fatalf("one machine per name; got %v", err)
}
}
func TestEnrolmentUnderANamedKeyMustUseTheName(t *testing.T) {
inv := fresh(t)
anAdoptedHub(t, inv)
if err := inv.NamePeer(t.Context(), "192.0.2.3", "home-server"); err != nil {
t.Fatal(err)
}
// The wrong name is refused where the operator can read it…
imposter, err := inv.AddNode(t.Context(), "some-other-name")
if err != nil {
t.Fatal(err)
}
if err := inv.RecordOverlayKey(t.Context(), imposter.ID, peerThree); err != nil {
t.Fatal(err)
}
if _, err := inv.AssignAddress(t.Context(), imposter.ID, "192.0.2.0/24"); err == nil ||
!strings.Contains(err.Error(), `named "home-server"`) {
t.Fatalf("enrolling a named peer under another name must refuse; got %v", err)
}
// …and the stated name enrols cleanly, keeping the carried address.
named, err := inv.AddNode(t.Context(), "home-server")
if err != nil {
t.Fatal(err)
}
if err := inv.RecordOverlayKey(t.Context(), named.ID, peerThree); err != nil {
t.Fatal(err)
}
address, err := inv.AssignAddress(t.Context(), named.ID, "192.0.2.0/24")
if err != nil {
t.Fatal(err)
}
if address != "192.0.2.3" {
t.Fatalf("the named peer keeps its carried address; got %s", address)
}
if err := inv.NamePeer(t.Context(), "192.0.2.3", "renamed"); err == nil ||
!strings.Contains(err.Error(), "enrolled as") {
t.Fatalf("an enrolled peer's name is the node's; got %v", err)
}
}
internal/inventory/nodes.go
+41 -2
View File
@@ -55,6 +55,29 @@ type Node struct {
// AdoptedSince is when it last became so; zero for a converged node.
Adopted bool
AdoptedSince time.Time
// Account is the operator's login on this machine — `jochens` on novox, `ace` on ace (novox/hq
// to-be 29). Empty when none is known yet. AccountHome is where that account's home is; empty
// means derive it (/root for root, /home/<account> otherwise), so the common case needs no
// entry. What decides who a file under a home is owned by, and which account `ssh <node>` uses.
Account string
AccountHome string
}
// Home is the account's home directory, derived when not stored: /root for root, /home/<account>
// otherwise. Empty only when there is no account at all.
func (n Node) Home() string {
if n.AccountHome != "" {
return n.AccountHome
}
switch n.Account {
case "":
return ""
case "root":
return "/root"
default:
return "/home/" + n.Account
}
}
// Silent is how long since this node was last heard from, and whether it ever was.
@@ -112,12 +135,13 @@ func (i *Inventory) AddNodeAs(ctx context.Context, name string, adopted bool) (N
// nodeColumns and scanNode are the one reading of a node row, so every way of finding a node
// says whether it is adopted.
const nodeColumns = `id, name, created, last_seen, adopted, adopted_since`
const nodeColumns = `id, name, created, last_seen, adopted, adopted_since, account, account_home`
func scanNode(row pgx.Row) (Node, error) {
var n Node
var seen, since *time.Time
if err := row.Scan(&n.ID, &n.Name, &n.Created, &seen, &n.Adopted, &since); err != nil {
if err := row.Scan(&n.ID, &n.Name, &n.Created, &seen, &n.Adopted, &since,
&n.Account, &n.AccountHome); err != nil {
return Node{}, err
}
if seen != nil {
@@ -129,6 +153,21 @@ func scanNode(row pgx.Row) (Node, error) {
return n, nil
}
// SetAccount records the operator account on a node — its human login — and optionally where that
// account's home is (novox/hq to-be 29). An empty home means the mesh derives it. Clearing the
// account (empty name) is allowed: a machine may stop having a known operator.
func (i *Inventory) SetAccount(ctx context.Context, node, account, home string) error {
tag, err := i.store.Pool().Exec(ctx,
`update node set account = $1, account_home = $2 where name = $3`, account, home, node)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("%w: %s", ErrNoSuchNode, node)
}
return nil
}
// Nodes are every node record, oldest first.
func (i *Inventory) Nodes(ctx context.Context) ([]Node, error) {
rows, err := i.store.Pool().Query(ctx,
internal/inventory/ports_test.go
+2 -2
View File
@@ -221,7 +221,7 @@ func TestWhatAMachineNoLongerHoldsIsAvailableAgain(t *testing.T) {
func TestUnassigningReleasesTheModulesPorts(t *testing.T) {
inv, node := aNodeWithModules(t, "mailu", "other-mail")
ctx := t.Context()
if err := inv.Assign(ctx, node, "mailu"); err != nil {
if _, err := inv.Assign(ctx, node, "mailu"); err != nil {
t.Fatal(err)
}
if _, err := inv.PortFor(ctx, node, "mailu", 25, true); err != nil {
@@ -230,7 +230,7 @@ func TestUnassigningReleasesTheModulesPorts(t *testing.T) {
if err := inv.Unassign(ctx, node, "mailu"); err != nil {
t.Fatal(err)
}
if err := inv.Assign(ctx, node, "other-mail"); err != nil {
if _, err := inv.Assign(ctx, node, "other-mail"); err != nil {
t.Fatal(err)
}
if _, err := inv.PortFor(ctx, node, "other-mail", 25, true); err != nil {
internal/inventory/seats.go
+108
View File
@@ -0,0 +1,108 @@
package inventory
import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The seats the mesh has, as data (novox/hq ADR 0122).
//
// The set the control plane reads is a table here, not a slice compiled into it. It is seeded from
// the binary's defaults the first time the mesh comes up (SeedSeats), and thereafter it is the live
// copy: a rename or an added seat is a write here, and the control plane loads it at startup rather
// than being rebuilt for it.
// Seats is every seat the mesh defines, read from the store.
func (i *Inventory) Seats(ctx context.Context) ([]catalogue.Seat, error) {
rows, err := i.store.Pool().Query(ctx,
`select name, scope, delivers, decided from seat order by name`)
if err != nil {
return nil, err
}
defer rows.Close()
var seats []catalogue.Seat
for rows.Next() {
var s catalogue.Seat
if err := rows.Scan(&s.Name, &s.Scope, &s.Delivers, &s.Decision); err != nil {
return nil, err
}
seats = append(seats, s)
}
return seats, rows.Err()
}
// SeedSeats writes the mesh's default set into the table where it is not already present.
//
// **Idempotent, and never overwriting.** Run every time the control plane migrates, it fills an
// empty table on first boot and adds a seat a new release ships — but it leaves a row already there
// exactly as it is, so an operator's rename in the table is not undone by the next deploy putting
// the old name back. What a release removes from the defaults is not deleted here either; retiring a
// seat is its own decision, not a silent consequence of it dropping out of the binary.
func (i *Inventory) SeedSeats(ctx context.Context, defaults []catalogue.Seat) (int, error) {
var added int
for _, s := range defaults {
tag, err := i.store.Pool().Exec(ctx,
`insert into seat (name, scope, delivers, decided) values ($1, $2, $3, $4)
on conflict (name) do nothing`,
s.Name, s.Scope, s.Delivers, s.Decision)
if err != nil {
return added, err
}
added += int(tag.RowsAffected())
}
return added, nil
}
// Aliases is every former seat name and the seat it now resolves to (novox/hq ADR 0122).
func (i *Inventory) Aliases(ctx context.Context) (map[string]string, error) {
rows, err := i.store.Pool().Query(ctx, `select alias, seat from seat_alias`)
if err != nil {
return nil, err
}
defer rows.Close()
aliases := map[string]string{}
for rows.Next() {
var alias, seat string
if err := rows.Scan(&alias, &seat); err != nil {
return nil, err
}
aliases[alias] = seat
}
return aliases, rows.Err()
}
// RenameSeat gives a seat a new name and keeps the old one as an alias (novox/hq ADR 0122).
//
// **This is the whole of a rename.** The seat's canonical name becomes `to`; `from` is remembered as
// an alias so every reference to it — a manifest's claim, a held record, the build machine's
// embedded set — goes on resolving to the same seat, unchanged. Nothing is rebuilt and nothing
// freezes. Any alias that pointed to `from` is repointed to `to`, so a chain of renames does not
// leave an older name resolving to a name that no longer exists.
func (i *Inventory) RenameSeat(ctx context.Context, from, to string) error {
if from == to {
return fmt.Errorf("a seat is renamed to a different name; %q is already its name", to)
}
tag, err := i.store.Pool().Exec(ctx, `update seat set name = $1 where name = $2`, to, from)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("no seat named %q to rename", from)
}
// The old name resolves to the new one; and any name that resolved to the old one now resolves
// to the new one, so no alias is left pointing at a name that is gone.
if _, err := i.store.Pool().Exec(ctx,
`insert into seat_alias (alias, seat) values ($1, $2)
on conflict (alias) do update set seat = excluded.seat`, from, to); err != nil {
return err
}
if _, err := i.store.Pool().Exec(ctx,
`update seat_alias set seat = $1 where seat = $2`, to, from); err != nil {
return err
}
return nil
}
internal/inventory/seats_test.go
+115
View File
@@ -0,0 +1,115 @@
package inventory
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The seat set is data the control plane owns (novox/hq ADR 0122): seeded from the binary's
// defaults, read back as the working set, and thereafter an operator's to change without a rebuild.
func TestSeatsAreSeededFromTheDefaultsAndReadBack(t *testing.T) {
inv := ForTest(t)
defaults := catalogue.DefaultSeats()
added, err := inv.SeedSeats(t.Context(), defaults)
if err != nil {
t.Fatal(err)
}
if added != len(defaults) {
t.Fatalf("seeded %d of %d seats", added, len(defaults))
}
got, err := inv.Seats(t.Context())
if err != nil {
t.Fatal(err)
}
if len(got) != len(defaults) {
t.Fatalf("read back %d seats, seeded %d", len(got), len(defaults))
}
// The set round-trips: name, scope and what it delivers survive the store.
by := map[string]catalogue.Seat{}
for _, s := range got {
by[s.Name] = s
}
for _, d := range defaults {
if by[d.Name].Scope != d.Scope || by[d.Name].Delivers != d.Delivers {
t.Errorf("%s came back as %+v, seeded %+v", d.Name, by[d.Name], d)
}
}
}
// Re-seeding an already-seeded set adds nothing and changes nothing — every deploy runs SeedSeats,
// and a mesh already holding the set must be left exactly as it is (an operator's edit to a row
// included). A row's fields are not overwritten: on conflict the insert does nothing.
//
// (Phase 1 keys the table by name, so a seat *renamed* in the table would have its old name
// re-seeded — the move to a stable id a rename does not touch is the next step, ADR 0122. This test
// asserts only the property that holds now: an unchanged set re-seeds to a no-op.)
func TestReSeedingAnUnchangedSetIsANoOp(t *testing.T) {
inv := ForTest(t)
defaults := catalogue.DefaultSeats()
if _, err := inv.SeedSeats(t.Context(), defaults); err != nil {
t.Fatal(err)
}
// An operator changes a row's scope in the table — the point of it being data.
if _, err := inv.store.Pool().Exec(t.Context(),
`update seat set scope = 'mesh' where name = 'node-uplink'`); err != nil {
t.Fatal(err)
}
added, err := inv.SeedSeats(t.Context(), defaults)
if err != nil {
t.Fatal(err)
}
if added != 0 {
t.Fatalf("re-seeding an already-present set added %d rows", added)
}
got, err := inv.Seats(t.Context())
if err != nil {
t.Fatal(err)
}
for _, s := range got {
if s.Name == "node-uplink" && s.Scope != "mesh" {
t.Fatalf("re-seeding overwrote the operator's change: node-uplink scope is %q", s.Scope)
}
}
}
// A rename is one operation: the seat gets the new name, the old name becomes an alias that still
// resolves to it (novox/hq ADR 0122).
func TestRenameSeatKeepsTheFormerNameAsAnAlias(t *testing.T) {
inv := ForTest(t)
if _, err := inv.SeedSeats(t.Context(), catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
if err := inv.RenameSeat(t.Context(), "node-packet-filter", "node-firewall"); err != nil {
t.Fatal(err)
}
seats, err := inv.Seats(t.Context())
if err != nil {
t.Fatal(err)
}
names := map[string]bool{}
for _, s := range seats {
names[s.Name] = true
}
if !names["node-firewall"] || names["node-packet-filter"] {
t.Fatalf("the seat was not renamed in place: %v", names)
}
aliases, err := inv.Aliases(t.Context())
if err != nil {
t.Fatal(err)
}
if aliases["node-packet-filter"] != "node-firewall" {
t.Fatalf("the former name is not an alias of the new one: %v", aliases)
}
// Renaming what has no seat is refused; renaming to the same name is refused.
if err := inv.RenameSeat(t.Context(), "no-such-seat", "x"); err == nil {
t.Fatal("renaming a seat that does not exist was accepted")
}
if err := inv.RenameSeat(t.Context(), "node-firewall", "node-firewall"); err == nil {
t.Fatal("renaming a seat to its own name was accepted")
}
}
internal/inventory/secrets_test.go
+1 -1
View File
@@ -350,7 +350,7 @@ func TestACredentialGoesWhenTheConsumerStopsAskingForIt(t *testing.T) {
}, Source{}); err != nil {
t.Fatal(err)
}
if err := inv.Assign(ctx, "consumer", "meshboard"); err != nil {
if _, err := inv.Assign(ctx, "consumer", "meshboard"); err != nil {
t.Fatal(err)
}
if _, err := inv.SecretFor(ctx, "postgres-database", "consumer", "gitea", "provider", ""); err != nil {
internal/inventory/tunnel.go
+48 -2
View File
@@ -33,6 +33,9 @@ type Tunnel struct {
// Port is the port the found interface listened on — one the hosting provider already lets
// through, which is why it is worth taking.
Port int `json:"port"`
// MTU is the found interface's, when it set one; the mesh's interface takes it over so a
// tuned path does not silently regress to the default (novox/hq: a taken tunnel carries its MTU).
MTU int `json:"mtu,omitempty"`
// Address is the interface's own address with its prefix length, 192.0.2.1/24; Range is the
// network that prefix names, 192.0.2.0/24.
Address string `json:"address"`
@@ -85,6 +88,9 @@ type CarriedPeer struct {
Address string
// EnrolledAs names the node that enrolled with this key, or is empty while none has.
EnrolledAs string
// Named is what the operator said this peer is, before it enrolled (novox/hq issue 112) —
// a statement the mesh records and cannot verify, which is why enrolment checks it.
Named string
}
// ErrNoTunnel is asking about a tunnel on a node that presented none.
@@ -247,7 +253,7 @@ func (i *Inventory) AdoptedTunnel(ctx context.Context) (Tunnel, string, bool, er
// adopted no tunnel.
func (i *Inventory) CarriedPeers(ctx context.Context) ([]CarriedPeer, error) {
rows, err := i.store.Pool().Query(ctx,
`select p.public_key, host(p.address), coalesce(n.name, '')
`select p.public_key, host(p.address), coalesce(n.name, ''), coalesce(p.named, '')
from tunnel_peer p
join node hub on hub.id = p.node and hub.is_hub
and hub.tunnel is not null and hub.overlay_key = hub.tunnel->>'public_key'
@@ -260,7 +266,7 @@ func (i *Inventory) CarriedPeers(ctx context.Context) ([]CarriedPeer, error) {
var out []CarriedPeer
for rows.Next() {
var p CarriedPeer
if err := rows.Scan(&p.PublicKey, &p.Address, &p.EnrolledAs); err != nil {
if err := rows.Scan(&p.PublicKey, &p.Address, &p.EnrolledAs, &p.Named); err != nil {
return nil, err
}
out = append(out, p)
@@ -268,6 +274,46 @@ func (i *Inventory) CarriedPeers(ctx context.Context) ([]CarriedPeer, error) {
return out, rows.Err()
}
// NamePeer records the operator's statement that a carried address is a particular machine
// (novox/hq issue 112). Refused when nothing carried has that address, when a node of the mesh
// already has the name — the statement would collide with something verified — and when another
// peer was already named it. Naming an enrolled peer is refused too: its name is the node's now.
func (i *Inventory) NamePeer(ctx context.Context, address, name string) error {
peers, err := i.CarriedPeers(ctx)
if err != nil {
return err
}
var at *CarriedPeer
for idx := range peers {
if peers[idx].Address == address {
at = &peers[idx]
continue
}
if peers[idx].Named == name {
return fmt.Errorf("the carried peer at %s is already named %q — one machine per name",
peers[idx].Address, name)
}
}
if at == nil {
return fmt.Errorf("no carried peer has the address %s — `overlay show` lists them", address)
}
if at.EnrolledAs != "" {
return fmt.Errorf("the peer at %s enrolled as %q — its name is the node's now", address, at.EnrolledAs)
}
if _, err := i.NodeByName(ctx, name); err == nil {
return fmt.Errorf("%q is a node of this mesh — a carried peer cannot be named after one", name)
}
tag, err := i.store.Pool().Exec(ctx,
`update tunnel_peer set named = $2 where host(address) = $1`, address, name)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("no carried peer has the address %s", address)
}
return nil
}
// FoundTunnel is a node's found tunnel with the node's mode, for composing: the takeover is
// declared to an adopted node only, since only there is a found unit kept to be stopped.
type FoundTunnel struct {
internal/link/enrolment.go
+2 -2
View File
@@ -149,7 +149,7 @@ func (e Enrolment) Enrol(ctx context.Context, request EnrolRequest) (reply Enrol
}
if err := e.Inventory.RecordTunnel(ctx, node.ID, inventory.Tunnel{
Interface: request.Tunnel.Interface, Unit: request.Tunnel.Unit,
Config: request.Tunnel.Config, Port: request.Tunnel.Port,
Config: request.Tunnel.Config, Port: request.Tunnel.Port, MTU: request.Tunnel.MTU,
Address: request.Tunnel.Address, Range: request.Tunnel.Range,
PublicKey: request.Tunnel.PublicKey, Peers: peers,
}); err != nil {
@@ -234,7 +234,7 @@ func (e Enrolment) rekey(ctx context.Context, node inventory.Node, r Rekey) erro
peers = append(peers, inventory.TunnelPeer{PublicKey: p.PublicKey, Address: p.Address})
}
err := e.Inventory.Rekey(ctx, node.ID, r.Previous, r.OverlayKey, inventory.Tunnel{
Interface: r.Tunnel.Interface, Unit: r.Tunnel.Unit, Config: r.Tunnel.Config, Port: r.Tunnel.Port,
Interface: r.Tunnel.Interface, Unit: r.Tunnel.Unit, Config: r.Tunnel.Config, Port: r.Tunnel.Port, MTU: r.Tunnel.MTU,
Address: r.Tunnel.Address, Range: r.Tunnel.Range, PublicKey: r.Tunnel.PublicKey, Peers: peers,
})
if err != nil {
internal/link/protocol.go
+1
View File
@@ -107,6 +107,7 @@ type Tunnel struct {
Unit string `json:"unit"`
Config string `json:"config"`
Port int `json:"port"`
MTU int `json:"mtu,omitempty"`
Address string `json:"address"`
Range string `json:"range"`
PublicKey string `json:"public_key"`
internal/overlay/declaration.go
+12
View File
@@ -123,6 +123,18 @@ func config(node Node, peers []Peer, keyPath string) string {
if port := portOf(node.Endpoint); port != "" {
fmt.Fprintf(&b, "ListenPort = %s\n", port)
}
} else if node.TakesOver != nil && node.TakesOver.Port != 0 {
// Not dialable from the hub, but a LAN peer dials this node on the tunnel it took over,
// so the mesh's interface must listen on that same port (novox/hq: a taken tunnel brings
// its port). Without this the takeover guard refuses overlay-up, and re-placing the node
// with an endpoint — the guard's suggested remedy — breaks a NAT'd node's path.
fmt.Fprintf(&b, "ListenPort = %d\n", node.TakesOver.Port)
}
// The MTU the found tunnel carried, when it set one: a path tuned to 1380 (say) stalls TLS
// and hangs transfers if the mesh's interface comes up at the 1420 default, and no ping shows
// it (novox/hq: a taken tunnel carries its MTU).
if node.TakesOver != nil && node.TakesOver.MTU != 0 {
fmt.Fprintf(&b, "MTU = %d\n", node.TakesOver.MTU)
}
// The private key is set from a file the node wrote, so it never appears here and never
// travelled. Everything else in this file came from the mesh; this one line is the node's.
internal/overlay/declaration_test.go
+49
View File
@@ -260,3 +260,52 @@ func TestTakingOverAFoundTunnelIsSaidOnTheInterfacesService(t *testing.T) {
}
}
}
func TestATakenTunnelBringsItsListenPortEvenWhenNotDialable(t *testing.T) {
// A home node behind NAT (no Endpoint, so not Reachable) that took over a tunnel must still
// listen on that tunnel's port, because its LAN peers dial it there (novox/hq: a taken tunnel
// brings its port). Without this the takeover guard refuses overlay-up.
config, _ := declarationFor(t, Node{
Name: "shanks", Key: "SPOKE", Address: "10.10.0.3",
TakesOver: &TakeOver{Interface: "wg0", Unit: "wg-quick@wg0", Port: 51820},
}, nil)
if !strings.Contains(config, "ListenPort = 51820") {
t.Fatalf("a taken tunnel's port must be the mesh interface's ListenPort:\n%s", config)
}
if strings.Contains(config, "Endpoint =") {
t.Error("a node that only listens for LAN peers must not advertise an endpoint")
}
}
func TestANodeWithNoTunnelAndNoEndpointStillListensOnNothing(t *testing.T) {
// The guard against over-emitting: a plain spoke with neither an endpoint nor a taken tunnel
// writes no ListenPort — it purely dials out.
config, _ := declarationFor(t, Node{Name: "laptop", Key: "K", Address: "10.10.0.9"}, nil)
if strings.Contains(config, "ListenPort") {
t.Fatalf("a dial-only node needs no ListenPort:\n%s", config)
}
}
func TestATakenTunnelCarriesItsMTU(t *testing.T) {
// A path tuned to a smaller MTU (1380 here) must survive the takeover — the mesh interface
// comes up with the same MTU, or transfers hang silently (novox/hq: a taken tunnel carries
// its MTU).
config, _ := declarationFor(t, Node{
Name: "shanks", Key: "SPOKE", Address: "10.10.0.3",
TakesOver: &TakeOver{Interface: "wg0", Port: 51820, MTU: 1380},
}, nil)
if !strings.Contains(config, "MTU = 1380") {
t.Fatalf("the tuned MTU was dropped:\n%s", config)
}
}
func TestNoMTULineWhenTheTunnelSetNone(t *testing.T) {
config, _ := declarationFor(t, Node{
Name: "shanks", Key: "SPOKE", Address: "10.10.0.3",
TakesOver: &TakeOver{Interface: "wg0", Port: 51820},
}, nil)
if strings.Contains(config, "MTU") {
t.Fatalf("no MTU was found, so none should be written:\n%s", config)
}
}
internal/overlay/generator.go
+21 -3
View File
@@ -52,7 +52,7 @@ const Addressing = "mesh-addressing"
// for two different purposes. Being **the** one the mesh runs over is singular, and without
// saying so a person who chose a different VPN can still end up with this one dragged back in by
// something that needed the mesh's own addresses. Which is exactly what happened, once.
const TheNetwork = "mesh-private-network"
const TheNetwork = "the-private-network"
// **A resolver's data went the same way as the names.** Two constants used to sit here — a
// `mesh-resolver` module that would write the machines as wildcards, and a `resolver-data`
@@ -161,6 +161,19 @@ func (g *Generator) Nodes() []Node { return g.nodes }
// Graph is the peer list per node, for showing.
func (g *Generator) Graph() Graph { return g.graph }
// hostsTemplate is the mesh's region of `/etc/hosts` — every machine's mesh name at its private
// address, written into a marked region and merged (RosterFile.Shared → `into: block`), so the rest
// of the file (localhost, the machine's own name, other tools' blocks) is kept byte for byte
// (novox/hq issue 128). It is a roster template like any module's: the mesh owns the data, this owns
// the format, and the control plane holds no formatter.
//
// - No floor: no header, no localhost, no `127.0.1.1` — those are the machine's, above the region.
// - A machine's own line is marked, and its mesh name resolves to its mesh address, not loopback.
// - `.Names` is every name the mesh serves (issue 111), so a container reaching a routed name
// finds the machine serving it; machines with no address yet are already left out of the set.
const hostsTemplate = "# The mesh's names. This region is replaced whenever a machine joins or leaves.\n" +
"{{range .Names}}{{.Address}}\t{{.FQDN}}\t{{.Name}}{{if eq .Name $.Node}}\t# this machine{{end}}\n{{end}}"
// Manifest is the module the mesh provides for itself.
//
// It ships with the control plane rather than coming from a repository, because the thing that
@@ -175,8 +188,13 @@ func Manifest() map[string]any {
// Being on the private network is what gives a machine a name, so the module that puts it
// there is what writes them. Asked for rather than generated by a module of its own: the
// mesh knows which machines exist and where; writing that into a hosts file is not a thing
// that needs a module to run nowhere.
"facts": map[string]string{"node-names": "/etc/hosts"},
// that needs a module to run nowhere. The format is a template like any other roster fact —
// the mesh's own module owns the `/etc/hosts` layout the way dnsmasq owns its zones, and the
// control plane holds no formatter (see catalogue.RosterFile). `shared`: the mesh owns only
// its region of the file and keeps the rest (novox/hq issue 128).
"facts": map[string]any{
"node-names": map[string]any{"path": "/etc/hosts", "template": hostsTemplate, "shared": true},
},
"claims": []map[string]any{{"name": TheNetwork, "scope": "node"}},
}
}
internal/overlay/graph.go
+7
View File
@@ -50,6 +50,13 @@ type TakeOver struct {
Interface string
Unit string
Config string
// MTU is the found tunnel's, when it set one — emitted so a tuned path keeps its MTU.
MTU int
// Port is the port the found tunnel listened on. The mesh's interface must listen on it too,
// even on a node that is not dialable from the hub: a home node's LAN peers dial it there
// (novox/hq: a taken tunnel brings its port). Listening is "a peer dials me here"; it is not
// "the hub can dial me", which is Reachable — the two were conflated.
Port int
}
// HostPrefix is one address as a route: /32 for IPv4, /128 for IPv6.
internal/overlay/names.go
+4 -3
View File
@@ -24,9 +24,10 @@ const DefaultSuffix = "internal"
//
// This is not the `/etc/hosts` floor the design removes. That floor existed because a node had to
// reach the mesh's database before its own DNS worked — a fallback for a circularity, and the
// circularity is gone. This is the mechanism itself: the complete set of names in this mesh,
// generated whole and owned by the mesh (novox/hq ADR 0011), rather than a patch written
// underneath something else.
// circularity is gone. This is the mechanism itself: the complete set of names in this mesh
// (novox/hq ADR 0011). Written as a marked region *into* the file rather than as the file: the
// rest of it — `localhost`, the machine's own name, other tools' blocks — is the machine's, and
// writing it whole replaced all of that (novox/hq issue 128).
//
// A file rather than a resolver daemon, deliberately, for now: it works on every Linux, needs no
// package, and has no failure mode of its own. A daemon becomes necessary when names are wanted