Compare commits

..
Author SHA1 Message Date
jschoubben c77d17fb42 A follower joining a watched state hears its current values, and a templates archive is fetched once
A read could start a state's watch before its follower joined, and a change landing in that gap woke nothing (review
of ADR 0580 §2): a joining follower now runs over the copy's current values before any later change. The watched
reader keeps each archive once per URL, and both caches are bounded.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:51:53 +01:00
jschoubben 8311ecb849 The hand-act recorder records through the seat alone, and the watched reader's reads are proved
The controller's hand-act and warranted are gone (issue 1953 L5b) and the bus refuses them, so the recorder no longer
falls back to them: an act the seat does not take is an error saying nothing was recorded. And ADR 0580 clause 1's
reader is shown reading a build by key each time and a manifest once per digest.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:33:17 +01:00
jschoubben ba8b8f28a3 One watched copy of the states a module reads, and the composer's reader over it
A whole state read key by key took the provisioner's composition of one node past 30 s and the assigner's compare past
its call bound (ADR 0580 clause 1): each state is now one watch's current values, kept by its changes; a build is read
by key and a manifest once per digest.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:29:05 +01:00
jschoubben 8af6e72669 Format streams.go as gofmt writes it, which the group's check refused
mesh/delivery merged: source.main moved to novox/mesh-sdk@da41823855a9
mesh/delivery-group group feat/judge-once merged: every member is merged
mesh/check pass 8af6e7266915 on ff5e00f1f894 | unit 8665 run, 1440 kept, 62 skipped | integration 10038 run, 975 kept, 70 skipped | over bound: unit, …
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:22:06 +01:00
jschoubben 62cddf4efc Refuse a manifest judge in mesh-sdk's own check, and never end the data items' read on one module (ADR 0578 §3, §4)
mesh/delivery-group group feat/judge-once rejected: composed together: the checker's verdict at the members' heads (8a8ede2fb2c8): unit: go.gofmt
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: go.gofmt: not as gofmt writes them (run gofmt -w): go/catalogue/streams.go | 62cddf4efc62 on ff5e00f1f894 | unit 0 run, 78 ke…
Review round 1 (issue 1585): L1, DataItemsHeld holds a module by the culprit rule PlansHeld uses, a held module's
data left out, so an item its composition refuses holds that module alone; L2, mesh-sdk's own tests scan its go/
tree as mesh-catalog's checks/sdkjudge does, with the same list of composition's refusals. Two tests merged into one.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:07:50 +01:00
jschoubben 19c204273b Let a verdict warn of what a stricter judge found in a manifest it accepted before (ADR 0579 §1)
Issue 1585: ManifestVerdict and the modules value carry Warnings; the finding manifest-outgrown names it.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:43:10 +01:00
jschoubben a20bdf7952 Carry mesh package versions in build asks and fingerprints
ADR 0573 §2 has every module build against the latest published version of
each mesh package it uses, chosen by the planner and written by the build,
never pinned by hand (issue 1585, S4). The ask, the fetch request and the
merge preview now carry the packages and their versions, and the new-form
fingerprint takes each version, so a package's trunk merge changes every
dependent's fingerprint. A module using no package keeps its fingerprint.

Folded onto feat/judge-once by builder-1585a: Module keeps both Packages and the verdict fields.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:40:12 +01:00
jschoubben 4dfc11b424 Take the reserved stream prefixes out of mesh-sdk with the judge that reads them (ADR 0578 §4)
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:38:05 +01:00
jschoubben 68aaf235d4 Take the judge out of mesh-sdk: it is mesh-delivery's now (ADR 0578 §4, §7)
Issue 1585, S19: the strict parser, every manifest problem function and the judge's tables (192 declarations,
about 5,200 lines) and their 244 tests leave; composition keeps its own refusals and the helpers it uses. The
catalogue's tests decode the manifests they build. ParseManifest, ParseManifestWith, ParseManifestForInputs,
CatalogueProblems and the rest are gone; mesh-delivery-indexer/judge holds them.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:36:24 +01:00
jschoubben 25a2cecd98 Let the tree readers decode, give the judge the manifest's data it reads, and stop judging outside the catalogue (ADR 0578 §3, §4)
Issue 1585, S6-S8: ReadTreeManifestWritten and WithTemplatesFrom decode (the judge in mesh-delivery judges what they
answer); Manifest.Predefined, SetPredefined and FormsWritten expose what the moved judge reads; buscompose names its
removals without the catalogue's allow-list; tests outside the catalogue decode, two tree-judging tests moved.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:33:19 +01:00
jschoubben 7cab079e2e Merge the set-aside field's three tests into one, so the judge work adds no test to the budget
Issue 1585: mesh-sdk and mesh-delivery are both over 500 unit tests; ADR 0578's tests replace, never add.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:20:31 +01:00
jschoubben 9904a03b51 Let the merge preview and the provided manifest decode instead of judging (ADR 0578 §3)
Issue 1585, S12 (mesh-sdk half): buildstep's named paths and delivery.ProvidedManifest read with catalogue.Decode.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:18:56 +01:00
jschoubben 7b09de1e55 Let the composer take each manifest's verdict from its input instead of judging (ADR 0578 §5)
Issue 1585, S11 (mesh-sdk half): Case.Verdicts carries the indexer's verdicts; a refused module is held as its
stub and named; an unknown field is set aside and named, never refused.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 01:57:36 +01:00
jschoubben 7631a7d5de Hold a refused manifest in the bus reader instead of ending the whole read (ADR 0578 §3)
Issue 1585, S2: one refused manifest silenced disk-load for 45 hours. frombus decodes leniently, takes the
indexer's verdict, stands a refused module in as its stub and names it in Input.Refused; DataItemsHeld names it.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 01:57:36 +01:00
jschoubben a343557554 Give the indexer's verdicts on manifests a contract readers take (ADR 0578 §1, §3)
Issue 1585, S3: state mesh-delivery-indexer_verdicts keyed by manifest digest, event manifest-refused, a modules
value carrying its verdict, and TakeManifestVerdict reading a missing verdict as accepted. Contract only, no rules.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 01:57:36 +01:00
jschoubben ccb367e68c Decode a manifest leniently, so a field a reader does not know never leaves its module out (ADR 0578 §3, §6)
Issue 1585, S1: catalogue.Decode sets an unknown field aside and names it (SetAside); only a manifest not read past
the field is still left out.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 01:57:36 +01:00
189 changed files with 1334 additions and 11660 deletions
+1 -1
View File
@@ -33,7 +33,7 @@ const KeptBucket = "mesh-controller_states-announced"
var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// Key is module's key in KeptBucket: the module itself, which must be one plain name, since a module's name is part of
// its buckets' names (catalogue.StateProblems).
// its buckets' names (the indexer's judge, novox/hq ADR 0578 §1).
func Key(module string) (string, error) {
if !name.MatchString(module) {
return "", fmt.Errorf("announce: %q is not a module that keeps state: lower-case letters, digits and hyphens", module)
+5 -4
View File
@@ -80,11 +80,12 @@ func ModuleTrees(ctx context.Context, run forgekey.Runner, env []string, dir, co
}
// narrow sets mt's Paths and its named Tree when the module's manifest names paths on every artifact (novox/hq ADR 0499
// clause 2): one `git ls-tree -z` gives each named path's object at commit. The merge preview reads the manifest here, as
// the indexer does, with catalogue.ParseManifestForInputs (issue 1751), and no other build step reads paths from one (ADR 0495 clause 1). A manifest that does not parse
// is not narrowed: the indexer gives it none. A named path absent at commit fails, naming the module and the path.
// clause 2): one `git ls-tree -z` gives each named path's object at commit. The merge preview decodes the manifest here
// (catalogue.Decode: a build step judges no manifest, novox/hq ADR 0578 §3), and no other build step reads paths from one
// (ADR 0495 clause 1). A manifest that does not decode is not narrowed: the indexer gives it none. A named path absent at
// commit fails, naming the module and the path.
func narrow(ctx context.Context, run forgekey.Runner, env []string, dir, commit, mdir string, raw []byte, mt *ModuleTree) error {
m, err := catalogue.ParseManifestForInputs(raw)
m, err := catalogue.Decode(raw)
if err != nil {
return nil
}
+5 -8
View File
@@ -75,20 +75,17 @@ func TestModuleTreesFailsNamingANamedPathAbsentAtTheCommit(t *testing.T) {
}
}
// A manifest whose config value falls back to a predefined key is narrowed here as the indexer narrows it: both read it
// with catalogue.ParseManifestForInputs (issue 1751), so a build module holding no published list computes the same
// tree (ADR 0499 clause 2).
// A manifest whose config value falls back to a predefined key is narrowed here: the merge preview decodes it and judges
// nothing (novox/hq ADR 0578 §3), so a build module holding no published list computes the tree the indexer does (ADR
// 0499 clause 2; issue 1751).
func TestModuleTreesNarrowsAManifestFallingBackToAPredefinedKey(t *testing.T) {
was := catalogue.ReplacePublished(nil) // the build modules' case: no published list
defer catalogue.ReplacePublished(was)
root := t.TempDir()
raw := `{"module":"xterm","version":"1","config":{"size":{"type":"number","description":"The text size.","predefined":"font-size","default":12,"runtime":true}},` +
`"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript","paths":["src"]}]}}`
if _, err := catalogue.ParseManifest([]byte(raw)); err == nil {
t.Fatal("ParseManifest reads the manifest with no list; the test proves nothing")
}
if _, err := catalogue.ParseManifestForInputs([]byte(raw)); err != nil {
t.Fatalf("the indexer's read refuses the manifest: %v", err)
if _, err := catalogue.Decode([]byte(raw)); err != nil {
t.Fatalf("the preview's read refuses the manifest: %v", err)
}
writeTreeFile(t, root, "module.json", raw)
run := func(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
+3 -2
View File
@@ -42,7 +42,7 @@ var (
// write's step, clause 5); the node-engine's acts bucket read (STREAM.INFO with a subject filter lists what a removal
// would take, so the tools read before they remove) and its keys removed, which nothing else may empty when it fills; the ASSIGNMENTS stream read and deleted whole (the lease
// write, research 072 row 33). A
// removal outside the set is refused, as the catalogue refuses it (catalogue.BusRemovals).
// removal outside the set is refused, as the indexer's judge refuses it in a manifest (novox/hq ADR 0578 §1).
func BusRemovalGrants(removes []string) ([]string, error) {
var out []string
for _, r := range removes {
@@ -56,7 +56,8 @@ func BusRemovalGrants(removes []string) ([]string, error) {
out = append(out, "$JS.API.STREAM.INFO."+AssignmentsStream, "$JS.API.STREAM.DELETE."+AssignmentsStream)
default:
return nil, fmt.Errorf("a bus account removes %q, which is not one of %s (novox/hq ADR 0564 Limits 4)", r,
strings.Join(catalogue.BusRemovals, ", "))
strings.Join([]string{catalogue.RemovesControllerDeadLetterConsumer, catalogue.RemovesEngineActs,
catalogue.RemovesAssignmentsStream}, ", "))
}
}
return out, nil
+1 -1
View File
@@ -108,7 +108,7 @@ func (s ModuleStream) Stream() Stream {
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
// to be the only one (catalogue.WorkQueueStreamProblems).
// to be the only one (the indexer's judge, novox/hq ADR 0578 §1).
type QueueOwner struct {
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
// 0436 §2).
+1 -1
View File
@@ -108,7 +108,7 @@ type ProcessPrincipal struct {
Emits []string
// Queues are the module's own work-queue streams this process is the one consumer of.
Queues []QueueOwner
// Removes are what its account may remove from the bus, as its bus declares them (catalogue.BusRemovals; novox/hq
// Removes are what its account may remove from the bus, as its bus declares them (the indexer's judge refuses any other; novox/hq
// ADR 0564 Limits 4): a tools bundle's account, whose whole authority is then BusRemovalGrants.
Removes []string
// State is the module's own state, by local name, this process reads and writes.
+1 -21
View File
@@ -2,7 +2,6 @@ package catalogue
import (
"encoding/json"
"strings"
"testing"
)
@@ -31,7 +30,7 @@ func aServiceModule(t *testing.T) Manifest {
"build": {"artifacts": [{"name": "code", "kind": "bundle", "language": "go", "system": "arch",
"from": "cmd/svc", "binary": "svc"}]}
}`
m, err := ParseManifest([]byte(raw))
m, err := decodeForTest([]byte(raw))
if err != nil {
t.Fatalf("the service's manifest is refused: %v", err)
}
@@ -145,22 +144,3 @@ func TestWhatAProcessReplacesIsNamedAsTheHostRecordedIt(t *testing.T) {
t.Fatalf("the process replaces %s", got)
}
}
func TestWhatReplacesMayNameIsRefusedNearItsAuthor(t *testing.T) {
for what, resource := range map[string]string{
"a container": `{"id":"c","type":"container","name":"c","image":"x@` + aServiceDigest + `","replaces":["old"]}`,
"a step": `{"id":"p","type":"process","name":"p","run":["./p"],"run-once":true,"replaces":["old"]}`,
"something declared": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["p"]}`,
"another module's": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["other.old"]}`,
"not a list": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":"old"}`,
} {
raw := `{"module":"m","version":"1","resources":[` + resource + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "replace") {
t.Errorf("replaces on %s was accepted: %v", what, err)
}
}
ok := `{"module":"m","version":"1","resources":[{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["old"]}]}`
if _, err := ParseManifest([]byte(ok)); err != nil {
t.Errorf("a process replacing what its module no longer declares was refused: %v", err)
}
}
-30
View File
@@ -76,33 +76,3 @@ func TestAModuleMayNotOwnWhatAnotherAccesses(t *testing.T) {
t.Errorf("the refusal does not explain the ownership conflict: %v", err)
}
}
// A manifest states an access's extent, the way a listening port states its source. An absolute
// path and a mode of read or read-write; anything else is refused rather than acted on wrongly.
func TestAnAccessIsValidated(t *testing.T) {
good := []byte(`{"module":"m","accesses":[{"path":"/services/media","mode":"read-write"}]}`)
if _, err := ParseManifest(good); err != nil {
t.Fatalf("a valid access was refused: %v", err)
}
// Mode is optional and narrows to read — the safe default, because the danger with an access
// is being given more than was meant, not less.
bare := []byte(`{"module":"m","accesses":[{"path":"/services/media"}]}`)
m, err := ParseManifest(bare)
if err != nil {
t.Fatalf("an access with no mode was refused: %v", err)
}
if m.Accesses[0].At() != AccessRead {
t.Errorf("an access with no mode is %q, want %q", m.Accesses[0].At(), AccessRead)
}
relative := []byte(`{"module":"m","accesses":[{"path":"services/media","mode":"read"}]}`)
if _, err := ParseManifest(relative); err == nil {
t.Error("a relative access path was accepted")
}
wrong := []byte(`{"module":"m","accesses":[{"path":"/services/media","mode":"append"}]}`)
if _, err := ParseManifest(wrong); err == nil {
t.Error("an access with an unknown mode was accepted")
}
}
-9
View File
@@ -258,15 +258,6 @@ WantedBy=multi-user.target
}
}
func TestAGuardedPortMustBeAPort(t *testing.T) {
if _, err := ParseManifest([]byte(`{"module":"postgres","guards":[5432]}`)); err != nil {
t.Fatalf("guards is refused: %v", err)
}
if _, err := ParseManifest([]byte(`{"module":"postgres","guards":[0]}`)); err == nil {
t.Fatal("guarding port 0 was accepted")
}
}
func keys[V any](m map[string]V) []string {
return sortedKeys(m)
}
+1 -35
View File
@@ -43,7 +43,7 @@ const agentModule = `{"module": "agent", "version": "1", "resources": [
]}`
func TestTheAgentAccountIsDeclaredNeverRootOnlyToAnEngineThatJudgesIt(t *testing.T) {
m, err := ParseManifest([]byte(agentModule))
m, err := decodeForTest([]byte(agentModule))
if err != nil {
t.Fatal(err)
}
@@ -83,37 +83,3 @@ func TestTheAgentAccountIsDeclaredNeverRootOnlyToAnEngineThatJudgesIt(t *testing
t.Errorf("with no agent account, the agent's directory is the operator's: %v", home)
}
}
func TestTheRuntimeIsToldTheAgentAccount(t *testing.T) {
with := Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}}
envOf := func(r Resolution) map[string]string {
t.Helper()
r.Node, r.Modules = "anchor", []Manifest{aToolsModule(t, "nftables", "tools/index.js"), theRuntime(t)}
out, err := r.Declaration(with)
if err != nil {
t.Fatal(err)
}
process := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())
if process == nil {
t.Fatal("no runtime process was composed")
}
return process["env"].(map[string]string)
}
env := envOf(Resolution{Account: "ops", AgentAccount: "agent"})
if env[RuntimeAgentAccount] != "agent" || env[RuntimeAgentHome] != "/home/agent" || env[RuntimeOperatorAccount] != "ops" {
t.Errorf("the runtime is not told whom agents run as: %v", env)
}
env = envOf(Resolution{Account: "ops"})
if env[RuntimeAgentAccount] != "ops" || env[RuntimeAgentHome] != "/home/ops" {
t.Errorf("with no agent account, agents run as the operator: %v", env)
}
env = envOf(Resolution{})
if _, set := env[RuntimeAgentAccount]; set {
t.Errorf("a machine with no account names an agent account: %v", env)
}
if problems := bundleEnvProblems("x", Artifact{Name: "b", Kind: ArtifactBundle, Loads: []string{"x"},
Env: map[string]string{RuntimeAgentAccount: "me"}}); len(problems) == 0 {
t.Error("a bundle may tell the runtime whom agents run as")
}
}
@@ -14,22 +14,6 @@ import (
// the old wire protocol asks for the one server being retired, so both directions are refused at the
// parser — this is judged from the manifest alone, no store needed.
func TestAManifestProvidingAmqpIsRefused(t *testing.T) {
raw := []byte(`{"module":"old-broker","version":"1","provides":[{"name":"amqp","scope":"mesh"}]}`)
_, err := ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), `provides "amqp", which is not a provision`) {
t.Fatalf("a module providing amqp was not refused, or not for the reason: %v", err)
}
}
func TestAManifestRequiringAmqpIsRefused(t *testing.T) {
raw := []byte(`{"module":"forwarder","version":"1","requires":["amqp"]}`)
_, err := ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), `requires "amqp", which is not a provision`) {
t.Fatalf("a module requiring amqp was not refused, or not for the reason: %v", err)
}
}
// And the catalogue (internal/beside) names it nowhere — the three modules that did are removed under
// design 28 task 5.4, not converted.
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
+2 -14
View File
@@ -5,28 +5,16 @@ import (
"testing"
)
// A backup contribution written by hand still names its module's own directories; one it does not
// declare is refused where it is written rather than reaching the holder as the literal text. (The
// catalogue check refuses a hand-written backup line at all since ADR 0233; parsing still reads one,
// because a module on the shelf was written before.)
func TestABackupNamingAnUndeclaredDirectoryIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:nowhere}"}]}`))
if err == nil || !strings.Contains(err.Error(), "nowhere") {
t.Fatalf("a backup of an undeclared directory was accepted: %v", err)
}
}
// A module on the shelf from before ADR 0233, which writes its backup lines by hand and declares no
// data, is still backed up: its lines are placed as written, each module's under a comment naming it.
func TestHandWrittenBackupLinesOfAnOlderModuleArePlaced(t *testing.T) {
pg, err := ParseManifest([]byte(`{"module":"pg","version":"1",
pg, err := decodeForTest([]byte(`{"module":"pg","version":"1",
"resources":[{"id":"dumps","type":"directory","path":"/srv/pg/dumps","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"run pg-dump-all\npath ${dir:dumps}"}]}`))
if err != nil {
t.Fatal(err)
}
mail, err := ParseManifest([]byte(`{"module":"mail","version":"1",
mail, err := decodeForTest([]byte(`{"module":"mail","version":"1",
"resources":[{"id":"spool","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:spool}"}]}`))
if err != nil {
+2 -24
View File
@@ -1,34 +1,12 @@
package catalogue
import (
"strings"
"testing"
)
// The module that provides the artifact store may not be delivered through it.
//
// Building publishes to the store and the builder will not start without one, so a module that
// provides the store and also builds something asks the mesh to put an artifact into the thing
// that artifact is needed to create. On a mesh new enough to have no registry, that is a build
// that never returns (novox/hq 04-ISSUES/029).
func TestTheArtifactStoreCannotBeDeliveredThroughItself(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"registry","version":"1",` +
`"provides":[{"name":"artifact-store","scope":"mesh"}],` +
`"build":{"artifacts":[{"name":"registry","kind":"upstream","from":"registry:2"}]},` +
`"resources":[{"id":"store","type":"container","name":"mesh-registry",` +
`"artifact":"registry","ports":["5000:5000"]}]}`))
if err == nil {
t.Fatal("a registry module that builds its own image was accepted; the build has " +
"nowhere to publish until the module it belongs to is already running")
}
if !strings.Contains(err.Error(), "artifact-store") {
t.Fatalf("refused without naming the provision the cycle turns on: %v", err)
}
}
// Naming the image directly is the way out, and must stay accepted.
func TestAnArtifactStoreThatNamesItsImageIsAccepted(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"registry","version":"1",` +
_, err := decodeForTest([]byte(`{"module":"registry","version":"1",` +
`"provides":[{"name":"artifact-store","scope":"mesh"}],` +
`"resources":[{"id":"store","type":"container","name":"mesh-registry",` +
`"image":"registry@sha256:` +
@@ -41,7 +19,7 @@ func TestAnArtifactStoreThatNamesItsImageIsAccepted(t *testing.T) {
// And an ordinary module still builds whatever it likes.
func TestAModuleThatDoesNotProvideTheStoreStillBuilds(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"forge","version":"1",` +
_, err := decodeForTest([]byte(`{"module":"forge","version":"1",` +
`"build":{"artifacts":[{"name":"forge","kind":"upstream","from":"gitea/gitea:1.22"}]},` +
`"resources":[{"id":"run","type":"container","name":"forge","artifact":"forge"}]}`))
if err != nil {
+1 -36
View File
@@ -174,7 +174,7 @@ func TestALocalRequirementIsStillAnsweredLocally(t *testing.T) {
func TestTheShortFormStillMeansHere(t *testing.T) {
// `"provides": ["shell"]` must keep meaning what it meant, or every existing manifest
// silently changes meaning.
m, err := ParseManifest([]byte(`{"module":"zsh","version":"1","provides":["shell"]}`))
m, err := decodeForTest([]byte(`{"module":"zsh","version":"1","provides":["shell"]}`))
if err != nil {
t.Fatal(err)
}
@@ -183,19 +183,6 @@ func TestTheShortFormStillMeansHere(t *testing.T) {
}
}
func TestASiteScopedProvisionIsRefused(t *testing.T) {
// Meaningful for a claim — one DHCP server per segment — and not yet meaningful for a
// provision, because nothing knows how to reach "the one at my site".
_, err := ParseManifest([]byte(
`{"module":"dns","version":"1","provides":[{"name":"resolver","scope":"site"}]}`))
if err == nil {
t.Fatal("a site-scoped provision was accepted")
}
if !strings.Contains(err.Error(), "site") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
// Being told, which is the difference between knowing and being able to.
func boundShelf() map[string]Manifest {
@@ -308,25 +295,3 @@ func TestBindingSomethingAnsweredHereWritesNothing(t *testing.T) {
}
}
}
func TestBindingWhatYouDoNotRequireIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"app","version":"1",
"binds":{"postgres-database":"/etc/app/db.json"}}`))
if err == nil {
t.Fatal("a module was told about something it never asked for")
}
if !strings.Contains(err.Error(), "does not require") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
func TestServingWhatYouDoNotProvideIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"app","version":"1",
"serves":{"postgres-database":{"port":5432}}}`))
if err == nil {
t.Fatal("a module served something it does not provide")
}
if !strings.Contains(err.Error(), "does not provide") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
+3 -3
View File
@@ -88,7 +88,7 @@ func TestAMalformedShareIsRefused(t *testing.T) {
"other require": `{"name":"route","shares":[{"to":"a","methods":["get"]}]}`,
}
for name, entry := range cases {
_, err := ParseManifest([]byte(`{"module":"files","version":"1","requires":[` + entry + `]}`))
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":[` + entry + `]}`))
if err == nil {
t.Errorf("%s: accepted", name)
continue
@@ -98,7 +98,7 @@ func TestAMalformedShareIsRefused(t *testing.T) {
}
}
// The stored form is held to the same rules.
_, err := ParseManifest([]byte(`{"module":"files","version":"1","requires":["s3-bucket"],
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":["s3-bucket"],
"requires-shares":{"s3-bucket":[{"to":"a","methods":[]}]}}`))
if err == nil || !strings.Contains(err.Error(), "ADR 0468 §8") {
t.Errorf("the stored form was accepted: %v", err)
@@ -169,7 +169,7 @@ func TestChangingOnlyTheSharesDoesNotRecreateTheConsumer(t *testing.T) {
// 1591/AC5: a consumer whose contributes to s3-bucket names `shares` itself is refused.
func TestAConsumerCannotContributeSharesItself(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"files","version":"1",
_, err := decodeForTest([]byte(`{"module":"files","version":"1",
"contributes":{"s3-bucket":{"shares":"x"}},
"requires":[{"name":"s3-bucket","shares":[{"to":"a","methods":["get"]}]}]}`))
if err == nil || !strings.Contains(err.Error(), "ADR 0468 §8") {
-350
View File
@@ -2,10 +2,7 @@ package catalogue
import (
"fmt"
"net"
"net/url"
"path"
"regexp"
"sort"
"strings"
)
@@ -237,152 +234,6 @@ func withVersion(value any, version string) any {
return value
}
// checkBuild is the manifest's own account of what it builds.
func (b *Build) problems(module string) []string {
if b == nil {
return nil
}
var problems []string
seen := map[string]bool{}
for _, a := range b.Artifacts {
if a.Name == "" {
problems = append(problems, module+" builds an artifact with no name")
continue
}
if seen[a.Name] {
problems = append(problems, fmt.Sprintf(
"%s builds two artifacts called %q, and a resource naming it could mean either",
module, a.Name))
}
seen[a.Name] = true
switch a.Kind {
case ArtifactImage, ArtifactArchive, ArtifactUpstream, ArtifactBundle, ArtifactPackage:
default:
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q, and an artifact is %q, %q, %q or %q",
module, a.Name, a.Kind, ArtifactImage, ArtifactArchive, ArtifactUpstream,
ArtifactBundle+", "+ArtifactPackage))
}
// **A bundle is built from the module itself, so it says a language instead.** Everything
// else names what it is built from: a Dockerfile, a directory, somebody else's reference.
// A bundle's source is the module's own directory by definition, and what it needs to say
// is which compiler — because the mesh chooses that, and cannot choose for a module that
// has not said.
if len(a.External) > 0 && (a.Kind != ArtifactBundle || a.Language != "typescript") {
problems = append(problems, fmt.Sprintf(
"%s: %q names packages it keeps external, and only a TypeScript bundle is bundled into "+
"one file with some kept out (novox/hq ADR 0193)", module, a.Name))
}
if len(a.Env) > 0 && a.Kind != ArtifactBundle {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and says what it is given (env). Only a bundle the node's runtime "+
"serves is given words (novox/hq ADR 0192); a container says its own environment",
module, a.Name, a.Kind))
}
if a.Kind == ArtifactBundle || a.Kind == ArtifactPackage {
// **Except for a language that compiles to a binary, where it names which one**
// (novox/hq 04-ISSUES/142). A bundle in an interpreted language is the module's own
// directory compiled whole, and naming a source would be describing its own build. A
// repository written in a compiled language holds several commands — the host and its
// bootstrap live in one, and the mesh needs the host — and "the module's own directory"
// is then not a package at all. So the compiled case may say which package, and says
// the module root by saying nothing.
if a.From != "" && !compilesToABinary(a.Language) {
problems = append(problems, fmt.Sprintf(
"%s: %q is a bundle and names what it is built from (%q). A bundle is built "+
"from the module's own directory; what it says is the language",
module, a.Name, a.From))
}
if strings.TrimSpace(a.Language) == "" {
problems = append(problems, fmt.Sprintf(
"%s: %q is a bundle and says no language, so nothing can choose a compiler "+
"for it", module, a.Name))
}
problems = append(problems, bundleEnvProblems(module, a)...)
// What the runtime loads is among what was compiled (ADR 0175): a name here that is
// not an entrypoint is a file the bundle does not contain, and the runtime would
// fail to import it on every machine rather than here.
for _, load := range a.Loads {
found := false
for _, e := range a.Entrypoints {
found = found || e == load
}
// A bundle compiled to a binary is one executable: the runtime loads that or nothing
// (novox/hq ADR 0193).
if bin := BinaryOf(a); bin != "" {
found = load == bin
}
if !found {
problems = append(problems, fmt.Sprintf(
"%s: %q says the runtime loads %q, which is not among its entrypoints — "+
"what is loaded is compiled, so it is named there too", module, a.Name, load))
}
}
// **A system, for a language that compiles to a binary** (novox/hq ADR 0142). A binary
// is pinned to one operating system at link time so a host refuses to touch a machine
// it was not built for (novox/hq ADR 0005); an artifact that says nothing would be
// compiled for whatever the build machine happened to be, which reads as portable and
// is not.
if compiled := compilesToABinary(a.Language); compiled && strings.TrimSpace(a.System) == "" {
problems = append(problems, fmt.Sprintf(
"%s: %q is compiled to a binary and says no system, so it would be built for "+
"whatever the build machine happens to be. Declare one artifact per "+
"system: %s", module, a.Name, spokenSystems()))
} else if !compiled && strings.TrimSpace(a.System) != "" {
problems = append(problems, fmt.Sprintf(
"%s: %q names the system %q and is written in %q, which compiles to code that "+
"runs anywhere — a system that decides nothing reads as though it did",
module, a.Name, a.System, a.Language))
} else if compiled && !knownSystem(a.System) {
problems = append(problems, fmt.Sprintf(
"%s: %q is built for %q, and a system is %s",
module, a.Name, a.System, spokenSystems()))
}
} else {
if a.From == "" && a.URL == "" {
problems = append(problems, fmt.Sprintf(
"%s: %q says nothing about what it is built from", module, a.Name))
}
if a.Language != "" {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and names a language. Only a bundle is compiled by the mesh; "+
"everything else brings its own recipe", module, a.Name, a.Kind))
}
}
if a.Compiles != "" {
if a.Kind != ArtifactImage {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and names a Go package it compiles. Only an image's recipe is "+
"told which; a bundle names its package in from (novox/hq ADR 0267)", module, a.Name, a.Kind))
} else if c := strings.TrimSpace(a.Compiles); strings.HasPrefix(c, "/") || c == ".." ||
strings.HasPrefix(c, "../") || strings.Contains(c, "/../") || strings.HasSuffix(c, "/..") {
problems = append(problems, fmt.Sprintf(
"%s: %q compiles %q, which leaves the tree it is built in", module, a.Name, a.Compiles))
}
}
// An upstream image is named, not read from the repository, so the path rule does not
// apply to it — and applying it anyway would refuse every reference with a registry host
// in it.
if a.Kind != ArtifactUpstream &&
(strings.HasPrefix(a.From, "/") || strings.Contains(a.From, "..")) {
// A build reads its own repository and nothing else. A path leaving it would make
// what gets built depend on whatever happens to be on the machine building it.
problems = append(problems, fmt.Sprintf(
"%s: %q is built from %q, which is outside its own repository",
module, a.Name, a.From))
}
if a.Kind == ArtifactUpstream && !strings.Contains(a.From, ":") {
// Without a tag or digest, what gets mirrored is whatever `latest` means today, and
// a module pinned to that is not pinned.
problems = append(problems, fmt.Sprintf(
"%s: %q mirrors %q, which names no tag or digest", module, a.Name, a.From))
}
problems = append(problems, upstreamFileProblems(module, a)...)
problems = append(problems, pathsProblems(module, a)...)
}
return problems
}
// NamedPaths are the paths a module's fingerprint takes its tree from (novox/hq ADR 0499 clause 2): the union of the
// paths its artifacts name, cleaned and sorted, and every true only when the module builds something and every artifact
// names paths; otherwise none, and the module keeps its directory's tree.
@@ -410,122 +261,6 @@ func (m Manifest) NamedPaths() (paths []string, every bool) {
return paths, true
}
// pathsProblems is what is wrong with an artifact's `paths` (novox/hq ADR 0499 clause 1): named on a kind whose build
// source is derived or named otherwise, or a path that leaves the module's directory, is its whole directory, or is a
// pattern.
func pathsProblems(module string, a Artifact) []string {
if a.Paths == nil {
return nil
}
switch {
case a.Kind == ArtifactBundle && a.Language == "go":
return []string{fmt.Sprintf("%s: %q is a Go program and names paths: its build source is derived from the "+
"package it compiles (novox/hq ADR 0267 clause 1, ADR 0499 clause 1)", module, a.Name)}
case a.Kind != ArtifactBundle && a.Kind != ArtifactPackage:
return []string{fmt.Sprintf("%s: %q is a %q and names paths; only a package or a bundle in a language other "+
"than Go names the paths it is built from (novox/hq ADR 0499 clause 1)", module, a.Name, a.Kind)}
case len(a.Paths) == 0:
return []string{fmt.Sprintf("%s: %q names no path in paths; leave the word out to build from the module's "+
"directory", module, a.Name)}
}
var problems []string
for _, p := range a.Paths {
clean := path.Clean(p)
switch {
case strings.TrimSpace(p) == "":
problems = append(problems, fmt.Sprintf("%s: %q names an empty path", module, a.Name))
case strings.HasPrefix(p, "/") || clean == ".." || strings.HasPrefix(clean, "../"):
problems = append(problems, fmt.Sprintf("%s: %q names the path %q, which leaves the module's directory",
module, a.Name, p))
case clean == ".":
problems = append(problems, fmt.Sprintf("%s: %q names the module's whole directory (%q) as a path; leave "+
"the word out for that", module, a.Name, p))
case strings.ContainsAny(p, "*?[]{}\\"):
problems = append(problems, fmt.Sprintf("%s: %q names %q, which is a pattern; paths names files and "+
"directories", module, a.Name, p))
}
}
return problems
}
// reservedSuffixes are names no public vendor publishes under: an installation's own and documentation's.
var reservedSuffixes = []string{".internal", ".local", ".lan", ".home", ".corp", ".localhost", ".home.arpa",
".test", ".invalid", ".example", ".intranet", ".private"}
// notAPublicHost says why a host is not a public vendor's name, or "".
func notAPublicHost(host string) string {
h := strings.ToLower(strings.TrimSuffix(host, "."))
if net.ParseIP(h) != nil {
return "an address rather than a name"
}
if !strings.Contains(h, ".") {
return "a name with no domain"
}
for _, s := range reservedSuffixes {
if strings.HasSuffix(h, s) {
return "a name under " + s + ", which is nobody's public domain"
}
}
return ""
}
// sha256Hex is a pinned sha256 as a manifest writes it: 64 lower-case hexadecimal characters.
var sha256Hex = regexp.MustCompile(`^[0-9a-f]{64}$`)
// upstreamFileProblems is what is wrong with an archive pinned by URL (novox/hq ADR 0316 §2, §6): it is an
// archive, fetched over https, pinned by a sha256 and a version, and it names no directory of the repository
// besides; a file name or a directory within is a plain path that stays where it is put.
func upstreamFileProblems(module string, a Artifact) []string {
pinned := a.URL != "" || a.SHA256 != "" || a.Version != "" || a.File != "" || a.Within != ""
if !pinned {
return nil
}
var problems []string
say := func(format string, args ...any) {
problems = append(problems, fmt.Sprintf("%s: %q ", module, a.Name)+fmt.Sprintf(format, args...))
}
if a.Kind != ArtifactArchive {
say("is a %q and names an upstream file (url, sha256, version, file, within): only an archive is "+
"mirrored from one (novox/hq ADR 0316)", a.Kind)
return problems
}
if a.URL == "" {
say("names a sha256, a version, a file or a directory within, and no url to fetch it from")
return problems
}
if a.From != "" {
say("is built from %q and fetched from %s: an archive is one or the other", a.From, a.URL)
}
if u, err := url.Parse(a.URL); err != nil || u.Scheme != "https" || u.Host == "" {
say("is fetched from %q, and an upstream file is fetched over https from a named host", a.URL)
} else if why := notAPublicHost(u.Hostname()); why != "" {
say("is fetched from %s, %s: an upstream file is somebody else's release, published under a public name", a.URL, why)
}
if !sha256Hex.MatchString(a.SHA256) {
say("is fetched from %s without a pinned sha256 (64 lower-case hexadecimal characters): what that "+
"address serves changes under the mesh, and only the digest says it is what was reviewed", a.URL)
}
if strings.TrimSpace(a.Version) == "" {
say("pins %s and says no version, so a person reading the pin cannot tell which release it is", a.URL)
}
if a.File != "" && a.Within != "" {
say("names a file (%q) and a directory within (%q): a file is kept whole, a directory is taken out "+
"of an archive", a.File, a.Within)
}
for field, p := range map[string]string{"file": a.File, "within": a.Within} {
if p == "" {
continue
}
clean := path.Clean(p)
if clean != p || strings.HasPrefix(p, "/") || clean == "." || clean == ".." || strings.HasPrefix(clean, "../") ||
(field == "file" && strings.Contains(p, "/")) {
say("names %s %q, which is not a plain relative %s", field, p, map[string]string{
"file": "file name", "within": "directory"}[field])
}
}
return problems
}
// oneOrOther keeps the message readable for one artifact and for several, because a message that
// says "neither" about one thing reads as a bug in the message.
func oneOrOther(n int) string {
@@ -535,30 +270,6 @@ func oneOrOther(n int) string {
return "them"
}
// Systems the mesh builds binaries for, which is the set a host may be pinned to (novox/hq ADR 0005).
//
// **A closed list, and the host's own, not the compiler's.** These are not the values a Go toolchain
// would call an operating system — the difference between two of them is a C library, not a kernel.
// They are what a machine reports itself to be and what a host is linked to refuse, so the list that
// matters is the one the host understands.
var systems = []string{"alpine", "android", "arch"}
// knownSystem is whether the mesh builds for it.
func knownSystem(system string) bool {
want := strings.ToLower(strings.TrimSpace(system))
for _, s := range systems {
if s == want {
return true
}
}
return false
}
// spokenSystems is the list as a refusal says it, so a reader is one edit from right.
func spokenSystems() string {
return strings.Join(systems, ", ")
}
// compilesToABinary is whether this language's bundle is a binary for one operating system rather
// than code that runs wherever its interpreter does.
//
@@ -599,42 +310,6 @@ func versionOf(digest string) string {
return hex
}
// bundleEnvWords are the words the runtime sets for itself; a bundle that named one would be
// telling the runtime what it is, which is the mesh's to say (novox/hq ADR 0192).
var bundleEnvWords = map[string]bool{
RuntimeToolModules: true, RuntimeBrokerFile: true, RuntimeOperatorAccount: true,
RuntimeOperatorHome: true, RuntimeToolEnv: true, RuntimeAgentAccount: true, RuntimeAgentHome: true,
}
// bundleEnvProblems says what is wrong with what a bundle says it is given (novox/hq ADR 0192):
// a value is a path or a constant written with the references a container's environment may use
// for a place or a port, and never a secret's content or another module's binding — a secret
// reaches a tool as a file whose path is named.
func bundleEnvProblems(module string, a Artifact) []string {
if len(a.Env) == 0 {
return nil
}
var problems []string
for _, word := range sortedKeys(a.Env) {
value := a.Env[word]
if bundleEnvWords[word] {
problems = append(problems, fmt.Sprintf(
"%s: %q gives itself %s, which the node's runtime sets for itself; a bundle is "+
"given its own words beside the runtime's, never in place of them (novox/hq ADR 0192)",
module, a.Name, word))
}
rest := ofPort.ReplaceAllString(dirRef.ReplaceAllString(value, ""), "")
if strings.Contains(rest, "${") {
problems = append(problems, fmt.Sprintf(
"%s: %q gives %s the value %q. A bundle's word is a path or a constant, written with "+
"${dir:…} and ${port:…} only; a secret reaches a tool as a file the mesh places, "+
"named by its path, never as its content (novox/hq ADR 0192)",
module, a.Name, word, value))
}
}
return problems
}
func copyWords(in map[string]string) map[string]string {
if len(in) == 0 {
return nil
@@ -675,31 +350,6 @@ func runByAResource(m Manifest) map[string]bool {
return named
}
// undeliveredBundles says which of a module's bundles nothing would ever put on a machine (novox/hq
// 04-ISSUES/216). A bundle reaches a machine three ways: the node's runtime serves it (it says
// `loads`, or its module declares `tools`), a resource names it (a process, a step, an archive), or
// it is the runtime itself. One reached by none of them was built, recorded and pushed as success,
// and was simply absent — seven modules' tools went missing that way on 2026-10-03. Refused here,
// naming the field that would deliver it.
func undeliveredBundles(m Manifest) []string {
if m.Build == nil || m.Module == RuntimeModule {
return nil
}
named := runByAResource(m)
var problems []string
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle || named[a.Name] || len(a.Loads) > 0 || len(m.Tools) > 0 {
continue
}
problems = append(problems, fmt.Sprintf(
"%s: the bundle %q would be built and never reach a machine: nothing loads it, runs it or "+
"unpacks it. A tools bundle says `loads` (the entrypoints the node's runtime serves) or its "+
"module lists its `tools`; a daemon or a step is a resource naming it (novox/hq 04-ISSUES/216)",
m.Module, a.Name))
}
return problems
}
// BundleLoads is what the runtime loads of the bundle artifact a of m (run: the bundles one of m's own resources runs,
// runByAResource): what the artifact said, else every entrypoint of a module that declares tools, else nothing; a
// bundle compiled to a binary is served by its binary. The rule a build's resolution keeps on Bundle.Loads, and what a
-74
View File
@@ -1,82 +1,8 @@
package catalogue
import (
"strings"
"testing"
)
// bundleFor is a manifest whose one artifact is a bundle in the given language and system.
func bundleFor(language, system string) Manifest {
return Manifest{Module: "a-component", Build: &Build{Artifacts: []Artifact{
{Name: "binary", Kind: ArtifactBundle, Language: language, System: system},
}}}
}
func problemsOf(t *testing.T, m Manifest) string {
t.Helper()
return strings.Join(m.Build.problems(m.Module), "\n")
}
// **A language that compiles to a binary must say which system.**
//
// A binary is pinned to one operating system at link time, so a host refuses to touch a machine it
// was not built for. An artifact that says nothing would be compiled for whatever the build machine
// happened to be — which reads as portable and is not, and is the fault this check exists for.
func TestABinaryMustSayWhichSystemItIsFor(t *testing.T) {
got := problemsOf(t, bundleFor("go", ""))
if !strings.Contains(got, "says no system") {
t.Fatalf("a compiled bundle with no system was accepted:\n%s", got)
}
// And the refusal names what it could have said, so a reader is one edit from right.
for _, system := range []string{"alpine", "android", "arch"} {
if !strings.Contains(got, system) {
t.Fatalf("the refusal does not name %q as a choice:\n%s", system, got)
}
}
}
func TestABinaryThatNamesASystemIsAccepted(t *testing.T) {
if got := problemsOf(t, bundleFor("go", "arch")); got != "" {
t.Fatalf("a compiled bundle naming a system was refused:\n%s", got)
}
}
// A system the mesh does not build for is refused where it is written. These are the host's own
// names, not a compiler's: the difference between two of them is a C library rather than a kernel,
// so a value that looks like an operating system to a toolchain is still wrong here.
func TestASystemTheMeshDoesNotBuildForIsRefused(t *testing.T) {
for _, wrong := range []string{"linux", "debian", "darwin"} {
got := problemsOf(t, bundleFor("go", wrong))
if !strings.Contains(got, "and a system is") {
t.Fatalf("%q was accepted as a system:\n%s", wrong, got)
}
}
}
// **And a language that runs anywhere must not name one.** A system that decides nothing reads as
// though it did, which is the same fault as a restriction that restricts nothing (novox/hq ADR 0045).
func TestAPortableBundleMayNotNameASystem(t *testing.T) {
got := problemsOf(t, bundleFor("typescript", "arch"))
if !strings.Contains(got, "runs anywhere") {
t.Fatalf("a portable bundle was allowed to name a system:\n%s", got)
}
}
func TestAPortableBundleNamingNoSystemIsAccepted(t *testing.T) {
if got := problemsOf(t, bundleFor("typescript", "")); got != "" {
t.Fatalf("an ordinary bundle was refused:\n%s", got)
}
}
// One component, one artifact per system: the shape the mesh's own binaries are declared in, and the
// reason the target is the artifact's rather than the recipe's.
func TestOneArtifactPerSystemIsAccepted(t *testing.T) {
m := Manifest{Module: "the-host", Build: &Build{Artifacts: []Artifact{
{Name: "arch", Kind: ArtifactBundle, Language: "go", System: "arch"},
{Name: "alpine", Kind: ArtifactBundle, Language: "go", System: "alpine"},
{Name: "android", Kind: ArtifactBundle, Language: "go", System: "android"},
}}}
if got := problemsOf(t, m); got != "" {
t.Fatalf("one artifact per system was refused:\n%s", got)
}
}
+1 -82
View File
@@ -108,68 +108,8 @@ func TestAModuleThatBuildsNothingIsOrdinary(t *testing.T) {
}
}
func TestTwoArtifactsWithOneNameAreRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"x","kind":"image","from":"Dockerfile"},
{"name":"x","kind":"archive","from":"files"}]}}`))
if err == nil {
t.Fatal("two artifacts with one name were accepted")
}
if !strings.Contains(err.Error(), "could mean either") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
func TestAnArtifactBuiltFromOutsideItsRepositoryIsRefused(t *testing.T) {
// A build reads its own repository and nothing else. A path leaving it would make what gets
// built depend on whatever happens to be on the machine doing the building.
for _, from := range []string{"/etc/passwd", "../elsewhere"} {
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"x","kind":"archive","from":"` + from + `"}]}}`))
if err == nil {
t.Fatalf("%q was accepted as a build input", from)
}
if !strings.Contains(err.Error(), "outside its own repository") {
t.Fatalf("refused for the wrong reason: %v", err)
}
}
}
func TestAnArtifactOfAnUnknownKindIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"x","kind":"binary","from":"main.go"}]}}`))
if err == nil {
t.Fatal("an artifact of an unknown kind was accepted")
}
}
func TestAPackageParsesLikeABundleAndNeedsALanguage(t *testing.T) {
// The SDK's shape: a package built from the module's own directory, naming a language.
m, err := ParseManifest([]byte(`{"module":"mesh-sdk","version":"1","slug":"sdk",
"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript"}]},
"resources":[]}`))
if err != nil {
t.Fatalf("the SDK's package manifest did not parse: %v", err)
}
if m.Build.Artifacts[0].Kind != ArtifactPackage {
t.Fatalf("expected a package artifact, got %q", m.Build.Artifacts[0].Kind)
}
// A package that names what it is built from is refused, exactly as a bundle is: it is built
// from the module's own directory.
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"lib","kind":"package","language":"typescript","from":"Dockerfile"}]}}`)); err == nil {
t.Fatal("a package naming a source was accepted")
}
// A package with no language cannot choose a toolchain.
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"lib","kind":"package"}]}}`)); err == nil {
t.Fatal("a package with no language was accepted")
}
}
func TestAResourceNamingAPackageIsRefused(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"a","version":"1","slug":"a",
m, err := decodeForTest([]byte(`{"module":"a","version":"1","slug":"a",
"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript"}]},
"resources":[{"id":"svc","type":"container","name":"a","artifact":"lib"}]}`))
if err != nil {
@@ -180,24 +120,3 @@ func TestAResourceNamingAPackageIsRefused(t *testing.T) {
t.Fatal("a resource backed by a package was accepted; a package is not a resource")
}
}
// An image names the Go package its recipe compiles (novox/hq ADR 0267): within the tree it is built in,
// and only an image says one.
func TestOnlyAnImageNamesThePackageItCompilesWithinItsTree(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"x","kind":"image","from":"Dockerfile","compiles":"cmd/x"}]}}`))
if err != nil || m.Build.Artifacts[0].Compiles != "cmd/x" {
t.Fatalf("an image naming its package was refused or lost it: %v", err)
}
for _, c := range []struct{ artifact, says string }{
{`{"name":"x","kind":"archive","from":"files","compiles":"cmd/x"}`, "Only an image"},
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"../x"}`, "leaves the tree"},
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"/x"}`, "leaves the tree"},
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"a/../../x"}`, "leaves the tree"},
} {
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[` + c.artifact + `]}}`))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v, wanted a refusal saying %q", c.artifact, err, c.says)
}
}
}
+1 -50
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"encoding/json"
"os"
"path/filepath"
"strings"
@@ -10,54 +9,6 @@ import (
"git.novox.be/novox/mesh-sdk/go/internal/beside"
)
// The module holding mesh-broker is the bus, and its account is granted the bus's snapshot API and
// nothing else (novox/hq ADR 0235). Anything it declared to say or hear on the bus would be granted
// nothing, so it is refused at the parser rather than silently dropped.
func TestTheBussAccountSaysNothingOnTheBus(t *testing.T) {
raw := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
"emits":["something.happened"]}`)
_, err := ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), "granted the bus's snapshot API and nothing else") {
t.Fatalf("a bus module declaring what it emits was not refused, or not for the reason: %v", err)
}
// Its tools bundle says what it emits (novox/hq ADR 0357): the node's runtime carries the bundle and is
// granted the module's events, so a bus module that ships one may declare them.
bundled := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
"emits":["bus.slow-consumer"],"tools":["bus_streams"],
"build":{"artifacts":[{"name":"tools","kind":"bundle","language":"go","system":"arch","from":"cmd/t","binary":"t"}]}}`)
m, err := ParseManifest(bundled)
if err != nil {
t.Fatalf("a bus module whose tools bundle declares what it emits was refused: %v", err)
}
// And so does its built manifest, its bundle in Bundles and its build read away (issue 774: nats's built manifest
// was refused, and with it every plan of the control-node).
built, err := m.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle, Reference: "store/tools:1", Digest: "sha256:1"}})
if err != nil {
t.Fatal(err)
}
raw, err = json.Marshal(built)
if err != nil {
t.Fatal(err)
}
if _, err := ParseManifestForInputs(raw); err != nil {
t.Fatalf("the built manifest of a bus module whose tools bundle declares what it emits was refused: %v", err)
}
quiet := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
"tools":["bus_streams"]}`)
m, err = ParseManifest(quiet)
if err != nil {
t.Fatalf("a bus module with an account and tools was refused: %v", err)
}
if !m.ClaimsSeat(BrokerSeat) {
t.Fatal("it does not read as holding the broker seat")
}
}
// The catalogue's bus protects its streams by the snapshot, not as live files: its streams' item is a
// dump into its snapshots, it has the account the dump runs as, and the dump runs the snapshot
// program in the bus's own container.
@@ -66,7 +17,7 @@ func TestTheCataloguesBusIsBackedUpBySnapshot(t *testing.T) {
if err != nil {
t.Fatal(err)
}
m, err := ParseManifest(raw)
m, err := decodeForTest(raw)
if err != nil {
t.Fatal(err)
}
-28
View File
@@ -80,34 +80,6 @@ func ReadBusy(r map[string]any) (Busy, bool, []string) {
return b, true, problems
}
// busyProblems is everything wrong with a manifest's `busy` fields, in the manifest's words: a command on
// one line, a timeout within its bounds, and only on a container that stays up.
func busyProblems(m Manifest) []string {
var problems []string
for _, r := range m.Resources {
b, has, read := ReadBusy(r)
if !has {
continue
}
where := m.Module + ": " + fmt.Sprint(r["id"])
for _, p := range read {
problems = append(problems, m.Module+": "+p)
}
if fmt.Sprint(r["type"]) != "container" || !LongRunning(r) {
problems = append(problems, fmt.Sprintf("%s declares busy and is not a container that stays up: busy is "+
"asked of a running container before it is replaced (ADR 0335)", where))
}
if b.Command == "" {
problems = append(problems, where+"'s busy names no command: one run inside the container, exiting 0 "+
"while it is busy")
}
if strings.ContainsAny(b.Command, "\n\r") {
problems = append(problems, where+"'s busy command is one line")
}
}
return problems
}
// busyInto composes a container's `busy` into the node-engine's words, or takes it away: for an engine that
// reads it, the command and its timeout written out; for one that does not, the field is not sent, and the
// container is replaced at once, as before.
+1 -29
View File
@@ -2,7 +2,6 @@ package catalogue
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/validate"
@@ -32,35 +31,8 @@ func busyManifest(t *testing.T, on string, busy any) []byte {
return out
}
func TestABusyIsHeldToItsShape(t *testing.T) {
if _, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "mesh-builder busy"})); err != nil {
t.Fatalf("a well-formed busy was refused: %v", err)
}
if _, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "x", "timeout": "30s"})); err != nil {
t.Fatalf("a busy with its timeout was refused: %v", err)
}
for name, c := range map[string]struct {
on string
busy any
says string
}{
"text": {"agent", "mesh-builder busy", "it is an object"},
"no command": {"agent", map[string]any{"timeout": "5s"}, "names no command"},
"two lines": {"agent", map[string]any{"command": "a\nb"}, "one line"},
"a long timeout": {"agent", map[string]any{"command": "x", "timeout": "5m"}, "at most 1m0s"},
"an unknown key": {"agent", map[string]any{"command": "x", "every": "1m"}, `says "every"`},
"on a step": {"seed", map[string]any{"command": "x"}, "not a container that stays up"},
"on a service": {"daemon", map[string]any{"command": "x"}, "not a container that stays up"},
} {
_, err := ParseManifest(busyManifest(t, c.on, c.busy))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: want a refusal saying %q, got %v", name, c.says, err)
}
}
}
func TestABusyIsSentOnlyToAnEngineThatReadsIt(t *testing.T) {
m, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "mesh-builder busy"}))
m, err := decodeForTest(busyManifest(t, "agent", map[string]any{"command": "mesh-builder busy"}))
if err != nil {
t.Fatal(err)
}
-76
View File
@@ -2,7 +2,6 @@ package catalogue
import (
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/internal/beside"
@@ -45,78 +44,3 @@ func TestEveryCatalogueManifestParses(t *testing.T) {
t.Fatal("no endpoint in the catalogue is named, so this proved nothing")
}
}
// TestNoCatalogueManifestNamesAnInstallation is ADR 0112's check, run over the real catalogue: no
// definition names a domain or a public address the mesh acts on, and every value that must for now
// carries its reason (novox/hq ADR 0155, issue 134). The list it prints is the one that shrinks.
func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
_, found := catalogueManifests(t)
var named []string
for _, r := range found {
m, err := r.manifest, r.err
if err != nil {
t.Errorf("%s: %v", r.path, err)
continue
}
named = append(named, InstallationProblems(m)...)
}
if len(named) > 0 {
t.Fatalf("%d value(s) name an installation:\n %s", len(named), strings.Join(named, "\n "))
}
}
// TestEveryCatalogueModuleDeclaresItsData is ADR 0233's check over the real catalogue (it replaced ADR
// 0214's store list): every provider that grants says what it keeps for its consumers, nothing writes a
// backup line by hand, every directory a container writes is declared, and every irreplaceable item is
// backed up — so a store added is a store backed up, without a list of stores to keep in step.
func TestEveryCatalogueModuleDeclaresItsData(t *testing.T) {
_, found := catalogueManifests(t)
shelf := Shelf{}
granting := 0
for _, r := range found {
m, err := r.manifest, r.err
if err != nil {
continue // TestEveryCatalogueManifestParses says why
}
if len(m.Grants) > 0 {
granting++
}
shelf[m.Module] = m
}
for _, problem := range DataProblems(shelf) {
t.Error(problem)
}
if granting == 0 {
t.Fatal("no module in the catalogue grants a provision, so this proved nothing")
}
}
// TestEveryCatalogueIdentityFitsWhatItRequires is ADR 0225's check over the real catalogue: every
// module's identity, on the longest machine name, fits the bound of every provision it wants. An
// overflow fails here, in the pull request that introduces it, rather than on the provider's machine
// the first time a real machine's name meets the module's (issue 263).
func TestEveryCatalogueIdentityFitsWhatItRequires(t *testing.T) {
_, found := catalogueManifests(t)
shelf := Shelf{}
for _, r := range found {
m, err := r.manifest, r.err
if err != nil {
t.Fatalf("%s: %v", r.path, err)
}
shelf[m.Module] = m
}
for _, p := range IdentityProblems(shelf, DefaultLongestMachine) {
t.Error(p)
}
// The night it was found: the resolver provision bounds nothing, and the object store still 20.
if dns, ok := shelf["dnsmasq"]; ok {
if b := dns.IdentityBoundOf("wildcard-resolution"); b.Bounded() {
t.Errorf("the resolver provision bounds its consumers' identities: %+v", b)
}
}
if store, ok := shelf["minio"]; ok {
if b := store.IdentityBoundOf("s3-bucket"); b.Max != 20 {
t.Errorf("the object store's access key is not bounded at 20: %+v", b)
}
}
}
+1 -210
View File
@@ -113,12 +113,7 @@ var oldCollectionNamespaces = []string{"shell", "environment", "contribution"}
var (
// ofCollected is where a collector places a collection. Loose inside the braces, so a misspelt name is refused.
ofCollected = regexp.MustCompile(`\$\{collected:([^}]*)\}`)
// collectionName is words joined by colons, so an old namespace's placeholder names its collection as it reads.
collectionName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*(:[a-z0-9][a-z0-9-]*)*$`)
// itemName, attributeKey and attributeValue are the words an item's name and attributes are made of; none holds
// a colon, a comma or an equals sign, which tell a selector from a collection's name.
itemName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
ofCollected = regexp.MustCompile(`\$\{collected:([^}]*)\}`)
attributeKey = regexp.MustCompile(`^[a-z][a-z0-9-]*$`)
attributeValue = regexp.MustCompile(`^[a-z0-9][a-z0-9_.-]*$`)
)
@@ -242,190 +237,6 @@ func placerOfOld(name string) string {
return canonicalSeat(seat)
}
// collectionProblems is what is wrong with a module's collections and items, from the manifest alone.
func collectionProblems(m Manifest) []string {
var problems []string
declared := map[string]bool{}
for i, c := range m.Collections {
switch {
case !collectionName.MatchString(c.Name):
problems = append(problems, fmt.Sprintf("%s's collection %d is %q, which is not a collection's name: "+
"words of lower-case letters, digits and hyphens, joined by colons", m.Module, i+1, c.Name))
case declared[c.Name]:
problems = append(problems, fmt.Sprintf("%s collects %s twice", m.Module, c.Name))
}
if _, old, _ := oldCollection(c.Name, false); old {
problems = append(problems, fmt.Sprintf("%s collects %s, which is read from its placeholder: a module "+
"places it as ${%s} where it holds the seat that places it, and declares nothing", m.Module, c.Name,
c.Name))
}
if strings.ContainsAny(c.Comment, "\n\r") {
problems = append(problems, fmt.Sprintf("%s's collection %s has a comment holding a line break",
m.Module, c.Name))
}
declared[c.Name] = true
}
// A collection declared and placed nowhere is a word read by nothing; judged where every file's text is here.
if !m.Templated() {
named := map[string]bool{}
for _, r := range m.Resources {
if text, ok := r["content"].(string); ok {
for _, f := range ofCollected.FindAllStringSubmatch(text, -1) {
base, _ := splitSelector(f[1])
named[base] = true
}
}
}
for _, c := range m.Collections {
if !named[c.Name] && collectionName.MatchString(c.Name) {
problems = append(problems, fmt.Sprintf("%s collects %s and no file of its places it as "+
"${collected:%s}", m.Module, c.Name, c.Name))
}
}
}
contributed := map[string]bool{}
for i, it := range m.Items {
where := fmt.Sprintf("%s's item %d", m.Module, i+1)
if !collectionName.MatchString(it.Collection) {
problems = append(problems, fmt.Sprintf("%s is to %q, which is not a collection's name", where,
it.Collection))
} else if _, old, wrong := oldCollection(it.Collection, true); old && wrong != nil {
problems = append(problems, fmt.Sprintf("%s: %v", where, wrong))
}
switch {
case it.Name != "" && !itemName.MatchString(it.Name):
problems = append(problems, fmt.Sprintf("%s: the name %q is not a name: lower-case letters, digits and "+
"hyphens", where, it.Name))
case contributed[it.key()]:
problems = append(problems, fmt.Sprintf("%s contributes to %s twice; a module contributes one item to "+
"a collection, or several each with a name of its own", m.Module, it.key()))
case it.Name == "" && slices.ContainsFunc(m.Items, func(o Item) bool {
return o.Collection == it.Collection && o.Name != ""
}):
problems = append(problems, fmt.Sprintf("%s contributes several items to %s and %s has no name; each "+
"of several items has a name of its own", m.Module, it.Collection, where))
}
contributed[it.key()] = true
problems = append(problems, itemAttributeProblems(m.Module, where, it)...)
if it.Collection == EnvironmentCollection {
problems = append(problems, environmentItemProblems(where, it)...)
}
switch {
case it.Template == "" && it.Content == "":
problems = append(problems, fmt.Sprintf("%s names no template: an item is a template of the module's "+
"own, \"template\": \"%s/<name>\" (novox/hq ADR 0349 §13)", where, m.TemplatesDir()))
case it.Template != "" && it.Content != "":
problems = append(problems, fmt.Sprintf("%s names its template and says its content too", where))
case it.Template != "":
if wrong := itemTemplateProblem(m, it.Template); wrong != "" {
problems = append(problems, fmt.Sprintf("%s: %s", where, wrong))
}
// **Filled by the provisioner's client alone**: the controller renders no item.
if !m.Moved {
problems = append(problems, fmt.Sprintf("%s: an item is filled by the provisioner's client of the "+
"node its contributor is placed on, so a module contributing one is marked moved "+
"(\"moved\": true; novox/hq ADR 0349 §13, §17)", where))
}
}
if it.Content != "" {
problems = append(problems, itemTextProblems(m.Module, it)...)
}
}
return problems
}
// itemAttributeProblems is what is wrong with an item's attributes and capability. The attributes of an item of a kind
// received as data are the fields its shape places by (and `order`), each with a value the shape takes.
func itemAttributeProblems(module, where string, it Item) []string {
var problems []string
for _, k := range sortedKeys(it.Attributes) {
v := it.Attributes[k]
switch {
case !attributeKey.MatchString(k):
problems = append(problems, fmt.Sprintf("%s: the attribute %q is not a name: lower-case letters, digits "+
"and hyphens", where, k))
continue
case !attributeFits(it.Collection, k, v):
problems = append(problems, fmt.Sprintf("%s: %s=%q is not a value: lower-case letters, digits, hyphens, "+
"dots and underscores (an environment variable's name, letters, digits and underscores)", where, k, v))
continue
}
if n, err := strconv.Atoi(v); k == OrderAttribute && (err != nil || n < 0 || n > 99) {
problems = append(problems, fmt.Sprintf("%s: order=%q is not a whole number from 0 to 99", where, v))
}
}
if namespace, key, _ := strings.Cut(it.Collection, ":"); namespace == "contribution" {
seat, kind, _ := strings.Cut(key, ":")
if _, r, ok := receivable(seat, kind); ok && r.Shape != nil {
for _, k := range sortedKeys(it.Attributes) {
var place *PlaceField
for i := range r.Shape.Places {
if r.Shape.Places[i].Field == k {
place = &r.Shape.Places[i]
}
}
switch {
case k == OrderAttribute:
case place == nil:
problems = append(problems, fmt.Sprintf("%s: %s is placed by %s, and %q is none of them", where,
it.Collection, placeFields(r), k))
case !oneOf(place.Values, it.Attributes[k]):
problems = append(problems, fmt.Sprintf("%s: %s=%s; %s is one of %s", where, k, it.Attributes[k], k,
strings.Join(place.Values, ", ")))
}
}
}
}
switch {
case it.IfCapability == "":
case !capabilityName.MatchString(it.IfCapability):
problems = append(problems, fmt.Sprintf("%s: if-capability %q is not a capability's name", where, it.IfCapability))
case !oneOf(KnownCapabilities, it.IfCapability):
problems = append(problems, fmt.Sprintf("%s: if-capability %q is one no machine reports; the node-engine "+
"detects %s", where, it.IfCapability, strings.Join(KnownCapabilities, ", ")))
}
return problems
}
// AttributeFits is whether an item's attribute is of the shape its collection allows: words of lower case, except
// the environment's `name`, which is a variable's name (design 57 L3, issue 795). The same rule is mesh-catalog's
// provisioner client's, and the inventory's.
func attributeFits(collection, k, v string) bool {
if collection == EnvironmentCollection && k == "name" {
return variableName.MatchString(v)
}
return attributeKey.MatchString(k) && attributeValue.MatchString(v)
}
// environmentItemProblems is what is wrong with an item of the environment collection: typed by `kind`, a variable
// with its `name`, a PATH entry `at` the start or the end, and no attribute besides `order`.
func environmentItemProblems(where string, it Item) []string {
var problems []string
for _, k := range sortedKeys(it.Attributes) {
if !slices.Contains([]string{"kind", "name", "at", OrderAttribute}, k) {
problems = append(problems, fmt.Sprintf("%s: %s is typed by kind, name and at, and %q is none of them",
where, it.Collection, k))
}
}
switch kind := it.Attributes["kind"]; kind {
case "variable":
if it.Attributes["name"] == "" || it.Attributes["at"] != "" {
problems = append(problems, fmt.Sprintf("%s: a variable names itself with name=<NAME> and has no at", where))
} else if it.Attributes["name"] == "PATH" {
problems = append(problems, fmt.Sprintf("%s: PATH is not a variable; a PATH entry is kind=path", where))
}
case "path":
if it.Attributes["name"] != "" || (it.Attributes["at"] != PathAtStart && it.Attributes["at"] != PathAtEnd) {
problems = append(problems, fmt.Sprintf("%s: a PATH entry is at=%s or at=%s and has no name", where,
PathAtStart, PathAtEnd))
}
default:
problems = append(problems, fmt.Sprintf("%s: an item of %s is kind=variable or kind=path, not %q", where,
it.Collection, kind))
}
return problems
}
// itemTemplateProblem is what is wrong with an item's template path, or "".
func itemTemplateProblem(m Manifest, name string) string {
dir := m.TemplatesDir()
@@ -437,26 +248,6 @@ func itemTemplateProblem(m Manifest, name string) string {
return ""
}
// itemTextProblems is what an item's text may not name: a secret or a bound value, since an item is kept in a
// bucket and no secret is (ADR 0349 clause 7), and a collection, since an item is not a collector. And every
// placeholder no pass fills, as in a file's content.
func itemTextProblems(module string, it Item) []string {
var problems []string
for _, f := range templatePlaceholder.FindAllStringSubmatch(it.Content, -1) {
namespace, key := f[1], f[2]
switch _, collection := collectionOfPlaceholder(namespace, key); {
case namespace == "secret" || namespace == "bound":
problems = append(problems, fmt.Sprintf("%s's item of %s names %s: an item is kept in a bucket, where "+
"no secret is (novox/hq ADR 0349 §7)", module, it.Collection, f[0]))
case collection:
problems = append(problems, fmt.Sprintf("%s's item of %s names %s: an item is not a collector",
module, it.Collection, f[0]))
}
}
return append(problems, unconsumedPlaceholders(module, map[string]any{"id": "item:" + it.key(),
"type": "file", "content": it.Content})...)
}
// isOld is whether a collection's name is an old namespace's.
func isOld(name string) bool {
_, old, _ := oldCollection(name, true)
+1 -28
View File
@@ -69,7 +69,7 @@ func TestASelectorIsRefusedWhenMalformedOrNamingACollectionNotCollected(t *testi
raw := strings.Replace(barCollector, `"template":"files/bars"`, `"content":"`+placeholder+`"`, 1)
raw = strings.Replace(raw, `"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},`, "", 1)
raw = strings.Replace(raw, `"moved":true,`, "", 1)
m, err := ParseManifest([]byte(raw))
m, err := decodeForTest([]byte(raw))
if err == nil {
_, err = (Resolution{Node: "n", Modules: []Manifest{m}}).Declaration(Rendering{})
}
@@ -82,33 +82,6 @@ func TestASelectorIsRefusedWhenMalformedOrNamingACollectionNotCollected(t *testi
}
}
func TestItemNamesAttributesAndCapabilitiesAreJudged(t *testing.T) {
item := func(extra string) string {
return `{"module":"p","moved":true,"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},` +
`"items":[` + extra + `]}`
}
for items, want := range map[string]string{
`{"collection":"c","name":"a","template":"files/a"},{"collection":"c","template":"files/b"}`: "has no name",
`{"collection":"c","name":"a","template":"files/a"},{"collection":"c","name":"a","template":"files/b"}`: "twice",
`{"collection":"c","name":"Big","template":"files/a"}`: "is not a name",
`{"collection":"c","attributes":{"order":"100"},"template":"files/a"}`: "0 to 99",
`{"collection":"c","attributes":{"bar":"a b"},"template":"files/a"}`: "is not a value",
`{"collection":"c","if-capability":"batery","template":"files/a"}`: "no machine reports",
`{"collection":"contribution:node-bar:block","attributes":{"bar":"left"},"template":"files/a"}`: "bar=left",
`{"collection":"contribution:node-bar:block","attributes":{"colour":"red"},"template":"files/a"}`: `"colour" is none of them`,
`{"collection":"c","name":"a","template":"files/a","colour":"red"}`: "colour",
} {
_, err := ParseManifest([]byte(item(items)))
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s: %v, want %q", items, err, want)
}
}
ok := item(`{"collection":"contribution:node-bar:block","name":"a","attributes":{"bar":"top","place":"status","order":"5"},"if-capability":"battery","template":"files/a"}`)
if _, err := ParseManifest([]byte(ok)); err != nil {
t.Fatal(err)
}
}
// A moved contributor's items, selected by the old placeholder's pair, render in the old order (order, module, item
// name) and with the same lines as the contributions they replace.
func TestASelectedMovedItemRendersIntoTheOldBarPlaceholderAsTheContributionDid(t *testing.T) {
+1 -37
View File
@@ -37,7 +37,7 @@ func collectionsOn(t *testing.T) Resolution {
t.Helper()
zsh := held(t, collectorZsh, builtWith("zsh", map[string]string{"zshrc": zshrcText}))
prompt := held(t, contributorP10k, builtWith("prompt", map[string]string{"zsh": contributorText}))
aliases, err := ParseManifest([]byte(oldFormAliases))
aliases, err := decodeForTest([]byte(oldFormAliases))
if err != nil {
t.Fatal(err)
}
@@ -141,39 +141,3 @@ func TestAMovedContributorsItemRendersAsItsOldFormDid(t *testing.T) {
}
}
}
func TestCollectionsAndItemsAreJudged(t *testing.T) {
for name, c := range map[string]struct{ raw, says string }{
"an item of a module not moved": {`{"module":"m","version":"1",
"items":[{"collection":"shell:zsh:normal","template":"files/x"}],
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is marked moved"},
"two items to one collection": {`{"module":"m","version":"1","moved":true,
"items":[{"collection":"a","template":"files/x"},{"collection":"a","template":"files/y"}],
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "contributes to a twice"},
"an item to no slot": {`{"module":"m","version":"1","moved":true,
"items":[{"collection":"shell:zsh:middle","template":"files/x"}],
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is no slot"},
"an item outside the templates": {`{"module":"m","version":"1","moved":true,
"items":[{"collection":"a","template":"x"}],
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is not a file under files/"},
"an item naming a secret": {`{"module":"m","version":"1",
"items":[{"collection":"a","content":"t=${secret:token}"}]}`, "where no secret is"},
"a collection not declared": {`{"module":"m","version":"1",
"resources":[{"id":"f","type":"file","path":"/f","content":"${collected:a}"}]}`, "does not collect"},
"an old namespace declared": {`{"module":"m","version":"1","collections":[{"name":"shell:zsh:normal"}],
"resources":[{"id":"f","type":"file","path":"/f","content":"x"}]}`, "read from its placeholder"},
"a collection placed nowhere": {`{"module":"m","version":"1","collections":[{"name":"a"}],
"resources":[{"id":"f","type":"file","path":"/f","content":"x"}]}`, "no file of its places it"},
"a collection in a path": {`{"module":"m","version":"1","collections":[{"name":"a"}],
"resources":[{"id":"f","type":"file","path":"/${collected:a}","content":"${collected:a}"}]}`,
"only in a file's content"},
} {
_, err := ParseManifest([]byte(c.raw))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v, want it to say %q", name, err, c.says)
}
}
if _, err := ParseManifest([]byte(collectorZsh)); err != nil {
t.Errorf("a collector: %v", err)
}
}
-13
View File
@@ -104,19 +104,6 @@ func TestAGeneratorThisControlPlaneDoesNotHaveIsRefused(t *testing.T) {
}
}
func TestAModuleIsEitherWrittenOrComputedNotBoth(t *testing.T) {
// Otherwise nobody could say where a given file on a machine came from.
_, err := ParseManifest([]byte(`{"module":"mesh-network","version":"1",
"computed":"mesh-network",
"resources":[{"id":"a","type":"package","package":"wireguard-tools"}]}`))
if err == nil {
t.Fatal("a module that both ships files and has them computed was accepted")
}
if !strings.Contains(err.Error(), "one or the other") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
func TestSettingsApplyToAComputedModuleToo(t *testing.T) {
// Being computed is about where the files come from, not about whether they are configurable.
// A line drawn there would be arbitrary and nobody could predict it.
-266
View File
@@ -5,7 +5,6 @@ import (
"fmt"
"regexp"
"slices"
"strings"
)
// A module declares its config (novox/hq ADR 0348 §1, §4, §5, §6, §8, §11; design 57 package K1, issue 660).
@@ -123,26 +122,6 @@ func toConfigForm(raw []byte) ([]byte, error) {
return raw, nil
}
// inConfigWords is a refusal said of a module in the config form in its words: `${config:<key>}` and
// takes-config, where the rules shared with the settings form say `${setting:<key>}` and takes-settings.
func inConfigWords(problem string) string {
problem = settingRef.ReplaceAllStringFunc(problem, func(ref string) string {
return "${config:" + settingRef.FindStringSubmatch(ref)[1] + "}"
})
return strings.ReplaceAll(problem, TakesSettingsField, TakesConfigField)
}
// MixedFormProblems is a manifest using both forms, said with the words of each (ADR 0348 §1).
func MixedFormProblems(m Manifest) []string {
if len(m.forms.config) == 0 || len(m.forms.settings) == 0 {
return nil
}
return []string{fmt.Sprintf("%s mixes the config form (%s) with the settings form (%s): a module uses one form "+
"or the other, never both; write %s, %s and ${config:<key>} throughout (novox/hq ADR 0348 §1)",
m.Module, strings.Join(m.forms.config, ", "), strings.Join(m.forms.settings, ", "),
ConfigField, TakesConfigField)}
}
// configDeclaration is the config form's declaration as written: nothing of the settings form's, and nothing
// unsaid.
type configDeclaration struct {
@@ -171,166 +150,6 @@ func (d SettingDeclaration) MarshalJSON() ([]byte, error) {
Default: d.Default, Required: d.Required, Runtime: d.Runtime})
}
// configShaped is whether a declaration says anything of the config form's.
func configShaped(d SettingDeclaration) bool {
return d.Type != "" || d.Description != "" || len(d.Enum) > 0 || d.Minimum != nil || d.Maximum != nil ||
d.Pattern != "" || d.Trusted || d.Predefined != "" || d.Required || d.Runtime
}
// TypedSettingProblems is a declaration under `settings` that says what only `config` says: a value with a type,
// a description and its validation is declared in the config form (ADR 0348 §1, §4).
func TypedSettingProblems(m Manifest) []string {
var problems []string
for _, key := range sortedSettingKeys(m.Settings) {
if configShaped(m.Settings[key]) {
problems = append(problems, fmt.Sprintf("%s's setting %q says its type, description or validation "+
"under %s: a value declared with those is declared under config, with ${config:<key>} and "+
"takes-config, never under %s (novox/hq ADR 0348 §1, §4)", m.Module, key, SettingsField, SettingsField))
}
}
return problems
}
// ConfigProblems is every way a module in the config form declares its config wrongly, each said by name with
// what to do (ADR 0348 How it is checked, design 57 package K1).
func ConfigProblems(m Manifest) []string {
if !m.DeclaresConfig() {
return nil
}
var problems []string
placed := configPlaced(m)
fallsBack := fileFallsBack(m)
renamed := renamedTaken(m)
for _, key := range sortedSettingKeys(m.Settings) {
d := m.Settings[key]
say := func(format string, args ...any) {
problems = append(problems, fmt.Sprintf("%s's config value %q: ", m.Module, key)+fmt.Sprintf(format, args...))
}
if !settingRef.MatchString("${setting:" + key + "}") {
say("not a usable key: lower-case letters, digits, dots, dashes and underscores")
continue
}
if meshWords[key] || accountFacts[key] {
say("%q is placement, which the mesh decides, never a config value (novox/hq ADR 0348 §11)", key)
continue
}
if d.Kind != "" || d.Why != "" {
say("kind and why are the settings form's; under config a value says its type and its description " +
"(novox/hq ADR 0348 §1, §4)")
}
switch d.Type {
case ConfigNumber, ConfigString, ConfigBoolean:
case ConfigObject, ConfigList:
if !d.Trusted {
say("is %s and not trusted: an object or a list decides more than one value can say, so it is "+
"trusted, \"trusted\": true, or split into numbers, strings and booleans (novox/hq ADR 0348 §4)", d.Type)
}
case "":
say("no type: say \"type\": number, string or boolean, or object or list when trusted (novox/hq ADR 0348 §4)")
default:
say("type %q is none of number, string, boolean, object and list (novox/hq ADR 0348 §4)", d.Type)
}
if strings.TrimSpace(d.Description) == "" {
say("no description: say in a sentence what the value is, including its unit (novox/hq ADR 0348 §4)")
}
switch {
case d.Required && d.Default != nil:
say("both a default and required: a required value is the operator's own and has no default; give " +
"one or the other (novox/hq ADR 0348 §4)")
case !d.Required && d.Default == nil:
say("neither a default nor required: give a default, or say \"required\": true for a value that is " +
"the operator's own (novox/hq ADR 0348 §4)")
}
if as := renamed[key]; as != "" && d.Required {
say("is laid into its file as %q, and only when it is set, so the file and its code fall back when "+
"it is not: it is never required; give it the file's default, the empty value of its type "+
"(novox/hq issue 751)", as)
}
for _, wrong := range validationProblems(d) {
say("%s", wrong)
}
if word := operatorsWord(key); word != "" && d.Default != nil && !fallsBack[key] {
say("a key naming %q is the operator's value: declare it \"required\": true, with no default, "+
"unless a mergeable file takes it and no placeholder asks for it, where the file falls back "+
"(novox/hq ADR 0348 §4, ADR 0112, issue 735)", word)
}
if d.Predefined != "" {
p, listed := m.predefined[d.Predefined]
switch {
case m.predefined == nil:
say("falls back to the predefined key %q, and the list of predefined keys (%s at the root of "+
"the catalogue's repository) is not present where it is judged: a value falls back only to a "+
"key on that list; judge it in the catalogue, or leave \"predefined\" out (novox/hq ADR 0348 §7)",
d.Predefined, PredefinedFile)
case !listed:
say("falls back to %q, which is not on the list of predefined keys: name a key on it, or leave "+
"\"predefined\" out (novox/hq ADR 0348 §7)", d.Predefined)
default:
if p.Type != d.Type {
say("falls back to the predefined key %q, which is a %s, and this value is a %s: a value falls "+
"back only to a key of its own type (novox/hq ADR 0348 §4)", d.Predefined, p.Type, d.Type)
}
if p.Trusted && !d.Trusted {
say("falls back to the trusted predefined key %q and is not trusted itself: a value that falls "+
"back to a trusted key is trusted, \"trusted\": true (novox/hq ADR 0348 §8)", d.Predefined)
}
}
}
// A template's text is not here yet. Resolve fills it from the build, WithTemplatesFrom from the tree; neither
// judges again itself, but WithTemplatesFrom parses its result again, and a resolved build is parsed again
// where it is registered, and this is judged then.
if !placed[key] && !d.Runtime && !m.Templated() {
say("placed nowhere: no file says ${config:%s} or takes it under %s, and it is not marked "+
"\"runtime\": true for a value the module's code reads itself (novox/hq ADR 0348 §5)", key, TakesConfigField)
}
}
used := settingKeysUsedBy(m)
for _, key := range sortedKeys(used) {
if _, declared := m.Settings[key]; !declared {
problems = append(problems, fmt.Sprintf("%s says ${config:%s} and declares no such value under %s: "+
"declare %q with its type, description and a default or required (novox/hq ADR 0348 §4, §5)",
m.Module, key, ConfigField, key))
}
}
problems = append(problems, configTakesProblems(m)...)
problems = append(problems, reflectsProblems(m)...)
return problems
}
// configPlaced is every value a file, a unit, a directory's owner, a contribution or a served fact asks for, and
// every value a mergeable file takes.
func configPlaced(m Manifest) map[string]bool {
placed := settingKeysUsedBy(m)
for _, r := range m.Resources {
for _, k := range takenKeyList(r) {
placed[k] = true
}
}
return placed
}
// fileFallsBack is every value a mergeable file takes and no placeholder asks for. Unset, such a value is simply
// not laid over the file: the file's own content, or the module's code reading the file, falls back. So one named
// after the operator's value (a url, a user, an address) is not `required`, and its default is the file's: its
// value in the content, else the empty value of its type, never laid over the file (the operator's decision on
// issue 664, issue 735). A placeholder has no such fallback: its default would be filled in where the operator's
// value belongs (ADR 0112).
func fileFallsBack(m Manifest) map[string]bool {
asked := settingKeysUsedBy(m)
out := map[string]bool{}
for _, r := range m.Resources {
if how, _ := r["merge"].(string); how == "" {
continue
}
for _, k := range takenKeyList(r) {
if !asked[k] {
out[k] = true
}
}
}
return out
}
// renamedTaken is every value a mergeable file takes under a name of its own, with that name (issue 751).
func renamedTaken(m Manifest) map[string]string {
out := map[string]string{}
@@ -417,88 +236,3 @@ func ofType(v any, kind string) bool {
}
return false
}
// configTakesProblems is what ADR 0348 §11 adds for a module in the config form: an account fact in
// takes-config (the mesh's own words are refused for both forms by TakenSettingsProblems), and a mergeable file
// naming no keys where none of the module's does (where one does, TakenSettingsProblems names every one that
// does not).
func configTakesProblems(m Manifest) []string {
var problems, silent []string
naming := false
for _, r := range m.Resources {
if how, _ := r["merge"].(string); how == "" {
continue
}
if _, said := r[TakesSettingsField]; !said {
silent = append(silent, fmt.Sprint(r["id"]))
continue
}
naming = true
for _, k := range takenKeyList(r) {
_, declared := m.Settings[k]
switch {
case accountFacts[k]:
problems = append(problems, fmt.Sprintf("%s's %v names %q in %s, and %q is an account fact the "+
"mesh places, never a config value (novox/hq ADR 0348 §11)", m.Module, r["id"], k,
TakesConfigField, k))
case meshWords[k]:
// Refused for both forms by TakenSettingsProblems, in its words.
case !declared:
problems = append(problems, fmt.Sprintf("%s's %v names %q in %s and declares no such value under "+
"%s: a value a file takes is declared like every other, with its type, description and a "+
"default or required, or the config verb could never set it (novox/hq ADR 0348 §4, §11)",
m.Module, r["id"], k, TakesConfigField, ConfigField))
}
}
}
if !naming {
for _, id := range silent {
problems = append(problems, fmt.Sprintf("%s's mergeable %s names no %s: a module that declares config "+
"names the keys each mergeable file takes, an empty list for none (novox/hq ADR 0348 §11)",
m.Module, id, TakesConfigField))
}
}
return problems
}
// meshMadeIDs are the resources the mesh composes into a module's own set, which its `restart-on` may name
// without declaring them; meshMadePrefixes are those named after what they are for.
var (
meshMadeIDs = []string{ComposedJailsID(), FilteringID(), CertificateID(), AuthorityID(), BusUsersID()}
meshMadePrefixes = []string{NeedID(""), SecretID(""), strings.TrimSuffix(GrantID("", ""), "-"), BoundID(""), ReceivedID(""), AccessID("/")}
)
// reflectsProblems is a `reload-on` or `restart-on` naming a file the module does not declare (ADR 0348 §6): not
// one of its own resources, not one the mesh composes for it, and not another module's, which is named
// `<module>.<id>`.
func reflectsProblems(m Manifest) []string {
own := map[string]bool{}
for _, r := range m.Resources {
own[fmt.Sprint(r["id"])] = true
}
meshMade := func(id string) bool {
if slices.Contains(meshMadeIDs, id) {
return true
}
for _, p := range meshMadePrefixes {
if strings.HasPrefix(id, p) && len(id) > len(p) {
return true
}
}
return false
}
var problems []string
for _, r := range m.Resources {
for _, field := range []string{"reload-on", "restart-on"} {
for _, id := range stringsOf(r[field]) {
if strings.Contains(id, ".") || own[id] || meshMade(id) {
continue
}
problems = append(problems, fmt.Sprintf("%s's %v names %q in its %s, and %s declares no such "+
"file: name a file of its own, or another module's as <module>.<id> (novox/hq ADR 0348 §6)",
m.Module, r["id"], id, field, m.Module))
}
}
}
return problems
}
+1 -37
View File
@@ -143,43 +143,7 @@ func TestARenamedEntryIsWrittenBackAsItWasSaid(t *testing.T) {
if !strings.Contains(string(raw), `"takes-config":["position","theme",{"as":"NOTIFIER_URL","key":"url"}]`) {
t.Fatalf("written back as %s", raw)
}
if _, err := ParseManifest(raw); err != nil {
if _, err := decodeForTest(raw); err != nil {
t.Fatalf("read again: %v", err)
}
}
func TestARenamedEntryIsRefusedWhereItCouldNotHold(t *testing.T) {
entry := func(m map[string]any, e any) map[string]any {
resourceOf(m, "rules")["takes-config"] = []any{"position", "theme", e}
return m
}
refused(t, entry(renaming(), map[string]any{"key": "url"}), "notifier's rules", `"key" and "as"`)
refused(t, entry(renaming(), map[string]any{"key": "url", "as": "NOTIFIER_URL", "when": "set"}), `"key" and "as"`)
refused(t, entry(renaming(), map[string]any{"key": "URL", "as": "NOTIFIER_URL"}), "not a usable key")
refused(t, entry(renaming(), map[string]any{"key": "url", "as": "NOTIFIER URL"}), "not a usable name")
refused(t, entry(renaming(), map[string]any{"key": "url", "as": "endpoints"}), `"endpoints"`, "the mesh reads")
refused(t, entry(renaming(), map[string]any{"key": "url", "as": "theme"}), `"theme"`, "twice")
refused(t, entry(renaming(), map[string]any{"key": "theme", "as": "THEME"}), `"theme"`, "twice")
refused(t, entry(renaming(), map[string]any{"key": "nothing", "as": "NOTHING"}), `"nothing"`, "declares no such value")
kept := entry(renaming(), map[string]any{"key": "url", "as": "NOTIFIER_URL"})
resourceOf(kept, "rules")["protected"] = []any{"NOTIFIER_URL"}
refused(t, kept, `"NOTIFIER_URL"`, "protected")
// Laid only when set, so the code's own fallback holds: never required.
required := renaming()
declaration(required, "url")["required"] = true
delete(declaration(required, "url"), "default")
refused(t, required, `"url"`, "only when it is set", "never required")
}
func TestARenamedEntryIsTheConfigFormsOnly(t *testing.T) {
m := map[string]any{
"module": "notifier", "version": "1",
"resources": []any{
map[string]any{"id": "rules", "type": "file", "path": "/x/rules.json", "merge": "json", "content": "{}",
"takes-settings": []any{map[string]any{"key": "url", "as": "NOTIFIER_URL"}}},
},
}
refused(t, m, "notifier's rules", "config form")
}
+3 -234
View File
@@ -63,7 +63,7 @@ func parseRaw(t *testing.T, m map[string]any) (Manifest, error) {
if err != nil {
t.Fatal(err)
}
return ParseManifest(raw)
return decodeForTest(raw)
}
// refused is that the manifest is refused with every one of these words.
@@ -97,66 +97,6 @@ func TestAModuleInTheConfigFormRegisters(t *testing.T) {
}
}
// Rule 1: a value without type or description.
func TestConfigRefusesAValueWithoutTypeOrDescription(t *testing.T) {
m := inConfigForm()
delete(declaration(m, "width"), "type")
refused(t, m, `notifier's config value "width"`, "no type")
m = inConfigForm()
declaration(m, "width")["type"] = "colour"
refused(t, m, `"width"`, `type "colour"`)
m = inConfigForm()
delete(declaration(m, "width"), "description")
refused(t, m, `"width"`, "no description")
}
// Rule 2: a value with neither a default nor required.
func TestConfigRefusesAValueWithNeitherDefaultNorRequired(t *testing.T) {
m := inConfigForm()
delete(declaration(m, "width"), "default")
refused(t, m, `"width"`, "neither a default nor required")
m = inConfigForm()
declaration(m, "sender")["default"] = "noreply@example.org"
refused(t, m, `"sender"`, "both a default and required")
}
// Rule 3: a default failing its validation.
func TestConfigRefusesADefaultFailingItsValidation(t *testing.T) {
for _, c := range []struct {
key string
value any
said string
}{
{"font-size", 4, "below its minimum"},
{"font-size", 41, "above its maximum"},
{"font-size", "10", "not a number"},
{"position", "bottom", "not one of"},
{"theme", "Dark", "does not match"},
} {
m := inConfigForm()
declaration(m, c.key)["default"] = c.value
refused(t, m, `"`+c.key+`"`, c.said)
}
}
// Rule 4: an object or a list not trusted.
func TestConfigRefusesAnObjectOrListNotTrusted(t *testing.T) {
for _, kind := range []string{"object", "list"} {
m := inConfigForm()
def := any(map[string]any{"a": 1})
if kind == "list" {
def = []any{"a"}
}
m["config"].(map[string]any)["rules"] = map[string]any{"type": kind, "description": "the rules, as " + kind,
"default": def, "runtime": true}
refused(t, m, `"rules"`, "is "+kind+" and not trusted")
m["config"].(map[string]any)["rules"].(map[string]any)["trusted"] = true
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("a trusted %s is refused: %v", kind, err)
}
}
}
// withPredefined is the list of predefined keys the mesh published, for the length of a test.
func withPredefined(t *testing.T, keys PredefinedList) {
t.Helper()
@@ -164,127 +104,6 @@ func withPredefined(t *testing.T, keys PredefinedList) {
t.Cleanup(func() { ReplacePublished(was) })
}
// Rule 5: a predefined key not on the list, or of another type. Where no list is present, a value naming one is
// refused saying so.
func TestConfigRefusesAPredefinedKeyNotOnTheListOrOfAnotherType(t *testing.T) {
withPredefined(t, nil)
m := inConfigForm()
declaration(m, "font-size")["predefined"] = "font-size"
refused(t, m, `"font-size"`, "the list of predefined keys", "is not present")
withPredefined(t, map[string]PredefinedKey{"font-size": {Type: "number"}, "accent": {Type: "string"}})
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("a value falling back to a listed key of its type is refused: %v", err)
}
m = inConfigForm()
declaration(m, "font-size")["predefined"] = "font-scale"
refused(t, m, `"font-size"`, `"font-scale"`, "not on the list")
m = inConfigForm()
declaration(m, "font-size")["predefined"] = "accent"
refused(t, m, `"font-size"`, `"accent"`, "is a string")
}
// Rule 6: an untrusted value falling back to a trusted predefined key.
func TestConfigRefusesAnUntrustedValueFallingBackToATrustedKey(t *testing.T) {
withPredefined(t, nil)
m := inConfigForm()
declaration(m, "theme")["predefined"] = "theme"
refused(t, m, `"theme"`, "is not present")
withPredefined(t, map[string]PredefinedKey{"theme": {Type: "string", Trusted: true}})
refused(t, m, `"theme"`, "trusted", "is not")
declaration(m, "theme")["trusted"] = true
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("a trusted value falling back to a trusted key is refused: %v", err)
}
}
// Rule 7: a value neither placed in a template nor read at runtime.
func TestConfigRefusesAValueNeitherPlacedNorReadAtRuntime(t *testing.T) {
m := inConfigForm()
delete(declaration(m, "history"), "runtime")
refused(t, m, `"history"`, "${config:history}", `"runtime": true`)
// Taken by a mergeable file is placed.
m = inConfigForm()
if _, err := parseRaw(t, m); err != nil {
t.Fatal(err)
}
}
// Rule 8: a placeholder naming an undeclared value.
func TestConfigRefusesAPlaceholderNamingAnUndeclaredValue(t *testing.T) {
m := inConfigForm()
resourceOf(m, "configuration")["content"] = "font = ${config:font-size}\nwidth = ${config:width}\n" +
"from = ${config:sender}\ncolour = ${config:colour}\n"
refused(t, m, "${config:colour}", "declares no such value")
}
// Rule 9: a reload-on or restart-on naming a file the module does not declare.
func TestConfigRefusesAReloadOrRestartOnNamingAnUndeclaredFile(t *testing.T) {
m := inConfigForm()
resourceOf(m, "notifier")["reload-on"] = []any{"configuration", "dunstrc"}
refused(t, m, "reload-on", `"dunstrc"`, "declares no such")
m = inConfigForm()
resourceOf(m, "notifier")["restart-on"] = []any{"rule"}
refused(t, m, "restart-on", `"rule"`)
// Another module's file, and one the mesh makes for the module, are not the module's to declare.
m = inConfigForm()
resourceOf(m, "notifier")["restart-on"] = []any{"rules", "resolver.fact-zones", "secret-mail", "composed-jails"}
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("refused: %v", err)
}
}
// Rule 10: a moved module's mergeable file not naming its keys under takes-config.
func TestConfigRefusesAMergeableFileNotNamingItsKeys(t *testing.T) {
m := inConfigForm()
delete(resourceOf(m, "rules"), "takes-config")
// position and theme are then placed nowhere; say they are read at runtime so only the file's rule speaks.
declaration(m, "position")["runtime"] = true
declaration(m, "theme")["runtime"] = true
refused(t, m, "rules", "names no takes-config")
}
// Rule 11: a mesh word, puid or pgid in takes-config.
func TestConfigRefusesAMeshWordOrAnAccountFactInTakesConfig(t *testing.T) {
for _, word := range []string{"ports", "expose", "reach", "endpoints", "places", "accesses", "networks", "puid", "pgid"} {
m := inConfigForm()
resourceOf(m, "rules")["takes-config"] = []any{"position", "theme", word}
refused(t, m, `"`+word+`"`, "takes-config")
}
}
// A module uses one form or the other, never both.
func TestAManifestMixingTheConfigAndSettingsFormsIsRefused(t *testing.T) {
m := inConfigForm()
resourceOf(m, "configuration")["content"] = "font = ${config:font-size}\nwidth = ${setting:width}\nfrom = ${config:sender}\n"
refused(t, m, "mixes", "${setting:", "config")
m = inConfigForm()
delete(resourceOf(m, "rules"), "takes-config")
resourceOf(m, "rules")["takes-settings"] = []any{"position", "theme"}
refused(t, m, "mixes", "takes-settings")
m = inConfigForm()
m["settings"] = map[string]any{"colour": map[string]any{"kind": "preference", "default": "red", "why": "red"}}
refused(t, m, "mixes", "settings")
}
// The settings form keeps its own declaration; the config form's has no kind or why.
func TestEachFormKeepsItsOwnDeclaration(t *testing.T) {
m := inConfigForm()
declaration(m, "width")["why"] = "fits a title"
refused(t, m, `"width"`, "why")
err := parsed(t, Manifest{Module: "notifier",
Settings: map[string]SettingDeclaration{"width": {Kind: KindPreference, Default: float64(250),
Why: "fits a title", Type: "number"}},
Resources: []map[string]any{{"id": "c", "type": "file", "path": "/x", "content": "${setting:width}"}}})
if err == nil || !strings.Contains(err.Error(), "under config") {
t.Fatalf("a typed value under settings: %v", err)
}
}
// What the mesh holds is what was written: a module in the config form is written back in it, and read again
// the same.
func TestAConfigManifestIsWrittenBackInItsOwnForm(t *testing.T) {
@@ -306,7 +125,7 @@ func TestAConfigManifestIsWrittenBackInItsOwnForm(t *testing.T) {
t.Fatalf("written back with %s: %s", unsaid, raw)
}
}
again, err := ParseManifest(raw)
again, err := decodeForTest(raw)
if err != nil {
t.Fatal(err)
}
@@ -365,7 +184,7 @@ const configDunst = `{"module": "dunst", "version": "1",
func TestAModuleRewrittenFromSettingsToConfigComposesByteIdentically(t *testing.T) {
composed := func(manifest string, layers []Layer) []byte {
t.Helper()
m, err := ParseManifest([]byte(manifest))
m, err := decodeForTest([]byte(manifest))
if err != nil {
t.Fatal(err)
}
@@ -404,14 +223,6 @@ func TestAModuleRewrittenFromSettingsToConfigComposesByteIdentically(t *testing.
}
}
// A key a mergeable file takes is a config value like every other: one not declared under config is refused by
// name (ADR 0348 §4, review of mesh-controller#320).
func TestConfigRefusesATakesConfigKeyNotDeclared(t *testing.T) {
m := inConfigForm()
resourceOf(m, "rules")["takes-config"] = []any{"position", "theme", "urgency"}
refused(t, m, `"urgency"`, "takes-config", "declares no such value under config")
}
// A required value nothing sets is refused at composition in the config form's words: ${config:<key>} and the
// config verb, never `settings set` (review of mesh-controller#320). The settings form's words are unchanged.
func TestAnUnsetRequiredValueIsRefusedInTheConfigFormsWords(t *testing.T) {
@@ -466,36 +277,6 @@ func TestAnUnsetRequiredValueIsRefusedInTheConfigFormsWords(t *testing.T) {
}
}
// A value named after the operator's own (a url, a user, an address) is `required` where nothing falls back when
// it is unset; where a mergeable file takes it and no placeholder asks for it, the file's own content (or the
// module's code reading the file) falls back, so it keeps a default and stays optional, as it is today: the
// operator's decision on issue 664's second question (issue 735).
func TestAnOperatorNamedValueAFileFallsBackOnMayHaveADefault(t *testing.T) {
m := inConfigForm()
m["config"].(map[string]any)["url"] = map[string]any{"type": "string",
"description": "the address the notifier posts to, a URL; empty for the one the file's software picks",
"default": ""}
resourceOf(m, "rules")["takes-config"] = []any{"position", "theme", "url"}
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("a taken value named after the operator's, with a default, was refused: %v", err)
}
// A placeholder asking for it has no fallback but the default, which would put a value in the definition
// where the operator's belongs (ADR 0112).
asked := inConfigForm()
asked["config"].(map[string]any)["url"] = map[string]any{"type": "string", "description": "a URL", "default": ""}
resourceOf(asked, "rules")["takes-config"] = []any{"position", "theme", "url"}
resourceOf(asked, "configuration")["content"] = "font = Inter ${config:font-size}\nwidth = ${config:width}\n" +
"from = ${config:sender}\npost = ${config:url}\n"
refused(t, asked, `"url"`, `a key naming "url" is the operator's value`)
// Read by the module's code alone, nothing in the definition says it falls back.
runtime := inConfigForm()
runtime["config"].(map[string]any)["url"] = map[string]any{"type": "string", "description": "a URL",
"default": "", "runtime": true}
refused(t, runtime, `"url"`, `a key naming "url" is the operator's value`)
}
// 1658/AC1: a value whose only placement is ${config:key} in an item's text is placed.
func TestConfigCountsAnItemsTextAsAPlacement(t *testing.T) {
m := map[string]any{
@@ -509,15 +290,3 @@ func TestConfigCountsAnItemsTextAsAPlacement(t *testing.T) {
t.Fatalf("a value placed only in an item's text is refused: %v", err)
}
}
// 1658/AC2: a declared value placed nowhere, with an item that does not say it, is still refused.
func TestConfigStillRefusesAValueNoItemPlaces(t *testing.T) {
m := map[string]any{
"module": "m", "version": "1",
"config": map[string]any{
"k": map[string]any{"type": "string", "description": "a word nothing says", "default": "x"},
},
"items": []any{map[string]any{"collection": "a", "content": "k = fixed\n"}},
}
refused(t, m, `"k"`, "placed nowhere")
}
-49
View File
@@ -124,55 +124,6 @@ func asDNSLabel(as string) string {
return strings.ReplaceAll(as, "_", "-")
}
// CheckServes refuses a `serves` block that names a consumer fact or an alphabet the mesh does not
// have, when the definition is parsed rather than when a consumer is resolved.
//
// A provision nobody consumes yet still has its rule read: a definition that would be refused the
// first time somebody required it is a definition that is wrong now.
func CheckServes(m Manifest) []string {
var problems []string
for _, provision := range sortedServes(m.Serves) {
for _, key := range sortedAnyKeys(m.Serves[provision]) {
text, ok := m.Serves[provision][key].(string)
if !ok {
// A provider's own name is filled only in a value of its own (novox/hq ADR 0346): one
// nested in a list or an object would be served as the placeholder, unsaid.
problems = append(problems, nestedOwnNames(m, provision, key, m.Serves[provision][key])...)
continue
}
// A probe identity, because what is checked is the shape of the statement and not
// what any consumer is called.
if _, err := consumerInto(text, "mesh_node_module"); err != nil {
problems = append(problems, fmt.Sprintf(
"%s serves %s, and the value it serves as %q %s", m.Module, provision, key, err))
}
// And the provider's own names it may serve (novox/hq ADR 0346).
problems = append(problems, checkServedOwnNames(m, provision, key, text)...)
// A label longer than DNS keeps is not truncated here (asDNSLabel), so the offer's own
// bound has to keep the identity inside one (ADR 0225).
if strings.Contains(text, "${consumer:as:dns}") {
if b := m.IdentityBoundOf(provision); !b.Bounded() || b.Max > dnsLabelLimit {
problems = append(problems, fmt.Sprintf(
"%s serves %s's consumers their identity as a DNS label in %q, and bounds "+
"that identity at %s: a label keeps %d — state an `identity` of at most %d",
m.Module, provision, key, boundWords(b), dnsLabelLimit, dnsLabelLimit))
}
}
}
}
return problems
}
// dnsLabelLimit is the longest DNS label (RFC 1035 §2.3.4).
const dnsLabelLimit = 63
func boundWords(b IdentityBound) string {
if !b.Bounded() {
return "nothing"
}
return fmt.Sprintf("%d", b.Max)
}
func sortedServes(serves map[string]map[string]any) []string {
out := make([]string, 0, len(serves))
for k := range serves {
+2 -2
View File
@@ -10,7 +10,7 @@ import (
func TestAConsumesEntryAskingWhereItStartsIsRefused(t *testing.T) { // 1279/AC4
for _, key := range []string{"from", "start", "deliver"} {
raw := `{"module":"audit","version":"1","consumes":[{"event":"shop.order.placed","` + key + `":"beginning"}]}`
_, err := ParseManifest([]byte(raw))
_, err := decodeForTest([]byte(raw))
if err == nil {
t.Fatalf("a consumes entry with %q was accepted", key)
}
@@ -18,7 +18,7 @@ func TestAConsumesEntryAskingWhereItStartsIsRefused(t *testing.T) { // 1279/AC4
t.Errorf("the refusal for %q does not name the key and the field: %v", key, err)
}
}
if _, err := ParseManifest([]byte(`{"module":"audit","version":"1","consumes":["shop.order.placed"]}`)); err != nil {
if _, err := decodeForTest([]byte(`{"module":"audit","version":"1","consumes":["shop.order.placed"]}`)); err != nil {
t.Errorf("a plain consumes entry was refused: %v", err)
}
}
-24
View File
@@ -218,30 +218,6 @@ func TestASettingReachesAContributionOnlyWhereItDeclaresTheKey(t *testing.T) {
}
}
func TestReceivingWhatYouDoNotProvideIsRefused(t *testing.T) {
// It would create a file nobody ever writes to, on a machine where nothing asked for it.
_, err := ParseManifest([]byte(`{"module":"traefik","version":"1",
"receives":{"reverse-proxy":"/etc/traefik/mesh.json"}}`))
if err == nil {
t.Fatal("a module received contributions to something it does not provide")
}
if !strings.Contains(err.Error(), "does not provide") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
func TestAnEmptyContributionIsRefused(t *testing.T) {
// Either a mistake or a requirement written the long way round, and both are better said.
_, err := ParseManifest([]byte(`{"module":"board","version":"1",
"contributes":{"reverse-proxy":{}}}`))
if err == nil {
t.Fatal("a module contributed nothing and was accepted")
}
if !strings.Contains(err.Error(), "require it") {
t.Fatalf("the refusal does not say what to do instead: %v", err)
}
}
// A provision answered from anywhere in the mesh has consumers on other machines, and the
// provider has to know who they are. Contributions were node-local until this, which meant the
// one case that most needed them was the one they did not reach.
@@ -113,24 +113,3 @@ func TestAPackageDeclaredPresentAndAbsentIsRefusedNamingBoth(t *testing.T) {
t.Errorf("an unrelated assignment is refused for a contradiction already there: %v", err)
}
}
func TestCodeForAPowerMomentDependsOnThePowerSeat(t *testing.T) {
for _, moment := range powerMoments {
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: moment, Slot: "normal", Code: "true"}}
if got := DependsOn(nil, m); !reflect.DeepEqual(got, []string{PowerSeat}) {
t.Errorf("code for %s depends on %v, want %s", moment, got, PowerSeat)
}
if p := m.shellProblems(); len(p) != 0 {
t.Errorf("code for %s is refused: %v", moment, p)
}
}
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: "after-lunch", Slot: "normal", Code: "true"}}
if p := m.shellProblems(); len(p) == 0 {
t.Error("code for a moment that does not exist was accepted")
}
if s, ok := SeatNamed(PowerSeat); !ok || s.Scope != ScopeNode {
t.Errorf("%s is not a node seat of the mesh's own: %+v %v", PowerSeat, s, ok)
}
}
+1 -1
View File
@@ -33,7 +33,7 @@ const provisionerClient = `{"module":"mesh-provisioner-client","version":"1",
func clientOn(t *testing.T) Resolution {
t.Helper()
m, err := ParseManifest([]byte(provisionerClient))
m, err := decodeForTest([]byte(provisionerClient))
if err != nil {
t.Fatal(err)
}
-77
View File
@@ -13,80 +13,3 @@ package catalogue
// The ordinary five fields — minute, hour, day-of-month, month, day-of-week — with `*`, lists
// (`,`), ranges (`-`) and steps (`/`). No names (jan, mon): a numeric cron is what the host
// evaluates, and refusing a name here is refusing it near whoever wrote it.
import (
"fmt"
"strconv"
"strings"
)
// validateCron reports why a string is not a five-field cron expression, or nil if it is one.
func validateCron(expr string) error {
fields := strings.Fields(expr)
if len(fields) != 5 {
return fmt.Errorf(
"a schedule is a five-field cron expression (minute hour day-of-month month "+
"day-of-week), and this has %d field(s)", len(fields))
}
bounds := []struct {
name string
min, max int
}{
{"minute", 0, 59},
{"hour", 0, 23},
{"day-of-month", 1, 31},
{"month", 1, 12},
{"day-of-week", 0, 7}, // 0 and 7 are both Sunday, the convention every cron keeps
}
for i, b := range bounds {
if err := validateCronField(fields[i], b.min, b.max); err != nil {
return fmt.Errorf("%s field: %w", b.name, err)
}
}
return nil
}
// validateCronField checks one field's elements are within range and well-formed.
func validateCronField(spec string, min, max int) error {
if spec == "" {
return fmt.Errorf("is empty")
}
for _, part := range strings.Split(spec, ",") {
if part == "" {
return fmt.Errorf("%q has an empty element between commas", spec)
}
rangePart := part
if slash := strings.IndexByte(part, '/'); slash >= 0 {
rangePart = part[:slash]
n, err := strconv.Atoi(part[slash+1:])
if err != nil || n < 1 {
return fmt.Errorf("step in %q is not a positive number", part)
}
}
switch {
case rangePart == "*":
// Any value; nothing to bound.
case strings.IndexByte(rangePart, '-') >= 0:
dash := strings.IndexByte(rangePart, '-')
lo, errLo := strconv.Atoi(rangePart[:dash])
hi, errHi := strconv.Atoi(rangePart[dash+1:])
if errLo != nil || errHi != nil {
return fmt.Errorf("range %q is not two numbers", rangePart)
}
if lo < min || hi > max || lo > hi {
return fmt.Errorf("range %q is outside the allowed %d-%d", part, min, max)
}
default:
v, err := strconv.Atoi(rangePart)
if err != nil {
return fmt.Errorf("%q is not a number", rangePart)
}
if v < min || v > max {
return fmt.Errorf("%q is outside the allowed range %d-%d", part, min, max)
}
}
}
return nil
}
-364
View File
@@ -5,7 +5,6 @@ import (
"encoding/json"
"fmt"
"regexp"
"sort"
"strconv"
"strings"
"time"
@@ -305,182 +304,6 @@ func keepsByClass(class string) bool {
return class == ClassIrreplaceable || class == ClassValuable || class == ClassRebuildable
}
// dataID is what an item's id may be: it is a token in a condition's key and a word on a line.
var dataID = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// dataPathRef is an item's path: one of the module's directories or accesses, and optionally a path
// beneath it.
var dataPathRef = regexp.MustCompile(`^\$\{(dir|access):([a-z0-9][a-z0-9-]*)\}(/[^\s$]*)?$`)
// dataProblems is what is wrong with the `data` section itself, from the manifest alone: judged at
// registration as every other per-manifest problem is. Whether a module declares what it should is
// the catalogue check's (DataProblems), because a module already running was written before it.
func (m Manifest) dataProblems() []string {
if m.Data == nil {
return nil
}
var problems []string
say := func(format string, args ...any) {
problems = append(problems, fmt.Sprintf("%s's data: ", m.Module)+fmt.Sprintf(format, args...))
}
dirs := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
dirs[fmt.Sprint(r["id"])] = true
}
}
accesses := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
accesses[a.ID] = true
}
}
ids := map[string]DataItem{}
paths := map[string]string{}
for i, it := range m.Data.Own {
label := it.ID
if label == "" {
label = fmt.Sprintf("item %d", i+1)
}
if !dataID.MatchString(it.ID) {
say("%s has no usable id: lower-case letters, digits and dashes", label)
} else if _, twice := ids[it.ID]; twice {
say("%s is declared twice", it.ID)
}
ids[it.ID] = it
ref := dataPathRef.FindStringSubmatch(it.Path)
switch {
case ref == nil:
say("%s's path %q is not one of the module's directories or accesses: ${dir:<id>} or ${access:<id>}, "+
"or a path beneath one (a definition names no machine path, novox/hq ADR 0112)", label, it.Path)
case ref[1] == "dir" && !dirs[ref[2]]:
say("%s's path names ${dir:%s}, and %s declares no directory %q", label, ref[2], m.Module, ref[2])
case ref[1] == "access" && !accesses[ref[2]]:
say("%s's path names ${access:%s}, and %s declares no access %q", label, ref[2], m.Module, ref[2])
case strings.Contains(it.Path, ".."):
say("%s's path %q climbs out of its directory", label, it.Path)
}
if other, twice := paths[it.Path]; twice && it.Path != "" {
say("%s and %s name the same path %s", other, label, it.Path)
}
paths[it.Path] = label
switch it.Class {
case ClassIrreplaceable, ClassValuable, ClassRebuildable, ClassCache:
case ClassNone:
say("%s is class none, which only a provision keeping nothing of its consumers' is; own data "+
"that is disposable is cache", label)
default:
say("%s's class %q is not one the mesh protects by: irreplaceable, valuable, rebuildable or cache",
label, it.Class)
}
if it.Class == ClassIrreplaceable && !it.BackedUp() && strings.TrimSpace(it.Redundancy) == "" {
say("%s is irreplaceable and is neither backed up nor said to be protected by redundancy; the only "+
"copy of something is protected one way or the other (novox/hq ADR 0233) — copy it, dump it into "+
"another item, or say `redundancy` with why that is enough", label)
}
if it.Class == ClassCache && it.Backup != nil && !it.Backup.None {
say("%s is a cache and asks to be backed up; a cache is disposable, or it is not a cache", label)
}
if it.Class == ClassCache && it.Redundancy != "" {
say("%s is a cache and says it is protected by redundancy; a cache is not protected", label)
}
switch it.Measure {
case "", MeasureWalk, MeasureDataset, MeasureShallow:
default:
say("%s's measure %q is walk, dataset or shallow", label, it.Measure)
}
if _, err := ParseDataDuration(it.Within); err != nil {
say("%s's within: %v", label, err)
}
if _, err := ParseDataDuration(it.Active); err != nil {
say("%s's active: %v", label, err)
}
if it.Within != "" && !it.BackedUp() {
say("%s states within, and is not backed up", label)
}
if it.Active != "" && !Watched(it.Class) {
say("%s is %s and expects writes; only irreplaceable and valuable data is watched", label, it.Class)
}
}
// Dumps go into an item of the same module, declared, and not into themselves.
for _, it := range m.Data.Own {
if it.Backup == nil || it.Backup.Copy || it.Backup.None {
continue
}
switch into, ok := ids[it.Backup.Into]; {
case strings.TrimSpace(it.Backup.Dump) == "":
say("%s's backup names no dump command", it.ID)
case it.Backup.Into == "":
say("%s's dump says no item it writes into", it.ID)
case !ok:
say("%s's dump writes into %q, which is not one of the module's data items", it.ID, it.Backup.Into)
case into.ID == it.ID:
say("%s's dump writes into itself; a dump is copied from where it lands", it.ID)
case into.Class == ClassCache:
say("%s's dump writes into %s, a cache; what is copied must not be disposable", it.ID, into.ID)
}
if it.Backup.Dump != "" {
declared := map[string]string{}
for d := range dirs {
declared[d] = ""
}
if _, err := dirFill(it.Backup.Dump, declared, m.Module); err != nil {
say("%s's dump: %v", it.ID, err)
}
}
}
provided := map[string]bool{}
for _, o := range m.Provides {
provided[o.Name] = true
}
wants := map[string]bool{}
for _, w := range m.Wants() {
wants[w] = true
}
for _, provision := range sortedKeys(m.Data.Consumers) {
c := m.Data.Consumers[provision]
if !provided[provision] {
say("says what it keeps for the consumers of %q, which it does not provide", provision)
}
switch c.Class {
case ClassIrreplaceable, ClassValuable, ClassRebuildable, ClassCache, ClassNone:
default:
say("its consumers' %s is class %q; one of irreplaceable, valuable, rebuildable, cache or none", provision, c.Class)
}
switch {
case c.Class == ClassNone && c.In != "":
say("its consumers' %s keeps nothing and says where it is kept (%s)", provision, c.In)
case keepsByClass(c.Class) && c.In == "":
say("its consumers' %s is %s and says nowhere it lives: `in` names one of the module's items, or a "+
"provision it requires", provision, c.Class)
case c.In != "":
if own, ok := ids[c.In]; ok {
if c.Class == ClassIrreplaceable && !own.BackedUp() && own.Redundancy == "" {
say("its consumers' %s is irreplaceable and lives in %s, which is not protected", provision, c.In)
}
if classRank[own.Class] < classRank[c.Class] {
say("its consumers' %s is %s and lives in %s, which is only %s", provision, c.Class, c.In, own.Class)
}
} else if !wants[c.In] {
say("its consumers' %s lives in %q, which is neither one of its data items nor a provision it "+
"requires", provision, c.In)
}
}
}
for _, provision := range sortedKeys(m.Data.KeptBy) {
k := m.Data.KeptBy[provision]
if !wants[provision] {
say("says it keeps data with the provider of %q, which it does not require", provision)
}
switch k.Class {
case ClassIrreplaceable, ClassValuable, ClassRebuildable, ClassCache:
default:
say("what it keeps with %s is class %q; one of irreplaceable, valuable, rebuildable or cache", provision, k.Class)
}
}
return problems
}
// KeepsConsumerData is whether this module, providing a provision, keeps what each consumer writes
// there (novox/hq ADR 0232, ADR 0233): whether a consumer bound to it is bound to its data.
//
@@ -596,196 +419,9 @@ func (m Manifest) allContributions() []SeatContribution {
// --- the catalogue check ------------------------------------------------------------------------
// DataProblems is what the catalogue check refuses about the data a module declares (novox/hq ADR
// 0233), judged over the manifests given together:
//
// - a provider that grants a provision says what it keeps for that provision's consumers;
// - nobody says it on the offer any more (`keeps-consumer-data`): the data section is the one place;
// - nobody writes a backup line by hand: it is derived from the data section;
// - a directory a container writes, mounted whole, is a data item of some class;
// - consumer data said to live in a required provision lives where that provision's provider keeps
// its consumers' data, as preciously, when the provider is given;
// - data a consumer keeps with a provider as irreplaceable is protected there, when the provider is
// given.
//
// **The check's, not registration's**, as ADR 0214's backup rule was: a module already on the shelf
// was written before the rule, and refusing it there would refuse the very providers whose data the
// rule protects. Each module meets it where it is written.
func DataProblems(shelf Shelf) []string {
var problems []string
for _, name := range shelfOrder(shelf) {
m := shelf[name]
for _, provision := range sortedKeys(m.Grants) {
if _, said := m.ConsumerDataOf(provision); !said {
problems = append(problems, fmt.Sprintf(
"%s grants %s and does not say what it keeps for its consumers: data.consumers.%s with a "+
"class — irreplaceable, valuable, rebuildable, cache, or none (novox/hq ADR 0233)", name, provision, provision))
}
}
for _, o := range m.Provides {
if o.KeepsConsumerData != nil {
problems = append(problems, fmt.Sprintf(
"%s says keeps-consumer-data on its offer of %s; it is said once, in data.consumers.%s, with a "+
"class (novox/hq ADR 0233)", name, o.Name, o.Name))
}
}
for i, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known && s.Name == BackupSeat {
problems = append(problems, fmt.Sprintf(
"%s's contribution %d is a backup line written by hand; backups are derived from the data "+
"section — declare the data, with its class and how it is protected (novox/hq ADR 0233)", name, i+1))
}
}
for _, dir := range writtenDirectories(m) {
if !coversDirectory(m, dir) {
problems = append(problems, fmt.Sprintf(
"%s mounts its directory %q into a container to be written, and declares no data in it: "+
"data.own with a class — cache if it is disposable (novox/hq ADR 0233)", name, dir))
}
}
if m.Data == nil {
continue
}
for _, provision := range sortedKeys(m.Data.Consumers) {
c := m.Data.Consumers[provision]
if c.In == "" {
continue
}
if _, own := m.DataItem(c.In); own {
continue
}
for _, pname := range shelfOrder(shelf) {
p := shelf[pname]
pc, said := p.ConsumerDataOf(c.In)
if !said || !providesName(p, c.In) {
continue
}
if classRank[pc.Class] < classRank[c.Class] {
problems = append(problems, fmt.Sprintf(
"%s keeps its consumers' %s, %s, in %s — and %s keeps its consumers' %s as %s, "+
"so it is not protected as %s", name, provision, c.Class, c.In, pname, c.In, pc.Class, c.Class))
}
}
}
for _, provision := range sortedKeys(m.Data.KeptBy) {
k := m.Data.KeptBy[provision]
if k.Class != ClassIrreplaceable {
continue
}
for _, pname := range shelfOrder(shelf) {
p := shelf[pname]
pc, said := p.ConsumerDataOf(provision)
if !said || !providesName(p, provision) {
continue
}
if !keepsByClass(pc.Class) {
problems = append(problems, fmt.Sprintf(
"%s keeps irreplaceable data with %s, and %s keeps its consumers' %s as %s — nothing of it is "+
"protected there", name, provision, pname, provision, pc.Class))
continue
}
if in, own := p.DataItem(pc.In); own && !in.BackedUp() && in.Redundancy == "" {
problems = append(problems, fmt.Sprintf(
"%s keeps irreplaceable data with %s, and %s keeps it in %s, which is neither backed up nor "+
"on declared redundancy", name, provision, pname, pc.In))
}
}
}
}
return problems
}
func providesName(m Manifest, provision string) bool {
for _, o := range m.Provides {
if o.Name == provision {
return true
}
}
return false
}
// writtenDirectories are the module's directories a container mounts whole and may write: a volume
// `${dir:<id>}:<target>` without `:ro`, or a directory stated by an absolute path mounted the same
// way. A file beneath a directory, or a read-only mount, is configuration the mesh wrote.
func writtenDirectories(m Manifest) []string {
absolute := map[string]string{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "directory" {
continue
}
if p, ok := r["path"].(string); ok && strings.HasPrefix(p, "/") {
absolute[strings.TrimRight(p, "/")] = fmt.Sprint(r["id"])
}
}
seen := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
vols, _ := r["volumes"].([]any)
for _, v := range vols {
s, _ := v.(string)
mount := volumeMount.FindStringSubmatch(s)
if mount == nil || volumeReadOnly(mount[3]) {
continue
}
src := strings.TrimRight(mount[1], "/")
if ref := dirPlain.FindStringSubmatch(src); ref != nil {
seen[ref[1]] = true
} else if id, ok := absolute[src]; ok {
seen[id] = true
}
}
}
out := make([]string, 0, len(seen))
for id := range seen {
out = append(out, id)
}
sort.Strings(out)
return out
}
// volumeMount is a container's volume, `<source>:<target>[:<options>]`, its source possibly a
// `${dir:<id>}` — whose own colon is not the separator.
var volumeMount = regexp.MustCompile(`^(\$\{(?:dir|access):[a-z0-9][a-z0-9-]*\}[^:]*|[^:]+):([^:]+)(?::(.*))?$`)
// volumeReadOnly is whether a volume's options mount it read-only.
func volumeReadOnly(options string) bool {
for _, o := range strings.Split(options, ",") {
if strings.TrimSpace(o) == "ro" {
return true
}
}
return false
}
// dirPlain is a whole directory, `${dir:<id>}` and nothing after it.
var dirPlain = regexp.MustCompile(`^\$\{dir:([a-z0-9][a-z0-9-]*)\}$`)
// coversDirectory is whether a data item names the directory, a path within it, or a directory it
// is placed beneath.
func coversDirectory(m Manifest, dir string) bool {
parents := map[string]string{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "directory" {
continue
}
if p, ok := r["path"].(string); ok {
if ref := dirRef.FindStringSubmatch(p); ref != nil && strings.HasPrefix(p, "${dir:") {
parents[fmt.Sprint(r["id"])] = ref[1]
}
}
}
for _, it := range m.DataItems() {
ref := dataPathRef.FindStringSubmatch(it.Path)
if ref == nil || ref[1] != "dir" {
continue
}
for d, hops := dir, 0; d != "" && hops < 4; d, hops = parents[d], hops+1 {
if ref[2] == d {
return true
}
}
}
return false
}
+1 -120
View File
@@ -2,7 +2,6 @@ package catalogue
import (
"slices"
"strings"
"testing"
)
@@ -24,111 +23,13 @@ const declaredStore = `{"module":"pg","version":"1",
func mustParse(t *testing.T, raw string) Manifest {
t.Helper()
m, err := ParseManifest([]byte(raw))
m, err := decodeForTest([]byte(raw))
if err != nil {
t.Fatalf("refused: %v", err)
}
return m
}
func TestADeclaredStoreParsesAndPassesTheCheck(t *testing.T) {
m := mustParse(t, declaredStore)
if problems := DataProblems(Shelf{"pg": m}); len(problems) > 0 {
t.Fatalf("a store declaring its data was refused: %v", problems)
}
if !m.KeepsConsumerData("postgres-database") {
t.Fatal("an irreplaceable consumer class does not keep the consumer's data")
}
if it, _ := m.DataItem("store"); !it.BackedUp() || it.BackupWithin() != DefaultBackupWithin || it.ActiveWithin().Hours() != 24 {
t.Fatalf("the store item reads %+v", it)
}
if it, _ := m.DataItem("dumps"); it.BackedUp() {
t.Fatal("a rebuildable item that says none is backed up")
}
}
// Every rule of the section itself, refused at parse in the words of the module.
func TestTheDataSectionIsRefusedWhereItIsWrong(t *testing.T) {
for _, c := range []struct{ name, data, want string }{
{"a class it does not know", `{"own":[{"id":"a","path":"${dir:d}","class":"precious"}]}`, "not one the mesh protects by"},
{"an access it does not have", `{"own":[{"id":"a","path":"${access:media}","class":"irreplaceable","redundancy":"an array"}]}`, "declares no access"},
{"irreplaceable and protected by nothing", `{"own":[{"id":"a","path":"${dir:d}","class":"irreplaceable","backup":"none"}]}`, "protected one way or the other"},
{"a cache on redundancy", `{"own":[{"id":"a","path":"${dir:d}","class":"cache","redundancy":"an array"}]}`, "a cache is not protected"},
{"kept with something not required", `{"kept-by":{"s3-bucket":{"class":"irreplaceable"}}}`, "does not require"},
{"a measure it does not know", `{"own":[{"id":"a","path":"${dir:d}","class":"cache","measure":"du"}]}`, "walk, dataset or shallow"},
{"a cache expecting writes", `{"own":[{"id":"a","path":"${dir:d}","class":"cache","active":"1d"}]}`, "only irreplaceable and valuable"},
{"a machine path", `{"own":[{"id":"a","path":"/srv/a","class":"cache"}]}`, "names no machine path"},
{"a directory it does not declare", `{"own":[{"id":"a","path":"${dir:nowhere}","class":"cache"}]}`, "declares no directory"},
{"a cache backed up", `{"own":[{"id":"a","path":"${dir:d}","class":"cache","backup":"copy"}]}`, "disposable"},
{"a dump into nothing", `{"own":[{"id":"a","path":"${dir:d}","class":"irreplaceable","backup":{"dump":"x","into":"b"}}]}`, "not one of the module's data items"},
{"a dump into itself", `{"own":[{"id":"a","path":"${dir:d}","class":"irreplaceable","backup":{"dump":"x","into":"a"}}]}`, "into itself"},
{"a bad duration", `{"own":[{"id":"a","path":"${dir:d}","class":"irreplaceable","within":"soon"}]}`, "not a duration"},
{"the same id twice", `{"own":[{"id":"a","path":"${dir:d}","class":"cache"},{"id":"a","path":"${dir:d}/x","class":"cache"}]}`, "declared twice"},
{"own data of class none", `{"own":[{"id":"a","path":"${dir:d}","class":"none"}]}`, "only a provision"},
{"consumers of something not provided", `{"consumers":{"s3-bucket":{"class":"cache"}}}`, "does not provide"},
{"consumer data that lives nowhere", `{"consumers":{"q":{"class":"irreplaceable"}}}`, "says nowhere it lives"},
{"consumer data in an item less precious", `{"own":[{"id":"a","path":"${dir:d}","class":"rebuildable"}],"consumers":{"q":{"class":"irreplaceable","in":"a"}}}`, "which is only rebuildable"},
{"consumer data in something unknown", `{"consumers":{"q":{"class":"rebuildable","in":"elsewhere"}}}`, "neither one of its data items"},
} {
raw := `{"module":"m","version":"1","provides":["q"],"resources":[{"id":"d","type":"directory","mode":"0700"}],"data":` + c.data + `}`
_, err := ParseManifest([]byte(raw))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: %v, want %q", c.name, err, c.want)
}
}
}
// The catalogue check: a granting provider says what it keeps, says it in one place, writes no backup
// line by hand, and every directory a container writes is declared (novox/hq ADR 0233).
func TestTheCatalogueCheckAsksEveryModuleForItsData(t *testing.T) {
unsaid := mustParse(t, `{"module":"q","version":"1","provides":[{"name":"queue","scope":"mesh"}],
"grants":{"queue":"${dir:g}"},"resources":[{"id":"g","type":"directory","mode":"0700"}]}`)
onTheOffer := mustParse(t, `{"module":"r","version":"1","provides":[{"name":"resolver","scope":"mesh","keeps-consumer-data":false}]}`)
byHand := mustParse(t, `{"module":"h","version":"1","resources":[{"id":"d","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:d}"}]}`)
writes := mustParse(t, `{"module":"w","version":"1","resources":[{"id":"d","type":"directory","mode":"0700"},
{"id":"c","type":"directory","mode":"0700"},
{"id":"app","type":"container","image":"x@sha256:aa","volumes":["${dir:d}:/data","${dir:c}:/etc/app:ro","${dir:d}/f:/f"]}]}`)
problems := strings.Join(DataProblems(Shelf{"q": unsaid, "r": onTheOffer, "h": byHand, "w": writes}), "\n")
for _, want := range []string{
"q grants queue and does not say what it keeps",
"r says keeps-consumer-data on its offer",
"h's contribution 1 is a backup line written by hand",
`w mounts its directory "d" into a container to be written`,
} {
if !strings.Contains(problems, want) {
t.Errorf("the check does not say %q:\n%s", want, problems)
}
}
if strings.Contains(problems, `"c"`) {
t.Errorf("a read-only mount was taken for written data:\n%s", problems)
}
// The same module declaring the directory, as a cache, passes.
declared := mustParse(t, `{"module":"w","version":"1","data":{"own":[{"id":"d","path":"${dir:d}","class":"cache"}]},
"resources":[{"id":"d","type":"directory","mode":"0700"},
{"id":"app","type":"container","image":"x@sha256:aa","volumes":["${dir:d}:/data"]}]}`)
if p := DataProblems(Shelf{"w": declared}); len(p) > 0 {
t.Fatalf("a declared directory is still refused: %v", p)
}
}
// Consumer data said to live in a provision the module requires is protected only as well as that
// provision's provider protects its consumers' data.
func TestConsumerDataInARequiredProvisionIsJudgedAgainstItsProvider(t *testing.T) {
idp := mustParse(t, `{"module":"idp","version":"1","requires":["postgres-database"],
"provides":[{"name":"oidc-client","scope":"mesh"}],"grants":{"oidc-client":"${dir:g}"},
"resources":[{"id":"g","type":"directory","mode":"0700"}],
"data":{"consumers":{"oidc-client":{"class":"irreplaceable","in":"postgres-database"}}}}`)
cachy := mustParse(t, strings.Replace(declaredStore, `"consumers":{"postgres-database":{"class":"irreplaceable","in":"store"}}`,
`"consumers":{"postgres-database":{"class":"cache"}}`, 1))
if p := strings.Join(DataProblems(Shelf{"idp": idp, "pg": cachy}), "\n"); !strings.Contains(p, "not protected as irreplaceable") {
t.Fatalf("irreplaceable data kept in a cache passed: %s", p)
}
if p := DataProblems(Shelf{"idp": idp, "pg": mustParse(t, declaredStore)}); len(p) > 0 {
t.Fatalf("refused against a provider that keeps its consumers' data: %v", p)
}
}
// What a binding is sticky by follows the class (ADR 0232, generalised by 0233): a cache's consumers
// move freely, a store's do not; an older definition saying nothing still follows its grant.
func TestKeepingConsumerDataFollowsTheClass(t *testing.T) {
@@ -215,23 +116,3 @@ func TestTheBackupHoldersLinesAreDerivedFromTheData(t *testing.T) {
t.Fatalf("an operator's path reads %+v", it)
}
}
// Data a consumer keeps with a provider as irreplaceable must be protected there: the check refuses a
// provider that keeps its consumers' data as a cache, or in an item with no protection.
func TestIrreplaceableDataKeptWithAProviderMustBeProtectedThere(t *testing.T) {
photos := mustParse(t, `{"module":"photos","version":"1","requires":["s3-bucket"],
"data":{"kept-by":{"s3-bucket":{"class":"irreplaceable"}}}}`)
store := func(backup string) Manifest {
return mustParse(t, `{"module":"objects","version":"1","provides":[{"name":"s3-bucket","scope":"mesh"}],
"grants":{"s3-bucket":"${dir:g}"},
"data":{"own":[{"id":"data","path":"${dir:data}","class":"valuable","backup":"`+backup+`"}],
"consumers":{"s3-bucket":{"class":"valuable","in":"data"}}},
"resources":[{"id":"g","type":"directory","mode":"0700"},{"id":"data","type":"directory","mode":"0700"}]}`)
}
if p := DataProblems(Shelf{"photos": photos, "objects": store("copy")}); len(p) > 0 {
t.Fatalf("a provider backing its consumers' data up was refused: %v", p)
}
if p := strings.Join(DataProblems(Shelf{"photos": photos, "objects": store("none")}), "\n"); !strings.Contains(p, "neither backed up") {
t.Fatalf("irreplaceable photos in an unprotected store passed: %s", p)
}
}
+42
View File
@@ -0,0 +1,42 @@
package catalogue
import (
"encoding/json"
"strings"
)
// Decode reads a manifest leniently, as every part but the judge reads one (novox/hq ADR 0578 §3): every field the
// contract knows as it declares it, and a field it does not know set aside and named (SetAside), never refused. It
// judges nothing: no name, seat, bus or catalogue rule is applied, since the indexer judged the manifest once and its
// verdict is in the indexer's state (delivery.VerdictsBucket); a reader takes the verdict beside it
// (delivery.TakeVerdict). An error is only bytes that are no manifest: not JSON, or a field of the wrong shape.
func Decode(raw []byte) (Manifest, error) {
var m Manifest
if err := json.Unmarshal(raw, &m); err != nil {
return Manifest{}, err
}
return m, nil
}
// SetAside names what Decode set aside of this manifest: each field it does not know, as the decoder words the first
// and with the path of every one it read without, or "" when it knew every field. A reader says it in its own words
// (its journal, its finding) and reads the rest; the part that needs the field takes the newer package (ADR 0578 §6).
func (m Manifest) SetAside() string { return m.unknown }
// ReadNoFurther is whether Decode could not read past a field it does not know, so the manifest holds its name and
// version alone: a reader holds such a module, as it holds a refused one, and names why (SetAside).
func (m Manifest) ReadNoFurther() bool { return strings.Contains(m.unknown, readNoFurther) }
// readNoFurther marks, in what a manifest set aside, that it was not read past it.
const readNoFurther = "(read no further: "
// Predefined is the list of predefined keys the manifest was read with (ReadTreeManifestWritten, or the judge's
// SetPredefined), nil when none: data of the contract a judge reads, never a rule.
func (m Manifest) Predefined() PredefinedList { return m.predefined }
// SetPredefined keeps the list of predefined keys a manifest is read with, for what reads it after (the judge).
func (m *Manifest) SetPredefined(list PredefinedList) { m.predefined = list }
// FormsWritten is the fields the manifest's config was written with, in the config form (`config`, `takes-config`,
// `${config:…}`) and in the settings form (novox/hq ADR 0348 §1): data a judge reads to say a manifest mixes them.
func (m Manifest) FormsWritten() (config, settings []string) { return m.forms.config, m.forms.settings }
+12
View File
@@ -0,0 +1,12 @@
package catalogue
// decodeForTest builds a manifest for a test as every reader reads one (Decode); the judge's own tests are in
// mesh-delivery's judge (novox/hq ADR 0578 §4).
func decodeForTest(raw []byte) (Manifest, error) { return Decode(raw) }
// decodeWithForTest is decodeForTest keeping the list of predefined keys it was read with.
func decodeWithForTest(raw []byte, list PredefinedList) (Manifest, error) {
m, err := Decode(raw)
m.SetPredefined(list)
return m, err
}
-25
View File
@@ -1,7 +1,5 @@
package catalogue
import "fmt"
// The delivery's owner (novox/hq ADR 0239, to-be 47): one holder for the mesh, the module mesh-delivery,
// which owns a delivery — one commit in one repository, from its pull request's head to every machine —
// and a delivery group, and asks the controller for every act. A seat of the mesh's own, so the role has
@@ -107,29 +105,6 @@ func SeatSays(emits []string, event string) bool {
return false
}
// seatEventProblems refuses a module that lists among its own `emits` an event a seat it claims says (the lead,
// 2026-10-29, issue 1952, the second of a kind: the self-check's `checked` and mesh-hand-acts' `act-recorded`). A
// consumer of `<seat>.<event>` hears the seat's subject (SeatSays), so the holder says it there, with stdio.SeatPublish
// on `mesh.seat.<seat>.event.<event>`; said through stdio.Emit as the module's own, it lands where nobody listens. Caught
// here, when the manifest is read at the build, not when nothing is heard on the machines.
func seatEventProblems(m Manifest) []string {
var problems []string
for _, c := range m.Claims {
seat, known := SeatNamed(c.Name)
if !known || len(seat.Emits) == 0 {
continue
}
for _, e := range m.Emits {
if SeatSays(seat.Emits, e) {
problems = append(problems, fmt.Sprintf("%s lists %q among its own emits, and the seat %s it claims says it: "+
"say it on the seat with stdio.SeatPublish(\"mesh.seat.%s.event.%s\", ...) and leave it out of emits, or "+
"no consumer of %s.%s hears it", m.Module, e, seat.Name, seat.Name, e, seat.Name, e))
}
}
}
return problems
}
// topicMatches is whether an event's name matches a declared pattern, `*` one dot-separated segment and
// `>` or `**` the rest.
func topicMatches(pattern, event string) bool {
-26
View File
@@ -193,32 +193,6 @@ func TestASettingComposesWithADerivedValue(t *testing.T) {
}
}
// A fact or an alphabet the mesh does not have is refused where the definition is, not where a
// consumer happens to be resolved — and the refusal says what may be said instead.
func TestAServedValueNamingSomethingTheMeshDoesNotHaveIsRefused(t *testing.T) {
for _, c := range []struct{ value, says string }{
{"${consumer:node}", "as"},
{"${consumer:as:punycode}", "dns"},
} {
m := store()
m.Serves["s3-bucket"]["bucket"] = c.value
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
_, err = ParseManifest(raw)
if err == nil {
t.Fatalf("%s was accepted", c.value)
}
if !strings.Contains(err.Error(), c.value) {
t.Errorf("the refusal of %s does not quote it: %v", c.value, err)
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("the refusal of %s does not say what may be said (%q): %v", c.value, c.says, err)
}
}
}
// `dns` is checked against an identity the mesh actually mints, not an invented string.
func TestTheDNSAlphabetIsTheMintedIdentityWithItsSeparatorRewritten(t *testing.T) {
as := ConsumerIdentity("anchor", IdentitySource("ncloud", "nextcloud"))
-111
View File
@@ -106,14 +106,6 @@ func dirsFor(m Manifest, with Rendering) map[string]string {
return dirs
}
// placedOrAbsolute says a path is usable where the mesh needs one: absolute already, or
// beginning with a placed reference — resolution makes it absolute before anything reads it.
// (unknownDirRefs is what checks the reference names a real directory.)
func placedOrAbsolute(path string) bool {
return strings.HasPrefix(path, "/") ||
(strings.HasPrefix(path, "${dir:") && dirRef.MatchString(path))
}
// dirFill resolves every ${dir:…} in one string, or refuses a reference naming no directory.
func dirFill(s string, dirs map[string]string, module string) (string, error) {
var missing error
@@ -260,109 +252,6 @@ func dirInto(resource map[string]any, dirs map[string]string, module string) err
return nil
}
// unknownDirRefs is every ${dir:…} in the definition that names no directory the definition
// declares — refused where the author is, not at composition on some later day (the same
// near-versus-far reasoning as the host's strict parse).
func (m Manifest) unknownDirRefs() []string {
declared := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
declared[fmt.Sprint(r["id"])] = true
}
}
referenced := func(s string) []string {
var ids []string
for _, match := range dirRef.FindAllStringSubmatch(s, -1) {
ids = append(ids, match[1])
}
return ids
}
var problems []string
for _, r := range m.Resources {
place, said := r["place"].(string)
if !said {
continue
}
if fmt.Sprint(r["type"]) != "directory" {
problems = append(problems, fmt.Sprintf(
"%s says place on %v, which is not a directory — only a directory is placed",
m.Module, r["id"]))
continue
}
if path, stated := r["path"].(string); stated && path != "" {
problems = append(problems, fmt.Sprintf(
"%s states both path and place on %v — a stated path IS the placement",
m.Module, r["id"]))
}
if place != placeOwn && place != placeMesh {
problems = append(problems, fmt.Sprintf(
"%s says place %q on %v, and the places are %q — the assignment's own root — and "+
"%q — where the mesh keeps what it writes for the module",
m.Module, place, r["id"], placeOwn, placeMesh))
}
}
seen := map[string]bool{}
refuse := func(id string, where any) {
if declared[id] || seen[id] {
return
}
seen[id] = true
problems = append(problems, fmt.Sprintf(
"%s says ${dir:%s} in %v, and declares no directory %q — a reference the mesh "+
"cannot place would reach the machine as a literal path",
m.Module, id, where, id))
}
for _, r := range m.Resources {
for _, field := range []string{"path", "content"} {
if s, ok := r[field].(string); ok {
for _, id := range referenced(s) {
refuse(id, r["id"])
}
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := r[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok {
for _, id := range referenced(s) {
refuse(id, r["id"])
}
}
}
}
}
if env, ok := r["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok {
for _, id := range referenced(s) {
refuse(id, r["id"])
}
}
}
}
}
maps := map[string]map[string]string{
"receives": m.Receives, "binds": m.Binds, "secrets": m.Secrets,
"own-secrets": m.OwnSecrets.Paths(), "grants": m.Grants,
}
for field, entries := range maps {
for _, value := range entries {
for _, id := range referenced(value) {
refuse(id, field)
}
}
}
for to, locals := range m.SecretsMany {
for _, value := range locals {
for _, id := range referenced(value) {
refuse(id, "secrets."+to)
}
}
}
sort.Strings(problems)
return problems
}
func namesOfDirs(dirs map[string]string) []string {
var names []string
for id := range dirs {
-51
View File
@@ -110,18 +110,6 @@ func TestFillingForOneNodeLeaksIntoNoOther(t *testing.T) {
}
}
func TestAReferenceToNoDirectoryRefusesAtTheManifest(t *testing.T) {
m := placedModule()
m.Resources[2]["volumes"] = []any{"${dir:date}:/data"} // a typo, the likely shape
problems := m.unknownDirRefs()
if len(problems) != 1 || !strings.Contains(problems[0], `${dir:date}`) {
t.Fatalf("a reference naming no directory is a manifest problem; got %v", problems)
}
if got := placedModule().unknownDirRefs(); len(got) != 0 {
t.Fatalf("a correct definition has none; got %v", got)
}
}
func TestAReferenceToNoDirectoryRefusesAtCompositionToo(t *testing.T) {
m := placedModule()
container := shallowCopy(m.Resources[2])
@@ -188,39 +176,6 @@ func TestTheManifestsMapsArePlaced(t *testing.T) {
}
}
func TestAMapReferenceToNoDirectoryRefusesAtTheManifest(t *testing.T) {
m := Manifest{
Module: "photos",
Resources: []map[string]any{{"id": "state", "type": "directory", "place": "."}},
Binds: map[string]string{"route": "${dir:stat}/route.json"},
}
problems := m.unknownDirRefs()
if len(problems) != 1 || !strings.Contains(problems[0], `${dir:stat}`) {
t.Fatalf("a map naming no directory is a manifest problem; got %v", problems)
}
}
func TestPlaceIsValidatedAtTheManifest(t *testing.T) {
both := Manifest{Module: "x", Resources: []map[string]any{
{"id": "d", "type": "directory", "place": ".", "path": "/somewhere"},
}}
if got := both.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], "both path and place") {
t.Fatalf("path beside place refuses; got %v", got)
}
elsewhere := Manifest{Module: "x", Resources: []map[string]any{
{"id": "f", "type": "file", "place": ".", "path": "/somewhere", "content": ""},
}}
if got := elsewhere.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], "not a directory") {
t.Fatalf("place on a file refuses; got %v", got)
}
wrong := Manifest{Module: "x", Resources: []map[string]any{
{"id": "d", "type": "directory", "place": "sub/dir"},
}}
if got := wrong.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], `the places are "."`) {
t.Fatalf("a place that is neither root refuses; got %v", got)
}
}
func TestTheMeshsDirectoryForAModuleIsAPlace(t *testing.T) {
// novox/hq issue 174. What the mesh writes for a module — its bus credential, its bindings —
// is the mesh's plumbing under <root>/mesh/<module>, and the definition names it by id.
@@ -234,9 +189,6 @@ func TestTheMeshsDirectoryForAModuleIsAPlace(t *testing.T) {
OwnSecrets: OwnSecrets{"broker": {Path: "${dir:mesh-state}/broker"}},
Binds: map[string]string{"route": "${dir:state}/route.json"},
}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("place %q is a place; got %v", "mesh", got)
}
dirs := dirsFor(m, Rendering{})
if dirs["mesh-state"] != "/var/lib/mesh/umami" || dirs["state"] != "/var/lib/umami" {
t.Fatalf("the mesh's directory sits beside the module's, not in it; got %v", dirs)
@@ -301,9 +253,6 @@ func TestADirectoryBeneathAPlacedOneIsPlacedWithIt(t *testing.T) {
{"id": "server", "type": "container", "image": "x@sha256:aa",
"volumes": []any{"${dir:runtime-state}:/data"}},
}}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("a path beneath a placed directory is well formed; got %v", got)
}
dirs := dirsFor(m, Rendering{DataRoot: "/srv"})
if dirs["runtime-state"] != "/srv/mesh/gitea/state" {
t.Fatalf("the subdirectory follows the placed one; got %v", dirs)
-47
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"strings"
"testing"
)
@@ -100,52 +99,6 @@ func TestTwoEndpointsOfDifferentShapesAreConfiguredSeparately(t *testing.T) {
}
}
// A route naming an endpoint the module does not declare reaches nothing, and is refused where it is
// written rather than resolving to no port and serving nothing.
func TestARouteNamingAnEndpointTheModuleLacksIsRefused(t *testing.T) {
m := aMediaServer()
m.Contributes["route"][RouteEndpoint] = "absent"
got := strings.Join(RouteProblems(m), "\n")
if !strings.Contains(got, "does not declare") {
t.Fatalf("a route naming an absent endpoint was accepted:\n%s", got)
}
}
// **Two endpoints called the same would make an assignment configure whichever was read last.** The
// point of a name is that it identifies one thing.
func TestTwoEndpointsWithOneNameAreRefused(t *testing.T) {
m := Manifest{Module: "twice", Listens: []Listening{
{Name: "web", Port: 80, From: FromMesh},
{Name: "web", Port: 8080, From: FromMesh},
}}
got := strings.Join(endpointNameProblems(m), "\n")
if !strings.Contains(got, "could mean either") {
t.Fatalf("two endpoints with one name were accepted:\n%s", got)
}
}
// A name that is not a name is refused where it is written: it ends up in something a person types.
func TestAnEndpointNameIsHeldToItsShape(t *testing.T) {
for _, wrong := range []string{"Web", "web port", "3000", "-web", "web_surface"} {
m := Manifest{Module: "odd", Listens: []Listening{{Name: wrong, Port: 80, From: FromMesh}}}
if got := strings.Join(endpointNameProblems(m), "\n"); !strings.Contains(got, "a name is lowercase") {
t.Fatalf("%q was accepted as an endpoint name:\n%s", wrong, got)
}
}
}
// **Every endpoint in the catalogue is unnamed today, and must stay valid.** The word ships one
// release before anything uses it.
func TestAnUnnamedEndpointIsStillValid(t *testing.T) {
m := Manifest{Module: "ordinary", Listens: []Listening{{Port: 443, From: FromEverywhere}}}
if got := endpointNameProblems(m); len(got) != 0 {
t.Fatalf("an unnamed endpoint was refused: %v", got)
}
if got := RouteProblems(m); len(got) != 0 {
t.Fatalf("a module with no route was refused: %v", got)
}
}
// **A route that names an endpoint still carries that endpoint's port.**
//
// Everything downstream reads the port: the provider is told where to reach the consumer, and the
-70
View File
@@ -1,71 +1 @@
package catalogue
import (
"go/ast"
"go/parser"
"go/token"
"path/filepath"
"slices"
"strconv"
"testing"
"git.novox.be/novox/mesh-sdk/go/internal/beside"
)
// The controller refuses a definition's resolved path by the node-engine's own rules, no more (novox/hq issue
// 496): the same lists, read here from mesh-host's internal/apply/placement_guard.go — in a merge check the clone
// beside it at the commit the mesh runs, elsewhere the copy captured in testdata/beside. When the engine's lists
// move, this fails until the controller's move with them, so the gate never refuses what the engine applies nor
// passes what it refuses by path alone.
func TestTheResolvedPathRulesAreTheNodeEnginesOwn(t *testing.T) {
file := filepath.Join(beside.Dir(t, "mesh-host"), "internal", "apply", "placement_guard.go")
parsed, err := parser.ParseFile(token.NewFileSet(), file, nil, 0)
if err != nil {
t.Fatalf("the node-engine's guard does not parse: %v", err)
}
lists := map[string][]string{}
consts := map[string]string{}
ast.Inspect(parsed, func(n ast.Node) bool {
spec, ok := n.(*ast.ValueSpec)
if !ok {
return true
}
for i, name := range spec.Names {
if i >= len(spec.Values) {
continue
}
switch v := spec.Values[i].(type) {
case *ast.CompositeLit:
for _, elt := range v.Elts {
if lit, ok := elt.(*ast.BasicLit); ok && lit.Kind == token.STRING {
s, _ := strconv.Unquote(lit.Value)
lists[name.Name] = append(lists[name.Name], s)
}
}
case *ast.BasicLit:
if v.Kind == token.STRING {
consts[name.Name], _ = strconv.Unquote(v.Value)
}
}
}
return true
})
for name, ours := range map[string][]string{
"protectedRoots": protectedRoots, "forbiddenBelow": forbiddenBelow, "engineTrees": engineTrees,
} {
theirs := lists[name]
if len(theirs) == 0 {
t.Errorf("the node-engine's guard names no %s any more; read it and say where its rule went", name)
continue
}
a, b := slices.Clone(ours), slices.Clone(theirs)
slices.Sort(a)
slices.Sort(b)
if !slices.Equal(a, b) {
t.Errorf("%s differs from the node-engine's:\n controller %v\n node-engine %v", name, a, b)
}
}
if consts["engineModule"] != engineModule {
t.Errorf("the node-engine's own module is %q there and %q here", consts["engineModule"], engineModule)
}
}
-98
View File
@@ -114,64 +114,6 @@ var (
ofContributed = regexp.MustCompile(`\$\{(shell|contribution|backup):([^}]*)\}`)
)
// variableName is a POSIX shell variable name, which is also what environment.d accepts.
var variableName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// environmentProblems is what is wrong with this module's environment contribution, from the
// manifest alone.
func (m Manifest) environmentProblems() []string {
if m.Environment == nil {
return nil
}
var problems []string
for _, n := range sortedKeys(m.Environment.Variables) {
switch {
case !variableName.MatchString(n):
problems = append(problems, fmt.Sprintf(
"%s sets the variable %q, which is not a name a shell accepts: a letter or an "+
"underscore, then letters, digits and underscores", m.Module, n))
continue
case n == "PATH":
// PATH is the one variable every module shares, so no module may set it whole: a second
// setter would replace the first's entries, and the account's own PATH with them.
problems = append(problems, fmt.Sprintf(
"%s sets PATH as a variable; a module adds an entry under environment.path, at the "+
"start or the end, and PATH is composed from every module's (novox/hq ADR 0203)", m.Module))
continue
}
if why := literalProblem(m.Environment.Variables[n]); why != "" {
problems = append(problems, fmt.Sprintf(
"%s sets %s to %q, which %s — %s", m.Module, n, m.Environment.Variables[n], why, literalRule))
}
}
seen := map[string]bool{}
for i, p := range m.Environment.Path {
switch {
case p.Entry == "":
problems = append(problems, fmt.Sprintf("%s's PATH entry %d names no directory", m.Module, i+1))
case strings.Contains(ofMachine.ReplaceAllString(p.Entry, ""), ":"):
// A colon is PATH's own separator, so an entry holding one is two entries, and the
// check that it is already present would look for the wrong thing.
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which holds a colon, PATH's own separator", m.Module, p.Entry))
case seen[p.Entry]:
problems = append(problems, fmt.Sprintf("%s puts %q on PATH twice", m.Module, p.Entry))
default:
if why := literalProblem(p.Entry); why != "" {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which %s — %s", m.Module, p.Entry, why, literalRule))
}
}
seen[p.Entry] = true
if p.At != PathAtStart && p.At != PathAtEnd {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH at %q; an entry goes at %q or %q of the account's PATH",
m.Module, p.Entry, p.At, PathAtStart, PathAtEnd))
}
}
return problems
}
// literalRule is why a value must be literal, said with every refusal of one.
const literalRule = "a value is literal, so a POSIX shell and the service manager read it alike, and " +
"names the machine only through the mesh's own ${machine:…} facts (novox/hq ADR 0203)"
@@ -195,46 +137,6 @@ func literalProblem(v string) string {
return ""
}
// shellProblems is what is wrong with this module's shell code, from the manifest alone. The code
// itself is not judged: it is the shell's syntax, which the controller does not read.
func (m Manifest) shellProblems() []string {
var problems []string
for i, c := range m.Shell {
if !oneOf(contributionTargets(), c.For) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d is for %q; the shells are %s, the session's files %s, and the power "+
"moments %s", m.Module, i+1, c.For, strings.Join(knownShells, ", "),
strings.Join(sessionFiles, ", "), strings.Join(powerMoments, ", ")))
}
if !oneOf(knownSlots, c.Slot) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d goes in the slot %q; the slots are %s", m.Module, i+1, c.Slot,
strings.Join(knownSlots, ", ")))
}
if strings.TrimSpace(c.Code) == "" {
problems = append(problems, fmt.Sprintf("%s's shell code %d has no code", m.Module, i+1))
}
}
return problems
}
// contributionPlaceholderProblems is every place this module's resources name the environment or
// the shell's code and may not — judged from the manifest, so the catalogue check refuses it before
// a mesh does, and again at composition in the same words.
func (m Manifest) contributionPlaceholderProblems() []string {
var problems []string
for _, r := range m.Resources {
problems = append(problems, placeholderProblems(m, r)...)
problems = append(problems, seatPlaceholderProblems(m, r)...)
problems = append(problems, collectedPlaceholderProblems(m, r)...)
problems = append(problems, consumersPlaceholderProblems(m, r)...)
// And every placeholder no pass fills where it stands: a misspelt namespace, a key its
// namespace cannot take, or a field its pass does not read (novox/hq issue 231).
problems = append(problems, unconsumedPlaceholders(m.Module, r)...)
}
return problems
}
// placeholderProblems is what is wrong with one resource's ${environment:…} and ${shell:…}.
//
// **The seat authorises it, not the placeholder** (novox/hq ADR 0203 §5, ADR 0204 §3), as the seat
-122
View File
@@ -276,102 +276,6 @@ func TestTheEnvironmentHolderPlacesBothRenderings(t *testing.T) {
}
}
// ADR 0203 §5 and ADR 0204 §3: a placeholder outside the seat's holder is refused — by the parser,
// which is what the catalogue check and registration run, and again at composition, in the same words.
func TestAPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:posix}", "toolchain's resource rc names ${environment:posix} and toolchain does not claim node-environment"},
{"${shell:zsh:normal}", "toolchain's resource rc names ${shell:zsh:normal} and toolchain does not claim node-login-shell"},
} {
raw := `{"module":"toolchain","resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("the catalogue check accepted %s outside its holder: %v", c.content, err)
}
m := Manifest{Module: "toolchain", Resources: []map[string]any{
{"id": "rc", "type": "file", "path": "/etc/rc", "content": c.content}}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{m}}
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("composition accepted %s outside its holder: %v", c.content, err)
}
}
}
// A key nobody renders is refused, not left in the file as a literal.
func TestAnUnknownPlaceholderKeyIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:foo}", "names ${environment:foo}; the environment is ${environment:posix} or ${environment:systemd}"},
{"${shell:zsh:middle}", "names ${shell:zsh:middle}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:tcsh:first}", "names ${shell:tcsh:first}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:zsh}", "names ${shell:zsh}; shell code is ${shell:<shell>:<slot>}"},
} {
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"},{"name":"node-login-shell","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s was accepted: %v", c.content, err)
}
}
// And outside a file's content, where nothing could be placed.
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/${environment:posix}","content":"x"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil ||
!strings.Contains(err.Error(), "names ${environment:posix} in its path; the environment and the shell's code are placed only in a file's content") {
t.Errorf("a placeholder in a path was accepted: %v", err)
}
}
// What ADR 0203 §2 allows a contribution to say, refused at parse when it says anything else.
func TestAMalformedEnvironmentIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ environment, want string }{
{`{"variables":{"1X":"a"}}`, `tool sets the variable "1X", which is not a name a shell accepts`},
{`{"variables":{"MY-VAR":"a"}}`, `tool sets the variable "MY-VAR", which is not a name a shell accepts`},
{`{"variables":{"PATH":"/bin"}}`, `tool sets PATH as a variable; a module adds an entry under environment.path`},
{`{"variables":{"A":"$HOME/x"}}`, `tool sets A to "$HOME/x", which holds a $ that is not one of the machine's ${machine:…} facts`},
{`{"variables":{"A":"${HOME}/x"}}`, `tool sets A to "${HOME}/x", which holds a $`},
{`{"variables":{"A":"it's"}}`, `tool sets A to "it's", which holds a quote`},
{`{"variables":{"A":"say \"hi\""}}`, `which holds a quote`},
{`{"variables":{"A":"a\\b"}}`, `which holds a backslash`},
{`{"variables":{"A":"a\nb"}}`, `which holds a line break`},
{`{"variables":{"A":"a\u0000b"}}`, `which holds a NUL`},
{`{"path":[{"entry":"","at":"start"}]}`, `tool's PATH entry 1 names no directory`},
{`{"path":[{"entry":"/a:/b","at":"start"}]}`, `tool puts "/a:/b" on PATH, which holds a colon`},
{`{"path":[{"entry":"$HOME/bin","at":"start"}]}`, `tool puts "$HOME/bin" on PATH, which holds a $`},
{`{"path":[{"entry":"/a","at":"middle"}]}`, `tool puts "/a" on PATH at "middle"; an entry goes at "start" or "end"`},
{`{"path":[{"entry":"/a"}]}`, `tool puts "/a" on PATH at ""`},
{`{"path":[{"entry":"/a","at":"start"},{"entry":"/a","at":"end"}]}`, `tool puts "/a" on PATH twice`},
{`{"variables":{"A":"x"},"paths":[]}`, `unknown field "paths"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","environment":` + c.environment + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.environment, c.want, err)
}
}
// What is allowed: a literal, and the machine's own facts.
if _, err := ParseManifest([]byte(`{"module":"tool","environment":{` +
`"variables":{"GOPATH":"${machine:account-home}/go","DISABLE_X":"1","ANSWER":"a b+c=d"},` +
`"path":[{"entry":"${machine:account-home}/go/bin","at":"start"},{"entry":"/opt/x","at":"end"}]}}`)); err != nil {
t.Fatalf("a well-formed environment was refused: %v", err)
}
}
func TestMalformedShellCodeIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ shell, want string }{
{`[{"for":"tcsh","slot":"normal","code":"x"}]`, `tool's shell code 1 is for "tcsh"; the shells are zsh, bash, fish`},
{`[{"for":"zsh","slot":"middle","code":"x"}]`, `tool's shell code 1 goes in the slot "middle"; the slots are first, normal, last`},
{`[{"for":"zsh","slot":"last","code":"x"},{"for":"zsh","slot":"last","code":" \n"}]`, `tool's shell code 2 has no code`},
{`[{"for":"zsh","slot":"last","code":"x","order":1}]`, `unknown field "order"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","shell":` + c.shell + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.shell, c.want, err)
}
}
// The code itself is never judged: a shell's own `${…}` is not the mesh's.
if _, err := ParseManifest([]byte(`{"module":"tool","shell":[{"for":"zsh","slot":"first",` +
`"code":"source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\""}]}`)); err != nil {
t.Fatalf("shell code was judged as if it were the mesh's: %v", err)
}
}
// ADR 0203 §1 and ADR 0204 §1: both seats are the mesh's own, held once per machine; the login
// shell's contract is `execute`, described and with a schema an agent can call.
func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
@@ -396,29 +300,3 @@ func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
t.Fatalf("execute takes no timeout: %v", props)
}
}
// ADR 0204 §1: the login shell is the mesh's, so no module declares it — neither under the mesh's
// name nor under the name a module gave it before.
func TestNoModuleMayDeclareTheLoginShell(t *testing.T) {
for _, n := range []string{"login-shell", "node-login-shell", "node-environment"} {
raw := `{"module":"zsh","seats":[{"name":"` + n + `","scope":"node","serves":["execute"]}],"tools":["execute"]}`
_, err := ParseManifest([]byte(raw))
if err == nil {
t.Errorf("a module declaring %q was accepted", n)
}
}
got := strings.Join(declaredSeatProblems(Manifest{Module: "zsh",
DefinesSeats: []SeatDeclaration{{Name: "login-shell", Scope: ScopeNode}}}), "; ")
if !strings.Contains(got, `zsh declares a seat named "login-shell"; the login shell is the mesh's own seat node-login-shell`) {
t.Fatalf("declaring login-shell was not refused by name: %q", got)
}
// And a shell module claiming the mesh's seat, serving execute, is what the seat is for.
m, err := ParseManifest([]byte(`{"module":"zsh","tools":["execute"],` +
`"claims":[{"name":"node-login-shell","scope":"node"}]}`))
if err != nil {
t.Fatal(err)
}
if err := CanHold(m, Seat{Name: LoginShellSeat, Scope: ScopeNode, Serves: loginShellVerbs()}); err != nil {
t.Fatalf("a shell module claiming the seat cannot hold it: %v", err)
}
}
+1 -25
View File
@@ -129,31 +129,7 @@ func TestTheEnvironmentOfTheCatalogueIsTheSameHandedOrNot(t *testing.T) {
}
}
func TestAnEnvironmentItemIsTypedAndNamesWhatItNeeds(t *testing.T) {
item := func(attrs map[string]string) []string {
return environmentItemProblems("x", Item{Collection: "environment", Attributes: attrs})
}
for _, ok := range []map[string]string{
{"kind": "variable", "name": "GTK_THEME"},
{"kind": "path", "at": "end", "order": "10"},
} {
if p := item(ok); len(p) != 0 {
t.Errorf("%v is refused: %v", ok, p)
}
}
for _, bad := range []map[string]string{
nil, {"kind": "other"}, {"kind": "variable"}, {"kind": "variable", "name": "PATH"},
{"kind": "path"}, {"kind": "path", "at": "middle"}, {"kind": "path", "at": "start", "name": "X"},
{"kind": "variable", "name": "A", "color": "red"},
} {
if p := item(bad); len(p) == 0 {
t.Errorf("%v is taken", bad)
}
}
if !attributeFits("environment", "name", "GTK_THEME") || attributeFits("shell:zsh:normal", "name", "GTK_THEME") ||
attributeFits("environment", "kind", "Variable") {
t.Error("an upper-case name is the environment's name alone")
}
func TestTwoModulesSettingOneVariableAreRefused(t *testing.T) {
// A module that sets a variable twice, in two modules, is refused as the old form refused it.
a := Manifest{Module: "a", Items: []Item{{Collection: "environment", Content: "1\n",
Attributes: map[string]string{"kind": "variable", "name": "X"}}}}
-127
View File
@@ -1,10 +1,8 @@
package catalogue
import (
"fmt"
"regexp"
"slices"
"strings"
)
// What a module may call an event, and what a consumer may ask for.
@@ -36,131 +34,6 @@ const (
TheRest = "**"
)
// EventProblems is what is wrong with a manifest's events.
//
// Refused at registration, because the alternative is a module that installs, starts, connects and
// reacts to nothing — and every log line says it is fine.
func EventProblems(m Manifest) []string {
var problems []string
for _, e := range m.Emits {
if was, stale := staleEventForm(e, m.Module); stale {
problems = append(problems, fmt.Sprintf(
"%s emits %q, which is the old bus's routing key. An event is named locally now, so "+
"write %q — the mesh derives the subject (novox/hq design 29 §1)",
m.Module, was, strings.TrimPrefix(was, "module."+m.Module+".")))
continue
}
if strings.HasPrefix(e, "module.") {
problems = append(problems, fmt.Sprintf(
"%s emits %q: `module.` is reserved, because it is how the old bus spelled a "+
"routing key and an event named that way derives into a namespace nobody owns",
m.Module, e))
continue
}
if err := localName(e); err != nil {
problems = append(problems, fmt.Sprintf("%s emits %q: %v", m.Module, e, err))
continue
}
// **Its own name, never another's.** The bus enforces that a namespace belongs to the module
// it is named for, so an event named for somebody else cannot be published at all. If the
// event is about a role rather than about this module, it belongs on the seat: a name that
// is stable across whoever fills it (04-ISSUES/127).
if first, _, split := strings.Cut(e, "."); split && isAModuleNameOtherThan(first, m.Module) {
problems = append(problems, fmt.Sprintf(
"%s emits %q, which reads as another module's event. A module publishes under its "+
"own name only. If this is about a role rather than about %s, declare it on that "+
"seat, where the name survives the holder changing",
m.Module, e, m.Module))
}
}
for _, c := range m.Consumes {
if strings.HasPrefix(c, "module.") {
problems = append(problems, fmt.Sprintf(
"%s consumes %q, which is the old bus's pattern. A consumed event names its emitter "+
"and the event: write %q", m.Module, c, strings.TrimPrefix(c, "module.")))
continue
}
if c == "#" {
problems = append(problems, fmt.Sprintf(
"%s consumes %q, which is the old bus's wildcard for everything. Write %q",
m.Module, c, TheRest))
continue
}
if err := consumePattern(c); err != nil {
problems = append(problems, fmt.Sprintf("%s consumes %q: %v", m.Module, c, err))
}
}
return problems
}
// staleEventForm says an emitted name is this module's own old routing key, and what it was.
func staleEventForm(event, module string) (string, bool) {
return event, module != "" && strings.HasPrefix(event, "module."+module+".")
}
// isAModuleNameOtherThan says a first token names some module of this mesh that is not this one.
//
// Only the mesh's own seats and the catalogue could answer this properly, and neither is reachable
// from a parser given one manifest. So this catches the case that actually happened — a name that
// is a *provision* the mesh defines, which is where "another module's event" comes from in practice
// — and the whole-catalogue check catches the rest.
func isAModuleNameOtherThan(first, module string) bool {
if first == module || first == "" {
return false
}
if _, isASeat := SeatNamed(first); isASeat {
return true
}
if _, isASeat := SeatDelivering(first); isASeat {
return true
}
return false
}
// localName checks one event name: dot-separated names, no wildcards, nothing else.
func localName(event string) error {
if event == "" {
return fmt.Errorf("an event needs a name")
}
for _, part := range strings.Split(event, ".") {
if part == OneName || part == TheRest {
return fmt.Errorf("an emitted event names one event, so it carries no wildcard")
}
if !eventName.MatchString(part) {
return fmt.Errorf("%q is not a usable name: lower-case letters, digits and dashes", part)
}
}
return nil
}
// consumePattern checks a consumed pattern: the emitter, then the event, with wildcards.
func consumePattern(pattern string) error {
if pattern == "" {
return fmt.Errorf("a consumed event needs an emitter and an event")
}
parts := strings.Split(pattern, ".")
for i, part := range parts {
switch {
case part == TheRest:
if i != len(parts)-1 {
return fmt.Errorf("%q stands for the rest of a name, so nothing may follow it", TheRest)
}
case part == OneName:
case !eventName.MatchString(part):
return fmt.Errorf("%q is not a usable name: lower-case letters, digits and dashes", part)
}
}
// `**` alone is every event from every module, which the audit logger wants and says plainly.
if len(parts) == 1 && parts[0] != TheRest {
return fmt.Errorf(
"%q names an emitter and no event. Write <emitter>.<event>, or %q for every event",
pattern, TheRest)
}
return nil
}
// The events a provider says about its consumers (novox/hq ADR 0224): a consumer it has failed
// without one success for minutes, and that consumer succeeding again or being withdrawn. The
// controller follows them from every module and `status` names a consumer failing until it recovers.
-10
View File
@@ -98,16 +98,6 @@ func filesVerbs() []Verb {
)
}
// FilesHolderProblems is what a holder of node-files is held to (novox/hq ADR 0394 §2): it runs as the tool
// runner's account, the node's operator account, so it declares no runs-as.
func FilesHolderProblems(m Manifest) []string {
if !m.ClaimsSeat(FilesSeat) || m.RunsAs == "" {
return nil
}
return []string{fmt.Sprintf("%s claims %s and declares runs-as %q: its holder runs as the tool runner's account, "+
"the node's operator account (novox/hq ADR 0394 §2)", m.Module, FilesSeat, m.RunsAs)}
}
// ownedPaths is every path a module owns on this node, as the holder is told (novox/hq ADR 0394 §5): the
// paths every seat declares (ADR 0287) with the module holding the seat here, and the paths and files each
// module's manifest places (declaredPaths). A seat's path wins where both name one.
-14
View File
@@ -46,20 +46,6 @@ func TestNodeFilesIsANodeSeatWithEveryVerbOfADR0394(t *testing.T) {
}
}
// issue 900, ADR 0394 §2: a holder of node-files runs as the tool runner's account, so it declares no runs-as.
func TestAHolderOfNodeFilesDeclaringRunsAsIsRefused(t *testing.T) {
p := FilesHolderProblems(Manifest{Module: "files", RunsAs: "mesh-files", Claims: []Claim{{Name: FilesSeat}}})
if len(p) != 1 || !strings.Contains(p[0], "0394") || !strings.Contains(p[0], "runs-as") {
t.Errorf("a holder declaring runs-as was not refused: %v", p)
}
if p := FilesHolderProblems(Manifest{Module: "files", Claims: []Claim{{Name: FilesSeat}}}); len(p) != 0 {
t.Errorf("a holder declaring none was refused: %v", p)
}
if p := FilesHolderProblems(Manifest{Module: "other", RunsAs: "x"}); len(p) != 0 {
t.Errorf("a non-holder was judged: %v", p)
}
}
// issue 900, ADR 0394 §4: the holder is told the secret places and the paths modules own on its node.
func TestTheHolderOfNodeFilesIsToldSecretPlacesAndOwnedPaths(t *testing.T) {
files := Manifest{Module: "files", Version: "1", Claims: []Claim{{Name: FilesSeat}},
-26
View File
@@ -1000,32 +1000,6 @@ func Reaches(m Manifest, layers []Layer) (map[int]string, error) {
// route that names the endpoint says what it means, and the mesh looks the port up.
const RouteEndpoint = "endpoint"
// RouteProblems holds a module's route contributions to naming an endpoint it actually has.
//
// A route naming an endpoint the module does not declare reaches nothing, and is refused where it is
// written rather than resolving to no port and serving nothing — the fault this repository names most
// often, a declaration that reads as though it did something.
func RouteProblems(m Manifest) []string {
var problems []string
check := func(where string, values map[string]any) {
name, ok := values[RouteEndpoint].(string)
if !ok || strings.TrimSpace(name) == "" {
return
}
if _, found := EndpointPort(m, name); !found {
problems = append(problems, fmt.Sprintf(
"%s routes %s to the endpoint %q, which it does not declare", m.Module, where, name))
}
}
if values, ok := m.Contributes["route"]; ok {
check("a name", values)
}
for local, values := range m.ContributesMany["route"] {
check(local, values)
}
return problems
}
// EndpointsSetting is the settings key that configures a module's endpoints by name, per node
// (novox/hq ADR 0138):
//
+1 -39
View File
@@ -7,22 +7,9 @@ import (
"testing"
)
// A manifest key nothing reads is worse than a key that is refused: five HAL manifests carried
// `scope:`, which read as a restriction and restricted nothing (novox/hq 04-ISSUES/003). The host's
// declaration parser has always refused unknown keys; manifests never did.
func TestAManifestKeyNothingReadsIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"web","scope":"mesh"}`))
if err == nil {
t.Fatal("a manifest with a key nothing reads was accepted, which is how scope: survived")
}
if !strings.Contains(err.Error(), "scope") {
t.Fatalf("refused without naming the key, which leaves the author guessing: %v", err)
}
}
// The same discipline must not refuse a manifest that is merely ordinary.
func TestAnOrdinaryManifestIsStillAccepted(t *testing.T) {
m, err := ParseManifest([]byte(
m, err := decodeForTest([]byte(
`{"module":"web","listens":[{"port":443,"from":"anywhere","why":"the site"}]}`))
if err != nil {
t.Fatalf("an ordinary manifest was refused: %v", err)
@@ -32,24 +19,6 @@ func TestAnOrdinaryManifestIsStillAccepted(t *testing.T) {
}
}
// A rule with no source is open, and must say so rather than appear to restrict something.
func TestAPortWithNoSourceIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"web","listens":[{"port":443}]}`))
if err == nil {
t.Fatal("a port that never said who may reach it was accepted; it reads as a restriction")
}
if !strings.Contains(err.Error(), "from where") {
t.Fatalf("refused without saying what is missing: %v", err)
}
}
func TestASourceNobodyDefinedIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"web","listens":[{"port":443,"from":"lan"}]}`))
if err == nil {
t.Fatal("a source the mesh cannot render was accepted, and would be silently dropped")
}
}
// The derivation is over the whole node, not one module.
func TestTheRuleSetIsEveryAssignedModulesPorts(t *testing.T) {
r := Resolution{Modules: []Manifest{
@@ -357,13 +326,6 @@ func TestTheModuleAskingForTheRuleSetGetsEveryModulesPorts(t *testing.T) {
}
}
func TestAskingForTheRuleSetWithNowhereToPutItIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"firewall","filtering":{"into":""}}`))
if err == nil {
t.Fatal("a module asked for the rule set and named no path; it would receive nothing")
}
}
// nftables matches the two address families separately, and one set holding both is a syntax
// error — so the file fails to load, the service reports a fault, and the machine filters nothing.
func TestAMeshOnBothAddressFamiliesRendersBoth(t *testing.T) {
-31
View File
@@ -1,37 +1,6 @@
package catalogue
import "testing"
// A bundle is the module's own directory compiled whole, and naming a source would be describing its
// own build. That holds for an interpreted language and cannot hold for a compiled one: a repository
// written in Go carries several commands — the host and its bootstrap live in one — and "the module's
// own directory" is then not a package at all (novox/hq 04-ISSUES/142).
func TestAGoBundleMayNameItsCommand(t *testing.T) {
b := &Build{Artifacts: []Artifact{{
Name: "host-arch", Kind: ArtifactBundle, Language: "go", System: "arch",
From: "cmd/mesh-host",
}}}
if p := b.problems("mesh-host"); len(p) != 0 {
t.Fatalf("a go bundle naming its command was refused: %v", p)
}
}
func TestAnInterpretedBundleStillMayNotNameASource(t *testing.T) {
b := &Build{Artifacts: []Artifact{{
Name: "tools", Kind: ArtifactBundle, Language: "typescript", From: "src",
}}}
p := b.problems("something")
if len(p) == 0 {
t.Fatal("an interpreted bundle naming what it is built from was accepted")
}
}
func TestACompiledBundleStillMustSayItsSystem(t *testing.T) {
b := &Build{Artifacts: []Artifact{{
Name: "host", Kind: ArtifactBundle, Language: "go", From: "cmd/mesh-host",
}}}
if len(b.problems("mesh-host")) == 0 {
t.Fatal("a compiled bundle with no system was accepted")
}
}
-85
View File
@@ -61,14 +61,6 @@ func TestTheGraphicalSessionsSeatsAreTheMeshsOwnWithTheirVerbs(t *testing.T) {
}
}
// §2: a module may claim one of them, and may not declare it as its own.
func TestNoModuleMayDeclareAGraphicalSessionSeat(t *testing.T) {
raw := `{"module":"xorg","seats":[{"name":"node-display-server","scope":"node"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "mesh's own namespace") {
t.Fatalf("a module declared node-display-server as its own: %v", err)
}
}
func displayServer(name, display string, claims ...string) Manifest {
m := Manifest{Module: name, Provides: []Offer{{Name: display, Reach: ReachMachine}}}
for _, c := range claims {
@@ -126,29 +118,6 @@ func TestAMachineReachRequirementIsNotPulledInNorAnsweredFromAnotherNode(t *test
}
}
func TestTheMachinesReachIsAProvisionsOnlyReachAndIsNodeScoped(t *testing.T) {
for _, c := range []struct{ provides, want string }{
{`{"name":"x11-display","reach":"internal"}`, `with reach "internal"; a provision's reach is "machine" or nothing`},
{`{"name":"x11-display","scope":"mesh","reach":"machine"}`, `at scope "mesh" with the machine's reach`},
} {
_, err := ParseManifest([]byte(`{"module":"xorg","provides":[` + c.provides + `]}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.provides, c.want, err)
}
}
m, err := ParseManifest([]byte(`{"module":"xorg","provides":[{"name":"x11-display","reach":"machine"}]}`))
if err != nil {
t.Fatal(err)
}
if !m.Provides[0].MachineReach() {
t.Fatal("the reach was not read")
}
back, _ := json.Marshal(m.Provides[0])
if string(back) != `{"name":"x11-display","reach":"machine"}` {
t.Errorf("written back as %s", back)
}
}
func TestACatalogueDisagreeingAboutAProvisionsReachIsRefused(t *testing.T) {
cat := shelf(windowManager(), displayServer("xorg", "x11-display"),
Manifest{Module: "fake-x", Provides: Offers("x11-display")})
@@ -157,57 +126,3 @@ func TestACatalogueDisagreeingAboutAProvisionsReachIsRefused(t *testing.T) {
t.Fatalf("a provision with and without the machine's reach gave %v", err)
}
}
// §4: xinitrc and xresources slots, placed by the display server's holder alone.
func TestTheSessionsFilesArePlacedByTheDisplayServersHolderAlone(t *testing.T) {
xorg := Manifest{Module: "xorg", Claims: []Claim{{Name: DisplayServerSeat}},
Shell: []ShellCode{{For: "xinitrc", Slot: "first", Code: "xset s off"}},
Resources: []map[string]any{
{"id": "xinitrc", "type": "file", "path": "/home/op/.xinitrc",
"content": "${shell:xinitrc:first}${shell:xinitrc:normal}${shell:xinitrc:last}"},
{"id": "xresources", "type": "file", "path": "/home/op/.Xresources",
"content": "${shell:xresources:normal}"},
}}
i3 := Manifest{Module: "i3", Shell: []ShellCode{{For: "xinitrc", Slot: "last", Code: "exec i3"}}}
theme := Manifest{Module: "theme", Shell: []ShellCode{
{For: "xresources", Slot: "normal", Code: "Xft.dpi: 96"},
{For: "zsh", Slot: "normal", Code: "not for the session"},
}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{xorg, i3, theme}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
by := map[string]any{}
for _, res := range out {
by[res["id"].(string)] = res["content"]
}
if got := by["xorg.xinitrc"]; got != "# xorg\nxset s off\n# i3\nexec i3\n" {
t.Errorf("the .xinitrc is %q", got)
}
if got := by["xorg.xresources"]; got != "# theme\nXft.dpi: 96\n" {
t.Errorf("the .Xresources is %q", got)
}
// The contributions parse; the placeholders parse only in the holder.
if _, err := ParseManifest([]byte(`{"module":"i3","shell":[{"for":"xinitrc","slot":"last","code":"exec i3"},` +
`{"for":"xresources","slot":"normal","code":"i3.font: x"}]}`)); err != nil {
t.Fatalf("a session contribution was refused: %v", err)
}
for _, c := range []struct{ claims, content, want string }{
{``, "${shell:xinitrc:normal}", "does not claim node-display-server; every module's xinitrc is placed by the display server's holder alone"},
{`{"name":"node-login-shell"}`, "${shell:xresources:normal}", "does not claim node-display-server"},
{`{"name":"node-display-server"}`, "${shell:zsh:normal}", "does not claim node-login-shell"},
{`{"name":"node-display-server"}`, "${shell:xsession:normal}", "the session's file one of xinitrc, xresources"},
} {
raw := `{"module":"holder","claims":[` + c.claims + `],"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` +
c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s with claims [%s]: want %q, got %v", c.content, c.claims, c.want, err)
}
}
if _, err := ParseManifest([]byte(`{"module":"xorg","claims":[{"name":"node-display-server"}],` +
`"resources":[{"id":"rc","type":"file","path":"/home/op/.xinitrc","content":"${shell:xinitrc:last}"}]}`)); err != nil {
t.Errorf("the display server's holder could not place the session's slots: %v", err)
}
}
-20
View File
@@ -58,26 +58,6 @@ func TestAGroupThatIsNotAWholeNumberAboveZeroIsRefused(t *testing.T) {
}
}
// A manifest in the config form registers with the group a config value fills, and the value is placed (a container's
// groups count as a place for it); a placeholder in any other shape is refused where the manifest is read.
func TestAManifestWhoseContainerTakesItsGroupFromConfigRegisters(t *testing.T) {
m := map[string]any{
"module": "n8n", "version": "1",
"config": map[string]any{"media-gid": map[string]any{"type": "number", "minimum": 1, "required": true,
"description": "the group that owns the media library on this machine"}},
"resources": []any{aContainerWithGroups("${config:media-gid}")},
}
if _, err := parseRaw(t, m); err != nil {
t.Fatalf("refused: %v", err)
}
for _, shape := range []string{"x${config:media-gid}", "${config:media-gid}:1", "${config:other}"} {
m["resources"] = []any{aContainerWithGroups(shape)}
if _, err := parseRaw(t, m); err == nil {
t.Errorf("groups %q registered", shape)
}
}
}
// 1636/AC3 composed for a machine, the way a push writes it: n8n's group is the number the machine's config value
// says, written as a number, and the module is left out by name when the machine says none.
func TestAComposedMachineGetsTheGroupTheConfigValueNames(t *testing.T) {
-37
View File
@@ -305,43 +305,6 @@ func takesOf(m Manifest, id string) string {
return takes
}
// movedProblems is what is wrong with a module's mark: a moved module must name a template (as written) or carry
// what its build kept (as held), and Handed is the build's word, never a repository's.
func movedProblems(m Manifest) []string {
switch {
case m.Handed != nil && !m.Moved:
return []string{fmt.Sprintf("%s carries what a moved module's build keeps (handed) and is not marked moved",
m.Module)}
case m.Handed != nil && m.Build != nil:
return []string{fmt.Sprintf("%s says handed in its repository: that is what its build keeps, never "+
"written by hand (novox/hq ADR 0349 §2)", m.Module)}
case m.Moved && m.Handed == nil && !m.Templated():
return []string{fmt.Sprintf("%s is marked moved and names no template: a moved module's files are filled "+
"from template files of its own, `\"template\": \"%s/<name>\"` (novox/hq ADR 0349 §2)", m.Module,
m.TemplatesDir())}
}
if !m.Moved {
return nil
}
// **A moved module's secrets are the vault's, handed to its machine's client** (design 57 V4): a secret the
// mesh makes, by name. A family's members and a value issued outside the mesh are given by a person and may be
// absent, which the client cannot tell from one the vault does not hold yet; they move with a package of their
// own.
var problems []string
if fams := m.OwnSecrets.Families(); len(fams) > 0 {
problems = append(problems, fmt.Sprintf("%s is marked moved and keeps a secret family (%s): a moved module's "+
"secrets are handed by the vault by name, and a family's members are not yet (design 57 V4)", m.Module,
strings.Join(fams, ", ")))
}
for _, name := range sortedKeys(m.OwnSecrets.Plain()) {
if m.OwnSecrets[name].IssuedBy == IssuedOutside {
problems = append(problems, fmt.Sprintf("%s is marked moved and its secret %s is issued outside the mesh: "+
"a value that may not be given is not yet handed by the vault (design 57 V4)", m.Module, name))
}
}
return problems
}
// handedBy is what a moved module's build keeps, from its templates archive as built, or nil for a module not
// moved.
func handedBy(m Manifest, archive Built) *Handed {
+1 -32
View File
@@ -1,8 +1,6 @@
package catalogue
import (
"os"
"path/filepath"
"strings"
"testing"
)
@@ -50,7 +48,7 @@ func TestWhatAConsumersPlaceholderMayNotBeIsRefused(t *testing.T) {
"a file the program keeps": {strings.Replace(handedProvider, `"mode":"0600","template"`,
`"mode":"0600","create-once":true,"template"`, 1), consumersText, "owned"},
} {
m, err := ParseManifest([]byte(c.raw))
m, err := decodeForTest([]byte(c.raw))
if err != nil {
t.Fatalf("%s: %v", why, err)
}
@@ -103,32 +101,3 @@ func TestAProvidersConsumersFileIsLeftOutWhereItIsNotHanded(t *testing.T) {
t.Fatalf("handed: said %q, filled %v", c.HandedOnly, c.Filled)
}
}
// What `module check` and the merge gate judge: the provider read from its tree, its text inline. Judged by the same
// rules, a provider naming a provision it does not provide is refused there too, and one that does composes.
func TestAProviderReadFromItsTreeIsJudgedByTheSameRules(t *testing.T) {
read := func(text string) (Manifest, error) {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "files", "consumers.json"), []byte(text), 0o644); err != nil {
t.Fatal(err)
}
m, err := ParseManifest([]byte(handedProvider))
if err != nil {
t.Fatal(err)
}
return WithTemplatesFrom(m, dir)
}
filled, err := read(consumersText)
if err != nil {
t.Fatalf("a provider read from its tree is refused: %v", err)
}
if _, err := notifierOn(t, filled).Compose(handingTo("store:5000")); err != nil {
t.Fatalf("a provider read from its tree does not compose where it would be handed: %v", err)
}
if _, err := read("${consumers:cache}"); err == nil || !strings.Contains(err.Error(), "does not provide") {
t.Fatalf("a provider read from its tree naming a provision it does not provide: %v", err)
}
}
+1 -63
View File
@@ -116,68 +116,6 @@ func TestAHandedModulesIssuedCredentialIsItsProvisionsOneEntry(t *testing.T) {
}
}
// Each entry names the credential form the provider's offer declares (issue 768), from the provider's manifest, so
// the vault makes a first value in that form for a provider it holds no row of: one of its own secrets shared by
// every consumer, a login per consumer when the offer says no credential, issued by its software, or given by a
// person (the offer's own secret issued outside).
func TestAHandedModulesPlanNamesTheCredentialFormOfItsProvidersOffer(t *testing.T) {
storeOffering := func(credential string) Manifest {
raw := `{"module":"store","version":"1","provides":[{"name":"postgres-database","scope":"mesh"` +
credential + `}],"serves":{"postgres-database":{"port":5432}}`
if strings.Contains(credential, `"own"`) {
raw += `,"own-secrets":{"admin":{"path":"/var/lib/store/admin","taken":"at-start"},` +
`"token":{"path":"/var/lib/store/token","taken":"at-start","issued-by":"outside"}}`
}
return mustParse(t, raw+`}`)
}
for credential, want := range map[string]string{
// No default (the lead, 2026-10-29): an offer that says nothing has no form, and the catalogue refuses it.
``: "",
`,"credential":{"form":"pair"}`: CredentialFormPair,
`,"credential":{"form":"none"}`: CredentialFormNone,
`,"credential":{"own":"admin"}`: CredentialFormShared,
`,"credential":{"own":"token"}`: CredentialFormGiven,
`,"credential":{"issued-by":"outside"}`: CredentialFormGiven,
`,"credential":{"issued-by":"software","tool":"store_issued_token"}`: CredentialFormIssued,
} {
node := workstation()
node.At = "workstation.internal"
consumer := notifierConsumer(t, true, false)
r, err := Resolve(shelf(consumer, storeOffering(credential)), []string{consumer.Module}, node, twoStores("anchor"))
if err != nil {
t.Fatal(err)
}
r.Account = "op"
c, _ := composeWith(t, r, handingTo("store:5000"))
got := c.Plans["notify"].Provisions
if len(got) != 1 || got[0].CredentialForm != want {
t.Errorf("an offer with %q: the plan names %+v, want the form %q", credential, got, want)
}
// Every form a plan names is in the one list each reader checks (issue 774); an undeclared one is not.
if IsCredentialForm(want) != (want != "") {
t.Errorf("the form %q of an offer with %q: in CredentialForms %v", want, credential, IsCredentialForm(want))
}
}
if forms := strings.Join(CredentialForms(), ","); forms != "shared,pair,issued,given,none" || IsCredentialForm("maybe") {
t.Errorf("the credential forms are %s, and maybe is one: %v", forms, IsCredentialForm("maybe"))
}
if got := (Manifest{Module: "x"}).CredentialFormOf("postgres-database"); got != "" {
t.Errorf("a module offering nothing has the credential form %q", got)
}
// The catalogue refuses an offer saying no form, and a form beside an own secret or not pair or none.
for credential, refused := range map[string]string{``: "says no credential form", `,"credential":{}`: "says no credential form",
`,"credential":{"form":"pair","own":"admin"}`: "beside an own secret", `,"credential":{"form":"maybe"}`: `not "pair" or "none"`,
`,"credential":{"form":"none"}`: ""} {
raw := `{"module":"store","version":"1","provides":[{"name":"postgres-database","scope":"mesh"` + credential + `}]}`
var m Manifest
_ = json.Unmarshal([]byte(raw), &m)
problems := strings.Join(CredentialFormProblems(m), "; ")
if (refused == "") != (problems == "") || !strings.Contains(problems, refused) {
t.Errorf("%q: %q, want %q", credential, problems, refused)
}
}
}
// A consumer's plan entry carries what its manifest contributes to that provision (issue 776), so the provider's
// helper learns a database's name from the bus; a consumer contributing nothing has the entry it had.
func TestAHandedModulesPlanEntryCarriesWhatItContributesToItsProvider(t *testing.T) {
@@ -258,7 +196,7 @@ func TestAnOfferAnsweredByTheHelperIsReadAndHandsNoFile(t *testing.T) {
if err != nil || !strings.Contains(string(raw), `"answered-by":"helper"`) {
t.Fatalf("written back as %s: %v", raw, err)
}
if _, err := ParseManifest([]byte(strings.Replace(with, `"helper"`, `"harness"`, 1))); err == nil {
if _, err := decodeForTest([]byte(strings.Replace(with, `"helper"`, `"harness"`, 1))); err == nil {
t.Fatal("an unknown answerer was accepted")
}
c1, before := composeWith(t, notifierOn(t, held(t, without, builtNotifier())), handingTo("store:5000"))
+1 -1
View File
@@ -22,7 +22,7 @@ func providing(raw string) string {
// onWorkstation resolves m on the workstation either assigned beside desk, or pulled in by desk alone.
func onWorkstation(t *testing.T, m Manifest, assigned bool) Resolution {
t.Helper()
desk, err := ParseManifest([]byte(pulledInBy))
desk, err := decodeForTest([]byte(pulledInBy))
if err != nil {
t.Fatal(err)
}
-31
View File
@@ -123,37 +123,6 @@ func TestAMovedModulesSecretsAreSealedAsBeforeWhereItIsNotHanded(t *testing.T) {
}
}
func TestWhatAHandedModuleCannotHaveFromTheVaultIsRefused(t *testing.T) {
// A template naming a credential the module requires: a binding's, which moves with its provider.
requires := strings.Replace(botMoved(), `"own-secrets":`, `"requires":["database"],"secrets":{"database":"/etc/bot/db"},"own-secrets":`, 1)
r, err := Resolve(shelf(held(t, requires, builtBot("db = ${secret:database}\n")),
Manifest{Module: "db", Version: "1", Provides: []Offer{{Name: "database", Scope: ScopeNode}}}),
[]string{"bot"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
r.Account = "op"
if _, err := r.Compose(handingTo("store:5000")); err == nil || !strings.Contains(err.Error(), "moves with its provider") {
t.Errorf("a template naming a required credential: %v", err)
}
// A file with its text inline naming a secret, in a handed module.
inline := strings.Replace(botMoved(), `{"id":"daemon"`,
`{"id":"legacy","type":"file","path":"/etc/bot/legacy","content":"t = ${secret:token}\n"},{"id":"daemon"`, 1)
r = notifierOn(t, held(t, inline, builtBot(botText)))
if _, err := r.Compose(handingTo("store:5000")); err == nil || !strings.Contains(err.Error(), "keep it in a template") {
t.Errorf("an inline file naming a secret in a handed module: %v", err)
}
// A family and a secret issued outside the mesh are not handed by name yet: the mark is refused.
for why, raw := range map[string]string{
"a family": strings.Replace(botMoved(), `"own-secrets":{"token":"${dir:state}/token"}`, `"own-secrets":{"token":"${dir:state}/token","key-*":"${dir:state}/keys"}`, 1),
"issued outside": strings.Replace(botMoved(), `"own-secrets":{"token":"${dir:state}/token"}`, `"own-secrets":{"token":{"path":"${dir:state}/token","issued-by":"outside"}}`, 1),
} {
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "design 57 V4") {
t.Errorf("a moved module with %s: %v", why, err)
}
}
}
// A credential of what a handed module requires that the vault keeps for it alone (issue 757): the forge's read
// token for one agent, issued by the forge's software per holder. Where the module is handed, its file is declared
// placed and without a value, and the plan names it with its provision and provider; anywhere else, and for a
+1 -19
View File
@@ -141,24 +141,6 @@ func TestAMovedModulesTemplatedFileIsHandedToTheProvisioner(t *testing.T) {
}
}
func TestAMovedModulesMarkIsJudged(t *testing.T) {
for name, raw := range map[string]string{
"marked moved and naming no template": `{"module":"bare","version":"1","moved":true,
"resources":[{"id":"conf","type":"file","path":"/etc/bare","content":"x"}]}`,
"handed written in a repository": `{"module":"bare","version":"1","moved":true,
"handed":{"source":"s","digest":"d","templates":{}},
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[{"id":"conf","type":"file","path":"/etc/bare","template":"files/bare"}]}`,
"handed and not moved": `{"module":"bare","version":"1",
"handed":{"source":"s","digest":"d","templates":{}},
"resources":[{"id":"conf","type":"file","path":"/etc/bare","content":"x"}]}`,
} {
if _, err := ParseManifest([]byte(raw)); err == nil {
t.Errorf("%s: accepted", name)
}
}
}
// What `module check` and the merge gate judge: a moved module read from its tree, its text inline. It is still
// marked moved, and composes as before, since nothing of its build's archive is there to hand.
func TestAMovedModuleReadFromItsTreeIsJudgedAsItComposesUnhanded(t *testing.T) {
@@ -169,7 +151,7 @@ func TestAMovedModuleReadFromItsTreeIsJudgedAsItComposesUnhanded(t *testing.T) {
if err := os.WriteFile(filepath.Join(dir, "files", "rc"), []byte(notifierText), 0o644); err != nil {
t.Fatal(err)
}
m, err := ParseManifest([]byte(notifierMoved()))
m, err := decodeForTest([]byte(notifierMoved()))
if err != nil {
t.Fatal(err)
}
-201
View File
@@ -174,207 +174,6 @@ func ReadHealth(r map[string]any) (Health, bool, []string) {
return h, true, problems
}
// healthProblems is everything wrong with a manifest's `health` fields, in the manifest's words (to-be
// 48 §8): a field on something that does not stay up, a kind its resource cannot have, an endpoint the
// module does not declare, a port or an address, a timing outside its bounds, a tool the module does not
// serve, and a tool check with no check of another kind beside it on the module.
func healthProblems(m Manifest) []string {
var problems []string
endpoints := map[string]Listening{}
for _, l := range m.Listens {
if n := strings.TrimSpace(l.Name); n != "" {
endpoints[n] = l
}
}
tools := map[string]bool{}
for _, t := range m.Tools {
tools[t] = true
}
wants := map[string]bool{}
for _, w := range m.Wants() {
wants[w] = true
}
var toolChecks []string
otherKinds := 0
resources := m.Resources
// A module's own runtime is judged as the process the controller writes for it (novox/hq ADR 0366 §1),
// and only a module that runs as its own account has one.
own, hasOwn := OwnRuntimeHealth(m)
if hasOwn {
if m.RunsAs == "" {
problems = append(problems, fmt.Sprintf("%s declares runtime-health and runs as no account of its own "+
"(runs-as): only a module served by a runtime of its own has that runtime's health to declare "+
"(novox/hq ADR 0366 §1)", m.Module))
hasOwn = false
} else {
resources = append(append([]map[string]any(nil), m.Resources...), own)
}
}
for i, r := range resources {
h, has, read := ReadHealth(r)
if !has {
continue
}
ownRuntime := hasOwn && i == len(resources)-1
id, typ := fmt.Sprint(r["id"]), fmt.Sprint(r["type"])
where := m.Module + ": " + id
if ownRuntime {
where = m.Module + ": its runtime-health"
}
for _, p := range read {
problems = append(problems, m.Module+": "+p)
}
if !LongRunning(r) {
problems = append(problems, fmt.Sprintf("%s declares health and does not stay up: a step, anything on a "+
"schedule and a service not stated running are judged by their step and their schedule (ADR 0240 rule 1)", where))
continue
}
switch h.Kind {
case HealthRuntime, HealthExec:
if typ != "container" {
problems = append(problems, fmt.Sprintf("%s is a %s and its health is %q, which only a container has: "+
"the runtime runs it inside the container", where, typ, h.Kind))
}
case HealthUnit:
if typ == "container" {
problems = append(problems, fmt.Sprintf("%s is a container and its health is %q, which is a service's "+
"or a process's own readiness", where, h.Kind))
}
case HealthHTTP, HealthTCP, HealthTool:
case "":
problems = append(problems, fmt.Sprintf("%s declares health with no kind: %s", where, healthKindsWords()))
default:
problems = append(problems, fmt.Sprintf("%s declares health of kind %q: %s", where, h.Kind, healthKindsWords()))
}
switch h.Kind {
case HealthHTTP, HealthTCP:
l, declared := endpoints[h.Endpoint]
switch {
case h.Endpoint == "":
problems = append(problems, fmt.Sprintf("%s's %s check names no endpoint: it names one the module "+
"declares under listens, by its name", where, h.Kind))
case !declared:
problems = append(problems, fmt.Sprintf("%s's %s check names the endpoint %q, which %s does not declare "+
"under listens (%s)", where, h.Kind, h.Endpoint, m.Module, namedEndpointsWords(endpoints)))
case l.At() != "tcp":
problems = append(problems, fmt.Sprintf("%s's %s check names %q, which is %s: a check connects over tcp",
where, h.Kind, h.Endpoint, l.At()))
}
default:
if h.Endpoint != "" {
problems = append(problems, fmt.Sprintf("%s's %s check names an endpoint, which only an http or tcp check "+
"looks at", where, h.Kind))
}
}
if h.Kind != HealthHTTP && (h.Path != "" || h.Status != 0 || h.Body != "" || h.Scheme != "") {
problems = append(problems, fmt.Sprintf("%s's %s check says a path, a status, a body or a scheme, which "+
"only an http check has", where, h.Kind))
}
if h.Kind == HealthHTTP {
if h.Path != "" && !strings.HasPrefix(h.Path, "/") {
problems = append(problems, fmt.Sprintf("%s's http check asks %q; a path starts with /", where, h.Path))
}
if h.Status != 0 && (h.Status < 100 || h.Status > 599) {
problems = append(problems, fmt.Sprintf("%s's http check expects status %d, which is not one", where, h.Status))
}
if h.Scheme != "" && h.Scheme != "http" && h.Scheme != "https" {
problems = append(problems, fmt.Sprintf("%s's http check is over %q; it is http or https", where, h.Scheme))
}
}
if (h.Command != "") != (h.Kind == HealthExec) {
if h.Kind == HealthExec {
problems = append(problems, fmt.Sprintf("%s's exec check says no command", where))
} else {
problems = append(problems, fmt.Sprintf("%s's %s check says a command, which only an exec check runs",
where, h.Kind))
}
}
if h.Kind == HealthTool {
switch {
case h.Tool == "":
problems = append(problems, fmt.Sprintf("%s's tool check names no tool", where))
case !tools[h.Tool]:
problems = append(problems, fmt.Sprintf("%s's tool check asks %q, which %s does not serve (its tools: %s)",
where, h.Tool, m.Module, orNoneWords(m.Tools)))
}
// Its own runtime being alive, judged by the node-engine, is the check the module does not run
// itself that this tool check stands beside (novox/hq ADR 0366 §2).
if !ownRuntime {
toolChecks = append(toolChecks, id)
}
} else {
if h.Tool != "" {
problems = append(problems, fmt.Sprintf("%s's %s check names a tool, which only a tool check asks", where, h.Kind))
}
if h.Kind != "" {
otherKinds++
}
}
if h.Needs != "" && !wants[h.Needs] {
problems = append(problems, fmt.Sprintf("%s's check needs %q, which %s does not require: a check names "+
"the provision it exercises, among those the module requires", where, h.Needs, m.Module))
}
problems = append(problems, healthTimingProblems(where, h)...)
}
if len(toolChecks) > 0 && otherKinds == 0 {
problems = append(problems, fmt.Sprintf("%s judges itself only by its own tool (%s): a tool check is for "+
"function no endpoint shows, and only beside a check of another kind the module does not run itself "+
"(ADR 0227 rule 8, ADR 0240 rule 2)", m.Module, strings.Join(toolChecks, ", ")))
}
return problems
}
// healthTimingProblems holds a check's timing to its bounds.
func healthTimingProblems(where string, h Health) []string {
var problems []string
if h.Interval < HealthIntervalFloor {
problems = append(problems, fmt.Sprintf("%s looks every %s; a check looks no more often than every %s — "+
"the mesh is a guest on the machine (ADR 0240 rule 2)", where, h.Interval, HealthIntervalFloor))
}
if h.Timeout <= 0 || h.Timeout >= h.Interval {
problems = append(problems, fmt.Sprintf("%s gives a look %s, which must be more than nothing and under its "+
"interval of %s", where, h.Timeout, h.Interval))
}
if h.Looks < HealthLooksFloor {
problems = append(problems, fmt.Sprintf("%s is unhealthy after %d failing look(s); it is at least %d — one "+
"look can be wrong (issue 277)", where, h.Looks, HealthLooksFloor))
}
if h.Grace < 0 {
problems = append(problems, fmt.Sprintf("%s has a grace of %s", where, h.Grace))
}
if h.Looks >= HealthLooksFloor && h.Interval >= HealthIntervalFloor {
if took := h.Grace + time.Duration(h.Looks)*h.Interval; took > HealthWithin {
problems = append(problems, fmt.Sprintf("%s is said unhealthy at the earliest %s after it starts (a grace "+
"of %s and %d looks every %s); it is at most %s, so a resource broken from its start is said within "+
"the gate's bound", where, took, h.Grace, h.Looks, h.Interval, HealthWithin))
}
}
return problems
}
func healthKindsWords() string {
return "a check is runtime (the image's own, adopted by name), http, tcp, exec, unit or tool"
}
func namedEndpointsWords(endpoints map[string]Listening) string {
if len(endpoints) == 0 {
return "it names none"
}
names := make([]string, 0, len(endpoints))
for n := range endpoints {
names = append(names, n)
}
sort.Strings(names)
return "it names " + strings.Join(names, ", ")
}
func orNoneWords(names []string) string {
if len(names) == 0 {
return "none"
}
return strings.Join(names, ", ")
}
// Undeclared is every long-running resource of the manifest without `health`, by id (ADR 0240 rule 8):
// what the catalogue's count counts.
func Undeclared(m Manifest) []string {
+7 -61
View File
@@ -53,78 +53,24 @@ func TestAWellFormedHealthIsAccepted(t *testing.T) {
{"kind": "runtime"},
{"kind": "exec", "command": "pg_isready -q", "grace": "4m", "looks": 2, "interval": "30s"},
} {
if _, err := ParseManifest(healthManifest(t, "server", h)); err != nil {
if _, err := decodeForTest(healthManifest(t, "server", h)); err != nil {
t.Errorf("%v was refused: %v", h, err)
}
}
if _, err := ParseManifest(healthManifest(t, "daemon", map[string]any{"kind": "unit"})); err != nil {
if _, err := decodeForTest(healthManifest(t, "daemon", map[string]any{"kind": "unit"})); err != nil {
t.Errorf("a service's own readiness was refused: %v", err)
}
// A tool beside a check of another kind on the module.
raw := healthManifest(t, "server", map[string]any{"kind": "http", "endpoint": "web"},
map[string]any{"id": "admin", "type": "service", "unit": "app-admin.service", "state": "running",
"health": map[string]any{"kind": "tool", "tool": "app_status"}})
if _, err := ParseManifest(raw); err != nil {
if _, err := decodeForTest(raw); err != nil {
t.Errorf("a tool check beside an http check was refused: %v", err)
}
}
func TestEveryOutOfBoundsHealthIsRefusedByName(t *testing.T) {
cases := []struct {
name string
on string
health map[string]any
says string
}{
{"no kind", "server", map[string]any{"endpoint": "web"}, "with no kind"},
{"an unknown kind", "server", map[string]any{"kind": "ping"}, `of kind "ping"`},
{"a port", "server", map[string]any{"kind": "tcp", "port": 8080}, "names a port"},
{"an address", "server", map[string]any{"kind": "http", "endpoint": "web", "address": "127.0.0.1"}, "names a address"},
{"a url", "server", map[string]any{"kind": "http", "url": "http://localhost:8080/"}, "names a url"},
{"an unknown key", "server", map[string]any{"kind": "tcp", "endpoint": "web", "retries": 3}, `"retries"`},
{"no endpoint", "server", map[string]any{"kind": "http"}, "names no endpoint"},
{"an undeclared endpoint", "server", map[string]any{"kind": "tcp", "endpoint": "admin"}, "does not declare"},
{"a udp endpoint", "server", map[string]any{"kind": "tcp", "endpoint": "beacon"}, "a check connects over tcp"},
{"an interval under the floor", "server", map[string]any{"kind": "tcp", "endpoint": "web", "interval": "5s", "timeout": "1s"}, "no more often than every 10s"},
{"a timeout of the interval", "server", map[string]any{"kind": "tcp", "endpoint": "web", "interval": "10s", "timeout": "10s"}, "under its interval"},
{"one failing look", "server", map[string]any{"kind": "tcp", "endpoint": "web", "looks": 1}, "at least 2"},
{"a grace and looks past five minutes", "server", map[string]any{"kind": "tcp", "endpoint": "web", "grace": "4m", "looks": 3, "interval": "30s"}, "at most 5m0s"},
{"a duration that is not one", "server", map[string]any{"kind": "tcp", "endpoint": "web", "interval": "often"}, "is a duration"},
{"looks that are not a number", "server", map[string]any{"kind": "tcp", "endpoint": "web", "looks": "three"}, "whole number"},
{"a path without a slash", "server", map[string]any{"kind": "http", "endpoint": "web", "path": "health"}, "starts with /"},
{"a status that is not one", "server", map[string]any{"kind": "http", "endpoint": "web", "status": 700}, "is not one"},
{"a scheme that is not one", "server", map[string]any{"kind": "http", "endpoint": "web", "scheme": "ftp"}, "http or https"},
{"a status on a tcp check", "server", map[string]any{"kind": "tcp", "endpoint": "web", "status": 200}, "only an http check has"},
{"an exec with no command", "server", map[string]any{"kind": "exec"}, "says no command"},
{"a command on an http check", "server", map[string]any{"kind": "http", "endpoint": "web", "command": "true"}, "only an exec check runs"},
{"a runtime check on a service", "daemon", map[string]any{"kind": "runtime"}, "only a container has"},
{"a unit check on a container", "server", map[string]any{"kind": "unit"}, "a service's or a process's"},
{"a tool the module does not serve", "server", map[string]any{"kind": "tool", "tool": "app_admin"}, "does not serve"},
{"a tool check alone", "server", map[string]any{"kind": "tool", "tool": "app_status"}, "judges itself only by its own tool"},
{"needs not required", "server", map[string]any{"kind": "tcp", "endpoint": "web", "needs": "redis"}, "does not require"},
{"health on a step", "seed", map[string]any{"kind": "runtime"}, "does not stay up"},
{"health on a schedule", "sweep", map[string]any{"kind": "runtime"}, "does not stay up"},
{"health that is not an object", "server", nil, "is an object"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
raw := healthManifest(t, c.on, c.health)
if c.health == nil {
raw = []byte(strings.Replace(string(raw), `"name":"app-server"`, `"name":"app-server","health":"tcp"`, 1))
}
_, err := ParseManifest(raw)
if err == nil {
t.Fatalf("%s was accepted", c.name)
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("refused, but not for saying %q: %v", c.says, err)
}
})
}
}
func TestHealthIsComposedAsThePortThisMachineGaveTheEndpoint(t *testing.T) {
m, err := ParseManifest(healthManifest(t, "server", map[string]any{"kind": "http", "endpoint": "web", "path": "/healthz",
m, err := decodeForTest(healthManifest(t, "server", map[string]any{"kind": "http", "endpoint": "web", "path": "/healthz",
"needs": "postgres-database"}))
if err != nil {
t.Fatal(err)
@@ -163,7 +109,7 @@ func TestHealthIsComposedAsThePortThisMachineGaveTheEndpoint(t *testing.T) {
}
func TestTheUndeclaredAreTheLongRunningWithoutHealth(t *testing.T) {
m, err := ParseManifest(healthManifest(t, "server", map[string]any{"kind": "runtime"}))
m, err := decodeForTest(healthManifest(t, "server", map[string]any{"kind": "runtime"}))
if err != nil {
t.Fatal(err)
}
@@ -176,7 +122,7 @@ func TestATooledHealthIsGrantedToTheEngine(t *testing.T) {
raw := healthManifest(t, "server", map[string]any{"kind": "http", "endpoint": "web"},
map[string]any{"id": "admin", "type": "service", "unit": "app-admin.service", "state": "running",
"health": map[string]any{"kind": "tool", "tool": "app_status"}})
m, err := ParseManifest(raw)
m, err := decodeForTest(raw)
if err != nil {
t.Fatal(err)
}
@@ -194,7 +140,7 @@ func TestEveryComposedHealthIsOneTheNodeEngineTakes(t *testing.T) {
{"kind": "runtime"},
{"kind": "exec", "command": "pg_isready -q"},
} {
m, err := ParseManifest(healthManifest(t, "server", h,
m, err := decodeForTest(healthManifest(t, "server", h,
map[string]any{"id": "admin", "type": "service", "unit": "app-admin.service", "state": "running",
"health": map[string]any{"kind": "tool", "tool": "app_status"}}))
if err != nil {
-44
View File
@@ -174,47 +174,3 @@ func (r Resolution) Overflowing() []Overflow {
// passes on every machine that mesh has. `module check --longest-machine-name` says another mesh's;
// a mesh that names a longer machine raises this in the same change (ADR 0225).
const DefaultLongestMachine = 6
// IdentityProblems is every module whose identity would overflow a provision it wants, on a machine
// whose name is `longestMachine` characters — judged before merge, over the catalogue alone, so the
// pull request that introduces an overflow is the one refused (novox/hq ADR 0225, issue 263). A
// provision no module in the shelf offers is not judged: its bound is not known here.
func IdentityProblems(shelf Shelf, longestMachine int) []string {
offeredBy := map[string][]string{}
for _, name := range shelfOrder(shelf) {
for _, o := range shelf[name].Offers() {
offeredBy[o] = append(offeredBy[o], name)
}
}
machine := strings.Repeat("n", longestMachine)
var problems []string
for _, name := range shelfOrder(shelf) {
m := shelf[name]
source := IdentitySource(m.Slug, m.Module)
for _, want := range m.Wants() {
// The tightest bound among the modules offering it: whichever one answers on a given
// machine, the identity has to fit it.
tightest, by := IdentityBound{}, ""
for _, provider := range offeredBy[want] {
if provider == name {
continue // a module answering its own requirement is not its own consumer
}
b := shelf[provider].IdentityBoundOf(want)
if b.Bounded() && (!tightest.Bounded() || b.Max < tightest.Max) {
tightest, by = b, provider
}
}
if CheckIdentityWithin(machine, source, tightest) == nil {
continue
}
got := ConsumerIdentity(machine, source)
problems = append(problems, fmt.Sprintf(
"%s wants %s, and %s keeps its consumers' identities in %s of at most %d characters: "+
"on a machine with a %d-character name it is identified as %q, %d — give %s a "+
"`slug` of at most %d characters",
name, want, by, tightest.In, tightest.Max, longestMachine, got, len(got), name,
tightest.Max-len(IdentityPrefix)-longestMachine-1))
}
}
return problems
}
-106
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
@@ -61,97 +60,6 @@ func TestAnUnstatedBoundFollowsWhatTheProviderIsTold(t *testing.T) {
}
}
// The field reads as written and writes back the same, and refuses what says nothing.
func TestAnOffersIdentityIsParsedStrictly(t *testing.T) {
for _, raw := range []string{
`{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"}}`,
`{"name":"wildcard-resolution","scope":"mesh","identity":false}`,
`{"name":"redis-cache","scope":"mesh","identity":{"in":"a Redis ACL user"}}`,
} {
var o Offer
if err := json.Unmarshal([]byte(raw), &o); err != nil {
t.Fatalf("%s: %v", raw, err)
}
back, err := json.Marshal(o)
if err != nil || string(back) != raw {
t.Errorf("did not round-trip:\n%s\n%s (%v)", raw, back, err)
}
}
for _, raw := range []string{
`{"name":"x","identity":true}`,
`{"name":"x","identity":{"max":20,"in":"y","most":3}}`,
} {
var o Offer
if err := json.Unmarshal([]byte(raw), &o); err == nil {
t.Errorf("accepted %s", raw)
}
}
bad := Manifest{Module: "bad", Version: "1", Provides: []Offer{
{Name: "unsaid", Scope: ScopeMesh, Identity: &OfferIdentity{Max: 20}},
{Name: "tiny", Scope: ScopeMesh, Identity: &OfferIdentity{Max: 4, In: "nothing usable"}},
}}
raw, err := json.Marshal(bad)
if err != nil {
t.Fatal(err)
}
_, err = ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), "without saying what keeps them") ||
!strings.Contains(err.Error(), "the shortest the mesh makes") {
t.Fatalf("a bound with no `in`, or too short for any identity, was accepted: %v", err)
}
}
// Refused before merge: the catalogue check judges each module's identity, on the longest machine
// name, against the bound of every provision it wants — and names the module and the slug to set.
func TestTheCatalogueCheckRefusesAnIdentityThatOverflowsWhatItRequires(t *testing.T) {
shelf := Shelf{
"objects": {Module: "objects", Provides: []Offer{{Name: "s3-bucket", Scope: ScopeMesh,
Identity: &OfferIdentity{Max: 20, In: "an S3 access key"}}},
Receives: map[string]string{"s3-bucket": "/var/lib/mesh/objects/mesh.json"}},
"photoalbum": {Module: "photoalbum", Requires: []string{"s3-bucket"}},
"files": {Module: "files", Requires: []string{"s3-bucket"}},
}
problems := IdentityProblems(shelf, 6)
if len(problems) != 1 || !strings.Contains(problems[0], "photoalbum wants s3-bucket") ||
!strings.Contains(problems[0], `"mesh_nnnnnn_photoalbum", 22`) ||
!strings.Contains(problems[0], "`slug` of at most 8 characters") {
t.Fatalf("one overflow, named with its remedy, was expected: %q", problems)
}
// A longer machine name refuses more: the check is about the mesh's machines, not one.
if got := IdentityProblems(shelf, 10); len(got) != 2 {
t.Fatalf("on a 10-character name both overflow (mesh_nnnnnnnnnn_files is 21): %q", got)
}
// And a slug is the remedy it names.
album := shelf["photoalbum"]
album.Slug = "album"
shelf["photoalbum"] = album
if got := IdentityProblems(shelf, 6); len(got) != 0 {
t.Fatalf("a slug that fits is still refused: %q", got)
}
}
// Tonight's case (issue 263): networkmanager, no slug, requiring the mesh's resolver provision on a
// machine with a six-character name. Under ADR 0049's one bound it was refused, and its provider's
// whole machine with it; the resolver keeps no name, so it is not refused at all.
func TestARequirementOnAKeylessProvisionComposesWithALongName(t *testing.T) {
shelf := Shelf{
"resolver": {Module: "resolver", Provides: []Offer{{Name: "wildcard-resolution", Scope: ScopeMesh}}},
"networkmanager": {Module: "networkmanager", Requires: []string{"wildcard-resolution"}},
}
if CheckIdentity("laptop", "networkmanager") == nil {
t.Fatal("the regression is not reproduced: mesh_laptop_networkmanager fits the old global bound")
}
if got := IdentityProblems(shelf, 6); len(got) != 0 {
t.Fatalf("a requirement on a keyless provision was refused for its length: %q", got)
}
r := Resolution{Node: "laptop", Modules: []Manifest{shelf["networkmanager"]},
Needs: []Needed{{Name: "wildcard-resolution", From: "anchor", For: "networkmanager",
Identity: shelf["resolver"].IdentityBoundOf("wildcard-resolution")}}}
if got := r.Overflowing(); len(got) != 0 {
t.Fatalf("a keyless requirement is reported as overflowing: %+v", got)
}
}
// The consumer's side of the same judgement the provider's composition makes: what `status` says.
func TestAnOverflowingRequirementIsNamedFromTheConsumersSide(t *testing.T) {
bound := IdentityBound{Max: 20, In: "an S3 access key"}
@@ -174,17 +82,3 @@ func TestAnOverflowingRequirementIsNamedFromTheConsumersSide(t *testing.T) {
t.Fatalf("the overflow does not say what keeps it or the remedy: %s", said)
}
}
// An identity served as a DNS label is never truncated, so the offer's bound must keep it in one.
func TestAnIdentityServedAsADNSLabelIsBoundedToOne(t *testing.T) {
serves := map[string]map[string]any{"bucket": {"name": "${consumer:as:dns}"}}
wide := Manifest{Module: "wide", Serves: serves, Provides: []Offer{{Name: "bucket", Scope: ScopeMesh,
Identity: &OfferIdentity{Max: 255, In: "a client id"}}}}
if got := CheckServes(wide); len(got) != 1 || !strings.Contains(got[0], "a label keeps 63") {
t.Fatalf("a 255-character bound on a DNS label passed: %q", got)
}
unsaid := Manifest{Module: "unsaid", Serves: serves, Provides: []Offer{{Name: "bucket", Scope: ScopeMesh}}}
if got := CheckServes(unsaid); len(got) != 0 {
t.Fatalf("the default bound (20) on a DNS label was refused: %q", got)
}
}
-209
View File
@@ -1,14 +1,5 @@
package catalogue
import (
"encoding/json"
"fmt"
"net"
"regexp"
"sort"
"strings"
)
// A definition names no installation (novox/hq ADR 0112, ADR 0155, issues 122 and 134).
//
// A module definition holds what is true of the module everywhere; what is particular to one mesh —
@@ -34,203 +25,3 @@ import (
// NamesOnPurpose is the catalogue-level word a resource carries for the names it means to name:
// each name mapped to its reason. The host never sees it.
const NamesOnPurpose = "names-on-purpose"
// prose is every key whose value the mesh never reads.
var prose = map[string]bool{"why": true, "description": true}
// Registries the world runs, which an image may name because an image reference must say where it
// is pulled from. Anything else in an image reference is a registry of some installation.
var worldsRegistries = map[string]bool{
"docker.io": true, "registry-1.docker.io": true, "index.docker.io": true, "ghcr.io": true,
"quay.io": true, "gcr.io": true, "registry.k8s.io": true, "k8s.gcr.io": true,
"mcr.microsoft.com": true, "lscr.io": true, "public.ecr.aws": true, "registry.gitlab.com": true,
"codeberg.org": true, "cgr.dev": true,
}
// Services the world runs that a definition may name as a policy default, the way it may name a
// public resolver: the public certificate authorities' ACME directories. Anything else a served
// fact or a file names is somebody's installation.
var worldsServices = map[string]bool{
"acme-v02.api.letsencrypt.org": true, "acme-staging-v02.api.letsencrypt.org": true,
"api.buypass.com": true, "api.test4.buypass.no": true, "dv.acme-v02.api.pki.goog": true,
"acme.zerossl.com": true,
}
// Resolvers the world runs, which a machine's resolver may forward to as a policy default.
var worldsResolvers = map[string]bool{
"1.1.1.1": true, "1.0.0.1": true, "8.8.8.8": true, "8.8.4.4": true, "9.9.9.9": true,
"149.112.112.112": true, "208.67.222.222": true, "208.67.220.220": true,
}
// hostname is a dotted name whose last label is a top-level domain a real installation would have.
// Not every dotted token: `module.json`, `index.html` and `docker.sock` are dotted and name nothing.
// Boundaries are checked by hand rather than in the pattern, because two names one character apart
// — `a.example.tld,b.example.tld` — would otherwise share the delimiter and the second would be lost.
var hostname = regexp.MustCompile(
`(?i)(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+` +
`(?:be|nl|de|fr|uk|eu|com|net|org|io|dev|app|cloud|site|online|me|co|ch|at|lu|` +
`internal|example|tld|test|invalid)`)
// upstreamVersion is where an artifact pinned by URL says which release it is (Artifact.Version).
var upstreamVersion = regexp.MustCompile(`^build\.artifacts\[[0-9]+\]\.version$`)
// address is a dotted quad.
var address = regexp.MustCompile(`(?:[0-9]{1,3}\.){3}[0-9]{1,3}`)
// isName is whether a byte may be part of a name; a match bordered by one is a longer token.
func isName(b byte) bool {
return b == '.' || b == '-' || (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || (b >= '0' && b <= '9')
}
// standalone are the matches of re in value that are whole tokens, not parts of a longer one.
func standalone(re *regexp.Regexp, value string) []string {
var out []string
for _, span := range re.FindAllStringIndex(value, -1) {
if span[0] > 0 && isName(value[span[0]-1]) {
continue
}
if span[1] < len(value) && isName(value[span[1]]) {
continue
}
out = append(out, value[span[0]:span[1]])
}
return out
}
// InstallationProblems is every value of a definition that names an installation, in the
// definition's own words: where it is, and what it names.
func InstallationProblems(m Manifest) []string {
raw, err := json.Marshal(m)
if err != nil {
return []string{fmt.Sprintf("%s could not be read back: %v", m.Module, err)}
}
var tree any
if err := json.Unmarshal(raw, &tree); err != nil {
return []string{fmt.Sprintf("%s could not be read back: %v", m.Module, err)}
}
var problems []string
// The module's own name is a value too: a module named after the domain it serves is a
// definition that can only be installed there (issue 134).
for _, name := range namesIn(m.Module) {
problems = append(problems, fmt.Sprintf(
"%s is named after %s, and a module is named for what it is, not for where it runs", m.Module, name))
}
walk(tree, "", nil, func(at string, value string, meant map[string]bool, isImage bool) {
for _, name := range namesIn(value) {
if (isImage && worldsRegistries[strings.ToLower(name)]) || meant[name] {
continue
}
problems = append(problems, fmt.Sprintf("%s names %s at %s", m.Module, name, at))
}
for _, ip := range addressesIn(value) {
// An upstream pin's version may have four dotted numbers (Owl 1.5.6.1), which is no address; a name
// in it is still judged above. Its URL is judged whole: the vendor's host goes through the artifact's
// names-on-purpose like any name a definition means (novox/hq ADR 0316).
if meant[ip] || upstreamVersion.MatchString(at) {
continue
}
problems = append(problems, fmt.Sprintf("%s names the public address %s at %s", m.Module, ip, at))
}
})
sort.Strings(problems)
return problems
}
// walk visits every string in the tree with its path, the names the enclosing resource means to
// name (with a reason), and whether it is an image reference.
func walk(node any, at string, meant map[string]bool, visit func(at, value string, meant map[string]bool, isImage bool)) {
switch v := node.(type) {
case map[string]any:
if declared, has := v[NamesOnPurpose].(map[string]any); has {
widened := map[string]bool{}
for name := range meant {
widened[name] = true
}
for name, reason := range declared {
if r, ok := reason.(string); ok && strings.TrimSpace(r) != "" {
widened[strings.ToLower(name)] = true
}
}
meant = widened
}
keys := make([]string, 0, len(v))
for k := range v {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
// Prose is a string a person reads. A key that is called `why` or `description` and holds
// anything else — a setting of that name, whose default the mesh writes — is walked like any
// other (novox/hq ADR 0262).
if _, isString := v[k].(string); (prose[k] && isString) || k == NamesOnPurpose || (at == "" && k == "module") {
continue
}
child := at + "." + k
if at == "" {
child = k
}
if s, isString := v[k].(string); isString {
visit(child, s, meant, k == "image")
continue
}
walk(v[k], child, meant, visit)
}
case []any:
for i, item := range v {
child := fmt.Sprintf("%s[%d]", at, i)
if s, isString := item.(string); isString {
visit(child, s, meant, false)
continue
}
walk(item, child, meant, visit)
}
}
}
// namesIn is every hostname in a value that could belong to an installation.
func namesIn(value string) []string {
var out []string
for _, found := range standalone(hostname, value) {
name := strings.ToLower(found)
switch {
case strings.HasSuffix(name, ".docker.internal"):
// The container runtime's alias for its own host: every machine running it has one.
case worldsServices[name]:
// A public authority named as a policy default, true of any mesh that wants it.
case name == "example.tld", strings.HasSuffix(name, ".example.tld"),
name == "example.com", name == "example.net", name == "example.org",
strings.HasSuffix(name, ".example.com"), strings.HasSuffix(name, ".example.net"),
strings.HasSuffix(name, ".example.org"), strings.HasSuffix(name, ".example"),
strings.HasSuffix(name, ".test"), strings.HasSuffix(name, ".invalid"):
// Documentation names, which is what a definition's own example should use.
default:
out = append(out, name)
}
}
return out
}
// addressesIn is every public address in a value: not a private range, loopback, link-local, the
// unspecified address, a documentation range, or a resolver the world runs.
func addressesIn(value string) []string {
var out []string
for _, found := range standalone(address, value) {
ip := net.ParseIP(found)
if ip == nil || ip.IsPrivate() || ip.IsLoopback() || ip.IsLinkLocalUnicast() ||
ip.IsUnspecified() || ip.IsMulticast() || worldsResolvers[found] || documentation(ip) {
continue
}
out = append(out, found)
}
return out
}
func documentation(ip net.IP) bool {
for _, cidr := range []string{"192.0.2.0/24", "198.51.100.0/24", "203.0.113.0/24", "100.64.0.0/10"} {
_, block, _ := net.ParseCIDR(cidr)
if block.Contains(ip) {
return true
}
}
return false
}
-92
View File
@@ -1,93 +1 @@
package catalogue
import (
"strings"
"testing"
)
// A definition names no installation (novox/hq ADR 0112, ADR 0155). What the mesh acts on is judged;
// prose is not; the world's registries and resolvers are the world's; a declared exception is a
// reason a reader sees.
func TestADefinitionNamingAnInstallationIsNamedBack(t *testing.T) {
m := Manifest{Module: "idp", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "quay.io/keycloak/keycloak@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}},
{"id": "env", "type": "file", "content": "REAL_IP_FROM=192.168.1.0/24,127.0.0.0/8,203.0.113.7,51.15.22.9\n"},
}, Listens: []Listening{{Port: 8080, From: FromMesh, Why: "the login page; login.mesh-one.be is a route grant"}}}
got := strings.Join(InstallationProblems(m), "\n")
for _, want := range []string{
"idp names login.mesh-one.be at resources[0].env.KC_HOSTNAME",
"idp names the public address 51.15.22.9 at resources[1].content",
} {
if !strings.Contains(got, want) {
t.Errorf("missing %q in:\n%s", want, got)
}
}
for _, mustNot := range []string{"quay.io", "why", "203.0.113.7", "192.168.1.0", "127.0.0.0"} {
if strings.Contains(got, mustNot) {
t.Errorf("%q was reported and should not be:\n%s", mustNot, got)
}
}
}
func TestTheWorldsNamesAreNotAnInstallations(t *testing.T) {
m := Manifest{Module: "resolver", Resources: []map[string]any{
{"id": "conf", "type": "file", "content": "server=1.1.1.1\nserver=8.8.8.8\nlisten=127.0.0.55\n"},
{"id": "proxy", "type": "container", "image": "docker.io/library/traefik@sha256:bb"},
{"id": "adapter", "type": "file", "content": "{\"machine\": \"host.docker.internal\"}\n"},
{"id": "doc", "type": "file", "content": "root = https://git.example.tld/\n"},
}}
if got := InstallationProblems(m); len(got) != 0 {
t.Fatalf("the world's names were reported: %v", got)
}
}
func TestANameMeantOnPurposeIsDeclaredWithItsReason(t *testing.T) {
// The federation's public directory in a homeserver's config: the world's, said so, and a name
// the map does not cover is still reported.
m := Manifest{Module: "homeserver", Resources: []map[string]any{
{"id": "conf", "type": "file", "content": "trusted_key_servers: matrix.org\nwell_known: https://mesh-one.be\n",
NamesOnPurpose: map[string]any{"matrix.org": "the federation's public key server, the world's"}},
}}
got := InstallationProblems(m)
if len(got) != 1 || !strings.Contains(got[0], "mesh-one.be") {
t.Fatalf("got %v", got)
}
}
func TestAnImageFromAnInstallationsRegistryNeedsAReason(t *testing.T) {
bare := Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc"},
}}
if got := InstallationProblems(bare); len(got) != 1 || !strings.Contains(got[0], "registry.mesh-one.be") {
t.Fatalf("an image on an installation's registry was not named: %v", got)
}
excepted := Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc",
NamesOnPurpose: map[string]any{"registry.mesh-one.be": "built outside the mesh until the site's repository is a build source here"}},
}}
if got := InstallationProblems(excepted); len(got) != 0 {
t.Fatalf("a declared exception was still reported: %v", got)
}
}
func TestAModuleNamedAfterADomainIsNamedBack(t *testing.T) {
got := InstallationProblems(Manifest{Module: "mesh-one.be"})
if len(got) != 1 || !strings.Contains(got[0], "named after mesh-one.be") {
t.Fatalf("got %v", got)
}
}
func TestABuildContextOnASeatNamesNoForge(t *testing.T) {
m := Manifest{Module: "packager", Build: &Build{Artifacts: []Artifact{
{Name: "server", Kind: "image", From: "Dockerfile",
Context: &ArtifactContext{Seat: "git", Repository: "org/controller", Ref: "main"}},
}}}
if got := InstallationProblems(m); len(got) != 0 {
t.Fatalf("a context on a seat was reported: %v", got)
}
m.Build.Artifacts[0].Context = &ArtifactContext{Repository: "https://git.mesh-one.be/org/controller.git"}
if got := InstallationProblems(m); len(got) != 1 {
t.Fatalf("a context by URL was not reported: %v", got)
}
}
+1 -14
View File
@@ -1,14 +1,13 @@
package catalogue
import (
"strings"
"testing"
)
// A manifest may say which tools its module calls (novox/hq ADR 0152), and the parser accepts the
// two shapes the grant has: a named tool, and every tool.
func TestAManifestMaySayWhatItInvokes(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-console","version":"1",` +
m, err := decodeForTest([]byte(`{"module":"mesh-console","version":"1",` +
`"invokes":["mesh-catalog.catalog_modules","*"]}`))
if err != nil {
t.Fatal(err)
@@ -17,15 +16,3 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
t.Fatalf("invokes not read: %v", m.Invokes)
}
}
// An entry that names a module and no tool is refused at parse, in the manifest's words, rather than
// at the composition of the bus's user list where it would stop the file for everybody.
func TestAnInvokeThatNamesNoToolIsRefusedAtParse(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"mesh-console","version":"1","invokes":["shop"]}`))
if err == nil {
t.Fatal("an invoke naming no tool was accepted")
}
if !strings.Contains(err.Error(), `invokes "shop", which does not name a tool`) {
t.Fatalf("refused for the wrong reason: %v", err)
}
}
-41
View File
@@ -1,42 +1 @@
package catalogue
import (
"strings"
"testing"
)
// A provision's credential may be issued by its provider's software (novox/hq ADR 0305, issue 526): the offer
// names the tool that answers it and no own secret, and every other shape is refused at parse.
func TestAnOfferMaySayItsSoftwareIssuesTheCredential(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"media","version":"1",
"provides":[{"name":"media-api","scope":"mesh","credential":{"issued-by":"software","tool":"media_issued_credential"}}],
"serves":{"media-api":{"port":32400}}}`))
if err != nil {
t.Fatal(err)
}
if tool, issued := m.IssuedCredentialOf("media-api"); !issued || tool != "media_issued_credential" {
t.Fatalf("the offer's issued credential was not read: %q %v", tool, issued)
}
if _, shared := m.SharedCredentialOf("media-api"); shared {
t.Fatal("an issued credential was read as an own secret shared")
}
if _, issued := m.IssuedCredentialOf("other"); issued {
t.Fatal("a provision the module does not offer reads as issued")
}
for want, raw := range map[string]string{
"the issuers an offer may name": `{"module":"d","version":"1",
"provides":[{"name":"d-api","credential":{"issued-by":"vendor","tool":"d_token"}}]}`,
"names no tool that answers it": `{"module":"d","version":"1",
"provides":[{"name":"d-api","credential":{"issued-by":"software"}}]}`,
"names its own secret": `{"module":"d","version":"1","own-secrets":{"token":{"path":"/p","taken":"at-start"}},
"provides":[{"name":"d-api","credential":{"issued-by":"software","tool":"d_token","own":"token"}}]}`,
"the issuers an offer may name ": `{"module":"d","version":"1",
"provides":[{"name":"d-api","credential":{"tool":"d_token"}}]}`,
} {
want = strings.TrimSpace(want)
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("expected a refusal saying %q, got %v", want, err)
}
}
}
-45
View File
@@ -1,46 +1 @@
package catalogue
import (
"strings"
"testing"
)
// A provision's credential may be one only a person outside the mesh can give (novox/hq ADR 0337 §1, issue 164):
// the offer says `issued-by: outside` and names neither an own secret nor a tool, and the parser refuses it beside
// either. An offer that says `software` keeps ADR 0305's shape.
func TestAnOfferMaySayAPersonOutsideTheMeshIssuesItsCredential(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"tv","version":"1",
"provides":[{"name":"tv-api","scope":"mesh","credential":{"issued-by":"outside"}}],
"serves":{"tv-api":{"port":8989}}}`))
if err != nil {
t.Fatal(err)
}
if !m.IssuedOutsideOf("tv-api") {
t.Fatal("the offer's credential issued outside was not read")
}
if m.IssuedOutsideOf("other") {
t.Fatal("a provision the module does not offer reads as issued outside")
}
if _, issued := m.IssuedCredentialOf("tv-api"); issued {
t.Fatal("a credential issued outside was read as issued by the software")
}
if _, shared := m.SharedCredentialOf("tv-api"); shared {
t.Fatal("a credential issued outside was read as an own secret shared")
}
if who, offered := OffersIssuedOutside(map[string]Manifest{"tv": m}, "tv-api"); !offered || who != "tv" {
t.Fatalf("the catalogue does not say tv offers tv-api issued outside: %q %v", who, offered)
}
for want, raw := range map[string]string{
"names its own secret": `{"module":"d","version":"1","own-secrets":{"key":{"path":"/p","taken":"at-start"}},
"provides":[{"name":"d-api","credential":{"issued-by":"outside","own":"key"}}]}`,
"names the tool": `{"module":"d","version":"1",
"provides":[{"name":"d-api","credential":{"issued-by":"outside","tool":"d_key"}}]}`,
"the issuers an offer may name": `{"module":"d","version":"1",
"provides":[{"name":"d-api","credential":{"issued-by":"a-person"}}]}`,
} {
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("expected a refusal saying %q, got %v", want, err)
}
}
}
-48
View File
@@ -1,49 +1 @@
package catalogue
import (
"strings"
"testing"
)
// A jail whose pattern names <HOST> twice is refused by name: fail2ban would read it as a duplicate
// capture group and refuse its whole configuration, leaving the machine with no bans at all — which
// is what happened on the control node the day this was built (novox/hq ADR 0179).
func TestAJailsPatternNamesTheHostOncePerShape(t *testing.T) {
twice := Manifest{Module: "route-proxy", Jails: []Jail{{
Name: "route-proxy",
Failregex: `^.*(?:refused for <HOST>:\d+|asked from <HOST>:\d+)$`,
Jail: "backend = systemd",
}}}
problems := twice.jailProblems()
if len(problems) != 1 || !strings.Contains(problems[0], "names <HOST> 2 times") {
t.Fatalf("a pattern naming <HOST> twice was not refused by name: %v", problems)
}
if !strings.Contains(problems[0], "no bans at all") {
t.Errorf("the refusal does not say what it costs: %q", problems[0])
}
// The same two shapes, one pattern each, is how it is written — and is accepted.
split := Manifest{Module: "route-proxy", Jails: []Jail{{
Name: "route-proxy",
Failregex: "^.*refused for <HOST>:\\d+$\n ^.*asked from <HOST>:\\d+$",
Jail: "backend = systemd",
}}}
if problems := split.jailProblems(); len(problems) != 0 {
t.Fatalf("one <HOST> per pattern was refused: %v", problems)
}
}
// A jail with no name, no pattern, or a name another jail of the same module already took is refused.
func TestAJailIsNamedOnceAndSaysWhatAFailureLooksLike(t *testing.T) {
m := Manifest{Module: "mailu", Jails: []Jail{
{Name: "", Failregex: "^.*rip=<HOST>$"},
{Name: "mailu-front", Failregex: " "},
{Name: "mailu-front", Failregex: "^.*rip=<HOST>$"},
}}
problems := strings.Join(m.jailProblems(), "\n")
for _, want := range []string{"a jail with no name", "says nothing a failed attempt looks like", "two jails called"} {
if !strings.Contains(problems, want) {
t.Errorf("missing %q in:\n%s", want, problems)
}
}
}
-10
View File
@@ -56,13 +56,3 @@ func TestOnlyAModuleThatKeepsGetsTheExport(t *testing.T) {
}
}
}
func TestKeepsMustBeAnAbsolutePath(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"mesh-vault","version":"1","keeps":"root"}`))
if err == nil || !strings.Contains(err.Error(), "keeps") {
t.Fatalf("a relative keeps path was not refused: %v", err)
}
if _, err := ParseManifest([]byte(`{"module":"mesh-vault","version":"1","keeps":"/var/lib/mesh-vault/root"}`)); err != nil {
t.Fatalf("an absolute keeps path was refused: %v", err)
}
}
-130
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"strings"
"testing"
)
@@ -25,62 +24,6 @@ func aChannel(module, kind string) Manifest {
{Name: "channel", Scope: ScopeMesh, Kind: kind}, {Name: "intake", Scope: ScopeMesh, Kind: kind}}}
}
func TestTwoChannelsOfDifferentKindsHoldTheBenches(t *testing.T) {
shelf := Shelf{"messenger": router(), "telegram": aChannel("telegram", "telegram"),
"desk-channel": aChannel("desk-channel", "desktop")}
if got := problemsFor(t, shelf); got != "" {
t.Fatalf("two kinds were refused: %s", got)
}
for _, m := range shelf {
if got := declaredSeatProblems(m); len(got) > 0 {
t.Fatalf("%s: %v", m.Module, got)
}
}
}
func TestASecondClaimOfOneKindIsRefused(t *testing.T) {
got := problemsFor(t, Shelf{"messenger": router(), "telegram": aChannel("telegram", "telegram"),
"telegram-two": aChannel("telegram-two", "telegram")})
if !strings.Contains(got, `of kind "telegram", which telegram already claims`) {
t.Fatalf("a second holder of one kind stood: %s", got)
}
}
func TestAKindedBenchNeedsAKindAndNoOtherSeatTakesOne(t *testing.T) {
got := problemsFor(t, Shelf{"messenger": router(), "nameless": aChannel("nameless", "")})
if !strings.Contains(got, "claims the kinded bench channel and names no kind") {
t.Fatalf("a claim without a kind stood: %s", got)
}
odd := Manifest{Module: "odd", Claims: []Claim{{Name: "operator-channel", Scope: ScopeMesh, Kind: "telegram"}}}
got = problemsFor(t, Shelf{"messenger": router(), "odd": odd})
if !strings.Contains(got, "only a kinded bench takes a kind") {
t.Fatalf("a kind on a seat that is not kinded stood: %s", got)
}
dotted := problemsFor(t, Shelf{"messenger": router(), "dotted": aChannel("dotted", "a.b")})
if !strings.Contains(dotted, "not a usable name") {
t.Fatalf("a kind that would widen a subject stood: %s", dotted)
}
}
func TestOnlyChannelAndIntakeAreKinded(t *testing.T) {
m := Manifest{Module: "x", DefinesSeats: []SeatDeclaration{{Name: "pager", Kinded: true, Accepts: []string{"page"}}}}
if got := strings.Join(declaredSeatProblems(m), "; "); !strings.Contains(got, "only channel and intake are kinded") {
t.Fatalf("another kinded bench was declared: %s", got)
}
}
func TestTheNewRulesAreHeldToWhatTheSeatSays(t *testing.T) {
m := Manifest{Module: "x", DefinesSeats: []SeatDeclaration{{Name: "thing", Accepts: []string{"do"},
ByCaller: []string{"undo"}, Proofs: []string{"code"}, Records: []string{"nothing"}}}}
got := strings.Join(declaredSeatProblems(m), "; ")
for _, want := range []string{"names thing.undo by its caller, which the seat neither accepts nor emits",
"declares proofs on thing, which is not kinded", `read its records "nothing", which it keeps no state of`} {
if !strings.Contains(got, want) {
t.Errorf("not refused: %q in %s", want, got)
}
}
}
// Two kinds on one machine are two holders, and one kind on two machines is a second claimant.
func TestEachKindIsItsOwnHolderWhenResolved(t *testing.T) {
modules := []Manifest{aChannel("telegram", "telegram"), aChannel("desk-channel", "desktop")}
@@ -93,76 +36,3 @@ func TestEachKindIsItsOwnHolderWhenResolved(t *testing.T) {
t.Fatal("one kind held on two machines was not refused")
}
}
// A channel's capabilities come from the fixed vocabulary, and only a kinded claim carries any.
func TestCapabilitiesAreTheVocabularysAndOnlyOnAKindedClaim(t *testing.T) {
good := aChannel("telegram", "telegram")
good.Claims[0].Capabilities = []string{"deliver", "choice", "verified-sender", "max-length:4096"}
good.RunsAs = "telegram"
if got := problemsFor(t, Shelf{"messenger": router(), "telegram": good}); got != "" {
t.Fatalf("the vocabulary was refused: %s", got)
}
bad := aChannel("telegram", "telegram")
bad.Claims[0].Capabilities = []string{"trusted", "max-length:lots"}
got := problemsFor(t, Shelf{"messenger": router(), "telegram": bad})
for _, w := range []string{`"trusted"`, `"max-length:lots"`} {
if !strings.Contains(got, w+", which channel-capabilities/1 does not have") {
t.Errorf("%s was not refused: %s", w, got)
}
}
odd := Manifest{Module: "odd", Claims: []Claim{{Name: "operator-channel", Scope: ScopeMesh, Capabilities: []string{"deliver"}}}}
if got := problemsFor(t, Shelf{"messenger": router(), "odd": odd}); !strings.Contains(got, "only a kinded bench's claim carries them") {
t.Errorf("capabilities on a seat that is not kinded stood: %s", got)
}
}
// novox/hq ADR 0259 §8: a module saying warrants, or speaking for a kind that proves its sender, runs as an
// account of its own — never carried by the machine's runtime, which runs as the operator's account.
func TestATrustedHolderMustRunAsAnAccountOfItsOwn(t *testing.T) {
r := router()
r.RunsAs = ""
if got := problemsFor(t, Shelf{"messenger": r}); !strings.Contains(got, "messenger must run as an account of its own") {
t.Errorf("a router on the machine's runtime stood: %s", got)
}
tg := aChannel("telegram", "telegram")
tg.Claims[0].Capabilities = []string{"choice", "verified-sender"}
if got := problemsFor(t, Shelf{"messenger": router(), "telegram": tg}); !strings.Contains(got, "telegram must run as an account of its own") {
t.Errorf("a verified channel on the machine's runtime stood: %s", got)
}
desk := aChannel("desk-channel", "desktop")
desk.Claims[0].Capabilities = []string{"choice"}
if got := problemsFor(t, Shelf{"messenger": router(), "desk-channel": desk}); got != "" {
t.Errorf("a channel proving nothing was held to it: %s", got)
}
// A kind that is `private` shows a link's code, which links an account as the operator: its holder is
// trusted with it, so it runs as its own account too (the confirmation review of 2026-10-09).
private := aChannel("desk-channel", "desktop")
private.Claims[0].Capabilities = []string{"choice", "private"}
if got := problemsFor(t, Shelf{"messenger": router(), "desk-channel": private}); !strings.Contains(got, "desk-channel must run as an account of its own") {
t.Errorf("a private channel on the machine's runtime stood: %s", got)
}
}
func TestRunsAsIsAnAccountOfTheModulesOwn(t *testing.T) {
ok := Manifest{Module: "telegram", RunsAs: "telegram", SecretsOwner: "telegram",
OwnSecrets: OwnSecrets{"broker": {Path: "/var/lib/telegram/broker"}},
Resources: []map[string]any{{"id": "account", "type": "user", "name": "telegram"}}}
if got := RunsAsProblems(ok); len(got) != 0 {
t.Fatalf("a sound runs-as was refused: %v", got)
}
for want, change := range map[string]func(*Manifest){
"never root": func(m *Manifest) { m.RunsAs, m.SecretsOwner = "root", "root" },
"not an account name": func(m *Manifest) { m.RunsAs = "${machine:account}" },
"which it does not make": func(m *Manifest) { m.Resources = nil },
"declares no own secret": func(m *Manifest) { m.OwnSecrets = nil },
"they are the account's own": func(m *Manifest) { m.SecretsOwner = "" },
} {
m := ok
m.Resources = append([]map[string]any(nil), ok.Resources...)
m.OwnSecrets = OwnSecrets{"broker": {Path: "/x"}}
change(&m)
if got := strings.Join(RunsAsProblems(m), "; "); !strings.Contains(got, want) {
t.Errorf("want %q, got %q", want, got)
}
}
}
-14
View File
@@ -118,20 +118,6 @@ func TestNoLocalNetworkIsSaidAndNotWidened(t *testing.T) {
}
}
// `local-network` is a source a manifest may name (§1, changing ADR 0045 §3), and an unknown source is
// still refused.
func TestAManifestMayListenFromTheLocalNetwork(t *testing.T) {
if _, err := ParseManifest([]byte(`{"module":"mosquitto","version":"1",
"listens":[{"port":1883,"from":"local-network"}]}`)); err != nil {
t.Fatalf("a listen from the local network is refused: %v", err)
}
_, err := ParseManifest([]byte(`{"module":"mosquitto","version":"1",
"listens":[{"port":1883,"from":"lan"}]}`))
if err == nil || !strings.Contains(err.Error(), `from "lan"`) {
t.Fatalf("a source the mesh cannot render is not refused: %v", err)
}
}
// **A fourth reach** (§4, changing ADR 0138 §4): `internal` with the local networks added to what the
// filter opens — the internal name, not the public one.
func TestTheLocalNetworkReachIsInternalWithTheLocalNetworkAdded(t *testing.T) {
-31
View File
@@ -87,34 +87,3 @@ func (l LogDeclaration) MarshalJSON() ([]byte, error) {
type plain LogDeclaration
return json.Marshal(plain(l))
}
// LogProblems is what is wrong with a manifest's logs.
//
// Refused at registration, for a bucket's reason: a stream name the bus cannot hold, or a cap the
// mesh would not grant, is a module that installs, starts, and is refused on its first append.
func LogProblems(m Manifest) []string {
var problems []string
if len(m.Logs) > 0 && !stateName.MatchString(m.Module) {
problems = append(problems, fmt.Sprintf(
"%s keeps a log, and a module's name is part of its logs' names, which take one plain "+
"name — no dot (novox/hq ADR 0297)", m.Module))
}
seen := map[string]bool{}
for _, l := range m.Logs {
switch {
case !stateName.MatchString(l.Name):
problems = append(problems, fmt.Sprintf(
"%s keeps log %q: a log is named locally — lower-case letters, digits and hyphens, "+
"no dot and no underscore; the mesh derives the stream (novox/hq ADR 0297)", m.Module, l.Name))
case seen[l.Name]:
problems = append(problems, fmt.Sprintf("%s keeps log %q twice", m.Module, l.Name))
}
seen[l.Name] = true
if l.MaxMiB != 0 && (l.MaxMiB < LogLeastMiB || l.MaxMiB > LogMostMiB) {
problems = append(problems, fmt.Sprintf(
"%s caps log %q at %d MiB; a log holds between %d and %d MiB (novox/hq ADR 0297)",
m.Module, l.Name, l.MaxMiB, LogLeastMiB, LogMostMiB))
}
}
return problems
}
+3 -41
View File
@@ -2,14 +2,13 @@ package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// A module declares the logs it keeps (novox/hq ADR 0297 §1): a log by its bare name, or with its cap
// in MiB; the default cap where none is said.
func TestAManifestMaySayWhatLogsItKeeps(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-issues","version":"1",` +
m, err := decodeForTest([]byte(`{"module":"mesh-issues","version":"1",` +
`"logs":["changes",{"name":"moves","max-mib":2048}]}`))
if err != nil {
t.Fatal(err)
@@ -25,7 +24,7 @@ func TestAManifestMaySayWhatLogsItKeeps(t *testing.T) {
t.Fatalf("written back as %s", out)
}
// 1953 S-age (ADR 0565 §10): a log may keep its entries for an age, written back with it.
aged, err := ParseManifest([]byte(`{"module":"mesh-hand-acts","version":"1","logs":[{"name":"acts","max-age-days":400}]}`))
aged, err := decodeForTest([]byte(`{"module":"mesh-hand-acts","version":"1","logs":[{"name":"acts","max-age-days":400}]}`))
if err != nil || aged.Logs[0].MaxAgeDays != 400 {
t.Fatalf("an aged log read as %+v, %v", aged.Logs, err)
}
@@ -33,45 +32,8 @@ func TestAManifestMaySayWhatLogsItKeeps(t *testing.T) {
t.Fatalf("an aged log written back as %s", out)
}
for _, edge := range []string{`{"name":"a","max-mib":1}`, `{"name":"a","max-mib":8192}`, `{"name":"a","max-age-days":3650}`} {
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","logs":[` + edge + `]}`)); err != nil {
if _, err := decodeForTest([]byte(`{"module":"a","version":"1","logs":[` + edge + `]}`)); err != nil {
t.Errorf("%s is inside the caps and was refused: %v", edge, err)
}
}
}
// The catalogue check refuses a log outside the caps, with a name that is not one plain token, with a
// field the mesh does not know, or kept by a module whose own name is not one plain token.
func TestALogIsNamedLocallyAndCapped(t *testing.T) {
for _, c := range []struct{ manifest, says string }{
{`{"module":"a","version":"1","logs":["mesh.changes"]}`, `keeps log "mesh.changes": a log is named locally`},
{`{"module":"a","version":"1","logs":["my_changes"]}`, `keeps log "my_changes"`},
{`{"module":"a","version":"1","logs":["Changes"]}`, `keeps log "Changes"`},
{`{"module":"a","version":"1","logs":["c","c"]}`, `keeps log "c" twice`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":8193}]}`, `between 1 and 8192 MiB`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":-1}]}`, `between 1 and 8192 MiB`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-mib":0}]}`, `max-mib is between 1 and 8192, not 0`},
{`{"module":"a","version":"1","logs":[{"name":"c","stream":"LOG_x"}]}`, `{name, max-mib, max-age-days}`},
{`{"module":"a","version":"1","logs":[{"name":"c","max-age-days":0}]}`, `max-age-days is between 1 and 3650, not 0`},
{`{"module":"a.b","version":"1","logs":["c"]}`, `no dot`},
} {
_, err := ParseManifest([]byte(c.manifest))
if err == nil {
t.Errorf("%s was accepted", c.manifest)
continue
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("%s refused for the wrong reason: %v", c.manifest, err)
}
}
}
// **Across the whole catalogue**: every log is named locally and capped within the mesh's caps.
func TestEveryManifestsLogsAreLocalAndCapped(t *testing.T) {
var problems []string
for _, m := range theCatalogue(t) {
problems = append(problems, LogProblems(m)...)
}
if len(problems) > 0 {
t.Fatalf("the catalogue's logs are not what ADR 0297 says:\n %s", strings.Join(problems, "\n "))
}
}
+1 -1295
View File
File diff suppressed because it is too large Load Diff
+4 -29
View File
@@ -22,20 +22,9 @@ func manifestMounting(from string, beside string) []byte {
return []byte(`{"module":"store","resources":[` + strings.Replace(res, "%s", from, 1) + `]}`)
}
func TestAMountNothingDeclaresIsRefused(t *testing.T) {
_, err := ParseManifest(manifestMounting("/services/store/data", ""))
if err == nil {
t.Fatal("a container mounting a path no resource declares was accepted; the runtime " +
"would create it as root and the module's owner and mode would never apply")
}
if !strings.Contains(err.Error(), "/services/store/data") {
t.Fatalf("refused without naming the path, which leaves the author guessing: %v", err)
}
}
// Declaring it is enough — including declaring the directory above it.
func TestAMountUnderADeclaredDirectoryIsAccepted(t *testing.T) {
_, err := ParseManifest(manifestMounting("/services/store/data",
_, err := decodeForTest(manifestMounting("/services/store/data",
`{"id":"state","type":"directory","path":"/services/store","mode":"0700"}`))
if err != nil {
t.Fatalf("a module that said where its data lives was refused anyway: %v", err)
@@ -45,27 +34,13 @@ func TestAMountUnderADeclaredDirectoryIsAccepted(t *testing.T) {
// The operator's path, granted for use (ADR 0051), is declared by `accesses`, not by a directory the
// module would then own.
func TestAMountOfAnAccessedPathIsAccepted(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"store","accesses":[{"path":"/services/media/movies","mode":"read"}],` +
_, err := decodeForTest([]byte(`{"module":"store","accesses":[{"path":"/services/media/movies","mode":"read"}],` +
`"resources":[` + strings.Replace(aContainerMounting, "%s", "/services/media/movies", 1) + `]}`))
if err != nil {
t.Fatalf("a mount of a path the module declares it accesses was refused: %v", err)
}
}
// The machine's facility — the container runtime's socket — is granted by the capability that names
// it, and declaring it as the module's own directory would be a lie the host would act on. This is
// the case the first version of this check refused, and was withdrawn for: the builder.
func TestTheRuntimeSocketIsGrantedByTheCapabilityAndNotOtherwise(t *testing.T) {
granted := `{"module":"builder","capabilities":["container-runtime"],"resources":[` +
strings.Replace(aContainerMounting, "%s", "/var/run/docker.sock", 1) + `]}`
if _, err := ParseManifest([]byte(granted)); err != nil {
t.Fatalf("a module with the container-runtime capability may mount its socket: %v", err)
}
if _, err := ParseManifest(manifestMounting("/var/run/docker.sock", "")); err == nil {
t.Fatal("a module mounted the container runtime's socket without declaring the capability, and was accepted")
}
}
// **Every manifest in the catalogue (internal/beside) passes**, so the rule is not one the catalogue
// is already breaking.
func TestEveryCatalogueManifestDeclaresWhatItMounts(t *testing.T) {
@@ -79,7 +54,7 @@ func TestEveryCatalogueManifestDeclaresWhatItMounts(t *testing.T) {
// Where a bound fact lands is the mesh's file too, and a container may mount it directly.
func TestAMountOfABoundFactIsAccepted(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"store","requires":["model-access"],` +
_, err := decodeForTest([]byte(`{"module":"store","requires":["model-access"],` +
`"binds":{"model-access":"/var/lib/store/model.json"},"resources":[` +
strings.Replace(aContainerMounting, "%s", "/var/lib/store/model.json", 1) + `]}`))
if err != nil {
@@ -90,7 +65,7 @@ func TestAMountOfABoundFactIsAccepted(t *testing.T) {
// The file a filter module's rule set is written to is declared by `filtering.into` (novox/hq ADR
// 0169): the module's runtime mounts it to reload the mesh's own table, and nothing else declares it.
func TestAMountOfTheFilterFileIsDeclaredByFilteringInto(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"nftables","filtering":{"into":"/etc/nftables.conf"},` +
_, err := decodeForTest([]byte(`{"module":"nftables","filtering":{"into":"/etc/nftables.conf"},` +
`"resources":[` + strings.Replace(aContainerMounting, "%s", "/etc/nftables.conf", 1) + `]}`))
if err != nil {
t.Fatalf("a filter module mounting its own filter file was refused: %v", err)
-16
View File
@@ -1,17 +1 @@
package catalogue
// Provision names that say a role where the consumer is coupled to an implementation.
//
// **A consumer's code is written against PostgreSQL or against SQL Server, never against a
// database** (novox/hq ADR 0027). A requirement naming the role matches any engine, resolves as
// satisfied, deploys, and fails on the first query — with nothing connecting the error back to a
// match made elsewhere by something that thought it had done its job.
//
// Refused rather than documented, because the previous naming *was* the documented convention.
// A rule states how it is checked (novox/hq 00-META/how-we-build.md §5).
var engineGeneric = map[string]string{
"database": "postgres-database, mssql-database, mariadb-database",
"db": "postgres-database, mssql-database, mariadb-database",
"sql": "postgres-database, mssql-database, mariadb-database",
"sql-database": "postgres-database, mssql-database, mariadb-database",
}
+2 -47
View File
@@ -1,36 +1,14 @@
package catalogue
import (
"strings"
"testing"
)
// A provision names what the consumer is coupled to (novox/hq ADR 0027).
//
// The fault this defends against does not look like a fault: resolution succeeds, the module
// deploys, and the first query fails somewhere else entirely. Every earlier test had exactly one
// provider of each name, so no mismatch was expressible and none was caught.
func TestAModuleMayNotProvideAGenericDatabase(t *testing.T) {
for _, hidden := range []string{"database", "db", "sql", "sql-database"} {
_, err := ParseManifest([]byte(`{"module":"postgres","version":"1",
"provides":[{"name":"` + hidden + `","scope":"mesh"}]}`))
if err == nil {
t.Fatalf("providing %q was accepted; a requirement for it matches any engine", hidden)
}
for _, want := range []string{hidden, "postgres-database", "first query"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("refusing %q did not mention %q, so nobody learns what to write:\n%v",
hidden, want, err)
}
}
}
}
// The rule is about coupling, not about specificity everywhere. Naming `route` after a particular
// proxy would be the same error in the other direction.
func TestARoleNameIsFineWhereTheConsumerCannotTellTheDifference(t *testing.T) {
for _, role := range []string{"route", "resolver", "artifact-store", "postgres-database"} {
if _, err := ParseManifest([]byte(`{"module":"m","version":"1",
if _, err := decodeForTest([]byte(`{"module":"m","version":"1",
"provides":[{"name":"` + role + `","scope":"mesh"}]}`)); err != nil {
t.Errorf("providing %q was refused, and it names what a consumer actually gets: %v",
role, err)
@@ -38,32 +16,9 @@ func TestARoleNameIsFineWhereTheConsumerCannotTellTheDifference(t *testing.T) {
}
}
// A module may not declare an action, and it is refused where it was written.
//
// The host refuses one arriving over the link, which is the bound the whole security argument
// rests on (novox/hq ADR 0005) and is enforced in the right place. But a module's resources reach
// a machine *over* the link, so a manifest carrying an action used to be accepted here, stored,
// resolved and pushed — and then refused on the machine, in a log, with nothing tying it back to
// the manifest. Enforced at the far end only is enforced and not usable.
func TestAModuleMayNotDeclareAnAction(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"probe","version":"1","resources":[
{"id":"migrate","type":"action","command":["/bin/true"],"verify":["/bin/true"]}]}`))
if err == nil {
t.Fatal("a manifest declaring an action was accepted, and the machine would refuse it")
}
if !strings.Contains(err.Error(), "may not") || !strings.Contains(err.Error(), "0005") {
t.Errorf("the refusal does not say what rule it is: %v", err)
}
// And it says what to do instead, because "you may not" without an alternative is where a
// module author stops.
if !strings.Contains(err.Error(), "ships a program") {
t.Errorf("the refusal names no alternative: %v", err)
}
}
// Every other shape a module may declare still passes, so the check above bounds one thing.
func TestTheOtherShapesAreStillAccepted(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"probe","version":"1","resources":[
_, err := decodeForTest([]byte(`{"module":"probe","version":"1","resources":[
{"id":"d","type":"directory","path":"/var/lib/probe","mode":"0700"},
{"id":"f","type":"file","path":"/var/lib/probe/x","content":"x\n"},
{"id":"n","type":"network","name":"probe"},
-41
View File
@@ -60,17 +60,6 @@ func TestADeclaredNeedThatWasNotMadeIsRefused(t *testing.T) {
}
}
func TestANeedIsAnAbsolutePath(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"postgres","version":"1",
"own-secrets":{"superuser":"superuser.txt"}}`))
if err == nil {
t.Fatal("a relative path was accepted")
}
if !strings.Contains(err.Error(), "absolute path") {
t.Fatalf("unhelpful refusal: %v", err)
}
}
func TestAModuleMayNeedSeveralThings(t *testing.T) {
// A password and a token, say. Telling them apart is the module's business, not the mesh's.
m := needy()
@@ -92,33 +81,3 @@ func TestAModuleMayNeedSeveralThings(t *testing.T) {
t.Fatalf("two needs did not land as two secrets: %v", seen)
}
}
// A manifest written against the old name is told what the field became.
//
// `needs` and `secrets` were both name-to-path and differed only in whose secret it was, so
// reaching for the wrong one parsed cleanly and failed somewhere else entirely. Refusing the old
// name with "unknown field" would be correct and unhelpful: whoever wrote it knew what they meant,
// and the mesh knows what it is called now.
func TestAManifestUsingTheOldNameIsToldTheNewOne(t *testing.T) {
_, err := ParseManifest([]byte(
`{"module":"postgres","version":"1","needs":{"superuser":"/var/lib/superuser"}}`))
if err == nil {
t.Fatal("a manifest using the old name was accepted, so two fields now mean one thing")
}
for _, want := range []string{"needs", "own-secrets"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal does not mention %q: %v", want, err)
}
}
}
// And an ordinary unknown key is still refused as one, rather than being guessed at.
func TestAnInventedKeyIsNotTreatedAsARename(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"postgres","version":"1","nonsense":{}}`))
if err == nil {
t.Fatal("an invented key was accepted")
}
if strings.Contains(err.Error(), "is now called") {
t.Fatalf("an invented key was reported as a rename: %v", err)
}
}
-19
View File
@@ -74,25 +74,6 @@ func TestNoManifestContainsASubject(t *testing.T) {
t.Logf("%d manifests hold no subject", checked)
}
// **Every module's event names are what design 29 says, across the whole catalogue.**
//
// The rule above holds by construction and turned out to be weaker than it reads: a manifest holds
// no subject, and every manifest in the catalogue still held the old bus's routing key, which
// derives into a namespace nobody owns (novox/hq 04-ISSUES/127). Nothing failed — the services
// started and none of them reacted. This is the check that was missing.
func TestEveryManifestsEventNamesAreLocal(t *testing.T) {
manifests := theCatalogue(t)
var problems []string
for _, m := range manifests {
problems = append(problems, EventProblems(m)...)
}
if len(problems) > 0 {
t.Fatalf("the catalogue holds %d event name(s) the mesh would derive wrongly:\n %s",
len(problems), strings.Join(problems, "\n "))
}
}
// theCatalogue is every manifest of the catalogue internal/beside finds, parsed the way registration
// parses one.
func theCatalogue(t *testing.T) []Manifest {
-99
View File
@@ -193,102 +193,3 @@ func withoutAbsent(resources []map[string]any, absent map[string]bool) []map[str
}
return out
}
// secretNamesOf is every `${secret:…}` name that carries a credential of this requirement: its own
// name, and each local name it keeps several under (ADR 0094).
func secretNamesOf(m Manifest, requirement string) []string {
names := []string{requirement}
for local := range m.SecretsMany[requirement] {
names = append(names, local)
}
sort.Strings(names)
return names
}
// OptionalProblems is what the module check refuses of a manifest's optional requirements and its
// only-when-bound resources (ADR 0330 §4–§6).
func OptionalProblems(m Manifest) []string {
var problems []string
required := map[string]bool{}
for _, r := range m.Requires {
if required[r] && m.IsOptional(r) {
problems = append(problems, fmt.Sprintf("%s requires %q twice; say it once, optional or not", m.Module, r))
}
required[r] = true
}
for _, r := range optionalRequiresList(m) {
if !required[r] {
problems = append(problems, fmt.Sprintf(
"%s says %q is optional in %s and does not require it", m.Module, r, OptionalRequiresKey))
}
}
// A contribution is to whoever answers the requirement, so it goes with it: contributing to one
// that is optional is allowed, and composed only while it is bound.
for _, r := range m.Resources {
id := fmt.Sprint(r["id"])
when, marked := r[OnlyWhenBound]
on := ""
if marked {
name, ok := when.(string)
switch {
case !ok || name == "":
problems = append(problems, fmt.Sprintf(
"%s's resource %s says %s %v; it names one requirement the module requires optionally",
m.Module, id, OnlyWhenBound, when))
continue
case fmt.Sprint(r["type"]) != "file":
problems = append(problems, fmt.Sprintf(
"%s's resource %s is a %v and says %s: only a file is left out while a requirement is "+
"absent (novox/hq ADR 0330 §5)", m.Module, id, r["type"], OnlyWhenBound))
continue
case !required[name]:
problems = append(problems, fmt.Sprintf(
"%s's resource %s is delivered only when %q is bound, and %s does not require it",
m.Module, id, name, m.Module))
continue
case !m.IsOptional(name):
problems = append(problems, fmt.Sprintf(
"%s's resource %s is delivered only when %q is bound, which is a hard requirement and "+
"always bound: mark the requirement optional, or drop %s", m.Module, id, name, OnlyWhenBound))
continue
}
on = name
}
if fmt.Sprint(r["type"]) != "file" {
continue
}
content, _ := r["content"].(string)
for _, use := range bound.FindAllStringSubmatch(content, -1) {
provision, key, hasDefault := use[1], use[2], use[3] != ""
if !m.IsOptional(provision) {
if hasDefault {
problems = append(problems, fmt.Sprintf(
"%s's resource %s gives ${bound:%s:%s} a default, and %q is a hard requirement, always "+
"bound: a default it never renders only misleads a reader", m.Module, id, provision, key, provision))
}
continue
}
if !hasDefault && on != provision {
problems = append(problems, fmt.Sprintf(
"%s's resource %s says ${bound:%s:%s}, and %q is optional: while it is absent this would "+
"be empty. Name its value then — ${bound:%s:%s|default:<value>} — or deliver the file "+
"only while it is bound, with \"%s\": %q (novox/hq ADR 0330 §4)",
m.Module, id, provision, key, provision, provision, key, OnlyWhenBound, provision))
}
}
for _, provision := range m.Requires {
if !m.IsOptional(provision) || on == provision {
continue
}
for _, name := range secretNamesOf(m, provision) {
if strings.Contains(content, "${secret:"+name+"}") {
problems = append(problems, fmt.Sprintf(
"%s's resource %s says ${secret:%s}, the credential of %q, which is optional and not "+
"delivered while absent: deliver the file only while it is bound, with \"%s\": %q "+
"(novox/hq ADR 0330 §6)", m.Module, id, name, provision, OnlyWhenBound, provision))
}
}
}
}
return problems
}
+1 -85
View File
@@ -54,60 +54,12 @@ func TestARequirementIsHardUnlessMarkedOptional(t *testing.T) {
}
}
// The stored form reads on a controller older than optional requirements (review of mesh-controller
// #281). Inventory.Catalogue stops at the first manifest it cannot read, so a long-form entry inside
// `requires` would have stopped every plan after a rollback. Stored, `requires` is plain names, as main's
// manifest type reads it, and which are optional sits beside it in a key such a controller sets aside.
func TestTheStoredFormReadsWhereRequiresIsAListOfNames(t *testing.T) {
m := mustParse(t, `{"module":"grafana","version":"1",
"requires":["route",{"name":"influxdb-api","optional":true}]}`)
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
// main's field, exactly: `Requires []string`.
var older struct {
Requires []string `json:"requires"`
}
if err := json.Unmarshal(raw, &older); err != nil {
t.Fatalf("a controller reading requires as names cannot read the stored form: %v\n%s", err, raw)
}
if strings.Join(older.Requires, " ") != "route influxdb-api" {
t.Fatalf("it reads the requirements as %v", older.Requires)
}
var keys map[string]json.RawMessage
if err := json.Unmarshal(raw, &keys); err != nil {
t.Fatal(err)
}
if string(keys[OptionalRequiresKey]) != `["influxdb-api"]` {
t.Fatalf("which are optional is stored as %s", keys[OptionalRequiresKey])
}
// And this controller reads the stored form back the same, leniently and strictly.
var back Manifest
if err := json.Unmarshal(raw, &back); err != nil {
t.Fatal(err)
}
strict, err := ParseManifest(raw)
if err != nil {
t.Fatal(err)
}
for _, got := range []Manifest{back, strict} {
if !got.IsOptional("influxdb-api") || got.IsOptional("route") || len(got.Requires) != 2 {
t.Fatalf("read back as %v optional %v", got.Requires, got.OptionalRequires)
}
}
// A stored list naming something the module does not require is refused.
if _, err := ParseManifest([]byte(`{"module":"x","version":"1","requires":["route"],"optional-requires":["mqtt-topic"]}`)); err == nil {
t.Error("an optional requirement the module does not require passed the check")
}
}
func TestARequirementsLongFormRefusesWhatItDoesNotKnow(t *testing.T) {
for _, raw := range []string{
`{"module":"x","version":"1","requires":[{"name":"influxdb-api","optinal":true}]}`,
`{"module":"x","version":"1","requires":[{"optional":true}]}`,
} {
if _, err := ParseManifest([]byte(raw)); err == nil {
if _, err := decodeForTest([]byte(raw)); err == nil {
t.Errorf("accepted %s", raw)
}
}
@@ -202,42 +154,6 @@ func optionalModule(resources string) string {
"resources":[` + resources + `]}`
}
func TestTheModuleCheckRefusesAnOptionalPlaceholderThatCouldRenderEmpty(t *testing.T) {
refused := map[string]string{
"a placeholder without a default": `{"id":"ds","type":"file","path":"/a","mode":"0644",
"content":"url: ${bound:influxdb-api:at}"}`,
"a secret outside an only-when-bound file": `{"id":"s","type":"file","path":"/a","mode":"0400",
"content":"${secret:influxdb-api}"}`,
"only-when-bound on a hard requirement": `{"id":"r","type":"file","path":"/a","mode":"0644",
"content":"x","only-when-bound":"route"}`,
"only-when-bound on something not required": `{"id":"r","type":"file","path":"/a","mode":"0644",
"content":"x","only-when-bound":"mqtt-topic"}`,
"only-when-bound on a directory": `{"id":"d","type":"directory","mode":"0700",
"only-when-bound":"influxdb-api"}`,
"a default on a hard requirement": `{"id":"r","type":"file","path":"/a","mode":"0644",
"content":"${bound:route:name|default:x}"}`,
}
for what, resource := range refused {
if _, err := ParseManifest([]byte(optionalModule(resource))); err == nil {
t.Errorf("%s passed the module check", what)
} else if what == "a placeholder without a default" && !strings.Contains(err.Error(), "|default:") {
t.Errorf("the refusal does not say how to write it: %v", err)
}
}
accepted := map[string]string{
"a placeholder with a default": `{"id":"ds","type":"file","path":"/a","mode":"0644",
"content":"url: ${bound:influxdb-api:at|default:none}"}`,
"a placeholder and a secret inside an only-when-bound file": `{"id":"ds","type":"file","path":"/a",
"mode":"0400","content":"url: ${bound:influxdb-api:at}\npassword: ${secret:influxdb-api}",
"only-when-bound":"influxdb-api"}`,
}
for what, resource := range accepted {
if _, err := ParseManifest([]byte(optionalModule(resource))); err != nil {
t.Errorf("%s was refused: %v", what, err)
}
}
}
func absentGrafana(t *testing.T, absent bool) Resolution {
t.Helper()
m := mustParse(t, optionalModule(`
+1 -1
View File
@@ -11,7 +11,7 @@ import (
func outsideManifest(t *testing.T) Manifest {
t.Helper()
m, err := ParseManifest([]byte(`{"module":"assistant","version":"1",
m, err := decodeForTest([]byte(`{"module":"assistant","version":"1",
"own-secrets":{
"password":{"path":"/var/lib/assistant/password.secret","taken":"at-start"},
"vendor-key":{"path":"/var/lib/assistant/vendor-key.secret","taken":"at-start","issued-by":"outside"}},
+1 -1
View File
@@ -14,7 +14,7 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
if err != nil {
t.Fatal(err)
}
m, err := ParseManifest(raw)
m, err := decodeForTest(raw)
if err != nil {
t.Fatalf("mesh-sdk's module.json does not parse: %v", err)
}
+1 -19
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"strings"
"testing"
)
@@ -20,7 +19,7 @@ func mailSenderManifest(t *testing.T, own string) Manifest {
ownSecrets += `,"owner":"` + own + `"`
}
ownSecrets += `}`
m, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","secrets-owner":"mesh-mail","own-secrets":{` +
m, err := decodeForTest([]byte(`{"module":"mesh-mail","version":"1","secrets-owner":"mesh-mail","own-secrets":{` +
ownSecrets + `},"resources":[{"id":"account","type":"user","name":"mesh-mail-send"}]}`))
if err != nil {
t.Fatalf("a manifest with own=%q was refused: %v", own, err)
@@ -73,23 +72,6 @@ func TestAnOwnSecretsOwnerIsGivenToItsFiles(t *testing.T) {
}
}
// 1170/AC2: an owner naming an account the module does not make, root, or the operator account's placeholder is
// refused, naming the secret and the account.
func TestAnOwnSecretsOwnerIsRefusedWhenItIsNotAnAccountTheModuleMakes(t *testing.T) {
for name, c := range map[string]struct{ owner, says string }{
"root": {"root", "root"},
"the operator account": {"${machine:account}", "operator account"},
"an account not declared": {"not-made", "not-made"},
} {
_, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{` +
`"mail-credential-*":{"path":"/var/lib/mesh-mail/mail-credential-*.secret","issued-by":"outside","owner":"` +
c.owner + `"}},"resources":[{"id":"account","type":"user","name":"mesh-mail-send"}]}`))
if err == nil || !strings.Contains(err.Error(), "mail-credential-*") || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v; want a refusal naming the secret and %q", name, err, c.says)
}
}
}
// 1170/AC3: a manifest with no owner on any own secret composes exactly as it did before this clause existed:
// every secret file belongs to the module's secrets-owner (or nothing, when that is absent too).
func TestNoOwnerOnAnySecretComposesAsMainDoes(t *testing.T) {
+3 -31
View File
@@ -10,7 +10,7 @@ import (
// its bare name (novox/hq issue 789): reviewed in the module's pull request, and the controller validates it in
// the module check.
func TestAnOwnSecretsPromptParsesAndIsWrittenBack(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
m, err := decodeForTest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
"mail-credential-*":{"path":"/var/lib/mesh-mail/mail-credential-*.secret","issued-by":"outside",
"prompt":"the password of the mail account {member}"},
"smtp-relay-key":{"path":"/var/lib/mesh-mail/smtp-relay-key","taken":"at-start",
@@ -41,7 +41,7 @@ func TestAnOwnSecretsPromptParsesAndIsWrittenBack(t *testing.T) {
// No prompt declared is fine: the desk asks by name alone, as it does today.
func TestAnOwnSecretWithNoPromptIsFine(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"telegram","version":"1","own-secrets":{
m, err := decodeForTest([]byte(`{"module":"telegram","version":"1","own-secrets":{
"telegram-token":{"path":"/var/lib/telegram/telegram-token","issued-by":"outside"}}}`))
if err != nil {
t.Fatal(err)
@@ -65,7 +65,7 @@ func TestAPromptOfMultiByteCharactersIsCountedByCharacter(t *testing.T) {
if n := len(prompt); n <= 120 {
t.Fatalf("the test prompt is %d bytes, not over 120 as intended", n)
}
_, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
_, err := decodeForTest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
"smtp-relay-key":{"path":"/var/lib/mesh-mail/smtp-relay-key","taken":"at-start",
"prompt":` + jsonString(prompt) + `}}}`))
if err != nil {
@@ -73,34 +73,6 @@ func TestAPromptOfMultiByteCharactersIsCountedByCharacter(t *testing.T) {
}
}
// {member} is replaced only for a family's member: a single secret's prompt may not hold it (novox/hq issue 789).
func TestAPromptHoldingMemberOnASingleSecretIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
"smtp-relay-key":{"path":"/var/lib/mesh-mail/smtp-relay-key","taken":"at-start",
"prompt":"For {member}"}}}`))
if err == nil || !strings.Contains(err.Error(), "{member} is replaced only for a family's member") {
t.Fatalf("a single secret's prompt holding {member}: %v; want a refusal saying so", err)
}
}
// A prompt that is not one short line naming only {member} is refused by the module check (novox/hq issue 789).
func TestAMalformedPromptIsRefused(t *testing.T) {
for name, c := range map[string]struct{ prompt, says string }{
"a newline": {"the password of\nthe mail account {member}", "one line"},
"over 120 characters": {"the password of the mail account {member}, which this mesh needs to log in to " +
"send and receive mail for the operator's account on this machine today", "120 characters"},
"a placeholder other than {member}": {"the password of the mail account {name}", "{member}"},
"two placeholders, one unknown": {"the {kind} secret of the mail account {member}", "{member}"},
} {
_, err := ParseManifest([]byte(`{"module":"mesh-mail","version":"1","own-secrets":{
"mail-credential-*":{"path":"/var/lib/mesh-mail/mail-credential-*.secret","issued-by":"outside",
"prompt":` + jsonString(c.prompt) + `}}}`))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v; want a refusal saying %q", name, err, c.says)
}
}
}
// jsonString quotes a Go string the way encoding/json would, for building a literal into a test's manifest JSON.
func jsonString(s string) string {
out, err := json.Marshal(s)
+1 -53
View File
@@ -10,7 +10,7 @@ import (
// nothing of it, an object says `at-start` or `applied`, and the bytes the mesh holds are the bytes
// it was given either way.
func TestAnOwnSecretSaysHowItIsTaken(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
m, err := decodeForTest([]byte(`{"module":"idp","version":"1","own-secrets":{
"broker":"/var/lib/mesh/idp/broker",
"admin":{"path":"/var/lib/idp/admin.secret","taken":"applied"},
"session":{"path":"/var/lib/idp/session.secret","taken":"at-start"}}}`))
@@ -39,55 +39,3 @@ func TestAnOwnSecretSaysHowItIsTaken(t *testing.T) {
t.Fatalf("a path alone is written back as a path: %s", out)
}
}
func TestAnOwnSecretTakenSomeOtherWayIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","taken":"sometimes"}}}`))
if err == nil || !strings.Contains(err.Error(), `taken "sometimes"`) {
t.Fatalf("an unknown word for how a secret is taken was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","rotate":"yes"}}}`))
if err == nil {
t.Fatal("an unknown field on an own secret was accepted")
}
}
// A secret an outside party issues says so (novox/hq ADR 0228), is written back as it was read, and
// is the one at-start secret the mesh may not make; any other word for who issued it is refused.
func TestAnOwnSecretSaysWhenAnOutsidePartyIssuesIt(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"letta","version":"1","own-secrets":{
"server-password":{"path":"/var/lib/letta/server-password","taken":"at-start"},
"openai-api-key":{"path":"/var/lib/letta/openai-api-key","taken":"at-start","issued-by":"outside"},
"telegram-token":{"path":"/var/lib/letta/telegram-token","issued-by":"outside"},
"logflare":{"path":"/var/lib/letta/logflare","taken":"applied"},
"broker":"/var/lib/mesh/letta/broker"}}`))
if err != nil {
t.Fatal(err)
}
for name, may := range map[string]bool{"server-password": true, "openai-api-key": false,
"telegram-token": false, "logflare": false, "broker": false} {
if got := m.OwnSecrets[name].MeshMayMake(); got != may {
t.Errorf("%s: the mesh may make it = %v, want %v", name, got, may)
}
}
out, err := json.Marshal(m.OwnSecrets)
if err != nil {
t.Fatal(err)
}
var again OwnSecrets
if err := json.Unmarshal(out, &again); err != nil {
t.Fatal(err)
}
if again["openai-api-key"] != m.OwnSecrets["openai-api-key"] || again["telegram-token"] != m.OwnSecrets["telegram-token"] {
t.Fatalf("the round trip lost who issued it: %s", out)
}
if strings.Contains(string(out), `"taken":""`) {
t.Fatalf("a secret that says only who issued it gained an empty taken: %s", out)
}
_, err = ParseManifest([]byte(`{"module":"letta","version":"1","own-secrets":{
"openai-api-key":{"path":"/var/lib/letta/openai-api-key","issued-by":"openai"}}}`))
if err == nil || !strings.Contains(err.Error(), `issued by "openai"`) {
t.Fatalf("an unknown word for who issued a secret was accepted: %v", err)
}
}
-40
View File
@@ -5,46 +5,6 @@ import (
"testing"
)
// An artifact names the paths it is built from with `paths` (novox/hq ADR 0499 clause 1): a package, or a bundle in a
// language other than Go, may; a Go program's artifact, an archive, an image and an upstream artifact may not, and the
// refusal names the artifact and its kind. Each path stays inside the module's directory: no `..`, no absolute path,
// no pattern.
func TestAnArtifactNamesThePathsItIsBuiltFrom(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-sdk","version":"1","slug":"sdk","build":{"artifacts":[
{"name":"lib","kind":"package","language":"typescript",
"paths":["src","test","package.json","package-lock.json","tsconfig.json"]}]},"resources":[]}`))
if err != nil {
t.Fatalf("a package naming its paths was refused: %v", err)
}
if got := strings.Join(m.Build.Artifacts[0].Paths, ","); got != "src,test,package.json,package-lock.json,tsconfig.json" {
t.Fatalf("the paths were not kept as written: %q", got)
}
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
{"name":"tools","kind":"bundle","language":"typescript","entrypoints":["index.js"],"loads":["index.js"],"paths":["src","package.json"]}]}}`)); err != nil {
t.Fatalf("a TypeScript bundle naming its paths was refused: %v", err)
}
for _, c := range []struct{ artifact, says string }{
{`{"name":"x","kind":"bundle","language":"go","system":"linux-amd64","paths":["cmd"]}`, `"x" is a Go program`},
{`{"name":"x","kind":"archive","from":"files","paths":["files"]}`, `"x" is a "archive"`},
{`{"name":"x","kind":"image","from":"Dockerfile","paths":["src"]}`, `"x" is a "image"`},
{`{"name":"x","kind":"upstream","from":"nats:2.10","paths":["src"]}`, `"x" is a "upstream"`},
{`{"name":"x","kind":"package","language":"typescript","paths":["../other"]}`, "leaves the module's directory"},
{`{"name":"x","kind":"package","language":"typescript","paths":["src/../../x"]}`, "leaves the module's directory"},
{`{"name":"x","kind":"package","language":"typescript","paths":["/etc"]}`, "leaves the module's directory"},
{`{"name":"x","kind":"package","language":"typescript","paths":["src/*.ts"]}`, "is a pattern"},
{`{"name":"x","kind":"package","language":"typescript","paths":["src/[ab]"]}`, "is a pattern"},
{`{"name":"x","kind":"package","language":"typescript","paths":[""]}`, "names an empty path"},
{`{"name":"x","kind":"package","language":"typescript","paths":["."]}`, "names the module's whole directory"},
{`{"name":"x","kind":"package","language":"typescript","paths":[]}`, "names no path"},
} {
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[` + c.artifact + `]}}`))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v, wanted a refusal saying %q", c.artifact, err, c.says)
}
}
}
// The paths a module's tree is taken from are the union its artifacts name, cleaned and sorted, and only when every
// artifact names some (novox/hq ADR 0499 clause 2); a module with an artifact that names none, or with none at all,
// keeps its directory's tree.
+2 -2
View File
@@ -39,12 +39,12 @@ func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
if string(old) == string(now) {
continue
}
m, err := ParseManifest(now)
m, err := decodeForTest(now)
if err != nil {
t.Errorf("%s: %v", module, err)
continue
}
earlier, err := ParseManifest(old)
earlier, err := decodeForTest(old)
if err != nil {
t.Errorf("%s before: %v", module, err)
continue
-147
View File
@@ -169,105 +169,9 @@ func systemPath(path string) string {
return ""
}
// Where no directory or file a module's definition resolves to may be (novox/hq issue 496).
//
// mesh-catalog #205 gave docker's `state` directory `"place": "."`, which resolves to <root>/docker: with the
// default root, /var/lib/docker, every container's filesystem. systemPath judged only the `places` and `accesses`
// settings, so the default layout and a definition's own paths reached the merge gate unjudged; it composed every
// machine and passed, and only the node-engine refused the directory, at apply, failing the walk.
//
// **Judged where a definition is judged, never where a machine is composed.** resolvedPathProblems runs in
// ParseManifest, which every route a definition takes into the mesh passes: `module check`, registration of a
// build (the builder's and the registry verbs'), and the merge gate's reading of the changed repository. A
// refusal there stops one definition before it reaches any machine. Composition reads registered manifests
// without ParseManifest, and judges nothing of this: refusing there would fail the whole machine's declaration and
// freeze every module on it for one module's path, where the node-engine fails only that resource.
//
// **The node-engine's rules, no more** (mesh-host's internal/apply/placement_guard.go, with files judged as
// directories are after novox/hq issue 495, rule 6). A path is refused when it is one of protectedRoots or holds
// one, when it is at or below a tree in forbiddenBelow, or at or below one of engineTrees and its module is not
// the node-engine's. What the engine judges with what only the machine knows stays the engine's: where the
// runtimes really keep their data, links, the accounts' homes, a directory's owner below /etc. The lists are the
// engine's own words, and a test (engine_guard_test.go) holds them equal to mesh-host's beside this repository.
// systemPath stays the stricter rule for a setting: a setting is an operator's word about one machine, and the
// trees it lists (/etc, /usr, /run, the mesh's own) are where the mesh's own modules write by design.
var (
protectedRoots = []string{"/", "/bin", "/boot", "/dev", "/etc", "/home", "/lib", "/lib32", "/lib64",
"/media", "/mnt", "/opt", "/proc", "/root", "/run", "/sbin", "/srv", "/sys", "/tmp", "/usr", "/usr/bin",
"/usr/lib", "/usr/lib64", "/usr/local", "/usr/local/bin", "/usr/local/lib", "/usr/local/sbin", "/usr/sbin",
"/usr/share", "/var", "/var/cache", "/var/lib", "/var/lib/mesh", "/var/log", "/var/run", "/var/tmp",
"/var/spool"}
forbiddenBelow = []string{"/proc", "/sys", "/dev", "/boot", "/root", "/var/spool", "/opt", "/var/lib/docker",
"/var/lib/containers", "/var/lib/containerd"}
engineTrees = []string{"/var/lib/mesh-host", "/usr/lib/nox-mesh-host"}
)
// engineModule is the node-engine's own module, the one that places in engineTrees.
const engineModule = "mesh-host"
// enginePath says why the node-engine refuses a directory or file at path for module, or "".
func enginePath(path, module string) string {
path = filepath.Clean(path)
if !filepath.IsAbs(path) {
return ""
}
atOrBelow := func(tree string) bool { return path == tree || strings.HasPrefix(path, tree+"/") }
for _, root := range protectedRoots {
if path == root || path == "/" || strings.HasPrefix(root, path+"/") {
return root + " is one of the machine's own directories, and owning it is owning everything in it"
}
}
for _, tree := range forbiddenBelow {
if atOrBelow(tree) {
return "nothing is placed in " + tree
}
}
if module != engineModule {
for _, tree := range engineTrees {
if atOrBelow(tree) {
return tree + " is the node-engine's own, placed in by its own module alone"
}
}
}
return ""
}
// resolvedPathProblems is every directory and file of the definition whose path, resolved as a node with the
// default root resolves it, the node-engine would refuse (novox/hq issue 496). A path still holding a placeholder
// only a machine fills (a setting, an access the definition gives no default) is the engine's to judge.
func (m Manifest) resolvedPathProblems() []string {
dirs := dirsFor(m, Rendering{})
accesses := map[string]string{}
for _, a := range m.Accesses {
if a.ID != "" && a.Path != "" {
accesses[a.ID] = a.Path
}
}
var problems []string
for _, r := range m.Resources {
kind := fmt.Sprint(r["type"])
if kind != "directory" && kind != "file" {
continue
}
id := fmt.Sprint(r["id"])
path, _ := r["path"].(string)
if kind == "directory" && path == "" {
path = dirs[id]
}
path, _ = dirFill(path, dirs, m.Module)
path, _ = accessFill(path, accesses, m.Module)
if path == "" || strings.Contains(path, "${") {
continue
}
if why := enginePath(path, m.Module); why != "" {
problems = append(problems, fmt.Sprintf("%s's %s %q resolves to %s, which the node-engine refuses: %s. "+
"A module's directories and files are judged when its definition is, so the merge gate refuses it "+
"before any machine does (novox/hq issue 496)", m.Module, kind, id, filepath.Clean(path), why))
}
}
return problems
}
// accessRef is how a module names one of its accesses: ${access:<id>}.
var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`)
@@ -534,57 +438,6 @@ func ownerInto(resource map[string]any, placed map[string]Placement) {
}
}
// unknownAccessRefs is every ${access:…} in the definition that names no access the definition
// declares by id — refused where the author is, as unknownDirRefs does for directories.
func (m Manifest) unknownAccessRefs() []string {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
declared[a.ID] = true
}
}
seen := map[string]bool{}
var problems []string
refuse := func(s string, where any) {
for _, match := range accessRef.FindAllStringSubmatch(s, -1) {
id := match[1]
if declared[id] || seen[id] {
continue
}
seen[id] = true
problems = append(problems, fmt.Sprintf(
"%s says ${access:%s} in %v, and declares no access %q — a reference the mesh "+
"cannot place would reach the machine as a literal path",
m.Module, id, where, id))
}
}
for _, r := range m.Resources {
for _, field := range []string{"path", "content"} {
if s, ok := r[field].(string); ok {
refuse(s, r["id"])
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := r[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
if env, ok := r["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
sort.Strings(problems)
return problems
}
func directoriesOf(m Manifest) map[string]string {
dirs := map[string]string{}
for _, r := range m.Resources {
-19
View File
@@ -161,25 +161,6 @@ func TestADefinitionsPathIsTheDefaultTheAssignmentReplaces(t *testing.T) {
}
}
// A reference to an access the definition does not declare is refused where the author is.
func TestAnUnknownAccessReferenceIsRefusedAtParse(t *testing.T) {
_, err := ParseManifest([]byte(`{
"module": "arr", "version": "1",
"accesses": [{"id": "series", "mode": "read-write"}],
"resources": [
{"id": "state", "type": "directory", "place": ".", "mode": "0700"},
{"id": "server", "type": "container", "name": "arr", "image": "arr:1",
"volumes": ["${access:movies}:/movies"]}
]}`))
if err == nil || !strings.Contains(err.Error(), "${access:movies}") {
t.Fatalf("a reference to an undeclared access was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module": "arr", "version": "1", "accesses": [{"mode": "read"}]}`))
if err == nil || !strings.Contains(err.Error(), "neither an id nor a path") {
t.Fatalf("an access with no id and no path was accepted: %v", err)
}
}
// **Placement is read from the placement, never from a layer's values** (novox/hq ADR 0348 clause 11, issue
// 651). A `places` or `accesses` among the config values places nothing; the same in Placement does. A
// mergeable file of today's form still takes every key of the layer as it is said, placement too, so it
+9 -4
View File
@@ -236,9 +236,9 @@ func ReplacePublished(list PredefinedList) (was PredefinedList) {
}
// ReadTreeManifestWritten is the manifest in a module's directory of a repository tree as written, still naming its
// templates, judged against the list of predefined keys at the tree's root as the merge gate judges it
// (cmd/mesh-controller). What a build resolves, given the text it reads back from the archive it packs the templates
// into (Built.Texts, ReadTemplates).
// templates, decoded and judged by nothing (novox/hq ADR 0578 §3: the indexer's judge judges manifests), the tree's
// list of predefined keys read so a tree whose list does not read is still refused. What a build resolves, given the
// text it reads back from the archive it packs the templates into (Built.Texts, ReadTemplates).
func ReadTreeManifestWritten(root, dir string) (Manifest, error) {
list, err := ReadPredefined(root)
if err != nil {
@@ -248,7 +248,12 @@ func ReadTreeManifestWritten(root, dir string) (Manifest, error) {
if err != nil {
return Manifest{}, err
}
return ParseManifestWith(raw, list)
m, err := Decode(raw)
if err != nil {
return Manifest{}, err
}
m.predefined = list
return m, nil
}
// ReadTreeManifest is ReadTreeManifestWritten with each template's text read from the module's directory as the
-40
View File
@@ -1,41 +1 @@
package catalogue
import "encoding/json"
// ParseManifestForInputs reads a manifest for the inputs of its build and its fingerprint: the indexer's and the
// builder's read (novox/hq issue 1751). It is never a judge of config values. A value falling back to a predefined key
// (ADR 0348 §7) is judged against the list the mesh published when this process holds one (PublishedPredefined);
// where it holds none, as in the indexer and the builder, each key a value names is taken as the value declares it (its
// type, trusted when it is), since the list is read where a manifest is judged: the merge gate from the tree's root,
// the controller from what it published. No input of a fingerprint or a build comes from the list. Everything else
// ParseManifest refuses is refused here too.
func ParseManifestForInputs(raw []byte) (Manifest, error) {
list := PublishedPredefined()
if list == nil {
list = fallbacksOf(raw)
}
return parseManifest(raw, list)
}
// fallbacksOf is the list of the predefined keys a manifest's config values fall back to, each as the value declares
// it. A manifest that is not JSON gives an empty list, and the parser says what is wrong with it.
func fallbacksOf(raw []byte) PredefinedList {
var m struct {
Config map[string]struct {
Type string `json:"type"`
Description string `json:"description"`
Predefined string `json:"predefined"`
Trusted bool `json:"trusted"`
} `json:"config"`
}
list := PredefinedList{}
if json.Unmarshal(raw, &m) != nil {
return list
}
for _, d := range m.Config {
if d.Predefined != "" {
list[d.Predefined] = PredefinedKey{Type: d.Type, Description: d.Description, Trusted: d.Trusted}
}
}
return list
}
+1 -21
View File
@@ -6,26 +6,6 @@ import (
"testing"
)
// The indexer and the builder hold no published list of predefined keys; a manifest whose config values fall back to
// one (claude-code, dunst, xterm at mesh-catalog 835bf22b) is still read for its build's and fingerprint's inputs
// (issue 1751).
func TestParseManifestForInputsReadsAValueFallingBackToAPredefinedKey(t *testing.T) { // issue 1751
was := ReplacePublished(nil) // the indexer's and the builder's case: no published list
defer ReplacePublished(was)
for _, module := range []string{"claude-code", "dunst", "xterm"} {
raw, err := os.ReadFile("testdata/catalogue-835bf22b-" + module + ".module.json")
if err != nil {
t.Fatal(err)
}
if _, err := ParseManifest(raw); err == nil {
t.Fatalf("%s: ParseManifest reads it with no list; the test proves nothing", module)
}
if m, err := ParseManifestForInputs(raw); err != nil || m.Module != module {
t.Errorf("%s: %v", module, err)
}
}
}
func TestParseManifestForInputsStillRefusesWhatTheParserRefuses(t *testing.T) { // issue 1751
was := ReplacePublished(nil)
defer ReplacePublished(was)
@@ -34,7 +14,7 @@ func TestParseManifestForInputsStillRefusesWhatTheParserRefuses(t *testing.T) {
if bad == string(raw) {
t.Fatal("the fixture no longer names font-size; the test proves nothing")
}
if _, err := ParseManifestForInputs([]byte(bad)); err == nil {
if _, err := decodeForTest([]byte(bad)); err == nil {
t.Error("a manifest the parser refuses was read")
}
}
+1 -32
View File
@@ -74,37 +74,6 @@ func fallingBackTo(t *testing.T, key string) []byte {
return raw
}
// A manifest naming font-size registers against the list; one naming a key not on it, or of another type, is
// refused — judged against the list given, whatever the mesh published.
func TestAManifestIsJudgedAgainstTheListOfItsTree(t *testing.T) {
withPredefined(t, nil)
list, err := ParsePredefined([]byte(theFirstKeys))
if err != nil {
t.Fatal(err)
}
if _, err := ParseManifestWith(fallingBackTo(t, "font-size"), list); err != nil {
t.Fatalf("a value falling back to font-size is refused: %v", err)
}
if _, err := ParseManifestWith(fallingBackTo(t, "font-scale"), list); err == nil ||
!strings.Contains(err.Error(), `"font-scale", which is not on the list`) {
t.Fatalf("a key not on the list: %v", err)
}
m := inConfigForm()
declaration(m, "theme")["predefined"] = "font-size"
raw, _ := json.Marshal(m)
if _, err := ParseManifestWith(raw, list); err == nil || !strings.Contains(err.Error(), "which is a number") {
t.Fatalf("a key of another type: %v", err)
}
// Where no tree is read, the published list judges: none published refuses.
if _, err := ParseManifest(fallingBackTo(t, "font-size")); err == nil || !strings.Contains(err.Error(), "is not present") {
t.Fatalf("judged against no list: %v", err)
}
withPredefined(t, list)
if _, err := ParseManifest(fallingBackTo(t, "font-size")); err != nil {
t.Fatalf("judged against the published list: %v", err)
}
}
// A template's text filled from the tree keeps the list the manifest was judged against: read again, it is not
// judged against the published one.
func TestTemplatesFilledKeepTheListTheManifestWasJudgedAgainst(t *testing.T) {
@@ -126,7 +95,7 @@ func TestTemplatesFilledKeepTheListTheManifestWasJudgedAgainst(t *testing.T) {
declaration(m, "width")["runtime"] = true
declaration(m, "sender")["runtime"] = true
raw, _ := json.Marshal(m)
parsed, err := ParseManifestWith(raw, list)
parsed, err := decodeWithForTest(raw, list)
if err != nil {
t.Fatal(err)
}
-21
View File
@@ -1,7 +1,6 @@
package catalogue
import (
"encoding/json"
"fmt"
"strings"
"testing"
@@ -119,23 +118,3 @@ func TestAModuleThatPreparesNothingGetsNoStep(t *testing.T) {
}
}
}
// A module whose own code the mesh cannot find has nothing to ask, and saying so where the manifest
// is read beats a declaration that quietly prepares nothing.
func TestAModuleThatPreparesAndRunsNoneOfItsOwnCodeIsRefused(t *testing.T) {
m := Manifest{
Module: "gitea",
Prepares: true,
Resources: []map[string]any{
// Only the upstream server it packages: nothing here runs gitea's own code.
{"id": "server", "type": "container", "name": "mesh-gitea-server", "image": "gitea/gitea@" + digest},
},
}
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
if _, err := ParseManifest(raw); err == nil {
t.Fatal("a module that prepares its state with nothing of its own to run was accepted")
}
}
-228
View File
@@ -4,8 +4,6 @@ import (
"bytes"
"encoding/json"
"fmt"
"slices"
"strings"
)
// ProcessBusField is the key a process resource declares its bus on. Controller-side only: never composed
@@ -46,9 +44,6 @@ const (
RemovesAssignmentsStream = "ASSIGNMENTS"
)
// BusRemovals is the closed set of what a bus account may remove.
var BusRemovals = []string{RemovesControllerDeadLetterConsumer, RemovesEngineActs, RemovesAssignmentsStream}
// BusAccount is one account of a module's own beside its `broker`: a process's (ADR 0436) or a tools bundle's (ADR 0564
// Limits 4), by its id (the process resource's, or the bundle's binary), with what its bus says.
type BusAccount struct {
@@ -99,124 +94,6 @@ func ToolsBusSecret(m Manifest, name string) bool {
return false
}
// toolsSecretProblems refuses a tools bundle's bus account secret the runtime's account could not read: it is declared
// as an own secret `broker-<binary>`, at `${dir:<id>}/broker-<binary>` for a directory resource every parent of which the
// runtime's account may pass (passableDir); the file itself is 0600 and the runtime account's.
func toolsSecretProblems(m Manifest) []string {
var problems []string
for _, a := range m.BusAccounts() {
if !a.Tools {
continue
}
name := "broker-" + a.ID
own, declared := m.OwnSecrets.Plain()[name]
if !declared {
problems = append(problems, fmt.Sprintf("%s's tools bundle %s declares bus and no own secret %s to hold its "+
"account in (novox/hq ADR 0564 Limits 4)", m.Module, a.ID, name))
continue
}
dir, _, _ := strings.Cut(strings.TrimPrefix(own.Path, "${dir:"), "}")
if !strings.HasPrefix(own.Path, "${dir:"+dir+"}/") || !passableDir(m, dir, 0) {
problems = append(problems, fmt.Sprintf("%s's own secret %s is at %s, which the node's tool runtime cannot "+
"reach: a tools bundle's account lies in a directory resource the runtime's account may pass, every parent "+
"included, such as {\"id\":\"tools-secret\",\"type\":\"directory\",\"path\":\"%s/%s-tools\",\"mode\":\"0711\"} "+
"with the secret at ${dir:tools-secret}/%s; never under the module's own or mesh place, which are its "+
"own (novox/hq ADR 0564 Limits 4)", m.Module, name, own.Path, meshStateRoot, m.Module, name))
}
}
return problems
}
// meshStateRoot is the mesh's directory of every module's mesh place on a node of the default data root: root's, mode
// 0755, so the runtime's account passes it, while each module's own place beneath it is 0700.
const meshStateRoot = defaultDataRoot + "/" + placeMesh
// passableDir says the runtime's account may pass the directory resource id and every parent the mesh makes for it: its
// own mode lets others pass (the last digit odd), and it is either stated directly under meshStateRoot or beneath a
// placed directory that is passable in turn. A pathless directory resolves under the module's own root or its mesh
// place, which are the module's own, so it never is.
func passableDir(m Manifest, id string, depth int) bool {
if depth > 4 {
return false
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "directory" || fmt.Sprint(r["id"]) != id {
continue
}
mode, _ := r["mode"].(string)
if mode == "" || !strings.ContainsAny(mode[len(mode)-1:], "1357") {
return false
}
path, _ := r["path"].(string)
switch {
case strings.HasPrefix(path, "${dir:"):
parent, _, _ := strings.Cut(strings.TrimPrefix(path, "${dir:"), "}")
return passableDir(m, parent, depth+1)
case strings.HasPrefix(path, "/"):
return filepathDir(strings.TrimRight(path, "/")) == meshStateRoot
}
return false
}
return false
}
func filepathDir(p string) string {
if i := strings.LastIndex(p, "/"); i > 0 {
return p[:i]
}
return "/"
}
// toolsBusProblems is what is wrong with a tools bundle's bus (ADR 0564 Limits 4): a bus on anything but a bundle
// compiled to a binary the runtime loads, a removal outside the closed set, or a removal beside any other grant or by a
// module that does not hold mesh-broker. Every other field is judged as a process's.
func toolsBusProblems(m Manifest) []string {
var problems []string
var artifacts []Artifact
if m.Build != nil {
artifacts = m.Build.Artifacts
}
claimsBroker := false
for _, c := range m.Claims {
claimsBroker = claimsBroker || c.Name == "mesh-broker"
}
for _, a := range artifacts {
if a.Bus == nil {
continue
}
if a.Kind != ArtifactBundle || BinaryOf(a) == "" {
problems = append(problems, fmt.Sprintf("%s's artifact %s declares bus, and only a tools bundle compiled to a "+
"binary has an account of its own (novox/hq ADR 0564 Limits 4)", m.Module, a.Name))
continue
}
b := *a.Bus
if len(b.Queues) > 0 || b.Asks {
problems = append(problems, fmt.Sprintf("%s's tools bundle %s's bus consumes a queue or reads asks; a tools "+
"bundle runs per call and holds no consumer (novox/hq ADR 0564 Limits 4)", m.Module, a.Name))
}
for _, r := range b.Removes {
if !slices.Contains(BusRemovals, r) {
problems = append(problems, fmt.Sprintf("%s's tools bundle %s's bus removes %q, which is not one of %s "+
"(novox/hq ADR 0564 Limits 4)", m.Module, a.Name, r, strings.Join(BusRemovals, ", ")))
}
}
if len(b.Removes) > 0 && (!claimsBroker || len(b.Emits)+len(b.State)+len(b.Invokes) > 0) {
problems = append(problems, fmt.Sprintf("%s's tools bundle %s's bus removes from the bus; only the bus "+
"module's (the holder of mesh-broker) may, and with nothing else (novox/hq ADR 0564 Limits 4)", m.Module, a.Name))
}
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" {
continue
}
if b, present, err := processBusOf(r); present && err == nil && len(b.Removes) > 0 {
problems = append(problems, fmt.Sprintf("%s's process %v's bus removes from the bus; only a tools bundle's "+
"may (novox/hq ADR 0564 Limits 4)", m.Module, r["id"]))
}
}
return problems
}
// UnmarshalJSON reads a process's bus strictly: an unknown key — serves, a seat to hold or claim, or
// anything else that would answer a tool call or a seat's verb — is refused here, by name (novox/hq ADR
// 0436 §3).
@@ -268,108 +145,3 @@ func (m Manifest) ProcessBuses() map[string]ProcessBus {
}
return out
}
// ProcessBusProblems is what is wrong with a manifest's processes' bus declarations (novox/hq ADR 0436 §1,
// §3): the account rule is modelled on RunsAsProblems (seats_declared.go:522-550), and an invoked tool or
// seat verb's shape on invokeProblems.
func ProcessBusProblems(m Manifest) []string {
problems := append(toolsBusProblems(m), toolsSecretProblems(m)...)
streams := map[string]bool{}
for _, s := range m.Streams {
streams[s.Name] = true
}
state := map[string]bool{}
for _, s := range m.State {
state[s.Name] = true
}
emits := map[string]bool{}
for _, e := range m.EmitsAll() {
emits[e] = true
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" {
continue
}
id := fmt.Sprint(r["id"])
if _, present := r["bus"]; !present {
continue
}
b, _, err := processBusOf(r)
if err != nil {
problems = append(problems, fmt.Sprintf("%s's process %s declares bus: %v", m.Module, id, err))
continue
}
user, _ := r["user"].(string)
problems = append(problems, processAccountProblems(m, id, user)...)
for _, e := range b.Emits {
if !emits[e] {
problems = append(problems, fmt.Sprintf(
"%s's process %s's bus emits %q, which %s does not declare as its own (novox/hq ADR 0436 §1)",
m.Module, id, e, m.Module))
}
}
for _, q := range b.Queues {
if !streams[q] {
problems = append(problems, fmt.Sprintf(
"%s's process %s's bus queues %q, which %s does not declare as its own stream (novox/hq "+
"ADR 0436 §1)", m.Module, id, q, m.Module))
}
}
for _, s := range b.State {
if !state[s] {
problems = append(problems, fmt.Sprintf(
"%s's process %s's bus reads and writes state %q, which %s does not declare as its own "+
"(novox/hq ADR 0436 §1)", m.Module, id, s, m.Module))
}
}
for _, t := range b.Invokes {
if !invokeNameValid(t) {
problems = append(problems, fmt.Sprintf(
"%s's process %s's bus invokes %q, which does not name a tool: a process invokes "+
"<module>.<tool>, or a role's seat:<seat>.<verb> (novox/hq ADR 0436 §1)", m.Module, id, t))
}
}
}
return problems
}
// processAccountProblems is what is wrong with one process's bus account (novox/hq ADR 0436 §1, §2): an
// account the module makes — a `user` resource of that name — never root and never the operator account,
// which a manifest cannot say safely (a node with no account has nothing to resolve it to, novox/hq ADR
// 0436, settled by the lead).
func processAccountProblems(m Manifest, processID, user string) []string {
switch {
case user == "":
return []string{fmt.Sprintf(
"%s's process %s declares bus and names no account; a process account is a user resource the "+
"module makes, never root and never the operator account (novox/hq ADR 0436 §1, §2)",
m.Module, processID)}
case user == "root":
return []string{fmt.Sprintf(
"%s's process %s runs as root; a process account is the module's own, never root (novox/hq ADR "+
"0436 §1, §2)", m.Module, processID)}
case user == "${machine:account}":
return []string{fmt.Sprintf(
"%s's process %s runs as the operator account; a process account is the module's own, never the "+
"operator's (novox/hq ADR 0436 §1, §2)", m.Module, processID)}
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "user" && fmt.Sprint(r["name"]) == user {
return nil
}
}
return []string{fmt.Sprintf(
"%s's process %s runs as %s, which it does not make: a user resource named %s (novox/hq ADR 0436 §1, "+
"§2)", m.Module, processID, user, user)}
}
// invokeNameValid says a module or process's invoked entry names a tool: <module>.<tool>, a role's
// seat:<seat>.<verb>, or * for every tool on the mesh (novox/hq ADR 0152, ADR 0436 §1).
func invokeNameValid(t string) bool {
if t == "*" {
return true
}
t = strings.TrimPrefix(t, "seat:")
first, second, named := strings.Cut(t, ".")
return named && name.MatchString(first) && toolName.MatchString(second)
}

Some files were not shown because too many files have changed in this diff Show More