Compare commits

...
Author SHA1 Message Date
mesh-tools 5c281ad99b Merge pull request 'An unissued bus account holds back only its own module, never the whole node' (#490) from fix/vault-minted-accounts into main 2026-11-01 08:26:02 +00:00
jschoubben 29dfb28f38 Hold only the module whose bus account nobody issued, never the whole node (issue 203)
mesh/delivery merged: source.main moved to novox/mesh-controller@5c281ad99bcc
mesh/check checking 29dfb28f3899 on 5c281ad99bcc
The 1585 group's refusal turned one unissued account into a refusal of the node's whole
composition: novox's mesh-hand-acts, which the vault had minted but the controller's store
does not hold, stopped every take-in on the machine that runs the bus. The module is now held
out of the composition alone, in issue 203's words, and the rest composes; no placeholder is
sealed. A held module of the former form keeps the node's own composition as last written, so
the node-engine never removes it.

Counting the vault's accounts as issued needs the controller to read mesh-vault.bus-accounts,
which mesh-sdk's named readers give to the grants part alone (ADR 0547 clause 9): not built here.

Controller research piece: none moved; this narrows a refusal in the composition the controller
hands over (blocking-fault recovery).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 09:12:54 +01:00
mesh-tools 88b16e9be2 Merge pull request 'The controller takes the indexer's verdict instead of judging manifests; route-proxy declares its credential form (ADR 0578, 0579)' (#489) from feat/judge-once into main 2026-11-01 07:54:23 +00:00
jschoubben b564c315d0 Test the controller as one that judges no manifest, and reset the new-module waits each test leaves (ADR 0578)
mesh/delivery merged: source.main moved to novox/mesh-controller@88b16e9be20d
mesh/delivery-group group feat/judge-once merged: every member is merged
mesh/check checking b564c315d012 on dc90b3f5bd13 | novox/mesh-controller#489 in group feat/judge-once
Issue 1585: registration publishes the predefined list and judges nothing against it; a definition naming an
installation is the indexer's judge's to refuse; a word the controller does not read is set aside and the module
registered (0578 §6 changes 0501 §7); a refused verdict holds the module and names it. The process-wide list of
new modules waiting is emptied at each take-in test's start and end, so no test reads another's.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 04:14:44 +01:00
jschoubben 5bd03b3984 Refuse a send that would seal a placeholder for a bus credential nobody issued (issue 203)
mesh-sdk #141 (issue 774) stands an unissued bus account in, in the Foreseeing mode, so a proposal's answer can be
asked before the vault mints it; the controller's send then took the stand-in for an own secret to make and would
seal a random value where the broker credential belongs. The send and the question refuse it now, in the words the
composition used to: module issue, then push.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 04:14:44 +01:00
jschoubben e02fcad739 Pin the group's mesh-sdk head after the gofmt fix
mesh/delivery-group group feat/judge-once rejected: composed together: the checker's verdict at the members' heads (0920e5637ce1): integration: github.com/novo…
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (5 tests): integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestABuildHeardIsRecordedAndRegistered: a definition na…
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:27:07 +01:00
jschoubben 4a92ec6102 Pin mesh-sdk at the head after review round 1
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/judge-once checking: 4 of 6 member(s) ready
mesh/check checking 4a92ec610227 on dc90b3f5bd13 | novox/mesh-controller#489 in group feat/judge-once
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 03:13:17 +01:00
jschoubben 909cc25169 Pin mesh-sdk at the judge-less release head in the controller and route-proxy
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:46:24 +01:00
jschoubben 272d5ebd61 Declare credential form none for the provision, which hands consumers no credential
The manifest judge refuses an offer that says no credential form (issue 1585).

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:43:24 +01:00
jschoubben a6386ff23a Decode manifests in tests and pin the mesh-sdk that no longer holds the judge
Issue 1585, S18/S19: the judge's tests live in mesh-delivery now, so an assertion that only tested judging goes
and every test that builds a manifest decodes it.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:40:39 +01:00
jschoubben b51105908a Stop judging manifests at registration; the controller registers what the indexer accepted
Issue 1585, S18 (ADR 0578 §3): registerJudged no longer re-judges the manifest or its predefined keys, and a
standing build is read leniently; module add decodes and judges nothing. Resolve never re-parsed a manifest,
so a key-key fallback needs no list there; a table row holds that.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:09:13 +01:00
jschoubben 8b4d536a11 Take the indexer's verdict on a manifest instead of judging it in the controller
Issue 1585, S18 (ADR 0578 §3, §6): a done key's manifest is decoded leniently and the verdict read from the
indexer's state; a refused one is held and named, an unread word set aside and named, never a wait. The
module check command, which judged files with no mesh and which nothing calls any more, goes with its tests.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-01 02:07:44 +01:00
mesh-tools dc90b3f5bd Merge pull request 'route-proxy dials the machine port its plans now carry (issue 774)' (#488) from fix/route-ports into main 2026-10-31 12:56:54 +00:00
jschoubben 0e9c4c8299 Check route-proxy's composed targets over the live shapes in an existing test's place, and pin the mesh-sdk that keeps unassigned bare ports (issue 774, research 072 row 34)
mesh/delivery merged: source.main moved to novox/mesh-controller@dc90b3f5bd13
mesh/delivery-group group fix/route-ports merged: every member is merged
mesh/check failed: integration: mesh-vault/cmd/mesh-vault.TestTheVaultWholeMintsOnTheBusWhoeverHoldsTheLease: never: the account minted under the cont…
The table covers gitea and grafana moved by the placement, influxdb's bare port, mailu's and
photos' fixed mappings and home-assistant on the host's network; it replaces the loopback test it
subsumes, so route-proxy's test count does not grow.

The controller, NATS-native: no piece moves; route-proxy only.
2026-10-31 13:22:00 +01:00
jschoubben 14d10051e5 Dial a route at the machine port its module's placement assigned, from the mesh-sdk whose plans carry it (issue 774, research 072 row 34)
route-proxy from plans dialled every route at the software's own port: gitea at 127.0.0.1:3000
(published at 20000) and grafana on ace at 3000 (published at 20012), and public sites answered
502. The proxy already reaches a module beside it over loopback and one elsewhere at the plan's
address; the plan now carries the assigned port, and a test composes the plan with the SDK and
reads it as the proxy does, so the two cannot drift again.

The controller, NATS-native: no piece moves; route-proxy only.
2026-10-31 13:11:00 +01:00
mesh-tools c58b311ea7 Merge pull request 'route-proxy from the provisioner's plans with its own contract; the controller forwards accounts to the vault (issue 774)' (#487) from feat/072-moves into main 2026-10-31 10:31:07 +00:00
jschoubben 7a169eae76 Keep the controller on main's mesh-sdk pin and mint until the grants part takes over, so its impact stays small and no account is minted that the bus it loads refuses (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@c58b311ea774
mesh/delivery-group group feat/072-moves merged: every member is merged
mesh/check failed: shanks: the check of novox/mesh-controller#487 failed: go mod download in modules/route-proxy: go: git.novox.be/novox/mesh-controll…
The controller is being removed, so its root module goes back to origin/main's mesh-sdk pin (2da0de7) and
testdata/beside's capture; route-proxy keeps its own pins. Its one functional change stays the account issue:

- Before the grants part's take-over (its key user-list.sealed not saying FollowGrants) the bus loads the
  controller's list, whose hashes come only from its store, so the controller mints as before and hands every
  account it mints, covered or not, to an installed vault at once: no account waits on a node reporting its hash.
- From the take-over it forwards the issue to the vault's issue-account (d791052f) and mints nothing.

And the controller no longer declares or asserts the ASSIGNMENTS stream, which the bus module's tools delete at the
lease write and nothing may make again (research 072 row 33); the verified-sender re-issue stops once it is gone.

Research 072 piece: the gap "vault mints after the lease", the minting piece moves to the vault from the take-over.
2026-10-31 10:37:13 +01:00
jschoubben a07621d761 Pin route-proxy at the group's mesh-sdk head acbf80a and its contract at the published tag v0.1.0, so it builds and reads the contributed payload; refresh its manifest digest for the events its contract names (issue 774) 2026-10-31 10:27:05 +01:00
jschoubben d791052f77 Forward module issue and an assignment's issue to the vault once one is installed, so the controller mints no bus account and none waits on a node reporting here (issue 774; research 072, the controller NATS-native: gap "vault mints after the lease", the minting piece moves to the vault) 2026-10-31 10:02:39 +01:00
jschoubben 22935b9202 Serve route-proxy's routes through its own contract: decode each plan entry's opaque contribution with it, say each name served, withdrawn or refused on the bus, and take who is inside from the seat mesh-placement, since the membership went with the assignments stream (issue 774, research 072 rows 33 and 34) 2026-10-31 09:44:07 +01:00
88 changed files with 1259 additions and 1728 deletions
@@ -85,7 +85,7 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err := catalogue.ParseManifest(raw) m, err := catalogue.Decode(raw)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -372,7 +372,7 @@ func TestTheAccountsAModuleSpeaksWithOnANode(t *testing.T) {
t.Error("sealed no password") t.Error("sealed no password")
} }
return c.sealErr return c.sealErr
}) }, func() error { acts.did("after"); return nil })
if (err != nil) != (c.sealErr != nil) { if (err != nil) != (c.sealErr != nil) {
t.Fatalf("error %v, want %v", err, c.sealErr) t.Fatalf("error %v, want %v", err, c.sealErr)
} }
@@ -385,6 +385,52 @@ func TestTheAccountsAModuleSpeaksWithOnANode(t *testing.T) {
}) })
} }
// Issue 774: where a vault is installed the controller mints nothing; `module issue` and an assignment's issue are
// forwarded to the vault's issue-account, and what the account needs beside its value still runs. A refusal fails
// `module issue` and only says itself on an assignment's issue: the vault mints on the want when it hears it.
for _, c := range []struct {
name string
only bool
answer string
refuse error
want string
fails bool
}{
{"module issue: forwarded, minted by the vault", false, `{"minted":true,"account":{"secret":"own/ace/mesh-build-go/broker","generation":1}}`, nil, "after", false},
{"an assignment's issue: forwarded, held already", true, `{"minted":false,"why":"held already"}`, nil, "after", false},
{"an assignment's issue: refused, said", true, "", errors.New("nobody wants it yet"), "after", false},
{"module issue: refused, failed", false, "", errors.New("nobody wants it"), "", true},
} {
t.Run("forwarded to the vault: "+c.name, func(t *testing.T) {
was := vaultIssuer
t.Cleanup(func() { vaultIssuer = was })
var asked []string
vaultIssuer = func(context.Context, accountIssuer) (vaultAsker, bool, error) {
return func(verb string, args map[string]any) (json.RawMessage, error) {
asked = append(asked, verb+" "+fmt.Sprint(args["user"]))
return json.RawMessage(c.answer), c.refuse
}, true, nil
}
acts := &anAccountStore{}
ctx, got := withIssued(t.Context())
if c.only {
ctx = onlyIfAbsent(ctx)
}
err := issueAccount(ctx, acts, inventory.BusUser{Username: "ace.mesh-build-go", Kind: inventory.BusModule},
func(string) error { acts.did("seal"); return nil }, func() error { acts.did("after"); return nil })
if (err != nil) != c.fails || strings.Join(acts.acts, " ") != c.want {
t.Fatalf("error %v, acts %q; want failed %v, acts %q (nothing minted, locked or sealed here)", err,
strings.Join(acts.acts, " "), c.fails, c.want)
}
if strings.Join(asked, ",") != "issue-account ace.mesh-build-go" {
t.Errorf("the vault was asked %v", asked)
}
if minted := len(got.list()) == 1; minted != strings.Contains(c.answer, `"minted":true`) {
t.Errorf("said issued %v", got.list())
}
})
}
// The accounts of a module that no longer resolves on its node are stale, its provisioner client's among them; a // The accounts of a module that no longer resolves on its node are stale, its provisioner client's among them; a
// module still resolved (assigned or required), an account of another kind, and a node whose resolution is unknown // module still resolved (assigned or required), an account of another kind, and a node whose resolution is unknown
// are left alone (issue 1945: ace's mesh-build-go rows from 17:19, marked sealed by migration 0108). // are left alone (issue 1945: ace's mesh-build-go rows from 17:19, marked sealed by migration 0108).
+5 -14
View File
@@ -5,10 +5,11 @@ import (
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
"github.com/novox/mesh-controller/internal/conditions"
"io" "io"
"os" "os"
"github.com/novox/mesh-controller/internal/conditions"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
@@ -127,17 +128,13 @@ type takenBuild struct {
carried func() error carried func() error
} }
// registerJudged judges a recorded build and registers its module, or says why not. // registerJudged registers its module, or says why not.
func registerJudged(ctx context.Context, inv *inventory.Inventory, j takenBuild) (catalogue.Manifest, error) { func registerJudged(ctx context.Context, inv *inventory.Inventory, j takenBuild) (catalogue.Manifest, error) {
manifest, kept := j.manifest, j.kept manifest, kept := j.manifest, j.kept
notRegistered := func(err error) (catalogue.Manifest, error) { notRegistered := func(err error) (catalogue.Manifest, error) {
return manifest, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", j.by, manifest.Module, return manifest, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", j.by, manifest.Module,
short(j.commit), err) short(j.commit), err)
} }
if err := namesNoInstallation(manifest); err != nil {
return manifest, fmt.Errorf("%s built %s (%s), and the mesh does not register it: %w",
j.by, j.what, short(j.commit), err)
}
// **Only from the repository the catalogue builds the module from** (novox/hq ADR 0266): else the trunk // **Only from the repository the catalogue builds the module from** (novox/hq ADR 0266): else the trunk
// below is the trunk of whatever repository was built, which may be one an agent made — and a module named // below is the trunk of whatever repository was built, which may be one an agent made — and a module named
// `sudo` from it would be what the next push sends. Another repository is the operator's, at the terminal. // `sudo` from it would be what the next push sends. Another repository is the operator's, at the terminal.
@@ -150,18 +147,12 @@ func registerJudged(ctx context.Context, inv *inventory.Inventory, j takenBuild)
return notRegistered(err) return notRegistered(err)
} }
} }
// Then the module is judged against the list the mesh publishes, as every manifest read again is: one naming a // The manifest was judged once, by the indexer, and its verdict taken before this (buildOfKey; ADR 0578 §3).
// key that list does not hold, or of another type, is refused here and never enters the catalogue.
if j.carried != nil { if j.carried != nil {
if err := j.carried(); err != nil { if err := j.carried(); err != nil {
return manifest, err return manifest, err
} }
} }
if manifest.NamesPredefined() {
if _, err := catalogue.ParseManifest(kept.Manifest); err != nil {
return notRegistered(fmt.Errorf("judged against the list of predefined keys the mesh publishes, %w", err))
}
}
// **A build whose source is unchanged is never a move** (novox/hq issue 280): a rebuild made from // **A build whose source is unchanged is never a move** (novox/hq issue 280): a rebuild made from
// what the build the mesh stands on was made from registers that build's artifacts at the new // what the build the mesh stands on was made from registers that build's artifacts at the new
// commit, so no machine is sent a new digest for a source nobody changed — an image is not // commit, so no machine is sent a new digest for a source nobody changed — an image is not
@@ -172,7 +163,7 @@ func registerJudged(ctx context.Context, inv *inventory.Inventory, j takenBuild)
return manifest, err return manifest, err
} }
if stands != "" && stands != kept.ID && len(raw) > 0 { if stands != "" && stands != kept.ID && len(raw) > 0 {
if was, err := catalogue.ParseManifest(raw); err == nil && was.Module == manifest.Module { if was, err := catalogue.Decode(raw); err == nil && was.Module == manifest.Module {
fmt.Printf("%s at %s was made from the source %s was: registered with its artifacts, no move\n", fmt.Printf("%s at %s was made from the source %s was: registered with its artifacts, no move\n",
manifest.Module, short(j.commit), stands) manifest.Module, short(j.commit), stands)
manifest = was manifest = was
@@ -49,8 +49,10 @@ func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}}) "env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ _, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named}) ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") { // Not registered from a source the catalogue does not build it from. That it names an installation is no longer the
t.Fatalf("a definition naming an installation was taken in: %v", err) // controller's to judge (ADR 0578 §3): the indexer's judge refuses it (mesh-delivery-indexer judge's installation rows).
if err == nil || !strings.Contains(err.Error(), "recorded and not registered") {
t.Fatalf("a definition from a source the catalogue does not build it from was taken in: %v", err)
} }
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" { if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway") t.Fatal("the refused module was registered anyway")
@@ -17,6 +17,8 @@ import (
type builderOnTheBus struct { type builderOnTheBus struct {
js *broker.JetStream js *broker.JetStream
builds, modules, manifests jetstream.KeyValue builds, modules, manifests jetstream.KeyValue
// verdicts is the indexer's state of verdicts on manifests (ADR 0578 §1).
verdicts jetstream.KeyValue
} }
func aBuilderOnTheBus(t *testing.T) builderOnTheBus { func aBuilderOnTheBus(t *testing.T) builderOnTheBus {
@@ -40,6 +42,7 @@ func aBuilderOnTheBus(t *testing.T) builderOnTheBus {
{&b.builds, sdkdelivery.BuilderBuildsBucket, sdkdelivery.BuilderBuildsHistory}, {&b.builds, sdkdelivery.BuilderBuildsBucket, sdkdelivery.BuilderBuildsHistory},
{&b.modules, sdkdelivery.ModulesBucket, sdkdelivery.ModulesHistory}, {&b.modules, sdkdelivery.ModulesBucket, sdkdelivery.ModulesHistory},
{&b.manifests, sdkdelivery.ManifestsBucket, 1}, {&b.manifests, sdkdelivery.ManifestsBucket, 1},
{&b.verdicts, sdkdelivery.VerdictsBucket, sdkdelivery.VerdictsHistory},
} { } {
kv, err := api.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: c.bucket, History: c.history}) kv, err := api.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: c.bucket, History: c.history})
if err != nil { if err != nil {
-337
View File
@@ -1,337 +0,0 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/link"
)
// moduleCheck judges manifests where they are written, with no mesh (novox/hq ADR 0037, issue 148).
//
// **The same functions registration runs, and nothing the command line adds** (ADR 0035): the strict
// parse with every per-manifest problem, then the rules no single manifest can be judged against,
// over exactly the manifests given. Somebody describing their own application in their own
// repository runs this before pushing and finds out there, rather than when a running mesh refuses
// the registration or, later, when a machine applies something that resolved and should not have.
//
// **What it cannot know without a store, it says.** The mesh's own seat set is the store's (ADR
// 0122); this binary carries a compiled copy that the store overrides when loaded, so a claim on a
// mesh seat is judged fully only at registration. A seat another module declares is unknown unless
// that module's manifest is passed too. Both are printed as a note, not as a problem — a check that
// refused what it could not see would teach people to ignore it.
//
// **And every identity against every bound it meets** (novox/hq ADR 0225, issue 263): each module's
// identity, on a machine whose name is `longestMachine` characters, against the bound of every
// provision it wants that a manifest given here offers. An overflow is refused in the pull request
// that introduces it — a new requirement, a lowered bound, a longer slug — instead of on the
// provider's machine when a real machine's name first meets the module's.
// SomeManifestsVar, set by the merge gate, says the manifests given are only some of their repository's.
const SomeManifestsVar = "MESH_MODULE_CHECK_SOME"
func moduleCheck(paths []string, out io.Writer) error {
return moduleCheckFor(paths, catalogue.DefaultLongestMachine, out)
}
func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
if len(paths) == 0 {
return errors.New("module check <manifest.json>... — one file per module; pass every " +
"manifest of a repository together so the rules between them are checked too")
}
shelf := catalogue.Shelf{}
faulted := map[string]bool{}
failed := 0
// The list of predefined keys at the root of the repository above each manifest (novox/hq ADR 0348 §7), read
// once per repository: a manifest naming a key is judged against the list its own tree holds.
lists := map[string]catalogue.PredefinedList{}
listOf := func(path string) (catalogue.PredefinedList, error) {
root := catalogue.RepositoryRoot(filepath.Dir(path))
if root == "" {
return nil, nil
}
if list, read := lists[root]; read {
return list, nil
}
list, err := catalogue.ReadPredefined(root)
if err != nil {
return nil, err
}
lists[root] = list
return list, nil
}
for _, path := range paths {
raw, err := os.ReadFile(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
list, err := listOf(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
m, err := catalogue.ParseManifestWith(raw, list)
if err == nil {
// Each template's text read from the tree beside the manifest, and judged as inline content is
// (novox/hq ADR 0349 §2): a missing template is refused by name.
m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(path))
}
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
if first, twice := shelf[m.Module]; twice {
_ = first
fmt.Fprintf(out, "%s: %s was already given; two manifests name one module\n", path, m.Module)
failed++
continue
}
// A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here, in the
// catalogue-wide test, and at registration, which refuses in the same words.
if wrong := catalogue.TrustProblems(m); len(wrong) > 0 {
for _, p := range wrong {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(wrong)
faulted[m.Module] = true
}
// A module that declares a setting names the keys of its mergeable files (novox/hq ADR 0326 §3,
// issue 574): judged where a definition is written, as at registration, never where a stored
// build is read again.
if wrong := catalogue.WrittenSettingsProblems(m); len(wrong) > 0 {
for _, p := range wrong {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(wrong)
faulted[m.Module] = true
}
// A module lives whole in one repository (novox/hq ADR 0420 §1, issue 1019): when the check is told the
// repository it runs for (MESH_CHECK_REPOSITORY, set by the build's check), an artifact building from
// another is refused, as the merge gate refuses it. A run by hand names no repository and judges none.
if repo := os.Getenv("MESH_CHECK_REPOSITORY"); repo != "" && m.Build != nil {
for _, a := range m.Build.Artifacts {
if a.Context != nil && a.Context.Repository != "" && !sameRepository(a.Context.Repository,
link.SourceMoved{Owner: ownerOf(repo), Repo: repoOf(repo)}) {
fmt.Fprintf(out, "%s: %s builds artifact %s from %s: a module lives whole in one repository "+
"(novox/hq ADR 0420 §1)\n", path, m.Module, a.Name, a.Context.Repository)
failed++
faulted[m.Module] = true
}
}
}
if named := catalogue.InstallationProblems(m); len(named) > 0 {
for _, p := range named {
fmt.Fprintf(out, "%s: %s\n", path, p)
}
failed += len(named)
faulted[m.Module] = true
}
shelf[m.Module] = m
}
// Between the manifests: a seat declared twice, a use of a seat nothing declares, a claim on
// a seat that does not exist. Run only over what parsed, because a problem inside one manifest
// has already been said and would be said again here in a worse form.
//
// **Over some of a repository's manifests, a seat none of them declares is a note** (novox/hq issue 364), as
// this command's own word says above: the merge gate passes only the manifests a change touches, and says so
// with SomeManifestsVar, so a module that uses or claims a seat another module declares (the operator
// channel's, a channel bench) was refused there for a manifest it was not given. Given every manifest — the
// catalogue's own check, and registration — it stays a refusal.
some := os.Getenv(SomeManifestsVar) != ""
problems := catalogue.CatalogueProblems(shelf)
sort.Strings(problems)
for _, p := range problems {
if some && catalogue.IsUndeclaredSeat(p) {
fmt.Fprintf(out, "note: %s among the manifests given; registration judges it against the whole catalogue, "+
"and passing the declaring module's manifest too judges it here\n", p)
continue
}
fmt.Fprintln(out, p)
failed++
}
// Between the manifests too: an identity against the bounds of the provisions it wants, which
// only the provider's manifest states.
identities := catalogue.IdentityProblems(shelf, longestMachine)
sort.Strings(identities)
for _, p := range identities {
fmt.Fprintln(out, p)
}
failed += len(identities)
// And the data each module keeps (novox/hq ADR 0233): a provider that grants says what it keeps for
// its consumers, a directory a container writes is declared, and no backup line is written by hand.
data := catalogue.DataProblems(shelf)
sort.Strings(data)
for _, p := range data {
fmt.Fprintln(out, p)
}
failed += len(data)
var names []string
for name := range shelf {
names = append(names, name)
}
sort.Strings(names)
// **Every long-running resource says how it is ready** (novox/hq ADR 0240 rule 8): warned until the
// date, refused from it. The count is the catalogue's: its merge check keeps the number and lets a
// change lower it, never raise it.
undeclared := 0
required := !checkNow().Before(catalogue.HealthRequiredFrom)
for _, name := range names {
missing := catalogue.Undeclared(shelf[name])
undeclared += len(missing)
if len(missing) == 0 {
continue
}
if required {
for _, id := range missing {
fmt.Fprintf(out, "%s: %s stays up and does not say how it is ready: a long-running resource declares "+
"health since %s (novox/hq ADR 0240 rule 8)\n", name, id, catalogue.HealthRequiredFrom.Format("2006-01-02"))
}
failed += len(missing)
faulted[name] = true
}
}
// **A file that asks for a setting and does not say whether it is trusted counts as trusted** (novox/hq issue
// 339): listed and counted, never refused, so an author can opt out a file nothing trusts.
unsaid := 0
for _, name := range names {
unsaid += len(catalogue.UnsaidTrust(shelf[name]))
}
for _, name := range names {
m := shelf[name]
if faulted[name] {
continue
}
fmt.Fprintf(out, "%s: ok", name)
if n := len(m.Tools); n > 0 {
fmt.Fprintf(out, ", %d tool(s)", n)
}
if len(m.Invokes) > 0 {
fmt.Fprintf(out, ", invokes %s", joinInvokes(m.Invokes))
}
// The state it keeps and reads (novox/hq ADR 0201), so a reviewer sees what lands on the bus.
if len(m.State) > 0 {
kept := make([]string, 0, len(m.State))
for _, s := range m.State {
kept = append(kept, s.Name)
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
// And the logs it keeps, with their caps (novox/hq ADR 0297).
if len(m.Logs) > 0 {
kept := make([]string, 0, len(m.Logs))
for _, l := range m.Logs {
kept = append(kept, fmt.Sprintf("%s (%d MiB)", l.Name, l.Cap()))
}
fmt.Fprintf(out, ", keeps log %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
// The data it keeps, by class, so a reviewer sees what the mesh will protect and how.
if items := m.DataItems(); len(items) > 0 {
kept := make([]string, 0, len(items))
for _, it := range items {
kept = append(kept, it.ID+" ("+it.Class+")")
}
fmt.Fprintf(out, ", keeps %s", strings.Join(kept, ", "))
}
// How what it runs is ready (ADR 0240): each declared check, and what is judged by liveness alone.
var checks []string
for _, r := range m.Resources {
if h, has, _ := catalogue.ReadHealth(r); has {
checks = append(checks, fmt.Sprintf("%v by %s", r["id"], catalogue.HealthWords(h)))
}
}
// And its own runtime's, where it runs as its own account (novox/hq ADR 0366).
if own, has := catalogue.OwnRuntimeHealth(m); has {
if h, _, _ := catalogue.ReadHealth(own); h.Kind != "" {
checks = append(checks, fmt.Sprintf("%v by %s", own["id"], catalogue.HealthWords(h)))
}
}
if len(checks) > 0 {
fmt.Fprintf(out, ", ready: %s", strings.Join(checks, "; "))
}
if missing := catalogue.UnsaidTrust(m); len(missing) > 0 {
fmt.Fprintf(out, "; WARNING: %s ask(s) for a setting and do(es) not say whether it is trusted, so it counts as "+
"trusted: set at the terminal alone; say %q false where nothing trusts it (novox/hq issue 339)",
strings.Join(missing, ", "), catalogue.TrustedField)
}
if missing := catalogue.Undeclared(m); len(missing) > 0 {
fmt.Fprintf(out, "; WARNING: %s stay(s) up and say(s) not how it is ready — judged by liveness alone, "+
"refused from %s (ADR 0240 rule 8)", strings.Join(missing, ", "), catalogue.HealthRequiredFrom.Format("2006-01-02"))
}
fmt.Fprintln(out)
}
// The count the catalogue keeps (ADR 0240 rule 8), in a line its merge check reads.
fmt.Fprintf(out, "%s %d\n", UndeclaredHealthLine, undeclared)
fmt.Fprintf(out, "%s %d\n", UnsaidTrustLine, unsaid)
if failed > 0 {
return fmt.Errorf("%d problem(s) in %d manifest(s)", failed, len(paths))
}
fmt.Fprintf(out, "%d manifest(s) checked. Judged against the seats this binary carries; a claim on "+
"one of the mesh's own seats is judged fully at registration, and a seat declared by a "+
"module not given here reads as unknown. Identities judged on a %d-character machine name, "+
"against the bounds of the providers given here\n", len(paths), longestMachine)
return nil
}
// UndeclaredHealthLine starts the line `module check` says the count of long-running resources without
// `health` in, over the manifests given: the catalogue's merge check compares it with the number it keeps.
const UndeclaredHealthLine = "long-running resources without health:"
// UnsaidTrustLine starts the line `module check` says the count of files that ask for a setting and do not say
// whether it is trusted, and so count as trusted (novox/hq issue 339).
const UnsaidTrustLine = "files asking for a setting without saying whether it is trusted:"
// checkNow is the clock `module check` judges the date by; a test sets it.
var checkNow = time.Now
func joinInvokes(invokes []string) string {
if len(invokes) == 1 && invokes[0] == "*" {
return "every tool"
}
s := ""
for i, t := range invokes {
if i > 0 {
s += ", "
}
s += t
}
return s
}
// manifestsUnder lists every module.json below a directory, for `module check <dir>`.
func manifestsUnder(dir string) ([]string, error) {
var found []string
err := filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "node_modules" || d.Name() == ".git" || d.Name() == "dist") {
return filepath.SkipDir
}
if !d.IsDir() && d.Name() == "module.json" {
found = append(found, path)
}
return nil
})
sort.Strings(found)
return found, err
}
-167
View File
@@ -1,167 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-controller/internal/beside"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
// with a known fault is named, and one without passes, with no store opened.
func TestModuleCheckNamesAFaultAndNeedsNoMesh(t *testing.T) {
dir := t.TempDir()
good := filepath.Join(dir, "good.json")
bad := filepath.Join(dir, "bad.json")
os.WriteFile(good, []byte(`{"module":"shop","version":"1","tools":["price"],"invokes":["mesh-catalog.catalog_modules"]}`), 0o600)
os.WriteFile(bad, []byte(`{"module":"till","version":"1","invokes":["shop"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil {
t.Fatalf("a sound manifest was refused: %v\n%s", err, out.String())
}
if !strings.Contains(out.String(), "shop: ok, 1 tool(s), invokes mesh-catalog.catalog_modules") {
t.Fatalf("the report does not say what it checked:\n%s", out.String())
}
out.Reset()
err := moduleCheck([]string{good, bad}, &out)
if err == nil {
t.Fatal("a manifest invoking a module and no tool passed")
}
if !strings.Contains(out.String(), `till invokes "shop", which does not name a tool`) {
t.Fatalf("the fault is not named in the manifest's words:\n%s", out.String())
}
}
// The rules between manifests run over what was given together: a seat two modules declare is
// refused, which no single-manifest check can see.
func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
dir := t.TempDir()
a := filepath.Join(dir, "a.json")
b := filepath.Join(dir, "b.json")
os.WriteFile(a, []byte(`{"module":"a","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
os.WriteFile(b, []byte(`{"module":"b","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{a, b}, &out); err == nil {
t.Fatalf("two declarations of one seat passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "a seat name means one protocol") {
t.Fatalf("the cross-manifest rule was not the one named:\n%s", out.String())
}
}
// The real catalogue passes the command, the way it passes the test that used to be the only check: the catalogue this
// controller is released with, captured in testdata/beside at the commit CAPTURED names. Not the clone a merge check puts
// beside, which is the catalogue's trunk even in a delivery group holding a catalogue pull request (ADR 0484 clause 4):
// a controller whose check reads a word that group's catalogue brings would fail on the trunk's catalogue before the
// group merges both (issue 774: the release's sdk refuses an offer that states no credential form, and the catalogue's
// trunk states none until the release's catalogue merges). Moving CAPTURED to the catalogue a change is released with is
// part of that change.
func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := filepath.Join(beside.CapturedDir(t, "mesh-catalog"), "modules")
paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
}
var out bytes.Buffer
if err := moduleCheck(paths, &out); err != nil {
t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String())
}
}
func TestRegistrationRefusesADefinitionNamingAnInstallation(t *testing.T) {
// novox/hq ADR 0155: the check moves to registration once the catalogue passes it. Both
// ways in — `module add` and a build's result — go through this, and a name declared on
// purpose passes with its reason.
named := catalogue.Manifest{Module: "idp", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}},
}}
err := namesNoInstallation(named)
if err == nil || !strings.Contains(err.Error(), "login.mesh-one.be") ||
!strings.Contains(err.Error(), catalogue.NamesOnPurpose) {
t.Fatalf("a definition naming an installation is refused with the name and the way out; got %v", err)
}
meant := catalogue.Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc",
catalogue.NamesOnPurpose: map[string]any{
"registry.mesh-one.be": "built outside the mesh until its repository is a build source here"}},
}}
if err := namesNoInstallation(meant); err != nil {
t.Fatalf("a name declared on purpose passes; got %v", err)
}
}
// **The controller's own manifest names every verb of its seat** (novox/hq ADR 0132): its tools lagged
// the seat's verbs for weeks, and `module check` — the gate's first step for a change touching it —
// refused it. Held here, so a verb added to the table without the manifest fails this repository's
// own suite rather than its next pull request's gate.
func TestTheControllersManifestServesEveryVerbOfItsSeat(t *testing.T) {
var out strings.Builder
if err := moduleCheck([]string{"../../module.json"}, &out); err != nil {
t.Fatalf("the controller's own module.json fails module check: %v\n%s", err, out.String())
}
}
// A module that uses a seat another module declares (novox/hq issue 364): refused over the whole catalogue when the
// declarer is missing, a note when the gate says it gives only the manifests a change touches.
func TestASeatAnotherModuleDeclaresIsANoteOverSomeManifests(t *testing.T) {
dir := t.TempDir()
user := filepath.Join(dir, "user.json")
os.WriteFile(user, []byte(`{"module":"asker","version":"1","uses":["operator-channel"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{user}, &out); err == nil {
t.Fatalf("a use of a seat nothing given declares passed the whole-catalogue check:\n%s", out.String())
}
t.Setenv(SomeManifestsVar, "1")
out.Reset()
if err := moduleCheck([]string{user}, &out); err != nil {
t.Fatalf("over some manifests the use was refused:\n%s", out.String())
}
if !strings.Contains(out.String(), "note: asker uses the seat \"operator-channel\"") {
t.Fatalf("the note was not said:\n%s", out.String())
}
}
// **A module lives whole in one repository** (novox/hq ADR 0420 §1, issue 1019): `module check`, told the
// repository it runs for, refuses an artifact whose build context names another; run by hand, it does not know.
func TestModuleCheckRefusesAnArtifactContextNamingAnotherRepository(t *testing.T) {
dir := t.TempDir()
write := func(name, context string) string {
path := filepath.Join(dir, name+".json")
raw := `{"module":"` + name + `","version":"1","build":{"artifacts":[{"name":"server","kind":"image","from":"Dockerfile"` + context + `}]}}`
if err := os.WriteFile(path, []byte(raw), 0o600); err != nil {
t.Fatal(err)
}
return path
}
other := write("other", `,"context":{"seat":"git","repository":"novox/mesh-controller","ref":"main"}`)
own := write("own", `,"context":{"seat":"git","repository":"novox/mesh-catalog","ref":"main"}`)
none := write("none", ``)
t.Setenv("MESH_CHECK_REPOSITORY", "novox/mesh-catalog")
var out bytes.Buffer
if err := moduleCheck([]string{other}, &out); err == nil {
t.Fatalf("a context naming another repository passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "other builds artifact server from novox/mesh-controller: a module lives whole in one repository (novox/hq ADR 0420 §1)") {
t.Errorf("the refusal is not in the words of the gate:\n%s", out.String())
}
for _, path := range []string{own, none} {
out.Reset()
if err := moduleCheck([]string{path}, &out); err != nil {
t.Errorf("%s was refused: %v\n%s", path, err, out.String())
}
}
t.Setenv("MESH_CHECK_REPOSITORY", "")
out.Reset()
if err := moduleCheck([]string{other}, &out); err != nil {
t.Errorf("a local run, with no repository named, refused it: %v\n%s", err, out.String())
}
}
@@ -11,7 +11,7 @@ import (
// configForm is a module in the config form, shaped as a real one (inventory's config_move_test). // configForm is a module in the config form, shaped as a real one (inventory's config_move_test).
func configForm(t *testing.T, name string) catalogue.Manifest { func configForm(t *testing.T, name string) catalogue.Manifest {
t.Helper() t.Helper()
m, err := catalogue.ParseManifest([]byte(`{"module": "` + name + `", "version": "1", m, err := catalogue.Decode([]byte(`{"module": "` + name + `", "version": "1",
"config": {"level": {"type": "string", "description": "how much it says", "default": "info"}}, "config": {"level": {"type": "string", "description": "how much it says", "default": "info"}},
"resources": [{"id": "conf", "type": "file", "path": "/etc/` + name + `.json", "mode": "0640", "resources": [{"id": "conf", "type": "file", "path": "/etc/` + name + `.json", "mode": "0640",
"content": "{\"level\": \"info\"}", "merge": "json", "takes-config": ["level"]}]}`)) "content": "{\"level\": \"info\"}", "merge": "json", "takes-config": ["level"]}]}`))
+2 -2
View File
@@ -86,8 +86,8 @@ func perform(ctx context.Context, open *stores, in *meshInput, node string, d co
} }
} }
// The node's own secrets: a module's superuser password, an administrator, an account. The composition refused what the // The node's own secrets: a module's superuser password, an administrator, an account. A bus credential nobody issued is
// send refuses (a bus credential nobody issued, a given value under an old key). // held out of the composition before this (holdBackUnissuedBusAccounts), a given value under an old key by the composition.
for _, one := range d.StoodIn { for _, one := range d.StoodIn {
module, name, ok := strings.Cut(one, "/") module, name, ok := strings.Cut(one, "/")
if !ok || strings.Contains(one, " ") { if !ok || strings.Contains(one, " ") {
+82
View File
@@ -184,8 +184,10 @@ func composeQuestion(ctx context.Context, open *stores, node string, mode compos
if err := refusedBus(in, node); err != nil { if err := refusedBus(in, node); err != nil {
return sendable{}, nil, err return sendable{}, nil, err
} }
held := holdBackUnissuedBusAccounts(in, node, &d)
sayFound(ctx, d) sayFound(ctx, d)
out := sendableOf(d) out := sendableOf(d)
out.heldForCredential = held
out.fileReferences = readsFileReferences(in.Input, node) out.fileReferences = readsFileReferences(in.Input, node)
if !withOrder { if !withOrder {
out.Sequence, out.Epoch, out.Generation = 0, 0, 0 out.Sequence, out.Epoch, out.Generation = 0, 0, 0
@@ -212,6 +214,82 @@ func refusedBus(in *meshInput, node string) error {
return nil return nil
} }
// holdBackUnissuedBusAccounts holds out of d every module that stands in for a bus credential nobody issued: the module's
// `broker` own secret (its account `<node>.<module>`) or the provisioner client's (`provisioner.<node>`). A bus credential is
// minted, never invented: a send that made one would seal a placeholder its process cannot read (novox/hq issue 203).
// mesh-sdk's composition stands one in, in the Foreseeing mode, since a node's answer to a proposal is asked before the vault
// mints it (mesh-sdk #141, issue 774).
//
// **That one module is held, never the node.** Its resources, its stand-ins and its build leave this composition, it is named
// among the left out with issue 203's words (said where the composition's other lines go), and the rest of the node composes:
// a refusal of the whole node stopped every take-in on the machine that runs the bus for one account (2026-11-01, novox's
// mesh-hand-acts, which the vault had in fact minted; the controller cannot read the vault's accounts yet, so such a module
// stays held until its account is in the controller's store). The modules held are answered, sorted.
func holdBackUnissuedBusAccounts(in *meshInput, node string, d *compose.Declared) []string {
why := map[string]string{}
for _, one := range d.StoodIn {
module, name, ok := strings.Cut(one, "/")
if !ok || strings.Contains(one, " ") {
continue
}
var user string
switch name {
case "broker":
user = node + "." + module
case compose.ProvisionerSecret:
user = "provisioner." + node
default:
continue
}
if _, said := why[module]; said || slices.Contains(in.Input.BusMinted, user) {
continue
}
why[module] = fmt.Sprintf("it has no bus credential: nothing was issued for %s, and a push would seal a placeholder "+
"its process cannot read (novox/hq issue 203). `module issue %s --node %s`, then push again", user, module, node)
}
if len(why) == 0 {
return nil
}
held := make([]string, 0, len(why))
for module := range why {
held = append(held, module)
}
sort.Strings(held)
ofHeld := func(s string) bool {
module, _, _ := strings.Cut(s, "/")
_, out := why[module]
return out
}
kept := d.Resources[:0:0]
for _, r := range d.Resources {
module := d.ModuleOf[fmt.Sprint(r["id"])]
if module == "" {
module, _ = r["module"].(string)
}
if _, out := why[module]; !out {
kept = append(kept, r)
}
}
d.Resources = kept
d.StoodIn = slices.DeleteFunc(d.StoodIn, ofHeld)
d.Foreseen = slices.DeleteFunc(d.Foreseen, ofHeld)
d.Modules = slices.DeleteFunc(d.Modules, func(m string) bool { _, out := why[m]; return out })
if d.LeftOutWhy == nil {
d.LeftOutWhy = map[string]string{}
}
for _, module := range held {
delete(d.BuildIDs, module)
d.LeftOutWhy[module] = why[module]
if !slices.Contains(d.LeftOut, module) {
d.LeftOut = append(d.LeftOut, module)
}
d.Said = append(d.Said, fmt.Sprintf("%s on %s is held out of %s's composition, and the rest of %s composes: %s",
module, node, node, node, why[module]))
}
sort.Strings(d.LeftOut)
return held
}
// markUnresolvable is an error of a node whose own set does not compose, as planFor marked it. // markUnresolvable is an error of a node whose own set does not compose, as planFor marked it.
func markUnresolvable(err error) error { func markUnresolvable(err error) error {
if compose.IsUnresolvable(err) { if compose.IsUnresolvable(err) {
@@ -256,6 +334,9 @@ func composeSendValidated(ctx context.Context, open *stores, node string) (senda
if err := refusedBus(in, node); err != nil { if err := refusedBus(in, node); err != nil {
return sendable{}, nil, err return sendable{}, nil, err
} }
// Before perform makes what the composition stands in for: a bus credential is never one of them. The module
// whose account nobody issued is held out of this composition, and the rest of the node composes.
held := holdBackUnissuedBusAccounts(in, node, &d)
if len(byController) > 0 { if len(byController) > 0 {
if d, problems, err = sealedByTheController(in, node, d, byController); err != nil { if d, problems, err = sealedByTheController(in, node, d, byController); err != nil {
return sendable{}, nil, err return sendable{}, nil, err
@@ -272,6 +353,7 @@ func composeSendValidated(ctx context.Context, open *stores, node string) (senda
if len(left) == 0 && len(in.wouldMake) == 0 { if len(left) == 0 && len(in.wouldMake) == 0 {
sayFound(ctx, d) sayFound(ctx, d)
out := sendableOf(d) out := sendableOf(d)
out.heldForCredential = held
out.fileReferences = readsFileReferences(in.Input, node) out.fileReferences = readsFileReferences(in.Input, node)
// The sender stamps the order it allots (order.stamp); a composition for a send carries none of its own. // The sender stamps the order it allots (order.stamp); a composition for a send carries none of its own.
out.Sequence, out.Epoch, out.Generation = 0, 0, 0 out.Sequence, out.Epoch, out.Generation = 0, 0, 0
+71
View File
@@ -7,6 +7,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose" "git.novox.be/novox/mesh-sdk/go/compose"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
) )
// A declaration as the SDK composes it is the one the controller sends, byte for byte, whatever it carries: the body is the // A declaration as the SDK composes it is the one the controller sends, byte for byte, whatever it carries: the body is the
@@ -85,3 +86,73 @@ func TestADeclarationThatStandsInForAValueIsStillForeseen(t *testing.T) {
} }
} }
} }
// 2026-11-01 (issue 203's refusal after the 1585 group): a module standing in for a bus account nobody issued is held out of
// the composition, alone; the rest of the node composes, with no stand-in left. An issued account holds nothing. And a held
// module with no fingerprint of the new form keeps the node's own composition as last written.
func TestAnUnissuedBusAccountHoldsItsModuleAloneAndNeverTheNode(t *testing.T) {
declared := func() compose.Declared {
return compose.Declared{
Resources: []map[string]any{
{"id": "hands.broker", "type": "file", "sealed": catalogue.ForeseenSealed},
{"id": "hands.unit", "type": "service"},
{"id": "client.broker", "type": "file", "sealed": catalogue.ForeseenSealed},
{"id": "web.config", "type": "file", "content": "ok"},
},
ModuleOf: map[string]string{"hands.broker": "hands", "hands.unit": "hands", "client.broker": "client",
"web.config": "web"},
BuildIDs: map[string]string{"hands": "b-1", "client": "b-2", "web": "b-3"},
Modules: []string{"client", "hands", "web"},
StoodIn: []string{"hands/broker", "client/" + compose.ProvisionerSecret},
Foreseen: []string{"hands/broker", "client/" + compose.ProvisionerSecret},
}
}
for name, c := range map[string]struct {
minted []string
held []string
}{
"issued by nobody": {held: []string{"client", "hands"}},
"the module's account issued": {minted: []string{"anchor.hands"}, held: []string{"client"}},
"both issued": {minted: []string{"anchor.hands", "provisioner.anchor"}},
} {
in := &meshInput{Input: compose.Input{BusMinted: c.minted}}
d := declared()
held := holdBackUnissuedBusAccounts(in, "anchor", &d)
if !reflect.DeepEqual(held, c.held) {
t.Errorf("%s: held %v, want %v", name, held, c.held)
continue
}
if len(c.held) < 2 {
continue // an issued account's stand-in is the send's to make (perform), not this one's
}
if left := stillForeseen(d); len(left) != 0 {
t.Errorf("%s: the rest of the node still stands in for %v", name, left)
}
if !reflect.DeepEqual(d.Modules, []string{"web"}) || len(d.Resources) != 1 || d.BuildIDs["hands"] != "" {
t.Errorf("%s: the rest of the node is not what composes: %v %v %v", name, d.Modules, d.Resources, d.BuildIDs)
}
if !reflect.DeepEqual(d.LeftOut, []string{"client", "hands"}) ||
!strings.Contains(d.LeftOutWhy["hands"], "module issue hands --node anchor") ||
!strings.Contains(d.LeftOutWhy["client"], "provisioner.anchor") || len(d.Said) != 2 {
t.Errorf("%s: the held modules are not said in issue 203's words: %v %v %v", name, d.LeftOut, d.LeftOutWhy, d.Said)
}
}
own, err := sdkdelivery.ComposedNodeKey("anchor")
if err != nil {
t.Fatal(err)
}
for name, c := range map[string]struct {
fps map[string]string
keepOwn bool
}{
"a held module of the new form": {fps: map[string]string{"hands": sdkdelivery.SourcePrefix2 + "x"}, keepOwn: true},
"a held module of the former form": {fps: map[string]string{}},
} {
comps := map[string]sdkdelivery.Composed{own: {Node: "anchor"}}
said := withoutNodeOwnForHeld("anchor", comps, c.fps, []string{"hands"})
if _, kept := comps[own]; kept != c.keepOwn || (said == "") != c.keepOwn {
t.Errorf("%s: the node's own composition kept %v (want %v), said %q", name, kept, c.keepOwn, said)
}
}
}
+3
View File
@@ -124,6 +124,9 @@ func composeModulesByNode(ctx context.Context, open *stores, nodes []string) (ma
errsOf[node] = append(errsOf[node], fmt.Errorf("%s: %w", node, err)) errsOf[node] = append(errsOf[node], fmt.Errorf("%s: %w", node, err))
return return
} }
if out := withoutNodeOwnForHeld(node, comps, fps, declared.heldForCredential); out != "" {
fmt.Printf("compositions: %s\n", out)
}
if len(moved) > 0 { if len(moved) > 0 {
fmt.Printf("compositions: %s's %s compose with the node's own: no fingerprint of the new form names them\n", fmt.Printf("compositions: %s's %s compose with the node's own: no fingerprint of the new form names them\n",
node, strings.Join(moved, ", ")) node, strings.Join(moved, ", "))
+26
View File
@@ -486,3 +486,29 @@ func controllerSigner(ctx context.Context, open *stores) (func(sdkdelivery.Compo
return sdkdelivery.SignComposedBy(c, public, func(m []byte) ([]byte, error) { return ident.Sign(ctx, m) }) return sdkdelivery.SignComposedBy(c, public, func(m []byte) ([]byte, error) { return ident.Sign(ctx, m) })
}, nil }, nil
} }
// withoutNodeOwnForHeld takes the node's own composition out of comps when a module held for a bus credential nobody
// issued (holdBackUnissuedBusAccounts) has no fingerprint of the new form: its resources would have been in the node's own,
// and the node's own written without them would have the node-engine remove a module that may be running. The node's
// own stays as last written; every module's composition is written. Answers what it said, "" when nothing was taken.
func withoutNodeOwnForHeld(node string, comps map[string]sdkdelivery.Composed, fps map[string]string, held []string) string {
var old []string
for _, module := range held {
if !strings.HasPrefix(fps[module], sdkdelivery.SourcePrefix2) {
old = append(old, module)
}
}
if len(old) == 0 {
return ""
}
own, err := sdkdelivery.ComposedNodeKey(node)
if err != nil {
return ""
}
if _, ok := comps[own]; !ok {
return ""
}
delete(comps, own)
return fmt.Sprintf("%s's own composition is not written this pass: %s, held for a bus account nobody issued, would "+
"leave it, and the node-engine would remove what may be running", node, strings.Join(old, ", "))
}
@@ -78,7 +78,7 @@ func aConfiguredMesh(t *testing.T) (*stores, configuredMesh, map[string]map[stri
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err := catalogue.ParseManifest(text) m, err := catalogue.Decode(text)
if err != nil { if err != nil {
t.Fatalf("%s: %v", name, err) t.Fatalf("%s: %v", name, err)
} }
+1 -1
View File
@@ -17,7 +17,7 @@ const n8nWithAGroup = `{"module":"n8n","version":"1",
// 1638/AC1: a node that reports the capability gets the module; one that does not has it held, the capability named. // 1638/AC1: a node that reports the capability gets the module; one that does not has it held, the capability named.
func TestAModuleWithGroupsIsHeldOnANodeThatDoesNotReportTheCapability(t *testing.T) { func TestAModuleWithGroupsIsHeldOnANodeThatDoesNotReportTheCapability(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(n8nWithAGroup)) m, err := catalogue.Decode([]byte(n8nWithAGroup))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -3,6 +3,7 @@
package main package main
import ( import (
"slices"
"strings" "strings"
"testing" "testing"
@@ -39,17 +40,23 @@ func TestAssigningAModuleThatSpeaksOnTheBusNamesItsCredential(t *testing.T) {
t.Fatalf("an assignment whose credential could not be issued does not name the verb:\n%s", said) t.Fatalf("an assignment whose credential could not be issued does not name the verb:\n%s", said)
} }
// And the push refuses to send it, naming the same verb, rather than sealing a placeholder. // And the push holds talker out, naming the same verb, rather than sealing a placeholder; the rest of the
// machine composes (2026-11-01: one unissued account refused the whole node that runs the bus).
plan, settings, err := resolveNode(ctx, open, "laptop") plan, settings, err := resolveNode(ctx, open, "laptop")
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
_, err = declarationFor(ctx, open, "laptop", plan, settings) held, err := declarationFor(ctx, open, "laptop", plan, settings)
if err == nil { if err != nil {
t.Fatal("a push sealed a placeholder where talker's bus credential belongs") t.Fatalf("one unissued account refused the whole machine: %v", err)
} }
if !strings.Contains(err.Error(), "module issue talker --node laptop") || !strings.Contains(err.Error(), "issue 203") { why := held.leftOutWhy["talker"]
t.Fatalf("the refusal does not say what to run: %v", err) if !slices.Contains(held.LeftOut, "talker") || !strings.Contains(why, "module issue talker --node laptop") ||
!strings.Contains(why, "issue 203") {
t.Fatalf("talker is not held out in issue 203's words: %v %q", held.LeftOut, why)
}
if body, err := held.Body(); err != nil || strings.Contains(string(body), catalogue.ForeseenSealed) {
t.Fatalf("a push would seal a placeholder where talker's bus credential belongs: %v\n%s", err, body)
} }
// Once the user is minted, the push goes on to the credential the mesh sealed, and re-assigning // Once the user is minted, the push goes on to the credential the mesh sealed, and re-assigning
+1 -1
View File
@@ -62,7 +62,7 @@ func TestNatsUnassigningIrreplaceableDataRetiresItAndAnEmptyReplacementIsUrgent(
Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}}, built); err != nil { Claims: []catalogue.Claim{{Name: catalogue.BackupSeat, Scope: catalogue.ScopeNode, Serves: []string{"backed-up", "now", "restore"}}}}, built); err != nil {
t.Fatal(err) t.Fatal(err)
} }
house, err := catalogue.ParseManifest([]byte(houseManifest)) house, err := catalogue.Decode([]byte(houseManifest))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
+1 -1
View File
@@ -24,7 +24,7 @@ func shelfFor(t *testing.T, manifests ...string) map[string]catalogue.Manifest {
t.Helper() t.Helper()
out := map[string]catalogue.Manifest{} out := map[string]catalogue.Manifest{}
for _, raw := range manifests { for _, raw := range manifests {
m, err := catalogue.ParseManifest([]byte(raw)) m, err := catalogue.Decode([]byte(raw))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
-72
View File
@@ -1,72 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Every catalogue module that runs something long-lived declares how it is ready (novox/hq ADR 0240 rule
// 8): `module check` warns and counts the undeclared before the date, and refuses them from it.
func TestModuleCheckCountsTheUndeclaredAndRefusesThemFromTheDate(t *testing.T) {
dir := t.TempDir()
digest := "@sha256:" + strings.Repeat("a", 64)
path := filepath.Join(dir, "module.json")
os.WriteFile(path, []byte(`{"module":"web","listens":[{"name":"web","port":80,"from":"mesh"}],"resources":[
{"id":"server","type":"container","name":"web","image":"registry.example/web`+digest+`","ports":["80"],
"health":{"kind":"http","endpoint":"web"}},
{"id":"worker","type":"container","name":"web-worker","image":"registry.example/web`+digest+`"},
{"id":"seed","type":"container","name":"web-seed","image":"registry.example/web`+digest+`","run-once":true}]}`), 0o600)
defer func() { checkNow = time.Now }()
checkNow = func() time.Time { return catalogue.HealthRequiredFrom.Add(-time.Hour) }
var out bytes.Buffer
if err := moduleCheck([]string{path}, &out); err != nil {
t.Fatalf("refused before the date: %v\n%s", err, out.String())
}
for _, want := range []string{"ready: server by http / on web every 30s", "WARNING: worker stay(s) up",
catalogue.HealthRequiredFrom.Format("2006-01-02"), UndeclaredHealthLine + " 1"} {
if !strings.Contains(out.String(), want) {
t.Errorf("the check does not say %q:\n%s", want, out.String())
}
}
checkNow = func() time.Time { return catalogue.HealthRequiredFrom }
out.Reset()
if err := moduleCheck([]string{path}, &out); err == nil {
t.Fatalf("a long-running resource without health passed after the date:\n%s", out.String())
}
if !strings.Contains(out.String(), "web: worker stays up and does not say how it is ready") {
t.Errorf("the refusal does not name the resource:\n%s", out.String())
}
}
// A file that asks for a setting without saying whether it is trusted counts as trusted (novox/hq issue 339):
// `module check` lists and counts it, and never refuses it — there is nothing unsafe to refuse.
func TestModuleCheckListsUnmarkedFilesAndNeverRefusesThem(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "module.json")
os.WriteFile(path, []byte(`{"module":"power","resources":[
{"id":"logind","type":"file","path":"/etc/systemd/logind.conf.d/power.conf","mode":"0644","trusted":true,
"content":"HandleLidSwitch=${setting:lid}\n"},
{"id":"note","type":"file","path":"/var/lib/power/note","mode":"0644","content":"${setting:greeting}\n"}]}`), 0o600)
defer func() { checkNow = time.Now }()
for _, at := range []time.Time{time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC), time.Date(2036, 1, 1, 0, 0, 0, 0, time.UTC)} {
checkNow = func() time.Time { return at }
var out bytes.Buffer
if err := moduleCheck([]string{path}, &out); err != nil {
t.Fatalf("refused at %v: %v\n%s", at, err, out.String())
}
for _, want := range []string{"note ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted",
UnsaidTrustLine + " 1"} {
if !strings.Contains(out.String(), want) {
t.Errorf("the check does not say %q:\n%s", want, out.String())
}
}
}
}
@@ -1,52 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
// novox/hq ADR 0225, issue 263: a consumer's identity is bounded by the provision it requires, an
// overflow is refused before merge by `module check`, and a provider's machine is never refused for
// one consumer's identity.
// `module check` refuses the pull request that introduces an overflow, naming the module.
func TestModuleCheckRefusesAnIdentityThatOverflowsWhatItRequires(t *testing.T) {
dir := t.TempDir()
write := func(name, body string) string {
p := filepath.Join(dir, name+".json")
if err := os.WriteFile(p, []byte(body), 0o600); err != nil {
t.Fatal(err)
}
return p
}
objects := write("objects", `{"module":"objects","version":"1",
"provides":[{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"},"credential":{"form":"pair"}}],
"receives":{"s3-bucket":"/var/lib/mesh/objects/mesh.json"}}`)
resolver := write("resolver", `{"module":"resolver","version":"1",
"provides":[{"name":"wildcard-resolution","scope":"mesh","identity":false,"credential":{"form":"none"}}]}`)
album := write("photoalbum", `{"module":"photoalbum","version":"1","requires":["s3-bucket"]}`)
nm := write("networkmanager", `{"module":"networkmanager","version":"1","requires":["wildcard-resolution"]}`)
var out bytes.Buffer
if err := moduleCheckFor([]string{resolver, nm}, 6, &out); err != nil {
t.Fatalf("a long name requiring a keyless provision was refused (issue 263): %v\n%s", err, out.String())
}
out.Reset()
err := moduleCheckFor([]string{objects, album, resolver, nm}, 6, &out)
if err == nil {
t.Fatalf("an identity overflowing an S3 access key passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "photoalbum wants s3-bucket") ||
!strings.Contains(out.String(), "`slug` of at most 8 characters") ||
strings.Contains(out.String(), "networkmanager wants") {
t.Fatalf("the refusal does not name the one overflowing module and its remedy:\n%s", out.String())
}
}
// Tonight's case, through the commands: networkmanager on a six-character machine requires the
// resolver provision, and a second consumer there overflows an object store's access key. The
// provider's machine still composes; the overflowing consumer is left out of its grants and named,
// by push and by `status`, and the keyless consumer is granted with its long name.
+53 -13
View File
@@ -2,9 +2,15 @@ package main
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
@@ -15,17 +21,19 @@ import (
// controller mints such an account and writes its hash to the bus's user list, as before, and then hands the whole // controller mints such an account and writes its hash to the bus's user list, as before, and then hands the whole
// value at once to the vault's `accept` with origin `issued`: sealed here to the key the vault made for that call, so // value at once to the vault's `accept` with origin `issued`: sealed here to the key the vault made for that call, so
// the vault alone opens it (ADR 0349 clause 8). The vault keeps it as the module's own secret on that machine and // the vault alone opens it (ADR 0349 clause 8). The vault keeps it as the module's own secret on that machine and
// raises `secret-changed`; a re-issue is the same hand with a new value, which replaces the old in place. An account // raises `secret-changed`; a re-issue is the same hand with a new value, which replaces the old in place. Until issue 774 an
// not covered is minted and kept as it was and handed to nobody. // account not covered was minted and kept as it was and handed to nobody.
//
// **Since issue 774 every account the controller mints is handed, covered or not**, once a vault is installed: until
// the grants part's take-over the controller still mints (the bus loads its list), and a vault that holds every value
// at once needs no node to report a hash for it to adopt (ADR 0547 clause 9). From the take-over the vault mints every
// account itself (issueAccount forwards to its issue-account), so this hand is reached only before it, and by
// `rollout mint`'s re-issue of a module's account onto a new bus. The vault never lets such a hand replace an account it
// minted.
// vaultAskerForIssue is the vault as `module issue` asks it. A test replaces it. // vaultAskerForIssue is the vault as `module issue` asks it. A test replaces it.
var vaultAskerForIssue = func(ctx context.Context) vaultAsker { return theVault(ctx) } var vaultAskerForIssue = func(ctx context.Context) vaultAsker { return theVault(ctx) }
// accountIsCovered says whether the bus accounts minted for m are the vault's to hold (ADR 0374 clause 1).
func accountIsCovered(m catalogue.Manifest) bool {
return catalogue.CarriesVaultSealed(m.Module) || m.Moved
}
// vaultInstalled says whether the vault is assigned to any machine. Until it is, as in genesis (ADR 0113 §15 to §18, // vaultInstalled says whether the vault is assigned to any machine. Until it is, as in genesis (ADR 0113 §15 to §18,
// mesh-host's bootstrap, which issues the bus's and the provisioner's accounts before the vault step), there is no // mesh-host's bootstrap, which issues the bus's and the provisioner's accounts before the vault step), there is no
// vault to hand to: the value is kept in the controller's store as before, and the vault takes it afterwards by the // vault to hand to: the value is kept in the controller's store as before, and the vault takes it afterwards by the
@@ -44,13 +52,45 @@ func vaultInstalled(ctx context.Context, inv *inventory.Inventory) (bool, error)
return false, nil return false, nil
} }
// handIssuedToVault hands the value just minted for module's own secret name on node to the vault, when the module's // grantsTookOver says whether the grants part took over the bus's user list: its key `user-list.sealed` says
// accounts are covered, and does nothing otherwise. Loud when the vault does not take it: the bus's user list // FollowGrants (novox/hq ADR 0547 clauses 7 and 8), which the bus node's engine follows. No bucket or no key yet is no
// already holds the new hash, so a vault that kept the old value would serve a password that no longer logs in. // take-over; a bus that cannot be read is an error, never a guess. A test replaces it.
func handIssuedToVault(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest, node, name, held string) error { var grantsTookOver = func(ctx context.Context) (bool, error) {
if !accountIsCovered(m) { over := false
return nil err := onTheBus(func(conn *nats.Conn) error {
js, err := jetstream.New(conn)
if err != nil {
return err
} }
kv, err := js.KeyValue(ctx, broker.GrantsUserListBucket)
if errors.Is(err, jetstream.ErrBucketNotFound) {
return nil
} else if err != nil {
return err
}
e, err := kv.Get(ctx, broker.GrantsUserListKey)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil
} else if err != nil {
return err
}
g, err := sdkdelivery.DecodeGrantsUserList(e.Value())
if err != nil {
return err
}
over = g.Follow == sdkdelivery.FollowGrants
return nil
})
if err != nil {
return false, fmt.Errorf("whether the grants part took over the bus's user list cannot be read: %w", err)
}
return over, nil
}
// handIssuedToVault hands the value just minted for module's own secret name on node to the vault, once one is
// installed, covered or not (issue 774), and does nothing before. Loud when the vault does not take it: the bus's user
// list already holds the new hash, so a vault that kept the old value would serve a password that no longer logs in.
func handIssuedToVault(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest, node, name, held string) error {
if installed, err := vaultInstalled(ctx, inv); err != nil { if installed, err := vaultInstalled(ctx, inv); err != nil {
return fmt.Errorf("whether the vault is installed cannot be read, so %s of %s was not handed to it: %w", name, return fmt.Errorf("whether the vault is installed cannot be read, so %s of %s was not handed to it: %w", name,
m.Module, err) m.Module, err)
+69 -8
View File
@@ -8,21 +8,50 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
sdkdelivery "git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/novox/mesh-controller/internal/broker" "github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/testbus"
) )
// Every other test mints without a vault to hand to: a stand-in that takes anything, replaced by the tests below. // grantsTookOverOnTheBus is the reader of the grants part's take-over a test reads on its own bus.
var grantsTookOverOnTheBus = grantsTookOver
// Every other test mints without a vault to hand to: a stand-in that takes anything, replaced by the tests below; and
// before the grants part's take-over, which no test bus says.
func init() { func init() {
vaultAskerForIssue = func(context.Context) vaultAsker { return newStandInVault(nil).ask } vaultAskerForIssue = func(context.Context) vaultAsker { return newStandInVault(nil).ask }
grantsTookOver = func(context.Context) (bool, error) { return false, nil }
} }
// A covered account is handed to the vault when it is minted (novox/hq ADR 0374, issue 787): the whole value, sealed // theGrantsPartTookOver says the grants part's take-over is on, or off, for the rest of the test.
// to the key the vault made for the call, with origin issued. A module that is neither core nor moved hands nothing. func theGrantsPartTookOver(t *testing.T, over bool) {
func TestModuleIssueHandsACoveredAccountToTheVaultAndOnlyThat(t *testing.T) { t.Helper()
was := grantsTookOver
grantsTookOver = func(context.Context) (bool, error) { return over, nil }
t.Cleanup(func() { grantsTookOver = was })
}
// An account is handed to the vault when it is minted (novox/hq ADR 0374, issue 787): the whole value, sealed to the
// key the vault made for the call, with origin issued. Since issue 774 every account is, covered or not, until the
// grants part's take-over, read on the bus as the bus node reads it; from the take-over nothing is minted here and the
// issue is forwarded to the vault's issue-account.
func TestModuleIssueHandsEveryAccountToTheVaultUntilTheTakeOver(t *testing.T) {
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
theVaultIsInstalled(t, open) theVaultIsInstalled(t, open)
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
servingBus.Store(js)
t.Cleanup(func() { servingBus.Store(nil) })
was := grantsTookOver
grantsTookOver = grantsTookOverOnTheBus
t.Cleanup(func() { grantsTookOver = was })
inv := open.inventory inv := open.inventory
known := broker.Broker{Fingerprint: "ab"} known := broker.Broker{Fingerprint: "ab"}
@@ -66,11 +95,11 @@ func TestModuleIssueHandsACoveredAccountToTheVaultAndOnlyThat(t *testing.T) {
t.Errorf("the vault was told %s, without %s", said, want) t.Errorf("the vault was told %s, without %s", said, want)
} }
} }
if n := strings.Count(said, `"origin":"issued"`); n != 3 { if n := strings.Count(said, `"origin":"issued"`); n != 4 {
t.Errorf("%d accounts were handed with origin issued, want 3 (the core's two and the moved module's): %s", n, said) t.Errorf("%d accounts were handed with origin issued, want 4 (the core's two, the moved and the plain module's): %s", n, said)
} }
if strings.Contains(said, "kept-bot") { if !strings.Contains(said, "kept-bot") {
t.Errorf("an account that is neither core nor moved was handed to the vault: %s", said) t.Errorf("an account that is neither core nor moved was not handed to the vault: %s", said)
} }
if strings.Contains(said, "first-core-password") || strings.Contains(said, "first-moved-password") { if strings.Contains(said, "first-core-password") || strings.Contains(said, "first-moved-password") {
t.Errorf("a value crossed the bus in the clear: %s", said) t.Errorf("a value crossed the bus in the clear: %s", said)
@@ -83,6 +112,38 @@ func TestModuleIssueHandsACoveredAccountToTheVaultAndOnlyThat(t *testing.T) {
!strings.Contains(got, `"url":"nats://10.0.0.1:4222"`) || !strings.Contains(got, `"fingerprint":"ab"`) { !strings.Contains(got, `"url":"nats://10.0.0.1:4222"`) || !strings.Contains(got, `"fingerprint":"ab"`) {
t.Errorf("the vault was handed %q after a re-issue, not the new account whole", got) t.Errorf("the vault was handed %q after a re-issue, not the new account whole", got)
} }
// The take-over: the grants part's key says the bus node follows its list. From then the vault mints.
api, err := jetstream.New(js.Conn())
if err != nil {
t.Fatal(err)
}
kv, err := api.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: broker.GrantsUserListBucket})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = api.DeleteKeyValue(context.Background(), broker.GrantsUserListBucket) })
client := broker.Principal{Kind: broker.KindProvisioner, Node: "laptop", Module: "nats"}.Username()
for _, follow := range []string{sdkdelivery.FollowController, sdkdelivery.FollowGrants} {
value, err := sdkdelivery.EncodeGrantsUserList(sdkdelivery.GrantsUserList{Follow: follow})
if err != nil {
t.Fatal(err)
}
if _, err := kv.Put(ctx, broker.GrantsUserListKey, value); err != nil {
t.Fatal(err)
}
before, _, _ := inv.BusUserHash(ctx, client)
if err := issueProvisionerClient(ctx, inv, core, "laptop", known, "10.0.0.1:4222"); err != nil {
t.Fatal(err)
}
after, _, _ := inv.BusUserHash(ctx, client)
if minted := before != after; minted != (follow == sdkdelivery.FollowController) {
t.Errorf("following %s's list, the controller minted %s: %v", follow, client, minted)
}
}
if said := vault.said(); strings.Count(said, `"verb":"issue-account"`) != 1 || !strings.Contains(said, `"user":"`+client+`"`) {
t.Errorf("the vault was not asked once, after the take-over, to issue %s: %s", client, said)
}
} }
// A vault that does not take the value fails the issue loudly: the bus's user list holds the new hash already. // A vault that does not take the value fails the issue loudly: the bus's user list holds the new hash already.
-12
View File
@@ -219,15 +219,3 @@ func theGraph(ctx context.Context, inv *inventory.Inventory) ([]inventory.Entry,
} }
return entries, read, edges, nil return entries, read, edges, nil
} }
// refusedWord is the word a build's failure says the reader did not know; empty when it says none.
func refusedWord(failed string) string {
if m := unknownWord.FindStringSubmatch(failed); m != nil {
return m[1]
}
return ""
}
// unknownWord is how the strict reading of a manifest says a word is not one it knows: encoding/json's own words, which
// the build-agent's answer carries whole. The build's answer has no typed field for it (Failed is the builder's words).
var unknownWord = regexp.MustCompile(`json: unknown field "([^"]+)"`)
+20 -8
View File
@@ -51,6 +51,8 @@ var errKeyFailed = errors.New("the builder's key failed")
type keyReads interface { type keyReads interface {
Build(ctx context.Context, module, fp string) (builderkeys.Key, error) Build(ctx context.Context, module, fp string) (builderkeys.Key, error)
Manifest(ctx context.Context, digest string) (sdkdelivery.Manifest, error) Manifest(ctx context.Context, digest string) (sdkdelivery.Manifest, error)
// Verdict is the indexer's verdict on the manifest of digest, nil when it wrote none (taken as accepted).
Verdict(ctx context.Context, digest string) (*sdkdelivery.ManifestVerdict, error)
} }
// keyBuildID is the id a key's build is recorded under: the key and its revision, so the same revision recorded twice // keyBuildID is the id a key's build is recorded under: the key and its revision, so the same revision recorded twice
@@ -335,16 +337,26 @@ func buildOfKey(ctx context.Context, keys keyReads, t keyTake, kc keyContext) (
if err != nil { if err != nil {
return waits("%s's manifest: %v", t.Module, err) return waits("%s's manifest: %v", t.Module, err)
} }
manifest, err := catalogue.ParseManifestWith([]byte(src.JSON), kc.list) // **The controller judges no manifest; the indexer did, once** (ADR 0578 §3, §6): the manifest is read leniently, and
// the indexer's verdict on its digest is taken. A refused one is held and named, never registered; a word this
// controller does not read is set aside and named, and the rest is registered with no second build.
manifest, err := catalogue.Decode([]byte(src.JSON))
if err != nil { if err != nil {
// **A word the running controller does not read waits for the controller that reads it** (ADR 0501 §7): the
// key stays done, and the controller that reads the word registers it with no second build.
if word := refusedWord(err.Error()); word != "" {
return waits("%s's manifest uses the word %q, which this controller does not read: it is registered "+
"when a controller that reads it runs", t.Module, word)
}
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the builder built %s and its manifest is "+ return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the builder built %s and its manifest is "+
"refused: %w", t.Module, err) "not read: %w", t.Module, err)
}
verdict, err := keys.Verdict(ctx, k.Build.Manifest)
if err != nil {
return waits("%s's verdict: %v", t.Module, err)
}
if refused, _ := sdkdelivery.TakeManifestVerdict(verdict); refused {
return waits("%s is held: %s", t.Module, verdict.Why())
}
if manifest.ReadNoFurther() {
return waits("%s is held: %s", t.Module, manifest.SetAside())
}
if aside := manifest.SetAside(); aside != "" {
fmt.Printf("%s's manifest has a field this controller does not read, set aside: %s\n", t.Module, aside)
} }
if manifest.Module != t.Module { if manifest.Module != t.Module {
return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the key of %s names the manifest of %s", return inventory.Build{}, catalogue.Manifest{}, fmt.Errorf("the key of %s names the manifest of %s",
+38 -7
View File
@@ -23,6 +23,14 @@ import (
type keyBuckets struct { type keyBuckets struct {
keys map[string]builderkeys.Key keys map[string]builderkeys.Key
manifests map[string]string manifests map[string]string
verdicts map[string]sdkdelivery.ManifestVerdict // by manifest digest; none is accepted
}
func (k keyBuckets) Verdict(_ context.Context, digest string) (*sdkdelivery.ManifestVerdict, error) {
if v, ok := k.verdicts[digest]; ok {
return &v, nil
}
return nil, nil
} }
func (k keyBuckets) Build(_ context.Context, module, fp string) (builderkeys.Key, error) { func (k keyBuckets) Build(_ context.Context, module, fp string) (builderkeys.Key, error) {
@@ -148,15 +156,38 @@ func TestAKeyNotReadyWaitsAndAFailedKeyFails(t *testing.T) {
} }
} }
// ADR 0501 §7: a manifest word this controller does not read waits for the controller that reads it; never fails. // ADR 0578 §3, §6 (changing ADR 0501 §7): the controller judges no manifest. A word it does not read is set aside and the
func TestAManifestWordTheControllerDoesNotReadWaits(t *testing.T) { // module registered with no second build; the module the indexer's verdict refuses is held and named, never failed.
raw := `{"module":"x","version":"1","not-a-word-yet":true,"build":{"artifacts":[{"name":"a","kind":"archive","from":"files"}]}, func TestAKeyIsTakenInOnTheIndexersVerdictAndNeverJudgedHere(t *testing.T) {
const tail = `"build":{"artifacts":[{"name":"a","kind":"archive","from":"files"}]},
"resources":[{"id":"a","type":"archive","path":"/opt/x","artifact":"a"}]}` "resources":[{"id":"a","type":"archive","path":"/opt/x","artifact":"a"}]}`
keys, trunk := aKeyOf(raw, 1, sdkdelivery.BuiltArtifact{Name: "a", Kind: "archive", Digest: shaOf("a"), refusal := func(digest string) map[string]sdkdelivery.ManifestVerdict {
return map[string]sdkdelivery.ManifestVerdict{digest: {Module: "x", State: sdkdelivery.VerdictRefused,
Problems: []string{"x names a seat nothing declares"}}}
}
for _, row := range []struct {
name, raw string
refused bool
holds string
}{
{"an unread word", `{"module":"x","version":"1","not-a-word-yet":true,` + tail, false, ""},
{"a fallback to a predefined key, no list at hand", `{"module":"x","version":"1","config":{"font-size":{"type":"number",
"description":"size","default":10,"predefined":"font-size"}},` + tail, false, ""},
{"no verdict is accepted", `{"module":"x","version":"1",` + tail, false, ""},
{"a refusal is held and named", `{"module":"x","version":"1",` + tail, true, "names a seat nothing declares"},
} {
keys, trunk := aKeyOf(row.raw, 1, sdkdelivery.BuiltArtifact{Name: "a", Kind: "archive", Digest: shaOf("a"),
Reference: "artifact-store://x/a/blobs/" + shaOf("a")}) Reference: "artifact-store://x/a/blobs/" + shaOf("a")})
if row.refused {
keys.verdicts = refusal(sdkdelivery.ManifestDigest([]byte(row.raw)))
}
_, _, err := buildOfKey(t.Context(), keys, keyTake{Module: "x", Trunk: trunk}, keyContext{}) _, _, err := buildOfKey(t.Context(), keys, keyTake{Module: "x", Trunk: trunk}, keyContext{})
if !errors.Is(err, errKeyWaits) || !strings.Contains(err.Error(), `"not-a-word-yet"`) { switch {
t.Fatalf("an unread word: %v", err) case row.holds == "" && err != nil:
t.Errorf("%s: %v", row.name, err)
case row.holds != "" && (!errors.Is(err, errKeyWaits) || !strings.Contains(err.Error(), row.holds)):
t.Errorf("%s: %v", row.name, err)
}
} }
} }
@@ -215,7 +246,7 @@ func TestATemplatedModulesTextsAreReadFromItsArchive(t *testing.T) {
// The launchers a key's bundle is resolved with are mesh-sdk's one rule (buildstep.Launchers), the names // The launchers a key's bundle is resolved with are mesh-sdk's one rule (buildstep.Launchers), the names
// mesh-build-node writes: a TypeScript bundle's entrypoint x.js runs as x.serve.mjs, a Go bundle has none. // mesh-build-node writes: a TypeScript bundle's entrypoint x.js runs as x.serve.mjs, a Go bundle has none.
func TestALaunchersRuleIsTheBuildModules(t *testing.T) { func TestALaunchersRuleIsTheBuildModules(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(tsBundle)) m, err := catalogue.Decode([]byte(tsBundle))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
+1 -1
View File
@@ -23,7 +23,7 @@ func leftOutNFS(t *testing.T) catalogue.Resolution {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err := catalogue.ParseManifest(raw) m, err := catalogue.Decode(raw)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -73,12 +73,10 @@ func buildFrom(result link.BuildResult) inventory.Build {
kept.Manifest = recordedManifest(result.Manifest, announced) kept.Manifest = recordedManifest(result.Manifest, announced)
// The module name comes from the manifest, which only exists when the build got that far. // The module name comes from the manifest, which only exists when the build got that far.
if len(kept.Manifest) > 0 { if len(kept.Manifest) > 0 {
if list, err := judgedWith(result); err == nil { if m, err := catalogue.Decode(kept.Manifest); err == nil {
if m, err := catalogue.ParseManifestWith(kept.Manifest, list); err == nil {
kept.Module = m.Module kept.Module = m.Module
} }
} }
}
return kept return kept
} }
@@ -130,15 +128,7 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s", return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed) result.On, result.Repository, result.Failed)
} }
// Judged first against the list the build was judged against: the catalogue's at the commit built (novox/hq manifest, err := catalogue.Decode(kept.Manifest)
// ADR 0348 §7), so a module naming a key its own commit adds is read; and again below against the list the
// mesh publishes, before it is registered.
list, err := judgedWith(result)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w", result.On,
result.Repository, short(result.Commit), err)
}
manifest, err := catalogue.ParseManifestWith(kept.Manifest, list)
if err != nil { if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w", return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err) result.On, result.Repository, err)
@@ -268,16 +258,6 @@ func carriedList(result link.BuildResult) (catalogue.PredefinedList, error) {
return list, nil return list, nil
} }
// judgedWith is the list a build's manifest is first judged against at registration: the catalogue's own at the
// commit built, which its build judged it against, where it carried one; the list kept otherwise.
func judgedWith(result link.BuildResult) (catalogue.PredefinedList, error) {
list, err := carriedList(result)
if err != nil || list != nil {
return list, err
}
return catalogue.PublishedPredefined(), nil
}
// publishPredefined keeps a trunk build's list when it is newer than the one kept, says so, and loads what the // publishPredefined keeps a trunk build's list when it is newer than the one kept, says so, and loads what the
// store keeps then — this build's, or a newer one another controller process kept. // store keeps then — this build's, or a newer one another controller process kept.
func publishPredefined(ctx context.Context, inv *inventory.Inventory, result link.BuildResult, func publishPredefined(ctx context.Context, inv *inventory.Inventory, result link.BuildResult,
-1
View File
@@ -223,7 +223,6 @@ var usageText = `mesh-controller — the control plane
overlay place <node> [flags] say where a node is and how it is reached overlay place <node> [flags] say where a node is and how it is reached
overlay show the private network, as the mesh computes it overlay show the private network, as the mesh computes it
module add <file> register a module from its manifest module add <file> register a module from its manifest
module check <file|dir>... judge manifests where they are written, with no mesh (exit 1 on any problem)
module list what modules this mesh knows about module list what modules this mesh knows about
module moved <name> <commit> the source has a newer commit than the mesh built module moved <name> <commit> the source has a newer commit than the mesh built
module forget <name> remove one, unless a node runs it or the mesh holds things for it module forget <name> remove one, unless a node runs it or the mesh holds things for it
+127 -93
View File
@@ -8,7 +8,6 @@ import (
"fmt" "fmt"
"net" "net"
"os" "os"
"path/filepath"
"slices" "slices"
"sort" "sort"
"strings" "strings"
@@ -64,35 +63,7 @@ var provided = providedModules()
func moduleCommand(ctx context.Context, args []string) error { func moduleCommand(ctx context.Context, args []string) error {
if len(args) == 0 { if len(args) == 0 {
return errors.New("module add <file>, module check <file>..., module list, or module forget <name>") return errors.New("module add <file>, module list, or module forget <name>")
}
// `check` needs no mesh, and must not: it is what somebody runs in their own repository before
// there is a mesh in reach (novox/hq issue 148). A directory expands to every manifest under it.
if args[0] == "check" {
set := flag.NewFlagSet("module check", flag.ContinueOnError)
// The longest machine name an identity must fit on (novox/hq ADR 0225): a mesh passes its own.
longest := set.Int("longest-machine-name", catalogue.DefaultLongestMachine,
"judge each module's identity on a machine name this many characters long")
given, err := parseAround(set, args[1:])
if err != nil {
return err
}
if *longest < 1 {
return errors.New("--longest-machine-name is a length, at least 1")
}
var paths []string
for _, a := range given {
if info, err := os.Stat(a); err == nil && info.IsDir() {
under, err := manifestsUnder(a)
if err != nil {
return err
}
paths = append(paths, under...)
continue
}
paths = append(paths, a)
}
return moduleCheckFor(paths, *longest, os.Stdout)
} }
open, err := openStores(ctx) open, err := openStores(ctx)
if err != nil { if err != nil {
@@ -126,21 +97,19 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
m, err := catalogue.ParseManifest(raw) m, err := catalogue.Decode(raw)
if err != nil { if err != nil {
return err return err
} }
// A manifest handed over by hand takes each template's text from the tree beside it (novox/hq ADR 0349 §2). // A manifest handed over by hand is not judged here (ADR 0578 §3): a templated one has no build to read its
if m, err = catalogue.WithTemplatesFrom(m, filepath.Dir(positionals[0])); err != nil { // texts from, so it is refused by name.
return err if p := catalogue.HeldTemplateProblem(m); p != "" {
return errors.New(p)
} }
from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self) from, err := whereItComesFrom(*repo, *ref, *commit, *path, *self)
if err != nil { if err != nil {
return err return err
} }
if err := namesNoInstallation(m); err != nil {
return err
}
if err := inv.RegisterModule(ctx, m, from); err != nil { if err := inv.RegisterModule(ctx, m, from); err != nil {
return err return err
} }
@@ -1018,8 +987,21 @@ const issueLockedFor = 2 * time.Minute
// left ace's mesh-build-go with no credential), under the store's lock on the account, so two issuers never interleave // left ace's mesh-build-go with no credential), under the store's lock on the account, so two issuers never interleave
// their look, mint and seal: the hash the bus holds is always the password sealed. A seal that fails leaves the // their look, mint and seal: the hash the bus holds is always the password sealed. A seal that fails leaves the
// account unsealed, issued again by the next issue. // account unsealed, issued again by the next issue.
func issueAccount(ctx context.Context, accounts accountIssuer, u inventory.BusUser, seal func(password string) error) error { //
// **From the grants part's take-over the controller mints nothing** (issue 774, research 072's gap "vault mints after
// the lease"; the controller is deprecated and hands its duties over): the bus then loads the grants part's list, whose
// hashes are the vault's, so the issue is forwarded to the vault's issue-account and only after runs (what the account
// needs beside its value: its durable consumers). **Before it** the bus loads the controller's own list, whose hashes
// come from its store alone, so an account the vault minted would be refused: the controller mints as before and hands
// every account it mints to an installed vault at once (handIssuedToVault), so none waits on a node reporting its hash.
func issueAccount(ctx context.Context, accounts accountIssuer, u inventory.BusUser, seal func(password string) error,
after func() error) error {
only, _ := ctx.Value(mintOnlyIfAbsent{}).(bool) only, _ := ctx.Value(mintOnlyIfAbsent{}).(bool)
if ask, installed, err := vaultIssuer(ctx, accounts); err != nil {
return fmt.Errorf("whether the vault issues bus accounts yet cannot be read, so %s is not issued: %w", u.Username, err)
} else if installed {
return forwardToTheVault(ctx, ask, u.Username, only, after)
}
if !only { if !only {
password, err := accounts.MintBusPassword(ctx, u) password, err := accounts.MintBusPassword(ctx, u)
if err != nil { if err != nil {
@@ -1062,6 +1044,67 @@ func issueAccount(ctx context.Context, accounts accountIssuer, u inventory.BusUs
return nil return nil
} }
// vaultIssuer answers the vault to forward an issue to, and whether the issue is the vault's: one is installed
// (vaultInstalled) and the grants part took over the bus's user list (grantsTookOver). A store that is not the
// controller's inventory (a test's) has none. A test replaces it.
var vaultIssuer = func(ctx context.Context, accounts accountIssuer) (vaultAsker, bool, error) {
inv, ok := accounts.(*inventory.Inventory)
if !ok {
return nil, false, nil
}
installed, err := vaultInstalled(ctx, inv)
if err != nil || !installed {
return nil, false, err
}
if over, err := grantsTookOver(ctx); err != nil || !over {
return nil, false, err
}
return vaultAskerForIssue(ctx), true, nil
}
// issuedByTheVault is the vault's answer to issue-account (mesh-vault heldBackOrMinted).
type issuedByTheVault struct {
Minted bool `json:"minted"`
Why string `json:"why"`
Account struct {
Secret string `json:"secret"`
Generation int `json:"generation"`
} `json:"account"`
}
// forwardToTheVault asks the vault to issue user (its verb issue-account) and then runs after. The vault mints only
// what the grants part wants and never anew a value it holds, so asking twice, or with the want's own mint, is harmless.
// An issue on an assignment (only) is not failed by a refusal: the vault mints on the want itself when it is heard, and
// the refusal is said; `module issue` fails with it.
func forwardToTheVault(ctx context.Context, ask vaultAsker, user string, only bool, after func() error) error {
raw, err := ask("issue-account", map[string]any{"user": user})
if err != nil {
if !only {
return fmt.Errorf("the vault is the one issuer of bus accounts and did not issue %s: %w", user, err)
}
fmt.Printf(" %s is the vault's to mint, on the grants part's want of it; issue-account said: %v\n", user, err)
} else {
var got issuedByTheVault
if err := json.Unmarshal(raw, &got); err != nil {
return fmt.Errorf("the vault's answer to issue-account %s does not read: %w", user, err)
}
if got.Minted {
fmt.Printf("bus user %s minted by the vault as %s (generation %d)\n", user, got.Account.Secret, got.Account.Generation)
if g, _ := ctx.Value(issuedNowKey{}).(*issuedNow); g != nil {
g.mu.Lock()
g.users = append(g.users, user)
g.mu.Unlock()
}
} else {
fmt.Printf("bus user %s not minted anew by the vault: %s\n", user, got.Why)
}
}
if after == nil {
return nil
}
return after()
}
// issueOnTheNewBus gives an assigned module its credential on the bus being built. // issueOnTheNewBus gives an assigned module its credential on the bus being built.
// //
// **Three things differ from a management call, and each is the point of the move.** The credential // **Three things differ from a management call, and each is the point of the move.** The credential
@@ -1090,7 +1133,7 @@ func issueOnTheNewBus(ctx context.Context, inv *inventory.Inventory, m catalogue
if err := issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: busKindOf(m.Module), Node: node, if err := issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: busKindOf(m.Module), Node: node,
Module: m.Module}, func(password string) error { Module: m.Module}, func(password string) error {
return issueWith(ctx, inv, m, node, busAddress, known, reachable, user, password) return issueWith(ctx, inv, m, node, busAddress, known, reachable, user, password)
}); err != nil { }, func() error { return ensureModuleConsumer(m, node, busAddress) }); err != nil {
return err return err
} }
} }
@@ -1118,7 +1161,7 @@ func issueProcess(ctx context.Context, inv *inventory.Inventory, m catalogue.Man
return issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: inventory.BusProcess, Node: node, Module: m.Module}, return issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: inventory.BusProcess, Node: node, Module: m.Module},
func(password string) error { func(password string) error {
return sealProcess(ctx, inv, m, pr, node, busAddress, known, reachable, user, password) return sealProcess(ctx, inv, m, pr, node, busAddress, known, reachable, user, password)
}) }, func() error { return ensureProcessQueues(m, pr, node, busAddress) })
} }
// sealProcess seals a process's minted account to its node, hands it to the vault and makes its work-queue consumers. // sealProcess seals a process's minted account to its node, hands it to the vault and makes its work-queue consumers.
@@ -1135,15 +1178,29 @@ func sealProcess(ctx context.Context, inv *inventory.Inventory, m catalogue.Mani
if err := handIssuedToVault(ctx, inv, m, node, name, string(held)); err != nil { if err := handIssuedToVault(ctx, inv, m, node, name, string(held)); err != nil {
return err return err
} }
if len(pr.Queues) > 0 { if err := ensureProcessQueues(m, pr, node, busAddress); err != nil {
return err
}
fmt.Printf("bus user %s minted for %s's process %s, scoped to its bus declaration\n", user, m.Module, pr.Process)
fmt.Printf(" sealed to %s as %s. It arrives with the next push — `push %s` to send it\n", node, name, node)
return nil
}
// ensureProcessQueues makes a process's work-queue consumers where the bus can be reached: what its account needs beside
// its value, whoever minted it.
func ensureProcessQueues(m catalogue.Manifest, pr broker.ProcessPrincipal, node, busAddress string) error {
if len(pr.Queues) == 0 {
return nil
}
if busAddress == "" { if busAddress == "" {
fmt.Printf(" %s's process %s consumes its work queue; its consumer is created when the bus is "+ fmt.Printf(" %s's process %s consumes its work queue; its consumer is created when the bus is "+
"reachable (`push --behind`, then `rollout mint` again is harmless)\n", m.Module, pr.Process) "reachable (`push --behind`, then `rollout mint` again is harmless)\n", m.Module, pr.Process)
} else { return nil
}
js, err := broker.Dial(busAddress) js, err := broker.Dial(busAddress)
if err != nil { if err != nil {
return fmt.Errorf("the credential is minted and the mesh cannot reach the bus to create "+ return fmt.Errorf("the mesh cannot reach the bus to create %s's process %s's work-queue consumer: %w",
"%s's process %s's work-queue consumer: %w", m.Module, pr.Process, err) m.Module, pr.Process, err)
} }
defer js.Close() defer js.Close()
who := broker.Principal{Kind: broker.KindProcess, Node: node, Module: m.Module, Process: pr.Process} who := broker.Principal{Kind: broker.KindProcess, Node: node, Module: m.Module, Process: pr.Process}
@@ -1152,10 +1209,6 @@ func sealProcess(ctx context.Context, inv *inventory.Inventory, m catalogue.Mani
return err return err
} }
} }
}
}
fmt.Printf("bus user %s minted for %s's process %s, scoped to its bus declaration\n", user, m.Module, pr.Process)
fmt.Printf(" sealed to %s as %s. It arrives with the next push — `push %s` to send it\n", node, name, node)
return nil return nil
} }
@@ -1191,7 +1244,7 @@ func issueProvisionerClient(ctx context.Context, inv *inventory.Inventory, m cat
return issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: inventory.BusProvisioner, Node: node, return issueAccount(ctx, inv, inventory.BusUser{Username: user, Kind: inventory.BusProvisioner, Node: node,
Module: m.Module}, func(password string) error { Module: m.Module}, func(password string) error {
return sealProvisionerClient(ctx, inv, m, node, known, reachable, user, password) return sealProvisionerClient(ctx, inv, m, node, known, reachable, user, password)
}) }, nil)
} }
// sealProvisionerClient seals the provisioner client's minted account to its machine and hands it to the vault. // sealProvisionerClient seals the provisioner client's minted account to its machine and hands it to the vault.
@@ -1263,29 +1316,9 @@ func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manife
if err := handIssuedToVault(ctx, inv, m, node, "broker", string(held)); err != nil { if err := handIssuedToVault(ctx, inv, m, node, "broker", string(held)); err != nil {
return err return err
} }
if err := ensureModuleConsumer(m, node, busAddress); err != nil {
// And how it hears what it consumes. Derived from its declaration, and only when it declared
// something: a module that consumes nothing needs no consumer, and creating one would be a
// durable subscription nobody reads.
if consumer, needed := broker.ConsumerFor(broker.Principal{
Kind: broker.KindModule, Node: node, Module: m.Module,
Emits: m.EmitsAll(), Consumes: m.Consumes, Serves: m.Tools,
}); needed {
if busAddress == "" {
fmt.Printf(" %s consumes; its consumer is created when the bus is reachable (`push --behind`, then "+
"`rollout mint` again is harmless)\n", m.Module)
} else {
js, err := broker.Dial(busAddress)
if err != nil {
return fmt.Errorf("the credential is minted and the mesh cannot reach the bus to create "+
"how %s hears what it consumes: %w", m.Module, err)
}
defer js.Close()
if err := js.EnsureConsumer(consumer); err != nil {
return err return err
} }
}
}
fmt.Printf("bus user %s minted for %s, scoped to what it emits and consumes\n", user, m.Module) fmt.Printf("bus user %s minted for %s, scoped to what it emits and consumes\n", user, m.Module)
fmt.Printf(" sealed to %s. It arrives with the next push — `push %s` to send it\n", node, node) fmt.Printf(" sealed to %s. It arrives with the next push — `push %s` to send it\n", node, node)
@@ -1294,6 +1327,30 @@ func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manife
return nil return nil
} }
// ensureModuleConsumer makes how a module hears what it consumes, where the bus can be reached: what its account needs
// beside its value, whoever minted it. Derived from its declaration, and only when it declared something: a module that
// consumes nothing needs no consumer, and creating one would be a durable subscription nobody reads.
func ensureModuleConsumer(m catalogue.Manifest, node, busAddress string) error {
consumer, needed := broker.ConsumerFor(broker.Principal{
Kind: broker.KindModule, Node: node, Module: m.Module,
Emits: m.EmitsAll(), Consumes: m.Consumes, Serves: m.Tools,
})
if !needed {
return nil
}
if busAddress == "" {
fmt.Printf(" %s consumes; its consumer is created when the bus is reachable (`push --behind`, then "+
"`rollout mint` again is harmless)\n", m.Module)
return nil
}
js, err := broker.Dial(busAddress)
if err != nil {
return fmt.Errorf("the mesh cannot reach the bus to create how %s hears what it consumes: %w", m.Module, err)
}
defer js.Close()
return js.EnsureConsumer(consumer)
}
// whereItComesFrom is the provenance a module handed over by hand records, and what a record must // whereItComesFrom is the provenance a module handed over by hand records, and what a record must
// say to be worth anything later. // say to be worth anything later.
// //
@@ -1328,29 +1385,6 @@ func whereItComesFrom(repository, ref, commit, path string, self bool) (inventor
return from, nil return from, nil
} }
// namesNoInstallation is the mesh refusing a definition that names an installation, at the moment
// it would enter the catalogue (novox/hq ADR 0112, ADR 0155). `module check` says the same thing
// earlier, where the author is; this is the last moment the mesh can still say no, and a
// definition that got past the check — written elsewhere, or checked by nobody — is refused here
// in the same words. A name meant on purpose is declared with its reason and passes.
func namesNoInstallation(m catalogue.Manifest) error {
if problems := catalogue.TrustProblems(m); len(problems) > 0 {
return fmt.Errorf("%s", strings.Join(problems, "; "))
}
// And a module that declares a setting names the keys of its mergeable files (novox/hq ADR 0326
// §3, issue 574): judged where a definition is written, never where a stored build is read again.
if problems := catalogue.WrittenSettingsProblems(m); len(problems) > 0 {
return fmt.Errorf("%s", strings.Join(problems, "; "))
}
named := catalogue.InstallationProblems(m)
if len(named) == 0 {
return nil
}
return fmt.Errorf("%s names an installation, and a definition names none — declare a name meant "+
"on purpose under %s with its reason, or take it out:\n - %s",
m.Module, catalogue.NamesOnPurpose, strings.Join(named, "\n - "))
}
// seatClaimed is one seat a module claims, as its runtime needs it: the name, the scope (a // seatClaimed is one seat a module claims, as its runtime needs it: the name, the scope (a
// node-scoped seat's verb carries the machine, design 33 §4) and the verbs the seat promises. // node-scoped seat's verb carries the machine, design 33 §4) and the verbs the seat promises.
type seatClaimed struct { type seatClaimed struct {
@@ -18,6 +18,7 @@ import (
// one module's directory and touches nothing else the mesh holds: nothing asks its first build, it opens no plan, and // one module's directory and touches nothing else the mesh holds: nothing asks its first build, it opens no plan, and
// the module is registered from the builder's first done key of it, assigned nowhere, with no build request written. // the module is registered from the builder's first done key of it, assigned nowhere, with no build request written.
func TestAMergeAddingAModuleEndsRegisteredFromItsKeyAndAsksNoBuild(t *testing.T) { func TestAMergeAddingAModuleEndsRegisteredFromItsKeyAndAsksNoBuild(t *testing.T) {
withNoNewModuleWaits(t)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
@@ -66,9 +67,11 @@ func TestAMergeAddingAModuleEndsRegisteredFromItsKeyAndAsksNoBuild(t *testing.T)
} }
} }
// ADR 0501 §7: a new module whose manifest uses a word this controller does not read is named waiting at once, never // ADR 0578 §6 (which changes ADR 0501 §7): a new module whose manifest uses a word this controller does not read is
// failed, and registered from the same key, with no second build, when a controller that reads it looks. // registered at once, the word set aside, never failed and with no second build; one whose manifest the indexer's judge
func TestANewModuleWithAnUnreadWordWaitsForTheControllerThatReadsIt(t *testing.T) { // refused is held, named in the indexer's words, and never registered (ADR 0578 §3).
func TestANewModuleWithAnUnreadWordIsRegisteredAndARefusedOneIsHeld(t *testing.T) {
withNoNewModuleWaits(t)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
inv := open.inventory inv := open.inventory
@@ -94,12 +97,38 @@ func TestANewModuleWithAnUnreadWordWaitsForTheControllerThatReadsIt(t *testing.T
} }
b.built(t, "watchman", "1", fp, digest, sdkdelivery.BuildDone) b.built(t, "watchman", "1", fp, digest, sdkdelivery.BuildDone)
takeNewModules(ctx, open, "watchman") takeNewModules(ctx, open, "watchman")
if _, known := inventoryHolds(t, inv, "watchman"); known { if _, known := inventoryHolds(t, inv, "watchman"); !known {
t.Fatal("0501 §7: a manifest with an unread word was registered") t.Fatalf("0578 §6: a manifest with a word set aside is not registered: %+v", newModuleObservations(time.Now()))
} }
obs := newModuleObservations(time.Now()) // The indexer refused another: held, named, not registered.
if len(obs) != 1 || obs[0].Kind != kindNewModuleWaits || !strings.Contains(obs[0].Summary, `"not-a-word-yet"`) { refusedFP := fpOf('9')
t.Fatalf("0501 §7: the module waiting for a reader of its word is named by %+v", obs) refusedRaw := `{"module":"refusedmod","version":"1","build":{"artifacts":[{"name":"a","kind":"archive","from":"files"}]},` +
`"resources":[{"id":"a","type":"archive","path":"/opt/refusedmod","artifact":"a"}]}`
refusedDigest := sdkdelivery.ManifestDigest([]byte(refusedRaw))
value, _ = sdkdelivery.EncodeManifest(sdkdelivery.Manifest{Module: "refusedmod", JSON: refusedRaw})
if _, err := b.manifests.Put(ctx, refusedDigest, value); err != nil {
t.Fatal(err)
}
verdict, _ := sdkdelivery.EncodeManifestVerdict(sdkdelivery.ManifestVerdict{Module: "refusedmod", State: sdkdelivery.VerdictRefused,
Problems: []string{"it breaks a rule"}, Judge: "exe:1", At: time.Now()})
if _, err := b.verdicts.Put(ctx, refusedDigest, verdict); err != nil {
t.Fatal(err)
}
trunk, _ = sdkdelivery.EncodeModule(sdkdelivery.Module{FP: "src1:" + refusedFP[5:], FP2: refusedFP, Manifest: refusedDigest,
Repository: "novox/mesh-catalog", Commit: "5555555500000000", Path: "modules/refusedmod"})
if _, err := b.modules.Put(ctx, "refusedmod", trunk); err != nil {
t.Fatal(err)
}
b.built(t, "refusedmod", "1", refusedFP, refusedDigest, sdkdelivery.BuildDone)
takeNewModules(ctx, open, "refusedmod")
if _, known := inventoryHolds(t, inv, "refusedmod"); known {
t.Fatal("0578 §3: a manifest the indexer refused was registered")
}
// Said at once in the controller's journal; a condition once it waits past the bound (the watchman raises the
// indexer's manifest-refused at once).
obs := newModuleObservations(time.Now().Add(newModuleWaitBound + time.Minute))
if len(obs) != 1 || obs[0].Kind != kindNewModuleWaits || !strings.Contains(obs[0].Summary, "refused the manifest of refusedmod: it breaks a rule") {
t.Fatalf("0578 §3: the refused module is not held and named: %+v", obs)
} }
builds, _ := inv.Builds(ctx, "watchman", 5) builds, _ := inv.Builds(ctx, "watchman", 5)
for _, b := range builds { for _, b := range builds {
+13 -5
View File
@@ -9,11 +9,7 @@ import (
// ADR 0501 §6, §7 (issue 1851, A5b): the condition of a new module reads its key's take-in: one waiting for a controller // ADR 0501 §6, §7 (issue 1851, A5b): the condition of a new module reads its key's take-in: one waiting for a controller
// that reads its manifest's word is named at once, one waiting for its key only past the bound, each reason said once. // that reads its manifest's word is named at once, one waiting for its key only past the bound, each reason said once.
func TestANewModuleWaitingIsNamedByWhatItWaitsFor(t *testing.T) { func TestANewModuleWaitingIsNamedByWhatItWaitsFor(t *testing.T) {
t.Cleanup(func() { withNoNewModuleWaits(t)
newModulesWaiting.Lock()
newModulesWaiting.by = map[string]newModuleWait{}
newModulesWaiting.Unlock()
})
word := `waits: watchman's manifest uses the word "key-ttl", which this controller does not read: it is registered when a controller that reads it runs` word := `waits: watchman's manifest uses the word "key-ttl", which this controller does not read: it is registered when a controller that reads it runs`
sayNewModuleWait(newModuleWait{Module: "watchman", Commit: "c1", Why: word, Word: wordIn(word)}) sayNewModuleWait(newModuleWait{Module: "watchman", Commit: "c1", Why: word, Word: wordIn(word)})
sayNewModuleWait(newModuleWait{Module: "view", Commit: "c1", Why: "waits: the build of view at aaaa is building on ace"}) sayNewModuleWait(newModuleWait{Module: "view", Commit: "c1", Why: "waits: the build of view at aaaa is building on ace"})
@@ -31,3 +27,15 @@ func TestANewModuleWaitingIsNamedByWhatItWaitsFor(t *testing.T) {
t.Fatalf("registered modules are still named: %+v", obs) t.Fatalf("registered modules are still named: %+v", obs)
} }
} }
// withNoNewModuleWaits empties the controller's list of new modules waiting, for a test, at its start and at its end: the
// list is the process's, so a test that leaves a module in it is read by the next.
func withNoNewModuleWaits(t *testing.T) {
reset := func() {
newModulesWaiting.Lock()
newModulesWaiting.by = map[string]newModuleWait{}
newModulesWaiting.Unlock()
}
reset()
t.Cleanup(reset)
}
+1 -1
View File
@@ -11,7 +11,7 @@ import (
// paths and its manifest alone. mesh-sdk's TypeScript package at its repository's root: a change to its go/ touches no // paths and its manifest alone. mesh-sdk's TypeScript package at its repository's root: a change to its go/ touches no
// module, one to its src/ or its package.json touches it. A module with an artifact naming no paths reads its directory. // module, one to its src/ or its package.json touches it. A module with an artifact naming no paths reads its directory.
func TestAModuleNamingItsPathsIsReadFromThemAlone(t *testing.T) { func TestAModuleNamingItsPathsIsReadFromThemAlone(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(`{"module":"mesh-sdk","version":"1","build":{"artifacts":[{"name":"lib", m, err := catalogue.Decode([]byte(`{"module":"mesh-sdk","version":"1","build":{"artifacts":[{"name":"lib",
"kind":"package","language":"typescript","paths":["src","package.json","package-lock.json","tsconfig.json"]}]}, "kind":"package","language":"typescript","paths":["src","package.json","package-lock.json","tsconfig.json"]}]},
"resources":[]}`)) "resources":[]}`))
if err != nil { if err != nil {
@@ -21,7 +21,7 @@ func aMeshWithTheWordsInItsLayers(t *testing.T) (*stores, wordedMesh) {
register(t, open, catalogue.Manifest{Module: "proxy", Version: "1", Provides: catalogue.FromAnywhere("route"), register(t, open, catalogue.Manifest{Module: "proxy", Version: "1", Provides: catalogue.FromAnywhere("route"),
Receives: map[string]string{"route": "/var/lib/proxy/routes.json"}}) Receives: map[string]string{"route": "/var/lib/proxy/routes.json"}})
for _, text := range w.Modules { for _, text := range w.Modules {
m, err := catalogue.ParseManifest([]byte(strings.ReplaceAll(string(text), "IMAGE", image))) m, err := catalogue.Decode([]byte(strings.ReplaceAll(string(text), "IMAGE", image)))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -10,7 +10,10 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
) )
func TestRegistrationPublishesTheNewestTrunkListAndJudgesEveryModuleAgainstIt(t *testing.T) { // Registration publishes the newest trunk's list of predefined keys. It judges no manifest against it (ADR 0578 §3): the
// indexer's judge does, with the list it keeps, and a fallback to a key not on it, or of another type, is refused there
// (mesh-delivery-indexer judge's tests).
func TestRegistrationPublishesTheNewestTrunkList(t *testing.T) {
withoutAPublishedList(t) withoutAPublishedList(t)
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -28,22 +31,6 @@ func TestRegistrationPublishesTheNewestTrunkListAndJudgesEveryModuleAgainstIt(t
t.Fatalf("the controller judges against %v, not the list published", got) t.Fatalf("the controller judges against %v, not the list published", got)
} }
// One naming a key not on the list, or of another type, is refused and never enters the catalogue.
for i, c := range []struct{ manifest, says string }{
{notifierFallingBackTo("font-scale", "number"), "not on the list"},
{notifierFallingBackTo("font-size", "string"), "which is a number"},
} {
id := []string{"b-2", "b-3"}[i]
if _, _, err := takeIn(ctx, open.inventory, catalogueBuild(id, c.manifest, theFirstKeys, 10)); err == nil ||
!strings.Contains(err.Error(), c.says) {
t.Errorf("wanted the registration refused saying %q, got %v", c.says, err)
}
}
src, err := open.inventory.SourceOf(ctx, "notifier")
if err != nil || src.BuiltFrom != strings.Repeat("1", 12) {
t.Fatalf("a refused build replaced the registered one: %+v %v", src, err)
}
// A newer build's list, which drops agent-managed-settings, replaces the published one. // A newer build's list, which drops agent-managed-settings, replaces the published one.
smaller := `{"font-size": {"type": "number", "description": "points", "minimum": 6, "maximum": 72}}` smaller := `{"font-size": {"type": "number", "description": "points", "minimum": 6, "maximum": 72}}`
if _, _, err := takeIn(ctx, open.inventory, catalogueBuild("b-4", if _, _, err := takeIn(ctx, open.inventory, catalogueBuild("b-4",
@@ -60,7 +47,7 @@ func TestRegistrationPublishesTheNewestTrunkListAndJudgesEveryModuleAgainstIt(t
t.Fatalf("an older build's list heard later replaced the newer one: %+v", kept) t.Fatalf("an older build's list heard later replaced the newer one: %+v", kept)
} }
// A build of another repository publishes nothing, and its manifest is judged against the published list. // A build of another repository publishes nothing.
other := catalogueBuild("b-6", notifierFallingBackTo("font-size", "number"), theFirstKeys, 99) other := catalogueBuild("b-6", notifierFallingBackTo("font-size", "number"), theFirstKeys, 99)
other.Repository, other.Source.Repository = "http://forge.internal:20000/novox/notifier.git", "novox/notifier" other.Repository, other.Source.Repository = "http://forge.internal:20000/novox/notifier.git", "novox/notifier"
other.Manifest = json.RawMessage(strings.Replace(notifierFallingBackTo("font-size", "number"), `"notifier"`, other.Manifest = json.RawMessage(strings.Replace(notifierFallingBackTo("font-size", "number"), `"notifier"`,
@@ -71,9 +58,4 @@ func TestRegistrationPublishesTheNewestTrunkListAndJudgesEveryModuleAgainstIt(t
if kept, _, _ = open.inventory.PredefinedKept(ctx); kept.Build != "b-4" { if kept, _, _ = open.inventory.PredefinedKept(ctx); kept.Build != "b-4" {
t.Fatalf("a build of another repository published its list: %+v", kept) t.Fatalf("a build of another repository published its list: %+v", kept)
} }
other.ID, other.Manifest = "b-7", json.RawMessage(strings.Replace(notifierFallingBackTo("agent-managed-settings",
"number"), `"notifier"`, `"chimes"`, 1))
if _, _, err := takeIn(ctx, open.inventory, other); err == nil || !strings.Contains(err.Error(), "not on the list") {
t.Fatalf("a key only another repository's list holds was registered: %v", err)
}
} }
+2 -59
View File
@@ -1,10 +1,7 @@
package main package main
import ( import (
"bytes"
"encoding/json" "encoding/json"
"os"
"path/filepath"
"strings" "strings"
"testing" "testing"
@@ -12,8 +9,8 @@ import (
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
) )
// The list of predefined keys (novox/hq ADR 0348 §7, design 57 package K2, issue 661): `module check` and the // The list of predefined keys (novox/hq ADR 0348 §7, design 57 package K2, issue 661): the
// merge gate judge each manifest against the list at the root of its tree; registration publishes the newest // merge gate judges each manifest against the list at the root of its tree; registration publishes the newest
// trunk build's list of the catalogue and judges every module against the list the mesh publishes. // trunk build's list of the catalogue and judges every module against the list the mesh publishes.
const theFirstKeys = `{ const theFirstKeys = `{
@@ -41,60 +38,6 @@ func withoutAPublishedList(t *testing.T) {
t.Cleanup(func() { catalogue.ReplacePublished(was) }) t.Cleanup(func() { catalogue.ReplacePublished(was) })
} }
// aRepositoryTree is a repository's checkout holding one module at modules/notifier, and the list when given.
func aRepositoryTree(t *testing.T, manifest string, list *string) string {
t.Helper()
root := t.TempDir()
if err := os.Mkdir(filepath.Join(root, ".git"), 0o755); err != nil {
t.Fatal(err)
}
if list != nil {
if err := os.WriteFile(filepath.Join(root, catalogue.PredefinedFile), []byte(*list), 0o644); err != nil {
t.Fatal(err)
}
}
return aTemplatedModule(t, filepath.Join(root, "modules", "notifier"), manifest, nil)
}
func TestModuleCheckJudgesAManifestAgainstTheListAtItsRepositoryRoot(t *testing.T) {
withoutAPublishedList(t)
list := theFirstKeys
check := func(path string) (string, error) {
var out bytes.Buffer
err := moduleCheck([]string{path}, &out)
return out.String(), err
}
if out, err := check(aRepositoryTree(t, notifierFallingBackTo("font-size", "number"), &list)); err != nil ||
!strings.Contains(out, "notifier: ok") {
t.Fatalf("a module naming font-size was refused: %v\n%s", err, out)
}
for _, c := range []struct {
manifest string
list *string
says string
}{
{notifierFallingBackTo("font-scale", "number"), &list, `"font-scale", which is not on the list`},
{notifierFallingBackTo("font-size", "string"), &list, "which is a number, and this value is a string"},
{notifierFallingBackTo("font-size", "number"), nil, "is not present"},
{notifierFallingBackTo("font-size", "number"), ptr(`{"font-size": {"type": "number"}}`), "no description"},
} {
if out, err := check(aRepositoryTree(t, c.manifest, c.list)); err == nil || !strings.Contains(out, c.says) {
t.Errorf("wanted the check refused saying %q, got %v:\n%s", c.says, err, out)
}
}
}
func ptr(s string) *string { return &s }
// writeTheList puts the list at the root of a tree the gate judges.
func writeTheList(t *testing.T, tree, list string) string {
t.Helper()
if err := os.WriteFile(filepath.Join(tree, catalogue.PredefinedFile), []byte(list), 0o644); err != nil {
t.Fatal(err)
}
return tree
}
// catalogueBuild is a trunk build of the catalogue carrying the list at a generation. // catalogueBuild is a trunk build of the catalogue carrying the list at a generation.
func catalogueBuild(id, manifest, list string, generation int) link.BuildResult { func catalogueBuild(id, manifest, list string, generation int) link.BuildResult {
r := link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git", r := link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
@@ -20,7 +20,7 @@ import (
func gainedAndPlain(t *testing.T) (plain, gained catalogue.Manifest) { func gainedAndPlain(t *testing.T) (plain, gained catalogue.Manifest) {
t.Helper() t.Helper()
plain, err := catalogue.ParseManifest([]byte(`{"module":"mesh-mail","version":"1", plain, err := catalogue.Decode([]byte(`{"module":"mesh-mail","version":"1",
"own-secrets":{"broker-send":{"path":"/var/lib/mesh-mail/broker-send","owner":"mesh-mail-send"}}, "own-secrets":{"broker-send":{"path":"/var/lib/mesh-mail/broker-send","owner":"mesh-mail-send"}},
"resources":[{"id":"account","type":"user","name":"mesh-mail-send"}]}`)) "resources":[{"id":"account","type":"user","name":"mesh-mail-send"}]}`))
if err != nil { if err != nil {
@@ -21,7 +21,9 @@ import (
// account against the store — no bus to dial, busAddress "" — for mesh-mail's real manifest shape (review // account against the store — no bus to dial, busAddress "" — for mesh-mail's real manifest shape (review
// round 1 finding 1: no own secret `broker` at all, its tools served by the tool runner), and issuing it // round 1 finding 1: no own secret `broker` at all, its tools served by the tool runner), and issuing it
// again is the path every bus account takes (`rollout mint`'s KindProcess case, and a re-run of `module // again is the path every bus account takes (`rollout mint`'s KindProcess case, and a re-run of `module
// issue`): it rotates the password, and the old hash no longer matches. // issue`): it rotates the password, and the old hash no longer matches. Issue 774: every account minted is handed to the
// vault, covered or not, until the grants part's take-over; from it the issue is forwarded to the vault's issue-account,
// and the controller's hash stays as it was.
func TestModuleIssueMintsAndDeliversAProcesssAccount(t *testing.T) { func TestModuleIssueMintsAndDeliversAProcesssAccount(t *testing.T) {
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
@@ -29,7 +31,6 @@ func TestModuleIssueMintsAndDeliversAProcesssAccount(t *testing.T) {
theVaultIsInstalled(t, open) theVaultIsInstalled(t, open)
m := mailSenderWithProcess() m := mailSenderWithProcess()
m.Moved = true // so handIssuedToVault's accountIsCovered is true, and the hand-off is exercised whole
register(t, open, m) register(t, open, m)
vault := newStandInVault(t) vault := newStandInVault(t)
@@ -84,4 +85,13 @@ func TestModuleIssueMintsAndDeliversAProcesssAccount(t *testing.T) {
if hash1 == hash2 { if hash1 == hash2 {
t.Fatal("issuing the process's account again did not rotate its password") t.Fatal("issuing the process's account again did not rotate its password")
} }
theGrantsPartTookOver(t, true)
issue()
if hash3, held, err := inv.BusUserHash(ctx, user); err != nil || !held || hash3 != hash2 {
t.Fatalf("the controller minted again after the grants part's take-over (held %v, %v)", held, err)
}
if said := vault.said(); !strings.Contains(said, `"verb":"issue-account"`) || !strings.Contains(said, `"user":"`+user+`"`) {
t.Errorf("the vault was not asked to issue %s after the take-over: %s", user, said)
}
} }
+1 -1
View File
@@ -11,7 +11,7 @@ import (
// declares bus, whose own secret broker-send is where module issue delivers its account (ADR 0436 §2), // declares bus, whose own secret broker-send is where module issue delivers its account (ADR 0436 §2),
// owned by mesh-mail-send, the user resource the process declares, at mode 0600 as every own secret is. // owned by mesh-mail-send, the user resource the process declares, at mode 0600 as every own secret is.
func mailSenderWithProcess() catalogue.Manifest { func mailSenderWithProcess() catalogue.Manifest {
m, err := catalogue.ParseManifest([]byte(`{"module":"mesh-mail","version":"1","emits":["mail.send.sent"], m, err := catalogue.Decode([]byte(`{"module":"mesh-mail","version":"1","emits":["mail.send.sent"],
"own-secrets":{"broker-send":{"path":"/var/lib/mesh-mail/broker-send","owner":"mesh-mail-send"}}, "own-secrets":{"broker-send":{"path":"/var/lib/mesh-mail/broker-send","owner":"mesh-mail-send"}},
"resources":[{"id":"account","type":"user","name":"mesh-mail-send"}, "resources":[{"id":"account","type":"user","name":"mesh-mail-send"},
{"id":"send","type":"process","run":["./mesh-mail","send"],"user":"mesh-mail-send", {"id":"send","type":"process","run":["./mesh-mail","send"],"user":"mesh-mail-send",
+1 -1
View File
@@ -23,7 +23,7 @@ import (
// invoked seat verb), so the account module issue delivers for it can be put through every grant at once. // invoked seat verb), so the account module issue delivers for it can be put through every grant at once.
func mailSenderLiveManifest(t *testing.T) catalogue.Manifest { func mailSenderLiveManifest(t *testing.T) catalogue.Manifest {
t.Helper() t.Helper()
m, err := catalogue.ParseManifest([]byte(`{"module":"mesh-mail","version":"1", m, err := catalogue.Decode([]byte(`{"module":"mesh-mail","version":"1",
"emits":["mail.send.sent","mail.send.failed"],"state":["sends"], "emits":["mail.send.sent","mail.send.failed"],"state":["sends"],
"streams":[{"name":"mail-sends","events":["mail.send.approved"],"max-age-days":1,"max-mib":8, "streams":[{"name":"mail-sends","events":["mail.send.approved"],"max-age-days":1,"max-mib":8,
"retention":"work-queue","max-deliveries":5}], "retention":"work-queue","max-deliveries":5}],
@@ -27,9 +27,10 @@ func TestTheProvisionerClientsAccountIsIssuedForItsServiceAndNeverInvented(t *te
Username: "laptop.mesh-provisioner", Kind: inventory.BusModule, Node: "laptop", Module: "mesh-provisioner"}); err != nil { Username: "laptop.mesh-provisioner", Kind: inventory.BusModule, Node: "laptop", Module: "mesh-provisioner"}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// The composition refuses it in the send's words (mesh-sdk's go/compose: TestTheSendsRefusalsAreRefusedInTheSendsWords). // The composition holds the module out in the send's words, and composes the rest of the machine.
if _, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false); err == nil || !strings.Contains(err.Error(), "provisioner.laptop") { if held, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false); err != nil ||
t.Fatalf("a push would seal a placeholder for the client's account before it was issued: %v", err) !strings.Contains(held.leftOutWhy[m.Module], "provisioner.laptop") {
t.Fatalf("a push would seal a placeholder for the client's account before it was issued: %v %v", err, held.leftOutWhy)
} }
if err := issueProvisionerClient(ctx, inv, m, "laptop", broker.Broker{Fingerprint: "ab"}, "10.0.0.1:4222"); err != nil { if err := issueProvisionerClient(ctx, inv, m, "laptop", broker.Broker{Fingerprint: "ab"}, "10.0.0.1:4222"); err != nil {
@@ -45,8 +46,9 @@ func TestTheProvisionerClientsAccountIsIssuedForItsServiceAndNeverInvented(t *te
if !slices.ContainsFunc(states, func(s inventory.SecretState) bool { return s.Name == broker.ProvisionerSecret }) { if !slices.ContainsFunc(states, func(s inventory.SecretState) bool { return s.Name == broker.ProvisionerSecret }) {
t.Fatalf("the client's account is not sealed to the machine for its service: %+v", states) t.Fatalf("the client's account is not sealed to the machine for its service: %+v", states)
} }
if _, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false); err != nil && strings.Contains(err.Error(), "provisioner.laptop") { if held, _, err := composeQuestion(ctx, open, "laptop", compose.Foreseeing, false); err != nil ||
t.Fatalf("an issued client account is refused: %v", err) slices.Contains(held.heldForCredential, m.Module) {
t.Fatalf("an issued client account is held: %v %v", err, held.leftOutWhy)
} }
if !runsTheProvisionerClient(m) { if !runsTheProvisionerClient(m) {
t.Fatal("the module is not read as running the client") t.Fatal("the module is not read as running the client")
@@ -207,8 +207,13 @@ func giveVerifiedSendersBack(ctx context.Context, d *doctor) {
} }
// reissueVerifiedSendersOn is reissueVerifiedSenders on the bus: memberships read from the AssignmentsStream and // reissueVerifiedSendersOn is reissueVerifiedSenders on the bus: memberships read from the AssignmentsStream and
// published to it. // published to it. Once the stream is gone (the bus module's tools delete it at the lease write, research 072 row 33)
// the memberships are the grants part's alone, and there is nothing here to issue again.
func reissueVerifiedSendersOn(ctx context.Context, bus link.OverNATS, records broker.Records, where broker.Placements) int { func reissueVerifiedSendersOn(ctx context.Context, bus link.OverNATS, records broker.Records, where broker.Placements) int {
if _, err := bus.JS.StreamInfo(broker.AssignmentsStream, nats.Context(ctx)); errors.Is(err, nats.ErrStreamNotFound) {
fmt.Printf(" the assignments stream is gone: the grants part issues the memberships, none is given a verified sender here\n")
return 0
}
return reissueVerifiedSenders(records, where, return reissueVerifiedSenders(records, where,
func(node, module string) ([]byte, uint64, error) { func(node, module string) ([]byte, uint64, error) {
m, err := bus.JS.GetLastMsg(broker.AssignmentsStream, broker.MembershipSubject(node, module)) m, err := bus.JS.GetLastMsg(broker.AssignmentsStream, broker.MembershipSubject(node, module))
@@ -6,6 +6,8 @@ import (
"encoding/json" "encoding/json"
"testing" "testing"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker" "github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link" "github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus" "github.com/novox/mesh-controller/internal/testbus"
@@ -17,7 +19,10 @@ func TestNatsAMembershipIssuedWithoutItsVerifiedSenderIsIssuedAgainWhenTheJudgem
t.Fatal(err) t.Fatal(err)
} }
t.Cleanup(js.Close) t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil { // The assignments stream is no longer the mesh's (research 072 row 33): it is asserted by nothing, and stays on a
// bus only until the lease write deletes it. Made here as it is left there.
if _, err := js.Context().AddStream(&nats.StreamConfig{Name: broker.AssignmentsStream,
Subjects: []string{"mesh.assignment.*.*"}, MaxMsgsPerSubject: 1, AllowDirect: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
bus := link.OverNATS{Conn: js.Conn(), JS: js.Context()} bus := link.OverNATS{Conn: js.Conn(), JS: js.Context()}
+1 -1
View File
@@ -33,7 +33,7 @@ func TestReplay239AModuleNameIsNotTakenOverByAnotherRepository(t *testing.T) {
register := func(raw, repository, path string) { register := func(raw, repository, path string) {
t.Helper() t.Helper()
m, err := catalogue.ParseManifest([]byte(raw)) m, err := catalogue.Decode([]byte(raw))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
+3
View File
@@ -73,6 +73,9 @@ type sendable struct {
LeftOut []string LeftOut []string
// leftOutWhy is why each was, for push and plan to say; never on the wire. // leftOutWhy is why each was, for push and plan to say; never on the wire.
leftOutWhy map[string]string leftOutWhy map[string]string
// heldForCredential are the modules held out because their bus account was issued by nobody
// (holdBackUnissuedBusAccounts), sorted; never on the wire.
heldForCredential []string
// withheld is every consumer this machine's grants leave out, because its identity overflows the // withheld is every consumer this machine's grants leave out, because its identity overflows the
// provision's bound (novox/hq ADR 0225); for push and plan to say, never on the wire. // provision's bound (novox/hq ADR 0225); for push and plan to say, never on the wire.
withheld []catalogue.Overflow withheld []catalogue.Overflow
+1 -1
View File
@@ -14,7 +14,7 @@ import (
func TestAProviderElsewhereServesItsOwnComposedName(t *testing.T) { func TestAProviderElsewhereServesItsOwnComposedName(t *testing.T) {
open := aMesh(t) open := aMesh(t)
ctx := t.Context() ctx := t.Context()
forge, err := catalogue.ParseManifest([]byte(`{"module":"forge","version":"1","requires":["route"], forge, err := catalogue.Decode([]byte(`{"module":"forge","version":"1","requires":["route"],
"provides":[{"name":"npm-package-registry","scope":"mesh"}], "provides":[{"name":"npm-package-registry","scope":"mesh"}],
"contributes":{"route":{"web":{"label":"git","port":3000}}}, "contributes":{"route":{"web":{"label":"git","port":3000}}},
"serves":{"npm-package-registry":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/", "serves":{"npm-package-registry":{"scheme":"http","port":3000,"npm-path":"/api/packages/novox/npm/",
+1 -28
View File
@@ -1,14 +1,10 @@
package main package main
import ( import (
"bytes"
"crypto/ecdh" "crypto/ecdh"
"crypto/rand" "crypto/rand"
"encoding/base64" "encoding/base64"
"encoding/json" "encoding/json"
"os"
"path/filepath"
"strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
@@ -41,7 +37,7 @@ func TestTheProvidersMachineReceivesTheSharesOfAnS3BucketRequirement(t *testing.
if err := json.Unmarshal(raw, &overlayModule); err != nil { if err := json.Unmarshal(raw, &overlayModule); err != nil {
t.Fatal(err) t.Fatal(err)
} }
consumer, err := catalogue.ParseManifest([]byte(filesRequiringShares)) consumer, err := catalogue.Decode([]byte(filesRequiringShares))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -79,26 +75,3 @@ func TestTheProvidersMachineReceivesTheSharesOfAnS3BucketRequirement(t *testing.
t.Fatalf("the provider's grant for the consumer carries shares %s, want %s", got, want) t.Fatalf("the provider's grant for the consumer carries shares %s, want %s", got, want)
} }
} }
// 1593/AC2 unit: the module check accepts a manifest with shares and refuses an unknown method, citing ADR 0468 §8.
func TestModuleCheckAcceptsSharesAndRefusesAnUnknownMethod(t *testing.T) {
dir := t.TempDir()
write := func(name, body string) string {
p := filepath.Join(dir, name+".json")
if err := os.WriteFile(p, []byte(body), 0o600); err != nil {
t.Fatal(err)
}
return p
}
var out bytes.Buffer
if err := moduleCheckFor([]string{write("files", filesRequiringShares)}, 6, &out); err != nil {
t.Fatalf("a manifest with shares was refused: %v\n%s", err, out.String())
}
out.Reset()
bad := write("bad", `{"module":"bad","version":"1","requires":[
{"name":"s3-bucket","shares":[{"to":"seat:node-files","methods":["delete"]}]}]}`)
err := moduleCheckFor([]string{bad}, 6, &out)
if err == nil || !strings.Contains(out.String(), "ADR 0468 §8") {
t.Fatalf("an unknown method passed or the refusal does not cite the ADR: %v\n%s", err, out.String())
}
}
+1 -1
View File
@@ -318,7 +318,7 @@ func printStatus(asked answers) error {
fmt.Printf(" %-12s %-20s %-22s %q is %d, %s keeps %d\n", o.Consumer, o.Module, o.Provision, fmt.Printf(" %-12s %-20s %-22s %q is %d, %s keeps %d\n", o.Consumer, o.Module, o.Provision,
o.Identity, len(o.Identity), o.Bound.In, o.Bound.Max) o.Identity, len(o.Identity), o.Bound.In, o.Bound.Max)
} }
fmt.Printf("\n a shorter `slug` in the module's definition fits it; `module check` refuses one before merge\n\n") fmt.Printf("\n a shorter `slug` in the module's definition fits it\n\n")
} }
// Repairs done by hand this week (novox/hq to-be 45 §7). Not a fault, so it does not break "all // Repairs done by hand this week (novox/hq to-be 45 §7). Not a fault, so it does not break "all
-123
View File
@@ -1,123 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
// A file's text kept as a template file in its module's repository (novox/hq ADR 0349 §2, design 57 package T1,
// issue 654): `module check` and the merge gate read it from the tree beside the manifest, so a change to a
// template alone is judged as a change to its module.
const templatedNotifier = `{"module":"notifier","version":"1",
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","template":"files/notifier.conf"}]}`
// inlineNotifier is the same module as the mesh holds it: built, its text inline.
const inlineNotifier = `{"module":"notifier","version":"1",
"resources":[{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","content":"volume = 3\n"}]}`
// aTemplatedModule writes a module's directory: its manifest and its files/.
func aTemplatedModule(t *testing.T, dir, manifest string, files map[string]string) string {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "module.json")
if err := os.WriteFile(path, []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
for name, text := range files {
full := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
return path
}
func TestModuleCheckReadsTemplatesFromTheTreeBesideTheManifest(t *testing.T) {
good := aTemplatedModule(t, t.TempDir(), templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil || !strings.Contains(out.String(), "notifier: ok") {
t.Fatalf("a module with its text in files/ was refused: %v\n%s", err, out.String())
}
// A change to the template alone, to text the controller refuses inline, is refused the same.
bad := aTemplatedModule(t, t.TempDir(), templatedNotifier,
map[string]string{"files/notifier.conf": "volume = 3\nstore = ${dir:nowhere}\n"})
out.Reset()
if err := moduleCheck([]string{bad}, &out); err == nil || !strings.Contains(out.String(), "nowhere") {
t.Fatalf("a template whose text is refused inline passed:\n%s", out.String())
}
// A symlink under files/ that no template names: the build's packing refuses it, so the check does too.
linked := t.TempDir()
aTemplatedModule(t, linked, templatedNotifier, map[string]string{"files/notifier.conf": "volume = 3\n"})
if err := os.Symlink("/etc/hostname", filepath.Join(linked, "files", "elsewhere")); err != nil {
t.Fatal(err)
}
out.Reset()
if err := moduleCheck([]string{filepath.Join(linked, "module.json")}, &out); err == nil ||
!strings.Contains(out.String(), "files/elsewhere is a symlink") {
t.Fatalf("a symlink under files/ passed the check:\n%s", out.String())
}
// And a template the tree lacks, by name.
missing := aTemplatedModule(t, t.TempDir(), templatedNotifier, nil)
out.Reset()
if err := moduleCheck([]string{missing}, &out); err == nil || !strings.Contains(out.String(), "files/notifier.conf") {
t.Fatalf("a missing template was not refused by name:\n%s", out.String())
}
}
// templatedMesh is catalogueMesh with the notifier on the laptop, registered as built: its text inline.
// What the gate composes for the module in its template form is what the mesh composes for it inline.
// A module whose files/ is unpacked onto the machine names its own templates directory (issue 731): `module check`
// reads its templates from there and says of it exactly what it says of the same module with its text inline.
func TestModuleCheckReadsTemplatesFromTheTemplatesDirectoryTheManifestNames(t *testing.T) {
const inline = `{"module":"notifier","version":"1",
"build":{"artifacts":[{"name":"scripts","kind":"archive","from":"files"}]},
"resources":[{"id":"scripts","type":"archive","path":"/usr/local/lib/notifier","artifact":"scripts"},
{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","content":"volume = 3\n"}]}`
const templated = `{"module":"notifier","version":"1","templates":"templates",
"build":{"artifacts":[{"name":"scripts","kind":"archive","from":"files"},
{"name":"templates","kind":"archive","from":"templates"}]},
"resources":[{"id":"scripts","type":"archive","path":"/usr/local/lib/notifier","artifact":"scripts"},
{"id":"conf","type":"file","path":"/etc/notifier/notifier.conf","mode":"0644","template":"templates/notifier.conf"}]}`
scripts := map[string]string{"files/bin/notify": "#!/bin/sh\n"}
var want bytes.Buffer
if err := moduleCheck([]string{aTemplatedModule(t, t.TempDir(), inline, scripts)}, &want); err != nil {
t.Fatalf("the inline module was refused: %v\n%s", err, want.String())
}
with := map[string]string{"files/bin/notify": "#!/bin/sh\n", "templates/notifier.conf": "volume = 3\n"}
var got bytes.Buffer
if err := moduleCheck([]string{aTemplatedModule(t, t.TempDir(), templated, with)}, &got); err != nil {
t.Fatalf("a module with its text in templates/ was refused: %v\n%s", err, got.String())
}
if got.String() != want.String() {
t.Fatalf("module check says otherwise of the templated module:\ninline: %s\ntemplated: %s", want.String(),
got.String())
}
// Its text in files/ instead, where the machine unpacks it, is not where its manifest says its templates are.
misplaced := strings.Replace(templated, `"templates/notifier.conf"`, `"files/notifier.conf"`, 1)
got.Reset()
if err := moduleCheck([]string{aTemplatedModule(t, t.TempDir(), misplaced,
map[string]string{"files/notifier.conf": "volume = 3\n"})}, &got); err == nil ||
!strings.Contains(got.String(), "files/notifier.conf") {
t.Fatalf("a template outside the templates directory the manifest names passed:\n%s", got.String())
}
}
// Issue 876: the gate judges a moved module that names its templates as registration reads it after a real build,
// handed its templates; it refused every such module as "marked moved and names no template". This calls the gate's
// own shelf (shelfWithChange), which needs no database.
+3 -2
View File
@@ -20,11 +20,12 @@ const (
) )
// unknownFieldObservations is one condition for each module of the catalogue whose stored manifest has // unknownFieldObservations is one condition for each module of the catalogue whose stored manifest has
// a key this controller does not know. // a key this controller does not know and read no further than its name and version (ADR 0578 §6: a manifest read past
// the key is registered and the key named, so it raises nothing).
func unknownFieldObservations(known map[string]catalogue.Manifest) []conditions.Observation { func unknownFieldObservations(known map[string]catalogue.Manifest) []conditions.Observation {
names := make([]string, 0, len(known)) names := make([]string, 0, len(known))
for name, m := range known { for name, m := range known {
if m.UnknownField() != "" { if m.ReadNoFurther() {
names = append(names, name) names = append(names, name)
} }
} }
+7 -3
View File
@@ -11,14 +11,18 @@ import (
// A module whose stored manifest has a key this controller does not know is a condition, in plain // A module whose stored manifest has a key this controller does not know is a condition, in plain
// words, until it is read again; every other module raises nothing (novox/hq ADR 0262). // words, until it is read again; every other module raises nothing (novox/hq ADR 0262).
func TestAModuleWithAnUnknownFieldIsACondition(t *testing.T) { func TestAModuleWithAnUnknownFieldIsACondition(t *testing.T) {
var later, now catalogue.Manifest var later, now, aside catalogue.Manifest
if err := json.Unmarshal([]byte(`{"module": "dunst", "version": "2", "settings": {}, "a-field-from-later": 1}`), &later); err != nil { if err := json.Unmarshal([]byte(`{"module": "dunst", "version": "2", "tools": "not-a-list", "a-field-from-later": 1}`), &later); err != nil {
t.Fatal(err)
}
// A field set aside with the rest read raises nothing (ADR 0578 §6): the module is registered and the field named.
if err := json.Unmarshal([]byte(`{"module": "sway", "version": "2", "settings": {}, "a-field-from-later": 1}`), &aside); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if err := json.Unmarshal([]byte(`{"module": "xorg", "version": "1"}`), &now); err != nil { if err := json.Unmarshal([]byte(`{"module": "xorg", "version": "1"}`), &now); err != nil {
t.Fatal(err) t.Fatal(err)
} }
observed := unknownFieldObservations(map[string]catalogue.Manifest{"dunst": later, "xorg": now}) observed := unknownFieldObservations(map[string]catalogue.Manifest{"dunst": later, "xorg": now, "sway": aside})
if len(observed) != 1 || observed[0].ID != "dunst" || observed[0].Kind != kindUnknownField { if len(observed) != 1 || observed[0].ID != "dunst" || observed[0].Kind != kindUnknownField {
t.Fatalf("observed: %+v", observed) t.Fatalf("observed: %+v", observed)
} }
@@ -49,6 +49,12 @@ func (v *standInVault) ask(verb string, args map[string]any) (json.RawMessage, e
// The vault asks the desk's prompt and the provider's tool itself, in its own name (issue 723): one step from // The vault asks the desk's prompt and the provider's tool itself, in its own name (issue 723): one step from
// the controller, which carries no key and no sealed value. // the controller, which carries no key and no sealed value.
switch verb { switch verb {
case "issue-account":
// The vault is the one issuer once installed (issue 774): the controller forwards, and keeps nothing.
if v.refuse != nil {
return nil, v.refuse
}
return json.Marshal(map[string]any{"minted": true, "account": map[string]any{"user": args["user"], "generation": 1}})
case "secret-ask": case "secret-ask":
if v.refuse != nil { if v.refuse != nil {
return nil, v.refuse return nil, v.refuse
@@ -1,46 +0,0 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// ADR 0326 §3 is judged where a definition is written (novox/hq issue 574): `module check`, and
// namesNoInstallation, which `module add`, a build taken in and the merge gate's registration run.
func TestWhereADefinitionIsWrittenADeclaringModuleNamesItsFilesKeys(t *testing.T) {
const silent = `{"module":"mail-client","version":"1",
"settings":{"prefs":{"kind":"preference","default":"// none","why":"its own defaults until a machine names one"}},
"resources":[
{"id":"user-js","type":"file","path":"/var/lib/mail-client/user.js","mode":"0644","content":"${setting:prefs}\n"},
{"id":"accounts","type":"file","path":"/var/lib/mail-client/accounts.json","mode":"0600","merge":"json",
"content":"{}\n"%s}]}`
dir := t.TempDir()
bad := filepath.Join(dir, "bad.json")
good := filepath.Join(dir, "good.json")
os.WriteFile(bad, []byte(strings.Replace(silent, "%s", "", 1)), 0o600)
os.WriteFile(good, []byte(strings.Replace(silent, "%s", `,"takes-settings":["accounts"]`, 1)), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{bad}, &out); err == nil ||
!strings.Contains(out.String(), "mergeable accounts names no takes-settings") {
t.Fatalf("module check passed a declaring module's silent mergeable file: %v\n%s", err, out.String())
}
out.Reset()
if err := moduleCheck([]string{good}, &out); err != nil {
t.Fatalf("the file naming its keys was refused: %v\n%s", err, out.String())
}
raw, _ := os.ReadFile(bad)
m, err := catalogue.ParseManifest(raw)
if err != nil {
t.Fatalf("reading it is not where it is judged: %v", err)
}
if err := namesNoInstallation(m); err == nil || !strings.Contains(err.Error(), "names no takes-settings") {
t.Fatalf("registration passed it: %v", err)
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module github.com/novox/mesh-controller
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915
github.com/jackc/pgx/v5 v5.10.0 github.com/jackc/pgx/v5 v5.10.0
github.com/nats-io/nats-server/v2 v2.11.17 github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
+2 -2
View File
@@ -1,7 +1,7 @@
git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5 h1:E5stfIeCN6uYgA9TCv+4K8JgaURg3zQppvUgJLaDMP4= git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5 h1:E5stfIeCN6uYgA9TCv+4K8JgaURg3zQppvUgJLaDMP4=
git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5/go.mod h1:4wyoCr6IIIIarEUuROThWDj7DAvJvIidqMjWcpSnOwE= git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5/go.mod h1:4wyoCr6IIIIarEUuROThWDj7DAvJvIidqMjWcpSnOwE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 h1:lKU3HjNXGiVbLW2hUgFjU/TMTUYMlNnnmZxG5T7biQA= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915 h1:x8k7IeMirMTxUo8BH+xUO+MDnHYgU+636jl1NIw1wvs=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+1 -52
View File
@@ -105,11 +105,7 @@ func theCataloguesEvents(t *testing.T) ([]AnEmitter, []AConsumer, []DeclaredSeat
emitters = append(emitters, *emitter) emitters = append(emitters, *emitter)
} }
if consumer != nil { if consumer != nil {
kept, setAside := setAsideRetiredSeats(*consumer) consumers = append(consumers, *consumer)
for _, note := range setAside {
t.Log(note)
}
consumers = append(consumers, kept)
} }
seats = append(seats, declared...) seats = append(seats, declared...)
} }
@@ -119,53 +115,6 @@ func theCataloguesEvents(t *testing.T) ([]AnEmitter, []AConsumer, []DeclaredSeat
return emitters, consumers, seats return emitters, consumers, seats
} }
// setAsideRetiredSeats takes out of a consumer the events of a seat the mesh has retired, and says each one.
// The check judges against the catalogue's trunk (ADR 0484 clause 4), and a group that retires a seat while
// its catalogue pull request drops the consume (ADR 0524 section 5) meets the trunk still holding it: against
// that catalogue the seat is gone and the subscription falls back to a module subject nobody owns. It is set
// aside and logged, never silently dropped; an unknown seat that is not retired is still judged.
func setAsideRetiredSeats(c AConsumer) (AConsumer, []string) {
kept := AConsumer{Module: c.Module}
var notes []string
for _, want := range c.Consumes {
seat, event, named := strings.Cut(want, ".")
if replacement, retired := catalogue.RetiredSeats[seat]; named && retired {
notes = append(notes, c.Module+" still consumes "+seat+"."+event+"; the seat is retired (its work is "+
replacement+"); the catalogue drops it in the same change (ADR 0524 §5), and the check judges against the catalogue's trunk (ADR 0484 clause 4)")
continue
}
kept.Consumes = append(kept.Consumes, want)
}
return kept, notes
}
func TestARetiredSeatsConsumeIsSetAsideAndAnUnknownOneIsNot(t *testing.T) {
var retired string
for name := range catalogue.RetiredSeats {
retired = name
break
}
if retired == "" {
t.Skip("no seat is retired")
}
kept, notes := setAsideRetiredSeats(AConsumer{Module: "m",
Consumes: []string{retired + ".built", "no-such-seat.built", "plain"}})
if len(notes) != 1 || !strings.Contains(notes[0], retired+".built") {
t.Fatalf("the retired seat's consume was not set aside and said: %v", notes)
}
if strings.Join(kept.Consumes, ",") != "no-such-seat.built,plain" {
t.Fatalf("what stays to be judged is wrong: %v", kept.Consumes)
}
if bad := Disagreements([]AnEmitter{{Module: "other", Emits: []string{"x"}}},
[]AConsumer{kept}, nil); len(bad) != 0 {
t.Fatalf("unexpected: %v", bad)
}
if bad := Disagreements([]AnEmitter{{Module: "no-such-seat", Emits: []string{"x"}}},
[]AConsumer{kept}, nil); len(bad) != 1 {
t.Fatalf("an unknown, non-retired consume no longer fails: %v", bad)
}
}
// **Do the derived subjects meet, not just the names?** // **Do the derived subjects meet, not just the names?**
// //
// The check above compares what a consumer asks for against what an emitter says it emits, by name. It // The check above compares what a consumer asks for against what an emitter says it emits, by name. It
+1 -1
View File
@@ -108,7 +108,7 @@ func (s ModuleStream) Stream() Stream {
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue // QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration // streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
// to be the only one (catalogue.WorkQueueStreamProblems). // to be the only one (the indexer's judge).
type QueueOwner struct { type QueueOwner struct {
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR // Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
// 0436 §2). // 0436 §2).
+7 -1
View File
@@ -21,7 +21,13 @@ func TestAModuleStreamOnARealBus(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
defer js.Close() defer js.Close()
for _, s := range []Stream{MeshStreams()[3], mailStream.Stream()} { var events Stream
for _, s := range MeshStreams() {
if s.Name == EventsStream {
events = s
}
}
for _, s := range []Stream{events, mailStream.Stream()} {
_ = js.js.DeleteStream(s.Name) _ = js.js.DeleteStream(s.Name)
if err := js.EnsureStream(s); err != nil { if err := js.EnsureStream(s); err != nil {
t.Fatalf("%s: %v", s.Name, err) t.Fatalf("%s: %v", s.Name, err)
@@ -14,7 +14,9 @@ import (
// queues are left on the bus until a cleanup removes them, and no module may take their names meanwhile. So the catalogue // queues are left on the bus until a cleanup removes them, and no module may take their names meanwhile. So the catalogue
// reserves every name of the broker's own, and may reserve these retired ones beside them. // reserves every name of the broker's own, and may reserve these retired ones beside them.
func TestTheCataloguesReservedStreamNamesAreTheMeshsOwn(t *testing.T) { func TestTheCataloguesReservedStreamNamesAreTheMeshsOwn(t *testing.T) {
retired := []string{"CHECKS_OTHER", "CHECKS_READY"} // And ASSIGNMENTS (research 072 row 33): the bus module's tools delete it at the lease write, and the controller
// asserts it no more, so it is never made again.
retired := []string{"CHECKS_OTHER", "CHECKS_READY", "ASSIGNMENTS"}
var own []string var own []string
for _, s := range MeshStreams() { for _, s := range MeshStreams() {
own = append(own, s.Name) own = append(own, s.Name)
+3 -8
View File
@@ -171,14 +171,9 @@ func MeshStreams() []Stream {
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " + Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
"n-1 by construction (issue 107)", "n-1 by construction (issue 107)",
}, },
{ // The assignments stream (ADR 0160) is no longer the mesh's: a runtime reads its membership from the grants
Name: AssignmentsStream, // part's state memberships (ADR 0571 clause 7), and the bus module's tools delete the stream at the lease write
Subjects: []string{"mesh.assignment.*.*"}, // (research 072 row 33), so it is not declared here, which would make it again at the controller's next start.
Retention: RetentionLastPerSubject,
Direct: true,
Why: "one membership per assignment, always the newest: what the mesh issued this module " +
"on this machine to serve and to reach (ADR 0160); read directly by the runtime it is for",
},
{ {
Name: EventsStream, Name: EventsStream,
// A seat's own events ride here too: they are 1:many like any event, and the // A seat's own events ride here too: they are 1:many like any event, and the
+2 -1
View File
@@ -126,7 +126,8 @@ func TestEachStreamCarriesTheRetentionItsShapeNeeds(t *testing.T) {
"CONTROL": RetentionWorkQueue, "CONTROL": RetentionWorkQueue,
"NODES": RetentionLastPerSubject, "NODES": RetentionLastPerSubject,
"EVENTS": RetentionLimits, "EVENTS": RetentionLimits,
"ASSIGNMENTS": RetentionLastPerSubject, // No ASSIGNMENTS: the bus module's tools delete it at the lease write and nothing makes it again (research 072
// row 33); a runtime reads its membership from the grants part's state.
// What a consumer gave up on (novox/hq issue 330): the server's notice taken once, the message // What a consumer gave up on (novox/hq issue 330): the server's notice taken once, the message
// kept until somebody acts. // kept until somebody acts.
"DEAD_LETTER_NOTICES": RetentionWorkQueue, "DEAD_LETTER_NOTICES": RetentionWorkQueue,
+25
View File
@@ -255,6 +255,31 @@ func (r Reader) Manifest(ctx context.Context, digest string) (delivery.Manifest,
return m, nil return m, nil
} }
// Verdict is the indexer's verdict on the manifest of digest (ADR 0578 §3), or nil when it holds none (no bucket yet,
// no key): a reader takes that as accepted (delivery.TakeManifestVerdict).
func (r Reader) Verdict(ctx context.Context, digest string) (*delivery.ManifestVerdict, error) {
key, err := delivery.VerdictsKey(digest)
if err != nil {
return nil, err
}
kv, err := r.bucket(ctx, delivery.VerdictsBucket)
if err != nil || kv == nil {
return nil, err
}
e, err := kv.Get(ctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("the verdict on %s: %w", short(digest), err)
}
v, err := delivery.DecodeManifestVerdict(e.Value())
if err != nil {
return nil, fmt.Errorf("the verdict on %s is not read: %w", short(digest), err)
}
return &v, nil
}
// Watch calls wake with the module of every key the builder or the indexer writes from now on, until ctx ends: what // Watch calls wake with the module of every key the builder or the indexer writes from now on, until ctx ends: what
// wakes a plan waiting on a key, with no timer (ADR 0501; the R4e context's A7). A bucket not there yet is watched once // wakes a plan waiting on a key, with no timer (ADR 0501; the R4e context's A7). A bucket not there yet is watched once
// it is: the watch looks again each time wake would have been called for the other bucket, and at every retry. // it is: the watch looks again each time wake would have been called for the other bucket, and at every retry.
+1 -1
View File
@@ -60,7 +60,7 @@ func TestTheAsksLabBus(t *testing.T) {
} }
var manifests []catalogue.Manifest var manifests []catalogue.Manifest
read := func(raw []byte, from string) { read := func(raw []byte, from string) {
m, err := catalogue.ParseManifest(raw) m, err := catalogue.Decode(raw)
if err != nil { if err != nil {
t.Fatalf("%s: %v", from, err) t.Fatalf("%s: %v", from, err)
} }
+1 -1
View File
@@ -476,7 +476,7 @@ func streamsOf(m catalogue.Manifest) []broker.ModuleStream {
// queueOwnersOf is every work-queue stream of a module's own that one of its processes names in bus.queues, // queueOwnersOf is every work-queue stream of a module's own that one of its processes names in bus.queues,
// with that process's id and the stream's declared max-deliveries (novox/hq ADR 0436 §1, §2, §6). In // with that process's id and the stream's declared max-deliveries (novox/hq ADR 0436 §1, §2, §6). In
// resource order, so composition is repeatable; a queue naming a stream the module does not own, or one two // resource order, so composition is repeatable; a queue naming a stream the module does not own, or one two
// processes name, is caught at registration (catalogue.WorkQueueStreamProblems) and never reaches here. // processes name, is caught at registration (the indexer's judge) and never reaches here.
func queueOwnersOf(m catalogue.Manifest) []broker.QueueOwner { func queueOwnersOf(m catalogue.Manifest) []broker.QueueOwner {
buses := m.ProcessBuses() buses := m.ProcessBuses()
if len(buses) == 0 { if len(buses) == 0 {
+1 -1
View File
@@ -15,7 +15,7 @@ import (
// configured is a module in the config form: one value a mergeable file takes, and a directory to place. // configured is a module in the config form: one value a mergeable file takes, and a directory to place.
func configured(t *testing.T, name string) catalogue.Manifest { func configured(t *testing.T, name string) catalogue.Manifest {
t.Helper() t.Helper()
m, err := catalogue.ParseManifest([]byte(`{"module": "` + name + `", "version": "1", m, err := catalogue.Decode([]byte(`{"module": "` + name + `", "version": "1",
"config": {"level": {"type": "string", "description": "how much it says", "default": "info"}}, "config": {"level": {"type": "string", "description": "how much it says", "default": "info"}},
"resources": [{"id": "data", "type": "directory", "mode": "0750"}, {"id": "conf", "type": "file", "path": "/etc/` + name + `.json", "mode": "0640", "resources": [{"id": "data", "type": "directory", "mode": "0750"}, {"id": "conf", "type": "file", "path": "/etc/` + name + `.json", "mode": "0640",
"content": "{\"level\": \"info\"}", "merge": "json", "takes-config": ["level"]}]}`)) "content": "{\"level\": \"info\"}", "merge": "json", "takes-config": ["level"]}]}`))
+2 -2
View File
@@ -44,12 +44,12 @@ func TestOnlyTheServingControllerMayAnswerRootFree(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
controller, err := catalogue.ParseManifest(raw) controller, err := catalogue.Decode(raw)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
parse := func(s string) catalogue.Manifest { parse := func(s string) catalogue.Manifest {
m, err := catalogue.ParseManifest([]byte(s)) m, err := catalogue.Decode([]byte(s))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -47,13 +47,10 @@ func TestAStoredBuildWithASilentMergeableFileIsStillReadAndComposed(t *testing.T
if err != nil || len(raw) == 0 { if err != nil || len(raw) == 0 {
t.Fatalf("the stored build is not read again: %d bytes, %v", len(raw), err) t.Fatalf("the stored build is not read again: %d bytes, %v", len(raw), err)
} }
m, err := catalogue.ParseManifest(raw) m, err := catalogue.Decode(raw)
if err != nil { if err != nil {
t.Fatalf("the stored manifest is not read again: %v", err) t.Fatalf("the stored manifest is not read again: %v", err)
} }
if written := catalogue.WrittenSettingsProblems(m); len(written) != 1 {
t.Fatalf("the rule this build predates should name its accounts file where it is written: %v", written)
}
// And composed, in today's form: the accounts file still takes its layer's keys. // And composed, in today's form: the accounts file still takes its layer's keys.
with := catalogue.Rendering{Settings: catalogue.SettingsBy{"mail-client": {{From: "workstation", with := catalogue.Rendering{Settings: catalogue.SettingsBy{"mail-client": {{From: "workstation",
+1 -1
View File
@@ -22,7 +22,7 @@ var adr0367Families = []string{`"mesh-knowledge.reviewed"`, `"mesh-delivery.tran
func composeWaits(t *testing.T, families []string, people map[string][]string) (granted, issued []string) { func composeWaits(t *testing.T, families []string, people map[string][]string) (granted, issued []string) {
t.Helper() t.Helper()
parse := func(s string) catalogue.Manifest { parse := func(s string) catalogue.Manifest {
m, err := catalogue.ParseManifest([]byte(s)) m, err := catalogue.Decode([]byte(s))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -18,7 +18,7 @@ func TestTheControllerIsAProcessAndNoContainer(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err := ParseManifest(raw) m, err := Decode(raw)
if err != nil { if err != nil {
t.Fatalf("the controller's own manifest does not parse:\n%v", err) t.Fatalf("the controller's own manifest does not parse:\n%v", err)
} }
+2 -2
View File
@@ -18,7 +18,7 @@ func TestTheControllersOwnManifestClaimsItsSeat(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("the controller's own manifest is unreadable: %v", err) t.Fatalf("the controller's own manifest is unreadable: %v", err)
} }
m, err := ParseManifest(raw) m, err := Decode(raw)
if err != nil { if err != nil {
t.Fatalf("the controller's own manifest does not parse: %v", err) t.Fatalf("the controller's own manifest does not parse: %v", err)
} }
@@ -41,7 +41,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err := ParseManifest(raw) m, err := Decode(raw)
if err != nil { if err != nil {
t.Fatalf("the control plane's own manifest does not parse:\n%v", err) t.Fatalf("the control plane's own manifest does not parse:\n%v", err)
} }
+1 -1
View File
@@ -34,7 +34,7 @@ func TestEveryManifestInUseClaimsASeatTheMeshDefines(t *testing.T) {
if raw, err = os.ReadFile(path); err != nil { if raw, err = os.ReadFile(path); err != nil {
t.Fatal(err) t.Fatal(err)
} }
m, err = catalogue.ParseManifest(raw) m, err = catalogue.Decode(raw)
} else { } else {
m, err = catalogue.ReadTreeManifest(root, filepath.Dir(path)) m, err = catalogue.ReadTreeManifest(root, filepath.Dir(path))
} }
+25 -6
View File
@@ -9,10 +9,27 @@ The shipping form of the reference reverse proxy (novox/hq ADR 0007, novox/hq 08
is simply deprecated/invalid"). A module that must be reachable declares `requires: ["route"]` and contributes the is simply deprecated/invalid"). A module that must be reachable declares `requires: ["route"]` and contributes the
name it wants and the port it listens on; this module provides `route`. The controller composes each consumer's plan, name it wants and the port it listens on; this module provides `route`. The controller composes each consumer's plan,
and the seat `mesh-provisioner` keeps it in its state `plans`: the plan's entry for the route names this proxy as and the seat `mesh-provisioner` keeps it in its state `plans`: the plan's entry for the route names this proxy as
its provider and carries what the consumer contributes, composed (`given`: its names joined to the node's domains, its its provider and carries what the consumer contributes, composed (its names joined to the node's domains, its
endpoint's port, its policy), and where the consumer's machine is on the private network (`consumer-at`). The proxy endpoint's port, its policy), and where the consumer's machine is on the private network, as one opaque payload
follows the plans on the bus and forwards by the `Host` header of each request to `http://<consumer-at>:<port>`, or (`contributed`) the seat never reads. The proxy follows the plans on the bus, reads each contribution with its own
over loopback for a consumer on its own machine. contract (below), and forwards by the `Host` header of each request to `http://<at>:<port>`, or over loopback for a
consumer on its own machine.
## Its contract: a package of its own
**route-proxy defines its own contract, and other modules use it** (the operator, 2026-10-31). It is the Go module
`git.novox.be/novox/mesh-controller/modules/route-proxy/contract`, in `contract/` here with its own `go.mod`, standard
library only, released by tags `modules/route-proxy/contract/vX.Y.Z`, the way mesh-sdk is released (`go/vX.Y.Z`); this
module depends on it by its version like any other. It holds:
- **the route a consumer asks** (`Request`, read strictly by `DecodeRequest`: no field it does not name, a name of
either kind, a port where the rule forwards, http or https, a positive body limit, a secret's file never a
credential) and its JSON schema (`route.schema.json`, `Schema`), the form of a manifest's `contributes.route`;
- **the events this proxy says** on `mesh.mod.route-proxy.event.<type>`: `route.served` and `route.withdrawn` as the
names it serves change, `route.refused` with why for a route a plan gives it and it does not serve (`Event`);
- **what it keeps on the bus**, its states `routes` and `unrouted`, their keys and records (`Route`, `Unrouted`,
`Other`, `Key`, `Split`);
- **its tools' names and arguments** (`ToolRoutes`, `ToolRoute`, `ToolUnrouted`, `RoutesArgs`, `RouteArgs`).
The controller does not write the proxy's configuration. It composes the facts into the plans and the proxy turns them The controller does not write the proxy's configuration. It composes the facts into the plans and the proxy turns them
into whatever it runs. That boundary is why swapping this proxy for another costs nothing to the modules that publish into whatever it runs. That boundary is why swapping this proxy for another costs nothing to the modules that publish
@@ -24,12 +41,14 @@ migration scaffolding still listed there.
|---|---|---| |---|---|---|
| `provides` | `route` (scope `mesh`, answered by the helper) | a module on any node may require it | | `provides` | `route` (scope `mesh`, answered by the helper) | a module on any node may require it |
| `serves.route` | `{}` | a route hands back a name, not a credential — the consumer already knows the name it asked for | | `serves.route` | `{}` | a route hands back a name, not a credential — the consumer already knows the name it asked for |
| `reads` | `mesh-provisioner.plans`, `mesh-provisioner.provisions` | the plans are where the routes are; the helper reads both | | `reads` | `mesh-provisioner.plans`, `mesh-provisioner.provisions`, `mesh-delivery-assigner.placement` | the plans are where the routes are; the helper reads both; the placement says who is inside |
| `emits` | `route.served`, `route.withdrawn`, `route.refused` | what it serves, said as it changes (the contract's `Event`) |
| `listens` | `80` and `443`, both `from: anywhere` | the one machine with a public opening; the firewall opens these for free (ADR 0045) | | `listens` | `80` and `443`, both `from: anywhere` | the one machine with a public opening; the firewall opens these for free (ADR 0045) |
Until the bus answers, the proxy routes nothing and says so on its log; it refuses to start without its broker Until the bus answers, the proxy routes nothing and says so on its log; it refuses to start without its broker
credential (`MESH_BROKER_FILE`). Who is inside the private network, and so may be served an internal-only name, is credential (`MESH_BROKER_FILE`). Who is inside the private network, and so may be served an internal-only name, is
still read from its membership (novox/hq ADR 0167). every node's address on it as the seat mesh-placement decides it (mesh-sdk `delivery.InsideAddresses`); until that
is read, an internal-only name is served to this machine alone.
## Which issuer: Let's Encrypt, stated here (novox/hq ADR 0226) ## Which issuer: Let's Encrypt, stated here (novox/hq ADR 0226)
@@ -14,7 +14,7 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/provider" "git.novox.be/novox/mesh-sdk/go/provider"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
func say(format string, args ...any) { func say(format string, args ...any) {
@@ -36,7 +36,7 @@ func main() {
} }
}() }()
} }
tools := front{routes: stdio.State(routestate.RoutesState), unrouted: stdio.State(routestate.UnroutedState), tools := front{routes: stdio.State(contract.RoutesState), unrouted: stdio.State(contract.UnroutedState),
now: time.Now}.tools() now: time.Now}.tools()
if err := stdio.Serve("", tools); err != nil { if err := stdio.Serve("", tools); err != nil {
say("%v", err) say("%v", err)
@@ -13,13 +13,13 @@ package main
import ( import (
"git.novox.be/novox/mesh-sdk/go/provider" "git.novox.be/novox/mesh-sdk/go/provider"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// ProvisionName is what route-proxy provides, as its manifest says, and ModuleName this module. // ProvisionName is what route-proxy provides, as its manifest says, and ModuleName this module.
const ( const (
ProvisionName = routestate.Provision ProvisionName = contract.Provision
ModuleName = routestate.Module ModuleName = contract.Module
) )
// newProvider is route-proxy answering its consumers over bus on node: the route, served as the manifest serves it // newProvider is route-proxy answering its consumers over bus on node: the route, served as the manifest serves it
@@ -130,12 +130,13 @@ func TestRouteProxyIsNotMovedAndKeepsItsOwnFiles(t *testing.T) {
for _, k := range []string{"build", "provides", "state", "reads", "tools", "replaces"} { for _, k := range []string{"build", "provides", "state", "reads", "tools", "replaces"} {
delete(m, k) delete(m, k)
} }
// Refreshed on purpose for the events its contract names (`emits`: route.served, route.withdrawn, route.refused).
pinned, err := json.Marshal(m) // sorted keys, compact pinned, err := json.Marshal(m) // sorted keys, compact
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
sum := sha256.Sum256(pinned) sum := sha256.Sum256(pinned)
if got := hex.EncodeToString(sum[:]); got != "e96702560003ae7f27490d27fd2353f8c6680d8e16864e15fdc0a833108bc2e5" { if got := hex.EncodeToString(sum[:]); got != "3a44c56cc021be4008f521d16fa02e1c0f0d2eedab2f4d0b60f7d3177269e542" {
t.Fatalf("route-proxy's own manifest changed: digest %s", got) t.Fatalf("route-proxy's own manifest changed: digest %s", got)
} }
// The two acme templates are pinned the same way (issue 849): they are what the proxy's certificates are asked // The two acme templates are pinned the same way (issue 849): they are what the proxy's certificates are asked
+18 -18
View File
@@ -14,14 +14,14 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// The tools' names, as the manifest declares them. // The tools' names, as the manifest declares them.
const ( const (
RoutesTool = "route_proxy_routes" RoutesTool = contract.ToolRoutes
RouteTool = "route_proxy_route" RouteTool = contract.ToolRoute
UnroutedTool = "route_proxy_unrouted" UnroutedTool = contract.ToolUnrouted
) )
// reader is one of the module's states, as stdio.KeptState reads it. // reader is one of the module's states, as stdio.KeptState reads it.
@@ -48,7 +48,7 @@ type RouteLine struct {
// Certificate is its outcome, issuer and expiry in words; ExpiresInDays is the whole days until not-after. // Certificate is its outcome, issuer and expiry in words; ExpiresInDays is the whole days until not-after.
Certificate string `json:"certificate"` Certificate string `json:"certificate"`
ExpiresInDays *int `json:"expires_in_days,omitempty"` ExpiresInDays *int `json:"expires_in_days,omitempty"`
LastAnswer *routestate.Answer `json:"last_answer,omitempty"` LastAnswer *contract.Answer `json:"last_answer,omitempty"`
} }
func (f front) tools() []stdio.Tool { func (f front) tools() []stdio.Tool {
@@ -99,7 +99,7 @@ func read[T any](state reader, node string) ([]T, error) {
} }
var out []T var out []T
for _, key := range keys { for _, key := range keys {
at, _ := routestate.Split(key) at, _ := contract.Split(key)
if node != "" && at != node { if node != "" && at != node {
continue continue
} }
@@ -120,7 +120,7 @@ func read[T any](state reader, node string) ([]T, error) {
} }
func (f front) list(node string) (map[string]any, error) { func (f front) list(node string) (map[string]any, error) {
routes, err := read[routestate.Route](f.routes, node) routes, err := read[contract.Route](f.routes, node)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -131,7 +131,7 @@ func (f front) list(node string) (map[string]any, error) {
if line.Reachable == "no" { if line.Reachable == "no" {
unreachable++ unreachable++
} }
if line.ExpiresInDays != nil && *line.ExpiresInDays < 14 || r.Certificate.Outcome == routestate.Failed { if line.ExpiresInDays != nil && *line.ExpiresInDays < 14 || r.Certificate.Outcome == contract.Failed {
expiring++ expiring++
} }
lines = append(lines, line) lines = append(lines, line)
@@ -146,7 +146,7 @@ func (f front) list(node string) (map[string]any, error) {
"certificates_failing_or_within_14_days": expiring}, nil "certificates_failing_or_within_14_days": expiring}, nil
} }
func (f front) line(r routestate.Route) RouteLine { func (f front) line(r contract.Route) RouteLine {
line := RouteLine{Name: r.Name, Proxy: r.Proxy, Reach: r.Reach, LastAnswer: r.LastAnswer, Reachable: "unknown", line := RouteLine{Name: r.Name, Proxy: r.Proxy, Reach: r.Reach, LastAnswer: r.LastAnswer, Reachable: "unknown",
Targets: []string{}} Targets: []string{}}
for _, t := range r.Targets { for _, t := range r.Targets {
@@ -182,7 +182,7 @@ func (f front) line(r routestate.Route) RouteLine {
} }
// describe is one rule in a few words: path → module@node at target, or what it answers with. // describe is one rule in a few words: path → module@node at target, or what it answers with.
func describe(t routestate.Target) string { func describe(t contract.Target) string {
path := t.Path path := t.Path
if path == "" { if path == "" {
path = "/" path = "/"
@@ -209,11 +209,11 @@ func (f front) one(name, node string) (map[string]any, error) {
if name == "" { if name == "" {
return nil, fmt.Errorf("name is the site to read, such as app.example.org") return nil, fmt.Errorf("name is the site to read, such as app.example.org")
} }
routes, err := read[routestate.Route](f.routes, node) routes, err := read[contract.Route](f.routes, node)
if err != nil { if err != nil {
return nil, err return nil, err
} }
var found []routestate.Route var found []contract.Route
for _, r := range routes { for _, r := range routes {
if r.Name == name { if r.Name == name {
found = append(found, r) found = append(found, r)
@@ -248,32 +248,32 @@ func (f front) refusals(node string) (map[string]any, error) {
} }
sort.Slice(asked, func(i, j int) bool { return asked[i].AskedAt.After(asked[j].AskedAt) }) sort.Slice(asked, func(i, j int) bool { return asked[i].AskedAt.After(asked[j].AskedAt) })
if asked == nil { if asked == nil {
asked = []routestate.Unrouted{} asked = []contract.Unrouted{}
} }
return map[string]any{"unrouted": asked, "names": len(asked), "other_refusals": others}, nil return map[string]any{"unrouted": asked, "names": len(asked), "other_refusals": others}, nil
} }
// asked is the names refused and kept by name, and each proxy's count of the refusals not kept by name. // asked is the names refused and kept by name, and each proxy's count of the refusals not kept by name.
func (f front) asked(node string) ([]routestate.Unrouted, []routestate.Other, error) { func (f front) asked(node string) ([]contract.Unrouted, []contract.Other, error) {
keys, err := f.unrouted.Keys() keys, err := f.unrouted.Keys()
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
names, others := keySet{}, keySet{} names, others := keySet{}, keySet{}
for _, key := range keys { for _, key := range keys {
if _, name := routestate.Split(key); name == routestate.OtherName { if _, name := contract.Split(key); name == contract.OtherName {
others[key] = true others[key] = true
} else { } else {
names[key] = true names[key] = true
} }
} }
asked, err := read[routestate.Unrouted](only{f.unrouted, names}, node) asked, err := read[contract.Unrouted](only{f.unrouted, names}, node)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
counted, err := read[routestate.Other](only{f.unrouted, others}, node) counted, err := read[contract.Other](only{f.unrouted, others}, node)
if counted == nil { if counted == nil {
counted = []routestate.Other{} counted = []contract.Other{}
} }
return asked, counted, err return asked, counted, err
} }
@@ -9,7 +9,7 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// held is a state as the bus holds it, read-only. // held is a state as the bus holds it, read-only.
@@ -40,19 +40,19 @@ func TestTheWebFrontsToolsAnswerFromTheBus(t *testing.T) {
now := time.Date(2026, 10, 30, 21, 0, 0, 0, time.UTC) now := time.Date(2026, 10, 30, 21, 0, 0, 0, time.UTC)
expires := now.Add(30 * 24 * time.Hour) expires := now.Add(30 * 24 * time.Hour)
routes := held{ routes := held{
"anchor.grafana.example": routestate.Route{Name: "grafana.example", Proxy: "anchor", Reach: routestate.ReachPublic, "anchor.grafana.example": contract.Route{Name: "grafana.example", Proxy: "anchor", Reach: contract.ReachPublic,
Targets: []routestate.Target{{Module: "grafana", Node: "home-server", Target: "http://10.0.0.2:3000"}}, Targets: []contract.Target{{Module: "grafana", Node: "home-server", Target: "http://10.0.0.2:3000"}},
Certificate: routestate.Certificate{Authority: routestate.ReachPublic, Issuer: "Let's Encrypt R11", NotAfter: &expires, Outcome: routestate.Issued}, Certificate: contract.Certificate{Authority: contract.ReachPublic, Issuer: "Let's Encrypt R11", NotAfter: &expires, Outcome: contract.Issued},
LastAnswer: &routestate.Answer{Status: 502, At: now, Error: "dial tcp 10.0.0.2:3000: connect: connection refused"}}, LastAnswer: &contract.Answer{Status: 502, At: now, Error: "dial tcp 10.0.0.2:3000: connect: connection refused"}},
"anchor.drive.example": routestate.Route{Name: "drive.example", Proxy: "anchor", Reach: routestate.ReachPublic, "anchor.drive.example": contract.Route{Name: "drive.example", Proxy: "anchor", Reach: contract.ReachPublic,
Targets: []routestate.Target{{Module: "nextcloud", Node: "home-server", Target: "http://10.0.0.2:8080"}}, Targets: []contract.Target{{Module: "nextcloud", Node: "home-server", Target: "http://10.0.0.2:8080"}},
Certificate: routestate.Certificate{Authority: routestate.ReachPublic}, Certificate: contract.Certificate{Authority: contract.ReachPublic},
LastAnswer: &routestate.Answer{Status: 200, At: now}}, LastAnswer: &contract.Answer{Status: 200, At: now}},
} }
unrouted := held{ unrouted := held{
"anchor.old.example": routestate.Unrouted{Name: "old.example", Proxy: "anchor", AskedAt: now.Add(-time.Hour), Times: 3}, "anchor.old.example": contract.Unrouted{Name: "old.example", Proxy: "anchor", AskedAt: now.Add(-time.Hour), Times: 3},
"anchor.wiki.example": routestate.Unrouted{Name: "wiki.example", Proxy: "anchor", AskedAt: now, Times: 1}, "anchor.wiki.example": contract.Unrouted{Name: "wiki.example", Proxy: "anchor", AskedAt: now, Times: 1},
"anchor._other": routestate.Other{Proxy: "anchor", Minute: now, LastMinute: 40, SinceStart: 900}, "anchor._other": contract.Other{Proxy: "anchor", Minute: now, LastMinute: 40, SinceStart: 900},
} }
tools := map[string]stdio.Tool{} tools := map[string]stdio.Tool{}
for _, tool := range (front{routes: routes, unrouted: unrouted, now: func() time.Time { return now }}).tools() { for _, tool := range (front{routes: routes, unrouted: unrouted, now: func() time.Time { return now }}).tools() {
+98 -32
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"fmt" "fmt"
"log" "log"
"sort"
"strings" "strings"
"sync/atomic" "sync/atomic"
"time" "time"
@@ -11,20 +12,21 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/announce" "git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/membership" "git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/process" "git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/provisions" "git.novox.be/novox/mesh-sdk/go/provisions"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// keptStates is every state route-proxy keeps, as both its halves write them: the routes and the refusals this // keptStates is every state route-proxy keeps, as both its halves write them: the routes and the refusals this
// proxy writes, readable by others, and the bound entries the provider helper writes. // proxy writes, readable by others, and the bound entries the provider helper writes.
func keptStates() []announce.State { func keptStates() []announce.State {
return []announce.State{ return []announce.State{
{Name: routestate.BoundState}, {Name: contract.BoundState},
{Name: routestate.RoutesState, Readable: true}, {Name: contract.RoutesState, Readable: true},
{Name: routestate.UnroutedState, Readable: true}, {Name: contract.UnroutedState, Readable: true},
} }
} }
@@ -32,12 +34,13 @@ func keptStates() []announce.State {
// //
// **The routes are the seat mesh-provisioner's plans, alone** (issue 774, research 072 row 34): every plan's entry // **The routes are the seat mesh-provisioner's plans, alone** (issue 774, research 072 row 34): every plan's entry
// naming this proxy as a route's provider, followed, so a route added or withdrawn reaches a running proxy without a // naming this proxy as a route's provider, followed, so a route added or withdrawn reaches a running proxy without a
// restart. **Who is inside** is this proxy's membership (novox/hq ADR 0160, ADR 0167): every machine's address on the // restart, and said on the bus as it changes (contract.EventServed, EventWithdrawn, EventRefused). **Who is inside**
// private network, which is who may be served an internal name; until it is read, an internal name is served to this // is every node's address on the private network as the seat mesh-placement decides it (mesh-sdk
// machine alone. // delivery.InsideAddresses), which is who may be served an internal name; until it is read, an internal name is
// served to this machine alone.
// follow connects with the credential in path, follows the plans and the membership, and retries the connection for as // follow connects with the credential in path, follows the plans and the placement, and retries the connection for
// long as it takes: until the bus answers, the proxy routes nothing, and says so. // as long as it takes: until the bus answers, the proxy routes nothing, and says so.
func follow(path string, held *table) { func follow(path string, held *table) {
for { for {
err := followOnce(path, held) err := followOnce(path, held)
@@ -64,39 +67,40 @@ func followOnce(path string, held *table) error {
if err := conn.Keeps(keptStates()...); err != nil { if err := conn.Keeps(keptStates()...); err != nil {
log.Printf("cannot say on the bus what this proxy keeps, so it keeps nothing there: %v", err) log.Printf("cannot say on the bus what this proxy keeps, so it keeps nothing there: %v", err)
} else { } else {
held.ledger.attach(account.Node, conn.State(routestate.RoutesState), conn.State(routestate.UnroutedState)) held.ledger.attach(account.Node, conn.State(contract.RoutesState), conn.State(contract.UnroutedState))
} }
if err := followPlans(conn.Read(provisions.PlansRead), newPlanned(account.Node), held); err != nil { say := func(eventType string, e contract.Event) {
if err := conn.Emit(eventType, e); err != nil {
log.Printf("cannot say %s of %q on the bus: %v", eventType, e.Name, err)
}
}
// Who is inside first, so the first table is served to the machines it is meant for.
if err := followInside(conn.Read(delivery.PlacementRead), held); err != nil {
log.Printf("cannot follow %s, so an internal name is served to this machine alone: %v", delivery.PlacementRead, err)
}
if err := followPlans(conn.Read(provisions.PlansRead), newPlanned(account.Node, say), held, say); err != nil {
conn.Close() conn.Close()
return err return err
} }
subject := membership.Subject(account.Node, account.Module)
err = conn.FollowMembership(func(issued membership.Membership) {
applyMembership(issued, held)
}, func(err error) { log.Print(err) })
if err != nil {
log.Printf("cannot follow this proxy's membership on %s, so an internal name is served to this machine "+
"alone: %v", subject, err)
}
return nil return nil
} }
// planWatch is the seat's plans as this proxy watches them: every current key, then every change. // planWatch is a state this proxy watches: every current key, then every change.
type planWatch interface { type planWatch interface {
Watch(key string, handle func(stdio.StateChange) error) error Watch(key string, handle func(stdio.StateChange) error) error
} }
// followPlans serves what the plans give this proxy: the whole table once every current plan is read, then again on // followPlans serves what the plans give this proxy: the whole table once every current plan is read, then again on
// every change that alters it. A table built from part of the plans would withdraw every route not yet read. // every change that alters it. A table built from part of the plans would withdraw every route not yet read.
func followPlans(plans planWatch, given *planned, held *table) error { func followPlans(plans planWatch, given *planned, held *table, say func(string, contract.Event)) error {
var read atomic.Bool // every current plan is read: from here, each change is served as it comes var read atomic.Bool // every current plan is read: from here, each change is served as it comes
err := plans.Watch("", func(c stdio.StateChange) error { err := plans.Watch("", func(c stdio.StateChange) error {
var value []byte var value []byte
if c.Op != "delete" { if c.Op != "delete" && c.Op != "purge" {
value = c.Value value = c.Value
} }
if changed := given.take(c.Key, value); changed && read.Load() { if changed := given.take(c.Key, value); changed && read.Load() {
serve(given, held) serve(given, held, say)
} }
return nil return nil
}) })
@@ -104,25 +108,87 @@ func followPlans(plans planWatch, given *planned, held *table) error {
return fmt.Errorf("watching %s: %w", provisions.PlansRead, err) return fmt.Errorf("watching %s: %w", provisions.PlansRead, err)
} }
read.Store(true) read.Store(true)
serve(given, held) serve(given, held, say)
return nil return nil
} }
// serve replaces what the proxy serves with what the plans give it. // serve replaces what the proxy serves with what the plans give it, and says each name it now serves or serves no
func serve(given *planned, held *table) { // more.
func serve(given *planned, held *table, say func(string, contract.Event)) {
before := held.served()
routes, public := routesOf(given.contributions()) routes, public := routesOf(given.contributions())
held.set(routes, public) held.set(routes, public)
after := held.served()
for _, e := range servedChanges(given.node, before, after) {
say(e.kind, e.Event)
}
log.Printf("serving %d route(s) from the seat's plans: %s", len(routes), strings.Join(held.names(), ", ")) log.Printf("serving %d route(s) from the seat's plans: %s", len(routes), strings.Join(held.names(), ", "))
} }
// applyMembership takes who is inside the mesh from a membership, and says whether it could read it. // change is one event serve says.
func applyMembership(issued membership.Membership, held *table) bool { type change struct {
inside, err := sourcesOf(issued.Mesh) kind string
contract.Event
}
// servedChanges is what changed between two tables' names, each name once, in order: a name now served, or now
// served at another reach or by another module, is served; a name no longer served is withdrawn.
func servedChanges(proxy string, before, after map[string]contract.Event) []change {
var out []change
for _, name := range sortedNames(after) {
if was, ok := before[name]; !ok || was != after[name] {
e := after[name]
e.Proxy = proxy
out = append(out, change{contract.EventServed, e})
}
}
for _, name := range sortedNames(before) {
if _, ok := after[name]; !ok {
out = append(out, change{contract.EventWithdrawn, contract.Event{Proxy: proxy, Name: name}})
}
}
return out
}
func sortedNames(m map[string]contract.Event) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// followInside follows the seat mesh-placement's state and serves an internal name to every node's address on the
// private network it decides, each time it changes (mesh-sdk delivery.InsideAddresses; research 072 rows 33 and 34:
// the membership that carried them went with the assignments stream). A key that does not read keeps who is inside
// as it was.
func followInside(placement planWatch, held *table) error {
keys := delivery.NewPlacementKeys()
var read atomic.Bool
apply := func() {
inside, err := sourcesOf(delivery.InsideAddresses(keys.All()))
if err != nil { if err != nil {
log.Printf("the mesh in this proxy's membership is unreadable, keeping who is inside as it was: %v", err) log.Printf("the placement says an address that is not one, so who is inside stays as it was: %v", err)
return false return
} }
held.setInside(inside) held.setInside(inside)
log.Printf("internal names are served to %d machine(s)", len(inside)) log.Printf("internal names are served to %d machine(s)", len(inside))
return true }
err := placement.Watch("", func(c stdio.StateChange) error {
if _, err := keys.Put(c.Key, c.Value, c.Revision, c.Op == "delete" || c.Op == "purge"); err != nil {
log.Printf("the placement's key %s does not read, so who is inside stays as it was: %v", c.Key, err)
return nil
}
if read.Load() {
apply()
}
return nil
})
if err != nil {
return err
}
read.Store(true)
apply()
return nil
} }
+33 -33
View File
@@ -17,7 +17,7 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// What this proxy says on the bus about the names it serves and the names it refuses (issue 774). // What this proxy says on the bus about the names it serves and the names it refuses (issue 774).
@@ -55,9 +55,9 @@ type ledger struct {
node string node string
routes, unrouted kvStore routes, unrouted kvStore
served map[string]*routestate.Route served map[string]*contract.Route
previous map[string]routestate.Route // what the bus held for this machine when it was reached previous map[string]contract.Route // what the bus held for this machine when it was reached
asked map[string]*routestate.Unrouted asked map[string]*contract.Unrouted
answerWritten map[string]time.Time answerWritten map[string]time.Time
askedWritten map[string]time.Time askedWritten map[string]time.Time
@@ -69,7 +69,7 @@ type ledger struct {
domains map[string]bool domains map[string]bool
newMinute time.Time newMinute time.Time
newInMinute int newInMinute int
other routestate.Other other contract.Other
otherMinute time.Time otherMinute time.Time
otherInMinute int otherInMinute int
otherDirty bool otherDirty bool
@@ -81,9 +81,9 @@ type ledger struct {
func newLedger() *ledger { func newLedger() *ledger {
return &ledger{ return &ledger{
now: time.Now, now: time.Now,
served: map[string]*routestate.Route{}, served: map[string]*contract.Route{},
previous: map[string]routestate.Route{}, previous: map[string]contract.Route{},
asked: map[string]*routestate.Unrouted{}, asked: map[string]*contract.Unrouted{},
answerWritten: map[string]time.Time{}, answerWritten: map[string]time.Time{},
askedWritten: map[string]time.Time{}, askedWritten: map[string]time.Time{},
dirtyRoutes: map[string]bool{}, dirtyRoutes: map[string]bool{},
@@ -113,12 +113,12 @@ func (l *ledger) attach(node string, routes, unrouted kvStore) {
if l == nil { if l == nil {
return return
} }
before := map[string]routestate.Route{} before := map[string]contract.Route{}
if keys, err := routes.Keys(); err != nil { if keys, err := routes.Keys(); err != nil {
log.Printf("cannot read back what this proxy said it serves: %v", err) log.Printf("cannot read back what this proxy said it serves: %v", err)
} else { } else {
for _, key := range keys { for _, key := range keys {
at, name := routestate.Split(key) at, name := contract.Split(key)
if at != node { if at != node {
continue continue
} }
@@ -126,7 +126,7 @@ func (l *ledger) attach(node string, routes, unrouted kvStore) {
if err != nil || entry == nil { if err != nil || entry == nil {
continue continue
} }
var r routestate.Route var r contract.Route
if json.Unmarshal(entry.Value, &r) == nil { if json.Unmarshal(entry.Value, &r) == nil {
before[name] = r before[name] = r
} }
@@ -152,7 +152,7 @@ func (l *ledger) attach(node string, routes, unrouted kvStore) {
} }
// keepWhatWasKnown carries a certificate and a last answer read back from the bus onto a route that knows none yet. // keepWhatWasKnown carries a certificate and a last answer read back from the bus onto a route that knows none yet.
func keepWhatWasKnown(r *routestate.Route, before routestate.Route) { func keepWhatWasKnown(r *contract.Route, before contract.Route) {
if r.Certificate.Outcome == "" && r.Certificate.NotAfter == nil && before.Certificate.Authority == r.Certificate.Authority { if r.Certificate.Outcome == "" && r.Certificate.NotAfter == nil && before.Certificate.Authority == r.Certificate.Authority {
r.Certificate = before.Certificate r.Certificate = before.Certificate
} }
@@ -170,14 +170,14 @@ func (l *ledger) serving(routes map[string][]rule, public map[string]bool) {
l.mu.Lock() l.mu.Lock()
defer l.mu.Unlock() defer l.mu.Unlock()
for name, rules := range routes { for name, rules := range routes {
reach := routestate.ReachInternal reach := contract.ReachInternal
if public[name] { if public[name] {
reach = routestate.ReachPublic reach = contract.ReachPublic
} }
targets := targetsOf(rules) targets := targetsOf(rules)
r := l.served[name] r := l.served[name]
if r == nil { if r == nil {
r = &routestate.Route{Name: name} r = &contract.Route{Name: name}
if before, ok := l.previous[name]; ok { if before, ok := l.previous[name]; ok {
r.Certificate.Authority = reach r.Certificate.Authority = reach
keepWhatWasKnown(r, before) keepWhatWasKnown(r, before)
@@ -190,7 +190,7 @@ func (l *ledger) serving(routes map[string][]rule, public map[string]bool) {
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
} }
if r.Certificate.Authority != reach { if r.Certificate.Authority != reach {
r.Certificate = routestate.Certificate{Authority: reach} r.Certificate = contract.Certificate{Authority: reach}
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
} }
if _, was := l.asked[name]; was { if _, was := l.asked[name]; was {
@@ -200,7 +200,7 @@ func (l *ledger) serving(routes map[string][]rule, public map[string]bool) {
} }
l.domains = map[string]bool{} l.domains = map[string]bool{}
for name := range routes { for name := range routes {
if d := routestate.Registrable(name); d != "" { if d := contract.Registrable(name); d != "" {
l.domains[d] = true l.domains[d] = true
} }
} }
@@ -214,10 +214,10 @@ func (l *ledger) serving(routes map[string][]rule, public map[string]bool) {
l.wake() l.wake()
} }
func targetsOf(rules []rule) []routestate.Target { func targetsOf(rules []rule) []contract.Target {
out := make([]routestate.Target, 0, len(rules)) out := make([]contract.Target, 0, len(rules))
for _, r := range rules { for _, r := range rules {
out = append(out, routestate.Target{Path: r.path, Module: r.from, Node: r.node, Target: r.target, out = append(out, contract.Target{Path: r.path, Module: r.from, Node: r.node, Target: r.target,
Deny: r.policy.deny, Redirect: r.policy.redirectTo, Auth: len(r.policy.users) > 0 || r.policy.sealed != "", Deny: r.policy.deny, Redirect: r.policy.redirectTo, Auth: len(r.policy.users) > 0 || r.policy.sealed != "",
Sealed: r.policy.sealed}) Sealed: r.policy.sealed})
} }
@@ -241,7 +241,7 @@ func (l *ledger) answered(name, path string, status int, failed error) {
why = failed.Error() why = failed.Error()
} }
before := r.LastAnswer before := r.LastAnswer
r.LastAnswer = &routestate.Answer{Status: status, At: now, Error: why, Path: path} r.LastAnswer = &contract.Answer{Status: status, At: now, Error: why, Path: path}
if before != nil && before.Status == status && before.Error == why && now.Sub(l.answerWritten[name]) < refreshAfter { if before != nil && before.Status == status && before.Error == why && now.Sub(l.answerWritten[name]) < refreshAfter {
return return
} }
@@ -268,11 +268,11 @@ func (l *ledger) refused(name string, internalOnly, handshake bool) {
now := l.now() now := l.now()
a := l.asked[name] a := l.asked[name]
if a == nil { if a == nil {
if !routestate.Keyable(name) || !l.domains[routestate.Registrable(name)] || !l.mayAddName(now) { if !contract.Keyable(name) || !l.domains[contract.Registrable(name)] || !l.mayAddName(now) {
l.countOther(now) l.countOther(now)
return return
} }
a = &routestate.Unrouted{Name: name} a = &contract.Unrouted{Name: name}
l.asked[name] = a l.asked[name] = a
} }
a.AskedAt, a.InternalOnly, a.Handshake = now, internalOnly, handshake a.AskedAt, a.InternalOnly, a.Handshake = now, internalOnly, handshake
@@ -345,7 +345,7 @@ func (l *ledger) certified(name string, leaf *x509.Certificate) {
notAfter := leaf.NotAfter.UTC() notAfter := leaf.NotAfter.UTC()
r.Certificate.Issuer, r.Certificate.NotAfter = issuerOf(leaf), &notAfter r.Certificate.Issuer, r.Certificate.NotAfter = issuerOf(leaf), &notAfter
r.Certificate.RenewedAt, r.Certificate.At = &now, &now r.Certificate.RenewedAt, r.Certificate.At = &now, &now
r.Certificate.Outcome, r.Certificate.Error = routestate.Issued, "" r.Certificate.Outcome, r.Certificate.Error = contract.Issued, ""
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
l.wake() l.wake()
} }
@@ -362,13 +362,13 @@ func (l *ledger) handedOut(name string, leaf *x509.Certificate) {
return return
} }
notAfter := leaf.NotAfter.UTC() notAfter := leaf.NotAfter.UTC()
if r.Certificate.NotAfter != nil && r.Certificate.NotAfter.Equal(notAfter) && r.Certificate.Outcome != routestate.Failed { if r.Certificate.NotAfter != nil && r.Certificate.NotAfter.Equal(notAfter) && r.Certificate.Outcome != contract.Failed {
return return
} }
now := l.now() now := l.now()
r.Certificate.Issuer, r.Certificate.NotAfter, r.Certificate.At = issuerOf(leaf), &notAfter, &now r.Certificate.Issuer, r.Certificate.NotAfter, r.Certificate.At = issuerOf(leaf), &notAfter, &now
if r.Certificate.Outcome != routestate.Issued || r.Certificate.Error != "" { if r.Certificate.Outcome != contract.Issued || r.Certificate.Error != "" {
r.Certificate.Outcome, r.Certificate.Error = routestate.Cached, "" r.Certificate.Outcome, r.Certificate.Error = contract.Cached, ""
} }
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
l.wake() l.wake()
@@ -386,11 +386,11 @@ func (l *ledger) certificateFailed(name string, failed error) {
return return
} }
now := l.now() now := l.now()
same := r.Certificate.Outcome == routestate.Failed && r.Certificate.Error == failed.Error() same := r.Certificate.Outcome == contract.Failed && r.Certificate.Error == failed.Error()
if same && r.Certificate.At != nil && now.Sub(*r.Certificate.At) < refreshAfter { if same && r.Certificate.At != nil && now.Sub(*r.Certificate.At) < refreshAfter {
return return
} }
r.Certificate.Outcome, r.Certificate.Error, r.Certificate.At = routestate.Failed, failed.Error(), &now r.Certificate.Outcome, r.Certificate.Error, r.Certificate.At = contract.Failed, failed.Error(), &now
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
l.wake() l.wake()
} }
@@ -423,7 +423,7 @@ func (l *ledger) flush() {
l.forgetOldestAsked() l.forgetOldestAsked()
var puts []put var puts []put
for name := range l.dirtyRoutes { for name := range l.dirtyRoutes {
p := put{store: l.routes, key: routestate.Key(l.node, name), route: true} p := put{store: l.routes, key: contract.Key(l.node, name), route: true}
if r := l.served[name]; r != nil { if r := l.served[name]; r != nil {
c := *r c := *r
c.Proxy = l.node c.Proxy = l.node
@@ -432,7 +432,7 @@ func (l *ledger) flush() {
puts = append(puts, p) puts = append(puts, p)
} }
for name := range l.dirtyAsked { for name := range l.dirtyAsked {
p := put{store: l.unrouted, key: routestate.Key(l.node, name)} p := put{store: l.unrouted, key: contract.Key(l.node, name)}
if a := l.asked[name]; a != nil { if a := l.asked[name]; a != nil {
c := *a c := *a
c.Proxy = l.node c.Proxy = l.node
@@ -443,7 +443,7 @@ func (l *ledger) flush() {
if l.otherDirty { if l.otherDirty {
o := l.other o := l.other
o.Proxy = l.node o.Proxy = l.node
puts = append(puts, put{store: l.unrouted, key: routestate.Key(l.node, routestate.OtherName), value: o}) puts = append(puts, put{store: l.unrouted, key: contract.Key(l.node, contract.OtherName), value: o})
} }
l.dirtyRoutes, l.dirtyAsked, l.otherDirty = map[string]bool{}, map[string]bool{}, false l.dirtyRoutes, l.dirtyAsked, l.otherDirty = map[string]bool{}, map[string]bool{}, false
l.mu.Unlock() l.mu.Unlock()
@@ -466,7 +466,7 @@ func (l *ledger) flush() {
log.Printf("cannot write %s to the bus, kept for the next change: %v", p.key, err) log.Printf("cannot write %s to the bus, kept for the next change: %v", p.key, err)
} }
l.routesFailing = true l.routesFailing = true
_, name := routestate.Split(p.key) _, name := contract.Split(p.key)
l.dirtyRoutes[name] = true l.dirtyRoutes[name] = true
default: default:
if !l.askFailed { if !l.askFailed {
@@ -25,7 +25,7 @@ import (
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// fakeState is a state on the bus, in memory, counting what was written to it. // fakeState is a state on the bus, in memory, counting what was written to it.
@@ -68,13 +68,13 @@ func (f *fakeState) Get(key string) (*stdio.StateEntry, error) {
return &stdio.StateEntry{Key: key, Value: v}, nil return &stdio.StateEntry{Key: key, Value: v}, nil
} }
func (f *fakeState) route(t *testing.T, key string) *routestate.Route { func (f *fakeState) route(t *testing.T, key string) *contract.Route {
t.Helper() t.Helper()
raw, ok := f.values[key] raw, ok := f.values[key]
if !ok { if !ok {
return nil return nil
} }
var r routestate.Route var r contract.Route
if err := json.Unmarshal(raw, &r); err != nil { if err := json.Unmarshal(raw, &r); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -117,8 +117,8 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
}() }()
_, port, _ := net.SplitHostPort(gone) _, port, _ := net.SplitHostPort(gone)
contributions := []contribution{ contributions := []contribution{
{From: "grafana", Node: "anchor", At: "127.0.0.1", Values: map[string]any{"name": "grafana.example", "port": mustFloat(t, port)}}, {From: "grafana", Node: "anchor", At: "127.0.0.1", Route: contract.Request{Name: "grafana.example", Port: int(mustFloat(t, port))}},
{From: "drive", Node: "home-server", At: "10.0.0.2", Values: map[string]any{"internal-name": "drive.internal", "port": 8080.0}}, {From: "drive", Node: "home-server", At: "10.0.0.2", Route: contract.Request{InternalName: "drive.internal", Port: 8080}},
} }
for _, row := range []struct { for _, row := range []struct {
@@ -130,12 +130,12 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
func(t *testing.T, held *table) {}, func(t *testing.T, held *table) {},
func(t *testing.T, routes, _ *fakeState) { func(t *testing.T, routes, _ *fakeState) {
r := routes.route(t, "anchor.grafana.example") r := routes.route(t, "anchor.grafana.example")
if r == nil || r.Proxy != "anchor" || r.Reach != routestate.ReachPublic || len(r.Targets) != 1 || if r == nil || r.Proxy != "anchor" || r.Reach != contract.ReachPublic || len(r.Targets) != 1 ||
r.Targets[0].Module != "grafana" || r.Targets[0].Node != "anchor" || r.Targets[0].Target != "http://127.0.0.1:"+port { r.Targets[0].Module != "grafana" || r.Targets[0].Node != "anchor" || r.Targets[0].Target != "http://127.0.0.1:"+port {
t.Fatalf("grafana's route is not on the bus as served: %+v", r) t.Fatalf("grafana's route is not on the bus as served: %+v", r)
} }
if d := routes.route(t, "anchor.drive.internal"); d == nil || d.Reach != routestate.ReachInternal || if d := routes.route(t, "anchor.drive.internal"); d == nil || d.Reach != contract.ReachInternal ||
d.Certificate.Authority != routestate.ReachInternal { d.Certificate.Authority != contract.ReachInternal {
t.Fatalf("an internal-only name is not on the bus as internal: %+v", d) t.Fatalf("an internal-only name is not on the bus as internal: %+v", d)
} }
}}, }},
@@ -172,7 +172,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
req.RemoteAddr = "192.0.2.7:51000" req.RemoteAddr = "192.0.2.7:51000"
handler(held).ServeHTTP(httptest.NewRecorder(), req) handler(held).ServeHTTP(httptest.NewRecorder(), req)
held.ledger.flush() held.ledger.flush()
var a routestate.Unrouted var a contract.Unrouted
raw := held.ledger.unrouted.(*fakeState).values["anchor.wiki.grafana.example"] raw := held.ledger.unrouted.(*fakeState).values["anchor.wiki.grafana.example"]
if err := json.Unmarshal(raw, &a); err != nil || a.Times != 1 || a.AskedAt.IsZero() { if err := json.Unmarshal(raw, &a); err != nil || a.Times != 1 || a.AskedAt.IsZero() {
t.Fatalf("the refused name is not on the bus as asked: %s", raw) t.Fatalf("the refused name is not on the bus as asked: %s", raw)
@@ -181,7 +181,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
t.Fatalf("who asked is on the bus: %s", raw) t.Fatalf("who asked is on the bus: %s", raw)
} }
routes, public := routesOf(append(contributions, contribution{From: "grafana", Node: "anchor", routes, public := routesOf(append(contributions, contribution{From: "grafana", Node: "anchor",
Values: map[string]any{"name": "wiki.grafana.example", "port": 3000.0}})) Route: contract.Request{Name: "wiki.grafana.example", Port: 3000}}))
held.set(routes, public) held.set(routes, public)
}, },
func(t *testing.T, _, unrouted *fakeState) { func(t *testing.T, _, unrouted *fakeState) {
@@ -197,7 +197,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
_, _ = pick(&tls.ClientHelloInfo{ServerName: "forgotten.grafana.example"}) _, _ = pick(&tls.ClientHelloInfo{ServerName: "forgotten.grafana.example"})
}, },
func(t *testing.T, _, unrouted *fakeState) { func(t *testing.T, _, unrouted *fakeState) {
var a routestate.Unrouted var a contract.Unrouted
if err := json.Unmarshal(unrouted.values["anchor.forgotten.grafana.example"], &a); err != nil || !a.Handshake { if err := json.Unmarshal(unrouted.values["anchor.forgotten.grafana.example"], &a); err != nil || !a.Handshake {
t.Fatalf("a handshake refused for its name is not on the bus: %v", unrouted.values) t.Fatalf("a handshake refused for its name is not on the bus: %v", unrouted.values)
} }
@@ -232,7 +232,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
held.ledger.refused("w.scan.test", false, false) held.ledger.refused("w.scan.test", false, false)
}, },
func(t *testing.T, _, unrouted *fakeState) { func(t *testing.T, _, unrouted *fakeState) {
var o routestate.Other var o contract.Other
if err := json.Unmarshal(unrouted.values["anchor._other"], &o); err != nil || o.LastMinute != 2 || o.SinceStart != 78 { if err := json.Unmarshal(unrouted.values["anchor._other"], &o); err != nil || o.LastMinute != 2 || o.SinceStart != 78 {
t.Fatalf("the refusals not kept by name are not counted on the bus: %s", unrouted.values["anchor._other"]) t.Fatalf("the refusals not kept by name are not counted on the bus: %s", unrouted.values["anchor._other"])
} }
@@ -247,7 +247,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
}, },
func(t *testing.T, routes, _ *fakeState) { func(t *testing.T, routes, _ *fakeState) {
c := routes.route(t, "anchor.grafana.example").Certificate c := routes.route(t, "anchor.grafana.example").Certificate
if c.Outcome != routestate.Issued || c.Issuer != "Let's Encrypt R11" || c.NotAfter == nil || if c.Outcome != contract.Issued || c.Issuer != "Let's Encrypt R11" || c.NotAfter == nil ||
!c.NotAfter.Equal(notAfter) || c.RenewedAt == nil { !c.NotAfter.Equal(notAfter) || c.RenewedAt == nil {
t.Fatalf("the issued certificate is not on the bus: %+v", c) t.Fatalf("the issued certificate is not on the bus: %+v", c)
} }
@@ -261,7 +261,7 @@ func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
}, },
func(t *testing.T, routes, _ *fakeState) { func(t *testing.T, routes, _ *fakeState) {
c := routes.route(t, "anchor.grafana.example").Certificate c := routes.route(t, "anchor.grafana.example").Certificate
if c.Outcome != routestate.Failed || c.Error != "acme: urn:ietf:params:acme:error:rateLimited" { if c.Outcome != contract.Failed || c.Error != "acme: urn:ietf:params:acme:error:rateLimited" {
t.Fatalf("the failed certificate is not on the bus: %+v", c) t.Fatalf("the failed certificate is not on the bus: %+v", c)
} }
}}, }},
+41 -131
View File
@@ -64,6 +64,8 @@ import (
"golang.org/x/crypto/acme" "golang.org/x/crypto/acme"
"golang.org/x/crypto/acme/autocert" "golang.org/x/crypto/acme/autocert"
"golang.org/x/crypto/bcrypt" "golang.org/x/crypto/bcrypt"
"git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// Where public certificates come from when nothing says otherwise. // Where public certificates come from when nothing says otherwise.
@@ -116,15 +118,16 @@ func onlyInternalNamesTheMeshSaid(held *table) autocert.HostPolicy {
} }
} }
// contribution is one route a consumer asked for: the module, its machine, where that machine is, and what it // contribution is one route a consumer asked for: the module, its machine, where that machine is, the local name of
// contributes, composed by the mesh (its plan's entry for the route, plans.go). // its contribution, and the route, composed by the mesh and read with route-proxy's contract (plans.go).
type contribution struct { type contribution struct {
From string `json:"from"` From string `json:"from"`
Node string `json:"node"` Node string `json:"node"`
// At is where that machine is on the private network. The mesh knows it; a proxy that had to // At is where that machine is on the private network. The mesh knows it; a proxy that had to
// build it from the node name would be a naming convention copied into every provider. // build it from the node name would be a naming convention copied into every provider.
At string `json:"at"` At string `json:"at"`
Values map[string]any `json:"values"` Local string `json:"local,omitempty"`
Route contract.Request `json:"route"`
} }
// policy is what a rule does with a request that matched it. // policy is what a rule does with a request that matched it.
@@ -407,6 +410,24 @@ func (t *table) namesSeenFrom(remote string) []string {
return out return out
} }
// served is every name served, as an event says it: how far it reaches and the module whose rule serves it first.
func (t *table) served() map[string]contract.Event {
t.mu.RLock()
defer t.mu.RUnlock()
out := make(map[string]contract.Event, len(t.to))
for name, rules := range t.to {
e := contract.Event{Name: name, Reach: contract.ReachInternal}
if t.public[name] {
e.Reach = contract.ReachPublic
}
if len(rules) > 0 {
e.Module, e.Node = rules[0].from, rules[0].node
}
out[name] = e
}
return out
}
func (t *table) names() []string { func (t *table) names() []string {
t.mu.RLock() t.mu.RLock()
defer t.mu.RUnlock() defer t.mu.RUnlock()
@@ -857,38 +878,15 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
out := map[string][]rule{} out := map[string][]rule{}
public := map[string]bool{} public := map[string]bool{}
for _, c := range contributions { for _, c := range contributions {
name, _ := c.Values["name"].(string) // The contract admitted the route whole (contract.Request.Validate): it names a host, a rule that forwards
name = strings.TrimSpace(name) // gives a port, its scheme is http or https, its body limit is whole and positive, and its auth names a
internal, _ := c.Values["internal-name"].(string) // secret's file rather than holding a credential (novox/hq ADR 0108).
internal = strings.TrimSpace(internal) r := c.Route
if name == "" && internal == "" { name := strings.TrimSpace(r.Name)
log.Printf("%s on %s asked for a route and named nothing; skipped", c.From, c.Node) internal := strings.TrimSpace(r.InternalName)
continue made := rule{path: r.Prefix(), priority: r.Priority}
} made.policy.deny, made.policy.redirectTo = r.Deny, r.Redirect
// What the route is called in a log line: its public name when it has one. if named := strings.TrimSpace(r.Auth); named != "" {
called := name
if called == "" {
called = internal
}
made := rule{path: asPath(c.Values["path"])}
if p, ok := asWhole(c.Values["priority"]); ok {
made.priority = p
}
made.policy.deny, _ = c.Values["deny"].(bool)
made.policy.redirectTo, _ = c.Values["redirect"].(string)
if named, carried := c.Values["auth"].(string); carried && strings.TrimSpace(named) != "" {
// **A declaration names a secret; it never holds one** (ADR 0108). Refused rather than
// tolerated, and the whole rule is dropped rather than served unprotected — the
// rejected option cannot come back by accident, which is the failure this check exists
// to make impossible.
if looksLikeACredential(named) {
log.Printf("%s on %s declared route %q with a credential in the declaration rather "+
"than the name of a secret; the whole route is refused (novox/hq ADR 0108)",
c.From, c.Node, called)
continue
}
users, err := usersFrom(named) users, err := usersFrom(named)
if err != nil { if err != nil {
// Fail closed: the rule is kept so the name stays routed and answers, and it // Fail closed: the rule is kept so the name stays routed and answers, and it
@@ -900,13 +898,7 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
} }
// Only a rule that actually proxies needs somewhere to send the request. // Only a rule that actually proxies needs somewhere to send the request.
if !made.policy.deny && made.policy.redirectTo == "" { if r.Forwards() {
port, ok := asPort(c.Values["port"])
if !ok {
log.Printf("%s on %s asked for route %q and gave no usable port; skipped",
c.From, c.Node, called)
continue
}
// Where the mesh says that machine is. Empty means it is this one — a workload beside // Where the mesh says that machine is. Empty means it is this one — a workload beside
// the proxy is ordinary, and reaching it over loopback is both correct and the only // the proxy is ordinary, and reaching it over loopback is both correct and the only
// thing that works when there is no private network. // thing that works when there is no private network.
@@ -916,32 +908,12 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
} }
// http unless the contribution says otherwise. A backend that terminates its own TLS // http unless the contribution says otherwise. A backend that terminates its own TLS
// with a certificate this proxy has no reason to trust — Mailu's webmail front is the // with a certificate this proxy has no reason to trust — Mailu's webmail front is the
// first of these — is the reason `insecure` exists, and it stays the exception: every // first of these — is the reason `insecure` exists, and it stays the exception.
// other target the mesh hands this proxy is a plain workload on the private network. made.insecure = r.Insecure
scheme, _ := c.Values["scheme"].(string) if r.MaxRequestBody != nil {
scheme = strings.ToLower(strings.TrimSpace(scheme)) made.maxRequestBody = *r.MaxRequestBody
if scheme == "" {
scheme = "http"
} }
if scheme != "http" && scheme != "https" { made.target = fmt.Sprintf("%s://%s:%d", r.SchemeOrDefault(), at, r.Port)
log.Printf("%s on %s asked for route %q with scheme %q, which is neither http "+
"nor https; skipped", c.From, c.Node, called, scheme)
continue
}
made.insecure, _ = c.Values["insecure"].(bool)
// A limit this proxy cannot read is a route it does not serve, named like a port that
// is not a port. Serving it without the limit would carry exactly what the module said
// not to carry, and report success doing it.
if asked, said := c.Values["max-request-body"]; said {
bytes, whole := asWhole(asked)
if !whole || bytes <= 0 {
log.Printf("%s on %s asked for route %q with a max-request-body of %v, which is "+
"not a whole positive number of bytes; skipped", c.From, c.Node, called, asked)
continue
}
made.maxRequestBody = int64(bytes)
}
made.target = fmt.Sprintf("%s://%s:%d", scheme, at, port)
} }
made.from, made.node = c.From, c.Node made.from, made.node = c.From, c.Node
@@ -955,9 +927,8 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
// answered both for one route, as a convenience (reaching a service over the VPN without a // answered both for one route, as a convenience (reaching a service over the VPN without a
// public TLS round trip), not as an access boundary; composing it here restores exactly // public TLS round trip), not as an access boundary; composing it here restores exactly
// that, nothing more. Absent whenever the node composed no internal name (novox/hq ADR // that, nothing more. Absent whenever the node composed no internal name (novox/hq ADR
// 0056's internalDomain half) — the same "nothing to join a label to" case the public name // 0056's internalDomain half). And the only name, when the endpoint reaches no further than
// already has. And the only name, when the endpoint reaches no further than the private // the private network.
// network.
if internal != "" { if internal != "" {
out[strings.ToLower(internal)] = append(out[strings.ToLower(internal)], made) out[strings.ToLower(internal)] = append(out[strings.ToLower(internal)], made)
} }
@@ -965,50 +936,6 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
return out, public return out, public
} }
// asWhole is any whole number the mesh wrote, whatever its magnitude.
//
// **Not asPort.** Priority was read with the port reader first, which caps at 65535 — so a rule
// declared at a priority above that silently became priority 0 and stopped shadowing the route it
// exists to shadow. The one real rule this has to reproduce is declared at 100000, so the bug was
// exactly load-bearing. A priority is an ordering, not a port: it has no range.
func asWhole(v any) (int, bool) {
switch n := v.(type) {
case float64:
// JSON makes a float of every number, so a non-integral one was not meant as a priority.
if n != float64(int(n)) {
return 0, false
}
return int(n), true
case int:
return n, true
}
return 0, false
}
// asPath is the path prefix a rule is scoped to, or "" for every path.
func asPath(v any) string {
p, _ := v.(string)
p = strings.TrimSpace(p)
if p == "" {
return ""
}
if !strings.HasPrefix(p, "/") {
p = "/" + p
}
return p
}
// looksLikeACredential is the check that keeps a secret out of a declaration.
//
// It errs towards refusing: a value holding a `:` (the htpasswd separator) or opening with a bcrypt
// identifier is a credential, not a path, and no filesystem path the mesh writes needs either. A
// false refusal is a loud log and a route that does not serve; a false accept is a credential
// committed to a declaration, which is the thing being prevented.
func looksLikeACredential(v string) bool {
v = strings.TrimSpace(v)
return strings.Contains(v, ":") || strings.HasPrefix(v, "$2")
}
// usersFrom reads the credentials the mesh mounted, in the one format every htpasswd already is. // usersFrom reads the credentials the mesh mounted, in the one format every htpasswd already is.
func usersFrom(path string) (map[string]string, error) { func usersFrom(path string) (map[string]string, error) {
raw, err := os.ReadFile(path) raw, err := os.ReadFile(path)
@@ -1033,23 +960,6 @@ func usersFrom(path string) (map[string]string, error) {
return users, nil return users, nil
} }
// asPort accepts what JSON makes of a number, which is a float even when it was written 8080.
func asPort(v any) (int, bool) {
switch n := v.(type) {
case float64:
if n < 1 || n > 65535 {
return 0, false
}
return int(n), true
case int:
if n < 1 || n > 65535 {
return 0, false
}
return n, true
}
return 0, false
}
// towards proxies to one backend and tells it what the client asked: **X-Forwarded-Proto, -Host and // towards proxies to one backend and tells it what the client asked: **X-Forwarded-Proto, -Host and
// -For**, set from the request this proxy received. A backend that builds its own addresses — a forge // -For**, set from the request this proxy received. A backend that builds its own addresses — a forge
// writing its clone URL into a page, a login redirect — otherwise sees the plain HTTP hop from this // writing its clone URL into a page, a login redirect — otherwise sees the plain HTTP hop from this
+54 -30
View File
@@ -3,12 +3,13 @@ package main
// The routes this proxy serves, read from the seat mesh-provisioner's plans and nothing else (issue 774; research 072 // The routes this proxy serves, read from the seat mesh-provisioner's plans and nothing else (issue 774; research 072
// row 34, the operator on 2026-10-31: "there is only 1 path, the other is simply deprecated/invalid"). // row 34, the operator on 2026-10-31: "there is only 1 path, the other is simply deprecated/invalid").
// //
// **A route is a provision.** A module that must be reachable requires `route`; the controller composes its plan on // **A route is a provision, in route-proxy's own contract.** A module that must be reachable requires `route`; the
// its node, and the plan's entry for the route names this proxy as its provider and carries what the module // controller composes its plan on its node, and the plan's entry for the route names this proxy as its provider and
// contributes, composed as the received file used to carry it (its names joined to the node's domains, its endpoint's // carries what the module contributes, composed (its names joined to the node's domains, the machine port its
// port, its policy), and where the module's machine is on the private network. The proxy follows the plans on the bus // placement assigned its endpoint, never the software's own), and where the module's machine is on the private
// network, as one opaque payload the seat never reads (mesh-sdk catalogue.PlanProvision.Contributed). The proxy decodes each contribution with its contract (contract.DecodeRequest)
// and serves every entry naming it. There is no file and no second source: a route the plans do not name is not // and serves every entry naming it. There is no file and no second source: a route the plans do not name is not
// served. // served, and a route the contract refuses is said refused (contract.EventRefused), never served half.
import ( import (
"fmt" "fmt"
@@ -19,23 +20,28 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/provisions" "git.novox.be/novox/mesh-sdk/go/provisions"
"route-proxy/internal/routestate" "git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// planned is the routes the seat's plans give this proxy on its node, by plan key, kept so a plan that cannot be read // planned is the routes the seat's plans give this proxy on its node, by plan key, kept so a plan that cannot be read
// leaves what it gave served (a bind missed would read as a route withdrawn). // leaves what it gave served (a bind missed would read as a route withdrawn).
type planned struct { type planned struct {
node string node string
// said is where a refusal is told: the bus's events, nothing in a test that does not ask.
said func(eventType string, e contract.Event)
mu sync.Mutex mu sync.Mutex
by map[string][]contribution by map[string][]contribution
} }
func newPlanned(node string) *planned { func newPlanned(node string, said func(string, contract.Event)) *planned {
return &planned{node: node, by: map[string][]contribution{}} if said == nil {
said = func(string, contract.Event) {}
}
return &planned{node: node, said: said, by: map[string][]contribution{}}
} }
// take is one change of a plan's key: its value, or nil when the key was deleted. It answers whether what is served // take is one change of a plan's key: its value, or nil when the key was deleted. It answers whether what is served
// changed, and says on the log what it could not serve. // changed, and says on the log and the bus what it could not serve.
func (p *planned) take(key string, value []byte) bool { func (p *planned) take(key string, value []byte) bool {
p.mu.Lock() p.mu.Lock()
defer p.mu.Unlock() defer p.mu.Unlock()
@@ -44,13 +50,14 @@ func (p *planned) take(key string, value []byte) bool {
delete(p.by, key) delete(p.by, key)
return had return had
} }
gave, problems, err := routesOfPlan(p.node, key, value) gave, refused, err := routesOfPlan(p.node, key, value)
if err != nil { if err != nil {
log.Printf("the plan %s is not readable, so what it routes is kept as it was: %v", key, err) log.Printf("the plan %s is not readable, so what it routes is kept as it was: %v", key, err)
return false return false
} }
for _, problem := range problems { for _, r := range refused {
log.Print(problem) log.Printf("the plan %s gives %s on %s a route this proxy does not serve: %s", key, r.Module, r.Node, r.Why)
p.said(contract.EventRefused, r)
} }
if len(gave) == 0 { if len(gave) == 0 {
_, had := p.by[key] _, had := p.by[key]
@@ -77,12 +84,12 @@ func (p *planned) contributions() []contribution {
return out return out
} }
// routesOfPlan is the routes one plan gives the proxy on node: each contribution of each entry naming this proxy on // routesOfPlan is the routes one plan gives the proxy on node: each contribution the plan's entries naming this proxy
// node as the route's provider, reached on its own machine over loopback and on another at the address the plan says. // on node as the route's provider carry, decoded with the contract, reached on its own machine over loopback and on
// A plan of another module, or one routing elsewhere, gives none. Problems are what the plan names and the proxy // another at the address the plan says. A plan of another module, or one routing elsewhere, gives none. Refused is
// cannot serve, said rather than skipped silently: an entry carrying nothing to serve (a controller before the plans // what the plan names and the proxy does not serve, each with why: an entry carrying nothing to serve, a payload or a
// carried it), or a consumer elsewhere with no address on the private network. // route the contract does not read, or a consumer elsewhere with no address on the private network.
func routesOfPlan(node, key string, value []byte) ([]contribution, []string, error) { func routesOfPlan(node, key string, value []byte) ([]contribution, []contract.Event, error) {
plan, err := provisions.DecodePlan(value) plan, err := provisions.DecodePlan(value)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
@@ -91,33 +98,50 @@ func routesOfPlan(node, key string, value []byte) ([]contribution, []string, err
return nil, nil, fmt.Errorf("it is the plan of %s.%s, under the key %s", plan.Node, plan.Module, key) return nil, nil, fmt.Errorf("it is the plan of %s.%s, under the key %s", plan.Node, plan.Module, key)
} }
var out []contribution var out []contribution
var problems []string var refused []contract.Event
refuse := func(local, why string) {
refused = append(refused, contract.Event{Proxy: node, Module: plan.Module, Node: plan.Node, Local: local, Why: why})
}
named, carried := false, false
for _, e := range plan.Plan.Provisions { for _, e := range plan.Plan.Provisions {
if !routesHere(node, e) { if !routesHere(node, e) {
continue continue
} }
if len(e.Given) == 0 { named = true
problems = append(problems, fmt.Sprintf("the plan %s names this proxy for its route and gives nothing "+ // What the module contributes is on the requirement's first entry alone, however many local names it has.
"to serve; it is not served", key)) if len(e.Contributed) == 0 {
continue
}
carried = true
given, err := catalogue.DecodeContributed(e.Contributed)
if err != nil {
refuse("", err.Error())
continue continue
} }
at := "" at := ""
if plan.Node != node { if plan.Node != node {
if e.ConsumerAt == "" { if given.At == "" {
problems = append(problems, fmt.Sprintf("the plan %s routes %s on %s here and says no address on "+ refuse("", fmt.Sprintf("it runs on %s and its plan says no address on the private network for it", plan.Node))
"the private network for it; it is not served", key, plan.Module, plan.Node))
continue continue
} }
at = e.ConsumerAt at = given.At
} }
for _, g := range e.Given { for _, one := range given.Each {
out = append(out, contribution{From: plan.Module, Node: plan.Node, At: at, Values: g.Values}) route, err := contract.DecodeRequest(one.Values)
if err != nil {
refuse(one.Local, err.Error())
continue
}
out = append(out, contribution{From: plan.Module, Node: plan.Node, At: at, Local: one.Local, Route: route})
} }
} }
return out, problems, nil if named && !carried {
refuse("", "its plan names this proxy for its route and carries nothing to serve")
}
return out, refused, nil
} }
// routesHere is whether a plan's entry is a route this proxy on node provides. // routesHere is whether a plan's entry is a route this proxy on node provides.
func routesHere(node string, e catalogue.PlanProvision) bool { func routesHere(node string, e catalogue.PlanProvision) bool {
return e.Provision == routestate.Provision && e.Provider == node && e.ProviderModule == routestate.Module return e.Provision == contract.Provision && e.Provider == node && e.ProviderModule == contract.Module
} }
@@ -10,11 +10,15 @@ import (
"net/http/httptest" "net/http/httptest"
"net/url" "net/url"
"reflect" "reflect"
"slices"
"sort"
"strings" "strings"
"testing" "testing"
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/membership" "git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// behind is a workload the proxy can send to, and a table routing one public name and one // behind is a workload the proxy can send to, and a table routing one public name and one
@@ -189,28 +193,42 @@ func plan(node, module, entries string) string {
`"provisions":[%s]}}`, node, module, entries) `"provisions":[%s]}}`, node, module, entries)
} }
// contributed is a plan entry's opaque payload: the consumer's machine at at, and each contribution by local name.
func contributed(at string, each ...string) string {
var parts []string
for i := 0; i+1 < len(each); i += 2 {
parts = append(parts, fmt.Sprintf(`{"local":%q,"values":%s}`, each[i], each[i+1]))
}
return fmt.Sprintf(`"contributed":{"v":1,"at":%q,"each":[%s]}`, at, strings.Join(parts, ","))
}
// What the plans give is what is served, and nothing else (issue 774, research 072 row 34): each entry naming this // What the plans give is what is served, and nothing else (issue 774, research 072 row 34): each entry naming this
// proxy on its node as the route's provider, every contribution of it, reached over loopback on this machine and at // proxy on its node as the route's provider, every contribution of it read with route-proxy's contract, reached over
// the plan's address on another; an entry routing elsewhere, giving nothing or naming no address is not served. A // loopback on this machine and at the plan's address on another; an entry routing elsewhere, carrying nothing, naming
// change is served as it comes, a plan that cannot be read keeps what it gave, and a deleted one withdraws it. The // no address or a route the contract refuses is not served, and is said refused. A change is served as it comes and
// membership says only who is inside: an internal-only name is served to the machines it names. // said served or withdrawn, a plan that cannot be read keeps what it gave, and a deleted one withdraws it. The
func TestThePlansAreServedAndTheMembershipSaysWhoIsInside(t *testing.T) { // placement says only who is inside: an internal-only name is served to the machines it decides addresses for.
const gitea = `{"provision":"route","provider":"anchor","provider-module":"route-proxy","consumer-at":"10.10.0.7",` + func TestThePlansAreServedAndThePlacementSaysWhoIsInside(t *testing.T) {
`"given":[{"local":"web","values":{"name":"git.example","internal-name":"git.anchor.internal","port":3000}},` + const route = `{"provision":"route","provider":"anchor","provider-module":"route-proxy",`
`{"local":"api","values":{"internal-name":"git-api.anchor.internal","port":3001}}]}` gitea := route + contributed("10.10.0.7",
"web", `{"name":"git.example","internal-name":"git.anchor.internal","port":3000}`,
"api", `{"internal-name":"git-api.anchor.internal","port":3001}`,
"refused", `{"name":"bad.example","port":3002,"host":"x"}`) + `}`
plans := &fakePlans{current: map[string]string{ plans := &fakePlans{current: map[string]string{
"laptop.gitea": plan("laptop", "gitea", gitea+`,{"provision":"postgres-database","provider":"anchor",`+ "laptop.gitea": plan("laptop", "gitea", gitea+`,{"provision":"postgres-database","provider":"anchor",`+
`"provider-module":"postgres","given":[{"values":{"name":"gitea"}}]}`), `"provider-module":"postgres",`+contributed("10.10.0.7", "", `{"name":"gitea"}`)+`}`),
"anchor.blog": plan("anchor", "blog", `{"provision":"route","provider":"anchor","provider-module":"route-proxy",`+ "anchor.blog": plan("anchor", "blog", route+contributed("10.10.0.1", "", `{"name":"blog.example","port":2368}`)+`}`),
`"consumer-at":"10.10.0.1","given":[{"values":{"name":"blog.example","port":2368}}]}`),
"anchor.other": plan("anchor", "other", `{"provision":"route","provider":"edge","provider-module":"route-proxy",`+ "anchor.other": plan("anchor", "other", `{"provision":"route","provider":"edge","provider-module":"route-proxy",`+
`"given":[{"values":{"name":"other.example","port":80}}]}`), contributed("", "", `{"name":"other.example","port":80}`)+`}`),
"anchor.older": plan("anchor", "older", `{"provision":"route","provider":"anchor","provider-module":"route-proxy"}`), "anchor.older": plan("anchor", "older", route+`"local":""}`),
"laptop.lost": plan("laptop", "lost", `{"provision":"route","provider":"anchor","provider-module":"route-proxy",`+ "laptop.lost": plan("laptop", "lost", route+contributed("", "", `{"name":"lost.example","port":80}`)+`}`),
`"given":[{"values":{"name":"lost.example","port":80}}]}`),
}} }}
held := newTable() held := newTable()
if err := followPlans(plans, newPlanned("anchor"), held); err != nil { var said []string
say := func(kind string, e contract.Event) {
said = append(said, kind+" "+e.Proxy+" "+e.Name+" "+e.Module+" "+e.Local+" "+e.Why)
}
if err := followPlans(plans, newPlanned("anchor", say), held, say); err != nil {
t.Fatal(err) t.Fatal(err)
} }
served := func() map[string]string { served := func() map[string]string {
@@ -230,17 +248,36 @@ func TestThePlansAreServedAndTheMembershipSaysWhoIsInside(t *testing.T) {
if !held.eligibleForACME("git.example") || held.eligibleForACME("git-api.anchor.internal") { if !held.eligibleForACME("git.example") || held.eligibleForACME("git-api.anchor.internal") {
t.Error("a public name is not certified publicly, or an internal-only one is") t.Error("a public name is not certified publicly, or an internal-only one is")
} }
// Each refusal said with why, and each name served said once.
sort.Strings(said)
for _, w := range []string{
"route.refused anchor gitea refused not a route this contract reads: json: unknown field \"host\"",
"route.refused anchor lost it runs on laptop and its plan says no address on the private network for it",
"route.refused anchor older its plan names this proxy for its route and carries nothing to serve",
"route.served anchor blog.example blog ",
"route.served anchor git-api.anchor.internal gitea ",
} {
if !slices.Contains(said, w) {
t.Errorf("not said %q, said:\n%s", w, strings.Join(said, "\n"))
}
}
if len(said) != 7 {
t.Errorf("said %d events, want 3 refusals and 4 names served:\n%s", len(said), strings.Join(said, "\n"))
}
for _, row := range []struct { for _, row := range []struct {
name, key, value string name, key, value string
served []string served []string
withdrawn []string withdrawn []string
say []string
}{ }{
{"a route arrives", "anchor.new", plan("anchor", "new", `{"provision":"route","provider":"anchor",`+ {"a route arrives", "anchor.new", plan("anchor", "new", route+contributed("", "", `{"name":"new.example","port":81}`)+`}`),
`"provider-module":"route-proxy","given":[{"values":{"name":"new.example","port":81}}]}`), []string{"new.example"}, nil, []string{"route.served anchor new.example new "}},
[]string{"new.example"}, nil}, {"an unreadable plan keeps what it gave", "laptop.gitea", `{"v":1,`, []string{"git.example"}, nil, nil},
{"an unreadable plan keeps what it gave", "laptop.gitea", `{"v":1,`, []string{"git.example"}, nil}, {"a deleted plan withdraws its routes", "laptop.gitea", "", nil, []string{"git.example", "git-api.anchor.internal"},
{"a deleted plan withdraws its routes", "laptop.gitea", "", nil, []string{"git.example", "git-api.anchor.internal"}}, []string{"route.withdrawn anchor git-api.anchor.internal ", "route.withdrawn anchor git.anchor.internal ",
"route.withdrawn anchor git.example "}},
} { } {
said = nil
change := stdio.StateChange{Key: row.key, Op: "put", Value: json.RawMessage(row.value)} change := stdio.StateChange{Key: row.key, Op: "put", Value: json.RawMessage(row.value)}
if row.value == "" { if row.value == "" {
change = stdio.StateChange{Key: row.key, Op: "delete"} change = stdio.StateChange{Key: row.key, Op: "delete"}
@@ -259,22 +296,36 @@ func TestThePlansAreServedAndTheMembershipSaysWhoIsInside(t *testing.T) {
t.Errorf("%s: %s is still served", row.name, host) t.Errorf("%s: %s is still served", row.name, host)
} }
} }
if !slices.Equal(said, row.say) {
t.Errorf("%s: said %q, want %q", row.name, said, row.say)
}
} }
held = behind(t) held = behind(t)
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code != http.StatusNotFound { if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code != http.StatusNotFound {
t.Errorf("before a membership, a machine of the mesh was served an internal-only name: %d", code) t.Errorf("before the placement is read, a machine of the mesh was served an internal-only name: %d", code)
} }
if !applyMembership(membership.Membership{Mesh: []string{"10.10.0.7"}}, held) { laptop, err := delivery.PlacementValues(delivery.Placement{Node: "laptop", Decided: delivery.NodeDecided{Address: "10.10.0.7"}})
t.Fatal("a membership naming the mesh was not applied") if err != nil {
t.Fatal(err)
}
placement := &fakePlans{current: map[string]string{}}
for k, v := range laptop {
placement.current[k] = string(v)
}
if err := followInside(placement, held); err != nil {
t.Fatal(err)
} }
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound { if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound {
t.Error("a machine the membership names was refused the internal-only route") t.Error("a machine the placement decides an address for was refused the internal-only route")
} }
if applyMembership(membership.Membership{Mesh: []string{"not-an-address"}}, held) { if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.8:1"); code != http.StatusNotFound {
t.Error("a membership whose mesh cannot be read was applied") t.Error("a machine the placement does not name was served the internal-only route")
}
if err := placement.handle(stdio.StateChange{Key: "laptop.node", Op: "put", Value: json.RawMessage(`{`)}); err != nil {
t.Fatal(err)
} }
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound { if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound {
t.Error("an unreadable membership changed who is inside") t.Error("an unreadable placement key changed who is inside")
} }
} }
@@ -9,18 +9,35 @@ import (
"net/http/httptest" "net/http/httptest"
"strings" "strings"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/provisions"
"git.novox.be/novox/mesh-controller/modules/route-proxy/contract"
) )
// routesFrom is the table a test's routes make, written as `{"given":[<contribution>…]}`: the contributions the plans // routesFrom is the table a test's routes make, written as `{"given":[{"from","node","at","values"}…]}`: the
// give the proxy (plans.go), each as routesOfPlan makes it. // contributions the plans give the proxy, each read with route-proxy's contract as routesOfPlan reads it, and one the
// contract refuses left out, as routesOfPlan leaves it out.
func routesFrom(body string) (map[string][]rule, map[string]bool, error) { func routesFrom(body string) (map[string][]rule, map[string]bool, error) {
var said struct { var said struct {
Given []contribution `json:"given"` Given []struct {
From, Node, At string
Values json.RawMessage
} `json:"given"`
} }
if err := json.Unmarshal([]byte(body), &said); err != nil { if err := json.Unmarshal([]byte(body), &said); err != nil {
return nil, nil, err return nil, nil, err
} }
routes, public := routesOf(said.Given) var given []contribution
for _, g := range said.Given {
route, err := contract.DecodeRequest(g.Values)
if err != nil {
continue
}
given = append(given, contribution{From: g.From, Node: g.Node, At: g.At, Route: route})
}
routes, public := routesOf(given)
return routes, public, nil return routes, public, nil
} }
@@ -137,17 +154,81 @@ func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) {
} }
} }
// A workload beside the proxy is ordinary, and reaching it over loopback is both correct and the // A route is dialled where the machine publishes its module's endpoint, as the module's declaration publishes it (issue
// only thing that works when there is no private network. // 774, research 072 row 34; 2026-10-31: gitea's 3000, published at 20000, and grafana's 3000 on another machine,
func TestAConsumerOnTheProxysOwnMachineIsReachedOverLoopback(t *testing.T) { // published at 20012, answered 502): over loopback beside the proxy, which is the only thing that works without a
routes, _, err := routesFrom(`{"given":[ // private network, and at its machine's address otherwise. Each plan is composed by the SDK as the seat
{"from":"app","node":"anchor","values":{"name":"app.example","port":9000}} // mesh-provisioner composes it, its ports as compose's portsFor gives them (the placement's for a port the mesh may
]}`) // move, the manifest's own otherwise), and read as this proxy reads it: the live shapes the old proxy served alike.
func TestAComposedPlanIsDialledWhereTheMachinePublishesIt(t *testing.T) {
proxy := catalogue.Manifest{Module: "route-proxy", Version: "1",
Provides: []catalogue.Offer{{Name: "route", Scope: catalogue.ScopeMesh, AnsweredBy: catalogue.AnsweredByHelper}}}
world := catalogue.World{Offered: map[string][]catalogue.Provider{
"route": {{Node: "anchor", At: "anchor.internal", Module: "route-proxy"}}}}
for _, c := range []struct {
name, node, module string
port int // the endpoint's port, as listens says it
published []any // the container's ports; nil for the host's network
placed map[int]int // what mesh-placement holds for the module
target string
}{
{"beside the proxy, moved by the placement (gitea)", "anchor", "gitea", 3000, []any{"3000"}, map[int]int{3000: 20000},
"http://127.0.0.1:20000"},
{"on another machine, moved by the placement (grafana)", "home", "grafana", 3000, []any{"3000"},
map[int]int{3000: 20012}, "http://home.internal:20012"},
{"a bare port the placement holds none for (influxdb)", "home", "influxdb", 8086, []any{"8086"}, nil,
"http://home.internal:8086"},
{"a fixed mapping (mailu 7080:80)", "anchor", "mailu", 7080, []any{"25", "7080:80"}, nil, "http://127.0.0.1:7080"},
{"a fixed mapping the placement names another port of (photos 9102:9000)", "anchor", "photos", 9102,
[]any{"9102:9000"}, map[int]int{9000: 20011}, "http://127.0.0.1:9102"},
{"the host's network (home-assistant)", "anchor", "home-assistant", 8123, nil, nil, "http://127.0.0.1:8123"},
} {
container := map[string]any{"type": "container", "name": c.module, "image": "x"}
if c.published != nil {
container["ports"] = c.published
} else {
container["network"] = "host"
}
module := catalogue.Manifest{Module: c.module, Version: "1", Requires: []string{"route"},
Listens: []catalogue.Listening{{Name: "web", Port: c.port, Protocol: "tcp", From: "mesh"}},
Contributes: map[string]map[string]any{"route": {"label": c.module, "endpoint": "web"}},
Resources: []map[string]any{container}}
node := catalogue.Node{Name: c.node, At: c.node + ".internal", PublicDomain: "example.org",
Capabilities: map[string]bool{"container-runtime": true}}
r, err := catalogue.Resolve(map[string]catalogue.Manifest{module.Module: module, proxy.Module: proxy},
[]string{module.Module}, node, world)
if err != nil { if err != nil {
t.Fatal(err) t.Fatalf("%s: %v", c.name, err)
}
r.Account = "op"
for i, n := range r.Needs {
r.Needs[i].AnsweredByHelper = n.Name == "route" // as the plan sets it (compose's answeredByHelper)
}
ports := map[int]int{}
for _, l := range module.Listens { // compose's portsFor
where, mayAssign := module.MachineSide(l.Port)
if at, known := c.placed[l.Port]; mayAssign && known {
where = at
}
ports[l.Port] = where
}
composed, err := r.Compose(catalogue.Rendering{Hands: true, Provisioner: "mesh-provisioner-client",
ArtifactStore: "store:5000", Planned: true, Ports: map[string]map[int]int{c.module: ports}})
if err != nil {
t.Fatalf("%s: %v", c.name, err)
}
value, err := provisions.EncodePlan(provisions.Plan{Node: c.node, Module: c.module, Plan: composed.Plans[c.module]})
if err != nil {
t.Fatalf("%s: %v", c.name, err)
}
given, refused, err := routesOfPlan("anchor", c.node+"."+c.module, value)
if err != nil || len(refused) != 0 {
t.Fatalf("%s: the plan gives %v, refused %v (%v)", c.name, given, refused, err)
}
routes, _ := routesOf(given)
if got := targetOf(routes, c.module+".example.org"); got != c.target {
t.Errorf("%s: dialled at %q, want %q", c.name, got, c.target)
} }
if targetOf(routes, "app.example") != "http://127.0.0.1:9000" {
t.Fatalf("a workload on this machine was not reachable: %v", routes)
} }
} }
+2 -1
View File
@@ -3,12 +3,13 @@ module route-proxy
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68 git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0 golang.org/x/crypto v0.57.0
) )
require ( require (
git.novox.be/novox/mesh-controller/modules/route-proxy/contract v0.1.0
github.com/klauspost/compress v1.20.0 // indirect github.com/klauspost/compress v1.20.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect github.com/nats-io/nuid v1.0.1 // indirect
+4 -2
View File
@@ -1,5 +1,7 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68 h1:2rMJX0RMIBaN5NDIaEZVwbj5MFism1O3bRayEz+WlOs= git.novox.be/novox/mesh-controller/modules/route-proxy/contract v0.1.0 h1:89jE1oBK+l7gFJjz6wx8mUD3KxrqSDa7pUTlQkzC1do=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY= git.novox.be/novox/mesh-controller/modules/route-proxy/contract v0.1.0/go.mod h1:Yr+Xp89kpMiKH6kA06EU+d9I2CyzrK9yMX7J3pSv0hE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915 h1:x8k7IeMirMTxUo8BH+xUO+MDnHYgU+636jl1NIw1wvs=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261101022206-8af6e7266915/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo= github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -1,158 +0,0 @@
// Package routestate is what the web front keeps on the bus about the names it serves, and about the names it was asked
// for and does not serve (issue 774): the shape both halves of route-proxy agree on. The proxy writes it, on change,
// in its states `routes` and `unrouted`, announced readable; the module's tools read it from there, so the question
// "is this site reachable, and is its certificate good" is answered from the bus rather than with curl.
//
// A key is `<node>.<name>`: the machine whose proxy answers, then the name as asked, lower-cased. A node's name holds
// no dot, so the first dot splits them.
package routestate
import (
"net"
"regexp"
"strings"
"time"
)
// Module is route-proxy, and Provision what it provides: what a plan's entry names when this proxy is a route's provider
// (both halves read the seat mesh-provisioner's plans for it, issue 774).
const (
Module = "route-proxy"
Provision = "route"
)
// The states route-proxy keeps, by their local names (the buckets are route-proxy_<name>).
const (
// RoutesState holds one key per served name: where it forwards, its certificate, the last answer it got.
RoutesState = "routes"
// UnroutedState holds one key per name the proxy answered "no route for" with, and when it was last asked.
UnroutedState = "unrouted"
// BoundState is the provider helper's bucket of consumer entries (mesh-sdk go/provider).
BoundState = "bound"
)
// The words a route's reach and a certificate's authority are said in.
const (
ReachPublic = "public"
ReachInternal = "internal"
)
// The words a certificate's outcome is said in.
const (
// Issued: the authority issued it while this proxy ran, at RenewedAt.
Issued = "issued"
// Cached: this proxy served it from what it kept, and saw no issuance of it since it started.
Cached = "cached"
// Failed: the last attempt to have one failed, Error says why.
Failed = "failed"
)
// Target is one rule a name is served by: what a request matching it reaches, or what it is answered with.
type Target struct {
// Path is the path prefix the rule is scoped to; empty is every path.
Path string `json:"path,omitempty"`
// Module and Node are the consumer that contributed the route and the machine it runs on.
Module string `json:"module,omitempty"`
Node string `json:"node,omitempty"`
// Target is where the request is forwarded, scheme://address:port; empty for a rule that answers itself.
Target string `json:"target,omitempty"`
Deny bool `json:"deny,omitempty"`
Redirect string `json:"redirect,omitempty"`
// Auth says the rule asks for credentials; Sealed says why it cannot check them, and so refuses everything.
Auth bool `json:"auth,omitempty"`
Sealed string `json:"sealed,omitempty"`
}
// Certificate is what this proxy knows of a name's certificate.
type Certificate struct {
// Authority is which issuer certifies the name: public (the ACME directory the module names) or internal (the
// mesh's own).
Authority string `json:"authority"`
Issuer string `json:"issuer,omitempty"`
NotAfter *time.Time `json:"not_after,omitempty"`
// RenewedAt is when an authority last issued it to this proxy.
RenewedAt *time.Time `json:"renewed_at,omitempty"`
// Outcome is issued, cached or failed; empty until a handshake or an issuance said anything.
Outcome string `json:"outcome,omitempty"`
Error string `json:"error,omitempty"`
At *time.Time `json:"at,omitempty"`
}
// Answer is the last answer the proxy got from upstream for a name: the status it carried back, or the error that
// stood in for one (a 502).
type Answer struct {
Status int `json:"status"`
At time.Time `json:"at"`
Error string `json:"error,omitempty"`
Path string `json:"path,omitempty"`
}
// Route is one served name, the value of a key of RoutesState.
type Route struct {
Name string `json:"name"`
// Proxy is the machine whose proxy serves it.
Proxy string `json:"proxy"`
// Reach is public (a route's own name) or internal (only its internal-name, served inside the private network).
Reach string `json:"reach"`
Targets []Target `json:"targets"`
Certificate Certificate `json:"certificate"`
LastAnswer *Answer `json:"last_answer,omitempty"`
}
// Unrouted is one name the proxy answered "no route for", the value of a key of UnroutedState.
type Unrouted struct {
Name string `json:"name"`
Proxy string `json:"proxy"`
// AskedAt is when the name was last asked for. Who asked is not kept here: the journal keeps it, for the
// machine's intrusion prevention.
AskedAt time.Time `json:"asked_at"`
// Times is how often it was asked since the proxy last started.
Times int `json:"times"`
// InternalOnly says the name is served, but only inside the private network, and was asked from outside.
InternalOnly bool `json:"internal_only,omitempty"`
// Handshake says it was asked in a TLS handshake (the name a client sent), not in an HTTP request.
Handshake bool `json:"handshake,omitempty"`
}
// OtherName is the name of the one key per proxy that counts the refusals not kept by name: names under no domain
// the proxy serves (what a scan sends), and names past the limit of new names a minute. Its key is
// `<node>._other`; no host name begins with an underscore.
const OtherName = "_other"
// Other is the value of a proxy's OtherName key.
type Other struct {
Proxy string `json:"proxy"`
// Minute is the last whole minute counted, and LastMinute how many refusals it held.
Minute time.Time `json:"minute"`
LastMinute int `json:"last_minute"`
// SinceStart is every refusal not kept by name since the proxy started.
SinceStart int `json:"since_start"`
}
// Registrable is the domain a name is registered under, as its last two labels; "" for an address or a single
// label. Two labels is wrong for a public suffix of two (co.uk); it only decides which refused names are kept by name.
func Registrable(name string) string {
if net.ParseIP(strings.Trim(name, "[]")) != nil {
return ""
}
labels := strings.Split(name, ".")
if len(labels) < 2 {
return ""
}
return strings.Join(labels[len(labels)-2:], ".")
}
var keyable = regexp.MustCompile(`^[a-z0-9_]([a-z0-9_.-]{0,251}[a-z0-9_])?$`)
// Keyable says whether a name can be a key's name: what a host name or an address is. A name asked for in a request
// is whatever a client sent; one no key could hold is no site anybody runs.
func Keyable(name string) bool { return keyable.MatchString(name) && !strings.Contains(name, "..") }
// Key is the key of name served or asked at node's proxy.
func Key(node, name string) string { return node + "." + name }
// Split is a key's node and name.
func Split(key string) (node, name string) {
node, name, _ = strings.Cut(key, ".")
return node, name
}
+9
View File
@@ -9,6 +9,9 @@
{ {
"name": "route", "name": "route",
"scope": "mesh", "scope": "mesh",
"credential": {
"form": "none"
},
"identity": false, "identity": false,
"answered-by": "helper" "answered-by": "helper"
} }
@@ -245,6 +248,11 @@
"route_proxy_route", "route_proxy_route",
"route_proxy_unrouted" "route_proxy_unrouted"
], ],
"emits": [
"route.served",
"route.withdrawn",
"route.refused"
],
"replaces": { "replaces": {
"route_proxy_routes": [ "route_proxy_routes": [
"curl -I" "curl -I"
@@ -257,6 +265,7 @@
] ]
}, },
"reads": [ "reads": [
"mesh-delivery-assigner.placement",
"mesh-provisioner.plans", "mesh-provisioner.plans",
"mesh-provisioner.provisions" "mesh-provisioner.provisions"
] ]
+1 -1
View File
@@ -9,7 +9,7 @@ a second place to clean besides the catalogue itself.
mesh-catalog f3a57dcc56fb6c198095cafd11d0decff9f720c4 modules/*/module.json, modules/*/files, modules/*/templates, predefined.json mesh-catalog f3a57dcc56fb6c198095cafd11d0decff9f720c4 modules/*/module.json, modules/*/files, modules/*/templates, predefined.json
mesh-host 9d9e3a46c4f5943cfffb413d962ce48b75e2fd39 examples/foundation-first-node.lock mesh-host 9d9e3a46c4f5943cfffb413d962ce48b75e2fd39 examples/foundation-first-node.lock
mesh-sdk 2da0de75c3f0087cff9c2f5ef789ca6dc3c59045 conformance/events/module-event.json mesh-sdk 8af6e72669151bf9b4f7524d38c28e29215514c6 conformance/events/module-event.json
To move them, from this repository's root, with the two repositories checked out beside it: To move them, from this repository's root, with the two repositories checked out beside it: