novox/mesh-controller
3
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

The mesh composes the accounts; the module owns its server

Browse Source 
The delivery question, decided. The alternative was a manifest field enumerating
the server's ports, TLS paths and store directory so the controller could write a
whole configuration file. That is wrong: those are properties of the container the
module raises, they live in its image and its mounts, and the controller would
have to be kept in step with a Dockerfile it never sees. So the mesh writes only
what only the mesh knows — who may connect — and the module's own configuration
includes it.

`ComposeAccounts` is that file. A test says what must *not* be in it as plainly as
what must: no port, no tls block, no store_dir. Each of those in the mesh's file
is a value the controller would then own, and the module could no longer change
its own image without the mesh agreeing.

`bus-users` is where a module wants it written, and **asking is not enough to
receive it**: the file holds every user's password hash, so a module that could ask
for it could read every credential on the bus. The claim on `mesh-broker`
authorises it, checked from the manifest alone. A holder with nothing composed is
refused rather than given an empty file, for the reason a certificate is — a bus
with no user list refuses every connection in the mesh and looks like a machine
problem.

Six claims checked against a running server before any of this was committed to,
and two of them changed what got written:

**An absolute include path is resolved relative to the including file's
directory.** `include /etc/nats/accounts.conf` from /etc/nats-server/nats.conf
makes the server look for /etc/nats-server/etc/nats/accounts.conf and refuse to
start. So both files share one directory, and the module declares its own as a
file resource beside the mesh's.

**`verify: true` was refusing every connection in the mesh.** It makes the server
demand a *client* certificate, and nothing in the mesh presents one: a host pins
this server's exact certificate and authenticates with the password the mesh
minted, and so does a module's runtime. Every connection died at the TLS handshake
before any password was looked at, with an error — "client didn't provide a
certificate" — that reads as a fault in the client. Removed. TLS is still
required; verify only decides whether client certificates are checked.

The other four: a user in an included file authenticates, an unknown user is
refused so the include is the whole authority rather than an addition, a publish
outside a grant is refused, and rewriting the mesh's half alone makes a new user
appear — noticed by the module's own watcher, with no signal from outside, and
without dropping the connection the mesh already had. That last one is task 1.2's
payoff, collected.
This commit is contained in:
jschoubben
2026-09-27 02:50:23 +02:00
parent ee1b8ffe24
commit f8ab9f2dcf
6 changed files with 239 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/catalogue/declaration.go
+36
View File
@@ -97,6 +97,13 @@ type Rendering struct {
// compose it a second time.
Suffix string
// BusUsers is the mesh's composed user list, for the module holding `mesh-broker`. Empty on
// every other node, and on this one until the controller has composed it.
//
// **Only the users, never the server's own settings**: those are the module's, in its image and
// its mounts (Manifest.BusUsers).
BusUsers string
// Kept is every operator-sealed secret in the mesh, for a module that `keeps` them. Nil when
// nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport
@@ -359,6 +366,35 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
})
}
}
if m.BusUsers != "" {
// **The claim authorises it, not the field.** This file holds every user's password
// hash, so a module that could ask for it could read every credential on the bus.
// Checked from this manifest alone, which is the cheapest check there is: whether some
// other module also claims the seat is resolution's business elsewhere, and one holder
// mesh-wide is already guaranteed.
if !m.ClaimsSeat("mesh-broker") {
return nil, fmt.Errorf(
"%s asks for the mesh's user list and does not claim mesh-broker. That file "+
"holds every user's password hash, so the seat is what authorises it",
m.Module)
}
if with.BusUsers == "" {
// Asked for and not composed. Refused rather than skipped, for the reason a
// certificate is: a bus with no user list refuses every connection in the mesh, and
// an empty file would look like a configuration problem on the machine.
return nil, fmt.Errorf(
"%s holds mesh-broker and the mesh composed no user list, so the bus would "+
"refuse every connection", m.Module)
}
first = append(first, map[string]any{
"id": BusUsersID(), "type": "file", "path": m.BusUsers,
"content": with.BusUsers,
// Readable by the server and nothing else. Hashes rather than passwords, so this is
// not a set of working credentials — but a list of every user in the mesh is worth
// keeping to the one process that needs it.
"mode": "0600",
})
}
for _, name := range sortedKeys(m.OwnSecrets) {
sealed := with.Needed[m.Module][name]
if sealed == "" {
internal/catalogue/declaration_test.go
+80
View File
@@ -0,0 +1,80 @@
package catalogue
import "testing"
// The mesh's user list reaches the module holding the bus, and nothing else.
//
// Three refusals and one delivery, because each of the refusals would be silent in a different way:
// a module that asked and was given it could read every credential on the bus; a bus given an empty
// file refuses every connection in the mesh and looks like a machine problem; and a bus that never
// asked gets nothing rather than a file it does not read.
func TestTheMeshsUserListGoesOnlyToTheModuleHoldingTheBus(t *testing.T) {
theBus := func() Manifest {
return Manifest{
Module: "nats", Version: "1",
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}},
BusUsers: "/var/lib/nats-module/conf/accounts.conf",
Resources: []map[string]any{},
}
}
on := func(t *testing.T, m Manifest, with Rendering) ([]map[string]any, error) {
t.Helper()
return Resolution{Node: "anchor", Modules: []Manifest{m}}.Declaration(with)
}
t.Run("the holder is given it", func(t *testing.T) {
resources, err := on(t, theBus(), Rendering{BusUsers: "accounts { MESH { users = [] } }"})
if err != nil {
t.Fatal(err)
}
// Prefixed with the module it came from, like every resource: two modules may reasonably
// both call something "config", and without the prefix the second would silently replace
// the first.
var found map[string]any
for _, r := range resources {
if r["id"] == "nats."+BusUsersID() {
found = r
}
}
if found == nil {
t.Fatalf("the bus was given no user list: %+v", resources)
}
if found["path"] != "/var/lib/nats-module/conf/accounts.conf" {
t.Errorf("written to %v rather than where the module asked", found["path"])
}
if found["mode"] != "0600" {
t.Errorf("mode %v: a list of every user in the mesh belongs to the one process that "+
"needs it", found["mode"])
}
})
t.Run("a module that does not claim the seat is refused", func(t *testing.T) {
m := theBus()
m.Claims = nil
if _, err := on(t, m, Rendering{BusUsers: "accounts {}"}); err == nil {
t.Fatal("a module that claims nothing was handed every user's password hash")
}
})
t.Run("the holder with nothing composed is refused", func(t *testing.T) {
if _, err := on(t, theBus(), Rendering{}); err == nil {
t.Fatal("the bus was given an empty user list, so it would refuse every connection in " +
"the mesh and look like a machine problem")
}
})
t.Run("a module that did not ask gets nothing", func(t *testing.T) {
m := theBus()
m.BusUsers = ""
resources, err := on(t, m, Rendering{BusUsers: "accounts {}"})
if err != nil {
t.Fatal(err)
}
for _, r := range resources {
if r["id"] == "nats."+BusUsersID() {
t.Fatal("a module that asked for no user list was given one")
}
}
})
}
internal/catalogue/manifest.go
+30 -1
View File
@@ -423,6 +423,20 @@ type Manifest struct {
// A directory rather than one document for the same reason as above: each value is sealed
// separately and the mesh cannot open any of them to build a list.
Grants map[string]string `json:"grants,omitempty"`
// BusUsers is where this module wants the mesh's user list written, and it is only ever
// answered for the module holding `mesh-broker`.
//
// **The mesh writes who may connect; the module owns everything else about its server**
// (novox/hq design 25 §4, task 1.7). Ports, TLS paths and a store directory live in this
// module's image and its mounts and change when it does, so the module's own configuration
// carries them and includes this file. A controller that wrote the whole configuration would
// have to be kept in step with a Dockerfile it never sees.
//
// **Asking for it is not enough to receive it.** This file holds every user's password hash, so
// a module that could ask for it could read every credential on the bus — and the claim on
// `mesh-broker` is what authorises it, checked from this manifest alone.
BusUsers string `json:"bus-users,omitempty"`
}
// Build says how to produce this module's artifacts from its source.
@@ -641,7 +655,22 @@ type Certificate struct {
// CertificateID and AuthorityID are the resource identities of what the mesh issued.
func CertificateID() string { return "certificate" }
func AuthorityID() string { return "certificate-authority" }
// ClaimsSeat says whether this manifest claims one named seat.
func (m Manifest) ClaimsSeat(seat string) bool {
for _, c := range m.Claims {
if c.Name == seat {
return true
}
}
return false
}
// BusUsersID names the mesh's composed user list, so it is the same resource across every
// declaration and a change to it is an update rather than a second file beside the old one — which
// on a bus reading a directory would be two account lists, and the server would take both.
func BusUsersID() string { return "bus-users" }
func AuthorityID() string { return "certificate-authority" }
// FilteringID names the computed rule set, so it is the same resource across every declaration
// and a change to it is an update rather than an addition beside the old one.