novox/mesh-controller
3
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

A consumer is a module on a machine, not a machine

Browse Source 
novox/hq 04-ISSUES/022. A credential was keyed by provision, consumer
node and provider node, so "who is asking" was answered by naming a
host. The node this mesh exists to take over runs eight modules against
one database server.

The symptom had two halves and only one was loud. The provider refused,
naming the modules and explaining they would share one credential, which
reads as a decision rather than a limit. The consumer did not refuse: it
resolved cleanly, wrote one module's credential file and left the others
absent — a service that starts and cannot authenticate, with nothing
saying why. That is 021 again on a different axis.

Three modules wanting one database produced one need, carrying whichever
module mentioned it first, because the resolution walk is a work-list
over names. The fan-out now happens in one place, after the walk. The
record path already did this correctly and said why: a consumer here is
a module on a machine. It is the same rule.

Downstream: the secret's key gains the consuming module, the grant file
is named after both halves, needs are matched by provision and module
rather than provision alone, and the provisioners name the role and the
access key after the module. The refusal in ContributionsTo is gone
because there is nothing left to refuse.

Worth stating plainly: without that refusal, gitea's login would have
opened keycloak's database. From the provisioner's side it created
exactly what it was asked to create.

Existing secrets are discarded rather than backfilled. They cannot say
which module they were for, and a secret is remade and delivered to both
ends on the next push — so this costs one rotation and invents nothing.

Also guards the role name against PostgreSQL's 63-byte truncation, which
is a notice rather than an error and would reintroduce exactly this
collision at a length nobody tests.

Three faults injected — the fan-out removed, needs matched by name
alone, the grant file named after the machine — each caught.
This commit is contained in:
jschoubben
2026-09-01 02:40:09 +02:00
parent 1314be5282
commit 0af3ea1acf
14 changed files with 440 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/catalogue/declaration.go
+50 -32
View File
@@ -46,10 +46,14 @@ type OpensPorts interface {
type Grant struct {
// Provision is what was required.
Provision string
// Consumer is the node that will use it, which is also what names the file.
// Consumer is the node that will use it.
Consumer string
// From is the module on that machine which asked, so the provider can name what it creates
// after the thing using it rather than after the machine.
// From is the module on that machine which asked.
//
// **Part of who this credential is for, not a label** (novox/hq 04-ISSUES/022). Together with
// Consumer it names one consumer; the node alone does not, because a machine routinely runs
// several modules wanting the same thing. It is also what the provider names the role or the
// bucket or the client after, so withdrawing one consumer does not take another's away.
From string
// Values are what that module contributed — the name it wants, and anything else the
// provision's own vocabulary defines.
@@ -180,7 +184,12 @@ func (r Resolution) Declaration(with Rendering) ([]map[string]any, error) {
for _, to := range sortedKeys(m.Secrets) {
var found *Needed
for i, n := range r.Needs {
if n.Name == to {
// **This module's need, not the provision's** (novox/hq 04-ISSUES/022). Matching
// on the name alone, every consumer of a provision took whichever credential
// happened to be last in the list — so on a node with two of them, one module
// would be given the other's password and fail to authenticate with a valid
// credential belonging to somebody else.
if n.Name == to && n.For == m.Module {
found = &r.Needs[i]
}
}
@@ -222,9 +231,9 @@ func (r Resolution) Declaration(with Rendering) ([]map[string]any, error) {
continue
}
first = append(first, map[string]any{
"id": GrantID(to, g.Consumer),
"id": GrantID(to, g.Consumer+"."+g.From),
"type": "file",
"path": grantPath(m.Grants[to], g.Consumer),
"path": grantPath(m.Grants[to], g.Consumer, g.From),
"sealed": g.Sealed,
})
}
@@ -232,7 +241,7 @@ func (r Resolution) Declaration(with Rendering) ([]map[string]any, error) {
for _, to := range sortedKeys(m.Binds) {
var found *Needed
for i, n := range r.Needs {
if n.Name == to {
if n.Name == to && n.For == m.Module {
found = &r.Needs[i]
}
}
@@ -389,8 +398,13 @@ type Contribution struct {
//
// Suffixed, so the directory can also hold whatever the module writing it keeps there and so a
// node named like something else in that directory cannot collide with it.
func grantPath(directory, consumer string) string {
return strings.TrimRight(directory, "/") + "/" + consumer + ".secret"
// One file per consumer, and a consumer is a module on a machine (novox/hq 04-ISSUES/022).
//
// Named after both. Named after the machine alone, two modules on one node wrote to one path: the
// second overwrote the first, and the provisioner — reading a directory — saw one consumer where
// there were two.
func grantPath(directory, consumer, module string) string {
return strings.TrimRight(directory, "/") + "/" + consumer + "." + module + ".secret"
}
// contributions collects what every module in this set contributes, by requirement.
@@ -411,7 +425,10 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
if sorted[i].Provision != sorted[j].Provision {
return sorted[i].Provision < sorted[j].Provision
}
return sorted[i].Consumer < sorted[j].Consumer
if sorted[i].Consumer != sorted[j].Consumer {
return sorted[i].Consumer < sorted[j].Consumer
}
return sorted[i].From < sorted[j].From
})
for _, g := range sorted {
if g.From == "" {
@@ -421,7 +438,7 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
}
out[g.Provision] = append(out[g.Provision], Contribution{
From: g.From, Node: g.Consumer, At: g.At, Values: g.Values,
Secret: grantPath(directories[g.Provision], g.Consumer),
Secret: grantPath(directories[g.Provision], g.Consumer, g.From),
})
}
for _, m := range modules {
@@ -495,8 +512,12 @@ func boundFile(n Needed, path string) (map[string]any, error) {
"from": n.From,
"at": where,
"serves": n.Serves,
// **Where the credential is, not what it is.** It stopped being true that the mesh
// cannot issue one when 021 was fixed, and a comment asserting a fact about the mesh that
// has become false is worse than none — somebody reads it and stops looking.
"generated": "by the mesh — do not edit; replaced whenever this changes. " +
"It carries no credential: the mesh has no way to issue one yet",
"The credential is not here: it is sealed, in the file this module's manifest " +
"names under `secrets`",
}, "", " ")
if err != nil {
return nil, err
@@ -507,34 +528,31 @@ func boundFile(n Needed, path string) (map[string]any, error) {
}, nil
}
// ContributionsTo is what this node's set asked of one requirement, settled.
// ContributionsFrom is what one module on this node asked of one requirement, settled.
//
// Exported because a provider's grants are assembled from its consumers' resolutions, one machine
// at a time, and the alternative was for the control plane to reimplement settling.
func (r Resolution) ContributionsTo(requirement string, settings SettingsBy) (
string, map[string]any, error) {
//
// **One module, not one machine** (novox/hq 04-ISSUES/022). This used to take a requirement alone
// and refuse whenever two modules wanted it — correctly, given what it had: the credential was
// keyed by node, so the two would have shared one, and sharing is worse than refusing. But the
// arrangement refused is the ordinary one. A node running eight services against one database is
// not an edge case; it is what a machine looks like. Now each consumer has its own credential and
// there is nothing left to refuse.
func (r Resolution) ContributionsFrom(requirement, module string, settings SettingsBy) (
map[string]any, bool, error) {
all, err := r.contributions(settings, nil, nil)
if err != nil {
return "", nil, err
return nil, false, err
}
given := all[requirement]
if len(given) == 0 {
return "", nil, nil
}
if len(given) > 1 {
// Two modules on one machine wanting the same provision would share one credential, and
// the provider would be told to create one thing under two names. Refused rather than
// resolved by picking, which is the rule everywhere else here.
var who []string
for _, g := range given {
who = append(who, g.From)
for _, g := range all[requirement] {
if g.From == module {
return g.Values, true, nil
}
sort.Strings(who)
return "", nil, fmt.Errorf(
"%s has %d modules asking for %q and they would share one credential: %s",
r.Node, len(given), requirement, strings.Join(who, ", "))
}
return given[0].From, given[0].Values, nil
// Nothing on that machine asks for this any more. Said as "not found" rather than as an
// error: it is how a credential is withdrawn, and the provider removes what nobody asks for.
return nil, false, nil
}
// here is the module on this same machine that answers a requirement, as a binding.