novox/mesh-controller
3
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

The two halves of an object-store edge, as a readable pair

Browse Source 
Manifests for a provider and a consumer, so the contract can be read
rather than only exercised through a lab fixture that stages the grants
by hand.

Checked as a pair rather than separately, because two manifests that
only ever parse alone are two manifests nobody has held against each
other. The test asserts the names match, that each side says where it
wants to be told, and that the consumer contributes the key the
provisioner actually reads.

That last one is the trap worth having a test for: a consumer
contributing "name" — which is exactly what a database consumer
contributes — resolves cleanly, deploys, and then fails on the machine
with "asked for a bucket and did not name it". Nothing in that message
points back at the manifest that caused it. Both mistakes were made
while writing these two files.
This commit is contained in:
jschoubben
2026-08-31 17:52:14 +02:00
parent ed9a30f22d
commit 950ebb9e28
4 changed files with 162 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/modules/README.md
+33
View File
@@ -40,3 +40,36 @@ somewhere — and a static manifest cannot know it.
Neither does, because both name things **the mesh itself named**: the private network's interface
is `mesh0` on every machine, and the address a resolver listens on for the machine's own use is
`127.0.0.54` on every machine. A name the mesh chose is a name a manifest can use.
## Asking for a bucket
`object-store.json` provides one, `photos.json` asks for one. Together they are the whole of an
edge, and they are here as a **pair** because that is the only way to see the halves line up:
| the provider says | the consumer says |
|---|---|
| `provides: s3-bucket` | `requires: s3-bucket` |
| `receives` — where to be told who asked | `contributes: {bucket: photos}` — what it wants |
| `grants` — where their credentials land | `secrets` — where to be given its key |
| `serves` — port, scheme, region | `binds` — where to be told all that |
**The mesh adds the half neither can know**: which machine the provider is on, and where it is on
the private network. Neither manifest names an address, and that is what lets the same pair work
on any mesh.
**`s3-bucket` names the protocol, not the product** ([ADR 0027](../../../hq/02-DECISIONS/)). A
consumer's code is written against the S3 API, and swapping one store for another does not break
it — so the coupling is to S3. A database is the other case: an application is written against
PostgreSQL or against SQL Server, so those provisions name the engine.
**What the pair is checked for.** That the names match, that each side says where it wants to be
told, and that the consumer contributes the key the provisioner actually reads — `bucket`, not
`name`. Contributing `name` (which is what a database consumer contributes) resolves perfectly and
then fails on the machine with *asked for a bucket and did not name it*, which is a long way from
the manifest that caused it.
The provisioner that makes the credential true lives in
[`../objectstore-provisioner`](../objectstore-provisioner), and is proven against a real store in
the lab — including the assertion a database does not need, that **a consumer cannot reach another
consumer's bucket**. One store holds every bucket behind one endpoint, so that isolation is a
policy somebody wrote rather than a boundary the product has.
examples/modules/modules_test.go
+49
View File
@@ -214,3 +214,52 @@ func TestTheResolverDoesNotAskItselfForUpstreams(t *testing.T) {
}
}
}
// The two halves of an object-store edge, as a pair.
//
// A provider and a consumer that only ever appear separately are two manifests nobody has checked
// against each other: the name one provides has to be the name the other requires, and the key a
// consumer contributes has to be the one the provisioner reads. Both were got wrong while writing
// them, and neither would have been caught by parsing either file alone.
func TestTheObjectStoreEdgeFitsTogether(t *testing.T) {
provider := read(t, "object-store.json")
consumer := read(t, "photos.json")
const provision = "s3-bucket"
var provides bool
for _, offer := range provider.Provides {
if offer.Name == provision {
provides = true
}
}
if !provides {
t.Fatalf("the provider does not offer %q", provision)
}
if !strings.Contains(strings.Join(consumer.Requires, ","), provision) {
t.Fatalf("the consumer does not require %q", provision)
}
// Where each side wants to be told. A provider that receives nowhere is a provider the mesh
// writes nothing for, and a provisioner with nothing to read.
if provider.Receives[provision] == "" {
t.Error("the provider says nowhere to write what its consumers asked for")
}
if provider.Grants[provision] == "" {
t.Error("the provider says nowhere to write its consumers' credentials")
}
if consumer.Binds[provision] == "" {
t.Error("the consumer says nowhere to be told where its bucket is")
}
if consumer.Secrets[provision] == "" {
t.Error("the consumer says nowhere to be given its key")
}
// The key the provisioner reads out of `values`. It looks for `bucket`, so a consumer
// contributing `name` — which is what the database one contributes — resolves cleanly and
// then fails on the machine with "asked for a bucket and did not name it".
if _, named := consumer.Contributes[provision]["bucket"]; !named {
t.Errorf("the consumer contributes %v, and the provisioner reads \"bucket\"",
consumer.Contributes[provision])
}
}
examples/modules/object-store.json
+51
View File
@@ -0,0 +1,51 @@
{
"module": "object-store",
"version": "1",
"provides": [{"name": "s3-bucket", "scope": "mesh"}],
"capabilities": ["container-runtime"],
"listens": [
{"port": 9000, "protocol": "tcp", "from": "mesh",
"why": "the S3 endpoint, for modules on any machine that were granted a bucket"}
],
"serves": {
"s3-bucket": {
"port": 9000,
"scheme": "http",
"region": "us-east-1"
}
},
"receives": {"s3-bucket": "/var/lib/objectstore/grants"},
"grants": {"s3-bucket": "/var/lib/objectstore/grants"},
"own-secrets": {"root": "/var/lib/objectstore/root.secret"},
"resources": [
{"id": "state", "type": "directory", "path": "/var/lib/objectstore", "mode": "0700"},
{"id": "grants", "type": "directory", "path": "/var/lib/objectstore/grants", "mode": "0700"},
{"id": "store", "type": "container", "name": "mesh-store",
"image": "minio/minio@sha256:0000000000000000000000000000000000000000000000000000000000000000",
"args": ["server", "/data"],
"env": {"MINIO_ROOT_USER": "meshroot"},
"ports": ["9000:9000"],
"volumes": ["mesh-store-data:/data", "/var/lib/objectstore/root.secret:/run/secrets/root:ro"]},
{"id": "provisioner", "type": "container", "name": "mesh-provision-objectstore",
"image": "mesh-provision-objectstore@sha256:0000000000000000000000000000000000000000000000000000000000000000",
"env": {
"GRANTS": "/var/lib/objectstore/grants",
"MESH_OBJECTSTORE_URL": "http://127.0.0.1:9000",
"MESH_OBJECTSTORE_ROOT_USER": "meshroot",
"MESH_OBJECTSTORE_ROOT_PASSWORD_FILE": "/run/secrets/root"
},
"volumes": [
"/var/lib/objectstore/grants:/var/lib/objectstore/grants:ro",
"/var/lib/objectstore/root.secret:/run/secrets/root:ro"
],
"restart-on": ["grants"]}
]
}
examples/modules/photos.json
+29
View File
@@ -0,0 +1,29 @@
{
"module": "photos",
"version": "1",
"requires": ["s3-bucket"],
"contributes": {
"s3-bucket": {"bucket": "photos"}
},
"binds": {"s3-bucket": "/etc/photos/store.json"},
"secrets": {"s3-bucket": "/etc/photos/store.secret"},
"resources": [
{"id": "config", "type": "directory", "path": "/etc/photos", "mode": "0750"},
{"id": "app", "type": "container", "name": "photos",
"image": "photos@sha256:0000000000000000000000000000000000000000000000000000000000000000",
"env": {
"PHOTOS_STORE": "/etc/photos/store.json",
"PHOTOS_STORE_SECRET_FILE": "/etc/photos/store.secret"
},
"volumes": [
"/etc/photos/store.json:/etc/photos/store.json:ro",
"/etc/photos/store.secret:/etc/photos/store.secret:ro"
],
"restart-on": ["config"]}
]
}