57/F2b: A settings layer no longer replaces what a provider serves (issue 696) #321

Merged
mesh-admin merged 1 commits from pkg/57-f2b-served-keys into feat/57-config-and-provisioning 2026-10-14 05:16:03 +00:00
Contributor

Work package F2b of hq design 57, issue 696, ADR 0348 clause 11. It finishes the part F2 (#316) left: Settle was unchanged there because influxdb on ace relied on it.

Merge order: mesh-catalog#290 (influxdb serves ${setting:org} and ${setting:bucket}) first, then this pull request. Merged before #290 is delivered, grafana's data source on ace would name the literal bucket default in place of the bucket ace's layer sets.

What changed

  • catalogue.Settle no longer lays a layer over served keys. It only fills ${setting:<key>} in served values from the layers, with module defaults lowest (filledFromSettings, shared with overridden). A key a layer sets that the provider serves as a literal changes nothing a consumer is told, and a setting still never adds a served key.
  • overridden keeps its override for contributions; that is not in this package's scope.
  • UnusedSettings no longer counts a literal served key as a place where a setting lands. A served key that asks for ${setting:k} still counts, through settingKeysUsedBy. A layer that sets a literal served key is now reported as reaching nothing, instead of being accepted silently.

Which live layers relied on the override

Only influxdb on ace. I read the settings layer, by key, of every provider assigned on ace, novox, g14 and shanks. That covered the mesh-wide and per-machine layers of distribution, gitea, influxdb, keycloak, mailu, minio, mongodb, mosquitto, mssql, postgres, redis, route-proxy, step-ca and umami. I compared each layer's keys with what the module serves:

  • mailu (domain) and keycloak (issuer) serve ${setting:<same key>}, so the value is the same with or without the override.
  • The other layers hold only ports, expose or endpoints. Settle skipped ports already, and the other two are not served keys.
  • No catalogue module declares a zone, so ZoneOn, which also calls Settle, is unaffected. Zones are ${setting:zone} in the tests.
  • No migration seeds a layer.

In the tests, three fixtures encoded the old override. step-ca serving an empty root that a layer replaced: three co-located tests now use stepCAWithItsRoot, which serves ${setting:root}. A store whose layer replaced the derived bucket: TestASettingComposesWithADerivedValue now serves ${setting:bucket-prefix}${consumer:as:dns}. TestASettingOverridesAServedKeyAndAddsNone asserted the override and is replaced by TestALayerNoLongerReplacesAServedKey. Each keeps its intent: an operator's value reaches what is served where the definition asks for it.

How it is proven

  • TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings: the seed is like ace. influxdb's layer holds org, bucket (made-up values) and an endpoints block, and grafana consumes influxdb-api with its data source file copied verbatim from the catalogue. Every grafana resource composed, including the data source file and the binding file, is compared byte for byte with golden files in testdata/influxdb-served-by-setting/. The golden files were composed by this branch's base (71362c1, Settle still overriding) from influxdb serving the literals. The test covers a machine with no layer as well. It passes before this commit (catalogue step alone) and after it (both steps).
  • TestALayerNoLongerReplacesAServedKey: Settle leaves a literal served key alone and adds no key. UnusedSettings says such a layer reaches nothing. influxdb as it was (literals) under an ace-like layer no longer passes the layer's bucket to grafana, which shows why the catalogue must go first. It fails before this commit.
  • TestInfluxdbsConfigTakesItsOrgAndBucket: influxdb's config.json takes org and bucket from an ace-like layer, and stays {} without a layer, because defaults never reach the file. Note that config.json on ace drops the endpoints key that today's form wrote there; see mesh-catalog#290.

Commands run: gofmt -l cmd internal examples (empty), go vet ./..., go test ./.... MESH_TEST_POSTGRES=postgres://…@127.0.0.1:<port>/postgres sh merge-check.sh against a throwaway PostgreSQL exits 0, and every listed store test passes, both at the first base and rebased on dc36536 (after V1, #317).

Left for other packages

  • overridden still lets a layer replace a contribution's declared keys. That is not served, and was not in this package.
  • K6 (issue 665) serves ${config:}. K5 (issue 664) moves ace's layer into mesh-config.

Reviewer's checklist

  1. Does what issue 696's step 2 says, nothing outside it: yes. The UnusedSettings change follows from it, because a setting that lands nowhere is said so.
  2. Done-when proof: the tests above. TestALayerNoLongerReplacesAServedKey fails before this commit. The three changed fixtures are justified above.
  3. Byte-identical composition: TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings against golden files from 71362c1.
  4. No secret anywhere, and the test values are made up; ace's values are not in the change. No schema change.
  5. Strict parsers: no new manifest or declaration field.
  6. gofmt, go vet, go test ./... and merge-check.sh pass (commands above).
  7. Words: settings layer, served, controller, walk. Issue and ADR numbers are given with what they are.
Work package F2b of hq design 57, issue 696, ADR 0348 clause 11. It finishes the part F2 (#316) left: `Settle` was unchanged there because influxdb on ace relied on it. **Merge order: mesh-catalog#290 (influxdb serves `${setting:org}` and `${setting:bucket}`) first, then this pull request.** Merged before #290 is delivered, grafana's data source on ace would name the literal bucket `default` in place of the bucket ace's layer sets. ## What changed - `catalogue.Settle` no longer lays a layer over served keys. It only fills `${setting:<key>}` in served values from the layers, with module defaults lowest (`filledFromSettings`, shared with `overridden`). A key a layer sets that the provider serves as a literal changes nothing a consumer is told, and a setting still never adds a served key. - `overridden` keeps its override for contributions; that is not in this package's scope. - `UnusedSettings` no longer counts a literal served key as a place where a setting lands. A served key that asks for `${setting:k}` still counts, through `settingKeysUsedBy`. A layer that sets a literal served key is now reported as reaching nothing, instead of being accepted silently. ## Which live layers relied on the override Only influxdb on ace. I read the settings layer, by key, of every provider assigned on ace, novox, g14 and shanks. That covered the mesh-wide and per-machine layers of distribution, gitea, influxdb, keycloak, mailu, minio, mongodb, mosquitto, mssql, postgres, redis, route-proxy, step-ca and umami. I compared each layer's keys with what the module serves: - mailu (`domain`) and keycloak (`issuer`) serve `${setting:<same key>}`, so the value is the same with or without the override. - The other layers hold only `ports`, `expose` or `endpoints`. Settle skipped `ports` already, and the other two are not served keys. - No catalogue module declares a zone, so `ZoneOn`, which also calls Settle, is unaffected. Zones are `${setting:zone}` in the tests. - No migration seeds a layer. In the tests, three fixtures encoded the old override. step-ca serving an empty `root` that a layer replaced: three co-located tests now use `stepCAWithItsRoot`, which serves `${setting:root}`. A store whose layer replaced the derived bucket: `TestASettingComposesWithADerivedValue` now serves `${setting:bucket-prefix}${consumer:as:dns}`. `TestASettingOverridesAServedKeyAndAddsNone` asserted the override and is replaced by `TestALayerNoLongerReplacesAServedKey`. Each keeps its intent: an operator's value reaches what is served where the definition asks for it. ## How it is proven - `TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings`: the seed is like ace. influxdb's layer holds org, bucket (made-up values) and an endpoints block, and grafana consumes influxdb-api with its data source file copied verbatim from the catalogue. Every grafana resource composed, including the data source file and the binding file, is compared byte for byte with golden files in `testdata/influxdb-served-by-setting/`. The golden files were composed by this branch's base (71362c1, Settle still overriding) from influxdb serving the literals. The test covers a machine with no layer as well. It passes before this commit (catalogue step alone) and after it (both steps). - `TestALayerNoLongerReplacesAServedKey`: Settle leaves a literal served key alone and adds no key. `UnusedSettings` says such a layer reaches nothing. influxdb as it was (literals) under an ace-like layer no longer passes the layer's bucket to grafana, which shows why the catalogue must go first. It fails before this commit. - `TestInfluxdbsConfigTakesItsOrgAndBucket`: influxdb's config.json takes org and bucket from an ace-like layer, and stays `{}` without a layer, because defaults never reach the file. Note that config.json on ace drops the `endpoints` key that today's form wrote there; see mesh-catalog#290. Commands run: `gofmt -l cmd internal examples` (empty), `go vet ./...`, `go test ./...`. `MESH_TEST_POSTGRES=postgres://…@127.0.0.1:<port>/postgres sh merge-check.sh` against a throwaway PostgreSQL exits 0, and every listed store test passes, both at the first base and rebased on dc36536 (after V1, #317). ## Left for other packages - `overridden` still lets a layer replace a contribution's declared keys. That is not served, and was not in this package. - K6 (issue 665) serves `${config:}`. K5 (issue 664) moves ace's layer into mesh-config. ## Reviewer's checklist 1. Does what issue 696's step 2 says, nothing outside it: yes. The `UnusedSettings` change follows from it, because a setting that lands nowhere is said so. 2. Done-when proof: the tests above. `TestALayerNoLongerReplacesAServedKey` fails before this commit. The three changed fixtures are justified above. 3. Byte-identical composition: `TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings` against golden files from 71362c1. 4. No secret anywhere, and the test values are made up; ace's values are not in the change. No schema change. 5. Strict parsers: no new manifest or declaration field. 6. gofmt, go vet, go test ./... and merge-check.sh pass (commands above). 7. Words: settings layer, served, controller, walk. Issue and ADR numbers are given with what they are.
mesh-admin added 1 commit 2026-10-14 05:03:56 +00:00
Stop a settings layer replacing what a provider serves (issue 696)
mesh/merge-gate head c5fe7d4795d7 merged into trunk dc36536e199e — pass: the change touches no module of the mesh's graph
mesh/repo-check head c5fe7d4795d7 merged into trunk dc36536e199e — pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group pkg/57-f2b-served-keys delivered: every member is delivered
c5fe7d4795
What a provider serves is its definition's (ADR 0348 clause 11): a layer
fills only the keys it asks for as ${setting:<key>}. Settle replaced any
served key a layer named, so a setting for one of a provider's files could
change what every consumer was told. influxdb on ace was the one live layer
relying on it and now serves its settings (mesh-catalog, merged first), so
grafana's data source composes byte for byte as before. UnusedSettings no
longer counts a literal served key as a place a setting lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
Contributor

Delivery novox/mesh-controller@c5fe7d4795d7 — delivered since 2026-10-14T05:16:18Z

Delivery plan — builds nothing: the change touches no module of the mesh's graph

Group pkg/57-f2b-served-keys, in order: novox/mesh-catalog@2dd6510ded → novox/mesh-controller@c5fe7d4795

  • composed together: pass — the change touches no module of the mesh's graph

Transitions

  • 2026-10-14 05:04 (new) → proposed (announced): novox/mesh-controller#321's head announced
  • 2026-10-14 05:08 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit
  • 2026-10-14 05:08 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-14 05:16 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-14 05:16 published → delivered (done): nothing for a walk to move

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-controller@c5fe7d4795d7` — **delivered** since 2026-10-14T05:16:18Z **Delivery plan** — builds nothing: the change touches no module of the mesh's graph **Group** `pkg/57-f2b-served-keys`, in order: novox/mesh-catalog@2dd6510ded29 → novox/mesh-controller@c5fe7d4795d7 - composed together: pass — the change touches no module of the mesh's graph **Transitions** - 2026-10-14 05:04 (new) → proposed (announced): novox/mesh-controller#321's head announced - 2026-10-14 05:08 proposed → checked (checked): a verdict names this commit and the trunk's head as its trunk commit - 2026-10-14 05:08 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-14 05:16 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-14 05:16 published → delivered (done): nothing for a walk to move The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
mesh-admin left a comment
Author
Contributor

Verdict: APPROVE at c5fe7d4795

Independent review for issue 696 (a settings layer still overrides what a provider serves). Merge after mesh-catalog#290.

What I ran on c5fe7d4:

  • gofmt -l cmd internal examples: empty.
  • go vet ./...: clean.
  • MESH_TEST_POSTGRES=postgres://postgres@127.0.0.1:<port>/postgres sh merge-check.sh against a throwaway PostgreSQL over TCP on 127.0.0.1 with mmap shared memory: exit 0, store tests included. I stopped the server afterwards.
  • I took the three new tests to the base (dc36536, the old Settle):
    • TestALayerNoLongerReplacesAServedKey fails there, on all three of its assertions.
    • TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings passes there with both the old and the new influxdb. That means the golden files are what the old controller composes.
    • TestInfluxdbsConfigTakesItsOrgAndBucket passes there, because it tests the catalogue's change.

What I checked:

  • The golden files match ace. The seed has the same key shape as influxdb's layer on ace (org, bucket, and endpoints.api with label, port and reach), and its values are made up. I did not copy ace's values.
  • No other live layer relied on the override. Every served key in the catalogue is a literal or asks for its own key, and only influxdb declares a setting with the same name as a served key. ollama (model) and showcase (path) are assigned nowhere. Of the providers' layers, I re-read minio, step-ca, gitea, distribution, mailu and influxdb on the whole mesh and on their machines, and mosquitto on ace: apart from influxdb on ace, they hold only ports, expose or endpoints, or nothing. No catalogue module declares a zone, so ZoneOn is unaffected.
  • Every caller of Settle (declaration.go:2045, consumer_into_serves.go:246, zones.go:71, plan.go:1640) now takes only the ${setting:} fill. That is the intended rule. The secret and protected refusals in settle never applied to a served map, so nothing is lost there.
  • The three rewritten fixtures still test what they tested. The co-located tests still prove that a value the operator sets on step-ca reaches route-proxy's file and binding, now through ${setting:root}. The derived-value test still proves that the setting is filled before the consumer's half is derived. The deleted override test is replaced by its inverse.

Findings. None of them blocks the merge, and all can follow:

  1. A comment in a file this pull request does not touch is now wrong. internal/catalogue/terminal_keys.go:18 still says "A setting overrides a served key (Settle)" as the reason every served key is a terminal key. Keeping served keys terminal is still the safe choice, but the reason should be reworded in a follow-up.
  2. overridden (settings.go:141) still lets a layer replace a contribution's declared keys. That is outside issue 696 and ADR 0348 clause 11, which are about served keys and the keys a file takes. Under ADR 0348 clauses 4 and 5, though, a value lands only in a template as ${config:} or is read at runtime, so this override is a second instance of the same rule. I recommend its own issue, linked to K6 (issue 665), instead of leaving it as a line in this pull request's description.
  3. The new UnusedSettings message (settings.go, "declares no %q in what it contributes") no longer says that a literal served key is not settable. A hint like "a served key is set only where the definition asks for ${setting:}" would explain the refusal to the person who sets it. This one is optional.
Verdict: APPROVE at c5fe7d4795d7 Independent review for issue 696 (a settings layer still overrides what a provider serves). Merge after mesh-catalog#290. What I ran on c5fe7d4: - `gofmt -l cmd internal examples`: empty. - `go vet ./...`: clean. - `MESH_TEST_POSTGRES=postgres://postgres@127.0.0.1:<port>/postgres sh merge-check.sh` against a throwaway PostgreSQL over TCP on 127.0.0.1 with mmap shared memory: exit 0, store tests included. I stopped the server afterwards. - I took the three new tests to the base (dc36536, the old Settle): - `TestALayerNoLongerReplacesAServedKey` fails there, on all three of its assertions. - `TestGrafanaComposesTheSameOnceInfluxdbServesItsSettings` passes there with both the old and the new influxdb. That means the golden files are what the old controller composes. - `TestInfluxdbsConfigTakesItsOrgAndBucket` passes there, because it tests the catalogue's change. What I checked: - The golden files match ace. The seed has the same key shape as influxdb's layer on ace (org, bucket, and endpoints.api with label, port and reach), and its values are made up. I did not copy ace's values. - No other live layer relied on the override. Every served key in the catalogue is a literal or asks for its own key, and only influxdb declares a setting with the same name as a served key. ollama (model) and showcase (path) are assigned nowhere. Of the providers' layers, I re-read minio, step-ca, gitea, distribution, mailu and influxdb on the whole mesh and on their machines, and mosquitto on ace: apart from influxdb on ace, they hold only `ports`, `expose` or `endpoints`, or nothing. No catalogue module declares a `zone`, so `ZoneOn` is unaffected. - Every caller of Settle (declaration.go:2045, consumer_into_serves.go:246, zones.go:71, plan.go:1640) now takes only the `${setting:}` fill. That is the intended rule. The secret and `protected` refusals in `settle` never applied to a served map, so nothing is lost there. - The three rewritten fixtures still test what they tested. The co-located tests still prove that a value the operator sets on step-ca reaches route-proxy's file and binding, now through `${setting:root}`. The derived-value test still proves that the setting is filled before the consumer's half is derived. The deleted override test is replaced by its inverse. Findings. None of them blocks the merge, and all can follow: 1. A comment in a file this pull request does not touch is now wrong. internal/catalogue/terminal_keys.go:18 still says "A setting overrides a served key (Settle)" as the reason every served key is a terminal key. Keeping served keys terminal is still the safe choice, but the reason should be reworded in a follow-up. 2. `overridden` (settings.go:141) still lets a layer replace a contribution's declared keys. That is outside issue 696 and ADR 0348 clause 11, which are about served keys and the keys a file takes. Under ADR 0348 clauses 4 and 5, though, a value lands only in a template as `${config:}` or is read at runtime, so this override is a second instance of the same rule. I recommend its own issue, linked to K6 (issue 665), instead of leaving it as a line in this pull request's description. 3. The new `UnusedSettings` message (settings.go, "declares no %q in what it contributes") no longer says that a literal served key is not settable. A hint like "a served key is set only where the definition asks for ${setting:<key>}" would explain the refusal to the person who sets it. This one is optional.
mesh-admin merged commit 60d390aded into feat/57-config-and-provisioning 2026-10-14 05:16:03 +00:00
mesh-admin deleted branch pkg/57-f2b-served-keys 2026-10-14 05:16:04 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-controller#321