Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dbb9d2bc16 | ||
|
|
28d53dcc28 | ||
|
|
df667eb710 | ||
|
|
098a2ca485 | ||
|
|
a34cedeb5d | ||
|
|
db5ff5a5ee |
@@ -124,6 +124,32 @@ This corrects a fact, not the decision: one statement per endpoint, three things
|
||||
none of them deciding on its own, all stand. The table in the decision should be read with the filter
|
||||
column applying to an unrouted endpoint.
|
||||
|
||||
## Progressive insight — 2026-10-02, from issue 191
|
||||
|
||||
**For a routed endpoint, "the proxy serves the internal name" has to mean "serves it to the private
|
||||
network", and only the proxy can make it mean that.** The decision says `internal` means the proxy
|
||||
serves the internal name and not the public one. It does not say to whom, and the proxy answered
|
||||
every name it routes to any request that carried it, on the same listeners as its public names. A
|
||||
name being internal kept nobody out: a request from the internet only had to send it. While every
|
||||
routed endpoint also had a public name, nothing showed it. Once an endpoint could be internal alone
|
||||
([issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)), serving
|
||||
its name to everyone would have published exactly what `internal` was chosen to keep private.
|
||||
|
||||
The earlier insight above says the port is not the path for a routed endpoint. This is its other
|
||||
half: the proxy is the path, so the proxy is where `internal` is enforced. It serves an internal name
|
||||
only to the machines of the mesh and to the machine itself
|
||||
([ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md)). Who the mesh is, it is told,
|
||||
not left to work out: its membership carries the same list of machine addresses the filter's "from the
|
||||
mesh" is rendered from ([ADR 0167](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
To anyone else, the name is answered as one never routed, in the handshake and in the request, and not
|
||||
listed among the names it serves. This holds for the internal name of a `both` endpoint too, whose
|
||||
outsiders have its public name.
|
||||
|
||||
The decision, the options and the consequences stand: one statement per endpoint, three things
|
||||
derived from it. Checked in the proxy's own tests: an internal-only name is served to a machine the
|
||||
membership names and to loopback, and refused, unlisted and uncertified for any other request; until
|
||||
the mesh is issued, it is served to the machine alone.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.**
|
||||
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
---
|
||||
|
||||
# 167. A membership carries what its module receives, and who the mesh is
|
||||
|
||||
## Context
|
||||
|
||||
A provider learns what it is given from a file. The controller composes every consumer's contribution
|
||||
to a requirement, and the node's declaration writes them into the provider's received file. The route
|
||||
proxy reads its routes that way: one JSON file, re-read every two seconds.
|
||||
|
||||
[Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) showed what
|
||||
that file leaves out. Since [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
a route whose endpoint reaches only the private network carries an internal name and no public one.
|
||||
The proxy dropped it. Serving it was not enough either: the proxy answers public and internal names on
|
||||
the same listeners, so an internal name served to every request is public under a guessable name. To
|
||||
serve it correctly the proxy needs a second fact, **who the mesh is**, and nothing gave it one.
|
||||
|
||||
The first attempt had the proxy work it out: the mesh's range from an environment variable written by
|
||||
the catalogue, and the machine's container bridges read from its own interfaces. That is a second
|
||||
definition of "the mesh", kept by one module, beside the one the packet filter already uses. The
|
||||
controller resolves "from the mesh" to every machine's address on the private network, and the filter
|
||||
is rendered from that list. Two definitions agree until one changes.
|
||||
|
||||
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
already gives every assignment one document on the bus, its membership, read once at connect and
|
||||
followed. It says what the assignment serves and reaches. It does not yet say what it is given.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the file, add the mesh to it.** The proxy keeps polling a file, and the controller writes the
|
||||
mesh's addresses beside the routes. It fixes the definition, but delivery stays a file re-read on a
|
||||
timer, written by a separate path from the one every other fact a module is told now takes.
|
||||
2. **Have the proxy work it out** from a range in its environment and the machine's interfaces. Rejected:
|
||||
it is the second definition this record exists to remove.
|
||||
3. **The membership carries it.** What each module receives, from the same composition its received
|
||||
file is written from, and the mesh's addresses, from the same list the filter is rendered from. The
|
||||
proxy follows its membership and serves exactly that.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 3.**
|
||||
|
||||
- **A membership carries what its module receives**, by requirement: the contributions every consumer
|
||||
made, exactly as composed for its received file. A requirement nobody contributed to is an empty
|
||||
list, never absent, for the reason the file is written empty: "nothing asked" and "never told" want
|
||||
different responses.
|
||||
- **A membership carries who the mesh is**: every machine's address on the private network, the list
|
||||
a rule saying "from the mesh" resolves to. One list, two readers: the filter and any module that
|
||||
must tell the mesh from the world.
|
||||
- **The route proxy reads its routes and the mesh from its membership**, with the bus account every
|
||||
module that speaks on the bus is given. It serves an internal name only to the machines the mesh
|
||||
names and to the machine itself, and answers anyone else as it answers a name it never routed: in
|
||||
the request, in the handshake, and in the list of names it serves.
|
||||
- **The file stays until the bus has spoken.** While a proxy has read no membership that carries routes,
|
||||
it serves the file, and an internal name only to its own machine: refused, never opened. A
|
||||
membership from a controller that issues no routes changes nothing.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every membership grows two fields. A machine joining or leaving republishes every membership, which
|
||||
a push already does.
|
||||
- A provider that receives something is told it twice for now, in its file and on the bus. The file
|
||||
goes when every provider reads its membership; that is its own change.
|
||||
- The route proxy needs a bus account. It is issued like any module's, so a machine running the proxy
|
||||
cannot be composed between the catalogue declaring the account and the operator issuing it. The
|
||||
machine keeps what it runs meanwhile.
|
||||
- The internal name of a route that also has a public one is now served to the mesh only. Outsiders
|
||||
have the public name.
|
||||
- A container on the same machine that calls that machine's own internal name arrives from its
|
||||
container network, not from a mesh address, and is refused. Calls between machines are unaffected:
|
||||
they leave by the machine's mesh address. Whether the mesh should also issue each machine's container
|
||||
networks is left open, because the mesh does not record them today.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| What a provider receives on the bus is what its received file says, same-node port fix included | a controller test composing a provider and a consumer on one machine and comparing the two |
|
||||
| An internal name is served to the machines the membership names and to loopback, and to nobody else | the proxy's tests: served from a named address and from loopback; refused, unlisted and uncertified from any other |
|
||||
| A membership that carries no routes, or a mesh that cannot be read, changes nothing | the proxy's tests |
|
||||
| Until the mesh is issued, an internal name is served to the machine alone | the proxy's tests |
|
||||
| Live: an internal-only route answers over the mesh and is refused from outside | by hand, after the release |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md) —
|
||||
the membership this extends
|
||||
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — reach, and the
|
||||
insight of 2026-10-02 that the proxy is where internal reach is kept
|
||||
- [ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md) — the machine itself is always inside
|
||||
- [Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) — what
|
||||
found it
|
||||
@@ -177,6 +177,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0161** — [What deserves a seat: a role of a module is a seat, a singular fact about machines is a placement with a capacity of one, and a holder's software is the machine's](0161-what-deserves-a-seat.md)
|
||||
- **0162** — [A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
|
||||
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
@@ -7,8 +7,9 @@ code:
|
||||
- mesh-controller internal/identity/authority.go
|
||||
- mesh-host internal/identity/serving.go
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-09-30
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md
|
||||
- 02-DECISIONS/0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md
|
||||
- 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
|
||||
@@ -842,6 +843,18 @@ One value, three readers:
|
||||
| `public` | the machine port, to anywhere | the public name | the public authority |
|
||||
| `both` | the machine port, to anywhere | both names | each name's own authority |
|
||||
|
||||
*2026-10-02.* **The proxy serves an internal name to the private network only**
|
||||
([ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
its insight of this date). It answers public and internal names on the same listeners, so the name a
|
||||
request carries is the request's own claim, not where the request came from. An internal name is
|
||||
served to the machines of the mesh and to the machine itself; to anyone else it is answered as a name
|
||||
never routed, in the handshake as well as the request. Without this, an endpoint with reach `internal`
|
||||
would be public under a name that is easy to guess
|
||||
([issue 191](../../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)).
|
||||
The proxy is told who the mesh is, and its routes, in its membership on the bus — the same machine
|
||||
addresses the filter's "from the mesh" is rendered from
|
||||
([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
|
||||
**An endpoint that is not routed is reached and never named.** No route contribution means no name is
|
||||
composed and no certificate requested, while the filter still acts on it. That is the case the model
|
||||
could not express at all, and it is the ordinary case for anything that is not HTTP.
|
||||
|
||||
@@ -7,8 +7,9 @@ code:
|
||||
- mesh-tools src/broker-amqp.ts (to be replaced)
|
||||
- mesh-catalog modules/nats (to be written)
|
||||
- mesh-sdk src (the protocol's NATS binding, step 3)
|
||||
updated: 2026-10-01
|
||||
updated: 2026-10-02
|
||||
decisions:
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
- 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
@@ -94,6 +95,12 @@ list; the account's grant is the same membership read the other way; the console
|
||||
tool's subject. The one rule a runtime keeps is the membership's own subject, from the two names in its
|
||||
credential.
|
||||
|
||||
**Revised 2026-10-02** ([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)):
|
||||
a membership also carries **what its module receives**, by requirement — the contributions its
|
||||
received file is written from, from the same composition — and **who the mesh is**, every machine's
|
||||
address on the private network, the list the filter's "from the mesh" is rendered from. The route
|
||||
proxy is the first reader of both.
|
||||
|
||||
**Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)):
|
||||
**`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the
|
||||
mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-01
|
||||
located-in: [mesh-controller examples/route-proxy/main.go (routesFrom requires a route's public `name` and treats `internal-name` only as an alias of it; the handler serves every routed name to any source), mesh-controller internal/broker/membership.go (a membership says nothing of what its module receives or who the mesh is)]
|
||||
fixed-by: mesh-controller PR 207 (the membership carries what a module receives and who the mesh is; the proxy follows it and serves internal names to the mesh only), mesh-catalog PR 211 (the proxy's bus account), mesh-controller PR 208 (the issue verb that delivers it), live 2026-10-02
|
||||
amended-design: [03-DESIGN/01-to-be/08-connectivity.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md]
|
||||
---
|
||||
|
||||
# 191 — A route with only an internal name is dropped as naming nothing
|
||||
|
||||
## What was observed
|
||||
|
||||
A module whose endpoint reaches only the private network could not be reached by its internal name.
|
||||
The module ran and answered on its own port. Its route's internal name resolved to the serving node.
|
||||
The request failed during the TLS handshake:
|
||||
|
||||
```
|
||||
http: TLS handshake error from …: no public route for "unifi.home-server.internal" in this mesh,
|
||||
so no certificate is asked for
|
||||
```
|
||||
|
||||
The proxy's own log said why, every time it re-read its routes:
|
||||
|
||||
```
|
||||
unifi on home-server asked for a route and named nothing; skipped
|
||||
```
|
||||
|
||||
The route it skipped was not empty. The mesh had given it an endpoint, a port, a scheme and an internal
|
||||
name, and no public name:
|
||||
|
||||
| route | `name` | `internal-name` | served |
|
||||
|---|---|---|---|
|
||||
| home-assistant | a public name | `home-assistant.home-server.internal` | under both |
|
||||
| unifi | — | `unifi.home-server.internal` | under neither |
|
||||
|
||||
Three other modules on the same node were skipped with the same line on the same pass.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**Reach is decided in one place, and the proxy reads the old shape of the decision.**
|
||||
[ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)
|
||||
made an endpoint's reach decide which names exist. The controller composes the public name, the
|
||||
internal name, or both, and composes no name that nobody asked for
|
||||
([issue 140](../140-an-endpoints-reach-is-not-declared/01-resolution.md)). An endpoint that reaches
|
||||
only the private network is the ordinary case for anything that should not face the internet. It is
|
||||
exactly the case the proxy drops.
|
||||
|
||||
**The failure is quiet and points the wrong way.** Nothing marks the module unhealthy. The handshake
|
||||
error says *no public route*, which reads as a certificate fault on the reader's side. The line that
|
||||
gives the real cause is one of four identical lines repeated every few seconds in a log nobody reads
|
||||
until they already suspect the proxy.
|
||||
|
||||
**The opposite move is not a workaround.** Giving the endpoint public reach makes the proxy serve it.
|
||||
It also publishes an administration interface to the internet to get a name on the private network.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the proxy refuse a route it cannot serve in a way the controller or an operator sees, not
|
||||
only in its own log? The same silent skip covers a route with no usable port or an unknown scheme.
|
||||
- What checks that what the controller composes and what the proxy serves stay the same shape? ADR
|
||||
0138 changed one side and nothing failed on the other.
|
||||
|
||||
## Resolved (2026-10-02)
|
||||
|
||||
Built as [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
decided, and live on both machines that run the proxy. Each logs that its routes now come from its
|
||||
membership, and serves internal names to the four machines the mesh names. Checked by hand:
|
||||
|
||||
- the internal-only route answers through the proxy from the serving machine and from two other
|
||||
machines of the mesh, over a certificate from the mesh's own authority that each verifies;
|
||||
- the same name asked from an address outside the mesh is answered as a name never routed, over plain
|
||||
HTTP, and refused in the TLS handshake; the list of served names it is shown leaves out every internal
|
||||
name.
|
||||
|
||||
Two things the rollout found are their own records: the proxy's bus account could be issued only from
|
||||
the controller's command line, until mesh-controller PR 208 added the `issue` verb, and the status line
|
||||
counting every module as a bus user without a credential is
|
||||
[issue 195](../195-every-assigned-module-is-counted-as-a-bus-user-without-a-credential/00-report.md).
|
||||
The serving machine also lacked the certificate-trust module, so it could not verify the mesh's own
|
||||
certificates until it was assigned there.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-01.*
|
||||
|
||||
**Ruled out first: the module itself.** Its container was up and had not restarted. The controller's
|
||||
status endpoint answered on its own port with `"up": true`. The tool wrapper beside it was serving
|
||||
all its tools.
|
||||
|
||||
**Ruled out: name resolution.** The internal name resolved to the serving node's private-network
|
||||
address, which is where the proxy listens. Plain HTTP to the name reached the proxy and got a 404.
|
||||
HTTPS failed in the handshake, and the proxy logged that it had no route for the name.
|
||||
|
||||
**The route as the proxy received it.** The mesh-written route file held a complete contribution
|
||||
for the module: endpoint `web`, port, scheme `https`, `insecure`, a label, and `internal-name`. It had
|
||||
no `name`. That is what the controller composes for an endpoint whose reach stops at the private
|
||||
network (ADR 0138, `composeName`). The contribution was correct.
|
||||
|
||||
**Located: `routesFrom` in the proxy.** It reads `name` first and skips the contribution if `name`
|
||||
is empty. It reads `internal-name` only at the end, as a second host for a rule that already has a
|
||||
public one. So the proxy can serve an internal name only next to a public one. That matched the
|
||||
mesh before ADR 0138, when both names were always composed. It has been wrong since then.
|
||||
|
||||
The other half of the proxy already handles the case. Certificates for a host are split by whether it
|
||||
is in the public set: hosts outside it go to the internal authority, and only hosts inside it are
|
||||
eligible for ACME. A host that is only ever an internal name falls on the correct side of both checks
|
||||
without change. For certificates, only reading the route was wrong; who may reach the route is the next section.
|
||||
|
||||
**The fix.** `routesFrom` takes a route that names either host, serves each name it carries, and
|
||||
marks only the public one as public. It still skips a route that names neither, with the same log line.
|
||||
A test proves an internal-only route is served, certified by the internal authority, and refused by
|
||||
the public one. That test fails against the code before the change.
|
||||
|
||||
## The first fix would have made the name public — 2026-10-02, from review
|
||||
|
||||
Serving the dropped route was not enough. The proxy picks a route from the name a request carries
|
||||
and never from where the request came from, and it answers public and internal names on the same
|
||||
listeners. Its public names resolve to an address the internet reaches. So once the internal-only
|
||||
route was served, any request from the internet carrying `unifi.home-server.internal` — a name of a
|
||||
fixed, guessable shape — would have reached an administration interface that reach `internal` was
|
||||
chosen to keep private. Before the fix the route was unreachable from everywhere. After it, it would
|
||||
have been reachable from everywhere. Two more leaks came with it: the proxy's answer for an unrouted
|
||||
name listed every name it serves, internal ones included, and the handshake handed a certificate
|
||||
naming the internal host to any client.
|
||||
|
||||
Nothing showed this while every routed endpoint also had a public name: its internal name exposed
|
||||
nothing the public one did not. It is a gap in the decision's wording, not only in the proxy — ADR
|
||||
0138 says the proxy *serves* the internal name without saying to whom — so it is recorded there as a
|
||||
progressive insight and in the to-be connectivity design.
|
||||
|
||||
**Where "inside" is decided: told, not worked out.** The first correction had the proxy work it out
|
||||
for itself — the mesh's range from an environment variable the catalogue wrote, and the machine's
|
||||
container bridges from its own interfaces. That was a second definition of "the mesh", kept by one
|
||||
module beside the one the controller already has: it resolves "from the mesh" to every machine's
|
||||
address on the private network, and the packet filter is rendered from that list. Reviewed, it was
|
||||
replaced: [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
has every membership on the bus carry what its module receives and that list, and the proxy follows
|
||||
its membership. One composition, read by the filter and by the proxy.
|
||||
|
||||
The proxy reads the source address, where the guard reads the interface, because it cannot see the
|
||||
interface a request arrived on. A claimed source does not carry here: a connection needs its replies,
|
||||
and replies to a mesh address leave by the tunnel.
|
||||
|
||||
**What changed with it.** The internal name of a route that also has a public one is now served to the
|
||||
mesh only, like any other internal name. Outsiders have the public name, so nothing they could reach is
|
||||
lost. A container calling its own machine's internal name arrives from its container network and is
|
||||
refused; whether the mesh should issue those networks too is left open in ADR 0167.
|
||||
|
||||
**Order of release.**
|
||||
|
||||
1. The catalogue change, which gives the proxy a bus account. A machine running the proxy is not
|
||||
composed until its account is issued, so the account is issued straight after
|
||||
(`module issue route-proxy --node <machine>`), and then the machine is pushed.
|
||||
2. The controller and proxy change. The push after it publishes memberships that carry the routes and
|
||||
the mesh, and each proxy takes them. Until then, a proxy serves its file, and internal names to its
|
||||
own machine alone.
|
||||
-78
@@ -1,78 +0,0 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-catalog modules/mesh-console, mesh-controller cmd/mesh-controller/plan.go (port assignment)]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 192 — The mesh's tools reach a person only by a registration made by hand
|
||||
|
||||
## What was observed
|
||||
|
||||
A design session on a workstation had none of the mesh's tools. The console was running on that
|
||||
machine and answering on its loopback port. It was reached over the bus as the console's account, and
|
||||
listed every running module's tools and every seat's verbs
|
||||
([ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md)). What was missing
|
||||
was the registration that tells the person's coding agent where the console is. That registration had
|
||||
been made by hand, once, while migrating the machine, and scoped to the one project directory it was
|
||||
made in. Every session started anywhere else had no mesh tools. Nothing said so: the agent simply
|
||||
offered no mesh tools, and the session fell back to a pull-request link for a person to open by hand.
|
||||
|
||||
The predecessor did this job itself: it wrote its tool server into the agent's user configuration on
|
||||
every machine. Migrating removed that entry, as it should have, and no module took the job over.
|
||||
|
||||
## Why this is here
|
||||
|
||||
Three gaps, each of which would have stopped a module from doing it even if one existed.
|
||||
|
||||
**1. The console tells nobody where it is.** Its definition listens on a port and provides nothing.
|
||||
A module that wanted to point an agent at the console has no requirement it could name, so it
|
||||
would have to write the address into its own definition as a literal. That is exactly what
|
||||
[ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) and
|
||||
[ADR 0155](../../02-DECISIONS/0155-a-definition-names-no-installation-and-how-that-is-checked.md)
|
||||
remove.
|
||||
|
||||
**2. The console's port is one its definition chose.** The definition names a port, and the mesh
|
||||
never assigned one: no port assignment exists for the console on any machine. The plan assigns a
|
||||
machine port only to a port a container publishes through a mapping, "without one the software binds
|
||||
what it binds". The console runs on the host network with no mapping, but it reads its listening
|
||||
address from `${port:…}`, so the mesh could move it and does not. That is a module choosing a
|
||||
machine port, which [ADR 0038](../../02-DECISIONS/0038-the-mesh-assigns-the-port.md) exists to
|
||||
prevent, through a gap in how the rule is applied rather than a decision against it. A module that
|
||||
reads its port from the mesh should be assigned one like any other.
|
||||
|
||||
**3. Nothing in the mesh owns a person's agent configuration.** No catalogue module writes the agent's
|
||||
settings, its tool-server registrations, or the rules and skills the predecessor delivered. On the
|
||||
four machines these are hand-kept, or left over from the predecessor, or missing.
|
||||
|
||||
## What a fix looks like (not decided)
|
||||
|
||||
- **The console provides its endpoint.** A provision, working name `mesh-tools`, served as the URL on
|
||||
the machine port the mesh gives it. The console listens only on loopback, so the provider must be on
|
||||
the consumer's own machine. Co-location already chooses it
|
||||
([ADR 0084](../../02-DECISIONS/0084-which-provider-serves-a-consumer.md)), and a machine with no
|
||||
console refuses the consumer, naming the provision.
|
||||
- **A module for the coding agent requires it** and writes the registration into the agent's
|
||||
system-wide managed settings. The agent reads tool servers from a `managedMcpServers` key there. That
|
||||
file is the machine's rather than a user's, so the module owns it whole and no home directory is
|
||||
named. People keep their own registrations beside it. The agent's separate *exclusive* managed
|
||||
file is the wrong one: it blocks every registration a person makes and hides the hosted connectors.
|
||||
The agent's per-user file is rewritten by the agent continuously and sits in a home directory,
|
||||
which would make its path an operator value. These facts come from the agent's documentation
|
||||
(managed MCP and managed settings pages), not yet verified on a machine.
|
||||
- **The same module owns the rest of the agent's configuration** the predecessor delivered: managed
|
||||
settings and the rules, skills and instructions every session reads. Each declared setting carries
|
||||
a default (ADR 0164,
|
||||
proposed on its own branch), so one configuration serves every machine and one machine may differ.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Is the agent's configuration one module or several?** Tool registration, managed settings, and
|
||||
the instruction files have different readers and change at different rates.
|
||||
- **Whose machine port is the console's?** Should a host-network container that reads its port from
|
||||
`${port:…}` be assigned one, or should a machine-only listener keep its declared number? The second
|
||||
needs a decision, because ADR 0038 does not allow it today.
|
||||
- **Credentials.** The console's authority is the machine's login (ADR 0152). A registration that
|
||||
reaches it carries no secret today. If the console ever listens beyond loopback, the registration
|
||||
needs one, from the vault.
|
||||
-112
@@ -1,112 +0,0 @@
|
||||
---
|
||||
status: resolved
|
||||
opened: 2026-10-02
|
||||
located-in: [mesh-catalog modules/postgres/client.ts (readOnlyQuery)]
|
||||
fixed-by: [mesh-catalog PR 209 (postgres), mesh-catalog PR 210 (mssql)]
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 193 — The store seat's read-only query is read-only by convention, and its answer is unreadable
|
||||
|
||||
## What was observed
|
||||
|
||||
Asking the store seat's `query` verb for a count through the console returned this. Rows are each
|
||||
wrapped in an object under a key named `BEGIN`: the column name, then the value, then the word
|
||||
`ROLLBACK`. A query returning nothing gave the column name and `ROLLBACK` alone. The answer to
|
||||
`select count(*) as n from <table>` was:
|
||||
|
||||
> `rows: [ {BEGIN: "n"}, {BEGIN: "46"}, {BEGIN: "ROLLBACK"} ]`
|
||||
|
||||
A reader can work it out. A program cannot, and a query with two columns loses which value belongs to
|
||||
which.
|
||||
|
||||
## Why this is here
|
||||
|
||||
**The cause is the same line that makes the query read-only.** The holder's tool sends
|
||||
`BEGIN TRANSACTION READ ONLY; <the caller's statement>; ROLLBACK;` to the command-line client as one
|
||||
string. The client prints a command tag for each of the three statements, and the parser takes the
|
||||
first line, `BEGIN`, as the header.
|
||||
|
||||
**And it is not read-only.** [ADR 0159](../../02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md)
|
||||
decided the store seat's `query` verb is "one read-only statement against one database". The only
|
||||
thing enforcing that is the wrapping transaction, and the caller's statement is pasted inside it as
|
||||
text. A statement that begins by ending the transaction (a commit, then anything) runs whatever
|
||||
follows it outside the read-only transaction, with the holder's own role, the administrative one that creates every
|
||||
consumer's role and database. A rule stated in a decision and enforced by string concatenation is enforced by nothing.
|
||||
|
||||
*This is read from the code, not tried against the live store, and it should not be tried there.*
|
||||
The lab bed is where it gets proven.
|
||||
|
||||
Every caller with `invokes` on the store seat's `query` can do this. The console has `invokes: ["*"]`,
|
||||
so that includes anyone logged in on a machine running the console.
|
||||
|
||||
## What a fix looks like
|
||||
|
||||
- **One statement, refused otherwise.** Send the caller's statement alone, through the client's
|
||||
single-statement path (the extended protocol takes one statement per call and refuses more). The
|
||||
read-only property then comes from the session, not from text around the statement.
|
||||
- **Read-only by role, not by transaction.** Run the verb as a role that can only read, granted
|
||||
`pg_read_all_data`, not as the administrative role. A statement that escapes every wrapper still cannot write.
|
||||
- **Rows as rows.** Parse the client's output with the column names it returns, or use a driver
|
||||
instead of the command-line client, so a row is an object keyed by its columns.
|
||||
- **The check 0159 lacks:** a test that sends a commit followed by a write and asserts the write is
|
||||
refused and nothing changed. Another asserts a two-column row comes back keyed by both columns.
|
||||
|
||||
## Proven, 2026-10-02
|
||||
|
||||
On a throwaway server — the same engine image, no network, reached over a socket — the module's code
|
||||
from the catalogue's main branch ran `COMMIT; COPY (select 1) TO PROGRAM '<a command>'` and **the
|
||||
command ran on the database host** as the server's own user. `COMMIT; DROP TABLE t` executed the drop
|
||||
outside the read-only transaction; the wrapper's own trailing rollback happened to undo it, which a
|
||||
caller ending their statement with a commit of their own would get past (not tried). Nothing was tried
|
||||
against the live store.
|
||||
|
||||
The fix (mesh-catalog PR 209) runs the caller's statement as a login granted `pg_read_all_data` and
|
||||
nothing else, read-only by its role and its session, with a password the mesh mints as one of the
|
||||
module's own secrets; without that password the call is refused rather than run as the admin. On the
|
||||
same throwaway server every escape above, and `SET ROLE`, `RESET SESSION AUTHORIZATION`, turning
|
||||
read-only off, creating a table, altering the role and reading a server file, is refused; a plain
|
||||
select comes back keyed by its columns. One attempt — turning the transaction's read-only off, then
|
||||
deleting — got past the first layer and was stopped by the second, which is why both exist.
|
||||
|
||||
**Not answered by the statement-count fix proposed above.** The command-line client sends one string
|
||||
in one message, so several statements still arrive together. They are harmless as the reader, and
|
||||
refusing them is left to whoever moves the module to a driver.
|
||||
|
||||
## The same hole, elsewhere — and two worse ones
|
||||
|
||||
The `mssql` module wrapped a caller's statement the same way (`BEGIN TRANSACTION; … ROLLBACK;` as its
|
||||
administrator) for its `mssql_query` tool. Its command-line client added two holes of its own. Both
|
||||
were proven on a throwaway server, running the client the way the module ran it:
|
||||
|
||||
- **It substitutes `$(NAME)` from its environment into the caller's text**, and the administrator's
|
||||
password is in that environment. Selecting it as a string returned the password.
|
||||
- **It reads a line beginning `:!!` as a command that starts a program**, in the container that holds
|
||||
the administrator's password and the module's bus credentials. Its switch for refusing such commands
|
||||
makes the shipped version ignore the statement entirely, so the switch cannot be the guard.
|
||||
|
||||
None of it was reachable on the live mesh, for a reason that is a defect of its own: the runtime image
|
||||
never installed the client, so every mssql tool failed (`spawn sqlcmd ENOENT`). The fix (mesh-catalog
|
||||
PR 210) installs the client at a pinned digest and runs the caller's statement as a login that can
|
||||
connect and read and do nothing else. Substitution is off. The statement must be one line, placed after
|
||||
the module's own text, so no line of it can begin a command; a line break is refused before the client
|
||||
starts. On the throwaway server, writes, `xp_cmdshell`, impersonating the administrator, and joining
|
||||
the administrators' role were all refused, and the variable came back as the literal text.
|
||||
|
||||
**The general lesson**, worth more than either module: *a command-line client is an interpreter with
|
||||
its own syntax, and a caller's text handed to it is a program in that syntax as well as in SQL.* A
|
||||
module that passes a caller's text to a client has two languages to defend, and a transaction drawn
|
||||
around the text defends neither.
|
||||
|
||||
## Resolved, 2026-10-02
|
||||
|
||||
Both pull requests merged, built and pushed to the two machines that run each module. Checked live, on
|
||||
every copy, by asking each one who it is:
|
||||
|
||||
- the store seat's `query`, and postgres's own tool on each machine, answer as the reader login —
|
||||
not a superuser, in a read-only transaction — with rows keyed by their columns;
|
||||
- mssql's tool, on each machine, answers as its reader login, outside the administrators' role, and
|
||||
returns `$(SQLCMDPASSWORD)` as the literal text it is. Its tools work for the first time.
|
||||
|
||||
The escapes themselves were tried only on the throwaway servers above; on the live mesh the check is
|
||||
the identity a statement runs as, which is what makes every escape a statement that the login cannot do.
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
---
|
||||
status: open
|
||||
opened: 2026-10-02
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 195 — Every assigned module is counted as a bus user without a credential, and the real gaps are lost in the count
|
||||
|
||||
## What was observed
|
||||
|
||||
`status`, and `plan` for any machine, open with one line before anything else:
|
||||
|
||||
```
|
||||
the bus's user list leaves out 49 user(s) the mesh has minted no credential for: <node>.<module>, …
|
||||
Each is a user that cannot connect until one is issued
|
||||
```
|
||||
|
||||
The 49 are spread over four machines and name 26 distinct modules. Checked against the catalogue on
|
||||
2026-10-02:
|
||||
|
||||
| what the module's definition says | modules |
|
||||
|---|---|
|
||||
| declares an own secret named `broker` | 1 — the route proxy, which needed a bus account for issue 191 |
|
||||
| declares no `broker` secret, and emits, consumes and serves nothing on the bus | 17 — the packet filter, the intrusion filter, the ssh daemon, the resolver configuration, the certificate authority, the broker itself and others |
|
||||
| declares no `broker` secret, and **emits events** | 1 |
|
||||
| not in this catalogue, so not checked | 7 |
|
||||
|
||||
So the line counts every module assigned anywhere as a bus user. For almost all of them that is not a
|
||||
missing credential. A module with no `broker` secret has nowhere to receive one, and the mesh already
|
||||
says an account nothing reads is an orphan ([issue 078](../078-a-delivered-secret-is-accepted-under-any-name/00-report.md)).
|
||||
|
||||
Two real gaps sit inside the count and cannot be told from the noise:
|
||||
|
||||
- **A declared `broker` secret was filled with a value that is not an account.** Before its account was
|
||||
issued, the route proxy's plan on both machines already carried a sealed `broker` file, while the same
|
||||
status line said no credential had been minted for it. A push had made the declared secret the way it
|
||||
makes any own secret. The module would have started with a credential the bus does not know, and
|
||||
nothing would have said why. It was found only because the account was being issued by hand.
|
||||
- **A module that emits events declares no way to reach the bus.** Its events can go nowhere, and no
|
||||
check refuses that.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**A warning that is always on is read as never on.** The line names 49 users on every `status` and every
|
||||
`plan`. An operator, or an agent, learns to scroll past it. The one entry that was a real fault looked
|
||||
exactly like the 48 that were not.
|
||||
|
||||
**The fault that was real is the silent kind.** A module whose broker credential is a generated value
|
||||
starts, fails to authenticate, and reports that three layers away from the cause. That is the failure
|
||||
the composition already refuses for a secret that was never made at all ("declared and not made"). Here
|
||||
a value was made, so the refusal never fired.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a bus user be composed for a module that declares no `broker` secret at all? If not, the line
|
||||
shrinks to the modules that can actually use an account.
|
||||
- Is a `broker` secret ever correctly made by the generic generator? If not, should composition refuse
|
||||
a declared `broker` until it is issued, or should the mesh issue it as part of placing the module?
|
||||
- Should a module that emits, consumes or serves on the bus be refused when it declares no `broker`
|
||||
secret?
|
||||
Reference in New Issue
Block a user