Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8c231102f8 |
@@ -124,32 +124,6 @@ This corrects a fact, not the decision: one statement per endpoint, three things
|
||||
none of them deciding on its own, all stand. The table in the decision should be read with the filter
|
||||
column applying to an unrouted endpoint.
|
||||
|
||||
## Progressive insight — 2026-10-02, from issue 191
|
||||
|
||||
**For a routed endpoint, "the proxy serves the internal name" has to mean "serves it to the private
|
||||
network", and only the proxy can make it mean that.** The decision says `internal` means the proxy
|
||||
serves the internal name and not the public one. It does not say to whom, and the proxy answered
|
||||
every name it routes to any request that carried it, on the same listeners as its public names. A
|
||||
name being internal kept nobody out: a request from the internet only had to send it. While every
|
||||
routed endpoint also had a public name, nothing showed it. Once an endpoint could be internal alone
|
||||
([issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)), serving
|
||||
its name to everyone would have published exactly what `internal` was chosen to keep private.
|
||||
|
||||
The earlier insight above says the port is not the path for a routed endpoint. This is its other
|
||||
half: the proxy is the path, so the proxy is where `internal` is enforced. It serves an internal name
|
||||
only to the machines of the mesh and to the machine itself
|
||||
([ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md)). Who the mesh is, it is told,
|
||||
not left to work out: its membership carries the same list of machine addresses the filter's "from the
|
||||
mesh" is rendered from ([ADR 0167](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
To anyone else, the name is answered as one never routed, in the handshake and in the request, and not
|
||||
listed among the names it serves. This holds for the internal name of a `both` endpoint too, whose
|
||||
outsiders have its public name.
|
||||
|
||||
The decision, the options and the consequences stand: one statement per endpoint, three things
|
||||
derived from it. Checked in the proxy's own tests: an internal-only name is served to a machine the
|
||||
membership names and to loopback, and refused, unlisted and uncertified for any other request; until
|
||||
the mesh is issued, it is served to the machine alone.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A manifest gains endpoint names, and a route contribution names an endpoint instead of a port.**
|
||||
|
||||
-99
@@ -1,99 +0,0 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-02
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
---
|
||||
|
||||
# 167. A membership carries what its module receives, and who the mesh is
|
||||
|
||||
## Context
|
||||
|
||||
A provider learns what it is given from a file. The controller composes every consumer's contribution
|
||||
to a requirement, and the node's declaration writes them into the provider's received file. The route
|
||||
proxy reads its routes that way: one JSON file, re-read every two seconds.
|
||||
|
||||
[Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) showed what
|
||||
that file leaves out. Since [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
a route whose endpoint reaches only the private network carries an internal name and no public one.
|
||||
The proxy dropped it. Serving it was not enough either: the proxy answers public and internal names on
|
||||
the same listeners, so an internal name served to every request is public under a guessable name. To
|
||||
serve it correctly the proxy needs a second fact, **who the mesh is**, and nothing gave it one.
|
||||
|
||||
The first attempt had the proxy work it out: the mesh's range from an environment variable written by
|
||||
the catalogue, and the machine's container bridges read from its own interfaces. That is a second
|
||||
definition of "the mesh", kept by one module, beside the one the packet filter already uses. The
|
||||
controller resolves "from the mesh" to every machine's address on the private network, and the filter
|
||||
is rendered from that list. Two definitions agree until one changes.
|
||||
|
||||
[ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
already gives every assignment one document on the bus, its membership, read once at connect and
|
||||
followed. It says what the assignment serves and reaches. It does not yet say what it is given.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the file, add the mesh to it.** The proxy keeps polling a file, and the controller writes the
|
||||
mesh's addresses beside the routes. It fixes the definition, but delivery stays a file re-read on a
|
||||
timer, written by a separate path from the one every other fact a module is told now takes.
|
||||
2. **Have the proxy work it out** from a range in its environment and the machine's interfaces. Rejected:
|
||||
it is the second definition this record exists to remove.
|
||||
3. **The membership carries it.** What each module receives, from the same composition its received
|
||||
file is written from, and the mesh's addresses, from the same list the filter is rendered from. The
|
||||
proxy follows its membership and serves exactly that.
|
||||
|
||||
## Decision
|
||||
|
||||
**Option 3.**
|
||||
|
||||
- **A membership carries what its module receives**, by requirement: the contributions every consumer
|
||||
made, exactly as composed for its received file. A requirement nobody contributed to is an empty
|
||||
list, never absent, for the reason the file is written empty: "nothing asked" and "never told" want
|
||||
different responses.
|
||||
- **A membership carries who the mesh is**: every machine's address on the private network, the list
|
||||
a rule saying "from the mesh" resolves to. One list, two readers: the filter and any module that
|
||||
must tell the mesh from the world.
|
||||
- **The route proxy reads its routes and the mesh from its membership**, with the bus account every
|
||||
module that speaks on the bus is given. It serves an internal name only to the machines the mesh
|
||||
names and to the machine itself, and answers anyone else as it answers a name it never routed: in
|
||||
the request, in the handshake, and in the list of names it serves.
|
||||
- **The file stays until the bus has spoken.** While a proxy has read no membership that carries routes,
|
||||
it serves the file, and an internal name only to its own machine: refused, never opened. A
|
||||
membership from a controller that issues no routes changes nothing.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every membership grows two fields. A machine joining or leaving republishes every membership, which
|
||||
a push already does.
|
||||
- A provider that receives something is told it twice for now, in its file and on the bus. The file
|
||||
goes when every provider reads its membership; that is its own change.
|
||||
- The route proxy needs a bus account. It is issued like any module's, so a machine running the proxy
|
||||
cannot be composed between the catalogue declaring the account and the operator issuing it. The
|
||||
machine keeps what it runs meanwhile.
|
||||
- The internal name of a route that also has a public one is now served to the mesh only. Outsiders
|
||||
have the public name.
|
||||
- A container on the same machine that calls that machine's own internal name arrives from its
|
||||
container network, not from a mesh address, and is refused. Calls between machines are unaffected:
|
||||
they leave by the machine's mesh address. Whether the mesh should also issue each machine's container
|
||||
networks is left open, because the mesh does not record them today.
|
||||
|
||||
## How this is checked
|
||||
|
||||
| Rule | Checked by |
|
||||
|---|---|
|
||||
| What a provider receives on the bus is what its received file says, same-node port fix included | a controller test composing a provider and a consumer on one machine and comparing the two |
|
||||
| An internal name is served to the machines the membership names and to loopback, and to nobody else | the proxy's tests: served from a named address and from loopback; refused, unlisted and uncertified from any other |
|
||||
| A membership that carries no routes, or a mesh that cannot be read, changes nothing | the proxy's tests |
|
||||
| Until the mesh is issued, an internal name is served to the machine alone | the proxy's tests |
|
||||
| Live: an internal-only route answers over the mesh and is refused from outside | by hand, after the release |
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0160](0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md) —
|
||||
the membership this extends
|
||||
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — reach, and the
|
||||
insight of 2026-10-02 that the proxy is where internal reach is kept
|
||||
- [ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md) — the machine itself is always inside
|
||||
- [Issue 191](../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md) — what
|
||||
found it
|
||||
@@ -177,7 +177,6 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0161** — [What deserves a seat: a role of a module is a seat, a singular fact about machines is a placement with a capacity of one, and a holder's software is the machine's](0161-what-deserves-a-seat.md)
|
||||
- **0162** — [A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
|
||||
- **0163** — [Taking a module over is a comparison: what it compares, what it refuses, and what it carries](0163-taking-a-module-over-is-a-comparison.md)
|
||||
- **0167** — [A membership carries what its module receives, and who the mesh is](0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
@@ -7,9 +7,8 @@ code:
|
||||
- mesh-controller internal/identity/authority.go
|
||||
- mesh-host internal/identity/serving.go
|
||||
- mesh-host internal/apply (the service that reflects a rule set)
|
||||
updated: 2026-10-02
|
||||
updated: 2026-09-30
|
||||
decisions:
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0148-the-meshs-names-are-resolved-not-copied-into-containers.md
|
||||
- 02-DECISIONS/0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md
|
||||
- 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
|
||||
@@ -843,18 +842,6 @@ One value, three readers:
|
||||
| `public` | the machine port, to anywhere | the public name | the public authority |
|
||||
| `both` | the machine port, to anywhere | both names | each name's own authority |
|
||||
|
||||
*2026-10-02.* **The proxy serves an internal name to the private network only**
|
||||
([ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md),
|
||||
its insight of this date). It answers public and internal names on the same listeners, so the name a
|
||||
request carries is the request's own claim, not where the request came from. An internal name is
|
||||
served to the machines of the mesh and to the machine itself; to anyone else it is answered as a name
|
||||
never routed, in the handshake as well as the request. Without this, an endpoint with reach `internal`
|
||||
would be public under a name that is easy to guess
|
||||
([issue 191](../../04-ISSUES/191-a-route-with-only-an-internal-name-is-dropped/00-report.md)).
|
||||
The proxy is told who the mesh is, and its routes, in its membership on the bus — the same machine
|
||||
addresses the filter's "from the mesh" is rendered from
|
||||
([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)).
|
||||
|
||||
**An endpoint that is not routed is reached and never named.** No route contribution means no name is
|
||||
composed and no certificate requested, while the filter still acts on it. That is the case the model
|
||||
could not express at all, and it is the ordinary case for anything that is not HTTP.
|
||||
|
||||
@@ -7,9 +7,8 @@ code:
|
||||
- mesh-tools src/broker-amqp.ts (to be replaced)
|
||||
- mesh-catalog modules/nats (to be written)
|
||||
- mesh-sdk src (the protocol's NATS binding, step 3)
|
||||
updated: 2026-10-02
|
||||
updated: 2026-10-01
|
||||
decisions:
|
||||
- 02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md
|
||||
- 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
|
||||
- 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
@@ -95,12 +94,6 @@ list; the account's grant is the same membership read the other way; the console
|
||||
tool's subject. The one rule a runtime keeps is the membership's own subject, from the two names in its
|
||||
credential.
|
||||
|
||||
**Revised 2026-10-02** ([ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)):
|
||||
a membership also carries **what its module receives**, by requirement — the contributions its
|
||||
received file is written from, from the same composition — and **who the mesh is**, every machine's
|
||||
address on the private network, the list the filter's "from the mesh" is rendered from. The route
|
||||
proxy is the first reader of both.
|
||||
|
||||
**Revised 2026-09-27** ([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)):
|
||||
**`mesh.build.request`, `mesh.control.built` and the BUILDS stream are gone.** A build is work submitted to a role, and the
|
||||
mesh already has a shape for that — a seat's `accept` subjects, on a work queue with a queue group of
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: proposed
|
||||
code: []
|
||||
updated: 2026-09-27
|
||||
status: in-progress
|
||||
code:
|
||||
- mesh-controller internal/inventory
|
||||
- mesh-controller internal/catalogue
|
||||
- mesh-controller cmd/mesh-controller
|
||||
updated: 2026-10-01
|
||||
decisions:
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md
|
||||
@@ -14,10 +17,10 @@ decisions:
|
||||
# 29 — A node has operator accounts, and the mesh owns what lives under a home
|
||||
|
||||
**The mesh models machines but not the people on them.** A node record holds its name, its
|
||||
address, its mode — and nothing about *who a person is* on it: `jochens` on novox, `ace` on ace,
|
||||
`jochen` on shanks and g14. That username is not incidental. It decides who a file under `~` is
|
||||
owned by, who a user service runs as, and — the case that surfaced this — which account `ssh
|
||||
<node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
address, its mode — and nothing about *who a person is* on it: one login name on the build node,
|
||||
another on the home-server, a third on both workstations. That username is not incidental. It
|
||||
decides who a file under `~` is owned by, who a user service runs as, and — the case that surfaced
|
||||
this — which account `ssh <node>` logs in as. The predecessor knew it (its per-node `user:`, and the modules that wrote a
|
||||
person's `~/.ssh/config`, `~/.zshrc`, `~/.config`); the mesh, taking those over, kept the machine
|
||||
facts and dropped the human one.
|
||||
|
||||
@@ -28,8 +31,9 @@ Several things are missing, and they are one idea.
|
||||
A node has one or more **operator accounts**: the human logins on it. At minimum a name; the
|
||||
mesh already knows the node and its address, so `<account>@<node>` is then a complete answer to
|
||||
"who am I, where." It is the mesh's to hold because everything below is derived from it, and
|
||||
because it is exactly the fact that was silently lost — `ssh ace` failed to `ace` because nothing
|
||||
in the mesh said ace's account is `ace`.
|
||||
because it is exactly the fact that was silently lost — `ssh home-server` logged in under the
|
||||
workstation's own name, because nothing in the mesh said the home-server's account is a different
|
||||
one.
|
||||
|
||||
## 2. A resource may live under a home, owned by its account
|
||||
|
||||
@@ -65,14 +69,14 @@ create `~/.ssh` at `0700`, chown it to the account, and own the files it places
|
||||
|
||||
**The boundary — and it is the reason this is safe:** `~/.ssh` is the one directory where a wrong
|
||||
declaration locks a person out of their own machine. So the mesh's *found-vs-owned* semantics
|
||||
([ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
([ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md),
|
||||
adoption) apply *inside* the home directory. The mesh **owns** the directory and the files above; it
|
||||
**holds as found — never rewrites, never removes** — the operator's own contents: their **private
|
||||
keys** and their **personal drop-ins** (`config.d/personal`, the personal `Host` aliases a
|
||||
workstation carries, exactly as `hosts.local` is the home the mesh never rewrites for `/etc/hosts`).
|
||||
Reconcile removing an unassigned `config.d/mesh` is fine; the same logic aimed at `id_ed25519` or an
|
||||
operator's own `authorized_keys` entry is a lockout. This is the login-channel cousin of the rule
|
||||
[ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
[ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md) draws for the uplink and the sshd
|
||||
module draws for the firewall: **the mesh must never be able to arrange the one failure that severs
|
||||
its own way back in.** The carve-out is not a convenience; it is that rule, in `~/.ssh`.
|
||||
|
||||
@@ -108,12 +112,12 @@ found-vs-owned boundary of §3 is exactly what guarantees nothing already there
|
||||
|
||||
None of this needs a node to discover the mesh, and none of it needs a control-plane module of its
|
||||
own. The ssh files are **roster facts**
|
||||
([ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)): once the
|
||||
roster view carries a node's **host key** and its **account** beside its name and address, the
|
||||
`ssh-client` module ships a template for `known_hosts`, `config` and `authorized_keys`, and the
|
||||
controller renders each node's copy from the full roster and pushes it. The mesh owns the data; the
|
||||
module owns ssh's format; the control plane gains no ssh syntax. It is the same act as composing a
|
||||
peer list or `/etc/hosts` — which is why there is **no novox-only "mesh-ssh" module**: the
|
||||
peer list or `/etc/hosts` — which is why there is **no control-node-only "mesh-ssh" module**: the
|
||||
centralization is the controller's composition, not a module that runs somewhere. Only non-secret
|
||||
facts travel (names, addresses, accounts, host keys, the CA public key); the private key stays the
|
||||
operator's, placed as an operator-owned file, referenced by path.
|
||||
@@ -129,6 +133,44 @@ operator's, placed as an operator-owned file, referenced by path.
|
||||
They meet at the account and the CA, not at a bespoke module. The `sshd` server side already exists;
|
||||
the client/identity side and the CA are the open pieces.
|
||||
|
||||
## What has shipped, and what has not
|
||||
|
||||
*Recorded 2026-10-01 from the controller's main branch, not from intent.*
|
||||
|
||||
**Built (mesh-controller, merged 2026-09-27):**
|
||||
|
||||
- **§1, the account as a node fact.** A node record carries an operator account and, optionally,
|
||||
its home. Empty is a real state — a freshly enrolled or headless machine has no operator account
|
||||
known yet — and an empty home means *derive it* (the superuser's home for the superuser, the
|
||||
conventional per-user home otherwise), so the common case needs no entry. The controller's node
|
||||
command sets it. One account per node is what exists; "one or several" below is still open.
|
||||
- **§2, resources under a home.** The account and its home are offered as machine facts, and a
|
||||
resource's *path and owner* resolve placeholders exactly as its content does — so a module places
|
||||
a file under a person's home, owned by that person, naming neither. A roster file may say it lives
|
||||
under the home: it is rendered per node, placed under that node's account's home, chowned to the
|
||||
account, and a node with no account gets none.
|
||||
- **§5, the composed ssh config.** The roster rendering carries each node's account, so the
|
||||
`ssh-client` template can emit a `Host` block per node with the right login name. Composed
|
||||
end-to-end in the controller's tests.
|
||||
|
||||
**Written but not shipped:** the `ssh-client` catalogue module itself exists on a branch of the
|
||||
module repository; its pull request was closed with a hold until this design is deployed, and
|
||||
nothing has deployed it since. The predecessor's generator still writes every workstation's ssh
|
||||
client blocks today — which is where [issue 172](../../04-ISSUES/172-the-ssh-client-block-matches-one-spelling-of-a-machine/00-report.md)
|
||||
was found.
|
||||
|
||||
**Not built:** the SSH CA and certificates (§4), `known_hosts` and `authorized_keys` as roster files,
|
||||
the found-vs-owned boundary inside `~/.ssh` (§3 — the controller has no rule yet that refuses to
|
||||
rewrite a private key), adoption of existing keys, the ssh-agent as a user service, and user-scoped
|
||||
services in general. The host vocabulary still has no user-scope unit at all; a workstation's
|
||||
per-user daemons (a bar watchdog, a config reloader, an audio service masked per user) have no form
|
||||
the mesh can send.
|
||||
|
||||
**A gap this surfaced:** §1 shipped as code before it had a decision record. The account as a node
|
||||
fact, the home as a placement root, and what the mesh may and may not do under a home are each a
|
||||
decision this document names but no record states. They are the next records to write, before the
|
||||
family of §2 modules is built.
|
||||
|
||||
## Why now, and why not yet
|
||||
|
||||
**Why it matters:** when HAL retires, the generators that keep `~/.ssh`, shell config and the
|
||||
@@ -137,7 +179,7 @@ alias and its trust, and a fresh machine has no operator dotfiles at all — the
|
||||
service and leave the human unable to work on the box.
|
||||
|
||||
**Why not build it reflexively:** it is a real addition to the node model, the resource model, and
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0128 is what lets
|
||||
the seat set, and must be gotten right. The mechanism half is now settled — ADR 0120 is what lets
|
||||
the ssh files be templates with no control-plane format — so what remains to decide here is the
|
||||
model:
|
||||
|
||||
@@ -156,15 +198,18 @@ model:
|
||||
unnecessary, and forwarding an agent into a node exposes the operator's keys to that node's root —
|
||||
so prefer certificates and `ProxyJump` over forwarding.
|
||||
|
||||
**Not urgent, not blocking.** ssh and dotfiles work today because HAL's generators still run as the
|
||||
substrate. This becomes load-bearing in the node-by-node retirement phase, not before — which is the
|
||||
right time to build it, once the account and CA model are decided here.
|
||||
**Now load-bearing.** The migration of every node to the mesh is complete; what remains of the
|
||||
predecessor is exactly the user environment this design covers — ssh config, dotfiles, the desktop
|
||||
stack and the per-user services of the two workstations. Those generators are the last thing
|
||||
keeping the predecessor running, so the model questions above are no longer deferred: the account
|
||||
record, the home as a placement root, user-scoped services and the one-off steps a hook used to run
|
||||
each need a decision before the modules that replace the generators can be written.
|
||||
|
||||
## References
|
||||
|
||||
- The gap was found generating `~/.ssh/config` from the *HAL* registry (`hal/terminal`'s
|
||||
postConfigure hook), which the nox mesh has no equivalent for.
|
||||
- [ADR 0128](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
- [ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md) — the roster
|
||||
fact mechanism that renders the ssh files, format owned by the module.
|
||||
- [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) — the
|
||||
system-path placement this mirrors for home paths.
|
||||
@@ -173,6 +218,6 @@ right time to build it, once the account and CA model are decided here.
|
||||
- [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md) — the CA key is a secret the
|
||||
vault makes; [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md)
|
||||
— short-lived certs as rotation.
|
||||
- [ADR 0125](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0126](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
- [ADR 0117](../../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md),
|
||||
[ADR 0118](../../02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) —
|
||||
the never-sever-the-channel rule and the found-vs-owned semantics, applied here to `~/.ssh`.
|
||||
|
||||
@@ -38,7 +38,7 @@ document is written and this one's status becomes `implemented`.
|
||||
| [`26-the-seats.md`](26-the-seats.md) | **Proposed.** What a mesh can have one of, who fills each, and a seat's holder answering for the provision it delivers — including the `git` seat a build's source can live on | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)), [ADR 0111](../../02-DECISIONS/0111-a-build-source-is-on-the-git-seat-or-external.md), [ADR 0109](../../02-DECISIONS/0109-a-package-registry-seat-is-one-per-ecosystem.md) |
|
||||
| [`27-a-module-requires-the-mesh-resolves.md`](27-a-module-requires-the-mesh-resolves.md) | **Proposed.** One concept for everything a module needs: a requirement with a contract, answered by one of four kinds of provider, resolved at assignment or refused. Retires settings, placeholders, facts and paths in definitions | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md)) |
|
||||
| [`28-building-the-bus.md`](28-building-the-bus.md) | **Proposed.** The five steps of the bus work in the order their dependencies allow, each ending at a bed — with the surface measured, so no step's size is a guess | [ADR 0116](../../02-DECISIONS/0116-the-bus-is-built-in-five-steps.md), [ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md), [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **Proposed.** The mesh models machines but not the humans on them: a node gains operator accounts, and a resource may live under a home owned by its account — what would own ~/.ssh, dotfiles and ~/.config when HAL retires | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
| [`29-a-node-has-operator-accounts.md`](29-a-node-has-operator-accounts.md) | **In progress.** A node has an operator account and a resource may live under its home — built in the controller; the ssh-client module, the SSH CA, the `~/.ssh` boundary and user-scoped services are not. The account fact still wants its decision record | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md), [ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md) |
|
||||
|
||||
| [`32-what-a-module-declares.md`](32-what-a-module-declares.md) | **Proposed.** What a module declares and what the bus derives from it: three namespaces, subjects from local names, queues never declared, the five relationships, and the build-publish-deploy lifecycle on one bus | [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md), [ADR 0127](../../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md), superseded by [ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) (superseding [ADR 0125](../../02-DECISIONS/0125-the-bus-is-the-only-broker.md)), [ADR 0041](../../02-DECISIONS/0041-events-are-a-relationship.md) |
|
||||
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-01
|
||||
located-in: [mesh-controller examples/route-proxy/main.go (routesFrom requires a route's public `name` and treats `internal-name` only as an alias of it; the handler serves every routed name to any source), mesh-controller internal/broker/membership.go (a membership says nothing of what its module receives or who the mesh is)]
|
||||
fixed-by:
|
||||
amended-design: [03-DESIGN/01-to-be/08-connectivity.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md]
|
||||
---
|
||||
|
||||
# 191 — A route with only an internal name is dropped as naming nothing
|
||||
|
||||
## What was observed
|
||||
|
||||
A module whose endpoint reaches only the private network could not be reached by its internal name.
|
||||
The module ran and answered on its own port. Its route's internal name resolved to the serving node.
|
||||
The request failed during the TLS handshake:
|
||||
|
||||
```
|
||||
http: TLS handshake error from …: no public route for "unifi.home-server.internal" in this mesh,
|
||||
so no certificate is asked for
|
||||
```
|
||||
|
||||
The proxy's own log said why, every time it re-read its routes:
|
||||
|
||||
```
|
||||
unifi on home-server asked for a route and named nothing; skipped
|
||||
```
|
||||
|
||||
The route it skipped was not empty. The mesh had given it an endpoint, a port, a scheme and an internal
|
||||
name, and no public name:
|
||||
|
||||
| route | `name` | `internal-name` | served |
|
||||
|---|---|---|---|
|
||||
| home-assistant | a public name | `home-assistant.home-server.internal` | under both |
|
||||
| unifi | — | `unifi.home-server.internal` | under neither |
|
||||
|
||||
Three other modules on the same node were skipped with the same line on the same pass.
|
||||
|
||||
## Why it matters
|
||||
|
||||
**Reach is decided in one place, and the proxy reads the old shape of the decision.**
|
||||
[ADR 0138](../../02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md)
|
||||
made an endpoint's reach decide which names exist. The controller composes the public name, the
|
||||
internal name, or both, and composes no name that nobody asked for
|
||||
([issue 140](../140-an-endpoints-reach-is-not-declared/01-resolution.md)). An endpoint that reaches
|
||||
only the private network is the ordinary case for anything that should not face the internet. It is
|
||||
exactly the case the proxy drops.
|
||||
|
||||
**The failure is quiet and points the wrong way.** Nothing marks the module unhealthy. The handshake
|
||||
error says *no public route*, which reads as a certificate fault on the reader's side. The line that
|
||||
gives the real cause is one of four identical lines repeated every few seconds in a log nobody reads
|
||||
until they already suspect the proxy.
|
||||
|
||||
**The opposite move is not a workaround.** Giving the endpoint public reach makes the proxy serve it.
|
||||
It also publishes an administration interface to the internet to get a name on the private network.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should the proxy refuse a route it cannot serve in a way the controller or an operator sees, not
|
||||
only in its own log? The same silent skip covers a route with no usable port or an unknown scheme.
|
||||
- What checks that what the controller composes and what the proxy serves stay the same shape? ADR
|
||||
0138 changed one side and nothing failed on the other.
|
||||
@@ -1,75 +0,0 @@
|
||||
# Diagnosis
|
||||
|
||||
*2026-10-01.*
|
||||
|
||||
**Ruled out first: the module itself.** Its container was up and had not restarted. The controller's
|
||||
status endpoint answered on its own port with `"up": true`. The tool wrapper beside it was serving
|
||||
all its tools.
|
||||
|
||||
**Ruled out: name resolution.** The internal name resolved to the serving node's private-network
|
||||
address, which is where the proxy listens. Plain HTTP to the name reached the proxy and got a 404.
|
||||
HTTPS failed in the handshake, and the proxy logged that it had no route for the name.
|
||||
|
||||
**The route as the proxy received it.** The mesh-written route file held a complete contribution
|
||||
for the module: endpoint `web`, port, scheme `https`, `insecure`, a label, and `internal-name`. It had
|
||||
no `name`. That is what the controller composes for an endpoint whose reach stops at the private
|
||||
network (ADR 0138, `composeName`). The contribution was correct.
|
||||
|
||||
**Located: `routesFrom` in the proxy.** It reads `name` first and skips the contribution if `name`
|
||||
is empty. It reads `internal-name` only at the end, as a second host for a rule that already has a
|
||||
public one. So the proxy can serve an internal name only next to a public one. That matched the
|
||||
mesh before ADR 0138, when both names were always composed. It has been wrong since then.
|
||||
|
||||
The other half of the proxy already handles the case. Certificates for a host are split by whether it
|
||||
is in the public set: hosts outside it go to the internal authority, and only hosts inside it are
|
||||
eligible for ACME. A host that is only ever an internal name falls on the correct side of both checks
|
||||
without change. For certificates, only reading the route was wrong; who may reach the route is the next section.
|
||||
|
||||
**The fix.** `routesFrom` takes a route that names either host, serves each name it carries, and
|
||||
marks only the public one as public. It still skips a route that names neither, with the same log line.
|
||||
A test proves an internal-only route is served, certified by the internal authority, and refused by
|
||||
the public one. That test fails against the code before the change.
|
||||
|
||||
## The first fix would have made the name public — 2026-10-02, from review
|
||||
|
||||
Serving the dropped route was not enough. The proxy picks a route from the name a request carries
|
||||
and never from where the request came from, and it answers public and internal names on the same
|
||||
listeners. Its public names resolve to an address the internet reaches. So once the internal-only
|
||||
route was served, any request from the internet carrying `unifi.home-server.internal` — a name of a
|
||||
fixed, guessable shape — would have reached an administration interface that reach `internal` was
|
||||
chosen to keep private. Before the fix the route was unreachable from everywhere. After it, it would
|
||||
have been reachable from everywhere. Two more leaks came with it: the proxy's answer for an unrouted
|
||||
name listed every name it serves, internal ones included, and the handshake handed a certificate
|
||||
naming the internal host to any client.
|
||||
|
||||
Nothing showed this while every routed endpoint also had a public name: its internal name exposed
|
||||
nothing the public one did not. It is a gap in the decision's wording, not only in the proxy — ADR
|
||||
0138 says the proxy *serves* the internal name without saying to whom — so it is recorded there as a
|
||||
progressive insight and in the to-be connectivity design.
|
||||
|
||||
**Where "inside" is decided: told, not worked out.** The first correction had the proxy work it out
|
||||
for itself — the mesh's range from an environment variable the catalogue wrote, and the machine's
|
||||
container bridges from its own interfaces. That was a second definition of "the mesh", kept by one
|
||||
module beside the one the controller already has: it resolves "from the mesh" to every machine's
|
||||
address on the private network, and the packet filter is rendered from that list. Reviewed, it was
|
||||
replaced: [ADR 0167](../../02-DECISIONS/0167-a-membership-carries-what-its-module-receives-and-who-the-mesh-is.md)
|
||||
has every membership on the bus carry what its module receives and that list, and the proxy follows
|
||||
its membership. One composition, read by the filter and by the proxy.
|
||||
|
||||
The proxy reads the source address, where the guard reads the interface, because it cannot see the
|
||||
interface a request arrived on. A claimed source does not carry here: a connection needs its replies,
|
||||
and replies to a mesh address leave by the tunnel.
|
||||
|
||||
**What changed with it.** The internal name of a route that also has a public one is now served to the
|
||||
mesh only, like any other internal name. Outsiders have the public name, so nothing they could reach is
|
||||
lost. A container calling its own machine's internal name arrives from its container network and is
|
||||
refused; whether the mesh should issue those networks too is left open in ADR 0167.
|
||||
|
||||
**Order of release.**
|
||||
|
||||
1. The catalogue change, which gives the proxy a bus account. A machine running the proxy is not
|
||||
composed until its account is issued, so the account is issued straight after
|
||||
(`module issue route-proxy --node <machine>`), and then the machine is pushed.
|
||||
2. The controller and proxy change. The push after it publishes memberships that carry the routes and
|
||||
the mesh, and each proxy takes them. Until then, a proxy serves its file, and internal names to its
|
||||
own machine alone.
|
||||
Reference in New Issue
Block a user