jschoubben
eba24a72af
Everything should be able to call what runs on the same machine, another machine's service exposed to the private network, and another machine's service exposed publicly. Three cases; the filter had two. The first was expressed as the machines' own addresses on the private network. A caller on the machine carries such an address; a caller in one of its containers carries a bridge address and matched nothing — measured, same destination and same machine: src 10.10.0.1 against src 172.17.0.8. The second case worked by accident, because the tunnel rewrites a caller's address to the sending machine's. Two of three working is why it read as correct. 0143 answered the wrong question. It proposed verifying each grant from the consumer's own network position and went to length about which position, because whether a caller sat in a container changed the answer — and that difference was the bug. Observing a configuration error is not its remedy. Superseded, and nothing replaces it; whether the mesh should check a grant is still open in issue 145 and must stand on its own. And a module is not a container: 61 of 72 happen to use one, 11 do not, and a rule reasoning about containers describes most of the mesh rather than the mesh.
7.0 KiB
topic, status, date, deciders, reconstructed, extends, supersedes
| topic | status | date | deciders | reconstructed | extends | supersedes |
|---|---|---|---|---|---|---|
| what runs on it | accepted | 2026-09-29 | jochen | false | 02-DECISIONS/0045-a-machine-firewall-is-the-sum-of-what-it-listens-on.md | 02-DECISIONS/0143-a-consumer-verifies-the-grant-it-is-given.md |
144. Anything on a machine may call anything on it, and that is the whole of "local"
Context
Everything in the mesh should be able to call:
- what runs on the same machine;
- another machine's service over the private network, if that service is exposed there;
- another machine's service over the public network, if it is exposed there.
Three cases. The filter had two of them.
The first was broken and the break was invisible. A service exposed to the private network rendered as the machines' own addresses on it. A caller on the machine carries such an address; a caller inside one of that machine's containers carries a bridge address and matched nothing. Measured:
the machine: local 10.10.0.1 dev lo src 10.10.0.1
a container: 10.10.0.1 via 172.17.0.1 dev eth0 src 172.17.0.8
Same destination, same machine, two source addresses. The rule named the first and silently refused the second, so a module reaching its database on its own machine's name timed out for eleven hours (issue 145).
The second case works, and by accident. A caller on another machine reaches the private network over the tunnel, and arrives carrying that machine's own address — so the rule matches. It would not have matched the caller's own address either; the tunnel rewrites it. That two of three cases worked is why this looked correct.
ADR 0143 answered the wrong question. Written hours earlier, it proposed that a consumer verify each grant it is given by opening a connection from its own network position — and it went to some length about which position, because whether a caller sat in a container changed the answer. That difference was the bug. A verification mechanism would have reported this outage sooner and would not have prevented it, and the machinery it needed existed only because the rule was wrong. The remedy for a configuration error is the correct configuration.
And a module is not a container. A module is software that delivers one or more services, and it may do that as a container, an installed package with a unit, a binary, or files something else reads. Of 72 modules in the catalogue, 61 happen to use a container and 11 do not — among them the resolver, the ssh daemon and the intrusion-prevention module. A rule that reasons about containers describes most of the mesh and not the mesh.
Considered Options
- A line per service admitting the machine's own callers. Rejected: it is what was written first, and it only ever covers the services somebody remembered to think about. It also states, service by service, a thing that is true of the machine.
- Verify each grant from the consumer's position (ADR 0143). Rejected as a remedy: it observes the fault rather than removing it, and the question it agonised over — which network position — exists only while the fault does.
- Enumerate the addresses a machine's callers may have. Rejected for the reason no address is named anywhere in this filter any more (ADR 0140): a range describes one machine and goes stale in silence.
- Local is not filtered, stated once. Adopted.
Decision
Anything on a machine may call anything on that machine, and the filter says so once. Not per service, not per port, and not by naming who the callers are: traffic that did not arrive from outside the machine and did not arrive over the private network is the machine's own, and is admitted. It is asked by the link the traffic arrived on, because that is a fact about the machine rather than a list that describes one.
Local is not a boundary this mesh draws. Whether a caller is a container, a unit, or the operator's shell changes nothing, because the thing being decided is "is this the same machine" and the answer does not depend on the form the caller takes.
The other two cases are unchanged and are now legible beside it. A service exposed to the private network admits the machines on it; a service exposed publicly admits anything. Three cases, three lines, and a reader can see all three at once.
ADR 0143 is superseded and nothing replaces it. Whether the mesh should check that a grant works is a real question — it reported this machine healthy for eleven hours — but it is a question about what the mesh can say, not about what it should do, and it must stand on its own rather than as the remedy for a rule that was wrong. It is not built.
Consequences
- The three things everything should be able to call are three lines, and the first is one line rather than one per service, so a service added tomorrow is reachable locally without anybody remembering to say so.
- A form of module stops mattering to the filter. The 11 modules that are not containers were never affected by this bug and were never the reason it was hard to see; they are the reason the rule should never have mentioned containers.
- The mesh still cannot say when a grant stops working. That is the live gap, recorded in issue 145 and no longer pretending to have an answer.
- What got harder: nothing. This removes a line per service and replaces it with one.
How it is checked
- A caller on the machine reaches a service on it, in the input chain, asserted on that chain's own body — because the forward chain carries the same line in the same words, and an assertion on the whole rendered file passed with the input chain's copy deleted. That is what ADR 0137's tests already say to do.
- It is one rule, not one per service. Asserted by rendering two services of different reach and refusing a per-port local line.
- The three reaches render as three lines, asserted together, so the whole of what the filter says about who may call what is one test.
- The measured case: from a container on the machine, a service exposed to the private network on that machine answers. This is the outage, and it fails against the rule this replaces.