Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0dce7faf20 | ||
|
|
0b1ddaa7e9 | ||
|
|
e7126bc1e6 |
@@ -0,0 +1,81 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-10-05
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md
|
||||
---
|
||||
|
||||
# 217. No change to a machine takes effect unseen
|
||||
|
||||
## Context
|
||||
|
||||
Two incidents in two days had one shape: **a change took effect that nobody saw before it did.**
|
||||
|
||||
- Issue 241: a provisioner read an unreadable grants file as "nobody asks" and withdrew every
|
||||
consumer; the postgres provider dropped seven databases. Nothing announced the withdrawal before it
|
||||
happened.
|
||||
- Issue 246: one placement was set for one module on one machine with `settings set`. The layer is
|
||||
replaced whole, so the module's other settings on that machine — three directory placements, eight
|
||||
media accesses, an exposure, four endpoints, the account's ids — were dropped, and the command
|
||||
answered only "places". The machine's plan then ran the module on an empty configuration
|
||||
directory. It was caught by reading the plan before a push, and the lost layer was read back from a
|
||||
database backup a few hours old.
|
||||
|
||||
Since issue 241 the host no longer destroys what it did not make (ADR 0030 for directories, issue
|
||||
241's fix for files) and no provider drops a store. What remains is the class above: the mesh doing
|
||||
exactly what it was told, when what it was told was not what the person meant, and nothing showing
|
||||
the difference until a machine had changed.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Confirm every push.** Rejected: every push would ask, and a confirmation asked every time is
|
||||
answered without reading. The guard must speak only when something is at stake.
|
||||
2. **Make `settings set` merge.** Rejected: removing a key would then need a second command, and the
|
||||
layer stops being a statement of the whole (ADR 0046, ADR 0164). The fault is that the removal was silent,
|
||||
not that it was possible.
|
||||
3. **Three guards, each where the change is made, each silent when nothing is at stake.** Adopted.
|
||||
|
||||
## Decision
|
||||
|
||||
**A change that removes, moves or replaces something a machine is running is shown before it takes
|
||||
effect; one that only adds is not interrupted.**
|
||||
|
||||
1. **A settings layer is read before it is replaced, and what a replacement removes is said.**
|
||||
`settings show` (and the verb) reads a layer. `settings set` answers with the keys it adds, changes
|
||||
and removes, and **refuses to remove a key unless told `--replace`**. Every change keeps the layer
|
||||
it replaced, so the previous value is one command away, not in a backup.
|
||||
2. **A push says what it will change.** `plan <node> --diff` lists what differs from what the machine
|
||||
was last sent: resources added, changed and removed, and every container that will be recreated
|
||||
with the reason. **`push` with no machine named is refused** unless `--all` says so.
|
||||
3. **Moving a running module's data is acknowledged.** When a push would give a running container a
|
||||
different host directory at a mount it already has — its data moving, or not following — the push
|
||||
to that machine is held, naming the module, the mount and both directories; the other machines in
|
||||
the same push go ahead. It is sent when the operator says so for that module (`--move <module>`).
|
||||
|
||||
None of the three adds a step to an ordinary change: a new module, a rebuilt image, a setting that
|
||||
only adds keys and a push to a named machine behave exactly as before.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The controller keeps what it last sent each machine**, not only its digest, so a plan can be
|
||||
compared with it. Migration 0014 declined to keep it — "a second account of what a machine should
|
||||
be" — and that reason stands for what it was about: the copy is never read as what a machine
|
||||
*should* be, only as what it *was told*, which is the one thing a comparison needs and the digest
|
||||
cannot give.
|
||||
- Settings carry a history; `settings show` reads it.
|
||||
- A script that ran `push` with no machine must say `--all`.
|
||||
- A changed placement for a running module needs one acknowledgement, once.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Tests in the controller: `settings set` that drops a key without `--replace` is refused and changes
|
||||
nothing; with it, the answer names the removed key and the history holds the previous layer. `plan
|
||||
--diff` of an unchanged node is empty. A push changing a running container's mount source is held,
|
||||
and goes ahead with `--move` for that module. `push` with no machine and no `--all` is refused.
|
||||
|
||||
## References
|
||||
|
||||
- [04-ISSUES/241](../04-ISSUES/241-one-unreadable-grants-file-dropped-every-database-on-the-control-node/00-report.md), [04-ISSUES/246](../04-ISSUES/246-setting-a-modules-settings-replaces-the-whole-layer-and-nothing-shows-it-first/00-report.md)
|
||||
- ADR 0030 (data outlives its declaration), ADR 0046 (a module's configuration is its assignments), ADR 0164 (a setting is declared with its default and its cost), ADR 0214 (backups)
|
||||
@@ -194,6 +194,7 @@ python3 00-META/checks/index.py fail if stale
|
||||
- **0207** — [A module depends on the node seats that apply its resources](0207-a-module-depends-on-the-node-seats-that-apply-its-resources.md)
|
||||
- **0210** — [A tool's configuration is its seat holder's, and every other module extends it through the seat](0210-a-tools-configuration-is-its-seat-holders-and-every-other-module-extends-it-through-the-seat.md)
|
||||
- **0212** — [A seat says what it receives, and the machine's hotkeys are a seat](0212-a-seat-says-what-it-receives-and-the-machines-hotkeys-are-a-seat.md)
|
||||
- **0217** — [No change to a machine takes effect unseen](0217-no-change-to-a-machine-takes-effect-unseen.md)
|
||||
|
||||
### Its tiers, from the bottom up
|
||||
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
layer: to-be
|
||||
status: in-progress
|
||||
code:
|
||||
- mesh-controller: cmd/mesh-controller/unseen.go, cmd/mesh-controller/push.go (--all, --move, the hold), cmd/mesh-controller/plan.go (--diff), cmd/mesh-controller/modules.go (settings show, --replace), internal/inventory/unseen.go, internal/inventory/migrations/0059-what-a-change-replaces.sql
|
||||
updated: 2026-10-05
|
||||
decisions:
|
||||
- 02-DECISIONS/0217-no-change-to-a-machine-takes-effect-unseen.md
|
||||
- 02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md
|
||||
---
|
||||
|
||||
# 44 — No change to a machine takes effect unseen
|
||||
|
||||
**A change that removes, moves or replaces something a machine runs is shown before it takes effect;
|
||||
a change that only adds goes through as before** (ADR 0217). Three guards, each at the place the
|
||||
change is made.
|
||||
|
||||
## 1. A settings layer is read before it is replaced
|
||||
|
||||
A module's settings are layers — the whole mesh, and one per machine — each replaced whole when set.
|
||||
That stays. Around it:
|
||||
|
||||
- **Reading.** `settings show <module> [--node]` prints the layers as they are, and the console verb
|
||||
answers the same.
|
||||
- **Saying what changed.** Setting a layer answers with the keys it adds, changes and removes,
|
||||
compared with the layer it replaces.
|
||||
- **Refusing a silent removal.** A set that would remove a key is refused, naming the keys, unless
|
||||
`--replace` says the removal is meant. A set that only adds or changes keys needs nothing.
|
||||
- **Keeping the previous layer.** Each set and each clear records the layer it replaced and when, so
|
||||
the previous value is read back with `settings show --history`, not from a backup.
|
||||
|
||||
## 2. A push says what it will change
|
||||
|
||||
- **What was sent is kept.** Each send records the declaration it sent the machine, beside the digest
|
||||
already kept. It is read only to compare; what a machine *should* be is still composed from the
|
||||
mesh's records every time.
|
||||
- **`plan <node> --diff`** compares what would be sent now with what was sent last: resources added,
|
||||
removed and changed by id, and for a changed container the fields that differ — which is what
|
||||
recreates it. A machine with nothing to change shows nothing.
|
||||
- **`push` with no machine is refused** unless `--all` names that intent. A push to one machine, or
|
||||
`--behind`, is unchanged; the console verb never pushed every machine and still does not.
|
||||
|
||||
## 3. Moving a running module's data is acknowledged
|
||||
|
||||
Before sending, each machine's declaration is compared with what it was last sent. **A container
|
||||
that keeps a mount at the same place inside it, with a different directory on the machine behind
|
||||
it**, is a module's data moving — or, as on 2026-10-05, a module about to run on an empty directory
|
||||
because a placement was lost. That machine's push is held, saying the module, the mount and both
|
||||
directories; the other machines in the same push go ahead. `push … --move <module>` sends it.
|
||||
|
||||
A first send to a machine, a new container, a mount added or removed, and a changed image are not
|
||||
moves and are not held.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Controller tests: a set dropping a key without `--replace` is refused and changes nothing, and with
|
||||
it the answer names the removed key and the history holds the layer it replaced; `plan --diff` of an
|
||||
unchanged machine is empty and names a changed container's changed field; a push changing a running
|
||||
container's mount source is held for that machine only and goes with `--move`; `push` with no machine
|
||||
and no `--all` is refused.
|
||||
Reference in New Issue
Block a user