50 lines
2.5 KiB
Markdown
50 lines
2.5 KiB
Markdown
---
|
|
status: resolved
|
|
opened: 2026-10-02
|
|
located-in:
|
|
- mesh-controller
|
|
fixed-by: mesh-controller PR #232
|
|
amended-design:
|
|
---
|
|
|
|
# 204 — A controller handover re-sent every node a declaration composed from a stale view
|
|
|
|
## What was observed
|
|
|
|
2026-10-02 21:29 UTC. Two machines were assigned the node tools runtime and had the console taken off
|
|
them, and were pushed; the host on each applied it (the console removed, the runtime created and
|
|
running). Two seconds later, on each, a second declaration arrived that undid it — the host's log:
|
|
|
|
```
|
|
23:29:18 created node-tools.runtime (node-tools): 239 file(s), running as node-tools.service
|
|
23:29:18 applied 569 resource(s)
|
|
23:29:20 removed node-tools.runtime (node-tools)
|
|
23:29:20 forgotten node-tools.interpreter (nodejs)
|
|
23:29:24 created mesh-console.needs-broker, created mesh-console.server (mesh-console)
|
|
```
|
|
|
|
The second declaration had the console assigned and the runtime absent: the assignments as they were
|
|
a minute earlier. The controller's status knew of one send per machine, the person's. At that minute a
|
|
plan from an unrelated merge was rolling a new controller build onto the control node, so an instance
|
|
was starting while another was stopping. A third push by hand, two minutes later, restored both
|
|
machines and nothing undid it again.
|
|
|
|
Which instance sent the stale declaration, and from what, is not established: the outgoing one on its
|
|
way down, the incoming one at start-up before its view was current, or the rolling plan sending what it
|
|
had composed when it was made — the same family as [issue 201](../201-a-push-recreated-the-controller-behind-the-row-its-successor-wrote/00-report.md),
|
|
where a plan sent a digest older than the one a successor had written.
|
|
|
|
## Why it matters beyond this instance
|
|
|
|
A declaration the mesh sends is the mesh's word on what a machine should be; a machine applies it in
|
|
full, including removing what it no longer names. A stale one is therefore not a no-op: it tears down
|
|
whatever was assigned since, and the controller's own record does not show the send, so the next
|
|
person reads "applied, current" over a machine that is wrong. Two people merging within a minute is
|
|
ordinary, and a controller handover happens on every controller merge.
|
|
|
|
## Questions HQ must answer
|
|
|
|
- Where does the stale view come from, and is every send recorded so status can show it?
|
|
- Should a declaration carry the assignment generation it was composed from, so a host refuses one
|
|
older than the last it applied, as it already refuses a declaration it cannot verify?
|