Compare commits

...
Author SHA1 Message Date
jschoubben 38dbe1d3a3 Let a node agent hear the creation of the seat's placement and candidates, so it binds them with no timer (ADR 0567 §9, issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d7c790d2422c
mesh/delivery-group group feat/1126-grants-first merged: every member is merged
mesh/check pass 38dbe1d3a388 on 98f2f2406921 | unit 1909 run, 715 kept, 3 skipped | integration 1931 run, 813 kept, 3 skipped | over bound: unit (slow…
2026-10-28 18:47:13 +01:00
jschoubben a2dbcf670b Bound a proposal's answers and ask by pulse keys whose expiry the holder hears, and name a hand-back proposal, so no timer waits (ADR 0567 §4, §7, issue 1126) 2026-10-28 18:43:39 +01:00
jschoubben e987a8280b Let a module's config values be proposed before the lease write: only a layer's places and accesses wait (ADR 0567 §13, issue 1126) 2026-10-28 18:42:05 +01:00
jschoubben 45f5f76811 Keep the placement ask among the assigner's asks: its Approve's level, not its kind, needs the proof (ADR 0567 §5, issue 1126) 2026-10-28 18:30:00 +01:00
jschoubben 4b4a173d84 Let withdraw mark a proposal for its holder to end, since the holder's process alone says the seat's events (ADR 0567 §2, issue 1126) 2026-10-28 18:21:59 +01:00
jschoubben 0282229fd9 Name the placement ask and a proposal's again and warrant, so the holder asks through the teller as its other asks (ADR 0567 §5, §10, issue 1126) 2026-10-28 18:20:27 +01:00
jschoubben 5a2a7eebfd Hold that a check's node names reach the digest tool too, so a kept verdict misses when they change (ADR 0567 §12, issue 1126) 2026-10-28 18:17:08 +01:00
jschoubben 004bbc2ce2 Carry every node's name on a check ask to the names check, so no repository lists them (ADR 0567 §12, issue 1126) 2026-10-28 18:13:57 +01:00
jschoubben fed64292f0 Read build capacity as an integer and build hours as one window, default 0: no repository names a node (ADR 0567 §12, issue 1126) 2026-10-28 18:13:18 +01:00
jschoubben 7a82384e21 Grant a node's agent the seat's placement and candidates, so it composes from the seat's state (ADR 0567 §3, §9, issue 1126) 2026-10-28 18:12:09 +01:00
jschoubben 7c565f2aa5 Move Intended and JudgeAnswers into mesh-sdk for a proposal: a difference is shown, only a refusal or silence fails (ADR 0567 §4, issue 1126) 2026-10-28 18:11:18 +01:00
jschoubben edc6f269f3 Name the seat's states, candidates by proposal and the proposal itself, so readers and the holder share one contract (ADR 0567 §1-§6, issue 1126) 2026-10-28 18:09:38 +01:00
jschoubben 902e3c9e96 Refuse placement files on a forge message and delete placement-seed: no repository holds placement (ADR 0567 §11, issue 1126) 2026-10-28 18:09:38 +01:00
jschoubben c16e6213e1 Define the seat mesh-placement with its seven verbs, so placement has one contract whoever holds it (ADR 0567, issue 1126) 2026-10-28 18:05:16 +01:00
mesh-tools 98f2f24069 Merge pull request 'A node holds the one module it cannot compose and applies the rest, as its shadow already answers (group fix/1126-apply-culprit, issue 1126)' (#118) from fix/1126-apply-culprit into main 2026-10-28 16:34:46 +00:00
jschoubben 0de3006415 A module refused alone is held, composed as what runs of it, on the node that applies as in its answer: one rule for both paths (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@98f2f2406921
mesh/delivery-group group fix/1126-apply-culprit merged: every member is merged
mesh/check pass 0de30064156a on 7aedd28f01dc | unit 2667 run, 590 kept, 36 skipped | integration 2676 run, 633 kept, 29 skipped | over bound: unit (sl…
The applying path (frombus.Compose) composed with no culprit, so one module's
refusal came back as the whole node's: novox's apply would have refused all its
modules for de-spiegel's unset route names. composeEach is now the one rule
Compose and Answer both follow:
- a module whose stub changes the refusal is held (compose.Node.Held); in the
  applying path a stand-in for a module placed here holds that module; only a
  refusal of no single module refuses the node;
- a held module is composed from what runs of it (compose.Carried of its own
  composition in Running) and carried in the node's own wherever that runs it,
  folded or leaving the fold, so the node's own apply never removes it (issue
  1945 by a new path) and what the others read of it is what runs;
- none of a held module's own composition is in what is applied;
- a module whose composition differs when the held ones are composed as stubs
  is held too; the node's own differing refuses the node;
- since a held module is composed from its new manifest, what reads that
  manifest is held with it: the providers of what it contributes to (the
  proxy its routes) and, when it listens, the filter's holder.

Quick fix weighed: a culprit loop in Compose alone. Thorough, chosen: one
shared function, since the kind met twice (seed-8 shadow, now the apply), and
review rounds 1 and 2 found its leaks.
2026-10-28 17:22:35 +01:00
mesh-tools 7aedd28f01 Merge pull request 'A module's own refusal refuses that module, not the node; contributions say unset values in config words; a folded module owns its manifest's paths (issue 1126, group fix/1126-seed-8)' (#117) from fix/1126-seed-8 into main 2026-10-28 13:27:05 +00:00
jschoubben f387581b6b A module's own refusal refuses that module, not the node: a stub whose contributions still fail the same is tried bare (issue 1126, seed-8: novox refused the whole candidate for de-spiegel's unset route names)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7aedd28f01dc
mesh/delivery-group group fix/1126-seed-8 merged: every member is merged
mesh/check pass f387581b6bfb on 3d021445f743 | unit 3018 run, 234 kept, 36 skipped | integration 3038 run, 266 kept, 29 skipped | over bound: unit (sl…
2026-10-28 14:07:59 +01:00
jschoubben eda82b1598 A folded module owns its manifest's paths, as the controller tells node-files them, not every path it runs at (issue 1126, seed-8: f1-circuits, snake and txt-game shifted node-tools' owned paths on ace) 2026-10-28 14:07:59 +01:00
jschoubben 06742d0448 A contribution's unset required value is refused in its module's own form's words, as a file's is (issue 1126, seed-8: de-spiegel in the config form was told ${setting:…} and settings set) 2026-10-28 14:07:59 +01:00
mesh-tools 3d021445f7 Merge pull request 'A build module may say its node is not eligible, so asks go to a node that can build (group fix/ace-tun)' (#116) from fix/ace-tun into main 2026-10-28 13:07:40 +00:00
jschoubben 1618837ea9 A build seat's puller takes nothing while its module finds its node cannot build, and says why
mesh/delivery merged: source.main moved to novox/mesh-sdk@3d021445f743
mesh/delivery-group group fix/ace-tun merged: every member is merged
mesh/check pass 1618837ea9fe on 6097e4f3e040 | carried from mesh-catalog 25f615456423 | unit 2293 run, 3984 kept, 16 skipped | integration 2331 run, 4…
A node whose module proves a need of the machine missing (ace's /dev/net/tun, 2026-10-28) pulled
image asks and failed each one; judged ineligible, its capacity is 0 with the reason, as build hours
are, so the asks go to a node that can build them (issue 1126 note).
2026-10-28 12:50:16 +01:00
mesh-tools 6097e4f3e0 Merge pull request 'A file's secrets are sealed values entry by entry, masked and named per secret (issue 1126; group fix/1126-seed-same-3)' (#115) from fix/1126-seed-same-3 into main 2026-10-28 11:40:35 +00:00
jschoubben 8520a4c099 Only a file's secrets are split entry by entry when masked and named: a vault box under vault-sealed is one masked word and one path, judged whole by what it opens to (issue 1126, C1; review of seed-same-3, round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@6097e4f3e040
mesh/delivery-group group fix/1126-seed-same-3 merged: every member is merged
mesh/check pass 8520a4c09960 on 79eb58efe4b0 | carried from mesh-catalog 774fcdfa93ad | unit 2188 run, 1064 kept, 36 skipped | integration 2197 run, 1…
2026-10-28 12:22:37 +01:00
jschoubben 70de9429e5 A file's secrets are sealed values entry by entry: each a slot of its own, kept where it opens to the value that runs, masked and named per secret (issue 1126, C1; the seed-7 probe: about 45 files on ace and novox differed by their secrets' bytes alone) 2026-10-28 12:02:39 +01:00
mesh-tools 79eb58efe4 Merge pull request 'Compose as the controller does, so a node's shadow can answer same (issue 1126, group fix/1126-seed-same)' (#114) from fix/1126-seed-same into main 2026-10-28 09:46:24 +00:00
jschoubben 3028c7ec4e A drifted own secret is read from a resource id as a node runs it, its module's name first (only-office.needs-jwt), so the node agent finds what to hand the vault (issue 1126, cause C4; review round 2)
mesh/delivery merged: source.main moved to novox/mesh-sdk@79eb58efe4b0
mesh/delivery-group group fix/1126-seed-same merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-host/cmd/mesh-host.TestNatsANodeThePlacementNamesComposesAndAppliesItsOwn: the node did not comp…
delivery.OwnNeedName reads <module>.needs-<name> and the bare needs-<name>; OwnValuesDiffer and the node agent use it.
The fixtures carry the id shape captured from novox's shadow answer.
2026-10-28 10:00:29 +01:00
jschoubben 1062b33d32 A node's shadow answer counts the values that drifted from the vault's, and names the own secrets among them (issue 1126, cause C4)
delivery.OwnValuesDiffer, ValuesDiffer and WithValuesDiffer read the shown bodies masked as another value; a candidate
answer carries values_differ. The node agent hands the vault those own secrets' running values through move-in's
replace_running, so no seat verb and no bus grant changes here.
2026-10-28 09:49:03 +01:00
jschoubben 16b8d8a1ee Compose as the controller does, so a node's shadow can answer same (issue 1126, C1 C2 C3 C5 C6, T1)
Seals are reused and compared by slot (resource, field), never across resources;
one rule says which resolved config values are laid; a seat path that steps back
is owned by the seat it steps back for, whatever the seat order; a folded module
of an old-form build is composed from what runs of it; the operator export's file
is carried as it runs; a differing resource names its differing JSON paths, found
before its bodies are cut, and an answer is fitted to the node's key.
2026-10-28 09:36:36 +01:00
mesh-tools be77f66563 Merge pull request 'frombus: blame only the modules a refusal names, and say which providers another node cannot run (issue 1126, group fix/1126-shadow-bodies)' (#113) from fix/1126-shadow-bodies into main 2026-10-28 07:46:23 +00:00
jschoubben 362a3f1378 A refusal for want of a provider names the modules another node cannot run in this composition, and a refusal naming no placed module is answered at once (issue 1126, probe #532)
mesh/delivery merged: source.main moved to novox/mesh-sdk@be77f6656326
mesh/delivery-group group fix/1126-shadow-bodies merged: every member is merged
mesh/check pass 362a3f1378aa on 03da552cb331 | unit 2860 run, 1141 kept, 36 skipped | integration 2887 run, 1241 kept, 29 skipped | over bound: unit (…
ace and g14 answered 'nothing in this mesh provides artifact-store … assign
distribution to a node' while novox runs it: a module another node cannot run
in this composition (catalogue Resolution.Unhostable) offers nothing, silently.
The refusal now names each such module and what it lacks, and each node left
out. And the culprit search composed the whole mesh once per placed module for a
refusal no stub changes (ace's answer took 7m46s): only the modules the refusal
names are tried, at most 8.
2026-10-28 01:39:07 +01:00
mesh-tools 03da552cb3 Merge pull request 'Shadow answers show both bodies of each difference (secrets masked); a folded module is judged inside; the bus membership is carried (issue 1126, group fix/1126-shadow-shows)' (#112) from fix/1126-shadow-shows into main 2026-10-28 00:21:47 +00:00
jschoubben 4e0c57771e Bound the bodies an answer shows, naming the rest (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-sdk@03da552cb331
mesh/delivery-group group fix/1126-shadow-shows merged: every member is merged
mesh/check pass 4e0c57771e8c on 47eb1009921d | unit 2980 run, 270 kept, 36 skipped | integration 2999 run, 302 kept, 29 skipped | over bound: unit (sl…
2026-10-28 01:09:02 +01:00
jschoubben b31c4580aa A candidate answer shows every differing resource, masked; a module folded in the node's own is judged there; the node's own carries its bus membership as it runs it (issue 1126, seed #531)
novox's answer to seed #531 named 15 resources by id alone, so why each differed
was guessed. ModuleDiffers.Bodies keeps every differing resource with both
bodies, each value under a key a secret could be in masked (the same, another
seal of the same value, another value). The five old-form modules read
'removed' because the agent also lists them by module from the declaration kept
before the switch; they are judged inside the node's own. And the node's own
lacked bus-membership, which only the controller composes from its store: the
membership the node runs is carried, until the grants part hands it sealed.
2026-10-28 01:08:49 +01:00
mesh-tools 47eb100992 Merge pull request 'frombus: a stub keeps its module's directories; compose returns nothing once its run ended (issue 1126, group fix/1126-switch-safe)' (#111) from fix/1126-switch-safe into main 2026-10-27 23:52:19 +00:00
jschoubben 7359d187db A stub keeps its module's directories, so the places its own secrets and data name still resolve (issue 1126: novox refused nats's ${dir:mesh-state})
mesh/delivery merged: source.main moved to novox/mesh-sdk@47eb1009921d
mesh/delivery-group group fix/1126-switch-safe merged: every member is merged
mesh/check pass 7359d187db8d on 47b1dcf08f85 | unit 2992 run, 1008 kept, 36 skipped | integration 3025 run, 1102 kept, 29 skipped | over bound: unit (…
2026-10-28 00:21:45 +01:00
mesh-tools 47b1dcf08f Merge pull request 'Shadow answers name every refusal, carry old-form builds as they run, and compare a deleted placement (issue 1126, group fix/1126-shadow-all)' (#110) from fix/1126-shadow-all into main 2026-10-27 22:44:08 +00:00
jschoubben e32d777530 A deleting head's answer compares the controller's compositions with what runs, and is refused only without returning; fold the shadow's tests into TestInput (issue 1126, review of the shadow, round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@47b1dcf08f85
mesh/delivery-group group fix/1126-shadow-all merged: every member is merged
mesh/check checking e32d77753024 on d48365aed010 | novox/mesh-sdk#110 in group fix/1126-shadow-all
2026-10-27 23:32:31 +01:00
jschoubben 472d5f3144 A node's have says whose compositions it applies, and its answer to a head deleting its placement says it returns to the controller's (issue 1126, the seed's revert)
The check of the seed's revert (mesh-catalog #526) failed on 'would stop
running node': the shadow answered a deleted placement as everything removed,
while the node keeps what runs. HaveNode.Composes and ComposesWhy say whether a
node applies its own compositions or the controller's; CandidateAnswer.Returns
answers a deleted placement, verdict same, naming nothing composed.
2026-10-27 23:22:54 +01:00
jschoubben 2fd5ffdd62 A node's answer names every module it cannot compose, a folded module of an old-form build is left to the node as it runs, and no store reached says why (issue 1126, the seed's shadow)
The seed's shadow answers named the first refusal only (ace: baserow's archive,
novox: de-spiegel), so each round showed one more. A build that cannot be read or
taken in, and a module the composition refuses, are now each composed as a stub
and named. A module folded into the node's own composition whose manifest names a
build (de-spiegel, nats, photos, systemd-resolved: haves naming a build-... id)
runs a build only the controller holds: it is composed as a stub and its
resources are carried as the node runs them.
2026-10-27 23:17:05 +01:00
60 changed files with 4126 additions and 642 deletions
+3
View File
@@ -200,6 +200,9 @@ func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string
if runner != "" {
args["runner"] = runner
}
if len(job.NodeNames) > 0 {
args[delivery.NodeNamesArg] = job.NodeNames // the names check's (ADR 0567 §12)
}
return args
}
test := func(args map[string]any) (bool, error) {
+9
View File
@@ -143,6 +143,15 @@ test {"level":"unit","packages":["x/pkg/a"],"workspace":"/w"}`
t.Fatalf("the unit level %+v", u)
}
// ADR 0567 §12: a check's node names reach the digest and test tools, for the names check (a kept verdict misses when
// they change).
r.calls = nil
if _, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, NodeNames: []string{"anchor"},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}})); err != nil || !strings.Contains(strings.Join(r.calls, "\n"), `digest {"level":"unit","node-names":["anchor"]`) ||
!strings.Contains(strings.Join(r.calls, "\n"), `test {"level":"unit","node-names":["anchor"]`) {
t.Fatalf("the node names are not handed to the digest and test tools: %v, %v", r.calls, err)
}
// Issue 890/AC2: a package declaring node-build-node is not tested on a node that lacks it but named unmet, and is
// tested on a node that has it.
for _, has := range []bool{false, true} {
+31 -37
View File
@@ -8,14 +8,15 @@ import (
"time"
)
// **A node may build only within its build hours** (the operator, 2026-10-26 21:18: "we should only use g14 as a builder
// outside office hours", "from 18 o'clock until 6 in the morning"). A build module's `build-hours` setting names, per
// node, the window in the node's local time in which it takes asks; outside it the capacity judged by is 0: steps running
// finish, no new one is taken. A node it does not name, or an empty setting, builds at any hour. The puller judges it at
// every judgement and wakes on one timer to the window's next edge (a clock edge, not a poll of another part).
// **A node may build only within its build hours** (the operator, 2026-10-26 21:18: the laptop builds only outside
// office hours, "from 18 o'clock until 6 in the morning"). A build module's `build-hours` setting is the node's window
// in its local time, a config value set on each node's layer of mesh-config (ADR 0567 §12: no repository names a node);
// outside it the capacity judged by is 0: steps running finish, no new one is taken. An empty value builds at any hour.
// The puller judges it at every judgement and wakes on one timer to the window's next edge (a clock edge, not a poll of
// another part).
// BuildHoursSetting is the build modules' setting of the nodes' build windows: `<node>=HH:MM-HH:MM` pairs separated by
// commas, in the node's local time; an end before its start crosses midnight (g14=18:00-06:00).
// BuildHoursSetting is the build modules' setting of the node's build window: HH:MM-HH:MM in the node's local time; an
// end before its start crosses midnight (18:00-06:00).
const BuildHoursSetting = "build-hours"
// BuildWindow is one node's build hours: minutes after local midnight, Start inclusive, End exclusive.
@@ -51,35 +52,28 @@ func (w BuildWindow) NextEdge(t time.Time) time.Time {
return next
}
// ParseBuildHours is node's window in a build-hours setting: nil when the setting is empty or does not name the node.
func ParseBuildHours(setting, node string) (*BuildWindow, error) {
var found *BuildWindow
for _, pair := range strings.Split(setting, ",") {
pair = strings.TrimSpace(pair)
if pair == "" {
continue
}
name, span, ok := strings.Cut(pair, "=")
from, to, ok2 := strings.Cut(span, "-")
if !ok || !ok2 || strings.TrimSpace(name) == "" {
return nil, fmt.Errorf("the build hours %q: %q is not <node>=HH:MM-HH:MM", setting, pair)
}
start, err := minuteOf(from)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
end, err := minuteOf(to)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
if start == end {
return nil, fmt.Errorf("the build hours %q: %s opens and closes at the same minute", setting, pair)
}
if strings.TrimSpace(name) == node {
found = &BuildWindow{Start: start, End: end}
}
// ParseBuildWindow reads a build-hours value, HH:MM-HH:MM: nil when it is empty.
func ParseBuildWindow(setting string) (*BuildWindow, error) {
setting = strings.TrimSpace(setting)
if setting == "" {
return nil, nil
}
return found, nil
from, to, ok := strings.Cut(setting, "-")
if !ok {
return nil, fmt.Errorf("the build hours %q are not HH:MM-HH:MM", setting)
}
start, err := minuteOf(from)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
end, err := minuteOf(to)
if err != nil {
return nil, fmt.Errorf("the build hours %q: %v", setting, err)
}
if start == end {
return nil, fmt.Errorf("the build hours %q open and close at the same minute", setting)
}
return &BuildWindow{Start: start, End: end}, nil
}
// minuteOf reads HH:MM as minutes after midnight.
@@ -92,7 +86,7 @@ func minuteOf(s string) (int, error) {
return h*60 + m, nil
}
// NodeBuildHours is node's build window, read from the module's settings file at path at every call, and why it is
// NodeBuildHours is the node's build window, read from the module's settings file at path at every call, and why it is
// none when the value cannot be read. No path, no file value or an empty one is no window and no fault. A value that
// cannot be read is no window too, so the node builds at any hour; the setting is a preference of hours, not a safety
// bound (a node that must never build has capacity 0). It is loud: the fault is said once per distinct fault through
@@ -125,7 +119,7 @@ func NodeBuildHours(path, node string, logf func(string, ...any)) func() (*Build
if v, ok := s[BuildHoursSetting]; ok && json.Unmarshal(v, &setting) != nil {
return fault(fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, v))
}
w, err := ParseBuildHours(setting, node)
w, err := ParseBuildWindow(setting)
if err != nil {
return fault(err.Error())
}
+31 -34
View File
@@ -4,45 +4,38 @@ import (
"encoding/json"
"fmt"
"os"
"git.novox.be/novox/mesh-sdk/go/delivery/buildrun"
)
// CapacitySetting is the build modules' setting of each node's build capacity (novox/hq ADR 0525 §5): `<node>=<asks at
// once>` pairs separated by commas, as the builder's was (ADR 0485 §1). Each build module declares it with
// DefaultCapacities as its default, and the controller lays it into the module's settings file.
// CapacitySetting is the build modules' setting of the node's build capacity (novox/hq ADR 0525 §5, ADR 0567 §12): how
// many asks the node takes at once, an integer config value set on each node's layer of mesh-config. No repository names
// a node: each build module declares it with the default 0, which pulls nothing, and the controller (or mesh-config)
// lays the node's value into the module's settings file.
const CapacitySetting = "capacity"
// DefaultCapacities is the setting's default, the builder's before R4g: the control-node (novox) and the laptop (g14)
// build nothing.
const DefaultCapacities = "shanks=3,ace=2,g14=0,novox=0"
// NodeCapacity is node's build capacity, read from the module's settings file at path at every call, so a change takes
// effect at the next pull; with no path it reads fallback. A node the setting does not name runs one ask at a time (as
// the builder's did). A file or a setting that cannot be read gives 0 and says why: pulling on a setting nobody can read
// could build on a node that must not.
func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func() int {
// NodeCapacity is the node's build capacity, read from the module's settings file at path at every call, so a change
// takes effect at the next pull. No path, no file value, or a value that is not a whole number of at least 0 gives 0 and
// a value that cannot be read says why: pulling on a setting nobody can read could build on a node that must not.
func NodeCapacity(path, node string, logf func(string, ...any)) func() int {
say := func(format string, args ...any) {
if logf != nil {
logf(format, args...)
}
}
return func() int {
setting := fallback
if path != "" {
raw, err := os.ReadFile(path)
if err != nil {
say("the build capacity cannot be read, so %s pulls nothing: %v", node, err)
return 0
}
var s map[string]any
if err := json.Unmarshal(raw, &s); err != nil {
say("the settings file %s is not JSON, so %s pulls nothing: %v", path, node, err)
return 0
}
setting, _ = s[CapacitySetting].(string)
if path == "" {
return 0
}
n, err := capacityOf(setting, node)
raw, err := os.ReadFile(path)
if err != nil {
say("the build capacity cannot be read, so %s pulls nothing: %v", node, err)
return 0
}
var s map[string]json.RawMessage
if err := json.Unmarshal(raw, &s); err != nil {
say("the settings file %s is not JSON, so %s pulls nothing: %v", path, node, err)
return 0
}
n, err := capacityOf(s[CapacitySetting])
if err != nil {
say("%v, so %s pulls nothing", err, node)
return 0
@@ -51,11 +44,15 @@ func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func()
}
}
// capacityOf is node's capacity in a setting's `<node>=<asks>` pairs: DefaultCapacity for a node it does not name.
func capacityOf(setting, node string) (int, error) {
caps, err := buildrun.ParseCapacity(setting)
if err != nil {
return 0, fmt.Errorf("the build capacity: %w", err)
// capacityOf is the capacity a config value holds: absent or null is 0; anything but a whole number of at least 0 is
// refused, naming it.
func capacityOf(value json.RawMessage) (int, error) {
if len(value) == 0 || string(value) == "null" {
return 0, nil
}
return (&buildrun.Places{Capacity: caps}).CapacityOf(node), nil
var n int
if err := json.Unmarshal(value, &n); err != nil || n < 0 {
return 0, fmt.Errorf("the build capacity %s is not a whole number of asks at once, 0 or more", value)
}
return n, nil
}
+2 -2
View File
@@ -14,7 +14,7 @@ import (
)
// **A build seat's capacity follows the CPU its node gives builds** (issue 1806's follow-up; issue 1890, 2026-10-26
// 19:41: under the 50 % cap shanks' mesh-build-go kept capacity 3, three full mesh-catalog checks shared 12 cores, and
// 19:41: under the 50 % cap the workstation's mesh-build-go kept capacity 3, three full mesh-catalog checks shared 12 cores, and
// each step ran past the 30-minute step bound with no test run). The configured capacity is an upper bound; the node's
// build slice bounds it again by the cores a step needs: floor(the slice's quota in cores / cores-per-step), one at
// least. The quota is read live from the cgroup at every judgement, so a changed cap is followed at the next take
@@ -27,7 +27,7 @@ import (
const CoresPerStepSetting = "cores-per-step"
// DefaultCoresPerStep are the build modules' defaults by language, from measured steps (2026-10-26): a full mesh-catalog
// check on mesh-build-go takes what it is given (three at once took 965 % of shanks' 1200 % slice, about 4 cores each,
// check on mesh-build-go takes what it is given (three at once took 965 % of the workstation's 1200 % slice, about 4 cores each,
// and ran past the 30-minute step bound), so a Go step is given 6, half again what starved; a Node step (tsc, esbuild, a
// test runner: median 15 s) and an image build (buildah, its RUN lines mostly fetches) need 2.
var DefaultCoresPerStep = map[string]float64{delivery.AskGo: 6, delivery.AskNode: 2, delivery.AskImage: 2}
+34 -33
View File
@@ -14,30 +14,20 @@ import (
// issue 1806's follow-up: the capacity judged by is the configured one bounded by the build slice's cores over the cores
// a step needs (the three build nodes as they stand on 2026-10-26), and cores-per-step is read from the settings file.
func TestNodeCapacityAndLimits(t *testing.T) {
if got := NodeCapacity("", DefaultCapacities, "novox", nil)(); got != 0 {
t.Errorf("novox = %d by default, want 0", got)
}
if got := NodeCapacity("", DefaultCapacities, "shanks", nil)(); got != 3 {
t.Errorf("shanks = %d by default, want 3", got)
// ADR 0567 §12: the capacity is the node's integer, its default 0, and no default names a node.
if got := NodeCapacity("", "laptop", nil)(); got != 0 {
t.Errorf("no settings file = %d, want 0", got)
}
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"capacity":"shanks=1,ace=2"}`), 0o600)
capacity := NodeCapacity(file, DefaultCapacities, "shanks", nil)
if got := capacity(); got != 1 {
t.Errorf("shanks = %d from the file, want 1", got)
capacity := NodeCapacity(file, "anchor", nil)
for value, want := range map[string]int{`{"capacity":1}`: 1, `{"capacity":4}`: 4, `{}`: 0, `{"capacity":null}`: 0,
`{"capacity":"anchor=3"}`: 0, `{"capacity":-1}`: 0, `{"capacity":1.5}`: 0} {
os.WriteFile(file, []byte(value), 0o600)
if got := capacity(); got != want {
t.Errorf("%s = %d, want %d", value, got, want)
}
}
os.WriteFile(file, []byte(`{"capacity":"shanks=4"}`), 0o600)
if got := capacity(); got != 4 {
t.Errorf("shanks = %d after the change, want 4", got)
}
if got := NodeCapacity(file, "", "pi", nil)(); got != 1 {
t.Errorf("a node not named = %d, want 1", got)
}
os.WriteFile(file, []byte(`{"capacity":"shanks=lots"}`), 0o600)
if got := capacity(); got != 0 {
t.Errorf("an unreadable setting = %d, want 0", got)
}
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), DefaultCapacities, "shanks", nil)(); got != 0 {
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), "anchor", nil)(); got != 0 {
t.Errorf("a missing file = %d, want 0", got)
}
t.Setenv("MESH_BUILD_CPU_SHARE", "")
@@ -74,7 +64,7 @@ func TestNodeCapacityAndLimits(t *testing.T) {
for _, c := range []struct {
file string
want float64
}{{`{"capacity":"shanks=2"}`, 6}, {`{"cores-per-step":4}`, 4}, {`{"cores-per-step":"3.5"}`, 3.5}, {`{"cores-per-step":0}`, 6}, {`{"cores-per-step":0}`, 6}} {
}{{`{"capacity":2}`, 6}, {`{"cores-per-step":4}`, 4}, {`{"cores-per-step":"3.5"}`, 3.5}, {`{"cores-per-step":0}`, 6}, {`{"cores-per-step":0}`, 6}} {
os.WriteFile(file, []byte(c.file), 0o600)
if got := cores(); got != c.want {
t.Errorf("cores-per-step from %s = %g, want %g", c.file, got, c.want)
@@ -86,6 +76,8 @@ func TestNodeCapacityAndLimits(t *testing.T) {
// The operator, 2026-10-26 21:18: g14 builds only from 18:00 to 06:00, its local time. Outside its build hours the
// capacity judged by is 0, said with the window and when it opens; a node not named, or a bad value, builds at any hour.
// And a node its module proves cannot build (ace without /dev/net/tun, 2026-10-28) judges by 0 at every hour, saying
// why, so its asks go to a node that can.
cet := time.FixedZone("CET", 3600)
day := func(hh, mm int) time.Time { return time.Date(2026, 10, 26, hh, mm, 0, 0, cet) }
was := now
@@ -96,17 +88,21 @@ func TestNodeCapacityAndLimits(t *testing.T) {
effective int
edge string // the window's next edge, HH:MM; "" for no window
bad bool
ineligible string // what the module's Eligible says; "" for eligible
}{
{"g14 in the office", "g14=18:00-06:00", "g14", day(10, 0), 0, "18:00", false},
{"g14 in the evening", "g14=18:00-06:00", "g14", day(18, 0), 2, "06:00", false},
{"g14 past midnight", "g14=18:00-06:00", "g14", day(5, 59), 2, "06:00", false},
{"g14 at the close", "g14=18:00-06:00", "g14", day(6, 0), 0, "18:00", false},
{"a day window", "shanks=09:00-17:30,g14=18:00-06:00", "shanks", day(17, 29), 2, "17:30", false},
{"a node not named", "g14=18:00-06:00", "ace", day(10, 0), 2, "", false},
{"a bad time", "g14=18:00-6", "g14", day(10, 0), 2, "", true},
{"no span", "g14=18:00-18:00", "g14", day(10, 0), 2, "", true},
{"the laptop in the office", "18:00-06:00", "laptop", day(10, 0), 0, "18:00", false, ""},
{"the laptop in the evening", "18:00-06:00", "laptop", day(18, 0), 2, "06:00", false, ""},
{"the laptop past midnight", "18:00-06:00", "laptop", day(5, 59), 2, "06:00", false, ""},
{"the laptop at the close", "18:00-06:00", "laptop", day(6, 0), 0, "18:00", false, ""},
{"a day window", "09:00-17:30", "anchor", day(17, 29), 2, "17:30", false, ""},
{"no window", "", "anchor", day(10, 0), 2, "", false, ""},
{"a bad time", "18:00-6", "laptop", day(10, 0), 2, "", true, ""},
{"no span", "18:00-18:00", "laptop", day(10, 0), 2, "", true, ""},
{"the old form naming a node", "laptop=18:00-06:00", "laptop", day(10, 0), 2, "", true, ""},
{"a node without /dev/net/tun", "", "anchor", day(10, 0), 0, "", false, "/dev/net/tun cannot be opened: no such device"},
{"not eligible within its hours", "18:00-06:00", "laptop", day(19, 0), 0, "06:00", false, "/dev/net/tun cannot be opened: no such device"},
} {
w, err := ParseBuildHours(c.setting, c.node)
w, err := ParseBuildWindow(c.setting)
if (err != nil) != c.bad {
t.Errorf("%s: parse %v", c.name, err)
}
@@ -121,11 +117,16 @@ func TestNodeCapacityAndLimits(t *testing.T) {
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"build-hours":"`+c.setting+`"}`), 0o600)
now = func() time.Time { return c.at }
p := &Puller{Language: "go", Node: c.node, Capacity: func() int { return 2 }, BuildHours: NodeBuildHours(file, c.node, nil)}
p := &Puller{Language: "go", Node: c.node, Capacity: func() int { return 2 }, BuildHours: NodeBuildHours(file, c.node, nil),
Eligible: func() string { return c.ineligible }}
got := p.CapacityDetail()
if got.Effective != c.effective || got.OutsideHours != (c.effective == 0) || (c.effective == 0 && !strings.Contains(got.Why, "outside its build hours 18:00-06:00 CET until 18:00")) {
outside := c.effective == 0 && c.ineligible == ""
if got.Effective != c.effective || got.OutsideHours != outside || (outside && !strings.Contains(got.Why, "outside its build hours 18:00-06:00 CET until 18:00")) {
t.Errorf("%s: %+v, want capacity %d", c.name, got, c.effective)
}
if got.Ineligible != c.ineligible || (c.ineligible != "" && !strings.HasPrefix(got.Why, "0, not eligible: "+c.ineligible+" (otherwise ")) {
t.Errorf("%s: not eligible is not said: %+v", c.name, got)
}
// A value that cannot be read builds at any hour, loudly: the record names the value and says why.
if c.bad && (!strings.Contains(got.BuildHoursFault, c.setting) || !strings.Contains(got.Why, "at any hour, because its build hours cannot be read")) {
t.Errorf("%s: a bad value is not said in the record: %+v", c.name, got)
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **A running ask says its phases** (issue 1890, 2026-10-26 19:09: a mesh-catalog try's step on shanks ran 30 minutes
// **A running ask says its phases** (issue 1890, 2026-10-26 19:09: a mesh-catalog try's step on the workstation ran 30 minutes
// and its module's journal held no line naming its ask, so nobody could tell whether it fetched, linted or tested). The
// puller hands every ask's run a phase reporter in its context; RunAsk and a module's own step bodies call Phase as they
// move (fetch and merge, a level's digest, lint, a level's tests), and each phase is logged with the ask's id and its
+19 -10
View File
@@ -86,6 +86,13 @@ type Puller struct {
// mesh-config resolves for `build-hours` takes its place once handed. Outside the window the capacity judged by is 0:
// steps running finish, no new one is taken; the puller wakes at the window's next edge. Nil builds at any hour.
BuildHours func() (*BuildWindow, string)
// Eligible says why this node cannot run the seat's asks at all now, and "" when it can: what the module proves of
// its machine and finds missing (mesh-build-image: /dev/net/tun cannot be opened, so buildah's network for a RUN line
// cannot be made; 2026-10-28 on ace). Read at every judgement, so it is cheap (a module answers its last look).
// While it says a reason the capacity judged by is 0, said with the reason: steps running finish, no new one is
// taken, and the asks go to a node that can run them instead of failing here. The module calls Wake when its answer
// changes. Nil is always eligible.
Eligible func() string
// Has says whether this node has a need an ask names (delivery.AskNeeds, issue 890). An ask needing what it lacks is
// declined: its step ends refused as declined (delivery.StepDeclined), the message is dropped for its asker to put
// again, and the worker it came from is not looked at again until the next pull's wait, so a node lacking the need
@@ -291,6 +298,13 @@ func (p *Puller) CapacityDetail() delivery.Capacity {
c.Why = fmt.Sprintf("%s; within its build hours %s until %s", c.Why, c.BuildHours, w.NextEdge(at).Format("15:04"))
}
}
if p.Eligible != nil {
if why := p.Eligible(); why != "" {
c.Ineligible = why
c.Why = fmt.Sprintf("0, not eligible: %s (otherwise %s)", why, c.Why)
c.Effective = 0
}
}
p.sayCapacity(c)
return c
}
@@ -326,7 +340,8 @@ func (p *Puller) sayCapacity(c delivery.Capacity) {
p.mu.Lock()
same := p.hasJudged && p.judged.Effective == c.Effective && p.judged.Configured == c.Configured && p.judged.ByCPU == c.ByCPU &&
p.judged.CoresPerStep == c.CoresPerStep && p.judged.QuotaCores == c.QuotaCores && p.judged.OutsideHours == c.OutsideHours &&
p.judged.BuildHours == c.BuildHours && p.judged.BuildHoursFault == c.BuildHoursFault
p.judged.BuildHours == c.BuildHours && p.judged.BuildHoursFault == c.BuildHoursFault &&
p.judged.Ineligible == c.Ineligible
if !same {
p.judged, p.hasJudged, p.capWritten = c, true, false
}
@@ -595,7 +610,7 @@ func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key
}
}
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: g14's
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: the laptop's
// puller started at capacity 0 sat parked with capacity 2 set, because nothing woke it to read its settings file
// again). The value comes as mesh-config resolved it for this node, the event of the change itself: the runtime hands
// the value now and then each change its machine's provisioner client says moved, so nothing polls, and the puller does
@@ -644,7 +659,7 @@ func (p *Puller) watchHours() {
why = fmt.Sprintf("the setting %s %s is not a string", BuildHoursSetting, c.Value)
default:
var err error
if w, err = ParseBuildHours(setting, p.Node); err != nil {
if w, err = ParseBuildWindow(setting); err != nil {
why = err.Error()
}
}
@@ -674,13 +689,7 @@ func (p *Puller) capacityFrom(c stdio.ProvisionChange) (int, string) {
if c.Refused != "" {
return 0, "its capacity is no longer answered: " + c.Refused
}
setting := ""
if len(c.Value) > 0 && string(c.Value) != "null" {
if err := json.Unmarshal(c.Value, &setting); err != nil {
return 0, fmt.Sprintf("its capacity %s is not a string", c.Value)
}
}
n, err := capacityOf(setting, p.Node)
n, err := capacityOf(c.Value)
if err != nil {
return 0, err.Error()
}
+2 -2
View File
@@ -22,7 +22,7 @@ func (w *configWatch) watch(key string, handle func(stdio.ProvisionChange) error
w.asked, w.handle = key, handle
value := w.value
w.mu.Unlock()
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)})
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(value)})
}
func (w *configWatch) set(t *testing.T, value string) {
@@ -33,7 +33,7 @@ func (w *configWatch) set(t *testing.T, value string) {
if handle == nil {
t.Fatal("the puller watches no config value")
}
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)}); err != nil {
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(value)}); err != nil {
t.Fatal(err)
}
}
+2 -2
View File
@@ -282,7 +282,7 @@ func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
t.Run("config watch wakes", func(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
w := &configWatch{value: "shanks=3,g14=0"}
w := &configWatch{value: "0"}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "g14", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 0 }, // the settings file, not refilled yet
WatchConfig: w.watch,
@@ -298,7 +298,7 @@ func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
if w.asked != CapacitySetting {
t.Fatalf("the puller watches %q, want %q", w.asked, CapacitySetting)
}
w.set(t, "shanks=3,g14=2")
w.set(t, "2")
waitFor(t, "the ask taken once the capacity moved to 2", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
for _, c := range []struct {
+1 -1
View File
@@ -149,7 +149,7 @@ func endRunning(scan Scan, key string, step delivery.Step, revision uint64, why
// asker takes the first end it reads, so an end written here would fail a check the redelivery would still run. Its
// asker's watch takes puts only and keeps waiting for the redelivery's key, its bound the backstop; the stale running
// key leaves the steps state, and with it the delivery view's and the checker's count of what runs (note 21: it
// held one of shanks' two slots).
// held one of a build node's two slots).
// - A step owned by this process is left: it may run now. A redelivered ask begun on this node is owned by it.
// - A step with no owner (written by an older build) keeps the earlier rule: a hand-started step started before
// `before` (the start of this process) is ended; a pulled ask is left to the stream's redelivery (isAsk) until its
+17 -1
View File
@@ -9,6 +9,8 @@ import (
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Defends issue 1377 and novox/hq ADR 0547 §5: this is the controller's bus composer copied, and it composes byte for
@@ -33,7 +35,7 @@ func beyondTheControllers(user string) map[string]bool {
if !ok {
return out
}
for _, s := range append(readGrants("node-engine_have", "*.node"), NodeComposeReads(node)...) {
for _, s := range append(append(readGrants("node-engine_have", "*.node"), NodeComposeReads(node)...), NodeComposeAdvisories()...) {
out[s] = true
}
return out
@@ -118,6 +120,20 @@ func grantsAsTheControllers(t *testing.T, got, want string) {
}
func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
// ADR 0567 §3, §9: a node reads every key of the seat's placement and its candidates (it composes with the whole
// proposal laid over), and none of its open proposals; it writes neither.
reads := strings.Join(NodeComposeReads("laptop"), " ")
for bucket, granted := range map[string]bool{delivery.PlacementBucket: true, delivery.PlacementCandidatesBucket: true,
delivery.PlacementProposalsBucket: false} {
if strings.Contains(reads, "$JS.API.DIRECT.GET.KV_"+bucket+".>") != granted || strings.Contains(reads, "$KV."+bucket+".laptop") {
t.Errorf("a node is granted %s: %v, want %v", bucket, !granted, granted)
}
}
// It hears the creation of those two buckets, and of no other.
if got := NodeComposeAdvisories(); len(got) != 2 || got[0] != "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-delivery-assigner_placement" ||
got[1] != "$JS.EVENT.ADVISORY.STREAM.CREATED.KV_mesh-delivery-assigner_candidates" {
t.Errorf("a node hears the creation of %v", got)
}
raw, err := os.ReadFile(filepath.Join("testdata", "controller-user-lists.json"))
if err != nil {
t.Fatal(err)
+5 -2
View File
@@ -898,8 +898,8 @@ func PermissionsFor(p Principal) (Permissions, error) {
// its own modules reads the other nodes' public records, which its firewall and resolver need (ADR 0547 §2). Read
// only: each node still writes its own keys alone.
pub = append(pub, readGrants(delivery.HaveBucket, "*."+delivery.HaveNodeWord)...)
// And what it composes its own modules from (ADR 0547 §2, ADR 0549 §3): the indexer's placement, modules and
// manifests, the builder's builds (the manifest of a build its want names), and mesh-config's resolved values of
// And what it composes its own modules from (ADR 0547 §2, ADR 0567 §9): the seat mesh-placement's placement and
// candidates, the indexer's modules and manifests, the builder's builds (the manifest of a build its want names), and mesh-config's resolved values of
// its own node alone. Read only.
pub = append(pub, NodeComposeReads(p.Node)...)
// And, of the other nodes' resolved values, the keys their modules serve from (issue 1126), each one key.
@@ -931,6 +931,9 @@ func PermissionsFor(p Principal) (Permissions, error) {
// And the backup holder's word that it backed the module up (ADR 0539 clause 16), whichever module holds
// node-backup: one event, by its name.
sub = append(sub, NodeAgentHears...)
// And the creation of the seat mesh-placement's two states it composes from (ADR 0567 §9), so a node agent started
// before they exist binds each when the bus says it was made, with no timer; no other bucket's.
sub = append(sub, NodeComposeAdvisories()...)
// And, while the vault serves it, the vault's move-in in its own name and nobody else's (issue 676): the
// values sealed to this machine, which only it opens, handed in sealed to the vault's key for the call.
if p.MovesIntoTheVault {
+12 -6
View File
@@ -311,12 +311,12 @@ func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err
// ResolvedBucket is mesh-config's state of resolved values, keyed `<node>.<module>.<key>`.
const ResolvedBucket = "mesh-config_resolved"
// NodeComposeReads is what a node's agent reads to compose its own modules (novox/hq ADR 0547 §2, ADR 0549 §3, issue
// 1126): every key of the indexer's placement, placement candidates, modules and manifests and of the builder's builds,
// and of mesh-config's resolved values only its own node's, `<node>.>`. Every candidate key, not only `*.<node>`: a node
// answering its candidate composes with the whole group's placement (the seed places every node at once, and a binding
// crosses nodes), and nothing in a placement is secret. Reads only; the node is checked as every subject token is by the
// caller's principal.
// NodeComposeReads is what a node's agent reads to compose its own modules (novox/hq ADR 0547 §2, ADR 0567 §3, §9,
// issue 1126): every key of the seat mesh-placement's placement and candidates (both the assigner's states), of the
// indexer's modules and manifests and of the builder's builds, and of mesh-config's resolved values only its own node's,
// `<node>.>`. Every candidate key, not only `*.<node>`: a node answering its candidate composes with the whole
// proposal laid over (the seed places every node at once, and a binding crosses nodes), and nothing in a placement is
// secret. Reads only; the node is checked as every subject token is by the caller's principal.
func NodeComposeReads(node string) []string {
var out []string
for _, bucket := range []string{delivery.PlacementBucket, delivery.PlacementCandidatesBucket, delivery.ModulesBucket,
@@ -326,6 +326,12 @@ func NodeComposeReads(node string) []string {
return append(out, readGrants(ResolvedBucket, node+".>")...)
}
// NodeComposeAdvisories are the stream-created advisories of the seat mesh-placement's placement and candidates
// (ADR 0567 §9): what a node agent hears to bind each once it is made. Those two and no wider.
func NodeComposeAdvisories() []string {
return []string{StreamCreatedAdvisory(delivery.PlacementBucket), StreamCreatedAdvisory(delivery.PlacementCandidatesBucket)}
}
// ServedReadGrants is the read of each key of mesh-config's resolved values named, `<node>.<module>.<key>`
// (Principal.ServedReads, issue 1126): a get and a watch of that key alone. Each part is checked as every subject token
// is, so a key cannot widen the grant.
+13
View File
@@ -445,6 +445,19 @@ func TestAnUnsetRequiredValueIsRefusedInTheConfigFormsWords(t *testing.T) {
t.Fatalf("not typed as a value nobody gave: %v", err)
}
// A contribution's value too (issue 1126, seed-8: de-spiegel's route names were refused with ${setting:…}).
withRoute := inConfigForm()
withRoute["contributes"] = map[string]any{"route": map[string]any{"site": map[string]any{"name": "${config:sender}", "port": 80}}}
routed, err := parseRaw(t, withRoute)
if err != nil {
t.Fatal(err)
}
_, err = Resolution{}.composed(routed, "route", routed.ContributesMany["route"]["site"], nil, "notifier contributing site to route")
if !errors.As(err, &unset) || unset.Module != "notifier" || !strings.Contains(err.Error(),
`notifier has a contribution site to route, in "name", that says ${config:sender}`) || strings.Contains(err.Error(), "${setting:") {
t.Fatalf("a contribution's unset value is not said in the config form's words: %v", err)
}
settingsForm := notifier()
settingsForm.Resources[0]["content"] = "from = ${setting:domain}\n"
err = unsetInConfigWords(settingInto(settingsForm.Resources[0], nil, "notifier"), settingsForm)
+20 -1
View File
@@ -10,6 +10,7 @@ package catalogue
import (
"encoding/json"
"errors"
"fmt"
"sort"
"strconv"
@@ -138,6 +139,9 @@ type Rendering struct {
// Kept is every operator-sealed secret in the mesh, for a module that `keeps` them. Nil when
// nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport
// KeptRuns is, by module, the file of the export as the node runs it, for a composer that holds no export (a node
// composing its own: the export is the controller's store's, issue 1126, C6): carried as it runs when Kept is nil.
KeptRuns map[string]map[string]any
// OutwardLinks is the links this machine reported as facing outside it, which the filter is
// written around (novox/hq ADR 0140). Empty means the machine has not said, and the mesh
@@ -929,6 +933,14 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
}
// Read by the module as its secrets' owner when it names one: the vault, as its own account (issue 723).
first = append(first, ownedBy(m.SecretsOwner, file))
} else if runs, ok := with.KeptRuns[m.Module]; m.Keeps != "" && ok {
file := make(map[string]any, len(runs))
for k, v := range runs {
file[k] = v
}
file["id"] = KeptID()
delete(file, "module")
first = append(first, file)
}
if m.Computed != "" {
mine, part := generated[m.Module]
@@ -1821,7 +1833,14 @@ func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers [
// module's files is no part of it (novox/hq 04-ISSUES/173).
values, err := overridden(raw, WithDefaults(m, layers), what)
if err != nil {
return nil, fmt.Errorf("%s: %w", what, err)
// A required value nothing sets is said in the words of the module's form, as a file's, a unit's and an owner's
// are (issue 1126, seed-8: de-spiegel in the config form was refused with ${setting:…} and `settings set`).
var unset *UnsetSettingError
if errors.As(err, &unset) && unset.Module == "" {
unset.Module = m.Module
unset.where = fmt.Sprintf("a contribution%s, in %q,", strings.TrimPrefix(what, m.Module+" contributing"), unset.field)
}
return nil, unsetInConfigWords(fmt.Errorf("%s: %w", what, err), m)
}
reaches, err := Reaches(m, layers)
if err != nil {
+17 -1
View File
@@ -121,7 +121,7 @@ func (r Resolution) ownedPaths() map[string]string {
for _, s := range Seats() {
for _, sp := range s.Paths {
for _, m := range r.Modules {
if m.ClaimsSeat(s.Name) && declaresPath(m, path.Clean(sp.Path)) {
if m.ClaimsSeat(s.Name) && declaresPath(m, path.Clean(sp.Path)) && !r.stepsBackHere(m, sp) {
owned[sp.Path] = m.Module + " (" + s.Name + ")"
}
}
@@ -130,6 +130,22 @@ func (r Resolution) ownedPaths() map[string]string {
return owned
}
// stepsBackHere says m's seat path steps back on this machine (novox/hq ADR 0287): it names a seat whose holder here
// declares the path too, so that holder owns it, as stepsBack leaves m's own rendering of it out. Without it the owner
// told was the last seat in the order of the seat table, which the controller reads sorted by name and a node in the
// order compiled (issue 1126, C3: /etc/resolv.conf, node-uplink against node-resolver).
func (r Resolution) stepsBackHere(m Manifest, sp SeatPath) bool {
if sp.StepsBackFor == "" {
return false
}
for _, o := range r.Modules {
if o.Module != m.Module && holdsSeat(o, sp.StepsBackFor) && declaresPath(o, path.Clean(sp.Path)) {
return true
}
}
return false
}
// filesWords is what the holder of node-files is given beside its own words: the node's secret places and the
// paths modules own here.
func (r Resolution) filesWords(m Manifest, words map[string]string, with Rendering) (map[string]string, error) {
+34
View File
@@ -2,6 +2,7 @@ package catalogue
import (
"encoding/json"
"slices"
"sort"
"strings"
"testing"
@@ -86,6 +87,39 @@ func TestTheHolderOfNodeFilesIsToldSecretPlacesAndOwnedPaths(t *testing.T) {
if !strings.HasPrefix(owned["/etc/app/app.conf"], "app") {
t.Errorf("a placed file is owned by %q, want app", owned["/etc/app/app.conf"])
}
// Issue 1126, C3: a seat path that steps back for a seat held here is that holder's, whatever the order of the seat
// table (the controller reads it sorted by name, a node in the order compiled).
resolver := Manifest{Module: "systemd-resolved", Version: "1", Claims: []Claim{{Name: ResolverSeat}},
Resources: []map[string]any{{"id": "stub", "type": "file", "path": ResolverFile}}}
uplink := Manifest{Module: "networkmanager", Version: "1", Claims: []Claim{{Name: "node-uplink"}},
Resources: []map[string]any{{"id": "resolv", "type": "file", "path": ResolverFile}}}
compiled := seats
t.Cleanup(func() { seats = compiled })
for _, order := range []string{"compiled", "reversed"} {
if order == "reversed" {
seats = append([]Seat(nil), compiled...)
slices.Reverse(seats)
}
both := Resolution{Node: "anchor", Modules: []Manifest{files, uplink, resolver}}
w, err := both.filesWords(files, nil, Rendering{})
if err != nil {
t.Fatal(err)
}
var o map[string]string
if err := json.Unmarshal([]byte(w[FilesOwnedPaths]), &o); err != nil || o[ResolverFile] != "systemd-resolved ("+ResolverSeat+")" {
t.Errorf("seat table %s: %s is owned by %q, want systemd-resolved (%s)", order, ResolverFile, o[ResolverFile], ResolverSeat)
}
}
seats = compiled
// Issue 1126, C5 and seed-8: a module carried as it runs (compose.Carried) owns its manifest's paths (OwnsPaths), as
// the controller tells them, and not the paths its running resources are at (a directory the mesh placed).
carried := Manifest{Module: "de-spiegel", Version: "1", OwnsPaths: []string{"/var/lib/de-spiegel/server.env"},
Resources: []map[string]any{{"id": "state", "type": "directory", "path": "/var/lib/de-spiegel"}}}
w, err := Resolution{Node: "anchor", Modules: []Manifest{files, carried}}.filesWords(files, nil, Rendering{})
if err != nil || !strings.Contains(w[FilesOwnedPaths], `"/var/lib/de-spiegel/server.env":"de-spiegel (its own tools)"`) ||
strings.Contains(w[FilesOwnedPaths], `"/var/lib/de-spiegel":`) {
t.Errorf("a carried module does not own its manifest's paths alone: %v %v", w[FilesOwnedPaths], err)
}
if w, _ := r.filesWords(app, nil, Rendering{}); w[FilesOwnedPaths] != "" {
t.Error("a module not holding node-files is told the owned paths")
}
+44
View File
@@ -0,0 +1,44 @@
package catalogue
import "encoding/json"
// The sources mesh-config says a resolved value came from, in mesh-config_resolved (novox/hq ADR 0348 §9): a key of
// it holds {"value": …, "source": …}.
const (
SourceMesh = "mesh"
SourceNode = "node"
SourceModule = "module"
SourceDefault = "default"
SourceUnset = "unset"
)
// LaidSource says whether a value resolved from source is laid over a module's files: mesh config, node config and the
// module's own value are; a manifest's default is not, since it is the composition's already and a taken key is laid
// only when a layer sets it, never from a default (TakesSettingsField); an unset value is not, since a required one is
// refused by the composition by name. **The one rule for every composer** (issue 1126, C2): the controller's config
// read and a node composing its own lay the same values, so the two compose the same files.
func LaidSource(source string) bool {
switch source {
case SourceMesh, SourceNode, SourceModule:
return true
}
return false
}
// LaidValue is one key of mesh-config_resolved read as a composer lays it: its value, and true when its source is laid
// (LaidSource) and it holds a value; false for a default, an unset value, a source not known and a value that cannot be
// read.
func LaidValue(raw []byte) (any, bool) {
var r struct {
Value json.RawMessage `json:"value"`
Source string `json:"source"`
}
if json.Unmarshal(raw, &r) != nil || !LaidSource(r.Source) || len(r.Value) == 0 {
return nil, false
}
var v any
if json.Unmarshal(r.Value, &v) != nil {
return nil, false
}
return v, true
}
+6
View File
@@ -864,6 +864,12 @@ type Manifest struct {
// that could only see its own ports would write a rule set that closed everything else.
Filtering *Filtering `json:"filtering,omitempty"`
// OwnsPaths is, for a module composed from what runs of it (compose.Carried: folded into its node's own composition
// at a build of the old form, issue 1126, C5), the paths it owns: its manifest's (DeclaredPaths), as the controller
// tells the holder of node-files them, in place of its resources', which are what runs (a directory placed at a
// path its manifest leaves to the mesh, a binding). Nil for any other module. Never in a manifest's JSON.
OwnsPaths []string `json:"-"`
// Jails are the fail2ban jails this module declares for its own service (novox/hq to-be 31).
// Written into whichever node runs the module, the same way `listens` become that node's rules.
Jails []Jail `json:"jails,omitempty"`
+74
View File
@@ -0,0 +1,74 @@
package catalogue
// The seat of placement (novox/hq ADR 0567): which module runs on which node, with its host ports, bindings, the
// node's pins and holdings, the config placement and values, and what a person decides of the node, is state on the
// bus held by this seat, changed only by a proposal the nodes it touches answer and the operator approves. Held once
// for the mesh by the assigner (mesh-delivery-assigner), beside its wants. Its states are named after the holding
// module, as every module state is (delivery.PlacementBucket, PlacementProposalsBucket, PlacementCandidatesBucket);
// they are the seat's contract, so every reader takes their names from mesh-sdk, and a change of holder is a migration
// the new holder's record decides (ADR 0567 §1). Its events are the holder's too, carried on the holder's stream
// (delivery.PlacementStream): the names are in delivery (PlacementProposedEvent and the rest).
// PlacementSeat is the seat the assigner holds for placement.
const PlacementSeat = "mesh-placement"
// The seat's seven verbs (ADR 0567 §10).
const (
PlacementVerbPlacement = "placement"
PlacementVerbProposals = "proposals"
PlacementVerbPropose = "propose"
PlacementVerbWithdraw = "withdraw"
PlacementVerbAgain = "again"
PlacementVerbAdopt = "adopt"
PlacementVerbHandBack = "hand-back"
)
// placementVerbs are the seat's verbs: what reads all it keeps and does by hand all it does by itself (ADR 0567 §10).
// Optional while the holder that serves them is not yet on every node (design 33 §7): the seat is new in the same
// delivery as its holder.
func placementVerbs() []Verb {
changes := map[string]any{"type": "array", "minItems": 1, "description": "the changes, each an object with `op`: " +
"place (node, module, ports, bindings), take-off (node, module), pin (node, provision, provider {node, module}), " +
"unpin (node, provision), hold (node, module, claim, scope, kind, site), unhold (node, module, claim), decide " +
"(node, field, value: a decided field of the node, its address among them), config (node, module, mesh-wide, " +
"layer, mesh-wide-values, values)",
"items": map[string]any{"type": "object", "required": []string{"op", "node"}}}
return []Verb{
{Name: PlacementVerbPlacement, Optional: true, Description: "Read placement: one node's key, the nodes one " +
"module is placed on, or every key, each with its revision, and how full the state is against its cap. With " +
"history, one node's or one module's changes in order: a revision the state keeps whole, an older one as the " +
"proposal's changes from its placement-changed event.",
Input: schema(map[string]string{"node": "one node", "module": "one module: the nodes it is placed on",
"history": "\"true\": the changes in order, from the stream and the state's revisions"}, nil, "history")},
{Name: PlacementVerbProposals, Optional: true, Description: "Every open proposal of placement: its changes, " +
"the revision of each key it was laid over, its candidates, every node's answer, the differences computed and " +
"answered, its verdict and its ask. Narrowed to one by id.",
Input: schema(map[string]string{"id": "one proposal's id"}, nil)},
{Name: PlacementVerbPropose, Optional: true, Description: "Propose a change of placement, with why: laid over " +
"the current keys and refused at once, naming every finding, when the result cannot be placed (two modules " +
"on one host port, a provider not placed, a seat held once held twice, two nodes with one address, two " +
"hubs, a module the indexer does not hold, a node with no record). Otherwise the nodes it touches compose " +
"it in shadow and answer, and on a passing verdict the operator is asked to approve it. Who proposes is the " +
"caller, as the runtime proves it.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"why", "changes"},
"properties": map[string]any{
"why": map[string]any{"type": "string", "description": "why, shown to the operator with the proposal"},
"changes": changes,
}}},
{Name: PlacementVerbWithdraw, Optional: true, Description: "Withdraw an open proposal: its key and its " +
"candidates are deleted and placement-withdrawn carries it whole. With why.",
Input: schema(map[string]string{"id": "the proposal's id", "why": "why, kept in the event"}, []string{"id", "why"})},
{Name: PlacementVerbAgain, Optional: true, Description: "Judge an open proposal now with the answers that " +
"stand, and raise its ask again when its verdict passes and the operator's channel does not hold the ask. " +
"Nothing more: the recovery of a proposal whose judgement or ask was lost.",
Input: schema(map[string]string{"id": "the proposal's id"}, []string{"id"})},
{Name: PlacementVerbAdopt, Optional: true, Description: "Take placement over from the controller: its " +
"placement-export is proposed whole as the seed, every node's key, which every node must answer as the same. " +
"Its Approve is the take-over; at the warrant the export is read again and a changed one is proposed anew.",
Input: schema(map[string]string{"why": "why, shown to the operator"}, []string{"why"})},
{Name: PlacementVerbHandBack, Optional: true, Description: "The operator's alone, recorded as a hand act: " +
"delete every placement key, the history kept, so every node returns to the controller's compositions. " +
"Refused after the lease write.",
Input: schema(map[string]string{"why": "why, recorded with the hand act"}, []string{"why"})},
}
}
+14
View File
@@ -123,6 +123,14 @@ func namesWithoutSteppingBack(s Seat, p string) bool {
// as written would pass for a path nobody named.
func declaredPaths(m Manifest) map[string]bool {
out := map[string]bool{}
if m.OwnsPaths != nil {
// Carried: what its manifest declares, never what runs of it (issue 1126, seed-8: f1-circuits, snake and txt-game
// on ace owned their /var/lib paths on the node and none on the controller).
for _, p := range m.OwnsPaths {
out[p] = true
}
return out
}
for _, r := range m.Resources {
if p, ok := r["path"].(string); ok && strings.HasPrefix(p, "/") {
out[path.Clean(p)] = true
@@ -136,6 +144,12 @@ func declaredPaths(m Manifest) map[string]bool {
return out
}
// DeclaredPaths is every machine path a module's manifest declares it owns (declaredPaths), sorted and never nil: what
// a carried module owns (Manifest.OwnsPaths).
func DeclaredPaths(m Manifest) []string {
return append([]string{}, sortedKeys(declaredPaths(m))...)
}
// declaresPath says whether a module declares one path, as a resource or as a file the mesh renders.
func declaresPath(m Manifest, p string) bool { return declaredPaths(m)[p] }
+3
View File
@@ -258,6 +258,9 @@ var defaultSeats = append([]Seat{
// machine, and a group of them, recorded and ordered by its holder, which asks the controller for each act.
// What it says it says as its module (`mesh-delivery.transition`), through the runtime that launches it.
{Name: DeliverySeat, Scope: ScopeMesh, Serves: deliveryVerbs(), Decision: "novox/hq ADR 0239"},
// Placement (novox/hq ADR 0567): which module runs on which node, state on the bus held by the assigner, changed only by
// a proposal the nodes answer and the operator approves. Its states and events are the holder module's.
{Name: PlacementSeat, Scope: ScopeMesh, Serves: placementVerbs(), Decision: "novox/hq ADR 0567"},
// Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a
// delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight.
// They keep their names until that migration is done deliberately, apart from the node-* pass.
+8 -2
View File
@@ -1,6 +1,7 @@
package catalogue
import (
"reflect"
"regexp"
"strings"
"testing"
@@ -43,6 +44,11 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// Forty-seven with mesh-placement (novox/hq ADR 0567), serving its seven verbs.
if s, ok := SeatNamed(PlacementSeat); !ok || s.Scope != ScopeMesh || !reflect.DeepEqual(VerbNames(s.Serves),
[]string{"placement", "proposals", "propose", "withdraw", "again", "adopt", "hand-back"}) {
t.Errorf("the placement seat is %v, %v", ok, s)
}
// Forty-six with the three build seats replacing node-build-agent and the retired mesh-build-machine (novox/hq
// ADR 0525 §2, §10); forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes.
// Forty-four with node-provisioner (novox/hq ADR 0349 clause 3); forty-three with mesh-provisioner (novox/hq ADR 0349 clause 3); forty-two with mesh-config (novox/hq ADR 0348 §9); forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
@@ -56,8 +62,8 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired
// mesh-build-machine row goes, when no registered manifest claims it.
if len(Seats()) != 46 {
t.Errorf("the mesh defines %d seats rather than 46; the set is closed, so a change here is "+
if len(Seats()) != 47 {
t.Errorf("the mesh defines %d seats rather than 47; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}
+2
View File
@@ -51,6 +51,8 @@ type UnsetSettingError struct {
// where says what asks for it — "a file", "a service whose unit", "a directory <id> whose owner" — and
// setToday which of the module's keys a layer sets, for saying it again in the config form's words.
where, setToday string
// field is the key of a contribution whose value asks for it, where the contribution's words are made by its caller.
field string
}
func (e *UnsetSettingError) Error() string { return e.said }
+11 -3
View File
@@ -5,6 +5,7 @@ import (
"crypto/x509"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"regexp"
"sort"
@@ -129,7 +130,12 @@ func ApplySettings(resource map[string]any, layers []Layer) (map[string]any, err
// contract. Until issue 696, a layer replaced any served key it named; influxdb on ace was the one
// layer relying on it, and serves `${setting:org}` and `${setting:bucket}` since.
func Settle(base map[string]any, layers []Layer) (map[string]any, error) {
return filledFromSettings(deepCopy(base), layers, "what is served")
out, err := filledFromSettings(deepCopy(base), layers, "what is served")
if err != nil {
// Said as it was, untyped: what is served names no module here to type it by.
return nil, errors.New(err.Error())
}
return out, nil
}
// ServedSettings is every key a module's served values and its zone's name fill from its settings (`${setting:<key>}`
@@ -193,10 +199,12 @@ func filledFromSettings(merged map[string]any, layers []Layer, what string) (map
for _, asked := range settingsUsed(s) {
v, set := settingValue(layers, asked)
if !set {
return nil, fmt.Errorf(
// Typed, its module and where left to the caller that knows them (Resolution.composed), so a module in
// the config form is refused in its own words (unsetInConfigWords; issue 1126, seed-8).
return nil, &UnsetSettingError{Setting: asked, field: key, setToday: orNoSettings(layers), said: fmt.Sprintf(
"%s says ${setting:%s} for %q, and nothing sets %q — an operator's value is the "+
"assignment's, never the definition's (novox/hq ADR 0112)%s",
what, asked, key, asked, orNoSettings(layers))
what, asked, key, asked, orNoSettings(layers))}
}
s = strings.ReplaceAll(s, "${setting:"+asked+"}", plainly(v))
}
-139
View File
@@ -1,139 +0,0 @@
// placement-seed writes the catalogue's first placement files (novox/hq ADR 0549 §4, ADR 0555 §6, issue 1126) from the
// answer of the controller's one read-only verb `placement-export`: one `placement/<node>.json` per node, each the
// node's placement as the controller's own code read it from its own store, checked as the catalogue's check checks it
// (delivery.CheckPlacements) before anything is written. The pull request it makes merges only when its composition check
// composes every node equal to what it runs.
//
// placement-seed -catalogue <checkout of mesh-catalog> < export.json
// placement-seed -catalogue <dir> -export export.json
//
// export.json is the verb's answer as it prints it ({"<node>": <placement>, ...}, or the verb's wrapping with it under
// "answer"). It opens no network connection, bus or store; it writes only <dir>/placement/<node>.json, and refuses to
// write over a placement file already there. Exit codes: 0 written; 1 the export cannot be read, is refused by the
// check, or a file is there; 2 the flags are wrong.
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func main() {
catalogue := flag.String("catalogue", "", "the checkout of the catalogue to write placement/<node>.json into")
export := flag.String("export", "", "the verb's answer; standard input when absent")
flag.Parse()
if *catalogue == "" || flag.NArg() > 0 {
fmt.Fprintln(os.Stderr, "usage: placement-seed -catalogue <dir> [-export <file>]")
os.Exit(2)
}
in := io.Reader(os.Stdin)
if *export != "" {
f, err := os.Open(*export)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer f.Close()
in = f
}
written, err := Seed(in, *catalogue)
for _, path := range written {
fmt.Println(path)
}
if err != nil {
fmt.Fprintln(os.Stderr, "placement-seed:", err)
os.Exit(1)
}
}
// Seed reads the export and writes each node's placement file under root, answering the paths written. Nothing is
// written when the export does not read, the check refuses it, or a placement file is already there.
func Seed(export io.Reader, root string) ([]string, error) {
raw, err := io.ReadAll(export)
if err != nil {
return nil, err
}
placements, err := read(raw)
if err != nil {
return nil, err
}
all := func(string) bool { return true }
if refused := delivery.CheckPlacements(placements, all, all); len(refused) > 0 {
return nil, fmt.Errorf("the export is refused by the catalogue's check: %v", refused)
}
nodes := make([]string, 0, len(placements))
for node := range placements {
nodes = append(nodes, node)
}
sort.Strings(nodes)
files := map[string][]byte{}
for _, node := range nodes {
p := placements[node]
p.Revision = "" // only the indexer writes a revision, into the key
if _, err := delivery.EncodePlacement(p); err != nil {
return nil, fmt.Errorf("%s: %w", node, err)
}
p.V = delivery.Version
body, err := json.MarshalIndent(p, "", " ")
if err != nil {
return nil, err
}
path := filepath.Join(root, filepath.FromSlash(delivery.PlacementFile(node)))
if _, err := os.Stat(path); err == nil {
return nil, fmt.Errorf("%s is already there; the seed is written once (ADR 0549 §4)", path)
} else if !errors.Is(err, os.ErrNotExist) {
return nil, err
}
files[path] = append(body, '\n')
}
var written []string
for _, node := range nodes {
path := filepath.Join(root, filepath.FromSlash(delivery.PlacementFile(node)))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return written, err
}
if err := os.WriteFile(path, files[path], 0o644); err != nil {
return written, err
}
written = append(written, path)
}
return written, nil
}
// read is the export as the verb prints it, or wrapped as a verb's answer (the controller's verbAnswer: its JSON under
// "answer").
func read(raw []byte) (map[string]delivery.Placement, error) {
var wrapped struct {
Answer json.RawMessage `json:"answer"`
}
if json.Unmarshal(raw, &wrapped) == nil && len(wrapped.Answer) > 0 {
raw = wrapped.Answer
}
var byNode map[string]json.RawMessage
if err := json.Unmarshal(raw, &byNode); err != nil {
return nil, fmt.Errorf("the export is not {<node>: <placement>}: %w", err)
}
if len(byNode) == 0 {
return nil, errors.New("the export names no node")
}
out := map[string]delivery.Placement{}
for node, body := range byNode {
p, err := delivery.DecodePlacement(body)
if err != nil {
return nil, fmt.Errorf("%s: %w", node, err)
}
if p.Node != node {
return nil, fmt.Errorf("the export's placement of %s names the node %q", node, p.Node)
}
out[node] = p
}
return out, nil
}
-61
View File
@@ -1,61 +0,0 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/compose/frombus"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Issue 1126 (ADR 0549 §4, ADR 0555 §6): the seed writes one placement file per node of the export, as the catalogue's
// check and the indexer read it, from the verb's answer bare or wrapped; it writes nothing for an export the check
// refuses, and never over a file already there.
func TestTheSeedWritesEachNodesPlacementFileOnce(t *testing.T) {
export := `{"answer":{"ace":{"v":1,"node":"ace","revision":"x","decided":{"address":"10.77.0.2"},"modules":[{"module":"plex","ports":{"32400":32400}}]},` +
`"novox":{"v":1,"node":"novox","decided":{"hub":true,"address":"10.77.0.1"},"modules":[]}}}`
root := t.TempDir()
written, err := Seed(strings.NewReader(export), root)
if err != nil || len(written) != 2 {
t.Fatalf("written %v, %v", written, err)
}
raw, err := os.ReadFile(filepath.Join(root, "placement", "ace.json"))
if err != nil {
t.Fatal(err)
}
p, err := delivery.DecodePlacement(raw)
if err != nil || p.Node != "ace" || p.Revision != "" || p.HostPorts()["plex"][32400] != 32400 {
t.Errorf("ace's file reads %+v, %v", p, err)
}
if _, err := Seed(strings.NewReader(export), root); err == nil || !strings.Contains(err.Error(), "already there") {
t.Errorf("a second seed over the files: %v", err)
}
clash := `{"ace":{"v":1,"node":"ace","decided":{"address":"10.77.0.1"},"modules":[]},"novox":{"v":1,"node":"novox","decided":{"address":"10.77.0.1"},"modules":[]}}`
other := t.TempDir()
if _, err := Seed(strings.NewReader(clash), other); err == nil || !strings.Contains(err.Error(), "both have the address") {
t.Errorf("an export the check refuses: %v", err)
}
if entries, _ := os.ReadDir(filepath.Join(other, "placement")); len(entries) != 0 {
t.Errorf("a refused export wrote %v", entries)
}
// What the controller's placement-export prints, frombus.Export per node, is what the seed takes: a node with no
// module placed included.
in := compose.Input{Nodes: []compose.Node{{Name: "solo", Place: compose.Place{Hub: true, Address: "10.77.0.9"}}}}
exported, err := frombus.Export(in, "solo")
if err != nil {
t.Fatal(err)
}
answer, _ := json.Marshal(map[string]any{"answer": map[string]delivery.Placement{"solo": exported}})
if written, err := Seed(strings.NewReader(string(answer)), t.TempDir()); err != nil || len(written) != 1 {
t.Errorf("Export's answer %s was not seeded: %v, %v", answer, written, err)
}
for _, bad := range []string{`[]`, `{}`, `{"ace":{"v":1,"node":"novox","modules":[]}}`} {
if _, err := Seed(strings.NewReader(bad), t.TempDir()); err == nil {
t.Errorf("%s was seeded", bad)
}
}
}
+242
View File
@@ -8,6 +8,7 @@ import (
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
@@ -54,11 +55,19 @@ func CompositionsShadow(in Input, node string, fps map[string]string) (out map[s
return compositions(in, node, fps, Foreseeing, false)
}
// CompositionsIn is the node's compositions in mode with what they stand in for answered (stood, sorted, the bus's user
// list left out) rather than refused: CompositionsShadow in Foreseeing, and in Reading what CompositionsApplied refuses
// before it is refused, so a caller that applies refuses each stand-in to the module it is for alone (issue 1126).
func CompositionsIn(in Input, node string, fps map[string]string, mode Mode) (out map[string]delivery.Composed, skipped, stood []string, err error) {
return compositions(in, node, fps, mode, false)
}
func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]delivery.Composed, skipped, stood []string, err error) {
reading := in
reading.Nodes = append([]Node(nil), in.Nodes...)
found := false
inOwn := map[string]bool{}
held := map[string]bool{}
for i := range reading.Nodes {
if reading.Nodes[i].Name == node {
reading.Nodes[i].ReadsModules = true
@@ -66,6 +75,9 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
for m, ok := range reading.Nodes[i].InOwn {
inOwn[m] = ok
}
for m, ok := range reading.Nodes[i].Held {
held[m] = ok
}
}
}
// The private network's module is built by nobody: no fingerprint of the new form ever names it.
@@ -76,6 +88,54 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
if !found {
return nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
}
// A folded module built by a build of the old form (Node.RunsOwn) is left to the node: composed from what runs of it
// (Carried), so what it provides and requires still shapes the rest and what the others read of it (its published
// ports, the paths it owns) is what runs, and its resources are then the ones the node runs.
carried := map[string]bool{}
var runsOwn *delivery.Composed
for i := range reading.Nodes {
n := &reading.Nodes[i]
if n.Name != node || n.RunsOwn == nil {
continue
}
runsOwn = n.RunsOwn
candidates := map[string]bool{}
for m := range inOwn {
candidates[m] = true
}
for m := range held {
candidates[m] = true
}
for m := range candidates {
// A held module is carried as it runs wherever the node's own runs it, folded or leaving the fold for a
// composition of its own (its want new, its names still unset): its stub is never what the node's own applies,
// and what the node's own applies never drops what runs of it (issue 1126, review of the apply; issue 1945).
heldHere := held[m] && RunsAnyOf(runsOwn, m)
if !folds(m) && !heldHere {
continue
}
man, ok := n.Kept[m]
if !ok {
man, ok = reading.Catalogue[m]
}
if !ok || (!NamesBuild(man) && !held[m]) {
continue
}
carried[m] = true
kept := make(map[string]catalogue.Manifest, len(n.Kept)+1)
for k, v := range n.Kept {
kept[k] = v
}
kept[m] = Carried(man, m, runsOwn)
n.Kept = kept
}
}
isCarried := func(module, id string) bool {
if module == "" {
module, _ = declaration.ModuleGuess(id)
}
return carried[module]
}
d, problems, err := ComposeIn(reading, node, mode)
if err != nil {
return nil, nil, nil, err
@@ -118,6 +178,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
left := map[string]bool{}
for _, raw := range envelope.Resources {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
Type string `json:"type"`
Path string `json:"path"`
@@ -125,6 +186,9 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
if err := json.Unmarshal(raw, &head); err != nil {
return nil, nil, nil, err
}
if isCarried(head.Module, head.ID) {
continue // what the stub composed of it: the node's own carries what runs of it instead
}
if folds(head.Module) {
// Into the node's own, in its place among the node's resources, its module taken off (mesh-controller's
// compositionsOf): the node's own composition is then the controller's, byte for byte.
@@ -151,6 +215,18 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
}
kept = append(kept, raw)
}
// What runs of each carried module, in the node's own composition, each after the resource it runs after.
if len(carried) > 0 {
var err error
if kept, foldedPaths, err = carry(kept, foldedPaths, runsOwn, isCarried); err != nil {
return nil, nil, nil, err
}
for _, p := range runsOwn.Filled {
if foldedPaths[p] && !slices.Contains(envelope.Filled, p) {
envelope.Filled = append(envelope.Filled, p)
}
}
}
// A filled file of a module's own composition goes with that module (SplitComposed); one of a folded module goes with
// the node's own, after them, in the declaration's order; one of a module left out is dropped.
var filled, ownFilled []string
@@ -177,6 +253,10 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
out[own] = c
}
for key, c := range out {
if held[c.Module] {
delete(out, key) // its stub's: the node keeps what it last applied of it (Node.Held)
continue
}
if m, ok := in.Catalogue[c.Module]; ok && c.Module != "" {
c.Data = len(m.DataItems()) > 0
out[key] = c
@@ -188,3 +268,165 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
sort.Strings(skipped)
return out, skipped, stood, nil
}
// RunsAnyOf says runs (a composition as it runs) holds a resource of module, by its module or the module its id names.
func RunsAnyOf(runs *delivery.Composed, module string) bool {
if runs == nil {
return false
}
for _, raw := range runs.Resources {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
}
if json.Unmarshal(raw, &head) != nil {
continue
}
owner := head.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(head.ID)
}
if owner == module {
return true
}
}
return false
}
// NamesBuild says a manifest names what a build makes: its build, or a resource naming an artifact. A node cannot
// compose such a module folded into its own composition: the build of the old form it runs is held by the controller
// alone (issue 1126).
func NamesBuild(m catalogue.Manifest) bool {
if m.Build != nil {
return true
}
for _, r := range m.Resources {
if _, ok := r["artifact"]; ok {
return true
}
}
return false
}
// Stub is m with nothing to build and no resource but its directories: what it provides, requires, serves and grants
// still shape the rest of the composition, the places its own secrets, data and grants name (${dir:…}) still resolve,
// and nothing else of its own is composed (issue 1126: nats's ${dir:mesh-state} on novox). A module refused in a
// node's answer is composed so; a folded one the node runs is Carried.
func Stub(m catalogue.Manifest) catalogue.Manifest {
// The paths it owns stay its manifest's, as the controller tells the holder of node-files them (issue 1126, seed-8).
if m.OwnsPaths == nil {
m.OwnsPaths = catalogue.DeclaredPaths(m)
}
m.Build = nil
var dirs []map[string]any
for _, r := range m.Resources {
if r["type"] == "directory" {
dirs = append(dirs, r)
}
}
m.Resources = dirs
return m
}
// Carried is m, folded into its node's own composition at a build of the old form, as the node runs it (issue 1126,
// C5): nothing to build; its containers and directories those the node's own composition (runs) carries of it, their
// module's word taken off their ids, with the manifest's directories it does not run. So what other modules read of it
// is what runs: the machine ports its containers publish (the filter's rules, the routes reaching it: MachineSide). The
// paths it owns, which the holder of node-files is told, are its manifest's, as for its Stub: the controller composes
// them from the manifest, not from every path a resource of it runs at (issue 1126, seed-8: f1-circuits, snake and
// txt-game on ace, whose manifests leave their directories to the mesh). Its other resources are not taken, since the
// composition makes some of them again from its manifest (its secrets' files) and one id twice is refused; none of them
// is composed into what is applied either way (compositions leaves a carried module's out, and carries the ones that
// run). What it provides, requires, listens on and grants is its manifest's. A module whose resources runs holds none
// of is its Stub, as before.
func Carried(m catalogue.Manifest, module string, runs *delivery.Composed) catalogue.Manifest {
if runs == nil {
return Stub(m)
}
var taken []map[string]any
runsAny := false
ids := map[string]bool{}
for _, raw := range runs.Resources {
var r map[string]any
if json.Unmarshal(raw, &r) != nil {
continue
}
id, _ := r["id"].(string)
owner, _ := r["module"].(string)
if owner == "" {
owner, _ = declaration.ModuleGuess(id)
}
if owner != module {
continue
}
runsAny = true
if typ := r["type"]; typ != "container" && typ != "directory" {
continue
}
r["id"] = strings.TrimPrefix(id, module+".")
delete(r, "module")
ids[r["id"].(string)] = true
taken = append(taken, r)
}
if !runsAny {
return Stub(m)
}
stub := Stub(m)
for _, r := range stub.Resources {
if id, _ := r["id"].(string); !ids[id] {
taken = append(taken, r)
}
}
stub.Resources = taken
return stub
}
// carry puts into the node's resources (kept, in the declaration's order) the resources of the node's own composition as
// it runs (runs) that belong to a carried module, each after the resource it runs after, or first when that one is
// not among them; a file among them is a folded path.
func carry(kept []json.RawMessage, foldedPaths map[string]bool, runs *delivery.Composed, carried func(module, id string) bool) ([]json.RawMessage, map[string]bool, error) {
idOf := func(raw json.RawMessage) (string, string, string, string, error) {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
Type string `json:"type"`
Path string `json:"path"`
}
err := json.Unmarshal(raw, &head)
return head.ID, head.Module, head.Type, head.Path, err
}
present := map[string]bool{}
for _, raw := range kept {
if id, module, _, _, err := idOf(raw); err == nil && module == "" {
present[id] = true
}
}
// By the id of the resource of the node's own it runs after, "" for first: the last one before it that is composed.
after := map[string][]json.RawMessage{}
anchor := ""
for _, raw := range runs.Resources {
id, module, typ, path, err := idOf(raw)
if err != nil {
return nil, nil, fmt.Errorf("compose: a resource the node runs in its own composition: %w", err)
}
if !carried(module, id) {
if module == "" && present[id] {
anchor = id
}
continue
}
after[anchor] = append(after[anchor], raw)
if typ == string(declaration.TypeFile) {
foldedPaths[path] = true
}
}
out := make([]json.RawMessage, 0, len(kept)+len(runs.Resources))
out = append(out, after[""]...)
for _, raw := range kept {
out = append(out, raw)
if id, module, _, _, err := idOf(raw); err == nil && module == "" {
out = append(out, after[id]...)
}
}
return out, foldedPaths, nil
}
+423 -11
View File
@@ -2,27 +2,31 @@ package frombus
import (
"context"
"encoding/json"
"fmt"
"regexp"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Answer is a node's answer to its group's placement candidates (novox/hq ADR 0559 §4): s.Node composed in shadow with
// Answer is a node's answer to its proposal's placement candidates (novox/hq ADR 0567 §3, ADR 0559 §4): s.Node composed in shadow with
// s.Placements laid over the trunk's placement (compose.CompositionsShadow, its own sealed values and boxes through r as
// for its own composition), compared with what it runs (delivery.CompareComposed). It applies nothing and writes
// nothing; head and revision are the candidate's, said back in the answer.
// nothing; the proposal and the revision are the candidate's, said back in the answer.
//
// The node's own composition is compared only where the node runs one. fps gives the build each module is composed at: what the node runs, and for a module the candidate places that the
// node does not run, the build the trunk holds; a placed module with none is added, with no build of it known. What the
// composition stands in for is judged: a value of a module the group adds (on this node or another: its credential, own
// composition stands in for is judged: a value of a module the proposal adds (on this node or another: its credential, own
// secrets and boxes are made only once it is placed, ADR 0549 §5) is not yet issued, which is no refusal; any other
// stand-in is a value missing for a module the node runs, a refusal naming it ("node" when it is the node's own).
func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string, running []delivery.Composed, head string, revision uint64) delivery.CandidateAnswer {
a := delivery.CandidateAnswer{Head: head, Revision: revision}
func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string, running []delivery.Composed, proposal string, revision uint64) delivery.CandidateAnswer {
a := delivery.CandidateAnswer{Proposal: proposal, Revision: revision}
refuse := func(err error) delivery.CandidateAnswer {
a.Refused = err.Error()
a.Verdict = delivery.CandidateRefused
@@ -32,7 +36,13 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
if err != nil {
return refuse(fmt.Errorf("frombus: placement: %w", err))
}
in, err := Input(ctx, r, mesh, s)
if s.Running == nil {
s.Running = running
}
// Every module it cannot compose is named, not the first only (issue 1126): a build it cannot read or take in, and
// below, a module the composition refuses, each composed as a stub so the rest is composed and compared.
failed := map[string]string{}
in, err := input(ctx, r, mesh, s, func(module string, err error) { failed[module] = err.Error() })
if err != nil {
return refuse(err)
}
@@ -62,9 +72,14 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err))
}
}
composed, skipped, stood, err := compose.CompositionsShadow(in, s.Node, fps)
composed, skipped, stood, err := composeEach(in, s.Node, fps, false, s.Running, failed)
if err != nil {
return refuse(err)
// Not one module's: the node's own (a value served elsewhere, the mesh's network), said once as the node's.
if len(failed) == 0 {
return refuse(err)
}
failed[delivery.HaveNodeWord] = err.Error()
composed, skipped, stood = nil, nil, nil
}
// The node's own composition is judged only where the node runs one: one that runs none applies it only once a want
// names it (the node agent's own composer hands it on that want alone), so a candidate does not add it.
@@ -79,10 +94,51 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
}
}
}
cmp := delivery.CompareComposed(composed, running)
a.Same, a.Differs, a.Removed = cmp.Same, cmp.Differs, cmp.Removed
if _, whole := failed[delivery.HaveNodeWord]; whole {
// Nothing composed to compare: every refusal is said, and nothing else.
for _, m := range sortedKeys(failed) {
a.Refusals = append(a.Refusals, delivery.ModuleRefused{Module: m, Why: failed[m]})
}
a.Verdict = delivery.VerdictOf(a)
return a
}
// A module the composition folds inside the node's own is judged there: the agent also says it runs by module, from
// the declaration it kept before the switch (agent.Everything), which is no removal (issue 1126, seed #531).
if len(s.InOwn) > 0 {
var judged []delivery.Composed
for _, c := range running {
if c.Module == "" || !slices.Contains(s.InOwn, c.Module) {
judged = append(judged, c)
}
}
running = judged
}
cmp := delivery.CompareComposedWith(composed, running, s.SameValue)
// The answer rides in the node's have key: its shown bodies are bounded, the rest named only.
budget := MaxShown
for i := range cmp.Differs {
kept := cmp.Differs[i].Bodies[:0]
for _, b := range cmp.Differs[i].Bodies {
if n := len(b.Composed) + len(b.Running); n <= budget {
budget -= n
kept = append(kept, b)
} else {
kept = append(kept, delivery.ResourceBodies{ID: b.ID, Why: b.Why})
a.Note = "some differing resources are named without their bodies: the answer holds at most " +
fmt.Sprint(MaxShown) + " bytes of them"
}
}
cmp.Differs[i].Bodies = kept
}
// A refused module was composed as a stub: what it composed is no difference; its refusal is said.
notFailed := func(m string) bool { _, ok := failed[m]; return !ok }
a.Same = slices.DeleteFunc(cmp.Same, func(m string) bool { return !notFailed(m) })
a.Removed = slices.DeleteFunc(cmp.Removed, func(m string) bool { return !notFailed(m) })
a.Differs = slices.DeleteFunc(cmp.Differs, func(d delivery.ModuleDiffers) bool { return !notFailed(d.Module) })
cmp.Added = slices.DeleteFunc(cmp.Added, func(d delivery.ModuleAdded) bool { return !notFailed(d.Module) })
skipped = slices.DeleteFunc(skipped, func(m string) bool { return !notFailed(m) })
// What the group adds, by node and module: placed by a candidate and not by the trunk on that node.
// What the proposal adds, by node and module: placed by a candidate and not by the trunk on that node.
added := map[string]bool{}
for node, p := range s.Placements {
if p == nil {
@@ -138,13 +194,25 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
if _, isAdded := addedHere[m]; isAdded {
continue // a module the candidate adds here: what it lacks is said under Added
}
if _, isFailed := failed[m]; isFailed {
continue // said below, as why it could not be composed
}
a.Refusals = append(a.Refusals, delivery.ModuleRefused{Module: m, Why: "stands in for " + strings.Join(refusals[m], ", ")})
}
for _, m := range sortedKeys(failed) {
a.Refusals = append(a.Refusals, delivery.ModuleRefused{Module: m, Why: failed[m]})
}
sort.Slice(a.Refusals, func(i, j int) bool { return a.Refusals[i].Module < a.Refusals[j].Module })
a.Verdict = delivery.VerdictOf(a)
return a
}
// maxCulprits bounds the modules one refusal is tried on, each a whole composition.
const maxCulprits = 8
// MaxShown bounds the bytes of shown resource bodies in one answer (delivery.ModuleDiffers.Bodies).
const MaxShown = 256 << 10
var (
grantStandIn = regexp.MustCompile(`^the grant of \S+ for (\S+) on (\S+)$`)
needsStandIn = regexp.MustCompile(`^(\S+) on (\S+) needs `)
@@ -170,3 +238,347 @@ func standsInFor(x, node string) (string, string) {
}
return node, ""
}
// composeEach is the one rule, for the node that applies and for its answer to a candidate alike, by which a module's
// own refusal refuses that module and never the node (issue 1126: seed-8 fixed it in the answer alone, and the
// applying path still refused all of novox for de-spiegel's unset route names). The node is composed; a refusal is laid
// on the module whose stub (compose.Stub) changes it (culprit), that module is held (compose.Node.Held) and said in
// failed with the refusal's words, and the rest is composed again, until it composes or the refusal is no single
// module's, which is answered as the node's (err). Applied composes as the node applies (compose.Reading) and holds the
// module placed here each stand-in is for; one for no module placed here (the node's own, a value this node holds for a
// module elsewhere) refuses the node (compose.ErrStoodIn). Not applied, it composes in compose.Foreseeing and answers
// every stand-in for the caller to judge (Answer).
//
// **A held module is what runs of it** (review of the apply, round 1): composed from what running holds of it
// (compose.Carried of its own composition there; the node's own carries it wherever it runs there), so what the others
// read of it (the machine ports its containers publish, the routes reaching it) is what runs, and none of its own
// composition is in out: the node keeps what it last applied. **What depends on how it is held is held too:** a module
// whose composition differs when the held ones are composed as their stubs instead is held with them, and a node's own
// composition that differs (but for the held modules' own resources) refuses the node. And since a held module is
// composed from its new manifest, which may not be the build's that runs, what reads that manifest is held with it
// (dependents): the providers of what it contributes to (the proxy its routes), and, when it listens, the filter.
func composeEach(in compose.Input, node string, fps map[string]string, applied bool, running []delivery.Composed, failed map[string]string) (out map[string]delivery.Composed, skipped, stood []string, err error) {
mode := compose.Foreseeing
if applied {
mode = compose.Reading
}
runs := map[string]*delivery.Composed{}
for i := range running {
if m := running[i].Module; m != "" {
runs[m] = &running[i]
}
}
src := map[string]catalogue.Manifest{} // each held module's manifest before it was held
bareOf := map[string]bool{}
var hold func(m, why string, bare bool)
hold = func(m, why string, bare bool) {
if _, done := failed[m]; !done {
failed[m] = why // a held module tried bare keeps the words it was first held for
}
first := false
if _, ok := src[m]; !ok {
man, ok := manifestOn(in, node, m)
if !ok {
return
}
src[m], first = man, true
}
bareOf[m] = bareOf[m] || bare
in = heldAs(in, node, m, src[m], bareOf[m], runs[m])
if !first {
return
}
// What the others read of its manifest (what it contributes, what it listens on) is its new manifest's, not
// necessarily the build's that runs: what reads it is held with it (review of the apply, round 2).
for _, r := range dependents(in, node, src[m], failed) {
hold(r, "it reads what "+m+" contributes or listens on, and "+m+" is held", false)
}
}
for _, m := range sortedKeys(failed) {
hold(m, failed[m], false)
}
for {
out, skipped, stood, err = compose.CompositionsIn(in, node, fps, mode)
if err != nil {
module, bare := culprit(in, node, fps, mode, failed, err)
if module == "" {
return nil, nil, nil, err
}
hold(module, err.Error(), bare)
continue
}
again := false
if applied && len(stood) > 0 {
placed := map[string]bool{}
for _, m := range assignedOn(in, node) {
placed[m] = true
}
of := map[string][]string{}
var nodes []string
for _, x := range stood {
if n, m := standsInFor(x, node); n == node && placed[m] {
of[m] = append(of[m], x)
} else {
nodes = append(nodes, x)
}
}
for _, m := range sortedKeys(of) {
if _, done := failed[m]; !done {
hold(m, "stands in for "+strings.Join(of[m], ", "), false)
again = true
}
}
if !again && len(nodes) > 0 {
return nil, nil, nil, fmt.Errorf("%w on %s: %s", compose.ErrStoodIn, node, strings.Join(nodes, ", "))
}
}
if !again && len(failed) > 0 {
dependent, own := dependsOnHeld(in, node, fps, mode, out, src, bareOf, failed)
if own {
return nil, nil, nil, fmt.Errorf("frombus: %s's own composition depends on how %s, held, is composed, and is not applied",
node, strings.Join(sortedKeys(failed), ", "))
}
for _, m := range dependent {
hold(m, "what it composes of "+strings.Join(sortedKeys(failed), ", ")+", held, is not what runs of them", false)
again = true
}
}
if !again {
break
}
}
skipped = slices.DeleteFunc(skipped, func(m string) bool { _, held := failed[m]; return held })
return out, skipped, stood, nil
}
// dependsOnHeld composes in again with each held module as its stub rather than as what runs of it, and answers the
// modules, not held, whose composition differs between the two (sorted), and own when the node's own composition does
// but for the held modules' own resources. A composition of stubs that fails says nothing more: what runs composed.
func dependsOnHeld(in compose.Input, node string, fps map[string]string, mode compose.Mode, asRuns map[string]delivery.Composed,
src map[string]catalogue.Manifest, bareOf map[string]bool, failed map[string]string) (dependent []string, own bool) {
stubs := in
for _, m := range sortedKeys(failed) {
if man, ok := src[m]; ok {
stubs = heldAs(stubs, node, m, man, bareOf[m], nil)
}
}
asStubs, _, _, err := compose.CompositionsIn(stubs, node, fps, mode)
if err != nil {
return nil, false
}
isHeld := func(m string) bool { _, ok := failed[m]; return ok }
keys := map[string]bool{}
for k := range asRuns {
keys[k] = true
}
for k := range asStubs {
keys[k] = true
}
for _, k := range sortedKeys(keys) {
a, inA := asRuns[k]
b, inB := asStubs[k]
m := a.Module
if !inA {
m = b.Module
}
if m != "" && isHeld(m) {
continue
}
if m == "" {
a, b = withoutHeld(a, isHeld), withoutHeld(b, isHeld)
}
da, errA := delivery.ComposedDigestOf(a)
db, errB := delivery.ComposedDigestOf(b)
if inA == inB && errA == nil && errB == nil && da == db {
continue
}
if m == "" {
own = true
continue
}
dependent = append(dependent, m)
}
return dependent, own
}
// withoutHeld is the node's own composition c without the resources of a held module (its id naming it) and without
// its filled paths, for comparing the rest.
func withoutHeld(c delivery.Composed, isHeld func(string) bool) delivery.Composed {
out := c
out.Filled = nil
out.Resources = nil
for _, raw := range c.Resources {
var head struct {
ID string `json:"id"`
Module string `json:"module"`
}
_ = json.Unmarshal(raw, &head)
owner := head.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(head.ID)
}
if owner != "" && isHeld(owner) {
continue
}
out.Resources = append(out.Resources, raw)
}
return out
}
// dependents is, of the modules placed on node and not held, those that read what man says to them: each that provides
// what man contributes to (the proxy its routes), and, when man listens on a port, each that holds the filter (the
// openings).
func dependents(in compose.Input, node string, man catalogue.Manifest, failed map[string]string) []string {
to := map[string]bool{}
for p := range man.Contributes {
to[p] = true
}
for p := range man.ContributesMany {
to[p] = true
}
listens := len(man.Listens) > 0
if len(to) == 0 && !listens {
return nil
}
var out []string
for _, m := range assignedOn(in, node) {
if _, held := failed[m]; held || m == man.Module {
continue
}
pm, ok := manifestOn(in, node, m)
if !ok {
continue
}
if listens && pm.Filtering != nil {
out = append(out, m)
continue
}
for _, o := range pm.Provides {
if to[o.Name] {
out = append(out, m)
break
}
}
}
return out
}
// assignedOn is the modules placed on node, in placement order.
func assignedOn(in compose.Input, node string) []string {
for _, n := range in.Nodes {
if n.Name == node {
return n.Assigned
}
}
return nil
}
// manifestOn is the manifest node composes module from: its Kept, else the catalogue's.
func manifestOn(in compose.Input, node, module string) (catalogue.Manifest, bool) {
for _, n := range in.Nodes {
if n.Name == node {
if man, ok := n.Kept[module]; ok {
return man, true
}
}
}
man, ok := in.Catalogue[module]
return man, ok
}
// culprit is the module placed on node whose composition the error is of: the first, in placement order, whose stub
// (compose.Stub) makes the composition in mode succeed or fail otherwise; empty when none does (the error is not one
// module's). Modules already refused are passed over. A module whose stub still fails the same is tried bare (its stub
// contributing nothing, bare true): a refusal of what it contributes is its own, which the stub keeps (issue 1126,
// seed-8: de-spiegel's route names on novox, an unset required value, refused the whole node).
func culprit(in compose.Input, node string, fps map[string]string, mode compose.Mode, failed map[string]string, err error) (string, bool) {
var assigned []string
for _, n := range in.Nodes {
if n.Name == node {
assigned = n.Assigned
}
}
// Only the modules the error names: the words of a module's refusal name it. Each try is a whole composition, so a
// refusal naming none is no module's, answered at once rather than after a composition per module (issue 1126: ace's
// answer took 7m46s).
var order []string
for _, m := range assigned {
if strings.Contains(err.Error(), m) && len(order) < maxCulprits {
order = append(order, m)
}
}
changes := func(m string, bare bool) bool {
_, _, _, again := compose.CompositionsIn(stubbed(in, node, m, bare), node, fps, mode)
return again == nil || again.Error() != err.Error()
}
for _, m := range order {
if _, done := failed[m]; done {
continue
}
if changes(m, false) {
return m, false
}
}
var kept map[string]catalogue.Manifest
for _, n := range in.Nodes {
if n.Name == node {
kept = n.Kept
}
}
for _, m := range order {
// One already held is tried bare too, once: held for its build, its stub may still refuse (its own secret
// never issued, issue 1126).
if _, done := failed[m]; done && isBare(kept[m]) {
continue
}
if changes(m, true) {
return m, true
}
}
return "", false
}
// isBare says a stub contributes nothing and declares no own secret (stubbed, bare).
func isBare(m catalogue.Manifest) bool {
return len(m.Contributes) == 0 && len(m.ContributesMany) == 0 && len(m.OwnSecrets) == 0
}
// stubbed is in with module held on node as its stub alone (heldAs, nothing of it running), from the manifest node
// composes it from: a culprit's try.
func stubbed(in compose.Input, node, module string, bare bool) compose.Input {
man, ok := manifestOn(in, node, module)
if !ok {
return in
}
return heldAs(in, node, module, man, bare, nil)
}
// heldAs is in with module held on node (compose.Node.Held): composed from man as what runs of it (compose.Carried of
// runs; its stub, compose.Stub, when runs is nil or holds nothing of it), as node's Kept; bare, contributing nothing to
// what it requires and declaring no own secret: none of a held module's own composition is applied, so what it has not
// been issued (restic's repository on a node it never ran on) is no refusal of the rest.
func heldAs(in compose.Input, node, module string, man catalogue.Manifest, bare bool, runs *delivery.Composed) compose.Input {
out := in
out.Nodes = append([]compose.Node(nil), in.Nodes...)
for i := range out.Nodes {
if out.Nodes[i].Name != node {
continue
}
if bare {
man.Contributes, man.ContributesMany, man.OwnSecrets = nil, nil, nil
}
kept := make(map[string]catalogue.Manifest, len(out.Nodes[i].Kept)+1)
for k, v := range out.Nodes[i].Kept {
kept[k] = v
}
kept[module] = compose.Carried(man, module, runs)
out.Nodes[i].Kept = kept
held := make(map[string]bool, len(out.Nodes[i].Held)+1)
for k, v := range out.Nodes[i].Held {
held[k] = v
}
held[module] = true
out.Nodes[i].Held = held
}
return out
}
+491
View File
@@ -0,0 +1,491 @@
package frombus
import (
"context"
"encoding/json"
"fmt"
"os"
"reflect"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// Issue 1126 (the seed's shadow, round 2): a node's answer names every module it cannot compose, not the first only;
// and a module folded into its own composition that names a build of the old form (de-spiegel on novox, nats, photos,
// systemd-resolved: their haves name a build-... id and no fingerprint) is left to the node, carried as it runs.
func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
ctx := context.Background()
image := "alpine@sha256:" + strings.Repeat("a", 64)
manifest := func(module string) []byte {
// Its data names its directory, as nats's names ${dir:mesh-state} (issue 1126: novox's own composition refused
// "nats says ${dir:mesh-state}, and nats declares no directory mesh-state" when its stub held no directory).
return []byte(`{"module":"` + module + `","version":"1","capabilities":["container-runtime"],` +
`"own-secrets":{"token":"${dir:state}/token"},` +
`"resources":[{"id":"state","type":"directory","path":"/var/lib/` + module + `","mode":"0700"},` +
`{"id":"server","type":"container","name":"` + module + `","artifact":"site"}],` +
`"build":{"artifacts":[{"name":"site","kind":"upstream","from":"` + image + `"}]}}`)
}
lemurs := []byte(`{"module":"lemurs","version":"1","resources":[{"id":"service","type":"service","unit":"lemurs.service","state":"running","boot":"enabled"}]}`)
fp := func(c string) string { return "src2:" + strings.Repeat(c, 64) }
mk := func(placed ...string) *bus {
p := delivery.Placement{Node: "n1", Decided: delivery.NodeDecided{Hub: true, Address: "10.77.0.1"}}
for _, m := range placed {
p.Modules = append(p.Modules, delivery.PlacedModule{Module: m})
}
b := &bus{placements: map[string]delivery.Placement{"n1": p},
records: map[string]delivery.NodeRecord{"n1": {Node: "n1", TunnelKey: publicKey(t), SealingKey: "sk", Capabilities: []string{"systemd", "container-runtime"}}},
modules: map[string]delivery.Module{"lemurs": {Manifest: delivery.ManifestDigest(lemurs)}},
manifests: map[string][]byte{delivery.ManifestDigest(lemurs): lemurs},
builds: map[string]*delivery.BuilderBuild{}}
for _, m := range []string{"web", "api", "spiegel"} {
b.own = append(b.own, compose.OwnSecretRow{Node: "n1", Module: m, Name: "token", Origin: "made", SealedTo: "sk", Sealed: `"box"`})
}
for _, m := range []string{"web", "api", "spiegel"} {
raw := manifest(m)
b.manifests[delivery.ManifestDigest(raw)] = raw
b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
}
// web and api are built, their keys done and waiting for the reference of their image; spiegel has no build of
// the new form at all.
for i, m := range []string{"web", "api"} {
mod := b.modules[m]
mod.FP2 = fp(string(rune('1' + i)))
b.modules[m] = mod
b.builds[m+" "+mod.FP2] = &delivery.BuilderBuild{State: delivery.BuildDone, Manifest: mod.Manifest,
Artifacts: []delivery.BuiltArtifact{{Name: "site", Kind: "image", Digest: "sha256:" + strings.Repeat("b", 64)}}}
}
return b
}
given := func(b *bus) map[string]*delivery.Placement {
p := b.placements["n1"]
return map[string]*delivery.Placement{"n1": &p}
}
// Every refusal: two builds taken in, and a module the composition refuses (its image never built), each named.
b := mk("lemurs", "web", "api", "spiegel")
a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b)},
map[string]string{"lemurs": fp("e"), "web": fp("1"), "api": fp("2"), "spiegel": fp("3")}, nil, "p-h1", 3)
var refused []string
for _, r := range a.Refusals {
refused = append(refused, r.Module)
}
if a.Verdict != delivery.CandidateRefused || a.Refused != "" || !reflect.DeepEqual(refused, []string{"api", "spiegel", "web"}) {
t.Errorf("every refusal: %+v, want api, spiegel and web refused, each named", a)
}
for _, r := range a.Refusals {
want := map[string]string{"api": "waits for the reference of site", "web": "waits for the reference of site",
"spiegel": "spiegel has not been built for this mesh"}[r.Module]
if !strings.Contains(r.Why, want) {
t.Errorf("%s refused as %q, want %q", r.Module, r.Why, want)
}
}
if raw, err := delivery.EncodeHaveNode(delivery.HaveNode{Candidates: map[string]delivery.CandidateAnswer{"g": a}}); err != nil {
t.Errorf("the answer is not one a have holds: %v (%s)", err, raw)
}
if lemursOK := slices.ContainsFunc(a.Added, func(m delivery.ModuleAdded) bool { return m.Module == "lemurs" }); !lemursOK {
t.Errorf("lemurs, which composes, is not answered beside the refusals: %+v", a)
}
// The fold: spiegel runs inside the node's own composition at a build of the old form. Its resources are carried as
// they run, each after the one it runs after: the node's own is the one it runs, and the node composes itself.
b = mk("lemurs", overlay.Name, "spiegel")
p := b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820"
b.placements["n1"] = p
fps := map[string]string{"lemurs": fp("e")}
stub := Scope{Node: "n1"}
in, err := Input(ctx, b, Mesh{}, stub)
if err != nil {
t.Fatal(err)
}
in.Nodes[0].Kept = map[string]catalogue.Manifest{"spiegel": compose.Stub(in.Catalogue["spiegel"])}
base, _, err := compose.Compositions(in, "n1", fps, compose.Reading)
if err != nil {
t.Fatal(err)
}
// A refusal naming no module placed here is no module's: no compose is spent stubbing each in turn (issue 1126: ace's
// answer took 7m46s, one compose per module), though here a stub would compose.
if m, _ := culprit(in, "n1", fps, compose.Foreseeing, map[string]string{}, fmt.Errorf("nothing in this mesh provides \"artifact-store\"")); m != "" {
t.Errorf("a refusal naming no placed module was laid on %s", m)
}
ownKey, _ := delivery.ComposedNodeKey("n1")
own := base[ownKey]
carriedRes := []json.RawMessage{
json.RawMessage(`{"id":"spiegel.state","type":"directory","path":"/var/lib/spiegel","mode":"0700"}`),
json.RawMessage(`{"id":"spiegel.server","type":"container","name":"spiegel","image":"registry.example/spiegel@sha256:` + strings.Repeat("c", 64) + `"}`)}
// They run after the node's first own resource.
own.Resources = append(append(append([]json.RawMessage{}, own.Resources[:1]...), carriedRes...), own.Resources[1:]...)
var running []delivery.Composed
for k, c := range base {
if k == ownKey {
c = own
}
running = append(running, c)
}
scope := Scope{Node: "n1", InOwn: []string{"spiegel"}, Running: running}
got, _, held, err := Compose(ctx, b, Mesh{}, scope, fps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatalf("the node does not compose itself with spiegel folded at a build of the old form: %v", err)
}
gotOwn, _ := json.Marshal(got[ownKey].Resources)
wantOwn, _ := json.Marshal(own.Resources)
if string(gotOwn) != string(wantOwn) {
t.Errorf("the node's own composition is\n%s\nwant what runs\n%s", gotOwn, wantOwn)
}
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"spiegel"}}, fps, running, "p-h1", 3)
if a.Verdict != delivery.CandidateSame {
t.Errorf("the fold's answer: %+v, want the same", a)
}
// (A) The agent also says spiegel runs by module, from the declaration kept before the switch (agent.Everything): a
// module the composition folds is judged inside the node's own, not as removed (issue 1126, seed #531's answer).
keptToo := append(append([]delivery.Composed{}, running...), delivery.Composed{Node: "n1", Module: "spiegel", Resources: carriedRes})
if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"spiegel"}}, fps, keptToo, "p-h1", 3); a.Verdict != delivery.CandidateSame || len(a.Removed) != 0 {
t.Errorf("a folded module running from the kept declaration is answered %+v, want the same", a)
}
// (B) The node's own composition carries its bus membership as it runs it (the controller's, from its store): composed
// the same, not 'in what runs, not composed' (seed #531, novox).
membership := json.RawMessage(`{"id":"bus-membership","type":"file","path":"` + catalogue.BusMembershipPath + `","sealed":"{\"box\":\"m\"}","mode":"0600"}`)
withMembership := make([]delivery.Composed, 0, len(running))
for _, c := range running {
if c.Module == "" {
c.Resources = append(append([]json.RawMessage{}, c.Resources...), membership)
}
withMembership = append(withMembership, c)
}
if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"spiegel"}}, fps, withMembership, "p-h1", 3); a.Verdict != delivery.CandidateSame {
t.Errorf("the node's own with its bus membership is answered %+v, want the same", a)
}
// Without what runs, the fold composes spiegel from its manifest, as before, and the composition refuses it.
if _, _, held, err := Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}}, fps); err != nil ||
!strings.Contains(held["spiegel"], "spiegel has not been built for this mesh") {
t.Errorf("composed spiegel without what runs: %v %v, want spiegel held alone", err, held)
}
// (C) Issue 1126, C5: what the others read of a module carried as it runs is what runs, never its stub's: the filter
// opens the machine port its running container publishes (de-spiegel's 20005 on novox, not its software's 8080; a
// stub had no container, so snake's 80 on ace was opened beside the route proxy's). And C6: the operator export's file
// a module that keeps it runs is carried as it runs (mesh-vault.kept on novox), since the node holds no export.
listening := []byte(`{"module":"spiegel","version":"1","capabilities":["container-runtime"],` +
`"listens":[{"port":8080,"from":"mesh","why":"the site"}],"own-secrets":{"token":"${dir:state}/token"},` +
`"resources":[{"id":"state","type":"directory","path":"/var/lib/spiegel","mode":"0700"},` +
`{"id":"server","type":"container","name":"spiegel","artifact":"site","ports":["8080"]}],` +
`"build":{"artifacts":[{"name":"site","kind":"upstream","from":"` + image + `"}]}}`)
nft := []byte(`{"module":"nft","version":"1","capabilities":["systemd"],"filtering":{"into":"/etc/nftables.conf"}}`)
vault := []byte(`{"module":"vault","version":"1","capabilities":["systemd"],"keeps":"/var/lib/vault/root",` +
`"resources":[{"id":"service","type":"service","unit":"vault.service","state":"running","boot":"enabled"}]}`)
b = mk("lemurs", overlay.Name, "spiegel", "nft", "vault")
for m, raw := range map[string][]byte{"spiegel": listening, "nft": nft, "vault": vault} {
b.manifests[delivery.ManifestDigest(raw)] = raw
b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
}
p = b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820"
for i := range p.Modules {
if p.Modules[i].Module == "spiegel" {
p.Modules[i].Ports = map[string]int{"8080": 20005}
}
}
b.placements["n1"] = p
rec := b.records["n1"]
rec.OutwardLinks = []string{"eth0"}
b.records["n1"] = rec
fps = map[string]string{"lemurs": fp("e"), "nft": fp("f"), "vault": fp("9")}
if in, err = Input(ctx, b, Mesh{}, stub); err != nil {
t.Fatal(err)
}
in.Nodes[0].Kept = map[string]catalogue.Manifest{"spiegel": compose.Stub(in.Catalogue["spiegel"])}
if base, _, err = compose.Compositions(in, "n1", fps, compose.Reading); err != nil {
t.Fatal(err)
}
keptFile := json.RawMessage(`{"digest":"sha256:` + strings.Repeat("d", 64) + `","id":"vault.kept","mode":"0600","module":"vault","path":"/var/lib/vault/root/export.json","size":143604,"type":"file"}`)
running = nil
for _, c := range base {
switch c.Module {
case "":
c.Resources = append(append([]json.RawMessage{}, c.Resources...),
json.RawMessage(`{"id":"spiegel.state","type":"directory","path":"/var/lib/spiegel","mode":"0700"}`),
json.RawMessage(`{"id":"spiegel.server","type":"container","name":"spiegel","image":"registry.example/spiegel@sha256:`+strings.Repeat("c", 64)+`","ports":["10.77.0.1:20005:8080"]}`),
json.RawMessage(`{"id":"spiegel.server-env","type":"file","path":"/var/lib/spiegel/server.env","mode":"0600","content":"A=1\n"}`))
case "vault":
c.Resources = append(append([]json.RawMessage{}, c.Resources...), keptFile)
}
running = append(running, c)
}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}, Running: running}, fps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatalf("the node does not compose itself with spiegel carried: %v", err)
}
var rules string
var keptComposed json.RawMessage
for _, c := range got {
for _, raw := range c.Resources {
var r struct {
ID string `json:"id"`
Content string `json:"content"`
}
_ = json.Unmarshal(raw, &r)
switch r.ID {
case "nft." + catalogue.FilteringID():
rules = r.Content
case "vault." + catalogue.KeptID():
keptComposed = raw
}
}
}
if !strings.Contains(rules, "# spiegel — the site") || !strings.Contains(rules, "dport 20005 accept") || strings.Contains(rules, "dport 8080") {
t.Errorf("the filter does not open the machine port spiegel runs at:\n%s", rules)
}
// The paths a carried module owns are its manifest's, as the controller composes them, never what runs of it (issue
// 1126, seed-8 #542: on ace, the folded f1-circuits, snake and txt-game, whose manifests leave their directories and
// files to ${dir:…}, owned their /var/lib paths on the node and none on the controller, shifting node-tools'
// MESH_OWNED_PATHS). f1-circuits' manifest at the indexer's commit (c0ea2c94) is captured; what runs of it is
// composed from that manifest by the controller's rule (the directory at /var/lib/f1-circuits, its file, its container).
for _, c := range running {
if c.Module == "" && !reflect.DeepEqual(compose.Carried(in.Catalogue["spiegel"], "spiegel", &c).OwnsPaths, []string{"/var/lib/spiegel"}) {
t.Errorf("spiegel carried owns %v, want its manifest's /var/lib/spiegel alone", compose.Carried(in.Catalogue["spiegel"], "spiegel", &c).OwnsPaths)
}
}
f1raw, err := os.ReadFile("testdata/f1-circuits-c0ea2c94.module.json")
if err != nil {
t.Fatal(err)
}
f1, err := catalogue.ParseManifest(f1raw)
if err != nil {
t.Fatal(err)
}
f1runs := delivery.Composed{Node: "ace", Resources: []json.RawMessage{
json.RawMessage(`{"id":"f1-circuits.state","module":"f1-circuits","type":"directory","path":"/var/lib/f1-circuits","mode":"0700"}`),
json.RawMessage(`{"id":"f1-circuits.cache","module":"f1-circuits","type":"directory","path":"/var/lib/f1-circuits/cache","mode":"0700"}`),
json.RawMessage(`{"id":"f1-circuits.site-env","module":"f1-circuits","type":"file","path":"/var/lib/f1-circuits/site.env","mode":"0644","content":"F1_CIRCUITS_BASE_URL=https://f1.example\n"}`),
json.RawMessage(`{"id":"f1-circuits.server","module":"f1-circuits","type":"container","name":"f1-circuits","image":"registry.example/f1@sha256:` + strings.Repeat("c", 64) + `"}`)}}
if got, want := catalogue.DeclaredPaths(compose.Carried(f1, "f1-circuits", &f1runs)), catalogue.DeclaredPaths(f1); !reflect.DeepEqual(got, want) {
t.Errorf("f1-circuits carried owns %v, want what the controller composes from its manifest, %v", got, want)
}
var k1, k2 any
_ = json.Unmarshal(keptComposed, &k1)
_ = json.Unmarshal(keptFile, &k2)
if !reflect.DeepEqual(k1, k2) {
t.Errorf("the operator export is composed as %s, want the file that runs %s", keptComposed, keptFile)
}
// (D) Issue 1126, seed-8 (mesh-catalog #542): a module's own refusal refuses that module, never the node. novox runs
// de-spiegel folded, and its manifest at the indexer's commit (e25e1beb, captured) asks four required config values
// for its route names that nothing sets. novox answered the whole candidate refused ("de-spiegel contributing
// praktijk to route says ${setting:praktijk-name} ...": its stub still contributed the route). Only de-spiegel is
// refused, in the config form's words, and proxy, which receives its routes (what it would compose lacks them: review
// of the apply, round 1); the rest is judged.
spiegel, err := os.ReadFile("testdata/de-spiegel-e25e1beb.module.json")
if err != nil {
t.Fatal(err)
}
proxy := []byte(`{"module":"proxy","version":"1","capabilities":["systemd"],"provides":[{"name":"route","scope":"mesh"},{"name":"smtp","scope":"mesh"}],` +
`"resources":[{"id":"service","type":"service","unit":"proxy.service","state":"running","boot":"enabled"}]}`)
b = mk("lemurs", overlay.Name, "proxy", "de-spiegel")
for m, raw := range map[string][]byte{"de-spiegel": spiegel, "proxy": proxy} {
b.manifests[delivery.ManifestDigest(raw)] = raw
b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
}
p = b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820"
b.placements["n1"] = p
fps = map[string]string{"lemurs": fp("e"), "proxy": fp("f")}
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given(b), InOwn: []string{"de-spiegel"}}, fps, nil, "p-h1", 3)
refused = nil
for _, r := range a.Refusals {
refused = append(refused, r.Module)
if r.Module == "de-spiegel" && (!strings.Contains(r.Why, "${config:praktijk-name}") || strings.Contains(r.Why, "${setting:")) {
t.Errorf("de-spiegel is refused in other words than its config form's: %s", r.Why)
}
}
if a.Refused != "" || !reflect.DeepEqual(refused, []string{"de-spiegel", "proxy"}) {
t.Errorf("a module's own refusal: %+v, want de-spiegel refused with proxy, which receives its routes, the node not", a)
}
if !slices.ContainsFunc(a.Added, func(m delivery.ModuleAdded) bool { return m.Module == "lemurs" }) {
t.Errorf("lemurs, which composes, is not judged beside de-spiegel's refusal: %+v", a)
}
// (E) Issue 1126, the applying path (review of the seed): the composition novox applies follows the same rule. It
// composed with no culprit, so de-spiegel's unset route names refused the whole node and every module it runs was
// marked not composed. Now de-spiegel is held, its refusal said in its config form's words, with proxy, which
// receives its routes; lemurs composes, and the node's own composition carries de-spiegel as it runs, so applying it
// removes nothing of it.
spiegelRuns := []json.RawMessage{
json.RawMessage(`{"id":"de-spiegel.state","type":"directory","path":"/var/lib/de-spiegel","mode":"0755"}`),
json.RawMessage(`{"id":"de-spiegel.server-env","type":"file","path":"/var/lib/de-spiegel/server.env","mode":"0600","content":"A=1\n"}`),
json.RawMessage(`{"id":"de-spiegel.server","type":"container","name":"de-spiegel","image":"registry.example/de-spiegel@sha256:` + strings.Repeat("c", 64) + `"}`)}
if in, err = Input(ctx, b, Mesh{}, stub); err != nil {
t.Fatal(err)
}
in.Nodes[0].Kept = map[string]catalogue.Manifest{"de-spiegel": compose.Stub(in.Catalogue["de-spiegel"])}
in.Nodes[0].Kept["de-spiegel"] = func(m catalogue.Manifest) catalogue.Manifest {
m.Contributes, m.ContributesMany = nil, nil
return m
}(in.Nodes[0].Kept["de-spiegel"])
if base, _, err = compose.Compositions(in, "n1", fps, compose.Reading); err != nil {
t.Fatal(err)
}
running = nil
for _, c := range base {
if c.Module == "" {
c.Resources = append(append([]json.RawMessage{}, c.Resources...), spiegelRuns...)
}
running = append(running, c)
}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"de-spiegel"}, Running: running}, fps)
if err != nil {
t.Fatalf("de-spiegel's own refusal refuses the node's whole composition: %v", err)
}
if len(held) != 2 || !strings.Contains(held["de-spiegel"], "${config:praktijk-name}") || !strings.Contains(held["proxy"], "de-spiegel") {
t.Errorf("held %v, want de-spiegel in its config form's words, and proxy for its routes", held)
}
lemursKey, _ := delivery.ComposedKey("n1", "lemurs", fps["lemurs"])
proxyKey, _ := delivery.ComposedKey("n1", "proxy", fps["proxy"])
if _, ok := got[lemursKey]; !ok {
t.Errorf("lemurs, which composes, is not composed beside de-spiegel's refusal: %v", sortedKeys(got))
}
if _, ok := got[proxyKey]; ok {
t.Errorf("proxy is composed without the routes of held de-spiegel: %v", sortedKeys(got))
}
carries := func(c delivery.Composed, what string) {
t.Helper()
ownRes, _ := json.Marshal(c.Resources)
for _, raw := range spiegelRuns {
var r struct{ ID string }
_ = json.Unmarshal(raw, &r)
if !strings.Contains(string(ownRes), `"id":"`+r.ID+`"`) {
t.Errorf("%s: the node's own composition drops %s, which de-spiegel runs: %s", what, r.ID, ownRes)
}
}
}
carries(got[ownKey], "folded")
// (F) Leaving the fold (review of the apply, round 1, finding 1): de-spiegel's want names a build of the new form,
// so it is no longer folded, while its names are still unset and the node's own still runs it. Held, it is carried
// in the node's own as it runs until a composition of its own is handed: else the node's own apply removes its
// container, files and units (issue 1945 by a new path).
fps["de-spiegel"] = fp("7")
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: running}, fps)
if err != nil || !strings.Contains(held["de-spiegel"], "${config:praktijk-name}") {
t.Fatalf("de-spiegel leaving the fold: %v, held %v", err, held)
}
spiegelKey, _ := delivery.ComposedKey("n1", "de-spiegel", fps["de-spiegel"])
if _, ok := got[spiegelKey]; ok {
t.Errorf("held de-spiegel's own composition is handed")
}
carries(got[ownKey], "leaving the fold")
// (G) A held module that runs a composition of its own (review of the apply, round 1, finding 2): what the others
// compose of it is what runs, never its stub's. site listens on 8080, placed at the machine's 20005, and its own
// secret was never issued here: held, the filter must not open 8080 and drop the 20005 that runs. A module whose
// composition depends on how site is held (here the filter) is held with it.
site := []byte(`{"module":"site","version":"1","capabilities":["container-runtime"],` +
`"listens":[{"port":8080,"from":"mesh","why":"the site"}],"own-secrets":{"key":"${dir:state}/key"},` +
`"resources":[{"id":"state","type":"directory","path":"/var/lib/site","mode":"0700"},` +
`{"id":"server","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["8080"]}]}`)
b = mk("lemurs", overlay.Name, "site", "nft")
for m, raw := range map[string][]byte{"site": site, "nft": nft} {
b.manifests[delivery.ManifestDigest(raw)] = raw
b.modules[m] = delivery.Module{Manifest: delivery.ManifestDigest(raw)}
}
p = b.placements["n1"]
p.Decided.Endpoint = "n1.example:51820"
for i := range p.Modules {
if p.Modules[i].Module == "site" {
p.Modules[i].Ports = map[string]int{"8080": 20005}
}
}
b.placements["n1"] = p
rec = b.records["n1"]
rec.OutwardLinks = []string{"eth0"}
b.records["n1"] = rec
fps = map[string]string{"lemurs": fp("e"), "nft": fp("f"), "site": fp("8")}
siteRuns := delivery.Composed{Node: "n1", Module: "site", FP: fps["site"], Resources: []json.RawMessage{
json.RawMessage(`{"id":"site.state","module":"site","type":"directory","path":"/var/lib/site","mode":"0700"}`),
json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["10.77.0.1:20005:8080"]}`)}}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps)
if err != nil || held["site"] == "" || held["nft"] == "" {
t.Fatalf("site, its own secret never issued: %v, held %v; want site held, and the filter, which depends on it", err, held)
}
for key, c := range got {
for _, raw := range c.Resources {
if strings.Contains(string(raw), "dport 8080") || (c.Module == "nft" && !strings.Contains(string(raw), "dport 20005") && strings.Contains(string(raw), "# site")) {
t.Errorf("%s composes held site from its stub, not from what runs: %s", key, raw)
}
}
}
// (H) Review of the apply, round 2: a held module is composed from its new manifest, so what the others read of its
// manifest (its listens, what it contributes) may not be what runs. site's new manifest moves it to 9090 while the
// build that runs listens on 8080: the filter, the holder of what site listens on, is held with it, and nothing
// applied opens 9090.
moved := []byte(strings.Replace(string(site), `"port":8080`, `"port":9090`, 1))
b.manifests[delivery.ManifestDigest(moved)] = moved
b.modules["site"] = delivery.Module{Manifest: delivery.ManifestDigest(moved)}
for i := range p.Modules {
p.Modules[i].Ports = nil
}
b.placements["n1"] = p
siteRuns.Resources[1] = json.RawMessage(`{"id":"site.server","module":"site","type":"container","name":"site","image":"registry.example/site@sha256:` + strings.Repeat("c", 64) + `","ports":["8080"]}`)
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []delivery.Composed{siteRuns}}, fps)
if err != nil || held["site"] == "" || held["nft"] == "" {
t.Errorf("site moved to 9090 in its new manifest: %v, held %v; want site held, and the filter with it", err, held)
}
for key, c := range got {
for _, raw := range c.Resources {
if strings.Contains(string(raw), "dport 9090") {
t.Errorf("%s opens 9090, which held site's new manifest names and nothing runs: %s", key, raw)
}
}
}
}
// A module another node cannot run in this composition offers nothing in it: the refusal names it and why
// (issue 1126, probe #532: ace and g14 answered "nothing in this mesh provides artifact-store … assign distribution to a
// node" while novox runs it). And a refusal no module's stub changes is answered at once, not after a compose per module
// (ace took 7m46s).
func otherNodeNotResolvedIsNamed(t *testing.T) {
ctx := context.Background()
prov := []byte(`{"module":"prov","version":"1","capabilities":["container-runtime"],"provides":[{"name":"thing","scope":"mesh"}]}`)
cons := []byte(`{"module":"cons","version":"1","requires":["thing"]}`)
// n2 runs prov, which needs a container runtime n2's record does not say it has: n2's set does not resolve here.
b := &bus{placements: map[string]delivery.Placement{
"n1": {Node: "n1", Decided: delivery.NodeDecided{Hub: true, Address: "10.77.0.1", Endpoint: "n1.example:51820"}, Modules: []delivery.PlacedModule{{Module: "cons"}, {Module: overlay.Name}}},
"n2": {Node: "n2", Decided: delivery.NodeDecided{Address: "10.77.0.2"}, Modules: []delivery.PlacedModule{{Module: "prov"}, {Module: overlay.Name}}}},
records: map[string]delivery.NodeRecord{"n1": {Node: "n1", TunnelKey: publicKey(t)}, "n2": {Node: "n2", TunnelKey: publicKey(t)}},
modules: map[string]delivery.Module{"prov": {Manifest: delivery.ManifestDigest(prov)}, "cons": {Manifest: delivery.ManifestDigest(cons)}},
manifests: map[string][]byte{delivery.ManifestDigest(prov): prov, delivery.ManifestDigest(cons): cons}}
given := map[string]*delivery.Placement{}
for n, p := range b.placements {
p := p
given[n] = &p
}
a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Placements: given}, map[string]string{"cons": "src2:" + strings.Repeat("1", 64)}, nil, "p-h1", 1)
if a.Verdict != delivery.CandidateRefused || !strings.Contains(a.Refused, "prov on n2 lacks container-runtime") {
t.Errorf("a refusal for want of n2's provider = %+v", a)
}
}
// A node that reaches no artifact store says what its composition holds of one (issue 1126: ace and g14 answered
// "reaches no artifact store", and nothing said why).
func noStoreReachedSaysWhy(t *testing.T) {
in := compose.Input{Catalogue: map[string]catalogue.Manifest{
"distribution": {Module: "distribution", Serves: map[string]map[string]any{catalogue.ArtifactStoreProvision: {"port": 5000}}}},
Nodes: []compose.Node{{Name: "g14", Assigned: []string{"adwaita"}}, {Name: "novox", Assigned: []string{"distribution"}}}}
if got := storeSought(in); got != "the composition holds g14, novox; artifact-store is served by distribution on novox, none of it on the private network with a port" {
t.Errorf("said %q", got)
}
in.Nodes = in.Nodes[:1]
if got := storeSought(in); got != "the composition holds g14; no module placed there serves artifact-store" {
t.Errorf("said %q", got)
}
}
+6 -5
View File
@@ -10,9 +10,10 @@ import (
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Export is one node's placement file as the composition input the controller fills says it (novox/hq ADR 0555 §6,
// issue 1126): the inverse of Input for what a person decides, so the controller's one read-only verb
// `placement-export` answers, from its own store by its own code, exactly what the seed writes into the catalogue.
// Export is one node's placement as the composition input the controller fills says it (novox/hq ADR 0555 §6, ADR 0567
// §7): the inverse of Input for what a person decides, so the controller's one read-only verb `placement-export`
// answers, from its own store by its own code, exactly what the seat's adopt proposes as the seed. Its answer must stay
// stable for the same store, since adopt checks its digest again at the warrant.
// What the machine observes (its record) is not in it.
func Export(in compose.Input, node string) (delivery.Placement, error) {
var n *compose.Node
@@ -24,8 +25,8 @@ func Export(in compose.Input, node string) (delivery.Placement, error) {
if n == nil {
return delivery.Placement{}, fmt.Errorf("frombus: the input has no node %q", node)
}
// At the version the placement file is read at, with a list of modules even when none is placed: the export is the
// file the seed writes (cmd/placement-seed), and the reader refuses a value of version 0 or without its modules.
// At the version a placement key is read at, with a list of modules even when none is placed: the export is the
// seed's value, and the reader refuses a value of version 0 or without its modules.
p := delivery.Placement{V: delivery.Version, Node: node, Modules: []delivery.PlacedModule{}, Decided: delivery.NodeDecided{
Endpoint: n.Place.Endpoint, Site: n.Place.Site, Hub: n.Place.Hub, Address: n.Place.Address,
PublicDomain: n.PublicDomain, Account: n.Account, AccountHome: n.AccountHome, AgentAccount: n.AgentAccount,
+169 -28
View File
@@ -2,13 +2,13 @@
// and the grants part compose with one composer over the same inputs (novox/hq ADR 0547 §2, ADR 0549 §3; issue 1126).
//
// **One composer, three callers.** compose.Compositions is the composer; Input is how each caller fills its
// compose.Input from the bus: the node agent for its own node (ADR 0547 §2), the assigner's composition check for every
// node (ADR 0539 §6's check, ADR 0547 §4), and the grants part (ADR 0547 §5). Each reads the bus with its own credential
// compose.Input from the bus: the node agent for its own node (ADR 0547 §2), the seat mesh-placement's holder judging a
// proposal for every node (ADR 0567 §4), and the grants part (ADR 0547 §5). Each reads the bus with its own credential
// through a Reader, so this package holds no bus client, no store and no timer: what it is given is all it reads.
//
// **What it reads, and from whom.**
// - which modules run on each node, with their pins, holdings, host ports, bindings and settings placement: the
// indexer's state `placement`, one key per node (ADR 0549 §2);
// seat mesh-placement's state `placement`, one key per node (ADR 0567 §1);
// - what a person decided of each node (its endpoint, site, hub, address on the private network, public domain and
// accounts): the same placement key, Placement.Decided;
// - what each node observes of itself (its tunnel key, capabilities, keys, certificate, links and networks): the
@@ -25,8 +25,8 @@
// **Own secrets and pair credentials** come from the vault sealed to the composing node's key (OwnSecrets, Credentials),
// for that node alone: as a consumer and as a provider, exactly the keys compose.CredentialsNeeded names. **Not yet
// read** (each a later subtask of issue 1126): licences (none is placed today) and core boxes. Without one
// the composition stands in for it, and Compose refuses a composition that stands in for anything (compose.ErrStoodIn):
// a node never applies a stand-in. No node is joining before ADR 0547's third milestone (ADR 0550 §6), so Joining is
// the composition stands in for it, and Compose holds the module it stands in for (or refuses the node, for a stand-in
// of no module placed there; compose.ErrStoodIn): a node never applies a stand-in. No node is joining before ADR 0547's third milestone (ADR 0550 §6), so Joining is
// always empty.
package frombus
@@ -107,18 +107,27 @@ type Mesh struct {
// build of at the trunk's FP2 composes from the indexer's manifest, as before. Every leaves the catalogue as the indexer
// read it: the check of every node judges placement, and reads no archive.
//
// Placements, when set, stands in for the indexer's placement of the nodes it names: a node's candidate placement
// (delivery.PlacementCandidate), so a node composes in shadow with a catalogue pull request's placement and compares
// the result with what it runs (issue 1126); a nil value removes the node's placement. The rest is read as always.
// Placements, when set, stands in for the placement of the nodes it names: a proposal's candidates
// (delivery.PlacementCandidate), so a node composes in shadow with a proposal laid over and compares the result with
// what it runs (ADR 0567 §3); a nil value removes the node's placement. The rest is read as always.
//
// InOwn is the modules the composing node runs inside its own composition (InOwn of what it runs), composed there again
// (compose.Node.InOwn) rather than cut out as modules no build is given for (issue 1126).
//
// Running is what the composing node runs; of it, its own composition (`<node>.node`) is given to the composition
// (compose.Node.RunsOwn), so a module folded there that names a build of the old form is left to the node, carried
// as it runs (issue 1126). Nil composes every folded module from its manifest.
type Scope struct {
Node string
Every bool
KeptAt map[string]string
Placements map[string]*delivery.Placement
InOwn []string
Running []delivery.Composed
// SameValue, in an answer, says whether two sealed values open to the same value (the node opens them with its own
// key), so a differing resource shows a value sealed again as such (delivery.CompareComposedWith). Nil shows them
// as differing.
SameValue func(a, b string) (same, known bool)
}
// InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs
@@ -154,6 +163,13 @@ func InOwn(running []delivery.Composed, p delivery.Placement) []string {
// Input is the composition input for s, read through r.
func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, error) {
return input(ctx, r, mesh, s, nil)
}
// input is Input; with refuse set, a module placed on the composing node whose build cannot be read or taken in is
// handed to refuse and composed as a stub (compose.Stub, kept for that node alone) rather than ending the read, so a
// node's answer names every module it cannot compose (issue 1126).
func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module string, err error)) (compose.Input, error) {
placements, err := r.Placements(ctx)
if err != nil {
return compose.Input{}, fmt.Errorf("frombus: placement: %w", err)
@@ -280,6 +296,37 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
}
n.InOwn[module] = true
}
for i := range s.Running {
// The operator export as the vault runs it: the controller composes it from its store, which the node
// does not read; until the export is on the bus, the node carries the file it runs (issue 1126, C6:
// mesh-vault.kept on novox, "in what runs, and not composed").
if m := s.Running[i].Module; m != "" {
if kept := keptOf(s.Running[i]); kept != nil {
if in.KeptRuns == nil {
in.KeptRuns = map[string]map[string]map[string]any{}
}
if in.KeptRuns[s.Node] == nil {
in.KeptRuns[s.Node] = map[string]map[string]any{}
}
in.KeptRuns[s.Node][m] = kept
}
continue
}
if s.Running[i].Module == "" {
own := s.Running[i]
n.RunsOwn = &own
// Its bus membership as it runs it: the controller composes it from its store, which the node does
// not read; until the grants part hands memberships sealed to the node, the node's own composition
// carries the one it runs (issue 1126, seed #531 on novox: "bus-membership in what runs, not
// composed", which applied would have taken the node's bus credential away).
if sealed := membershipOf(own); sealed != "" {
if in.BusMemberships == nil {
in.BusMemberships = map[string]string{}
}
in.BusMemberships[s.Node] = sealed
}
}
}
}
in.Nodes = append(in.Nodes, n)
in.Holdings = append(in.Holdings, p.Held()...)
@@ -309,7 +356,7 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
if _, ok := records[s.Node]; !ok {
return compose.Input{}, fmt.Errorf("frombus: %s has written no record of itself yet", s.Node)
}
if err := fromBuilds(ctx, r, &in, s, placements[s.Node], modules, read, parse); err != nil {
if err := fromBuilds(ctx, r, &in, s, placements[s.Node], modules, read, parse, refuse); err != nil {
return compose.Input{}, err
}
// Its own modules' secrets, sealed to its own key by the vault: a composition carries them sealed (ADR 0550 §5).
@@ -332,25 +379,31 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
return in, nil
}
// Compose is the node's compositions over the bus (ADR 0547 §2): Input for s, then compose.CompositionsApplied at the
// fingerprints fps gives each module: composed in the Reading mode as the controller composes them, and refused when it
// stands in for anything the input does not carry (compose.ErrStoodIn), since the node applies what it composed.
func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (map[string]delivery.Composed, []string, error) {
in, err := Input(ctx, r, mesh, s)
// Compose is the node's compositions over the bus (ADR 0547 §2): input for s, then composed at the fingerprints fps
// gives each module in the Reading mode, as the controller composes them, by the one rule its answer to a candidate
// follows too (composeEach, issue 1126): a module refused alone (its build unread or not taken in, its composition
// refused, a stand-in for a value of its own the input does not carry, compose.ErrStoodIn) is held, named in held with
// why, and none of its own composition is in out, so the node keeps what it last applied of it and applies the rest; a
// node never applies a stand-in. Only a refusal that is no single module's (the node's own resources, a value served
// elsewhere, a stand-in for the node's own) refuses the node, err.
func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (out map[string]delivery.Composed, skipped []string, held map[string]string, err error) {
held = map[string]string{}
in, err := input(ctx, r, mesh, s, func(module string, err error) { held[module] = err.Error() })
if err != nil {
return nil, nil, err
return nil, nil, held, err
}
// The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5).
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil {
return nil, nil, err
return nil, nil, held, err
}
if len(asConsumer)+len(asProvider) > 0 {
if in.Credentials, in.Issued, err = r.Credentials(ctx, s.Node, asConsumer, asProvider); err != nil {
return nil, nil, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)
return nil, nil, held, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)
}
}
return compose.CompositionsApplied(in, s.Node, fps)
out, skipped, _, err = composeEach(in, s.Node, fps, true, s.Running, held)
return out, skipped, held, err
}
// fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each
@@ -358,7 +411,29 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
// by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it
// (issue 1126).
func fromBuilds(ctx context.Context, r Reader, in *compose.Input, s Scope, p delivery.Placement, modules map[string]delivery.Module,
read func(module, digest string) (catalogue.Manifest, error), parse func([]byte) (catalogue.Manifest, error)) error {
read func(module, digest string) (catalogue.Manifest, error), parse func([]byte) (catalogue.Manifest, error),
refuse func(module string, err error)) error {
// stub keeps a refused module in the composition as a stub for this node alone (compose.Stub): what it provides
// and requires still shapes the rest.
stub := func(module string, err error) {
refuse(module, err)
for i := range in.Nodes {
if in.Nodes[i].Name != s.Node {
continue
}
man, ok := in.Nodes[i].Kept[module]
if !ok {
man, ok = in.Catalogue[module]
}
if !ok {
return
}
if in.Nodes[i].Kept == nil {
in.Nodes[i].Kept = map[string]catalogue.Manifest{}
}
in.Nodes[i].Kept[module] = compose.Stub(man)
}
}
address, addressErr := "", error(nil)
addressed := false
archive := func(ctx context.Context, reference, digest string) ([]byte, error) {
@@ -370,7 +445,7 @@ func fromBuilds(ctx context.Context, r Reader, in *compose.Input, s Scope, p del
case addressErr != nil:
return nil, addressErr
case address == "":
return nil, fmt.Errorf("%s reaches no artifact store", s.Node)
return nil, fmt.Errorf("%s reaches no artifact store (%s)", s.Node, storeSought(*in))
}
return r.Archive(ctx, catalogue.Routed(reference, address), digest)
}
@@ -409,14 +484,23 @@ func fromBuilds(ctx context.Context, r Reader, in *compose.Input, s Scope, p del
}
b, err := r.Build(ctx, pm.Module, m.FP2)
if err != nil {
return fmt.Errorf("frombus: the build of %s at %s: %w", pm.Module, m.FP2, err)
err = fmt.Errorf("frombus: the build of %s at %s: %w", pm.Module, m.FP2, err)
if refuse == nil {
return err
}
stub(pm.Module, err)
continue
}
if b == nil || b.State != delivery.BuildDone {
continue // no done build at the trunk's: composed from the indexer's manifest, as before
}
resolved, err := built(pm.Module, m.FP2, b)
if err != nil {
return err
if refuse == nil {
return err
}
stub(pm.Module, err)
continue
}
in.Catalogue[pm.Module] = resolved
}
@@ -427,15 +511,22 @@ func fromBuilds(ctx context.Context, r Reader, in *compose.Input, s Scope, p del
for _, module := range sortedKeys(s.KeptAt) {
fp := s.KeptAt[module]
b, err := r.Build(ctx, module, fp)
if err != nil {
return fmt.Errorf("frombus: the build of %s at %s: %w", module, fp, err)
switch {
case err != nil:
err = fmt.Errorf("frombus: the build of %s at %s: %w", module, fp, err)
case b == nil || b.State != delivery.BuildDone:
err = fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build", module, fp)
}
if b == nil || b.State != delivery.BuildDone {
return fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build", module, fp)
var resolved catalogue.Manifest
if err == nil {
resolved, err = built(module, fp, b)
}
resolved, err := built(module, fp, b)
if err != nil {
return err
if refuse == nil {
return err
}
stub(module, err)
continue
}
if in.Nodes[i].Kept == nil {
in.Nodes[i].Kept = map[string]catalogue.Manifest{}
@@ -613,3 +704,53 @@ func sortedKeys[V any](m map[string]V) []string {
sort.Strings(out)
return out
}
// storeSought says, for a node that reaches no artifact store, what the composition holds of one: the nodes it holds and
// where each module serving the artifact store is placed, so the refusal names why (issue 1126).
func storeSought(in compose.Input) string {
var nodes, where []string
for _, n := range in.Nodes {
nodes = append(nodes, n.Name)
for _, a := range n.Assigned {
if m, ok := in.Catalogue[a]; ok {
if _, serves := m.Serves[catalogue.ArtifactStoreProvision]; serves {
where = append(where, a+" on "+n.Name)
}
}
}
}
sort.Strings(nodes)
sort.Strings(where)
said := "the composition holds " + strings.Join(nodes, ", ")
if len(where) == 0 {
return said + "; no module placed there serves " + catalogue.ArtifactStoreProvision
}
return said + "; " + catalogue.ArtifactStoreProvision + " is served by " + strings.Join(where, ", ") +
", none of it on the private network with a port"
}
// keptOf is the operator export's file (catalogue.KeptID) of a module's composition as it runs, or nil.
func keptOf(c delivery.Composed) map[string]any {
for _, raw := range c.Resources {
var r map[string]any
if json.Unmarshal(raw, &r) == nil && r["id"] == c.Module+"."+catalogue.KeptID() {
return r
}
}
return nil
}
// membershipOf is the sealed value of the bus membership resource (catalogue.BusMembershipID) of a node's own
// composition, or "".
func membershipOf(own delivery.Composed) string {
for _, raw := range own.Resources {
var r struct {
ID string `json:"id"`
Sealed string `json:"sealed"`
}
if json.Unmarshal(raw, &r) == nil && r.ID == catalogue.BusMembershipID() {
return r.Sealed
}
}
return ""
}
+47 -15
View File
@@ -119,6 +119,11 @@ func publicKey(t *testing.T) string {
// controller's store fills for compose's own two-machine mesh (compose's aMesh: anchor the hub, laptop), and that Input
// composes; a node reads only its own config values; what cannot be composed is refused, naming it.
func TestInput(t *testing.T) {
// Issue 1126, the seed's shadow: every refusal named, an old-form module left to the node, and no store reached
// saying why (answer_rows_test.go).
t.Run("every refusal, and an old build left to the node", answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode)
t.Run("no store reached says why", noStoreReachedSaysWhy)
t.Run("another node not resolved is named", otherNodeNotResolvedIsNamed)
ctx := context.Background()
xterm, err := os.ReadFile("../../catalogue/testdata/catalogue-835bf22b-xterm.module.json")
if err != nil {
@@ -218,7 +223,10 @@ func TestInput(t *testing.T) {
// The same composer over both: the compositions from the bus are the store's, byte for byte.
fps := map[string]string{overlay.Name: "src2:" + strings.Repeat("a", 64), "xterm": "src2:" + strings.Repeat("b", 64),
"media": "src2:" + strings.Repeat("c", 64)}
fromBus, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop"}, fps)
fromBus, _, held, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop"}, fps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatal(err)
}
@@ -238,7 +246,10 @@ func TestInput(t *testing.T) {
running = append(running, c)
}
laptop := mk().placements["laptop"]
asIs, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop}}, fps)
asIs, _, held, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop}}, fps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatal(err)
}
@@ -247,7 +258,10 @@ func TestInput(t *testing.T) {
}
moved := mk().placements["anchor"]
moved.Decided.Endpoint = "anchor.example:51821"
changed, _, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop, "anchor": &moved}}, fps)
changed, _, held, err := Compose(ctx, mk(), Mesh{}, Scope{Node: "laptop", Placements: map[string]*delivery.Placement{"laptop": &laptop, "anchor": &moved}}, fps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatal(err)
}
@@ -284,7 +298,10 @@ func TestInput(t *testing.T) {
}
oneFps := map[string]string{"keeper": "src2:" + strings.Repeat("d", 64), "lemurs": "src2:" + strings.Repeat("e", 64),
"objects": "src2:" + strings.Repeat("f", 64), "files": "src2:" + strings.Repeat("0", 64)}
runs, _, err := Compose(ctx, one("keeper", "lemurs"), Mesh{}, Scope{Node: "n1"}, oneFps)
runs, _, held, err := Compose(ctx, one("keeper", "lemurs"), Mesh{}, Scope{Node: "n1"}, oneFps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatal(err)
}
@@ -292,7 +309,10 @@ func TestInput(t *testing.T) {
for _, c := range runs {
runsNow = append(runsNow, c)
}
objectsRun, _, err := Compose(ctx, one("objects"), Mesh{}, Scope{Node: "n1"}, oneFps)
objectsRun, _, held, err := Compose(ctx, one("objects"), Mesh{}, Scope{Node: "n1"}, oneFps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatal(err)
}
@@ -381,7 +401,9 @@ func TestInput(t *testing.T) {
check func(delivery.CandidateAnswer) bool
}{
"the same": {one("keeper", "lemurs"), placedAs("keeper", "lemurs"), runsNow, delivery.CandidateSame,
func(a delivery.CandidateAnswer) bool { return len(a.Same) == 3 && a.Head == "h1" && a.Revision == 7 }},
func(a delivery.CandidateAnswer) bool {
return len(a.Same) == 3 && a.Proposal == "p-h1" && a.Revision == 7
}},
"the node runs no own composition": {one("keeper", "lemurs"), placedAs("keeper", "lemurs"), withoutOwn(runsNow), delivery.CandidateSame,
func(a delivery.CandidateAnswer) bool { return len(a.Same) == 2 && len(a.Added) == 0 }},
"one removed": {one("keeper", "lemurs"), placedAs("keeper"), runsNow, delivery.CandidateDiffers,
@@ -422,7 +444,7 @@ func TestInput(t *testing.T) {
if name == "mesh-wireguard and an old-form module, folded" {
fps = map[string]string{"keeper": oneFps["keeper"]} // the node's wants: no build of lemurs or the private network
}
a := Answer(ctx, c.b, Mesh{}, Scope{Node: "n1", Placements: c.given, InOwn: InOwn(c.running, *c.given["n1"])}, fps, c.running, "h1", 7)
a := Answer(ctx, c.b, Mesh{}, Scope{Node: "n1", Placements: c.given, InOwn: InOwn(c.running, *c.given["n1"])}, fps, c.running, "p-h1", 7)
if a.Verdict != c.verdict || !c.check(a) {
t.Errorf("%s: %+v", name, a)
}
@@ -574,11 +596,21 @@ func TestInput(t *testing.T) {
}
return b
}
if _, _, err := Compose(ctx, movedMesh(false), Mesh{}, Scope{Node: "n1"}, movedFps); err == nil || !strings.Contains(err.Error(), `restic needs a secret called "repository"`) {
t.Errorf("restic composed from the indexer's manifest: %v, want its own secret refused", err)
if out, _, held, err := Compose(ctx, movedMesh(false), Mesh{}, Scope{Node: "n1"}, movedFps); err != nil || !strings.Contains(held["restic"], `restic needs a secret called "repository"`) {
t.Errorf("restic composed from the indexer's manifest: %v %v, want its own secret refusing restic alone", err, held)
} else {
for key, c := range out {
if c.Module == "restic" {
// Its stub's composition, applied, would remove what restic runs: a held module keeps its last one.
t.Errorf("held restic is composed as %s, want none of its own composition handed", key)
}
}
}
builtBus := movedMesh(true)
fromBuild, _, err := Compose(ctx, builtBus, Mesh{}, Scope{Node: "n1"}, movedFps)
fromBuild, _, held, err := Compose(ctx, builtBus, Mesh{}, Scope{Node: "n1"}, movedFps)
if err == nil && len(held) > 0 {
err = fmt.Errorf("modules held: %v", held)
}
if err != nil {
t.Fatalf("restic composed from its build: %v", err)
}
@@ -592,8 +624,8 @@ func TestInput(t *testing.T) {
// A done key an artifact of which names no reference is waited for, in the controller's words (ADR 0498 §8).
unreferenced := movedMesh(true)
unreferenced.builds["restic "+movedFps["restic"]].Artifacts = []delivery.BuiltArtifact{{Name: "tools", Kind: "bundle", Digest: digestOf("6")}, resticArtifacts[1]}
if _, _, err := Compose(ctx, unreferenced, Mesh{}, Scope{Node: "n1"}, movedFps); err == nil || !strings.Contains(err.Error(), "the build of restic is done and waits for the reference of tools") {
t.Errorf("a done key with an unreferenced artifact: %v", err)
if _, _, held, err := Compose(ctx, unreferenced, Mesh{}, Scope{Node: "n1"}, movedFps); err != nil || !strings.Contains(held["restic"], "the build of restic is done and waits for the reference of tools") {
t.Errorf("a done key with an unreferenced artifact: %v %v, want restic held alone", err, held)
}
// A module the node runs inside its own composition (the fold) composes from the indexer's manifest though a done
// build of it is at the trunk's FP2: its build is not read.
@@ -657,12 +689,12 @@ func TestInput(t *testing.T) {
}
return b
}
if _, _, err := Compose(ctx, withMailer(false), Mesh{}, Scope{Node: "laptop"}, fps); err == nil || !strings.Contains(err.Error(), `nothing sets "domain"`) {
if _, _, _, err := Compose(ctx, withMailer(false), Mesh{}, Scope{Node: "laptop"}, fps); err == nil || !strings.Contains(err.Error(), `nothing sets "domain"`) {
t.Errorf("a provider's served setting nobody sets: %v", err)
}
served := withMailer(true)
if _, _, err := Compose(ctx, served, Mesh{}, Scope{Node: "laptop"}, fps); err != nil {
t.Errorf("laptop with what anchor's mailer serves: %v", err)
if _, _, held, err := Compose(ctx, served, Mesh{}, Scope{Node: "laptop"}, fps); err != nil || len(held) > 0 {
t.Errorf("laptop with what anchor's mailer serves: %v %v", err, held)
}
if !reflect.DeepEqual(served.servedRead, []string{"anchor.mailer:domain"}) || slices.Contains(served.valuesRead, "anchor.mailer") {
t.Errorf("laptop read %v of other nodes (and %v of config values): the served keys of anchor's mailer alone", served.servedRead, served.valuesRead)
@@ -0,0 +1,129 @@
{
"module": "de-spiegel",
"version": "1",
"slug": "spiegel",
"capabilities": [
"container-runtime"
],
"requires": [
"route",
"smtp"
],
"contributes": {
"route": {
"site": {
"label": "de-spiegel",
"port": 35621
},
"praktijk": {
"name": "${config:praktijk-name}",
"port": 35621
},
"praktijk-www": {
"name": "${config:praktijk-www-name}",
"redirect": "https://${config:praktijk-name}"
},
"psychologie": {
"name": "${config:psychologie-name}",
"port": 35621
},
"psychologie-www": {
"name": "${config:psychologie-www-name}",
"redirect": "https://${config:psychologie-name}"
}
},
"smtp": {
"account": "de-spiegel"
}
},
"binds": {
"route": "/var/lib/de-spiegel/route.json",
"smtp": "/var/lib/de-spiegel/smtp.json"
},
"config": {
"praktijk-name": {
"type": "string",
"description": "The public name the practice's site is served on; the operator's, with no default (novox/hq ADR 0112).",
"trusted": true,
"required": true
},
"praktijk-www-name": {
"type": "string",
"description": "The public name that redirects to the practice's site (its www form); the operator's, with no default (novox/hq ADR 0112).",
"trusted": true,
"required": true
},
"psychologie-name": {
"type": "string",
"description": "The public name the psychology practice's site is served on; the operator's, with no default (novox/hq ADR 0112).",
"trusted": true,
"required": true
},
"psychologie-www-name": {
"type": "string",
"description": "The public name that redirects to the psychology practice's site (its www form); the operator's, with no default (novox/hq ADR 0112).",
"trusted": true,
"required": true
}
},
"listens": [
{
"port": 35621,
"protocol": "tcp",
"from": "mesh",
"why": "the de-spiegel site and its /contact endpoint over http; the practice's three public names are route grants, and the proxy reaches every one of them here"
}
],
"resources": [
{
"id": "state",
"type": "directory",
"path": "/var/lib/de-spiegel",
"mode": "0700"
},
{
"id": "server-env",
"type": "file",
"path": "/var/lib/de-spiegel/server.env",
"mode": "0600",
"content": "SMTP_HOST=${bound:smtp:at}\nSMTP_PORT=${bound:smtp:port}\nSMTP_AUTH_USER=de-spiegel@${bound:smtp:domain}\nSMTP_AUTH_PASS=${secret:smtp}\n"
},
{
"id": "net",
"type": "network",
"name": "de-spiegel"
},
{
"id": "server",
"type": "container",
"name": "de-spiegel",
"artifact": "site",
"network": "de-spiegel",
"env-file": [
"/var/lib/de-spiegel/server.env"
],
"ports": [
"35621"
],
"secrets-in-environment": "the application's own code reads SMTP_AUTH_USER/PASS from the environment (server/index.js); converting is this repository's change"
}
],
"build": {
"on": [
{
"arg": "NODE_BASE",
"image": "node@sha256:48e4b67d85f87bd551df43704e24d252f56cc5f8e9718841aace50f19948f0f9"
}
],
"artifacts": [
{
"name": "site",
"kind": "image",
"from": "Dockerfile"
}
]
},
"secrets": {
"smtp": "/var/lib/de-spiegel/smtp.secret"
}
}
@@ -0,0 +1,82 @@
{
"module": "f1-circuits",
"slug": "f1",
"version": "1",
"capabilities": [
"container-runtime"
],
"requires": [
"route"
],
"contributes": {
"route": {
"label": "f1-circuits",
"endpoint": "web"
}
},
"binds": {
"route": "${dir:state}/route.json"
},
"listens": [
{
"name": "web",
"port": 8000,
"protocol": "tcp",
"from": "mesh",
"why": "the track-layout CDN, the race posters and the Kometa metadata over http; its public name is a route grant and the proxy reaches it here. Kometa fetches the metadata and every poster by that public name"
}
],
"resources": [
{
"id": "state",
"type": "directory",
"mode": "0700",
"place": "."
},
{
"id": "cache",
"type": "directory",
"mode": "0700"
},
{
"id": "site-env",
"type": "file",
"path": "${dir:state}/site.env",
"mode": "0644",
"content": "F1_CIRCUITS_BASE_URL=https://${bound:route:name}\n"
},
{
"id": "server",
"type": "container",
"name": "f1-circuits",
"artifact": "cdn",
"ports": [
"8000"
],
"env-file": [
"${dir:state}/site.env"
],
"volumes": [
"${dir:cache}:/app/cache"
],
"restart-on": [
"site-env"
]
}
],
"build": {
"on": [
{
"arg": "PYTHON_BASE",
"image": "python@sha256:6d43704baacd1bfbe7c295d7f13079d5d8104ed33568873133f8fc69980419df"
}
],
"artifacts": [
{
"name": "cdn",
"kind": "image",
"from": "service/Dockerfile"
}
]
}
}
+16
View File
@@ -23,6 +23,7 @@ import (
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Input is everything the composition reads. Each field is what one read of the controller's inventory answers, named
@@ -67,6 +68,10 @@ type Input struct {
// OperatorExport is every operator-sealed secret in the mesh, for a module that keeps them (inv.OperatorExport): sealed
// values the composition carries without opening. Nil when the mesh has no operator key.
OperatorExport *catalogue.KeptExport
// KeptRuns is, by node and module, the file of the export as that node runs it (its resource of a module's running
// composition): what a node composing its own carries where it holds no OperatorExport (catalogue.Rendering.KeptRuns,
// issue 1126, C6). Used only while OperatorExport is nil.
KeptRuns map[string]map[string]map[string]any
// Issued is the credentials made for the consumers of each provider (inv.SecretsFrom): one row for each pair credential
// that exists, by the provider node it is for. The grants a provider must create are read from these, so a credential not
@@ -194,6 +199,17 @@ type Node struct {
// resources into the node's own, as the controller's composed writer does, unless fps gives one of the new form.
// The private network's module (overlay.Name), which nobody builds, is folded whether or not it is named here.
InOwn map[string]bool
// RunsOwn is the node's own composition (`<node>.node`) as it runs, when the node composes itself: a module folded
// there (InOwn) whose manifest names a build (NamesBuild) runs a build of the old form only the controller holds,
// so it is left to the node: Compositions composes it as a stub (Stub) and carries its resources as they run, each
// after the resource it runs after (issue 1126). Nil composes every folded module from its manifest, as before.
RunsOwn *delivery.Composed
// Held is the modules refused alone on this node, each composed as what runs of it (Carried, in Kept; its Stub when
// nothing of it runs) so the rest is composed (novox/hq issue 1126): none of a held module's own composition is in what
// Compositions answers, so the node keeps what it last applied of it; one whose resources the node's own runs
// (RunsOwn), folded or leaving the fold, is carried there as it runs, whatever its manifest names, so the node's own
// never drops it.
Held map[string]bool
}
// Place is where a node is, as its placement says (inv.Overlay).
+243
View File
@@ -0,0 +1,243 @@
// Package judge is how the seat mesh-placement's holder judges a proposal (novox/hq ADR 0567 §3, §4): which nodes must
// answer it and what it changes on each (Intended), and the verdict of their answers (JudgeAnswers). Moved here from
// the assigner, where it judged a catalogue pull request's placement (ADR 0559 §5), with a proposal in place of a pull
// request's change. It holds no bus client and no timer: the holder gives it what it read.
package judge
import (
"fmt"
"reflect"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// AnswerBound is how long the holder waits for a node's answer after its candidate was written (ADR 0567 §4): the time to
// live of the node's pulse key (delivery.PulseAnswerKey), whose expiry names it silent, never passed. A variable so a test
// on the bus can shorten it.
var AnswerBound = 10 * time.Minute
// Intended is, by node, the modules whose composition the proposal changes, and the nodes that must answer it (§3).
// current is the composition input with the current placement and laidInput the same with the proposal's placements
// (laid, by node; nil removes the node's key) laid over it; nodes is every node to look at (every node with a record).
// A node's set holds the modules the proposal adds, removes or whose placed entry differs on that node, and those whose
// composition differs between the two inputs (delivery.CompareComposed), the node's own named "node". Must is every node
// whose key the proposal changes and every other whose set is not empty, sorted. A node that has no key now gets an
// empty set (§7: the seed is answered as the same). A node whose compositions cannot be made on either side keeps the
// modules its entries name, said in notes; such a node is in Must only when its key changes.
func Intended(current, laidInput *compose.Input, currentPlaced map[string]delivery.Placement, laid map[string]*delivery.Placement,
nodes []string, fps map[string]string) (intended map[string]map[string]bool, must []string, notes []string) {
intended = map[string]map[string]bool{}
inMust := map[string]bool{}
all := map[string]bool{}
for _, n := range nodes {
all[n] = true
}
for n := range laid {
all[n] = true
inMust[n] = true
}
names := make([]string, 0, len(all))
for n := range all {
names = append(names, n)
}
sort.Strings(names)
for _, node := range names {
before, placed := currentPlaced[node]
set := map[string]bool{}
after, changesKey := laid[node]
if changesKey && !placed {
intended[node] = set // a node new to placement: every answer must be the same
continue
}
if changesKey {
var afterModules []delivery.PlacedModule
if after != nil {
afterModules = after.Modules
}
entries := map[string]delivery.PlacedModule{}
for _, m := range before.Modules {
entries[m.Module] = m
}
seen := map[string]bool{}
for _, m := range afterModules {
seen[m.Module] = true
if e, ok := entries[m.Module]; !ok || !reflect.DeepEqual(e, m) {
set[m.Module] = true
}
}
for name := range entries {
if !seen[name] {
set[name] = true
}
}
}
if placed && current != nil && laidInput != nil {
was, _, _, errB := compose.CompositionsShadow(*current, node, fps)
now, _, _, errA := compose.CompositionsShadow(*laidInput, node, fps)
if errA == nil && errB == nil {
running := make([]delivery.Composed, 0, len(was))
for _, c := range was {
running = append(running, c)
}
cmp := delivery.CompareComposed(now, running)
for _, d := range cmp.Differs {
set[d.Module] = true
}
for _, d := range cmp.Added {
set[d.Module] = true
}
for _, m := range cmp.Removed {
set[m] = true
}
} else if changesKey {
notes = append(notes, fmt.Sprintf("what the proposal changes on %s is read from its placement alone: its "+
"compositions cannot be made (now: %v; proposed: %v)", node, errB, errA))
}
}
if len(set) > 0 {
inMust[node] = true
}
if inMust[node] {
intended[node] = set
}
}
for n := range inMust {
must = append(must, n)
}
sort.Strings(must)
return intended, must, notes
}
// JudgeAnswers is the verdict of a proposal's answers (§4). answers is by node its HaveNode.Candidates entry for the
// proposal (nil when it has none) and revisions by node the revision of its candidate key: an answer counts when it
// names the proposal and that revision. It fails on a refusal by a node that must answer, and on every node of Must with
// no answer that counts once its pulse key has ended (pulsing false: the bound's expiry, heard on the bus); while it
// stands it waits for them.
// A difference fails nothing: one Intended computed is listed under Intended, one it did not under Unintended, both
// shown in the ask's brief. The seed alone fails on any difference.
func JudgeAnswers(p delivery.PlacementProposal, intended map[string]map[string]bool, answers map[string]*delivery.CandidateAnswer,
revisions map[string]uint64, online map[string]bool, pulsing map[string]bool, now time.Time) delivery.PlacementVerdict {
v := delivery.PlacementVerdict{At: now.UTC(), Intended: map[string][]string{}, Unintended: map[string][]string{},
Answers: map[string]delivery.CandidateAnswer{}}
for node, set := range intended {
for m := range set {
v.Intended[node] = append(v.Intended[node], m)
}
sort.Strings(v.Intended[node])
}
must := p.Must
if len(must) == 0 {
must = p.Nodes()
}
for _, node := range must {
a := answers[node]
if a == nil || a.Proposal != p.ID || a.Revision == 0 || a.Revision != revisions[node] {
v.Silent = append(v.Silent, node)
continue
}
v.Answers[node] = *a
if a.Refused != "" {
v.Fails = append(v.Fails, fmt.Sprintf("%s cannot compose the proposal: %s", node, a.Refused))
continue
}
for _, r := range a.Refusals {
v.Fails = append(v.Fails, fmt.Sprintf("%s refuses the proposal for %s: %s", node, r.Module, r.Why))
}
want := intended[node]
differs := func(module, words string) {
if p.Seed {
v.Fails = append(v.Fails, fmt.Sprintf("%s %s with the seed, which must be the same as what runs", node, words))
return
}
if !want[module] {
v.Unintended[node] = append(v.Unintended[node], module)
}
}
for _, d := range a.Differs {
differs(d.Module, fmt.Sprintf("composes %s differently (%s)", d.Module, d.Why))
}
for _, d := range a.Added {
differs(d.Module, "would run "+d.Module)
}
for _, m := range a.Removed {
differs(m, "would stop running "+m)
}
sort.Strings(v.Unintended[node])
}
if len(v.Intended) == 0 {
v.Intended = nil
}
if len(v.Unintended) == 0 {
v.Unintended = nil
}
if len(v.Answers) == 0 {
v.Answers = nil
}
var waiting, gone []string
for _, node := range v.Silent {
named := node
if online != nil && !online[node] {
named += " (offline)"
}
if pulsing[node] {
waiting = append(waiting, named)
} else {
gone = append(gone, named)
}
}
if len(waiting) > 0 {
v.Notes = append(v.Notes, fmt.Sprintf("waits for the answers of %s, each for %s from its candidate's write",
strings.Join(waiting, ", "), AnswerBound))
}
if len(gone) > 0 {
v.Fails = append(v.Fails, fmt.Sprintf("%s did not answer within %s of its candidate's write; judged again when it "+
"answers", strings.Join(gone, ", "), AnswerBound))
}
switch {
case len(v.Fails) > 0:
v.State = delivery.VerdictFail
case len(waiting) > 0:
v.State = delivery.VerdictWaits
default:
v.State = delivery.VerdictPass
}
return v
}
// Brief is the verdict as the ask's brief shows it, by node (§5): what the proposal changes and what each node answered
// beside it, marked, with the proposal's why.
func Brief(p delivery.PlacementProposal, v delivery.PlacementVerdict) string {
var b strings.Builder
fmt.Fprintf(&b, "%s (proposed by %s)", p.Why, p.By)
nodes := map[string]bool{}
for n := range v.Intended {
nodes[n] = true
}
for n := range v.Unintended {
nodes[n] = true
}
for _, n := range p.Nodes() {
nodes[n] = true
}
names := make([]string, 0, len(nodes))
for n := range nodes {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
fmt.Fprintf(&b, "; %s:", n)
if m := v.Intended[n]; len(m) > 0 {
fmt.Fprintf(&b, " changes %s", strings.Join(m, ", "))
} else {
b.WriteString(" composes the same")
}
if m := v.Unintended[n]; len(m) > 0 {
fmt.Fprintf(&b, "; also differs, not computed: %s", strings.Join(m, ", "))
}
}
return b.String()
}
+68
View File
@@ -0,0 +1,68 @@
package judge
import (
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// ADR 0567 §3, §4 (rows moved from the assigner's check of a pull request's placement, ADR 0559 §5): what a proposal
// changes by node, and the verdict of the answers: differences shown and failing nothing, a refusal and a silent node
// failing, a later answer judging again, the seed failing on any difference.
func TestJudge(t *testing.T) {
current := map[string]delivery.Placement{"anchor": {Node: "anchor", Modules: []delivery.PlacedModule{{Module: "mail"}}}, "server": {Node: "server"}}
laid := map[string]*delivery.Placement{
"anchor": {Node: "anchor", Modules: []delivery.PlacedModule{{Module: "mail", Ports: map[string]int{"25": 2525}}, {Module: "web"}}},
"laptop": {Node: "laptop", Modules: []delivery.PlacedModule{{Module: "web"}}}}
got, must, _ := Intended(nil, nil, current, laid, []string{"anchor", "laptop", "server"}, nil)
if !got["anchor"]["mail"] || !got["anchor"]["web"] || len(got["laptop"]) != 0 || !slices.Equal(must, []string{"anchor", "laptop"}) {
t.Errorf("intended = %v, must %v: want anchor's mail and web, nothing on the laptop that has no key, server untouched", got, must)
}
at := time.Date(2026, 10, 27, 2, 0, 0, 0, time.UTC)
p := delivery.PlacementProposal{ID: "p-one", By: "op", Why: "move web", Written: at, Base: map[string]uint64{"anchor": 3, "laptop": 0},
Must: []string{"anchor", "laptop"}}
revisions := map[string]uint64{"anchor": 7, "laptop": 8}
intended := map[string]map[string]bool{"anchor": {"web": true}}
same := func(rev uint64) *delivery.CandidateAnswer {
return &delivery.CandidateAnswer{Proposal: "p-one", Revision: rev, Verdict: delivery.CandidateSame, Same: []string{"mail"}}
}
differs := func(rev uint64, module string) *delivery.CandidateAnswer {
return &delivery.CandidateAnswer{Proposal: "p-one", Revision: rev, Verdict: delivery.CandidateDiffers,
Differs: []delivery.ModuleDiffers{{Module: module, Why: "the port differs"}}}
}
for _, row := range []struct {
name string
seed bool
answers map[string]*delivery.CandidateAnswer
pulsing map[string]bool
state string
words string
marked string
}{
{"all the same", false, map[string]*delivery.CandidateAnswer{"anchor": same(7), "laptop": same(8)}, nil, delivery.VerdictPass, "", ""},
{"a module added where intended", false, map[string]*delivery.CandidateAnswer{"anchor": {Proposal: "p-one", Revision: 7, Verdict: delivery.CandidateDiffers,
Added: []delivery.ModuleAdded{{Module: "web", Unissued: []string{"its own secret"}}}}, "laptop": same(8)}, nil, delivery.VerdictPass, "", ""},
{"a difference not computed passes, marked", false, map[string]*delivery.CandidateAnswer{"anchor": differs(7, "mail"), "laptop": same(8)}, nil, delivery.VerdictPass, "", "mail"},
{"the seed with a difference", true, map[string]*delivery.CandidateAnswer{"anchor": same(7), "laptop": differs(8, "web")}, nil, delivery.VerdictFail, "laptop composes web differently", ""},
{"a refusal", false, map[string]*delivery.CandidateAnswer{"anchor": same(7), "laptop": {Proposal: "p-one", Revision: 8, Verdict: delivery.CandidateRefused,
Refusals: []delivery.ModuleRefused{{Module: "web", Why: "no own secret"}}}}, nil, delivery.VerdictFail, "laptop refuses", ""},
{"a node not answering, within its bound", false, map[string]*delivery.CandidateAnswer{"anchor": same(7)}, map[string]bool{"laptop": true}, delivery.VerdictWaits, "laptop (offline)", ""},
{"an answer of an earlier revision, past its bound", false, map[string]*delivery.CandidateAnswer{"anchor": same(6), "laptop": same(8)}, nil, delivery.VerdictFail, "anchor did not answer", ""},
{"its later answer", false, map[string]*delivery.CandidateAnswer{"anchor": same(7), "laptop": same(8)}, nil, delivery.VerdictPass, "", ""},
} {
q := p
q.Seed = row.seed
v := JudgeAnswers(q, intended, row.answers, revisions, map[string]bool{"anchor": true}, row.pulsing, at)
all := strings.Join(append(append([]string{}, v.Fails...), v.Notes...), "; ")
if v.State != row.state || !strings.Contains(all, row.words) {
t.Errorf("%s: %s, %q", row.name, v.State, all)
}
if row.marked != "" && (!slices.Contains(v.Unintended["anchor"], row.marked) || !strings.Contains(Brief(q, v), "not computed: "+row.marked)) {
t.Errorf("%s: the difference is not marked: %+v, %s", row.name, v.Unintended, Brief(q, v))
}
}
}
+3
View File
@@ -24,6 +24,9 @@ type composer struct {
in Input
said []string
seen map[string]bool
// unhostable is, from the last look at the rest of the mesh, each module assigned to another node that this
// composition finds that node cannot run: what it would have offered is missing here (issue 1126, probe #532).
unhostable []string
}
func newComposer(in Input) *composer { return &composer{in: in, seen: map[string]bool{}} }
+3
View File
@@ -128,6 +128,9 @@ func (c *composer) places(nodeName string, resolved Resolved) (Placed, error) {
continue
}
r.Kept = c.in.OperatorExport
if r.Kept == nil {
r.KeptRuns = c.in.KeptRuns[node.Name]
}
break
}
+15 -1
View File
@@ -3,6 +3,7 @@ package compose
import (
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -203,7 +204,7 @@ func (c *composer) resolution(nodeName string) (catalogue.Resolution, error) {
return catalogue.Resolution{}, err
}
shelf := c.shelfFor(node)
world, _, _, err := c.theRestOfTheMesh(nodeName)
world, _, left, err := c.theRestOfTheMesh(nodeName)
if err != nil {
return catalogue.Resolution{}, err
}
@@ -226,6 +227,19 @@ func (c *composer) resolution(nodeName string) (catalogue.Resolution, error) {
Account: node.Account, AccountHome: node.AccountHome,
AgentAccount: node.AgentAccount, AgentAccountHome: node.AgentAccountHome}, world)
if err != nil {
// What another node's set would have offered is missing when that set does not resolve here: said with the
// refusal, so "nothing provides" is read beside why (issue 1126, probe #532).
if len(left) > 0 {
var why []string
for _, l := range left {
why = append(why, l.String())
}
err = fmt.Errorf("%w\n\nand %d node(s) offer nothing here, as they could not be resolved: %s", err, len(left), strings.Join(why, "; "))
}
if len(c.unhostable) > 0 {
err = fmt.Errorf("%w\n\nand these modules offer nothing here, as their node cannot run them in this composition: %s",
err, strings.Join(c.unhostable, "; "))
}
// The node's own set does not compose: the only failure a mesh-wide gatherer may pass over.
return catalogue.Resolution{}, notResolvable{err}
}
+5
View File
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -100,6 +101,7 @@ func (c *composer) theRestOfTheMesh(exclude string) (catalogue.World, Passes, []
var left []LeftOut
offered := map[string][]catalogue.Provider{}
var firstHeld []catalogue.Held
c.unhostable = nil
for _, o := range others {
got, err := catalogue.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil {
@@ -107,6 +109,9 @@ func (c *composer) theRestOfTheMesh(exclude string) (catalogue.World, Passes, []
left = append(left, LeftOut{Node: o.node.Name, Err: err})
continue
}
for _, u := range got.Unhostable {
c.unhostable = append(c.unhostable, fmt.Sprintf("%s on %s lacks %s", u.Module, o.node.Name, strings.Join(u.Missing, ", ")))
}
firstHeld = append(firstHeld, got.Claims...)
for _, m := range got.Modules {
for _, name := range m.OffersAt(catalogue.ScopeMesh) {
+177
View File
@@ -0,0 +1,177 @@
package delivery
import (
"bytes"
"encoding/json"
"sort"
"strconv"
"strings"
)
// DifferingPaths is every JSON path at which two bodies of one resource differ, sorted (issue 1126, T1: the shadow
// shows its work): object keys and list places joined by dots (`content.url`, `links.0`), a string that holds a JSON
// object or list read as that JSON and walked on (a runtime's config carried as text), a key or place one side alone
// holds named as it is. A key a secret could be under (secretWords) is named and not walked into. A body that is not
// JSON differs at "" (the whole), and a resource one side alone holds has none. At most MaxPaths: the last then says how many more there are. A path names keys
// alone, never a value, so it holds no secret.
func DifferingPaths(composed, running json.RawMessage) []string {
if composed == nil || running == nil {
return nil // one side alone holds it: its why says so
}
var a, b any
okA := unmarshalNumbers(composed, &a) == nil
okB := unmarshalNumbers(running, &b) == nil
if !okA || !okB {
if bytes.Equal(composed, running) {
return nil
}
return []string{""}
}
var out []string
walkDiffering(a, b, "", &out)
sort.Strings(out)
if len(out) > MaxPaths {
more := len(out) - (MaxPaths - 1)
out = append(out[:MaxPaths-1], "… "+strconv.Itoa(more)+" more")
}
return out
}
func unmarshalNumbers(raw []byte, into *any) error {
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
return d.Decode(into)
}
// asJSON is a string holding a JSON object or list read as that value; v itself otherwise.
func asJSON(v any) any {
s, ok := v.(string)
if !ok {
return v
}
t := strings.TrimSpace(s)
if t == "" || (t[0] != '{' && t[0] != '[') {
return v
}
var parsed any
if unmarshalNumbers([]byte(t), &parsed) != nil {
return v
}
return parsed
}
func join(at, key string) string {
if at == "" {
return key
}
return at + "." + key
}
func walkDiffering(a, b any, at string, out *[]string) {
a, b = asJSON(a), asJSON(b)
ma, isMA := a.(map[string]any)
mb, isMB := b.(map[string]any)
if isMA && isMB {
keys := map[string]bool{}
for k := range ma {
keys[k] = true
}
for k := range mb {
keys[k] = true
}
for k := range keys {
x, inA := ma[k]
y, inB := mb[k]
switch {
case !inA || !inB:
*out = append(*out, join(at, k))
case secretKey(k):
// A file's secrets (SecretsField) are named entry by entry, each entry never entered; any other
// object under a secret key, a vault box included, is one path.
sx, isSX := x.(map[string]any)
sy, isSY := y.(map[string]any)
if k != SecretsField || !isSX || !isSY {
if !sameAny(x, y) {
*out = append(*out, join(at, k))
}
continue
}
for name := range sx {
if v, in := sy[name]; !in || !sameAny(sx[name], v) {
*out = append(*out, join(join(at, k), name))
}
}
for name := range sy {
if _, in := sx[name]; !in {
*out = append(*out, join(join(at, k), name))
}
}
default:
walkDiffering(x, y, join(at, k), out)
}
}
return
}
la, isLA := a.([]any)
lb, isLB := b.([]any)
if isLA && isLB {
for i := range max(len(la), len(lb)) {
if i >= len(la) || i >= len(lb) {
*out = append(*out, join(at, strconv.Itoa(i)))
continue
}
walkDiffering(la[i], lb[i], join(at, strconv.Itoa(i)), out)
}
return
}
if !sameAny(a, b) {
*out = append(*out, at)
}
}
func sameAny(a, b any) bool {
x, errA := json.Marshal(a)
y, errB := json.Marshal(b)
return errA == nil && errB == nil && bytes.Equal(x, y)
}
// FitAnswer is a with its shown bodies cut until its JSON is within limit bytes (MaxAnswerBytes for the node's key):
// first the bodies' texts, from the last differing module back, keeping their paths; then the paths too. What is cut
// is said in the answer's note. The verdict and every list of modules are never cut.
func FitAnswer(a CandidateAnswer, limit int) CandidateAnswer {
size := func() int {
raw, err := json.Marshal(a)
if err != nil {
return 0
}
return len(raw)
}
if size() <= limit {
return a
}
a.Differs = append([]ModuleDiffers(nil), a.Differs...)
for i := range a.Differs {
a.Differs[i].Bodies = append([]ResourceBodies(nil), a.Differs[i].Bodies...)
}
cutTexts, cutPaths := false, false
for i := len(a.Differs) - 1; i >= 0 && size() > limit; i-- {
for j := range a.Differs[i].Bodies {
a.Differs[i].Bodies[j].Composed, a.Differs[i].Bodies[j].Running = "", ""
}
cutTexts = true
}
for i := len(a.Differs) - 1; i >= 0 && size() > limit; i-- {
for j := range a.Differs[i].Bodies {
a.Differs[i].Bodies[j].Paths = nil
}
cutPaths = true
}
why := "the bodies shown were cut to keep the answer within " + strconv.Itoa(limit>>10) + " KiB"
if cutPaths {
why += ", their paths too"
}
if cutTexts {
a.Note = strings.TrimPrefix(a.Note+"; ", "; ") + why
}
return a
}
-26
View File
@@ -4,38 +4,12 @@ import (
"context"
"fmt"
"sort"
"strconv"
"strings"
"sync"
)
// DefaultCapacity is the build capacity of a node that has no setting: one build step at a time.
const DefaultCapacity = 1
// ParseCapacity reads a capacity setting: `node=steps` pairs separated by commas, such as
// `shanks=3,ace=2`. A node without a pair has DefaultCapacity. A pair with no node, a count that is not a whole number of
// at least 0, and a node named twice are refused. A count of 0 keeps the part off the node: it is never chosen.
func ParseCapacity(s string) (map[string]int, error) {
out := map[string]int{}
for _, pair := range strings.Split(s, ",") {
pair = strings.TrimSpace(pair)
if pair == "" {
continue
}
node, count, ok := strings.Cut(pair, "=")
node, count = strings.TrimSpace(node), strings.TrimSpace(count)
n, err := strconv.Atoi(count)
if !ok || node == "" || err != nil || n < 0 {
return nil, fmt.Errorf("the capacity %q is not <node>=<steps at once, 0 or more>", pair)
}
if _, dup := out[node]; dup {
return nil, fmt.Errorf("the capacity names %s twice", node)
}
out[node] = n
}
return out, nil
}
// Places counts the build steps a part runs on each node and holds a lane back while its node is at its
// capacity (the operator's decision of 2026-10-21; ADR 0478 clause 3). A lane is the fetch and the builds of one build
// module on one node, run one step at a time, so a lane holds one place while it runs. A node is chosen for a lane by
+2 -24
View File
@@ -7,21 +7,6 @@ import (
"time"
)
func TestParseCapacityReadsNodeEqualsCountPairs(t *testing.T) {
c, err := ParseCapacity(" shanks=3, ace=2 ")
if err != nil || c["shanks"] != 3 || c["ace"] != 2 || len(c) != 2 {
t.Errorf("capacity = %v, %v", c, err)
}
if c, err := ParseCapacity(""); err != nil || len(c) != 0 {
t.Errorf("empty: %v, %v", c, err)
}
for _, bad := range []string{"shanks", "shanks=x", "=2", "a=1,a=2", "shanks=-1"} {
if _, err := ParseCapacity(bad); err == nil {
t.Errorf("%q accepted", bad)
}
}
}
func acquireAsync(p *Places, node string) chan func() {
got := make(chan func(), 1)
go func() {
@@ -95,17 +80,10 @@ func TestAWaitingStepStopsWhenItsContextEnds(t *testing.T) { // round 3
// The operator's decision of 2026-10-21 13:40: the builder builds on shanks and ace only; a capacity of 0 keeps it off a node.
func TestACapacityOfZeroKeepsThePartOffTheNodeAndANodeNotNamedKeepsOne(t *testing.T) { // issue 1661
c, err := ParseCapacity("shanks=3,ace=2,g14=0,novox=0")
if err != nil || c["g14"] != 0 || c["novox"] != 0 || c["shanks"] != 3 || len(c) != 4 {
t.Fatalf("capacity = %v, %v", c, err)
}
p := &Places{Capacity: c}
p := &Places{Capacity: map[string]int{"shanks": 3, "ace": 2, "g14": 0, "novox": 0}}
if p.CapacityOf("g14") != 0 || p.CapacityOf("novox") != 0 || p.CapacityOf("ace") != 2 || p.CapacityOf("other") != DefaultCapacity {
t.Errorf("capacities = %d %d %d %d, want 0 0 2 and the default for a node not named", p.CapacityOf("g14"), p.CapacityOf("novox"), p.CapacityOf("ace"), p.CapacityOf("other"))
}
if _, err := ParseCapacity("shanks=-1"); err == nil {
t.Error("a negative capacity accepted")
}
entries := tcs("go", "go1.27.1", "ace", "g14", "novox", "shanks", "other")
for i := 0; i < 6; i++ {
nodes, _, err := ChooseNodes("go", entries, p, nil, 2)
@@ -119,7 +97,7 @@ func TestACapacityOfZeroKeepsThePartOffTheNodeAndANodeNotNamedKeepsOne(t *testin
}
}
// Only nodes of capacity 0 report: nothing qualifies, and the build says why.
_, _, err = ChooseNodes("go", tcs("go", "go1.27.1", "g14", "novox"), p, nil, 2)
_, _, err := ChooseNodes("go", tcs("go", "go1.27.1", "g14", "novox"), p, nil, 2)
if err == nil || !strings.Contains(err.Error(), "capacity") {
t.Errorf("only capacity-0 nodes: error = %v, want it to name the capacity", err)
}
+225 -5
View File
@@ -28,6 +28,13 @@ type Comparison struct {
// resource is equal it says what else differs (their order, the files filled, the data, the node's own parts, a file
// carried by digest).
func CompareComposed(composed map[string]Composed, running []Composed) Comparison {
return CompareComposedWith(composed, running, nil)
}
// CompareComposedWith is CompareComposed showing, for each module that differs, every differing resource with both
// bodies masked (MaskedPair); sameValue, when given, says whether two sealed values open to the same value (the node
// opens them with its own key), so a value sealed again reads as such.
func CompareComposedWith(composed map[string]Composed, running []Composed, sameValue func(a, b string) (same, known bool)) Comparison {
name := func(c Composed) string {
if c.Module == "" {
return HaveNodeWord
@@ -60,7 +67,7 @@ func CompareComposed(composed map[string]Composed, running []Composed) Compariso
case !composedHere:
out.Removed = append(out.Removed, m)
default:
if d, ok := compareOne(m, a, b); ok {
if d, ok := compareOne(m, a, b, sameValue); ok {
out.Same = append(out.Same, m)
} else {
out.Differs = append(out.Differs, d)
@@ -71,8 +78,11 @@ func CompareComposed(composed map[string]Composed, running []Composed) Compariso
}
// compareOne is one module's comparison: true when the digests agree, else what differs first.
func compareOne(module string, composed, running Composed) (ModuleDiffers, bool) {
func compareOne(module string, composed, running Composed, sameValue func(a, b string) (bool, bool)) (ModuleDiffers, bool) {
composed, running = withoutBusUsers(composed), withoutBusUsers(running)
// A sealed value is judged by the value it opens to, slot by slot (issue 1126, C1): a value sealed again at the same
// field of the same resource is the box that runs, never a difference; and a box is never matched across resources.
composed = KeepSeals(composed, SealsIn(running), sameValue)
dc, errC := ComposedDigestOf(composed)
dr, errR := ComposedDigestOf(running)
if errC != nil {
@@ -99,17 +109,38 @@ func compareOne(module string, composed, running Composed) (ModuleDiffers, bool)
for id := range rb {
ids[id] = true
}
var first ModuleDiffers
found := false
var bodies []ResourceBodies
for _, id := range sortedKeys(ids) {
x, inA := ra[id]
y, inB := rb[id]
why := ""
switch {
case !inB:
return differs(id, "composed, and not in what runs")
why = "composed, and not in what runs"
case !inA:
return differs(id, "in what runs, and not composed")
why = "in what runs, and not composed"
case !bytes.Equal(x, y):
return differs(id, "the resource differs")
why = "the resource differs"
default:
continue
}
if !found {
first, _ = differs(id, why)
found = true
}
if len(bodies) < MaxBodies {
// The paths are found on the whole bodies, before either is cut (issue 1126, T1: three resources over
// MaxBody could not be told apart).
c, r := MaskedPair(x, y, sameValue)
bodies = append(bodies, ResourceBodies{ID: id, Why: why, Paths: DifferingPaths(x, y),
Composed: cut(c), Running: cut(r)})
}
}
if found {
first.Bodies = bodies
return first, false
}
switch {
case !jsonEqual(a.Filled, b.Filled):
@@ -165,3 +196,192 @@ func resourcesByID(resources []json.RawMessage) map[string][]byte {
}
return out
}
// cut is s cut at MaxBody bytes, saying so.
func cut(s string) string {
if len(s) <= MaxBody {
return s
}
return s[:MaxBody] + "… (cut at " + strconv.Itoa(MaxBody) + " bytes)"
}
// secretWords are the words of a key whose value may hold a secret or a sealed one: masked in a shown body.
var secretWords = []string{"sealed", "secret", "password", "passwd", "token", "private", "key", "credential", "jwt", "salt", "hash"}
func secretKey(k string) bool {
k = strings.ToLower(k)
for _, w := range secretWords {
if strings.Contains(k, w) {
return true
}
}
return false
}
// MaskedPair is two bodies of one resource (either may be nil) as JSON text with every value under a key a secret could
// be in (secretWords, at any depth) masked: "<masked: the same on both sides>", "<masked: another seal of the same
// value>" or "<masked: another value>" when sameValue says so, "<masked: differs>" otherwise, "<masked>" on one side
// alone. A body that is not JSON is shown as its length only.
func MaskedPair(composed, running json.RawMessage, sameValue func(a, b string) (same, known bool)) (string, string) {
var a, b any
okA := composed != nil && json.Unmarshal(composed, &a) == nil
okB := running != nil && json.Unmarshal(running, &b) == nil
if okA && okB {
a, b = maskBoth(a, b, sameValue)
} else if okA {
a, _ = maskBoth(a, nil, nil)
} else if okB {
_, b = maskBoth(nil, b, nil)
}
show := func(v any, raw json.RawMessage, ok bool) string {
if raw == nil {
return ""
}
if !ok {
return "<not JSON: " + strconv.Itoa(len(raw)) + " bytes>"
}
var out bytes.Buffer
enc := json.NewEncoder(&out)
enc.SetEscapeHTML(false)
_ = enc.Encode(v)
return strings.TrimSuffix(out.String(), "\n")
}
return show(a, composed, okA), show(b, running, okB)
}
func maskBoth(a, b any, sameValue func(a, b string) (bool, bool)) (any, any) {
ma, isA := a.(map[string]any)
mb, isB := b.(map[string]any)
if isA || isB {
outA, outB := map[string]any{}, map[string]any{}
keys := map[string]bool{}
for k := range ma {
keys[k] = true
}
for k := range mb {
keys[k] = true
}
for k := range keys {
va, inA := ma[k]
vb, inB := mb[k]
if secretKey(k) {
x, y := maskAll(k, va, vb, inA && inB, sameValue)
if inA {
outA[k] = x
}
if inB {
outB[k] = y
}
continue
}
x, y := maskBoth(va, vb, sameValue)
if inA {
outA[k] = x
}
if inB {
outB[k] = y
}
}
var ra, rb any
if isA {
ra = outA
} else {
ra = a
}
if isB {
rb = outB
} else {
rb = b
}
return ra, rb
}
la, isLA := a.([]any)
lb, isLB := b.([]any)
if isLA || isLB {
outA := make([]any, len(la))
outB := make([]any, len(lb))
for i := range max(len(la), len(lb)) {
var x, y any
if i < len(la) {
x = la[i]
}
if i < len(lb) {
y = lb[i]
}
x, y = maskBoth(x, y, sameValue)
if i < len(la) {
outA[i] = x
}
if i < len(lb) {
outB[i] = y
}
}
var ra, rb any = a, b
if isLA {
ra = outA
}
if isLB {
rb = outB
}
return ra, rb
}
return a, b
}
// maskAll is a value under a key a secret could be under, masked as one word (maskWord); a file's secrets
// (SecretsField) alone entry by entry, each its own sealed value: "another seal of the same value" or "another value"
// per secret, its name shown (issue 1126, C1). Any other object, a vault box included, is one word, judged whole.
func maskAll(key string, a, b any, both bool, sameValue func(a, b string) (bool, bool)) (any, any) {
ma, isA := a.(map[string]any)
mb, isB := b.(map[string]any)
if key != SecretsField || (!isA && !isB) {
w := maskWord(a, b, both, sameValue)
return w, w
}
outA, outB := map[string]any{}, map[string]any{}
for k, va := range ma {
vb, inB := mb[k]
outA[k] = maskWord(va, vb, isB && inB, sameValue)
}
for k, vb := range mb {
va, inA := ma[k]
outB[k] = maskWord(va, vb, isA && inA, sameValue)
}
// A map on one side and another value on the other: the other is one word.
var ra, rb any = outA, outB
if !isA {
ra = maskWord(a, b, both, sameValue)
}
if !isB {
rb = maskWord(a, b, both, sameValue)
}
return ra, rb
}
func maskWord(a, b any, both bool, sameValue func(a, b string) (bool, bool)) string {
if !both {
return "<masked>"
}
x, _ := json.Marshal(a)
y, _ := json.Marshal(b)
if bytes.Equal(x, y) {
return "<masked: the same on both sides>"
}
if sameValue != nil {
sa, okA := a.(string)
sb, okB := b.(string)
if !okA {
sa = string(x)
}
if !okB {
sb = string(y)
}
if same, known := sameValue(sa, sb); known {
if same {
return "<masked: another seal of the same value>"
}
return "<masked: another value>"
}
}
return "<masked: differs>"
}
+194
View File
@@ -0,0 +1,194 @@
package delivery
import (
"bytes"
"encoding/json"
"sort"
"strconv"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Seals is, by slot, each value one composition carries sealed to its node's key (issue 1126, C1): a file's `sealed`
// a vault box (catalogue.VaultSealedField) and each entry of a file's secrets (SecretsField), as their JSON text (a box is carried as its JSON, a sealed value as
// the string it is). A slot is one field of one resource of the composition's module (SealSlot): a box is reused only
// at the slot that holds it, never for another resource whose value opens to the same (a provider's grant and its
// consumer's secret on one node hold one value).
type Seals map[string]string
// SealedFields are the fields of a resource that carry a value sealed to the node's key.
var SealedFields = []string{"sealed", catalogue.VaultSealedField}
// SealSlot is the slot of one sealed field of one resource, by the resource's id (its place, `#<i>`, when it has none).
// The module is the composition's: a slot is never compared across compositions.
func SealSlot(resource, field string) string { return resource + "#" + field }
// SecretsField is the field of a file that carries, by name, each sealed value its content asks for
// (${secret:<name>}, catalogue's intoFile): each entry is a slot of its own (SecretSlotField), so a secret sealed again
// is judged by the value it opens to, entry by entry, like a file's `sealed` (issue 1126, C1; the seed-7 probe: about 45
// files on ace and novox differed by their secrets' bytes alone).
const SecretsField = "secrets"
// SecretSlotField is the field part of the slot of one entry of a file's secrets (SealSlot(id, SecretSlotField(name))).
func SecretSlotField(name string) string { return SecretsField + "." + name }
// sealAt is one sealed value of a resource: its slot's field, its place (a top-level field, or an entry of the
// secrets), and its raw JSON.
type sealAt struct {
field, entry string
raw json.RawMessage
}
// sealsAt is every sealed value of one resource's fields, in a fixed order: SealedFields, then the secrets by name.
func sealsAt(fields map[string]json.RawMessage) []sealAt {
var out []sealAt
for _, field := range SealedFields {
if f, has := fields[field]; has {
out = append(out, sealAt{field: field, raw: f})
}
}
var secrets map[string]json.RawMessage
if f, has := fields[SecretsField]; has && json.Unmarshal(f, &secrets) == nil {
names := make([]string, 0, len(secrets))
for name := range secrets {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
out = append(out, sealAt{field: SecretSlotField(name), entry: name, raw: secrets[name]})
}
}
return out
}
// SealsIn is every sealed value c carries, by slot; nil when it carries none.
func SealsIn(c Composed) Seals {
var out Seals
for i, raw := range c.Resources {
id, fields, ok := sealedFieldsOf(raw, i)
if !ok {
continue
}
for _, s := range sealsAt(fields) {
if text := sealedText(s.raw); text != "" {
if out == nil {
out = Seals{}
}
out[SealSlot(id, s.field)] = text
}
}
}
return out
}
// KeepSeals is c with each sealed value replaced by the one kept at the same slot, where same says the two open to the
// same value (known and same): a value the vault sealed again is then the box that runs, so the composition is the
// one that runs (ADR 0547 §3, issue 1126). A slot kept holds nothing for, or whose value differs or cannot be opened,
// keeps c's. Every other byte of a resource is kept as it is; c itself is not changed.
func KeepSeals(c Composed, kept Seals, same func(a, b string) (same, known bool)) Composed {
if len(kept) == 0 || same == nil {
return c
}
out := c
out.Resources = nil
for i, raw := range c.Resources {
id, fields, ok := sealedFieldsOf(raw, i)
if ok {
for _, s := range sealsAt(fields) {
ours := sealedText(s.raw)
theirs, held := kept[SealSlot(id, s.field)]
if !held || ours == "" || theirs == ours {
continue
}
if opens, known := same(ours, theirs); !known || !opens {
continue
}
value := json.RawMessage(theirs)
if s.raw[0] == '"' {
value, _ = json.Marshal(theirs)
}
if s.entry == "" {
if swapped, ok := withField(raw, s.field, value); ok {
raw = swapped
}
continue
}
// An entry of the secrets: swapped inside the secrets as they stand in raw, then the secrets in raw.
var now map[string]json.RawMessage
if json.Unmarshal(raw, &now) != nil {
continue
}
if secrets, ok := withField(now[SecretsField], s.entry, value); ok {
if swapped, ok := withField(raw, SecretsField, secrets); ok {
raw = swapped
}
}
}
}
out.Resources = append(out.Resources, raw)
}
return out
}
// sealedFieldsOf is a resource's id (`#<i>` when it has none) and its top-level fields, false when it is no object.
func sealedFieldsOf(raw json.RawMessage, i int) (string, map[string]json.RawMessage, bool) {
var fields map[string]json.RawMessage
if json.Unmarshal(raw, &fields) != nil {
return "", nil, false
}
var id string
if f, ok := fields["id"]; !ok || json.Unmarshal(f, &id) != nil || id == "" {
id = "#" + strconv.Itoa(i)
}
return id, fields, true
}
// sealedText is a sealed field as its text: a string as it is, a box as its JSON.
func sealedText(f json.RawMessage) string {
var text string
if json.Unmarshal(f, &text) == nil {
return text
}
var compact bytes.Buffer
if json.Compact(&compact, f) != nil {
return string(f)
}
return compact.String()
}
// withField is the object raw with its top-level field replaced by value, every other byte as it was (so a composition
// whose sealed values are those that run is the one that runs, byte for byte); false when raw is no object holding it.
func withField(raw json.RawMessage, field string, value json.RawMessage) (json.RawMessage, bool) {
d := json.NewDecoder(bytes.NewReader(raw))
if t, err := d.Token(); err != nil || t != json.Delim('{') {
return nil, false
}
for d.More() {
t, err := d.Token()
if err != nil {
return nil, false
}
key, _ := t.(string)
before := d.InputOffset()
var skip json.RawMessage
if err := d.Decode(&skip); err != nil {
return nil, false
}
if key != field {
continue
}
after := d.InputOffset()
// The span holds the colon, any space and the value: kept up to the colon, the value replaced.
span := raw[before:after]
colon := bytes.IndexByte(span, ':')
if colon < 0 {
return nil, false
}
out := make([]byte, 0, len(raw)-len(span)+colon+1+len(value))
out = append(out, raw[:int(before)+colon+1]...)
out = append(out, value...)
out = append(out, raw[after:]...)
return out, true
}
return nil, false
}
+29 -3
View File
@@ -48,6 +48,7 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"reflect"
"regexp"
"slices"
@@ -355,6 +356,10 @@ type Have struct {
// after a restart while it opens to the vault's value, so a value sealed again moves no composition. Openable by the
// node alone, as the vault hands them over the bus. Empty on every other have.
Sealed []string `json:"sealed,omitempty"`
// SealedAt is Sealed by slot (Seals: a field of a resource, issue 1126, C1): the composer reuses a box only at the
// slot that holds it, so a provider and a consumer of one value on one node each keep their own. Written in Sealed's
// place by a node-engine that knows it; Sealed is read only from a have written before.
SealedAt Seals `json:"sealed-at,omitempty"`
}
// UserListWritten is the bus's user list as the node-engine of the bus node wrote it to the bus module's file, applied
@@ -406,12 +411,26 @@ type HaveNode struct {
// applied (Have.Sealed for the node's own, issue 1126): a value the vault seals again after a restart moves no
// composition, so a restart applies nothing again.
Sealed []string `json:"sealed,omitempty"`
// Candidates is the node's answer to each placement candidate naming it, by group (PlacementCandidate): written by
// SealedAt is Sealed by slot (Have.SealedAt).
SealedAt Seals `json:"sealed-at,omitempty"`
// Candidates is the node's answer to each placement candidate naming it, by proposal id (PlacementCandidate): written by
// its shadow composer whenever an answer changes, removed when the candidate key goes. The node applies nothing of a
// candidate.
Candidates map[string]CandidateAnswer `json:"candidates,omitempty"`
// Composes is whose compositions the node applies: its own (ComposesOwn, placement places it) or the
// controller's (ComposesController); ComposesWhy says why in words. Written by the node agent at each start, so a
// node returning to the controller's compositions when its placement key is deleted says so (issue 1126). Empty
// from an engine older than the field.
Composes string `json:"composes,omitempty"`
ComposesWhy string `json:"composes-why,omitempty"`
}
// Whose compositions a node applies (HaveNode.Composes).
const (
ComposesOwn = "own"
ComposesController = "controller"
)
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
// writes it.
type Presence struct {
@@ -506,6 +525,11 @@ func DecodeHave(data []byte) (Have, error) {
}
func (n HaveNode) check() error {
switch n.Composes {
case "", ComposesOwn, ComposesController:
default:
return fmt.Errorf("delivery: a node composes %q, neither %s nor %s", n.Composes, ComposesOwn, ComposesController)
}
for group, a := range n.Candidates {
if group == "" {
return fmt.Errorf("delivery: a node's candidate answer names no group")
@@ -569,10 +593,11 @@ func DecodePulse(data []byte) (Pulse, error) {
func SameHave(a, b Have) bool {
ua, ub := a.UserList, b.UserList
a.UserList, b.UserList = nil, nil
if !slices.Equal(a.Sealed, b.Sealed) {
if !slices.Equal(a.Sealed, b.Sealed) || !maps.Equal(a.SealedAt, b.SealedAt) {
return false
}
a.Sealed, b.Sealed = nil, nil
a.SealedAt, b.SealedAt = nil, nil
if !reflect.DeepEqual(a, b) || (ua == nil) != (ub == nil) {
return false
}
@@ -590,7 +615,8 @@ func SameHaveNode(a, b HaveNode) bool {
return a.V == b.V && a.Composed == b.Composed && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile) &&
(a.Record == nil) == (b.Record == nil) && (a.Record == nil || SameNodeRecord(*a.Record, *b.Record)) &&
jsonEqual(a.Sealed, b.Sealed) && jsonEqual(a.Candidates, b.Candidates)
jsonEqual(a.Sealed, b.Sealed) && maps.Equal(a.SealedAt, b.SealedAt) && jsonEqual(a.Candidates, b.Candidates) && a.Composes == b.Composes &&
a.ComposesWhy == b.ComposesWhy
}
// SamePresence says whether two presences are the same value.
+7 -16
View File
@@ -9,10 +9,9 @@ const oddManifest = "{\n\t\"module\": \"a\",\n \"z\": 1, \"a\": [ \"x\",\"y\"
func mergedWithInputs() Forge {
return Forge{Kind: ForgeMerged, Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa", Merge: "cccc", Root: "t0",
Trees: map[string]string{"a": "t1", "b": "t2"},
Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("ab", 32)}},
Placements: map[string]string{"ace": `{"v":1,"node":"ace","modules":[]}`}, PlacementsRead: true}
Trees: map[string]string{"a": "t1", "b": "t2"},
Manifests: map[string]string{"a": oddManifest},
Locks: map[string]map[string]string{"a": {"go.sum": strings.Repeat("ab", 32)}}}
}
// 1442/AC1
@@ -22,7 +21,7 @@ func TestForgeMergedCarriesManifestsAndLocks(t *testing.T) {
if err != nil {
t.Fatal(err)
}
for _, frag := range []string{`"manifests":{"a":`, `"locks":{"a":{"go.sum":`, `"placements":{"ace":`, `"placements-read":true`} {
for _, frag := range []string{`"manifests":{"a":`, `"locks":{"a":{"go.sum":`} {
if !strings.Contains(string(data), frag) {
t.Errorf("the encoding lacks %s: %s", frag, data)
}
@@ -48,9 +47,9 @@ func TestForgeMergedRefusesBadInputs(t *testing.T) {
"locks outside trees": {func(f *Forge) { f.Locks["yy"] = map[string]string{"go.sum": "h"} }, `"yy"`},
"empty manifest": {func(f *Forge) { f.Manifests["b"] = "" }, `"b"`},
"empty lock hash": {func(f *Forge) { f.Locks["a"]["go.sum"] = "" }, `"a"`},
"placements unsaid": {func(f *Forge) { f.PlacementsRead = false }, "placements-read"},
"placement of no node": {func(f *Forge) { f.Placements["a.b"] = "{}" }, "names no node"},
"placements on a pull": {func(f *Forge) { f.Kind, f.Action, f.Merge, f.Root, f.Trees = ForgePull, PullOpened, "", "", nil }, "only a merged message"},
// No repository holds placement (ADR 0567 §11): a message carrying placement files is refused, on a merge or a pull.
"placement files": {func(f *Forge) { f.RetiredPlacements = []byte(`{"anchor":"{}"}`) }, "mesh-placement"},
"placements read": {func(f *Forge) { f.RetiredRead = true }, "mesh-placement"},
} {
f := mergedWithInputs()
tc.mut(&f)
@@ -86,14 +85,6 @@ func TestForgePullCarriesManifestsAndLocks(t *testing.T) {
if _, err := EncodeForge(f); err == nil || !strings.Contains(err.Error(), `"c"`) {
t.Errorf("locks outside Dirs: %v", err)
}
// A pull message of the catalogue carries its head's placement files whole (placement candidates, issue 1126).
f = Forge{Kind: ForgePull, Repository: CatalogueRepository, Number: 2, Head: "h", Action: PullUpdated,
Placements: map[string]string{"ace": `{"v":1,"node":"ace","modules":[]}`}, PlacementsRead: true}
if data, err := EncodeForge(f); err != nil {
t.Errorf("a catalogue pull with placement files: %v", err)
} else if got, err := DecodeForge(data); err != nil || !SameForge(got, withV(f)) {
t.Errorf("a catalogue pull's placement files changed on the way: %+v, %v", got, err)
}
for name, f := range map[string]Forge{
"refused manifests": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Manifests: map[string]string{"a": "{}"}},
"refused locks": {Kind: ForgeRefused, Repository: "a/b", Act: "x", Words: "no", Locks: map[string]map[string]string{"a": {"go.sum": "h"}}},
+95
View File
@@ -0,0 +1,95 @@
package delivery
// The own secrets a node's shadow answer finds the vault holds another value for than the node runs (novox/hq issue 1126,
// cause C4): the vault's `move-in` never replaces a value it holds, so a value the controller made anew later (a rotation
// at start, a re-issue) runs while the vault keeps the first one it was handed. The node opens both seals with its own
// key and its answer's shown bodies mask the value as MaskedAnotherValue; this reads which own secrets those are, for the
// node agent (which hands the vault what runs) and for the vault's `own_drift` (which lists them mesh-wide), so both read
// the answer one way.
import (
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// MaskedAnotherValue is the word a shown body masks a value with when the node opened both sides with its key and they
// are two values (MaskedPair, maskWord).
const MaskedAnotherValue = "<masked: another value>"
// OwnValueDiffers is one own secret of a module whose value composed (the vault's) is another than the one that runs.
type OwnValueDiffers struct {
Module string `json:"module"`
Name string `json:"name"`
}
// ownNeedPrefix starts the id of the file a module's own secret is placed in (catalogue.NeedID).
var ownNeedPrefix = catalogue.NeedID("")
// OwnNeedName is the name of module's own secret whose file a resource id names, and whether it names one. On a node a
// resource's id carries its module's name (`only-office.needs-jwt`, as a live shadow answer shows it); the bare form a
// declaration composes (`needs-jwt`, catalogue.NeedID) is read too. An id of another module's is not this module's.
func OwnNeedName(module, id string) (string, bool) {
if module != "" {
id = strings.TrimPrefix(id, module+".")
}
name, own := strings.CutPrefix(id, ownNeedPrefix)
return name, own && name != "" && !strings.Contains(name, ".")
}
// OwnValuesDiffer is every own secret a's shown bodies say holds another value composed than running: a body of a
// module's own secret's file (OwnNeedName: `<module>.needs-<name>` on a node) masked MaskedAnotherValue on either side. Sorted by module and name, each
// once. A body the answer did not show (beyond MaxBodies) is not read.
func OwnValuesDiffer(a CandidateAnswer) []OwnValueDiffers {
seen := map[OwnValueDiffers]bool{}
var out []OwnValueDiffers
for _, d := range a.Differs {
for _, b := range d.Bodies {
name, own := OwnNeedName(d.Module, b.ID)
if !own || !anotherValue(b) {
continue
}
o := OwnValueDiffers{Module: d.Module, Name: name}
if !seen[o] {
seen[o] = true
out = append(out, o)
}
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Module != out[j].Module {
return out[i].Module < out[j].Module
}
return out[i].Name < out[j].Name
})
return out
}
// ValuesDiffer is how many shown bodies of a hold a value masked MaskedAnotherValue, own secrets and every other
// resource alike: the count a node's answer carries as CandidateAnswer.ValuesDiffer.
func ValuesDiffer(a CandidateAnswer) int {
n := 0
for _, d := range a.Differs {
for _, b := range d.Bodies {
if anotherValue(b) {
n++
}
}
}
return n
}
// WithValuesDiffer is answers with each one's ValuesDiffer counted.
func WithValuesDiffer(answers map[string]CandidateAnswer) map[string]CandidateAnswer {
out := make(map[string]CandidateAnswer, len(answers))
for g, a := range answers {
a.ValuesDiffer = ValuesDiffer(a)
out[g] = a
}
return out
}
func anotherValue(b ResourceBodies) bool {
return strings.Contains(b.Composed, MaskedAnotherValue) || strings.Contains(b.Running, MaskedAnotherValue)
}
+49 -53
View File
@@ -3,7 +3,6 @@ package delivery
import (
"encoding/json"
"fmt"
"path"
"sort"
"strconv"
"strings"
@@ -11,29 +10,38 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Placement of modules on nodes lives in the catalogue (novox/hq ADR 0549): one file per node, `placement/<node>.json`
// in the catalogue repository, changed only by a pull request. At each trunk merge of the catalogue the indexer reads
// the files and writes each node's key `placement.<node>` in its state `placement`, only when it differs, and says
// `placement-changed` naming the node, the revision and the modules added, removed or changed (ADR 0549 §2). The node
// agents, the assigner, the grants part and mesh-config read it (§3); nothing in it is secret.
// Placement of modules on nodes is the state of the seat mesh-placement (novox/hq ADR 0567), held once for the mesh by
// the assigner: one key `placement.<node>` per node in its state `placement`, written only on the operator's Approve of a
// proposal the nodes it touches answered (§2-§5), by compare-and-set at the revision the proposal was laid over. No
// repository holds it. The node agents, the grants part, mesh-config and the status view read it (§9); nothing in it is
// secret. Its states are named after the holding module, as every module state is, and these constants are the seat's
// contract: every reader takes the names from here (§1).
const (
// PlacementState is the indexer's state of placements, and PlacementBucket its bucket.
// PlacementState is the seat's state of placements, and PlacementBucket its bucket. PlacementRead is the name a
// reader's manifest declares in its reads.
PlacementState = "placement"
PlacementBucket = IndexerModule + "_" + PlacementState
// PlacementHistory is the revisions the placement bucket keeps per key.
PlacementHistory = 10
// PlacementDir is the catalogue's directory of placement files, one `<node>.json` per node.
PlacementDir = "placement"
// PlacementChangedEvent is what the indexer says when a node's placement key changed.
PlacementChangedEvent = "placement-changed"
PlacementBucket = AssignerModule + "_" + PlacementState
PlacementRead = AssignerModule + "." + PlacementState
// PlacementHistory is the revisions the placement bucket keeps per key: the bus's most (§6).
PlacementHistory = 64
// PlacementStream is the holder's stream of the seat's events, kept PlacementStreamDays (§6, within the mesh's bound
// on a module's stream, catalogue.StreamMostDays), and declared as the holder's data so the mesh's backups keep it.
PlacementStream = "placement"
PlacementStreamDays = 30
// The seat's events, said by its holder as its own (§2, §4, §5, §6), each carrying a PlacementEvent.
PlacementProposedEvent = "placement-proposed"
PlacementWithdrawnEvent = "placement-withdrawn"
PlacementJudgedEvent = "placement-judged"
PlacementChangedEvent = "placement-changed"
placementWord = "placement"
// CatalogueRepository is the repository whose placement files are the mesh's placement (ADR 0549 §1): the forge bridge
// reads them at each of its merges, and no other repository's.
CatalogueRepository = "novox/mesh-catalog"
)
// PlacementEvents are the seat's four events, in the order a proposal says them.
var PlacementEvents = []string{PlacementProposedEvent, PlacementJudgedEvent, PlacementChangedEvent, PlacementWithdrawnEvent}
// PlacementKey is the key, in PlacementBucket, of one node's placement: `placement.<node>`.
func PlacementKey(node string) (string, error) {
if err := checkName("node", node); err != nil {
@@ -51,20 +59,6 @@ func ParsePlacementKey(key string) (node string, err error) {
return node, nil
}
// PlacementFile is the path, in the catalogue repository, of one node's placement file.
func PlacementFile(node string) string { return path.Join(PlacementDir, node+".json") }
// PlacementFileNode is the node a catalogue path is the placement file of, and whether it is one: `placement/<node>.json`
// directly under PlacementDir.
func PlacementFileNode(p string) (string, bool) {
dir, file := path.Split(p)
node, ok := strings.CutSuffix(file, ".json")
if path.Clean(dir) != PlacementDir || !ok || checkName("node", node) != nil {
return "", false
}
return node, true
}
// Provider is the module on a node that answers a requirement or holds a seat: a pin's or a binding's provider.
type Provider struct {
Node string `json:"node"`
@@ -109,8 +103,8 @@ type Holding struct {
Kind string `json:"kind,omitempty"`
}
// Placement is one node's placement: the file the catalogue keeps and, with Revision, the value of its key. Modules are
// in the order the file names them, which is the order the composition reads the node's modules in. Pins is, by
// Placement is one node's placement, the value of its key. Modules are in the order the composition reads the node's
// modules in. Revision is unused since placement left the catalogue (ADR 0567): the key's own revision is the bus's. Pins is, by
// provision, the provider chosen for the node where more than one could answer.
type Placement struct {
V int `json:"v"`
@@ -122,13 +116,13 @@ type Placement struct {
Holds []Holding `json:"holds,omitempty"`
}
// NodeDecided is what a person decides of the node, kept in its placement file (issue 1126's K2: what a person decides
// lives in the catalogue, what the machine observes lives in its record, NodeRecord): where it is dialled (endpoint) and
// NodeDecided is what a person decides of the node, kept in its placement key (issue 1126's K2: what a person decides
// lives in placement, what the machine observes lives in its record, NodeRecord): where it is dialled (endpoint) and
// its site, whether it is the private network's hub, its address on the private network, the domain its routed names
// are composed under, its operator and agent logins by name and their homes, whether it is adopted rather than
// converged, the modules taken on an adopted node, the tunnel found on it at adoption and, on the hub, the names given
// to that tunnel's peers. The address is a decision written once, at enrolment, by a
// catalogue pull request: nothing allocates it at run time, and CheckPlacements refuses two nodes with one address.
// to that tunnel's peers. The address is a decision written once, at enrolment, by an
// approved proposal (ADR 0567 §2): nothing allocates it at run time, and CheckPlacements refuses two nodes with one address.
// No secret and no key material: accounts are names, never credentials.
type NodeDecided struct {
Endpoint string `json:"endpoint,omitempty"`
@@ -211,8 +205,8 @@ func DecodePlacement(data []byte) (Placement, error) {
return p, nil
}
// SamePlacement says whether two placements are the same value, so the indexer writes only when they differ. The
// revision is not compared: a merge that changes no placement writes nothing (ADR 0549 §2).
// SamePlacement says whether two placements are the same value, so the holder writes only when they differ. The
// revision is not compared.
func SamePlacement(a, b Placement) bool {
a.Revision, b.Revision = "", ""
ja, errA := EncodePlacement(a)
@@ -220,21 +214,23 @@ func SamePlacement(a, b Placement) bool {
return errA == nil && errB == nil && string(ja) == string(jb)
}
// PlacementChange is what `placement-changed` says: the node, the catalogue revision, and its modules added, removed
// or changed against the placement before.
// PlacementChange is what `placement-changed` says of one node (ADR 0567 §6): the node, the revision of its key before
// (0: it had none) and after (0: the key was deleted), and its modules added, removed or changed against the placement
// before.
type PlacementChange struct {
Node string `json:"node"`
Revision string `json:"revision"`
Added []string `json:"added,omitempty"`
Removed []string `json:"removed,omitempty"`
Changed []string `json:"changed,omitempty"`
Node string `json:"node"`
Before uint64 `json:"before"`
After uint64 `json:"after"`
Added []string `json:"added,omitempty"`
Removed []string `json:"removed,omitempty"`
Changed []string `json:"changed,omitempty"`
}
// ChangeOf is what changed from before to after on one node; before is the zero Placement for a node new to placement.
// A change of the node's pins or holdings counts as a change of every module placed on it both times, since the
// composition of each may read them.
func ChangeOf(before, after Placement) PlacementChange {
c := PlacementChange{Node: after.Node, Revision: after.Revision}
c := PlacementChange{Node: after.Node}
if c.Node == "" {
c.Node = before.Node
}
@@ -368,8 +364,8 @@ func (p Placement) Held() []catalogue.Held {
return out
}
// CheckPlacements is the catalogue's own check of every placement file (ADR 0549 §1), over the placements by node, the
// modules the catalogue holds and the nodes enrolled. It returns every refusal, in node order, never only the first: a
// CheckPlacements is the holder's check of a proposal laid over every placement (ADR 0567 §2), over the placements by
// node, the modules the indexer holds and the nodes with a record. It returns every refusal, in node order, never only the first: a
// placement naming a node not enrolled or a module the catalogue does not hold; two modules of one node on the same host
// port; a pin, binding or holding naming a provider that is not placed where it says; a seat of scope mesh held twice, unless the seat is replicated (catalogue.Seat's Replicated, ADR 0223), which
// is held once per node;
@@ -401,7 +397,7 @@ func CheckPlacements(placements map[string]Placement, held func(module string) b
for _, node := range nodes {
p := placements[node]
if p.Node != node {
refusals = append(refusals, fmt.Sprintf("the placement file of %s names the node %q", node, p.Node))
refusals = append(refusals, fmt.Sprintf("the placement of %s names the node %q", node, p.Node))
}
if a := strings.TrimSpace(p.Decided.Address); a != "" {
if other, twice := addresses[a]; twice {
@@ -413,12 +409,12 @@ func CheckPlacements(placements map[string]Placement, held func(module string) b
hubs = append(hubs, node)
}
if !enrolled(node) {
refusals = append(refusals, fmt.Sprintf("%s is not an enrolled node", node))
refusals = append(refusals, fmt.Sprintf("%s is a node with no record", node))
}
hostPorts := map[int]string{}
for _, m := range p.Modules {
if !held(m.Module) {
refusals = append(refusals, fmt.Sprintf("%s: the catalogue holds no module %s", node, m.Module))
refusals = append(refusals, fmt.Sprintf("%s: the indexer holds no module %s", node, m.Module))
}
for _, port := range sortedKeys(m.Ports) {
at := m.Ports[port]
+83 -47
View File
@@ -6,70 +6,62 @@ import (
"strings"
)
// Placement candidates (issue 1126; ADR 0549 §4: a placement change passes the composition check before any node sees
// it). A catalogue pull request that changes a placement file carries every placement file at its head on its pull
// message (Forge.Placements). The indexer writes, for each node whose file at the head differs from the trunk's, the
// key `<group>.<node>` of its state `placement-candidates`, and deletes it when the pull request merges or closes. It is
// a state of its own, since the readers of `placement` refuse any key that is not `placement.<node>`.
// Placement candidates (novox/hq ADR 0567 §3, ADR 0559 §4): for each node a proposal must hear from, the seat's holder
// writes one key `<proposal>.<node>` in its state `candidates`: the node's placement with the proposal laid over it for a
// node whose key it changes, or the node's current key for one whose composition it changes only through another node
// (Reached: a binding, a firewall or a resolver entry reaching it). It deletes them when the proposal ends. A state of its
// own, since the readers of `placement` refuse any key that is not `placement.<node>`.
//
// The node it names composes in shadow with the candidate (its own sealed values, the boxes it already runs) and
// compares the result with what it runs; it applies nothing. Its answer rides in its own have key, `<node>.node`
// (HaveNode.Candidates, by group), which that node alone writes; the assigner's check of the group reads every placed
// node's answer at the head and fails on any difference, refusal or silence. Event-driven both ways: the candidate key's
// write is the node's event, the `<node>.node` change the check's.
// The node watches `*.<node>`, composes in shadow with every candidate of its proposal laid over the current placement
// (its own sealed values, the boxes it already runs) and compares the result with what it runs; it applies nothing. Its
// answer rides in its own have key, `<node>.node` (HaveNode.Candidates, by proposal), which that node alone writes; the
// holder reads it and judges (ADR 0567 §4). Event-driven both ways: the candidate key's write is the node's event, the
// `<node>.node` change the holder's.
const (
// PlacementCandidatesState is the indexer's state of placement candidates, and PlacementCandidatesBucket its bucket.
PlacementCandidatesState = "placement-candidates"
PlacementCandidatesBucket = IndexerModule + "_" + PlacementCandidatesState
// PlacementCandidatesState is the seat's state of placement candidates, PlacementCandidatesBucket its bucket and
// PlacementCandidatesRead the name a reader's manifest declares.
PlacementCandidatesState = "candidates"
PlacementCandidatesBucket = AssignerModule + "_" + PlacementCandidatesState
PlacementCandidatesRead = AssignerModule + "." + PlacementCandidatesState
// PlacementCandidatesHistory is the revisions the bucket keeps per key.
PlacementCandidatesHistory = 5
)
// PlacementCandidateKey is the key, in PlacementCandidatesBucket, of one node's candidate placement in one group:
// `<group>.<node>`, the group escaped as CandidatesKey escapes it, so a node watches `*.<node>` and a group's keys are
// `<group>.*`.
func PlacementCandidateKey(group, node string) (string, error) {
g, err := groupKey(group)
if err != nil {
// PlacementCandidateKey is the key, in PlacementCandidatesBucket, of one node's candidate in one proposal:
// `<proposal>.<node>`, so a node watches `*.<node>` and a proposal's keys are `<proposal>.*`.
func PlacementCandidateKey(proposal, node string) (string, error) {
if err := CheckProposalID(proposal); err != nil {
return "", err
}
if err := checkName("node", node); err != nil {
return "", err
}
return g + "." + node, nil
return proposal + "." + node, nil
}
// ParsePlacementCandidateKey is the inverse of PlacementCandidateKey; it refuses any other key, naming it.
func ParsePlacementCandidateKey(key string) (group, node string, err error) {
g, node, ok := strings.Cut(key, ".")
if !ok || checkName("node", node) != nil {
return "", "", fmt.Errorf("delivery: %q is not a placement candidate key, <group>.<node>", key)
func ParsePlacementCandidateKey(key string) (proposal, node string, err error) {
proposal, node, ok := strings.Cut(key, ".")
if !ok || checkName("node", node) != nil || CheckProposalID(proposal) != nil {
return "", "", fmt.Errorf("delivery: %q is not a placement candidate key, <proposal>.<node>", key)
}
if group, err = parseGroupKey(g); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a placement candidate key: %w", key, err)
}
return group, node, nil
return proposal, node, nil
}
// PlacementCandidate is the value of a candidate key: the group, the catalogue pull request and the head its placement
// files were read at, and the node's placement at that head; nil Placement says the head removes the node's file.
// PlacementCandidate is the value of a candidate key: the proposal it is of, and the node's placement with the proposal
// laid over it; nil Placement says the proposal takes the node's key away. Reached says the proposal changes not this
// node's key but its composition, through another node's: Placement is then its current key (nil when it has none).
type PlacementCandidate struct {
V int `json:"v"`
Group string `json:"group"`
Pull Head `json:"pull"`
Proposal string `json:"proposal"`
Reached bool `json:"reached,omitempty"`
Placement *Placement `json:"placement,omitempty"`
}
func (c PlacementCandidate) check() error {
if c.Group == "" {
return fmt.Errorf("delivery: a placement candidate names no group")
}
if err := checkRepository(c.Pull.Repository); err != nil {
if err := CheckProposalID(c.Proposal); err != nil {
return err
}
if c.Pull.Number <= 0 || c.Pull.Head == "" {
return fmt.Errorf("delivery: a placement candidate needs its pull request's number and head")
}
if c.Placement != nil {
return c.Placement.check()
}
@@ -90,8 +82,8 @@ func EncodePlacementCandidate(c PlacementCandidate) ([]byte, error) {
return json.Marshal(c)
}
// DecodePlacementCandidate reads a candidate; it refuses an unknown version, no group, a pull request without number or
// head, and a placement DecodePlacement refuses.
// DecodePlacementCandidate reads a candidate; it refuses an unknown version, a proposal id that is not one, and a
// placement DecodePlacement refuses.
func DecodePlacementCandidate(data []byte) (PlacementCandidate, error) {
var c PlacementCandidate
if err := decode(data, &c); err != nil {
@@ -115,14 +107,13 @@ const (
CandidateRefused = "refused"
)
// CandidateAnswer is a node's answer to its group's candidate keys, in HaveNode.Candidates (ADR 0559 §4): the head
// and the revision of the node's own key in the group it answered (0 when the group has no key of its own for it: its
// file is unchanged and it answers as placed at the head or on the trunk, judged by Head alone), its verdict, the modules composed the same, those that differ with both digests, those
// CandidateAnswer is a node's answer to its proposal's candidate keys, in HaveNode.Candidates (ADR 0567 §3): the
// proposal and the revision of the node's own candidate key it answered, its verdict, the modules composed the same, those that differ with both digests, those
// the candidate adds (naming the values not yet issued for each, which is no refusal: a module's credential and boxes
// are made only once it is placed, ADR 0549 §5) or removes, and each refusal. The node's own composition is named
// "node" (HaveNodeWord).
type CandidateAnswer struct {
Head string `json:"head"`
Proposal string `json:"proposal"`
Revision uint64 `json:"revision"`
Verdict string `json:"verdict"`
Same []string `json:"same,omitempty"`
@@ -132,8 +123,23 @@ type CandidateAnswer struct {
Refusals []ModuleRefused `json:"refusals,omitempty"`
// Refused is why the node could not compose with the candidate at all; Refusals is then empty.
Refused string `json:"refused,omitempty"`
// Returns is, for a proposal that takes the node's placement key away, what the node does then: it returns to the
// controller's compositions (ReturnsToController), as a node with no placement key composes none of its own. Same, Differs, Added and Removed then compare the controller's compositions at
// the node's wants with what it runs, and the verdict is theirs; a want the controller holds no composition for is
// refused (Refused, without Returns).
Returns string `json:"returns,omitempty"`
// Note is what the node says of how it answered, in words, beside the verdict: a pass that ran over a minute, inputs
// it waited for (issue 1126: the shadow shows its work). It changes no verdict.
Note string `json:"note,omitempty"`
// ValuesDiffer is how many shown bodies hold a value the node opened on both sides to two values (ValuesDiffer,
// MaskedAnotherValue): the vault's value and the one that runs drifted apart (issue 1126, cause C4). It changes no
// verdict.
ValuesDiffer int `json:"values_differ,omitempty"`
}
// ReturnsToController is what a node answers a proposal that takes its placement key away.
const ReturnsToController = "the proposal takes this node's placement away: it returns to the controller's compositions"
// ModuleDiffers is one module the node runs and the candidate still places whose composition is not the one it runs:
// both digests (Composed with the candidate, Running what runs), the first resource that differs by its id, or empty
// when every resource is equal, and why in words.
@@ -143,8 +149,34 @@ type ModuleDiffers struct {
Running string `json:"running,omitempty"`
Resource string `json:"resource,omitempty"`
Why string `json:"why"`
// Bodies is every resource that differs, both sides, each value a secret could be in masked (issue 1126: the shadow
// shows its work, so why a module differs is read, not guessed). At most MaxBodies, each side cut at MaxBody bytes.
Bodies []ResourceBodies `json:"bodies,omitempty"`
}
// ResourceBodies is one differing resource of a module: its id, why, and its body composed and as it runs, masked
// (MaskedPair); a side that does not hold it is empty.
type ResourceBodies struct {
ID string `json:"id"`
Why string `json:"why"`
// Paths is every JSON path at which the two bodies differ (DifferingPaths: `sealed`, `content.url`, `links.0`), found
// on the whole bodies before they are cut, a JSON value written in a string read as JSON; a path names keys alone,
// never a value. At most MaxPaths, the last then saying how many more.
Paths []string `json:"paths,omitempty"`
Composed string `json:"composed,omitempty"`
Running string `json:"running,omitempty"`
}
// The bounds of a module's shown bodies, and of one answer as the node's own key holds it: an answer over
// MaxAnswerBytes is fitted (FitAnswer). The node's key (HaveBucket) takes a value of about 1 MiB
// (buscompose.ChainValueBytes) and holds every proposal's answer beside the node's facts, so one answer keeps to a sixth.
const (
MaxBodies = 20
MaxBody = 8192
MaxPaths = 64
MaxAnswerBytes = 160 << 10
)
// ModuleAdded is one module the candidate places on the node and the node does not run: its composition's digest when
// it composed, and the values not yet issued for it (its own secrets, its grants, a host port), each in the words the
// composition stood in for it with.
@@ -173,8 +205,8 @@ func VerdictOf(a CandidateAnswer) string {
}
func (a CandidateAnswer) check() error {
if a.Head == "" {
return fmt.Errorf("delivery: a candidate answer names no head")
if CheckProposalID(a.Proposal) != nil || a.Revision == 0 {
return fmt.Errorf("delivery: a candidate answer names no proposal or no revision of its candidate")
}
switch a.Verdict {
case CandidateSame, CandidateDiffers, CandidateRefused:
@@ -187,6 +219,10 @@ func (a CandidateAnswer) check() error {
if a.Refused != "" && len(a.Same)+len(a.Differs)+len(a.Added)+len(a.Removed)+len(a.Refusals) > 0 {
return fmt.Errorf("delivery: a candidate the node could not compose names modules")
}
if a.Returns != "" && (a.Refused != "" || len(a.Refusals) > 0) {
return fmt.Errorf("delivery: a candidate the node returns to the controller's compositions on is compared, never refused: " +
"a composition the controller holds none of is refused without returning")
}
for _, d := range a.Differs {
if d.Module == "" || d.Why == "" {
return fmt.Errorf("delivery: a candidate difference names no module or no why")
+513
View File
@@ -0,0 +1,513 @@
package delivery
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"regexp"
"sort"
"strings"
"time"
)
// A placement changes only by a proposal (novox/hq ADR 0567 §2): `mesh-placement.propose` takes why and a list of
// changes; the holder lays them over the current keys, names the revision of each key it laid them over, and either
// refuses at once with every finding or writes the proposal under its id in its state `proposals` and a candidate for
// each node it must hear from (PlacementCandidate). The proposal's key leaves the state when it is applied, withdrawn,
// declined or expired; its whole content stays in the event that ends it (PlacementEvent), so no proposal is kept in
// the state once it has ended.
const (
// PlacementProposalsState is the seat's state of open proposals, PlacementProposalsBucket its bucket and
// PlacementProposalsRead the name a reader's manifest declares; a key is the proposal's id.
PlacementProposalsState = "proposals"
PlacementProposalsBucket = AssignerModule + "_" + PlacementProposalsState
PlacementProposalsRead = AssignerModule + "." + PlacementProposalsState
// PlacementProposalsHistory is the revisions the bucket keeps per key.
PlacementProposalsHistory = 5
// PlacementPulsesState is the holder's state of bounds (§4): a key per node a proposal waits on, written with a time
// to live of the answer bound, and a key per raised ask, with its ask's; the server's delete marker when one ends is
// the event that fails a silent node or ends an unanswered ask, so the holder keeps no timer. Its bucket allows a
// time to live per key (catalogue.StateDeclaration.KeyTTL).
PlacementPulsesState = "pulses"
PlacementPulsesBucket = AssignerModule + "_" + PlacementPulsesState
)
// PulseAnswerKey is the key of the bound on one node's answer to a proposal: `answer.<proposal>.<node>`.
func PulseAnswerKey(proposal, node string) (string, error) {
if err := CheckProposalID(proposal); err != nil {
return "", err
}
if err := checkName("node", node); err != nil {
return "", err
}
return "answer." + proposal + "." + node, nil
}
// PulseAskKey is the key of the bound on a proposal's raised ask: `ask.<proposal>`.
func PulseAskKey(proposal string) (string, error) {
if err := CheckProposalID(proposal); err != nil {
return "", err
}
return "ask." + proposal, nil
}
// The operations a proposal's change names (ADR 0567 §2).
const (
OpPlace = "place" // place a module on a node, with its host ports and bindings
OpTakeOff = "take-off" // take a module off a node
OpPin = "pin" // pin the provider of a provision on a node
OpUnpin = "unpin" // remove a pin
OpHold = "hold" // a module placed on the node holds a seat
OpUnhold = "unhold" // it holds it no more
OpDecide = "decide" // set a decided field of the node (NodeDecided by its JSON name), its address among them
OpConfig = "config" // set a module's config placement or values on the node
OpSeed = "seed" // the node's whole placement: the take-over (adopt, §7), never given by a caller
)
// The states of an open proposal.
const (
// ProposalAnswering: its candidates are written and the nodes it must hear from have not all answered.
ProposalAnswering = "answering"
// ProposalFailed: its verdict failed; it stays open, for its proposer to read and withdraw, or for a node's later
// answer to judge it again.
ProposalFailed = "failed"
// ProposalAsked: its verdict passed and the operator is asked to approve it.
ProposalAsked = "asked"
)
// VerdictWaits is a proposal's verdict while a node it must hear from has not answered within the bound; it passes
// (VerdictPass) or fails (VerdictFail) as a check's verdict does (ADR 0567 §4).
const VerdictWaits = "waits"
// PlacementProposalChange is one change of a proposal. Node is always named; the rest is the operation's.
type PlacementProposalChange struct {
Op string `json:"op"`
Node string `json:"node"`
Module string `json:"module,omitempty"`
Ports map[string]int `json:"ports,omitempty"`
Bindings map[string]Provider `json:"bindings,omitempty"`
Provision string `json:"provision,omitempty"`
Provider *Provider `json:"provider,omitempty"`
// Claim, Scope, Kind and Site are a holding's (OpHold, OpUnhold).
Claim string `json:"claim,omitempty"`
Scope string `json:"scope,omitempty"`
Kind string `json:"kind,omitempty"`
Site string `json:"site,omitempty"`
// Field and Value are a decided field's (OpDecide): its JSON name in NodeDecided and its value.
Field string `json:"field,omitempty"`
Value json.RawMessage `json:"value,omitempty"`
// MeshWide, Layer, MeshValues and Values are a module's config placement and values (OpConfig), as PlacedModule
// carries them; each given replaces what the module has.
MeshWide map[string]any `json:"mesh-wide,omitempty"`
Layer map[string]any `json:"layer,omitempty"`
MeshValues map[string]any `json:"mesh-wide-values,omitempty"`
Values map[string]any `json:"values,omitempty"`
// Whole is the node's whole placement (OpSeed).
Whole *Placement `json:"whole,omitempty"`
}
// BeforeLease is why the change cannot be proposed before the lease write, or "" (ADR 0567 §8): until then the holder
// keeps the controller's assignments in step through its verbs assign, unassign, pin and unpin, so only those are
// proposable — a module placed or taken off with no host port or binding, a pin set or removed, a module's config values
// and the seed. A host port, a binding, a holding, a decided field and a config layer's places or accesses wait for the
// lease write, which the refusal names.
func (c PlacementProposalChange) BeforeLease() string {
const lease = "which the controller's verbs cannot take from a caller, so it waits for the lease write (ADR 0567 §8)"
switch c.Op {
case OpPlace:
if len(c.Ports)+len(c.Bindings) > 0 {
return fmt.Sprintf("the place of %s on %s names host ports or bindings, %s", c.Module, c.Node, lease)
}
case OpTakeOff, OpPin, OpUnpin, OpSeed:
case OpConfig:
// A module's config values are the state's alone (§13); its layers' places and accesses are the controller's
// until the lease write.
if c.MeshWide != nil || c.Layer != nil {
return fmt.Sprintf("the config of %s on %s sets a layer's places or accesses, %s", c.Module, c.Node, lease)
}
default:
return fmt.Sprintf("a change %s of %s is no assignment or pin, %s", c.Op, c.Node, lease)
}
return ""
}
func (c PlacementProposalChange) check() error {
if err := checkName("node", c.Node); err != nil {
return err
}
needs := func(what string, ok bool) error {
if !ok {
return fmt.Errorf("delivery: a %s change of %s names no %s", c.Op, c.Node, what)
}
return nil
}
switch c.Op {
case OpPlace, OpTakeOff, OpConfig:
return needs("module", checkName("module", c.Module) == nil)
case OpPin:
return needs("provision or provider", c.Provision != "" && c.Provider != nil && c.Provider.Node != "" && c.Provider.Module != "")
case OpUnpin:
return needs("provision", c.Provision != "")
case OpHold, OpUnhold:
return needs("module or claim", checkName("module", c.Module) == nil && c.Claim != "")
case OpDecide:
return needs("field or value", c.Field != "" && len(c.Value) > 0)
case OpSeed:
if c.Whole == nil || c.Whole.Node != c.Node {
return fmt.Errorf("delivery: the seed of %s names no placement of that node", c.Node)
}
return c.Whole.check()
}
return fmt.Errorf("delivery: %q is no change of placement: place, take-off, pin, unpin, hold, unhold, decide or config", c.Op)
}
// LayChanges is the placements of the nodes the changes name with the changes laid over current, in the order given;
// current is every node's placement now (absent: the node has no key). It returns, by node, the laid placement, and
// every finding: a change that does not fit its node (a module taken off that is not placed, configured or holding
// there; a field NodeDecided does not have) or a placement DecodePlacement refuses. What CheckPlacements refuses
// across nodes is the caller's to ask of the result laid over current.
func LayChanges(current map[string]Placement, changes []PlacementProposalChange) (map[string]Placement, []string) {
laid := map[string]Placement{}
var findings []string
of := func(node string) Placement {
if p, ok := laid[node]; ok {
return p
}
p, ok := current[node]
if !ok {
return Placement{V: Version, Node: node, Modules: []PlacedModule{}}
}
return clonePlacement(p)
}
for i, c := range changes {
if err := c.check(); err != nil {
findings = append(findings, fmt.Sprintf("change %d: %v", i+1, err))
continue
}
p := of(c.Node)
at := -1
for j, m := range p.Modules {
if m.Module == c.Module {
at = j
}
}
notPlaced := func() {
findings = append(findings, fmt.Sprintf("change %d: %s is not placed on %s", i+1, c.Module, c.Node))
}
switch c.Op {
case OpSeed:
p = clonePlacement(*c.Whole)
case OpPlace:
m := PlacedModule{Module: c.Module, Ports: c.Ports, Bindings: c.Bindings}
if at >= 0 {
m.MeshWide, m.HasMesh, m.Layer, m.HasLayer = p.Modules[at].MeshWide, p.Modules[at].HasMesh, p.Modules[at].Layer, p.Modules[at].HasLayer
m.MeshValues, m.Values = p.Modules[at].MeshValues, p.Modules[at].Values
p.Modules[at] = m
} else {
p.Modules = append(p.Modules, m)
}
case OpTakeOff:
if at < 0 {
notPlaced()
continue
}
p.Modules = append(p.Modules[:at:at], p.Modules[at+1:]...)
var holds []Holding
for _, h := range p.Holds {
if h.Module != c.Module {
holds = append(holds, h)
}
}
p.Holds = holds
case OpConfig:
if at < 0 {
notPlaced()
continue
}
m := &p.Modules[at]
if c.MeshWide != nil {
m.MeshWide, m.HasMesh = c.MeshWide, true
}
if c.Layer != nil {
m.Layer, m.HasLayer = c.Layer, true
}
if c.MeshValues != nil {
m.MeshValues = c.MeshValues
}
if c.Values != nil {
m.Values = c.Values
}
case OpPin:
if p.Pins == nil {
p.Pins = map[string]Provider{}
}
p.Pins[c.Provision] = *c.Provider
case OpUnpin:
delete(p.Pins, c.Provision)
if len(p.Pins) == 0 {
p.Pins = nil
}
case OpHold:
if at < 0 {
notPlaced()
continue
}
p.Holds = append(p.Holds, Holding{Claim: c.Claim, Scope: c.Scope, Module: c.Module, Site: c.Site, Kind: c.Kind})
case OpUnhold:
kept := p.Holds[:0:0]
for _, h := range p.Holds {
if h.Module != c.Module || h.Claim != c.Claim {
kept = append(kept, h)
}
}
if len(kept) == len(p.Holds) {
findings = append(findings, fmt.Sprintf("change %d: %s holds no %s on %s", i+1, c.Module, c.Claim, c.Node))
continue
}
p.Holds = kept
case OpDecide:
d, err := decide(p.Decided, c.Field, c.Value)
if err != nil {
findings = append(findings, fmt.Sprintf("change %d: %v", i+1, err))
continue
}
p.Decided = d
}
laid[c.Node] = p
}
for _, node := range sortedKeys(laid) {
if err := laid[node].check(); err != nil {
findings = append(findings, err.Error())
}
}
return laid, findings
}
// decide is d with the field named by its JSON name set to value, refused when NodeDecided has no such field or the
// value does not fit it.
func decide(d NodeDecided, field string, value json.RawMessage) (NodeDecided, error) {
raw, _ := json.Marshal(d)
var fields map[string]json.RawMessage
_ = json.Unmarshal(raw, &fields)
if fields == nil {
fields = map[string]json.RawMessage{}
}
if !decidedFields[field] {
return d, fmt.Errorf("a node has no decided field %q", field)
}
fields[field] = value
raw, _ = json.Marshal(fields)
var out NodeDecided
dec := json.NewDecoder(strings.NewReader(string(raw)))
dec.DisallowUnknownFields()
if err := dec.Decode(&out); err != nil {
return d, fmt.Errorf("the decided field %q does not take %s: %v", field, value, err)
}
return out, nil
}
// decidedFields are NodeDecided's fields by their JSON names.
var decidedFields = func() map[string]bool {
raw, _ := json.Marshal(NodeDecided{Endpoint: "x", Site: "x", Hub: true, Address: "x", PublicDomain: "x", Account: "x",
AccountHome: "x", AgentAccount: "x", AgentHome: "x", Adopted: true, Taken: []string{"x"}, Tunnel: &RecordTunnel{},
PeerNames: map[string]string{"x": "x"}})
var m map[string]any
_ = json.Unmarshal(raw, &m)
out := map[string]bool{}
for k := range m {
out[k] = true
}
return out
}()
func clonePlacement(p Placement) Placement {
raw, _ := json.Marshal(p)
var out Placement
_ = json.Unmarshal(raw, &out)
if out.Modules == nil {
out.Modules = []PlacedModule{}
}
return out
}
var proposalIDOK = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{2,62}$`)
// CheckProposalID refuses an id that is no proposal's: lower-case letters, digits and dashes, 3 to 63 of them, so it is
// one key token.
func CheckProposalID(id string) error {
if !proposalIDOK.MatchString(id) {
return fmt.Errorf("delivery: %q is no proposal id (lower-case letters, digits and dashes, one key token)", id)
}
return nil
}
// NewProposalID is a fresh proposal id: its time in UTC and six random hex digits, `p-20261028t183000-1a2b3c`.
func NewProposalID(at time.Time) string {
var b [3]byte
_, _ = rand.Read(b[:])
return "p-" + strings.ToLower(at.UTC().Format("20060102T150405")) + "-" + hex.EncodeToString(b[:])
}
// PlacementVerdict is the holder's judgement of a proposal's answers (ADR 0567 §4). It fails only on a refusal by a
// node that must answer or on its silence past the bound (and, for the seed alone, on any difference); every difference
// is shown by node: Intended is what the holder computed the proposal changes, Unintended what a node answered beside
// it, marked as such in the brief.
type PlacementVerdict struct {
State string `json:"state"`
At time.Time `json:"at"`
Fails []string `json:"fails,omitempty"`
Silent []string `json:"silent,omitempty"`
Until time.Time `json:"until,omitempty"`
Intended map[string][]string `json:"intended,omitempty"`
Unintended map[string][]string `json:"unintended,omitempty"`
Answers map[string]CandidateAnswer `json:"answers,omitempty"`
Notes []string `json:"notes,omitempty"`
}
// PlacementProposal is an open proposal, the value of its key in PlacementProposalsBucket.
type PlacementProposal struct {
V int `json:"v"`
ID string `json:"id"`
By string `json:"by"`
Why string `json:"why"`
// At is when it was proposed, Written when its newest candidate was written (the answers' bound runs from it).
At time.Time `json:"at"`
Written time.Time `json:"written,omitempty"`
// Changes are as proposed; Seed says they are the take-over (adopt, §7), ExportDigest the digest of the export it
// proposed (ExportDigest), checked again at the warrant.
Changes []PlacementProposalChange `json:"changes"`
Seed bool `json:"seed,omitempty"`
ExportDigest string `json:"export-digest,omitempty"`
// Base is, by node whose key it changes, the revision of the key it was laid over (0: the node had none).
Base map[string]uint64 `json:"base"`
// Must is every node it must hear from (§3): each whose key it changes and each whose composition it changes
// through another's, sorted; set by the holder once it composed, before then the nodes of Base.
Must []string `json:"must,omitempty"`
// HandBack says it is the operator's hand-back (§7): every key of Base deleted on its warrant, with no change and no
// node's answer.
HandBack bool `json:"hand-back,omitempty"`
// State is ProposalAnswering, ProposalFailed or ProposalAsked; Verdict is the latest judgement, Ask the authorising
// ask raised on a passing one.
State string `json:"state"`
Verdict *PlacementVerdict `json:"verdict,omitempty"`
Ask string `json:"ask,omitempty"`
// Words is what happened last, in words (laid again over moved keys, asked again).
Words string `json:"words,omitempty"`
// Again is when `again` asked the holder to judge it now and raise its ask again (§10); the holder does once per
// value.
Again time.Time `json:"again,omitempty"`
// Withdrawn is who withdrew it and why (`withdraw`, §2): the holder then deletes it and its candidates and says
// placement-withdrawn, which only the holder's process can say on its stream.
Withdrawn string `json:"withdrawn,omitempty"`
}
// Nodes is every node whose key the proposal changes, sorted.
func (p PlacementProposal) Nodes() []string { return sortedKeys(p.Base) }
func (p PlacementProposal) check() error {
if err := CheckProposalID(p.ID); err != nil {
return err
}
if strings.TrimSpace(p.By) == "" || strings.TrimSpace(p.Why) == "" {
return fmt.Errorf("delivery: the proposal %s names no proposer or no why", p.ID)
}
if len(p.Base) == 0 || (len(p.Changes) == 0) != p.HandBack || (p.HandBack && p.Seed) {
return fmt.Errorf("delivery: the proposal %s changes nothing, or is a hand-back with changes", p.ID)
}
for i, c := range p.Changes {
if err := c.check(); err != nil {
return fmt.Errorf("delivery: the proposal %s, change %d: %w", p.ID, i+1, err)
}
if (c.Op == OpSeed) != p.Seed {
return fmt.Errorf("delivery: the proposal %s mixes the seed with other changes", p.ID)
}
if _, ok := p.Base[c.Node]; !ok {
return fmt.Errorf("delivery: the proposal %s changes %s and names no base revision of its key", p.ID, c.Node)
}
}
switch p.State {
case ProposalAnswering, ProposalFailed:
case ProposalAsked:
if p.Ask == "" || p.Verdict == nil || p.Verdict.State != VerdictPass {
return fmt.Errorf("delivery: the proposal %s is asked with no ask or no passing verdict", p.ID)
}
default:
return fmt.Errorf("delivery: the proposal %s is in no state %q", p.ID, p.State)
}
if p.Seed && p.ExportDigest == "" {
return fmt.Errorf("delivery: the seed %s names no export digest", p.ID)
}
return nil
}
// EncodePlacementProposal is p as JSON at Version; it refuses what DecodePlacementProposal refuses.
func EncodePlacementProposal(p PlacementProposal) ([]byte, error) {
if err := p.check(); err != nil {
return nil, err
}
p.V = Version
return json.Marshal(p)
}
// DecodePlacementProposal reads a proposal; it refuses an unknown version, an id that is none, no proposer or why, no
// change, a change it does not take or of a node with no base revision, the seed mixed with other changes or without its
// export digest, and a state it does not know.
func DecodePlacementProposal(data []byte) (PlacementProposal, error) {
var p PlacementProposal
if err := decode(data, &p); err != nil {
return PlacementProposal{}, err
}
if err := p.check(); err != nil {
return PlacementProposal{}, err
}
return p, nil
}
// ExportDigest is the digest of an export's answer, every node's placement, as the seed proposes it: the same
// placements give the same digest, whatever order they came in.
func ExportDigest(placements map[string]Placement) string {
h := sha256.New()
for _, node := range sortedKeys(placements) {
raw, _ := EncodePlacement(placements[node])
fmt.Fprintf(h, "%s\n%s\n", node, raw)
}
return "sha256:" + hex.EncodeToString(h.Sum(nil))
}
// ChangesDigest is the digest of a proposal's changes, which its authorising ask is bound to (§5).
func ChangesDigest(p PlacementProposal) string {
raw, _ := json.Marshal(struct {
ID string `json:"id"`
Changes []PlacementProposalChange `json:"changes"`
Base map[string]uint64 `json:"base"`
}{p.ID, p.Changes, p.Base})
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:])
}
// PlacementWarrant is who approved a proposal, through which channel and when (§5, §6): the warrant's id as the teller
// wrote it (the ask's id and when), and its words, which name the person and the channel that proved them
// (asks.Warrant.Says).
type PlacementWarrant struct {
Warrant string `json:"warrant"`
Words string `json:"words"`
At time.Time `json:"at"`
}
// PlacementEvent is what each of the seat's events carries (§6): the proposal whole (its id, by, why, changes, base
// revisions and verdict with every node's answer); for placement-changed, by node the revisions before and after and
// the modules added, removed or changed, and the warrant; for placement-withdrawn, why it ended.
type PlacementEvent struct {
Proposal PlacementProposal `json:"proposal"`
Nodes []PlacementChange `json:"nodes,omitempty"`
Warrant *PlacementWarrant `json:"warrant,omitempty"`
Ended string `json:"ended,omitempty"`
}
// SortChanges orders a placement-changed event's nodes by node.
func SortChanges(c []PlacementChange) {
sort.Slice(c, func(i, j int) bool { return c[i].Node < c[j].Node })
}
+321 -43
View File
@@ -1,11 +1,15 @@
package delivery
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"reflect"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -29,15 +33,18 @@ func TestPlacement(t *testing.T) {
if _, err := PlacementKey("a.b"); err == nil {
t.Error("PlacementKey took a dotted node")
}
if PlacementBucket != "mesh-delivery-indexer_placement" || PlacementFile("ace") != "placement/ace.json" {
t.Fatalf("bucket %q, file %q", PlacementBucket, PlacementFile("ace"))
}
for p, want := range map[string]string{"placement/ace.json": "ace", "./placement/ace.json": "ace",
"placement/x/ace.json": "", "modules/ace.json": "", "placement/ace.yaml": ""} {
if n, ok := PlacementFileNode(p); n != want || ok != (want != "") {
t.Errorf("PlacementFileNode(%q) = %q, %v", p, n, ok)
// ADR 0567 §1: the seat's three states are named after the holding module, and every reader takes the names here.
for got, want := range map[string]string{PlacementBucket: "mesh-delivery-assigner_placement",
PlacementProposalsBucket: "mesh-delivery-assigner_proposals", PlacementCandidatesBucket: "mesh-delivery-assigner_candidates",
PlacementRead: "mesh-delivery-assigner.placement", PlacementCandidatesRead: "mesh-delivery-assigner.candidates",
PlacementProposalsRead: "mesh-delivery-assigner.proposals"} {
if got != want {
t.Errorf("a bucket is named %q, want %q", got, want)
}
}
if PlacementHistory != 64 || PlacementStreamDays != 30 {
t.Errorf("history %d, stream %d days", PlacementHistory, PlacementStreamDays)
}
})
g14 := Placement{Node: "g14", Revision: "abc",
@@ -126,10 +133,9 @@ func TestPlacement(t *testing.T) {
t.Run("what a change says", func(t *testing.T) {
after := g14
after.Revision = "def"
after.Modules = []PlacedModule{{Module: "app", Ports: map[string]int{"8080": 18081}}, {Module: "new"}}
got := ChangeOf(g14, after)
want := PlacementChange{Node: "g14", Revision: "def", Added: []string{"new"}, Removed: []string{"zsh"}, Changed: []string{"app"}}
want := PlacementChange{Node: "g14", Added: []string{"new"}, Removed: []string{"zsh"}, Changed: []string{"app"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("ChangeOf = %+v, want %+v", got, want)
}
@@ -168,7 +174,7 @@ func TestPlacement(t *testing.T) {
twice.Decided.Address = "10.77.0.2"
r := CheckPlacements(map[string]Placement{"g14": clash, "novox": novox, "stranger": twice}, held, enrolled)
for _, want := range []string{"holds no module ghost", "ghost and app are both published on host port 18080",
"bound to ace/postgres", "pin of shell names g14/bash", "stranger is not an enrolled node",
"bound to ace/postgres", "pin of shell names g14/bash", "stranger is a node with no record",
"mesh-store, held once for the mesh, is held on novox and stranger",
"g14 and stranger both have the address 10.77.0.2", "novox, stranger are each placed as it"} {
found := false
@@ -181,67 +187,79 @@ func TestPlacement(t *testing.T) {
}
})
// Issue 1126: a catalogue pull request's placement, one candidate key per node, and each node's answer in its have.
// ADR 0567 §3: a proposal's candidates, one key `<proposal>.<node>` per node it must hear from, and each node's answer
// in its have.
t.Run("candidates", func(t *testing.T) {
key, err := PlacementCandidateKey("feat/1.2", "g14")
if err != nil || key != "feat/1=2e2.g14" {
const id = "p-20261028t183000-1a2b3c"
key, err := PlacementCandidateKey(id, "g14")
if err != nil || key != id+".g14" {
t.Fatalf("PlacementCandidateKey = %q, %v", key, err)
}
if g, n, err := ParsePlacementCandidateKey(key); err != nil || g != "feat/1.2" || n != "g14" {
t.Fatalf("ParsePlacementCandidateKey = %q, %q, %v", g, n, err)
if p, n, err := ParsePlacementCandidateKey(key); err != nil || p != id || n != "g14" {
t.Fatalf("ParsePlacementCandidateKey = %q, %q, %v", p, n, err)
}
for _, bad := range []string{"g14", ".g14", "feat.g.14", "feat.", "fe=t.g14"} {
for _, bad := range []string{"g14", ".g14", "p-x.g.14", "p-x.", "P-X.g14", "feat/1.g14"} {
if _, _, err := ParsePlacementCandidateKey(bad); err == nil {
t.Errorf("ParsePlacementCandidateKey(%q) took it", bad)
}
}
if PlacementCandidatesBucket != "mesh-delivery-indexer_placement-candidates" {
t.Errorf("bucket %q", PlacementCandidatesBucket)
}
pull := Head{Repository: CatalogueRepository, Number: 7, Head: "h1"}
for name, c := range map[string]PlacementCandidate{
"placed": {Group: "feat/x", Pull: pull, Placement: &g14},
"removed": {Group: "feat/x", Pull: pull},
"placed": {Proposal: id, Placement: &g14},
"removed": {Proposal: id},
"reached": {Proposal: id, Reached: true, Placement: &g14},
} {
raw, err := EncodePlacementCandidate(c)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
back, err := DecodePlacementCandidate(raw)
if err != nil || back.Group != c.Group || back.Pull != pull || (back.Placement == nil) != (c.Placement == nil) ||
if err != nil || back.Proposal != id || back.Reached != c.Reached || (back.Placement == nil) != (c.Placement == nil) ||
(c.Placement != nil && !SamePlacement(*back.Placement, g14)) {
t.Errorf("%s: round trip %+v, %v", name, back, err)
}
}
for name, c := range map[string]PlacementCandidate{
"no group": {Pull: pull},
"no head": {Group: "g", Pull: Head{Repository: CatalogueRepository, Number: 7}},
"bad place": {Group: "g", Pull: pull, Placement: &Placement{Node: "a.b"}},
"no repo form": {Group: "g", Pull: Head{Repository: "mesh-catalog", Number: 7, Head: "h"}},
"no proposal": {},
"a group": {Proposal: "feat/x"},
"bad place": {Proposal: id, Placement: &Placement{Node: "a.b"}},
} {
if _, err := EncodePlacementCandidate(c); err == nil {
t.Errorf("%s: taken", name)
}
}
answers := map[string]CandidateAnswer{
"same": {Head: "h1", Revision: 3, Verdict: CandidateSame, Same: []string{"app", "node"}},
"differs": {Head: "h1", Revision: 3, Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Module: "app", Resource: "file:/etc/app", Why: "the resource differs"}}},
"refused": {Head: "h1", Revision: 3, Verdict: CandidateRefused, Refused: "unbound postgres"},
"added": {Head: "h1", Verdict: CandidateDiffers, Same: []string{"node"}, Added: []ModuleAdded{{Module: "app", Unissued: []string{"app/token"}}}, Removed: []string{"old"}},
"a value": {Head: "h1", Verdict: CandidateRefused, Same: []string{"node"}, Refusals: []ModuleRefused{{Module: "db", Why: "stands in for db/password"}}},
"same": {Proposal: id, Revision: 3, Verdict: CandidateSame, Same: []string{"app", "node"}},
"differs": {Proposal: id, Revision: 3, Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Module: "app", Resource: "file:/etc/app", Why: "the resource differs"}}},
"refused": {Proposal: id, Revision: 3, Verdict: CandidateRefused, Refused: "unbound postgres"},
"added": {Proposal: id, Revision: 1, Verdict: CandidateDiffers, Same: []string{"node"}, Added: []ModuleAdded{{Module: "app", Unissued: []string{"app/token"}}}, Removed: []string{"old"}},
"a value": {Proposal: id, Revision: 1, Verdict: CandidateRefused, Same: []string{"node"}, Refusals: []ModuleRefused{{Module: "db", Why: "stands in for db/password"}}},
// A head that deletes the node's placement file: the node returns to the controller's compositions (issue 1126,
// the seed's revert, mesh-catalog #526).
"returns": {Proposal: id, Revision: 1, Verdict: CandidateSame, Returns: ReturnsToController, Same: []string{"app", "node"}},
"returns, the controller's differs": {Proposal: id, Revision: 1, Verdict: CandidateDiffers, Returns: ReturnsToController, Same: []string{"node"},
Differs: []ModuleDiffers{{Module: "app", Resource: "file:/etc/app", Why: "the resource differs"}}, Removed: []string{"old"}},
}
n := HaveNode{Composed: "sha256:" + strings.Repeat("a", 64), Sealed: []string{"box"}, Candidates: answers}
n := HaveNode{Composed: "sha256:" + strings.Repeat("a", 64), Sealed: []string{"box"}, Candidates: answers,
Composes: ComposesOwn, ComposesWhy: "the catalogue places it (placement.n1)"}
raw, err := EncodeHaveNode(n)
if err != nil {
t.Fatal(err)
}
back, err := DecodeHaveNode(raw)
if err != nil || !SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Candidates: answers}) {
if err != nil || !SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Candidates: answers,
Composes: ComposesOwn, ComposesWhy: n.ComposesWhy}) {
t.Fatalf("a node's answers changed on the way: %+v, %v", back, err)
}
if SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed}) {
if SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Composes: ComposesOwn, ComposesWhy: n.ComposesWhy}) {
t.Error("a node's have without its answers counts as the same")
}
if SameHaveNode(back, HaveNode{V: Version, Composed: n.Composed, Sealed: n.Sealed, Candidates: answers,
Composes: ComposesController, ComposesWhy: n.ComposesWhy}) {
t.Error("a node's have composing otherwise counts as the same")
}
if _, err := EncodeHaveNode(HaveNode{Composes: "both"}); err == nil {
t.Error("a node composing neither its own nor the controller's was taken")
}
// CompareComposed: the bus's user list is left out on both sides; anything else that differs is named.
file := func(id, content string) json.RawMessage {
return json.RawMessage(`{"id":"` + id + `","type":"file","module":"nats","path":"/etc/` + id + `","mode":"0600","content":"` + content + `"}`)
@@ -269,6 +287,155 @@ func TestPlacement(t *testing.T) {
t.Errorf("%s: differs %+v, want %s: %s with both digests", name, c.Differs, tc.resource, tc.why)
}
}
// Issue 1126, the shadow shows its work: every differing resource with both bodies, each value a secret could be
// in masked (never a secret in an answer), saying whether the two sides hold the same value.
sealedFile := func(id, sealed, mode string) json.RawMessage {
return json.RawMessage(`{"id":"` + id + `","type":"file","module":"nats","path":"/etc/` + id + `","mode":"` + mode + `","sealed":"` + sealed + `"}`)
}
// Issue 1126, C1: a sealed value is judged by the value it opens to at its own slot: b sealed again (box-2 and
// box-3 open to one value) is no difference; d holds at its slot a box of another value, though that value is
// the one b holds, and differs at its path `sealed`.
sameBox := func(a, b string) (same, known bool) {
return (a == "box-2" && b == "box-3") || a == b, true
}
shown := CompareComposedWith(map[string]Composed{"k": comp(sealedFile("a", "box-1", "0600"), sealedFile("b", "box-2", "0600"),
file("c", "x"), sealedFile("d", "box-2", "0600"))},
[]Composed{comp(sealedFile("a", "box-1", "0644"), sealedFile("b", "box-3", "0600"), file("c", "y"), sealedFile("d", "box-4", "0600"))},
sameBox)
if len(shown.Differs) != 1 || len(shown.Differs[0].Bodies) != 3 {
t.Fatalf("every differing resource is not shown, or b sealed again is shown: %+v", shown.Differs)
}
if c := CompareComposedWith(map[string]Composed{"k": comp(sealedFile("b", "box-2", "0600"))},
[]Composed{comp(sealedFile("b", "box-3", "0600"))}, sameBox); len(c.Same) != 1 {
t.Errorf("a value sealed again at its own slot is not the same: %+v", c)
}
for _, row := range []struct {
id, has, hasNot string
paths []string
}{
{"a", `"mode":"0600"`, "box-1", []string{"mode"}},
{"c", `"content":"x"`, "", []string{"content"}},
{"d", "<masked: another value>", "box-2", []string{"sealed"}},
} {
var body ResourceBodies
for _, b := range shown.Differs[0].Bodies {
if b.ID == row.id {
body = b
}
}
both := body.Composed + body.Running
if !strings.Contains(both, row.has) || (row.hasNot != "" && strings.Contains(both, row.hasNot)) ||
!slices.Equal(body.Paths, row.paths) {
t.Errorf("%s shown as %+v, want paths %v", row.id, body, row.paths)
}
}
// Issue 1126, C1, the seed-7 probe: a file naming secrets carries each sealed value in its `secrets`, and each entry
// is a slot of its own. The shape is intoFile's output, captured from catalogue for a file naming two secrets
// (2026-10-28), with the module a composition names each resource of; only the boxes are replaced.
withSecrets := func(db, key string) json.RawMessage {
return json.RawMessage(`{"content":"[database]\npassword = ${secret:postgres-database}\n[security]\nsecret_key = ${secret:secret-key}\n",` +
`"id":"grafana.ini","mode":"0600","module":"nats","path":"/etc/grafana/grafana.ini","secrets":{"postgres-database":"` + db +
`","secret-key":"` + key + `"},"type":"file"}`)
}
if c := CompareComposedWith(map[string]Composed{"k": comp(withSecrets("box-2", "box-1"))},
[]Composed{comp(withSecrets("box-3", "box-1"))}, sameBox); len(c.Same) != 1 {
t.Errorf("a secret sealed again at its own entry is not the same: %+v", c)
}
if kept := KeepSeals(comp(withSecrets("box-2", "box-1")), SealsIn(comp(withSecrets("box-3", "box-1"))), sameBox); !bytes.Equal(kept.Resources[0], withSecrets("box-3", "box-1")) {
t.Errorf("the box that runs is not kept byte for byte: %s", kept.Resources[0])
}
if s := SealsIn(comp(withSecrets("box-2", "box-1"))); s[SealSlot("grafana.ini", SecretSlotField("postgres-database"))] != "box-2" || len(s) != 2 {
t.Errorf("the secrets' slots are %v", s)
}
// Another value at one entry differs there alone, each entry masked by what it opens to.
// box-2 and box-4 open to one value, so the box that runs is kept there; box-5 and box-6 are two values.
mixed := CompareComposedWith(map[string]Composed{"k": comp(withSecrets("box-2", "box-5"))},
[]Composed{comp(withSecrets("box-4", "box-6"))}, func(a, b string) (bool, bool) {
return (a == "box-2" && b == "box-4") || a == b, true
})
if len(mixed.Differs) != 1 || len(mixed.Differs[0].Bodies) != 1 {
t.Fatalf("a secret of another value is not shown: %+v", mixed)
}
if body := mixed.Differs[0].Bodies[0]; !slices.Equal(body.Paths, []string{"secrets.secret-key"}) ||
!strings.Contains(body.Composed, `"secret-key":"<masked: another value>"`) || strings.Contains(body.Composed+body.Running, "box-") ||
!strings.Contains(body.Running, `"postgres-database":"<masked: the same on both sides>"`) {
t.Errorf("a file's secrets shown as %+v", body)
}
if c, _ := MaskedPair(withSecrets("box-2", "box-5"), withSecrets("box-4", "box-6"), func(a, b string) (bool, bool) {
return a == "box-2", true
}); !strings.Contains(c, `"postgres-database":"<masked: another seal of the same value>"`) || !strings.Contains(c, `"secret-key":"<masked: another value>"`) {
t.Errorf("each secret is not masked by what it opens to: %s", c)
}
// Only a file's secrets are split by entry: a vault box (an object under `vault-sealed`) is one masked word and one
// path, judged whole by what it opens to (rev-seed-same-3, round 1).
box := func(ct string) json.RawMessage {
return json.RawMessage(`{"id":"core-box","type":"file","module":"nats","path":"/etc/core","mode":"0600","vault-sealed":{"v":1,"eph":"e-` +
ct + `","ct":"` + ct + `"}}`)
}
vault := CompareComposedWith(map[string]Composed{"k": comp(box("c1"))}, []Composed{comp(box("c2"))},
func(a, b string) (bool, bool) {
return false, strings.Contains(a, `"ct":"c1"`) && strings.Contains(b, `"ct":"c2"`)
})
if len(vault.Differs) != 1 || len(vault.Differs[0].Bodies) != 1 {
t.Fatalf("a box of another value is not shown: %+v", vault)
}
if body := vault.Differs[0].Bodies[0]; !slices.Equal(body.Paths, []string{"vault-sealed"}) ||
!strings.Contains(body.Composed, `"vault-sealed":"<masked: another value>"`) || strings.Contains(body.Composed+body.Running, "c1") {
t.Errorf("a vault box shown as %+v", body)
}
// Issue 1126, T1: the paths are found on the whole bodies, before they are cut; JSON in a string is walked into, a
// key a secret could be under is named and never shown.
long := strings.Repeat("x", MaxBody)
conf := func(link, password string) json.RawMessage {
inner, _ := json.Marshal(map[string]any{"pad": long, "links": []string{"a", link}, "password": password})
outer, _ := json.Marshal(map[string]any{"id": "conf", "type": "file", "module": "nats", "path": "/etc/conf",
"mode": "0600", "content": string(inner)})
return outer
}
far := CompareComposedWith(map[string]Composed{"k": comp(conf("b", "hunter2"))}, []Composed{comp(conf("c", "hunter3"))}, nil)
if len(far.Differs) != 1 || len(far.Differs[0].Bodies) != 1 ||
!slices.Equal(far.Differs[0].Bodies[0].Paths, []string{"content.links.1", "content.password"}) ||
!strings.Contains(far.Differs[0].Bodies[0].Composed, "cut at") || strings.Contains(fmt.Sprint(far), "hunter") {
t.Errorf("a difference past the cut: %+v", far.Differs)
}
// And an answer too large for the node's key is fitted: the texts go first, the verdict and the paths stay.
big := CandidateAnswer{Proposal: "p-x1", Revision: 1, Verdict: CandidateDiffers, Differs: far.Differs}
if fitted := FitAnswer(big, 4<<10); len(fitted.Differs[0].Bodies[0].Composed) != 0 || len(fitted.Differs[0].Bodies[0].Paths) != 2 ||
fitted.Verdict != CandidateDiffers || !strings.Contains(fitted.Note, "cut") || big.Differs[0].Bodies[0].Composed == "" {
t.Errorf("fitted as %+v", fitted)
}
if c, r := MaskedPair(json.RawMessage(`{"env":{"DB_PASSWORD":"hunter2","PORT":"80"},"list":[{"token":"t1"}]}`),
json.RawMessage(`{"env":{"DB_PASSWORD":"hunter3","PORT":"81"},"list":[{"token":"t1"}]}`), nil); strings.Contains(c+r, "hunter") ||
strings.Contains(c+r, `"t1"`) || !strings.Contains(r, `"PORT":"81"`) || !strings.Contains(c, "<masked: differs>") ||
!strings.Contains(c, "<masked: the same on both sides>") {
t.Errorf("masked as %s and %s", c, r)
}
// Issue 1126, cause C4: the own secrets whose two sides open to two values are read from the shown bodies, and
// every body so masked is counted, own or not.
// The ids are as a node runs them, its module's name first (captured from novox's shadow answer, 2026-10-28:
// `only-office.needs-jwt`); another module's need, named here, is not nats's.
drift := CompareComposedWith(map[string]Composed{"k": comp(sealedFile("nats.needs-jwt", "box-1", "0600"),
sealedFile("nats.needs-salt", "box-4", "0600"), sealedFile("nats.conf", "box-6", "0600"),
sealedFile("other.needs-key", "box-8", "0600"))},
[]Composed{comp(sealedFile("nats.needs-jwt", "box-2", "0600"), sealedFile("nats.needs-salt", "box-5", "0600"),
sealedFile("nats.conf", "box-7", "0600"), sealedFile("other.needs-key", "box-9", "0600"))},
func(a, b string) (same, known bool) { return a == "box-4", true })
got := CandidateAnswer{Proposal: "p-x1", Revision: 1, Differs: drift.Differs}
if own := OwnValuesDiffer(got); len(own) != 1 || own[0] != (OwnValueDiffers{Module: "nats", Name: "jwt"}) {
t.Errorf("the own secrets with another value are %+v, want nats jwt alone", own)
}
if n := ValuesDiffer(got); n != 3 {
t.Errorf("%d bodies hold another value, want 3 (jwt, conf and other's key)", n)
}
for id, want := range map[string]string{"nats.needs-jwt": "jwt", "needs-jwt": "jwt", "other.needs-jwt": "",
"nats.conf": "", "nats.needs-": ""} {
if name, own := OwnNeedName("nats", id); name != want && (own || want != "") {
t.Errorf("%s names nats's own secret %q (%v), want %q", id, name, own, want)
}
}
if w := WithValuesDiffer(map[string]CandidateAnswer{"g": got}); w["g"].ValuesDiffer != 3 {
t.Errorf("the answer carries values_differ %d", w["g"].ValuesDiffer)
}
if c := CompareComposed(nil, []Composed{{Node: "novox", Resources: []json.RawMessage{}}}); len(c.Removed) != 1 || c.Removed[0] != "node" {
t.Errorf("the node's own composition run and not composed is removed: %+v", c)
}
@@ -277,14 +444,16 @@ func TestPlacement(t *testing.T) {
}
for name, a := range map[string]CandidateAnswer{
"no head": {Verdict: CandidateSame},
"same with a diff": {Head: "h", Verdict: CandidateSame, Differs: answers["differs"].Differs},
"differs with none": {Head: "h", Verdict: CandidateDiffers},
"refused no reason": {Head: "h", Verdict: CandidateRefused},
"unknown verdict": {Head: "h", Verdict: "maybe"},
"diff with no module": {Head: "h", Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Why: "x"}}},
"same with an added": {Head: "h", Verdict: CandidateSame, Added: []ModuleAdded{{Module: "a"}}},
"differs refusing": {Head: "h", Verdict: CandidateDiffers, Removed: []string{"a"}, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"refused and modules": {Head: "h", Verdict: CandidateRefused, Refused: "x", Same: []string{"a"}},
"same with a diff": {Proposal: "p-x1", Revision: 1, Verdict: CandidateSame, Differs: answers["differs"].Differs},
"differs with none": {Proposal: "p-x1", Revision: 1, Verdict: CandidateDiffers},
"refused no reason": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused},
"unknown verdict": {Proposal: "p-x1", Revision: 1, Verdict: "maybe"},
"diff with no module": {Proposal: "p-x1", Revision: 1, Verdict: CandidateDiffers, Differs: []ModuleDiffers{{Why: "x"}}},
"same with an added": {Proposal: "p-x1", Revision: 1, Verdict: CandidateSame, Added: []ModuleAdded{{Module: "a"}}},
"differs refusing": {Proposal: "p-x1", Revision: 1, Verdict: CandidateDiffers, Removed: []string{"a"}, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"refused and modules": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Refused: "x", Same: []string{"a"}},
"returns refusing": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToController, Refusals: []ModuleRefused{{Module: "b", Why: "x"}}},
"returns refused": {Proposal: "p-x1", Revision: 1, Verdict: CandidateRefused, Returns: ReturnsToController, Refused: "x"},
} {
if _, err := EncodeHaveNode(HaveNode{Candidates: map[string]CandidateAnswer{"g": a}}); err == nil {
t.Errorf("%s: taken", name)
@@ -292,4 +461,113 @@ func TestPlacement(t *testing.T) {
}
})
// ADR 0567 §2, §8: a proposal's changes laid over the current keys, what waits for the lease write, and the proposal's
// encoding and refusals.
t.Run("proposal", func(t *testing.T) {
current := map[string]Placement{"g14": g14, "novox": novox}
laid, findings := LayChanges(current, []PlacementProposalChange{
{Op: OpPlace, Node: "g14", Module: "new"},
{Op: OpTakeOff, Node: "g14", Module: "zsh"},
{Op: OpUnpin, Node: "g14", Provision: "shell"},
{Op: OpDecide, Node: "novox", Field: "site", Value: json.RawMessage(`"attic"`)},
})
if len(findings) != 0 || len(laid) != 2 || !reflect.DeepEqual(laid["g14"].Assigned(), []string{"app", "new"}) ||
len(laid["g14"].Holds) != 0 || laid["g14"].Pins != nil || laid["novox"].Decided.Site != "attic" || !laid["novox"].Decided.Hub {
t.Fatalf("laid %+v, %v", laid, findings)
}
if !reflect.DeepEqual(current["g14"].Assigned(), []string{"zsh", "app"}) {
t.Error("laying changed the current placement")
}
if fresh, f := LayChanges(current, []PlacementProposalChange{{Op: OpPlace, Node: "pi", Module: "zsh"}}); len(f) != 0 || fresh["pi"].Node != "pi" {
t.Errorf("a node with no key: %+v, %v", fresh, f)
}
for name, c := range map[string]struct {
change PlacementProposalChange
want string
}{
"not placed": {PlacementProposalChange{Op: OpTakeOff, Node: "g14", Module: "nope"}, "not placed"},
"no such field": {PlacementProposalChange{Op: OpDecide, Node: "g14", Field: "colour", Value: json.RawMessage(`"x"`)}, "no decided field"},
"wrong type": {PlacementProposalChange{Op: OpDecide, Node: "g14", Field: "hub", Value: json.RawMessage(`"x"`)}, "does not take"},
"unknown op": {PlacementProposalChange{Op: "move", Node: "g14"}, "no change of placement"},
"pin of nobody": {PlacementProposalChange{Op: OpPin, Node: "g14", Provision: "x"}, "provider"},
"not held": {PlacementProposalChange{Op: OpUnhold, Node: "g14", Module: "app", Claim: "x"}, "holds no"},
"bad port": {PlacementProposalChange{Op: OpPlace, Node: "g14", Module: "x", Ports: map[string]int{"a": 1}}, "port"},
"seed of another": {PlacementProposalChange{Op: OpSeed, Node: "g14", Whole: &novox}, "seed"},
} {
if _, f := LayChanges(current, []PlacementProposalChange{c.change}); len(f) != 1 || !strings.Contains(f[0], c.want) {
t.Errorf("%s: %v, want a finding naming %q", name, f, c.want)
}
}
for change, waits := range map[*PlacementProposalChange]bool{
{Op: OpPlace, Node: "g14", Module: "x"}: false, {Op: OpTakeOff, Node: "g14", Module: "x"}: false,
{Op: OpPin, Node: "g14"}: false, {Op: OpUnpin, Node: "g14"}: false, {Op: OpSeed, Node: "g14"}: false,
{Op: OpPlace, Node: "g14", Module: "x", Ports: map[string]int{"80": 80}}: true,
{Op: OpPlace, Node: "g14", Module: "x", Bindings: map[string]Provider{"p": {}}}: true,
{Op: OpHold, Node: "g14"}: true, {Op: OpDecide, Node: "g14"}: true,
{Op: OpConfig, Node: "g14", Values: map[string]any{"puid": 1}}: false, {Op: OpConfig, Node: "g14", MeshValues: map[string]any{"a": 1}}: false,
{Op: OpConfig, Node: "g14", Layer: map[string]any{"places": 1}}: true, {Op: OpConfig, Node: "g14", MeshWide: map[string]any{"accesses": 1}}: true,
} {
if got := change.BeforeLease(); (got != "") != waits || (waits && !strings.Contains(got, "lease write")) {
t.Errorf("%+v before the lease write: %q", *change, got)
}
}
const id = "p-20261028t183000-1a2b3c"
if got := NewProposalID(time.Date(2026, 10, 28, 18, 30, 0, 0, time.UTC)); !strings.HasPrefix(got, "p-20261028t183000-") || CheckProposalID(got) != nil {
t.Errorf("NewProposalID = %q", got)
}
p := PlacementProposal{ID: id, By: "operator@anchor", Why: "move the app", At: time.Unix(1, 0).UTC(),
Changes: []PlacementProposalChange{{Op: OpTakeOff, Node: "g14", Module: "zsh"}}, Base: map[string]uint64{"g14": 4},
State: ProposalAnswering}
raw, err := EncodePlacementProposal(p)
if err != nil {
t.Fatal(err)
}
if back, err := DecodePlacementProposal(raw); err != nil || back.ID != id || back.Base["g14"] != 4 || !reflect.DeepEqual(back.Nodes(), []string{"g14"}) {
t.Fatalf("round trip %+v, %v", back, err)
}
seed := p
seed.Seed, seed.ExportDigest = true, ExportDigest(current)
seed.Changes = []PlacementProposalChange{{Op: OpSeed, Node: "g14", Whole: &g14}}
if _, err := EncodePlacementProposal(seed); err != nil {
t.Errorf("the seed: %v", err)
}
if ExportDigest(current) != ExportDigest(map[string]Placement{"novox": novox, "g14": g14}) || ExportDigest(current) == ExportDigest(nil) {
t.Error("the export digest depends on order or not on the placements")
}
back := p
back.HandBack, back.Changes = true, nil
if _, err := EncodePlacementProposal(back); err != nil {
t.Errorf("a hand-back: %v", err)
}
if k, err := PulseAnswerKey(id, "g14"); err != nil || k != "answer."+id+".g14" {
t.Errorf("PulseAnswerKey = %q, %v", k, err)
}
if k, err := PulseAskKey(id); err != nil || k != "ask."+id || PlacementPulsesBucket != "mesh-delivery-assigner_pulses" {
t.Errorf("PulseAskKey = %q, %v", k, err)
}
if ChangesDigest(p) == ChangesDigest(seed) {
t.Error("two proposals' changes digest alike")
}
for name, mut := range map[string]func(*PlacementProposal){
"no id": func(p *PlacementProposal) { p.ID = "" },
"no proposer": func(p *PlacementProposal) { p.By = "" },
"no why": func(p *PlacementProposal) { p.Why = " " },
"no change": func(p *PlacementProposal) { p.Changes = nil },
"no base": func(p *PlacementProposal) { p.Base = map[string]uint64{"novox": 1} },
"no state": func(p *PlacementProposal) { p.State = "" },
"asked, no ask": func(p *PlacementProposal) { p.State = ProposalAsked },
"seed mixed": func(p *PlacementProposal) { p.Seed, p.ExportDigest = true, "d" },
"bad change": func(p *PlacementProposal) { p.Changes[0].Op = "swap" },
"seed, no digest": func(p *PlacementProposal) { *p = seed; p.ExportDigest = "" },
"hand-back with changes": func(p *PlacementProposal) { p.HandBack = true },
} {
q := p
q.Changes = append([]PlacementProposalChange(nil), p.Changes...)
mut(&q)
if _, err := EncodePlacementProposal(q); err == nil {
t.Errorf("%s: taken", name)
}
}
})
}
+8 -19
View File
@@ -537,13 +537,11 @@ type Forge struct {
Act string `json:"act,omitempty"`
Words string `json:"words,omitempty"`
Open []OpenPull `json:"open,omitempty"`
// Placements are, on a merged message of the catalogue repository, every placement file of the merge commit, the
// bytes exact, by node (PlacementFile; novox/hq ADR 0549 §2), and PlacementsRead says they were read, so a catalogue
// whose last placement file was removed is told from a message that read none (older writers, another repository).
// On a pull message of the catalogue (CatalogueRepository) they are every placement file at the pull request's head,
// read whole or not at all, which the indexer writes as placement candidates (PlacementCandidate).
Placements map[string]string `json:"placements,omitempty"`
PlacementsRead bool `json:"placements-read,omitempty"`
// RetiredPlacements is the placement files a bridge before ADR 0567 carried (ADR 0549 §2, ADR 0559 §2): placement is
// the seat mesh-placement's state now and no repository holds it (ADR 0567 §11), so a message carrying them is
// refused, loudly, rather than read with them dropped.
RetiredPlacements json.RawMessage `json:"placements,omitempty"`
RetiredRead bool `json:"placements-read,omitempty"`
}
func (f Forge) check() error {
@@ -556,18 +554,9 @@ func (f Forge) check() error {
if len(f.Open) > 0 {
return fmt.Errorf("delivery: only an open message names open pull requests")
}
// A pull message of the catalogue carries its head's placement files, so its change of placement is composed and
// answered by the nodes it places before it merges (placement candidates, issue 1126; ADR 0549 §4).
if (len(f.Placements) > 0 || f.PlacementsRead) && f.Kind != ForgeMerged && (f.Kind != ForgePull || f.Repository != CatalogueRepository) {
return fmt.Errorf("delivery: only a merged message, or a pull message of the catalogue, carries placement files")
}
if len(f.Placements) > 0 && !f.PlacementsRead {
return fmt.Errorf("delivery: a message with placement files must say it read them (placements-read)")
}
for node := range f.Placements {
if err := checkName("node", node); err != nil {
return fmt.Errorf("delivery: a placement file names no node: %w", err)
}
if len(f.RetiredPlacements) > 0 || f.RetiredRead {
return fmt.Errorf("delivery: a forge message carries placement files, which no repository holds since ADR 0567: " +
"placement is the seat mesh-placement's state")
}
if err := checkRepository(f.Repository); err != nil {
return err
+22
View File
@@ -340,8 +340,20 @@ type CheckJob struct {
// Packages, when set, are the only packages each level tests (issue 890, ADR 0552 §4): a placed ask names the
// packages the first ask left unmet, and the others are neither tested nor taken kept, so nothing is judged twice.
Packages []string `json:"packages,omitempty"`
// NodeNames are the names of every node with a record (novox/hq ADR 0567 §12), read by the checker from each
// `<node>.node` of node-engine's have: the names check of ADR 0507 §9 looks for them in the tree, so no repository
// has to list them. The build step hands them to its test tool as the argument NodeNamesArg, and the build module
// gives them to the tests in NodeNamesEnv.
NodeNames []string `json:"node-names,omitempty"`
}
// NodeNamesArg is the test tool's argument that carries a check's node names, and NodeNamesEnv the environment
// variable a build module sets for the tests it runs from it, the names separated by commas.
const (
NodeNamesArg = "node-names"
NodeNamesEnv = "MESH_NODE_NAMES"
)
func (a StepAsk) check() error {
if err := checkAskID(a.ID); err != nil {
return err
@@ -447,6 +459,13 @@ func (c CheckJob) check(id string) error {
return fmt.Errorf("delivery: the check ask %s names a package %q that is no import path", id, p)
}
}
seen := map[string]bool{}
for _, n := range c.NodeNames {
if err := checkName("node", n); err != nil || strings.Contains(n, ",") || seen[n] {
return fmt.Errorf("delivery: the check ask %s names the node %q, which is no node name or named twice", id, n)
}
seen[n] = true
}
return checkFetchArgs(id, c.Fetch)
}
@@ -730,6 +749,9 @@ type Capacity struct {
OutsideHours bool `json:"outside-hours,omitempty"`
// BuildHoursFault is why the node's build-hours value cannot be read, naming the value: it then builds at any hour.
BuildHoursFault string `json:"build-hours-fault,omitempty"`
// Ineligible is why the module found its node cannot run its asks (buildstep.Puller.Eligible), absent when it can:
// Effective is then 0 at every hour.
Ineligible string `json:"ineligible,omitempty"`
// Why is the capacity in words: what bounds it.
Why string `json:"why"`
At time.Time `json:"at,omitzero"`
+10 -1
View File
@@ -24,7 +24,8 @@ func checkAsk() StepAsk {
return StepAsk{ID: "mh3k2x9qa1b2c4", Kind: AskCheck, Language: AskNode, Asker: "mesh-delivery-checker",
At: time.Date(2026, 10, 23, 13, 0, 0, 0, time.UTC),
Check: &CheckJob{Fetch: map[string]any{"repository": "novox/mesh-catalog", "commit": askCommit, "ref": "refs/pull/3/head", "base": askCommit},
Levels: []string{LevelUnit, LevelIntegration}, Kept: true, Runner: "workspace", RunnerWhen: "modules/mesh-build-go/", Run: ""}}
Levels: []string{LevelUnit, LevelIntegration}, Kept: true, Runner: "workspace", RunnerWhen: "modules/mesh-build-go/", Run: "",
NodeNames: []string{"anchor", "laptop"}}}
}
// 1887/A1a: an ask encodes and decodes with every field.
@@ -65,6 +66,14 @@ func TestAskRefusesUnknownKindAndLanguage(t *testing.T) {
t.Errorf("%s: encoded", name)
}
}
// ADR 0567 §12: a check's node names are each a node name, once.
for _, names := range [][]string{{"anchor", "anchor"}, {"a.b"}, {""}, {"a,b"}} {
c := checkAsk()
c.Check.NodeNames = names
if _, err := EncodeStepAsk(c); err == nil {
t.Errorf("the node names %q: encoded", names)
}
}
// A copy alone fetches nothing and needs no path.
a := buildAsk()
a.Build.Fetch, a.Build.Path, a.Build.Artifacts = nil, "", []AskArtifact{{Artifact: "base", From: "registry/x@sha256:00"}}
+23 -4
View File
@@ -38,14 +38,19 @@ const (
// AskKindRecord is the kind of the assigner's ask about a move a person must allow, for a module of policy record:
// a composition judged to recreate a container or move data, or one that cannot be judged (ADR 0539 clause 14).
AskKindRecord = "record"
// AskKindPlacement is the kind of the seat mesh-placement's authorising ask about a proposal (novox/hq ADR 0567 §5):
// `placement.<proposal id>`. The kind needs no proof, as the bus's put does not: its yes is an Approve, which only a
// channel proving who answered takes and the view never does (ViewAnswerFits).
AskKindPlacement = "placement"
viewWord = "view"
)
// AskKey is an ask's key read whole: its Kind, and the parts its kind has: a merge's Group and heads Hash, a hold's
// Module and fingerprint FP, a restore's and a backup's Node, Module and FP, a record's Node, Module and the digest
// Composed of the composition it would allow.
// Composed of the composition it would allow, a placement's Proposal.
type AskKey struct {
Proposal string
Kind string
Group string
Hash string
@@ -61,11 +66,11 @@ func AskNeedsProof(kind string) (bool, error) {
switch kind {
case AskKindMerge:
return true, nil
case AskKindHold, AskKindRestore, AskKindBackup, AskKindRecord:
case AskKindHold, AskKindRestore, AskKindBackup, AskKindRecord, AskKindPlacement:
return false, nil
}
return false, fmt.Errorf("delivery: %q is not a kind of ask, %s, %s, %s, %s or %s", kind, AskKindMerge, AskKindHold,
AskKindRestore, AskKindBackup, AskKindRecord)
return false, fmt.Errorf("delivery: %q is not a kind of ask, %s, %s, %s, %s, %s or %s", kind, AskKindMerge, AskKindHold,
AskKindRestore, AskKindBackup, AskKindRecord, AskKindPlacement)
}
// HoldAskKey is the key, in AssignerAsksBucket, of the ask about a module's held fingerprint: `hold.<module>.<hex>`.
@@ -100,6 +105,15 @@ func BackupAskKey(node, module, fingerprint string) (string, error) {
return AskKindBackup + "." + key, nil
}
// PlacementAskKey is the key, in AssignerAsksBucket, of the authorising ask about a proposal of placement:
// `placement.<proposal id>` (ADR 0567 §5).
func PlacementAskKey(proposal string) (string, error) {
if err := CheckProposalID(proposal); err != nil {
return "", err
}
return AskKindPlacement + "." + proposal, nil
}
// RecordAskKey is the key, in AssignerAsksBucket, of the ask about a move of a module on a node to the composition
// whose digest it names: `record.<node>.<module>.<digest hex>`, so a new revision at the same fingerprint is asked
// again (ADR 0539 clause 14).
@@ -154,6 +168,11 @@ func ParseAnyAskKey(key string) (AskKey, error) {
return refused(fmt.Errorf("a backup ask is backup.<node>.<module>.<fingerprint hex>"))
}
return AskKey{Kind: kind, Node: node, Module: module, FP: fp}, nil
case AskKindPlacement:
if err := CheckProposalID(rest); err != nil {
return refused(err)
}
return AskKey{Kind: kind, Proposal: rest}, nil
case AskKindRecord:
parts := strings.Split(rest, ".")
if len(parts) != 3 {
+11 -3
View File
@@ -33,9 +33,17 @@ func TestParseAnyAskKey(t *testing.T) {
if err != nil || restore != "restore.ace.mesh-mail."+strings.Repeat("d", 64) {
t.Fatalf("RestoreAskKey = %q, %v", restore, err)
}
placement, err := PlacementAskKey("p-one")
if err != nil || placement != "placement.p-one" {
t.Fatalf("PlacementAskKey = %q, %v", placement, err)
}
if proof, err := AskNeedsProof(AskKindPlacement); proof || err != nil {
t.Error("the assigner cannot keep a placement ask: its kind needs proof (its yes's level does)")
}
for key, want := range map[string]AskKey{
hold: {Kind: AskKindHold, Module: "mesh-mail", FP: fp},
restore: {Kind: AskKindRestore, Node: "ace", Module: "mesh-mail", FP: fp},
hold: {Kind: AskKindHold, Module: "mesh-mail", FP: fp},
restore: {Kind: AskKindRestore, Node: "ace", Module: "mesh-mail", FP: fp},
placement: {Kind: AskKindPlacement, Proposal: "p-one"},
} {
if got, err := ParseAnyAskKey(key); err != nil || got != want {
t.Errorf("ParseAnyAskKey(%q) = %+v, %v", key, got, err)
@@ -44,7 +52,7 @@ func TestParseAnyAskKey(t *testing.T) {
if got, err := ParseAnyAskKey(merge); err != nil || got.Kind != AskKindMerge || got.Group != "feat/r4g" || len(got.Hash) != 12 {
t.Errorf("ParseAnyAskKey(%q) = %+v, %v", merge, got, err)
}
for _, bad := range []string{"want.ace.mesh-mail", "hold.mesh-mail", "restore.mesh-mail." + strings.Repeat("d", 64), "approve.x.y", "restore.a.b.c.d", ""} {
for _, bad := range []string{"want.ace.mesh-mail", "hold.mesh-mail", "restore.mesh-mail." + strings.Repeat("d", 64), "approve.x.y", "restore.a.b.c.d", "placement.P", ""} {
if _, err := ParseAnyAskKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseAnyAskKey(%q) = %v, want a refusal naming it", bad, err)
}