Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jochen
|
ef76b9f2ea |
@@ -1,3 +1,2 @@
|
||||
node_modules/
|
||||
dist/
|
||||
vendor/
|
||||
|
||||
@@ -9,15 +9,6 @@ does not belong.* Per-module code (a service's API client, its tool implementati
|
||||
create-a-resource adapter) lives **in the module**, never here — that coupling is exactly what
|
||||
turned the old SDK into constant maintenance and made every edit rebuild every module.
|
||||
|
||||
**A seat's code is not here** (novox/hq ADR 0625, ADR 0627). mesh-sdk is the runtime every module uses (the bus
|
||||
connection, the tools bundle's runtime, state and its helpers) and a seat-neutral part two or more seats' clients or
|
||||
holders share (sending a file in parts, `go/parts`). A seat defined whole has its generated contract, and its client
|
||||
where callers share logic over it, in mesh-contracts (`go/seats/<seat>`, `go/clients/<seat>client`); the seat list is
|
||||
mesh-contracts' `go/seats`, which `catalogue` alone imports. The value layouts of seats not yet defined whole stay
|
||||
here until their domain moves, and no new seat-specific type is added. `packages_list_test.go` holds all of it:
|
||||
mesh-sdk imports no contracts module but the seat set in `catalogue`, names none of the seats with a client outside
|
||||
`catalogue`, and holds none of the packages that moved (`asks`, `asker`, `publish`).
|
||||
|
||||
## What is in it
|
||||
|
||||
Five small areas, each a thing a module's `tools/` or `provisioner/` imports:
|
||||
@@ -115,20 +106,21 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
|
||||
values and then every change. Through the runtime that launched the bundle, which refuses what the
|
||||
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
|
||||
|
||||
### `seats` and `channel` — publishing on a seat, the channel's benches (Go)
|
||||
### `seats` and `asks` — asking a person, answering on a channel (Go)
|
||||
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
|
||||
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
|
||||
and `stdio.SeatAnswer` carry a proof (a code typed by a person, by request and reply, never kept),
|
||||
`stdio.SeatRecord` reads a record kept for the module, and `State(...).Create` and `.Update` write by
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/channel` holds the
|
||||
shapes of the two benches a channel holds, `channel` and `intake` (the `Message` a channel takes, the `Choice`,
|
||||
`LinkAsked`, `Failed`, `Standing` and `Code` it says, with the subjects each travels on): those benches are not
|
||||
defined whole, so their layouts stay here (ADR 0625 §9).
|
||||
Asking a person is the `operator-channel` seat's, defined whole in mesh-contracts: its `Ask` and `Warrant` are its
|
||||
generated contract (`go/seats/operatorchannel`), and the checks an ask is held to, the warrant's (`For`, `Says`,
|
||||
`Performs`) and an asking module's client (`Ask`, `Cancel`, `Where`, `Listen`, `Take`, each warrant handed once,
|
||||
marked acted on by compare-and-set) are its client, `go/clients/operatorchannelclient` (ADR 0627 §1). A lost
|
||||
compare-and-set is `stdio.ErrChanged`, told by the runtime's error code.
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/asks` is the
|
||||
contract on top: the `Ask` an asker publishes and the checks it is held to (its `Whole` is what a channel that proves
|
||||
who answers shows in place of its `Explanation`, and its `Details` what the Details answer shows, hq issue 383), the `Warrant` it acts on
|
||||
(`For`, `Says`), and the `Message`, `Choice`, `LinkAsked` and `Code` a channel takes and says, with the
|
||||
subjects each travels on.
|
||||
`go/asker` is an asking module's client (ADR 0259 §10): `Ask` keeps an ask in the module's own state and then
|
||||
publishes it under the module's name; `Cancel`; `Where` reads its state from the router's record; `Listen` and
|
||||
`Take` hand the module each warrant once — marked acted on by compare-and-set before the module acts, and only
|
||||
for an ask it made, an option it offered, at that option's level, before it expired. A lost compare-and-set is
|
||||
`stdio.ErrChanged`, told by the runtime's error code.
|
||||
`State(...).Increment(key, attempts)` is a counter on top of the two (for numbers handed out once, such as an
|
||||
issue's): it reads the key, writes one more at the revision read, and on a lost write reads and tries again, at
|
||||
most `attempts` times; past that it is `stdio.ErrChanged` and nothing was counted.
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
example
|
||||
@@ -0,0 +1,8 @@
|
||||
CFLAGS ?= -O2 -Wall -Wextra
|
||||
JANSSON := $(shell pkg-config --cflags --libs jansson)
|
||||
|
||||
example: example.c mesh_sdk.c mesh_sdk.h
|
||||
$(CC) $(CFLAGS) -o $@ example.c mesh_sdk.c $(JANSSON)
|
||||
|
||||
clean:
|
||||
rm -f example
|
||||
@@ -0,0 +1,9 @@
|
||||
# mesh-sdk for C
|
||||
|
||||
The skeleton (novox/hq ADR 0188): `mesh_serve(name, tools, count)` speaks MCP over stdio to the
|
||||
node's tool runtime. A `mesh_tool` is a name, a description, an argument schema and a function
|
||||
returning JSON. A tool named `<seat>.<verb>` is the seat's implementation. JSON is jansson's, the
|
||||
one dependency.
|
||||
|
||||
What it holds, under ADR 0039's test: the stdio loop and the tool type. Nothing else — no bus, no
|
||||
client, nothing of any module's. `make` and pipe a `tools/list` request into `./example`.
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
/* A tools bundle in C: one tool of its own and one seat verb, served over stdio. */
|
||||
#include "mesh_sdk.h"
|
||||
#include <stdio.h>
|
||||
|
||||
static json_t *greet(json_t *args, char **error) {
|
||||
(void)error;
|
||||
const char *who = json_string_value(json_object_get(args, "who"));
|
||||
char greeting[128];
|
||||
snprintf(greeting, sizeof greeting, "hello %s", who ? who : "world");
|
||||
return json_pack("{s:s,s:s}", "greeting", greeting, "language", "c");
|
||||
}
|
||||
|
||||
static json_t *on(json_t *args, char **error) {
|
||||
(void)args; (void)error;
|
||||
return json_pack("{s:b,s:s}", "on", 1, "language", "c");
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
json_t *who = json_pack("{s:{s:s,s:s}}", "who", "type", "string", "description", "whom");
|
||||
const mesh_tool tools[] = {
|
||||
{"greet", "say hello", who, greet},
|
||||
{"node-lamp.on", "the seat's verb", NULL, on},
|
||||
};
|
||||
int rc = mesh_serve("c-example", tools, 2);
|
||||
json_decref(who);
|
||||
return rc;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
#include "mesh_sdk.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static void say(json_t *m) {
|
||||
json_object_set_new(m, "jsonrpc", json_string("2.0"));
|
||||
char *text = json_dumps(m, JSON_COMPACT);
|
||||
fprintf(stdout, "%s\n", text);
|
||||
fflush(stdout);
|
||||
free(text);
|
||||
json_decref(m);
|
||||
}
|
||||
|
||||
static json_t *as_schema(json_t *given) {
|
||||
if (!given || !json_is_object(given) || json_object_size(given) == 0)
|
||||
return json_pack("{s:s,s:{}}", "type", "object", "properties");
|
||||
if (json_object_get(given, "type") || json_object_get(given, "properties"))
|
||||
return json_incref(given);
|
||||
return json_pack("{s:s,s:O}", "type", "object", "properties", given);
|
||||
}
|
||||
|
||||
int mesh_serve(const char *name, const mesh_tool *tools, size_t count) {
|
||||
char *line = NULL;
|
||||
size_t cap = 0;
|
||||
ssize_t got;
|
||||
while ((got = getline(&line, &cap, stdin)) >= 0) {
|
||||
json_error_t err;
|
||||
json_t *req = json_loads(line, 0, &err);
|
||||
if (!req) continue;
|
||||
json_t *id = json_object_get(req, "id");
|
||||
const char *method = json_string_value(json_object_get(req, "method"));
|
||||
json_t *params = json_object_get(req, "params");
|
||||
if (!method) { json_decref(req); continue; }
|
||||
if (strcmp(method, "initialize") == 0) {
|
||||
say(json_pack("{s:O,s:{s:s,s:{s:{}},s:{s:s,s:s}}}", "id", id ? id : json_null(), "result",
|
||||
"protocolVersion", MESH_SDK_PROTOCOL, "capabilities", "tools",
|
||||
"serverInfo", "name", name, "version", "1"));
|
||||
} else if (strcmp(method, "notifications/initialized") == 0) {
|
||||
} else if (strcmp(method, "ping") == 0) {
|
||||
if (id) say(json_pack("{s:O,s:{}}", "id", id, "result"));
|
||||
} else if (strcmp(method, "tools/list") == 0) {
|
||||
json_t *listed = json_array();
|
||||
for (size_t i = 0; i < count; i++)
|
||||
json_array_append_new(listed, json_pack("{s:s,s:s,s:o}", "name", tools[i].name,
|
||||
"description", tools[i].description, "inputSchema", as_schema(tools[i].input)));
|
||||
say(json_pack("{s:O,s:{s:o}}", "id", id ? id : json_null(), "result", "tools", listed));
|
||||
} else if (strcmp(method, "tools/call") == 0) {
|
||||
const char *wanted = params ? json_string_value(json_object_get(params, "name")) : NULL;
|
||||
const mesh_tool *tool = NULL;
|
||||
for (size_t i = 0; wanted && i < count; i++)
|
||||
if (strcmp(tools[i].name, wanted) == 0) tool = &tools[i];
|
||||
if (!tool) {
|
||||
char message[256];
|
||||
snprintf(message, sizeof message, "%s has no tool %s", name, wanted ? wanted : "");
|
||||
say(json_pack("{s:O,s:{s:i,s:s}}", "id", id ? id : json_null(), "error", "code", -32602, "message", message));
|
||||
} else {
|
||||
json_t *args = params ? json_object_get(params, "arguments") : NULL;
|
||||
json_t *empty = args ? NULL : json_object();
|
||||
char *error = NULL;
|
||||
json_t *result = tool->run(args ? args : empty, &error);
|
||||
if (empty) json_decref(empty);
|
||||
if (result) {
|
||||
char *text = json_dumps(result, JSON_COMPACT);
|
||||
say(json_pack("{s:O,s:{s:[{s:s,s:s}]}}", "id", id ? id : json_null(), "result", "content", "type", "text", "text", text));
|
||||
free(text);
|
||||
json_decref(result);
|
||||
} else {
|
||||
say(json_pack("{s:O,s:{s:[{s:s,s:s}],s:b}}", "id", id ? id : json_null(), "result", "content",
|
||||
"type", "text", "text", error ? error : "the tool failed", "isError", 1));
|
||||
free(error);
|
||||
}
|
||||
}
|
||||
} else if (id) {
|
||||
char message[256];
|
||||
snprintf(message, sizeof message, "no %s", method);
|
||||
say(json_pack("{s:O,s:{s:i,s:s}}", "id", id, "error", "code", -32601, "message", message));
|
||||
}
|
||||
json_decref(req);
|
||||
}
|
||||
free(line);
|
||||
return ferror(stdin) ? -1 : 0;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
/* The Novox Mesh SDK for C — the skeleton (novox/hq ADR 0188).
|
||||
*
|
||||
* A tools bundle is a process the node's runtime launches and speaks MCP over stdio to:
|
||||
* `initialize`, `tools/list` once, `tools/call` per call, newline-framed JSON-RPC on stdin and
|
||||
* stdout. This is the loop, so a bundle writes its tools and nothing else. A tool named
|
||||
* `<seat>.<verb>` is the module's implementation of that seat's verb; any other name is the
|
||||
* module's own. JSON is jansson's.
|
||||
*
|
||||
* No transport here: the bus, the subjects and the memberships are the runtime's (ADR 0039, 0175).
|
||||
* stdout is the protocol; log to stderr. */
|
||||
#ifndef MESH_SDK_H
|
||||
#define MESH_SDK_H
|
||||
#include <jansson.h>
|
||||
|
||||
#define MESH_SDK_PROTOCOL "2025-03-26"
|
||||
|
||||
/* One tool. `run` returns a new JSON value (the answer) or NULL with *error set to a message the
|
||||
* caller frees. `input` is the schema of the arguments, or NULL for a tool that takes nothing. */
|
||||
typedef struct {
|
||||
const char *name;
|
||||
const char *description;
|
||||
json_t *input;
|
||||
json_t *(*run)(json_t *args, char **error);
|
||||
} mesh_tool;
|
||||
|
||||
/* Serve these tools on stdin/stdout until stdin closes. Returns 0, or -1 on an I/O fault. */
|
||||
int mesh_serve(const char *name, const mesh_tool *tools, size_t count);
|
||||
|
||||
#endif
|
||||
+1
-1
@@ -16,6 +16,6 @@ at the same version as the TypeScript package.
|
||||
|
||||
What it holds, under ADR 0039's test: the stdio loop and the tool type, and `cli`, the local protocol
|
||||
`mesh-cli` speaks to the node-engine on its machine (novox/hq ADR 0272): the socket's path and one request and one
|
||||
answer; and `seal`, the mesh's sealed box (novox/hq ADR 0206: X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}`), byte for byte the box the licence manager made before it lived here. Nothing else — no bus, no client, nothing of any module's.
|
||||
answer; and `seal`, the mesh's sealed box (novox/hq ADR 0206: X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}`), byte for byte the box the licence manager made before it lived here and the TypeScript `@novox/mesh-sdk/seal` makes. Nothing else — no bus, no client, nothing of any module's. `go build ./example` and pipe a `tools/list` request into it.
|
||||
|
||||
From release 4a of design 47 (novox/hq ADR 0453 clause 3, ADR 0446 clause 4) it also holds the composition of a machine's declaration, over a plain input and with nothing behind it: `declaration` and `validate`, the node-engine's parser and its validator, copied from mesh-host; `catalogue`, the manifests, their resolution and rendering, and `overlay`, the private network's generator, copied from mesh-controller; `compose`, the controller's planFor, renderingFor and declarationWith as functions of an `Input` (the composition never allocates, makes, keeps or writes: what a send would do it names as foreseen or returns); `sdkversion`, which mesh-sdk version a program was built with; and, in `delivery`, the layouts of the keys the builder, the checker and the indexer write (the source fingerprint's new form, the builds, the verdicts and kept verdicts, the manifests by digest). None of them holds a bus, a store or a vault client, which `go list -deps` of `compose` shows.
|
||||
|
||||
@@ -1,152 +0,0 @@
|
||||
// Package announce is how a module says on the bus which states it keeps and which of them others may read (novox/hq
|
||||
// ADR 0495 clauses 2 and 3; issue 1790): the subject it says it on, the value it says, and where the controller keeps
|
||||
// the last one said by each module. What a module announced is the truth of what it keeps; a reader is granted a read
|
||||
// from it, never from a list or from the owner's manifest.
|
||||
//
|
||||
// A module announces from its code (process.Conn.Keeps, stdio.Keeps, the TypeScript SDK's keeps), never from its
|
||||
// manifest, which only the core reads (clause 1). A module that keeps no state on the bus announces nothing.
|
||||
package announce
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Version is the version of the value this package writes.
|
||||
const Version = 1
|
||||
|
||||
// Event is the event type of an announcement, under the module's own namespace.
|
||||
const Event = "states"
|
||||
|
||||
// Subject is where module announces what it keeps: its own event subject, mesh.mod.<module>.event.states, which its
|
||||
// account may already publish to and the events stream keeps.
|
||||
func Subject(module string) string { return "mesh.mod." + module + ".event." + Event }
|
||||
|
||||
// SubjectAll is every module's announcement.
|
||||
const SubjectAll = "mesh.mod.*.event." + Event
|
||||
|
||||
var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||
|
||||
// Key is module's key where the last announcement of each module is kept: the module itself, which must be one plain name, since a module's name is part of
|
||||
// its buckets' names (the indexer's judge, novox/hq ADR 0578 §1).
|
||||
func Key(module string) (string, error) {
|
||||
if !name.MatchString(module) {
|
||||
return "", fmt.Errorf("announce: %q is not a module that keeps state: lower-case letters, digits and hyphens", module)
|
||||
}
|
||||
return module, nil
|
||||
}
|
||||
|
||||
// State is one state a module keeps, by its local name, and whether others may read it. Others never write it: a
|
||||
// state another module writes is the owner's own code's business.
|
||||
type State struct {
|
||||
Name string `json:"name"`
|
||||
Readable bool `json:"readable,omitempty"`
|
||||
// Readers, when named, are the only modules that may read the state (novox/hq ADR 0547 clause 9, issue 1945): a
|
||||
// state readable by some and not all, such as the vault's bus account hashes. Only ReadableBy grants them; Readable,
|
||||
// which a composer that knows no readers asks, stays false for it, so such a composer grants nobody.
|
||||
Readers []string `json:"readers,omitempty"`
|
||||
}
|
||||
|
||||
// States is an announcement: the module, the machine whose instance said it, and every state the module keeps, sorted
|
||||
// by name. An empty list says the module keeps nothing.
|
||||
type States struct {
|
||||
V int `json:"v"`
|
||||
Module string `json:"module"`
|
||||
Node string `json:"node,omitempty"`
|
||||
States []State `json:"states"`
|
||||
}
|
||||
|
||||
// Keeps says whether the module announced keeping the state.
|
||||
func (s States) Keeps(state string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name == state {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ReadableBy says whether the module announced the state readable by reader: readable, and reader among the readers
|
||||
// when it names any.
|
||||
func (s States) ReadableBy(state, reader string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name != state {
|
||||
continue
|
||||
}
|
||||
if len(k.Readers) > 0 {
|
||||
for _, r := range k.Readers {
|
||||
if r == reader {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
return k.Readable
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Readable says whether the module announced the state readable by others.
|
||||
func (s States) Readable(state string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name == state {
|
||||
return k.Readable
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (s *States) check() error {
|
||||
if s.Module == "" {
|
||||
return fmt.Errorf("announce: an announcement names no module")
|
||||
}
|
||||
if _, err := Key(s.Module); err != nil {
|
||||
return err
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, k := range s.States {
|
||||
if !name.MatchString(k.Name) {
|
||||
return fmt.Errorf("announce: %s announces the state %q, and a state is named locally: lower-case letters, digits and hyphens", s.Module, k.Name)
|
||||
}
|
||||
if seen[k.Name] {
|
||||
return fmt.Errorf("announce: %s announces the state %q twice", s.Module, k.Name)
|
||||
}
|
||||
seen[k.Name] = true
|
||||
for _, r := range k.Readers {
|
||||
if !name.MatchString(r) {
|
||||
return fmt.Errorf("announce: %s names %q as a reader of %s, and a reader is a module: lower-case letters, digits and hyphens", s.Module, r, k.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Encode is s as JSON at Version, its states sorted; it refuses a module or a state no bucket could be named by and a
|
||||
// state announced twice, naming it.
|
||||
func Encode(s States) ([]byte, error) {
|
||||
if err := s.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.V = Version
|
||||
s.States = append([]State{}, s.States...)
|
||||
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
|
||||
return json.Marshal(s)
|
||||
}
|
||||
|
||||
// Decode reads an announcement; it refuses another version and what Encode refuses.
|
||||
func Decode(data []byte) (States, error) {
|
||||
var s States
|
||||
if err := json.Unmarshal(data, &s); err != nil {
|
||||
return States{}, fmt.Errorf("announce: the announcement is not JSON: %w", err)
|
||||
}
|
||||
if s.V != Version {
|
||||
return States{}, fmt.Errorf("announce: the announcement is version %d, and this reads version %d", s.V, Version)
|
||||
}
|
||||
if err := s.check(); err != nil {
|
||||
return States{}, err
|
||||
}
|
||||
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
|
||||
return s, nil
|
||||
}
|
||||
@@ -1,75 +0,0 @@
|
||||
package announce
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A module says on the bus which states it keeps and which of them others may read (novox/hq ADR 0495 clause 2), on its
|
||||
// own event subject, in a versioned value the controller keeps by module.
|
||||
func TestAnAnnouncementRoundTripsSortedAndSaysWhatIsReadable(t *testing.T) {
|
||||
data, err := Encode(States{Module: "mesh-controller", Node: "novox", States: []State{
|
||||
{Name: "plans-building", Readable: true}, {Name: "assignments", Readable: true}, {Name: "own"},
|
||||
{Name: "hashes", Readers: []string{"mesh-grants"}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
back, err := Decode(data)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.V != Version || back.Module != "mesh-controller" || back.Node != "novox" {
|
||||
t.Fatalf("decoded %+v", back)
|
||||
}
|
||||
var names []string
|
||||
for _, s := range back.States {
|
||||
names = append(names, s.Name)
|
||||
}
|
||||
if strings.Join(names, ",") != "assignments,hashes,own,plans-building" {
|
||||
t.Fatalf("the states are not sorted: %v", names)
|
||||
}
|
||||
if !back.Readable("plans-building") || back.Readable("own") || back.Readable("absent") {
|
||||
t.Fatal("Readable does not say what was announced readable")
|
||||
}
|
||||
// Named readers (ADR 0547 clause 9): readable by them alone, and not readable to a composer that knows no readers.
|
||||
if back.Readable("hashes") || !back.ReadableBy("hashes", "mesh-grants") || back.ReadableBy("hashes", "spy") ||
|
||||
!back.ReadableBy("plans-building", "spy") || back.ReadableBy("own", "spy") {
|
||||
t.Fatal("ReadableBy does not say who was named")
|
||||
}
|
||||
if !back.Keeps("own") || back.Keeps("absent") {
|
||||
t.Fatal("Keeps does not say what was announced")
|
||||
}
|
||||
if Subject("mesh-controller") != "mesh.mod.mesh-controller.event.states" || SubjectAll != "mesh.mod.*.event.states" {
|
||||
t.Fatalf("the subjects are %q and %q", Subject("mesh-controller"), SubjectAll)
|
||||
}
|
||||
if k, err := Key("mesh-delivery-builder"); err != nil || k != "mesh-delivery-builder" {
|
||||
t.Fatalf("Key = %q, %v", k, err)
|
||||
}
|
||||
if nothing, err := Encode(States{Module: "quiet"}); err != nil || !strings.Contains(string(nothing), `"states":[]`) {
|
||||
t.Fatalf("a module keeping nothing: %s, %v", nothing, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAnnouncementRefusesWhatNoBucketCouldBe(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
s States
|
||||
says string
|
||||
}{
|
||||
{States{Module: ""}, "no module"},
|
||||
{States{Module: "a.b"}, `"a.b"`},
|
||||
{States{Module: "a", States: []State{{Name: "x_y"}}}, `"x_y"`},
|
||||
{States{Module: "a", States: []State{{Name: ""}}}, `""`},
|
||||
{States{Module: "a", States: []State{{Name: "x"}, {Name: "x", Readable: true}}}, "twice"},
|
||||
{States{Module: "a", States: []State{{Name: "x", Readers: []string{"b.c"}}}}, `"b.c"`},
|
||||
} {
|
||||
if _, err := Encode(c.s); err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%+v: %v, wanted a refusal saying %s", c.s, err, c.says)
|
||||
}
|
||||
}
|
||||
if _, err := Decode([]byte(`{"v":2,"module":"a","states":[]}`)); err == nil {
|
||||
t.Error("an unknown version was read")
|
||||
}
|
||||
if _, err := Key("a.b"); err == nil {
|
||||
t.Error("a module with a dot became a key")
|
||||
}
|
||||
}
|
||||
@@ -1,91 +0,0 @@
|
||||
// Package announcetest is the test each module runs with its own checks to hold that it announces on the bus every
|
||||
// state it keeps (novox/hq ADR 0495 clause 5; issue 1790): it starts the module against a bus of the test's own and
|
||||
// compares what the module announced with what its code declares. With the bind guard of process.Conn.Keeps,
|
||||
// stdio.Keeps and the TypeScript keeps, which refuses a state the code did not announce, announcement and code agree.
|
||||
//
|
||||
// It replaces the merge gate's check of one module's reads against another's manifest (clause 5); a test using it
|
||||
// carries `//go:build integration`, since it starts a bus.
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
"git.novox.be/novox/mesh-sdk/go/testbus"
|
||||
)
|
||||
|
||||
// Wait is how long Check waits for the module's announcement once it started it.
|
||||
var Wait = 10 * time.Second
|
||||
|
||||
// Check starts a bus of the test's own, with a stream taking module's events, calls start with its address (start
|
||||
// runs the module, or its part that connects, against it, and answers how to stop it), waits for the module's
|
||||
// announcement and fails the test unless it is module's and names exactly want (in any order), readable as want says.
|
||||
// It answers the announcement.
|
||||
func Check(t testing.TB, module string, want []announce.State, start func(url string) (stop func())) announce.States {
|
||||
t.Helper()
|
||||
url := testbus.URL(t)
|
||||
nc, err := nats.Connect(url)
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: connecting to the test bus: %v", err)
|
||||
}
|
||||
defer nc.Close()
|
||||
js, err := jetstream.New(nc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), Wait)
|
||||
defer cancel()
|
||||
stream, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "ANNOUNCETEST", Subjects: []string{"mesh.mod." + module + ".event.>"}})
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: making the events stream: %v", err)
|
||||
}
|
||||
stop := start(url)
|
||||
if stop != nil {
|
||||
defer stop()
|
||||
}
|
||||
got, err := waitFor(ctx, stream, module)
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: %v", err)
|
||||
}
|
||||
if problem := differs(got, module, want); problem != "" {
|
||||
t.Fatalf("announcetest: %s", problem)
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
// waitFor reads the stream until module's announcement is in it.
|
||||
func waitFor(ctx context.Context, stream jetstream.Stream, module string) (announce.States, error) {
|
||||
for {
|
||||
msg, err := stream.GetLastMsgForSubject(ctx, announce.Subject(module))
|
||||
if err == nil {
|
||||
return announce.Decode(msg.Data)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return announce.States{}, fmt.Errorf("%s announced nothing on %s within %s", module, announce.Subject(module), Wait)
|
||||
case <-time.After(50 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// differs says how got is not module's announcement of want, or "".
|
||||
func differs(got announce.States, module string, want []announce.State) string {
|
||||
if got.Module != module {
|
||||
return fmt.Sprintf("the announcement names the module %q, not %q", got.Module, module)
|
||||
}
|
||||
w := append([]announce.State{}, want...)
|
||||
sort.Slice(w, func(i, j int) bool { return w[i].Name < w[j].Name })
|
||||
g := append([]announce.State{}, got.States...)
|
||||
if !reflect.DeepEqual(g, w) && !(len(g) == 0 && len(w) == 0) {
|
||||
return fmt.Sprintf("%s announced %v, and its code declares %v", module, g, w)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
"git.novox.be/novox/mesh-sdk/go/process"
|
||||
)
|
||||
|
||||
// 1790/S11: a module started against the test bus announces through process.Conn.Keeps, and Check reads that
|
||||
// announcement back against what the code declares.
|
||||
func TestCheckReadsTheModulesAnnouncementOffARealBus(t *testing.T) {
|
||||
declared := []announce.State{{Name: "accounts", Readable: true}, {Name: "drafts"}}
|
||||
got := Check(t, "mesh-mail", declared, func(url string) func() {
|
||||
conn, err := process.Connect(process.Account{URL: url, Module: "mesh-mail", Node: "novox", User: "u", Password: "p"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := conn.Keeps(declared...); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return conn.Close
|
||||
})
|
||||
if got.Node != "novox" {
|
||||
t.Fatalf("announced from %q", got.Node)
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
)
|
||||
|
||||
// The comparison is exact up to order: a state missing, one extra, a readability that differs or another module's
|
||||
// announcement each fail, naming what differs.
|
||||
func TestAnAnnouncementDiffersFromTheCodeByNameAndReadability(t *testing.T) {
|
||||
got := announce.States{Module: "m", States: []announce.State{{Name: "a", Readable: true}, {Name: "b"}}}
|
||||
if p := differs(got, "m", []announce.State{{Name: "b"}, {Name: "a", Readable: true}}); p != "" {
|
||||
t.Fatalf("an equal announcement differs: %s", p)
|
||||
}
|
||||
for _, want := range [][]announce.State{
|
||||
{{Name: "a", Readable: true}},
|
||||
{{Name: "a", Readable: true}, {Name: "b"}, {Name: "c"}},
|
||||
{{Name: "a"}, {Name: "b"}},
|
||||
} {
|
||||
if p := differs(got, "m", want); p == "" || !strings.Contains(p, "declares") {
|
||||
t.Errorf("want %v: %q", want, p)
|
||||
}
|
||||
}
|
||||
if p := differs(got, "other", nil); !strings.Contains(p, `"other"`) {
|
||||
t.Errorf("another module's announcement: %q", p)
|
||||
}
|
||||
if p := differs(announce.States{Module: "m"}, "m", nil); p != "" {
|
||||
t.Errorf("a module keeping nothing: %q", p)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
// Package asker is a module's client for asking a person (novox/hq ADR 0259 §10): it publishes an ask under
|
||||
// the module's own name, cancels it, reads its state from the router's record, and hands the module each
|
||||
// warrant once — only one for an ask it made, for an option it offered, at that option's level, before it
|
||||
// expired. The module then acts with its own grants.
|
||||
//
|
||||
// It keeps what it asked, and which warrants it acted on, in a state of the module's own (declared in its
|
||||
// manifest, by default `asked`): the ask is written before it is published, and a warrant is marked acted on
|
||||
// by compare-and-set before the module's handler runs, so a redelivery, or a second instance, acts once.
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
// Client asks as one module.
|
||||
type Client struct {
|
||||
// Module is the asking module's name: the one the bus grants its asks under.
|
||||
Module string
|
||||
// State is the module's own state the asks are kept in.
|
||||
State Store
|
||||
// Now is the clock; nil is time.Now.
|
||||
Now func() time.Time
|
||||
// Publish, Record and Subscribe reach the bus; nil is the stdio SDK's, through the module's runtime.
|
||||
Publish func(subject string, body any, id string) error
|
||||
Record func(key string) (*stdio.StateEntry, error)
|
||||
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
|
||||
}
|
||||
|
||||
// Store is where the asks are kept: stdio.State(name) is one.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
}
|
||||
|
||||
// New is a client for the module the runtime serves, keeping its asks in the state named.
|
||||
func New(state string) *Client {
|
||||
return &Client{Module: os.Getenv("MESH_MODULE"), State: stdio.State(state)}
|
||||
}
|
||||
|
||||
func (c *Client) now() time.Time {
|
||||
if c.Now != nil {
|
||||
return c.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
// Kept is one ask as the module keeps it.
|
||||
type Kept struct {
|
||||
Ask asks.Ask `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended asks.Outcome `json:"ended,omitempty"`
|
||||
Answer *asks.Warrant `json:"answer,omitempty"`
|
||||
}
|
||||
|
||||
func key(id string) string { return "ask." + id }
|
||||
|
||||
// Ask checks an ask, keeps it, and publishes it under the module's own name. An ask with an id already kept
|
||||
// is refused: a new question is a new id.
|
||||
func (c *Client) Ask(a asks.Ask) error {
|
||||
if err := a.Check(c.now()); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := c.State.Create(key(a.ID), Kept{Ask: a, Asked: c.now()}); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return fmt.Errorf("an ask %s was already made: a new question is a new id", a.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return c.publish(asks.AskSubject(c.Module), a, a.ID)
|
||||
}
|
||||
|
||||
// Cancel takes an ask back: the router ends it and edits every copy.
|
||||
func (c *Client) Cancel(id string) error {
|
||||
return c.publish(asks.CancelSubject(c.Module), map[string]string{"id": id}, "cancel."+id)
|
||||
}
|
||||
|
||||
func (c *Client) publish(subject string, body any, id string) error {
|
||||
if c.Publish != nil {
|
||||
return c.Publish(subject, body, id)
|
||||
}
|
||||
return stdio.SeatPublish(subject, body, id)
|
||||
}
|
||||
|
||||
// Where is an ask's state as the router keeps it: its state and its warrant, if any — read under the
|
||||
// module's own name and no other's.
|
||||
func (c *Client) Where(id string) (state string, w *asks.Warrant, err error) {
|
||||
read := c.Record
|
||||
if read == nil {
|
||||
read = func(k string) (*stdio.StateEntry, error) { return stdio.SeatRecord("", k) }
|
||||
}
|
||||
e, err := read(id)
|
||||
if err != nil || e == nil {
|
||||
return "", nil, err
|
||||
}
|
||||
var rec struct {
|
||||
State string `json:"state"`
|
||||
Warrant *asks.Warrant `json:"warrant"`
|
||||
}
|
||||
if err := json.Unmarshal(e.Value, &rec); err != nil {
|
||||
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
|
||||
}
|
||||
return rec.State, rec.Warrant, nil
|
||||
}
|
||||
|
||||
// Handled is what came of a warrant: acted on, refused (and why), or already done.
|
||||
type Handled string
|
||||
|
||||
const (
|
||||
Acted Handled = "acted"
|
||||
Refused Handled = "refused"
|
||||
Done Handled = "already acted on"
|
||||
Recorded Handled = "ended without a choice"
|
||||
)
|
||||
|
||||
// Take hands act a warrant once, after holding it to the ask it answers. act performs the option with the
|
||||
// module's own grants; its error is said and the warrant stays acted on (a person's decision is not
|
||||
// performed twice; the module says the failure in its own record).
|
||||
func (c *Client) Take(w asks.Warrant, act func(asks.Option, asks.Warrant) error) (Handled, error) {
|
||||
if w.Asker != c.Module {
|
||||
return Refused, fmt.Errorf("a warrant for %s reached %s", w.Asker, c.Module)
|
||||
}
|
||||
e, err := c.State.Get(key(w.Ask))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if e == nil {
|
||||
return Refused, fmt.Errorf("%s holds no ask %s", c.Module, w.Ask)
|
||||
}
|
||||
var k Kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if k.Acted || k.Ended != "" {
|
||||
return Done, nil
|
||||
}
|
||||
if w.Outcome != asks.OutcomeChosen {
|
||||
k.Ended = w.Outcome
|
||||
k.Answer = &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil && !errors.Is(err, stdio.ErrChanged) {
|
||||
return "", err
|
||||
}
|
||||
return Recorded, nil
|
||||
}
|
||||
o, err := w.For(c.Module, k.Ask)
|
||||
if err != nil {
|
||||
return Refused, err
|
||||
}
|
||||
// Marked before acting: of two deliveries, or two instances, only the one whose write stands acts.
|
||||
k.Acted, k.Ended, k.Answer = true, asks.OutcomeChosen, &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return Done, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return Acted, act(o, w)
|
||||
}
|
||||
|
||||
// Listen hands every warrant for this module to Take, with act. It asks the runtime for the module's
|
||||
// events matching the router's `decided` subject for this module.
|
||||
func (c *Client) Listen(act func(asks.Option, asks.Warrant) error, said func(Handled, asks.Warrant, error)) error {
|
||||
subscribe := c.Subscribe
|
||||
if subscribe == nil {
|
||||
subscribe = stdio.Subscribe
|
||||
}
|
||||
return subscribe(asks.Seat+".decided."+c.Module, func(e stdio.Envelope) error {
|
||||
if !strings.HasPrefix(e.Key, asks.Seat+".decided.") {
|
||||
return nil
|
||||
}
|
||||
var w asks.Warrant
|
||||
if err := json.Unmarshal(e.Body, &w); err != nil {
|
||||
if said != nil {
|
||||
said(Refused, w, fmt.Errorf("a warrant could not be read: %w", err))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
h, err := c.Take(w, act)
|
||||
if h == "" && err != nil {
|
||||
return err // the state could not be reached: offered again
|
||||
}
|
||||
if said != nil {
|
||||
said(h, w, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
type mem struct {
|
||||
vals map[string]json.RawMessage
|
||||
revs map[string]uint64
|
||||
n uint64
|
||||
}
|
||||
|
||||
func (m *mem) Get(k string) (*stdio.StateEntry, error) {
|
||||
v, ok := m.vals[k]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: k, Value: v, Revision: m.revs[k]}, nil
|
||||
}
|
||||
func (m *mem) Create(k string, v any) (uint64, error) {
|
||||
if _, ok := m.vals[k]; ok {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) Update(k string, v any, rev uint64) (uint64, error) {
|
||||
if rev == 0 || m.revs[k] != rev {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) put(k string, v any) (uint64, error) {
|
||||
raw, _ := json.Marshal(v)
|
||||
m.n++
|
||||
m.vals[k], m.revs[k] = raw, m.n
|
||||
return m.n, nil
|
||||
}
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func client() (*Client, map[string][]string) {
|
||||
sent := map[string][]string{}
|
||||
c := &Client{Module: "mesh-delivery", State: &mem{vals: map[string]json.RawMessage{}, revs: map[string]uint64{}},
|
||||
Now: func() time.Time { return now },
|
||||
Publish: func(subject string, body any, id string) error {
|
||||
sent[subject] = append(sent[subject], id)
|
||||
return nil
|
||||
}}
|
||||
return c, sent
|
||||
}
|
||||
|
||||
// releaseHQ is the act the ask's one option binds.
|
||||
var releaseHQ = map[string]string{"verb": "release", "delivery": "hq"}
|
||||
|
||||
func held() asks.Ask {
|
||||
binds, _ := asks.ActDigest(releaseHQ)
|
||||
return asks.Ask{ID: "d-1", Headline: "Delivery of hq held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: asks.Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held",
|
||||
Options: []asks.Option{{ID: "release", Label: "Release", Does: "it goes on", Level: asks.Approve, Binds: binds}}}
|
||||
}
|
||||
|
||||
func warrant() asks.Warrant {
|
||||
return asks.Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: asks.OutcomeChosen, Option: "release",
|
||||
Label: "Release", Level: asks.Approve, At: now.Add(time.Minute), AskDigest: held().Digest(),
|
||||
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
|
||||
}
|
||||
|
||||
func TestAnAskIsKeptThenPublishedUnderTheModulesOwnName(t *testing.T) {
|
||||
c, sent := client()
|
||||
if err := c.Ask(held()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := sent[asks.AskSubject("mesh-delivery")]; len(got) != 1 || got[0] != "d-1" {
|
||||
t.Fatalf("%v", sent)
|
||||
}
|
||||
if err := c.Ask(held()); err == nil || !strings.Contains(err.Error(), "already made") {
|
||||
t.Errorf("the same id was asked twice: %v", err)
|
||||
}
|
||||
bad := held()
|
||||
bad.Who = "agent"
|
||||
if err := c.Ask(bad); err == nil {
|
||||
t.Error("an ask the contract refuses was published")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWarrantIsActedOnOnceAndOnlyForItsOwnAsk(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
acted := 0
|
||||
act := func(o asks.Option, _ asks.Warrant) error {
|
||||
if err := o.Performs(releaseHQ); err != nil {
|
||||
t.Errorf("the bound act: %v", err)
|
||||
}
|
||||
acted++
|
||||
return nil
|
||||
}
|
||||
if h, err := c.Take(warrant(), act); h != Acted || err != nil || acted != 1 {
|
||||
t.Fatalf("%s %v %d", h, err, acted)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), act); h != Done || acted != 1 {
|
||||
t.Errorf("a redelivery was acted on: %s", h)
|
||||
}
|
||||
for why, change := range map[string]func(*asks.Warrant){
|
||||
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-controller" },
|
||||
"an ask not made": func(w *asks.Warrant) { w.Ask = "d-9" },
|
||||
"another level": func(w *asks.Warrant) { w.Level = asks.Acknowledge },
|
||||
"an option not offered": func(w *asks.Warrant) { w.Option = "delete" },
|
||||
"after its expiry": func(w *asks.Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *asks.Warrant) { w.AskDigest = "sha256:00" },
|
||||
} {
|
||||
c2, _ := client()
|
||||
_ = c2.Ask(held())
|
||||
w := warrant()
|
||||
change(&w)
|
||||
if h, _ := c2.Take(w, act); h == Acted {
|
||||
t.Errorf("a warrant from %s was acted on", why)
|
||||
}
|
||||
}
|
||||
if acted != 1 {
|
||||
t.Errorf("acted %d times", acted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskEndedWithoutAChoiceIsRecordedAndNothingDone(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
w := warrant()
|
||||
w.Outcome, w.Option, w.By = asks.OutcomeExpired, "", nil
|
||||
if h, err := c.Take(w, func(asks.Option, asks.Warrant) error { t.Fatal("acted"); return nil }); h != Recorded || err != nil {
|
||||
t.Fatalf("%s %v", h, err)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), func(asks.Option, asks.Warrant) error { t.Fatal("acted after it ended"); return nil }); h != Done {
|
||||
t.Errorf("%s", h)
|
||||
}
|
||||
}
|
||||
+573
@@ -0,0 +1,573 @@
|
||||
// Package asks is the contract of asking a person and answering on a channel (novox/hq ADR 0259): the
|
||||
// shapes an asker, the router and a channel exchange on the bus, and the subjects they travel on. No
|
||||
// transport and no channel's service: an asker publishes an Ask under its own name and acts on the Warrant
|
||||
// it hears; the router holds the ask, sends channels a Message, and judges the Choice a channel says; a
|
||||
// channel shows a Message and says what was chosen and by whom, as its service authenticated it.
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The seats (novox/hq ADR 0259 §3).
|
||||
const (
|
||||
// Seat is held by the router: an ask and its cancel are its accepts, a warrant its event, each named by
|
||||
// the asker.
|
||||
Seat = "operator-channel"
|
||||
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
|
||||
ChannelSeat = "channel"
|
||||
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
|
||||
// the kind.
|
||||
IntakeSeat = "intake"
|
||||
)
|
||||
|
||||
// AskSubject is where an asker publishes an ask, CancelSubject its cancel, and DecidedSubject where it
|
||||
// hears the warrant, or the ask's end without one.
|
||||
func AskSubject(asker string) string { return "mesh.seat." + Seat + ".accept.ask." + asker }
|
||||
func CancelSubject(asker string) string { return "mesh.seat." + Seat + ".accept.cancel." + asker }
|
||||
func DecidedSubject(asker string) string { return "mesh.seat." + Seat + ".event.decided." + asker }
|
||||
|
||||
// The work a channel takes, on ChannelSubject.
|
||||
const (
|
||||
Show = "show" // a message offering answers
|
||||
Edit = "edit" // a message shown before, replaced
|
||||
Send = "send" // a message offering nothing
|
||||
)
|
||||
|
||||
// ChannelSubject is where the router sends a channel of a kind its work.
|
||||
func ChannelSubject(verb, kind string) string {
|
||||
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
|
||||
}
|
||||
|
||||
// What a channel says, on IntakeSubject.
|
||||
const (
|
||||
Chosen = "choice" // a button tapped
|
||||
Link = "link" // somebody asked to be linked as the operator
|
||||
)
|
||||
|
||||
// IntakeSubject is where a channel of a kind says what arrived.
|
||||
func IntakeSubject(what, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
|
||||
}
|
||||
|
||||
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
|
||||
const CodeProof = "code"
|
||||
|
||||
// ProofSubject is where a channel of a kind asks a proof.
|
||||
func ProofSubject(verb, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
|
||||
}
|
||||
|
||||
// A Level is how much proof an option's answer needs (novox/hq ADR 0234 §8, the glossary's assurance level).
|
||||
type Level string
|
||||
|
||||
const (
|
||||
// Acknowledge performs only what any granted principal may already do: silencing, details. No proof.
|
||||
Acknowledge Level = "acknowledge"
|
||||
// Approve needs one proof: a verified sender (a linked account, linked an hour or more) or a code.
|
||||
Approve Level = "approve"
|
||||
// Destroy needs two proofs, one of them a code.
|
||||
Destroy Level = "destroy"
|
||||
)
|
||||
|
||||
// Rank orders the levels; an unknown level ranks above every known one, so it is never taken as less.
|
||||
func (l Level) Rank() int {
|
||||
switch l {
|
||||
case Acknowledge:
|
||||
return 0
|
||||
case Approve:
|
||||
return 1
|
||||
case Destroy:
|
||||
return 2
|
||||
}
|
||||
return 3
|
||||
}
|
||||
|
||||
// Operator is the one role an ask may be answered by today.
|
||||
const Operator = "operator"
|
||||
|
||||
// Option is one answer an ask offers: a label for the button, what it does in plain words, its level.
|
||||
type Option struct {
|
||||
ID string `json:"id"`
|
||||
Label string `json:"label"`
|
||||
Does string `json:"does"`
|
||||
Level Level `json:"level"`
|
||||
// Binds is the digest of exactly what the asker performs when this option is chosen — the verb, the
|
||||
// machine and every argument — as ActDigest gives it. It travels in the ask, so the router's warrant,
|
||||
// which names the ask's digest, names it too: a warrant then authorises that act and no other, and an
|
||||
// asker whose record of the act changed after it asked finds the digests differ and does nothing.
|
||||
// Required on an option above acknowledge.
|
||||
Binds string `json:"binds,omitempty"`
|
||||
}
|
||||
|
||||
// An Act is what an option binds: named fields, each a string — the verb, the machine, the level, and each
|
||||
// argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over these names and
|
||||
// values alone, never over how a language happens to encode a struct.
|
||||
type Act map[string]string
|
||||
|
||||
// ActDigest is the digest an asker puts in Option.Binds: SHA-256 over the act's canonical encoding (canonical),
|
||||
// written "sha256:<hex>". The same names and values give the same digest in any language, whatever order
|
||||
// they were set in; a field renamed, added or emptied gives another.
|
||||
func ActDigest(act Act) (string, error) {
|
||||
if len(act) == 0 {
|
||||
return "", fmt.Errorf("the act cannot be digested: it names nothing")
|
||||
}
|
||||
keys := make([]string, 0, len(act))
|
||||
for k := range act {
|
||||
if k == "" {
|
||||
return "", fmt.Errorf("the act cannot be digested: a field has no name")
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.act.v1\n")
|
||||
for _, k := range keys {
|
||||
canonical(&b, k)
|
||||
canonical(&b, act[k])
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read
|
||||
// as another's end or start, so two different sequences of values never encode the same.
|
||||
func canonical(b *strings.Builder, v string) {
|
||||
b.WriteString(strconv.Itoa(len(v)))
|
||||
b.WriteByte(':')
|
||||
b.WriteString(v)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
|
||||
// Digest is the digest of the ask exactly as its asker published it: its id, words, options with what each
|
||||
// binds, who answers, and its expiry. The router puts it in the warrant (Warrant.AskDigest), and an asker
|
||||
// acts only on a warrant whose digest is that of the ask it keeps — so a warrant answers one ask, as the
|
||||
// person was shown it, and nothing published under the same id before or after.
|
||||
//
|
||||
// It is over the ask's named fields in a fixed order, each written canonically, and the expiry as UTC
|
||||
// RFC 3339 to the nanosecond — never over a language's encoding of the struct, so a field added to Ask
|
||||
// later changes no digest until it is added here, on purpose. Whole and Details (novox/hq issue 383) follow
|
||||
// the options only when the ask gives either: an ask without them digests as it did before they existed, so
|
||||
// a router and an asker of different builds still agree on every such ask.
|
||||
func (a Ask) Digest() string {
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.ask.v1\n")
|
||||
for _, v := range []string{a.ID, a.Headline, a.Explanation, a.Who,
|
||||
a.Expires.UTC().Format(time.RFC3339Nano), a.OnExpiry, a.About, strconv.FormatBool(a.Urgent),
|
||||
strconv.Itoa(len(a.Options))} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
for _, o := range a.Options {
|
||||
for _, v := range []string{o.ID, o.Label, o.Does, string(o.Level), o.Binds} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
}
|
||||
// Whole and Details are written whenever either or WholeDetails is given, and WholeDetails after them whenever
|
||||
// it is given (novox/hq ADR 0347): the fields after the options are then always one of three fixed shapes —
|
||||
// none, the two, or the three — and the count of values the sequence holds tells them apart, so no two asks
|
||||
// encode the same (the review of mesh-sdk #25). An ask with none of them digests as before they existed, and
|
||||
// one without WholeDetails as before it existed.
|
||||
if a.Whole != "" || a.Details != "" || a.WholeDetails != "" {
|
||||
canonical(&b, a.Whole)
|
||||
canonical(&b, a.Details)
|
||||
}
|
||||
if a.WholeDetails != "" {
|
||||
canonical(&b, a.WholeDetails)
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// Ask is a request for a person's word (novox/hq ADR 0259 §4).
|
||||
type Ask struct {
|
||||
// ID is the asker's own, unique to it.
|
||||
ID string `json:"id"`
|
||||
// Headline names the thing and what is wrong, in a few plain words; Explanation is what happened and
|
||||
// what it means. Both are held to the plain rule and the content rule by the router.
|
||||
Headline string `json:"headline"`
|
||||
Explanation string `json:"explanation"`
|
||||
Options []Option `json:"options"`
|
||||
// Who may answer: Operator.
|
||||
Who string `json:"who"`
|
||||
// Expires is when the ask ends unanswered; OnExpiry is what the asker then does, in words the person
|
||||
// is shown ("the delivery stays held"). An ask that authorises never defaults.
|
||||
Expires time.Time `json:"expires"`
|
||||
OnExpiry string `json:"on-expiry"`
|
||||
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
|
||||
About string `json:"about,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
// Whole is the explanation with its exact values whole, shown in place of Explanation on a channel kind
|
||||
// that proves who answers (verified-sender) and carries the ask's answers (novox/hq issue 383): what the
|
||||
// person approves — a mount point, a share, a private address — must be readable where they approve it.
|
||||
// The asker withholds a value shaped like a secret in it, and the router refuses the ask when one is left;
|
||||
// Explanation stays under the whole content rule everywhere else. Empty, Explanation is shown everywhere.
|
||||
Whole string `json:"whole,omitempty"`
|
||||
// Details are the ask's secondary words, line by line under the content rule: who proposed it, a
|
||||
// fingerprint, how to read it whole. The router shows them in the note of the ask's own message
|
||||
// (Message.Note), after what the ask's options do and when it expires — never in its body (novox/hq
|
||||
// issue 639).
|
||||
Details string `json:"details,omitempty"`
|
||||
// WholeDetails are the Details with their exact values whole, shown in place of Details where Whole is shown
|
||||
// in place of the explanation (a kind that proves who answers, given the ask's answers): every value an
|
||||
// approval sets, whole, so the person reads exactly what they approve (novox/hq issue 383). Held, like
|
||||
// Whole, to the secret rule alone; the router refuses the ask when a secret's shape is left in it. Empty,
|
||||
// Details are shown everywhere.
|
||||
WholeDetails string `json:"whole-details,omitempty"`
|
||||
}
|
||||
|
||||
// What an ask's explanation is held to, so it reads at a glance on a phone (novox/hq issue 639): a summary
|
||||
// of two or three short sentences on its first line — what changes, why, what it costs or risks — then at
|
||||
// most three points, a line each, only where they add something. Whole is held to the same. Who asked, when,
|
||||
// the fingerprint and how to read it whole are Details, never the explanation.
|
||||
const (
|
||||
SummaryLength = 300
|
||||
MostPoints = 3
|
||||
PointLength = 160
|
||||
)
|
||||
|
||||
// Glance says why an explanation does not read at a glance, or nil: its first line, the summary, longer than
|
||||
// SummaryLength; more than MostPoints lines after it; a point longer than PointLength. Blank lines are not
|
||||
// counted. An asker holds its asks to it in its own tests; the router says, without refusing, an ask that
|
||||
// does not hold to it.
|
||||
func Glance(explanation string) error {
|
||||
var lines []string
|
||||
for _, l := range strings.Split(explanation, "\n") {
|
||||
if strings.TrimSpace(l) != "" {
|
||||
lines = append(lines, strings.TrimSpace(l))
|
||||
}
|
||||
}
|
||||
var problems []string
|
||||
switch {
|
||||
case len(lines) == 0:
|
||||
problems = append(problems, "it has no summary")
|
||||
case len([]rune(lines[0])) > SummaryLength:
|
||||
problems = append(problems, fmt.Sprintf("its summary is %d characters, more than %d", len([]rune(lines[0])), SummaryLength))
|
||||
}
|
||||
if len(lines) > 1+MostPoints {
|
||||
problems = append(problems, fmt.Sprintf("it has %d points after its summary, more than %d", len(lines)-1, MostPoints))
|
||||
}
|
||||
for i, l := range lines {
|
||||
if i > 0 && len([]rune(l)) > PointLength {
|
||||
problems = append(problems, fmt.Sprintf("its point %d is %d characters, more than %d", i, len([]rune(l)), PointLength))
|
||||
}
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the explanation does not read at a glance: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
|
||||
const (
|
||||
MostOptions = 4
|
||||
MostOpen = 3
|
||||
ApproveLasts = 24 * time.Hour
|
||||
DestroyLasts = 10 * time.Minute
|
||||
HeadlineLength = 60
|
||||
LabelLength = 24
|
||||
)
|
||||
|
||||
var usableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`)
|
||||
|
||||
// UsableID says whether a name can be an ask's or an option's id: a key and a subject token both.
|
||||
func UsableID(id string) bool { return usableID.MatchString(id) }
|
||||
|
||||
// Highest is the highest level among the ask's options.
|
||||
func (a Ask) Highest() Level {
|
||||
high := Acknowledge
|
||||
for _, o := range a.Options {
|
||||
if o.Level.Rank() > high.Rank() {
|
||||
high = o.Level
|
||||
}
|
||||
}
|
||||
return high
|
||||
}
|
||||
|
||||
// Option is the option of this id, or false.
|
||||
func (a Ask) Option(id string) (Option, bool) {
|
||||
for _, o := range a.Options {
|
||||
if o.ID == id {
|
||||
return o, true
|
||||
}
|
||||
}
|
||||
return Option{}, false
|
||||
}
|
||||
|
||||
// Check is what an ask is held to before anything is shown: every refusal, in words its asker can act on.
|
||||
func (a Ask) Check(now time.Time) error {
|
||||
var problems []string
|
||||
say := func(format string, args ...any) { problems = append(problems, fmt.Sprintf(format, args...)) }
|
||||
if !UsableID(a.ID) {
|
||||
say("its id %q is not letters, digits, - and _, at most 64", a.ID)
|
||||
}
|
||||
if strings.TrimSpace(a.Headline) == "" || len([]rune(a.Headline)) > HeadlineLength {
|
||||
say("its headline is empty or longer than %d characters", HeadlineLength)
|
||||
}
|
||||
if strings.TrimSpace(a.Explanation) == "" {
|
||||
say("it explains nothing")
|
||||
}
|
||||
if a.Who != Operator {
|
||||
say("it is answered by %q, and only the operator answers today", a.Who)
|
||||
}
|
||||
if len(a.Options) == 0 || len(a.Options) > MostOptions {
|
||||
say("it offers %d options, and an ask offers one to %d", len(a.Options), MostOptions)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, o := range a.Options {
|
||||
switch {
|
||||
case !UsableID(o.ID):
|
||||
say("an option's id %q is not letters, digits, - and _", o.ID)
|
||||
case seen[o.ID]:
|
||||
say("the option %s is offered twice", o.ID)
|
||||
}
|
||||
seen[o.ID] = true
|
||||
if strings.TrimSpace(o.Label) == "" || len([]rune(o.Label)) > LabelLength {
|
||||
say("the option %s's label is empty or longer than %d characters", o.ID, LabelLength)
|
||||
}
|
||||
if strings.TrimSpace(o.Does) == "" {
|
||||
say("the option %s does not say what it does", o.ID)
|
||||
}
|
||||
if o.Level.Rank() > Destroy.Rank() {
|
||||
say("the option %s has the level %q, which is none of acknowledge, approve, destroy", o.ID, o.Level)
|
||||
}
|
||||
if o.Level != Acknowledge && !strings.HasPrefix(o.Binds, "sha256:") {
|
||||
say("the option %s authorises and does not bind what it performs (its binds is not an ActDigest)", o.ID)
|
||||
}
|
||||
}
|
||||
if !a.Expires.After(now) {
|
||||
say("it expires before it is asked")
|
||||
}
|
||||
switch a.Highest() {
|
||||
case Approve:
|
||||
if a.Expires.After(now.Add(ApproveLasts)) {
|
||||
say("an ask that approves lasts at most %s", ApproveLasts)
|
||||
}
|
||||
case Destroy:
|
||||
if a.Expires.After(now.Add(DestroyLasts)) {
|
||||
say("an ask that destroys lasts at most %s", DestroyLasts)
|
||||
}
|
||||
}
|
||||
if a.Highest() != Acknowledge && strings.TrimSpace(a.OnExpiry) == "" {
|
||||
say("it does not say what happens when nobody answers, and an ask that authorises never defaults")
|
||||
}
|
||||
if a.About != "" && strings.ContainsAny(a.About, " \n") {
|
||||
say("what it is about is a key, without spaces")
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the ask is refused: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Outcome is how an ask ended.
|
||||
type Outcome string
|
||||
|
||||
const (
|
||||
OutcomeChosen Outcome = "chosen" // a person chose an option: a warrant
|
||||
OutcomeExpired Outcome = "expired" // nobody answered in time
|
||||
OutcomeCancelled Outcome = "cancelled" // its asker took it back
|
||||
OutcomeReplaced Outcome = "replaced" // a newer ask about the same thing replaced it
|
||||
OutcomeRefused Outcome = "refused" // it was never shown: Words says why
|
||||
)
|
||||
|
||||
// Person is who chose, as the router verified them.
|
||||
type Person struct {
|
||||
// Who is the role: Operator.
|
||||
Who string `json:"who"`
|
||||
// Kind is the channel kind they answered on, Identity their account on that service, Display the name
|
||||
// the service shows, and Verified how the router knew it was them.
|
||||
Kind string `json:"kind"`
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Verified string `json:"verified"`
|
||||
}
|
||||
|
||||
// Warrant is the router's record that a person chose one option of one ask, or the ask's end without one
|
||||
// (novox/hq ADR 0259 §6). It carries no secret.
|
||||
type Warrant struct {
|
||||
Ask string `json:"ask"`
|
||||
Asker string `json:"asker"`
|
||||
About string `json:"about,omitempty"`
|
||||
Outcome Outcome `json:"outcome"`
|
||||
// Option, Label and Level are the option chosen; By who chose it, Channel the module it came through,
|
||||
// Proofs which proofs were present (P1, P2, P3).
|
||||
Option string `json:"option,omitempty"`
|
||||
Label string `json:"label,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
By *Person `json:"by,omitempty"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
Proofs []string `json:"proofs,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
// AskDigest is the digest of the ask as the router took it (Ask.Digest): the warrant answers that ask
|
||||
// alone, with the options it bound.
|
||||
AskDigest string `json:"ask-digest,omitempty"`
|
||||
// Words are why an ask ended without a choice, or what refused it.
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
// Says is the warrant in the words an asker records with its act: "the operator, via telegram (user id
|
||||
// verified), chose Release".
|
||||
func (w Warrant) Says() string {
|
||||
if w.Outcome != OutcomeChosen || w.By == nil {
|
||||
return fmt.Sprintf("no person chose: the ask %s %s", w.Ask, w.Outcome)
|
||||
}
|
||||
via := w.By.Kind
|
||||
if w.By.Verified != "" {
|
||||
via += " (" + w.By.Verified + ")"
|
||||
}
|
||||
return fmt.Sprintf("the %s, via %s, chose %s", w.By.Who, via, w.Label)
|
||||
}
|
||||
|
||||
// For checks a warrant against the ask its asker made: the same asker and ask, the same ask's digest (so the
|
||||
// same words, options and binds), a choice, an option the ask offered, at that option's level, before the
|
||||
// ask expired. An asker acts on nothing else, and then performs only what the option's Binds names.
|
||||
func (w Warrant) For(asker string, a Ask) (Option, error) {
|
||||
if w.Asker != asker || w.Ask != a.ID {
|
||||
return Option{}, fmt.Errorf("the warrant is for %s's ask %s, not %s's %s", w.Asker, w.Ask, asker, a.ID)
|
||||
}
|
||||
if w.Outcome != OutcomeChosen || w.By == nil || w.By.Who != a.Who {
|
||||
return Option{}, fmt.Errorf("the ask %s ended %s; no person chose", a.ID, w.Outcome)
|
||||
}
|
||||
if d := a.Digest(); w.AskDigest != d {
|
||||
return Option{}, fmt.Errorf("the warrant answers an ask whose digest is %q, and the ask %s kept here is %s: "+
|
||||
"it was not the ask the person was shown", w.AskDigest, a.ID, d)
|
||||
}
|
||||
o, offered := a.Option(w.Option)
|
||||
if !offered {
|
||||
return Option{}, fmt.Errorf("the ask %s offered no option %s", a.ID, w.Option)
|
||||
}
|
||||
if w.Level != o.Level {
|
||||
return Option{}, fmt.Errorf("the option %s is %s, and the warrant says %s", o.ID, o.Level, w.Level)
|
||||
}
|
||||
// A choice made after the ask expired is no answer to it, whatever the router said. An ask that says no
|
||||
// expiry, or a warrant that says no time, is no answer either: neither can be shown to be in time (the
|
||||
// confirmation review of 2026-10-09).
|
||||
if a.Expires.IsZero() {
|
||||
return Option{}, fmt.Errorf("the ask %s says no expiry, so no answer to it can be in time", a.ID)
|
||||
}
|
||||
if w.At.IsZero() {
|
||||
return Option{}, fmt.Errorf("the warrant for the ask %s says no time it was given, so it cannot be shown to be in time", a.ID)
|
||||
}
|
||||
if w.At.After(a.Expires) {
|
||||
return Option{}, fmt.Errorf("the warrant was given at %s, after the ask %s expired at %s",
|
||||
w.At.UTC().Format(time.RFC3339), a.ID, a.Expires.UTC().Format(time.RFC3339))
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Performs checks that the act an asker is about to perform is the one the chosen option bound when it
|
||||
// asked: the act's digest equals the option's Binds. An acknowledge option that bound nothing passes.
|
||||
func (o Option) Performs(act Act) error {
|
||||
if o.Binds == "" && o.Level == Acknowledge {
|
||||
return nil
|
||||
}
|
||||
d, err := ActDigest(act)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d != o.Binds {
|
||||
return fmt.Errorf("the option %s bound %s, and the act about to be performed is %s: nothing is done", o.ID, o.Binds, d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
|
||||
type Button struct {
|
||||
Label string `json:"label"`
|
||||
Ticket string `json:"ticket"`
|
||||
}
|
||||
|
||||
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
|
||||
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
|
||||
// which of its own messages that is. The words are the router's, shown as given.
|
||||
type Message struct {
|
||||
Handle string `json:"handle"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Buttons []Button `json:"buttons,omitempty"`
|
||||
// Note is the message's secondary words, after its body and its buttons: who asked and when, what each
|
||||
// answer does, when it expires, and the ask's Details. A channel shows it smaller, collapsed or set apart
|
||||
// where it can, and after the body where it cannot (novox/hq issue 639). A channel that bounds a message's
|
||||
// length cuts the body first and the note only when the title and the note alone do not fit, saying each
|
||||
// cut (novox/hq ADR 0347 §5).
|
||||
Note string `json:"note,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
Silent bool `json:"silent,omitempty"`
|
||||
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
|
||||
Reply string `json:"reply,omitempty"`
|
||||
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
|
||||
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
|
||||
To string `json:"to,omitempty"`
|
||||
// Secret says the words carry something shown once (a link's code): the channel shows it and keeps no
|
||||
// copy of it — no state, no history of its own.
|
||||
Secret bool `json:"secret,omitempty"`
|
||||
}
|
||||
|
||||
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
|
||||
// service authenticated it. The router alone judges whether that is the operator.
|
||||
type Sender struct {
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Authenticated bool `json:"authenticated"`
|
||||
}
|
||||
|
||||
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
|
||||
type Failed struct {
|
||||
Handle string `json:"handle"`
|
||||
Why string `json:"why"`
|
||||
Permanent bool `json:"permanent,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
|
||||
// whether it can send now, why not, and whether its edits notify nobody.
|
||||
type Standing struct {
|
||||
Ready bool `json:"ready"`
|
||||
Why string `json:"why,omitempty"`
|
||||
EditsSilently bool `json:"edits-silently,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// What a channel says of its own delivery, on IntakeSubject.
|
||||
const (
|
||||
FailedWhat = "failed"
|
||||
StandingWhat = "standing"
|
||||
)
|
||||
|
||||
// Choice is a button chosen on a channel.
|
||||
type Choice struct {
|
||||
// ID is the channel's own for this arrival, unique, so the router acts on it once.
|
||||
ID string `json:"id"`
|
||||
Ticket string `json:"ticket"`
|
||||
Handle string `json:"handle,omitempty"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// LinkAsked is somebody on a channel asking to be linked as the operator.
|
||||
type LinkAsked struct {
|
||||
ID string `json:"id"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
|
||||
type Code struct {
|
||||
Sender Sender `json:"sender"`
|
||||
Code string `json:"code"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
|
||||
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
|
||||
type ProofAnswer struct {
|
||||
Accepted bool `json:"accepted"`
|
||||
Words string `json:"words"`
|
||||
}
|
||||
@@ -0,0 +1,367 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func held() Ask {
|
||||
return Ask{ID: "d-1", Headline: "openrazer delivery held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held", About: "delivery.openrazer",
|
||||
Options: []Option{
|
||||
{ID: "release", Label: "Release", Does: "the delivery goes on", Level: Approve, Binds: bind("release", "openrazer")},
|
||||
{ID: "stop", Label: "Stop", Does: "the delivery ends", Level: Approve, Binds: bind("stop", "openrazer")},
|
||||
{ID: "silence", Label: "Silence for a week", Does: "nothing more is said for a week", Level: Acknowledge},
|
||||
}}
|
||||
}
|
||||
|
||||
func bind(verb, what string) string {
|
||||
d, err := ActDigest(Act{"verb": verb, "delivery": what})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func TestASoundAskPasses(t *testing.T) {
|
||||
if err := held().Check(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskIsRefusedInWordsForEachFault(t *testing.T) {
|
||||
for want, change := range map[string]func(*Ask){
|
||||
"is not letters": func(a *Ask) { a.ID = "a.b" },
|
||||
"headline is empty": func(a *Ask) { a.Headline = "" },
|
||||
"explains nothing": func(a *Ask) { a.Explanation = " " },
|
||||
"only the operator answers": func(a *Ask) { a.Who = "agent" },
|
||||
"offers 0 options": func(a *Ask) { a.Options = nil },
|
||||
"offers 5 options": func(a *Ask) { a.Options = append(a.Options, a.Options[0], a.Options[1]) },
|
||||
"is offered twice": func(a *Ask) { a.Options[1].ID = "release" },
|
||||
"does not say what it does": func(a *Ask) { a.Options[0].Does = "" },
|
||||
"none of acknowledge": func(a *Ask) { a.Options[0].Level = "admin" },
|
||||
"expires before it is asked": func(a *Ask) { a.Expires = now },
|
||||
"approves lasts at most": func(a *Ask) { a.Expires = now.Add(25 * time.Hour) },
|
||||
"destroys lasts at most": func(a *Ask) { a.Options[0].Level = Destroy },
|
||||
"never defaults": func(a *Ask) { a.OnExpiry = "" },
|
||||
"is a key, without spaces": func(a *Ask) { a.About = "a b" },
|
||||
"does not bind what it": func(a *Ask) { a.Options[0].Binds = "" },
|
||||
} {
|
||||
a := held()
|
||||
change(&a)
|
||||
if err := a.Check(now); err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("want a refusal saying %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
ack := held()
|
||||
ack.Options = ack.Options[2:]
|
||||
ack.OnExpiry = ""
|
||||
ack.Expires = now.Add(7 * 24 * time.Hour)
|
||||
if err := ack.Check(now); err != nil {
|
||||
t.Errorf("an ask that only acknowledges needs no expiry words and may last a week: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskerActsOnlyOnAWarrantForItsOwnAskAndOption(t *testing.T) {
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
|
||||
AskDigest: held().Digest(), At: now.Add(time.Minute)}
|
||||
if o, err := w.For("mesh-delivery", held()); err != nil || o.ID != "release" {
|
||||
t.Fatalf("a sound warrant was refused: %v", err)
|
||||
}
|
||||
if got := w.Says(); got != "the operator, via telegram (user id verified), chose Release" {
|
||||
t.Errorf("the warrant says %q", got)
|
||||
}
|
||||
for why, change := range map[string]func(*Warrant){
|
||||
"another asker": func(w *Warrant) { w.Asker = "mesh-controller" },
|
||||
"another ask": func(w *Warrant) { w.Ask = "d-2" },
|
||||
"no choice": func(w *Warrant) { w.Outcome = OutcomeExpired },
|
||||
"nobody": func(w *Warrant) { w.By = nil },
|
||||
"not offered": func(w *Warrant) { w.Option = "delete" },
|
||||
"a lower level": func(w *Warrant) { w.Level = Acknowledge },
|
||||
"another role": func(w *Warrant) { w.By = &Person{Who: "agent"} },
|
||||
"no digest": func(w *Warrant) { w.AskDigest = "" },
|
||||
"no time given": func(w *Warrant) { w.At = time.Time{} },
|
||||
"after expiry": func(w *Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *Warrant) {
|
||||
other := held()
|
||||
other.Options[0].Binds = bind("release", "another")
|
||||
w.AskDigest = other.Digest()
|
||||
},
|
||||
} {
|
||||
c := w
|
||||
by := *w.By
|
||||
c.By = &by
|
||||
change(&c)
|
||||
if _, err := c.For("mesh-delivery", held()); err == nil {
|
||||
t.Errorf("a warrant with %s was acted on", why)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSubjectsNameTheirPublisher(t *testing.T) {
|
||||
for got, want := range map[string]string{
|
||||
AskSubject("mesh-delivery"): "mesh.seat.operator-channel.accept.ask.mesh-delivery",
|
||||
DecidedSubject("mesh-delivery"): "mesh.seat.operator-channel.event.decided.mesh-delivery",
|
||||
ChannelSubject(Show, "telegram"): "mesh.seat.channel.accept.show.telegram",
|
||||
IntakeSubject(Chosen, "desktop"): "mesh.seat.intake.event.choice.desktop",
|
||||
ProofSubject(CodeProof, "telegram"): "mesh.seat.intake.proof.code.telegram",
|
||||
} {
|
||||
if got != want {
|
||||
t.Errorf("%s, want %s", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A warrant answers one ask as it was shown, and authorises the act its option bound and no other
|
||||
// (novox/hq ADR 0259 §6: the warrant is bound to the exact ask and its arguments).
|
||||
func TestAWarrantAuthorisesOnlyTheActItsOptionBound(t *testing.T) {
|
||||
a := held()
|
||||
// The digest survives the trip through the bus: what the router decodes and digests is what was sent.
|
||||
raw, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var decoded Ask
|
||||
if err := json.Unmarshal(raw, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Digest() != a.Digest() {
|
||||
t.Fatalf("the ask's digest changed on the way: %s, %s", decoded.Digest(), a.Digest())
|
||||
}
|
||||
local := a
|
||||
local.Expires = a.Expires.In(time.FixedZone("CEST", 2*3600))
|
||||
if local.Digest() != a.Digest() {
|
||||
t.Error("the same instant in another zone digests differently")
|
||||
}
|
||||
// Changing anything the person was shown, or what an option binds, changes the digest.
|
||||
for what, change := range map[string]func(*Ask){
|
||||
"the headline": func(a *Ask) { a.Headline += "!" },
|
||||
"the explanation": func(a *Ask) { a.Explanation += " " },
|
||||
"a label": func(a *Ask) { a.Options[0].Label = "Go" },
|
||||
"what it binds": func(a *Ask) { a.Options[0].Binds = bind("release", "another") },
|
||||
"the expiry": func(a *Ask) { a.Expires = a.Expires.Add(time.Second) },
|
||||
"what it is about": func(a *Ask) { a.About = "delivery.other" },
|
||||
} {
|
||||
c := held()
|
||||
change(&c)
|
||||
if c.Digest() == a.Digest() {
|
||||
t.Errorf("changing %s leaves the digest", what)
|
||||
}
|
||||
}
|
||||
o, _ := a.Option("release")
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "openrazer"}); err != nil {
|
||||
t.Errorf("the bound act was refused: %v", err)
|
||||
}
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "another"}); err == nil {
|
||||
t.Error("an act the option did not bind was let through")
|
||||
}
|
||||
ack, _ := a.Option("silence")
|
||||
if err := ack.Performs(Act{"anything": "x"}); err != nil {
|
||||
t.Errorf("an acknowledgement that bound nothing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The digests are over named fields in a canonical encoding (the review of 2026-10-09): not over Go's
|
||||
// encoding of a struct, so a field added to Ask or Option, a JSON tag renamed, or a map built in another
|
||||
// order changes nothing it should not — and every field that is shown or bound is in it.
|
||||
func TestDigestsAreCanonicalOverNamedFields(t *testing.T) {
|
||||
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
|
||||
b := Act{}
|
||||
b["arg.delivery"] = "hq"
|
||||
b["verb"] = "release"
|
||||
a2, _ := ActDigest(b)
|
||||
if a1 != a2 {
|
||||
t.Error("the order an act was built in changed its digest")
|
||||
}
|
||||
// A value cannot be read as two: "a"+"b=c" is not "a=b"+"c".
|
||||
x, _ := ActDigest(Act{"a": "b\n1:c"})
|
||||
y, _ := ActDigest(Act{"a": "b", "c": ""})
|
||||
if x == y {
|
||||
t.Error("two different acts encode the same")
|
||||
}
|
||||
if _, err := ActDigest(Act{}); err == nil {
|
||||
t.Error("an empty act was digested")
|
||||
}
|
||||
// The encoding, by hand: any language reproduces it.
|
||||
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
|
||||
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Errorf("the act's encoding is not the one written down: %s", got)
|
||||
}
|
||||
|
||||
// Every exported field of Ask and Option is in the digest: changing any one changes it.
|
||||
base := held()
|
||||
mutations := map[string]func(*Ask){
|
||||
"ID": func(a *Ask) { a.ID = "other" },
|
||||
"Headline": func(a *Ask) { a.Headline += "!" },
|
||||
"Explanation": func(a *Ask) { a.Explanation += "!" },
|
||||
"Who": func(a *Ask) { a.Who = "someone" },
|
||||
"Expires": func(a *Ask) { a.Expires = a.Expires.Add(time.Nanosecond) },
|
||||
"OnExpiry": func(a *Ask) { a.OnExpiry += "!" },
|
||||
"About": func(a *Ask) { a.About += "!" },
|
||||
"Urgent": func(a *Ask) { a.Urgent = !a.Urgent },
|
||||
"Options": func(a *Ask) { a.Options = a.Options[:1] },
|
||||
"Option.ID": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].ID = "x" },
|
||||
"Option.Label": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Label += "!"
|
||||
},
|
||||
"Option.Does": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].Does += "!" },
|
||||
"Option.Level": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Level = Destroy
|
||||
},
|
||||
"Option.Binds": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Binds = "sha256:00"
|
||||
},
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
c := base
|
||||
mutate(&c)
|
||||
if c.Digest() == base.Digest() {
|
||||
t.Errorf("changing %s left the digest the same", name)
|
||||
}
|
||||
}
|
||||
// Whole and Details (issue 383) are in the digest once given, and change it; an ask without either digests
|
||||
// exactly as before they existed — the encoding by hand, so no build can drift from it unseen.
|
||||
plain := Ask{ID: "p", Headline: "h", Explanation: "e", Who: Operator, Expires: now, OnExpiry: "o", About: "a",
|
||||
Options: []Option{{ID: "ok", Label: "OK", Does: "d", Level: Acknowledge}}}
|
||||
byHand := sha256.Sum256([]byte("novox.ask.v1\n1:p\n1:h\n1:e\n8:operator\n20:2026-10-08T14:00:00Z\n1:o\n1:a\n5:false\n1:1\n" +
|
||||
"2:ok\n2:OK\n1:d\n11:acknowledge\n0:\n"))
|
||||
if plain.Digest() != "sha256:"+hex.EncodeToString(byHand[:]) {
|
||||
t.Errorf("an ask without Whole and Details does not digest as it did before them: %s", plain.Digest())
|
||||
}
|
||||
withWhole, withDetails := plain, plain
|
||||
withWhole.Whole = "e, whole"
|
||||
withDetails.Details = "fingerprint 1234"
|
||||
if withWhole.Digest() == plain.Digest() || withDetails.Digest() == plain.Digest() || withWhole.Digest() == withDetails.Digest() {
|
||||
t.Error("Whole or Details left the digest the same")
|
||||
}
|
||||
both := withWhole
|
||||
both.Details = "fingerprint 1234"
|
||||
changed := both
|
||||
changed.Whole += "!"
|
||||
if changed.Digest() == both.Digest() {
|
||||
t.Error("changing Whole beside Details left the digest the same")
|
||||
}
|
||||
// A field added to Ask or Option without being added to Digest fails here.
|
||||
if n := reflect.TypeOf(Ask{}).NumField(); n != 12 {
|
||||
t.Errorf("Ask has %d fields, and Digest covers 12: add the new one to Digest and to this test", n)
|
||||
}
|
||||
if n := reflect.TypeOf(Option{}).NumField(); n != 5 {
|
||||
t.Errorf("Option has %d fields, and Digest covers 5: add the new one to Digest and to this test", n)
|
||||
}
|
||||
// The same instant in another zone is the same ask.
|
||||
z := base
|
||||
z.Expires = z.Expires.In(time.FixedZone("x", 3600))
|
||||
if z.Digest() != base.Digest() {
|
||||
t.Error("the expiry's zone changed the digest")
|
||||
}
|
||||
}
|
||||
|
||||
// An ask that says no expiry is answered by no warrant (the confirmation review of 2026-10-09): nothing could
|
||||
// show the answer was in time.
|
||||
func TestAWarrantForAnAskWithNoExpiryIsRefused(t *testing.T) {
|
||||
a := held()
|
||||
a.Expires = time.Time{}
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42"}, AskDigest: a.Digest(),
|
||||
At: now.Add(time.Minute)}
|
||||
if _, err := w.For("mesh-delivery", a); err == nil {
|
||||
t.Error("a warrant for an ask with no expiry was taken")
|
||||
}
|
||||
}
|
||||
|
||||
// An explanation that reads at a glance passes; a wall of text says each way it is one (novox/hq issue 639).
|
||||
func TestAnExplanationReadsAtAGlance(t *testing.T) {
|
||||
if err := Glance("Your agents' memory follows you to every machine. A file changed on two keeps both.\n\n• Only the memory folders are carried\n• A conflict reaches you as a warning"); err != nil {
|
||||
t.Fatalf("a summary and two points: %v", err)
|
||||
}
|
||||
var wall strings.Builder
|
||||
wall.WriteString("ADR 0343 was proposed by fix-367 and approved by the knowledge base's own review.")
|
||||
for i := 1; i <= 13; i++ {
|
||||
wall.WriteString("\n" + strings.Repeat("A clause lead. ", 3))
|
||||
}
|
||||
err := Glance(wall.String())
|
||||
if err == nil || !strings.Contains(err.Error(), "13 points") {
|
||||
t.Fatalf("thirteen clause lines pass as a glance: %v", err)
|
||||
}
|
||||
if err := Glance(strings.Repeat("word ", 70)); err == nil || !strings.Contains(err.Error(), "summary is 349") {
|
||||
t.Fatalf("a long summary passes: %v", err)
|
||||
}
|
||||
if err := Glance("short.\n" + strings.Repeat("p", PointLength+1)); err == nil || !strings.Contains(err.Error(), "point 1") {
|
||||
t.Fatalf("a long point passes: %v", err)
|
||||
}
|
||||
if err := Glance(" \n"); err == nil {
|
||||
t.Fatal("an empty explanation passes")
|
||||
}
|
||||
}
|
||||
|
||||
// A message's note travels beside its body, and is left out of the encoding when empty, so a channel older
|
||||
// than it reads every message it read before.
|
||||
func TestAMessageCarriesItsNoteApart(t *testing.T) {
|
||||
raw, _ := json.Marshal(Message{Handle: "h", Title: "t", Body: "b", Note: "Asked by x."})
|
||||
if !strings.Contains(string(raw), `"note":"Asked by x."`) {
|
||||
t.Fatalf("the note is not encoded: %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(Message{Handle: "h", Title: "t"})
|
||||
if strings.Contains(string(raw), "note") {
|
||||
t.Fatalf("an empty note is encoded: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// WholeDetails is digested when given, and an ask without it digests as before it existed: a router and an asker
|
||||
// of different builds still agree on every such ask.
|
||||
func TestWholeDetailsAreDigestedOnlyWhenGiven(t *testing.T) {
|
||||
a := Ask{ID: "a", Headline: "h", Explanation: "e", Details: "d"}
|
||||
before := a.Digest()
|
||||
a.WholeDetails = "+ shares: media=/storage/media"
|
||||
if a.Digest() == before {
|
||||
t.Fatal("the whole details are not in the digest")
|
||||
}
|
||||
b := a
|
||||
b.WholeDetails = "+ shares: media=/storage/other"
|
||||
if a.Digest() == b.Digest() {
|
||||
t.Fatal("two whole details digest the same")
|
||||
}
|
||||
a.WholeDetails = ""
|
||||
if a.Digest() != before {
|
||||
t.Fatal("an ask without whole details digests otherwise than before")
|
||||
}
|
||||
}
|
||||
|
||||
// No two asks encode the same (the review of mesh-sdk #25): whole details given beside nothing else are not read as
|
||||
// a Whole and Details that happen to spell the same values, nor as any other split of the three fields.
|
||||
func TestWholeDetailsCollideWithNoOtherAsk(t *testing.T) {
|
||||
base := Ask{ID: "a", Headline: "h", Explanation: "e"}
|
||||
shapes := []Ask{}
|
||||
for _, f := range []struct{ whole, details, wholeDetails string }{
|
||||
{"", "", "+ x: 1"},
|
||||
{"whole-details", "+ x: 1", ""},
|
||||
{"", "+ x: 1", ""},
|
||||
{"+ x: 1", "", ""},
|
||||
{"", "", "whole-details"},
|
||||
{"", "+ x: 1", "+ x: 1"},
|
||||
{"+ x: 1", "+ x: 1", ""},
|
||||
{"", "", ""},
|
||||
} {
|
||||
a := base
|
||||
a.Whole, a.Details, a.WholeDetails = f.whole, f.details, f.wholeDetails
|
||||
shapes = append(shapes, a)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
for i, a := range shapes {
|
||||
if j, ok := seen[a.Digest()]; ok {
|
||||
t.Errorf("asks %d and %d digest the same: %+v / %+v", j, i, shapes[j], a)
|
||||
}
|
||||
seen[a.Digest()] = i
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ConditionSubject is where a module says a condition to the operator: a by-caller accept of the operator-channel
|
||||
// seat, `mesh.seat.operator-channel.accept.condition.<publisher>` (novox/hq ADR 0437 §9). The router reads the
|
||||
// verb at the fifth part and the caller at the sixth, so the publisher is one token, the module's own name; it is
|
||||
// refused when empty or when it holds a dot, a wildcard or whitespace.
|
||||
func ConditionSubject(publisher string) (string, error) {
|
||||
if publisher == "" {
|
||||
return "", errors.New("asks: the condition's publisher is empty")
|
||||
}
|
||||
if strings.ContainsAny(publisher, ".*> \t\r\n") || strings.ContainsFunc(publisher, func(r rune) bool { return r <= ' ' }) {
|
||||
return "", fmt.Errorf("asks: the condition's publisher %q is not one token: it holds a dot, a wildcard or whitespace", publisher)
|
||||
}
|
||||
return "mesh.seat." + Seat + ".accept.condition." + publisher, nil
|
||||
}
|
||||
|
||||
// ConditionVersion is the version of the condition value this package writes and the only one it reads.
|
||||
const ConditionVersion = 1
|
||||
|
||||
// ErrUnknownConditionVersion is what DecodeCondition returns for a value whose version is missing or not
|
||||
// ConditionVersion. Test it with errors.Is.
|
||||
var ErrUnknownConditionVersion = errors.New("asks: unknown condition version")
|
||||
|
||||
// The words a condition's explanation opens with (novox/hq ADR 0253 §1).
|
||||
const (
|
||||
NothingToDo = "Nothing for you to do."
|
||||
NeedsYou = "Needs you: "
|
||||
)
|
||||
|
||||
// The severities and states of a condition.
|
||||
const (
|
||||
SeverityUrgent = "urgent"
|
||||
SeverityWarning = "warning"
|
||||
StateRaised = "raised"
|
||||
StateCleared = "cleared"
|
||||
)
|
||||
|
||||
// Scope is what a condition is about, under the controller's names.
|
||||
type Scope struct {
|
||||
Scope string `json:"scope"`
|
||||
ID string `json:"id"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
}
|
||||
|
||||
// Action is one answer a notification offers, under the controller's names.
|
||||
type Action struct {
|
||||
Label string `json:"label"`
|
||||
Verb string `json:"verb"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
Arguments map[string]string `json:"arguments,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
}
|
||||
|
||||
// Condition is a condition a module says to the operator, under the names of the controller's conditions: its key,
|
||||
// severity, subject, state (raised or cleared), summary and the plain words of ADR 0253. A cleared condition needs
|
||||
// only its key, subject and state. The router alone chooses the channels.
|
||||
type Condition struct {
|
||||
V int `json:"v"`
|
||||
Key string `json:"key"`
|
||||
Severity string `json:"severity,omitempty"`
|
||||
Subject Scope `json:"subject"`
|
||||
State string `json:"state"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Headline string `json:"headline,omitempty"`
|
||||
Needs string `json:"needs,omitempty"`
|
||||
Explanation string `json:"explanation,omitempty"`
|
||||
Resolved string `json:"resolved,omitempty"`
|
||||
Actions []Action `json:"actions,omitempty"`
|
||||
}
|
||||
|
||||
func (c Condition) check() error {
|
||||
if c.Key == "" {
|
||||
return errors.New("asks: a condition has no key")
|
||||
}
|
||||
if c.Subject.Scope == "" || c.Subject.ID == "" {
|
||||
return errors.New("asks: a condition's subject needs its scope and id")
|
||||
}
|
||||
switch c.State {
|
||||
case StateCleared:
|
||||
return nil
|
||||
case StateRaised:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's state %q is not raised or cleared", c.State)
|
||||
}
|
||||
switch c.Severity {
|
||||
case SeverityUrgent, SeverityWarning:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's severity %q is not urgent or warning", c.Severity)
|
||||
}
|
||||
if c.Headline == "" {
|
||||
return errors.New("asks: a raised condition has no headline")
|
||||
}
|
||||
if n := len([]rune(c.Headline)); n > HeadlineLength {
|
||||
return fmt.Errorf("asks: the condition's headline is %d characters, more than %d", n, HeadlineLength)
|
||||
}
|
||||
verdict := NothingToDo
|
||||
if c.Needs != "" {
|
||||
verdict = NeedsYou + c.Needs
|
||||
}
|
||||
if !strings.HasPrefix(c.Explanation, verdict) {
|
||||
return fmt.Errorf("asks: the condition's explanation must open with %q", verdict)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeCondition is c as JSON at ConditionVersion; it refuses what ADR 0253 and ADR 0437 §9 refuse.
|
||||
func EncodeCondition(c Condition) ([]byte, error) {
|
||||
if err := c.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
return json.Marshal(c)
|
||||
}
|
||||
|
||||
// DecodeCondition reads a condition; it refuses an unknown version (ErrUnknownConditionVersion) and what
|
||||
// EncodeCondition refuses.
|
||||
func DecodeCondition(data []byte) (Condition, error) {
|
||||
var probe struct {
|
||||
V *int `json:"v"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &probe); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition is not JSON: %w", err)
|
||||
}
|
||||
if probe.V == nil {
|
||||
return Condition{}, fmt.Errorf("%w: no version in the value, this reader reads %d", ErrUnknownConditionVersion, ConditionVersion)
|
||||
}
|
||||
if *probe.V != ConditionVersion {
|
||||
return Condition{}, fmt.Errorf("%w: the value is version %d, this reader reads %d", ErrUnknownConditionVersion, *probe.V, ConditionVersion)
|
||||
}
|
||||
var c Condition
|
||||
if err := json.Unmarshal(data, &c); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition does not fit its shape: %w", err)
|
||||
}
|
||||
if err := c.check(); err != nil {
|
||||
return Condition{}, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// SameCondition says whether two conditions are the same value.
|
||||
func SameCondition(a, b Condition) bool { return reflect.DeepEqual(a, b) }
|
||||
@@ -0,0 +1,143 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestConditionSubject(t *testing.T) { // 1202/AC1
|
||||
s, err := ConditionSubject("mesh-delivery-teller")
|
||||
if err != nil || s != "mesh.seat.operator-channel.accept.condition.mesh-delivery-teller" {
|
||||
t.Fatalf("subject = %q, %v", s, err)
|
||||
}
|
||||
// The router splits an accept subject in six parts and reads the verb at 4 and the caller at 5.
|
||||
parts := strings.Split(s, ".")
|
||||
if len(parts) != 6 || parts[4] != "condition" || parts[5] != "mesh-delivery-teller" {
|
||||
t.Errorf("subject %q does not keep the router's shape: %v", s, parts)
|
||||
}
|
||||
for _, bad := range []string{"", "a.b", "a*", "a>", "a b", "a\tb", "a\nb"} {
|
||||
if _, err := ConditionSubject(bad); err == nil {
|
||||
t.Errorf("publisher %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func raisedCondition() Condition {
|
||||
return Condition{
|
||||
Key: "delivery.stuck.mesh-mail", Severity: "warning",
|
||||
Subject: Scope{Scope: "module", ID: "mesh-mail", Machine: "ace"}, State: "raised",
|
||||
Summary: "plan 12 stopped", Headline: "Delivery of mesh-mail is stuck",
|
||||
Needs: "approve the merge.", Explanation: "Needs you: approve the merge. The plan waits.",
|
||||
Resolved: "The delivery of mesh-mail moved on.",
|
||||
Actions: []Action{{Label: "Approve", Verb: "mesh-controller.assign", Machine: "ace",
|
||||
Arguments: map[string]string{"why": ""}, Level: Approve}},
|
||||
}
|
||||
}
|
||||
|
||||
func clearedCondition() Condition {
|
||||
return Condition{Key: "delivery.stuck.mesh-mail", Subject: Scope{Scope: "module", ID: "mesh-mail"}, State: "cleared"}
|
||||
}
|
||||
|
||||
func TestConditionRoundTrip(t *testing.T) { // 1202/AC2
|
||||
for _, c := range []Condition{raisedCondition(), clearedCondition()} {
|
||||
data, err := EncodeCondition(c)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", c.State, err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if err := json.Unmarshal(data, &probe); err != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("%s: %s does not carry v:1", c.State, data)
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
got, err := DecodeCondition(data)
|
||||
if err != nil || !SameCondition(got, c) {
|
||||
t.Errorf("%s round trip: %+v %v", c.State, got, err)
|
||||
}
|
||||
}
|
||||
data, _ := EncodeCondition(raisedCondition())
|
||||
var probe map[string]any
|
||||
_ = json.Unmarshal(data, &probe)
|
||||
for _, name := range []string{"key", "severity", "subject", "summary", "headline", "needs", "explanation", "resolved", "actions", "state"} {
|
||||
if _, ok := probe[name]; !ok {
|
||||
t.Errorf("the JSON has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
subject := probe["subject"].(map[string]any)
|
||||
for _, name := range []string{"scope", "id", "machine"} {
|
||||
if _, ok := subject[name]; !ok {
|
||||
t.Errorf("the subject has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
action := probe["actions"].([]any)[0].(map[string]any)
|
||||
for _, name := range []string{"label", "verb", "machine", "arguments", "level"} {
|
||||
if _, ok := action[name]; !ok {
|
||||
t.Errorf("the action has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
for _, in := range []string{`{"key":"k"}`, `{"v":2,"key":"k"}`} {
|
||||
if _, err := DecodeCondition([]byte(in)); !errors.Is(err, ErrUnknownConditionVersion) {
|
||||
t.Errorf("decode of %s = %v, want ErrUnknownConditionVersion", in, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionVerdict(t *testing.T) { // 1202/AC3
|
||||
c := raisedCondition()
|
||||
c.Needs, c.Explanation = "", "The plan waits."
|
||||
_, err := EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Nothing for you to do.") {
|
||||
t.Errorf("empty needs without its verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
if _, err := EncodeCondition(c); err != nil {
|
||||
t.Errorf("empty needs with its verdict refused: %v", err)
|
||||
}
|
||||
c = raisedCondition()
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
_, err = EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Needs you: approve the merge.") {
|
||||
t.Errorf("needs set with the wrong verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Needs you: something else. The plan waits."
|
||||
if _, err = EncodeCondition(c); err == nil {
|
||||
t.Error("needs set with other needs in the verdict accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionRefused(t *testing.T) { // 1202/AC4
|
||||
mutate := func(f func(*Condition)) error {
|
||||
c := raisedCondition()
|
||||
f(&c)
|
||||
_, err := EncodeCondition(c)
|
||||
return err
|
||||
}
|
||||
for name, f := range map[string]func(*Condition){
|
||||
"severity": func(c *Condition) { c.Severity = "critical" },
|
||||
"state": func(c *Condition) { c.State = "open" },
|
||||
"key": func(c *Condition) { c.Key = "" },
|
||||
"headline": func(c *Condition) { c.Headline = "" },
|
||||
"long headline": func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength+1) },
|
||||
} {
|
||||
if err := mutate(f); err == nil {
|
||||
t.Errorf("a condition with a bad %s accepted", name)
|
||||
}
|
||||
}
|
||||
if err := mutate(func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength) }); err != nil {
|
||||
t.Errorf("a headline of the greatest length refused: %v", err)
|
||||
}
|
||||
if _, err := EncodeCondition(clearedCondition()); err != nil {
|
||||
t.Errorf("a cleared condition with only key, subject and state refused: %v", err)
|
||||
}
|
||||
c := clearedCondition()
|
||||
c.Key = ""
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a cleared condition without its key accepted")
|
||||
}
|
||||
c = clearedCondition()
|
||||
c.State = "gone"
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a condition with an unknown state accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
// Option changes FetchCommit.
|
||||
type Option func(*fetchOptions)
|
||||
|
||||
type fetchOptions struct{ allowFile bool }
|
||||
|
||||
// AllowFileRemote lets FetchCommit take a file:// remote. It is for a test's local repository; production
|
||||
// code does not pass it, and FetchCommit then refuses anything but ssh://.
|
||||
func AllowFileRemote() Option { return func(o *fetchOptions) { o.allowFile = true } }
|
||||
|
||||
var (
|
||||
commitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
|
||||
refOK = regexp.MustCompile(`^refs/[A-Za-z0-9._/-]+$`)
|
||||
)
|
||||
|
||||
// FetchCommit fetches one commit of a forge repository into dir, an empty directory, and checks it out detached
|
||||
// (novox/hq ADR 0466 §4, §5; issue 1474). remote is ssh://<ssh name>/<owner>/<repo>.git, the ssh name being the
|
||||
// one the build module's templates set up (ADR 0409 §1); commit is a full 40-hex sha; ref is optional, and names
|
||||
// the pull request's head (refs/pull/<n>/head) when the commit is only there. Every git command runs through run
|
||||
// as `git -C dir ...` with GIT_TERMINAL_PROMPT=0 and no credential helper, so nothing is asked for and no
|
||||
// credential is used. A remote that is not ssh://, or one with user information, and a commit that is not 40
|
||||
// hex digits are refused before anything runs.
|
||||
func FetchCommit(ctx context.Context, run forgekey.Runner, dir, remote, commit, ref string, opts ...Option) error {
|
||||
var o fetchOptions
|
||||
for _, opt := range opts {
|
||||
opt(&o)
|
||||
}
|
||||
if err := checkFetch(o, dir, remote, commit, ref); err != nil {
|
||||
return err
|
||||
}
|
||||
env := []string{"GIT_TERMINAL_PROMPT=0"}
|
||||
git := func(args ...string) (string, error) {
|
||||
full := append([]string{"-c", "credential.helper=", "-C", dir}, args...)
|
||||
r := run(ctx, env, "git", full...)
|
||||
if r.Status != 0 || r.Err != "" {
|
||||
return "", fmt.Errorf("git %s: %s", args[0], firstLine(r.Err+" "+r.Stderr))
|
||||
}
|
||||
return r.Stdout, nil
|
||||
}
|
||||
if _, err := git("init", "-q"); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := git("remote", "add", "origin", remote); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := git("fetch", "--no-tags", "--depth", "1", "origin", commit); err != nil {
|
||||
if ref == "" {
|
||||
return err
|
||||
}
|
||||
if _, rerr := git("fetch", "--no-tags", "origin", ref); rerr != nil {
|
||||
return fmt.Errorf("%v; then the ref %s: %w", err, ref, rerr)
|
||||
}
|
||||
if _, verr := git("cat-file", "-e", commit+"^{commit}"); verr != nil {
|
||||
return fmt.Errorf("the commit %s is not in %s: %w", commit, ref, verr)
|
||||
}
|
||||
}
|
||||
if _, err := git("checkout", "-q", "--detach", commit); err != nil {
|
||||
return err
|
||||
}
|
||||
out, err := git("rev-parse", "HEAD")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if head := strings.TrimSpace(out); head != commit {
|
||||
return fmt.Errorf("the checkout is at %s, not at the commit %s", head, commit)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkFetch(o fetchOptions, dir, remote, commit, ref string) error {
|
||||
if dir == "" {
|
||||
return fmt.Errorf("buildstep: no workspace directory")
|
||||
}
|
||||
if !commitOK.MatchString(commit) {
|
||||
return fmt.Errorf("buildstep: %q is not a full commit (40 hex digits)", commit)
|
||||
}
|
||||
if ref != "" && !refOK.MatchString(ref) {
|
||||
return fmt.Errorf("buildstep: %q is not a ref (refs/...)", ref)
|
||||
}
|
||||
u, err := url.Parse(remote)
|
||||
if err != nil {
|
||||
return fmt.Errorf("buildstep: the remote is not a URL: %w", err)
|
||||
}
|
||||
if u.User != nil {
|
||||
return fmt.Errorf("buildstep: the remote carries user information; the key is the credential")
|
||||
}
|
||||
switch {
|
||||
case u.Scheme == "ssh" && u.Host != "":
|
||||
case u.Scheme == "file" && o.allowFile:
|
||||
default:
|
||||
return fmt.Errorf("buildstep: the remote %q is not ssh://", remote)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func firstLine(s string) string {
|
||||
for _, l := range strings.Split(s, "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" {
|
||||
return l
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
//go:build integration
|
||||
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
func gitIn(t *testing.T, dir string, args ...string) string {
|
||||
t.Helper()
|
||||
cmd := exec.Command("git", append([]string{"-C", dir, "-c", "user.name=t", "-c", "user.email=t@example.invalid"}, args...)...)
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("git %v: %v\n%s", args, err, out)
|
||||
}
|
||||
return strings.TrimSpace(string(out))
|
||||
}
|
||||
|
||||
// 1474/AC5: a local bare repository with a commit only under refs/pull/1/head (a file:// remote, allowed by the
|
||||
// test-only option): the workspace's HEAD is the commit. The forge itself is proven live after the build
|
||||
// modules' delivery (ADR 0466 §5).
|
||||
func TestFetchCommitFromAPullRefOnly(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
bare := filepath.Join(root, "r.git")
|
||||
work := filepath.Join(root, "work")
|
||||
for _, d := range []string{bare, work} {
|
||||
if err := os.MkdirAll(d, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
gitIn(t, bare, "init", "-q", "--bare")
|
||||
gitIn(t, work, "init", "-q")
|
||||
if err := os.WriteFile(filepath.Join(work, "f"), []byte("x"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
gitIn(t, work, "add", "f")
|
||||
gitIn(t, work, "commit", "-q", "-m", "pull request head")
|
||||
commit := gitIn(t, work, "rev-parse", "HEAD")
|
||||
gitIn(t, work, "push", "-q", bare, "HEAD:refs/pull/1/head")
|
||||
|
||||
ws := filepath.Join(root, "ws")
|
||||
if err := os.MkdirAll(ws, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := FetchCommit(context.Background(), forgekey.ExecRunner, ws, "file://"+bare, commit, "refs/pull/1/head", AllowFileRemote())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if head := gitIn(t, ws, "rev-parse", "HEAD"); head != commit {
|
||||
t.Errorf("HEAD = %s, want %s", head, commit)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/forgekey"
|
||||
)
|
||||
|
||||
const sha = "0123456789abcdef0123456789abcdef01234567"
|
||||
|
||||
// fakeGit records every command and answers by a rule.
|
||||
type fakeGit struct {
|
||||
calls []string
|
||||
envs [][]string
|
||||
rule func(args string) forgekey.Ran
|
||||
}
|
||||
|
||||
func (f *fakeGit) run(ctx context.Context, env []string, name string, args ...string) forgekey.Ran {
|
||||
line := name + " " + strings.Join(args, " ")
|
||||
f.calls = append(f.calls, line)
|
||||
f.envs = append(f.envs, env)
|
||||
if f.rule != nil {
|
||||
if r := f.rule(line); r.Status != 0 || r.Stdout != "" || r.Err != "" {
|
||||
return r
|
||||
}
|
||||
}
|
||||
if len(args) > 4 && args[4] == "rev-parse" {
|
||||
return forgekey.Ran{Stdout: sha + "\n"}
|
||||
}
|
||||
return forgekey.Ran{}
|
||||
}
|
||||
|
||||
const pre = "git -c credential.helper= -C /w "
|
||||
|
||||
// 1474/AC1: the git commands are exactly the sequence, none with a credential or an https remote.
|
||||
func TestFetchCommitRunsTheSequence(t *testing.T) {
|
||||
f := &fakeGit{}
|
||||
if err := FetchCommit(context.Background(), f.run, "/w", "ssh://mesh-forge/novox/mesh-sdk.git", sha, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := []string{
|
||||
pre + "init -q",
|
||||
pre + "remote add origin ssh://mesh-forge/novox/mesh-sdk.git",
|
||||
pre + "fetch --no-tags --depth 1 origin " + sha,
|
||||
pre + "checkout -q --detach " + sha,
|
||||
pre + "rev-parse HEAD",
|
||||
}
|
||||
if strings.Join(f.calls, "\n") != strings.Join(want, "\n") {
|
||||
t.Errorf("commands:\n%s\nwant:\n%s", strings.Join(f.calls, "\n"), strings.Join(want, "\n"))
|
||||
}
|
||||
for i, c := range f.calls {
|
||||
if strings.Contains(c, "https://") || strings.Contains(c, "@") || !strings.Contains(c, "credential.helper= ") {
|
||||
t.Errorf("command %q carries a credential or https", c)
|
||||
}
|
||||
if len(f.envs[i]) != 1 || f.envs[i][0] != "GIT_TERMINAL_PROMPT=0" {
|
||||
t.Errorf("env %v", f.envs[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1474/AC2: the depth-1 fetch fails and a pull ref is given: the ref is fetched and the commit verified; with no
|
||||
// ref, the fetch's failure is the answer.
|
||||
func TestFetchCommitFallsBackToThePullRef(t *testing.T) {
|
||||
f := &fakeGit{rule: func(l string) forgekey.Ran {
|
||||
if strings.Contains(l, "--depth 1") {
|
||||
return forgekey.Ran{Status: 128, Stderr: "fatal: couldn't find remote ref " + sha + "\n"}
|
||||
}
|
||||
return forgekey.Ran{}
|
||||
}}
|
||||
if err := FetchCommit(context.Background(), f.run, "/w", "ssh://mesh-forge/o/r.git", sha, "refs/pull/1/head"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := []string{pre + "fetch --no-tags origin refs/pull/1/head", pre + "cat-file -e " + sha + "^{commit}", pre + "checkout -q --detach " + sha}
|
||||
got := strings.Join(f.calls[3:6], "\n")
|
||||
if got != strings.Join(want, "\n") {
|
||||
t.Errorf("after the failed fetch:\n%s\nwant:\n%s", got, strings.Join(want, "\n"))
|
||||
}
|
||||
|
||||
g := &fakeGit{rule: f.rule}
|
||||
err := FetchCommit(context.Background(), g.run, "/w", "ssh://mesh-forge/o/r.git", sha, "")
|
||||
if err == nil || !strings.Contains(err.Error(), "couldn't find remote ref") || len(g.calls) != 3 {
|
||||
t.Errorf("without a ref: %v after %d commands", err, len(g.calls))
|
||||
}
|
||||
|
||||
h := &fakeGit{rule: func(l string) forgekey.Ran {
|
||||
if strings.Contains(l, "--depth 1") || strings.Contains(l, "cat-file") {
|
||||
return forgekey.Ran{Status: 1, Stderr: "no"}
|
||||
}
|
||||
return forgekey.Ran{}
|
||||
}}
|
||||
if err := FetchCommit(context.Background(), h.run, "/w", "ssh://mesh-forge/o/r.git", sha, "refs/pull/1/head"); err == nil {
|
||||
t.Error("a commit that is not in the ref was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// 1474/AC3: HEAD differs from the commit after the checkout: an error naming both.
|
||||
func TestFetchCommitAnswersAHeadThatIsNotTheCommit(t *testing.T) {
|
||||
other := strings.Repeat("a", 40)
|
||||
f := &fakeGit{rule: func(l string) forgekey.Ran {
|
||||
if strings.HasSuffix(l, "rev-parse HEAD") {
|
||||
return forgekey.Ran{Stdout: other + "\n"}
|
||||
}
|
||||
return forgekey.Ran{}
|
||||
}}
|
||||
err := FetchCommit(context.Background(), f.run, "/w", "ssh://mesh-forge/o/r.git", sha, "")
|
||||
if err == nil || !strings.Contains(err.Error(), other) || !strings.Contains(err.Error(), sha) {
|
||||
t.Errorf("error = %v, want both shas", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 1474/AC4: an https remote, user information, a file remote without the option, or a commit that is not 40
|
||||
// hex digits is refused before anything runs.
|
||||
func TestFetchCommitRefusesBeforeRunningAnything(t *testing.T) {
|
||||
for _, c := range []struct{ remote, commit, ref string }{
|
||||
{"https://forge.example/o/r.git", sha, ""},
|
||||
{"http://forge.example/o/r.git", sha, ""},
|
||||
{"ssh://git@mesh-forge/o/r.git", sha, ""},
|
||||
{"ssh://user:pw@mesh-forge/o/r.git", sha, ""},
|
||||
{"file:///tmp/r.git", sha, ""},
|
||||
{"/tmp/r.git", sha, ""},
|
||||
{"ssh://mesh-forge/o/r.git", "abc123", ""},
|
||||
{"ssh://mesh-forge/o/r.git", strings.ToUpper(sha), ""},
|
||||
{"ssh://mesh-forge/o/r.git", "main", ""},
|
||||
{"ssh://mesh-forge/o/r.git", sha, "--upload-pack=x"},
|
||||
} {
|
||||
f := &fakeGit{}
|
||||
if err := FetchCommit(context.Background(), f.run, "/w", c.remote, c.commit, c.ref); err == nil || len(f.calls) != 0 {
|
||||
t.Errorf("%+v: err %v, %d commands", c, err, len(f.calls))
|
||||
}
|
||||
}
|
||||
f := &fakeGit{}
|
||||
if err := FetchCommit(context.Background(), f.run, "/w", "file:///tmp/r.git", sha, "", AllowFileRemote()); err != nil {
|
||||
t.Errorf("a file remote with the option: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// Guard is a build module's check that the toolchain installed on its node is the one its recipe builds with
|
||||
// (novox/hq ADR 0469 §1; issue 1473). The same guard serves ADR 0470 §4's postgres major for the integration
|
||||
// level, with its own Probe and Want.
|
||||
type Guard struct {
|
||||
// Language is the toolchain's key part: go or typescript (delivery.ToolchainKey).
|
||||
Language string
|
||||
// Want is the toolchain version the recipe builds with, and Recipe the recipe's own version. The indexer
|
||||
// compares both texts as they are written here.
|
||||
Want, Recipe string
|
||||
// Probe reads the installed version (go version, node --version).
|
||||
Probe func(ctx context.Context) (installed string, err error)
|
||||
Node string
|
||||
// Put is the toolchains state's Put (stdio.State(delivery.ToolchainsState).Put).
|
||||
Put Put
|
||||
}
|
||||
|
||||
// Check probes the installed version, writes it to <node>.<language> of the toolchains state every time it runs,
|
||||
// and answers nil when it is the recipe's. Otherwise it answers a *Refusal naming both versions; the installed
|
||||
// version is written even then, so the indexer sees that builders disagree. A probe that fails is a *Refusal
|
||||
// naming its failure, and nothing is written. A write that fails is an error that is not a Refusal.
|
||||
func (g Guard) Check(ctx context.Context) error {
|
||||
installed, err := g.Probe(ctx)
|
||||
if err != nil {
|
||||
return &Refusal{Text: fmt.Sprintf("the installed %s version could not be read: %v", g.Language, err), Want: g.Want}
|
||||
}
|
||||
key, err := delivery.ToolchainKey(g.Node, g.Language)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
raw, err := delivery.EncodeToolchain(delivery.Toolchain{Language: g.Language, Version: installed, Recipe: g.Recipe, At: now().UTC()})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := g.Put(key, json.RawMessage(raw)); err != nil {
|
||||
return fmt.Errorf("buildstep: writing the toolchain %s: %w", key, err)
|
||||
}
|
||||
if installed != g.Want {
|
||||
return &Refusal{Text: fmt.Sprintf("%s %s installed, the recipe builds with %s %s", g.Language, installed, g.Language, g.Want),
|
||||
Want: g.Want, Have: installed}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
type toolchainKeys map[string]delivery.Toolchain
|
||||
|
||||
func (k toolchainKeys) put(key string, value any) (uint64, error) {
|
||||
tc, err := delivery.DecodeToolchain(value.(json.RawMessage))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
k[key] = tc
|
||||
return uint64(len(k)), nil
|
||||
}
|
||||
|
||||
func guardFor(k toolchainKeys, installed string, probeErr error) Guard {
|
||||
return Guard{Language: "go", Want: "1.27.0", Recipe: "r7", Node: "ace", Put: k.put,
|
||||
Probe: func(context.Context) (string, error) { return installed, probeErr }}
|
||||
}
|
||||
|
||||
// 1473/AC1: the probe answers the recipe's version: nil, and <node>.go holds that version and the recipe's.
|
||||
func TestCheckPassesAndWritesTheToolchain(t *testing.T) {
|
||||
k := toolchainKeys{}
|
||||
if err := guardFor(k, "1.27.0", nil).Check(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tc, ok := k["ace.go"]
|
||||
if !ok || tc.Version != "1.27.0" || tc.Recipe != "r7" || tc.Language != "go" || tc.At.IsZero() {
|
||||
t.Errorf("keys = %+v", k)
|
||||
}
|
||||
}
|
||||
|
||||
// 1473/AC2: another version: a Refusal naming both, and the key holds the installed version.
|
||||
func TestCheckRefusesAnotherVersionAndWritesTheInstalledOne(t *testing.T) {
|
||||
k := toolchainKeys{}
|
||||
err := guardFor(k, "1.27.1", nil).Check(context.Background())
|
||||
var ref *Refusal
|
||||
if !errors.As(err, &ref) {
|
||||
t.Fatalf("Check = %v, want a Refusal", err)
|
||||
}
|
||||
if ref.Text != "go 1.27.1 installed, the recipe builds with go 1.27.0" || ref.Want != "1.27.0" || ref.Have != "1.27.1" {
|
||||
t.Errorf("refusal = %+v", ref)
|
||||
}
|
||||
if k["ace.go"].Version != "1.27.1" {
|
||||
t.Errorf("the key holds %q, want the installed 1.27.1", k["ace.go"].Version)
|
||||
}
|
||||
}
|
||||
|
||||
// 1473/AC3: a probe that fails: a Refusal naming the failure, and nothing put.
|
||||
func TestCheckRefusesAFailedProbeAndPutsNothing(t *testing.T) {
|
||||
k := toolchainKeys{}
|
||||
err := guardFor(k, "", errors.New("go: not found")).Check(context.Background())
|
||||
var ref *Refusal
|
||||
if !errors.As(err, &ref) || !strings.Contains(ref.Text, "go: not found") {
|
||||
t.Fatalf("Check = %v, want a Refusal naming the probe's failure", err)
|
||||
}
|
||||
if len(k) != 0 {
|
||||
t.Errorf("put %v", k)
|
||||
}
|
||||
}
|
||||
|
||||
// A failed write is an error, not a refusal: the step fails instead of refusing.
|
||||
func TestCheckAnswersAFailedWriteAsAnError(t *testing.T) {
|
||||
g := guardFor(toolchainKeys{}, "1.27.0", nil)
|
||||
g.Put = func(string, any) (uint64, error) { return 0, errors.New("no grant") }
|
||||
err := g.Check(context.Background())
|
||||
var ref *Refusal
|
||||
if err == nil || errors.As(err, &ref) {
|
||||
t.Errorf("Check = %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Pack tars and gzips a directory the way the build agent does, byte for byte, and answers the archive and its
|
||||
// digest, "sha256:" and the hex of the archive's bytes (issue 1479; novox/hq ADR 0405 §7, ADR 0453 §7). It is a
|
||||
// copy of mesh-controller's internal/builder/builder.go pack (origin/main 073ccc29, :1351-1425), which stays
|
||||
// where it is until the build agent goes: a build module's digest is compared with the build agent's for one
|
||||
// fingerprint, so the bytes must be equal.
|
||||
//
|
||||
// Deterministically: regular files only (a symlink or a device is refused), sorted relative paths with
|
||||
// slashes, each header carrying its name, permission bits and size and nothing else, the gzip default writer.
|
||||
func Pack(root string) (archive []byte, digest string, err error) {
|
||||
raw, err := pack(root)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
sum := sha256.Sum256(raw)
|
||||
return raw, "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
func pack(root string) ([]byte, error) {
|
||||
info, err := os.Stat(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("%s is not a directory", root)
|
||||
}
|
||||
|
||||
var paths []string
|
||||
err = filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.IsDir() || !info.Mode().IsRegular() {
|
||||
// Only files. A symlink or a device in an archive is refused by the host that unpacks
|
||||
// it, so putting one in would build something that cannot be applied.
|
||||
if !info.IsDir() && !info.Mode().IsRegular() {
|
||||
return fmt.Errorf("%s is neither a file nor a directory, and an archive carries "+
|
||||
"only those", path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
paths = append(paths, path)
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// filepath.Walk walks in lexical order; the sort is kept as the build agent keeps it, since an archive
|
||||
// whose digest changes because the traversal did is a silent failure.
|
||||
sort.Strings(paths)
|
||||
|
||||
var out bytes.Buffer
|
||||
zipped := gzip.NewWriter(&out)
|
||||
writer := tar.NewWriter(zipped)
|
||||
for _, path := range paths {
|
||||
body, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
relative, err := filepath.Rel(root, path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
info, err := os.Stat(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mode := int64(info.Mode().Perm())
|
||||
if err := writer.WriteHeader(&tar.Header{
|
||||
Name: filepath.ToSlash(relative), Mode: mode, Size: int64(len(body)),
|
||||
Typeflag: tar.TypeReg,
|
||||
// Everything else left at its zero value on purpose: no times, owners or names.
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := writer.Write(body); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := zipped.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fixture writes three files in two directories (modes 0644 and 0755).
|
||||
func fixture(t *testing.T) string {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
for _, f := range []struct {
|
||||
p string
|
||||
mode os.FileMode
|
||||
body string
|
||||
}{{"a/x.txt", 0o644, "x\n"}, {"a/run.sh", 0o755, "#!/bin/sh\necho hi\n"}, {"b/c/y.txt", 0o644, "why\n"}} {
|
||||
path := filepath.Join(root, f.p)
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(f.body), f.mode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(path, f.mode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
// 1479/AC1: the tar stream inside the archive equals the bytes mesh-controller's pack makes for this tree.
|
||||
// testdata/pack-golden.tar is the uncompressed stream, made once by running mesh-controller
|
||||
// internal/builder/builder.go pack (origin/main 073ccc29, :1351-1425) on this fixture (above, modes set by chmod)
|
||||
// and gunzipping its archive. The gzip layer is not compared byte for byte, since compress/gzip's output differs
|
||||
// between Go versions; its header is checked to carry no name, no comment and no time.
|
||||
func TestPackEqualsTheBuildAgentsArchive(t *testing.T) {
|
||||
golden, err := os.ReadFile("testdata/pack-golden.tar")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
archive, digest, err := Pack(fixture(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
zr, err := gzip.NewReader(bytes.NewReader(archive))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if zr.Name != "" || zr.Comment != "" || !zr.ModTime.IsZero() {
|
||||
t.Errorf("gzip header = name %q, comment %q, time %v; want none", zr.Name, zr.Comment, zr.ModTime)
|
||||
}
|
||||
tarBytes, err := io.ReadAll(zr)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !bytes.Equal(tarBytes, golden) {
|
||||
t.Errorf("the tar stream is %d bytes, the build agent's %d, and they differ", len(tarBytes), len(golden))
|
||||
}
|
||||
sum := sha256.Sum256(archive)
|
||||
if digest != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Errorf("digest = %s, not the sha256 of the archive", digest)
|
||||
}
|
||||
}
|
||||
|
||||
// 1479/AC2: touching every file's time changes nothing.
|
||||
func TestPackDigestIgnoresFileTimes(t *testing.T) {
|
||||
root := fixture(t)
|
||||
_, first, err := Pack(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
later := time.Now().Add(48 * time.Hour)
|
||||
filepath.Walk(root, func(p string, _ os.FileInfo, _ error) error { return os.Chtimes(p, later, later) })
|
||||
if _, second, err := Pack(root); err != nil || second != first {
|
||||
t.Errorf("digests %s and %s (%v)", first, second, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
//go:build unix
|
||||
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1479/AC3: anything but a file or a directory is refused, naming it. The rule of this repository is that no
|
||||
// test makes a symlink, so the non-regular entry is a named pipe: Pack refuses a symlink on the same branch
|
||||
// (filepath.Walk reports it by lstat as not regular).
|
||||
func TestPackRefusesAnEntryThatIsNotAFile(t *testing.T) {
|
||||
root := fixture(t)
|
||||
pipe := filepath.Join(root, "a", "pipe")
|
||||
if err := syscall.Mkfifo(pipe, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, _, err := Pack(root)
|
||||
if err == nil || !strings.Contains(err.Error(), pipe) {
|
||||
t.Errorf("Pack = %v, want a refusal naming %s", err, pipe)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
package buildstep
|
||||
|
||||
// Refusal is a step that is not run on purpose, as against one that failed: the toolchain installed is not the
|
||||
// recipe's, the integration level's postgres major is not the one it is built for (novox/hq ADR 0469 §1, ADR 0470
|
||||
// §4; issue 1473). A step's run returns one and the runner ends the step refused with its text.
|
||||
type Refusal struct {
|
||||
// Text is the reason, in words the operator reads.
|
||||
Text string
|
||||
// Want is the version the recipe asks for and Have the one found; either may be empty.
|
||||
Want, Have string
|
||||
}
|
||||
|
||||
func (r *Refusal) Error() string { return r.Text }
|
||||
@@ -0,0 +1,99 @@
|
||||
// Package buildstep is what both build modules share: the runner of a long step that answers a tool call at once
|
||||
// and ends on its key in the module's `steps` state (novox/hq ADR 0469 §2; issue 1471), the toolchain guard
|
||||
// (issue 1473) and the fetch of one commit over SSH (issue 1474).
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// Put writes one key of a state: stdio.State(delivery.StepsState).Put's shape (go/state.go).
|
||||
type Put func(key string, value any) (uint64, error)
|
||||
|
||||
// EndAttempts and EndRetry bound how a step's end is written: a write that fails is tried again, since the end
|
||||
// is the only way the step's reader learns it is over.
|
||||
var (
|
||||
EndAttempts = 5
|
||||
EndRetry = 2 * time.Second
|
||||
)
|
||||
|
||||
var now = time.Now
|
||||
|
||||
// NewID is a step id: the time in milliseconds in base 36 and six random hex digits, lower-case letters and
|
||||
// digits only, so it is one key token and differs between two steps of one node even in one millisecond.
|
||||
func NewID() string {
|
||||
var b [3]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return strconv.FormatInt(now().UnixMilli(), 36) + hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// Start begins a step of tool on node and answers its id at once: the key <node>.<id> holds state running when
|
||||
// Start returns. run is then called in a goroutine of its own, under ctx (the process's, not the tool call's: the
|
||||
// call is cut at 30 s, the step is not). When it returns, the key is written passed with run's result, failed with
|
||||
// the error's text, or refused when the error is a *Refusal; a panic in run ends the step failed, naming it, and
|
||||
// the process goes on. If the running write fails, nothing was started and Start answers the error.
|
||||
func Start(ctx context.Context, put Put, node, tool string, run func(ctx context.Context) (result any, err error)) (string, error) {
|
||||
step := delivery.Step{ID: NewID(), Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC()}
|
||||
key, err := delivery.StepKey(node, step.ID)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := write(put, key, step); err != nil {
|
||||
return "", fmt.Errorf("buildstep: writing the running step %s: %w", key, err)
|
||||
}
|
||||
go func() {
|
||||
result, err := protect(ctx, run)
|
||||
step.Ended = now().UTC()
|
||||
switch ref := (*Refusal)(nil); {
|
||||
case err == nil:
|
||||
step.State = delivery.StepPassed
|
||||
if result != nil {
|
||||
raw, merr := json.Marshal(result)
|
||||
if merr != nil {
|
||||
step.State, step.Reason = delivery.StepFailed, "the result is not JSON: "+merr.Error()
|
||||
} else {
|
||||
step.Result = raw
|
||||
}
|
||||
}
|
||||
case errors.As(err, &ref):
|
||||
step.State, step.Reason = delivery.StepRefused, err.Error()
|
||||
default:
|
||||
step.State, step.Reason = delivery.StepFailed, err.Error()
|
||||
}
|
||||
for attempt := 1; ; attempt++ {
|
||||
if werr := write(put, key, step); werr == nil || attempt >= EndAttempts {
|
||||
return
|
||||
}
|
||||
time.Sleep(EndRetry)
|
||||
}
|
||||
}()
|
||||
return step.ID, nil
|
||||
}
|
||||
|
||||
// protect is run with a panic turned into the error that ends the step.
|
||||
func protect(ctx context.Context, run func(ctx context.Context) (any, error)) (result any, err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
result, err = nil, fmt.Errorf("the step panicked: %v", p)
|
||||
}
|
||||
}()
|
||||
return run(ctx)
|
||||
}
|
||||
|
||||
func write(put Put, key string, step delivery.Step) error {
|
||||
raw, err := delivery.EncodeStep(step)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = put(key, json.RawMessage(raw))
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
package buildstep
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/delivery"
|
||||
)
|
||||
|
||||
// keys is a steps state in memory: every value put, in order, per key, and a signal for each end.
|
||||
type keys struct {
|
||||
mu sync.Mutex
|
||||
puts map[string][]delivery.Step
|
||||
ended chan string
|
||||
}
|
||||
|
||||
func newKeys() *keys { return &keys{puts: map[string][]delivery.Step{}, ended: make(chan string, 16)} }
|
||||
|
||||
func (k *keys) put(key string, value any) (uint64, error) {
|
||||
step, err := delivery.DecodeStep(value.(json.RawMessage))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
k.mu.Lock()
|
||||
k.puts[key] = append(k.puts[key], step)
|
||||
n := uint64(len(k.puts[key]))
|
||||
k.mu.Unlock()
|
||||
if step.State != delivery.StepRunning {
|
||||
k.ended <- key
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func (k *keys) last(key string) delivery.Step {
|
||||
k.mu.Lock()
|
||||
defer k.mu.Unlock()
|
||||
s := k.puts[key]
|
||||
return s[len(s)-1]
|
||||
}
|
||||
|
||||
func (k *keys) waitEnd(t *testing.T) string {
|
||||
t.Helper()
|
||||
select {
|
||||
case key := <-k.ended:
|
||||
return key
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the step never ended on its key")
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// 1471/AC1: a run that blocks: Start answers an id within 100 ms and the key holds running.
|
||||
func TestStartAnswersAtOnceWithTheStepRunning(t *testing.T) {
|
||||
k := newKeys()
|
||||
release := make(chan struct{})
|
||||
defer close(release)
|
||||
began := time.Now()
|
||||
id, err := Start(context.Background(), k.put, "ace", "fetch", func(ctx context.Context) (any, error) {
|
||||
<-release
|
||||
return nil, nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if d := time.Since(began); d > 100*time.Millisecond {
|
||||
t.Errorf("Start took %v", d)
|
||||
}
|
||||
s := k.last("ace." + id)
|
||||
if s.State != delivery.StepRunning || s.Tool != "fetch" || s.Node != "ace" || s.ID != id || s.Started.IsZero() {
|
||||
t.Errorf("step = %+v", s)
|
||||
}
|
||||
}
|
||||
|
||||
// 1471/AC2 and 1473/AC4: a result, an error and a Refusal end the step passed, failed and refused.
|
||||
func TestStepEndsPassedFailedOrRefusedOnItsKey(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
run func(context.Context) (any, error)
|
||||
state string
|
||||
reason string
|
||||
result string
|
||||
}{
|
||||
{"result", func(context.Context) (any, error) { return map[string]string{"sha": "abc"}, nil }, delivery.StepPassed, "", `{"sha":"abc"}`},
|
||||
{"error", func(context.Context) (any, error) { return nil, errors.New("exit 2") }, delivery.StepFailed, "exit 2", ""},
|
||||
{"refusal", func(context.Context) (any, error) {
|
||||
return nil, &Refusal{Text: "go 1.27.1 installed, the recipe builds with go 1.27.0", Want: "1.27.0", Have: "1.27.1"}
|
||||
}, delivery.StepRefused, "go 1.27.1 installed, the recipe builds with go 1.27.0", ""},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
k := newKeys()
|
||||
id, err := Start(context.Background(), k.put, "ace", "test", c.run)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := k.waitEnd(t); got != "ace."+id {
|
||||
t.Fatalf("ended key %q", got)
|
||||
}
|
||||
s := k.last("ace." + id)
|
||||
if s.State != c.state || s.Reason != c.reason || string(s.Result) != c.result || s.Ended.IsZero() {
|
||||
t.Errorf("step = %+v", s)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1471/AC3: a panic in run ends the step failed, naming it, and the test process goes on.
|
||||
func TestAPanicEndsTheStepFailed(t *testing.T) {
|
||||
k := newKeys()
|
||||
id, _ := Start(context.Background(), k.put, "ace", "build", func(context.Context) (any, error) { panic("boom") })
|
||||
k.waitEnd(t)
|
||||
s := k.last("ace." + id)
|
||||
if s.State != delivery.StepFailed || s.Reason != "the step panicked: boom" {
|
||||
t.Errorf("step = %+v", s)
|
||||
}
|
||||
}
|
||||
|
||||
// 1471/AC5: ids made in one and the same millisecond differ, and each is one key token. The clock is fixed
|
||||
// while no step runs, so no goroutine reads it meanwhile.
|
||||
func TestTwoIDsInOneMillisecondDiffer(t *testing.T) {
|
||||
was := now
|
||||
defer func() { now = was }()
|
||||
at := time.Date(2026, 10, 20, 12, 0, 0, 0, time.UTC)
|
||||
now = func() time.Time { return at }
|
||||
seen := map[string]bool{}
|
||||
for i := 0; i < 50; i++ {
|
||||
id := NewID()
|
||||
if seen[id] {
|
||||
t.Fatalf("id %q handed out twice", id)
|
||||
}
|
||||
seen[id] = true
|
||||
if _, err := delivery.StepKey("ace", id); err != nil {
|
||||
t.Errorf("id %q: %v", id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A first write that fails starts nothing.
|
||||
func TestStartAnswersTheErrorWhenTheRunningWriteFails(t *testing.T) {
|
||||
ran := make(chan struct{}, 1)
|
||||
_, err := Start(context.Background(), func(string, any) (uint64, error) { return 0, errors.New("no grant") }, "ace", "fetch",
|
||||
func(context.Context) (any, error) { ran <- struct{}{}; return nil, nil })
|
||||
if err == nil {
|
||||
t.Fatal("Start answered no error")
|
||||
}
|
||||
select {
|
||||
case <-ran:
|
||||
t.Error("run was called")
|
||||
case <-time.After(50 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
@@ -1,50 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// Unwrap reads a tool's answer whatever wraps it on its way from the tool runner (the lead's ruling 2026-11-04 13:12:
|
||||
// one reading, not a copy per module): a JSON string holding the answer; the protocol's content list, whose first
|
||||
// text is the answer and whose isError says it failed; the {output, ok, answer} form; the tool runner's {result,
|
||||
// error} envelope, whose error is the call's. It answers the answer itself, the words said with it (the output, the
|
||||
// error, or the answer's own text), and whether the tool says it succeeded. A bare answer is itself, succeeded.
|
||||
func Unwrap(raw json.RawMessage) (answer json.RawMessage, said string, ok bool) {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
return Unwrap(json.RawMessage(s))
|
||||
}
|
||||
var m map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &m) != nil {
|
||||
return raw, string(raw), true
|
||||
}
|
||||
if content, has := m["content"]; has {
|
||||
var items []struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
var e struct {
|
||||
IsError bool `json:"isError"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &e)
|
||||
if json.Unmarshal(content, &items) == nil && len(items) > 0 {
|
||||
inner, out, ok := Unwrap(json.RawMessage(items[0].Text))
|
||||
return inner, out, ok && !e.IsError
|
||||
}
|
||||
}
|
||||
var v struct {
|
||||
Output string `json:"output"`
|
||||
OK *bool `json:"ok"`
|
||||
Answer json.RawMessage `json:"answer"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if json.Unmarshal(raw, &v) == nil {
|
||||
switch {
|
||||
case v.OK != nil:
|
||||
return v.Answer, v.Output, *v.OK
|
||||
case v.Error != "":
|
||||
return nil, v.Error, false
|
||||
case len(v.Result) > 0:
|
||||
return Unwrap(v.Result)
|
||||
}
|
||||
}
|
||||
return raw, string(raw), true
|
||||
}
|
||||
@@ -1,382 +0,0 @@
|
||||
// Package buscalls is the contract of the calls a bundle makes on its own bus state through the tool runner, over the
|
||||
// bundle's pipe (novox/hq ADR 0595 §9, ADR 0597 §10, design 25 § 12.5): the object store's, a stream's read-back and
|
||||
// fill, a work queue's size and dead letters, and the ledger's. Each call names an entry by the local name its module
|
||||
// asks for it by in `requires`; the tool runner serves it from the module's membership's bindings, its own entries
|
||||
// only. Only the shapes and the method names are here: what the tool runner refuses, and why, is its own.
|
||||
//
|
||||
// It is a package of its own, imported only by the modules that use these calls and by the tool runner, so a change
|
||||
// here rebuilds those and not the module library's importers (ADR 0595 §9). It imports nothing of mesh-sdk.
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the tool runner hands a bundle the id of the tool call in flight: the id
|
||||
// ApplyRequest.Call names, so the runner stamps the caller it proved for that call as the operation's author (novox/hq
|
||||
// ADR 0597 §7). The runner writes `_meta`; a caller writes only `arguments`, so no caller can name another call.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// The methods a bundle asks the tool runner, one per call.
|
||||
const (
|
||||
ObjectsPut = "mesh/objects.put"
|
||||
ObjectsGet = "mesh/objects.get"
|
||||
ObjectsList = "mesh/objects.list"
|
||||
ObjectsDelete = "mesh/objects.delete"
|
||||
ObjectsInfo = "mesh/objects.info"
|
||||
ObjectsWatch = "mesh/objects.watch"
|
||||
|
||||
StreamRead = "mesh/stream.read"
|
||||
StreamFill = "mesh/stream.fill"
|
||||
|
||||
WorkQueueSize = "mesh/workqueue.size"
|
||||
WorkQueueDead = "mesh/workqueue.dead"
|
||||
|
||||
LedgerApply = "mesh/ledger.apply"
|
||||
LedgerGet = "mesh/ledger.get"
|
||||
LedgerHistory = "mesh/ledger.history"
|
||||
LedgerWatch = "mesh/ledger.watch"
|
||||
LedgerQuery = "mesh/ledger.query"
|
||||
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
|
||||
// ledgers only (design 25 § 12.5).
|
||||
LedgerRebuild = "mesh/ledger.rebuild"
|
||||
LedgerReindex = "mesh/ledger.reindex"
|
||||
LedgerSnapshot = "mesh/ledger.snapshot"
|
||||
LedgerLag = "mesh/ledger.lag"
|
||||
)
|
||||
|
||||
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
|
||||
// does for a log.
|
||||
const (
|
||||
ObjectsWatched = "mesh/objects"
|
||||
LedgerWatched = "mesh/ledger"
|
||||
)
|
||||
|
||||
// Fill is how much an entry holds against its cap: its messages (or objects), its bytes and its cap in bytes.
|
||||
type Fill struct {
|
||||
Count uint64 `json:"count"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
MaxBytes uint64 `json:"max_bytes"`
|
||||
}
|
||||
|
||||
// Done is the answer of a call that answers nothing but that it was done.
|
||||
type Done struct{}
|
||||
|
||||
// --- the object store
|
||||
|
||||
// ObjectsPutRequest stores one object whole under its name, replacing one of that name.
|
||||
type ObjectsPutRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
Data []byte `json:"data"`
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsNameRequest names one object of an object store: get, delete.
|
||||
type ObjectsNameRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// ObjectsRequest names an object store: list, info.
|
||||
type ObjectsRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsWatchRequest asks the tool runner to send every object stored or deleted, as ObjectsChange under the watch's id.
|
||||
type ObjectsWatchRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// ObjectInfo is one object as its store holds it: put's answer, and each of list's.
|
||||
type ObjectInfo struct {
|
||||
Name string `json:"name"`
|
||||
Size uint64 `json:"size"`
|
||||
Chunks uint32 `json:"chunks"`
|
||||
Digest string `json:"digest"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Deleted bool `json:"deleted,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectData is get's answer: the object's info and its bytes.
|
||||
type ObjectData struct {
|
||||
Info ObjectInfo `json:"info"`
|
||||
Data []byte `json:"data"`
|
||||
}
|
||||
|
||||
// ObjectsListed is list's answer: every object the store holds.
|
||||
type ObjectsListed struct {
|
||||
Objects []ObjectInfo `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsChange is one object stored or deleted, sent under ObjectsWatched; Current says it was there when the watch
|
||||
// began.
|
||||
type ObjectsChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Object ObjectInfo `json:"object"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// --- a stream
|
||||
|
||||
// StreamReadRequest reads a module's own stream from a sequence on, at most Limit messages, Subjects narrowing it.
|
||||
type StreamReadRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
From uint64 `json:"from"`
|
||||
Subjects string `json:"subjects,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// StreamRequest names a stream: fill.
|
||||
type StreamRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
}
|
||||
|
||||
// StreamMessage is one message of a stream.
|
||||
type StreamMessage struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// StreamPage is read's answer: the messages in order, the sequence to read on from (0 when nothing follows) and the
|
||||
// stream's last sequence when it was read.
|
||||
type StreamPage struct {
|
||||
Messages []StreamMessage `json:"messages"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
|
||||
type StreamFilled struct {
|
||||
Fill
|
||||
First uint64 `json:"first"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// --- a work queue
|
||||
|
||||
// WorkQueueRequest names a work queue: size.
|
||||
type WorkQueueRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadRequest asks a work queue's dead letters, at most Limit of them.
|
||||
type WorkQueueDeadRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// WorkQueueSized is size's answer: the messages waiting to be taken, against the queue's cap.
|
||||
type WorkQueueSized struct {
|
||||
Fill
|
||||
}
|
||||
|
||||
// DeadLetter is one message delivered as often as its queue allows and never settled.
|
||||
type DeadLetter struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Deliveries int `json:"deliveries"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadLetters is dead's answer.
|
||||
type WorkQueueDeadLetters struct {
|
||||
Dead []DeadLetter `json:"dead"`
|
||||
}
|
||||
|
||||
// --- the ledger (ADR 0597)
|
||||
|
||||
// ApplyRequest applies one operation to one instance of a ledger. Call is the id of the tool call the operation
|
||||
// serves, as the tool runner handed it under MetaCall, whose caller the tool runner proved and stamps as the author
|
||||
// (ADR 0597 §7); empty, the operation carries the owning module's account alone. An author the bundle writes into Body
|
||||
// is never taken as the author.
|
||||
type ApplyRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Call string `json:"call,omitempty"`
|
||||
}
|
||||
|
||||
// Applied is apply's answer: the operation was taken, with its sequence in the instance's history; StateBehind says
|
||||
// the state could not follow because its bucket is full, and catches up at the next apply after room is made (ADR
|
||||
// 0597 §6).
|
||||
type Applied struct {
|
||||
Taken bool `json:"taken"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
StateBehind bool `json:"state_behind,omitempty"`
|
||||
}
|
||||
|
||||
// Author is who an operation is by, as `apply` stamped it (ADR 0597 §7): the owning module's account, and the caller
|
||||
// of the tool call it served, as the tool runner proved it.
|
||||
type Author struct {
|
||||
Account string `json:"account"`
|
||||
Caller string `json:"caller,omitempty"`
|
||||
}
|
||||
|
||||
// InstanceRequest names one instance of a ledger: get, snapshot.
|
||||
type InstanceRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
}
|
||||
|
||||
// InstanceState is get's answer: the instance's state and the sequence of the last operation it holds.
|
||||
type InstanceState struct {
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// HistoryRequest reads an instance's operations from a sequence on, at most Limit of them.
|
||||
type HistoryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
From uint64 `json:"from,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// Operation is one operation as the history keeps it.
|
||||
type Operation struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Author Author `json:"author"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// HistoryPage is history's answer: the operations oldest first, the sequence to read on from (0 when nothing
|
||||
// follows) and the instance's last sequence when it was read.
|
||||
type HistoryPage struct {
|
||||
Operations []Operation `json:"operations"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// LedgerWatchRequest asks the tool runner to send every change of the instances Instances names (an instance, with
|
||||
// `*` for one name and `>` for the rest) as InstanceChange under the watch's id: the current states first.
|
||||
type LedgerWatchRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instances string `json:"instances"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// InstanceChange is one instance's new state, sent under LedgerWatched; Current says it was the state when the watch
|
||||
// began.
|
||||
type InstanceChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// QueryRequest reads a declared index of a ledger: the entries whose key Key names (a key, with `*` for one name and
|
||||
// `>` for the rest; empty is every key), at most Limit of them.
|
||||
type QueryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// IndexEntry is the value of one key of a declared index (IndexKey): the instance it points at, and the sequence of
|
||||
// the operation it was written from. The owner's watcher and the tool runner's reindex both write it so.
|
||||
type IndexEntry struct {
|
||||
Instance string `json:"instance"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// IndexKey is the key of an instance in a declared index: the values of the index's fields, in its order, then the
|
||||
// parts of the instance's path, all joined by ".". Each value and each path part is encoded (keyToken): every byte
|
||||
// outside [A-Za-z0-9_-] is written "=" and two upper-case hex digits ("." is "=2E", " " is "=20", "=" is "=3D"), so two
|
||||
// different keys never meet and every part is a token the bus takes. `17950.listing.a17` is the key of price 17950
|
||||
// and instance `listing/a17`. The owner's watcher and the tool runner's reindex both write it, so a reindex leaves the
|
||||
// index as the watcher keeps it.
|
||||
func IndexKey(fields []string, instance string) string {
|
||||
parts := make([]string, 0, len(fields)+2)
|
||||
for _, f := range fields {
|
||||
parts = append(parts, keyToken(f))
|
||||
}
|
||||
for _, p := range strings.Split(instance, "/") {
|
||||
parts = append(parts, keyToken(p))
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
// keyToken is v with every byte outside [A-Za-z0-9_-] written "=" and two upper-case hex digits (IndexKey).
|
||||
func keyToken(v string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(v); i++ {
|
||||
c := v[i]
|
||||
if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_' || c == '-' {
|
||||
b.WriteByte(c)
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, "=%02X", c)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// QueryAnswer is query's answer.
|
||||
type QueryAnswer struct {
|
||||
Entries []IndexEntry `json:"entries"`
|
||||
}
|
||||
|
||||
// RebuildFrom is what a rebuild derives the state from: the history, or the snapshot and the operations after it.
|
||||
const (
|
||||
RebuildFromHistory = "history"
|
||||
RebuildFromSnapshot = "snapshot"
|
||||
)
|
||||
|
||||
// RebuildRequest derives an instance's state again (every instance's when Instance is empty), From the history or the
|
||||
// snapshot.
|
||||
type RebuildRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance,omitempty"`
|
||||
From string `json:"from,omitempty"`
|
||||
}
|
||||
|
||||
// ReindexRequest builds a declared index again from the history.
|
||||
type ReindexRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
}
|
||||
|
||||
// LedgerRequest names a ledger: lag.
|
||||
type LedgerRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
}
|
||||
|
||||
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
|
||||
type Repaired struct {
|
||||
Written int `json:"written"`
|
||||
}
|
||||
|
||||
// InstanceLag is one instance whose state or snapshot is behind its history: the sequences each holds.
|
||||
type InstanceLag struct {
|
||||
Instance string `json:"instance"`
|
||||
History uint64 `json:"history"`
|
||||
State uint64 `json:"state"`
|
||||
Snapshot uint64 `json:"snapshot"`
|
||||
}
|
||||
|
||||
// IndexLag is how far one declared index is behind the history.
|
||||
type IndexLag struct {
|
||||
Index string `json:"index"`
|
||||
Behind uint64 `json:"behind"`
|
||||
}
|
||||
|
||||
// Lagged is lag's answer: every instance and index behind its history; none when nothing is behind.
|
||||
type Lagged struct {
|
||||
Instances []InstanceLag `json:"instances,omitempty"`
|
||||
Indexes []IndexLag `json:"indexes,omitempty"`
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import "time"
|
||||
|
||||
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash. The state of
|
||||
// wanted accounts it mints from is the grants part's (grantsstate.AccountsState; novox/hq ADR 0634 §1).
|
||||
const AccountIssuedEvent = "account-issued"
|
||||
|
||||
// AccountIssued is the vault's account-issued: which account it minted, and its generation. Never a value or a hash.
|
||||
type AccountIssued struct {
|
||||
User string `json:"user"`
|
||||
Kind Kind `json:"kind"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
Process string `json:"process,omitempty"`
|
||||
Generation int `json:"generation"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"git.novox.be/novox/mesh-sdk/go/declaration"
|
||||
)
|
||||
|
||||
// DeclarationFilesBucket is the prefix of the buckets' names, which mesh-sdk fixes for the node-engine and the controller alike.
|
||||
const DeclarationFilesBucket = declaration.FileBucketPrefix
|
||||
@@ -1,59 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// A Consumer is a durable subscription the controller creates on a module's behalf. A module
|
||||
// declares what it reacts to, never how delivery works, so it does not name these and cannot
|
||||
// misconfigure them.
|
||||
type Consumer struct {
|
||||
Name string
|
||||
Stream string
|
||||
// Filters are the subjects this consumer receives. One consumer per module with several
|
||||
// filters, rather than one per consumed event: its ack subject is derived from its name, and
|
||||
// a module with five consumers would need five ack permissions to ack its own deliveries.
|
||||
Filters []string
|
||||
// Queue is the queue group, set for a seat's worker so that "exactly one holder" survives a
|
||||
// seat later being relaxed to several. Authority and delivery are kept separate on purpose.
|
||||
Queue string
|
||||
// Push asks the server to deliver to a subject rather than wait to be pulled.
|
||||
//
|
||||
// For the mesh's own consumer, where the controller wants every message to arrive in the one
|
||||
// loop it already runs: pulling would mean a second goroutine fetching batches and handing
|
||||
// them over, and a loop that acts on one message at a time is the property the store window
|
||||
// depends on. A queue group implies this, because a group has nothing to pull from.
|
||||
Push bool
|
||||
// TakesOverFrom names a consumer of the same stream whose filter this one takes over (issue 973): made while
|
||||
// that consumer still filters this one's subject, it starts after what that consumer has acknowledged, so a
|
||||
// message the old one held unacknowledged is not lost to the change. Otherwise it starts as FromNow says.
|
||||
TakesOverFrom string
|
||||
// DeliverSubject, when set, is where a push consumer delivers instead of DeliverSubjectFor's.
|
||||
DeliverSubject string
|
||||
// AckWaitSeconds before an unacknowledged delivery is redelivered.
|
||||
AckWaitSeconds int
|
||||
// MaxDeliver is how often a message is handed over before the consumer gives it up; zero for no
|
||||
// bound. What a consumer gives up is kept in DEAD_LETTERS by the controller (novox/hq issue 330).
|
||||
MaxDeliver int
|
||||
// MaxAckPending is how many deliveries the server lets stand unacknowledged at once; zero for
|
||||
// the server's default, which is many. **One, for a consumer handled one at a time**
|
||||
// (novox/hq issue 175): a handler that builds for minutes keeps its own message alive with a
|
||||
// heartbeat, but everything handed over behind it times out unacknowledged and comes back —
|
||||
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
|
||||
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
|
||||
MaxAckPending int
|
||||
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
|
||||
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
|
||||
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
|
||||
// holds, replays every merge and every build of that week as if it had just happened. A consumer
|
||||
// that exists keeps where it is, whatever this says; only its making is decided here.
|
||||
FromNow bool
|
||||
Why string
|
||||
// Resettable says why this consumer may be re-made to deliver from now by the mesh itself, with
|
||||
// nobody asked (novox/hq to-be 45 §7, healer H4): what a reset drops, something else catches up.
|
||||
// Empty for every consumer where nothing would — a module's, whose events would be lost to it.
|
||||
// Not a property of the consumer on the bus: nothing here is sent to the server.
|
||||
Resettable string
|
||||
}
|
||||
|
||||
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
|
||||
type ModuleConsumer struct {
|
||||
Node, Module string
|
||||
Consumer Consumer
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
// Package buscompose is the bus's names and shapes every part reads (novox/hq ADR 0593 §1): the mesh's streams and
|
||||
// dead letters, a module's log, stream and buckets as the bus names them, the account a part asks for, a principal's
|
||||
// shape, and the grants part's memberships key. The composer of the bus's user list, which reads these and makes every
|
||||
// user's permissions, is the grants part's own (mesh-delivery modules/mesh-grants internal/buscompose).
|
||||
package buscompose
|
||||
@@ -1,12 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// EngineModule is the node-engine's manifest's module name, and EngineActsState its state the acts asked through the
|
||||
// node seat and its own asks are kept in (ADR 0563 §2, §3: "the engine's own state"): the bucket mesh-host_acts, keys
|
||||
// `<node>.act.<id>`, `<node>.open.<act>.<digest>` and `<node>.ask.<id>`, each written by that node's engine alone.
|
||||
const (
|
||||
EngineModule = "mesh-host"
|
||||
EngineActsState = "acts"
|
||||
)
|
||||
|
||||
// EngineActsBucket is that state's bucket.
|
||||
var EngineActsBucket = BucketName(EngineModule, EngineActsState)
|
||||
@@ -1,87 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// The mesh's caps on a log: what an entry's value may weigh, what a message on the log's stream may
|
||||
// weigh — the value and its headers, which the server counts in a message's size, so a value of the
|
||||
// full size still fits with the expected-last-sequence header an append carries — and what a log holds
|
||||
// when its module says nothing and at most.
|
||||
const (
|
||||
LogMaxEntryBytes = 256 * 1024
|
||||
LogMaxMessageBytes = LogMaxEntryBytes + 4*1024
|
||||
LogDefaultMiB = 1024
|
||||
)
|
||||
|
||||
// A Log is one module's declared log as the bus holds it.
|
||||
type Log struct {
|
||||
Module string
|
||||
Name string
|
||||
// MaxMiB is its cap in MiB; zero is LogDefaultMiB.
|
||||
MaxMiB int
|
||||
// MaxAgeDays is how long an entry is kept; zero is as long as the log (catalogue.LogDeclaration).
|
||||
MaxAgeDays int
|
||||
}
|
||||
|
||||
// LogStreamPrefix starts every log's stream name, which no other stream of the mesh's starts with.
|
||||
const LogStreamPrefix = "LOG_"
|
||||
|
||||
// LogStreamName is the stream a module's log lives in: `LOG_<module>_<name>`. The module and the
|
||||
// local name are each one token with no underscore, so two modules can never derive one stream.
|
||||
func LogStreamName(module, name string) string { return LogStreamPrefix + module + "_" + name }
|
||||
|
||||
// LogSubject is the subject a log's entries are published under, without the key: an entry for key K
|
||||
// is on `<LogSubject>.<K>`.
|
||||
func LogSubject(module, name string) string { return "mesh.log." + module + "." + name }
|
||||
|
||||
// Stream is this log's stream name.
|
||||
func (l Log) Stream() string { return LogStreamName(l.Module, l.Name) }
|
||||
|
||||
// Subject is this log's subject, without the key.
|
||||
func (l Log) Subject() string { return LogSubject(l.Module, l.Name) }
|
||||
|
||||
// MaxBytes is this log's cap in bytes.
|
||||
func (l Log) MaxBytes() int64 {
|
||||
mib := l.MaxMiB
|
||||
if mib <= 0 {
|
||||
mib = LogDefaultMiB
|
||||
}
|
||||
return int64(mib) * 1024 * 1024
|
||||
}
|
||||
|
||||
// Why is carried into the server's description of the stream, so somebody reading the server's own
|
||||
// state finds whose it is and why it is kept.
|
||||
func (l Log) Why() string {
|
||||
return fmt.Sprintf("%s's log %q (novox/hq ADR 0297): its record of operations, one entry per operation "+
|
||||
"under its key, appended by %s alone and kept as long as the log; never removed by the mesh, because "+
|
||||
"it is data", l.Module, l.Name, l.Module)
|
||||
}
|
||||
|
||||
// Config is the stream a log is, field by field as novox/hq ADR 0297 fixes it: a file stream kept by
|
||||
// limits, with no maximum age unless the log says one (ADR 0565 §10) and no cap per key, whose message is an entry's value and 4 KiB of
|
||||
// headers at most, that refuses a new entry when full rather than
|
||||
// drop an old one, and that refuses deleting an entry or purging it. Direct gets are allowed, which
|
||||
// is how the runtime reads it.
|
||||
func (l Log) Config() jetstream.StreamConfig {
|
||||
return jetstream.StreamConfig{
|
||||
Name: l.Stream(),
|
||||
Description: l.Why(),
|
||||
Subjects: []string{l.Subject() + ".>"},
|
||||
Storage: jetstream.FileStorage,
|
||||
Retention: jetstream.LimitsPolicy,
|
||||
MaxAge: time.Duration(l.MaxAgeDays) * 24 * time.Hour,
|
||||
MaxMsgs: -1,
|
||||
MaxMsgsPerSubject: -1,
|
||||
MaxBytes: l.MaxBytes(),
|
||||
MaxMsgSize: LogMaxMessageBytes,
|
||||
Discard: jetstream.DiscardNew,
|
||||
AllowDirect: true,
|
||||
DenyDelete: true,
|
||||
DenyPurge: true,
|
||||
Replicas: 1,
|
||||
}
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// moduleStreamPrefix starts every subject a module's own stream takes.
|
||||
const moduleStreamPrefix = "mesh.stream."
|
||||
|
||||
// ModuleStreamSubject is where a module's event on its own stream lands: `mesh.stream.<module>.event.<event>`.
|
||||
// The token before `.event.` is the emitter, as on EVENTS, so a runtime reads the same key from it.
|
||||
func ModuleStreamSubject(module, event string) string {
|
||||
return moduleStreamPrefix + module + ".event." + event
|
||||
}
|
||||
|
||||
// A ModuleStream is one module's declared stream as the bus holds it.
|
||||
type ModuleStream struct {
|
||||
Module string
|
||||
// Name is its name on the bus, `MAIL`.
|
||||
Name string
|
||||
// Events are the events it carries, as the bus spells them (WirePattern).
|
||||
Events []string
|
||||
MaxAgeDays int
|
||||
MaxMiB int
|
||||
// Restricted says a trusted config value of the module grants who hears it (ADR 0363 §4).
|
||||
Restricted bool
|
||||
// WorkQueue says this stream holds ADR 0436 §6's work-queue retention: a message is removed once a
|
||||
// consumer acknowledges it, and the bus refuses a new one when full rather than dropping its oldest.
|
||||
WorkQueue bool
|
||||
// Duplicates is the work-queue's duplicate window in hours (novox/hq ADR 0436 §6); meaningless unless
|
||||
// WorkQueue.
|
||||
Duplicates int
|
||||
}
|
||||
|
||||
// Subjects are the subjects the stream takes.
|
||||
func (s ModuleStream) Subjects() []string {
|
||||
var out []string
|
||||
for _, e := range s.Events {
|
||||
out = append(out, ModuleStreamSubject(s.Module, e))
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// Stream is the stream the controller asserts: kept by limits, its age and size the module's, dropping
|
||||
// its oldest when full — a consumer that was away catches up within the age, and the module's own record
|
||||
// answers for anything older.
|
||||
func (s ModuleStream) Stream() Stream {
|
||||
why := fmt.Sprintf("%s's own stream (novox/hq ADR 0363): its events, kept %d days and at most %d MiB so a "+
|
||||
"consumer that was away catches up", s.Module, s.MaxAgeDays, s.MaxMiB)
|
||||
if s.Restricted {
|
||||
why += "; heard only by the consumers its trusted config value grants"
|
||||
}
|
||||
if s.WorkQueue {
|
||||
why += "; a work-queue (novox/hq ADR 0436 §6): a message is removed once its one consumer " +
|
||||
"acknowledges it, and a full stream refuses a new one rather than dropping what it holds"
|
||||
}
|
||||
out := Stream{
|
||||
Name: s.Name,
|
||||
Subjects: s.Subjects(),
|
||||
Retention: RetentionLimits,
|
||||
MaxAge: s.MaxAgeDays * 24 * 60 * 60,
|
||||
MaxBytes: int64(s.MaxMiB) * 1024 * 1024,
|
||||
Why: why,
|
||||
// Said on the bus, so a dead letter kept from it ages out with it (novox/hq ADR 0376 §3).
|
||||
Restricted: s.Restricted,
|
||||
}
|
||||
if s.WorkQueue {
|
||||
out.Retention = RetentionWorkQueue
|
||||
out.DiscardNew = true
|
||||
out.DuplicatesSeconds = s.Duplicates * 60 * 60
|
||||
// Said on the bus too, reusing the one flag a restricted stream's dead letters already age and hide
|
||||
// by (novox/hq ADR 0436 §6, ADR 0376 §3-4): a work-queue stream's grant is always empty (ADR 0436
|
||||
// §6 refuses one that carries a grant), so this never collides with a trusted-config restriction.
|
||||
out.Restricted = true
|
||||
// And its own metadata, apart from the bus's work-queue retention SEAT_* and other mesh-internal
|
||||
// streams share without being one of this clause's: only this flag says a dead letter kept from the
|
||||
// stream is drop-only (novox/hq ADR 0436 §6), never the stream's Retention alone.
|
||||
out.WorkQueue = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
|
||||
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
|
||||
// to be the only one (the indexer's judge, novox/hq ADR 0578 §1).
|
||||
type QueueOwner struct {
|
||||
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
|
||||
// 0436 §2).
|
||||
Process string
|
||||
// Stream is the work-queue stream's name on the bus.
|
||||
Stream string
|
||||
// MaxDeliver bounds how many times a message is handed over before it is given up as a dead letter
|
||||
// (novox/hq ADR 0436 §6).
|
||||
MaxDeliver int
|
||||
}
|
||||
|
||||
// Heard is a module's stream another module hears, and the subjects it hears of it: its `consumes`
|
||||
// within the stream and, for a restricted stream, within what the provider's trusted config value grants it
|
||||
// (ADR 0363 §5). Composed only with at least one subject.
|
||||
type Heard struct {
|
||||
Stream string
|
||||
Emitter string
|
||||
Filters []string
|
||||
// Restricted says the stream is restricted by its provider's trusted config value (ADR 0363 §4): then it
|
||||
// is heard only by a module on an account of its own (ADR 0376 §1, ComposedHears).
|
||||
Restricted bool
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/membership"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// The names this package keeps for the grants part's composer (mesh-grants internal/buscompose, ADR 0593 §1) agree with
|
||||
// what the runtime reads: the memberships bucket, and a log's stream and subject.
|
||||
func TestTheBusNamesAgree(t *testing.T) {
|
||||
if GrantsMembershipsBucket != membership.GrantsBucket {
|
||||
t.Fatalf("the memberships bucket is %q here and %q in membership", GrantsMembershipsBucket, membership.GrantsBucket)
|
||||
}
|
||||
if MembershipKey("ace", "netcheck") != "ace.netcheck" {
|
||||
t.Fatal("a membership's key is not <node>.<module>")
|
||||
}
|
||||
if LogStreamName("mesh-delivery-builder", "build-log") == "" || LogSubject("mesh-delivery-builder", "build-log") == "" {
|
||||
t.Fatal("a log has no stream or subject")
|
||||
}
|
||||
// 1966/AC20: a ledger's history and snapshot subjects are one contract, which the grants part grants a ledger
|
||||
// account its publish on.
|
||||
if h, sn := LedgerHistorySubject("m", "records"), LedgerSnapshotSubject("m", "records"); h != "mesh.ledger.m.records.history" ||
|
||||
sn != "mesh.ledger.m.records.snapshot" {
|
||||
t.Fatalf("a ledger's subjects are %q and %q", h, sn)
|
||||
}
|
||||
// 1966/AC23: a ledger account for m on a is named ledger.a.m.
|
||||
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
|
||||
t.Fatalf("a ledger account is named %q", got)
|
||||
}
|
||||
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
|
||||
for _, st := range MeshStreams() {
|
||||
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
|
||||
t.Errorf("the shared events stream keeps %d s, a seat's definition says %d days", st.MaxAge, seatdef.SharedEventsMaxAgeDays)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,351 +0,0 @@
|
||||
// Composing the bus's own configuration.
|
||||
//
|
||||
// An account is *composed*, never called for: the controller writes accounts, users and
|
||||
// per-subject permissions into one file the host keeps current, and the server reloads it in
|
||||
// place (novox/hq ADR 0106 — never through a management API; design 25 §4).
|
||||
//
|
||||
// Everything here is pure. Given the principals, it returns the file's text — so the whole of the
|
||||
// mesh's authority model is testable as strings, with no server.
|
||||
//
|
||||
// **Permissions are per subject, so a module's own name is the server's to enforce.** ADR 0042
|
||||
// reserves a module's origin — it publishes only under its own name — and here that is a refusal
|
||||
// rather than something a library promises.
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A Kind is what a principal is, which decides the shape of its authority rather than its
|
||||
// contents: a module's comes from its declaration, a host's from its node, and the controller's
|
||||
// and the enrolment user's are fixed.
|
||||
type Kind string
|
||||
|
||||
const (
|
||||
KindModule Kind = "module"
|
||||
KindNode Kind = "node"
|
||||
KindEnrolment Kind = "enrolment"
|
||||
// KindPerson is somebody reaching the mesh's tools from a workstation (design 25 §7). Its
|
||||
// authority is a list of tools and nothing else — not control, not declarations, not builds,
|
||||
// and no ability to answer anything, because a person asks.
|
||||
KindPerson Kind = "person"
|
||||
// KindNodeTools is a machine's tool runtime (novox/hq ADR 0175, to-be 38): one process per
|
||||
// node, on the host side, serving every assigned module's tools and every held seat's verbs.
|
||||
// Its authority is the union of what the modules it carries would each have had for their
|
||||
// tools — and nothing of what they consume, because tools are what it runs, not reactions.
|
||||
KindNodeTools Kind = "node-tools"
|
||||
// KindProvisioner is a machine's provisioner client (novox/hq ADR 0349 clauses 6 and 11, design 57 P3): a
|
||||
// service of the module holding NodeProvisionerSeat, outside the tool runner, under an account of its own so
|
||||
// the vault can tell it from an agent. Its authority is provisionerGrants and nothing else.
|
||||
KindProvisioner Kind = "provisioner"
|
||||
// KindProcess is one process of a module that declares bus (novox/hq ADR 0436 §1, §2): a service of that
|
||||
// module, outside the tool runner and outside the module's own account, under a bus account scoped to
|
||||
// exactly what its bus says — never root, never the operator account, and never able to answer a tool
|
||||
// call or a seat's verb (§3). One per process and machine.
|
||||
KindProcess Kind = "process"
|
||||
// KindLedger is one module assignment's ledger credential (novox/hq ADR 0597 §9, design 25 § 12.4): held by that
|
||||
// machine's tool runner, never by the module's own code, and the only account that publishes on that module's ledger
|
||||
// histories and snapshots. One per module assignment, whose module asks for a ledger in `requires`.
|
||||
KindLedger Kind = "ledger"
|
||||
)
|
||||
|
||||
// LedgerHistorySubject and LedgerSnapshotSubject are the subject prefixes of one ledger's history and of its instances'
|
||||
// snapshots (novox/hq ADR 0597 §1, §8, §9): an instance's subject is the prefix, ".", and the instance's path with each
|
||||
// "/" written "." (`decision/0290` is `….decision.0290`). One contract: the grants part grants a ledger account its
|
||||
// publish on them, the tool runner appends and snapshots on them, and the bus module creates the history's stream over
|
||||
// them.
|
||||
func LedgerHistorySubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".history"
|
||||
}
|
||||
|
||||
// LedgerSnapshotSubject is the subject prefix of one ledger's instances' snapshots (LedgerHistorySubject).
|
||||
func LedgerSnapshotSubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".snapshot"
|
||||
}
|
||||
|
||||
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
|
||||
// emits (novox/hq ADR 0118, design 29 §5).
|
||||
type Seat struct {
|
||||
Name string
|
||||
// Scope is where the seat has one holder. A node-scoped seat's tool carries the node in its
|
||||
// subject, because one subject reaching six machines' holders is not an address
|
||||
// (novox/hq ADR 0132, design 33 §4). Empty reads as mesh.
|
||||
Scope string
|
||||
Accepts []string
|
||||
Emits []string
|
||||
Serves []string
|
||||
Versions []string // protocol versions served beside the current one; empty for v1 only
|
||||
|
||||
// Kinded says the seat is a kinded bench (novox/hq ADR 0234 §2, ADR 0259 §3): each holder claims one
|
||||
// kind, and its verbs' subjects carry it. Kind is the kind this principal's claim names, for a seat it
|
||||
// holds.
|
||||
Kinded bool
|
||||
Kind string
|
||||
// ByCaller are the accepts and emits whose last token names the calling module (ADR 0259 §3).
|
||||
ByCaller []string
|
||||
// Proofs are the seat's proof verbs: core request and reply, never on a stream (ADR 0259 §3).
|
||||
Proofs []string
|
||||
// Records are the holder's buckets, by their full name, each user reads under its own name.
|
||||
Records []string
|
||||
// Capabilities are what this principal's claim of a kinded bench promises (ADR 0234 §2).
|
||||
Capabilities []string
|
||||
// DeclaredBy is the module that declares the seat. On a kinded bench it alone submits work to a kind
|
||||
// and answers its proofs (novox/hq ADR 0259 §8): the router, not any user or watcher of the bench.
|
||||
DeclaredBy string
|
||||
// PulledByKind says each accepted kind is a worker of its own on the seat's queue, filtered to that kind, which
|
||||
// every holder pulls in the order Accepts lists them: the build seats' (novox/hq ADR 0525 §2, ADR 0536 §4).
|
||||
PulledByKind bool
|
||||
}
|
||||
|
||||
// A Principal is one user of the bus. Its permissions are derived from what it declares and
|
||||
// nothing else (novox/hq ADR 0043), over the three namespaces of design 29 §2: its own, the seats
|
||||
// it holds, and the seats it uses.
|
||||
type Principal struct {
|
||||
Kind Kind
|
||||
Node string
|
||||
Module string
|
||||
// Process is the process resource's id, for a KindProcess principal alone (novox/hq ADR 0436 §2):
|
||||
// Username and inbox are <node>.<module>.<process id>.
|
||||
Process string
|
||||
|
||||
// FilesFor are the nodes whose declaration-file buckets (novox/hq ADR 0486 §2) this principal may remove, for the bus
|
||||
// module's tools removing the ended controller's leftovers alone (ADR 0590 clause 6): every node of the mesh, one
|
||||
// bucket each.
|
||||
FilesFor []string
|
||||
|
||||
Emits []string
|
||||
Consumes []string
|
||||
Serves []string
|
||||
|
||||
Holds []Seat
|
||||
Uses []Seat
|
||||
// Watches are seats whose events this principal consumes. Separate from Consumes because a
|
||||
// role's event lives under the seat's namespace and not a module's, and this package cannot tell
|
||||
// a seat's name from a module's by looking at it — whoever resolved the declaration can, and
|
||||
// does (novox/hq ADR 0121).
|
||||
//
|
||||
// **Found by a consumer reading nothing.** The catalogue consumes the build machine's outcome;
|
||||
// with that name read as a module's, its subscription pointed at `mesh.mod.mesh-build-machine.…`,
|
||||
// a namespace no such module owns. Every service started and the graph stayed empty.
|
||||
Watches []Seat
|
||||
|
||||
// Carries are the modules whose tools this principal serves, for a KindNodeTools principal
|
||||
// (novox/hq ADR 0175): every module assigned to its node, as each declares itself. Its
|
||||
// serving authority is the union of theirs — each module's own tool namespace and each held
|
||||
// seat's verbs on this node — derived from the same declarations the modules' own principals
|
||||
// are, so the runtime can serve nothing a module could not have served for itself.
|
||||
Carries []Declared
|
||||
|
||||
// Invokes are the tools this principal may call, as `<module>.<tool>`; a single `*` is every
|
||||
// tool. A person's whole authority (design 25 §7), and a module's only if its manifest says so
|
||||
// (novox/hq ADR 0152) — the console's does, and nothing else's.
|
||||
//
|
||||
// **A list, not a role.** A person is not a module and holds no seat: nothing is addressed
|
||||
// to them, nothing is delivered to them, and they have no durable consumer to acknowledge.
|
||||
// What they have is permission to ask. A module that invokes gains exactly the same
|
||||
// permission and nothing beside it.
|
||||
Invokes []string
|
||||
|
||||
// AsksIssued are, for the vault's holder, the tools it asks in its own name for a credential a provider's software
|
||||
// issued, each `<module>.<tool>` (novox/hq ADR 0305, issue 723, vaultCalls): on the subject that names its caller,
|
||||
// so the tool knows the vault asks.
|
||||
AsksIssued []string
|
||||
|
||||
// AsksTheDesk says this principal is the vault's holder, which asks the desks' secret prompt in its own name
|
||||
// (novox/hq ADR 0349 clause 9, issue 723): `node-launcher.secret` on any machine, on the subject that names
|
||||
// its caller, and nothing else of that seat. With AsksIssued on the same principal, the vault's whole reach
|
||||
// outside its own seat: it asks each issuing provider's tool in its own name (clause 10).
|
||||
AsksTheDesk bool
|
||||
|
||||
// State is the local names of the state this principal's module keeps, and Reads the state of
|
||||
// others it reads as `<module>.<name>` (novox/hq ADR 0201): a bucket each, kept by the owner's
|
||||
// instances and read by whoever declares it.
|
||||
State []string
|
||||
Reads []string
|
||||
// PerMachine is the local names of its state keyed by machine, reaching the key named for Node and
|
||||
// no other; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
|
||||
PerMachine []string
|
||||
KeyedReads []KeyedRead
|
||||
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
|
||||
Logs []string
|
||||
// ReadLogs are, of Reads, those naming another module's log (novox/hq ADR 0571 clause 1).
|
||||
ReadLogs []string
|
||||
// Streams are the streams this principal's module owns, and Hears the module streams it hears with the
|
||||
// subjects it was granted (novox/hq ADR 0363).
|
||||
Streams []ModuleStream
|
||||
Hears []Heard
|
||||
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
|
||||
// §1, §2, §6); for a KindProcess principal, its own alone.
|
||||
Queues []QueueOwner
|
||||
// Asks says a KindProcess principal reads the records of its module's own asks (novox/hq ADR 0436 §1),
|
||||
// on whichever seat of Uses keeps them (novox/hq ADR 0259 §3) — never set for any other kind.
|
||||
Asks bool
|
||||
|
||||
// SnapshotsTheBus is the bus's own module, the one holding mesh-broker (novox/hq ADR 0235). Its
|
||||
// whole authority is BusSnapshotGrants: it copies the streams for the night's backup and nothing
|
||||
// else — not its declarations, which the node's tool runtime serves for it.
|
||||
SnapshotsTheBus bool
|
||||
// Removes are, for a process or tools bundle account whose bus declares them, what it may remove from the bus
|
||||
// (novox/hq ADR 0564 Limits 4): its whole authority is then BusRemovalGrants of them.
|
||||
Removes []string
|
||||
// DeadLetterSources are, for the watchman, the streams a dead letter it keeps may come from, by name
|
||||
// (deadLetterSourcesOf, novox/hq ADR 0564 clause 5): never a bucket's.
|
||||
DeadLetterSources []string
|
||||
|
||||
// Checks are the tools a machine principal's node-engine asks as a declared health check, each
|
||||
// `<module>.<tool>` (novox/hq ADR 0240, to-be 48 §3): asked of the instance on its own machine and
|
||||
// nowhere else.
|
||||
Checks []string
|
||||
// MovesIntoTheVault is a machine principal whose node-engine may ask the vault's `move-in` in its own name (design
|
||||
// 57's V3, issue 676): set while the vault's holder serves it, so the engine can hand in the values sealed to its
|
||||
// machine, which nothing else can open.
|
||||
MovesIntoTheVault bool
|
||||
// StatesItsKeyToTheVault is a machine principal whose node-engine may ask the vault's `node-key` in its own name
|
||||
// (issue 786, novox/hq ADR 0372): set while the vault's holder serves it. That one verb, and nothing else new.
|
||||
StatesItsKeyToTheVault bool
|
||||
// AsksItsOwnSecrets is a machine principal whose node-engine may ask the vault's `own` in its own name (novox/hq
|
||||
// ADR 0547 §2, ADR 0550 §5, issue 1126): its modules' own secrets sealed to its key, for the compositions it makes
|
||||
// itself. Set while the vault's holder serves it.
|
||||
AsksItsOwnSecrets bool
|
||||
// AsksTheVaultToSign is a machine principal whose node-engine may ask the vault's `sign` in its own name, read the
|
||||
// vault's public signing halves (`signers`) and the controller's lease key (novox/hq ADR 0550 §2): its own node's
|
||||
// certificate, the keys it trusts, and the lease write that drops the controller's key. Set while the vault's holder
|
||||
// serves `sign`.
|
||||
AsksTheVaultToSign bool
|
||||
// ServedReads are, for a machine principal, the keys of mesh-config's resolved values that other nodes' modules serve
|
||||
// from (`<node>.<module>.<key>`, catalogue.ServedSettings): what every consumer is told, and so what a node composing
|
||||
// its own modules reads of a provider elsewhere (issue 1126), each key alone. Sorted, none twice.
|
||||
ServedReads []string
|
||||
// FollowsGrantsUserList is the machine principal of the node holding mesh-broker: its node-engine reads the grants
|
||||
// part's sealed user list, one key and an object by name (novox/hq ADR 0547 clause 7, GrantsUserListFollows).
|
||||
FollowsGrantsUserList bool
|
||||
// BoundBuckets are the providers' buckets of entries, `<module>_bound`, a provisioner client reads its own
|
||||
// machine's keys of (mesh-sdk go/provider, issue 679): every bucket named bound of a module the mesh places.
|
||||
BoundBuckets []string
|
||||
// CreatesStreams are the streams the grants part's principal alone may create (novox/hq ADR 0547 §5; the operator,
|
||||
// 2026-10-26 08:54): the stream of every bucket, log and stream a manifest of a module placed anywhere declares,
|
||||
// by name. Created only, never updated or deleted: a new module state is on the bus when the grants part hears its
|
||||
// manifest, not at the controller's next start (issue 1790).
|
||||
CreatesStreams []string
|
||||
// CreatesConsumers are the durable consumers the grants part's principal alone may create beside the controller
|
||||
// (novox/hq ADR 0561 §6): each placed module's on EVENTS and each process's on its work queue (AccountConsumers), by
|
||||
// stream and name. Created only, never updated or deleted.
|
||||
CreatesConsumers []ModuleConsumer
|
||||
|
||||
// Login is a node principal's login when it is not `node.<n>`: its second login, `node.<n>--b`, composed beside or
|
||||
// in place of the first as the vault's node-logins says (novox/hq ADR 0583 §1, issue 774). Each login is granted
|
||||
// the node's permissions and the node's inbox `_INBOX.node.<n>`; what it calls, it calls in its own name, so the
|
||||
// vault tells the login that confirms a rotation from the one it replaces (§3).
|
||||
Login string
|
||||
// RotatesItsLogin is a machine principal whose node-engine reads its own node's key of the vault's node-logins,
|
||||
// and no other key, and confirms a switch with node-account-confirm in its own name (novox/hq ADR 0583 §1 insight,
|
||||
// §3). Set while the vault's holder serves rotate-node.
|
||||
RotatesItsLogin bool
|
||||
|
||||
// Redeems is an enrolment principal that may call the seat mesh-enrolment's redeem in its own name, and nothing else
|
||||
// of it (novox/hq ADR 0587 §4): set while the seat's holder serves redeem. LeaseHandedAway drops the controller's
|
||||
// enrolment subject from it, which is served no more from the lease write (§6).
|
||||
Redeems bool
|
||||
LeaseHandedAway bool
|
||||
|
||||
// VaultEnrols says the vault's holder serves an enrolment verb (VerbsTheEnrolmentAsksTheVault): every principal but
|
||||
// the module mesh-enrolment and its processes is then denied them, however wide its grant (ADR 0587 §1).
|
||||
VaultEnrols bool
|
||||
|
||||
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
|
||||
// and never appears here: this file is written to a node's disk and read by a server, and a
|
||||
// secret that can be read from a configuration file is a secret with a wider blast radius
|
||||
// than the one it protects (novox/hq design 29 §10).
|
||||
PasswordHash string
|
||||
}
|
||||
|
||||
// enrolmentPrefix is the space every enrolling node's user and inbox live under, so the one place the
|
||||
// controller may answer an enrolment is derived from the same constant the user is named from.
|
||||
const enrolmentPrefix = "enrol"
|
||||
|
||||
// safeSubject refuses anything that would change the meaning of a subject rather than sit inside
|
||||
// one. A name carrying a dot would silently widen a permission by adding a token; a name carrying
|
||||
// `>` or `*` would widen it to a wildcard, which is the whole authority model gone.
|
||||
var safeSubject = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// Username is how a principal is named to the server. The node is part of it, so the same module
|
||||
// on two machines holds two users, each sealed to its own — the rule management.go already
|
||||
// applies, kept.
|
||||
func (p Principal) Username() string {
|
||||
switch p.Kind {
|
||||
case KindPerson:
|
||||
return "person." + p.Module
|
||||
case KindModule, KindNodeTools:
|
||||
// The runtime is named exactly as the module it stands for would have been: the mesh
|
||||
// issues its credential through the same path a module's takes (`module issue`), and
|
||||
// that path knows the node and the module, not the kind.
|
||||
return p.Node + "." + p.Module
|
||||
case KindProcess:
|
||||
// One account per process and machine (novox/hq ADR 0436 §2): `<machine>.<module>.<process id>`,
|
||||
// never the module's own name alone, which is a different account with a different secret.
|
||||
return p.Node + "." + p.Module + "." + p.Process
|
||||
case KindNode:
|
||||
if p.Login != "" {
|
||||
return p.Login
|
||||
}
|
||||
return "node." + p.Node
|
||||
case KindLedger:
|
||||
// `ledger.<machine>.<module>`: never `<node>.<module>`, which is the module's own account or the tool runner's,
|
||||
// so the module's code never holds it.
|
||||
return "ledger." + p.Node + "." + p.Module
|
||||
case KindProvisioner:
|
||||
// Named for the machine, as a node is: one client per machine, whichever module runs it, and never
|
||||
// `<node>.<module>`, which is the module's own account or the tool runner's.
|
||||
return "provisioner." + p.Node
|
||||
case KindEnrolment:
|
||||
// Per token, not one shared user. **The inbox is the reason**: with a single `enrolment`
|
||||
// user every machine enrolling at once could read every other's answer, and an answer
|
||||
// carries that node's credentials sealed to it. Design 25 §6 says the inbox a token
|
||||
// derives, and a permission belongs to a user, so the user is per token.
|
||||
//
|
||||
// Named after the node, which **is** the token's id: a token is issued for a node record,
|
||||
// the mesh holds one live claim per record, and the node's name is the one identifier both
|
||||
// sides already have before anything else is agreed. It is also exactly what the other
|
||||
// transport does, where the account is named after the node and the secret is its password.
|
||||
return enrolmentPrefix + "." + p.Node
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// inbox is a principal's own reply space. No user is ever granted a bare `_INBOX.>` (design 25
|
||||
// §4): with one account, inbox privacy is the permission list or it is nothing, so each user's
|
||||
// inbox is derived from its own identity and its permissions name that prefix and no other.
|
||||
//
|
||||
// A node's two logins share the node's inbox (novox/hq ADR 0583 §1): its answers reach it whichever login it dials as.
|
||||
func (p Principal) inbox() string {
|
||||
if p.Kind == KindNode {
|
||||
return "_INBOX.node." + p.Node + ".>"
|
||||
}
|
||||
return "_INBOX." + p.Username() + ".>"
|
||||
}
|
||||
|
||||
// ResponseTTL is how long the bus lets a principal answer a request it received. Its one answer has
|
||||
// to come inside this, and a seat's holder answers within link.AnswerWithin — inside it by design.
|
||||
// **A broker reloading its user list forgets every answer it was about to permit**, whatever this
|
||||
// says (novox/hq issue 265): a call that is still running when the list reloads has its answer
|
||||
// refused, which is why a holder answers before it does what can reload it.
|
||||
const ResponseTTL = time.Minute
|
||||
|
||||
// GrantsModule is the one module whose principal publishes the modules' memberships (novox/hq ADR 0547 §5): the grants
|
||||
// part, which composes them from the manifests and placement. It defines no stream and is granted no other subject here.
|
||||
const GrantsModule = "mesh-grants"
|
||||
|
||||
// The grants part's state memberships (novox/hq ADR 0571 clause 7): one key per assignment, `<node>.<module>`, its
|
||||
// membership (Membership) bare, as the assignments stream carries it, written only when it differs. A module's runtime
|
||||
// reads its own key, the tool runner its node's; the grants part alone writes it. Both sides go through
|
||||
// membership.EncodeGrantsMembership and membership.DecodeGrantsMembership (or BareGrantsMembership for a reader with a
|
||||
// type of its own), never json of a type of their own: the grants part once wrote its wrapper here and every reader
|
||||
// served nothing (issue 774). The decoder still reads that wrapper, for the transition.
|
||||
const (
|
||||
GrantsMembershipsState = "memberships"
|
||||
GrantsMembershipsBucket = GrantsModule + "_" + GrantsMembershipsState
|
||||
)
|
||||
|
||||
// MembershipKey is an assignment's key in the grants part's memberships: `<node>.<module>`.
|
||||
func MembershipKey(node, module string) string { return node + "." + module }
|
||||
@@ -1,8 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"git.novox.be/novox/mesh-sdk/go/membership"
|
||||
)
|
||||
|
||||
// AssignmentsStream holds every assignment's membership, the newest per subject.
|
||||
const AssignmentsStream = membership.AssignmentsStream
|
||||
@@ -1,7 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// isNewTraffic says whether a seat carries any of the rules above, so a seat that carries none is
|
||||
// composed exactly as before them.
|
||||
func (s Seat) isNewTraffic() bool {
|
||||
return s.Kinded || len(s.ByCaller) > 0 || len(s.Proofs) > 0 || len(s.Records) > 0 || s.PulledByKind
|
||||
}
|
||||
@@ -1,48 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
|
||||
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
|
||||
const (
|
||||
StateMaxValueBytes = 256 * 1024
|
||||
)
|
||||
|
||||
// A Bucket is one module's declared state as the bus holds it.
|
||||
type Bucket struct {
|
||||
Module string
|
||||
Name string
|
||||
// History is how many values a key keeps; zero is one.
|
||||
History int
|
||||
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
|
||||
TTLSeconds int
|
||||
// PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
|
||||
// 0260), and is granted that key and no other.
|
||||
PerMachine bool
|
||||
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
|
||||
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
|
||||
KeyTTL bool
|
||||
}
|
||||
|
||||
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
|
||||
// underscore, which neither may contain, so two modules can never derive one bucket.
|
||||
func BucketName(module, name string) string { return module + "_" + name }
|
||||
|
||||
// Bucket is this bucket's name on the bus.
|
||||
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
|
||||
|
||||
// Why is carried into the server's description of the bucket, so somebody reading the server's
|
||||
// own state finds whose it is and why it is kept.
|
||||
func (b Bucket) Why() string {
|
||||
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
|
||||
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
|
||||
b.Module, b.Name, b.Module, b.Module)
|
||||
}
|
||||
|
||||
// KeyedRead is one key of another module's state a principal reads, and no other key of it.
|
||||
type KeyedRead struct {
|
||||
Read string `json:"read"`
|
||||
Key string `json:"key"`
|
||||
}
|
||||
@@ -1,196 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Retention is how a stream decides what to keep, which is the whole of what distinguishes the
|
||||
// mesh's four relationships on the wire (design 29 §4).
|
||||
type Retention string
|
||||
|
||||
const (
|
||||
// RetentionWorkQueue: a message is removed once a consumer acknowledges it. Exactly one
|
||||
// worker does the work, and a worker that dies has its message redelivered.
|
||||
RetentionWorkQueue Retention = "workqueue"
|
||||
// RetentionLastPerSubject: only the newest message on each subject survives. This is the
|
||||
// state shape — a node that was away gets exactly the current declaration and nothing older.
|
||||
RetentionLastPerSubject Retention = "last_per_subject"
|
||||
// RetentionLimits: kept until it ages or the stream fills. Events, where a subscriber that
|
||||
// was down catches up and nobody is obliged to act.
|
||||
RetentionLimits Retention = "limits"
|
||||
)
|
||||
|
||||
// A Stream is one of the mesh's own, as the controller asserts it.
|
||||
type Stream struct {
|
||||
Name string
|
||||
Subjects []string
|
||||
Retention Retention
|
||||
// MaxAge in seconds, zero for unbounded. Per stream — JetStream has no per-subject age,
|
||||
// which is why differing retention between modules would mean a stream each.
|
||||
MaxAge int
|
||||
// MaxMsgsPerSubject caps each subject independently, so one noisy emitter cannot push
|
||||
// another's events out of a shared stream. Verified: with a cap of 3, ten messages on one
|
||||
// subject and one on another leave four in the stream, not three.
|
||||
MaxMsgsPerSubject int
|
||||
// Why is carried into the assertion so an operator reading the server's own state finds the
|
||||
// reason there, rather than only in a repository they may not have.
|
||||
Why string
|
||||
// Direct lets a client read a subject's last message without a consumer, which is how a
|
||||
// runtime reads its own membership with no JetStream API beyond one request (ADR 0160).
|
||||
Direct bool
|
||||
// MaxBytes bounds the stream's size, zero for unbounded. With DiscardNew a full stream refuses
|
||||
// what comes next rather than dropping what it holds: the publisher is told, and says so.
|
||||
MaxBytes int64
|
||||
DiscardNew bool
|
||||
// DuplicatesSeconds is the window in which a message id published twice is kept once; zero for the
|
||||
// server's default (two minutes).
|
||||
DuplicatesSeconds int
|
||||
// AllowMsgTTL lets a message carry a time to live of its own (`Nats-TTL`), for what must not outlive the
|
||||
// stream it came from (novox/hq ADR 0376 §3, §5).
|
||||
AllowMsgTTL bool
|
||||
// Restricted marks a module's stream restricted by a grant (novox/hq ADR 0363 §4), in the stream's own
|
||||
// metadata, so whoever keeps one of its messages reads it from the bus (ADR 0376 §3).
|
||||
Restricted bool
|
||||
// WorkQueue marks a module stream declared retention: work-queue (novox/hq ADR 0436 §6), in the
|
||||
// stream's own metadata, so a dead letter kept from it is read as drop-only from the bus, by the
|
||||
// declaration and not by the bus's work-queue retention alone — which SEAT_* and other mesh-internal
|
||||
// streams share without being one of ADR 0436's, and whose dead letters are still delivered again
|
||||
// (ADR 0264 §4).
|
||||
WorkQueue bool
|
||||
}
|
||||
|
||||
// RestrictedMetadata is the key a restricted stream's metadata says it under.
|
||||
const RestrictedMetadata = "mesh-restricted"
|
||||
|
||||
// WorkQueueMetadata is the key a module stream declared retention: work-queue (novox/hq ADR 0436 §6) says
|
||||
// it under: never set by a mesh-internal work-queue stream (a seat's, a check's), only by Stream.WorkQueue.
|
||||
const WorkQueueMetadata = "mesh-work-queue"
|
||||
|
||||
// What a durable consumer gave up on is kept (novox/hq issue 330, design 25 §3).
|
||||
//
|
||||
// **The server says it and keeps it; the controller copies it.** A consumer that handed a message over
|
||||
// as often as it may stops offering it and publishes `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES` with
|
||||
// the stream and the message's sequence. DeadLetterNoticesStream captures those advisories as the
|
||||
// server publishes them, so one said while no controller listens is still there when one starts. The
|
||||
// controller's consumer on it fetches the given-up message by its sequence, while the source stream
|
||||
// still holds it, and keeps a copy in DeadLettersStream under DeadLetterSubject, with the consumer,
|
||||
// the subject, how often it was handed over and when it was given up. It stays there until a person
|
||||
// delivers it again or drops it, with why; a condition is open for as long as it does.
|
||||
const (
|
||||
DeadLetterNoticesStream = "DEAD_LETTER_NOTICES"
|
||||
DeadLettersStream = "DEAD_LETTERS"
|
||||
// MaxDeliveriesAdvisories is the subject the server says a given-up message on.
|
||||
MaxDeliveriesAdvisories = "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>"
|
||||
deadLetterPrefix = "mesh.events.dead."
|
||||
againPrefix = "mesh.again."
|
||||
// DeadLettersBytes bounds the kept copies. Full, the stream refuses the next copy, which is said
|
||||
// as the consumer's condition; it never drops one it holds.
|
||||
DeadLettersBytes = 256 << 20
|
||||
)
|
||||
|
||||
// DeadLetterSubject is where a message one consumer gave up on is kept: `mesh.events.dead.<stream>.<consumer>`.
|
||||
func DeadLetterSubject(stream, consumer string) string {
|
||||
return deadLetterPrefix + stream + "." + consumer
|
||||
}
|
||||
|
||||
// DeadLetterOf is the stream and consumer a kept message's subject names; false for any other subject.
|
||||
func DeadLetterOf(subject string) (stream, consumer string, ok bool) {
|
||||
rest, found := strings.CutPrefix(subject, deadLetterPrefix)
|
||||
if !found {
|
||||
return "", "", false
|
||||
}
|
||||
stream, consumer, ok = strings.Cut(rest, ".")
|
||||
return stream, consumer, ok && stream != "" && consumer != "" && !strings.Contains(consumer, ".")
|
||||
}
|
||||
|
||||
// MeshStreams is the foundation set, in the order a person reads it.
|
||||
//
|
||||
// **EVENTS filters on the `event` token**, which is the reason that token exists. A module's
|
||||
// namespace carries both its events and its tool calls; a filter of `mesh.mod.*.>` would persist
|
||||
// every tool invocation in the mesh, and a tool call must never be persisted (design 25 §3 keeps
|
||||
// tools on core NATS, where a lost call is a timeout the caller already handles).
|
||||
func MeshStreams() []Stream {
|
||||
return []Stream{
|
||||
// CONTROL, the ended controller's work queue of what nodes reported, is no longer the mesh's (novox/hq ADR 0590
|
||||
// clause 6): not declared here, which would make it again; the bus module's tools remove it.
|
||||
{
|
||||
Name: "NODES",
|
||||
Subjects: []string{"mesh.node.*.declare"},
|
||||
Retention: RetentionLastPerSubject,
|
||||
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
|
||||
"n-1 by construction (issue 107)",
|
||||
},
|
||||
// The assignments stream (ADR 0160) is no longer the mesh's: a runtime reads its membership from the grants
|
||||
// part's state memberships (ADR 0571 clause 7), and the bus module's tools delete the stream at the lease write
|
||||
// (research 072 row 33), so it is not declared here, which would make it again.
|
||||
{
|
||||
Name: EventsStream,
|
||||
// A seat's own events ride here too: they are 1:many like any event, and the
|
||||
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
|
||||
// tools (`tool`), which must not be persisted.
|
||||
// And an event given up on, delivered again to the one consumer that gave it up
|
||||
// (novox/hq issue 330): under `mesh.again.<consumer>.`, which only that consumer filters.
|
||||
Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>", againPrefix + ">"},
|
||||
Retention: RetentionLimits,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
MaxMsgsPerSubject: 10000,
|
||||
// A restricted dead letter delivered again keeps what is left of its stream's age (ADR 0376 §5).
|
||||
AllowMsgTTL: true,
|
||||
Why: "a subscriber that was down catches up; tool traffic under the same prefix is " +
|
||||
"excluded by the event token; per-subject caps keep a noisy emitter from " +
|
||||
"evicting a quiet one without splitting the stream",
|
||||
},
|
||||
{
|
||||
Name: DeadLetterNoticesStream,
|
||||
Subjects: []string{MaxDeliveriesAdvisories},
|
||||
Retention: RetentionWorkQueue,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
Why: "the server's word that a consumer gave up on a message, kept until the controller has " +
|
||||
"copied the message into DEAD_LETTERS (novox/hq issue 330); a week, the longest the source " +
|
||||
"streams keep what they are about",
|
||||
},
|
||||
{
|
||||
Name: DeadLettersStream,
|
||||
Subjects: []string{deadLetterPrefix + ">"},
|
||||
Retention: RetentionLimits,
|
||||
MaxBytes: DeadLettersBytes,
|
||||
DiscardNew: true,
|
||||
DuplicatesSeconds: 24 * 60 * 60,
|
||||
AllowMsgTTL: true,
|
||||
Why: "every message a consumer gave up on, with its consumer, subject, deliveries and when, kept " +
|
||||
"until a person delivers it again or drops it with why (novox/hq issue 330); no age but a restricted " +
|
||||
"stream's message, which ages out with its stream (ADR 0376), and full it refuses the next copy " +
|
||||
"rather than drop one it holds",
|
||||
},
|
||||
{
|
||||
Name: ProvAppliedStream,
|
||||
Subjects: []string{ProvAppliedSubjects},
|
||||
Retention: RetentionLastPerSubject,
|
||||
Direct: true,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
Why: "each machine's provisioner client's newest word on each module's files, applied, waiting or " +
|
||||
"failed with the reason (novox/hq ADR 0349 §4): what a walk's end and a gate wait for (ADR 0362, " +
|
||||
"ADR 0349 §15), read by subject; a week, as the events are",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// ProvAppliedStream keeps what each machine's provisioner client last said of each module's files, newest per
|
||||
// subject (novox/hq ADR 0349 §4, ADR 0362, design 57 P6): `prov.applied.<node>.<module>` lies outside `mesh.*`, so
|
||||
// no other stream captures it, and a report said while no controller listened is still there to be read.
|
||||
const ProvAppliedStream = "PROV_APPLIED"
|
||||
|
||||
// ProvAppliedSubjects is every machine's client's reports: `prov.applied.<node>.<module>`, a module's name dots and
|
||||
// all (ProvisionerAppliedSubject).
|
||||
const ProvAppliedSubjects = "prov.applied.>"
|
||||
|
||||
// The ended controller's consumers' names (novox/hq ADR 0590 clause 6): kept only so its leftovers are named for their
|
||||
// removal (ControllerLeftoverConsumers) and the watchman can read whether `controller` on DEAD_LETTER_NOTICES still
|
||||
// stands.
|
||||
const (
|
||||
ControllerName = "controller"
|
||||
MergesConsumerName = "controller-merges"
|
||||
)
|
||||
|
||||
// EventsStream holds every module's and every role's events, a build's log among them.
|
||||
const EventsStream = "EVENTS"
|
||||
@@ -1,102 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// Declared is one module on one node, as composing its authority needs it.
|
||||
type Declared struct {
|
||||
Module string
|
||||
Emits []string
|
||||
Consumes []string
|
||||
Serves []string
|
||||
// Served are the config keys its served values and zone name fill from its settings (catalogue.ServedSettings): what
|
||||
// every node may read of its resolved values where it runs (Principal.ServedReads, issue 1126).
|
||||
Served []string
|
||||
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
|
||||
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
|
||||
// which is right: those seats are about who does a job, not about who may say what.
|
||||
Holds []Seat
|
||||
// Uses are the seats this module sends to.
|
||||
Uses []Seat
|
||||
// Issued are the tools that answer a credential its software issued, one per offer that says so (novox/hq
|
||||
// ADR 0305): the controller is granted to ask each in its own name, and no other tool so.
|
||||
Issued []string
|
||||
// Watches are the seats whose events it consumes.
|
||||
Watches []Seat
|
||||
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
|
||||
Invokes []string
|
||||
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
|
||||
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
|
||||
Waits []Wait
|
||||
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
|
||||
State []Bucket
|
||||
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
|
||||
Logs []Log
|
||||
// Streams are the streams it owns, each its events' one place (novox/hq ADR 0363).
|
||||
Streams []ModuleStream
|
||||
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
|
||||
// §1, §2, §6): one durable consumer per entry, named `<node>_<module>_<process id>`.
|
||||
Queues []QueueOwner
|
||||
// Hears are the module streams it hears on this machine, each with the subjects it was granted
|
||||
// (novox/hq ADR 0363 §5); HearsAStream says its `consumes` names a module that owns a stream, granted
|
||||
// or not, so its membership is issued again when a grant changes.
|
||||
Hears []Heard
|
||||
HearsAStream bool
|
||||
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
|
||||
Reads []string
|
||||
// ReadsRefused are the reads its manifest names that the owner's announcement does not make readable (novox/hq ADR
|
||||
// 0495 clause 3, issue 1790): granted nothing, kept to be said.
|
||||
ReadsRefused []string
|
||||
// ReadLogs are, of its reads, those naming another module's log (`<module>.<log>`, novox/hq ADR 0571 clause 1): filled
|
||||
// by Users and PlacementsOf from the logs the modules placed anywhere declare, never by a caller.
|
||||
ReadLogs []string
|
||||
// KeyedReads are keys of other modules' state it reads, each one key alone: what a seat's holder is
|
||||
// granted for the pieces the modules beside it offer (novox/hq ADR 0260).
|
||||
KeyedReads []KeyedRead
|
||||
// NoAccount says the module declares no own secret named broker, so no account could ever be
|
||||
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
|
||||
// record that does not say is composed as it always was.
|
||||
NoAccount bool
|
||||
// SnapshotsTheBus says the module holds mesh-broker — it is the bus — and so is the one module
|
||||
// granted the snapshot API, to copy the bus's streams for the night's backup (novox/hq ADR 0235).
|
||||
SnapshotsTheBus bool
|
||||
// Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be
|
||||
// 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more.
|
||||
Checks []string
|
||||
// Loads says the module has a compiled bundle whose entrypoints the machine's tool runtime loads and runs
|
||||
// (novox/hq ADR 0175; issue 1329): a helper that reaches the bus on the runner's account, so the runner
|
||||
// must carry the module although it declares no tools.
|
||||
Loads bool
|
||||
// RunsAs is the account the module runs as in a runtime of its own (novox/hq ADR 0259 §8): it is never
|
||||
// carried by the machine's runtime, and reaches the bus on its own account.
|
||||
RunsAs string
|
||||
// ProvisionsNode says the module runs the machine's provisioner client: it claims NodeProvisionerSeat and
|
||||
// declares the own secret ProvisionerSecret its account is delivered in (novox/hq ADR 0349 clause 11, design 57
|
||||
// P3). The client is given a bus account of its own, never the module's or the tool runner's.
|
||||
ProvisionsNode bool
|
||||
// Processes are this module's own processes that declare bus (novox/hq ADR 0436 §1), each getting a
|
||||
// principal of its own: granted exactly what its bus says, and nothing of this module's other grants —
|
||||
// composed whether or not the module itself has an account (NoAccount), because a process account is
|
||||
// delivered in its own secret, broker-<process id>, never the module's `broker`.
|
||||
Processes []ProcessPrincipal
|
||||
}
|
||||
|
||||
// ProcessPrincipal is one process of a module that declares bus (novox/hq ADR 0436 §1, §2), as the broker
|
||||
// composes its own account from: its own emits, its own work-queue consumers, its own state, whether it
|
||||
// reads the router's records of its module's own asks, and the tools and seat verbs it calls.
|
||||
type ProcessPrincipal struct {
|
||||
// Process is the process resource's id, naming the account <node>.<module>.<process id>.
|
||||
Process string
|
||||
// Emits are the events this process may publish, named as the module's own Emits are.
|
||||
Emits []string
|
||||
// Queues are the module's own work-queue streams this process is the one consumer of.
|
||||
Queues []QueueOwner
|
||||
// Removes are what its account may remove from the bus, as its bus declares them (the indexer's judge refuses any other; novox/hq
|
||||
// ADR 0564 Limits 4): a tools bundle's account, whose whole authority is then BusRemovalGrants.
|
||||
Removes []string
|
||||
// State is the module's own state, by local name, this process reads and writes.
|
||||
State []string
|
||||
// Asks says whether this process reads the records of its module's own asks (novox/hq ADR 0259 §3), on
|
||||
// whichever seats the module uses that keep them (the router's, operator-channel, among others).
|
||||
Asks bool
|
||||
// Invokes are the tools and seat verbs this process calls, each <module>.<tool> or a role's
|
||||
// seat:<seat>.<verb>, or * for every tool on the mesh.
|
||||
Invokes []string
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
|
||||
// emitter — a seat or a module — and the event, as the inventory resolved them.
|
||||
type Wait struct {
|
||||
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
|
||||
Name string
|
||||
Emitter string
|
||||
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
|
||||
Seat bool
|
||||
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
|
||||
// seat that names it last (ADR 0259 §3).
|
||||
Event string
|
||||
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
|
||||
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
|
||||
// on with it given, and refused without.
|
||||
Last string
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
// Package busretry waits out a bus that answers nothing for a few seconds (issue 774, the sweep of 2026-10-30: bus
|
||||
// blackouts at every user-list reload). nats-server 2.11's reloadAuthorization rebuilds the accounts, and for a few
|
||||
// seconds the mesh's account has none of its imports, $JS.API.> among them: every request is answered "no responders"
|
||||
// and every JetStream call "jetstream not enabled". A write or a call made in that window failed, and was lost.
|
||||
//
|
||||
// Only an error that says the request reached no one is waited out (Undelivered): nothing was done, so asking again
|
||||
// is safe even for a write that is not idempotent. A timeout, or any answer, is never asked again: the request may
|
||||
// have been delivered. The wait is bounded (Bound, about 30 s, backing off), and said on standard error when it ends
|
||||
// without an answer.
|
||||
//
|
||||
// This is the quick fix. The thorough one is a reload that keeps the account's imports, in the bus's holder.
|
||||
package busretry
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// Bound is how long a request that reached no one is asked again; FirstWait the first pause, doubled to MostWait.
|
||||
var (
|
||||
Bound = 30 * time.Second
|
||||
FirstWait = 250 * time.Millisecond
|
||||
MostWait = 4 * time.Second
|
||||
)
|
||||
|
||||
// Waits sets Bound, FirstWait and MostWait and returns what puts the earlier values back. They are state every caller
|
||||
// in a process shares, so a test sets them only through it, as t.Cleanup(busretry.Waits(...)): a test that set them
|
||||
// and left them set shortened the next test's bound to milliseconds, and a loaded check machine overran it (the check
|
||||
// of release sdk1, 2026-11-02). A bound a test waits on to succeed is seconds: no slow machine reaches it.
|
||||
func Waits(bound, first, most time.Duration) (restore func()) {
|
||||
was := [3]time.Duration{Bound, FirstWait, MostWait}
|
||||
Bound, FirstWait, MostWait = bound, first, most
|
||||
return func() { Bound, FirstWait, MostWait = was[0], was[1], was[2] }
|
||||
}
|
||||
|
||||
// Undelivered says err means the request reached no one: no responders, JetStream not enabled (for the account), or
|
||||
// no response from the stream after the client's own retries on no responders; also said in words by a runtime that
|
||||
// passed the error on.
|
||||
func Undelivered(err error) bool {
|
||||
var answered *Answered
|
||||
if err == nil || errors.As(err, &answered) {
|
||||
return false
|
||||
}
|
||||
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, jetstream.ErrJetStreamNotEnabled) ||
|
||||
errors.Is(err, jetstream.ErrJetStreamNotEnabledForAccount) || errors.Is(err, jetstream.ErrNoStreamResponse) ||
|
||||
errors.Is(err, nats.ErrJetStreamNotEnabled) || errors.Is(err, nats.ErrJetStreamNotEnabledForAccount) {
|
||||
return true
|
||||
}
|
||||
words := strings.ToLower(err.Error())
|
||||
return strings.Contains(words, "no responders") || strings.Contains(words, "jetstream not enabled")
|
||||
}
|
||||
|
||||
// Answered is an error a callee answered with: the request was delivered, whatever its words say ("jetstream not
|
||||
// enabled" in a callee's own answer), so Undelivered never counts it and Do never asks it again.
|
||||
type Answered struct{ Words string }
|
||||
|
||||
func (a *Answered) Error() string { return a.Words }
|
||||
|
||||
// Do runs try, and again while its error says the request reached no one, until Bound has passed. blackout, when
|
||||
// given, is asked after each such error and Do stops waiting when it says the bus is not blacked out: a call nobody
|
||||
// answers because its callee is not running fails at once, as before. what names the request in the error.
|
||||
func Do[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
|
||||
v, err := try()
|
||||
if !Undelivered(err) || (blackout != nil && !blackout()) {
|
||||
return v, err
|
||||
}
|
||||
start, wait := time.Now(), FirstWait
|
||||
for time.Since(start) < Bound {
|
||||
time.Sleep(wait)
|
||||
wait = min(wait*2, MostWait)
|
||||
v, err = try()
|
||||
if !Undelivered(err) || (blackout != nil && !blackout()) {
|
||||
return v, err
|
||||
}
|
||||
}
|
||||
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
|
||||
"account's imports for a few seconds; this was longer): %w", what, time.Since(start).Round(time.Second), err)
|
||||
fmt.Fprintf(os.Stderr, "[mesh-sdk] %v\n", err)
|
||||
return v, err
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
package busretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the
|
||||
// bound; a blackout probe that says the bus answers ends the wait at once.
|
||||
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
errs []error // what each try answers; the last repeats
|
||||
blackout func() bool
|
||||
tries int // at least
|
||||
atMost int
|
||||
delivered bool
|
||||
}{
|
||||
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true},
|
||||
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true},
|
||||
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true},
|
||||
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false},
|
||||
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false},
|
||||
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false},
|
||||
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false},
|
||||
} {
|
||||
// A row that ends in an answer waits on a bound of seconds, which no slow machine reaches; only the row that
|
||||
// runs past it has a short one, and asks only that it was asked again at least once.
|
||||
bound := 10 * time.Second
|
||||
if strings.HasPrefix(row.name, "blacked out") {
|
||||
bound = 100 * time.Millisecond
|
||||
}
|
||||
t.Cleanup(Waits(bound, time.Millisecond, 4*time.Millisecond))
|
||||
tries := 0
|
||||
_, err := Do("the test", func() (int, error) {
|
||||
e := row.errs[min(tries, len(row.errs)-1)]
|
||||
tries++
|
||||
return tries, e
|
||||
}, row.blackout)
|
||||
if tries < row.tries || tries > row.atMost || (err == nil) != row.delivered {
|
||||
t.Errorf("%s: %d tries, %v", row.name, tries, err)
|
||||
}
|
||||
if strings.HasPrefix(row.name, "blacked out") && (!Undelivered(err) || !strings.Contains(fmt.Sprint(err), "the bus answered nothing")) {
|
||||
t.Errorf("%s: %v", row.name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// busStateCalls are the method prefixes of the calls a bundle makes on its own bus state through the runtime (novox/hq
|
||||
// ADR 0595 §9, design 25 § 12.5): their shapes and names are package buscalls's, and package busstate the typed way
|
||||
// to make them.
|
||||
var busStateCalls = []string{"mesh/objects.", "mesh/stream.", "mesh/workqueue.", "mesh/ledger."}
|
||||
|
||||
// BusState asks the runtime one call on this bundle's own bus state, an entry its module asks for in `requires`, named
|
||||
// by its local name in params; the runtime serves it from the module's own bindings only. A method that is none of
|
||||
// those calls is refused here, before the runtime is asked.
|
||||
func BusState(method string, params any) (json.RawMessage, error) {
|
||||
known := false
|
||||
for _, p := range busStateCalls {
|
||||
known = known || strings.HasPrefix(method, p)
|
||||
}
|
||||
if !known {
|
||||
return nil, fmt.Errorf("%s is no call on a bundle's own bus state (%s)", method, strings.Join(busStateCalls, ", "))
|
||||
}
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.ask(method, params, 60*time.Second)
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
// Package busstate is how a bundle uses its own bus state through the runtime (novox/hq ADR 0595 §9, design 25 §
|
||||
// 12.5): the object store's put, get, list, delete and fill, each naming an entry by the local name its module asks for
|
||||
// it by in `requires`. The runtime serves the calls from the module's own bindings, its own entries only. The shapes
|
||||
// are package buscalls's; this package makes the calls.
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// Objects is one object store the module asks for in `requires` (the template objects, ADR 0595 §2), by its local name.
|
||||
type Objects struct{ local string }
|
||||
|
||||
// ObjectsOf is the object store of that local name.
|
||||
func ObjectsOf(local string) Objects { return Objects{local: local} }
|
||||
|
||||
func call[T any](method string, params any) (T, error) {
|
||||
var out T
|
||||
raw, err := stdio.BusState(method, params)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return out, fmt.Errorf("the runtime's answer to %s: %w", method, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it.
|
||||
func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
|
||||
return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name,
|
||||
Data: data, Description: description})
|
||||
}
|
||||
|
||||
// Get is one object, its info and its bytes.
|
||||
func (o Objects) Get(name string) (buscalls.ObjectData, error) {
|
||||
return call[buscalls.ObjectData](buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
}
|
||||
|
||||
// List is every object the store holds.
|
||||
func (o Objects) List() ([]buscalls.ObjectInfo, error) {
|
||||
l, err := call[buscalls.ObjectsListed](buscalls.ObjectsList, buscalls.ObjectsRequest{Objects: o.local})
|
||||
return l.Objects, err
|
||||
}
|
||||
|
||||
// Delete removes one object.
|
||||
func (o Objects) Delete(name string) error {
|
||||
_, err := call[buscalls.Done](buscalls.ObjectsDelete, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
return err
|
||||
}
|
||||
|
||||
// Fill is how much the store holds against its cap.
|
||||
func (o Objects) Fill() (buscalls.Fill, error) {
|
||||
return call[buscalls.Fill](buscalls.ObjectsInfo, buscalls.ObjectsRequest{Objects: o.local})
|
||||
}
|
||||
@@ -1,102 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's
|
||||
// incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the
|
||||
// runtime answered; a method that is no bus-state call is refused before the runtime is asked.
|
||||
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
|
||||
t.Setenv(stdio.ServedAs, "distribution")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
type result struct {
|
||||
what string
|
||||
err error
|
||||
got any
|
||||
}
|
||||
results := make(chan result, 8)
|
||||
go func() {
|
||||
_ = stdio.ServeOn("", []stdio.Tool{{Name: "keep", Run: func(map[string]any) (any, error) {
|
||||
go func() {
|
||||
in := ObjectsOf("incoming")
|
||||
info, err := in.Put("t1.part0", []byte("hello"), "a part")
|
||||
results <- result{"put", err, info.Digest}
|
||||
data, err := in.Get("t1.part0")
|
||||
results <- result{"get", err, string(data.Data)}
|
||||
list, err := in.List()
|
||||
results <- result{"list", err, len(list)}
|
||||
results <- result{"delete", in.Delete("t1.part0"), nil}
|
||||
fill, err := in.Fill()
|
||||
results <- result{"fill", err, fill.MaxBytes}
|
||||
_, err = stdio.BusState("mesh/state.get", nil)
|
||||
results <- result{"refused", err, nil}
|
||||
}()
|
||||
return "started", nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
lines := bufio.NewScanner(outR)
|
||||
next := func() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "keep"}})
|
||||
answers := map[string]any{
|
||||
buscalls.ObjectsPut: buscalls.ObjectInfo{Name: "t1.part0", Size: 5, Digest: "SHA-256=x"},
|
||||
buscalls.ObjectsGet: buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: "t1.part0"}, Data: []byte("hello")},
|
||||
buscalls.ObjectsList: buscalls.ObjectsListed{Objects: []buscalls.ObjectInfo{{Name: "t1.part0"}}},
|
||||
buscalls.ObjectsDelete: buscalls.Done{},
|
||||
buscalls.ObjectsInfo: buscalls.Fill{Count: 0, MaxBytes: 20 << 30},
|
||||
}
|
||||
asked := map[string]map[string]any{}
|
||||
for len(asked) < len(answers) {
|
||||
m := next()
|
||||
method, _ := m["method"].(string)
|
||||
if method == "" {
|
||||
continue // the tool's own answer
|
||||
}
|
||||
p, _ := m["params"].(map[string]any)
|
||||
asked[method] = p
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": answers[method]})
|
||||
}
|
||||
want := map[string]any{"put": "SHA-256=x", "get": "hello", "list": 1, "delete": nil, "fill": uint64(20 << 30)}
|
||||
for range 5 {
|
||||
r := <-results
|
||||
if r.err != nil || r.got != want[r.what] {
|
||||
t.Errorf("%s answered %v, %v; want %v", r.what, r.got, r.err, want[r.what])
|
||||
}
|
||||
}
|
||||
if r := <-results; r.err == nil {
|
||||
t.Error("a call that is not on the bundle's bus state was asked of the runtime")
|
||||
}
|
||||
for method, p := range asked {
|
||||
if p["objects"] != "incoming" {
|
||||
t.Errorf("%s named the store %v, not its local name", method, p["objects"])
|
||||
}
|
||||
}
|
||||
if p := asked[buscalls.ObjectsPut]; p["name"] != "t1.part0" || p["data"] != "aGVsbG8=" || p["description"] != "a part" {
|
||||
t.Errorf("put asked %v", p)
|
||||
}
|
||||
}
|
||||
@@ -1,108 +0,0 @@
|
||||
// Package calladdress reads the key a module calls a tool or a seat verb by, as a module's invokes names it, with the
|
||||
// machine it is asked of after '@': `seat:<seat>.<verb>[@<node>]` or `<module>.<tool>[@<node>]` (novox/hq ADR 0159,
|
||||
// ADR 0469 §3). Every caller in the SDK parses its key here before it publishes anything, so a malformed key fails at
|
||||
// once and says why, never as a call nobody serves and a timeout (novox/hq issue 774, the lease write's stop asked on
|
||||
// `...call.stop@novox-be...`).
|
||||
package calladdress
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Address is a parsed key: a seat and its verb, or a module and its tool, and the machine named, if any.
|
||||
type Address struct {
|
||||
Seat, Verb string
|
||||
Module, Tool string
|
||||
Node string
|
||||
}
|
||||
|
||||
// Key is the address as a module's invokes names it, without the machine.
|
||||
func (a Address) Key() string {
|
||||
if a.Seat != "" {
|
||||
return "seat:" + a.Seat + "." + a.Verb
|
||||
}
|
||||
return a.Module + "." + a.Tool
|
||||
}
|
||||
|
||||
// Parse reads a key, refusing one with more than one '@', a node that is not one subject token, or a seat, verb, module
|
||||
// or tool that holds a wildcard, white space or (but a tool) a dot.
|
||||
func Parse(key string) (Address, error) {
|
||||
name, node, err := cutNode(key)
|
||||
if err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
if rest, ok := strings.CutPrefix(name, "seat:"); ok {
|
||||
seat, verb, ok := strings.Cut(rest, ".")
|
||||
if !ok || seat == "" || verb == "" {
|
||||
return Address{}, fmt.Errorf("%q names no seat and verb (seat:<seat>.<verb>)", key)
|
||||
}
|
||||
if err := tokens(key, seat, verb); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
return Address{Seat: seat, Verb: verb, Node: node}, nil
|
||||
}
|
||||
module, tool, ok := strings.Cut(name, ".")
|
||||
if !ok || module == "" || tool == "" {
|
||||
return Address{}, fmt.Errorf("%q is not <module>.<tool>, and does not start seat:<seat>.<verb>", key)
|
||||
}
|
||||
if err := tokens(key, module); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
if err := checkTool(key, tool); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
return Address{Module: module, Tool: tool, Node: node}, nil
|
||||
}
|
||||
|
||||
// CheckOwn is Parse for a caller that may also name a tool of its own module by its bare name (the runtime's mesh/ask).
|
||||
func CheckOwn(key string) error {
|
||||
name, _, err := cutNode(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.HasPrefix(name, "seat:") && !strings.Contains(name, ".") {
|
||||
return tokens(key, name)
|
||||
}
|
||||
_, err = Parse(key)
|
||||
return err
|
||||
}
|
||||
|
||||
// CheckNode refuses a node name that is not one subject token.
|
||||
func CheckNode(node string) error {
|
||||
if node == "" || strings.ContainsAny(node, ".*>@ \t\r\n") {
|
||||
return fmt.Errorf("%q is not a node name (one subject token: no '.', '*', '>', '@' or space)", node)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cutNode(key string) (name, node string, err error) {
|
||||
name, node, at := strings.Cut(key, "@")
|
||||
if !at {
|
||||
return key, "", nil
|
||||
}
|
||||
if strings.Contains(node, "@") {
|
||||
return "", "", fmt.Errorf("%q holds more than one '@': a key names at most one machine", key)
|
||||
}
|
||||
if err := CheckNode(node); err != nil {
|
||||
return "", "", fmt.Errorf("%q: %w", key, err)
|
||||
}
|
||||
return name, node, nil
|
||||
}
|
||||
|
||||
func tokens(key string, parts ...string) error {
|
||||
for _, p := range parts {
|
||||
if p == "" || strings.ContainsAny(p, ".*> \t\r\n") {
|
||||
return fmt.Errorf("%q: %q is not one subject token (no '.', '*', '>' or space)", key, p)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkTool refuses a tool name with a wildcard or white space; a dot is kept, as the tool runner serves it.
|
||||
func checkTool(key, tool string) error {
|
||||
if strings.ContainsAny(tool, "*> \t\r\n") || strings.HasPrefix(tool, ".") || strings.HasSuffix(tool, ".") {
|
||||
return fmt.Errorf("%q: %q is not one subject token (no '*', '>' or space)", key, tool)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
package calladdress
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Issue 774 notes 84-87 (the lease write's stop asked of `seat:node-service-manager.stop@novox-be` published on a subject
|
||||
// nobody serves and timed out after 30 s): a key names its machine after '@', and every other malformed key is refused
|
||||
// before anything is published, naming why.
|
||||
func TestAKeyIsParsedOrRefusedBeforeItIsPublished(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
key, want string // want: "seat|<seat>|<verb>|<node>", "mod|<module>|<tool>|<node>", or a refusal's words
|
||||
}{
|
||||
{"seat:node-service-manager.stop@novox", "seat|node-service-manager|stop|novox"},
|
||||
{"seat:operator-channel.tell", "seat|operator-channel|tell|"},
|
||||
{"mesh-build-go.fetch@ace", "mod|mesh-build-go|fetch|ace"},
|
||||
{"nats.nats_users", "mod|nats|nats_users|"},
|
||||
{"seat:node-service-manager.stop@novox@g14", "more than one '@'"},
|
||||
{"seat:node-service-manager.stop@", "not a node name"},
|
||||
{"seat:node-service-manager.stop@nov.ox", "not a node name"},
|
||||
{"seat:node-files.*", "not one subject token"},
|
||||
{"mesh-build-go.fe tch", "not one subject token"},
|
||||
{"mesh>build.fetch", "not one subject token"},
|
||||
{"seat:noverb", "names no seat and verb"},
|
||||
{"nodot", "is not <module>.<tool>"},
|
||||
{"", "is not <module>.<tool>"},
|
||||
} {
|
||||
a, err := Parse(row.key)
|
||||
got := ""
|
||||
switch {
|
||||
case err != nil:
|
||||
got = err.Error()
|
||||
if !strings.Contains(got, row.want) {
|
||||
t.Errorf("Parse(%q) refused %q, want %q", row.key, got, row.want)
|
||||
}
|
||||
continue
|
||||
case a.Seat != "":
|
||||
got = "seat|" + a.Seat + "|" + a.Verb + "|" + a.Node
|
||||
default:
|
||||
got = "mod|" + a.Module + "|" + a.Tool + "|" + a.Node
|
||||
}
|
||||
if got != row.want {
|
||||
t.Errorf("Parse(%q) = %s, want %s", row.key, got, row.want)
|
||||
}
|
||||
}
|
||||
if a, err := Parse("seat:node-files.read@g14"); err != nil || a.Key() != "seat:node-files.read" {
|
||||
t.Errorf("the key without its machine: %q, %v", a.Key(), err)
|
||||
}
|
||||
if err := CheckOwn("list@ace"); err != nil {
|
||||
t.Errorf("a module's own tool by its bare name was refused: %v", err)
|
||||
}
|
||||
if err := CheckOwn("li*st"); err == nil {
|
||||
t.Error("a bare name that is not one token was accepted")
|
||||
}
|
||||
}
|
||||
+6
-38
@@ -22,19 +22,8 @@ const MetaCaller = "mesh/caller"
|
||||
// parts are safe in a subject. Anything else the runtime named is not a caller and reads as Unproven.
|
||||
var busUser = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the runtime names the id of the call in flight, the id a ledger's
|
||||
// apply names so the runtime stamps the caller it proved as the operation's author (novox/hq ADR 0597 §7):
|
||||
// buscalls.MetaCall, written here too so this package imports nothing of buscalls.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// callID is the shape of a call's id: one subject-safe token, as the runtime makes it. Anything else reads as none.
|
||||
var callID = regexp.MustCompile(`^[A-Za-z0-9_-]{1,128}$`)
|
||||
|
||||
// inFlight is what the runtime said of one call while its tool runs.
|
||||
type inFlight struct{ caller, call string }
|
||||
|
||||
// calls holds the caller and the id of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → inFlight
|
||||
// calls holds the caller of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → string
|
||||
|
||||
func argsKey(args map[string]any) unsafe.Pointer {
|
||||
return reflect.ValueOf(args).UnsafePointer()
|
||||
@@ -48,32 +37,11 @@ func CallerOf(args map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).caller
|
||||
return c.(string)
|
||||
}
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// CallIDOf is the id of the call that was handed args, as the runtime named it under MetaCall: what a ledger's apply
|
||||
// names (buscalls.ApplyRequest.Call) so the operation's author is the caller the runtime proved. Empty for arguments
|
||||
// no call was handed, or a call the runtime named no id for; read from `_meta`, never from the arguments.
|
||||
func CallIDOf(args map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).call
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callIn is the call id a `tools/call`'s `_meta` names, or empty.
|
||||
func callIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCall].(string); ok && callID.MatchString(s) {
|
||||
return s
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callerIn is the caller a `tools/call`'s `_meta` names, or Unproven.
|
||||
func callerIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCaller].(string); ok && busUser.MatchString(s) {
|
||||
@@ -82,10 +50,10 @@ func callerIn(meta map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// runAs runs a tool with the caller and the call's id held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller, call string, args map[string]any) (any, error) {
|
||||
// runAs runs a tool with the caller held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller string, args map[string]any) (any, error) {
|
||||
key := argsKey(args)
|
||||
calls.Store(key, inFlight{caller, call})
|
||||
calls.Store(key, caller)
|
||||
defer calls.Delete(key)
|
||||
return t.Run(args)
|
||||
}
|
||||
|
||||
+5
-85
@@ -6,33 +6,24 @@ import (
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// callWith serves one tool that answers the caller CallerOf reads from its arguments, sends it one tools/call
|
||||
// with the params given, and returns the caller it answered.
|
||||
func callWith(t *testing.T, params map[string]any) string {
|
||||
caller, _ := callAnswering(t, params)
|
||||
return caller
|
||||
}
|
||||
|
||||
// callAnswering is callWith answering the call's id CallIDOf read too.
|
||||
func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
servesAlone(t)
|
||||
t.Helper()
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("tracker", []Tool{{Name: "who", Run: func(args map[string]any) (any, error) {
|
||||
return map[string]any{"caller": CallerOf(args), "call": CallIDOf(args)}, nil
|
||||
return map[string]any{"caller": CallerOf(args)}, nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
defer inW.Close()
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": params})
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
lines := bufio.NewScanner(outR)
|
||||
done := make(chan [2]string, 1)
|
||||
done := make(chan string, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m struct {
|
||||
@@ -45,20 +36,19 @@ func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
var answer struct {
|
||||
Caller string `json:"caller"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
if len(m.Result.Content) > 0 {
|
||||
_ = json.Unmarshal([]byte(m.Result.Content[0].Text), &answer)
|
||||
}
|
||||
done <- [2]string{answer.Caller, answer.Call}
|
||||
done <- answer.Caller
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case got := <-done:
|
||||
return got[0], got[1]
|
||||
return got
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle did not answer the call")
|
||||
return "", ""
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -71,28 +61,6 @@ func TestAToolReadsTheCallerTheRuntimeNamed(t *testing.T) {
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("the tool read caller %q, want the one the runtime named", got)
|
||||
}
|
||||
// The call's id is the runtime's too (ADR 0597 §7): read from `_meta`, never from the arguments, and none when it
|
||||
// is not one token.
|
||||
for _, c := range []struct {
|
||||
args, meta map[string]any
|
||||
want string
|
||||
}{
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools", MetaCall: "c-7Hq2"}, "c-7Hq2"},
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools"}, ""},
|
||||
{map[string]any{MetaCall: "c-forged"}, map[string]any{}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: "a b"}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: 7}, ""},
|
||||
} {
|
||||
if _, call := callAnswering(t, map[string]any{"name": "who", "arguments": c.args, "_meta": c.meta}); call != c.want {
|
||||
t.Errorf("args %v, meta %v: the tool read call %q, want %q", c.args, c.meta, call, c.want)
|
||||
}
|
||||
}
|
||||
if CallIDOf(map[string]any{"x": 1}) != "" || CallIDOf(nil) != "" {
|
||||
t.Error("arguments no call was handed have a call id")
|
||||
}
|
||||
if MetaCall != buscalls.MetaCall {
|
||||
t.Errorf("the call id's place is %q here and %q in buscalls", MetaCall, buscalls.MetaCall)
|
||||
}
|
||||
}
|
||||
|
||||
// A caller named in the arguments is not the caller: an argument is the caller's to write.
|
||||
@@ -107,54 +75,6 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("with a caller in the arguments the tool read %q, want the runtime's", got)
|
||||
}
|
||||
// A module's trusted list of callers (issue 774, ADR 0563 §8) grants the callers it names as the runtime proved them:
|
||||
// a module on any machine, <machine>.<module> exactly, person.<name> exactly; no wildcard, never one unproven.
|
||||
callers, err := ParseCallers([]byte(`{"callers":["node-tools","ace.claude-code","person.jo","*"]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for caller, want := range map[string]bool{"shanks.node-tools": true, "ace.claude-code": true, "g14.claude-code": false,
|
||||
"person.jo": true, "person.node-tools": false, Unproven: false, "": false, "novox.mesh-vault": false} {
|
||||
if callers.Allows(caller) != want {
|
||||
t.Errorf("%q allowed: %v, want %v", caller, !want, want)
|
||||
}
|
||||
}
|
||||
if _, err := ParseCallers([]byte(`{"callers":["x"],"more":1}`)); err == nil {
|
||||
t.Error("a list with a field it does not know was read")
|
||||
}
|
||||
// A holder's list per verb (ADR 0627 §2): each verb's entries as above, plus every proved caller (AnyCaller) or
|
||||
// every operator's session (AnyPerson); an unproven caller never, a verb the holder does not serve refused.
|
||||
verbs, err := ParseVerbCallers(`{"publish":["mesh-build-go"],"versions":["*"],"release":["operator","sweep"]}`,
|
||||
[]string{"publish", "versions", "release", "remove"}, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range []struct {
|
||||
verb, caller string
|
||||
want bool
|
||||
}{{"publish", "ace.mesh-build-go", true}, {"publish", "ace.mesh-build-node", false}, {"versions", "g14.anything", true},
|
||||
{"versions", Unproven, false}, {"release", "person.jo", true}, {"release", "ace.operator", false},
|
||||
{"release", "novox.sweep", true}, {"remove", "novox.sweep", false}} {
|
||||
if got := verbs.Allowed(c.verb, c.caller) == nil; got != c.want {
|
||||
t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want)
|
||||
}
|
||||
}
|
||||
if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil {
|
||||
t.Error("a list naming a verb the holder does not serve was read")
|
||||
}
|
||||
if d, _ := ParseVerbCallers(" ", nil, VerbCallers{"publish": {"x"}}); len(d["publish"]) != 1 {
|
||||
t.Error("an empty value is not the holder's defaults")
|
||||
}
|
||||
// As mesh-config resolves the value: the object itself, or written as a string; null is the defaults.
|
||||
for _, raw := range []string{`{"publish":["a"]}`, `"{\"publish\":[\"a\"]}"`, `null`} {
|
||||
c, err := parseConfiguredVerbCallers(json.RawMessage(raw), []string{"publish"}, VerbCallers{"publish": {"a"}})
|
||||
if err != nil || c.Allowed("publish", "ace.a") != nil {
|
||||
t.Errorf("the config value %s read as %v, %v", raw, c, err)
|
||||
}
|
||||
}
|
||||
if ModuleOf("ace.mesh-build-go") != "mesh-build-go" {
|
||||
t.Error("ModuleOf does not read the module of a proved caller")
|
||||
}
|
||||
}
|
||||
|
||||
// A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is
|
||||
|
||||
-131
@@ -1,131 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
|
||||
// writes a list setting into a file the module reads: `{"callers": [...]}`. An entry is a module's name (that module on
|
||||
// any machine), exactly <machine>.<module>, or exactly person.<name>. Each caller is the runtime's proof (CallerOf), never
|
||||
// an argument; nobody beyond the list is granted, a wildcard is no entry, and an unproven caller never matches.
|
||||
type Callers []string
|
||||
|
||||
// ParseCallers reads a callers file: the list under `callers`, and nothing else.
|
||||
func ParseCallers(raw []byte) (Callers, error) {
|
||||
var f struct {
|
||||
Callers Callers `json:"callers"`
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&f); err != nil {
|
||||
return nil, fmt.Errorf("the callers setting is not a list of callers: %v", err)
|
||||
}
|
||||
return f.Callers, nil
|
||||
}
|
||||
|
||||
// Allows says whether caller, as the runtime proved it, is one the list names.
|
||||
func (c Callers) Allows(caller string) bool {
|
||||
if caller == "" || caller == Unproven {
|
||||
return false
|
||||
}
|
||||
for _, entry := range c {
|
||||
if entry == "" || strings.ContainsAny(entry, "*> ") {
|
||||
continue
|
||||
}
|
||||
if entry == caller {
|
||||
return true
|
||||
}
|
||||
if !strings.Contains(entry, ".") {
|
||||
if _, module, ok := strings.Cut(caller, "."); ok && module == entry && !strings.HasPrefix(caller, "person.") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// VerbCallers is a holder's trusted list per verb (novox/hq ADR 0627 §2, design 51 *Who may call which verb*): for each
|
||||
// verb of the seats it holds, the callers that may call it, each entry read as Callers reads it. Two entries more stand
|
||||
// for a whole class: AnyCaller lets every proved caller call the verb (a read the mesh keeps open), AnyPerson every
|
||||
// operator's session the runtime proved as person.<name>. It names no seat and no verb: the holder gives its verbs and
|
||||
// its defaults, and keeps any rule of its own seat (who may never be on a list) beside it.
|
||||
type VerbCallers map[string][]string
|
||||
|
||||
// The entries of a VerbCallers list that stand for a class of callers.
|
||||
const (
|
||||
AnyCaller = "*"
|
||||
AnyPerson = "operator"
|
||||
)
|
||||
|
||||
// ParseVerbCallers reads a holder's config value of verb to callers; an empty value is defaults. A verb not among verbs
|
||||
// is refused, so a list that names a verb the holder does not serve is said at once.
|
||||
func ParseVerbCallers(raw string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return defaults, nil
|
||||
}
|
||||
var c VerbCallers
|
||||
if err := json.Unmarshal([]byte(raw), &c); err != nil {
|
||||
return nil, fmt.Errorf("callers is not a JSON object of verb to callers: %w", err)
|
||||
}
|
||||
for verb := range c {
|
||||
if !slices.Contains(verbs, verb) {
|
||||
return nil, fmt.Errorf("callers names the verb %q, which the holder does not serve", verb)
|
||||
}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven
|
||||
// caller is never allowed, not even by AnyCaller.
|
||||
func (c VerbCallers) Allowed(verb, caller string) error {
|
||||
if caller == "" || caller == Unproven {
|
||||
return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb)
|
||||
}
|
||||
person := strings.HasPrefix(caller, "person.")
|
||||
for _, entry := range c[verb] {
|
||||
switch entry {
|
||||
case AnyCaller:
|
||||
return nil
|
||||
case AnyPerson:
|
||||
if person {
|
||||
return nil
|
||||
}
|
||||
default:
|
||||
if (Callers{entry}).Allows(caller) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("%s: refused for %s: not on the holder's caller list for %s", verb, caller, verb)
|
||||
}
|
||||
|
||||
// ModuleOf is the module of a proved module caller, which the runtime names <machine>.<module>.
|
||||
func ModuleOf(proved string) string {
|
||||
if _, module, ok := strings.Cut(proved, "."); ok {
|
||||
return module
|
||||
}
|
||||
return proved
|
||||
}
|
||||
|
||||
// ConfiguredVerbCallers is the holder's list as mesh-config resolved its config value key for this machine (Config):
|
||||
// a JSON object of verb to callers, or that object written as a string; an unset or empty value is defaults. A value
|
||||
// that cannot be read is an error, never the defaults, so a broken list is said, not widened.
|
||||
func ConfiguredVerbCallers(key string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
raw, err := Config(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
}
|
||||
return parseConfiguredVerbCallers(raw, verbs, defaults)
|
||||
}
|
||||
|
||||
func parseConfiguredVerbCallers(raw json.RawMessage, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
var text string
|
||||
if err := json.Unmarshal(raw, &text); err == nil || len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
|
||||
return ParseVerbCallers(text, verbs, defaults)
|
||||
}
|
||||
return ParseVerbCallers(string(raw), verbs, defaults)
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -29,7 +31,7 @@ func aServiceModule(t *testing.T) Manifest {
|
||||
"build": {"artifacts": [{"name": "code", "kind": "bundle", "language": "go", "system": "arch",
|
||||
"from": "cmd/svc", "binary": "svc"}]}
|
||||
}`
|
||||
m, err := decodeForTest([]byte(raw))
|
||||
m, err := ParseManifest([]byte(raw))
|
||||
if err != nil {
|
||||
t.Fatalf("the service's manifest is refused: %v", err)
|
||||
}
|
||||
@@ -41,6 +43,17 @@ func aServiceModule(t *testing.T) Manifest {
|
||||
return resolved
|
||||
}
|
||||
|
||||
func composeTheService(t *testing.T, with Rendering) []map[string]any {
|
||||
t.Helper()
|
||||
with.Needed = map[string]map[string]string{"svc": {"store": "sealed-store"}}
|
||||
with.ArtifactStore = "anchor.internal:5100"
|
||||
out, err := Resolution{Node: "anchor", Modules: []Manifest{aServiceModule(t)}}.Declaration(with)
|
||||
if err != nil {
|
||||
t.Fatalf("the service does not compose: %v", err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func indexOf(out []map[string]any, id string) int {
|
||||
for i, r := range out {
|
||||
if r["id"] == id {
|
||||
@@ -75,3 +88,79 @@ func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
|
||||
}
|
||||
}
|
||||
|
||||
// The account is created before anything is given to it. Its secrets are mesh-computed and so
|
||||
// placed before the module's own resources; given to a user the machine did not have yet, they were
|
||||
// refused on the first apply and the process started without them.
|
||||
func TestAModulesAccountComesBeforeWhatBelongsToIt(t *testing.T) {
|
||||
out := composeTheService(t, Rendering{})
|
||||
account, secret := indexOf(out, "svc.account"), indexOf(out, "svc."+NeedID("store"))
|
||||
if account < 0 || secret < 0 {
|
||||
t.Fatalf("the account or the secret is missing: %v", out)
|
||||
}
|
||||
if account > secret {
|
||||
t.Fatalf("the secret owned by svc is written before svc exists: account at %d, secret at %d",
|
||||
account, secret)
|
||||
}
|
||||
if owner := out[secret]["owner"]; owner != "svc" {
|
||||
t.Errorf("the secret belongs to %v, not the account its process runs as", owner)
|
||||
}
|
||||
}
|
||||
|
||||
// The process is the module's program; its preparation is the same program asked to prepare, as a
|
||||
// step before it — with the same account and environment, and handing nothing over.
|
||||
func TestAProcessIsPreparedByItsOwnProgram(t *testing.T) {
|
||||
out := composeTheService(t, Rendering{})
|
||||
step, process := indexOf(out, "svc.service-prepare"), indexOf(out, "svc.service")
|
||||
if step < 0 || process < 0 || step > process {
|
||||
t.Fatalf("the preparation is not a step before the process (%d, %d): %v", step, process, out)
|
||||
}
|
||||
s := out[step]
|
||||
if s["type"] != "process" || s["run-once"] != true || s["name"] != "svc-prepare" {
|
||||
t.Errorf("the preparation is not a run-once process: %v", s)
|
||||
}
|
||||
if run, _ := json.Marshal(s["run"]); string(run) != `["./svc","prepare"]` {
|
||||
t.Errorf("the preparation runs %s", run)
|
||||
}
|
||||
if s["user"] != "svc" || s["source"] != out[process]["source"] {
|
||||
t.Errorf("the preparation does not run the same bundle as the same account: %v", s)
|
||||
}
|
||||
if env, _ := s["env"].(map[string]any); env["SVC_STORE_FILE"] == nil {
|
||||
t.Errorf("the preparation is not given the process's environment: %v", s["env"])
|
||||
}
|
||||
if _, has := s["replaces"]; has {
|
||||
t.Errorf("the preparation would hand over what the process replaces: %v", s)
|
||||
}
|
||||
if _, has := s["args"]; has {
|
||||
t.Errorf("the preparation carries a container's args: %v", s)
|
||||
}
|
||||
}
|
||||
|
||||
// What the process replaces is named as the host recorded it, `<module>.<id>`; unprefixed, the host
|
||||
// matches nothing and removes the container first, as before.
|
||||
func TestWhatAProcessReplacesIsNamedAsTheHostRecordedIt(t *testing.T) {
|
||||
out := composeTheService(t, Rendering{})
|
||||
p := out[indexOf(out, "svc.service")]
|
||||
if got, _ := json.Marshal(p["replaces"]); string(got) != `["svc.server"]` {
|
||||
t.Fatalf("the process replaces %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWhatReplacesMayNameIsRefusedNearItsAuthor(t *testing.T) {
|
||||
for what, resource := range map[string]string{
|
||||
"a container": `{"id":"c","type":"container","name":"c","image":"x@` + aServiceDigest + `","replaces":["old"]}`,
|
||||
"a step": `{"id":"p","type":"process","name":"p","run":["./p"],"run-once":true,"replaces":["old"]}`,
|
||||
"something declared": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["p"]}`,
|
||||
"another module's": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["other.old"]}`,
|
||||
"not a list": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":"old"}`,
|
||||
} {
|
||||
raw := `{"module":"m","version":"1","resources":[` + resource + `]}`
|
||||
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "replace") {
|
||||
t.Errorf("replaces on %s was accepted: %v", what, err)
|
||||
}
|
||||
}
|
||||
ok := `{"module":"m","version":"1","resources":[{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["old"]}]}`
|
||||
if _, err := ParseManifest([]byte(ok)); err != nil {
|
||||
t.Errorf("a process replacing what its module no longer declares was refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The fault 04-ISSUES/036 records: a media stack is several modules — a library server, the
|
||||
// acquisition managers, a download client — that must share directories on one machine. Written
|
||||
// as owned `directory` resources, two of them on one node are refused as rivalrous owners, which
|
||||
// is right in general and wrong here: sharing those directories is the whole point of the stack.
|
||||
//
|
||||
// Declared as accesses to an operator-owned path (novox/hq ADR 0051), they co-resolve. An access
|
||||
// is not a resource and never enters the duplicate-owner map, so this is the exact case the
|
||||
// resolver refuses above when the same path is owned — and does not, when it is merely accessed.
|
||||
func TestTwoModulesAccessingOnePathCoResolve(t *testing.T) {
|
||||
a := mod("a", nil, nil, nil)
|
||||
a.Accesses = []Access{{Path: "/services/media/downloads", Mode: AccessReadWrite}}
|
||||
b := mod("b", nil, nil, nil)
|
||||
b.Accesses = []Access{{Path: "/services/media/downloads", Mode: AccessRead}}
|
||||
|
||||
got, err := Resolve(shelf(a, b), []string{"a", "b"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatalf("two modules accessing one operator-owned path were refused: %v", err)
|
||||
}
|
||||
|
||||
// And each accessing module's grant reaches the machine as its own `access` resource, so the
|
||||
// host can find the path present or refuse — both naming the one operator-owned path.
|
||||
var accesses int
|
||||
for _, r := range mustDeclare(t, got) {
|
||||
if r["type"] == "access" && r["path"] == "/services/media/downloads" {
|
||||
accesses++
|
||||
}
|
||||
}
|
||||
if accesses != 2 {
|
||||
t.Errorf("expected both modules' accesses in the declaration, got %d", accesses)
|
||||
}
|
||||
}
|
||||
|
||||
// The mirror of the case above, so the sharing vocabulary does not weaken the rule it sits beside:
|
||||
// two modules OWNING one path is still refused, exactly as before accesses existed.
|
||||
func TestTwoModulesOwningOnePathAreStillRefused(t *testing.T) {
|
||||
a := mod("a", nil, nil, nil)
|
||||
a.Resources = []map[string]any{
|
||||
{"id": "lib", "type": "directory", "path": "/services/media/movies", "mode": "0755"}}
|
||||
b := mod("b", nil, nil, nil)
|
||||
b.Resources = []map[string]any{
|
||||
{"id": "lib", "type": "directory", "path": "/services/media/movies", "mode": "0755"}}
|
||||
|
||||
_, err := Resolve(shelf(a, b), []string{"a", "b"}, workstation(), World{})
|
||||
if err == nil {
|
||||
t.Fatal("two modules owning the same directory were both assigned")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "/services/media/movies") {
|
||||
t.Errorf("the refusal does not name the path: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Shared data is the operator's, owned by no module (novox/hq ADR 0051). A module owning the path
|
||||
// another module was told to expect the operator to provide would create and chown it — and the
|
||||
// two intentions cannot both hold, so it is refused by name. This is what keeps the distinction
|
||||
// machine-checkable rather than a convention nobody enforces.
|
||||
func TestAModuleMayNotOwnWhatAnotherAccesses(t *testing.T) {
|
||||
owns := mod("owns", nil, nil, nil)
|
||||
owns.Resources = []map[string]any{
|
||||
{"id": "lib", "type": "directory", "path": "/services/media/movies", "mode": "0755"}}
|
||||
uses := mod("uses", nil, nil, nil)
|
||||
uses.Accesses = []Access{{Path: "/services/media/movies", Mode: AccessRead}}
|
||||
|
||||
_, err := Resolve(shelf(owns, uses), []string{"owns", "uses"}, workstation(), World{})
|
||||
if err == nil {
|
||||
t.Fatal("a module owning a path another accesses was allowed")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "/services/media/movies") ||
|
||||
!strings.Contains(err.Error(), "the operator's") {
|
||||
t.Errorf("the refusal does not explain the ownership conflict: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A manifest states an access's extent, the way a listening port states its source. An absolute
|
||||
// path and a mode of read or read-write; anything else is refused rather than acted on wrongly.
|
||||
func TestAnAccessIsValidated(t *testing.T) {
|
||||
good := []byte(`{"module":"m","accesses":[{"path":"/services/media","mode":"read-write"}]}`)
|
||||
if _, err := ParseManifest(good); err != nil {
|
||||
t.Fatalf("a valid access was refused: %v", err)
|
||||
}
|
||||
|
||||
// Mode is optional and narrows to read — the safe default, because the danger with an access
|
||||
// is being given more than was meant, not less.
|
||||
bare := []byte(`{"module":"m","accesses":[{"path":"/services/media"}]}`)
|
||||
m, err := ParseManifest(bare)
|
||||
if err != nil {
|
||||
t.Fatalf("an access with no mode was refused: %v", err)
|
||||
}
|
||||
if m.Accesses[0].At() != AccessRead {
|
||||
t.Errorf("an access with no mode is %q, want %q", m.Accesses[0].At(), AccessRead)
|
||||
}
|
||||
|
||||
relative := []byte(`{"module":"m","accesses":[{"path":"services/media","mode":"read"}]}`)
|
||||
if _, err := ParseManifest(relative); err == nil {
|
||||
t.Error("a relative access path was accepted")
|
||||
}
|
||||
|
||||
wrong := []byte(`{"module":"m","accesses":[{"path":"/services/media","mode":"append"}]}`)
|
||||
if _, err := ParseManifest(wrong); err == nil {
|
||||
t.Error("an access with an unknown mode was accepted")
|
||||
}
|
||||
}
|
||||
@@ -2,9 +2,18 @@ package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// What an adopted node is declared in place of a filter (novox/hq ADR 0100).
|
||||
//
|
||||
// On an adopted node the firewall found on the machine stays in force: the mesh loads no table
|
||||
// that drops by default or holds an accept. What the mesh needs reachable is declared as
|
||||
// openings, which the host converges through the found firewall in its own terms; and the mesh
|
||||
// guards its own foundation ports itself, in a table that only refuses.
|
||||
|
||||
// AdoptionPrefix is the id prefix of what the mesh declares of its own on an adopted node. It is
|
||||
// never a module's, so none of it is ever held as found.
|
||||
const AdoptionPrefix = "adoption."
|
||||
@@ -30,9 +39,87 @@ const (
|
||||
GuardUnitPath = "/etc/systemd/system/" + GuardUnit
|
||||
)
|
||||
|
||||
// GuardID, GuardUnitID and GuardRunningID are the guard's resource identities. The installer
|
||||
// raises the same three on an adopted genesis, so the first push finds them already there.
|
||||
func GuardID() string { return AdoptionPrefix + "guard" }
|
||||
func GuardUnitID() string { return AdoptionPrefix + "guard-unit" }
|
||||
func GuardRunningID() string { return AdoptionPrefix + "guard-running" }
|
||||
|
||||
// GuardPackageID is the tool that loads the guard, declared first: a node joining adopted has
|
||||
// no filter module and may have no nft at all, and a table nothing can load guards nothing.
|
||||
func GuardPackageID() string { return AdoptionPrefix + "guard-package" }
|
||||
|
||||
// GuardPackage is the package that carries nft.
|
||||
const GuardPackage = "nftables"
|
||||
|
||||
// OpeningID is an opening's resource identity: its protocol, port and path say what it is.
|
||||
func OpeningID(protocol string, port int, path string) string {
|
||||
return fmt.Sprintf("%sopening-%s-%d-%s", AdoptionPrefix, protocol, port, path)
|
||||
}
|
||||
|
||||
// Openings are what the mesh needs reachable on an adopted node, from the same inputs as the
|
||||
// filter it would load were the node converged, each from where that filter would admit it.
|
||||
//
|
||||
// `rules` is Filtering's answer — every module's listens, the hub's port, the per-node exposure —
|
||||
// and `foundation` is the ports the mesh itself needs, from everywhere. A rule for this machine
|
||||
// only opens nothing. `published` maps a machine port a container publishes to the container's
|
||||
// port: a published port is forwarded, not received, so its opening names the forwarded path and
|
||||
// the port the packet is forwarded to.
|
||||
func Openings(rules []Rule, foundation []int, published map[string]map[int]int) []map[string]any {
|
||||
type key struct {
|
||||
protocol string
|
||||
port int
|
||||
}
|
||||
from := map[key]string{}
|
||||
var order []key
|
||||
widen := func(k key, f string) {
|
||||
was, seen := from[k]
|
||||
if !seen {
|
||||
order = append(order, k)
|
||||
}
|
||||
if !seen || was != OpeningFromEverywhere {
|
||||
from[k] = f
|
||||
}
|
||||
}
|
||||
for _, rule := range rules {
|
||||
switch rule.From {
|
||||
case FromEverywhere:
|
||||
widen(key{rule.Protocol, rule.Port}, OpeningFromEverywhere)
|
||||
case FromMesh:
|
||||
widen(key{rule.Protocol, rule.Port}, OpeningFromMesh)
|
||||
case FromLocalNetwork:
|
||||
// The private network's part only (novox/hq ADR 0340). An adopted node's found firewall
|
||||
// stays, and it is what admits the machine's local network; an opening has no word for a
|
||||
// local network, and widening it to everywhere would open the port to the internet.
|
||||
widen(key{rule.Protocol, rule.Port}, OpeningFromMesh)
|
||||
}
|
||||
}
|
||||
for _, port := range foundation {
|
||||
widen(key{"tcp", port}, OpeningFromEverywhere)
|
||||
}
|
||||
sort.Slice(order, func(a, b int) bool {
|
||||
if order[a].port != order[b].port {
|
||||
return order[a].port < order[b].port
|
||||
}
|
||||
return order[a].protocol < order[b].protocol
|
||||
})
|
||||
out := make([]map[string]any, 0, len(order))
|
||||
for _, k := range order {
|
||||
opening := map[string]any{"type": "opening", "port": k.port, "protocol": k.protocol,
|
||||
"from": from[k]}
|
||||
if to, forwarded := published[k.protocol][k.port]; forwarded {
|
||||
opening["id"] = OpeningID(k.protocol, k.port, PathForwarded)
|
||||
opening["path"] = PathForwarded
|
||||
opening["to"] = to
|
||||
} else {
|
||||
opening["id"] = OpeningID(k.protocol, k.port, PathIncoming)
|
||||
opening["path"] = PathIncoming
|
||||
}
|
||||
out = append(out, opening)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Published is every port the given containers publish on the machine, by protocol and machine
|
||||
// port, mapped to the container's own port. A mapping bound to loopback is left out: nothing off
|
||||
// the machine reaches it, forwarded or not.
|
||||
@@ -66,3 +153,90 @@ func Published(resources []map[string]any) map[string]map[int]int {
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// AsGuard renders the mesh's refusal-only table for the given machine ports.
|
||||
//
|
||||
// It passes everything by default and holds nothing but a refusal, so it cannot close anything
|
||||
// the machine serves; and it is the mesh's own table, so the found firewall reloading does not
|
||||
// touch it. It refuses only packets addressed to this machine, and the ports except from the
|
||||
// machine itself — its loopback and the container runtime's own networks — and from the private
|
||||
// network, known by the interface a packet arrives on and never by its source address. At
|
||||
// prerouting, ahead of the runtime's destination translation, so it matches the port the packet
|
||||
// was sent to; in the inet family, so both address families.
|
||||
//
|
||||
// **The machine's own interfaces are named, where the derived filter names address ranges.** The
|
||||
// filter accepts the container runtime's networks by CIDR; this excludes its bridges by name — lo,
|
||||
// docker0, the br-* a compose network gets, and the mesh's own mesh0. A runtime whose bridge is
|
||||
// named anything else (a podman or libvirt bridge, or a docker network created with a fixed name)
|
||||
// would have its containers' traffic to a guarded port refused, which reads as the port being
|
||||
// down. Names rather than addresses is deliberate: a source address can be claimed by whoever
|
||||
// sends the packet, and this table exists to refuse what the found firewall never sees. Widening
|
||||
// it means adding names here and in the installer's copy together, which the golden test holds to
|
||||
// one text.
|
||||
//
|
||||
// The same text the installer raises on an adopted genesis; a test holds both to it.
|
||||
func AsGuard(ports []int) string {
|
||||
sorted := append([]int{}, ports...)
|
||||
sort.Ints(sorted)
|
||||
listed := make([]string, len(sorted))
|
||||
for i, p := range sorted {
|
||||
listed[i] = strconv.Itoa(p)
|
||||
}
|
||||
var b strings.Builder
|
||||
b.WriteString("table inet mesh_guard {}\n")
|
||||
b.WriteString("delete table inet mesh_guard\n")
|
||||
b.WriteString("table inet mesh_guard {\n")
|
||||
b.WriteString("\tchain prerouting {\n")
|
||||
b.WriteString("\t\ttype filter hook prerouting priority raw; policy accept;\n")
|
||||
// Only packets addressed to this machine: traffic it routes for others — a predecessor's hub,
|
||||
// say — is never the guard's business (novox/hq ADR 0103).
|
||||
fmt.Fprintf(&b, "\t\tfib daddr type local iifname != \"lo\" iifname != \"docker0\" "+
|
||||
"iifname != \"br-*\" iifname != \"mesh0\" tcp dport { %s } drop\n",
|
||||
strings.Join(listed, ", "))
|
||||
b.WriteString("\t}\n")
|
||||
b.WriteString("}\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// GuardUnitText is the unit that loads the guard. Stopping it deletes only its own table: never
|
||||
// a flush, which would take the container runtime's rules and the found firewall with it.
|
||||
func GuardUnitText() string {
|
||||
return "[Unit]\n" +
|
||||
"Description=The mesh's guard: refuses its own ports from outside (novox/hq ADR 0100)\n" +
|
||||
// Early, before the network is up, and without the default dependencies that would
|
||||
// order it after the network; stopped at shutdown like any unit.
|
||||
"DefaultDependencies=no\n" +
|
||||
"Wants=network-pre.target\n" +
|
||||
"Before=network-pre.target shutdown.target\n" +
|
||||
"Conflicts=shutdown.target\n" +
|
||||
"\n" +
|
||||
"[Service]\n" +
|
||||
"Type=oneshot\n" +
|
||||
"RemainAfterExit=yes\n" +
|
||||
"ExecStart=nft -f " + GuardPath + "\n" +
|
||||
"ExecReload=nft -f " + GuardPath + "\n" +
|
||||
"ExecStop=nft delete table inet mesh_guard\n" +
|
||||
"\n" +
|
||||
"[Install]\n" +
|
||||
"WantedBy=multi-user.target\n"
|
||||
}
|
||||
|
||||
// GuardResources are the guard as four resources of the existing kinds: the tool that loads it,
|
||||
// the table, the unit, and the unit running — reloaded when the table changes, so the new table
|
||||
// replaces the old in one `nft -f` through the unit's ExecReload with no moment unguarded, and
|
||||
// restarted only when the unit itself changes. Nothing when there is nothing to guard: an empty
|
||||
// set is not a table nft loads.
|
||||
func GuardResources(ports []int) []map[string]any {
|
||||
if len(ports) == 0 {
|
||||
return nil
|
||||
}
|
||||
return []map[string]any{
|
||||
{"id": GuardPackageID(), "type": "package", "package": GuardPackage},
|
||||
{"id": GuardID(), "type": "file", "path": GuardPath, "content": AsGuard(ports),
|
||||
"mode": "0644"},
|
||||
{"id": GuardUnitID(), "type": "file", "path": GuardUnitPath, "content": GuardUnitText(),
|
||||
"mode": "0644"},
|
||||
{"id": GuardRunningID(), "type": "service", "unit": GuardUnit, "state": "running",
|
||||
"boot": "enabled", "restart-on": []any{GuardUnitID()}, "reload-on": []any{GuardID()}},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -12,6 +15,14 @@ import (
|
||||
// hub is the private network's generator on the hub: it opens the hub's port from anywhere.
|
||||
type hub struct{}
|
||||
|
||||
func (hub) Resources(string) ([]map[string]any, bool, error) {
|
||||
return []map[string]any{{"id": "config", "type": "file", "path": "/etc/wireguard/mesh0.conf",
|
||||
"content": "[Interface]\n"}}, true, nil
|
||||
}
|
||||
func (hub) Listens(string) ([]Listening, error) {
|
||||
return []Listening{{Port: 51820, Protocol: "udp", From: FromEverywhere}}, nil
|
||||
}
|
||||
|
||||
// anAdoptedAnchor is the control-node's set: the store, the bus, the registry, the private network,
|
||||
// a served module and the filter module.
|
||||
func anAdoptedAnchor() Resolution {
|
||||
@@ -40,6 +51,24 @@ func anAdoptedAnchor() Resolution {
|
||||
}}
|
||||
}
|
||||
|
||||
func anchorRendering(adopted bool) Rendering {
|
||||
return Rendering{
|
||||
Generators: map[string]Generator{"overlay": hub{}},
|
||||
Ports: map[string]map[int]int{"distribution": {5000: 5000}, "hello-web": {8080: 20001}},
|
||||
Settings: SettingsBy{"distribution": {{From: "node anchor",
|
||||
Placement: map[string]any{ExposeSetting: map[string]any{"5000": FromEverywhere}}}}},
|
||||
Mesh: []string{"10.42.0.1"},
|
||||
Foundation: []int{5671},
|
||||
// What the machine reported faces outside, which every rule in the filter is written
|
||||
// around (novox/hq ADR 0140).
|
||||
OutwardLinks: []string{"eth0"},
|
||||
TunnelInterface: "mesh0",
|
||||
Adopted: adopted,
|
||||
// Genesis takes the foundation's modules.
|
||||
Taken: map[string]bool{"postgres": true, "lavinmq": true},
|
||||
}
|
||||
}
|
||||
|
||||
func byID(resources []map[string]any) map[string]map[string]any {
|
||||
out := map[string]map[string]any{}
|
||||
for _, r := range resources {
|
||||
@@ -48,6 +77,320 @@ func byID(resources []map[string]any) map[string]map[string]any {
|
||||
return out
|
||||
}
|
||||
|
||||
func TestAnAdoptedNodeIsDeclaredOpeningsFromTheSameInputsAsTheFilter(t *testing.T) {
|
||||
composed, err := anAdoptedAnchor().Compose(anchorRendering(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
want := map[string]map[string]any{
|
||||
// The hub's port, from anywhere, received.
|
||||
"adoption.opening-udp-51820-incoming": {"port": 51820, "protocol": "udp",
|
||||
"from": "everywhere", "path": "incoming"},
|
||||
// The store's port from the private network only, and forwarded: a container publishes it.
|
||||
"adoption.opening-tcp-5432-forwarded": {"port": 5432, "protocol": "tcp", "from": "mesh",
|
||||
"path": "forwarded", "to": 5432},
|
||||
// The bus from anywhere: a node enrols over it before it has a private address.
|
||||
"adoption.opening-tcp-5671-forwarded": {"port": 5671, "protocol": "tcp",
|
||||
"from": "everywhere", "path": "forwarded", "to": 5671},
|
||||
"adoption.opening-tcp-5672-forwarded": {"port": 5672, "protocol": "tcp", "from": "mesh",
|
||||
"path": "forwarded", "to": 5672},
|
||||
// The registry from anywhere, by its node's exposure setting.
|
||||
"adoption.opening-tcp-5000-forwarded": {"port": 5000, "protocol": "tcp",
|
||||
"from": "everywhere", "path": "forwarded", "to": 5000},
|
||||
// A published port names the machine port and the container port it is forwarded to.
|
||||
"adoption.opening-tcp-20001-forwarded": {"port": 20001, "protocol": "tcp",
|
||||
"from": "everywhere", "path": "forwarded", "to": 8080},
|
||||
}
|
||||
for id, fields := range want {
|
||||
opening, ok := got[id]
|
||||
if !ok {
|
||||
t.Errorf("no %s among %v", id, keys(got))
|
||||
continue
|
||||
}
|
||||
if opening["type"] != "opening" {
|
||||
t.Errorf("%s is a %v", id, opening["type"])
|
||||
}
|
||||
for k, v := range fields {
|
||||
if opening[k] != v {
|
||||
t.Errorf("%s: %s is %v, want %v", id, k, opening[k], v)
|
||||
}
|
||||
}
|
||||
}
|
||||
for id := range got {
|
||||
if strings.HasPrefix(id, "adoption.opening-") && want[id] == nil {
|
||||
t.Errorf("an opening nothing asked for: %s", id)
|
||||
}
|
||||
}
|
||||
// A port for this machine only opens nothing, and the management port is not opened at all.
|
||||
for id := range got {
|
||||
if strings.Contains(id, "-9000-") || strings.Contains(id, "-15672-") {
|
||||
t.Errorf("%s is opened", id)
|
||||
}
|
||||
}
|
||||
|
||||
// And openings come first, in the order the machine applies them.
|
||||
if !strings.HasPrefix(composed.Resources[0]["id"].(string), "adoption.opening-") {
|
||||
t.Errorf("openings are not first: %v", composed.Resources[0]["id"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAdoptedNodeLoadsNoFilterOfTheMeshs(t *testing.T) {
|
||||
composed, err := anAdoptedAnchor().Compose(anchorRendering(true))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range composed.Resources {
|
||||
if r["path"] == "/etc/nftables.conf" || strings.HasPrefix(r["id"].(string), "nftables.") {
|
||||
t.Fatalf("an adopted node is declared the filter module's %v", r["id"])
|
||||
}
|
||||
if content, _ := r["content"].(string); strings.Contains(content, "policy drop") {
|
||||
t.Fatalf("an adopted node is declared a table that drops by default: %v", r["id"])
|
||||
}
|
||||
// Nothing but refusals: the only accept in the guard is its policy.
|
||||
if content, _ := r["content"].(string); r["id"] == GuardID() &&
|
||||
strings.Count(content, "accept") != 1 {
|
||||
t.Fatalf("the guard holds an accept:\n%s", content)
|
||||
}
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
guard := got[GuardID()]
|
||||
if guard == nil || got[GuardUnitID()] == nil || got[GuardRunningID()] == nil {
|
||||
t.Fatalf("no guard: %v", keys(got))
|
||||
}
|
||||
if guard["content"] != AsGuard([]int{5432, 5672, 15672}) {
|
||||
t.Fatalf("the guard does not guard the store, the broker and its management port:\n%s",
|
||||
guard["content"])
|
||||
}
|
||||
// The tool that loads it comes first, and the table after it: a node joining adopted has no
|
||||
// filter module and may have no nft.
|
||||
pkg, table := -1, -1
|
||||
for i, r := range composed.Resources {
|
||||
switch r["id"] {
|
||||
case GuardPackageID():
|
||||
pkg = i
|
||||
if r["type"] != "package" || r["package"] != "nftables" {
|
||||
t.Fatalf("the guard's package is %v", r)
|
||||
}
|
||||
case GuardID():
|
||||
table = i
|
||||
}
|
||||
}
|
||||
if pkg < 0 || pkg > table {
|
||||
t.Fatalf("nftables is not declared before the guard's table (%d, %d)", pkg, table)
|
||||
}
|
||||
// A changed table is reloaded — one `nft -f`, atomic — never restarted, which would delete the
|
||||
// table and leave the ports unguarded until it is loaded again. Only a changed unit restarts.
|
||||
if !reflect.DeepEqual(got[GuardRunningID()]["reload-on"], []any{GuardID()}) ||
|
||||
!reflect.DeepEqual(got[GuardRunningID()]["restart-on"], []any{GuardUnitID()}) {
|
||||
t.Fatalf("the guard is not reloaded on its table and restarted on its unit: %v",
|
||||
got[GuardRunningID()])
|
||||
}
|
||||
// Nothing of the mesh's own is anybody's to hold.
|
||||
for id, module := range composed.Owner {
|
||||
if strings.HasPrefix(id, AdoptionPrefix) {
|
||||
t.Fatalf("%s is owned by %s", id, module)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAConvergedNodeIsDeclaredItsFilterAndNoOpenings(t *testing.T) {
|
||||
composed, err := anAdoptedAnchor().Compose(anchorRendering(false))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
if got["nftables.filtering"] == nil || got["nftables.load"] == nil {
|
||||
t.Fatalf("a converged node lost its filter: %v", keys(got))
|
||||
}
|
||||
for id := range got {
|
||||
if strings.HasPrefix(id, AdoptionPrefix) {
|
||||
t.Fatalf("a converged node is declared %s", id)
|
||||
}
|
||||
}
|
||||
plain, err := anAdoptedAnchor().Declaration(anchorRendering(false))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a, _ := json.Marshal(plain)
|
||||
b, _ := json.Marshal(composed.Resources)
|
||||
if string(a) != string(b) {
|
||||
t.Fatal("Compose and Declaration disagree on a converged node")
|
||||
}
|
||||
}
|
||||
|
||||
// The table the installer raises and the controller declares, character for character.
|
||||
func TestTheGuardIsExactlyThisTable(t *testing.T) {
|
||||
const golden = `table inet mesh_guard {}
|
||||
delete table inet mesh_guard
|
||||
table inet mesh_guard {
|
||||
chain prerouting {
|
||||
type filter hook prerouting priority raw; policy accept;
|
||||
fib daddr type local iifname != "lo" iifname != "docker0" iifname != "br-*" iifname != "mesh0" tcp dport { 5432, 15672 } drop
|
||||
}
|
||||
}
|
||||
`
|
||||
if got := AsGuard([]int{15672, 5432}); got != golden {
|
||||
t.Fatalf("the guard changed:\n%s", got)
|
||||
}
|
||||
const unit = `[Unit]
|
||||
Description=The mesh's guard: refuses its own ports from outside (novox/hq ADR 0100)
|
||||
DefaultDependencies=no
|
||||
Wants=network-pre.target
|
||||
Before=network-pre.target shutdown.target
|
||||
Conflicts=shutdown.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=yes
|
||||
ExecStart=nft -f /etc/mesh/guard.nft
|
||||
ExecReload=nft -f /etc/mesh/guard.nft
|
||||
ExecStop=nft delete table inet mesh_guard
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
`
|
||||
if got := GuardUnitText(); got != unit {
|
||||
t.Fatalf("the guard's unit changed:\n%s", got)
|
||||
}
|
||||
if GuardResources(nil) != nil {
|
||||
t.Fatal("a guard with nothing to guard is an empty set nft refuses to load")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAGuardedPortMustBeAPort(t *testing.T) {
|
||||
if _, err := ParseManifest([]byte(`{"module":"postgres","guards":[5432]}`)); err != nil {
|
||||
t.Fatalf("guards is refused: %v", err)
|
||||
}
|
||||
if _, err := ParseManifest([]byte(`{"module":"postgres","guards":[0]}`)); err == nil {
|
||||
t.Fatal("guarding port 0 was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func keys[V any](m map[string]V) []string {
|
||||
return sortedKeys(m)
|
||||
}
|
||||
|
||||
// novox/hq ADR 0100: the foundation's ports are the node's. Given 5433 for the store, every place
|
||||
// that uses the port reads it from there: the container, the filter, the openings, the guard.
|
||||
func TestAGivenPortIsUsedEverywhereThePortIs(t *testing.T) {
|
||||
given := map[string]map[int]int{"postgres": {5432: 5433}, "lavinmq": {15672: 15673}}
|
||||
for _, adopted := range []bool{true, false} {
|
||||
with := anchorRendering(adopted)
|
||||
with.Given = given
|
||||
with.Ports["postgres"] = map[int]int{5432: 5433}
|
||||
composed, err := anAdoptedAnchor().Compose(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
if ports := got["postgres.server"]["ports"]; !reflect.DeepEqual(ports, []any{"5433:5432"}) {
|
||||
t.Fatalf("the store's container publishes %v", ports)
|
||||
}
|
||||
if ports := got["lavinmq.server"]["ports"]; !reflect.DeepEqual(ports,
|
||||
[]any{"5671:5671", "5672:5672", "127.0.0.1:15673:15672"}) {
|
||||
t.Fatalf("the broker's container publishes %v", ports)
|
||||
}
|
||||
if !adopted {
|
||||
filter, _ := got["nftables.filtering"]["content"].(string)
|
||||
if !strings.Contains(filter, "tcp dport 5433 accept") || strings.Contains(filter, "5432") {
|
||||
t.Fatalf("the filter does not use the given port:\n%s", filter)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if o := got["adoption.opening-tcp-5433-forwarded"]; o == nil || o["to"] != 5432 {
|
||||
t.Fatalf("no opening for the given port: %v", keys(got))
|
||||
}
|
||||
if guard := got[GuardID()]["content"]; guard != AsGuard([]int{5433, 5672, 15673}) {
|
||||
t.Fatalf("the guard does not guard the given ports:\n%s", guard)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAGivenPortIsTheNodesAndReachesSomething(t *testing.T) {
|
||||
store := anAdoptedAnchor().Modules[1]
|
||||
node := func(v any) []Layer {
|
||||
return []Layer{{From: "anchor", Placement: map[string]any{PortsSetting: v}}}
|
||||
}
|
||||
if got, err := GivenPorts(store, node(map[string]any{"5432": float64(5433)})); err != nil ||
|
||||
got[5432] != 5433 {
|
||||
t.Fatalf("a node's given port was not read: %v %v", got, err)
|
||||
}
|
||||
if _, err := GivenPorts(store, []Layer{{From: MeshWideLayer,
|
||||
Placement: map[string]any{PortsSetting: map[string]any{"5432": float64(5433)}}}}); err == nil {
|
||||
t.Fatal("a port given for the whole mesh was accepted")
|
||||
}
|
||||
if _, err := GivenPorts(store, node(map[string]any{"6000": float64(6001)})); err == nil {
|
||||
t.Fatal("a port the module neither listens on, publishes nor guards was given")
|
||||
}
|
||||
if _, err := GivenPorts(store, node(map[string]any{"5432": float64(70000)})); err == nil {
|
||||
t.Fatal("a machine port that is not a port was given")
|
||||
}
|
||||
if _, err := GivenPorts(store, node(map[string]any{"5432": float64(22)})); err == nil {
|
||||
t.Fatal("ssh's port was given")
|
||||
}
|
||||
broker := anAdoptedAnchor().Modules[2]
|
||||
if _, err := GivenPorts(broker, node(map[string]any{"5671": float64(5700),
|
||||
"5672": float64(5700)})); err == nil {
|
||||
t.Fatal("one machine port was given for two of the module's ports")
|
||||
}
|
||||
if stray := UnusedSettings(store, node(map[string]any{"5432": float64(5433)})); len(stray) != 0 {
|
||||
t.Fatalf("a given port is called stray: %v", stray)
|
||||
}
|
||||
}
|
||||
|
||||
// novox/hq ADR 0103: the guard is derived, and from taken modules only — every machine port a taken
|
||||
// module publishes that the filter admits from the private network only, and the ports its
|
||||
// manifest guards. A module assigned but not taken is not guarded: its port may still be the
|
||||
// predecessor's.
|
||||
func TestTheGuardIsDerivedFromTakenModulesOnly(t *testing.T) {
|
||||
guardOf := func(with Rendering) string {
|
||||
t.Helper()
|
||||
composed, err := anAdoptedAnchor().Compose(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content, _ := byID(composed.Resources)[GuardID()]["content"].(string)
|
||||
return content
|
||||
}
|
||||
|
||||
// The broker taken, the store not: the broker's plain port follows from its listens (from
|
||||
// the mesh, published), its management port from its manifest; the store is not guarded, and
|
||||
// neither is the bus, which the mesh needs from everywhere.
|
||||
with := anchorRendering(true)
|
||||
with.Taken = map[string]bool{"lavinmq": true}
|
||||
if got := guardOf(with); got != AsGuard([]int{5672, 15672}) {
|
||||
t.Fatalf("the guard is not the taken broker's ports:\n%s", got)
|
||||
}
|
||||
|
||||
// A taken module publishing a port admitted from everywhere is not guarded; one admitted from
|
||||
// the mesh is. The registry is exposed everywhere on this node, and hello-web listens from
|
||||
// everywhere.
|
||||
with.Taken = map[string]bool{"distribution": true, "hello-web": true}
|
||||
if got := guardOf(with); got != "" {
|
||||
t.Fatalf("a port admitted from everywhere is guarded:\n%s", got)
|
||||
}
|
||||
with.Settings = nil
|
||||
if got := guardOf(with); got != AsGuard([]int{5000}) {
|
||||
t.Fatalf("the registry, from the mesh only, is not guarded:\n%s", got)
|
||||
}
|
||||
|
||||
// Nothing taken, nothing guarded — and no guard at all rather than an empty set.
|
||||
with = anchorRendering(true)
|
||||
with.Taken = nil
|
||||
if got := guardOf(with); got != "" {
|
||||
t.Fatalf("an untaken store is guarded:\n%s", got)
|
||||
}
|
||||
|
||||
// A given port is followed: where the machine put it is what is refused.
|
||||
with = anchorRendering(true)
|
||||
with.Given = map[string]map[int]int{"lavinmq": {5672: 5682, 15672: 15673}}
|
||||
with.Ports["lavinmq"] = map[int]int{5671: 5671, 5672: 5682}
|
||||
if got := guardOf(with); got != AsGuard([]int{5432, 5682, 15673}) {
|
||||
t.Fatalf("the guard does not follow the given ports:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A mapping bound to loopback is not published to anything off the machine — in either address
|
||||
// family — and an address's own colons never shift the ports.
|
||||
func TestPublishedLeavesOutLoopbackInBothFamilies(t *testing.T) {
|
||||
@@ -59,3 +402,282 @@ func TestPublishedLeavesOutLoopbackInBothFamilies(t *testing.T) {
|
||||
t.Fatalf("published is %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// novox/hq ADR 0038: only a published port is the mesh's to move. A module that binds the machine
|
||||
// itself listens where its software was told to, so giving it a machine port is refused.
|
||||
func TestAGivenPortIsRefusedForAPortNoContainerPublishes(t *testing.T) {
|
||||
onTheMachine := Manifest{Module: "daemon",
|
||||
Listens: []Listening{{Port: 9000, From: FromMesh}}, Guards: []int{9000}}
|
||||
layers := []Layer{{From: "node anchor",
|
||||
Placement: map[string]any{PortsSetting: map[string]any{"9000": float64(9100)}}}}
|
||||
_, err := GivenPorts(onTheMachine, layers)
|
||||
if err == nil || !strings.Contains(err.Error(), "does not publish") {
|
||||
t.Fatalf("a port no container publishes was given: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// novox/hq ADR 0103: a guarded port this node is told to open to everyone is opened, not guarded.
|
||||
// An opening from everywhere beside a guard dropping it is one statement refusing the other.
|
||||
func TestAGuardedPortOpenedToEveryoneIsNotGuarded(t *testing.T) {
|
||||
with := anchorRendering(true)
|
||||
with.Settings["postgres"] = []Layer{{From: "node anchor",
|
||||
Placement: map[string]any{ExposeSetting: map[string]any{"5432": FromEverywhere}}}}
|
||||
composed, err := anAdoptedAnchor().Compose(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
if o := got["adoption.opening-tcp-5432-forwarded"]; o == nil || o["from"] != OpeningFromEverywhere {
|
||||
t.Fatalf("the store's port is not opened to everyone: %v", o)
|
||||
}
|
||||
if guard, _ := got[GuardID()]["content"].(string); guard != AsGuard([]int{5672, 15672}) {
|
||||
t.Fatalf("a port opened to everyone is still guarded:\n%s", guard)
|
||||
}
|
||||
}
|
||||
|
||||
// A module that publishes its ssh port the long way — `2222:22`, because the machine's own daemon
|
||||
// holds 22 — and says it listens on the machine side of that mapping, which is what anything
|
||||
// reaching it dials.
|
||||
func aForge() Manifest {
|
||||
return Manifest{Module: "forge",
|
||||
Provides: []Offer{{Name: "git-over-ssh", Scope: ScopeMesh}},
|
||||
Serves: map[string]map[string]any{"git-over-ssh": {"port": 2222}},
|
||||
Listens: []Listening{{Port: 3000, From: FromMesh}, {Port: 2222, From: FromMesh}},
|
||||
Resources: []map[string]any{{"id": "server", "type": "container", "name": "forge",
|
||||
"ports": []any{"3000", "2222:22"}}}}
|
||||
}
|
||||
|
||||
// portsAsThePlanWould is where this machine puts each of a module's ports, derived the way
|
||||
// cmd/mesh-controller/plan.go derives it: a given port first, looked up by the port the module
|
||||
// says it listens on, and otherwise wherever the manifest's own mapping already put it. Written
|
||||
// here because everything below — the filter, the openings, the guard, what a consumer is told —
|
||||
// reads that map, and a given port that the lookup does not find moves the container's mapping
|
||||
// and nothing else.
|
||||
//
|
||||
// It stands in for the plan only where the plan does not allocate: a port the manifest already
|
||||
// placed, or one a node was given. For a short form with no given port the real plan asks the
|
||||
// inventory for a machine port and may come back with one from the pool, which needs a store and
|
||||
// is what cmd/mesh-controller's own tests exercise. So an assertion here about such a port asserts
|
||||
// this helper, not the mesh; keep the assertions to the ports under test.
|
||||
func portsAsThePlanWould(m Manifest, given map[int]int) map[int]int {
|
||||
out := map[int]int{}
|
||||
for _, l := range m.Listens {
|
||||
if at, is := given[l.Port]; is {
|
||||
out[l.Port] = at
|
||||
continue
|
||||
}
|
||||
at, _ := m.MachineSide(l.Port)
|
||||
out[l.Port] = at
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// novox/hq ADR 0100 and 0038: the machine side of a long-form mapping is a port the module
|
||||
// publishes, so a node may move it — and a module may name a mapping by either end.
|
||||
func TestAGivenPortNamesEitherEndOfWhatTheModulePublishes(t *testing.T) {
|
||||
forge := aForge()
|
||||
node := func(v any) []Layer {
|
||||
return []Layer{{From: "node anchor", Placement: map[string]any{PortsSetting: v}}}
|
||||
}
|
||||
|
||||
// The machine side — the number this module says it listens on, and the one the predecessor
|
||||
// had somewhere else. Answered under 2222, the end the module itself names, which is what
|
||||
// every reader of this map asks for. One entry, not two: a second key for the same answer is
|
||||
// an entry another mapping's reader could find instead.
|
||||
given, err := GivenPorts(forge, node(map[string]any{"2222": float64(222)}))
|
||||
if err != nil {
|
||||
t.Fatalf("the machine side of a mapping cannot be given a port: %v", err)
|
||||
}
|
||||
if want := map[int]int{2222: 222}; !reflect.DeepEqual(given, want) {
|
||||
t.Fatalf("the forge was given %v; the mapping it names is filed under %v", given, want)
|
||||
}
|
||||
|
||||
// The container's own port names the same mapping and means the same thing — and is filed
|
||||
// under the same name, because the module's name for it has not changed.
|
||||
if inside, err := GivenPorts(forge, node(map[string]any{"22": float64(222)})); err != nil ||
|
||||
!reflect.DeepEqual(inside, map[int]int{2222: 222}) {
|
||||
t.Fatalf("the container's end of the mapping was given %v: %v", inside, err)
|
||||
}
|
||||
|
||||
// A short form is published on the number it names, and is unchanged by any of this.
|
||||
if short, err := GivenPorts(forge, node(map[string]any{"3000": float64(2999)})); err != nil ||
|
||||
short[3000] != 2999 {
|
||||
t.Fatalf("the short form was given %v: %v", short, err)
|
||||
}
|
||||
|
||||
// A port no container publishes is still refused, in the same words.
|
||||
if _, err := GivenPorts(forge, node(map[string]any{"9000": float64(9100)})); err == nil ||
|
||||
!strings.Contains(err.Error(), "does not publish") {
|
||||
t.Fatalf("a port the forge does not publish was given: %v", err)
|
||||
}
|
||||
|
||||
// And the two ends of one mapping given two different numbers is one setting contradicting
|
||||
// the other: the machine publishes it once.
|
||||
if _, err := GivenPorts(forge, node(map[string]any{
|
||||
"2222": float64(222), "22": float64(300)})); err == nil ||
|
||||
!strings.Contains(err.Error(), "one mapping") {
|
||||
t.Fatalf("the two ends of one mapping were given different ports: %v", err)
|
||||
}
|
||||
// Said at both ends with the same number, it is still said twice, and refused where every
|
||||
// other repeated machine port is — as the inventory refuses it when the setting is stored,
|
||||
// which is the layer that sees it first.
|
||||
if _, err := GivenPorts(forge, node(map[string]any{
|
||||
"2222": float64(222), "22": float64(222)})); err == nil ||
|
||||
!strings.Contains(err.Error(), "to both its 22 and its 2222") {
|
||||
t.Fatalf("one mapping given one machine port at both ends: %v", err)
|
||||
}
|
||||
// A number that names two different mappings names neither: which one to move is not said.
|
||||
twice := aForge()
|
||||
twice.Resources[0]["ports"] = []any{"22", "2222:22"}
|
||||
if _, err := GivenPorts(twice, node(map[string]any{"22": float64(222)})); err == nil ||
|
||||
!strings.Contains(err.Error(), "twice") {
|
||||
t.Fatalf("a number naming two of the module's mappings was accepted: %v", err)
|
||||
}
|
||||
|
||||
// And the same refusal when the number naming two mappings is not the one the setting used
|
||||
// but the one the answer would be filed under. Here `80` is the module's own name for a
|
||||
// mapping, and two mappings wear it; filing an answer there is one container's port standing
|
||||
// where the other's is read, and both containers then publish it.
|
||||
shared := Manifest{Module: "gallery",
|
||||
Listens: []Listening{{Port: 80, From: FromMesh}},
|
||||
Resources: []map[string]any{
|
||||
{"id": "a", "type": "container", "name": "a", "ports": []any{"4001:80"}},
|
||||
{"id": "b", "type": "container", "name": "b", "ports": []any{"4002:80"}}}}
|
||||
if _, err := GivenPorts(shared, node(map[string]any{"4001": float64(1234)})); err == nil ||
|
||||
!strings.Contains(err.Error(), "twice") {
|
||||
t.Fatalf("two containers were put on one machine port: %v", err)
|
||||
}
|
||||
|
||||
// A module whose mappings chain — one's machine side is another's container port — keeps them
|
||||
// apart, because each is filed under the port the module names it by and neither name is
|
||||
// shared. Given both, each moves on its own and neither overwrites the other.
|
||||
chained := Manifest{Module: "chain",
|
||||
Listens: []Listening{{Port: 80, From: FromMesh}, {Port: 9090, From: FromMesh}},
|
||||
Resources: []map[string]any{{"id": "server", "type": "container", "name": "chain",
|
||||
"ports": []any{"8080:80", "9090:8080"}}}}
|
||||
both, err := GivenPorts(chained, node(map[string]any{"80": float64(1234), "9090": float64(5678)}))
|
||||
if err != nil || !reflect.DeepEqual(both, map[int]int{80: 1234, 9090: 5678}) {
|
||||
t.Fatalf("chained mappings were given %v: %v", both, err)
|
||||
}
|
||||
if moved := givenOuter("8080:80", both); moved != "1234:80" {
|
||||
t.Fatalf("the first mapping moved to %q, and it was given 1234", moved)
|
||||
}
|
||||
if moved := givenOuter("9090:8080", both); moved != "5678:8080" {
|
||||
t.Fatalf("the second mapping moved to %q, and it was given 5678", moved)
|
||||
}
|
||||
}
|
||||
|
||||
// And the number reaches everything derived from it. The fault this is written against moved the
|
||||
// container's mapping alone: the filter opened the port the software had left, the adopted node's
|
||||
// opening named it too, the guard refused it, and a consumer was sent to it.
|
||||
func TestAGivenMachineSideReachesTheFilterTheOpeningAndTheConsumer(t *testing.T) {
|
||||
forge := aForge()
|
||||
given, err := GivenPorts(forge, []Layer{{From: "node anchor",
|
||||
Placement: map[string]any{PortsSetting: map[string]any{"2222": float64(222)}}}})
|
||||
if err != nil {
|
||||
t.Fatalf("the machine side of a mapping cannot be given a port: %v", err)
|
||||
}
|
||||
r := Resolution{Node: "anchor", Modules: []Manifest{forge}}
|
||||
with := Rendering{
|
||||
Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, given)},
|
||||
Given: map[string]map[int]int{"forge": given},
|
||||
Mesh: []string{"10.77.0.1"},
|
||||
Adopted: true,
|
||||
OutwardLinks: []string{"eth0"},
|
||||
TunnelInterface: "mesh0",
|
||||
Taken: map[string]bool{"forge": true},
|
||||
}
|
||||
|
||||
// What the runtime is handed: the machine's own port on the outside, the container's within.
|
||||
composed, err := r.Compose(with)
|
||||
if err != nil {
|
||||
t.Fatalf("the forge does not compose: %v", err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
if ports := got["forge.server"]["ports"]; !reflect.DeepEqual(ports, []any{"3000:3000", "222:22"}) {
|
||||
t.Fatalf("the forge's container publishes %v", ports)
|
||||
}
|
||||
|
||||
// What the filter would open, were the node converged.
|
||||
rules, err := r.Rules(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var opened []int
|
||||
for _, rule := range rules {
|
||||
opened = append(opened, rule.Port)
|
||||
}
|
||||
// Only the moved port is asserted: the forge's other port is a short form the real plan would
|
||||
// allocate rather than read off the manifest, so its number here is portsAsThePlanWould's and
|
||||
// not the mesh's.
|
||||
if !slices.Contains(opened, 222) || slices.Contains(opened, 2222) {
|
||||
t.Fatalf("the filter opens %v, not where the machine puts the forge", opened)
|
||||
}
|
||||
|
||||
// And what it is declared instead, adopted: an opening on the machine's port, naming the
|
||||
// container's port on the forwarded path — a published port is forwarded, never received.
|
||||
opening := got[OpeningID("tcp", 222, PathForwarded)]
|
||||
if opening == nil || opening["to"] != 22 || opening["from"] != OpeningFromMesh {
|
||||
t.Fatalf("no opening for the port this node gave the forge: %v", keys(got))
|
||||
}
|
||||
for id := range got {
|
||||
if strings.HasPrefix(id, "adoption.opening-tcp-2222-") {
|
||||
t.Errorf("an opening for the port the forge was moved off: %s", id)
|
||||
}
|
||||
}
|
||||
|
||||
// The guard refuses it where the machine put it, and nothing where it used to be.
|
||||
guard, _ := got[GuardID()]["content"].(string)
|
||||
if !strings.Contains(guard, "222") || strings.Contains(guard, "2222") {
|
||||
t.Fatalf("the guard does not follow the given port:\n%s", guard)
|
||||
}
|
||||
|
||||
// And a consumer is sent to the same number, which is read from what the module serves.
|
||||
if told := ServedOn(forge, "git-over-ssh", with.Ports["forge"])["port"]; told != 222 {
|
||||
t.Fatalf("a consumer is told the forge answers on %v", told)
|
||||
}
|
||||
}
|
||||
|
||||
// And the mapping itself moves under either name, because Rendering.Given is a map anybody
|
||||
// composing a declaration hands in: keyed by the machine side, which is what a module declaring
|
||||
// 2222 calls its port, only the outside moves and the container's own port stays as written.
|
||||
func TestAMappingIsMovedUnderEitherOfItsNames(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
written string
|
||||
given map[int]int
|
||||
want string
|
||||
}{
|
||||
{"2222:22", map[int]int{2222: 222}, "222:22"},
|
||||
{"2222:22", map[int]int{22: 222}, "222:22"},
|
||||
{"127.0.0.1:15672:15672/tcp", map[int]int{15672: 15673}, "127.0.0.1:15673:15672/tcp"},
|
||||
{"2222:22", map[int]int{3000: 2999}, "2222:22"},
|
||||
{"2222:22", nil, "2222:22"},
|
||||
} {
|
||||
if got := givenOuter(c.written, c.given); got != c.want {
|
||||
t.Errorf("%s given %v is published as %s, want %s", c.written, c.given, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The same on a node that was given nothing, which is where the derivation was never at fault:
|
||||
// a long-form mapping is published where the manifest put it, and an adopted node opens that port
|
||||
// on the forwarded path like any other. What broke it was the number reaching only the mapping.
|
||||
func TestALongFormPortIsOpenedWhereTheManifestPublishesIt(t *testing.T) {
|
||||
forge := aForge()
|
||||
r := Resolution{Node: "anchor", Modules: []Manifest{forge}}
|
||||
composed, err := r.Compose(Rendering{
|
||||
Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, nil)},
|
||||
Mesh: []string{"10.77.0.1"},
|
||||
Adopted: true,
|
||||
OutwardLinks: []string{"eth0"},
|
||||
TunnelInterface: "mesh0",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := byID(composed.Resources)
|
||||
opening := got[OpeningID("tcp", 2222, PathForwarded)]
|
||||
if opening == nil || opening["to"] != 22 {
|
||||
t.Fatalf("no opening for the forge's published ssh port: %v", keys(got))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The account agents run as (novox/hq ADR 0266): a module names it as a machine fact — the agent account
|
||||
// where the node names one, the operator's otherwise — and asks the node-engine to judge it never to become
|
||||
// root only where it is the agents' own.
|
||||
|
||||
func TestTheAgentAccountFactFallsBackToTheOperatorAndIsNeverRootOnlyWhenItsOwn(t *testing.T) {
|
||||
facts := machineFacts(Resolution{Node: "anchor", Account: "ops"}, nil, "")
|
||||
if facts["agent-account"] != "ops" || facts["agent-home"] != "/home/ops" || facts["agent-root"] != "" {
|
||||
t.Errorf("with no agent account named, agents run as the operator: %v", facts)
|
||||
}
|
||||
facts = machineFacts(Resolution{Node: "anchor", Account: "ops", AccountHome: "/srv/ops"}, nil, "")
|
||||
if facts["agent-home"] != "/srv/ops" {
|
||||
t.Errorf("the operator's stated home is the agent's home when they are one account: %v", facts)
|
||||
}
|
||||
facts = machineFacts(Resolution{Node: "anchor", Account: "ops", AgentAccount: "agent"}, nil, "")
|
||||
if facts["agent-account"] != "agent" || facts["agent-home"] != "/home/agent" || facts["agent-root"] != RootNever {
|
||||
t.Errorf("a named agent account is the agents', never root: %v", facts)
|
||||
}
|
||||
if facts["account"] != "ops" {
|
||||
t.Errorf("the operator account is still the operator's: %v", facts)
|
||||
}
|
||||
facts = machineFacts(Resolution{Node: "anchor", AgentAccount: "agent", AgentAccountHome: "/var/lib/agent"}, nil, "")
|
||||
if facts["agent-home"] != "/var/lib/agent" || facts["agent-root"] != RootNever {
|
||||
t.Errorf("an agent account with a stated home on a machine with no operator: %v", facts)
|
||||
}
|
||||
if _, has := machineFacts(Resolution{Node: "anchor"}, nil, "")["agent-account"]; has {
|
||||
t.Error("a machine with no account at all names an agent account")
|
||||
}
|
||||
}
|
||||
|
||||
// The agent's module, in the shape the catalogue's declares it: the account, never root where it is its
|
||||
// own; its directory under that home, owned by it.
|
||||
const agentModule = `{"module": "agent", "version": "1", "resources": [
|
||||
{"id": "account", "type": "user", "name": "${machine:agent-account}", "home": "${machine:agent-home}",
|
||||
"root": "${machine:agent-root}"},
|
||||
{"id": "home", "type": "directory", "path": "${machine:agent-home}/.agent", "mode": "0700",
|
||||
"owner": "${machine:agent-account}"}
|
||||
]}`
|
||||
|
||||
func TestTheAgentAccountIsDeclaredNeverRootOnlyToAnEngineThatJudgesIt(t *testing.T) {
|
||||
m, err := ParseManifest([]byte(agentModule))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
compose := func(r Resolution, with Rendering) (user, home map[string]any) {
|
||||
t.Helper()
|
||||
r.Node, r.Modules = "anchor", []Manifest{m}
|
||||
out, err := r.Declaration(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return fileNamed(out, "agent.account"), fileNamed(out, "agent.home")
|
||||
}
|
||||
|
||||
user, home := compose(Resolution{Account: "ops", AgentAccount: "agent"}, Rendering{JudgesRoot: true})
|
||||
if user["name"] != "agent" || user[RootField] != RootNever || user["home"] != "/home/agent" {
|
||||
t.Errorf("an engine that judges root is sent the agent account never to become root: %v", user)
|
||||
}
|
||||
if home["path"] != "/home/agent/.agent" || home["owner"] != "agent" {
|
||||
t.Errorf("the agent's directory is under its own home, its own: %v", home)
|
||||
}
|
||||
|
||||
user, _ = compose(Resolution{Account: "ops", AgentAccount: "agent", AgentAccountHome: "/srv/agent"}, Rendering{JudgesRoot: true})
|
||||
if user["home"] != "/srv/agent" {
|
||||
t.Errorf("an agent account named with a home of its own is made there: %v", user)
|
||||
}
|
||||
|
||||
user, _ = compose(Resolution{Account: "ops", AgentAccount: "agent"}, Rendering{})
|
||||
if _, sent := user[RootField]; sent || user["name"] != "agent" {
|
||||
t.Errorf("an older engine, which parses strictly, is sent root: %v", user)
|
||||
}
|
||||
|
||||
user, home = compose(Resolution{Account: "ops"}, Rendering{JudgesRoot: true})
|
||||
if _, sent := user[RootField]; sent || user["name"] != "ops" {
|
||||
t.Errorf("where agents run as the operator, root asserts nothing and is not sent: %v", user)
|
||||
}
|
||||
if home["path"] != "/home/ops/.agent" || home["owner"] != "ops" {
|
||||
t.Errorf("with no agent account, the agent's directory is the operator's: %v", home)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheRuntimeIsToldTheAgentAccount(t *testing.T) {
|
||||
with := Rendering{ArtifactStore: "anchor.internal:5101",
|
||||
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}}
|
||||
envOf := func(r Resolution) map[string]string {
|
||||
t.Helper()
|
||||
r.Node, r.Modules = "anchor", []Manifest{aToolsModule(t, "nftables", "tools/index.js"), theRuntime(t)}
|
||||
out, err := r.Declaration(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
process := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())
|
||||
if process == nil {
|
||||
t.Fatal("no runtime process was composed")
|
||||
}
|
||||
return process["env"].(map[string]string)
|
||||
}
|
||||
env := envOf(Resolution{Account: "ops", AgentAccount: "agent"})
|
||||
if env[RuntimeAgentAccount] != "agent" || env[RuntimeAgentHome] != "/home/agent" || env[RuntimeOperatorAccount] != "ops" {
|
||||
t.Errorf("the runtime is not told whom agents run as: %v", env)
|
||||
}
|
||||
env = envOf(Resolution{Account: "ops"})
|
||||
if env[RuntimeAgentAccount] != "ops" || env[RuntimeAgentHome] != "/home/ops" {
|
||||
t.Errorf("with no agent account, agents run as the operator: %v", env)
|
||||
}
|
||||
env = envOf(Resolution{})
|
||||
if _, set := env[RuntimeAgentAccount]; set {
|
||||
t.Errorf("a machine with no account names an agent account: %v", env)
|
||||
}
|
||||
if problems := bundleEnvProblems("x", Artifact{Name: "b", Kind: ArtifactBundle, Loads: []string{"x"},
|
||||
Env: map[string]string{RuntimeAgentAccount: "me"}}); len(problems) == 0 {
|
||||
t.Error("a bundle may tell the runtime whom agents run as")
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,8 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -18,11 +14,26 @@ import (
|
||||
// the old wire protocol asks for the one server being retired, so both directions are refused at the
|
||||
// parser — this is judged from the manifest alone, no store needed.
|
||||
|
||||
func TestAManifestProvidingAmqpIsRefused(t *testing.T) {
|
||||
raw := []byte(`{"module":"old-broker","version":"1","provides":[{"name":"amqp","scope":"mesh"}]}`)
|
||||
_, err := ParseManifest(raw)
|
||||
if err == nil || !strings.Contains(err.Error(), `provides "amqp", which is not a provision`) {
|
||||
t.Fatalf("a module providing amqp was not refused, or not for the reason: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAManifestRequiringAmqpIsRefused(t *testing.T) {
|
||||
raw := []byte(`{"module":"forwarder","version":"1","requires":["amqp"]}`)
|
||||
_, err := ParseManifest(raw)
|
||||
if err == nil || !strings.Contains(err.Error(), `requires "amqp", which is not a provision`) {
|
||||
t.Fatalf("a module requiring amqp was not refused, or not for the reason: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// And the catalogue (internal/beside) names it nowhere — the three modules that did are removed under
|
||||
// design 28 task 5.4, not converted.
|
||||
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
|
||||
modules, err := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
|
||||
modules0 := modules
|
||||
if err != nil || len(modules) == 0 {
|
||||
t.Fatalf("no manifests in the catalogue, so this proved nothing: %v", err)
|
||||
}
|
||||
@@ -36,42 +47,4 @@ func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
|
||||
filepath.Base(filepath.Dir(path)))
|
||||
}
|
||||
}
|
||||
// Nor does this package's code name any module the catalogue holds (ADR 0601 §4): what a module is, its manifest
|
||||
// says. The names are read from the catalogue's module directories, never from a list here; every string literal of
|
||||
// the package's non-test files is read, and a module's name is refused unless it is also a seat's name or one of the
|
||||
// words below, which name something else.
|
||||
t.Run("the package names no module", func(t *testing.T) {
|
||||
modules := map[string]bool{}
|
||||
for _, path := range modules0 {
|
||||
modules[filepath.Base(filepath.Dir(path))] = true
|
||||
}
|
||||
// Words that are a module's name and name something else here: a language, an environment's rendering, a package.
|
||||
elsewhere := map[string]bool{"go": true, "systemd": true, "nftables": true}
|
||||
fset := token.NewFileSet()
|
||||
files, _ := filepath.Glob("*.go")
|
||||
for _, f := range files {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
parsed, err := parser.ParseFile(fset, f, nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ast.Inspect(parsed, func(n ast.Node) bool {
|
||||
lit, ok := n.(*ast.BasicLit)
|
||||
if !ok || lit.Kind != token.STRING {
|
||||
return true
|
||||
}
|
||||
word, err := strconv.Unquote(lit.Value)
|
||||
if err != nil || !modules[word] || elsewhere[word] {
|
||||
return true
|
||||
}
|
||||
if _, seat := SeatNamed(word); seat {
|
||||
return true
|
||||
}
|
||||
t.Errorf("%s names the module %s", fset.Position(lit.Pos()), word)
|
||||
return true
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
package catalogue
|
||||
|
||||
import "testing"
|
||||
|
||||
// A claim written before the rename still holds (novox/hq ADR 0122, ADR 0156): with the store's
|
||||
// aliases loaded, the former name resolves to the seat.
|
||||
func TestTheArtifactStoresFormerNameResolvesToIt(t *testing.T) {
|
||||
was := aliases
|
||||
t.Cleanup(func() { aliases = was })
|
||||
UseAliases(map[string]string{"the-artifact-store": "mesh-artifact-store"})
|
||||
seat, known := SeatNamed("the-artifact-store")
|
||||
if !known || seat.Name != "mesh-artifact-store" || seat.Delivers != "artifact-store" {
|
||||
t.Fatalf("the former name did not resolve: %+v %v", seat, known)
|
||||
}
|
||||
if _, known := SeatNamed("mesh-artifact-store"); !known {
|
||||
t.Fatal("the seat is not in the set under its name")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The artifact store's seat is one per mesh, read from the catalogue beside this checkout.
|
||||
//
|
||||
// **A second store anywhere is refused by name, not discovered as a consumer failure.** The seat
|
||||
// was node-scoped, so a second `distribution` on another machine resolved cleanly there — and a
|
||||
// node-scoped requirement with one candidate installs that candidate on the node, so anything that
|
||||
// required the store's presence beside it would have raised a fresh, empty store on the wrong
|
||||
// machine. Only afterwards did the mesh notice: `artifact-store` offered by two nodes, and every
|
||||
// consumer elsewhere refusing to choose. The claim says it first, where the second store is
|
||||
// assigned.
|
||||
func TestASecondArtifactStoreAnywhereIsRefusedByName(t *testing.T) {
|
||||
store := catalogueManifest(t, "distribution")
|
||||
|
||||
// The first store resolves as it always has.
|
||||
if _, err := Resolve(shelf(store), []string{"distribution"}, workstation(), World{}); err != nil {
|
||||
t.Fatalf("the store alone does not resolve: %v", err)
|
||||
}
|
||||
|
||||
// A second one, on any other machine, is refused — and the refusal names the seat.
|
||||
elsewhere := World{Held: []Held{{Claim: "mesh-artifact-store", Scope: ScopeMesh,
|
||||
Node: "anchor", Module: "distribution"}}}
|
||||
other := workstation()
|
||||
other.Name = "laptop"
|
||||
_, err := Resolve(shelf(store), []string{"distribution"}, other, elsewhere)
|
||||
if err == nil {
|
||||
t.Fatal("a second store was accepted on another machine; it would offer artifact-store a " +
|
||||
"second time and every consumer elsewhere would refuse to choose")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "mesh-artifact-store") || !strings.Contains(err.Error(), "one per mesh") {
|
||||
t.Fatalf("refused without naming the seat: %v", err)
|
||||
}
|
||||
}
|
||||
+141
-2
@@ -1,6 +1,145 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// What the mesh built, named by what it is rather than by where it was pushed.
|
||||
//
|
||||
// **An image is recorded by its digest and its path; the registry's address is a route to it**
|
||||
// (novox/hq 04-ISSUES/102). A build used to be recorded as `<registry>:<port>/<module>/<artifact>@sha256:…`
|
||||
// — the reference the builder pushed to, kept whole — and every declaration carried that literal.
|
||||
// Move the registry's port, or the registry, and every fresh pull of a mesh image fails: a new
|
||||
// node, a recreate after eviction. The digest is the identity; the address is the node's setting
|
||||
// for the module that serves the artifact store, and it is read from there when a reference is
|
||||
// composed, never written into a record.
|
||||
//
|
||||
// So a kept reference has a scheme of the mesh's own, named after the provision that answers it:
|
||||
//
|
||||
// artifact-store://<module>/<artifact>@sha256:<hex> an image
|
||||
// artifact-store://<module>/<artifact>/blobs/sha256:<hex> an archive
|
||||
//
|
||||
// No runtime knows the scheme. That is the point of it being one: a reference that leaks to a
|
||||
// machine uncomposed is refused by the runtime as malformed, in front of whoever sent it, rather
|
||||
// than pulled from a public registry that happens to have a repository by that name — which is
|
||||
// what an address-less `<module>/<artifact>@sha256:…` would be.
|
||||
|
||||
// ArtifactStoreScheme marks a reference to something in the mesh's artifact store, kept without
|
||||
// the store's address. How a reference is recorded, read and routed (the registry seat's form beside this one) is the
|
||||
// registry clients' (mesh-delivery's contracts, `containerimageregistryclient/reference`; ADR 0634 §4).
|
||||
// the store's address.
|
||||
const ArtifactStoreScheme = ArtifactStoreProvision + "://"
|
||||
|
||||
// Recorded is a reference as the mesh records it: the artifact store's address, if the builder wrote
|
||||
// one, taken off.
|
||||
//
|
||||
// For a reference the builder announced — one it pushed to the store — which is the only kind
|
||||
// this is called on. An image the builder named `<host>/<path>@sha256:…` is kept as its path; an
|
||||
// archive it named `http://<host>/v2/<path>/blobs/<digest>` likewise. A reference with no address
|
||||
// in it — an image id from a genesis build that had nowhere to publish, a package version — is
|
||||
// what it was.
|
||||
func Recorded(reference string) string {
|
||||
if strings.HasPrefix(reference, ArtifactStoreScheme) {
|
||||
return reference
|
||||
}
|
||||
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
|
||||
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
return reference
|
||||
}
|
||||
host, path, ok := strings.Cut(reference, "/")
|
||||
if !ok || !isRegistryHost(host) || !strings.Contains(path, "@sha256:") {
|
||||
return reference
|
||||
}
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
|
||||
// isRegistryHost is the runtime's own rule for reading the first component of a reference as a
|
||||
// registry rather than as a namespace: it has a dot or a port in it, or it is localhost.
|
||||
func isRegistryHost(component string) bool {
|
||||
return component == "localhost" || strings.ContainsAny(component, ".:")
|
||||
}
|
||||
|
||||
// InArtifactStore reports whether a reference is a kept one, and what it names there.
|
||||
func InArtifactStore(reference string) (path string, kept bool) {
|
||||
return strings.CutPrefix(reference, ArtifactStoreScheme)
|
||||
}
|
||||
|
||||
// Routed is a kept reference as a machine fetches it, through the artifact store at `address`
|
||||
// (host:port). A reference that is not a kept one is what it was.
|
||||
func Routed(reference, address string) string {
|
||||
path, kept := InArtifactStore(reference)
|
||||
if !kept {
|
||||
return reference
|
||||
}
|
||||
if strings.Contains(path, "/blobs/") {
|
||||
return "http://" + address + "/v2/" + path
|
||||
}
|
||||
return address + "/" + path
|
||||
}
|
||||
|
||||
// Rerouted is a reference the mesh recorded, whichever way it was recorded, as a machine fetches
|
||||
// it now: a kept one composed with the store's address, and one recorded before references were
|
||||
// kept without their address — the builder's own `<host>/<path>@sha256:…` — re-routed to where
|
||||
// the store is now. Only for references that are the mesh's own: everything a build record
|
||||
// holds is, by construction.
|
||||
func Rerouted(reference, address string) string {
|
||||
return Routed(Recorded(reference), address)
|
||||
}
|
||||
|
||||
// artifactsInto composes the artifact store's address into a resource's `image` and `source`.
|
||||
//
|
||||
// **Composed here, at the last moment before a machine, and stored nowhere.** A kept reference is
|
||||
// routed through the store as this network reaches it now. A reference recorded with an address
|
||||
// before references were kept without one is re-routed the same way — but only when the mesh
|
||||
// built it (`with.Built` names every `<module>/<artifact>` it has), because a module may run an
|
||||
// image from a public registry under its own name and that one is exactly where it says.
|
||||
//
|
||||
// A kept reference with no store to route it through is refused: sent as it is, the runtime would
|
||||
// refuse the scheme on the machine, one push away from the reason.
|
||||
//
|
||||
// **What this does not reach: the images genesis pinned.** The installer builds the control plane
|
||||
// and the builder before the mesh exists, pushes them itself and pins their manifests to
|
||||
// `<registry>:<port>/mesh-controller@…` and `<registry>:<port>/mesh-builder@…` — single-segment
|
||||
// repositories with no build record, so `with.Built` does not name them and they are left as
|
||||
// written until each is rebuilt through the mesh, which records it by digest and path. Until then
|
||||
// a registry that moves strands exactly those two on a recreate, and the control plane's is the
|
||||
// one that cannot be repaired through the mesh. Rebuild both through `build` before moving the
|
||||
// store (novox/hq 04-ISSUES/102, finding F4).
|
||||
func artifactsInto(resource map[string]any, module string, with Rendering) error {
|
||||
for _, key := range []string{"image", "source"} {
|
||||
written, ok := resource[key].(string)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if _, kept := InArtifactStore(written); kept {
|
||||
if with.ArtifactStore == "" {
|
||||
return fmt.Errorf(
|
||||
"%s's %v names %s, which is in the mesh's artifact store, and this mesh has no "+
|
||||
"artifact store on its network to fetch it from — nothing assigned offers "+
|
||||
"%s, or the machine offering it is not on the private network",
|
||||
module, resource["id"], written, ArtifactStoreProvision)
|
||||
}
|
||||
resource[key] = Routed(written, with.ArtifactStore)
|
||||
continue
|
||||
}
|
||||
if with.ArtifactStore == "" {
|
||||
continue
|
||||
}
|
||||
recorded := Recorded(written)
|
||||
if recorded == written {
|
||||
continue
|
||||
}
|
||||
path, _ := InArtifactStore(recorded)
|
||||
repository := path
|
||||
if at := strings.IndexAny(path, "@"); at >= 0 {
|
||||
repository = path[:at]
|
||||
} else if blobs := strings.Index(path, "/blobs/"); blobs >= 0 {
|
||||
repository = path[:blobs]
|
||||
}
|
||||
if with.Built[repository] {
|
||||
resource[key] = Routed(recorded, with.ArtifactStore)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -1,6 +1,137 @@
|
||||
package catalogue
|
||||
|
||||
import "strings"
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A build is recorded by what it is; where it is pushed is composed where it is used (novox/hq
|
||||
// 04-ISSUES/102).
|
||||
|
||||
// digest is an artifact digest the tests build references with.
|
||||
var digest = "sha256:" + strings.Repeat("d", 64)
|
||||
|
||||
func TestAReferenceIsRecordedWithoutTheStoresAddress(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"anchor.internal:5100/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
"localhost:5000/gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
"http://anchor.internal:5100/v2/gitea/config/blobs/" + digest: ArtifactStoreScheme + "gitea/config/blobs/" + digest,
|
||||
ArtifactStoreScheme + "gitea/server@" + digest: ArtifactStoreScheme + "gitea/server@" + digest,
|
||||
digest: digest,
|
||||
"@novox/sdk@1.2.3": "@novox/sdk@1.2.3",
|
||||
"gitea/gitea@" + digest: "gitea/gitea@" + digest,
|
||||
"https://registry.example/v2/gitea/config/blobs/" + digest: "https://registry.example/v2/gitea/config/blobs/" + digest,
|
||||
}
|
||||
for announced, want := range cases {
|
||||
if got := Recorded(announced); got != want {
|
||||
t.Errorf("Recorded(%q) = %q, want %q", announced, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestARecordedReferenceIsRoutedThroughTheStoreAsItIsNow(t *testing.T) {
|
||||
if got := Routed(ArtifactStoreScheme+"gitea/server@"+digest, "anchor.internal:5101"); got != "anchor.internal:5101/gitea/server@"+digest {
|
||||
t.Errorf("an image is fetched as %q", got)
|
||||
}
|
||||
if got := Routed(ArtifactStoreScheme+"gitea/config/blobs/"+digest, "anchor.internal:5101"); got != "http://anchor.internal:5101/v2/gitea/config/blobs/"+digest {
|
||||
t.Errorf("an archive is fetched as %q", got)
|
||||
}
|
||||
if got := Routed("gitea/gitea@"+digest, "anchor.internal:5101"); got != "gitea/gitea@"+digest {
|
||||
t.Errorf("a reference that is not the store's was routed: %q", got)
|
||||
}
|
||||
// One recorded before references were kept without their address follows the store too.
|
||||
if got := Rerouted("anchor.internal:5100/gitea/server@"+digest, "anchor.internal:5101"); got != "anchor.internal:5101/gitea/server@"+digest {
|
||||
t.Errorf("a reference recorded with the old address stays there: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// **The address is composed into a declaration, and the record never carries it.**
|
||||
func TestAnImageTheMeshBuiltIsRoutedThroughTheStoreWhenDeclared(t *testing.T) {
|
||||
m := Manifest{Module: "gitea", Version: "1", Resources: []map[string]any{
|
||||
{"id": "server", "type": "container", "name": "mesh-gitea", "artifact": "server"},
|
||||
{"id": "config", "type": "archive", "path": "/etc/gitea", "artifact": "config"},
|
||||
{"id": "cache", "type": "container", "name": "mesh-gitea-cache", "image": "valkey/valkey@" + digest},
|
||||
}, Build: &Build{Artifacts: []Artifact{
|
||||
{Name: "server", Kind: ArtifactImage, From: "Dockerfile"},
|
||||
{Name: "config", Kind: ArtifactArchive, From: "config"},
|
||||
}}}
|
||||
resolved, err := m.Resolve([]Built{
|
||||
{Name: "server", Kind: ArtifactImage, Reference: ArtifactStoreScheme + "gitea/server@" + digest},
|
||||
{Name: "config", Kind: ArtifactArchive, Reference: ArtifactStoreScheme + "gitea/config/blobs/" + digest, Digest: digest},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := Resolution{Node: "anchor", Modules: []Manifest{resolved}}
|
||||
|
||||
out, err := r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := fileNamed(out, "gitea.server")["image"]; got != "anchor.internal:5101/gitea/server@"+digest {
|
||||
t.Errorf("the image the mesh built is fetched as %v", got)
|
||||
}
|
||||
if got := fileNamed(out, "gitea.config")["source"]; got != "http://anchor.internal:5101/v2/gitea/config/blobs/"+digest {
|
||||
t.Errorf("the archive the mesh built is fetched from %v", got)
|
||||
}
|
||||
if got := fileNamed(out, "gitea.cache")["image"]; got != "valkey/valkey@"+digest {
|
||||
t.Errorf("an image from a public registry was routed through the store: %v", got)
|
||||
}
|
||||
// The manifest the mesh holds still says what it is, not where it was fetched from.
|
||||
if got := resolved.Resources[0]["image"]; got != ArtifactStoreScheme+"gitea/server@"+digest {
|
||||
t.Errorf("composing wrote the address into the catalogue's copy: %v", got)
|
||||
}
|
||||
|
||||
// And the store moves: the same record, another address, without a rebuild.
|
||||
out, err = r.Declaration(Rendering{ArtifactStore: "laptop.internal:5000"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := fileNamed(out, "gitea.server")["image"]; got != "laptop.internal:5000/gitea/server@"+digest {
|
||||
t.Errorf("after the store moved, the image is still fetched as %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnImageInTheStoreWithNoStoreToFetchItFromIsRefused(t *testing.T) {
|
||||
m := Manifest{Module: "gitea", Version: "1", Resources: []map[string]any{
|
||||
{"id": "server", "type": "container", "name": "mesh-gitea",
|
||||
"image": ArtifactStoreScheme + "gitea/server@" + digest},
|
||||
}}
|
||||
_, err := Resolution{Node: "anchor", Modules: []Manifest{m}}.Declaration(Rendering{})
|
||||
if err == nil || !strings.Contains(err.Error(), "no artifact store") {
|
||||
t.Fatalf("a reference nothing can fetch was sent to a machine: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// **A reference recorded with an address before this follows the store too** — when the mesh
|
||||
// built it. A module running an image straight from a public registry under its own name is left
|
||||
// exactly where it says: `quay.io/keycloak/keycloak` is not the mesh's, whatever it is called.
|
||||
func TestAReferenceRecordedWithAnAddressFollowsTheStoreWhenTheMeshBuiltIt(t *testing.T) {
|
||||
m := Manifest{Module: "keycloak", Version: "1", Resources: []map[string]any{
|
||||
{"id": "server", "type": "container", "name": "mesh-keycloak",
|
||||
"image": "anchor.internal:5100/keycloak/server@" + digest},
|
||||
{"id": "upstream", "type": "container", "name": "mesh-keycloak-upstream",
|
||||
"image": "quay.io/keycloak/keycloak@" + digest},
|
||||
}}
|
||||
r := Resolution{Node: "anchor", Modules: []Manifest{m}}
|
||||
out, err := r.Declaration(Rendering{
|
||||
ArtifactStore: "anchor.internal:5101",
|
||||
Built: map[string]bool{"keycloak/server": true},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := fileNamed(out, "keycloak.server")["image"]; got != "anchor.internal:5101/keycloak/server@"+digest {
|
||||
t.Errorf("an image the mesh built, recorded with the old address, is fetched as %v", got)
|
||||
}
|
||||
if got := fileNamed(out, "keycloak.upstream")["image"]; got != "quay.io/keycloak/keycloak@"+digest {
|
||||
t.Errorf("a public image was re-routed through the store: %v", got)
|
||||
}
|
||||
// Nothing known to be built: nothing re-routed, nothing refused.
|
||||
out, err = r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := fileNamed(out, "keycloak.server")["image"]; got != "anchor.internal:5100/keycloak/server@"+digest {
|
||||
t.Errorf("with no build record, a reference was rewritten: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,53 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// What the machine's modules back up is placement (design 57 L3, issue 794). The holder of node-backup names it as
|
||||
// `${backup:lines}` (what to run and which paths to keep) and `${backup:data}` (every item with its class and
|
||||
// protection): the text `${contribution:node-backup:backup}` and `${contribution:node-backup:data}` render, from the
|
||||
// same function, derived from each placed module's declared data (novox/hq ADR 0233). A handed holder has the text in
|
||||
// its plan's placement for its client to fill; one not handed has the controller fill it where it stands.
|
||||
|
||||
// BackupNamespace is the placeholder kind of the machine's backup lines and data.
|
||||
const BackupNamespace = "backup"
|
||||
|
||||
// ofBackup is where a holder places them: ${backup:<key>}. Loose inside the braces, so a misspelt key is found and
|
||||
// refused rather than written out as text.
|
||||
var ofBackup = regexp.MustCompile(`\$\{backup:([^}]*)\}`)
|
||||
|
||||
// backupKinds is the contribution kind each key of the backup placement renders.
|
||||
var backupKinds = map[string]string{"lines": BackupKindBackup, "data": BackupKindData}
|
||||
|
||||
// backupPlacer is the machine's modules the backup placement is composed over, as the old contributions are.
|
||||
type backupPlacer struct {
|
||||
modules []Manifest
|
||||
with Rendering
|
||||
facts map[string]string
|
||||
caps map[string]bool
|
||||
unplaced *[]string
|
||||
}
|
||||
|
||||
// placed is the text of one key for a holder, or what is wrong with naming it.
|
||||
func (b backupPlacer) placed(holder Manifest, key string, item bool) (string, string) {
|
||||
kind, known := backupKinds[key]
|
||||
switch {
|
||||
case item:
|
||||
return "", "an item is kept per item; the machine's backup lines are placed only in a file"
|
||||
case !known:
|
||||
return "", fmt.Sprintf("the backup placement has `lines` and `data`, not %q", key)
|
||||
case !holder.ClaimsSeat(BackupSeat):
|
||||
return "", fmt.Sprintf("%s does not claim %s; the machine's backup lines are placed by its holder alone "+
|
||||
"(novox/hq ADR 0212)", holder.Module, BackupSeat)
|
||||
}
|
||||
text, left, err := seatContributions(b.modules, holder, BackupSeat+":"+kind, b.with, b.facts, b.caps)
|
||||
if b.unplaced != nil {
|
||||
*b.unplaced = append(*b.unplaced, left...)
|
||||
}
|
||||
if err != nil {
|
||||
return "", err.Error()
|
||||
}
|
||||
return text, ""
|
||||
}
|
||||
|
||||
@@ -1,10 +1,126 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A machine's backup lines are placement (design 57 L3, issue 794): the holder of node-backup names them as
|
||||
// `${backup:lines}` and `${backup:data}`, which a handed holder has its client fill from the plan and an unhanded
|
||||
// one has the controller fill as it fills `${contribution:node-backup:backup}` and `${...:data}`: the same text.
|
||||
|
||||
const (
|
||||
oldBackupText = "# lines\n${contribution:node-backup:backup}"
|
||||
oldDataText = "# data\n${contribution:node-backup:data}"
|
||||
placedBackupTxt = "# lines\n${backup:lines}"
|
||||
placedDataTxt = "# data\n${backup:data}"
|
||||
backupHolder = `{"module":"restic","version":"1","moved":true,"claims":[{"name":"node-backup","scope":"node"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
|
||||
"resources":[{"id":"declared","type":"file","path":"/etc/backups.conf","template":"files/backups"},
|
||||
{"id":"data-declared","type":"file","path":"/etc/data.conf","template":"files/data"}]}`
|
||||
)
|
||||
|
||||
func dataModule(name string) string {
|
||||
return `{"module":"` + name + `","version":"1","resources":[{"id":"d","type":"directory","mode":"0700"}],
|
||||
"data":{"own":[{"id":"d","path":"${dir:d}","class":"irreplaceable","why":"the ` + name + ` data"}]}}`
|
||||
}
|
||||
|
||||
// backupsOn is a machine of a holder whose files say these texts and the modules that declare data.
|
||||
func backupsOn(t *testing.T, lines, data string, modules ...string) Resolution {
|
||||
t.Helper()
|
||||
all := []Manifest{held(t, backupHolder, builtWith("restic", map[string]string{"backups": lines, "data": data}))}
|
||||
names := []string{"restic"}
|
||||
for _, name := range modules {
|
||||
all = append(all, mustParse(t, dataModule(name)))
|
||||
names = append(names, name)
|
||||
}
|
||||
r, err := Resolve(shelf(all...), names, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Account = "op"
|
||||
return r
|
||||
}
|
||||
|
||||
var backupModules = []string{"alpha", "beta", "gamma"}
|
||||
|
||||
// gammaLeftOut is a machine whose gamma does not place: its lines are kept, best effort.
|
||||
func gammaLeftOut(with Rendering) Rendering {
|
||||
with.NotPlaced = map[string]string{"gamma": "its placement setting does not read"}
|
||||
return with
|
||||
}
|
||||
|
||||
func TestTheBackupPlacementIsTheTextTheOldPlaceholdersRender(t *testing.T) {
|
||||
// The old form is an unmoved holder: its placeholders are the controller's alone.
|
||||
oldHolder := strings.Replace(backupHolder, `"moved":true,`, "", 1)
|
||||
all := []Manifest{held(t, oldHolder, builtWith("restic", map[string]string{"backups": oldBackupText, "data": oldDataText}))}
|
||||
for _, name := range backupModules {
|
||||
all = append(all, mustParse(t, dataModule(name)))
|
||||
}
|
||||
old, err := Resolve(shelf(all...), append([]string{"restic"}, backupModules...), workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
old.Account = "op"
|
||||
wantLines, wantData := fileOf(t, old, gammaLeftOut(Rendering{}), "restic.declared"), fileOf(t, old, gammaLeftOut(Rendering{}), "restic.data-declared")
|
||||
if !strings.Contains(wantLines, "# alpha\npath ") || !strings.Contains(wantLines, "# gamma\n") || !strings.Contains(wantData, "item d") {
|
||||
t.Fatalf("the old form renders\n%s\n%s", wantLines, wantData)
|
||||
}
|
||||
moved := backupsOn(t, placedBackupTxt, placedDataTxt, backupModules...)
|
||||
// Not handed: the controller fills the new placeholders with the old text.
|
||||
if got := fileOf(t, moved, gammaLeftOut(Rendering{}), "restic.declared"); got != wantLines {
|
||||
t.Errorf("not handed, the lines are\n%s\nwant\n%s", got, wantLines)
|
||||
}
|
||||
if got := fileOf(t, moved, gammaLeftOut(Rendering{}), "restic.data-declared"); got != wantData {
|
||||
t.Errorf("not handed, the data is\n%s\nwant\n%s", got, wantData)
|
||||
}
|
||||
// Handed: the plan carries the same text as placement, and the files are the client's.
|
||||
c, _ := composeWith(t, moved, gammaLeftOut(handingTo("store:5000")))
|
||||
p := c.Plans["restic"].Placement
|
||||
if got := "# lines\n" + p["backup:lines"]; got != wantLines {
|
||||
t.Errorf("handed, the plan's lines are\n%s\nwant\n%s", got, wantLines)
|
||||
}
|
||||
if got := "# data\n" + p["backup:data"]; got != wantData {
|
||||
t.Errorf("handed, the plan's data is\n%s\nwant\n%s", got, wantData)
|
||||
}
|
||||
if len(p) != 2 {
|
||||
t.Errorf("the plan's placement is %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAModuleAddedOrTakenAwayChangesTheBackupPlan(t *testing.T) {
|
||||
plan := func(modules ...string) map[string]string {
|
||||
c, _ := composeWith(t, backupsOn(t, placedBackupTxt, placedDataTxt, modules...), handingTo("store:5000"))
|
||||
return c.Plans["restic"].Placement
|
||||
}
|
||||
two, three := plan("alpha", "beta"), plan("alpha", "beta", "gamma")
|
||||
if reflect.DeepEqual(two, three) || strings.Contains(two["backup:lines"], "gamma") || !strings.Contains(three["backup:lines"], "# gamma\n") {
|
||||
t.Errorf("two modules give %v, three give %v", two, three)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnUnknownBackupKeyIsRefused(t *testing.T) {
|
||||
r := backupsOn(t, "${backup:rows}", placedDataTxt, backupModules...)
|
||||
if _, err := r.Compose(handingTo("store:5000")); err == nil || !strings.Contains(err.Error(), "backup:rows") {
|
||||
t.Errorf("handed, an unknown key is %v", err)
|
||||
}
|
||||
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), "backup:rows") {
|
||||
t.Errorf("not handed, an unknown key is %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func fileOf(t *testing.T, r Resolution, with Rendering, id string) string {
|
||||
t.Helper()
|
||||
out, err := r.Declaration(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, res := range out {
|
||||
if res["id"] == id {
|
||||
return res["content"].(string)
|
||||
}
|
||||
}
|
||||
t.Fatalf("%s was not composed", id)
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A backup contribution written by hand still names its module's own directories; one it does not
|
||||
// declare is refused where it is written rather than reaching the holder as the literal text. (The
|
||||
// catalogue check refuses a hand-written backup line at all since ADR 0233; parsing still reads one,
|
||||
// because a module on the shelf was written before.)
|
||||
func TestABackupNamingAnUndeclaredDirectoryIsRefused(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"pg","version":"1",
|
||||
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:nowhere}"}]}`))
|
||||
if err == nil || !strings.Contains(err.Error(), "nowhere") {
|
||||
t.Fatalf("a backup of an undeclared directory was accepted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A module on the shelf from before ADR 0233, which writes its backup lines by hand and declares no
|
||||
// data, is still backed up: its lines are placed as written, each module's under a comment naming it.
|
||||
func TestHandWrittenBackupLinesOfAnOlderModuleArePlaced(t *testing.T) {
|
||||
pg, err := ParseManifest([]byte(`{"module":"pg","version":"1",
|
||||
"resources":[{"id":"dumps","type":"directory","path":"/srv/pg/dumps","mode":"0700"}],
|
||||
"contributions":[{"seat":"node-backup","kind":"backup","content":"run pg-dump-all\npath ${dir:dumps}"}]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
mail, err := ParseManifest([]byte(`{"module":"mail","version":"1",
|
||||
"resources":[{"id":"spool","type":"directory","mode":"0700"}],
|
||||
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:spool}"}]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
placed, _, err := seatContributions([]Manifest{pg, mail}, Manifest{}, BackupSeat+":"+"backup", Rendering{}, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{"# pg\nrun pg-dump-all\npath /srv/pg/dumps\n", "# mail\npath /var/lib/mail/spool\n"} {
|
||||
if !strings.Contains(placed, want) {
|
||||
t.Errorf("placed contributions lack %q:\n%s", want, placed)
|
||||
}
|
||||
}
|
||||
if strings.Contains(placed, "${dir:") {
|
||||
t.Errorf("a directory reached the holder unfilled:\n%s", placed)
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,34 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The module that provides the artifact store may not be delivered through it.
|
||||
//
|
||||
// Building publishes to the store and the builder will not start without one, so a module that
|
||||
// provides the store and also builds something asks the mesh to put an artifact into the thing
|
||||
// that artifact is needed to create. On a mesh new enough to have no registry, that is a build
|
||||
// that never returns (novox/hq 04-ISSUES/029).
|
||||
func TestTheArtifactStoreCannotBeDeliveredThroughItself(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"registry","version":"1",` +
|
||||
`"provides":[{"name":"artifact-store","scope":"mesh"}],` +
|
||||
`"build":{"artifacts":[{"name":"registry","kind":"upstream","from":"registry:2"}]},` +
|
||||
`"resources":[{"id":"store","type":"container","name":"mesh-registry",` +
|
||||
`"artifact":"registry","ports":["5000:5000"]}]}`))
|
||||
if err == nil {
|
||||
t.Fatal("a registry module that builds its own image was accepted; the build has " +
|
||||
"nowhere to publish until the module it belongs to is already running")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "artifact-store") {
|
||||
t.Fatalf("refused without naming the provision the cycle turns on: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Naming the image directly is the way out, and must stay accepted.
|
||||
func TestAnArtifactStoreThatNamesItsImageIsAccepted(t *testing.T) {
|
||||
_, err := decodeForTest([]byte(`{"module":"registry","version":"1",` +
|
||||
_, err := ParseManifest([]byte(`{"module":"registry","version":"1",` +
|
||||
`"provides":[{"name":"artifact-store","scope":"mesh"}],` +
|
||||
`"resources":[{"id":"store","type":"container","name":"mesh-registry",` +
|
||||
`"image":"registry@sha256:` +
|
||||
@@ -19,7 +41,7 @@ func TestAnArtifactStoreThatNamesItsImageIsAccepted(t *testing.T) {
|
||||
|
||||
// And an ordinary module still builds whatever it likes.
|
||||
func TestAModuleThatDoesNotProvideTheStoreStillBuilds(t *testing.T) {
|
||||
_, err := decodeForTest([]byte(`{"module":"forge","version":"1",` +
|
||||
_, err := ParseManifest([]byte(`{"module":"forge","version":"1",` +
|
||||
`"build":{"artifacts":[{"name":"forge","kind":"upstream","from":"gitea/gitea:1.22"}]},` +
|
||||
`"resources":[{"id":"run","type":"container","name":"forge","artifact":"forge"}]}`))
|
||||
if err != nil {
|
||||
|
||||
@@ -2,9 +2,25 @@ package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// A consumer of a provision that keeps its data stays bound where its data is (novox/hq ADR 0232).
|
||||
//
|
||||
// **What a resolution chooses is not where a consumer's data is.** Resolving answers "which provider
|
||||
// would I pick now", from the seats' holders, the pins and what is assigned where, and every one of
|
||||
// those can change under a consumer without anybody meaning to move it. For a resolver that is the
|
||||
// point: any provider answers alike. For a database it is the consumer's whole state: on 2026-10-05
|
||||
// one change to how a seat's holder answers (issue 258) re-bound five database consumers on one
|
||||
// machine to the store on another, each was given a fresh, empty database there, and nothing warned
|
||||
// for twenty hours (issue 273). Nothing was lost only because the old provider kept everything.
|
||||
//
|
||||
// So the mesh records where each such consumer was last sent (the store's `binding` table), and a
|
||||
// resolution that would answer it from anywhere else keeps the recorded provider instead and says so.
|
||||
// **Only a pin moves it**, because a pin is a person: the one act that says "answer this machine's
|
||||
// consumers from there", taken knowing the data must go first.
|
||||
|
||||
// KeptBinding is one consumer this resolution would have moved, and did not.
|
||||
type KeptBinding struct {
|
||||
// Machine is where the consumer runs, and Consumer the module there that is bound.
|
||||
@@ -24,6 +40,99 @@ func (k KeptBinding) String() string {
|
||||
k.Consumer, k.Provision, k.Bound, k.Would, k.Bound, k.Machine, k.Provision, k.Would.Node, k.Would.Module)
|
||||
}
|
||||
|
||||
// keepBound holds every need for a provision that keeps its consumers' data at the provider its
|
||||
// consumer was bound to, where the resolution chose another.
|
||||
//
|
||||
// A need with no record is a binding being made, and is left as resolved: it is recorded when it is
|
||||
// first sent. A pin naming the provider the resolution chose is a person moving it, and is left too.
|
||||
// Otherwise the recorded provider answers, if it still provides the provision — beside the consumer,
|
||||
// or offered from elsewhere — and the move is returned as kept. **One that no longer does is refused,
|
||||
// never answered by the provider chosen**: answering it there is exactly the silent move this exists
|
||||
// to stop, and the consumer's data is still wherever it was.
|
||||
func keepBound(needs []Needed, catalogue map[string]Manifest, node Node, world World,
|
||||
keeps map[string]bool, here func(string) bool) ([]Needed, []KeptBinding, []string) {
|
||||
var kept []KeptBinding
|
||||
var problems []string
|
||||
refused := map[[2]string]bool{}
|
||||
out := make([]Needed, 0, len(needs))
|
||||
for _, n := range needs {
|
||||
if n.ByRecord || !keeps[n.Name] {
|
||||
out = append(out, n)
|
||||
continue
|
||||
}
|
||||
n.KeepsData = true
|
||||
bound, recorded := world.Bound[n.For][n.Name]
|
||||
chose := Chosen{Node: n.From, Module: n.Module}
|
||||
if !recorded || sameProvider(bound, chose) {
|
||||
out = append(out, n)
|
||||
continue
|
||||
}
|
||||
if pin, pinned := world.Pinned[n.Name]; pinned && pin.matches(Provider{Node: chose.Node, Module: chose.Module}) {
|
||||
out = append(out, n)
|
||||
continue
|
||||
}
|
||||
held, ok, why := boundNeed(n, bound, catalogue, node, world, here)
|
||||
if !ok {
|
||||
if key := [2]string{n.For, n.Name}; !refused[key] {
|
||||
refused[key] = true
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s on %s is bound to %s for %q, which keeps its data, and %s — it would move to %s, "+
|
||||
"which holds none of it. Move its data and say so with `pin %s %s %s %s`, or give "+
|
||||
"%s back what it provided",
|
||||
n.For, node.Name, bound, n.Name, why, chose, node.Name, n.Name, chose.Node, chose.Module,
|
||||
bound.Node))
|
||||
}
|
||||
continue
|
||||
}
|
||||
out = append(out, held)
|
||||
kept = append(kept, KeptBinding{Machine: node.Name, Consumer: n.For, Provision: n.Name, Bound: bound, Would: chose})
|
||||
}
|
||||
return out, kept, problems
|
||||
}
|
||||
|
||||
// sameProvider is whether a recorded provider is the one chosen. A record naming no module (none
|
||||
// is written without one, but a person's hand might) matches any module on its node.
|
||||
func sameProvider(bound, chose Chosen) bool {
|
||||
return bound.Node == chose.Node && (bound.Module == "" || bound.Module == chose.Module)
|
||||
}
|
||||
|
||||
// boundNeed is the need answered by the recorded provider, or why it cannot be.
|
||||
func boundNeed(n Needed, bound Chosen, catalogue map[string]Manifest, node Node, world World,
|
||||
here func(string) bool) (Needed, bool, string) {
|
||||
held := Needed{Name: n.Name, For: n.For, Local: n.Local, KeepsData: true}
|
||||
if bound.Node == node.Name {
|
||||
m, known := catalogue[bound.Module]
|
||||
if !known || !here(bound.Module) || !providesAt(m, n.Name, ScopeMesh) {
|
||||
return Needed{}, false, fmt.Sprintf("%s no longer runs on %s", bound.Module, node.Name)
|
||||
}
|
||||
at := node.At
|
||||
if at == "" {
|
||||
at = "127.0.0.1"
|
||||
}
|
||||
held.From, held.At, held.Module = node.Name, at, m.Module
|
||||
held.Serves, held.SharedOwn, held.Identity = servedByOne(m, n.Name), sharedByOne(m, n.Name), m.IdentityBoundOf(n.Name)
|
||||
held.CredentialForm, held.CredentialTool = m.CredentialFormOf(n.Name), m.CredentialToolOf(n.Name)
|
||||
return held, true, ""
|
||||
}
|
||||
matching := bound.among(world.Offered[n.Name])
|
||||
if len(matching) != 1 {
|
||||
return Needed{}, false, fmt.Sprintf("%s no longer provides it", bound)
|
||||
}
|
||||
p := matching[0]
|
||||
if node.At == "" || p.At == "" {
|
||||
return Needed{}, false, fmt.Sprintf("%s and %s are not both on the private network", node.Name, p.Node)
|
||||
}
|
||||
identity := DefaultIdentityBound
|
||||
if pm, known := catalogue[p.Module]; known {
|
||||
held.SharedOwn, _ = pm.SharedCredentialOf(n.Name)
|
||||
held.CredentialForm, held.CredentialTool = pm.CredentialFormOf(n.Name), pm.CredentialToolOf(n.Name)
|
||||
identity = pm.IdentityBoundOf(n.Name)
|
||||
}
|
||||
held.From, held.At, held.Module, held.Serves, held.Identity = p.Node, p.At, p.Module, p.Serves, identity
|
||||
held.LeftOut = p.LeftOut
|
||||
return held, true, ""
|
||||
}
|
||||
|
||||
// Unbound is one consumer that still asks for a provision and whose own resolution binds it to
|
||||
// another provider than the one a pair credential on record was made with (novox/hq issue 274).
|
||||
//
|
||||
@@ -59,3 +168,19 @@ func (u Unbound) String() string {
|
||||
"login and keeps its data until `cleanup delete` (ADR 0230); the credential from %s stays on "+
|
||||
"record while that login does", who, u.Consumer, now, u.Provision, u.Provider, u.Provider, u.Provider)
|
||||
}
|
||||
|
||||
// BindsFrom is the providers this resolution binds a pair credential's need to — the provision, the
|
||||
// module that requires it and the credential's local name — answered by a machine rather than by a
|
||||
// record. None means this machine states no such binding.
|
||||
func (r Resolution) BindsFrom(provision, module, local string) []string {
|
||||
var from []string
|
||||
for _, n := range r.Needs {
|
||||
if n.ByRecord || n.Name != provision || n.For != module || n.Local != local {
|
||||
continue
|
||||
}
|
||||
if !slices.Contains(from, n.From) {
|
||||
from = append(from, n.From)
|
||||
}
|
||||
}
|
||||
return from
|
||||
}
|
||||
|
||||
@@ -1,5 +1,189 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The half of a connection that is not secret, put where the program reading it can find it.
|
||||
//
|
||||
// **The asymmetry this removes was backwards** (novox/hq 04-ISSUES/023). A sealed credential can
|
||||
// be placed inside any configuration file a module writes: the module leaves a hole, the mesh
|
||||
// delivers the value sealed beside it, and the host — the only thing that sees both — fills it in.
|
||||
// The host and the port and the name to present are ordinary facts the mesh holds in the clear,
|
||||
// and they were the ones stuck: readable only inside a JSON binding, which a program reading
|
||||
// `KEY=value` cannot use.
|
||||
//
|
||||
// So the same shape, and simpler. These values are not secret, so **the control plane substitutes
|
||||
// them itself** before the declaration is sent. Nothing new reaches the host, which learns no
|
||||
// formats and gains no fields.
|
||||
//
|
||||
// **It stays name-agnostic** ([ADR 0027]). The mesh does not learn what a `postgres-database` is:
|
||||
// `at`, `as`, `from` and `address` (the providing machine's private address, novox/hq ADR 0194) are
|
||||
// facts about any provision at all, and everything else comes from what the provider said it
|
||||
// serves — whose keys are agreed by the requirement's name, not by this file.
|
||||
|
||||
// bound is where a module says a value from one of its bindings belongs:
|
||||
// ${bound:<provision>:<key>}, and for an optional requirement the value it takes while absent,
|
||||
// ${bound:<provision>:<key>|default:<value>} (novox/hq ADR 0330 §4). A default is never empty: an
|
||||
// absent requirement is never rendered as an empty string.
|
||||
var bound = regexp.MustCompile(`\$\{bound:([a-z0-9][a-z0-9.-]*[a-z0-9]):([a-z0-9][a-z0-9_-]*)(?:\|default:([^}\n]+))?\}`)
|
||||
|
||||
// knownFor is everything a module may name from one of its bindings.
|
||||
//
|
||||
// Three facts the mesh states about any provision, plus whatever the provider said it serves. A
|
||||
// module may not reach a binding it does not have — the same boundary as a secret, for the same
|
||||
// reason.
|
||||
func knownFor(m Manifest, needs []Needed, node string) (map[string]map[string]string, error) {
|
||||
out := map[string]map[string]string{}
|
||||
for _, want := range m.Wants() {
|
||||
for i := range needs {
|
||||
n := needs[i]
|
||||
if n.Name != want || n.For != m.Module {
|
||||
continue
|
||||
}
|
||||
as := ConsumerIdentity(node, IdentitySource(m.Slug, m.Module))
|
||||
values := map[string]string{
|
||||
"at": n.At,
|
||||
"from": n.From,
|
||||
"as": as,
|
||||
}
|
||||
// What the provider derives for this consumer rather than for all of them
|
||||
// (novox/hq ADR 0202). Filled here, the one place a provision and the module
|
||||
// requiring it are both in hand.
|
||||
served, err := ServedTo(n.Serves, as)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s requires %s: %w", m.Module, want, err)
|
||||
}
|
||||
for key, value := range served {
|
||||
// The provider's own vocabulary. Rendered plainly: a port is 5432, not 5432.000000,
|
||||
// which is what a float would write and what a connection string would refuse.
|
||||
values[key] = plainly(value)
|
||||
}
|
||||
out[want] = values
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// withOwnNames adds a module's own composed names to what it may name from one binding:
|
||||
// `${bound:<provision>:name}` and `:internal-name`, and for several contributions to one requirement
|
||||
// `:name-<local>` / `:internal-name-<local>`. Set over anything the provider serves under those keys:
|
||||
// what the module is called is the mesh's statement, not the provider's.
|
||||
func withOwnNames(values map[string]string, own map[string]any) {
|
||||
for _, key := range []string{"name", "internal-name"} {
|
||||
if v, ok := own[key].(string); ok {
|
||||
values[key] = v
|
||||
}
|
||||
}
|
||||
many, _ := own["names"].(map[string]any)
|
||||
for local, raw := range many {
|
||||
names, _ := raw.(map[string]any)
|
||||
for _, key := range []string{"name", "internal-name"} {
|
||||
if v, ok := names[key].(string); ok {
|
||||
values[key+"-"+local] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// plainly renders a served value as a program would expect to read it.
|
||||
func plainly(value any) string {
|
||||
switch v := value.(type) {
|
||||
case string:
|
||||
return v
|
||||
case float64:
|
||||
if v == float64(int64(v)) {
|
||||
return fmt.Sprintf("%d", int64(v))
|
||||
}
|
||||
return strings.TrimRight(strings.TrimRight(fmt.Sprintf("%f", v), "0"), ".")
|
||||
case bool:
|
||||
return fmt.Sprintf("%t", v)
|
||||
case nil:
|
||||
return ""
|
||||
default:
|
||||
return fmt.Sprint(v)
|
||||
}
|
||||
}
|
||||
|
||||
// boundInto replaces a file's ${bound:…} placeholders with what the mesh knows.
|
||||
//
|
||||
// A placeholder naming something the module does not require, or a key the provider does not
|
||||
// serve, is refused. Left as it was, the literal `${bound:x:y}` would be written into a
|
||||
// configuration file and read as a value — a connection to a host called `${bound:x:y}`, failing
|
||||
// somewhere that names neither the module nor the mesh.
|
||||
//
|
||||
// **An optional requirement left absent takes the default its placeholder names** (novox/hq ADR
|
||||
// 0330 §4), and one that names none is refused rather than rendered empty. The module check refuses
|
||||
// that already; this is the same rule where the file is made, for a manifest registered before it.
|
||||
func boundInto(resource map[string]any, known map[string]map[string]string, module string, absent map[string]bool) error {
|
||||
if fmt.Sprint(resource["type"]) != "file" {
|
||||
return nil
|
||||
}
|
||||
content, ok := resource["content"].(string)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
var refused error
|
||||
filled := bound.ReplaceAllStringFunc(content, func(token string) string {
|
||||
if refused != nil {
|
||||
return token
|
||||
}
|
||||
m := bound.FindStringSubmatch(token)
|
||||
provision, key, fallback := m[1], m[2], m[3]
|
||||
values, has := known[provision]
|
||||
if !has {
|
||||
if absent[provision] {
|
||||
if fallback != "" {
|
||||
return fallback
|
||||
}
|
||||
refused = fmt.Errorf(
|
||||
"%s has a file that says ${bound:%s:%s}, and %q is optional and absent here: it names no "+
|
||||
"value for then (${bound:%s:%s|default:<value>}), so it would be written empty "+
|
||||
"(novox/hq ADR 0330 §4)", module, provision, key, provision, provision, key)
|
||||
return token
|
||||
}
|
||||
refused = fmt.Errorf(
|
||||
"%s has a file that says ${bound:%s:%s}, and %s does not require %q. It may name %s",
|
||||
module, provision, key, module, provision, orNothing(namesOfBindings(known)))
|
||||
return token
|
||||
}
|
||||
value, said := values[key]
|
||||
if !said {
|
||||
refused = fmt.Errorf(
|
||||
"%s asks its %s binding for %q, and what answers it says %s",
|
||||
module, provision, key, orNothing(namesOfKeys(values)))
|
||||
return token
|
||||
}
|
||||
return value
|
||||
})
|
||||
if refused != nil {
|
||||
return refused
|
||||
}
|
||||
resource["content"] = filled
|
||||
return nil
|
||||
}
|
||||
|
||||
func namesOfBindings(known map[string]map[string]string) []string {
|
||||
var names []string
|
||||
for name := range known {
|
||||
names = append(names, fmt.Sprintf("%q", name))
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
func namesOfKeys(values map[string]string) []string {
|
||||
var names []string
|
||||
for key := range values {
|
||||
names = append(names, fmt.Sprintf("%q", key))
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
func orNothing(names []string) string {
|
||||
if len(names) == 0 {
|
||||
return "nothing"
|
||||
|
||||
@@ -1,10 +1,217 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A consumer writes its own connection string (novox/hq 04-ISSUES/023).
|
||||
//
|
||||
// Everything needed for one now reaches the file that needs it: the address and port from what the
|
||||
// provider serves, the name to present from what the mesh decided, the database name from what the
|
||||
// module itself contributed, and the password through the sealed hole the host fills.
|
||||
func TestAConsumerCanWriteAConnectionString(t *testing.T) {
|
||||
r := Resolution{
|
||||
Node: "workstation",
|
||||
Modules: []Manifest{{
|
||||
Module: "keycloak",
|
||||
Requires: []string{"postgres-database"},
|
||||
Secrets: map[string]string{"postgres-database": "/var/lib/keycloak/db.secret"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "dbenv", "type": "file", "path": "/var/lib/keycloak/database.env",
|
||||
"mode": "0600",
|
||||
"content": "KC_DB_URL=jdbc:postgresql://${bound:postgres-database:at}:" +
|
||||
"${bound:postgres-database:port}/keycloak\n" +
|
||||
"KC_DB_USERNAME=${bound:postgres-database:as}\n" +
|
||||
"KC_DB_PASSWORD=${secret:postgres-database}\n",
|
||||
}},
|
||||
}},
|
||||
Needs: []Needed{{
|
||||
Name: "postgres-database", From: "anchor", At: "anchor.internal", For: "keycloak",
|
||||
Sealed: "sealed-db", Serves: map[string]any{"port": float64(5432)},
|
||||
}},
|
||||
}
|
||||
out, err := r.Declaration(Rendering{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
file := fileNamed(out, "keycloak.dbenv")
|
||||
if file == nil {
|
||||
t.Fatalf("no file in %v", out)
|
||||
}
|
||||
got := file["content"].(string)
|
||||
|
||||
want := "KC_DB_URL=jdbc:postgresql://anchor.internal:5432/keycloak\n" +
|
||||
"KC_DB_USERNAME=mesh_workstation_keycloak\n" +
|
||||
"KC_DB_PASSWORD=${secret:postgres-database}\n"
|
||||
if got != want {
|
||||
t.Errorf("the consumer cannot connect with what it was given:\n got %q\nwant %q", got, want)
|
||||
}
|
||||
// The password is still a hole. Only the host may fill that one, and only on the machine.
|
||||
if sealed, _ := file["secrets"].(map[string]any); sealed["postgres-database"] != "sealed-db" {
|
||||
t.Errorf("the credential did not travel with the file: %v", file)
|
||||
}
|
||||
}
|
||||
|
||||
// A provider answered on its consumer's own machine is announced at the port the machine PUBLISHES
|
||||
// it on, not the port the module declared (novox/hq 04-ISSUES/038).
|
||||
//
|
||||
// Issue 018 made the same-node provider announced at all; this is the promise after it — that the
|
||||
// port a co-located consumer is told is the one actually listening at the address it is handed. The
|
||||
// mesh assigns a bare `ports` mapping a host port (here the container's 5432 becomes 15432 on this
|
||||
// machine), and it must announce 15432, or the consumer dials workstation.internal:5432 where the
|
||||
// provider is not published. Cross-node consumers already got this (the plan re-derives the served
|
||||
// facts with the provider's assignment); the same-node need was settled before the assignment and
|
||||
// carried the declared port until now.
|
||||
func TestASameNodeProviderIsAnnouncedAtThePortItIsPublishedOn(t *testing.T) {
|
||||
provider := Manifest{
|
||||
Module: "postgres", Version: "1",
|
||||
Provides: FromAnywhere("postgres-database"),
|
||||
Listens: []Listening{{Port: 5432, Protocol: "tcp", From: "mesh"}},
|
||||
Serves: map[string]map[string]any{"postgres-database": {"port": 5432}},
|
||||
Resources: []map[string]any{{
|
||||
"id": "server", "type": "container", "name": "postgres",
|
||||
"ports": []any{"5432"},
|
||||
}},
|
||||
}
|
||||
consumer := Manifest{
|
||||
Module: "keycloak", Version: "1", Requires: []string{"postgres-database"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "dbenv", "type": "file", "path": "/var/lib/keycloak/database.env", "mode": "0600",
|
||||
"content": "KC_DB_URL=jdbc:postgresql://${bound:postgres-database:at}:" +
|
||||
"${bound:postgres-database:port}/keycloak\n",
|
||||
}},
|
||||
}
|
||||
r, err := Resolve(shelf(provider, consumer),
|
||||
[]string{"postgres", "keycloak"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The mesh assigned the container's 5432 to 15432 on this machine.
|
||||
out, err := r.Declaration(Rendering{Ports: map[string]map[int]int{"postgres": {5432: 15432}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Where the provider is actually published: the assigned host port, on the machine at large.
|
||||
server := fileNamed(out, "postgres.server")
|
||||
if server == nil {
|
||||
t.Fatalf("the provider container is not in the declaration: %v", out)
|
||||
}
|
||||
if published := fmt.Sprint(server["ports"]); !strings.Contains(published, "15432:5432") {
|
||||
t.Fatalf("the provider was not published on the assigned port: %v", server["ports"])
|
||||
}
|
||||
|
||||
// What the consumer was told to dial: the SAME port, at the announced address — not the 5432 the
|
||||
// module declared, which is closed on that address.
|
||||
dbenv := fileNamed(out, "keycloak.dbenv")
|
||||
if dbenv == nil {
|
||||
t.Fatalf("the consumer's binding file is not in the declaration: %v", out)
|
||||
}
|
||||
got := dbenv["content"].(string)
|
||||
want := "KC_DB_URL=jdbc:postgresql://workstation.internal:15432/keycloak\n"
|
||||
if got != want {
|
||||
t.Errorf("the consumer dials a port the provider does not listen on:\n got %q\nwant %q",
|
||||
got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A port is 5432, not 5432.000000 — which is what a number decoded from JSON would write, and what
|
||||
// every connection string in the world would refuse.
|
||||
func TestAPortIsWrittenAsAPort(t *testing.T) {
|
||||
if got := plainly(float64(5432)); got != "5432" {
|
||||
t.Errorf("a port was rendered as %q", got)
|
||||
}
|
||||
if got := plainly("postgres"); got != "postgres" {
|
||||
t.Errorf("a string was rendered as %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The name the provider will create is the name the consumer is told to present. One derivation,
|
||||
// so the two agree by construction rather than by two conventions written on the same day.
|
||||
func TestTheProviderAndTheConsumerAgreeOnTheName(t *testing.T) {
|
||||
provider, err := Resolve(shelf(Manifest{
|
||||
Module: "postgres", Version: "1", Provides: FromAnywhere("postgres-database"),
|
||||
Grants: map[string]string{"postgres-database": "/var/lib/postgres/grants"},
|
||||
Receives: map[string]string{"postgres-database": "/var/lib/postgres/grants/mesh.json"},
|
||||
}), []string{"postgres"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := provider.Declaration(Rendering{Grants: []Grant{{
|
||||
Provision: "postgres-database", Consumer: "workstation", From: "keycloak",
|
||||
Values: map[string]any{"name": "keycloak"}, Sealed: "sealed",
|
||||
}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var told string
|
||||
for _, res := range out {
|
||||
if res["path"] == "/var/lib/postgres/grants/mesh.json" {
|
||||
told = res["content"].(string)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(told, `"as": "mesh_workstation_keycloak"`) {
|
||||
t.Errorf("the provider was not told what to call the login: %s", told)
|
||||
}
|
||||
if told == "" {
|
||||
t.Fatal("the provider was told nothing at all")
|
||||
}
|
||||
// Which is exactly what the consumer, on its own machine, is told to present.
|
||||
if ConsumerIdentity("workstation", "keycloak") != "mesh_workstation_keycloak" {
|
||||
t.Errorf("got %q", ConsumerIdentity("workstation", "keycloak"))
|
||||
}
|
||||
}
|
||||
|
||||
// A binding the module does not have is a typo, refused where it was written.
|
||||
func TestAFileCannotNameABindingTheModuleDoesNotHave(t *testing.T) {
|
||||
r := Resolution{
|
||||
Node: "workstation",
|
||||
Modules: []Manifest{{
|
||||
Module: "keycloak", Requires: []string{"postgres-database"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "conf", "type": "file", "path": "/etc/keycloak.conf",
|
||||
"content": "host=${bound:s3-bucket:at}\n",
|
||||
}},
|
||||
}},
|
||||
Needs: []Needed{{Name: "postgres-database", From: "anchor", At: "a", For: "keycloak"}},
|
||||
}
|
||||
_, err := r.Declaration(Rendering{})
|
||||
if err == nil {
|
||||
t.Fatal("a module read a binding it does not have")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "postgres-database") {
|
||||
t.Errorf("the refusal does not say what it could have named: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A key the provider does not serve is refused too. Written through, the literal placeholder
|
||||
// becomes a hostname, and the failure names neither the module nor the mesh.
|
||||
func TestAKeyTheProviderDoesNotServeIsRefused(t *testing.T) {
|
||||
r := Resolution{
|
||||
Node: "workstation",
|
||||
Modules: []Manifest{{
|
||||
Module: "keycloak", Requires: []string{"postgres-database"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "conf", "type": "file", "path": "/etc/keycloak.conf",
|
||||
"content": "sock=${bound:postgres-database:socket}\n",
|
||||
}},
|
||||
}},
|
||||
Needs: []Needed{{
|
||||
Name: "postgres-database", From: "anchor", At: "a", For: "keycloak",
|
||||
Serves: map[string]any{"port": float64(5432)},
|
||||
}},
|
||||
}
|
||||
_, err := r.Declaration(Rendering{})
|
||||
if err == nil {
|
||||
t.Fatal("a module read a key nothing serves, and the placeholder became a value")
|
||||
}
|
||||
if !strings.Contains(err.Error(), `"port"`) {
|
||||
t.Errorf("the refusal does not say what is served: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A name too long to stay distinct is refused before anything is created.
|
||||
func TestAnIdentityTooLongToBeDistinctIsRefused(t *testing.T) {
|
||||
if err := CheckIdentity("anchor", "keycloak"); err != nil {
|
||||
@@ -25,3 +232,34 @@ func TestTwoModulesOnOneNodeAreTwoIdentities(t *testing.T) {
|
||||
t.Fatal("one module on two machines shares an identity")
|
||||
}
|
||||
}
|
||||
|
||||
// A machine's resolver configuration must name its resolver by address — it cannot resolve the name
|
||||
// of the thing it resolves names with (novox/hq ADR 0194). So a binding offers the providing
|
||||
// machine's private address beside its name, and only when the machine has one.
|
||||
func TestABindingOffersTheProvidersAddress(t *testing.T) {
|
||||
consumer := func() Resolution {
|
||||
return Resolution{
|
||||
Node: "workstation",
|
||||
Modules: []Manifest{{
|
||||
Module: "resolv-conf",
|
||||
Requires: []string{"wildcard-resolution"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "resolv", "type": "file", "path": "/etc/resolv.conf", "mode": "0644",
|
||||
"content": "nameserver ${bound:wildcard-resolution:address}\n",
|
||||
}},
|
||||
}},
|
||||
Needs: []Needed{{Name: "wildcard-resolution", From: "anchor", At: "anchor.internal", For: "resolv-conf"}},
|
||||
}
|
||||
}
|
||||
out, err := consumer().Declaration(Rendering{Machines: map[string]string{"anchor.internal": "10.77.0.1"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := fileNamed(out, "resolv-conf.resolv")["content"]; got != "nameserver 10.77.0.1\n" {
|
||||
t.Fatalf("the resolver is not named by its address: %q", got)
|
||||
}
|
||||
// A machine with no address yet: refused, never written with a blank where the address belongs.
|
||||
if _, err := consumer().Declaration(Rendering{}); err == nil {
|
||||
t.Fatal("a file naming an address the mesh does not have was composed")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,6 +14,200 @@ import (
|
||||
// before this machine's own provider — re-bound all five to the holder, each was made a fresh, empty
|
||||
// database there, and nothing said so.
|
||||
|
||||
var incidentConsumers = []string{"board", "listings", "workflows", "agents", "game"}
|
||||
|
||||
func storeMesh() map[string]Manifest {
|
||||
shelf := map[string]Manifest{
|
||||
"store": {Module: "store", Version: "1",
|
||||
Provides: []Offer{{Name: "postgres-database", Scope: ScopeMesh}},
|
||||
Claims: []Claim{{Name: "mesh-store", Scope: ScopeMesh}},
|
||||
Serves: map[string]map[string]any{"postgres-database": {"port": 5432}},
|
||||
Grants: map[string]string{"postgres-database": "/grants"}},
|
||||
"resolver": {Module: "resolver", Version: "1",
|
||||
Provides: []Offer{{Name: "wildcard-resolution", Scope: ScopeMesh}},
|
||||
Claims: []Claim{{Name: "mesh-dns-resolver", Scope: ScopeMesh}}},
|
||||
"network": {Module: "network", Version: "1", Requires: []string{"wildcard-resolution"}},
|
||||
}
|
||||
for _, c := range incidentConsumers {
|
||||
shelf[c] = Manifest{Module: c, Version: "1", Requires: []string{"postgres-database"}}
|
||||
}
|
||||
return shelf
|
||||
}
|
||||
|
||||
// storeWorld is the rest of the mesh as the home server's plan sees it: the anchor runs a store and a
|
||||
// resolver and holds both seats; the home server runs its own of each.
|
||||
func storeWorld() World {
|
||||
w := World{Offered: map[string][]Provider{
|
||||
"postgres-database": {
|
||||
{Node: "anchor", At: "anchor.internal", Module: "store", Serves: map[string]any{"port": 5432}},
|
||||
{Node: "home", At: "home.internal", Module: "store", Serves: map[string]any{"port": 5434}},
|
||||
},
|
||||
"wildcard-resolution": {
|
||||
{Node: "anchor", At: "anchor.internal", Module: "resolver"},
|
||||
{Node: "home", At: "home.internal", Module: "resolver"},
|
||||
},
|
||||
}}
|
||||
w.Held = []Held{
|
||||
{Claim: "mesh-store", Scope: ScopeMesh, Node: "anchor", Module: "store"},
|
||||
{Claim: "mesh-dns-resolver", Scope: ScopeMesh, Node: "anchor", Module: "resolver"},
|
||||
}
|
||||
w.Holdings = w.Held
|
||||
return w
|
||||
}
|
||||
|
||||
var home = Node{Name: "home", At: "home.internal"}
|
||||
|
||||
func homeAssigned() []string {
|
||||
return append([]string{"store", "resolver", "network"}, incidentConsumers...)
|
||||
}
|
||||
|
||||
func boundFrom(t *testing.T, r Resolution, consumer, provision string) Needed {
|
||||
t.Helper()
|
||||
for _, n := range r.Needs {
|
||||
if n.For == consumer && n.Name == provision {
|
||||
return n
|
||||
}
|
||||
}
|
||||
t.Fatalf("no binding of %s for %s: %+v", consumer, provision, r.Needs)
|
||||
return Needed{}
|
||||
}
|
||||
|
||||
func TestTheIncidentAMachinesOwnStoreKeepsItsConsumersWhenTheSeatIsHeldElsewhere(t *testing.T) {
|
||||
got, err := Resolve(storeMesh(), homeAssigned(), home, storeWorld())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range incidentConsumers {
|
||||
if n := boundFrom(t, got, c, "postgres-database"); n.From != "home" || n.Module != "store" {
|
||||
t.Errorf("%s bound to %s/%s; its data is on the store beside it (issue 273)", c, n.From, n.Module)
|
||||
}
|
||||
}
|
||||
// And the resolver, which keeps nothing of anybody's, still answers from the seat's holder (258).
|
||||
if n := boundFrom(t, got, "network", "wildcard-resolution"); n.From != "anchor" {
|
||||
t.Errorf("the resolver bound to %s; the seat is held on anchor (issue 258)", n.From)
|
||||
}
|
||||
if len(got.Kept) != 0 {
|
||||
t.Errorf("nothing was moved, and %d kept: %+v", len(got.Kept), got.Kept)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheIncidentWithEveryConsumerOnRecordStillMovesNothing(t *testing.T) {
|
||||
w := storeWorld()
|
||||
w.Bound = map[string]map[string]Chosen{}
|
||||
for _, c := range incidentConsumers {
|
||||
w.Bound[c] = map[string]Chosen{"postgres-database": {Node: "home", Module: "store"}}
|
||||
}
|
||||
got, err := Resolve(storeMesh(), homeAssigned(), home, w)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range incidentConsumers {
|
||||
if n := boundFrom(t, got, c, "postgres-database"); n.From != "home" {
|
||||
t.Errorf("%s bound to %s", c, n.From)
|
||||
}
|
||||
}
|
||||
if len(got.Kept) != 0 {
|
||||
t.Errorf("kept %+v", got.Kept)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPinElsewhereStillMovesAStoresConsumers(t *testing.T) {
|
||||
w := storeWorld()
|
||||
w.Pinned = map[string]Chosen{"postgres-database": {Node: "anchor", Module: "store"}}
|
||||
w.Bound = map[string]map[string]Chosen{"board": {"postgres-database": {Node: "home", Module: "store"}}}
|
||||
got, err := Resolve(storeMesh(), homeAssigned(), home, w)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n := boundFrom(t, got, "board", "postgres-database"); n.From != "anchor" || n.Module != "store" {
|
||||
t.Errorf("bound to %s/%s; a person pinned it to anchor", n.From, n.Module)
|
||||
}
|
||||
if len(got.Kept) != 0 {
|
||||
t.Errorf("a pin is a person's move, not one to keep: %+v", got.Kept)
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer on a machine with no store of its own, bound to one store, when the seat moves to
|
||||
// another: the holder would answer, and the binding stays.
|
||||
func laptopWorld(holder string) World {
|
||||
w := storeWorld()
|
||||
w.Held = []Held{{Claim: "mesh-store", Scope: ScopeMesh, Node: holder, Module: "store"}}
|
||||
w.Holdings = w.Held
|
||||
return w
|
||||
}
|
||||
|
||||
var laptop = Node{Name: "laptop", At: "laptop.internal"}
|
||||
|
||||
func TestABoundConsumerStaysWhenTheSeatsHolderChanges(t *testing.T) {
|
||||
w := laptopWorld("home")
|
||||
w.Bound = map[string]map[string]Chosen{"board": {"postgres-database": {Node: "anchor", Module: "store"}}}
|
||||
got, err := Resolve(storeMesh(), []string{"board"}, laptop, w)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
n := boundFrom(t, got, "board", "postgres-database")
|
||||
if n.From != "anchor" || n.At != "anchor.internal" || n.Serves["port"] != 5432 {
|
||||
t.Fatalf("bound to %s at %s %v; its data is on anchor", n.From, n.At, n.Serves)
|
||||
}
|
||||
if len(got.Kept) != 1 {
|
||||
t.Fatalf("the move was not said: %+v", got.Kept)
|
||||
}
|
||||
k := got.Kept[0]
|
||||
if k.Bound != (Chosen{"anchor", "store"}) || k.Would != (Chosen{"home", "store"}) || k.Consumer != "board" {
|
||||
t.Fatalf("kept %+v", k)
|
||||
}
|
||||
for _, want := range []string{"would move board's postgres-database from anchor/store to home/store",
|
||||
"its data is on anchor/store", "pin laptop postgres-database home store", "move the data first"} {
|
||||
if !strings.Contains(k.String(), want) {
|
||||
t.Errorf("%q does not say %q", k.String(), want)
|
||||
}
|
||||
}
|
||||
// Without a record it is a binding being made, and the holder answers it as before.
|
||||
w.Bound = nil
|
||||
got, err = Resolve(storeMesh(), []string{"board"}, laptop, w)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n := boundFrom(t, got, "board", "postgres-database"); n.From != "home" {
|
||||
t.Errorf("a new consumer bound to %s; the seat is held on home", n.From)
|
||||
}
|
||||
}
|
||||
|
||||
func TestABoundConsumerWhoseProviderIsGoneIsRefusedNotMoved(t *testing.T) {
|
||||
w := laptopWorld("anchor")
|
||||
w.Offered["postgres-database"] = w.Offered["postgres-database"][:1] // only anchor's store is left
|
||||
w.Bound = map[string]map[string]Chosen{"board": {"postgres-database": {Node: "home", Module: "store"}}}
|
||||
_, err := Resolve(storeMesh(), []string{"board"}, laptop, w)
|
||||
if err == nil {
|
||||
t.Fatal("bound to home's store, which is gone, and answered by anchor's — the silent move")
|
||||
}
|
||||
for _, want := range []string{"board on laptop is bound to home/store", "home/store no longer provides it",
|
||||
"pin laptop postgres-database anchor store"} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("%q does not say %q", err, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAMachinesOwnStoreUnassignedRefusesItsBoundConsumers(t *testing.T) {
|
||||
w := storeWorld()
|
||||
w.Bound = map[string]map[string]Chosen{"board": {"postgres-database": {Node: "home", Module: "store"}}}
|
||||
_, err := Resolve(storeMesh(), []string{"board"}, home, w)
|
||||
if err == nil || !strings.Contains(err.Error(), "store no longer runs on home") {
|
||||
t.Fatalf("got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The first pass asks only what a machine offers, and is never refused by a binding.
|
||||
func TestTheFirstPassKeepsNoBinding(t *testing.T) {
|
||||
w := storeWorld()
|
||||
w.Unchecked = true
|
||||
w.Bound = map[string]map[string]Chosen{"board": {"postgres-database": {Node: "gone", Module: "store"}}}
|
||||
if _, err := Resolve(storeMesh(), []string{"board"}, laptop, w); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnOfferSaysWhetherItKeepsConsumerData(t *testing.T) {
|
||||
var o Offer
|
||||
if err := json.Unmarshal([]byte(`{"name":"wildcard-resolution","scope":"mesh","keeps-consumer-data":false}`), &o); err != nil {
|
||||
@@ -44,3 +238,20 @@ func TestAnOfferSaysWhetherItKeepsConsumerData(t *testing.T) {
|
||||
t.Error("a name any provider says keeps data keeps data")
|
||||
}
|
||||
}
|
||||
|
||||
// An offer saying it keeps nothing is answered by the seat's holder as the resolver is, even where it
|
||||
// grants a credential.
|
||||
func TestAStoreSayingItKeepsNothingFollowsTheSeat(t *testing.T) {
|
||||
shelf := storeMesh()
|
||||
no := false
|
||||
s := shelf["store"]
|
||||
s.Provides = []Offer{{Name: "postgres-database", Scope: ScopeMesh, KeepsConsumerData: &no}}
|
||||
shelf["store"] = s
|
||||
got, err := Resolve(shelf, homeAssigned(), home, storeWorld())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n := boundFrom(t, got, "board", "postgres-database"); n.From != "anchor" {
|
||||
t.Errorf("bound to %s; it keeps nothing, and the seat is held on anchor", n.From)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,9 +1,51 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The mesh's bus is one per mesh, read from the catalogue beside this checkout.
|
||||
//
|
||||
// **This is step 2's claim, and it is checked here rather than in a bed** (novox/hq ADR 0116):
|
||||
// adoption puts the NATS server into the `mesh-broker` seat on a mesh that is already running,
|
||||
// and the property that matters is that a second one anywhere is refused *when it is assigned*,
|
||||
// not discovered later as two servers holding different halves of the mesh's traffic. A second
|
||||
// bus is not a degraded mesh; it is two meshes that both believe they are the one.
|
||||
func TestASecondMeshBusAnywhereIsRefusedByName(t *testing.T) {
|
||||
nats := catalogueManifest(t, "nats")
|
||||
|
||||
if _, err := Resolve(shelf(nats), []string{"nats"}, workstation(), World{}); err != nil {
|
||||
t.Fatalf("the bus alone does not resolve: %v", err)
|
||||
}
|
||||
|
||||
elsewhere := World{Held: []Held{{Claim: "mesh-broker", Scope: ScopeMesh,
|
||||
Node: "anchor", Module: "nats"}}}
|
||||
other := workstation()
|
||||
other.Name = "laptop"
|
||||
_, err := Resolve(shelf(nats), []string{"nats"}, other, elsewhere)
|
||||
if err == nil {
|
||||
t.Fatal("a second bus was accepted on another machine")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "mesh-broker") || !strings.Contains(err.Error(), "one per mesh") {
|
||||
t.Fatalf("refused without naming the seat: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The seat is the server's role, not the product's name (novox/hq ADR 0079). A different
|
||||
// implementation of the bus claims the same seat, and the mesh refuses it for the same reason —
|
||||
// which is the property that lets the bus be replaced at all.
|
||||
func TestTheSeatRefusesADifferentBusToo(t *testing.T) {
|
||||
nats := catalogueManifest(t, "nats")
|
||||
held := World{Held: []Held{{Claim: "mesh-broker", Scope: ScopeMesh,
|
||||
Node: "anchor", Module: "some-other-broker"}}}
|
||||
other := workstation()
|
||||
other.Name = "laptop"
|
||||
if _, err := Resolve(shelf(nats), []string{"nats"}, other, held); err == nil {
|
||||
t.Fatal("the seat admitted a second holder because the module's name differed")
|
||||
}
|
||||
}
|
||||
|
||||
// The old broker is gone from the catalogue (novox/hq ADR 0131, design 28 task 5.4), so it is no
|
||||
// longer a fixture here. That two eligible holders stand beside each other with one on record is
|
||||
// pinned in holdings_test.go against manifests this package owns.
|
||||
@@ -17,11 +59,7 @@ func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
|
||||
for _, pair := range []struct{ seat, delivers string }{
|
||||
{"git", "git"},
|
||||
{"npm-package-registry", "npm-package-registry"},
|
||||
// The store's seats deliver nothing since the provision artifact-store retired (novox/hq ADR 0632 §2): the
|
||||
// store's address is its holder's listen, read through the seat.
|
||||
{"mesh-artifact-store", ""},
|
||||
{"container-image-registry", ""},
|
||||
{"oci-artifact-registry", ""},
|
||||
{"mesh-artifact-store", "artifact-store"},
|
||||
{"mesh-store", "postgres-database"},
|
||||
{"mesh-broker", "mesh-bus"},
|
||||
} {
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -20,10 +22,159 @@ func reachable() Node {
|
||||
return n
|
||||
}
|
||||
|
||||
func onNetwork(nodes ...string) map[string][]Provider {
|
||||
out := make([]Provider, 0, len(nodes))
|
||||
for _, n := range nodes {
|
||||
out = append(out, Provider{Node: n, At: n + ".internal", Module: "postgres"})
|
||||
}
|
||||
return map[string][]Provider{"postgres-database": out}
|
||||
}
|
||||
|
||||
func brokeredShelf() map[string]Manifest {
|
||||
return shelf(
|
||||
Manifest{Module: "postgres", Version: "1", Provides: FromAnywhere("postgres-database")},
|
||||
Manifest{Module: "meshboard", Version: "1", Requires: []string{"postgres-database"}},
|
||||
)
|
||||
}
|
||||
|
||||
func TestADatabaseIsNotInstalledOnEveryMachineThatUsesOne(t *testing.T) {
|
||||
// The fault this whole distinction exists for.
|
||||
got, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: onNetwork("anchor")})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if have := strings.Join(names(got), " "); strings.Contains(have, "postgres") {
|
||||
t.Fatalf("using a database installed one here: %s", have)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWhatAMachineTakesFromElsewhereIsRecorded(t *testing.T) {
|
||||
// It is the only part of a node's set that stops working when a *different* machine goes
|
||||
// away, and it is where a credential will have to be handed back.
|
||||
got, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: onNetwork("anchor")})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Needs) != 1 {
|
||||
t.Fatalf("got %v", got.Needs)
|
||||
}
|
||||
if got.Needs[0].Name != "postgres-database" || got.Needs[0].From != "anchor" {
|
||||
t.Fatalf("got %v", got.Needs[0])
|
||||
}
|
||||
if got.Needs[0].For != "meshboard" {
|
||||
t.Fatalf("it does not say what wanted it: %v", got.Needs[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestNothingInTheMeshProvidingItIsRefusedWithSomewhereToPutIt(t *testing.T) {
|
||||
// Refused rather than installed here. Choosing a machine to put a database on is a decision
|
||||
// with consequences, and nothing resolving a web application should make it silently.
|
||||
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, workstation(), World{})
|
||||
if err == nil {
|
||||
t.Fatal("a database was found in a mesh that has none")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "assign") || !strings.Contains(err.Error(), "postgres") {
|
||||
t.Fatalf("the refusal does not say what to do: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTwoNodesProvidingItIsRefusedRatherThanPicked(t *testing.T) {
|
||||
// Same rule as everywhere else. Picking one would be a guess about which database a person
|
||||
// meant, and the wrong guess is somebody's data in the wrong place.
|
||||
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: onNetwork("anchor", "archive")})
|
||||
if err == nil {
|
||||
t.Fatal("one of two databases was picked silently")
|
||||
}
|
||||
for _, want := range []string{"anchor", "archive", "pin"} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("the refusal does not name %s: %v", want, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSayingWhichOneSettlesIt(t *testing.T) {
|
||||
got, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{
|
||||
Offered: onNetwork("anchor", "archive"),
|
||||
Pinned: map[string]Chosen{"postgres-database": {Node: "archive", Module: "postgres"}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Needs) != 1 || got.Needs[0].From != "archive" {
|
||||
t.Fatalf("the choice was not taken: %v", got.Needs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBeingPointedAtAMachineThatDoesNotProvideItIsRefused(t *testing.T) {
|
||||
// Rather than falling back to one that does. A fallback would quietly move somebody's data to
|
||||
// a machine they did not choose, which is the whole reason the question is asked.
|
||||
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{
|
||||
Offered: onNetwork("anchor", "archive"),
|
||||
Pinned: map[string]Chosen{"postgres-database": {Node: "somewhere-else", Module: "postgres"}},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("a machine was silently given a different database from the one chosen")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "somewhere-else") {
|
||||
t.Fatalf("the refusal does not say what was chosen: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOneProviderDoesNotOverruleAChoice(t *testing.T) {
|
||||
// A single answer is normally taken silently. Not when somebody said they wanted a different
|
||||
// one -- that is the mesh overruling a person, which it does nowhere else.
|
||||
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
|
||||
World{
|
||||
Offered: onNetwork("anchor"),
|
||||
Pinned: map[string]Chosen{"postgres-database": {Node: "archive", Module: "postgres"}},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("the only database was used although another was chosen")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "only anchor/postgres provides it") {
|
||||
t.Fatalf("the refusal does not say what is available: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestACatalogueThatDisagreesAboutScopeIsRefused(t *testing.T) {
|
||||
// If one module says a database is local and another says it is anywhere, the same
|
||||
// requirement means two things depending on which one happens to answer it.
|
||||
_, err := Resolve(shelf(
|
||||
Manifest{Module: "postgres", Version: "1", Provides: FromAnywhere("postgres-database")},
|
||||
Manifest{Module: "sqlite", Version: "1", Provides: Offers("postgres-database")},
|
||||
Manifest{Module: "meshboard", Version: "1", Requires: []string{"postgres-database"}},
|
||||
), []string{"meshboard"}, workstation(), World{})
|
||||
if err == nil {
|
||||
t.Fatal("a catalogue that disagrees about where a database lives was accepted")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "two things") {
|
||||
t.Fatalf("unhelpful refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestALocalRequirementIsStillAnsweredLocally(t *testing.T) {
|
||||
// The change must not have made everything brokered. A shell is still installed here.
|
||||
got, err := Resolve(shelf(
|
||||
Manifest{Module: "zsh", Version: "1", Provides: Offers("shell")},
|
||||
Manifest{Module: "tools", Version: "1", Requires: []string{"shell"}},
|
||||
), []string{"tools"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if have := strings.Join(names(got), " "); !strings.Contains(have, "zsh") {
|
||||
t.Fatalf("a shell was not installed on the machine that needs one: %s", have)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheShortFormStillMeansHere(t *testing.T) {
|
||||
// `"provides": ["shell"]` must keep meaning what it meant, or every existing manifest
|
||||
// silently changes meaning.
|
||||
m, err := decodeForTest([]byte(`{"module":"zsh","version":"1","provides":["shell"]}`))
|
||||
m, err := ParseManifest([]byte(`{"module":"zsh","version":"1","provides":["shell"]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -32,4 +183,150 @@ func TestTheShortFormStillMeansHere(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestASiteScopedProvisionIsRefused(t *testing.T) {
|
||||
// Meaningful for a claim — one DHCP server per segment — and not yet meaningful for a
|
||||
// provision, because nothing knows how to reach "the one at my site".
|
||||
_, err := ParseManifest([]byte(
|
||||
`{"module":"dns","version":"1","provides":[{"name":"resolver","scope":"site"}]}`))
|
||||
if err == nil {
|
||||
t.Fatal("a site-scoped provision was accepted")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "site") {
|
||||
t.Fatalf("unhelpful refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Being told, which is the difference between knowing and being able to.
|
||||
|
||||
func boundShelf() map[string]Manifest {
|
||||
return shelf(
|
||||
Manifest{Module: "postgres", Version: "1", Provides: FromAnywhere("postgres-database"),
|
||||
Serves: map[string]map[string]any{"postgres-database": {"port": 5432, "driver": "postgres"}}},
|
||||
Manifest{Module: "meshboard", Version: "1", Requires: []string{"postgres-database"},
|
||||
Binds: map[string]string{"postgres-database": "/etc/meshboard/database.json"}},
|
||||
)
|
||||
}
|
||||
|
||||
func binding(t *testing.T, out []map[string]any) map[string]any {
|
||||
t.Helper()
|
||||
for _, r := range out {
|
||||
if r["path"] != "/etc/meshboard/database.json" {
|
||||
continue
|
||||
}
|
||||
var parsed map[string]any
|
||||
if err := json.Unmarshal([]byte(r["content"].(string)), &parsed); err != nil {
|
||||
t.Fatalf("what the app is told is not readable: %v", err)
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
t.Fatalf("the app was told nothing: %v", out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestAnAppIsToldWhereItsDatabaseIs(t *testing.T) {
|
||||
// Knowing it needs the anchor's database is useless to the program that needs it unless the
|
||||
// program is told. This is the whole point of the field.
|
||||
got, err := Resolve(boundShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: map[string][]Provider{"postgres-database": {{Node: "anchor", At: "anchor.internal",
|
||||
Serves: map[string]any{"port": 5432, "driver": "postgres"}}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
told := binding(t, mustDeclare(t, got))
|
||||
if told["from"] != "anchor" || told["at"] != "anchor.internal" {
|
||||
t.Fatalf("it was not told where: %v", told)
|
||||
}
|
||||
serves, _ := told["serves"].(map[string]any)
|
||||
if serves["port"] != float64(5432) || serves["driver"] != "postgres" {
|
||||
t.Fatalf("it was not told how: %v", serves)
|
||||
}
|
||||
}
|
||||
|
||||
func TestItSaysWhereTheCredentialIsInstead(t *testing.T) {
|
||||
// A missing field looks like a bug; a stated absence looks like a boundary. Somebody wiring
|
||||
// this up must not spend an afternoon looking for the password field.
|
||||
//
|
||||
// It used to say only that the mesh had no way to issue one, which stopped being true when
|
||||
// 021 was fixed. A stated absence is only useful while it is accurate — once it is not, it
|
||||
// sends the reader somewhere there is nothing to find.
|
||||
got, _ := Resolve(boundShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: map[string][]Provider{"postgres-database": {{Node: "anchor", At: "anchor.internal"}}}})
|
||||
told := binding(t, mustDeclare(t, got))
|
||||
note, _ := told["generated"].(string)
|
||||
if !strings.Contains(note, "not here") || !strings.Contains(note, "secrets") {
|
||||
t.Fatalf("the file does not say where the credential is instead: %v", note)
|
||||
}
|
||||
for key := range told {
|
||||
if strings.Contains(key, "password") || strings.Contains(key, "secret") {
|
||||
t.Fatalf("something that looks like a credential appeared: %q", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTwoMachinesThatCannotReachEachOtherAreRefused(t *testing.T) {
|
||||
// An app on one machine and a database on another that share no private network is a mesh
|
||||
// that reports itself configured and does not work. Said here rather than discovered as a
|
||||
// connection timing out.
|
||||
_, err := Resolve(boundShelf(), []string{"meshboard"}, workstation(), // not on the network
|
||||
World{Offered: map[string][]Provider{"postgres-database": {{Node: "anchor", At: "anchor.internal"}}}})
|
||||
if err == nil {
|
||||
t.Fatal("an app was pointed at a database it has no path to")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "private network") {
|
||||
t.Fatalf("the refusal does not say what is wrong: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), meshNetwork) {
|
||||
t.Fatalf("the refusal does not say what to assign: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheProviderBeingOffTheNetworkIsAlsoRefused(t *testing.T) {
|
||||
// Both directions, because the failure is identical from either end and the remedy differs.
|
||||
_, err := Resolve(boundShelf(), []string{"meshboard"}, reachable(),
|
||||
World{Offered: map[string][]Provider{"postgres-database": {{Node: "anchor"}}}})
|
||||
if err == nil {
|
||||
t.Fatal("an app was pointed at a database that is not on the private network")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "anchor") {
|
||||
t.Fatalf("the refusal does not name the unreachable end: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBindingSomethingAnsweredHereWritesNothing(t *testing.T) {
|
||||
// A file saying "it is on this node" is a fact nobody needs and one more thing to keep true.
|
||||
got, err := Resolve(shelf(
|
||||
Manifest{Module: "zsh", Version: "1", Provides: Offers("shell")},
|
||||
Manifest{Module: "tools", Version: "1", Requires: []string{"shell"},
|
||||
Binds: map[string]string{"shell": "/etc/tools/shell.json"}},
|
||||
), []string{"tools"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range mustDeclare(t, got) {
|
||||
if r["path"] == "/etc/tools/shell.json" {
|
||||
t.Fatalf("a binding was written for something on this machine: %v", r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBindingWhatYouDoNotRequireIsRefused(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"app","version":"1",
|
||||
"binds":{"postgres-database":"/etc/app/db.json"}}`))
|
||||
if err == nil {
|
||||
t.Fatal("a module was told about something it never asked for")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "does not require") {
|
||||
t.Fatalf("unhelpful refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestServingWhatYouDoNotProvideIsRefused(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"app","version":"1",
|
||||
"serves":{"postgres-database":{"port":5432}}}`))
|
||||
if err == nil {
|
||||
t.Fatal("a module served something it does not provide")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "does not provide") {
|
||||
t.Fatalf("unhelpful refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// BucketShare is one entry of `shares` on a module's requirement of `s3-bucket` (novox/hq ADR 0468 §8): who else may
|
||||
// put or get in the consumer's bucket, and with which methods. Not the NFS shares of shares.go (ADR 0263).
|
||||
//
|
||||
// "requires": [{"name": "s3-bucket", "shares": [{"to": "seat:node-files", "methods": ["put"]},
|
||||
// {"to": "print-and-scan", "methods": ["get"]}]}]
|
||||
//
|
||||
// `to` is a module name, or `seat:<seat name>`. `methods` is a non-empty subset of get and put, without a repeat.
|
||||
// Shares are allowed on `s3-bucket` only, and no two entries name the same `to`.
|
||||
type BucketShare struct {
|
||||
To string `json:"to"`
|
||||
Methods []string `json:"methods"`
|
||||
}
|
||||
|
||||
// BucketRequirement is the requirement `shares` may be written on.
|
||||
const BucketRequirement = "s3-bucket"
|
||||
|
||||
// RequireSharesKey is where a stored manifest keeps the shares of its requirements: beside `requires`, which stays a
|
||||
// plain list of names, for the reason OptionalRequiresKey is beside it. An older controller sets the unknown key
|
||||
// aside instead of failing the catalogue.
|
||||
const RequireSharesKey = "requires-shares"
|
||||
|
||||
// SharesValue is the key of the value a consumer gives the provider of s3-bucket in its grant: the list of
|
||||
// {to, methods} sorted by `to`. A consumer's own `contributes` may not name it.
|
||||
const SharesValue = "shares"
|
||||
|
||||
var shareTarget = regexp.MustCompile(`^(seat:)?[a-z0-9][a-z0-9-]*$`)
|
||||
|
||||
// checkBucketShares refuses a malformed `shares` of one requirement, naming the entry and ADR 0468 §8.
|
||||
func checkBucketShares(requirement string, shares []BucketShare) error {
|
||||
if len(shares) == 0 {
|
||||
return nil
|
||||
}
|
||||
if requirement != BucketRequirement {
|
||||
return fmt.Errorf("requires %s: `shares` is allowed on %s only (novox/hq ADR 0468 §8)", requirement, BucketRequirement)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for i, s := range shares {
|
||||
at := fmt.Sprintf("requires %s: shares[%d] (to %q)", requirement, i, s.To)
|
||||
if !shareTarget.MatchString(s.To) {
|
||||
return fmt.Errorf("%s: `to` is a module name or seat:<seat name> (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
if seen[s.To] {
|
||||
return fmt.Errorf("%s: a second entry for the same `to` (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
seen[s.To] = true
|
||||
if len(s.Methods) == 0 {
|
||||
return fmt.Errorf("%s: `methods` is empty; name get, put or both (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
used := map[string]bool{}
|
||||
for _, method := range s.Methods {
|
||||
if method != "get" && method != "put" {
|
||||
return fmt.Errorf("%s: method %q is not get or put (novox/hq ADR 0468 §8)", at, method)
|
||||
}
|
||||
if used[method] {
|
||||
return fmt.Errorf("%s: method %q twice (novox/hq ADR 0468 §8)", at, method)
|
||||
}
|
||||
used[method] = true
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// readRequireShares reads the stored form's side key.
|
||||
func readRequireShares(raw json.RawMessage) (map[string][]BucketShare, error) {
|
||||
var out map[string][]BucketShare
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return nil, fmt.Errorf("%s: an object of requirement to a list of {to, methods}: %w", RequireSharesKey, err)
|
||||
}
|
||||
for requirement, shares := range out {
|
||||
if err := checkBucketShares(requirement, shares); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -1,72 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Issue 1591: `shares` on a requirement of s3-bucket (novox/hq ADR 0468 §8).
|
||||
|
||||
const filesWithShares = `{"module":"files","version":"1","requires":["shell",
|
||||
{"name":"s3-bucket","shares":[{"to":"seat:node-files","methods":["put"]},{"to":"print-and-scan","methods":["get","put"]}]}]}`
|
||||
|
||||
// 1591/AC1: parsed, the shares are on the manifest; stored, `requires` is plain names and the side key holds them.
|
||||
func TestSharesParseAndAreStoredBesideRequires(t *testing.T) {
|
||||
m := mustParse(t, filesWithShares)
|
||||
want := []BucketShare{{To: "seat:node-files", Methods: []string{"put"}}, {To: "print-and-scan", Methods: []string{"get", "put"}}}
|
||||
if !reflect.DeepEqual(m.RequireShares["s3-bucket"], want) {
|
||||
t.Fatalf("the shares read as %+v", m.RequireShares)
|
||||
}
|
||||
raw, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var keys map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &keys); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(keys["requires"]) != `["shell","s3-bucket"]` {
|
||||
t.Errorf("the stored requires is not plain names: %s", keys["requires"])
|
||||
}
|
||||
if !strings.Contains(string(keys[RequireSharesKey]), `"seat:node-files"`) {
|
||||
t.Errorf("the side key does not hold the shares: %s", raw)
|
||||
}
|
||||
var again Manifest
|
||||
if err := json.Unmarshal(raw, &again); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(again, m) {
|
||||
t.Fatalf("not lossless:\n%+v\n%+v", again, m)
|
||||
}
|
||||
}
|
||||
|
||||
// 1591/AC2: a malformed share is refused, naming the entry and ADR 0468 §8.
|
||||
func TestAMalformedShareIsRefused(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"unknown method": `{"name":"s3-bucket","shares":[{"to":"a","methods":["delete"]}]}`,
|
||||
"empty to": `{"name":"s3-bucket","shares":[{"to":"","methods":["get"]}]}`,
|
||||
"malformed to": `{"name":"s3-bucket","shares":[{"to":"Seat:X y","methods":["get"]}]}`,
|
||||
"empty methods": `{"name":"s3-bucket","shares":[{"to":"a","methods":[]}]}`,
|
||||
"duplicate to": `{"name":"s3-bucket","shares":[{"to":"a","methods":["get"]},{"to":"a","methods":["put"]}]}`,
|
||||
"duplicate verb": `{"name":"s3-bucket","shares":[{"to":"a","methods":["get","get"]}]}`,
|
||||
"other require": `{"name":"route","shares":[{"to":"a","methods":["get"]}]}`,
|
||||
}
|
||||
for name, entry := range cases {
|
||||
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":[` + entry + `]}`))
|
||||
if err == nil {
|
||||
t.Errorf("%s: accepted", name)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(err.Error(), "ADR 0468 §8") || !strings.Contains(err.Error(), "requires") {
|
||||
t.Errorf("%s: the error does not name the entry and the ADR: %v", name, err)
|
||||
}
|
||||
}
|
||||
// The stored form is held to the same rules.
|
||||
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":["s3-bucket"],
|
||||
"requires-shares":{"s3-bucket":[{"to":"a","methods":[]}]}}`))
|
||||
if err == nil || !strings.Contains(err.Error(), "ADR 0468 §8") {
|
||||
t.Errorf("the stored form was accepted: %v", err)
|
||||
}
|
||||
}
|
||||
+331
-61
@@ -2,11 +2,25 @@ package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net"
|
||||
"net/url"
|
||||
"path"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Turning a manifest that names artifacts into one that names digests.
|
||||
//
|
||||
// **Two documents, deliberately.** The manifest in a repository says *this resource uses the
|
||||
// archive called `config`*; the manifest the mesh holds says *this resource is sha256:…*. A digest
|
||||
// is not knowable until something is built, so a repository carrying one would be a repository
|
||||
// whose file is wrong the moment anybody edits anything — and the mesh would be pinning a value
|
||||
// nobody could have checked.
|
||||
//
|
||||
// So the built manifest is **derived**, and the record of which commit it was derived from is what
|
||||
// makes "is this current?" answerable without building (novox/hq ADR 0009).
|
||||
|
||||
// Built is one artifact after it exists: where it is and what it hashes to.
|
||||
type Built struct {
|
||||
// Name is what the manifest called it.
|
||||
@@ -112,17 +126,33 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
|
||||
continue
|
||||
}
|
||||
made := by[a.Name]
|
||||
loads := BundleLoads(m, a, run)
|
||||
// What the runtime loads: what the artifact said, else every entrypoint of a module
|
||||
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
|
||||
//
|
||||
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
|
||||
// A process the host runs is the module's service, not its tools: the controller declares
|
||||
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
|
||||
// have been launched a second time by the node's runtime, as an MCP child it is not.
|
||||
loads := append([]string(nil), a.Loads...)
|
||||
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
|
||||
loads = append([]string(nil), a.Entrypoints...)
|
||||
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
|
||||
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
|
||||
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
|
||||
if bin := BinaryOf(a); bin != "" {
|
||||
loads = []string{bin}
|
||||
}
|
||||
}
|
||||
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
|
||||
// it reached it by, and a manifest carrying that address names an installation —
|
||||
// registration refused node-tools for exactly this on 2026-10-02. The build record
|
||||
// already keeps the store-relative form; the resolved manifest keeps the same, and
|
||||
// composition routes it through the store a machine reaches (the registry clients' Routed).
|
||||
// composition routes it through the store a machine reaches (Routed).
|
||||
out.Bundles = append(out.Bundles, Bundle{
|
||||
Name: a.Name, Source: kept(made.Reference), Digest: made.Digest,
|
||||
Name: a.Name, Source: Recorded(made.Reference), Digest: made.Digest,
|
||||
Language: a.Language, Entrypoints: append([]string(nil), a.Entrypoints...),
|
||||
Loads: loads, Env: copyWords(a.Env), Launchers: copyWords(made.Launchers),
|
||||
Binary: BinaryOf(a), Bus: a.Bus,
|
||||
Binary: BinaryOf(a),
|
||||
})
|
||||
}
|
||||
sort.Slice(out.Bundles, func(i, j int) bool { return out.Bundles[i].Name < out.Bundles[j].Name })
|
||||
@@ -171,7 +201,7 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
|
||||
// how they were made — one packed as it stood, the other compiled first — and a
|
||||
// machine has no reason to care which.
|
||||
// Kept, not routed, for the reason the bundles above are (ADR 0155).
|
||||
filled["source"] = kept(artifact.Reference)
|
||||
filled["source"] = Recorded(artifact.Reference)
|
||||
filled["digest"] = artifact.Digest
|
||||
// **And `${version}`, so a resource can name a place that is this build's alone**
|
||||
// (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named
|
||||
@@ -223,31 +253,227 @@ func withVersion(value any, version string) any {
|
||||
return value
|
||||
}
|
||||
|
||||
// NamedPaths are the paths a module's fingerprint takes its tree from (novox/hq ADR 0499 clause 2): the union of the
|
||||
// paths its artifacts name, cleaned and sorted, and every true only when the module builds something and every artifact
|
||||
// names paths; otherwise none, and the module keeps its directory's tree.
|
||||
//
|
||||
// **Read by the core alone** (ADR 0495 clause 1, the lead's decision L8 of R4e): the indexer and its merge preview
|
||||
// (buildstep.ModuleTrees) call it. A build module's fetch step is handed the paths by the builder, from the indexer's
|
||||
// kept inputs (delivery.KeptInputs2.Paths), and reads no manifest for them.
|
||||
func (m Manifest) NamedPaths() (paths []string, every bool) {
|
||||
if m.Build == nil || len(m.Build.Artifacts) == 0 {
|
||||
return nil, false
|
||||
// checkBuild is the manifest's own account of what it builds.
|
||||
func (b *Build) problems(module string) []string {
|
||||
if b == nil {
|
||||
return nil
|
||||
}
|
||||
var problems []string
|
||||
seen := map[string]bool{}
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if len(a.Paths) == 0 {
|
||||
return nil, false
|
||||
for _, a := range b.Artifacts {
|
||||
if a.Name == "" {
|
||||
problems = append(problems, module+" builds an artifact with no name")
|
||||
continue
|
||||
}
|
||||
for _, p := range a.Paths {
|
||||
seen[path.Clean(p)] = true
|
||||
if seen[a.Name] {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s builds two artifacts called %q, and a resource naming it could mean either",
|
||||
module, a.Name))
|
||||
}
|
||||
seen[a.Name] = true
|
||||
switch a.Kind {
|
||||
case ArtifactImage, ArtifactArchive, ArtifactUpstream, ArtifactBundle, ArtifactPackage:
|
||||
default:
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a %q, and an artifact is %q, %q, %q or %q",
|
||||
module, a.Name, a.Kind, ArtifactImage, ArtifactArchive, ArtifactUpstream,
|
||||
ArtifactBundle+", "+ArtifactPackage))
|
||||
}
|
||||
// **A bundle is built from the module itself, so it says a language instead.** Everything
|
||||
// else names what it is built from: a Dockerfile, a directory, somebody else's reference.
|
||||
// A bundle's source is the module's own directory by definition, and what it needs to say
|
||||
// is which compiler — because the mesh chooses that, and cannot choose for a module that
|
||||
// has not said.
|
||||
if len(a.External) > 0 && (a.Kind != ArtifactBundle || a.Language != "typescript") {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q names packages it keeps external, and only a TypeScript bundle is bundled into "+
|
||||
"one file with some kept out (novox/hq ADR 0193)", module, a.Name))
|
||||
}
|
||||
if len(a.Env) > 0 && a.Kind != ArtifactBundle {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a %q and says what it is given (env). Only a bundle the node's runtime "+
|
||||
"serves is given words (novox/hq ADR 0192); a container says its own environment",
|
||||
module, a.Name, a.Kind))
|
||||
}
|
||||
if a.Kind == ArtifactBundle || a.Kind == ArtifactPackage {
|
||||
// **Except for a language that compiles to a binary, where it names which one**
|
||||
// (novox/hq 04-ISSUES/142). A bundle in an interpreted language is the module's own
|
||||
// directory compiled whole, and naming a source would be describing its own build. A
|
||||
// repository written in a compiled language holds several commands — the host and its
|
||||
// bootstrap live in one, and the mesh needs the host — and "the module's own directory"
|
||||
// is then not a package at all. So the compiled case may say which package, and says
|
||||
// the module root by saying nothing.
|
||||
if a.From != "" && !compilesToABinary(a.Language) {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a bundle and names what it is built from (%q). A bundle is built "+
|
||||
"from the module's own directory; what it says is the language",
|
||||
module, a.Name, a.From))
|
||||
}
|
||||
if strings.TrimSpace(a.Language) == "" {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a bundle and says no language, so nothing can choose a compiler "+
|
||||
"for it", module, a.Name))
|
||||
}
|
||||
problems = append(problems, bundleEnvProblems(module, a)...)
|
||||
// What the runtime loads is among what was compiled (ADR 0175): a name here that is
|
||||
// not an entrypoint is a file the bundle does not contain, and the runtime would
|
||||
// fail to import it on every machine rather than here.
|
||||
for _, load := range a.Loads {
|
||||
found := false
|
||||
for _, e := range a.Entrypoints {
|
||||
found = found || e == load
|
||||
}
|
||||
// A bundle compiled to a binary is one executable: the runtime loads that or nothing
|
||||
// (novox/hq ADR 0193).
|
||||
if bin := BinaryOf(a); bin != "" {
|
||||
found = load == bin
|
||||
}
|
||||
if !found {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q says the runtime loads %q, which is not among its entrypoints — "+
|
||||
"what is loaded is compiled, so it is named there too", module, a.Name, load))
|
||||
}
|
||||
}
|
||||
// **A system, for a language that compiles to a binary** (novox/hq ADR 0142). A binary
|
||||
// is pinned to one operating system at link time so a host refuses to touch a machine
|
||||
// it was not built for (novox/hq ADR 0005); an artifact that says nothing would be
|
||||
// compiled for whatever the build machine happened to be, which reads as portable and
|
||||
// is not.
|
||||
if compiled := compilesToABinary(a.Language); compiled && strings.TrimSpace(a.System) == "" {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is compiled to a binary and says no system, so it would be built for "+
|
||||
"whatever the build machine happens to be. Declare one artifact per "+
|
||||
"system: %s", module, a.Name, spokenSystems()))
|
||||
} else if !compiled && strings.TrimSpace(a.System) != "" {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q names the system %q and is written in %q, which compiles to code that "+
|
||||
"runs anywhere — a system that decides nothing reads as though it did",
|
||||
module, a.Name, a.System, a.Language))
|
||||
} else if compiled && !knownSystem(a.System) {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is built for %q, and a system is %s",
|
||||
module, a.Name, a.System, spokenSystems()))
|
||||
}
|
||||
} else {
|
||||
if a.From == "" && a.URL == "" {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q says nothing about what it is built from", module, a.Name))
|
||||
}
|
||||
if a.Language != "" {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a %q and names a language. Only a bundle is compiled by the mesh; "+
|
||||
"everything else brings its own recipe", module, a.Name, a.Kind))
|
||||
}
|
||||
}
|
||||
if a.Compiles != "" {
|
||||
if a.Kind != ArtifactImage {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is a %q and names a Go package it compiles. Only an image's recipe is "+
|
||||
"told which; a bundle names its package in from (novox/hq ADR 0267)", module, a.Name, a.Kind))
|
||||
} else if c := strings.TrimSpace(a.Compiles); strings.HasPrefix(c, "/") || c == ".." ||
|
||||
strings.HasPrefix(c, "../") || strings.Contains(c, "/../") || strings.HasSuffix(c, "/..") {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q compiles %q, which leaves the tree it is built in", module, a.Name, a.Compiles))
|
||||
}
|
||||
}
|
||||
// An upstream image is named, not read from the repository, so the path rule does not
|
||||
// apply to it — and applying it anyway would refuse every reference with a registry host
|
||||
// in it.
|
||||
if a.Kind != ArtifactUpstream &&
|
||||
(strings.HasPrefix(a.From, "/") || strings.Contains(a.From, "..")) {
|
||||
// A build reads its own repository and nothing else. A path leaving it would make
|
||||
// what gets built depend on whatever happens to be on the machine building it.
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q is built from %q, which is outside its own repository",
|
||||
module, a.Name, a.From))
|
||||
}
|
||||
if a.Kind == ArtifactUpstream && !strings.Contains(a.From, ":") {
|
||||
// Without a tag or digest, what gets mirrored is whatever `latest` means today, and
|
||||
// a module pinned to that is not pinned.
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q mirrors %q, which names no tag or digest", module, a.Name, a.From))
|
||||
}
|
||||
problems = append(problems, upstreamFileProblems(module, a)...)
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// reservedSuffixes are names no public vendor publishes under: an installation's own and documentation's.
|
||||
var reservedSuffixes = []string{".internal", ".local", ".lan", ".home", ".corp", ".localhost", ".home.arpa",
|
||||
".test", ".invalid", ".example", ".intranet", ".private"}
|
||||
|
||||
// notAPublicHost says why a host is not a public vendor's name, or "".
|
||||
func notAPublicHost(host string) string {
|
||||
h := strings.ToLower(strings.TrimSuffix(host, "."))
|
||||
if net.ParseIP(h) != nil {
|
||||
return "an address rather than a name"
|
||||
}
|
||||
if !strings.Contains(h, ".") {
|
||||
return "a name with no domain"
|
||||
}
|
||||
for _, s := range reservedSuffixes {
|
||||
if strings.HasSuffix(h, s) {
|
||||
return "a name under " + s + ", which is nobody's public domain"
|
||||
}
|
||||
}
|
||||
for p := range seen {
|
||||
paths = append(paths, p)
|
||||
return ""
|
||||
}
|
||||
|
||||
// sha256Hex is a pinned sha256 as a manifest writes it: 64 lower-case hexadecimal characters.
|
||||
var sha256Hex = regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
|
||||
// upstreamFileProblems is what is wrong with an archive pinned by URL (novox/hq ADR 0316 §2, §6): it is an
|
||||
// archive, fetched over https, pinned by a sha256 and a version, and it names no directory of the repository
|
||||
// besides; a file name or a directory within is a plain path that stays where it is put.
|
||||
func upstreamFileProblems(module string, a Artifact) []string {
|
||||
pinned := a.URL != "" || a.SHA256 != "" || a.Version != "" || a.File != "" || a.Within != ""
|
||||
if !pinned {
|
||||
return nil
|
||||
}
|
||||
sort.Strings(paths)
|
||||
return paths, true
|
||||
var problems []string
|
||||
say := func(format string, args ...any) {
|
||||
problems = append(problems, fmt.Sprintf("%s: %q ", module, a.Name)+fmt.Sprintf(format, args...))
|
||||
}
|
||||
if a.Kind != ArtifactArchive {
|
||||
say("is a %q and names an upstream file (url, sha256, version, file, within): only an archive is "+
|
||||
"mirrored from one (novox/hq ADR 0316)", a.Kind)
|
||||
return problems
|
||||
}
|
||||
if a.URL == "" {
|
||||
say("names a sha256, a version, a file or a directory within, and no url to fetch it from")
|
||||
return problems
|
||||
}
|
||||
if a.From != "" {
|
||||
say("is built from %q and fetched from %s: an archive is one or the other", a.From, a.URL)
|
||||
}
|
||||
if u, err := url.Parse(a.URL); err != nil || u.Scheme != "https" || u.Host == "" {
|
||||
say("is fetched from %q, and an upstream file is fetched over https from a named host", a.URL)
|
||||
} else if why := notAPublicHost(u.Hostname()); why != "" {
|
||||
say("is fetched from %s, %s: an upstream file is somebody else's release, published under a public name", a.URL, why)
|
||||
}
|
||||
if !sha256Hex.MatchString(a.SHA256) {
|
||||
say("is fetched from %s without a pinned sha256 (64 lower-case hexadecimal characters): what that "+
|
||||
"address serves changes under the mesh, and only the digest says it is what was reviewed", a.URL)
|
||||
}
|
||||
if strings.TrimSpace(a.Version) == "" {
|
||||
say("pins %s and says no version, so a person reading the pin cannot tell which release it is", a.URL)
|
||||
}
|
||||
if a.File != "" && a.Within != "" {
|
||||
say("names a file (%q) and a directory within (%q): a file is kept whole, a directory is taken out "+
|
||||
"of an archive", a.File, a.Within)
|
||||
}
|
||||
for field, p := range map[string]string{"file": a.File, "within": a.Within} {
|
||||
if p == "" {
|
||||
continue
|
||||
}
|
||||
clean := path.Clean(p)
|
||||
if clean != p || strings.HasPrefix(p, "/") || clean == "." || clean == ".." || strings.HasPrefix(clean, "../") ||
|
||||
(field == "file" && strings.Contains(p, "/")) {
|
||||
say("names %s %q, which is not a plain relative %s", field, p, map[string]string{
|
||||
"file": "file name", "within": "directory"}[field])
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// oneOrOther keeps the message readable for one artifact and for several, because a message that
|
||||
@@ -259,6 +485,30 @@ func oneOrOther(n int) string {
|
||||
return "them"
|
||||
}
|
||||
|
||||
// Systems the mesh builds binaries for, which is the set a host may be pinned to (novox/hq ADR 0005).
|
||||
//
|
||||
// **A closed list, and the host's own, not the compiler's.** These are not the values a Go toolchain
|
||||
// would call an operating system — the difference between two of them is a C library, not a kernel.
|
||||
// They are what a machine reports itself to be and what a host is linked to refuse, so the list that
|
||||
// matters is the one the host understands.
|
||||
var systems = []string{"alpine", "android", "arch"}
|
||||
|
||||
// knownSystem is whether the mesh builds for it.
|
||||
func knownSystem(system string) bool {
|
||||
want := strings.ToLower(strings.TrimSpace(system))
|
||||
for _, s := range systems {
|
||||
if s == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// spokenSystems is the list as a refusal says it, so a reader is one edit from right.
|
||||
func spokenSystems() string {
|
||||
return strings.Join(systems, ", ")
|
||||
}
|
||||
|
||||
// compilesToABinary is whether this language's bundle is a binary for one operating system rather
|
||||
// than code that runs wherever its interpreter does.
|
||||
//
|
||||
@@ -299,6 +549,42 @@ func versionOf(digest string) string {
|
||||
return hex
|
||||
}
|
||||
|
||||
// bundleEnvWords are the words the runtime sets for itself; a bundle that named one would be
|
||||
// telling the runtime what it is, which is the mesh's to say (novox/hq ADR 0192).
|
||||
var bundleEnvWords = map[string]bool{
|
||||
RuntimeToolModules: true, RuntimeBrokerFile: true, RuntimeOperatorAccount: true,
|
||||
RuntimeOperatorHome: true, RuntimeToolEnv: true, RuntimeAgentAccount: true, RuntimeAgentHome: true,
|
||||
}
|
||||
|
||||
// bundleEnvProblems says what is wrong with what a bundle says it is given (novox/hq ADR 0192):
|
||||
// a value is a path or a constant written with the references a container's environment may use
|
||||
// for a place or a port, and never a secret's content or another module's binding — a secret
|
||||
// reaches a tool as a file whose path is named.
|
||||
func bundleEnvProblems(module string, a Artifact) []string {
|
||||
if len(a.Env) == 0 {
|
||||
return nil
|
||||
}
|
||||
var problems []string
|
||||
for _, word := range sortedKeys(a.Env) {
|
||||
value := a.Env[word]
|
||||
if bundleEnvWords[word] {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q gives itself %s, which the node's runtime sets for itself; a bundle is "+
|
||||
"given its own words beside the runtime's, never in place of them (novox/hq ADR 0192)",
|
||||
module, a.Name, word))
|
||||
}
|
||||
rest := ofPort.ReplaceAllString(dirRef.ReplaceAllString(value, ""), "")
|
||||
if strings.Contains(rest, "${") {
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: %q gives %s the value %q. A bundle's word is a path or a constant, written with "+
|
||||
"${dir:…} and ${port:…} only; a secret reaches a tool as a file the mesh places, "+
|
||||
"named by its path, never as its content (novox/hq ADR 0192)",
|
||||
module, a.Name, word, value))
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
func copyWords(in map[string]string) map[string]string {
|
||||
if len(in) == 0 {
|
||||
return nil
|
||||
@@ -339,43 +625,27 @@ func runByAResource(m Manifest) map[string]bool {
|
||||
return named
|
||||
}
|
||||
|
||||
// BundleLoads is what the runtime loads of the bundle artifact a of m (run: the bundles one of m's own resources runs,
|
||||
// runByAResource): what the artifact said, else every entrypoint of a module that declares tools, else nothing; a
|
||||
// bundle compiled to a binary is served by its binary. The rule a build's resolution keeps on Bundle.Loads, and what a
|
||||
// module composed as what runs of it reads again when the runtime already dropped it (issue 1126).
|
||||
func BundleLoads(m Manifest, a Artifact, run map[string]bool) []string {
|
||||
// What the runtime loads: what the artifact said, else every entrypoint of a module
|
||||
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
|
||||
//
|
||||
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
|
||||
// A process the host runs is the module's service, not its tools: the controller declares
|
||||
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
|
||||
// have been launched a second time by the node's runtime, as an MCP child it is not.
|
||||
loads := append([]string(nil), a.Loads...)
|
||||
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
|
||||
loads = append([]string(nil), a.Entrypoints...)
|
||||
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
|
||||
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
|
||||
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
|
||||
if bin := BinaryOf(a); bin != "" {
|
||||
loads = []string{bin}
|
||||
// undeliveredBundles says which of a module's bundles nothing would ever put on a machine (novox/hq
|
||||
// 04-ISSUES/216). A bundle reaches a machine three ways: the node's runtime serves it (it says
|
||||
// `loads`, or its module declares `tools`), a resource names it (a process, a step, an archive), or
|
||||
// it is the runtime itself. One reached by none of them was built, recorded and pushed as success,
|
||||
// and was simply absent — seven modules' tools went missing that way on 2026-10-03. Refused here,
|
||||
// naming the field that would deliver it.
|
||||
func undeliveredBundles(m Manifest) []string {
|
||||
if m.Build == nil || m.Module == RuntimeModule {
|
||||
return nil
|
||||
}
|
||||
named := runByAResource(m)
|
||||
var problems []string
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a.Kind != ArtifactBundle || named[a.Name] || len(a.Loads) > 0 || len(m.Tools) > 0 {
|
||||
continue
|
||||
}
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"%s: the bundle %q would be built and never reach a machine: nothing loads it, runs it or "+
|
||||
"unpacks it. A tools bundle says `loads` (the entrypoints the node's runtime serves) or its "+
|
||||
"module lists its `tools`; a daemon or a step is a resource naming it (novox/hq 04-ISSUES/216)",
|
||||
m.Module, a.Name))
|
||||
}
|
||||
return loads
|
||||
}
|
||||
|
||||
// kept is a bundle's or an archive's reference without the artifact store's address, if the builder wrote one (ADR 0155:
|
||||
// a resolved manifest names no installation). Only the address is read here: an `http://<host>/v2/<path>/blobs/<digest>`
|
||||
// becomes `artifact-store://<path>/blobs/<digest>`; a kept reference, or one in a registry seat's form (whose first part,
|
||||
// a seat's name, has no dot or port), is what it was. How the registry seats' forms are read is the registry clients'.
|
||||
func kept(reference string) string {
|
||||
if strings.HasPrefix(reference, ArtifactStoreScheme) {
|
||||
return reference
|
||||
}
|
||||
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
|
||||
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
}
|
||||
return reference
|
||||
return problems
|
||||
}
|
||||
|
||||
@@ -1,8 +1,82 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// bundleFor is a manifest whose one artifact is a bundle in the given language and system.
|
||||
func bundleFor(language, system string) Manifest {
|
||||
return Manifest{Module: "a-component", Build: &Build{Artifacts: []Artifact{
|
||||
{Name: "binary", Kind: ArtifactBundle, Language: language, System: system},
|
||||
}}}
|
||||
}
|
||||
|
||||
func problemsOf(t *testing.T, m Manifest) string {
|
||||
t.Helper()
|
||||
return strings.Join(m.Build.problems(m.Module), "\n")
|
||||
}
|
||||
|
||||
// **A language that compiles to a binary must say which system.**
|
||||
//
|
||||
// A binary is pinned to one operating system at link time, so a host refuses to touch a machine it
|
||||
// was not built for. An artifact that says nothing would be compiled for whatever the build machine
|
||||
// happened to be — which reads as portable and is not, and is the fault this check exists for.
|
||||
func TestABinaryMustSayWhichSystemItIsFor(t *testing.T) {
|
||||
got := problemsOf(t, bundleFor("go", ""))
|
||||
if !strings.Contains(got, "says no system") {
|
||||
t.Fatalf("a compiled bundle with no system was accepted:\n%s", got)
|
||||
}
|
||||
// And the refusal names what it could have said, so a reader is one edit from right.
|
||||
for _, system := range []string{"alpine", "android", "arch"} {
|
||||
if !strings.Contains(got, system) {
|
||||
t.Fatalf("the refusal does not name %q as a choice:\n%s", system, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestABinaryThatNamesASystemIsAccepted(t *testing.T) {
|
||||
if got := problemsOf(t, bundleFor("go", "arch")); got != "" {
|
||||
t.Fatalf("a compiled bundle naming a system was refused:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A system the mesh does not build for is refused where it is written. These are the host's own
|
||||
// names, not a compiler's: the difference between two of them is a C library rather than a kernel,
|
||||
// so a value that looks like an operating system to a toolchain is still wrong here.
|
||||
func TestASystemTheMeshDoesNotBuildForIsRefused(t *testing.T) {
|
||||
for _, wrong := range []string{"linux", "debian", "darwin"} {
|
||||
got := problemsOf(t, bundleFor("go", wrong))
|
||||
if !strings.Contains(got, "and a system is") {
|
||||
t.Fatalf("%q was accepted as a system:\n%s", wrong, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// **And a language that runs anywhere must not name one.** A system that decides nothing reads as
|
||||
// though it did, which is the same fault as a restriction that restricts nothing (novox/hq ADR 0045).
|
||||
func TestAPortableBundleMayNotNameASystem(t *testing.T) {
|
||||
got := problemsOf(t, bundleFor("typescript", "arch"))
|
||||
if !strings.Contains(got, "runs anywhere") {
|
||||
t.Fatalf("a portable bundle was allowed to name a system:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPortableBundleNamingNoSystemIsAccepted(t *testing.T) {
|
||||
if got := problemsOf(t, bundleFor("typescript", "")); got != "" {
|
||||
t.Fatalf("an ordinary bundle was refused:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// One component, one artifact per system: the shape the mesh's own binaries are declared in, and the
|
||||
// reason the target is the artifact's rather than the recipe's.
|
||||
func TestOneArtifactPerSystemIsAccepted(t *testing.T) {
|
||||
m := Manifest{Module: "the-host", Build: &Build{Artifacts: []Artifact{
|
||||
{Name: "arch", Kind: ArtifactBundle, Language: "go", System: "arch"},
|
||||
{Name: "alpine", Kind: ArtifactBundle, Language: "go", System: "alpine"},
|
||||
{Name: "android", Kind: ArtifactBundle, Language: "go", System: "android"},
|
||||
}}}
|
||||
if got := problemsOf(t, m); got != "" {
|
||||
t.Fatalf("one artifact per system was refused:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -108,8 +108,68 @@ func TestAModuleThatBuildsNothingIsOrdinary(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTwoArtifactsWithOneNameAreRefused(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"x","kind":"image","from":"Dockerfile"},
|
||||
{"name":"x","kind":"archive","from":"files"}]}}`))
|
||||
if err == nil {
|
||||
t.Fatal("two artifacts with one name were accepted")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "could mean either") {
|
||||
t.Fatalf("unhelpful refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnArtifactBuiltFromOutsideItsRepositoryIsRefused(t *testing.T) {
|
||||
// A build reads its own repository and nothing else. A path leaving it would make what gets
|
||||
// built depend on whatever happens to be on the machine doing the building.
|
||||
for _, from := range []string{"/etc/passwd", "../elsewhere"} {
|
||||
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"x","kind":"archive","from":"` + from + `"}]}}`))
|
||||
if err == nil {
|
||||
t.Fatalf("%q was accepted as a build input", from)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "outside its own repository") {
|
||||
t.Fatalf("refused for the wrong reason: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnArtifactOfAnUnknownKindIsRefused(t *testing.T) {
|
||||
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"x","kind":"binary","from":"main.go"}]}}`))
|
||||
if err == nil {
|
||||
t.Fatal("an artifact of an unknown kind was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPackageParsesLikeABundleAndNeedsALanguage(t *testing.T) {
|
||||
// The SDK's shape: a package built from the module's own directory, naming a language.
|
||||
m, err := ParseManifest([]byte(`{"module":"mesh-sdk","version":"1","slug":"sdk",
|
||||
"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript"}]},
|
||||
"resources":[]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("the SDK's package manifest did not parse: %v", err)
|
||||
}
|
||||
if m.Build.Artifacts[0].Kind != ArtifactPackage {
|
||||
t.Fatalf("expected a package artifact, got %q", m.Build.Artifacts[0].Kind)
|
||||
}
|
||||
|
||||
// A package that names what it is built from is refused, exactly as a bundle is: it is built
|
||||
// from the module's own directory.
|
||||
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"lib","kind":"package","language":"typescript","from":"Dockerfile"}]}}`)); err == nil {
|
||||
t.Fatal("a package naming a source was accepted")
|
||||
}
|
||||
// A package with no language cannot choose a toolchain.
|
||||
if _, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"lib","kind":"package"}]}}`)); err == nil {
|
||||
t.Fatal("a package with no language was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAResourceNamingAPackageIsRefused(t *testing.T) {
|
||||
m, err := decodeForTest([]byte(`{"module":"a","version":"1","slug":"a",
|
||||
m, err := ParseManifest([]byte(`{"module":"a","version":"1","slug":"a",
|
||||
"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript"}]},
|
||||
"resources":[{"id":"svc","type":"container","name":"a","artifact":"lib"}]}`))
|
||||
if err != nil {
|
||||
@@ -120,3 +180,24 @@ func TestAResourceNamingAPackageIsRefused(t *testing.T) {
|
||||
t.Fatal("a resource backed by a package was accepted; a package is not a resource")
|
||||
}
|
||||
}
|
||||
|
||||
// An image names the Go package its recipe compiles (novox/hq ADR 0267): within the tree it is built in,
|
||||
// and only an image says one.
|
||||
func TestOnlyAnImageNamesThePackageItCompilesWithinItsTree(t *testing.T) {
|
||||
m, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[
|
||||
{"name":"x","kind":"image","from":"Dockerfile","compiles":"cmd/x"}]}}`))
|
||||
if err != nil || m.Build.Artifacts[0].Compiles != "cmd/x" {
|
||||
t.Fatalf("an image naming its package was refused or lost it: %v", err)
|
||||
}
|
||||
for _, c := range []struct{ artifact, says string }{
|
||||
{`{"name":"x","kind":"archive","from":"files","compiles":"cmd/x"}`, "Only an image"},
|
||||
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"../x"}`, "leaves the tree"},
|
||||
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"/x"}`, "leaves the tree"},
|
||||
{`{"name":"x","kind":"image","from":"Dockerfile","compiles":"a/../../x"}`, "leaves the tree"},
|
||||
} {
|
||||
_, err := ParseManifest([]byte(`{"module":"a","version":"1","build":{"artifacts":[` + c.artifact + `]}}`))
|
||||
if err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%s: %v, wanted a refusal saying %q", c.artifact, err, c.says)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,6 +9,40 @@ import (
|
||||
"git.novox.be/novox/mesh-sdk/go/internal/beside"
|
||||
)
|
||||
|
||||
// The module holding mesh-broker is the bus, and its account is granted the bus's snapshot API and
|
||||
// nothing else (novox/hq ADR 0235). Anything it declared to say or hear on the bus would be granted
|
||||
// nothing, so it is refused at the parser rather than silently dropped.
|
||||
func TestTheBussAccountSaysNothingOnTheBus(t *testing.T) {
|
||||
raw := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
|
||||
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
|
||||
"emits":["something.happened"]}`)
|
||||
_, err := ParseManifest(raw)
|
||||
if err == nil || !strings.Contains(err.Error(), "granted the bus's snapshot API and nothing else") {
|
||||
t.Fatalf("a bus module declaring what it emits was not refused, or not for the reason: %v", err)
|
||||
}
|
||||
|
||||
// Its tools bundle says what it emits (novox/hq ADR 0357): the node's runtime carries the bundle and is
|
||||
// granted the module's events, so a bus module that ships one may declare them.
|
||||
bundled := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
|
||||
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
|
||||
"emits":["bus.slow-consumer"],"tools":["bus_streams"],
|
||||
"build":{"artifacts":[{"name":"tools","kind":"bundle","language":"go","system":"arch","from":"cmd/t","binary":"t"}]}}`)
|
||||
if _, err := ParseManifest(bundled); err != nil {
|
||||
t.Fatalf("a bus module whose tools bundle declares what it emits was refused: %v", err)
|
||||
}
|
||||
|
||||
quiet := []byte(`{"module":"bus","version":"1","provides":[{"name":"mesh-bus","scope":"mesh"}],
|
||||
"claims":[{"name":"mesh-broker","scope":"mesh"}],"own-secrets":{"broker":"/run/broker"},
|
||||
"tools":["bus_streams"]}`)
|
||||
m, err := ParseManifest(quiet)
|
||||
if err != nil {
|
||||
t.Fatalf("a bus module with an account and tools was refused: %v", err)
|
||||
}
|
||||
if !m.ClaimsSeat(BrokerSeat) {
|
||||
t.Fatal("it does not read as holding the broker seat")
|
||||
}
|
||||
}
|
||||
|
||||
// The catalogue's bus protects its streams by the snapshot, not as live files: its streams' item is a
|
||||
// dump into its snapshots, it has the account the dump runs as, and the dump runs the snapshot
|
||||
// program in the bus's own container.
|
||||
@@ -17,7 +51,7 @@ func TestTheCataloguesBusIsBackedUpBySnapshot(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := decodeForTest(raw)
|
||||
m, err := ParseManifest(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -1,207 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Bus state is what a module keeps on the bus, asked for as a provision of the bus module in `requires` (novox/hq ADR
|
||||
// 0595 §1, design 32 § 4): an entry of `requires` in its typed long form, the local name, the storage type, the template
|
||||
// and the template's options. This is the contract only (ADR 0578 §7, ADR 0593 §1): the entry's shape, decoded
|
||||
// leniently. Every rule that judges an entry (a template known, an option within its bound, a fixed setting not given)
|
||||
// is the indexer's judge's.
|
||||
|
||||
// The storage types (design 25 § 12.1): the stream is the base; the key-value bucket, the object store and the ledger
|
||||
// are built on it, each with calls of its own (ADR 0595 §2).
|
||||
const (
|
||||
StorageStream = "stream"
|
||||
StorageKeyValue = "key-value"
|
||||
StorageObjectStore = "object-store"
|
||||
StorageLedger = "ledger"
|
||||
)
|
||||
|
||||
// The six templates, each a storage type with a predefined configuration (ADR 0595 §2, design 25 § 12.1): state on
|
||||
// key-value, events, work queue and log on stream, objects on object store, ledger on ledger. A bare stream names
|
||||
// none and gives its own Configuration.
|
||||
const (
|
||||
TemplateState = "state"
|
||||
TemplateEvents = "events"
|
||||
TemplateWorkQueue = "work-queue"
|
||||
TemplateLog = "log"
|
||||
TemplateObjects = "objects"
|
||||
TemplateLedger = "ledger"
|
||||
)
|
||||
|
||||
// The option names, a template's settings a module may change (design 25 § 12.1's table), written as the old sections
|
||||
// wrote them, so a module that moves carries its old settings as options unchanged (ADR 0595 §11). Which option a
|
||||
// template takes, and its bound, is the judge's.
|
||||
const (
|
||||
// OptionPastValues is how many past values a state's key keeps (the old `state`'s `history`).
|
||||
OptionPastValues = "history"
|
||||
// OptionTimeToLive is how long a state's value lives, in seconds.
|
||||
OptionTimeToLive = "ttl-seconds"
|
||||
// OptionLifetimePerKey is whether a state's key may be given a lifetime of its own.
|
||||
OptionLifetimePerKey = "key-ttl"
|
||||
// OptionPerMachine is whether a state is kept one bucket per machine.
|
||||
OptionPerMachine = "per-machine"
|
||||
// OptionAgeDays is how long a stream keeps a message, in days.
|
||||
OptionAgeDays = "max-age-days"
|
||||
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
|
||||
OptionCapMiB = "max-mib"
|
||||
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
|
||||
OptionGrant = "grant"
|
||||
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
|
||||
OptionDeliveries = "max-deliveries"
|
||||
// OptionDuplicates is how long, in seconds, a work queue detects a message sent twice.
|
||||
OptionDuplicates = "duplicates"
|
||||
)
|
||||
|
||||
// ConfigWhenFull is what a configuration of its own says a bare stream does when full: WhenFullRefuse or
|
||||
// WhenFullDropOldest. In a template it is fixed, never an option (ADR 0595 §3).
|
||||
const (
|
||||
ConfigWhenFull = "when-full"
|
||||
WhenFullRefuse = "refuse"
|
||||
WhenFullDropOldest = "drop-oldest"
|
||||
)
|
||||
|
||||
// The ledger effects, what an operation does to an instance's state (ADR 0597 §2): one of this fixed set.
|
||||
const (
|
||||
// EffectSetFields sets the state's fields to the operation's Fields.
|
||||
EffectSetFields = "set-fields"
|
||||
// EffectSetEntry sets one entry of the state's map Into, under the key the operation's field Key holds, to its Fields.
|
||||
EffectSetEntry = "set-entry"
|
||||
// EffectAppend appends the operation's Fields to the state's list Into.
|
||||
EffectAppend = "append"
|
||||
// EffectRemove removes the entry of the map Into under the operation's field Key, or, naming no Into, the instance's
|
||||
// state.
|
||||
EffectRemove = "remove"
|
||||
)
|
||||
|
||||
// LedgerType is one type of a ledger (ADR 0597 §1): a schema and the operations allowed on it, with its indexes. An
|
||||
// instance of it is named by a path whose first part is the type's name (`decision/0290`).
|
||||
type LedgerType struct {
|
||||
Name string `json:"name"`
|
||||
// Schema is the type's fields, by name.
|
||||
Schema map[string]LedgerField `json:"schema"`
|
||||
// Operations are the operations allowed on an instance, each with its effect.
|
||||
Operations []LedgerOperation `json:"operations"`
|
||||
// Indexes are its declared indexes, each a second key-value bucket kept by a watcher (ADR 0597 §4).
|
||||
Indexes []LedgerIndex `json:"indexes,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerField is one field of a ledger type's schema.
|
||||
type LedgerField struct {
|
||||
// Type is the field's value type, as the schema names it.
|
||||
Type string `json:"type"`
|
||||
// DigestOf names the text field this field is the digest of (ADR 0597 §2): `apply` refuses an operation whose
|
||||
// digest does not match its text.
|
||||
DigestOf string `json:"digest-of,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerOperation is one operation a ledger type allows, and its effect on the state.
|
||||
type LedgerOperation struct {
|
||||
Name string `json:"name"`
|
||||
// Effect is one of the ledger effects (EffectSetFields, EffectSetEntry, EffectAppend, EffectRemove).
|
||||
Effect string `json:"effect"`
|
||||
// Fields are the operation's fields its effect copies into the state; it may carry more (ADR 0597 §2).
|
||||
Fields []string `json:"fields,omitempty"`
|
||||
// Into is the state's map (EffectSetEntry, EffectRemove) or list (EffectAppend) the effect writes.
|
||||
Into string `json:"into,omitempty"`
|
||||
// Key is the operation's field holding the entry's key (EffectSetEntry, EffectRemove).
|
||||
Key string `json:"key,omitempty"`
|
||||
// Sequence and Author are whether the effect also writes the operation's sequence and its proven author.
|
||||
Sequence bool `json:"sequence,omitempty"`
|
||||
Author bool `json:"author,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerIndex is one declared index of a ledger type: its name, and the fields it is kept on.
|
||||
type LedgerIndex struct {
|
||||
Name string `json:"name"`
|
||||
On []string `json:"on"`
|
||||
}
|
||||
|
||||
// BusStateField is where a stored manifest keeps its bus-state entries: beside `requires`, which stays a plain list of
|
||||
// names, as OptionalRequiresKey keeps the optional ones.
|
||||
const BusStateField = "bus-state"
|
||||
|
||||
// BusStateEntry is one piece of a module's bus state, as its manifest asks for it.
|
||||
type BusStateEntry struct {
|
||||
// Name is its local name; the mesh, never the module, names it on the bus (ADR 0201 §4).
|
||||
Name string `json:"name"`
|
||||
// Storage is its storage type (design 25 § 12.1): how the bus keeps it.
|
||||
Storage string `json:"storage"`
|
||||
// Template is the predefined configuration of that storage type it uses; empty for a bare stream, which gives
|
||||
// Configuration instead.
|
||||
Template string `json:"template,omitempty"`
|
||||
// Options are the template's settings the module changes, by option name; each value as the manifest wrote it.
|
||||
Options map[string]json.RawMessage `json:"options,omitempty"`
|
||||
// Configuration is a configuration of the module's own, where no template fits: a bare stream's, or a ledger's
|
||||
// whose history expires (ADR 0595 §2).
|
||||
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
|
||||
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
|
||||
Types []LedgerType `json:"types,omitempty"`
|
||||
}
|
||||
|
||||
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
|
||||
func isBusStateEntry(raw json.RawMessage) bool {
|
||||
var keys map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &keys) != nil {
|
||||
return false
|
||||
}
|
||||
_, has := keys["storage"]
|
||||
return has
|
||||
}
|
||||
|
||||
// readBusStateEntry reads one bus-state entry leniently (ADR 0578 §3): a key it does not know, at any depth, is set
|
||||
// aside and its path named, never refused. An error is only an entry of the wrong shape.
|
||||
func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
|
||||
var tree any
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&tree); err != nil {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
var aside []string
|
||||
strict := func(e *BusStateEntry) error {
|
||||
b, err := json.Marshal(tree)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d := json.NewDecoder(bytes.NewReader(b))
|
||||
d.DisallowUnknownFields()
|
||||
return typedUnknown(d.Decode(e))
|
||||
}
|
||||
for tries := 0; ; tries++ {
|
||||
var e BusStateEntry
|
||||
err := strict(&e)
|
||||
if err == nil {
|
||||
return e, aside, nil
|
||||
}
|
||||
unknown := asUnknownField(err)
|
||||
if unknown == nil || tries > 64 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path, ok := removalThatHelps(tree, unknown.Field, err.Error(), func() error { var e BusStateEntry; return strict(&e) })
|
||||
if !ok {
|
||||
if removeEvery(tree, unknown.Field) == 0 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path = "…." + unknown.Field
|
||||
}
|
||||
aside = append(aside, path)
|
||||
}
|
||||
}
|
||||
|
||||
// setAsideAlso adds paths read without to what a manifest set aside (SetAside), in its words.
|
||||
func setAsideAlso(unknown string, paths []string) string {
|
||||
if len(paths) == 0 {
|
||||
return unknown
|
||||
}
|
||||
if unknown == "" {
|
||||
field := paths[0][strings.LastIndex(paths[0], ".")+1:]
|
||||
unknown = fmt.Sprintf("json: unknown field %q", field)
|
||||
}
|
||||
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
|
||||
}
|
||||
@@ -7,6 +7,18 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// A long-running container may say it is busy (novox/hq ADR 0335 §1).
|
||||
//
|
||||
// **Beside its health, on the container**: a command run inside it, by its shell, as an exec health check
|
||||
// is, that exits 0 while the container holds work a replacement would lose and non-zero when it is idle;
|
||||
// and how long one ask may take. Being asked is also its notice that it is about to be replaced: from the
|
||||
// first ask it takes no new work. The node-engine asks before it replaces the container, leaves it running
|
||||
// while it says busy, and replaces it on the first pass it says idle, or at 15 minutes — the engine's bound,
|
||||
// never the module's.
|
||||
//
|
||||
// Sent only to a node-engine that reads it (link.BusyContract): an older, strict engine refuses a field it
|
||||
// does not know, whole, and to it the container is replaced at once, as before.
|
||||
|
||||
// BusyField is the resource field.
|
||||
const BusyField = "busy"
|
||||
|
||||
@@ -67,3 +79,46 @@ func ReadBusy(r map[string]any) (Busy, bool, []string) {
|
||||
}
|
||||
return b, true, problems
|
||||
}
|
||||
|
||||
// busyProblems is everything wrong with a manifest's `busy` fields, in the manifest's words: a command on
|
||||
// one line, a timeout within its bounds, and only on a container that stays up.
|
||||
func busyProblems(m Manifest) []string {
|
||||
var problems []string
|
||||
for _, r := range m.Resources {
|
||||
b, has, read := ReadBusy(r)
|
||||
if !has {
|
||||
continue
|
||||
}
|
||||
where := m.Module + ": " + fmt.Sprint(r["id"])
|
||||
for _, p := range read {
|
||||
problems = append(problems, m.Module+": "+p)
|
||||
}
|
||||
if fmt.Sprint(r["type"]) != "container" || !LongRunning(r) {
|
||||
problems = append(problems, fmt.Sprintf("%s declares busy and is not a container that stays up: busy is "+
|
||||
"asked of a running container before it is replaced (ADR 0335)", where))
|
||||
}
|
||||
if b.Command == "" {
|
||||
problems = append(problems, where+"'s busy names no command: one run inside the container, exiting 0 "+
|
||||
"while it is busy")
|
||||
}
|
||||
if strings.ContainsAny(b.Command, "\n\r") {
|
||||
problems = append(problems, where+"'s busy command is one line")
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// busyInto composes a container's `busy` into the node-engine's words, or takes it away: for an engine that
|
||||
// reads it, the command and its timeout written out; for one that does not, the field is not sent, and the
|
||||
// container is replaced at once, as before.
|
||||
func busyInto(resource map[string]any, with Rendering) {
|
||||
if _, has := resource[BusyField]; !has {
|
||||
return
|
||||
}
|
||||
if !with.ReadsBusy {
|
||||
delete(resource, BusyField)
|
||||
return
|
||||
}
|
||||
b, _, _ := ReadBusy(resource)
|
||||
resource[BusyField] = map[string]any{"command": b.Command, "timeout": b.Timeout.String()}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/validate"
|
||||
)
|
||||
|
||||
// A long-running container may say it is busy (novox/hq ADR 0335 §1): `module check` holds the field to its
|
||||
// shape near its author, and it is sent only to a node-engine that reads it, in words the engine's own
|
||||
// validator accepts.
|
||||
|
||||
func busyManifest(t *testing.T, on string, busy any) []byte {
|
||||
t.Helper()
|
||||
extra := map[string]any{"id": "agent", "type": "container", "name": "app-agent",
|
||||
"image": "registry.example/app" + healthDigest}
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(healthManifest(t, "", nil, extra), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range m["resources"].([]any) {
|
||||
if r := r.(map[string]any); r["id"] == on {
|
||||
r[BusyField] = busy
|
||||
}
|
||||
}
|
||||
out, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestABusyIsHeldToItsShape(t *testing.T) {
|
||||
if _, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "mesh-builder busy"})); err != nil {
|
||||
t.Fatalf("a well-formed busy was refused: %v", err)
|
||||
}
|
||||
if _, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "x", "timeout": "30s"})); err != nil {
|
||||
t.Fatalf("a busy with its timeout was refused: %v", err)
|
||||
}
|
||||
for name, c := range map[string]struct {
|
||||
on string
|
||||
busy any
|
||||
says string
|
||||
}{
|
||||
"text": {"agent", "mesh-builder busy", "it is an object"},
|
||||
"no command": {"agent", map[string]any{"timeout": "5s"}, "names no command"},
|
||||
"two lines": {"agent", map[string]any{"command": "a\nb"}, "one line"},
|
||||
"a long timeout": {"agent", map[string]any{"command": "x", "timeout": "5m"}, "at most 1m0s"},
|
||||
"an unknown key": {"agent", map[string]any{"command": "x", "every": "1m"}, `says "every"`},
|
||||
"on a step": {"seed", map[string]any{"command": "x"}, "not a container that stays up"},
|
||||
"on a service": {"daemon", map[string]any{"command": "x"}, "not a container that stays up"},
|
||||
} {
|
||||
_, err := ParseManifest(busyManifest(t, c.on, c.busy))
|
||||
if err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%s: want a refusal saying %q, got %v", name, c.says, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestABusyIsSentOnlyToAnEngineThatReadsIt(t *testing.T) {
|
||||
m, err := ParseManifest(busyManifest(t, "agent", map[string]any{"command": "mesh-builder busy"}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
compose := func(with Rendering) []map[string]any {
|
||||
out, err := Resolution{Node: "laptop", Modules: []Manifest{m}}.Declaration(with)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
older := compose(Rendering{})
|
||||
if b, sent := older[indexOfID(older, "app.agent")][BusyField]; sent {
|
||||
t.Fatalf("busy was sent to an engine that does not read it: %v", b)
|
||||
}
|
||||
newer := compose(Rendering{ReadsBusy: true})
|
||||
got, _ := newer[indexOfID(newer, "app.agent")][BusyField].(map[string]any)
|
||||
if got["command"] != "mesh-builder busy" || got["timeout"] != "10s" {
|
||||
t.Fatalf("busy composed as %v", got)
|
||||
}
|
||||
// And the engine's own validator takes the container with it.
|
||||
body, err := json.Marshal(map[string]any{"declaration": validate.Version, "resources": newer})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if problems := validate.Declaration(body); len(problems) > 0 {
|
||||
t.Fatalf("the node-engine's validator refuses the composed busy: %v", problems)
|
||||
}
|
||||
if _, has := m.Resources[len(m.Resources)-1][BusyField].(map[string]any)["timeout"]; has {
|
||||
t.Error("composing wrote the timeout into the catalogue's own manifest")
|
||||
}
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// CatalogSeat is the seat of the catalog: what the mesh holds of every module (novox/hq ADR 0604 §1). Its states are
|
||||
// its holder's: the modules the trunk holds, their manifests by digest and the module graph (the indexer's states
|
||||
// `modules`, `manifests` and `graph`), announced with who may read them; a reader that only watches reads them and calls
|
||||
// no verb. It names no event. What must be built again is no verb: a module whose fingerprint differs from its newest
|
||||
// build, which the delivery view derives. The verbs' answers are CatalogModules, CatalogOne, CatalogProvides and
|
||||
// CatalogDependents, whose field names are those of the catalog module's tools they replace.
|
||||
|
||||
// CatalogEntry is one module of the catalog: its repository, its directory there ("." for the root), the merge commit it
|
||||
// was last changed at, and its new-form fingerprint or why it has none.
|
||||
type CatalogEntry struct {
|
||||
Module string `json:"module"`
|
||||
Repository string `json:"repository"`
|
||||
Path string `json:"path"`
|
||||
Commit string `json:"commit"`
|
||||
FP string `json:"fp,omitempty"`
|
||||
None string `json:"none,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogModule is one module whole: its entry and its manifest as the trunk holds it.
|
||||
type CatalogModule struct {
|
||||
CatalogEntry
|
||||
Manifest json.RawMessage `json:"manifest"`
|
||||
}
|
||||
|
||||
// CatalogEnd is a module that provides, requires or depends on something, with its commit; Scope is a provider's scope
|
||||
// (a module provides its own name at node scope).
|
||||
type CatalogEnd struct {
|
||||
Module string `json:"module"`
|
||||
Commit string `json:"commit"`
|
||||
Scope string `json:"scope,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogModules is the answer of the verb modules: every module the trunk holds, sorted by name.
|
||||
type CatalogModules struct {
|
||||
Modules []CatalogEntry `json:"modules"`
|
||||
}
|
||||
|
||||
// CatalogOne is the answer of the verb module.
|
||||
type CatalogOne struct {
|
||||
Module CatalogModule `json:"module"`
|
||||
}
|
||||
|
||||
// CatalogProvides is the answer of the verb provides: the modules that provide the provision and those that require it.
|
||||
type CatalogProvides struct {
|
||||
Provision string `json:"provision"`
|
||||
Provides []CatalogEnd `json:"provides"`
|
||||
Requires []CatalogEnd `json:"requires"`
|
||||
}
|
||||
|
||||
// CatalogDependents is the answer of the verb dependents: the modules whose build edges reach the module.
|
||||
type CatalogDependents struct {
|
||||
Module string `json:"module"`
|
||||
Dependents []CatalogEnd `json:"dependents"`
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/internal/beside"
|
||||
@@ -18,3 +20,119 @@ func catalogueRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
return beside.Dir(t, "mesh-catalog")
|
||||
}
|
||||
|
||||
func TestEveryCatalogueManifestParses(t *testing.T) {
|
||||
root := catalogueRoot(t)
|
||||
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil || len(found) == 0 {
|
||||
t.Fatalf("no manifests under %s: %v", root, err)
|
||||
}
|
||||
named, routed := 0, 0
|
||||
for _, p := range found {
|
||||
m, err := ReadTreeManifest(root, filepath.Dir(p))
|
||||
if err != nil {
|
||||
t.Errorf("%s: %v", filepath.Base(filepath.Dir(p)), err)
|
||||
continue
|
||||
}
|
||||
for _, l := range m.Listens {
|
||||
if l.Name != "" {
|
||||
named++
|
||||
}
|
||||
}
|
||||
for port := range RoutedPorts(m) {
|
||||
_ = port
|
||||
routed++
|
||||
}
|
||||
}
|
||||
t.Logf("%d manifests, %d named endpoints, %d routed endpoints resolved", len(found), named, routed)
|
||||
if named == 0 {
|
||||
t.Fatal("no endpoint in the catalogue is named, so this proved nothing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestNoCatalogueManifestNamesAnInstallation is ADR 0112's check, run over the real catalogue: no
|
||||
// definition names a domain or a public address the mesh acts on, and every value that must for now
|
||||
// carries its reason (novox/hq ADR 0155, issue 134). The list it prints is the one that shrinks.
|
||||
func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
|
||||
root := catalogueRoot(t)
|
||||
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil || len(found) == 0 {
|
||||
t.Fatalf("no manifests under %s: %v", root, err)
|
||||
}
|
||||
var named []string
|
||||
for _, p := range found {
|
||||
m, err := ReadTreeManifest(root, filepath.Dir(p))
|
||||
if err != nil {
|
||||
t.Errorf("%s: %v", p, err)
|
||||
continue
|
||||
}
|
||||
named = append(named, InstallationProblems(m)...)
|
||||
}
|
||||
if len(named) > 0 {
|
||||
t.Fatalf("%d value(s) name an installation:\n %s", len(named), strings.Join(named, "\n "))
|
||||
}
|
||||
}
|
||||
|
||||
// TestEveryCatalogueModuleDeclaresItsData is ADR 0233's check over the real catalogue (it replaced ADR
|
||||
// 0214's store list): every provider that grants says what it keeps for its consumers, nothing writes a
|
||||
// backup line by hand, every directory a container writes is declared, and every irreplaceable item is
|
||||
// backed up — so a store added is a store backed up, without a list of stores to keep in step.
|
||||
func TestEveryCatalogueModuleDeclaresItsData(t *testing.T) {
|
||||
root := catalogueRoot(t)
|
||||
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil || len(found) == 0 {
|
||||
t.Fatalf("no manifests under %s: %v", root, err)
|
||||
}
|
||||
shelf := Shelf{}
|
||||
granting := 0
|
||||
for _, p := range found {
|
||||
m, err := ReadTreeManifest(root, filepath.Dir(p))
|
||||
if err != nil {
|
||||
continue // TestEveryCatalogueManifestParses says why
|
||||
}
|
||||
if len(m.Grants) > 0 {
|
||||
granting++
|
||||
}
|
||||
shelf[m.Module] = m
|
||||
}
|
||||
for _, problem := range DataProblems(shelf) {
|
||||
t.Error(problem)
|
||||
}
|
||||
if granting == 0 {
|
||||
t.Fatal("no module in the catalogue grants a provision, so this proved nothing")
|
||||
}
|
||||
}
|
||||
|
||||
// TestEveryCatalogueIdentityFitsWhatItRequires is ADR 0225's check over the real catalogue: every
|
||||
// module's identity, on the longest machine name, fits the bound of every provision it wants. An
|
||||
// overflow fails here, in the pull request that introduces it, rather than on the provider's machine
|
||||
// the first time a real machine's name meets the module's (issue 263).
|
||||
func TestEveryCatalogueIdentityFitsWhatItRequires(t *testing.T) {
|
||||
root := catalogueRoot(t)
|
||||
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
|
||||
if err != nil || len(found) == 0 {
|
||||
t.Fatalf("no manifests under %s: %v", root, err)
|
||||
}
|
||||
shelf := Shelf{}
|
||||
for _, p := range found {
|
||||
m, err := ReadTreeManifest(root, filepath.Dir(p))
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", p, err)
|
||||
}
|
||||
shelf[m.Module] = m
|
||||
}
|
||||
for _, p := range IdentityProblems(shelf, DefaultLongestMachine) {
|
||||
t.Error(p)
|
||||
}
|
||||
// The night it was found: the resolver provision bounds nothing, and the object store still 20.
|
||||
if dns, ok := shelf["dnsmasq"]; ok {
|
||||
if b := dns.IdentityBoundOf("wildcard-resolution"); b.Bounded() {
|
||||
t.Errorf("the resolver provision bounds its consumers' identities: %+v", b)
|
||||
}
|
||||
}
|
||||
if store, ok := shelf["minio"]; ok {
|
||||
if b := store.IdentityBoundOf("s3-bucket"); b.Max != 20 {
|
||||
t.Errorf("the object store's access key is not bounded at 20: %+v", b)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// readManifest is one manifest of the catalogue as ReadTreeManifest read it.
|
||||
type readManifest struct {
|
||||
path string // the module.json
|
||||
manifest Manifest
|
||||
err error
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/internal/beside"
|
||||
)
|
||||
|
||||
// **A machine trusts the mesh's authority because a module put its root there** (novox/hq ADR
|
||||
// 0147, issue 129). The module carries a shell script and a unit, and both are worthless unless
|
||||
// the mesh fills in where the authority is — which is the one thing about it the manifest cannot
|
||||
// state, because the authority's address is a fact about the mesh and not about the module.
|
||||
//
|
||||
// So what is checked here is the rendering, not the parsing: the script the machine will run
|
||||
// names the authority it was bound to, and the unit runs that script both ways. The verification
|
||||
// itself — a plain client trusting an internal name on a machine holding this, and failing on one
|
||||
// that does not — is the lab's, and cannot be had here.
|
||||
func TestCaTrustRendersTheAuthorityItWasBoundTo(t *testing.T) {
|
||||
m, err := ReadTreeManifest(beside.Dir(t, "mesh-catalog"), filepath.Join(beside.Catalogue(t), "ca-trust"))
|
||||
if err != nil {
|
||||
t.Fatalf("the trust module does not parse:\n%v", err)
|
||||
}
|
||||
|
||||
r := Resolution{
|
||||
Node: "workstation",
|
||||
Modules: []Manifest{m},
|
||||
Needs: []Needed{{
|
||||
Name: "internal-acme-ca", From: "anchor", At: "anchor.internal", For: "ca-trust",
|
||||
Serves: map[string]any{
|
||||
"port": float64(9000), "path": "/acme/acme/directory", "roots": "/roots.pem",
|
||||
},
|
||||
}},
|
||||
}
|
||||
out, err := r.Declaration(Rendering{})
|
||||
if err != nil {
|
||||
t.Fatalf("the trust module could not be composed for a machine: %v", err)
|
||||
}
|
||||
|
||||
script := fileNamed(out, "ca-trust.anchor")
|
||||
if script == nil {
|
||||
t.Fatalf("nothing writes the script the unit runs: %v", out)
|
||||
}
|
||||
body, _ := script["content"].(string)
|
||||
if !strings.Contains(body, "https://anchor.internal:9000/roots.pem") {
|
||||
t.Errorf("the script does not fetch from the authority it was bound to:\n%s", body)
|
||||
}
|
||||
if script["mode"] != "0755" {
|
||||
t.Errorf("the script is written %v, which systemd cannot execute", script["mode"])
|
||||
}
|
||||
|
||||
unit := fileNamed(out, "ca-trust.unit")
|
||||
if unit == nil {
|
||||
t.Fatalf("no unit: %v", out)
|
||||
}
|
||||
text, _ := unit["content"].(string)
|
||||
// Both halves. A unit that only installs the anchor leaves a machine trusting an authority
|
||||
// nobody assigned it to any more, which is the half issue 129 asked for by name.
|
||||
for _, want := range []string{
|
||||
"ExecStart=" + script["path"].(string) + " install",
|
||||
"ExecStop=" + script["path"].(string) + " remove",
|
||||
"RemainAfterExit=yes",
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("the unit does not say %q:\n%s", want, text)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// CheckerSeat is the seat mesh-delivery-checker holds.
|
||||
//
|
||||
// Deprecated: a whole seat's name is mesh-contracts' (seats.MeshDeliveryChecker); kept for its importers until their
|
||||
// next change takes it from there (testdata/seat-names.txt).
|
||||
@@ -1,5 +1,9 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Chosen is the provider somebody named for a provision: the module, and the node it runs on. Both,
|
||||
// always (novox/hq #258) — a provision comes from a module, and the same module on two machines is
|
||||
// two answers, so neither half alone says which. Module is empty only on a record made before this
|
||||
@@ -16,6 +20,76 @@ func (c Chosen) String() string {
|
||||
return c.Node + "/" + c.Module
|
||||
}
|
||||
|
||||
// matches is whether this provider is the one chosen.
|
||||
func (c Chosen) matches(p Provider) bool {
|
||||
return p.Node == c.Node && (c.Module == "" || p.Module == c.Module)
|
||||
}
|
||||
|
||||
// among is every offered provider the choice names — one, when the choice is whole.
|
||||
func (c Chosen) among(where []Provider) []Provider {
|
||||
var out []Provider
|
||||
for _, p := range where {
|
||||
if c.matches(p) {
|
||||
out = append(out, p)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// nameOf is how a refusal names a provider: the node and the module on it.
|
||||
func nameOf(p Provider) string {
|
||||
return Chosen{Node: p.Node, Module: p.Module}.String()
|
||||
}
|
||||
|
||||
// providerNames is every provider named, sorted, for a refusal to list.
|
||||
func providerNames(where []Provider) []string {
|
||||
out := make([]string, 0, len(where))
|
||||
for _, p := range where {
|
||||
out = append(out, nameOf(p))
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// providersHere is which modules in this node's own set offer a provision, sorted.
|
||||
func providersHere(catalogue map[string]Manifest, here func(string) bool, want string) []string {
|
||||
var out []string
|
||||
for name, m := range catalogue {
|
||||
if !here(name) {
|
||||
continue
|
||||
}
|
||||
for _, o := range m.Offers() {
|
||||
if o == want {
|
||||
out = append(out, name)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// servedByOne is what one provider beside the consumer says a consumer needs to know, or nothing.
|
||||
//
|
||||
// Serving is *whether* a need is created at all when the provider is on this same machine (novox/hq
|
||||
// 04-ISSUES/038's sibling): a need never created is a binding the consumer never gets. The manifest
|
||||
// alone answers that; the values are settled later, with the node's settings.
|
||||
func servedByOne(m Manifest, want string) map[string]any {
|
||||
if _, ok := m.Serves[want]; ok {
|
||||
return ServedOn(m, want, nil)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sharedByOne is the own secret that provider names as its credential (ADR 0158), or "" when it
|
||||
// gives each consumer its own.
|
||||
func sharedByOne(m Manifest, want string) string {
|
||||
if own, shared := m.SharedCredentialOf(want); shared {
|
||||
return own
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func oneOf(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A provider and its consumer on ONE machine, which is what ADR 0066's anchor is.
|
||||
//
|
||||
// **Every fault in this file is the same shape: the same-node path diverging from the cross-node
|
||||
// one.** A provider on another machine is walked by the control plane — its served facts are
|
||||
// re-derived from its manifest with that machine's port assignments and settled with that node's
|
||||
// settings layers — and only then handed to the consumer. A provider on the consumer's OWN machine
|
||||
// never passes through that walk, so every step of it had to be repeated here, and each step that
|
||||
// was not is a promise the co-located arrangement quietly breaks.
|
||||
//
|
||||
// 04-ISSUES/038 was the first of them (the port). These are the rest.
|
||||
|
||||
// A root certificate, as a certificate authority serves one: a real, self-signed one made for these tests (its key
|
||||
// thrown away), because the one setting that may hold lines must parse as a certificate (novox/hq issue 339).
|
||||
const servedRoot = `-----BEGIN CERTIFICATE-----
|
||||
MIIBPjCB8aADAgECAhRZG3p93hUUB5bz00uxhYo/nJnTQjAFBgMrZXAwFDESMBAG
|
||||
A1UEAwwJdGVzdCByb290MCAXDTI2MTAwODIyMjE0NloYDzIxMjYwOTE0MjIyMTQ2
|
||||
WjAUMRIwEAYDVQQDDAl0ZXN0IHJvb3QwKjAFBgMrZXADIQA0pt/ld+W0MXwBhPfO
|
||||
cuAt56kIW6Qcn+4vqWpuvHTiqaNTMFEwHQYDVR0OBBYEFIjgaLk4OVGIOeZQiqnN
|
||||
p9vhciFjMB8GA1UdIwQYMBaAFIjgaLk4OVGIOeZQiqnNp9vhciFjMA8GA1UdEwEB
|
||||
/wQFMAMBAf8wBQYDK2VwA0EAl9uWeSM2XAV8u0reyV3BLRxNVik+4FCRO1QKPs2k
|
||||
IlB1rK9oAOYManH+VFuBMI/JJ31ajSti81q4E0CSw8r5DQ==
|
||||
-----END CERTIFICATE-----
|
||||
`
|
||||
|
||||
// stepCA is an internal ACME authority: it answers `acme-ca` from anywhere in the mesh, and what a
|
||||
// consumer must know is the directory URL and the root to trust. **The root is not knowable when
|
||||
// the module is written** — it exists only once the CA has been initialised — so the manifest
|
||||
// declares the key and the operator supplies the value as a setting, which is exactly the shape the
|
||||
// cross-node path settles and the same-node path did not.
|
||||
func stepCA() Manifest {
|
||||
return Manifest{
|
||||
Module: "step-ca", Version: "1",
|
||||
Provides: FromAnywhere("acme-ca"),
|
||||
Listens: []Listening{{Port: 9000, Protocol: "tcp", From: FromMesh}},
|
||||
Serves: map[string]map[string]any{"acme-ca": {
|
||||
"directory": "https://anchor.internal/acme/acme/directory",
|
||||
// Declared empty: a manifest is the same on every mesh, and this mesh's root is not.
|
||||
"root": "",
|
||||
}},
|
||||
Resources: []map[string]any{{
|
||||
"id": "server", "type": "container", "name": "step-ca", "ports": []any{"9000"},
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
// stepCAWithItsRoot is the authority whose root the operator supplies: the served key asks for it as
|
||||
// `${setting:root}` (ADR 0155), the only way a setting reaches what a provider serves since a layer
|
||||
// stopped replacing a served key (novox/hq issue 696; before it, the empty `root` was replaced whole).
|
||||
func stepCAWithItsRoot() Manifest {
|
||||
m := stepCA()
|
||||
m.Serves["acme-ca"]["root"] = "${setting:root}"
|
||||
return m
|
||||
}
|
||||
|
||||
// routeProxy issues from that authority, so it must be told the root to verify it with.
|
||||
func routeProxy() Manifest {
|
||||
return Manifest{
|
||||
Module: "route-proxy", Version: "1",
|
||||
Requires: []string{"acme-ca"},
|
||||
Provides: FromAnywhere("route"),
|
||||
Binds: map[string]string{"acme-ca": "/var/lib/route-proxy/acme-ca.json"},
|
||||
Receives: map[string]string{"route": "/var/lib/route-proxy/routes.json"},
|
||||
Resources: []map[string]any{{
|
||||
"id": "bundle", "type": "file", "path": "/var/lib/route-proxy/ca.pem", "mode": "0644",
|
||||
"content": "${bound:acme-ca:root}",
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
// A co-located provider's served VALUES reach its consumer, not just its served keys.
|
||||
//
|
||||
// The mesh walks a provider on ANOTHER machine and settles what it serves with that node's settings
|
||||
// layers before offering it (cmd/mesh-controller plan.go, theRestOfTheMesh). A provider on the
|
||||
// consumer's own machine was never settled at all: resolve.go's servedHere and declaration.go's
|
||||
// here() both read the manifest and stop there. So a served value the operator supplied — the one
|
||||
// kind of value a manifest cannot carry, because it is different on every mesh — arrived as the
|
||||
// manifest's empty default.
|
||||
//
|
||||
// The consequence is silent and total: route-proxy wrote an empty CA bundle, fell back to the
|
||||
// system trust store, could not verify the internal authority, and no certificate was ever issued.
|
||||
func TestACoLocatedProvidersServedValuesReachItsConsumer(t *testing.T) {
|
||||
r, err := Resolve(shelf(stepCAWithItsRoot(), routeProxy()),
|
||||
[]string{"step-ca", "route-proxy"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// What the operator set on the provider, on this node — the CA's root, which only exists once
|
||||
// the CA has been initialised.
|
||||
out, err := r.Declaration(Rendering{Settings: SettingsBy{
|
||||
"step-ca": {{From: "the operator", Values: map[string]any{"root": servedRoot}}},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
bundle := fileNamed(out, "route-proxy.bundle")
|
||||
if bundle == nil {
|
||||
t.Fatalf("the consumer was given no bundle at all: %v", out)
|
||||
}
|
||||
if got := bundle["content"]; got != servedRoot {
|
||||
t.Errorf("the co-located consumer was not told the root it must trust:\n got %q\nwant %q",
|
||||
got, servedRoot)
|
||||
}
|
||||
}
|
||||
|
||||
// And the binding file says the same thing, for a consumer that reads the binding rather than a
|
||||
// substituted placeholder. The two are one fact and must not be able to disagree.
|
||||
func TestACoLocatedConsumersBindingCarriesTheSettledValues(t *testing.T) {
|
||||
r, err := Resolve(shelf(stepCAWithItsRoot(), routeProxy()),
|
||||
[]string{"step-ca", "route-proxy"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := r.Declaration(Rendering{
|
||||
Settings: SettingsBy{
|
||||
"step-ca": {{From: "the operator", Values: map[string]any{"root": servedRoot}}},
|
||||
},
|
||||
// And the machine moved the provider's port while it was at it, so both halves of the
|
||||
// cross-node derivation are exercised at once.
|
||||
Ports: map[string]map[int]int{"step-ca": {9000: 19000}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
binding := fileNamed(out, "route-proxy.bound-acme-ca")
|
||||
if binding == nil {
|
||||
t.Fatalf("the consumer was given no binding at all: %v", out)
|
||||
}
|
||||
var said struct {
|
||||
Serves map[string]any `json:"serves"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(binding["content"].(string)), &said); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if said.Serves["root"] != servedRoot {
|
||||
t.Errorf("the binding does not carry the settled root: %q", said.Serves["root"])
|
||||
}
|
||||
if port, _ := asPort(said.Serves["port"]); port != 19000 {
|
||||
t.Errorf("the binding does not carry the port the machine publishes: %v", said.Serves["port"])
|
||||
}
|
||||
}
|
||||
|
||||
// A provider PULLED IN rather than assigned is settled the same way.
|
||||
//
|
||||
// The resolver's same-node need is built during the walk, from whichever modules had been chosen by
|
||||
// the time the requirement came up — so which path a co-located binding took depended on the order
|
||||
// a person happened to assign things in. Settling after the closure is known removes that: both
|
||||
// orders now produce the same file.
|
||||
func TestACoLocatedProviderIsSettledWhicheverWayItWasPulledIn(t *testing.T) {
|
||||
for _, assigned := range [][]string{
|
||||
{"step-ca", "route-proxy"},
|
||||
{"route-proxy", "step-ca"},
|
||||
} {
|
||||
r, err := Resolve(shelf(stepCAWithItsRoot(), routeProxy()), assigned, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := r.Declaration(Rendering{Settings: SettingsBy{
|
||||
"step-ca": {{From: "the operator", Values: map[string]any{"root": servedRoot}}},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
bundle := fileNamed(out, "route-proxy.bundle")
|
||||
if bundle == nil || bundle["content"] != servedRoot {
|
||||
t.Errorf("assigned %v: the consumer was not told the root", assigned)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A same-node contribution names the port the MACHINE publishes, not the one the module declared.
|
||||
//
|
||||
// 04-ISSUES/038 fixed this for what a consumer is TOLD about a provider. This is the other
|
||||
// direction: what a workload TELLS the provider about itself. gitea declares a bare container port
|
||||
// 3000, the mesh publishes it as 20000:3000 — and gitea's route contribution still said 3000, so
|
||||
// the proxy beside it dialled a port nothing listens on and answered 502 for every request.
|
||||
//
|
||||
// The redirect uses the CONTRIBUTING module's assignment: the port belongs to the workload, and
|
||||
// using the provider's map would move it to wherever the proxy happens to be published.
|
||||
func TestASameNodeContributionNamesThePortTheMachinePublishes(t *testing.T) {
|
||||
gitea := Manifest{
|
||||
Module: "gitea", Version: "1",
|
||||
Listens: []Listening{{Port: 3000, Protocol: "tcp", From: FromMesh}},
|
||||
Contributes: map[string]map[string]any{"route": {"label": "git", "port": 3000}},
|
||||
Resources: []map[string]any{{
|
||||
"id": "server", "type": "container", "name": "gitea", "ports": []any{"3000"},
|
||||
}},
|
||||
}
|
||||
r, err := Resolve(shelf(gitea, routeProxy(), stepCA()),
|
||||
[]string{"gitea", "route-proxy", "step-ca"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := r.Declaration(Rendering{
|
||||
// The machine put gitea's 3000 on 20000, and published it that way.
|
||||
Ports: map[string]map[int]int{"gitea": {3000: 20000}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Where the workload is actually published.
|
||||
server := fileNamed(out, "gitea.server")
|
||||
if published := strings.Join(asStrings(server["ports"]), ","); published != "20000:3000" {
|
||||
t.Fatalf("the workload was not published on the assigned port: %v", server["ports"])
|
||||
}
|
||||
|
||||
// What the proxy beside it was told to dial: the SAME port.
|
||||
routes := fileNamed(out, "route-proxy.received-route")
|
||||
if routes == nil {
|
||||
t.Fatalf("the proxy was given no routes file at all: %v", out)
|
||||
}
|
||||
var given struct {
|
||||
Given []Contribution `json:"given"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(routes["content"].(string)), &given); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(given.Given) != 1 {
|
||||
t.Fatalf("expected one route, got %v", given.Given)
|
||||
}
|
||||
port, ok := asPort(given.Given[0].Values["port"])
|
||||
if !ok || port != 20000 {
|
||||
t.Errorf("the proxy was told to dial a port nothing listens on: %v",
|
||||
given.Given[0].Values["port"])
|
||||
}
|
||||
}
|
||||
|
||||
// A contribution from ANOTHER machine is left exactly as it was.
|
||||
//
|
||||
// Its port belongs to that machine's assignment, which this node's map knows nothing about —
|
||||
// applying this node's map to it would move a remote workload's port to wherever a local module of
|
||||
// the same name happens to be published, which is worse than the fault being fixed.
|
||||
func TestAContributionFromAnotherMachineKeepsItsOwnPort(t *testing.T) {
|
||||
r, err := Resolve(shelf(routeProxy(), stepCA()),
|
||||
[]string{"route-proxy", "step-ca"}, reachable(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := r.Declaration(Rendering{
|
||||
// A same-named module on this machine, moved. The grant below is a laptop's, and must not
|
||||
// be dragged along with it.
|
||||
Ports: map[string]map[int]int{"gitea": {3000: 20000}},
|
||||
Grants: []Grant{{
|
||||
Provision: "route", Consumer: "laptop", From: "gitea", At: "laptop.internal",
|
||||
Values: map[string]any{"name": "git.example", "port": 3000},
|
||||
}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
routes := fileNamed(out, "route-proxy.received-route")
|
||||
var given struct {
|
||||
Given []Contribution `json:"given"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(routes["content"].(string)), &given); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(given.Given) != 1 {
|
||||
t.Fatalf("expected one route, got %v", given.Given)
|
||||
}
|
||||
if port, _ := asPort(given.Given[0].Values["port"]); port != 3000 {
|
||||
t.Errorf("another machine's contribution was rewritten with this machine's ports: %v",
|
||||
given.Given[0].Values["port"])
|
||||
}
|
||||
}
|
||||
|
||||
// The cross-node half of the same fault: a grant assembled on the providing machine carries the
|
||||
// consumer's declared port until the CONSUMER's machine's assignments are applied to it. The
|
||||
// declaration layer cannot do it — those assignments are not this machine's, which is the line the
|
||||
// test above holds — so the grant layer is handed them and does it there.
|
||||
func TestAContributionIsRedirectedToWhereItsOwnMachinePublishedIt(t *testing.T) {
|
||||
published := map[int]int{3000: 20000}
|
||||
|
||||
got := AtPublishedPort(map[string]any{"name": "git.example.tld", "port": 3000}, "forge", published)
|
||||
if port, _ := asPort(got["port"]); port != 20000 {
|
||||
t.Errorf("the proxy would dial a port that machine never published: %v", got["port"])
|
||||
}
|
||||
if got["name"] != "git.example.tld" {
|
||||
t.Errorf("redirecting the port disturbed the rest of the contribution: %v", got)
|
||||
}
|
||||
|
||||
// Idempotent, and silent about a port nobody moved: a module that pinned its own mapping has no
|
||||
// assignment, and must come back exactly as it was written.
|
||||
again := AtPublishedPort(got, "forge", published)
|
||||
if port, _ := asPort(again["port"]); port != 20000 {
|
||||
t.Errorf("applying it twice moved the port again: %v", again["port"])
|
||||
}
|
||||
pinned := AtPublishedPort(map[string]any{"port": 9070}, "office", published)
|
||||
if port, _ := asPort(pinned["port"]); port != 9070 {
|
||||
t.Errorf("a port the mesh never assigned was rewritten: %v", pinned["port"])
|
||||
}
|
||||
}
|
||||
|
||||
func asStrings(v any) []string {
|
||||
listed, ok := v.([]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(listed))
|
||||
for _, one := range listed {
|
||||
out = append(out, strings.TrimSpace(plainly(one)))
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -3,9 +3,35 @@ package catalogue
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Collections (novox/hq ADR 0349 clause 13, design 57 L1, issue 684): a provision with many providers.
|
||||
//
|
||||
// **A collector requires a collection; a contributor provides an item from a template.** A module taking additions
|
||||
// from others names a collection in a file of its own, `${collected:<collection>}`, and declares it under
|
||||
// `collections`, with the comment its tool writes a line in. Each contributor ships its item as a template of its
|
||||
// own repository and declares it under `items`: `{"collection": …, "template": "files/<name>"}`. Its build packs the
|
||||
// template with the module's others, and the mesh holds its text as it holds a file's (template.go).
|
||||
//
|
||||
// **The node's provisioner client fills both.** An item is filled on the node by the client of the node its
|
||||
// contributor is placed on, from the same owners as a file, and recorded in mesh-provisioner_collect; a collector's
|
||||
// file is filled by the same client from every item of its collection recorded for that node, so an item goes when
|
||||
// its contributor leaves. So a module contributing an item is a moved module (handed.go), and the controller renders
|
||||
// no item: the plan of a contributor names its items and their templates, and the plan of a collector names each
|
||||
// collection it reads and its comment. Neither carries a contributor's text.
|
||||
//
|
||||
// **The three old namespaces read as collections.** `${shell:<shell>:<slot>}`, `${environment:<format>}` and
|
||||
// `${contribution:<seat>:<kind>}` name, in a handed collector's file, the collection of the same name —
|
||||
// `shell:zsh:normal` — with the comment the controller wrote before each module's piece. An item is contributed to
|
||||
// one by that name, by a module the old form would have let contribute it. Each file is filled one way, never
|
||||
// both (ADR 0349 clause 17): a handed collector is filled from items alone and an unhanded one from the old forms
|
||||
// alone, and a piece of the other form on the same machine is said as not placed, never dropped quietly.
|
||||
|
||||
// Item is one item a module contributes to a collection: its template in the module's repository, or, in the
|
||||
// manifest the mesh holds, the template's text, read back from the build's templates archive like a file's.
|
||||
//
|
||||
@@ -33,9 +59,39 @@ func ItemKey(collection, name string) string {
|
||||
|
||||
func (it Item) key() string { return ItemKey(it.Collection, it.Name) }
|
||||
|
||||
// order is the item's place among the items of one placeholder: its `order` attribute, or OrderDefault.
|
||||
func (it Item) order() int {
|
||||
if n, err := strconv.Atoi(it.Attributes[OrderAttribute]); err == nil && n >= 0 && n <= 99 {
|
||||
return n
|
||||
}
|
||||
return OrderDefault
|
||||
}
|
||||
|
||||
// OrderAttribute is the attribute that orders items.
|
||||
const OrderAttribute = "order"
|
||||
|
||||
// matches is whether the item's attributes carry every pair of the selector.
|
||||
func (it Item) matches(where map[string]string) bool {
|
||||
for k, v := range where {
|
||||
if got, has := it.Attributes[k]; !has || got != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// itemsInOrder is the module's items by order, then name; the order the old form placed a module's pieces in.
|
||||
func itemsInOrder(items []Item) []Item {
|
||||
out := append([]Item(nil), items...)
|
||||
sort.SliceStable(out, func(a, b int) bool {
|
||||
if oa, ob := out[a].order(), out[b].order(); oa != ob {
|
||||
return oa < ob
|
||||
}
|
||||
return out[a].Name < out[b].Name
|
||||
})
|
||||
return out
|
||||
}
|
||||
|
||||
// Collection is one collection a module collects, and the comment its tool writes a line in: each contributor's item
|
||||
// is introduced by `<comment> <contributor>`, and by nothing without one.
|
||||
type Collection struct {
|
||||
@@ -52,6 +108,324 @@ const CollectedNamespace = "collected"
|
||||
// `${collected:environment:systemd}`, which is not a selector.
|
||||
const EnvironmentCollection = "environment"
|
||||
|
||||
// oldCollectionNamespaces are the namespaces read as collections of their own name.
|
||||
var oldCollectionNamespaces = []string{"shell", "environment", "contribution"}
|
||||
|
||||
var (
|
||||
// ofCollected is where a collector places a collection. Loose inside the braces, so a misspelt name is refused.
|
||||
ofCollected = regexp.MustCompile(`\$\{collected:([^}]*)\}`)
|
||||
// collectionName is words joined by colons, so an old namespace's placeholder names its collection as it reads.
|
||||
collectionName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*(:[a-z0-9][a-z0-9-]*)*$`)
|
||||
// itemName, attributeKey and attributeValue are the words an item's name and attributes are made of; none holds
|
||||
// a colon, a comma or an equals sign, which tell a selector from a collection's name.
|
||||
itemName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||
attributeKey = regexp.MustCompile(`^[a-z][a-z0-9-]*$`)
|
||||
attributeValue = regexp.MustCompile(`^[a-z0-9][a-z0-9_.-]*$`)
|
||||
)
|
||||
|
||||
// splitSelector is a placeholder's name without its selector, and the selector: `widgets:bar=top,place=left` is
|
||||
// `widgets` and `bar=top,place=left`. A collection's name holds no `=`, so the selector begins at the last colon
|
||||
// before the first one.
|
||||
func splitSelector(name string) (collection, selector string) {
|
||||
eq := strings.Index(name, "=")
|
||||
if eq < 0 {
|
||||
return name, ""
|
||||
}
|
||||
colon := strings.LastIndex(name[:eq], ":")
|
||||
if colon < 0 {
|
||||
return name, ""
|
||||
}
|
||||
return name[:colon], name[colon+1:]
|
||||
}
|
||||
|
||||
// selectorOf is a selector's pairs, or why it is none.
|
||||
func selectorOf(selector string) (map[string]string, error) {
|
||||
out := map[string]string{}
|
||||
if selector == "" {
|
||||
return out, nil
|
||||
}
|
||||
for _, pair := range strings.Split(selector, ",") {
|
||||
k, v, ok := strings.Cut(pair, "=")
|
||||
switch {
|
||||
case !ok || !attributeKey.MatchString(k) || !attributeValue.MatchString(v):
|
||||
return nil, fmt.Errorf("%q in %q is not <attribute>=<value>", pair, selector)
|
||||
case out[k] != "":
|
||||
return nil, fmt.Errorf("%q names %s twice", selector, k)
|
||||
}
|
||||
out[k] = v
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// collectionOfPlaceholder is the collection a placeholder's namespace and key name, and whether they name one.
|
||||
func collectionOfPlaceholder(namespace, key string) (string, bool) {
|
||||
key, _ = splitSelector(key)
|
||||
switch {
|
||||
case namespace == CollectedNamespace && strings.HasPrefix(key, EnvironmentCollection+":"):
|
||||
// The environment is one collection, and the rest of the name picks the rendering (issue 795).
|
||||
return EnvironmentCollection, true
|
||||
case namespace == CollectedNamespace:
|
||||
return key, true
|
||||
case namespace == "environment":
|
||||
return EnvironmentCollection, true
|
||||
case slices.Contains(oldCollectionNamespaces, namespace):
|
||||
return namespace + ":" + key, true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// oldCollection is an old namespace's collection by its name: its comment, or why no item may be contributed to it.
|
||||
// ok is false for a name that is not an old namespace's.
|
||||
//
|
||||
// forItem is whether an item is to be contributed to it: an item may be contributed to a kind received as data,
|
||||
// which a collector may place as a collection only where its shape says its pieces are text (design 57 L3).
|
||||
func oldCollection(name string, forItem bool) (comment string, old bool, wrong error) {
|
||||
namespace, key, _ := strings.Cut(name, ":")
|
||||
if !slices.Contains(oldCollectionNamespaces, namespace) {
|
||||
return "", false, nil
|
||||
}
|
||||
switch namespace {
|
||||
case "shell":
|
||||
target, slot, two := strings.Cut(key, ":")
|
||||
if !two || !oneOf(contributionTargets(), target) || !oneOf(knownSlots, slot) {
|
||||
return "", true, fmt.Errorf("%s is no slot: shell code is shell:<shell>:<slot>, the shell one of %s, "+
|
||||
"the session's file one of %s or the power moment one of %s, and the slot one of %s", name,
|
||||
strings.Join(knownShells, ", "), strings.Join(sessionFiles, ", "), strings.Join(powerMoments, ", "),
|
||||
strings.Join(knownSlots, ", "))
|
||||
}
|
||||
// shellCode's line before each module's code.
|
||||
return "#", true, nil
|
||||
case "environment":
|
||||
if key == "" {
|
||||
// The collection itself, which an item is contributed to; a placeholder names a rendering.
|
||||
return "#", true, nil
|
||||
}
|
||||
if key != EnvironmentPOSIX && key != EnvironmentSystemd {
|
||||
return "", true, fmt.Errorf("%s is no format: the environment is environment:%s or environment:%s",
|
||||
name, EnvironmentPOSIX, EnvironmentSystemd)
|
||||
}
|
||||
return "#", true, nil
|
||||
}
|
||||
seat, kind, _ := strings.Cut(key, ":")
|
||||
s, r, ok := receivable(seat, kind)
|
||||
switch {
|
||||
case !ok && s.Name == "":
|
||||
return "", true, fmt.Errorf("%s names the seat %q, which the mesh does not define", name, seat)
|
||||
case !ok:
|
||||
return "", true, fmt.Errorf("%s: %s; %s", name, s.Name, kindsOf(s))
|
||||
case r.Shape != nil && (forItem || collectableAsText(r)):
|
||||
return r.Comment, true, nil
|
||||
case r.Shape != nil:
|
||||
return "", true, fmt.Errorf("%s is received by %s as data in its own shape, which no item of text is "+
|
||||
"(novox/hq ADR 0255)", name, s.Name)
|
||||
case r.Dirs:
|
||||
return "", true, fmt.Errorf("%s names its contributors' directories, which the holder reads as paths "+
|
||||
"the controller fills; it is not yet a collection (design 57 L1)", name)
|
||||
case s.Name != canonicalSeat(seat):
|
||||
return "", true, fmt.Errorf("%s names the seat by another name; a collection names it %s", name, s.Name)
|
||||
}
|
||||
return r.Comment, true, nil
|
||||
}
|
||||
|
||||
// placerOfOld is the seat whose holder alone may collect an old namespace's collection (ADR 0203, 0204, 0208,
|
||||
// 0211, 0212): the same seat that places it the old way.
|
||||
func placerOfOld(name string) string {
|
||||
namespace, key, _ := strings.Cut(name, ":")
|
||||
switch namespace {
|
||||
case "shell":
|
||||
target, _, _ := strings.Cut(key, ":")
|
||||
return placerOf(target)
|
||||
case "environment":
|
||||
return EnvironmentSeat
|
||||
}
|
||||
seat, _, _ := strings.Cut(key, ":")
|
||||
return canonicalSeat(seat)
|
||||
}
|
||||
|
||||
// collectionProblems is what is wrong with a module's collections and items, from the manifest alone.
|
||||
func collectionProblems(m Manifest) []string {
|
||||
var problems []string
|
||||
declared := map[string]bool{}
|
||||
for i, c := range m.Collections {
|
||||
switch {
|
||||
case !collectionName.MatchString(c.Name):
|
||||
problems = append(problems, fmt.Sprintf("%s's collection %d is %q, which is not a collection's name: "+
|
||||
"words of lower-case letters, digits and hyphens, joined by colons", m.Module, i+1, c.Name))
|
||||
case declared[c.Name]:
|
||||
problems = append(problems, fmt.Sprintf("%s collects %s twice", m.Module, c.Name))
|
||||
}
|
||||
if _, old, _ := oldCollection(c.Name, false); old {
|
||||
problems = append(problems, fmt.Sprintf("%s collects %s, which is read from its placeholder: a module "+
|
||||
"places it as ${%s} where it holds the seat that places it, and declares nothing", m.Module, c.Name,
|
||||
c.Name))
|
||||
}
|
||||
if strings.ContainsAny(c.Comment, "\n\r") {
|
||||
problems = append(problems, fmt.Sprintf("%s's collection %s has a comment holding a line break",
|
||||
m.Module, c.Name))
|
||||
}
|
||||
declared[c.Name] = true
|
||||
}
|
||||
// A collection declared and placed nowhere is a word read by nothing; judged where every file's text is here.
|
||||
if !m.Templated() {
|
||||
named := map[string]bool{}
|
||||
for _, r := range m.Resources {
|
||||
if text, ok := r["content"].(string); ok {
|
||||
for _, f := range ofCollected.FindAllStringSubmatch(text, -1) {
|
||||
base, _ := splitSelector(f[1])
|
||||
named[base] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, c := range m.Collections {
|
||||
if !named[c.Name] && collectionName.MatchString(c.Name) {
|
||||
problems = append(problems, fmt.Sprintf("%s collects %s and no file of its places it as "+
|
||||
"${collected:%s}", m.Module, c.Name, c.Name))
|
||||
}
|
||||
}
|
||||
}
|
||||
contributed := map[string]bool{}
|
||||
for i, it := range m.Items {
|
||||
where := fmt.Sprintf("%s's item %d", m.Module, i+1)
|
||||
if !collectionName.MatchString(it.Collection) {
|
||||
problems = append(problems, fmt.Sprintf("%s is to %q, which is not a collection's name", where,
|
||||
it.Collection))
|
||||
} else if _, old, wrong := oldCollection(it.Collection, true); old && wrong != nil {
|
||||
problems = append(problems, fmt.Sprintf("%s: %v", where, wrong))
|
||||
}
|
||||
switch {
|
||||
case it.Name != "" && !itemName.MatchString(it.Name):
|
||||
problems = append(problems, fmt.Sprintf("%s: the name %q is not a name: lower-case letters, digits and "+
|
||||
"hyphens", where, it.Name))
|
||||
case contributed[it.key()]:
|
||||
problems = append(problems, fmt.Sprintf("%s contributes to %s twice; a module contributes one item to "+
|
||||
"a collection, or several each with a name of its own", m.Module, it.key()))
|
||||
case it.Name == "" && slices.ContainsFunc(m.Items, func(o Item) bool {
|
||||
return o.Collection == it.Collection && o.Name != ""
|
||||
}):
|
||||
problems = append(problems, fmt.Sprintf("%s contributes several items to %s and %s has no name; each "+
|
||||
"of several items has a name of its own", m.Module, it.Collection, where))
|
||||
}
|
||||
contributed[it.key()] = true
|
||||
problems = append(problems, itemAttributeProblems(m.Module, where, it)...)
|
||||
if it.Collection == EnvironmentCollection {
|
||||
problems = append(problems, environmentItemProblems(where, it)...)
|
||||
}
|
||||
switch {
|
||||
case it.Template == "" && it.Content == "":
|
||||
problems = append(problems, fmt.Sprintf("%s names no template: an item is a template of the module's "+
|
||||
"own, \"template\": \"%s/<name>\" (novox/hq ADR 0349 §13)", where, m.TemplatesDir()))
|
||||
case it.Template != "" && it.Content != "":
|
||||
problems = append(problems, fmt.Sprintf("%s names its template and says its content too", where))
|
||||
case it.Template != "":
|
||||
if wrong := itemTemplateProblem(m, it.Template); wrong != "" {
|
||||
problems = append(problems, fmt.Sprintf("%s: %s", where, wrong))
|
||||
}
|
||||
// **Filled by the provisioner's client alone**: the controller renders no item.
|
||||
if !m.Moved {
|
||||
problems = append(problems, fmt.Sprintf("%s: an item is filled by the provisioner's client of the "+
|
||||
"node its contributor is placed on, so a module contributing one is marked moved "+
|
||||
"(\"moved\": true; novox/hq ADR 0349 §13, §17)", where))
|
||||
}
|
||||
}
|
||||
if it.Content != "" {
|
||||
problems = append(problems, itemTextProblems(m.Module, it)...)
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// itemAttributeProblems is what is wrong with an item's attributes and capability. The attributes of an item of a kind
|
||||
// received as data are the fields its shape places by (and `order`), each with a value the shape takes.
|
||||
func itemAttributeProblems(module, where string, it Item) []string {
|
||||
var problems []string
|
||||
for _, k := range sortedKeys(it.Attributes) {
|
||||
v := it.Attributes[k]
|
||||
switch {
|
||||
case !attributeKey.MatchString(k):
|
||||
problems = append(problems, fmt.Sprintf("%s: the attribute %q is not a name: lower-case letters, digits "+
|
||||
"and hyphens", where, k))
|
||||
continue
|
||||
case !attributeFits(it.Collection, k, v):
|
||||
problems = append(problems, fmt.Sprintf("%s: %s=%q is not a value: lower-case letters, digits, hyphens, "+
|
||||
"dots and underscores (an environment variable's name, letters, digits and underscores)", where, k, v))
|
||||
continue
|
||||
}
|
||||
if n, err := strconv.Atoi(v); k == OrderAttribute && (err != nil || n < 0 || n > 99) {
|
||||
problems = append(problems, fmt.Sprintf("%s: order=%q is not a whole number from 0 to 99", where, v))
|
||||
}
|
||||
}
|
||||
if namespace, key, _ := strings.Cut(it.Collection, ":"); namespace == "contribution" {
|
||||
seat, kind, _ := strings.Cut(key, ":")
|
||||
if _, r, ok := receivable(seat, kind); ok && r.Shape != nil {
|
||||
for _, k := range sortedKeys(it.Attributes) {
|
||||
var place *PlaceField
|
||||
for i := range r.Shape.Places {
|
||||
if r.Shape.Places[i].Field == k {
|
||||
place = &r.Shape.Places[i]
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case k == OrderAttribute:
|
||||
case place == nil:
|
||||
problems = append(problems, fmt.Sprintf("%s: %s is placed by %s, and %q is none of them", where,
|
||||
it.Collection, placeFields(r), k))
|
||||
case !oneOf(place.Values, it.Attributes[k]):
|
||||
problems = append(problems, fmt.Sprintf("%s: %s=%s; %s is one of %s", where, k, it.Attributes[k], k,
|
||||
strings.Join(place.Values, ", ")))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case it.IfCapability == "":
|
||||
case !capabilityName.MatchString(it.IfCapability):
|
||||
problems = append(problems, fmt.Sprintf("%s: if-capability %q is not a capability's name", where, it.IfCapability))
|
||||
case !oneOf(KnownCapabilities, it.IfCapability):
|
||||
problems = append(problems, fmt.Sprintf("%s: if-capability %q is one no machine reports; the node-engine "+
|
||||
"detects %s", where, it.IfCapability, strings.Join(KnownCapabilities, ", ")))
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// AttributeFits is whether an item's attribute is of the shape its collection allows: words of lower case, except
|
||||
// the environment's `name`, which is a variable's name (design 57 L3, issue 795). The same rule is mesh-catalog's
|
||||
// provisioner client's, and the inventory's.
|
||||
func attributeFits(collection, k, v string) bool {
|
||||
if collection == EnvironmentCollection && k == "name" {
|
||||
return variableName.MatchString(v)
|
||||
}
|
||||
return attributeKey.MatchString(k) && attributeValue.MatchString(v)
|
||||
}
|
||||
|
||||
// environmentItemProblems is what is wrong with an item of the environment collection: typed by `kind`, a variable
|
||||
// with its `name`, a PATH entry `at` the start or the end, and no attribute besides `order`.
|
||||
func environmentItemProblems(where string, it Item) []string {
|
||||
var problems []string
|
||||
for _, k := range sortedKeys(it.Attributes) {
|
||||
if !slices.Contains([]string{"kind", "name", "at", OrderAttribute}, k) {
|
||||
problems = append(problems, fmt.Sprintf("%s: %s is typed by kind, name and at, and %q is none of them",
|
||||
where, it.Collection, k))
|
||||
}
|
||||
}
|
||||
switch kind := it.Attributes["kind"]; kind {
|
||||
case "variable":
|
||||
if it.Attributes["name"] == "" || it.Attributes["at"] != "" {
|
||||
problems = append(problems, fmt.Sprintf("%s: a variable names itself with name=<NAME> and has no at", where))
|
||||
} else if it.Attributes["name"] == "PATH" {
|
||||
problems = append(problems, fmt.Sprintf("%s: PATH is not a variable; a PATH entry is kind=path", where))
|
||||
}
|
||||
case "path":
|
||||
if it.Attributes["name"] != "" || (it.Attributes["at"] != PathAtStart && it.Attributes["at"] != PathAtEnd) {
|
||||
problems = append(problems, fmt.Sprintf("%s: a PATH entry is at=%s or at=%s and has no name", where,
|
||||
PathAtStart, PathAtEnd))
|
||||
}
|
||||
default:
|
||||
problems = append(problems, fmt.Sprintf("%s: an item of %s is kind=variable or kind=path, not %q", where,
|
||||
it.Collection, kind))
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// itemTemplateProblem is what is wrong with an item's template path, or "".
|
||||
func itemTemplateProblem(m Manifest, name string) string {
|
||||
dir := m.TemplatesDir()
|
||||
@@ -62,3 +436,200 @@ func itemTemplateProblem(m Manifest, name string) string {
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// itemTextProblems is what an item's text may not name: a secret or a bound value, since an item is kept in a
|
||||
// bucket and no secret is (ADR 0349 clause 7), and a collection, since an item is not a collector. And every
|
||||
// placeholder no pass fills, as in a file's content.
|
||||
func itemTextProblems(module string, it Item) []string {
|
||||
var problems []string
|
||||
for _, f := range templatePlaceholder.FindAllStringSubmatch(it.Content, -1) {
|
||||
namespace, key := f[1], f[2]
|
||||
switch _, collection := collectionOfPlaceholder(namespace, key); {
|
||||
case namespace == "secret" || namespace == "bound":
|
||||
problems = append(problems, fmt.Sprintf("%s's item of %s names %s: an item is kept in a bucket, where "+
|
||||
"no secret is (novox/hq ADR 0349 §7)", module, it.Collection, f[0]))
|
||||
case collection:
|
||||
problems = append(problems, fmt.Sprintf("%s's item of %s names %s: an item is not a collector",
|
||||
module, it.Collection, f[0]))
|
||||
}
|
||||
}
|
||||
return append(problems, unconsumedPlaceholders(module, map[string]any{"id": "item:" + it.key(),
|
||||
"type": "file", "content": it.Content})...)
|
||||
}
|
||||
|
||||
// isOld is whether a collection's name is an old namespace's.
|
||||
func isOld(name string) bool {
|
||||
_, old, _ := oldCollection(name, true)
|
||||
return old
|
||||
}
|
||||
|
||||
// collectedPlaceholderProblems is what is wrong with one resource's ${collected:…}: placed only in a file's content,
|
||||
// and naming a collection the module declares it collects.
|
||||
func collectedPlaceholderProblems(m Manifest, r map[string]any) []string {
|
||||
var problems []string
|
||||
for _, field := range sortedKeys(r) {
|
||||
v, ok := r[field].(string)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for _, f := range ofCollected.FindAllStringSubmatch(v, -1) {
|
||||
base, selector := splitSelector(f[1])
|
||||
if _, err := selectorOf(selector); err != nil {
|
||||
problems = append(problems, fmt.Sprintf("%s's resource %v names %s: %v", m.Module, r["id"], f[0], err))
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case field != "content":
|
||||
problems = append(problems, fmt.Sprintf("%s's resource %v names %s in its %s; a collection is placed "+
|
||||
"only in a file's content", m.Module, r["id"], f[0], field))
|
||||
case isOld(base):
|
||||
if _, _, wrong := oldCollection(base, false); wrong != nil {
|
||||
problems = append(problems, fmt.Sprintf("%s's resource %v names %s: %v", m.Module, r["id"], f[0], wrong))
|
||||
}
|
||||
case !slices.ContainsFunc(m.Collections, func(c Collection) bool { return c.Name == base }):
|
||||
problems = append(problems, fmt.Sprintf("%s's resource %v names %s, and %s does not collect %q: "+
|
||||
"declare it under collections (novox/hq ADR 0349 §13)", m.Module, r["id"], f[0], m.Module, base))
|
||||
}
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// collectionComment is the comment of a collection a module's file names: declared, or the old namespace's.
|
||||
func collectionComment(m Manifest, name string) (string, error) {
|
||||
if comment, old, wrong := oldCollection(name, false); old {
|
||||
return comment, wrong
|
||||
}
|
||||
for _, c := range m.Collections {
|
||||
if c.Name == name {
|
||||
return c.Comment, nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("%s names ${collected:%s} and does not collect it", m.Module, name)
|
||||
}
|
||||
|
||||
// itemsLeftOut says each item of a collection a file is filled with the old way that the controller could not render
|
||||
// there (heldItemText): the collector is not handed to the provisioner on this machine (ADR 0349 clause 17). Said,
|
||||
// never dropped quietly.
|
||||
func itemsLeftOut(modules []Manifest, holder Manifest, collection string, facts map[string]string) []string {
|
||||
return itemsLeftOutFor(modules, holder, collection, nil, facts, nil)
|
||||
}
|
||||
|
||||
// itemsLeftOutFor is itemsLeftOut for a placeholder that selects by attributes (where), on a machine with these
|
||||
// capabilities: an item the selector does not take, or whose capability the machine lacks, is not left out but not
|
||||
// wanted here.
|
||||
func itemsLeftOutFor(modules []Manifest, holder Manifest, collection string, where map[string]string,
|
||||
facts map[string]string, caps map[string]bool) []string {
|
||||
var out []string
|
||||
for _, m := range inModuleOrder(modules) {
|
||||
for _, it := range itemsInOrder(m.Items) {
|
||||
if it.Collection != collection || !it.matches(where) || (it.IfCapability != "" && !caps[it.IfCapability]) {
|
||||
continue
|
||||
}
|
||||
if _, rendered := heldText(it, facts); rendered && renderedOldWay(collection) {
|
||||
continue
|
||||
}
|
||||
out = append(out, fmt.Sprintf("%s's item of %s is not placed: %s is filled the old way on this "+
|
||||
"machine, from contributions, and an item only by the provisioner's client (design 57 L1)",
|
||||
m.Module, it.key(), holder.Module))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// oldFormsLeftOut says each piece of an old form a handed collector's collections would have held: the collector is
|
||||
// filled from items alone, and the piece's module contributes none (ADR 0349 clause 17).
|
||||
func oldFormsLeftOut(modules []Manifest, holder Manifest, collections []PlanCollection) []string {
|
||||
var out []string
|
||||
for _, c := range collections {
|
||||
namespace, key, _ := strings.Cut(c.Name, ":")
|
||||
for _, m := range inModuleOrder(modules) {
|
||||
old := false
|
||||
switch namespace {
|
||||
case "shell":
|
||||
target, slot, _ := strings.Cut(key, ":")
|
||||
old = slices.ContainsFunc(m.Shell, func(s ShellCode) bool { return s.For == target && s.Slot == slot })
|
||||
case "environment":
|
||||
old = m.Environment != nil && (len(m.Environment.Variables) > 0 || len(m.Environment.Path) > 0)
|
||||
case "contribution":
|
||||
seat, kind, _ := strings.Cut(key, ":")
|
||||
old = slices.ContainsFunc(m.allContributions(), func(s SeatContribution) bool {
|
||||
return canonicalSeat(s.Seat) == canonicalSeat(seat) && s.Kind == kind
|
||||
})
|
||||
}
|
||||
if old {
|
||||
out = append(out, fmt.Sprintf("%s's %s is not placed: %s is filled by the provisioner's client on "+
|
||||
"this machine, from items, and %s contributes it the old way (design 57 L1)", m.Module, c.Name,
|
||||
holder.Module, m.Module))
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// heldItemText is a moved contributor's items of an old namespace's collection as the controller renders them into a
|
||||
// collector not handed on this machine (design 57 L2, issue 681): the text the mesh holds, read back from the
|
||||
// contributor's templates archive, with the machine's facts filled, each ending its line. As a moved module's file
|
||||
// keeps its text for a machine that cannot take it handed (handed.go), an item keeps its place in a collector filled
|
||||
// the old way, so the file stays what its old form made it. Only `shell:` and `contribution:` collections, which the
|
||||
// controller renders as text; false for a module with no such item, or an item whose text the mesh does not hold or
|
||||
// which names anything but a machine fact this machine has, which is then said as not placed.
|
||||
func heldItemText(m Manifest, collection string, facts map[string]string) (string, bool) {
|
||||
items := heldItems(m, collection, nil, facts, nil)
|
||||
var b strings.Builder
|
||||
for _, h := range items {
|
||||
b.WriteString(h.text)
|
||||
}
|
||||
return b.String(), len(items) > 0
|
||||
}
|
||||
|
||||
// renderedOldWay is whether the controller renders a collection's items into a collector not handed.
|
||||
func renderedOldWay(collection string) bool {
|
||||
namespace, _, _ := strings.Cut(collection, ":")
|
||||
return namespace == "shell" || namespace == "contribution" || namespace == EnvironmentCollection
|
||||
}
|
||||
|
||||
// heldItem is one item of a module rendered the old way.
|
||||
type heldItem struct {
|
||||
item Item
|
||||
text string
|
||||
}
|
||||
|
||||
// heldItems is a module's items of the collection that the selector takes and the machine's capabilities allow, by
|
||||
// order then name, each rendered as heldText does; the ones it cannot render are left out (itemsLeftOutFor says so).
|
||||
func heldItems(m Manifest, collection string, where map[string]string, facts map[string]string,
|
||||
caps map[string]bool) []heldItem {
|
||||
if !renderedOldWay(collection) {
|
||||
return nil
|
||||
}
|
||||
var out []heldItem
|
||||
for _, it := range itemsInOrder(m.Items) {
|
||||
if it.Collection != collection || !it.matches(where) || (it.IfCapability != "" && !caps[it.IfCapability]) {
|
||||
continue
|
||||
}
|
||||
if text, ok := heldText(it, facts); ok {
|
||||
out = append(out, heldItem{item: it, text: text})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// heldText is an item's text with the machine's facts filled, ending its line; false for an item whose text the mesh
|
||||
// does not hold or which names anything but a machine fact this machine has.
|
||||
func heldText(it Item, facts map[string]string) (string, bool) {
|
||||
if it.Content == "" {
|
||||
return "", false
|
||||
}
|
||||
text := it.Content
|
||||
for _, f := range templatePlaceholder.FindAllStringSubmatch(text, -1) {
|
||||
value, has := facts[f[2]]
|
||||
if f[1] != "machine" || !has {
|
||||
return "", false
|
||||
}
|
||||
text = strings.ReplaceAll(text, f[0], value)
|
||||
}
|
||||
if !strings.HasSuffix(text, "\n") {
|
||||
text += "\n"
|
||||
}
|
||||
return text, true
|
||||
}
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A bar's blocks are a collection of text (design 57 L3, issue 781): the holder of node-bar names the kind as
|
||||
// `${collected:contribution:node-bar:block:bar=<b>,place=<p>}`, which a handed collector has its client fill from the
|
||||
// items and an unhanded one has the controller fill as it fills the old placeholder.
|
||||
|
||||
const (
|
||||
collectedBarsText = "top\n${collected:contribution:node-bar:block:bar=top,place=status}" +
|
||||
"bottom\n${collected:contribution:node-bar:block:bar=bottom,place=status}"
|
||||
collectedBars = `{"module":"bars","version":"1","moved":true,
|
||||
"claims":[{"name":"node-bar","scope":"node","renders":{"block":"{{.shows}}"}}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
|
||||
"resources":[{"id":"conf","type":"file","path":"/etc/bars.conf","template":"files/bars"}]}`
|
||||
itemsOfPower = `{"module":"power","version":"1","moved":true,
|
||||
"items":[{"collection":"contribution:node-bar:block","name":"battery","if-capability":"battery","attributes":{"bar":"bottom","place":"status","order":"40"},"template":"files/battery"},
|
||||
{"collection":"contribution:node-bar:block","name":"draw","attributes":{"bar":"bottom","place":"status","order":"45"},"template":"files/draw"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`
|
||||
)
|
||||
|
||||
func collectedBarsOn(t *testing.T, caps map[string]bool) Resolution {
|
||||
t.Helper()
|
||||
bars := held(t, collectedBars, builtWith("bars", map[string]string{"bars": collectedBarsText}))
|
||||
power := held(t, itemsOfPower, builtWith("power", map[string]string{"battery": "B\n", "draw": "D ${machine:name}\n"}))
|
||||
r, err := Resolve(shelf(bars, power), []string{"bars", "power"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Account, r.Capabilities = "op", caps
|
||||
return r
|
||||
}
|
||||
|
||||
func TestACollectorMayNameTheBarsBlocksAsACollectionAndNoOtherDataKind(t *testing.T) {
|
||||
bars := held(t, collectedBars, builtWith("bars", map[string]string{"bars": collectedBarsText}))
|
||||
if _, err := collectedBarsOn(t, nil).Compose(handingTo("store:5000")); err != nil {
|
||||
t.Fatalf("the handed collector is refused: %v", err)
|
||||
}
|
||||
if got := collectedPlaceholderProblems(bars, map[string]any{"id": "x",
|
||||
"content": "${collected:contribution:node-bar:block:bar=top,place=status}"}); len(got) != 0 {
|
||||
t.Fatalf("the check refuses it: %v", got)
|
||||
}
|
||||
// A data kind whose shape does not say its items are text stays refused.
|
||||
_, plain, _ := receivable(BarSeat, BarKindBlock)
|
||||
plain.Shape = &Shape{}
|
||||
if collectableAsText(plain) {
|
||||
t.Fatal("a shape not marked as text is collectable")
|
||||
}
|
||||
if _, _, err := oldCollection("contribution:node-bar:block", false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheBarsCollectionIsFilledTheOldWayWhereTheCollectorIsNotHanded(t *testing.T) {
|
||||
for _, caps := range []map[string]bool{nil, {"battery": true}} {
|
||||
got := composedFile(t, collectedBarsOn(t, caps), "bars.conf")
|
||||
want := "top\nbottom\n# power\nD workstation\n"
|
||||
if caps["battery"] {
|
||||
want = "top\nbottom\n# power\nB\n# power\nD workstation\n"
|
||||
}
|
||||
if got != want {
|
||||
t.Errorf("caps %v: the file is %q, want %q", caps, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheBarsCollectionIsNamedInTheHandedPlan(t *testing.T) {
|
||||
c, _ := composeWith(t, collectedBarsOn(t, map[string]bool{"battery": true}), handingTo("store:5000"))
|
||||
if !reflect.DeepEqual(c.Plans["bars"].Collections, []PlanCollection{{Name: "contribution:node-bar:block", Comment: "#"}}) {
|
||||
t.Fatalf("the plan collects %+v", c.Plans["bars"].Collections)
|
||||
}
|
||||
if n := len(c.Plans["power"].Items); n != 2 {
|
||||
t.Fatalf("the contributor's plan names %d items", n)
|
||||
}
|
||||
}
|
||||
|
||||
// The bars of the catalogue's i3status-rust, with the blocks of power's items, compose to the files the old shaped
|
||||
// contribution made (issue 781). testdata/bars holds the catalogue's bars and block texts as of the move and the bars as
|
||||
// composed before it, for a machine with neither capability, the battery, the power meter and both. Where the
|
||||
// collector is not handed the controller fills the files; where it is, it names the items for the client, which
|
||||
// fills the same files (mesh-catalog's provisioner client holds that half against the same golden files).
|
||||
func TestTheBarsOfTheCatalogueAreTheSameHandedOrNot(t *testing.T) {
|
||||
read := func(name string) string {
|
||||
b, err := os.ReadFile(filepath.Join("testdata", "bars", name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
collector := `{"module":"bars","version":"1","moved":true,"claims":[{"name":"node-bar","scope":"node"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
|
||||
"resources":[{"id":"top","type":"file","path":"/t","template":"files/top"},
|
||||
{"id":"bottom","type":"file","path":"/b","template":"files/bottom"}]}`
|
||||
power := `{"module":"power","version":"1","moved":true,
|
||||
"items":[{"collection":"contribution:node-bar:block","name":"battery","if-capability":"battery","attributes":{"bar":"bottom","place":"status","order":"40"},"template":"files/battery"},
|
||||
{"collection":"contribution:node-bar:block","name":"power-draw","if-capability":"power-meter","attributes":{"bar":"bottom","place":"status","order":"45"},"template":"files/draw"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`
|
||||
for name, caps := range map[string]map[string]bool{"none": nil, "battery": {"battery": true},
|
||||
"power-meter": {"power-meter": true}, "both": {"battery": true, "power-meter": true}} {
|
||||
bars := held(t, collector, builtWith("bars", map[string]string{"top": read("top-bar.toml"), "bottom": read("bottom-bar.toml")}))
|
||||
blocks := held(t, power, builtWith("power", map[string]string{"battery": read("bar-battery"), "draw": read("bar-power-draw")}))
|
||||
r, err := Resolve(shelf(bars, blocks), []string{"bars", "power"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Account, r.Capabilities = "op", caps
|
||||
// Not handed: the controller fills them.
|
||||
for id, which := range map[string]string{"bars.top": "top", "bars.bottom": "bottom"} {
|
||||
if got := composedFile(t, r, id); got != read(name+"."+which+"-bar.toml") {
|
||||
t.Errorf("%s, %s bar, not handed:\n%s\nwant\n%s", name, which, got, read(name+"."+which+"-bar.toml"))
|
||||
}
|
||||
}
|
||||
// Handed: the plan names the items the machine's capabilities allow, for the client to fill.
|
||||
c, _ := composeWith(t, r, handingTo("store:5000"))
|
||||
var names []string
|
||||
for _, it := range c.Plans["power"].Items {
|
||||
names = append(names, it.Name)
|
||||
}
|
||||
want := []string{}
|
||||
if caps["battery"] {
|
||||
want = append(want, "battery")
|
||||
}
|
||||
if caps["power-meter"] {
|
||||
want = append(want, "power-draw")
|
||||
}
|
||||
if !reflect.DeepEqual(append([]string{}, names...), want) {
|
||||
t.Errorf("%s: the plan names %v, want %v", name, names, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The collectors design 57 L2 moved (issue 681), composed with nothing handed: zsh's rc, triggerhappy's triggers and
|
||||
// power's after-wake moment, from the catalogue's real templates and its moved contributors' real items, are byte
|
||||
// for byte what the controller composed from the old forms before the move (testdata/l2-moved: templates/ and items/
|
||||
// copied from mesh-catalog at the move, want/ composed at the feature branch before it).
|
||||
|
||||
func l2Moved(t *testing.T, parts ...string) string {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile(filepath.Join(append([]string{"testdata", "l2-moved"}, parts...)...))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
func TestTheCollectorsL2MovedComposeAsBeforeWhereNothingIsHanded(t *testing.T) {
|
||||
// Each moved contributor, its items read back from its templates archive as the mesh holds them.
|
||||
items := map[string][]string{
|
||||
"asus-zephyrus-g14": {"shell:after-wake:normal", "contribution:node-hotkeys:trigger"},
|
||||
"claude-code": {"shell:zsh:normal"},
|
||||
"mesh-cli": {"shell:zsh:normal"},
|
||||
"powerlevel10k": {"shell:zsh:normal"},
|
||||
"zsh-autosuggestions": {"shell:zsh:normal"},
|
||||
"zsh-syntax-highlighting": {"shell:zsh:last"},
|
||||
}
|
||||
var modules []Manifest
|
||||
for _, module := range sortedKeys(items) {
|
||||
m := Manifest{Module: module, Moved: true}
|
||||
for _, collection := range items[module] {
|
||||
m.Items = append(m.Items, Item{Collection: collection,
|
||||
Content: l2Moved(t, "items", module+"."+strings.ReplaceAll(collection, ":", "-"))})
|
||||
}
|
||||
modules = append(modules, m)
|
||||
}
|
||||
for _, c := range []struct{ module, seat string }{{"zsh", LoginShellSeat}, {"triggerhappy", HotkeysSeat},
|
||||
{"power", PowerSeat}} {
|
||||
holder := Manifest{Module: c.module, Moved: true, Claims: []Claim{{Name: c.seat}}}
|
||||
resource := map[string]any{"id": "file", "type": "file", "path": "/etc/" + c.module,
|
||||
"content": l2Moved(t, "templates", c.module)}
|
||||
var unplaced []string
|
||||
if err := contributionsInto(resource, holder, append([]Manifest{holder}, modules...),
|
||||
map[string]string{"name": "g14"}, Rendering{}, map[string]bool{}, &unplaced); err != nil {
|
||||
t.Fatalf("%s: %v", c.module, err)
|
||||
}
|
||||
if got, want := resource["content"], l2Moved(t, "want", c.module); got != want {
|
||||
t.Errorf("%s's file is not what the old forms made:\n%s\nwant:\n%s", c.module, got, want)
|
||||
}
|
||||
if len(unplaced) != 0 {
|
||||
t.Errorf("%s: unplaced %v", c.module, unplaced)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,14 +1,158 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// An item may carry attributes and a capability, a module may give several items to one collection, and a
|
||||
// placeholder may select by attributes (design 57 L3, issue 777). The controller leaves out of a plan an item whose
|
||||
// capability the machine lacks, accepts a selecting placeholder where it accepts the collection, and renders a
|
||||
// selected moved item into an old placeholder exactly as the contribution it replaces.
|
||||
|
||||
const (
|
||||
barCollector = `{"module":"bars","version":"1","moved":true,
|
||||
"collections":[{"name":"blocks","comment":"#"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},
|
||||
"resources":[{"id":"conf","type":"file","path":"/etc/bars.conf","template":"files/bars"}]}`
|
||||
barsText = "top\n${collected:blocks:bar=top,place=status}bottom\n${collected:blocks:bar=bottom,place=status}${collected:blocks}"
|
||||
barContribute = `{"module":"power","version":"1","moved":true,
|
||||
"items":[{"collection":"blocks","name":"battery","if-capability":"battery","attributes":{"bar":"bottom","place":"status","order":"20"},"template":"files/battery"},
|
||||
{"collection":"blocks","name":"clock","attributes":{"bar":"top","place":"status"},"template":"files/clock"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`
|
||||
)
|
||||
|
||||
func blocksOn(t *testing.T, caps map[string]bool) Composed {
|
||||
t.Helper()
|
||||
bars := held(t, barCollector, builtWith("bars", map[string]string{"bars": barsText}))
|
||||
power := held(t, barContribute, builtWith("power", map[string]string{"battery": "B\n", "clock": "C\n"}))
|
||||
r, err := Resolve(shelf(bars, power), []string{"bars", "power"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Account, r.Capabilities = "op", caps
|
||||
c, _ := composeWith(t, r, handingTo("store:5000"))
|
||||
return c
|
||||
}
|
||||
|
||||
func TestAnItemWhoseCapabilityTheMachineLacksIsLeftOutOfItsPlan(t *testing.T) {
|
||||
with := blocksOn(t, map[string]bool{"battery": true}).Plans["power"]
|
||||
want := []PlanItem{
|
||||
{Collection: "blocks", Name: "battery", Template: "battery",
|
||||
Attributes: map[string]string{"bar": "bottom", "place": "status", "order": "20"}},
|
||||
{Collection: "blocks", Name: "clock", Template: "clock", Attributes: map[string]string{"bar": "top", "place": "status"}},
|
||||
}
|
||||
if !reflect.DeepEqual(with.Items, want) {
|
||||
t.Fatalf("with the capability the plan carries %+v", with.Items)
|
||||
}
|
||||
without := blocksOn(t, map[string]bool{"seat": true}).Plans["power"]
|
||||
if !reflect.DeepEqual(without.Items, want[1:]) {
|
||||
t.Fatalf("without it the plan carries %+v", without.Items)
|
||||
}
|
||||
}
|
||||
|
||||
func TestACollectorSelectingByAttributesCollectsTheCollectionOnce(t *testing.T) {
|
||||
bars := blocksOn(t, nil).Plans["bars"]
|
||||
if !reflect.DeepEqual(bars.Collections, []PlanCollection{{Name: "blocks", Comment: "#"}}) {
|
||||
t.Fatalf("the plan collects %+v", bars.Collections)
|
||||
}
|
||||
}
|
||||
|
||||
func TestASelectorIsRefusedWhenMalformedOrNamingACollectionNotCollected(t *testing.T) {
|
||||
for placeholder, want := range map[string]string{
|
||||
"${collected:blocks:bar}": "", // no equals sign: a collection named blocks:bar
|
||||
"${collected:blocks:bar=top,bar=bottom}": "names bar twice",
|
||||
"${collected:blocks:bar=}": "is not <attribute>=<value>",
|
||||
"${collected:other:bar=top}": `does not collect "other"`,
|
||||
} {
|
||||
raw := strings.Replace(barCollector, `"template":"files/bars"`, `"content":"`+placeholder+`"`, 1)
|
||||
raw = strings.Replace(raw, `"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},`, "", 1)
|
||||
raw = strings.Replace(raw, `"moved":true,`, "", 1)
|
||||
m, err := ParseManifest([]byte(raw))
|
||||
if err == nil {
|
||||
_, err = (Resolution{Node: "n", Modules: []Manifest{m}}).Declaration(Rendering{})
|
||||
}
|
||||
if want == "" {
|
||||
continue
|
||||
}
|
||||
if err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("%s: %v, want %q", placeholder, err, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestItemNamesAttributesAndCapabilitiesAreJudged(t *testing.T) {
|
||||
item := func(extra string) string {
|
||||
return `{"module":"p","moved":true,"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]},` +
|
||||
`"items":[` + extra + `]}`
|
||||
}
|
||||
for items, want := range map[string]string{
|
||||
`{"collection":"c","name":"a","template":"files/a"},{"collection":"c","template":"files/b"}`: "has no name",
|
||||
`{"collection":"c","name":"a","template":"files/a"},{"collection":"c","name":"a","template":"files/b"}`: "twice",
|
||||
`{"collection":"c","name":"Big","template":"files/a"}`: "is not a name",
|
||||
`{"collection":"c","attributes":{"order":"100"},"template":"files/a"}`: "0 to 99",
|
||||
`{"collection":"c","attributes":{"bar":"a b"},"template":"files/a"}`: "is not a value",
|
||||
`{"collection":"c","if-capability":"batery","template":"files/a"}`: "no machine reports",
|
||||
`{"collection":"contribution:node-bar:block","attributes":{"bar":"left"},"template":"files/a"}`: "bar=left",
|
||||
`{"collection":"contribution:node-bar:block","attributes":{"colour":"red"},"template":"files/a"}`: `"colour" is none of them`,
|
||||
`{"collection":"c","name":"a","template":"files/a","colour":"red"}`: "colour",
|
||||
} {
|
||||
_, err := ParseManifest([]byte(item(items)))
|
||||
if err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("%s: %v, want %q", items, err, want)
|
||||
}
|
||||
}
|
||||
ok := item(`{"collection":"contribution:node-bar:block","name":"a","attributes":{"bar":"top","place":"status","order":"5"},"if-capability":"battery","template":"files/a"}`)
|
||||
if _, err := ParseManifest([]byte(ok)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// A moved contributor's items, selected by the old placeholder's pair, render in the old order (order, module, item
|
||||
// name) and with the same lines as the contributions they replace.
|
||||
func TestASelectedMovedItemRendersIntoTheOldBarPlaceholderAsTheContributionDid(t *testing.T) {
|
||||
piece := func(order, command string) string {
|
||||
return "[[block]]\nblock = \"custom\"\ncommand = \"" + command + "\"\ninterval = 5\n"
|
||||
}
|
||||
asOld := Manifest{Module: "power", Contributions: []SeatContribution{
|
||||
block(`{"bar":"bottom","place":"status","order":20,"shows":"command","options":{"command":"two","interval":5}}`),
|
||||
block(`{"bar":"bottom","place":"status","shows":"command","options":{"command":"one","interval":5}}`),
|
||||
block(`{"bar":"top","place":"resources","shows":"command","options":{"command":"up","interval":5}}`),
|
||||
}}
|
||||
asItems := Manifest{Module: "power", Moved: true, Items: []Item{
|
||||
{Collection: "contribution:node-bar:block", Name: "b", Content: piece("", "one"),
|
||||
Attributes: map[string]string{"bar": "bottom", "place": "status"}},
|
||||
{Collection: "contribution:node-bar:block", Name: "a", Content: piece("", "two"),
|
||||
Attributes: map[string]string{"bar": "bottom", "place": "status", "order": "20"}},
|
||||
{Collection: "contribution:node-bar:block", Name: "c", Content: piece("", "up"),
|
||||
Attributes: map[string]string{"bar": "top", "place": "resources"}},
|
||||
{Collection: "contribution:node-bar:block", Name: "d", Content: "x\n", IfCapability: "battery",
|
||||
Attributes: map[string]string{"bar": "bottom", "place": "status"}},
|
||||
}}
|
||||
other := Manifest{Module: "another", Contributions: []SeatContribution{
|
||||
block(`{"bar":"bottom","place":"status","order":20,"shows":"command","options":{"command":"mid","interval":5}}`)}}
|
||||
for _, caps := range []map[string]bool{nil, {"battery": true}} {
|
||||
was := composedFile(t, Resolution{Node: "n", Account: "op", Capabilities: caps,
|
||||
Modules: []Manifest{barHolder(), asOld, other}}, "a-bar.bottom")
|
||||
is := composedFile(t, Resolution{Node: "n", Account: "op", Capabilities: caps,
|
||||
Modules: []Manifest{barHolder(), asItems, other}}, "a-bar.bottom")
|
||||
// The old form puts the contributions of a module under one line naming it and the item form puts one under
|
||||
// each item: the same pieces in the same order, otherwise the same bytes.
|
||||
want := "own-first\nown-middle\n# another\n" + piece("", "mid") + "# power\n" + piece("", "two") +
|
||||
"# power\n" + piece("", "one")
|
||||
if caps["battery"] {
|
||||
want += "# power\nx\n"
|
||||
}
|
||||
want += "own-last\n"
|
||||
if is != want {
|
||||
t.Errorf("caps %v: the items render\n%s\nnot\n%s", caps, is, want)
|
||||
}
|
||||
if !strings.Contains(was, piece("", "two")) {
|
||||
t.Errorf("the contributions made\n%s", was)
|
||||
}
|
||||
}
|
||||
if got := composedFile(t, Resolution{Node: "n", Account: "op", Modules: []Manifest{barHolder(), asItems}}, "a-bar.top"); !strings.Contains(got, "\"up\"") || strings.Contains(got, `"one"`) {
|
||||
t.Errorf("the top bar renders\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A collection is declared and filled by the client (novox/hq ADR 0349 §13, design 57 L1, issue 684): a collector
|
||||
@@ -30,3 +32,148 @@ func builtWith(module string, texts map[string]string) []Built {
|
||||
Reference: ArtifactStoreScheme + module + "/files/blobs/sha256:" + strings.Repeat("e", 64),
|
||||
Digest: "sha256:" + strings.Repeat("e", 64), Texts: texts}}
|
||||
}
|
||||
|
||||
func collectionsOn(t *testing.T) Resolution {
|
||||
t.Helper()
|
||||
zsh := held(t, collectorZsh, builtWith("zsh", map[string]string{"zshrc": zshrcText}))
|
||||
prompt := held(t, contributorP10k, builtWith("prompt", map[string]string{"zsh": contributorText}))
|
||||
aliases, err := ParseManifest([]byte(oldFormAliases))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r, err := Resolve(shelf(zsh, prompt, aliases), []string{"zsh", "prompt", "aliases"}, workstation(), World{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Account = "op"
|
||||
return r
|
||||
}
|
||||
|
||||
func TestAContributorsBuildKeepsItsItemsTemplateAndText(t *testing.T) {
|
||||
m := held(t, contributorP10k, builtWith("prompt", map[string]string{"zsh": contributorText}))
|
||||
if m.Handed == nil || !reflect.DeepEqual(m.Handed.Items, map[string]string{"shell:zsh:normal": "zsh"}) {
|
||||
t.Fatalf("the build kept %+v", m.Handed)
|
||||
}
|
||||
if !reflect.DeepEqual(m.Items, []Item{{Collection: "shell:zsh:normal", Content: contributorText}}) {
|
||||
t.Fatalf("the held items are %+v: the template's text, read back", m.Items)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAHandedCollectorAndContributorArePlannedAndNeitherIsRendered(t *testing.T) {
|
||||
c, got := composeWith(t, collectionsOn(t), handingTo("store:5000"))
|
||||
if strings.Contains(got, ".p10k.zsh") || strings.Contains(got, "alias ll") {
|
||||
t.Fatalf("the controller rendered a contribution into a handed collector: %s", got)
|
||||
}
|
||||
zsh, prompt := c.Plans["zsh"], c.Plans["prompt"]
|
||||
wantCollections := []PlanCollection{{Name: "shell:zsh:normal", Comment: "#"}, {Name: "widgets", Comment: "#"}}
|
||||
if !reflect.DeepEqual(zsh.Collections, wantCollections) || len(zsh.Items) != 0 {
|
||||
t.Fatalf("zsh's plan collects %+v and contributes %+v", zsh.Collections, zsh.Items)
|
||||
}
|
||||
if !reflect.DeepEqual(prompt.Items, []PlanItem{{Collection: "shell:zsh:normal", Template: "zsh"}}) ||
|
||||
len(prompt.Files) != 0 || prompt.Placement["machine:account-home"] != "/home/op" {
|
||||
t.Fatalf("prompt's plan is %+v: its item, by template, and the placement its template names", prompt)
|
||||
}
|
||||
// The old form on the same machine is not placed, and said so.
|
||||
if len(c.Unplaced) != 1 || !strings.Contains(c.Unplaced[0], "aliases's shell:zsh:normal is not placed") {
|
||||
t.Fatalf("unplaced: %v", c.Unplaced)
|
||||
}
|
||||
}
|
||||
|
||||
// A collector filled the old way holds the old forms and each moved contributor's item, rendered from the text the
|
||||
// mesh holds as its old form was (design 57 L2, issue 681), so the file loses no line where nothing is handed; a
|
||||
// collection of the new form stays empty.
|
||||
func TestACollectorFilledTheOldWayHoldsTheOldFormsAndEachHeldItem(t *testing.T) {
|
||||
c, got := composeWith(t, collectionsOn(t), Rendering{ArtifactStore: "store:5000"})
|
||||
if c.Plans != nil {
|
||||
t.Fatalf("handed on a machine that cannot take it: %v", c.Plans)
|
||||
}
|
||||
if !strings.Contains(got, `"content":"# zsh\n# aliases\nalias ll='ls -l'\n# prompt\nsource /home/op/.p10k.zsh\n"`) {
|
||||
t.Fatalf("the old way does not hold the old forms and the held item in module order: %s", got)
|
||||
}
|
||||
if len(c.Unplaced) != 0 {
|
||||
t.Fatalf("unplaced: %v", c.Unplaced)
|
||||
}
|
||||
}
|
||||
|
||||
// An item the controller cannot render the old way (it names a value but a machine fact) is said as not placed.
|
||||
func TestAHeldItemNamingMoreThanAFactIsSaidNotPlacedTheOldWay(t *testing.T) {
|
||||
zsh := held(t, collectorZsh, builtWith("zsh", map[string]string{"zshrc": zshrcText}))
|
||||
if _, ok := heldItemText(Manifest{Module: "p", Items: []Item{{Collection: "shell:zsh:normal",
|
||||
Content: "export THEME=${config:theme}\n"}}}, "shell:zsh:normal", map[string]string{}); ok {
|
||||
t.Fatal("an item naming a config value was rendered by the controller")
|
||||
}
|
||||
if _, ok := heldItemText(Manifest{Module: "p", Items: []Item{{Collection: "shell:zsh:normal",
|
||||
Content: "cd ${machine:account-home}\n"}}}, "shell:zsh:normal", map[string]string{}); ok {
|
||||
t.Fatal("an item naming a fact this machine lacks was rendered")
|
||||
}
|
||||
if _, ok := heldItemText(Manifest{Module: "p", Items: []Item{{Collection: "widgets", Content: "x\n"}}},
|
||||
"widgets", nil); ok {
|
||||
t.Fatal("an item of a collection of the new form was rendered the old way")
|
||||
}
|
||||
left := itemsLeftOut([]Manifest{zsh, {Module: "p", Items: []Item{{Collection: "shell:zsh:normal",
|
||||
Content: "export THEME=${config:theme}\n"}}}}, zsh, "shell:zsh:normal", map[string]string{})
|
||||
if len(left) != 1 || !strings.Contains(left[0], "p's item of shell:zsh:normal is not placed") {
|
||||
t.Fatalf("left out: %v", left)
|
||||
}
|
||||
}
|
||||
|
||||
// Each old namespace L2 moves renders a moved contributor's item exactly as its old form did, where the collector is
|
||||
// not handed: the same contributor written both ways composes the same bytes (design 57 L2, issue 681).
|
||||
func TestAMovedContributorsItemRendersAsItsOldFormDid(t *testing.T) {
|
||||
code := "# two lines\nalias a=b"
|
||||
asOld := Manifest{Module: "laptop", Shell: []ShellCode{{For: "zsh", Slot: "normal", Code: code}},
|
||||
Contributions: []SeatContribution{{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_F21 1 touchpad\n"}}}
|
||||
asItem := Manifest{Module: "laptop", Moved: true, Items: []Item{{Collection: "shell:zsh:normal", Content: code},
|
||||
{Collection: "contribution:node-hotkeys:trigger", Content: "KEY_F21 1 touchpad\n"}}}
|
||||
other := Manifest{Module: "zz-other", Shell: []ShellCode{{For: "zsh", Slot: "normal", Code: "x=1\n"}},
|
||||
Contributions: []SeatContribution{{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_MUTE 1 mute\n"}}}
|
||||
first := Manifest{Module: "aa-first", Shell: []ShellCode{{For: "zsh", Slot: "normal", Code: "y=2\n"}}}
|
||||
for _, mods := range [][]Manifest{{other, first}, {}} {
|
||||
was := append(append([]Manifest(nil), mods...), asOld)
|
||||
is := append(append([]Manifest(nil), mods...), asItem)
|
||||
if a, b := shellCodeOn(was, "zsh", "normal", nil), shellCodeOn(is, "zsh", "normal", nil); a != b || a == "" {
|
||||
t.Errorf("shell: the old form made %q, the item %q", a, b)
|
||||
}
|
||||
a, _, errA := seatContributions(was, hotkeysHolder(), "node-hotkeys:trigger", Rendering{}, nil, nil)
|
||||
b, left, errB := seatContributions(is, hotkeysHolder(), "node-hotkeys:trigger", Rendering{}, nil, nil)
|
||||
if errA != nil || errB != nil || a != b || a == "" || len(left) != 0 {
|
||||
t.Errorf("hotkeys: the old form made %q, the item %q (%v %v %v)", a, b, errA, errB, left)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollectionsAndItemsAreJudged(t *testing.T) {
|
||||
for name, c := range map[string]struct{ raw, says string }{
|
||||
"an item of a module not moved": {`{"module":"m","version":"1",
|
||||
"items":[{"collection":"shell:zsh:normal","template":"files/x"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is marked moved"},
|
||||
"two items to one collection": {`{"module":"m","version":"1","moved":true,
|
||||
"items":[{"collection":"a","template":"files/x"},{"collection":"a","template":"files/y"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "contributes to a twice"},
|
||||
"an item to no slot": {`{"module":"m","version":"1","moved":true,
|
||||
"items":[{"collection":"shell:zsh:middle","template":"files/x"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is no slot"},
|
||||
"an item outside the templates": {`{"module":"m","version":"1","moved":true,
|
||||
"items":[{"collection":"a","template":"x"}],
|
||||
"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`, "is not a file under files/"},
|
||||
"an item naming a secret": {`{"module":"m","version":"1",
|
||||
"items":[{"collection":"a","content":"t=${secret:token}"}]}`, "where no secret is"},
|
||||
"a collection not declared": {`{"module":"m","version":"1",
|
||||
"resources":[{"id":"f","type":"file","path":"/f","content":"${collected:a}"}]}`, "does not collect"},
|
||||
"an old namespace declared": {`{"module":"m","version":"1","collections":[{"name":"shell:zsh:normal"}],
|
||||
"resources":[{"id":"f","type":"file","path":"/f","content":"x"}]}`, "read from its placeholder"},
|
||||
"a collection placed nowhere": {`{"module":"m","version":"1","collections":[{"name":"a"}],
|
||||
"resources":[{"id":"f","type":"file","path":"/f","content":"x"}]}`, "no file of its places it"},
|
||||
"a collection in a path": {`{"module":"m","version":"1","collections":[{"name":"a"}],
|
||||
"resources":[{"id":"f","type":"file","path":"/${collected:a}","content":"${collected:a}"}]}`,
|
||||
"only in a file's content"},
|
||||
} {
|
||||
_, err := ParseManifest([]byte(c.raw))
|
||||
if err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%s: %v, want it to say %q", name, err, c.says)
|
||||
}
|
||||
}
|
||||
if _, err := ParseManifest([]byte(collectorZsh)); err != nil {
|
||||
t.Errorf("a collector: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user