Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f185aedbf5 |
@@ -1,3 +1,2 @@
|
||||
node_modules/
|
||||
dist/
|
||||
vendor/
|
||||
|
||||
@@ -9,15 +9,6 @@ does not belong.* Per-module code (a service's API client, its tool implementati
|
||||
create-a-resource adapter) lives **in the module**, never here — that coupling is exactly what
|
||||
turned the old SDK into constant maintenance and made every edit rebuild every module.
|
||||
|
||||
**A seat's code is not here** (novox/hq ADR 0625, ADR 0627). mesh-sdk is the runtime every module uses (the bus
|
||||
connection, the tools bundle's runtime, state and its helpers) and a seat-neutral part two or more seats' clients or
|
||||
holders share (sending a file in parts, `go/parts`). A seat defined whole has its generated contract, and its client
|
||||
where callers share logic over it, in mesh-contracts (`go/seats/<seat>`, `go/clients/<seat>client`); the seat list is
|
||||
mesh-contracts' `go/seats`, which `catalogue` alone imports. The value layouts of seats not yet defined whole stay
|
||||
here until their domain moves, and no new seat-specific type is added. `packages_list_test.go` holds all of it:
|
||||
mesh-sdk imports no contracts module but the seat set in `catalogue`, names none of the seats with a client outside
|
||||
`catalogue`, and holds none of the packages that moved (`asks`, `asker`, `publish`).
|
||||
|
||||
## What is in it
|
||||
|
||||
Five small areas, each a thing a module's `tools/` or `provisioner/` imports:
|
||||
@@ -115,31 +106,21 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
|
||||
values and then every change. Through the runtime that launched the bundle, which refuses what the
|
||||
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
|
||||
|
||||
### `runtime`, `callers` and `views` — holding a seat from TypeScript
|
||||
The TypeScript side of what the Go side's `stdio` and `views` give a seat's holder, with the same rules (design 47
|
||||
*The forge module's tools come from mesh-sdk's TypeScript side*): `runtime.seatPublish` says an event or submits an
|
||||
accept on a seat (`mesh/seat.publish`) and `runtime.config` reads a config value; `callers.callerOf(args)` is the
|
||||
caller the runtime proved in the call's `_meta`, never an argument, and `callers.refusal` judges it against a
|
||||
holder's list per verb (`parseVerbCallers`, `configuredVerbCallers`: `*` any proved caller, `operator` any person);
|
||||
`views.Follower` keeps a first view of an owner's state from its pages and its events, asking again only on a gap,
|
||||
and `views.statusTools` serves its read and ask-again tools (`<module>_first_views`, `<module>_first_view_again`)
|
||||
over the reader's state `first-views`. `waitFor(view)` and `holds(view)` say the revision view a part waits for and
|
||||
holds; past five minutes `raiseWaits` is told and `viewWaitsFinding` says the part's `view-waits` finding. Checked by `test/stdio.test.ts` and `test/sdk.test.ts`.
|
||||
|
||||
### `seats` and `channel` — publishing on a seat, the channel's benches (Go)
|
||||
### `seats` and `asks` — asking a person, answering on a channel (Go)
|
||||
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
|
||||
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
|
||||
and `stdio.SeatAnswer` carry a proof (a code typed by a person, by request and reply, never kept),
|
||||
`stdio.SeatRecord` reads a record kept for the module, and `State(...).Create` and `.Update` write by
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/channel` holds the
|
||||
shapes of the two benches a channel holds, `channel` and `intake` (the `Message` a channel takes, the `Choice`,
|
||||
`LinkAsked`, `Failed`, `Standing` and `Code` it says, with the subjects each travels on): those benches are not
|
||||
defined whole, so their layouts stay here (ADR 0625 §9).
|
||||
Asking a person is the `operator-channel` seat's, defined whole in mesh-contracts: its `Ask` and `Warrant` are its
|
||||
generated contract (`go/seats/operatorchannel`), and the checks an ask is held to, the warrant's (`For`, `Says`,
|
||||
`Performs`) and an asking module's client (`Ask`, `Cancel`, `Where`, `Listen`, `Take`, each warrant handed once,
|
||||
marked acted on by compare-and-set) are its client, `go/clients/operatorchannelclient` (ADR 0627 §1). A lost
|
||||
compare-and-set is `stdio.ErrChanged`, told by the runtime's error code.
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/asks` is the
|
||||
contract on top: the `Ask` an asker publishes and the checks it is held to (its `Whole` is what a channel that proves
|
||||
who answers shows in place of its `Explanation`, and its `Details` what the Details answer shows, hq issue 383), the `Warrant` it acts on
|
||||
(`For`, `Says`), and the `Message`, `Choice`, `LinkAsked` and `Code` a channel takes and says, with the
|
||||
subjects each travels on.
|
||||
`go/asker` is an asking module's client (ADR 0259 §10): `Ask` keeps an ask in the module's own state and then
|
||||
publishes it under the module's name; `Cancel`; `Where` reads its state from the router's record; `Listen` and
|
||||
`Take` hand the module each warrant once — marked acted on by compare-and-set before the module acts, and only
|
||||
for an ask it made, an option it offered, at that option's level, before it expired. A lost compare-and-set is
|
||||
`stdio.ErrChanged`, told by the runtime's error code.
|
||||
`State(...).Increment(key, attempts)` is a counter on top of the two (for numbers handed out once, such as an
|
||||
issue's): it reads the key, writes one more at the revision read, and on a lost write reads and tries again, at
|
||||
most `attempts` times; past that it is `stdio.ErrChanged` and nothing was counted.
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
example
|
||||
@@ -0,0 +1,8 @@
|
||||
CFLAGS ?= -O2 -Wall -Wextra
|
||||
JANSSON := $(shell pkg-config --cflags --libs jansson)
|
||||
|
||||
example: example.c mesh_sdk.c mesh_sdk.h
|
||||
$(CC) $(CFLAGS) -o $@ example.c mesh_sdk.c $(JANSSON)
|
||||
|
||||
clean:
|
||||
rm -f example
|
||||
@@ -0,0 +1,9 @@
|
||||
# mesh-sdk for C
|
||||
|
||||
The skeleton (novox/hq ADR 0188): `mesh_serve(name, tools, count)` speaks MCP over stdio to the
|
||||
node's tool runtime. A `mesh_tool` is a name, a description, an argument schema and a function
|
||||
returning JSON. A tool named `<seat>.<verb>` is the seat's implementation. JSON is jansson's, the
|
||||
one dependency.
|
||||
|
||||
What it holds, under ADR 0039's test: the stdio loop and the tool type. Nothing else — no bus, no
|
||||
client, nothing of any module's. `make` and pipe a `tools/list` request into `./example`.
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
/* A tools bundle in C: one tool of its own and one seat verb, served over stdio. */
|
||||
#include "mesh_sdk.h"
|
||||
#include <stdio.h>
|
||||
|
||||
static json_t *greet(json_t *args, char **error) {
|
||||
(void)error;
|
||||
const char *who = json_string_value(json_object_get(args, "who"));
|
||||
char greeting[128];
|
||||
snprintf(greeting, sizeof greeting, "hello %s", who ? who : "world");
|
||||
return json_pack("{s:s,s:s}", "greeting", greeting, "language", "c");
|
||||
}
|
||||
|
||||
static json_t *on(json_t *args, char **error) {
|
||||
(void)args; (void)error;
|
||||
return json_pack("{s:b,s:s}", "on", 1, "language", "c");
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
json_t *who = json_pack("{s:{s:s,s:s}}", "who", "type", "string", "description", "whom");
|
||||
const mesh_tool tools[] = {
|
||||
{"greet", "say hello", who, greet},
|
||||
{"node-lamp.on", "the seat's verb", NULL, on},
|
||||
};
|
||||
int rc = mesh_serve("c-example", tools, 2);
|
||||
json_decref(who);
|
||||
return rc;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
#include "mesh_sdk.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
static void say(json_t *m) {
|
||||
json_object_set_new(m, "jsonrpc", json_string("2.0"));
|
||||
char *text = json_dumps(m, JSON_COMPACT);
|
||||
fprintf(stdout, "%s\n", text);
|
||||
fflush(stdout);
|
||||
free(text);
|
||||
json_decref(m);
|
||||
}
|
||||
|
||||
static json_t *as_schema(json_t *given) {
|
||||
if (!given || !json_is_object(given) || json_object_size(given) == 0)
|
||||
return json_pack("{s:s,s:{}}", "type", "object", "properties");
|
||||
if (json_object_get(given, "type") || json_object_get(given, "properties"))
|
||||
return json_incref(given);
|
||||
return json_pack("{s:s,s:O}", "type", "object", "properties", given);
|
||||
}
|
||||
|
||||
int mesh_serve(const char *name, const mesh_tool *tools, size_t count) {
|
||||
char *line = NULL;
|
||||
size_t cap = 0;
|
||||
ssize_t got;
|
||||
while ((got = getline(&line, &cap, stdin)) >= 0) {
|
||||
json_error_t err;
|
||||
json_t *req = json_loads(line, 0, &err);
|
||||
if (!req) continue;
|
||||
json_t *id = json_object_get(req, "id");
|
||||
const char *method = json_string_value(json_object_get(req, "method"));
|
||||
json_t *params = json_object_get(req, "params");
|
||||
if (!method) { json_decref(req); continue; }
|
||||
if (strcmp(method, "initialize") == 0) {
|
||||
say(json_pack("{s:O,s:{s:s,s:{s:{}},s:{s:s,s:s}}}", "id", id ? id : json_null(), "result",
|
||||
"protocolVersion", MESH_SDK_PROTOCOL, "capabilities", "tools",
|
||||
"serverInfo", "name", name, "version", "1"));
|
||||
} else if (strcmp(method, "notifications/initialized") == 0) {
|
||||
} else if (strcmp(method, "ping") == 0) {
|
||||
if (id) say(json_pack("{s:O,s:{}}", "id", id, "result"));
|
||||
} else if (strcmp(method, "tools/list") == 0) {
|
||||
json_t *listed = json_array();
|
||||
for (size_t i = 0; i < count; i++)
|
||||
json_array_append_new(listed, json_pack("{s:s,s:s,s:o}", "name", tools[i].name,
|
||||
"description", tools[i].description, "inputSchema", as_schema(tools[i].input)));
|
||||
say(json_pack("{s:O,s:{s:o}}", "id", id ? id : json_null(), "result", "tools", listed));
|
||||
} else if (strcmp(method, "tools/call") == 0) {
|
||||
const char *wanted = params ? json_string_value(json_object_get(params, "name")) : NULL;
|
||||
const mesh_tool *tool = NULL;
|
||||
for (size_t i = 0; wanted && i < count; i++)
|
||||
if (strcmp(tools[i].name, wanted) == 0) tool = &tools[i];
|
||||
if (!tool) {
|
||||
char message[256];
|
||||
snprintf(message, sizeof message, "%s has no tool %s", name, wanted ? wanted : "");
|
||||
say(json_pack("{s:O,s:{s:i,s:s}}", "id", id ? id : json_null(), "error", "code", -32602, "message", message));
|
||||
} else {
|
||||
json_t *args = params ? json_object_get(params, "arguments") : NULL;
|
||||
json_t *empty = args ? NULL : json_object();
|
||||
char *error = NULL;
|
||||
json_t *result = tool->run(args ? args : empty, &error);
|
||||
if (empty) json_decref(empty);
|
||||
if (result) {
|
||||
char *text = json_dumps(result, JSON_COMPACT);
|
||||
say(json_pack("{s:O,s:{s:[{s:s,s:s}]}}", "id", id ? id : json_null(), "result", "content", "type", "text", "text", text));
|
||||
free(text);
|
||||
json_decref(result);
|
||||
} else {
|
||||
say(json_pack("{s:O,s:{s:[{s:s,s:s}],s:b}}", "id", id ? id : json_null(), "result", "content",
|
||||
"type", "text", "text", error ? error : "the tool failed", "isError", 1));
|
||||
free(error);
|
||||
}
|
||||
}
|
||||
} else if (id) {
|
||||
char message[256];
|
||||
snprintf(message, sizeof message, "no %s", method);
|
||||
say(json_pack("{s:O,s:{s:i,s:s}}", "id", id, "error", "code", -32601, "message", message));
|
||||
}
|
||||
json_decref(req);
|
||||
}
|
||||
free(line);
|
||||
return ferror(stdin) ? -1 : 0;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
/* The Novox Mesh SDK for C — the skeleton (novox/hq ADR 0188).
|
||||
*
|
||||
* A tools bundle is a process the node's runtime launches and speaks MCP over stdio to:
|
||||
* `initialize`, `tools/list` once, `tools/call` per call, newline-framed JSON-RPC on stdin and
|
||||
* stdout. This is the loop, so a bundle writes its tools and nothing else. A tool named
|
||||
* `<seat>.<verb>` is the module's implementation of that seat's verb; any other name is the
|
||||
* module's own. JSON is jansson's.
|
||||
*
|
||||
* No transport here: the bus, the subjects and the memberships are the runtime's (ADR 0039, 0175).
|
||||
* stdout is the protocol; log to stderr. */
|
||||
#ifndef MESH_SDK_H
|
||||
#define MESH_SDK_H
|
||||
#include <jansson.h>
|
||||
|
||||
#define MESH_SDK_PROTOCOL "2025-03-26"
|
||||
|
||||
/* One tool. `run` returns a new JSON value (the answer) or NULL with *error set to a message the
|
||||
* caller frees. `input` is the schema of the arguments, or NULL for a tool that takes nothing. */
|
||||
typedef struct {
|
||||
const char *name;
|
||||
const char *description;
|
||||
json_t *input;
|
||||
json_t *(*run)(json_t *args, char **error);
|
||||
} mesh_tool;
|
||||
|
||||
/* Serve these tools on stdin/stdout until stdin closes. Returns 0, or -1 on an I/O fault. */
|
||||
int mesh_serve(const char *name, const mesh_tool *tools, size_t count);
|
||||
|
||||
#endif
|
||||
+1
-3
@@ -16,6 +16,4 @@ at the same version as the TypeScript package.
|
||||
|
||||
What it holds, under ADR 0039's test: the stdio loop and the tool type, and `cli`, the local protocol
|
||||
`mesh-cli` speaks to the node-engine on its machine (novox/hq ADR 0272): the socket's path and one request and one
|
||||
answer; and `seal`, the mesh's sealed box (novox/hq ADR 0206: X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}`), byte for byte the box the licence manager made before it lived here. Nothing else — no bus, no client, nothing of any module's.
|
||||
|
||||
From release 4a of design 47 (novox/hq ADR 0453 clause 3, ADR 0446 clause 4) it also holds the composition of a machine's declaration, over a plain input and with nothing behind it: `declaration` and `validate`, the node-engine's parser and its validator, copied from mesh-host; `catalogue`, the manifests, their resolution and rendering, and `overlay`, the private network's generator, copied from mesh-controller; `compose`, the controller's planFor, renderingFor and declarationWith as functions of an `Input` (the composition never allocates, makes, keeps or writes: what a send would do it names as foreseen or returns); `sdkversion`, which mesh-sdk version a program was built with; and, in `delivery`, the layouts of the keys the builder, the checker and the indexer write (the source fingerprint's new form, the builds, the verdicts and kept verdicts, the manifests by digest). None of them holds a bus, a store or a vault client, which `go list -deps` of `compose` shows.
|
||||
answer; and `seal`, the mesh's sealed box (novox/hq ADR 0206: X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}`), byte for byte the box the licence manager made before it lived here and the TypeScript `@novox/mesh-sdk/seal` makes. Nothing else — no bus, no client, nothing of any module's. `go build ./example` and pipe a `tools/list` request into it.
|
||||
|
||||
@@ -1,152 +0,0 @@
|
||||
// Package announce is how a module says on the bus which states it keeps and which of them others may read (novox/hq
|
||||
// ADR 0495 clauses 2 and 3; issue 1790): the subject it says it on, the value it says, and where the controller keeps
|
||||
// the last one said by each module. What a module announced is the truth of what it keeps; a reader is granted a read
|
||||
// from it, never from a list or from the owner's manifest.
|
||||
//
|
||||
// A module announces from its code (process.Conn.Keeps, stdio.Keeps, the TypeScript SDK's keeps), never from its
|
||||
// manifest, which only the core reads (clause 1). A module that keeps no state on the bus announces nothing.
|
||||
package announce
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Version is the version of the value this package writes.
|
||||
const Version = 1
|
||||
|
||||
// Event is the event type of an announcement, under the module's own namespace.
|
||||
const Event = "states"
|
||||
|
||||
// Subject is where module announces what it keeps: its own event subject, mesh.mod.<module>.event.states, which its
|
||||
// account may already publish to and the events stream keeps.
|
||||
func Subject(module string) string { return "mesh.mod." + module + ".event." + Event }
|
||||
|
||||
// SubjectAll is every module's announcement.
|
||||
const SubjectAll = "mesh.mod.*.event." + Event
|
||||
|
||||
var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||
|
||||
// Key is module's key where the last announcement of each module is kept: the module itself, which must be one plain name, since a module's name is part of
|
||||
// its buckets' names (the indexer's judge, novox/hq ADR 0578 §1).
|
||||
func Key(module string) (string, error) {
|
||||
if !name.MatchString(module) {
|
||||
return "", fmt.Errorf("announce: %q is not a module that keeps state: lower-case letters, digits and hyphens", module)
|
||||
}
|
||||
return module, nil
|
||||
}
|
||||
|
||||
// State is one state a module keeps, by its local name, and whether others may read it. Others never write it: a
|
||||
// state another module writes is the owner's own code's business.
|
||||
type State struct {
|
||||
Name string `json:"name"`
|
||||
Readable bool `json:"readable,omitempty"`
|
||||
// Readers, when named, are the only modules that may read the state (novox/hq ADR 0547 clause 9, issue 1945): a
|
||||
// state readable by some and not all, such as the vault's bus account hashes. Only ReadableBy grants them; Readable,
|
||||
// which a composer that knows no readers asks, stays false for it, so such a composer grants nobody.
|
||||
Readers []string `json:"readers,omitempty"`
|
||||
}
|
||||
|
||||
// States is an announcement: the module, the machine whose instance said it, and every state the module keeps, sorted
|
||||
// by name. An empty list says the module keeps nothing.
|
||||
type States struct {
|
||||
V int `json:"v"`
|
||||
Module string `json:"module"`
|
||||
Node string `json:"node,omitempty"`
|
||||
States []State `json:"states"`
|
||||
}
|
||||
|
||||
// Keeps says whether the module announced keeping the state.
|
||||
func (s States) Keeps(state string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name == state {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ReadableBy says whether the module announced the state readable by reader: readable, and reader among the readers
|
||||
// when it names any.
|
||||
func (s States) ReadableBy(state, reader string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name != state {
|
||||
continue
|
||||
}
|
||||
if len(k.Readers) > 0 {
|
||||
for _, r := range k.Readers {
|
||||
if r == reader {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
return k.Readable
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Readable says whether the module announced the state readable by others.
|
||||
func (s States) Readable(state string) bool {
|
||||
for _, k := range s.States {
|
||||
if k.Name == state {
|
||||
return k.Readable
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (s *States) check() error {
|
||||
if s.Module == "" {
|
||||
return fmt.Errorf("announce: an announcement names no module")
|
||||
}
|
||||
if _, err := Key(s.Module); err != nil {
|
||||
return err
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, k := range s.States {
|
||||
if !name.MatchString(k.Name) {
|
||||
return fmt.Errorf("announce: %s announces the state %q, and a state is named locally: lower-case letters, digits and hyphens", s.Module, k.Name)
|
||||
}
|
||||
if seen[k.Name] {
|
||||
return fmt.Errorf("announce: %s announces the state %q twice", s.Module, k.Name)
|
||||
}
|
||||
seen[k.Name] = true
|
||||
for _, r := range k.Readers {
|
||||
if !name.MatchString(r) {
|
||||
return fmt.Errorf("announce: %s names %q as a reader of %s, and a reader is a module: lower-case letters, digits and hyphens", s.Module, r, k.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Encode is s as JSON at Version, its states sorted; it refuses a module or a state no bucket could be named by and a
|
||||
// state announced twice, naming it.
|
||||
func Encode(s States) ([]byte, error) {
|
||||
if err := s.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.V = Version
|
||||
s.States = append([]State{}, s.States...)
|
||||
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
|
||||
return json.Marshal(s)
|
||||
}
|
||||
|
||||
// Decode reads an announcement; it refuses another version and what Encode refuses.
|
||||
func Decode(data []byte) (States, error) {
|
||||
var s States
|
||||
if err := json.Unmarshal(data, &s); err != nil {
|
||||
return States{}, fmt.Errorf("announce: the announcement is not JSON: %w", err)
|
||||
}
|
||||
if s.V != Version {
|
||||
return States{}, fmt.Errorf("announce: the announcement is version %d, and this reads version %d", s.V, Version)
|
||||
}
|
||||
if err := s.check(); err != nil {
|
||||
return States{}, err
|
||||
}
|
||||
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
|
||||
return s, nil
|
||||
}
|
||||
@@ -1,75 +0,0 @@
|
||||
package announce
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A module says on the bus which states it keeps and which of them others may read (novox/hq ADR 0495 clause 2), on its
|
||||
// own event subject, in a versioned value the controller keeps by module.
|
||||
func TestAnAnnouncementRoundTripsSortedAndSaysWhatIsReadable(t *testing.T) {
|
||||
data, err := Encode(States{Module: "mesh-controller", Node: "novox", States: []State{
|
||||
{Name: "plans-building", Readable: true}, {Name: "assignments", Readable: true}, {Name: "own"},
|
||||
{Name: "hashes", Readers: []string{"mesh-grants"}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
back, err := Decode(data)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.V != Version || back.Module != "mesh-controller" || back.Node != "novox" {
|
||||
t.Fatalf("decoded %+v", back)
|
||||
}
|
||||
var names []string
|
||||
for _, s := range back.States {
|
||||
names = append(names, s.Name)
|
||||
}
|
||||
if strings.Join(names, ",") != "assignments,hashes,own,plans-building" {
|
||||
t.Fatalf("the states are not sorted: %v", names)
|
||||
}
|
||||
if !back.Readable("plans-building") || back.Readable("own") || back.Readable("absent") {
|
||||
t.Fatal("Readable does not say what was announced readable")
|
||||
}
|
||||
// Named readers (ADR 0547 clause 9): readable by them alone, and not readable to a composer that knows no readers.
|
||||
if back.Readable("hashes") || !back.ReadableBy("hashes", "mesh-grants") || back.ReadableBy("hashes", "spy") ||
|
||||
!back.ReadableBy("plans-building", "spy") || back.ReadableBy("own", "spy") {
|
||||
t.Fatal("ReadableBy does not say who was named")
|
||||
}
|
||||
if !back.Keeps("own") || back.Keeps("absent") {
|
||||
t.Fatal("Keeps does not say what was announced")
|
||||
}
|
||||
if Subject("mesh-controller") != "mesh.mod.mesh-controller.event.states" || SubjectAll != "mesh.mod.*.event.states" {
|
||||
t.Fatalf("the subjects are %q and %q", Subject("mesh-controller"), SubjectAll)
|
||||
}
|
||||
if k, err := Key("mesh-delivery-builder"); err != nil || k != "mesh-delivery-builder" {
|
||||
t.Fatalf("Key = %q, %v", k, err)
|
||||
}
|
||||
if nothing, err := Encode(States{Module: "quiet"}); err != nil || !strings.Contains(string(nothing), `"states":[]`) {
|
||||
t.Fatalf("a module keeping nothing: %s, %v", nothing, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAnnouncementRefusesWhatNoBucketCouldBe(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
s States
|
||||
says string
|
||||
}{
|
||||
{States{Module: ""}, "no module"},
|
||||
{States{Module: "a.b"}, `"a.b"`},
|
||||
{States{Module: "a", States: []State{{Name: "x_y"}}}, `"x_y"`},
|
||||
{States{Module: "a", States: []State{{Name: ""}}}, `""`},
|
||||
{States{Module: "a", States: []State{{Name: "x"}, {Name: "x", Readable: true}}}, "twice"},
|
||||
{States{Module: "a", States: []State{{Name: "x", Readers: []string{"b.c"}}}}, `"b.c"`},
|
||||
} {
|
||||
if _, err := Encode(c.s); err == nil || !strings.Contains(err.Error(), c.says) {
|
||||
t.Errorf("%+v: %v, wanted a refusal saying %s", c.s, err, c.says)
|
||||
}
|
||||
}
|
||||
if _, err := Decode([]byte(`{"v":2,"module":"a","states":[]}`)); err == nil {
|
||||
t.Error("an unknown version was read")
|
||||
}
|
||||
if _, err := Key("a.b"); err == nil {
|
||||
t.Error("a module with a dot became a key")
|
||||
}
|
||||
}
|
||||
@@ -1,91 +0,0 @@
|
||||
// Package announcetest is the test each module runs with its own checks to hold that it announces on the bus every
|
||||
// state it keeps (novox/hq ADR 0495 clause 5; issue 1790): it starts the module against a bus of the test's own and
|
||||
// compares what the module announced with what its code declares. With the bind guard of process.Conn.Keeps,
|
||||
// stdio.Keeps and the TypeScript keeps, which refuses a state the code did not announce, announcement and code agree.
|
||||
//
|
||||
// It replaces the merge gate's check of one module's reads against another's manifest (clause 5); a test using it
|
||||
// carries `//go:build integration`, since it starts a bus.
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
"git.novox.be/novox/mesh-sdk/go/testbus"
|
||||
)
|
||||
|
||||
// Wait is how long Check waits for the module's announcement once it started it.
|
||||
var Wait = 10 * time.Second
|
||||
|
||||
// Check starts a bus of the test's own, with a stream taking module's events, calls start with its address (start
|
||||
// runs the module, or its part that connects, against it, and answers how to stop it), waits for the module's
|
||||
// announcement and fails the test unless it is module's and names exactly want (in any order), readable as want says.
|
||||
// It answers the announcement.
|
||||
func Check(t testing.TB, module string, want []announce.State, start func(url string) (stop func())) announce.States {
|
||||
t.Helper()
|
||||
url := testbus.URL(t)
|
||||
nc, err := nats.Connect(url)
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: connecting to the test bus: %v", err)
|
||||
}
|
||||
defer nc.Close()
|
||||
js, err := jetstream.New(nc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), Wait)
|
||||
defer cancel()
|
||||
stream, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "ANNOUNCETEST", Subjects: []string{"mesh.mod." + module + ".event.>"}})
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: making the events stream: %v", err)
|
||||
}
|
||||
stop := start(url)
|
||||
if stop != nil {
|
||||
defer stop()
|
||||
}
|
||||
got, err := waitFor(ctx, stream, module)
|
||||
if err != nil {
|
||||
t.Fatalf("announcetest: %v", err)
|
||||
}
|
||||
if problem := differs(got, module, want); problem != "" {
|
||||
t.Fatalf("announcetest: %s", problem)
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
// waitFor reads the stream until module's announcement is in it.
|
||||
func waitFor(ctx context.Context, stream jetstream.Stream, module string) (announce.States, error) {
|
||||
for {
|
||||
msg, err := stream.GetLastMsgForSubject(ctx, announce.Subject(module))
|
||||
if err == nil {
|
||||
return announce.Decode(msg.Data)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return announce.States{}, fmt.Errorf("%s announced nothing on %s within %s", module, announce.Subject(module), Wait)
|
||||
case <-time.After(50 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// differs says how got is not module's announcement of want, or "".
|
||||
func differs(got announce.States, module string, want []announce.State) string {
|
||||
if got.Module != module {
|
||||
return fmt.Sprintf("the announcement names the module %q, not %q", got.Module, module)
|
||||
}
|
||||
w := append([]announce.State{}, want...)
|
||||
sort.Slice(w, func(i, j int) bool { return w[i].Name < w[j].Name })
|
||||
g := append([]announce.State{}, got.States...)
|
||||
if !reflect.DeepEqual(g, w) && !(len(g) == 0 && len(w) == 0) {
|
||||
return fmt.Sprintf("%s announced %v, and its code declares %v", module, g, w)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
"git.novox.be/novox/mesh-sdk/go/process"
|
||||
)
|
||||
|
||||
// 1790/S11: a module started against the test bus announces through process.Conn.Keeps, and Check reads that
|
||||
// announcement back against what the code declares.
|
||||
func TestCheckReadsTheModulesAnnouncementOffARealBus(t *testing.T) {
|
||||
declared := []announce.State{{Name: "accounts", Readable: true}, {Name: "drafts"}}
|
||||
got := Check(t, "mesh-mail", declared, func(url string) func() {
|
||||
conn, err := process.Connect(process.Account{URL: url, Module: "mesh-mail", Node: "novox", User: "u", Password: "p"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := conn.Keeps(declared...); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return conn.Close
|
||||
})
|
||||
if got.Node != "novox" {
|
||||
t.Fatalf("announced from %q", got.Node)
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
package announcetest
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/announce"
|
||||
)
|
||||
|
||||
// The comparison is exact up to order: a state missing, one extra, a readability that differs or another module's
|
||||
// announcement each fail, naming what differs.
|
||||
func TestAnAnnouncementDiffersFromTheCodeByNameAndReadability(t *testing.T) {
|
||||
got := announce.States{Module: "m", States: []announce.State{{Name: "a", Readable: true}, {Name: "b"}}}
|
||||
if p := differs(got, "m", []announce.State{{Name: "b"}, {Name: "a", Readable: true}}); p != "" {
|
||||
t.Fatalf("an equal announcement differs: %s", p)
|
||||
}
|
||||
for _, want := range [][]announce.State{
|
||||
{{Name: "a", Readable: true}},
|
||||
{{Name: "a", Readable: true}, {Name: "b"}, {Name: "c"}},
|
||||
{{Name: "a"}, {Name: "b"}},
|
||||
} {
|
||||
if p := differs(got, "m", want); p == "" || !strings.Contains(p, "declares") {
|
||||
t.Errorf("want %v: %q", want, p)
|
||||
}
|
||||
}
|
||||
if p := differs(got, "other", nil); !strings.Contains(p, `"other"`) {
|
||||
t.Errorf("another module's announcement: %q", p)
|
||||
}
|
||||
if p := differs(announce.States{Module: "m"}, "m", nil); p != "" {
|
||||
t.Errorf("a module keeping nothing: %q", p)
|
||||
}
|
||||
}
|
||||
@@ -1,296 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/outward"
|
||||
)
|
||||
|
||||
// Call is one call the runner serves, as the gate judges it: a module's tool, or a seat verb its module serves.
|
||||
type Call struct {
|
||||
// Address is the address as called: `<module>.<tool>` or `<seat>.<verb>`.
|
||||
Address string
|
||||
// Module is the module whose bundle serves it; Seat the seat of a verb, empty for a tool; Name the tool or the verb.
|
||||
Module, Seat, Name string
|
||||
// Fingerprint is the build the bundle serves, from its composed environment (§5).
|
||||
Fingerprint string
|
||||
// Caller is the caller the bus proved.
|
||||
Caller string
|
||||
// Args are the call's arguments, a JSON object.
|
||||
Args json.RawMessage
|
||||
// Description is the tool's or verb's own description, whose first line the ask explains it by.
|
||||
Description string
|
||||
// Own says it is one of the runner's own tools served under the module's name (its ledger repairs), not its bundle's.
|
||||
Own bool
|
||||
}
|
||||
|
||||
// MostArguments is the most a marked call's arguments may hold: more is refused, asking nothing (§9).
|
||||
const MostArguments = 16 * 1024
|
||||
|
||||
// arguments are a call's arguments by name; a body that is not an object is refused.
|
||||
func arguments(raw json.RawMessage) (map[string]json.RawMessage, error) {
|
||||
args := map[string]json.RawMessage{}
|
||||
if len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
|
||||
return args, nil
|
||||
}
|
||||
if err := json.Unmarshal(raw, &args); err != nil {
|
||||
return nil, fmt.Errorf("its arguments are not a JSON object: %v", err)
|
||||
}
|
||||
return args, nil
|
||||
}
|
||||
|
||||
// refusedUnasked says why a marked call is refused at once, asking nothing (§9): its arguments hold a secret's shape, or are
|
||||
// more than MostArguments.
|
||||
func refusedUnasked(raw json.RawMessage) string {
|
||||
if len(raw) > MostArguments {
|
||||
return fmt.Sprintf("its arguments are %d bytes, more than the %d a call that needs the operator's approval may carry", len(raw), MostArguments)
|
||||
}
|
||||
if r, ok := outward.CheckSecret(string(raw)); !ok {
|
||||
return fmt.Sprintf("its arguments hold %s, which no ask may carry", r.What)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// ActOf is the act a marked call binds (§8): `address`, `node` (none given for a process holder, lead answer A-255), `module` and `tool` or `seat` and `verb`, `build`, and
|
||||
// `arg.<name>` for each argument given, a string as itself and any other value as its JSON in RFC 8785's canonical form.
|
||||
func ActOf(c Call, node string) (operatorask.Act, error) {
|
||||
act := operatorask.Act{"address": c.Address, "build": c.Fingerprint}
|
||||
if node != "" { // a process holder's act names none (lead answer A-255)
|
||||
act["node"] = node
|
||||
}
|
||||
if c.Seat == "" {
|
||||
act["module"], act["tool"] = c.Module, c.Name
|
||||
} else {
|
||||
act["seat"], act["verb"] = c.Seat, c.Name
|
||||
}
|
||||
args, err := arguments(c.Args)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for name, raw := range args {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
act["arg."+name] = s
|
||||
continue
|
||||
}
|
||||
canon, err := Canonical(raw)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the argument %s cannot be put in canonical form: %v", name, err)
|
||||
}
|
||||
act["arg."+name] = canon
|
||||
}
|
||||
return act, nil
|
||||
}
|
||||
|
||||
// Canonical is a JSON value in RFC 8785's canonical form: no white space, an object's members sorted by their names' UTF-16
|
||||
// code units, a number as ECMAScript writes a double, a string with only the escapes the RFC names.
|
||||
func Canonical(raw json.RawMessage) (string, error) {
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
var v any
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
var b strings.Builder
|
||||
if err := canonicalValue(&b, v); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return b.String(), nil
|
||||
}
|
||||
|
||||
func canonicalValue(b *strings.Builder, v any) error {
|
||||
switch x := v.(type) {
|
||||
case nil:
|
||||
b.WriteString("null")
|
||||
case bool:
|
||||
b.WriteString(strconv.FormatBool(x))
|
||||
case json.Number:
|
||||
f, err := strconv.ParseFloat(string(x), 64)
|
||||
if err != nil || math.IsInf(f, 0) || math.IsNaN(f) {
|
||||
return fmt.Errorf("the number %s is not a double", x)
|
||||
}
|
||||
b.WriteString(es6Number(f))
|
||||
case string:
|
||||
canonicalString(b, x)
|
||||
case []any:
|
||||
b.WriteByte('[')
|
||||
for i, e := range x {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
if err := canonicalValue(b, e); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
b.WriteByte(']')
|
||||
case map[string]any:
|
||||
names := make([]string, 0, len(x))
|
||||
for k := range x {
|
||||
names = append(names, k)
|
||||
}
|
||||
sort.Slice(names, func(i, j int) bool { return utf16Less(names[i], names[j]) })
|
||||
b.WriteByte('{')
|
||||
for i, k := range names {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
canonicalString(b, k)
|
||||
b.WriteByte(':')
|
||||
if err := canonicalValue(b, x[k]); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
b.WriteByte('}')
|
||||
default:
|
||||
return fmt.Errorf("a value of type %T", v)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// es6Number is a double as ECMAScript's Number.prototype.toString writes it (RFC 8785 §3.2.2.3).
|
||||
func es6Number(f float64) string {
|
||||
if f == 0 {
|
||||
return "0"
|
||||
}
|
||||
abs := math.Abs(f)
|
||||
if abs >= 1e-6 && abs < 1e21 {
|
||||
return strconv.FormatFloat(f, 'f', -1, 64)
|
||||
}
|
||||
s := strconv.FormatFloat(f, 'e', -1, 64) // 1e+21, 1.5e-07
|
||||
mant, exp, _ := strings.Cut(s, "e")
|
||||
sign, digits := exp[:1], strings.TrimLeft(exp[1:], "0")
|
||||
return mant + "e" + sign + digits
|
||||
}
|
||||
|
||||
func canonicalString(b *strings.Builder, s string) {
|
||||
b.WriteByte('"')
|
||||
for _, r := range s {
|
||||
switch {
|
||||
case r == '"':
|
||||
b.WriteString(`\"`)
|
||||
case r == '\\':
|
||||
b.WriteString(`\\`)
|
||||
case r == '\b':
|
||||
b.WriteString(`\b`)
|
||||
case r == '\f':
|
||||
b.WriteString(`\f`)
|
||||
case r == '\n':
|
||||
b.WriteString(`\n`)
|
||||
case r == '\r':
|
||||
b.WriteString(`\r`)
|
||||
case r == '\t':
|
||||
b.WriteString(`\t`)
|
||||
case r < 0x20:
|
||||
fmt.Fprintf(b, `\u%04x`, r)
|
||||
default:
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
b.WriteByte('"')
|
||||
}
|
||||
|
||||
func utf16Less(a, b string) bool {
|
||||
ua, ub := utf16Of(a), utf16Of(b)
|
||||
for i := 0; i < len(ua) && i < len(ub); i++ {
|
||||
if ua[i] != ub[i] {
|
||||
return ua[i] < ub[i]
|
||||
}
|
||||
}
|
||||
return len(ua) < len(ub)
|
||||
}
|
||||
|
||||
func utf16Of(s string) []uint16 {
|
||||
out := make([]uint16, 0, utf8.RuneCountInString(s))
|
||||
for _, r := range s {
|
||||
if r >= 0x10000 {
|
||||
r -= 0x10000
|
||||
out = append(out, uint16(0xD800+(r>>10)), uint16(0xDC00+(r&0x3FF)))
|
||||
} else {
|
||||
out = append(out, uint16(r))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Fingerprint is how an act is shown: its digest's first 24 hexadecimal digits in groups of four (§10).
|
||||
func Fingerprint(digest string) string {
|
||||
hex := strings.TrimPrefix(digest, "sha256:")
|
||||
if len(hex) > 24 {
|
||||
hex = hex[:24]
|
||||
}
|
||||
var groups []string
|
||||
for len(hex) > 4 {
|
||||
groups, hex = append(groups, hex[:4]), hex[4:]
|
||||
}
|
||||
return strings.Join(append(groups, hex), " ")
|
||||
}
|
||||
|
||||
// The bounds of the arguments shown in an ask (§11).
|
||||
const (
|
||||
ShownBytes, ShownLines = 2000, 40
|
||||
PartBytes, PartLines = 1200, 20
|
||||
)
|
||||
|
||||
// shown is a call's arguments as an ask shows them (§11), one per line as "name: value", and whether all were shown whole:
|
||||
// an argument whole where outward.Check passes it, else its shape, ‹path›, ‹address› or ‹withheld›; the whole when it is at
|
||||
// most ShownBytes and ShownLines, else its first PartLines lines and PartBytes bytes, marked NOT SHOWN IN FULL.
|
||||
func shown(raw json.RawMessage) (string, bool) {
|
||||
args, _ := arguments(raw)
|
||||
names := make([]string, 0, len(args))
|
||||
for n := range args {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
whole := true
|
||||
var lines []string
|
||||
for _, n := range names {
|
||||
var value string
|
||||
if json.Unmarshal(args[n], &value) != nil {
|
||||
value = string(args[n])
|
||||
}
|
||||
if r, ok := outward.Check(value); !ok {
|
||||
value, whole = shapeOf(r), false
|
||||
}
|
||||
lines = append(lines, n+": "+value)
|
||||
}
|
||||
text := strings.Join(lines, "\n")
|
||||
if all := strings.Split(text, "\n"); len(text) > ShownBytes || len(all) > ShownLines {
|
||||
if len(all) > PartLines {
|
||||
all = all[:PartLines]
|
||||
}
|
||||
text = strings.Join(all, "\n")
|
||||
if len(text) > PartBytes {
|
||||
text = cutBytes(text, PartBytes)
|
||||
}
|
||||
text += "\nNOT SHOWN IN FULL"
|
||||
whole = false
|
||||
}
|
||||
return text, whole
|
||||
}
|
||||
|
||||
// shapeOf is what an argument outward.Check refuses is shown as.
|
||||
func shapeOf(r outward.Refusal) string {
|
||||
switch r.Class {
|
||||
case "path":
|
||||
return "‹path›"
|
||||
case "address":
|
||||
return "‹address›"
|
||||
}
|
||||
return "‹withheld›"
|
||||
}
|
||||
|
||||
// cutBytes is s at most n bytes, cut at a character's end.
|
||||
func cutBytes(s string, n int) string {
|
||||
for n > 0 && !utf8.RuneStart(s[n]) {
|
||||
n--
|
||||
}
|
||||
return s[:n]
|
||||
}
|
||||
@@ -1,80 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// call is a fixture call of the marked tool wipe.
|
||||
func call(name, args string) Call {
|
||||
return Call{Address: "shell." + name, Module: "shell", Name: name, Fingerprint: "src2:1", Caller: "agent", Args: json.RawMessage(args),
|
||||
Description: "Wipes a scratch directory.\nMore words."}
|
||||
}
|
||||
|
||||
// Design 46 7-approval-mark §8, §10, §11: the act's fields give each call its digest, and the ask shows a call's arguments
|
||||
// in their shape and its headline cut (the gate's other cases run on the tool runner's fixture mesh, node-tools
|
||||
// internal/approval).
|
||||
func TestTheActAndItsAsk(t *testing.T) {
|
||||
base := call("wipe", `{"dir":"scratch","n":1}`)
|
||||
digest := func(c Call) string {
|
||||
act, err := ActOf(c, "n1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
d, _ := operatorask.ActDigest(act)
|
||||
return d
|
||||
}
|
||||
act, _ := ActOf(base, "n1")
|
||||
want := operatorask.Act{"address": "shell.wipe", "node": "n1", "module": "shell", "tool": "wipe", "build": "src2:1",
|
||||
"arg.dir": "scratch", "arg.n": "1"}
|
||||
if len(act) != len(want) {
|
||||
t.Fatalf("act %v", act)
|
||||
}
|
||||
for k, v := range want {
|
||||
if act[k] != v {
|
||||
t.Errorf("act[%s] = %q, want %q", k, act[k], v)
|
||||
}
|
||||
}
|
||||
seen := map[string]string{digest(base): "the call"}
|
||||
for name, c := range map[string]Call{
|
||||
"an omitted argument": call("wipe", `{"dir":"scratch"}`),
|
||||
"another number": call("wipe", `{"dir":"scratch","n":2}`),
|
||||
"an object": call("wipe", `{"dir":{"name":"scratch"},"n":1}`),
|
||||
"a changed build": func() Call { c := base; c.Fingerprint = "src2:2"; return c }(),
|
||||
"another node's seat": {Address: "node-login-shell.execute", Module: "shell", Seat: "node-login-shell", Name: "execute", Fingerprint: "src2:1"},
|
||||
} {
|
||||
d := digest(c)
|
||||
if other, dup := seen[d]; dup {
|
||||
t.Errorf("%s gives the digest of %s", name, other)
|
||||
}
|
||||
seen[d] = name
|
||||
}
|
||||
if got, _ := Canonical(json.RawMessage(`{"b":[1.0,2.5e-7,1e21],"a":"é<"}`)); got != `{"a":"é<","b":[1,2.5e-7,1e+21]}` {
|
||||
t.Errorf("canonical form %s", got)
|
||||
}
|
||||
// §11: a path, an address and a long argument are shown in their shape and cut.
|
||||
text, whole := shown(json.RawMessage(`{"dir":"/home/someone/scratch","host":"10.0.0.7","ok":"yes"}`))
|
||||
if whole || text != "dir: ‹path›\nhost: ‹address›\nok: yes" {
|
||||
t.Errorf("shown %q whole %v", text, whole)
|
||||
}
|
||||
long, _ := json.Marshal(map[string]string{"text": strings.Repeat("word ", 500)})
|
||||
if text, whole := shown(long); whole || !strings.HasSuffix(text, "NOT SHOWN IN FULL") || len(text) > PartBytes+len("\nNOT SHOWN IN FULL") {
|
||||
t.Errorf("a long argument shown whole %v, %d bytes", whole, len(text))
|
||||
}
|
||||
// §10: the headline, cut with … to 60 characters, ending Part shown.
|
||||
for _, tc := range []struct{ name, node, want string }{
|
||||
{"wipe", "n1", "Run wipe on n1?"},
|
||||
{strings.Repeat("x", 50), "n1", "Run " + strings.Repeat("x", 50) + "?"},
|
||||
{strings.Repeat("y", 80), "n1", "Run " + strings.Repeat("y", 54) + "…?"},
|
||||
} {
|
||||
if got := headlineOf("Run", tc.name, tc.node, true); got != tc.want || len([]rune(got)) > 60 {
|
||||
t.Errorf("headline %q, want %q", got, tc.want)
|
||||
}
|
||||
}
|
||||
if got := headlineOf("Run", "wipe", "n1", false); got != "Run wipe on n1? Part shown" {
|
||||
t.Errorf("headline %q", got)
|
||||
}
|
||||
}
|
||||
@@ -1,239 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// StateName is the runner's own state its asks and claims are kept in, per machine (§13).
|
||||
const StateName = "approvals"
|
||||
|
||||
// Store is the runner's state `approvals` (§13): a key's value, a create that refuses a key that exists and an update by
|
||||
// compare-and-set, each failing with stdio.ErrChanged when another write came first; a put, and its keys.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
Put(key string, value any) (uint64, error)
|
||||
Keys() ([]string, error)
|
||||
}
|
||||
|
||||
// The bounds of an ask (§10, §12, §13, §14, §17).
|
||||
const (
|
||||
AskLasts = 60 * time.Minute
|
||||
RecordWithin = 5 * time.Second
|
||||
RunWithin = 30 * time.Minute
|
||||
DeclinedFor = 10 * time.Minute
|
||||
MostKeptValue = 256 * 1024
|
||||
OnExpiry = "nothing runs; call again to ask anew"
|
||||
)
|
||||
|
||||
// The options every ask offers (§10), both at level approve, both binding the act's digest.
|
||||
const (
|
||||
OptionApprove = "approve"
|
||||
OptionDecline = "decline"
|
||||
)
|
||||
|
||||
// actKey is the key naming the newest ask of an act (§13), its digest escaped as a key's name is, since the bus holds no `:` in a key (lead answer A-209).
|
||||
func (g *Gate) actKey(digest string) string { return g.prefix() + "act." + keys.EscapeName(digest) }
|
||||
|
||||
// prefix is what the gate's keys begin with: the tool runner's machine, `<node>.`, since its state is shared by every
|
||||
// machine's runner (§13); none for a module's process, whose state approvals is its module's own (design 47 G §677).
|
||||
func (g *Gate) prefix() string {
|
||||
if g.d.Holding {
|
||||
return ""
|
||||
}
|
||||
return g.d.Node + "."
|
||||
}
|
||||
|
||||
// askKey is the key the client keeps an ask under (§13).
|
||||
func (g *Gate) askKey(id string) string { return g.prefix() + "ask." + id }
|
||||
|
||||
// newID is an ask's id: 128 random bits.
|
||||
func newID() string {
|
||||
var b [16]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// askFor is the ask a marked call raises (§10, §11).
|
||||
func (g *Gate) askFor(c Call, digest, id string) Ask {
|
||||
args, whole := shown(c.Args)
|
||||
name := c.Name
|
||||
if c.Seat != "" {
|
||||
name = c.Seat + "." + c.Name
|
||||
}
|
||||
headline := headlineOf("Run", name, g.d.Node, whole)
|
||||
first, _, _ := strings.Cut(strings.TrimSpace(c.Description), "\n")
|
||||
if first == "" {
|
||||
first = name
|
||||
}
|
||||
how := "called through the mesh's tools, which cannot tell who called."
|
||||
if g.d.Holding {
|
||||
how = "called by " + c.Caller + "."
|
||||
}
|
||||
lines := []string{first, "On " + g.d.Node + ", " + how}
|
||||
if args != "" {
|
||||
lines = append(lines, args)
|
||||
}
|
||||
if !whole {
|
||||
read := "mesh_approvals on " + g.d.Node
|
||||
if g.d.Holding {
|
||||
read = c.Seat + ".approvals"
|
||||
}
|
||||
lines = append(lines, "The whole is read with "+read+", whose fingerprint must be the one on the phone.")
|
||||
}
|
||||
lines = append(lines, "Fingerprint "+Fingerprint(digest))
|
||||
return Ask{ID: id, About: digest, Headline: headline,
|
||||
Explanation: strings.Join(lines, "\n"), Expires: formatTime(g.d.Now().Add(AskLasts)), OnExpiry: OnExpiry,
|
||||
Options: []Option{
|
||||
{ID: OptionApprove, Label: "Approve", Does: "the same call, made again within 30 minutes, runs once", Binds: digest},
|
||||
{ID: OptionDecline, Label: "Decline", Does: "nothing runs; the same call is not asked again for 10 minutes", Binds: digest}}}
|
||||
}
|
||||
|
||||
// headlineOf is "<verb> <names> on <node>?", "<verb> <names>?" when that is longer than the router allows, and the names cut
|
||||
// with "…" when that too is longer; ending "Part shown" whenever an argument is not shown whole (§10, Q-HL). verb is "Run"
|
||||
// for a call and "Wait on" for a wait (design 34 every-event §35).
|
||||
func headlineOf(verb, name, node string, whole bool) string {
|
||||
suffix := ""
|
||||
if !whole {
|
||||
suffix = " Part shown"
|
||||
}
|
||||
max := HeadlineLength
|
||||
fits := func(s string) bool { return len([]rune(s)) <= max }
|
||||
if h := verb + " " + name + " on " + node + "?" + suffix; fits(h) {
|
||||
return h
|
||||
}
|
||||
if h := verb + " " + name + "?" + suffix; fits(h) {
|
||||
return h
|
||||
}
|
||||
room := max - len([]rune(verb+" …?"+suffix))
|
||||
r := []rune(name)
|
||||
if room < 1 {
|
||||
room = 1
|
||||
}
|
||||
if len(r) > room {
|
||||
r = r[:room]
|
||||
}
|
||||
return verb + " " + string(r) + "…?" + suffix
|
||||
}
|
||||
|
||||
// firstCall is a marked call whose act has no ask open: refused at once when its arguments may not be asked (§9), else
|
||||
// asked (§10, §12).
|
||||
func (g *Gate) firstCall(c Call) (any, error) {
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
return nil, g.raise(c, act, digest, "")
|
||||
}
|
||||
|
||||
// raise raises the ask of a marked call whose act has none open (§10): kept in the runner's state with the act and the whole
|
||||
// arguments, published under the module's name, its act's key naming it; then the 5 seconds for the router's record and
|
||||
// the first answer (§12), always an error, never a result.
|
||||
func (g *Gate) raise(c Call, act operatorask.Act, digest, markedSince string) error {
|
||||
id := newID()
|
||||
if err := g.keepAsk(c, act, digest, g.askFor(c, digest, id)); err != nil {
|
||||
return err
|
||||
}
|
||||
return g.firstAnswer(c, id, digest, markedSince)
|
||||
}
|
||||
|
||||
// keepAsk raises an ask and keeps it in the runner's state with the act and the whole arguments, its act's key naming it
|
||||
// (§10, §13); refused, as an error, when it cannot be.
|
||||
func (g *Gate) keepAsk(c Call, act operatorask.Act, digest string, a Ask) error {
|
||||
if size := keptSize(a, act, c.Args); size > MostKeptValue {
|
||||
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
|
||||
}
|
||||
if err := g.d.Channel.Ask(c.Module, a); err != nil {
|
||||
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
|
||||
}
|
||||
if err := g.keepCall(a.ID, act, c.Args); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
|
||||
}
|
||||
if _, err := g.d.State.Put(g.actKey(digest), a.ID); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its act could not be kept: "+err.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// keptSize is the bytes the kept ask would take.
|
||||
func keptSize(a Ask, act operatorask.Act, args json.RawMessage) int {
|
||||
raw, _ := json.Marshal(Kept[Ask, Warrant]{Ask: a, Act: act, Arguments: args})
|
||||
return len(raw)
|
||||
}
|
||||
|
||||
// keepCall writes the act and the whole arguments beside the ask the client kept (§13).
|
||||
func (g *Gate) keepCall(id string, act operatorask.Act, args json.RawMessage) error {
|
||||
e, err := g.d.State.Get(g.askKey(id))
|
||||
if err != nil || e == nil {
|
||||
return fmt.Errorf("the kept ask %s cannot be read: %v", id, err)
|
||||
}
|
||||
var k kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return err
|
||||
}
|
||||
k.Act, k.Arguments = act, args
|
||||
_, err = g.d.State.Update(g.askKey(id), k, e.Revision)
|
||||
return err
|
||||
}
|
||||
|
||||
// firstAnswer waits up to RecordWithin for the router's record of the ask and says the call was not run (§12).
|
||||
func (g *Gate) firstAnswer(c Call, id, digest, markedSince string) error {
|
||||
rec, _ := g.awaitRecord(c.Module, id)
|
||||
since := ""
|
||||
if markedSince != "" {
|
||||
since = "; marked since revision " + markedSince
|
||||
}
|
||||
if rec == nil {
|
||||
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make the same call again later%s",
|
||||
id, Fingerprint(digest), since)
|
||||
}
|
||||
return fmt.Errorf("Not run: this needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make the same call "+
|
||||
"again within 30 minutes of the approval; the ask expires at %s%s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
|
||||
id, Fingerprint(digest), rec.Expires, since)
|
||||
}
|
||||
|
||||
// record is the router's record of an ask under the module's name, nil when it holds none.
|
||||
func (g *Gate) record(module, id string) (*Record, error) {
|
||||
return g.d.Channel.Record(module, id)
|
||||
}
|
||||
|
||||
// awaitRecord reads the router's record of a new ask until it is there, within RecordWithin: the seat answers a key's
|
||||
// value and says nothing when the router takes an ask, so it is read again after a growing pause (§12).
|
||||
func (g *Gate) awaitRecord(module, id string) (*Record, error) {
|
||||
until := g.d.Now().Add(RecordWithin)
|
||||
pause := 50 * time.Millisecond
|
||||
for {
|
||||
rec, err := g.record(module, id)
|
||||
if rec != nil || g.d.Now().Add(pause).After(until) {
|
||||
return rec, err
|
||||
}
|
||||
<-g.d.After(pause)
|
||||
pause *= 2
|
||||
}
|
||||
}
|
||||
|
||||
// refused is a marked call refused, run nothing, written in the runner's log with the ask's fingerprint and never its
|
||||
// arguments (§19).
|
||||
func (g *Gate) refused(c Call, digest, why string) error {
|
||||
g.d.Logf("[approval] %s on %s is refused (fingerprint %s): %s", c.Address, g.d.Node, orNone(Fingerprint(digest)), why)
|
||||
return fmt.Errorf("Not run: %s needs the operator's approval and %s", c.Address, why)
|
||||
}
|
||||
@@ -1,115 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// What the gate reads beyond itself crosses in the gate's own terms, its caller translating from the generated contracts
|
||||
// (design 47 G lead answers A-219, A-224, A-228): the operator channel (Channel), the catalog's entries (Entry, through
|
||||
// Catalog), the finding writer (FindingWriter) and the seat holders (Deps.Holder). No form of a generated contract is
|
||||
// copied here: what the router keeps whole (an ask, a warrant) the gate holds as the bytes it was given.
|
||||
|
||||
// Channel is the operator channel as the gate reaches it (A-219, A-228): operatorchannelclient implements it over the
|
||||
// generated types, keeping each ask in the gate's own state under the marked module's name.
|
||||
type Channel interface {
|
||||
// Ask checks the ask, keeps it in the gate's state as a Kept and publishes it under module's name, at the level
|
||||
// approve, to the operator (design 46 7-approval-mark §10).
|
||||
Ask(module string, a Ask) error
|
||||
// Record is the router's record of module's ask id, read under module's own name; nil while the router holds none
|
||||
// (design 47 G §700).
|
||||
Record(module, id string) (*Record, error)
|
||||
// Claim takes the warrant once (design 46 7-approval-mark §14, §15): it holds it to the ask kept, checks that the
|
||||
// chosen option binds act and marks the kept ask acted, then calls run; act's run happens only when it answers
|
||||
// Claimed. A warrant already taken answers ClaimDone, one that does not hold ClaimRefused with why.
|
||||
Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error)
|
||||
// Cancel takes module's ask back.
|
||||
Cancel(module, id string) error
|
||||
// Decided is the subject the router's `decided` for module is heard on.
|
||||
Decided(module string) string
|
||||
// Seat is the seat an ask is raised on, whose holder must be there for a marked call to be asked (§19).
|
||||
Seat() string
|
||||
}
|
||||
|
||||
// Claim is what came of a warrant handed to Channel.Claim.
|
||||
type Claim string
|
||||
|
||||
const (
|
||||
Claimed Claim = "acted"
|
||||
ClaimRefused Claim = "refused"
|
||||
ClaimDone Claim = "already acted on"
|
||||
// ClaimEnded is a warrant of an ask that ended without a choice.
|
||||
ClaimEnded Claim = "ended without a choice"
|
||||
)
|
||||
|
||||
// Ask is an ask the gate raises: two options, each binding the act's digest (design 46 7-approval-mark §10, §11).
|
||||
type Ask struct {
|
||||
ID, About, Headline, Explanation, Expires, OnExpiry string
|
||||
Options []Option
|
||||
}
|
||||
|
||||
// Option is one option of an ask.
|
||||
type Option struct{ ID, Label, Does, Binds string }
|
||||
|
||||
// Record is the router's record of an ask as the gate reads it.
|
||||
type Record struct {
|
||||
State, Answer, Opened, Expires string
|
||||
OfferedOn []string
|
||||
Warrant *Warrant
|
||||
}
|
||||
|
||||
// Warrant is a person's choice as the gate reads it; Whole is the warrant as the router recorded it, which Claim holds to
|
||||
// the kept ask.
|
||||
type Warrant struct {
|
||||
Ask, Asker, Option, Channel, At string
|
||||
Whole json.RawMessage
|
||||
}
|
||||
|
||||
// Kept is one ask as its asker keeps it (design 46 7-approval-mark §13): the ask, beside it the act the ask binds, the
|
||||
// call's whole arguments, and whether the act's run was recorded in the hand-act log. A is the ask's form and W the
|
||||
// warrant's: operatorchannelclient's Kept is Kept of the generated contract's, the gate's of the bytes kept (A-228).
|
||||
type Kept[A, W any] struct {
|
||||
Ask A `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended string `json:"ended,omitempty"`
|
||||
Answer *W `json:"answer,omitempty"`
|
||||
// Act is the act the ask's options bind, as its asker digested it.
|
||||
Act operatorask.Act `json:"act,omitempty"`
|
||||
// Arguments are the asked call's arguments whole.
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
// Recorded says the act's run is in the hand-act log (`mesh-hand-acts.warranted` answered).
|
||||
Recorded bool `json:"recorded,omitempty"`
|
||||
}
|
||||
|
||||
// kept is the gate's own reading of a Kept: the ask and the warrant as the bytes kept, so a write back keeps them whole.
|
||||
type kept = Kept[json.RawMessage, json.RawMessage]
|
||||
|
||||
// HeadlineLength is the longest headline the router takes, in characters; operatorchannelclient re-exports it.
|
||||
const HeadlineLength = 60
|
||||
|
||||
// Entry is a module's entry in the catalog as the marks are read from it (design 46 7-approval-mark §5): its module, the
|
||||
// build's fingerprint, the seat verbs it serves as `<seat>.<verb>`, and its manifest.
|
||||
type Entry struct {
|
||||
Module, Fingerprint string
|
||||
Serves []string
|
||||
Manifest json.RawMessage
|
||||
}
|
||||
|
||||
// formatTime is a time as the router's records carry it: RFC 3339 to the nanosecond; the zero time is empty.
|
||||
func formatTime(t time.Time) string {
|
||||
if t.IsZero() {
|
||||
return ""
|
||||
}
|
||||
return t.Format(time.RFC3339Nano)
|
||||
}
|
||||
|
||||
// parseTime reads such a time; empty is the zero time.
|
||||
func parseTime(s string) (time.Time, error) {
|
||||
if s == "" {
|
||||
return time.Time{}, nil
|
||||
}
|
||||
return time.Parse(time.RFC3339Nano, s)
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// UnreadableAfter is how long the state approvals fails before the runner's finding says so (§19).
|
||||
const UnreadableAfter = 5 * time.Minute
|
||||
|
||||
// unreadableKey is the bound key whose expiry raises `approvals-unreadable` (lead answer A-157).
|
||||
func (g *Gate) unreadableKey() string { return g.d.Node + ".unreadable." + StateName }
|
||||
|
||||
// held says why no marked call can be asked now, or "" (§19): no module holds operator-channel, as the assigner's view
|
||||
// of the seat's holders says, or that view is not read.
|
||||
func (g *Gate) held() string {
|
||||
if g.d.Channel == nil {
|
||||
return "the tool runner reaches no operator channel to ask on"
|
||||
}
|
||||
if g.d.Holder == nil && g.d.Holding {
|
||||
return "" // a process holder reads no view of the seats' holders: an ask nobody takes is said by its gate-unanswered (design 47 G §677)
|
||||
}
|
||||
if g.d.Holder == nil {
|
||||
return "the tool runner reads no view of the seats' holders, so it cannot tell that a module holds " + g.d.Channel.Seat()
|
||||
}
|
||||
on := g.d.Channel.Seat()
|
||||
ok, err := g.d.Holder(on)
|
||||
switch {
|
||||
case err != nil:
|
||||
return fmt.Sprintf("whether a module holds %s cannot be read: %v", on, err)
|
||||
case !ok:
|
||||
return "no module holds " + on + ", so nobody can be asked"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// stateFailed notes that the state approvals could not be read or written: no marked call runs on this machine until it
|
||||
// can, the failed read or write is made again at the next marked call and at the next membership change (A-157), and
|
||||
// five minutes on the runner's finding says it (§19).
|
||||
func (g *Gate) stateFailed(err error) {
|
||||
g.mu.Lock()
|
||||
first := g.failingSince.IsZero()
|
||||
if first {
|
||||
g.failingSince = g.d.Now()
|
||||
}
|
||||
g.mu.Unlock()
|
||||
if first {
|
||||
g.d.Logf("[approval] the state %s on %s cannot be read or written (%v): no marked call runs here until it can", StateName, g.d.Node, err)
|
||||
if g.d.Bounds != nil && !g.d.Holding {
|
||||
if e := g.d.Bounds.CreateTTL(g.unreadableKey(), map[string]string{"since": g.d.Now().UTC().Format(time.RFC3339)}, UnreadableAfter); e != nil {
|
||||
g.d.Logf("[approval] the bound of the state %s's failure is not kept: %v", StateName, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// stateRead notes that the state approvals was read or written: a failure noted before ends, its finding cleared.
|
||||
func (g *Gate) stateRead() {
|
||||
g.mu.Lock()
|
||||
was := !g.failingSince.IsZero()
|
||||
g.failingSince = time.Time{}
|
||||
g.mu.Unlock()
|
||||
if !was {
|
||||
return
|
||||
}
|
||||
if g.d.Bounds != nil && !g.d.Holding {
|
||||
_ = g.d.Bounds.Delete(g.unreadableKey())
|
||||
}
|
||||
g.finding(StateName, KindApprovalsUnreadable, false, "the state "+StateName+" is read again", "")
|
||||
}
|
||||
|
||||
// failing says the state approvals is failing now.
|
||||
func (g *Gate) failing() bool {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return !g.failingSince.IsZero()
|
||||
}
|
||||
|
||||
// StateAgain makes the failed read of the state approvals again: at the next membership change the runner hears (A-157).
|
||||
func (g *Gate) StateAgain() {
|
||||
if g.d.State == nil || !g.failing() {
|
||||
return
|
||||
}
|
||||
if _, err := g.d.State.Keys(); err != nil {
|
||||
return
|
||||
}
|
||||
g.stateRead()
|
||||
}
|
||||
|
||||
// unreadableExpired is the bound key of a failing state expiring: the finding raised while the state still fails.
|
||||
func (g *Gate) unreadableExpired() {
|
||||
g.mu.Lock()
|
||||
since := g.failingSince
|
||||
g.mu.Unlock()
|
||||
if since.IsZero() {
|
||||
return
|
||||
}
|
||||
g.finding(StateName, KindApprovalsUnreadable, true, fmt.Sprintf("the state %s on %s cannot be read or written since %s: no marked call "+
|
||||
"runs here until it can", StateName, g.d.Node, since.UTC().Format(time.RFC3339)), "approvals unreadable on "+g.d.Node)
|
||||
}
|
||||
@@ -1,178 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The runner's findings of the gate (design 46 7-approval-mark §4, §16, §19), written through its held writer.
|
||||
const (
|
||||
KindMarkUnserved = "mark-unserved"
|
||||
KindCallUnanswered = "call-unanswered"
|
||||
KindApprovalsUnreadable = "approvals-unreadable"
|
||||
)
|
||||
|
||||
// Kinds are the finding kinds the gate writes through the runner's writer.
|
||||
var Kinds = []string{KindMarkUnserved, KindCallUnanswered, KindApprovalsUnreadable}
|
||||
|
||||
// FindingWriter raises and clears the gate's findings by kind and id (A-228): its caller writes each as the contract's
|
||||
// finding, about its own module on its machine, through its held writer.
|
||||
type FindingWriter interface {
|
||||
Raise(ctx context.Context, kind, id, summary, headline string) error
|
||||
Clear(ctx context.Context, kind, id, why string) error
|
||||
}
|
||||
|
||||
// Deps is what the gate reaches the mesh through: the runner's connection in its one place, fakes in a test.
|
||||
type Deps struct {
|
||||
// Node is the machine the runner serves; Runner its own module's name, the subject of its findings.
|
||||
Node, Runner string
|
||||
// Catalog is the catalog half the marks are read from.
|
||||
Catalog Catalog
|
||||
// Wait waits, within the bound, for the catalog half's first view, and is nil once it is read or why not.
|
||||
Wait func(within time.Duration) error
|
||||
// Findings is the runner's held writer; nil says a finding in the log only.
|
||||
Findings FindingWriter
|
||||
Logf func(string, ...any)
|
||||
Now func() time.Time
|
||||
// After is the clock's pause; nil is time.After.
|
||||
After func(time.Duration) <-chan time.Time
|
||||
// State is the runner's state approvals; nil when its membership binds none, so every marked call is refused (§13).
|
||||
State Store
|
||||
// Channel is the operator channel, its asks kept in State under this machine's keys (A-219, A-228).
|
||||
Channel Channel
|
||||
// Warranted asks `mesh-hand-acts.warranted` {asker, ask}, each try a kept call open while open says so (§16, A-157).
|
||||
Warranted func(asker, ask string, open func() (bool, error)) error
|
||||
// Bounds is the runner's key-ttl state bound (A-157, A-158), or a process holder's state approvals itself (A-255); nil
|
||||
// keeps no bound.
|
||||
Bounds Bounds
|
||||
// Holder says whether a module holds a seat, from the assigner's view of the seats' holders the runner follows (§19).
|
||||
Holder func(seat string) (bool, error)
|
||||
// Build is the tool runner's own fingerprint, read from its MESH_FINGERPRINT at its start: the build a marked wait's
|
||||
// act binds; empty while unknown, when every marked wait is refused (design 34 every-event §35).
|
||||
Build string
|
||||
// Hear waits within the bound for the first event on the subjects whose body match takes, or the caller's going: a
|
||||
// marked wait's ask's `decided` (every-event §36); false when none came.
|
||||
Hear func(ctx context.Context, subjects []string, within time.Duration, match func(json.RawMessage) bool) (bool, error)
|
||||
// Go runs what goes on beside the call (a run's recording); nil is a goroutine.
|
||||
Go func(func())
|
||||
// Holding says the gate is a module's process's seat server's (design 47 G §677): its keys are its module's own, its
|
||||
// calls held as waiting calls and their ends said through Say; Runner is then the holder's module.
|
||||
Holding bool
|
||||
// Say publishes one gated-ended on its subject with the message id given (§677).
|
||||
Say func(subject string, body any, id string) error
|
||||
}
|
||||
|
||||
// Gate is the approval gate: every call of a module the tool runner serves goes through it before the bundle (§20), and a
|
||||
// marked verb a module's process serves before its handler (design 47 G §677; lead answer A-224).
|
||||
type Gate struct {
|
||||
d Deps
|
||||
mu sync.Mutex
|
||||
// registered is each module's tools as its bundle registered them at its last load.
|
||||
registered map[string]map[string]bool
|
||||
// unserved is each module whose marks name a tool its bundle does not register, with those tools.
|
||||
unserved map[string][]string
|
||||
// unmarkedAt is each caller's address it last called unmarked, so a mark added since is named (§12).
|
||||
unmarkedAt map[string]bool
|
||||
// inFlight is each ask whose recording runs now.
|
||||
inFlight map[string]bool
|
||||
// failingSince is when the state approvals began to fail; zero while it is read.
|
||||
failingSince time.Time
|
||||
// handlers are a process holder's marked verbs' handlers, by `<seat>.<verb>` (design 47 G §677).
|
||||
handlers map[string]func(string, json.RawMessage) (any, error)
|
||||
// served is each module this runner serves, with the build its bundle serves, as its last load said (§18).
|
||||
served map[string]string
|
||||
}
|
||||
|
||||
// New is a gate over d.
|
||||
func New(d Deps) *Gate {
|
||||
if d.Logf == nil {
|
||||
d.Logf = func(string, ...any) {}
|
||||
}
|
||||
if d.Now == nil {
|
||||
d.Now = time.Now
|
||||
}
|
||||
if d.After == nil {
|
||||
d.After = time.After
|
||||
}
|
||||
if d.Go == nil {
|
||||
d.Go = func(f func()) { go f() }
|
||||
}
|
||||
return &Gate{d: d, registered: map[string]map[string]bool{}, unserved: map[string][]string{}}
|
||||
}
|
||||
|
||||
// Unserved is, of the module's marked tools, those its bundle does not register (§4): such a bundle is not loaded, its
|
||||
// finding `mark-unserved` raised; a bundle that serves them, or marks that no longer name them, clear it. Before the marks
|
||||
// are read nothing is known, and every marked call is refused until they are (§19).
|
||||
func (g *Gate) Unserved(module, fingerprint string, registered []string) []string {
|
||||
set := map[string]bool{}
|
||||
for _, t := range registered {
|
||||
set[t] = true
|
||||
}
|
||||
g.mu.Lock()
|
||||
g.registered[module] = set
|
||||
if g.served == nil {
|
||||
g.served = map[string]string{}
|
||||
}
|
||||
g.served[module] = fingerprint
|
||||
g.mu.Unlock()
|
||||
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
|
||||
return nil
|
||||
}
|
||||
return g.checkServed(Derive(g.d.Catalog, module, fingerprint))
|
||||
}
|
||||
|
||||
// checkServed raises or clears the module's `mark-unserved` as its marks and its bundle's registrations stand now, and is
|
||||
// the tools it marks and does not register.
|
||||
func (g *Gate) checkServed(m Marks) []string {
|
||||
g.mu.Lock()
|
||||
set := g.registered[m.Module]
|
||||
var missing []string
|
||||
for t := range m.Tools {
|
||||
if !set[t] {
|
||||
missing = append(missing, t)
|
||||
}
|
||||
}
|
||||
sort.Strings(missing)
|
||||
was := g.unserved[m.Module]
|
||||
if len(missing) > 0 {
|
||||
g.unserved[m.Module] = missing
|
||||
} else {
|
||||
delete(g.unserved, m.Module)
|
||||
}
|
||||
g.mu.Unlock()
|
||||
switch {
|
||||
case len(missing) > 0 && strings.Join(was, ",") != strings.Join(missing, ","):
|
||||
g.finding(m.Module, KindMarkUnserved, true, fmt.Sprintf("%s's marks (%s, catalog revision %s) name %s, which its bundle on %s does not "+
|
||||
"register: no tool of %s is served here", m.Module, sourceOf(m, missing[0]), m.Revision, strings.Join(missing, ", "), g.d.Node, m.Module),
|
||||
fmt.Sprintf("%s is not served on %s", m.Module, g.d.Node))
|
||||
case len(missing) == 0 && len(was) > 0:
|
||||
g.finding(m.Module, KindMarkUnserved, false, "a bundle that serves every marked tool, or marks without the tool, is loaded", "")
|
||||
}
|
||||
return missing
|
||||
}
|
||||
|
||||
func sourceOf(m Marks, tool string) string { return m.Tools[tool] }
|
||||
|
||||
// finding writes or clears one of the gate's findings, its id `<node>.<what>` escaped as a key's name is,
|
||||
// about the runner's module on this machine (lead answer A-154).
|
||||
func (g *Gate) finding(what, kind string, raised bool, summary, headline string) {
|
||||
id := g.findingID(what)
|
||||
if g.d.Findings == nil {
|
||||
g.d.Logf("[approval] %s %s: %s", kind, id, summary)
|
||||
return
|
||||
}
|
||||
var err error
|
||||
if raised {
|
||||
err = g.d.Findings.Raise(context.Background(), kind, id, summary, headline)
|
||||
} else {
|
||||
err = g.d.Findings.Clear(context.Background(), kind, id, summary)
|
||||
}
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the finding %s of %s is not kept: %v", kind, id, err)
|
||||
}
|
||||
}
|
||||
@@ -1,699 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// A marked verb a module's process serves is one call with a gate before its running (design 47 G §677): the holder keeps
|
||||
// the call, asks under its own module's name, and on the Approve runs that same call, saying its end on the seat's event
|
||||
// `gated-ended`. These are the holder's half of the gate; the caller's is process.Conn's call.
|
||||
|
||||
// WaitHeader is the bus header a call carries to say it waits for its end on `gated-ended` (lead answer A-244): a header,
|
||||
// never an argument, so no verb's input and no act's digest changes. WaitHeld is its one value.
|
||||
const (
|
||||
WaitHeader = "x-wait"
|
||||
WaitHeld = "held"
|
||||
)
|
||||
|
||||
// Held is the holder's answer to a call that waits (§677): the ask it waits on and its own call's id, in the reply's field
|
||||
// `held` beside result, error and node (A-244).
|
||||
type Held struct {
|
||||
Asked string `json:"asked"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
|
||||
// GatedEndedEvent is the seat's event a call's end is said on, kept last per subject for GatedEndedKeptFor (§677).
|
||||
const (
|
||||
GatedEndedEvent = seatdef.GatedEndedEvent
|
||||
GatedEndedKeptFor = seatdef.GatedEndedKeptDays * 24 * time.Hour
|
||||
)
|
||||
|
||||
// The outcomes of a held call (§677).
|
||||
const (
|
||||
OutcomeRan = "ran"
|
||||
OutcomeDeclined = "declined"
|
||||
OutcomeExpired = "expired"
|
||||
OutcomeRefused = "refused"
|
||||
)
|
||||
|
||||
// KindGateUnanswered is the holder's finding of a waiting call past its ask's 60 minutes with no decided heard (§677),
|
||||
// its id the call's.
|
||||
const KindGateUnanswered = "gate-unanswered"
|
||||
|
||||
// GatedEnded is a held call's end as the holder says it (§677): no argument is in it.
|
||||
type GatedEnded struct {
|
||||
Call string `json:"call"`
|
||||
Ask string `json:"ask"`
|
||||
Act string `json:"act"`
|
||||
Verb string `json:"verb"`
|
||||
Outcome string `json:"outcome"`
|
||||
Answer any `json:"answer,omitempty"`
|
||||
Approval string `json:"approval,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// GatedEndedSubject is the subject one call's end is said on: `mesh.seat.<seat>.event.gated-ended.<caller>.<call>`, the
|
||||
// caller the account the bus proved, escaped as a key's name is (§677, §73).
|
||||
func GatedEndedSubject(seat, caller, call string) string {
|
||||
return "mesh.seat." + seat + ".event." + GatedEndedEvent + "." + keys.EscapeName(caller) + "." + call
|
||||
}
|
||||
|
||||
// WaitingCall is one call the holder keeps under `call.<id>` (§677): the call whole, the caller the bus proved, its ask,
|
||||
// and once it ended its outcome, and whether that end was said.
|
||||
type WaitingCall struct {
|
||||
ID string `json:"id"`
|
||||
Caller string `json:"caller"`
|
||||
Ask string `json:"ask"`
|
||||
Digest string `json:"digest"`
|
||||
Act operatorask.Act `json:"act"`
|
||||
Address string `json:"address"`
|
||||
Seat string `json:"seat"`
|
||||
Verb string `json:"verb"`
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
// Made is when the call was made; Asked when its ask (the newest, after an ask anew) was raised or taken.
|
||||
Made time.Time `json:"made"`
|
||||
Asked time.Time `json:"asked"`
|
||||
// End is its end once it ended; Said says End was said on gated-ended.
|
||||
End *GatedEnded `json:"end,omitempty"`
|
||||
Said bool `json:"said,omitempty"`
|
||||
}
|
||||
|
||||
func callKey(id string) string { return "call." + id }
|
||||
func claimedKey(id string) string { return "claimed." + id }
|
||||
|
||||
// Serves names the handler a waiting call of seat's verb runs on its Approve (§677): process.Conn's AnswerSeat gives each
|
||||
// marked verb's.
|
||||
func (g *Gate) Serves(seat, verb string, h func(caller string, args json.RawMessage) (any, error)) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.handlers == nil {
|
||||
g.handlers = map[string]func(string, json.RawMessage) (any, error){}
|
||||
}
|
||||
g.handlers[seat+"."+verb] = h
|
||||
}
|
||||
|
||||
func (g *Gate) handler(seat, verb string) func(string, json.RawMessage) (any, error) {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.handlers[seat+"."+verb]
|
||||
}
|
||||
|
||||
// Hold is a call of a marked verb the holder serves (§677). A call whose arguments may not be asked, or that comes while
|
||||
// the state approvals cannot be read or written, is refused at once and nothing is kept. One that waits (the header
|
||||
// WaitHeader) is kept as a waiting call and answered at once with Held; one that does not takes ADR 0286 §37's path: it is
|
||||
// refused naming its act's ask, and the same call made again after the Approve runs. A call of an act declined within 10
|
||||
// minutes ends declined at once, unasked.
|
||||
func (g *Gate) Hold(c Call, wait bool, run func() (any, error)) (any, *Held, error) {
|
||||
if c.Fingerprint == "" {
|
||||
c.Fingerprint = g.d.Build
|
||||
}
|
||||
if c.Fingerprint == "" {
|
||||
return nil, nil, g.refused(c, "", "the holder's own fingerprint is unknown, so no warrant can bind its build")
|
||||
}
|
||||
if g.d.State == nil {
|
||||
return nil, nil, g.refused(c, "", "its holder keeps no state "+StateName+" to keep its calls in")
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, nil, g.refused(c, "", why)
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, "") // a process holder's act names no node, so two instances raise one ask (A-255)
|
||||
if err != nil {
|
||||
return nil, nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
if !wait {
|
||||
answer, err := g.unheld(c, act, digest, run)
|
||||
return answer, nil, err
|
||||
}
|
||||
id, err := g.askOf(c, act, digest)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
now := g.d.Now().UTC()
|
||||
w := WaitingCall{ID: newID(), Caller: c.Caller, Ask: id, Digest: digest, Act: act, Address: c.Address, Seat: c.Seat,
|
||||
Verb: c.Name, Arguments: c.Args, Made: now, Asked: now}
|
||||
if _, err := g.d.State.Create(callKey(w.ID), w); err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, nil, g.refused(c, digest, "its call cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
g.armUnanswered(w)
|
||||
g.d.Go(func() { g.Decided(id) }) // a choice made before the call was kept is acted on now
|
||||
return nil, &Held{Asked: id, Call: w.ID}, nil
|
||||
}
|
||||
|
||||
// askOf is the ask a call of the act takes (§677): the one `act.<digest>` names while it is open, else a new one, written
|
||||
// with the bus's create (so two instances raise one ask) or, over an ended one, by compare-and-set on its revision. A call
|
||||
// of an act declined within 10 minutes is refused with the time it may be asked again.
|
||||
func (g *Gate) askOf(c Call, act operatorask.Act, digest string) (string, error) {
|
||||
key := g.actKey(digest)
|
||||
for {
|
||||
e, err := g.d.State.Get(key)
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
id := newID()
|
||||
if e == nil {
|
||||
if _, err := g.d.State.Create(key, id); err != nil {
|
||||
if isChanged(err) {
|
||||
continue
|
||||
}
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
return id, g.raiseHeld(c, act, digest, id)
|
||||
}
|
||||
var named string
|
||||
if err := json.Unmarshal(e.Value, &named); err != nil || named == "" {
|
||||
return "", g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
|
||||
}
|
||||
open, until, err := g.openAsk(c.Module, named)
|
||||
if err != nil {
|
||||
return "", g.refused(c, digest, err.Error())
|
||||
}
|
||||
if !until.IsZero() {
|
||||
return "", fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s", named, formatTime(until))
|
||||
}
|
||||
if open {
|
||||
return named, nil
|
||||
}
|
||||
if _, err := g.d.State.Update(key, id, e.Revision); err != nil {
|
||||
if isChanged(err) {
|
||||
continue
|
||||
}
|
||||
g.stateFailed(err)
|
||||
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
|
||||
}
|
||||
return id, g.raiseHeld(c, act, digest, id)
|
||||
}
|
||||
}
|
||||
|
||||
// openAsk says whether an ask is open for another call of its act: raised and not yet ended by the router; and, when it was
|
||||
// declined within 10 minutes, until when no call of its act is asked.
|
||||
func (g *Gate) openAsk(module, id string) (bool, time.Time, error) {
|
||||
rec, err := g.record(module, id)
|
||||
if err != nil {
|
||||
return false, time.Time{}, fmt.Errorf("the router's record of its ask %s cannot be read: %v", id, err)
|
||||
}
|
||||
if rec == nil {
|
||||
k, _, err := g.kept(id)
|
||||
return err == nil && k != nil && k.Ended == "", time.Time{}, nil // raised and not yet taken by the router
|
||||
}
|
||||
switch rec.State {
|
||||
case "open", "held-back":
|
||||
return true, time.Time{}, nil
|
||||
case "chosen":
|
||||
if w := rec.Warrant; w != nil && w.Option == OptionDecline {
|
||||
at, _ := parseTime(w.At)
|
||||
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
|
||||
return false, until, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return false, time.Time{}, nil
|
||||
}
|
||||
|
||||
// raiseHeld raises the ask the act's key names, under the holder's module's name, keeping the act and the arguments beside
|
||||
// it (§677; design 46 7-approval-mark §10).
|
||||
func (g *Gate) raiseHeld(c Call, act operatorask.Act, digest, id string) error {
|
||||
return g.keepAskHeld(c, act, digest, g.askFor(c, digest, id))
|
||||
}
|
||||
|
||||
// keepAskHeld is keepAsk without its write of the act's key, which askOf made first.
|
||||
func (g *Gate) keepAskHeld(c Call, act operatorask.Act, digest string, a Ask) error {
|
||||
if size := keptSize(a, act, c.Args); size > MostKeptValue {
|
||||
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
|
||||
}
|
||||
if err := g.d.Channel.Ask(c.Module, a); err != nil {
|
||||
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
|
||||
}
|
||||
if err := g.keepCall(a.ID, act, c.Args); err != nil {
|
||||
g.stateFailed(err)
|
||||
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// unheld is a call that does not wait (ADR 0286 §37's path, §677): run when its act's ask is chosen Approve within 30
|
||||
// minutes and unclaimed, claimed once; otherwise refused naming the ask, raising one when none is open.
|
||||
func (g *Gate) unheld(c Call, act operatorask.Act, digest string, run func() (any, error)) (any, error) {
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
if named != nil {
|
||||
var id string
|
||||
if json.Unmarshal(named.Value, &id) == nil && id != "" {
|
||||
rec, err := g.record(c.Module, id)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
|
||||
}
|
||||
if rec != nil && rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionApprove {
|
||||
if at, _ := parseTime(rec.Warrant.At); !g.d.Now().After(at.Add(RunWithin)) {
|
||||
out := g.claimRun(c.Module, id, *rec.Warrant, act, run)
|
||||
switch out.outcome {
|
||||
case OutcomeRan:
|
||||
return g.ran(c, id, rec, out.answer, out.err)
|
||||
case OutcomeRefused:
|
||||
return nil, g.refused(c, digest, out.why)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
id, err := g.askOf(c, act, digest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return nil, g.firstAnswer(c, id, digest, "")
|
||||
}
|
||||
|
||||
// claimed is what came of a claim of a warrant for one call.
|
||||
type claimed struct {
|
||||
outcome string // OutcomeRan, OutcomeRefused, or "" when another took it
|
||||
answer any
|
||||
err error
|
||||
why string
|
||||
}
|
||||
|
||||
// claimRun holds the warrant to the ask and the call's act, claims it once at `claimed.<ask id>` with the bus's create, and
|
||||
// runs the call (§677; ADR 0286 §27, §28, §30, §31): of two instances, or two deliveries, one runs.
|
||||
func (g *Gate) claimRun(module, id string, w Warrant, act operatorask.Act, run func() (any, error)) claimed {
|
||||
var out claimed
|
||||
var claimErr error
|
||||
h, err := g.d.Channel.Claim(module, w, act, func() {
|
||||
if _, err := g.d.State.Create(claimedKey(id), map[string]string{"at": formatTime(g.d.Now().UTC())}); err != nil {
|
||||
claimErr = err
|
||||
return
|
||||
}
|
||||
out.outcome = OutcomeRan
|
||||
out.answer, out.err = run()
|
||||
})
|
||||
switch {
|
||||
case h == Claimed && claimErr != nil && !isChanged(claimErr):
|
||||
g.stateFailed(claimErr)
|
||||
return claimed{outcome: OutcomeRefused, why: "its claim cannot be written in " + StateName + ": " + claimErr.Error()}
|
||||
case h == Claimed:
|
||||
return out
|
||||
case h == ClaimDone:
|
||||
return claimed{}
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return claimed{outcome: OutcomeRefused, why: "its state " + StateName + " cannot be written: " + err.Error()}
|
||||
default: // refused, or the act about to run is not the one the option bound: nothing runs
|
||||
return claimed{outcome: OutcomeRefused, why: "its warrant does not hold: " + errText(err)}
|
||||
}
|
||||
}
|
||||
|
||||
// Decided acts on the router's own record of one of the holder's asks (§677): heard on the holder's `decided`, never from
|
||||
// its body, and at the start for each ask a waiting call waits on. On the Approve the oldest waiting call of the ask runs,
|
||||
// within 30 minutes of the choice, else it ends expired; every other is asked anew. A Decline ends every one declined, an
|
||||
// ask that ended without a choice every one expired, and one the router refused every one refused with its words.
|
||||
func (g *Gate) Decided(id string) {
|
||||
module := g.d.Runner
|
||||
rec, err := g.record(module, id)
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the router's record of ask %s cannot be read (%v): read again at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if rec == nil || rec.State == "open" || rec.State == "held-back" {
|
||||
return
|
||||
}
|
||||
calls, err := g.waiting(func(w WaitingCall) bool { return w.Ask == id })
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
g.d.Logf("[approval] the waiting calls of ask %s cannot be read (%v): read again at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if len(calls) == 0 {
|
||||
return
|
||||
}
|
||||
end := func(w WaitingCall, outcome, why string) {
|
||||
g.end(w, GatedEnded{Outcome: outcome, Error: why})
|
||||
}
|
||||
switch {
|
||||
case rec.State == "refused":
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeRefused, "the router refused its ask "+id+": "+orNone(rec.Answer))
|
||||
}
|
||||
return
|
||||
case rec.State != "chosen":
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeExpired, "its ask "+id+" ended "+rec.State+" with no choice; nothing ran")
|
||||
}
|
||||
return
|
||||
case rec.Warrant == nil:
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeRefused, "the router's record of its ask "+id+" is chosen and carries no warrant")
|
||||
}
|
||||
return
|
||||
case rec.Warrant.Option == OptionDecline:
|
||||
for _, w := range calls {
|
||||
end(w, OutcomeDeclined, "the operator declined ask "+id+"; nothing ran")
|
||||
}
|
||||
return
|
||||
}
|
||||
first, rest := calls[0], calls[1:]
|
||||
at, _ := parseTime(rec.Warrant.At)
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
end(first, OutcomeExpired, "the Approve of ask "+id+" was heard more than 30 minutes after it was given; nothing ran")
|
||||
} else {
|
||||
g.runWaiting(first, rec)
|
||||
}
|
||||
for _, w := range rest {
|
||||
g.anew(w)
|
||||
}
|
||||
}
|
||||
|
||||
// runWaiting claims and runs one waiting call on its ask's warrant, saying its end.
|
||||
func (g *Gate) runWaiting(w WaitingCall, rec *Record) {
|
||||
h := g.handler(w.Seat, w.Verb)
|
||||
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments}
|
||||
if h == nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: "the holder serves no handler of " + w.Address + " now; nothing ran"})
|
||||
return
|
||||
}
|
||||
out := g.claimRun(g.d.Runner, w.Ask, *rec.Warrant, w.Act, func() (any, error) { return h(w.Caller, w.Arguments) })
|
||||
switch out.outcome {
|
||||
case "": // another instance, or another call of the same ask, took the warrant
|
||||
return
|
||||
case OutcomeRefused:
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: out.why})
|
||||
return
|
||||
}
|
||||
answered, err := g.ran(c, w.Ask, rec, out.answer, out.err)
|
||||
if err != nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRan, Error: err.Error()})
|
||||
return
|
||||
}
|
||||
a := answered.(Answered)
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRan, Answer: a.Answer, Approval: a.Approval})
|
||||
}
|
||||
|
||||
// anew asks a waiting call's act anew, as a new ask (ADR 0286 §33: a second run takes a second approval); its end comes
|
||||
// when its own ask ends.
|
||||
func (g *Gate) anew(w WaitingCall) {
|
||||
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments,
|
||||
Fingerprint: w.Act["build"]}
|
||||
id, err := g.askOf(c, w.Act, w.Digest)
|
||||
if err != nil {
|
||||
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: err.Error()})
|
||||
return
|
||||
}
|
||||
if err := g.updateCall(w.ID, func(k *WaitingCall) { k.Ask, k.Asked = id, g.d.Now().UTC() }); err != nil {
|
||||
g.d.Logf("[approval] the call %s, asked anew as ask %s, cannot be kept (%v)", w.ID, id, err)
|
||||
return
|
||||
}
|
||||
w.Ask, w.Asked = id, g.d.Now().UTC()
|
||||
g.rearmUnanswered(w)
|
||||
g.d.Go(func() { g.Decided(id) })
|
||||
}
|
||||
|
||||
// end keeps a waiting call's end, then says it on gated-ended (§677); an end not said is said again at the next start.
|
||||
func (g *Gate) end(w WaitingCall, e GatedEnded) {
|
||||
e.Call, e.Ask, e.Act, e.Verb = w.ID, w.Ask, w.Digest, w.Verb
|
||||
ended := false
|
||||
if err := g.updateCall(w.ID, func(k *WaitingCall) {
|
||||
if k.End == nil {
|
||||
k.End, ended = &e, true
|
||||
}
|
||||
}); err != nil {
|
||||
g.d.Logf("[approval] the end %s of call %s cannot be kept (%v)", e.Outcome, w.ID, err)
|
||||
return
|
||||
}
|
||||
if ended {
|
||||
g.say(w.ID)
|
||||
}
|
||||
}
|
||||
|
||||
// say says a kept end on gated-ended and marks it said; its finding gate-unanswered is cleared.
|
||||
func (g *Gate) say(id string) {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return
|
||||
}
|
||||
var w WaitingCall
|
||||
if json.Unmarshal(e.Value, &w) != nil || w.End == nil || w.Said {
|
||||
return
|
||||
}
|
||||
// Its bound key still kept says gate-unanswered was not raised, and the key goes; one the bus ended is cleared once the
|
||||
// end is said (A-255).
|
||||
b, err := g.d.State.Get(unansweredHeldKey(id))
|
||||
raised := err == nil && b == nil
|
||||
if b != nil && g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(unansweredHeldKey(id))
|
||||
}
|
||||
if g.d.Say == nil {
|
||||
g.d.Logf("[approval] the end %s of call %s is not said: the holder says no gated-ended", w.End.Outcome, id)
|
||||
return
|
||||
}
|
||||
if err := g.d.Say(GatedEndedSubject(w.Seat, w.Caller, w.ID), *w.End, GatedEndedEvent+"."+w.ID); err != nil {
|
||||
g.d.Logf("[approval] the end %s of call %s is not said (%v): said again at the next start", w.End.Outcome, id, err)
|
||||
return
|
||||
}
|
||||
g.d.Logf("[approval] %s by %s ended %s (call %s, ask %s)", w.Address, w.Caller, w.End.Outcome, id, w.Ask)
|
||||
_ = g.updateCall(id, func(k *WaitingCall) { k.Said = true })
|
||||
if raised && g.d.Findings != nil {
|
||||
_ = g.d.Findings.Clear(nilCtx(), KindGateUnanswered, id, "the call's end, "+w.End.Outcome+", is said")
|
||||
}
|
||||
}
|
||||
|
||||
// updateCall changes a kept call by compare-and-set, again when another write came first.
|
||||
func (g *Gate) updateCall(id string, change func(*WaitingCall)) error {
|
||||
for {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return fmt.Errorf("the call %s cannot be read: %v", id, err)
|
||||
}
|
||||
var w WaitingCall
|
||||
if err := json.Unmarshal(e.Value, &w); err != nil {
|
||||
return err
|
||||
}
|
||||
change(&w)
|
||||
if _, err := g.d.State.Update(callKey(id), w, e.Revision); err == nil {
|
||||
return nil
|
||||
} else if !isChanged(err) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// waiting is every kept call that has not ended and that keep takes, oldest first.
|
||||
func (g *Gate) waiting(keep func(WaitingCall) bool) ([]WaitingCall, error) {
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []WaitingCall
|
||||
for _, w := range all {
|
||||
if w.End == nil && keep(w) {
|
||||
out = append(out, w)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// calls is every kept call, oldest first.
|
||||
func (g *Gate) calls() ([]WaitingCall, error) {
|
||||
names, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []WaitingCall
|
||||
for _, key := range names {
|
||||
if !strings.HasPrefix(key, "call.") {
|
||||
continue
|
||||
}
|
||||
e, err := g.d.State.Get(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var w WaitingCall
|
||||
if e == nil || json.Unmarshal(e.Value, &w) != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, w)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Made.Before(out[j].Made) })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// unansweredHeldKey is a waiting call's bound key in the holder's state approvals (lead answer A-255): created with the
|
||||
// ask's 60 minutes as its own time to live, its expiry heard as the bus's event raises gate-unanswered.
|
||||
func unansweredHeldKey(call string) string { return "unanswered." + call }
|
||||
|
||||
// armUnanswered keeps the waiting call's bound key (§677; A-255): when its ask's 60 minutes pass with no end of it, the bus
|
||||
// ends the key and the holder, hearing that, raises gate-unanswered; no clock waits in the process. A call already past its
|
||||
// 60 minutes (its key ended while the holder was away) is judged at once.
|
||||
func (g *Gate) armUnanswered(w WaitingCall) {
|
||||
left := w.Asked.Add(AskLasts).Sub(g.d.Now())
|
||||
if left <= 0 {
|
||||
g.heldUnanswered(w.ID)
|
||||
return
|
||||
}
|
||||
if g.d.Bounds == nil {
|
||||
g.d.Logf("[approval] the call %s waits on ask %s with no bound kept: the holder's %s keeps no key with its own time to live",
|
||||
w.ID, w.Ask, StateName)
|
||||
return
|
||||
}
|
||||
if err := g.d.Bounds.CreateTTL(unansweredHeldKey(w.ID), map[string]string{"call": w.ID, "ask": w.Ask}, left); err != nil && !isChanged(err) {
|
||||
g.d.Logf("[approval] the bound of call %s on ask %s is not kept: %v", w.ID, w.Ask, err)
|
||||
}
|
||||
}
|
||||
|
||||
// rearmUnanswered keeps a call asked anew under a new bound of its new ask's 60 minutes.
|
||||
func (g *Gate) rearmUnanswered(w WaitingCall) {
|
||||
if g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(unansweredHeldKey(w.ID))
|
||||
}
|
||||
g.armUnanswered(w)
|
||||
}
|
||||
|
||||
// heldUnanswered is a waiting call's bound ended (A-255): gate-unanswered raised while the call has no end.
|
||||
func (g *Gate) heldUnanswered(id string) {
|
||||
e, err := g.d.State.Get(callKey(id))
|
||||
if err != nil || e == nil {
|
||||
return
|
||||
}
|
||||
var w WaitingCall
|
||||
if json.Unmarshal(e.Value, &w) != nil || w.End != nil || g.d.Findings == nil {
|
||||
return
|
||||
}
|
||||
if err := g.d.Findings.Raise(nilCtx(), KindGateUnanswered, w.ID, fmt.Sprintf("the call %s of %s by %s waits on ask %s, "+
|
||||
"past its %s with no decided heard: nothing ran; the holder's %s.approvals reads it", w.ID, w.Address, w.Caller, w.Ask,
|
||||
AskLasts, w.Seat), "a call waits on an unanswered ask"); err != nil {
|
||||
g.d.Logf("[approval] the finding %s of call %s is not kept: %v", KindGateUnanswered, w.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
// watchHeld hears the waiting calls' bound keys end (A-255) for the life of the holder's connection.
|
||||
func (g *Gate) watchHeld() error {
|
||||
if g.d.Bounds == nil {
|
||||
return nil
|
||||
}
|
||||
_, err := g.d.Bounds.Watch("unanswered.*", func(key string) { g.heldUnanswered(strings.TrimPrefix(key, "unanswered.")) })
|
||||
return err
|
||||
}
|
||||
|
||||
// StartHolding is the holder's read at its start (§677): every waiting call is read, each ask already decided acted on as
|
||||
// when its decided is heard, each end not yet said said, each waiting call's 60 minutes armed, and every acted, unrecorded
|
||||
// run recorded. A state that cannot be read leaves every marked call refused until it can.
|
||||
func (g *Gate) StartHolding() error {
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return fmt.Errorf("the waiting calls in %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
g.stateRead()
|
||||
if err := g.watchHeld(); err != nil {
|
||||
return fmt.Errorf("the bound keys in %s cannot be watched: %w", StateName, err)
|
||||
}
|
||||
asks := map[string]bool{}
|
||||
for _, w := range all {
|
||||
switch {
|
||||
case w.End != nil && !w.Said:
|
||||
g.say(w.ID)
|
||||
case w.End == nil:
|
||||
g.armUnanswered(w)
|
||||
asks[w.Ask] = true
|
||||
}
|
||||
}
|
||||
ids := make([]string, 0, len(asks))
|
||||
for id := range asks {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
for _, id := range ids {
|
||||
g.Decided(id)
|
||||
}
|
||||
g.Recover()
|
||||
return nil
|
||||
}
|
||||
|
||||
// Approvals is the seat's verb `approvals` (§677): the waiting and ended calls with their caller, ask and state, the
|
||||
// `act.` keys and the claims; a call's arguments only to that call's caller.
|
||||
type Approvals struct {
|
||||
Calls []ListedCall `json:"calls"`
|
||||
Acts []ListedAct `json:"acts"`
|
||||
Claims []string `json:"claims"`
|
||||
}
|
||||
|
||||
// ListedCall is one kept call as `approvals` lists it.
|
||||
type ListedCall struct {
|
||||
Call string `json:"call"`
|
||||
Caller string `json:"caller"`
|
||||
Ask string `json:"ask"`
|
||||
Verb string `json:"verb"`
|
||||
State string `json:"state"`
|
||||
Made time.Time `json:"made"`
|
||||
End *GatedEnded `json:"end,omitempty"`
|
||||
Arguments json.RawMessage `json:"arguments,omitempty"`
|
||||
}
|
||||
|
||||
// ListedAct is one `act.` key: its digest, escaped as kept, and the ask it names.
|
||||
type ListedAct struct {
|
||||
Act string `json:"act"`
|
||||
Ask string `json:"ask"`
|
||||
}
|
||||
|
||||
// Approvals answers the verb for caller, the account the bus proved.
|
||||
func (g *Gate) Approvals(caller string) (Approvals, error) {
|
||||
out := Approvals{Calls: []ListedCall{}, Acts: []ListedAct{}, Claims: []string{}}
|
||||
if g.d.State == nil {
|
||||
return out, fmt.Errorf("the holder keeps no state %s", StateName)
|
||||
}
|
||||
all, err := g.calls()
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
for _, w := range all {
|
||||
l := ListedCall{Call: w.ID, Caller: w.Caller, Ask: w.Ask, Verb: w.Verb, State: "waiting", Made: w.Made, End: w.End}
|
||||
if w.End != nil {
|
||||
l.State = w.End.Outcome
|
||||
}
|
||||
if caller != "" && caller == w.Caller {
|
||||
l.Arguments = w.Arguments
|
||||
}
|
||||
out.Calls = append(out.Calls, l)
|
||||
}
|
||||
names, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, key := range names {
|
||||
switch {
|
||||
case strings.HasPrefix(key, "act."):
|
||||
e, err := g.d.State.Get(key)
|
||||
var id string
|
||||
if err == nil && e != nil {
|
||||
_ = json.Unmarshal(e.Value, &id)
|
||||
}
|
||||
out.Acts = append(out.Acts, ListedAct{Act: strings.TrimPrefix(key, "act."), Ask: id})
|
||||
case strings.HasPrefix(key, "claimed."):
|
||||
out.Claims = append(out.Claims, strings.TrimPrefix(key, "claimed."))
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func nilCtx() context.Context { return context.Background() }
|
||||
@@ -1,439 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// memStore is the holder's state approvals in memory: create refuses a key that exists, update is compare-and-set.
|
||||
type memStore struct {
|
||||
mu sync.Mutex
|
||||
vals map[string][]byte
|
||||
revs map[string]uint64
|
||||
rev uint64
|
||||
fail error
|
||||
}
|
||||
|
||||
func newMem() *memStore { return &memStore{vals: map[string][]byte{}, revs: map[string]uint64{}} }
|
||||
|
||||
func (m *memStore) Get(key string) (*stdio.StateEntry, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return nil, m.fail
|
||||
}
|
||||
v, ok := m.vals[key]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: key, Value: v, Revision: m.revs[key]}, nil
|
||||
}
|
||||
|
||||
func (m *memStore) write(key string, value any, check func(exists bool, rev uint64) bool) (uint64, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return 0, m.fail
|
||||
}
|
||||
_, exists := m.vals[key]
|
||||
if !check(exists, m.revs[key]) {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
raw, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
m.rev++
|
||||
m.vals[key], m.revs[key] = raw, m.rev
|
||||
return m.rev, nil
|
||||
}
|
||||
|
||||
func (m *memStore) Create(key string, value any) (uint64, error) {
|
||||
return m.write(key, value, func(exists bool, _ uint64) bool { return !exists })
|
||||
}
|
||||
func (m *memStore) Update(key string, value any, revision uint64) (uint64, error) {
|
||||
return m.write(key, value, func(exists bool, rev uint64) bool { return exists && rev == revision })
|
||||
}
|
||||
func (m *memStore) Put(key string, value any) (uint64, error) {
|
||||
return m.write(key, value, func(bool, uint64) bool { return true })
|
||||
}
|
||||
func (m *memStore) Keys() ([]string, error) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.fail != nil {
|
||||
return nil, m.fail
|
||||
}
|
||||
var out []string
|
||||
for k := range m.vals {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// bounds is the holder's bound keys in its state approvals as a fake (A-255): each key kept in the state with its time to
|
||||
// live, expire ending it as the bus does and handing it to the watch.
|
||||
type bounds struct {
|
||||
state *memStore
|
||||
mu sync.Mutex
|
||||
ttl map[string]time.Duration
|
||||
watched func(string)
|
||||
}
|
||||
|
||||
func (b *bounds) CreateTTL(key string, value any, ttl time.Duration) error {
|
||||
if _, err := b.state.Create(key, value); err != nil {
|
||||
return err
|
||||
}
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
b.ttl[key] = ttl
|
||||
return nil
|
||||
}
|
||||
func (b *bounds) Delete(key string) error {
|
||||
b.state.delete(key)
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
delete(b.ttl, key)
|
||||
return nil
|
||||
}
|
||||
func (b *bounds) Watch(pattern string, expired func(string)) (func(), error) {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
b.watched = expired
|
||||
return func() {}, nil
|
||||
}
|
||||
func (b *bounds) expire(key string) {
|
||||
b.mu.Lock()
|
||||
_, ok := b.ttl[key]
|
||||
delete(b.ttl, key)
|
||||
w := b.watched
|
||||
b.mu.Unlock()
|
||||
b.state.delete(key)
|
||||
if ok && w != nil {
|
||||
w(key)
|
||||
}
|
||||
}
|
||||
|
||||
func (m *memStore) delete(key string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
delete(m.vals, key)
|
||||
delete(m.revs, key)
|
||||
}
|
||||
|
||||
// router is the operator channel as a fake: it keeps each ask the gate raises, and the test decides its record.
|
||||
type router struct {
|
||||
mu sync.Mutex
|
||||
state Store
|
||||
asks []Ask
|
||||
records map[string]*Record
|
||||
}
|
||||
|
||||
func (r *router) Ask(module string, a Ask) error {
|
||||
raw, _ := json.Marshal(a)
|
||||
if _, err := r.state.Create("ask."+a.ID, kept{Ask: raw, Asked: time.Now()}); err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.asks = append(r.asks, a)
|
||||
return nil
|
||||
}
|
||||
func (r *router) Record(module, id string) (*Record, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
return r.records[id], nil
|
||||
}
|
||||
func (r *router) Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error) {
|
||||
e, _ := r.state.Get("ask." + w.Ask)
|
||||
if e == nil {
|
||||
return ClaimRefused, fmt.Errorf("no ask %s", w.Ask)
|
||||
}
|
||||
var k kept
|
||||
_ = json.Unmarshal(e.Value, &k)
|
||||
if k.Acted {
|
||||
return ClaimDone, nil
|
||||
}
|
||||
var a Ask
|
||||
_ = json.Unmarshal(k.Ask, &a)
|
||||
digest, _ := operatorask.ActDigest(act)
|
||||
if a.About != digest {
|
||||
return ClaimRefused, fmt.Errorf("the option binds another act")
|
||||
}
|
||||
k.Acted = true
|
||||
if _, err := r.state.Update("ask."+w.Ask, k, e.Revision); err != nil {
|
||||
return ClaimDone, nil
|
||||
}
|
||||
run()
|
||||
return Claimed, nil
|
||||
}
|
||||
func (r *router) Cancel(module, id string) error { return nil }
|
||||
func (r *router) Decided(module string) string { return "decided." + module }
|
||||
func (r *router) Seat() string { return "operator-channel" }
|
||||
|
||||
func (r *router) choose(id, option string, at time.Time) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.records[id] = &Record{State: "chosen", OfferedOn: []string{"telegram"},
|
||||
Warrant: &Warrant{Ask: id, Asker: "holder", Option: option, Channel: "telegram", At: formatTime(at)}}
|
||||
}
|
||||
|
||||
type findings struct {
|
||||
mu sync.Mutex
|
||||
raised map[string]bool
|
||||
heard chan string
|
||||
}
|
||||
|
||||
func (f *findings) Raise(_ context.Context, kind, id, _, _ string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.raised[kind+" "+id] = true
|
||||
if f.heard != nil {
|
||||
f.heard <- kind + " " + id
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (f *findings) Clear(_ context.Context, kind, id, _ string) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
delete(f.raised, kind+" "+id)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Design 47 G §677 (lead answers A-158, A-209, A-244): a process holder's marked verb is one call with a gate before its
|
||||
// running, its waiting calls, act keys and claims kept in its own state.
|
||||
func TestAHolderHoldsAMarkedCall(t *testing.T) {
|
||||
now := time.Date(2026, 11, 12, 10, 0, 0, 0, time.UTC)
|
||||
type world struct {
|
||||
g *Gate
|
||||
state *memStore
|
||||
router *router
|
||||
said map[string]GatedEnded
|
||||
ran *int
|
||||
recs *int
|
||||
finds *findings
|
||||
after chan time.Time
|
||||
bounds *bounds
|
||||
}
|
||||
newGate := func(state *memStore, r *router, ran, recs *int, said map[string]GatedEnded, after chan time.Time, f *findings, b *bounds) *Gate {
|
||||
g := New(Deps{Node: "n1", Runner: "holder", Holding: true, Build: "fp1", State: state, Channel: r, Findings: f, Bounds: b,
|
||||
Now: func() time.Time { return now }, Go: func(f func()) { f() },
|
||||
After: func(d time.Duration) <-chan time.Time {
|
||||
if d < time.Minute { // the record's short pauses pass at once
|
||||
fired := make(chan time.Time)
|
||||
close(fired)
|
||||
return fired
|
||||
}
|
||||
return after
|
||||
},
|
||||
Warranted: func(asker, ask string, open func() (bool, error)) error {
|
||||
*recs++
|
||||
return nil
|
||||
},
|
||||
Say: func(subject string, body any, id string) error {
|
||||
said[subject] = body.(GatedEnded)
|
||||
return nil
|
||||
}})
|
||||
g.Serves("wiper", "wipe", func(caller string, args json.RawMessage) (any, error) { *ran++; return "wiped", nil })
|
||||
return g
|
||||
}
|
||||
setup := func() world {
|
||||
state := newMem()
|
||||
r := &router{state: state, records: map[string]*Record{}}
|
||||
ran, recs := 0, 0
|
||||
w := world{state: state, router: r, said: map[string]GatedEnded{}, ran: &ran, recs: &recs,
|
||||
finds: &findings{raised: map[string]bool{}}, after: make(chan time.Time), bounds: &bounds{state: state, ttl: map[string]time.Duration{}}}
|
||||
w.g = newGate(state, r, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
return w
|
||||
}
|
||||
call := func(caller, args string) Call {
|
||||
return Call{Address: "wiper.wipe", Module: "holder", Seat: "wiper", Name: "wipe", Caller: caller, Args: json.RawMessage(args)}
|
||||
}
|
||||
hold := func(t *testing.T, w world, c Call, wait bool) (any, *Held, error) {
|
||||
return w.g.Hold(c, wait, func() (any, error) { *w.ran++; return "wiped", nil })
|
||||
}
|
||||
ended := func(w world, caller, id string) GatedEnded { return w.said[GatedEndedSubject("wiper", caller, id)] }
|
||||
|
||||
t.Run("asked, run once on the Approve, its end said to its caller", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, err := hold(t, w, call("n1.caller", `{"dir":"scratch"}`), true)
|
||||
if err != nil || h == nil || h.Asked == "" || h.Call == "" {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
if e, _ := w.state.Get("call." + h.Call); e == nil {
|
||||
t.Fatal("no call kept")
|
||||
}
|
||||
w.router.choose(h.Asked, OptionApprove, now)
|
||||
w.g.Decided(h.Asked)
|
||||
w.g.Decided(h.Asked) // heard twice: run once
|
||||
end := ended(w, "n1.caller", h.Call)
|
||||
if *w.ran != 1 || end.Outcome != OutcomeRan || end.Answer != "wiped" || !strings.Contains(end.Approval, h.Asked) || *w.recs != 1 {
|
||||
t.Fatalf("ran %d, recorded %d, end %+v", *w.ran, *w.recs, end)
|
||||
}
|
||||
if e, _ := w.state.Get("claimed." + h.Asked); e == nil {
|
||||
t.Error("no claim kept")
|
||||
}
|
||||
a, _ := w.g.Approvals("someone.else")
|
||||
if len(a.Calls) != 1 || a.Calls[0].Arguments != nil || a.Calls[0].State != OutcomeRan || len(a.Acts) != 1 || len(a.Claims) != 1 {
|
||||
t.Errorf("approvals %+v", a)
|
||||
}
|
||||
if a, _ := w.g.Approvals("n1.caller"); string(a.Calls[0].Arguments) != `{"dir":"scratch"}` {
|
||||
t.Errorf("its caller reads %s", a.Calls[0].Arguments)
|
||||
}
|
||||
})
|
||||
t.Run("a second call of the act takes the same ask, asked anew after the first runs", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
now = now.Add(time.Second)
|
||||
_, h2, _ := hold(t, w, call("n1.b", `{}`), true)
|
||||
if h1.Asked != h2.Asked || h1.Call == h2.Call {
|
||||
t.Fatalf("asks %s and %s", h1.Asked, h2.Asked)
|
||||
}
|
||||
w.router.choose(h1.Asked, OptionApprove, now)
|
||||
w.g.Decided(h1.Asked)
|
||||
if *w.ran != 1 || ended(w, "n1.a", h1.Call).Outcome != OutcomeRan || ended(w, "n1.b", h2.Call).Outcome != "" {
|
||||
t.Fatalf("ran %d; ends %v", *w.ran, w.said)
|
||||
}
|
||||
e, _ := w.state.Get("call." + h2.Call)
|
||||
var second WaitingCall
|
||||
_ = json.Unmarshal(e.Value, &second)
|
||||
if second.Ask == h1.Asked || len(w.router.asks) != 2 {
|
||||
t.Errorf("the second call waits on %s (asks %d)", second.Ask, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("a decline runs nothing; a call within 10 minutes ends declined unasked", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionDecline, now)
|
||||
w.g.Decided(h.Asked)
|
||||
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeDeclined {
|
||||
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
|
||||
}
|
||||
if _, h2, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h2 != nil || !strings.Contains(err.Error(), "declined") || len(w.router.asks) != 1 {
|
||||
t.Errorf("a call within 10 minutes: %v, %d asks", err, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("heard 31 minutes after the choice, it ends expired", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionApprove, now.Add(-31*time.Minute))
|
||||
w.g.Decided(h.Asked)
|
||||
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeExpired {
|
||||
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
|
||||
}
|
||||
})
|
||||
t.Run("a secret-shaped argument refused, nothing kept", func(t *testing.T) {
|
||||
w := setup()
|
||||
if _, h, err := hold(t, w, call("n1.a", `{"token":"ghp_0123456789abcdefghijklmnopqrstuvwxyzAB"}`), true); err == nil || h != nil {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
if keys, _ := w.state.Keys(); len(keys) != 0 || len(w.router.asks) != 0 || len(w.bounds.ttl) != 0 {
|
||||
t.Errorf("kept %v, asked %d", keys, len(w.router.asks))
|
||||
}
|
||||
})
|
||||
t.Run("a call with no wait refused with the ask, and run when made again after the Approve", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, err := hold(t, w, call("n1.a", `{}`), false)
|
||||
if err == nil || h != nil || len(w.router.asks) != 1 || !strings.Contains(err.Error(), w.router.asks[0].ID) {
|
||||
t.Fatalf("first: %v", err)
|
||||
}
|
||||
w.router.choose(w.router.asks[0].ID, OptionApprove, now)
|
||||
answer, _, err := hold(t, w, call("n1.a", `{}`), false)
|
||||
if a, ok := answer.(Answered); err != nil || !ok || a.Answer != "wiped" || *w.ran != 1 {
|
||||
t.Fatalf("again: %+v, %v", answer, err)
|
||||
}
|
||||
})
|
||||
t.Run("a restart between the ask and the Approve runs the waiting call once", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.router.choose(h.Asked, OptionApprove, now)
|
||||
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
if err := again.StartHolding(); err != nil || *w.ran != 1 || ended(w, "n1.a", h.Call).Outcome != OutcomeRan {
|
||||
t.Fatalf("start: %v, ran %d", err, *w.ran)
|
||||
}
|
||||
_ = again.StartHolding()
|
||||
if *w.ran != 1 {
|
||||
t.Errorf("ran %d after a second start", *w.ran)
|
||||
}
|
||||
})
|
||||
t.Run("past its ask's 60 minutes with no decided, gate-unanswered on its bound key's end", func(t *testing.T) {
|
||||
// A-255: a bound key with the ask's 60 minutes to live in the state approvals, never a clock wait.
|
||||
w := setup()
|
||||
if err := w.g.StartHolding(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
key := "unanswered." + h.Call
|
||||
if ttl := w.bounds.ttl[key]; ttl != AskLasts {
|
||||
t.Fatalf("bound %s kept for %s (%v)", key, ttl, w.bounds.ttl)
|
||||
}
|
||||
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Fatal("raised before its bound ended")
|
||||
}
|
||||
w.bounds.expire(key)
|
||||
if !w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Errorf("not raised on its bound's end: %v", w.finds.raised)
|
||||
}
|
||||
w.router.choose(h.Asked, OptionDecline, now)
|
||||
w.g.Decided(h.Asked)
|
||||
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Error("not cleared once its end is said")
|
||||
}
|
||||
})
|
||||
t.Run("a call whose bound ended while the holder was away is judged at its start", func(t *testing.T) {
|
||||
w := setup()
|
||||
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
w.bounds.watched = nil
|
||||
w.bounds.expire("unanswered." + h.Call) // the bus ended it with nobody watching
|
||||
now = now.Add(AskLasts + time.Minute)
|
||||
defer func() { now = now.Add(-AskLasts - time.Minute) }()
|
||||
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
|
||||
if err := again.StartHolding(); err != nil || !w.finds.raised[KindGateUnanswered+" "+h.Call] {
|
||||
t.Errorf("start: %v, raised %v", err, w.finds.raised)
|
||||
}
|
||||
})
|
||||
t.Run("two instances of the holder raise one ask and run once", func(t *testing.T) {
|
||||
// A-255: a process holder's act names no node.
|
||||
w := setup()
|
||||
other := New(Deps{Node: "n2", Runner: "holder", Holding: true, Build: "fp1", State: w.state, Channel: w.router,
|
||||
Findings: w.finds, Bounds: w.bounds, Now: func() time.Time { return now }, Go: func(f func()) { f() },
|
||||
Say: func(subject string, body any, id string) error { w.said[subject] = body.(GatedEnded); return nil }})
|
||||
other.Serves("wiper", "wipe", func(string, json.RawMessage) (any, error) { *w.ran++; return "wiped", nil })
|
||||
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
|
||||
_, h2, err := other.Hold(call("n2.b", `{}`), true, nil)
|
||||
if err != nil || h2 == nil || h2.Asked != h1.Asked || len(w.router.asks) != 1 {
|
||||
t.Fatalf("second instance: %+v, %v; asks %d", h2, err, len(w.router.asks))
|
||||
}
|
||||
if strings.Contains(w.router.asks[0].About, "n1") {
|
||||
t.Errorf("the ask binds %s", w.router.asks[0].About)
|
||||
}
|
||||
var first WaitingCall
|
||||
e, _ := w.state.Get("call." + h1.Call)
|
||||
_ = json.Unmarshal(e.Value, &first)
|
||||
if _, ok := first.Act["node"]; ok {
|
||||
t.Errorf("the act names a node: %v", first.Act)
|
||||
}
|
||||
w.router.choose(h1.Asked, OptionApprove, now)
|
||||
w.g.Decided(h1.Asked)
|
||||
other.Decided(h1.Asked)
|
||||
if *w.ran != 1 {
|
||||
t.Errorf("ran %d", *w.ran)
|
||||
}
|
||||
})
|
||||
t.Run("a state that cannot be read refuses every marked call", func(t *testing.T) {
|
||||
w := setup()
|
||||
w.state.fail = fmt.Errorf("the bus is away")
|
||||
if _, h, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h != nil || *w.ran != 0 {
|
||||
t.Fatalf("held %+v, %v", h, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
// Package approval is the one gate on a call that needs the operator's approval (design 46 7-approval-mark; design 47 G
|
||||
// §677, lead answer A-224): the tool runner's and a module's process's seat server's. It holds the marks it reads from the
|
||||
// catalog half, the act it digests, the ask it raises on the seat operator-channel under the marked module's name, the judging of the same call again on the router's record, the claim, the recording in the
|
||||
// hand-act log, and the refusals when a part of the check is missing (§5, §8 to §20). No module's name is written here.
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscompose"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// Catalog is what the marks are read from: the catalog half the runner follows (design 46 7-approval-mark §5, design 47 G
|
||||
// §196, §443) and the seat definitions it carries (§2).
|
||||
type Catalog interface {
|
||||
// Ready is nil once the catalog half's first view is read, else says what it waits for.
|
||||
Ready() error
|
||||
// Revision is the catalog revision the marks are read at, as the runner names it.
|
||||
Revision() string
|
||||
// Served is the entry of the build a bundle serves, by the fingerprint its composed environment carries; false when
|
||||
// none is read (not yet, or not found), with why.
|
||||
Served(fingerprint string) (Entry, bool, string)
|
||||
// Newest is the module's entry at the newest revision; false when the catalog half lists none.
|
||||
Newest(module string) (Entry, bool)
|
||||
// Definitions are the seat definitions the catalog half carries, by seat.
|
||||
Definitions() map[string]seatdef.Definition
|
||||
// Entries are the newest entry of every module in every followed branch's half, a marked wait's marks (lead answer
|
||||
// A-168).
|
||||
Entries() []Entry
|
||||
}
|
||||
|
||||
// Marks is what one module's calls are gated by: its marked tools and seat verbs, and the subjects of its marked events and
|
||||
// restricted streams, each with its source (§18: its module's entry in the catalog half, or its seat's definition).
|
||||
type Marks struct {
|
||||
Module string
|
||||
// Tools are its marked tools by name, Verbs its marked seat verbs as `<seat>.<verb>`, Subjects the bus subjects of its
|
||||
// marked events and restricted streams (design 34 every-event §33); each to its source.
|
||||
Tools, Verbs, Subjects map[string]string
|
||||
// Revision is the catalog revision read; Served the served entry's fingerprint, empty when it could not be read.
|
||||
Revision, Served string
|
||||
// Unread says why the served entry could not be read: every marked call is then refused (§5, §19).
|
||||
Unread string
|
||||
}
|
||||
|
||||
// Marked says whether a tool or a seat verb of the module needs the operator's approval, and its source.
|
||||
func (m Marks) Marked(seat, name string) (bool, string) {
|
||||
if seat == "" {
|
||||
src, ok := m.Tools[name]
|
||||
return ok, src
|
||||
}
|
||||
src, ok := m.Verbs[seat+"."+name]
|
||||
return ok, src
|
||||
}
|
||||
|
||||
// Any says the module marks anything at all.
|
||||
func (m Marks) Any() bool { return len(m.Tools)+len(m.Verbs)+len(m.Subjects) > 0 }
|
||||
|
||||
// marked is what an entry's manifest says of its marks: `needs-approval`, its events and its streams.
|
||||
type marked struct {
|
||||
NeedsApproval []string `json:"needs-approval"`
|
||||
Emits []string `json:"emits"`
|
||||
// Streams are read for their events and whether each carries a `grant`, and no other word, so a word added to a
|
||||
// stream's declaration later never hides its marks.
|
||||
Streams []struct {
|
||||
Events []string `json:"events"`
|
||||
Grant json.RawMessage `json:"grant"`
|
||||
} `json:"streams"`
|
||||
}
|
||||
|
||||
// Derive is a module's marks from its served entry and its newest one, unioned while they differ so a mark is never lost
|
||||
// before or after the module moves (§5, ADR 0286 §3), and the seat definitions' verb marks for every verb an entry serves,
|
||||
// which no holder unmarks (§2, §3). fingerprint is the build its bundle serves; empty, or an entry not read, leaves the
|
||||
// marks of the newest entry and the seats, with Unread saying why.
|
||||
func Derive(c Catalog, module, fingerprint string) Marks {
|
||||
m := Marks{Module: module, Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{},
|
||||
Revision: c.Revision()}
|
||||
defs := c.Definitions()
|
||||
var entries []Entry
|
||||
if fingerprint == "" {
|
||||
m.Unread = "its bundle's environment carries no fingerprint, so the build it serves cannot be read"
|
||||
} else if e, ok, why := c.Served(fingerprint); ok {
|
||||
entries, m.Served = append(entries, e), fingerprint
|
||||
} else {
|
||||
m.Unread = fmt.Sprintf("the catalog half's entry of %s cannot be read: %s", fingerprint, why)
|
||||
}
|
||||
if e, ok := c.Newest(module); ok && (len(entries) == 0 || e.Fingerprint != entries[0].Fingerprint) {
|
||||
entries = append(entries, e)
|
||||
}
|
||||
for _, e := range entries {
|
||||
m.add(e, defs)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// add takes one entry's marks into m: its `needs-approval` (a tool, a verb it serves or an event it declares), the seat
|
||||
// definitions' marks of every verb it serves, and every stream it declares with a `grant`, each with its source.
|
||||
func (m *Marks) add(e Entry, defs map[string]seatdef.Definition) {
|
||||
source := "entry " + e.Module + " " + orNone(e.Fingerprint)
|
||||
var said marked
|
||||
_ = json.Unmarshal(e.Manifest, &said)
|
||||
serves := map[string]bool{}
|
||||
for _, sv := range e.Serves {
|
||||
serves[sv] = true
|
||||
seat, verb, _ := strings.Cut(sv, ".")
|
||||
for _, v := range defs[seat].Verbs {
|
||||
if v.Name == verb && v.NeedsApproval {
|
||||
m.Verbs[sv] = "seat " + seat
|
||||
}
|
||||
}
|
||||
}
|
||||
events := map[string]bool{}
|
||||
for _, ev := range said.Emits {
|
||||
events[ev] = true
|
||||
}
|
||||
for _, s := range said.Streams {
|
||||
for _, ev := range s.Events {
|
||||
events[ev] = true
|
||||
}
|
||||
}
|
||||
for _, name := range said.NeedsApproval {
|
||||
switch {
|
||||
case serves[name]:
|
||||
if _, bySeat := m.Verbs[name]; !bySeat {
|
||||
m.Verbs[name] = source
|
||||
}
|
||||
case events[name]:
|
||||
for _, s := range eventSubjects(e.Module, name) {
|
||||
m.Subjects[s] = source
|
||||
}
|
||||
default:
|
||||
m.Tools[name] = source
|
||||
}
|
||||
}
|
||||
for _, s := range said.Streams {
|
||||
if len(s.Grant) == 0 || string(s.Grant) == "null" {
|
||||
continue
|
||||
}
|
||||
wire := make([]string, 0, len(s.Events))
|
||||
for _, ev := range s.Events {
|
||||
wire = append(wire, buscompose.WirePattern(ev))
|
||||
}
|
||||
for _, sub := range (buscompose.ModuleStream{Module: e.Module, Events: wire}).Subjects() {
|
||||
m.Subjects[sub] = source
|
||||
if !strings.HasSuffix(sub, ">") {
|
||||
m.Subjects[sub+".>"] = source
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// eventSubjects are a marked event's subjects on the bus (design 34 every-event §33): said on the module's events and on its
|
||||
// own stream, each also with every subject below it.
|
||||
func eventSubjects(module, event string) []string {
|
||||
ev := buscompose.WirePattern(event)
|
||||
out := []string{"mesh.mod." + module + ".event." + ev, buscompose.ModuleStreamSubject(module, ev)}
|
||||
if !strings.HasSuffix(ev, ">") {
|
||||
out = append(out, out[0]+".>", out[1]+".>")
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func orNone(s string) string {
|
||||
if s == "" {
|
||||
return "none"
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -1,255 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/keys"
|
||||
)
|
||||
|
||||
// The bounds of a run's recording (§13, §16; lead answers A-157, A-158).
|
||||
const (
|
||||
UnansweredAfter = 5 * time.Minute
|
||||
KeptFor = 48 * time.Hour
|
||||
SaidBefore = time.Hour
|
||||
)
|
||||
|
||||
// Bounds is the runner's key-ttl state `bound` (lead answers A-157, A-158): a key created with its own time to live, which
|
||||
// the bus removes when it passes, every watch hearing the removal as an expiry, so the gate's waits need no timer.
|
||||
type Bounds interface {
|
||||
CreateTTL(key string, value any, ttl time.Duration) error
|
||||
Delete(key string) error
|
||||
// Watch hands expired every key under the pattern whose time to live passed, until the stop is called.
|
||||
Watch(pattern string, expired func(key string)) (stop func(), err error)
|
||||
}
|
||||
|
||||
// Answered is a call run on the operator's approval: the tool's own answer and the line naming the approval (§16), in the
|
||||
// form a process holder's `gated-ended` says it (design 47 G §677; lead answer A-158).
|
||||
type Answered struct {
|
||||
Answer any `json:"answer"`
|
||||
Approval string `json:"approval"`
|
||||
}
|
||||
|
||||
// The bound keys of a run (lead answer A-158; §13).
|
||||
func (g *Gate) unansweredKey(id string) string { return g.d.Node + ".unanswered." + id }
|
||||
func (g *Gate) unrecordedKey(id string) string { return g.d.Node + ".unrecorded." + id }
|
||||
|
||||
// ran is a call run on the operator's approval: answered with the tool's answer and the approval's line, and its warrant
|
||||
// recorded in the hand-act log (§16).
|
||||
func (g *Gate) ran(c Call, id string, rec *Record, answer any, err error) (any, error) {
|
||||
channel := ""
|
||||
at := ""
|
||||
if w := rec.Warrant; w != nil {
|
||||
channel, at = w.Channel, w.At
|
||||
}
|
||||
if channel == "" {
|
||||
channel = strings.Join(rec.OfferedOn, ", ")
|
||||
}
|
||||
line := fmt.Sprintf("run on the operator's approval via %s at %s, ask %s", orNone(channel), at, id)
|
||||
g.d.Logf("[approval] %s on %s %s", c.Address, g.d.Node, line)
|
||||
g.bindRecording(c.Module, id)
|
||||
g.d.Go(func() { g.recordRun(c.Module, id) })
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w (%s)", err, line)
|
||||
}
|
||||
return Answered{Answer: answer, Approval: line}, nil
|
||||
}
|
||||
|
||||
// bindRecording keeps the run's two bound keys: one that raises `call-unanswered` when `warranted` has not answered within
|
||||
// five minutes, one that says an unrecorded run an hour before its key's 48 hours end (§13, §16).
|
||||
func (g *Gate) bindRecording(module, id string) {
|
||||
if g.d.Bounds == nil || g.d.Holding { // a process holder's bound keys are its waiting calls' alone (A-255)
|
||||
return
|
||||
}
|
||||
for key, ttl := range map[string]time.Duration{g.unansweredKey(id): UnansweredAfter, g.unrecordedKey(id): KeptFor - SaidBefore} {
|
||||
if err := g.d.Bounds.CreateTTL(key, map[string]string{"asker": module, "ask": id}, ttl); err != nil {
|
||||
g.d.Logf("[approval] the bound %s of the run of ask %s is not kept: %v", key, id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// recordRun calls `mesh-hand-acts.warranted` for one acted ask, each try a kept call open while the ask lacks `recorded`
|
||||
// (§16, A-157), and on its answer writes `recorded` and clears what waited on it. One recording per ask at a time.
|
||||
func (g *Gate) recordRun(module, id string) {
|
||||
if !g.recording(id, true) {
|
||||
return
|
||||
}
|
||||
defer g.recording(id, false)
|
||||
if g.d.Warranted == nil {
|
||||
g.d.Logf("[approval] the run of ask %s is not recorded: the tool runner reaches no hand-act log", id)
|
||||
return
|
||||
}
|
||||
err := g.d.Warranted(module, id, func() (bool, error) { return g.unrecorded(id) })
|
||||
g.judged(id) // made again: its answer, or the bound of its key, judges it from now
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] mesh-hand-acts.warranted has not answered for ask %s (%v): asked again when the hand-act seat's "+
|
||||
"holder is heard again and at the next start", id, err)
|
||||
return
|
||||
}
|
||||
if err := g.markRecorded(id); err != nil {
|
||||
g.d.Logf("[approval] ask %s is recorded in the hand-act log and its mark recorded is not kept: %v", id, err)
|
||||
return
|
||||
}
|
||||
if g.d.Bounds != nil {
|
||||
_ = g.d.Bounds.Delete(g.unansweredKey(id))
|
||||
_ = g.d.Bounds.Delete(g.unrecordedKey(id))
|
||||
}
|
||||
g.finding(id, KindCallUnanswered, false, "mesh-hand-acts.warranted answered for ask "+id, "")
|
||||
}
|
||||
|
||||
// recording claims or releases an ask's recording: false when one runs already.
|
||||
func (g *Gate) recording(id string, start bool) bool {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.inFlight == nil {
|
||||
g.inFlight = map[string]bool{}
|
||||
}
|
||||
if start {
|
||||
if g.inFlight[id] {
|
||||
return false
|
||||
}
|
||||
g.inFlight[id] = true
|
||||
return true
|
||||
}
|
||||
delete(g.inFlight, id)
|
||||
return true
|
||||
}
|
||||
|
||||
// kept is the ask the client keeps under its id, with its revision; nil when there is none.
|
||||
func (g *Gate) kept(id string) (*kept, uint64, error) {
|
||||
e, err := g.d.State.Get(g.askKey(id))
|
||||
if err != nil || e == nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
var k kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
return &k, e.Revision, nil
|
||||
}
|
||||
|
||||
// unrecorded says the ask is acted on and its run not yet recorded: the key that keeps `warranted` open (§16).
|
||||
func (g *Gate) unrecorded(id string) (bool, error) {
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil || k == nil {
|
||||
return false, err
|
||||
}
|
||||
return k.Acted && !k.Recorded, nil
|
||||
}
|
||||
|
||||
// markRecorded writes `recorded` on the kept ask by compare-and-set, again when another write came first.
|
||||
func (g *Gate) markRecorded(id string) error {
|
||||
for {
|
||||
k, rev, err := g.kept(id)
|
||||
if err != nil || k == nil {
|
||||
return fmt.Errorf("the kept ask cannot be read: %v", err)
|
||||
}
|
||||
if k.Recorded {
|
||||
return nil
|
||||
}
|
||||
k.Recorded = true
|
||||
if _, err := g.d.State.Update(g.askKey(id), k, rev); err == nil {
|
||||
return nil
|
||||
} else if !isChanged(err) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Recover calls `warranted` again for every acted, unrecorded ask this machine's keys hold within their 48 hours (§16): at
|
||||
// each start, and when the hand-act seat's holder is heard again (A-157).
|
||||
func (g *Gate) Recover() {
|
||||
if g.d.State == nil {
|
||||
return
|
||||
}
|
||||
keys, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
g.d.Logf("[approval] the acted, unrecorded asks cannot be read (%v): read again at the next start or holder heard", err)
|
||||
return
|
||||
}
|
||||
prefix := g.prefix() + "ask."
|
||||
for _, key := range keys {
|
||||
if !strings.HasPrefix(key, prefix) {
|
||||
continue
|
||||
}
|
||||
id := strings.TrimPrefix(key, prefix)
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil || k == nil || !k.Acted || k.Recorded || g.d.Now().Sub(k.Asked) > KeptFor {
|
||||
continue
|
||||
}
|
||||
module := moduleOf(k)
|
||||
g.pend(id)
|
||||
g.d.Go(func() { g.recordRun(module, id) })
|
||||
}
|
||||
}
|
||||
|
||||
// moduleOf is the module an ask was asked under: the warrant's asker.
|
||||
func moduleOf(k *kept) string {
|
||||
var w struct {
|
||||
Asker string `json:"asker"`
|
||||
}
|
||||
if k.Answer != nil {
|
||||
_ = json.Unmarshal(*k.Answer, &w)
|
||||
}
|
||||
return w.Asker
|
||||
}
|
||||
|
||||
// WatchBounds hears the run's bound keys expire (A-157): `call-unanswered` raised when `warranted` has not answered within
|
||||
// five minutes; an unrecorded run said in the log an hour before its key's 48 hours end (§13).
|
||||
func (g *Gate) WatchBounds() (func(), error) {
|
||||
if g.d.Bounds == nil {
|
||||
return func() {}, nil
|
||||
}
|
||||
return g.d.Bounds.Watch(g.d.Node+".**", func(key string) {
|
||||
if key == g.unreadableKey() {
|
||||
g.unreadableExpired()
|
||||
return
|
||||
}
|
||||
rest := strings.TrimPrefix(key, g.d.Node+".")
|
||||
kind, id, _ := strings.Cut(rest, ".")
|
||||
if open, err := g.unrecorded(id); err != nil || !open {
|
||||
return
|
||||
}
|
||||
switch kind {
|
||||
case "unanswered":
|
||||
g.finding(id, KindCallUnanswered, true, fmt.Sprintf("mesh-hand-acts.warranted has not answered for ask %s, run on the "+
|
||||
"operator's approval on %s, within %s: the run is not yet in the hand-act log", id, g.d.Node, UnansweredAfter),
|
||||
"a run's hand-act is unrecorded on "+g.d.Node)
|
||||
case "unrecorded":
|
||||
g.d.Logf("[approval] ERROR the run of ask %s on %s is still unrecorded an hour before its key's 48 hours end", id, g.d.Node)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Pender is the held writer's pending (views.HeldWriter's Pend and Judged), by a finding's kind and id: the findings it
|
||||
// cannot judge yet since its start, named in its `findings-stated` (design 47 G §384).
|
||||
type Pender interface {
|
||||
Pend(kind, id string, since time.Time, bound time.Duration)
|
||||
Judged(kind, id string)
|
||||
}
|
||||
|
||||
// pend names an acted, unrecorded ask's `call-unanswered` in the runner's `findings-stated` until its `warranted` is made
|
||||
// again (§16; design 47 G §384), its bound the five minutes the finding waits.
|
||||
func (g *Gate) pend(id string) {
|
||||
if p, ok := g.d.Findings.(Pender); ok {
|
||||
p.Pend(KindCallUnanswered, g.findingID(id), g.d.Now().UTC(), UnansweredAfter)
|
||||
}
|
||||
}
|
||||
|
||||
// judged ends that pending.
|
||||
func (g *Gate) judged(id string) {
|
||||
if p, ok := g.d.Findings.(Pender); ok {
|
||||
p.Judged(KindCallUnanswered, g.findingID(id))
|
||||
}
|
||||
}
|
||||
|
||||
// findingID is the id of a finding about what on this machine, `<node>.<what>` escaped as a key's name is (lead answer
|
||||
// A-154).
|
||||
func (g *Gate) findingID(what string) string { return keys.EscapeName(g.d.Node + "." + what) }
|
||||
|
||||
// isChanged says a compare-and-set lost to another write.
|
||||
func isChanged(err error) bool { return errors.Is(err, stdio.ErrChanged) }
|
||||
@@ -1,176 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
)
|
||||
|
||||
// Run is a call through the gate (§20): an unmarked call runs; a marked one runs only on the operator's approval of that
|
||||
// one call, judged on the router's record and claimed once (§14, §15), and is otherwise answered as an error saying why
|
||||
// (§12, §17, §19). run is called at most once.
|
||||
func (g *Gate) Run(c Call, run func() (any, error)) (any, error) {
|
||||
m, err := g.marksOf(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if missing := g.unservedOf(c.Module); len(missing) > 0 {
|
||||
return nil, fmt.Errorf("Not run: %s's marks name %s, which its bundle on %s does not register, so none of its tools is "+
|
||||
"served here (design 46 7-approval-mark §4)", c.Module, strings.Join(missing, ", "), g.d.Node)
|
||||
}
|
||||
marked, _ := m.Marked(c.Seat, c.Name)
|
||||
since := g.markedSince(c, marked, m.Revision)
|
||||
if !marked {
|
||||
return run()
|
||||
}
|
||||
if c.Own {
|
||||
return nil, g.refused(c, "", "it is one of the tool runner's own tools, which no ask covers")
|
||||
}
|
||||
if m.Unread != "" {
|
||||
return nil, g.refused(c, "", m.Unread)
|
||||
}
|
||||
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
|
||||
return nil, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in")
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, g.refused(c, "", why)
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, "", err.Error())
|
||||
}
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
|
||||
}
|
||||
g.stateRead()
|
||||
if named == nil {
|
||||
return nil, g.raise(c, act, digest, since)
|
||||
}
|
||||
var id string
|
||||
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
|
||||
return nil, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
|
||||
}
|
||||
return g.judge(c, act, digest, id, since, run)
|
||||
}
|
||||
|
||||
// Binding is a Store that says whether the runner's membership binds the state now (§13).
|
||||
type Binding interface{ Bound() bool }
|
||||
|
||||
// marksOf is the module's marks as the catalog half has them now; before its first view, the call waits for it within
|
||||
// RecordWithin and is then judged, refused naming mesh_first_views when it is still unread (§19).
|
||||
func (g *Gate) marksOf(c Call) (Marks, error) {
|
||||
if g.d.Catalog == nil {
|
||||
return Marks{}, g.refused(c, "", "the tool runner reads no catalog half, so its marks are not read")
|
||||
}
|
||||
if err := g.d.Catalog.Ready(); err != nil {
|
||||
if g.d.Wait != nil {
|
||||
err = g.d.Wait(RecordWithin)
|
||||
}
|
||||
if err != nil {
|
||||
return Marks{}, fmt.Errorf("Not run: the marks of %s are not read yet (%v); mesh_first_views says what the tool runner "+
|
||||
"waits for", c.Module, err)
|
||||
}
|
||||
}
|
||||
m := Derive(g.d.Catalog, c.Module, c.Fingerprint)
|
||||
g.checkServed(m)
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func (g *Gate) unservedOf(module string) []string {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.unserved[module]
|
||||
}
|
||||
|
||||
// markedSince is the revision a call was marked at when the same caller last called it unmarked (§12), empty otherwise.
|
||||
func (g *Gate) markedSince(c Call, marked bool, revision string) string {
|
||||
key := c.Caller + " " + c.Address
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
if g.unmarkedAt == nil {
|
||||
g.unmarkedAt = map[string]bool{}
|
||||
}
|
||||
was := g.unmarkedAt[key]
|
||||
if marked {
|
||||
delete(g.unmarkedAt, key)
|
||||
if was {
|
||||
return revision
|
||||
}
|
||||
return ""
|
||||
}
|
||||
g.unmarkedAt[key] = true
|
||||
return ""
|
||||
}
|
||||
|
||||
// judge is the same call made again, judged on the router's own record of its act's newest ask (§14 to §17).
|
||||
func (g *Gate) judge(c Call, act operatorask.Act, digest, id, since string, run func() (any, error)) (any, error) {
|
||||
rec, err := g.record(c.Module, id)
|
||||
if err != nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
|
||||
}
|
||||
if rec == nil {
|
||||
return nil, g.firstAnswer(c, id, digest, since) // §17: not yet taken by the router
|
||||
}
|
||||
again := func(why string) (any, error) {
|
||||
err := g.raise(c, act, digest, since)
|
||||
return nil, fmt.Errorf("The ask %s %s; asked again. %w", id, why, err)
|
||||
}
|
||||
switch rec.State {
|
||||
case "open", "held-back":
|
||||
return nil, fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
case "refused":
|
||||
return nil, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer))
|
||||
case "chosen":
|
||||
default: // expired, cancelled, replaced: each asks again (§15, §17)
|
||||
return again("ended " + rec.State)
|
||||
}
|
||||
w := rec.Warrant
|
||||
if w == nil {
|
||||
return nil, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant")
|
||||
}
|
||||
at, _ := parseTime(w.At)
|
||||
if w.Option == OptionDecline {
|
||||
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
|
||||
return nil, fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s",
|
||||
id, formatTime(until))
|
||||
}
|
||||
return again("was declined more than 10 minutes ago")
|
||||
}
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
return again("was approved too long ago (more than 30 minutes)")
|
||||
}
|
||||
var answer any
|
||||
var ran error
|
||||
h, err := g.d.Channel.Claim(c.Module, *w, act, func() { answer, ran = run() })
|
||||
switch {
|
||||
case h == ClaimDone:
|
||||
return again("was already run")
|
||||
case h == ClaimRefused:
|
||||
return nil, g.refused(c, digest, "its warrant does not hold: "+errText(err))
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return nil, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error())
|
||||
case err != nil: // the act about to run is not the one the option bound: nothing runs, the warrant stays used
|
||||
return nil, g.refused(c, digest, err.Error())
|
||||
}
|
||||
return g.ran(c, id, rec, answer, ran)
|
||||
}
|
||||
|
||||
func errText(err error) string {
|
||||
if err == nil {
|
||||
return "no reason given"
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The mesh MCP server's mesh_approvals reads and steers what the gate asked on this machine (§18).
|
||||
|
||||
// Listed is one ask of the last day as mesh_approvals lists it: its state and its act's fingerprint.
|
||||
type Listed struct {
|
||||
Ask string `json:"ask"`
|
||||
Address string `json:"address"`
|
||||
Module string `json:"module"`
|
||||
Asked time.Time `json:"asked"`
|
||||
State string `json:"state"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
}
|
||||
|
||||
// Whole is one ask read whole: its arguments and the act's digest.
|
||||
type Whole struct {
|
||||
Listed
|
||||
Digest string `json:"digest"`
|
||||
Act map[string]string `json:"act"`
|
||||
Arguments json.RawMessage `json:"arguments"`
|
||||
Recorded bool `json:"recorded"`
|
||||
}
|
||||
|
||||
// stateOf is a kept ask's state as the runner knows it.
|
||||
func stateOf(acted, recorded bool, ended string) string {
|
||||
switch {
|
||||
case acted && recorded:
|
||||
return "run, recorded"
|
||||
case acted:
|
||||
return "run, not yet recorded"
|
||||
case ended != "":
|
||||
return ended
|
||||
}
|
||||
return "asked"
|
||||
}
|
||||
|
||||
// List is the calls asked on this machine in the last day, newest first (§18).
|
||||
func (g *Gate) List() ([]Listed, error) {
|
||||
if g.d.State == nil {
|
||||
return nil, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
|
||||
}
|
||||
keys, err := g.d.State.Keys()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
prefix := g.d.Node + ".ask."
|
||||
out := []Listed{}
|
||||
for _, key := range keys {
|
||||
if !strings.HasPrefix(key, prefix) {
|
||||
continue
|
||||
}
|
||||
w, err := g.Read(strings.TrimPrefix(key, prefix))
|
||||
if err != nil || g.d.Now().Sub(w.Asked) > 24*time.Hour {
|
||||
continue
|
||||
}
|
||||
out = append(out, w.Listed)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Asked.After(out[j].Asked) })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Read is one ask whole: its arguments and the act's digest (§18).
|
||||
func (g *Gate) Read(id string) (Whole, error) {
|
||||
if g.d.State == nil {
|
||||
return Whole{}, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
|
||||
}
|
||||
k, _, err := g.kept(id)
|
||||
if err != nil {
|
||||
return Whole{}, err
|
||||
}
|
||||
if k == nil {
|
||||
return Whole{}, fmt.Errorf("no ask %s is kept on %s", id, g.d.Node)
|
||||
}
|
||||
module := k.Act["module"]
|
||||
if module == "" {
|
||||
module = moduleOf(k)
|
||||
}
|
||||
var ask struct {
|
||||
About string `json:"about"`
|
||||
}
|
||||
_ = json.Unmarshal(k.Ask, &ask)
|
||||
return Whole{Listed: Listed{Ask: id, Address: k.Act["address"], Module: module, Asked: k.Asked,
|
||||
State: stateOf(k.Acted, k.Recorded, k.Ended), Fingerprint: Fingerprint(ask.About)},
|
||||
Digest: ask.About, Act: k.Act, Arguments: k.Arguments, Recorded: k.Recorded}, nil
|
||||
}
|
||||
|
||||
// Applied are the marks the gate applies on this machine, each module's marked tools and verbs with their source and the
|
||||
// catalog revision read (§18).
|
||||
func (g *Gate) Applied() []Marks {
|
||||
g.mu.Lock()
|
||||
modules := make([]string, 0, len(g.served))
|
||||
for m := range g.served {
|
||||
modules = append(modules, m)
|
||||
}
|
||||
g.mu.Unlock()
|
||||
sort.Strings(modules)
|
||||
var out []Marks
|
||||
if g.d.Catalog == nil {
|
||||
return out
|
||||
}
|
||||
for _, m := range modules {
|
||||
if marks := Derive(g.d.Catalog, m, g.servedFingerprint(m)); marks.Any() {
|
||||
out = append(out, marks)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (g *Gate) servedFingerprint(module string) string {
|
||||
g.mu.Lock()
|
||||
defer g.mu.Unlock()
|
||||
return g.served[module]
|
||||
}
|
||||
|
||||
// Describe says whether a tool or a seat verb of a module is marked, and its source, for mesh_describe (§18): the served
|
||||
// build's marks where this machine serves the module, else the newest entry's and its seat's.
|
||||
func (g *Gate) Describe(module, seat, name string) (bool, string) {
|
||||
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
|
||||
return false, ""
|
||||
}
|
||||
if seat != "" && module == "" {
|
||||
for s, d := range g.d.Catalog.Definitions() {
|
||||
if s != seat {
|
||||
continue
|
||||
}
|
||||
for _, v := range d.Verbs {
|
||||
if v.Name == name && v.NeedsApproval {
|
||||
return true, "seat " + seat
|
||||
}
|
||||
}
|
||||
}
|
||||
return false, ""
|
||||
}
|
||||
return Derive(g.d.Catalog, module, g.servedFingerprint(module)).Marked(seat, name)
|
||||
}
|
||||
|
||||
// Again asks `warranted` again for one acted, unrecorded ask, by hand, in the kept-call form of §16 (§18); the caller
|
||||
// records the hand act first.
|
||||
func (g *Gate) Again(id string) error {
|
||||
w, err := g.Read(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.HasPrefix(w.State, "run, not yet") {
|
||||
return fmt.Errorf("the ask %s is %s: only a run not yet recorded is asked again", id, w.State)
|
||||
}
|
||||
g.recordRun(w.Module, id)
|
||||
if again, _ := g.Read(id); !again.Recorded {
|
||||
return fmt.Errorf("mesh-hand-acts.warranted has not answered for ask %s yet", id)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Cancel takes an open ask back through operator-channel's client (§18); the caller records the hand act first.
|
||||
func (g *Gate) Cancel(id string) error {
|
||||
w, err := g.Read(id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if w.State != "asked" {
|
||||
return fmt.Errorf("the ask %s is %s, not open", id, w.State)
|
||||
}
|
||||
return g.d.Channel.Cancel(w.Module, id)
|
||||
}
|
||||
@@ -1,290 +0,0 @@
|
||||
package approval
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscompose"
|
||||
"git.novox.be/novox/mesh-sdk/go/operatorask"
|
||||
"git.novox.be/novox/mesh-sdk/go/outward"
|
||||
)
|
||||
|
||||
// Wait is one mesh_wait call as the gate judges it (design 34 every-event §34 to §36): the event names given, the subjects
|
||||
// they became (after `from`), the filter and the timeout.
|
||||
type Wait struct {
|
||||
Names []string
|
||||
From string
|
||||
Subjects []string
|
||||
Filter map[string]any
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// Waited is a wait through the gate (every-event §34 to §36). An unmarked wait is heard at once by hear, given how long it
|
||||
// may wait. A marked one is heard only on the operator's warrant for that one wait, its timeout running from then; with
|
||||
// none it subscribes to nothing, its ask raised once under the runner's own module's name, and is answered as an error at
|
||||
// its timeout, when its caller goes, or as soon as the router's record says the ask was chosen, never in a hot loop. c is
|
||||
// the call as the console makes it: its address, its tool's name, its caller and its arguments as given.
|
||||
func (g *Gate) Waited(ctx context.Context, c Call, w Wait, hear func(within time.Duration) (any, error)) (any, error) {
|
||||
deadline := g.d.Now().Add(w.Timeout)
|
||||
// A wait from a seat is never marked, and a seat's event never is (§32, §34).
|
||||
if w.From == "seat" || !outsideSeats(w.Subjects) {
|
||||
return hear(w.Timeout)
|
||||
}
|
||||
if g.d.Catalog == nil {
|
||||
return nil, g.holdRefused(ctx, deadline, "this wait is refused: the tool runner reads no catalog half, so the marks are not read")
|
||||
}
|
||||
if err := g.d.Catalog.Ready(); err != nil {
|
||||
// Before the marks are read nothing is asked: the wait waits for the first view within its own timeout and is
|
||||
// then judged, and is refused at its timeout when it does not come (§34).
|
||||
if g.d.Wait != nil {
|
||||
err = g.waitCatalog(ctx, w.Timeout)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, fmt.Sprintf("this wait is refused at its timeout: the marks are not yet read (%v); "+
|
||||
"mesh_first_views says what the tool runner waits for", err))
|
||||
}
|
||||
}
|
||||
if !overlapsAny(w.Subjects, g.waitMarks()) {
|
||||
return hear(deadline.Sub(g.d.Now()))
|
||||
}
|
||||
c.Module, c.Seat, c.Fingerprint = g.d.Runner, "", g.d.Build
|
||||
if c.Fingerprint == "" {
|
||||
return nil, g.holdRefused(ctx, deadline, "this wait needs the operator's approval, and the tool runner's own fingerprint is "+
|
||||
"unknown (its MESH_FINGERPRINT was not set at its start), so no warrant can bind its build")
|
||||
}
|
||||
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in").Error())
|
||||
}
|
||||
if why := refusedUnasked(c.Args); why != "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
|
||||
}
|
||||
if why := g.held(); why != "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
|
||||
}
|
||||
act, err := ActOf(c, g.d.Node)
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
|
||||
}
|
||||
digest, err := operatorask.ActDigest(act)
|
||||
if err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
|
||||
}
|
||||
named, err := g.d.State.Get(g.actKey(digest))
|
||||
if err != nil {
|
||||
g.stateFailed(err)
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error()).Error())
|
||||
}
|
||||
g.stateRead()
|
||||
anew := func() (any, error) {
|
||||
id := newID()
|
||||
if err := g.keepAsk(c, act, digest, g.askForWait(c, w, digest, id)); err != nil {
|
||||
return nil, g.holdRefused(ctx, deadline, err.Error())
|
||||
}
|
||||
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, true)
|
||||
}
|
||||
if named == nil {
|
||||
return anew()
|
||||
}
|
||||
var id string
|
||||
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName)).Error())
|
||||
}
|
||||
rec, err := g.record(c.Module, id)
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error()).Error())
|
||||
case rec == nil, rec.State == "open", rec.State == "held-back":
|
||||
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, false)
|
||||
case rec.State == "refused":
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer)).Error())
|
||||
case rec.State != "chosen":
|
||||
return anew() // expired, cancelled, replaced: asked again (§15, §17)
|
||||
}
|
||||
wt := rec.Warrant
|
||||
if wt == nil {
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant").Error())
|
||||
}
|
||||
at, _ := parseTime(wt.At)
|
||||
if wt.Option == OptionDecline {
|
||||
if g.d.Now().Before(at.Add(DeclinedFor)) {
|
||||
return nil, g.holdRefused(ctx, deadline, declinedWords(id, at))
|
||||
}
|
||||
return anew()
|
||||
}
|
||||
if g.d.Now().After(at.Add(RunWithin)) {
|
||||
return anew()
|
||||
}
|
||||
var answer any
|
||||
var ran error
|
||||
// On the warrant it subscribes, and the wait's timeout runs from then (§35).
|
||||
h, err := g.d.Channel.Claim(c.Module, *wt, act, func() { answer, ran = hear(w.Timeout) })
|
||||
switch {
|
||||
case h == ClaimDone:
|
||||
return anew() // already run: one warrant is one wait (§35)
|
||||
case h == ClaimRefused:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its warrant does not hold: "+errText(err)).Error())
|
||||
case h == "" && err != nil:
|
||||
g.stateFailed(err)
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error()).Error())
|
||||
case err != nil:
|
||||
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, err.Error()).Error())
|
||||
}
|
||||
return g.ran(c, id, rec, answer, ran)
|
||||
}
|
||||
|
||||
// outsideSeats says a subject is not a seat's: only such a subject can be marked (§32, §34).
|
||||
func outsideSeats(subjects []string) bool {
|
||||
for _, s := range subjects {
|
||||
if !strings.HasPrefix(s, "mesh.seat.") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// waitMarks are the marked subjects of every module the catalog half lists (§33): the newest entry of every followed
|
||||
// branch, and the served entry of each module this runner serves (lead answer A-168), so a mark is never lost.
|
||||
func (g *Gate) waitMarks() map[string]string {
|
||||
m := Marks{Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{}}
|
||||
defs := g.d.Catalog.Definitions()
|
||||
for _, e := range g.d.Catalog.Entries() {
|
||||
m.add(e, defs)
|
||||
}
|
||||
g.mu.Lock()
|
||||
served := make(map[string]string, len(g.served))
|
||||
for module, fp := range g.served {
|
||||
served[module] = fp
|
||||
}
|
||||
g.mu.Unlock()
|
||||
for _, fp := range served {
|
||||
if e, ok, _ := g.d.Catalog.Served(fp); ok {
|
||||
m.add(e, defs)
|
||||
}
|
||||
}
|
||||
return m.Subjects
|
||||
}
|
||||
|
||||
// overlapsAny says a subject a wait subscribes overlaps a marked subject (§34).
|
||||
func overlapsAny(subjects []string, marked map[string]string) bool {
|
||||
for _, s := range subjects {
|
||||
for mk := range marked {
|
||||
if buscompose.Overlaps(s, mk) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// waitCatalog waits for the catalog half's first view within the bound, or until the caller goes.
|
||||
func (g *Gate) waitCatalog(ctx context.Context, within time.Duration) error {
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- g.d.Wait(within) }()
|
||||
select {
|
||||
case err := <-done:
|
||||
return err
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
|
||||
// pause waits until the deadline or the caller's going.
|
||||
func (g *Gate) pause(ctx context.Context, deadline time.Time) {
|
||||
if left := deadline.Sub(g.d.Now()); left > 0 {
|
||||
select {
|
||||
case <-g.d.After(left):
|
||||
case <-ctx.Done():
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// holdRefused is a refused marked wait, said at its timeout or when its caller goes, never at once, so a watcher's loop
|
||||
// is never a hot one (§36).
|
||||
func (g *Gate) holdRefused(ctx context.Context, deadline time.Time, words string) error {
|
||||
g.pause(ctx, deadline)
|
||||
if !strings.HasPrefix(words, "Not run") {
|
||||
words = "Not run: " + words
|
||||
}
|
||||
return fmt.Errorf("%s", words)
|
||||
}
|
||||
|
||||
// holdAsk holds a marked wait whose ask is open, subscribing to nothing but its ask's `decided` (§36): it answers, as an
|
||||
// error, at its timeout, when its caller goes, or once the router's record says the ask was chosen, the event only waking it.
|
||||
func (g *Gate) holdAsk(ctx context.Context, deadline time.Time, module, id, digest string, first bool) error {
|
||||
for {
|
||||
left := deadline.Sub(g.d.Now())
|
||||
heard := false
|
||||
if left > 0 && ctx.Err() == nil {
|
||||
var err error
|
||||
if g.d.Hear != nil {
|
||||
heard, err = g.d.Hear(ctx, []string{g.d.Channel.Decided(module)}, left, func(body json.RawMessage) bool {
|
||||
var w struct {
|
||||
Ask string `json:"ask"`
|
||||
}
|
||||
return json.Unmarshal(body, &w) == nil && w.Ask == id
|
||||
})
|
||||
}
|
||||
if g.d.Hear == nil || err != nil {
|
||||
g.pause(ctx, deadline)
|
||||
}
|
||||
}
|
||||
rec, _ := g.record(module, id)
|
||||
if heard && (rec == nil || rec.State != "chosen") {
|
||||
continue // the event only wakes it: the record says nothing chosen yet
|
||||
}
|
||||
return g.waitWords(rec, id, digest, first)
|
||||
}
|
||||
}
|
||||
|
||||
// waitWords are §12's and §17's words for a marked wait not run, "this wait" for "the call" (§36).
|
||||
func (g *Gate) waitWords(rec *Record, id, digest string, first bool) error {
|
||||
switch {
|
||||
case rec == nil:
|
||||
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make this wait again later", id, Fingerprint(digest))
|
||||
case rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionDecline:
|
||||
at, _ := parseTime(rec.Warrant.At)
|
||||
return fmt.Errorf("%s", declinedWords(id, at))
|
||||
case rec.State == "chosen", first:
|
||||
return fmt.Errorf("Not run: this wait needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make this wait "+
|
||||
"again within 30 minutes of the approval; the ask expires at %s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
case rec.State == "open", rec.State == "held-back":
|
||||
return fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
|
||||
id, Fingerprint(digest), rec.Expires)
|
||||
}
|
||||
return fmt.Errorf("Not run: the ask %s ended %s; make this wait again to ask anew", id, rec.State)
|
||||
}
|
||||
|
||||
func declinedWords(id string, at time.Time) string {
|
||||
return fmt.Sprintf("Not run: the operator declined ask %s; this wait is not asked again until %s", id,
|
||||
formatTime(at.Add(DeclinedFor)))
|
||||
}
|
||||
|
||||
// askForWait is a marked wait's ask (§35): "Wait on <names> on <node>?", shortened as §10 says, and an explanation naming
|
||||
// every event name one per line, the filter, and the bound as the timeout in minutes, then the act's fingerprint.
|
||||
func (g *Gate) askForWait(c Call, w Wait, digest, id string) Ask {
|
||||
a := g.askFor(c, digest, id)
|
||||
filter, whole := "none", true
|
||||
if len(w.Filter) > 0 {
|
||||
raw, _ := json.Marshal(w.Filter)
|
||||
filter = string(raw)
|
||||
if r, ok := outward.Check(filter); !ok {
|
||||
filter, whole = shapeOf(r), false
|
||||
}
|
||||
}
|
||||
a.Headline = headlineOf("Wait on", strings.Join(w.Names, ", "), g.d.Node, whole)
|
||||
lines := append([]string{"Wait on " + g.d.Node + ", called through the mesh's tools, which cannot tell who called, for:"}, w.Names...)
|
||||
lines = append(lines, "Filter: "+filter, fmt.Sprintf("Timeout: %s minutes", minutes(w.Timeout)), "Fingerprint "+Fingerprint(digest))
|
||||
a.Explanation = strings.Join(lines, "\n")
|
||||
a.Options[0].Does = "the same wait, made again within 30 minutes, subscribes once, its timeout running from then"
|
||||
a.Options[1].Does = "nothing is heard; the same wait is not asked again for 10 minutes"
|
||||
return a
|
||||
}
|
||||
|
||||
// minutes is a timeout in minutes, as few digits as say it.
|
||||
func minutes(d time.Duration) string {
|
||||
return strings.TrimSuffix(strings.TrimRight(fmt.Sprintf("%.2f", d.Minutes()), "0"), ".")
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
// Package asker is a module's client for asking a person (novox/hq ADR 0259 §10): it publishes an ask under
|
||||
// the module's own name, cancels it, reads its state from the router's record, and hands the module each
|
||||
// warrant once — only one for an ask it made, for an option it offered, at that option's level, before it
|
||||
// expired. The module then acts with its own grants.
|
||||
//
|
||||
// It keeps what it asked, and which warrants it acted on, in a state of the module's own (declared in its
|
||||
// manifest, by default `asked`): the ask is written before it is published, and a warrant is marked acted on
|
||||
// by compare-and-set before the module's handler runs, so a redelivery, or a second instance, acts once.
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
// Client asks as one module.
|
||||
type Client struct {
|
||||
// Module is the asking module's name: the one the bus grants its asks under.
|
||||
Module string
|
||||
// State is the module's own state the asks are kept in.
|
||||
State Store
|
||||
// Now is the clock; nil is time.Now.
|
||||
Now func() time.Time
|
||||
// Publish, Record and Subscribe reach the bus; nil is the stdio SDK's, through the module's runtime.
|
||||
Publish func(subject string, body any, id string) error
|
||||
Record func(key string) (*stdio.StateEntry, error)
|
||||
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
|
||||
}
|
||||
|
||||
// Store is where the asks are kept: stdio.State(name) is one.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
}
|
||||
|
||||
// New is a client for the module the runtime serves, keeping its asks in the state named.
|
||||
func New(state string) *Client {
|
||||
return &Client{Module: os.Getenv("MESH_MODULE"), State: stdio.State(state)}
|
||||
}
|
||||
|
||||
func (c *Client) now() time.Time {
|
||||
if c.Now != nil {
|
||||
return c.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
// Kept is one ask as the module keeps it.
|
||||
type Kept struct {
|
||||
Ask asks.Ask `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended asks.Outcome `json:"ended,omitempty"`
|
||||
Answer *asks.Warrant `json:"answer,omitempty"`
|
||||
}
|
||||
|
||||
func key(id string) string { return "ask." + id }
|
||||
|
||||
// Ask checks an ask, keeps it, and publishes it under the module's own name. An ask with an id already kept
|
||||
// is refused: a new question is a new id.
|
||||
func (c *Client) Ask(a asks.Ask) error {
|
||||
if err := a.Check(c.now()); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := c.State.Create(key(a.ID), Kept{Ask: a, Asked: c.now()}); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return fmt.Errorf("an ask %s was already made: a new question is a new id", a.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return c.publish(asks.AskSubject(c.Module), a, a.ID)
|
||||
}
|
||||
|
||||
// Cancel takes an ask back: the router ends it and edits every copy.
|
||||
func (c *Client) Cancel(id string) error {
|
||||
return c.publish(asks.CancelSubject(c.Module), map[string]string{"id": id}, "cancel."+id)
|
||||
}
|
||||
|
||||
func (c *Client) publish(subject string, body any, id string) error {
|
||||
if c.Publish != nil {
|
||||
return c.Publish(subject, body, id)
|
||||
}
|
||||
return stdio.SeatPublish(subject, body, id)
|
||||
}
|
||||
|
||||
// Where is an ask's state as the router keeps it: its state and its warrant, if any — read under the
|
||||
// module's own name and no other's.
|
||||
func (c *Client) Where(id string) (state string, w *asks.Warrant, err error) {
|
||||
read := c.Record
|
||||
if read == nil {
|
||||
read = func(k string) (*stdio.StateEntry, error) { return stdio.SeatRecord("", k) }
|
||||
}
|
||||
e, err := read(id)
|
||||
if err != nil || e == nil {
|
||||
return "", nil, err
|
||||
}
|
||||
var rec struct {
|
||||
State string `json:"state"`
|
||||
Warrant *asks.Warrant `json:"warrant"`
|
||||
}
|
||||
if err := json.Unmarshal(e.Value, &rec); err != nil {
|
||||
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
|
||||
}
|
||||
return rec.State, rec.Warrant, nil
|
||||
}
|
||||
|
||||
// Handled is what came of a warrant: acted on, refused (and why), or already done.
|
||||
type Handled string
|
||||
|
||||
const (
|
||||
Acted Handled = "acted"
|
||||
Refused Handled = "refused"
|
||||
Done Handled = "already acted on"
|
||||
Recorded Handled = "ended without a choice"
|
||||
)
|
||||
|
||||
// Take hands act a warrant once, after holding it to the ask it answers. act performs the option with the
|
||||
// module's own grants; its error is said and the warrant stays acted on (a person's decision is not
|
||||
// performed twice; the module says the failure in its own record).
|
||||
func (c *Client) Take(w asks.Warrant, act func(asks.Option, asks.Warrant) error) (Handled, error) {
|
||||
if w.Asker != c.Module {
|
||||
return Refused, fmt.Errorf("a warrant for %s reached %s", w.Asker, c.Module)
|
||||
}
|
||||
e, err := c.State.Get(key(w.Ask))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if e == nil {
|
||||
return Refused, fmt.Errorf("%s holds no ask %s", c.Module, w.Ask)
|
||||
}
|
||||
var k Kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if k.Acted || k.Ended != "" {
|
||||
return Done, nil
|
||||
}
|
||||
if w.Outcome != asks.OutcomeChosen {
|
||||
k.Ended = w.Outcome
|
||||
k.Answer = &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil && !errors.Is(err, stdio.ErrChanged) {
|
||||
return "", err
|
||||
}
|
||||
return Recorded, nil
|
||||
}
|
||||
o, err := w.For(c.Module, k.Ask)
|
||||
if err != nil {
|
||||
return Refused, err
|
||||
}
|
||||
// Marked before acting: of two deliveries, or two instances, only the one whose write stands acts.
|
||||
k.Acted, k.Ended, k.Answer = true, asks.OutcomeChosen, &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return Done, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return Acted, act(o, w)
|
||||
}
|
||||
|
||||
// Listen hands every warrant for this module to Take, with act. It asks the runtime for the module's
|
||||
// events matching the router's `decided` subject for this module.
|
||||
func (c *Client) Listen(act func(asks.Option, asks.Warrant) error, said func(Handled, asks.Warrant, error)) error {
|
||||
subscribe := c.Subscribe
|
||||
if subscribe == nil {
|
||||
subscribe = stdio.Subscribe
|
||||
}
|
||||
return subscribe(asks.Seat+".decided."+c.Module, func(e stdio.Envelope) error {
|
||||
if !strings.HasPrefix(e.Key, asks.Seat+".decided.") {
|
||||
return nil
|
||||
}
|
||||
var w asks.Warrant
|
||||
if err := json.Unmarshal(e.Body, &w); err != nil {
|
||||
if said != nil {
|
||||
said(Refused, w, fmt.Errorf("a warrant could not be read: %w", err))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
h, err := c.Take(w, act)
|
||||
if h == "" && err != nil {
|
||||
return err // the state could not be reached: offered again
|
||||
}
|
||||
if said != nil {
|
||||
said(h, w, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
type mem struct {
|
||||
vals map[string]json.RawMessage
|
||||
revs map[string]uint64
|
||||
n uint64
|
||||
}
|
||||
|
||||
func (m *mem) Get(k string) (*stdio.StateEntry, error) {
|
||||
v, ok := m.vals[k]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: k, Value: v, Revision: m.revs[k]}, nil
|
||||
}
|
||||
func (m *mem) Create(k string, v any) (uint64, error) {
|
||||
if _, ok := m.vals[k]; ok {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) Update(k string, v any, rev uint64) (uint64, error) {
|
||||
if rev == 0 || m.revs[k] != rev {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) put(k string, v any) (uint64, error) {
|
||||
raw, _ := json.Marshal(v)
|
||||
m.n++
|
||||
m.vals[k], m.revs[k] = raw, m.n
|
||||
return m.n, nil
|
||||
}
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func client() (*Client, map[string][]string) {
|
||||
sent := map[string][]string{}
|
||||
c := &Client{Module: "mesh-delivery", State: &mem{vals: map[string]json.RawMessage{}, revs: map[string]uint64{}},
|
||||
Now: func() time.Time { return now },
|
||||
Publish: func(subject string, body any, id string) error {
|
||||
sent[subject] = append(sent[subject], id)
|
||||
return nil
|
||||
}}
|
||||
return c, sent
|
||||
}
|
||||
|
||||
// releaseHQ is the act the ask's one option binds.
|
||||
var releaseHQ = map[string]string{"verb": "release", "delivery": "hq"}
|
||||
|
||||
func held() asks.Ask {
|
||||
binds, _ := asks.ActDigest(releaseHQ)
|
||||
return asks.Ask{ID: "d-1", Headline: "Delivery of hq held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: asks.Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held",
|
||||
Options: []asks.Option{{ID: "release", Label: "Release", Does: "it goes on", Level: asks.Approve, Binds: binds}}}
|
||||
}
|
||||
|
||||
func warrant() asks.Warrant {
|
||||
return asks.Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: asks.OutcomeChosen, Option: "release",
|
||||
Label: "Release", Level: asks.Approve, At: now.Add(time.Minute), AskDigest: held().Digest(),
|
||||
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
|
||||
}
|
||||
|
||||
func TestAnAskIsKeptThenPublishedUnderTheModulesOwnName(t *testing.T) {
|
||||
c, sent := client()
|
||||
if err := c.Ask(held()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := sent[asks.AskSubject("mesh-delivery")]; len(got) != 1 || got[0] != "d-1" {
|
||||
t.Fatalf("%v", sent)
|
||||
}
|
||||
if err := c.Ask(held()); err == nil || !strings.Contains(err.Error(), "already made") {
|
||||
t.Errorf("the same id was asked twice: %v", err)
|
||||
}
|
||||
bad := held()
|
||||
bad.Who = "agent"
|
||||
if err := c.Ask(bad); err == nil {
|
||||
t.Error("an ask the contract refuses was published")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWarrantIsActedOnOnceAndOnlyForItsOwnAsk(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
acted := 0
|
||||
act := func(o asks.Option, _ asks.Warrant) error {
|
||||
if err := o.Performs(releaseHQ); err != nil {
|
||||
t.Errorf("the bound act: %v", err)
|
||||
}
|
||||
acted++
|
||||
return nil
|
||||
}
|
||||
if h, err := c.Take(warrant(), act); h != Acted || err != nil || acted != 1 {
|
||||
t.Fatalf("%s %v %d", h, err, acted)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), act); h != Done || acted != 1 {
|
||||
t.Errorf("a redelivery was acted on: %s", h)
|
||||
}
|
||||
for why, change := range map[string]func(*asks.Warrant){
|
||||
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-controller" },
|
||||
"an ask not made": func(w *asks.Warrant) { w.Ask = "d-9" },
|
||||
"another level": func(w *asks.Warrant) { w.Level = asks.Acknowledge },
|
||||
"an option not offered": func(w *asks.Warrant) { w.Option = "delete" },
|
||||
"after its expiry": func(w *asks.Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *asks.Warrant) { w.AskDigest = "sha256:00" },
|
||||
} {
|
||||
c2, _ := client()
|
||||
_ = c2.Ask(held())
|
||||
w := warrant()
|
||||
change(&w)
|
||||
if h, _ := c2.Take(w, act); h == Acted {
|
||||
t.Errorf("a warrant from %s was acted on", why)
|
||||
}
|
||||
}
|
||||
if acted != 1 {
|
||||
t.Errorf("acted %d times", acted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskEndedWithoutAChoiceIsRecordedAndNothingDone(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
w := warrant()
|
||||
w.Outcome, w.Option, w.By = asks.OutcomeExpired, "", nil
|
||||
if h, err := c.Take(w, func(asks.Option, asks.Warrant) error { t.Fatal("acted"); return nil }); h != Recorded || err != nil {
|
||||
t.Fatalf("%s %v", h, err)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), func(asks.Option, asks.Warrant) error { t.Fatal("acted after it ended"); return nil }); h != Done {
|
||||
t.Errorf("%s", h)
|
||||
}
|
||||
}
|
||||
+573
@@ -0,0 +1,573 @@
|
||||
// Package asks is the contract of asking a person and answering on a channel (novox/hq ADR 0259): the
|
||||
// shapes an asker, the router and a channel exchange on the bus, and the subjects they travel on. No
|
||||
// transport and no channel's service: an asker publishes an Ask under its own name and acts on the Warrant
|
||||
// it hears; the router holds the ask, sends channels a Message, and judges the Choice a channel says; a
|
||||
// channel shows a Message and says what was chosen and by whom, as its service authenticated it.
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The seats (novox/hq ADR 0259 §3).
|
||||
const (
|
||||
// Seat is held by the router: an ask and its cancel are its accepts, a warrant its event, each named by
|
||||
// the asker.
|
||||
Seat = "operator-channel"
|
||||
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
|
||||
ChannelSeat = "channel"
|
||||
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
|
||||
// the kind.
|
||||
IntakeSeat = "intake"
|
||||
)
|
||||
|
||||
// AskSubject is where an asker publishes an ask, CancelSubject its cancel, and DecidedSubject where it
|
||||
// hears the warrant, or the ask's end without one.
|
||||
func AskSubject(asker string) string { return "mesh.seat." + Seat + ".accept.ask." + asker }
|
||||
func CancelSubject(asker string) string { return "mesh.seat." + Seat + ".accept.cancel." + asker }
|
||||
func DecidedSubject(asker string) string { return "mesh.seat." + Seat + ".event.decided." + asker }
|
||||
|
||||
// The work a channel takes, on ChannelSubject.
|
||||
const (
|
||||
Show = "show" // a message offering answers
|
||||
Edit = "edit" // a message shown before, replaced
|
||||
Send = "send" // a message offering nothing
|
||||
)
|
||||
|
||||
// ChannelSubject is where the router sends a channel of a kind its work.
|
||||
func ChannelSubject(verb, kind string) string {
|
||||
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
|
||||
}
|
||||
|
||||
// What a channel says, on IntakeSubject.
|
||||
const (
|
||||
Chosen = "choice" // a button tapped
|
||||
Link = "link" // somebody asked to be linked as the operator
|
||||
)
|
||||
|
||||
// IntakeSubject is where a channel of a kind says what arrived.
|
||||
func IntakeSubject(what, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
|
||||
}
|
||||
|
||||
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
|
||||
const CodeProof = "code"
|
||||
|
||||
// ProofSubject is where a channel of a kind asks a proof.
|
||||
func ProofSubject(verb, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
|
||||
}
|
||||
|
||||
// A Level is how much proof an option's answer needs (novox/hq ADR 0234 §8, the glossary's assurance level).
|
||||
type Level string
|
||||
|
||||
const (
|
||||
// Acknowledge performs only what any granted principal may already do: silencing, details. No proof.
|
||||
Acknowledge Level = "acknowledge"
|
||||
// Approve needs one proof: a verified sender (a linked account, linked an hour or more) or a code.
|
||||
Approve Level = "approve"
|
||||
// Destroy needs two proofs, one of them a code.
|
||||
Destroy Level = "destroy"
|
||||
)
|
||||
|
||||
// Rank orders the levels; an unknown level ranks above every known one, so it is never taken as less.
|
||||
func (l Level) Rank() int {
|
||||
switch l {
|
||||
case Acknowledge:
|
||||
return 0
|
||||
case Approve:
|
||||
return 1
|
||||
case Destroy:
|
||||
return 2
|
||||
}
|
||||
return 3
|
||||
}
|
||||
|
||||
// Operator is the one role an ask may be answered by today.
|
||||
const Operator = "operator"
|
||||
|
||||
// Option is one answer an ask offers: a label for the button, what it does in plain words, its level.
|
||||
type Option struct {
|
||||
ID string `json:"id"`
|
||||
Label string `json:"label"`
|
||||
Does string `json:"does"`
|
||||
Level Level `json:"level"`
|
||||
// Binds is the digest of exactly what the asker performs when this option is chosen — the verb, the
|
||||
// machine and every argument — as ActDigest gives it. It travels in the ask, so the router's warrant,
|
||||
// which names the ask's digest, names it too: a warrant then authorises that act and no other, and an
|
||||
// asker whose record of the act changed after it asked finds the digests differ and does nothing.
|
||||
// Required on an option above acknowledge.
|
||||
Binds string `json:"binds,omitempty"`
|
||||
}
|
||||
|
||||
// An Act is what an option binds: named fields, each a string — the verb, the machine, the level, and each
|
||||
// argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over these names and
|
||||
// values alone, never over how a language happens to encode a struct.
|
||||
type Act map[string]string
|
||||
|
||||
// ActDigest is the digest an asker puts in Option.Binds: SHA-256 over the act's canonical encoding (canonical),
|
||||
// written "sha256:<hex>". The same names and values give the same digest in any language, whatever order
|
||||
// they were set in; a field renamed, added or emptied gives another.
|
||||
func ActDigest(act Act) (string, error) {
|
||||
if len(act) == 0 {
|
||||
return "", fmt.Errorf("the act cannot be digested: it names nothing")
|
||||
}
|
||||
keys := make([]string, 0, len(act))
|
||||
for k := range act {
|
||||
if k == "" {
|
||||
return "", fmt.Errorf("the act cannot be digested: a field has no name")
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.act.v1\n")
|
||||
for _, k := range keys {
|
||||
canonical(&b, k)
|
||||
canonical(&b, act[k])
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read
|
||||
// as another's end or start, so two different sequences of values never encode the same.
|
||||
func canonical(b *strings.Builder, v string) {
|
||||
b.WriteString(strconv.Itoa(len(v)))
|
||||
b.WriteByte(':')
|
||||
b.WriteString(v)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
|
||||
// Digest is the digest of the ask exactly as its asker published it: its id, words, options with what each
|
||||
// binds, who answers, and its expiry. The router puts it in the warrant (Warrant.AskDigest), and an asker
|
||||
// acts only on a warrant whose digest is that of the ask it keeps — so a warrant answers one ask, as the
|
||||
// person was shown it, and nothing published under the same id before or after.
|
||||
//
|
||||
// It is over the ask's named fields in a fixed order, each written canonically, and the expiry as UTC
|
||||
// RFC 3339 to the nanosecond — never over a language's encoding of the struct, so a field added to Ask
|
||||
// later changes no digest until it is added here, on purpose. Whole and Details (novox/hq issue 383) follow
|
||||
// the options only when the ask gives either: an ask without them digests as it did before they existed, so
|
||||
// a router and an asker of different builds still agree on every such ask.
|
||||
func (a Ask) Digest() string {
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.ask.v1\n")
|
||||
for _, v := range []string{a.ID, a.Headline, a.Explanation, a.Who,
|
||||
a.Expires.UTC().Format(time.RFC3339Nano), a.OnExpiry, a.About, strconv.FormatBool(a.Urgent),
|
||||
strconv.Itoa(len(a.Options))} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
for _, o := range a.Options {
|
||||
for _, v := range []string{o.ID, o.Label, o.Does, string(o.Level), o.Binds} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
}
|
||||
// Whole and Details are written whenever either or WholeDetails is given, and WholeDetails after them whenever
|
||||
// it is given (novox/hq ADR 0347): the fields after the options are then always one of three fixed shapes —
|
||||
// none, the two, or the three — and the count of values the sequence holds tells them apart, so no two asks
|
||||
// encode the same (the review of mesh-sdk #25). An ask with none of them digests as before they existed, and
|
||||
// one without WholeDetails as before it existed.
|
||||
if a.Whole != "" || a.Details != "" || a.WholeDetails != "" {
|
||||
canonical(&b, a.Whole)
|
||||
canonical(&b, a.Details)
|
||||
}
|
||||
if a.WholeDetails != "" {
|
||||
canonical(&b, a.WholeDetails)
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// Ask is a request for a person's word (novox/hq ADR 0259 §4).
|
||||
type Ask struct {
|
||||
// ID is the asker's own, unique to it.
|
||||
ID string `json:"id"`
|
||||
// Headline names the thing and what is wrong, in a few plain words; Explanation is what happened and
|
||||
// what it means. Both are held to the plain rule and the content rule by the router.
|
||||
Headline string `json:"headline"`
|
||||
Explanation string `json:"explanation"`
|
||||
Options []Option `json:"options"`
|
||||
// Who may answer: Operator.
|
||||
Who string `json:"who"`
|
||||
// Expires is when the ask ends unanswered; OnExpiry is what the asker then does, in words the person
|
||||
// is shown ("the delivery stays held"). An ask that authorises never defaults.
|
||||
Expires time.Time `json:"expires"`
|
||||
OnExpiry string `json:"on-expiry"`
|
||||
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
|
||||
About string `json:"about,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
// Whole is the explanation with its exact values whole, shown in place of Explanation on a channel kind
|
||||
// that proves who answers (verified-sender) and carries the ask's answers (novox/hq issue 383): what the
|
||||
// person approves — a mount point, a share, a private address — must be readable where they approve it.
|
||||
// The asker withholds a value shaped like a secret in it, and the router refuses the ask when one is left;
|
||||
// Explanation stays under the whole content rule everywhere else. Empty, Explanation is shown everywhere.
|
||||
Whole string `json:"whole,omitempty"`
|
||||
// Details are the ask's secondary words, line by line under the content rule: who proposed it, a
|
||||
// fingerprint, how to read it whole. The router shows them in the note of the ask's own message
|
||||
// (Message.Note), after what the ask's options do and when it expires — never in its body (novox/hq
|
||||
// issue 639).
|
||||
Details string `json:"details,omitempty"`
|
||||
// WholeDetails are the Details with their exact values whole, shown in place of Details where Whole is shown
|
||||
// in place of the explanation (a kind that proves who answers, given the ask's answers): every value an
|
||||
// approval sets, whole, so the person reads exactly what they approve (novox/hq issue 383). Held, like
|
||||
// Whole, to the secret rule alone; the router refuses the ask when a secret's shape is left in it. Empty,
|
||||
// Details are shown everywhere.
|
||||
WholeDetails string `json:"whole-details,omitempty"`
|
||||
}
|
||||
|
||||
// What an ask's explanation is held to, so it reads at a glance on a phone (novox/hq issue 639): a summary
|
||||
// of two or three short sentences on its first line — what changes, why, what it costs or risks — then at
|
||||
// most three points, a line each, only where they add something. Whole is held to the same. Who asked, when,
|
||||
// the fingerprint and how to read it whole are Details, never the explanation.
|
||||
const (
|
||||
SummaryLength = 300
|
||||
MostPoints = 3
|
||||
PointLength = 160
|
||||
)
|
||||
|
||||
// Glance says why an explanation does not read at a glance, or nil: its first line, the summary, longer than
|
||||
// SummaryLength; more than MostPoints lines after it; a point longer than PointLength. Blank lines are not
|
||||
// counted. An asker holds its asks to it in its own tests; the router says, without refusing, an ask that
|
||||
// does not hold to it.
|
||||
func Glance(explanation string) error {
|
||||
var lines []string
|
||||
for _, l := range strings.Split(explanation, "\n") {
|
||||
if strings.TrimSpace(l) != "" {
|
||||
lines = append(lines, strings.TrimSpace(l))
|
||||
}
|
||||
}
|
||||
var problems []string
|
||||
switch {
|
||||
case len(lines) == 0:
|
||||
problems = append(problems, "it has no summary")
|
||||
case len([]rune(lines[0])) > SummaryLength:
|
||||
problems = append(problems, fmt.Sprintf("its summary is %d characters, more than %d", len([]rune(lines[0])), SummaryLength))
|
||||
}
|
||||
if len(lines) > 1+MostPoints {
|
||||
problems = append(problems, fmt.Sprintf("it has %d points after its summary, more than %d", len(lines)-1, MostPoints))
|
||||
}
|
||||
for i, l := range lines {
|
||||
if i > 0 && len([]rune(l)) > PointLength {
|
||||
problems = append(problems, fmt.Sprintf("its point %d is %d characters, more than %d", i, len([]rune(l)), PointLength))
|
||||
}
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the explanation does not read at a glance: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
|
||||
const (
|
||||
MostOptions = 4
|
||||
MostOpen = 3
|
||||
ApproveLasts = 24 * time.Hour
|
||||
DestroyLasts = 10 * time.Minute
|
||||
HeadlineLength = 60
|
||||
LabelLength = 24
|
||||
)
|
||||
|
||||
var usableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`)
|
||||
|
||||
// UsableID says whether a name can be an ask's or an option's id: a key and a subject token both.
|
||||
func UsableID(id string) bool { return usableID.MatchString(id) }
|
||||
|
||||
// Highest is the highest level among the ask's options.
|
||||
func (a Ask) Highest() Level {
|
||||
high := Acknowledge
|
||||
for _, o := range a.Options {
|
||||
if o.Level.Rank() > high.Rank() {
|
||||
high = o.Level
|
||||
}
|
||||
}
|
||||
return high
|
||||
}
|
||||
|
||||
// Option is the option of this id, or false.
|
||||
func (a Ask) Option(id string) (Option, bool) {
|
||||
for _, o := range a.Options {
|
||||
if o.ID == id {
|
||||
return o, true
|
||||
}
|
||||
}
|
||||
return Option{}, false
|
||||
}
|
||||
|
||||
// Check is what an ask is held to before anything is shown: every refusal, in words its asker can act on.
|
||||
func (a Ask) Check(now time.Time) error {
|
||||
var problems []string
|
||||
say := func(format string, args ...any) { problems = append(problems, fmt.Sprintf(format, args...)) }
|
||||
if !UsableID(a.ID) {
|
||||
say("its id %q is not letters, digits, - and _, at most 64", a.ID)
|
||||
}
|
||||
if strings.TrimSpace(a.Headline) == "" || len([]rune(a.Headline)) > HeadlineLength {
|
||||
say("its headline is empty or longer than %d characters", HeadlineLength)
|
||||
}
|
||||
if strings.TrimSpace(a.Explanation) == "" {
|
||||
say("it explains nothing")
|
||||
}
|
||||
if a.Who != Operator {
|
||||
say("it is answered by %q, and only the operator answers today", a.Who)
|
||||
}
|
||||
if len(a.Options) == 0 || len(a.Options) > MostOptions {
|
||||
say("it offers %d options, and an ask offers one to %d", len(a.Options), MostOptions)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, o := range a.Options {
|
||||
switch {
|
||||
case !UsableID(o.ID):
|
||||
say("an option's id %q is not letters, digits, - and _", o.ID)
|
||||
case seen[o.ID]:
|
||||
say("the option %s is offered twice", o.ID)
|
||||
}
|
||||
seen[o.ID] = true
|
||||
if strings.TrimSpace(o.Label) == "" || len([]rune(o.Label)) > LabelLength {
|
||||
say("the option %s's label is empty or longer than %d characters", o.ID, LabelLength)
|
||||
}
|
||||
if strings.TrimSpace(o.Does) == "" {
|
||||
say("the option %s does not say what it does", o.ID)
|
||||
}
|
||||
if o.Level.Rank() > Destroy.Rank() {
|
||||
say("the option %s has the level %q, which is none of acknowledge, approve, destroy", o.ID, o.Level)
|
||||
}
|
||||
if o.Level != Acknowledge && !strings.HasPrefix(o.Binds, "sha256:") {
|
||||
say("the option %s authorises and does not bind what it performs (its binds is not an ActDigest)", o.ID)
|
||||
}
|
||||
}
|
||||
if !a.Expires.After(now) {
|
||||
say("it expires before it is asked")
|
||||
}
|
||||
switch a.Highest() {
|
||||
case Approve:
|
||||
if a.Expires.After(now.Add(ApproveLasts)) {
|
||||
say("an ask that approves lasts at most %s", ApproveLasts)
|
||||
}
|
||||
case Destroy:
|
||||
if a.Expires.After(now.Add(DestroyLasts)) {
|
||||
say("an ask that destroys lasts at most %s", DestroyLasts)
|
||||
}
|
||||
}
|
||||
if a.Highest() != Acknowledge && strings.TrimSpace(a.OnExpiry) == "" {
|
||||
say("it does not say what happens when nobody answers, and an ask that authorises never defaults")
|
||||
}
|
||||
if a.About != "" && strings.ContainsAny(a.About, " \n") {
|
||||
say("what it is about is a key, without spaces")
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the ask is refused: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Outcome is how an ask ended.
|
||||
type Outcome string
|
||||
|
||||
const (
|
||||
OutcomeChosen Outcome = "chosen" // a person chose an option: a warrant
|
||||
OutcomeExpired Outcome = "expired" // nobody answered in time
|
||||
OutcomeCancelled Outcome = "cancelled" // its asker took it back
|
||||
OutcomeReplaced Outcome = "replaced" // a newer ask about the same thing replaced it
|
||||
OutcomeRefused Outcome = "refused" // it was never shown: Words says why
|
||||
)
|
||||
|
||||
// Person is who chose, as the router verified them.
|
||||
type Person struct {
|
||||
// Who is the role: Operator.
|
||||
Who string `json:"who"`
|
||||
// Kind is the channel kind they answered on, Identity their account on that service, Display the name
|
||||
// the service shows, and Verified how the router knew it was them.
|
||||
Kind string `json:"kind"`
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Verified string `json:"verified"`
|
||||
}
|
||||
|
||||
// Warrant is the router's record that a person chose one option of one ask, or the ask's end without one
|
||||
// (novox/hq ADR 0259 §6). It carries no secret.
|
||||
type Warrant struct {
|
||||
Ask string `json:"ask"`
|
||||
Asker string `json:"asker"`
|
||||
About string `json:"about,omitempty"`
|
||||
Outcome Outcome `json:"outcome"`
|
||||
// Option, Label and Level are the option chosen; By who chose it, Channel the module it came through,
|
||||
// Proofs which proofs were present (P1, P2, P3).
|
||||
Option string `json:"option,omitempty"`
|
||||
Label string `json:"label,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
By *Person `json:"by,omitempty"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
Proofs []string `json:"proofs,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
// AskDigest is the digest of the ask as the router took it (Ask.Digest): the warrant answers that ask
|
||||
// alone, with the options it bound.
|
||||
AskDigest string `json:"ask-digest,omitempty"`
|
||||
// Words are why an ask ended without a choice, or what refused it.
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
// Says is the warrant in the words an asker records with its act: "the operator, via telegram (user id
|
||||
// verified), chose Release".
|
||||
func (w Warrant) Says() string {
|
||||
if w.Outcome != OutcomeChosen || w.By == nil {
|
||||
return fmt.Sprintf("no person chose: the ask %s %s", w.Ask, w.Outcome)
|
||||
}
|
||||
via := w.By.Kind
|
||||
if w.By.Verified != "" {
|
||||
via += " (" + w.By.Verified + ")"
|
||||
}
|
||||
return fmt.Sprintf("the %s, via %s, chose %s", w.By.Who, via, w.Label)
|
||||
}
|
||||
|
||||
// For checks a warrant against the ask its asker made: the same asker and ask, the same ask's digest (so the
|
||||
// same words, options and binds), a choice, an option the ask offered, at that option's level, before the
|
||||
// ask expired. An asker acts on nothing else, and then performs only what the option's Binds names.
|
||||
func (w Warrant) For(asker string, a Ask) (Option, error) {
|
||||
if w.Asker != asker || w.Ask != a.ID {
|
||||
return Option{}, fmt.Errorf("the warrant is for %s's ask %s, not %s's %s", w.Asker, w.Ask, asker, a.ID)
|
||||
}
|
||||
if w.Outcome != OutcomeChosen || w.By == nil || w.By.Who != a.Who {
|
||||
return Option{}, fmt.Errorf("the ask %s ended %s; no person chose", a.ID, w.Outcome)
|
||||
}
|
||||
if d := a.Digest(); w.AskDigest != d {
|
||||
return Option{}, fmt.Errorf("the warrant answers an ask whose digest is %q, and the ask %s kept here is %s: "+
|
||||
"it was not the ask the person was shown", w.AskDigest, a.ID, d)
|
||||
}
|
||||
o, offered := a.Option(w.Option)
|
||||
if !offered {
|
||||
return Option{}, fmt.Errorf("the ask %s offered no option %s", a.ID, w.Option)
|
||||
}
|
||||
if w.Level != o.Level {
|
||||
return Option{}, fmt.Errorf("the option %s is %s, and the warrant says %s", o.ID, o.Level, w.Level)
|
||||
}
|
||||
// A choice made after the ask expired is no answer to it, whatever the router said. An ask that says no
|
||||
// expiry, or a warrant that says no time, is no answer either: neither can be shown to be in time (the
|
||||
// confirmation review of 2026-10-09).
|
||||
if a.Expires.IsZero() {
|
||||
return Option{}, fmt.Errorf("the ask %s says no expiry, so no answer to it can be in time", a.ID)
|
||||
}
|
||||
if w.At.IsZero() {
|
||||
return Option{}, fmt.Errorf("the warrant for the ask %s says no time it was given, so it cannot be shown to be in time", a.ID)
|
||||
}
|
||||
if w.At.After(a.Expires) {
|
||||
return Option{}, fmt.Errorf("the warrant was given at %s, after the ask %s expired at %s",
|
||||
w.At.UTC().Format(time.RFC3339), a.ID, a.Expires.UTC().Format(time.RFC3339))
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Performs checks that the act an asker is about to perform is the one the chosen option bound when it
|
||||
// asked: the act's digest equals the option's Binds. An acknowledge option that bound nothing passes.
|
||||
func (o Option) Performs(act Act) error {
|
||||
if o.Binds == "" && o.Level == Acknowledge {
|
||||
return nil
|
||||
}
|
||||
d, err := ActDigest(act)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d != o.Binds {
|
||||
return fmt.Errorf("the option %s bound %s, and the act about to be performed is %s: nothing is done", o.ID, o.Binds, d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
|
||||
type Button struct {
|
||||
Label string `json:"label"`
|
||||
Ticket string `json:"ticket"`
|
||||
}
|
||||
|
||||
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
|
||||
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
|
||||
// which of its own messages that is. The words are the router's, shown as given.
|
||||
type Message struct {
|
||||
Handle string `json:"handle"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Buttons []Button `json:"buttons,omitempty"`
|
||||
// Note is the message's secondary words, after its body and its buttons: who asked and when, what each
|
||||
// answer does, when it expires, and the ask's Details. A channel shows it smaller, collapsed or set apart
|
||||
// where it can, and after the body where it cannot (novox/hq issue 639). A channel that bounds a message's
|
||||
// length cuts the body first and the note only when the title and the note alone do not fit, saying each
|
||||
// cut (novox/hq ADR 0347 §5).
|
||||
Note string `json:"note,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
Silent bool `json:"silent,omitempty"`
|
||||
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
|
||||
Reply string `json:"reply,omitempty"`
|
||||
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
|
||||
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
|
||||
To string `json:"to,omitempty"`
|
||||
// Secret says the words carry something shown once (a link's code): the channel shows it and keeps no
|
||||
// copy of it — no state, no history of its own.
|
||||
Secret bool `json:"secret,omitempty"`
|
||||
}
|
||||
|
||||
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
|
||||
// service authenticated it. The router alone judges whether that is the operator.
|
||||
type Sender struct {
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Authenticated bool `json:"authenticated"`
|
||||
}
|
||||
|
||||
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
|
||||
type Failed struct {
|
||||
Handle string `json:"handle"`
|
||||
Why string `json:"why"`
|
||||
Permanent bool `json:"permanent,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
|
||||
// whether it can send now, why not, and whether its edits notify nobody.
|
||||
type Standing struct {
|
||||
Ready bool `json:"ready"`
|
||||
Why string `json:"why,omitempty"`
|
||||
EditsSilently bool `json:"edits-silently,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// What a channel says of its own delivery, on IntakeSubject.
|
||||
const (
|
||||
FailedWhat = "failed"
|
||||
StandingWhat = "standing"
|
||||
)
|
||||
|
||||
// Choice is a button chosen on a channel.
|
||||
type Choice struct {
|
||||
// ID is the channel's own for this arrival, unique, so the router acts on it once.
|
||||
ID string `json:"id"`
|
||||
Ticket string `json:"ticket"`
|
||||
Handle string `json:"handle,omitempty"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// LinkAsked is somebody on a channel asking to be linked as the operator.
|
||||
type LinkAsked struct {
|
||||
ID string `json:"id"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
|
||||
type Code struct {
|
||||
Sender Sender `json:"sender"`
|
||||
Code string `json:"code"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
|
||||
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
|
||||
type ProofAnswer struct {
|
||||
Accepted bool `json:"accepted"`
|
||||
Words string `json:"words"`
|
||||
}
|
||||
@@ -0,0 +1,367 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func held() Ask {
|
||||
return Ask{ID: "d-1", Headline: "openrazer delivery held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held", About: "delivery.openrazer",
|
||||
Options: []Option{
|
||||
{ID: "release", Label: "Release", Does: "the delivery goes on", Level: Approve, Binds: bind("release", "openrazer")},
|
||||
{ID: "stop", Label: "Stop", Does: "the delivery ends", Level: Approve, Binds: bind("stop", "openrazer")},
|
||||
{ID: "silence", Label: "Silence for a week", Does: "nothing more is said for a week", Level: Acknowledge},
|
||||
}}
|
||||
}
|
||||
|
||||
func bind(verb, what string) string {
|
||||
d, err := ActDigest(Act{"verb": verb, "delivery": what})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func TestASoundAskPasses(t *testing.T) {
|
||||
if err := held().Check(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskIsRefusedInWordsForEachFault(t *testing.T) {
|
||||
for want, change := range map[string]func(*Ask){
|
||||
"is not letters": func(a *Ask) { a.ID = "a.b" },
|
||||
"headline is empty": func(a *Ask) { a.Headline = "" },
|
||||
"explains nothing": func(a *Ask) { a.Explanation = " " },
|
||||
"only the operator answers": func(a *Ask) { a.Who = "agent" },
|
||||
"offers 0 options": func(a *Ask) { a.Options = nil },
|
||||
"offers 5 options": func(a *Ask) { a.Options = append(a.Options, a.Options[0], a.Options[1]) },
|
||||
"is offered twice": func(a *Ask) { a.Options[1].ID = "release" },
|
||||
"does not say what it does": func(a *Ask) { a.Options[0].Does = "" },
|
||||
"none of acknowledge": func(a *Ask) { a.Options[0].Level = "admin" },
|
||||
"expires before it is asked": func(a *Ask) { a.Expires = now },
|
||||
"approves lasts at most": func(a *Ask) { a.Expires = now.Add(25 * time.Hour) },
|
||||
"destroys lasts at most": func(a *Ask) { a.Options[0].Level = Destroy },
|
||||
"never defaults": func(a *Ask) { a.OnExpiry = "" },
|
||||
"is a key, without spaces": func(a *Ask) { a.About = "a b" },
|
||||
"does not bind what it": func(a *Ask) { a.Options[0].Binds = "" },
|
||||
} {
|
||||
a := held()
|
||||
change(&a)
|
||||
if err := a.Check(now); err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("want a refusal saying %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
ack := held()
|
||||
ack.Options = ack.Options[2:]
|
||||
ack.OnExpiry = ""
|
||||
ack.Expires = now.Add(7 * 24 * time.Hour)
|
||||
if err := ack.Check(now); err != nil {
|
||||
t.Errorf("an ask that only acknowledges needs no expiry words and may last a week: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskerActsOnlyOnAWarrantForItsOwnAskAndOption(t *testing.T) {
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
|
||||
AskDigest: held().Digest(), At: now.Add(time.Minute)}
|
||||
if o, err := w.For("mesh-delivery", held()); err != nil || o.ID != "release" {
|
||||
t.Fatalf("a sound warrant was refused: %v", err)
|
||||
}
|
||||
if got := w.Says(); got != "the operator, via telegram (user id verified), chose Release" {
|
||||
t.Errorf("the warrant says %q", got)
|
||||
}
|
||||
for why, change := range map[string]func(*Warrant){
|
||||
"another asker": func(w *Warrant) { w.Asker = "mesh-controller" },
|
||||
"another ask": func(w *Warrant) { w.Ask = "d-2" },
|
||||
"no choice": func(w *Warrant) { w.Outcome = OutcomeExpired },
|
||||
"nobody": func(w *Warrant) { w.By = nil },
|
||||
"not offered": func(w *Warrant) { w.Option = "delete" },
|
||||
"a lower level": func(w *Warrant) { w.Level = Acknowledge },
|
||||
"another role": func(w *Warrant) { w.By = &Person{Who: "agent"} },
|
||||
"no digest": func(w *Warrant) { w.AskDigest = "" },
|
||||
"no time given": func(w *Warrant) { w.At = time.Time{} },
|
||||
"after expiry": func(w *Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *Warrant) {
|
||||
other := held()
|
||||
other.Options[0].Binds = bind("release", "another")
|
||||
w.AskDigest = other.Digest()
|
||||
},
|
||||
} {
|
||||
c := w
|
||||
by := *w.By
|
||||
c.By = &by
|
||||
change(&c)
|
||||
if _, err := c.For("mesh-delivery", held()); err == nil {
|
||||
t.Errorf("a warrant with %s was acted on", why)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSubjectsNameTheirPublisher(t *testing.T) {
|
||||
for got, want := range map[string]string{
|
||||
AskSubject("mesh-delivery"): "mesh.seat.operator-channel.accept.ask.mesh-delivery",
|
||||
DecidedSubject("mesh-delivery"): "mesh.seat.operator-channel.event.decided.mesh-delivery",
|
||||
ChannelSubject(Show, "telegram"): "mesh.seat.channel.accept.show.telegram",
|
||||
IntakeSubject(Chosen, "desktop"): "mesh.seat.intake.event.choice.desktop",
|
||||
ProofSubject(CodeProof, "telegram"): "mesh.seat.intake.proof.code.telegram",
|
||||
} {
|
||||
if got != want {
|
||||
t.Errorf("%s, want %s", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A warrant answers one ask as it was shown, and authorises the act its option bound and no other
|
||||
// (novox/hq ADR 0259 §6: the warrant is bound to the exact ask and its arguments).
|
||||
func TestAWarrantAuthorisesOnlyTheActItsOptionBound(t *testing.T) {
|
||||
a := held()
|
||||
// The digest survives the trip through the bus: what the router decodes and digests is what was sent.
|
||||
raw, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var decoded Ask
|
||||
if err := json.Unmarshal(raw, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Digest() != a.Digest() {
|
||||
t.Fatalf("the ask's digest changed on the way: %s, %s", decoded.Digest(), a.Digest())
|
||||
}
|
||||
local := a
|
||||
local.Expires = a.Expires.In(time.FixedZone("CEST", 2*3600))
|
||||
if local.Digest() != a.Digest() {
|
||||
t.Error("the same instant in another zone digests differently")
|
||||
}
|
||||
// Changing anything the person was shown, or what an option binds, changes the digest.
|
||||
for what, change := range map[string]func(*Ask){
|
||||
"the headline": func(a *Ask) { a.Headline += "!" },
|
||||
"the explanation": func(a *Ask) { a.Explanation += " " },
|
||||
"a label": func(a *Ask) { a.Options[0].Label = "Go" },
|
||||
"what it binds": func(a *Ask) { a.Options[0].Binds = bind("release", "another") },
|
||||
"the expiry": func(a *Ask) { a.Expires = a.Expires.Add(time.Second) },
|
||||
"what it is about": func(a *Ask) { a.About = "delivery.other" },
|
||||
} {
|
||||
c := held()
|
||||
change(&c)
|
||||
if c.Digest() == a.Digest() {
|
||||
t.Errorf("changing %s leaves the digest", what)
|
||||
}
|
||||
}
|
||||
o, _ := a.Option("release")
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "openrazer"}); err != nil {
|
||||
t.Errorf("the bound act was refused: %v", err)
|
||||
}
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "another"}); err == nil {
|
||||
t.Error("an act the option did not bind was let through")
|
||||
}
|
||||
ack, _ := a.Option("silence")
|
||||
if err := ack.Performs(Act{"anything": "x"}); err != nil {
|
||||
t.Errorf("an acknowledgement that bound nothing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The digests are over named fields in a canonical encoding (the review of 2026-10-09): not over Go's
|
||||
// encoding of a struct, so a field added to Ask or Option, a JSON tag renamed, or a map built in another
|
||||
// order changes nothing it should not — and every field that is shown or bound is in it.
|
||||
func TestDigestsAreCanonicalOverNamedFields(t *testing.T) {
|
||||
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
|
||||
b := Act{}
|
||||
b["arg.delivery"] = "hq"
|
||||
b["verb"] = "release"
|
||||
a2, _ := ActDigest(b)
|
||||
if a1 != a2 {
|
||||
t.Error("the order an act was built in changed its digest")
|
||||
}
|
||||
// A value cannot be read as two: "a"+"b=c" is not "a=b"+"c".
|
||||
x, _ := ActDigest(Act{"a": "b\n1:c"})
|
||||
y, _ := ActDigest(Act{"a": "b", "c": ""})
|
||||
if x == y {
|
||||
t.Error("two different acts encode the same")
|
||||
}
|
||||
if _, err := ActDigest(Act{}); err == nil {
|
||||
t.Error("an empty act was digested")
|
||||
}
|
||||
// The encoding, by hand: any language reproduces it.
|
||||
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
|
||||
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Errorf("the act's encoding is not the one written down: %s", got)
|
||||
}
|
||||
|
||||
// Every exported field of Ask and Option is in the digest: changing any one changes it.
|
||||
base := held()
|
||||
mutations := map[string]func(*Ask){
|
||||
"ID": func(a *Ask) { a.ID = "other" },
|
||||
"Headline": func(a *Ask) { a.Headline += "!" },
|
||||
"Explanation": func(a *Ask) { a.Explanation += "!" },
|
||||
"Who": func(a *Ask) { a.Who = "someone" },
|
||||
"Expires": func(a *Ask) { a.Expires = a.Expires.Add(time.Nanosecond) },
|
||||
"OnExpiry": func(a *Ask) { a.OnExpiry += "!" },
|
||||
"About": func(a *Ask) { a.About += "!" },
|
||||
"Urgent": func(a *Ask) { a.Urgent = !a.Urgent },
|
||||
"Options": func(a *Ask) { a.Options = a.Options[:1] },
|
||||
"Option.ID": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].ID = "x" },
|
||||
"Option.Label": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Label += "!"
|
||||
},
|
||||
"Option.Does": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].Does += "!" },
|
||||
"Option.Level": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Level = Destroy
|
||||
},
|
||||
"Option.Binds": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Binds = "sha256:00"
|
||||
},
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
c := base
|
||||
mutate(&c)
|
||||
if c.Digest() == base.Digest() {
|
||||
t.Errorf("changing %s left the digest the same", name)
|
||||
}
|
||||
}
|
||||
// Whole and Details (issue 383) are in the digest once given, and change it; an ask without either digests
|
||||
// exactly as before they existed — the encoding by hand, so no build can drift from it unseen.
|
||||
plain := Ask{ID: "p", Headline: "h", Explanation: "e", Who: Operator, Expires: now, OnExpiry: "o", About: "a",
|
||||
Options: []Option{{ID: "ok", Label: "OK", Does: "d", Level: Acknowledge}}}
|
||||
byHand := sha256.Sum256([]byte("novox.ask.v1\n1:p\n1:h\n1:e\n8:operator\n20:2026-10-08T14:00:00Z\n1:o\n1:a\n5:false\n1:1\n" +
|
||||
"2:ok\n2:OK\n1:d\n11:acknowledge\n0:\n"))
|
||||
if plain.Digest() != "sha256:"+hex.EncodeToString(byHand[:]) {
|
||||
t.Errorf("an ask without Whole and Details does not digest as it did before them: %s", plain.Digest())
|
||||
}
|
||||
withWhole, withDetails := plain, plain
|
||||
withWhole.Whole = "e, whole"
|
||||
withDetails.Details = "fingerprint 1234"
|
||||
if withWhole.Digest() == plain.Digest() || withDetails.Digest() == plain.Digest() || withWhole.Digest() == withDetails.Digest() {
|
||||
t.Error("Whole or Details left the digest the same")
|
||||
}
|
||||
both := withWhole
|
||||
both.Details = "fingerprint 1234"
|
||||
changed := both
|
||||
changed.Whole += "!"
|
||||
if changed.Digest() == both.Digest() {
|
||||
t.Error("changing Whole beside Details left the digest the same")
|
||||
}
|
||||
// A field added to Ask or Option without being added to Digest fails here.
|
||||
if n := reflect.TypeOf(Ask{}).NumField(); n != 12 {
|
||||
t.Errorf("Ask has %d fields, and Digest covers 12: add the new one to Digest and to this test", n)
|
||||
}
|
||||
if n := reflect.TypeOf(Option{}).NumField(); n != 5 {
|
||||
t.Errorf("Option has %d fields, and Digest covers 5: add the new one to Digest and to this test", n)
|
||||
}
|
||||
// The same instant in another zone is the same ask.
|
||||
z := base
|
||||
z.Expires = z.Expires.In(time.FixedZone("x", 3600))
|
||||
if z.Digest() != base.Digest() {
|
||||
t.Error("the expiry's zone changed the digest")
|
||||
}
|
||||
}
|
||||
|
||||
// An ask that says no expiry is answered by no warrant (the confirmation review of 2026-10-09): nothing could
|
||||
// show the answer was in time.
|
||||
func TestAWarrantForAnAskWithNoExpiryIsRefused(t *testing.T) {
|
||||
a := held()
|
||||
a.Expires = time.Time{}
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42"}, AskDigest: a.Digest(),
|
||||
At: now.Add(time.Minute)}
|
||||
if _, err := w.For("mesh-delivery", a); err == nil {
|
||||
t.Error("a warrant for an ask with no expiry was taken")
|
||||
}
|
||||
}
|
||||
|
||||
// An explanation that reads at a glance passes; a wall of text says each way it is one (novox/hq issue 639).
|
||||
func TestAnExplanationReadsAtAGlance(t *testing.T) {
|
||||
if err := Glance("Your agents' memory follows you to every machine. A file changed on two keeps both.\n\n• Only the memory folders are carried\n• A conflict reaches you as a warning"); err != nil {
|
||||
t.Fatalf("a summary and two points: %v", err)
|
||||
}
|
||||
var wall strings.Builder
|
||||
wall.WriteString("ADR 0343 was proposed by fix-367 and approved by the knowledge base's own review.")
|
||||
for i := 1; i <= 13; i++ {
|
||||
wall.WriteString("\n" + strings.Repeat("A clause lead. ", 3))
|
||||
}
|
||||
err := Glance(wall.String())
|
||||
if err == nil || !strings.Contains(err.Error(), "13 points") {
|
||||
t.Fatalf("thirteen clause lines pass as a glance: %v", err)
|
||||
}
|
||||
if err := Glance(strings.Repeat("word ", 70)); err == nil || !strings.Contains(err.Error(), "summary is 349") {
|
||||
t.Fatalf("a long summary passes: %v", err)
|
||||
}
|
||||
if err := Glance("short.\n" + strings.Repeat("p", PointLength+1)); err == nil || !strings.Contains(err.Error(), "point 1") {
|
||||
t.Fatalf("a long point passes: %v", err)
|
||||
}
|
||||
if err := Glance(" \n"); err == nil {
|
||||
t.Fatal("an empty explanation passes")
|
||||
}
|
||||
}
|
||||
|
||||
// A message's note travels beside its body, and is left out of the encoding when empty, so a channel older
|
||||
// than it reads every message it read before.
|
||||
func TestAMessageCarriesItsNoteApart(t *testing.T) {
|
||||
raw, _ := json.Marshal(Message{Handle: "h", Title: "t", Body: "b", Note: "Asked by x."})
|
||||
if !strings.Contains(string(raw), `"note":"Asked by x."`) {
|
||||
t.Fatalf("the note is not encoded: %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(Message{Handle: "h", Title: "t"})
|
||||
if strings.Contains(string(raw), "note") {
|
||||
t.Fatalf("an empty note is encoded: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// WholeDetails is digested when given, and an ask without it digests as before it existed: a router and an asker
|
||||
// of different builds still agree on every such ask.
|
||||
func TestWholeDetailsAreDigestedOnlyWhenGiven(t *testing.T) {
|
||||
a := Ask{ID: "a", Headline: "h", Explanation: "e", Details: "d"}
|
||||
before := a.Digest()
|
||||
a.WholeDetails = "+ shares: media=/storage/media"
|
||||
if a.Digest() == before {
|
||||
t.Fatal("the whole details are not in the digest")
|
||||
}
|
||||
b := a
|
||||
b.WholeDetails = "+ shares: media=/storage/other"
|
||||
if a.Digest() == b.Digest() {
|
||||
t.Fatal("two whole details digest the same")
|
||||
}
|
||||
a.WholeDetails = ""
|
||||
if a.Digest() != before {
|
||||
t.Fatal("an ask without whole details digests otherwise than before")
|
||||
}
|
||||
}
|
||||
|
||||
// No two asks encode the same (the review of mesh-sdk #25): whole details given beside nothing else are not read as
|
||||
// a Whole and Details that happen to spell the same values, nor as any other split of the three fields.
|
||||
func TestWholeDetailsCollideWithNoOtherAsk(t *testing.T) {
|
||||
base := Ask{ID: "a", Headline: "h", Explanation: "e"}
|
||||
shapes := []Ask{}
|
||||
for _, f := range []struct{ whole, details, wholeDetails string }{
|
||||
{"", "", "+ x: 1"},
|
||||
{"whole-details", "+ x: 1", ""},
|
||||
{"", "+ x: 1", ""},
|
||||
{"+ x: 1", "", ""},
|
||||
{"", "", "whole-details"},
|
||||
{"", "+ x: 1", "+ x: 1"},
|
||||
{"+ x: 1", "+ x: 1", ""},
|
||||
{"", "", ""},
|
||||
} {
|
||||
a := base
|
||||
a.Whole, a.Details, a.WholeDetails = f.whole, f.details, f.wholeDetails
|
||||
shapes = append(shapes, a)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
for i, a := range shapes {
|
||||
if j, ok := seen[a.Digest()]; ok {
|
||||
t.Errorf("asks %d and %d digest the same: %+v / %+v", j, i, shapes[j], a)
|
||||
}
|
||||
seen[a.Digest()] = i
|
||||
}
|
||||
}
|
||||
@@ -1,81 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A bus function called before Serve waits for Serve and proceeds (issue 1966): a module that starts a goroutine
|
||||
// before stdio.Serve and calls the bus there is no longer refused. It is refused only when the process never serves.
|
||||
// The test drives the wait itself: serveWait is the test's own timer channel, no row sleeps on a clock.
|
||||
func subBusFunctionsWaitForServe(t *testing.T) {
|
||||
const refusal = "before Serve: a bundle reaches the bus through the runtime that launched it"
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
serve string // "first": before the call; "during": while the call waits; "never"
|
||||
wantOK bool
|
||||
}{
|
||||
{"a call before Serve completes after Serve", "during", true},
|
||||
{"a call whose wait ends before Serve is refused", "never", false},
|
||||
{"a call after Serve is as before", "first", true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
servesAlone(t)
|
||||
expire := make(chan time.Time)
|
||||
oldAfter := serveTimer
|
||||
serveTimer = func() (<-chan time.Time, func()) { return expire, func() {} }
|
||||
t.Cleanup(func() { serveTimer = oldAfter })
|
||||
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
stopped := make(chan struct{})
|
||||
serve := func() {
|
||||
go func() { _ = ServeOn("m", nil, inR, outW); close(stopped) }()
|
||||
}
|
||||
t.Cleanup(func() { _ = inW.Close(); go func() { _, _ = io.Copy(io.Discard, outR) }() })
|
||||
|
||||
type result struct {
|
||||
c *channel
|
||||
err error
|
||||
}
|
||||
done := make(chan result, 1)
|
||||
if tc.serve == "first" {
|
||||
serve()
|
||||
<-Served()
|
||||
c, err := live()
|
||||
done <- result{c, err}
|
||||
} else {
|
||||
go func() { c, err := live(); done <- result{c, err} }()
|
||||
select {
|
||||
case r := <-done:
|
||||
t.Fatalf("live() returned before Serve and before its wait ended: %v", r.err)
|
||||
default:
|
||||
}
|
||||
if tc.serve == "during" {
|
||||
serve()
|
||||
} else {
|
||||
expire <- time.Time{}
|
||||
}
|
||||
}
|
||||
r := <-done
|
||||
if tc.wantOK != (r.err == nil && r.c != nil) {
|
||||
t.Fatalf("live() = %v, %v; want ok=%v", r.c, r.err, tc.wantOK)
|
||||
}
|
||||
if !tc.wantOK && r.err.Error() != refusal {
|
||||
t.Fatalf("refusal words changed: %v", r.err)
|
||||
}
|
||||
if tc.serve != "never" {
|
||||
_ = inW.Close()
|
||||
<-stopped
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestBusFunctionsWaitForServe runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestBusFunctionsWaitForServe(t *testing.T) {
|
||||
t.Run("BusFunctionsWaitForServe", subBusFunctionsWaitForServe)
|
||||
t.Run("ACapabilityWordIsReadOnceAndNamedWhenNotStated", subACapabilityWordIsReadOnceAndNamedWhenNotStated)
|
||||
t.Run("ABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages", subABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages)
|
||||
}
|
||||
@@ -1,162 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
// Archive is a snapshot read back: its manifest, and each file beside it unpacked into a scratch
|
||||
// directory of its own, removed by Close.
|
||||
type Archive struct {
|
||||
Manifest Manifest
|
||||
dir string
|
||||
files map[string]string
|
||||
}
|
||||
|
||||
// ReadArchive unpacks a snapshot. Only the files the manifest names are kept, and only under names
|
||||
// that are one stream's directory: an archive is data, and a path in it is never trusted to say where
|
||||
// on the machine a file goes.
|
||||
func ReadArchive(r io.Reader, scratch string) (*Archive, error) {
|
||||
dir, err := os.MkdirTemp(scratch, "mesh-nats-restore-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a := &Archive{dir: dir, files: map[string]string{}}
|
||||
tr := tar.NewReader(r)
|
||||
first := true
|
||||
for {
|
||||
hdr, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive does not read: %w", err)
|
||||
}
|
||||
if first {
|
||||
if hdr.Name != ManifestName {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive starts with %q, not its manifest: it is not a snapshot of the bus", hdr.Name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &a.Manifest); err != nil {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("its manifest: %w", err)
|
||||
}
|
||||
if a.Manifest.Format != 1 {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("its manifest is format %d; this program reads format 1", a.Manifest.Format)
|
||||
}
|
||||
first = false
|
||||
continue
|
||||
}
|
||||
if !a.named(hdr.Name) {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the archive holds %q, which its manifest does not name", hdr.Name)
|
||||
}
|
||||
local := filepath.Join(dir, fmt.Sprintf("%04d", len(a.files)))
|
||||
f, err := os.OpenFile(local, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
_, err = io.Copy(f, tr)
|
||||
f.Close()
|
||||
if err != nil {
|
||||
a.Close()
|
||||
return nil, err
|
||||
}
|
||||
a.files[hdr.Name] = local
|
||||
}
|
||||
if first {
|
||||
a.Close()
|
||||
return nil, errors.New("the archive is empty")
|
||||
}
|
||||
for _, s := range a.Manifest.Streams {
|
||||
if !ValidName(s.Name) || s.Meta != "streams/"+s.Name+"/backup.json" || s.Archive != "streams/"+s.Name+"/stream.tar.s2" {
|
||||
a.Close()
|
||||
return nil, fmt.Errorf("the manifest names a stream %q at paths that are not its own", s.Name)
|
||||
}
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
func (a *Archive) named(name string) bool {
|
||||
for _, s := range a.Manifest.Streams {
|
||||
if name == s.Meta || name == s.Archive {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Close removes the unpacked files.
|
||||
func (a *Archive) Close() { os.RemoveAll(a.dir) }
|
||||
|
||||
// Verify holds every file to the manifest's size and checksum, and reads every stream's archive to
|
||||
// its end.
|
||||
func (a *Archive) Verify() error {
|
||||
for _, s := range a.Manifest.Streams {
|
||||
for _, f := range []struct{ name, sum string }{{s.Meta, s.MetaSHA256}, {s.Archive, s.ArchiveSHA256}} {
|
||||
local, ok := a.files[f.name]
|
||||
if !ok {
|
||||
return fmt.Errorf("%s is named in the manifest and missing from the archive", f.name)
|
||||
}
|
||||
got, size, err := sha256Of(local)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if got != f.sum {
|
||||
return fmt.Errorf("%s does not match the checksum its manifest gives", f.name)
|
||||
}
|
||||
if f.name == s.Archive && size != s.ArchiveBytes {
|
||||
return fmt.Errorf("%s is %d bytes; its manifest says %d", f.name, size, s.ArchiveBytes)
|
||||
}
|
||||
}
|
||||
af, err := os.Open(a.files[s.Archive])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
err = ReadsThrough(af)
|
||||
af.Close()
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s does not read: %w", s.Archive, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func sha256Of(path string) (string, int64, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
defer f.Close()
|
||||
h := sha256.New()
|
||||
n, err := io.Copy(h, f)
|
||||
if err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
return hex.EncodeToString(h.Sum(nil)), n, nil
|
||||
}
|
||||
|
||||
// Open is one stream's meta and archive.
|
||||
func (a *Archive) Open(s StreamEntry) (meta []byte, archive *os.File, err error) {
|
||||
meta, err = os.ReadFile(a.files[s.Meta])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
archive, err = os.Open(a.files[s.Archive])
|
||||
return meta, archive, err
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
// Package busarchive is the bus's snapshot archive: its manifest, reading one back and holding it to
|
||||
// its checksums. The snapshot program (cmd/mesh-nats-snapshot) writes and restores with it and the bus's
|
||||
// tool nats_snapshot_verify (cmd/nats-tools) checks with it, so the two never differ on what a good
|
||||
// snapshot is.
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"io"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/klauspost/compress/s2"
|
||||
)
|
||||
|
||||
// Manifest is the first entry of an archive: what was taken, when, how long it took, and the checksum
|
||||
// of every file beside it. It is what `verify` and `restore` hold the archive to.
|
||||
type Manifest struct {
|
||||
Format int `json:"format"`
|
||||
Server string `json:"server_version"`
|
||||
Started time.Time `json:"started"`
|
||||
Finished time.Time `json:"finished"`
|
||||
Seconds float64 `json:"seconds"`
|
||||
// Messages and StreamBytes are the streams' own totals at their snapshots; ArchiveBytes what the
|
||||
// archives take, compressed.
|
||||
Messages uint64 `json:"messages"`
|
||||
StreamBytes int64 `json:"stream_bytes"`
|
||||
ArchiveBytes int64 `json:"archive_bytes"`
|
||||
Streams []StreamEntry `json:"streams"`
|
||||
// Skipped are streams that hold nothing across a restart (memory storage), said rather than
|
||||
// silently left out.
|
||||
Skipped []Skipped `json:"skipped,omitempty"`
|
||||
}
|
||||
|
||||
// StreamEntry is one stream as it was snapshotted.
|
||||
type StreamEntry struct {
|
||||
Name string `json:"name"`
|
||||
Subjects []string `json:"subjects,omitempty"`
|
||||
Retention string `json:"retention,omitempty"`
|
||||
Messages uint64 `json:"messages"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
FirstSeq uint64 `json:"first_seq"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
Deleted int `json:"deleted,omitempty"`
|
||||
Consumers int `json:"consumers"`
|
||||
// Meta is the server's own description of the stream (its configuration and state), in the
|
||||
// shape the restore API and the nats CLI take back: backup.json.
|
||||
Meta string `json:"meta"`
|
||||
MetaSHA256 string `json:"meta_sha256"`
|
||||
// Archive is the server's snapshot of the stream, as the server sent it: stream.tar.s2.
|
||||
Archive string `json:"archive"`
|
||||
ArchiveBytes int64 `json:"archive_bytes"`
|
||||
ArchiveSHA256 string `json:"archive_sha256"`
|
||||
Seconds float64 `json:"seconds"`
|
||||
}
|
||||
|
||||
// Skipped is a stream not snapshotted, and why.
|
||||
type Skipped struct {
|
||||
Name string `json:"name"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
|
||||
// ManifestName is the archive's first entry.
|
||||
const ManifestName = "manifest.json"
|
||||
|
||||
// ReadsThrough is whether a stream's archive decompresses and lists to its end.
|
||||
func ReadsThrough(r io.Reader) error {
|
||||
tr := tar.NewReader(s2.NewReader(r))
|
||||
for {
|
||||
_, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := io.Copy(io.Discard, tr); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ValidName is a stream name that is one safe path component.
|
||||
func ValidName(name string) bool {
|
||||
return name != "" && name != "." && name != ".." && !strings.ContainsAny(name, "/\\\x00 *>")
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// StreamReport is one stream of a verified snapshot.
|
||||
type StreamReport struct {
|
||||
Name string `json:"name"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
}
|
||||
|
||||
// Report is the answer of VerifyFile: which file was held to its manifest, and whether it passed.
|
||||
type Report struct {
|
||||
File string `json:"file"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Size int64 `json:"size"`
|
||||
Taken *time.Time `json:"taken,omitempty"`
|
||||
Streams []StreamReport `json:"streams"`
|
||||
Skipped []Skipped `json:"skipped,omitempty"`
|
||||
Passed bool `json:"passed"`
|
||||
// Fault is the first thing wrong, empty when the snapshot passed.
|
||||
Fault string `json:"fault,omitempty"`
|
||||
}
|
||||
|
||||
// VerifyFile verifies one snapshot inside dir: the file called name, or the newest *.tar when name is
|
||||
// empty. Nothing outside dir is read and nothing is written there; a snapshot that fails is an answer
|
||||
// (Passed false, the first Fault), and only a directory without a snapshot is an error.
|
||||
func VerifyFile(dir, name string) (*Report, error) {
|
||||
path, info, err := pick(dir, name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r := &Report{File: filepath.Base(path), Modified: info.ModTime().UTC(), Size: info.Size(), Streams: []StreamReport{}}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
defer f.Close()
|
||||
a, err := ReadArchive(f, "")
|
||||
if err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
defer a.Close()
|
||||
taken := a.Manifest.Finished
|
||||
r.Taken = &taken
|
||||
r.Skipped = a.Manifest.Skipped
|
||||
for _, s := range a.Manifest.Streams {
|
||||
r.Streams = append(r.Streams, StreamReport{Name: s.Name, LastSeq: s.LastSeq})
|
||||
}
|
||||
if err := a.Verify(); err != nil {
|
||||
r.Fault = err.Error()
|
||||
return r, nil
|
||||
}
|
||||
r.Passed = true
|
||||
return r, nil
|
||||
}
|
||||
|
||||
// pick names the regular file to verify, and nothing outside dir: a name is one path component, and a
|
||||
// symbolic link is not followed out of the directory.
|
||||
func pick(dir, name string) (string, fs.FileInfo, error) {
|
||||
if dir == "" {
|
||||
return "", nil, errors.New("the snapshots directory is not set")
|
||||
}
|
||||
if name != "" {
|
||||
if name != filepath.Base(name) || name == "." || name == ".." || strings.ContainsRune(name, 0) {
|
||||
return "", nil, fmt.Errorf("%q is not a file name inside the snapshots directory", name)
|
||||
}
|
||||
p := filepath.Join(dir, name)
|
||||
info, err := os.Lstat(p)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("no snapshot %s in the snapshots directory: %w", name, err)
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return "", nil, fmt.Errorf("%s is not a regular file", name)
|
||||
}
|
||||
return p, info, nil
|
||||
}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("the snapshots directory does not read: %w", err)
|
||||
}
|
||||
var best string
|
||||
var bestInfo fs.FileInfo
|
||||
for _, e := range entries {
|
||||
if !strings.HasSuffix(e.Name(), ".tar") {
|
||||
continue
|
||||
}
|
||||
info, err := e.Info()
|
||||
if err != nil || !info.Mode().IsRegular() {
|
||||
continue
|
||||
}
|
||||
if bestInfo == nil || info.ModTime().After(bestInfo.ModTime()) {
|
||||
best, bestInfo = e.Name(), info
|
||||
}
|
||||
}
|
||||
if bestInfo == nil {
|
||||
return "", nil, errors.New("the snapshots directory holds no .tar snapshot")
|
||||
}
|
||||
return filepath.Join(dir, best), bestInfo, nil
|
||||
}
|
||||
@@ -1,182 +0,0 @@
|
||||
package busarchive
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/klauspost/compress/s2"
|
||||
)
|
||||
|
||||
func sum(b []byte) string { h := sha256.Sum256(b); return hex.EncodeToString(h[:]) }
|
||||
|
||||
// streamArchive is a stream's archive as the server sends it: a tar of one file, s2-compressed.
|
||||
func streamArchive(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
var raw bytes.Buffer
|
||||
tw := tar.NewWriter(&raw)
|
||||
body := []byte(strings.Repeat("a message of the stream\n", 50))
|
||||
if err := tw.WriteHeader(&tar.Header{Name: "msgs.blk", Mode: 0o600, Size: int64(len(body))}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tw.Write(body)
|
||||
tw.Close()
|
||||
var out bytes.Buffer
|
||||
w := s2.NewWriter(&out)
|
||||
w.Write(raw.Bytes())
|
||||
w.Close()
|
||||
return out.Bytes()
|
||||
}
|
||||
|
||||
// snapshotTar builds a snapshot of two streams; the mutations change what the manifest says or what is
|
||||
// stored, so each fault is made in the one place it is found.
|
||||
func snapshotTar(t *testing.T, mutate func(m *Manifest, files map[string][]byte)) []byte {
|
||||
t.Helper()
|
||||
files := map[string][]byte{}
|
||||
m := Manifest{Format: 1, Server: "2.11.17", Finished: time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)}
|
||||
for i, name := range []string{"EVENTS", "KV_conditions"} {
|
||||
meta := []byte(`{"config":{"name":"` + name + `"}}`)
|
||||
arc := streamArchive(t)
|
||||
e := StreamEntry{Name: name, LastSeq: uint64(10 * (i + 1)),
|
||||
Meta: "streams/" + name + "/backup.json", MetaSHA256: sum(meta),
|
||||
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: int64(len(arc)), ArchiveSHA256: sum(arc)}
|
||||
m.Streams = append(m.Streams, e)
|
||||
files[e.Meta], files[e.Archive] = meta, arc
|
||||
}
|
||||
if mutate != nil {
|
||||
mutate(&m, files)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
tw := tar.NewWriter(&out)
|
||||
put := func(name string, b []byte) {
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(b))}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tw.Write(b)
|
||||
}
|
||||
raw, _ := json.Marshal(m)
|
||||
put(ManifestName, raw)
|
||||
for _, s := range m.Streams {
|
||||
put(s.Meta, files[s.Meta])
|
||||
put(s.Archive, files[s.Archive])
|
||||
}
|
||||
tw.Close()
|
||||
return out.Bytes()
|
||||
}
|
||||
|
||||
func put(t *testing.T, dir, name string, b []byte, at time.Time) {
|
||||
t.Helper()
|
||||
p := filepath.Join(dir, name)
|
||||
if err := os.WriteFile(p, b, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
os.Chtimes(p, at, at)
|
||||
}
|
||||
|
||||
func subAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
good := snapshotTar(t, nil)
|
||||
put(t, dir, "bus.tar", good, time.Now())
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !r.Passed || r.Fault != "" || r.File != "bus.tar" || r.Size != int64(len(good)) {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
if r.Taken == nil || !r.Taken.Equal(time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)) {
|
||||
t.Errorf("taken %v", r.Taken)
|
||||
}
|
||||
if len(r.Streams) != 2 || r.Streams[0] != (StreamReport{"EVENTS", 10}) || r.Streams[1] != (StreamReport{"KV_conditions", 20}) {
|
||||
t.Errorf("streams %+v", r.Streams)
|
||||
}
|
||||
}
|
||||
|
||||
func subACorruptedChecksumFailsNamingTheFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
|
||||
m.Streams[1].ArchiveSHA256 = sum([]byte("something else"))
|
||||
}), time.Now())
|
||||
r, err := VerifyFile(dir, "bus.tar")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.Passed || !strings.Contains(r.Fault, "streams/KV_conditions/stream.tar.s2") || !strings.Contains(r.Fault, "checksum") {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
if len(r.Streams) != 2 {
|
||||
t.Errorf("a failed snapshot still says its streams: %+v", r.Streams)
|
||||
}
|
||||
}
|
||||
|
||||
func subATruncatedSnapshotFails(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
good := snapshotTar(t, nil)
|
||||
put(t, dir, "bus.tar", good[:len(good)*2/3], time.Now())
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.Passed || r.Fault == "" {
|
||||
t.Fatalf("a truncated snapshot passed: %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
|
||||
bad := []byte("not an s2 stream at all")
|
||||
s := &m.Streams[0]
|
||||
files[s.Archive] = bad
|
||||
s.ArchiveSHA256, s.ArchiveBytes = sum(bad), int64(len(bad))
|
||||
}), time.Now())
|
||||
r, _ := VerifyFile(dir, "")
|
||||
if r.Passed || !strings.Contains(r.Fault, "does not read") {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
now := time.Now()
|
||||
put(t, dir, "old.tar", []byte("junk"), now.Add(-48*time.Hour))
|
||||
put(t, dir, "bus.tar", snapshotTar(t, nil), now)
|
||||
put(t, dir, "bus.tar.partial", []byte("half"), now.Add(time.Hour))
|
||||
r, err := VerifyFile(dir, "")
|
||||
if err != nil || r.File != "bus.tar" || !r.Passed {
|
||||
t.Fatalf("%+v %v", r, err)
|
||||
}
|
||||
if r, _ := VerifyFile(dir, "old.tar"); r.Passed || r.File != "old.tar" {
|
||||
t.Errorf("a named file is the one checked: %+v", r)
|
||||
}
|
||||
outside := t.TempDir()
|
||||
put(t, outside, "x.tar", snapshotTar(t, nil), now)
|
||||
for _, name := range []string{"../" + filepath.Base(outside) + "/x.tar", outside + "/x.tar", ".."} {
|
||||
if _, err := VerifyFile(dir, name); err == nil {
|
||||
t.Errorf("%q was read", name)
|
||||
}
|
||||
}
|
||||
os.Symlink(filepath.Join(outside, "x.tar"), filepath.Join(dir, "link.tar"))
|
||||
if _, err := VerifyFile(dir, "link.tar"); err == nil {
|
||||
t.Error("a symbolic link out of the directory was followed")
|
||||
}
|
||||
if _, err := VerifyFile(t.TempDir(), ""); err == nil {
|
||||
t.Error("an empty directory is not a snapshot")
|
||||
}
|
||||
}
|
||||
|
||||
// TestAGoodSnapshotPassesAndSaysItsStreams runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
|
||||
t.Run("AGoodSnapshotPassesAndSaysItsStreams", subAGoodSnapshotPassesAndSaysItsStreams)
|
||||
t.Run("ACorruptedChecksumFailsNamingTheFile", subACorruptedChecksumFailsNamingTheFile)
|
||||
t.Run("ATruncatedSnapshotFails", subATruncatedSnapshotFails)
|
||||
t.Run("AnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums", subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums)
|
||||
t.Run("TheNewestTarIsVerifiedAndOnlyInsideTheDirectory", subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory)
|
||||
}
|
||||
@@ -1,50 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// Unwrap reads a tool's answer whatever wraps it on its way from the tool runner (the lead's ruling 2026-11-04 13:12:
|
||||
// one reading, not a copy per module): a JSON string holding the answer; the protocol's content list, whose first
|
||||
// text is the answer and whose isError says it failed; the {output, ok, answer} form; the tool runner's {result,
|
||||
// error} envelope, whose error is the call's. It answers the answer itself, the words said with it (the output, the
|
||||
// error, or the answer's own text), and whether the tool says it succeeded. A bare answer is itself, succeeded.
|
||||
func Unwrap(raw json.RawMessage) (answer json.RawMessage, said string, ok bool) {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
return Unwrap(json.RawMessage(s))
|
||||
}
|
||||
var m map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &m) != nil {
|
||||
return raw, string(raw), true
|
||||
}
|
||||
if content, has := m["content"]; has {
|
||||
var items []struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
var e struct {
|
||||
IsError bool `json:"isError"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &e)
|
||||
if json.Unmarshal(content, &items) == nil && len(items) > 0 {
|
||||
inner, out, ok := Unwrap(json.RawMessage(items[0].Text))
|
||||
return inner, out, ok && !e.IsError
|
||||
}
|
||||
}
|
||||
var v struct {
|
||||
Output string `json:"output"`
|
||||
OK *bool `json:"ok"`
|
||||
Answer json.RawMessage `json:"answer"`
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if json.Unmarshal(raw, &v) == nil {
|
||||
switch {
|
||||
case v.OK != nil:
|
||||
return v.Answer, v.Output, *v.OK
|
||||
case v.Error != "":
|
||||
return nil, v.Error, false
|
||||
case len(v.Result) > 0:
|
||||
return Unwrap(v.Result)
|
||||
}
|
||||
}
|
||||
return raw, string(raw), true
|
||||
}
|
||||
@@ -1,497 +0,0 @@
|
||||
// Package buscalls is the contract of the calls a bundle makes on its own bus state through the tool runner, over the
|
||||
// bundle's pipe (novox/hq ADR 0595 §9, ADR 0597 §10, design 25 § 12.5): the object store's, a stream's read-back and
|
||||
// fill, a work queue's size and dead letters, and the ledger's. Each call names an entry by the local name its module
|
||||
// asks for it by in `requires`; the tool runner serves it from the module's membership's bindings, its own entries
|
||||
// only. Only the shapes and the method names are here: what the tool runner refuses, and why, is its own.
|
||||
//
|
||||
// It is a package of its own, imported only by the modules that use these calls and by the tool runner, so a change
|
||||
// here rebuilds those and not the module library's importers (ADR 0595 §9). It imports nothing of mesh-sdk.
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the tool runner hands a bundle the id of the tool call in flight: the id
|
||||
// ApplyRequest.Call names, so the runner stamps the caller it proved for that call as the operation's author (novox/hq
|
||||
// ADR 0597 §7). The runner writes `_meta`; a caller writes only `arguments`, so no caller can name another call.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// The methods a bundle asks the tool runner, one per call.
|
||||
const (
|
||||
ObjectsPut = "mesh/objects.put"
|
||||
ObjectsGet = "mesh/objects.get"
|
||||
ObjectsList = "mesh/objects.list"
|
||||
ObjectsDelete = "mesh/objects.delete"
|
||||
ObjectsInfo = "mesh/objects.info"
|
||||
ObjectsWatch = "mesh/objects.watch"
|
||||
// ObjectsSentGet reads an object of a sender's store of sent objects by the reference a reply or an event carried
|
||||
// (design 47 G §347, §407), served under the standing grant the grants part derives from the manifests; the answer is
|
||||
// ObjectData.
|
||||
ObjectsSentGet = "mesh/objects.sent-get"
|
||||
// ObjectsPutChunk and ObjectsGetChunk are the put session's and the get session's calls (design 47 G §518, §345, §490),
|
||||
// under the capability word put-sessions (stdio.CapPutSessions): a value larger than ChunkBytes is put as chunks in order,
|
||||
// each answered before the next, and a get that answers ObjectsChunked is read chunk by chunk.
|
||||
ObjectsPutChunk = "mesh/objects.put-chunk"
|
||||
ObjectsGetChunk = "mesh/objects.get-chunk"
|
||||
|
||||
StreamRead = "mesh/stream.read"
|
||||
StreamFill = "mesh/stream.fill"
|
||||
// StreamPublish says one event on the module's own bound stream (A-42).
|
||||
StreamPublish = "mesh/stream.publish"
|
||||
|
||||
WorkQueueSize = "mesh/workqueue.size"
|
||||
WorkQueueDead = "mesh/workqueue.dead"
|
||||
|
||||
LedgerApply = "mesh/ledger.apply"
|
||||
LedgerGet = "mesh/ledger.get"
|
||||
LedgerHistory = "mesh/ledger.history"
|
||||
LedgerWatch = "mesh/ledger.watch"
|
||||
LedgerQuery = "mesh/ledger.query"
|
||||
// LedgerFill is one ledger's history's fill against its cap, asked with a LedgerRequest and answered as a Fill
|
||||
// (design 47 G §462, §587); LedgerRemove removes one instance, asked with a LedgerRemoveRequest and answered as
|
||||
// LedgerRemoved. Both are under the capability word LedgerUpkeepCalls.
|
||||
LedgerFill = "mesh/ledger.fill"
|
||||
LedgerRemove = "mesh/ledger.remove"
|
||||
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
|
||||
// ledgers only (design 25 § 12.5).
|
||||
LedgerRebuild = "mesh/ledger.rebuild"
|
||||
LedgerReindex = "mesh/ledger.reindex"
|
||||
LedgerSnapshot = "mesh/ledger.snapshot"
|
||||
LedgerLag = "mesh/ledger.lag"
|
||||
)
|
||||
|
||||
// The capability words of the families this package's calls belong to (design 47 G §488, §622): a tool runner states a
|
||||
// word once it answers the family, and a bundle fails at once, naming the word and its node, on a call whose word is not
|
||||
// stated. Which calls each word covers is the tool runner's (§462).
|
||||
const (
|
||||
BusStateCalls = "bus-state-calls"
|
||||
LedgerCalls = "ledger-calls"
|
||||
LedgerUpkeepCalls = "ledger-upkeep-calls"
|
||||
SentObjectsCalls = "sent-objects-calls"
|
||||
// StreamPublishWord is the family of StreamPublish alone, the word a module asks for in `capabilities` before it
|
||||
// says an event on its own bound stream (design 42 zfs-owns-zfs, A-44).
|
||||
StreamPublishWord = "stream-publish"
|
||||
)
|
||||
|
||||
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
|
||||
// does for a log.
|
||||
const (
|
||||
ObjectsWatched = "mesh/objects"
|
||||
LedgerWatched = "mesh/ledger"
|
||||
)
|
||||
|
||||
// Fill is how much an entry holds against its cap: its messages (or objects), its bytes and its cap in bytes.
|
||||
type Fill struct {
|
||||
Count uint64 `json:"count"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
MaxBytes uint64 `json:"max_bytes"`
|
||||
}
|
||||
|
||||
// Done is the answer of a call that answers nothing but that it was done.
|
||||
type Done struct{}
|
||||
|
||||
// --- the object store
|
||||
|
||||
// ObjectsPutRequest stores one object whole under its name, replacing one of that name.
|
||||
type ObjectsPutRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
Data []byte `json:"data"`
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ChunkBytes is the size of one chunk of a put or get session, and so the largest value one call puts or gets (design 47 G
|
||||
// §518, A-21): it fits under the bus's default maximum payload, whatever the bus sets it to.
|
||||
const ChunkBytes = 512 << 10
|
||||
|
||||
// ObjectsPutChunkRequest is one chunk of a put session (design 47 G §518, A-21): the object `name` in the store `objects`,
|
||||
// `call` the bundle's id for this put, `seq` from 1 and one more each chunk, `last` true on the last chunk alone, and the
|
||||
// chunk's bytes; the first chunk also carries the put's `description` (A-25).
|
||||
type ObjectsPutChunkRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
Call string `json:"call"`
|
||||
Seq uint32 `json:"seq"`
|
||||
Last bool `json:"last,omitempty"`
|
||||
Data []byte `json:"data"`
|
||||
// Description is the put's, carried on the first chunk alone (design 47 G A-25).
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsChunkTaken is the answer to a chunk that is not the last: the sequence taken, or the reason it was not.
|
||||
type ObjectsChunkTaken struct {
|
||||
Seq uint32 `json:"seq"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsChunked is the answer of a get of a value over one chunk (design 47 G §490, §518, §620): the count of chunks (or
|
||||
// true, for a sent object), the whole size, its `sha256:` digest and the session's `call`. Its Chunked is absent from the
|
||||
// answer of a value read in one call.
|
||||
type ObjectsChunked struct {
|
||||
Chunked json.RawMessage `json:"chunked"`
|
||||
Size uint64 `json:"size"`
|
||||
Digest string `json:"digest"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
|
||||
// ObjectsGetChunkRequest asks one chunk of a get session by its call and sequence.
|
||||
type ObjectsGetChunkRequest struct {
|
||||
Call string `json:"call"`
|
||||
Seq uint32 `json:"seq"`
|
||||
}
|
||||
|
||||
// ObjectsChunk is a get session's chunk: the sequence, whether it is the last, its bytes, or the reason it is none.
|
||||
type ObjectsChunk struct {
|
||||
Seq uint32 `json:"seq"`
|
||||
Last bool `json:"last"`
|
||||
Data []byte `json:"data"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsNameRequest names one object of an object store: get, delete.
|
||||
type ObjectsNameRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// ObjectsRequest names an object store: list, info.
|
||||
type ObjectsRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsWatchRequest asks the tool runner to send every object stored or deleted, as ObjectsChange under the watch's id.
|
||||
type ObjectsWatchRequest struct {
|
||||
Objects string `json:"objects"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// ObjectInfo is one object as its store holds it: put's answer, and each of list's.
|
||||
type ObjectInfo struct {
|
||||
Name string `json:"name"`
|
||||
Size uint64 `json:"size"`
|
||||
Chunks uint32 `json:"chunks"`
|
||||
Digest string `json:"digest"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Modified time.Time `json:"modified"`
|
||||
Deleted bool `json:"deleted,omitempty"`
|
||||
}
|
||||
|
||||
// ObjectsSentGetRequest names one sent object by the reference that carried it (stateobjects.Ref's fields, which this
|
||||
// package does not import): the sender's store, the object's name, and its size and digest, which the reader checks.
|
||||
type ObjectsSentGetRequest struct {
|
||||
Store string `json:"store"`
|
||||
Object string `json:"object"`
|
||||
Size int64 `json:"size"`
|
||||
Digest string `json:"sha256"`
|
||||
}
|
||||
|
||||
// ObjectData is get's answer: the object's info and its bytes.
|
||||
type ObjectData struct {
|
||||
Info ObjectInfo `json:"info"`
|
||||
Data []byte `json:"data"`
|
||||
}
|
||||
|
||||
// ObjectsListed is list's answer: every object the store holds.
|
||||
type ObjectsListed struct {
|
||||
Objects []ObjectInfo `json:"objects"`
|
||||
}
|
||||
|
||||
// ObjectsChange is one object stored or deleted, sent under ObjectsWatched; Current says it was there when the watch
|
||||
// began.
|
||||
type ObjectsChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Object ObjectInfo `json:"object"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// --- a stream
|
||||
|
||||
// StreamReadRequest reads a module's own stream from a sequence on, at most Limit messages, Subjects narrowing it.
|
||||
type StreamReadRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
From uint64 `json:"from"`
|
||||
Subjects string `json:"subjects,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// StreamRequest names a stream: fill.
|
||||
type StreamRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
}
|
||||
|
||||
// StreamMessage is one message of a stream.
|
||||
type StreamMessage struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// StreamPage is read's answer: the messages in order, the sequence to read on from (0 when nothing follows) and the
|
||||
// stream's last sequence when it was read.
|
||||
type StreamPage struct {
|
||||
Messages []StreamMessage `json:"messages"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// StreamPublishRequest says one event on a module's own stream, by the stream's local name: the tool runner publishes
|
||||
// Body on the binding's subject plus "." plus Event, acknowledged, with ID as the message id (design 42 zfs-owns-zfs, A-42).
|
||||
type StreamPublishRequest struct {
|
||||
Stream string `json:"stream"`
|
||||
Event string `json:"event"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
ID string `json:"id,omitempty"`
|
||||
}
|
||||
|
||||
// StreamPublished is publish's answer: the sequence the stream acknowledged the event at.
|
||||
type StreamPublished struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
|
||||
type StreamFilled struct {
|
||||
Fill
|
||||
First uint64 `json:"first"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// --- a work queue
|
||||
|
||||
// WorkQueueRequest names a work queue: size.
|
||||
type WorkQueueRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadRequest asks a work queue's dead letters, at most Limit of them.
|
||||
type WorkQueueDeadRequest struct {
|
||||
Queue string `json:"queue"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// WorkQueueSized is size's answer: the messages waiting to be taken, against the queue's cap.
|
||||
type WorkQueueSized struct {
|
||||
Fill
|
||||
}
|
||||
|
||||
// DeadLetter is one message delivered as often as its queue allows and never settled.
|
||||
type DeadLetter struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Subject string `json:"subject"`
|
||||
Deliveries int `json:"deliveries"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// WorkQueueDeadLetters is dead's answer.
|
||||
type WorkQueueDeadLetters struct {
|
||||
Dead []DeadLetter `json:"dead"`
|
||||
}
|
||||
|
||||
// --- the ledger (ADR 0597)
|
||||
|
||||
// ApplyRequest applies one operation to one instance of a ledger. Call is the id of the tool call the operation
|
||||
// serves, as the tool runner handed it under MetaCall, whose caller the tool runner proved and stamps as the author
|
||||
// (ADR 0597 §7); empty, the operation carries the owning module's account alone. An author the bundle writes into Body
|
||||
// is never taken as the author.
|
||||
type ApplyRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Call string `json:"call,omitempty"`
|
||||
}
|
||||
|
||||
// Applied is apply's answer: the operation was taken, with its sequence in the instance's history; StateBehind says
|
||||
// the state could not follow because its bucket is full, and catches up at the next apply after room is made (ADR
|
||||
// 0597 §6).
|
||||
type Applied struct {
|
||||
Taken bool `json:"taken"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
StateBehind bool `json:"state_behind,omitempty"`
|
||||
}
|
||||
|
||||
// Author is who an operation is by, as `apply` stamped it (ADR 0597 §7): the owning module's account, and the caller
|
||||
// of the tool call it served, as the tool runner proved it.
|
||||
type Author struct {
|
||||
Account string `json:"account"`
|
||||
Caller string `json:"caller,omitempty"`
|
||||
}
|
||||
|
||||
// InstanceRequest names one instance of a ledger: get, snapshot.
|
||||
type InstanceRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
}
|
||||
|
||||
// InstanceState is get's answer: the instance's state and the sequence of the last operation it holds.
|
||||
type InstanceState struct {
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// HistoryRequest reads an instance's operations from a sequence on, at most Limit of them.
|
||||
type HistoryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
From uint64 `json:"from,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// Operation is one operation as the history keeps it.
|
||||
type Operation struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Instance string `json:"instance"`
|
||||
Operation string `json:"operation"`
|
||||
Body json.RawMessage `json:"body,omitempty"`
|
||||
Author Author `json:"author"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// HistoryPage is history's answer: the operations oldest first, the sequence to read on from (0 when nothing
|
||||
// follows) and the instance's last sequence when it was read.
|
||||
type HistoryPage struct {
|
||||
Operations []Operation `json:"operations"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// LedgerWatchRequest asks the tool runner to send every change of the instances Instances names (an instance, with
|
||||
// `*` for one name and `>` for the rest) as InstanceChange under the watch's id: the current states first.
|
||||
type LedgerWatchRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instances string `json:"instances"`
|
||||
Watch string `json:"watch"`
|
||||
}
|
||||
|
||||
// InstanceChange is one instance's new state, sent under LedgerWatched; Current says it was the state when the watch
|
||||
// began.
|
||||
type InstanceChange struct {
|
||||
Watch string `json:"watch"`
|
||||
Instance string `json:"instance"`
|
||||
State json.RawMessage `json:"state"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// QueryRequest reads a declared index of a ledger: the entries whose key Key names (a key, with `*` for one name and
|
||||
// `>` for the rest; empty is every key), at most Limit of them.
|
||||
type QueryRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// IndexEntry is the value of one key of a declared index (IndexKey): the instance it points at, and the sequence of
|
||||
// the operation it was written from. The owner's watcher and the tool runner's reindex both write it so.
|
||||
type IndexEntry struct {
|
||||
Instance string `json:"instance"`
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
|
||||
// IndexKey is the key of an instance in a declared index: the values of the index's fields, in its order, then the
|
||||
// parts of the instance's path, all joined by ".". Each value and each path part is encoded (keyToken): every byte
|
||||
// outside [A-Za-z0-9_-] is written "=" and two upper-case hex digits ("." is "=2E", " " is "=20", "=" is "=3D"), so two
|
||||
// different keys never meet and every part is a token the bus takes. `17950.listing.a17` is the key of price 17950
|
||||
// and instance `listing/a17`. The owner's watcher and the tool runner's reindex both write it, so a reindex leaves the
|
||||
// index as the watcher keeps it.
|
||||
func IndexKey(fields []string, instance string) string {
|
||||
parts := make([]string, 0, len(fields)+2)
|
||||
for _, f := range fields {
|
||||
parts = append(parts, keyToken(f))
|
||||
}
|
||||
for _, p := range strings.Split(instance, "/") {
|
||||
parts = append(parts, keyToken(p))
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
// keyToken is v with every byte outside [A-Za-z0-9_-] written "=" and two upper-case hex digits (IndexKey).
|
||||
func keyToken(v string) string {
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(v); i++ {
|
||||
c := v[i]
|
||||
if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '_' || c == '-' {
|
||||
b.WriteByte(c)
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, "=%02X", c)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// QueryAnswer is query's answer.
|
||||
type QueryAnswer struct {
|
||||
Entries []IndexEntry `json:"entries"`
|
||||
}
|
||||
|
||||
// RebuildFrom is what a rebuild derives the state from: the history, or the snapshot and the operations after it.
|
||||
const (
|
||||
RebuildFromHistory = "history"
|
||||
RebuildFromSnapshot = "snapshot"
|
||||
)
|
||||
|
||||
// RebuildRequest derives an instance's state again (every instance's when Instance is empty), From the history or the
|
||||
// snapshot.
|
||||
type RebuildRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance,omitempty"`
|
||||
From string `json:"from,omitempty"`
|
||||
}
|
||||
|
||||
// ReindexRequest builds a declared index again from the history.
|
||||
type ReindexRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Index string `json:"index"`
|
||||
}
|
||||
|
||||
// LedgerRequest names a ledger: lag, fill.
|
||||
type LedgerRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
}
|
||||
|
||||
// LedgerRemoveRequest names one instance of a ledger to remove from its history, its state and its indexes (design 47 G
|
||||
// §587).
|
||||
type LedgerRemoveRequest struct {
|
||||
Ledger string `json:"ledger"`
|
||||
Instance string `json:"instance"`
|
||||
}
|
||||
|
||||
// LedgerRemoved is remove's answer: the instance, the operations removed and the bytes the history held less after it;
|
||||
// an instance the ledger does not hold answers 0 and 0, so a removal said twice changes nothing.
|
||||
type LedgerRemoved struct {
|
||||
Instance string `json:"instance"`
|
||||
Operations uint64 `json:"operations"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
}
|
||||
|
||||
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
|
||||
type Repaired struct {
|
||||
Written int `json:"written"`
|
||||
}
|
||||
|
||||
// InstanceLag is one instance whose state or snapshot is behind its history: the sequences each holds.
|
||||
type InstanceLag struct {
|
||||
Instance string `json:"instance"`
|
||||
History uint64 `json:"history"`
|
||||
State uint64 `json:"state"`
|
||||
Snapshot uint64 `json:"snapshot"`
|
||||
}
|
||||
|
||||
// IndexLag is how far one declared index is behind the history.
|
||||
type IndexLag struct {
|
||||
Index string `json:"index"`
|
||||
Behind uint64 `json:"behind"`
|
||||
}
|
||||
|
||||
// Lagged is lag's answer: every instance and index behind its history; none when nothing is behind.
|
||||
type Lagged struct {
|
||||
Instances []InstanceLag `json:"instances,omitempty"`
|
||||
Indexes []IndexLag `json:"indexes,omitempty"`
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A-42: stream.publish is a method of its own, beside read and fill, and its request and answer keep their field names.
|
||||
func TestStreamPublishIsAMethodWithItsShapes(t *testing.T) {
|
||||
if StreamPublish != "mesh/stream.publish" || StreamPublish == StreamRead || StreamPublish == StreamFill {
|
||||
t.Fatalf("method %q", StreamPublish)
|
||||
}
|
||||
b, _ := json.Marshal(StreamPublishRequest{Stream: "events", Event: "scrub.requested", Body: json.RawMessage(`{"a":1}`), ID: "x"})
|
||||
if string(b) != `{"stream":"events","event":"scrub.requested","body":{"a":1},"id":"x"}` {
|
||||
t.Errorf("request %s", b)
|
||||
}
|
||||
a, _ := json.Marshal(StreamPublished{Sequence: 7})
|
||||
if string(a) != `{"sequence":7}` {
|
||||
t.Errorf("answer %s", a)
|
||||
}
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// SeatChunks is the send of one seat verb's answer file in chunks (design 47 G §681, A-64, A-139), a verb whose seat
|
||||
// definition says `chunks: true`: the holder's process sends the nth chunk on mesh.seat.<seat>.chunk.<verb>.<call>.<n>,
|
||||
// and the caller acknowledges it on mesh.seat.<seat>.ack.<verb>.<call>.<n> within SeatAckWithin before the next is sent.
|
||||
// `call` is the 128 random bits the caller chose. The holder sends with it and the caller receives with it: one form.
|
||||
type SeatChunks struct {
|
||||
Seat, Verb, Call string
|
||||
}
|
||||
|
||||
// SeatChunkBytes is at most what one chunk carries: 512 KiB.
|
||||
const SeatChunkBytes = 512 << 10
|
||||
|
||||
// SeatAckWithin is how long the holder waits for a chunk's acknowledgement, and so how long a caller waits for the next chunk.
|
||||
const SeatAckWithin = 30 * time.Second
|
||||
|
||||
// SeatChunk is the body of one chunk: its number from 1, how many there are, and its bytes.
|
||||
type SeatChunk struct {
|
||||
N int64 `json:"n"`
|
||||
Of int64 `json:"of"`
|
||||
Data []byte `json:"data"`
|
||||
}
|
||||
|
||||
// Chunk is the subject of the nth chunk.
|
||||
func (c SeatChunks) Chunk(n int64) string {
|
||||
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
|
||||
}
|
||||
|
||||
// Ack is the subject the nth chunk is acknowledged on.
|
||||
func (c SeatChunks) Ack(n int64) string {
|
||||
return "mesh.seat." + c.Seat + ".ack." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
|
||||
}
|
||||
|
||||
// Chunks is every chunk subject of this call: what a caller subscribes to before it asks the verb.
|
||||
func (c SeatChunks) Chunks() string {
|
||||
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + ".*"
|
||||
}
|
||||
|
||||
// SeatAcks is every acknowledgement of a verb's sends: what the holder subscribes to.
|
||||
func SeatAcks(seat, verb string) string {
|
||||
return "mesh.seat." + seat + ".ack." + verb + ".*.*"
|
||||
}
|
||||
|
||||
// ErrSeatChunkMissed is a chunk that did not come within its bound: the holder ended the send.
|
||||
var ErrSeatChunkMissed = errors.New("a chunk did not come within its bound, so the send ended")
|
||||
|
||||
// Receive reads `of` chunks in order, writes each one's bytes to w and acknowledges it before the next is sent. next
|
||||
// yields the subject and body of the next chunk heard on Chunks(), or ErrSeatChunkMissed at its bound (SeatAckWithin);
|
||||
// ack publishes on a subject. A chunk out of order, of another count or on another subject ends the receive, and so the
|
||||
// send, since it is never acknowledged. It answers the bytes written.
|
||||
func (c SeatChunks) Receive(of int64, next func() (subject string, body []byte, err error), ack func(subject string) error, w io.Writer) (int64, error) {
|
||||
var written int64
|
||||
for n := int64(1); n <= of; n++ {
|
||||
subject, body, err := next()
|
||||
if err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d: %w", n, of, err)
|
||||
}
|
||||
if subject != c.Chunk(n) {
|
||||
return written, fmt.Errorf("chunk %d of %d came on %s, not %s", n, of, subject, c.Chunk(n))
|
||||
}
|
||||
var ch SeatChunk
|
||||
if err := json.Unmarshal(body, &ch); err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d does not read: %w", n, of, err)
|
||||
}
|
||||
if ch.N != n || ch.Of != of {
|
||||
return written, fmt.Errorf("chunk %d of %d says it is %d of %d", n, of, ch.N, ch.Of)
|
||||
}
|
||||
k, err := w.Write(ch.Data)
|
||||
written += int64(k)
|
||||
if err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d cannot be written: %w", n, of, err)
|
||||
}
|
||||
if err := ack(c.Ack(n)); err != nil {
|
||||
return written, fmt.Errorf("chunk %d of %d cannot be acknowledged: %w", n, of, err)
|
||||
}
|
||||
}
|
||||
return written, nil
|
||||
}
|
||||
|
||||
// ChunkLine is the core connection as a send of chunks reaches it, which a *nats.Conn is: a chunk is published on its subject and
|
||||
// kept on no stream, and the acknowledgements are subscribed to, since the caller asks each as a request.
|
||||
type ChunkLine interface {
|
||||
Publish(subject string, data []byte) error
|
||||
Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error)
|
||||
}
|
||||
|
||||
// Send sends body in chunks of at most SeatChunkBytes, the holder's side of Receive (design 47 G §681, A-139, A-237): it subscribes
|
||||
// to the verb's acknowledgements (SeatAcks), answers each as the request it is, publishes the nth chunk on Chunk(n) only once the
|
||||
// n-1th is acknowledged on Ack(n-1), and ends the subscription when the send ends. An acknowledgement that does not come within
|
||||
// `within` (SeatAckWithin where the caller has no other bound) ends the send with ErrSeatChunkMissed, and logf says where. Another
|
||||
// call's acknowledgements are skipped. It blocks until the send ends, so a verb answers its chunk count first and runs it after.
|
||||
func (c SeatChunks) Send(line ChunkLine, body []byte, within time.Duration, logf func(string, ...any)) error {
|
||||
if logf == nil {
|
||||
logf = func(string, ...any) {}
|
||||
}
|
||||
acks := make(chan string, 1)
|
||||
sub, err := line.Subscribe(SeatAcks(c.Seat, c.Verb), func(m *nats.Msg) {
|
||||
if m.Reply != "" {
|
||||
_ = m.Respond([]byte("{}"))
|
||||
}
|
||||
select {
|
||||
case acks <- m.Subject:
|
||||
default:
|
||||
}
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("the acknowledgements cannot be heard, so no chunk is sent: %w", err)
|
||||
}
|
||||
defer func() { _ = sub.Unsubscribe() }()
|
||||
total := int64((len(body) + SeatChunkBytes - 1) / SeatChunkBytes)
|
||||
for n := int64(1); n <= total; n++ {
|
||||
from := (n - 1) * SeatChunkBytes
|
||||
to := min(int64(len(body)), from+SeatChunkBytes)
|
||||
data, err := json.Marshal(SeatChunk{N: n, Of: total, Data: body[from:to]})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := line.Publish(c.Chunk(n), data); err != nil {
|
||||
logf("the send %s of %s ends at chunk %d of %d: %v", c.Call, c.Verb, n, total, err)
|
||||
return fmt.Errorf("chunk %d of %d: %w", n, total, err)
|
||||
}
|
||||
if !c.acknowledged(acks, c.Ack(n), within) {
|
||||
logf("the send %s of %s ends at chunk %d of %d: no acknowledgement within %s", c.Call, c.Verb, n, total, within)
|
||||
return fmt.Errorf("chunk %d of %d: %w", n, total, ErrSeatChunkMissed)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// acknowledged waits within for the acknowledgement on subject, skipping another call's.
|
||||
func (c SeatChunks) acknowledged(acks <-chan string, subject string, within time.Duration) bool {
|
||||
timer := time.NewTimer(within)
|
||||
defer timer.Stop()
|
||||
for {
|
||||
select {
|
||||
case got := <-acks:
|
||||
if got == subject {
|
||||
return true
|
||||
}
|
||||
case <-timer.C:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,72 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestSeatChunksReceive: the chunks of a send are written in order and each acknowledged before the next; a missed,
|
||||
// misplaced or miscounted chunk ends the receive unacknowledged (design 47 G §681, A-139).
|
||||
func TestSeatChunksReceive(t *testing.T) {
|
||||
c := SeatChunks{Seat: "mesh-broker", Verb: "snapshot-get", Call: "0123456789abcdef0123456789abcdef"}
|
||||
if c.Chunk(2) != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
|
||||
c.Ack(2) != "mesh.seat.mesh-broker.ack.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
|
||||
c.Chunks() != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.*" ||
|
||||
SeatAcks("mesh-broker", "snapshot-get") != "mesh.seat.mesh-broker.ack.snapshot-get.*.*" {
|
||||
t.Fatalf("subjects: %s %s %s", c.Chunk(2), c.Ack(2), c.Chunks())
|
||||
}
|
||||
type sent struct {
|
||||
subject string
|
||||
body SeatChunk
|
||||
missed bool
|
||||
}
|
||||
good := []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 3, Data: []byte("ab")}},
|
||||
{subject: c.Chunk(2), body: SeatChunk{N: 2, Of: 3, Data: []byte("cd")}},
|
||||
{subject: c.Chunk(3), body: SeatChunk{N: 3, Of: 3, Data: []byte("e")}}}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
sends []sent
|
||||
want string
|
||||
acks int
|
||||
err string
|
||||
}{
|
||||
{name: "three chunks", sends: good, want: "abcde", acks: 3},
|
||||
{name: "a missed chunk", sends: append(good[:1:1], sent{missed: true}), want: "ab", acks: 1, err: "chunk 2 of 3"},
|
||||
{name: "out of order", sends: []sent{good[1]}, acks: 0, err: "came on"},
|
||||
{name: "another count", sends: []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 2, Data: []byte("x")}}}, err: "says it is 1 of 2"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
i, acks := 0, 0
|
||||
next := func() (string, []byte, error) {
|
||||
if i >= len(tc.sends) || tc.sends[i].missed {
|
||||
return "", nil, ErrSeatChunkMissed
|
||||
}
|
||||
s := tc.sends[i]
|
||||
i++
|
||||
body, _ := json.Marshal(s.body)
|
||||
return s.subject, body, nil
|
||||
}
|
||||
ack := func(subject string) error {
|
||||
acks++
|
||||
if subject != c.Ack(int64(acks)) {
|
||||
t.Fatalf("ack %d on %s", acks, subject)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var out bytes.Buffer
|
||||
n, err := c.Receive(3, next, ack, &out)
|
||||
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
|
||||
t.Fatalf("err %v, want %q", err, tc.err)
|
||||
}
|
||||
if tc.name == "a missed chunk" && !errors.Is(err, ErrSeatChunkMissed) {
|
||||
t.Fatalf("a missed chunk is not ErrSeatChunkMissed: %v", err)
|
||||
}
|
||||
if out.String() != tc.want || n != int64(len(tc.want)) || acks != tc.acks {
|
||||
t.Fatalf("wrote %q (%d), %d acks; want %q, %d acks", out.String(), n, acks, tc.want, tc.acks)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
package buscalls
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// lineMem is a core connection in memory: what Send publishes is kept, and when ackOf says so the chunk is acknowledged as the
|
||||
// caller does, with a request on the ack subject, whose reply Send must give.
|
||||
type lineMem struct {
|
||||
mu sync.Mutex
|
||||
handler nats.MsgHandler
|
||||
subject string
|
||||
chunks map[string][]byte
|
||||
order []string
|
||||
ackOf func(n int64) (string, bool)
|
||||
}
|
||||
|
||||
func (l *lineMem) Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
l.handler, l.subject = cb, subject
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (l *lineMem) Publish(subject string, data []byte) error {
|
||||
l.mu.Lock()
|
||||
l.chunks[subject] = data
|
||||
l.order = append(l.order, subject)
|
||||
h := l.handler
|
||||
l.mu.Unlock()
|
||||
var c SeatChunk
|
||||
_ = json.Unmarshal(data, &c)
|
||||
if sub, ok := l.ackOf(c.N); ok {
|
||||
go h(&nats.Msg{Subject: sub, Reply: "_INBOX.x", Sub: nil})
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TestSeatChunksSend: the bytes go in chunks of at most SeatChunkBytes, each published only after the one before is
|
||||
// acknowledged, and the receive of the same form reads them back; an acknowledgement missed or of another call ends the
|
||||
// send (design 47 G §681, A-237, A-139).
|
||||
func TestSeatChunksSend(t *testing.T) {
|
||||
c := SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "0123456789abcdef"}
|
||||
body := bytes.Repeat([]byte("x"), 2*SeatChunkBytes+5)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
ackOf func(int64) (string, bool)
|
||||
err string
|
||||
sent int
|
||||
}{
|
||||
{name: "every chunk acknowledged", ackOf: func(n int64) (string, bool) { return c.Ack(n), true }, sent: 3},
|
||||
{name: "no acknowledgement", ackOf: func(int64) (string, bool) { return "", false }, err: "chunk 1 of 3", sent: 1},
|
||||
{name: "another call's acknowledgement", ackOf: func(n int64) (string, bool) {
|
||||
return SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "fedcba9876543210"}.Ack(n), true
|
||||
}, err: "chunk 1 of 3", sent: 1},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
l := &lineMem{chunks: map[string][]byte{}, ackOf: tc.ackOf}
|
||||
err := c.Send(l, body, time.Millisecond*50, func(string, ...any) {})
|
||||
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
|
||||
t.Fatalf("err %v, want %q", err, tc.err)
|
||||
}
|
||||
if len(l.order) != tc.sent {
|
||||
t.Fatalf("%d chunks sent, want %d", len(l.order), tc.sent)
|
||||
}
|
||||
if l.subject != SeatAcks("mesh-catalog", "input") {
|
||||
t.Fatalf("acknowledgements heard on %q", l.subject)
|
||||
}
|
||||
if tc.err != "" {
|
||||
return
|
||||
}
|
||||
var got bytes.Buffer
|
||||
i := 0
|
||||
_, err = c.Receive(3, func() (string, []byte, error) {
|
||||
s := l.order[i]
|
||||
i++
|
||||
return s, l.chunks[s], nil
|
||||
}, func(string) error { return nil }, &got)
|
||||
if err != nil || !bytes.Equal(got.Bytes(), body) {
|
||||
t.Fatalf("received %d bytes, %v", got.Len(), err)
|
||||
}
|
||||
})
|
||||
}
|
||||
if err := c.Send(&lineMem{chunks: map[string][]byte{}}, nil, time.Second, nil); err != nil {
|
||||
t.Errorf("no bytes: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import "time"
|
||||
|
||||
// AccountIssuedEvent is what the vault says when it minted an account: AccountIssued, never a value or a hash. The state of
|
||||
// wanted accounts it mints from is the grants part's (grantsstate.AccountsState; novox/hq ADR 0634 §1).
|
||||
const AccountIssuedEvent = "account-issued"
|
||||
|
||||
// AccountIssued is the vault's account-issued: which account it minted, and its generation. Never a value or a hash.
|
||||
type AccountIssued struct {
|
||||
User string `json:"user"`
|
||||
Kind Kind `json:"kind"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
Process string `json:"process,omitempty"`
|
||||
Generation int `json:"generation"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"git.novox.be/novox/mesh-sdk/go/declaration"
|
||||
)
|
||||
|
||||
// DeclarationFilesBucket is the prefix of the buckets' names, which mesh-sdk fixes for the node-engine and the controller alike.
|
||||
const DeclarationFilesBucket = declaration.FileBucketPrefix
|
||||
@@ -1,59 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// A Consumer is a durable subscription the controller creates on a module's behalf. A module
|
||||
// declares what it reacts to, never how delivery works, so it does not name these and cannot
|
||||
// misconfigure them.
|
||||
type Consumer struct {
|
||||
Name string
|
||||
Stream string
|
||||
// Filters are the subjects this consumer receives. One consumer per module with several
|
||||
// filters, rather than one per consumed event: its ack subject is derived from its name, and
|
||||
// a module with five consumers would need five ack permissions to ack its own deliveries.
|
||||
Filters []string
|
||||
// Queue is the queue group, set for a seat's worker so that "exactly one holder" survives a
|
||||
// seat later being relaxed to several. Authority and delivery are kept separate on purpose.
|
||||
Queue string
|
||||
// Push asks the server to deliver to a subject rather than wait to be pulled.
|
||||
//
|
||||
// For the mesh's own consumer, where the controller wants every message to arrive in the one
|
||||
// loop it already runs: pulling would mean a second goroutine fetching batches and handing
|
||||
// them over, and a loop that acts on one message at a time is the property the store window
|
||||
// depends on. A queue group implies this, because a group has nothing to pull from.
|
||||
Push bool
|
||||
// TakesOverFrom names a consumer of the same stream whose filter this one takes over (issue 973): made while
|
||||
// that consumer still filters this one's subject, it starts after what that consumer has acknowledged, so a
|
||||
// message the old one held unacknowledged is not lost to the change. Otherwise it starts as FromNow says.
|
||||
TakesOverFrom string
|
||||
// DeliverSubject, when set, is where a push consumer delivers instead of DeliverSubjectFor's.
|
||||
DeliverSubject string
|
||||
// AckWaitSeconds before an unacknowledged delivery is redelivered.
|
||||
AckWaitSeconds int
|
||||
// MaxDeliver is how often a message is handed over before the consumer gives it up; zero for no
|
||||
// bound. What a consumer gives up is kept in DEAD_LETTERS by the controller (novox/hq issue 330).
|
||||
MaxDeliver int
|
||||
// MaxAckPending is how many deliveries the server lets stand unacknowledged at once; zero for
|
||||
// the server's default, which is many. **One, for a consumer handled one at a time**
|
||||
// (novox/hq issue 175): a handler that builds for minutes keeps its own message alive with a
|
||||
// heartbeat, but everything handed over behind it times out unacknowledged and comes back —
|
||||
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
|
||||
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
|
||||
MaxAckPending int
|
||||
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
|
||||
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
|
||||
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
|
||||
// holds, replays every merge and every build of that week as if it had just happened. A consumer
|
||||
// that exists keeps where it is, whatever this says; only its making is decided here.
|
||||
FromNow bool
|
||||
Why string
|
||||
// Resettable says why this consumer may be re-made to deliver from now by the mesh itself, with
|
||||
// nobody asked (novox/hq to-be 45 §7, healer H4): what a reset drops, something else catches up.
|
||||
// Empty for every consumer where nothing would — a module's, whose events would be lost to it.
|
||||
// Not a property of the consumer on the bus: nothing here is sent to the server.
|
||||
Resettable string
|
||||
}
|
||||
|
||||
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
|
||||
type ModuleConsumer struct {
|
||||
Node, Module string
|
||||
Consumer Consumer
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
// Package buscompose is the bus's names and shapes every part reads (novox/hq ADR 0593 §1): the mesh's streams and
|
||||
// dead letters, a module's log, stream and buckets as the bus names them, the account a part asks for, a principal's
|
||||
// shape, and the grants part's memberships key. The composer of the bus's user list, which reads these and makes every
|
||||
// user's permissions, is the grants part's own (mesh-delivery modules/mesh-grants internal/buscompose).
|
||||
package buscompose
|
||||
@@ -1,106 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The one place the mesh's event subjects are named (design 34 every-event §1, §25, §26), so neither the grants part
|
||||
// nor the tool runner spells the event namespace itself.
|
||||
|
||||
// The namespaces an event lands in, as `from` names them (design 34 every-event §13, §26).
|
||||
const (
|
||||
fromSeat = "seat"
|
||||
fromModule = "module"
|
||||
fromStream = "stream"
|
||||
)
|
||||
|
||||
// eventNamespaces are the three, each with the prefix its subjects start with, in EveryEvent's order.
|
||||
var eventNamespaces = []struct{ from, prefix string }{
|
||||
{fromSeat, "mesh.seat."},
|
||||
{fromModule, "mesh.mod."},
|
||||
{fromStream, moduleStreamPrefix},
|
||||
}
|
||||
|
||||
// EveryEvent is the subjects the tool runner's account hears: every seat's events, every module's, and every module
|
||||
// stream's, a restricted one included, in that order (novox/hq ADR 0670 §1, ADR 0671 §1).
|
||||
func EveryEvent() []string {
|
||||
out := make([]string, 0, len(eventNamespaces))
|
||||
for _, ns := range eventNamespaces {
|
||||
out = append(out, ns.prefix+"*.event.>")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// eventToken is the form of each token of an event name.
|
||||
var eventToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// EventSubjects is the subjects one event name lands on. A name is `<emitter>.<event>`, optionally followed by further
|
||||
// tokens (`operator-channel.decided.claude-code`), each token of the form `[A-Za-z0-9_-]+`. With from empty it answers
|
||||
// six subjects, per namespace (seat, module, stream) `mesh.<ns>.<emitter>.event.<rest>` and the same followed by `.>`;
|
||||
// with from `seat`, `module` or `stream` only that namespace's two. It refuses, naming the rule, a name of fewer than two
|
||||
// tokens, an empty token or a token outside that form (so `*` and `>` are refused), and a from of any other word.
|
||||
func EventSubjects(name, from string) ([]string, error) {
|
||||
tokens := strings.Split(name, ".")
|
||||
if len(tokens) < 2 {
|
||||
return nil, fmt.Errorf("event name %q has fewer than two tokens: a name is <seat or module>.<event>", name)
|
||||
}
|
||||
for _, t := range tokens {
|
||||
if !eventToken.MatchString(t) {
|
||||
return nil, fmt.Errorf("event name %q has the token %q: each token of a name matches [A-Za-z0-9_-]+, "+
|
||||
"so no token is empty and none is * or >", name, t)
|
||||
}
|
||||
}
|
||||
emitter, rest := tokens[0], strings.Join(tokens[1:], ".")
|
||||
var out []string
|
||||
for _, ns := range eventNamespaces {
|
||||
if from != "" && from != ns.from {
|
||||
continue
|
||||
}
|
||||
subject := ns.prefix + emitter + ".event." + rest
|
||||
out = append(out, subject, subject+".>")
|
||||
}
|
||||
if out == nil {
|
||||
return nil, fmt.Errorf("from %q is none of %s, %s or %s", from, fromSeat, fromModule, fromStream)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// EventName is the inverse of EventSubjects for a heard subject: for `mesh.seat.<emitter>.event.<rest>`,
|
||||
// `mesh.mod.<emitter>.event.<rest>` or `mesh.stream.<emitter>.event.<rest>` it answers the name `<emitter>.<rest>` and
|
||||
// its namespace (`seat`, `module` or `stream`); any other subject is answered as not an event (ok false).
|
||||
func EventName(subject string) (name, from string, ok bool) {
|
||||
for _, ns := range eventNamespaces {
|
||||
after, found := strings.CutPrefix(subject, ns.prefix)
|
||||
if !found {
|
||||
continue
|
||||
}
|
||||
emitter, rest, found := strings.Cut(after, ".event.")
|
||||
if !found || emitter == "" || rest == "" || strings.Contains(emitter, ".") {
|
||||
return "", "", false
|
||||
}
|
||||
return emitter + "." + rest, ns.from, true
|
||||
}
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
// Overlaps says whether some subject matches both a and b, a wildcard on either side: `*` stands for one token and
|
||||
// `>`, last, for one token or more (design 34 every-event §34).
|
||||
func Overlaps(a, b string) bool {
|
||||
at, bt := strings.Split(a, "."), strings.Split(b, ".")
|
||||
for i := 0; ; i++ {
|
||||
switch {
|
||||
case i == len(at) && i == len(bt):
|
||||
return true
|
||||
case i == len(at) || i == len(bt):
|
||||
return false
|
||||
case at[i] == ">" || bt[i] == ">":
|
||||
return true
|
||||
case at[i] == "*" || bt[i] == "*" || at[i] == bt[i]:
|
||||
continue
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The mesh's event subjects are named in one place: EveryEvent, EventSubjects, EventName and Overlaps (design 34
|
||||
// every-event §1, §25, §26, §34, §37).
|
||||
func TestTheEventSubjectsAreNamedInOnePlace(t *testing.T) {
|
||||
// EE-S1/AC1 (E §1, §25): the three subjects, in order.
|
||||
if got, want := EveryEvent(), []string{"mesh.seat.*.event.>", "mesh.mod.*.event.>", "mesh.stream.*.event.>"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("EveryEvent is %v, want %v", got, want)
|
||||
}
|
||||
|
||||
// EE-S1/AC2, AC3 (E §1, §26): a name's subjects, all six or one namespace's two, and the refusals naming the rule.
|
||||
subjects := []struct {
|
||||
name, from string
|
||||
want []string
|
||||
refused string
|
||||
}{
|
||||
{name: "operator-channel.decided.claude-code", want: []string{
|
||||
"mesh.seat.operator-channel.event.decided.claude-code", "mesh.seat.operator-channel.event.decided.claude-code.>",
|
||||
"mesh.mod.operator-channel.event.decided.claude-code", "mesh.mod.operator-channel.event.decided.claude-code.>",
|
||||
"mesh.stream.operator-channel.event.decided.claude-code", "mesh.stream.operator-channel.event.decided.claude-code.>",
|
||||
}},
|
||||
{name: "mesh-delivery-view.delivered", from: "seat", want: []string{
|
||||
"mesh.seat.mesh-delivery-view.event.delivered", "mesh.seat.mesh-delivery-view.event.delivered.>"}},
|
||||
{name: "mesh-delivery-view.delivered", from: "module", want: []string{
|
||||
"mesh.mod.mesh-delivery-view.event.delivered", "mesh.mod.mesh-delivery-view.event.delivered.>"}},
|
||||
{name: "mesh-mail.mail", from: "stream", want: []string{
|
||||
"mesh.stream.mesh-mail.event.mail", "mesh.stream.mesh-mail.event.mail.>"}},
|
||||
{name: "mesh-delivery-view", refused: "fewer than two tokens"},
|
||||
{name: "mesh-delivery-view..delivered", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.*", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.>", refused: "[A-Za-z0-9_-]+"},
|
||||
{name: "mesh-delivery-view.delivered", from: "bench", refused: "none of seat, module or stream"},
|
||||
}
|
||||
for _, c := range subjects {
|
||||
got, err := EventSubjects(c.name, c.from)
|
||||
if c.refused != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), c.refused) {
|
||||
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want refused naming %q", c.name, c.from, got, err, c.refused)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil || !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want %v", c.name, c.from, got, err, c.want)
|
||||
}
|
||||
// EE-S1/AC5 (E §1): every subject it answers is covered by one of EveryEvent's.
|
||||
for _, s := range got {
|
||||
covered := false
|
||||
for _, e := range EveryEvent() {
|
||||
covered = covered || Overlaps(e, s)
|
||||
}
|
||||
if !covered {
|
||||
t.Errorf("%s is covered by none of EveryEvent", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// EE-S1/AC4 (E §1, §26): a heard subject's name and namespace, and any other subject not an event.
|
||||
names := []struct{ subject, name, from string }{
|
||||
{"mesh.seat.operator-channel.event.decided.claude-code", "operator-channel.decided.claude-code", "seat"},
|
||||
{"mesh.mod.mesh-delivery-view.event.delivered", "mesh-delivery-view.delivered", "module"},
|
||||
{"mesh.stream.mesh-delivery-checker.event.verdict", "mesh-delivery-checker.verdict", "stream"},
|
||||
{"mesh.seat.intake.call.accept.ace", "", ""},
|
||||
{"mesh.mod.netcheck.call.netcheck_dns", "", ""},
|
||||
{"$KV.views.x", "", ""},
|
||||
{"mesh.seat..event.x", "", ""},
|
||||
}
|
||||
for _, c := range names {
|
||||
name, from, ok := EventName(c.subject)
|
||||
if name != c.name || from != c.from || ok != (c.name != "") {
|
||||
t.Errorf("EventName(%q) answered %q, %q, %v; want %q, %q", c.subject, name, from, ok, c.name, c.from)
|
||||
}
|
||||
}
|
||||
|
||||
// EE-S1/AC6 (E §34, §37): overlapping subjects, a wildcard on either side.
|
||||
overlaps := []struct {
|
||||
a, b string
|
||||
want bool
|
||||
}{
|
||||
{"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.*.message", true},
|
||||
{"mesh.mod.mail.event.x", "mesh.stream.mail.event.x", false},
|
||||
{"mesh.mod.mail.event.x.>", "mesh.mod.mail.event.x", false},
|
||||
{"mesh.mod.mail.event.*", "mesh.mod.*.event.x", true},
|
||||
{"mesh.mod.mail.event.x", "mesh.mod.mail.event.x", true},
|
||||
{"mesh.mod.*.event.>", "mesh.mod.mail.event.x.ace", true},
|
||||
{"mesh.mod.mail.event.x.ace", "mesh.mod.mail.event.x", false},
|
||||
}
|
||||
for _, c := range overlaps {
|
||||
if Overlaps(c.a, c.b) != c.want || Overlaps(c.b, c.a) != c.want {
|
||||
t.Errorf("Overlaps(%q, %q) is not %v both ways", c.a, c.b, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// The mesh's caps on a log: what an entry's value may weigh, what a message on the log's stream may
|
||||
// weigh — the value and its headers, which the server counts in a message's size, so a value of the
|
||||
// full size still fits with the expected-last-sequence header an append carries — and what a log holds
|
||||
// when its module says nothing and at most.
|
||||
const (
|
||||
LogMaxEntryBytes = 256 * 1024
|
||||
LogMaxMessageBytes = LogMaxEntryBytes + 4*1024
|
||||
LogDefaultMiB = 1024
|
||||
)
|
||||
|
||||
// A Log is one module's declared log as the bus holds it.
|
||||
type Log struct {
|
||||
Module string
|
||||
Name string
|
||||
// MaxMiB is its cap in MiB; zero is LogDefaultMiB.
|
||||
MaxMiB int
|
||||
// MaxAgeDays is how long an entry is kept; zero is as long as the log (catalogue.LogDeclaration).
|
||||
MaxAgeDays int
|
||||
}
|
||||
|
||||
// LogStreamPrefix starts every log's stream name, which no other stream of the mesh's starts with.
|
||||
const LogStreamPrefix = "LOG_"
|
||||
|
||||
// LogStreamName is the stream a module's log lives in: `LOG_<module>_<name>`. The module and the
|
||||
// local name are each one token with no underscore, so two modules can never derive one stream.
|
||||
func LogStreamName(module, name string) string { return LogStreamPrefix + module + "_" + name }
|
||||
|
||||
// LogSubject is the subject a log's entries are published under, without the key: an entry for key K
|
||||
// is on `<LogSubject>.<K>`.
|
||||
func LogSubject(module, name string) string { return "mesh.log." + module + "." + name }
|
||||
|
||||
// Stream is this log's stream name.
|
||||
func (l Log) Stream() string { return LogStreamName(l.Module, l.Name) }
|
||||
|
||||
// Subject is this log's subject, without the key.
|
||||
func (l Log) Subject() string { return LogSubject(l.Module, l.Name) }
|
||||
|
||||
// MaxBytes is this log's cap in bytes.
|
||||
func (l Log) MaxBytes() int64 {
|
||||
mib := l.MaxMiB
|
||||
if mib <= 0 {
|
||||
mib = LogDefaultMiB
|
||||
}
|
||||
return int64(mib) * 1024 * 1024
|
||||
}
|
||||
|
||||
// Why is carried into the server's description of the stream, so somebody reading the server's own
|
||||
// state finds whose it is and why it is kept.
|
||||
func (l Log) Why() string {
|
||||
return fmt.Sprintf("%s's log %q (novox/hq ADR 0297): its record of operations, one entry per operation "+
|
||||
"under its key, appended by %s alone and kept as long as the log; never removed by the mesh, because "+
|
||||
"it is data", l.Module, l.Name, l.Module)
|
||||
}
|
||||
|
||||
// Config is the stream a log is, field by field as novox/hq ADR 0297 fixes it: a file stream kept by
|
||||
// limits, with no maximum age unless the log says one (ADR 0565 §10) and no cap per key, whose message is an entry's value and 4 KiB of
|
||||
// headers at most, that refuses a new entry when full rather than
|
||||
// drop an old one, and that refuses deleting an entry or purging it. Direct gets are allowed, which
|
||||
// is how the runtime reads it.
|
||||
func (l Log) Config() jetstream.StreamConfig {
|
||||
return jetstream.StreamConfig{
|
||||
Name: l.Stream(),
|
||||
Description: l.Why(),
|
||||
Subjects: []string{l.Subject() + ".>"},
|
||||
Storage: jetstream.FileStorage,
|
||||
Retention: jetstream.LimitsPolicy,
|
||||
MaxAge: time.Duration(l.MaxAgeDays) * 24 * time.Hour,
|
||||
MaxMsgs: -1,
|
||||
MaxMsgsPerSubject: -1,
|
||||
MaxBytes: l.MaxBytes(),
|
||||
MaxMsgSize: LogMaxMessageBytes,
|
||||
Discard: jetstream.DiscardNew,
|
||||
AllowDirect: true,
|
||||
DenyDelete: true,
|
||||
DenyPurge: true,
|
||||
Replicas: 1,
|
||||
}
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// moduleStreamPrefix starts every subject a module's own stream takes.
|
||||
const moduleStreamPrefix = "mesh.stream."
|
||||
|
||||
// ModuleStreamSubject is where a module's event on its own stream lands: `mesh.stream.<module>.event.<event>`.
|
||||
// The token before `.event.` is the emitter, as on EVENTS, so a runtime reads the same key from it.
|
||||
func ModuleStreamSubject(module, event string) string {
|
||||
return moduleStreamPrefix + module + ".event." + event
|
||||
}
|
||||
|
||||
// WirePattern is a stream's local event pattern as the bus spells it: `**` is `>`, `*` stays `*`. A reader turns a
|
||||
// stream's events into the bus's subjects with it (design 46 7-approval-mark §5): the grants part composing a stream, and
|
||||
// the tool runner deriving a restricted stream's marked subjects.
|
||||
func WirePattern(local string) string {
|
||||
parts := strings.Split(local, ".")
|
||||
for i, p := range parts {
|
||||
if p == "**" {
|
||||
parts[i] = ">"
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
// A ModuleStream is one module's declared stream as the bus holds it.
|
||||
type ModuleStream struct {
|
||||
Module string
|
||||
// Name is its name on the bus, `MAIL`.
|
||||
Name string
|
||||
// Events are the events it carries, as the bus spells them (WirePattern).
|
||||
Events []string
|
||||
MaxAgeDays int
|
||||
MaxMiB int
|
||||
// Restricted says a trusted config value of the module grants who hears it (ADR 0363 §4).
|
||||
Restricted bool
|
||||
// WorkQueue says this stream holds ADR 0436 §6's work-queue retention: a message is removed once a
|
||||
// consumer acknowledges it, and the bus refuses a new one when full rather than dropping its oldest.
|
||||
WorkQueue bool
|
||||
// Duplicates is the work-queue's duplicate window in hours (novox/hq ADR 0436 §6); meaningless unless
|
||||
// WorkQueue.
|
||||
Duplicates int
|
||||
}
|
||||
|
||||
// Subjects are the subjects the stream takes.
|
||||
func (s ModuleStream) Subjects() []string {
|
||||
var out []string
|
||||
for _, e := range s.Events {
|
||||
out = append(out, ModuleStreamSubject(s.Module, e))
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// Stream is the stream the controller asserts: kept by limits, its age and size the module's, dropping
|
||||
// its oldest when full — a consumer that was away catches up within the age, and the module's own record
|
||||
// answers for anything older.
|
||||
func (s ModuleStream) Stream() Stream {
|
||||
why := fmt.Sprintf("%s's own stream (novox/hq ADR 0363): its events, kept %d days and at most %d MiB so a "+
|
||||
"consumer that was away catches up", s.Module, s.MaxAgeDays, s.MaxMiB)
|
||||
if s.Restricted {
|
||||
why += "; heard only by the consumers its trusted config value grants"
|
||||
}
|
||||
if s.WorkQueue {
|
||||
why += "; a work-queue (novox/hq ADR 0436 §6): a message is removed once its one consumer " +
|
||||
"acknowledges it, and a full stream refuses a new one rather than dropping what it holds"
|
||||
}
|
||||
out := Stream{
|
||||
Name: s.Name,
|
||||
Subjects: s.Subjects(),
|
||||
Retention: RetentionLimits,
|
||||
MaxAge: s.MaxAgeDays * 24 * 60 * 60,
|
||||
MaxBytes: int64(s.MaxMiB) * 1024 * 1024,
|
||||
Why: why,
|
||||
// Said on the bus, so a dead letter kept from it ages out with it (novox/hq ADR 0376 §3).
|
||||
Restricted: s.Restricted,
|
||||
}
|
||||
if s.WorkQueue {
|
||||
out.Retention = RetentionWorkQueue
|
||||
out.DiscardNew = true
|
||||
out.DuplicatesSeconds = s.Duplicates * 60 * 60
|
||||
// Said on the bus too, reusing the one flag a restricted stream's dead letters already age and hide
|
||||
// by (novox/hq ADR 0436 §6, ADR 0376 §3-4): a work-queue stream's grant is always empty (ADR 0436
|
||||
// §6 refuses one that carries a grant), so this never collides with a trusted-config restriction.
|
||||
out.Restricted = true
|
||||
// And its own metadata, apart from the bus's work-queue retention SEAT_* and other mesh-internal
|
||||
// streams share without being one of this clause's: only this flag says a dead letter kept from the
|
||||
// stream is drop-only (novox/hq ADR 0436 §6), never the stream's Retention alone.
|
||||
out.WorkQueue = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
|
||||
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
|
||||
// to be the only one (the indexer's judge, novox/hq ADR 0578 §1).
|
||||
type QueueOwner struct {
|
||||
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
|
||||
// 0436 §2).
|
||||
Process string
|
||||
// Stream is the work-queue stream's name on the bus.
|
||||
Stream string
|
||||
// MaxDeliver bounds how many times a message is handed over before it is given up as a dead letter
|
||||
// (novox/hq ADR 0436 §6).
|
||||
MaxDeliver int
|
||||
}
|
||||
|
||||
// Heard is a module's stream another module hears, and the subjects it hears of it: its `consumes`
|
||||
// within the stream and, for a restricted stream, within what the provider's trusted config value grants it
|
||||
// (ADR 0363 §5). Composed only with at least one subject.
|
||||
type Heard struct {
|
||||
Stream string
|
||||
Emitter string
|
||||
Filters []string
|
||||
// Restricted says the stream is restricted by its provider's trusted config value (ADR 0363 §4): then it
|
||||
// is heard only by a module on an account of its own (ADR 0376 §1, ComposedHears).
|
||||
Restricted bool
|
||||
}
|
||||
@@ -1,52 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/membership"
|
||||
"git.novox.be/novox/mesh-sdk/go/seatdef"
|
||||
)
|
||||
|
||||
// The names this package keeps for the grants part's composer (mesh-grants internal/buscompose, ADR 0593 §1) agree with
|
||||
// what the runtime reads: the memberships bucket, and a log's stream and subject.
|
||||
func TestTheBusNamesAgree(t *testing.T) {
|
||||
if GrantsMembershipsBucket != membership.GrantsBucket {
|
||||
t.Fatalf("the memberships bucket is %q here and %q in membership", GrantsMembershipsBucket, membership.GrantsBucket)
|
||||
}
|
||||
if MembershipKey("ace", "netcheck") != "ace.netcheck" {
|
||||
t.Fatal("a membership's key is not <node>.<module>")
|
||||
}
|
||||
if LogStreamName("mesh-delivery-builder", "build-log") == "" || LogSubject("mesh-delivery-builder", "build-log") == "" {
|
||||
t.Fatal("a log has no stream or subject")
|
||||
}
|
||||
// 1966/AC20: a ledger's history and snapshot subjects are one contract, which the grants part grants a ledger
|
||||
// account its publish on.
|
||||
if h, sn := LedgerHistorySubject("m", "records"), LedgerSnapshotSubject("m", "records"); h != "mesh.ledger.m.records.history" ||
|
||||
sn != "mesh.ledger.m.records.snapshot" {
|
||||
t.Fatalf("a ledger's subjects are %q and %q", h, sn)
|
||||
}
|
||||
// 1966/AC23: a ledger account for m on a is named ledger.a.m.
|
||||
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
|
||||
t.Fatalf("a ledger account is named %q", got)
|
||||
}
|
||||
// G §469: a state's bucket is capped at its max-mib; naming none is unbounded.
|
||||
if (Bucket{}).MaxBytes() != -1 || (Bucket{MaxMiB: 2}).MaxBytes() != 2<<20 {
|
||||
t.Fatal("a bucket's cap is not its max-mib in bytes")
|
||||
}
|
||||
// Design 46 7-approval-mark §5: a stream's local event pattern as the bus spells it.
|
||||
for local, wire := range map[string]string{"mail.**": "mail.>", "mail.*.message": "mail.*.message", "sent": "sent"} {
|
||||
if got := WirePattern(local); got != wire {
|
||||
t.Errorf("WirePattern(%q) = %q, want %q", local, got, wire)
|
||||
}
|
||||
}
|
||||
// Design 46 7-approval-mark §13: a per-machine state reaches its machine's key and the keys below it, no other machine's.
|
||||
if got := PerMachineKeys("ace"); len(got) != 2 || got[0] != "ace" || got[1] != "ace.>" {
|
||||
t.Errorf("a per-machine state reaches %v on ace", got)
|
||||
}
|
||||
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
|
||||
for _, st := range MeshStreams() {
|
||||
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
|
||||
t.Errorf("the shared events stream keeps %d s, a seat's definition says %d days", st.MaxAge, seatdef.SharedEventsMaxAgeDays)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,351 +0,0 @@
|
||||
// Composing the bus's own configuration.
|
||||
//
|
||||
// An account is *composed*, never called for: the controller writes accounts, users and
|
||||
// per-subject permissions into one file the host keeps current, and the server reloads it in
|
||||
// place (novox/hq ADR 0106 — never through a management API; design 25 §4).
|
||||
//
|
||||
// Everything here is pure. Given the principals, it returns the file's text — so the whole of the
|
||||
// mesh's authority model is testable as strings, with no server.
|
||||
//
|
||||
// **Permissions are per subject, so a module's own name is the server's to enforce.** ADR 0042
|
||||
// reserves a module's origin — it publishes only under its own name — and here that is a refusal
|
||||
// rather than something a library promises.
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A Kind is what a principal is, which decides the shape of its authority rather than its
|
||||
// contents: a module's comes from its declaration, a host's from its node, and the controller's
|
||||
// and the enrolment user's are fixed.
|
||||
type Kind string
|
||||
|
||||
const (
|
||||
KindModule Kind = "module"
|
||||
KindNode Kind = "node"
|
||||
KindEnrolment Kind = "enrolment"
|
||||
// KindPerson is somebody reaching the mesh's tools from a workstation (design 25 §7). Its
|
||||
// authority is a list of tools and nothing else — not control, not declarations, not builds,
|
||||
// and no ability to answer anything, because a person asks.
|
||||
KindPerson Kind = "person"
|
||||
// KindNodeTools is a machine's tool runtime (novox/hq ADR 0175, to-be 38): one process per
|
||||
// node, on the host side, serving every assigned module's tools and every held seat's verbs.
|
||||
// Its authority is the union of what the modules it carries would each have had for their
|
||||
// tools — and nothing of what they consume, because tools are what it runs, not reactions.
|
||||
KindNodeTools Kind = "node-tools"
|
||||
// KindProvisioner is a machine's provisioner client (novox/hq ADR 0349 clauses 6 and 11, design 57 P3): a
|
||||
// service of the module holding NodeProvisionerSeat, outside the tool runner, under an account of its own so
|
||||
// the vault can tell it from an agent. Its authority is provisionerGrants and nothing else.
|
||||
KindProvisioner Kind = "provisioner"
|
||||
// KindProcess is one process of a module that declares bus (novox/hq ADR 0436 §1, §2): a service of that
|
||||
// module, outside the tool runner and outside the module's own account, under a bus account scoped to
|
||||
// exactly what its bus says — never root, never the operator account, and never able to answer a tool
|
||||
// call or a seat's verb (§3). One per process and machine.
|
||||
KindProcess Kind = "process"
|
||||
// KindLedger is one module assignment's ledger credential (novox/hq ADR 0597 §9, design 25 § 12.4): held by that
|
||||
// machine's tool runner, never by the module's own code, and the only account that publishes on that module's ledger
|
||||
// histories and snapshots. One per module assignment, whose module asks for a ledger in `requires`.
|
||||
KindLedger Kind = "ledger"
|
||||
)
|
||||
|
||||
// LedgerHistorySubject and LedgerSnapshotSubject are the subject prefixes of one ledger's history and of its instances'
|
||||
// snapshots (novox/hq ADR 0597 §1, §8, §9): an instance's subject is the prefix, ".", and the instance's path with each
|
||||
// "/" written "." (`decision/0290` is `….decision.0290`). One contract: the grants part grants a ledger account its
|
||||
// publish on them, the tool runner appends and snapshots on them, and the bus module creates the history's stream over
|
||||
// them.
|
||||
func LedgerHistorySubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".history"
|
||||
}
|
||||
|
||||
// LedgerSnapshotSubject is the subject prefix of one ledger's instances' snapshots (LedgerHistorySubject).
|
||||
func LedgerSnapshotSubject(module, local string) string {
|
||||
return "mesh.ledger." + module + "." + local + ".snapshot"
|
||||
}
|
||||
|
||||
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
|
||||
// emits (novox/hq ADR 0118, design 29 §5).
|
||||
type Seat struct {
|
||||
Name string
|
||||
// Scope is where the seat has one holder. A node-scoped seat's tool carries the node in its
|
||||
// subject, because one subject reaching six machines' holders is not an address
|
||||
// (novox/hq ADR 0132, design 33 §4). Empty reads as mesh.
|
||||
Scope string
|
||||
Accepts []string
|
||||
Emits []string
|
||||
Serves []string
|
||||
Versions []string // protocol versions served beside the current one; empty for v1 only
|
||||
|
||||
// Kinded says the seat is a kinded bench (novox/hq ADR 0234 §2, ADR 0259 §3): each holder claims one
|
||||
// kind, and its verbs' subjects carry it. Kind is the kind this principal's claim names, for a seat it
|
||||
// holds.
|
||||
Kinded bool
|
||||
Kind string
|
||||
// ByCaller are the accepts and emits whose last token names the calling module (ADR 0259 §3).
|
||||
ByCaller []string
|
||||
// Proofs are the seat's proof verbs: core request and reply, never on a stream (ADR 0259 §3).
|
||||
Proofs []string
|
||||
// Records are the holder's buckets, by their full name, each user reads under its own name.
|
||||
Records []string
|
||||
// Capabilities are what this principal's claim of a kinded bench promises (ADR 0234 §2).
|
||||
Capabilities []string
|
||||
// DeclaredBy is the module that declares the seat. On a kinded bench it alone submits work to a kind
|
||||
// and answers its proofs (novox/hq ADR 0259 §8): the router, not any user or watcher of the bench.
|
||||
DeclaredBy string
|
||||
// PulledByKind says each accepted kind is a worker of its own on the seat's queue, filtered to that kind, which
|
||||
// every holder pulls in the order Accepts lists them: the build seats' (novox/hq ADR 0525 §2, ADR 0536 §4).
|
||||
PulledByKind bool
|
||||
}
|
||||
|
||||
// A Principal is one user of the bus. Its permissions are derived from what it declares and
|
||||
// nothing else (novox/hq ADR 0043), over the three namespaces of design 29 §2: its own, the seats
|
||||
// it holds, and the seats it uses.
|
||||
type Principal struct {
|
||||
Kind Kind
|
||||
Node string
|
||||
Module string
|
||||
// Process is the process resource's id, for a KindProcess principal alone (novox/hq ADR 0436 §2):
|
||||
// Username and inbox are <node>.<module>.<process id>.
|
||||
Process string
|
||||
|
||||
// FilesFor are the nodes whose declaration-file buckets (novox/hq ADR 0486 §2) this principal may remove, for the bus
|
||||
// module's tools removing the ended controller's leftovers alone (ADR 0590 clause 6): every node of the mesh, one
|
||||
// bucket each.
|
||||
FilesFor []string
|
||||
|
||||
Emits []string
|
||||
Consumes []string
|
||||
Serves []string
|
||||
|
||||
Holds []Seat
|
||||
Uses []Seat
|
||||
// Watches are seats whose events this principal consumes. Separate from Consumes because a
|
||||
// role's event lives under the seat's namespace and not a module's, and this package cannot tell
|
||||
// a seat's name from a module's by looking at it — whoever resolved the declaration can, and
|
||||
// does (novox/hq ADR 0121).
|
||||
//
|
||||
// **Found by a consumer reading nothing.** The catalogue consumes the build machine's outcome;
|
||||
// with that name read as a module's, its subscription pointed at `mesh.mod.mesh-build-machine.…`,
|
||||
// a namespace no such module owns. Every service started and the graph stayed empty.
|
||||
Watches []Seat
|
||||
|
||||
// Carries are the modules whose tools this principal serves, for a KindNodeTools principal
|
||||
// (novox/hq ADR 0175): every module assigned to its node, as each declares itself. Its
|
||||
// serving authority is the union of theirs — each module's own tool namespace and each held
|
||||
// seat's verbs on this node — derived from the same declarations the modules' own principals
|
||||
// are, so the runtime can serve nothing a module could not have served for itself.
|
||||
Carries []Declared
|
||||
|
||||
// Invokes are the tools this principal may call, as `<module>.<tool>`; a single `*` is every
|
||||
// tool. A person's whole authority (design 25 §7), and a module's only if its manifest says so
|
||||
// (novox/hq ADR 0152) — the console's does, and nothing else's.
|
||||
//
|
||||
// **A list, not a role.** A person is not a module and holds no seat: nothing is addressed
|
||||
// to them, nothing is delivered to them, and they have no durable consumer to acknowledge.
|
||||
// What they have is permission to ask. A module that invokes gains exactly the same
|
||||
// permission and nothing beside it.
|
||||
Invokes []string
|
||||
|
||||
// AsksIssued are, for the vault's holder, the tools it asks in its own name for a credential a provider's software
|
||||
// issued, each `<module>.<tool>` (novox/hq ADR 0305, issue 723, vaultCalls): on the subject that names its caller,
|
||||
// so the tool knows the vault asks.
|
||||
AsksIssued []string
|
||||
|
||||
// AsksTheDesk says this principal is the vault's holder, which asks the desks' secret prompt in its own name
|
||||
// (novox/hq ADR 0349 clause 9, issue 723): `node-launcher.secret` on any machine, on the subject that names
|
||||
// its caller, and nothing else of that seat. With AsksIssued on the same principal, the vault's whole reach
|
||||
// outside its own seat: it asks each issuing provider's tool in its own name (clause 10).
|
||||
AsksTheDesk bool
|
||||
|
||||
// State is the local names of the state this principal's module keeps, and Reads the state of
|
||||
// others it reads as `<module>.<name>` (novox/hq ADR 0201): a bucket each, kept by the owner's
|
||||
// instances and read by whoever declares it.
|
||||
State []string
|
||||
Reads []string
|
||||
// PerMachine is the local names of its state keyed by machine, reaching the keys of Node (PerMachineKeys) and
|
||||
// no other machine's; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
|
||||
PerMachine []string
|
||||
KeyedReads []KeyedRead
|
||||
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
|
||||
Logs []string
|
||||
// ReadLogs are, of Reads, those naming another module's log (novox/hq ADR 0571 clause 1).
|
||||
ReadLogs []string
|
||||
// Streams are the streams this principal's module owns, and Hears the module streams it hears with the
|
||||
// subjects it was granted (novox/hq ADR 0363).
|
||||
Streams []ModuleStream
|
||||
Hears []Heard
|
||||
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
|
||||
// §1, §2, §6); for a KindProcess principal, its own alone.
|
||||
Queues []QueueOwner
|
||||
// Asks says a KindProcess principal reads the records of its module's own asks (novox/hq ADR 0436 §1),
|
||||
// on whichever seat of Uses keeps them (novox/hq ADR 0259 §3) — never set for any other kind.
|
||||
Asks bool
|
||||
|
||||
// SnapshotsTheBus is the bus's own module, the one holding mesh-broker (novox/hq ADR 0235). Its
|
||||
// whole authority is BusSnapshotGrants: it copies the streams for the night's backup and nothing
|
||||
// else — not its declarations, which the node's tool runtime serves for it.
|
||||
SnapshotsTheBus bool
|
||||
// Removes are, for a process or tools bundle account whose bus declares them, what it may remove from the bus
|
||||
// (novox/hq ADR 0564 Limits 4): its whole authority is then BusRemovalGrants of them.
|
||||
Removes []string
|
||||
// DeadLetterSources are, for the watchman, the streams a dead letter it keeps may come from, by name
|
||||
// (deadLetterSourcesOf, novox/hq ADR 0564 clause 5): never a bucket's.
|
||||
DeadLetterSources []string
|
||||
|
||||
// Checks are the tools a machine principal's node-engine asks as a declared health check, each
|
||||
// `<module>.<tool>` (novox/hq ADR 0240, to-be 48 §3): asked of the instance on its own machine and
|
||||
// nowhere else.
|
||||
Checks []string
|
||||
// MovesIntoTheVault is a machine principal whose node-engine may ask the vault's `move-in` in its own name (design
|
||||
// 57's V3, issue 676): set while the vault's holder serves it, so the engine can hand in the values sealed to its
|
||||
// machine, which nothing else can open.
|
||||
MovesIntoTheVault bool
|
||||
// StatesItsKeyToTheVault is a machine principal whose node-engine may ask the vault's `node-key` in its own name
|
||||
// (issue 786, novox/hq ADR 0372): set while the vault's holder serves it. That one verb, and nothing else new.
|
||||
StatesItsKeyToTheVault bool
|
||||
// AsksItsOwnSecrets is a machine principal whose node-engine may ask the vault's `own` in its own name (novox/hq
|
||||
// ADR 0547 §2, ADR 0550 §5, issue 1126): its modules' own secrets sealed to its key, for the compositions it makes
|
||||
// itself. Set while the vault's holder serves it.
|
||||
AsksItsOwnSecrets bool
|
||||
// AsksTheVaultToSign is a machine principal whose node-engine may ask the vault's `sign` in its own name, read the
|
||||
// vault's public signing halves (`signers`) and the controller's lease key (novox/hq ADR 0550 §2): its own node's
|
||||
// certificate, the keys it trusts, and the lease write that drops the controller's key. Set while the vault's holder
|
||||
// serves `sign`.
|
||||
AsksTheVaultToSign bool
|
||||
// ServedReads are, for a machine principal, the keys of mesh-config's resolved values that other nodes' modules serve
|
||||
// from (`<node>.<module>.<key>`, catalogue.ServedSettings): what every consumer is told, and so what a node composing
|
||||
// its own modules reads of a provider elsewhere (issue 1126), each key alone. Sorted, none twice.
|
||||
ServedReads []string
|
||||
// FollowsGrantsUserList is the machine principal of the node holding mesh-broker: its node-engine reads the grants
|
||||
// part's sealed user list, one key and an object by name (novox/hq ADR 0547 clause 7, GrantsUserListFollows).
|
||||
FollowsGrantsUserList bool
|
||||
// BoundBuckets are the providers' buckets of entries, `<module>_bound`, a provisioner client reads its own
|
||||
// machine's keys of (mesh-sdk go/provider, issue 679): every bucket named bound of a module the mesh places.
|
||||
BoundBuckets []string
|
||||
// CreatesStreams are the streams the grants part's principal alone may create (novox/hq ADR 0547 §5; the operator,
|
||||
// 2026-10-26 08:54): the stream of every bucket, log and stream a manifest of a module placed anywhere declares,
|
||||
// by name. Created only, never updated or deleted: a new module state is on the bus when the grants part hears its
|
||||
// manifest, not at the controller's next start (issue 1790).
|
||||
CreatesStreams []string
|
||||
// CreatesConsumers are the durable consumers the grants part's principal alone may create beside the controller
|
||||
// (novox/hq ADR 0561 §6): each placed module's on EVENTS and each process's on its work queue (AccountConsumers), by
|
||||
// stream and name. Created only, never updated or deleted.
|
||||
CreatesConsumers []ModuleConsumer
|
||||
|
||||
// Login is a node principal's login when it is not `node.<n>`: its second login, `node.<n>--b`, composed beside or
|
||||
// in place of the first as the vault's node-logins says (novox/hq ADR 0583 §1, issue 774). Each login is granted
|
||||
// the node's permissions and the node's inbox `_INBOX.node.<n>`; what it calls, it calls in its own name, so the
|
||||
// vault tells the login that confirms a rotation from the one it replaces (§3).
|
||||
Login string
|
||||
// RotatesItsLogin is a machine principal whose node-engine reads its own node's key of the vault's node-logins,
|
||||
// and no other key, and confirms a switch with node-account-confirm in its own name (novox/hq ADR 0583 §1 insight,
|
||||
// §3). Set while the vault's holder serves rotate-node.
|
||||
RotatesItsLogin bool
|
||||
|
||||
// Redeems is an enrolment principal that may call the seat mesh-enrolment's redeem in its own name, and nothing else
|
||||
// of it (novox/hq ADR 0587 §4): set while the seat's holder serves redeem. LeaseHandedAway drops the controller's
|
||||
// enrolment subject from it, which is served no more from the lease write (§6).
|
||||
Redeems bool
|
||||
LeaseHandedAway bool
|
||||
|
||||
// VaultEnrols says the vault's holder serves an enrolment verb (VerbsTheEnrolmentAsksTheVault): every principal but
|
||||
// the module mesh-enrolment and its processes is then denied them, however wide its grant (ADR 0587 §1).
|
||||
VaultEnrols bool
|
||||
|
||||
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
|
||||
// and never appears here: this file is written to a node's disk and read by a server, and a
|
||||
// secret that can be read from a configuration file is a secret with a wider blast radius
|
||||
// than the one it protects (novox/hq design 29 §10).
|
||||
PasswordHash string
|
||||
}
|
||||
|
||||
// enrolmentPrefix is the space every enrolling node's user and inbox live under, so the one place the
|
||||
// controller may answer an enrolment is derived from the same constant the user is named from.
|
||||
const enrolmentPrefix = "enrol"
|
||||
|
||||
// safeSubject refuses anything that would change the meaning of a subject rather than sit inside
|
||||
// one. A name carrying a dot would silently widen a permission by adding a token; a name carrying
|
||||
// `>` or `*` would widen it to a wildcard, which is the whole authority model gone.
|
||||
var safeSubject = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// Username is how a principal is named to the server. The node is part of it, so the same module
|
||||
// on two machines holds two users, each sealed to its own — the rule management.go already
|
||||
// applies, kept.
|
||||
func (p Principal) Username() string {
|
||||
switch p.Kind {
|
||||
case KindPerson:
|
||||
return "person." + p.Module
|
||||
case KindModule, KindNodeTools:
|
||||
// The runtime is named exactly as the module it stands for would have been: the mesh
|
||||
// issues its credential through the same path a module's takes (`module issue`), and
|
||||
// that path knows the node and the module, not the kind.
|
||||
return p.Node + "." + p.Module
|
||||
case KindProcess:
|
||||
// One account per process and machine (novox/hq ADR 0436 §2): `<machine>.<module>.<process id>`,
|
||||
// never the module's own name alone, which is a different account with a different secret.
|
||||
return p.Node + "." + p.Module + "." + p.Process
|
||||
case KindNode:
|
||||
if p.Login != "" {
|
||||
return p.Login
|
||||
}
|
||||
return "node." + p.Node
|
||||
case KindLedger:
|
||||
// `ledger.<machine>.<module>`: never `<node>.<module>`, which is the module's own account or the tool runner's,
|
||||
// so the module's code never holds it.
|
||||
return "ledger." + p.Node + "." + p.Module
|
||||
case KindProvisioner:
|
||||
// Named for the machine, as a node is: one client per machine, whichever module runs it, and never
|
||||
// `<node>.<module>`, which is the module's own account or the tool runner's.
|
||||
return "provisioner." + p.Node
|
||||
case KindEnrolment:
|
||||
// Per token, not one shared user. **The inbox is the reason**: with a single `enrolment`
|
||||
// user every machine enrolling at once could read every other's answer, and an answer
|
||||
// carries that node's credentials sealed to it. Design 25 §6 says the inbox a token
|
||||
// derives, and a permission belongs to a user, so the user is per token.
|
||||
//
|
||||
// Named after the node, which **is** the token's id: a token is issued for a node record,
|
||||
// the mesh holds one live claim per record, and the node's name is the one identifier both
|
||||
// sides already have before anything else is agreed. It is also exactly what the other
|
||||
// transport does, where the account is named after the node and the secret is its password.
|
||||
return enrolmentPrefix + "." + p.Node
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// inbox is a principal's own reply space. No user is ever granted a bare `_INBOX.>` (design 25
|
||||
// §4): with one account, inbox privacy is the permission list or it is nothing, so each user's
|
||||
// inbox is derived from its own identity and its permissions name that prefix and no other.
|
||||
//
|
||||
// A node's two logins share the node's inbox (novox/hq ADR 0583 §1): its answers reach it whichever login it dials as.
|
||||
func (p Principal) inbox() string {
|
||||
if p.Kind == KindNode {
|
||||
return "_INBOX.node." + p.Node + ".>"
|
||||
}
|
||||
return "_INBOX." + p.Username() + ".>"
|
||||
}
|
||||
|
||||
// ResponseTTL is how long the bus lets a principal answer a request it received. Its one answer has
|
||||
// to come inside this, and a seat's holder answers within link.AnswerWithin — inside it by design.
|
||||
// **A broker reloading its user list forgets every answer it was about to permit**, whatever this
|
||||
// says (novox/hq issue 265): a call that is still running when the list reloads has its answer
|
||||
// refused, which is why a holder answers before it does what can reload it.
|
||||
const ResponseTTL = time.Minute
|
||||
|
||||
// GrantsModule is the one module whose principal publishes the modules' memberships (novox/hq ADR 0547 §5): the grants
|
||||
// part, which composes them from the manifests and placement. It defines no stream and is granted no other subject here.
|
||||
const GrantsModule = "mesh-grants"
|
||||
|
||||
// The grants part's state memberships (novox/hq ADR 0571 clause 7): one key per assignment, `<node>.<module>`, its
|
||||
// membership (Membership) bare, as the assignments stream carries it, written only when it differs. A module's runtime
|
||||
// reads its own key, the tool runner its node's; the grants part alone writes it. Both sides go through
|
||||
// membership.EncodeGrantsMembership and membership.DecodeGrantsMembership (or BareGrantsMembership for a reader with a
|
||||
// type of its own), never json of a type of their own: the grants part once wrote its wrapper here and every reader
|
||||
// served nothing (issue 774). The decoder still reads that wrapper, for the transition.
|
||||
const (
|
||||
GrantsMembershipsState = "memberships"
|
||||
GrantsMembershipsBucket = GrantsModule + "_" + GrantsMembershipsState
|
||||
)
|
||||
|
||||
// MembershipKey is an assignment's key in the grants part's memberships: `<node>.<module>`.
|
||||
func MembershipKey(node, module string) string { return node + "." + module }
|
||||
@@ -1,8 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"git.novox.be/novox/mesh-sdk/go/membership"
|
||||
)
|
||||
|
||||
// AssignmentsStream holds every assignment's membership, the newest per subject.
|
||||
const AssignmentsStream = membership.AssignmentsStream
|
||||
@@ -1,7 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// isNewTraffic says whether a seat carries any of the rules above, so a seat that carries none is
|
||||
// composed exactly as before them.
|
||||
func (s Seat) isNewTraffic() bool {
|
||||
return s.Kinded || len(s.ByCaller) > 0 || len(s.Proofs) > 0 || len(s.Records) > 0 || s.PulledByKind
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
|
||||
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
|
||||
const (
|
||||
StateMaxValueBytes = 256 * 1024
|
||||
)
|
||||
|
||||
// A Bucket is one module's declared state as the bus holds it.
|
||||
type Bucket struct {
|
||||
Module string
|
||||
Name string
|
||||
// History is how many values a key keeps; zero is one.
|
||||
History int
|
||||
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
|
||||
TTLSeconds int
|
||||
// PerMachine says each machine's instance writes only the keys of its machine (novox/hq ADR 0260; design 46
|
||||
// 7-approval-mark §13): the key named for its machine and the keys below it, PerMachineKeys, and no other machine's.
|
||||
PerMachine bool
|
||||
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
|
||||
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
|
||||
KeyTTL bool
|
||||
// MaxMiB is the bucket's cap in MiB (design 47 G §469); zero is no cap.
|
||||
MaxMiB int
|
||||
}
|
||||
|
||||
// MaxBytes is the bucket's cap in bytes as the bus's maximum size: -1, unbounded, where it names none.
|
||||
func (b Bucket) MaxBytes() int64 {
|
||||
if b.MaxMiB <= 0 {
|
||||
return -1
|
||||
}
|
||||
return int64(b.MaxMiB) * 1024 * 1024
|
||||
}
|
||||
|
||||
// PerMachineKeys are the keys a machine's instance of a per-machine state reaches (design 46 7-approval-mark §13): the key
|
||||
// named for its machine, as every module's per-machine key keeps its form, and the keys below it, `<node>.>`; no other
|
||||
// machine's. A membership's per-machine state (membership.StateIssued's Key) reaches the same.
|
||||
func PerMachineKeys(node string) []string { return []string{node, node + ".>"} }
|
||||
|
||||
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
|
||||
// underscore, which neither may contain, so two modules can never derive one bucket.
|
||||
func BucketName(module, name string) string { return module + "_" + name }
|
||||
|
||||
// Bucket is this bucket's name on the bus.
|
||||
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
|
||||
|
||||
// Why is carried into the server's description of the bucket, so somebody reading the server's
|
||||
// own state finds whose it is and why it is kept.
|
||||
func (b Bucket) Why() string {
|
||||
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
|
||||
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
|
||||
b.Module, b.Name, b.Module, b.Module)
|
||||
}
|
||||
|
||||
// KeyedRead is one key of another module's state a principal reads, and no other key of it.
|
||||
type KeyedRead struct {
|
||||
Read string `json:"read"`
|
||||
Key string `json:"key"`
|
||||
}
|
||||
@@ -1,196 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Retention is how a stream decides what to keep, which is the whole of what distinguishes the
|
||||
// mesh's four relationships on the wire (design 29 §4).
|
||||
type Retention string
|
||||
|
||||
const (
|
||||
// RetentionWorkQueue: a message is removed once a consumer acknowledges it. Exactly one
|
||||
// worker does the work, and a worker that dies has its message redelivered.
|
||||
RetentionWorkQueue Retention = "workqueue"
|
||||
// RetentionLastPerSubject: only the newest message on each subject survives. This is the
|
||||
// state shape — a node that was away gets exactly the current declaration and nothing older.
|
||||
RetentionLastPerSubject Retention = "last_per_subject"
|
||||
// RetentionLimits: kept until it ages or the stream fills. Events, where a subscriber that
|
||||
// was down catches up and nobody is obliged to act.
|
||||
RetentionLimits Retention = "limits"
|
||||
)
|
||||
|
||||
// A Stream is one of the mesh's own, as the controller asserts it.
|
||||
type Stream struct {
|
||||
Name string
|
||||
Subjects []string
|
||||
Retention Retention
|
||||
// MaxAge in seconds, zero for unbounded. Per stream — JetStream has no per-subject age,
|
||||
// which is why differing retention between modules would mean a stream each.
|
||||
MaxAge int
|
||||
// MaxMsgsPerSubject caps each subject independently, so one noisy emitter cannot push
|
||||
// another's events out of a shared stream. Verified: with a cap of 3, ten messages on one
|
||||
// subject and one on another leave four in the stream, not three.
|
||||
MaxMsgsPerSubject int
|
||||
// Why is carried into the assertion so an operator reading the server's own state finds the
|
||||
// reason there, rather than only in a repository they may not have.
|
||||
Why string
|
||||
// Direct lets a client read a subject's last message without a consumer, which is how a
|
||||
// runtime reads its own membership with no JetStream API beyond one request (ADR 0160).
|
||||
Direct bool
|
||||
// MaxBytes bounds the stream's size, zero for unbounded. With DiscardNew a full stream refuses
|
||||
// what comes next rather than dropping what it holds: the publisher is told, and says so.
|
||||
MaxBytes int64
|
||||
DiscardNew bool
|
||||
// DuplicatesSeconds is the window in which a message id published twice is kept once; zero for the
|
||||
// server's default (two minutes).
|
||||
DuplicatesSeconds int
|
||||
// AllowMsgTTL lets a message carry a time to live of its own (`Nats-TTL`), for what must not outlive the
|
||||
// stream it came from (novox/hq ADR 0376 §3, §5).
|
||||
AllowMsgTTL bool
|
||||
// Restricted marks a module's stream restricted by a grant (novox/hq ADR 0363 §4), in the stream's own
|
||||
// metadata, so whoever keeps one of its messages reads it from the bus (ADR 0376 §3).
|
||||
Restricted bool
|
||||
// WorkQueue marks a module stream declared retention: work-queue (novox/hq ADR 0436 §6), in the
|
||||
// stream's own metadata, so a dead letter kept from it is read as drop-only from the bus, by the
|
||||
// declaration and not by the bus's work-queue retention alone — which SEAT_* and other mesh-internal
|
||||
// streams share without being one of ADR 0436's, and whose dead letters are still delivered again
|
||||
// (ADR 0264 §4).
|
||||
WorkQueue bool
|
||||
}
|
||||
|
||||
// RestrictedMetadata is the key a restricted stream's metadata says it under.
|
||||
const RestrictedMetadata = "mesh-restricted"
|
||||
|
||||
// WorkQueueMetadata is the key a module stream declared retention: work-queue (novox/hq ADR 0436 §6) says
|
||||
// it under: never set by a mesh-internal work-queue stream (a seat's, a check's), only by Stream.WorkQueue.
|
||||
const WorkQueueMetadata = "mesh-work-queue"
|
||||
|
||||
// What a durable consumer gave up on is kept (novox/hq issue 330, design 25 §3).
|
||||
//
|
||||
// **The server says it and keeps it; the controller copies it.** A consumer that handed a message over
|
||||
// as often as it may stops offering it and publishes `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES` with
|
||||
// the stream and the message's sequence. DeadLetterNoticesStream captures those advisories as the
|
||||
// server publishes them, so one said while no controller listens is still there when one starts. The
|
||||
// controller's consumer on it fetches the given-up message by its sequence, while the source stream
|
||||
// still holds it, and keeps a copy in DeadLettersStream under DeadLetterSubject, with the consumer,
|
||||
// the subject, how often it was handed over and when it was given up. It stays there until a person
|
||||
// delivers it again or drops it, with why; a condition is open for as long as it does.
|
||||
const (
|
||||
DeadLetterNoticesStream = "DEAD_LETTER_NOTICES"
|
||||
DeadLettersStream = "DEAD_LETTERS"
|
||||
// MaxDeliveriesAdvisories is the subject the server says a given-up message on.
|
||||
MaxDeliveriesAdvisories = "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>"
|
||||
deadLetterPrefix = "mesh.events.dead."
|
||||
againPrefix = "mesh.again."
|
||||
// DeadLettersBytes bounds the kept copies. Full, the stream refuses the next copy, which is said
|
||||
// as the consumer's condition; it never drops one it holds.
|
||||
DeadLettersBytes = 256 << 20
|
||||
)
|
||||
|
||||
// DeadLetterSubject is where a message one consumer gave up on is kept: `mesh.events.dead.<stream>.<consumer>`.
|
||||
func DeadLetterSubject(stream, consumer string) string {
|
||||
return deadLetterPrefix + stream + "." + consumer
|
||||
}
|
||||
|
||||
// DeadLetterOf is the stream and consumer a kept message's subject names; false for any other subject.
|
||||
func DeadLetterOf(subject string) (stream, consumer string, ok bool) {
|
||||
rest, found := strings.CutPrefix(subject, deadLetterPrefix)
|
||||
if !found {
|
||||
return "", "", false
|
||||
}
|
||||
stream, consumer, ok = strings.Cut(rest, ".")
|
||||
return stream, consumer, ok && stream != "" && consumer != "" && !strings.Contains(consumer, ".")
|
||||
}
|
||||
|
||||
// MeshStreams is the foundation set, in the order a person reads it.
|
||||
//
|
||||
// **EVENTS filters on the `event` token**, which is the reason that token exists. A module's
|
||||
// namespace carries both its events and its tool calls; a filter of `mesh.mod.*.>` would persist
|
||||
// every tool invocation in the mesh, and a tool call must never be persisted (design 25 §3 keeps
|
||||
// tools on core NATS, where a lost call is a timeout the caller already handles).
|
||||
func MeshStreams() []Stream {
|
||||
return []Stream{
|
||||
// CONTROL, the ended controller's work queue of what nodes reported, is no longer the mesh's (novox/hq ADR 0590
|
||||
// clause 6): not declared here, which would make it again; the bus module's tools remove it.
|
||||
{
|
||||
Name: "NODES",
|
||||
Subjects: []string{"mesh.node.*.declare"},
|
||||
Retention: RetentionLastPerSubject,
|
||||
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
|
||||
"n-1 by construction (issue 107)",
|
||||
},
|
||||
// The assignments stream (ADR 0160) is no longer the mesh's: a runtime reads its membership from the grants
|
||||
// part's state memberships (ADR 0571 clause 7), and the bus module's tools delete the stream at the lease write
|
||||
// (research 072 row 33), so it is not declared here, which would make it again.
|
||||
{
|
||||
Name: EventsStream,
|
||||
// A seat's own events ride here too: they are 1:many like any event, and the
|
||||
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
|
||||
// tools (`tool`), which must not be persisted.
|
||||
// And an event given up on, delivered again to the one consumer that gave it up
|
||||
// (novox/hq issue 330): under `mesh.again.<consumer>.`, which only that consumer filters.
|
||||
Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>", againPrefix + ">"},
|
||||
Retention: RetentionLimits,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
MaxMsgsPerSubject: 10000,
|
||||
// A restricted dead letter delivered again keeps what is left of its stream's age (ADR 0376 §5).
|
||||
AllowMsgTTL: true,
|
||||
Why: "a subscriber that was down catches up; tool traffic under the same prefix is " +
|
||||
"excluded by the event token; per-subject caps keep a noisy emitter from " +
|
||||
"evicting a quiet one without splitting the stream",
|
||||
},
|
||||
{
|
||||
Name: DeadLetterNoticesStream,
|
||||
Subjects: []string{MaxDeliveriesAdvisories},
|
||||
Retention: RetentionWorkQueue,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
Why: "the server's word that a consumer gave up on a message, kept until the controller has " +
|
||||
"copied the message into DEAD_LETTERS (novox/hq issue 330); a week, the longest the source " +
|
||||
"streams keep what they are about",
|
||||
},
|
||||
{
|
||||
Name: DeadLettersStream,
|
||||
Subjects: []string{deadLetterPrefix + ">"},
|
||||
Retention: RetentionLimits,
|
||||
MaxBytes: DeadLettersBytes,
|
||||
DiscardNew: true,
|
||||
DuplicatesSeconds: 24 * 60 * 60,
|
||||
AllowMsgTTL: true,
|
||||
Why: "every message a consumer gave up on, with its consumer, subject, deliveries and when, kept " +
|
||||
"until a person delivers it again or drops it with why (novox/hq issue 330); no age but a restricted " +
|
||||
"stream's message, which ages out with its stream (ADR 0376), and full it refuses the next copy " +
|
||||
"rather than drop one it holds",
|
||||
},
|
||||
{
|
||||
Name: ProvAppliedStream,
|
||||
Subjects: []string{ProvAppliedSubjects},
|
||||
Retention: RetentionLastPerSubject,
|
||||
Direct: true,
|
||||
MaxAge: 7 * 24 * 60 * 60,
|
||||
Why: "each machine's provisioner client's newest word on each module's files, applied, waiting or " +
|
||||
"failed with the reason (novox/hq ADR 0349 §4): what a walk's end and a gate wait for (ADR 0362, " +
|
||||
"ADR 0349 §15), read by subject; a week, as the events are",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// ProvAppliedStream keeps what each machine's provisioner client last said of each module's files, newest per
|
||||
// subject (novox/hq ADR 0349 §4, ADR 0362, design 57 P6): `prov.applied.<node>.<module>` lies outside `mesh.*`, so
|
||||
// no other stream captures it, and a report said while no controller listened is still there to be read.
|
||||
const ProvAppliedStream = "PROV_APPLIED"
|
||||
|
||||
// ProvAppliedSubjects is every machine's client's reports: `prov.applied.<node>.<module>`, a module's name dots and
|
||||
// all (ProvisionerAppliedSubject).
|
||||
const ProvAppliedSubjects = "prov.applied.>"
|
||||
|
||||
// The ended controller's consumers' names (novox/hq ADR 0590 clause 6): kept only so its leftovers are named for their
|
||||
// removal (ControllerLeftoverConsumers) and the watchman can read whether `controller` on DEAD_LETTER_NOTICES still
|
||||
// stands.
|
||||
const (
|
||||
ControllerName = "controller"
|
||||
MergesConsumerName = "controller-merges"
|
||||
)
|
||||
|
||||
// EventsStream holds every module's and every role's events, a build's log among them.
|
||||
const EventsStream = "EVENTS"
|
||||
@@ -1,99 +0,0 @@
|
||||
package buscompose
|
||||
|
||||
// Declared is one module on one node, as composing its authority needs it.
|
||||
type Declared struct {
|
||||
Module string
|
||||
Emits []string
|
||||
Consumes []string
|
||||
Serves []string
|
||||
// Served are the config keys its served values and zone name fill from its settings (catalogue.ServedSettings): what
|
||||
// every node may read of its resolved values where it runs (Principal.ServedReads, issue 1126).
|
||||
Served []string
|
||||
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
|
||||
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
|
||||
// which is right: those seats are about who does a job, not about who may say what.
|
||||
Holds []Seat
|
||||
// Uses are the seats this module sends to.
|
||||
Uses []Seat
|
||||
// Issued are the tools that answer a credential its software issued, one per offer that says so (novox/hq
|
||||
// ADR 0305): the controller is granted to ask each in its own name, and no other tool so.
|
||||
Issued []string
|
||||
// Watches are the seats whose events it consumes.
|
||||
Watches []Seat
|
||||
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
|
||||
Invokes []string
|
||||
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
|
||||
State []Bucket
|
||||
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
|
||||
Logs []Log
|
||||
// Streams are the streams it owns, each its events' one place (novox/hq ADR 0363).
|
||||
Streams []ModuleStream
|
||||
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
|
||||
// §1, §2, §6): one durable consumer per entry, named `<node>_<module>_<process id>`.
|
||||
Queues []QueueOwner
|
||||
// Hears are the module streams it hears on this machine, each with the subjects it was granted
|
||||
// (novox/hq ADR 0363 §5); HearsAStream says its `consumes` names a module that owns a stream, granted
|
||||
// or not, so its membership is issued again when a grant changes.
|
||||
Hears []Heard
|
||||
HearsAStream bool
|
||||
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
|
||||
Reads []string
|
||||
// ReadsRefused are the reads its manifest names that the owner's announcement does not make readable (novox/hq ADR
|
||||
// 0495 clause 3, issue 1790): granted nothing, kept to be said.
|
||||
ReadsRefused []string
|
||||
// ReadLogs are, of its reads, those naming another module's log (`<module>.<log>`, novox/hq ADR 0571 clause 1): filled
|
||||
// by Users and PlacementsOf from the logs the modules placed anywhere declare, never by a caller.
|
||||
ReadLogs []string
|
||||
// KeyedReads are keys of other modules' state it reads, each one key alone: what a seat's holder is
|
||||
// granted for the pieces the modules beside it offer (novox/hq ADR 0260).
|
||||
KeyedReads []KeyedRead
|
||||
// NoAccount says the module declares no own secret named broker, so no account could ever be
|
||||
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
|
||||
// record that does not say is composed as it always was.
|
||||
NoAccount bool
|
||||
// SnapshotsTheBus says the module holds mesh-broker — it is the bus — and so is the one module
|
||||
// granted the snapshot API, to copy the bus's streams for the night's backup (novox/hq ADR 0235).
|
||||
SnapshotsTheBus bool
|
||||
// Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be
|
||||
// 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more.
|
||||
Checks []string
|
||||
// Loads says the module has a compiled bundle whose entrypoints the machine's tool runtime loads and runs
|
||||
// (novox/hq ADR 0175; issue 1329): a helper that reaches the bus on the runner's account, so the runner
|
||||
// must carry the module although it declares no tools.
|
||||
Loads bool
|
||||
// RunsAs is the account the module runs as in a runtime of its own (novox/hq ADR 0259 §8): it is never
|
||||
// carried by the machine's runtime, and reaches the bus on its own account.
|
||||
RunsAs string
|
||||
// ProvisionsNode says the module runs the machine's provisioner client: it claims NodeProvisionerSeat and
|
||||
// declares the own secret ProvisionerSecret its account is delivered in (novox/hq ADR 0349 clause 11, design 57
|
||||
// P3). The client is given a bus account of its own, never the module's or the tool runner's.
|
||||
ProvisionsNode bool
|
||||
// Processes are this module's own processes that declare bus (novox/hq ADR 0436 §1), each getting a
|
||||
// principal of its own: granted exactly what its bus says, and nothing of this module's other grants —
|
||||
// composed whether or not the module itself has an account (NoAccount), because a process account is
|
||||
// delivered in its own secret, broker-<process id>, never the module's `broker`.
|
||||
Processes []ProcessPrincipal
|
||||
}
|
||||
|
||||
// ProcessPrincipal is one process of a module that declares bus (novox/hq ADR 0436 §1, §2), as the broker
|
||||
// composes its own account from: its own emits, its own work-queue consumers, its own state, whether it
|
||||
// reads the router's records of its module's own asks, and the tools and seat verbs it calls.
|
||||
type ProcessPrincipal struct {
|
||||
// Process is the process resource's id, naming the account <node>.<module>.<process id>.
|
||||
Process string
|
||||
// Emits are the events this process may publish, named as the module's own Emits are.
|
||||
Emits []string
|
||||
// Queues are the module's own work-queue streams this process is the one consumer of.
|
||||
Queues []QueueOwner
|
||||
// Removes are what its account may remove from the bus, as its bus declares them (the indexer's judge refuses any other; novox/hq
|
||||
// ADR 0564 Limits 4): a tools bundle's account, whose whole authority is then BusRemovalGrants.
|
||||
Removes []string
|
||||
// State is the module's own state, by local name, this process reads and writes.
|
||||
State []string
|
||||
// Asks says whether this process reads the records of its module's own asks (novox/hq ADR 0259 §3), on
|
||||
// whichever seats the module uses that keep them (the router's, operator-channel, among others).
|
||||
Asks bool
|
||||
// Invokes are the tools and seat verbs this process calls, each <module>.<tool> or a role's
|
||||
// seat:<seat>.<verb>, or * for every tool on the mesh.
|
||||
Invokes []string
|
||||
}
|
||||
@@ -1,159 +0,0 @@
|
||||
// Package busretry waits out a bus that answers nothing for a few seconds (issue 774, the sweep of 2026-10-30: bus
|
||||
// blackouts at every user-list reload). nats-server 2.11's reloadAuthorization rebuilds the accounts, and for a few
|
||||
// seconds the mesh's account has none of its imports, $JS.API.> among them: every request is answered "no responders"
|
||||
// and every JetStream call "jetstream not enabled". A write or a call made in that window failed, and was lost.
|
||||
//
|
||||
// Only an error that says the request reached no one is waited out (Undelivered): nothing was done, so asking again
|
||||
// is safe even for a write that is not idempotent. A timeout, or any answer, is never asked again: the request may
|
||||
// have been delivered. The wait is bounded (Bound, about 30 s, backing off), and said on standard error when it ends
|
||||
// without an answer.
|
||||
//
|
||||
// DoRepeatable is the form for a call its caller keeps the answer of and marks safe to make again (a read, or a write
|
||||
// carrying its own message id): it also asks again on a timeout, and its blackout probe is optional (novox/hq ADR 0668,
|
||||
// extending ADR 0615 §6 to every kept call; issue 1966).
|
||||
package busretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// Bound is how long a request that reached no one is asked again; FirstWait the first pause, doubled to MostWait.
|
||||
var (
|
||||
Bound = 30 * time.Second
|
||||
FirstWait = 250 * time.Millisecond
|
||||
MostWait = 4 * time.Second
|
||||
)
|
||||
|
||||
// Waits sets Bound, FirstWait and MostWait and returns what puts the earlier values back. They are state every caller
|
||||
// in a process shares, so a test sets them only through it, as t.Cleanup(busretry.Waits(...)): a test that set them
|
||||
// and left them set shortened the next test's bound to milliseconds, and a loaded check machine overran it (the check
|
||||
// of release sdk1, 2026-11-02). A bound a test waits on to succeed is seconds: no slow machine reaches it.
|
||||
func Waits(bound, first, most time.Duration) (restore func()) {
|
||||
was := [3]time.Duration{Bound, FirstWait, MostWait}
|
||||
Bound, FirstWait, MostWait = bound, first, most
|
||||
return func() { Bound, FirstWait, MostWait = was[0], was[1], was[2] }
|
||||
}
|
||||
|
||||
// Undelivered says err means the request reached no one: no responders, JetStream not enabled (for the account), or
|
||||
// no response from the stream after the client's own retries on no responders (either JetStream client's: the old
|
||||
// one's publish says it for no responders, review round 1 of issue 1966); also said in words by a runtime that
|
||||
// passed the error on.
|
||||
func Undelivered(err error) bool {
|
||||
var answered *Answered
|
||||
if err == nil || errors.As(err, &answered) {
|
||||
return false
|
||||
}
|
||||
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, jetstream.ErrJetStreamNotEnabled) ||
|
||||
errors.Is(err, jetstream.ErrJetStreamNotEnabledForAccount) || errors.Is(err, jetstream.ErrNoStreamResponse) ||
|
||||
errors.Is(err, nats.ErrNoStreamResponse) ||
|
||||
errors.Is(err, nats.ErrJetStreamNotEnabled) || errors.Is(err, nats.ErrJetStreamNotEnabledForAccount) {
|
||||
return true
|
||||
}
|
||||
words := strings.ToLower(err.Error())
|
||||
return strings.Contains(words, "no responders") || strings.Contains(words, "jetstream not enabled")
|
||||
}
|
||||
|
||||
// Answered is an error a callee answered with: the request was delivered, whatever its words say ("jetstream not
|
||||
// enabled" in a callee's own answer), so Undelivered never counts it and Do never asks it again.
|
||||
type Answered struct{ Words string }
|
||||
|
||||
func (a *Answered) Error() string { return a.Words }
|
||||
|
||||
// Do runs try, and again while its error says the request reached no one, until Bound has passed. blackout, when
|
||||
// given, is asked after each such error and Do stops waiting when it says the bus is not blacked out: a call nobody
|
||||
// answers because its callee is not running fails at once, as before. what names the request in the error.
|
||||
func Do[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
|
||||
return do(what, try, func(err error) bool { return Undelivered(err) && (blackout == nil || blackout()) })
|
||||
}
|
||||
|
||||
// DoRepeatable is Do for a call the caller marks safe to make again (a read, or a write carrying its message id, which
|
||||
// the callee deduplicates): a timeout is asked again too, and with no blackout probe a request that reached no one is
|
||||
// asked again whether or not the bus is blacked out, since a callee that is restarting answers nothing for seconds
|
||||
// (novox/hq ADR 0668; issue 1966). Past Bound the error says which fault it last met.
|
||||
func DoRepeatable[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
|
||||
return do(what, try, func(err error) bool {
|
||||
return TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))
|
||||
})
|
||||
}
|
||||
|
||||
// Open is what an asker passes a kept call beside the ask's key (design 47 G §351, §425, §460): Open reads the asker's own
|
||||
// state and says whether the key is still open; Replaced, when the asker has one, says an event that replaces the ask was
|
||||
// heard. The ask is made again only while Open answers yes and Replaced says no. An error of Open is written to Logf
|
||||
// (standard error when nil), naming the key, and the ask is made again: a kept call is one the callee takes once by a
|
||||
// key, so asking again is safe and not asking would be silent. Open is always given.
|
||||
type Open struct {
|
||||
Key string
|
||||
Open func(key string) (bool, error)
|
||||
Replaced func() bool
|
||||
Logf func(format string, args ...any)
|
||||
}
|
||||
|
||||
func (o Open) stillOpen() bool {
|
||||
if o.Replaced != nil && o.Replaced() {
|
||||
return false
|
||||
}
|
||||
open, err := o.Open(o.Key)
|
||||
if err != nil {
|
||||
logf := o.Logf
|
||||
if logf == nil {
|
||||
logf = func(f string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-sdk] "+f+"\n", a...) }
|
||||
}
|
||||
logf("busretry: whether the ask %q is still open could not be read, so it is asked again: %v", o.Key, err)
|
||||
return true
|
||||
}
|
||||
return open
|
||||
}
|
||||
|
||||
// DoKept is DoRepeatable for a kept call that is asked again only while its key is open and not replaced (Open).
|
||||
func DoKept[T any](what string, o Open, try func() (T, error), blackout func() bool) (T, error) {
|
||||
if o.Open == nil {
|
||||
var zero T
|
||||
return zero, fmt.Errorf("%s: a kept call names its key's open function", what)
|
||||
}
|
||||
return do(what, try, func(err error) bool {
|
||||
return (TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))) && o.stillOpen()
|
||||
})
|
||||
}
|
||||
|
||||
// TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a
|
||||
// call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts.
|
||||
func TimedOut(err error) bool {
|
||||
var answered *Answered
|
||||
if err == nil || errors.As(err, &answered) {
|
||||
return false
|
||||
}
|
||||
return errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout)
|
||||
}
|
||||
|
||||
func do[T any](what string, try func() (T, error), again func(error) bool) (T, error) {
|
||||
v, err := try()
|
||||
if !again(err) {
|
||||
return v, err
|
||||
}
|
||||
start, wait := time.Now(), FirstWait
|
||||
for time.Since(start) < Bound {
|
||||
time.Sleep(wait)
|
||||
wait = min(wait*2, MostWait)
|
||||
v, err = try()
|
||||
if !again(err) {
|
||||
return v, err
|
||||
}
|
||||
}
|
||||
if TimedOut(err) {
|
||||
err = fmt.Errorf("%s: no answer within its deadline, asked again for %s: %w", what, time.Since(start).Round(time.Second), err)
|
||||
} else {
|
||||
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
|
||||
"account's imports for a few seconds, a callee that restarts answers nothing; this was longer): %w", what,
|
||||
time.Since(start).Round(time.Second), err)
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "[mesh-sdk] %v\n", err)
|
||||
return v, err
|
||||
}
|
||||
@@ -1,117 +0,0 @@
|
||||
package busretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the
|
||||
// bound; a blackout probe that says the bus answers ends the wait at once.
|
||||
func subABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
errs []error // what each try answers; the last repeats
|
||||
blackout func() bool
|
||||
tries int // at least
|
||||
atMost int
|
||||
delivered bool
|
||||
repeatable bool // DoRepeatable, a call marked safe to make again (ADR 0668)
|
||||
}{
|
||||
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true, false},
|
||||
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true, false},
|
||||
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true, false},
|
||||
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false, false},
|
||||
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false, false},
|
||||
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, false},
|
||||
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false, false},
|
||||
// Issue 1966, ADR 0668: a call kept and safe to make again is asked again on a timeout, and on no responders
|
||||
// from a callee that restarts on a bus that answers; Do still asks neither.
|
||||
{"repeatable: timed out twice, then an answer", []error{context.DeadlineExceeded, nats.ErrTimeout, nil}, nil, 3, 3, true, true},
|
||||
{"repeatable: no responders with no probe, then an answer", []error{nats.ErrNoResponders, nil}, nil, 2, 2, true, true},
|
||||
{"repeatable: a callee's answer is an answer", []error{&Answered{Words: "context deadline exceeded"}}, nil, 1, 1, false, true},
|
||||
{"repeatable: nobody answers and the probe says the bus answers", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, true},
|
||||
{"repeatable: no response from the stream, either client's, then an answer", []error{nats.ErrNoStreamResponse, jetstream.ErrNoStreamResponse, nil}, nil, 3, 3, true, true},
|
||||
{"timed out past the bound: said", []error{context.DeadlineExceeded}, nil, 2, 1000, false, true},
|
||||
} {
|
||||
// A row that ends in an answer waits on a bound of seconds, which no slow machine reaches; only the row that
|
||||
// runs past it has a short one, and asks only that it was asked again at least once.
|
||||
bound := 10 * time.Second
|
||||
if strings.Contains(row.name, "past the bound") {
|
||||
bound = 100 * time.Millisecond
|
||||
}
|
||||
t.Cleanup(Waits(bound, time.Millisecond, 4*time.Millisecond))
|
||||
tries := 0
|
||||
do := Do[int]
|
||||
if row.repeatable {
|
||||
do = DoRepeatable[int]
|
||||
}
|
||||
_, err := do("the test", func() (int, error) {
|
||||
e := row.errs[min(tries, len(row.errs)-1)]
|
||||
tries++
|
||||
return tries, e
|
||||
}, row.blackout)
|
||||
if tries < row.tries || tries > row.atMost || (err == nil) != row.delivered {
|
||||
t.Errorf("%s: %d tries, %v", row.name, tries, err)
|
||||
}
|
||||
if strings.HasPrefix(row.name, "blacked out") && (!Undelivered(err) || !strings.Contains(fmt.Sprint(err), "the bus answered nothing")) {
|
||||
t.Errorf("%s: %v", row.name, err)
|
||||
}
|
||||
if strings.HasPrefix(row.name, "timed out") && (!TimedOut(err) || !strings.Contains(fmt.Sprint(err), "the test: no answer within its deadline")) {
|
||||
t.Errorf("%s: %v", row.name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Design 47 G §351, §460: a kept call is asked again only while its key is open and not replaced; an error of `open` is
|
||||
// logged naming the key and the ask is made again.
|
||||
func subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
name string
|
||||
open func(n int) (bool, error) // by the number of asks made so far
|
||||
replaced bool
|
||||
tries int
|
||||
logged bool
|
||||
}{
|
||||
{"open throughout, then answered", func(int) (bool, error) { return true, nil }, false, 3, false},
|
||||
{"closed after the first ask: not asked again", func(int) (bool, error) { return false, nil }, false, 1, false},
|
||||
{"replaced: not asked again", func(int) (bool, error) { return true, nil }, true, 1, false},
|
||||
{"open errs: logged and asked again", func(int) (bool, error) { return false, errors.New("state unreadable") }, false, 3, true},
|
||||
} {
|
||||
t.Cleanup(Waits(10*time.Second, time.Millisecond, 4*time.Millisecond))
|
||||
tries, logs := 0, []string{}
|
||||
o := Open{Key: "ask-7", Open: func(k string) (bool, error) {
|
||||
if k != "ask-7" {
|
||||
t.Errorf("%s: open asked about %q", row.name, k)
|
||||
}
|
||||
return row.open(tries)
|
||||
}, Replaced: func() bool { return row.replaced },
|
||||
Logf: func(f string, a ...any) { logs = append(logs, fmt.Sprintf(f, a...)) }}
|
||||
_, err := DoKept("the test", o, func() (int, error) {
|
||||
tries++
|
||||
if tries < 3 {
|
||||
return 0, nats.ErrNoResponders
|
||||
}
|
||||
return tries, nil
|
||||
}, nil)
|
||||
if tries != row.tries || (row.tries == 3) != (err == nil) || (len(logs) > 0) != row.logged ||
|
||||
(row.logged && !strings.Contains(logs[0], "ask-7")) {
|
||||
t.Errorf("%s: %d asks, err %v, logs %v", row.name, tries, err, logs)
|
||||
}
|
||||
}
|
||||
if _, err := DoKept("x", Open{Key: "k"}, func() (int, error) { return 0, nil }, nil); err == nil {
|
||||
t.Errorf("a kept call with no open function was made")
|
||||
}
|
||||
}
|
||||
|
||||
// TestABusThatAnswersNothingIsWaitedOut runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
|
||||
t.Run("ABusThatAnswersNothingIsWaitedOut", subABusThatAnswersNothingIsWaitedOut)
|
||||
t.Run("AKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen", subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen)
|
||||
}
|
||||
@@ -1,158 +0,0 @@
|
||||
package bussnap
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/busarchive"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// The keeper's restore in place of a snapshot it took (design 47 G §681): each stream of the file put back under its own
|
||||
// name, only where no stream of that name is live.
|
||||
|
||||
// APIStreamDelete is the subject the keeper removes its own partial stream with.
|
||||
const APIStreamDelete = "$JS.API.STREAM.DELETE."
|
||||
|
||||
const restoreChunk = 64 * 1024
|
||||
|
||||
// ManifestOf reads the manifest of a snapshot file: the streams it holds, without unpacking them.
|
||||
func ManifestOf(path string) (*busarchive.Manifest, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
tr := tar.NewReader(f)
|
||||
hdr, err := tr.Next()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the snapshot does not read: %w", err)
|
||||
}
|
||||
if hdr.Name != busarchive.ManifestName {
|
||||
return nil, fmt.Errorf("the snapshot starts with %q, not its manifest", hdr.Name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var m busarchive.Manifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return nil, fmt.Errorf("its manifest: %w", err)
|
||||
}
|
||||
return &m, nil
|
||||
}
|
||||
|
||||
// Live says whether a stream of that name exists.
|
||||
func Live(ctx context.Context, nc *nats.Conn, name string) (bool, error) {
|
||||
err := Request(ctx, nc, APIStreamInfo+name, nil, nil)
|
||||
var api *APIError
|
||||
switch {
|
||||
case err == nil:
|
||||
return true, nil
|
||||
case errors.As(err, &api) && api.Code == 404:
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
|
||||
// DeleteStream removes a stream; one that is not there is no error.
|
||||
func DeleteStream(ctx context.Context, nc *nats.Conn, name string) error {
|
||||
err := Request(ctx, nc, APIStreamDelete+name, nil, nil)
|
||||
var api *APIError
|
||||
if errors.As(err, &api) && api.Code == 404 {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// RestoreFile puts every stream of the snapshot file back under its own name, each sent in chunks the server answers; it
|
||||
// ends with the context's bound. A stream already there is the caller's to have refused beforehand.
|
||||
func RestoreFile(ctx context.Context, nc *nats.Conn, path, scratch string, log func(string, ...any)) error {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
a, err := busarchive.ReadArchive(f, scratch)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer a.Close()
|
||||
if err := a.Verify(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, s := range a.Manifest.Streams {
|
||||
started := time.Now()
|
||||
if err := restoreStream(ctx, nc, a, s); err != nil {
|
||||
return fmt.Errorf("stream %s: %w", s.Name, err)
|
||||
}
|
||||
log("%s: restored in %.2fs", s.Name, time.Since(started).Seconds())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func restoreStream(ctx context.Context, nc *nats.Conn, a *busarchive.Archive, s busarchive.StreamEntry) error {
|
||||
meta, archive, err := a.Open(s)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer archive.Close()
|
||||
var req map[string]json.RawMessage
|
||||
if err := json.Unmarshal(meta, &req); err != nil || req["config"] == nil {
|
||||
return errors.New("its meta is not a stream's configuration and state")
|
||||
}
|
||||
var resp struct {
|
||||
DeliverSubject string `json:"deliver_subject"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamRestore+s.Name, req, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.DeliverSubject == "" {
|
||||
return errors.New("the server gave nowhere to send the archive")
|
||||
}
|
||||
buf := make([]byte, restoreChunk)
|
||||
for {
|
||||
n, err := archive.Read(buf)
|
||||
if n > 0 {
|
||||
reply, rerr := nc.RequestWithContext(ctx, resp.DeliverSubject, buf[:n])
|
||||
if rerr != nil {
|
||||
return fmt.Errorf("sending the archive: %w", rerr)
|
||||
}
|
||||
if len(reply.Data) > 0 {
|
||||
return fmt.Errorf("the server refused the archive: %s", Truncate(reply.Data))
|
||||
}
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
final, err := nc.RequestWithContext(ctx, resp.DeliverSubject, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("finishing the restore: %w", err)
|
||||
}
|
||||
var created struct {
|
||||
Error *APIError `json:"error"`
|
||||
State StreamState `json:"state"`
|
||||
}
|
||||
if err := json.Unmarshal(final.Data, &created); err != nil {
|
||||
return fmt.Errorf("the server's last answer is not JSON: %q", Truncate(final.Data))
|
||||
}
|
||||
if created.Error != nil {
|
||||
return created.Error
|
||||
}
|
||||
if created.State.LastSeq < s.LastSeq || (s.Messages > 0 && created.State.Messages == 0) {
|
||||
return fmt.Errorf("restored with %d message(s) up to sequence %d; the snapshot held %d up to %d",
|
||||
created.State.Messages, created.State.LastSeq, s.Messages, s.LastSeq)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
//go:build integration
|
||||
|
||||
package bussnap_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/bussnap"
|
||||
"git.novox.be/novox/mesh-sdk/go/testbus"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// TestOneStreamIsTakenRemovedAndRestored: the take and restore of one stream is mesh-sdk's (design 47 G §655, A-125):
|
||||
// a stream's snapshot taken with Into, the stream deleted, and RestoreFile puts it back under its own name with its
|
||||
// messages; a second restore over the live name is refused by the server.
|
||||
func TestOneStreamIsTakenRemovedAndRestored(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
nc, err := nats.Connect(testbus.URL(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer nc.Close()
|
||||
js, err := nc.JetStream()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := js.AddStream(&nats.StreamConfig{Name: "KEPT_one", Subjects: []string{"kept.one.>"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, m := range []string{"a", "b", "c"} {
|
||||
if _, err := js.Publish("kept.one."+m, []byte(m)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
file, err := bussnap.Into(nc, t.TempDir())(ctx, []string{"KEPT_one"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := bussnap.ManifestOf(file)
|
||||
if err != nil || len(m.Streams) != 1 || m.Streams[0].Name != "KEPT_one" {
|
||||
t.Fatalf("the manifest says %+v, %v; want the one stream KEPT_one", m, err)
|
||||
}
|
||||
if err := bussnap.DeleteStream(ctx, nc, "KEPT_one"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if live, err := bussnap.Live(ctx, nc, "KEPT_one"); err != nil || live {
|
||||
t.Fatalf("live after its deletion: %v, %v", live, err)
|
||||
}
|
||||
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, err := js.StreamInfo("KEPT_one")
|
||||
if err != nil || info.State.Msgs != 3 || info.State.LastSeq != 3 {
|
||||
t.Fatalf("restored as %+v, %v; want 3 messages up to sequence 3", info, err)
|
||||
}
|
||||
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err == nil {
|
||||
t.Fatal("a restore over the live name was taken")
|
||||
}
|
||||
}
|
||||
@@ -1,433 +0,0 @@
|
||||
// Package bussnap takes a snapshot of the bus's streams with the server's own snapshot API (novox/hq ADR 0235): the
|
||||
// nightly dump of every stream, and the bus module's keeper's snapshot of one entry before it is rebuilt or removed
|
||||
// (design 25 § 12.3).
|
||||
package bussnap
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/busarchive"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// The JetStream API's subjects this program speaks. Each is granted to the bus's own module and
|
||||
// nothing else is (mesh-controller internal/broker, BusSnapshotGrants); a test holds the two lists
|
||||
// equal against a real server.
|
||||
const (
|
||||
APIStreamNames = "$JS.API.STREAM.NAMES"
|
||||
APIStreamInfo = "$JS.API.STREAM.INFO."
|
||||
APIStreamSnapshot = "$JS.API.STREAM.SNAPSHOT."
|
||||
APIStreamRestore = "$JS.API.STREAM.RESTORE."
|
||||
)
|
||||
|
||||
// ChunkSize is what the server is asked to send at a time: its own default. The server sends up to
|
||||
// an 8 MiB window ahead of the acknowledgements and waits for them past it, so one stream costs the
|
||||
// server at most that much in flight, whatever its size.
|
||||
const ChunkSize = 128 * 1024
|
||||
|
||||
// SnapshotOptions bounds a snapshot.
|
||||
type SnapshotOptions struct {
|
||||
// StreamTimeout is the longest one stream may take; past it the snapshot fails, naming it.
|
||||
StreamTimeout time.Duration
|
||||
// Pause is a rest between two streams, so a night's snapshot is never one long burst.
|
||||
Pause time.Duration
|
||||
// TempDir is where one stream's archive waits until it is written out; the system's when empty.
|
||||
TempDir string
|
||||
Log func(format string, args ...any)
|
||||
// Streams, when set, are the only streams taken (the keeper's snapshot of one entry); every stream when empty.
|
||||
Streams []string
|
||||
}
|
||||
|
||||
// APIError is the error every JetStream API answer may carry.
|
||||
type APIError struct {
|
||||
Code int `json:"code"`
|
||||
ErrCode int `json:"err_code"`
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
func (e *APIError) Error() string { return fmt.Sprintf("%s (%d)", e.Description, e.Code) }
|
||||
|
||||
// StreamState is what this program reads of a stream's state; the whole of it is kept as the server
|
||||
// gave it, in the stream's meta.
|
||||
type StreamState struct {
|
||||
Messages uint64 `json:"messages"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
FirstSeq uint64 `json:"first_seq"`
|
||||
LastSeq uint64 `json:"last_seq"`
|
||||
NumDelete int `json:"num_deleted"`
|
||||
Consumers int `json:"consumer_count"`
|
||||
}
|
||||
|
||||
type streamConfig struct {
|
||||
Name string `json:"name"`
|
||||
Subjects []string `json:"subjects"`
|
||||
Retention string `json:"retention"`
|
||||
Storage string `json:"storage"`
|
||||
}
|
||||
|
||||
// Request asks the JetStream API and decodes its answer, refusing an answer that carries an error.
|
||||
func Request(ctx context.Context, nc *nats.Conn, subject string, body any, into any) error {
|
||||
var payload []byte
|
||||
if body != nil {
|
||||
var err error
|
||||
if payload, err = json.Marshal(body); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
defer cancel()
|
||||
msg, err := nc.RequestWithContext(rctx, subject, payload)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout) || errors.Is(err, nats.ErrNoResponders) {
|
||||
return fmt.Errorf("%s was not answered (%v): the bus refuses a subject this user is not granted by not answering it", subject, err)
|
||||
}
|
||||
return fmt.Errorf("%s: %w", subject, err)
|
||||
}
|
||||
var wrapped struct {
|
||||
Error *APIError `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(msg.Data, &wrapped); err != nil {
|
||||
return fmt.Errorf("%s answered something that is not JSON: %q", subject, Truncate(msg.Data))
|
||||
}
|
||||
if wrapped.Error != nil {
|
||||
return fmt.Errorf("%s: %w", subject, wrapped.Error)
|
||||
}
|
||||
if into != nil {
|
||||
return json.Unmarshal(msg.Data, into)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func Truncate(b []byte) string {
|
||||
if len(b) > 200 {
|
||||
return string(b[:200]) + "…"
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// StreamNames is every stream in the account, in name order, all pages of them.
|
||||
func StreamNames(ctx context.Context, nc *nats.Conn) ([]string, error) {
|
||||
var names []string
|
||||
for {
|
||||
var page struct {
|
||||
Total int `json:"total"`
|
||||
Offset int `json:"offset"`
|
||||
Limit int `json:"limit"`
|
||||
Streams []string `json:"streams"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamNames, map[string]int{"offset": len(names)}, &page); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names = append(names, page.Streams...)
|
||||
if len(page.Streams) == 0 || len(names) >= page.Total {
|
||||
break
|
||||
}
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names, nil
|
||||
}
|
||||
|
||||
// Snapshot writes every stream, one at a time, as a tar: the manifest first, then each stream's
|
||||
// meta and archive under streams/<name>/. Any stream failing fails the whole: a partial copy of the
|
||||
// bus written where a whole one is expected is the silent failure this exists to prevent.
|
||||
func Snapshot(ctx context.Context, nc *nats.Conn, out io.Writer, o SnapshotOptions) (*busarchive.Manifest, error) {
|
||||
if o.StreamTimeout <= 0 {
|
||||
o.StreamTimeout = 10 * time.Minute
|
||||
}
|
||||
if o.Log == nil {
|
||||
o.Log = func(string, ...any) {}
|
||||
}
|
||||
m := &busarchive.Manifest{Format: 1, Server: nc.ConnectedServerVersion(), Started: time.Now().UTC()}
|
||||
names, err := StreamNames(ctx, nc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(o.Streams) > 0 {
|
||||
var only []string
|
||||
for _, n := range names {
|
||||
if slices.Contains(o.Streams, n) {
|
||||
only = append(only, n)
|
||||
}
|
||||
}
|
||||
if len(only) != len(o.Streams) {
|
||||
return nil, fmt.Errorf("the bus holds %v of the streams %v asked for: a snapshot missing one would pass for a whole one", only, o.Streams)
|
||||
}
|
||||
names = only
|
||||
}
|
||||
if len(names) == 0 {
|
||||
// The mesh's own streams always exist once the controller has run; a bus with none is not
|
||||
// one to call backed up.
|
||||
return nil, errors.New("the bus holds no streams; a snapshot of nothing would pass for a backup")
|
||||
}
|
||||
|
||||
type taken struct {
|
||||
entry busarchive.StreamEntry
|
||||
meta []byte
|
||||
archive *os.File
|
||||
}
|
||||
var files []taken
|
||||
defer func() {
|
||||
for _, t := range files {
|
||||
t.archive.Close()
|
||||
os.Remove(t.archive.Name())
|
||||
}
|
||||
}()
|
||||
|
||||
for i, name := range names {
|
||||
if i > 0 && o.Pause > 0 {
|
||||
select {
|
||||
case <-time.After(o.Pause):
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
var info struct {
|
||||
Config streamConfig `json:"config"`
|
||||
}
|
||||
if err := Request(ctx, nc, APIStreamInfo+name, nil, &info); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if info.Config.Storage == "memory" {
|
||||
m.Skipped = append(m.Skipped, busarchive.Skipped{Name: name, Why: "memory storage: it holds nothing across a restart, and the server cannot snapshot it"})
|
||||
o.Log("%s: skipped, memory storage", name)
|
||||
continue
|
||||
}
|
||||
f, err := os.CreateTemp(o.TempDir, "mesh-nats-snapshot-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
files = append(files, taken{archive: f})
|
||||
sctx, cancel := context.WithTimeout(ctx, o.StreamTimeout)
|
||||
entry, meta, err := snapshotStream(sctx, nc, name, f)
|
||||
cancel()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("stream %s: %w", name, err)
|
||||
}
|
||||
files[len(files)-1].entry, files[len(files)-1].meta = entry, meta
|
||||
m.Messages += entry.Messages
|
||||
m.StreamBytes += int64(entry.Bytes)
|
||||
m.ArchiveBytes += entry.ArchiveBytes
|
||||
o.Log("%s: %d message(s), %s, last sequence %d, %d consumer(s) → %s in %.2fs",
|
||||
name, entry.Messages, Human(int64(entry.Bytes)), entry.LastSeq, entry.Consumers, Human(entry.ArchiveBytes), entry.Seconds)
|
||||
}
|
||||
m.Finished = time.Now().UTC()
|
||||
m.Seconds = m.Finished.Sub(m.Started).Seconds()
|
||||
for _, t := range files {
|
||||
m.Streams = append(m.Streams, t.entry)
|
||||
}
|
||||
|
||||
tw := tar.NewWriter(out)
|
||||
manifest, err := json.MarshalIndent(m, "", " ")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := writeEntry(tw, busarchive.ManifestName, append(manifest, '\n'), m.Finished); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, t := range files {
|
||||
if err := writeEntry(tw, t.entry.Meta, t.meta, m.Finished); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := t.archive.Seek(0, io.SeekStart); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hdr := &tar.Header{Name: t.entry.Archive, Mode: 0o600, Size: t.entry.ArchiveBytes, ModTime: m.Finished, Format: tar.FormatPAX}
|
||||
if err := tw.WriteHeader(hdr); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := io.Copy(tw, t.archive); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if err := tw.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func writeEntry(tw *tar.Writer, name string, body []byte, at time.Time) error {
|
||||
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(body)), ModTime: at, Format: tar.FormatPAX}); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := tw.Write(body)
|
||||
return err
|
||||
}
|
||||
|
||||
// snapshotStream takes one stream's snapshot into f: the server's chunks, each acknowledged as it
|
||||
// arrives so the server's window keeps moving, until the server's last, empty message. It answers
|
||||
// the stream's entry (the archive's size and checksum included) and its meta.
|
||||
//
|
||||
// The protocol is the server's (nats-server jetstream_api.go, streamSnapshot): the request names a
|
||||
// subject to deliver to; the server answers with the stream's configuration and its state at the
|
||||
// snapshot, then sends the archive in chunks whose reply subject is the acknowledgement it waits for
|
||||
// past its window — two seconds without one and it gives up with "408 No Flow Response". The end is
|
||||
// an empty message whose status header says 204, or the error that ended it.
|
||||
func snapshotStream(ctx context.Context, nc *nats.Conn, name string, f *os.File) (busarchive.StreamEntry, []byte, error) {
|
||||
started := time.Now()
|
||||
deliver := nc.NewRespInbox()
|
||||
|
||||
hash := sha256.New()
|
||||
var written int64
|
||||
var mu sync.Mutex
|
||||
var failed error
|
||||
done := make(chan struct{})
|
||||
var once sync.Once
|
||||
finish := func(err error) {
|
||||
mu.Lock()
|
||||
if failed == nil {
|
||||
failed = err
|
||||
}
|
||||
mu.Unlock()
|
||||
once.Do(func() { close(done) })
|
||||
}
|
||||
sub, err := nc.Subscribe(deliver, func(msg *nats.Msg) {
|
||||
if len(msg.Data) == 0 {
|
||||
if status := msg.Header.Get("Status"); status != "" && status != "204" {
|
||||
finish(fmt.Errorf("the server ended the snapshot: %s %s", status, msg.Header.Get("Description")))
|
||||
return
|
||||
}
|
||||
finish(nil)
|
||||
return
|
||||
}
|
||||
// Acknowledged before it is written: the chunk is already in memory, and a slow disk must not
|
||||
// read to the server as a reader that went away.
|
||||
if msg.Reply != "" {
|
||||
if err := nc.Publish(msg.Reply, nil); err != nil {
|
||||
finish(fmt.Errorf("acknowledging a chunk: %w", err))
|
||||
return
|
||||
}
|
||||
}
|
||||
if _, err := f.Write(msg.Data); err != nil {
|
||||
finish(fmt.Errorf("writing the archive: %w", err))
|
||||
return
|
||||
}
|
||||
hash.Write(msg.Data)
|
||||
written += int64(len(msg.Data))
|
||||
})
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
defer sub.Unsubscribe()
|
||||
// Every chunk is held until the callback has it; the window bounds how many there can be.
|
||||
if err := sub.SetPendingLimits(-1, -1); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if err := nc.FlushWithContext(ctx); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Config json.RawMessage `json:"config"`
|
||||
State json.RawMessage `json:"state"`
|
||||
}
|
||||
req := map[string]any{"deliver_subject": deliver, "chunk_size": ChunkSize}
|
||||
if err := Request(ctx, nc, APIStreamSnapshot+name, req, &resp); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
select {
|
||||
case <-done:
|
||||
case <-ctx.Done():
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("not finished within its bound: %w", ctx.Err())
|
||||
}
|
||||
// The subscription is drained before the archive is read back, so no callback is still writing.
|
||||
_ = sub.Unsubscribe()
|
||||
mu.Lock()
|
||||
err = failed
|
||||
mu.Unlock()
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if written == 0 {
|
||||
return busarchive.StreamEntry{}, nil, errors.New("the server sent an empty archive")
|
||||
}
|
||||
if err := f.Sync(); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
|
||||
var cfg streamConfig
|
||||
var state StreamState
|
||||
if err := json.Unmarshal(resp.Config, &cfg); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("its configuration: %w", err)
|
||||
}
|
||||
if err := json.Unmarshal(resp.State, &state); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("its state: %w", err)
|
||||
}
|
||||
// The archive reads to its end — s2 frames, then a tar — before it is called a snapshot.
|
||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
if err := busarchive.ReadsThrough(f); err != nil {
|
||||
return busarchive.StreamEntry{}, nil, fmt.Errorf("the archive the server sent does not read: %w", err)
|
||||
}
|
||||
|
||||
// The meta, in the shape the restore API takes and the nats CLI writes (backup.json): the
|
||||
// server's own configuration and state, kept as raw JSON so a field this program does not know
|
||||
// is never dropped on the way back.
|
||||
meta, err := json.MarshalIndent(map[string]json.RawMessage{"config": resp.Config, "state": resp.State}, "", " ")
|
||||
if err != nil {
|
||||
return busarchive.StreamEntry{}, nil, err
|
||||
}
|
||||
metaSum := sha256.Sum256(meta)
|
||||
return busarchive.StreamEntry{
|
||||
Name: name, Subjects: cfg.Subjects, Retention: cfg.Retention,
|
||||
Messages: state.Messages, Bytes: state.Bytes, FirstSeq: state.FirstSeq, LastSeq: state.LastSeq,
|
||||
Deleted: state.NumDelete, Consumers: state.Consumers,
|
||||
Meta: "streams/" + name + "/backup.json", MetaSHA256: hex.EncodeToString(metaSum[:]),
|
||||
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: written,
|
||||
ArchiveSHA256: hex.EncodeToString(hash.Sum(nil)), Seconds: time.Since(started).Seconds(),
|
||||
}, meta, nil
|
||||
}
|
||||
|
||||
// Human is a size in bytes as a person reads it: 1.5 MiB.
|
||||
func Human(n int64) string {
|
||||
const unit = 1024
|
||||
if n < unit {
|
||||
return fmt.Sprintf("%d B", n)
|
||||
}
|
||||
div, exp := int64(unit), 0
|
||||
for v := n / unit; v >= unit; v /= unit {
|
||||
div *= unit
|
||||
exp++
|
||||
}
|
||||
return fmt.Sprintf("%.1f %ciB", float64(n)/float64(div), "KMGTPE"[exp])
|
||||
}
|
||||
|
||||
// Into is the keeper's snapshot of some streams before it rebuilds or removes them (design 25 § 12.3): one tar under
|
||||
// dir, named for the streams and the time, written whole before its name is given, so a partial one is never taken
|
||||
// for it. It answers the file's path.
|
||||
func Into(nc *nats.Conn, dir string) func(ctx context.Context, streams []string) (string, error) {
|
||||
return func(ctx context.Context, streams []string) (string, error) {
|
||||
if dir == "" {
|
||||
return "", errors.New("no snapshots directory is set (MESH_NATS_SNAPSHOTS_DIR), so no snapshot is taken and nothing is removed")
|
||||
}
|
||||
name := filepath.Join(dir, "entry-"+strings.Join(streams, "+")+"-"+time.Now().UTC().Format("20060102T150405.000000Z")+".tar")
|
||||
f, err := os.CreateTemp(dir, ".entry-*.partial")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if _, err := Snapshot(ctx, nc, f, SnapshotOptions{Streams: streams, TempDir: dir}); err != nil {
|
||||
f.Close()
|
||||
return "", err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, os.Rename(f.Name(), name)
|
||||
}
|
||||
}
|
||||
@@ -1,31 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// busStateCalls are the method prefixes of the calls a bundle makes on its own bus state through the runtime (novox/hq
|
||||
// ADR 0595 §9, design 25 § 12.5): their shapes and names are package buscalls's, and package busstate the typed way
|
||||
// to make them.
|
||||
var busStateCalls = []string{"mesh/objects.", "mesh/stream.", "mesh/workqueue.", "mesh/ledger."}
|
||||
|
||||
// BusState asks the runtime one call on this bundle's own bus state, an entry its module asks for in `requires`, named
|
||||
// by its local name in params; the runtime serves it from the module's own bindings only. A method that is none of
|
||||
// those calls is refused here, before the runtime is asked.
|
||||
func BusState(method string, params any) (json.RawMessage, error) {
|
||||
known := false
|
||||
for _, p := range busStateCalls {
|
||||
known = known || strings.HasPrefix(method, p)
|
||||
}
|
||||
if !known {
|
||||
return nil, fmt.Errorf("%s is no call on a bundle's own bus state (%s)", method, strings.Join(busStateCalls, ", "))
|
||||
}
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.ask(method, params, 60*time.Second)
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// Design 47 G §587, §622: the ledger's upkeep calls have their requests and answers, and every family its word.
|
||||
func subTheLedgerUpkeepCallsAndTheirWords(t *testing.T) {
|
||||
var removal buscalls.LedgerRemoveRequest
|
||||
if err := json.Unmarshal([]byte(`{"ledger":"records","instance":"r1"}`), &removal); err != nil || removal.Ledger != "records" || removal.Instance != "r1" {
|
||||
t.Errorf("the removal read as %+v (%v)", removal, err)
|
||||
}
|
||||
raw, _ := json.Marshal(buscalls.LedgerRemoved{Instance: "r1", Operations: 3, Bytes: 900})
|
||||
if string(raw) != `{"instance":"r1","operations":3,"bytes":900}` {
|
||||
t.Errorf("the removal's answer reads %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(buscalls.Fill{Count: 2, Bytes: 10, MaxBytes: 100})
|
||||
if string(raw) != `{"count":2,"bytes":10,"max_bytes":100}` || buscalls.LedgerFill != "mesh/ledger.fill" || buscalls.LedgerRemove != "mesh/ledger.remove" {
|
||||
t.Errorf("the fill reads %s, calls %s %s", raw, buscalls.LedgerFill, buscalls.LedgerRemove)
|
||||
}
|
||||
for word, want := range map[string]string{buscalls.BusStateCalls: "bus-state-calls", buscalls.LedgerCalls: "ledger-calls",
|
||||
buscalls.LedgerUpkeepCalls: "ledger-upkeep-calls", buscalls.SentObjectsCalls: "sent-objects-calls"} {
|
||||
if word != want {
|
||||
t.Errorf("the word %q is %q", want, word)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
// Package busstate is how a bundle uses its own bus state through the runtime (novox/hq ADR 0595 §9, design 25 §
|
||||
// 12.5): the object store's put, get, list, delete and fill, each naming an entry by the local name its module asks for
|
||||
// it by in `requires`. The runtime serves the calls from the module's own bindings, its own entries only. The shapes
|
||||
// are package buscalls's; this package makes the calls.
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// Objects is one object store the module asks for in `requires` (the template objects, ADR 0595 §2), by its local name.
|
||||
type Objects struct{ local string }
|
||||
|
||||
// ObjectsOf is the object store of that local name.
|
||||
func ObjectsOf(local string) Objects { return Objects{local: local} }
|
||||
|
||||
func call[T any](method string, params any) (T, error) {
|
||||
var out T
|
||||
raw, err := stdio.BusState(method, params)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return out, fmt.Errorf("the runtime's answer to %s: %w", method, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it.
|
||||
//
|
||||
// A value larger than buscalls.ChunkBytes is put as a session of chunks (design 47 G §518, §345): every object is one
|
||||
// object, never split over several.
|
||||
func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
|
||||
if len(data) > buscalls.ChunkBytes {
|
||||
return o.putSession(name, data, description)
|
||||
}
|
||||
return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name,
|
||||
Data: data, Description: description})
|
||||
}
|
||||
|
||||
// Get is one object, its info and its bytes.
|
||||
//
|
||||
// A value the runtime answers as a session (`chunked`) is read chunk by chunk and checked against its digest (design 47 G §490).
|
||||
func (o Objects) Get(name string) (buscalls.ObjectData, error) {
|
||||
raw, err := stdio.BusState(buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
if err != nil {
|
||||
return buscalls.ObjectData{}, err
|
||||
}
|
||||
return ReadObject(name, raw)
|
||||
}
|
||||
|
||||
// ReadObject is the object a runtime's answer to a get (or to a sent-get, design 47 G §620) holds: the value itself, or the
|
||||
// session of chunks it opened, read whole.
|
||||
func ReadObject(name string, raw json.RawMessage) (buscalls.ObjectData, error) {
|
||||
if c, ok := chunkedAnswer(raw); ok {
|
||||
return getSession(name, c)
|
||||
}
|
||||
var out buscalls.ObjectData
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return out, fmt.Errorf("the runtime's answer for %s: %w", name, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// List is every object the store holds.
|
||||
func (o Objects) List() ([]buscalls.ObjectInfo, error) {
|
||||
l, err := call[buscalls.ObjectsListed](buscalls.ObjectsList, buscalls.ObjectsRequest{Objects: o.local})
|
||||
return l.Objects, err
|
||||
}
|
||||
|
||||
// Delete removes one object.
|
||||
func (o Objects) Delete(name string) error {
|
||||
_, err := call[buscalls.Done](buscalls.ObjectsDelete, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
|
||||
return err
|
||||
}
|
||||
|
||||
// Fill is how much the store holds against its cap.
|
||||
func (o Objects) Fill() (buscalls.Fill, error) {
|
||||
return call[buscalls.Fill](buscalls.ObjectsInfo, buscalls.ObjectsRequest{Objects: o.local})
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's
|
||||
// incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the
|
||||
// runtime answered; a method that is no bus-state call is refused before the runtime is asked.
|
||||
func subABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
|
||||
t.Setenv(stdio.ServedAs, "distribution")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
type result struct {
|
||||
what string
|
||||
err error
|
||||
got any
|
||||
}
|
||||
results := make(chan result, 8)
|
||||
go func() {
|
||||
_ = stdio.ServeOn("", []stdio.Tool{{Name: "keep", Run: func(map[string]any) (any, error) {
|
||||
go func() {
|
||||
in := ObjectsOf("incoming")
|
||||
info, err := in.Put("t1.part0", []byte("hello"), "a part")
|
||||
results <- result{"put", err, info.Digest}
|
||||
data, err := in.Get("t1.part0")
|
||||
results <- result{"get", err, string(data.Data)}
|
||||
list, err := in.List()
|
||||
results <- result{"list", err, len(list)}
|
||||
results <- result{"delete", in.Delete("t1.part0"), nil}
|
||||
fill, err := in.Fill()
|
||||
results <- result{"fill", err, fill.MaxBytes}
|
||||
_, err = stdio.BusState("mesh/state.get", nil)
|
||||
results <- result{"refused", err, nil}
|
||||
}()
|
||||
return "started", nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
lines := bufio.NewScanner(outR)
|
||||
next := func() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "keep"}})
|
||||
answers := map[string]any{
|
||||
buscalls.ObjectsPut: buscalls.ObjectInfo{Name: "t1.part0", Size: 5, Digest: "SHA-256=x"},
|
||||
buscalls.ObjectsGet: buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: "t1.part0"}, Data: []byte("hello")},
|
||||
buscalls.ObjectsList: buscalls.ObjectsListed{Objects: []buscalls.ObjectInfo{{Name: "t1.part0"}}},
|
||||
buscalls.ObjectsDelete: buscalls.Done{},
|
||||
buscalls.ObjectsInfo: buscalls.Fill{Count: 0, MaxBytes: 20 << 30},
|
||||
}
|
||||
asked := map[string]map[string]any{}
|
||||
for len(asked) < len(answers) {
|
||||
m := next()
|
||||
method, _ := m["method"].(string)
|
||||
if method == "" {
|
||||
continue // the tool's own answer
|
||||
}
|
||||
p, _ := m["params"].(map[string]any)
|
||||
asked[method] = p
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": answers[method]})
|
||||
}
|
||||
want := map[string]any{"put": "SHA-256=x", "get": "hello", "list": 1, "delete": nil, "fill": uint64(20 << 30)}
|
||||
for range 5 {
|
||||
r := <-results
|
||||
if r.err != nil || r.got != want[r.what] {
|
||||
t.Errorf("%s answered %v, %v; want %v", r.what, r.got, r.err, want[r.what])
|
||||
}
|
||||
}
|
||||
if r := <-results; r.err == nil {
|
||||
t.Error("a call that is not on the bundle's bus state was asked of the runtime")
|
||||
}
|
||||
for method, p := range asked {
|
||||
if p["objects"] != "incoming" {
|
||||
t.Errorf("%s named the store %v, not its local name", method, p["objects"])
|
||||
}
|
||||
}
|
||||
if p := asked[buscalls.ObjectsPut]; p["name"] != "t1.part0" || p["data"] != "aGVsbG8=" || p["description"] != "a part" {
|
||||
t.Errorf("put asked %v", p)
|
||||
}
|
||||
}
|
||||
|
||||
// TestABundleKeepsObjectsThroughTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
|
||||
t.Run("ABundleKeepsObjectsThroughTheRuntime", subABundleKeepsObjectsThroughTheRuntime)
|
||||
t.Run("TheLedgerUpkeepCallsAndTheirWords", subTheLedgerUpkeepCallsAndTheirWords)
|
||||
t.Run("AValueLargerThanOneChunkIsPutAndGotBySession", subAValueLargerThanOneChunkIsPutAndGotBySession)
|
||||
}
|
||||
@@ -1,113 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// The put and get sessions of a value larger than one chunk (design 47 G §518, §345, §490, A-21): a bundle sends and asks the
|
||||
// chunks as calls on its channel to the tool runner, which holds the session; nothing of it is on the bus. The capability
|
||||
// word put-sessions names the two calls, so a tool runner that does not state it refuses the session at once.
|
||||
|
||||
// callID is the bundle's id for one session: random, so two sessions never share one.
|
||||
func callID() string {
|
||||
var b [12]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
return hex.EncodeToString(b[:])
|
||||
}
|
||||
|
||||
// put sends data as a put session: chunks of buscalls.ChunkBytes in order, each answered before the next is sent, the
|
||||
// answer to the last being the put's.
|
||||
func (o Objects) putSession(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
|
||||
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, larger than one call carries: %w", name, err)
|
||||
}
|
||||
id := callID()
|
||||
for seq, at := uint32(1), 0; ; seq++ {
|
||||
end := min(at+buscalls.ChunkBytes, len(data))
|
||||
last := end == len(data)
|
||||
req := buscalls.ObjectsPutChunkRequest{Objects: o.local, Name: name, Call: id, Seq: seq, Last: last, Data: data[at:end]}
|
||||
if seq == 1 {
|
||||
req.Description = description // the first chunk carries it (design 47 G A-25)
|
||||
}
|
||||
raw, err := stdio.BusState(buscalls.ObjectsPutChunk, req)
|
||||
if err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %w", name, seq, err)
|
||||
}
|
||||
var taken buscalls.ObjectsChunkTaken
|
||||
if err := json.Unmarshal(raw, &taken); err != nil {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("the runtime's answer to a chunk of %s: %w", name, err)
|
||||
}
|
||||
if taken.Error != "" {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %s", name, seq, taken.Error)
|
||||
}
|
||||
if last {
|
||||
var info buscalls.ObjectInfo
|
||||
if err := json.Unmarshal(raw, &info); err != nil {
|
||||
return info, fmt.Errorf("the runtime's answer to the put of %s: %w", name, err)
|
||||
}
|
||||
return info, nil
|
||||
}
|
||||
if taken.Seq != seq {
|
||||
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s: chunk %d was answered as chunk %d", name, seq, taken.Seq)
|
||||
}
|
||||
at = end
|
||||
}
|
||||
}
|
||||
|
||||
// getSession reads the chunks of the session the runtime opened for name, and checks the whole against the digest and the
|
||||
// size it answered.
|
||||
func getSession(name string, c buscalls.ObjectsChunked) (buscalls.ObjectData, error) {
|
||||
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, larger than one call carries: %w", name, err)
|
||||
}
|
||||
var whole []byte
|
||||
for seq := uint32(1); ; seq++ {
|
||||
raw, err := stdio.BusState(buscalls.ObjectsGetChunk, buscalls.ObjectsGetChunkRequest{Call: c.Call, Seq: seq})
|
||||
if err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %w", name, seq, err)
|
||||
}
|
||||
var chunk buscalls.ObjectsChunk
|
||||
if err := json.Unmarshal(raw, &chunk); err != nil {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("the runtime's chunk %d of %s: %w", seq, name, err)
|
||||
}
|
||||
if chunk.Error != "" {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %s", name, seq, chunk.Error)
|
||||
}
|
||||
if chunk.Seq != seq {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: chunk %d was answered as chunk %d", name, seq, chunk.Seq)
|
||||
}
|
||||
whole = append(whole, chunk.Data...)
|
||||
if chunk.Last {
|
||||
break
|
||||
}
|
||||
}
|
||||
sum := sha256.Sum256(whole)
|
||||
if c.Digest != "" && !strings.EqualFold(c.Digest, "sha256:"+hex.EncodeToString(sum[:])) {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: what was read is not the value the runtime named by %s", name, c.Digest)
|
||||
}
|
||||
if c.Size != 0 && c.Size != uint64(len(whole)) {
|
||||
return buscalls.ObjectData{}, fmt.Errorf("getting %s: %d bytes were read, the runtime said %d", name, len(whole), c.Size)
|
||||
}
|
||||
return buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: name, Size: uint64(len(whole)), Digest: c.Digest}, Data: whole}, nil
|
||||
}
|
||||
|
||||
// chunkedAnswer is the session an answer to a get opens, if it is not the value itself.
|
||||
func chunkedAnswer(raw json.RawMessage) (buscalls.ObjectsChunked, bool) {
|
||||
var c buscalls.ObjectsChunked
|
||||
if json.Unmarshal(raw, &c) != nil {
|
||||
return c, false
|
||||
}
|
||||
switch strings.TrimSpace(string(c.Chunked)) {
|
||||
case "", "null", "0", "false":
|
||||
return c, false
|
||||
}
|
||||
return c, c.Call != ""
|
||||
}
|
||||
@@ -1,126 +0,0 @@
|
||||
package busstate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// serveRuntime runs a bundle whose tool `go` runs fn in a goroutine and answers every call the bundle makes through handle,
|
||||
// until done is closed by fn returning.
|
||||
func serveRuntime(t *testing.T, fn func(), handle func(method string, params map[string]any) any) {
|
||||
t.Helper()
|
||||
t.Setenv(stdio.ServedAs, "distribution")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
_ = stdio.ServeOn("", []stdio.Tool{{Name: "go", Run: func(map[string]any) (any, error) {
|
||||
go func() { fn(); close(done) }()
|
||||
return "started", nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
lines := bufio.NewScanner(outR)
|
||||
lines.Buffer(make([]byte, 0, 1<<20), 64<<20)
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "go"}})
|
||||
deadline := time.After(10 * time.Second)
|
||||
for {
|
||||
got := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
got <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
return
|
||||
case <-deadline:
|
||||
t.Fatal("the bundle did not finish")
|
||||
case m := <-got:
|
||||
method, _ := m["method"].(string)
|
||||
if method == "" {
|
||||
continue
|
||||
}
|
||||
p, _ := m["params"].(map[string]any)
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": handle(method, p)})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Design 47 G §518, §345, §490 (A-21): a value larger than one chunk is put as chunks in order under one call id, each
|
||||
// answered before the next, with the store named in each; a get the runtime answers as a session is read chunk by chunk and
|
||||
// checked against its digest.
|
||||
func subAValueLargerThanOneChunkIsPutAndGotBySession(t *testing.T) {
|
||||
value := bytes.Repeat([]byte("0123456789abcdef"), (2*buscalls.ChunkBytes+1000)/16)
|
||||
sum := sha256.Sum256(value)
|
||||
digest := "sha256:" + hex.EncodeToString(sum[:])
|
||||
var stored []byte
|
||||
var seqs []uint32
|
||||
calls := map[string]bool{}
|
||||
var gotValue []byte
|
||||
var putErr, getErr, tamperErr error
|
||||
var tampered bool
|
||||
serveRuntime(t, func() {
|
||||
o := ObjectsOf("incoming")
|
||||
_, putErr = o.Put("big", value, "a big one")
|
||||
d, err := o.Get("big")
|
||||
gotValue, getErr = d.Data, err
|
||||
tampered = true
|
||||
_, tamperErr = o.Get("big")
|
||||
}, func(method string, p map[string]any) any {
|
||||
switch method {
|
||||
case stdio.CapabilitiesCall:
|
||||
return map[string]any{"words": []string{stdio.CapPutSessions}}
|
||||
case buscalls.ObjectsPutChunk:
|
||||
calls[p["call"].(string)] = true
|
||||
if p["objects"] != "incoming" || p["name"] != "big" {
|
||||
t.Errorf("a chunk named %v in %v", p["name"], p["objects"])
|
||||
}
|
||||
raw, _ := json.Marshal(p["data"])
|
||||
var b []byte
|
||||
_ = json.Unmarshal(raw, &b)
|
||||
if (p["description"] == "a big one") != (p["seq"].(float64) == 1) {
|
||||
t.Errorf("chunk %v carries the description %v: the first chunk alone does", p["seq"], p["description"])
|
||||
}
|
||||
stored = append(stored, b...)
|
||||
seqs = append(seqs, uint32(p["seq"].(float64)))
|
||||
if p["last"] == true {
|
||||
return buscalls.ObjectInfo{Name: "big", Size: uint64(len(stored))}
|
||||
}
|
||||
return buscalls.ObjectsChunkTaken{Seq: uint32(p["seq"].(float64))}
|
||||
case buscalls.ObjectsGet:
|
||||
d := digest
|
||||
if tampered {
|
||||
d = "sha256:" + hex.EncodeToString(make([]byte, 32))
|
||||
}
|
||||
return map[string]any{"chunked": 3, "size": len(stored), "digest": d, "call": "g1"}
|
||||
case buscalls.ObjectsGetChunk:
|
||||
seq := int(p["seq"].(float64))
|
||||
at := (seq - 1) * buscalls.ChunkBytes
|
||||
return buscalls.ObjectsChunk{Seq: uint32(seq), Last: seq == 3, Data: stored[at:min(at+buscalls.ChunkBytes, len(stored))]}
|
||||
}
|
||||
t.Errorf("unexpected call %s", method)
|
||||
return nil
|
||||
})
|
||||
if putErr != nil || getErr != nil {
|
||||
t.Fatalf("put %v, get %v", putErr, getErr)
|
||||
}
|
||||
if len(seqs) != 3 || seqs[0] != 1 || seqs[1] != 2 || seqs[2] != 3 || len(calls) != 1 || !bytes.Equal(stored, value) || !bytes.Equal(gotValue, value) {
|
||||
t.Errorf("chunks %v under %d call ids, stored %d bytes, got %d", seqs, len(calls), len(stored), len(gotValue))
|
||||
}
|
||||
if tamperErr == nil {
|
||||
t.Error("a value that is not the one the runtime named by its digest was accepted")
|
||||
}
|
||||
}
|
||||
@@ -1,108 +0,0 @@
|
||||
// Package calladdress reads the key a module calls a tool or a seat verb by, as a module's invokes names it, with the
|
||||
// machine it is asked of after '@': `seat:<seat>.<verb>[@<node>]` or `<module>.<tool>[@<node>]` (novox/hq ADR 0159,
|
||||
// ADR 0469 §3). Every caller in the SDK parses its key here before it publishes anything, so a malformed key fails at
|
||||
// once and says why, never as a call nobody serves and a timeout (novox/hq issue 774, the lease write's stop asked on
|
||||
// `...call.stop@novox-be...`).
|
||||
package calladdress
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Address is a parsed key: a seat and its verb, or a module and its tool, and the machine named, if any.
|
||||
type Address struct {
|
||||
Seat, Verb string
|
||||
Module, Tool string
|
||||
Node string
|
||||
}
|
||||
|
||||
// Key is the address as a module's invokes names it, without the machine.
|
||||
func (a Address) Key() string {
|
||||
if a.Seat != "" {
|
||||
return "seat:" + a.Seat + "." + a.Verb
|
||||
}
|
||||
return a.Module + "." + a.Tool
|
||||
}
|
||||
|
||||
// Parse reads a key, refusing one with more than one '@', a node that is not one subject token, or a seat, verb, module
|
||||
// or tool that holds a wildcard, white space or (but a tool) a dot.
|
||||
func Parse(key string) (Address, error) {
|
||||
name, node, err := cutNode(key)
|
||||
if err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
if rest, ok := strings.CutPrefix(name, "seat:"); ok {
|
||||
seat, verb, ok := strings.Cut(rest, ".")
|
||||
if !ok || seat == "" || verb == "" {
|
||||
return Address{}, fmt.Errorf("%q names no seat and verb (seat:<seat>.<verb>)", key)
|
||||
}
|
||||
if err := tokens(key, seat, verb); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
return Address{Seat: seat, Verb: verb, Node: node}, nil
|
||||
}
|
||||
module, tool, ok := strings.Cut(name, ".")
|
||||
if !ok || module == "" || tool == "" {
|
||||
return Address{}, fmt.Errorf("%q is not <module>.<tool>, and does not start seat:<seat>.<verb>", key)
|
||||
}
|
||||
if err := tokens(key, module); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
if err := checkTool(key, tool); err != nil {
|
||||
return Address{}, err
|
||||
}
|
||||
return Address{Module: module, Tool: tool, Node: node}, nil
|
||||
}
|
||||
|
||||
// CheckOwn is Parse for a caller that may also name a tool of its own module by its bare name (the runtime's mesh/ask).
|
||||
func CheckOwn(key string) error {
|
||||
name, _, err := cutNode(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !strings.HasPrefix(name, "seat:") && !strings.Contains(name, ".") {
|
||||
return tokens(key, name)
|
||||
}
|
||||
_, err = Parse(key)
|
||||
return err
|
||||
}
|
||||
|
||||
// CheckNode refuses a node name that is not one subject token.
|
||||
func CheckNode(node string) error {
|
||||
if node == "" || strings.ContainsAny(node, ".*>@ \t\r\n") {
|
||||
return fmt.Errorf("%q is not a node name (one subject token: no '.', '*', '>', '@' or space)", node)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func cutNode(key string) (name, node string, err error) {
|
||||
name, node, at := strings.Cut(key, "@")
|
||||
if !at {
|
||||
return key, "", nil
|
||||
}
|
||||
if strings.Contains(node, "@") {
|
||||
return "", "", fmt.Errorf("%q holds more than one '@': a key names at most one machine", key)
|
||||
}
|
||||
if err := CheckNode(node); err != nil {
|
||||
return "", "", fmt.Errorf("%q: %w", key, err)
|
||||
}
|
||||
return name, node, nil
|
||||
}
|
||||
|
||||
func tokens(key string, parts ...string) error {
|
||||
for _, p := range parts {
|
||||
if p == "" || strings.ContainsAny(p, ".*> \t\r\n") {
|
||||
return fmt.Errorf("%q: %q is not one subject token (no '.', '*', '>' or space)", key, p)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkTool refuses a tool name with a wildcard or white space; a dot is kept, as the tool runner serves it.
|
||||
func checkTool(key, tool string) error {
|
||||
if strings.ContainsAny(tool, "*> \t\r\n") || strings.HasPrefix(tool, ".") || strings.HasSuffix(tool, ".") {
|
||||
return fmt.Errorf("%q: %q is not one subject token (no '*', '>' or space)", key, tool)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
package calladdress
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Issue 774 notes 84-87 (the lease write's stop asked of `seat:node-service-manager.stop@novox-be` published on a subject
|
||||
// nobody serves and timed out after 30 s): a key names its machine after '@', and every other malformed key is refused
|
||||
// before anything is published, naming why.
|
||||
func TestAKeyIsParsedOrRefusedBeforeItIsPublished(t *testing.T) {
|
||||
for _, row := range []struct {
|
||||
key, want string // want: "seat|<seat>|<verb>|<node>", "mod|<module>|<tool>|<node>", or a refusal's words
|
||||
}{
|
||||
{"seat:node-service-manager.stop@novox", "seat|node-service-manager|stop|novox"},
|
||||
{"seat:operator-channel.tell", "seat|operator-channel|tell|"},
|
||||
{"mesh-build-go.fetch@ace", "mod|mesh-build-go|fetch|ace"},
|
||||
{"nats.nats_users", "mod|nats|nats_users|"},
|
||||
{"seat:node-service-manager.stop@novox@g14", "more than one '@'"},
|
||||
{"seat:node-service-manager.stop@", "not a node name"},
|
||||
{"seat:node-service-manager.stop@nov.ox", "not a node name"},
|
||||
{"seat:node-files.*", "not one subject token"},
|
||||
{"mesh-build-go.fe tch", "not one subject token"},
|
||||
{"mesh>build.fetch", "not one subject token"},
|
||||
{"seat:noverb", "names no seat and verb"},
|
||||
{"nodot", "is not <module>.<tool>"},
|
||||
{"", "is not <module>.<tool>"},
|
||||
} {
|
||||
a, err := Parse(row.key)
|
||||
got := ""
|
||||
switch {
|
||||
case err != nil:
|
||||
got = err.Error()
|
||||
if !strings.Contains(got, row.want) {
|
||||
t.Errorf("Parse(%q) refused %q, want %q", row.key, got, row.want)
|
||||
}
|
||||
continue
|
||||
case a.Seat != "":
|
||||
got = "seat|" + a.Seat + "|" + a.Verb + "|" + a.Node
|
||||
default:
|
||||
got = "mod|" + a.Module + "|" + a.Tool + "|" + a.Node
|
||||
}
|
||||
if got != row.want {
|
||||
t.Errorf("Parse(%q) = %s, want %s", row.key, got, row.want)
|
||||
}
|
||||
}
|
||||
if a, err := Parse("seat:node-files.read@g14"); err != nil || a.Key() != "seat:node-files.read" {
|
||||
t.Errorf("the key without its machine: %q, %v", a.Key(), err)
|
||||
}
|
||||
if err := CheckOwn("list@ace"); err != nil {
|
||||
t.Errorf("a module's own tool by its bare name was refused: %v", err)
|
||||
}
|
||||
if err := CheckOwn("li*st"); err == nil {
|
||||
t.Error("a bare name that is not one token was accepted")
|
||||
}
|
||||
}
|
||||
+6
-38
@@ -22,19 +22,8 @@ const MetaCaller = "mesh/caller"
|
||||
// parts are safe in a subject. Anything else the runtime named is not a caller and reads as Unproven.
|
||||
var busUser = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
|
||||
|
||||
// MetaCall is where in a `tools/call`'s `_meta` the runtime names the id of the call in flight, the id a ledger's
|
||||
// apply names so the runtime stamps the caller it proved as the operation's author (novox/hq ADR 0597 §7):
|
||||
// buscalls.MetaCall, written here too so this package imports nothing of buscalls.
|
||||
const MetaCall = "mesh/call"
|
||||
|
||||
// callID is the shape of a call's id: one subject-safe token, as the runtime makes it. Anything else reads as none.
|
||||
var callID = regexp.MustCompile(`^[A-Za-z0-9_-]{1,128}$`)
|
||||
|
||||
// inFlight is what the runtime said of one call while its tool runs.
|
||||
type inFlight struct{ caller, call string }
|
||||
|
||||
// calls holds the caller and the id of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → inFlight
|
||||
// calls holds the caller of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → string
|
||||
|
||||
func argsKey(args map[string]any) unsafe.Pointer {
|
||||
return reflect.ValueOf(args).UnsafePointer()
|
||||
@@ -48,32 +37,11 @@ func CallerOf(args map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).caller
|
||||
return c.(string)
|
||||
}
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// CallIDOf is the id of the call that was handed args, as the runtime named it under MetaCall: what a ledger's apply
|
||||
// names (buscalls.ApplyRequest.Call) so the operation's author is the caller the runtime proved. Empty for arguments
|
||||
// no call was handed, or a call the runtime named no id for; read from `_meta`, never from the arguments.
|
||||
func CallIDOf(args map[string]any) string {
|
||||
if args == nil {
|
||||
return ""
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(inFlight).call
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callIn is the call id a `tools/call`'s `_meta` names, or empty.
|
||||
func callIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCall].(string); ok && callID.MatchString(s) {
|
||||
return s
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// callerIn is the caller a `tools/call`'s `_meta` names, or Unproven.
|
||||
func callerIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCaller].(string); ok && busUser.MatchString(s) {
|
||||
@@ -82,10 +50,10 @@ func callerIn(meta map[string]any) string {
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// runAs runs a tool with the caller and the call's id held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller, call string, args map[string]any) (any, error) {
|
||||
// runAs runs a tool with the caller held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller string, args map[string]any) (any, error) {
|
||||
key := argsKey(args)
|
||||
calls.Store(key, inFlight{caller, call})
|
||||
calls.Store(key, caller)
|
||||
defer calls.Delete(key)
|
||||
return t.Run(args)
|
||||
}
|
||||
|
||||
+5
-143
@@ -3,37 +3,27 @@ package stdio
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/buscalls"
|
||||
)
|
||||
|
||||
// callWith serves one tool that answers the caller CallerOf reads from its arguments, sends it one tools/call
|
||||
// with the params given, and returns the caller it answered.
|
||||
func callWith(t *testing.T, params map[string]any) string {
|
||||
caller, _ := callAnswering(t, params)
|
||||
return caller
|
||||
}
|
||||
|
||||
// callAnswering is callWith answering the call's id CallIDOf read too.
|
||||
func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
servesAlone(t)
|
||||
t.Helper()
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("tracker", []Tool{{Name: "who", Run: func(args map[string]any) (any, error) {
|
||||
return map[string]any{"caller": CallerOf(args), "call": CallIDOf(args)}, nil
|
||||
return map[string]any{"caller": CallerOf(args)}, nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
defer inW.Close()
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": params})
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
lines := bufio.NewScanner(outR)
|
||||
done := make(chan [2]string, 1)
|
||||
done := make(chan string, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m struct {
|
||||
@@ -46,20 +36,19 @@ func callAnswering(t *testing.T, params map[string]any) (string, string) {
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
var answer struct {
|
||||
Caller string `json:"caller"`
|
||||
Call string `json:"call"`
|
||||
}
|
||||
if len(m.Result.Content) > 0 {
|
||||
_ = json.Unmarshal([]byte(m.Result.Content[0].Text), &answer)
|
||||
}
|
||||
done <- [2]string{answer.Caller, answer.Call}
|
||||
done <- answer.Caller
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case got := <-done:
|
||||
return got[0], got[1]
|
||||
return got
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle did not answer the call")
|
||||
return "", ""
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -72,28 +61,6 @@ func TestAToolReadsTheCallerTheRuntimeNamed(t *testing.T) {
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("the tool read caller %q, want the one the runtime named", got)
|
||||
}
|
||||
// The call's id is the runtime's too (ADR 0597 §7): read from `_meta`, never from the arguments, and none when it
|
||||
// is not one token.
|
||||
for _, c := range []struct {
|
||||
args, meta map[string]any
|
||||
want string
|
||||
}{
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools", MetaCall: "c-7Hq2"}, "c-7Hq2"},
|
||||
{map[string]any{}, map[string]any{MetaCaller: "shanks.node-tools"}, ""},
|
||||
{map[string]any{MetaCall: "c-forged"}, map[string]any{}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: "a b"}, ""},
|
||||
{map[string]any{}, map[string]any{MetaCall: 7}, ""},
|
||||
} {
|
||||
if _, call := callAnswering(t, map[string]any{"name": "who", "arguments": c.args, "_meta": c.meta}); call != c.want {
|
||||
t.Errorf("args %v, meta %v: the tool read call %q, want %q", c.args, c.meta, call, c.want)
|
||||
}
|
||||
}
|
||||
if CallIDOf(map[string]any{"x": 1}) != "" || CallIDOf(nil) != "" {
|
||||
t.Error("arguments no call was handed have a call id")
|
||||
}
|
||||
if MetaCall != buscalls.MetaCall {
|
||||
t.Errorf("the call id's place is %q here and %q in buscalls", MetaCall, buscalls.MetaCall)
|
||||
}
|
||||
}
|
||||
|
||||
// A caller named in the arguments is not the caller: an argument is the caller's to write.
|
||||
@@ -108,70 +75,6 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("with a caller in the arguments the tool read %q, want the runtime's", got)
|
||||
}
|
||||
// A module's trusted list of callers (issue 774, ADR 0563 §8) grants the callers it names as the runtime proved them:
|
||||
// a module on any machine, <machine>.<module> exactly, person.<name> exactly; no wildcard, never one unproven.
|
||||
callers, err := ParseCallers([]byte(`{"callers":["node-tools","ace.claude-code","person.jo","*"]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for caller, want := range map[string]bool{"shanks.node-tools": true, "ace.claude-code": true, "g14.claude-code": false,
|
||||
"person.jo": true, "person.node-tools": false, Unproven: false, "": false, "novox.mesh-vault": false} {
|
||||
if callers.Allows(caller) != want {
|
||||
t.Errorf("%q allowed: %v, want %v", caller, !want, want)
|
||||
}
|
||||
}
|
||||
if _, err := ParseCallers([]byte(`{"callers":["x"],"more":1}`)); err == nil {
|
||||
t.Error("a list with a field it does not know was read")
|
||||
}
|
||||
// A holder's list per verb (ADR 0627 §2): each verb's entries as above, plus every proved caller (AnyCaller) or
|
||||
// every operator's session (AnyPerson); an unproven caller never, a verb the holder does not serve refused.
|
||||
verbs, err := ParseVerbCallers(`{"publish":["mesh-build-go"],"versions":["*"],"release":["operator","sweep"]}`,
|
||||
[]string{"publish", "versions", "release", "remove"}, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range []struct {
|
||||
verb, caller string
|
||||
want bool
|
||||
}{{"publish", "ace.mesh-build-go", true}, {"publish", "ace.mesh-build-node", false}, {"versions", "g14.anything", true},
|
||||
{"versions", Unproven, false}, {"release", "person.jo", true}, {"release", "ace.operator", false},
|
||||
{"release", "novox.sweep", true}, {"remove", "novox.sweep", false}} {
|
||||
if got := verbs.Allowed(c.verb, c.caller) == nil; got != c.want {
|
||||
t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want)
|
||||
}
|
||||
}
|
||||
// A-295: an entry `seat:<seat>` allows the modules the part's holder function answers for the seat, and
|
||||
// nobody without holders.
|
||||
seats := VerbCallers{"again": {AnyPerson, "seat:node-tool-runner", "seat:mesh-healers"}}
|
||||
holders := func(seat string) []string {
|
||||
return map[string][]string{"node-tool-runner": {"node-tools"}, "mesh-healers": {"mesh-delivery-watchman"}}[seat]
|
||||
}
|
||||
for _, c := range []struct {
|
||||
caller string
|
||||
holders func(string) []string
|
||||
want bool
|
||||
}{{"ace.node-tools", holders, true}, {"g14.mesh-delivery-watchman", holders, true}, {"ace.mesh-build-go", holders, false},
|
||||
{"person.jo", holders, true}, {"ace.node-tools", nil, false}} {
|
||||
if got := seats.AllowedWith("again", c.caller, c.holders) == nil; got != c.want {
|
||||
t.Errorf("again by %q (holders %v) allowed: %v, want %v", c.caller, c.holders != nil, got, c.want)
|
||||
}
|
||||
}
|
||||
if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil {
|
||||
t.Error("a list naming a verb the holder does not serve was read")
|
||||
}
|
||||
if d, _ := ParseVerbCallers(" ", nil, VerbCallers{"publish": {"x"}}); len(d["publish"]) != 1 {
|
||||
t.Error("an empty value is not the holder's defaults")
|
||||
}
|
||||
// As mesh-config resolves the value: the object itself, or written as a string; null is the defaults.
|
||||
for _, raw := range []string{`{"publish":["a"]}`, `"{\"publish\":[\"a\"]}"`, `null`} {
|
||||
c, err := parseConfiguredVerbCallers(json.RawMessage(raw), []string{"publish"}, VerbCallers{"publish": {"a"}})
|
||||
if err != nil || c.Allowed("publish", "ace.a") != nil {
|
||||
t.Errorf("the config value %s read as %v, %v", raw, c, err)
|
||||
}
|
||||
}
|
||||
if ModuleOf("ace.mesh-build-go") != "mesh-build-go" {
|
||||
t.Error("ModuleOf does not read the module of a proved caller")
|
||||
}
|
||||
}
|
||||
|
||||
// A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is
|
||||
@@ -200,44 +103,3 @@ func TestArgumentsOfNoCallHaveNoCaller(t *testing.T) {
|
||||
t.Fatalf("CallerOf(nil) answered %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A holder's caller list is held (issue 1966): the runtime is asked once, by a watch; a later call never asks again, so
|
||||
// an ask that timed out refuses no verb, and a change that cannot be read keeps the list last read.
|
||||
func TestAHeldCallerListIsAskedOnceAndSurvivesAnUnreadableChange(t *testing.T) {
|
||||
verbs, defaults := []string{"publish"}, VerbCallers{"publish": {"a"}}
|
||||
var asks int
|
||||
var push func(ProvisionChange) error
|
||||
fail := true
|
||||
read := holdVerbCallers(func(key string, h func(ProvisionChange) error) error {
|
||||
asks++
|
||||
if fail {
|
||||
return errors.New("nats: timeout")
|
||||
}
|
||||
push = h
|
||||
return h(ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`{"publish":["b"]}`)})
|
||||
}, "callers", verbs, defaults)
|
||||
if _, err := read(); err == nil {
|
||||
t.Fatal("a list never read was answered")
|
||||
}
|
||||
fail = false
|
||||
c, err := read()
|
||||
if err != nil || c.Allowed("publish", "ace.b") != nil || asks != 2 {
|
||||
t.Fatalf("after the ask answered: %v, %v, asked %d times", c, err, asks)
|
||||
}
|
||||
for i := 0; i < 3; i++ {
|
||||
if _, err := read(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if asks != 2 {
|
||||
t.Errorf("a held list was asked again: %d asks", asks)
|
||||
}
|
||||
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"nothing":["x"]}`)})
|
||||
if c, err := read(); err != nil || c.Allowed("publish", "ace.b") != nil {
|
||||
t.Errorf("an unreadable change dropped the list last read: %v, %v", c, err)
|
||||
}
|
||||
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"publish":["c"]}`)})
|
||||
if c, _ := read(); c.Allowed("publish", "ace.c") != nil || c.Allowed("publish", "ace.b") == nil {
|
||||
t.Errorf("a change was not held: %v", c)
|
||||
}
|
||||
}
|
||||
|
||||
-207
@@ -1,207 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
|
||||
// writes a list setting into a file the module reads: `{"callers": [...]}`. An entry is a module's name (that module on
|
||||
// any machine), exactly <machine>.<module>, or exactly person.<name>. Each caller is the runtime's proof (CallerOf), never
|
||||
// an argument; nobody beyond the list is granted, a wildcard is no entry, and an unproven caller never matches.
|
||||
type Callers []string
|
||||
|
||||
// ParseCallers reads a callers file: the list under `callers`, and nothing else.
|
||||
func ParseCallers(raw []byte) (Callers, error) {
|
||||
var f struct {
|
||||
Callers Callers `json:"callers"`
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&f); err != nil {
|
||||
return nil, fmt.Errorf("the callers setting is not a list of callers: %v", err)
|
||||
}
|
||||
return f.Callers, nil
|
||||
}
|
||||
|
||||
// Allows says whether caller, as the runtime proved it, is one the list names.
|
||||
func (c Callers) Allows(caller string) bool {
|
||||
if caller == "" || caller == Unproven {
|
||||
return false
|
||||
}
|
||||
for _, entry := range c {
|
||||
if entry == "" || strings.ContainsAny(entry, "*> ") {
|
||||
continue
|
||||
}
|
||||
if entry == caller {
|
||||
return true
|
||||
}
|
||||
if !strings.Contains(entry, ".") {
|
||||
if _, module, ok := strings.Cut(caller, "."); ok && module == entry && !strings.HasPrefix(caller, "person.") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// VerbCallers is a holder's trusted list per verb (novox/hq ADR 0627 §2, design 51 *Who may call which verb*): for each
|
||||
// verb of the seats it holds, the callers that may call it, each entry read as Callers reads it. Two entries more stand
|
||||
// for a whole class: AnyCaller lets every proved caller call the verb (a read the mesh keeps open), AnyPerson every
|
||||
// operator's session the runtime proved as person.<name>. It names no seat and no verb: the holder gives its verbs and
|
||||
// its defaults, and keeps any rule of its own seat (who may never be on a list) beside it.
|
||||
type VerbCallers map[string][]string
|
||||
|
||||
// The entries of a VerbCallers list that stand for a class of callers.
|
||||
const (
|
||||
AnyCaller = "*"
|
||||
AnyPerson = "operator"
|
||||
)
|
||||
|
||||
// ParseVerbCallers reads a holder's config value of verb to callers; an empty value is defaults. A verb not among verbs
|
||||
// is refused, so a list that names a verb the holder does not serve is said at once.
|
||||
func ParseVerbCallers(raw string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return defaults, nil
|
||||
}
|
||||
var c VerbCallers
|
||||
if err := json.Unmarshal([]byte(raw), &c); err != nil {
|
||||
return nil, fmt.Errorf("callers is not a JSON object of verb to callers: %w", err)
|
||||
}
|
||||
for verb := range c {
|
||||
if !slices.Contains(verbs, verb) {
|
||||
return nil, fmt.Errorf("callers names the verb %q, which the holder does not serve", verb)
|
||||
}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// CallerSeatPrefix opens an entry naming a seat, `seat:<seat>`: every module holding it, so a list names no module (design 47
|
||||
// G A-210, A-295).
|
||||
const CallerSeatPrefix = "seat:"
|
||||
|
||||
// Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven
|
||||
// caller is never allowed, not even by AnyCaller. An entry `seat:<seat>` allows no one here (AllowedWith).
|
||||
func (c VerbCallers) Allowed(verb, caller string) error { return c.AllowedWith(verb, caller, nil) }
|
||||
|
||||
// AllowedWith is Allowed whose entries `seat:<seat>` allow a module caller whose module holders answers for that seat:
|
||||
// the part's holder function, which answers from what it last heard of the assigner's seat-holders view (design 47 G A-295),
|
||||
// so no call is made per check. A nil holders, or a seat it answers none for, allows no one by that entry.
|
||||
func (c VerbCallers) AllowedWith(verb, caller string, holders func(seat string) []string) error {
|
||||
if caller == "" || caller == Unproven {
|
||||
return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb)
|
||||
}
|
||||
person := strings.HasPrefix(caller, "person.")
|
||||
for _, entry := range c[verb] {
|
||||
if seat, ok := strings.CutPrefix(entry, CallerSeatPrefix); ok {
|
||||
if !person && holders != nil && slices.Contains(holders(seat), ModuleOf(caller)) {
|
||||
return nil
|
||||
}
|
||||
continue
|
||||
}
|
||||
switch entry {
|
||||
case AnyCaller:
|
||||
return nil
|
||||
case AnyPerson:
|
||||
if person {
|
||||
return nil
|
||||
}
|
||||
default:
|
||||
if (Callers{entry}).Allows(caller) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("%s: refused for %s: not on the holder's caller list for %s", verb, caller, verb)
|
||||
}
|
||||
|
||||
// ModuleOf is the module of a proved module caller, which the runtime names <machine>.<module>.
|
||||
func ModuleOf(proved string) string {
|
||||
if _, module, ok := strings.Cut(proved, "."); ok {
|
||||
return module
|
||||
}
|
||||
return proved
|
||||
}
|
||||
|
||||
// ConfiguredVerbCallers is the holder's list as mesh-config resolved its config value key for this machine (Config):
|
||||
// a JSON object of verb to callers, or that object written as a string; an unset or empty value is defaults. A value
|
||||
// that cannot be read is an error, never the defaults, so a broken list is said, not widened.
|
||||
func ConfiguredVerbCallers(key string, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
raw, err := Config(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
}
|
||||
return parseConfiguredVerbCallers(raw, verbs, defaults)
|
||||
}
|
||||
|
||||
// HoldVerbCallers is ConfiguredVerbCallers held: the returned reader asks the runtime once, by a watch of the config
|
||||
// value key (WatchConfig), and from then on answers the list it holds, which each change of the value replaces. A call
|
||||
// never asks the runtime again, so a runtime that is slow or away for a moment (a timed-out ask) refuses no verb; a
|
||||
// change that cannot be read keeps the list last read and is said on the bundle's standard error, so a broken list is
|
||||
// said and never widened. Only a reader that has not yet read any value answers an error, and its next call asks again.
|
||||
func HoldVerbCallers(key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
|
||||
return holdVerbCallers(WatchConfig, key, verbs, defaults)
|
||||
}
|
||||
|
||||
func holdVerbCallers(watch func(string, func(ProvisionChange) error) error, key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
|
||||
var start, mu sync.Mutex
|
||||
var held VerbCallers
|
||||
var bad error
|
||||
handle := func(c ProvisionChange) error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if c.Refused != "" {
|
||||
bad = fmt.Errorf("the config value %s is not answered: %s", key, c.Refused)
|
||||
} else if l, err := parseConfiguredVerbCallers(c.Value, verbs, defaults); err != nil {
|
||||
bad = fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
} else {
|
||||
held, bad = l, nil
|
||||
return nil
|
||||
}
|
||||
if held != nil {
|
||||
sayf("%v: the list last read stays", bad)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
read := func() (VerbCallers, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if held != nil {
|
||||
return held, nil
|
||||
}
|
||||
if bad != nil {
|
||||
return nil, bad
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
return func() (VerbCallers, error) {
|
||||
if l, err := read(); l != nil || err != nil {
|
||||
return l, err
|
||||
}
|
||||
start.Lock()
|
||||
defer start.Unlock()
|
||||
if l, err := read(); l != nil || err != nil {
|
||||
return l, err
|
||||
}
|
||||
if err := watch(key, handle); err != nil {
|
||||
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
|
||||
}
|
||||
l, err := read()
|
||||
if l == nil && err == nil {
|
||||
err = fmt.Errorf("the config value %s cannot be read: the runtime handed over no value", key)
|
||||
}
|
||||
return l, err
|
||||
}
|
||||
}
|
||||
|
||||
func parseConfiguredVerbCallers(raw json.RawMessage, verbs []string, defaults VerbCallers) (VerbCallers, error) {
|
||||
var text string
|
||||
if err := json.Unmarshal(raw, &text); err == nil || len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
|
||||
return ParseVerbCallers(text, verbs, defaults)
|
||||
}
|
||||
return ParseVerbCallers(string(raw), verbs, defaults)
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"slices"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The call a bundle reads its tool runner's capability words with, and the words of the families that sit beside
|
||||
// `mesh/publish` and `mesh/subscribe` (design 47 G §576, §622). The words of the bus-state families are in buscalls.
|
||||
const (
|
||||
// CapabilitiesCall is answered with {words}: the capability words the tool runner states.
|
||||
CapabilitiesCall = "mesh/capabilities"
|
||||
// CapPutSessions is the put and get sessions' word (§457, §518), CapFindingCalls the acknowledged publish and the
|
||||
// subscription for new messages the finding writer uses (§559). CapSeatSettleDelay (seat.go) is the third.
|
||||
CapPutSessions = "put-sessions"
|
||||
CapFindingCalls = "finding-calls"
|
||||
)
|
||||
|
||||
// capabilityWords are the words the tool runner states, read once at the first call that needs one and kept for as long
|
||||
// as the bundle runs: a bundle lives no longer than the tool runner that started it (design 47 G §576).
|
||||
type capabilityWords struct {
|
||||
mu sync.Mutex
|
||||
read bool
|
||||
words []string
|
||||
}
|
||||
|
||||
var capabilities capabilityWords
|
||||
|
||||
// get answers the words, asking once with ask. A tool runner of before G answers `mesh/capabilities` as an unknown call,
|
||||
// which reads as no word stated; any other failure is returned and asked again at the next call.
|
||||
func (c *capabilityWords) get(ask func() (json.RawMessage, error)) ([]string, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.read {
|
||||
return c.words, nil
|
||||
}
|
||||
raw, err := ask()
|
||||
switch {
|
||||
case errors.Is(err, errors.ErrUnsupported):
|
||||
c.words = nil
|
||||
case err != nil:
|
||||
return nil, fmt.Errorf("reading the tool runner's capability words: %w", err)
|
||||
default:
|
||||
var a struct {
|
||||
Words []string `json:"words"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &a); err != nil {
|
||||
return nil, fmt.Errorf("the tool runner's capability words: %w", err)
|
||||
}
|
||||
c.words = a.Words
|
||||
}
|
||||
c.read = true
|
||||
return c.words, nil
|
||||
}
|
||||
|
||||
// require is nil when word is among the stated ones, else an error naming the word and the node (design 47 G §488, §576).
|
||||
func (c *capabilityWords) require(word string, ask func() (json.RawMessage, error)) error {
|
||||
words, err := c.get(ask)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if slices.Contains(words, word) {
|
||||
return nil
|
||||
}
|
||||
node := os.Getenv("MESH_NODE")
|
||||
if node == "" {
|
||||
node = "unknown"
|
||||
}
|
||||
return fmt.Errorf("the tool runner of node %s does not state the capability word %q: a call of its family is refused until it does", node, word)
|
||||
}
|
||||
|
||||
// RequireCapability fails at once, naming the word and the node, when the tool runner this bundle runs under does not
|
||||
// state word; it reads the words once (design 47 G §488, §576).
|
||||
func RequireCapability(word string) error {
|
||||
return capabilities.require(word, func() (json.RawMessage, error) {
|
||||
c := awaitChannel()
|
||||
if c == nil {
|
||||
return nil, errors.New("a capability is read through the runtime that launched the bundle, and Serve has not run")
|
||||
}
|
||||
return c.ask(CapabilitiesCall, map[string]any{}, 30*time.Second)
|
||||
})
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Design 47 G §576: the words are read once, an unknown-call answer reads as none, and a word not stated fails naming it
|
||||
// and the node (from MESH_NODE).
|
||||
func subACapabilityWordIsReadOnceAndNamedWhenNotStated(t *testing.T) {
|
||||
t.Setenv("MESH_NODE", "ace")
|
||||
asked := 0
|
||||
var c capabilityWords
|
||||
ask := func() (json.RawMessage, error) { asked++; return json.RawMessage(`{"words":["put-sessions"]}`), nil }
|
||||
if err := c.require(CapPutSessions, ask); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := c.require(CapFindingCalls, ask)
|
||||
if err == nil || !strings.Contains(err.Error(), CapFindingCalls) || !strings.Contains(err.Error(), "ace") || asked != 1 {
|
||||
t.Errorf("a word not stated: %v after %d asks", err, asked)
|
||||
}
|
||||
var old capabilityWords
|
||||
unknown := func() (json.RawMessage, error) {
|
||||
asked++
|
||||
return nil, &refused{message: "no mesh/capabilities", code: CodeNoMethod}
|
||||
}
|
||||
if err := old.require(CapPutSessions, unknown); err == nil || !strings.Contains(err.Error(), CapPutSessions) {
|
||||
t.Errorf("a tool runner of before G stated %v", err)
|
||||
}
|
||||
var failing capabilityWords
|
||||
if err := failing.require(CapPutSessions, func() (json.RawMessage, error) { return nil, errors.New("pipe closed") }); err == nil ||
|
||||
!strings.Contains(err.Error(), "pipe closed") || failing.read {
|
||||
t.Errorf("a failed read: %v, kept %v", err, failing.read)
|
||||
}
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// novox/hq issue 213 (ADR 0188 §1, §3): a module's own Go service is a bundle the host runs as a
|
||||
// process, not an image. Each test holds one thing that had to change in the composer for the
|
||||
// controller to be declared that way.
|
||||
|
||||
const aServiceDigest = "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
|
||||
|
||||
// aServiceModule is the controller's shape in miniature: it answers tools of its own, its code is a
|
||||
// Go bundle a process runs as an account it declares, its secrets belong to that account, it
|
||||
// prepares its state, and its process replaces the container it used to run as.
|
||||
func aServiceModule(t *testing.T) Manifest {
|
||||
t.Helper()
|
||||
raw := `{
|
||||
"module": "svc", "version": "1", "tools": ["status"], "prepares": true,
|
||||
"own-secrets": {"store": "${dir:state}/store"},
|
||||
"secrets-owner": "svc",
|
||||
"resources": [
|
||||
{"id": "state", "type": "directory", "mode": "0700", "place": "mesh", "owner": "svc"},
|
||||
{"id": "service", "type": "process", "name": "svc", "artifact": "code",
|
||||
"run": ["./svc", "serve"], "user": "svc", "replaces": ["server"],
|
||||
"env": {"SVC_STORE_FILE": "${dir:state}/store", "SVC_STORE_PORT": "${seat:mesh-store:5432}"}},
|
||||
{"id": "account", "type": "user", "name": "svc", "shell": "/usr/bin/nologin", "home": "/var/lib/svc"}
|
||||
],
|
||||
"build": {"artifacts": [{"name": "code", "kind": "bundle", "language": "go", "system": "arch",
|
||||
"from": "cmd/svc", "binary": "svc"}]}
|
||||
}`
|
||||
m, err := decodeForTest([]byte(raw))
|
||||
if err != nil {
|
||||
t.Fatalf("the service's manifest is refused: %v", err)
|
||||
}
|
||||
resolved, err := m.Resolve([]Built{{Name: "code", Kind: ArtifactBundle,
|
||||
Reference: ArtifactStoreScheme + "svc/code@" + aServiceDigest, Digest: aServiceDigest}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
func indexOf(out []map[string]any, id string) int {
|
||||
for i, r := range out {
|
||||
if r["id"] == id {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// A module that declares tools has every bundle served by the node's runtime unless it says
|
||||
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
|
||||
// by its own process; launched a second time by the runtime it would be a second controller
|
||||
// pretending to be an MCP server.
|
||||
func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
m := aServiceModule(t)
|
||||
if len(m.Bundles) != 1 {
|
||||
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
|
||||
}
|
||||
if loads := m.Bundles[0].Loads; len(loads) != 0 {
|
||||
t.Fatalf("the runtime would launch the service's own bundle as tools: %v", loads)
|
||||
}
|
||||
// And a bundle no resource runs still is served, as a module declaring tools always had it.
|
||||
tools := Manifest{Module: "t", Version: "1", Tools: []string{"x"},
|
||||
Build: &Build{Artifacts: []Artifact{{Name: "tools", Kind: ArtifactBundle, Language: "go",
|
||||
System: "arch", From: "cmd/t"}}}}
|
||||
resolved, err := tools.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle,
|
||||
Reference: ArtifactStoreScheme + "t/tools@" + aServiceDigest, Digest: aServiceDigest}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if loads := resolved.Bundles[0].Loads; len(loads) != 1 || loads[0] != "t" {
|
||||
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
|
||||
}
|
||||
}
|
||||
|
||||
// TestABundleItsOwnProcessRunsIsNotServedByTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
|
||||
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
|
||||
t.Run("ABundleItsOwnProcessRunsIsNotServedByTheRuntime", subABundleItsOwnProcessRunsIsNotServedByTheRuntime)
|
||||
t.Run("TheBusNamesOfAnEntryAreOneRule", subTheBusNamesOfAnEntryAreOneRule)
|
||||
t.Run("AStarEmitterHearsEachDeclaringModulesUnrestrictedStream", subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream)
|
||||
}
|
||||
@@ -1,68 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// AdoptionPrefix is the id prefix of what the mesh declares of its own on an adopted node. It is
|
||||
// never a module's, so none of it is ever held as found.
|
||||
const AdoptionPrefix = "adoption."
|
||||
|
||||
// Where an opening admits from, on the wire.
|
||||
const (
|
||||
OpeningFromEverywhere = "everywhere"
|
||||
OpeningFromMesh = "mesh"
|
||||
)
|
||||
|
||||
// The two paths a packet reaches a port by: received by the machine, or forwarded to a
|
||||
// container that publishes it.
|
||||
const (
|
||||
PathIncoming = "incoming"
|
||||
PathForwarded = "forwarded"
|
||||
)
|
||||
|
||||
// Guard resources: the refusal-only table, the unit that loads it, and that unit running.
|
||||
const (
|
||||
GuardPath = "/etc/mesh/guard.nft"
|
||||
GuardUnit = "mesh-guard.service"
|
||||
// GuardUnitPath is where the unit is written.
|
||||
GuardUnitPath = "/etc/systemd/system/" + GuardUnit
|
||||
)
|
||||
|
||||
// GuardPackage is the package that carries nft.
|
||||
const GuardPackage = "nftables"
|
||||
|
||||
// Published is every port the given containers publish on the machine, by protocol and machine
|
||||
// port, mapped to the container's own port. A mapping bound to loopback is left out: nothing off
|
||||
// the machine reaches it, forwarded or not.
|
||||
func Published(resources []map[string]any) map[string]map[int]int {
|
||||
out := map[string]map[int]int{}
|
||||
for _, r := range resources {
|
||||
if fmt.Sprint(r["type"]) != "container" {
|
||||
continue
|
||||
}
|
||||
listed, _ := r["ports"].([]any)
|
||||
for _, entry := range listed {
|
||||
written := strings.TrimSpace(fmt.Sprint(entry))
|
||||
protocol := "tcp"
|
||||
if cut := strings.LastIndex(written, "/"); cut >= 0 {
|
||||
protocol = written[cut+1:]
|
||||
}
|
||||
// Indexed from the end, so an IPv6 address's own colons never shift the ports.
|
||||
outer, inner, address, ok := mapping(written)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
switch strings.Trim(address, "[]") {
|
||||
case "127.0.0.1", "localhost", "::1":
|
||||
continue
|
||||
}
|
||||
if out[protocol] == nil {
|
||||
out[protocol] = map[int]int{}
|
||||
}
|
||||
out[protocol][outer] = inner
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// novox/hq ADR 0100: on an adopted node the found firewall stays in force. The mesh declares
|
||||
// openings where it would have loaded a filter, and guards its own ports in a table that only
|
||||
// refuses.
|
||||
|
||||
// hub is the private network's generator on the hub: it opens the hub's port from anywhere.
|
||||
type hub struct{}
|
||||
|
||||
// anAdoptedAnchor is the control-node's set: the store, the bus, the registry, the private network,
|
||||
// a served module and the filter module.
|
||||
func anAdoptedAnchor() Resolution {
|
||||
return Resolution{Node: "anchor", Modules: []Manifest{
|
||||
{Module: "network", Computed: "overlay"},
|
||||
{Module: "postgres", Guards: []int{5432},
|
||||
Listens: []Listening{{Port: 5432, From: FromMesh}},
|
||||
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-store",
|
||||
"ports": []any{"5432:5432"}}}},
|
||||
{Module: "lavinmq", Guards: []int{15672},
|
||||
Listens: []Listening{{Port: 5671, From: FromMesh}, {Port: 5672, From: FromMesh}},
|
||||
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker",
|
||||
"ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}}}},
|
||||
{Module: "distribution",
|
||||
Listens: []Listening{{Port: 5000, From: FromMesh}},
|
||||
Resources: []map[string]any{{"id": "store", "type": "container", "name": "registry",
|
||||
"ports": []any{"5000"}}}},
|
||||
{Module: "hello-web",
|
||||
Listens: []Listening{{Port: 8080, From: FromEverywhere}},
|
||||
Resources: []map[string]any{{"id": "server", "type": "container", "name": "hello-web",
|
||||
"ports": []any{"8080"}}}},
|
||||
{Module: "helper", Listens: []Listening{{Port: 9000, From: FromMachine}}},
|
||||
{Module: "nftables", Filtering: &Filtering{Into: "/etc/nftables.conf"},
|
||||
Resources: []map[string]any{{"id": "load", "type": "service", "unit": "mesh-filter.service",
|
||||
"state": "running", "restart-on": []any{"filtering"}}}},
|
||||
}}
|
||||
}
|
||||
|
||||
func byID(resources []map[string]any) map[string]map[string]any {
|
||||
out := map[string]map[string]any{}
|
||||
for _, r := range resources {
|
||||
out[r["id"].(string)] = r
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// A mapping bound to loopback is not published to anything off the machine — in either address
|
||||
// family — and an address's own colons never shift the ports.
|
||||
func TestPublishedLeavesOutLoopbackInBothFamilies(t *testing.T) {
|
||||
got := Published([]map[string]any{{"type": "container", "ports": []any{
|
||||
"127.0.0.1:15672:15672", "[::1]:8080:80", "localhost:9090:90",
|
||||
"[::]:8443:443", "0.0.0.0:5000:5000", "5353:53/udp"}}})
|
||||
want := map[string]map[int]int{"tcp": {8443: 443, 5000: 5000}, "udp": {5353: 53}}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("published is %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
@@ -1,77 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/internal/beside"
|
||||
)
|
||||
|
||||
// **The word does not come back through a manifest** (novox/hq ADR 0131). A module that wants
|
||||
// messaging wants the mesh's bus, reached through the sdk and named by the `mesh-broker` seat. Naming
|
||||
// the old wire protocol asks for the one server being retired, so both directions are refused at the
|
||||
// parser — this is judged from the manifest alone, no store needed.
|
||||
|
||||
// And the catalogue (internal/beside) names it nowhere — the three modules that did are removed under
|
||||
// design 28 task 5.4, not converted.
|
||||
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
|
||||
modules, err := filepath.Glob(filepath.Join(beside.Catalogue(t), "*", "module.json"))
|
||||
modules0 := modules
|
||||
if err != nil || len(modules) == 0 {
|
||||
t.Fatalf("no manifests in the catalogue, so this proved nothing: %v", err)
|
||||
}
|
||||
for _, path := range modules {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(raw), `"amqp"`) {
|
||||
t.Errorf("%s names amqp, which is not a provision (novox/hq ADR 0131)",
|
||||
filepath.Base(filepath.Dir(path)))
|
||||
}
|
||||
}
|
||||
// Nor does this package's code name any module the catalogue holds (ADR 0601 §4): what a module is, its manifest
|
||||
// says. The names are read from the catalogue's module directories, never from a list here; every string literal of
|
||||
// the package's non-test files is read, and a module's name is refused unless it is also a seat's name or one of the
|
||||
// words below, which name something else.
|
||||
t.Run("the package names no module", func(t *testing.T) {
|
||||
modules := map[string]bool{}
|
||||
for _, path := range modules0 {
|
||||
modules[filepath.Base(filepath.Dir(path))] = true
|
||||
}
|
||||
// Words that are a module's name and name something else here: a language, an environment's rendering, a package.
|
||||
elsewhere := map[string]bool{"go": true, "systemd": true, "nftables": true}
|
||||
fset := token.NewFileSet()
|
||||
files, _ := filepath.Glob("*.go")
|
||||
for _, f := range files {
|
||||
if strings.HasSuffix(f, "_test.go") {
|
||||
continue
|
||||
}
|
||||
parsed, err := parser.ParseFile(fset, f, nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ast.Inspect(parsed, func(n ast.Node) bool {
|
||||
lit, ok := n.(*ast.BasicLit)
|
||||
if !ok || lit.Kind != token.STRING {
|
||||
return true
|
||||
}
|
||||
word, err := strconv.Unquote(lit.Value)
|
||||
if err != nil || !modules[word] || elsewhere[word] {
|
||||
return true
|
||||
}
|
||||
if _, seat := SeatNamed(word); seat {
|
||||
return true
|
||||
}
|
||||
t.Errorf("%s names the module %s", fset.Position(lit.Pos()), word)
|
||||
return true
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// ArtifactStoreScheme marks a reference to something in the mesh's artifact store, kept without
|
||||
// the store's address. How a reference is recorded, read and routed (the registry seat's form beside this one) is the
|
||||
// registry clients' (mesh-delivery's contracts, `containerimageregistryclient/reference`; ADR 0634 §4).
|
||||
const ArtifactStoreScheme = ArtifactStoreProvision + "://"
|
||||
@@ -1,6 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import "strings"
|
||||
|
||||
// digest is an artifact digest the tests build references with.
|
||||
var digest = "sha256:" + strings.Repeat("d", 64)
|
||||
@@ -1,4 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// BackupNamespace is the placeholder kind of the machine's backup lines and data.
|
||||
const BackupNamespace = "backup"
|
||||
@@ -1,10 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// A machine's backup lines are placement (design 57 L3, issue 794): the holder of node-backup names them as
|
||||
// `${backup:lines}` and `${backup:data}`, which a handed holder has its client fill from the plan and an unhanded
|
||||
// one has the controller fill as it fills `${contribution:node-backup:backup}` and `${...:data}`: the same text.
|
||||
|
||||
func dataModule(name string) string {
|
||||
return `{"module":"` + name + `","version":"1","resources":[{"id":"d","type":"directory","mode":"0700"}],
|
||||
"data":{"own":[{"id":"d","path":"${dir:d}","class":"irreplaceable","why":"the ` + name + ` data"}]}}`
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Naming the image directly is the way out, and must stay accepted.
|
||||
func TestAnArtifactStoreThatNamesItsImageIsAccepted(t *testing.T) {
|
||||
_, err := decodeForTest([]byte(`{"module":"registry","version":"1",` +
|
||||
`"provides":[{"name":"artifact-store","scope":"mesh"}],` +
|
||||
`"resources":[{"id":"store","type":"container","name":"mesh-registry",` +
|
||||
`"image":"registry@sha256:` +
|
||||
`266f282fabd7cd3df053ee7c658c77b42380d1a2f0d8e5a1c0d7a6d5b5c4a3b2",` +
|
||||
`"ports":["5000:5000"]}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("the one way an artifact store can be delivered was refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// And an ordinary module still builds whatever it likes.
|
||||
func TestAModuleThatDoesNotProvideTheStoreStillBuilds(t *testing.T) {
|
||||
_, err := decodeForTest([]byte(`{"module":"forge","version":"1",` +
|
||||
`"build":{"artifacts":[{"name":"forge","kind":"upstream","from":"gitea/gitea:1.22"}]},` +
|
||||
`"resources":[{"id":"run","type":"container","name":"forge","artifact":"forge"}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("an ordinary module was caught by a rule about the artifact store: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A mapping that names an address still names the port, and the machine side is still the middle.
|
||||
//
|
||||
// The foundation bundle writes "127.0.0.1:5432:5432" today, so the shape is not hypothetical.
|
||||
// Found in review: the first cut split on the first colon, read "127.0.0.1" as the machine port,
|
||||
// failed to parse it, and silently skipped the mapping — which put the filter back on the
|
||||
// declared port, the exact fault MachineSide was written to end.
|
||||
func TestAnAddressedMappingStillNamesThePort(t *testing.T) {
|
||||
m := Manifest{Module: "store", Resources: []map[string]any{
|
||||
{"type": "container", "id": "server", "ports": []any{"127.0.0.1:8080:80"}},
|
||||
}}
|
||||
for _, named := range []int{8080, 80} {
|
||||
at, mayAssign := m.MachineSide(named)
|
||||
if mayAssign {
|
||||
t.Fatalf("%d was reassigned though the manifest published it explicitly", named)
|
||||
}
|
||||
if at != 8080 {
|
||||
t.Fatalf("naming %d gave %d; the machine side of 127.0.0.1:8080:80 is 8080", named, at)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// KeptBinding is one consumer this resolution would have moved, and did not.
|
||||
type KeptBinding struct {
|
||||
// Machine is where the consumer runs, and Consumer the module there that is bound.
|
||||
Machine string
|
||||
Consumer string
|
||||
// Provision is what it is bound for.
|
||||
Provision string
|
||||
// Bound is the provider it was recorded at, and still is; Would is the one the resolution chose.
|
||||
Bound Chosen
|
||||
Would Chosen
|
||||
}
|
||||
|
||||
// String is the condition's sentence: the move, where the data is, and the act that confirms it.
|
||||
func (k KeptBinding) String() string {
|
||||
return fmt.Sprintf("would move %s's %s from %s to %s — its data is on %s; kept there. "+
|
||||
"`pin %s %s %s %s` to confirm a move (and move the data first)",
|
||||
k.Consumer, k.Provision, k.Bound, k.Would, k.Bound, k.Machine, k.Provision, k.Would.Node, k.Would.Module)
|
||||
}
|
||||
|
||||
// Unbound is one consumer that still asks for a provision and whose own resolution binds it to
|
||||
// another provider than the one a pair credential on record was made with (novox/hq issue 274).
|
||||
//
|
||||
// **A provider is granted exactly the consumers bound to it.** The credential from the old provider
|
||||
// stays on record — it is the key to a login that provider keeps, disabled, with the consumer's data,
|
||||
// until a person deletes it with `cleanup delete` (ADR 0230), and a pin back must find it — but it is
|
||||
// no longer granted, so the provider stops being asked for it and retires it. Said on every plan and
|
||||
// push of the provider, so a credential the mesh keeps and does not use is never kept silently.
|
||||
type Unbound struct {
|
||||
// Provision is what was required, Provider the machine the credential on record is from.
|
||||
Provision string `json:"provision"`
|
||||
Provider string `json:"provider"`
|
||||
// Consumer is the machine, Module the module on it that requires it, Local the credential's
|
||||
// name inside it where it keeps several (ADR 0094).
|
||||
Consumer string `json:"consumer"`
|
||||
Module string `json:"module"`
|
||||
Local string `json:"local,omitempty"`
|
||||
// BoundTo is every provider the consumer's resolution binds this credential to now; empty when
|
||||
// it binds it nowhere — the module asks for the provision under other local names.
|
||||
BoundTo []string `json:"bound_to,omitempty"`
|
||||
}
|
||||
|
||||
func (u Unbound) String() string {
|
||||
who := u.Module
|
||||
if u.Local != "" {
|
||||
who += " (as " + u.Local + ")"
|
||||
}
|
||||
now := "is bound to no provider under that name"
|
||||
if len(u.BoundTo) > 0 {
|
||||
now = "is bound to " + strings.Join(u.BoundTo, ", ")
|
||||
}
|
||||
return fmt.Sprintf("%s on %s %s for %s, not to %s — %s no longer grants it, so it retires that "+
|
||||
"login and keeps its data until `cleanup delete` (ADR 0230); the credential from %s stays on "+
|
||||
"record while that login does", who, u.Consumer, now, u.Provision, u.Provider, u.Provider, u.Provider)
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
func orNothing(names []string) string {
|
||||
if len(names) == 0 {
|
||||
return "nothing"
|
||||
}
|
||||
return join(names)
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A name too long to stay distinct is refused before anything is created.
|
||||
func TestAnIdentityTooLongToBeDistinctIsRefused(t *testing.T) {
|
||||
if err := CheckIdentity("anchor", "keycloak"); err != nil {
|
||||
t.Fatalf("an ordinary consumer was refused: %v", err)
|
||||
}
|
||||
if err := CheckIdentity(strings.Repeat("n", 40), strings.Repeat("m", 40)); err == nil {
|
||||
t.Fatal("an identity that providers would shorten was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
// Two modules on one node get two identities, which is the whole point of both issues.
|
||||
func TestTwoModulesOnOneNodeAreTwoIdentities(t *testing.T) {
|
||||
if ConsumerIdentity("anchor", "gitea") == ConsumerIdentity("anchor", "keycloak") {
|
||||
t.Fatal("two consumers on one machine share an identity")
|
||||
}
|
||||
// And one module on two nodes, likewise.
|
||||
if ConsumerIdentity("anchor", "gitea") == ConsumerIdentity("laptop", "gitea") {
|
||||
t.Fatal("one module on two machines shares an identity")
|
||||
}
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A consumer of a provision that keeps its data stays bound where its data is (novox/hq ADR 0232,
|
||||
// issue 273).
|
||||
//
|
||||
// The incident, exactly: a machine runs its own store and five consumers of it; the mesh's store seat
|
||||
// is held by the store on another machine. Issue 258's rule — the seat's holder elsewhere answers
|
||||
// before this machine's own provider — re-bound all five to the holder, each was made a fresh, empty
|
||||
// database there, and nothing said so.
|
||||
|
||||
func TestAnOfferSaysWhetherItKeepsConsumerData(t *testing.T) {
|
||||
var o Offer
|
||||
if err := json.Unmarshal([]byte(`{"name":"wildcard-resolution","scope":"mesh","keeps-consumer-data":false}`), &o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if o.KeepsConsumerData == nil || *o.KeepsConsumerData {
|
||||
t.Fatalf("read %+v", o)
|
||||
}
|
||||
out, err := json.Marshal(o)
|
||||
if err != nil || !strings.Contains(string(out), `"keeps-consumer-data":false`) {
|
||||
t.Fatalf("wrote %s, %v", out, err)
|
||||
}
|
||||
yes, no := true, false
|
||||
granting := Manifest{Module: "g", Provides: []Offer{{Name: "p", Scope: ScopeMesh}}, Grants: map[string]string{"p": "/g"}}
|
||||
if !granting.KeepsConsumerData("p") {
|
||||
t.Error("a provider granting each consumer a credential keeps what it writes, unsaid")
|
||||
}
|
||||
if (Manifest{Module: "r", Provides: []Offer{{Name: "p", Scope: ScopeMesh}}}).KeepsConsumerData("p") {
|
||||
t.Error("a provider granting nothing keeps nothing, unsaid")
|
||||
}
|
||||
granting.Provides[0].KeepsConsumerData = &no
|
||||
if granting.KeepsConsumerData("p") {
|
||||
t.Error("what an offer says, it gets")
|
||||
}
|
||||
said := Manifest{Module: "s", Provides: []Offer{{Name: "p", Scope: ScopeMesh, KeepsConsumerData: &yes}}}
|
||||
if !said.KeepsConsumerData("p") || !KeepsConsumerData(map[string]Manifest{"s": said, "r": {Module: "r",
|
||||
Provides: []Offer{{Name: "p", Scope: ScopeMesh, KeepsConsumerData: &no}}}}, "p") {
|
||||
t.Error("a name any provider says keeps data keeps data")
|
||||
}
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The old broker is gone from the catalogue (novox/hq ADR 0131, design 28 task 5.4), so it is no
|
||||
// longer a fixture here. That two eligible holders stand beside each other with one on record is
|
||||
// pinned in holdings_test.go against manifests this package owns.
|
||||
|
||||
// **A seat and the interface it delivers are different names, and renaming one must not rename
|
||||
// the other** (novox/hq ADR 0118). This nearly went wrong: the seats were renamed to the `mesh-*`
|
||||
// prefix, and a blanket search-and-replace also renamed `npm-package-registry` and `git` where
|
||||
// they are *provisions* — which a consumer requires and a provider offers. The tests failed with
|
||||
// "the package registry is served on <nil>", which does not say "you renamed an interface".
|
||||
func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
|
||||
for _, pair := range []struct{ seat, delivers string }{
|
||||
{"git", "git"},
|
||||
{"npm-package-registry", "npm-package-registry"},
|
||||
// The store's seats deliver nothing since the provision artifact-store retired (novox/hq ADR 0632 §2): the
|
||||
// store's address is its holder's listen, read through the seat.
|
||||
{"mesh-artifact-store", ""},
|
||||
{"container-image-registry", ""},
|
||||
{"oci-artifact-registry", ""},
|
||||
{"mesh-store", "postgres-database"},
|
||||
{"mesh-broker", "mesh-bus"},
|
||||
} {
|
||||
s, known := SeatNamed(pair.seat)
|
||||
if !known {
|
||||
t.Fatalf("%q is not a seat", pair.seat)
|
||||
}
|
||||
if s.Delivers != pair.delivers {
|
||||
t.Errorf("the %s seat delivers %q, expected %q — renaming the seat moved the "+
|
||||
"interface with it, and every consumer requiring it would stop resolving",
|
||||
pair.seat, s.Delivers, pair.delivers)
|
||||
}
|
||||
// **Three of these deliberately share a name with what they deliver**, and that is not an
|
||||
// incomplete rename. Renaming a seat that delivers a provision cascades to every consumer
|
||||
// requiring it, with a mesh-wide window where a holder stops resolving mid-flight — so the
|
||||
// trunk deferred exactly those three (novox/hq ADR 0121) while renaming the node-scoped ones.
|
||||
// What this test is for is the other direction: that renaming a seat never moves the
|
||||
// interface, which once produced "the package registry is served on <nil>".
|
||||
}
|
||||
}
|
||||
|
||||
// A manifest written against an old seat name is told what it became rather than refused as
|
||||
// unknown. **That map is the controller's store now, not this package** (novox/hq ADR 0122): a
|
||||
// rename is a row, so the courtesy survives a rename nobody recompiled for. Checked where the
|
||||
// table is read, not here, where there is no longer a hardcoded list to check against.
|
||||
@@ -1,35 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Where the answer to a requirement is allowed to live.
|
||||
//
|
||||
// A shell, a display server and a private network have to be on the machine that needs them. A
|
||||
// database does not — it runs somewhere and is reached over the network. Both were written
|
||||
// `requires`, so both were answered the same way, and the second answer was to install PostgreSQL
|
||||
// on every machine that runs a web application.
|
||||
|
||||
// onNetwork is a set of providers, all of them reachable. Written as a helper because a machine
|
||||
// that cannot reach the one answering its requirement is its own case, tested separately.
|
||||
// reachable is this machine, on the private network.
|
||||
func reachable() Node {
|
||||
n := workstation()
|
||||
n.At = "workstation.internal"
|
||||
return n
|
||||
}
|
||||
|
||||
func TestTheShortFormStillMeansHere(t *testing.T) {
|
||||
// `"provides": ["shell"]` must keep meaning what it meant, or every existing manifest
|
||||
// silently changes meaning.
|
||||
m, err := decodeForTest([]byte(`{"module":"zsh","version":"1","provides":["shell"]}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(m.Provides) != 1 || m.Provides[0].At() != ScopeNode {
|
||||
t.Fatalf("a plain name is no longer node-scoped: %v", m.Provides)
|
||||
}
|
||||
}
|
||||
|
||||
// Being told, which is the difference between knowing and being able to.
|
||||
@@ -1,83 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// BucketShare is one entry of `shares` on a module's requirement of `s3-bucket` (novox/hq ADR 0468 §8): who else may
|
||||
// put or get in the consumer's bucket, and with which methods. Not the NFS shares of shares.go (ADR 0263).
|
||||
//
|
||||
// "requires": [{"name": "s3-bucket", "shares": [{"to": "seat:node-files", "methods": ["put"]},
|
||||
// {"to": "print-and-scan", "methods": ["get"]}]}]
|
||||
//
|
||||
// `to` is a module name, or `seat:<seat name>`. `methods` is a non-empty subset of get and put, without a repeat.
|
||||
// Shares are allowed on `s3-bucket` only, and no two entries name the same `to`.
|
||||
type BucketShare struct {
|
||||
To string `json:"to"`
|
||||
Methods []string `json:"methods"`
|
||||
}
|
||||
|
||||
// BucketRequirement is the requirement `shares` may be written on.
|
||||
const BucketRequirement = "s3-bucket"
|
||||
|
||||
// RequireSharesKey is where a stored manifest keeps the shares of its requirements: beside `requires`, which stays a
|
||||
// plain list of names, for the reason OptionalRequiresKey is beside it. An older controller sets the unknown key
|
||||
// aside instead of failing the catalogue.
|
||||
const RequireSharesKey = "requires-shares"
|
||||
|
||||
// SharesValue is the key of the value a consumer gives the provider of s3-bucket in its grant: the list of
|
||||
// {to, methods} sorted by `to`. A consumer's own `contributes` may not name it.
|
||||
const SharesValue = "shares"
|
||||
|
||||
var shareTarget = regexp.MustCompile(`^(seat:)?[a-z0-9][a-z0-9-]*$`)
|
||||
|
||||
// checkBucketShares refuses a malformed `shares` of one requirement, naming the entry and ADR 0468 §8.
|
||||
func checkBucketShares(requirement string, shares []BucketShare) error {
|
||||
if len(shares) == 0 {
|
||||
return nil
|
||||
}
|
||||
if requirement != BucketRequirement {
|
||||
return fmt.Errorf("requires %s: `shares` is allowed on %s only (novox/hq ADR 0468 §8)", requirement, BucketRequirement)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for i, s := range shares {
|
||||
at := fmt.Sprintf("requires %s: shares[%d] (to %q)", requirement, i, s.To)
|
||||
if !shareTarget.MatchString(s.To) {
|
||||
return fmt.Errorf("%s: `to` is a module name or seat:<seat name> (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
if seen[s.To] {
|
||||
return fmt.Errorf("%s: a second entry for the same `to` (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
seen[s.To] = true
|
||||
if len(s.Methods) == 0 {
|
||||
return fmt.Errorf("%s: `methods` is empty; name get, put or both (novox/hq ADR 0468 §8)", at)
|
||||
}
|
||||
used := map[string]bool{}
|
||||
for _, method := range s.Methods {
|
||||
if method != "get" && method != "put" {
|
||||
return fmt.Errorf("%s: method %q is not get or put (novox/hq ADR 0468 §8)", at, method)
|
||||
}
|
||||
if used[method] {
|
||||
return fmt.Errorf("%s: method %q twice (novox/hq ADR 0468 §8)", at, method)
|
||||
}
|
||||
used[method] = true
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// readRequireShares reads the stored form's side key.
|
||||
func readRequireShares(raw json.RawMessage) (map[string][]BucketShare, error) {
|
||||
var out map[string][]BucketShare
|
||||
if err := json.Unmarshal(raw, &out); err != nil {
|
||||
return nil, fmt.Errorf("%s: an object of requirement to a list of {to, methods}: %w", RequireSharesKey, err)
|
||||
}
|
||||
for requirement, shares := range out {
|
||||
if err := checkBucketShares(requirement, shares); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -1,72 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Issue 1591: `shares` on a requirement of s3-bucket (novox/hq ADR 0468 §8).
|
||||
|
||||
const filesWithShares = `{"module":"files","version":"1","requires":["shell",
|
||||
{"name":"s3-bucket","shares":[{"to":"seat:node-files","methods":["put"]},{"to":"print-and-scan","methods":["get","put"]}]}]}`
|
||||
|
||||
// 1591/AC1: parsed, the shares are on the manifest; stored, `requires` is plain names and the side key holds them.
|
||||
func TestSharesParseAndAreStoredBesideRequires(t *testing.T) {
|
||||
m := mustParse(t, filesWithShares)
|
||||
want := []BucketShare{{To: "seat:node-files", Methods: []string{"put"}}, {To: "print-and-scan", Methods: []string{"get", "put"}}}
|
||||
if !reflect.DeepEqual(m.RequireShares["s3-bucket"], want) {
|
||||
t.Fatalf("the shares read as %+v", m.RequireShares)
|
||||
}
|
||||
raw, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var keys map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &keys); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(keys["requires"]) != `["shell","s3-bucket"]` {
|
||||
t.Errorf("the stored requires is not plain names: %s", keys["requires"])
|
||||
}
|
||||
if !strings.Contains(string(keys[RequireSharesKey]), `"seat:node-files"`) {
|
||||
t.Errorf("the side key does not hold the shares: %s", raw)
|
||||
}
|
||||
var again Manifest
|
||||
if err := json.Unmarshal(raw, &again); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(again, m) {
|
||||
t.Fatalf("not lossless:\n%+v\n%+v", again, m)
|
||||
}
|
||||
}
|
||||
|
||||
// 1591/AC2: a malformed share is refused, naming the entry and ADR 0468 §8.
|
||||
func TestAMalformedShareIsRefused(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"unknown method": `{"name":"s3-bucket","shares":[{"to":"a","methods":["delete"]}]}`,
|
||||
"empty to": `{"name":"s3-bucket","shares":[{"to":"","methods":["get"]}]}`,
|
||||
"malformed to": `{"name":"s3-bucket","shares":[{"to":"Seat:X y","methods":["get"]}]}`,
|
||||
"empty methods": `{"name":"s3-bucket","shares":[{"to":"a","methods":[]}]}`,
|
||||
"duplicate to": `{"name":"s3-bucket","shares":[{"to":"a","methods":["get"]},{"to":"a","methods":["put"]}]}`,
|
||||
"duplicate verb": `{"name":"s3-bucket","shares":[{"to":"a","methods":["get","get"]}]}`,
|
||||
"other require": `{"name":"route","shares":[{"to":"a","methods":["get"]}]}`,
|
||||
}
|
||||
for name, entry := range cases {
|
||||
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":[` + entry + `]}`))
|
||||
if err == nil {
|
||||
t.Errorf("%s: accepted", name)
|
||||
continue
|
||||
}
|
||||
if !strings.Contains(err.Error(), "ADR 0468 §8") || !strings.Contains(err.Error(), "requires") {
|
||||
t.Errorf("%s: the error does not name the entry and the ADR: %v", name, err)
|
||||
}
|
||||
}
|
||||
// The stored form is held to the same rules.
|
||||
_, err := decodeForTest([]byte(`{"module":"files","version":"1","requires":["s3-bucket"],
|
||||
"requires-shares":{"s3-bucket":[{"to":"a","methods":[]}]}}`))
|
||||
if err == nil || !strings.Contains(err.Error(), "ADR 0468 §8") {
|
||||
t.Errorf("the stored form was accepted: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,381 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Built is one artifact after it exists: where it is and what it hashes to.
|
||||
type Built struct {
|
||||
// Name is what the manifest called it.
|
||||
Name string
|
||||
// Kind is "image" or "archive".
|
||||
Kind string
|
||||
// Reference is what a machine uses to get it — an image reference for an image, a URL for an
|
||||
// archive. Both already carry the digest for an image; an archive carries it separately.
|
||||
Reference string
|
||||
// Digest is "sha256:<hex>", for an archive. An image reference already ends in one.
|
||||
Digest string
|
||||
// Launchers are, for a bundle in an interpreted language, the executable the build wrote beside
|
||||
// each entrypoint, by entrypoint (novox/hq ADR 0193): what the node's runtime starts to serve it.
|
||||
Launchers map[string]string
|
||||
// Texts are, for the archive a module packs its templates into (TemplatesArtifact), each file's text as the
|
||||
// build read it back from the packed archive, by its path inside it (novox/hq ADR 0349 §2). Never carried on
|
||||
// the wire or kept: Resolve reads them into the files' content, which is what the mesh holds.
|
||||
Texts map[string]string `json:"-"`
|
||||
}
|
||||
|
||||
// Resolve fills a manifest's resources in from what was built.
|
||||
//
|
||||
// Every resource naming an artifact is rewritten to name the thing itself, and the `artifact` key
|
||||
// is removed — because it is a build-time word and the host has never heard of it. A resource
|
||||
// naming an artifact nothing produced is refused: it would otherwise reach a machine with an
|
||||
// empty image or an unpinned archive, which is the shape of failure that looks like success.
|
||||
func (m Manifest) Resolve(built []Built) (Manifest, error) {
|
||||
if m.Build == nil && len(built) == 0 {
|
||||
return m, nil
|
||||
}
|
||||
|
||||
by := map[string]Built{}
|
||||
for _, b := range built {
|
||||
by[b.Name] = b
|
||||
}
|
||||
// Declared and not produced is a build that did not do what the manifest asked, and saying so
|
||||
// here beats a machine reporting it later.
|
||||
var missing []string
|
||||
if m.Build != nil {
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if _, ok := by[a.Name]; !ok {
|
||||
missing = append(missing, a.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
sort.Strings(missing)
|
||||
return Manifest{}, fmt.Errorf(
|
||||
"%s says it builds %s and the build did not produce %s — the build did not do what "+
|
||||
"the manifest asked, which is a different fault from a resource asking for the "+
|
||||
"wrong thing",
|
||||
m.Module, strings.Join(missing, " and "), oneOrOther(len(missing)))
|
||||
}
|
||||
|
||||
// **Each template's text, as the build read it back from its templates archive** (novox/hq ADR 0349 §2): into
|
||||
// the file's content, so what the mesh holds and composes is what the same text inline would be.
|
||||
var handed *Handed
|
||||
if m.Templated() {
|
||||
name := TemplatesArtifact(m)
|
||||
texts := by[name].Texts
|
||||
if name == "" || texts == nil {
|
||||
return Manifest{}, fmt.Errorf("%s names templates, and the build carried no text read back from the "+
|
||||
"archive it packs %s/ into (%q): the builder that built it does not read templates",
|
||||
m.Module, m.TemplatesDir(), name)
|
||||
}
|
||||
dir := m.TemplatesDir()
|
||||
filled, err := fillTemplates(m.Module, dir, m.Resources, func(template string) (string, bool) {
|
||||
t, ok := texts[strings.TrimPrefix(template, dir+"/")]
|
||||
return t, ok
|
||||
}, fmt.Sprintf("the archive %q", name))
|
||||
if err != nil {
|
||||
return Manifest{}, err
|
||||
}
|
||||
items, err := fillItemTemplates(m, m.Items, func(template string) (string, bool) {
|
||||
t, ok := texts[strings.TrimPrefix(template, dir+"/")]
|
||||
return t, ok
|
||||
}, fmt.Sprintf("the archive %q", name))
|
||||
if err != nil {
|
||||
return Manifest{}, err
|
||||
}
|
||||
// And, for a moved module, where its templates are and which one each file and item is filled from
|
||||
// (handed.go): read while its resources and items still name their templates.
|
||||
handed = handedBy(m, by[name])
|
||||
m.Resources = filled
|
||||
m.Items = items
|
||||
}
|
||||
|
||||
out := m
|
||||
out.Handed = handed
|
||||
out.Build = nil
|
||||
// Read with the build, like Build: the manifest the mesh holds carries each file's text (issue 731).
|
||||
out.Templates = ""
|
||||
out.Resources = nil
|
||||
// What the build compiled, kept on the resolved manifest (novox/hq ADR 0175): a tools bundle is
|
||||
// named by no resource of the module's own — the node's runtime loads it — so this is the only
|
||||
// place the mesh would otherwise not have it. In artifact order, so two resolutions of one
|
||||
// build compare equal.
|
||||
out.Bundles = nil
|
||||
if m.Build != nil {
|
||||
run := runByAResource(m)
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a.Kind != ArtifactBundle {
|
||||
continue
|
||||
}
|
||||
made := by[a.Name]
|
||||
loads := BundleLoads(m, a, run)
|
||||
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
|
||||
// it reached it by, and a manifest carrying that address names an installation —
|
||||
// registration refused node-tools for exactly this on 2026-10-02. The build record
|
||||
// already keeps the store-relative form; the resolved manifest keeps the same, and
|
||||
// composition routes it through the store a machine reaches (the registry clients' Routed).
|
||||
out.Bundles = append(out.Bundles, Bundle{
|
||||
Name: a.Name, Source: kept(made.Reference), Digest: made.Digest,
|
||||
Language: a.Language, Entrypoints: append([]string(nil), a.Entrypoints...),
|
||||
Loads: loads, Env: copyWords(a.Env), Launchers: copyWords(made.Launchers),
|
||||
Binary: BinaryOf(a), Bus: a.Bus,
|
||||
})
|
||||
}
|
||||
sort.Slice(out.Bundles, func(i, j int) bool { return out.Bundles[i].Name < out.Bundles[j].Name })
|
||||
}
|
||||
for _, r := range m.Resources {
|
||||
named, _ := r["artifact"].(string)
|
||||
if named == "" {
|
||||
out.Resources = append(out.Resources, r)
|
||||
continue
|
||||
}
|
||||
artifact, ok := by[named]
|
||||
if !ok {
|
||||
return Manifest{}, fmt.Errorf(
|
||||
"%s: %v uses the artifact %q, and this module builds no such thing",
|
||||
m.Module, r["id"], named)
|
||||
}
|
||||
|
||||
filled := map[string]any{}
|
||||
for k, v := range r {
|
||||
filled[k] = v
|
||||
}
|
||||
delete(filled, "artifact")
|
||||
switch artifact.Kind {
|
||||
case ArtifactPackage:
|
||||
// A package is not a resource on any machine; it is consumed by other builds. A
|
||||
// resource that names one is a manifest error, named here rather than shipped.
|
||||
return Manifest{}, fmt.Errorf(
|
||||
"%s: %v uses %q, which is a package — a build input, not a resource a machine runs",
|
||||
m.Module, r["id"], named)
|
||||
case ArtifactImage, ArtifactUpstream:
|
||||
filled["image"] = artifact.Reference
|
||||
// An image is not unpacked anywhere, so it has no directory to be named for its
|
||||
// version and `${version}` has nothing to mean. Refused rather than left as literal
|
||||
// text in a path, which is how it would reach a machine and be created as a directory
|
||||
// called `${version}`.
|
||||
for key, value := range filled {
|
||||
if text, isText := value.(string); isText && strings.Contains(text, versionRef) {
|
||||
return Manifest{}, fmt.Errorf(
|
||||
"%s: %v says %s in %q, and %q is an image — an image is not unpacked, so "+
|
||||
"it has no versioned place. %s is for an archive or a bundle",
|
||||
m.Module, r["id"], versionRef, key, named, versionRef)
|
||||
}
|
||||
}
|
||||
case ArtifactArchive, ArtifactBundle:
|
||||
// The same on the wire: both are bytes fetched by digest and unpacked. They differ in
|
||||
// how they were made — one packed as it stood, the other compiled first — and a
|
||||
// machine has no reason to care which.
|
||||
// Kept, not routed, for the reason the bundles above are (ADR 0155).
|
||||
filled["source"] = kept(artifact.Reference)
|
||||
filled["digest"] = artifact.Digest
|
||||
// **And `${version}`, so a resource can name a place that is this build's alone**
|
||||
// (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named
|
||||
// for its version so it can read its own version from its path — and until this,
|
||||
// nothing could compose that path: an archive named a fixed one in the manifest and
|
||||
// nothing interpolated the build into it, so nothing could ask for
|
||||
// `…/versions/<version>/` and every machine took a hand-placed fallback.
|
||||
//
|
||||
// The version is the artifact's own digest, short. Not the commit: two builds of one
|
||||
// commit are meant to be the same bytes (the toolchains are `-trimpath` for this), and
|
||||
// a content-addressed version means an unchanged build resolves to the path it already
|
||||
// had — so re-composing a declaration moves nothing, where a commit would move the
|
||||
// path of an identical binary and recreate everything that reads it.
|
||||
for key, value := range filled {
|
||||
filled[key] = withVersion(value, versionOf(artifact.Digest))
|
||||
}
|
||||
default:
|
||||
return Manifest{}, fmt.Errorf("%s: %q is a %q, and an artifact is %q, %q, %q or %q",
|
||||
m.Module, named, artifact.Kind, ArtifactImage, ArtifactArchive, ArtifactUpstream,
|
||||
ArtifactBundle)
|
||||
}
|
||||
out.Resources = append(out.Resources, filled)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// withVersion is a resource's value with `${version}` filled: in a string, and in each string of a map —
|
||||
// a process's env, where the controller is told which build it is (novox/hq issue 352). Anything else is
|
||||
// left as it is.
|
||||
func withVersion(value any, version string) any {
|
||||
switch v := value.(type) {
|
||||
case string:
|
||||
if strings.Contains(v, versionRef) {
|
||||
return strings.ReplaceAll(v, versionRef, version)
|
||||
}
|
||||
case map[string]any:
|
||||
out := make(map[string]any, len(v))
|
||||
for k, x := range v {
|
||||
out[k] = withVersion(x, version)
|
||||
}
|
||||
return out
|
||||
case map[string]string:
|
||||
out := make(map[string]string, len(v))
|
||||
for k, x := range v {
|
||||
out[k] = strings.ReplaceAll(x, versionRef, version)
|
||||
}
|
||||
return out
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
// NamedPaths are the paths a module's fingerprint takes its tree from (novox/hq ADR 0499 clause 2): the union of the
|
||||
// paths its artifacts name, cleaned and sorted, and every true only when the module builds something and every artifact
|
||||
// names paths; otherwise none, and the module keeps its directory's tree.
|
||||
//
|
||||
// **Read by the core alone** (ADR 0495 clause 1, the lead's decision L8 of R4e): the indexer and its merge preview
|
||||
// (buildstep.ModuleTrees) call it. A build module's fetch step is handed the paths by the builder, from the indexer's
|
||||
// kept inputs (delivery.KeptInputs2.Paths), and reads no manifest for them.
|
||||
func (m Manifest) NamedPaths() (paths []string, every bool) {
|
||||
if m.Build == nil || len(m.Build.Artifacts) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if len(a.Paths) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
for _, p := range a.Paths {
|
||||
seen[path.Clean(p)] = true
|
||||
}
|
||||
}
|
||||
for p := range seen {
|
||||
paths = append(paths, p)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
return paths, true
|
||||
}
|
||||
|
||||
// oneOrOther keeps the message readable for one artifact and for several, because a message that
|
||||
// says "neither" about one thing reads as a bug in the message.
|
||||
func oneOrOther(n int) string {
|
||||
if n == 1 {
|
||||
return "it"
|
||||
}
|
||||
return "them"
|
||||
}
|
||||
|
||||
// compilesToABinary is whether this language's bundle is a binary for one operating system rather
|
||||
// than code that runs wherever its interpreter does.
|
||||
//
|
||||
// **Asked of the language, not of the artifact.** A module says what it is written in; what that
|
||||
// implies is the mesh's to know, exactly as the compiler is (novox/hq ADR 0142). Asking the artifact
|
||||
// would let two artifacts in one language disagree about whether they are portable.
|
||||
func compilesToABinary(language string) bool {
|
||||
switch strings.ToLower(strings.TrimSpace(language)) {
|
||||
case "go":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// versionRef is how a resource names the version of the artifact it uses: ${version}.
|
||||
//
|
||||
// No artifact name in it, because the resource already says which artifact it is for — a second
|
||||
// name would be a second thing to keep in step with the first.
|
||||
const versionRef = "${version}"
|
||||
|
||||
// versionOf is an artifact's version as a path names it: its digest, short.
|
||||
//
|
||||
// **Content-addressed on purpose.** The alternative is the commit a build came from, and two builds
|
||||
// of one commit are meant to produce the same bytes — every toolchain here is `-trimpath` for that
|
||||
// reason. A commit-named path would move for an identical binary, and everything reading that path
|
||||
// would be recreated for a change that is not one. A digest-named path moves exactly when the bytes
|
||||
// do.
|
||||
//
|
||||
// Twelve hex characters: enough that two of this mesh's builds will not collide, short enough to
|
||||
// read in a path and in a journal line. The `sha256:` prefix goes, because a directory name carrying
|
||||
// a colon is a directory name people quote wrong.
|
||||
func versionOf(digest string) string {
|
||||
hex := strings.TrimPrefix(strings.TrimSpace(digest), "sha256:")
|
||||
if len(hex) > 12 {
|
||||
return hex[:12]
|
||||
}
|
||||
return hex
|
||||
}
|
||||
|
||||
func copyWords(in map[string]string) map[string]string {
|
||||
if len(in) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]string, len(in))
|
||||
for k, v := range in {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// BinaryOf is what a bundle compiled to a binary is called once built: what the artifact names, else
|
||||
// the package it is built from, else the artifact's own name (novox/hq 04-ISSUES/142). Empty for a
|
||||
// language that does not compile to one. The builder writes the binary under this name, and the
|
||||
// composer runs it by it, so both ask here.
|
||||
func BinaryOf(a Artifact) string {
|
||||
if !compilesToABinary(a.Language) {
|
||||
return ""
|
||||
}
|
||||
if name := strings.TrimSpace(a.Binary); name != "" {
|
||||
return name
|
||||
}
|
||||
if from := strings.Trim(a.From, "./"); from != "" {
|
||||
return path.Base(from)
|
||||
}
|
||||
return a.Name
|
||||
}
|
||||
|
||||
// runByAResource is the artifacts one of a module's own resources names — a process that runs it,
|
||||
// a step, an archive that unpacks it — by name.
|
||||
func runByAResource(m Manifest) map[string]bool {
|
||||
named := map[string]bool{}
|
||||
for _, r := range m.Resources {
|
||||
if a, ok := r["artifact"].(string); ok && a != "" {
|
||||
named[a] = true
|
||||
}
|
||||
}
|
||||
return named
|
||||
}
|
||||
|
||||
// BundleLoads is what the runtime loads of the bundle artifact a of m (run: the bundles one of m's own resources runs,
|
||||
// runByAResource): what the artifact said, else every entrypoint of a module that declares tools, else nothing; a
|
||||
// bundle compiled to a binary is served by its binary. The rule a build's resolution keeps on Bundle.Loads, and what a
|
||||
// module composed as what runs of it reads again when the runtime already dropped it (issue 1126).
|
||||
func BundleLoads(m Manifest, a Artifact, run map[string]bool) []string {
|
||||
// What the runtime loads: what the artifact said, else every entrypoint of a module
|
||||
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
|
||||
//
|
||||
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
|
||||
// A process the host runs is the module's service, not its tools: the controller declares
|
||||
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
|
||||
// have been launched a second time by the node's runtime, as an MCP child it is not.
|
||||
loads := append([]string(nil), a.Loads...)
|
||||
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
|
||||
loads = append([]string(nil), a.Entrypoints...)
|
||||
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
|
||||
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
|
||||
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
|
||||
if bin := BinaryOf(a); bin != "" {
|
||||
loads = []string{bin}
|
||||
}
|
||||
}
|
||||
return loads
|
||||
}
|
||||
|
||||
// kept is a bundle's or an archive's reference without the artifact store's address, if the builder wrote one (ADR 0155:
|
||||
// a resolved manifest names no installation). Only the address is read here: an `http://<host>/v2/<path>/blobs/<digest>`
|
||||
// becomes `artifact-store://<path>/blobs/<digest>`; a kept reference, or one in a registry seat's form (whose first part,
|
||||
// a seat's name, has no dot or port), is what it was. How the registry seats' forms are read is the registry clients'.
|
||||
func kept(reference string) string {
|
||||
if strings.HasPrefix(reference, ArtifactStoreScheme) {
|
||||
return reference
|
||||
}
|
||||
if rest, isURL := strings.CutPrefix(reference, "http://"); isURL {
|
||||
if _, path, ok := strings.Cut(rest, "/v2/"); ok && strings.Contains(path, "/blobs/") {
|
||||
return ArtifactStoreScheme + path
|
||||
}
|
||||
}
|
||||
return reference
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
// bundleFor is a manifest whose one artifact is a bundle in the given language and system.
|
||||
func bundleFor(language, system string) Manifest {
|
||||
return Manifest{Module: "a-component", Build: &Build{Artifacts: []Artifact{
|
||||
{Name: "binary", Kind: ArtifactBundle, Language: language, System: system},
|
||||
}}}
|
||||
}
|
||||
@@ -1,122 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The manifest in a repository and the manifest the mesh holds are two documents.
|
||||
//
|
||||
// A digest is not knowable until something is built, so a repository carrying one would be a
|
||||
// repository whose file is wrong the moment anybody edits anything — and the mesh would be pinning
|
||||
// a value nobody could have checked.
|
||||
|
||||
func buildable() Manifest {
|
||||
return Manifest{
|
||||
Module: "meshboard", Version: "1",
|
||||
Build: &Build{Artifacts: []Artifact{
|
||||
{Name: "server", Kind: ArtifactImage, From: "Dockerfile"},
|
||||
{Name: "theme", Kind: ArtifactArchive, From: "files"},
|
||||
}},
|
||||
Resources: []map[string]any{
|
||||
{"id": "svc", "type": "container", "name": "meshboard", "artifact": "server"},
|
||||
{"id": "look", "type": "archive", "path": "/opt/meshboard", "artifact": "theme"},
|
||||
{"id": "dir", "type": "directory", "path": "/opt/meshboard"},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func wasBuilt() []Built {
|
||||
return []Built{
|
||||
{Name: "server", Kind: ArtifactImage,
|
||||
Reference: "registry.invalid/meshboard@sha256:" + strings.Repeat("a", 64)},
|
||||
{Name: "theme", Kind: ArtifactArchive,
|
||||
Reference: "https://store.invalid/theme.tar.gz",
|
||||
Digest: "sha256:" + strings.Repeat("b", 64)},
|
||||
}
|
||||
}
|
||||
|
||||
func TestAResourceNamingAnArtifactBecomesOneNamingTheThing(t *testing.T) {
|
||||
got, err := buildable().Resolve(wasBuilt())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range got.Resources {
|
||||
if r["artifact"] != nil {
|
||||
// A build-time word the host has never heard of. Leaving it would be a field the
|
||||
// strict decoder refuses on the machine, at the worst moment.
|
||||
t.Fatalf("%v still names an artifact: %v", r["id"], r)
|
||||
}
|
||||
}
|
||||
if got.Resources[0]["image"] != "registry.invalid/meshboard@sha256:"+strings.Repeat("a", 64) {
|
||||
t.Fatalf("the image was not filled in: %v", got.Resources[0])
|
||||
}
|
||||
if got.Resources[1]["source"] != "https://store.invalid/theme.tar.gz" ||
|
||||
got.Resources[1]["digest"] != "sha256:"+strings.Repeat("b", 64) {
|
||||
t.Fatalf("the archive was not filled in: %v", got.Resources[1])
|
||||
}
|
||||
// And a resource that names nothing is untouched.
|
||||
if got.Resources[2]["path"] != "/opt/meshboard" || len(got.Resources[2]) != 3 {
|
||||
t.Fatalf("an ordinary resource was changed: %v", got.Resources[2])
|
||||
}
|
||||
// The built manifest carries no build section: it is the derived document, and something
|
||||
// holding both would invite somebody to build from it again.
|
||||
if got.Build != nil {
|
||||
t.Fatal("the built manifest still says how to build itself")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnArtifactDeclaredAndNotBuiltIsRefused(t *testing.T) {
|
||||
// A build that did not do what the manifest asked. Said here rather than by a machine later
|
||||
// reporting an empty image.
|
||||
_, err := buildable().Resolve(wasBuilt()[:1])
|
||||
if err == nil {
|
||||
t.Fatal("a manifest resolved with an artifact missing")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "theme") {
|
||||
t.Fatalf("the refusal does not name what is missing: %v", err)
|
||||
}
|
||||
// And it must blame the build rather than the resource. A resource asking for something that
|
||||
// does not exist is a manifest fault; a build not producing what the manifest declared is a
|
||||
// build fault, and telling somebody to fix the wrong one costs an afternoon.
|
||||
if !strings.Contains(err.Error(), "the build did not produce") {
|
||||
t.Fatalf("the refusal blames the wrong thing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAResourceNamingSomethingTheModuleDoesNotBuildIsRefused(t *testing.T) {
|
||||
m := buildable()
|
||||
m.Resources = append(m.Resources,
|
||||
map[string]any{"id": "other", "type": "archive", "path": "/x", "artifact": "nothing"})
|
||||
if _, err := m.Resolve(wasBuilt()); err == nil {
|
||||
t.Fatal("a resource naming an artifact nobody builds was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAModuleThatBuildsNothingIsOrdinary(t *testing.T) {
|
||||
// Most of what a person installs is configuration. Requiring an empty build section would be
|
||||
// a field that exists to be left blank.
|
||||
m := Manifest{Module: "shell", Version: "1", Resources: []map[string]any{
|
||||
{"id": "rc", "type": "file", "path": "/etc/zsh/zshrc", "content": "setopt"},
|
||||
}}
|
||||
got, err := m.Resolve(nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Resources) != 1 {
|
||||
t.Fatalf("got %v", got.Resources)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAResourceNamingAPackageIsRefused(t *testing.T) {
|
||||
m, err := decodeForTest([]byte(`{"module":"a","version":"1","slug":"a",
|
||||
"build":{"artifacts":[{"name":"lib","kind":"package","language":"typescript"}]},
|
||||
"resources":[{"id":"svc","type":"container","name":"a","artifact":"lib"}]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("parse: %v", err)
|
||||
}
|
||||
_, err = m.Resolve([]Built{{Name: "lib", Kind: ArtifactPackage, Reference: "@novox/a@1.0.0"}})
|
||||
if err == nil {
|
||||
t.Fatal("a resource backed by a package was accepted; a package is not a resource")
|
||||
}
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/internal/beside"
|
||||
)
|
||||
|
||||
// The catalogue's bus protects its streams by the snapshot, not as live files: its streams' item is a
|
||||
// dump into its snapshots, it has the account the dump runs as, and the dump runs the snapshot
|
||||
// program in the bus's own container.
|
||||
func TestTheCataloguesBusIsBackedUpBySnapshot(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join(beside.Catalogue(t), "nats", "module.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := decodeForTest(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, account := m.OwnSecrets["broker"]; !account {
|
||||
t.Fatal("the bus declares no account, so its snapshot could not reach it")
|
||||
}
|
||||
var found bool
|
||||
if m.Data == nil {
|
||||
t.Fatal("the bus declares no data")
|
||||
}
|
||||
for _, it := range m.Data.Own {
|
||||
if it.ID != "jetstream" {
|
||||
continue
|
||||
}
|
||||
found = true
|
||||
if !it.Backup.IsDump() || it.Backup.Into != "snapshots" {
|
||||
t.Fatalf("the bus's streams are protected by %+v, not a snapshot into its snapshots", it.Backup)
|
||||
}
|
||||
if !strings.Contains(it.Backup.Dump, "mesh-nats-snapshot snapshot") {
|
||||
t.Fatalf("the dump does not run the snapshot program: %s", it.Backup.Dump)
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("the bus declares no item for its streams")
|
||||
}
|
||||
}
|
||||
@@ -1,88 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The names of what a module keeps on the bus, as one rule (design 25 § 12.1; novox/hq ADR 0666 §16): the bus module
|
||||
// creates them from a module's entries and the grants part grants them, and both take the names from here.
|
||||
|
||||
// BusNameSeparator joins a module's name and an entry's local name into the name the mesh gives the entry on the bus.
|
||||
const BusNameSeparator = "_"
|
||||
|
||||
// SentObjectsLocal is the local name of a module's store of sent objects (the bodies of a reply or an event over the
|
||||
// per-message limit): the bus module plans `<module>_sent-objects` for every module, and no entry may take the name.
|
||||
const SentObjectsLocal = "sent-objects"
|
||||
|
||||
// ObjectsSuffix ends the local name of the object store beside a state: `<state>-objects`.
|
||||
const ObjectsSuffix = "-objects"
|
||||
|
||||
// safeToken is a name that is one plain subject token.
|
||||
var safeToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||
|
||||
// BusName is `<module>_<local>`, the name the mesh gives an entry on the bus.
|
||||
func BusName(module, local string) string { return module + BusNameSeparator + local }
|
||||
|
||||
// StateObjectsName is the local name of the object store that holds the large values of the state `state`.
|
||||
func StateObjectsName(state string) string { return state + ObjectsSuffix }
|
||||
|
||||
// SentObjectsName is the bus name of a module's store of sent objects.
|
||||
func SentObjectsName(module string) string { return BusName(module, SentObjectsLocal) }
|
||||
|
||||
// BusStateStreams are the streams one bus-state entry of module is kept in on the bus, by the mesh's names
|
||||
// `<module>_<local>`: a stream's own; a key-value bucket's KV_ stream with the OBJ_ stream of its object store
|
||||
// `<local>-objects`; an object store's OBJ_ stream; a ledger's history stream, its snapshot stream, and its state and
|
||||
// index buckets' KV_ streams. A name that is not one plain subject token names nothing.
|
||||
func BusStateStreams(module string, e BusStateEntry) []string {
|
||||
base := BusName(module, e.Name)
|
||||
if !safeToken.MatchString(base) || strings.Contains(e.Name, BusNameSeparator) || strings.Contains(module, BusNameSeparator) {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
switch e.Storage {
|
||||
case StorageKeyValue:
|
||||
out = []string{"KV_" + base}
|
||||
if e.Template == TemplateState {
|
||||
out = append(out, "OBJ_"+BusName(module, StateObjectsName(e.Name)))
|
||||
}
|
||||
case StorageObjectStore:
|
||||
out = []string{"OBJ_" + base}
|
||||
case StorageLedger:
|
||||
out = []string{base, base + "_snapshot", "KV_" + base + "_state"}
|
||||
for _, t := range e.Types {
|
||||
for _, ix := range t.Indexes {
|
||||
if safeToken.MatchString(ix.Name) {
|
||||
out = append(out, "KV_"+base+"_index-"+ix.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
default:
|
||||
out = []string{base}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return slices.Compact(out)
|
||||
}
|
||||
|
||||
// ObjectNameClashes are the local names of a module's bus-state entries that meet a name the mesh makes itself: the
|
||||
// name of a state's object store (`<state>-objects` beside the state `<state>`) and the store of sent objects. The
|
||||
// indexer's judge refuses each.
|
||||
func ObjectNameClashes(entries []BusStateEntry) []string {
|
||||
states := map[string]bool{}
|
||||
for _, e := range entries {
|
||||
if e.Storage == StorageKeyValue && e.Template == TemplateState {
|
||||
states[e.Name] = true
|
||||
}
|
||||
}
|
||||
var out []string
|
||||
for _, e := range entries {
|
||||
stem, ok := strings.CutSuffix(e.Name, ObjectsSuffix)
|
||||
if e.Name == SentObjectsLocal || (ok && states[stem]) {
|
||||
out = append(out, e.Name)
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return slices.Compact(out)
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"slices"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ADR 0666 §16: the bus names of an entry are one rule. A state entry answers its bucket's stream and the stream of
|
||||
// its object store; a local name that meets a name the mesh makes is a clash for the judge.
|
||||
func subTheBusNamesOfAnEntryAreOneRule(t *testing.T) {
|
||||
state := BusStateEntry{Name: "views", Storage: StorageKeyValue, Template: TemplateState}
|
||||
rows := []struct {
|
||||
name string
|
||||
entry BusStateEntry
|
||||
streams []string
|
||||
}{
|
||||
{"a state: its bucket and its object store", state, []string{"KV_m_views", "OBJ_m_views-objects"}},
|
||||
{"a key-value that is not a state template: its bucket only", BusStateEntry{Name: "x", Storage: StorageKeyValue}, []string{"KV_m_x"}},
|
||||
{"an object store", BusStateEntry{Name: "blobs", Storage: StorageObjectStore, Template: TemplateObjects}, []string{"OBJ_m_blobs"}},
|
||||
{"a stream", BusStateEntry{Name: "log", Storage: StorageStream, Template: TemplateLog}, []string{"m_log"}},
|
||||
{"a ledger", BusStateEntry{Name: "l", Storage: StorageLedger, Template: TemplateLedger}, []string{"KV_m_l_state", "m_l", "m_l_snapshot"}},
|
||||
{"a name that is not one token", BusStateEntry{Name: "a.b", Storage: StorageStream}, nil},
|
||||
{"a local name holding the separator", BusStateEntry{Name: "a_b", Storage: StorageStream}, nil},
|
||||
}
|
||||
for _, r := range rows {
|
||||
if got := BusStateStreams("m", r.entry); !slices.Equal(got, r.streams) {
|
||||
t.Errorf("%s: streams %v, want %v", r.name, got, r.streams)
|
||||
}
|
||||
}
|
||||
if got := SentObjectsName("m"); got != "m_sent-objects" {
|
||||
t.Errorf("sent store %q", got)
|
||||
}
|
||||
clash := []BusStateEntry{state, {Name: "views-objects", Storage: StorageObjectStore}, {Name: "other-objects", Storage: StorageObjectStore},
|
||||
{Name: SentObjectsLocal, Storage: StorageObjectStore}}
|
||||
if got := ObjectNameClashes(clash); !slices.Equal(got, []string{"sent-objects", "views-objects"}) {
|
||||
t.Errorf("clashes %v", got)
|
||||
}
|
||||
}
|
||||
@@ -1,232 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Bus state is what a module keeps on the bus, asked for as a provision of the bus module in `requires` (novox/hq ADR
|
||||
// 0595 §1, design 32 § 4): an entry of `requires` in its typed long form, the local name, the storage type, the template
|
||||
// and the template's options. This is the contract only (ADR 0578 §7, ADR 0593 §1): the entry's shape, decoded
|
||||
// leniently. Every rule that judges an entry (a template known, an option within its bound, a fixed setting not given)
|
||||
// is the indexer's judge's.
|
||||
|
||||
// The storage types (design 25 § 12.1): the stream is the base; the key-value bucket, the object store and the ledger
|
||||
// are built on it, each with calls of its own (ADR 0595 §2).
|
||||
const (
|
||||
StorageStream = "stream"
|
||||
StorageKeyValue = "key-value"
|
||||
StorageObjectStore = "object-store"
|
||||
StorageLedger = "ledger"
|
||||
)
|
||||
|
||||
// The six templates, each a storage type with a predefined configuration (ADR 0595 §2, design 25 § 12.1): state on
|
||||
// key-value, events, work queue and log on stream, objects on object store, ledger on ledger. A bare stream names
|
||||
// none and gives its own Configuration.
|
||||
const (
|
||||
TemplateState = "state"
|
||||
TemplateEvents = "events"
|
||||
TemplateWorkQueue = "work-queue"
|
||||
TemplateLog = "log"
|
||||
TemplateObjects = "objects"
|
||||
TemplateLedger = "ledger"
|
||||
)
|
||||
|
||||
// The option names, a template's settings a module may change (design 25 § 12.1's table), written as the old sections
|
||||
// wrote them, so a module that moves carries its old settings as options unchanged (ADR 0595 §11). Which option a
|
||||
// template takes, and its bound, is the judge's.
|
||||
const (
|
||||
// OptionPastValues is how many past values a state's key keeps (the old `state`'s `history`).
|
||||
OptionPastValues = "history"
|
||||
// OptionTimeToLive is how long a state's value lives, in seconds.
|
||||
OptionTimeToLive = "ttl-seconds"
|
||||
// OptionLifetimePerKey is whether a state's key may be given a lifetime of its own.
|
||||
OptionLifetimePerKey = "key-ttl"
|
||||
// OptionPerMachine is whether a state is kept one bucket per machine.
|
||||
OptionPerMachine = "per-machine"
|
||||
// OptionAgeDays is how long a stream keeps a message, in days.
|
||||
OptionAgeDays = "max-age-days"
|
||||
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
|
||||
OptionCapMiB = "max-mib"
|
||||
// OptionObjectsCapMiB is the cap of a state's objects store `<state>-objects`, in MiB, where the state needs
|
||||
// another than the one the bus module plans (design 47 G §513, §547).
|
||||
OptionObjectsCapMiB = "objects-max-mib"
|
||||
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
|
||||
OptionGrant = "grant"
|
||||
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
|
||||
OptionDeliveries = "max-deliveries"
|
||||
// OptionDuplicates is how long, in seconds, a work queue detects a message sent twice.
|
||||
OptionDuplicates = "duplicates"
|
||||
)
|
||||
|
||||
// ConfigWhenFull is what a configuration of its own says a bare stream does when full: WhenFullRefuse or
|
||||
// WhenFullDropOldest. In a template it is fixed, never an option (ADR 0595 §3).
|
||||
const (
|
||||
ConfigWhenFull = "when-full"
|
||||
WhenFullRefuse = "refuse"
|
||||
WhenFullDropOldest = "drop-oldest"
|
||||
)
|
||||
|
||||
// The ledger effects, what an operation does to an instance's state (ADR 0597 §2): one of this fixed set.
|
||||
const (
|
||||
// EffectSetFields sets the state's fields to the operation's Fields.
|
||||
EffectSetFields = "set-fields"
|
||||
// EffectSetEntry sets one entry of the state's map Into, under the key the operation's field Key holds, to its Fields.
|
||||
EffectSetEntry = "set-entry"
|
||||
// EffectAppend appends the operation's Fields to the state's list Into.
|
||||
EffectAppend = "append"
|
||||
// EffectRemove removes the entry of the map Into under the operation's field Key, or, naming no Into, the instance's
|
||||
// state.
|
||||
EffectRemove = "remove"
|
||||
)
|
||||
|
||||
// LedgerType is one type of a ledger (ADR 0597 §1): a schema and the operations allowed on it, with its indexes. An
|
||||
// instance of it is named by a path whose first part is the type's name (`decision/0290`).
|
||||
type LedgerType struct {
|
||||
Name string `json:"name"`
|
||||
// Schema is the type's fields, by name.
|
||||
Schema map[string]LedgerField `json:"schema"`
|
||||
// Operations are the operations allowed on an instance, each with its effect.
|
||||
Operations []LedgerOperation `json:"operations"`
|
||||
// Indexes are its declared indexes, each a second key-value bucket kept by a watcher (ADR 0597 §4).
|
||||
Indexes []LedgerIndex `json:"indexes,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerField is one field of a ledger type's schema.
|
||||
type LedgerField struct {
|
||||
// Type is the field's value type, as the schema names it.
|
||||
Type string `json:"type"`
|
||||
// DigestOf names the text field this field is the digest of (ADR 0597 §2): `apply` refuses an operation whose
|
||||
// digest does not match its text.
|
||||
DigestOf string `json:"digest-of,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerOperation is one operation a ledger type allows, and its effect on the state.
|
||||
type LedgerOperation struct {
|
||||
Name string `json:"name"`
|
||||
// Effect is one of the ledger effects (EffectSetFields, EffectSetEntry, EffectAppend, EffectRemove).
|
||||
Effect string `json:"effect"`
|
||||
// Fields are the operation's fields its effect copies into the state; it may carry more (ADR 0597 §2).
|
||||
Fields []string `json:"fields,omitempty"`
|
||||
// Into is the state's map (EffectSetEntry, EffectRemove) or list (EffectAppend) the effect writes.
|
||||
Into string `json:"into,omitempty"`
|
||||
// Key is the operation's field holding the entry's key (EffectSetEntry, EffectRemove).
|
||||
Key string `json:"key,omitempty"`
|
||||
// Sequence and Author are whether the effect also writes the operation's sequence and its proven author.
|
||||
Sequence bool `json:"sequence,omitempty"`
|
||||
Author bool `json:"author,omitempty"`
|
||||
}
|
||||
|
||||
// LedgerIndex is one declared index of a ledger type: its name, and the fields it is kept on.
|
||||
type LedgerIndex struct {
|
||||
Name string `json:"name"`
|
||||
On []string `json:"on"`
|
||||
}
|
||||
|
||||
// BusStateField is where a stored manifest keeps its bus-state entries: beside `requires`, which stays a plain list of
|
||||
// names, as OptionalRequiresKey keeps the optional ones.
|
||||
const BusStateField = "bus-state"
|
||||
|
||||
// BusStateEntry is one piece of a module's bus state, as its manifest asks for it.
|
||||
type BusStateEntry struct {
|
||||
// Name is its local name; the mesh, never the module, names it on the bus (ADR 0201 §4).
|
||||
Name string `json:"name"`
|
||||
// Storage is its storage type (design 25 § 12.1): how the bus keeps it.
|
||||
Storage string `json:"storage"`
|
||||
// Template is the predefined configuration of that storage type it uses; empty for a bare stream, which gives
|
||||
// Configuration instead.
|
||||
Template string `json:"template,omitempty"`
|
||||
// Options are the template's settings the module changes, by option name; each value as the manifest wrote it.
|
||||
Options map[string]json.RawMessage `json:"options,omitempty"`
|
||||
// Configuration is a configuration of the module's own, where no template fits: a bare stream's, or a ledger's
|
||||
// whose history expires (ADR 0595 §2).
|
||||
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
|
||||
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
|
||||
Types []LedgerType `json:"types,omitempty"`
|
||||
// Events are the events an entry of the template events carries, named locally, as a part's finding stream
|
||||
// lists its three finding events; the catalog graph entry carries them (design 47 G §529).
|
||||
Events []string `json:"events,omitempty"`
|
||||
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
|
||||
KeptLayout
|
||||
}
|
||||
|
||||
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
|
||||
func isBusStateEntry(raw json.RawMessage) bool {
|
||||
var keys map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &keys) != nil {
|
||||
return false
|
||||
}
|
||||
_, has := keys["storage"]
|
||||
return has
|
||||
}
|
||||
|
||||
// readBusStateEntry reads one bus-state entry leniently (ADR 0578 §3): a key it does not know, at any depth, is set
|
||||
// aside and its path named, never refused. An error is only an entry of the wrong shape.
|
||||
func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
|
||||
var tree any
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&tree); err != nil {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
var aside []string
|
||||
strict := func(e *BusStateEntry) error {
|
||||
b, err := json.Marshal(tree)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d := json.NewDecoder(bytes.NewReader(b))
|
||||
d.DisallowUnknownFields()
|
||||
return typedUnknown(d.Decode(e))
|
||||
}
|
||||
for tries := 0; ; tries++ {
|
||||
var e BusStateEntry
|
||||
err := strict(&e)
|
||||
if err == nil {
|
||||
if err := e.KeptLayout.refuse(fmt.Sprintf("requires entry %q", e.Name)); err != nil {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
return e, aside, nil
|
||||
}
|
||||
unknown := asUnknownField(err)
|
||||
if unknown == nil || tries > 64 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path, ok := removalThatHelps(tree, unknown.Field, err.Error(), func() error { var e BusStateEntry; return strict(&e) })
|
||||
if !ok {
|
||||
if removeEvery(tree, unknown.Field) == 0 {
|
||||
return BusStateEntry{}, nil, err
|
||||
}
|
||||
path = "…." + unknown.Field
|
||||
}
|
||||
aside = append(aside, path)
|
||||
}
|
||||
}
|
||||
|
||||
// setAsideAlso adds paths read without to what a manifest set aside (SetAside), in its words.
|
||||
func setAsideAlso(unknown string, paths []string) string {
|
||||
if len(paths) == 0 {
|
||||
return unknown
|
||||
}
|
||||
if unknown == "" {
|
||||
field := paths[0][strings.LastIndex(paths[0], ".")+1:]
|
||||
unknown = fmt.Sprintf("json: unknown field %q", field)
|
||||
}
|
||||
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
|
||||
}
|
||||
|
||||
// KeptStates are the local names of the state a manifest keeps, each a key-value bucket of its own: the old `state`
|
||||
// section's and every key-value entry it asks for in `requires` (ADR 0595 §1, §11), sorted, once each. It is the one
|
||||
// rule every reader of what a module keeps calls (novox/hq issue 1966): a process's `bus.state` and a seat
|
||||
// contribution's state name one of these, in either form.
|
||||
func KeptStates(m Manifest) []string {
|
||||
var out []string
|
||||
for _, k := range KeptDeclarations(m) {
|
||||
if k.Kind == KeptKindState {
|
||||
out = append(out, k.Name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -1,69 +0,0 @@
|
||||
package catalogue
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// BusyField is the resource field.
|
||||
const BusyField = "busy"
|
||||
|
||||
// The bounds of one ask (ADR 0335), the node-engine's own, held here near the author.
|
||||
const (
|
||||
BusyTimeoutDefault = 10 * time.Second
|
||||
BusyTimeoutCeiling = time.Minute
|
||||
)
|
||||
|
||||
// Busy is one container's busy, read.
|
||||
type Busy struct {
|
||||
Command string
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// ReadBusy reads a resource's `busy` field, its default applied. False when it carries none.
|
||||
func ReadBusy(r map[string]any) (Busy, bool, []string) {
|
||||
raw, present := r[BusyField]
|
||||
if !present {
|
||||
return Busy{}, false, nil
|
||||
}
|
||||
id := fmt.Sprint(r["id"])
|
||||
fields, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return Busy{}, true, []string{fmt.Sprintf("%s: busy is %T; it is an object with a command", id, raw)}
|
||||
}
|
||||
var problems []string
|
||||
keys := make([]string, 0, len(fields))
|
||||
for k := range fields {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, k := range keys {
|
||||
if k != "command" && k != "timeout" {
|
||||
problems = append(problems, fmt.Sprintf("%s: its busy says %q; busy has a command and a timeout", id, k))
|
||||
}
|
||||
}
|
||||
b := Busy{Timeout: BusyTimeoutDefault}
|
||||
if v, has := fields["command"]; has {
|
||||
s, ok := v.(string)
|
||||
if !ok {
|
||||
problems = append(problems, fmt.Sprintf("%s: its busy's command is %T; it is text", id, v))
|
||||
}
|
||||
b.Command = strings.TrimSpace(s)
|
||||
}
|
||||
if v, has := fields["timeout"]; has {
|
||||
s, _ := v.(string)
|
||||
d, err := time.ParseDuration(strings.TrimSpace(s))
|
||||
switch {
|
||||
case err != nil:
|
||||
problems = append(problems, fmt.Sprintf("%s: its busy's timeout is %v; it is a duration such as \"10s\"", id, v))
|
||||
case d <= 0 || d > BusyTimeoutCeiling:
|
||||
problems = append(problems, fmt.Sprintf("%s: an ask of its busy takes more than nothing and at most %s, not %s",
|
||||
id, BusyTimeoutCeiling, d))
|
||||
default:
|
||||
b.Timeout = d
|
||||
}
|
||||
}
|
||||
return b, true, problems
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user