Compare commits
82
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1c20b9fb9b | ||
|
|
91763dd70e | ||
|
|
eaec209a37 | ||
|
|
9ebfb87b5d | ||
|
|
9ade199740 | ||
|
|
0596f44414 | ||
|
|
d792646110 | ||
|
|
732238cb6a | ||
|
|
845c8b0e56 | ||
|
|
aa68b67389 | ||
|
|
a39661aa36 | ||
|
|
662fb0a57c | ||
|
|
0bc5659076 | ||
|
|
c23833d3e5 | ||
|
|
0ae600dc97 | ||
|
|
15af148520 | ||
|
|
6d620f24b7 | ||
|
|
34392e7491 | ||
|
|
cfa20f05ad | ||
|
|
f1438dff6a | ||
|
|
1ffbf9ea5b | ||
|
|
e05e7130ae | ||
|
|
80d3975762 | ||
|
|
f73ec7457a | ||
|
|
94702bcd79 | ||
|
|
de2d944c2a | ||
|
|
0796341c8d | ||
|
|
96155dea23 | ||
|
|
1361c1b456 | ||
|
|
c8bd137082 | ||
|
|
9908faaa02 | ||
|
|
238440d6f8 | ||
|
|
953446cc8f | ||
|
|
f692fe7da8 | ||
|
|
2b420603fe | ||
|
|
f70a99dbf4 | ||
|
|
a4e7f480b3 | ||
|
|
d7cac79585 | ||
|
|
377fd6d67d | ||
|
|
bb7c82cac6 | ||
|
|
0db956dd83 | ||
|
|
40f3658557 | ||
|
|
219bedbdd2 | ||
|
|
1bc5719b8b | ||
|
|
a2d2af00dc | ||
|
|
5933eb757d | ||
|
|
f4745447ba | ||
|
|
9588871d66 | ||
|
|
a0fc82fdcd | ||
|
|
d49d3a14d8 | ||
|
|
c1ae30b0a2 | ||
|
|
67bfa22132 | ||
|
|
4dc5f29d3e | ||
|
|
9747f3e1b1 | ||
|
|
d41d255f28 | ||
|
|
5794ccd569 | ||
|
|
eaed934629 | ||
|
|
4cf45b5ca0 | ||
|
|
d9e24e95a2 | ||
|
|
ee1e1a8303 | ||
|
|
24f5ee81bd | ||
|
|
b4900f52b8 | ||
|
|
19cfe0ef03 | ||
|
|
33917f91ac | ||
|
|
622b6b4f55 | ||
|
|
67dfeafd51 | ||
|
|
4f3efc98e0 | ||
|
|
7605a3fa56 | ||
|
|
9c548753b8 | ||
|
|
c58ce2c7fe | ||
|
|
ff1fcd4441 | ||
|
|
f047d0a4a9 | ||
|
|
97f9acfb51 | ||
|
|
16984aafc6 | ||
|
|
81a8e8a537 | ||
|
|
f7b57fb1ad | ||
|
|
e53ceadcf4 | ||
|
|
c65a9bd324 | ||
|
|
a2b619b638 | ||
|
|
8c3480c845 | ||
|
|
d7be28a1a9 | ||
|
|
3d0c8b02b2 |
@@ -1,2 +1,3 @@
|
||||
node_modules/
|
||||
dist/
|
||||
vendor/
|
||||
|
||||
@@ -20,7 +20,13 @@ Not the manifest schema (the control plane owns and parses that, in Go). The sha
|
||||
contains — the contract both a provider and a consumer conform to), and the tool and event types.
|
||||
These change rarely and deliberately; when one does, a rebuild of everything is *correct*.
|
||||
|
||||
### `provisioner` — the reconcile harness every provider shares
|
||||
### `provider-loop` — the reconcile harness every provider on the grant files shares
|
||||
Called `provisioner` until design 57 (novox/hq issue 678); that word is mesh-provisioner's now. The old path and
|
||||
names (`/provisioner`, `runProvisioner`, `provisionerTools`) stay aliases until the providers importing them move to
|
||||
the Go helper that answers consumers over the bus (`go/provider`, design 57 B2 and B3) and this loop retires with the
|
||||
grant files. Its events and tools keep their names (`provisioner.failing`, `provisioner_retirement`, …): the
|
||||
controller and the operator read them.
|
||||
|
||||
The loop is identical for postgres, redis, minio and umami: watch the grants directory, for each
|
||||
requested grant call the provider's adapter, write the sealed credential, handle withdrawal, and
|
||||
keep converging (`watch`). That loop lives here. A **module provides only the adapter** — "create
|
||||
@@ -39,11 +45,21 @@ never called to retire**; it is the delete of an adapter without `delete`, reach
|
||||
person's `provisioner_delete`. A stable set of more than 3, or more than half of those held where more
|
||||
than one is held, is not retired without a person: the provider waits for
|
||||
`provisioner_retire_approve` or `provisioner_retire_reject`, and says `provisioner.retirement`
|
||||
(`waiting`) through the `announce` option. Asked for again, a retired consumer is created — re-enabled
|
||||
(`waiting`). Asked for again, a retired consumer is created — re-enabled
|
||||
— at once. Given `module`, the four tools (`provisioner_retirement`, `provisioner_retire_approve`,
|
||||
`provisioner_retire_reject`, `provisioner_delete`) are registered for it; `provisionerTools(handle)`
|
||||
builds them otherwise. Additions and changes are never delayed.
|
||||
|
||||
**A provider that keeps failing a consumer says so on the bus** (novox/hq ADR 0224). A consumer whose
|
||||
create, check (`holds`) or secret has failed for five minutes without one success between
|
||||
(`failingAfterMs`) is announced `provisioner.failing` — the consumer, its machine, the class of error
|
||||
(`credentials-rejected`, `unreachable`, `secret-unreadable`, `refused`; an adapter's `classify` is
|
||||
believed over the words), the error with the password removed, since when and how many attempts — and
|
||||
again every fifteen minutes while it lasts. The first success after it is `provisioner.recovered`, as is
|
||||
a consumer no longer asked for or retired, and the first success for each consumer since the process
|
||||
started. Every event goes through the `announce` option, which defaults to the SDK's `emit`: a provider
|
||||
says them without asking, and a failed publish is said in its journal, never failing the act.
|
||||
|
||||
### `tools` — the tool-serving harness
|
||||
`registerModuleTools`, the `ToolDefinition` type, and the registry/worker that loads a module's
|
||||
tools and exposes them through the mesh's command surface. *How* a tool is declared and served is
|
||||
@@ -90,6 +106,25 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
|
||||
values and then every change. Through the runtime that launched the bundle, which refuses what the
|
||||
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
|
||||
|
||||
### `seats` and `asks` — asking a person, answering on a channel (Go)
|
||||
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
|
||||
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
|
||||
and `stdio.SeatAnswer` carry a proof (a code typed by a person, by request and reply, never kept),
|
||||
`stdio.SeatRecord` reads a record kept for the module, and `State(...).Create` and `.Update` write by
|
||||
compare-and-set. The runtime does each only as far as the module's membership lists. `go/asks` is the
|
||||
contract on top: the `Ask` an asker publishes and the checks it is held to (its `Whole` is what a channel that proves
|
||||
who answers shows in place of its `Explanation`, and its `Details` what the Details answer shows, hq issue 383), the `Warrant` it acts on
|
||||
(`For`, `Says`), and the `Message`, `Choice`, `LinkAsked` and `Code` a channel takes and says, with the
|
||||
subjects each travels on.
|
||||
`go/asker` is an asking module's client (ADR 0259 §10): `Ask` keeps an ask in the module's own state and then
|
||||
publishes it under the module's name; `Cancel`; `Where` reads its state from the router's record; `Listen` and
|
||||
`Take` hand the module each warrant once — marked acted on by compare-and-set before the module acts, and only
|
||||
for an ask it made, an option it offered, at that option's level, before it expired. A lost compare-and-set is
|
||||
`stdio.ErrChanged`, told by the runtime's error code.
|
||||
`State(...).Increment(key, attempts)` is a counter on top of the two (for numbers handed out once, such as an
|
||||
issue's): it reads the key, writes one more at the revision read, and on a lost write reads and tries again, at
|
||||
most `attempts` times; past that it is `stdio.ErrChanged` and nothing was counted.
|
||||
|
||||
### `primitives` — sealing, semver, resolved-env
|
||||
Seal/unseal for secrets, semantic version comparison, and the accessor a module uses to read its
|
||||
own resolved environment.
|
||||
@@ -117,12 +152,12 @@ import { UmamiClient } from "../client.js"; // the client lives in the module, n
|
||||
registerModuleTools("umami", (env) => getUmamiTools(new UmamiClient(env.UMAMI_URL)));
|
||||
```
|
||||
|
||||
A provider's provisioner:
|
||||
A provider's loop:
|
||||
|
||||
```ts
|
||||
import { runProvisioner, Grant, Credential } from "@novox/mesh-sdk/provisioner";
|
||||
import { runProviderLoop, Grant, Credential } from "@novox/mesh-sdk/provider-loop";
|
||||
|
||||
runProvisioner("analytics", {
|
||||
runProviderLoop("analytics", {
|
||||
async create(grant: Grant): Promise<Credential> { /* create a umami site, return the grant */ },
|
||||
async remove(grant: Grant): Promise<void> { /* delete the site */ },
|
||||
});
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Release 57: config and provisioning
|
||||
|
||||
This branch carries the work of hq design 57 (ADR 0348, ADR 0349) in this repository. Each package
|
||||
merges here through its own reviewed pull request; the branch reaches main as one pull request, in one
|
||||
delivery group with the branch of the same name in mesh-controller, mesh-host, mesh-sdk, mesh-catalog
|
||||
and hq, after a rehearsal in the lab. This file is deleted by the last commit before that pull request.
|
||||
+3
-3
@@ -2,8 +2,8 @@
|
||||
|
||||
`stdio.Serve("", tools)` speaks MCP over stdio to the node's tool runtime, as the module the runtime
|
||||
names (novox/hq ADR 0188, ADR 0193). A `Tool` is a name, a description, an argument schema and a
|
||||
function; calls run concurrently. A tool named `<seat>.<verb>` is the seat's implementation.
|
||||
`stdio.Emit(type, body)` publishes an event as the module, `stdio.Subscribe(pattern, handler)` receives the module's events (taken when the handler returns nil, offered again when it returns an error — the one rule is in the repository README, "Taking an event") and `stdio.Ask(key, body)` calls a tool, and `stdio.State(name)` gets, puts, deletes, lists and watches the module's declared state — all through the runtime (novox/hq ADR 0193, 0198, 0201). Standard library only.
|
||||
function; calls run concurrently. `stdio.CallerOf(args)` inside a tool is who called it: the bus user the runtime took from the subject the call arrived on, named in the call's `_meta`, never read from the arguments — or `stdio.Unproven` when the runtime named none (novox/hq issue 365). A tool named `<seat>.<verb>` is the seat's implementation.
|
||||
`stdio.Emit(type, body)` publishes an event as the module, `stdio.Subscribe(pattern, handler)` receives the module's events (taken when the handler returns nil, offered again when it returns an error — the one rule is in the repository README, "Taking an event") and `stdio.Ask(key, body)` calls a tool, and `stdio.State(name)` gets, puts, deletes, lists, watches and reads the kept history of the module's declared state, and `stdio.Log(name)` appends to, reads, and measures a log of operations the module declared — all through the runtime (novox/hq ADR 0193, 0198, 0201, 0297, 0307). `stdio.Config(key)` and `stdio.Bound(provision, key)` read, and `stdio.WatchConfig` and `stdio.WatchBound` watch, the module's own config values and the non-secret values of what it is bound to, for what its manifest declares: the runtime asks its machine's provisioner client as the module it launched, never a module the code names. `stdio.Secret(name)` is refused while the module's code runs in the tool runner; its secrets reach it through its files (novox/hq ADR 0349 §14, ADR 0370). `provider` is a providing module's half of a binding (novox/hq ADR 0349 §12, issue 678): it sees each consumer that binds the provider in the plans of the assignments mirror, writes one entry per consumer to `<module>_bound` (`<node>.<module>.<provision>[.<local>]`: served values composed with the consumer's identity, the login, and the name of the vault's credential, never a secret), hands the provider's adapter the vault's value (`provision/<provider>/<provision>` shared, `pair/<provider>/<provision>/<node>/<module>[/<local>]` per consumer), refuses a derived value to a consumer bound under a local name, naming both ends, and removes the entry of one that unbinds; the bus and the vault are interfaces the provider wires, the vault as `provider.ConsumersFile`, the file its machine's provisioner client fills from `${consumers:<provision>}` with each bound consumer's credential (novox/hq ADR 0373), since a provider in the tool runner never asks the vault itself. Standard library only.
|
||||
|
||||
## Installing
|
||||
|
||||
@@ -16,4 +16,4 @@ at the same version as the TypeScript package.
|
||||
|
||||
What it holds, under ADR 0039's test: the stdio loop and the tool type, and `cli`, the local protocol
|
||||
`mesh-cli` speaks to the node-engine on its machine (novox/hq ADR 0272): the socket's path and one request and one
|
||||
answer. Nothing else — no bus, no client, nothing of any module's. `go build ./example` and pipe a `tools/list` request into it.
|
||||
answer; and `seal`, the mesh's sealed box (novox/hq ADR 0206: X25519, HKDF-SHA256, AES-256-GCM, `{v, eph, iv, tag, ct}`), byte for byte the box the licence manager made before it lived here and the TypeScript `@novox/mesh-sdk/seal` makes. Nothing else — no bus, no client, nothing of any module's. `go build ./example` and pipe a `tools/list` request into it.
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
// Package asker is a module's client for asking a person (novox/hq ADR 0259 §10): it publishes an ask under
|
||||
// the module's own name, cancels it, reads its state from the router's record, and hands the module each
|
||||
// warrant once — only one for an ask it made, for an option it offered, at that option's level, before it
|
||||
// expired. The module then acts with its own grants.
|
||||
//
|
||||
// It keeps what it asked, and which warrants it acted on, in a state of the module's own (declared in its
|
||||
// manifest, by default `asked`): the ask is written before it is published, and a warrant is marked acted on
|
||||
// by compare-and-set before the module's handler runs, so a redelivery, or a second instance, acts once.
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
// Client asks as one module.
|
||||
type Client struct {
|
||||
// Module is the asking module's name: the one the bus grants its asks under.
|
||||
Module string
|
||||
// State is the module's own state the asks are kept in.
|
||||
State Store
|
||||
// Now is the clock; nil is time.Now.
|
||||
Now func() time.Time
|
||||
// Publish, Record and Subscribe reach the bus; nil is the stdio SDK's, through the module's runtime.
|
||||
Publish func(subject string, body any, id string) error
|
||||
Record func(key string) (*stdio.StateEntry, error)
|
||||
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
|
||||
}
|
||||
|
||||
// Store is where the asks are kept: stdio.State(name) is one.
|
||||
type Store interface {
|
||||
Get(key string) (*stdio.StateEntry, error)
|
||||
Create(key string, value any) (uint64, error)
|
||||
Update(key string, value any, revision uint64) (uint64, error)
|
||||
}
|
||||
|
||||
// New is a client for the module the runtime serves, keeping its asks in the state named.
|
||||
func New(state string) *Client {
|
||||
return &Client{Module: os.Getenv("MESH_MODULE"), State: stdio.State(state)}
|
||||
}
|
||||
|
||||
func (c *Client) now() time.Time {
|
||||
if c.Now != nil {
|
||||
return c.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
// Kept is one ask as the module keeps it.
|
||||
type Kept struct {
|
||||
Ask asks.Ask `json:"ask"`
|
||||
Asked time.Time `json:"asked"`
|
||||
Acted bool `json:"acted,omitempty"`
|
||||
Ended asks.Outcome `json:"ended,omitempty"`
|
||||
Answer *asks.Warrant `json:"answer,omitempty"`
|
||||
}
|
||||
|
||||
func key(id string) string { return "ask." + id }
|
||||
|
||||
// Ask checks an ask, keeps it, and publishes it under the module's own name. An ask with an id already kept
|
||||
// is refused: a new question is a new id.
|
||||
func (c *Client) Ask(a asks.Ask) error {
|
||||
if err := a.Check(c.now()); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := c.State.Create(key(a.ID), Kept{Ask: a, Asked: c.now()}); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return fmt.Errorf("an ask %s was already made: a new question is a new id", a.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return c.publish(asks.AskSubject(c.Module), a, a.ID)
|
||||
}
|
||||
|
||||
// Cancel takes an ask back: the router ends it and edits every copy.
|
||||
func (c *Client) Cancel(id string) error {
|
||||
return c.publish(asks.CancelSubject(c.Module), map[string]string{"id": id}, "cancel."+id)
|
||||
}
|
||||
|
||||
func (c *Client) publish(subject string, body any, id string) error {
|
||||
if c.Publish != nil {
|
||||
return c.Publish(subject, body, id)
|
||||
}
|
||||
return stdio.SeatPublish(subject, body, id)
|
||||
}
|
||||
|
||||
// Where is an ask's state as the router keeps it: its state and its warrant, if any — read under the
|
||||
// module's own name and no other's.
|
||||
func (c *Client) Where(id string) (state string, w *asks.Warrant, err error) {
|
||||
read := c.Record
|
||||
if read == nil {
|
||||
read = func(k string) (*stdio.StateEntry, error) { return stdio.SeatRecord("", k) }
|
||||
}
|
||||
e, err := read(id)
|
||||
if err != nil || e == nil {
|
||||
return "", nil, err
|
||||
}
|
||||
var rec struct {
|
||||
State string `json:"state"`
|
||||
Warrant *asks.Warrant `json:"warrant"`
|
||||
}
|
||||
if err := json.Unmarshal(e.Value, &rec); err != nil {
|
||||
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
|
||||
}
|
||||
return rec.State, rec.Warrant, nil
|
||||
}
|
||||
|
||||
// Handled is what came of a warrant: acted on, refused (and why), or already done.
|
||||
type Handled string
|
||||
|
||||
const (
|
||||
Acted Handled = "acted"
|
||||
Refused Handled = "refused"
|
||||
Done Handled = "already acted on"
|
||||
Recorded Handled = "ended without a choice"
|
||||
)
|
||||
|
||||
// Take hands act a warrant once, after holding it to the ask it answers. act performs the option with the
|
||||
// module's own grants; its error is said and the warrant stays acted on (a person's decision is not
|
||||
// performed twice; the module says the failure in its own record).
|
||||
func (c *Client) Take(w asks.Warrant, act func(asks.Option, asks.Warrant) error) (Handled, error) {
|
||||
if w.Asker != c.Module {
|
||||
return Refused, fmt.Errorf("a warrant for %s reached %s", w.Asker, c.Module)
|
||||
}
|
||||
e, err := c.State.Get(key(w.Ask))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if e == nil {
|
||||
return Refused, fmt.Errorf("%s holds no ask %s", c.Module, w.Ask)
|
||||
}
|
||||
var k Kept
|
||||
if err := json.Unmarshal(e.Value, &k); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if k.Acted || k.Ended != "" {
|
||||
return Done, nil
|
||||
}
|
||||
if w.Outcome != asks.OutcomeChosen {
|
||||
k.Ended = w.Outcome
|
||||
k.Answer = &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil && !errors.Is(err, stdio.ErrChanged) {
|
||||
return "", err
|
||||
}
|
||||
return Recorded, nil
|
||||
}
|
||||
o, err := w.For(c.Module, k.Ask)
|
||||
if err != nil {
|
||||
return Refused, err
|
||||
}
|
||||
// Marked before acting: of two deliveries, or two instances, only the one whose write stands acts.
|
||||
k.Acted, k.Ended, k.Answer = true, asks.OutcomeChosen, &w
|
||||
if _, err := c.State.Update(key(w.Ask), k, e.Revision); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
return Done, nil
|
||||
}
|
||||
return "", err
|
||||
}
|
||||
return Acted, act(o, w)
|
||||
}
|
||||
|
||||
// Listen hands every warrant for this module to Take, with act. It asks the runtime for the module's
|
||||
// events matching the router's `decided` subject for this module.
|
||||
func (c *Client) Listen(act func(asks.Option, asks.Warrant) error, said func(Handled, asks.Warrant, error)) error {
|
||||
subscribe := c.Subscribe
|
||||
if subscribe == nil {
|
||||
subscribe = stdio.Subscribe
|
||||
}
|
||||
return subscribe(asks.Seat+".decided."+c.Module, func(e stdio.Envelope) error {
|
||||
if !strings.HasPrefix(e.Key, asks.Seat+".decided.") {
|
||||
return nil
|
||||
}
|
||||
var w asks.Warrant
|
||||
if err := json.Unmarshal(e.Body, &w); err != nil {
|
||||
if said != nil {
|
||||
said(Refused, w, fmt.Errorf("a warrant could not be read: %w", err))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
h, err := c.Take(w, act)
|
||||
if h == "" && err != nil {
|
||||
return err // the state could not be reached: offered again
|
||||
}
|
||||
if said != nil {
|
||||
said(h, w, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
package asker
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
type mem struct {
|
||||
vals map[string]json.RawMessage
|
||||
revs map[string]uint64
|
||||
n uint64
|
||||
}
|
||||
|
||||
func (m *mem) Get(k string) (*stdio.StateEntry, error) {
|
||||
v, ok := m.vals[k]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return &stdio.StateEntry{Key: k, Value: v, Revision: m.revs[k]}, nil
|
||||
}
|
||||
func (m *mem) Create(k string, v any) (uint64, error) {
|
||||
if _, ok := m.vals[k]; ok {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) Update(k string, v any, rev uint64) (uint64, error) {
|
||||
if rev == 0 || m.revs[k] != rev {
|
||||
return 0, stdio.ErrChanged
|
||||
}
|
||||
return m.put(k, v)
|
||||
}
|
||||
func (m *mem) put(k string, v any) (uint64, error) {
|
||||
raw, _ := json.Marshal(v)
|
||||
m.n++
|
||||
m.vals[k], m.revs[k] = raw, m.n
|
||||
return m.n, nil
|
||||
}
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func client() (*Client, map[string][]string) {
|
||||
sent := map[string][]string{}
|
||||
c := &Client{Module: "mesh-delivery", State: &mem{vals: map[string]json.RawMessage{}, revs: map[string]uint64{}},
|
||||
Now: func() time.Time { return now },
|
||||
Publish: func(subject string, body any, id string) error {
|
||||
sent[subject] = append(sent[subject], id)
|
||||
return nil
|
||||
}}
|
||||
return c, sent
|
||||
}
|
||||
|
||||
// releaseHQ is the act the ask's one option binds.
|
||||
var releaseHQ = map[string]string{"verb": "release", "delivery": "hq"}
|
||||
|
||||
func held() asks.Ask {
|
||||
binds, _ := asks.ActDigest(releaseHQ)
|
||||
return asks.Ask{ID: "d-1", Headline: "Delivery of hq held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: asks.Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held",
|
||||
Options: []asks.Option{{ID: "release", Label: "Release", Does: "it goes on", Level: asks.Approve, Binds: binds}}}
|
||||
}
|
||||
|
||||
func warrant() asks.Warrant {
|
||||
return asks.Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: asks.OutcomeChosen, Option: "release",
|
||||
Label: "Release", Level: asks.Approve, At: now.Add(time.Minute), AskDigest: held().Digest(),
|
||||
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"}}
|
||||
}
|
||||
|
||||
func TestAnAskIsKeptThenPublishedUnderTheModulesOwnName(t *testing.T) {
|
||||
c, sent := client()
|
||||
if err := c.Ask(held()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := sent[asks.AskSubject("mesh-delivery")]; len(got) != 1 || got[0] != "d-1" {
|
||||
t.Fatalf("%v", sent)
|
||||
}
|
||||
if err := c.Ask(held()); err == nil || !strings.Contains(err.Error(), "already made") {
|
||||
t.Errorf("the same id was asked twice: %v", err)
|
||||
}
|
||||
bad := held()
|
||||
bad.Who = "agent"
|
||||
if err := c.Ask(bad); err == nil {
|
||||
t.Error("an ask the contract refuses was published")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWarrantIsActedOnOnceAndOnlyForItsOwnAsk(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
acted := 0
|
||||
act := func(o asks.Option, _ asks.Warrant) error {
|
||||
if err := o.Performs(releaseHQ); err != nil {
|
||||
t.Errorf("the bound act: %v", err)
|
||||
}
|
||||
acted++
|
||||
return nil
|
||||
}
|
||||
if h, err := c.Take(warrant(), act); h != Acted || err != nil || acted != 1 {
|
||||
t.Fatalf("%s %v %d", h, err, acted)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), act); h != Done || acted != 1 {
|
||||
t.Errorf("a redelivery was acted on: %s", h)
|
||||
}
|
||||
for why, change := range map[string]func(*asks.Warrant){
|
||||
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-controller" },
|
||||
"an ask not made": func(w *asks.Warrant) { w.Ask = "d-9" },
|
||||
"another level": func(w *asks.Warrant) { w.Level = asks.Acknowledge },
|
||||
"an option not offered": func(w *asks.Warrant) { w.Option = "delete" },
|
||||
"after its expiry": func(w *asks.Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *asks.Warrant) { w.AskDigest = "sha256:00" },
|
||||
} {
|
||||
c2, _ := client()
|
||||
_ = c2.Ask(held())
|
||||
w := warrant()
|
||||
change(&w)
|
||||
if h, _ := c2.Take(w, act); h == Acted {
|
||||
t.Errorf("a warrant from %s was acted on", why)
|
||||
}
|
||||
}
|
||||
if acted != 1 {
|
||||
t.Errorf("acted %d times", acted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskEndedWithoutAChoiceIsRecordedAndNothingDone(t *testing.T) {
|
||||
c, _ := client()
|
||||
_ = c.Ask(held())
|
||||
w := warrant()
|
||||
w.Outcome, w.Option, w.By = asks.OutcomeExpired, "", nil
|
||||
if h, err := c.Take(w, func(asks.Option, asks.Warrant) error { t.Fatal("acted"); return nil }); h != Recorded || err != nil {
|
||||
t.Fatalf("%s %v", h, err)
|
||||
}
|
||||
if h, _ := c.Take(warrant(), func(asks.Option, asks.Warrant) error { t.Fatal("acted after it ended"); return nil }); h != Done {
|
||||
t.Errorf("%s", h)
|
||||
}
|
||||
}
|
||||
+573
@@ -0,0 +1,573 @@
|
||||
// Package asks is the contract of asking a person and answering on a channel (novox/hq ADR 0259): the
|
||||
// shapes an asker, the router and a channel exchange on the bus, and the subjects they travel on. No
|
||||
// transport and no channel's service: an asker publishes an Ask under its own name and acts on the Warrant
|
||||
// it hears; the router holds the ask, sends channels a Message, and judges the Choice a channel says; a
|
||||
// channel shows a Message and says what was chosen and by whom, as its service authenticated it.
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The seats (novox/hq ADR 0259 §3).
|
||||
const (
|
||||
// Seat is held by the router: an ask and its cancel are its accepts, a warrant its event, each named by
|
||||
// the asker.
|
||||
Seat = "operator-channel"
|
||||
// ChannelSeat is the kinded bench a channel holds to show and say: its accepts carry the kind.
|
||||
ChannelSeat = "channel"
|
||||
// IntakeSeat is the kinded bench a channel holds to say what was chosen: its events and proofs carry
|
||||
// the kind.
|
||||
IntakeSeat = "intake"
|
||||
)
|
||||
|
||||
// AskSubject is where an asker publishes an ask, CancelSubject its cancel, and DecidedSubject where it
|
||||
// hears the warrant, or the ask's end without one.
|
||||
func AskSubject(asker string) string { return "mesh.seat." + Seat + ".accept.ask." + asker }
|
||||
func CancelSubject(asker string) string { return "mesh.seat." + Seat + ".accept.cancel." + asker }
|
||||
func DecidedSubject(asker string) string { return "mesh.seat." + Seat + ".event.decided." + asker }
|
||||
|
||||
// The work a channel takes, on ChannelSubject.
|
||||
const (
|
||||
Show = "show" // a message offering answers
|
||||
Edit = "edit" // a message shown before, replaced
|
||||
Send = "send" // a message offering nothing
|
||||
)
|
||||
|
||||
// ChannelSubject is where the router sends a channel of a kind its work.
|
||||
func ChannelSubject(verb, kind string) string {
|
||||
return "mesh.seat." + ChannelSeat + ".accept." + verb + "." + kind
|
||||
}
|
||||
|
||||
// What a channel says, on IntakeSubject.
|
||||
const (
|
||||
Chosen = "choice" // a button tapped
|
||||
Link = "link" // somebody asked to be linked as the operator
|
||||
)
|
||||
|
||||
// IntakeSubject is where a channel of a kind says what arrived.
|
||||
func IntakeSubject(what, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".event." + what + "." + kind
|
||||
}
|
||||
|
||||
// CodeProof is the one proof verb: a code the operator typed, carried by request and reply, never kept.
|
||||
const CodeProof = "code"
|
||||
|
||||
// ProofSubject is where a channel of a kind asks a proof.
|
||||
func ProofSubject(verb, kind string) string {
|
||||
return "mesh.seat." + IntakeSeat + ".proof." + verb + "." + kind
|
||||
}
|
||||
|
||||
// A Level is how much proof an option's answer needs (novox/hq ADR 0234 §8, the glossary's assurance level).
|
||||
type Level string
|
||||
|
||||
const (
|
||||
// Acknowledge performs only what any granted principal may already do: silencing, details. No proof.
|
||||
Acknowledge Level = "acknowledge"
|
||||
// Approve needs one proof: a verified sender (a linked account, linked an hour or more) or a code.
|
||||
Approve Level = "approve"
|
||||
// Destroy needs two proofs, one of them a code.
|
||||
Destroy Level = "destroy"
|
||||
)
|
||||
|
||||
// Rank orders the levels; an unknown level ranks above every known one, so it is never taken as less.
|
||||
func (l Level) Rank() int {
|
||||
switch l {
|
||||
case Acknowledge:
|
||||
return 0
|
||||
case Approve:
|
||||
return 1
|
||||
case Destroy:
|
||||
return 2
|
||||
}
|
||||
return 3
|
||||
}
|
||||
|
||||
// Operator is the one role an ask may be answered by today.
|
||||
const Operator = "operator"
|
||||
|
||||
// Option is one answer an ask offers: a label for the button, what it does in plain words, its level.
|
||||
type Option struct {
|
||||
ID string `json:"id"`
|
||||
Label string `json:"label"`
|
||||
Does string `json:"does"`
|
||||
Level Level `json:"level"`
|
||||
// Binds is the digest of exactly what the asker performs when this option is chosen — the verb, the
|
||||
// machine and every argument — as ActDigest gives it. It travels in the ask, so the router's warrant,
|
||||
// which names the ask's digest, names it too: a warrant then authorises that act and no other, and an
|
||||
// asker whose record of the act changed after it asked finds the digests differ and does nothing.
|
||||
// Required on an option above acknowledge.
|
||||
Binds string `json:"binds,omitempty"`
|
||||
}
|
||||
|
||||
// An Act is what an option binds: named fields, each a string — the verb, the machine, the level, and each
|
||||
// argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over these names and
|
||||
// values alone, never over how a language happens to encode a struct.
|
||||
type Act map[string]string
|
||||
|
||||
// ActDigest is the digest an asker puts in Option.Binds: SHA-256 over the act's canonical encoding (canonical),
|
||||
// written "sha256:<hex>". The same names and values give the same digest in any language, whatever order
|
||||
// they were set in; a field renamed, added or emptied gives another.
|
||||
func ActDigest(act Act) (string, error) {
|
||||
if len(act) == 0 {
|
||||
return "", fmt.Errorf("the act cannot be digested: it names nothing")
|
||||
}
|
||||
keys := make([]string, 0, len(act))
|
||||
for k := range act {
|
||||
if k == "" {
|
||||
return "", fmt.Errorf("the act cannot be digested: a field has no name")
|
||||
}
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.act.v1\n")
|
||||
for _, k := range keys {
|
||||
canonical(&b, k)
|
||||
canonical(&b, act[k])
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read
|
||||
// as another's end or start, so two different sequences of values never encode the same.
|
||||
func canonical(b *strings.Builder, v string) {
|
||||
b.WriteString(strconv.Itoa(len(v)))
|
||||
b.WriteByte(':')
|
||||
b.WriteString(v)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
|
||||
// Digest is the digest of the ask exactly as its asker published it: its id, words, options with what each
|
||||
// binds, who answers, and its expiry. The router puts it in the warrant (Warrant.AskDigest), and an asker
|
||||
// acts only on a warrant whose digest is that of the ask it keeps — so a warrant answers one ask, as the
|
||||
// person was shown it, and nothing published under the same id before or after.
|
||||
//
|
||||
// It is over the ask's named fields in a fixed order, each written canonically, and the expiry as UTC
|
||||
// RFC 3339 to the nanosecond — never over a language's encoding of the struct, so a field added to Ask
|
||||
// later changes no digest until it is added here, on purpose. Whole and Details (novox/hq issue 383) follow
|
||||
// the options only when the ask gives either: an ask without them digests as it did before they existed, so
|
||||
// a router and an asker of different builds still agree on every such ask.
|
||||
func (a Ask) Digest() string {
|
||||
var b strings.Builder
|
||||
b.WriteString("novox.ask.v1\n")
|
||||
for _, v := range []string{a.ID, a.Headline, a.Explanation, a.Who,
|
||||
a.Expires.UTC().Format(time.RFC3339Nano), a.OnExpiry, a.About, strconv.FormatBool(a.Urgent),
|
||||
strconv.Itoa(len(a.Options))} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
for _, o := range a.Options {
|
||||
for _, v := range []string{o.ID, o.Label, o.Does, string(o.Level), o.Binds} {
|
||||
canonical(&b, v)
|
||||
}
|
||||
}
|
||||
// Whole and Details are written whenever either or WholeDetails is given, and WholeDetails after them whenever
|
||||
// it is given (novox/hq ADR 0347): the fields after the options are then always one of three fixed shapes —
|
||||
// none, the two, or the three — and the count of values the sequence holds tells them apart, so no two asks
|
||||
// encode the same (the review of mesh-sdk #25). An ask with none of them digests as before they existed, and
|
||||
// one without WholeDetails as before it existed.
|
||||
if a.Whole != "" || a.Details != "" || a.WholeDetails != "" {
|
||||
canonical(&b, a.Whole)
|
||||
canonical(&b, a.Details)
|
||||
}
|
||||
if a.WholeDetails != "" {
|
||||
canonical(&b, a.WholeDetails)
|
||||
}
|
||||
sum := sha256.Sum256([]byte(b.String()))
|
||||
return "sha256:" + hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// Ask is a request for a person's word (novox/hq ADR 0259 §4).
|
||||
type Ask struct {
|
||||
// ID is the asker's own, unique to it.
|
||||
ID string `json:"id"`
|
||||
// Headline names the thing and what is wrong, in a few plain words; Explanation is what happened and
|
||||
// what it means. Both are held to the plain rule and the content rule by the router.
|
||||
Headline string `json:"headline"`
|
||||
Explanation string `json:"explanation"`
|
||||
Options []Option `json:"options"`
|
||||
// Who may answer: Operator.
|
||||
Who string `json:"who"`
|
||||
// Expires is when the ask ends unanswered; OnExpiry is what the asker then does, in words the person
|
||||
// is shown ("the delivery stays held"). An ask that authorises never defaults.
|
||||
Expires time.Time `json:"expires"`
|
||||
OnExpiry string `json:"on-expiry"`
|
||||
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
|
||||
About string `json:"about,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
// Whole is the explanation with its exact values whole, shown in place of Explanation on a channel kind
|
||||
// that proves who answers (verified-sender) and carries the ask's answers (novox/hq issue 383): what the
|
||||
// person approves — a mount point, a share, a private address — must be readable where they approve it.
|
||||
// The asker withholds a value shaped like a secret in it, and the router refuses the ask when one is left;
|
||||
// Explanation stays under the whole content rule everywhere else. Empty, Explanation is shown everywhere.
|
||||
Whole string `json:"whole,omitempty"`
|
||||
// Details are the ask's secondary words, line by line under the content rule: who proposed it, a
|
||||
// fingerprint, how to read it whole. The router shows them in the note of the ask's own message
|
||||
// (Message.Note), after what the ask's options do and when it expires — never in its body (novox/hq
|
||||
// issue 639).
|
||||
Details string `json:"details,omitempty"`
|
||||
// WholeDetails are the Details with their exact values whole, shown in place of Details where Whole is shown
|
||||
// in place of the explanation (a kind that proves who answers, given the ask's answers): every value an
|
||||
// approval sets, whole, so the person reads exactly what they approve (novox/hq issue 383). Held, like
|
||||
// Whole, to the secret rule alone; the router refuses the ask when a secret's shape is left in it. Empty,
|
||||
// Details are shown everywhere.
|
||||
WholeDetails string `json:"whole-details,omitempty"`
|
||||
}
|
||||
|
||||
// What an ask's explanation is held to, so it reads at a glance on a phone (novox/hq issue 639): a summary
|
||||
// of two or three short sentences on its first line — what changes, why, what it costs or risks — then at
|
||||
// most three points, a line each, only where they add something. Whole is held to the same. Who asked, when,
|
||||
// the fingerprint and how to read it whole are Details, never the explanation.
|
||||
const (
|
||||
SummaryLength = 300
|
||||
MostPoints = 3
|
||||
PointLength = 160
|
||||
)
|
||||
|
||||
// Glance says why an explanation does not read at a glance, or nil: its first line, the summary, longer than
|
||||
// SummaryLength; more than MostPoints lines after it; a point longer than PointLength. Blank lines are not
|
||||
// counted. An asker holds its asks to it in its own tests; the router says, without refusing, an ask that
|
||||
// does not hold to it.
|
||||
func Glance(explanation string) error {
|
||||
var lines []string
|
||||
for _, l := range strings.Split(explanation, "\n") {
|
||||
if strings.TrimSpace(l) != "" {
|
||||
lines = append(lines, strings.TrimSpace(l))
|
||||
}
|
||||
}
|
||||
var problems []string
|
||||
switch {
|
||||
case len(lines) == 0:
|
||||
problems = append(problems, "it has no summary")
|
||||
case len([]rune(lines[0])) > SummaryLength:
|
||||
problems = append(problems, fmt.Sprintf("its summary is %d characters, more than %d", len([]rune(lines[0])), SummaryLength))
|
||||
}
|
||||
if len(lines) > 1+MostPoints {
|
||||
problems = append(problems, fmt.Sprintf("it has %d points after its summary, more than %d", len(lines)-1, MostPoints))
|
||||
}
|
||||
for i, l := range lines {
|
||||
if i > 0 && len([]rune(l)) > PointLength {
|
||||
problems = append(problems, fmt.Sprintf("its point %d is %d characters, more than %d", i, len([]rune(l)), PointLength))
|
||||
}
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the explanation does not read at a glance: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
|
||||
const (
|
||||
MostOptions = 4
|
||||
MostOpen = 3
|
||||
ApproveLasts = 24 * time.Hour
|
||||
DestroyLasts = 10 * time.Minute
|
||||
HeadlineLength = 60
|
||||
LabelLength = 24
|
||||
)
|
||||
|
||||
var usableID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`)
|
||||
|
||||
// UsableID says whether a name can be an ask's or an option's id: a key and a subject token both.
|
||||
func UsableID(id string) bool { return usableID.MatchString(id) }
|
||||
|
||||
// Highest is the highest level among the ask's options.
|
||||
func (a Ask) Highest() Level {
|
||||
high := Acknowledge
|
||||
for _, o := range a.Options {
|
||||
if o.Level.Rank() > high.Rank() {
|
||||
high = o.Level
|
||||
}
|
||||
}
|
||||
return high
|
||||
}
|
||||
|
||||
// Option is the option of this id, or false.
|
||||
func (a Ask) Option(id string) (Option, bool) {
|
||||
for _, o := range a.Options {
|
||||
if o.ID == id {
|
||||
return o, true
|
||||
}
|
||||
}
|
||||
return Option{}, false
|
||||
}
|
||||
|
||||
// Check is what an ask is held to before anything is shown: every refusal, in words its asker can act on.
|
||||
func (a Ask) Check(now time.Time) error {
|
||||
var problems []string
|
||||
say := func(format string, args ...any) { problems = append(problems, fmt.Sprintf(format, args...)) }
|
||||
if !UsableID(a.ID) {
|
||||
say("its id %q is not letters, digits, - and _, at most 64", a.ID)
|
||||
}
|
||||
if strings.TrimSpace(a.Headline) == "" || len([]rune(a.Headline)) > HeadlineLength {
|
||||
say("its headline is empty or longer than %d characters", HeadlineLength)
|
||||
}
|
||||
if strings.TrimSpace(a.Explanation) == "" {
|
||||
say("it explains nothing")
|
||||
}
|
||||
if a.Who != Operator {
|
||||
say("it is answered by %q, and only the operator answers today", a.Who)
|
||||
}
|
||||
if len(a.Options) == 0 || len(a.Options) > MostOptions {
|
||||
say("it offers %d options, and an ask offers one to %d", len(a.Options), MostOptions)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, o := range a.Options {
|
||||
switch {
|
||||
case !UsableID(o.ID):
|
||||
say("an option's id %q is not letters, digits, - and _", o.ID)
|
||||
case seen[o.ID]:
|
||||
say("the option %s is offered twice", o.ID)
|
||||
}
|
||||
seen[o.ID] = true
|
||||
if strings.TrimSpace(o.Label) == "" || len([]rune(o.Label)) > LabelLength {
|
||||
say("the option %s's label is empty or longer than %d characters", o.ID, LabelLength)
|
||||
}
|
||||
if strings.TrimSpace(o.Does) == "" {
|
||||
say("the option %s does not say what it does", o.ID)
|
||||
}
|
||||
if o.Level.Rank() > Destroy.Rank() {
|
||||
say("the option %s has the level %q, which is none of acknowledge, approve, destroy", o.ID, o.Level)
|
||||
}
|
||||
if o.Level != Acknowledge && !strings.HasPrefix(o.Binds, "sha256:") {
|
||||
say("the option %s authorises and does not bind what it performs (its binds is not an ActDigest)", o.ID)
|
||||
}
|
||||
}
|
||||
if !a.Expires.After(now) {
|
||||
say("it expires before it is asked")
|
||||
}
|
||||
switch a.Highest() {
|
||||
case Approve:
|
||||
if a.Expires.After(now.Add(ApproveLasts)) {
|
||||
say("an ask that approves lasts at most %s", ApproveLasts)
|
||||
}
|
||||
case Destroy:
|
||||
if a.Expires.After(now.Add(DestroyLasts)) {
|
||||
say("an ask that destroys lasts at most %s", DestroyLasts)
|
||||
}
|
||||
}
|
||||
if a.Highest() != Acknowledge && strings.TrimSpace(a.OnExpiry) == "" {
|
||||
say("it does not say what happens when nobody answers, and an ask that authorises never defaults")
|
||||
}
|
||||
if a.About != "" && strings.ContainsAny(a.About, " \n") {
|
||||
say("what it is about is a key, without spaces")
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return errors.New("the ask is refused: " + strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Outcome is how an ask ended.
|
||||
type Outcome string
|
||||
|
||||
const (
|
||||
OutcomeChosen Outcome = "chosen" // a person chose an option: a warrant
|
||||
OutcomeExpired Outcome = "expired" // nobody answered in time
|
||||
OutcomeCancelled Outcome = "cancelled" // its asker took it back
|
||||
OutcomeReplaced Outcome = "replaced" // a newer ask about the same thing replaced it
|
||||
OutcomeRefused Outcome = "refused" // it was never shown: Words says why
|
||||
)
|
||||
|
||||
// Person is who chose, as the router verified them.
|
||||
type Person struct {
|
||||
// Who is the role: Operator.
|
||||
Who string `json:"who"`
|
||||
// Kind is the channel kind they answered on, Identity their account on that service, Display the name
|
||||
// the service shows, and Verified how the router knew it was them.
|
||||
Kind string `json:"kind"`
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Verified string `json:"verified"`
|
||||
}
|
||||
|
||||
// Warrant is the router's record that a person chose one option of one ask, or the ask's end without one
|
||||
// (novox/hq ADR 0259 §6). It carries no secret.
|
||||
type Warrant struct {
|
||||
Ask string `json:"ask"`
|
||||
Asker string `json:"asker"`
|
||||
About string `json:"about,omitempty"`
|
||||
Outcome Outcome `json:"outcome"`
|
||||
// Option, Label and Level are the option chosen; By who chose it, Channel the module it came through,
|
||||
// Proofs which proofs were present (P1, P2, P3).
|
||||
Option string `json:"option,omitempty"`
|
||||
Label string `json:"label,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
By *Person `json:"by,omitempty"`
|
||||
Channel string `json:"channel,omitempty"`
|
||||
Proofs []string `json:"proofs,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
// AskDigest is the digest of the ask as the router took it (Ask.Digest): the warrant answers that ask
|
||||
// alone, with the options it bound.
|
||||
AskDigest string `json:"ask-digest,omitempty"`
|
||||
// Words are why an ask ended without a choice, or what refused it.
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
// Says is the warrant in the words an asker records with its act: "the operator, via telegram (user id
|
||||
// verified), chose Release".
|
||||
func (w Warrant) Says() string {
|
||||
if w.Outcome != OutcomeChosen || w.By == nil {
|
||||
return fmt.Sprintf("no person chose: the ask %s %s", w.Ask, w.Outcome)
|
||||
}
|
||||
via := w.By.Kind
|
||||
if w.By.Verified != "" {
|
||||
via += " (" + w.By.Verified + ")"
|
||||
}
|
||||
return fmt.Sprintf("the %s, via %s, chose %s", w.By.Who, via, w.Label)
|
||||
}
|
||||
|
||||
// For checks a warrant against the ask its asker made: the same asker and ask, the same ask's digest (so the
|
||||
// same words, options and binds), a choice, an option the ask offered, at that option's level, before the
|
||||
// ask expired. An asker acts on nothing else, and then performs only what the option's Binds names.
|
||||
func (w Warrant) For(asker string, a Ask) (Option, error) {
|
||||
if w.Asker != asker || w.Ask != a.ID {
|
||||
return Option{}, fmt.Errorf("the warrant is for %s's ask %s, not %s's %s", w.Asker, w.Ask, asker, a.ID)
|
||||
}
|
||||
if w.Outcome != OutcomeChosen || w.By == nil || w.By.Who != a.Who {
|
||||
return Option{}, fmt.Errorf("the ask %s ended %s; no person chose", a.ID, w.Outcome)
|
||||
}
|
||||
if d := a.Digest(); w.AskDigest != d {
|
||||
return Option{}, fmt.Errorf("the warrant answers an ask whose digest is %q, and the ask %s kept here is %s: "+
|
||||
"it was not the ask the person was shown", w.AskDigest, a.ID, d)
|
||||
}
|
||||
o, offered := a.Option(w.Option)
|
||||
if !offered {
|
||||
return Option{}, fmt.Errorf("the ask %s offered no option %s", a.ID, w.Option)
|
||||
}
|
||||
if w.Level != o.Level {
|
||||
return Option{}, fmt.Errorf("the option %s is %s, and the warrant says %s", o.ID, o.Level, w.Level)
|
||||
}
|
||||
// A choice made after the ask expired is no answer to it, whatever the router said. An ask that says no
|
||||
// expiry, or a warrant that says no time, is no answer either: neither can be shown to be in time (the
|
||||
// confirmation review of 2026-10-09).
|
||||
if a.Expires.IsZero() {
|
||||
return Option{}, fmt.Errorf("the ask %s says no expiry, so no answer to it can be in time", a.ID)
|
||||
}
|
||||
if w.At.IsZero() {
|
||||
return Option{}, fmt.Errorf("the warrant for the ask %s says no time it was given, so it cannot be shown to be in time", a.ID)
|
||||
}
|
||||
if w.At.After(a.Expires) {
|
||||
return Option{}, fmt.Errorf("the warrant was given at %s, after the ask %s expired at %s",
|
||||
w.At.UTC().Format(time.RFC3339), a.ID, a.Expires.UTC().Format(time.RFC3339))
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Performs checks that the act an asker is about to perform is the one the chosen option bound when it
|
||||
// asked: the act's digest equals the option's Binds. An acknowledge option that bound nothing passes.
|
||||
func (o Option) Performs(act Act) error {
|
||||
if o.Binds == "" && o.Level == Acknowledge {
|
||||
return nil
|
||||
}
|
||||
d, err := ActDigest(act)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d != o.Binds {
|
||||
return fmt.Errorf("the option %s bound %s, and the act about to be performed is %s: nothing is done", o.ID, o.Binds, d)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Button is one answer a channel offers: its label and the router's one-time ticket for it.
|
||||
type Button struct {
|
||||
Label string `json:"label"`
|
||||
Ticket string `json:"ticket"`
|
||||
}
|
||||
|
||||
// Message is the work a channel takes: shown with buttons (Show), shown again in place (Edit), or said
|
||||
// (Send). Handle is the router's name for it, the same across a show and its edits; the channel keeps
|
||||
// which of its own messages that is. The words are the router's, shown as given.
|
||||
type Message struct {
|
||||
Handle string `json:"handle"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
Buttons []Button `json:"buttons,omitempty"`
|
||||
// Note is the message's secondary words, after its body and its buttons: who asked and when, what each
|
||||
// answer does, when it expires, and the ask's Details. A channel shows it smaller, collapsed or set apart
|
||||
// where it can, and after the body where it cannot (novox/hq issue 639). A channel that bounds a message's
|
||||
// length cuts the body first and the note only when the title and the note alone do not fit, saying each
|
||||
// cut (novox/hq ADR 0347 §5).
|
||||
Note string `json:"note,omitempty"`
|
||||
Urgent bool `json:"urgent,omitempty"`
|
||||
Silent bool `json:"silent,omitempty"`
|
||||
// Reply is the Choice or LinkAsked this answers, by its ID: the channel shows it where that was made.
|
||||
Reply string `json:"reply,omitempty"`
|
||||
// To is the account linked as the operator on this kind, for a channel that verifies its sender: where
|
||||
// the channel sends what is not a reply. The router's word, from its list; empty when none is linked.
|
||||
To string `json:"to,omitempty"`
|
||||
// Secret says the words carry something shown once (a link's code): the channel shows it and keeps no
|
||||
// copy of it — no state, no history of its own.
|
||||
Secret bool `json:"secret,omitempty"`
|
||||
}
|
||||
|
||||
// Sender is who a channel's service says sent something: the account, the name it shows, and whether the
|
||||
// service authenticated it. The router alone judges whether that is the operator.
|
||||
type Sender struct {
|
||||
Identity string `json:"identity"`
|
||||
Display string `json:"display,omitempty"`
|
||||
Authenticated bool `json:"authenticated"`
|
||||
}
|
||||
|
||||
// Failed is a message a channel could not deliver: its handle, why, and whether trying again could help.
|
||||
type Failed struct {
|
||||
Handle string `json:"handle"`
|
||||
Why string `json:"why"`
|
||||
Permanent bool `json:"permanent,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Standing is what a channel says of itself, at least every five minutes and whenever it changes:
|
||||
// whether it can send now, why not, and whether its edits notify nobody.
|
||||
type Standing struct {
|
||||
Ready bool `json:"ready"`
|
||||
Why string `json:"why,omitempty"`
|
||||
EditsSilently bool `json:"edits-silently,omitempty"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// What a channel says of its own delivery, on IntakeSubject.
|
||||
const (
|
||||
FailedWhat = "failed"
|
||||
StandingWhat = "standing"
|
||||
)
|
||||
|
||||
// Choice is a button chosen on a channel.
|
||||
type Choice struct {
|
||||
// ID is the channel's own for this arrival, unique, so the router acts on it once.
|
||||
ID string `json:"id"`
|
||||
Ticket string `json:"ticket"`
|
||||
Handle string `json:"handle,omitempty"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// LinkAsked is somebody on a channel asking to be linked as the operator.
|
||||
type LinkAsked struct {
|
||||
ID string `json:"id"`
|
||||
Sender Sender `json:"sender"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Code is a code a person typed on a channel, asked as a proof: never in an event, never kept.
|
||||
type Code struct {
|
||||
Sender Sender `json:"sender"`
|
||||
Code string `json:"code"`
|
||||
Purpose string `json:"purpose"`
|
||||
}
|
||||
|
||||
// ProofAnswer is the router's answer to a proof: whether it was taken, and words to say to the person.
|
||||
type ProofAnswer struct {
|
||||
Accepted bool `json:"accepted"`
|
||||
Words string `json:"words"`
|
||||
}
|
||||
@@ -0,0 +1,367 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var now = time.Date(2026, 10, 8, 14, 0, 0, 0, time.UTC)
|
||||
|
||||
func held() Ask {
|
||||
return Ask{ID: "d-1", Headline: "openrazer delivery held", Explanation: "Needs you: release it, or stop it.",
|
||||
Who: Operator, Expires: now.Add(time.Hour), OnExpiry: "the delivery stays held", About: "delivery.openrazer",
|
||||
Options: []Option{
|
||||
{ID: "release", Label: "Release", Does: "the delivery goes on", Level: Approve, Binds: bind("release", "openrazer")},
|
||||
{ID: "stop", Label: "Stop", Does: "the delivery ends", Level: Approve, Binds: bind("stop", "openrazer")},
|
||||
{ID: "silence", Label: "Silence for a week", Does: "nothing more is said for a week", Level: Acknowledge},
|
||||
}}
|
||||
}
|
||||
|
||||
func bind(verb, what string) string {
|
||||
d, err := ActDigest(Act{"verb": verb, "delivery": what})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func TestASoundAskPasses(t *testing.T) {
|
||||
if err := held().Check(now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskIsRefusedInWordsForEachFault(t *testing.T) {
|
||||
for want, change := range map[string]func(*Ask){
|
||||
"is not letters": func(a *Ask) { a.ID = "a.b" },
|
||||
"headline is empty": func(a *Ask) { a.Headline = "" },
|
||||
"explains nothing": func(a *Ask) { a.Explanation = " " },
|
||||
"only the operator answers": func(a *Ask) { a.Who = "agent" },
|
||||
"offers 0 options": func(a *Ask) { a.Options = nil },
|
||||
"offers 5 options": func(a *Ask) { a.Options = append(a.Options, a.Options[0], a.Options[1]) },
|
||||
"is offered twice": func(a *Ask) { a.Options[1].ID = "release" },
|
||||
"does not say what it does": func(a *Ask) { a.Options[0].Does = "" },
|
||||
"none of acknowledge": func(a *Ask) { a.Options[0].Level = "admin" },
|
||||
"expires before it is asked": func(a *Ask) { a.Expires = now },
|
||||
"approves lasts at most": func(a *Ask) { a.Expires = now.Add(25 * time.Hour) },
|
||||
"destroys lasts at most": func(a *Ask) { a.Options[0].Level = Destroy },
|
||||
"never defaults": func(a *Ask) { a.OnExpiry = "" },
|
||||
"is a key, without spaces": func(a *Ask) { a.About = "a b" },
|
||||
"does not bind what it": func(a *Ask) { a.Options[0].Binds = "" },
|
||||
} {
|
||||
a := held()
|
||||
change(&a)
|
||||
if err := a.Check(now); err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("want a refusal saying %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
ack := held()
|
||||
ack.Options = ack.Options[2:]
|
||||
ack.OnExpiry = ""
|
||||
ack.Expires = now.Add(7 * 24 * time.Hour)
|
||||
if err := ack.Check(now); err != nil {
|
||||
t.Errorf("an ask that only acknowledges needs no expiry words and may last a week: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAskerActsOnlyOnAWarrantForItsOwnAskAndOption(t *testing.T) {
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
|
||||
AskDigest: held().Digest(), At: now.Add(time.Minute)}
|
||||
if o, err := w.For("mesh-delivery", held()); err != nil || o.ID != "release" {
|
||||
t.Fatalf("a sound warrant was refused: %v", err)
|
||||
}
|
||||
if got := w.Says(); got != "the operator, via telegram (user id verified), chose Release" {
|
||||
t.Errorf("the warrant says %q", got)
|
||||
}
|
||||
for why, change := range map[string]func(*Warrant){
|
||||
"another asker": func(w *Warrant) { w.Asker = "mesh-controller" },
|
||||
"another ask": func(w *Warrant) { w.Ask = "d-2" },
|
||||
"no choice": func(w *Warrant) { w.Outcome = OutcomeExpired },
|
||||
"nobody": func(w *Warrant) { w.By = nil },
|
||||
"not offered": func(w *Warrant) { w.Option = "delete" },
|
||||
"a lower level": func(w *Warrant) { w.Level = Acknowledge },
|
||||
"another role": func(w *Warrant) { w.By = &Person{Who: "agent"} },
|
||||
"no digest": func(w *Warrant) { w.AskDigest = "" },
|
||||
"no time given": func(w *Warrant) { w.At = time.Time{} },
|
||||
"after expiry": func(w *Warrant) { w.At = now.Add(2 * time.Hour) },
|
||||
"another ask's digest": func(w *Warrant) {
|
||||
other := held()
|
||||
other.Options[0].Binds = bind("release", "another")
|
||||
w.AskDigest = other.Digest()
|
||||
},
|
||||
} {
|
||||
c := w
|
||||
by := *w.By
|
||||
c.By = &by
|
||||
change(&c)
|
||||
if _, err := c.For("mesh-delivery", held()); err == nil {
|
||||
t.Errorf("a warrant with %s was acted on", why)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSubjectsNameTheirPublisher(t *testing.T) {
|
||||
for got, want := range map[string]string{
|
||||
AskSubject("mesh-delivery"): "mesh.seat.operator-channel.accept.ask.mesh-delivery",
|
||||
DecidedSubject("mesh-delivery"): "mesh.seat.operator-channel.event.decided.mesh-delivery",
|
||||
ChannelSubject(Show, "telegram"): "mesh.seat.channel.accept.show.telegram",
|
||||
IntakeSubject(Chosen, "desktop"): "mesh.seat.intake.event.choice.desktop",
|
||||
ProofSubject(CodeProof, "telegram"): "mesh.seat.intake.proof.code.telegram",
|
||||
} {
|
||||
if got != want {
|
||||
t.Errorf("%s, want %s", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A warrant answers one ask as it was shown, and authorises the act its option bound and no other
|
||||
// (novox/hq ADR 0259 §6: the warrant is bound to the exact ask and its arguments).
|
||||
func TestAWarrantAuthorisesOnlyTheActItsOptionBound(t *testing.T) {
|
||||
a := held()
|
||||
// The digest survives the trip through the bus: what the router decodes and digests is what was sent.
|
||||
raw, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var decoded Ask
|
||||
if err := json.Unmarshal(raw, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Digest() != a.Digest() {
|
||||
t.Fatalf("the ask's digest changed on the way: %s, %s", decoded.Digest(), a.Digest())
|
||||
}
|
||||
local := a
|
||||
local.Expires = a.Expires.In(time.FixedZone("CEST", 2*3600))
|
||||
if local.Digest() != a.Digest() {
|
||||
t.Error("the same instant in another zone digests differently")
|
||||
}
|
||||
// Changing anything the person was shown, or what an option binds, changes the digest.
|
||||
for what, change := range map[string]func(*Ask){
|
||||
"the headline": func(a *Ask) { a.Headline += "!" },
|
||||
"the explanation": func(a *Ask) { a.Explanation += " " },
|
||||
"a label": func(a *Ask) { a.Options[0].Label = "Go" },
|
||||
"what it binds": func(a *Ask) { a.Options[0].Binds = bind("release", "another") },
|
||||
"the expiry": func(a *Ask) { a.Expires = a.Expires.Add(time.Second) },
|
||||
"what it is about": func(a *Ask) { a.About = "delivery.other" },
|
||||
} {
|
||||
c := held()
|
||||
change(&c)
|
||||
if c.Digest() == a.Digest() {
|
||||
t.Errorf("changing %s leaves the digest", what)
|
||||
}
|
||||
}
|
||||
o, _ := a.Option("release")
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "openrazer"}); err != nil {
|
||||
t.Errorf("the bound act was refused: %v", err)
|
||||
}
|
||||
if err := o.Performs(Act{"verb": "release", "delivery": "another"}); err == nil {
|
||||
t.Error("an act the option did not bind was let through")
|
||||
}
|
||||
ack, _ := a.Option("silence")
|
||||
if err := ack.Performs(Act{"anything": "x"}); err != nil {
|
||||
t.Errorf("an acknowledgement that bound nothing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The digests are over named fields in a canonical encoding (the review of 2026-10-09): not over Go's
|
||||
// encoding of a struct, so a field added to Ask or Option, a JSON tag renamed, or a map built in another
|
||||
// order changes nothing it should not — and every field that is shown or bound is in it.
|
||||
func TestDigestsAreCanonicalOverNamedFields(t *testing.T) {
|
||||
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
|
||||
b := Act{}
|
||||
b["arg.delivery"] = "hq"
|
||||
b["verb"] = "release"
|
||||
a2, _ := ActDigest(b)
|
||||
if a1 != a2 {
|
||||
t.Error("the order an act was built in changed its digest")
|
||||
}
|
||||
// A value cannot be read as two: "a"+"b=c" is not "a=b"+"c".
|
||||
x, _ := ActDigest(Act{"a": "b\n1:c"})
|
||||
y, _ := ActDigest(Act{"a": "b", "c": ""})
|
||||
if x == y {
|
||||
t.Error("two different acts encode the same")
|
||||
}
|
||||
if _, err := ActDigest(Act{}); err == nil {
|
||||
t.Error("an empty act was digested")
|
||||
}
|
||||
// The encoding, by hand: any language reproduces it.
|
||||
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
|
||||
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
|
||||
t.Errorf("the act's encoding is not the one written down: %s", got)
|
||||
}
|
||||
|
||||
// Every exported field of Ask and Option is in the digest: changing any one changes it.
|
||||
base := held()
|
||||
mutations := map[string]func(*Ask){
|
||||
"ID": func(a *Ask) { a.ID = "other" },
|
||||
"Headline": func(a *Ask) { a.Headline += "!" },
|
||||
"Explanation": func(a *Ask) { a.Explanation += "!" },
|
||||
"Who": func(a *Ask) { a.Who = "someone" },
|
||||
"Expires": func(a *Ask) { a.Expires = a.Expires.Add(time.Nanosecond) },
|
||||
"OnExpiry": func(a *Ask) { a.OnExpiry += "!" },
|
||||
"About": func(a *Ask) { a.About += "!" },
|
||||
"Urgent": func(a *Ask) { a.Urgent = !a.Urgent },
|
||||
"Options": func(a *Ask) { a.Options = a.Options[:1] },
|
||||
"Option.ID": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].ID = "x" },
|
||||
"Option.Label": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Label += "!"
|
||||
},
|
||||
"Option.Does": func(a *Ask) { a.Options = append([]Option(nil), a.Options...); a.Options[0].Does += "!" },
|
||||
"Option.Level": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Level = Destroy
|
||||
},
|
||||
"Option.Binds": func(a *Ask) {
|
||||
a.Options = append([]Option(nil), a.Options...)
|
||||
a.Options[0].Binds = "sha256:00"
|
||||
},
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
c := base
|
||||
mutate(&c)
|
||||
if c.Digest() == base.Digest() {
|
||||
t.Errorf("changing %s left the digest the same", name)
|
||||
}
|
||||
}
|
||||
// Whole and Details (issue 383) are in the digest once given, and change it; an ask without either digests
|
||||
// exactly as before they existed — the encoding by hand, so no build can drift from it unseen.
|
||||
plain := Ask{ID: "p", Headline: "h", Explanation: "e", Who: Operator, Expires: now, OnExpiry: "o", About: "a",
|
||||
Options: []Option{{ID: "ok", Label: "OK", Does: "d", Level: Acknowledge}}}
|
||||
byHand := sha256.Sum256([]byte("novox.ask.v1\n1:p\n1:h\n1:e\n8:operator\n20:2026-10-08T14:00:00Z\n1:o\n1:a\n5:false\n1:1\n" +
|
||||
"2:ok\n2:OK\n1:d\n11:acknowledge\n0:\n"))
|
||||
if plain.Digest() != "sha256:"+hex.EncodeToString(byHand[:]) {
|
||||
t.Errorf("an ask without Whole and Details does not digest as it did before them: %s", plain.Digest())
|
||||
}
|
||||
withWhole, withDetails := plain, plain
|
||||
withWhole.Whole = "e, whole"
|
||||
withDetails.Details = "fingerprint 1234"
|
||||
if withWhole.Digest() == plain.Digest() || withDetails.Digest() == plain.Digest() || withWhole.Digest() == withDetails.Digest() {
|
||||
t.Error("Whole or Details left the digest the same")
|
||||
}
|
||||
both := withWhole
|
||||
both.Details = "fingerprint 1234"
|
||||
changed := both
|
||||
changed.Whole += "!"
|
||||
if changed.Digest() == both.Digest() {
|
||||
t.Error("changing Whole beside Details left the digest the same")
|
||||
}
|
||||
// A field added to Ask or Option without being added to Digest fails here.
|
||||
if n := reflect.TypeOf(Ask{}).NumField(); n != 12 {
|
||||
t.Errorf("Ask has %d fields, and Digest covers 12: add the new one to Digest and to this test", n)
|
||||
}
|
||||
if n := reflect.TypeOf(Option{}).NumField(); n != 5 {
|
||||
t.Errorf("Option has %d fields, and Digest covers 5: add the new one to Digest and to this test", n)
|
||||
}
|
||||
// The same instant in another zone is the same ask.
|
||||
z := base
|
||||
z.Expires = z.Expires.In(time.FixedZone("x", 3600))
|
||||
if z.Digest() != base.Digest() {
|
||||
t.Error("the expiry's zone changed the digest")
|
||||
}
|
||||
}
|
||||
|
||||
// An ask that says no expiry is answered by no warrant (the confirmation review of 2026-10-09): nothing could
|
||||
// show the answer was in time.
|
||||
func TestAWarrantForAnAskWithNoExpiryIsRefused(t *testing.T) {
|
||||
a := held()
|
||||
a.Expires = time.Time{}
|
||||
w := Warrant{Ask: "d-1", Asker: "mesh-delivery", Outcome: OutcomeChosen, Option: "release", Label: "Release",
|
||||
Level: Approve, By: &Person{Who: Operator, Kind: "telegram", Identity: "42"}, AskDigest: a.Digest(),
|
||||
At: now.Add(time.Minute)}
|
||||
if _, err := w.For("mesh-delivery", a); err == nil {
|
||||
t.Error("a warrant for an ask with no expiry was taken")
|
||||
}
|
||||
}
|
||||
|
||||
// An explanation that reads at a glance passes; a wall of text says each way it is one (novox/hq issue 639).
|
||||
func TestAnExplanationReadsAtAGlance(t *testing.T) {
|
||||
if err := Glance("Your agents' memory follows you to every machine. A file changed on two keeps both.\n\n• Only the memory folders are carried\n• A conflict reaches you as a warning"); err != nil {
|
||||
t.Fatalf("a summary and two points: %v", err)
|
||||
}
|
||||
var wall strings.Builder
|
||||
wall.WriteString("ADR 0343 was proposed by fix-367 and approved by the knowledge base's own review.")
|
||||
for i := 1; i <= 13; i++ {
|
||||
wall.WriteString("\n" + strings.Repeat("A clause lead. ", 3))
|
||||
}
|
||||
err := Glance(wall.String())
|
||||
if err == nil || !strings.Contains(err.Error(), "13 points") {
|
||||
t.Fatalf("thirteen clause lines pass as a glance: %v", err)
|
||||
}
|
||||
if err := Glance(strings.Repeat("word ", 70)); err == nil || !strings.Contains(err.Error(), "summary is 349") {
|
||||
t.Fatalf("a long summary passes: %v", err)
|
||||
}
|
||||
if err := Glance("short.\n" + strings.Repeat("p", PointLength+1)); err == nil || !strings.Contains(err.Error(), "point 1") {
|
||||
t.Fatalf("a long point passes: %v", err)
|
||||
}
|
||||
if err := Glance(" \n"); err == nil {
|
||||
t.Fatal("an empty explanation passes")
|
||||
}
|
||||
}
|
||||
|
||||
// A message's note travels beside its body, and is left out of the encoding when empty, so a channel older
|
||||
// than it reads every message it read before.
|
||||
func TestAMessageCarriesItsNoteApart(t *testing.T) {
|
||||
raw, _ := json.Marshal(Message{Handle: "h", Title: "t", Body: "b", Note: "Asked by x."})
|
||||
if !strings.Contains(string(raw), `"note":"Asked by x."`) {
|
||||
t.Fatalf("the note is not encoded: %s", raw)
|
||||
}
|
||||
raw, _ = json.Marshal(Message{Handle: "h", Title: "t"})
|
||||
if strings.Contains(string(raw), "note") {
|
||||
t.Fatalf("an empty note is encoded: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// WholeDetails is digested when given, and an ask without it digests as before it existed: a router and an asker
|
||||
// of different builds still agree on every such ask.
|
||||
func TestWholeDetailsAreDigestedOnlyWhenGiven(t *testing.T) {
|
||||
a := Ask{ID: "a", Headline: "h", Explanation: "e", Details: "d"}
|
||||
before := a.Digest()
|
||||
a.WholeDetails = "+ shares: media=/storage/media"
|
||||
if a.Digest() == before {
|
||||
t.Fatal("the whole details are not in the digest")
|
||||
}
|
||||
b := a
|
||||
b.WholeDetails = "+ shares: media=/storage/other"
|
||||
if a.Digest() == b.Digest() {
|
||||
t.Fatal("two whole details digest the same")
|
||||
}
|
||||
a.WholeDetails = ""
|
||||
if a.Digest() != before {
|
||||
t.Fatal("an ask without whole details digests otherwise than before")
|
||||
}
|
||||
}
|
||||
|
||||
// No two asks encode the same (the review of mesh-sdk #25): whole details given beside nothing else are not read as
|
||||
// a Whole and Details that happen to spell the same values, nor as any other split of the three fields.
|
||||
func TestWholeDetailsCollideWithNoOtherAsk(t *testing.T) {
|
||||
base := Ask{ID: "a", Headline: "h", Explanation: "e"}
|
||||
shapes := []Ask{}
|
||||
for _, f := range []struct{ whole, details, wholeDetails string }{
|
||||
{"", "", "+ x: 1"},
|
||||
{"whole-details", "+ x: 1", ""},
|
||||
{"", "+ x: 1", ""},
|
||||
{"+ x: 1", "", ""},
|
||||
{"", "", "whole-details"},
|
||||
{"", "+ x: 1", "+ x: 1"},
|
||||
{"+ x: 1", "+ x: 1", ""},
|
||||
{"", "", ""},
|
||||
} {
|
||||
a := base
|
||||
a.Whole, a.Details, a.WholeDetails = f.whole, f.details, f.wholeDetails
|
||||
shapes = append(shapes, a)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
for i, a := range shapes {
|
||||
if j, ok := seen[a.Digest()]; ok {
|
||||
t.Errorf("asks %d and %d digest the same: %+v / %+v", j, i, shapes[j], a)
|
||||
}
|
||||
seen[a.Digest()] = i
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ConditionSubject is where a module says a condition to the operator: a by-caller accept of the operator-channel
|
||||
// seat, `mesh.seat.operator-channel.accept.condition.<publisher>` (novox/hq ADR 0437 §9). The router reads the
|
||||
// verb at the fifth part and the caller at the sixth, so the publisher is one token, the module's own name; it is
|
||||
// refused when empty or when it holds a dot, a wildcard or whitespace.
|
||||
func ConditionSubject(publisher string) (string, error) {
|
||||
if publisher == "" {
|
||||
return "", errors.New("asks: the condition's publisher is empty")
|
||||
}
|
||||
if strings.ContainsAny(publisher, ".*> \t\r\n") || strings.ContainsFunc(publisher, func(r rune) bool { return r <= ' ' }) {
|
||||
return "", fmt.Errorf("asks: the condition's publisher %q is not one token: it holds a dot, a wildcard or whitespace", publisher)
|
||||
}
|
||||
return "mesh.seat." + Seat + ".accept.condition." + publisher, nil
|
||||
}
|
||||
|
||||
// ConditionVersion is the version of the condition value this package writes and the only one it reads.
|
||||
const ConditionVersion = 1
|
||||
|
||||
// ErrUnknownConditionVersion is what DecodeCondition returns for a value whose version is missing or not
|
||||
// ConditionVersion. Test it with errors.Is.
|
||||
var ErrUnknownConditionVersion = errors.New("asks: unknown condition version")
|
||||
|
||||
// The words a condition's explanation opens with (novox/hq ADR 0253 §1).
|
||||
const (
|
||||
NothingToDo = "Nothing for you to do."
|
||||
NeedsYou = "Needs you: "
|
||||
)
|
||||
|
||||
// The severities and states of a condition.
|
||||
const (
|
||||
SeverityUrgent = "urgent"
|
||||
SeverityWarning = "warning"
|
||||
StateRaised = "raised"
|
||||
StateCleared = "cleared"
|
||||
)
|
||||
|
||||
// Scope is what a condition is about, under the controller's names.
|
||||
type Scope struct {
|
||||
Scope string `json:"scope"`
|
||||
ID string `json:"id"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
}
|
||||
|
||||
// Action is one answer a notification offers, under the controller's names.
|
||||
type Action struct {
|
||||
Label string `json:"label"`
|
||||
Verb string `json:"verb"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
Arguments map[string]string `json:"arguments,omitempty"`
|
||||
Level Level `json:"level,omitempty"`
|
||||
}
|
||||
|
||||
// Condition is a condition a module says to the operator, under the names of the controller's conditions: its key,
|
||||
// severity, subject, state (raised or cleared), summary and the plain words of ADR 0253. A cleared condition needs
|
||||
// only its key, subject and state. The router alone chooses the channels.
|
||||
type Condition struct {
|
||||
V int `json:"v"`
|
||||
Key string `json:"key"`
|
||||
Severity string `json:"severity,omitempty"`
|
||||
Subject Scope `json:"subject"`
|
||||
State string `json:"state"`
|
||||
Summary string `json:"summary,omitempty"`
|
||||
Headline string `json:"headline,omitempty"`
|
||||
Needs string `json:"needs,omitempty"`
|
||||
Explanation string `json:"explanation,omitempty"`
|
||||
Resolved string `json:"resolved,omitempty"`
|
||||
Actions []Action `json:"actions,omitempty"`
|
||||
}
|
||||
|
||||
func (c Condition) check() error {
|
||||
if c.Key == "" {
|
||||
return errors.New("asks: a condition has no key")
|
||||
}
|
||||
if c.Subject.Scope == "" || c.Subject.ID == "" {
|
||||
return errors.New("asks: a condition's subject needs its scope and id")
|
||||
}
|
||||
switch c.State {
|
||||
case StateCleared:
|
||||
return nil
|
||||
case StateRaised:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's state %q is not raised or cleared", c.State)
|
||||
}
|
||||
switch c.Severity {
|
||||
case SeverityUrgent, SeverityWarning:
|
||||
default:
|
||||
return fmt.Errorf("asks: the condition's severity %q is not urgent or warning", c.Severity)
|
||||
}
|
||||
if c.Headline == "" {
|
||||
return errors.New("asks: a raised condition has no headline")
|
||||
}
|
||||
if n := len([]rune(c.Headline)); n > HeadlineLength {
|
||||
return fmt.Errorf("asks: the condition's headline is %d characters, more than %d", n, HeadlineLength)
|
||||
}
|
||||
verdict := NothingToDo
|
||||
if c.Needs != "" {
|
||||
verdict = NeedsYou + c.Needs
|
||||
}
|
||||
if !strings.HasPrefix(c.Explanation, verdict) {
|
||||
return fmt.Errorf("asks: the condition's explanation must open with %q", verdict)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeCondition is c as JSON at ConditionVersion; it refuses what ADR 0253 and ADR 0437 §9 refuse.
|
||||
func EncodeCondition(c Condition) ([]byte, error) {
|
||||
if err := c.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
return json.Marshal(c)
|
||||
}
|
||||
|
||||
// DecodeCondition reads a condition; it refuses an unknown version (ErrUnknownConditionVersion) and what
|
||||
// EncodeCondition refuses.
|
||||
func DecodeCondition(data []byte) (Condition, error) {
|
||||
var probe struct {
|
||||
V *int `json:"v"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &probe); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition is not JSON: %w", err)
|
||||
}
|
||||
if probe.V == nil {
|
||||
return Condition{}, fmt.Errorf("%w: no version in the value, this reader reads %d", ErrUnknownConditionVersion, ConditionVersion)
|
||||
}
|
||||
if *probe.V != ConditionVersion {
|
||||
return Condition{}, fmt.Errorf("%w: the value is version %d, this reader reads %d", ErrUnknownConditionVersion, *probe.V, ConditionVersion)
|
||||
}
|
||||
var c Condition
|
||||
if err := json.Unmarshal(data, &c); err != nil {
|
||||
return Condition{}, fmt.Errorf("asks: the condition does not fit its shape: %w", err)
|
||||
}
|
||||
if err := c.check(); err != nil {
|
||||
return Condition{}, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// SameCondition says whether two conditions are the same value.
|
||||
func SameCondition(a, b Condition) bool { return reflect.DeepEqual(a, b) }
|
||||
@@ -0,0 +1,143 @@
|
||||
package asks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestConditionSubject(t *testing.T) { // 1202/AC1
|
||||
s, err := ConditionSubject("mesh-delivery-teller")
|
||||
if err != nil || s != "mesh.seat.operator-channel.accept.condition.mesh-delivery-teller" {
|
||||
t.Fatalf("subject = %q, %v", s, err)
|
||||
}
|
||||
// The router splits an accept subject in six parts and reads the verb at 4 and the caller at 5.
|
||||
parts := strings.Split(s, ".")
|
||||
if len(parts) != 6 || parts[4] != "condition" || parts[5] != "mesh-delivery-teller" {
|
||||
t.Errorf("subject %q does not keep the router's shape: %v", s, parts)
|
||||
}
|
||||
for _, bad := range []string{"", "a.b", "a*", "a>", "a b", "a\tb", "a\nb"} {
|
||||
if _, err := ConditionSubject(bad); err == nil {
|
||||
t.Errorf("publisher %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func raisedCondition() Condition {
|
||||
return Condition{
|
||||
Key: "delivery.stuck.mesh-mail", Severity: "warning",
|
||||
Subject: Scope{Scope: "module", ID: "mesh-mail", Machine: "ace"}, State: "raised",
|
||||
Summary: "plan 12 stopped", Headline: "Delivery of mesh-mail is stuck",
|
||||
Needs: "approve the merge.", Explanation: "Needs you: approve the merge. The plan waits.",
|
||||
Resolved: "The delivery of mesh-mail moved on.",
|
||||
Actions: []Action{{Label: "Approve", Verb: "mesh-controller.assign", Machine: "ace",
|
||||
Arguments: map[string]string{"why": ""}, Level: Approve}},
|
||||
}
|
||||
}
|
||||
|
||||
func clearedCondition() Condition {
|
||||
return Condition{Key: "delivery.stuck.mesh-mail", Subject: Scope{Scope: "module", ID: "mesh-mail"}, State: "cleared"}
|
||||
}
|
||||
|
||||
func TestConditionRoundTrip(t *testing.T) { // 1202/AC2
|
||||
for _, c := range []Condition{raisedCondition(), clearedCondition()} {
|
||||
data, err := EncodeCondition(c)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", c.State, err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if err := json.Unmarshal(data, &probe); err != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("%s: %s does not carry v:1", c.State, data)
|
||||
}
|
||||
c.V = ConditionVersion
|
||||
got, err := DecodeCondition(data)
|
||||
if err != nil || !SameCondition(got, c) {
|
||||
t.Errorf("%s round trip: %+v %v", c.State, got, err)
|
||||
}
|
||||
}
|
||||
data, _ := EncodeCondition(raisedCondition())
|
||||
var probe map[string]any
|
||||
_ = json.Unmarshal(data, &probe)
|
||||
for _, name := range []string{"key", "severity", "subject", "summary", "headline", "needs", "explanation", "resolved", "actions", "state"} {
|
||||
if _, ok := probe[name]; !ok {
|
||||
t.Errorf("the JSON has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
subject := probe["subject"].(map[string]any)
|
||||
for _, name := range []string{"scope", "id", "machine"} {
|
||||
if _, ok := subject[name]; !ok {
|
||||
t.Errorf("the subject has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
action := probe["actions"].([]any)[0].(map[string]any)
|
||||
for _, name := range []string{"label", "verb", "machine", "arguments", "level"} {
|
||||
if _, ok := action[name]; !ok {
|
||||
t.Errorf("the action has no %q: %s", name, data)
|
||||
}
|
||||
}
|
||||
for _, in := range []string{`{"key":"k"}`, `{"v":2,"key":"k"}`} {
|
||||
if _, err := DecodeCondition([]byte(in)); !errors.Is(err, ErrUnknownConditionVersion) {
|
||||
t.Errorf("decode of %s = %v, want ErrUnknownConditionVersion", in, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionVerdict(t *testing.T) { // 1202/AC3
|
||||
c := raisedCondition()
|
||||
c.Needs, c.Explanation = "", "The plan waits."
|
||||
_, err := EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Nothing for you to do.") {
|
||||
t.Errorf("empty needs without its verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
if _, err := EncodeCondition(c); err != nil {
|
||||
t.Errorf("empty needs with its verdict refused: %v", err)
|
||||
}
|
||||
c = raisedCondition()
|
||||
c.Explanation = "Nothing for you to do. The plan waits."
|
||||
_, err = EncodeCondition(c)
|
||||
if err == nil || !strings.Contains(err.Error(), "Needs you: approve the merge.") {
|
||||
t.Errorf("needs set with the wrong verdict: %v", err)
|
||||
}
|
||||
c.Explanation = "Needs you: something else. The plan waits."
|
||||
if _, err = EncodeCondition(c); err == nil {
|
||||
t.Error("needs set with other needs in the verdict accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestConditionRefused(t *testing.T) { // 1202/AC4
|
||||
mutate := func(f func(*Condition)) error {
|
||||
c := raisedCondition()
|
||||
f(&c)
|
||||
_, err := EncodeCondition(c)
|
||||
return err
|
||||
}
|
||||
for name, f := range map[string]func(*Condition){
|
||||
"severity": func(c *Condition) { c.Severity = "critical" },
|
||||
"state": func(c *Condition) { c.State = "open" },
|
||||
"key": func(c *Condition) { c.Key = "" },
|
||||
"headline": func(c *Condition) { c.Headline = "" },
|
||||
"long headline": func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength+1) },
|
||||
} {
|
||||
if err := mutate(f); err == nil {
|
||||
t.Errorf("a condition with a bad %s accepted", name)
|
||||
}
|
||||
}
|
||||
if err := mutate(func(c *Condition) { c.Headline = strings.Repeat("h", HeadlineLength) }); err != nil {
|
||||
t.Errorf("a headline of the greatest length refused: %v", err)
|
||||
}
|
||||
if _, err := EncodeCondition(clearedCondition()); err != nil {
|
||||
t.Errorf("a cleared condition with only key, subject and state refused: %v", err)
|
||||
}
|
||||
c := clearedCondition()
|
||||
c.Key = ""
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a cleared condition without its key accepted")
|
||||
}
|
||||
c = clearedCondition()
|
||||
c.State = "gone"
|
||||
if _, err := EncodeCondition(c); err == nil {
|
||||
t.Error("a condition with an unknown state accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"regexp"
|
||||
"sync"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// Unproven is the caller of a call the runtime named no caller for: one that arrived on the subject that names
|
||||
// none, or from a runtime that names none (novox/hq issue 365). Said as a word, never left empty, so a record
|
||||
// shows it was not proven.
|
||||
const Unproven = "unproven"
|
||||
|
||||
// MetaCaller is where in a `tools/call`'s `_meta` the runtime names the caller: the bus user that published the
|
||||
// call, which the runtime takes from the subject the call arrived on — that subject's last token, which the
|
||||
// bus lets each user publish with its own name alone (novox/hq issue 365, by ADR 0259 §3's precedent). The
|
||||
// runtime writes `_meta`; a caller writes only `arguments`, so no caller can name another.
|
||||
const MetaCaller = "mesh/caller"
|
||||
|
||||
// busUser is the shape of a bus user's name: `<node>.<module>`, `person.<name>`, `controller` — names whose
|
||||
// parts are safe in a subject. Anything else the runtime named is not a caller and reads as Unproven.
|
||||
var busUser = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
|
||||
|
||||
// calls holds the caller of every call while its tool runs, by the arguments map that call was handed.
|
||||
var calls sync.Map // unsafe.Pointer → string
|
||||
|
||||
func argsKey(args map[string]any) unsafe.Pointer {
|
||||
return reflect.ValueOf(args).UnsafePointer()
|
||||
}
|
||||
|
||||
// CallerOf is who called the tool that was handed args, as the runtime proved it: the bus user that
|
||||
// published the call, or Unproven. Read from the runtime, never from the arguments' contents — a `caller`
|
||||
// argument names nobody. Valid while the tool runs; arguments no call was handed have no caller.
|
||||
func CallerOf(args map[string]any) string {
|
||||
if args == nil {
|
||||
return Unproven
|
||||
}
|
||||
if c, ok := calls.Load(argsKey(args)); ok {
|
||||
return c.(string)
|
||||
}
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// callerIn is the caller a `tools/call`'s `_meta` names, or Unproven.
|
||||
func callerIn(meta map[string]any) string {
|
||||
if s, ok := meta[MetaCaller].(string); ok && busUser.MatchString(s) {
|
||||
return s
|
||||
}
|
||||
return Unproven
|
||||
}
|
||||
|
||||
// runAs runs a tool with the caller held for its arguments for as long as it runs.
|
||||
func runAs(t Tool, caller string, args map[string]any) (any, error) {
|
||||
key := argsKey(args)
|
||||
calls.Store(key, caller)
|
||||
defer calls.Delete(key)
|
||||
return t.Run(args)
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// callWith serves one tool that answers the caller CallerOf reads from its arguments, sends it one tools/call
|
||||
// with the params given, and returns the caller it answered.
|
||||
func callWith(t *testing.T, params map[string]any) string {
|
||||
t.Helper()
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("tracker", []Tool{{Name: "who", Run: func(args map[string]any) (any, error) {
|
||||
return map[string]any{"caller": CallerOf(args)}, nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
defer inW.Close()
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": params})
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
lines := bufio.NewScanner(outR)
|
||||
done := make(chan string, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m struct {
|
||||
Result struct {
|
||||
Content []struct {
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
} `json:"result"`
|
||||
}
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
var answer struct {
|
||||
Caller string `json:"caller"`
|
||||
}
|
||||
if len(m.Result.Content) > 0 {
|
||||
_ = json.Unmarshal([]byte(m.Result.Content[0].Text), &answer)
|
||||
}
|
||||
done <- answer.Caller
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case got := <-done:
|
||||
return got
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle did not answer the call")
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// **A tool reads its caller from the runtime, never from the call's arguments** (novox/hq issue 365): the
|
||||
// runtime names the caller it took from the subject the call arrived on in the call's `_meta`, and CallerOf
|
||||
// hands it to the tool by the arguments the tool was given.
|
||||
func TestAToolReadsTheCallerTheRuntimeNamed(t *testing.T) {
|
||||
got := callWith(t, map[string]any{"name": "who", "arguments": map[string]any{"title": "x"},
|
||||
"_meta": map[string]any{MetaCaller: "shanks.node-tools"}})
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("the tool read caller %q, want the one the runtime named", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A caller named in the arguments is not the caller: an argument is the caller's to write.
|
||||
func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
|
||||
got := callWith(t, map[string]any{"name": "who", "arguments": map[string]any{"caller": "novox.node-tools",
|
||||
"_meta": map[string]any{MetaCaller: "novox.node-tools"}, MetaCaller: "novox.node-tools"}})
|
||||
if got != Unproven {
|
||||
t.Fatalf("a caller the arguments claimed was read as %q, want %q", got, Unproven)
|
||||
}
|
||||
got = callWith(t, map[string]any{"name": "who", "arguments": map[string]any{"caller": "novox.node-tools"},
|
||||
"_meta": map[string]any{MetaCaller: "shanks.node-tools"}})
|
||||
if got != "shanks.node-tools" {
|
||||
t.Fatalf("with a caller in the arguments the tool read %q, want the runtime's", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A runtime that names no caller — one from before issue 365, or a call on the subject that names none — is
|
||||
// said as Unproven, explicitly; a caller that is not a bus user's name is Unproven too.
|
||||
func TestACallTheRuntimeNamedNoCallerForIsUnproven(t *testing.T) {
|
||||
for _, params := range []map[string]any{
|
||||
{"name": "who", "arguments": map[string]any{}},
|
||||
{"name": "who"},
|
||||
{"name": "who", "arguments": map[string]any{}, "_meta": map[string]any{}},
|
||||
{"name": "who", "arguments": map[string]any{}, "_meta": map[string]any{MetaCaller: ""}},
|
||||
{"name": "who", "arguments": map[string]any{}, "_meta": map[string]any{MetaCaller: "a b"}},
|
||||
{"name": "who", "arguments": map[string]any{}, "_meta": map[string]any{MetaCaller: 7}},
|
||||
} {
|
||||
if got := callWith(t, params); got != Unproven {
|
||||
t.Errorf("%v: the tool read caller %q, want %q", params, got, Unproven)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Arguments the runtime never handed a tool have no caller.
|
||||
func TestArgumentsOfNoCallHaveNoCaller(t *testing.T) {
|
||||
if got := CallerOf(map[string]any{"x": 1}); got != Unproven {
|
||||
t.Fatalf("CallerOf of a map no call was given answered %q", got)
|
||||
}
|
||||
if got := CallerOf(nil); got != Unproven {
|
||||
t.Fatalf("CallerOf(nil) answered %q", got)
|
||||
}
|
||||
}
|
||||
+44
-7
@@ -24,9 +24,13 @@ import (
|
||||
// address (ADR 0272 §3).
|
||||
const Socket = "/run/nox-mesh-host/cli.sock"
|
||||
|
||||
// MaxRequest bounds what the node-engine reads of a request. A command line is a few hundred bytes; settings given
|
||||
// inline are a few kilobytes.
|
||||
const MaxRequest = 64 << 10
|
||||
// MaxStdin bounds the standard input mesh-cli carries for a line: a secret, a key, a settings file — kilobytes. More
|
||||
// is refused by mesh-cli before anything is asked, and by the node-engine and the controller again.
|
||||
const MaxStdin = 64 << 10
|
||||
|
||||
// MaxRequest bounds what the node-engine reads of a request: a command line of a few hundred bytes, settings given
|
||||
// inline of a few kilobytes, and standard input of at most MaxStdin, which JSON writes as base64 (4 bytes for every 3).
|
||||
const MaxRequest = 64<<10 + MaxStdin*4/3 + 4
|
||||
|
||||
// MaxAnswer bounds what mesh-cli reads of an answer. The controller cuts what it runs to fit one bus message, and
|
||||
// says so; this is the local bound above that.
|
||||
@@ -36,9 +40,16 @@ const MaxAnswer = 16 << 20
|
||||
// and a minute for the link on either side.
|
||||
const AskWithin = 6 * time.Minute
|
||||
|
||||
// Request is what mesh-cli asks: the controller's command line, as words.
|
||||
// Request is what mesh-cli asks: the controller's command line, as words, and what mesh-cli's own standard input
|
||||
// held when it was not a terminal.
|
||||
//
|
||||
// **Stdin is for the controller's terminal alone** (novox/hq ADR 0259 §10, ADR 0272): a secret given with
|
||||
// `secret accept … --from -`, so it never stands on a command line. The controller gives it to a line that runs as
|
||||
// its terminal and refuses any other line that carries it; nobody keeps it — not the node-engine, not the
|
||||
// controller's record of calls, not its journal.
|
||||
type Request struct {
|
||||
Line []string `json:"line"`
|
||||
Line []string `json:"line"`
|
||||
Stdin []byte `json:"stdin,omitempty"`
|
||||
}
|
||||
|
||||
// Answer is what comes back: what the command printed, how it exited, and whether it ran as the controller's
|
||||
@@ -87,6 +98,9 @@ func ReadRequest(r io.Reader) (Request, error) {
|
||||
if len(req.Line) == 0 {
|
||||
return Request{}, errors.New("the request names no command")
|
||||
}
|
||||
if len(req.Stdin) > MaxStdin {
|
||||
return Request{}, fmt.Errorf("%w: standard input of more than %d bytes", ErrTooLong, MaxStdin)
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
@@ -102,9 +116,18 @@ func ReadAnswer(r io.Reader) (Answer, error) {
|
||||
|
||||
// Ask is mesh-cli's side: connect to the socket, ask, and wait for the answer within AskWithin or ctx.
|
||||
func Ask(ctx context.Context, socket string, line []string) (Answer, error) {
|
||||
return AskWith(ctx, socket, Request{Line: line})
|
||||
}
|
||||
|
||||
// AskWith asks a whole request: a line, and the standard input it carries, at most MaxStdin.
|
||||
func AskWith(ctx context.Context, socket string, req Request) (Answer, error) {
|
||||
line := req.Line
|
||||
if len(line) == 0 {
|
||||
return Answer{}, errors.New("no command given")
|
||||
}
|
||||
if len(req.Stdin) > MaxStdin {
|
||||
return Answer{}, fmt.Errorf("standard input is more than %d bytes, which mesh-cli does not carry; nothing was asked", MaxStdin)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, AskWithin)
|
||||
defer cancel()
|
||||
var d net.Dialer
|
||||
@@ -116,8 +139,22 @@ func Ask(ctx context.Context, socket string, line []string) (Answer, error) {
|
||||
if deadline, ok := ctx.Deadline(); ok {
|
||||
_ = conn.SetDeadline(deadline)
|
||||
}
|
||||
if err := WriteRequest(conn, Request{Line: line}); err != nil {
|
||||
return Answer{}, fmt.Errorf("asking the node-engine: %w", err)
|
||||
if err := WriteRequest(conn, req); err != nil {
|
||||
// The engine may have answered before it read the request — a refusal (busy, an unknown account) written
|
||||
// and the connection closed — and the write then fails on the close. That answer is already waiting, and
|
||||
// it says why where the write error says only that the pipe broke (novox/hq issue 444, found under issue
|
||||
// 434). So read it, within the deadline the connection already has, and only when none came say the write.
|
||||
// Only a refusal is taken: the request was never wholly written, so nothing can have run, and any other
|
||||
// answer here is not one a correct engine gives — passing it on as success would hide the failure.
|
||||
a, rerr := ReadAnswer(conn)
|
||||
if rerr != nil {
|
||||
return Answer{}, fmt.Errorf("asking the node-engine: %w; and no answer came (%v)", err, rerr)
|
||||
}
|
||||
if a.Refused == "" {
|
||||
return Answer{}, fmt.Errorf("asking the node-engine: %w; and the answer that came is not a refusal, "+
|
||||
"though the request was never wholly written", err)
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
a, err := ReadAnswer(conn)
|
||||
if err != nil {
|
||||
|
||||
@@ -3,6 +3,7 @@ package cli
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net"
|
||||
"path/filepath"
|
||||
@@ -44,6 +45,17 @@ func TestARequestAndAnAnswerSurviveTheirRoundTrip(t *testing.T) {
|
||||
// A request over the bound is refused, not read whole; one naming no command is refused.
|
||||
func TestARequestOverTheBoundOrNamingNothingIsRefused(t *testing.T) {
|
||||
long := Request{Line: []string{strings.Repeat("x", MaxRequest)}}
|
||||
big := Request{Line: []string{"secret"}, Stdin: bytes.Repeat([]byte("s"), MaxStdin+1)}
|
||||
var bb bytes.Buffer
|
||||
if err := WriteRequest(&bb, big); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := ReadRequest(&bb); !errors.Is(err, ErrTooLong) {
|
||||
t.Fatalf("standard input over its bound was read: %v", err)
|
||||
}
|
||||
if _, err := AskWith(context.Background(), "/nonexistent.sock", big); err == nil || !strings.Contains(err.Error(), "nothing was asked") {
|
||||
t.Fatalf("mesh-cli would ask with standard input over its bound: %v", err)
|
||||
}
|
||||
var b bytes.Buffer
|
||||
if err := WriteRequest(&b, long); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -103,3 +115,27 @@ func TestAnUnknownOutcomeIsNotARefusal(t *testing.T) {
|
||||
t.Fatalf("an unknown outcome with no call names one: %+v", b)
|
||||
}
|
||||
}
|
||||
|
||||
// Standard input up to its bound travels whole, beside the line: a secret given with --from -.
|
||||
func TestStandardInputTravelsWithItsLine(t *testing.T) {
|
||||
given := append(bytes.Repeat([]byte{0, 0xff, '\n', 'x'}, MaxStdin/4-1), "end"...)
|
||||
var b bytes.Buffer
|
||||
if err := WriteRequest(&b, Request{Line: []string{"secret", "accept", "anchor", "telegram", "telegram-token", "--from", "-"},
|
||||
Stdin: given}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b.Len() > MaxRequest {
|
||||
t.Fatalf("a request with standard input at its bound is %d bytes, over MaxRequest %d", b.Len(), MaxRequest)
|
||||
}
|
||||
got, err := ReadRequest(&b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !bytes.Equal(got.Stdin, given) || got.Line[1] != "accept" {
|
||||
t.Fatalf("standard input did not travel whole: %d bytes, want %d", len(got.Stdin), len(given))
|
||||
}
|
||||
none, _ := json.Marshal(Request{Line: []string{"status"}})
|
||||
if strings.Contains(string(none), "stdin") {
|
||||
t.Fatalf("a request without standard input says it has one: %s", none)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// anEngineThatClosesFirst listens like the node-engine before mesh-host#80: it writes its answer, if any, and
|
||||
// closes without reading the request (novox/hq issue 434). It answers one connection.
|
||||
func anEngineThatClosesFirst(t *testing.T, answer *Answer) string {
|
||||
t.Helper()
|
||||
socket := filepath.Join(t.TempDir(), "cli.sock")
|
||||
l, err := net.Listen("unix", socket)
|
||||
if err != nil {
|
||||
t.Skipf("no unix socket here: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { l.Close() })
|
||||
go func() {
|
||||
conn, err := l.Accept()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if answer != nil {
|
||||
_ = WriteAnswer(conn, *answer)
|
||||
}
|
||||
conn.Close()
|
||||
}()
|
||||
return socket
|
||||
}
|
||||
|
||||
// aLineTheSocketCannotHold is a request larger than a unix socket buffers, so its write cannot finish before
|
||||
// the engine reads it — and an engine that closes first makes it fail every time, not only when it loses a race.
|
||||
func aLineTheSocketCannotHold() []string { return []string{"status", strings.Repeat("x", 4<<20)} }
|
||||
|
||||
// An engine that refuses and closes before reading the request is heard: its refusal reaches the caller, not
|
||||
// the broken pipe its close left the client's write (novox/hq issue 444).
|
||||
func TestARefusalWrittenBeforeTheRequestIsReadReachesTheCaller(t *testing.T) {
|
||||
busy := Refusal("the node-engine is busy: too many lines at once")
|
||||
socket := anEngineThatClosesFirst(t, &busy)
|
||||
a, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
|
||||
if err != nil {
|
||||
t.Fatalf("the engine said %q and the caller heard: %v", busy.Refused, err)
|
||||
}
|
||||
if a.Refused != busy.Refused || a.Exit != 1 {
|
||||
t.Fatalf("the engine said %q and the caller heard %+v", busy.Refused, a)
|
||||
}
|
||||
}
|
||||
|
||||
// When the write fails and no answer is waiting, the error says both: what failed, and that no answer came.
|
||||
func TestAFailedWriteWithNoAnswerSaysBoth(t *testing.T) {
|
||||
socket := anEngineThatClosesFirst(t, nil)
|
||||
start := time.Now()
|
||||
_, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
|
||||
if err == nil {
|
||||
t.Fatal("a request the engine never read, and never answered, came back without an error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "asking the node-engine") || !strings.Contains(err.Error(), "no answer came") {
|
||||
t.Fatalf("the error does not say what failed and that no answer came: %v", err)
|
||||
}
|
||||
if waited := time.Since(start); waited > 30*time.Second {
|
||||
t.Fatalf("waited %s on a connection the engine had closed", waited)
|
||||
}
|
||||
}
|
||||
|
||||
// After a failed write only a refusal is taken: nothing can have run, so an answer saying it ran and exited 0
|
||||
// is an error, never a success (novox/hq issue 444).
|
||||
func TestAnAnswerOtherThanARefusalAfterAFailedWriteIsAnError(t *testing.T) {
|
||||
ran := Answer{Exit: 0, Stdout: []byte("ok")}
|
||||
socket := anEngineThatClosesFirst(t, &ran)
|
||||
a, err := Ask(context.Background(), socket, aLineTheSocketCannotHold())
|
||||
if err == nil {
|
||||
t.Fatalf("a request never wholly written came back as success: %+v", a)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "not a refusal") {
|
||||
t.Fatalf("the error does not say the answer was not a refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Reading after a failed write never waits longer than the ask already allowed: a peer that never reads and
|
||||
// never answers is given up on at the context's deadline, not later (novox/hq issue 444).
|
||||
func TestAFailedWriteWaitsNoLongerThanTheDeadline(t *testing.T) {
|
||||
socket := filepath.Join(t.TempDir(), "cli.sock")
|
||||
l, err := net.Listen("unix", socket)
|
||||
if err != nil {
|
||||
t.Skipf("no unix socket here: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { l.Close() })
|
||||
held := make(chan net.Conn, 1)
|
||||
go func() {
|
||||
conn, err := l.Accept()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
held <- conn // neither read nor answered, nor closed, until the test ends
|
||||
}()
|
||||
t.Cleanup(func() {
|
||||
select {
|
||||
case c := <-held:
|
||||
c.Close()
|
||||
default:
|
||||
}
|
||||
})
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
|
||||
defer cancel()
|
||||
start := time.Now()
|
||||
_, err = Ask(ctx, socket, aLineTheSocketCannotHold())
|
||||
waited := time.Since(start)
|
||||
if err == nil {
|
||||
t.Fatal("a peer that never read nor answered gave no error")
|
||||
}
|
||||
if !errors.Is(err, os.ErrDeadlineExceeded) {
|
||||
t.Fatalf("the error is not the deadline: %v", err)
|
||||
}
|
||||
if waited > 3*time.Second {
|
||||
t.Fatalf("waited %s with a deadline of 1s", waited)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package delivery
|
||||
|
||||
import "time"
|
||||
|
||||
// BuildRecord is the layout mesh-controller internal/chain/records.go Build writes (line 91), the fields a reader outside
|
||||
// the controller reads (novox/hq ADR 0437 §4 §5, issue 1223). The controller's record carries more; a decode ignores
|
||||
// what is not here. Path is the module's directory in its repository, as the controller writes it (Build.Path, records.go:100,
|
||||
// set from the build request trimmed of slashes at internal/inventory/chain_requests.go:19); empty for a module whose
|
||||
// manifest is the repository's root. SourceInputs has no json keys of its own: it is written under its Go field names, as the controller
|
||||
// writes it.
|
||||
type BuildRecord struct {
|
||||
ID string `json:"id"`
|
||||
State string `json:"state,omitempty"`
|
||||
Repository string `json:"repository"`
|
||||
Ref string `json:"ref,omitempty"`
|
||||
Module string `json:"module,omitempty"`
|
||||
Path string `json:"path,omitempty"`
|
||||
Commit string `json:"commit,omitempty"`
|
||||
At time.Time `json:"at,omitzero"`
|
||||
SourceFingerprint string `json:"source-fingerprint,omitempty"`
|
||||
SourceInputs *SourceInputs `json:"source-inputs,omitempty"`
|
||||
}
|
||||
|
||||
// BuildsBucket is the bucket the builds are kept in: mesh-controller internal/broker/controller_buckets.go:101,
|
||||
// BucketName(ControllerSeat, ChainBuilds), with ControllerSeat "mesh-controller" (internal/broker/streams.go:359) and
|
||||
// ChainBuilds "builds" (controller_buckets.go:113).
|
||||
const BuildsBucket = "mesh-controller_builds"
|
||||
|
||||
// BuildKey is a build's record under its module, a build that knew none under the request's key: copied from
|
||||
// mesh-controller internal/chain/keys.go:35-40 (BuildKey, with RequestKey at line 42).
|
||||
func BuildKey(module, id string) string {
|
||||
if module == "" {
|
||||
return "req." + id
|
||||
}
|
||||
return module + ".build." + id
|
||||
}
|
||||
|
||||
// NewestKey is the key of a module's newest build, of any state: copied from mesh-controller internal/chain/keys.go:44.
|
||||
// The key holds no record: its value is the newest build's id as raw bytes (internal/inventory/chain_builds.go:210 and
|
||||
// :546), and the record lives at BuildKey(module, id). The pointer is a hint, checked against that record.
|
||||
func NewestKey(module string) string { return module + ".newest" }
|
||||
@@ -0,0 +1,123 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The fixture's path is what the controller writes for a build request: strings.Trim(path, "/") at mesh-controller
|
||||
// internal/inventory/chain_requests.go:19, the module's directory in its repository (Build.Path, internal/chain/records.go:100).
|
||||
// The fixture is a record the controller's own encoder wrote (json.Marshal of a chain.Build, mesh-controller
|
||||
// internal/chain, a throwaway test run once and deleted) with build-agent's build-1792395447088936520 values, plus a
|
||||
// manifest-digest and one made artifact, which a BuildRecord does not carry.
|
||||
func loadBuildFixture(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("testdata/build-record-build-agent.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
func TestBuildRecordDecodes(t *testing.T) { // 1223/AC1
|
||||
var r BuildRecord
|
||||
if err := json.Unmarshal(loadBuildFixture(t), &r); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range []struct{ name, got, want string }{
|
||||
{"id", r.ID, "build-1792395447088936520"},
|
||||
{"module", r.Module, "build-agent"},
|
||||
{"repository", r.Repository, "novox/mesh-controller"},
|
||||
{"ref", r.Ref, "main"},
|
||||
{"commit", r.Commit, "7a74cee83f88e6585a5511c3cc53bdf0f5303d20"},
|
||||
{"state", r.State, "built"},
|
||||
{"source-fingerprint", r.SourceFingerprint, "src1:e846efe73716b78e80860baaac96153de2cb76551325753838ca245d2160a3d7"},
|
||||
} {
|
||||
if c.got != c.want {
|
||||
t.Errorf("%s = %q, want %q", c.name, c.got, c.want)
|
||||
}
|
||||
}
|
||||
if want := time.Date(2026, 10, 19, 9, 38, 0, 0, time.UTC); !r.At.Equal(want) {
|
||||
t.Errorf("at = %v, want %v", r.At, want)
|
||||
}
|
||||
if r.SourceInputs == nil {
|
||||
t.Fatal("source-inputs missing")
|
||||
}
|
||||
fp, none := SourceFingerprint(*r.SourceInputs)
|
||||
if none != "" || fp != r.SourceFingerprint {
|
||||
t.Errorf("SourceFingerprint of the decoded inputs = %q (none %q), the record says %q", fp, none, r.SourceFingerprint)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRecordPath(t *testing.T) { // 1233/AC1
|
||||
var r BuildRecord
|
||||
if err := json.Unmarshal(loadBuildFixture(t), &r); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.Path != "modules/build-agent" {
|
||||
t.Errorf("path = %q, want modules/build-agent", r.Path)
|
||||
}
|
||||
// mesh-controller's manifest is its repository's root: the controller writes no path (omitempty), so the path is "".
|
||||
var m map[string]json.RawMessage
|
||||
if err := json.Unmarshal(loadBuildFixture(t), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
delete(m, "path")
|
||||
raw, _ := json.Marshal(m)
|
||||
var root BuildRecord
|
||||
if err := json.Unmarshal(raw, &root); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if root.Path != "" {
|
||||
t.Errorf("path of a root module = %q, want empty", root.Path)
|
||||
}
|
||||
// Every field 1223/AC1 checks is still equal in both.
|
||||
if root.ID != r.ID || root.Module != r.Module || root.Repository != r.Repository || root.Ref != r.Ref ||
|
||||
root.Commit != r.Commit || root.State != r.State || root.SourceFingerprint != r.SourceFingerprint || !root.At.Equal(r.At) {
|
||||
t.Errorf("fields differ: %+v vs %+v", root, r)
|
||||
}
|
||||
// And the path survives a round trip through the SDK's own type.
|
||||
back, _ := json.Marshal(r)
|
||||
var again BuildRecord
|
||||
if err := json.Unmarshal(back, &again); err != nil || again.Path != r.Path {
|
||||
t.Errorf("round trip path = %q (%v)", again.Path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRecordWithoutInputs(t *testing.T) { // 1223/AC2
|
||||
var m map[string]json.RawMessage
|
||||
if err := json.Unmarshal(loadBuildFixture(t), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
delete(m, "source-inputs")
|
||||
raw, _ := json.Marshal(m)
|
||||
var r BuildRecord
|
||||
if err := json.Unmarshal(raw, &r); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r.SourceInputs != nil {
|
||||
t.Errorf("source-inputs = %+v, want nil", r.SourceInputs)
|
||||
}
|
||||
if r.Module != "build-agent" {
|
||||
t.Errorf("module = %q, want build-agent", r.Module)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildKeys(t *testing.T) { // 1223/AC3
|
||||
// Literal values of mesh-controller internal/broker/controller_buckets.go:101 (BucketName(ControllerSeat, ChainBuilds)
|
||||
// is "mesh-controller_builds") and internal/chain/keys.go:35-44.
|
||||
if BuildsBucket != "mesh-controller_builds" {
|
||||
t.Errorf("BuildsBucket = %q", BuildsBucket)
|
||||
}
|
||||
if got := BuildKey("build-agent", "build-1"); got != "build-agent.build.build-1" {
|
||||
t.Errorf("BuildKey = %q", got)
|
||||
}
|
||||
if got := BuildKey("", "build-1"); got != "req.build-1" {
|
||||
t.Errorf("BuildKey without a module = %q", got)
|
||||
}
|
||||
if got := NewestKey("build-agent"); got != "build-agent.newest" {
|
||||
t.Errorf("NewestKey = %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,390 @@
|
||||
// Package delivery is the one home of the keys and values the parts of delivery read and write on the bus
|
||||
// (novox/hq ADR 0429 §2: the key layouts another repository reads live in mesh-sdk; research 063 section 01). The
|
||||
// node-engine writes the have keys, the watchman of the repository mesh-delivery writes presence and pulse, the
|
||||
// view reads all of them, and the controller grants them; every one imports this package, and nothing else defines
|
||||
// these names.
|
||||
//
|
||||
// **Buckets** are named `<owner module>_<name>` as the broker names a module's state (provider.BucketOf): the
|
||||
// node-engine's `have`, and the watchman's `presence` and `pulse`.
|
||||
//
|
||||
// **Keys.** A have key is `<node>.<module>`, and a node's own facts are `<node>.node`, so a module may not be named
|
||||
// `node`. A presence key is `<node>` for a node and `part.<part>.<instance>` for a part's instance, so a node may
|
||||
// not be named `part`. A pulse key is `<bucket>.<key>`: the full name of the bucket watched, then that bucket's key,
|
||||
// which may hold dots, so it splits at the first dot. A name is refused when it is empty or holds a dot, a wildcard,
|
||||
// whitespace or a character a bus key refuses, and the error names it.
|
||||
//
|
||||
// **Values** are JSON of under about 500 bytes, each carrying its version in `v`. A reader refuses a value whose
|
||||
// version is missing or one it does not know, with an error that is ErrUnknownVersion (research 063 section 05
|
||||
// question 9 item 7): a writer writes a new version only once every reader's have shows a version that reads it. A
|
||||
// writer writes only when the new value differs from the one before (the Same functions).
|
||||
//
|
||||
// **Source, candidates, modules and the forge** (issue 1200; ADR 0437 §5, release 3): the gate's bucket `source` with
|
||||
// its one key `main` (every repository's merged head and the group that moved it), the indexer's `candidates.<group>`
|
||||
// and `modules.<module>` (a module's source fingerprint, or none and why), and the forge bridge's messages named
|
||||
// `forge.<owner>.<name>.<kind>` (source.go).
|
||||
//
|
||||
// **Merges, asks, owed and answers** (issue 1201; ADR 0437 §6, §8): the gate's `merges.<group>`, its asks
|
||||
// `merge.<group>.<heads hash>` (the router's ask whole), its owed keys `<repository>.<number>.<head>.<act>` that the
|
||||
// forge bridge performs, and the teller's answer under the ask's own key (gate.go).
|
||||
//
|
||||
// **The source fingerprint** (issue 1203; ADR 0437 §5): one function of named inputs that the controller's builder and
|
||||
// the indexer both call, so they agree byte for byte (fingerprint.go).
|
||||
//
|
||||
// Standard library only, and this module's own asks package.
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The buckets of delivery's parts. Each is written by one owner: every node's node-engine writes the have keys of
|
||||
// its own node only, and the watchman writes presence and pulse.
|
||||
const (
|
||||
HaveBucket = "node-engine_have"
|
||||
PresenceBucket = WatchmanModule + "_presence"
|
||||
PulseBucket = WatchmanModule + "_pulse"
|
||||
)
|
||||
|
||||
// HaveHistory and PresenceHistory are the revisions the have and presence buckets keep per key (research 063
|
||||
// section 01).
|
||||
const (
|
||||
HaveHistory = 20
|
||||
PresenceHistory = 20
|
||||
)
|
||||
|
||||
// PulseMoveBound is how long a move may take: a pulse key lives for this time minus the move's age, and the bucket
|
||||
// expires it on its own (the server writes a delete marker), so a move that overruns it is seen to stall.
|
||||
const PulseMoveBound = 120 * time.Second
|
||||
|
||||
// Version is the version of every value this package writes and the only one it reads.
|
||||
const Version = 1
|
||||
|
||||
// ErrUnknownVersion is what a decode returns for a value whose version is missing or not Version. Test it with
|
||||
// errors.Is; the message names the version seen and the one read.
|
||||
var ErrUnknownVersion = errors.New("delivery: unknown value version")
|
||||
|
||||
// Phase is how far a module's change has come on a node.
|
||||
type Phase string
|
||||
|
||||
// The phases of a module's change on a node.
|
||||
const (
|
||||
PhaseBackingUp Phase = "backing-up"
|
||||
PhaseApplying Phase = "applying"
|
||||
PhaseChecking Phase = "checking"
|
||||
PhaseRunning Phase = "running"
|
||||
PhaseFailed Phase = "failed"
|
||||
)
|
||||
|
||||
// Health is what the module's own check says.
|
||||
type Health string
|
||||
|
||||
// The healths a have carries.
|
||||
const (
|
||||
HealthOK Health = "ok"
|
||||
HealthFailed Health = "failed"
|
||||
HealthWaits Health = "waits"
|
||||
HealthUnknown Health = "unknown"
|
||||
)
|
||||
|
||||
var nameOK = regexp.MustCompile(`^[-/_=a-zA-Z0-9]+$`)
|
||||
|
||||
// checkName refuses a name that is empty or holds a dot, a wildcard, whitespace or a character a bus key refuses.
|
||||
func checkName(kind, name string) error {
|
||||
if name == "" {
|
||||
return fmt.Errorf("delivery: the %s name is empty", kind)
|
||||
}
|
||||
if !nameOK.MatchString(name) {
|
||||
return fmt.Errorf("delivery: the %s name %q holds a dot, a wildcard, whitespace or a character a key refuses", kind, name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// HaveKey is the key, in HaveBucket, of a module's have on a node: `<node>.<module>`. A module named `node` is
|
||||
// refused, since `<node>.node` is the node's own facts.
|
||||
func HaveKey(node, module string) (string, error) {
|
||||
if err := checkName("node", node); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := checkName("module", module); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if module == "node" {
|
||||
return "", fmt.Errorf(`delivery: the module name "node" is refused: %s.node is the node's own key`, node)
|
||||
}
|
||||
return node + "." + module, nil
|
||||
}
|
||||
|
||||
// HaveNodeKey is the key, in HaveBucket, of a node's own facts: `<node>.node`.
|
||||
func HaveNodeKey(node string) (string, error) {
|
||||
if err := checkName("node", node); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return node + ".node", nil
|
||||
}
|
||||
|
||||
// ParseHaveKey is the inverse of HaveKey and HaveNodeKey: isNode is true for `<node>.node`, with module empty.
|
||||
func ParseHaveKey(key string) (node, module string, isNode bool, err error) {
|
||||
node, module, ok := strings.Cut(key, ".")
|
||||
if !ok {
|
||||
return "", "", false, fmt.Errorf("delivery: %q is not a have key, <node>.<module>", key)
|
||||
}
|
||||
if err := checkName("node", node); err != nil {
|
||||
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
|
||||
}
|
||||
if err := checkName("module", module); err != nil {
|
||||
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
|
||||
}
|
||||
if module == "node" {
|
||||
return node, "", true, nil
|
||||
}
|
||||
return node, module, false, nil
|
||||
}
|
||||
|
||||
// PresenceNodeKey is the key, in PresenceBucket, of a node: `<node>`. A node named `part` is refused.
|
||||
func PresenceNodeKey(node string) (string, error) {
|
||||
if err := checkName("node", node); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if node == "part" {
|
||||
return "", errors.New(`delivery: the node name "part" is refused: part.<part>.<instance> is a part's key`)
|
||||
}
|
||||
return node, nil
|
||||
}
|
||||
|
||||
// PresencePartKey is the key, in PresenceBucket, of a part's instance: `part.<part>.<instance>`.
|
||||
func PresencePartKey(part, instance string) (string, error) {
|
||||
if err := checkName("part", part); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := checkName("instance", instance); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return "part." + part + "." + instance, nil
|
||||
}
|
||||
|
||||
// ParsePresenceKey is the inverse of PresenceNodeKey and PresencePartKey: for a node's key part and instance are
|
||||
// empty, for a part's key node is.
|
||||
func ParsePresenceKey(key string) (node, part, instance string, err error) {
|
||||
if !strings.Contains(key, ".") {
|
||||
node, err = PresenceNodeKey(key)
|
||||
if err != nil {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not a presence key: %w", key, err)
|
||||
}
|
||||
return node, "", "", nil
|
||||
}
|
||||
fields := strings.Split(key, ".")
|
||||
if len(fields) != 3 || fields[0] != "part" {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not a presence key, <node> or part.<part>.<instance>", key)
|
||||
}
|
||||
if _, err := PresencePartKey(fields[1], fields[2]); err != nil {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not a presence key: %w", key, err)
|
||||
}
|
||||
return "", fields[1], fields[2], nil
|
||||
}
|
||||
|
||||
// PulseKey is the key, in PulseBucket, of the move watched at key of bucket: `<bucket>.<key>`. The bucket is the
|
||||
// full bucket name and holds no dot; the key may.
|
||||
func PulseKey(bucket, key string) (string, error) {
|
||||
if err := checkName("bucket", bucket); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if key == "" || strings.HasPrefix(key, ".") || strings.HasSuffix(key, ".") || strings.Contains(key, "..") {
|
||||
return "", fmt.Errorf("delivery: the pulse key %q is empty or has an empty part", key)
|
||||
}
|
||||
for _, part := range strings.Split(key, ".") {
|
||||
if err := checkName("key", part); err != nil {
|
||||
return "", err
|
||||
}
|
||||
}
|
||||
return bucket + "." + key, nil
|
||||
}
|
||||
|
||||
// ParsePulseKey is the inverse of PulseKey, splitting at the first dot.
|
||||
func ParsePulseKey(pulseKey string) (bucket, key string, err error) {
|
||||
bucket, key, ok := strings.Cut(pulseKey, ".")
|
||||
if !ok {
|
||||
return "", "", fmt.Errorf("delivery: %q is not a pulse key, <bucket>.<key>", pulseKey)
|
||||
}
|
||||
if _, err := PulseKey(bucket, key); err != nil {
|
||||
return "", "", fmt.Errorf("delivery: %q is not a pulse key: %w", pulseKey, err)
|
||||
}
|
||||
return bucket, key, nil
|
||||
}
|
||||
|
||||
// PulseTTL is the time to live of a pulse key for a move of the given age: PulseMoveBound minus the age. At or past
|
||||
// the bound the move is stalled, with no time left and nothing to write: the watchman raises it at once.
|
||||
func PulseTTL(moveAge time.Duration) (ttl time.Duration, stalled bool) {
|
||||
if moveAge >= PulseMoveBound {
|
||||
return 0, true
|
||||
}
|
||||
return PulseMoveBound - moveAge, false
|
||||
}
|
||||
|
||||
// Have is a module's state on a node, written by that node's node-engine under HaveKey. FP is the module's
|
||||
// fingerprint once the indexer exists (design 47 release 3); until then Build is the build the node was sent, and
|
||||
// the view compares by build.
|
||||
type Have struct {
|
||||
V int `json:"v"`
|
||||
FP string `json:"fp,omitempty"`
|
||||
Build string `json:"build,omitempty"`
|
||||
Phase Phase `json:"phase"`
|
||||
Health Health `json:"health"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
}
|
||||
|
||||
// HaveNode is a node's own facts, written under HaveNodeKey. Each field is the node-engine's own JSON, carried whole:
|
||||
// this package names the facts and does not type them.
|
||||
type HaveNode struct {
|
||||
V int `json:"v"`
|
||||
Windows json.RawMessage `json:"windows,omitempty"`
|
||||
Busy json.RawMessage `json:"busy,omitempty"`
|
||||
Strays json.RawMessage `json:"strays,omitempty"`
|
||||
Filters json.RawMessage `json:"filters,omitempty"`
|
||||
Profile json.RawMessage `json:"profile,omitempty"`
|
||||
}
|
||||
|
||||
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
|
||||
// writes it.
|
||||
type Presence struct {
|
||||
V int `json:"v"`
|
||||
Online bool `json:"online"`
|
||||
}
|
||||
|
||||
// Pulse is the value of a pulse key: the watched key mirrored, its revision, and who owes the next move (a node or
|
||||
// a part instance). The watchman writes it with the time to live PulseTTL gives.
|
||||
type Pulse struct {
|
||||
V int `json:"v"`
|
||||
Bucket string `json:"bucket"`
|
||||
Key string `json:"key"`
|
||||
Revision uint64 `json:"revision"`
|
||||
Owes string `json:"owes,omitempty"`
|
||||
}
|
||||
|
||||
func checkVersion(data []byte) error {
|
||||
var probe struct {
|
||||
V *int `json:"v"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &probe); err != nil {
|
||||
return fmt.Errorf("delivery: value is not JSON: %w", err)
|
||||
}
|
||||
if probe.V == nil {
|
||||
return fmt.Errorf("%w: no version in the value, this reader reads %d", ErrUnknownVersion, Version)
|
||||
}
|
||||
if *probe.V != Version {
|
||||
return fmt.Errorf("%w: the value is version %d, this reader reads %d", ErrUnknownVersion, *probe.V, Version)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func decode(data []byte, into any) error {
|
||||
if err := checkVersion(data); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(data, into); err != nil {
|
||||
return fmt.Errorf("delivery: value does not fit its shape: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (h Have) check() error {
|
||||
switch h.Phase {
|
||||
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed:
|
||||
default:
|
||||
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running or failed", h.Phase)
|
||||
}
|
||||
switch h.Health {
|
||||
case HealthOK, HealthFailed, HealthWaits, HealthUnknown:
|
||||
default:
|
||||
return fmt.Errorf("delivery: health %q is not ok, failed, waits or unknown", h.Health)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeHave is h as JSON at Version; it refuses a phase or health outside the lists.
|
||||
func EncodeHave(h Have) ([]byte, error) {
|
||||
if err := h.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
h.V = Version
|
||||
return json.Marshal(h)
|
||||
}
|
||||
|
||||
// DecodeHave reads a have; it refuses an unknown version (ErrUnknownVersion) and a phase or health outside the lists.
|
||||
func DecodeHave(data []byte) (Have, error) {
|
||||
var h Have
|
||||
if err := decode(data, &h); err != nil {
|
||||
return Have{}, err
|
||||
}
|
||||
if err := h.check(); err != nil {
|
||||
return Have{}, err
|
||||
}
|
||||
return h, nil
|
||||
}
|
||||
|
||||
// EncodeHaveNode is n as JSON at Version.
|
||||
func EncodeHaveNode(n HaveNode) ([]byte, error) {
|
||||
n.V = Version
|
||||
return json.Marshal(n)
|
||||
}
|
||||
|
||||
// DecodeHaveNode reads a node's facts; it refuses an unknown version (ErrUnknownVersion).
|
||||
func DecodeHaveNode(data []byte) (HaveNode, error) {
|
||||
var n HaveNode
|
||||
err := decode(data, &n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// EncodePresence is p as JSON at Version.
|
||||
func EncodePresence(p Presence) ([]byte, error) {
|
||||
p.V = Version
|
||||
return json.Marshal(p)
|
||||
}
|
||||
|
||||
// DecodePresence reads a presence; it refuses an unknown version (ErrUnknownVersion).
|
||||
func DecodePresence(data []byte) (Presence, error) {
|
||||
var p Presence
|
||||
err := decode(data, &p)
|
||||
return p, err
|
||||
}
|
||||
|
||||
// EncodePulse is p as JSON at Version.
|
||||
func EncodePulse(p Pulse) ([]byte, error) {
|
||||
p.V = Version
|
||||
return json.Marshal(p)
|
||||
}
|
||||
|
||||
// DecodePulse reads a pulse; it refuses an unknown version (ErrUnknownVersion).
|
||||
func DecodePulse(data []byte) (Pulse, error) {
|
||||
var p Pulse
|
||||
err := decode(data, &p)
|
||||
return p, err
|
||||
}
|
||||
|
||||
// SameHave says whether two haves are the same value, so a writer writes only when they differ.
|
||||
func SameHave(a, b Have) bool { return a == b }
|
||||
|
||||
// SameHaveNode says whether two node facts are the same value, comparing each carried JSON without its spacing.
|
||||
func SameHaveNode(a, b HaveNode) bool {
|
||||
return a.V == b.V && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
|
||||
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile)
|
||||
}
|
||||
|
||||
// SamePresence says whether two presences are the same value.
|
||||
func SamePresence(a, b Presence) bool { return a == b }
|
||||
|
||||
// SamePulse says whether two pulses are the same value.
|
||||
func SamePulse(a, b Pulse) bool { return a == b }
|
||||
|
||||
func sameJSON(a, b json.RawMessage) bool {
|
||||
var ca, cb bytes.Buffer
|
||||
if json.Compact(&ca, a) != nil || json.Compact(&cb, b) != nil {
|
||||
return bytes.Equal(a, b)
|
||||
}
|
||||
return bytes.Equal(ca.Bytes(), cb.Bytes())
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// 1131/AC1: the have key forms and parses back; bad names are refused, naming them.
|
||||
func TestHaveKey(t *testing.T) {
|
||||
if HaveBucket != "node-engine_have" {
|
||||
t.Fatalf("HaveBucket = %q", HaveBucket)
|
||||
}
|
||||
key, err := HaveKey("ace", "plex")
|
||||
if err != nil || key != "ace.plex" {
|
||||
t.Fatalf("HaveKey = %q, %v", key, err)
|
||||
}
|
||||
node, module, isNode, err := ParseHaveKey(key)
|
||||
if err != nil || node != "ace" || module != "plex" || isNode {
|
||||
t.Fatalf("ParseHaveKey = %q %q %v %v", node, module, isNode, err)
|
||||
}
|
||||
if got, err := HaveNodeKey("ace"); err != nil || got != "ace.node" {
|
||||
t.Fatalf("HaveNodeKey = %q, %v", got, err)
|
||||
}
|
||||
if _, err := HaveNodeKey("a b"); err == nil {
|
||||
t.Error("HaveNodeKey with a space accepted")
|
||||
}
|
||||
node, module, isNode, err = ParseHaveKey("ace.node")
|
||||
if err != nil || node != "ace" || module != "" || !isNode {
|
||||
t.Fatalf("ParseHaveKey node = %q %q %v %v", node, module, isNode, err)
|
||||
}
|
||||
for _, c := range []struct{ node, module, named string }{
|
||||
{"ace", "node", "node"},
|
||||
{"ace", "", "module"},
|
||||
{"", "plex", "node"},
|
||||
{"a.b", "plex", "a.b"},
|
||||
{"ace", "p*", "p*"},
|
||||
{"ace", "p>", "p>"},
|
||||
{"ace", "p x", "p x"},
|
||||
{"ace", "p\tx", "module"},
|
||||
} {
|
||||
_, err := HaveKey(c.node, c.module)
|
||||
if err == nil || !strings.Contains(err.Error(), c.named) {
|
||||
t.Errorf("HaveKey(%q, %q) = %v, want an error naming %q", c.node, c.module, err, c.named)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"", "ace", "ace.", ".plex", "ace.plex.x", "ace.p x"} {
|
||||
if _, _, _, err := ParseHaveKey(bad); err == nil {
|
||||
t.Errorf("ParseHaveKey(%q) accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC2: the presence keys form and parse back; a node named part is refused.
|
||||
func TestPresenceKey(t *testing.T) {
|
||||
if PresenceBucket != "mesh-delivery-watchman_presence" {
|
||||
t.Fatalf("PresenceBucket = %q", PresenceBucket)
|
||||
}
|
||||
key, err := PresenceNodeKey("ace")
|
||||
if err != nil || key != "ace" {
|
||||
t.Fatalf("PresenceNodeKey = %q, %v", key, err)
|
||||
}
|
||||
node, part, instance, err := ParsePresenceKey(key)
|
||||
if err != nil || node != "ace" || part != "" || instance != "" {
|
||||
t.Fatalf("ParsePresenceKey node = %q %q %q %v", node, part, instance, err)
|
||||
}
|
||||
key, err = PresencePartKey("view", "novox")
|
||||
if err != nil || key != "part.view.novox" {
|
||||
t.Fatalf("PresencePartKey = %q, %v", key, err)
|
||||
}
|
||||
node, part, instance, err = ParsePresenceKey(key)
|
||||
if err != nil || node != "" || part != "view" || instance != "novox" {
|
||||
t.Fatalf("ParsePresenceKey part = %q %q %q %v", node, part, instance, err)
|
||||
}
|
||||
if _, err := PresenceNodeKey("part"); err == nil || !strings.Contains(err.Error(), "part") {
|
||||
t.Errorf("PresenceNodeKey(part) = %v", err)
|
||||
}
|
||||
if _, err := PresenceNodeKey("a.b"); err == nil {
|
||||
t.Error("PresenceNodeKey(a.b) accepted")
|
||||
}
|
||||
if _, err := PresencePartKey("", "x"); err == nil {
|
||||
t.Error("PresencePartKey with empty part accepted")
|
||||
}
|
||||
if _, err := PresencePartKey("view", "x y"); err == nil {
|
||||
t.Error("PresencePartKey with a space accepted")
|
||||
}
|
||||
for _, bad := range []string{"", "part", "part.view", "part.view.a.b", "a.b", "part..x"} {
|
||||
if _, _, _, err := ParsePresenceKey(bad); err == nil {
|
||||
t.Errorf("ParsePresenceKey(%q) accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC3: the pulse key splits at the first dot, the key holding dots.
|
||||
func TestPulseKey(t *testing.T) {
|
||||
if PulseBucket != "mesh-delivery-watchman_pulse" {
|
||||
t.Fatalf("PulseBucket = %q", PulseBucket)
|
||||
}
|
||||
key, err := PulseKey("node-engine_have", "ace.plex")
|
||||
if err != nil || key != "node-engine_have.ace.plex" {
|
||||
t.Fatalf("PulseKey = %q, %v", key, err)
|
||||
}
|
||||
bucket, rest, err := ParsePulseKey(key)
|
||||
if err != nil || bucket != "node-engine_have" || rest != "ace.plex" {
|
||||
t.Fatalf("ParsePulseKey = %q %q %v", bucket, rest, err)
|
||||
}
|
||||
for _, c := range [][2]string{{"", "k"}, {"a.b", "k"}, {"b", ""}, {"b", "k x"}, {"b", ".k"}, {"b", "k."}, {"b", "k*"}} {
|
||||
if _, err := PulseKey(c[0], c[1]); err == nil {
|
||||
t.Errorf("PulseKey(%q, %q) accepted", c[0], c[1])
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"", "nodot", ".k", "b."} {
|
||||
if _, _, err := ParsePulseKey(bad); err == nil {
|
||||
t.Errorf("ParsePulseKey(%q) accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC4: each value encodes, decodes equal, and carries "v":1.
|
||||
func TestRoundTrip(t *testing.T) {
|
||||
hasV1 := func(t *testing.T, b []byte) {
|
||||
t.Helper()
|
||||
var m map[string]any
|
||||
if err := json.Unmarshal(b, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m["v"] != float64(1) {
|
||||
t.Fatalf("v = %v in %s", m["v"], b)
|
||||
}
|
||||
if len(b) > 500 {
|
||||
t.Fatalf("value is %d bytes", len(b))
|
||||
}
|
||||
}
|
||||
t.Run("have", func(t *testing.T) {
|
||||
in := Have{Build: "b1", FP: "f1", Phase: PhaseRunning, Health: HealthOK, Detail: "up"}
|
||||
b, err := EncodeHave(in)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hasV1(t, b)
|
||||
out, err := DecodeHave(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
in.V = Version
|
||||
if out != in {
|
||||
t.Fatalf("got %+v want %+v", out, in)
|
||||
}
|
||||
})
|
||||
t.Run("have node", func(t *testing.T) {
|
||||
in := HaveNode{Windows: json.RawMessage(`{"a":1}`), Busy: json.RawMessage(`true`)}
|
||||
b, err := EncodeHaveNode(in)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hasV1(t, b)
|
||||
out, err := DecodeHaveNode(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !SameHaveNode(out, HaveNode{V: Version, Windows: in.Windows, Busy: in.Busy}) {
|
||||
t.Fatalf("got %+v", out)
|
||||
}
|
||||
})
|
||||
t.Run("presence", func(t *testing.T) {
|
||||
b, err := EncodePresence(Presence{Online: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hasV1(t, b)
|
||||
out, err := DecodePresence(b)
|
||||
if err != nil || out != (Presence{V: Version, Online: true}) {
|
||||
t.Fatalf("got %+v %v", out, err)
|
||||
}
|
||||
})
|
||||
t.Run("pulse", func(t *testing.T) {
|
||||
in := Pulse{Bucket: "node-engine_have", Key: "ace.plex", Revision: 7, Owes: "ace"}
|
||||
b, err := EncodePulse(in)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hasV1(t, b)
|
||||
out, err := DecodePulse(b)
|
||||
in.V = Version
|
||||
if err != nil || out != in {
|
||||
t.Fatalf("got %+v %v", out, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// 1131/AC5: a missing or unknown version is refused as ErrUnknownVersion, naming both versions.
|
||||
func TestUnknownVersion(t *testing.T) {
|
||||
decoders := map[string]func([]byte) error{
|
||||
"have": func(b []byte) error { _, err := DecodeHave(b); return err },
|
||||
"have node": func(b []byte) error { _, err := DecodeHaveNode(b); return err },
|
||||
"presence": func(b []byte) error { _, err := DecodePresence(b); return err },
|
||||
"pulse": func(b []byte) error { _, err := DecodePulse(b); return err },
|
||||
}
|
||||
for name, dec := range decoders {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
err := dec([]byte(`{"v":2,"phase":"running","health":"ok"}`))
|
||||
if !errors.Is(err, ErrUnknownVersion) || !strings.Contains(err.Error(), "2") || !strings.Contains(err.Error(), "1") {
|
||||
t.Errorf("v 2: %v", err)
|
||||
}
|
||||
err = dec([]byte(`{"phase":"running","health":"ok"}`))
|
||||
if !errors.Is(err, ErrUnknownVersion) {
|
||||
t.Errorf("missing v: %v", err)
|
||||
}
|
||||
if err := dec([]byte(`not json`)); err == nil || errors.Is(err, ErrUnknownVersion) {
|
||||
t.Errorf("garbage: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC6: a phase or health outside the lists is refused on encode and decode.
|
||||
func TestPhaseHealthRefused(t *testing.T) {
|
||||
for _, p := range []Phase{PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed} {
|
||||
for _, h := range []Health{HealthOK, HealthFailed, HealthWaits, HealthUnknown} {
|
||||
if _, err := EncodeHave(Have{Phase: p, Health: h}); err != nil {
|
||||
t.Errorf("%s/%s refused: %v", p, h, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if _, err := EncodeHave(Have{Phase: "sleeping", Health: HealthOK}); err == nil || !strings.Contains(err.Error(), "sleeping") {
|
||||
t.Errorf("bad phase encoded: %v", err)
|
||||
}
|
||||
if _, err := EncodeHave(Have{Phase: PhaseRunning, Health: "fine"}); err == nil || !strings.Contains(err.Error(), "fine") {
|
||||
t.Errorf("bad health encoded: %v", err)
|
||||
}
|
||||
if _, err := EncodeHave(Have{Phase: PhaseRunning}); err == nil {
|
||||
t.Error("empty health encoded")
|
||||
}
|
||||
if _, err := DecodeHave([]byte(`{"v":1,"phase":"sleeping","health":"ok"}`)); err == nil {
|
||||
t.Error("bad phase decoded")
|
||||
}
|
||||
if _, err := DecodeHave([]byte(`{"v":1,"phase":"running","health":"fine"}`)); err == nil {
|
||||
t.Error("bad health decoded")
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC7: the pulse's time to live is 120 seconds minus the move's age; at the bound the move is stalled.
|
||||
func TestPulseTTL(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
age time.Duration
|
||||
ttl time.Duration
|
||||
stalled bool
|
||||
}{
|
||||
{0, 120 * time.Second, false},
|
||||
{30 * time.Second, 90 * time.Second, false},
|
||||
{119 * time.Second, time.Second, false},
|
||||
{120 * time.Second, 0, true},
|
||||
{200 * time.Second, 0, true},
|
||||
} {
|
||||
ttl, stalled := PulseTTL(c.age)
|
||||
if ttl != c.ttl || stalled != c.stalled {
|
||||
t.Errorf("PulseTTL(%v) = %v, %v; want %v, %v", c.age, ttl, stalled, c.ttl, c.stalled)
|
||||
}
|
||||
}
|
||||
if PulseMoveBound != 120*time.Second || HaveHistory != 20 || PresenceHistory != 20 {
|
||||
t.Error("constants")
|
||||
}
|
||||
}
|
||||
|
||||
// 1131/AC8: equal values say same, different ones do not.
|
||||
func TestSame(t *testing.T) {
|
||||
a := Have{V: 1, Build: "b", Phase: PhaseRunning, Health: HealthOK}
|
||||
b := a
|
||||
if !SameHave(a, b) {
|
||||
t.Error("equal haves differ")
|
||||
}
|
||||
b.Detail = "x"
|
||||
if SameHave(a, b) {
|
||||
t.Error("different haves same")
|
||||
}
|
||||
n := HaveNode{V: 1, Busy: json.RawMessage(`true`)}
|
||||
if !SameHaveNode(n, HaveNode{V: 1, Busy: json.RawMessage(`true`)}) || SameHaveNode(n, HaveNode{V: 1}) {
|
||||
t.Error("have node")
|
||||
}
|
||||
if !SamePresence(Presence{V: 1, Online: true}, Presence{V: 1, Online: true}) || SamePresence(Presence{V: 1}, Presence{V: 1, Online: true}) {
|
||||
t.Error("presence")
|
||||
}
|
||||
p := Pulse{V: 1, Bucket: "b", Key: "k", Revision: 1, Owes: "ace"}
|
||||
q := p
|
||||
if !SamePulse(p, q) {
|
||||
t.Error("equal pulses differ")
|
||||
}
|
||||
q.Revision = 2
|
||||
if SamePulse(p, q) {
|
||||
t.Error("different pulses same")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// SourceInputs are what a module's source fingerprint is of (novox/hq ADR 0437 §5): the module, the git tree of
|
||||
// its directory, its bases (references, of which only the digest counts), the root tree of each repository its
|
||||
// artifacts read (by artifact), each artifact's toolchain line, and the reason the source does not pin the build
|
||||
// when it does not (a build that resolves packages when built). A context is the value the builder records for it:
|
||||
// a repository's root tree, or for an image that compiles a Go program the `narrow:` hash of the build source it was
|
||||
// handed (mesh-controller internal/builder/builder.go:1146-1156).
|
||||
type SourceInputs struct {
|
||||
Module string
|
||||
Tree string
|
||||
Bases []string
|
||||
Contexts map[string]string
|
||||
Toolchains map[string]string
|
||||
Unpinned string
|
||||
}
|
||||
|
||||
// SourceFingerprint is the module's source fingerprint, `src1:` and the hex sha256 of its inputs' lines, or none
|
||||
// and the reason when the source does not pin the build: Unpinned when set, "no tree" when Tree is empty. It is the
|
||||
// one function the controller's builder and the indexer both call, so they agree byte for byte. The lines are
|
||||
// "module <m>" and "tree <t>", then "base <digest>" (once each), "context <artifact> <tree>" and "toolchain
|
||||
// <artifact> <line>" sorted, all joined by a newline.
|
||||
func SourceFingerprint(in SourceInputs) (fp, none string) {
|
||||
if in.Unpinned != "" {
|
||||
return "", in.Unpinned
|
||||
}
|
||||
if in.Tree == "" {
|
||||
return "", "no tree"
|
||||
}
|
||||
lines := []string{"module " + in.Module, "tree " + in.Tree}
|
||||
bases := map[string]bool{}
|
||||
for _, b := range in.Bases {
|
||||
bases[ReferenceDigest(b)] = true
|
||||
}
|
||||
for b := range bases {
|
||||
lines = append(lines, "base "+b)
|
||||
}
|
||||
for a, t := range in.Contexts {
|
||||
lines = append(lines, "context "+a+" "+t)
|
||||
}
|
||||
for a, t := range in.Toolchains {
|
||||
lines = append(lines, "toolchain "+a+" "+t)
|
||||
}
|
||||
sort.Strings(lines[2:])
|
||||
sum := sha256.Sum256([]byte(strings.Join(lines, "\n")))
|
||||
return SourcePrefix + hex.EncodeToString(sum[:]), ""
|
||||
}
|
||||
|
||||
// ReferenceDigest is a reference's digest, `sha256:...`, so the registry address a base was copied into does not
|
||||
// enter the fingerprint; the reference itself when it carries none.
|
||||
func ReferenceDigest(reference string) string {
|
||||
if _, digest, pinned := strings.Cut(reference, "@"); pinned && digest != "" {
|
||||
return digest
|
||||
}
|
||||
return reference
|
||||
}
|
||||
@@ -0,0 +1,178 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func inputs() SourceInputs {
|
||||
return SourceInputs{
|
||||
Module: "mesh-mail", Tree: "t-module",
|
||||
Bases: []string{"registry.example/base@sha256:aaa", "registry.example/other@sha256:bbb"},
|
||||
Contexts: map[string]string{"mail": "t-root", "tool": "t-root2"},
|
||||
Toolchains: map[string]string{"mail": "go sha256:ccc 0123456789abcdef", "tool": "go sha256:ccc fedcba9876543210"},
|
||||
}
|
||||
}
|
||||
|
||||
func mustFP(t *testing.T, in SourceInputs) string {
|
||||
t.Helper()
|
||||
fp, none := SourceFingerprint(in)
|
||||
if fp == "" || none != "" {
|
||||
t.Fatalf("fingerprint = %q, none = %q", fp, none)
|
||||
}
|
||||
return fp
|
||||
}
|
||||
|
||||
func TestFingerprintForm(t *testing.T) {
|
||||
fp := mustFP(t, inputs())
|
||||
hex := strings.TrimPrefix(fp, SourcePrefix)
|
||||
if SourcePrefix != "src1:" || hex == fp || len(hex) != 64 || strings.Trim(hex, "0123456789abcdef") != "" {
|
||||
t.Errorf("fingerprint %q is not src1: and 64 hex digits", fp)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFingerprintOrderAndDuplicates(t *testing.T) { // 1203/AC2
|
||||
want := mustFP(t, inputs())
|
||||
in := inputs()
|
||||
in.Bases = []string{in.Bases[1], in.Bases[0]}
|
||||
if got := mustFP(t, in); got != want {
|
||||
t.Errorf("the order of bases changes it: %s", got)
|
||||
}
|
||||
in = inputs()
|
||||
in.Bases = append(in.Bases, in.Bases[0])
|
||||
if got := mustFP(t, in); got != want {
|
||||
t.Errorf("a base given twice changes it: %s", got)
|
||||
}
|
||||
in = inputs()
|
||||
in.Bases = append(in.Bases, "other.registry.example/base@sha256:aaa")
|
||||
if got := mustFP(t, in); got != want {
|
||||
t.Errorf("two references to one digest change it: %s", got)
|
||||
}
|
||||
// Maps have no order, but build them in many insertion orders anyway.
|
||||
for i := 0; i < 20; i++ {
|
||||
in = inputs()
|
||||
in.Contexts = map[string]string{}
|
||||
in.Toolchains = map[string]string{}
|
||||
for _, k := range []string{"tool", "mail"} {
|
||||
in.Contexts[k] = inputs().Contexts[k]
|
||||
in.Toolchains[k] = inputs().Toolchains[k]
|
||||
}
|
||||
if got := mustFP(t, in); got != want {
|
||||
t.Fatalf("map order changes it: %s", got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFingerprintChanges(t *testing.T) { // 1203/AC3
|
||||
want := mustFP(t, inputs())
|
||||
for name, f := range map[string]func(*SourceInputs){
|
||||
"module": func(in *SourceInputs) { in.Module = "other" },
|
||||
"tree": func(in *SourceInputs) { in.Tree = "t-other" },
|
||||
"base": func(in *SourceInputs) { in.Bases[0] = "registry.example/base@sha256:zzz" },
|
||||
"a base added": func(in *SourceInputs) { in.Bases = append(in.Bases, "x@sha256:new") },
|
||||
"context": func(in *SourceInputs) { in.Contexts["mail"] = "t-changed" },
|
||||
"context artifact": func(in *SourceInputs) { in.Contexts["extra"] = "t-root" },
|
||||
"toolchain": func(in *SourceInputs) { in.Toolchains["tool"] = "go sha256:ccc changed" },
|
||||
} {
|
||||
in := inputs()
|
||||
f(&in)
|
||||
if got := mustFP(t, in); got == want {
|
||||
t.Errorf("a changed %s keeps the fingerprint", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFingerprintNone(t *testing.T) { // 1203/AC4
|
||||
in := inputs()
|
||||
in.Unpinned = "the build resolves packages when built"
|
||||
fp, none := SourceFingerprint(in)
|
||||
if fp != "" || none != in.Unpinned {
|
||||
t.Errorf("unpinned: fp = %q, none = %q", fp, none)
|
||||
}
|
||||
in = inputs()
|
||||
in.Tree = ""
|
||||
fp, none = SourceFingerprint(in)
|
||||
if fp != "" || none != "no tree" {
|
||||
t.Errorf("no tree: fp = %q, none = %q", fp, none)
|
||||
}
|
||||
in = inputs()
|
||||
in.Tree, in.Unpinned = "", "the build resolves packages when built"
|
||||
if _, none = SourceFingerprint(in); none != in.Unpinned {
|
||||
t.Errorf("the reason for unpinned stands over no tree: %q", none)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReferenceDigest(t *testing.T) {
|
||||
for in, want := range map[string]string{
|
||||
"registry.example/base@sha256:aaa": "sha256:aaa",
|
||||
"registry.example/base": "registry.example/base",
|
||||
"registry.example/base@": "registry.example/base@",
|
||||
"": "",
|
||||
} {
|
||||
if got := ReferenceDigest(in); got != want {
|
||||
t.Errorf("ReferenceDigest(%q) = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1203/AC1: three builds the live builder recorded a source fingerprint for. Each value is captured, never written
|
||||
// by hand: the recorded fingerprint was read with mesh-controller.builds (after issues 1204 and 1208 were
|
||||
// delivered); each tree is `git rev-parse <commit>:<dir>` in the repository's clone; the bases are from the
|
||||
// build's `[bases]` log lines; the toolchain lines are mesh-controller's toolchainOf over its toolchains values.
|
||||
func TestFingerprintLiveBuilds(t *testing.T) {
|
||||
const ts = "typescript sha256:4b81682df5665c1f461d3f720a3a0fd7593f4aa71e4fc272dbfdc4bbd235b489 20125bd590041690"
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
in SourceInputs
|
||||
recorded string
|
||||
}{
|
||||
{
|
||||
// mailu, build-1792391134620131370: mesh-catalog at b4527d9091e972f28cb365623d9280045e7741a3, dir modules/mailu.
|
||||
name: "mailu",
|
||||
in: SourceInputs{
|
||||
Module: "mailu",
|
||||
Tree: "cfc6fa8ff5fc1e220132e51ade653b9d3f387b3e",
|
||||
Bases: []string{"sha256:25f3cfeaceca14921366af4d1240b56457ef46273bdb508c7b0e8f469f6fd228"},
|
||||
Toolchains: map[string]string{"code": ts},
|
||||
},
|
||||
recorded: "src1:539f2e64000d436c572ef53b40e524012455945f8c5a1c6bb173ee0707e88ef4",
|
||||
},
|
||||
{
|
||||
// radarr, build-1792391147149753355: mesh-media-catalog at 197abb061d9b77d67c7f11e9e567528ea7a85436,
|
||||
// dir modules/radarr; no bases.
|
||||
name: "radarr",
|
||||
in: SourceInputs{
|
||||
Module: "radarr",
|
||||
Tree: "42b9c918b5af75cba2bbc75845012e7f64e16df9",
|
||||
Toolchains: map[string]string{
|
||||
"code": ts,
|
||||
"tools-go": "go sha256:2054591ca034b68323d87ec887ba34f56819cb3364034360c0777da4b314dd54 0995d85edc9a6a23",
|
||||
},
|
||||
},
|
||||
recorded: "src1:b3c150d39c395b99dd68974a48490e9a42467aa5d486d0c604d6ef1e8e9c910a",
|
||||
},
|
||||
{
|
||||
// build-agent, build-1792395447088936520: mesh-controller at 7a74cee83f88e6585a5511c3cc53bdf0f5303d20,
|
||||
// dir modules/build-agent; the context is narrowTree's hash of the 185 paths GoBuildSource gave for
|
||||
// cmd/mesh-builder at 7a74cee8.
|
||||
name: "build-agent",
|
||||
in: SourceInputs{
|
||||
Module: "build-agent",
|
||||
Tree: "11c0b4f90cae0d6e4ef9cc48cbfdf9f6dbe52bca",
|
||||
Bases: []string{
|
||||
"sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bc",
|
||||
"sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c",
|
||||
},
|
||||
Contexts: map[string]string{"server": "narrow:d51ca744fd11af59d616c858f76c27b3e58e16e11fbdc0c720eb9d7d19d55fac"},
|
||||
},
|
||||
recorded: "src1:e846efe73716b78e80860baaac96153de2cb76551325753838ca245d2160a3d7",
|
||||
},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
fp, none := SourceFingerprint(c.in)
|
||||
if none != "" || fp != c.recorded {
|
||||
t.Errorf("fingerprint = %q (none %q), the live builder recorded %q", fp, none, c.recorded)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,404 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
// The gate's buckets and the teller's (ADR 0437 §6, §8). The gate writes merges (each pull request's expected
|
||||
// head, its state and each merge commit), its asks (`merge.<group>.<heads hash>`, the router's ask whole) and owed
|
||||
// (one key per act it owes the forge, which the forge bridge performs once per revision). The teller writes its
|
||||
// answer under the ask's own key in its bucket answers.
|
||||
const (
|
||||
MergesBucket = GateModule + "_merges"
|
||||
AsksBucket = GateModule + "_asks"
|
||||
OwedBucket = GateModule + "_owed"
|
||||
AnswersBucket = TellerModule + "_answers"
|
||||
MergesHistory = 10
|
||||
AsksHistory = 5
|
||||
OwedHistory = 5
|
||||
AnswersHistory = 5
|
||||
)
|
||||
|
||||
// AskKindMerge is the kind of the gate's ask for a group's merge, the first part of its key.
|
||||
const AskKindMerge = "merge"
|
||||
|
||||
// The acts the gate owes the forge, the last part of an owed key.
|
||||
const (
|
||||
OwedStatus = "status"
|
||||
OwedView = "view"
|
||||
OwedNote = "note"
|
||||
OwedComment = "comment"
|
||||
)
|
||||
|
||||
// The states of a pull request in a merge.
|
||||
type MergeState string
|
||||
|
||||
// The states a merge's pull request is in.
|
||||
const (
|
||||
MergeMerging MergeState = "merging"
|
||||
MergeMerged MergeState = "merged"
|
||||
MergeRefused MergeState = "refused"
|
||||
)
|
||||
|
||||
// The states a status the gate owes the forge may carry.
|
||||
var statusStates = []string{"pending", "success", "failure", "error", "warning"}
|
||||
|
||||
var hashOK = regexp.MustCompile(`^[0-9a-f]{12}$`)
|
||||
|
||||
// MergesKey is the key, in MergesBucket, of a group: `<group>`.
|
||||
func MergesKey(group string) (string, error) {
|
||||
if err := checkName("group", group); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return group, nil
|
||||
}
|
||||
|
||||
// MergeAskKey is the key, in AsksBucket, of the ask for merging a group at these heads: `merge.<group>.<heads
|
||||
// hash>`. The teller's answer has the same key in AnswersBucket (AnswerKey).
|
||||
func MergeAskKey(group string, heads []Head) (string, error) {
|
||||
if err := checkName("group", group); err != nil {
|
||||
return "", err
|
||||
}
|
||||
hash, err := HeadsHash(heads)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return AskKindMerge + "." + group + "." + hash, nil
|
||||
}
|
||||
|
||||
// ParseAskKey is the inverse of MergeAskKey: the kind, the group and the heads hash.
|
||||
func ParseAskKey(key string) (kind, group, hash string, err error) {
|
||||
parts := strings.Split(key, ".")
|
||||
if len(parts) != 3 || parts[0] != AskKindMerge {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not an ask key, merge.<group>.<heads hash>", key)
|
||||
}
|
||||
if err := checkName("group", parts[1]); err != nil {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not an ask key: %w", key, err)
|
||||
}
|
||||
if !hashOK.MatchString(parts[2]) {
|
||||
return "", "", "", fmt.Errorf("delivery: %q is not an ask key: %q is not 12 hex digits", key, parts[2])
|
||||
}
|
||||
return parts[0], parts[1], parts[2], nil
|
||||
}
|
||||
|
||||
// AnswerKey is the key, in AnswersBucket, of the teller's answer to the ask at askKey: the same string.
|
||||
func AnswerKey(askKey string) string { return askKey }
|
||||
|
||||
// AskID is the id the teller gives the router for the ask at askKey. The router's ids refuse dots (asks.UsableID),
|
||||
// so in the readable part a dot becomes `-` and any other byte outside letters, digits, `_` and `-` becomes `_`; that part is cut to fit, and every id ends in `-` and
|
||||
// the first 12 hex digits of the sha256 of the whole key, so two keys never give one id. It cannot be read back:
|
||||
// the teller keeps the key beside it.
|
||||
func AskID(askKey string) string {
|
||||
const most = 64 // the router's bound on an id
|
||||
const tail = 13 // "-" and 12 hex digits
|
||||
readable := []byte(askKey)
|
||||
for i, c := range readable {
|
||||
switch {
|
||||
case c >= 'A' && c <= 'Z', c >= 'a' && c <= 'z', c >= '0' && c <= '9':
|
||||
case c == '.':
|
||||
readable[i] = '-'
|
||||
default: // a slash, an equals sign or any other byte the router's id refuses
|
||||
readable[i] = '_'
|
||||
}
|
||||
}
|
||||
if len(readable) > 0 && (readable[0] == '-' || readable[0] == '_') {
|
||||
readable[0] = 'k' // an id starts with a letter or digit
|
||||
}
|
||||
if len(readable) > most-tail {
|
||||
readable = readable[:most-tail]
|
||||
}
|
||||
sum := sha256.Sum256([]byte(askKey))
|
||||
return string(readable) + "-" + hex.EncodeToString(sum[:])[:12]
|
||||
}
|
||||
|
||||
func checkOwedAct(act string) (kind, context string, err error) {
|
||||
kind, context, hasContext := strings.Cut(act, "=")
|
||||
switch kind {
|
||||
case OwedStatus:
|
||||
if !hasContext {
|
||||
return "", "", fmt.Errorf("delivery: the owed act %q is a status without its context, status=<context>", act)
|
||||
}
|
||||
if err := checkName("status context", context); err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return kind, context, nil
|
||||
case OwedView, OwedNote, OwedComment:
|
||||
if hasContext {
|
||||
return "", "", fmt.Errorf("delivery: the owed act %q takes no context", act)
|
||||
}
|
||||
return kind, "", nil
|
||||
}
|
||||
return "", "", fmt.Errorf("delivery: the owed act %q is not status=<context>, view, note or comment", act)
|
||||
}
|
||||
|
||||
// ParseOwedAct splits an owed act into its kind and, for a status, its context.
|
||||
func ParseOwedAct(act string) (kind, context string, err error) { return checkOwedAct(act) }
|
||||
|
||||
// OwedKey is the key, in OwedBucket, of an act the gate owes the forge on a pull request at a head:
|
||||
// `<repository>.<number>.<head>.<act>`, the act being `status=<context>` (mesh/merge-gate), `view`, `note` or
|
||||
// `comment`.
|
||||
func OwedKey(repository string, number int, head, act string) (string, error) {
|
||||
if err := checkRepository(repository); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if number <= 0 {
|
||||
return "", fmt.Errorf("delivery: an owed key needs the pull request number, not %d", number)
|
||||
}
|
||||
if err := checkName("head", head); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, _, err := checkOwedAct(act); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return repository + "." + strconv.Itoa(number) + "." + head + "." + act, nil
|
||||
}
|
||||
|
||||
// ParseOwedKey is the inverse of OwedKey; act is whole, as `status=<context>` for a status.
|
||||
func ParseOwedKey(key string) (repository string, number int, head, act string, err error) {
|
||||
parts := strings.Split(key, ".")
|
||||
if len(parts) != 4 {
|
||||
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key, <repository>.<number>.<head>.<act>", key)
|
||||
}
|
||||
number, convErr := strconv.Atoi(parts[1])
|
||||
if convErr != nil {
|
||||
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key: %q is not a pull request number", key, parts[1])
|
||||
}
|
||||
if _, err := OwedKey(parts[0], number, parts[2], parts[3]); err != nil {
|
||||
return "", 0, "", "", fmt.Errorf("delivery: %q is not an owed key: %w", key, err)
|
||||
}
|
||||
return parts[0], number, parts[2], parts[3], nil
|
||||
}
|
||||
|
||||
// Merge is the value of a merges key: the ask approved and, for each pull request, the head the approval expects
|
||||
// and how far its merge has come.
|
||||
type Merge struct {
|
||||
V int `json:"v"`
|
||||
Group string `json:"group"`
|
||||
Ask string `json:"ask"`
|
||||
Pulls []MergePull `json:"pulls"`
|
||||
}
|
||||
|
||||
// MergePull is one pull request of a merge: its expected head, its state, the merge commit once merged, the
|
||||
// forge's words when refused.
|
||||
type MergePull struct {
|
||||
Head
|
||||
State MergeState `json:"state"`
|
||||
Commit string `json:"commit,omitempty"`
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
func (m Merge) check() error {
|
||||
if err := checkName("group", m.Group); err != nil {
|
||||
return err
|
||||
}
|
||||
if m.Ask == "" {
|
||||
return fmt.Errorf("delivery: a merge names no ask")
|
||||
}
|
||||
if len(m.Pulls) == 0 {
|
||||
return fmt.Errorf("delivery: a merge names no pull request")
|
||||
}
|
||||
for _, p := range m.Pulls {
|
||||
who := fmt.Sprintf("%s#%d", p.Repository, p.Number)
|
||||
switch p.State {
|
||||
case MergeMerging:
|
||||
case MergeMerged:
|
||||
if p.Commit == "" {
|
||||
return fmt.Errorf("delivery: the merged pull request %s has no merge commit", who)
|
||||
}
|
||||
case MergeRefused:
|
||||
if p.Words == "" {
|
||||
return fmt.Errorf("delivery: the refused pull request %s has no words of the forge", who)
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("delivery: the state %q of %s is not merging, merged or refused", p.State, who)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Ask is the value of an asks key: the router's ask whole as the gate composes it, so the teller publishes it
|
||||
// unchanged, and the group and heads it was composed for.
|
||||
type Ask struct {
|
||||
V int `json:"v"`
|
||||
Group string `json:"group"`
|
||||
Heads []Head `json:"heads"`
|
||||
Ask asks.Ask `json:"ask"`
|
||||
}
|
||||
|
||||
func (a Ask) check() error {
|
||||
key, err := MergeAskKey(a.Group, a.Heads)
|
||||
if err != nil {
|
||||
return fmt.Errorf("delivery: an ask needs its group and at least one pull request's head: %w", err)
|
||||
}
|
||||
if want := AskID(key); a.Ask.ID != want {
|
||||
return fmt.Errorf("delivery: the ask's id %q is not %q, the id its group and heads give", a.Ask.ID, want)
|
||||
}
|
||||
if len(a.Ask.Options) == 0 {
|
||||
return fmt.Errorf("delivery: the ask has no option")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Answer is the teller's answer to an ask: the option chosen, the warrant's id and its words.
|
||||
type Answer struct {
|
||||
V int `json:"v"`
|
||||
Option string `json:"option"`
|
||||
Warrant string `json:"warrant"`
|
||||
Words string `json:"words"`
|
||||
}
|
||||
|
||||
func (a Answer) check() error {
|
||||
for _, f := range []struct{ name, value string }{{"option", a.Option}, {"warrant", a.Warrant}, {"words", a.Words}} {
|
||||
if f.value == "" {
|
||||
return fmt.Errorf("delivery: an answer has no %s", f.name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Owed is the value of an owed key: what the forge bridge performs. A status needs Context and State, a view and a
|
||||
// comment need Text, a note needs Text and Ref (the notes ref, mesh-plan).
|
||||
type Owed struct {
|
||||
V int `json:"v"`
|
||||
Act string `json:"act"`
|
||||
Context string `json:"context,omitempty"`
|
||||
State string `json:"state,omitempty"`
|
||||
Text string `json:"text,omitempty"`
|
||||
Ref string `json:"ref,omitempty"`
|
||||
}
|
||||
|
||||
func (o Owed) check() error {
|
||||
switch o.Act {
|
||||
case OwedStatus:
|
||||
if o.Context == "" {
|
||||
return fmt.Errorf("delivery: an owed status has no context")
|
||||
}
|
||||
for _, s := range statusStates {
|
||||
if o.State == s {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("delivery: the owed status state %q is not %s", o.State, strings.Join(statusStates, ", "))
|
||||
case OwedView, OwedComment:
|
||||
if o.Text == "" {
|
||||
return fmt.Errorf("delivery: an owed %s has no text", o.Act)
|
||||
}
|
||||
case OwedNote:
|
||||
if o.Text == "" {
|
||||
return fmt.Errorf("delivery: an owed note has no text")
|
||||
}
|
||||
if o.Ref == "" {
|
||||
return fmt.Errorf("delivery: an owed note has no ref")
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("delivery: the owed act %q is not status, view, note or comment", o.Act)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeMerge is m as JSON at Version; it refuses a merged pull request without its merge commit, a refused one
|
||||
// without the forge's words and a state outside the list.
|
||||
func EncodeMerge(m Merge) ([]byte, error) {
|
||||
if err := m.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.V = Version
|
||||
return json.Marshal(m)
|
||||
}
|
||||
|
||||
// DecodeMerge reads a merge; it refuses an unknown version (ErrUnknownVersion) and what EncodeMerge refuses.
|
||||
func DecodeMerge(data []byte) (Merge, error) {
|
||||
var m Merge
|
||||
if err := decode(data, &m); err != nil {
|
||||
return Merge{}, err
|
||||
}
|
||||
if err := m.check(); err != nil {
|
||||
return Merge{}, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// EncodeAsk is a as JSON at Version; it refuses an ask whose id is not the one its group and heads give, or with
|
||||
// no option.
|
||||
func EncodeAsk(a Ask) ([]byte, error) {
|
||||
if err := a.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.V = Version
|
||||
return json.Marshal(a)
|
||||
}
|
||||
|
||||
// DecodeAsk reads an ask; it refuses an unknown version (ErrUnknownVersion) and what EncodeAsk refuses.
|
||||
func DecodeAsk(data []byte) (Ask, error) {
|
||||
var a Ask
|
||||
if err := decode(data, &a); err != nil {
|
||||
return Ask{}, err
|
||||
}
|
||||
if err := a.check(); err != nil {
|
||||
return Ask{}, err
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// EncodeAnswer is a as JSON at Version; it refuses an answer missing its option, warrant or words.
|
||||
func EncodeAnswer(a Answer) ([]byte, error) {
|
||||
if err := a.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.V = Version
|
||||
return json.Marshal(a)
|
||||
}
|
||||
|
||||
// DecodeAnswer reads an answer; it refuses an unknown version (ErrUnknownVersion) and what EncodeAnswer refuses.
|
||||
func DecodeAnswer(data []byte) (Answer, error) {
|
||||
var a Answer
|
||||
if err := decode(data, &a); err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
if err := a.check(); err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// EncodeOwed is o as JSON at Version; it refuses an act missing what it needs.
|
||||
func EncodeOwed(o Owed) ([]byte, error) {
|
||||
if err := o.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
o.V = Version
|
||||
return json.Marshal(o)
|
||||
}
|
||||
|
||||
// DecodeOwed reads an owed act; it refuses an unknown version (ErrUnknownVersion) and what EncodeOwed refuses.
|
||||
func DecodeOwed(data []byte) (Owed, error) {
|
||||
var o Owed
|
||||
if err := decode(data, &o); err != nil {
|
||||
return Owed{}, err
|
||||
}
|
||||
if err := o.check(); err != nil {
|
||||
return Owed{}, err
|
||||
}
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// SameMerge says whether two merges are the same value, so a writer writes only when they differ.
|
||||
func SameMerge(a, b Merge) bool { return reflect.DeepEqual(a, b) }
|
||||
|
||||
// SameAsk says whether two asks are the same value.
|
||||
func SameAsk(a, b Ask) bool { return reflect.DeepEqual(a, b) }
|
||||
|
||||
// SameAnswer says whether two answers are the same value.
|
||||
func SameAnswer(a, b Answer) bool { return a == b }
|
||||
|
||||
// SameOwed says whether two owed acts are the same value.
|
||||
func SameOwed(a, b Owed) bool { return a == b }
|
||||
@@ -0,0 +1,301 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||
)
|
||||
|
||||
func gateHeads() []Head {
|
||||
return []Head{
|
||||
{Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa"},
|
||||
{Repository: "novox/mesh-control", Number: 7, Head: "bbbb"},
|
||||
}
|
||||
}
|
||||
|
||||
func TestMergeKeys(t *testing.T) { // 1201/AC1
|
||||
if k, err := MergesKey("feat/x"); err != nil || k != "feat/x" {
|
||||
t.Fatalf("merges key = %q, %v", k, err)
|
||||
}
|
||||
if _, err := MergesKey("a.b"); err == nil {
|
||||
t.Error("a group with a dot accepted")
|
||||
}
|
||||
hh, _ := HeadsHash(gateHeads())
|
||||
key, err := MergeAskKey("feat/x", gateHeads())
|
||||
if err != nil || key != "merge.feat/x."+hh {
|
||||
t.Fatalf("ask key = %q, %v", key, err)
|
||||
}
|
||||
kind, group, hash, err := ParseAskKey(key)
|
||||
if err != nil || kind != AskKindMerge || group != "feat/x" || hash != hh {
|
||||
t.Fatalf("parse ask key = %q %q %q %v", kind, group, hash, err)
|
||||
}
|
||||
if AnswerKey(key) != key {
|
||||
t.Error("the answer's key is not the ask's key")
|
||||
}
|
||||
if _, err := MergeAskKey("feat/x", nil); err == nil {
|
||||
t.Error("an ask key without heads accepted")
|
||||
}
|
||||
if _, err := MergeAskKey("", gateHeads()); err == nil {
|
||||
t.Error("an ask key without a group accepted")
|
||||
}
|
||||
for _, bad := range []string{"", "merge", "merge.g", "merge.g.h.x", "other.g.0123456789ab", "merge..0123456789ab", "merge.g.nothex"} {
|
||||
if _, _, _, err := ParseAskKey(bad); err == nil {
|
||||
t.Errorf("parse ask key %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAskID(t *testing.T) { // 1201/AC2
|
||||
k1, _ := MergeAskKey("feat/x", gateHeads())
|
||||
k2, _ := MergeAskKey("feat/y", gateHeads())
|
||||
k3, _ := MergeAskKey("feat/x", gateHeads()[:1])
|
||||
long, _ := MergeAskKey("feat/"+strings.Repeat("long-", 20), gateHeads())
|
||||
long2, _ := MergeAskKey("feat/"+strings.Repeat("long-", 20)+"2", gateHeads())
|
||||
slash, _ := MergeAskKey("a/b", gateHeads())
|
||||
under, _ := MergeAskKey("a_b", gateHeads())
|
||||
eq, _ := MergeAskKey("feat=x", gateHeads())
|
||||
eqs, _ := MergeAskKey("feat_x", gateHeads())
|
||||
seen := map[string]string{}
|
||||
for _, k := range []string{k1, k2, k3, long, long2, slash, under, eq, eqs, "=odd.key"} {
|
||||
id := AskID(k)
|
||||
if strings.Contains(id, ".") {
|
||||
t.Errorf("id %q holds a dot", id)
|
||||
}
|
||||
if !asks.UsableID(id) {
|
||||
t.Errorf("id %q is not usable by the router", id)
|
||||
}
|
||||
if other, dup := seen[id]; dup {
|
||||
t.Errorf("keys %q and %q give the id %q", k, other, id)
|
||||
}
|
||||
seen[id] = k
|
||||
}
|
||||
if AskID(k1) != AskID(k1) {
|
||||
t.Error("the id is not stable")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOwedKey(t *testing.T) { // 1201/AC3
|
||||
for _, act := range []string{"status=mesh/merge-gate", "view", "note", "comment"} {
|
||||
key, err := OwedKey("novox/mesh-sdk", 36, "aaaa", act)
|
||||
if err != nil || key != "novox/mesh-sdk.36.aaaa."+act {
|
||||
t.Fatalf("owed key for %s = %q, %v", act, key, err)
|
||||
}
|
||||
repo, number, head, gotAct, err := ParseOwedKey(key)
|
||||
if err != nil || repo != "novox/mesh-sdk" || number != 36 || head != "aaaa" || gotAct != act {
|
||||
t.Fatalf("parse %q = %q %d %q %q %v", key, repo, number, head, gotAct, err)
|
||||
}
|
||||
}
|
||||
kind, context, err := ParseOwedAct("status=mesh/merge-gate")
|
||||
if err != nil || kind != OwedStatus || context != "mesh/merge-gate" {
|
||||
t.Errorf("owed act = %q %q %v", kind, context, err)
|
||||
}
|
||||
for _, bad := range []string{"", "merge", "status", "status=", "view=x", "status=a.b", "statuses=x"} {
|
||||
if _, err := OwedKey("novox/mesh-sdk", 36, "aaaa", bad); err == nil {
|
||||
t.Errorf("owed act %q accepted", bad)
|
||||
}
|
||||
}
|
||||
if _, err := OwedKey("nope", 36, "aaaa", "view"); err == nil {
|
||||
t.Error("a bad repository accepted")
|
||||
}
|
||||
if _, err := OwedKey("novox/mesh-sdk", 0, "aaaa", "view"); err == nil {
|
||||
t.Error("pull request number 0 accepted")
|
||||
}
|
||||
if _, err := OwedKey("novox/mesh-sdk", 36, "", "view"); err == nil {
|
||||
t.Error("an empty head accepted")
|
||||
}
|
||||
for _, bad := range []string{"", "novox/mesh-sdk.36.aaaa", "novox/mesh-sdk.x.aaaa.view", "novox/mesh-sdk.36.aaaa.merge", "a.b.36.aaaa.view"} {
|
||||
if _, _, _, _, err := ParseOwedKey(bad); err == nil {
|
||||
t.Errorf("parse owed key %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sampleMerge() Merge {
|
||||
return Merge{Group: "feat/x", Ask: "merge.feat/x.0123456789ab", Pulls: []MergePull{
|
||||
{Head: gateHeads()[0], State: MergeMerged, Commit: "cccc"},
|
||||
{Head: gateHeads()[1], State: MergeRefused, Words: "the forge says no"},
|
||||
{Head: Head{Repository: "novox/other", Number: 1, Head: "dddd"}, State: MergeMerging},
|
||||
}}
|
||||
}
|
||||
|
||||
func sampleAsk() Ask {
|
||||
key, _ := MergeAskKey("feat/x", gateHeads())
|
||||
return Ask{Group: "feat/x", Heads: gateHeads(), Ask: asks.Ask{
|
||||
ID: AskID(key), Headline: "Merge into main: group feat/x", Explanation: "The group passed its checks.",
|
||||
Who: asks.Operator, Expires: time.Date(2026, 10, 20, 8, 0, 0, 0, time.UTC), OnExpiry: "nothing is merged",
|
||||
Options: []asks.Option{{ID: "approve", Label: "Approve", Does: "merges", Level: asks.Approve, Binds: "sha256:00"}},
|
||||
}}
|
||||
}
|
||||
|
||||
func sampleOwed() []Owed {
|
||||
return []Owed{
|
||||
{Act: OwedStatus, Context: "mesh/merge-gate", State: "success"},
|
||||
{Act: OwedView, Text: "the delivery reaches ace"},
|
||||
{Act: OwedNote, Text: "plan: mesh-mail to ace", Ref: "mesh-plan"},
|
||||
{Act: OwedComment, Text: "merged"},
|
||||
}
|
||||
}
|
||||
|
||||
func TestGateValuesRoundTrip(t *testing.T) { // 1201/AC4
|
||||
hasV := func(name string, data []byte, err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if json.Unmarshal(data, &probe) != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("%s: %s does not carry v:1", name, data)
|
||||
}
|
||||
}
|
||||
m := sampleMerge()
|
||||
data, err := EncodeMerge(m)
|
||||
hasV("merge", data, err)
|
||||
m.V = Version
|
||||
if got, err := DecodeMerge(data); err != nil || !SameMerge(got, m) {
|
||||
t.Errorf("merge round trip: %+v %v", got, err)
|
||||
}
|
||||
a := sampleAsk()
|
||||
data, err = EncodeAsk(a)
|
||||
hasV("ask", data, err)
|
||||
a.V = Version
|
||||
if got, err := DecodeAsk(data); err != nil || !SameAsk(got, a) {
|
||||
t.Errorf("ask round trip: %+v %v", got, err)
|
||||
}
|
||||
an := Answer{Option: "approve", Warrant: "w-1", Words: "approved on telegram"}
|
||||
data, err = EncodeAnswer(an)
|
||||
hasV("answer", data, err)
|
||||
an.V = Version
|
||||
if got, err := DecodeAnswer(data); err != nil || !SameAnswer(got, an) {
|
||||
t.Errorf("answer round trip: %+v %v", got, err)
|
||||
}
|
||||
for _, o := range sampleOwed() {
|
||||
data, err = EncodeOwed(o)
|
||||
hasV("owed "+o.Act, data, err)
|
||||
o.V = Version
|
||||
if got, err := DecodeOwed(data); err != nil || !SameOwed(got, o) {
|
||||
t.Errorf("owed %s round trip: %+v %v", o.Act, got, err)
|
||||
}
|
||||
}
|
||||
for name, dec := range map[string]func([]byte) error{
|
||||
"merge": func(b []byte) error { _, err := DecodeMerge(b); return err },
|
||||
"ask": func(b []byte) error { _, err := DecodeAsk(b); return err },
|
||||
"answer": func(b []byte) error { _, err := DecodeAnswer(b); return err },
|
||||
"owed": func(b []byte) error { _, err := DecodeOwed(b); return err },
|
||||
} {
|
||||
for _, in := range []string{`{"group":"g"}`, `{"v":2}`} {
|
||||
if err := dec([]byte(in)); !errors.Is(err, ErrUnknownVersion) {
|
||||
t.Errorf("%s decode of %s = %v, want ErrUnknownVersion", name, in, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGateValuesRefused(t *testing.T) { // 1201/AC5
|
||||
wantErr := func(name string, err error, words ...string) {
|
||||
t.Helper()
|
||||
if err == nil {
|
||||
t.Errorf("%s accepted", name)
|
||||
return
|
||||
}
|
||||
for _, w := range words {
|
||||
if !strings.Contains(err.Error(), w) {
|
||||
t.Errorf("%s: error %q does not name %q", name, err, w)
|
||||
}
|
||||
}
|
||||
}
|
||||
m := sampleMerge()
|
||||
m.Pulls[0].Commit = ""
|
||||
_, err := EncodeMerge(m)
|
||||
wantErr("merged without a merge commit", err, "commit")
|
||||
m = sampleMerge()
|
||||
m.Pulls[1].Words = ""
|
||||
_, err = EncodeMerge(m)
|
||||
wantErr("refused without words", err, "words")
|
||||
m = sampleMerge()
|
||||
m.Pulls[2].State = "done"
|
||||
_, err = EncodeMerge(m)
|
||||
wantErr("unknown state", err, "done")
|
||||
m = sampleMerge()
|
||||
m.Pulls = nil
|
||||
_, err = EncodeMerge(m)
|
||||
wantErr("merge without pulls", err, "pull")
|
||||
m = sampleMerge()
|
||||
m.Ask = ""
|
||||
_, err = EncodeMerge(m)
|
||||
wantErr("merge without its ask", err, "ask")
|
||||
|
||||
a := sampleAsk()
|
||||
a.Ask.ID = "something-else"
|
||||
_, err = EncodeAsk(a)
|
||||
wantErr("ask with another id", err, "something-else", AskID("merge.feat/x."+mustHash(gateHeads())))
|
||||
a = sampleAsk()
|
||||
a.Ask.Options = nil
|
||||
_, err = EncodeAsk(a)
|
||||
wantErr("ask without options", err, "option")
|
||||
a = sampleAsk()
|
||||
a.Heads = nil
|
||||
_, err = EncodeAsk(a)
|
||||
wantErr("ask without heads", err, "pull request")
|
||||
|
||||
for name, an := range map[string]Answer{
|
||||
"option": {Warrant: "w", Words: "x"},
|
||||
"warrant": {Option: "approve", Words: "x"},
|
||||
"words": {Option: "approve", Warrant: "w"},
|
||||
} {
|
||||
_, err = EncodeAnswer(an)
|
||||
wantErr("answer without "+name, err, name)
|
||||
}
|
||||
|
||||
for name, o := range map[string]Owed{
|
||||
"context": {Act: OwedStatus, State: "success"},
|
||||
"state": {Act: OwedStatus, Context: "mesh/merge-gate", State: "green"},
|
||||
"text": {Act: OwedView},
|
||||
"ref": {Act: OwedNote, Text: "line"},
|
||||
"comment": {Act: OwedComment},
|
||||
"act": {Act: "merge"},
|
||||
} {
|
||||
_, err = EncodeOwed(o)
|
||||
wantErr("owed missing "+name, err)
|
||||
}
|
||||
_, err = EncodeOwed(Owed{Act: OwedNote, Text: "line"})
|
||||
wantErr("note without ref", err, "ref")
|
||||
_, err = EncodeOwed(Owed{Act: OwedStatus, State: "success"})
|
||||
wantErr("status without context", err, "context")
|
||||
}
|
||||
|
||||
func mustHash(h []Head) string { s, _ := HeadsHash(h); return s }
|
||||
|
||||
func TestGateSame(t *testing.T) { // 1201/AC6
|
||||
if !SameMerge(sampleMerge(), sampleMerge()) || !SameAsk(sampleAsk(), sampleAsk()) {
|
||||
t.Error("equal values differ")
|
||||
}
|
||||
m := sampleMerge()
|
||||
m.Pulls[0].Commit = "other"
|
||||
if SameMerge(m, sampleMerge()) {
|
||||
t.Error("different merges are the same")
|
||||
}
|
||||
a := sampleAsk()
|
||||
a.Ask.Headline = "other"
|
||||
if SameAsk(a, sampleAsk()) {
|
||||
t.Error("different asks are the same")
|
||||
}
|
||||
if !SameAnswer(Answer{Option: "a", Warrant: "w", Words: "x"}, Answer{Option: "a", Warrant: "w", Words: "x"}) ||
|
||||
SameAnswer(Answer{Option: "a", Warrant: "w", Words: "x"}, Answer{Option: "b", Warrant: "w", Words: "x"}) {
|
||||
t.Error("SameAnswer wrong")
|
||||
}
|
||||
if !SameOwed(sampleOwed()[0], sampleOwed()[0]) || SameOwed(sampleOwed()[0], sampleOwed()[1]) {
|
||||
t.Error("SameOwed wrong")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGateBuckets(t *testing.T) {
|
||||
if MergesBucket != "mesh-delivery-gate_merges" || AsksBucket != "mesh-delivery-gate_asks" ||
|
||||
OwedBucket != "mesh-delivery-gate_owed" || AnswersBucket != "mesh-delivery-teller_answers" ||
|
||||
MergesHistory != 10 || AsksHistory != 5 || OwedHistory != 5 || AnswersHistory != 5 {
|
||||
t.Error("a bucket or history differs from the decided layout")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,420 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The modules of the parts release 3 adds (novox/hq ADR 0437): their buckets are their own state, `<module>_<name>`.
|
||||
const (
|
||||
GateModule = "mesh-delivery-gate"
|
||||
IndexerModule = "mesh-delivery-indexer"
|
||||
ForgeBridgeModule = "mesh-delivery-forge-bridge"
|
||||
TellerModule = "mesh-delivery-teller"
|
||||
)
|
||||
|
||||
// The gate's source bucket and the indexer's candidates and modules buckets (ADR 0437 §5). The source bucket has
|
||||
// one key, SourceMainKey: every repository's merged head and the group that moved it, written by the gate alone by
|
||||
// compare-and-set; its revision is the mesh's source revision. The indexer writes the other two.
|
||||
const (
|
||||
SourceBucket = GateModule + "_source"
|
||||
SourceMainKey = "main"
|
||||
SourceHistory = 64
|
||||
CandidatesBucket = IndexerModule + "_candidates"
|
||||
// CandidatesHistory is the revisions the candidates bucket keeps per key.
|
||||
CandidatesHistory = 10
|
||||
ModulesBucket = IndexerModule + "_modules"
|
||||
// ModulesHistory is the revisions the modules bucket keeps per key.
|
||||
ModulesHistory = 20
|
||||
)
|
||||
|
||||
// ForgeStream is the forge bridge's stream as its manifest declares it; the broker derives the real names.
|
||||
const ForgeStream = "forge"
|
||||
|
||||
// The kinds of forge message, the last part of a forge event's name.
|
||||
const (
|
||||
ForgePull = "pull"
|
||||
ForgeMerged = "merged"
|
||||
ForgeRefused = "refused"
|
||||
)
|
||||
|
||||
// The actions of a pull message.
|
||||
const (
|
||||
PullOpened = "opened"
|
||||
PullUpdated = "updated"
|
||||
PullClosed = "closed"
|
||||
)
|
||||
|
||||
// SourcePrefix opens every source fingerprint.
|
||||
const SourcePrefix = "src1:"
|
||||
|
||||
// CandidatesKey is the key, in CandidatesBucket, of a group: `<group>`, the branch name the group's pull requests
|
||||
// share (a slash is allowed); a lone pull request is a group of one under its branch name.
|
||||
func CandidatesKey(group string) (string, error) {
|
||||
if err := checkName("group", group); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return group, nil
|
||||
}
|
||||
|
||||
// ParseCandidatesKey is the inverse of CandidatesKey.
|
||||
func ParseCandidatesKey(key string) (group string, err error) {
|
||||
if group, err = CandidatesKey(key); err != nil {
|
||||
return "", fmt.Errorf("delivery: %q is not a candidates key: %w", key, err)
|
||||
}
|
||||
return group, nil
|
||||
}
|
||||
|
||||
// ModulesKey is the key, in ModulesBucket, of a module: `<module>`.
|
||||
func ModulesKey(module string) (string, error) {
|
||||
if err := checkName("module", module); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return module, nil
|
||||
}
|
||||
|
||||
// ParseModulesKey is the inverse of ModulesKey.
|
||||
func ParseModulesKey(key string) (module string, err error) {
|
||||
if module, err = ModulesKey(key); err != nil {
|
||||
return "", fmt.Errorf("delivery: %q is not a modules key: %w", key, err)
|
||||
}
|
||||
return module, nil
|
||||
}
|
||||
|
||||
// checkRepository refuses a repository that is not `<owner>/<name>`: exactly one slash, each part a valid name.
|
||||
func checkRepository(repository string) error {
|
||||
owner, name, ok := strings.Cut(repository, "/")
|
||||
if !ok || strings.Contains(name, "/") {
|
||||
return fmt.Errorf("delivery: the repository %q is not <owner>/<name>, with exactly one slash", repository)
|
||||
}
|
||||
if err := checkName("repository owner", owner); err != nil {
|
||||
return fmt.Errorf("delivery: the repository %q: %w", repository, err)
|
||||
}
|
||||
if err := checkName("repository name", name); err != nil {
|
||||
return fmt.Errorf("delivery: the repository %q: %w", repository, err)
|
||||
}
|
||||
if strings.Contains(owner, "/") {
|
||||
return fmt.Errorf("delivery: the repository %q is not <owner>/<name>, with exactly one slash", repository)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func checkForgeKind(kind string) error {
|
||||
switch kind {
|
||||
case ForgePull, ForgeMerged, ForgeRefused:
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("delivery: the forge kind %q is not pull, merged or refused", kind)
|
||||
}
|
||||
|
||||
// ForgeEvent is the name of a forge message of the given kind: `forge.<owner>.<name>.<kind>`.
|
||||
func ForgeEvent(repository, kind string) (string, error) {
|
||||
if err := checkRepository(repository); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := checkForgeKind(kind); err != nil {
|
||||
return "", err
|
||||
}
|
||||
owner, name, _ := strings.Cut(repository, "/")
|
||||
return "forge." + owner + "." + name + "." + kind, nil
|
||||
}
|
||||
|
||||
// ParseForgeEvent is the inverse of ForgeEvent.
|
||||
func ParseForgeEvent(event string) (repository, kind string, err error) {
|
||||
parts := strings.Split(event, ".")
|
||||
if len(parts) != 4 || parts[0] != "forge" {
|
||||
return "", "", fmt.Errorf("delivery: %q is not a forge event, forge.<owner>.<name>.<kind>", event)
|
||||
}
|
||||
repository = parts[1] + "/" + parts[2]
|
||||
if _, err := ForgeEvent(repository, parts[3]); err != nil {
|
||||
return "", "", fmt.Errorf("delivery: %q is not a forge event: %w", event, err)
|
||||
}
|
||||
return repository, parts[3], nil
|
||||
}
|
||||
|
||||
// Head is one pull request at one commit: the repository as `<owner>/<name>` and the full sha.
|
||||
type Head struct {
|
||||
Repository string `json:"repository"`
|
||||
Number int `json:"number"`
|
||||
Head string `json:"head"`
|
||||
}
|
||||
|
||||
// HeadsHash is the first 12 hex digits of the sha256 over the lines `<repository>#<number>=<head>`, sorted and
|
||||
// joined by a newline: the same for every order of heads, different when any head changes. An empty list is refused.
|
||||
func HeadsHash(heads []Head) (string, error) {
|
||||
if len(heads) == 0 {
|
||||
return "", fmt.Errorf("delivery: a heads hash needs at least one pull request")
|
||||
}
|
||||
lines := make([]string, len(heads))
|
||||
for i, h := range heads {
|
||||
lines[i] = fmt.Sprintf("%s#%d=%s", h.Repository, h.Number, h.Head)
|
||||
}
|
||||
sort.Strings(lines)
|
||||
sum := sha256.Sum256([]byte(strings.Join(lines, "\n")))
|
||||
return hex.EncodeToString(sum[:])[:12], nil
|
||||
}
|
||||
|
||||
// Source is the value of SourceMainKey: every repository's merged head (repository to full sha, not only the
|
||||
// group's) and the group that moved it.
|
||||
type Source struct {
|
||||
V int `json:"v"`
|
||||
Group string `json:"group"`
|
||||
Heads map[string]string `json:"heads"`
|
||||
}
|
||||
|
||||
func (s Source) check() error {
|
||||
if len(s.Heads) == 0 {
|
||||
return fmt.Errorf("delivery: a source names no repository")
|
||||
}
|
||||
for repository, head := range s.Heads {
|
||||
if repository == "" {
|
||||
return fmt.Errorf("delivery: a source names a repository with no name")
|
||||
}
|
||||
if head == "" {
|
||||
return fmt.Errorf("delivery: the source's head of %s is empty", repository)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Candidate is the value of a candidates key: the group's pull requests and heads, the heads hash, the base (the
|
||||
// source revision it was previewed onto) and the modules each pull request touches. It carries no fingerprint
|
||||
// until release 4 (ADR 0437 §5).
|
||||
type Candidate struct {
|
||||
V int `json:"v"`
|
||||
Group string `json:"group"`
|
||||
Base uint64 `json:"base"`
|
||||
HeadsHash string `json:"heads"`
|
||||
Pulls []CandidatePull `json:"pulls"`
|
||||
}
|
||||
|
||||
// CandidatePull is one pull request of a candidate and the modules it touches.
|
||||
type CandidatePull struct {
|
||||
Head
|
||||
Modules []string `json:"modules"`
|
||||
}
|
||||
|
||||
func (c Candidate) check() error {
|
||||
if len(c.Pulls) == 0 {
|
||||
return fmt.Errorf("delivery: a candidate names no pull request")
|
||||
}
|
||||
heads := make([]Head, len(c.Pulls))
|
||||
for i, p := range c.Pulls {
|
||||
heads[i] = p.Head
|
||||
}
|
||||
want, _ := HeadsHash(heads)
|
||||
if c.HeadsHash != want {
|
||||
return fmt.Errorf("delivery: the candidate's heads hash %q is not the hash %q of its pull requests' heads", c.HeadsHash, want)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Module is the value of a modules key: the module's source fingerprint, or none and why (a build that resolves
|
||||
// packages when built gets none, said here), and where it was computed: Repository and Commit are the trunk merge,
|
||||
// Src the source revision then (0 before the gate writes source.main). Exactly one of FP and None is set.
|
||||
type Module struct {
|
||||
V int `json:"v"`
|
||||
FP string `json:"fp,omitempty"`
|
||||
None string `json:"none,omitempty"`
|
||||
Src uint64 `json:"src,omitempty"`
|
||||
Repository string `json:"repository"`
|
||||
Commit string `json:"commit"`
|
||||
}
|
||||
|
||||
func (m Module) check() error {
|
||||
switch {
|
||||
case m.FP != "" && m.None != "":
|
||||
return fmt.Errorf("delivery: a module has both a fingerprint and a reason for none")
|
||||
case m.FP == "" && m.None == "":
|
||||
return fmt.Errorf("delivery: a module has neither a fingerprint nor a reason for none")
|
||||
case m.FP != "" && !strings.HasPrefix(m.FP, SourcePrefix):
|
||||
return fmt.Errorf("delivery: the module's fingerprint %q does not start with %s", m.FP, SourcePrefix)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Forge is a message of the forge bridge's stream, trimmed to repository, branch, pull request and head. A pull
|
||||
// message says Action; a merged message carries the merge commit, the tree of every directory holding a
|
||||
// module.json and the repository's root tree; a refused message names the owed act it answers and the forge's own
|
||||
// words. A pull message may also carry the module directories its head changes (Dirs, in the order the forge said) and
|
||||
// DirsKnown, true when the forge could say them: known and empty is DirsKnown with no Dirs, not known is neither
|
||||
// (the forge's module_dirs and module_dirs_said, mesh-catalog modules/gitea/trunk.ts). Other messages carry neither.
|
||||
type Forge struct {
|
||||
V int `json:"v"`
|
||||
Kind string `json:"kind"`
|
||||
Repository string `json:"repository"`
|
||||
Branch string `json:"branch,omitempty"`
|
||||
Number int `json:"number,omitempty"`
|
||||
Head string `json:"head,omitempty"`
|
||||
Action string `json:"action,omitempty"`
|
||||
Dirs []string `json:"dirs,omitempty"`
|
||||
DirsKnown bool `json:"dirs-known,omitempty"`
|
||||
Merge string `json:"merge,omitempty"`
|
||||
Trees map[string]string `json:"trees,omitempty"`
|
||||
Root string `json:"root,omitempty"`
|
||||
Act string `json:"act,omitempty"`
|
||||
Words string `json:"words,omitempty"`
|
||||
}
|
||||
|
||||
func (f Forge) check() error {
|
||||
if err := checkRepository(f.Repository); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkForgeKind(f.Kind); err != nil {
|
||||
return err
|
||||
}
|
||||
switch f.Kind {
|
||||
case ForgePull:
|
||||
if f.Number <= 0 {
|
||||
return fmt.Errorf("delivery: a pull message needs its pull request number")
|
||||
}
|
||||
if f.Head == "" {
|
||||
return fmt.Errorf("delivery: a pull message needs its head")
|
||||
}
|
||||
switch f.Action {
|
||||
case PullOpened, PullUpdated, PullClosed:
|
||||
default:
|
||||
return fmt.Errorf("delivery: the pull action %q is not opened, updated or closed", f.Action)
|
||||
}
|
||||
if len(f.Dirs) > 0 && !f.DirsKnown {
|
||||
return fmt.Errorf("delivery: a pull message with module directories must say it knows them (dirs-known)")
|
||||
}
|
||||
case ForgeMerged:
|
||||
if len(f.Dirs) > 0 || f.DirsKnown {
|
||||
return fmt.Errorf("delivery: a merged message carries no module directories")
|
||||
}
|
||||
if f.Number <= 0 {
|
||||
return fmt.Errorf("delivery: a merged message needs its pull request number")
|
||||
}
|
||||
if f.Head == "" {
|
||||
return fmt.Errorf("delivery: a merged message needs its head")
|
||||
}
|
||||
if f.Merge == "" {
|
||||
return fmt.Errorf("delivery: a merged message needs its merge commit")
|
||||
}
|
||||
if f.Root == "" {
|
||||
return fmt.Errorf("delivery: a merged message needs the repository's root tree")
|
||||
}
|
||||
if len(f.Trees) == 0 {
|
||||
return fmt.Errorf("delivery: a merged message needs the tree of at least one directory holding a module.json")
|
||||
}
|
||||
for dir, tree := range f.Trees {
|
||||
if tree == "" {
|
||||
return fmt.Errorf("delivery: a merged message's tree of the directory %q is empty", dir)
|
||||
}
|
||||
}
|
||||
case ForgeRefused:
|
||||
if len(f.Dirs) > 0 || f.DirsKnown {
|
||||
return fmt.Errorf("delivery: a refused message carries no module directories")
|
||||
}
|
||||
if f.Act == "" {
|
||||
return fmt.Errorf("delivery: a refused message needs the act it answers")
|
||||
}
|
||||
if f.Words == "" {
|
||||
return fmt.Errorf("delivery: a refused message needs the forge's words")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EncodeSource is s as JSON at Version; it refuses a source with no repository or an empty head.
|
||||
func EncodeSource(s Source) ([]byte, error) {
|
||||
if err := s.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.V = Version
|
||||
return json.Marshal(s)
|
||||
}
|
||||
|
||||
// DecodeSource reads a source; it refuses an unknown version (ErrUnknownVersion) and what EncodeSource refuses.
|
||||
func DecodeSource(data []byte) (Source, error) {
|
||||
var s Source
|
||||
if err := decode(data, &s); err != nil {
|
||||
return Source{}, err
|
||||
}
|
||||
if err := s.check(); err != nil {
|
||||
return Source{}, err
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// EncodeCandidate is c as JSON at Version; it refuses a heads hash that is not the hash of its pull requests' heads.
|
||||
func EncodeCandidate(c Candidate) ([]byte, error) {
|
||||
if err := c.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.V = Version
|
||||
return json.Marshal(c)
|
||||
}
|
||||
|
||||
// DecodeCandidate reads a candidate; it refuses an unknown version (ErrUnknownVersion) and what EncodeCandidate refuses.
|
||||
func DecodeCandidate(data []byte) (Candidate, error) {
|
||||
var c Candidate
|
||||
if err := decode(data, &c); err != nil {
|
||||
return Candidate{}, err
|
||||
}
|
||||
if err := c.check(); err != nil {
|
||||
return Candidate{}, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// EncodeModule is m as JSON at Version; it refuses both or neither of a fingerprint and a reason, and a
|
||||
// fingerprint not of the form src1:.
|
||||
func EncodeModule(m Module) ([]byte, error) {
|
||||
if err := m.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.V = Version
|
||||
return json.Marshal(m)
|
||||
}
|
||||
|
||||
// DecodeModule reads a module; it refuses an unknown version (ErrUnknownVersion) and what EncodeModule refuses.
|
||||
func DecodeModule(data []byte) (Module, error) {
|
||||
var m Module
|
||||
if err := decode(data, &m); err != nil {
|
||||
return Module{}, err
|
||||
}
|
||||
if err := m.check(); err != nil {
|
||||
return Module{}, err
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// EncodeForge is f as JSON at Version; it refuses a message missing what its kind needs.
|
||||
func EncodeForge(f Forge) ([]byte, error) {
|
||||
if err := f.check(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f.V = Version
|
||||
return json.Marshal(f)
|
||||
}
|
||||
|
||||
// DecodeForge reads a forge message; it refuses an unknown version (ErrUnknownVersion) and what EncodeForge refuses.
|
||||
func DecodeForge(data []byte) (Forge, error) {
|
||||
var f Forge
|
||||
if err := decode(data, &f); err != nil {
|
||||
return Forge{}, err
|
||||
}
|
||||
if err := f.check(); err != nil {
|
||||
return Forge{}, err
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// SameSource says whether two sources are the same value, so a writer writes only when they differ.
|
||||
func SameSource(a, b Source) bool { return reflect.DeepEqual(a, b) }
|
||||
|
||||
// SameCandidate says whether two candidates are the same value.
|
||||
func SameCandidate(a, b Candidate) bool { return reflect.DeepEqual(a, b) }
|
||||
|
||||
// SameModule says whether two modules are the same value.
|
||||
func SameModule(a, b Module) bool { return a == b }
|
||||
|
||||
// SameForge says whether two forge messages are the same value.
|
||||
func SameForge(a, b Forge) bool { return reflect.DeepEqual(a, b) }
|
||||
@@ -0,0 +1,340 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSourceKeys(t *testing.T) { // 1200/AC1
|
||||
if k, err := CandidatesKey("feat/delivery-source-keys"); err != nil || k != "feat/delivery-source-keys" {
|
||||
t.Fatalf("candidates key = %q, %v", k, err)
|
||||
}
|
||||
if g, err := ParseCandidatesKey("feat/x"); err != nil || g != "feat/x" {
|
||||
t.Fatalf("parse candidates = %q, %v", g, err)
|
||||
}
|
||||
if k, err := ModulesKey("mesh-mail"); err != nil || k != "mesh-mail" {
|
||||
t.Fatalf("modules key = %q, %v", k, err)
|
||||
}
|
||||
if m, err := ParseModulesKey("mesh-mail"); err != nil || m != "mesh-mail" {
|
||||
t.Fatalf("parse modules = %q, %v", m, err)
|
||||
}
|
||||
for _, bad := range []string{"", "a.b", "a*b", "a>b", "a b", "a\tb"} {
|
||||
if _, err := CandidatesKey(bad); err == nil {
|
||||
t.Errorf("candidates key %q accepted", bad)
|
||||
}
|
||||
if _, err := ModulesKey(bad); err == nil {
|
||||
t.Errorf("modules key %q accepted", bad)
|
||||
}
|
||||
if _, err := ParseCandidatesKey(bad); err == nil {
|
||||
t.Errorf("parse candidates %q accepted", bad)
|
||||
}
|
||||
if _, err := ParseModulesKey(bad); err == nil {
|
||||
t.Errorf("parse modules %q accepted", bad)
|
||||
}
|
||||
}
|
||||
ev, err := ForgeEvent("novox/mesh-sdk", ForgeMerged)
|
||||
if err != nil || ev != "forge.novox.mesh-sdk.merged" {
|
||||
t.Fatalf("forge event = %q, %v", ev, err)
|
||||
}
|
||||
repo, kind, err := ParseForgeEvent(ev)
|
||||
if err != nil || repo != "novox/mesh-sdk" || kind != ForgeMerged {
|
||||
t.Fatalf("parse forge event = %q %q %v", repo, kind, err)
|
||||
}
|
||||
for _, k := range []string{ForgePull, ForgeMerged, ForgeRefused} {
|
||||
if _, err := ForgeEvent("a/b", k); err != nil {
|
||||
t.Errorf("kind %s refused: %v", k, err)
|
||||
}
|
||||
}
|
||||
for _, bad := range []struct{ repo, kind string }{
|
||||
{"", "pull"}, {"novox", "pull"}, {"a/b/c", "pull"}, {"/b", "pull"}, {"a/", "pull"},
|
||||
{"a.b/c", "pull"}, {"a/b c", "pull"}, {"a/b", "closed"}, {"a/b", ""},
|
||||
} {
|
||||
if _, err := ForgeEvent(bad.repo, bad.kind); err == nil {
|
||||
t.Errorf("forge event %q %q accepted", bad.repo, bad.kind)
|
||||
} else if bad.kind == "closed" && !strings.Contains(err.Error(), "closed") {
|
||||
t.Errorf("error does not name the kind: %v", err)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"", "forge.a.b", "forge.a.b.closed", "x.a.b.pull", "forge.a.b.pull.x", "forge..b.pull"} {
|
||||
if _, _, err := ParseForgeEvent(bad); err == nil {
|
||||
t.Errorf("parse forge event %q accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHeadsHash(t *testing.T) { // 1200/AC2
|
||||
a := Head{Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa"}
|
||||
b := Head{Repository: "novox/mesh-control", Number: 7, Head: "bbbb"}
|
||||
h1, err := HeadsHash([]Head{a, b})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h2, _ := HeadsHash([]Head{b, a})
|
||||
if h1 != h2 {
|
||||
t.Errorf("order changes the hash: %s %s", h1, h2)
|
||||
}
|
||||
if len(h1) != 12 || strings.Trim(h1, "0123456789abcdef") != "" {
|
||||
t.Errorf("hash %q is not 12 hex digits", h1)
|
||||
}
|
||||
for _, changed := range []Head{
|
||||
{Repository: "novox/mesh-sdk", Number: 36, Head: "aaab"},
|
||||
{Repository: "novox/mesh-sdk", Number: 37, Head: "aaaa"},
|
||||
{Repository: "novox/other", Number: 36, Head: "aaaa"},
|
||||
} {
|
||||
h3, _ := HeadsHash([]Head{changed, b})
|
||||
if h3 == h1 {
|
||||
t.Errorf("changing %v keeps the hash", changed)
|
||||
}
|
||||
}
|
||||
if _, err := HeadsHash(nil); err == nil {
|
||||
t.Error("an empty list is accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func sampleSource() Source {
|
||||
return Source{Group: "feat/x", Heads: map[string]string{"novox/mesh-sdk": "abc123", "novox/mesh-control": "def456"}}
|
||||
}
|
||||
|
||||
func sampleCandidate() Candidate {
|
||||
pulls := []CandidatePull{
|
||||
{Head: Head{Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa"}, Modules: []string{"mesh-mail"}},
|
||||
{Head: Head{Repository: "novox/mesh-control", Number: 7, Head: "bbbb"}, Modules: []string{"mesh-controller"}},
|
||||
}
|
||||
hh, _ := HeadsHash([]Head{pulls[0].Head, pulls[1].Head})
|
||||
return Candidate{Group: "feat/x", Base: 12, HeadsHash: hh, Pulls: pulls}
|
||||
}
|
||||
|
||||
func sampleForges() []Forge {
|
||||
return []Forge{
|
||||
{Kind: ForgePull, Repository: "novox/mesh-sdk", Branch: "feat/x", Number: 36, Head: "aaaa", Action: "opened"},
|
||||
{Kind: ForgeMerged, Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa", Merge: "cccc",
|
||||
Trees: map[string]string{"modules/a": "t1"}, Root: "t0"},
|
||||
{Kind: ForgeRefused, Repository: "novox/mesh-sdk", Act: "novox/mesh-sdk.36.aaaa.view", Words: "403 forbidden"},
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceValuesRoundTrip(t *testing.T) { // 1200/AC3
|
||||
check := func(name string, data []byte, err error) {
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if err := json.Unmarshal(data, &probe); err != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("%s: json %s does not carry v:1", name, data)
|
||||
}
|
||||
}
|
||||
s := sampleSource()
|
||||
data, err := EncodeSource(s)
|
||||
check("source", data, err)
|
||||
s.V = Version
|
||||
if got, err := DecodeSource(data); err != nil || !SameSource(got, s) {
|
||||
t.Errorf("source round trip: %+v %v", got, err)
|
||||
}
|
||||
c := sampleCandidate()
|
||||
data, err = EncodeCandidate(c)
|
||||
check("candidate", data, err)
|
||||
c.V = Version
|
||||
if got, err := DecodeCandidate(data); err != nil || !SameCandidate(got, c) {
|
||||
t.Errorf("candidate round trip: %+v %v", got, err)
|
||||
}
|
||||
for _, m := range []Module{
|
||||
{FP: "src1:abc", Src: 5, Repository: "novox/mesh-sdk", Commit: "cccc"},
|
||||
{None: "a build that resolves packages when built", Repository: "novox/mesh-sdk", Commit: "cccc"},
|
||||
} {
|
||||
data, err = EncodeModule(m)
|
||||
check("module", data, err)
|
||||
m.V = Version
|
||||
if got, err := DecodeModule(data); err != nil || !SameModule(got, m) {
|
||||
t.Errorf("module round trip: %+v %v", got, err)
|
||||
}
|
||||
}
|
||||
for _, f := range sampleForges() {
|
||||
data, err = EncodeForge(f)
|
||||
check("forge "+f.Kind, data, err)
|
||||
f.V = Version
|
||||
if got, err := DecodeForge(data); err != nil || !SameForge(got, f) {
|
||||
t.Errorf("forge %s round trip: %+v %v", f.Kind, got, err)
|
||||
}
|
||||
}
|
||||
for name, dec := range map[string]func([]byte) error{
|
||||
"source": func(b []byte) error { _, err := DecodeSource(b); return err },
|
||||
"candidate": func(b []byte) error { _, err := DecodeCandidate(b); return err },
|
||||
"module": func(b []byte) error { _, err := DecodeModule(b); return err },
|
||||
"forge": func(b []byte) error { _, err := DecodeForge(b); return err },
|
||||
} {
|
||||
for _, in := range []string{`{"group":"g"}`, `{"v":2,"group":"g"}`} {
|
||||
if err := dec([]byte(in)); !errors.Is(err, ErrUnknownVersion) {
|
||||
t.Errorf("%s decode of %s = %v, want ErrUnknownVersion", name, in, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceValuesRefused(t *testing.T) { // 1200/AC4
|
||||
wantErr := func(name string, err error, words ...string) {
|
||||
t.Helper()
|
||||
if err == nil {
|
||||
t.Errorf("%s accepted", name)
|
||||
return
|
||||
}
|
||||
for _, w := range words {
|
||||
if !strings.Contains(err.Error(), w) {
|
||||
t.Errorf("%s: error %q does not name %q", name, err, w)
|
||||
}
|
||||
}
|
||||
}
|
||||
_, err := EncodeSource(Source{Group: "g"})
|
||||
wantErr("source without repositories", err, "repositor")
|
||||
_, err = EncodeSource(Source{Group: "g", Heads: map[string]string{"a/b": ""}})
|
||||
wantErr("source with an empty head", err, "a/b")
|
||||
_, err = EncodeSource(Source{Group: "g", Heads: map[string]string{"": "x"}})
|
||||
wantErr("source with an empty repository", err, "repositor")
|
||||
|
||||
c := sampleCandidate()
|
||||
c.HeadsHash = "000000000000"
|
||||
_, err = EncodeCandidate(c)
|
||||
wantErr("candidate with a stale hash", err, "heads hash")
|
||||
c = sampleCandidate()
|
||||
c.Pulls = nil
|
||||
_, err = EncodeCandidate(c)
|
||||
wantErr("candidate without pulls", err, "pull")
|
||||
|
||||
_, err = EncodeModule(Module{FP: "src1:x", None: "why", Repository: "a/b", Commit: "c"})
|
||||
wantErr("module with both", err, "both")
|
||||
_, err = EncodeModule(Module{Repository: "a/b", Commit: "c"})
|
||||
wantErr("module with neither", err, "fingerprint", "none")
|
||||
_, err = EncodeModule(Module{FP: "abc", Repository: "a/b", Commit: "c"})
|
||||
wantErr("module with a bad fingerprint", err, "src1:")
|
||||
|
||||
f := sampleForges()[1]
|
||||
f.Merge = ""
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("merged without merge commit", err, "merge")
|
||||
f = sampleForges()[1]
|
||||
f.Root = ""
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("merged without root", err, "root")
|
||||
f = sampleForges()[1]
|
||||
f.Trees = nil
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("merged without trees", err, "tree")
|
||||
f = sampleForges()[1]
|
||||
f.Trees = map[string]string{"modules/a": ""}
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("merged with an empty tree", err, "modules/a")
|
||||
f = sampleForges()[0]
|
||||
f.Action = "reopened"
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("pull with an unknown action", err, "reopened")
|
||||
f = sampleForges()[0]
|
||||
f.Head = ""
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("pull without head", err, "head")
|
||||
f = sampleForges()[2]
|
||||
f.Words = ""
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("refused without words", err, "words")
|
||||
f = sampleForges()[2]
|
||||
f.Act = ""
|
||||
_, err = EncodeForge(f)
|
||||
wantErr("refused without act", err, "act")
|
||||
_, err = EncodeForge(Forge{Kind: "closed", Repository: "a/b"})
|
||||
wantErr("unknown kind", err, "closed")
|
||||
_, err = EncodeForge(Forge{Kind: ForgePull, Repository: "nope", Number: 1, Head: "h", Action: "opened"})
|
||||
wantErr("bad repository", err, "nope")
|
||||
}
|
||||
|
||||
func TestSourceSame(t *testing.T) { // 1200/AC5
|
||||
if !SameSource(sampleSource(), sampleSource()) {
|
||||
t.Error("equal sources differ")
|
||||
}
|
||||
s := sampleSource()
|
||||
s.Heads["novox/mesh-sdk"] = "zzz"
|
||||
if SameSource(s, sampleSource()) {
|
||||
t.Error("different sources are the same")
|
||||
}
|
||||
if !SameCandidate(sampleCandidate(), sampleCandidate()) {
|
||||
t.Error("equal candidates differ")
|
||||
}
|
||||
c := sampleCandidate()
|
||||
c.Pulls[0].Modules = []string{"other"}
|
||||
if SameCandidate(c, sampleCandidate()) {
|
||||
t.Error("different candidates are the same")
|
||||
}
|
||||
m := Module{FP: "src1:a", Repository: "a/b", Commit: "c"}
|
||||
if !SameModule(m, m) || SameModule(m, Module{FP: "src1:b", Repository: "a/b", Commit: "c"}) {
|
||||
t.Error("SameModule wrong")
|
||||
}
|
||||
f := sampleForges()[1]
|
||||
g := sampleForges()[1]
|
||||
if !SameForge(f, g) {
|
||||
t.Error("equal forge messages differ")
|
||||
}
|
||||
g.Trees["modules/a"] = "other"
|
||||
if SameForge(f, g) {
|
||||
t.Error("different forge messages are the same")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceBuckets(t *testing.T) {
|
||||
if SourceBucket != "mesh-delivery-gate_source" || CandidatesBucket != "mesh-delivery-indexer_candidates" ||
|
||||
ModulesBucket != "mesh-delivery-indexer_modules" || SourceMainKey != "main" ||
|
||||
SourceHistory != 64 || CandidatesHistory != 10 || ModulesHistory != 20 {
|
||||
t.Error("a bucket or history differs from the decided layout")
|
||||
}
|
||||
}
|
||||
|
||||
func TestForgeDirs(t *testing.T) {
|
||||
pull := func(dirs []string, known bool) Forge {
|
||||
return Forge{Kind: ForgePull, Repository: "novox/mesh-sdk", Number: 36, Head: "aaaa", Action: "updated", Dirs: dirs, DirsKnown: known}
|
||||
}
|
||||
round := func(f Forge) Forge {
|
||||
t.Helper()
|
||||
data, err := EncodeForge(f)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := DecodeForge(data)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return got
|
||||
}
|
||||
// 1233/AC2: the directories come back equal and in order.
|
||||
got := round(pull([]string{"modules/a", "modules/b"}, true))
|
||||
if !reflect.DeepEqual(got.Dirs, []string{"modules/a", "modules/b"}) || !got.DirsKnown {
|
||||
t.Errorf("dirs = %v known %v", got.Dirs, got.DirsKnown)
|
||||
}
|
||||
// 1233/AC3: a head changing no module directory is told apart from one whose directories could not be said.
|
||||
none, unknown := round(pull(nil, true)), round(pull(nil, false))
|
||||
if !none.DirsKnown || len(none.Dirs) != 0 {
|
||||
t.Errorf("known and empty: dirs %v known %v", none.Dirs, none.DirsKnown)
|
||||
}
|
||||
if unknown.DirsKnown || len(unknown.Dirs) != 0 {
|
||||
t.Errorf("not known: dirs %v known %v", unknown.Dirs, unknown.DirsKnown)
|
||||
}
|
||||
// 1233/AC4: a merged or a refused message carrying directories is refused.
|
||||
m := sampleForges()[1]
|
||||
m.Dirs, m.DirsKnown = []string{"modules/a"}, true
|
||||
if _, err := EncodeForge(m); err == nil {
|
||||
t.Error("merged message with dirs accepted")
|
||||
}
|
||||
r := sampleForges()[2]
|
||||
r.Dirs, r.DirsKnown = []string{"modules/a"}, true
|
||||
if _, err := EncodeForge(r); err == nil {
|
||||
t.Error("refused message with dirs accepted")
|
||||
}
|
||||
r = sampleForges()[2]
|
||||
r.DirsKnown = true
|
||||
if _, err := EncodeForge(r); err == nil {
|
||||
t.Error("refused message with dirs-known accepted")
|
||||
}
|
||||
// directories on a pull message that does not say it knows them are refused too.
|
||||
if _, err := EncodeForge(pull([]string{"modules/a"}, false)); err == nil {
|
||||
t.Error("dirs without dirs-known accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// WatchmanModule is the watchman's module: its buckets are its own state, `<module>_<name>`, and the controller's
|
||||
// broker grants it alone the bus's connect and disconnect events (novox/hq ADR 0431).
|
||||
const WatchmanModule = "mesh-delivery-watchman"
|
||||
|
||||
// StuckBucket holds what stopped moving (novox/hq ADR 0433 §2): one key per key whose pulse expired, written by the
|
||||
// watchman, read by the controller's self-check, which raises each as a condition until it says moved. No key of it
|
||||
// is removed: a key that moves again is written as moved.
|
||||
const StuckBucket = WatchmanModule + "_stuck"
|
||||
|
||||
// StuckKey is the key, in StuckBucket, of the key that stopped moving: `<bucket>.<key>`, formed as PulseKey forms a
|
||||
// pulse's, so a node or part gone offline is `mesh-delivery-watchman_presence.<presence key>`.
|
||||
func StuckKey(bucket, key string) (string, error) {
|
||||
stuck, err := PulseKey(bucket, key)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("delivery: no stuck key: %w", err)
|
||||
}
|
||||
return stuck, nil
|
||||
}
|
||||
|
||||
// ParseStuckKey is the inverse of StuckKey, splitting at the first dot.
|
||||
func ParseStuckKey(stuckKey string) (bucket, key string, err error) {
|
||||
bucket, key, err = ParsePulseKey(stuckKey)
|
||||
if err != nil {
|
||||
return "", "", fmt.Errorf("delivery: %q is not a stuck key: %w", stuckKey, err)
|
||||
}
|
||||
return bucket, key, nil
|
||||
}
|
||||
|
||||
// Stuck is the value of a stuck key: the key that stopped moving, what it waited for in words, since when, and
|
||||
// whether it has moved again since. The watchman writes it; the controller reads it.
|
||||
type Stuck struct {
|
||||
V int `json:"v"`
|
||||
Bucket string `json:"bucket"`
|
||||
Key string `json:"key"`
|
||||
Waited string `json:"waited,omitempty"`
|
||||
Since time.Time `json:"since"`
|
||||
Moved bool `json:"moved,omitempty"`
|
||||
}
|
||||
|
||||
// EncodeStuck is s as JSON at Version.
|
||||
func EncodeStuck(s Stuck) ([]byte, error) {
|
||||
s.V = Version
|
||||
return json.Marshal(s)
|
||||
}
|
||||
|
||||
// DecodeStuck reads a stuck value; it refuses an unknown version (ErrUnknownVersion).
|
||||
func DecodeStuck(data []byte) (Stuck, error) {
|
||||
var s Stuck
|
||||
err := decode(data, &s)
|
||||
return s, err
|
||||
}
|
||||
|
||||
// SameStuck says whether two stuck values are the same, so a writer writes only when they differ.
|
||||
func SameStuck(a, b Stuck) bool {
|
||||
return a.V == b.V && a.Bucket == b.Bucket && a.Key == b.Key && a.Waited == b.Waited && a.Since.Equal(b.Since) &&
|
||||
a.Moved == b.Moved
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package delivery
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// 1153/AC4: the watchman's module and its three buckets, each that module's state.
|
||||
func TestWatchmanBuckets(t *testing.T) {
|
||||
if WatchmanModule != "mesh-delivery-watchman" {
|
||||
t.Fatalf("WatchmanModule = %q", WatchmanModule)
|
||||
}
|
||||
for name, bucket := range map[string]string{"presence": PresenceBucket, "pulse": PulseBucket, "stuck": StuckBucket} {
|
||||
if bucket != WatchmanModule+"_"+name {
|
||||
t.Errorf("the %s bucket is %q, not %s_%s", name, bucket, WatchmanModule, name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1153/AC1: the stuck key is the stopped key's bucket and key, split at the first dot, refused as a pulse key is.
|
||||
func TestStuckKey(t *testing.T) {
|
||||
key, err := StuckKey(PresenceBucket, "ace")
|
||||
if err != nil || key != "mesh-delivery-watchman_presence.ace" {
|
||||
t.Fatalf("StuckKey = %q, %v", key, err)
|
||||
}
|
||||
key, err = StuckKey("mesh-controller_builds", "zsh.build.b-1")
|
||||
if err != nil || key != "mesh-controller_builds.zsh.build.b-1" {
|
||||
t.Fatalf("StuckKey = %q, %v", key, err)
|
||||
}
|
||||
bucket, rest, err := ParseStuckKey(key)
|
||||
if err != nil || bucket != "mesh-controller_builds" || rest != "zsh.build.b-1" {
|
||||
t.Fatalf("ParseStuckKey = %q %q %v", bucket, rest, err)
|
||||
}
|
||||
for _, c := range [][2]string{{"", "k"}, {"a.b", "k"}, {"b", ""}, {"b", "k x"}, {"b", ".k"}, {"b", "k."}, {"b", "k>"}} {
|
||||
if _, err := StuckKey(c[0], c[1]); err == nil {
|
||||
t.Errorf("StuckKey(%q, %q) accepted", c[0], c[1])
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"", "nodot", ".k", "b."} {
|
||||
if _, _, err := ParseStuckKey(bad); err == nil {
|
||||
t.Errorf("ParseStuckKey(%q) accepted", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1153/AC2: a stuck value round-trips with "v":1, and an unknown or missing version is refused loudly.
|
||||
func TestStuckValue(t *testing.T) {
|
||||
s := Stuck{Bucket: PresenceBucket, Key: "ace", Waited: "ace to come back online",
|
||||
Since: time.Date(2026, 10, 18, 20, 0, 0, 0, time.UTC)}
|
||||
raw, err := EncodeStuck(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var probe map[string]any
|
||||
if err := json.Unmarshal(raw, &probe); err != nil || probe["v"] != float64(1) {
|
||||
t.Fatalf("encoded %s: v = %v (%v)", raw, probe["v"], err)
|
||||
}
|
||||
back, err := DecodeStuck(raw)
|
||||
s.V = Version
|
||||
if err != nil || !SameStuck(back, s) {
|
||||
t.Fatalf("decoded %+v, %v; want %+v", back, err, s)
|
||||
}
|
||||
for _, bad := range []string{`{"v":2,"bucket":"b","key":"k"}`, `{"bucket":"b","key":"k"}`} {
|
||||
_, err := DecodeStuck([]byte(bad))
|
||||
if !errors.Is(err, ErrUnknownVersion) {
|
||||
t.Errorf("DecodeStuck(%s) = %v, want ErrUnknownVersion", bad, err)
|
||||
}
|
||||
if strings.Contains(bad, `"v":2`) && (err == nil || !strings.Contains(err.Error(), "2") || !strings.Contains(err.Error(), "1")) {
|
||||
t.Errorf("DecodeStuck(%s) = %v: does not name the version seen and the one read", bad, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1153/AC3: equal stuck values are the same; one moved is not.
|
||||
func TestSameStuck(t *testing.T) {
|
||||
a := Stuck{V: Version, Bucket: "b", Key: "k", Waited: "w", Since: time.Unix(100, 0).UTC()}
|
||||
b := a
|
||||
if !SameStuck(a, b) {
|
||||
t.Fatal("equal values are not the same")
|
||||
}
|
||||
b.Moved = true
|
||||
if SameStuck(a, b) {
|
||||
t.Fatal("a moved value is the same as a stuck one")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"id":"build-1792395447088936520","state":"built","repository":"novox/mesh-controller","ref":"main","module":"build-agent","commit":"7a74cee83f88e6585a5511c3cc53bdf0f5303d20","path":"modules/build-agent","manifest-digest":"sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","source-fingerprint":"src1:e846efe73716b78e80860baaac96153de2cb76551325753838ca245d2160a3d7","source-inputs":{"Module":"build-agent","Tree":"11c0b4f90cae0d6e4ef9cc48cbfdf9f6dbe52bca","Bases":["sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bc","sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c"],"Contexts":{"server":"narrow:d51ca744fd11af59d616c858f76c27b3e58e16e11fbdc0c720eb9d7d19d55fac"},"Toolchains":null,"Unpinned":""},"made":[{"name":"server","kind":"image","reference":"registry.example/build-agent@sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bc"}],"at":"2026-10-19T09:38:00Z"}
|
||||
@@ -0,0 +1,110 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// emitRuntime drives a bundle over pipes as the runtime does, reading every line it says.
|
||||
type emitRuntime struct {
|
||||
t *testing.T
|
||||
lines *bufio.Scanner
|
||||
in io.Writer
|
||||
}
|
||||
|
||||
func (r *emitRuntime) next() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if r.lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(r.lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
r.t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func serveEmit(t *testing.T, tools []Tool) *emitRuntime {
|
||||
t.Helper()
|
||||
t.Setenv(ServedAs, "chime")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() { _ = ServeOn("", tools, inR, outW) }()
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
return &emitRuntime{t: t, lines: bufio.NewScanner(outR), in: inW}
|
||||
}
|
||||
|
||||
func (r *emitRuntime) call(id int, tool string) {
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": id, "method": "tools/call",
|
||||
"params": map[string]any{"name": tool, "arguments": map[string]any{}}})
|
||||
_, _ = r.in.Write(append(b, '\n'))
|
||||
}
|
||||
|
||||
func (r *emitRuntime) answer(id any) {
|
||||
b, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": id, "result": map[string]any{}})
|
||||
_, _ = r.in.Write(append(b, '\n'))
|
||||
}
|
||||
|
||||
// eventIDOf is a "mesh/publish" ask's x-event-id header.
|
||||
func eventIDOf(asked map[string]any) string {
|
||||
return asked["params"].(map[string]any)["headers"].(map[string]any)["x-event-id"].(string)
|
||||
}
|
||||
|
||||
// 1187/AC1: given EmitOptions with a message id, when a bundle emits, then the event's x-event-id header is
|
||||
// that id; and a random one when none is given, as on main.
|
||||
func TestEmitSendsTheChosenMessageIDAsXEventID(t *testing.T) {
|
||||
rt := serveEmit(t, []Tool{
|
||||
{Name: "chosen", Run: func(map[string]any) (any, error) {
|
||||
return nil, Emit("sent", map[string]any{}, EmitOptions{MessageID: "send-42"})
|
||||
}},
|
||||
{Name: "unchosen", Run: func(map[string]any) (any, error) {
|
||||
return nil, Emit("sent", map[string]any{})
|
||||
}},
|
||||
})
|
||||
rt.call(1, "chosen")
|
||||
asked := rt.next()
|
||||
if got := eventIDOf(asked); got != "send-42" {
|
||||
t.Fatalf("x-event-id: %q, want the chosen id %q", got, "send-42")
|
||||
}
|
||||
rt.answer(asked["id"])
|
||||
rt.next() // the tool's own answer
|
||||
|
||||
rt.call(2, "unchosen")
|
||||
asked = rt.next()
|
||||
if got := eventIDOf(asked); len(got) != 36 {
|
||||
t.Fatalf("x-event-id with no message id chosen: %q, want a random UUID as on main", got)
|
||||
}
|
||||
rt.answer(asked["id"])
|
||||
rt.next()
|
||||
}
|
||||
|
||||
// 1187/AC2: given two emits with the same chosen id, when sent through the runtime's protocol stand-in,
|
||||
// then both carry the same x-event-id, which the tool runner gives the bus as Nats-Msg-Id (mesh-tools
|
||||
// node-tools/internal/bus/bus.go ~748-773, :770): a repeated publish is kept to one by the bus's dedup.
|
||||
func TestEmitRepeatedWithTheSameChosenIDCarriesTheSameXEventIDTwice(t *testing.T) {
|
||||
rt := serveEmit(t, []Tool{
|
||||
{Name: "chosen", Run: func(map[string]any) (any, error) {
|
||||
return nil, Emit("sent", map[string]any{}, EmitOptions{MessageID: "send-77"})
|
||||
}},
|
||||
})
|
||||
var ids []string
|
||||
for i := 1; i <= 2; i++ {
|
||||
rt.call(i, "chosen")
|
||||
asked := rt.next()
|
||||
ids = append(ids, eventIDOf(asked))
|
||||
rt.answer(asked["id"])
|
||||
rt.next()
|
||||
}
|
||||
if ids[0] != "send-77" || ids[1] != "send-77" {
|
||||
t.Fatalf("the two emits' x-event-id: %v, want %q both times", ids, "send-77")
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,21 @@
|
||||
module git.novox.be/novox/mesh-sdk/go
|
||||
|
||||
go 1.22
|
||||
go 1.26.0
|
||||
|
||||
require (
|
||||
github.com/nats-io/nats-server/v2 v2.11.17
|
||||
github.com/nats-io/nats.go v1.54.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
|
||||
github.com/google/go-tpm v0.9.8 // indirect
|
||||
github.com/klauspost/compress v1.20.0 // indirect
|
||||
github.com/minio/highwayhash v1.0.4 // indirect
|
||||
github.com/nats-io/jwt/v2 v2.8.1 // indirect
|
||||
github.com/nats-io/nkeys v0.4.16 // indirect
|
||||
github.com/nats-io/nuid v1.0.1 // indirect
|
||||
golang.org/x/crypto v0.57.0 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/time v0.15.0 // indirect
|
||||
)
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
|
||||
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
|
||||
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
|
||||
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
|
||||
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
|
||||
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
|
||||
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
|
||||
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
|
||||
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
|
||||
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
|
||||
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
|
||||
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
|
||||
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
|
||||
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
|
||||
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
|
||||
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
|
||||
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
|
||||
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
|
||||
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
||||
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
||||
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
||||
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||
@@ -0,0 +1,191 @@
|
||||
// Package health is the answer a module's health tool gives, and how the node-engine reads it (novox/hq ADR 0240
|
||||
// rule 2, ADR 0283).
|
||||
//
|
||||
// A health tool answers one of three:
|
||||
//
|
||||
// - healthy: everything the module runs works;
|
||||
// - unhealthy: something is wrong, said in Why;
|
||||
// - waits: nothing is wrong but a part that cannot work until the operator gives something — one of the module's
|
||||
// own secrets, or one of its settings. Each Wait names exactly one of them, the part, and what the operator
|
||||
// gives, in words with no path, address or value.
|
||||
//
|
||||
// The answer keeps `healthy` beside `state`, so a node-engine older than ADR 0283 reads a wait as not healthy, as
|
||||
// it did before. An answer without `state` is read from `healthy`.
|
||||
//
|
||||
// The module writes its answer with Healthy, Unhealthy or Waiting, and the node-engine reads it with Read. Read
|
||||
// never trusts a shape it cannot check: a wait it cannot read makes the answer unhealthy, saying why.
|
||||
package health
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The three states an answer says.
|
||||
const (
|
||||
StateHealthy = "healthy"
|
||||
StateUnhealthy = "unhealthy"
|
||||
StateWaits = "waits"
|
||||
)
|
||||
|
||||
// Answer is a health tool's answer.
|
||||
type Answer struct {
|
||||
// State is healthy, unhealthy or waits.
|
||||
State string `json:"state"`
|
||||
// Healthy is State == healthy, kept for a reader older than State.
|
||||
Healthy bool `json:"healthy"`
|
||||
// Why is the answer in one or more plain sentences.
|
||||
Why string `json:"why"`
|
||||
// Waits is what the module waits for, when State is waits; and, beside a fault, what it also waits for.
|
||||
Waits []Wait `json:"waits,omitempty"`
|
||||
}
|
||||
|
||||
// Wait is one thing a module waits for the operator to give: exactly one of Secret and Setting.
|
||||
type Wait struct {
|
||||
// Part is what waits, in words: "the source games".
|
||||
Part string `json:"part"`
|
||||
// Secret is one of the module's own secrets, by name (a member of a secret family by its full name).
|
||||
Secret string `json:"secret,omitempty"`
|
||||
// Setting is one of the module's settings, by name.
|
||||
Setting string `json:"setting,omitempty"`
|
||||
// What is what the operator gives, in words: "the password of the source games".
|
||||
What string `json:"what"`
|
||||
}
|
||||
|
||||
// Healthy is a healthy answer.
|
||||
func Healthy(why string) Answer {
|
||||
return Answer{State: StateHealthy, Healthy: true, Why: why}
|
||||
}
|
||||
|
||||
// Unhealthy is an unhealthy answer. Waits may be given beside the fault: they are said, and excuse nothing.
|
||||
func Unhealthy(why string, waits ...Wait) Answer {
|
||||
return Answer{State: StateUnhealthy, Why: why, Waits: waits}
|
||||
}
|
||||
|
||||
// Waiting is an answer that says nothing is wrong but what the operator has yet to give. With no wait it is
|
||||
// unhealthy: a module that waits names what for.
|
||||
func Waiting(why string, waits ...Wait) Answer {
|
||||
if len(waits) == 0 {
|
||||
return Unhealthy(orSaid(why, "it says it waits") + " (and names nothing it waits for)")
|
||||
}
|
||||
for _, w := range waits {
|
||||
if err := w.Check(); err != nil {
|
||||
return Unhealthy(orSaid(why, "it says it waits") + " (" + err.Error() + ")")
|
||||
}
|
||||
}
|
||||
return Answer{State: StateWaits, Why: why, Waits: waits}
|
||||
}
|
||||
|
||||
// name is a secret's or a setting's name: what a manifest may declare.
|
||||
var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
|
||||
|
||||
// looksLikeAddress finds what the operator's words never carry: a URL, an IPv4 or IPv6 address, a host and port, a
|
||||
// path (absolute, from a home or relative, on its own or after a bracket or a quote), a UNC path or a drive path.
|
||||
// It cannot find a value typed in plain words ("the password hunter2"): a module never puts one there, and the
|
||||
// words are its own, not the operator's.
|
||||
var looksLikeAddress = regexp.MustCompile(strings.Join([]string{
|
||||
`://`,
|
||||
`\b[0-9]{1,3}(\.[0-9]{1,3}){3}\b`,
|
||||
`[0-9A-Fa-f]{0,4}::[0-9A-Fa-f]{0,4}`,
|
||||
`\b[0-9A-Fa-f]{1,4}(:[0-9A-Fa-f]{1,4}){3,}\b`,
|
||||
`\b[A-Za-z0-9.-]+:[0-9]{1,5}\b`,
|
||||
`(^|[\s(\[{"'\x60])[~.]{0,2}/`,
|
||||
`\\\\`,
|
||||
`\b[A-Za-z]:\\`,
|
||||
}, "|"))
|
||||
|
||||
// Check says why a wait cannot be read, or nil.
|
||||
func (w Wait) Check() error {
|
||||
switch {
|
||||
case w.Secret != "" && w.Setting != "":
|
||||
return fmt.Errorf("a wait names both the secret %q and the setting %q; it names one", w.Secret, w.Setting)
|
||||
case w.Secret == "" && w.Setting == "":
|
||||
return fmt.Errorf("a wait names neither a secret nor a setting")
|
||||
case w.Secret != "" && !name.MatchString(w.Secret):
|
||||
return fmt.Errorf("a wait names the secret %q, which is not a name", w.Secret)
|
||||
case w.Setting != "" && !name.MatchString(w.Setting):
|
||||
return fmt.Errorf("a wait names the setting %q, which is not a name", w.Setting)
|
||||
case strings.TrimSpace(w.What) == "":
|
||||
return fmt.Errorf("a wait for %s does not say what the operator gives", w.Names())
|
||||
case strings.TrimSpace(w.Part) == "":
|
||||
return fmt.Errorf("a wait for %s does not say which part waits", w.Names())
|
||||
}
|
||||
for _, words := range []string{w.What, w.Part} {
|
||||
if strings.ContainsAny(words, "\n\r") {
|
||||
return fmt.Errorf("a wait for %s says more than one line", w.Names())
|
||||
}
|
||||
if looksLikeAddress.MatchString(words) {
|
||||
return fmt.Errorf("a wait for %s says a path or an address, which the operator's words never carry", w.Names())
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Names is what the wait names, as "the secret x" or "the setting y".
|
||||
func (w Wait) Names() string {
|
||||
if w.Secret != "" {
|
||||
return "the secret " + w.Secret
|
||||
}
|
||||
if w.Setting != "" {
|
||||
return "the setting " + w.Setting
|
||||
}
|
||||
return "nothing"
|
||||
}
|
||||
|
||||
// Read reads a health tool's result. An answer it cannot read at all is an error; an answer whose wait it cannot
|
||||
// read is unhealthy, saying why, so a malformed wait never excuses anything.
|
||||
func Read(result []byte) (Answer, error) {
|
||||
var a Answer
|
||||
if err := json.Unmarshal(result, &a); err != nil {
|
||||
return Answer{}, fmt.Errorf("it answered something other than healthy or not: %w", err)
|
||||
}
|
||||
switch a.State {
|
||||
case "":
|
||||
if a.Healthy {
|
||||
a.State = StateHealthy
|
||||
} else {
|
||||
a.State = StateUnhealthy
|
||||
}
|
||||
case StateHealthy:
|
||||
if !a.Healthy {
|
||||
return Unhealthy(fmt.Sprintf("it answered the state healthy and healthy false, which contradict each other (%s)",
|
||||
orSaid(a.Why, "and said nothing more"))), nil
|
||||
}
|
||||
// A healthy module waits for nothing: a wait beside healthy is a contradiction, and is dropped.
|
||||
a.Waits = nil
|
||||
case StateUnhealthy:
|
||||
a.Healthy = false
|
||||
case StateWaits:
|
||||
a.Healthy = false
|
||||
if len(a.Waits) == 0 {
|
||||
return Unhealthy(orSaid(a.Why, "it says it waits") + " (and names nothing it waits for)"), nil
|
||||
}
|
||||
for _, w := range a.Waits {
|
||||
if err := w.Check(); err != nil {
|
||||
return Unhealthy(orSaid(a.Why, "it says it waits") + " (" + err.Error() + ")"), nil
|
||||
}
|
||||
}
|
||||
default:
|
||||
return Unhealthy(fmt.Sprintf("it answered the state %q, which is none of healthy, unhealthy or waits", a.State)), nil
|
||||
}
|
||||
if a.State != StateWaits {
|
||||
// Waits beside a fault are said and excuse nothing; one that cannot be read is dropped, not believed.
|
||||
var kept []Wait
|
||||
for _, w := range a.Waits {
|
||||
if w.Check() == nil {
|
||||
kept = append(kept, w)
|
||||
}
|
||||
}
|
||||
a.Waits = kept
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
func orSaid(s, otherwise string) string {
|
||||
if strings.TrimSpace(s) == "" {
|
||||
return otherwise
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package health
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func roundTrip(t *testing.T, a Answer) Answer {
|
||||
t.Helper()
|
||||
b, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := Read(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
var password = Wait{Part: "the source games", Secret: "smb-password-games", What: "the password of the source games"}
|
||||
|
||||
func TestEachConstructorSaysItsStateAndKeepsHealthyForAnOlderReader(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
a Answer
|
||||
state string
|
||||
healthy bool
|
||||
}{
|
||||
{Healthy("all armed"), StateHealthy, true},
|
||||
{Unhealthy("the share is unreachable"), StateUnhealthy, false},
|
||||
{Waiting("the source games waits for its password", password), StateWaits, false},
|
||||
} {
|
||||
b, _ := json.Marshal(c.a)
|
||||
var old struct {
|
||||
Healthy bool `json:"healthy"`
|
||||
Why string `json:"why"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &old); err != nil || old.Healthy != c.healthy || old.Why == "" {
|
||||
t.Fatalf("an older reader of %s reads %+v (%v)", b, old, err)
|
||||
}
|
||||
if got := roundTrip(t, c.a); got.State != c.state || got.Healthy != c.healthy {
|
||||
t.Fatalf("%s read as %+v", b, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWaitIsReadWithWhatItNames(t *testing.T) {
|
||||
got := roundTrip(t, Waiting("waits", password, Wait{Part: "the source games", Setting: "smb-users", What: "the username of the source games"}))
|
||||
if got.State != StateWaits || len(got.Waits) != 2 || got.Waits[0].Secret != "smb-password-games" || got.Waits[1].Setting != "smb-users" {
|
||||
t.Fatalf("read as %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAnswerWithoutStateIsReadFromHealthy(t *testing.T) {
|
||||
for raw, want := range map[string]string{`{"healthy":true,"why":"ok"}`: StateHealthy, `{"healthy":false,"why":"down"}`: StateUnhealthy} {
|
||||
got, err := Read([]byte(raw))
|
||||
if err != nil || got.State != want {
|
||||
t.Fatalf("%s read as %+v, %v", raw, got, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWaitThatCannotBeReadIsUnhealthyAndSaysWhy(t *testing.T) {
|
||||
for name, c := range map[string]struct {
|
||||
w []Wait
|
||||
says string
|
||||
}{
|
||||
"no wait": {nil, "names nothing"},
|
||||
"neither": {[]Wait{{Part: "p", What: "w"}}, "neither a secret nor a setting"},
|
||||
"both": {[]Wait{{Part: "p", Secret: "a", Setting: "b", What: "w"}}, "both"},
|
||||
"not a name": {[]Wait{{Part: "p", Secret: "Smb Password", What: "w"}}, "not a name"},
|
||||
"no what": {[]Wait{{Part: "p", Secret: "a"}}, "does not say what"},
|
||||
"no part": {[]Wait{{Secret: "a", What: "w"}}, "which part"},
|
||||
"two lines": {[]Wait{{Part: "p", Secret: "a", What: "the password\nhunter2"}}, "more than one line"},
|
||||
"an address": {[]Wait{{Part: "p", Secret: "a", What: "the password of 192.0.2.76"}}, "path or an address"},
|
||||
"a url": {[]Wait{{Part: "p", Secret: "a", What: "the password of smb://host/share"}}, "path or an address"},
|
||||
"a path": {[]Wait{{Part: "the source at /mnt/games", Secret: "a", What: "the password"}}, "path or an address"},
|
||||
} {
|
||||
b, _ := json.Marshal(Answer{State: StateWaits, Why: "it waits", Waits: c.w})
|
||||
got, err := Read(b)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
if got.State != StateUnhealthy || got.Healthy || !strings.Contains(got.Why, c.says) {
|
||||
t.Fatalf("%s: read as %+v, want unhealthy saying %q", name, got, c.says)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnUnknownStateIsUnhealthy(t *testing.T) {
|
||||
got, err := Read([]byte(`{"state":"fine","healthy":true}`))
|
||||
if err != nil || got.State != StateUnhealthy || got.Healthy {
|
||||
t.Fatalf("read as %+v, %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWaitsBesideAFaultAreSaidAndExcuseNothing(t *testing.T) {
|
||||
got := roundTrip(t, Unhealthy("the share is unreachable", password, Wait{Part: "p", What: "w"}))
|
||||
if got.State != StateUnhealthy || len(got.Waits) != 1 {
|
||||
t.Fatalf("read as %+v: the readable wait is kept, the other dropped, the state stays unhealthy", got)
|
||||
}
|
||||
if h := roundTrip(t, Answer{State: StateHealthy, Healthy: true, Why: "ok", Waits: []Wait{password}}); len(h.Waits) != 0 {
|
||||
t.Fatalf("a healthy answer kept its waits: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWaitingWithNothingIsUnhealthy(t *testing.T) {
|
||||
if a := Waiting("it waits"); a.State != StateUnhealthy {
|
||||
t.Fatalf("Waiting with no wait is %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNotAnAnswerIsAnError(t *testing.T) {
|
||||
if _, err := Read([]byte(`"yes"`)); err == nil {
|
||||
t.Fatal("a string read as an answer")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryFormOfPathOrAddressIsRefused(t *testing.T) {
|
||||
for _, words := range []string{
|
||||
"the source (/mnt/games)", `the source "/mnt/games"`, "the folder ~/games", "the folder ./games",
|
||||
"2001:db8::1", "fe80::1%eth0", "2001:0db8:0000:0000:0000:0000:0000:0001", "nas.lan:445", "192.0.2.7:445",
|
||||
`\\nas\games`, `C:\Users\x`, "smb://nas/games", "198.51.100.4",
|
||||
} {
|
||||
w := Wait{Part: "the source games", Secret: "smb-password-games", What: "the password of " + words}
|
||||
if err := w.Check(); err == nil {
|
||||
t.Errorf("%q passed as words for the operator", words)
|
||||
}
|
||||
}
|
||||
for _, words := range []string{"the password of the source games", "the username of the source games-2", "the API key, issued by the vendor"} {
|
||||
w := Wait{Part: "the source games", Secret: "smb-password-games", What: words}
|
||||
if err := w.Check(); err != nil {
|
||||
t.Errorf("%q refused: %v", words, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthyStateWithHealthyFalseIsUnhealthy(t *testing.T) {
|
||||
got, err := Read([]byte(`{"state":"healthy","healthy":false,"why":"x"}`))
|
||||
if err != nil || got.State != StateUnhealthy || !strings.Contains(got.Why, "contradict") {
|
||||
t.Fatalf("read as %+v, %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWaitingChecksItsWaits(t *testing.T) {
|
||||
if a := Waiting("it waits", Wait{Part: "p", What: "w"}); a.State != StateUnhealthy || !strings.Contains(a.Why, "neither") {
|
||||
t.Fatalf("Waiting with a malformed wait is %+v", a)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
// Package testbus is the bus an integration test of the SDK's Go binding runs against: a server of
|
||||
// its own, of the release go.mod pins, started for the one test and gone after it.
|
||||
//
|
||||
// **Why one per test.** A test that reaches a shared bus asserts, reads and removes objects by fixed
|
||||
// name — a stream, a consumer — and two tests at once would step on each other's: one deleting what
|
||||
// the other is reading, one leaving a stream behind for the next to trip on. A server per test makes
|
||||
// every test start from nothing and leave nothing, so the suite is the same run in any order, at any
|
||||
// parallelism, under the race detector, on a workstation or on the build seat.
|
||||
//
|
||||
// **No catalogue pin check.** mesh-controller's testbus also checks the server's release against the
|
||||
// catalogue's bus image and the mesh's facts snapshot (novox/hq ADR 0227 rule 9) — the SDK has no
|
||||
// catalogue of its own to check against, so that test is left out here. Only the version this
|
||||
// module's go.mod pins is what these tests run.
|
||||
//
|
||||
// A person may still point a run at a bus of their own — another release, a lab's — with
|
||||
// MESH_TEST_NATS_EXTERNAL=1 and MESH_TEST_NATS; then the run is theirs to serialise.
|
||||
package testbus
|
||||
|
||||
import (
|
||||
"os"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats-server/v2/server"
|
||||
)
|
||||
|
||||
// Version is the release of the server the tests run.
|
||||
const Version = server.VERSION
|
||||
|
||||
// URL is the bus of this test alone: started at its first ask, the same one at every ask after — a
|
||||
// test that dials twice, or hands the address to the code it tests, reaches one bus — and shut down
|
||||
// when the test ends.
|
||||
func URL(t testing.TB) string {
|
||||
t.Helper()
|
||||
if os.Getenv("MESH_TEST_NATS_EXTERNAL") == "1" {
|
||||
if url := os.Getenv("MESH_TEST_NATS"); url != "" {
|
||||
return url
|
||||
}
|
||||
t.Fatal("MESH_TEST_NATS_EXTERNAL=1 and MESH_TEST_NATS names no bus")
|
||||
}
|
||||
if url, ok := buses.Load(t); ok {
|
||||
return url.(string)
|
||||
}
|
||||
url := Start(t).ClientURL()
|
||||
buses.Store(t, url)
|
||||
t.Cleanup(func() { buses.Delete(t) })
|
||||
return url
|
||||
}
|
||||
|
||||
// buses are the running tests' buses, by test.
|
||||
var buses sync.Map
|
||||
|
||||
// Start is a server with JetStream, for this test alone: on loopback, on a port the system chooses, its
|
||||
// store in the test's own directory.
|
||||
func Start(t testing.TB) *server.Server {
|
||||
t.Helper()
|
||||
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
|
||||
NoLog: true, NoSigs: true}
|
||||
s, err := server.NewServer(opts)
|
||||
if err != nil {
|
||||
t.Fatalf("a bus for this test could not be made: %v", err)
|
||||
}
|
||||
go s.Start()
|
||||
if !s.ReadyForConnections(30 * time.Second) {
|
||||
s.Shutdown()
|
||||
t.Fatal("a bus for this test did not come up within 30s")
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
s.Shutdown()
|
||||
s.WaitForShutdown()
|
||||
})
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
//go:build integration
|
||||
|
||||
package testbus
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// 1175/AC3, 1171/AC1: a JetStream bus of this test alone answers a publish and a key-value create.
|
||||
func TestBusAnswersPublishAndKeyValueCreate(t *testing.T) {
|
||||
conn, err := nats.Connect(URL(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
js, err := jetstream.New(conn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
|
||||
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "EVENTS", Subjects: []string{"mesh.>"}}); err != nil {
|
||||
t.Fatalf("the bus did not answer a stream create: %v", err)
|
||||
}
|
||||
ack, err := js.Publish(ctx, "mesh.test", []byte("hello"))
|
||||
if err != nil {
|
||||
t.Fatalf("the bus did not answer a publish: %v", err)
|
||||
}
|
||||
if ack.Stream != "EVENTS" {
|
||||
t.Errorf("published into %q, want EVENTS", ack.Stream)
|
||||
}
|
||||
|
||||
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: "TESTKV"})
|
||||
if err != nil {
|
||||
t.Fatalf("the bus did not answer a key-value create: %v", err)
|
||||
}
|
||||
if _, err := kv.Put(ctx, "k", []byte("v")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry, err := kv.Get(ctx, "k")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(entry.Value()) != "v" {
|
||||
t.Errorf("read back %q, want %q", entry.Value(), "v")
|
||||
}
|
||||
}
|
||||
|
||||
// 1175/AC3, 1171/AC1: the bus is gone when the test that started it ends.
|
||||
func TestBusGoneWhenItsTestEnds(t *testing.T) {
|
||||
var url string
|
||||
t.Run("one", func(t *testing.T) {
|
||||
url = URL(t)
|
||||
conn, err := nats.Connect(url)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
conn.Close()
|
||||
})
|
||||
if _, err := nats.Connect(url, nats.Timeout(time.Second)); err == nil {
|
||||
t.Fatal("the subtest's bus still answers now that the subtest has ended")
|
||||
}
|
||||
}
|
||||
|
||||
// 1175/AC3: two tests never share a bus.
|
||||
func TestEveryTestHasABusOfItsOwn(t *testing.T) {
|
||||
seen := map[string]bool{}
|
||||
for i := 0; i < 2; i++ {
|
||||
t.Run("one", func(t *testing.T) {
|
||||
url := URL(t)
|
||||
if seen[url] {
|
||||
t.Fatalf("two tests were handed %s", url)
|
||||
}
|
||||
seen[url] = true
|
||||
})
|
||||
}
|
||||
// Asked twice, one test reaches one bus.
|
||||
if a, b := URL(t), URL(t); a != b {
|
||||
t.Errorf("one test was handed two buses: %s and %s", a, b)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1171/AC4: before Serve, a bundle's bus functions still refuse exactly as on main (stdio.go:391-398,
|
||||
// unchanged for bundles). Run in a fresh process: once any other test in this package has called Serve,
|
||||
// current stays set for the rest of the test binary, so "before Serve" can only be checked honestly in a
|
||||
// process of its own (the self-exec pattern Go's own testing package uses for os.Exit).
|
||||
func TestBusFunctionsRefuseBeforeServeInAFreshProcess(t *testing.T) {
|
||||
const marker = "MESH_SDK_TEST_FRESH_PROCESS"
|
||||
if os.Getenv(marker) == "1" {
|
||||
if err := Emit("x", map[string]any{}); err == nil ||
|
||||
err.Error() != "emit before Serve: a bundle emits through the runtime that launched it" {
|
||||
t.Fatalf("Emit before Serve: %v", err)
|
||||
}
|
||||
if _, err := Ask("m.t", nil); err == nil ||
|
||||
err.Error() != "before Serve: a bundle reaches the bus through the runtime that launched it" {
|
||||
t.Fatalf("Ask before Serve: %v", err)
|
||||
}
|
||||
if err := Subscribe("x", func(Envelope) error { return nil }); err == nil ||
|
||||
err.Error() != "before Serve: a bundle reaches the bus through the runtime that launched it" {
|
||||
t.Fatalf("Subscribe before Serve: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
cmd := exec.Command(os.Args[0], "-test.run=^TestBusFunctionsRefuseBeforeServeInAFreshProcess$", "-test.v")
|
||||
cmd.Env = append(os.Environ(), marker+"=1")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("a fresh process did not refuse every bus function before Serve: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Log — a module's record of operations on the bus, kept as long as the log is (novox/hq ADR 0297). Not the
|
||||
// current value: that is state, and a log's state is its fold. Each entry is one operation on one key, its
|
||||
// sequence in the log is its identity, and the order of the log is the order of the operations. A module
|
||||
// declares each log it owns in its manifest (`logs`) and names it here as it named it there; a log is its
|
||||
// owner's alone, and the runtime refuses, with the reason, one it did not issue.
|
||||
|
||||
// LogEntry is one operation on one key, as the log holds it.
|
||||
type LogEntry struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
Key string `json:"key"`
|
||||
Value json.RawMessage `json:"value"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// LogPage is one page of a read: its entries, in order, the sequence to read on from (0 when nothing
|
||||
// follows) and the log's last sequence when it was read.
|
||||
type LogPage struct {
|
||||
Entries []LogEntry `json:"entries"`
|
||||
Next uint64 `json:"next"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// LogInfo is how much a log holds: its entries and bytes, its cap in bytes, and its first and last sequence.
|
||||
type LogInfo struct {
|
||||
Entries uint64 `json:"entries"`
|
||||
Bytes uint64 `json:"bytes"`
|
||||
MaxBytes uint64 `json:"max_bytes"`
|
||||
First uint64 `json:"first"`
|
||||
Last uint64 `json:"last"`
|
||||
}
|
||||
|
||||
// KeptLog is one log this module owns.
|
||||
type KeptLog struct{ name string }
|
||||
|
||||
// Log is a log this module owns, by its local name.
|
||||
func Log(name string) KeptLog { return KeptLog{name: name} }
|
||||
|
||||
const logTimeout = 30 * time.Second
|
||||
|
||||
func (l KeptLog) ask(verb string, params map[string]any) (json.RawMessage, error) {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
params["log"] = l.name
|
||||
return c.ask("mesh/log."+verb, params, logTimeout)
|
||||
}
|
||||
|
||||
func (l KeptLog) append(params map[string]any) (uint64, error) {
|
||||
raw, err := l.ask("append", params)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var appended struct {
|
||||
Sequence uint64 `json:"sequence"`
|
||||
}
|
||||
return appended.Sequence, json.Unmarshal(raw, &appended)
|
||||
}
|
||||
|
||||
// Append writes an entry for key, whatever came before it, and returns its sequence. A key is one or more
|
||||
// names of letters, digits, `-` and `_`, joined by `.`; a value is JSON of at most 256 KiB. A full log
|
||||
// refuses the entry: it never drops an older one to make room.
|
||||
func (l KeptLog) Append(key string, value any) (uint64, error) {
|
||||
return l.append(map[string]any{"key": key, "value": value})
|
||||
}
|
||||
|
||||
// AppendAt writes an entry for key only if the key's last entry is still at expect — the sequence the caller
|
||||
// read, 0 for a key with no entry yet — and returns its sequence. When another entry for the key came in
|
||||
// between, nothing is written and the error is ErrChanged, as for a lost State update: read again and
|
||||
// decide again.
|
||||
func (l KeptLog) AppendAt(key string, value any, expect uint64) (uint64, error) {
|
||||
return l.append(map[string]any{"key": key, "value": value, "expect": expect})
|
||||
}
|
||||
|
||||
// Read is the entries from the sequence from on, oldest first, at most limit of them (0 is the runtime's
|
||||
// page, 100; it gives at most 1000). keys narrows it: a key, with `*` for one name and `>` for the rest;
|
||||
// empty is every key. Read on from the page's Next until it is 0.
|
||||
func (l KeptLog) Read(from uint64, keys string, limit int) (*LogPage, error) {
|
||||
if limit < 0 {
|
||||
return nil, fmt.Errorf("a read of %s asks for at least no entries: limit %d", l.name, limit)
|
||||
}
|
||||
params := map[string]any{"from": from}
|
||||
if keys != "" {
|
||||
params["keys"] = keys
|
||||
}
|
||||
if limit > 0 {
|
||||
params["limit"] = limit
|
||||
}
|
||||
raw, err := l.ask("read", params)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var p LogPage
|
||||
return &p, json.Unmarshal(raw, &p)
|
||||
}
|
||||
|
||||
// Last is a key's last entry, or nil when the key has none.
|
||||
func (l KeptLog) Last(key string) (*LogEntry, error) {
|
||||
raw, err := l.ask("last", map[string]any{"key": key})
|
||||
if err != nil || len(raw) == 0 || string(raw) == "null" {
|
||||
return nil, err
|
||||
}
|
||||
var e LogEntry
|
||||
return &e, json.Unmarshal(raw, &e)
|
||||
}
|
||||
|
||||
// Info is how much the log holds and how much it may.
|
||||
func (l KeptLog) Info() (*LogInfo, error) {
|
||||
raw, err := l.ask("info", map[string]any{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var i LogInfo
|
||||
return &i, json.Unmarshal(raw, &i)
|
||||
}
|
||||
+354
@@ -0,0 +1,354 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// logRuntime is the runtime's side of mesh/log.append, read, last and info as the bus does them (novox/hq
|
||||
// ADR 0297): one sequence per entry across the log, an append with expect refused unless the key's last
|
||||
// entry is still at it (0 for a key with none), that refusal answered with CodeChanged as a lost state
|
||||
// update is, and a read paged by its limit with the sequence to read on from. asked keeps every ask's
|
||||
// method and params, as the bundle sent them.
|
||||
type logRuntime struct {
|
||||
mu sync.Mutex
|
||||
entries []LogEntry
|
||||
asked []message
|
||||
}
|
||||
|
||||
func (r *logRuntime) last(key string) uint64 {
|
||||
for i := len(r.entries) - 1; i >= 0; i-- {
|
||||
if r.entries[i].Key == key {
|
||||
return r.entries[i].Sequence
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// matches is a key pattern as the runtime reads it: `*` one name, `>` the rest.
|
||||
func matches(pattern, key string) bool {
|
||||
if pattern == "" {
|
||||
return true
|
||||
}
|
||||
p, k := strings.Split(pattern, "."), strings.Split(key, ".")
|
||||
for i, t := range p {
|
||||
if t == ">" {
|
||||
return len(k) > i
|
||||
}
|
||||
if i >= len(k) || (t != "*" && t != k[i]) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return len(p) == len(k)
|
||||
}
|
||||
|
||||
func (r *logRuntime) answer(m message) map[string]any {
|
||||
var p struct {
|
||||
Log string `json:"log"`
|
||||
Key string `json:"key"`
|
||||
Value json.RawMessage `json:"value"`
|
||||
Expect *uint64 `json:"expect"`
|
||||
From uint64 `json:"from"`
|
||||
Keys string `json:"keys"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
r.asked = append(r.asked, m)
|
||||
if p.Log != "changes" {
|
||||
return map[string]any{"error": map[string]any{"code": -32000, "message": "issues was issued no log " + p.Log}}
|
||||
}
|
||||
switch strings.TrimPrefix(m.Method, "mesh/log.") {
|
||||
case "append":
|
||||
if p.Expect != nil && r.last(p.Key) != *p.Expect {
|
||||
return map[string]any{"error": map[string]any{"code": CodeChanged, "message": p.Key + ": the key was written by another writer first"}}
|
||||
}
|
||||
seq := uint64(len(r.entries)) + 1
|
||||
r.entries = append(r.entries, LogEntry{Sequence: seq, Key: p.Key, Value: p.Value,
|
||||
At: time.Date(2026, 10, 11, 18, 0, int(seq), 0, time.UTC)})
|
||||
return map[string]any{"result": map[string]any{"sequence": seq}}
|
||||
case "read":
|
||||
limit := p.Limit
|
||||
if limit == 0 {
|
||||
limit = 100
|
||||
}
|
||||
page := map[string]any{"entries": []LogEntry{}, "next": 0, "last": len(r.entries)}
|
||||
var got []LogEntry
|
||||
for _, e := range r.entries {
|
||||
if e.Sequence < p.From || !matches(p.Keys, e.Key) {
|
||||
continue
|
||||
}
|
||||
if len(got) == limit {
|
||||
page["next"] = e.Sequence
|
||||
break
|
||||
}
|
||||
got = append(got, e)
|
||||
}
|
||||
if got != nil {
|
||||
page["entries"] = got
|
||||
}
|
||||
return map[string]any{"result": page}
|
||||
case "last":
|
||||
seq := r.last(p.Key)
|
||||
if seq == 0 {
|
||||
return map[string]any{"result": nil}
|
||||
}
|
||||
return map[string]any{"result": r.entries[seq-1]}
|
||||
case "info":
|
||||
var bytes int
|
||||
for _, e := range r.entries {
|
||||
bytes += len(e.Value)
|
||||
}
|
||||
info := map[string]any{"entries": len(r.entries), "bytes": bytes, "max_bytes": 1 << 30, "first": 0, "last": len(r.entries)}
|
||||
if len(r.entries) > 0 {
|
||||
info["first"] = 1
|
||||
}
|
||||
return map[string]any{"result": info}
|
||||
}
|
||||
return map[string]any{"error": map[string]any{"code": -32601, "message": "no " + m.Method}}
|
||||
}
|
||||
|
||||
// logging serves a bundle whose one tool, "log", runs run against r, answering each mesh/log ask beside
|
||||
// the loop as the runtime does. It returns what the tool answered.
|
||||
func logging(t *testing.T, r *logRuntime, run func() (any, error)) map[string]any {
|
||||
t.Helper()
|
||||
t.Setenv(ServedAs, "issues")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{{Name: "log", Run: func(map[string]any) (any, error) { return run() }}}, inR, outW)
|
||||
}()
|
||||
var wmu sync.Mutex
|
||||
send := func(m map[string]any) {
|
||||
m["jsonrpc"] = "2.0"
|
||||
b, _ := json.Marshal(m)
|
||||
wmu.Lock()
|
||||
defer wmu.Unlock()
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
}
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
lines := bufio.NewScanner(outR)
|
||||
lines.Buffer(make([]byte, 1<<20), 1<<24)
|
||||
for lines.Scan() {
|
||||
var m message
|
||||
if json.Unmarshal(lines.Bytes(), &m) != nil {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(m.Method, "mesh/log.") {
|
||||
go func(m message) {
|
||||
a := r.answer(m)
|
||||
a["id"] = m.ID
|
||||
send(a)
|
||||
}(m)
|
||||
continue
|
||||
}
|
||||
var answer map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &answer)
|
||||
if answer["id"] == float64(1) {
|
||||
done <- answer
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
send(map[string]any{"id": 1, "method": "tools/call", "params": map[string]any{"name": "log"}})
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
select {
|
||||
case a := <-done:
|
||||
return a
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("the bundle did not answer")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// params is what the bundle sent with its n-th ask, and the ask's method.
|
||||
func (r *logRuntime) params(t *testing.T, n int) (string, map[string]any) {
|
||||
t.Helper()
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if n >= len(r.asked) {
|
||||
t.Fatalf("the bundle asked %d times, not %d", len(r.asked), n+1)
|
||||
}
|
||||
var p map[string]any
|
||||
_ = json.Unmarshal(r.asked[n].Params, &p)
|
||||
return r.asked[n].Method, p
|
||||
}
|
||||
|
||||
// A bundle appends to, reads, and measures its log through the runtime (novox/hq ADR 0297): each verb is
|
||||
// asked as mesh/log.<verb> naming the log, its answer is handed back whole, a key with no entry is nil,
|
||||
// and a read pages on from Next until it is 0.
|
||||
func TestABundleKeepsALogThroughTheRuntime(t *testing.T) {
|
||||
r := &logRuntime{}
|
||||
var (
|
||||
first, second, third uint64
|
||||
pages []*LogPage
|
||||
last, none *LogEntry
|
||||
info *LogInfo
|
||||
narrowed *LogPage
|
||||
)
|
||||
got := logging(t, r, func() (any, error) {
|
||||
changes := Log("changes")
|
||||
var err error
|
||||
if first, err = changes.Append("401", map[string]any{"op": "opened", "by": "fix@g14"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if second, err = changes.Append("402", map[string]any{"op": "opened"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if third, err = changes.AppendAt("401", map[string]any{"op": "moved", "to": "fixing"}, first); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, k := range []string{"403", "404", "405"} {
|
||||
if _, err = changes.Append(k, map[string]any{"op": "opened"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for from := uint64(1); ; {
|
||||
p, err := changes.Read(from, "", 2)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pages = append(pages, p)
|
||||
if p.Next == 0 {
|
||||
break
|
||||
}
|
||||
from = p.Next
|
||||
}
|
||||
if narrowed, err = changes.Read(0, "401", 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if last, err = changes.Last("401"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if none, err = changes.Last("999"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
info, err = changes.Info()
|
||||
return "done", err
|
||||
})
|
||||
if out, isError := text(t, got); isError || out != `"done"` {
|
||||
t.Fatalf("the tool answered %s", out)
|
||||
}
|
||||
if first != 1 || second != 2 || third != 3 {
|
||||
t.Fatalf("appended at %d, %d, %d", first, second, third)
|
||||
}
|
||||
|
||||
// Asked as the contract names them: the verb, the log, and only what was given.
|
||||
method, p := r.params(t, 0)
|
||||
if method != "mesh/log.append" || p["log"] != "changes" || p["key"] != "401" || p["value"].(map[string]any)["by"] != "fix@g14" {
|
||||
t.Fatalf("an append asked %s %v", method, p)
|
||||
}
|
||||
if _, has := p["expect"]; has {
|
||||
t.Fatalf("an Append without expect named one: %v", p)
|
||||
}
|
||||
if method, p = r.params(t, 2); method != "mesh/log.append" || p["expect"] != float64(1) {
|
||||
t.Fatalf("an AppendAt asked %s %v", method, p)
|
||||
}
|
||||
if method, p = r.params(t, 6); method != "mesh/log.read" || p["from"] != float64(1) || p["limit"] != float64(2) || p["keys"] != nil {
|
||||
t.Fatalf("a read asked %s %v", method, p)
|
||||
}
|
||||
|
||||
// Paged two at a time through six entries: three pages, every entry once, in order.
|
||||
if len(pages) != 3 {
|
||||
t.Fatalf("read %d pages, not 3", len(pages))
|
||||
}
|
||||
var seqs []uint64
|
||||
for _, pg := range pages {
|
||||
if pg.Last != 6 {
|
||||
t.Fatalf("a page said the last sequence is %d", pg.Last)
|
||||
}
|
||||
for _, e := range pg.Entries {
|
||||
seqs = append(seqs, e.Sequence)
|
||||
}
|
||||
}
|
||||
if fmt.Sprint(seqs) != "[1 2 3 4 5 6]" || pages[0].Next != 3 || pages[2].Next != 0 {
|
||||
t.Fatalf("paged %v, next %d then %d", seqs, pages[0].Next, pages[2].Next)
|
||||
}
|
||||
if e := pages[1].Entries[0]; e.Key != "401" || string(e.Value) != `{"op":"moved","to":"fixing"}` ||
|
||||
!e.At.Equal(time.Date(2026, 10, 11, 18, 0, 3, 0, time.UTC)) {
|
||||
t.Fatalf("an entry read as %+v", e)
|
||||
}
|
||||
|
||||
// Narrowed to a key, with the runtime's page: no limit sent.
|
||||
method, p = r.params(t, 9)
|
||||
if method != "mesh/log.read" || p["keys"] != "401" || p["limit"] != nil {
|
||||
t.Fatalf("a narrowed read asked %s %v", method, p)
|
||||
}
|
||||
if len(narrowed.Entries) != 2 || narrowed.Entries[0].Sequence != 1 || narrowed.Entries[1].Sequence != 3 || narrowed.Next != 0 {
|
||||
t.Fatalf("narrowed to 401: %+v", narrowed)
|
||||
}
|
||||
|
||||
if last == nil || last.Sequence != 3 || last.Key != "401" {
|
||||
t.Fatalf("the last of 401 is %+v", last)
|
||||
}
|
||||
if method, p = r.params(t, 10); method != "mesh/log.last" || p["key"] != "401" {
|
||||
t.Fatalf("a last asked %s %v", method, p)
|
||||
}
|
||||
if none != nil {
|
||||
t.Fatalf("a key with no entry has a last: %+v", none)
|
||||
}
|
||||
if info == nil || info.Entries != 6 || info.MaxBytes != 1<<30 || info.First != 1 || info.Last != 6 || info.Bytes == 0 {
|
||||
t.Fatalf("the log's info is %+v", info)
|
||||
}
|
||||
if method, p = r.params(t, 12); method != "mesh/log.info" || p["log"] != "changes" || len(p) != 1 {
|
||||
t.Fatalf("an info asked %s %v", method, p)
|
||||
}
|
||||
}
|
||||
|
||||
// An AppendAt on a stale expect is ErrChanged and writes nothing: another entry for the key came in
|
||||
// between. Expect 0 is a key with no entry yet, and is ErrChanged once the key has one. A log not issued
|
||||
// is refused with the runtime's reason, and is not ErrChanged.
|
||||
func TestAnAppendAtAStaleSequenceIsErrChangedAndWritesNothing(t *testing.T) {
|
||||
r := &logRuntime{}
|
||||
var errs []error
|
||||
got := logging(t, r, func() (any, error) {
|
||||
changes := Log("changes")
|
||||
read, err := changes.AppendAt("401", map[string]any{"op": "opened"}, 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Another writer is first.
|
||||
if _, err := changes.Append("401", map[string]any{"op": "noted"}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_, stale := changes.AppendAt("401", map[string]any{"op": "moved"}, read)
|
||||
_, again := changes.AppendAt("401", map[string]any{"op": "opened"}, 0)
|
||||
_, other := Log("elsewhere").Append("401", map[string]any{})
|
||||
errs = []error{stale, again, other}
|
||||
return "done", nil
|
||||
})
|
||||
if out, isError := text(t, got); isError || out != `"done"` {
|
||||
t.Fatalf("the tool answered %s", out)
|
||||
}
|
||||
if !errors.Is(errs[0], ErrChanged) || !errors.Is(errs[1], ErrChanged) {
|
||||
t.Fatalf("a stale expect answered %v and %v, not ErrChanged", errs[0], errs[1])
|
||||
}
|
||||
if errs[2] == nil || errors.Is(errs[2], ErrChanged) || !strings.Contains(errs[2].Error(), "no log elsewhere") {
|
||||
t.Fatalf("a log not issued answered %v", errs[2])
|
||||
}
|
||||
if len(r.entries) != 2 {
|
||||
t.Fatalf("the log holds %d entries, not 2: a refused append wrote", len(r.entries))
|
||||
}
|
||||
}
|
||||
|
||||
// A read asking for fewer than no entries is refused before the runtime is asked.
|
||||
func TestAReadOfANegativeLimitIsRefused(t *testing.T) {
|
||||
r := &logRuntime{}
|
||||
var err error
|
||||
logging(t, r, func() (any, error) {
|
||||
_, err = Log("changes").Read(0, "", -1)
|
||||
return "done", nil
|
||||
})
|
||||
if err == nil || len(r.asked) != 0 {
|
||||
t.Fatalf("a negative limit answered %v after %d asks", err, len(r.asked))
|
||||
}
|
||||
}
|
||||
+115
-4
@@ -5,16 +5,127 @@
|
||||
# seat in the mesh's Go toolchain, its own container, because the SDK's merge-check.sh declares it
|
||||
# (`# mesh-check-also: go go/merge-check.sh`) — and by hand, from anywhere: `sh go/merge-check.sh`.
|
||||
#
|
||||
# Formatted, vetted, and its whole suite run: the stdio loop and the local protocol mesh-cli speaks to the
|
||||
# node-engine. Standard library only, so nothing is fetched.
|
||||
# Formatted, vetted (with and without the build tag `integration`), and run at two levels (novox/hq ADR 0391):
|
||||
#
|
||||
# 1. the unit level: `go test ./...` untagged, no bus or other process, timed and held to a bound
|
||||
# (unit_bound below); over it the check fails and names the slowest tests (issue 895);
|
||||
# 2. the integration level: the whole suite with `-tags integration`, each test against a bus of its
|
||||
# own (internal/testbus, which issue 1171 brings with the first test that needs a bus) so nothing is
|
||||
# serialised. With no tagged test yet, the level runs the unit tests again and says their count.
|
||||
#
|
||||
# **A test that starts a bus or another process carries `//go:build integration`** as the first line of
|
||||
# its file; a file that held both kinds is two files (`<name>_integration_test.go` beside `<name>_test.go`).
|
||||
# Each level says its test count and its time.
|
||||
#
|
||||
# No dependency is committed (no vendor directory: .gitignore; the operator, 2026-10-19; issue 1304). Every dependency
|
||||
# is resolved by the version and checksum go.sum names (novox/hq ADR 0405 §5), from its public source: this binding
|
||||
# requires nothing of the mesh's own forge. `go mod verify` fails by module name when a downloaded dependency does not
|
||||
# match go.sum, and -mod=readonly fails a build whose go.sum lacks an entry instead of writing it. gofmt reads only
|
||||
# the binding's own files.
|
||||
#
|
||||
# Exit 0 is a pass; anything else fails `mesh/repo-check` with the last line printed.
|
||||
set -eu
|
||||
cd "$(dirname "$0")"
|
||||
# The toolchain the check runs in sets -mod=vendor; there is no vendor directory, so it is replaced.
|
||||
GOFLAGS="$(printf '%s' "${GOFLAGS:-}" | sed 's/-mod=[a-z]*//g') -mod=readonly"
|
||||
export GOFLAGS
|
||||
export GOPRIVATE=git.novox.be
|
||||
|
||||
unformatted=$(gofmt -l .)
|
||||
# **The unit level's bound, in seconds** (issue 895): the slowest package's own test time of the untagged run,
|
||||
# the compile not counted. A run over it fails the check and names the slowest tests. The SDK's unit suite is
|
||||
# small (its slowest package measured well under a second on a quiet machine), so 2 s leaves room for a busier
|
||||
# build node without hiding a test that should carry `integration` instead.
|
||||
unit_bound=2
|
||||
|
||||
# judge_unit_run says the unit level as `go test -v` reported it and fails when it was over the bound: $1 the
|
||||
# file holding the output of the untagged `go test -v` of every package (packages one after another, each
|
||||
# ending in its `ok <package> <time>` line), $2 the bound in seconds. Packages run side by side, so the
|
||||
# level's time is that of its slowest package; the five slowest tests of the run are named when it is over,
|
||||
# so that the bound is kept by making them cheap or by tagging them `integration`, and never by raising it
|
||||
# (issue 895).
|
||||
judge_unit_run() {
|
||||
out=$1 bound=$2
|
||||
awk '
|
||||
/^--- (PASS|FAIL|SKIP): / {
|
||||
name = $3; took = $4; gsub(/[()s]/, "", took)
|
||||
pending[++n] = took "\t" name
|
||||
if ($2 == "SKIP:") skipped++
|
||||
next
|
||||
}
|
||||
/^(ok|FAIL)[ \t]/ && NF >= 3 && $3 ~ /^[0-9.]+s$/ {
|
||||
t = $3; sub(/s$/, "", t)
|
||||
for (i = 1; i <= n; i++) print "T\t" pending[i] "\t" $2
|
||||
n = 0
|
||||
print "P\t" t "\t" $2
|
||||
}
|
||||
END { print "S\t" skipped + 0 }
|
||||
' "$out" >"$out.judged"
|
||||
tests=$(grep -c '^T' "$out.judged" || true)
|
||||
skipped=$(awk -F'\t' '$1 == "S" {print $2}' "$out.judged")
|
||||
level=$(awk -F'\t' '$1 == "P" && $2 + 0 > m {m = $2 + 0} END {printf "%.2f", m}' "$out.judged")
|
||||
echo "unit level: $tests tests ($skipped skipped), $level s (bound $bound s)"
|
||||
if awk -v t="$level" -v b="$bound" 'BEGIN {exit !(t + 0 > b + 0)}'; then
|
||||
echo "the unit level took $level s, over its bound of $bound s; the slowest tests:"
|
||||
awk -F'\t' '$1 == "T" {print $2 "\t" $4 "\t" $3}' "$out.judged" | sort -t"$(printf '\t')" -k1,1 -nr | head -n 5 |
|
||||
awk -F'\t' '{printf " %s s %s %s\n", $1, $3, $2}'
|
||||
echo "make them cheap (no wait, no process), or give their files //go:build integration if they start a bus or another process"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
# `sh go/merge-check.sh judge-unit-run <output> <bound>` runs only the judging of the unit level, on a
|
||||
# recorded output.
|
||||
if [ "${1:-}" = judge-unit-run ]; then
|
||||
shift
|
||||
judge_unit_run "$@"
|
||||
exit
|
||||
fi
|
||||
|
||||
if ! fetched=$(go mod download 2>&1); then
|
||||
echo "FAILED: the dependencies cannot be fetched: $(printf '%s\n' "$fetched" | tail -n 1)"
|
||||
exit 1
|
||||
fi
|
||||
if ! verified=$(go mod verify 2>&1); then
|
||||
echo "FAILED: a dependency does not match go.sum: $verified"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
unformatted=$(find . -name '*.go' -not -path './vendor/*' | xargs gofmt -l)
|
||||
if [ -n "$unformatted" ]; then
|
||||
echo "not gofmt'd:"
|
||||
echo "$unformatted"
|
||||
exit 1
|
||||
fi
|
||||
go vet ./...
|
||||
go test -count=1 ./...
|
||||
# The files tagged `integration` are vetted too, or nothing would read them.
|
||||
go vet -tags integration ./...
|
||||
|
||||
# NOT RACE-CHECKED here: the SDK's suite is small and carries no store or other shared state between
|
||||
# packages, so the race detector is not required of this check (novox/hq issue 1175); a module that needs
|
||||
# it asks for it in its own merge-check.sh.
|
||||
echo "NOT RACE-CHECKED: the Go binding's check runs without the race detector"
|
||||
|
||||
work=$(mktemp -d)
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
|
||||
# The unit level first and alone: every package untagged, no bus or other process (judged above).
|
||||
status=0
|
||||
go test -count=1 -v ./... >"$work/unit.out" 2>&1 || status=$?
|
||||
if [ "$status" -ne 0 ]; then
|
||||
grep -vE '^(=== |[[:space:]]*--- PASS)' "$work/unit.out"
|
||||
echo "the unit level failed (go test exited $status)"
|
||||
exit 1
|
||||
fi
|
||||
judge_unit_run "$work/unit.out" "$unit_bound"
|
||||
|
||||
# The integration level: the whole suite with the tag, each test against its own bus (internal/testbus).
|
||||
integration_start=$(date +%s)
|
||||
status=0
|
||||
go test -tags integration -count=1 -v ./... >"$work/integration.out" 2>&1 || status=$?
|
||||
grep -vE '^(=== |[[:space:]]*--- PASS)' "$work/integration.out" || true
|
||||
tests=$(grep -cE '^--- (PASS|FAIL|SKIP): ' "$work/integration.out" || true)
|
||||
skipped=$(grep -cE '^--- SKIP: ' "$work/integration.out" || true)
|
||||
echo "integration level: $tests tests ($skipped skipped), $(($(date +%s) - integration_start)) s"
|
||||
if [ "$status" -ne 0 ]; then
|
||||
echo "the integration level failed (go test exited $status)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
// Package outward is the content rule (novox/hq ADR 0310 §4, to-be 45 §5, research 028 Q8, ADR 0227): what
|
||||
// may leave the mesh, and what an answer may carry at all. One rule, here, so every module that answers outward
|
||||
// applies the same one rather than a copy: the operator channel's router and every node-reads answer.
|
||||
//
|
||||
// Two strengths:
|
||||
//
|
||||
// - Check is the operator channel's: roles and words only. A message carrying an address, a path or
|
||||
// anything shaped like a secret is refused, because a channel such as Telegram is not end-to-end
|
||||
// encrypted for bots and this rule is the only thing between a condition's words and someone else's
|
||||
// server. Deliberately wider than it must be: a commit id or a long random name is refused too.
|
||||
// - CheckSecret is the secret class alone: what leaves the mesh nowhere and is answered nowhere. A path
|
||||
// or an address is what a file read or an operator's approval is about; a secret's shape never is.
|
||||
//
|
||||
// A refusal says its class and a few words, never the text itself.
|
||||
package outward
|
||||
|
||||
import (
|
||||
"math"
|
||||
"regexp"
|
||||
"strings"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// Refusal says why a text may not leave: the class of what it carried, never the text itself.
|
||||
type Refusal struct {
|
||||
Class string // address, path or secret
|
||||
What string // a few words: "an IPv4 address", "a URL", …
|
||||
}
|
||||
|
||||
func (r Refusal) String() string { return r.Class + " (" + r.What + ")" }
|
||||
|
||||
var (
|
||||
reURL = regexp.MustCompile(`(?i)\b[a-z][a-z0-9+.-]*://`)
|
||||
reEmail = regexp.MustCompile(`[A-Za-z0-9._%+-]+@[A-Za-z0-9-]+(\.[A-Za-z0-9-]+)*\.[A-Za-z]{2,}`)
|
||||
reIPv4 = regexp.MustCompile(`\b\d{1,3}(\.\d{1,3}){3}\b`)
|
||||
reIPv6 = regexp.MustCompile(`(?i)(^|[^0-9a-z:])(([0-9a-f]{1,4}:){4,7}[0-9a-f]{1,4}|([0-9a-f]{1,4}:)*[0-9a-f]{0,4}::([0-9a-f]{1,4}:)*[0-9a-f]{0,4})([^0-9a-z:]|$)`)
|
||||
reMAC = regexp.MustCompile(`(?i)\b([0-9a-f]{2}[:-]){5}[0-9a-f]{2}\b`)
|
||||
rePEM = regexp.MustCompile(`-----BEGIN [A-Z ]+-----`)
|
||||
reJWT = regexp.MustCompile(`\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}`)
|
||||
reBotToken = regexp.MustCompile(`\b\d{6,}:[A-Za-z0-9_-]{30,}`)
|
||||
reKnown = regexp.MustCompile(`\b(gh[pousr]_[A-Za-z0-9]{20,}|glpat-[A-Za-z0-9_-]{16,}|sk-[A-Za-z0-9_-]{16,}|xox[abprs]-[A-Za-z0-9-]{10,}|AKIA[0-9A-Z]{16})`)
|
||||
reAssigned = regexp.MustCompile(`(?i)\b(password|passwd|passphrase|secret|token|api[_-]?key|apikey|credential|private[_-]?key)\s*[=:]\s*\S`)
|
||||
// reNamedSecret is a value given to a name that ends in a secret's word after `_` or `-`, as an environment
|
||||
// or configuration name is written: MESH_TOKEN=, DB_PASS=, aws_secret_access_key = …, SUDO_PASSWORD value.
|
||||
// The word ends the name: what follows it is `=`, `:` or a space and then a value.
|
||||
reNamedSecret = regexp.MustCompile(`(?i)(^|[^a-z0-9_-])[a-z0-9]+[_-]([a-z0-9]+[_-])*(password|passwd|passphrase|pass|pwd|secret|token|api[_-]?key|apikey|access[_-]?key|secret[_-]?key|private[_-]?key|credentials?|psk)(\s*[=:]\s*|\s+)[^\s=:]`)
|
||||
// reURLCredential is a password inside an address: scheme://user:password@host.
|
||||
reURLCredential = regexp.MustCompile(`(?i)\b[a-z][a-z0-9+.-]*://[^/\s:@]+:[^/\s@]+@`)
|
||||
// reSecretRun is a run CheckSecret judges: reRun with `-`, so a base64url value is judged whole.
|
||||
reSecretRun = regexp.MustCompile(`[A-Za-z0-9+/=_-]{20,}`)
|
||||
reUUID = regexp.MustCompile(`^(?i)[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$`)
|
||||
reHex = regexp.MustCompile(`(?i)\b[0-9a-f]{32,}\b`)
|
||||
reRun = regexp.MustCompile(`[A-Za-z0-9+/=_]{20,}`)
|
||||
reWinPath = regexp.MustCompile(`(?i)\b[a-z]:\\`)
|
||||
)
|
||||
|
||||
// topLevel are names that end a host name — the generic and country ones a mesh's names use, and the
|
||||
// private ones (.internal, .lan, .home, .local) a mesh is likelier to.
|
||||
var topLevel = map[string]bool{}
|
||||
|
||||
func init() {
|
||||
for _, t := range strings.Fields(`com net org edu gov mil int io dev app cloud ai co me info biz xyz
|
||||
site online tech page link
|
||||
be nl de fr uk lu eu ch at it es pt se no dk fi pl cz us ca au nz jp cn ru in br ie
|
||||
internal lan home local localdomain corp intranet private arpa test example invalid localhost`) {
|
||||
topLevel[t] = true
|
||||
}
|
||||
}
|
||||
|
||||
// Check says whether a text may leave the mesh, and when not, why.
|
||||
func Check(text string) (Refusal, bool) {
|
||||
switch {
|
||||
case reURL.MatchString(text):
|
||||
return Refusal{"address", "a URL"}, false
|
||||
case reEmail.MatchString(text):
|
||||
return Refusal{"address", "a mail address"}, false
|
||||
case reIPv4.MatchString(text):
|
||||
return Refusal{"address", "an IPv4 address"}, false
|
||||
case reMAC.MatchString(text):
|
||||
return Refusal{"address", "a hardware address"}, false
|
||||
case reIPv6.MatchString(text):
|
||||
return Refusal{"address", "an IPv6 address"}, false
|
||||
}
|
||||
if refusal, ok := secretShape(text); !ok {
|
||||
return refusal, false
|
||||
}
|
||||
if reWinPath.MatchString(text) {
|
||||
return Refusal{"path", "a drive path"}, false
|
||||
}
|
||||
if refusal, ok := randomRun(text); !ok {
|
||||
return refusal, false
|
||||
}
|
||||
for _, word := range strings.FieldsFunc(text, func(r rune) bool {
|
||||
return unicode.IsSpace(r) || strings.ContainsRune("\"'`()[]{}<>,;|", r)
|
||||
}) {
|
||||
w := strings.TrimRight(word, ".:!?")
|
||||
if isPath(w) {
|
||||
return Refusal{"path", "a file path"}, false
|
||||
}
|
||||
if isHostName(w) {
|
||||
return Refusal{"address", "a host name"}, false
|
||||
}
|
||||
}
|
||||
return Refusal{}, true
|
||||
}
|
||||
|
||||
// CheckSecret says whether a text carries a secret's shape — the one class that leaves the mesh nowhere. It is
|
||||
// what an ask's whole words (asks.Ask.Whole, novox/hq issue 383) are held to: on a kind that proves who answers,
|
||||
// a path or an address is what the operator approves and may be shown, a secret never. A path is read as one:
|
||||
// a run with a slash in it is judged piece by piece between the slashes, so "/mnt/Photos_2024/Jochen" is a
|
||||
// path and not a random string (the review of 2026-10-10); the named shapes — a key block, a signed token, a
|
||||
// bot token, a known token shape, a value given to a secret's name, a long hexadecimal string — hold whatever
|
||||
// the run holds.
|
||||
func CheckSecret(text string) (Refusal, bool) {
|
||||
if refusal, ok := secretShape(text); !ok {
|
||||
return refusal, false
|
||||
}
|
||||
return randomRunOutsidePaths(text)
|
||||
}
|
||||
|
||||
// randomRunOutsidePaths judges every long run of a value's characters, a path's pieces between its slashes
|
||||
// alone (the review of mesh-sdk #23): a run is read as a path only where it is one — it starts with a slash,
|
||||
// follows a dot or a tilde (a host's or a home's path), or a name's `=` gives it one — so a base64 value, whose
|
||||
// slashes fall anywhere, is judged whole. A UUID is a name, never a secret.
|
||||
func randomRunOutsidePaths(text string) (Refusal, bool) {
|
||||
for _, loc := range reSecretRun.FindAllStringIndex(text, -1) {
|
||||
var before byte
|
||||
if loc[0] > 0 {
|
||||
before = text[loc[0]-1]
|
||||
}
|
||||
run := text[loc[0]:loc[1]]
|
||||
// Standard base64 ends in its padding: such a run is a value, even when it starts with a slash.
|
||||
padded := strings.HasSuffix(run, "=") && !strings.Contains(strings.TrimRight(run, "="), "=")
|
||||
for i, seg := range strings.Split(run, "=") {
|
||||
path := !padded && (strings.HasPrefix(seg, "/") || (i == 0 && (before == '.' || before == '~')))
|
||||
pieces := []string{seg}
|
||||
if path {
|
||||
pieces = strings.Split(seg, "/")
|
||||
}
|
||||
for _, piece := range pieces {
|
||||
if len(piece) >= 20 && !reUUID.MatchString(piece) && looksRandomValue(piece) {
|
||||
return Refusal{"secret", "a long random-looking string"}, false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return Refusal{}, true
|
||||
}
|
||||
|
||||
// secretShape is the secret shapes a pattern names.
|
||||
func secretShape(text string) (Refusal, bool) {
|
||||
switch {
|
||||
case rePEM.MatchString(text):
|
||||
return Refusal{"secret", "a key block"}, false
|
||||
case reJWT.MatchString(text):
|
||||
return Refusal{"secret", "a signed token"}, false
|
||||
case reBotToken.MatchString(text):
|
||||
return Refusal{"secret", "a bot token"}, false
|
||||
case reKnown.MatchString(text):
|
||||
return Refusal{"secret", "a known token shape"}, false
|
||||
case reURLCredential.MatchString(text):
|
||||
return Refusal{"secret", "a password in an address"}, false
|
||||
case reAssigned.MatchString(text), reNamedSecret.MatchString(text):
|
||||
return Refusal{"secret", "a value given to a secret's name"}, false
|
||||
case reHex.MatchString(text):
|
||||
return Refusal{"secret", "a long hexadecimal string"}, false
|
||||
}
|
||||
return Refusal{}, true
|
||||
}
|
||||
|
||||
// randomRun is a long unbroken run of characters spread as a random string's are.
|
||||
func randomRun(text string) (Refusal, bool) {
|
||||
for _, run := range reRun.FindAllString(text, -1) {
|
||||
if looksRandom(run) {
|
||||
return Refusal{"secret", "a long random-looking string"}, false
|
||||
}
|
||||
}
|
||||
return Refusal{}, true
|
||||
}
|
||||
|
||||
// isPath: absolute, home-relative or dot-relative, or two separators deep. A mesh address names one
|
||||
// machine and one tool (`ace/postgres.query`) and has one; a ratio ("3/4") has digits only.
|
||||
func isPath(w string) bool {
|
||||
if w == "" {
|
||||
return false
|
||||
}
|
||||
if strings.HasPrefix(w, "/") && len(w) > 1 {
|
||||
return true
|
||||
}
|
||||
for _, p := range []string{"~/", "./", "../", "$HOME", "${"} {
|
||||
if strings.HasPrefix(w, p) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return strings.Count(w, "/") >= 2 || strings.Contains(w, "\\")
|
||||
}
|
||||
|
||||
// isHostName: two names or more, the last a top-level one. A condition key's last name is its kind
|
||||
// (`machine.ace.silent`), which none of these is.
|
||||
func isHostName(w string) bool {
|
||||
w = strings.ToLower(w)
|
||||
if w == "localhost" {
|
||||
return true
|
||||
}
|
||||
parts := strings.Split(w, ".")
|
||||
if len(parts) < 2 {
|
||||
return false
|
||||
}
|
||||
for _, p := range parts {
|
||||
if p == "" {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return topLevel[parts[len(parts)-1]]
|
||||
}
|
||||
|
||||
// looksRandom: letters and digits mixed, and the characters spread as a random string's are. A
|
||||
// sentence's words are separated, so only an unbroken run reaches here.
|
||||
func looksRandom(s string) bool {
|
||||
var letters, digits int
|
||||
counts := map[rune]int{}
|
||||
for _, r := range s {
|
||||
counts[r]++
|
||||
switch {
|
||||
case unicode.IsLetter(r):
|
||||
letters++
|
||||
case unicode.IsDigit(r):
|
||||
digits++
|
||||
}
|
||||
}
|
||||
if letters == 0 || digits == 0 {
|
||||
// One class only: a word, or a number. A long number of digits alone is a count or a time.
|
||||
return letters > 0 && hasUpperAndLower(s) && entropy(counts, len(s)) >= 4.0
|
||||
}
|
||||
return entropy(counts, len(s)) >= 3.3
|
||||
}
|
||||
|
||||
// looksRandomValue is looksRandom for a value in a file: letters and digits mixed and spread as a random string's
|
||||
// are; letters alone when the case changes as a random string's does — about as many capitals as small letters,
|
||||
// where a CamelCase name (sshd's PasswordAuthentication) has one a word.
|
||||
func looksRandomValue(s string) bool {
|
||||
var upper, lower, digits int
|
||||
counts := map[rune]int{}
|
||||
for _, r := range s {
|
||||
counts[r]++
|
||||
switch {
|
||||
case unicode.IsUpper(r):
|
||||
upper++
|
||||
case unicode.IsLower(r):
|
||||
lower++
|
||||
case unicode.IsDigit(r):
|
||||
digits++
|
||||
}
|
||||
}
|
||||
if upper+lower == 0 {
|
||||
return false
|
||||
}
|
||||
if digits > 0 {
|
||||
return entropy(counts, len(s)) >= 3.3
|
||||
}
|
||||
share := float64(upper) / float64(upper+lower)
|
||||
return share >= 0.3 && share <= 0.7 && entropy(counts, len(s)) >= 3.5
|
||||
}
|
||||
|
||||
func hasUpperAndLower(s string) bool {
|
||||
return strings.IndexFunc(s, unicode.IsUpper) >= 0 && strings.IndexFunc(s, unicode.IsLower) >= 0
|
||||
}
|
||||
|
||||
func entropy(counts map[rune]int, n int) float64 {
|
||||
e := 0.0
|
||||
for _, c := range counts {
|
||||
p := float64(c) / float64(n)
|
||||
e -= p * math.Log2(p)
|
||||
}
|
||||
return e
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
package outward
|
||||
|
||||
// The operator channel's cases, carried here with the rule (novox/hq ADR 0310 §4): the rule moved, its meaning did not.
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"math/rand"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRolesAndWordsMayLeave(t *testing.T) {
|
||||
for _, s := range []string{
|
||||
"URGENT: the home server has not been heard for 15 min",
|
||||
"key: provider.keycloak.ace.gitea.failing",
|
||||
"more: conditions show machine.ace.silent",
|
||||
"since: 2026-10-06 12:30 UTC",
|
||||
"call call-1759752000123456789-12 has run past its bound of 10 min",
|
||||
"ace/postgres.query answers no more",
|
||||
"3/4 machines answered; plan 41 waits on build 7f3a9c1e",
|
||||
"the bus advisory slow-consumer for ace_records",
|
||||
"core.controller.novox.rolled-back",
|
||||
"seat node-notifier has no live holder",
|
||||
"HELD BACK: 5 message(s) over the cap of 20 an hour",
|
||||
"STILL OPEN after 1.5 h: the controller's event loop takes no message",
|
||||
"node-engine and node tools builds differ from the plan's",
|
||||
} {
|
||||
if r, ok := Check(s); !ok {
|
||||
t.Errorf("refused %q: %s", s, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAddressAPathOrASecretMayNot(t *testing.T) {
|
||||
for s, class := range map[string]string{
|
||||
"cannot reach 192.168.1.20": "address",
|
||||
"listening on 10.0.0.7:5432": "address",
|
||||
"route fd00:1234:5678::1 is gone": "address",
|
||||
"fe80::1 answered": "address",
|
||||
"2001:db8:0:0:0:0:2:1 answered": "address",
|
||||
"see https://git.example.org/x": "address",
|
||||
"git.novox.internal does not answer": "address",
|
||||
"the mail for admin@example.org bounced": "address",
|
||||
"zurag.be is down": "address",
|
||||
"localhost refused": "address",
|
||||
"the card aa:bb:cc:dd:ee:ff went away": "address",
|
||||
"/var/lib/mesh-controller is full": "path",
|
||||
"~/.config/hal/env changed": "path",
|
||||
"read ./grants.json": "path",
|
||||
"services/postgres/data/pg_hba.conf": "path",
|
||||
"C:\\Users\\x": "path",
|
||||
"token 123456789:AAHdqTcvCH1vGWJxfSeofSAs0K5PALDsaw": "secret",
|
||||
"ghp_abcdefghijklmnopqrstuvwxyz0123456789": "secret",
|
||||
"password=hunter2": "secret",
|
||||
"api_key: x": "secret",
|
||||
"-----BEGIN OPENSSH PRIVATE KEY-----": "secret",
|
||||
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0": "secret",
|
||||
"commit 3f2a9c1e5b7d4f6a8c0e2b4d6f8a0c2e4b6d8f0a": "secret",
|
||||
"value Zx8Qp2Lm9Rt4Vb7Nc1Kd5Hs3Jf6": "secret",
|
||||
} {
|
||||
r, ok := Check(s)
|
||||
if ok {
|
||||
t.Errorf("let %q leave", s)
|
||||
continue
|
||||
}
|
||||
if r.Class != class {
|
||||
t.Errorf("%q refused as %s, want %s", s, r, class)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The secret class alone (novox/hq ADR 0310 §4, issue 383): a path and an address pass, a path's pieces are
|
||||
// judged between its slashes, and every named secret shape and a random run outside a path are refused.
|
||||
func TestCheckSecretRefusesASecretsShapeAndNothingElse(t *testing.T) {
|
||||
for text, may := range map[string]bool{
|
||||
"+ shares: photos=/mnt/Photos_2024/Jochen": true,
|
||||
"+ sources: games=smb://nas.lan/Recalbox_Games2024": true,
|
||||
"+ library: /srv/media/Series_Archive/Season01": true,
|
||||
"root:x:0:0::/root:/usr/bin/bash": true,
|
||||
"nameserver 192.168.1.1": true,
|
||||
"drwxr-x--- jochen users /home/jochen": true,
|
||||
"+ key: xK9mQ2vL8pR4tW7yB3nF6hJ1dG5sA0zC": false,
|
||||
"-----BEGIN OPENSSH PRIVATE KEY-----": false,
|
||||
"password=hunter2": false,
|
||||
"Authorization: token ghp_abcdefghijklmnopqrstuvwxyz0123456789": false,
|
||||
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0": false,
|
||||
"sha 3f2a9c1e5b7d4f6a8c0e2b4d6f8a0c2e4b6d8f0a": false,
|
||||
} {
|
||||
r, ok := CheckSecret(text)
|
||||
if ok != may {
|
||||
t.Errorf("CheckSecret(%q) = %v (%s), want %v", text, ok, r, may)
|
||||
}
|
||||
if !ok && r.Class != "secret" {
|
||||
t.Errorf("CheckSecret(%q) refused as %s: only the secret class is refused there", text, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The review of mesh-sdk #23: the shapes node-reads meets in env files, unit files, configurations and
|
||||
// history files, each refused by its shape whatever the value looks like.
|
||||
func TestCheckSecretRefusesTheShapesOfConfigurationFiles(t *testing.T) {
|
||||
for _, text := range []string{
|
||||
"url = nats://user:pa55word@host:4222",
|
||||
"DATABASE_URL=postgres://app:hunter2@db/app",
|
||||
"MESH_TOKEN=s3cr3t-v4lue",
|
||||
"DB_PASS=hunter2",
|
||||
"export GITEA_SECRET=abc",
|
||||
"Environment=APP_API_KEY=xyz",
|
||||
"SUDO_PASSWORD hunter2",
|
||||
"aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
"password = hunter2",
|
||||
"wifi_psk: correcthorse",
|
||||
"value: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
} {
|
||||
if r, ok := CheckSecret(text); ok || r.Class != "secret" {
|
||||
t.Errorf("CheckSecret(%q) let it through (%v)", text, r)
|
||||
}
|
||||
}
|
||||
// What a configuration file says that is not a secret.
|
||||
for _, text := range []string{
|
||||
"ExecStart=/usr/lib/systemd/systemd-resolved",
|
||||
"UUID=3f2a9c1e-5b7d-4f6a-8c0e-2b4d6f8a0c2e /home btrfs defaults 0 0",
|
||||
"password required pam_unix.so",
|
||||
"bypass = true",
|
||||
"PasswordAuthentication no",
|
||||
"url = https://git.example.org/novox/mesh-sdk.git",
|
||||
"HOME=/var/lib/mesh-reads",
|
||||
"TOKEN_FILE=/var/lib/app/token",
|
||||
} {
|
||||
if r, ok := CheckSecret(text); !ok {
|
||||
t.Errorf("CheckSecret(%q) withheld it: %s", text, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Random values as base64 and base64url, wherever their slashes fall: every one is refused.
|
||||
func TestCheckSecretRefusesRandomValuesAsBase64(t *testing.T) {
|
||||
rng := rand.New(rand.NewSource(418))
|
||||
passed := map[string]int{}
|
||||
for i := 0; i < 1000; i++ {
|
||||
for _, n := range []int{16, 32} {
|
||||
b := make([]byte, n)
|
||||
rng.Read(b)
|
||||
for name, enc := range map[string]*base64.Encoding{"std": base64.StdEncoding, "url": base64.RawURLEncoding} {
|
||||
v := enc.EncodeToString(b)
|
||||
if _, ok := CheckSecret("key " + v); ok {
|
||||
passed[fmt.Sprintf("%s/%d", name, n)]++
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(passed) > 0 {
|
||||
t.Errorf("random values passed as clean, of 1000 each: %v", passed)
|
||||
}
|
||||
}
|
||||
|
||||
// A CamelCase name is a word, not a value: the case changes once a word.
|
||||
func TestCheckSecretLetsACamelCaseNameThrough(t *testing.T) {
|
||||
for _, text := range []string{"PasswordAuthenticationIsOff", "ChallengeResponseAuthentication no", "KbdInteractiveAuthentication"} {
|
||||
if r, ok := CheckSecret(text); !ok {
|
||||
t.Errorf("CheckSecret(%q) withheld it: %s", text, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
// Package process is a module's own process's connection to the bus (novox/hq ADR 0436 clause 4): it reads
|
||||
// the account the controller delivers, connects as it with its pinned fingerprint and its inbox prefix, and
|
||||
// offers emitting an event, the module's states with compare-and-set, consuming a work queue, reading the
|
||||
// router's records of the module's own asks, and calling a tool or verb — each only as the account's grants
|
||||
// allow. It offers no Serve and no way to serve a tool: a process never answers a call. A sibling of stdio,
|
||||
// which it imports only for stdio's pure types (ErrChanged, CodeChanged, StateEntry, EmitOptions); it never
|
||||
// touches stdio's current, channel or live(), which stay the bundle's own way to the bus.
|
||||
package process
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
// Account is a process's own bus account, exactly as the controller delivers it as the module's own secret
|
||||
// broker-<process id> (mesh-controller cmd/mesh-controller/modules.go issueProcess, heldProcessAccount, and
|
||||
// issueWith for a module's own broker account, which also carries claims). A process account carries no
|
||||
// claims at all; this type reads either file without error.
|
||||
type Account struct {
|
||||
URL string `json:"url"`
|
||||
Fingerprint string `json:"fingerprint,omitempty"`
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
User string `json:"user"`
|
||||
Password string `json:"password"`
|
||||
// Claims rides on a module's own broker account only; a process holds no seat, so its own account
|
||||
// carries none (novox/hq ADR 0436 §2, §3). Kept raw: this package never reads it.
|
||||
Claims json.RawMessage `json:"claims,omitempty"`
|
||||
}
|
||||
|
||||
// InboxPrefix is this account's own inbox, and nothing wider: `_INBOX.<user>`.
|
||||
func (a Account) InboxPrefix() string { return "_INBOX." + a.User }
|
||||
|
||||
// ReadAccount reads and validates the account file the controller delivers: it refuses a file missing url,
|
||||
// user or password, naming the field — never the password, which does not appear in an error.
|
||||
func ReadAccount(path string) (Account, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return Account{}, fmt.Errorf("reading the process account at %s: %w", path, err)
|
||||
}
|
||||
var a Account
|
||||
if err := json.Unmarshal(data, &a); err != nil {
|
||||
return Account{}, fmt.Errorf("the process account at %s is not JSON: %w", path, err)
|
||||
}
|
||||
for _, f := range []struct{ name, value string }{
|
||||
{"url", a.URL}, {"user", a.User}, {"password", a.Password},
|
||||
} {
|
||||
if f.value == "" {
|
||||
return Account{}, fmt.Errorf("the process account at %s has no %s", path, f.name)
|
||||
}
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1171/AC2: given an account file broker-<process id> in the controller's form {url, fingerprint, node,
|
||||
// module, user, password, claims}, when the process package reads it, then it yields those fields and
|
||||
// tolerates claims.
|
||||
func TestReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
|
||||
path := writeAccount(t, map[string]any{
|
||||
"url": "nats://novox:4222", "fingerprint": "sha256:ab", "node": "novox", "module": "mesh-mail",
|
||||
"user": "novox.mesh-mail.send", "password": "secret-pw",
|
||||
"claims": []map[string]any{{"seat": "operator-channel", "verbs": []string{"tell"}}},
|
||||
})
|
||||
a, err := ReadAccount(path)
|
||||
if err != nil {
|
||||
t.Fatalf("a well-formed account file was refused: %v", err)
|
||||
}
|
||||
if a.URL != "nats://novox:4222" || a.Fingerprint != "sha256:ab" || a.Node != "novox" ||
|
||||
a.Module != "mesh-mail" || a.User != "novox.mesh-mail.send" || a.Password != "secret-pw" {
|
||||
t.Fatalf("the account read back: %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
// 1171/AC2: a process account carries no claims at all (heldProcessAccount); reading it is not refused for
|
||||
// the field being absent.
|
||||
func TestReadAccountToleratesNoClaimsAtAll(t *testing.T) {
|
||||
path := writeAccount(t, map[string]any{
|
||||
"url": "nats://novox:4222", "node": "novox", "module": "mesh-mail",
|
||||
"user": "novox.mesh-mail.send", "password": "secret-pw",
|
||||
})
|
||||
a, err := ReadAccount(path)
|
||||
if err != nil {
|
||||
t.Fatalf("a process account with no claims was refused: %v", err)
|
||||
}
|
||||
if a.User != "novox.mesh-mail.send" {
|
||||
t.Fatalf("the account read back: %+v", a)
|
||||
}
|
||||
}
|
||||
|
||||
// 1171/AC2: a file missing url, user or password is refused, naming the field; the password never appears
|
||||
// in the error even when it is the field present and another is missing.
|
||||
func TestReadAccountRefusesAFileMissingAField(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
field string
|
||||
values map[string]any
|
||||
}{
|
||||
{"url", map[string]any{"node": "novox", "module": "mesh-mail", "user": "novox.mesh-mail.send", "password": "secret-pw"}},
|
||||
{"user", map[string]any{"url": "nats://novox:4222", "node": "novox", "module": "mesh-mail", "password": "secret-pw"}},
|
||||
{"password", map[string]any{"url": "nats://novox:4222", "node": "novox", "module": "mesh-mail", "user": "novox.mesh-mail.send"}},
|
||||
} {
|
||||
t.Run(tc.field, func(t *testing.T) {
|
||||
path := writeAccount(t, tc.values)
|
||||
_, err := ReadAccount(path)
|
||||
if err == nil {
|
||||
t.Fatalf("an account file missing %s was accepted", tc.field)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.field) {
|
||||
t.Errorf("the error %q does not name the missing field %s", err.Error(), tc.field)
|
||||
}
|
||||
if strings.Contains(err.Error(), "secret-pw") {
|
||||
t.Errorf("the password appears in the error: %q", err.Error())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func writeAccount(t *testing.T, fields map[string]any) string {
|
||||
t.Helper()
|
||||
b, err := json.Marshal(fields)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "broker-send")
|
||||
if err := os.WriteFile(path, b, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// Ask reads the router's record of one of this process's module's own asks, under its own module and no
|
||||
// other's (novox/hq ADR 0436 clause 4, ADR 0259 §3): by direct get on bucket, the same subject the tool
|
||||
// runner reads (mesh-tools node-tools/internal/bus/seat.go ~300-345):
|
||||
//
|
||||
// $JS.API.DIRECT.GET.KV_<bucket>.$KV.<bucket>.<module>.<id>
|
||||
//
|
||||
// bucket is the router's records bucket as the caller's own manifest names it (operator-channel's is
|
||||
// messenger_asks in the controller's own test fixtures) — never hard-coded to one bucket here, since a
|
||||
// process may read records of more than one seat's router. A missing, deleted or purged key is nil, not
|
||||
// an error.
|
||||
func (c *Conn) Ask(bucket, id string) (*stdio.StateEntry, error) {
|
||||
if err := checkAskID(id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
subject := "$JS.API.DIRECT.GET.KV_" + bucket + ".$KV." + bucket + "." + c.account.Module + "." + id
|
||||
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
|
||||
defer cancel()
|
||||
reply, err := c.nc.RequestWithContext(ctx, subject, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading %s's ask %s from %s: %w", c.account.Module, id, bucket, err)
|
||||
}
|
||||
if status := reply.Header.Get("Status"); status == "404" {
|
||||
return nil, nil
|
||||
} else if status != "" {
|
||||
return nil, fmt.Errorf("the record %s.%s could not be read: %s %s", c.account.Module, id, status,
|
||||
reply.Header.Get("Description"))
|
||||
}
|
||||
if op := reply.Header.Get("KV-Operation"); op == "DEL" || op == "PURGE" {
|
||||
return nil, nil
|
||||
}
|
||||
var revision uint64
|
||||
if _, err := fmt.Sscan(reply.Header.Get("Nats-Sequence"), &revision); err != nil {
|
||||
return nil, fmt.Errorf("the record %s.%s carries no readable revision: %w", c.account.Module, id, err)
|
||||
}
|
||||
return &stdio.StateEntry{Key: c.account.Module + "." + id, Value: askValueOf(reply.Data), Revision: revision}, nil
|
||||
}
|
||||
|
||||
// checkAskID refuses an id holding a dot or a wildcard before anything is sent: either would name a
|
||||
// different key under the direct-get subject's own dot-joined path, never this one (novox/hq issue 1189
|
||||
// AC2).
|
||||
func checkAskID(id string) error {
|
||||
if id == "" || strings.ContainsAny(id, ".*>") {
|
||||
return fmt.Errorf("%q is not an ask id: an ask id holds no dot and no wildcard, naming one key and "+
|
||||
"no other", id)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// askValueOf is a record's raw bytes as JSON, quoting them when they are not already valid JSON
|
||||
// (mesh-tools node-tools/internal/bus/state.go:351-357, valueOf).
|
||||
func askValueOf(raw []byte) json.RawMessage {
|
||||
if len(raw) == 0 || !json.Valid(raw) {
|
||||
b, _ := json.Marshal(string(raw))
|
||||
return b
|
||||
}
|
||||
return json.RawMessage(append([]byte(nil), raw...))
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// 1189/AC1: given the router's records bucket on a test bus holding a record under <module>.<ask id>,
|
||||
// when the process connection reads its module's ask of that id, then it gets the record's bytes and
|
||||
// revision by direct get; a deleted or purged key is said as absent; a missing key as absent.
|
||||
func TestAskReadsTheRoutersRecordOfItsOwnAsk(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
kv, err := adminJetStream.CreateKeyValue(ctxFor(t), jetstream.KeyValueConfig{Bucket: "messenger_asks"})
|
||||
if err != nil {
|
||||
t.Fatalf("the admin could not make the records bucket: %v", err)
|
||||
}
|
||||
rev, err := kv.Put(ctxFor(t), "mesh-mail.ask-1", []byte(`{"state":"decided"}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
e, err := conn.Ask("messenger_asks", "ask-1")
|
||||
if err != nil {
|
||||
t.Fatalf("reading an existing record: %v", err)
|
||||
}
|
||||
if e == nil || e.Revision != rev || string(e.Value) != `{"state":"decided"}` {
|
||||
t.Fatalf("read back %+v, want revision %d and {\"state\":\"decided\"}", e, rev)
|
||||
}
|
||||
|
||||
if err := kv.Delete(ctxFor(t), "mesh-mail.ask-1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e, err := conn.Ask("messenger_asks", "ask-1"); err != nil || e != nil {
|
||||
t.Fatalf("a deleted record: %+v, %v, want absent", e, err)
|
||||
}
|
||||
|
||||
if _, err := kv.Create(ctxFor(t), "mesh-mail.ask-2", []byte(`{"state":"asked"}`)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := kv.Purge(ctxFor(t), "mesh-mail.ask-2"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e, err := conn.Ask("messenger_asks", "ask-2"); err != nil || e != nil {
|
||||
t.Fatalf("a purged record: %+v, %v, want absent", e, err)
|
||||
}
|
||||
|
||||
if e, err := conn.Ask("messenger_asks", "ask-missing"); err != nil || e != nil {
|
||||
t.Fatalf("a missing record: %+v, %v, want absent", e, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package process
|
||||
|
||||
import "testing"
|
||||
|
||||
// 1189/AC2: given an ask id holding a dot or a wildcard, when read, then it is refused before anything is
|
||||
// sent — proven on a Conn with no live connection at all: if Ask tried to send, it would panic on the nil
|
||||
// *nats.Conn instead of returning a clean error.
|
||||
func TestAskRefusesAnInvalidIDBeforeSendingAnything(t *testing.T) {
|
||||
c := &Conn{account: Account{Module: "mesh-mail"}}
|
||||
for _, id := range []string{"", "a.b", "a*", "a>", "ask.1", "ask.*"} {
|
||||
if _, err := c.Ask("messenger_asks", id); err == nil {
|
||||
t.Errorf("Ask(%q) was not refused", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCheckAskID(t *testing.T) {
|
||||
for _, id := range []string{"", "a.b", "a*b", "a>b", ".", "*"} {
|
||||
if err := checkAskID(id); err == nil {
|
||||
t.Errorf("checkAskID(%q) was accepted", id)
|
||||
}
|
||||
}
|
||||
for _, id := range []string{"ask-1", "a1b2c3", "send_42"} {
|
||||
if err := checkAskID(id); err != nil {
|
||||
t.Errorf("checkAskID(%q) was refused: %v", id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// CallTimeout bounds a call to a tool or a seat verb (mesh-tools node-tools/internal/bus/bus.go:35,
|
||||
// RequestTimeout): what a module already expects.
|
||||
var CallTimeout = 30 * time.Second
|
||||
|
||||
// Call invokes a tool or a seat verb as this process's own account (novox/hq ADR 0436 clause 4), on the
|
||||
// caller-named subject, falling back to the subject that names no caller when nothing answers it —
|
||||
// exactly the provisioner client's own pattern (mesh-catalog modules/mesh-provisioner-client/cmd/
|
||||
// mesh-provisioner-client/secrets.go:59-61, 93-97, 119, 125-133). key is "seat:<seat>.<verb>" for a seat's
|
||||
// verb, or "<module>.<tool>" for a module's own tool, as a process's bus.invokes names it. The reply is
|
||||
// {result, error, node} (mesh-tools node-tools/internal/bus/bus.go answerOf); a non-empty error is the
|
||||
// call's failure. It offers no way to answer a call — a process reaches the bus only as itself, and
|
||||
// never serves anything.
|
||||
func (c *Conn) Call(key string, body any) (json.RawMessage, error) {
|
||||
called, fallback, err := callSubjects(key, c.callerToken())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result, err := c.request(called, data)
|
||||
if errors.Is(err, nats.ErrNoResponders) {
|
||||
result, err = c.request(fallback, data)
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
|
||||
// callerToken is this account's user name as the last token of a call it publishes, each dot written ~
|
||||
// (mesh-tools node-tools/internal/bus/caller.go:43, CallerToken).
|
||||
func (c *Conn) callerToken() string { return strings.ReplaceAll(c.account.User, ".", "~") }
|
||||
|
||||
// callSubjects is the caller-named subject key is called on, and its plain fallback (mesh-tools
|
||||
// node-tools/internal/bus/caller.go CalledSubject, toolShape, calledPattern).
|
||||
func callSubjects(key, token string) (called, fallback string, err error) {
|
||||
if rest, ok := strings.CutPrefix(key, "seat:"); ok {
|
||||
seat, verb, ok := strings.Cut(rest, ".")
|
||||
if !ok || seat == "" || verb == "" {
|
||||
return "", "", fmt.Errorf("%q names no seat and verb (seat:<seat>.<verb>)", key)
|
||||
}
|
||||
return "mesh.seat." + seat + ".call." + verb + "." + token, "mesh.seat." + seat + ".tool." + verb, nil
|
||||
}
|
||||
module, tool, ok := strings.Cut(key, ".")
|
||||
if !ok || module == "" || tool == "" {
|
||||
return "", "", fmt.Errorf("%q is not <module>.<tool>, and does not start seat:<seat>.<verb>", key)
|
||||
}
|
||||
return "mesh.mod." + module + ".call." + tool + "." + token, "mesh.mod." + module + ".tool." + tool, nil
|
||||
}
|
||||
|
||||
// request asks one subject once and reads its reply ({result, error, node}); a reply naming an error is
|
||||
// the call's failure, not a transport one.
|
||||
func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), CallTimeout)
|
||||
defer cancel()
|
||||
msg, err := c.nc.RequestWithContext(ctx, subject, data)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var r struct {
|
||||
Result json.RawMessage `json:"result"`
|
||||
Error string `json:"error"`
|
||||
Node string `json:"node"`
|
||||
}
|
||||
if err := json.Unmarshal(msg.Data, &r); err != nil {
|
||||
return nil, fmt.Errorf("%s answered something that is not a seat's answer: %w", subject, err)
|
||||
}
|
||||
if r.Error != "" {
|
||||
return nil, errors.New(r.Error)
|
||||
}
|
||||
return r.Result, nil
|
||||
}
|
||||
@@ -0,0 +1,237 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// 1191/AC1: given a stand-in holder answering seat operator-channel verb tell on a test bus, when the
|
||||
// process connection of novox.mesh-mail.send calls seat:operator-channel.tell with {headline, lines,
|
||||
// subject}, then the request goes to mesh.seat.operator-channel.call.tell.novox~mesh-mail~send with the
|
||||
// arguments as the body, and the holder's {result} comes back; a reply with error set comes back as an
|
||||
// error with those words.
|
||||
func TestCallReachesTheCallerNamedSubjectAndReadsTheReply(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
adminNC, _ := adminJS(t, url)
|
||||
|
||||
heard := make(chan *nats.Msg, 1)
|
||||
sub, err := adminNC.Subscribe("mesh.seat.operator-channel.call.tell.novox~mesh-mail~send", func(m *nats.Msg) {
|
||||
heard <- m
|
||||
_ = m.Respond([]byte(`{"result":{"sent":true}}`))
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = sub.Unsubscribe() }()
|
||||
if err := adminNC.Flush(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
result, err := conn.Call("seat:operator-channel.tell",
|
||||
map[string]any{"headline": "h", "lines": []string{"l1"}, "subject": "s"})
|
||||
if err != nil {
|
||||
t.Fatalf("Call: %v", err)
|
||||
}
|
||||
if string(result) != `{"sent":true}` {
|
||||
t.Fatalf("result: %s", result)
|
||||
}
|
||||
|
||||
select {
|
||||
case msg := <-heard:
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(msg.Data, &body); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if body["headline"] != "h" || body["subject"] != "s" {
|
||||
t.Errorf("the arguments were not the call's body: %v", body)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the holder never heard the call")
|
||||
}
|
||||
}
|
||||
|
||||
// 1191/AC1: a reply with error set comes back as an error with those words.
|
||||
func TestCallReadsAnErrorReplyAsAnError(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
adminNC, _ := adminJS(t, url)
|
||||
sub, err := adminNC.Subscribe("mesh.seat.operator-channel.call.tell.novox~mesh-mail~send", func(m *nats.Msg) {
|
||||
_ = m.Respond([]byte(`{"error":"refused: no such warrant"}`))
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = sub.Unsubscribe() }()
|
||||
if err := adminNC.Flush(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
_, err = conn.Call("seat:operator-channel.tell", map[string]any{})
|
||||
if err == nil || err.Error() != "refused: no such warrant" {
|
||||
t.Fatalf("Call with an error reply: %v, want the words of the error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 1191/AC2: given no responder on the caller-named subject, when the call is made, then it is asked
|
||||
// again on mesh.seat.operator-channel.tool.tell (the tool runner's fallback).
|
||||
func TestCallFallsBackToThePlainSubjectWhenNothingAnswersTheCallerNamedOne(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
adminNC, _ := adminJS(t, url)
|
||||
sub, err := adminNC.Subscribe("mesh.seat.operator-channel.tool.tell", func(m *nats.Msg) {
|
||||
_ = m.Respond([]byte(`{"result":{"via":"fallback"}}`))
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = sub.Unsubscribe() }()
|
||||
if err := adminNC.Flush(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
result, err := conn.Call("seat:operator-channel.tell", map[string]any{})
|
||||
if err != nil {
|
||||
t.Fatalf("Call: %v", err)
|
||||
}
|
||||
if string(result) != `{"via":"fallback"}` {
|
||||
t.Fatalf("result: %s, want the fallback holder's answer", result)
|
||||
}
|
||||
}
|
||||
|
||||
// 1191/AC4, release 2's acceptance test: given the test bus with accounts composed as the controller
|
||||
// composes a process account (the subjects of mesh-controller internal/broker/process_grants.go), when
|
||||
// one process connection emits, updates a state, reads an own ask's record, consumes and acknowledges its
|
||||
// queue and calls tell, then each succeeds; and its attempt to subscribe a tool-call subject is refused
|
||||
// by the bus.
|
||||
func TestReleaseAcceptanceAProcessDoesExactlyItsBusDeclaration(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
|
||||
if _, err := adminJetStream.CreateKeyValue(ctxFor(t), jetstream.KeyValueConfig{Bucket: "mesh-mail_sends"}); err != nil {
|
||||
t.Fatalf("the admin could not make the state bucket: %v", err)
|
||||
}
|
||||
asksKV, err := adminJetStream.CreateKeyValue(ctxFor(t), jetstream.KeyValueConfig{Bucket: "messenger_asks"})
|
||||
if err != nil {
|
||||
t.Fatalf("the admin could not make the records bucket: %v", err)
|
||||
}
|
||||
if _, err := asksKV.Put(ctxFor(t), "mesh-mail.ask-1", []byte(`{"state":"decided"}`)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := adminJetStream.CreateStream(ctxFor(t), jetstream.StreamConfig{
|
||||
Name: "MAIL_EVENTS", Subjects: []string{"mesh.mod.mesh-mail.event.>"}, Duplicates: time.Minute,
|
||||
}); err != nil {
|
||||
t.Fatalf("the admin could not make the events stream: %v", err)
|
||||
}
|
||||
makeQueue(t, url, account, 30*time.Second)
|
||||
publishApproved(t, url, `{"send":"s1"}`)
|
||||
|
||||
adminNC, _ := adminJS(t, url)
|
||||
holderSub, err := adminNC.Subscribe("mesh.seat.operator-channel.call.tell.novox~mesh-mail~send", func(m *nats.Msg) {
|
||||
_ = m.Respond([]byte(`{"result":{}}`))
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = holderSub.Unsubscribe() }()
|
||||
if err := adminNC.Flush(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
// 1. Emits.
|
||||
if err := conn.Emit("mail.send.sent", map[string]any{"send": "s1"}, stdio.EmitOptions{MessageID: "ev-1"}); err != nil {
|
||||
t.Fatalf("Emit: %v", err)
|
||||
}
|
||||
|
||||
// 2. Updates a state.
|
||||
s := conn.State("sends")
|
||||
rev, err := s.Create("k", map[string]any{"v": 1})
|
||||
if err != nil {
|
||||
t.Fatalf("State.Create: %v", err)
|
||||
}
|
||||
if _, err := s.Update("k", map[string]any{"v": 2}, rev); err != nil {
|
||||
t.Fatalf("State.Update: %v", err)
|
||||
}
|
||||
|
||||
// 3. Reads an own ask's record.
|
||||
if e, err := conn.Ask("messenger_asks", "ask-1"); err != nil || e == nil {
|
||||
t.Fatalf("Ask: %+v, %v", e, err)
|
||||
}
|
||||
|
||||
// 4. Consumes and acknowledges its queue.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
consumed := make(chan error, 1)
|
||||
go func() {
|
||||
consumed <- conn.Consume(ctx, "MAIL_SENDS", func(m *Message) {
|
||||
if err := m.Ack(); err != nil {
|
||||
t.Errorf("Ack: %v", err)
|
||||
}
|
||||
cancel()
|
||||
})
|
||||
}()
|
||||
select {
|
||||
case <-consumed:
|
||||
case <-time.After(8 * time.Second):
|
||||
t.Fatal("the process did not consume its own queue")
|
||||
}
|
||||
|
||||
// 5. Calls tell.
|
||||
if _, err := conn.Call("seat:operator-channel.tell", map[string]any{}); err != nil {
|
||||
t.Fatalf("Call: %v", err)
|
||||
}
|
||||
|
||||
// 6. Refused: subscribing a tool-call subject — a fresh connection, so the error handler catches
|
||||
// exactly this one refusal and nothing earlier.
|
||||
refusals := make(chan error, 1)
|
||||
watched, err := Connect(account, nats.ErrorHandler(func(_ *nats.Conn, _ *nats.Subscription, err error) { refusals <- err }))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer watched.Close()
|
||||
if _, err := watched.nc.Subscribe("mesh.mod.mesh-mail.tool.>", func(*nats.Msg) {}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := watched.nc.Flush(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
select {
|
||||
case err := <-refusals:
|
||||
if !errors.Is(err, nats.ErrPermissionViolation) {
|
||||
t.Errorf("subscribing a tool-call subject was not refused as a permissions violation: %v", err)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Error("subscribing a tool-call subject the process does not serve was not refused")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package process
|
||||
|
||||
import "testing"
|
||||
|
||||
// 1191/AC3: given a module tool key <module>.<tool>, when called, then the subject is
|
||||
// mesh.mod.<module>.call.<tool>.<token> with the same fallback (mesh.mod.<module>.tool.<tool>).
|
||||
func TestCallSubjectsForAModuleTool(t *testing.T) {
|
||||
called, fallback, err := callSubjects("mesh-mail.send", "novox~mesh-mail~send")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if called != "mesh.mod.mesh-mail.call.send.novox~mesh-mail~send" {
|
||||
t.Errorf("called: %q", called)
|
||||
}
|
||||
if fallback != "mesh.mod.mesh-mail.tool.send" {
|
||||
t.Errorf("fallback: %q", fallback)
|
||||
}
|
||||
}
|
||||
|
||||
// 1191/AC1's subject (the seat form): seat:<seat>.<verb> calls mesh.seat.<seat>.call.<verb>.<token>,
|
||||
// falling back to mesh.seat.<seat>.tool.<verb>.
|
||||
func TestCallSubjectsForASeatVerb(t *testing.T) {
|
||||
called, fallback, err := callSubjects("seat:operator-channel.tell", "novox~mesh-mail~send")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if called != "mesh.seat.operator-channel.call.tell.novox~mesh-mail~send" {
|
||||
t.Errorf("called: %q", called)
|
||||
}
|
||||
if fallback != "mesh.seat.operator-channel.tool.tell" {
|
||||
t.Errorf("fallback: %q", fallback)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallSubjectsRefusesAKeyNamingNeither(t *testing.T) {
|
||||
for _, key := range []string{"", "nodot", "seat:", "seat:noverb", "."} {
|
||||
if _, _, err := callSubjects(key, "token"); err == nil {
|
||||
t.Errorf("callSubjects(%q) was accepted", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sync"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// Conn is one process's own connection to the bus (novox/hq ADR 0436 clause 4): dialled with the account
|
||||
// the controller delivered, pinned to its fingerprint, on its own inbox prefix. It offers no Serve and no
|
||||
// way to serve a tool — a process reaches the bus only as itself, and never answers a call.
|
||||
type Conn struct {
|
||||
nc *nats.Conn
|
||||
account Account
|
||||
|
||||
mu sync.Mutex
|
||||
next uint64
|
||||
pending map[string]map[uint64]context.CancelCauseFunc
|
||||
}
|
||||
|
||||
// errPermissionRefused marks a context cancelled because the bus refused this account's publish of the
|
||||
// subject a pending call was waiting an answer on: a denied publish is refused only as an async protocol
|
||||
// error with no reply (mesh-tools node-tools/internal/bus/caller.go publishRefused), so without this a
|
||||
// call would wait out its whole timeout instead of failing at once.
|
||||
var errPermissionRefused = errors.New("the bus refused this account's publish")
|
||||
|
||||
// refusedPublish reads the subject out of the bus's "-ERR 'Permissions Violation for Publish to
|
||||
// "<subject>"'" (mesh-tools node-tools/internal/bus/caller.go's own regexp).
|
||||
var refusedPublish = regexp.MustCompile(`(?i)Publish to "(\S+)"`)
|
||||
|
||||
// Connect dials the bus with account: pinned to its fingerprint when it carries one (refused with any
|
||||
// other certificate), authenticated as its user and password, on the inbox prefix _INBOX.<user> and no
|
||||
// other, reconnecting without limit as the tool runner does (novox/hq issue 1192: a process outlives a
|
||||
// bus restart, never giving up the way the client library's own default of 60 attempts would). Ported
|
||||
// from mesh-controller internal/broker/jetstream.go:106-132 (DialPinned, PinnedToFingerprint) since this
|
||||
// package may not import the controller.
|
||||
//
|
||||
// An ErrorHandler among opts is chained behind this package's own, which watches every async error for a
|
||||
// permission violation refusing a publish this connection is waiting an answer to, and cancels that wait
|
||||
// at once.
|
||||
func Connect(account Account, opts ...nats.Option) (*Conn, error) {
|
||||
c := &Conn{account: account, pending: map[string]map[uint64]context.CancelCauseFunc{}}
|
||||
nc, err := nats.Connect(account.URL, c.options(opts)...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("connecting the process account %s: %w", account.User, err)
|
||||
}
|
||||
c.nc = nc
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// options is the full option list Connect dials with, apart from Connect so a unit test can check what
|
||||
// it carries without a live bus: this package's own (the inbox prefix, the credentials, the pin,
|
||||
// reconnecting without limit, and an ErrorHandler chaining any the caller passed), then opts, then this
|
||||
// package's own ErrorHandler again last, so it is the one AsyncErrorCB nats.Connect keeps however opts
|
||||
// ordered theirs.
|
||||
func (c *Conn) options(opts []nats.Option) []nats.Option {
|
||||
// opts are functions over a *nats.Options; applied to a scratch value, this reads off a caller's own
|
||||
// ErrorHandler (if any) to chain, without needing nats.Connect's own URL/Servers parsing here.
|
||||
var scratch nats.Options
|
||||
for _, opt := range opts {
|
||||
if opt != nil {
|
||||
_ = opt(&scratch)
|
||||
}
|
||||
}
|
||||
chained := scratch.AsyncErrorCB
|
||||
|
||||
all := []nats.Option{
|
||||
nats.CustomInboxPrefix(c.account.InboxPrefix()),
|
||||
nats.UserInfo(c.account.User, c.account.Password),
|
||||
nats.MaxReconnects(-1),
|
||||
}
|
||||
if c.account.Fingerprint != "" {
|
||||
all = append(all, nats.Secure(pinnedToFingerprint(c.account.Fingerprint)))
|
||||
}
|
||||
all = append(all, opts...)
|
||||
return append(all, nats.ErrorHandler(func(nc *nats.Conn, sub *nats.Subscription, err error) {
|
||||
c.publishRefused(err)
|
||||
if chained != nil {
|
||||
chained(nc, sub, err)
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
// Close ends the connection.
|
||||
func (c *Conn) Close() { c.nc.Close() }
|
||||
|
||||
// withRefusal derives a context from ctx that is also cancelled, with errPermissionRefused as its cause,
|
||||
// the moment the bus refuses this account's publish of subject — before ctx's own deadline, which a
|
||||
// denied publish would otherwise run out, refused only as an async error with no reply. Every call that
|
||||
// requests on the bus registers the subject it is about to publish to here first.
|
||||
func (c *Conn) withRefusal(ctx context.Context, subject string) (context.Context, func()) {
|
||||
derived, cancel := context.WithCancelCause(ctx)
|
||||
c.mu.Lock()
|
||||
c.next++
|
||||
id := c.next
|
||||
if c.pending[subject] == nil {
|
||||
c.pending[subject] = map[uint64]context.CancelCauseFunc{}
|
||||
}
|
||||
c.pending[subject][id] = cancel
|
||||
c.mu.Unlock()
|
||||
return derived, func() {
|
||||
c.mu.Lock()
|
||||
if list := c.pending[subject]; list != nil {
|
||||
delete(list, id)
|
||||
if len(list) == 0 {
|
||||
delete(c.pending, subject)
|
||||
}
|
||||
}
|
||||
c.mu.Unlock()
|
||||
cancel(nil)
|
||||
}
|
||||
}
|
||||
|
||||
// publishRefused ends every call waiting on the subject the bus's error names, with errPermissionRefused
|
||||
// as the cause (mesh-tools node-tools/internal/bus/caller.go publishRefused): true when there was one
|
||||
// waiting.
|
||||
func (c *Conn) publishRefused(err error) bool {
|
||||
if !errors.Is(err, nats.ErrPermissionViolation) {
|
||||
return false
|
||||
}
|
||||
m := refusedPublish.FindStringSubmatch(err.Error())
|
||||
if len(m) < 2 {
|
||||
return false
|
||||
}
|
||||
c.mu.Lock()
|
||||
cancels := c.pending[m[1]]
|
||||
delete(c.pending, m[1])
|
||||
c.mu.Unlock()
|
||||
for _, cancel := range cancels {
|
||||
cancel(errPermissionRefused)
|
||||
}
|
||||
return len(cancels) > 0
|
||||
}
|
||||
|
||||
// pinnedToFingerprint accepts exactly the certificate with this SHA-256 and no other — the leaf alone, its
|
||||
// name and chain deliberately not consulted, as mesh-controller's own pin does (jetstream.go:116-132).
|
||||
func pinnedToFingerprint(want string) *tls.Config {
|
||||
return &tls.Config{
|
||||
InsecureSkipVerify: true, //nolint:gosec // replaced by the pin below, which is stricter
|
||||
MinVersion: tls.VersionTLS12,
|
||||
VerifyPeerCertificate: func(rawCerts [][]byte, _ [][]*x509.Certificate) error {
|
||||
if len(rawCerts) == 0 {
|
||||
return errors.New("the bus presented no certificate")
|
||||
}
|
||||
sum := sha256.Sum256(rawCerts[0])
|
||||
got := "sha256:" + hex.EncodeToString(sum[:])
|
||||
if got != want {
|
||||
return fmt.Errorf("the bus presented a certificate this mesh does not know (%s…), expected %s…",
|
||||
got[:23], want[:23])
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"math/big"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats-server/v2/server"
|
||||
)
|
||||
|
||||
// selfSignedServer is a nats-server with JetStream and TLS, self-signed, for this test alone (mirroring
|
||||
// go/internal/testbus.Start, with a certificate added since testbus has none): a mesh's bus presents its own
|
||||
// certificate in no public trust store, which is exactly what a process pins to (ADR 0436 clause 4).
|
||||
func selfSignedServer(t *testing.T) (url, fingerprint string) {
|
||||
t.Helper()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tmpl := &x509.Certificate{
|
||||
SerialNumber: big.NewInt(1),
|
||||
Subject: pkix.Name{CommonName: "mesh-sdk-test-bus"},
|
||||
NotBefore: time.Now().Add(-time.Hour),
|
||||
NotAfter: time.Now().Add(time.Hour),
|
||||
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sum := sha256.Sum256(der)
|
||||
fingerprint = "sha256:" + hex.EncodeToString(sum[:])
|
||||
cert := tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
|
||||
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
|
||||
NoLog: true, NoSigs: true, TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}},
|
||||
Username: "novox.mesh-mail.send", Password: "the-pw"}
|
||||
s, err := server.NewServer(opts)
|
||||
if err != nil {
|
||||
t.Fatalf("a TLS bus for this test could not be made: %v", err)
|
||||
}
|
||||
go s.Start()
|
||||
if !s.ReadyForConnections(10 * time.Second) {
|
||||
s.Shutdown()
|
||||
t.Fatal("a TLS bus for this test did not come up within 10s")
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
s.Shutdown()
|
||||
s.WaitForShutdown()
|
||||
})
|
||||
return s.ClientURL(), fingerprint
|
||||
}
|
||||
|
||||
// mailSenderAccount is the account novox.mesh-mail.send would be delivered as, the module's own secret
|
||||
// broker-send: written to a file and read back through ReadAccount (1171/AC2), exactly as a process
|
||||
// reads it, rather than built by hand.
|
||||
func mailSenderAccount(t *testing.T, url, fingerprint string) Account {
|
||||
t.Helper()
|
||||
data, err := json.Marshal(map[string]any{
|
||||
"url": url, "fingerprint": fingerprint, "node": "novox", "module": "mesh-mail",
|
||||
"user": "novox.mesh-mail.send", "password": "the-pw",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "broker-send")
|
||||
if err := os.WriteFile(path, data, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
account, err := ReadAccount(path)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadAccount on a well-formed file: %v", err)
|
||||
}
|
||||
return account
|
||||
}
|
||||
|
||||
// 1171/AC3: given a test bus with TLS and the account composed for novox.mesh-mail.send, when the package
|
||||
// connects with the file, then it connects with the inbox prefix _INBOX.novox.mesh-mail.send.
|
||||
func TestConnectPinsTheFingerprintAndSetsTheInboxPrefix(t *testing.T) {
|
||||
url, fingerprint := selfSignedServer(t)
|
||||
account := mailSenderAccount(t, url, fingerprint)
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatalf("connecting with the right fingerprint was refused: %v", err)
|
||||
}
|
||||
defer conn.Close()
|
||||
if !conn.nc.IsConnected() {
|
||||
t.Fatal("the connection did not come up")
|
||||
}
|
||||
if got := conn.nc.Opts.InboxPrefix; got != "_INBOX.novox.mesh-mail.send" {
|
||||
t.Errorf("inbox prefix: %q, want %q", got, "_INBOX.novox.mesh-mail.send")
|
||||
}
|
||||
}
|
||||
|
||||
// 1171/AC3: connecting with a fingerprint that is not the bus's own leaf certificate is refused.
|
||||
func TestConnectRefusesTheWrongFingerprint(t *testing.T) {
|
||||
url, _ := selfSignedServer(t)
|
||||
account := mailSenderAccount(t, url, "sha256:0000000000000000000000000000000000000000000000000000000000000000")
|
||||
if _, err := Connect(account); err == nil {
|
||||
t.Fatal("connecting with the wrong fingerprint was not refused")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// Emit publishes an event as this process's module (novox/hq ADR 0436 clause 4), under its module's own
|
||||
// namespace (mesh.mod.<module>.event.<type>, the same subject the module's own principal would publish it
|
||||
// on), with the SDK's own envelope and headers (stdio.go:498-509). A chosen message id
|
||||
// (EmitOptions.MessageID) is sent as x-event-id, and as the bus's own dedup id (Nats-Msg-Id), so a second
|
||||
// publish inside the stream's duplicate window is dropped; empty is a random one, as stdio.Emit on main.
|
||||
func (c *Conn) Emit(eventType string, body any, opts ...stdio.EmitOptions) error {
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id := ""
|
||||
for _, o := range opts {
|
||||
if o.MessageID != "" {
|
||||
id = o.MessageID
|
||||
}
|
||||
}
|
||||
if id == "" {
|
||||
id = eventID()
|
||||
}
|
||||
msg := nats.NewMsg("mesh.mod." + c.account.Module + ".event." + eventType)
|
||||
msg.Header.Set("x-event-id", id)
|
||||
msg.Header.Set("x-source", c.account.Module)
|
||||
msg.Header.Set("x-node", c.account.Node)
|
||||
msg.Header.Set("x-time", time.Now().UTC().Format(time.RFC3339Nano))
|
||||
msg.Header.Set("content-type", "application/json")
|
||||
for _, o := range opts {
|
||||
if o.CausationID != "" {
|
||||
msg.Header.Set("x-causation-id", o.CausationID)
|
||||
}
|
||||
if o.Schema != "" {
|
||||
msg.Header.Set("x-schema", o.Schema)
|
||||
}
|
||||
}
|
||||
msg.Data = data
|
||||
js, err := jetstream.New(c.nc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
|
||||
defer cancel()
|
||||
_, err = js.PublishMsg(ctx, msg, jetstream.WithMsgID(id))
|
||||
return err
|
||||
}
|
||||
|
||||
// eventID is a random UUID, version 4, as stdio's own eventID (go/stdio.go:515-522) — duplicated rather
|
||||
// than imported, since stdio's is unexported and this package touches none of stdio's bundle machinery.
|
||||
func eventID() string {
|
||||
var b [16]byte
|
||||
_, _ = rand.Read(b[:])
|
||||
b[6] = (b[6] & 0x0f) | 0x40
|
||||
b[8] = (b[8] & 0x3f) | 0x80
|
||||
h := hex.EncodeToString(b[:])
|
||||
return h[0:8] + "-" + h[8:12] + "-" + h[12:16] + "-" + h[16:20] + "-" + h[20:32]
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// 1188/AC3: given a stream accepting mail.send.sent, when the process emits it twice with the same
|
||||
// message id inside the duplicate window, then the stream holds one message; with the headers stdio's
|
||||
// Emit sets (stdio.go:498-509).
|
||||
func TestEmitTwiceWithTheSameIDIsKeptToOneWithTheSDKsHeaders(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
account.Node = "novox"
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
stream, err := adminJetStream.CreateStream(ctxFor(t), jetstream.StreamConfig{
|
||||
Name: "MAIL_EVENTS", Subjects: []string{"mesh.mod.mesh-mail.event.>"}, Duplicates: time.Minute,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("the admin could not make the events stream: %v", err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
for i := 0; i < 2; i++ {
|
||||
if err := conn.Emit("mail.send.sent", map[string]any{"send": "s1"}, stdio.EmitOptions{MessageID: "send-s1"}); err != nil {
|
||||
t.Fatalf("emit %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
info, err := stream.Info(ctxFor(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.State.Msgs != 1 {
|
||||
t.Fatalf("the stream holds %d messages, want exactly 1 (the second publish is a duplicate)", info.State.Msgs)
|
||||
}
|
||||
|
||||
msg, err := stream.GetMsg(ctxFor(t), 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if msg.Header.Get("x-event-id") != "send-s1" || msg.Header.Get("x-source") != "mesh-mail" ||
|
||||
msg.Header.Get("x-node") != "novox" || msg.Header.Get("content-type") != "application/json" ||
|
||||
msg.Header.Get("x-time") == "" {
|
||||
t.Errorf("the stored message's headers are not the SDK's: %v", msg.Header)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats-server/v2/server"
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// sendUser and sendPassword are mesh-mail's process "send" on novox, as mesh-controller would mint it
|
||||
// (mesh-controller internal/broker/process_grants_test.go mailSenderProcess, Principal.Username).
|
||||
const (
|
||||
sendUser = "novox.mesh-mail.send"
|
||||
sendPassword = "the-pw"
|
||||
adminUser = "admin"
|
||||
adminPass = "admin-pw"
|
||||
)
|
||||
|
||||
// sendGrants is the process principal mailSenderProcess's whole publish set, copied from
|
||||
// mesh-controller internal/broker/process_grants_test.go (TestAProcessIsGrantedExactlyItsBusDeclaration,
|
||||
// "dumped from a passing run... 16 subjects, correct today"), since this package may not import the
|
||||
// controller: the exact grant the controller composes for novox.mesh-mail.send.
|
||||
var sendGrants = []string{
|
||||
"$JS.ACK.MAIL_SENDS.novox_mesh-mail_send.>",
|
||||
"$JS.API.CONSUMER.CREATE.KV_mesh-mail_sends.>",
|
||||
"$JS.API.CONSUMER.DELETE.KV_mesh-mail_sends.>",
|
||||
"$JS.API.CONSUMER.INFO.MAIL_SENDS.novox_mesh-mail_send",
|
||||
"$JS.API.CONSUMER.MSG.NEXT.MAIL_SENDS.novox_mesh-mail_send",
|
||||
"$JS.API.DIRECT.GET.KV_mesh-mail_sends.>",
|
||||
"$JS.API.DIRECT.GET.KV_messenger_asks.$KV.messenger_asks.mesh-mail.>",
|
||||
"$JS.API.STREAM.INFO.KV_mesh-mail_sends",
|
||||
"$JS.FC.KV_mesh-mail_sends.>",
|
||||
"$KV.mesh-mail_sends.>",
|
||||
"mesh.mod.mesh-mail.event.mail.send.failed",
|
||||
"mesh.mod.mesh-mail.event.mail.send.sent",
|
||||
"mesh.seat.operator-channel.call.tell.*.novox~mesh-mail~send",
|
||||
"mesh.seat.operator-channel.call.tell.novox~mesh-mail~send",
|
||||
"mesh.seat.operator-channel.tool.tell",
|
||||
"mesh.seat.operator-channel.tool.tell.*",
|
||||
}
|
||||
|
||||
// grantedSendProcess is a test bus with JetStream, with two users: novox.mesh-mail.send, permissioned to
|
||||
// exactly sendGrants and its own inbox (nats-server server.Options Users with Permissions, as
|
||||
// mesh-controller's cmd/mesh-controller/process_live_test.go composes accounts for a real bus — copied as
|
||||
// literal grants here since this package may not import the controller); and an unrestricted admin for the
|
||||
// test's own setup (creating streams and buckets, standing in for the holder and the runtime).
|
||||
func grantedSendProcess(t *testing.T) (url string, account Account) {
|
||||
t.Helper()
|
||||
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
|
||||
NoLog: true, NoSigs: true,
|
||||
Users: []*server.User{
|
||||
{Username: adminUser, Password: adminPass},
|
||||
{Username: sendUser, Password: sendPassword, Permissions: &server.Permissions{
|
||||
Publish: &server.SubjectPermission{Allow: sendGrants},
|
||||
Subscribe: &server.SubjectPermission{Allow: []string{"_INBOX." + sendUser + ".>"}},
|
||||
}},
|
||||
},
|
||||
}
|
||||
s, err := server.NewServer(opts)
|
||||
if err != nil {
|
||||
t.Fatalf("a granted test bus could not be made: %v", err)
|
||||
}
|
||||
go s.Start()
|
||||
if !s.ReadyForConnections(10 * time.Second) {
|
||||
s.Shutdown()
|
||||
t.Fatal("a granted test bus did not come up within 10s")
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
s.Shutdown()
|
||||
s.WaitForShutdown()
|
||||
})
|
||||
url = s.ClientURL()
|
||||
return url, Account{URL: url, Node: "novox", Module: "mesh-mail", User: sendUser, Password: sendPassword}
|
||||
}
|
||||
|
||||
// adminJS is the test's own unrestricted connection to url, for setting up fixtures.
|
||||
func adminJS(t *testing.T, url string) (*nats.Conn, jetstream.JetStream) {
|
||||
t.Helper()
|
||||
nc, err := nats.Connect(url, nats.UserInfo(adminUser, adminPass), nats.CustomInboxPrefix("_INBOX."+adminUser))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(nc.Close)
|
||||
js, err := jetstream.New(nc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return nc, js
|
||||
}
|
||||
|
||||
func ctxFor(t *testing.T) context.Context {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
t.Cleanup(cancel)
|
||||
return ctx
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// 1171/AC3: it offers no Serve and no way to serve a tool — a process reaches the bus only as itself, and
|
||||
// never answers a call. Checked by reflection over *Conn's whole method set and field list, so a later
|
||||
// addition cannot bring one back without this test naming it.
|
||||
func TestConnOffersNoServeAndExposesNoRawConnection(t *testing.T) {
|
||||
ptrType := reflect.TypeOf(&Conn{})
|
||||
for _, forbidden := range []string{"Serve", "ServeOn", "Handle", "Subscribe", "Respond", "Answer"} {
|
||||
if _, ok := ptrType.MethodByName(forbidden); ok {
|
||||
t.Errorf("*Conn has a method %s: a process offers no way to serve a tool", forbidden)
|
||||
}
|
||||
}
|
||||
|
||||
natsConnType := reflect.TypeOf(&nats.Conn{})
|
||||
structType := reflect.TypeOf(Conn{})
|
||||
for i := 0; i < structType.NumField(); i++ {
|
||||
f := structType.Field(i)
|
||||
if f.IsExported() && (f.Type == natsConnType || f.Type == natsConnType.Elem()) {
|
||||
t.Errorf("Conn exposes its raw connection through the exported field %s", f.Name)
|
||||
}
|
||||
}
|
||||
for i := 0; i < ptrType.NumMethod(); i++ {
|
||||
m := ptrType.Method(i)
|
||||
for o := 0; o < m.Type.NumOut(); o++ {
|
||||
if m.Type.Out(o) == natsConnType || m.Type.Out(o) == natsConnType.Elem() {
|
||||
t.Errorf("*Conn.%s returns the raw *nats.Conn", m.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// Message is one piece of work this process's queue consumer delivered, with how many times the bus has
|
||||
// delivered it. Nothing acts on its behalf: the handler decides its fate exactly once, by calling Ack,
|
||||
// NakWithDelay or Term (mesh-tools node-tools/internal/bus/seat.go's seat work Consume, with the same
|
||||
// three outcomes, offered here explicitly instead of by a handler's return value, since the caller
|
||||
// chooses its own delay before a message comes back).
|
||||
type Message struct {
|
||||
Subject string
|
||||
Body []byte
|
||||
Headers map[string]string
|
||||
// Delivered is how many times the bus has handed this message over, this time included.
|
||||
Delivered uint64
|
||||
|
||||
ack func() error
|
||||
nak func(time.Duration) error
|
||||
term func() error
|
||||
}
|
||||
|
||||
// Ack acknowledges the message: the bus never delivers it again.
|
||||
func (m *Message) Ack() error { return m.ack() }
|
||||
|
||||
// NakWithDelay asks for the message to be delivered again no sooner than delay, its delivery count
|
||||
// raised by one.
|
||||
func (m *Message) NakWithDelay(delay time.Duration) error { return m.nak(delay) }
|
||||
|
||||
// Term terminates the message: the bus never delivers it again, and does not count it a dead letter the
|
||||
// way running out of deliveries does.
|
||||
func (m *Message) Term() error { return m.term() }
|
||||
|
||||
// ProcessID is this account's own process id, the last segment of its user name <node>.<module>.<id>
|
||||
// (mesh-controller internal/broker, Principal.Username for KindProcess).
|
||||
func (a Account) ProcessID() string {
|
||||
prefix := a.Node + "." + a.Module + "."
|
||||
if !strings.HasPrefix(a.User, prefix) {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimPrefix(a.User, prefix)
|
||||
}
|
||||
|
||||
// QueueConsumerName is the one durable consumer a process's bus.queues names it on its work-queue stream
|
||||
// (novox/hq ADR 0436 clauses 1, 2, 6; mesh-controller internal/broker/modstreams.go QueueConsumer):
|
||||
// <node>_<module>_<process id>.
|
||||
func (a Account) QueueConsumerName() string { return a.Node + "_" + a.Module + "_" + a.ProcessID() }
|
||||
|
||||
// errNoMessage is one pull that fetched nothing within its wait: not an error to the caller, just a
|
||||
// reason to pull again.
|
||||
var errNoMessage = errors.New("no message this pull")
|
||||
|
||||
// Consume binds this process's one durable consumer on stream (QueueConsumerName) and pulls from it, one
|
||||
// message at a time, handing each to handle until ctx ends: it then stops pulling and returns ctx.Err(),
|
||||
// never acknowledging a message still in handle's hands — only handle decides a message's fate, by
|
||||
// calling Ack, NakWithDelay or Term on it.
|
||||
func (c *Conn) Consume(ctx context.Context, stream string, handle func(*Message)) error {
|
||||
js, err := jetstream.New(c.nc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
name := c.account.QueueConsumerName()
|
||||
cons, err := js.Consumer(ctx, stream, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("binding %s's queue consumer %s on %s: %w", c.account.Module, name, stream, err)
|
||||
}
|
||||
return runQueue(ctx, fetchOne(cons), handle)
|
||||
}
|
||||
|
||||
// fetchOne is one pull of at most one message, non-blocking past ctx: Fetch itself does not take a
|
||||
// context, so this selects on its message channel and ctx together, and returns promptly once ctx ends
|
||||
// even while a fetch is still outstanding.
|
||||
func fetchOne(cons jetstream.Consumer) func(ctx context.Context) (*Message, error) {
|
||||
return func(ctx context.Context) (*Message, error) {
|
||||
batch, err := cons.Fetch(1, jetstream.FetchMaxWait(5*time.Second))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
select {
|
||||
case msg, ok := <-batch.Messages():
|
||||
if !ok {
|
||||
return nil, errNoMessage
|
||||
}
|
||||
headers := map[string]string{}
|
||||
for k := range msg.Headers() {
|
||||
headers[k] = msg.Headers().Get(k)
|
||||
}
|
||||
var delivered uint64 = 1
|
||||
if meta, err := msg.Metadata(); err == nil && meta != nil {
|
||||
delivered = meta.NumDelivered
|
||||
}
|
||||
return &Message{Subject: msg.Subject(), Body: append([]byte(nil), msg.Data()...), Headers: headers,
|
||||
Delivered: delivered, ack: msg.Ack, nak: msg.NakWithDelay, term: msg.Term}, nil
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// runQueue is Consume's loop, apart from the bus so a unit test can drive it with a fake next
|
||||
// (novox/hq issue 1190 AC4): it checks ctx before every pull, and stops as soon as next or the check
|
||||
// says ctx has ended, without touching the message past handing it to handle.
|
||||
func runQueue(ctx context.Context, next func(context.Context) (*Message, error), handle func(*Message)) error {
|
||||
for {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
msg, err := next(ctx)
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
if errors.Is(err, errNoMessage) {
|
||||
continue
|
||||
}
|
||||
return err
|
||||
}
|
||||
handle(msg)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,223 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
)
|
||||
|
||||
// mailSendsApprovedSubject is where the runtime publishes a work-queue message on mesh-mail's own stream
|
||||
// (mesh-controller internal/broker/modstreams.go ModuleStreamSubject, mail-sends' one event).
|
||||
const mailSendsApprovedSubject = "mesh.stream.mesh-mail.event.mail.send.approved"
|
||||
|
||||
// makeQueue sets up MAIL_SENDS as a work-queue stream and binds the one durable consumer a process's
|
||||
// bus.queues names it (QueueConsumerName), both as the admin, mirroring what the controller composes
|
||||
// (mesh-controller internal/broker/modstreams.go QueueConsumer), named <node>_<module>_<process id>.
|
||||
// ackWait is the controller's 30s in most tests, shortened in the Term test so waiting past it (to prove
|
||||
// no redelivery) does not make that test slow.
|
||||
func makeQueue(t *testing.T, url string, account Account, ackWait time.Duration) {
|
||||
t.Helper()
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
if _, err := adminJetStream.CreateStream(ctxFor(t), jetstream.StreamConfig{
|
||||
Name: "MAIL_SENDS", Subjects: []string{mailSendsApprovedSubject}, Retention: jetstream.WorkQueuePolicy,
|
||||
Discard: jetstream.DiscardNew,
|
||||
}); err != nil {
|
||||
t.Fatalf("the admin could not make the work-queue stream: %v", err)
|
||||
}
|
||||
if _, err := adminJetStream.CreateConsumer(ctxFor(t), "MAIL_SENDS", jetstream.ConsumerConfig{
|
||||
Durable: account.QueueConsumerName(), AckPolicy: jetstream.AckExplicitPolicy, AckWait: ackWait,
|
||||
MaxDeliver: 5,
|
||||
}); err != nil {
|
||||
t.Fatalf("the admin could not make the queue consumer: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func publishApproved(t *testing.T, url string, body string) {
|
||||
t.Helper()
|
||||
nc, _ := adminJS(t, url)
|
||||
if err := nc.Publish(mailSendsApprovedSubject, []byte(body)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := nc.FlushTimeout(3 * time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// recv reads one value off ch, failing the test rather than hanging forever if nothing arrives within
|
||||
// timeout.
|
||||
func recv[T any](t *testing.T, ch <-chan T, timeout time.Duration, what string) T {
|
||||
t.Helper()
|
||||
select {
|
||||
case v := <-ch:
|
||||
return v
|
||||
case <-time.After(timeout):
|
||||
t.Fatalf("%s: nothing arrived within %s", what, timeout)
|
||||
var zero T
|
||||
return zero
|
||||
}
|
||||
}
|
||||
|
||||
// streamMsgs is how many messages MAIL_SENDS still holds.
|
||||
func streamMsgs(t *testing.T, url string) uint64 {
|
||||
t.Helper()
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
stream, err := adminJetStream.Stream(ctxFor(t), "MAIL_SENDS")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, err := stream.Info(ctxFor(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return info.State.Msgs
|
||||
}
|
||||
|
||||
// 1190/AC1: given a work-queue stream with one durable pull consumer and a process connection, when a
|
||||
// message is consumed and acknowledged, then it is gone from the stream.
|
||||
func TestConsumeAckRemovesTheMessageFromTheStream(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
makeQueue(t, url, account, 30*time.Second)
|
||||
publishApproved(t, url, `{"send":"s1"}`)
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
got := make(chan *Message, 1)
|
||||
go func() {
|
||||
_ = conn.Consume(ctx, "MAIL_SENDS", func(m *Message) {
|
||||
got <- m
|
||||
cancel()
|
||||
})
|
||||
}()
|
||||
msg := recv(t, got, 8*time.Second, "the message")
|
||||
<-ctx.Done()
|
||||
if err := msg.Ack(); err != nil {
|
||||
t.Fatalf("Ack: %v", err)
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
if n := streamMsgs(t, url); n == 0 {
|
||||
return
|
||||
} else if time.Now().After(deadline) {
|
||||
t.Fatalf("the stream still holds %d messages after the ack", n)
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
// 1190/AC2: given a consumed message, when the handler asks for it to be delivered again after a delay,
|
||||
// then it comes back no sooner than the delay with its delivery count raised by one.
|
||||
func TestConsumeNakWithDelayRedeliversAfterTheDelayWithARaisedCount(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
makeQueue(t, url, account, 30*time.Second)
|
||||
publishApproved(t, url, `{"send":"s2"}`)
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
const delay = 2 * time.Second
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
defer cancel()
|
||||
type delivery struct {
|
||||
at time.Time
|
||||
delivered uint64
|
||||
}
|
||||
deliveries := make(chan delivery, 2)
|
||||
go func() {
|
||||
_ = conn.Consume(ctx, "MAIL_SENDS", func(m *Message) {
|
||||
deliveries <- delivery{time.Now(), m.Delivered}
|
||||
if m.Delivered == 1 {
|
||||
_ = m.NakWithDelay(delay)
|
||||
} else {
|
||||
_ = m.Ack()
|
||||
cancel()
|
||||
}
|
||||
})
|
||||
}()
|
||||
|
||||
first := recv(t, deliveries, 8*time.Second, "the first delivery")
|
||||
if first.delivered != 1 {
|
||||
t.Fatalf("first delivery count: %d, want 1", first.delivered)
|
||||
}
|
||||
second := recv(t, deliveries, delay+8*time.Second, "the redelivery")
|
||||
if second.delivered != 2 {
|
||||
t.Fatalf("second delivery count: %d, want 2", second.delivered)
|
||||
}
|
||||
if elapsed := second.at.Sub(first.at); elapsed < delay {
|
||||
t.Fatalf("redelivered after %s, want no sooner than %s", elapsed, delay)
|
||||
}
|
||||
}
|
||||
|
||||
// 1190/AC3: given a consumed message, when the handler terminates it, then it is never delivered again.
|
||||
// A short AckWait (well under the 4s this then waits past) is what would otherwise make this pass
|
||||
// whether or not Term did anything at all: without it, the message simply sits unredelivered until the
|
||||
// controller's ordinary 30s AckWait, and the test would see the same "nothing arrived" result either way.
|
||||
func TestConsumeTermNeverDeliversTheMessageAgain(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
const ackWait = time.Second
|
||||
makeQueue(t, url, account, ackWait)
|
||||
publishApproved(t, url, `{"send":"s3"}`)
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
deliveries := make(chan *Message, 4)
|
||||
go func() {
|
||||
_ = conn.Consume(ctx, "MAIL_SENDS", func(m *Message) {
|
||||
deliveries <- m
|
||||
_ = m.Term()
|
||||
})
|
||||
}()
|
||||
|
||||
first := recv(t, deliveries, 8*time.Second, "the message")
|
||||
if first.Delivered != 1 {
|
||||
t.Fatalf("first delivery count: %d, want 1", first.Delivered)
|
||||
}
|
||||
|
||||
// Term removes it from the work-queue stream and leaves the consumer with nothing pending — not just
|
||||
// "has not come back yet", which a message merely sitting past an un-acted AckWait would also show.
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
if n := streamMsgs(t, url); n == 0 {
|
||||
break
|
||||
} else if time.Now().After(deadline) {
|
||||
t.Fatalf("the stream still holds %d messages after Term", n)
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
}
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
cons, err := adminJetStream.Consumer(ctxFor(t), "MAIL_SENDS", account.QueueConsumerName())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, err := cons.Info(ctxFor(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.NumAckPending != 0 {
|
||||
t.Fatalf("the consumer still has %d acks pending after Term", info.NumAckPending)
|
||||
}
|
||||
|
||||
select {
|
||||
case again := <-deliveries:
|
||||
t.Fatalf("a terminated message was delivered again: %+v", again)
|
||||
case <-time.After(4 * ackWait):
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// 1190/AC4: given the consume loop, when its context ends, then it stops pulling and returns without
|
||||
// acknowledging a message in hand.
|
||||
func TestRunQueueStopsPullingAndNeverActsOnAMessageWhenContextEnds(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
|
||||
acked := false
|
||||
msg := &Message{Subject: "test", ack: func() error { acked = true; return nil },
|
||||
nak: func(time.Duration) error { return nil }, term: func() error { return nil }}
|
||||
|
||||
pulls := 0
|
||||
next := func(ctx context.Context) (*Message, error) {
|
||||
pulls++
|
||||
return msg, nil
|
||||
}
|
||||
|
||||
handled := 0
|
||||
done := make(chan error, 1)
|
||||
go func() {
|
||||
done <- runQueue(ctx, next, func(m *Message) {
|
||||
handled++
|
||||
// The handler does not act on the message at all: a redelivery, a termination or an
|
||||
// acknowledgement is its choice alone, never the loop's.
|
||||
cancel()
|
||||
})
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("runQueue returned %v, want context.Canceled", err)
|
||||
}
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("runQueue did not stop when its context ended")
|
||||
}
|
||||
if handled != 1 {
|
||||
t.Fatalf("handle was called %d times, want 1", handled)
|
||||
}
|
||||
if acked {
|
||||
t.Error("runQueue acknowledged the message itself; only the handler may")
|
||||
}
|
||||
if pulls != 1 {
|
||||
t.Fatalf("next was called %d times, want exactly 1: once the handler ends the context, runQueue's "+
|
||||
"own check stops it before pulling again", pulls)
|
||||
}
|
||||
}
|
||||
|
||||
// 1190/AC4: a context already ended before the loop starts pulls nothing at all.
|
||||
func TestRunQueueNeverPullsOnAnAlreadyEndedContext(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
calls := 0
|
||||
err := runQueue(ctx, func(context.Context) (*Message, error) {
|
||||
calls++
|
||||
return nil, context.Canceled
|
||||
}, func(*Message) { t.Fatal("handle was called on an already-ended context") })
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("runQueue returned %v, want context.Canceled", err)
|
||||
}
|
||||
if calls != 0 {
|
||||
t.Fatalf("next was called %d times, want 0", calls)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
)
|
||||
|
||||
// novox/hq issue 1192 (taken into this round, release 2's reviewer): Connect reconnects without limit, as
|
||||
// the tool runner does, so a process outlives a bus restart instead of giving up after the client
|
||||
// library's default of 60 attempts. Checked without a live bus: the option list Connect dials with is
|
||||
// applied to a scratch nats.Options and MaxReconnect read back.
|
||||
func TestConnectOptionsReconnectWithoutLimit(t *testing.T) {
|
||||
c := &Conn{account: Account{Node: "novox", Module: "mesh-mail", User: "novox.mesh-mail.send", Password: "pw"}}
|
||||
var o nats.Options
|
||||
for _, opt := range c.options(nil) {
|
||||
if opt == nil {
|
||||
continue
|
||||
}
|
||||
if err := opt(&o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if o.MaxReconnect != -1 {
|
||||
t.Errorf("MaxReconnect: %d, want -1 (no limit)", o.MaxReconnect)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// stateTimeout bounds a state call (Get, Create, Update) and an Emit (emit.go): a denied publish is
|
||||
// caught at once by Conn's own ErrorHandler (conn.go withRefusal/publishRefused), never waited out, so
|
||||
// this stays the ordinary request bound and is not exposed for a test to shrink.
|
||||
const stateTimeout = 30 * time.Second
|
||||
|
||||
// State is one of this process's module's states, read and written on this connection (novox/hq ADR 0436
|
||||
// clause 4), in the bucket <module>_<name> (mesh-controller internal/broker/state.go:49) — only as the
|
||||
// account's grants allow. It satisfies go/asker's Store.
|
||||
type State struct {
|
||||
Module string
|
||||
Name string
|
||||
// kv binds this state's underlying key-value store: a live connection's own (Conn.State), or, in a
|
||||
// unit test, a fake that never reaches a bus.
|
||||
kv func(ctx context.Context) (jetstream.KeyValue, error)
|
||||
}
|
||||
|
||||
// bucket is this state's name on the bus.
|
||||
func (s State) bucket() string { return s.Module + "_" + s.Name }
|
||||
|
||||
// State is the named state this connection's module keeps, bound lazily to its bucket on each call so a
|
||||
// bucket the account is not granted fails at the call that needed it, naming the bucket. The bind itself
|
||||
// (js.KeyValue) is what asks $JS.API.STREAM.INFO.KV_<bucket>, so it is what withRefusal watches: a bucket
|
||||
// not granted is refused there, promptly, as nats.ErrPermissionViolation.
|
||||
func (c *Conn) State(name string) State {
|
||||
bucket := c.account.Module + "_" + name
|
||||
return State{Module: c.account.Module, Name: name, kv: func(ctx context.Context) (jetstream.KeyValue, error) {
|
||||
js, err := jetstream.New(c.nc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
subject := "$JS.API.STREAM.INFO.KV_" + bucket
|
||||
ctx, stop := c.withRefusal(ctx, subject)
|
||||
defer stop()
|
||||
kv, err := js.KeyValue(ctx, bucket)
|
||||
if err != nil {
|
||||
if errors.Is(context.Cause(ctx), errPermissionRefused) {
|
||||
return nil, fmt.Errorf("%w: publishing %s", nats.ErrPermissionViolation, subject)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return kv, nil
|
||||
}}
|
||||
}
|
||||
|
||||
func (s State) store(ctx context.Context) (jetstream.KeyValue, error) {
|
||||
kv, err := s.kv(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s's state %s (bucket %s): %w", s.Module, s.Name, s.bucket(), err)
|
||||
}
|
||||
return kv, nil
|
||||
}
|
||||
|
||||
// Get is a key's current value, or nil when it has none.
|
||||
func (s State) Get(key string) (*stdio.StateEntry, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
|
||||
defer cancel()
|
||||
kv, err := s.store(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entry, err := kv.Get(ctx, key)
|
||||
if errors.Is(err, jetstream.ErrKeyNotFound) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
|
||||
}
|
||||
return &stdio.StateEntry{Key: key, Value: json.RawMessage(entry.Value()), Revision: entry.Revision()}, nil
|
||||
}
|
||||
|
||||
// Create writes a key only when it has none, and returns the revision; a key that has one is refused as
|
||||
// stdio.ErrChanged.
|
||||
func (s State) Create(key string, value any) (uint64, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
|
||||
defer cancel()
|
||||
kv, err := s.store(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
data, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
rev, err := kv.Create(ctx, key, data)
|
||||
if errors.Is(err, jetstream.ErrKeyExists) {
|
||||
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
|
||||
}
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
|
||||
}
|
||||
return rev, nil
|
||||
}
|
||||
|
||||
// Update writes a key only when it is still at revision, and returns the new one; a key changed meanwhile
|
||||
// is refused as stdio.ErrChanged.
|
||||
func (s State) Update(key string, value any, revision uint64) (uint64, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
|
||||
defer cancel()
|
||||
kv, err := s.store(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
data, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
rev, err := kv.Update(ctx, key, data, revision)
|
||||
if errors.Is(err, jetstream.ErrKeyRevisionMismatch) {
|
||||
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
|
||||
}
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
|
||||
}
|
||||
return rev, nil
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
//go:build integration
|
||||
|
||||
package process
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go"
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// 1188/AC1: given a process connection to a test bus granted the bucket mesh-mail_sends, when it creates a
|
||||
// key, reads it, and updates it with the revision it read, then each succeeds and returns the new
|
||||
// revision; an update with a stale revision fails as stdio.ErrChanged; a create of an existing key fails
|
||||
// as stdio.ErrChanged.
|
||||
func TestStateCreateGetUpdateOnAGrantedBucket(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
if _, err := adminJetStream.CreateKeyValue(ctxFor(t), jetstream.KeyValueConfig{Bucket: "mesh-mail_sends"}); err != nil {
|
||||
t.Fatalf("the admin could not make the bucket: %v", err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
s := conn.State("sends")
|
||||
|
||||
rev, err := s.Create("key", map[string]any{"v": 1})
|
||||
if err != nil {
|
||||
t.Fatalf("Create on a granted bucket: %v", err)
|
||||
}
|
||||
if rev == 0 {
|
||||
t.Fatal("Create returned no revision")
|
||||
}
|
||||
|
||||
e, err := s.Get("key")
|
||||
if err != nil || e == nil {
|
||||
t.Fatalf("Get after Create: %+v, %v", e, err)
|
||||
}
|
||||
if e.Revision != rev || string(e.Value) != `{"v":1}` {
|
||||
t.Fatalf("Get read back %+v, want revision %d and {\"v\":1}", e, rev)
|
||||
}
|
||||
|
||||
rev2, err := s.Update("key", map[string]any{"v": 2}, rev)
|
||||
if err != nil {
|
||||
t.Fatalf("Update at the revision read: %v", err)
|
||||
}
|
||||
if rev2 == rev {
|
||||
t.Fatal("Update did not return a new revision")
|
||||
}
|
||||
|
||||
if _, err := s.Update("key", map[string]any{"v": 3}, rev); !errors.Is(err, stdio.ErrChanged) {
|
||||
t.Fatalf("Update at a stale revision: %v, want stdio.ErrChanged", err)
|
||||
}
|
||||
|
||||
if _, err := s.Create("key", map[string]any{"v": 4}); !errors.Is(err, stdio.ErrChanged) {
|
||||
t.Fatalf("Create of an existing key: %v, want stdio.ErrChanged", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 1188/AC4: given an account not granted a bucket, when it writes there, then the bus's refusal comes
|
||||
// back as an error naming the bucket. The bucket exists (made by the admin) so the only reason Create
|
||||
// can fail is the missing grant, never "bucket not found"; and Conn's own ErrorHandler (conn.go
|
||||
// withRefusal/publishRefused) catches the denied publish at once, so this returns promptly rather than
|
||||
// after a timeout.
|
||||
func TestStateRefusesAnUngrantedBucketNamingIt(t *testing.T) {
|
||||
url, account := grantedSendProcess(t)
|
||||
_, adminJetStream := adminJS(t, url)
|
||||
if _, err := adminJetStream.CreateKeyValue(ctxFor(t), jetstream.KeyValueConfig{Bucket: "mesh-mail_unrelated"}); err != nil {
|
||||
t.Fatalf("the admin could not make the bucket: %v", err)
|
||||
}
|
||||
|
||||
conn, err := Connect(account)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
s := conn.State("unrelated")
|
||||
_, err = s.Create("key", map[string]any{"v": 1})
|
||||
if err == nil {
|
||||
t.Fatal("writing to an ungranted bucket was not refused")
|
||||
}
|
||||
if !errors.Is(err, nats.ErrPermissionViolation) {
|
||||
t.Errorf("the refusal %v does not wrap nats.ErrPermissionViolation", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "mesh-mail_unrelated") {
|
||||
t.Errorf("the refusal %q does not name the bucket mesh-mail_unrelated", err.Error())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package process
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/nats-io/nats.go/jetstream"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
"git.novox.be/novox/mesh-sdk/go/asker"
|
||||
)
|
||||
|
||||
// fakeEntry is a fakeKV's KeyValueEntry: only Key, Value and Revision are read by State.
|
||||
type fakeEntry struct {
|
||||
jetstream.KeyValueEntry
|
||||
key string
|
||||
value []byte
|
||||
rev uint64
|
||||
}
|
||||
|
||||
func (e fakeEntry) Key() string { return e.key }
|
||||
func (e fakeEntry) Value() []byte { return e.value }
|
||||
func (e fakeEntry) Revision() uint64 { return e.rev }
|
||||
|
||||
// fakeKV is an in-memory jetstream.KeyValue: embedding the nil interface satisfies every method this test
|
||||
// never calls, so only Get, Create and Update need a body (1188/AC2's "a fake KV").
|
||||
type fakeKV struct {
|
||||
jetstream.KeyValue
|
||||
mu sync.Mutex
|
||||
values map[string][]byte
|
||||
revs map[string]uint64
|
||||
next uint64
|
||||
}
|
||||
|
||||
func newFakeKV() *fakeKV { return &fakeKV{values: map[string][]byte{}, revs: map[string]uint64{}} }
|
||||
|
||||
func (f *fakeKV) Get(_ context.Context, key string) (jetstream.KeyValueEntry, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
v, ok := f.values[key]
|
||||
if !ok {
|
||||
return nil, jetstream.ErrKeyNotFound
|
||||
}
|
||||
return fakeEntry{key: key, value: v, rev: f.revs[key]}, nil
|
||||
}
|
||||
|
||||
func (f *fakeKV) Create(_ context.Context, key string, value []byte, _ ...jetstream.KVCreateOpt) (uint64, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if _, ok := f.values[key]; ok {
|
||||
return 0, jetstream.ErrKeyExists
|
||||
}
|
||||
f.next++
|
||||
f.values[key], f.revs[key] = value, f.next
|
||||
return f.next, nil
|
||||
}
|
||||
|
||||
func (f *fakeKV) Update(_ context.Context, key string, value []byte, revision uint64) (uint64, error) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if f.revs[key] != revision {
|
||||
return 0, jetstream.ErrKeyRevisionMismatch
|
||||
}
|
||||
f.next++
|
||||
f.values[key], f.revs[key] = value, f.next
|
||||
return f.next, nil
|
||||
}
|
||||
|
||||
func stateOn(kv jetstream.KeyValue) State {
|
||||
return State{Module: "mesh-mail", Name: "sends", kv: func(context.Context) (jetstream.KeyValue, error) { return kv, nil }}
|
||||
}
|
||||
|
||||
// 1188/AC2: given the process connection's state type, when used as go/asker's Store, then it compiles and
|
||||
// behaves as the interface says, against a fake KV.
|
||||
func TestStateSatisfiesAskerStore(t *testing.T) {
|
||||
var _ asker.Store = State{}
|
||||
|
||||
s := stateOn(newFakeKV())
|
||||
if e, err := s.Get("ask.1"); err != nil || e != nil {
|
||||
t.Fatalf("Get of an absent key: %v, %v", e, err)
|
||||
}
|
||||
rev, err := s.Create("ask.1", map[string]any{"a": 1})
|
||||
if err != nil {
|
||||
t.Fatalf("Create: %v", err)
|
||||
}
|
||||
e, err := s.Get("ask.1")
|
||||
if err != nil || e == nil || e.Revision != rev {
|
||||
t.Fatalf("Get after Create: %+v, %v", e, err)
|
||||
}
|
||||
if _, err := s.Create("ask.1", map[string]any{"a": 2}); !errors.Is(err, stdio.ErrChanged) {
|
||||
t.Fatalf("Create of an existing key: %v, want stdio.ErrChanged", err)
|
||||
}
|
||||
rev2, err := s.Update("ask.1", map[string]any{"a": 3}, rev)
|
||||
if err != nil {
|
||||
t.Fatalf("Update at the revision read: %v", err)
|
||||
}
|
||||
if _, err := s.Update("ask.1", map[string]any{"a": 4}, rev); !errors.Is(err, stdio.ErrChanged) {
|
||||
t.Fatalf("Update at a stale revision: %v, want stdio.ErrChanged", err)
|
||||
}
|
||||
if rev2 == rev {
|
||||
t.Fatal("Update did not advance the revision")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
)
|
||||
|
||||
// The provider's consumers' credentials, from the file its machine's provisioner client fills (novox/hq ADR 0373,
|
||||
// issue 679).
|
||||
//
|
||||
// **The provider never asks the vault.** A provider in the tool runner has no account of its own, and the vault
|
||||
// hands a secret only to a node's provisioner client or to a module under an account of its own (ADR 0349 clause
|
||||
// 11, ADR 0370 clause 2). So the provider's template names `${consumers:<provision>}`, and its machine's client
|
||||
// fills that file with one entry per consumer bound to it for that provision, each credential asked of the vault in
|
||||
// the client's own name; the file is the provider's alone (mode 0600, owned). ConsumersFile is that file as the
|
||||
// helper's Vault: it reads the file on each ask, so a consumer that binds or a credential the vault remakes reaches
|
||||
// the provider at its next pass.
|
||||
|
||||
// ConsumerCredential is one entry of the file: the consumer, the vault's name of its credential, and its value.
|
||||
type ConsumerCredential struct {
|
||||
// Consumer and Module are the consumer's machine and module, Local its local name for the provision, if any.
|
||||
Consumer string `json:"consumer"`
|
||||
Module string `json:"module"`
|
||||
Local string `json:"local,omitempty"`
|
||||
// Credential is the vault's name of the consumer's credential, PairName's.
|
||||
Credential string `json:"credential"`
|
||||
// Value is the credential itself.
|
||||
Value string `json:"value"`
|
||||
}
|
||||
|
||||
// ConsumersFile answers a credential from the files the provider machine's client fills: one per provision whose
|
||||
// template names `${consumers:<provision>}`, each a JSON list of ConsumerCredential.
|
||||
type ConsumersFile struct {
|
||||
Paths []string
|
||||
}
|
||||
|
||||
// Value is the credential named name, from the first file carrying it. A file not there yet is passed over; one not
|
||||
// in the client's form is an error, never read as no consumers.
|
||||
func (f ConsumersFile) Value(_ context.Context, name string) ([]byte, error) {
|
||||
var missing []string
|
||||
for _, at := range f.Paths {
|
||||
raw, err := os.ReadFile(at)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
missing = append(missing, at)
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the consumers' file %s: %w", at, err)
|
||||
}
|
||||
var entries []ConsumerCredential
|
||||
if err := json.Unmarshal(raw, &entries); err != nil {
|
||||
return nil, fmt.Errorf("the consumers' file %s is not the provisioner client's list of consumers", at)
|
||||
}
|
||||
for _, e := range entries {
|
||||
if e.Credential == name {
|
||||
if e.Value == "" {
|
||||
return nil, fmt.Errorf("the consumers' file %s carries %s with no value", at, name)
|
||||
}
|
||||
return []byte(e.Value), nil
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(missing) == len(f.Paths) {
|
||||
return nil, fmt.Errorf("%s: no consumers' file is filled yet (%v)", name, f.Paths)
|
||||
}
|
||||
return nil, fmt.Errorf("%s is in no consumers' file: the vault holds none for it yet, or its machine's client "+
|
||||
"has not filled it", name)
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The provider reads its consumers' credentials only from the file its machine's provisioner client fills (novox/hq
|
||||
// ADR 0373 clause 3, issue 679): one entry per consumer, naming the consumer and the vault's name for its credential,
|
||||
// and carrying the value. The helper is wired to it as its Vault and asks nothing of the vault itself.
|
||||
func TestTheConsumersFileAnswersEachConsumersCredential(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
at := filepath.Join(dir, "consumers.json")
|
||||
file := `[{"consumer":"ace","module":"letta","credential":"pair/novox/postgres-database/ace/letta","value":"s3cret"},
|
||||
{"consumer":"g14","module":"gitea","local":"cache","credential":"pair/novox/postgres-database/g14/gitea/cache","value":"other"}]`
|
||||
if err := os.WriteFile(at, []byte(file), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v := ConsumersFile{Paths: []string{filepath.Join(dir, "absent.json"), at}}
|
||||
got, err := v.Value(context.Background(), PairName("novox", "postgres-database", "ace", "letta", ""))
|
||||
if err != nil || string(got) != "s3cret" {
|
||||
t.Fatalf("ace's letta: %q, %v; want s3cret", got, err)
|
||||
}
|
||||
got, err = v.Value(context.Background(), PairName("novox", "postgres-database", "g14", "gitea", "cache"))
|
||||
if err != nil || string(got) != "other" {
|
||||
t.Fatalf("g14's gitea under cache: %q, %v; want other", got, err)
|
||||
}
|
||||
// A consumer the file does not carry is waited for, and never told of a login its provider has not made.
|
||||
_, err = v.Value(context.Background(), PairName("novox", "postgres-database", "shanks", "photos", ""))
|
||||
if err == nil || !strings.Contains(err.Error(), "pair/novox/postgres-database/shanks/photos") {
|
||||
t.Fatalf("a consumer the file does not carry: %v", err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "s3cret") || strings.Contains(err.Error(), "other") {
|
||||
t.Fatalf("an error names a credential: %v", err)
|
||||
}
|
||||
|
||||
// Wired as the helper's Vault, the file is what makes each consumer's login.
|
||||
bus := newStandIn()
|
||||
bus.Put(context.Background(), AssignmentsBucket, "ace.letta", []byte(`{"node":"ace","module":"letta",
|
||||
"provisioning":{"provisions":[{"provision":"postgres-database","provider":"novox","provider-module":"postgres"}]}}`))
|
||||
var made []string
|
||||
p := New(Options{Node: "novox", Module: "postgres", Bus: bus, Vault: v,
|
||||
Offers: map[string]Offer{"postgres-database": {Serves: map[string]any{"port": 5432}}},
|
||||
Make: func(_ context.Context, c Consumer) error {
|
||||
made = append(made, c.Entry.User+"="+string(c.Credential))
|
||||
return nil
|
||||
}, Logf: func(string, ...any) {}})
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(made) != 1 || made[0] != "mesh_ace_letta=s3cret" {
|
||||
t.Fatalf("made %v, want ace's letta under the file's credential", made)
|
||||
}
|
||||
}
|
||||
|
||||
// A file that is not the client's form is said, never read as no consumers.
|
||||
func TestAConsumersFileNotInItsFormIsRefused(t *testing.T) {
|
||||
at := filepath.Join(t.TempDir(), "consumers.json")
|
||||
if err := os.WriteFile(at, []byte(`{"not": "a list"}`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, err := ConsumersFile{Paths: []string{at}}.Value(context.Background(), "pair/novox/p/ace/m")
|
||||
if err == nil || !strings.Contains(err.Error(), at) {
|
||||
t.Fatalf("a file not in the client's form: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
package harness
|
||||
|
||||
import "context"
|
||||
|
||||
var ctx = context.Background()
|
||||
@@ -0,0 +1,62 @@
|
||||
package harness
|
||||
|
||||
// The moved path started before its runtime answers asks again until it does (issue 943).
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/provider"
|
||||
)
|
||||
|
||||
// earlyBus refuses its first watches as the runtime does before Serve runs, then answers as the bucket bus does.
|
||||
type earlyBus struct {
|
||||
*bucketBus
|
||||
refuse int
|
||||
}
|
||||
|
||||
func (b *earlyBus) Watch(ctx context.Context, bucket string, handle func(string, []byte)) error {
|
||||
if b.refuse > 0 {
|
||||
b.refuse--
|
||||
return errors.New("before Serve: a bundle reaches the bus through the runtime that launched it")
|
||||
}
|
||||
return b.bucketBus.Watch(ctx, bucket, handle)
|
||||
}
|
||||
|
||||
func TestAMovedPathStartedTooEarlyAsksAgainUntilTheRuntimeAnswers(t *testing.T) {
|
||||
w := newMovedWorld(t, map[[2]string]string{})
|
||||
w.run.FirstRetry = time.Millisecond
|
||||
w.run.loaded = false
|
||||
bus := &earlyBus{bucketBus: w.bus, refuse: 3}
|
||||
p := provider.New(provider.Options{Node: "novox", Module: "postgres", Bus: w.bus,
|
||||
Vault: provider.ConsumersFile{Paths: []string{w.file}}, Make: w.run.Make, Gone: w.run.Gone,
|
||||
Every: time.Hour, Logf: func(string, ...any) {}})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- w.run.Run(ctx, p, bus) }()
|
||||
deadline := time.After(time.Second)
|
||||
for {
|
||||
w.run.mu.Lock()
|
||||
loaded := w.run.loaded
|
||||
w.run.mu.Unlock()
|
||||
if loaded {
|
||||
break
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
cancel()
|
||||
t.Fatalf("Run stopped before the runtime answered: %v", err)
|
||||
case <-deadline:
|
||||
cancel()
|
||||
t.Fatal("the plans were never watched")
|
||||
case <-time.After(time.Millisecond):
|
||||
}
|
||||
}
|
||||
cancel()
|
||||
<-done
|
||||
if bus.refuse != 0 {
|
||||
t.Fatalf("%d refusals left", bus.refuse)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
package harness
|
||||
|
||||
// postgres handed to the provisioner still serves the consumers the controller's grant file gives (issue 937): a
|
||||
// consumer not moved has no plan naming postgres, only its grant, until R1 (issue 682).
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// granting gives the moved world a grant file, as the controller writes it for the consumers not moved.
|
||||
func (w *movedWorld) granting(given ...map[string]any) {
|
||||
if w.h.Receives == "" {
|
||||
w.h.Receives = filepath.Join(w.t.TempDir(), "mesh.json")
|
||||
}
|
||||
dir := filepath.Dir(w.h.Receives)
|
||||
for _, g := range given {
|
||||
secret := filepath.Join(dir, g["as"].(string)+".secret")
|
||||
if err := os.WriteFile(secret, []byte("pw-"+g["as"].(string)+"\n"), 0o600); err != nil {
|
||||
w.t.Fatal(err)
|
||||
}
|
||||
g["secret"] = secret
|
||||
}
|
||||
if given == nil {
|
||||
given = []map[string]any{}
|
||||
}
|
||||
raw, _ := json.Marshal(map[string]any{"requirement": ProvisionName, "given": given})
|
||||
if err := os.WriteFile(w.h.Receives, raw, 0o600); err != nil {
|
||||
w.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// settled is enough passes and time for a retirement to be stable.
|
||||
func (w *movedWorld) settled() {
|
||||
for i := 0; i < 8; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(15 * time.Minute)
|
||||
w.pass()
|
||||
w.pass()
|
||||
}
|
||||
|
||||
// The logins the grant file gives are asked for: made as the grant path makes them, never offered for retirement,
|
||||
// beside a moved consumer the helper makes from its plan.
|
||||
func TestAConsumerOnlyTheGrantFileGivesIsAskedForAndMade(t *testing.T) {
|
||||
bound, logins := bindings(1)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.a.inv.Active = []string{"mesh_ace_baserow", "mesh_ace_letta", logins[0]}
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.granting(
|
||||
map[string]any{"as": "mesh_ace_baserow", "node": "ace", "values": map[string]any{"name": "baserow"}},
|
||||
map[string]any{"as": "mesh_ace_letta", "node": "ace",
|
||||
"values": map[string]any{"name": "letta", "extensions": []any{"vector"}}},
|
||||
)
|
||||
w.settled()
|
||||
if len(w.a.removed) != 0 || w.h.r.waiting != nil {
|
||||
t.Fatalf("retired %v, waiting %+v: the grant file's logins were counted unasked", w.a.removed, w.h.r.waiting)
|
||||
}
|
||||
made := map[string]Provision{}
|
||||
for _, c := range w.a.created {
|
||||
made[c.As] = c
|
||||
}
|
||||
letta := made["mesh_ace_letta"]
|
||||
if letta.Password != "pw-mesh_ace_letta" || letta.Consumer != "ace" ||
|
||||
!reflect.DeepEqual(letta.Values, map[string]any{"name": "letta", "extensions": []any{"vector"}}) {
|
||||
t.Fatalf("made %+v for letta, as its grant gives it", letta)
|
||||
}
|
||||
if _, ok := made["mesh_ace_baserow"]; !ok {
|
||||
t.Fatalf("baserow not made: %+v", w.a.created)
|
||||
}
|
||||
if _, ok := made[logins[0]]; !ok {
|
||||
t.Fatalf("the moved consumer not made: %+v", w.a.created)
|
||||
}
|
||||
}
|
||||
|
||||
// A login both a plan and the grant file name is the plan's: made once, with the plan's credential.
|
||||
func TestALoginAPlanNamesIsNotMadeFromTheGrantFile(t *testing.T) {
|
||||
bound, logins := bindings(1)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.granting(map[string]any{"as": logins[0], "node": "ace", "values": map[string]any{"name": "app1"}})
|
||||
w.pass()
|
||||
w.pass()
|
||||
for _, c := range w.a.created {
|
||||
if c.As == logins[0] && c.Password != "pw-app1" {
|
||||
t.Fatalf("made %s with the grant's password %q, not the plan's", c.As, c.Password)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An unreadable grant file is not nobody asking (issue 241): nothing is retired while it cannot be read.
|
||||
func TestAnUnreadableGrantFileRetiresNothing(t *testing.T) {
|
||||
bound, logins := bindings(1)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.a.inv.Active = []string{"mesh_ace_baserow", logins[0]}
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.h.Receives = filepath.Join(t.TempDir(), "absent.json")
|
||||
w.settled()
|
||||
if len(w.a.removed) != 0 || w.h.r.waiting != nil {
|
||||
t.Fatalf("retired %v, waiting %+v with the grant file unreadable", w.a.removed, w.h.r.waiting)
|
||||
}
|
||||
}
|
||||
|
||||
// A login the grant file stops giving is retired by the usual rules: stable, then retired, never dropped.
|
||||
func TestALoginTheGrantFileStopsGivingIsRetired(t *testing.T) {
|
||||
bound, _ := bindings(1)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.granting(
|
||||
map[string]any{"as": "mesh_ace_baserow", "node": "ace", "values": map[string]any{"name": "baserow"}},
|
||||
map[string]any{"as": "mesh_ace_letta", "node": "ace", "values": map[string]any{"name": "letta"}},
|
||||
)
|
||||
w.pass()
|
||||
w.granting(map[string]any{"as": "mesh_ace_baserow", "node": "ace", "values": map[string]any{"name": "baserow"}})
|
||||
w.settled()
|
||||
if !reflect.DeepEqual(w.a.removed, []string{"mesh_ace_letta"}) || len(w.a.deleted) != 0 {
|
||||
t.Fatalf("retired %v, deleted %v", w.a.removed, w.a.deleted)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,572 @@
|
||||
// Package harness is the reconcile loop and the retirement of a provider that keeps a resource per consumer; the
|
||||
// module supplies an Adapter.
|
||||
package harness
|
||||
|
||||
// The reconcile loop every provider shares, as the TypeScript SDK's runProvisioner runs it
|
||||
// (@novox/mesh-sdk/provisioner, 0.1.12), line for line in behaviour. It moved here from the postgres and keycloak
|
||||
// providers, which each carried an identical copy (novox/hq ADR 0039: the loop is the SDK's, the adapter is the
|
||||
// module's; issue 849).
|
||||
//
|
||||
// Read the contributions the mesh delivered; bring each consumer's resource into being through the
|
||||
// adapter, under the login and password the mesh minted; retire what the mesh no longer asks for.
|
||||
// **A provider creates the credential the mesh minted, and seals nothing (novox/hq ADR 0048).**
|
||||
//
|
||||
// **A provider that keeps failing a consumer says so on the bus (novox/hq ADR 0224).** A consumer
|
||||
// whose create, check or secret has failed without one success in between for FailingAfter is
|
||||
// announced as `provisioner.failing` — naming the consumer, its machine and the class of error — and
|
||||
// again every SayAgainEvery while it lasts; the first success after that is `provisioner.recovered`.
|
||||
// The controller keeps the newest per provider and consumer and `status` names it. On 2026-10-05 the
|
||||
// identity provider failed every consumer 31,000 times in a day and said so only in its journal
|
||||
// (novox/hq issue 179).
|
||||
//
|
||||
// **A consumer the mesh stops asking for is retired, not withdrawn, and deleted only by a person
|
||||
// (novox/hq ADR 0230, the operator's model of 2026-10-06).** Retiring disables its access — reversibly —
|
||||
// and marks its login and data "to delete" with when and why; nothing is deleted. Asked for again, the
|
||||
// ordinary create re-enables it as it was. It is retired only once the same set has gone unasked in
|
||||
// StablePasses consecutive passes and for StableFor, and a set larger than the bound waits for a
|
||||
// person. retirement.go.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Provision is one consumer's resource to bring into being — everything the mesh derived and delivered.
|
||||
type Provision struct {
|
||||
// As is the login the mesh derived and gave the consumer to present.
|
||||
As string
|
||||
// Password is the one the mesh minted, read from the file the host unsealed.
|
||||
Password string
|
||||
// Values are what the consumer contributed (e.g. {"name": "letta", "extensions": ["vector"]}).
|
||||
Values map[string]any
|
||||
// Derived is what this provider's own definition derives for the consumer (novox/hq ADR 0201).
|
||||
Derived map[string]any
|
||||
// At is where the consumer is; Consumer is its node.
|
||||
At string
|
||||
Consumer string
|
||||
}
|
||||
|
||||
// Adapter is the per-service half.
|
||||
type Adapter interface {
|
||||
// Create brings the consumer into being under the mesh's login and password — and, for one that
|
||||
// was retired, re-enables it as it was and clears its mark to delete.
|
||||
Create(ctx context.Context, p Provision) error
|
||||
// Retire disables the consumer's access, reversibly, and marks its login and data to delete with
|
||||
// when and why. It deletes nothing (novox/hq ADR 0230). derived is what the mesh last derived,
|
||||
// remembered here; nil for a consumer this process did not make.
|
||||
Retire(ctx context.Context, as string, derived map[string]any, why string, at time.Time) error
|
||||
// Holds says whether the backend still holds the consumer exactly as p says. Read-only.
|
||||
Holds(ctx context.Context, p Provision) (bool, error)
|
||||
// Inventory is what the backend holds that the mesh made: consumers active and retired, read from
|
||||
// the backend itself so a restart forgets nothing. Read-only.
|
||||
Inventory(ctx context.Context) (Inventory, error)
|
||||
// Delete removes one retired consumer — its login and its data — for good. Only ever asked by a
|
||||
// person, through `cleanup delete`; the harness has checked it is retired and not asked for.
|
||||
Delete(ctx context.Context, r Retired) (freedBytes int64, err error)
|
||||
}
|
||||
|
||||
// Harness is the loop's settings and memory.
|
||||
type Harness struct {
|
||||
Resource string
|
||||
Receives string
|
||||
Adapter Adapter
|
||||
Every time.Duration // 5s
|
||||
VerifyEvery time.Duration // 60s
|
||||
HoldsTimeout time.Duration // 30s
|
||||
Log func(format string, args ...any)
|
||||
Now func() time.Time
|
||||
// Announce publishes one of the provider's standing events; nil announces nothing. Node is the
|
||||
// machine this provider runs on, said in each.
|
||||
Announce func(event string, body map[string]any)
|
||||
Node string
|
||||
// FailingAfter is how long a consumer fails without a success before it is announced (5m);
|
||||
// SayAgainEvery is how often it is announced again while it lasts (15m), so a controller that
|
||||
// missed the first hears the next, and a standing nobody repeats can be told from one that holds.
|
||||
FailingAfter time.Duration
|
||||
SayAgainEvery time.Duration
|
||||
// StablePasses (5) is how many consecutive passes must see the same set no longer asked for, and
|
||||
// StableFor (10m) how long it must have held since the first of them, before it is retired. RetireAtOnce (3) and RetireFraction (0.5) bound what is retired without a person:
|
||||
// more consumers than RetireAtOnce, or — where more than one is held — a larger share of those held
|
||||
// than RetireFraction, waits for `retire approve` (novox/hq ADR 0230).
|
||||
StablePasses int
|
||||
StableFor time.Duration
|
||||
RetireAtOnce int
|
||||
RetireFraction float64
|
||||
|
||||
// mu is held by a pass and by every tool a person asks, so the two never interleave.
|
||||
mu sync.Mutex
|
||||
verifiedAt time.Time
|
||||
r retirement
|
||||
applied map[string]appliedEntry
|
||||
lost map[string]brake
|
||||
waiting map[string]int
|
||||
failing map[string]failure
|
||||
trouble map[string]*standing
|
||||
cleared map[string]bool
|
||||
lastWarning string
|
||||
}
|
||||
|
||||
// standing is one consumer's unbroken run of failures: since when, how often, and the last error.
|
||||
type standing struct {
|
||||
node string
|
||||
since time.Time
|
||||
attempts int
|
||||
class string
|
||||
text string
|
||||
saidAt time.Time
|
||||
}
|
||||
|
||||
// The events a provider's standing is announced as (novox/hq ADR 0224). The controller derives the
|
||||
// permission to emit them for every module that receives contributions; no manifest lists them.
|
||||
const (
|
||||
EventFailing = "provisioner.failing"
|
||||
EventRecovered = "provisioner.recovered"
|
||||
)
|
||||
|
||||
// The classes of error a standing is announced with: what a person reading `status` needs to know
|
||||
// before reading the journal. An adapter may say better (Classifier).
|
||||
const (
|
||||
ClassCredentials = "credentials-rejected"
|
||||
ClassUnreachable = "unreachable"
|
||||
ClassSecret = "secret-unreadable"
|
||||
ClassRefused = "refused"
|
||||
)
|
||||
|
||||
// Classifier is an adapter that can say what class an error of its own is.
|
||||
type Classifier interface {
|
||||
Class(err error) string
|
||||
}
|
||||
|
||||
type appliedEntry struct {
|
||||
hash string
|
||||
derived map[string]any
|
||||
node string
|
||||
}
|
||||
|
||||
type brake struct {
|
||||
times int
|
||||
nextAt time.Time
|
||||
}
|
||||
|
||||
type failure struct {
|
||||
text string
|
||||
times int
|
||||
}
|
||||
|
||||
// The longest a consumer whose create keeps failing to satisfy holds waits between checks.
|
||||
const maxBackoff = time.Hour
|
||||
|
||||
// How many passes a secret may be unreadable before it stops being called a race (issue 225), and
|
||||
// once said loudly, how often it is repeated. The same cadence quiets a create that keeps failing
|
||||
// the same way.
|
||||
const (
|
||||
patiently = 12
|
||||
loudlyEvery = 240
|
||||
)
|
||||
|
||||
type contribution struct {
|
||||
As string `json:"as"`
|
||||
Secret string `json:"secret"`
|
||||
Node string `json:"node"`
|
||||
At string `json:"at"`
|
||||
Values map[string]any `json:"values"`
|
||||
Derived map[string]any `json:"derived"`
|
||||
}
|
||||
|
||||
func (h *Harness) init() {
|
||||
if h.Every == 0 {
|
||||
h.Every = 5 * time.Second
|
||||
}
|
||||
if h.VerifyEvery == 0 {
|
||||
h.VerifyEvery = time.Minute
|
||||
}
|
||||
if h.HoldsTimeout == 0 {
|
||||
h.HoldsTimeout = 30 * time.Second
|
||||
}
|
||||
if h.Now == nil {
|
||||
h.Now = time.Now
|
||||
}
|
||||
if h.FailingAfter == 0 {
|
||||
h.FailingAfter = 5 * time.Minute
|
||||
}
|
||||
if h.SayAgainEvery == 0 {
|
||||
h.SayAgainEvery = 15 * time.Minute
|
||||
}
|
||||
if h.StablePasses == 0 {
|
||||
h.StablePasses = 5
|
||||
}
|
||||
if h.StableFor == 0 {
|
||||
h.StableFor = StableFor
|
||||
}
|
||||
if h.RetireAtOnce == 0 {
|
||||
h.RetireAtOnce = 3
|
||||
}
|
||||
if h.RetireFraction == 0 {
|
||||
h.RetireFraction = 0.5
|
||||
}
|
||||
if h.Log == nil {
|
||||
h.Log = func(format string, args ...any) { fmt.Fprintf(os.Stderr, format+"\n", args...) }
|
||||
}
|
||||
if h.applied == nil {
|
||||
h.applied = map[string]appliedEntry{}
|
||||
h.lost = map[string]brake{}
|
||||
h.waiting = map[string]int{}
|
||||
h.failing = map[string]failure{}
|
||||
h.trouble = map[string]*standing{}
|
||||
h.cleared = map[string]bool{}
|
||||
h.r.retired = map[string]retiredEntry{}
|
||||
}
|
||||
}
|
||||
|
||||
// Run reconciles until ctx ends. One consumer's failure never stops the others'.
|
||||
func (h *Harness) Run(ctx context.Context) {
|
||||
h.init()
|
||||
for {
|
||||
h.Reconcile(ctx)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(h.Every):
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Harness) say(format string, args ...any) {
|
||||
h.Log("[provisioner:"+h.Resource+"] "+format, args...)
|
||||
}
|
||||
|
||||
func (h *Harness) warn(why string) {
|
||||
if why == h.lastWarning {
|
||||
return
|
||||
}
|
||||
if why != "" {
|
||||
h.say("%s; nothing applied or removed until it can be read", why)
|
||||
} else {
|
||||
h.say("contributions file readable again")
|
||||
}
|
||||
h.lastWarning = why
|
||||
}
|
||||
|
||||
// readContributions answers the consumers asked for, or nil when the file says nothing usable.
|
||||
// **Nothing read is not nobody asking** (novox/hq issue 241): only a file that was read can retire anything.
|
||||
func (h *Harness) readContributions() []contribution {
|
||||
raw, err := os.ReadFile(h.Receives)
|
||||
if err != nil {
|
||||
h.warn(fmt.Sprintf("contributions file unreadable (%s): %v", h.Receives, err))
|
||||
return nil
|
||||
}
|
||||
var doc struct {
|
||||
Requirement string `json:"requirement"`
|
||||
Given json.RawMessage `json:"given"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||
h.warn(fmt.Sprintf("contributions file is not JSON (%s): %v", h.Receives, err))
|
||||
return nil
|
||||
}
|
||||
if doc.Requirement != "" && doc.Requirement != h.Resource {
|
||||
h.warn(fmt.Sprintf("%s is for %s, not %s", h.Receives, doc.Requirement, h.Resource))
|
||||
return nil
|
||||
}
|
||||
var given []contribution
|
||||
if len(doc.Given) == 0 || string(doc.Given) == "null" || json.Unmarshal(doc.Given, &given) != nil {
|
||||
h.warn(fmt.Sprintf("%s has no given list", h.Receives))
|
||||
return nil
|
||||
}
|
||||
h.warn("")
|
||||
out := []contribution{}
|
||||
for _, g := range given {
|
||||
// No `as` is not a credential grant: nothing to create for it.
|
||||
if g.As != "" && g.Secret != "" {
|
||||
out = append(out, g)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func hashOf(as, password string, values, derived map[string]any) string {
|
||||
// Derived is in the hash: a provider that renames what it derives gave a different resource.
|
||||
b, _ := json.Marshal([]any{as, password, orEmpty(values), orEmpty(derived)})
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func orEmpty(m map[string]any) map[string]any {
|
||||
if m == nil {
|
||||
return map[string]any{}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Reconcile is one pass.
|
||||
func (h *Harness) Reconcile(ctx context.Context) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
given := h.readContributions()
|
||||
if given == nil {
|
||||
// Not a result: the passes that must agree before anything is retired start again.
|
||||
h.r.key, h.r.count = "", 0
|
||||
return
|
||||
}
|
||||
want := map[string]bool{}
|
||||
for _, g := range given {
|
||||
want[g.As] = true
|
||||
}
|
||||
h.seed(ctx, want)
|
||||
h.apply(ctx, given)
|
||||
|
||||
// Retire what the mesh has stably stopped asking for — within the bound, or with a person.
|
||||
h.retireUnasked(ctx, want)
|
||||
h.r.lastWant = want
|
||||
for as := range h.failing {
|
||||
if !want[as] {
|
||||
delete(h.failing, as)
|
||||
}
|
||||
}
|
||||
// A consumer the mesh stopped asking for that this provider holds nothing for is no longer failed by
|
||||
// anyone: said, so a standing the controller keeps for it is cleared rather than left naming a
|
||||
// consumer that is gone. One still held is said recovered when it is retired.
|
||||
for as := range h.trouble {
|
||||
if _, held := h.applied[as]; !want[as] && !held {
|
||||
h.recovered(as, "no longer asked for")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// apply makes each consumer the grant file gives, and checks every VerifyEvery that the backend still holds it. The
|
||||
// caller holds h.mu. One consumer's failure never stops the others'.
|
||||
func (h *Harness) apply(ctx context.Context, given []contribution) {
|
||||
verifying := h.Now().Sub(h.verifiedAt) >= h.VerifyEvery
|
||||
if verifying {
|
||||
h.verifiedAt = h.Now()
|
||||
}
|
||||
|
||||
for _, g := range given {
|
||||
raw, err := os.ReadFile(g.Secret)
|
||||
if err != nil {
|
||||
// A secret the host has not written yet is a race on the first pass; past a minute it is
|
||||
// a person's to look at, and said so (novox/hq issue 225).
|
||||
n := h.waiting[g.As] + 1
|
||||
h.waiting[g.As] = n
|
||||
if n <= patiently {
|
||||
h.say("%s: secret not readable yet (%s): %v", g.As, g.Secret, err)
|
||||
} else if n == patiently+1 || n%loudlyEvery == 0 {
|
||||
h.say("%s: CANNOT READ the secret after %d attempts (%s): %v. This is not a race any more — "+
|
||||
"nothing has been provisioned for this consumer and nothing will be until somebody looks. "+
|
||||
"Check who owns the file and who this process runs as (novox/hq issue 225)", g.As, n, g.Secret, err)
|
||||
}
|
||||
h.failed(g.As, g.Node, ClassSecret, fmt.Sprintf("secret not readable (%s): %v", g.Secret, err))
|
||||
continue
|
||||
}
|
||||
delete(h.waiting, g.As)
|
||||
password := strings.TrimSuffix(string(raw), "\n")
|
||||
p := Provision{As: g.As, Password: password, Values: orEmpty(g.Values), Derived: orEmpty(g.Derived), At: g.At, Consumer: g.Node}
|
||||
hash := hashOf(g.As, password, g.Values, g.Derived)
|
||||
|
||||
reapplying := 0
|
||||
if was, ok := h.applied[g.As]; ok && was.hash == hash {
|
||||
if !verifying {
|
||||
continue
|
||||
}
|
||||
b, braked := h.lost[g.As]
|
||||
if braked && h.Now().Before(b.nextAt) {
|
||||
continue
|
||||
}
|
||||
hctx, cancel := context.WithTimeout(ctx, h.HoldsTimeout)
|
||||
held, err := h.Adapter.Holds(hctx, p)
|
||||
timedOut := errors.Is(hctx.Err(), context.DeadlineExceeded)
|
||||
cancel()
|
||||
if err != nil {
|
||||
// Unable to ask is not evidence of loss. A backend that timed out will time out for
|
||||
// the next consumer too, so the rest of this pass is not asked.
|
||||
text := scrub(err, password)
|
||||
h.say("%s: could not check the backend, will ask again: %s", g.As, text)
|
||||
h.failed(g.As, g.Node, h.classOf(err, text), text)
|
||||
if timedOut {
|
||||
verifying = false
|
||||
}
|
||||
continue
|
||||
}
|
||||
if held {
|
||||
delete(h.lost, g.As)
|
||||
h.succeeded(g.As)
|
||||
continue
|
||||
}
|
||||
reapplying = b.times + 1
|
||||
if reapplying == 1 {
|
||||
h.say("%s: the backend no longer holds it; applying again", g.As)
|
||||
} else {
|
||||
h.say("%s: still not held after being applied again (%d times in a row) — create does not "+
|
||||
"produce what holds checks; applying again", g.As, reapplying)
|
||||
}
|
||||
}
|
||||
if err := h.Adapter.Create(ctx, p); err != nil {
|
||||
text := scrub(err, password)
|
||||
f := h.failing[g.As]
|
||||
if f.text != text {
|
||||
f = failure{text: text}
|
||||
}
|
||||
f.times++
|
||||
h.failing[g.As] = f
|
||||
// Said each time it changes, and while it stays the same, as rarely as a lost secret.
|
||||
if f.times == 1 || f.times%loudlyEvery == 0 {
|
||||
h.say("%s: create failed, will retry: %s", g.As, text)
|
||||
}
|
||||
h.failed(g.As, g.Node, h.classOf(err, text), text)
|
||||
if reapplying > 0 {
|
||||
h.lost[g.As] = brake{times: reapplying - 1}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if f, was := h.failing[g.As]; was {
|
||||
h.say("%s: created, after %d failed attempt(s)", g.As, f.times)
|
||||
delete(h.failing, g.As)
|
||||
}
|
||||
h.succeeded(g.As)
|
||||
h.applied[g.As] = appliedEntry{hash: hash, derived: p.Derived, node: g.Node}
|
||||
h.reenabled(g.As, g.Node)
|
||||
if reapplying == 0 {
|
||||
delete(h.lost, g.As)
|
||||
} else {
|
||||
wait := h.VerifyEvery << (reapplying - 1)
|
||||
if wait > maxBackoff || wait <= 0 {
|
||||
wait = maxBackoff
|
||||
}
|
||||
h.lost[g.As] = brake{times: reapplying, nextAt: h.Now().Add(wait)}
|
||||
if reapplying > 1 {
|
||||
h.say("%s: next check in %s", g.As, wait.Round(time.Second))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// failed counts one more failure in a consumer's unbroken run, and announces the run once it has
|
||||
// lasted FailingAfter — then again every SayAgainEvery while it lasts.
|
||||
func (h *Harness) failed(as, node, class, text string) {
|
||||
now := h.Now()
|
||||
s := h.trouble[as]
|
||||
if s == nil {
|
||||
s = &standing{since: now}
|
||||
h.trouble[as] = s
|
||||
}
|
||||
s.node, s.class, s.text = node, class, text
|
||||
s.attempts++
|
||||
if now.Sub(s.since) < h.FailingAfter {
|
||||
return
|
||||
}
|
||||
if !s.saidAt.IsZero() && now.Sub(s.saidAt) < h.SayAgainEvery {
|
||||
return
|
||||
}
|
||||
first := s.saidAt.IsZero()
|
||||
s.saidAt = now
|
||||
if first {
|
||||
h.say("%s: FAILING for %s (%d attempts, %s): %s. Announced as %s; `status` names it until it "+
|
||||
"succeeds (novox/hq ADR 0224)", as, now.Sub(s.since).Round(time.Second), s.attempts, class, text, EventFailing)
|
||||
}
|
||||
h.announce(EventFailing, map[string]any{
|
||||
"provider": h.Resource, "provider-node": h.Node,
|
||||
"consumer": as, "node": node,
|
||||
"class": class, "error": clip(text),
|
||||
"since": s.since.UTC().Format(time.RFC3339), "attempts": s.attempts,
|
||||
})
|
||||
}
|
||||
|
||||
// succeeded ends a consumer's run of failures; one that was announced is announced recovered.
|
||||
//
|
||||
// **And the first success for a consumer since this process started is announced too**, failing or
|
||||
// not: a provider that announced a failure and was restarted has forgotten it, and without this the
|
||||
// controller would name the consumer failing for ever after it recovered unheard.
|
||||
func (h *Harness) succeeded(as string) {
|
||||
if h.trouble[as] == nil && !h.cleared[as] {
|
||||
h.cleared[as] = true
|
||||
h.announce(EventRecovered, map[string]any{
|
||||
"provider": h.Resource, "provider-node": h.Node, "consumer": as, "why": "first-success",
|
||||
})
|
||||
return
|
||||
}
|
||||
h.cleared[as] = true
|
||||
h.recovered(as, "")
|
||||
}
|
||||
|
||||
func (h *Harness) recovered(as, why string) {
|
||||
s := h.trouble[as]
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
delete(h.trouble, as)
|
||||
if s.saidAt.IsZero() {
|
||||
return // never announced, so there is nothing to take back
|
||||
}
|
||||
if why == "" {
|
||||
h.say("%s: recovered after %s and %d failed attempt(s)", as, h.Now().Sub(s.since).Round(time.Second), s.attempts)
|
||||
}
|
||||
body := map[string]any{
|
||||
"provider": h.Resource, "provider-node": h.Node, "consumer": as, "node": s.node,
|
||||
"since": s.since.UTC().Format(time.RFC3339), "attempts": s.attempts,
|
||||
}
|
||||
if why != "" {
|
||||
body["why"] = why
|
||||
}
|
||||
h.announce(EventRecovered, body)
|
||||
}
|
||||
|
||||
func (h *Harness) announce(event string, body map[string]any) {
|
||||
if h.Announce != nil {
|
||||
h.Announce(event, body)
|
||||
}
|
||||
}
|
||||
|
||||
// classOf is an error's class: the adapter's word when it has one, else read from the text.
|
||||
func (h *Harness) classOf(err error, text string) string {
|
||||
if c, ok := h.Adapter.(Classifier); ok {
|
||||
if class := c.Class(err); class != "" {
|
||||
return class
|
||||
}
|
||||
}
|
||||
return ClassOf(text)
|
||||
}
|
||||
|
||||
// ClassOf reads an error's class from its text — the words the backends the mesh runs use.
|
||||
func ClassOf(text string) string {
|
||||
t := strings.ToLower(text)
|
||||
for _, w := range []string{"invalid_grant", "invalid user credentials", "password authentication failed",
|
||||
"authentication failed", "unauthorized", " 401"} {
|
||||
if strings.Contains(t, w) {
|
||||
return ClassCredentials
|
||||
}
|
||||
}
|
||||
for _, w := range []string{"connection refused", "no such host", "i/o timeout", "deadline exceeded",
|
||||
"connection reset", "network is unreachable", "no route to host", "eof"} {
|
||||
if strings.Contains(t, w) {
|
||||
return ClassUnreachable
|
||||
}
|
||||
}
|
||||
return ClassRefused
|
||||
}
|
||||
|
||||
// clip keeps an announced error to what belongs in a status line.
|
||||
func clip(text string) string {
|
||||
const most = 300
|
||||
if len(text) <= most {
|
||||
return text
|
||||
}
|
||||
return text[:most] + "…"
|
||||
}
|
||||
|
||||
// scrub is an error's text with the consumer's password removed, raw and URL-encoded.
|
||||
func scrub(err error, password string) string {
|
||||
text := err.Error()
|
||||
if password == "" {
|
||||
return text
|
||||
}
|
||||
for _, form := range []string{password, url.QueryEscape(password), url.PathEscape(password)} {
|
||||
text = strings.ReplaceAll(text, form, "***")
|
||||
}
|
||||
return text
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
package harness
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type world struct {
|
||||
t *testing.T
|
||||
dir string
|
||||
receives string
|
||||
now time.Time
|
||||
h *Harness
|
||||
a *recorder
|
||||
said []string
|
||||
}
|
||||
|
||||
func newWorld(t *testing.T) *world {
|
||||
w := &world{t: t, dir: t.TempDir(), now: time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC), a: &recorder{held: true}}
|
||||
w.receives = filepath.Join(w.dir, "mesh.json")
|
||||
w.h = &Harness{Resource: "postgres-database", Receives: w.receives, Adapter: w.a,
|
||||
Now: func() time.Time { return w.now },
|
||||
Log: func(f string, a ...any) { w.said = append(w.said, f) }}
|
||||
return w
|
||||
}
|
||||
|
||||
func (w *world) give(given ...map[string]any) {
|
||||
for _, g := range given {
|
||||
secret := filepath.Join(w.dir, g["as"].(string)+".secret")
|
||||
if err := os.WriteFile(secret, []byte("pw-"+g["as"].(string)+"\n"), 0o600); err != nil {
|
||||
w.t.Fatal(err)
|
||||
}
|
||||
g["secret"] = secret
|
||||
}
|
||||
if given == nil {
|
||||
given = []map[string]any{}
|
||||
}
|
||||
raw, _ := json.Marshal(map[string]any{"requirement": "postgres-database", "given": given})
|
||||
if err := os.WriteFile(w.receives, raw, 0o600); err != nil {
|
||||
w.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAConsumerIsCreatedOnceUnderTheMeshsLoginAndPassword(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.give(map[string]any{"as": "mesh_ace_letta", "node": "ace", "values": map[string]any{"name": "letta"}})
|
||||
w.h.Reconcile(ctx)
|
||||
w.h.Reconcile(ctx)
|
||||
if len(w.a.created) != 1 {
|
||||
t.Fatalf("created %d times", len(w.a.created))
|
||||
}
|
||||
p := w.a.created[0]
|
||||
if p.As != "mesh_ace_letta" || p.Password != "pw-mesh_ace_letta" || p.Consumer != "ace" {
|
||||
t.Fatalf("%+v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAConsumerNoLongerAskedForIsRetiredOnceStable(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.give(map[string]any{"as": "a"}, map[string]any{"as": "b"})
|
||||
w.h.Reconcile(ctx)
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.settle() // five passes and ten minutes
|
||||
if strings.Join(w.a.removed, ",") != "b" {
|
||||
t.Fatal(w.a.removed)
|
||||
}
|
||||
// Only a file that says nobody asks retires the last one.
|
||||
w.give()
|
||||
w.settle()
|
||||
if strings.Join(w.a.removed, ",") != "b,a" {
|
||||
t.Fatal(w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNothingReadIsNotNobodyAsking(t *testing.T) {
|
||||
for name, content := range map[string]string{
|
||||
"unreadable": "",
|
||||
"not JSON": "{",
|
||||
"no given": `{"requirement": "postgres-database"}`,
|
||||
"another": `{"requirement": "mssql-database", "given": []}`,
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.h.Reconcile(ctx)
|
||||
if content == "" {
|
||||
os.Remove(w.receives)
|
||||
} else {
|
||||
os.WriteFile(w.receives, []byte(content), 0o600)
|
||||
}
|
||||
w.h.Reconcile(ctx)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v on a file it could not use", w.a.removed)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestALostConsumerIsAppliedAgainAndBraked(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.h.Reconcile(ctx)
|
||||
w.a.held = false
|
||||
w.now = w.now.Add(2 * time.Minute)
|
||||
w.h.Reconcile(ctx)
|
||||
if len(w.a.created) != 2 {
|
||||
t.Fatalf("created %d times", len(w.a.created))
|
||||
}
|
||||
// Still not held a minute later: applied again, then braked for two minutes.
|
||||
w.now = w.now.Add(61 * time.Second)
|
||||
w.h.Reconcile(ctx)
|
||||
w.now = w.now.Add(61 * time.Second)
|
||||
w.h.Reconcile(ctx)
|
||||
if len(w.a.created) != 3 {
|
||||
t.Fatalf("not braked: created %d times", len(w.a.created))
|
||||
}
|
||||
}
|
||||
|
||||
func TestAFailingCreateIsRetriedAndSaidOnce(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.a.failing = &pgErr{"extension \"nope\" refused, password pw-a"}
|
||||
w.give(map[string]any{"as": "a"})
|
||||
for i := 0; i < 5; i++ {
|
||||
w.h.Reconcile(ctx)
|
||||
}
|
||||
n := 0
|
||||
for _, s := range w.said {
|
||||
if strings.Contains(s, "create failed") {
|
||||
n++
|
||||
}
|
||||
}
|
||||
if n != 1 {
|
||||
t.Fatalf("said %d times", n)
|
||||
}
|
||||
w.a.failing = nil
|
||||
w.h.Reconcile(ctx)
|
||||
if len(w.a.created) != 1 {
|
||||
t.Fatal("not retried")
|
||||
}
|
||||
}
|
||||
|
||||
type pgErr struct{ s string }
|
||||
|
||||
func (e *pgErr) Error() string { return e.s }
|
||||
|
||||
func TestScrubRemovesThePassword(t *testing.T) {
|
||||
if got := scrub(&pgErr{"bad pw a/b c and a%2Fb+c"}, "a/b c"); strings.Contains(got, "a/b c") || strings.Contains(got, "a%2Fb+c") {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,339 @@
|
||||
package harness
|
||||
|
||||
// A provider handed to the provisioner (novox/hq ADR 0373, ADR 0349 clause 12, issues 780 and 805): where its
|
||||
// consumers file is set, it runs on the provider helper (package provider) instead of the harness reading the
|
||||
// controller's grant files. Moved is that path, shared by every provider that keeps a resource per consumer; a module
|
||||
// says only what differs, in a MovedSpec.
|
||||
//
|
||||
// The helper learns who binds the provider from the plans, composes each consumer's entry and reads its credential
|
||||
// through ConsumersFile, the file this machine's provisioner client fills. It leaves the rest to the two hooks
|
||||
// here: Make brings the consumer's resource into being exactly as the harness's adapter does, and Gone leaves
|
||||
// retirement to the harness's guards (retirement.go): the resource is disabled and marked to delete, never removed.
|
||||
//
|
||||
// **A consumer not moved is still the grant file's** (issue 937, ADR 0349 clause 17), for a provider whose spec says
|
||||
// WithGrants: only a moved consumer's plan names its provider, so the controller still gives every other consumer
|
||||
// through the grant file (MESH_RECEIVES) until R1 (issue 682). Each login it gives is asked for, and one no plan
|
||||
// names is made and checked by the harness's grant pass exactly as on the grant path; a login a plan names is the
|
||||
// helper's alone. A grant file that cannot be read is not nobody asking (issue 241): nothing is retired until it is
|
||||
// read again.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/provider"
|
||||
)
|
||||
|
||||
// MovedSpec is what differs per provider on the moved path.
|
||||
type MovedSpec struct {
|
||||
// Module and Provision are the provider module's name and what it provides, as its manifest says.
|
||||
Module, Provision string
|
||||
// Offer is what it serves: its credential form and served values.
|
||||
Offer provider.Offer
|
||||
// Login is the identity a consumer is made under at the provider's end (postgres: the entry's User; keycloak: its
|
||||
// As); an entry with none is not made.
|
||||
Login func(e provider.Entry) string
|
||||
// Values are the values the adapter is handed for a consumer; the entry's contribution when not set.
|
||||
Values func(e provider.Entry) map[string]any
|
||||
// Noun names what Login is in words ("login", "client id"), Made what is made ("login", "client"), and Backend
|
||||
// the service in "no longer holds it" ("the backend", "Keycloak").
|
||||
Noun, Made, Backend string
|
||||
// WithGrants makes the logins the grant file gives, that no plan names, asked for and made by the harness's grant
|
||||
// pass (issue 937).
|
||||
WithGrants bool
|
||||
}
|
||||
|
||||
// Moved is a provider on the provider helper, with the harness's own memory and retirement behind it: what is held,
|
||||
// what is counted unasked, what waits for a person (retirement.go). The harness is never Run here; the helper's Make
|
||||
// and Gone and the loop below drive the same state, so the guards are the harness's, not a copy.
|
||||
type Moved struct {
|
||||
h *Harness
|
||||
spec MovedSpec
|
||||
|
||||
// FirstRetry is the first wait before a watch asked too early is asked again (issue 943); a second when zero.
|
||||
FirstRetry time.Duration
|
||||
|
||||
mu sync.Mutex // guards assignments and loaded
|
||||
assignments map[string][]byte
|
||||
loaded bool
|
||||
verified map[string]time.Time
|
||||
}
|
||||
|
||||
func newMoved(h *Harness, spec MovedSpec) *Moved {
|
||||
h.mu.Lock()
|
||||
h.init()
|
||||
h.mu.Unlock()
|
||||
return &Moved{h: h, spec: spec, assignments: map[string][]byte{}, verified: map[string]time.Time{}}
|
||||
}
|
||||
|
||||
// Make brings one consumer's resource into being under its login and the vault's credential, as the harness does:
|
||||
// at once when it is new or changed, and otherwise only when the backend no longer holds it, which is asked every
|
||||
// VerifyEvery.
|
||||
func (m *Moved) Make(ctx context.Context, c provider.Consumer) error {
|
||||
h, e := m.h, c.Entry
|
||||
as := m.spec.Login(e)
|
||||
if as == "" {
|
||||
return fmt.Errorf("%s has no %s to make", e.Key(), m.spec.Noun)
|
||||
}
|
||||
if len(c.Credential) == 0 {
|
||||
return fmt.Errorf("%s has no credential to make its %s with", e.Key(), m.spec.Made)
|
||||
}
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
password := string(c.Credential)
|
||||
values := orEmpty(e.Contributes)
|
||||
if m.spec.Values != nil {
|
||||
values = m.spec.Values(e)
|
||||
}
|
||||
p := Provision{As: as, Password: password, Values: values, Consumer: e.Consumer}
|
||||
hash := hashOf(as, password, p.Values, nil)
|
||||
if was, ok := h.applied[as]; ok && was.hash == hash {
|
||||
if h.Now().Sub(m.verified[as]) < h.VerifyEvery {
|
||||
return nil
|
||||
}
|
||||
m.verified[as] = h.Now()
|
||||
hctx, cancel := context.WithTimeout(ctx, h.HoldsTimeout)
|
||||
held, err := h.Adapter.Holds(hctx, p)
|
||||
cancel()
|
||||
if err != nil {
|
||||
text := scrub(err, password)
|
||||
h.failed(as, e.Consumer, h.classOf(err, text), text)
|
||||
return errors.New(text)
|
||||
}
|
||||
if held {
|
||||
h.succeeded(as)
|
||||
return nil
|
||||
}
|
||||
h.say("%s: %s no longer holds it; applying again", as, m.spec.Backend)
|
||||
}
|
||||
if err := h.Adapter.Create(ctx, p); err != nil {
|
||||
text := scrub(err, password)
|
||||
h.failed(as, e.Consumer, h.classOf(err, text), text)
|
||||
return errors.New(text)
|
||||
}
|
||||
h.succeeded(as)
|
||||
h.applied[as] = appliedEntry{hash: hash, node: e.Consumer}
|
||||
m.verified[as] = h.Now()
|
||||
h.reenabled(as, e.Consumer)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Gone is the helper asking whether it may forget a consumer that no longer binds. **It never retires**: retiring is
|
||||
// the harness's rule, the same unasked set over StablePasses passes and StableFor, at most RetireAtOnce at a time
|
||||
// (retirement.go, ADR 0230), run by retire below. So a consumer missing from one read of the plans loses nothing:
|
||||
// the helper is told no, and asks again at its next pass, until the login is retired, or asked for again.
|
||||
func (m *Moved) Gone(_ context.Context, e provider.Entry) error {
|
||||
h := m.h
|
||||
as := m.spec.Login(e)
|
||||
if as == "" {
|
||||
return nil // refused: nothing was made for it
|
||||
}
|
||||
// The plans and the grant file as they are now, not as the last retirement pass read them: with either not
|
||||
// read, nothing is let go.
|
||||
want, read := m.asked(h.Node)
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
asked := want[as]
|
||||
if m.spec.WithGrants {
|
||||
_, granted, ok := m.granted(want)
|
||||
read = read && ok
|
||||
asked = asked || granted[as]
|
||||
}
|
||||
_, held := h.applied[as]
|
||||
_, retired := h.r.retired[as]
|
||||
switch {
|
||||
case !read:
|
||||
case asked:
|
||||
return nil // another holder of the same login still binds: nothing to retire
|
||||
case retired, !held && h.r.seeded:
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%s is kept until its retirement is stable (%d passes and %s) and within the bound, or a "+
|
||||
"person approves it", as, h.StablePasses, h.StableFor)
|
||||
}
|
||||
|
||||
// plan is as much of an assignment as is read to learn who binds the provider.
|
||||
type plan struct {
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
Provisioning *struct {
|
||||
Provisions []struct {
|
||||
Provision string `json:"provision"`
|
||||
Provider string `json:"provider"`
|
||||
ProviderModule string `json:"provider-module"`
|
||||
// As is who the consumer is at its provider's end, its slug first (issue 788).
|
||||
As string `json:"as"`
|
||||
} `json:"provisions"`
|
||||
} `json:"provisioning"`
|
||||
}
|
||||
|
||||
// asked is the logins the plans bind the provider for, and whether they could all be read: a plan that is not the
|
||||
// controller's, or no read yet, is not an answer, so nothing is counted unasked from it (novox/hq issue 241).
|
||||
func (m *Moved) asked(node string) (map[string]bool, bool) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if !m.loaded || len(m.assignments) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
want := map[string]bool{}
|
||||
for key, raw := range m.assignments {
|
||||
var a plan
|
||||
if err := json.Unmarshal(raw, &a); err != nil || a.Node+"."+a.Module != key {
|
||||
return nil, false
|
||||
}
|
||||
if a.Provisioning == nil {
|
||||
continue
|
||||
}
|
||||
for _, b := range a.Provisioning.Provisions {
|
||||
if b.Provider == node && b.ProviderModule == m.spec.Module && b.Provision == m.spec.Provision {
|
||||
as := b.As
|
||||
if as == "" {
|
||||
as = provider.ConsumerIdentity(a.Node, a.Module) // a plan that names none
|
||||
}
|
||||
want[as] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
return want, true
|
||||
}
|
||||
|
||||
// granted is what the grant file gives that no plan names (issue 937), and every login it gives; and whether it
|
||||
// could be read. The caller holds h.mu. With no grant file named there is nothing given, and that is a read.
|
||||
func (m *Moved) granted(planned map[string]bool) ([]contribution, map[string]bool, bool) {
|
||||
h := m.h
|
||||
if h.Receives == "" {
|
||||
return nil, map[string]bool{}, true
|
||||
}
|
||||
given := h.readContributions()
|
||||
if given == nil {
|
||||
return nil, nil, false
|
||||
}
|
||||
var own []contribution
|
||||
logins := map[string]bool{}
|
||||
for _, g := range given {
|
||||
logins[g.As] = true
|
||||
if !planned[g.As] {
|
||||
own = append(own, g)
|
||||
}
|
||||
}
|
||||
return own, logins, true
|
||||
}
|
||||
|
||||
// retire is one pass of the harness's retirement over what the plans bind and the grant file gives now; the logins
|
||||
// only the grant file gives are made and checked first, as the grant path does.
|
||||
func (m *Moved) retire(ctx context.Context) {
|
||||
h := m.h
|
||||
want, ok := m.asked(h.Node)
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
var given []contribution
|
||||
if ok && m.spec.WithGrants {
|
||||
var granted map[string]bool
|
||||
given, granted, ok = m.granted(want)
|
||||
for as := range granted {
|
||||
want[as] = true
|
||||
}
|
||||
}
|
||||
if !ok {
|
||||
// Not a result: the passes that must agree before anything is retired start again.
|
||||
h.r.key, h.r.count = "", 0
|
||||
return
|
||||
}
|
||||
h.seed(ctx, want)
|
||||
h.apply(ctx, given)
|
||||
h.retireUnasked(ctx, want)
|
||||
h.r.lastWant = want
|
||||
for as := range h.failing {
|
||||
if !want[as] {
|
||||
delete(h.failing, as)
|
||||
}
|
||||
}
|
||||
for as := range h.trouble {
|
||||
if _, held := h.applied[as]; !want[as] && !held {
|
||||
h.recovered(as, "no longer asked for")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// watch keeps the plans as the bus holds them, from the first read on.
|
||||
func (m *Moved) watch(ctx context.Context, bus provider.Bus) error {
|
||||
err := bus.Watch(ctx, provider.AssignmentsBucket, func(key string, value []byte) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if value == nil {
|
||||
delete(m.assignments, key)
|
||||
} else {
|
||||
m.assignments[key] = append([]byte(nil), value...)
|
||||
}
|
||||
})
|
||||
if err == nil {
|
||||
m.mu.Lock()
|
||||
m.loaded = true
|
||||
m.mu.Unlock()
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// Run answers the consumers through the helper and retires through the harness's rules, until ctx ends.
|
||||
//
|
||||
// The bus is reached through the runtime that launched this bundle, which answers only once Serve runs, and Run starts
|
||||
// beside Serve: a watch asked too early is asked again, after a wait that doubles from FirstRetry up to a minute,
|
||||
// until it is answered or ctx ends, so a start that loses the race never leaves the consumers unanswered (issue 943).
|
||||
func (m *Moved) Run(ctx context.Context, p *provider.Provider, bus provider.Bus) error {
|
||||
wait := m.FirstRetry
|
||||
if wait <= 0 {
|
||||
wait = time.Second
|
||||
}
|
||||
for tries := 1; ; tries++ {
|
||||
err := m.watch(ctx, bus)
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
if tries == 4 || tries%20 == 0 {
|
||||
m.h.say("the plans cannot be watched yet (%d tries), asking again: %v", tries, err)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-time.After(wait):
|
||||
}
|
||||
wait = min(wait*2, time.Minute)
|
||||
}
|
||||
go func() {
|
||||
for {
|
||||
m.retire(ctx)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(m.h.Every):
|
||||
}
|
||||
}
|
||||
}()
|
||||
return p.Run(ctx)
|
||||
}
|
||||
|
||||
// NewMoved is a provider handed to the provisioner: the helper answering its consumers, and the harness (never run)
|
||||
// whose retirement and tools it shares. consumersFile is where its machine's client fills its consumers. Run the
|
||||
// result with Moved.Run, passing the provider and the bus returned with it.
|
||||
func NewMoved(consumersFile string, h *Harness, spec MovedSpec) (*Moved, *provider.Provider, provider.Bus, error) {
|
||||
if consumersFile == "" {
|
||||
return nil, nil, nil, errors.New("no consumers' file")
|
||||
}
|
||||
run := newMoved(h, spec)
|
||||
bus := provider.StateBus(spec.Module)
|
||||
return run, provider.New(provider.Options{
|
||||
Node: h.Node,
|
||||
Module: spec.Module,
|
||||
Offers: map[string]provider.Offer{spec.Provision: spec.Offer},
|
||||
Bus: bus,
|
||||
Vault: provider.ConsumersFile{Paths: []string{consumersFile}},
|
||||
Make: run.Make,
|
||||
Gone: run.Gone,
|
||||
Every: h.Every,
|
||||
Logf: func(f string, a ...any) { h.say(f, a...) },
|
||||
}), bus, nil
|
||||
}
|
||||
@@ -0,0 +1,480 @@
|
||||
package harness
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/provider"
|
||||
)
|
||||
|
||||
const (
|
||||
ProvisionName = "postgres-database"
|
||||
servedPort = 5432
|
||||
)
|
||||
|
||||
// postgresSpec is what a postgres-like provider says: a login per consumer, and the grant file's logins.
|
||||
var postgresSpec = MovedSpec{Module: "postgres", Provision: ProvisionName,
|
||||
Offer: provider.Offer{Credential: provider.PerConsumer, Serves: map[string]any{"port": servedPort}},
|
||||
Login: func(e provider.Entry) string { return e.User },
|
||||
Noun: "login", Made: "login", Backend: "the backend", WithGrants: true}
|
||||
|
||||
// bucketBus is the helper's bus over in-memory buckets.
|
||||
type bucketBus struct {
|
||||
mu sync.Mutex
|
||||
buckets map[string]map[string][]byte
|
||||
}
|
||||
|
||||
func (b *bucketBus) Watch(_ context.Context, bucket string, handle func(string, []byte)) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
for k, v := range b.buckets[bucket] {
|
||||
handle(k, v)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *bucketBus) Put(_ context.Context, bucket, key string, value []byte) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
if b.buckets[bucket] == nil {
|
||||
b.buckets[bucket] = map[string][]byte{}
|
||||
}
|
||||
b.buckets[bucket][key] = append([]byte(nil), value...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *bucketBus) Delete(_ context.Context, bucket, key string) error {
|
||||
b.mu.Lock()
|
||||
defer b.mu.Unlock()
|
||||
delete(b.buckets[bucket], key)
|
||||
return nil
|
||||
}
|
||||
|
||||
// bind is one consumer's assignment, whose plan names this machine's postgres as the provider and carries what the
|
||||
// consumer contributes (issue 776).
|
||||
func (b *bucketBus) bind(node, module string, contributes map[string]any) {
|
||||
b.bindNamed("postgres", node, module, contributes, nil)
|
||||
}
|
||||
|
||||
// bindNamed is bind to a provider module, with the names the controller composed for the consumer (issue 792).
|
||||
func (b *bucketBus) bindNamed(provides, node, module string, contributes map[string]any, names map[string]string) {
|
||||
plan, _ := json.Marshal(map[string]any{"node": node, "module": module, "provisioning": map[string]any{
|
||||
"provisions": []any{map[string]any{"provision": ProvisionName, "provider": "novox", "provider-module": provides,
|
||||
"contributes": contributes, "names": names}}}})
|
||||
if err := b.Put(context.Background(), provider.AssignmentsBucket, node+"."+module, plan); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
// consumersFile writes the file the machine's client fills: one credential per consumer.
|
||||
func consumersFile(t *testing.T, bound map[[2]string]string) string {
|
||||
var list []provider.ConsumerCredential
|
||||
for k, password := range bound {
|
||||
list = append(list, provider.ConsumerCredential{Consumer: k[0], Module: k[1],
|
||||
Credential: provider.PairName("novox", ProvisionName, k[0], k[1], ""), Value: password})
|
||||
}
|
||||
raw, _ := json.Marshal(list)
|
||||
at := filepath.Join(t.TempDir(), "consumers.json")
|
||||
if err := os.WriteFile(at, raw, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return at
|
||||
}
|
||||
|
||||
// movedWorld is postgres on the helper over an in-memory bus, with a recorder for a server and a clock to move.
|
||||
type movedWorld struct {
|
||||
t *testing.T
|
||||
a *recorder
|
||||
bus *bucketBus
|
||||
h *Harness
|
||||
spec MovedSpec
|
||||
run *Moved
|
||||
p *provider.Provider
|
||||
now time.Time
|
||||
file string
|
||||
}
|
||||
|
||||
func newMovedWorld(t *testing.T, bound map[[2]string]string) *movedWorld {
|
||||
return newMovedWorldOf(t, bound, postgresSpec)
|
||||
}
|
||||
|
||||
func newMovedWorldOf(t *testing.T, bound map[[2]string]string, spec MovedSpec) *movedWorld {
|
||||
w := &movedWorld{t: t, spec: spec, a: &recorder{}, bus: &bucketBus{buckets: map[string]map[string][]byte{}},
|
||||
now: time.Date(2026, 10, 15, 12, 0, 0, 0, time.UTC)}
|
||||
w.file = consumersFile(t, bound)
|
||||
w.h = &Harness{Resource: ProvisionName, Adapter: w.a, Node: "novox", Now: func() time.Time { return w.now },
|
||||
Log: func(string, ...any) {}}
|
||||
var bus provider.Bus
|
||||
w.run, w.p, bus, _ = NewMoved(w.file, w.h, spec)
|
||||
_ = bus
|
||||
if err := w.run.watch(ctx, w.bus); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
// pass is one helper pass and one retirement pass, five seconds after the last.
|
||||
func (w *movedWorld) pass() []provider.Outcome {
|
||||
w.now = w.now.Add(5 * time.Second)
|
||||
// the plans as the bus holds them now, as the watch would hand them
|
||||
w.run.assignments = map[string][]byte{}
|
||||
for k, v := range w.bus.buckets[provider.AssignmentsBucket] {
|
||||
w.run.assignments[k] = v
|
||||
}
|
||||
// the helper reads its buckets from the in-memory bus
|
||||
w.p = provider.New(provider.Options{Node: "novox", Module: w.spec.Module, Bus: w.bus,
|
||||
Vault: provider.ConsumersFile{Paths: []string{w.file}},
|
||||
Offers: map[string]provider.Offer{ProvisionName: w.spec.Offer},
|
||||
Make: w.run.Make, Gone: w.run.Gone, Logf: func(string, ...any) {}})
|
||||
out, err := w.p.Pass(ctx)
|
||||
if err != nil {
|
||||
w.t.Fatal(err)
|
||||
}
|
||||
w.run.retire(ctx)
|
||||
return out
|
||||
}
|
||||
|
||||
func bindings(n int) (map[[2]string]string, []string) {
|
||||
bound := map[[2]string]string{}
|
||||
var logins []string
|
||||
for i := 1; i <= n; i++ {
|
||||
module := fmt.Sprintf("app%d", i)
|
||||
bound[[2]string{"ace", module}] = "pw-" + module
|
||||
logins = append(logins, provider.ConsumerIdentity("ace", module))
|
||||
}
|
||||
return bound, logins
|
||||
}
|
||||
|
||||
// The consumers' file makes the same calls on the adapter as the equivalent grant files do (the harness).
|
||||
func TestTheConsumersFileMakesWhatTheGrantFilesMake(t *testing.T) {
|
||||
letta := provider.ConsumerIdentity("ace", "letta")
|
||||
plain := provider.ConsumerIdentity("ace", "plain")
|
||||
|
||||
// Today: the grant files, through the harness.
|
||||
g := newWorld(t)
|
||||
g.give(
|
||||
map[string]any{"as": letta, "node": "ace", "values": map[string]any{"name": "letta", "extensions": []any{"vector"}}},
|
||||
map[string]any{"as": plain, "node": "ace", "values": map[string]any{"name": "plain"}},
|
||||
)
|
||||
g.h.Reconcile(ctx)
|
||||
|
||||
// The consumers' file, through the provider helper.
|
||||
w := newMovedWorld(t, map[[2]string]string{{"ace", "letta"}: "pw-" + letta, {"ace", "plain"}: "pw-" + plain})
|
||||
w.bus.bind("ace", "letta", map[string]any{"name": "letta", "extensions": []any{"vector"}})
|
||||
w.bus.bind("ace", "plain", map[string]any{"name": "plain"})
|
||||
for _, o := range w.pass() {
|
||||
if o.Status != provider.Answered {
|
||||
t.Fatalf("%+v", o)
|
||||
}
|
||||
}
|
||||
byAs := map[string]Provision{}
|
||||
for _, c := range w.a.created {
|
||||
byAs[c.As] = c
|
||||
}
|
||||
if len(w.a.created) != 2 {
|
||||
t.Fatalf("created %d, want 2", len(w.a.created))
|
||||
}
|
||||
for _, want := range g.a.created {
|
||||
c := byAs[want.As]
|
||||
if c.As != want.As || c.Password != want.Password || c.Consumer != want.Consumer ||
|
||||
!reflect.DeepEqual(orEmpty(c.Values), orEmpty(want.Values)) {
|
||||
t.Fatalf("made %+v, the grant files make %+v", c, want)
|
||||
}
|
||||
}
|
||||
// Unchanged, and held: nothing is made again, within a minute or after it.
|
||||
w.a.held = true
|
||||
w.pass()
|
||||
w.now = w.now.Add(2 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.created) != 2 {
|
||||
t.Fatalf("an unchanged pass created again: %d", len(w.a.created))
|
||||
}
|
||||
// No longer held: made again.
|
||||
w.a.held = false
|
||||
w.now = w.now.Add(2 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.created) != 4 {
|
||||
t.Fatalf("not made again: %d", len(w.a.created))
|
||||
}
|
||||
}
|
||||
|
||||
// One read without a consumer retires nothing: it takes the harness's stable passes and time, and then the entry
|
||||
// goes and the login is retired, never dropped.
|
||||
func TestAConsumerMissingFromOneReadRetiresNothing(t *testing.T) {
|
||||
bound, logins := bindings(2)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.bus.bind("ace", "app2", nil)
|
||||
w.pass()
|
||||
|
||||
// One read without app2.
|
||||
saved := w.bus.buckets[provider.AssignmentsBucket]["ace.app2"]
|
||||
w.bus.Delete(ctx, provider.AssignmentsBucket, "ace.app2")
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v on one read", w.a.removed)
|
||||
}
|
||||
if _, kept := w.bus.buckets[provider.BucketOf("postgres")]["ace.app2.postgres-database"]; !kept {
|
||||
t.Fatal("the entry went on one read")
|
||||
}
|
||||
// It is back: nothing was lost, and the entry never left the bucket.
|
||||
w.bus.Put(ctx, provider.AssignmentsBucket, "ace.app2", saved)
|
||||
for i := 0; i < 8; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(15 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 || len(w.bus.buckets[provider.BucketOf("postgres")]) != 2 {
|
||||
t.Fatalf("retired %v, entries %d", w.a.removed, len(w.bus.buckets[provider.BucketOf("postgres")]))
|
||||
}
|
||||
|
||||
// Gone for good: not retired before StablePasses and StableFor, then retired and its entry removed.
|
||||
w.bus.Delete(ctx, provider.AssignmentsBucket, "ace.app2")
|
||||
for i := 0; i < 6; i++ {
|
||||
w.pass()
|
||||
}
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v before the time was out", w.a.removed)
|
||||
}
|
||||
w.now = w.now.Add(11 * time.Minute)
|
||||
w.pass()
|
||||
w.pass()
|
||||
if !reflect.DeepEqual(w.a.removed, []string{logins[1]}) || len(w.a.deleted) != 0 {
|
||||
t.Fatalf("retired %v, deleted %v", w.a.removed, w.a.deleted)
|
||||
}
|
||||
if _, kept := w.bus.buckets[provider.BucketOf("postgres")]["ace.app2.postgres-database"]; kept {
|
||||
t.Fatal("the entry stayed after the login was retired")
|
||||
}
|
||||
}
|
||||
|
||||
// More than three at once waits for a person, as the harness's bound says.
|
||||
func TestMoreThanTheBoundWaitsForAPerson(t *testing.T) {
|
||||
bound, _ := bindings(5)
|
||||
w := newMovedWorld(t, bound)
|
||||
for i := 1; i <= 5; i++ {
|
||||
w.bus.bind("ace", fmt.Sprintf("app%d", i), nil)
|
||||
}
|
||||
w.pass()
|
||||
for i := 1; i <= 4; i++ {
|
||||
w.bus.Delete(ctx, provider.AssignmentsBucket, fmt.Sprintf("ace.app%d", i))
|
||||
}
|
||||
for i := 0; i < 6; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(11 * time.Minute)
|
||||
w.pass()
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 || w.h.r.waiting == nil {
|
||||
t.Fatalf("retired %v, waiting %v", w.a.removed, w.h.r.waiting)
|
||||
}
|
||||
}
|
||||
|
||||
// A plan list that was never read is not nobody asking.
|
||||
func TestNoPlansReadRetiresNothing(t *testing.T) {
|
||||
bound, _ := bindings(1)
|
||||
w := newMovedWorld(t, bound)
|
||||
w.bus.bind("ace", "app1", nil)
|
||||
w.pass()
|
||||
w.bus.Delete(ctx, provider.AssignmentsBucket, "ace.app1")
|
||||
for i := 0; i < 8; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(20 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v with no plans read", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer whose credential is not in the file yet is waited for: nothing is made for it.
|
||||
func TestAConsumerWithoutACredentialIsWaitedFor(t *testing.T) {
|
||||
w := newMovedWorld(t, map[[2]string]string{})
|
||||
w.bus.bind("ace", "letta", map[string]any{"name": "letta"})
|
||||
outcomes := w.pass()
|
||||
if len(outcomes) != 1 || outcomes[0].Status != provider.Waiting || len(w.a.created) != 0 {
|
||||
t.Fatalf("%+v, created %v", outcomes, w.a.created)
|
||||
}
|
||||
}
|
||||
|
||||
// Where the file is not there, postgres takes the grant directory, as before; where it is, the helper.
|
||||
// The grant path restarts itself when the consumers file is first filled.
|
||||
// A consumer whose module has a slug keeps the login its plan names, the slug first (issue 788): the adapter makes it
|
||||
// under that login, and the retirement counts it as asked for, so a pass retires nothing.
|
||||
func TestASluggedConsumerKeepsTheLoginItsPlanNames(t *testing.T) {
|
||||
agent := provider.ConsumerIdentity("ace", "agent")
|
||||
w := newMovedWorld(t, map[[2]string]string{{"ace", "claude-code"}: "pw-agent"})
|
||||
plan, _ := json.Marshal(map[string]any{"node": "ace", "module": "claude-code", "provisioning": map[string]any{
|
||||
"provisions": []any{map[string]any{"provision": ProvisionName, "provider": "novox",
|
||||
"provider-module": "postgres", "as": agent}}}})
|
||||
if err := w.bus.Put(ctx, provider.AssignmentsBucket, "ace.claude-code", plan); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, o := range w.pass() {
|
||||
if o.Status != provider.Answered {
|
||||
t.Fatalf("%+v", o)
|
||||
}
|
||||
}
|
||||
if len(w.a.created) != 1 || w.a.created[0].As != agent {
|
||||
t.Fatalf("made %+v, want one under %s", w.a.created, agent)
|
||||
}
|
||||
for i := 0; i < 8; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(15 * time.Minute)
|
||||
w.pass()
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v: the plan's login was counted unasked", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
// A provider whose spec names the plan's `as` as the login and no grant file (the keycloak shape) makes the consumer
|
||||
// under that name with the values its spec composes, and a grant file that cannot be read does not hold up its
|
||||
// retirement pass.
|
||||
func TestASpecWithoutGrantsMakesUnderThePlansAsAndIgnoresTheGrantFile(t *testing.T) {
|
||||
a := &recorder{}
|
||||
bus := &bucketBus{buckets: map[string]map[string][]byte{}}
|
||||
now := time.Date(2026, 10, 15, 12, 0, 0, 0, time.UTC)
|
||||
h := &Harness{Resource: ProvisionName, Adapter: a, Node: "novox", Now: func() time.Time { return now },
|
||||
Receives: filepath.Join(t.TempDir(), "unreadable-or-missing.json"), Log: func(string, ...any) {}}
|
||||
spec := MovedSpec{Module: "keycloak", Provision: ProvisionName,
|
||||
Offer: provider.Offer{Credential: provider.PerConsumer},
|
||||
Login: func(e provider.Entry) string { return e.As }, Noun: "client id", Made: "client", Backend: "Keycloak",
|
||||
Values: func(e provider.Entry) map[string]any { return map[string]any{"label": e.Contributes["label"]} }}
|
||||
file := consumersFile(t, map[[2]string]string{{"ace", "wiki"}: "pw-wiki"})
|
||||
run, _, _, err := NewMoved(file, h, spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
plan, _ := json.Marshal(map[string]any{"node": "ace", "module": "wiki", "provisioning": map[string]any{
|
||||
"provisions": []any{map[string]any{"provision": ProvisionName, "provider": "novox",
|
||||
"provider-module": "keycloak", "as": "wiki-client", "contributes": map[string]any{"label": "Wiki"}}}}})
|
||||
_ = bus.Put(ctx, provider.AssignmentsBucket, "ace.wiki", plan)
|
||||
if err := run.watch(ctx, bus); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p := provider.New(provider.Options{Node: "novox", Module: "keycloak", Bus: bus,
|
||||
Vault: provider.ConsumersFile{Paths: []string{file}},
|
||||
Offers: map[string]provider.Offer{ProvisionName: spec.Offer}, Make: run.Make, Gone: run.Gone,
|
||||
Logf: func(string, ...any) {}})
|
||||
out, err := p.Pass(ctx)
|
||||
if err != nil || len(out) != 1 || out[0].Status != provider.Answered {
|
||||
t.Fatalf("%v %+v", err, out)
|
||||
}
|
||||
run.retire(ctx)
|
||||
if len(a.created) != 1 || a.created[0].As != "wiki-client" || a.created[0].Password != "pw-wiki" ||
|
||||
a.created[0].Values["label"] != "Wiki" {
|
||||
t.Fatalf("made %+v", a.created)
|
||||
}
|
||||
}
|
||||
|
||||
const testIssuer = "https://id.example.org/realms/Novox"
|
||||
|
||||
var (
|
||||
grafanaContributes = map[string]any{"label": "grafana", "endpoint": "web", "callback": "/login/generic_oauth"}
|
||||
grafanaNames = map[string]string{"name": "grafana.example.org", "internal-name": "grafana.home.internal"}
|
||||
)
|
||||
|
||||
// keycloakSpec is what a keycloak-like provider says: the client id is the plan's `as`, the values are what the
|
||||
// consumer contributes with the controller's names over them, and no grant file is consulted.
|
||||
var keycloakSpec = MovedSpec{Module: "keycloak", Provision: ProvisionName,
|
||||
Offer: provider.Offer{Credential: provider.PerConsumer, Serves: map[string]any{"issuer": testIssuer,
|
||||
"token-path": "/protocol/openid-connect/token"}},
|
||||
Login: func(e provider.Entry) string { return e.As },
|
||||
Values: func(e provider.Entry) map[string]any {
|
||||
values := map[string]any{}
|
||||
for k, v := range e.Contributes {
|
||||
values[k] = v
|
||||
}
|
||||
for k, name := range e.Names {
|
||||
values[k] = name
|
||||
}
|
||||
return values
|
||||
},
|
||||
Noun: "client id", Made: "client", Backend: "Keycloak"}
|
||||
|
||||
// The consumers' file makes the same call on the adapter as the equivalent grant file does (the harness), for a
|
||||
// provider of the keycloak shape: the same client id, secret and values; and the entry tells the consumer what the
|
||||
// provider serves.
|
||||
func TestTheConsumersFileMakesWhatTheGrantFileMakesForTheKeycloakShape(t *testing.T) {
|
||||
grafana := provider.ConsumerIdentity("home", "grafana")
|
||||
values := map[string]any{"name": grafanaNames["name"], "internal-name": grafanaNames["internal-name"]}
|
||||
for k, v := range grafanaContributes {
|
||||
values[k] = v
|
||||
}
|
||||
g := newWorld(t)
|
||||
g.give(map[string]any{"as": grafana, "node": "home", "values": values})
|
||||
g.h.Reconcile(ctx)
|
||||
|
||||
w := newMovedWorldOf(t, map[[2]string]string{{"home", "grafana"}: "pw-" + grafana}, keycloakSpec)
|
||||
w.bus.bindNamed("keycloak", "home", "grafana", grafanaContributes, grafanaNames)
|
||||
for _, o := range w.pass() {
|
||||
if o.Status != provider.Answered {
|
||||
t.Fatalf("%+v", o)
|
||||
}
|
||||
}
|
||||
if len(w.a.created) != 1 || len(g.a.created) != 1 {
|
||||
t.Fatalf("created %d through the helper, %d through the harness, want 1 each", len(w.a.created), len(g.a.created))
|
||||
}
|
||||
c, want := w.a.created[0], g.a.created[0]
|
||||
if c.As != want.As || c.Password != want.Password || c.Consumer != want.Consumer ||
|
||||
!reflect.DeepEqual(orEmpty(c.Values), orEmpty(want.Values)) {
|
||||
t.Fatalf("made %+v, the grant file makes %+v", c, want)
|
||||
}
|
||||
var entry provider.Entry
|
||||
if err := json.Unmarshal(w.bus.buckets[provider.BucketOf("keycloak")]["home.grafana."+ProvisionName], &entry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entry.Values["issuer"] != testIssuer || entry.Values["token-path"] != "/protocol/openid-connect/token" {
|
||||
t.Fatalf("served %v", entry.Values)
|
||||
}
|
||||
w.a.held = true
|
||||
w.pass()
|
||||
w.now = w.now.Add(2 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.created) != 1 {
|
||||
t.Fatalf("an unchanged pass created again: %d", len(w.a.created))
|
||||
}
|
||||
w.a.held = false
|
||||
w.now = w.now.Add(2 * time.Minute)
|
||||
w.pass()
|
||||
if len(w.a.created) != 2 {
|
||||
t.Fatalf("not made again: %d", len(w.a.created))
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer whose plan names its client id (a slugged module) keeps it, and the retirement counts it as asked for.
|
||||
func TestAConsumerNamedByItsPlanKeepsItsClientId(t *testing.T) {
|
||||
agent := provider.ConsumerIdentity("home", "dashboard")
|
||||
w := newMovedWorldOf(t, map[[2]string]string{{"home", "grafana-2"}: "pw"}, keycloakSpec)
|
||||
plan, _ := json.Marshal(map[string]any{"node": "home", "module": "grafana-2", "provisioning": map[string]any{
|
||||
"provisions": []any{map[string]any{"provision": ProvisionName, "provider": "novox", "provider-module": "keycloak",
|
||||
"as": agent, "contributes": grafanaContributes, "names": grafanaNames}}}})
|
||||
if err := w.bus.Put(ctx, provider.AssignmentsBucket, "home.grafana-2", plan); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, o := range w.pass() {
|
||||
if o.Status != provider.Answered {
|
||||
t.Fatalf("%+v", o)
|
||||
}
|
||||
}
|
||||
if len(w.a.created) != 1 || w.a.created[0].As != agent {
|
||||
t.Fatalf("made %+v, want one under %s", w.a.created, agent)
|
||||
}
|
||||
for i := 0; i < 8; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.now = w.now.Add(15 * time.Minute)
|
||||
w.pass()
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired %v: the plan's client id was counted unasked", w.a.removed)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,623 @@
|
||||
package harness
|
||||
|
||||
// What becomes of a consumer the mesh no longer asks for (novox/hq ADR 0230 — the operator's model,
|
||||
// decided 2026-10-06; it replaces ADR 0229's withdrawal brake, which let one consumer go every hour).
|
||||
//
|
||||
// A consumer — a login, a client, a key, and the data behind it — is in one of three states:
|
||||
//
|
||||
// 1. ACTIVE.
|
||||
// 2. RETIRED. The mesh stopped asking for it: its access is disabled, reversibly, and its login and
|
||||
// data are marked "to delete" with when and why. Nothing is deleted. Asked for again, the ordinary
|
||||
// create re-enables it at once, as it was.
|
||||
// 3. DELETED, only through `cleanup delete`, a person's act, which this provider executes because it
|
||||
// owns its backend.
|
||||
//
|
||||
// **Stable removals.** A consumer is retired only once the same set of consumers has gone unasked BOTH
|
||||
// in StablePasses (5) consecutive passes that read the file AND for at least StableFor (10 minutes)
|
||||
// since the first pass that saw it. Five passes alone are twenty-five seconds — shorter than a
|
||||
// controller restart, a store reconnecting or a file half written. A pass that could not read the file
|
||||
// is not a result and starts both again; so does a different set. Additions and changes are never
|
||||
// delayed.
|
||||
//
|
||||
// **Too many is a person.** A stable set of more than RetireAtOnce (3), or — where more than one is
|
||||
// held — of more than RetireFraction (half) of those held, retires nothing: it WAITS, said in the
|
||||
// journal and announced `provisioner.retirement` `waiting` (again every SayAgainEvery), which the
|
||||
// controller raises as an urgent condition, until `retire approve <node> <module>` or `retire reject`.
|
||||
// An unassignment the controller made itself is no exception: many changes at once means a person is
|
||||
// at work, and a person confirms once. On 2026-10-04 one misread file withdrew seven consumers at once
|
||||
// (issue 241); under this rule that is a question, not an act.
|
||||
//
|
||||
// **The backend remembers, not this process.** What is retired, since when and why is read from the
|
||||
// backend's own mark (Adapter.Inventory), so a restart forgets nothing; a consumer the backend holds
|
||||
// active that the mesh no longer asks for is retired by the same rules after a restart as before one.
|
||||
// A consumer found disabled without the mark — withdrawn before this record — is ADOPTED as retired:
|
||||
// marked, said, announced `adopted`, and its clock starts then.
|
||||
//
|
||||
// **A rejection lives as long as this process.** Rejected, the set is kept active and not asked about
|
||||
// again while it stays the same set; a restart asks again — loudly, never silently.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// EventRetirement is the one event a provider says about retirement, its `change` saying what
|
||||
// happened. The controller derives the permission to emit it for every module that receives
|
||||
// contributions, as it does the standing events (novox/hq ADR 0224, 0230).
|
||||
const EventRetirement = "provisioner.retirement"
|
||||
|
||||
// The changes EventRetirement carries.
|
||||
const (
|
||||
ChangeWaiting = "waiting" // a stable set over the bound waits for a person
|
||||
ChangeSettled = "settled" // a waiting or rejected set ended without being retired
|
||||
ChangeApproved = "approved" // a person approved the waiting (or rejected) set
|
||||
ChangeRejected = "rejected" // a person kept the waiting set active
|
||||
ChangeRetired = "retired" // consumers disabled and marked to delete
|
||||
ChangeReenabled = "reenabled" // a retired consumer asked for again, enabled as it was
|
||||
ChangeDeleted = "deleted" // a retired consumer deleted, by a person
|
||||
ChangeAdopted = "adopted" // found disabled without the mark, now retired on record
|
||||
)
|
||||
|
||||
// StableFor is the least time the same unasked set must hold before it is retired (novox/hq ADR 0230):
|
||||
// longer than a controller restart, a store reconnecting or a file half written.
|
||||
const StableFor = 10 * time.Minute
|
||||
|
||||
// KindConsumer is a retired consumer. A backend may list other things set aside for deletion under a
|
||||
// word of its own (postgres: a database renamed aside).
|
||||
const KindConsumer = "consumer"
|
||||
|
||||
// Retired is one thing a provider holds retired, as its backend's mark says.
|
||||
type Retired struct {
|
||||
Consumer string
|
||||
// Node is the consumer's machine, where the mark records it.
|
||||
Node string
|
||||
// RetiredAt is when it was retired; zero for one found disabled without the mesh's mark.
|
||||
RetiredAt time.Time
|
||||
Why string
|
||||
// SizeBytes is what deleting it would free; -1 when the backend cannot say.
|
||||
SizeBytes int64
|
||||
Kind string
|
||||
}
|
||||
|
||||
// Inventory is what a backend holds that the mesh made.
|
||||
type Inventory struct {
|
||||
Active []string
|
||||
Retired []Retired
|
||||
// Sizes is what each active consumer keeps on the backend, in bytes, where the backend can say: what
|
||||
// the controller tells an empty replacement of a consumer's data by (novox/hq ADR 0233).
|
||||
Sizes map[string]int64
|
||||
}
|
||||
|
||||
// retirement is the loop's memory of the sets it is counting, waiting on and was told to keep.
|
||||
type retirement struct {
|
||||
key string // the unasked set, joined
|
||||
count int // consecutive passes that saw it
|
||||
firstSeen time.Time
|
||||
waiting *waitingSet
|
||||
rejected *rejectedSet
|
||||
retired map[string]retiredEntry
|
||||
lastWant map[string]bool
|
||||
seeded bool
|
||||
seedSaid string
|
||||
// noneSaid is whether this process has said that nothing waits: an earlier process may have left a set
|
||||
// waiting that this one never knew (issue 937).
|
||||
noneSaid bool
|
||||
}
|
||||
|
||||
type waitingSet struct {
|
||||
set []string
|
||||
since time.Time
|
||||
saidAt time.Time
|
||||
held int
|
||||
}
|
||||
|
||||
type rejectedSet struct {
|
||||
set []string
|
||||
by, why string
|
||||
at time.Time
|
||||
}
|
||||
|
||||
type retiredEntry struct {
|
||||
node string
|
||||
at time.Time
|
||||
why string
|
||||
}
|
||||
|
||||
// seed reads what the backend holds, once per process: an active consumer the mesh no longer asks
|
||||
// for is held, so it is retired by the same rules as one this process made; one found disabled
|
||||
// without the mark is adopted as retired.
|
||||
func (h *Harness) seed(ctx context.Context, want map[string]bool) {
|
||||
if h.r.seeded {
|
||||
return
|
||||
}
|
||||
inv, err := h.Adapter.Inventory(ctx)
|
||||
if err != nil {
|
||||
if said := err.Error(); said != h.r.seedSaid {
|
||||
h.say("cannot list what the backend holds (%v); a consumer the mesh stopped asking for while this "+
|
||||
"provider was down is not retired until it can", err)
|
||||
h.r.seedSaid = said
|
||||
}
|
||||
return
|
||||
}
|
||||
h.r.seeded = true
|
||||
for _, as := range inv.Active {
|
||||
if _, held := h.applied[as]; !held && !want[as] {
|
||||
// No hash: asked for again, it is applied again, which is harmless and marks it.
|
||||
h.applied[as] = appliedEntry{}
|
||||
h.say("%s: held by the backend and no longer asked for; retired once that holds for %d passes "+
|
||||
"and %s (novox/hq ADR 0230)", as, h.StablePasses, h.StableFor)
|
||||
}
|
||||
}
|
||||
now := h.Now()
|
||||
for _, r := range inv.Retired {
|
||||
if r.Kind != "" && r.Kind != KindConsumer {
|
||||
continue
|
||||
}
|
||||
if !r.RetiredAt.IsZero() {
|
||||
h.r.retired[r.Consumer] = retiredEntry{node: r.Node, at: r.RetiredAt, why: r.Why}
|
||||
continue
|
||||
}
|
||||
if want[r.Consumer] {
|
||||
continue // asked for: this pass's create enables it
|
||||
}
|
||||
why := "found disabled without the mesh's mark — withdrawn before retirement was recorded " +
|
||||
"(novox/hq ADR 0230); retired as found"
|
||||
if err := h.Adapter.Retire(ctx, r.Consumer, nil, why, now); err != nil {
|
||||
h.say("%s: found disabled and could not be marked retired, will try at the next start: %v", r.Consumer, err)
|
||||
continue
|
||||
}
|
||||
h.r.retired[r.Consumer] = retiredEntry{node: r.Node, at: now, why: why}
|
||||
h.say("%s: ADOPTED as retired — %s", r.Consumer, why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeAdopted, []map[string]any{
|
||||
{"consumer": r.Consumer, "node": r.Node, "retired_at": stamp(now), "why": why, "size_bytes": r.SizeBytes},
|
||||
}, map[string]any{"why": why}))
|
||||
}
|
||||
}
|
||||
|
||||
// retireUnasked counts the passes that saw the same set no longer asked for and, once it is stable,
|
||||
// retires it — or, past the bound, waits for a person.
|
||||
func (h *Harness) retireUnasked(ctx context.Context, want map[string]bool) {
|
||||
var unasked []string
|
||||
for as := range h.applied {
|
||||
if !want[as] {
|
||||
unasked = append(unasked, as)
|
||||
}
|
||||
}
|
||||
sort.Strings(unasked)
|
||||
key := strings.Join(unasked, "\x00")
|
||||
now := h.Now()
|
||||
switch {
|
||||
case len(unasked) == 0:
|
||||
h.settle("every consumer held is asked for again")
|
||||
h.r.key, h.r.count = "", 0
|
||||
return
|
||||
case key != h.r.key:
|
||||
if h.r.waiting != nil || h.r.rejected != nil {
|
||||
h.settle("the set the mesh no longer asks for changed")
|
||||
}
|
||||
h.r.key, h.r.count, h.r.firstSeen = key, 1, now
|
||||
h.say("no longer asked for: %s; retired once the same set holds for %d passes and %s",
|
||||
strings.Join(unasked, ", "), h.StablePasses, h.StableFor)
|
||||
default:
|
||||
h.r.count++
|
||||
}
|
||||
if h.r.rejected != nil || h.r.count < h.StablePasses || now.Sub(h.r.firstSeen) < h.StableFor {
|
||||
return
|
||||
}
|
||||
if h.overTheBound(len(unasked), len(h.applied)) {
|
||||
h.wait(unasked)
|
||||
return
|
||||
}
|
||||
why := fmt.Sprintf("the mesh stopped asking for it: the same result in %d consecutive passes over %s, from %s",
|
||||
h.r.count, now.Sub(h.r.firstSeen).Round(time.Second), stamp(h.r.firstSeen))
|
||||
h.retire(ctx, unasked, why, nil)
|
||||
}
|
||||
|
||||
// overTheBound says retiring n of the held consumers at once needs a person.
|
||||
func (h *Harness) overTheBound(n, held int) bool {
|
||||
if n == 0 {
|
||||
return false
|
||||
}
|
||||
return n > h.RetireAtOnce || (held > 1 && float64(n) > h.RetireFraction*float64(held))
|
||||
}
|
||||
|
||||
func (h *Harness) bound(held int) string {
|
||||
return fmt.Sprintf("more than %d at once, or more than %.0f%% of the %d held", h.RetireAtOnce,
|
||||
h.RetireFraction*100, held)
|
||||
}
|
||||
|
||||
// wait holds a stable set over the bound for a person, said once and announced again every
|
||||
// SayAgainEvery so a controller that missed the first hears the next.
|
||||
func (h *Harness) wait(set []string) {
|
||||
now := h.Now()
|
||||
w := h.r.waiting
|
||||
if w == nil {
|
||||
w = &waitingSet{set: set, since: now, held: len(h.applied)}
|
||||
h.r.waiting = w
|
||||
h.say("RETIREMENT WAITS FOR A PERSON: the mesh no longer asks for %d of the %d consumer(s) this provider "+
|
||||
"holds (%s), %s. Nothing is retired; each stays active until `retire approve %s <module>` or "+
|
||||
"`retire reject` (novox/hq ADR 0230)", len(set), w.held, strings.Join(set, ", "), h.bound(w.held), h.Node)
|
||||
} else if now.Sub(w.saidAt) < h.SayAgainEvery {
|
||||
return
|
||||
}
|
||||
w.saidAt = now
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeWaiting, h.named(set), map[string]any{
|
||||
"held": w.held, "bound": h.bound(w.held), "since": stamp(w.since),
|
||||
}))
|
||||
}
|
||||
|
||||
// settle ends a waiting or rejected set that the mesh's own answer made moot.
|
||||
func (h *Harness) settle(why string) {
|
||||
var set []string
|
||||
switch {
|
||||
case h.r.waiting != nil:
|
||||
set = h.r.waiting.set
|
||||
case h.r.rejected != nil:
|
||||
set = h.r.rejected.set
|
||||
case h.r.noneSaid:
|
||||
return
|
||||
default:
|
||||
// A restart forgets a set an earlier process waited on, and so would never say it ended: the
|
||||
// controller's condition would ask a person to approve what nobody waits on any more (issue 937).
|
||||
// The first time this process finds every consumer held asked for, it says so once, with no set.
|
||||
h.r.noneSaid = true
|
||||
h.say("nothing waits for a person: %s", why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeSettled, []map[string]any{}, map[string]any{"why": why}))
|
||||
return
|
||||
}
|
||||
h.r.waiting, h.r.rejected = nil, nil
|
||||
h.say("retirement of %s settled without retiring: %s", strings.Join(set, ", "), why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeSettled, h.named(set), map[string]any{"why": why}))
|
||||
}
|
||||
|
||||
// retire disables and marks each consumer of set; one that fails stays held and is tried again once
|
||||
// its set is stable again.
|
||||
func (h *Harness) retire(ctx context.Context, set []string, why string, extra map[string]any) []string {
|
||||
now := h.Now()
|
||||
held := len(h.applied)
|
||||
var done []string
|
||||
var said []map[string]any
|
||||
for _, as := range set {
|
||||
was, ok := h.applied[as]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if err := h.Adapter.Retire(ctx, as, was.derived, why, now); err != nil {
|
||||
h.say("%s: retiring failed, will try again: %v", as, err)
|
||||
continue
|
||||
}
|
||||
delete(h.applied, as)
|
||||
delete(h.lost, as)
|
||||
delete(h.failing, as)
|
||||
h.r.retired[as] = retiredEntry{node: was.node, at: now, why: why}
|
||||
h.recovered(as, "retired")
|
||||
h.say("%s: RETIRED — its access disabled and its data kept, marked to delete (%s). Asked for again "+
|
||||
"it is re-enabled as it was; it is deleted only by `cleanup delete` (novox/hq ADR 0230)", as, why)
|
||||
done = append(done, as)
|
||||
said = append(said, map[string]any{"consumer": as, "node": was.node, "retired_at": stamp(now), "why": why})
|
||||
}
|
||||
h.r.key, h.r.count, h.r.waiting, h.r.rejected = "", 0, nil, nil
|
||||
if len(done) > 0 {
|
||||
body := map[string]any{"held": held, "why": why}
|
||||
for k, v := range extra {
|
||||
body[k] = v
|
||||
}
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeRetired, said, body))
|
||||
}
|
||||
return done
|
||||
}
|
||||
|
||||
// reenabled says a retired consumer was asked for again and its create enabled it.
|
||||
func (h *Harness) reenabled(as, node string) {
|
||||
e, was := h.r.retired[as]
|
||||
if !was {
|
||||
return
|
||||
}
|
||||
delete(h.r.retired, as)
|
||||
h.say("%s: asked for again — re-enabled as it was, its mark to delete cleared (retired %s: %s)", as,
|
||||
stamp(e.at), e.why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeReenabled, []map[string]any{
|
||||
{"consumer": as, "node": node, "retired_at": stamp(e.at), "why": e.why},
|
||||
}, nil))
|
||||
}
|
||||
|
||||
// Approve retires exactly the set waiting (or the set rejected) — a person's confirmation.
|
||||
func (h *Harness) Approve(ctx context.Context, consumers []string, why, by, via string) ([]string, error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
if strings.TrimSpace(why) == "" {
|
||||
return nil, errors.New("approve: say why")
|
||||
}
|
||||
set := sorted(consumers)
|
||||
var held []string
|
||||
switch {
|
||||
case h.r.waiting != nil && same(set, h.r.waiting.set):
|
||||
held = h.r.waiting.set
|
||||
case h.r.rejected != nil && same(set, h.r.rejected.set):
|
||||
held = h.r.rejected.set
|
||||
default:
|
||||
return nil, fmt.Errorf("approve: %s is not the set waiting here — %s", orNone(set), h.waitingWords())
|
||||
}
|
||||
h.say("retirement of %s APPROVED by %s: %s", strings.Join(held, ", "), orSomebody(by), why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeApproved, h.named(held),
|
||||
map[string]any{"why": why, "by": by, "via": via}))
|
||||
reason := fmt.Sprintf("the mesh stopped asking for it; approved by %s: %s", orSomebody(by), why)
|
||||
return h.retire(ctx, held, reason, map[string]any{"by": by, "via": via}), nil
|
||||
}
|
||||
|
||||
// Reject keeps the waiting set active; it is not asked about again while it stays the same set.
|
||||
func (h *Harness) Reject(consumers []string, why, by, via string) ([]string, error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
if strings.TrimSpace(why) == "" {
|
||||
return nil, errors.New("reject: say why")
|
||||
}
|
||||
set := sorted(consumers)
|
||||
if h.r.waiting == nil || !same(set, h.r.waiting.set) {
|
||||
return nil, fmt.Errorf("reject: %s is not the set waiting here — %s", orNone(set), h.waitingWords())
|
||||
}
|
||||
h.r.rejected = &rejectedSet{set: h.r.waiting.set, by: by, why: why, at: h.Now()}
|
||||
h.r.waiting = nil
|
||||
h.say("retirement of %s REJECTED by %s: %s. Kept active, and not asked about again while the mesh goes on "+
|
||||
"not asking for exactly these (until this provider restarts)", strings.Join(set, ", "), orSomebody(by), why)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeRejected, h.named(set),
|
||||
map[string]any{"why": why, "by": by, "via": via}))
|
||||
return set, nil
|
||||
}
|
||||
|
||||
// DeleteRetired deletes one retired consumer, by a person's word: never an active one, never one the
|
||||
// mesh asks for.
|
||||
func (h *Harness) DeleteRetired(ctx context.Context, consumer, why, by, via string) (int64, error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
if strings.TrimSpace(why) == "" {
|
||||
return 0, errors.New("delete: say why")
|
||||
}
|
||||
if h.r.lastWant[consumer] {
|
||||
return 0, fmt.Errorf("delete: the mesh asks for %s — it is not retired", consumer)
|
||||
}
|
||||
if _, held := h.applied[consumer]; held {
|
||||
return 0, fmt.Errorf("delete: %s is active here — only a retired consumer is deleted", consumer)
|
||||
}
|
||||
inv, err := h.Adapter.Inventory(ctx)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("delete: what the backend holds cannot be read, so nothing is deleted: %w", err)
|
||||
}
|
||||
var found *Retired
|
||||
for i := range inv.Retired {
|
||||
if inv.Retired[i].Consumer == consumer {
|
||||
found = &inv.Retired[i]
|
||||
}
|
||||
}
|
||||
if found == nil {
|
||||
return 0, fmt.Errorf("delete: %s is not retired here — only a retired consumer is deleted", consumer)
|
||||
}
|
||||
freed, err := h.Adapter.Delete(ctx, *found)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
delete(h.r.retired, consumer)
|
||||
h.say("%s: DELETED by %s: %s (retired %s: %s; %d bytes freed)", consumer, orSomebody(by), why,
|
||||
stampOr(found.RetiredAt), found.Why, freed)
|
||||
h.announce(EventRetirement, h.retirementBody(ChangeDeleted, []map[string]any{{
|
||||
"consumer": consumer, "node": found.Node, "retired_at": stampOr(found.RetiredAt), "why": found.Why,
|
||||
"size_bytes": found.SizeBytes, "kind": kindOf(*found),
|
||||
}}, map[string]any{"why": why, "by": by, "via": via, "freed_bytes": freed}))
|
||||
return freed, nil
|
||||
}
|
||||
|
||||
// Retirement is what a person (and the controller's `cleanup list` and its probe) asks: what is
|
||||
// held, waiting, rejected and retired here.
|
||||
func (h *Harness) Retirement(ctx context.Context) (map[string]any, error) {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
inv, err := h.Adapter.Inventory(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var held []string
|
||||
for as := range h.applied {
|
||||
held = append(held, as)
|
||||
}
|
||||
sort.Strings(held)
|
||||
retired := []map[string]any{}
|
||||
for _, r := range inv.Retired {
|
||||
retired = append(retired, map[string]any{"consumer": r.Consumer, "node": r.Node,
|
||||
"retired_at": stampOr(r.RetiredAt), "why": r.Why, "size_bytes": r.SizeBytes, "kind": kindOf(r)})
|
||||
}
|
||||
sizes := map[string]int64{}
|
||||
for _, as := range held {
|
||||
if size, ok := inv.Sizes[as]; ok && size >= 0 {
|
||||
sizes[as] = size
|
||||
}
|
||||
}
|
||||
out := map[string]any{"resource": h.Resource, "node": h.Node, "held": orEmptyList(held), "held_sizes": sizes,
|
||||
"stable_passes": h.StablePasses, "stable_for_seconds": int(h.StableFor.Seconds()), "bound": h.bound(len(h.applied)), "waiting": nil, "rejected": nil,
|
||||
"retired": retired}
|
||||
if w := h.r.waiting; w != nil {
|
||||
out["waiting"] = map[string]any{"consumers": h.named(w.set), "since": stamp(w.since), "held": w.held}
|
||||
}
|
||||
if r := h.r.rejected; r != nil {
|
||||
out["rejected"] = map[string]any{"consumers": h.named(r.set), "by": r.by, "why": r.why, "at": stamp(r.at)}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (h *Harness) waitingWords() string {
|
||||
switch {
|
||||
case h.r.waiting != nil:
|
||||
return "waiting: " + strings.Join(h.r.waiting.set, ", ")
|
||||
case h.r.rejected != nil:
|
||||
return "nothing waits; rejected and kept: " + strings.Join(h.r.rejected.set, ", ")
|
||||
}
|
||||
return "nothing waits for a person here"
|
||||
}
|
||||
|
||||
// named is a set with each consumer's machine, as the events and the tools say it.
|
||||
func (h *Harness) named(set []string) []map[string]any {
|
||||
out := make([]map[string]any, 0, len(set))
|
||||
for _, as := range set {
|
||||
out = append(out, map[string]any{"consumer": as, "node": h.applied[as].node})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (h *Harness) retirementBody(change string, consumers []map[string]any, extra map[string]any) map[string]any {
|
||||
body := map[string]any{"provider": h.Resource, "provider-node": h.Node, "change": change,
|
||||
"consumers": consumers, "at": stamp(h.Now())}
|
||||
for k, v := range extra {
|
||||
body[k] = v
|
||||
}
|
||||
return body
|
||||
}
|
||||
|
||||
// RetirementTools are the four tools every provider serves for retirement, the same names in every
|
||||
// module: the controller's `retire` and `cleanup` verbs ask them on the provider's machine.
|
||||
func RetirementTools(h *Harness) []stdio.Tool {
|
||||
if h == nil {
|
||||
return nil
|
||||
}
|
||||
ask := func() (context.Context, context.CancelFunc) {
|
||||
return context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
}
|
||||
who := map[string]any{
|
||||
"why": map[string]any{"type": "string", "description": "why — required, kept in the hand-act log"},
|
||||
"by": map[string]any{"type": "string", "description": "who decided"},
|
||||
"via": map[string]any{"type": "string", "description": "mesh-controller when asked through its verbs"},
|
||||
}
|
||||
withSet := map[string]any{"consumers": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
"description": "the set, exactly as provisioner_retirement names it"}}
|
||||
for k, v := range who {
|
||||
withSet[k] = v
|
||||
}
|
||||
withOne := map[string]any{
|
||||
"consumer": map[string]any{"type": "string", "description": "the retired consumer to delete"},
|
||||
"confirm": map[string]any{"type": "string", "description": "the consumer's name again, to say this is meant"},
|
||||
}
|
||||
for k, v := range who {
|
||||
withOne[k] = v
|
||||
}
|
||||
return []stdio.Tool{
|
||||
{Name: "provisioner_retirement",
|
||||
Description: "What this provider holds, what waits for a person to approve its retirement, what was rejected, " +
|
||||
"and every retired consumer with when, why and its size (novox/hq ADR 0230).",
|
||||
Run: func(map[string]any) (any, error) {
|
||||
ctx, cancel := ask()
|
||||
defer cancel()
|
||||
return h.Retirement(ctx)
|
||||
}},
|
||||
{Name: "provisioner_retire_approve",
|
||||
Description: "Retire exactly the set waiting for a person (or the set rejected earlier): access disabled, data " +
|
||||
"kept and marked to delete. Use the controller's `retire approve`, which records the hand act.",
|
||||
Input: withSet,
|
||||
Run: func(args map[string]any) (any, error) {
|
||||
ctx, cancel := ask()
|
||||
defer cancel()
|
||||
done, err := h.Approve(ctx, strs(args["consumers"]), argString(args, "why"), argString(args, "by"), argString(args, "via"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"retired": orEmptyList(done)}, nil
|
||||
}},
|
||||
{Name: "provisioner_retire_reject",
|
||||
Description: "Keep the set waiting for a person active. Use the controller's `retire reject`.",
|
||||
Input: withSet,
|
||||
Run: func(args map[string]any) (any, error) {
|
||||
kept, err := h.Reject(strs(args["consumers"]), argString(args, "why"), argString(args, "by"), argString(args, "via"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"kept": kept}, nil
|
||||
}},
|
||||
{Name: "provisioner_delete",
|
||||
Description: "Delete one RETIRED consumer — its login and its data — for good. Refused for anything active or " +
|
||||
"asked for. Use the controller's `cleanup delete`, which records the hand act.",
|
||||
Input: withOne,
|
||||
Run: func(args map[string]any) (any, error) {
|
||||
consumer := argString(args, "consumer")
|
||||
if consumer == "" || argString(args, "confirm") != consumer {
|
||||
return nil, errors.New("delete: name the consumer, and repeat it in confirm")
|
||||
}
|
||||
ctx, cancel := ask()
|
||||
defer cancel()
|
||||
freed, err := h.DeleteRetired(ctx, consumer, argString(args, "why"), argString(args, "by"), argString(args, "via"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"deleted": consumer, "freed_bytes": freed}, nil
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
func strs(v any) []string {
|
||||
list, _ := v.([]any)
|
||||
out := []string{}
|
||||
for _, x := range list {
|
||||
if s, ok := x.(string); ok && s != "" {
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func sorted(list []string) []string {
|
||||
out := append([]string(nil), list...)
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func same(a, b []string) bool {
|
||||
return strings.Join(sorted(a), "\x00") == strings.Join(sorted(b), "\x00")
|
||||
}
|
||||
|
||||
func orNone(set []string) string {
|
||||
if len(set) == 0 {
|
||||
return "an empty set"
|
||||
}
|
||||
return strings.Join(set, ", ")
|
||||
}
|
||||
|
||||
func orSomebody(by string) string {
|
||||
if by == "" {
|
||||
return "a person"
|
||||
}
|
||||
return by
|
||||
}
|
||||
|
||||
func orEmptyList(list []string) []string {
|
||||
if list == nil {
|
||||
return []string{}
|
||||
}
|
||||
return list
|
||||
}
|
||||
|
||||
func kindOf(r Retired) string {
|
||||
if r.Kind == "" {
|
||||
return KindConsumer
|
||||
}
|
||||
return r.Kind
|
||||
}
|
||||
|
||||
func stamp(t time.Time) string { return t.UTC().Format(time.RFC3339) }
|
||||
|
||||
func stampOr(t time.Time) string {
|
||||
if t.IsZero() {
|
||||
return ""
|
||||
}
|
||||
return stamp(t)
|
||||
}
|
||||
|
||||
func argString(args map[string]any, key string) string {
|
||||
s, _ := args[key].(string)
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,571 @@
|
||||
package harness
|
||||
|
||||
// Retirement (novox/hq ADR 0230), as the shared loop does it: a consumer the mesh stops asking for is
|
||||
// retired only after the same result in five consecutive passes, too many at once wait for a person,
|
||||
// asked for again it is re-enabled, and only a person deletes. The same file in every Go provider.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// recorder is a backend that remembers what it holds, active and retired, as a real one's mark does.
|
||||
type recorder struct {
|
||||
created []Provision
|
||||
removed []string // retired, in order
|
||||
deleted []string
|
||||
held bool
|
||||
failing error
|
||||
holdsErr error
|
||||
retErr error
|
||||
inv Inventory
|
||||
invErr error
|
||||
}
|
||||
|
||||
func (r *recorder) Create(_ context.Context, p Provision) error {
|
||||
if r.failing != nil {
|
||||
return r.failing
|
||||
}
|
||||
r.created = append(r.created, p)
|
||||
r.inv.Retired = withoutRetired(r.inv.Retired, p.As)
|
||||
if !listHas(r.inv.Active, p.As) {
|
||||
r.inv.Active = append(r.inv.Active, p.As)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *recorder) Retire(_ context.Context, as string, _ map[string]any, why string, at time.Time) error {
|
||||
if r.retErr != nil {
|
||||
return r.retErr
|
||||
}
|
||||
r.removed = append(r.removed, as)
|
||||
r.inv.Active = without(r.inv.Active, as)
|
||||
r.inv.Retired = append(withoutRetired(r.inv.Retired, as),
|
||||
Retired{Consumer: as, RetiredAt: at, Why: why, SizeBytes: 42, Kind: KindConsumer})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *recorder) Holds(context.Context, Provision) (bool, error) {
|
||||
if r.holdsErr != nil {
|
||||
return false, r.holdsErr
|
||||
}
|
||||
return r.held, nil
|
||||
}
|
||||
|
||||
func (r *recorder) Inventory(context.Context) (Inventory, error) { return r.inv, r.invErr }
|
||||
|
||||
func (r *recorder) Delete(_ context.Context, x Retired) (int64, error) {
|
||||
r.deleted = append(r.deleted, x.Consumer)
|
||||
r.inv.Retired = withoutRetired(r.inv.Retired, x.Consumer)
|
||||
return x.SizeBytes, nil
|
||||
}
|
||||
|
||||
func listHas(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func without(list []string, s string) []string {
|
||||
var out []string
|
||||
for _, x := range list {
|
||||
if x != s {
|
||||
out = append(out, x)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func withoutRetired(list []Retired, s string) []Retired {
|
||||
var out []Retired
|
||||
for _, x := range list {
|
||||
if x.Consumer != s {
|
||||
out = append(out, x)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// holding gives the provider n consumers c1…cn and applies them.
|
||||
func holding(w *world, n int) []map[string]any {
|
||||
var given []map[string]any
|
||||
for i := 1; i <= n; i++ {
|
||||
given = append(given, map[string]any{"as": fmt.Sprintf("c%d", i), "node": "anchor"})
|
||||
}
|
||||
w.give(given...)
|
||||
w.h.Reconcile(ctx)
|
||||
return given
|
||||
}
|
||||
|
||||
// pass is one reconcile five seconds after the last.
|
||||
func (w *world) pass() {
|
||||
w.now = w.now.Add(5 * time.Second)
|
||||
w.h.Reconcile(ctx)
|
||||
}
|
||||
|
||||
func retirements(said []announced) []string {
|
||||
var out []string
|
||||
for _, a := range said {
|
||||
if a.event == EventRetirement {
|
||||
out = append(out, a.body["change"].(string))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func noneWaits(a announced) bool {
|
||||
c, _ := a.body["consumers"].([]map[string]any)
|
||||
return a.body["change"] == ChangeSettled && len(c) == 0
|
||||
}
|
||||
|
||||
func lastRetirement(t *testing.T, said []announced) map[string]any {
|
||||
t.Helper()
|
||||
for i := len(said) - 1; i >= 0; i-- {
|
||||
if said[i].event == EventRetirement {
|
||||
return said[i].body
|
||||
}
|
||||
}
|
||||
t.Fatal("nothing about retirement was announced")
|
||||
return nil
|
||||
}
|
||||
|
||||
func namesOf(body map[string]any) string {
|
||||
var out []string
|
||||
for _, c := range body["consumers"].([]map[string]any) {
|
||||
out = append(out, c["consumer"].(string))
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
// settle passes every five seconds until the same answer has held for StableFor, and one pass more.
|
||||
func (w *world) settle() { w.passes(StableFor + 5*time.Second) }
|
||||
|
||||
func TestATransientEmptyListForFourPassesRetiresNothing(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 1)
|
||||
w.give()
|
||||
for i := 0; i < 4; i++ {
|
||||
w.pass()
|
||||
}
|
||||
w.give(given...)
|
||||
w.pass()
|
||||
if len(w.a.removed) != 0 || len(retirements(*said)) != 0 {
|
||||
t.Fatalf("four passes retired %v and said %v", w.a.removed, retirements(*said))
|
||||
}
|
||||
}
|
||||
|
||||
// Five identical passes are twenty-five seconds — shorter than a controller restart. Both must hold:
|
||||
// five passes AND ten minutes of the same answer (novox/hq ADR 0230).
|
||||
func TestFivePassesInTwentyFiveSecondsRetireNothingTenMinutesDo(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
holding(w, 1)
|
||||
w.give()
|
||||
for i := 0; i < 5; i++ {
|
||||
w.pass()
|
||||
}
|
||||
if len(w.a.removed) != 0 || len(retirements(*said)) != 0 {
|
||||
t.Fatalf("five passes in 25 s retired %v", w.a.removed)
|
||||
}
|
||||
w.passes(StableFor - 30*time.Second)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("retired before the set held %s: %v", StableFor, w.a.removed)
|
||||
}
|
||||
w.passes(time.Minute)
|
||||
if strings.Join(w.a.removed, ",") != "c1" {
|
||||
t.Fatalf("the same set held %s and was not retired: %v", StableFor, w.a.removed)
|
||||
}
|
||||
// Ten minutes in one slow pass are not five passes.
|
||||
w2 := newWorld(t)
|
||||
holding(w2, 1)
|
||||
w2.give()
|
||||
w2.pass()
|
||||
w2.now = w2.now.Add(StableFor)
|
||||
w2.pass()
|
||||
if len(w2.a.removed) != 0 {
|
||||
t.Fatalf("two passes ten minutes apart retired %v", w2.a.removed)
|
||||
}
|
||||
w2.passes(15 * time.Second)
|
||||
if len(w2.a.removed) != 1 {
|
||||
t.Fatalf("five passes over ten minutes did not retire: %v", w2.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAStableSetIsRetiredAndAskedAgainReEnables(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 3)
|
||||
w.give(given[:2]...)
|
||||
w.settle()
|
||||
if strings.Join(w.a.removed, ",") != "c3" {
|
||||
t.Fatalf("retired %v", w.a.removed)
|
||||
}
|
||||
body := lastRetirement(t, *said)
|
||||
if body["change"] != ChangeRetired || namesOf(body) != "c3" || body["held"] != 3 || body["provider-node"] != "anchor" ||
|
||||
!strings.Contains(body["why"].(string), "consecutive passes over 10m") {
|
||||
t.Fatalf("%v", body)
|
||||
}
|
||||
if len(w.a.inv.Retired) != 1 || w.a.inv.Retired[0].Consumer != "c3" {
|
||||
t.Fatalf("the backend does not hold it retired: %+v", w.a.inv)
|
||||
}
|
||||
// Asked for again: the ordinary create, on the first pass, and said.
|
||||
created := len(w.a.created)
|
||||
w.give(given...)
|
||||
w.pass()
|
||||
if len(w.a.created) != created+1 || w.a.created[created].As != "c3" {
|
||||
t.Fatalf("not created again: %v", w.a.created)
|
||||
}
|
||||
if body := lastRetirement(t, *said); body["change"] != ChangeReenabled || namesOf(body) != "c3" {
|
||||
t.Fatalf("%v", body)
|
||||
}
|
||||
if len(w.a.inv.Retired) != 0 {
|
||||
t.Fatalf("still marked retired: %+v", w.a.inv.Retired)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnUnreadablePassStartsTheCountAgain(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
holding(w, 1)
|
||||
w.give()
|
||||
w.passes(StableFor - time.Minute)
|
||||
os.WriteFile(w.receives, []byte("{"), 0o600)
|
||||
w.pass()
|
||||
w.give()
|
||||
w.passes(StableFor - time.Minute)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("an unreadable pass counted: %v", w.a.removed)
|
||||
}
|
||||
w.passes(2 * time.Minute)
|
||||
if len(w.a.removed) != 1 {
|
||||
t.Fatalf("not retired after ten minutes of readable passes: %v", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestADifferentSetStartsTheCountAgain(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
given := holding(w, 5)
|
||||
w.give(given[:4]...)
|
||||
w.passes(StableFor - time.Minute)
|
||||
w.give(given[:3]...)
|
||||
w.passes(StableFor - time.Minute)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("a changed set kept its count: %v", w.a.removed)
|
||||
}
|
||||
w.passes(2 * time.Minute)
|
||||
if strings.Join(w.a.removed, ",") != "c4,c5" {
|
||||
t.Fatalf("%v", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdditionsAndChangesAreNeverDelayed(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
holding(w, 1)
|
||||
w.give(map[string]any{"as": "c1", "node": "anchor"}, map[string]any{"as": "c2"})
|
||||
w.pass()
|
||||
if len(w.a.created) != 2 || w.a.created[1].As != "c2" {
|
||||
t.Fatalf("an addition waited: %v", w.a.created)
|
||||
}
|
||||
w.give(map[string]any{"as": "c1", "node": "anchor", "values": map[string]any{"name": "rotated"}}, map[string]any{"as": "c2"})
|
||||
w.pass()
|
||||
if len(w.a.created) != 3 || w.a.created[2].As != "c1" {
|
||||
t.Fatalf("a change waited: %v", w.a.created)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTooManyWaitsForAPersonAndApproveRetiresExactlyThatSet(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
holding(w, 4)
|
||||
w.give()
|
||||
w.passes(15 * time.Minute)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("four of four were retired without a person: %v", w.a.removed)
|
||||
}
|
||||
if got := strings.Join(retirements(*said), ","); got != ChangeWaiting {
|
||||
t.Fatalf("said %q, want one waiting", got)
|
||||
}
|
||||
body := lastRetirement(t, *said)
|
||||
if namesOf(body) != "c1,c2,c3,c4" || body["held"] != 4 || body["bound"] == "" {
|
||||
t.Fatalf("%v", body)
|
||||
}
|
||||
if !strings.Contains(strings.Join(w.said, "\n"), "RETIREMENT WAITS FOR A PERSON") {
|
||||
t.Fatal("the wait was not said")
|
||||
}
|
||||
// Said again every quarter of an hour while it waits.
|
||||
w.passes(11 * time.Minute)
|
||||
if got := strings.Join(retirements(*said), ","); got != "waiting,waiting" {
|
||||
t.Fatalf("%q", got)
|
||||
}
|
||||
|
||||
if _, err := w.h.Approve(ctx, []string{"c1", "c2"}, "tidy", "operator", "mesh-controller"); err == nil {
|
||||
t.Fatal("approved a set that is not the one waiting")
|
||||
}
|
||||
if _, err := w.h.Approve(ctx, []string{"c4", "c3", "c2", "c1"}, "", "operator", ""); err == nil {
|
||||
t.Fatal("approved without a why")
|
||||
}
|
||||
done, err := w.h.Approve(ctx, []string{"c4", "c3", "c2", "c1"}, "the old machine is gone", "operator", "mesh-controller")
|
||||
if err != nil || strings.Join(done, ",") != "c1,c2,c3,c4" || strings.Join(w.a.removed, ",") != "c1,c2,c3,c4" {
|
||||
t.Fatal(done, err, w.a.removed)
|
||||
}
|
||||
changes := retirements(*said)
|
||||
if strings.Join(changes[len(changes)-2:], ",") != "approved,retired" {
|
||||
t.Fatalf("%v", changes)
|
||||
}
|
||||
if b := lastRetirement(t, *said); b["by"] != "operator" || b["via"] != "mesh-controller" ||
|
||||
!strings.Contains(b["why"].(string), "the old machine is gone") {
|
||||
t.Fatalf("%v", b)
|
||||
}
|
||||
// Nothing more waits.
|
||||
w.passes(time.Minute)
|
||||
if r, _ := w.h.Retirement(ctx); r["waiting"] != nil || len(r["retired"].([]map[string]any)) != 4 {
|
||||
t.Fatalf("%v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectedIsKeptAndNotAskedAgainUntilTheSetChanges(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 4)
|
||||
w.give()
|
||||
w.settle()
|
||||
if _, err := w.h.Reject([]string{"c1"}, "no", "operator", ""); err == nil {
|
||||
t.Fatal("rejected a set that is not the one waiting")
|
||||
}
|
||||
kept, err := w.h.Reject([]string{"c1", "c2", "c3", "c4"}, "a person is moving them", "operator", "mesh-controller")
|
||||
if err != nil || len(kept) != 4 {
|
||||
t.Fatal(kept, err)
|
||||
}
|
||||
w.passes(time.Hour)
|
||||
if len(w.a.removed) != 0 {
|
||||
t.Fatalf("a rejected set was retired: %v", w.a.removed)
|
||||
}
|
||||
if got := strings.Join(retirements(*said), ","); got != "waiting,rejected" {
|
||||
t.Fatalf("asked again after a rejection: %q", got)
|
||||
}
|
||||
r, _ := w.h.Retirement(ctx)
|
||||
if r["rejected"] == nil || r["waiting"] != nil {
|
||||
t.Fatalf("%v", r)
|
||||
}
|
||||
// One of them asked for again: a different answer, so the rejection is settled and counting
|
||||
// starts over — three of four is over the bound again, and waits again.
|
||||
w.give(given[0])
|
||||
w.pass()
|
||||
if got := strings.Join(retirements(*said), ","); got != "waiting,rejected,settled" {
|
||||
t.Fatalf("%q", got)
|
||||
}
|
||||
w.settle()
|
||||
if got := strings.Join(retirements(*said), ","); got != "waiting,rejected,settled,waiting" {
|
||||
t.Fatalf("%q", got)
|
||||
}
|
||||
// A rejected set can still be approved later.
|
||||
w2, _ := standingWorld(t)
|
||||
holding(w2, 4)
|
||||
w2.give()
|
||||
w2.settle()
|
||||
w2.h.Reject([]string{"c1", "c2", "c3", "c4"}, "wait", "operator", "")
|
||||
if done, err := w2.h.Approve(ctx, []string{"c1", "c2", "c3", "c4"}, "now", "operator", ""); err != nil || len(done) != 4 {
|
||||
t.Fatal(done, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWaitingSetAskedForAgainSettles(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 4)
|
||||
w.give()
|
||||
w.settle()
|
||||
w.give(given...)
|
||||
w.pass()
|
||||
if got := strings.Join(retirements(*said), ","); got != "waiting,settled" || len(w.a.removed) != 0 {
|
||||
t.Fatalf("%q %v", got, w.a.removed)
|
||||
}
|
||||
if _, err := w.h.Approve(ctx, []string{"c1", "c2", "c3", "c4"}, "late", "operator", ""); err == nil {
|
||||
t.Fatal("approved a set that no longer waits")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteRemovesOnlyThatRetiredConsumer(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 5)
|
||||
w.give(given[:3]...)
|
||||
w.settle()
|
||||
if strings.Join(w.a.removed, ",") != "c4,c5" {
|
||||
t.Fatalf("%v", w.a.removed)
|
||||
}
|
||||
if _, err := w.h.DeleteRetired(ctx, "c1", "tidy", "operator", ""); err == nil {
|
||||
t.Fatal("deleted an active consumer")
|
||||
}
|
||||
if _, err := w.h.DeleteRetired(ctx, "c5", "", "operator", ""); err == nil {
|
||||
t.Fatal("deleted without a why")
|
||||
}
|
||||
if _, err := w.h.DeleteRetired(ctx, "nobody", "tidy", "operator", ""); err == nil {
|
||||
t.Fatal("deleted something not retired")
|
||||
}
|
||||
freed, err := w.h.DeleteRetired(ctx, "c5", "the experiment is over", "operator", "mesh-controller")
|
||||
if err != nil || freed != 42 || strings.Join(w.a.deleted, ",") != "c5" {
|
||||
t.Fatal(freed, err, w.a.deleted)
|
||||
}
|
||||
if b := lastRetirement(t, *said); b["change"] != ChangeDeleted || namesOf(b) != "c5" || b["freed_bytes"] != int64(42) {
|
||||
t.Fatalf("%v", b)
|
||||
}
|
||||
r, _ := w.h.Retirement(ctx)
|
||||
if left := r["retired"].([]map[string]any); len(left) != 1 || left[0]["consumer"] != "c4" {
|
||||
t.Fatalf("%v", r)
|
||||
}
|
||||
// Retired and asked for again before anybody deleted it: refused, it is the mesh's again.
|
||||
w.give(given[:4]...)
|
||||
w.pass()
|
||||
if _, err := w.h.DeleteRetired(ctx, "c4", "tidy", "operator", ""); err == nil {
|
||||
t.Fatal("deleted a consumer the mesh asks for")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheBound(t *testing.T) {
|
||||
h := &Harness{}
|
||||
h.init()
|
||||
for _, c := range []struct{ n, held int }{{1, 1}, {1, 2}, {1, 3}, {3, 7}, {3, 6}, {2, 5}} {
|
||||
if h.overTheBound(c.n, c.held) {
|
||||
t.Errorf("%d of %d waits for a person", c.n, c.held)
|
||||
}
|
||||
}
|
||||
for _, c := range []struct{ n, held int }{{2, 2}, {2, 3}, {4, 7}, {4, 10}, {7, 7}} {
|
||||
if !h.overTheBound(c.n, c.held) {
|
||||
t.Errorf("%d of %d is retired without a person", c.n, c.held)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestARestartRetiresWhatTheBackendHoldsUnaskedAndAdoptsWhatItFindsDisabled(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.a.inv = Inventory{Active: []string{"kept", "orphan"}, Retired: []Retired{
|
||||
{Consumer: "locked-before", SizeBytes: 7, Kind: KindConsumer},
|
||||
{Consumer: "old_deleted_20261005", RetiredAt: time.Date(2026, 10, 5, 0, 0, 0, 0, time.UTC), Kind: "set-aside-database"},
|
||||
}}
|
||||
w.give(map[string]any{"as": "kept"})
|
||||
w.h.Reconcile(ctx)
|
||||
if b := lastRetirement(t, *said); b["change"] != ChangeAdopted || namesOf(b) != "locked-before" {
|
||||
t.Fatalf("%v", b)
|
||||
}
|
||||
if strings.Join(w.a.removed, ",") != "locked-before" {
|
||||
t.Fatalf("adopting did not mark it: %v", w.a.removed)
|
||||
}
|
||||
w.settle()
|
||||
if strings.Join(w.a.removed, ",") != "locked-before,orphan" {
|
||||
t.Fatalf("an orphan the backend holds was not retired: %v", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestABackendThatCannotBeListedIsSaidAndAskedAgain(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
w.a.invErr = errors.New("connection refused")
|
||||
holding(w, 1)
|
||||
if !strings.Contains(strings.Join(w.said, "\n"), "cannot list what the backend holds") {
|
||||
t.Fatal(w.said)
|
||||
}
|
||||
w.a.invErr = nil
|
||||
w.a.inv = Inventory{Active: []string{"c1", "orphan"}}
|
||||
w.pass()
|
||||
w.settle()
|
||||
if strings.Join(w.a.removed, ",") != "orphan" {
|
||||
t.Fatalf("%v", w.a.removed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheToolsAnswer(t *testing.T) {
|
||||
w, _ := standingWorld(t)
|
||||
holding(w, 4)
|
||||
w.give()
|
||||
w.settle()
|
||||
tools := map[string]func(map[string]any) (any, error){}
|
||||
for _, tool := range RetirementTools(w.h) {
|
||||
tools[tool.Name] = tool.Run
|
||||
}
|
||||
if len(tools) != 4 {
|
||||
t.Fatalf("%d tools", len(tools))
|
||||
}
|
||||
got, err := tools["provisioner_retirement"](nil)
|
||||
if err != nil || got.(map[string]any)["waiting"] == nil {
|
||||
t.Fatal(got, err)
|
||||
}
|
||||
set := []any{"c1", "c2", "c3", "c4"}
|
||||
if _, err := tools["provisioner_retire_approve"](map[string]any{"consumers": set}); err == nil {
|
||||
t.Fatal("approved without a why")
|
||||
}
|
||||
if _, err := tools["provisioner_retire_approve"](map[string]any{"consumers": set, "why": "gone", "by": "operator"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := tools["provisioner_delete"](map[string]any{"consumer": "c1", "why": "gone"}); err == nil {
|
||||
t.Fatal("deleted without confirm")
|
||||
}
|
||||
if _, err := tools["provisioner_delete"](map[string]any{"consumer": "c1", "confirm": "c1", "why": "gone"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Join(w.a.deleted, ",") != "c1" {
|
||||
t.Fatal(w.a.deleted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAFailingConsumerRetiredIsSaidRecovered(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
given := holding(w, 2)
|
||||
w.a.held = false
|
||||
w.a.failing = errors.New("boom")
|
||||
w.passes(7 * time.Minute)
|
||||
w.give(given[0])
|
||||
w.settle()
|
||||
recovered := false
|
||||
for _, a := range *said {
|
||||
if a.event == EventRecovered && a.body["consumer"] == "c2" && a.body["why"] == "retired" {
|
||||
recovered = true
|
||||
}
|
||||
}
|
||||
if !recovered {
|
||||
t.Fatalf("%v", *said)
|
||||
}
|
||||
}
|
||||
|
||||
// everyWord is a world that keeps every word the provider announces.
|
||||
func everyWord(t *testing.T) (*world, *[]announced) {
|
||||
w := newWorld(t)
|
||||
var said []announced
|
||||
w.h.Announce = func(e string, b map[string]any) { said = append(said, announced{e, b}) }
|
||||
w.h.Node = "anchor"
|
||||
return w, &said
|
||||
}
|
||||
|
||||
// A restarted provider that finds every consumer it holds asked for says once that nothing waits, so the controller
|
||||
// clears a waiting condition an earlier process raised and this one never knew (issue 937); it says it no more.
|
||||
func TestARestartedProviderSaysOnceThatNothingWaits(t *testing.T) {
|
||||
w, said := everyWord(t)
|
||||
holding(w, 2)
|
||||
w.pass()
|
||||
w.pass()
|
||||
var words []announced
|
||||
for _, a := range *said {
|
||||
if a.event == EventRetirement {
|
||||
words = append(words, a)
|
||||
}
|
||||
}
|
||||
if len(words) != 1 || !noneWaits(words[0]) || words[0].body["provider-node"] != w.h.Node {
|
||||
t.Fatalf("said %+v, want one settled with no set", words)
|
||||
}
|
||||
}
|
||||
|
||||
// A provider whose own set waits does not say that nothing waits.
|
||||
func TestAProviderThatWaitsDoesNotSayNothingWaits(t *testing.T) {
|
||||
w, said := everyWord(t)
|
||||
w.a.inv.Active = []string{"c1", "c2", "c3", "c4", "c5"}
|
||||
w.give(map[string]any{"as": "c1", "node": "anchor"})
|
||||
w.settle()
|
||||
for _, a := range *said {
|
||||
if a.event == EventRetirement && noneWaits(a) {
|
||||
t.Fatalf("said nothing waits while %v waited", w.h.r.waiting)
|
||||
}
|
||||
}
|
||||
if w.h.r.waiting == nil {
|
||||
t.Fatal("nothing waited: the test does not hold the case")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
package harness
|
||||
|
||||
// A provider that keeps failing a consumer says so on the bus (novox/hq ADR 0224): not on the first
|
||||
// failure, which may be a restart; after FailingAfter of failures with no success between; again
|
||||
// every SayAgainEvery while it lasts; and recovered on the first success, or when the consumer goes.
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type announced struct {
|
||||
event string
|
||||
body map[string]any
|
||||
}
|
||||
|
||||
func standingWorld(t *testing.T) (*world, *[]announced) {
|
||||
w := newWorld(t)
|
||||
var said []announced
|
||||
w.h.Announce = func(e string, b map[string]any) {
|
||||
// The first success since start, and the first word that nothing waits (issue 937), are their own
|
||||
// tests'; every other test reads past them.
|
||||
if c, _ := b["consumers"].([]map[string]any); b["why"] != "first-success" &&
|
||||
!(e == EventRetirement && b["change"] == ChangeSettled && len(c) == 0) {
|
||||
said = append(said, announced{e, b})
|
||||
}
|
||||
}
|
||||
w.h.Node = "anchor"
|
||||
return w, &said
|
||||
}
|
||||
|
||||
// passes reconciles every five seconds for d, as Run would.
|
||||
func (w *world) passes(d time.Duration) {
|
||||
for end := w.now.Add(d); w.now.Before(end); w.now = w.now.Add(5 * time.Second) {
|
||||
w.h.Reconcile(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
func events(said []announced) string {
|
||||
var out []string
|
||||
for _, a := range said {
|
||||
out = append(out, a.event)
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
func TestAConsumerFailedForMinutesIsAnnouncedNamingItAndTheClass(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.a.failing = errors.New(`token request failed: 401 {"error":"invalid_grant","error_description":"Invalid user credentials"}`)
|
||||
w.give(map[string]any{"as": "mesh_home_grafana", "node": "home-server"})
|
||||
|
||||
w.passes(4 * time.Minute)
|
||||
if len(*said) != 0 {
|
||||
t.Fatalf("announced before FailingAfter: %v", events(*said))
|
||||
}
|
||||
w.passes(2 * time.Minute)
|
||||
if events(*said) != EventFailing {
|
||||
t.Fatalf("want one %s, got %q", EventFailing, events(*said))
|
||||
}
|
||||
b := (*said)[0].body
|
||||
if b["consumer"] != "mesh_home_grafana" || b["node"] != "home-server" || b["class"] != ClassCredentials ||
|
||||
b["provider"] != "postgres-database" || b["provider-node"] != "anchor" || b["attempts"].(int) < 60 {
|
||||
t.Fatalf("%v", b)
|
||||
}
|
||||
|
||||
// Said again while it lasts, not every pass.
|
||||
w.passes(14 * time.Minute)
|
||||
if events(*said) != EventFailing {
|
||||
t.Fatalf("repeated too soon: %q", events(*said))
|
||||
}
|
||||
w.passes(2 * time.Minute)
|
||||
if events(*said) != EventFailing+","+EventFailing {
|
||||
t.Fatalf("not repeated: %q", events(*said))
|
||||
}
|
||||
|
||||
// The first success takes it back.
|
||||
w.a.failing = nil
|
||||
w.passes(5 * time.Second)
|
||||
if events(*said) != EventFailing+","+EventFailing+","+EventRecovered {
|
||||
t.Fatalf("no recovery: %q", events(*said))
|
||||
}
|
||||
if (*said)[2].body["consumer"] != "mesh_home_grafana" {
|
||||
t.Fatal((*said)[2].body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOneSuccessBetweenFailuresStartsTheRunAgain(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.a.failing = errors.New("connection refused")
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.passes(4 * time.Minute)
|
||||
w.a.failing = nil
|
||||
w.passes(5 * time.Second)
|
||||
w.give(map[string]any{"as": "a", "values": map[string]any{"name": "changed"}})
|
||||
w.a.failing = errors.New("connection refused")
|
||||
w.passes(4 * time.Minute)
|
||||
if len(*said) != 0 {
|
||||
t.Fatalf("two runs of four minutes are not one of eight: %q", events(*said))
|
||||
}
|
||||
}
|
||||
|
||||
// The check that failed for a day on 2026-10-05: clients already made, every minute's check refused
|
||||
// at the token. A check that cannot be asked is a failure too.
|
||||
func TestACheckThatKeepsFailingIsAFailureToo(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.h.Reconcile(ctx)
|
||||
w.a.holdsErr = errors.New("401 invalid_grant")
|
||||
w.passes(7 * time.Minute)
|
||||
if events(*said) != EventFailing || (*said)[0].body["class"] != ClassCredentials {
|
||||
t.Fatalf("%q %v", events(*said), *said)
|
||||
}
|
||||
w.a.holdsErr = nil
|
||||
w.passes(time.Minute + 5*time.Second)
|
||||
if events(*said) != EventFailing+","+EventRecovered {
|
||||
t.Fatalf("%q", events(*said))
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnUnreadableSecretIsAnnouncedAsSuch(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.give(map[string]any{"as": "a"})
|
||||
os.Remove(w.dir + "/a.secret")
|
||||
w.passes(6 * time.Minute)
|
||||
if events(*said) != EventFailing || (*said)[0].body["class"] != ClassSecret {
|
||||
t.Fatalf("%q %v", events(*said), *said)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAConsumerNoLongerAskedForIsNoLongerFailing(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.a.failing = errors.New("boom")
|
||||
w.give(map[string]any{"as": "a"}, map[string]any{"as": "b"})
|
||||
w.passes(6 * time.Minute)
|
||||
if events(*said) != EventFailing+","+EventFailing {
|
||||
t.Fatalf("%q", events(*said))
|
||||
}
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.passes(5 * time.Second)
|
||||
last := (*said)[len(*said)-1]
|
||||
if last.event != EventRecovered || last.body["consumer"] != "b" || last.body["why"] != "no longer asked for" {
|
||||
t.Fatalf("%v", *said)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnErrorIsClassedByItsWords(t *testing.T) {
|
||||
for text, want := range map[string]string{
|
||||
`Keycloak token request failed: 401 {"error":"invalid_grant"}`: ClassCredentials,
|
||||
`FATAL: password authentication failed for user "postgres"`: ClassCredentials,
|
||||
`dial tcp 127.0.0.1:5432: connect: connection refused`: ClassUnreachable,
|
||||
`context deadline exceeded`: ClassUnreachable,
|
||||
`extension "nope" is not available`: ClassRefused,
|
||||
} {
|
||||
if got := ClassOf(text); got != want {
|
||||
t.Errorf("%s: %s, want %s", text, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAdapterThatClassesItsOwnErrorsIsBelieved(t *testing.T) {
|
||||
w, said := standingWorld(t)
|
||||
w.h.Adapter = classing{w.a}
|
||||
w.a.failing = errors.New("anything")
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.passes(6 * time.Minute)
|
||||
if (*said)[0].body["class"] != "its-own" {
|
||||
t.Fatal((*said)[0].body)
|
||||
}
|
||||
}
|
||||
|
||||
type classing struct{ *recorder }
|
||||
|
||||
func (classing) Class(error) string { return "its-own" }
|
||||
|
||||
// A provider restarted after announcing a failure has forgotten it; its first success for each
|
||||
// consumer is announced, so the controller clears what it kept rather than naming it for ever.
|
||||
func TestTheFirstSuccessSinceStartIsAnnouncedOnce(t *testing.T) {
|
||||
w := newWorld(t)
|
||||
var said []announced
|
||||
w.h.Announce = func(e string, b map[string]any) {
|
||||
if e != EventRetirement {
|
||||
said = append(said, announced{e, b})
|
||||
}
|
||||
}
|
||||
w.give(map[string]any{"as": "a"})
|
||||
w.passes(3 * time.Minute)
|
||||
if events(said) != EventRecovered || said[0].body["why"] != "first-success" || said[0].body["consumer"] != "a" {
|
||||
t.Fatalf("%v", said)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,746 @@
|
||||
// Package provider is a providing module's half of a binding: it answers the provider's consumers itself, over the
|
||||
// bus (novox/hq ADR 0349 clause 12, issue 678): the provider module, not the controller, makes each bound
|
||||
// consumer's entry.
|
||||
//
|
||||
// **Who binds the provider.** Each node's provisioner client records the needs of the modules handed to it from
|
||||
// their plans, and the controller names in each plan, per required provision and local name, the provider that
|
||||
// resolution chose: `provisioning.provisions` in `<node>.<module>` of the assignments mirror (issue 760). A recorded
|
||||
// need names no provider (issue 677's note 7), so the plan's entry is where a bind is seen, as the vault sees it
|
||||
// (mesh-vault pair.go). A consumer binds this provider when an entry names this machine and this module, and
|
||||
// unbinds when the entry, or the assignment, goes. A consumer still filled the old way has no plan and is not seen
|
||||
// here; the controller answers it as before (ADR 0349 clause 17).
|
||||
//
|
||||
// **One entry per consumer, in the provider's own bucket.** For each bind the helper composes the consumer's
|
||||
// served values from the provider's offer (its config already composed into it, ADR 0202 §7) and the consumer's
|
||||
// identity (`${consumer:as}`, `${consumer:as:dns}`, ADR 0202 §1), and writes one entry, `<node>.<module>.
|
||||
// <provision>[.<local>]` of `<provider module>_bound`: the one statement both ends read. It never carries a
|
||||
// secret. It names the credential the vault keeps for the pair, which the consumer's client asks the vault for.
|
||||
//
|
||||
// **The credential is the vault's** (ADR 0349 clause 8). The helper mints nothing and reads nothing of the
|
||||
// provider's own secrets: it reads the value from the vault and hands it to the provider's adapter, which makes the
|
||||
// login its software keeps. An offer whose software holds one credential (ADR 0158) is the vault's one value per
|
||||
// provider and provision, `provision/<provider node>/<provision>`, with the provider's one user name; an offer with
|
||||
// a login per consumer (ADR 0048, ADR 0158 §11, the default) is each consumer's own value,
|
||||
// `pair/<provider node>/<provision>/<node>/<module>[/<local>]`, with the consumer's identity as its login (issue
|
||||
// 677's notes 12 and 13). A credential the vault does not hold yet is waited for: nothing is made and no entry is
|
||||
// written, so a consumer is never told of a login its provider has not made. A provider in the tool runner never asks
|
||||
// the vault itself: its Vault is ConsumersFile, the file its machine's provisioner client fills (ADR 0373).
|
||||
//
|
||||
// **A multi-holder consumer gets no derived value** (ADR 0202 §10 as ADR 0349 clause 12 changes it): a consumer
|
||||
// bound under a local name keeps several holders of the provision, each with its own login, while what the
|
||||
// provider derives names one; its entry refuses it, naming both ends, and its client reports it.
|
||||
//
|
||||
// The helper speaks to the bus and the vault through the two interfaces below, so a provider wires it to its
|
||||
// runtime and a test to a stand-in. Standard library only.
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// AssignmentsBucket is the controller's assignments mirror, `<node>.<module>`, whose value carries the plan of a
|
||||
// module handed to the provisioner.
|
||||
const AssignmentsBucket = "mesh-controller_assignments"
|
||||
|
||||
// BucketOf is a providing module's own bucket of entries, one per consumer: `<module>_bound`, the module's state
|
||||
// `bound`.
|
||||
func BucketOf(module string) string { return module + "_bound" }
|
||||
|
||||
// Bus is what the helper needs of the bus: a key-value watch and the provider's writes to its own bucket.
|
||||
type Bus interface {
|
||||
// Watch hands handle every current key and value of bucket, then each change as it comes, a removed key with a
|
||||
// nil value; it returns once the current values are handed, and the watch lasts until ctx ends.
|
||||
Watch(ctx context.Context, bucket string, handle func(key string, value []byte)) error
|
||||
Put(ctx context.Context, bucket, key string, value []byte) error
|
||||
Delete(ctx context.Context, bucket, key string) error
|
||||
}
|
||||
|
||||
// ConditionalBus is a Bus that can also delete a key only while it still holds what the caller read. The helper uses
|
||||
// it when the Bus offers it, so a pass that lets a consumer go never removes an entry another instance of the
|
||||
// provider wrote after the pass read it; a Bus without it deletes unconditionally. DeleteIf returns an error that
|
||||
// is stdio.ErrChanged when the key holds something else now.
|
||||
type ConditionalBus interface {
|
||||
Bus
|
||||
DeleteIf(ctx context.Context, bucket, key string, held []byte) error
|
||||
}
|
||||
|
||||
// Vault is how the provider reads a credential the vault keeps, by its name, opened.
|
||||
type Vault interface {
|
||||
Value(ctx context.Context, name string) ([]byte, error)
|
||||
}
|
||||
|
||||
// CredentialForm is which credential an offer's consumers present.
|
||||
type CredentialForm int
|
||||
|
||||
const (
|
||||
// PerConsumer is a login of each consumer's own (ADR 0048, ADR 0158 §11): the default.
|
||||
PerConsumer CredentialForm = iota
|
||||
// Shared is the one credential the provider's software holds, shared with every consumer (ADR 0158).
|
||||
Shared
|
||||
// NoCredential is an offer whose consumers present none.
|
||||
NoCredential
|
||||
)
|
||||
|
||||
// Offer is what the provider serves one provision with.
|
||||
type Offer struct {
|
||||
// Serves are the values served to each consumer, the provider's config already composed into them; a string
|
||||
// may name the consumer's identity, `${consumer:as}` or `${consumer:as:dns}`.
|
||||
Serves map[string]any
|
||||
// Credential is the form of the credential its consumers present. A consumer's plan entry that names a
|
||||
// credential form of its own wins over it (the plan's `credential-form`).
|
||||
Credential CredentialForm
|
||||
// User is the provider's one user name, for a Shared credential.
|
||||
User string
|
||||
}
|
||||
|
||||
// Entry is one consumer's entry in the provider's bucket: the one statement both ends read. Never a secret.
|
||||
type Entry struct {
|
||||
Provider string `json:"provider"`
|
||||
ProviderModule string `json:"provider-module"`
|
||||
Provision string `json:"provision"`
|
||||
Consumer string `json:"consumer"`
|
||||
Module string `json:"module"`
|
||||
Local string `json:"local,omitempty"`
|
||||
// As is the consumer's identity, the mesh's (ADR 0202 §1).
|
||||
As string `json:"as,omitempty"`
|
||||
// User is the login the consumer presents: its identity, or the provider's one user name for a shared credential.
|
||||
User string `json:"user,omitempty"`
|
||||
// Values are the served values, composed for this consumer: what `${bound:<provision>:<key>}` reads.
|
||||
Values map[string]any `json:"values,omitempty"`
|
||||
// VaultName is the name under which the vault keeps this consumer's credential, which the consumer's client asks
|
||||
// the vault for: a name, never the credential. Its key is not "credential" because the runtime's state guard
|
||||
// refuses any key ending in that word, whatever the value (issue 945).
|
||||
VaultName string `json:"vault-name,omitempty"`
|
||||
// Refused, when set, is why the consumer is not served, naming both ends.
|
||||
Refused string `json:"refused,omitempty"`
|
||||
// Contributes is what the consumer's module tells its provider about this provision (a database's name, its
|
||||
// extensions), from its plan's entry (issue 776). Handed to Make and Gone; never written to the bucket.
|
||||
Contributes map[string]any `json:"-"`
|
||||
// At and From are where the consumer's plan says the provider is and which machine it is (issue 788): the values
|
||||
// today's binding file gives that consumer. Written into Values as `at` and `from`; never alone to the bucket.
|
||||
At, From string `json:"-"`
|
||||
// Names are the names the controller composed for this consumer on this provision (issue 792): its own name, its
|
||||
// internal name, and one of each per local name it contributes under, keyed as its binding file names them.
|
||||
// Written into Values over anything the provider serves under the key; never alone to the bucket.
|
||||
Names map[string]string `json:"-"`
|
||||
// CredentialForm and Name are from the consumer's plan entry (issue 825): the form of the credential the
|
||||
// controller chose for this bind (`shared`, `pair`, `issued` or `given`) and the holder's name. When set, they
|
||||
// win over the offer's form. Never written to the bucket.
|
||||
CredentialForm, Name string `json:"-"`
|
||||
}
|
||||
|
||||
// Key is the entry's key in the provider's bucket: `<node>.<module>.<provision>`, and `.<local>` for a bind under a
|
||||
// local name.
|
||||
func (e Entry) Key() string {
|
||||
k := e.Consumer + "." + e.Module + "." + e.Provision
|
||||
if e.Local != "" {
|
||||
k += "." + e.Local
|
||||
}
|
||||
return k
|
||||
}
|
||||
|
||||
// Consumer is what the provider's adapter makes: the consumer's entry and the credential the vault keeps for it.
|
||||
type Consumer struct {
|
||||
Entry Entry
|
||||
Credential []byte
|
||||
}
|
||||
|
||||
// Options are the provider's.
|
||||
type Options struct {
|
||||
// Node and Module are this provider: the machine it runs on and the module serving.
|
||||
Node, Module string
|
||||
// Offers are the provisions it serves, by name.
|
||||
Offers map[string]Offer
|
||||
Bus Bus
|
||||
Vault Vault
|
||||
// Make brings a consumer into being in the provider's software under the vault's credential. Asked on every
|
||||
// pass for every consumer answered, so it must leave one that is already as asked alone; a rotated credential
|
||||
// reaches the software this way.
|
||||
Make func(ctx context.Context, c Consumer) error
|
||||
// Gone, when set, hears that a consumer no longer binds the provider, before its entry is removed; an error
|
||||
// keeps the entry, and it is heard again on the next pass. Retiring what was kept for it is the provider's
|
||||
// (ADR 0230).
|
||||
Gone func(ctx context.Context, e Entry) error
|
||||
// Identity is the consumer's identity for an entry whose plan names none; ConsumerIdentity of the node and the
|
||||
// module when not set.
|
||||
Identity func(node, module string) string
|
||||
// Every is how often a pass runs without a change watched; a minute when not set.
|
||||
Every time.Duration
|
||||
// Logf says what a pass did that differs from the last; the standard log when not set.
|
||||
Logf func(format string, args ...any)
|
||||
}
|
||||
|
||||
// Provider answers its consumers.
|
||||
type Provider struct {
|
||||
opts Options
|
||||
|
||||
mu sync.Mutex
|
||||
assignments map[string][]byte
|
||||
entries map[string][]byte
|
||||
said map[string]string
|
||||
poke chan struct{}
|
||||
}
|
||||
|
||||
// New is a provider answering its consumers as opts say.
|
||||
func New(opts Options) *Provider {
|
||||
if opts.Identity == nil {
|
||||
opts.Identity = ConsumerIdentity
|
||||
}
|
||||
if opts.Every <= 0 {
|
||||
opts.Every = time.Minute
|
||||
}
|
||||
if opts.Logf == nil {
|
||||
opts.Logf = func(f string, a ...any) { fmt.Printf("[provider] "+f+"\n", a...) }
|
||||
}
|
||||
return &Provider{opts: opts, assignments: map[string][]byte{}, entries: map[string][]byte{},
|
||||
said: map[string]string{}, poke: make(chan struct{}, 1)}
|
||||
}
|
||||
|
||||
// What a pass did for one bind.
|
||||
const (
|
||||
Answered = "answered"
|
||||
Refused = "refused"
|
||||
Waiting = "waiting"
|
||||
Failed = "failed"
|
||||
Removed = "removed"
|
||||
// Superseded is an entry no bind asks for any more that another instance of the provider wrote after this pass
|
||||
// read it: it is left standing.
|
||||
Superseded = "superseded"
|
||||
)
|
||||
|
||||
// Outcome is what a pass did for one consumer's entry.
|
||||
type Outcome struct {
|
||||
Key string `json:"key"`
|
||||
Status string `json:"status"`
|
||||
Why string `json:"why,omitempty"`
|
||||
}
|
||||
|
||||
// Run watches the binds and the provider's own entries, and passes once at start, on every change and every
|
||||
// Every, until ctx ends.
|
||||
func (p *Provider) Run(ctx context.Context) error {
|
||||
if err := p.watch(ctx, BucketOf(p.opts.Module), p.entries); err != nil {
|
||||
return fmt.Errorf("watching %s: %w", BucketOf(p.opts.Module), err)
|
||||
}
|
||||
if err := p.watch(ctx, AssignmentsBucket, p.assignments); err != nil {
|
||||
return fmt.Errorf("watching %s: %w", AssignmentsBucket, err)
|
||||
}
|
||||
tick := time.NewTicker(p.opts.Every)
|
||||
defer tick.Stop()
|
||||
for {
|
||||
p.mu.Lock()
|
||||
assignments, entries := clone(p.assignments), clone(p.entries)
|
||||
p.mu.Unlock()
|
||||
if _, err := p.pass(ctx, assignments, entries); err != nil {
|
||||
p.say("pass", err.Error())
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-p.poke:
|
||||
case <-tick.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Provider) watch(ctx context.Context, bucket string, into map[string][]byte) error {
|
||||
return p.opts.Bus.Watch(ctx, bucket, func(key string, value []byte) {
|
||||
p.mu.Lock()
|
||||
if value == nil {
|
||||
delete(into, key)
|
||||
} else {
|
||||
into[key] = append([]byte(nil), value...)
|
||||
}
|
||||
p.mu.Unlock()
|
||||
if bucket == AssignmentsBucket {
|
||||
select {
|
||||
case p.poke <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Pass reads the binds and the provider's entries from the bus and answers once: each bind of this provider gets
|
||||
// its entry, and each entry of this provider no bind asks for is removed. An assignment that cannot be read, or that
|
||||
// is not the one its key names, is skipped and said (an outcome of status Failed under the assignment's key, and a
|
||||
// line of the log), and the entries of its consumer are left as they are, since a bind missed would read as an
|
||||
// unbind; the pass goes on for the rest. Run passes by itself; Pass is for a provider that passes when it chooses.
|
||||
//
|
||||
// Over StateBus each call to Pass leaves two watches open for the life of the process, silenced once the call has
|
||||
// ended (see StateBus): a provider that passes often should use Run, which watches once.
|
||||
func (p *Provider) Pass(ctx context.Context) ([]Outcome, error) {
|
||||
watchCtx, cancel := context.WithCancel(ctx)
|
||||
defer cancel()
|
||||
var mu sync.Mutex
|
||||
read := func(bucket string) (map[string][]byte, error) {
|
||||
got := map[string][]byte{}
|
||||
err := p.opts.Bus.Watch(watchCtx, bucket, func(key string, value []byte) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if value == nil {
|
||||
delete(got, key)
|
||||
} else {
|
||||
got[key] = append([]byte(nil), value...)
|
||||
}
|
||||
})
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return clone(got), err
|
||||
}
|
||||
entries, err := read(BucketOf(p.opts.Module))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading %s: %w", BucketOf(p.opts.Module), err)
|
||||
}
|
||||
assignments, err := read(AssignmentsBucket)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading %s: %w", AssignmentsBucket, err)
|
||||
}
|
||||
return p.pass(ctx, assignments, entries)
|
||||
}
|
||||
|
||||
// planned is as much of an assignment as the helper reads: lenient, since the plan is the client's to act on.
|
||||
type planned struct {
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
Provisioning *struct {
|
||||
Provisions []struct {
|
||||
Provision string `json:"provision"`
|
||||
Local string `json:"local"`
|
||||
Provider string `json:"provider"`
|
||||
ProviderModule string `json:"provider-module"`
|
||||
Contributes map[string]any `json:"contributes"`
|
||||
At string `json:"at"`
|
||||
From string `json:"from"`
|
||||
As string `json:"as"`
|
||||
Names map[string]string `json:"names"`
|
||||
CredentialForm string `json:"credential-form"`
|
||||
Name string `json:"name"`
|
||||
} `json:"provisions"`
|
||||
} `json:"provisioning"`
|
||||
}
|
||||
|
||||
func (p *Provider) pass(ctx context.Context, assignments, entries map[string][]byte) ([]Outcome, error) {
|
||||
var binds []Entry
|
||||
var out []Outcome
|
||||
unknown := map[string]bool{} // consumers whose assignment was skipped: their entries are not judged
|
||||
for _, key := range sortedKeys(assignments) {
|
||||
var a planned
|
||||
if err := json.Unmarshal(assignments[key], &a); err != nil || a.Node+"."+a.Module != key {
|
||||
o := Outcome{Key: key, Status: Failed, Why: fmt.Sprintf("the assignment %s is not the controller's, so "+
|
||||
"who it binds is not known; it is skipped, and its entries are left as they are", key)}
|
||||
out = append(out, o)
|
||||
p.say(key, o.Status+" "+o.Why)
|
||||
unknown[key] = true
|
||||
continue
|
||||
}
|
||||
if a.Provisioning == nil {
|
||||
continue
|
||||
}
|
||||
for _, b := range a.Provisioning.Provisions {
|
||||
if b.Provider != p.opts.Node || b.ProviderModule != p.opts.Module {
|
||||
continue
|
||||
}
|
||||
binds = append(binds, Entry{Provider: p.opts.Node, ProviderModule: p.opts.Module,
|
||||
Provision: b.Provision, Consumer: a.Node, Module: a.Module, Local: b.Local,
|
||||
Contributes: b.Contributes, At: b.At, From: b.From, As: b.As, Names: b.Names,
|
||||
CredentialForm: b.CredentialForm, Name: b.Name})
|
||||
}
|
||||
}
|
||||
|
||||
want := map[string]bool{}
|
||||
for _, b := range binds {
|
||||
key := b.Key()
|
||||
want[key] = true
|
||||
o := p.answer(ctx, b, entries[key])
|
||||
out = append(out, o)
|
||||
p.say(key, o.Status+" "+o.Why)
|
||||
}
|
||||
for _, key := range sortedKeys(entries) {
|
||||
if want[key] || unknown[consumerOf(key)] {
|
||||
continue
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(entries[key], &e); err != nil || e.Provider != p.opts.Node ||
|
||||
e.ProviderModule != p.opts.Module {
|
||||
continue // another machine's instance of this provider wrote it, or nothing of this helper's
|
||||
}
|
||||
if p.opts.Gone != nil {
|
||||
if err := p.opts.Gone(ctx, e); err != nil {
|
||||
o := Outcome{Key: key, Status: Failed, Why: "no longer bound, and the provider could not let it go: " +
|
||||
err.Error()}
|
||||
out = append(out, o)
|
||||
p.say(key, o.Status+" "+o.Why)
|
||||
continue
|
||||
}
|
||||
}
|
||||
if err := p.remove(ctx, key, entries[key]); err != nil {
|
||||
if errors.Is(err, stdio.ErrChanged) {
|
||||
o := Outcome{Key: key, Status: Superseded, Why: "another instance of the provider wrote its entry " +
|
||||
"since this pass read it, so it stands"}
|
||||
out = append(out, o)
|
||||
p.say(key, o.Status+" "+o.Why)
|
||||
continue
|
||||
}
|
||||
o := Outcome{Key: key, Status: Failed, Why: "no longer bound, and its entry could not be removed: " +
|
||||
err.Error()}
|
||||
out = append(out, o)
|
||||
p.say(key, o.Status+" "+o.Why)
|
||||
continue
|
||||
}
|
||||
out = append(out, Outcome{Key: key, Status: Removed})
|
||||
p.say(key, Removed)
|
||||
p.mu.Lock()
|
||||
delete(p.said, key)
|
||||
p.mu.Unlock()
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// consumerOf is the `<node>.<module>` an entry's key starts with: its first two parts.
|
||||
func consumerOf(key string) string {
|
||||
parts := strings.SplitN(key, ".", 3)
|
||||
if len(parts) < 3 {
|
||||
return key
|
||||
}
|
||||
return parts[0] + "." + parts[1]
|
||||
}
|
||||
|
||||
// remove deletes an entry this instance read as held, and only while it is still as read when the Bus can say so.
|
||||
func (p *Provider) remove(ctx context.Context, key string, held []byte) error {
|
||||
if c, ok := p.opts.Bus.(ConditionalBus); ok {
|
||||
return c.DeleteIf(ctx, BucketOf(p.opts.Module), key, held)
|
||||
}
|
||||
return p.opts.Bus.Delete(ctx, BucketOf(p.opts.Module), key)
|
||||
}
|
||||
|
||||
// answer makes one bind's entry and writes it when it differs from what the bucket holds.
|
||||
func (p *Provider) answer(ctx context.Context, e Entry, have []byte) Outcome {
|
||||
key := e.Key()
|
||||
offer, offered := p.opts.Offers[e.Provision]
|
||||
switch {
|
||||
case !offered:
|
||||
e.Refused = fmt.Sprintf("%s on %s is bound to %s on %s for %s, which %s does not serve", e.Module,
|
||||
e.Consumer, e.ProviderModule, e.Provider, e.Provision, e.ProviderModule)
|
||||
case e.Local != "" && derives(offer.Serves):
|
||||
e.Refused = fmt.Sprintf("%s on %s keeps several holders of %s (this one is %q), and %s on %s derives %s for "+
|
||||
"each consumer from its login: each holder has a login of its own and a consumer is told one value per "+
|
||||
"requirement, so the two ends would name different things (novox/hq ADR 0202 §10, ADR 0349 clause 12)",
|
||||
e.Module, e.Consumer, e.Provision, e.Local, e.ProviderModule, e.Provider,
|
||||
strings.Join(derivedKeys(offer.Serves), ", "))
|
||||
}
|
||||
if e.Refused != "" {
|
||||
return p.write(ctx, e, have, Outcome{Key: key, Status: Refused, Why: e.Refused})
|
||||
}
|
||||
|
||||
// The plan's identity is the controller's, slug first; Identity only answers for an entry without one.
|
||||
if e.As == "" {
|
||||
e.As = p.opts.Identity(e.Consumer, e.Module)
|
||||
}
|
||||
values, err := ServedTo(offer.Serves, e.As)
|
||||
if err != nil {
|
||||
e.Refused = fmt.Sprintf("%s on %s serving %s to %s on %s: %v", e.ProviderModule, e.Provider, e.Provision,
|
||||
e.Module, e.Consumer, err)
|
||||
e.As = ""
|
||||
return p.write(ctx, e, have, Outcome{Key: key, Status: Refused, Why: e.Refused})
|
||||
}
|
||||
if e.At != "" || e.From != "" {
|
||||
if values == nil {
|
||||
values = map[string]any{}
|
||||
}
|
||||
if e.At != "" {
|
||||
values["at"] = e.At
|
||||
}
|
||||
if e.From != "" {
|
||||
values["from"] = e.From
|
||||
}
|
||||
}
|
||||
if len(e.Names) > 0 {
|
||||
if values == nil {
|
||||
values = map[string]any{}
|
||||
}
|
||||
for key, name := range e.Names {
|
||||
values[key] = name
|
||||
}
|
||||
}
|
||||
e.Values = values
|
||||
var credential []byte
|
||||
// The plan's form wins; the offer's is the default.
|
||||
// An offer that declares no credential has none whatever the plan says: the controller writes "pair" for it.
|
||||
form := e.CredentialForm
|
||||
if offer.Credential == NoCredential {
|
||||
form = ""
|
||||
} else if form == "" {
|
||||
form = map[CredentialForm]string{Shared: "shared", PerConsumer: "pair"}[offer.Credential]
|
||||
}
|
||||
switch form {
|
||||
case "shared":
|
||||
e.User = offer.User
|
||||
e.VaultName = SharedName(e.Provider, e.Provision)
|
||||
case "pair":
|
||||
e.User = e.As
|
||||
e.VaultName = PairName(e.Provider, e.Provision, e.Consumer, e.Module, e.Local)
|
||||
case "issued", "given":
|
||||
// A per-holder value, as mesh-vault names it (perholder.go).
|
||||
e.User = e.As
|
||||
e.VaultName = HolderName(e.Provider, e.Provision, e.Consumer, e.Module)
|
||||
}
|
||||
if e.VaultName != "" {
|
||||
credential, err = p.opts.Vault.Value(ctx, e.VaultName)
|
||||
if err != nil {
|
||||
return Outcome{Key: key, Status: Waiting, Why: fmt.Sprintf("the vault's %s: %v", e.VaultName, err)}
|
||||
}
|
||||
}
|
||||
if p.opts.Make != nil {
|
||||
if err := p.opts.Make(ctx, Consumer{Entry: e, Credential: credential}); err != nil {
|
||||
why := err.Error()
|
||||
why = redact(why, credential)
|
||||
return Outcome{Key: key, Status: Failed, Why: "the provider could not make it: " + why}
|
||||
}
|
||||
}
|
||||
return p.write(ctx, e, have, Outcome{Key: key, Status: Answered})
|
||||
}
|
||||
|
||||
// redact takes a credential out of text in the forms it can come back in: as it is, in base64 (standard and URL,
|
||||
// padded and not) and as a JSON string says it (escapes included).
|
||||
func redact(text string, credential []byte) string {
|
||||
if len(credential) == 0 {
|
||||
return text
|
||||
}
|
||||
quoted, _ := json.Marshal(string(credential))
|
||||
forms := []string{string(credential), base64.StdEncoding.EncodeToString(credential),
|
||||
base64.RawStdEncoding.EncodeToString(credential), base64.URLEncoding.EncodeToString(credential),
|
||||
base64.RawURLEncoding.EncodeToString(credential), string(quoted[1 : len(quoted)-1])}
|
||||
sort.SliceStable(forms, func(i, j int) bool { return len(forms[i]) > len(forms[j]) })
|
||||
for _, f := range forms {
|
||||
text = strings.ReplaceAll(text, f, "<credential>")
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func (p *Provider) write(ctx context.Context, e Entry, have []byte, o Outcome) Outcome {
|
||||
body, err := json.Marshal(e)
|
||||
if err != nil {
|
||||
return Outcome{Key: o.Key, Status: Failed, Why: err.Error()}
|
||||
}
|
||||
if sameJSON(have, body) {
|
||||
return o
|
||||
}
|
||||
if err := p.opts.Bus.Put(ctx, BucketOf(p.opts.Module), o.Key, body); err != nil {
|
||||
return Outcome{Key: o.Key, Status: Failed, Why: "its entry could not be written: " + err.Error()}
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
// say logs what a key's pass said when it differs from the last.
|
||||
func (p *Provider) say(key, what string) {
|
||||
p.mu.Lock()
|
||||
same := p.said[key] == what
|
||||
p.said[key] = what
|
||||
p.mu.Unlock()
|
||||
if !same {
|
||||
p.opts.Logf("%s: %s", key, strings.TrimSpace(what))
|
||||
}
|
||||
}
|
||||
|
||||
// SharedName is the vault's one value per provider and provision (ADR 0158, issue 677).
|
||||
func SharedName(provider, provision string) string { return "provision/" + provider + "/" + provision }
|
||||
|
||||
// HolderName is the vault's value of one holder's own credential, issued or given for it (mesh-vault perholder.go).
|
||||
func HolderName(provider, provision, node, module string) string {
|
||||
return "provision/" + provider + "/" + provision + "/" + node + "/" + module
|
||||
}
|
||||
|
||||
// PairName is the vault's value of one consumer's own login (ADR 0048, ADR 0158 §11, issue 677's note 12).
|
||||
func PairName(provider, provision, node, module, local string) string {
|
||||
n := "pair/" + provider + "/" + provision + "/" + node + "/" + module
|
||||
if local != "" {
|
||||
n += "/" + local
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// identityUnusable is every character a consumer's identity does not hold (mesh-controller internal/catalogue
|
||||
// identity.go, the one derivation).
|
||||
var identityUnusable = regexp.MustCompile(`[^a-z0-9_]+`)
|
||||
|
||||
// ConsumerIdentity is what one module on one machine is called wherever it authenticates: `mesh_<node>_<module>`,
|
||||
// lower case, anything but letters, digits and `_` written `_` (mesh-controller ConsumerIdentity). module is the
|
||||
// identity source: the module's slug when it declares one, otherwise its name (ADR 0049).
|
||||
func ConsumerIdentity(node, module string) string {
|
||||
clean := func(s string) string {
|
||||
return strings.Trim(identityUnusable.ReplaceAllString(strings.ToLower(s), "_"), "_")
|
||||
}
|
||||
return "mesh_" + clean(node) + "_" + clean(module)
|
||||
}
|
||||
|
||||
// consumerFact is `${consumer:<fact>}` or `${consumer:<fact>:<alphabet>}`.
|
||||
var consumerFact = regexp.MustCompile(`\$\{consumer:([a-z][a-z0-9-]*)(?::([a-z][a-z0-9-]*))?\}`)
|
||||
|
||||
// ServedTo fills an offer's served values for one consumer whose identity is as (ADR 0202 §1): `${consumer:as}` as
|
||||
// it is, `${consumer:as:dns}` with `-` for `_`. Only strings carry placeholders; any other fact or alphabet is
|
||||
// refused, never left standing.
|
||||
func ServedTo(serves map[string]any, as string) (map[string]any, error) {
|
||||
if serves == nil {
|
||||
return nil, nil
|
||||
}
|
||||
out := make(map[string]any, len(serves))
|
||||
for _, key := range sortedKeys(serves) {
|
||||
text, ok := serves[key].(string)
|
||||
if !ok || !strings.Contains(text, "${consumer:") {
|
||||
out[key] = serves[key]
|
||||
continue
|
||||
}
|
||||
var failed error
|
||||
out[key] = consumerFact.ReplaceAllStringFunc(text, func(match string) string {
|
||||
parts := consumerFact.FindStringSubmatch(match)
|
||||
switch {
|
||||
case parts[1] == "as" && parts[2] == "":
|
||||
return as
|
||||
case parts[1] == "as" && parts[2] == "dns":
|
||||
return strings.ReplaceAll(as, "_", "-")
|
||||
}
|
||||
if failed == nil {
|
||||
failed = fmt.Errorf("the value served as %q says %s, and the mesh states only ${consumer:as} and "+
|
||||
"${consumer:as:dns}", key, match)
|
||||
}
|
||||
return match
|
||||
})
|
||||
if failed != nil {
|
||||
return nil, failed
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func derives(serves map[string]any) bool { return len(derivedKeys(serves)) > 0 }
|
||||
|
||||
func derivedKeys(serves map[string]any) []string {
|
||||
var out []string
|
||||
for _, k := range sortedKeys(serves) {
|
||||
if text, ok := serves[k].(string); ok && strings.Contains(text, "${consumer:") {
|
||||
out = append(out, k)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func clone(m map[string][]byte) map[string][]byte {
|
||||
out := make(map[string][]byte, len(m))
|
||||
for k, v := range m {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func sameJSON(a, b []byte) bool {
|
||||
if a == nil || b == nil {
|
||||
return false
|
||||
}
|
||||
var x, y any
|
||||
if json.Unmarshal(a, &x) != nil || json.Unmarshal(b, &y) != nil {
|
||||
return false
|
||||
}
|
||||
ja, _ := json.Marshal(x)
|
||||
jb, _ := json.Marshal(y)
|
||||
return string(ja) == string(jb)
|
||||
}
|
||||
|
||||
func sortedKeys[V any](m map[string]V) []string {
|
||||
if len(m) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// StateBus is the Bus of a provider that runs as a module bundle over the runtime's stdio: the provider's own bucket,
|
||||
// BucketOf(module), is the module's state `bound`, and AssignmentsBucket is the state it reads,
|
||||
// `mesh-controller.assignments` (both declared in the module's manifest). Any other bucket is refused.
|
||||
//
|
||||
// A stdio state watch lasts as long as the process and cannot be ended (stdio.KeptState.Watch), so Watch honours the
|
||||
// Bus contract by silencing instead: once ctx has ended the handler is never called again, though the runtime keeps
|
||||
// sending changes that are dropped. A Pass, which watches twice per call, is therefore correct, only the dropped
|
||||
// changes of its two spent watches remain.
|
||||
func StateBus(module string) Bus { return stateBus{module: module} }
|
||||
|
||||
type stateBus struct{ module string }
|
||||
|
||||
func (b stateBus) state(bucket string) (stdio.KeptState, error) {
|
||||
switch bucket {
|
||||
case BucketOf(b.module):
|
||||
return stdio.State("bound"), nil
|
||||
case AssignmentsBucket:
|
||||
return stdio.State("mesh-controller.assignments"), nil
|
||||
}
|
||||
return stdio.KeptState{}, fmt.Errorf("%s is not a bucket this provider keeps or reads", bucket)
|
||||
}
|
||||
|
||||
func (b stateBus) Watch(ctx context.Context, bucket string, handle func(key string, value []byte)) error {
|
||||
s, err := b.state(bucket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.Watch("", func(ch stdio.StateChange) error {
|
||||
if ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
if ch.Op == "delete" {
|
||||
handle(ch.Key, nil)
|
||||
} else {
|
||||
handle(ch.Key, ch.Value)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func (b stateBus) Put(_ context.Context, bucket, key string, value []byte) error {
|
||||
s, err := b.state(bucket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = s.Put(key, json.RawMessage(value))
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteIf removes the key only while it still holds what the caller read; it reads the key first, so another
|
||||
// writer's change in that short while can still be lost.
|
||||
func (b stateBus) DeleteIf(_ context.Context, bucket, key string, held []byte) error {
|
||||
s, err := b.state(bucket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cur, err := s.Get(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if cur == nil {
|
||||
return nil
|
||||
}
|
||||
if !sameJSON(cur.Value, held) {
|
||||
return fmt.Errorf("%s is not as it was read: %w", key, stdio.ErrChanged)
|
||||
}
|
||||
return s.Delete(key)
|
||||
}
|
||||
|
||||
func (b stateBus) Delete(_ context.Context, bucket, key string) error {
|
||||
s, err := b.state(bucket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.Delete(key)
|
||||
}
|
||||
@@ -0,0 +1,803 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// standIn is a stand-in bus: buckets of keys and values, and watches that hand the current values first and then
|
||||
// every change, as the bus's key-value watch does.
|
||||
type standIn struct {
|
||||
mu sync.Mutex
|
||||
buckets map[string]map[string][]byte
|
||||
watchers map[string][]func(string, []byte)
|
||||
}
|
||||
|
||||
func newStandIn() *standIn {
|
||||
return &standIn{buckets: map[string]map[string][]byte{}, watchers: map[string][]func(string, []byte){}}
|
||||
}
|
||||
|
||||
func (s *standIn) Watch(ctx context.Context, bucket string, handle func(key string, value []byte)) error {
|
||||
s.mu.Lock()
|
||||
current := map[string][]byte{}
|
||||
for k, v := range s.buckets[bucket] {
|
||||
current[k] = v
|
||||
}
|
||||
s.watchers[bucket] = append(s.watchers[bucket], handle)
|
||||
s.mu.Unlock()
|
||||
for _, k := range sortedKeys(current) {
|
||||
handle(k, current[k])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *standIn) Put(_ context.Context, bucket, key string, value []byte) error {
|
||||
s.mu.Lock()
|
||||
if s.buckets[bucket] == nil {
|
||||
s.buckets[bucket] = map[string][]byte{}
|
||||
}
|
||||
s.buckets[bucket][key] = append([]byte(nil), value...)
|
||||
ws := slices.Clone(s.watchers[bucket])
|
||||
s.mu.Unlock()
|
||||
for _, w := range ws {
|
||||
w(key, value)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *standIn) Delete(_ context.Context, bucket, key string) error {
|
||||
s.mu.Lock()
|
||||
delete(s.buckets[bucket], key)
|
||||
ws := slices.Clone(s.watchers[bucket])
|
||||
s.mu.Unlock()
|
||||
for _, w := range ws {
|
||||
w(key, nil)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DeleteIf removes the key only while it still holds what the caller read.
|
||||
func (s *standIn) DeleteIf(ctx context.Context, bucket, key string, held []byte) error {
|
||||
s.mu.Lock()
|
||||
cur := s.buckets[bucket][key]
|
||||
s.mu.Unlock()
|
||||
if !bytes.Equal(cur, held) {
|
||||
return fmt.Errorf("%s: %w", key, stdio.ErrChanged)
|
||||
}
|
||||
return s.Delete(ctx, bucket, key)
|
||||
}
|
||||
|
||||
func (s *standIn) get(bucket, key string) []byte {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.buckets[bucket][key]
|
||||
}
|
||||
|
||||
func (s *standIn) keys(bucket string) []string {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return sortedKeys(s.buckets[bucket])
|
||||
}
|
||||
|
||||
// everything is every value the bus holds, for a test that no secret is in a bucket.
|
||||
func (s *standIn) everything() []byte {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
var all []byte
|
||||
for _, b := range s.buckets {
|
||||
for k, v := range b {
|
||||
all = append(all, k...)
|
||||
all = append(all, v...)
|
||||
}
|
||||
}
|
||||
return all
|
||||
}
|
||||
|
||||
// vault is a stand-in vault: a value per name, or nothing yet.
|
||||
type vault struct {
|
||||
mu sync.Mutex
|
||||
values map[string]string
|
||||
asked []string
|
||||
}
|
||||
|
||||
func (v *vault) Value(_ context.Context, name string) ([]byte, error) {
|
||||
v.mu.Lock()
|
||||
defer v.mu.Unlock()
|
||||
v.asked = append(v.asked, name)
|
||||
val, ok := v.values[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("the vault holds no %s yet", name)
|
||||
}
|
||||
return []byte(val), nil
|
||||
}
|
||||
|
||||
// maker records what the provider's adapter was asked to make, with the credential it was handed.
|
||||
type maker struct {
|
||||
mu sync.Mutex
|
||||
made map[string]string
|
||||
// contributed is what each consumer's entry carried of its plan's contribution.
|
||||
contributed map[string]map[string]any
|
||||
// as is the identity each consumer's entry was made under.
|
||||
as map[string]string
|
||||
gone []string
|
||||
fail error
|
||||
calls int
|
||||
}
|
||||
|
||||
func (m *maker) Make(_ context.Context, c Consumer) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.calls++
|
||||
if m.fail != nil {
|
||||
return m.fail
|
||||
}
|
||||
if m.made == nil {
|
||||
m.made = map[string]string{}
|
||||
}
|
||||
m.made[c.Entry.Key()] = string(c.Credential)
|
||||
if m.contributed == nil {
|
||||
m.contributed = map[string]map[string]any{}
|
||||
}
|
||||
m.contributed[c.Entry.Key()] = c.Entry.Contributes
|
||||
if m.as == nil {
|
||||
m.as = map[string]string{}
|
||||
}
|
||||
m.as[c.Entry.Key()] = c.Entry.As
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *maker) Gone(_ context.Context, e Entry) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.gone = append(m.gone, e.Key())
|
||||
return nil
|
||||
}
|
||||
|
||||
// assign puts a consumer's assignment with its plan's provisions into the stand-in mirror.
|
||||
func assign(t *testing.T, bus *standIn, node, module string, provisions ...map[string]string) {
|
||||
t.Helper()
|
||||
ps := []map[string]string{}
|
||||
ps = append(ps, provisions...)
|
||||
raw, err := json.Marshal(map[string]any{"node": node, "module": module, "assigned": "2026-10-15T00:00:00Z",
|
||||
"provisioning": map[string]any{"templates": map[string]string{"source": "x", "digest": "y"},
|
||||
"provisions": ps}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := bus.Put(context.Background(), AssignmentsBucket, node+"."+module, raw); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func bind(provision, local, provider, providerModule string) map[string]string {
|
||||
b := map[string]string{"provision": provision, "provider": provider, "provider-module": providerModule}
|
||||
if local != "" {
|
||||
b["local"] = local
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func entryOf(t *testing.T, bus *standIn, key string) Entry {
|
||||
t.Helper()
|
||||
raw := bus.get(BucketOf("postgres"), key)
|
||||
if raw == nil {
|
||||
t.Fatalf("no entry %s in %s; it holds %v", key, BucketOf("postgres"), bus.keys(BucketOf("postgres")))
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(raw, &e); err != nil {
|
||||
t.Fatalf("entry %s: %v", key, err)
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
// postgres is a provider of postgres-database on novox with a login per consumer (ADR 0048, ADR 0158 §11), its
|
||||
// served values derived from each consumer's identity (ADR 0202 §1).
|
||||
func postgres(bus *standIn, v *vault, m *maker) *Provider {
|
||||
return New(Options{
|
||||
Node: "novox", Module: "postgres", Bus: bus, Vault: v, Make: m.Make, Gone: m.Gone,
|
||||
Offers: map[string]Offer{
|
||||
"postgres-database": {Credential: PerConsumer, Serves: map[string]any{
|
||||
"host": "postgres.novox.mesh", "port": 5432, "database": "${consumer:as}"}},
|
||||
},
|
||||
Logf: func(string, ...any) {},
|
||||
})
|
||||
}
|
||||
|
||||
// A provider answers each consumer that binds it with an entry of its own in its own bucket: composed from its
|
||||
// served values and the consumer's identity, naming the credential the vault keeps for that consumer, and never
|
||||
// the credential itself; its adapter is handed each consumer's own value read from the vault.
|
||||
func TestEachConsumerGetsAnEntryOfItsOwnAndTheVaultsValue(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{
|
||||
"pair/novox/postgres-database/ace/letta": "letta-secret-1",
|
||||
"pair/novox/postgres-database/g14/umami": "umami-secret-2",
|
||||
}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
assign(t, bus, "g14", "umami", bind("postgres-database", "", "novox", "postgres"))
|
||||
p := postgres(bus, v, m)
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := bus.keys(BucketOf("postgres")); !slices.Equal(got, []string{"ace.letta.postgres-database",
|
||||
"g14.umami.postgres-database"}) {
|
||||
t.Fatalf("entries %v", got)
|
||||
}
|
||||
e := entryOf(t, bus, "ace.letta.postgres-database")
|
||||
if e.Provider != "novox" || e.ProviderModule != "postgres" || e.Consumer != "ace" || e.Module != "letta" ||
|
||||
e.As != "mesh_ace_letta" || e.User != "mesh_ace_letta" ||
|
||||
e.VaultName != "pair/novox/postgres-database/ace/letta" || e.Refused != "" {
|
||||
t.Fatalf("entry %+v", e)
|
||||
}
|
||||
if e.Values["database"] != "mesh_ace_letta" || e.Values["host"] != "postgres.novox.mesh" ||
|
||||
e.Values["port"] != float64(5432) {
|
||||
t.Fatalf("values %v", e.Values)
|
||||
}
|
||||
if m.made["ace.letta.postgres-database"] != "letta-secret-1" ||
|
||||
m.made["g14.umami.postgres-database"] != "umami-secret-2" {
|
||||
t.Fatalf("made %v", m.made)
|
||||
}
|
||||
if bytes.Contains(bus.everything(), []byte("secret-")) {
|
||||
t.Fatal("a credential is in a bucket")
|
||||
}
|
||||
for _, o := range out {
|
||||
if o.Status != Answered {
|
||||
t.Fatalf("outcome %+v", o)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A provider whose offer shares one credential (ADR 0158) names the vault's one value for the provision, and its
|
||||
// one user name, to every consumer.
|
||||
func TestASharedCredentialIsTheVaultsOneValueAndTheProvidersOneUser(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"provision/novox/sonarr-api": "one-value"}}
|
||||
assign(t, bus, "ace", "overseerr", bind("sonarr-api", "", "novox", "sonarr"))
|
||||
assign(t, bus, "g14", "homepage", bind("sonarr-api", "", "novox", "sonarr"))
|
||||
p := New(Options{Node: "novox", Module: "sonarr", Bus: bus, Vault: v, Make: m.Make, Logf: func(string, ...any) {},
|
||||
Offers: map[string]Offer{"sonarr-api": {Credential: Shared, User: "sonarr",
|
||||
Serves: map[string]any{"url": "http://sonarr.novox.mesh:8989"}}}})
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, key := range []string{"ace.overseerr.sonarr-api", "g14.homepage.sonarr-api"} {
|
||||
var e Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("sonarr"), key), &e); err != nil {
|
||||
t.Fatalf("%s: %v", key, err)
|
||||
}
|
||||
if e.VaultName != "provision/novox/sonarr-api" || e.User != "sonarr" || e.Values["url"] == nil {
|
||||
t.Fatalf("%s: %+v", key, e)
|
||||
}
|
||||
if m.made[key] != "one-value" {
|
||||
t.Fatalf("%s made with %q", key, m.made[key])
|
||||
}
|
||||
}
|
||||
if bytes.Contains(bus.everything(), []byte("one-value")) {
|
||||
t.Fatal("the shared credential is in a bucket")
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer that keeps several holders of one provision is served no derived value: its entry refuses it, naming
|
||||
// both ends, so its client can report it (ADR 0349 clause 12, ADR 0202 §10); nothing is made for it.
|
||||
func TestAMultiHolderConsumerIsRefusedADerivedValueNamingBothEnds(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta/archive": "x-secret"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "archive", "novox", "postgres"))
|
||||
out, err := postgres(bus, v, m).Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e := entryOf(t, bus, "ace.letta.postgres-database.archive")
|
||||
for _, says := range []string{"letta on ace", "postgres on novox", "postgres-database", "archive"} {
|
||||
if !strings.Contains(e.Refused, says) {
|
||||
t.Fatalf("the refusal does not name %q: %s", says, e.Refused)
|
||||
}
|
||||
}
|
||||
if e.Values != nil || e.VaultName != "" {
|
||||
t.Fatalf("a refused entry carries values: %+v", e)
|
||||
}
|
||||
if m.calls != 0 {
|
||||
t.Fatal("the adapter made a refused consumer")
|
||||
}
|
||||
if len(out) != 1 || out[0].Status != Refused {
|
||||
t.Fatalf("outcomes %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer bound under a local name is served when the offer derives nothing from it: its credential is its own
|
||||
// pair row under that local name.
|
||||
func TestALocalNameIsServedWhenNothingIsDerived(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/redis-cache/ace/letta/queue": "q-secret"}}
|
||||
assign(t, bus, "ace", "letta", bind("redis-cache", "queue", "novox", "redis"))
|
||||
p := New(Options{Node: "novox", Module: "redis", Bus: bus, Vault: v, Make: m.Make, Logf: func(string, ...any) {},
|
||||
Offers: map[string]Offer{"redis-cache": {Credential: PerConsumer, Serves: map[string]any{"port": 6379}}}})
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("redis"), "ace.letta.redis-cache.queue"), &e); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e.Refused != "" || e.VaultName != "pair/novox/redis-cache/ace/letta/queue" || e.Local != "queue" {
|
||||
t.Fatalf("entry %+v", e)
|
||||
}
|
||||
if m.made["ace.letta.redis-cache.queue"] != "q-secret" {
|
||||
t.Fatalf("made %v", m.made)
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer that unbinds loses its entry and the adapter hears it is gone; an entry another machine's instance of
|
||||
// the same provider wrote is not this one's to remove.
|
||||
func TestAnUnbindRemovesTheEntryAndLeavesAnotherProvidersAlone(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s1"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
other, _ := json.Marshal(Entry{Provider: "shanks", ProviderModule: "postgres", Provision: "postgres-database",
|
||||
Consumer: "g14", Module: "gitea"})
|
||||
_ = bus.Put(context.Background(), BucketOf("postgres"), "g14.gitea.postgres-database", other)
|
||||
p := postgres(bus, v, m)
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = bus.Delete(context.Background(), AssignmentsBucket, "ace.letta")
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := bus.keys(BucketOf("postgres")); !slices.Equal(got, []string{"g14.gitea.postgres-database"}) {
|
||||
t.Fatalf("entries left %v", got)
|
||||
}
|
||||
if !slices.Equal(m.gone, []string{"ace.letta.postgres-database"}) {
|
||||
t.Fatalf("gone %v", m.gone)
|
||||
}
|
||||
if len(out) != 1 || out[0].Status != Removed {
|
||||
t.Fatalf("outcomes %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A bind whose credential the vault does not hold yet is waited for: nothing is made and no entry is written, so a
|
||||
// consumer is never told of a login its provider has not made; once the vault holds it, it is answered.
|
||||
func TestACredentialTheVaultDoesNotHoldYetIsWaitedFor(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
p := postgres(bus, v, m)
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(out) != 1 || out[0].Status != Waiting || !strings.Contains(out[0].Why, "pair/novox/postgres-database/ace/letta") {
|
||||
t.Fatalf("outcomes %+v", out)
|
||||
}
|
||||
if bus.keys(BucketOf("postgres")) != nil || m.calls != 0 {
|
||||
t.Fatal("a consumer was answered before its credential was there")
|
||||
}
|
||||
v.values["pair/novox/postgres-database/ace/letta"] = "late"
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entryOf(t, bus, "ace.letta.postgres-database").Refused != "" || m.made["ace.letta.postgres-database"] != "late" {
|
||||
t.Fatal("not answered once the vault held the value")
|
||||
}
|
||||
}
|
||||
|
||||
// Only the binds naming this provider, on this machine, are its own; a bind naming it for a provision it does not
|
||||
// offer is refused, naming both ends.
|
||||
func TestOnlyBindsNamingThisProviderAreAnswered(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "shanks", "postgres"),
|
||||
bind("mongodb-database", "", "novox", "postgres"))
|
||||
assign(t, bus, "ace", "other", bind("postgres-database", "", "novox", "mssql"))
|
||||
out, err := postgres(bus, v, m).Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := bus.keys(BucketOf("postgres")); !slices.Equal(got, []string{"ace.letta.mongodb-database"}) {
|
||||
t.Fatalf("entries %v", got)
|
||||
}
|
||||
e := entryOf(t, bus, "ace.letta.mongodb-database")
|
||||
if !strings.Contains(e.Refused, "postgres on novox") || !strings.Contains(e.Refused, "letta on ace") {
|
||||
t.Fatalf("refusal %q", e.Refused)
|
||||
}
|
||||
if len(out) != 1 || m.calls != 0 {
|
||||
t.Fatalf("outcomes %+v, %d made", out, m.calls)
|
||||
}
|
||||
}
|
||||
|
||||
// An assignment that cannot be read is skipped and said, by its key and why; the rest of the pass goes on (issue 825,
|
||||
// finding 3). Its consumer's own entry is left alone, since a bind missed would read as an unbind.
|
||||
func TestAnUnreadableAssignmentIsSkippedAndSaidAndTheRestGoesOn(t *testing.T) {
|
||||
for name, bad := range map[string]string{"not json": "{not json",
|
||||
"another key": `{"node":"g14","module":"umami"}`} {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s",
|
||||
"pair/novox/postgres-database/g14/umami": "u"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
p := postgres(bus, v, m)
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = bus.Put(context.Background(), AssignmentsBucket, "ace.letta", []byte(bad))
|
||||
assign(t, bus, "g14", "umami", bind("postgres-database", "", "novox", "postgres"))
|
||||
var logged []string
|
||||
p.opts.Logf = func(f string, a ...any) { logged = append(logged, fmt.Sprintf(f, a...)) }
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("%s: %v", name, err)
|
||||
}
|
||||
if bus.get(BucketOf("postgres"), "g14.umami.postgres-database") == nil {
|
||||
t.Fatalf("%s: the good consumer was not answered", name)
|
||||
}
|
||||
if bus.get(BucketOf("postgres"), "ace.letta.postgres-database") == nil || len(m.gone) != 0 {
|
||||
t.Fatalf("%s: the entry of the unreadable assignment's consumer was removed", name)
|
||||
}
|
||||
said := false
|
||||
for _, o := range out {
|
||||
if o.Key == "ace.letta" && o.Status == Failed && o.Why != "" {
|
||||
said = true
|
||||
}
|
||||
}
|
||||
if !said || !strings.Contains(strings.Join(logged, "\n"), "ace.letta") {
|
||||
t.Fatalf("%s: the skipped assignment is not said: %+v, %q", name, out, logged)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An adapter that fails is said with the credential taken out of its words, and the entry is not written.
|
||||
func TestAFailedMakeIsSaidWithoutTheCredential(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{fail: errors.New(`password "hunter2-secret" rejected`)}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "hunter2-secret"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
var logged []string
|
||||
p := postgres(bus, v, m)
|
||||
p.opts.Logf = func(f string, a ...any) { logged = append(logged, fmt.Sprintf(f, a...)) }
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(out) != 1 || out[0].Status != Failed || strings.Contains(out[0].Why, "hunter2") {
|
||||
t.Fatalf("outcomes %+v", out)
|
||||
}
|
||||
if strings.Contains(strings.Join(logged, "\n"), "hunter2") || len(logged) == 0 {
|
||||
t.Fatalf("logged %q", logged)
|
||||
}
|
||||
if bus.keys(BucketOf("postgres")) != nil {
|
||||
t.Fatal("an entry was written for a consumer the adapter could not make")
|
||||
}
|
||||
}
|
||||
|
||||
// Run watches the binds on the stand-in bus: a consumer that binds is answered, and one that unbinds is removed,
|
||||
// without a pass asked by hand.
|
||||
func TestRunAnswersBindsAndUnbindsAsTheyAreWatched(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s"}}
|
||||
p := postgres(bus, v, m)
|
||||
p.opts.Every = time.Hour
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- p.Run(ctx) }()
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
waitFor(t, func() bool { return bus.get(BucketOf("postgres"), "ace.letta.postgres-database") != nil })
|
||||
_ = bus.Delete(context.Background(), AssignmentsBucket, "ace.letta")
|
||||
waitFor(t, func() bool { return bus.get(BucketOf("postgres"), "ace.letta.postgres-database") == nil })
|
||||
cancel()
|
||||
if err := <-done; err != nil && !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func waitFor(t *testing.T, ok func() bool) {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for !ok() {
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("not within 3s")
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
// The consumer's identity is the mesh's (ADR 0202 §1): mesh_<node>_<module>, lower case, `_` for anything else,
|
||||
// and its DNS form writes `-` for `_`.
|
||||
func TestTheIdentityIsTheMeshsAndItsDNSForm(t *testing.T) {
|
||||
if got := ConsumerIdentity("Home.Server", "my-app"); got != "mesh_home_server_my_app" {
|
||||
t.Fatal(got)
|
||||
}
|
||||
got, err := ServedTo(map[string]any{"bucket": "b-${consumer:as:dns}", "n": 3}, "mesh_ace_letta")
|
||||
if err != nil || got["bucket"] != "b-mesh-ace-letta" || got["n"] != 3 {
|
||||
t.Fatalf("%v %v", got, err)
|
||||
}
|
||||
if _, err := ServedTo(map[string]any{"x": "${consumer:node}"}, "mesh_ace_letta"); err == nil {
|
||||
t.Fatal("a fact the mesh does not state was filled")
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer's plan entry may carry what its module contributes to the provider (issue 776): the helper hands it to
|
||||
// the provider's adapter with the consumer, and does not write it to the bucket.
|
||||
func TestAConsumersContributionIsHandedToTheProviderPerConsumer(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{
|
||||
"pair/novox/postgres-database/ace/nextcloud": "a", "pair/novox/postgres-database/g14/umami": "b"}}
|
||||
raw := `{"node":"ace","module":"nextcloud","provisioning":{"templates":{"source":"x","digest":"y"},` +
|
||||
`"provisions":[{"provision":"postgres-database","provider":"novox","provider-module":"postgres",` +
|
||||
`"contributes":{"name":"nextcloud","extensions":["pg_trgm"]}}]}}`
|
||||
if err := bus.Put(context.Background(), AssignmentsBucket, "ace.nextcloud", []byte(raw)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assign(t, bus, "g14", "umami", bind("postgres-database", "", "novox", "postgres"))
|
||||
if _, err := postgres(bus, v, m).Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := map[string]any{"name": "nextcloud", "extensions": []any{"pg_trgm"}}
|
||||
if got := m.contributed["ace.nextcloud.postgres-database"]; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("the adapter was handed the contribution %v, want %v", got, want)
|
||||
}
|
||||
if got := m.contributed["g14.umami.postgres-database"]; got != nil {
|
||||
t.Fatalf("a consumer contributing nothing was handed %v", got)
|
||||
}
|
||||
if bytes.Contains(bus.get(BucketOf("postgres"), "ace.nextcloud.postgres-database"), []byte("pg_trgm")) {
|
||||
t.Fatal("a contribution is in the provider's bucket")
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer's plan entry carries where its provider is, which machine, and who the consumer is (issue 788): the
|
||||
// entry's values carry `at` and `from` as the binding file gives them, and its `as` is the plan's, so a module with a
|
||||
// slug keeps the login it has today. An entry of a plan without them is as it was.
|
||||
func TestAConsumersEntryCarriesAtFromAndTheAsOfItsPlan(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{
|
||||
"pair/novox/postgres-database/ace/claude-code": "a", "pair/novox/postgres-database/g14/umami": "b"}}
|
||||
raw := `{"node":"ace","module":"claude-code","provisioning":{"templates":{"source":"x","digest":"y"},` +
|
||||
`"provisions":[{"provision":"postgres-database","provider":"novox","provider-module":"postgres",` +
|
||||
`"at":"10.7.0.1","from":"novox","as":"mesh_ace_agent"}]}}`
|
||||
if err := bus.Put(context.Background(), AssignmentsBucket, "ace.claude-code", []byte(raw)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assign(t, bus, "g14", "umami", bind("postgres-database", "", "novox", "postgres"))
|
||||
if _, err := postgres(bus, v, m).Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("postgres"), "ace.claude-code.postgres-database"), &e); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e.Values["at"] != "10.7.0.1" || e.Values["from"] != "novox" || e.Values["port"] == nil {
|
||||
t.Fatalf("the entry's values are %v", e.Values)
|
||||
}
|
||||
if e.As != "mesh_ace_agent" || e.User != "mesh_ace_agent" || m.as["ace.claude-code.postgres-database"] != "mesh_ace_agent" {
|
||||
t.Fatalf("the entry is %+v, made as %q", e, m.as["ace.claude-code.postgres-database"])
|
||||
}
|
||||
var plain Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("postgres"), "g14.umami.postgres-database"), &plain); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, has := plain.Values["at"]; has || plain.As != "mesh_g14_umami" {
|
||||
t.Fatalf("an entry of a plan without them changed: %+v", plain)
|
||||
}
|
||||
}
|
||||
|
||||
// A consumer's plan entry may carry the names the controller composed for it (issue 792): each is copied into the
|
||||
// entry's values under its key, over anything the provider serves under it. An entry without them is as it was.
|
||||
func TestAConsumersEntryCarriesTheNamesOfItsPlan(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{
|
||||
"pair/novox/postgres-database/ace/gitea": "a", "pair/novox/postgres-database/g14/umami": "b"}}
|
||||
raw := `{"node":"ace","module":"gitea","provisioning":{"templates":{"source":"x","digest":"y"},` +
|
||||
`"provisions":[{"provision":"postgres-database","provider":"novox","provider-module":"postgres",` +
|
||||
`"names":{"name":"git.example.org","internal-name":"git.mesh","name-web":"web.example.org"}}]}}`
|
||||
if err := bus.Put(context.Background(), AssignmentsBucket, "ace.gitea", []byte(raw)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assign(t, bus, "g14", "umami", bind("postgres-database", "", "novox", "postgres"))
|
||||
if _, err := postgres(bus, v, m).Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("postgres"), "ace.gitea.postgres-database"), &e); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e.Values["name"] != "git.example.org" || e.Values["internal-name"] != "git.mesh" ||
|
||||
e.Values["name-web"] != "web.example.org" || e.Values["port"] == nil {
|
||||
t.Fatalf("the entry's values are %v", e.Values)
|
||||
}
|
||||
var plain Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("postgres"), "g14.umami.postgres-database"), &plain); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, has := plain.Values["name"]; has {
|
||||
t.Fatalf("an entry of a plan without names changed: %+v", plain)
|
||||
}
|
||||
}
|
||||
|
||||
// refusedByTheStateGuard mirrors the key rule of the runtime's state guard (mesh-tools node-tools internal/bus
|
||||
// state.go): a JSON key whose lowercased name, with `-`, `_` and spaces taken out, ends in one of these is refused.
|
||||
func refusedByTheStateGuard(key string) bool {
|
||||
k := strings.NewReplacer("-", "", "_", "", " ", "").Replace(strings.ToLower(key))
|
||||
for _, end := range []string{"password", "passwd", "secret", "token", "credential", "credentials",
|
||||
"authorization", "apikey", "privatekey", "accesskey", "cookie"} {
|
||||
if strings.HasSuffix(k, end) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func keysOf(v any, into *[]string) {
|
||||
switch x := v.(type) {
|
||||
case map[string]any:
|
||||
for k, c := range x {
|
||||
*into = append(*into, k)
|
||||
keysOf(c, into)
|
||||
}
|
||||
case []any:
|
||||
for _, c := range x {
|
||||
keysOf(c, into)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Every entry the helper writes passes the state guard's key rule (issue 945), and names where the vault keeps the
|
||||
// credential under `vault-name`, for a login per consumer and for a shared one.
|
||||
func TestAnEntryHasNoKeyTheStateGuardRefuses(t *testing.T) {
|
||||
for name, form := range map[string]CredentialForm{"pair": PerConsumer, "shared": Shared} {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s1",
|
||||
"provision/novox/postgres-database": "s2"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
p := New(Options{Node: "novox", Module: "postgres", Bus: bus, Vault: v, Make: m.Make, Logf: func(string, ...any) {},
|
||||
Offers: map[string]Offer{"postgres-database": {Credential: form, User: "shared-user"}}})
|
||||
if _, err := p.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err := json.Unmarshal(bus.get(BucketOf("postgres"), "ace.letta.postgres-database"), &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var keys []string
|
||||
keysOf(doc, &keys)
|
||||
for _, k := range keys {
|
||||
if refusedByTheStateGuard(k) {
|
||||
t.Fatalf("%s: the entry carries the key %q, which the state guard refuses", name, k)
|
||||
}
|
||||
}
|
||||
want := map[string]string{"pair": "pair/novox/postgres-database/ace/letta",
|
||||
"shared": "provision/novox/postgres-database"}[name]
|
||||
if doc["vault-name"] != want {
|
||||
t.Fatalf("%s: vault-name is %v, want %s", name, doc["vault-name"], want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The plan's credential form and holder name win over the offer's; the offer's form is the default (issue 825,
|
||||
// finding 4).
|
||||
func TestThePlansCredentialFormWinsOverTheOffers(t *testing.T) {
|
||||
run := func(offer CredentialForm, plan map[string]string) Entry {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "p",
|
||||
"provision/novox/postgres-database": "s", "provision/novox/postgres-database/ace/letta": "h"}}
|
||||
b := bind("postgres-database", "", "novox", "postgres")
|
||||
for k, val := range plan {
|
||||
b[k] = val
|
||||
}
|
||||
assign(t, bus, "ace", "letta", b)
|
||||
p := New(Options{Node: "novox", Module: "postgres", Bus: bus, Vault: v, Make: m.Make, Logf: func(string, ...any) {},
|
||||
Offers: map[string]Offer{"postgres-database": {Credential: offer, User: "shared-user"}}})
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil || len(out) != 1 || out[0].Status != Answered {
|
||||
t.Fatalf("%v %+v", err, out)
|
||||
}
|
||||
return entryOf(t, bus, "ace.letta.postgres-database")
|
||||
}
|
||||
if e := run(PerConsumer, map[string]string{"credential-form": "shared"}); e.VaultName !=
|
||||
"provision/novox/postgres-database" || e.User != "shared-user" {
|
||||
t.Fatalf("shared over pair: %+v", e)
|
||||
}
|
||||
if e := run(Shared, map[string]string{"credential-form": "pair"}); e.VaultName !=
|
||||
"pair/novox/postgres-database/ace/letta" || e.User != "mesh_ace_letta" {
|
||||
t.Fatalf("pair over shared: %+v", e)
|
||||
}
|
||||
for _, form := range []string{"issued", "given"} {
|
||||
if e := run(PerConsumer, map[string]string{"credential-form": form, "name": "main"}); e.VaultName !=
|
||||
"provision/novox/postgres-database/ace/letta" || e.User != "mesh_ace_letta" {
|
||||
t.Fatalf("%s holder: %+v", form, e)
|
||||
}
|
||||
}
|
||||
if e := run(Shared, nil); e.VaultName != "provision/novox/postgres-database" {
|
||||
t.Fatalf("the offer's form is the default: %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// An instance deletes only an entry it wrote, and only while the entry is as it read it: one that another instance of
|
||||
// the provider wrote after the consumer moved stands (issue 825, finding 5).
|
||||
func TestAnEntryAnotherInstanceWroteAfterTheMoveStands(t *testing.T) {
|
||||
bus, m := newStandIn(), &maker{}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": "s"}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
a := postgres(bus, v, m)
|
||||
if _, err := a.Pass(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The consumer moves to the instance on shanks: A's pass reads its own entry and the unbind, and while it lets
|
||||
// the consumer go, B writes the consumer's entry.
|
||||
_ = bus.Delete(context.Background(), AssignmentsBucket, "ace.letta")
|
||||
theirs, _ := json.Marshal(Entry{Provider: "shanks", ProviderModule: "postgres", Provision: "postgres-database",
|
||||
Consumer: "ace", Module: "letta", VaultName: "pair/shanks/postgres-database/ace/letta"})
|
||||
a.opts.Gone = func(ctx context.Context, e Entry) error {
|
||||
return bus.Put(ctx, BucketOf("postgres"), e.Key(), theirs)
|
||||
}
|
||||
out, err := a.Pass(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := bus.get(BucketOf("postgres"), "ace.letta.postgres-database"); !bytes.Equal(got, theirs) {
|
||||
t.Fatalf("the other instance's entry was removed: %s", got)
|
||||
}
|
||||
for _, o := range out {
|
||||
if o.Status == Removed {
|
||||
t.Fatalf("outcomes %+v", out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A failed Make is said with the credential taken out in the forms it can come back in: as it is, in base64 (standard
|
||||
// and URL, with and without padding) and as a JSON string says it (issue 825, finding 6).
|
||||
func TestAFailedMakeIsSaidWithoutTheCredentialInAnyOfItsForms(t *testing.T) {
|
||||
cred := "pa/ss+w\"rd?>>x"
|
||||
b := []byte(cred)
|
||||
jsonQuoted, _ := json.Marshal(cred)
|
||||
forms := map[string]string{
|
||||
"raw": cred, "std": base64.StdEncoding.EncodeToString(b), "std unpadded": base64.RawStdEncoding.EncodeToString(b),
|
||||
"url": base64.URLEncoding.EncodeToString(b), "url unpadded": base64.RawURLEncoding.EncodeToString(b),
|
||||
"json": string(jsonQuoted[1 : len(jsonQuoted)-1]),
|
||||
}
|
||||
if forms["std"] == forms["url"] || forms["json"] == forms["raw"] {
|
||||
t.Fatal("the test credential does not tell the forms apart")
|
||||
}
|
||||
for name, form := range forms {
|
||||
bus, m := newStandIn(), &maker{fail: errors.New("login failed for " + form + " at the server")}
|
||||
v := &vault{values: map[string]string{"pair/novox/postgres-database/ace/letta": cred}}
|
||||
assign(t, bus, "ace", "letta", bind("postgres-database", "", "novox", "postgres"))
|
||||
out, err := postgres(bus, v, m).Pass(context.Background())
|
||||
if err != nil || len(out) != 1 {
|
||||
t.Fatalf("%v %+v", err, out)
|
||||
}
|
||||
if strings.Contains(out[0].Why, form) || !strings.Contains(out[0].Why, "<credential>") {
|
||||
t.Fatalf("%s form: %q", name, out[0].Why)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// An offer that declares no credential is answered with none whatever form the plan says (the controller writes
|
||||
// "pair" for such an offer): no vault name, and the Vault is never asked, so a provider with no Vault works (issue 825,
|
||||
// finding 4; issue 923).
|
||||
func TestAnOfferWithNoCredentialIgnoresThePlansForm(t *testing.T) {
|
||||
bus := newStandIn()
|
||||
b := bind("route", "", "novox", "route-proxy")
|
||||
b["credential-form"] = "pair"
|
||||
assign(t, bus, "ace", "letta", b)
|
||||
p := New(Options{Node: "novox", Module: "route-proxy", Bus: bus, Logf: func(string, ...any) {},
|
||||
Offers: map[string]Offer{"route": {Credential: NoCredential}}})
|
||||
out, err := p.Pass(context.Background())
|
||||
if err != nil || len(out) != 1 || out[0].Status != Answered {
|
||||
t.Fatalf("%v %+v", err, out)
|
||||
}
|
||||
var e Entry
|
||||
if err := json.Unmarshal(bus.get(BucketOf("route-proxy"), "ace.letta.route"), &e); err != nil || e.VaultName != "" ||
|
||||
e.User != "" {
|
||||
t.Fatalf("%v %+v", err, e)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// runtime drives a bundle over pipes as the runtime does: it reads what the bundle says and sends what the runtime would.
|
||||
type runtime struct {
|
||||
t *testing.T
|
||||
lines *bufio.Scanner
|
||||
in io.Writer
|
||||
}
|
||||
|
||||
func (r *runtime) next() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if r.lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(r.lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
r.t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func (r *runtime) send(m map[string]any) {
|
||||
b, _ := json.Marshal(m)
|
||||
_, _ = r.in.Write(append(b, '\n'))
|
||||
}
|
||||
|
||||
func (r *runtime) call(id int, tool string, args map[string]any) {
|
||||
r.send(map[string]any{"jsonrpc": "2.0", "id": id, "method": "tools/call", "params": map[string]any{"name": tool, "arguments": args}})
|
||||
}
|
||||
|
||||
func serve(t *testing.T, tools []stdio.Tool) *runtime {
|
||||
t.Setenv(stdio.ServedAs, "keeper")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() { _ = stdio.ServeOn("", tools, inR, outW) }()
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
return &runtime{t: t, lines: bufio.NewScanner(outR), in: inW}
|
||||
}
|
||||
|
||||
// The bus of a moved provider maps its own bucket to the state `bound` and the assignments to the state
|
||||
// `mesh-controller.assignments`, writes only the own bucket's state, and refuses any other bucket.
|
||||
func TestStateBusMapsTheProvidersBucketsToTheRuntimesStates(t *testing.T) {
|
||||
bus := StateBus("postgres")
|
||||
rt := serve(t, []stdio.Tool{
|
||||
{Name: "watch", Run: func(a map[string]any) (any, error) {
|
||||
return nil, bus.Watch(context.Background(), a["bucket"].(string), func(string, []byte) {})
|
||||
}},
|
||||
{Name: "put", Run: func(map[string]any) (any, error) {
|
||||
return "put", bus.Put(context.Background(), BucketOf("postgres"), "k", []byte(`{"a":1}`))
|
||||
}},
|
||||
{Name: "delete", Run: func(map[string]any) (any, error) {
|
||||
return "deleted", bus.Delete(context.Background(), BucketOf("postgres"), "k")
|
||||
}},
|
||||
})
|
||||
for i, c := range []struct{ bucket, state string }{
|
||||
{BucketOf("postgres"), "bound"},
|
||||
{AssignmentsBucket, "mesh-controller.assignments"},
|
||||
} {
|
||||
rt.call(i+1, "watch", map[string]any{"bucket": c.bucket})
|
||||
w := rt.next()
|
||||
if w["method"] != "mesh/state.watch" || w["params"].(map[string]any)["state"] != c.state {
|
||||
t.Fatalf("bucket %s asked %v, want the state %s", c.bucket, w, c.state)
|
||||
}
|
||||
rt.send(map[string]any{"jsonrpc": "2.0", "id": w["id"], "result": map[string]any{}})
|
||||
rt.next()
|
||||
}
|
||||
rt.call(3, "put", nil)
|
||||
p := rt.next()
|
||||
if p["method"] != "mesh/state.put" || p["params"].(map[string]any)["state"] != "bound" || p["params"].(map[string]any)["key"] != "k" {
|
||||
t.Fatalf("put asked %v", p)
|
||||
}
|
||||
rt.send(map[string]any{"jsonrpc": "2.0", "id": p["id"], "result": map[string]any{"revision": 1}})
|
||||
rt.next()
|
||||
rt.call(4, "delete", nil)
|
||||
d := rt.next()
|
||||
if d["method"] != "mesh/state.delete" || d["params"].(map[string]any)["state"] != "bound" {
|
||||
t.Fatalf("delete asked %v", d)
|
||||
}
|
||||
rt.send(map[string]any{"jsonrpc": "2.0", "id": d["id"], "result": map[string]any{}})
|
||||
rt.next()
|
||||
|
||||
for _, bucket := range []string{"other_bound", "elsewhere"} {
|
||||
if err := bus.Watch(context.Background(), bucket, func(string, []byte) {}); err == nil || !strings.Contains(err.Error(), "not a bucket") {
|
||||
t.Fatalf("a watch of %s gave %v", bucket, err)
|
||||
}
|
||||
if err := bus.Put(context.Background(), bucket, "k", nil); err == nil {
|
||||
t.Fatalf("a put to %s was allowed", bucket)
|
||||
}
|
||||
if err := bus.Delete(context.Background(), bucket, "k"); err == nil {
|
||||
t.Fatalf("a delete in %s was allowed", bucket)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A watch whose ctx has ended calls its handler no more, though the runtime keeps sending changes: the Bus contract
|
||||
// says a watch lasts until ctx ends, and a stdio state watch lasts as long as the process.
|
||||
func TestStateBusWatchCallsItsHandlerNoMoreOnceItsContextEnded(t *testing.T) {
|
||||
bus := StateBus("postgres")
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
handled := make(chan string, 8)
|
||||
rt := serve(t, []stdio.Tool{
|
||||
{Name: "start", Run: func(map[string]any) (any, error) {
|
||||
return "started", bus.Watch(ctx, AssignmentsBucket, func(key string, value []byte) {
|
||||
handled <- key + "=" + string(value)
|
||||
})
|
||||
}},
|
||||
{Name: "stop", Run: func(map[string]any) (any, error) { cancel(); return "stopped", nil }},
|
||||
})
|
||||
change := func(id int, watch any, key, op string) {
|
||||
rt.send(map[string]any{"jsonrpc": "2.0", "id": id, "method": "mesh/state", "params": map[string]any{"watch": watch,
|
||||
"change": map[string]any{"state": "mesh-controller.assignments", "key": key, "op": op, "value": "v"}}})
|
||||
if r := rt.next(); r["id"] != float64(id) || r["error"] != nil {
|
||||
t.Fatalf("a change was answered %v", r)
|
||||
}
|
||||
}
|
||||
rt.call(1, "start", nil)
|
||||
w := rt.next()
|
||||
rt.send(map[string]any{"jsonrpc": "2.0", "id": w["id"], "result": map[string]any{}})
|
||||
rt.next()
|
||||
|
||||
change(50, w["id"], "before", "put")
|
||||
if got := <-handled; got != `before="v"` {
|
||||
t.Fatalf("handled %q", got)
|
||||
}
|
||||
rt.call(2, "stop", nil)
|
||||
rt.next()
|
||||
change(51, w["id"], "after", "put")
|
||||
change(52, w["id"], "after", "delete")
|
||||
select {
|
||||
case got := <-handled:
|
||||
t.Fatalf("the handler was called after its context ended: %q", got)
|
||||
case <-time.After(100 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A module's code reads its values at runtime (novox/hq ADR 0349 clause 14, ADR 0370, and the part of the feature 57 design that lets a running module read, rather than be handed in a file, its provisioned values): its config
|
||||
// values and its bindings' values that are not secret, for what its manifest declares, and watches them. The
|
||||
// runtime that launched this bundle asks its machine's provisioner client **as this bundle's module** — the module it
|
||||
// launched, never a name this code passes — and the client answers only what that module declares, only while the
|
||||
// mesh places it on that machine. A secret is refused while the module's code runs in the tool runner: the vault
|
||||
// hands a secret only to a caller the bus proves, and the bus proves only the tool runner there, so a module's
|
||||
// secrets reach it through its files until it runs under a bus account of its own (novox/hq issue 569).
|
||||
|
||||
// ProvisionChange is what a watch hands over: the value now, or — once it is no longer answered, the module taken
|
||||
// off the machine or the key no longer declared — Refused, why.
|
||||
type ProvisionChange struct {
|
||||
Kind string `json:"kind"`
|
||||
Key string `json:"key"`
|
||||
Value json.RawMessage `json:"value,omitempty"`
|
||||
Refused string `json:"refused,omitempty"`
|
||||
}
|
||||
|
||||
const provisionTimeout = 30 * time.Second
|
||||
|
||||
func provisionGet(kind, key string) (json.RawMessage, error) {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.ask("mesh/provision.get", map[string]any{"kind": kind, "key": key}, provisionTimeout)
|
||||
}
|
||||
|
||||
func provisionWatch(kind, key string, handle func(ProvisionChange) error) error {
|
||||
if handle == nil {
|
||||
return errors.New("a watch needs a handler")
|
||||
}
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id := c.newID()
|
||||
c.mu.Lock()
|
||||
c.provisioned[id] = handle
|
||||
c.mu.Unlock()
|
||||
if _, err := c.askAs(id, "mesh/provision.watch", map[string]any{"kind": kind, "key": key}, provisionTimeout); err != nil {
|
||||
c.mu.Lock()
|
||||
delete(c.provisioned, id)
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Config is one of this module's config values as mesh-config resolved it for this machine, as JSON: the key as
|
||||
// its manifest declares it under `config`. A key it does not declare is refused, with the reason.
|
||||
func Config(key string) (json.RawMessage, error) { return provisionGet("config", key) }
|
||||
|
||||
// WatchConfig hands handle the config value now and then every change, in order, and returns once the value now is
|
||||
// handled. The watch lasts as long as the process.
|
||||
func WatchConfig(key string, handle func(ProvisionChange) error) error {
|
||||
return provisionWatch("config", key, handle)
|
||||
}
|
||||
|
||||
// Bound is one value, not a secret, of what this module is bound to: the provision as its manifest requires it,
|
||||
// and the key its provider answers (`${bound:<provision>:<key>}`).
|
||||
func Bound(provision, key string) (json.RawMessage, error) {
|
||||
return provisionGet("bound", provision+":"+key)
|
||||
}
|
||||
|
||||
// WatchBound is WatchConfig for a bound value.
|
||||
func WatchBound(provision, key string, handle func(ProvisionChange) error) error {
|
||||
return provisionWatch("bound", provision+":"+key, handle)
|
||||
}
|
||||
|
||||
// Secret is one of this module's secrets. Refused, with the reason, while the module's code runs in the tool
|
||||
// runner (novox/hq ADR 0349 §11, ADR 0370 §2): its secrets reach it through its files.
|
||||
func Secret(name string) ([]byte, error) {
|
||||
raw, err := provisionGet("secret", name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var s string
|
||||
if err := json.Unmarshal(raw, &s); err != nil {
|
||||
return nil, errors.New("the runtime answered a secret that is not a string")
|
||||
}
|
||||
return []byte(s), nil
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A Go bundle reads its values at runtime through the runtime (novox/hq ADR 0349 §14, ADR 0370, issue 685): Config,
|
||||
// Bound and their watches ask the runtime, which asks as the module; a watch's value now is handled before it
|
||||
// returns and each change after is named by the watch's id; what the module does not declare and every secret come
|
||||
// back refused, in the runtime's words; a change for no watch of this bundle is refused.
|
||||
func TestABundleReadsAndWatchesItsValuesThroughTheRuntime(t *testing.T) {
|
||||
t.Setenv(ServedAs, "claude-code")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
seen := make(chan string, 16)
|
||||
watched := make(chan error, 1)
|
||||
answers := make(chan string, 4)
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{
|
||||
{Name: "start", Run: func(map[string]any) (any, error) {
|
||||
go func() {
|
||||
watched <- WatchConfig("managed_settings", func(c ProvisionChange) error {
|
||||
seen <- c.Kind + " " + c.Key + " " + string(c.Value) + c.Refused
|
||||
return nil
|
||||
})
|
||||
}()
|
||||
return "started", nil
|
||||
}},
|
||||
{Name: "read", Run: func(a map[string]any) (any, error) {
|
||||
var v json.RawMessage
|
||||
var err error
|
||||
switch a["what"] {
|
||||
case "config":
|
||||
v, err = Config(a["key"].(string))
|
||||
case "bound":
|
||||
v, err = Bound("mcp-endpoint", "url")
|
||||
case "secret":
|
||||
var s []byte
|
||||
s, err = Secret("gitea-read")
|
||||
v = json.RawMessage(s)
|
||||
}
|
||||
if err != nil {
|
||||
answers <- "refused: " + err.Error()
|
||||
} else {
|
||||
answers <- string(v)
|
||||
}
|
||||
return "read", nil
|
||||
}},
|
||||
}, inR, outW)
|
||||
}()
|
||||
lines := bufio.NewScanner(outR)
|
||||
next := func() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
// asked is the bundle's next ask of the runtime, beside its answers to this tool call and those before.
|
||||
asked := func(callID int) map[string]any {
|
||||
t.Helper()
|
||||
var ask map[string]any
|
||||
for ask == nil {
|
||||
m := next()
|
||||
if m["method"] != nil {
|
||||
ask = m
|
||||
} else if id, ok := m["id"].(float64); !ok || id > float64(callID) || m["error"] != nil {
|
||||
t.Fatalf("said %v", m)
|
||||
}
|
||||
}
|
||||
return ask
|
||||
}
|
||||
|
||||
// The watch: asked of the runtime with the kind and the key and no module, the module being the runtime's word.
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "start"}})
|
||||
watch := asked(1)
|
||||
if watch["method"] != "mesh/provision.watch" {
|
||||
t.Fatalf("asked %v", watch)
|
||||
}
|
||||
if p := watch["params"].(map[string]any); p["kind"] != "config" || p["key"] != "managed_settings" || len(p) != 2 {
|
||||
t.Fatalf("asked to watch %v", p)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 50, "method": "mesh/provision", "params": map[string]any{"watch": watch["id"],
|
||||
"change": map[string]any{"kind": "config", "key": "managed_settings", "value": map[string]any{"spinnerTipsEnabled": false}}}})
|
||||
for {
|
||||
if r := next(); r["id"] == float64(50) {
|
||||
if r["error"] != nil {
|
||||
t.Fatalf("the value now was answered %v", r)
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
if s := <-seen; s != `config managed_settings {"spinnerTipsEnabled":false}` {
|
||||
t.Fatalf("handled %q", s)
|
||||
}
|
||||
select {
|
||||
case err := <-watched:
|
||||
t.Fatalf("the watch returned before it was answered: %v", err)
|
||||
default:
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": watch["id"], "result": map[string]any{}})
|
||||
if err := <-watched; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A change, and then the value no longer answered.
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 51, "method": "mesh/provision", "params": map[string]any{"watch": watch["id"],
|
||||
"change": map[string]any{"kind": "config", "key": "managed_settings", "value": map[string]any{"spinnerTipsEnabled": true}}}})
|
||||
if r := next(); r["id"] != float64(51) || r["error"] != nil {
|
||||
t.Fatalf("a change was answered %v", r)
|
||||
}
|
||||
if s := <-seen; s != `config managed_settings {"spinnerTipsEnabled":true}` {
|
||||
t.Fatalf("handled %q", s)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 52, "method": "mesh/provision", "params": map[string]any{"watch": watch["id"],
|
||||
"change": map[string]any{"kind": "config", "key": "managed_settings", "refused": "claude-code is not placed on g14"}}})
|
||||
if r := next(); r["id"] != float64(52) || r["error"] != nil {
|
||||
t.Fatalf("a refusal was answered %v", r)
|
||||
}
|
||||
if s := <-seen; s != `config managed_settings claude-code is not placed on g14` {
|
||||
t.Fatalf("handled %q", s)
|
||||
}
|
||||
// A change for a watch this bundle never asked for is refused.
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 53, "method": "mesh/provision", "params": map[string]any{"watch": "bundle-999",
|
||||
"change": map[string]any{"kind": "config", "key": "x"}}})
|
||||
if r := next(); r["id"] != float64(53) || !strings.Contains(r["error"].(map[string]any)["message"].(string), "names no watch") {
|
||||
t.Fatalf("a change for no watch was answered %v", r)
|
||||
}
|
||||
|
||||
// Each reader asks the runtime, and hands back its answer or its refusal.
|
||||
for i, c := range []struct {
|
||||
what, key, kind, askedKey string
|
||||
result any
|
||||
refusal string
|
||||
want string
|
||||
}{
|
||||
{what: "config", key: "role", kind: "config", askedKey: "role", result: "the build machine", want: `"the build machine"`},
|
||||
{what: "config", key: "nothing", kind: "config", askedKey: "nothing",
|
||||
refusal: "claude-code declares no config value nothing", want: "refused: claude-code declares no config value nothing"},
|
||||
{what: "bound", kind: "bound", askedKey: "mcp-endpoint:url", result: "http://127.0.0.1:9000", want: `"http://127.0.0.1:9000"`},
|
||||
{what: "secret", kind: "secret", askedKey: "gitea-read",
|
||||
refusal: "the vault hands a secret only to a caller the bus proves", want: "refused: the vault hands a secret only to a caller the bus proves"},
|
||||
} {
|
||||
id := 10 + i
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": id, "method": "tools/call", "params": map[string]any{"name": "read",
|
||||
"arguments": map[string]any{"what": c.what, "key": c.key}}})
|
||||
get := asked(id)
|
||||
if p := get["params"].(map[string]any); get["method"] != "mesh/provision.get" || p["kind"] != c.kind || p["key"] != c.askedKey || len(p) != 2 {
|
||||
t.Fatalf("%s: asked %v", c.what, get)
|
||||
}
|
||||
if c.refusal != "" {
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": get["id"], "error": map[string]any{"code": -32000, "message": c.refusal}})
|
||||
} else {
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": get["id"], "result": c.result})
|
||||
}
|
||||
if got := <-answers; got != c.want {
|
||||
t.Errorf("%s %s: %q, want %q", c.what, c.key, got, c.want)
|
||||
}
|
||||
}
|
||||
_ = inW.Close()
|
||||
}
|
||||
+203
@@ -0,0 +1,203 @@
|
||||
// Package seal is the mesh's sealed box (novox/hq ADR 0183, ADR 0206): a value sealed to one recipient's
|
||||
// public key, which only that recipient's private key opens. X25519 for the agreement, HKDF-SHA256 for the
|
||||
// key, AES-256-GCM for the box — the box the licence manager made and opened before it lived here, byte
|
||||
// for byte, and the one the SDK's TypeScript `seal` makes and opens.
|
||||
//
|
||||
// A box is `{ v: 1, eph, iv, tag, ct }`, every field standard base64; `eph` is the one-time public key as
|
||||
// SPKI DER. The key is derived with the salt eph-DER ‖ the recipient's raw public key and the info
|
||||
// "novox-mesh sealed box v1", so a box cannot be re-addressed to another key.
|
||||
//
|
||||
// Keys travel as PEM: the public key as SPKI ("PUBLIC KEY"), the private key as PKCS #8 ("PRIVATE KEY").
|
||||
// Standard library only.
|
||||
package seal
|
||||
|
||||
import (
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/ecdh"
|
||||
"crypto/hmac"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/x509"
|
||||
"encoding/base64"
|
||||
"encoding/pem"
|
||||
"errors"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// Box is a value sealed to one recipient, as it travels in JSON.
|
||||
type Box struct {
|
||||
V int `json:"v"`
|
||||
Eph string `json:"eph"`
|
||||
IV string `json:"iv"`
|
||||
Tag string `json:"tag"`
|
||||
Ct string `json:"ct"`
|
||||
}
|
||||
|
||||
// KeyPair is a recipient's keypair as the two PEM strings it is kept and sent as.
|
||||
type KeyPair struct {
|
||||
PublicKey string `json:"publicKey"`
|
||||
PrivateKey string `json:"privateKey"`
|
||||
}
|
||||
|
||||
// Version is the only box version Open opens.
|
||||
const Version = 1
|
||||
|
||||
const info = "novox-mesh sealed box v1"
|
||||
|
||||
// GenerateKeyPair makes a fresh X25519 keypair, PEM-encoded.
|
||||
func GenerateKeyPair() (KeyPair, error) {
|
||||
priv, err := ecdh.X25519().GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
return KeyPair{}, err
|
||||
}
|
||||
pubDER, err := x509.MarshalPKIXPublicKey(priv.PublicKey())
|
||||
if err != nil {
|
||||
return KeyPair{}, err
|
||||
}
|
||||
privDER, err := x509.MarshalPKCS8PrivateKey(priv)
|
||||
if err != nil {
|
||||
return KeyPair{}, err
|
||||
}
|
||||
return KeyPair{
|
||||
PublicKey: string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: pubDER})),
|
||||
PrivateKey: string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privDER})),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Seal seals plaintext to the recipient's PEM public key, with a fresh one-time key and iv.
|
||||
func Seal(plaintext, recipientPEM string) (Box, error) {
|
||||
eph, err := ecdh.X25519().GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
iv := make([]byte, 12)
|
||||
if _, err := rand.Read(iv); err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
return sealWith(plaintext, recipientPEM, eph, iv)
|
||||
}
|
||||
|
||||
// sealWith is Seal with its two sources of randomness given, for the byte-for-byte tests.
|
||||
func sealWith(plaintext, recipientPEM string, eph *ecdh.PrivateKey, iv []byte) (Box, error) {
|
||||
recipient, err := publicFromPEM(recipientPEM)
|
||||
if err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
secret, err := eph.ECDH(recipient)
|
||||
if err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
ephDER, err := x509.MarshalPKIXPublicKey(eph.PublicKey())
|
||||
if err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
gcm, err := newGCM(boxKey(secret, ephDER, recipient.Bytes()))
|
||||
if err != nil {
|
||||
return Box{}, err
|
||||
}
|
||||
out := gcm.Seal(nil, iv, []byte(plaintext), nil)
|
||||
ct, tag := out[:len(out)-gcm.Overhead()], out[len(out)-gcm.Overhead():]
|
||||
b64 := base64.StdEncoding.EncodeToString
|
||||
return Box{V: Version, Eph: b64(ephDER), IV: b64(iv), Tag: b64(tag), Ct: b64(ct)}, nil
|
||||
}
|
||||
|
||||
// Open opens a box with the recipient's PEM private key. It fails for a box sealed to another key, one
|
||||
// changed in any field, or one of another version.
|
||||
func Open(box Box, privatePEM string) (string, error) {
|
||||
if box.V != Version {
|
||||
return "", fmt.Errorf("a sealed box of version %d, not %d", box.V, Version)
|
||||
}
|
||||
priv, err := privateFromPEM(privatePEM)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
d := base64.StdEncoding.DecodeString
|
||||
ephDER, err := d(box.Eph)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("the box's eph: %w", err)
|
||||
}
|
||||
ephKey, err := x509.ParsePKIXPublicKey(ephDER)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("the box's eph: %w", err)
|
||||
}
|
||||
eph, ok := ephKey.(*ecdh.PublicKey)
|
||||
if !ok || eph.Curve() != ecdh.X25519() {
|
||||
return "", errors.New("the box's eph is not an X25519 key")
|
||||
}
|
||||
secret, err := priv.ECDH(eph)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
iv, err1 := d(box.IV)
|
||||
tag, err2 := d(box.Tag)
|
||||
ct, err3 := d(box.Ct)
|
||||
if err := errors.Join(err1, err2, err3); err != nil {
|
||||
return "", fmt.Errorf("the box's fields: %w", err)
|
||||
}
|
||||
gcm, err := newGCM(boxKey(secret, ephDER, priv.PublicKey().Bytes()))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(iv) != gcm.NonceSize() || len(tag) != gcm.Overhead() {
|
||||
return "", errors.New("the box does not open with this key")
|
||||
}
|
||||
plain, err := gcm.Open(nil, iv, append(append([]byte{}, ct...), tag...), nil)
|
||||
if err != nil {
|
||||
return "", errors.New("the box does not open with this key")
|
||||
}
|
||||
return string(plain), nil
|
||||
}
|
||||
|
||||
func publicFromPEM(p string) (*ecdh.PublicKey, error) {
|
||||
block, _ := pem.Decode([]byte(p))
|
||||
if block == nil {
|
||||
return nil, errors.New("not a PEM public key")
|
||||
}
|
||||
k, err := x509.ParsePKIXPublicKey(block.Bytes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pub, ok := k.(*ecdh.PublicKey)
|
||||
if !ok || pub.Curve() != ecdh.X25519() {
|
||||
return nil, errors.New("not an X25519 public key")
|
||||
}
|
||||
return pub, nil
|
||||
}
|
||||
|
||||
func privateFromPEM(p string) (*ecdh.PrivateKey, error) {
|
||||
block, _ := pem.Decode([]byte(p))
|
||||
if block == nil {
|
||||
return nil, errors.New("not a PEM private key")
|
||||
}
|
||||
k, err := x509.ParsePKCS8PrivateKey(block.Bytes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
priv, ok := k.(*ecdh.PrivateKey)
|
||||
if !ok || priv.Curve() != ecdh.X25519() {
|
||||
return nil, errors.New("not an X25519 private key")
|
||||
}
|
||||
return priv, nil
|
||||
}
|
||||
|
||||
// boxKey is HKDF-SHA256 (RFC 5869) for one 32-byte block: written out because crypto/hkdf needs Go 1.24
|
||||
// and this module promises 1.22; the vectors prove it equals the licence manager's hkdf.Key.
|
||||
func boxKey(secret, ephDER, recipientRaw []byte) []byte {
|
||||
salt := append(append([]byte{}, ephDER...), recipientRaw...)
|
||||
extract := hmac.New(sha256.New, salt)
|
||||
extract.Write(secret)
|
||||
prk := extract.Sum(nil)
|
||||
expand := hmac.New(sha256.New, prk)
|
||||
expand.Write([]byte(info))
|
||||
expand.Write([]byte{1})
|
||||
return expand.Sum(nil)
|
||||
}
|
||||
|
||||
func newGCM(key []byte) (cipher.AEAD, error) {
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return cipher.NewGCM(block)
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
package seal
|
||||
|
||||
import (
|
||||
"crypto/ecdh"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type fixture struct {
|
||||
Recipient KeyPair `json:"recipient"`
|
||||
Sealed []struct {
|
||||
Plaintext string `json:"plaintext"`
|
||||
Box Box `json:"box"`
|
||||
} `json:"sealed"`
|
||||
Vectors []struct {
|
||||
Plaintext string `json:"plaintext"`
|
||||
EphPrivate string `json:"ephPrivate"`
|
||||
IV string `json:"iv"`
|
||||
Box Box `json:"box"`
|
||||
} `json:"vectors"`
|
||||
}
|
||||
|
||||
func readFixture(t *testing.T) fixture {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("testdata/sealed-by-licence-manager.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var f fixture
|
||||
if err := json.Unmarshal(raw, &f); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(f.Sealed) == 0 || len(f.Vectors) == 0 {
|
||||
t.Fatal("the fixture holds no boxes")
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
// Every box the licence manager's own Seal made opens here to its plaintext.
|
||||
func TestABoxTheLicenceManagerSealedOpens(t *testing.T) {
|
||||
f := readFixture(t)
|
||||
for _, s := range f.Sealed {
|
||||
got, err := Open(s.Box, f.Recipient.PrivateKey)
|
||||
if err != nil || got != s.Plaintext {
|
||||
t.Fatalf("opened %q, %v; want %q", got, err, s.Plaintext)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Given the same one-time key and iv, this package seals byte for byte what the licence manager sealed:
|
||||
// the reverse direction, so whatever opens the licence manager's boxes opens these.
|
||||
func TestSealIsByteForByteTheLicenceManagersBox(t *testing.T) {
|
||||
f := readFixture(t)
|
||||
for _, v := range f.Vectors {
|
||||
d := base64.StdEncoding.DecodeString
|
||||
ephRaw, err1 := d(v.EphPrivate)
|
||||
iv, err2 := d(v.IV)
|
||||
if err1 != nil || err2 != nil {
|
||||
t.Fatal(err1, err2)
|
||||
}
|
||||
eph, err := ecdh.X25519().NewPrivateKey(ephRaw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := sealWith(v.Plaintext, f.Recipient.PublicKey, eph, iv)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != v.Box {
|
||||
t.Fatalf("sealed %+v\nthe licence manager sealed %+v", got, v.Box)
|
||||
}
|
||||
if plain, err := Open(got, f.Recipient.PrivateKey); err != nil || plain != v.Plaintext {
|
||||
t.Fatalf("reopened %q, %v", plain, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A box the agent module's TypeScript sealed (the fixture the licence manager keeps) opens here too.
|
||||
func TestABoxSealedInTypeScriptOpens(t *testing.T) {
|
||||
raw, err := os.ReadFile("testdata/sealed-by-typescript.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var f struct {
|
||||
PrivateKey string `json:"privateKey"`
|
||||
Box Box `json:"box"`
|
||||
Plaintext string `json:"plaintext"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &f); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, err := Open(f.Box, f.PrivateKey); err != nil || got != f.Plaintext {
|
||||
t.Fatalf("opened %q, %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
// What Seal makes opens with its recipient's key and no other, and a fresh box differs every time.
|
||||
func TestABoxOpensOnlyForItsRecipient(t *testing.T) {
|
||||
a, err := GenerateKeyPair()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b, _ := GenerateKeyPair()
|
||||
box, err := Seal("a token", a.PublicKey)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, err := Open(box, a.PrivateKey); err != nil || got != "a token" {
|
||||
t.Fatalf("opened %q, %v", got, err)
|
||||
}
|
||||
if _, err := Open(box, b.PrivateKey); err == nil {
|
||||
t.Fatal("a box opened for another key")
|
||||
}
|
||||
again, _ := Seal("a token", a.PublicKey)
|
||||
if again.Eph == box.Eph || again.IV == box.IV {
|
||||
t.Fatal("two boxes share a one-time key or iv")
|
||||
}
|
||||
}
|
||||
|
||||
// A box changed in any field, or of another version, does not open.
|
||||
func TestATamperedBoxDoesNotOpen(t *testing.T) {
|
||||
f := readFixture(t)
|
||||
good := f.Sealed[1].Box
|
||||
flip := func(s string) string {
|
||||
b, _ := base64.StdEncoding.DecodeString(s)
|
||||
b[len(b)-1] ^= 1
|
||||
return base64.StdEncoding.EncodeToString(b)
|
||||
}
|
||||
for name, box := range map[string]Box{
|
||||
"ct": {V: 1, Eph: good.Eph, IV: good.IV, Tag: good.Tag, Ct: flip(good.Ct)},
|
||||
"tag": {V: 1, Eph: good.Eph, IV: good.IV, Tag: flip(good.Tag), Ct: good.Ct},
|
||||
"iv": {V: 1, Eph: good.Eph, IV: flip(good.IV), Tag: good.Tag, Ct: good.Ct},
|
||||
"eph": {V: 1, Eph: flip(good.Eph), IV: good.IV, Tag: good.Tag, Ct: good.Ct},
|
||||
"version": {V: 2, Eph: good.Eph, IV: good.IV, Tag: good.Tag, Ct: good.Ct},
|
||||
} {
|
||||
if _, err := Open(box, f.Recipient.PrivateKey); err == nil {
|
||||
t.Fatalf("a box with a changed %s opened", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A key that is not X25519 PEM is refused, not misread.
|
||||
func TestAKeyOfAnotherKindIsRefused(t *testing.T) {
|
||||
if _, err := Seal("x", "not a key"); err == nil {
|
||||
t.Fatal("sealed to text")
|
||||
}
|
||||
f := readFixture(t)
|
||||
if _, err := Open(f.Sealed[0].Box, f.Recipient.PublicKey); err == nil {
|
||||
t.Fatal("opened with a public key")
|
||||
}
|
||||
if _, err := Seal("x", f.Recipient.PrivateKey); err == nil {
|
||||
t.Fatal("sealed to a private key")
|
||||
}
|
||||
}
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
{
|
||||
"about": "Made by the claude-licence-manager's seal.go (mesh-catalog feat/57-config-and-provisioning at 8aa37e8), unchanged but for its two sources of randomness, which the vectors fix. The recipient is a test key derived from a fixed label; it guards nothing.",
|
||||
"recipient": {
|
||||
"publicKey": "-----BEGIN PUBLIC KEY-----\nMCowBQYDK2VuAyEAkZXnghyfA1BR5rWtvBLmjdZMCbC99xxF3WKHe1E+3XM=\n-----END PUBLIC KEY-----\n",
|
||||
"privateKey": "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VuBCIEIHjV954xXPa72FyGUw5PsWCbsHrliVp/7xoAgG9fGs7q\n-----END PRIVATE KEY-----\n"
|
||||
},
|
||||
"sealed": [
|
||||
{
|
||||
"plaintext": "",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAwN2gt32hv+9VyFh0oQ8oetPBHsZ9dt80DzwPwcB0bHg=",
|
||||
"iv": "0xHCegxP6o48EGLq",
|
||||
"tag": "VYQZm/lBZB0YMPUqYVPKaQ==",
|
||||
"ct": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "a grant sealed by the licence manager",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEA54v96LEb8SwT2uJZCa90/2jcLvEbK/BZpbMS6TxAAW0=",
|
||||
"iv": "xm6Jg4oOnRpRpDGK",
|
||||
"tag": "cmbOzIV63Nv0lPfpTgP2kg==",
|
||||
"ct": "InWe8PZ8hkd7F9QaHujjhdiBECKzLAK22d2aIEXwBhtxIowc/Q=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "sk-ant-api03-test-0123456789abcdefghij\n",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAIXdum2FYLzvLDj/Of2253hH8Lu0szU4tLxJzfXTahDs=",
|
||||
"iv": "KMPNa60C/Qjhtp0Z",
|
||||
"tag": "PPa82ZdFyWz9kbLSSGkl3g==",
|
||||
"ct": "Phu3ZsxIWGlBWbYOGzPopncEbc0VhTAloxzeHs4KDBMFZU9Pet2V"
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "ünïcødé ✓ and a NUL \u0000 inside",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAo1WaRyrAJFM6If/cpFgdpZjsC6XebuJtVSugbI+m8RU=",
|
||||
"iv": "H9+WvM6ZYbCYJj6D",
|
||||
"tag": "K5p986nSpeItYK1m4bcVPQ==",
|
||||
"ct": "eyZs3uVeZguNbqSmrMc1rWgxhW3nP0GABH6Ek06/8UuBqw=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; ",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAzjZdTMu08I0Z+FWXSr9f++EADfqg3qr2FVt+3Q/hxm0=",
|
||||
"iv": "lTG2VwPA5SwHbZE/",
|
||||
"tag": "Il6ICxoZgZr9dOgeIZOPrg==",
|
||||
"ct": "ZWQ8m1nxrqq0oIGRoeNlUu9/JpTMrxlncTCL5cDqsJrx9NjS0nZX292JJ5p0+BYSVLUN+yGUuBOnH860ywvKK7XhPOc0Yo2nq7X2O3OYn9Za4y+HK6dVDM8zhsS8P3Di/MJCwS+AnremVCaj2k3xHL3oc331RBZZ5w2JfgSX3ouwl2+VbIPyJVSDsxnLLoHsiIR7nB0GNCe0fynhVkoIEwC8IXLnL6m3rjjN3YCQDmVP2UoftniLAypd5ioc9GCtpvUL6ktjs82JkvEtFbHtc+YzIjP6ZnjTj/bvbRHkIqN4MDiu+fvGOr0jtGWO24ecGC+6KtrKGOaEYHDqOLaK3c+UcrDsFYC+cuKCdCNhs5UNhYYylVXE648KFa488eFk"
|
||||
}
|
||||
}
|
||||
],
|
||||
"vectors": [
|
||||
{
|
||||
"plaintext": "",
|
||||
"ephPrivate": "W8iT0SrRDnazANHYsXHC72sM1wzx+/Y8vowREeGGz/A=",
|
||||
"iv": "DD7WV6Y8hLGvaG0b",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAdCCqlFhxbKBpv1JClXceH43AqEvcaM6SGfyXx05UJUs=",
|
||||
"iv": "DD7WV6Y8hLGvaG0b",
|
||||
"tag": "ctb+ZQjx8FyynvT88rfwkQ==",
|
||||
"ct": ""
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "a grant sealed by the licence manager",
|
||||
"ephPrivate": "lzIbkYmIbkKTYDL4dZiAFUFv2h+GWBiqlQfnsVD2An8=",
|
||||
"iv": "xz2FIBXNM3scOAJb",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAS3L49p4KrGMo/NmHZq8osoPbk2ZCQWLoNIKnpB4T5hg=",
|
||||
"iv": "xz2FIBXNM3scOAJb",
|
||||
"tag": "7dC8bYqzsJ/17EC7y6lhCA==",
|
||||
"ct": "FEMfh4HMvMwqx4QE4YMMR3d3g5rNuxa/+u6u+i/aRLed3q8kBw=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "sk-ant-api03-test-0123456789abcdefghij\n",
|
||||
"ephPrivate": "tAjRQTQHk6nNJLblzdslDbWmyYm8cE6/G4ipXHR0zlE=",
|
||||
"iv": "BPF1JDQ5mDNhKKr3",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEA1GianODxSXUHYMdUmZLcQztp5ry2/xtwUSeHZWrGFAo=",
|
||||
"iv": "BPF1JDQ5mDNhKKr3",
|
||||
"tag": "uiIIFD7LgPmLV3Y6uleSzg==",
|
||||
"ct": "+maa0GDDOkcyzD0chFQ4a/0NvBrvl23NVqcGv1BEKgmMUenf9HHd"
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "ünïcødé ✓ and a NUL \u0000 inside",
|
||||
"ephPrivate": "QBYUxuk4jwiWKOPEWeE7v6WxY8BDmZZCDwlY1WJ1PSA=",
|
||||
"iv": "u76JO8baqokfnjcK",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAePakOoCmmXLtnN3EykbpqfdbfXpkg04F2djxnJWH220=",
|
||||
"iv": "u76JO8baqokfnjcK",
|
||||
"tag": "sfBPRnwhfP/p7vlsUx+e9w==",
|
||||
"ct": "B9yc0FbdKYDtDeX1jmKa+DBP8fPN1O9SbeYBDn9b3+Bq2Q=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"plaintext": "a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; a longer value, past one AES block; ",
|
||||
"ephPrivate": "2VYOyzqLrb+zWUhVpHA9aV3vMAyRWlNPEq2ASf60F9M=",
|
||||
"iv": "0Gv6ejq07LwbM8cY",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEARSOBczrJ6X1MkxT7Mrd1ryDLmXAMBgYIgpfwBjHz1GA=",
|
||||
"iv": "0Gv6ejq07LwbM8cY",
|
||||
"tag": "Vm+E0myXgdMwA6knJDg/+w==",
|
||||
"ct": "tJwZpUXeQYOu2PG1g9AwIV8RhKKDorUHWQSygEus3xHOpuTwh3+D4j2htIerMB/3AKToQYABeZMyEAOzhL2x+Cg06WlUKrggZjvdX5W7X813KQBgzg0YnnCZTKTYlviDWe1E9TZ3aAauhlcgpBaQXUh/akkDiqeq2Ua5ruY6IGH5HgqWWPYJo3b4zkbArhvdP2HJX9Pn1mQa0BtGMinyutyD0am5vZuemdKGYS1Ao00M81sMXYc2CBy9K48Ofw8GapcmYNHsE0s6NSgNZumnVaR8TQRy/oqEdL3Q6z31hAZMzTdS0ckXg1NCgcgXVuzwNov/D4320E3DxFS3v8Z5p74ZkqAmo/1JcCNJwwQyRCWk/4iJFjP1OKEavOGo+bcp"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"privateKey": "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VuBCIEIAi2NK/bN+p7cqYUwv/kz72TgLdmJUfOHCDZTrMpQzpS\n-----END PRIVATE KEY-----\n",
|
||||
"publicKey": "-----BEGIN PUBLIC KEY-----\nMCowBQYDK2VuAyEARYvD/w+9ah0KWS9T9pd6Ea6CBymUE48vEVk983QPKyY=\n-----END PUBLIC KEY-----\n",
|
||||
"box": {
|
||||
"v": 1,
|
||||
"eph": "MCowBQYDK2VuAyEAIYlSGrJvF8qSjR1aTFWbEQM/NFbZXrarglN0aRLZZ0E=",
|
||||
"iv": "ABqT/hMukn6+xpjh",
|
||||
"tag": "POzmsWTPHSn9xLMMJJ9Akg==",
|
||||
"ct": "m2u0kuQ0PwrsGelM99Z5aBw6FCd6lFISUbwiK5TzzDw4ZrGtsHEvxytoIeFC"
|
||||
},
|
||||
"plaintext": "a grant sealed by the TypeScript agent module"
|
||||
}
|
||||
+153
@@ -0,0 +1,153 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Seats that name their caller or their kind (novox/hq ADR 0259 §3): a module submits an accept or says an
|
||||
// event under its own name or kind, takes a held seat's work, asks a proof and answers one, and reads a
|
||||
// record a seat keeps for it. The runtime does each as the module, only as far as its membership lists,
|
||||
// and refuses the rest with the reason.
|
||||
|
||||
// Work is one piece of a seat's work this bundle takes: the subject it was submitted on, its body, and the
|
||||
// headers the submitter's runtime set (x-source is the submitting module, x-event-id its id).
|
||||
type Work struct {
|
||||
Worker string `json:"worker,omitempty"`
|
||||
Subject string `json:"subject"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
Headers map[string]string `json:"headers,omitempty"`
|
||||
}
|
||||
|
||||
const seatTimeout = 30 * time.Second
|
||||
|
||||
// SeatPublish submits an accept, or says an event, on a seat, de-duplicated by id: the same id twice is
|
||||
// one message. An empty id is the runtime's to make.
|
||||
func SeatPublish(subject string, body any, id string) error {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = c.ask("mesh/seat.publish", map[string]any{"subject": subject, "body": body, "id": id}, seatTimeout)
|
||||
return err
|
||||
}
|
||||
|
||||
// SeatProve asks a proof — core request and reply, never kept — and returns its answer.
|
||||
func SeatProve(subject string, body any) (json.RawMessage, error) {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.ask("mesh/seat.prove", map[string]any{"subject": subject, "body": body}, seatTimeout)
|
||||
}
|
||||
|
||||
// SeatAnswer answers every proof on a subject (a pattern, `*` one token) with what handle returns.
|
||||
func SeatAnswer(subject string, handle func(subject string, body json.RawMessage) (any, error)) error {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.mu.Lock()
|
||||
c.provers[subject] = handle
|
||||
c.mu.Unlock()
|
||||
_, err = c.ask("mesh/seat.answer", map[string]any{"subject": subject}, seatTimeout)
|
||||
return err
|
||||
}
|
||||
|
||||
// SeatTake takes a held seat's work from a worker, handing each piece to handle: taken when it returns
|
||||
// nil, offered again after an error. Make the work safe to do twice, or skip an x-event-id already done.
|
||||
func SeatTake(worker string, handle func(Work) error) error {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.mu.Lock()
|
||||
c.takers[worker] = handle
|
||||
c.mu.Unlock()
|
||||
_, err = c.ask("mesh/seat.take", map[string]any{"worker": worker}, seatTimeout)
|
||||
return err
|
||||
}
|
||||
|
||||
// SeatRecord reads one key of a record a seat keeps for this module, under its own name, or nil.
|
||||
func SeatRecord(bucket, key string) (*StateEntry, error) {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
raw, err := c.ask("mesh/seat.record", map[string]any{"bucket": bucket, "key": key}, seatTimeout)
|
||||
if err != nil || len(raw) == 0 || string(raw) == "null" {
|
||||
return nil, err
|
||||
}
|
||||
var e StateEntry
|
||||
return &e, json.Unmarshal(raw, &e)
|
||||
}
|
||||
|
||||
// KindHeld is one kind of a kinded bench and who holds it, with what it promises (novox/hq ADR 0259 §5).
|
||||
type KindHeld struct {
|
||||
Seat string `json:"seat"`
|
||||
Kind string `json:"kind"`
|
||||
Module string `json:"module"`
|
||||
Node string `json:"node"`
|
||||
Capabilities []string `json:"capabilities,omitempty"`
|
||||
}
|
||||
|
||||
// Promises says whether this kind's claim declares a capability.
|
||||
func (k KindHeld) Promises(capability string) bool {
|
||||
for _, c := range k.Capabilities {
|
||||
if c == capability {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// SeatKinds is every kind of the kinded benches this module uses or watches, as the controller issued
|
||||
// them in its membership: the one account of which channel is which, never a channel's own word.
|
||||
func SeatKinds() ([]KindHeld, error) {
|
||||
c, err := live()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
raw, err := c.ask("mesh/seat.kinds", map[string]any{}, seatTimeout)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []KindHeld
|
||||
return out, json.Unmarshal(raw, &out)
|
||||
}
|
||||
|
||||
// Create writes a key only when it has none, and returns the revision; a key that has one is refused.
|
||||
func (s KeptState) Create(key string, value any) (uint64, error) {
|
||||
return s.revisionOf(s.ask("create", map[string]any{"key": key, "value": value}))
|
||||
}
|
||||
|
||||
// Update writes a key only when it is still at revision, and returns the new one; a key changed
|
||||
// meanwhile is refused, so of two writers that read one value only the first writes.
|
||||
func (s KeptState) Update(key string, value any, revision uint64) (uint64, error) {
|
||||
return s.revisionOf(s.ask("update", map[string]any{"key": key, "value": value, "revision": revision}))
|
||||
}
|
||||
|
||||
func (s KeptState) revisionOf(raw json.RawMessage, err error) (uint64, error) {
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var put struct {
|
||||
Revision uint64 `json:"revision"`
|
||||
}
|
||||
return put.Revision, json.Unmarshal(raw, &put)
|
||||
}
|
||||
|
||||
// subjectMatches is a subject against a pattern in the bus's wildcards: `*` one token, `>` the rest.
|
||||
func subjectMatches(pattern, subject string) bool {
|
||||
p, s := strings.Split(pattern, "."), strings.Split(subject, ".")
|
||||
for i, part := range p {
|
||||
if part == ">" {
|
||||
return len(s) > i
|
||||
}
|
||||
if i >= len(s) || (part != "*" && part != s[i]) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return len(p) == len(s)
|
||||
}
|
||||
@@ -0,0 +1,98 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// novox/hq ADR 0259 §3: a bundle takes a seat's work and answers proofs through the runtime; a piece of
|
||||
// work it fails is answered as an error, so the runtime offers it again, and a proof's error is said.
|
||||
func TestABundleTakesWorkAndAnswersProofsThroughTheRuntime(t *testing.T) {
|
||||
t.Setenv(ServedAs, "router")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
taken := make(chan string, 4)
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{{Name: "start", Run: func(map[string]any) (any, error) {
|
||||
if err := SeatTake("SEAT_OPERATOR_CHANNEL_worker", func(w Work) error {
|
||||
taken <- w.Subject
|
||||
if string(w.Body) == `{"fail":true}` {
|
||||
return errors.New("not recorded")
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := SeatAnswer("mesh.seat.intake.proof.code.*", func(subject string, body json.RawMessage) (any, error) {
|
||||
if subject != "mesh.seat.intake.proof.code.telegram" {
|
||||
return nil, errors.New("no such kind")
|
||||
}
|
||||
return map[string]any{"accepted": true}, nil
|
||||
}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{}, nil
|
||||
}}}, inR, outW)
|
||||
}()
|
||||
lines := bufio.NewScanner(outR)
|
||||
next := func() map[string]any {
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
if lines.Scan() {
|
||||
var m map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &m)
|
||||
done <- m
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case m := <-done:
|
||||
return m
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("the bundle said nothing")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "start"}})
|
||||
asked := next()
|
||||
if asked["method"] != "mesh/seat.take" || asked["params"].(map[string]any)["worker"] != "SEAT_OPERATOR_CHANNEL_worker" {
|
||||
t.Fatalf("the take was not asked of the runtime: %v", asked)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": asked["id"], "result": map[string]any{}})
|
||||
asked = next()
|
||||
if asked["method"] != "mesh/seat.answer" {
|
||||
t.Fatalf("the answer was not asked of the runtime: %v", asked)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": asked["id"], "result": map[string]any{}})
|
||||
next() // the tool's answer
|
||||
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 2, "method": "mesh/work", "params": map[string]any{"work": map[string]any{
|
||||
"worker": "SEAT_OPERATOR_CHANNEL_worker", "subject": "mesh.seat.operator-channel.accept.ask.x", "body": map[string]any{"fail": true}}}})
|
||||
if got := next(); got["error"] == nil {
|
||||
t.Errorf("work the handler failed was answered as taken: %v", got)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 3, "method": "mesh/work", "params": map[string]any{"work": map[string]any{
|
||||
"worker": "SEAT_OPERATOR_CHANNEL_worker", "subject": "mesh.seat.operator-channel.accept.ask.x", "body": map[string]any{}}}})
|
||||
if got := next(); got["error"] != nil {
|
||||
t.Errorf("work the handler took was answered as an error: %v", got)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 4, "method": "mesh/proof", "params": map[string]any{
|
||||
"subject": "mesh.seat.intake.proof.code.telegram", "body": map[string]any{"code": "1"}}})
|
||||
if got := next(); got["result"].(map[string]any)["accepted"] != true {
|
||||
t.Errorf("the proof was answered %v", got)
|
||||
}
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 5, "method": "mesh/proof", "params": map[string]any{
|
||||
"subject": "mesh.seat.intake.proof.code.desktop", "body": map[string]any{}}})
|
||||
if got := next(); got["error"] == nil {
|
||||
t.Errorf("a proof the handler refused was answered as accepted: %v", got)
|
||||
}
|
||||
if len(taken) != 2 {
|
||||
t.Errorf("the handler was handed %d pieces of work, want 2", len(taken))
|
||||
}
|
||||
_ = inW.Close()
|
||||
}
|
||||
+72
@@ -2,6 +2,8 @@ package stdio
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -29,6 +31,16 @@ type StateChange struct {
|
||||
Current bool `json:"current"`
|
||||
}
|
||||
|
||||
// StateRevision is one value a bucket still keeps for a key: what was written, at which revision, by which
|
||||
// operation ("put" or "delete", whose value is null) and when (novox/hq ADR 0307).
|
||||
type StateRevision struct {
|
||||
Key string `json:"key"`
|
||||
Value json.RawMessage `json:"value"`
|
||||
Revision uint64 `json:"revision"`
|
||||
Op string `json:"op"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// KeptState is one state this module keeps, or reads.
|
||||
type KeptState struct{ name string }
|
||||
|
||||
@@ -56,6 +68,21 @@ func (s KeptState) Get(key string) (*StateEntry, error) {
|
||||
return &e, json.Unmarshal(raw, &e)
|
||||
}
|
||||
|
||||
// History is every value the bucket still keeps for key, oldest first — as many as the state declared it
|
||||
// keeps, deletes included — and empty for a key it holds nothing for (novox/hq ADR 0307). A read, like Get:
|
||||
// the owner's instances and its readers may ask it, and it writes nothing.
|
||||
func (s KeptState) History(key string) ([]StateRevision, error) {
|
||||
raw, err := s.ask("history", map[string]any{"key": key})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
revisions := []StateRevision{}
|
||||
if len(raw) == 0 || string(raw) == "null" {
|
||||
return revisions, nil
|
||||
}
|
||||
return revisions, json.Unmarshal(raw, &revisions)
|
||||
}
|
||||
|
||||
// Put writes a key — the owner's instances only — and returns the revision written.
|
||||
func (s KeptState) Put(key string, value any) (uint64, error) {
|
||||
raw, err := s.ask("put", map[string]any{"key": key, "value": value})
|
||||
@@ -74,6 +101,51 @@ func (s KeptState) Delete(key string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// Increment adds one to a counter kept at key and returns the number it now holds: one for a key with no
|
||||
// value. Read, then written only if the key is still at the revision read (Create when it had none, Update
|
||||
// when it had one), so of any number of writers each number is handed out once and no increment is lost.
|
||||
// A write another writer won is read again and tried again, at most attempts times in all; past that the
|
||||
// error is ErrChanged and nothing was counted. Any other error is returned at once, as is a value that is
|
||||
// not a whole number (null included): nothing is ever reported counted that was not written. The reverse
|
||||
// does not hold: an error that is not ErrChanged — a timeout above all — may come after the write landed, so
|
||||
// that number may be spent and never handed out. A caller tolerates gaps, never a number handed out twice.
|
||||
// Deleting the key starts the count again at one; a key whose numbers must stay unique is never deleted.
|
||||
func (s KeptState) Increment(key string, attempts int) (uint64, error) {
|
||||
if attempts < 1 {
|
||||
return 0, fmt.Errorf("an increment of %s is tried at least once: %d attempts", key, attempts)
|
||||
}
|
||||
for i := 0; i < attempts; i++ {
|
||||
e, err := s.Get(key)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var next uint64 = 1
|
||||
if e == nil {
|
||||
_, err = s.Create(key, next)
|
||||
} else {
|
||||
var n uint64
|
||||
if string(e.Value) == "null" {
|
||||
return 0, fmt.Errorf("%s holds null, not a counter", key)
|
||||
}
|
||||
if err := json.Unmarshal(e.Value, &n); err != nil {
|
||||
return 0, fmt.Errorf("%s holds %s, not a counter: %w", key, e.Value, err)
|
||||
}
|
||||
if n == ^uint64(0) {
|
||||
return 0, fmt.Errorf("%s is at the largest number a counter holds", key)
|
||||
}
|
||||
next = n + 1
|
||||
_, err = s.Update(key, next, e.Revision)
|
||||
}
|
||||
if err == nil {
|
||||
return next, nil
|
||||
}
|
||||
if !errors.Is(err, ErrChanged) {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return 0, fmt.Errorf("%s was written by another writer first %d times: %w", key, attempts, ErrChanged)
|
||||
}
|
||||
|
||||
// Keys is every key with a current value, sorted.
|
||||
func (s KeptState) Keys() ([]string, error) {
|
||||
raw, err := s.ask("keys", map[string]any{})
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// casRuntime is the runtime's side of mesh/state.get, create and update as the bus does them (novox/hq ADR
|
||||
// 0201, ADR 0259): one revision per write across the bucket, create refused when the key has a value,
|
||||
// update refused unless the key is still at the revision named, each refusal answered with CodeChanged.
|
||||
// refuse, when set, is asked before each write and may refuse it with an error of its own.
|
||||
type casRuntime struct {
|
||||
mu sync.Mutex
|
||||
revision uint64
|
||||
keys map[string]StateEntry
|
||||
writes map[string]int
|
||||
lost int
|
||||
refuse func(verb string) (code int, message string, refused bool)
|
||||
}
|
||||
|
||||
func (r *casRuntime) answer(m message) map[string]any {
|
||||
var p struct {
|
||||
Key string `json:"key"`
|
||||
Value json.RawMessage `json:"value"`
|
||||
Revision uint64 `json:"revision"`
|
||||
}
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
verb := strings.TrimPrefix(m.Method, "mesh/state.")
|
||||
lost := func() map[string]any {
|
||||
r.lost++
|
||||
return map[string]any{"error": map[string]any{"code": CodeChanged, "message": p.Key + ": the key was written by another writer first"}}
|
||||
}
|
||||
if verb != "get" && r.refuse != nil {
|
||||
if code, msg, refused := r.refuse(verb); refused {
|
||||
return map[string]any{"error": map[string]any{"code": code, "message": msg}}
|
||||
}
|
||||
}
|
||||
e, has := r.keys[p.Key]
|
||||
switch verb {
|
||||
case "get":
|
||||
if !has {
|
||||
return map[string]any{"result": nil}
|
||||
}
|
||||
return map[string]any{"result": e}
|
||||
case "create":
|
||||
if has {
|
||||
return lost()
|
||||
}
|
||||
case "update":
|
||||
if p.Revision == 0 {
|
||||
return map[string]any{"error": map[string]any{"code": -32000, "message": "an update names the revision it read"}}
|
||||
}
|
||||
if !has || e.Revision != p.Revision {
|
||||
return lost()
|
||||
}
|
||||
case "put":
|
||||
default:
|
||||
return map[string]any{"error": map[string]any{"code": -32601, "message": "no " + m.Method}}
|
||||
}
|
||||
r.revision++
|
||||
r.keys[p.Key] = StateEntry{Key: p.Key, Value: p.Value, Revision: r.revision}
|
||||
r.writes[verb]++
|
||||
return map[string]any{"result": map[string]any{"revision": r.revision}}
|
||||
}
|
||||
|
||||
// counting serves a bundle whose one tool, "count", runs run, against r; it answers the bundle's asks each
|
||||
// beside the loop, as the runtime does, so two writers' reads and writes interleave. It returns what the
|
||||
// tool answered.
|
||||
func counting(t *testing.T, r *casRuntime, run func() (any, error)) map[string]any {
|
||||
t.Helper()
|
||||
t.Setenv(ServedAs, "issues")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{{Name: "count", Run: func(map[string]any) (any, error) { return run() }}}, inR, outW)
|
||||
}()
|
||||
var wmu sync.Mutex
|
||||
send := func(m map[string]any) {
|
||||
m["jsonrpc"] = "2.0"
|
||||
b, _ := json.Marshal(m)
|
||||
wmu.Lock()
|
||||
defer wmu.Unlock()
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
}
|
||||
done := make(chan map[string]any, 1)
|
||||
go func() {
|
||||
lines := bufio.NewScanner(outR)
|
||||
lines.Buffer(make([]byte, 1<<20), 1<<24)
|
||||
for lines.Scan() {
|
||||
var m message
|
||||
if json.Unmarshal(lines.Bytes(), &m) != nil {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(m.Method, "mesh/state.") {
|
||||
go func(m message) {
|
||||
a := r.answer(m)
|
||||
a["id"] = m.ID
|
||||
send(a)
|
||||
}(m)
|
||||
continue
|
||||
}
|
||||
var answer map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &answer)
|
||||
if answer["id"] == float64(1) {
|
||||
done <- answer
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
send(map[string]any{"id": 1, "method": "tools/call", "params": map[string]any{"name": "count"}})
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
select {
|
||||
case a := <-done:
|
||||
return a
|
||||
case <-time.After(60 * time.Second):
|
||||
t.Fatal("the bundle did not answer the count")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func text(t *testing.T, a map[string]any) (string, bool) {
|
||||
t.Helper()
|
||||
res, _ := a["result"].(map[string]any)
|
||||
content, _ := res["content"].([]any)
|
||||
if len(content) != 1 {
|
||||
t.Fatalf("answered %v", a)
|
||||
}
|
||||
isError, _ := res["isError"].(bool)
|
||||
return content[0].(map[string]any)["text"].(string), isError
|
||||
}
|
||||
|
||||
// Two writers incrementing one counter a thousand times each end at two thousand: every number handed out
|
||||
// once, none lost, each writer's numbers rising — read, then written only at the revision read.
|
||||
func TestTwoWritersIncrementingOneCounterLoseNoUpdate(t *testing.T) {
|
||||
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
|
||||
const each = 1000
|
||||
got := counting(t, r, func() (any, error) {
|
||||
var wg sync.WaitGroup
|
||||
numbers := make([][]uint64, 2)
|
||||
errs := make([]error, 2)
|
||||
for w := range numbers {
|
||||
wg.Add(1)
|
||||
go func(w int) {
|
||||
defer wg.Done()
|
||||
for i := 0; i < each; i++ {
|
||||
n, err := State("numbers").Increment("issue", 1000)
|
||||
if err != nil {
|
||||
errs[w] = err
|
||||
return
|
||||
}
|
||||
numbers[w] = append(numbers[w], n)
|
||||
}
|
||||
}(w)
|
||||
}
|
||||
wg.Wait()
|
||||
if err := errors.Join(errs...); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
seen := map[uint64]bool{}
|
||||
for w, ns := range numbers {
|
||||
for i, n := range ns {
|
||||
if seen[n] {
|
||||
t.Errorf("the number %d was handed out twice", n)
|
||||
}
|
||||
seen[n] = true
|
||||
if i > 0 && n <= ns[i-1] {
|
||||
t.Errorf("writer %d was handed %d after %d", w, n, ns[i-1])
|
||||
}
|
||||
}
|
||||
}
|
||||
return len(seen), nil
|
||||
})
|
||||
if out, isError := text(t, got); isError || out != "2000" {
|
||||
t.Fatalf("two writers counted %s (error: %v)", out, isError)
|
||||
}
|
||||
var final uint64
|
||||
_ = json.Unmarshal(r.keys["issue"].Value, &final)
|
||||
if final != 2*each {
|
||||
t.Fatalf("the counter ended at %d, not %d: an increment was lost", final, 2*each)
|
||||
}
|
||||
if r.lost == 0 {
|
||||
t.Fatal("the two writers never collided: the test proved nothing about contention")
|
||||
}
|
||||
t.Logf("2000 increments, %d writes lost to the other writer and tried again", r.lost)
|
||||
if r.writes["create"] != 1 || r.writes["update"] != 2*each-1 || r.writes["put"] != 0 {
|
||||
t.Fatalf("written by %v: one create, then updates at the revision read, never a put", r.writes)
|
||||
}
|
||||
}
|
||||
|
||||
// A lost write is read again and tried again, never reported counted; past its attempts it is ErrChanged
|
||||
// with nothing counted; any other refusal, and a value that is no counter, is returned at once.
|
||||
func TestAnIncrementThatLosesIsTriedAgainAndNeverReportedCounted(t *testing.T) {
|
||||
t.Run("lost twice then won", func(t *testing.T) {
|
||||
lost := 0
|
||||
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`41`), Revision: 7}},
|
||||
writes: map[string]int{}, revision: 7}
|
||||
r.refuse = func(string) (int, string, bool) {
|
||||
if lost < 2 {
|
||||
lost++
|
||||
// Another writer was first: the counter moved on by one, to 42 and then 43.
|
||||
r.revision++
|
||||
r.keys["issue"] = StateEntry{Key: "issue", Value: json.RawMessage(fmt.Sprint(41 + lost)), Revision: r.revision}
|
||||
return CodeChanged, "issue: the key was written by another writer first", true
|
||||
}
|
||||
return 0, "", false
|
||||
}
|
||||
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 3) })
|
||||
if out, isError := text(t, got); isError || out != "44" {
|
||||
t.Fatalf("answered %s (error: %v), not 44", out, isError)
|
||||
}
|
||||
})
|
||||
t.Run("lost every time", func(t *testing.T) {
|
||||
tries := 0
|
||||
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
|
||||
r.refuse = func(string) (int, string, bool) {
|
||||
tries++
|
||||
return CodeChanged, "issue: the key was written by another writer first", true
|
||||
}
|
||||
var err error
|
||||
got := counting(t, r, func() (any, error) {
|
||||
n, e := State("numbers").Increment("issue", 5)
|
||||
err = e
|
||||
return n, e
|
||||
})
|
||||
if _, isError := text(t, got); !isError || !errors.Is(err, ErrChanged) || tries != 5 {
|
||||
t.Fatalf("after %d tries: %v", tries, err)
|
||||
}
|
||||
})
|
||||
t.Run("refused otherwise", func(t *testing.T) {
|
||||
tries := 0
|
||||
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
|
||||
r.refuse = func(string) (int, string, bool) {
|
||||
tries++
|
||||
return -32000, "issues keeps no state numbers", true
|
||||
}
|
||||
var err error
|
||||
got := counting(t, r, func() (any, error) {
|
||||
n, e := State("numbers").Increment("issue", 5)
|
||||
err = e
|
||||
return n, e
|
||||
})
|
||||
if _, isError := text(t, got); !isError || errors.Is(err, ErrChanged) || tries != 1 {
|
||||
t.Fatalf("after %d tries: %v", tries, err)
|
||||
}
|
||||
})
|
||||
t.Run("no counter", func(t *testing.T) {
|
||||
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`"x"`), Revision: 1}},
|
||||
writes: map[string]int{}, revision: 1}
|
||||
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 5) })
|
||||
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
|
||||
t.Fatalf("a value that is no counter was counted: %v, writes %v", got, r.writes)
|
||||
}
|
||||
})
|
||||
t.Run("null", func(t *testing.T) {
|
||||
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`null`), Revision: 1}},
|
||||
writes: map[string]int{}, revision: 1}
|
||||
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 5) })
|
||||
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
|
||||
t.Fatalf("a null was counted from zero: %v, writes %v", got, r.writes)
|
||||
}
|
||||
})
|
||||
t.Run("no attempts", func(t *testing.T) {
|
||||
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
|
||||
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 0) })
|
||||
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
|
||||
t.Fatalf("zero attempts counted: %v", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
package stdio
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// historying serves a bundle whose one tool, "history", runs run, answering each mesh/state ask with
|
||||
// answer beside the loop, as the runtime does; it returns what the bundle asked.
|
||||
func historying(t *testing.T, answer func(message) map[string]any, run func()) []message {
|
||||
t.Helper()
|
||||
t.Setenv(ServedAs, "issues")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{{Name: "history", Run: func(map[string]any) (any, error) { run(); return "done", nil }}}, inR, outW)
|
||||
}()
|
||||
var wmu, amu sync.Mutex
|
||||
var asked []message
|
||||
send := func(m map[string]any) {
|
||||
m["jsonrpc"] = "2.0"
|
||||
b, _ := json.Marshal(m)
|
||||
wmu.Lock()
|
||||
defer wmu.Unlock()
|
||||
_, _ = inW.Write(append(b, '\n'))
|
||||
}
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
lines := bufio.NewScanner(outR)
|
||||
for lines.Scan() {
|
||||
var m message
|
||||
if json.Unmarshal(lines.Bytes(), &m) != nil {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(m.Method, "mesh/state.") {
|
||||
amu.Lock()
|
||||
asked = append(asked, m)
|
||||
amu.Unlock()
|
||||
a := answer(m)
|
||||
a["id"] = m.ID
|
||||
send(a)
|
||||
continue
|
||||
}
|
||||
if m.Method == "" {
|
||||
close(done)
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
send(map[string]any{"id": 1, "method": "tools/call", "params": map[string]any{"name": "history"}})
|
||||
t.Cleanup(func() { _ = inW.Close() })
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("the bundle did not answer")
|
||||
}
|
||||
amu.Lock()
|
||||
defer amu.Unlock()
|
||||
return asked
|
||||
}
|
||||
|
||||
// History asks mesh/state.history with the state and the key alone, and hands back every kept value oldest
|
||||
// first, a delete with its null value; nothing kept is an empty history; a refusal is the runtime's error
|
||||
// (novox/hq ADR 0307).
|
||||
func TestHistoryIsEveryKeptValueOldestFirst(t *testing.T) {
|
||||
kept := []map[string]any{
|
||||
{"key": "401", "value": map[string]any{"status": "open"}, "revision": 4, "op": "put", "at": "2026-10-10T19:46:00Z"},
|
||||
{"key": "401", "value": map[string]any{"status": "fixing"}, "revision": 9, "op": "put", "at": "2026-10-11T08:01:02Z"},
|
||||
{"key": "401", "value": nil, "revision": 12, "op": "delete", "at": "2026-10-11T09:30:00+02:00"},
|
||||
}
|
||||
var got, none, null []StateRevision
|
||||
var refused error
|
||||
asked := historying(t, func(m message) map[string]any {
|
||||
var p map[string]any
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
switch {
|
||||
case p["state"] != "issues":
|
||||
return map[string]any{"error": map[string]any{"code": -32000, "message": "issues was issued no state " + p["state"].(string)}}
|
||||
case p["key"] == "401":
|
||||
return map[string]any{"result": kept}
|
||||
case p["key"] == "402":
|
||||
return map[string]any{"result": nil}
|
||||
}
|
||||
return map[string]any{"result": []any{}}
|
||||
}, func() {
|
||||
var err error
|
||||
if got, err = State("issues").History("401"); err != nil {
|
||||
t.Errorf("history of 401: %v", err)
|
||||
}
|
||||
if none, err = State("issues").History("999"); err != nil {
|
||||
t.Errorf("history of 999: %v", err)
|
||||
}
|
||||
if null, err = State("issues").History("402"); err != nil {
|
||||
t.Errorf("history of 402: %v", err)
|
||||
}
|
||||
var h []StateRevision
|
||||
h, refused = State("elsewhere.issues").History("401")
|
||||
if h != nil {
|
||||
t.Errorf("a refused history handed back %v", h)
|
||||
}
|
||||
})
|
||||
|
||||
if len(asked) != 4 {
|
||||
t.Fatalf("asked %d times, not 4", len(asked))
|
||||
}
|
||||
var p map[string]any
|
||||
_ = json.Unmarshal(asked[0].Params, &p)
|
||||
if asked[0].Method != "mesh/state.history" || p["state"] != "issues" || p["key"] != "401" || len(p) != 2 {
|
||||
t.Fatalf("asked %s %v", asked[0].Method, p)
|
||||
}
|
||||
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("history of 401 is %d values, not 3: %+v", len(got), got)
|
||||
}
|
||||
want := []struct {
|
||||
rev uint64
|
||||
op string
|
||||
value string
|
||||
at time.Time
|
||||
}{
|
||||
{4, "put", `{"status":"open"}`, time.Date(2026, 10, 10, 19, 46, 0, 0, time.UTC)},
|
||||
{9, "put", `{"status":"fixing"}`, time.Date(2026, 10, 11, 8, 1, 2, 0, time.UTC)},
|
||||
{12, "delete", `null`, time.Date(2026, 10, 11, 7, 30, 0, 0, time.UTC)},
|
||||
}
|
||||
for i, w := range want {
|
||||
r := got[i]
|
||||
if r.Key != "401" || r.Revision != w.rev || r.Op != w.op || string(r.Value) != w.value || !r.At.Equal(w.at) {
|
||||
t.Fatalf("value %d is %+v, not %+v", i, r, w)
|
||||
}
|
||||
}
|
||||
if none == nil || len(none) != 0 || null == nil || len(null) != 0 {
|
||||
t.Fatalf("nothing kept read as %v and %v, not an empty history", none, null)
|
||||
}
|
||||
if refused == nil || !strings.Contains(refused.Error(), "issued no state elsewhere.issues") {
|
||||
t.Fatalf("a state not issued answered %v", refused)
|
||||
}
|
||||
}
|
||||
+133
-12
@@ -39,7 +39,8 @@ type Tool struct {
|
||||
Description string
|
||||
// Input is the JSON schema of the arguments; nil is a tool that takes nothing.
|
||||
Input map[string]any
|
||||
Run func(args map[string]any) (any, error)
|
||||
// Run answers one call. Who called is CallerOf(args): the bus user the runtime proved, or Unproven.
|
||||
Run func(args map[string]any) (any, error)
|
||||
}
|
||||
|
||||
type message struct {
|
||||
@@ -48,6 +49,7 @@ type message struct {
|
||||
Params json.RawMessage `json:"params,omitempty"`
|
||||
Result json.RawMessage `json:"result,omitempty"`
|
||||
Error *struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
} `json:"error,omitempty"`
|
||||
}
|
||||
@@ -55,7 +57,11 @@ type message struct {
|
||||
// channel is the one conversation with the runtime: what this bundle writes, serialised, and what
|
||||
// it asked and waits for.
|
||||
type channel struct {
|
||||
mu sync.Mutex
|
||||
mu sync.Mutex
|
||||
// wmu orders the writes to the runtime and nothing else: a write waits on the pipe for as long as
|
||||
// the runtime does not read, and mu must never wait with it, or the loop that reads the runtime's
|
||||
// answers stops (issue 1286).
|
||||
wmu sync.Mutex
|
||||
enc *json.Encoder
|
||||
next int
|
||||
waiting map[string]chan message
|
||||
@@ -64,6 +70,11 @@ type channel struct {
|
||||
subscribed int
|
||||
// watches are this bundle's state watches by the id each was asked with (ADR 0201).
|
||||
watches map[string]func(StateChange) error
|
||||
// provisioned are this bundle's watches of its values, by the id each was asked with (provision.go).
|
||||
provisioned map[string]func(ProvisionChange) error
|
||||
// takers and provers are this bundle's seat work and proof handlers, by worker and subject (ADR 0259).
|
||||
takers map[string]func(Work) error
|
||||
provers map[string]func(subject string, body json.RawMessage) (any, error)
|
||||
}
|
||||
|
||||
type listener struct {
|
||||
@@ -83,8 +94,8 @@ type Envelope struct {
|
||||
|
||||
func (c *channel) say(m map[string]any) {
|
||||
m["jsonrpc"] = "2.0"
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.wmu.Lock()
|
||||
defer c.wmu.Unlock()
|
||||
_ = c.enc.Encode(m)
|
||||
}
|
||||
|
||||
@@ -111,7 +122,7 @@ func (c *channel) askAs(id, method string, params any, timeout time.Duration) (j
|
||||
select {
|
||||
case m := <-answer:
|
||||
if m.Error != nil {
|
||||
return nil, &refused{m.Error.Message}
|
||||
return nil, &refused{message: m.Error.Message, code: m.Error.Code}
|
||||
}
|
||||
return m.Result, nil
|
||||
case <-time.After(timeout):
|
||||
@@ -123,10 +134,23 @@ func (c *channel) askAs(id, method string, params any, timeout time.Duration) (j
|
||||
}
|
||||
|
||||
// refused is the runtime's error answer, as opposed to no answer: what it refused, it did not do.
|
||||
type refused struct{ message string }
|
||||
type refused struct {
|
||||
message string
|
||||
code int
|
||||
}
|
||||
|
||||
func (r *refused) Error() string { return r.message }
|
||||
|
||||
// Is makes a compare-and-set the runtime refused because another writer was first ErrChanged.
|
||||
func (r *refused) Is(target error) bool { return target == ErrChanged && r.code == CodeChanged }
|
||||
|
||||
// ErrChanged is a compare-and-set that lost: the key was made or changed by another writer first (novox/hq
|
||||
// ADR 0259). Told apart by the runtime's error code, never by its words.
|
||||
var ErrChanged = errors.New("the key was written by another writer first")
|
||||
|
||||
// CodeChanged is the error code the runtime answers a lost compare-and-set with.
|
||||
const CodeChanged = -32010
|
||||
|
||||
var (
|
||||
current *channel
|
||||
currentMu sync.Mutex
|
||||
@@ -150,7 +174,9 @@ func ServeOn(name string, tools []Tool, in io.Reader, out io.Writer) error {
|
||||
for _, t := range tools {
|
||||
byName[t.Name] = t
|
||||
}
|
||||
c := &channel{enc: json.NewEncoder(out), waiting: map[string]chan message{}, watches: map[string]func(StateChange) error{}}
|
||||
c := &channel{enc: json.NewEncoder(out), waiting: map[string]chan message{}, watches: map[string]func(StateChange) error{},
|
||||
provisioned: map[string]func(ProvisionChange) error{},
|
||||
takers: map[string]func(Work) error{}, provers: map[string]func(string, json.RawMessage) (any, error){}}
|
||||
currentMu.Lock()
|
||||
current = c
|
||||
currentMu.Unlock()
|
||||
@@ -205,6 +231,8 @@ func ServeOn(name string, tools []Tool, in io.Reader, out io.Writer) error {
|
||||
var p struct {
|
||||
Name string `json:"name"`
|
||||
Arguments map[string]any `json:"arguments"`
|
||||
// Meta is the runtime's, never the caller's: who called, taken from the subject (CallerOf).
|
||||
Meta map[string]any `json:"_meta"`
|
||||
}
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
t, ok := byName[p.Name]
|
||||
@@ -214,18 +242,18 @@ func ServeOn(name string, tools []Tool, in io.Reader, out io.Writer) error {
|
||||
}
|
||||
// Beside the loop, never in it: a tool that emits waits for the runtime's answer, which
|
||||
// arrives on the line this loop reads next — and calls do not queue behind a slow one.
|
||||
go func(id any, t Tool, args map[string]any) {
|
||||
go func(id any, t Tool, args map[string]any, caller string) {
|
||||
if args == nil {
|
||||
args = map[string]any{}
|
||||
}
|
||||
result, err := t.Run(args)
|
||||
result, err := runAs(t, caller, args)
|
||||
if err != nil {
|
||||
c.say(map[string]any{"id": id, "result": map[string]any{"content": []map[string]any{{"type": "text", "text": err.Error()}}, "isError": true}})
|
||||
return
|
||||
}
|
||||
text, _ := json.Marshal(result)
|
||||
c.say(map[string]any{"id": id, "result": map[string]any{"content": []map[string]any{{"type": "text", "text": string(text)}}}})
|
||||
}(id, t, p.Arguments)
|
||||
}(id, t, p.Arguments, callerIn(p.Meta))
|
||||
case "mesh/event":
|
||||
// One event the runtime read from this module's consumer (ADR 0198), answered once every
|
||||
// handler it is for has taken it; an error is answered as one, so the runtime does not
|
||||
@@ -276,6 +304,85 @@ func ServeOn(name string, tools []Tool, in io.Reader, out io.Writer) error {
|
||||
}
|
||||
c.say(map[string]any{"id": id, "result": map[string]any{}})
|
||||
}(id, *p.Change)
|
||||
case "mesh/provision":
|
||||
// One change to a value of this module's that this bundle watches (provision.go), named by the
|
||||
// watch's id, answered once its handler has taken it. Beside the loop, like a tool call.
|
||||
var p struct {
|
||||
Watch string `json:"watch"`
|
||||
Change *ProvisionChange `json:"change"`
|
||||
}
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
c.mu.Lock()
|
||||
handle := c.provisioned[p.Watch]
|
||||
c.mu.Unlock()
|
||||
if handle == nil || p.Change == nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32602, "message": "mesh/provision names no watch of this bundle"}})
|
||||
continue
|
||||
}
|
||||
go func(id any, change ProvisionChange) {
|
||||
if err := handle(change); err != nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32000, "message": err.Error()}})
|
||||
return
|
||||
}
|
||||
c.say(map[string]any{"id": id, "result": map[string]any{}})
|
||||
}(id, *p.Change)
|
||||
case "mesh/work":
|
||||
// One piece of a seat's work this bundle takes (novox/hq ADR 0259): taken by an answer that is
|
||||
// not an error, offered again after one. Beside the loop, like a tool call.
|
||||
var p struct {
|
||||
Work *Work `json:"work"`
|
||||
}
|
||||
if json.Unmarshal(m.Params, &p) != nil || p.Work == nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32602, "message": "mesh/work carries no work"}})
|
||||
continue
|
||||
}
|
||||
c.mu.Lock()
|
||||
handle := c.takers[p.Work.Worker]
|
||||
if handle == nil && len(c.takers) == 1 {
|
||||
for _, h := range c.takers {
|
||||
handle = h
|
||||
}
|
||||
}
|
||||
c.mu.Unlock()
|
||||
go func(id any, w Work) {
|
||||
if handle == nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32000, "message": "nothing here takes " + w.Subject}})
|
||||
return
|
||||
}
|
||||
if err := handle(w); err != nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32000, "message": err.Error()}})
|
||||
return
|
||||
}
|
||||
c.say(map[string]any{"id": id, "result": map[string]any{}})
|
||||
}(id, *p.Work)
|
||||
case "mesh/proof":
|
||||
// One proof this bundle answers (novox/hq ADR 0259): what the handler returns is the answer; an
|
||||
// error is answered in words, never as silence.
|
||||
var p struct {
|
||||
Subject string `json:"subject"`
|
||||
Body json.RawMessage `json:"body"`
|
||||
}
|
||||
_ = json.Unmarshal(m.Params, &p)
|
||||
c.mu.Lock()
|
||||
var handle func(string, json.RawMessage) (any, error)
|
||||
for pattern, h := range c.provers {
|
||||
if subjectMatches(pattern, p.Subject) {
|
||||
handle = h
|
||||
}
|
||||
}
|
||||
c.mu.Unlock()
|
||||
go func(id any, subject string, body json.RawMessage) {
|
||||
if handle == nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32000, "message": "nothing here answers " + subject}})
|
||||
return
|
||||
}
|
||||
out, err := handle(subject, body)
|
||||
if err != nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32000, "message": err.Error()}})
|
||||
return
|
||||
}
|
||||
c.say(map[string]any{"id": id, "result": out})
|
||||
}(id, p.Subject, p.Body)
|
||||
default:
|
||||
if id != nil {
|
||||
c.say(map[string]any{"id": id, "error": map[string]any{"code": -32601, "message": "no " + m.Method}})
|
||||
@@ -369,10 +476,15 @@ func matchFrom(p []string, pi int, k []string, ki int) bool {
|
||||
return matchFrom(p, pi+1, k, ki+1)
|
||||
}
|
||||
|
||||
// EmitOptions is the provenance an emitter may attach (ADR 0042).
|
||||
// EmitOptions is the provenance an emitter may attach (ADR 0042), and the message id it may choose.
|
||||
type EmitOptions struct {
|
||||
CausationID string
|
||||
Schema string
|
||||
// MessageID is the id this emit is sent under, as x-event-id (and, by the tool runner, as the bus's
|
||||
// own Nats-Msg-Id): chosen by the emitter so a repeated publish inside the stream's duplicate window
|
||||
// is kept to one, never generated once an id is given (novox/hq ADR 0436 clause 4). Empty is a random
|
||||
// one, as on main.
|
||||
MessageID string
|
||||
}
|
||||
|
||||
// Emit asks the runtime to publish an event as this bundle's module, and returns once the bus has
|
||||
@@ -392,8 +504,17 @@ func Emit(eventType string, body any, opts ...EmitOptions) error {
|
||||
if node == "" {
|
||||
node = "unknown"
|
||||
}
|
||||
id := ""
|
||||
for _, o := range opts {
|
||||
if o.MessageID != "" {
|
||||
id = o.MessageID
|
||||
}
|
||||
}
|
||||
if id == "" {
|
||||
id = eventID()
|
||||
}
|
||||
headers := map[string]string{
|
||||
"x-event-id": eventID(), "x-source": source, "x-node": node,
|
||||
"x-event-id": id, "x-source": source, "x-node": node,
|
||||
"x-time": time.Now().UTC().Format(time.RFC3339Nano), "content-type": "application/json",
|
||||
}
|
||||
for _, o := range opts {
|
||||
|
||||
@@ -224,3 +224,54 @@ func TestARefusedSubscribeLeavesNoHandlerBehind(t *testing.T) {
|
||||
t.Fatalf("one event was handled %d times", handled)
|
||||
}
|
||||
}
|
||||
|
||||
// A message the runtime has not read yet does not stop the channel taking its answers: with one tool
|
||||
// answer blocked on the pipe, a refused subscribe is still delivered to Subscribe, which returns the
|
||||
// refusal — no lock of the channel is held across a write to the pipe (novox/hq issue 1286/AC1, 1286/AC2).
|
||||
func TestARefusedSubscribeIsDeliveredWhileAMessageWaitsOnThePipe(t *testing.T) {
|
||||
t.Setenv(ServedAs, "plex")
|
||||
inR, inW := io.Pipe()
|
||||
outR, outW := io.Pipe()
|
||||
defer func() { _ = inW.Close() }()
|
||||
// Not the channel an earlier run left behind: Subscribe goes to the one this run serves.
|
||||
currentMu.Lock()
|
||||
current = nil
|
||||
currentMu.Unlock()
|
||||
go func() {
|
||||
_ = ServeOn("", []Tool{{Name: "quick", Run: func(map[string]any) (any, error) { return "ok", nil }}}, inR, outW)
|
||||
}()
|
||||
subscribed := make(chan error, 1)
|
||||
go func() {
|
||||
for {
|
||||
if _, err := live(); err == nil {
|
||||
break
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
subscribed <- Subscribe("*.download.completed", func(Envelope) error { return nil })
|
||||
}()
|
||||
go func() {
|
||||
lines := bufio.NewScanner(outR)
|
||||
if !lines.Scan() {
|
||||
return
|
||||
}
|
||||
var req map[string]any
|
||||
_ = json.Unmarshal(lines.Bytes(), &req)
|
||||
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
|
||||
// A tool answer the runtime has not read: its write waits on the pipe, and no one reads further.
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "quick"}})
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
send(map[string]any{"jsonrpc": "2.0", "id": req["id"], "error": map[string]any{"code": -32000, "message": "not yet"}})
|
||||
}()
|
||||
deadline := time.After(2 * time.Second)
|
||||
select {
|
||||
case err := <-subscribed:
|
||||
if err == nil || err.Error() != "not yet" {
|
||||
t.Fatalf("the refused subscribe said %v", err)
|
||||
}
|
||||
case <-deadline:
|
||||
t.Fatal("Subscribe did not return in 2 s: the channel is stuck on a write to a pipe nobody reads")
|
||||
}
|
||||
// Let the waiting answer go, so nothing is left blocked.
|
||||
go func() { _, _ = io.Copy(io.Discard, outR) }()
|
||||
}
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ node node_modules/typescript/bin/tsc --noEmit
|
||||
npm test
|
||||
|
||||
if command -v go >/dev/null 2>&1; then
|
||||
(cd go && test -z "$(gofmt -l .)" && go vet ./... && go test -count=1 ./...)
|
||||
sh go/merge-check.sh
|
||||
else
|
||||
echo "NOT TESTED HERE: go/ (the Go binding) — the Go toolchain runs it from go/merge-check.sh"
|
||||
fi
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "@novox/mesh-sdk",
|
||||
"version": "0.1.13",
|
||||
"version": "0.1.14",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "@novox/mesh-sdk",
|
||||
"version": "0.1.13",
|
||||
"version": "0.1.14",
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"typescript": "^5.6.0"
|
||||
|
||||
+5
-3
@@ -1,18 +1,20 @@
|
||||
{
|
||||
"name": "@novox/mesh-sdk",
|
||||
"version": "0.1.13",
|
||||
"version": "0.1.14",
|
||||
"description": "The stable spine a Novox Mesh module's own code builds against.",
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": "./dist/index.js",
|
||||
"./contracts": "./dist/contracts/index.js",
|
||||
"./provisioner": "./dist/provisioner/index.js",
|
||||
"./provider-loop": "./dist/provider-loop/index.js",
|
||||
"./provisioner": "./dist/provider-loop/index.js",
|
||||
"./tools": "./dist/tools/index.js",
|
||||
"./messaging": "./dist/messaging/index.js",
|
||||
"./events": "./dist/events/index.js",
|
||||
"./primitives": "./dist/primitives/index.js",
|
||||
"./stdio": "./dist/stdio/index.js",
|
||||
"./state": "./dist/state/index.js"
|
||||
"./state": "./dist/state/index.js",
|
||||
"./seal": "./dist/seal/index.js"
|
||||
},
|
||||
"files": [
|
||||
"dist"
|
||||
|
||||
+5
-2
@@ -1,13 +1,16 @@
|
||||
// @novox/mesh-sdk — the stable spine a module's own code builds against (novox/hq ADR 0039).
|
||||
//
|
||||
// Import the area you need directly (`@novox/mesh-sdk/tools`, `/provisioner`, …) or the whole
|
||||
// Import the area you need directly (`@novox/mesh-sdk/tools`, `/provider-loop`, …) or the whole
|
||||
// surface from here. Everything specific to one module — its API client, its tool implementations,
|
||||
// its create-a-resource adapter — lives in the module, never here.
|
||||
|
||||
export * from "./contracts/index.js";
|
||||
export * as tools from "./tools/index.js";
|
||||
export * as provisioner from "./provisioner/index.js";
|
||||
export * as providerLoop from "./provider-loop/index.js";
|
||||
/** @deprecated The provider loop's name before design 57 (novox/hq issue 678): `providerLoop`. */
|
||||
export * as provisioner from "./provider-loop/index.js";
|
||||
export * as messaging from "./messaging/index.js";
|
||||
export * as events from "./events/index.js";
|
||||
export * as primitives from "./primitives/index.js";
|
||||
export * as state from "./state/index.js";
|
||||
export * as seal from "./seal/index.js";
|
||||
|
||||
@@ -1,3 +1,13 @@
|
||||
// The provider loop: the reconcile harness every provider on the controller's grant files shares.
|
||||
// Called `provisioner` until design 57 (novox/hq issue 678): that word is mesh-provisioner's, which
|
||||
// places every value a module needs (ADR 0349), so the loop is named for what it is. The old path
|
||||
// `@novox/mesh-sdk/provisioner` and the names `runProvisioner`, `provisionerTools`, `Provisioner` and
|
||||
// `ProvisionerOptions` stay as aliases of these until the providers that import them move to the Go
|
||||
// helper that answers consumers over the bus (mesh-sdk go/provider, design 57 B2 and B3) and this loop
|
||||
// retires with the grant files. The events (`provisioner.failing`, `.recovered`, `.retirement`) and the
|
||||
// tool names (`provisioner_retirement`, …) are what the controller and the operator read on the bus,
|
||||
// and keep their names.
|
||||
//
|
||||
// The reconcile harness every provider shares. The loop below is identical for postgres, redis,
|
||||
// minio and umami: read the contributions the mesh delivered, bring each consumer's resource into
|
||||
// being through the provider's adapter under the login and password the mesh minted, and retire
|
||||
@@ -10,10 +20,20 @@
|
||||
// does not hand one back — the consumer already receives its copy through the mesh's own channel.
|
||||
// This is why the old $MESH_SEAL_KEY, the symmetric seal, and the *.grant.json / *.credential files
|
||||
// are gone: they described a second, contradictory credential path the mesh does not have.
|
||||
//
|
||||
// **A provider that keeps failing a consumer says so on the bus (novox/hq ADR 0224).** A consumer
|
||||
// whose create, check or secret has failed without one success in between for `failingAfterMs` is
|
||||
// announced as `provisioner.failing` — naming the consumer, its machine and the class of error — and
|
||||
// again every `sayAgainEveryMs` while it lasts; the first success after that is
|
||||
// `provisioner.recovered`, as is a consumer the mesh stopped asking for, and the first success for
|
||||
// each consumer since this process started. The controller keeps the newest per provider and consumer
|
||||
// and `status` names it. Until novox/hq issue 272 only the Go copy of this loop (postgres, keycloak)
|
||||
// said it; every provider on this one failed its consumers in its journal alone.
|
||||
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
import type { ToolDefinition } from "../contracts/index.js";
|
||||
import { emit } from "../events/index.js";
|
||||
import { registerModuleTools } from "../tools/index.js";
|
||||
|
||||
/** One consumer's resource to bring into being — everything the mesh derived and delivered. */
|
||||
@@ -86,6 +106,9 @@ export interface Adapter {
|
||||
/** Optional: every consumer the backend holds ACTIVE that the mesh made, read at start, so one the
|
||||
* mesh stopped asking for while this process was down is still retired (by the same rules). */
|
||||
held?(): Promise<string[]>;
|
||||
/** Optional: the class of one of the adapter's own errors (novox/hq ADR 0224), when its code knows
|
||||
* better than the error's words; "" or undefined falls back to `classOf` of the text. */
|
||||
classify?(err: unknown): string | undefined;
|
||||
}
|
||||
|
||||
/** One retired consumer, as `provisioner_retirement` answers it. */
|
||||
@@ -127,9 +150,9 @@ export interface Act {
|
||||
readonly via?: string;
|
||||
}
|
||||
|
||||
/** The running provisioner: a stop function, as before, that also answers and takes the person's
|
||||
/** The running provider loop: a stop function, as before, that also answers and takes the person's
|
||||
* acts on retirement (novox/hq ADR 0230). */
|
||||
export type Provisioner = (() => void) & {
|
||||
export type ProviderLoop = (() => void) & {
|
||||
retirement(): Promise<Retirement>;
|
||||
approve(consumers: readonly string[], act: Act): Promise<{ retired: string[] }>;
|
||||
reject(consumers: readonly string[], act: Act): Promise<{ kept: string[] }>;
|
||||
@@ -139,7 +162,22 @@ export type Provisioner = (() => void) & {
|
||||
/** The one event a provider says about retirement (novox/hq ADR 0230). */
|
||||
export const RETIREMENT_EVENT = "provisioner.retirement";
|
||||
|
||||
export interface ProvisionerOptions {
|
||||
/** The events a provider's standing toward one consumer is announced as (novox/hq ADR 0224). The
|
||||
* controller derives the permission to emit them for every module that receives contributions. */
|
||||
export const FAILING_EVENT = "provisioner.failing";
|
||||
export const RECOVERED_EVENT = "provisioner.recovered";
|
||||
|
||||
/** The classes a failing consumer is announced with: what a person reading `status` needs to know
|
||||
* before reading the journal (novox/hq ADR 0224). An adapter may say better (`classify`). */
|
||||
export const CLASS_CREDENTIALS = "credentials-rejected";
|
||||
export const CLASS_UNREACHABLE = "unreachable";
|
||||
export const CLASS_SECRET = "secret-unreadable";
|
||||
export const CLASS_REFUSED = "refused";
|
||||
|
||||
/** How long a consumer fails without one success before it is announced failing (ADR 0224). */
|
||||
export const FAILING_AFTER_MS = 5 * 60_000;
|
||||
|
||||
export interface ProviderLoopOptions {
|
||||
/** The contributions file the mesh writes for this resource (the module's `receives` path).
|
||||
* Defaults to $MESH_RECEIVES. */
|
||||
receives?: string;
|
||||
@@ -165,10 +203,16 @@ export interface ProvisionerOptions {
|
||||
/** Nor a set larger than this share of the consumers held, where more than one is held. Defaults
|
||||
* to 0.5. */
|
||||
retireFraction?: number;
|
||||
/** How often a waiting retirement is said again, in ms. Defaults to 15 minutes. */
|
||||
/** How often a waiting retirement, and a consumer still failing, is said again, in ms. Defaults to
|
||||
* 15 minutes. */
|
||||
sayAgainEveryMs?: number;
|
||||
/** Publishes one of the provider's events (`provisioner.retirement`); usually the SDK's `emit`.
|
||||
* Without it the provider's retirement is said in its journal only. */
|
||||
/** How long, in ms, a consumer fails without one success before it is announced
|
||||
* `provisioner.failing` (novox/hq ADR 0224). Defaults to FAILING_AFTER_MS, five minutes. */
|
||||
failingAfterMs?: number;
|
||||
/** Publishes one of the provider's events (`provisioner.failing`, `provisioner.recovered`,
|
||||
* `provisioner.retirement`). Defaults to the SDK's `emit`, so every provider says them on the bus
|
||||
* without asking (novox/hq issue 272); a failed publish is said in the journal and never fails
|
||||
* the act it reports. */
|
||||
announce?: (event: string, body: Record<string, unknown>) => Promise<void> | void;
|
||||
/** The machine this provider runs on, said in each event. Defaults to $MESH_NODE. */
|
||||
node?: string;
|
||||
@@ -216,7 +260,7 @@ interface Contribution {
|
||||
* `retireFraction` of those held where more than one is held, is not retired without a person: the
|
||||
* provider waits for approve or reject. RETIRED → DELETED only through `provisioner_delete`.
|
||||
*/
|
||||
export function runProvisioner(resource: string, adapter: Adapter, opts: ProvisionerOptions = {}): Provisioner {
|
||||
export function runProviderLoop(resource: string, adapter: Adapter, opts: ProviderLoopOptions = {}): ProviderLoop {
|
||||
const receives = opts.receives ?? envOrThrow("MESH_RECEIVES");
|
||||
const everyMs = opts.everyMs ?? 5000;
|
||||
const verifyEveryMs = opts.verifyEveryMs ?? 60_000;
|
||||
@@ -227,6 +271,8 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
const retireAtOnce = opts.retireAtOnce ?? 3;
|
||||
const retireFraction = opts.retireFraction ?? 0.5;
|
||||
const sayAgainEveryMs = opts.sayAgainEveryMs ?? 15 * 60_000;
|
||||
const failingAfterMs = opts.failingAfterMs ?? FAILING_AFTER_MS;
|
||||
const announce = opts.announce ?? ((event: string, body: Record<string, unknown>) => emit(event, body));
|
||||
const node = opts.node ?? process.env.MESH_NODE ?? "";
|
||||
const say = (line: string): void => console.error(`[provisioner:${resource}] ${line}`);
|
||||
// Retirement (novox/hq ADR 0230). The unasked set last seen and in how many consecutive readable
|
||||
@@ -245,6 +291,13 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
// itself. Each repeat doubles the wait before asking again, so such a bug costs one re-apply an
|
||||
// hour rather than one a minute, and it is said loudly rather than quietly repeated.
|
||||
const lost = new Map<string, { times: number; nextAt: number }>();
|
||||
// Each consumer's unbroken run of failures — since when, how often, the last error, and when it was
|
||||
// last announced (0: not yet) — and every consumer that has succeeded since this process started
|
||||
// (novox/hq ADR 0224).
|
||||
const trouble = new Map<string, {
|
||||
node?: string; since: number; attempts: number; cls: string; text: string; saidAt: number;
|
||||
}>();
|
||||
const cleared = new Set<string>();
|
||||
|
||||
// login (`as`) -> what was last applied: the hash that says whether it changed, and the
|
||||
// derivation to hand back on withdrawal, which by then is no longer in the file.
|
||||
@@ -311,6 +364,7 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
`file and who this process runs as (novox/hq issue 225)`,
|
||||
);
|
||||
}
|
||||
await failed(g.as, g.node, CLASS_SECRET, `secret not readable (${g.secret}): ${err}`);
|
||||
continue;
|
||||
}
|
||||
waiting.delete(g.as);
|
||||
@@ -328,11 +382,15 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
try {
|
||||
if (await withTimeout(adapter.holds!(p), holdsTimeoutMs)) {
|
||||
lost.delete(g.as);
|
||||
await succeeded(g.as);
|
||||
continue;
|
||||
}
|
||||
} catch (err) {
|
||||
// Unable to ask is not evidence of loss. Kept as applied, asked again next time.
|
||||
console.error(`[provisioner:${resource}] ${g.as}: could not check the backend, will ask again: ${scrub(err, password)}`);
|
||||
// Unable to ask is not evidence of loss. Kept as applied, asked again next time — but a
|
||||
// check that keeps failing is a consumer failed all the same (ADR 0224, issue 179).
|
||||
const text = scrub(err, password);
|
||||
console.error(`[provisioner:${resource}] ${g.as}: could not check the backend, will ask again: ${text}`);
|
||||
await failed(g.as, g.node, classed(err, text), text);
|
||||
// A backend that did not answer in time will not answer for the next consumer either, and
|
||||
// each would cost a timeout, so the rest of this pass is not asked. Any other failure may
|
||||
// be this consumer's alone, and the others are still asked.
|
||||
@@ -356,6 +414,7 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
await adapter.reenable({ as: g.as, derived: was.derived ?? p.derived });
|
||||
}
|
||||
await adapter.create(p);
|
||||
await succeeded(g.as);
|
||||
applied.set(g.as, { hash: h, derived: p.derived, node: g.node });
|
||||
if (retired.has(g.as)) {
|
||||
// Asked for again: the ordinary create re-enabled it and cleared its mark (ADR 0230).
|
||||
@@ -376,7 +435,9 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`[provisioner:${resource}] ${g.as}: create failed, will retry: ${scrub(err, password)}`);
|
||||
const text = scrub(err, password);
|
||||
console.error(`[provisioner:${resource}] ${g.as}: create failed, will retry: ${text}`);
|
||||
await failed(g.as, g.node, classed(err, text), text);
|
||||
// A failed create is not a disagreement: asked again at the next check, the count unchanged.
|
||||
if (reapplying !== undefined) lost.set(g.as, { times: reapplying - 1, nextAt: 0 });
|
||||
}
|
||||
@@ -386,6 +447,12 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
// `stablePasses` passes, and within the bound, or with a person's word (novox/hq ADR 0230).
|
||||
// Only consumers this harness applied, or the backend says the mesh made, are touched.
|
||||
const unasked = [...applied.keys()].filter((as) => !wantByAs.has(as)).sort();
|
||||
// A consumer the mesh stopped asking for that this provider holds nothing for is no longer failed
|
||||
// by anyone: said, so the standing the controller keeps for it is cleared rather than left naming a
|
||||
// consumer that is gone. One still held is said recovered when it is retired (ADR 0224, 0230).
|
||||
for (const as of [...trouble.keys()]) {
|
||||
if (!wantByAs.has(as) && !applied.has(as)) await recovered(as, "no longer asked for");
|
||||
}
|
||||
const key = unasked.join(",");
|
||||
if (pending && pending.consumers.join(",") !== key) {
|
||||
say(`the retirement waiting for a person (${pending.consumers.join(", ")}) is no longer what the mesh says; settled, nothing retired`);
|
||||
@@ -464,6 +531,7 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
}
|
||||
applied.delete(as);
|
||||
lost.delete(as);
|
||||
await recovered(as, "retired");
|
||||
retired.set(as, { node: was.node, at: at.getTime(), why, derived: was.derived, access: adapter.retire ? "disabled" : "kept" });
|
||||
done.push(as);
|
||||
} catch (err) {
|
||||
@@ -485,16 +553,86 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
|
||||
// Said on the bus, never failing the act it reports (the journal still has it).
|
||||
async function tell(change: string, body: Record<string, unknown>): Promise<void> {
|
||||
if (!opts.announce) return;
|
||||
await said(RETIREMENT_EVENT, {
|
||||
provider: resource, "provider-node": node, change, bound: bound(applied.size), at: iso(Date.now()), ...body,
|
||||
}, change);
|
||||
}
|
||||
|
||||
async function said(event: string, body: Record<string, unknown>, what = ""): Promise<void> {
|
||||
try {
|
||||
await opts.announce(RETIREMENT_EVENT, {
|
||||
provider: resource, "provider-node": node, change, bound: bound(applied.size), at: iso(Date.now()), ...body,
|
||||
});
|
||||
await announce(event, body);
|
||||
} catch (err) {
|
||||
say(`could not announce ${RETIREMENT_EVENT} ${change}: ${err}`);
|
||||
say(`could not announce ${event}${what ? ` ${what}` : ""}: ${err}`);
|
||||
}
|
||||
}
|
||||
|
||||
// An error's class: the adapter's word when it has one, else read from the text (ADR 0224).
|
||||
function classed(err: unknown, text: string): string {
|
||||
try {
|
||||
const own = adapter.classify?.(err);
|
||||
if (own) return own;
|
||||
} catch {
|
||||
// An adapter that cannot class its own error is read by the words.
|
||||
}
|
||||
// A check that did not answer in time is a backend out of reach, whatever its words.
|
||||
if (err instanceof HoldsTimeout) return CLASS_UNREACHABLE;
|
||||
return classOf(text);
|
||||
}
|
||||
|
||||
// One more failure in a consumer's unbroken run, announced once the run has lasted failingAfterMs,
|
||||
// then again every sayAgainEveryMs while it lasts (novox/hq ADR 0224).
|
||||
async function failed(as: string, consumerNode: string | undefined, cls: string, text: string): Promise<void> {
|
||||
const now = Date.now();
|
||||
let s = trouble.get(as);
|
||||
if (!s) {
|
||||
s = { since: now, attempts: 0, cls, text, saidAt: 0 };
|
||||
trouble.set(as, s);
|
||||
}
|
||||
s.node = consumerNode;
|
||||
s.cls = cls;
|
||||
s.text = text;
|
||||
s.attempts++;
|
||||
if (now - s.since < failingAfterMs) return;
|
||||
if (s.saidAt !== 0 && now - s.saidAt < sayAgainEveryMs) return;
|
||||
if (s.saidAt === 0) {
|
||||
say(
|
||||
`${as}: FAILING for ${Math.round((now - s.since) / 1000)}s (${s.attempts} attempts, ${cls}): ${text}. ` +
|
||||
`Announced as ${FAILING_EVENT}; \`status\` names it until it succeeds (novox/hq ADR 0224)`,
|
||||
);
|
||||
}
|
||||
s.saidAt = now;
|
||||
await said(FAILING_EVENT, {
|
||||
provider: resource, "provider-node": node, consumer: as, node: consumerNode ?? "",
|
||||
class: cls, error: clip(text), since: iso(s.since), attempts: s.attempts,
|
||||
});
|
||||
}
|
||||
|
||||
// A success ends a consumer's run of failures; one that was announced is announced recovered. And
|
||||
// the first success for a consumer since this process started is announced too, failing or not: a
|
||||
// provider that announced a failure and was restarted has forgotten it, and without this the
|
||||
// controller would name the consumer failing for ever after it recovered unheard.
|
||||
async function succeeded(as: string): Promise<void> {
|
||||
if (!trouble.has(as) && !cleared.has(as)) {
|
||||
cleared.add(as);
|
||||
await said(RECOVERED_EVENT, { provider: resource, "provider-node": node, consumer: as, why: "first-success" });
|
||||
return;
|
||||
}
|
||||
cleared.add(as);
|
||||
await recovered(as, "");
|
||||
}
|
||||
|
||||
async function recovered(as: string, why: string): Promise<void> {
|
||||
const s = trouble.get(as);
|
||||
if (!s) return;
|
||||
trouble.delete(as);
|
||||
if (s.saidAt === 0) return; // never announced, so there is nothing to take back
|
||||
if (!why) say(`${as}: recovered after ${Math.round((Date.now() - s.since) / 1000)}s and ${s.attempts} failed attempt(s)`);
|
||||
await said(RECOVERED_EVENT, {
|
||||
provider: resource, "provider-node": node, consumer: as, node: s.node ?? "",
|
||||
since: iso(s.since), attempts: s.attempts, ...(why ? { why } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
// What the backend holds that the mesh made: active ones become held, so one the mesh stopped asking
|
||||
// for while this process was down is still retired, by the same rules; retired ones are known.
|
||||
async function seed(): Promise<void> {
|
||||
@@ -629,8 +767,8 @@ export function runProvisioner(resource: string, adapter: Adapter, opts: Provisi
|
||||
const stop = () => {
|
||||
stopped = true;
|
||||
};
|
||||
const handle: Provisioner = Object.assign(stop, { retirement, approve, reject, deleteRetired });
|
||||
if (opts.module) registerModuleTools(opts.module, () => provisionerTools(handle));
|
||||
const handle: ProviderLoop = Object.assign(stop, { retirement, approve, reject, deleteRetired });
|
||||
if (opts.module) registerModuleTools(opts.module, () => providerLoopTools(handle));
|
||||
return handle;
|
||||
}
|
||||
|
||||
@@ -639,7 +777,7 @@ const text = (v: unknown): string => (typeof v === "string" ? v : "");
|
||||
|
||||
/** The four tools every provider serves for retirement (novox/hq ADR 0230), the controller's `retire`
|
||||
* and `cleanup` verbs asking them per machine. */
|
||||
export function provisionerTools(p: Provisioner): ToolDefinition[] {
|
||||
export function providerLoopTools(p: ProviderLoop): ToolDefinition[] {
|
||||
const act = { type: "string" };
|
||||
return [
|
||||
{
|
||||
@@ -713,6 +851,27 @@ async function readContributions(
|
||||
return (doc.given as Contribution[]).filter((g) => g.as && g.secret);
|
||||
}
|
||||
|
||||
/** An error's class read from its text — the words the backends the mesh runs use (novox/hq ADR
|
||||
* 0224): the same words, in the same order, as the Go copy of this loop reads. */
|
||||
export function classOf(text: string): string {
|
||||
const t = text.toLowerCase();
|
||||
const credentials = ["invalid_grant", "invalid user credentials", "password authentication failed",
|
||||
"authentication failed", "unauthorized", " 401"];
|
||||
if (credentials.some((w) => t.includes(w))) return CLASS_CREDENTIALS;
|
||||
const unreachable = ["connection refused", "no such host", "i/o timeout", "deadline exceeded",
|
||||
"connection reset", "network is unreachable", "no route to host", "eof",
|
||||
// Node's own words for the same faults.
|
||||
"econnrefused", "enotfound", "etimedout", "econnreset", "ehostunreach", "enetunreach", "eai_again"];
|
||||
if (unreachable.some((w) => t.includes(w))) return CLASS_UNREACHABLE;
|
||||
return CLASS_REFUSED;
|
||||
}
|
||||
|
||||
/** An announced error kept to what belongs in a status line. */
|
||||
function clip(text: string): string {
|
||||
const most = 300;
|
||||
return text.length <= most ? text : `${text.slice(0, most)}…`;
|
||||
}
|
||||
|
||||
/** A check that did not answer in time: the backend, not the consumer, is the likely cause. */
|
||||
class HoldsTimeout extends Error {}
|
||||
|
||||
@@ -751,3 +910,12 @@ function envOrThrow(name: string): string {
|
||||
if (!v) throw new Error(`${name} is not set — the provisioner cannot run without it`);
|
||||
return v;
|
||||
}
|
||||
|
||||
/** @deprecated The provider loop's name before design 57: {@link runProviderLoop}. */
|
||||
export const runProvisioner = runProviderLoop;
|
||||
/** @deprecated {@link providerLoopTools}. */
|
||||
export const provisionerTools = providerLoopTools;
|
||||
/** @deprecated {@link ProviderLoop}. */
|
||||
export type Provisioner = ProviderLoop;
|
||||
/** @deprecated {@link ProviderLoopOptions}. */
|
||||
export type ProvisionerOptions = ProviderLoopOptions;
|
||||
@@ -0,0 +1,90 @@
|
||||
// The sealed box itself (novox/hq ADR 0183, ADR 0206), with its two sources of randomness as arguments so
|
||||
// the tests can prove it byte for byte against the Go package's vectors. Modules import `./index.js`.
|
||||
|
||||
import {
|
||||
createCipheriv,
|
||||
createDecipheriv,
|
||||
createPrivateKey,
|
||||
createPublicKey,
|
||||
diffieHellman,
|
||||
hkdfSync,
|
||||
type KeyObject,
|
||||
} from "node:crypto";
|
||||
|
||||
/** A value sealed to one recipient, as it travels in JSON; every field but `v` is standard base64. */
|
||||
export interface SealedBox {
|
||||
readonly v: number;
|
||||
/** The one-time public key, SPKI DER. */
|
||||
readonly eph: string;
|
||||
readonly iv: string;
|
||||
readonly tag: string;
|
||||
readonly ct: string;
|
||||
}
|
||||
|
||||
/** The only box version `open` opens. */
|
||||
export const VERSION = 1;
|
||||
|
||||
const INFO = "novox-mesh sealed box v1";
|
||||
const IV_BYTES = 12;
|
||||
const TAG_BYTES = 16;
|
||||
|
||||
function x25519(key: KeyObject, what: string): KeyObject {
|
||||
if (key.asymmetricKeyType !== "x25519") throw new Error(`not an X25519 ${what} key`);
|
||||
return key;
|
||||
}
|
||||
|
||||
export function publicFromPem(pem: string): KeyObject {
|
||||
return x25519(createPublicKey({ key: pem, format: "pem" }), "public");
|
||||
}
|
||||
|
||||
export function privateFromPem(pem: string): KeyObject {
|
||||
return x25519(createPrivateKey({ key: pem, format: "pem" }), "private");
|
||||
}
|
||||
|
||||
/** The recipient's raw 32-byte public key: the last 32 bytes of its SPKI DER. */
|
||||
function raw(pub: KeyObject): Buffer {
|
||||
const der = pub.export({ format: "der", type: "spki" });
|
||||
return der.subarray(der.length - 32);
|
||||
}
|
||||
|
||||
function boxKey(secret: Buffer, ephDer: Buffer, recipientRaw: Buffer): Buffer {
|
||||
return Buffer.from(hkdfSync("sha256", secret, Buffer.concat([ephDer, recipientRaw]), INFO, 32));
|
||||
}
|
||||
|
||||
/** `seal` with the one-time private key and the iv given. */
|
||||
export function sealWith(plaintext: string, recipientPem: string, eph: KeyObject, iv: Buffer): SealedBox {
|
||||
const recipient = publicFromPem(recipientPem);
|
||||
x25519(eph, "one-time");
|
||||
if (iv.length !== IV_BYTES) throw new Error(`an iv of ${iv.length} bytes, not ${IV_BYTES}`);
|
||||
const secret = diffieHellman({ privateKey: eph, publicKey: recipient });
|
||||
const ephDer = createPublicKey(eph).export({ format: "der", type: "spki" });
|
||||
const cipher = createCipheriv("aes-256-gcm", boxKey(secret, ephDer, raw(recipient)), iv);
|
||||
const ct = Buffer.concat([cipher.update(plaintext, "utf8"), cipher.final()]);
|
||||
return {
|
||||
v: VERSION,
|
||||
eph: ephDer.toString("base64"),
|
||||
iv: iv.toString("base64"),
|
||||
tag: cipher.getAuthTag().toString("base64"),
|
||||
ct: ct.toString("base64"),
|
||||
};
|
||||
}
|
||||
|
||||
/** Open a box with the recipient's PEM private key; throws for a box to another key, changed, or of another version. */
|
||||
export function open(box: SealedBox, privatePem: string): string {
|
||||
if (box.v !== VERSION) throw new Error(`a sealed box of version ${box.v}, not ${VERSION}`);
|
||||
const priv = privateFromPem(privatePem);
|
||||
const ephDer = Buffer.from(box.eph, "base64");
|
||||
const eph = x25519(createPublicKey({ key: ephDer, format: "der", type: "spki" }), "one-time");
|
||||
const iv = Buffer.from(box.iv, "base64");
|
||||
const tag = Buffer.from(box.tag, "base64");
|
||||
// node accepts a shorter tag than 16 bytes; the box never has one, so a short one is refused here.
|
||||
if (iv.length !== IV_BYTES || tag.length !== TAG_BYTES) throw new Error("the box does not open with this key");
|
||||
const secret = diffieHellman({ privateKey: priv, publicKey: eph });
|
||||
const decipher = createDecipheriv("aes-256-gcm", boxKey(secret, ephDer, raw(createPublicKey(priv))), iv);
|
||||
decipher.setAuthTag(tag);
|
||||
try {
|
||||
return Buffer.concat([decipher.update(Buffer.from(box.ct, "base64")), decipher.final()]).toString("utf8");
|
||||
} catch {
|
||||
throw new Error("the box does not open with this key");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
// Sealing to one recipient (novox/hq ADR 0183, ADR 0206): X25519 for the agreement, HKDF-SHA256 for the
|
||||
// key, AES-256-GCM for the box. The TypeScript twin of the Go package `go/seal`, byte for byte: the same
|
||||
// fixtures prove both. A box is `{ v: 1, eph, iv, tag, ct }`; keys travel as PEM (SPKI public, PKCS #8
|
||||
// private).
|
||||
|
||||
import { generateKeyPairSync, randomBytes } from "node:crypto";
|
||||
import { sealWith, open, VERSION, type SealedBox } from "./box.js";
|
||||
|
||||
export { open, VERSION, type SealedBox };
|
||||
|
||||
/** A recipient's keypair as the two PEM strings it is kept and sent as. */
|
||||
export interface KeyPair {
|
||||
readonly publicKey: string;
|
||||
readonly privateKey: string;
|
||||
}
|
||||
|
||||
/** A fresh X25519 keypair, PEM-encoded. */
|
||||
export function generateKeyPair(): KeyPair {
|
||||
const { publicKey, privateKey } = generateKeyPairSync("x25519");
|
||||
return {
|
||||
publicKey: publicKey.export({ format: "pem", type: "spki" }) as string,
|
||||
privateKey: privateKey.export({ format: "pem", type: "pkcs8" }) as string,
|
||||
};
|
||||
}
|
||||
|
||||
/** Seal plaintext to the recipient's PEM public key, with a fresh one-time key and iv. */
|
||||
export function seal(plaintext: string, recipientPem: string): SealedBox {
|
||||
return sealWith(plaintext, recipientPem, generateKeyPairSync("x25519").privateKey, randomBytes(12));
|
||||
}
|
||||
+8
-7
@@ -8,7 +8,7 @@ import { mkdtemp, writeFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { runProvisioner, provisionerTools, RETIREMENT_EVENT, type Provisioner } from "../dist/provisioner/index.js";
|
||||
import { runProviderLoop, providerLoopTools, RETIREMENT_EVENT, type ProviderLoop } from "../dist/provider-loop/index.js";
|
||||
import { resetTools, collectTools } from "../dist/tools/index.js";
|
||||
|
||||
const EVERY = 10;
|
||||
@@ -32,7 +32,7 @@ interface World {
|
||||
reenabled: string[];
|
||||
events: Record<string, unknown>[];
|
||||
said: string[];
|
||||
p: Provisioner;
|
||||
p: ProviderLoop;
|
||||
done(): void;
|
||||
}
|
||||
|
||||
@@ -62,9 +62,10 @@ async function world(
|
||||
...(opts.retire === false ? {} : { async retire(p: { as: string; why: string }) { w.retired.push({ as: p.as, why: p.why }); } }),
|
||||
...(opts.held ? { async held() { return opts.held!; } } : {}),
|
||||
};
|
||||
w.p = runProvisioner("postgres-database", adapter, {
|
||||
w.p = runProviderLoop("postgres-database", adapter, {
|
||||
receives, everyMs: EVERY, node: "anchor", stableForMs: opts.stableForMs ?? 0,
|
||||
announce: (event, body) => { assert.equal(event, RETIREMENT_EVENT); w.events.push(body); },
|
||||
// Only retirement is these tests'; the standing (provisioner.failing, .recovered) is standing.test.ts's.
|
||||
announce: (event, body) => { if (event === RETIREMENT_EVENT) w.events.push(body); },
|
||||
});
|
||||
// A pass is counted by the file being read: wait wall time for n passes and some slack.
|
||||
w.done = () => { w.p(); console.error = was; };
|
||||
@@ -167,7 +168,7 @@ test("five passes are not enough: the same set must also hold for stableForMs",
|
||||
});
|
||||
|
||||
test("the default hold is ten minutes", async () => {
|
||||
const { STABLE_FOR_MS } = await import("../dist/provisioner/index.js");
|
||||
const { STABLE_FOR_MS } = await import("../dist/provider-loop/index.js");
|
||||
assert.equal(STABLE_FOR_MS, 10 * 60_000);
|
||||
});
|
||||
|
||||
@@ -326,7 +327,7 @@ test("the four tools are registered for the module and answer", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "retire-tools-"));
|
||||
const receives = join(dir, "r.json");
|
||||
await writeFile(receives, JSON.stringify({ requirement: "r", given: [] }));
|
||||
const p = runProvisioner("r", { async create() {}, async remove() {} }, { receives, everyMs: EVERY, module: "postgres" });
|
||||
const p = runProviderLoop("r", { async create() {}, async remove() {} }, { receives, everyMs: EVERY, module: "postgres" });
|
||||
try {
|
||||
const mine = collectTools({}).find((c) => c.module === "postgres");
|
||||
assert.deepEqual(mine?.tools.map((t) => t.name), [
|
||||
@@ -335,7 +336,7 @@ test("the four tools are registered for the module and answer", async () => {
|
||||
const answer = (await mine!.tools[0].run({})) as { resource: string; waiting: unknown };
|
||||
assert.equal(answer.resource, "r");
|
||||
assert.equal(answer.waiting, null);
|
||||
assert.equal(provisionerTools(p).length, 4);
|
||||
assert.equal(providerLoopTools(p).length, 4);
|
||||
} finally {
|
||||
p();
|
||||
resetTools();
|
||||
|
||||
+26
-13
@@ -6,7 +6,7 @@ import { join } from "node:path";
|
||||
|
||||
import { compareVersions } from "../dist/primitives/index.js";
|
||||
import { registerModuleTools, collectTools, collectToolsEach, resetTools, serveTools, listTools, invokeTool } from "../dist/tools/index.js";
|
||||
import { runProvisioner } from "../dist/provisioner/index.js";
|
||||
import { runProviderLoop } from "../dist/provider-loop/index.js";
|
||||
import { emit, on, type Event } from "../dist/events/index.js";
|
||||
import { useBroker } from "../dist/messaging/index.js";
|
||||
|
||||
@@ -47,7 +47,7 @@ test("provisioner creates each consumer with the mesh's login and password; remo
|
||||
doc([{ from: "webapp", node: "anchor", as: "webapp-anchor", secret: join(dir, "webapp.secret"), values: { name: "webapp" } }]),
|
||||
);
|
||||
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"analytics",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -101,7 +101,7 @@ test("provisioner hands the adapter what the mesh derived, and still has it at w
|
||||
});
|
||||
await writeFile(receives, doc([entry("mesh-anchor-ncloud")]));
|
||||
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"s3-bucket",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -153,7 +153,7 @@ test("provisioner stops calling an unreadable secret a race, and says what to lo
|
||||
console.error = (line: unknown) => { said.push(String(line)); };
|
||||
let stop: (() => void) | undefined;
|
||||
try {
|
||||
stop = runProvisioner("db", { async create() {}, async remove() {} }, { receives, everyMs: 5 });
|
||||
stop = runProviderLoop("db", { async create() {}, async remove() {} }, { receives, everyMs: 5 });
|
||||
await waitFor(() => said.some((l) => l.includes("CANNOT READ")), 3000);
|
||||
} finally {
|
||||
stop?.();
|
||||
@@ -185,7 +185,7 @@ test("provisioner removes nothing while its contributions file cannot be used (n
|
||||
]);
|
||||
await writeFile(receives, both);
|
||||
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"postgres-database",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -239,7 +239,7 @@ test("provisioner applies again what the backend no longer holds, and trusts mem
|
||||
const backend = new Set<string>();
|
||||
let creates = 0;
|
||||
let asked = 0;
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -271,7 +271,7 @@ test("provisioner applies again what the backend no longer holds, and trusts mem
|
||||
// An adapter without holds is trusted from memory, as before: a forgotten backend stays forgotten.
|
||||
const forgetful = new Set<string>();
|
||||
let plainCreates = 0;
|
||||
const stopPlain = runProvisioner(
|
||||
const stopPlain = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -299,7 +299,7 @@ test("provisioner keeps a consumer applied when the backend cannot be asked", as
|
||||
);
|
||||
let creates = 0;
|
||||
let asked = 0;
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create() {
|
||||
@@ -330,7 +330,7 @@ test("provisioner backs off when create does not satisfy holds, and says so", as
|
||||
const original = console.error;
|
||||
console.error = (...a: unknown[]) => logged.push(a.join(" "));
|
||||
try {
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create() {
|
||||
@@ -371,7 +371,7 @@ test("provisioner treats a failed or hung holds as could-not-ask, and never logs
|
||||
const original = console.error;
|
||||
console.error = (...a: unknown[]) => logged.push(a.join(" "));
|
||||
try {
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create() {
|
||||
@@ -404,7 +404,7 @@ test("provisioner treats a failed or hung holds as could-not-ask, and never logs
|
||||
const hungLogged: string[] = [];
|
||||
console.error = (...a: unknown[]) => hungLogged.push(a.join(" "));
|
||||
try {
|
||||
const stopHung = runProvisioner(
|
||||
const stopHung = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create() {
|
||||
@@ -438,7 +438,7 @@ test("provisioner stops a verify pass at a timeout, and retries a failed re-appl
|
||||
try {
|
||||
// A timeout on "a" means "b" is not asked in that pass.
|
||||
const askedFor: string[] = [];
|
||||
const stopHung = runProvisioner(
|
||||
const stopHung = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create() {},
|
||||
@@ -460,7 +460,7 @@ test("provisioner stops a verify pass at a timeout, and retries a failed re-appl
|
||||
let createCalls = 0;
|
||||
const logged: string[] = [];
|
||||
console.error = (...a: unknown[]) => logged.push(a.join(" "));
|
||||
const stop = runProvisioner(
|
||||
const stop = runProviderLoop(
|
||||
"cache",
|
||||
{
|
||||
async create(p) {
|
||||
@@ -649,3 +649,16 @@ test("each registration is given the environment asked for it, and sees no other
|
||||
assert.deepEqual(seen.beta, { BETA_URL: "http://127.0.0.1:3000" });
|
||||
resetTools();
|
||||
});
|
||||
|
||||
test("the provider loop is named for what it is, and its old name stays an alias until its providers move (issue 678)", async () => {
|
||||
const loop = await import("../dist/provider-loop/index.js");
|
||||
const old = await import("../dist/index.js");
|
||||
assert.equal(old.provisioner.runProvisioner, loop.runProviderLoop);
|
||||
assert.equal(old.provisioner.provisionerTools, loop.providerLoopTools);
|
||||
assert.equal(old.providerLoop.runProviderLoop, loop.runProviderLoop);
|
||||
const pkg = JSON.parse(await (await import("node:fs/promises")).readFile(new URL("../package.json", import.meta.url), "utf8"));
|
||||
assert.equal(pkg.exports["./provider-loop"], "./dist/provider-loop/index.js");
|
||||
assert.equal(pkg.exports["./provisioner"], pkg.exports["./provider-loop"]);
|
||||
// The events and tools are what the controller and the operator read on the bus: their names stay.
|
||||
assert.equal(loop.FAILING_EVENT, "provisioner.failing");
|
||||
});
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { createPrivateKey } from "node:crypto";
|
||||
|
||||
import { seal, open, generateKeyPair, type SealedBox } from "../dist/seal/index.js";
|
||||
import { sealWith } from "../dist/seal/box.js";
|
||||
|
||||
// The fixtures the Go package proves itself against: boxes the licence manager's own seal.go made, and
|
||||
// the one the agent module's TypeScript made. One format, three implementations.
|
||||
const fixtures = new URL("../go/seal/testdata/", import.meta.url);
|
||||
const lm = JSON.parse(readFileSync(new URL("sealed-by-licence-manager.json", fixtures), "utf8")) as {
|
||||
recipient: { publicKey: string; privateKey: string };
|
||||
sealed: { plaintext: string; box: SealedBox }[];
|
||||
vectors: { plaintext: string; ephPrivate: string; iv: string; box: SealedBox }[];
|
||||
};
|
||||
|
||||
// PKCS #8 for an X25519 private key is this prefix and the 32 raw bytes.
|
||||
const pkcs8 = (raw: Buffer) =>
|
||||
createPrivateKey({ key: Buffer.concat([Buffer.from("302e020100300506032b656e04220420", "hex"), raw]), format: "der", type: "pkcs8" });
|
||||
|
||||
test("seal: every box the licence manager sealed opens to its plaintext", () => {
|
||||
assert.ok(lm.sealed.length > 0);
|
||||
for (const s of lm.sealed) assert.equal(open(s.box, lm.recipient.privateKey), s.plaintext);
|
||||
});
|
||||
|
||||
test("seal: given the same one-time key and iv, the box is byte for byte the licence manager's", () => {
|
||||
assert.ok(lm.vectors.length > 0);
|
||||
for (const v of lm.vectors) {
|
||||
const box = sealWith(v.plaintext, lm.recipient.publicKey, pkcs8(Buffer.from(v.ephPrivate, "base64")), Buffer.from(v.iv, "base64"));
|
||||
assert.deepEqual(box, v.box);
|
||||
assert.equal(open(box, lm.recipient.privateKey), v.plaintext);
|
||||
}
|
||||
});
|
||||
|
||||
test("seal: the agent module's TypeScript box opens", () => {
|
||||
const f = JSON.parse(readFileSync(new URL("sealed-by-typescript.json", fixtures), "utf8"));
|
||||
assert.equal(open(f.box, f.privateKey), f.plaintext);
|
||||
});
|
||||
|
||||
test("seal: a box opens for its recipient only, and two boxes share no one-time key or iv", () => {
|
||||
const a = generateKeyPair();
|
||||
const b = generateKeyPair();
|
||||
const box = seal("a token", a.publicKey);
|
||||
assert.equal(open(box, a.privateKey), "a token");
|
||||
assert.throws(() => open(box, b.privateKey));
|
||||
const again = seal("a token", a.publicKey);
|
||||
assert.notEqual(again.eph, box.eph);
|
||||
assert.notEqual(again.iv, box.iv);
|
||||
});
|
||||
|
||||
test("seal: a box changed in any field, or of another version, does not open", () => {
|
||||
const good = lm.sealed[1].box;
|
||||
const flip = (s: string) => {
|
||||
const b = Buffer.from(s, "base64");
|
||||
b[b.length - 1] ^= 1;
|
||||
return b.toString("base64");
|
||||
};
|
||||
for (const box of [
|
||||
{ ...good, ct: flip(good.ct) },
|
||||
{ ...good, tag: flip(good.tag) },
|
||||
{ ...good, iv: flip(good.iv) },
|
||||
{ ...good, eph: flip(good.eph) },
|
||||
{ ...good, v: 2 },
|
||||
]) assert.throws(() => open(box as SealedBox, lm.recipient.privateKey));
|
||||
});
|
||||
|
||||
test("seal: a key that is not X25519 PEM is refused", () => {
|
||||
assert.throws(() => seal("x", "not a key"));
|
||||
assert.throws(() => open(lm.sealed[0].box, lm.recipient.publicKey));
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user