46 Commits
Author SHA1 Message Date
mesh-tools 30b2a72f9b Merge pull request 'node-tools: a probe line, so node-tools gets a pipeline on a revision that is kept' (#76) from probe/node-tools-pipeline-4 into main 2026-11-16 22:49:02 +00:00
jschoubben 470e57124e node-tools: a probe line, so node-tools gets a pipeline on a revision that is kept 2026-11-16 23:48:42 +01:00
mesh-tools e554402e47 Merge pull request 'node-tools: a probe line, so node-tools builds against mesh-sdk go/v0.1.16' (#75) from probe/node-tools-pipeline-3 into main 2026-11-16 22:17:42 +00:00
jschoubben ef0a61d348 node-tools: a probe line, so node-tools builds against mesh-sdk go/v0.1.16 2026-11-16 23:17:31 +01:00
mesh-tools 42a752f052 Merge pull request 'node-tools: a probe line, so node-tools gets a pipeline again' (#74) from probe/node-tools-pipeline-2 into main 2026-11-16 21:55:42 +00:00
jschoubben 4043dd1176 node-tools: a probe line, so node-tools gets a pipeline again (main.970 met node-engines older than G) 2026-11-16 22:55:30 +01:00
mesh-tools b232e39c96 Merge pull request 'node-tools: a probe line, so node-tools gets a pipeline again' (#73) from probe/node-tools-pipeline into main 2026-11-16 20:38:28 +00:00
jschoubben 20b6b02341 node-tools: a probe line, so node-tools gets a pipeline again (its composition step at main.969 was lost with the assigner it ran on) 2026-11-16 21:38:17 +01:00
mesh-tools 72b0d4ca4e Merge pull request 'Design 47 G start: tool runner fixes placed by hand' (#72) from fix/47-G-start-3 into main 2026-11-16 14:15:09 +00:00
jschoubben 4333d5b944 node-tools: the tool runner admits every caller of a seat's verb
The operator, 2026-11-15 23:25 CET: every module, agent and operator may call anything; gates and guards can be added
later. The runner refused every call of the knowledge base and the issue tracker while it had not read their seat
definitions. The approval mark stays, held by the gate.
2026-11-15 23:27:12 +01:00
jschoubben 9e8bf6dc0c Runner: a get session's object reader reads on a context that ends with the session, not with the call that opened it
Builds: design 47 G §490, §518, §620 (a get over one chunk is a session read chunk by chunk)
2026-11-15 01:45:37 +01:00
jschoubben 45fbc813df Runner: a page of a state reads every current value in one stream, not a bus request per key: a page of the assignments took 13 s from a node far from the bus 2026-11-14 21:47:28 +01:00
jschoubben 3d6daad925 Runner: reads each module's events on its own durable <node>_<module>_tools, beside the program's (A-397) 2026-11-14 18:58:20 +01:00
jschoubben 903ac4ce93 Runner: a module's marks are checked once against all its bundles (A-398)
Live on novox every gitea tool was refused: each of gitea's eight bundles overwrote the gate's registered set, so the last one, which does not carry gitea_branch_protection_loosen, made the mark unserved.
2026-11-14 14:36:42 +01:00
jschoubben 9fa29ad801 Tool runner: serve a node-scoped seat's verb also with no machine, in a queue group
A verb asked with no machine (the indexer's seed asks node-build-go's steps) is
answered by any one holder, on the tool and the caller-named subject.

Builds: lead answer A-393 (design 47 G)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-14 11:33:57 +01:00
jschoubben 30027347e9 Tool runner: hear an adopted seat event by its key's prefixes, say an unheard key once
The mesh-catalog seat definition publishes catalog-revision on
mesh.seat.mesh-catalog.event.catalog-revision.<branch>, so the bus key is
mesh-catalog.catalog-revision.<branch>; the exact-key lookup missed and the event
was dropped silently. Look up the key and its shorter prefixes (as mesh-sdk's
views.adoptedHears does) and say an event no view hears once per key.

Builds: design 47 G (catalog half's revision events)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-14 04:26:14 +01:00
jschoubben 7a6174bb3e Reach an old objects entry by the store grants creates for it, not by a membership field (A-380)
Builds: design 47 G A-380 (corrects A-379)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 23:19:04 +01:00
jschoubben faa1064214 Fill the gate's Call.Env from the bundle's composed environment (A-377)
Builds: design 47 G A-377

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 23:17:54 +01:00
jschoubben 27a45d414d Reach a module's old objects section by its membership when no binding names it (A-379)
Builds: design 47 G A-379

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 23:17:29 +01:00
mesh-tools a96df478d4 Merge pull request 'Milestone 47 G: mesh-tools's part of the one release' (#71) from feat/47-G-join12 into main 2026-11-13 13:09:37 +00:00
jschoubben 6be5679a0d Merge main into feat/47-G-join11: the tool runner's mesh/stream.publish (issue 1979)
mesh/delivery merged: source.main moved to novox/mesh-tools@a96df478d498
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 6be5679a0d58 on a96df478d498
The capability stream-publish is stated under mesh-sdk's word (buscalls.StreamPublishWord) and in the runtime artifact's states, so G's test that the stated words are the manifest's states holds; the publish uses G's exported JetStream.

Builds: design 47 G §545, §622; design 42 zfs-owns-zfs A-42, A-44

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools af83854956 Merge pull request 'The tool runner publishes a bundle's event onto its own bound events stream (issue 1979)' (#70) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:53 +00:00
jschoubben df2162e381 tool runner: mesh/stream.publish is the capability stream-publish, taken out of bus-state-calls (A-44)
mesh/delivery merged: source.main moved to novox/mesh-tools@af8385495688
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to df2162e381f5 on af8385495688 | from 447a8d035935, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-host 0193949…
Builds: design 42 zfs-owns-zfs §25, lead answer A-44
2026-11-13 01:52:15 +01:00
jschoubben a775b42e4d tool runner: mesh/stream.publish says an event on the calling bundle's own bound stream (A-42)
Builds: design 42 zfs-owns-zfs §25, §24, lead answer A-42
2026-11-13 01:49:02 +01:00
jschoubben 481a83e11c Round 2 T2, T3, T4, T5: the tool runner ends G with its merge-base test count; the wait test runs its rows side by side
mesh/delivery-group group feat/47-G-join11 checking: 15 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 481a83e11c4f on af8385495688 | novox/mesh-tools#69 in group feat/47-G-join11
The tests G added fold into existing ones as subtests (design 47 G §626, A-306: 171 tests down to 154, integration 9
down to 7). TestMeshWaitRefusesLoudlyAndNeverLoops runs its groups of one-second rows as parallel subtests, each
on a runtime of its own, 15 s down to 5 s (A-311: the one-second floor of a wait's timeout stands, no clock is
injected). The go.mod no longer requires meshconditionsclient, which no package loads (T4). Node names in the
lines G adds are role names (T5).

Builds: design 47 G §626, design 34 every-event §19, A-306, A-311; review round 2 findings T2, T3, T4, T5
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 22:24:02 +01:00
jschoubben 28bece26b0 Join 11 (try 2): the chunked-answer tests flush the holder's verb before another connection asks it
The tests of A-249 and A-257 served the verb on one connection and asked it from another at once; the
subscription was still in the holder's buffer when the ask reached the bus, so the ask was answered 503
no responders (13 of 300 runs with -cpu 1 before, 0 of 2000 after). The runner flushes after serving
(runtime.go:352, :807); the tests now do as it does.

Builds: design 47 G A-249, A-257
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 15:14:01 +01:00
jschoubben 816df217cd Join 11: every go.mod requires, at the zero version, each mesh module its packages load
The checker's fetch pins only the zero-version requirements a directory's go.mod names
(mesh-build-go workspace.go goModOf), so a module reached only through another, such as
meshhandacts through meshhandactsclient or mesh-delivery-sdk through meshcatalogclient,
was never downloaded and the shut-off load failed (fetch.go:267). The join's merges had
dropped these indirect requirements that the trunk's go.mod files list.

Builds: design 47 G (join 11, the check's fetch of the group's modules)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 14:07:29 +01:00
jschoubben c35c27f664 Area 1 (part): the tool runner's manifest states approval-mark/1
The runner states the approval mark's word beside its call families, so its
artifact's states are exactly what it says (the join of A3's and A5's lines);
TestTheWordsTheRunnerStatesAreItsManifestsStates failed without it.

Builds: design 47 G §545, design 46 7-approval-mark §6; join 11 area 1

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:44:02 +01:00
jschoubben d350c98823 Join 11: the merged heads compile where no area's fix is owed
The join's importers name the delivery repository's moved contracts and sdk
(MV1), node-tools' tests use mesh-sdk's gate, the duplicate node-engine claim
rule of the reader release gives way to G's, and the assigner names its first
watched branch, never a constant. The seat holders' AnswerSeat callers are
area 6's.

Builds: design 47 G join 11 (merge of fix round 11)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:41:44 +01:00
jschoubben 29757dabc3 Merge branch 'feat/47-G-fix11-a3h' into feat/47-G-join11
# Conflicts:
#	node-tools/internal/runtime/runtime.go
2026-11-12 10:27:48 +01:00
jschoubben 95a265e730 Merge branch 'feat/47-G-fix11-c1c' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben d6c267d2f0 Merge branch 'feat/47-G-fix11-b5g3' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben e03d8d3c7b Merge branch 'feat/47-G-fix11-b4b' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben a91386b3b5 Merge branch 'feat/47-G-fix11-a5' into feat/47-G-join11
# Conflicts:
#	node-tools/internal/firstview/firstview_test.go
#	node-tools/internal/runtime/announce_test.go
#	node-tools/internal/runtime/busstate.go
#	node-tools/internal/runtime/ledgers_test.go
2026-11-12 10:27:27 +01:00
jschoubben 8b980a9480 Merge branch 'feat/47-G-fix11-a3' into feat/47-G-join11 2026-11-12 10:27:01 +01:00
jschoubben 9fcb14e741 A3h item 1: the tool runner serves a held seat's approvals from its gate's record
The seat verb approvals is the gate's own, served wherever a marked verb is: the runner answers it by caller from the
calls its gate holds, with no bundle naming it; a marked verb already reaches the gate by caller. Tested with zsh's
node-login-shell execute as the case.

Builds: design 47 G §677, §697; lead answers A-264, A-265, A-266, A-271

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:21:21 +01:00
jschoubben 14f464e8bc C1c-4: the tool runner sends a bundle's chunked seat answer in chunks, and a node's chunked verb is asked of the node
A seat verb whose definition says chunks, served by a bundle that answers the whole file as the member `file`, is sent by the
runner with buscalls.SeatChunks.Send and answered with the chunk count (Conn.AnswerChunked, called from serveSeats). A chunked verb
asked of one machine (`@<node>`) is received in chunks as the verb is.

Builds: design 47 G A-257, A-249, A-254, §681

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:43:08 +01:00
jschoubben 2d95c0eb59 AM-D3 path: the tool runner states its own fingerprint beside its capabilities
A capability reading must be tied to the runner that made it, so the grants
part can tell an older runner's reading from the running one's.

Builds: design 46 7 §6 (the announcement carries the runner's fingerprint), lead answer A-203 (key `fingerprint`)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:02:18 +01:00
jschoubben b5c7bf13ea A-9 tests: a bundle not serving its marked tool is refused at the load; a withheld marked call is said so
The runtime's load test gains rows (run with the Python fixture, before the bundles' skip): a bundle whose
marks name a tool it does not register serves no tool of its module, and one that registers it serves.
mesh_call's failure words move into Surface.callFailed, whose rows hold that only a marked tool reaching no
responder names withheld and mesh-grants.grants_user_list.

Builds: design 46 7-approval-mark §4, §6, §22

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:54:49 +01:00
jschoubben 99f8c8c050 Finding 6: every module requires the mesh modules it builds with
The SDK, the assigner, the builder's buildrun, the indexer, the
pipeline, the watchman, mesh-enrolment and mesh-grants (and through the
SDK the checker's checkermsg and mesh-tools' node-tools) imported
packages of modules their go.mod did not require at the zero version;
the placement's go.work over the whole repository hid it, and a build
of the module alone with -mod=readonly refused it. Each now requires
them (beside a relative replace where the module keeps its siblings so),
and the indexer's and watchman's duplicated lines go.

Builds: design 47 G §427 (the group's check builds every importer); delivery review round 1, finding 6

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:21:14 +01:00
jschoubben 24f257e929 MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben e63d707eac R-11: mesh/publish with ack answers its sequence or {error}, mesh/subscribe with deliver new makes an ordered consumer; finding-calls is stated
The subscription's key resolves through mesh-sdk's buscompose.EventSubjects (lead answer A-153; the sdk commit EE-S1 is
cherry-picked into this change's sdk placement), a wildcard is refused.

Builds: design 47 G §559, §488, §576, §622

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:04:18 +01:00
jschoubben 5e373af70d R-1 rest, R-12, R-13: the capability words are mesh-sdk's, the ledger's fill and removal are answered, the artifact states the words
sent-get is under sent-objects-calls; mesh/ledger.fill answers the history's fill against its cap and mesh/ledger.remove
removes an instance's operations, snapshot, state and index entries, both under ledger-upkeep-calls; the runtime artifact's
`states` is exactly the words the runner states.

Builds: design 47 G §462, §488, §545, §587, §622, §155

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:02:04 +01:00
jschoubben 64015b5952 R-8: the tool runner's own fingerprint reaches no bundle; a bundle reads the one its module is composed with
Builds: design 47 G §519, §618

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:57:42 +01:00
jschoubben e873cb3b86 R-7: mesh_first_views names the tool runner's own build, or says own build unknown
Builds: design 47 G §205, §424

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:57:12 +01:00
jschoubben e88e0c825c R-5: the tool runner follows the assigner's views of the module that claims the seat, named by the seat
The holder is read from the catalog half's entries serving a verb of mesh-module-assigner, on the branch follower's change
callback; the manifest's consumes names the two views by the seat (seat:mesh-module-assigner.views.*).

Builds: design 47 G §616, §428, §7

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:33 +01:00
58 changed files with 2182 additions and 662 deletions
@@ -12,7 +12,7 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
seat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"github.com/nats-io/nats.go"
+1
View File
@@ -1,4 +1,5 @@
// node-tools — the node's tool runtime, in Go (novox/hq ADR 0175, ADR 0193; to-be 38 WP4d).
// (Its pipeline is asked again by this line, 2026-11-16 23:45: main.981 passed all but composition, whose revision 433 was removed; the assigner now waits 3 minutes for composing nodes.)
//
// One process per machine: it connects to the bus on the node's credential, launches every assigned
// module's tools bundle and serves its tools and its seats' verbs; as the node-tools module — or
+6 -3
View File
@@ -5,14 +5,13 @@ go 1.26.0
require (
git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshhandacts v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
github.com/nats-io/nats.go v1.54.0
golang.org/x/sys v0.48.0
)
@@ -30,3 +29,7 @@ require (
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
require git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect
require git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk v0.0.0-00010101000000-000000000000 // indirect
+5
View File
@@ -48,6 +48,11 @@ const Digest = "digest"
// module needing one is placed only where the runtime states it.
const Capabilities = "capabilities"
// Fingerprint is the service metadata key the tool runner states its own fingerprint under, beside Capabilities: the build
// its composed environment names (MESH_FINGERPRINT, design 47 G §424), so the node-engine ties the capabilities it takes
// into the node's record to the runner that stated them (design 46 approval mark §6, lead answer A-203).
const Fingerprint = "fingerprint"
// Shortened is the service metadata key a runtime sets when its announcement was cut to fit the bus.
const Shortened = "shortened"
+84 -1
View File
@@ -3,6 +3,7 @@ package approval
import (
"encoding/json"
"errors"
"reflect"
"strings"
"sync"
"testing"
@@ -10,7 +11,7 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
seat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/keys"
@@ -506,4 +507,86 @@ func TestTheRefusalsAndAnswers(t *testing.T) {
} {
t.Run(tc.name, func(t *testing.T) { tc.run(t, newWorld()) })
}
// Design 46 7-approval-mark §5, §3, §4 and design 34 every-event §33: the marks a module's calls are gated by, derived from
// fixture catalog halves and served entries.
t.Run("TheMarks", func(t *testing.T) {
login := map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "sessions"}}}}
v1 := entry("src2:1", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"wipe"}})
v2 := entry("src2:2", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"erase", "node-login-shell.sessions"}})
mail := catalogseat.CatalogEntry{Module: "mail", Fp: "src2:m", Manifest: map[string]any{
"needs-approval": []any{"mail.received"}, "emits": []any{"mail.received"},
"streams": []any{map[string]any{"name": "MAIL", "events": []any{"mail.**"}, "grant": map[string]any{"config": "readers", "events": "mail.{key}.**"}}}}}
for _, tc := range []struct {
name string
cat *fakeCatalog
module string
fp string
tools []string
verbs map[string]string
subjects []string
unread bool
}{
{name: "a seat's mark stands for every holder, and the holder's own adds up", cat: &fakeCatalog{revision: "7",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell"}},
{name: "a holder's needs-approval cannot unmark its seat's verb, and marks another", cat: &fakeCatalog{revision: "8",
byFp: map[string]catalogseat.CatalogEntry{"src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:2", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "across a version change the union stands: a dropped mark is asked while the old bundle runs", cat: &fakeCatalog{revision: "9",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1, "src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase", "wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "a served entry that cannot be read leaves the newest marks, every marked call refused", cat: &fakeCatalog{revision: "9",
newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}, unread: true},
{name: "a bundle with no fingerprint has no served entry", cat: &fakeCatalog{revision: "9", defs: login},
module: "shell", unread: true},
{name: "a marked event and a stream with a grant give their subjects and each .>", cat: &fakeCatalog{revision: "3",
byFp: map[string]catalogseat.CatalogEntry{"src2:m": mail}, newest: map[string]catalogseat.CatalogEntry{"mail": mail}},
module: "mail", fp: "src2:m", subjects: []string{"mesh.mod.mail.event.mail.received", "mesh.mod.mail.event.mail.received.>",
"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.received", "mesh.stream.mail.event.mail.received.>"}},
} {
t.Run(tc.name, func(t *testing.T) {
m := gate.Derive(tc.cat, tc.module, tc.fp)
if got := keysOf(m.Tools); !reflect.DeepEqual(got, tc.tools) {
t.Errorf("tools %v, want %v", got, tc.tools)
}
if tc.verbs == nil {
tc.verbs = map[string]string{}
}
if !reflect.DeepEqual(m.Verbs, tc.verbs) {
t.Errorf("verbs %v, want %v", m.Verbs, tc.verbs)
}
if got := keysOf(m.Subjects); !reflect.DeepEqual(got, tc.subjects) {
t.Errorf("subjects %v, want %v", got, tc.subjects)
}
if (m.Unread != "") != tc.unread || m.Revision != tc.cat.revision {
t.Errorf("unread %q, revision %q", m.Unread, m.Revision)
}
})
}
// §4: a bundle whose marks name a tool it does not register is not loaded, its finding mark-unserved raised; a bundle that
// serves it clears it, and a catalog without the mark too.
t.Run("mark-unserved raised and cleared", func(t *testing.T) {
cat := &fakeCatalog{revision: "7", byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login}
w := &findings{}
g := gate.New(gate.Deps{Node: "n1", Runner: "node-tools", Catalog: cat, Findings: Findings(w, "n1", "node-tools", nil)})
if got := g.Unserved("shell", "src2:1", []string{"other"}); !reflect.DeepEqual(got, []string{"wipe"}) || w.raised[keys.EscapeName("n1.shell")] != gate.KindMarkUnserved {
t.Fatalf("unserved %v, findings %v", got, w.raised)
}
if got := g.Unserved("shell", "src2:1", []string{"wipe"}); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a bundle serving the mark: unserved %v, findings %v", got, w.raised)
}
_ = g.Unserved("shell", "src2:1", nil)
cat.byFp["src2:1"], cat.newest["shell"] = entry("src2:1", nil, nil), entry("src2:1", nil, nil)
if got := g.Unserved("shell", "src2:1", nil); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a catalog without the mark: unserved %v, findings %v", got, w.raised)
}
})
})
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
assignerseat "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/views"
)
+1 -87
View File
@@ -2,15 +2,12 @@ package approval
import (
"context"
"reflect"
"sort"
"testing"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/views"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
@@ -47,89 +44,6 @@ func entry(fp string, serves []string, manifest map[string]any) catalogseat.Cata
return catalogseat.CatalogEntry{Module: "shell", Fp: fp, Serves: serves, Manifest: manifest}
}
// Design 46 7-approval-mark §5, §3, §4 and design 34 every-event §33: the marks a module's calls are gated by, derived from
// fixture catalog halves and served entries.
func TestTheMarks(t *testing.T) {
login := map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "sessions"}}}}
v1 := entry("src2:1", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"wipe"}})
v2 := entry("src2:2", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"erase", "node-login-shell.sessions"}})
mail := catalogseat.CatalogEntry{Module: "mail", Fp: "src2:m", Manifest: map[string]any{
"needs-approval": []any{"mail.received"}, "emits": []any{"mail.received"},
"streams": []any{map[string]any{"name": "MAIL", "events": []any{"mail.**"}, "grant": map[string]any{"config": "readers", "events": "mail.{key}.**"}}}}}
for _, tc := range []struct {
name string
cat *fakeCatalog
module string
fp string
tools []string
verbs map[string]string
subjects []string
unread bool
}{
{name: "a seat's mark stands for every holder, and the holder's own adds up", cat: &fakeCatalog{revision: "7",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell"}},
{name: "a holder's needs-approval cannot unmark its seat's verb, and marks another", cat: &fakeCatalog{revision: "8",
byFp: map[string]catalogseat.CatalogEntry{"src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:2", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "across a version change the union stands: a dropped mark is asked while the old bundle runs", cat: &fakeCatalog{revision: "9",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1, "src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase", "wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "a served entry that cannot be read leaves the newest marks, every marked call refused", cat: &fakeCatalog{revision: "9",
newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}, unread: true},
{name: "a bundle with no fingerprint has no served entry", cat: &fakeCatalog{revision: "9", defs: login},
module: "shell", unread: true},
{name: "a marked event and a stream with a grant give their subjects and each .>", cat: &fakeCatalog{revision: "3",
byFp: map[string]catalogseat.CatalogEntry{"src2:m": mail}, newest: map[string]catalogseat.CatalogEntry{"mail": mail}},
module: "mail", fp: "src2:m", subjects: []string{"mesh.mod.mail.event.mail.received", "mesh.mod.mail.event.mail.received.>",
"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.received", "mesh.stream.mail.event.mail.received.>"}},
} {
t.Run(tc.name, func(t *testing.T) {
m := gate.Derive(tc.cat, tc.module, tc.fp)
if got := keysOf(m.Tools); !reflect.DeepEqual(got, tc.tools) {
t.Errorf("tools %v, want %v", got, tc.tools)
}
if tc.verbs == nil {
tc.verbs = map[string]string{}
}
if !reflect.DeepEqual(m.Verbs, tc.verbs) {
t.Errorf("verbs %v, want %v", m.Verbs, tc.verbs)
}
if got := keysOf(m.Subjects); !reflect.DeepEqual(got, tc.subjects) {
t.Errorf("subjects %v, want %v", got, tc.subjects)
}
if (m.Unread != "") != tc.unread || m.Revision != tc.cat.revision {
t.Errorf("unread %q, revision %q", m.Unread, m.Revision)
}
})
}
// §4: a bundle whose marks name a tool it does not register is not loaded, its finding mark-unserved raised; a bundle that
// serves it clears it, and a catalog without the mark too.
t.Run("mark-unserved raised and cleared", func(t *testing.T) {
cat := &fakeCatalog{revision: "7", byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login}
w := &findings{}
g := gate.New(gate.Deps{Node: "n1", Runner: "node-tools", Catalog: cat, Findings: Findings(w, "n1", "node-tools", nil)})
if got := g.Unserved("shell", "src2:1", []string{"other"}); !reflect.DeepEqual(got, []string{"wipe"}) || w.raised[keys.EscapeName("n1.shell")] != gate.KindMarkUnserved {
t.Fatalf("unserved %v, findings %v", got, w.raised)
}
if got := g.Unserved("shell", "src2:1", []string{"wipe"}); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a bundle serving the mark: unserved %v, findings %v", got, w.raised)
}
_ = g.Unserved("shell", "src2:1", nil)
cat.byFp["src2:1"], cat.newest["shell"] = entry("src2:1", nil, nil), entry("src2:1", nil, nil)
if got := g.Unserved("shell", "src2:1", nil); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a catalog without the mark: unserved %v, findings %v", got, w.raised)
}
})
}
// findings is the runner's held writer as a fixture: each finding raised now, by id.
type findings struct {
raised map[string]string
+61 -2
View File
@@ -62,8 +62,28 @@ func (c *Conn) Bindings(module string) []membership.BusBinding {
// A-16, A-27).
func (c *Conn) JetStream() (jetstream.JetStream, error) { return jetstream.New(c.nc) }
func (c *Conn) objectStore(ctx context.Context, module, local string) (jetstream.ObjectStore, membership.BusBinding, error) {
// objectBucketOf is the bus name of the module's object store of a local name: its binding's when it has one, else, for a
// name no binding of any type holds, the store grants creates for a module's old `objects` entry, `<module>_<name>`
// (catalogue.ObjectsBucket; design 47 G lead answers A-380): the bus's rights decide whether it is reached. A name bound
// with another storage type is refused as the binding refuses it.
func (c *Conn) objectBucketOf(module, local string) (membership.BusBinding, error) {
b, err := c.Binding(module, local, catalogue.StorageObjectStore)
if err == nil {
return b, nil
}
if m := c.Membership(module); m != nil {
for _, held := range m.Bindings {
if held.Local == local {
return b, err
}
}
return membership.BusBinding{Local: local, Storage: catalogue.StorageObjectStore, ObjectStore: catalogue.ObjectsBucket(module, local)}, nil
}
return b, err
}
func (c *Conn) objectStore(ctx context.Context, module, local string) (jetstream.ObjectStore, membership.BusBinding, error) {
b, err := c.objectBucketOf(module, local)
if err != nil {
return nil, b, err
}
@@ -73,6 +93,10 @@ func (c *Conn) objectStore(ctx context.Context, module, local string) (jetstream
}
obs, err := js.ObjectStore(ctx, b.ObjectStore)
if errors.Is(err, jetstream.ErrBucketNotFound) || errors.Is(err, jetstream.ErrStreamNotFound) {
if b.Template == "" {
return nil, b, fmt.Errorf("%s was bound no object-store called %q, and the store %s grants creates for an old `objects` entry of that name is not on the bus",
module, local, b.ObjectStore)
}
return nil, b, fmt.Errorf("%s's objects %q are bound and their store %s is not on the bus", module, local, b.ObjectStore)
}
return obs, b, err
@@ -178,7 +202,7 @@ func (c *Conn) ObjectsInfo(module string, r buscalls.ObjectsRequest) (buscalls.F
if err != nil {
return buscalls.Fill{}, err
}
b, _ := c.Binding(module, r.Objects, catalogue.StorageObjectStore)
b, _ := c.objectBucketOf(module, r.Objects)
fill, err := c.streamFill("OBJ_" + b.ObjectStore)
if err != nil {
return buscalls.Fill{}, err
@@ -276,6 +300,41 @@ func (c *Conn) StreamFill(module string, r buscalls.StreamRequest) (buscalls.Str
return c.streamFill(b.Stream)
}
// StreamPublish says one event on the module's own bound stream, by an acknowledged publish on the binding's subject plus
// "." plus the event, the id as the message id (design 42 zfs-owns-zfs, A-42). The module's own binding alone: another
// module's, and a binding that is not a stream, are refused by Binding.
func (c *Conn) StreamPublish(module string, r buscalls.StreamPublishRequest) (buscalls.StreamPublished, error) {
b, err := c.streamBinding(module, r.Stream)
if err != nil {
return buscalls.StreamPublished{}, err
}
if b.Subject == "" {
return buscalls.StreamPublished{}, fmt.Errorf("%s's stream %q has no subject to say an event on", module, r.Stream)
}
if r.Event == "" || strings.ContainsAny(r.Event, " *>\t\r\n") || strings.HasPrefix(r.Event, ".") || strings.HasSuffix(r.Event, ".") {
return buscalls.StreamPublished{}, fmt.Errorf("the event %q is not a name to say on %s's stream %q", r.Event, module, r.Stream)
}
body := []byte(r.Body)
if len(body) == 0 {
body = []byte("null")
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
js, err := c.JetStream()
if err != nil {
return buscalls.StreamPublished{}, err
}
var opts []jetstream.PublishOpt
if r.ID != "" {
opts = append(opts, jetstream.WithMsgID(r.ID))
}
ack, err := js.Publish(ctx, b.Subject+"."+r.Event, body, opts...)
if err != nil {
return buscalls.StreamPublished{}, fmt.Errorf("the event %s was not acknowledged on %s's stream %q: %w", r.Event, module, r.Stream, err)
}
return buscalls.StreamPublished{Sequence: ack.Sequence}, nil
}
// StreamReadMost is the most messages one read answers.
const StreamReadMost = 1000
+37 -1
View File
@@ -2,6 +2,7 @@ package bus_test
import (
"context"
"encoding/json"
"errors"
"io"
"strings"
@@ -32,6 +33,9 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
if _, err := js.CreateObjectStore(ctx, jetstream.ObjectStoreConfig{Bucket: "m_files"}); err != nil {
t.Fatal(err)
}
if _, err := js.CreateObjectStore(ctx, jetstream.ObjectStoreConfig{Bucket: "m_old"}); err != nil {
t.Fatal(err)
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "m_events", Subjects: []string{"mesh.state.m.events.>"}}); err != nil {
t.Fatal(err)
}
@@ -91,6 +95,13 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
if listed, err := conn.ObjectsList("m", buscalls.ObjectsRequest{Objects: "files"}); err != nil || len(listed.Objects) != 1 {
t.Fatalf("listed %+v, %v", listed, err)
}
// A-380: an old `objects` entry is reached by the store grants creates for it, `<module>_<name>`, with no binding of that name.
if _, err := conn.ObjectsPut("m", buscalls.ObjectsPutRequest{Objects: "old", Name: "r", Data: []byte("old section")}); err != nil {
t.Fatalf("put into the old objects section: %v", err)
}
if listed, err := conn.ObjectsList("m", buscalls.ObjectsRequest{Objects: "old"}); err != nil || len(listed.Objects) != 1 {
t.Fatalf("listed the old objects section %+v, %v", listed, err)
}
page, err := conn.StreamRead("m", buscalls.StreamReadRequest{Stream: "events", Limit: 1})
if err != nil || len(page.Messages) != 1 || page.Next != 2 || page.Last != 2 {
t.Fatalf("read %+v, %v", page, err)
@@ -99,11 +110,28 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
t.Fatalf("fill %+v, %v", fill, err)
}
// A-42: an event said on the module's own stream lands on the binding's subject plus the event, acknowledged, and a
// second say with the same id is the same message.
said, err := conn.StreamPublish("m", buscalls.StreamPublishRequest{Stream: "events", Event: "scrub.requested",
Body: json.RawMessage(`{"pool":"tank"}`), ID: "id-1"})
if err != nil || said.Sequence != 3 {
t.Fatalf("said %+v, %v", said, err)
}
if again, err := conn.StreamPublish("m", buscalls.StreamPublishRequest{Stream: "events", Event: "scrub.requested",
Body: json.RawMessage(`{"pool":"tank"}`), ID: "id-1"}); err != nil || again.Sequence != 3 {
t.Fatalf("said again %+v, %v", again, err)
}
if page, err := conn.StreamRead("m", buscalls.StreamReadRequest{Stream: "events", From: 3,
Subjects: "mesh.state.m.events.scrub.requested"}); err != nil || len(page.Messages) != 1 ||
string(page.Messages[0].Body) != `{"pool":"tank"}` {
t.Fatalf("read back %+v, %v", page, err)
}
for _, c := range []struct {
name, module, refusal string
call func(module string) error
}{
{"a local name not bound", "m", `bound no object-store called "photos"`, func(module string) error {
{"a local name not bound", "m", `bound no object-store called "photos", and the store m_photos grants creates`, func(module string) error {
_, err := conn.ObjectsGet(module, buscalls.ObjectsNameRequest{Objects: "photos", Name: "a"})
return err
}},
@@ -115,6 +143,14 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
_, err := conn.ObjectsList(module, buscalls.ObjectsRequest{Objects: "events"})
return err
}},
{"say on another module's stream", "other", `other was bound no stream called "events"`, func(module string) error {
_, err := conn.StreamPublish(module, buscalls.StreamPublishRequest{Stream: "events", Event: "x"})
return err
}},
{"say on a binding that is not a stream", "m", `"files" is bound with the storage type object-store, not stream`, func(module string) error {
_, err := conn.StreamPublish(module, buscalls.StreamPublishRequest{Stream: "files", Event: "x"})
return err
}},
{"a work queue not bound", "m", `bound no stream called "jobs"`, func(module string) error {
_, err := conn.WorkQueueSize(module, buscalls.WorkQueueRequest{Queue: "jobs"})
return err
+25 -3
View File
@@ -682,6 +682,12 @@ func (c *Conn) HandleSubjectCalled(subject string, h CalledHandler) (func(), err
return c.answerBoth(subject, "", h)
}
// HandleSubjectCalledQueue is HandleSubjectCalled in one queue group: of every holder that serves the subject in the
// group, one answers (lead answer A-393: a node-scoped seat's verb asked with no machine).
func (c *Conn) HandleSubjectCalledQueue(subject, queue string, h CalledHandler) (func(), error) {
return c.answerBoth(subject, queue, h)
}
// answerBoth answers a subject and, when it is a tool's, the caller-named pattern beside it, in one queue.
func (c *Conn) answerBoth(subject, queue string, h CalledHandler) (func(), error) {
stop, err := c.answerCalledOn(subject, queue, h)
@@ -849,6 +855,12 @@ func answerOf(data []byte) (Answered, error) {
// PublishAs emits an event as a module: published into JetStream and awaited, de-duplicated by its
// own id (ADR 0042). The body is the payload; the metadata rides as headers.
func (c *Conn) PublishAs(module string, env Envelope) error {
_, err := c.PublishAsAcked(module, env)
return err
}
// PublishAsAcked is PublishAs that also answers the sequence the bus acknowledged the event at.
func (c *Conn) PublishAsAcked(module string, env Envelope) (uint64, error) {
// On the module's own stream when its membership names one carrying the event (ADR 0363 §2), on EVENTS
// otherwise — and only there, as before, for a membership that names no stream.
msg := nats.NewMsg(eventSubject(c.Membership(module), module, env.Key))
@@ -870,8 +882,11 @@ func (c *Conn) PublishAs(module string, env Envelope) error {
if id := env.Headers["x-event-id"]; id != "" {
opts = append(opts, nats.MsgId(id))
}
_, err := c.js.PublishMsg(msg, opts...)
return err
ack, err := c.js.PublishMsg(msg, opts...)
if err != nil {
return 0, err
}
return ack.Sequence, nil
}
// Say publishes one message on core NATS, kept by nobody: a heartbeat, whose loss is the next one.
@@ -1060,6 +1075,13 @@ func ConsumerOf(node, module string) string {
return node + "_" + module
}
// ToolsSuffix ends the name of the tool runner's own durable consumer of a module: the runner and the module's program each
// need every event, so each reads under a name of its own (lead answer A-397); only a work queue shares one name.
const ToolsSuffix = "_tools"
// ToolsConsumerOf is the tool runner's durable consumer of a module on a machine: `<node>_<module>_tools`.
func ToolsConsumerOf(node, module string) string { return ConsumerOf(node, module) + ToolsSuffix }
// NakDelay is how long an event a handler failed waits before it is offered again: a transient cause
// gets another attempt, a permanent one exhausts the consumer's max-deliver rather than spinning. The
// consumer then gives the event up, and the mesh keeps it in DEAD_LETTERS until a person delivers
@@ -1110,7 +1132,7 @@ var tickHook func()
// answers true (nil is always): a module whose children asked for no heartbeat costs the bus no consumer-info request.
func (c *Conn) ConsumeAsBeatingWhen(module string, deliver func(Envelope) error, beat func(Beat), wanted func() bool) (func(), error) {
asked := func() bool { return beat != nil && (wanted == nil || wanted()) }
durable := ConsumerOf(c.node, module)
durable := ToolsConsumerOf(c.node, module)
js, err := jetstream.New(c.nc)
if err != nil {
return nil, err
+51 -3
View File
@@ -25,9 +25,12 @@ const FileField = "file"
// counts with buscalls.SeatChunks, each acknowledged before the next, and answers the verb's own answer with the whole
// file added as FileField. The asking bundle receives no chunk itself.
func (c *Conn) AskAsChunked(module, key string, body any) (Answered, error) {
seat, verb, ok := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
if !strings.HasPrefix(key, "seat:") || !ok || strings.Contains(verb, "@") {
return Answered{}, fmt.Errorf("%q is no seat:<seat>.<verb> of no one machine, which a chunked answer needs", key)
// The verb may be asked of the machine that ran the ask (`@<node>`, A-254): the chunk subjects name the seat, the verb and
// the call alone.
name, _, _ := strings.Cut(key, "@")
seat, verb, ok := strings.Cut(strings.TrimPrefix(name, "seat:"), ".")
if !strings.HasPrefix(key, "seat:") || !ok {
return Answered{}, fmt.Errorf("%q is no seat:<seat>.<verb>, which a chunked answer needs", key)
}
args := map[string]any{}
if raw, err := json.Marshal(body); err != nil {
@@ -93,3 +96,48 @@ func (c *Conn) AskAsChunked(module, key string, body any) (Answered, error) {
answered.Result = out
return answered, nil
}
// AnswerChunked is a seat verb's answer as its definition says `chunks` (design 47 G A-257, §681): a bundle answers the whole file
// as the member FileField (base64), and the runner that serves the verb sends it in chunks with buscalls.SeatChunks on the call
// token the body names, each acknowledged by the caller before the next, and answers the verb's own members with `chunks`, the
// count, in place of the file. The bundle sends no chunk itself. An answer with no file is answered as it is. The send runs after
// the answer is returned: the caller hears the chunks before it asks (AskAsChunked) and acknowledges each once it has the answer.
func (c *Conn) AnswerChunked(seat, verb string, body json.RawMessage, answer any) (any, error) {
raw, err := json.Marshal(answer)
if err != nil {
return nil, err
}
fields := map[string]json.RawMessage{}
if err := json.Unmarshal(raw, &fields); err != nil {
return answer, nil
}
enc, has := fields[FileField]
if !has {
return answer, nil
}
var text string
if err := json.Unmarshal(enc, &text); err != nil {
return nil, fmt.Errorf("%s.%s answers a %s that is no string: %w", seat, verb, FileField, err)
}
file, err := base64.StdEncoding.DecodeString(text)
if err != nil {
return nil, fmt.Errorf("%s.%s answers a %s in no base64: %w", seat, verb, FileField, err)
}
var asked struct {
Call string `json:"call"`
}
if err := json.Unmarshal(body, &asked); err != nil || asked.Call == "" {
return nil, fmt.Errorf("%s.%s is asked in chunks with no call token", seat, verb)
}
delete(fields, FileField)
count := int64((len(file) + buscalls.SeatChunkBytes - 1) / buscalls.SeatChunkBytes)
fields["chunks"], _ = json.Marshal(count)
send := buscalls.SeatChunks{Seat: seat, Verb: verb, Call: asked.Call}
go func() { _ = send.Send(c.nc, file, buscalls.SeatAckWithin, c.Logf) }()
return json.RawMessage(mustMarshal(fields)), nil
}
func mustMarshal(v any) []byte {
b, _ := json.Marshal(v)
return b
}
+42
View File
@@ -57,6 +57,7 @@ func TestAChunkedSeatAnswerIsReceivedByTheRunnerAndHandedWhole(t *testing.T) {
t.Fatal(err)
}
defer stop()
holder.Flush() // the bus holds the verb's subscription before it is asked: the asker is another connection
got, err := asker.AskAsChunked("mesh-build-go", "seat:mesh-catalog.input", map[string]any{"digest": "d1"})
if err != nil {
t.Fatal(err)
@@ -72,4 +73,45 @@ func TestAChunkedSeatAnswerIsReceivedByTheRunnerAndHandedWhole(t *testing.T) {
if body, err := base64.StdEncoding.DecodeString(ans.File); err != nil || string(body) != string(file) {
t.Errorf("the file handed is %d bytes, want the %d sent: %v", len(body), len(file), err)
}
// Design 47 G A-257, A-254: a bundle answers a chunked verb with the whole file in the member `file`; the runner that serves the
// verb sends the file in chunks (each acknowledged before the next) and answers the count, so the bundle sends none, and the asker
// of one node's verb receives it whole.
t.Run("ABundlesChunkedAnswerIsSentByTheRunner", func(t *testing.T) {
url := mt.URL(t)
holder := dial(t, url, "mesh-build-go", "anchor", "")
asker := dial(t, url, "mesh-delivery-indexer", "anchor", "")
file := make([]byte, buscalls.SeatChunkBytes*2+7)
for i := range file {
file[i] = byte(i * 7)
}
stop, err := holder.Handle("seat:node-build-go.output@anchor", func(body json.RawMessage) (any, error) {
answer := map[string]any{"ask": "a1", "file": base64.StdEncoding.EncodeToString(file)}
return holder.AnswerChunked("node-build-go", "output", body, answer)
})
if err != nil {
t.Fatal(err)
}
defer stop()
holder.Flush() // the bus holds the verb's subscription before it is asked: the asker is another connection
got, err := asker.AskAsChunked("mesh-delivery-indexer", "seat:node-build-go.output@anchor", map[string]any{"ask": "a1"})
if err != nil {
t.Fatal(err)
}
var ans struct {
Ask string `json:"ask"`
Chunks int `json:"chunks"`
File string `json:"file"`
}
if err := json.Unmarshal(got.Result, &ans); err != nil || ans.Ask != "a1" || ans.Chunks != 3 {
t.Fatalf("the answer %s: %v", got.Result, err)
}
if body, err := base64.StdEncoding.DecodeString(ans.File); err != nil || string(body) != string(file) {
t.Errorf("the file handed is %d bytes, want the %d sent: %v", len(body), len(file), err)
}
// an answer with no file is the verb's own: no chunks are sent
plain, err := holder.AnswerChunked("node-build-go", "output", json.RawMessage(`{"call":"c"}`), map[string]any{"ask": "x", "chunks": 0})
if m, _ := plain.(map[string]any); err != nil || m["chunks"] != 0 {
t.Fatalf("an answer with no file was %v, %v", plain, err)
}
})
}
+76
View File
@@ -10,6 +10,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/membership"
"github.com/nats-io/nats.go"
@@ -143,6 +144,76 @@ func (h *busHistory) Read(keys string, from uint64, limit int) ([]ledger.Kept, e
}
}
// Purge removes every operation of one instance from the history stream, on the module's ledger connection, the one
// granted the purge of its own ledger's streams.
func (h *busHistory) Purge(key string) error {
return purgeSubject(h.read, h.write, h.stream, h.prefix+"."+key)
}
// Fill is the history stream's fill against its cap, the cap the manifest gives the ledger.
func (h *busHistory) Fill() (buscalls.Fill, error) {
name := h.stream
if name == "" {
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
js, err := h.read.JetStream()
if err != nil {
return buscalls.Fill{}, err
}
if name, err = js.StreamNameBySubject(ctx, h.prefix+".>"); err != nil {
return buscalls.Fill{}, fmt.Errorf("no stream on the bus keeps %s: %w", h.prefix, err)
}
}
f, err := h.read.streamFill(name)
return f.Fill, err
}
// purgeSubject purges one subject's messages from the stream that keeps it: the stream named, or found on the runner's
// own connection by its subject, purged on the ledger connection.
func purgeSubject(read, write *Conn, stream, subject string) error {
if write == nil {
return errors.New("a ledger's streams are purged only on its module's ledger connection, and none is held")
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
if stream == "" {
js, err := read.JetStream()
if err != nil {
return err
}
if stream, err = js.StreamNameBySubject(ctx, subject); err != nil {
return fmt.Errorf("no stream on the bus keeps %s: %w", subject, err)
}
}
// The purge is asked of the stream by its name, with no read of its info first: the ledger account is granted the
// purge of its own ledger's two streams and nothing else (PurgeAPI).
body, err := json.Marshal(map[string]string{"filter": subject})
if err != nil {
return err
}
reply, err := write.nc.RequestWithContext(ctx, PurgeAPI(stream), body)
if err != nil {
return fmt.Errorf("purging %s from %s: %w", subject, stream, err)
}
var ack struct {
Success bool `json:"success"`
Error *struct {
Description string `json:"description"`
} `json:"error"`
}
if err := json.Unmarshal(reply.Data, &ack); err != nil || !ack.Success {
why := string(reply.Data)
if ack.Error != nil {
why = ack.Error.Description
}
return fmt.Errorf("the bus refused the purge of %s from %s: %s", subject, stream, why)
}
return nil
}
// PurgeAPI is the bus's subject that purges a stream, the one a ledger account is granted for its own ledger's streams.
func PurgeAPI(stream string) string { return "$JS.API.STREAM.PURGE." + stream }
type busState struct {
c *Conn
bucket string
@@ -231,6 +302,11 @@ func (s *busSnapshots) Get(key string) ([]byte, uint64, error) {
return lastOn(s.read, "", s.prefix+"."+key)
}
// Purge removes one instance's snapshot, on the module's ledger connection.
func (s *busSnapshots) Purge(key string) error {
return purgeSubject(s.read, s.write, "", s.prefix+"."+key)
}
func (s *busSnapshots) Put(key string, value []byte, expect uint64) error {
if s.write == nil {
return errors.New("a ledger's snapshot is published only on its module's ledger connection, and none is held")
-27
View File
@@ -1,27 +0,0 @@
package bus
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// Design 47 G §347, §59, §462: a sent object is read by the reference that carried it, from a sender's sent store only; a
// state's objects, and a reference that is none are refused before the bus is asked.
func TestSentGetRefusesWhatIsNoSentObjectsReference(t *testing.T) {
digest := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
for _, c := range []struct {
name string
req buscalls.ObjectsSentGetRequest
want string
}{
{"no digest", buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3}, "named by the reference"},
{"a state's objects", buscalls.ObjectsSentGetRequest{Store: "m_asks-objects", Object: "o", Size: 3, Digest: digest}, "no sent store"},
} {
_, err := (&Conn{}).SentGet(c.req)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: got %v, want an error containing %q", c.name, err, c.want)
}
}
}
+52 -39
View File
@@ -5,7 +5,9 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"regexp"
"slices"
"sort"
"strings"
"sync"
@@ -174,64 +176,50 @@ func (c *Conn) StatePage(module, name, after string, maxBytes int, budget time.D
each := func() (context.Context, context.CancelFunc) {
return context.WithTimeout(context.Background(), StateTimeout)
}
var keys []string
var get func(key string) (*StateEntry, error)
// Every current value is read in one stream per bucket (currentOf), never one bus request per key: a page of the
// assignments (about 330 keys) read key by key took 13 s from a node far from the bus (2026-11-14).
current := map[string]*StateEntry{}
if s.PerNode {
fam := c.family(s)
if keys, err = fam.Keys(ctx); err != nil {
keys, err := fam.Keys(ctx)
if err != nil {
return StatePage{}, err
}
buckets := map[string]jetstream.KeyValue{} // each node's bucket bound once
get = func(key string) (*StateEntry, error) {
node, err := pernode.NodeOf(key)
if err != nil {
return nil, err
nodes := map[string]bool{}
for _, k := range keys {
if node, err := pernode.NodeOf(k); err == nil {
nodes[node] = true
}
kv, bound := buckets[node]
if !bound {
octx, ocancel := each() // the open is a bus request in the loop: its own deadline, never ctx's
kv, err = fam.Open(octx, node)
ocancel()
if err != nil {
return nil, err
}
buckets[node] = kv
}
for node := range nodes {
octx, ocancel := each() // each node's bucket is a bus request of its own: its own deadline
kv, err := fam.Open(octx, node)
ocancel()
if err != nil {
return StatePage{}, err
}
if kv == nil {
return nil, nil
}
gctx, gcancel := each()
defer gcancel()
e, err := kv.Get(gctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
continue
}
wctx, wcancel := each()
got, err := currentOf(wctx, kv)
wcancel()
if err != nil {
return nil, err
return StatePage{}, err
}
return &StateEntry{Key: e.Key(), Value: valueOf(e.Value()), Revision: e.Revision(), At: e.Created().UTC()}, nil
maps.Copy(current, got)
}
} else {
kv, err := c.bucket(ctx, s)
if err != nil {
return StatePage{}, err
}
if keys, err = keysOf(ctx, kv); err != nil {
if current, err = currentOf(ctx, kv); err != nil {
return StatePage{}, err
}
get = func(key string) (*StateEntry, error) {
gctx, gcancel := each()
defer gcancel()
e, err := kv.Get(gctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, err
}
return &StateEntry{Key: e.Key(), Value: valueOf(e.Value()), Revision: e.Revision(), At: e.Created().UTC()}, nil
}
}
keys := slices.Collect(maps.Keys(current))
get := func(key string) (*StateEntry, error) { return current[key], nil }
sort.Strings(keys)
page := StatePage{Entries: []StateEntry{}}
started := PageClock()
@@ -719,3 +707,28 @@ func providerReference(v any) bool {
// referenceName is a node's or a module's name as mesh-sdk names them: lower case, digits and dashes.
var referenceName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// currentOf is every current value of a bucket, read in one stream (a watch of every key, deletes left out, ended at its
// first nil): the values a page answers, without a bus request per key.
func currentOf(ctx context.Context, kv jetstream.KeyValue) (map[string]*StateEntry, error) {
w, err := kv.WatchAll(ctx, jetstream.IgnoreDeletes())
if err != nil {
return nil, err
}
defer func() { _ = w.Stop() }()
out := map[string]*StateEntry{}
for {
select {
case <-ctx.Done():
return nil, fmt.Errorf("the values of %s are not all read: %w", kv.Bucket(), ctx.Err())
case e, ok := <-w.Updates():
if !ok {
return out, nil
}
if e == nil {
return out, nil
}
out[e.Key()] = &StateEntry{Key: e.Key(), Value: valueOf(e.Value()), Revision: e.Revision(), At: e.Created().UTC()}
}
}
}
+68 -69
View File
@@ -87,6 +87,74 @@ func TestAWatchHoldsWhileItsCurrentValuesKeepArriving(t *testing.T) {
}
return nil
})
// **A watch given up, or stopped, hands over nothing more** (issue 774). The runtime's watch stopped its watcher
// when the current values did not arrive within the bound, but the watcher still held values, and its goroutine
// went on handing them over: to a watch id the bundle no longer held (on the provisioner, 51 failed watches were
// followed by 1257 orphan changes). Each row blocks the bundle on its first value, gives the watch up, lets the
// bundle go and counts what it is still handed.
t.Run("AWatchGivenUpHandsOverNothingMore", func(t *testing.T) {
for _, tc := range []struct {
name string
refused bool // the bound gives it up, or the caller stops it
}{
{"refused at the bound", true},
{"stopped by its caller", false},
} {
t.Run(tc.name, func(t *testing.T) {
was := bus.StateTimeout
bus.StateTimeout = 300 * time.Millisecond
t.Cleanup(func() { bus.StateTimeout = was })
conn, kv := owedWatcher(t)
put := func(i int) {
if _, err := kv.Put(fmt.Sprintf("novox/hq.%d.%040x.status=mesh/check", i+1, i), []byte(`{"v":1}`)); err != nil {
t.Fatal(err)
}
}
if tc.refused {
for i := 0; i < 4; i++ {
put(i)
}
}
var calls atomic.Int32
first := make(chan struct{}, 1)
release := make(chan struct{})
deliver := func(ch bus.StateChange) error {
if calls.Add(1) == 1 {
first <- struct{}{}
<-release
}
return nil
}
var stop func()
if tc.refused {
if _, err := conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err == nil {
t.Fatal("the watch whose bundle took nothing was not refused")
}
} else {
var err error
if stop, err = conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err != nil {
t.Fatal(err)
}
for i := 0; i < 4; i++ {
put(i)
}
select {
case <-first:
case <-time.After(5 * time.Second):
t.Fatal("the bundle was handed no change")
}
stop()
}
// Let the watcher's server side deliver what it held, then free the bundle.
time.Sleep(300 * time.Millisecond)
close(release)
time.Sleep(500 * time.Millisecond)
if n := calls.Load(); n != 1 {
t.Fatalf("the bundle was handed %d changes of a watch given up, want 1 (the one in hand)", n)
}
})
}
})
}
// A watch whose bundle takes nothing for longer than the bound is still refused, so a bundle that hangs
@@ -187,72 +255,3 @@ func owedWatcher(t *testing.T) (*bus.Conn, interface {
})
return conn, kv
}
// **A watch given up, or stopped, hands over nothing more** (issue 774). The runtime's watch stopped its watcher
// when the current values did not arrive within the bound, but the watcher still held values, and its goroutine
// went on handing them over: to a watch id the bundle no longer held (on the provisioner, 51 failed watches were
// followed by 1257 orphan changes). Each row blocks the bundle on its first value, gives the watch up, lets the
// bundle go and counts what it is still handed.
func TestAWatchGivenUpHandsOverNothingMore(t *testing.T) {
for _, tc := range []struct {
name string
refused bool // the bound gives it up, or the caller stops it
}{
{"refused at the bound", true},
{"stopped by its caller", false},
} {
t.Run(tc.name, func(t *testing.T) {
was := bus.StateTimeout
bus.StateTimeout = 300 * time.Millisecond
t.Cleanup(func() { bus.StateTimeout = was })
conn, kv := owedWatcher(t)
put := func(i int) {
if _, err := kv.Put(fmt.Sprintf("novox/hq.%d.%040x.status=mesh/check", i+1, i), []byte(`{"v":1}`)); err != nil {
t.Fatal(err)
}
}
if tc.refused {
for i := 0; i < 4; i++ {
put(i)
}
}
var calls atomic.Int32
first := make(chan struct{}, 1)
release := make(chan struct{})
deliver := func(ch bus.StateChange) error {
if calls.Add(1) == 1 {
first <- struct{}{}
<-release
}
return nil
}
var stop func()
if tc.refused {
if _, err := conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err == nil {
t.Fatal("the watch whose bundle took nothing was not refused")
}
} else {
var err error
if stop, err = conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err != nil {
t.Fatal(err)
}
for i := 0; i < 4; i++ {
put(i)
}
select {
case <-first:
case <-time.After(5 * time.Second):
t.Fatal("the bundle was handed no change")
}
stop()
}
// Let the watcher's server side deliver what it held, then free the bundle.
time.Sleep(300 * time.Millisecond)
close(release)
time.Sleep(500 * time.Millisecond)
if n := calls.Load(); n != 1 {
t.Fatalf("the bundle was handed %d changes of a watch given up, want 1 (the one in hand)", n)
}
})
}
}
+2
View File
@@ -269,6 +269,8 @@ func (h *hearing) reconcile() {
if m := h.c.Membership(h.module); m != nil {
for _, heard := range m.Hears {
if heard.Stream != "" && heard.Consumer != "" {
// The runner's own durable beside the program's (A-397).
heard.Consumer += ToolsSuffix
want[heard] = true
}
}
@@ -1,6 +1,11 @@
package bus
import "testing"
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// novox/hq ADR 0363 §2, §3: an event goes to the stream whose pattern carries it, as the bus matches subjects.
func TestAnEventFallsUnderAStreamsPatternAsTheBusMatchesIt(t *testing.T) {
@@ -22,6 +27,24 @@ func TestAnEventFallsUnderAStreamsPatternAsTheBusMatchesIt(t *testing.T) {
t.Errorf("%q under %q: %v, want %v", c.event, c.pattern, got, c.want)
}
}
// Design 47 G §347, §59, §462: a sent object is read by the reference that carried it, from a sender's sent store only; a
// state's objects, and a reference that is none are refused before the bus is asked.
t.Run("SentGetRefusesWhatIsNoSentObjectsReference", func(t *testing.T) {
digest := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
for _, c := range []struct {
name string
req buscalls.ObjectsSentGetRequest
want string
}{
{"no digest", buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3}, "named by the reference"},
{"a state's objects", buscalls.ObjectsSentGetRequest{Store: "m_asks-objects", Object: "o", Size: 3, Digest: digest}, "no sent store"},
} {
_, err := (&Conn{}).SentGet(c.req)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: got %v, want an error containing %q", c.name, err, c.want)
}
}
})
}
// A membership naming no stream publishes every event on EVENTS, exactly as before (issue 725).
+87
View File
@@ -0,0 +1,87 @@
package bus
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"github.com/nats-io/nats.go/jetstream"
)
// SubscribeNew hears one event, named `<emitter>.<event>`, from the next message on, on an ordered consumer of its own on each
// stream that carries it, and hands every message to deliver as the envelope a module sees. The subjects the name lands on are
// mesh-sdk's one rule (buscompose.EventSubjects, design 47 G §559, A-153), so a name with a wildcard is refused, naming the rule;
// a name no stream on the bus carries is an error, never a subscription that hears nothing. stop ends every consumer.
func (c *Conn) SubscribeNew(name string, deliver func(json.RawMessage) error) (func(), error) {
subjects, err := buscompose.EventSubjects(name, "")
if err != nil {
return nil, err
}
js, err := c.JetStream()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
// The streams that carry each subject, found by the bus: a namespace no stream carries hears nothing and is skipped.
byStream := map[string][]string{}
for _, subject := range subjects {
if strings.HasSuffix(subject, ".>") {
continue
}
stream, err := js.StreamNameBySubject(ctx, subject)
if err != nil || stream == "" {
continue
}
byStream[stream] = append(byStream[stream], subject, subject+".>")
}
if len(byStream) == 0 {
return nil, fmt.Errorf("no stream on the bus carries %s (%s): nothing would be heard", name, strings.Join(subjects, ", "))
}
var stops []func()
stopAll := func() {
for _, s := range stops {
s()
}
}
for _, stream := range slices.Sorted(mapKeys(byStream)) {
cons, err := js.OrderedConsumer(ctx, stream, jetstream.OrderedConsumerConfig{FilterSubjects: byStream[stream],
DeliverPolicy: jetstream.DeliverNewPolicy})
if err != nil {
stopAll()
return nil, fmt.Errorf("an ordered consumer of %s on %s: %w", name, stream, err)
}
cc, err := cons.Consume(func(m jetstream.Msg) {
env, ok := envelopeOf(m.Subject(), m.Headers(), m.Data())
if !ok {
return
}
raw, err := json.Marshal(env)
if err != nil {
return
}
if err := deliver(raw); err != nil && c.Logf != nil {
c.Logf("[mesh-tools] a subscription for new %s messages did not take one: %v", name, err)
}
})
if err != nil {
stopAll()
return nil, fmt.Errorf("consuming %s on %s: %w", name, stream, err)
}
stops = append(stops, cc.Stop)
}
return stopAll, nil
}
func mapKeys(m map[string][]string) func(func(string) bool) {
return func(yield func(string) bool) {
for k := range m {
if !yield(k) {
return
}
}
}
}
+17 -11
View File
@@ -19,7 +19,7 @@ package console
import (
assigner "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"context"
@@ -927,6 +927,21 @@ func schemaWithoutNode(raw json.RawMessage) map[string]any {
}
// answerText is a discovery verb's answer as MCP content: JSON, indented.
// callFailed is why a call failed, as mesh_call says it: a marked tool or verb that reaches no responder names `withheld`
// and the grants part's read (design 46 7-approval-mark §6), and a target known only from an earlier round says so.
func (s *Surface) callFailed(t target, err error) string {
why := whyItFailed(t.Address, err)
if ok, _ := s.markedTool(t); ok && errors.Is(err, bus.ErrNoResponders) {
// A marked subject withheld from a machine whose tool runner does not honour the mark (§6).
why += "; it needs the operator's approval, so it may be withheld where the machine's tool runner does not state " +
"approval-mark/1: mesh-grants.grants_user_list names it under withheld"
}
if t.Stale != nil {
why += " (" + t.Stale.say() + ")"
}
return why
}
func answerText(v any) map[string]any {
var b strings.Builder
enc := json.NewEncoder(&b)
@@ -1137,16 +1152,7 @@ func (s *Surface) discover(ctx context.Context, name string, args map[string]any
got, err = callTool(s.conn, t.Key, callArgs, nil, x.Listing)
}
if err != nil {
why := whyItFailed(t.Address, err)
if ok, _ := s.markedTool(t); ok && errors.Is(err, bus.ErrNoResponders) {
// A marked subject withheld from a machine whose tool runner does not honour the mark (§6).
why += "; it needs the operator's approval, so it may be withheld where the machine's tool runner does not state " +
"approval-mark/1: mesh-grants.grants_user_list names it under withheld"
}
if t.Stale != nil {
why += " (" + t.Stale.say() + ")"
}
return failure(why)
return failure(s.callFailed(t, err))
}
result, unread := resolveSent(s.conn, got.Result)
content := []map[string]any{{"type": "text", "text": pretty(result)}}
@@ -6,6 +6,7 @@ import (
"log"
"strings"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
)
@@ -20,6 +21,10 @@ type FirstViewer interface {
// state and since when.
type entryLister interface{ Entries() any }
// ownBuilder is a FirstViewer that also says the build the tool runner itself was started with (design 47 G §205, §424):
// mesh_first_views names it, or says `own build unknown` and why.
type ownBuilder interface{ OwnBuild() process.Build }
// firstViewsAnswer is mesh_first_views and mesh_first_view_again.
func (s *Surface) firstViewsAnswer(ctx context.Context, name string, args map[string]any) map[string]any {
fv, ok := s.src.(FirstViewer)
@@ -29,6 +34,9 @@ func (s *Surface) firstViewsAnswer(ctx context.Context, name string, args map[st
}
if name == verbFirstViews {
out := map[string]any{"first views": fv.Status()}
if ob, ok := fv.(ownBuilder); ok {
out["own build"] = ob.OwnBuild().String()
}
if el, ok := fv.(entryLister); ok {
out["entries"] = el.Entries()
}
@@ -7,6 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
)
@@ -14,12 +15,16 @@ import (
type viewer struct {
busSource
asked []string
build process.Build
}
func (v *viewer) Status() []views.Status {
return []views.Status{{Owner: "mesh-delivery-assigner", Tool: "assignments", State: "assignments", Revision: 4, Asks: 1}}
}
// OwnBuild is the build the viewer was started with.
func (v *viewer) OwnBuild() process.Build { return v.build }
// Entries are the entries by fingerprint the viewer keeps (design 47 G A-103).
func (v *viewer) Entries() any { return []string{"fp:a asked"} }
@@ -41,6 +46,13 @@ func TestTheConsoleShowsAndAsksAgainTheRunnersFirstViews(t *testing.T) {
if got := text(s.discover(context.Background(), "mesh_first_views", nil)); !strings.Contains(got, "mesh-delivery-assigner") || !strings.Contains(got, "assignments") || !strings.Contains(got, "fp:a asked") {
t.Errorf("mesh_first_views shows %s", got)
}
// The tool runner's own build is named, or said unknown with the reason (design 47 G §205, §424).
for build, want := range map[string]string{"abcd1234": "abcd1234", "": "own build unknown"} {
v.build = process.ReadOwnBuild(func(string) string { return build })
if got := text(s.discover(context.Background(), "mesh_first_views", nil)); !strings.Contains(got, `own build`) || !strings.Contains(got, want) {
t.Errorf("mesh_first_views with the build %q shows %s", build, got)
}
}
got := s.discover(context.Background(), "mesh_first_view_again", map[string]any{"why": "doubted"})
if len(v.asked) != 1 || v.asked[0] != "doubted" || !strings.Contains(text(got), "asked again") {
t.Errorf("mesh_first_view_again asked %v and answered %s", v.asked, text(got))
@@ -70,8 +70,8 @@ func TestTheManifestNamesNoWaitsAndKeepsItsApprovals(t *testing.T) {
// `registration-changed`, and the gate's `source-moved`; never the indexer's `views.modules` or `views.manifests`), the two
// events of the seat that serves the seat list (ADR 0636 §4) and the keeper's `findings-asked`, and nothing else.
wantConsumes := []string{"mesh-catalog.catalog-revision", "mesh-catalog.registration-changed", "mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed", "mesh-conditions.findings-asked", "mesh-delivery-assigner.views.assignments",
"mesh-delivery-assigner.views.seat-holders", "mesh-delivery.source-moved"}
"mesh-catalog.seat-definition-removed", "mesh-conditions.findings-asked", "seat:mesh-module-assigner.views.assignments",
"seat:mesh-module-assigner.views.seat-holders", "mesh-delivery.source-moved"}
if !reflect.DeepEqual(m.Consumes, wantConsumes) {
t.Errorf("the tool runner consumes %v, want %v", m.Consumes, wantConsumes)
}
@@ -11,7 +11,8 @@ import (
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// A tool's own error that mentions 503 is the tool's answer, not a routing fault (issue 1529); a real no-responders still says
// A marked tool or verb that reaches no responder names withheld and the grants part's read (design 46 7-approval-mark §6);
// an unmarked one, or a marked one failing otherwise, does not. A tool's own error that mentions 503 is the tool's answer, not a routing fault (issue 1529); a real no-responders still says
// nothing serves.
func TestARealNoRespondersStillSaysNothingServes(t *testing.T) {
got := whyItFailed("scanner.scan", errors.New("the scanner refused the scan job: 503 Service Unavailable"))
@@ -23,4 +24,20 @@ func TestARealNoRespondersStillSaysNothingServes(t *testing.T) {
t.Fatalf("%v: got %q", err, got)
}
}
s := &Surface{Approvals: &fakeApprovals{marked: "shell.wipe"}}
for _, c := range []struct {
target target
err error
hinted bool
}{
{target{Address: "shell.wipe", Tool: Tool{Module: "shell", Name: "wipe"}}, bus.ErrNoResponders, true},
{target{Address: "node-login-shell.execute", Seat: true, Tool: Tool{Module: "node-login-shell", Name: "execute"}}, bus.ErrNoResponders, false},
{target{Address: "shell.tidy", Tool: Tool{Module: "shell", Name: "tidy"}}, bus.ErrNoResponders, false},
{target{Address: "shell.wipe", Tool: Tool{Module: "shell", Name: "wipe"}}, errors.New("its own failure"), false},
} {
got := s.callFailed(c.target, c.err)
if hinted := strings.Contains(got, "approval-mark/1: mesh-grants.grants_user_list names it under withheld"); hinted != c.hinted {
t.Errorf("%s (%v): %q", c.target.Address, c.err, got)
}
}
}
@@ -1,9 +1,18 @@
package console
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/stateobjects"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// A refusal is said only for the bus's own refusals; a tool's answer that mentions authorization is the
@@ -19,6 +28,45 @@ func TestARefusalIsSaidOnlyForTheBussOwn(t *testing.T) {
t.Errorf("%q read as a refusal: %v, want %v", msg, got, refusal)
}
}
// ADR 0666 §16: an answer a module sent as a reference is read from its sent store for the person asking; one the bus does
// not let the runner read is shown as its reference with its size and why it was not read.
t.Run("AnAnswerSentAsAReferenceIsReadForThePersonAsking", func(t *testing.T) {
nc, err := nats.Connect(meshtest.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
obs, err := js.CreateObjectStore(context.Background(), jetstream.ObjectStoreConfig{Bucket: "m_sent-objects"})
if err != nil {
t.Fatal(err)
}
conn, err := bus.Connect(bus.Credential{URL: meshtest.URL(t), Module: "console", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer conn.Close()
body := []byte(`{"lines":"` + strings.Repeat("x", 9<<20) + `"}`)
sum := sha256.Sum256(body)
ref := stateobjects.Ref{Store: "m_sent-objects", Object: "tool.lines." + hex.EncodeToString(sum[:]), Size: int64(len(body)),
Digest: hex.EncodeToString(sum[:]), Form: stateobjects.FormVersion}
if _, err := obs.PutBytes(context.Background(), ref.Object, body); err != nil {
t.Fatal(err)
}
if got, unread := resolveSent(conn, stateobjects.Wrap(ref)); unread != "" || string(got) != string(body) {
t.Fatalf("read %d bytes of %d, %q", len(got), len(body), unread)
}
missing := ref
missing.Store = "gone_sent-objects"
wrapped := stateobjects.Wrap(missing)
got, unread := resolveSent(conn, wrapped)
if string(got) != string(wrapped) || !strings.Contains(unread, "9437") || !strings.Contains(unread, "gone_sent-objects") {
t.Fatalf("an unread answer showed %q and %q", got, unread)
}
if got, unread := resolveSent(conn, []byte(`{"a":1}`)); unread != "" || string(got) != `{"a":1}` {
t.Fatalf("a plain answer showed %q, %q", got, unread)
}
})
}
// **A timeout says it is not a failure** (novox/hq issue 265): a push that outlasted the wait had
-56
View File
@@ -1,56 +0,0 @@
package console
import (
"context"
"crypto/sha256"
"encoding/hex"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/stateobjects"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// ADR 0666 §16: an answer a module sent as a reference is read from its sent store for the person asking; one the bus does
// not let the runner read is shown as its reference with its size and why it was not read.
func TestAnAnswerSentAsAReferenceIsReadForThePersonAsking(t *testing.T) {
nc, err := nats.Connect(meshtest.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
obs, err := js.CreateObjectStore(context.Background(), jetstream.ObjectStoreConfig{Bucket: "m_sent-objects"})
if err != nil {
t.Fatal(err)
}
conn, err := bus.Connect(bus.Credential{URL: meshtest.URL(t), Module: "console", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer conn.Close()
body := []byte(`{"lines":"` + strings.Repeat("x", 9<<20) + `"}`)
sum := sha256.Sum256(body)
ref := stateobjects.Ref{Store: "m_sent-objects", Object: "tool.lines." + hex.EncodeToString(sum[:]), Size: int64(len(body)),
Digest: hex.EncodeToString(sum[:]), Form: stateobjects.FormVersion}
if _, err := obs.PutBytes(context.Background(), ref.Object, body); err != nil {
t.Fatal(err)
}
if got, unread := resolveSent(conn, stateobjects.Wrap(ref)); unread != "" || string(got) != string(body) {
t.Fatalf("read %d bytes of %d, %q", len(got), len(body), unread)
}
missing := ref
missing.Store = "gone_sent-objects"
wrapped := stateobjects.Wrap(missing)
got, unread := resolveSent(conn, wrapped)
if string(got) != string(wrapped) || !strings.Contains(unread, "9437") || !strings.Contains(unread, "gone_sent-objects") {
t.Fatalf("an unread answer showed %q and %q", got, unread)
}
if got, unread := resolveSent(conn, []byte(`{"a":1}`)); unread != "" || string(got) != `{"a":1}` {
t.Fatalf("a plain answer showed %q, %q", got, unread)
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"encoding/json"
"sync"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
+1 -1
View File
@@ -17,7 +17,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/catalogue"
+106 -61
View File
@@ -16,8 +16,8 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
opseat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
@@ -203,7 +203,7 @@ func TestMeshWaitsTimeoutIsBounded(t *testing.T) {
// a wildcard, of one token, more than 16 names, none, or a from of another word is refused at once naming the rule; a
// wait whose every subscription the bus refuses answers after its timeout, in words naming no wait grant.
func TestMeshWaitRefusesLoudlyAndNeverLoops(t *testing.T) {
s, mesh := waitingRuntime(t)
s, _ := waitingRuntime(t)
var seventeen []any
for i := 0; i < 17; i++ {
seventeen = append(seventeen, fmt.Sprintf("mesh-knowledge.reviewed%d", i))
@@ -230,63 +230,80 @@ func TestMeshWaitRefusesLoudlyAndNeverLoops(t *testing.T) {
}
}
// E §14: against the catalog half, a misspelt emitter is refused naming the half and its revision, a known emitter's
// undeclared event is waited on and its timeout says so, and a declared one says nothing of it.
half := NewSurface(s.conn, halfSource{busSource{s.conn}, map[string]meshcatalogclient.Half{"main": {
Revision: catalogseat.CatalogRevision{Revision: 7, Definitions: []catalogseat.CatalogRevisionDefinitionsItem{{Seat: "mesh-catalog"}}},
Entries: map[string]catalogseat.CatalogEntry{"mesh-delivery-view": {Module: "mesh-delivery-view",
Manifest: map[string]any{"emits": []any{"delivered"}}}},
}}}, "desk.node-tools")
for _, c := range []struct {
name, says string
refused bool
}{
{"mesh-delivry-view.delivered", "mesh-delivry-view is neither a seat nor a module of the catalog half (mesh-catalog.revision, revision 7 of main)", true},
{"mesh-delivery-view.planned", "no declaration names mesh-delivery-view.planned", false},
{"mesh-catalog.catalog-revision", "", false},
} {
text, isErr := callWait(half, map[string]any{"subjects": []any{c.name}, "timeout": 1})
if isErr != c.refused || c.says != "" && !strings.Contains(text, c.says) ||
c.says == "" && (strings.Contains(text, "undeclared") || strings.Contains(text, "unchecked")) {
t.Errorf("%s against the catalog half: %s", c.name, text)
t.Run("the catalog half", func(t *testing.T) {
t.Parallel()
// E §14: against the catalog half, a misspelt emitter is refused naming the half and its revision, a known emitter's
// undeclared event is waited on and its timeout says so, and a declared one says nothing of it.
half := NewSurface(s.conn, halfSource{busSource{s.conn}, map[string]meshcatalogclient.Half{"main": {
Revision: catalogseat.CatalogRevision{Revision: 7, Definitions: []catalogseat.CatalogRevisionDefinitionsItem{{Seat: "mesh-catalog"}}},
Entries: map[string]catalogseat.CatalogEntry{"mesh-delivery-view": {Module: "mesh-delivery-view",
Manifest: map[string]any{"emits": []any{"delivered"}}}},
}}}, "desk.node-tools")
for _, c := range []struct {
name, says string
refused bool
}{
{"mesh-delivry-view.delivered", "mesh-delivry-view is neither a seat nor a module of the catalog half (mesh-catalog.revision, revision 7 of main)", true},
{"mesh-delivery-view.planned", "no declaration names mesh-delivery-view.planned", false},
{"mesh-catalog.catalog-revision", "", false},
} {
text, isErr := callWait(half, map[string]any{"subjects": []any{c.name}, "timeout": 1})
if isErr != c.refused || c.says != "" && !strings.Contains(text, c.says) ||
c.says == "" && (strings.Contains(text, "undeclared") || strings.Contains(text, "unchecked")) {
t.Errorf("%s against the catalog half: %s", c.name, text)
}
}
}
})
t.Run("a bus that refuses every subscription", func(t *testing.T) {
t.Parallel()
// The bus refusing every subscription of the call: answered after the timeout, in §16's words.
b, err := server.NewServer(&server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, NoLog: true, NoSigs: true,
Users: []*server.User{{Username: "desk.node-tools", Password: "pw", Permissions: &server.Permissions{
Subscribe: &server.SubjectPermission{Allow: []string{"_INBOX.desk.node-tools.>", "mesh.assignment.desk.*"}},
}}}})
if err != nil {
t.Fatal(err)
}
go b.Start()
if !b.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(b.Shutdown)
conn, err := bus.Connect(bus.Credential{URL: b.ClientURL(), Module: "node-tools", Node: "desk", User: "desk.node-tools", Password: "pw"})
if err != nil {
t.Fatal(err)
}
conn.Logf = func(string, ...any) {}
t.Cleanup(conn.Close)
start := time.Now()
text, isErr := callWait(NewSurface(conn, nil, "desk.node-tools"), map[string]any{"subjects": []any{"mesh-delivery-view.delivered"}, "timeout": 1})
if took := time.Since(start); took < time.Second || took > 3*time.Second {
t.Errorf("a wait the bus refused whole answered after %s, want its timeout of 1s", took)
}
if !isErr || !strings.Contains(text, "the bus refused this account a subscription on mesh.seat.mesh-delivery-view.event.delivered") ||
!strings.Contains(text, "novox/hq ADR 0670 §1") || strings.Contains(text, "wait grant") || strings.Contains(text, "0367 §2") {
t.Errorf("a wait the bus refused whole: %s", text)
}
})
// The bus refusing every subscription of the call: answered after the timeout, in §16's words.
b, err := server.NewServer(&server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, NoLog: true, NoSigs: true,
Users: []*server.User{{Username: "desk.node-tools", Password: "pw", Permissions: &server.Permissions{
Subscribe: &server.SubjectPermission{Allow: []string{"_INBOX.desk.node-tools.>", "mesh.assignment.desk.*"}},
}}}})
if err != nil {
t.Fatal(err)
}
go b.Start()
if !b.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(b.Shutdown)
conn, err := bus.Connect(bus.Credential{URL: b.ClientURL(), Module: "node-tools", Node: "desk", User: "desk.node-tools", Password: "pw"})
if err != nil {
t.Fatal(err)
}
conn.Logf = func(string, ...any) {}
t.Cleanup(conn.Close)
start := time.Now()
text, isErr := callWait(NewSurface(conn, nil, "desk.node-tools"), map[string]any{"subjects": []any{"mesh-delivery-view.delivered"}, "timeout": 1})
if took := time.Since(start); took < time.Second || took > 3*time.Second {
t.Errorf("a wait the bus refused whole answered after %s, want its timeout of 1s", took)
}
if !isErr || !strings.Contains(text, "the bus refused this account a subscription on mesh.seat.mesh-delivery-view.event.delivered") ||
!strings.Contains(text, "novox/hq ADR 0670 §1") || strings.Contains(text, "wait grant") || strings.Contains(text, "0367 §2") {
t.Errorf("a wait the bus refused whole: %s", text)
}
// Design 34 every-event §37: a wait whose subjects overlap a marked event's, or a stream's with a `grant`, runs only on the
// operator's warrant for that one wait; with none it subscribes nothing, asks once and answers at its timeout or the choice.
markedWaits(t, s, mesh)
// Each group of rows has a runtime of its own, so the waits of the groups run side by side (A-311: the one-second floor of
// a wait's timeout stands, and no clock is injected).
for name, group := range map[string]func(*testing.T, *Surface, *mt.Mesh){
"no warrant": markedWaitsWithNoWarrant, "a warrant": markedWaitsOnAWarrant, "the catalog half unread": markedWaitsBeforeTheHalfIsRead,
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
s, mesh := waitingRuntime(t)
group(t, s, mesh)
})
}
}
// markedWaits are §37's rows, against a gate on the test bus, the router's records written there as the router writes them.
func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
// markedWaitsWithNoWarrant are §37's rows of a wait with no warrant, against a gate on the test bus, the router's records written there as the router writes them.
func markedWaitsWithNoWarrant(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
@@ -321,6 +338,17 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
!strings.Contains(text, `"timed-out": true`) || gw.asks() != 3 {
t.Errorf("from a seat: %s, %d asks", text, gw.asks())
}
}
// markedWaitsOnAWarrant are §37's rows of a wait chosen while it waits and of one on a warrant.
func markedWaitsOnAWarrant(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
start := time.Now()
text, isErr := callWait(g, args)
return text, isErr, time.Since(start)
}
// The most timeout a wait takes, its ask chosen while it waits: answered at the choice; the same call made then
// subscribes and hears the event published after it, answered with its approval; a warrant for other names, another
// filter or another timeout is refused, asked anew.
@@ -328,7 +356,7 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
done := make(chan string, 1)
go func() { text, _, d := took(long); done <- fmt.Sprintf("%s after %s", text, d) }()
mt.Until(t, func() error {
if gw.asks() < 4 {
if gw.asks() < 1 {
return os.ErrNotExist
}
return nil
@@ -342,7 +370,7 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
case <-time.After(10 * time.Second):
t.Fatal("a long marked wait was not answered at the choice")
}
before = g.conn.Subscriptions()
before := g.conn.Subscriptions()
go func() {
listening(t, g, before, 6)
mesh.Emit(t, "mesh.mod.mail.event.message", map[string]any{"n": 2})
@@ -361,6 +389,17 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
t.Errorf("another wait than the warrant's %v: %s", other, text)
}
}
}
// markedWaitsBeforeTheHalfIsRead are §37's rows of a wait before the catalog half is read, and of one whose runner build is unknown.
func markedWaitsBeforeTheHalfIsRead(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
start := time.Now()
text, isErr := callWait(g, args)
return text, isErr, time.Since(start)
}
// Before the catalog half is read nothing is asked: judged when its first view comes within the timeout, refused at
// the timeout when it does not.
gw.cat.mu.Lock()
@@ -562,17 +601,23 @@ func TestMeshWaitIsAnnouncedAmongSeven(t *testing.T) {
func fmtAnswer(out map[string]any) string { raw, _ := json.Marshal(out); return string(raw) }
// fakeApprovals is a gate as mesh_approvals reads it.
type fakeApprovals struct{ cancelled string }
type fakeApprovals struct {
cancelled string
// marked is the one `<module>.<tool>` its Describe says is marked.
marked string
}
func (f *fakeApprovals) List() ([]approval.Listed, error) {
return []approval.Listed{{Ask: "a1", State: "asked"}}, nil
}
func (f *fakeApprovals) Read(id string) (approval.Whole, error) { return approval.Whole{}, nil }
func (f *fakeApprovals) Applied() []approval.Marks { return nil }
func (f *fakeApprovals) Describe(_, _, _ string) (bool, string) { return false, "" }
func (f *fakeApprovals) Again(string) error { return nil }
func (f *fakeApprovals) Cancel(id string) error { f.cancelled = id; return nil }
func (f *fakeApprovals) Fill() (uint64, int64, error) { return 10, 64 << 20, nil }
func (f *fakeApprovals) Describe(module, _, name string) (bool, string) {
return f.marked != "" && f.marked == module+"."+name, "entry " + module
}
func (f *fakeApprovals) Again(string) error { return nil }
func (f *fakeApprovals) Cancel(id string) error { f.cancelled = id; return nil }
func (f *fakeApprovals) Fill() (uint64, int64, error) { return 10, 64 << 20, nil }
// A client that hangs up or cancels ends its wait at once: the subscription is closed and nothing is left
// waiting for the timeout (the review of issue 745). Over the console's own HTTP, as an agent host calls it.
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
+158 -44
View File
@@ -20,53 +20,57 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// The owner and its first-view tools (the tools' addresses are `<owner>.<tool>`).
const Assigner = "mesh-delivery-assigner"
// AssignerSeat is the seat whose holder's views the runner follows: the holder is read from the catalog half's claims of the
// seat, never named in code (design 47 G §616, §428), and the names the console reads the two states by are the seat's
// (`<seat>.<state>`; the manifest's `consumes` names their views families by the seat, as `seat:<seat>.views.<state>`).
const AssignerSeat = meshmoduleassigner.Seat
// The names the console reads the assigner's two states by (`<owner>.<state>`), as the manifest's `consumes` names their views
// families (no seat's client names a holder, novox/hq ADR 0625 §4; TestTheFirstViewsAreTheManifestsViews).
const (
AssignmentsRead = Assigner + ".assignments"
SeatHoldersRead = Assigner + ".seat-holders"
AssignmentsRead = AssignerSeat + ".assignments"
SeatHoldersRead = AssignerSeat + ".seat-holders"
)
// Spec is one first view the runner holds.
type Spec struct{ Owner, Tool, State string }
// Spec is one first view the runner holds of the seat's holder: its first-view tool and the state.
type Spec struct{ Tool, State string }
// Specs are the two owner states asked of an owner's tool; the catalog half's views are adopted by the branch follower.
// Specs are the two states asked of the holder's tool; the catalog half's views are adopted by the branch follower.
var Specs = []Spec{
{Assigner, "assignments", "assignments"},
{Assigner, "seat-holders", "seat-holders"},
{"assignments", "assignments"},
{"seat-holders", "seat-holders"},
}
// Name is the name a state is read by, `<owner>.<state>`: AssignmentsRead and SeatHoldersRead.
func (s Spec) Name() string { return s.Owner + "." + s.State }
// Name is the name a state is read by, `<seat>.<state>`: AssignmentsRead and SeatHoldersRead.
func (s Spec) Name() string { return AssignerSeat + "." + s.State }
// Views is the first views: the assigner's two, and the catalog half's per watched branch.
type Views struct {
logf func(string, ...any)
mu sync.Mutex
followers map[string]*views.Follower // by `<owner>.<state>`
byState map[string]*views.Follower // by the state an event names, the owners' views families
adopted map[string]*adoption // by `<owner>.<state>`, the views the branch follower adopted
hearers map[string][]*adoption // event key `<seat>.<event>` -> the adopted views that hear it
branches *meshcatalogclient.Branches
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
deps Deps
logf func(string, ...any)
mu sync.Mutex
followers map[string]*views.Follower // by `<owner>.<state>`
byState map[string]*views.Follower // by the state an event names, the owners' views families
adopted map[string]*adoption // by `<owner>.<state>`, the views the branch follower adopted
unheardKeys map[string]bool // event keys said as heard by no view
hearers map[string][]*adoption // event key `<seat>.<event>` -> the adopted views that hear it
branches *meshcatalogclient.Branches
holder *holding // the module that claims the assigner seat, once the catalog half names it
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
deps Deps
// heard is closed and made again at every event heard, so a wait for the picture wakes on events, never on a timer.
heard chan struct{}
// onKey are the hearers of one key of an owner's state, by `<state> <key>` (OnSeatHolder).
@@ -89,6 +93,9 @@ type Deps struct {
Findings FindingWriter
// StartFindings starts that writer, and its stop ends it; nil starts none.
StartFindings func(ctx context.Context) (stop func(), err error)
// Build is the build this process was started with (process.OwnBuild, read once at the start): its fingerprint, or
// why it is unknown (design 47 G §205, §424).
Build process.Build
// Node is the machine the runner is on, named in its findings' subject.
Node string
// Bound and Wait are the follower's (zero: two minutes, five seconds).
@@ -111,7 +118,7 @@ func Start(parent context.Context, conn *bus.Conn, logf func(string, ...any)) (*
// state adapter, JetStream and node are this connection's, in this one place (design 47 G A-16, A-27).
func DepsOf(conn *bus.Conn, logf func(string, ...any)) Deps {
me := conn.Module()
d := Deps{Module: me, Node: conn.Node(),
d := Deps{Module: me, Node: conn.Node(), Build: process.OwnBuild(),
Ask: func(address string, body any) (json.RawMessage, error) {
got, err := conn.Ask(address, body, "")
return got.Result, err
@@ -168,22 +175,6 @@ func StartWith(parent context.Context, d Deps, logf func(string, ...any)) (*View
ctx, cancel := context.WithCancel(parent)
v := &Views{logf: logf, ctx: ctx, cancel: cancel, followers: map[string]*views.Follower{}, byState: map[string]*views.Follower{},
adopted: map[string]*adoption{}, hearers: map[string][]*adoption{}, deps: d}
for _, s := range Specs {
s := s
call := func(tool string, body any) (json.RawMessage, error) { return d.Ask(s.Owner+"."+tool, body) }
f := &views.Follower{Owner: s.Owner, Tool: s.Tool, State: s.State, Ask: views.AskVia(call, s.Tool), Logf: logf,
Bound: d.Bound, Wait: d.Wait,
Raise: func(st views.Status) {
v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, true)
},
Clear: func(st views.Status) {
v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, false)
}}
v.followers[s.Name()] = f
v.byState[s.State] = f
v.wg.Add(1)
go func() { defer v.wg.Done(); _ = f.Start(ctx) }()
}
stopFindings := func() {}
if d.StartFindings != nil {
if stop, err := d.StartFindings(ctx); err != nil {
@@ -199,6 +190,9 @@ func StartWith(parent context.Context, d Deps, logf func(string, ...any)) (*View
logf("[mesh-tools] the catalog half is not followed (%v): the console cannot say which modules exist", err)
} else {
branches.Logf = func(format string, args ...any) { logf("[mesh-tools] "+format, args...) }
branches.Changed = func(_, module string, entry json.RawMessage, deleted bool, _ time.Time) {
v.claimed(module, entry, deleted)
}
v.branches = branches
}
stop, err := d.Consume(v.hear)
@@ -214,7 +208,7 @@ func StartWith(parent context.Context, d Deps, logf func(string, ...any)) (*View
func (v *Views) hear(env bus.Envelope) error {
defer v.heardOne()
v.mu.Lock()
hs := append([]*adoption(nil), v.hearers[env.Key]...)
hs := v.hearersOf(env.Key)
v.mu.Unlock()
if len(hs) > 0 { // an event of the catalog half or the gate's source, heard by the views the branch follower adopted
for _, a := range hs {
@@ -226,9 +220,12 @@ func (v *Views) hear(env bus.Envelope) error {
}
ev, err := views.DecodeEvent(env.Body)
if err != nil {
v.unheard(env.Key)
return nil // not a views event: nothing to hold, nothing to retry
}
v.mu.Lock()
f := v.byState[ev.State]
v.mu.Unlock()
if f == nil || f.Owner != ev.Owner {
return nil
}
@@ -240,6 +237,37 @@ func (v *Views) hear(env bus.Envelope) error {
return nil
}
// hearersOf are the adopted views that hear the event key `<seat>.<event>` or, as its subject carries further tokens
// (`<event>.<branch>`, the mesh-catalog seat's catalog-revision), the key's shorter prefixes: the way mesh-sdk's views reader
// finds its adopted hears. The caller holds v.mu.
func (v *Views) hearersOf(key string) []*adoption {
for k := key; k != ""; {
if hs := v.hearers[k]; len(hs) > 0 {
return append([]*adoption(nil), hs...)
}
i := strings.LastIndex(k, ".")
if i <= 0 {
break
}
k = k[:i]
}
return nil
}
// unheard says once per key an event no adopted view and no owner's state hears, never dropping it silently.
func (v *Views) unheard(key string) {
v.mu.Lock()
if v.unheardKeys == nil {
v.unheardKeys = map[string]bool{}
}
seen := v.unheardKeys[key]
v.unheardKeys[key] = true
v.mu.Unlock()
if !seen {
v.logf("[mesh-tools] an event %s is heard by no view of this tool runner: it is dropped (said once per key)", key)
}
}
// finding writes or clears the runner's finding of a first view unanswered past its bound, through its writer (design 47 G A-28).
func (v *Views) finding(f condseat.Finding, tool string, raised bool) {
if v.deps.Findings == nil {
@@ -261,6 +289,13 @@ func (v *Views) finding(f condseat.Finding, tool string, raised bool) {
// not answered yet, so nothing is claimed of it.
func (v *Views) State(name string) (map[string][]byte, error) {
v.mu.Lock()
if state, ok := strings.CutPrefix(name, AssignerSeat+"."); ok { // the seat's name stands for its holder's
if v.holder == nil {
v.mu.Unlock()
return nil, fmt.Errorf("the catalog half names no holder of the seat %s yet, so %s has no first view", AssignerSeat, name)
}
name = v.holder.module + "." + state
}
f := v.followers[name]
v.mu.Unlock()
if f == nil {
@@ -297,6 +332,10 @@ func (v *Views) Status() []views.Status {
return out
}
// OwnBuild is the build the tool runner was started with: the fingerprint of the want its node-engine applied, or why it is
// unknown (`own build unknown: started without MESH_FINGERPRINT`).
func (v *Views) OwnBuild() process.Build { return v.deps.Build }
// Entries are the entries by fingerprint the catalog half's follower keeps, with their state and since when (design 47 G A-103).
func (v *Views) Entries() any {
if v.branches == nil {
@@ -489,3 +528,78 @@ func (v *Views) Modules() (map[string]catalogseat.CatalogEntry, error) {
}
return out, nil
}
// holding is the views of the module that claims the assigner seat, and what stops them.
type holding struct {
module string
cancel context.CancelFunc
views []*views.Follower
}
// claimed is the branch follower's word of one entry changed: an entry that serves a verb of the assigner seat makes its module
// the holder whose views the runner follows (a holder before it is dropped), the deletion of the holder's entry drops them
// (design 47 G §616, §428). It acts on the event alone; nothing waits on a timer for the claim.
func (v *Views) claimed(module string, entry json.RawMessage, deleted bool) {
claims := false
if !deleted {
var e catalogseat.CatalogEntry
if json.Unmarshal(entry, &e) == nil {
claims = slices.ContainsFunc(e.Serves, func(s string) bool { return strings.HasPrefix(s, AssignerSeat+".") })
}
}
v.mu.Lock()
cur := v.holder
switch {
case claims && cur != nil && cur.module == module, !claims && (cur == nil || cur.module != module), v.ctx.Err() != nil:
v.mu.Unlock()
return
}
if cur != nil {
v.dropHolderLocked()
}
if claims {
v.startHolderLocked(module)
}
v.mu.Unlock()
}
// startHolderLocked follows the first views of module, the holder of the seat, on the runner's own consumer.
func (v *Views) startHolderLocked(module string) {
ctx, cancel := context.WithCancel(v.ctx)
h := &holding{module: module, cancel: cancel}
d := v.deps
for _, s := range Specs {
s := s
call := func(tool string, body any) (json.RawMessage, error) { return d.Ask(module+"."+tool, body) }
f := &views.Follower{Owner: module, Tool: s.Tool, State: s.State, Ask: views.AskVia(call, s.Tool), Logf: v.logf,
Bound: d.Bound, Wait: d.Wait,
Raise: func(st views.Status) {
v.finding(unansweredFinding(d.Module, d.Node, st, time.Now()), st.Tool, true)
},
Clear: func(st views.Status) {
v.finding(unansweredFinding(d.Module, d.Node, st, time.Now()), st.Tool, false)
}}
v.followers[module+"."+s.State] = f
v.byState[s.State] = f
h.views = append(h.views, f)
v.wg.Add(1)
go func() { defer v.wg.Done(); _ = f.Start(ctx) }()
}
v.holder = h
v.logf("[mesh-tools] the seat %s is held by %s: its views are followed", AssignerSeat, module)
}
// dropHolderLocked stops the views of the holder and clears the findings they raised.
func (v *Views) dropHolderLocked() {
h := v.holder
v.holder = nil
h.cancel()
for _, f := range h.views {
delete(v.followers, f.Owner+"."+f.State)
delete(v.byState, f.State)
if st := f.Status(); st.Raised {
go v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, false)
}
}
v.logf("[mesh-tools] %s no longer holds the seat %s: its views are no longer followed", h.module, AssignerSeat)
}
@@ -17,6 +17,10 @@ import (
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
@@ -41,7 +45,9 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
if err := json.Unmarshal(raw, &manifest); err != nil {
t.Fatal(err)
}
streamOf := map[string]string{Assigner: catalogue.StreamBusName("assigner-views")}
// the manifest names the assigner's views by the seat; the module the catalog half says claims it is the fixture's
const holder = "assigner-fixture"
streamOf := map[string]string{"seat:" + AssignerSeat: catalogue.StreamBusName("assigner-views")}
filters := map[string][]string{}
families := 0
for _, c := range manifest.Consumes {
@@ -53,16 +59,16 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
if stream == "" {
t.Fatalf("the manifest consumes %s, whose views stream the test does not know", c)
}
filters[stream] = append(filters[stream], process.ModuleStreamSubject(emitter, views.EventPrefix+event))
filters[stream] = append(filters[stream], process.ModuleStreamSubject(holder, views.EventPrefix+event))
families++
}
if families != len(Specs) {
t.Fatalf("the manifest consumes %d families, the runner holds %d first views", families, len(Specs))
}
runner := mt.MembershipOf("node-tools", "anchor", false, nil)
for _, owner := range []string{Assigner} {
stream := streamOf[owner]
mesh.ModuleStream(t, owner, stream)
for _, seat := range []string{"seat:" + AssignerSeat} {
stream := streamOf[seat]
mesh.ModuleStream(t, holder, stream)
mesh.ConsumerOn(t, stream, "anchor", "node-tools", filters[stream], 2*time.Second)
runner.Hears = append(runner.Hears, bus.HeardIssued{Stream: stream, Consumer: "anchor_node-tools"})
}
@@ -93,13 +99,13 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
t.Cleanup(stop)
// The owners are down: no tool answers, nothing is claimed, and past the bound each view is a finding.
if _, err := v.State(Assigner + ".assignments"); err == nil {
if _, err := v.State(AssignmentsRead); err == nil {
t.Fatal("a state its owner has not answered is claimed")
}
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(findings) != len(Specs)+catalogViews {
if len(findings) != catalogViews {
return errFindings(len(findings))
}
return nil
@@ -111,10 +117,25 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
t.Fatal(err)
}
t.Cleanup(nc.Close)
respond := func(subject string, answer any) {
if _, err := nc.Subscribe(subject, func(m *nats.Msg) {
raw, _ := json.Marshal(answer)
body, _ := json.Marshal(map[string]json.RawMessage{"result": raw})
_ = m.Respond(body)
}); err != nil {
t.Fatal(err)
}
}
// the catalog half answers: the gate's source names main, whose entries say the fixture claims the assigner seat
respond("mesh.seat.mesh-delivery.tool.source", delivery.SourceAnswer{Branches: []string{"main"}})
respond("mesh.seat.mesh-catalog.tool.registrations", catalogseat.RegistrationsAnswer{Registrations: []catalogseat.Registration{}})
respond("mesh.seat.mesh-catalog.tool.revision", catalogseat.RevisionAnswer{
Revision: &catalogseat.CatalogRevision{Revision: 3, Kind: meshcatalogclient.KindTrunk, Branch: "main"},
Entries: []catalogseat.CatalogEntry{{Module: holder, Serves: []string{AssignerSeat + ".assignments"}}}})
for _, s := range Specs {
s := s
if _, err := nc.Subscribe("mesh.mod."+s.Owner+".tool."+s.Tool, func(m *nats.Msg) {
answer, _ := json.Marshal(views.Answer{V: 1, Owner: s.Owner, State: s.State, Revision: 4, Created: time.Unix(1000, 0).UTC(),
if _, err := nc.Subscribe("mesh.mod."+holder+".tool."+s.Tool, func(m *nats.Msg) {
answer, _ := json.Marshal(views.Answer{V: 1, Owner: holder, State: s.State, Revision: 4, Created: time.Unix(1000, 0).UTC(),
Values: map[string]json.RawMessage{"first": json.RawMessage(`{"n":1}`)}})
body, _ := json.Marshal(map[string]json.RawMessage{"result": answer})
_ = m.Respond(body)
@@ -130,7 +151,7 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(findings) != catalogViews { // the seats' verbs are not served in this test: the catalog half stays unanswered
if len(findings) != 0 { // every view answered clears its finding
return errFindings(len(findings))
}
return nil
@@ -138,9 +159,9 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
// Each owner's next event arrives on the runner's consumer: all four families are bound.
for _, s := range Specs {
ev, _ := json.Marshal(views.Event{V: 1, Owner: s.Owner, State: s.State, Key: "next", Revision: 5, Created: time.Unix(1000, 0).UTC(),
ev, _ := json.Marshal(views.Event{V: 1, Owner: holder, State: s.State, Key: "next", Revision: 5, Created: time.Unix(1000, 0).UTC(),
Value: json.RawMessage(`{"n":2}`)})
mesh.Emit(t, process.ModuleStreamSubject(s.Owner, views.EventPrefix+s.State), json.RawMessage(ev))
mesh.Emit(t, process.ModuleStreamSubject(holder, views.EventPrefix+s.State), json.RawMessage(ev))
}
for _, s := range Specs {
name := s.Name()
+79 -26
View File
@@ -12,9 +12,9 @@ import (
"time"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/novox/mesh-tools/node-tools/internal/bus"
@@ -32,12 +32,14 @@ type owners struct {
// catalog is the revision of the watched branch main the seat mesh-catalog answers, and modules its entries
catalog int64
modules []string
// holder is the module among them that claims the assigner seat
holder string
}
func newOwners() *owners {
return &owners{state: map[string]map[string]string{"assignments": {"anchor": `{"p":1}`}, "seat-holders": {"seat": `{"h":1}`}},
rev: map[string]uint64{"assignments": 4, "seat-holders": 2}, asked: map[string]int{}, finds: map[string]bool{},
catalog: 3, modules: []string{"web"}}
catalog: 3, modules: []string{"web", fixtureHolder}, holder: fixtureHolder}
}
// Write and Clear are the runner's finding writer in a test: the findings written and not cleared, by their key.
@@ -71,7 +73,11 @@ func (o *owners) ask(address string, _ any) (json.RawMessage, error) {
case "seat:mesh-catalog.revision":
a := catalogseat.RevisionAnswer{Revision: &catalogseat.CatalogRevision{Revision: o.catalog, Kind: meshcatalogclient.KindTrunk, Branch: "main"}}
for _, m := range o.modules {
a.Entries = append(a.Entries, catalogseat.CatalogEntry{Module: m, Manifest: map[string]any{"module": m, "tools": []string{m + "_check"}}})
e := catalogseat.CatalogEntry{Module: m, Manifest: map[string]any{"module": m, "tools": []string{m + "_check"}}}
if m == o.holder {
e.Serves = []string{AssignerSeat + ".assignments", AssignerSeat + ".seat-holders"}
}
a.Entries = append(a.Entries, e)
}
return json.Marshal(a)
}
@@ -94,14 +100,10 @@ func (o *owners) say(t *testing.T, state, key, value string, revision uint64, ga
}
}
func owner(state string) string {
for _, s := range Specs {
if s.State == state {
return s.Owner
}
}
return ""
}
// fixtureHolder is the module the fixture catalog half says claims the assigner seat: no name is in the code.
const fixtureHolder = "assigner-fixture"
func owner(state string) string { return fixtureHolder }
func eventually(t *testing.T, what string, ok func() bool) {
t.Helper()
@@ -127,39 +129,43 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
defer stop()
// Down: nothing is claimed, and past the bound each unanswered view is a finding.
if _, err := v.State("mesh-delivery-assigner.assignments"); err == nil || !strings.Contains(err.Error(), "not been answered") {
t.Fatalf("a state its owner has not answered is claimed: %v", err)
if _, err := v.State(AssignmentsRead); err == nil || !strings.Contains(err.Error(), "names no holder") {
t.Fatalf("a state whose holder is not known is claimed: %v", err)
}
eventually(t, "no finding past the bound", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 4 })
eventually(t, "no finding past the bound", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 2 })
// Up: answered, the findings are cleared, and the pictures hold the owners' values.
o.mu.Lock()
o.down = false
o.mu.Unlock()
eventually(t, "not answered", func() bool { _, err := v.State("mesh-delivery-assigner.seat-holders"); return err == nil })
eventually(t, "not answered", func() bool { _, err := v.State(SeatHoldersRead); return err == nil })
eventually(t, "findings not cleared", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 0 })
if got, _ := v.Entry("mesh-delivery-assigner.seat-holders", "none"); got != nil {
if got, _ := v.Entry(AssignerSeat+".seat-holders", "none"); got != nil {
t.Errorf("an absent key is %q", got)
}
// The catalog half is followed by the shared branch follower on the runner's own consumer: the gate's source names main,
// main's view is adopted, and its entries are the modules, each with its manifest (design 47 G §196, §614).
eventually(t, "the catalog half holds no modules", func() bool { m, err := v.Modules(); return err == nil && len(m) == 1 })
eventually(t, "the catalog half holds no modules", func() bool { m, err := v.Modules(); return err == nil && len(m) == 2 })
m, _ := v.Modules()
if m["web"].Manifest["module"] != "web" {
t.Errorf("the module web is %+v", m["web"])
}
// A catalog-revision of the branch is heard on the same consumer and asks the view once more.
o.mu.Lock()
o.catalog, o.modules = 4, []string{"web", "db"}
o.catalog, o.modules = 4, []string{"web", "db", fixtureHolder}
o.mu.Unlock()
ev, _ := json.Marshal(catalogseat.CatalogRevisionEvent{V: 1, Branch: "main", Kind: meshcatalogclient.KindTrunk, Revision: 4})
if err := o.deliver(bus.Envelope{Key: catalogseat.Seat + "." + catalogseat.EventCatalogRevision, Body: ev}); err != nil {
if err := o.deliver(bus.Envelope{Key: catalogseat.Seat + "." + catalogseat.EventCatalogRevision + ".main", Body: ev}); err != nil {
t.Fatal(err)
}
eventually(t, "the catalog-revision was not taken", func() bool { m, err := v.Modules(); return err == nil && len(m) == 2 })
eventually(t, "the catalog-revision was not taken", func() bool { m, err := v.Modules(); return err == nil && len(m) == 3 })
// An event no view hears is said once per key, not dropped silently.
for i := 0; i < 2; i++ {
_ = o.deliver(bus.Envelope{Key: "some-seat.unknown-event.x", Body: json.RawMessage(`"x"`)})
}
// And the halves by branch, with their revision, which mesh_wait names checking an emitter (design 34 every-event §14).
if h, err := v.Halves(); err != nil || h["main"].Revision.Revision != 4 || len(h["main"].Entries) != 2 {
if h, err := v.Halves(); err != nil || h["main"].Revision.Revision != 4 || len(h["main"].Entries) != 3 {
t.Errorf("the halves: %+v, %v", h, err)
}
@@ -170,7 +176,7 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
o.say(t, "assignments", "laptop", `{"p":0}`, 5, false)
o.say(t, "assignments", "anchor", "", 6, false)
// (a deleted mark carries no value: Deleted is the encoding's, set below)
if got, _ := v.Entry("mesh-delivery-assigner.assignments", "laptop"); string(got) != `{"p":2}` {
if got, _ := v.Entry(AssignerSeat+".assignments", "laptop"); string(got) != `{"p":2}` {
t.Errorf("the next revision is not applied: %q", got)
}
if n := asked("assignments"); n != before {
@@ -188,7 +194,7 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
assignments = s
}
}
if assignments.Owner != "mesh-delivery-assigner" || assignments.Tool != "assignments" || assignments.Asks < 2 || assignments.Answered.IsZero() {
if assignments.Owner != fixtureHolder || assignments.Tool != "assignments" || assignments.Asks < 2 || assignments.Answered.IsZero() {
t.Errorf("the status of assignments is %+v", assignments)
}
manifests := asked("seat-holders")
@@ -205,6 +211,9 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
}
lmu.Lock()
defer lmu.Unlock()
if n := strings.Count(strings.Join(logs, "\n"), "heard by no view"); n != 1 {
t.Errorf("an unheard event was said %d times, want once: %v", n, logs)
}
if !strings.Contains(strings.Join(logs, "\n"), "a gap in") {
t.Errorf("the gap was not said in the log: %v", logs)
}
@@ -228,13 +237,19 @@ func TestTheFirstViewsAreTheManifestsViews(t *testing.T) {
t.Errorf("the first view %s is read by no name here", s.Name())
}
delete(want, s.Name())
if view := s.Owner + ".views." + s.Tool; !slices.Contains(m.Consumes, view) {
// the family is named by the seat, never by its holder (design 47 G §616)
if view := "seat:" + AssignerSeat + ".views." + s.Tool; !slices.Contains(m.Consumes, view) {
t.Errorf("the first view %s has no family in the manifest's consumes %v", view, m.Consumes)
}
}
if len(want) > 0 {
t.Errorf("no first view reads %v", want)
}
for _, c := range m.Consumes {
if strings.Contains(c, "assigner") && !strings.HasPrefix(c, "seat:") {
t.Errorf("the manifest names the assigner by a module: %s", c)
}
}
// the catalog half's follower hears these events and asks these verbs, and the manifest says so (design 47 G §614)
for _, ev := range []string{"mesh-catalog.catalog-revision", "mesh-catalog.registration-changed", "mesh-delivery.source-moved"} {
if !slices.Contains(m.Consumes, ev) {
@@ -246,4 +261,42 @@ func TestTheFirstViewsAreTheManifestsViews(t *testing.T) {
t.Errorf("the manifest still consumes the indexer's %s", ev)
}
}
// design 47 G §616: the holder of the assigner seat is whichever module the catalog half says serves it; when another one
// claims it the views are of the new holder, and when none does the seat's names answer that the catalog half names no holder.
t.Run("TheAssignersViewsFollowTheModuleThatClaimsTheSeat", func(t *testing.T) {
o := newOwners()
deps := Deps{Module: "node-tools", Ask: o.ask, Bound: time.Second, Wait: 10 * time.Millisecond,
Consume: func(deliver func(bus.Envelope) error) (func(), error) { o.deliver = deliver; return func() {}, nil }, Findings: o}
v, stop := StartWith(context.Background(), deps, func(string, ...any) {})
defer stop()
eventually(t, "not answered", func() bool { _, err := v.State(AssignmentsRead); return err == nil })
ownerOf := func() string {
for _, s := range v.Status() {
if s.State == "assignments" {
return s.Owner
}
}
return ""
}
if ownerOf() != fixtureHolder {
t.Fatalf("the views are of %q", ownerOf())
}
revise := func(rev int64, holder string, modules ...string) {
o.mu.Lock()
o.catalog, o.modules, o.holder = rev, modules, holder
o.mu.Unlock()
ev, _ := json.Marshal(catalogseat.CatalogRevisionEvent{V: 1, Branch: "main", Kind: meshcatalogclient.KindTrunk, Revision: rev})
if err := o.deliver(bus.Envelope{Key: catalogseat.Seat + "." + catalogseat.EventCatalogRevision, Body: ev}); err != nil {
t.Fatal(err)
}
}
revise(4, "other-holder", "web", "other-holder")
eventually(t, "the views did not move to the new holder", func() bool { return ownerOf() == "other-holder" })
eventually(t, "not answered", func() bool { _, err := v.State(SeatHoldersRead); return err == nil })
revise(5, "", "web")
eventually(t, "the views stayed after the claim went", func() bool {
_, err := v.State(AssignmentsRead)
return err != nil && strings.Contains(err.Error(), "names no holder")
})
})
}
+56 -2
View File
@@ -121,7 +121,7 @@ type Bus interface {
// busStateCalls are the methods of mesh-sdk buscalls a bundle asks and BusState answers.
var busStateCalls = map[string]bool{
buscalls.ObjectsPut: true, buscalls.ObjectsGet: true, buscalls.ObjectsList: true, buscalls.ObjectsDelete: true,
buscalls.ObjectsInfo: true, buscalls.StreamRead: true, buscalls.StreamFill: true, buscalls.WorkQueueSize: true,
buscalls.ObjectsInfo: true, buscalls.StreamRead: true, buscalls.StreamFill: true, buscalls.StreamPublish: true, buscalls.WorkQueueSize: true,
buscalls.WorkQueueDead: true, buscalls.LedgerApply: true, buscalls.LedgerGet: true, buscalls.LedgerHistory: true,
buscalls.LedgerQuery: true, buscalls.LedgerRebuild: true, buscalls.LedgerReindex: true, buscalls.LedgerSnapshot: true,
buscalls.LedgerLag: true, buscalls.ObjectsSentGet: true, buscalls.ObjectsPutChunk: true, buscalls.ObjectsGetChunk: true,
@@ -387,7 +387,29 @@ func Start(module, entry string, env []string, mesh Bus, logf func(string, ...an
asked := time.Now()
switch m.Method {
case "mesh/publish":
err = mesh.Publish(m.Params)
var said struct {
Ack bool `json:"ack"`
}
_ = json.Unmarshal(m.Params, &said)
if !said.Ack {
err = mesh.Publish(m.Params)
break
}
// With ack the runtime answers only once the bus acknowledged the event: its sequence, or {error}
// (design 47 G §559). A refusal is an answer, so a writer says it as a finding's resend.
acker, ok := mesh.(interface {
PublishAcked(params json.RawMessage) (uint64, error)
})
switch {
case !ok:
result = map[string]any{"error": "this tool runner's bus publishes no event with an acknowledgement"}
default:
if seq, perr := acker.PublishAcked(m.Params); perr != nil {
result = map[string]any{"error": perr.Error()}
} else {
result = map[string]any{"sequence": seq}
}
}
case "mesh/ask":
var asked json.RawMessage
if asked, err = mesh.Ask(m.Params); err == nil {
@@ -404,6 +426,38 @@ func Start(module, entry string, env []string, mesh Bus, logf func(string, ...an
result = json.RawMessage(raw)
}
case "mesh/subscribe":
var said struct {
Pattern string `json:"pattern"`
Deliver string `json:"deliver"`
}
_ = json.Unmarshal(m.Params, &said)
if said.Deliver == "new" {
// An ordered consumer of its own from the next message on (design 47 G §559, A-153): each message
// is handed to this child as an event is, and the consumer ends with the child.
newer, ok := mesh.(interface {
SubscribeNew(pattern string, deliver func(json.RawMessage) error) (func(), error)
})
if !ok {
err = errors.New("this tool runner's bus makes no subscription for new messages")
break
}
var stop func()
stop, err = newer.SubscribeNew(said.Pattern, func(envelope json.RawMessage) error {
_, err := c.ask(module, "mesh/event", map[string]any{"envelope": envelope}, EventTimeout)
return err
})
if err == nil {
c.mu.Lock()
if c.why != nil {
c.mu.Unlock()
stop()
} else {
c.watches = append(c.watches, stop)
c.mu.Unlock()
}
}
break
}
mu.Lock()
subscriber = c
mu.Unlock()
+60
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
@@ -68,6 +69,65 @@ func TestABundleTakesASeatsWorkThroughItsOwnChannel(t *testing.T) {
if err != nil || string(got) != `{"taken":true}` {
t.Fatalf("the work was not handed to the bundle: %s %v", got, err)
}
// Design 47 G §576, §518: a bundle reads the runner's capability words with mesh/capabilities, then runs its put session
// through the runner's put-chunk, and the sessions of its channel are dropped when the channel closes.
t.Run("ABundleReadsTheCapabilityWordsRunsAPutSessionAndItsChannelsClosingDropsThem", func(t *testing.T) {
dir := t.TempDir()
entry := filepath.Join(dir, "bundle")
if err := os.WriteFile(entry, []byte(puttingBundle), 0o755); err != nil {
t.Fatal(err)
}
b := &sessionBus{dropped: make(chan struct{})}
l, err := Start("putter", entry, os.Environ(), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.dropped:
case <-time.After(5 * time.Second):
t.Fatal("the bundle's sessions were not dropped when its channel closed")
}
b.mu.Lock()
defer b.mu.Unlock()
if len(b.chunks) != 1 || !strings.HasPrefix(b.chunks[0], "mesh/objects.put-chunk ") || !strings.Contains(b.chunks[0], `"seq":1`) {
t.Fatalf("the runner took %v", b.chunks)
}
})
// Design 47 G §559: `mesh/publish` with `ack` is answered with its sequence once the bus acknowledged it, or `{error}`;
// `mesh/subscribe` with `deliver: new` makes a subscription for new messages that ends with the bundle's channel.
t.Run("APublishWithAckIsAnsweredWithItsSequenceAndASubscriptionForNewMessagesEndsWithTheChannel", func(t *testing.T) {
dir := t.TempDir()
entry, out := filepath.Join(dir, "bundle"), filepath.Join(dir, "out")
if err := os.WriteFile(entry, []byte(findingBundle), 0o755); err != nil {
t.Fatal(err)
}
b := &findingBus{stopped: make(chan struct{})}
l, err := Start("writer", entry, append(os.Environ(), "FINDING_OUT="+out), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.stopped:
case <-time.After(5 * time.Second):
t.Fatal("the subscription for new messages was not ended with the bundle's channel")
}
got, _ := os.ReadFile(out)
for _, want := range []string{`"sequence":42`, `"error":"the bus refused it"`} {
if !strings.Contains(string(got), want) {
t.Errorf("the bundle was answered %s, want it to hold %s", got, want)
}
}
b.mu.Lock()
defer b.mu.Unlock()
if b.pattern != "mesh-conditions.findings-asked" {
t.Errorf("the subscription was made for %q", b.pattern)
}
if len(b.published) != 0 {
t.Errorf("a publish with ack went the unacknowledged way: %v", b.published)
}
})
}
// A bundle that pulls one ask from its worker when it is listed.
+36 -26
View File
@@ -2,12 +2,9 @@ package launch
import (
"encoding/json"
"os"
"path/filepath"
"errors"
"strings"
"sync"
"testing"
"time"
)
// puttingBundle asks the capability words, and once they are answered with put-sessions sends the one chunk of a put
@@ -41,28 +38,41 @@ func (b *sessionBus) BusState(method string, params json.RawMessage) (json.RawMe
}
func (b *sessionBus) DropSessions() { close(b.dropped) }
// Design 47 G §576, §518: a bundle reads the runner's capability words with mesh/capabilities, then runs its put session
// through the runner's put-chunk, and the sessions of its channel are dropped when the channel closes.
func TestABundleReadsTheCapabilityWordsRunsAPutSessionAndItsChannelsClosingDropsThem(t *testing.T) {
dir := t.TempDir()
entry := filepath.Join(dir, "bundle")
if err := os.WriteFile(entry, []byte(puttingBundle), 0o755); err != nil {
t.Fatal(err)
// findingBundle asks, as mesh-sdk's finding writer does, a publish with ack that the bus takes, one it refuses, and a
// subscription for new messages; each answer it gets is written to $FINDING_OUT, and it ends after the subscription's.
const findingBundle = `#!/bin/sh
while IFS= read -r line; do
id=$(printf '%s' "$line" | sed -n 's/.*"id":\([0-9]*\)[,}].*/\1/p')
case "$line" in
*'"method":"initialize"'*) printf '{"jsonrpc":"2.0","id":%s,"result":{}}\n' "$id" ;;
*'"method":"tools/list"'*) printf '{"jsonrpc":"2.0","id":%s,"result":{"tools":[]}}\n' "$id"
printf '{"jsonrpc":"2.0","id":"a1","method":"mesh/publish","params":{"key":"taken","ack":true,"body":{}}}\n' ;;
*'"id":"a1"'*) printf '%s\n' "$line" >>"$FINDING_OUT"
printf '{"jsonrpc":"2.0","id":"a2","method":"mesh/publish","params":{"key":"refused","ack":true,"body":{}}}\n' ;;
*'"id":"a2"'*) printf '%s\n' "$line" >>"$FINDING_OUT"
printf '{"jsonrpc":"2.0","id":"s1","method":"mesh/subscribe","params":{"pattern":"mesh-conditions.findings-asked","deliver":"new"}}\n' ;;
*'"id":"s1"'*) printf '%s\n' "$line" >>"$FINDING_OUT"; exit 0 ;;
esac
done
`
type findingBus struct {
subscribing
mu sync.Mutex
pattern string
stopped chan struct{}
}
func (b *findingBus) PublishAcked(params json.RawMessage) (uint64, error) {
if strings.Contains(string(params), `"refused"`) {
return 0, errors.New("the bus refused it")
}
b := &sessionBus{dropped: make(chan struct{})}
l, err := Start("putter", entry, os.Environ(), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.dropped:
case <-time.After(5 * time.Second):
t.Fatal("the bundle's sessions were not dropped when its channel closed")
}
return 42, nil
}
func (b *findingBus) SubscribeNew(pattern string, _ func(json.RawMessage) error) (func(), error) {
b.mu.Lock()
defer b.mu.Unlock()
if len(b.chunks) != 1 || !strings.HasPrefix(b.chunks[0], "mesh/objects.put-chunk ") || !strings.Contains(b.chunks[0], `"seq":1`) {
t.Fatalf("the runner took %v", b.chunks)
}
b.pattern = pattern
b.mu.Unlock()
return func() { close(b.stopped) }, nil
}
@@ -0,0 +1,14 @@
package launch
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// A-42: a launched bundle may ask mesh/stream.publish.
func TestABundleMayAskStreamPublish(t *testing.T) {
if !busStateCalls[buscalls.StreamPublish] {
t.Fatal("mesh/stream.publish is not among the methods a bundle may ask")
}
}
+79
View File
@@ -58,6 +58,10 @@ type History interface {
// Read is the operations of the instances keys names (a key, with `*` for one name and `>` for the rest) from a
// sequence on, oldest first, at most limit of them (every one when limit is 0).
Read(keys string, from uint64, limit int) ([]Kept, error)
// Purge removes every operation of the instance key names, and nothing of another (an instance's removal).
Purge(key string) error
// Fill is the history's fill against its cap, the ledger's cap (buscalls.LedgerFill; design 47 G §587).
Fill() (buscalls.Fill, error)
}
// State is a ledger's state bucket: each instance's record under its key.
@@ -69,6 +73,8 @@ type State interface {
Write(key string, value []byte, revision uint64) (uint64, error)
// Keys is every key the bucket holds.
Keys() ([]string, error)
// Delete removes the key; a key it does not hold is no fault.
Delete(key string) error
}
// Snapshots are a ledger's snapshots: the latest record of each instance, on a subject of its own, never expiring.
@@ -77,6 +83,8 @@ type Snapshots interface {
Get(key string) ([]byte, uint64, error)
// Put writes the key's snapshot, expecting its last to be at expect (0: none); ErrRaced when it is not.
Put(key string, value []byte, expect uint64) error
// Purge removes the key's snapshot; one it does not hold is no fault.
Purge(key string) error
}
// Ledger is one ledger of one module: its types and where it is kept.
@@ -495,3 +503,74 @@ func (l *Ledger) Apply(r buscalls.ApplyRequest, author buscalls.Author) (buscall
}
return buscalls.Applied{Taken: true, Sequence: seq, StateBehind: behind}, nil
}
// Fill is the fill of the ledger's history against the ledger's cap (buscalls.LedgerFill; design 47 G §587): its messages,
// its bytes and the cap in bytes. The history holds every operation, and the cap is its alone: neither the fullest of the
// stores nor their sum is answered. A history that cannot say its fill is an error naming the ledger, never a fill of
// nothing.
func (l *Ledger) Fill(buscalls.LedgerRequest) (buscalls.Fill, error) {
f, err := l.History.Fill()
if err != nil {
return buscalls.Fill{}, fmt.Errorf("%s's ledger %s cannot say the fill of its history: %w", l.Module, l.Name, err)
}
return f, nil
}
// Remove removes one instance with everything it kept (buscalls.LedgerRemove; design 47 G §587, §155): its operations, its
// snapshot, its key in the state bucket and its keys in every declared index, and answers how many operations and how many
// bytes of the history that was. An instance the ledger does not hold answers 0 and 0, and a removal that left something
// (said, and answered as an error) is made again by calling it again, which removes the rest.
func (l *Ledger) Remove(r buscalls.LedgerRemoveRequest) (buscalls.LedgerRemoved, error) {
key, err := KeyOf(r.Instance)
if err != nil {
return buscalls.LedgerRemoved{}, err
}
out := buscalls.LedgerRemoved{Instance: r.Instance}
before, err := l.History.Fill()
if err != nil {
return out, fmt.Errorf("%s's ledger %s cannot say the fill of its history, so a removal is not made: %w", l.Module, l.Name, err)
}
ops, err := l.History.Read(key, 1, 0)
if err != nil {
return out, err
}
for _, step := range []struct {
what string
remove func(string) error
}{{"operations", l.History.Purge}, {"snapshot", l.Snapshots.Purge}, {"state", l.State.Delete}, {"index entries", l.dropFromIndexes}} {
if err := step.remove(key); err != nil {
l.say("[mesh-tools] %s's ledger %s removes %s and its %s could not be removed: %v; removing it again removes what "+
"is left", l.Module, l.Name, r.Instance, step.what, err)
return out, fmt.Errorf("%s's ledger %s removed part of %s and its %s are left: %w", l.Module, l.Name, r.Instance, step.what, err)
}
}
after, err := l.History.Fill()
if err != nil {
return out, fmt.Errorf("%s's ledger %s removed %s and cannot say the fill of its history after: %w", l.Module, l.Name, r.Instance, err)
}
out.Operations = uint64(len(ops))
if before.Bytes > after.Bytes {
out.Bytes = before.Bytes - after.Bytes
}
return out, nil
}
// dropFromIndexes removes the entries of the instance key names from every index bucket the ledger is bound to.
func (l *Ledger) dropFromIndexes(key string) error {
instance := InstanceOf(key)
for name, ix := range l.Indexes {
held, err := ix.Entries("", 0)
if err != nil {
return fmt.Errorf("the index %s: %w", name, err)
}
for _, e := range held {
var v buscalls.IndexEntry
if json.Unmarshal(e.Value, &v) == nil && v.Instance == instance {
if err := ix.Delete(e.Key); err != nil {
return fmt.Errorf("the index %s: %w", name, err)
}
}
}
}
return nil
}
+46
View File
@@ -99,4 +99,50 @@ func TestApplyAppendsDerivesAndSnapshots(t *testing.T) {
c.run(t, l, f)
})
}
// design 47 G §587: a ledger's fill is its history's against the ledger's cap, a store that cannot say it is an error naming
// the ledger; a removal takes an instance's operations, snapshot, state and index entries and answers what that was, an
// unknown instance answers none, and nothing of another instance goes.
t.Run("TheLedgerSaysItsFillAndRemovesAnInstanceWhole", func(t *testing.T) {
l, f := newLedger()
f.history.MaxBytes = 1 << 20
for _, n := range []string{"a", "b"} {
applied(t, l, "decision/0290", "note", `{"says":"`+n+`"}`, owner)
}
applied(t, l, "decision/0291", "note", `{"says":"other"}`, owner)
entry, _ := json.Marshal(buscalls.IndexEntry{Instance: "decision/0290", Sequence: 2})
other, _ := json.Marshal(buscalls.IndexEntry{Instance: "decision/0291", Sequence: 1})
_ = f.index.Put("t.decision.0290", entry)
_ = f.index.Put("t.decision.0291", other)
fill, err := l.Fill(buscalls.LedgerRequest{Ledger: "records"})
if err != nil || fill.Count != 3 || fill.Bytes == 0 || fill.MaxBytes != 1<<20 {
t.Fatalf("the fill is %+v (%v), want 3 messages against 1 MiB", fill, err)
}
got, err := l.Remove(buscalls.LedgerRemoveRequest{Ledger: "records", Instance: "decision/0290"})
if err != nil || got.Instance != "decision/0290" || got.Operations != 2 || got.Bytes == 0 {
t.Fatalf("the removal answered %+v (%v), want 2 operations and their bytes", got, err)
}
if kept, _ := f.history.Read("decision.0290", 1, 0); len(kept) != 0 {
t.Errorf("operations are left: %d", len(kept))
}
if raw, _, _ := f.state.Get("decision.0290"); raw != nil {
t.Errorf("the state is left: %s", raw)
}
if raw, _, _ := f.snapshots.Get("decision.0290"); raw != nil {
t.Errorf("the snapshot is left: %s", raw)
}
if held, _ := f.index.Entries("", 0); len(held) != 1 || held[0].Key != "t.decision.0291" {
t.Errorf("the index holds %v, want the other instance's entry alone", held)
}
if kept, _ := f.history.Read("decision.0291", 1, 0); len(kept) != 1 {
t.Errorf("another instance lost operations: %d left", len(kept))
}
if again, err := l.Remove(buscalls.LedgerRemoveRequest{Ledger: "records", Instance: "decision/0290"}); err != nil || again.Operations != 0 || again.Bytes != 0 {
t.Errorf("a removal said twice answered %+v (%v), want none", again, err)
}
f.history.FillFails = true
if _, err := l.Fill(buscalls.LedgerRequest{}); err == nil || !strings.Contains(err.Error(), "ledger records cannot say the fill") {
t.Errorf("a history that cannot say its fill answered %v", err)
}
})
}
+50 -3
View File
@@ -1,9 +1,12 @@
package ledger
import (
"errors"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// The memory stores keep what the bus keeps, as the bus answers it, for a test of a ledger without a bus: a stream whose
@@ -12,9 +15,13 @@ import (
// MemoryHistory is a history in memory.
type MemoryHistory struct {
mu sync.Mutex
seq uint64
kept []struct {
// MaxBytes is the cap its fill is said against.
MaxBytes uint64
// FillFails makes Fill an error, as a store that cannot say its fill.
FillFails bool
mu sync.Mutex
seq uint64
kept []struct {
key string
Kept
}
@@ -73,6 +80,32 @@ func (h *MemoryHistory) Read(keys string, from uint64, limit int) ([]Kept, error
return out, nil
}
func (h *MemoryHistory) Purge(key string) error {
h.mu.Lock()
defer h.mu.Unlock()
kept := h.kept[:0]
for _, k := range h.kept {
if k.key != key {
kept = append(kept, k)
}
}
h.kept = kept
return nil
}
func (h *MemoryHistory) Fill() (buscalls.Fill, error) {
h.mu.Lock()
defer h.mu.Unlock()
if h.FillFails {
return buscalls.Fill{}, errors.New("the history does not say its fill")
}
f := buscalls.Fill{Count: uint64(len(h.kept)), MaxBytes: h.MaxBytes}
for _, k := range h.kept {
f.Bytes += uint64(len(k.Data))
}
return f, nil
}
// Expire drops every operation of the history, as a ledger's keeping days end them.
func (h *MemoryHistory) Expire() {
h.mu.Lock()
@@ -129,6 +162,13 @@ func (s *MemoryState) Keys() ([]string, error) {
return out, nil
}
func (s *MemoryState) Delete(key string) error {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.kv, key)
return nil
}
// MemorySnapshots are snapshots in memory.
type MemorySnapshots struct {
mu sync.Mutex
@@ -157,6 +197,13 @@ func (s *MemorySnapshots) Put(key string, value []byte, expect uint64) error {
return nil
}
func (s *MemorySnapshots) Purge(key string) error {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.kv, key)
return nil
}
// MemoryIndex is an index's bucket in memory.
type MemoryIndex struct {
mu sync.Mutex
+7 -7
View File
@@ -23,7 +23,7 @@ import (
assigner "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
seatassigner "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
meshcatalog "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
meshcatalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/membership"
"github.com/nats-io/nats-server/v2/server"
@@ -156,7 +156,7 @@ func sdkMembership(t *testing.T, mem bus.Membership) membership.Membership {
// no holder).
const (
catalogHolder = "mesh-catalog"
assignerHolder = "mesh-delivery-assigner"
assignerHolder = "mesh-module-assigner" // the seat's name stands for its holder in the console's state names (firstview.AssignmentsRead)
// AssignmentsBucket and SeatHoldersBucket are the assigner's states assignments and seat-holders.
AssignmentsBucket = assignerHolder + "_assignments"
SeatHoldersBucket = assignerHolder + "_seat-holders"
@@ -303,7 +303,7 @@ func (l *Logs) All() string { return strings.Join(l.lines, "\n") }
// redelivery in seconds rather than the mesh's minutes.
func (m *Mesh) Consumer(t *testing.T, node, module string, filters []string, ackWait time.Duration) {
t.Helper()
cfg := &nats.ConsumerConfig{Durable: node + "_" + module, AckPolicy: nats.AckExplicitPolicy,
cfg := &nats.ConsumerConfig{Durable: node + "_" + module + "_tools", AckPolicy: nats.AckExplicitPolicy,
AckWait: ackWait, MaxDeliver: 10, DeliverPolicy: nats.DeliverNewPolicy}
if len(filters) == 1 {
cfg.FilterSubject = filters[0]
@@ -340,7 +340,7 @@ func (m *Mesh) Publish(t *testing.T, subject string, body any) {
// delivered.
func (m *Mesh) Pending(t *testing.T, node, module string) (ackPending, notDelivered uint64) {
t.Helper()
info, err := m.js.ConsumerInfo("EVENTS", node+"_"+module)
info, err := m.js.ConsumerInfo("EVENTS", node+"_"+module+"_tools")
if err != nil {
t.Fatal(err)
}
@@ -430,7 +430,7 @@ func (m *Mesh) ModuleStream(t *testing.T, module, name string) {
// ConsumerOn makes a module's durable consumer on a machine on any stream, as Consumer does on EVENTS.
func (m *Mesh) ConsumerOn(t *testing.T, stream, node, module string, filters []string, ackWait time.Duration) {
t.Helper()
cfg := &nats.ConsumerConfig{Durable: node + "_" + module, AckPolicy: nats.AckExplicitPolicy,
cfg := &nats.ConsumerConfig{Durable: node + "_" + module + "_tools", AckPolicy: nats.AckExplicitPolicy,
AckWait: ackWait, MaxDeliver: 10, DeliverPolicy: nats.DeliverNewPolicy}
if len(filters) == 1 {
cfg.FilterSubject = filters[0]
@@ -445,7 +445,7 @@ func (m *Mesh) ConsumerOn(t *testing.T, stream, node, module string, filters []s
// PendingOn is what a module's consumer on a stream still holds, as Pending is on EVENTS.
func (m *Mesh) PendingOn(t *testing.T, stream, node, module string) (ackPending, notDelivered uint64) {
t.Helper()
info, err := m.js.ConsumerInfo(stream, node+"_"+module)
info, err := m.js.ConsumerInfo(stream, node+"_"+module+"_tools")
if err != nil {
t.Fatal(err)
}
@@ -466,7 +466,7 @@ func (m *Mesh) Messages(t *testing.T, stream string) uint64 {
// it cannot narrow is withdrawn (novox/hq ADR 0363 §6).
func (m *Mesh) RemoveConsumer(t *testing.T, stream, node, module string) {
t.Helper()
if err := m.js.DeleteConsumer(stream, node+"_"+module); err != nil {
if err := m.js.DeleteConsumer(stream, node+"_"+module+"_tools"); err != nil {
t.Fatal(err)
}
}
+62 -1
View File
@@ -2,11 +2,15 @@ package runtime
import (
"encoding/json"
"os"
"slices"
"sort"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/micro"
@@ -121,6 +125,7 @@ func TestTheRuntimeAnnouncesWhatItServesInTheServicesProtocol(t *testing.T) {
// issue 1966 TLS-3 (ENG-2): the runner states the capabilities of the calls it answers in its PING, on its own instance, which
// is what its machine's node-engine asks and takes into the node's record; no bundle is needed for it.
func TestTheRunnerStatesTheCapabilitiesOfItsCallsInItsPing(t *testing.T) {
t.Setenv(FingerprintWord, "sha256:runner")
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
@@ -139,7 +144,63 @@ func TestTheRunnerStatesTheCapabilitiesOfItsCallsInItsPing(t *testing.T) {
t.Fatalf("ping of the instance: %v", err)
}
var ping micro.Ping
if err := json.Unmarshal(msg.Data, &ping); err != nil || ping.Metadata[announce.Capabilities] != "approval-mark/1,bus-state-calls,ledger-calls,put-sessions,seat-settle-delay" {
if err := json.Unmarshal(msg.Data, &ping); err != nil || ping.Metadata[announce.Capabilities] != "approval-mark/1,bus-state-calls,finding-calls,ledger-calls,ledger-upkeep-calls,put-sessions,seat-settle-delay,sent-objects-calls,stream-publish" {
t.Errorf("the runner states %q (%v), want its bus-state and ledger calls", ping.Metadata[announce.Capabilities], err)
}
// Its own fingerprint beside them (design 46 approval mark §6).
if ping.Metadata[announce.Fingerprint] != "sha256:runner" {
t.Errorf("the runner states its fingerprint as %q", ping.Metadata[announce.Fingerprint])
}
// Design 47 G §462, §488, §622: each call is stated under the capability word of its own family, the words mesh-sdk's buscalls
// names: the read of a sent object under sent-objects-calls (no longer under bus-state-calls), the ledger's fill and removal
// under ledger-upkeep-calls, the others where they were.
t.Run("EachCallIsStatedUnderTheWordOfItsFamily", func(t *testing.T) {
for method, word := range map[string]string{
buscalls.ObjectsSentGet: buscalls.SentObjectsCalls,
buscalls.LedgerFill: buscalls.LedgerUpkeepCalls,
buscalls.LedgerRemove: buscalls.LedgerUpkeepCalls,
buscalls.LedgerApply: buscalls.LedgerCalls,
buscalls.ObjectsPut: buscalls.BusStateCalls,
buscalls.StreamPublish: buscalls.StreamPublishWord,
} {
var in []string
for c, methods := range callFamilies {
if slices.Contains(methods, method) {
in = append(in, c)
}
}
if len(in) != 1 || in[0] != word {
t.Errorf("%s is stated under %v, want %s alone", method, in, word)
}
}
})
// Design 47 G §545: the words the tool runner states are exactly its manifest's artifact `states`, so the delivery plan's edge
// of §605 runs to it for each word a module's artifact `calls`.
t.Run("TheWordsTheRunnerStatesAreItsManifestsStates", func(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Build struct {
Artifacts []struct {
Name string `json:"name"`
States []string `json:"states"`
} `json:"artifacts"`
} `json:"build"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
var declared []string
for _, a := range m.Build.Artifacts {
if a.Name == "runtime" {
declared = a.States
}
}
slices.Sort(declared)
if got := CallCapabilities(); !slices.Equal(got, declared) {
t.Errorf("the runner states %v, its manifest's states are %v", got, declared)
}
})
}
+54 -13
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"sort"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
@@ -18,28 +19,39 @@ import (
// (announce.Capabilities) for its machine's node-engine to take into the node's record. A module whose manifest asks for one is
// held off a machine whose runner does not state it.
const (
CapBusStateCalls = "bus-state-calls"
CapLedgerCalls = "ledger-calls"
CapBusStateCalls = buscalls.BusStateCalls
CapLedgerCalls = buscalls.LedgerCalls
// CapLedgerUpkeepCalls is the ledger's fill and removal (§462, §488) and CapSentObjectsCalls the read of a sent object (§488):
// each is stated once the runner answers its family, the words mesh-sdk's buscalls names (§622).
CapLedgerUpkeepCalls = buscalls.LedgerUpkeepCalls
CapSentObjectsCalls = buscalls.SentObjectsCalls
// CapPutSessions is the put and get sessions' word (design 47 G §518, §622), mesh-sdk's.
CapPutSessions = stdio.CapPutSessions
// CapApprovalMark says the runner honours the approval mark: its gate sits on every handler it serves (design 46
// 7-approval-mark §6, §20), so the grants part composes no deny of its modules' marked subjects.
CapApprovalMark = "approval-mark/1"
// CapStreamPublish is the call that says an event on the module's own bound stream (design 42 zfs-owns-zfs, A-44), the
// word mesh-sdk's buscalls names (§622).
CapStreamPublish = buscalls.StreamPublishWord
)
// callFamilies are the methods each capability stands for, the ones BusState and BusStateWatch answer.
var callFamilies = map[string][]string{
CapBusStateCalls: {buscalls.ObjectsPut, buscalls.ObjectsGet, buscalls.ObjectsList, buscalls.ObjectsDelete, buscalls.ObjectsInfo,
buscalls.ObjectsWatch, buscalls.ObjectsSentGet, buscalls.StreamRead, buscalls.StreamFill, buscalls.WorkQueueSize, buscalls.WorkQueueDead},
buscalls.ObjectsWatch, buscalls.StreamRead, buscalls.StreamFill, buscalls.WorkQueueSize, buscalls.WorkQueueDead},
CapStreamPublish: {buscalls.StreamPublish},
CapLedgerCalls: {buscalls.LedgerApply, buscalls.LedgerGet, buscalls.LedgerHistory, buscalls.LedgerWatch, buscalls.LedgerQuery,
buscalls.LedgerRebuild, buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag},
CapPutSessions: {buscalls.ObjectsPutChunk, buscalls.ObjectsGetChunk},
CapLedgerUpkeepCalls: {buscalls.LedgerFill, buscalls.LedgerRemove},
CapSentObjectsCalls: {buscalls.ObjectsSentGet},
CapPutSessions: {buscalls.ObjectsPutChunk, buscalls.ObjectsGetChunk},
}
// CallCapabilities is every capability of the calls this runner answers, sorted. The delayed settle of a pulled seat
// message names a fate, not a method, so it stands beside the method families (novox/hq ADR 0664 §5).
// message names a fate, not a method, so it stands beside the method families (novox/hq ADR 0664 §5); so does `finding-calls`,
// the acknowledged `mesh/publish` and the `mesh/subscribe` for new messages the launch answers (design 47 G §559).
func CallCapabilities() []string {
out := []string{stdio.CapSeatSettleDelay, CapApprovalMark}
out := []string{stdio.CapSeatSettleDelay, stdio.CapFindingCalls, CapApprovalMark}
for c := range callFamilies {
out = append(out, c)
}
@@ -103,6 +115,11 @@ func (b *moduleBus) BusState(method string, params json.RawMessage) (json.RawMes
if err = decode(method, params, &r); err == nil {
answer, err = conn.StreamFill(b.module, r)
}
case buscalls.StreamPublish:
var r buscalls.StreamPublishRequest
if err = decode(method, params, &r); err == nil {
answer, err = conn.StreamPublish(b.module, r)
}
case buscalls.WorkQueueSize:
var r buscalls.WorkQueueRequest
if err = decode(method, params, &r); err == nil {
@@ -114,7 +131,7 @@ func (b *moduleBus) BusState(method string, params json.RawMessage) (json.RawMes
answer, err = conn.WorkQueueDead(b.module, r)
}
case buscalls.LedgerApply, buscalls.LedgerGet, buscalls.LedgerHistory, buscalls.LedgerQuery, buscalls.LedgerRebuild,
buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag:
buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag, buscalls.LedgerFill, buscalls.LedgerRemove:
answer, err = b.ledgerCall(method, params)
default:
return nil, fmt.Errorf("the runtime answers no %s", method)
@@ -161,15 +178,18 @@ func (c *consumers) objectsGet(module string, r buscalls.ObjectsNameRequest) (an
if st == nil {
return nil, fmt.Errorf("the runner has no bus to read %s of %s from", r.Name, module)
}
ctx, cancel := context.WithTimeout(context.Background(), bus.StateTimeout)
defer cancel()
ctx, cancel, opened := openingContext()
info, rd, found, err := st.ObjectsOpen(ctx, module, r)
opened()
if err != nil || !found {
cancel()
return nil, err
}
if info.Size > buscalls.ChunkBytes {
return c.openGet(module, info, rd, sessionDigest(info.Digest), false)
// the reader reads on ctx after this call returns: it ends with the session, not with this call
return c.openGet(module, info, cancelOnClose{rd, cancel}, sessionDigest(info.Digest), false)
}
defer cancel()
defer rd.Close()
data, err := io.ReadAll(rd)
if err != nil {
@@ -192,11 +212,32 @@ func (c *consumers) sentGet(module string, r buscalls.ObjectsSentGetRequest) (an
if st == nil {
return nil, fmt.Errorf("the runner has no bus to read %s from", r.Object)
}
ctx, cancel := context.WithTimeout(context.Background(), bus.StateTimeout)
defer cancel()
ctx, cancel, opened := openingContext()
info, rd, err := st.SentOpen(ctx, r)
opened()
if err != nil {
cancel()
return nil, err
}
return c.openGet(module, info, rd, sessionDigest(r.Digest), true)
return c.openGet(module, info, cancelOnClose{rd, cancel}, sessionDigest(r.Digest), true)
}
// openingContext is the context an object is opened on: it is bound at bus.StateTimeout while opening, and once opened (the
// returned function) is bound no more: a reader of an object that outlives the call reads on it until its session ends.
func openingContext() (context.Context, context.CancelFunc, func()) {
ctx, cancel := context.WithCancel(context.Background())
t := time.AfterFunc(bus.StateTimeout, cancel)
return ctx, cancel, func() { t.Stop() }
}
// cancelOnClose ends the context an object's reader reads on when the reader is closed: at the end of its get session.
type cancelOnClose struct {
io.ReadCloser
cancel context.CancelFunc
}
func (c cancelOnClose) Close() error {
err := c.ReadCloser.Close()
c.cancel()
return err
}
+75 -6
View File
@@ -6,6 +6,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"github.com/nats-io/nats.go"
@@ -101,9 +102,10 @@ func TestTheSubjectThatNamesNoCallerStillWorksAndNamesNone(t *testing.T) {
}
return err
})
// Every caller is admitted, an unproven one too (the operator, 2026-11-15 23:25 CET).
if strings.HasPrefix(subject, "mesh.seat.") {
if !strings.Contains(reply.Error, "node-desk.who takes no unproven caller") || reply.Result.Call != "" {
t.Errorf("%s: an unproven caller of a seat's verb was answered %+v (error %q), want refused before the bundle", subject, reply.Result, reply.Error)
if reply.Error != "" {
t.Errorf("%s: an unproven caller of a seat's verb was refused: %q, want answered", subject, reply.Error)
}
continue
}
@@ -115,15 +117,82 @@ func TestTheSubjectThatNamesNoCallerStillWorksAndNamesNone(t *testing.T) {
defs map[string]seatdef.Definition
says string
}{
{nil, "is not yet read"},
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk"}}, "names no verb who"},
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk", Verbs: []seatdef.Verb{{Name: "who", Callers: &seatdef.Callers{List: []string{}}}}}}, "open to no caller"},
{nil, ""},
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk"}}, ""},
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk", Verbs: []seatdef.Verb{{Name: "who", Callers: &seatdef.Callers{List: []string{}}}}}}, ""},
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk", Verbs: []seatdef.Verb{{Name: "who", Class: seatdef.ClassAct}}}}, ""},
} {
defs := c.defs
err := (&consumers{definitions: func() map[string]seatdef.Definition { return defs }}).admitted("node-desk", "who", "shanks.node-tools")
err := (&consumers{definitions: func() map[string]seatdef.Definition { return defs }}).admitted("node-desk", "who", "workstation.node-tools")
if c.says == "" && err != nil || c.says != "" && (err == nil || !strings.Contains(err.Error(), c.says)) {
t.Errorf("a proven caller with the definitions %v: %v, want %q", c.defs, err, c.says)
}
}
// Design 47 G A-249: a bundle's seat call is received by the runner in chunks only for a verb whose definition says `chunks`,
// of any machine: a verb a node holds is asked of the node that ran the ask (A-254).
t.Run("OnlyAChunkedSeatVerbIsAskedInChunks", func(t *testing.T) {
c := &consumers{definitions: func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"mesh-catalog": {Verbs: []seatdef.Verb{{Name: "input", Chunks: true}, {Name: "entry"}}}}
}}
for key, want := range map[string]bool{
"seat:mesh-catalog.input": true,
"seat:mesh-catalog.entry": false,
"seat:mesh-catalog.input@home-server": true,
"seat:mesh-catalog.entry@home-server": false,
"seat:mesh-unread.input": false,
"mesh-delivery-indexer.indexer": false,
} {
if got := c.chunked(key); got != want {
t.Errorf("chunked(%s) = %v, want %v", key, got, want)
}
}
if (&consumers{}).chunked("seat:mesh-catalog.input") {
t.Error("a runner that holds no seat definitions asks no chunks")
}
})
// **The tool runner answers a held seat's marked verb by caller and serves the seat's `approvals`** (design 47 G §677; lead
// answers A-265, A-271), with zsh's node-login-shell `execute` as the case: the marked verb reaches the gate with the caller
// its subject names, and `approvals` is answered from the runner's own record of the calls its gate holds, with the
// arguments for the call's own caller only, though no bundle names the verb.
t.Run("TheRunnerServesApprovalsForTheSeatsItsBundlesHold", func(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("zsh", "anchor", true, map[string][]string{"node-login-shell": {"execute", seatdef.ApprovalsVerb}}))
nodeTools := connect(t, "node-tools", "anchor")
gate := &gateHolding{}
defined := WithSeatDefinitions(func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell", Verbs: []seatdef.Verb{
{Name: "execute", NeedsApproval: true}, {Name: seatdef.ApprovalsVerb}}}}
})
stop, err := Run(nodeTools, []Served{{"zsh", []string{mt.Fixture("login-shell-execute.py")}}}, nil, func(string, ...any) {}, defined, WithGate(gate))
if err != nil {
t.Fatal(err)
}
defer stop()
asker, err := bus.Connect(bus.Credential{URL: mt.URL(t), Module: "node-tools", Node: "workstation", User: "workstation.node-tools", Password: "unused"})
if err != nil {
t.Fatal(err)
}
asker.Logf = func(string, ...any) {}
defer asker.Close()
mt.Until(t, func() error {
_, err := asker.Ask("seat:node-login-shell.execute@anchor", map[string]any{"command": "ls"}, "")
return err
})
if len(gate.calls) != 1 || gate.calls[0].Seat != "node-login-shell" || gate.calls[0].Name != "execute" || gate.calls[0].Caller != "workstation.node-tools" {
t.Fatalf("the marked verb reached the gate as %+v, want the seat's execute by workstation.node-tools", gate.calls)
}
var got approval.Approvals
mt.Until(t, func() error {
answered, err := asker.Ask("seat:node-login-shell.approvals@anchor", map[string]any{}, "")
if err == nil {
err = json.Unmarshal(answered.Result, &got)
}
return err
})
if len(got.Calls) != 1 || got.Calls[0].Caller != "workstation.node-tools" || got.Calls[0].Verb != "node-login-shell.execute" ||
string(got.Calls[0].Arguments) != `{"command":"ls"}` {
t.Fatalf("approvals answered %+v, want the one call the gate holds, with its caller's arguments", got)
}
})
}
@@ -1,29 +0,0 @@
package runtime
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Design 47 G A-249: a bundle's seat call is received by the runner in chunks only for a verb whose definition says `chunks`,
// of no one machine.
func TestOnlyAChunkedSeatVerbIsAskedInChunks(t *testing.T) {
c := &consumers{definitions: func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"mesh-catalog": {Verbs: []seatdef.Verb{{Name: "input", Chunks: true}, {Name: "entry"}}}}
}}
for key, want := range map[string]bool{
"seat:mesh-catalog.input": true,
"seat:mesh-catalog.entry": false,
"seat:mesh-catalog.input@ace": false,
"seat:mesh-unread.input": false,
"mesh-delivery-indexer.indexer": false,
} {
if got := c.chunked(key); got != want {
t.Errorf("chunked(%s) = %v, want %v", key, got, want)
}
}
if (&consumers{}).chunked("seat:mesh-catalog.input") {
t.Error("a runner that holds no seat definitions asks no chunks")
}
}
+17 -29
View File
@@ -1,7 +1,6 @@
package runtime
import (
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/approval"
@@ -19,6 +18,8 @@ type Gate interface {
Unserved(module, fingerprint string, registered []string) []string
// Run is a call through the gate: run is called only when the call may run now, once.
Run(call approval.Call, run func() (any, error)) (any, error)
// Approvals is the seat verb `approvals` for a caller: the calls the gate holds, its acts and claims (A-265, A-271).
Approvals(caller string) (approval.Approvals, error)
}
// gatedRun is how a handler calls a tool: through the gate where there is one.
@@ -33,6 +34,14 @@ func (c *consumers) gated(call approval.Call, t launch.Tool, caller string) (any
return c.gate.Run(call, run)
}
// approvalsOf is how the runner answers a held seat's verb `approvals`: from its gate's record, nil without a gate.
func (r *runner) approvalsOf() func(caller string) (any, error) {
if r.events.gate == nil {
return nil
}
return func(caller string) (any, error) { return r.events.gate.Approvals(caller) }
}
// WithGate serves every call through the approval gate.
func WithGate(g Gate) Option { return func(c *consumers) { c.gate = g } }
@@ -45,32 +54,10 @@ func WithSeatDefinitions(defs func() map[string]seatdef.Definition) Option {
return func(c *consumers) { c.definitions = defs }
}
// admitted says why a seat's verb is refused to a caller, nil when it is answered: an unproven caller is refused, and so is
// every caller while the seat's definition is not read, names no such verb, or the verb's effective callers are empty; a
// proven caller is among them, since the grants part composes a verb's call subject only for its effective callers (A-179).
func (c *consumers) admitted(seat, verb, caller string) error {
if c.definitions == nil {
return nil
}
if caller == "" {
return fmt.Errorf("Not run: %s.%s takes no unproven caller: call it on its subject that names its caller, "+
"mesh.seat.%s.call.%s.<caller>", seat, verb, seat, verb)
}
def, ok := c.definitions()[seat]
if !ok {
return fmt.Errorf("Not run: the definition of the seat %s is not yet read by this tool runner's seat list, so no caller "+
"of %s.%s is admitted", seat, seat, verb)
}
for _, v := range def.Verbs {
if v.Name == verb {
if len(v.EffectiveCallers()) == 0 {
return fmt.Errorf("Not run: %s.%s is open to no caller", seat, verb)
}
return nil
}
}
return fmt.Errorf("Not run: the seat %s's definition names no verb %s, so no caller is admitted", seat, verb)
}
// admitted says why a seat's verb is refused to a caller: never. Every caller is admitted, whether or not the seat's definition
// is read and whatever callers it names (the operator, 2026-11-15 23:25 CET: every module, agent and operator may call anything;
// approval stays as the mark on a verb, held by the gate).
func (c *consumers) admitted(seat, verb, caller string) error { return nil }
// unservedMarks is, of the module's marked tools, those its bundle's registrations under its own name do not register.
func (r *runner) unservedMarks(module string, regs []launch.Registration) []string {
@@ -94,8 +81,9 @@ func (c *consumers) chunked(key string) bool {
if c.definitions == nil || !strings.HasPrefix(key, "seat:") {
return false
}
seat, verb, ok := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
if !ok || strings.Contains(verb, "@") {
name, _, _ := strings.Cut(key, "@") // a verb asked of the machine that ran the ask is chunked as the verb is (A-254)
seat, verb, ok := strings.Cut(strings.TrimPrefix(name, "seat:"), ".")
if !ok {
return false
}
for _, v := range c.definitions()[seat].Verbs {
@@ -0,0 +1,26 @@
package runtime
import (
"git.novox.be/novox/mesh-sdk/go/approval"
)
// gateHolding is the runner's gate as far as the runtime reaches it: it records the calls it is run for and answers
// `approvals` from them.
type gateHolding struct{ calls []approval.Call }
func (g *gateHolding) Unserved(string, string, []string) []string { return nil }
func (g *gateHolding) Run(c approval.Call, run func() (any, error)) (any, error) {
g.calls = append(g.calls, c)
return run()
}
func (g *gateHolding) Approvals(caller string) (approval.Approvals, error) {
out := approval.Approvals{Calls: []approval.ListedCall{}, Acts: []approval.ListedAct{}, Claims: []string{}}
for i, c := range g.calls {
l := approval.ListedCall{Call: string(rune('a' + i)), Caller: c.Caller, Verb: c.Seat + "." + c.Name, State: "waiting"}
if caller == c.Caller {
l.Arguments = c.Args
}
out.Calls = append(out.Calls, l)
}
return out, nil
}
@@ -0,0 +1,43 @@
package runtime
import (
"testing"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// Design 47 G lead answers A-377: the call a module's tool reaches the gate by carries the module's composed
// environment, so the gate tells a first-view tool (the one whose MESH_VIEW_CALLERS_<TOOL> the bundle holds) from an
// ordinary one while its catalog half is unread.
func TestAModuleToolsCallCarriesItsBundlesComposedEnvironment(t *testing.T) {
for name, tc := range map[string]struct {
envs map[string]map[string]string
want bool
}{
"a first-view tool": {map[string]map[string]string{"ledger": {"MESH_VIEW_CALLERS_WHO": "grants"}}, true},
"an ordinary tool": {nil, false},
} {
t.Run(name, func(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("ledger", "anchor", true, map[string][]string{"node-desk": {"who"}}))
nodeTools := connect(t, "node-tools", "anchor")
gate := &gateHolding{}
stop, err := Run(nodeTools, []Served{{"ledger", []string{mt.Fixture("caller-echo.py")}}}, tc.envs, func(string, ...any) {}, deskDefined, WithGate(gate))
if err != nil {
t.Fatal(err)
}
defer stop()
asker := connect(t, "node-tools", "anchor")
mt.Until(t, func() error { _, err := asker.Ask("ledger.who@anchor", map[string]any{}, ""); return err })
var found bool
for _, c := range gate.calls {
if c.Seat == "" && c.Name == "who" {
_, found = c.Env["MESH_VIEW_CALLERS_WHO"]
}
}
if found != tc.want {
t.Errorf("the gate saw the first-view word: %v, want %v (calls %+v)", found, tc.want, gate.calls)
}
})
}
}
@@ -8,6 +8,7 @@ import (
"errors"
"fmt"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
@@ -240,6 +241,35 @@ func TestTheLedgerKeepsItsPromisesOnTheBus(t *testing.T) {
t.Fatalf("queried %+v, %v", q, err)
}
}},
{"fill and removal are answered through the calls and remove an instance from the bus whole", "decision.gone", func(t *testing.T) {
must(t, open(t), "decision/gone", "title", `{"title":"Gone"}`)
must(t, open(t), "decision/gone", "note", `{"says":"two"}`)
must(t, open(t), "decision/stays", "note", `{"says":"kept"}`)
raw, err := m.BusState(buscalls.LedgerFill, json.RawMessage(`{"ledger":"records"}`))
var fill buscalls.Fill
if _ = json.Unmarshal(raw, &fill); err != nil || fill.Count < 3 || fill.Bytes == 0 {
t.Fatalf("the fill answered %s (%v)", raw, err)
}
raw, err = m.BusState(buscalls.LedgerRemove, json.RawMessage(`{"ledger":"records","instance":"decision/gone"}`))
var removed buscalls.LedgerRemoved
if _ = json.Unmarshal(raw, &removed); err != nil || removed.Operations != 2 || removed.Bytes == 0 {
t.Fatalf("the removal answered %s (%v)", raw, err)
}
l := open(t)
if last, _ := l.History.Last("decision.gone"); last != 0 {
t.Fatalf("operations are left, at %d", last)
}
if st, _, _ := l.State.Get("decision.gone"); st != nil {
t.Fatalf("the state is left: %s", st)
}
if last, _ := l.History.Last("decision.stays"); last == 0 {
t.Fatal("another instance lost its operations")
}
if raw, err = m.BusState(buscalls.LedgerRemove, json.RawMessage(`{"ledger":"records","instance":"decision/gone"}`)); err != nil ||
!strings.Contains(string(raw), `"operations":0`) {
t.Fatalf("a removal said twice answered %s (%v)", raw, err)
}
}},
{"a forged author is refused", "decision.forged", func(t *testing.T) {
_, err := m.BusState(buscalls.LedgerApply, json.RawMessage(`{"ledger":"records","instance":"decision/forged","operation":"title","body":{"title":"T","author":"op@shanks"},"call":"forged"}`))
if err == nil || !strings.Contains(err.Error(), "the call forged, which is not a tool call in flight") {
@@ -258,4 +288,70 @@ func TestTheLedgerKeepsItsPromisesOnTheBus(t *testing.T) {
}
})
}
// Design 47 G §559, A-153: against the real bus a publish with ack answers the sequence the bus gave it, and a subscription
// for new messages hears only what is published after it was made, on the subject mesh-sdk's one rule names for the event;
// a wildcard is refused.
t.Run("AnAckedPublishAnswersItsSequenceAndANewSubscriptionHearsOnlyWhatComesAfter", func(t *testing.T) {
mt.New(t)
nc, err := nats.Connect(mt.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(nc.Close)
js, _ := jetstream.New(nc)
ctx := context.Background()
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "SEATEVENTS", Subjects: []string{"mesh.seat.mesh-conditions.event.>"}}); err != nil {
t.Fatal(err)
}
conn := connect(t, "node-tools", "anchor")
publish := func(body string) {
t.Helper()
if _, err := js.Publish(ctx, "mesh.seat.mesh-conditions.event.findings-asked", []byte(body)); err != nil {
t.Fatal(err)
}
}
publish(`{"before":true}`)
var mu sync.Mutex
var heard []string
stop, err := conn.SubscribeNew("mesh-conditions.findings-asked", func(raw json.RawMessage) error {
var env bus.Envelope
_ = json.Unmarshal(raw, &env)
mu.Lock()
heard = append(heard, env.Key+" "+string(env.Body))
mu.Unlock()
return nil
})
if err != nil {
t.Fatal(err)
}
defer stop()
publish(`{"after":true}`)
publish(`{"after":2}`)
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(heard) < 2 {
return context.DeadlineExceeded
}
return nil
})
mu.Lock()
if len(heard) != 2 || heard[0] != `mesh-conditions.findings-asked {"after":true}` {
t.Errorf("heard %v, want the messages after the subscription alone, in order", heard)
}
mu.Unlock()
for name, want := range map[string]string{"mesh-conditions.*": "[A-Za-z0-9_-]"} {
if _, err := conn.SubscribeNew(name, func(json.RawMessage) error { return nil }); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("a subscription for %s answered %v, want one naming %q", name, err, want)
}
}
first, err := conn.PublishAsAcked("node-tools", bus.Envelope{Key: "finding-written", Body: json.RawMessage(`{}`)})
second, err2 := conn.PublishAsAcked("node-tools", bus.Envelope{Key: "finding-written", Body: json.RawMessage(`{}`)})
if err != nil || err2 != nil || first == 0 || second != first+1 {
t.Errorf("acknowledged publishes answered %d (%v) and %d (%v), want consecutive sequences", first, err, second, err2)
}
})
}
+11
View File
@@ -171,6 +171,17 @@ func (b *moduleBus) ledgerCall(method string, params json.RawMessage) (any, erro
var r buscalls.LedgerRequest
_ = json.Unmarshal(params, &r)
return l.Lag(r)
case buscalls.LedgerFill:
var r buscalls.LedgerRequest
_ = json.Unmarshal(params, &r)
return l.Fill(r)
case buscalls.LedgerRemove:
// Never on an apply: a removal is no operation of the history and has no author (design 47 G §587).
var r buscalls.LedgerRemoveRequest
if err := decode(method, params, &r); err != nil {
return nil, err
}
return l.Remove(r)
}
return nil, fmt.Errorf("the runtime answers no %s", method)
}
+1 -6
View File
@@ -4,7 +4,6 @@ import (
"context"
"encoding/json"
"errors"
"slices"
"sort"
"strings"
"testing"
@@ -205,13 +204,9 @@ func TestABundleToolNamedAsALedgerRepairIsRefused(t *testing.T) {
// no other name; none answers "the runtime answers no ...".
func TestEveryCallOfAStatedCapabilityIsAnswered(t *testing.T) {
m, _, _ := aLedger(t)
if got := strings.Join(CallCapabilities(), ","); got != "approval-mark/1,bus-state-calls,ledger-calls,put-sessions,seat-settle-delay" {
if got := strings.Join(CallCapabilities(), ","); got != "approval-mark/1,bus-state-calls,finding-calls,ledger-calls,ledger-upkeep-calls,put-sessions,seat-settle-delay,sent-objects-calls,stream-publish" {
t.Fatalf("the runner states %s", got)
}
// Design 47 G §462: the call that reads a sent object is listed under the bus state capability.
if !slices.Contains(callFamilies[CapBusStateCalls], buscalls.ObjectsSentGet) {
t.Errorf("%s does not list %s", CapBusStateCalls, buscalls.ObjectsSentGet)
}
for capability, methods := range callFamilies {
for _, method := range methods {
t.Run(capability+" "+method, func(t *testing.T) {
@@ -36,6 +36,25 @@ func TestNoBusWordReachesABundle(t *testing.T) {
t.Errorf("%s did not reach the bundle", want)
}
}
// design 47 G §618, §519: the runtime's own fingerprint reaches no bundle; the fingerprint a module's composed words hold does.
t.Run("TheRuntimesOwnFingerprintReachesNoBundle", func(t *testing.T) {
base := []string{"MESH_FINGERPRINT=runner-build", "PATH=/usr/bin"}
for name, c := range map[string]struct {
given map[string]string
want string
}{
"composed with none sees none": {nil, ""},
"composed with one sees its own": {map[string]string{"MESH_FINGERPRINT": "module-build"}, "MESH_FINGERPRINT=module-build"},
} {
env := strings.Join(BundleEnv(base, c.given, "telegram", "anchor"), "\n")
if strings.Contains(env, "runner-build") {
t.Errorf("%s: the runtime's own fingerprint reached the bundle: %s", name, env)
}
if c.want == "" && strings.Contains(env, "MESH_FINGERPRINT") || !strings.Contains(env, c.want) {
t.Errorf("%s: the environment is %s", name, env)
}
}
})
}
// A tool call reaches only a tool's subject: whatever key a caller names, it is never a seat's event, accept
+103 -17
View File
@@ -62,6 +62,10 @@ type ListedTool struct {
// NodeRuntimeModule is the module that is a machine's runtime, serving every module on it.
const NodeRuntimeModule = "node-tools"
// fingerprintEnv is the word the node-engine starts each process with, the fingerprint of the want it applies (mesh-sdk's
// process.FingerprintEnv; this package imports no SDK): the runtime's own is no bundle's.
const fingerprintEnv = "MESH_FINGERPRINT"
// runtimeOnly are the words of the runtime's own environment no bundle is given.
var runtimeOnly = map[string]bool{ToolEnv: true, ToolModules: true, "MESH_BROKER_FILE": true,
"MESH_BROKER_URL": true, "MESH_CONSOLE_LISTEN": true, LedgersFile: true}
@@ -237,6 +241,9 @@ type runner struct {
// §424, §618), never the runner's own.
fingerprintOf func(module string) string
// envOf is a module's composed environment, the words the gate tells a first-view tool by (lead answers A-377).
envOf func(module string) map[string]string
seatsAgain func()
stops []func() // the watch, the announcement and the seats, in the order they began
}
@@ -271,6 +278,7 @@ func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, log
base := os.Environ()
r.envFor = func(module string) []string { return BundleEnv(base, envs[module], module, node) }
r.fingerprintOf = func(module string) string { return envs[module][FingerprintWord] }
r.envOf = func(module string) map[string]string { return envs[module] }
// The module's events, for every child of it that subscribes (ADR 0198): one consumer per module,
// bound the first time any of its children subscribes, each event handed to every child that did.
@@ -306,7 +314,8 @@ func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, log
}
logf("%s", line)
conn.OnMembership(func(bus.Membership) { go func() { r.refreshRepairs(); conn.Flush() }() })
stopSeats, again := serveSeats(conn, r.modules, r.registrations, r.events.gated, r.events.admitted, r.fingerprintOf, logf)
stopSeats, again := serveSeats(conn, r.modules, r.registrations, r.events.gated, r.events.admitted, r.events.chunked, r.approvalsOf(),
r.fingerprintOf, logf)
r.seatsAgain = again
r.stops = append(r.stops, stopSeats)
@@ -316,7 +325,10 @@ func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, log
announced, err := announce.Serve(conn, announce.Service{
Name: conn.Module(), ID: instanceOf(conn),
Description: "the mesh's tool runtime on " + node + ": every assigned module's tools and the seats they hold",
Metadata: map[string]string{"node": node, announce.Capabilities: strings.Join(CallCapabilities(), ",")},
// Its own fingerprint beside the capabilities, so a reading is tied to the runner that made it (design 46 approval
// mark §6, lead answer A-203).
Metadata: map[string]string{"node": node, announce.Capabilities: strings.Join(CallCapabilities(), ","),
announce.Fingerprint: os.Getenv(FingerprintWord)},
}, func() []announce.Endpoint { return endpointsOf(conn, r.own(), r.registrations(), r.modules) })
if err != nil {
logf("[mesh-tools] cannot announce what it serves: %v", err)
@@ -387,6 +399,7 @@ func (r *runner) claimed() map[string]bool {
func (r *runner) load(s Served) *loadedModule {
l := &loadedModule{served: s, seen: map[string]string{}}
claimed := r.claimed()
var all []launch.Registration // every bundle's registrations, the module's marks checked against them together (A-398)
for _, entry := range s.Entrypoints {
path, _ := filepath.Abs(entry)
module := s.Module
@@ -417,13 +430,7 @@ func (r *runner) load(s Served) *loadedModule {
fail(clashed)
continue
}
// A bundle whose marks name a tool it does not register is not loaded: no tool of its module is served here, rather
// than one served unmarked that the manifest meant to mark (design 46 7-approval-mark §4).
if unserved := r.unservedMarks(module, child.Registrations); len(unserved) > 0 {
child.Stop()
fail(fmt.Sprintf("its marks name %s, which it does not register (design 46 7-approval-mark §4)", strings.Join(unserved, ", ")))
continue
}
all = append(all, child.Registrations...)
l.children = append(l.children, child.Stop)
for _, reg := range child.Registrations {
got := registration{module: reg.Module, owner: module, tools: reg.Tools}
@@ -437,6 +444,18 @@ func (r *runner) load(s Served) *loadedModule {
}
}
}
// A module whose marks name a tool none of its bundles registers is not loaded: no tool of it is served here, rather
// than one served unmarked that the manifest meant to mark (design 46 7-approval-mark §4). Checked once, against every
// bundle of the module together (lead answer A-398: gitea's eight bundles each overwrote the gate's set).
if unserved := r.unservedMarks(s.Module, all); len(unserved) > 0 {
for _, stop := range l.children {
stop()
}
l.children, l.regs, l.own = nil, nil, nil
l.failed = fmt.Sprintf("its marks name %s, which none of its bundles registers (design 46 7-approval-mark §4)", strings.Join(unserved, ", "))
r.logf("[mesh-tools] %s is not loaded: %s; its tools are not served here", s.Module, l.failed)
return l
}
if repairs := r.repairsOf(s.Module); repairs != nil {
l.regs, l.own = append(l.regs, *repairs), append(l.own, *repairs)
}
@@ -471,7 +490,8 @@ func (r *runner) serve(l *loadedModule) {
module, own := reg.module, reg.repairs
stop, err := r.conn.HandleCalled(reg.module+"."+t.Name, func(caller string, body json.RawMessage) (any, error) {
return r.events.gated(approval.Call{Address: module + "." + t.Name, Module: module, Name: t.Name, Own: own,
Fingerprint: r.fingerprintOf(module), Caller: caller, Args: argsOf(body), Description: t.Description}, t, caller)
Fingerprint: r.fingerprintOf(module), Env: r.envOf(module), Caller: caller, Args: argsOf(body),
Description: t.Description}, t, caller)
})
if err != nil {
r.logf("[mesh-tools] cannot serve %s.%s: %v", reg.module, t.Name, err)
@@ -655,6 +675,9 @@ func endpointsOf(conn *bus.Conn, own []registration, registrations []registratio
}
for _, v := range m.Seats {
t, ok := impl[v.Seat][v.Verb]
if v.Verb == seatdef.ApprovalsVerb && !ok {
t, ok = launch.Tool{Name: v.Verb, Description: "The calls this seat's holder holds for the operator's approval, its acts and its claims."}, true
}
if !ok || have[v.Subject] {
continue
}
@@ -710,7 +733,8 @@ func subjectsOf(conn *bus.Conn, module, tool string) []string {
// whenever a membership changes, and when again is called because a bundle was loaded again. Whether
// this machine holds the seat is the bus's to decide. It answers its stop and again.
func serveSeats(conn *bus.Conn, modules []string, current func() []registration, gated gatedRun,
admitted func(seat, verb, caller string) error, fingerprintOf func(string) string, logf func(string, ...any)) (func(), func()) {
admitted func(seat, verb, caller string) error, chunked func(key string) bool, approvals func(caller string) (any, error),
fingerprintOf func(string) string, logf func(string, ...any)) (func(), func()) {
var mu sync.Mutex
var stops []func()
stopped := false
@@ -744,6 +768,24 @@ func serveSeats(conn *bus.Conn, modules []string, current func() []registration,
continue
}
have[v.Subject] = true
// The seat's verb `approvals` is the gate's own, served wherever a marked verb is: the runner answers it from its
// record of the calls its gate holds, by caller, and no bundle names it (lead answers A-265, A-266, A-271).
if v.Verb == seatdef.ApprovalsVerb && approvals != nil {
seat := v.Seat
stop, err := conn.HandleSubjectCalled(v.Subject, func(caller string, _ json.RawMessage) (any, error) {
if err := admitted(seat, seatdef.ApprovalsVerb, caller); err != nil {
return nil, err
}
return approvals(caller)
})
if err != nil {
logf("[mesh-tools] cannot serve %s's approvals on %s: %v", seat, v.Subject, err)
continue
}
stops = append(stops, stop)
logf("[mesh-tools] serving %s's approvals on %s, the gate's own verb", seat, v.Subject)
continue
}
t, ok := impl[v.Seat][v.Verb]
if !ok {
logf("[mesh-tools] %s claims %s and implements no %s, which that seat promises; not served", module, v.Seat, v.Verb)
@@ -752,19 +794,37 @@ func serveSeats(conn *bus.Conn, modules []string, current func() []registration,
// Its callers checked, then through the approval gate, before the bundle (design 47 G §610, A-179; design 46
// 7-approval-mark §20).
module, v := module, v
stop, err := conn.HandleSubjectCalled(v.Subject, func(caller string, body json.RawMessage) (any, error) {
handle := func(caller string, body json.RawMessage) (any, error) {
if err := admitted(v.Seat, v.Verb, caller); err != nil {
return nil, err
}
return gated(approval.Call{Address: v.Seat + "." + v.Verb, Module: module, Seat: v.Seat, Name: v.Verb,
answer, err := gated(approval.Call{Address: v.Seat + "." + v.Verb, Module: module, Seat: v.Seat, Name: v.Verb,
Fingerprint: fingerprintOf(module), Caller: caller, Args: argsOf(body), Description: t.Description}, t, caller)
})
if err != nil || !chunked("seat:"+v.Seat+"."+v.Verb) {
return answer, err
}
// A verb marked chunks: the bundle answers the whole file, the runner sends it in chunks (design 47 G A-257).
return conn.AnswerChunked(v.Seat, v.Verb, body, answer)
}
stop, err := conn.HandleSubjectCalled(v.Subject, handle)
if err != nil {
logf("[mesh-tools] cannot serve %s's %s on %s: %v", v.Seat, v.Verb, v.Subject, err)
continue
}
stops = append(stops, stop)
logf("[mesh-tools] serving %s's %s on %s, admitted where %s holds the seat", v.Seat, v.Verb, v.Subject, module)
// A verb of a node-scoped seat is also answered on the subject that names no machine, by any one holder, in
// one queue group per seat and verb (lead answer A-393).
if open, ok := anyHolderSubject(v, m.Node); ok && !have[open] {
have[open] = true
openStop, err := conn.HandleSubjectCalledQueue(open, "seat."+v.Seat+"."+v.Verb, handle)
if err != nil {
logf("[mesh-tools] cannot serve %s's %s on %s: %v", v.Seat, v.Verb, open, err)
continue
}
stops = append(stops, openStop)
logf("[mesh-tools] serving %s's %s on %s, in a queue group, for a call that names no machine", v.Seat, v.Verb, open)
}
}
}
}
@@ -781,6 +841,15 @@ func serveSeats(conn *bus.Conn, modules []string, current func() []registration,
}, serve
}
// anyHolderSubject is a node-scoped seat verb's subject without its machine, the one asked with no machine named; false
// where the verb's subject does not end in the machine (a seat held once).
func anyHolderSubject(v bus.SeatVerb, node string) (string, bool) {
if node == "" || !strings.HasSuffix(v.Subject, "."+v.Verb+"."+node) {
return "", false
}
return strings.TrimSuffix(v.Subject, "."+node), true
}
// consumers holds, per module the runtime serves, the one durable consumer its events arrive on and
// the children its events are handed to (novox/hq ADR 0198).
type consumers struct {
@@ -902,6 +971,21 @@ func (b *moduleBus) Publish(params json.RawMessage) error {
return b.all.conn.PublishAs(b.module, env)
}
// PublishAcked is Publish that answers the sequence the bus acknowledged the event at (`mesh/publish` with `ack`, design 47 G §559).
func (b *moduleBus) PublishAcked(params json.RawMessage) (uint64, error) {
var env bus.Envelope
if err := json.Unmarshal(params, &env); err != nil {
return 0, fmt.Errorf("not an event envelope: %w", err)
}
return b.all.conn.PublishAsAcked(b.module, env)
}
// SubscribeNew hands the bundle one event's messages published from now on, on an ordered consumer of its own (`mesh/subscribe`
// with `deliver: new`, design 47 G §559); the consumer ends with the bundle's channel.
func (b *moduleBus) SubscribeNew(pattern string, deliver func(json.RawMessage) error) (func(), error) {
return b.all.conn.SubscribeNew(pattern, deliver)
}
// Ask calls a tool as the module: `{key, body}`, answered with the tool's result (ADR 0198).
func (b *moduleBus) Ask(params json.RawMessage) (json.RawMessage, error) {
var asked struct {
@@ -979,7 +1063,7 @@ func (b *moduleBus) Subscribe(deliver func(json.RawMessage) error, beat func(jso
return err
}
m.stop = stop
c.logf("[mesh-tools] %s's events arrive on its consumer %s", module, bus.ConsumerOf(c.conn.Node(), module))
c.logf("[mesh-tools] %s's events arrive on its consumer %s", module, bus.ToolsConsumerOf(c.conn.Node(), module))
return nil
}
@@ -1403,11 +1487,13 @@ func (b *moduleBus) bind(key string, stop func()) {
// BundleEnv is what one module's bundle is started with: the runtime's environment without its own words,
// the module's composed words over it, and the module's and machine's names. **No bus word reaches a
// bundle** (security review of 2026-10-08): the credential and the bus's address are the runtime's, and a
// bundle reaches the bus only through the runtime.
// bundle reaches the bus only through the runtime. **The runtime's own fingerprint reaches none either**: a bundle reads
// only the `MESH_FINGERPRINT` its module's composed words hold, the build the node-engine applied for that module, and with
// none it has none (design 47 G §519, §618).
func BundleEnv(base []string, given map[string]string, module, node string) []string {
words := map[string]string{}
for _, kv := range base {
if k, v, ok := strings.Cut(kv, "="); ok && !runtimeOnly[k] {
if k, v, ok := strings.Cut(kv, "="); ok && !runtimeOnly[k] && k != fingerprintEnv {
words[k] = v
}
}
+88 -2
View File
@@ -4,10 +4,12 @@ import (
"encoding/json"
"errors"
"os"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/approval"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/launch"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
@@ -192,10 +194,50 @@ func TestEachBundleIsGivenItsOwnEnvironment(t *testing.T) {
}
}
// An entrypoint that is not executable is refused by name, and the others serve (ADR 0193).
// unservingGate is an approval gate whose marks of each module name the tools given, so a bundle not registering them is
// refused at the load (design 46 7-approval-mark §4).
type unservingGate map[string][]string
func (g unservingGate) Unserved(module, _ string, registered []string) []string {
var missing []string
for _, tool := range g[module] {
if !slices.Contains(registered, tool) {
missing = append(missing, tool)
}
}
return missing
}
func (unservingGate) Run(_ approval.Call, run func() (any, error)) (any, error) { return run() }
func (unservingGate) Approvals(string) (approval.Approvals, error) { return approval.Approvals{}, nil }
// An entrypoint that is not executable is refused by name, and the others serve (ADR 0193). A bundle whose marks name a
// tool it does not register is refused at the load, serving no tool of its module, and one that registers it serves
// (design 46 7-approval-mark §4).
func TestAnEntrypointThatIsNotExecutableIsRefused(t *testing.T) {
mt.NeedBundles(t)
mesh := mt.New(t)
{
mesh.Issue(t, mt.MembershipOf("marked", "anchor", true, nil))
mesh.Issue(t, mt.MembershipOf("served", "anchor", true, nil))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
logs := &mt.Logs{}
stop, err := Run(nodeTools, []Served{{"marked", []string{mt.Fixture("caller-echo.py")}}, {"served", []string{mt.Fixture("caller-echo.py")}}},
nil, logs.Logf, WithGate(unservingGate{"marked": {"wipe"}, "served": {"who"}}))
if err != nil {
t.Fatal(err)
}
if !logs.Has("marked is not loaded: its marks name wipe, which none of its bundles registers (design 46 7-approval-mark §4)") {
t.Errorf("a bundle not serving its marked tool was not refused at the load:\n%s", logs.All())
}
if _, err := call(t, asker, "served.who@anchor", map[string]any{}); err != nil {
t.Errorf("a bundle serving its marked tool: %v", err)
}
if _, err := asker.Ask("marked.who@anchor", map[string]any{}, ""); !errors.Is(err, bus.ErrNoResponders) {
t.Errorf("a refused bundle's tool answered: %v", err)
}
stop()
}
mt.NeedBundles(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("plain", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
@@ -270,3 +312,47 @@ func TestAnUntakenEventSaysWhoFailed(t *testing.T) {
t.Errorf("the event is not named by its id: %q", got)
}
}
// A node-scoped seat's verb asked with no machine is answered by a holder, by the tool form the SDK asks
// (lead answer A-393).
func TestANodeScopedSeatVerbIsAlsoAnsweredWithNoMachine(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("theta", "anchor", false, map[string][]string{"node-shelf": {"list"}}))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
regs := func() []registration {
return []registration{{module: "node-shelf", owner: "theta", tools: []launch.Tool{{Name: "list",
Run: func(string, json.RawMessage) (json.RawMessage, error) { return json.RawMessage(`{"shelf":["x"]}`), nil }}}}}
}
gated := func(_ approval.Call, tool launch.Tool, caller string) (any, error) {
return tool.Run(caller, nil)
}
nodeTools.Follow("theta")
stop, _ := serveSeats(nodeTools, []string{"theta"}, regs, gated, func(_, _, _ string) error { return nil },
func(string) bool { return false }, nil, func(string) string { return "" }, (&mt.Logs{}).Logf)
defer stop()
for _, key := range []string{"seat:node-shelf.list@anchor", "seat:node-shelf.list"} {
t.Run(key, func(t *testing.T) {
mt.Until(t, func() error {
got, err := call(t, asker, key, map[string]any{})
if err == nil {
same(t, got, `{"shelf":["x"]}`)
}
return err
})
})
}
}
func TestAnyHolderSubject(t *testing.T) {
for _, c := range []struct{ subject, node, want string }{
{"mesh.seat.node-build-go.tool.steps.shanks", "shanks", "mesh.seat.node-build-go.tool.steps"},
{"mesh.seat.mesh-delivery.tool.show", "shanks", ""},
{"mesh.seat.node-build-go.tool.steps.shanks", "", ""},
} {
got, ok := anyHolderSubject(bus.SeatVerb{Seat: "s", Verb: "steps", Subject: c.subject}, c.node)
if got != c.want || ok != (c.want != "") {
t.Errorf("%s on %q: %q %v", c.subject, c.node, got, ok)
}
}
}
+76 -64
View File
@@ -37,12 +37,26 @@ func (f *fakeObjects) ObjectsPutStream(ctx context.Context, _ string, r buscalls
return buscalls.ObjectInfo{Name: r.Name, Size: uint64(len(data)), Description: description}, nil
}
func (f *fakeObjects) ObjectsOpen(context.Context, string, buscalls.ObjectsNameRequest) (buscalls.ObjectInfo, io.ReadCloser, bool, error) {
return buscalls.ObjectInfo{Name: "o", Size: uint64(len(f.read))}, io.NopCloser(bytes.NewReader(f.read)), true, nil
// ctxReader reads as the bus's object reader does: on the context it was opened on, failing once that is cancelled.
type ctxReader struct {
ctx context.Context
r io.Reader
}
func (f *fakeObjects) SentOpen(context.Context, buscalls.ObjectsSentGetRequest) (buscalls.ObjectInfo, io.ReadCloser, error) {
return buscalls.ObjectInfo{Size: uint64(len(f.read))}, io.NopCloser(bytes.NewReader(f.read)), nil
func (c ctxReader) Read(p []byte) (int, error) {
if err := c.ctx.Err(); err != nil {
return 0, err
}
return c.r.Read(p)
}
func (c ctxReader) Close() error { return nil }
func (f *fakeObjects) ObjectsOpen(ctx context.Context, _ string, _ buscalls.ObjectsNameRequest) (buscalls.ObjectInfo, io.ReadCloser, bool, error) {
return buscalls.ObjectInfo{Name: "o", Size: uint64(len(f.read))}, ctxReader{ctx, bytes.NewReader(f.read)}, true, nil
}
func (f *fakeObjects) SentOpen(ctx context.Context, _ buscalls.ObjectsSentGetRequest) (buscalls.ObjectInfo, io.ReadCloser, error) {
return buscalls.ObjectInfo{Size: uint64(len(f.read))}, ctxReader{ctx, bytes.NewReader(f.read)}, nil
}
func aSession(t *testing.T, f *fakeObjects) *moduleBus {
@@ -64,66 +78,6 @@ func asked(t *testing.T, b *moduleBus, method string, req any, into any) {
}
}
// Design 47 G §518, A-21, A-25: a put session writes each chunk as it is taken, carries the description of its first chunk,
// and answers the put's own info for the last; a chunk out of order fails the put naming both sequences and leaves none.
func TestAPutSessionSealsOnTheLastChunkAndDropsOnADisorder(t *testing.T) {
f := &fakeObjects{put: map[string][]byte{}}
b := aSession(t, f)
whole := bytes.Repeat([]byte("x"), 2*buscalls.ChunkBytes+5)
var taken buscalls.ObjectsChunkTaken
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(whole))
req := buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c1", Seq: seq, Last: end == len(whole), Data: whole[at:end]}
if seq == 1 {
req.Description = "what it is"
}
if req.Last {
var info buscalls.ObjectInfo
asked(t, b, buscalls.ObjectsPutChunk, req, &info)
if info.Size != uint64(len(whole)) || info.Description != "what it is" {
t.Fatalf("the put answered %+v", info)
}
break
}
asked(t, b, buscalls.ObjectsPutChunk, req, &taken)
if taken.Seq != seq || taken.Error != "" {
t.Fatalf("chunk %d was answered %+v", seq, taken)
}
at = end
}
if !bytes.Equal(f.put["a"], whole) {
t.Fatalf("the store holds %d bytes, want %d", len(f.put["a"]), len(whole))
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 1, Data: []byte("one")}, &taken)
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 3, Data: []byte("three")}, &taken)
if !strings.Contains(taken.Error, "chunk 3") || !strings.Contains(taken.Error, "chunk 2") {
t.Fatalf("a disorder was answered %+v", taken)
}
if _, ok := f.put["b"]; ok || len(f.deleted) != 1 || f.deleted[0] != "b" {
t.Fatalf("a dropped put left %v, deleted %v", f.put, f.deleted)
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 2, Data: []byte("two")}, &taken)
if taken.Error == "" {
t.Fatal("a chunk of a dropped put was taken")
}
}
// §518 "The bound": a put not sealed within the bound is dropped, its object deleted.
func TestAPutNotSealedWithinTheBoundIsDropped(t *testing.T) {
was := SessionBound
SessionBound = time.Millisecond
t.Cleanup(func() { SessionBound = was })
f := &fakeObjects{put: map[string][]byte{}, ctxEnded: make(chan struct{})}
b := aSession(t, f)
var taken buscalls.ObjectsChunkTaken
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c", Seq: 1, Data: []byte("x")}, &taken)
<-f.ctxEnded
if len(f.deleted) != 1 {
t.Fatalf("the unsealed object was left: %v", f.deleted)
}
}
// §490, §518, §620: a value over one chunk is got as a session (the chunked answer, then each chunk by its call and sequence,
// the whole checked against the digest); a sent object over one chunk the same, with `chunked` true.
func TestAGetOverOneChunkRunsAsASession(t *testing.T) {
@@ -192,4 +146,62 @@ func TestAGetOverOneChunkRunsAsASession(t *testing.T) {
if string(small.Data) != "small" {
t.Fatalf("a small value was answered %+v", small)
}
// Design 47 G §518, A-21, A-25: a put session writes each chunk as it is taken, carries the description of its first chunk,
// and answers the put's own info for the last; a chunk out of order fails the put naming both sequences and leaves none.
t.Run("APutSessionSealsOnTheLastChunkAndDropsOnADisorder", func(t *testing.T) {
f := &fakeObjects{put: map[string][]byte{}}
b := aSession(t, f)
whole := bytes.Repeat([]byte("x"), 2*buscalls.ChunkBytes+5)
var taken buscalls.ObjectsChunkTaken
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(whole))
req := buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c1", Seq: seq, Last: end == len(whole), Data: whole[at:end]}
if seq == 1 {
req.Description = "what it is"
}
if req.Last {
var info buscalls.ObjectInfo
asked(t, b, buscalls.ObjectsPutChunk, req, &info)
if info.Size != uint64(len(whole)) || info.Description != "what it is" {
t.Fatalf("the put answered %+v", info)
}
break
}
asked(t, b, buscalls.ObjectsPutChunk, req, &taken)
if taken.Seq != seq || taken.Error != "" {
t.Fatalf("chunk %d was answered %+v", seq, taken)
}
at = end
}
if !bytes.Equal(f.put["a"], whole) {
t.Fatalf("the store holds %d bytes, want %d", len(f.put["a"]), len(whole))
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 1, Data: []byte("one")}, &taken)
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 3, Data: []byte("three")}, &taken)
if !strings.Contains(taken.Error, "chunk 3") || !strings.Contains(taken.Error, "chunk 2") {
t.Fatalf("a disorder was answered %+v", taken)
}
if _, ok := f.put["b"]; ok || len(f.deleted) != 1 || f.deleted[0] != "b" {
t.Fatalf("a dropped put left %v, deleted %v", f.put, f.deleted)
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 2, Data: []byte("two")}, &taken)
if taken.Error == "" {
t.Fatal("a chunk of a dropped put was taken")
}
})
// §518 "The bound": a put not sealed within the bound is dropped, its object deleted.
t.Run("APutNotSealedWithinTheBoundIsDropped", func(t *testing.T) {
was := SessionBound
SessionBound = time.Millisecond
t.Cleanup(func() { SessionBound = was })
f := &fakeObjects{put: map[string][]byte{}, ctxEnded: make(chan struct{})}
b := aSession(t, f)
var taken buscalls.ObjectsChunkTaken
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c", Seq: 1, Data: []byte("x")}, &taken)
<-f.ctxEnded
if len(f.deleted) != 1 {
t.Fatalf("the unsealed object was left: %v", f.deleted)
}
})
}
@@ -0,0 +1,18 @@
package runtime
import (
"slices"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// A-44: mesh/stream.publish is a family of its own, the capability stream-publish, and no longer of bus-state-calls.
func TestStreamPublishIsACapabilityOfItsOwn(t *testing.T) {
if !slices.Equal(callFamilies[CapStreamPublish], []string{buscalls.StreamPublish}) || CapStreamPublish != "stream-publish" {
t.Fatal("stream-publish does not state only mesh/stream.publish")
}
if slices.Contains(callFamilies[CapBusStateCalls], buscalls.StreamPublish) {
t.Fatal("bus-state-calls still states mesh/stream.publish")
}
}
+14 -3
View File
@@ -58,7 +58,18 @@
"language": "go",
"system": "arch",
"from": "cmd/node-tools",
"binary": "node-tools"
"binary": "node-tools",
"states": [
"approval-mark/1",
"bus-state-calls",
"finding-calls",
"ledger-calls",
"ledger-upkeep-calls",
"put-sessions",
"seat-settle-delay",
"sent-objects-calls",
"stream-publish"
]
}
]
},
@@ -77,8 +88,8 @@
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed",
"mesh-conditions.findings-asked",
"mesh-delivery-assigner.views.assignments",
"mesh-delivery-assigner.views.seat-holders",
"seat:mesh-module-assigner.views.assignments",
"seat:mesh-module-assigner.views.seat-holders",
"mesh-delivery.source-moved"
],
"streams": [
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env python3
# The login shell's bundle as the seat node-login-shell has it: MCP over stdio, no SDK, one seat verb, execute, which says
# the command it was given and the caller the call's `_meta` named.
import json, sys
def say(m):
m["jsonrpc"] = "2.0"; sys.stdout.write(json.dumps(m) + "\n"); sys.stdout.flush()
TOOLS = [
{"name": "node-login-shell.execute", "description": "run a command", "inputSchema": {"type": "object", "properties": {"command": {"type": "string"}}}},
]
for line in sys.stdin:
req = json.loads(line); rid = req.get("id"); m = req.get("method"); p = req.get("params") or {}
if m == "initialize":
say({"id": rid, "result": {"protocolVersion": "2025-03-26", "capabilities": {"tools": {}}, "serverInfo": {"name": "zsh", "version": "1"}}})
elif m == "tools/list":
say({"id": rid, "result": {"tools": TOOLS}})
elif m == "tools/call":
meta = p.get("_meta") or {}
said = {"ran": (p.get("arguments") or {}).get("command", ""), "caller": meta.get("mesh/caller", "")}
say({"id": rid, "result": {"content": [{"type": "text", "text": json.dumps(said)}]}})
elif rid is not None:
say({"id": rid, "result": {}})