Author SHA1 Message Date
jschoubben 6be5679a0d Merge main into feat/47-G-join11: the tool runner's mesh/stream.publish (issue 1979)
mesh/delivery merged: source.main moved to novox/mesh-tools@a96df478d498
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 6be5679a0d58 on a96df478d498
The capability stream-publish is stated under mesh-sdk's word (buscalls.StreamPublishWord) and in the runtime artifact's states, so G's test that the stated words are the manifest's states holds; the publish uses G's exported JetStream.

Builds: design 47 G §545, §622; design 42 zfs-owns-zfs A-42, A-44

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools af83854956 Merge pull request 'The tool runner publishes a bundle's event onto its own bound events stream (issue 1979)' (#70) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:53 +00:00
jschoubben df2162e381 tool runner: mesh/stream.publish is the capability stream-publish, taken out of bus-state-calls (A-44)
mesh/delivery merged: source.main moved to novox/mesh-tools@af8385495688
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to df2162e381f5 on af8385495688 | from 447a8d035935, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-host 0193949…
Builds: design 42 zfs-owns-zfs §25, lead answer A-44
2026-11-13 01:52:15 +01:00
jschoubben a775b42e4d tool runner: mesh/stream.publish says an event on the calling bundle's own bound stream (A-42)
Builds: design 42 zfs-owns-zfs §25, §24, lead answer A-42
2026-11-13 01:49:02 +01:00
jschoubben 481a83e11c Round 2 T2, T3, T4, T5: the tool runner ends G with its merge-base test count; the wait test runs its rows side by side
mesh/delivery-group group feat/47-G-join11 checking: 15 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 481a83e11c4f on af8385495688 | novox/mesh-tools#69 in group feat/47-G-join11
The tests G added fold into existing ones as subtests (design 47 G §626, A-306: 171 tests down to 154, integration 9
down to 7). TestMeshWaitRefusesLoudlyAndNeverLoops runs its groups of one-second rows as parallel subtests, each
on a runtime of its own, 15 s down to 5 s (A-311: the one-second floor of a wait's timeout stands, no clock is
injected). The go.mod no longer requires meshconditionsclient, which no package loads (T4). Node names in the
lines G adds are role names (T5).

Builds: design 47 G §626, design 34 every-event §19, A-306, A-311; review round 2 findings T2, T3, T4, T5
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 22:24:02 +01:00
jschoubben 28bece26b0 Join 11 (try 2): the chunked-answer tests flush the holder's verb before another connection asks it
The tests of A-249 and A-257 served the verb on one connection and asked it from another at once; the
subscription was still in the holder's buffer when the ask reached the bus, so the ask was answered 503
no responders (13 of 300 runs with -cpu 1 before, 0 of 2000 after). The runner flushes after serving
(runtime.go:352, :807); the tests now do as it does.

Builds: design 47 G A-249, A-257
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 15:14:01 +01:00
jschoubben 816df217cd Join 11: every go.mod requires, at the zero version, each mesh module its packages load
The checker's fetch pins only the zero-version requirements a directory's go.mod names
(mesh-build-go workspace.go goModOf), so a module reached only through another, such as
meshhandacts through meshhandactsclient or mesh-delivery-sdk through meshcatalogclient,
was never downloaded and the shut-off load failed (fetch.go:267). The join's merges had
dropped these indirect requirements that the trunk's go.mod files list.

Builds: design 47 G (join 11, the check's fetch of the group's modules)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 14:07:29 +01:00
jschoubben c35c27f664 Area 1 (part): the tool runner's manifest states approval-mark/1
The runner states the approval mark's word beside its call families, so its
artifact's states are exactly what it says (the join of A3's and A5's lines);
TestTheWordsTheRunnerStatesAreItsManifestsStates failed without it.

Builds: design 47 G §545, design 46 7-approval-mark §6; join 11 area 1

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:44:02 +01:00
jschoubben d350c98823 Join 11: the merged heads compile where no area's fix is owed
The join's importers name the delivery repository's moved contracts and sdk
(MV1), node-tools' tests use mesh-sdk's gate, the duplicate node-engine claim
rule of the reader release gives way to G's, and the assigner names its first
watched branch, never a constant. The seat holders' AnswerSeat callers are
area 6's.

Builds: design 47 G join 11 (merge of fix round 11)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:41:44 +01:00
jschoubben 29757dabc3 Merge branch 'feat/47-G-fix11-a3h' into feat/47-G-join11
# Conflicts:
#	node-tools/internal/runtime/runtime.go
2026-11-12 10:27:48 +01:00
jschoubben 95a265e730 Merge branch 'feat/47-G-fix11-c1c' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben d6c267d2f0 Merge branch 'feat/47-G-fix11-b5g3' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben e03d8d3c7b Merge branch 'feat/47-G-fix11-b4b' into feat/47-G-join11 2026-11-12 10:27:27 +01:00
jschoubben a91386b3b5 Merge branch 'feat/47-G-fix11-a5' into feat/47-G-join11
# Conflicts:
#	node-tools/internal/firstview/firstview_test.go
#	node-tools/internal/runtime/announce_test.go
#	node-tools/internal/runtime/busstate.go
#	node-tools/internal/runtime/ledgers_test.go
2026-11-12 10:27:27 +01:00
jschoubben 8b980a9480 Merge branch 'feat/47-G-fix11-a3' into feat/47-G-join11 2026-11-12 10:27:01 +01:00
jschoubben 9fcb14e741 A3h item 1: the tool runner serves a held seat's approvals from its gate's record
The seat verb approvals is the gate's own, served wherever a marked verb is: the runner answers it by caller from the
calls its gate holds, with no bundle naming it; a marked verb already reaches the gate by caller. Tested with zsh's
node-login-shell execute as the case.

Builds: design 47 G §677, §697; lead answers A-264, A-265, A-266, A-271

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:21:21 +01:00
jschoubben 14f464e8bc C1c-4: the tool runner sends a bundle's chunked seat answer in chunks, and a node's chunked verb is asked of the node
A seat verb whose definition says chunks, served by a bundle that answers the whole file as the member `file`, is sent by the
runner with buscalls.SeatChunks.Send and answered with the chunk count (Conn.AnswerChunked, called from serveSeats). A chunked verb
asked of one machine (`@<node>`) is received in chunks as the verb is.

Builds: design 47 G A-257, A-249, A-254, §681

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:43:08 +01:00
jschoubben 2d95c0eb59 AM-D3 path: the tool runner states its own fingerprint beside its capabilities
A capability reading must be tied to the runner that made it, so the grants
part can tell an older runner's reading from the running one's.

Builds: design 46 7 §6 (the announcement carries the runner's fingerprint), lead answer A-203 (key `fingerprint`)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:02:18 +01:00
jschoubben b5c7bf13ea A-9 tests: a bundle not serving its marked tool is refused at the load; a withheld marked call is said so
The runtime's load test gains rows (run with the Python fixture, before the bundles' skip): a bundle whose
marks name a tool it does not register serves no tool of its module, and one that registers it serves.
mesh_call's failure words move into Surface.callFailed, whose rows hold that only a marked tool reaching no
responder names withheld and mesh-grants.grants_user_list.

Builds: design 46 7-approval-mark §4, §6, §22

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:54:49 +01:00
jschoubben 99f8c8c050 Finding 6: every module requires the mesh modules it builds with
The SDK, the assigner, the builder's buildrun, the indexer, the
pipeline, the watchman, mesh-enrolment and mesh-grants (and through the
SDK the checker's checkermsg and mesh-tools' node-tools) imported
packages of modules their go.mod did not require at the zero version;
the placement's go.work over the whole repository hid it, and a build
of the module alone with -mod=readonly refused it. Each now requires
them (beside a relative replace where the module keeps its siblings so),
and the indexer's and watchman's duplicated lines go.

Builds: design 47 G §427 (the group's check builds every importer); delivery review round 1, finding 6

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:21:14 +01:00
jschoubben 24f257e929 MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben e63d707eac R-11: mesh/publish with ack answers its sequence or {error}, mesh/subscribe with deliver new makes an ordered consumer; finding-calls is stated
The subscription's key resolves through mesh-sdk's buscompose.EventSubjects (lead answer A-153; the sdk commit EE-S1 is
cherry-picked into this change's sdk placement), a wildcard is refused.

Builds: design 47 G §559, §488, §576, §622

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:04:18 +01:00
jschoubben 5e373af70d R-1 rest, R-12, R-13: the capability words are mesh-sdk's, the ledger's fill and removal are answered, the artifact states the words
sent-get is under sent-objects-calls; mesh/ledger.fill answers the history's fill against its cap and mesh/ledger.remove
removes an instance's operations, snapshot, state and index entries, both under ledger-upkeep-calls; the runtime artifact's
`states` is exactly the words the runner states.

Builds: design 47 G §462, §488, §545, §587, §622, §155

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:02:04 +01:00
jschoubben 64015b5952 R-8: the tool runner's own fingerprint reaches no bundle; a bundle reads the one its module is composed with
Builds: design 47 G §519, §618

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:57:42 +01:00
jschoubben e873cb3b86 R-7: mesh_first_views names the tool runner's own build, or says own build unknown
Builds: design 47 G §205, §424

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:57:12 +01:00
jschoubben e88e0c825c R-5: the tool runner follows the assigner's views of the module that claims the seat, named by the seat
The holder is read from the catalog half's entries serving a verb of mesh-module-assigner, on the branch follower's change
callback; the manifest's consumes names the two views by the seat (seat:mesh-module-assigner.views.*).

Builds: design 47 G §616, §428, §7

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:33 +01:00
54 changed files with 1827 additions and 548 deletions
@@ -12,7 +12,7 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
seat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"github.com/nats-io/nats.go"
+6 -3
View File
@@ -5,14 +5,13 @@ go 1.26.0
require (
git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshhandacts v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
github.com/nats-io/nats.go v1.54.0
golang.org/x/sys v0.48.0
)
@@ -30,3 +29,7 @@ require (
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
require git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect
require git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk v0.0.0-00010101000000-000000000000 // indirect
+5
View File
@@ -48,6 +48,11 @@ const Digest = "digest"
// module needing one is placed only where the runtime states it.
const Capabilities = "capabilities"
// Fingerprint is the service metadata key the tool runner states its own fingerprint under, beside Capabilities: the build
// its composed environment names (MESH_FINGERPRINT, design 47 G §424), so the node-engine ties the capabilities it takes
// into the node's record to the runner that stated them (design 46 approval mark §6, lead answer A-203).
const Fingerprint = "fingerprint"
// Shortened is the service metadata key a runtime sets when its announcement was cut to fit the bus.
const Shortened = "shortened"
+84 -1
View File
@@ -3,6 +3,7 @@ package approval
import (
"encoding/json"
"errors"
"reflect"
"strings"
"sync"
"testing"
@@ -10,7 +11,7 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
seat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/keys"
@@ -506,4 +507,86 @@ func TestTheRefusalsAndAnswers(t *testing.T) {
} {
t.Run(tc.name, func(t *testing.T) { tc.run(t, newWorld()) })
}
// Design 46 7-approval-mark §5, §3, §4 and design 34 every-event §33: the marks a module's calls are gated by, derived from
// fixture catalog halves and served entries.
t.Run("TheMarks", func(t *testing.T) {
login := map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "sessions"}}}}
v1 := entry("src2:1", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"wipe"}})
v2 := entry("src2:2", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"erase", "node-login-shell.sessions"}})
mail := catalogseat.CatalogEntry{Module: "mail", Fp: "src2:m", Manifest: map[string]any{
"needs-approval": []any{"mail.received"}, "emits": []any{"mail.received"},
"streams": []any{map[string]any{"name": "MAIL", "events": []any{"mail.**"}, "grant": map[string]any{"config": "readers", "events": "mail.{key}.**"}}}}}
for _, tc := range []struct {
name string
cat *fakeCatalog
module string
fp string
tools []string
verbs map[string]string
subjects []string
unread bool
}{
{name: "a seat's mark stands for every holder, and the holder's own adds up", cat: &fakeCatalog{revision: "7",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell"}},
{name: "a holder's needs-approval cannot unmark its seat's verb, and marks another", cat: &fakeCatalog{revision: "8",
byFp: map[string]catalogseat.CatalogEntry{"src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:2", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "across a version change the union stands: a dropped mark is asked while the old bundle runs", cat: &fakeCatalog{revision: "9",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1, "src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase", "wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "a served entry that cannot be read leaves the newest marks, every marked call refused", cat: &fakeCatalog{revision: "9",
newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}, unread: true},
{name: "a bundle with no fingerprint has no served entry", cat: &fakeCatalog{revision: "9", defs: login},
module: "shell", unread: true},
{name: "a marked event and a stream with a grant give their subjects and each .>", cat: &fakeCatalog{revision: "3",
byFp: map[string]catalogseat.CatalogEntry{"src2:m": mail}, newest: map[string]catalogseat.CatalogEntry{"mail": mail}},
module: "mail", fp: "src2:m", subjects: []string{"mesh.mod.mail.event.mail.received", "mesh.mod.mail.event.mail.received.>",
"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.received", "mesh.stream.mail.event.mail.received.>"}},
} {
t.Run(tc.name, func(t *testing.T) {
m := gate.Derive(tc.cat, tc.module, tc.fp)
if got := keysOf(m.Tools); !reflect.DeepEqual(got, tc.tools) {
t.Errorf("tools %v, want %v", got, tc.tools)
}
if tc.verbs == nil {
tc.verbs = map[string]string{}
}
if !reflect.DeepEqual(m.Verbs, tc.verbs) {
t.Errorf("verbs %v, want %v", m.Verbs, tc.verbs)
}
if got := keysOf(m.Subjects); !reflect.DeepEqual(got, tc.subjects) {
t.Errorf("subjects %v, want %v", got, tc.subjects)
}
if (m.Unread != "") != tc.unread || m.Revision != tc.cat.revision {
t.Errorf("unread %q, revision %q", m.Unread, m.Revision)
}
})
}
// §4: a bundle whose marks name a tool it does not register is not loaded, its finding mark-unserved raised; a bundle that
// serves it clears it, and a catalog without the mark too.
t.Run("mark-unserved raised and cleared", func(t *testing.T) {
cat := &fakeCatalog{revision: "7", byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login}
w := &findings{}
g := gate.New(gate.Deps{Node: "n1", Runner: "node-tools", Catalog: cat, Findings: Findings(w, "n1", "node-tools", nil)})
if got := g.Unserved("shell", "src2:1", []string{"other"}); !reflect.DeepEqual(got, []string{"wipe"}) || w.raised[keys.EscapeName("n1.shell")] != gate.KindMarkUnserved {
t.Fatalf("unserved %v, findings %v", got, w.raised)
}
if got := g.Unserved("shell", "src2:1", []string{"wipe"}); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a bundle serving the mark: unserved %v, findings %v", got, w.raised)
}
_ = g.Unserved("shell", "src2:1", nil)
cat.byFp["src2:1"], cat.newest["shell"] = entry("src2:1", nil, nil), entry("src2:1", nil, nil)
if got := g.Unserved("shell", "src2:1", nil); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a catalog without the mark: unserved %v, findings %v", got, w.raised)
}
})
})
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
assignerseat "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/views"
)
+1 -87
View File
@@ -2,15 +2,12 @@ package approval
import (
"context"
"reflect"
"sort"
"testing"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/views"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
gate "git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
@@ -47,89 +44,6 @@ func entry(fp string, serves []string, manifest map[string]any) catalogseat.Cata
return catalogseat.CatalogEntry{Module: "shell", Fp: fp, Serves: serves, Manifest: manifest}
}
// Design 46 7-approval-mark §5, §3, §4 and design 34 every-event §33: the marks a module's calls are gated by, derived from
// fixture catalog halves and served entries.
func TestTheMarks(t *testing.T) {
login := map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "sessions"}}}}
v1 := entry("src2:1", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"wipe"}})
v2 := entry("src2:2", []string{"node-login-shell.execute", "node-login-shell.sessions"},
map[string]any{"needs-approval": []any{"erase", "node-login-shell.sessions"}})
mail := catalogseat.CatalogEntry{Module: "mail", Fp: "src2:m", Manifest: map[string]any{
"needs-approval": []any{"mail.received"}, "emits": []any{"mail.received"},
"streams": []any{map[string]any{"name": "MAIL", "events": []any{"mail.**"}, "grant": map[string]any{"config": "readers", "events": "mail.{key}.**"}}}}}
for _, tc := range []struct {
name string
cat *fakeCatalog
module string
fp string
tools []string
verbs map[string]string
subjects []string
unread bool
}{
{name: "a seat's mark stands for every holder, and the holder's own adds up", cat: &fakeCatalog{revision: "7",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell"}},
{name: "a holder's needs-approval cannot unmark its seat's verb, and marks another", cat: &fakeCatalog{revision: "8",
byFp: map[string]catalogseat.CatalogEntry{"src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:2", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "across a version change the union stands: a dropped mark is asked while the old bundle runs", cat: &fakeCatalog{revision: "9",
byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1, "src2:2": v2}, newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase", "wipe"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}},
{name: "a served entry that cannot be read leaves the newest marks, every marked call refused", cat: &fakeCatalog{revision: "9",
newest: map[string]catalogseat.CatalogEntry{"shell": v2}, defs: login},
module: "shell", fp: "src2:1", tools: []string{"erase"}, verbs: map[string]string{"node-login-shell.execute": "seat node-login-shell",
"node-login-shell.sessions": "entry shell src2:2"}, unread: true},
{name: "a bundle with no fingerprint has no served entry", cat: &fakeCatalog{revision: "9", defs: login},
module: "shell", unread: true},
{name: "a marked event and a stream with a grant give their subjects and each .>", cat: &fakeCatalog{revision: "3",
byFp: map[string]catalogseat.CatalogEntry{"src2:m": mail}, newest: map[string]catalogseat.CatalogEntry{"mail": mail}},
module: "mail", fp: "src2:m", subjects: []string{"mesh.mod.mail.event.mail.received", "mesh.mod.mail.event.mail.received.>",
"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.received", "mesh.stream.mail.event.mail.received.>"}},
} {
t.Run(tc.name, func(t *testing.T) {
m := gate.Derive(tc.cat, tc.module, tc.fp)
if got := keysOf(m.Tools); !reflect.DeepEqual(got, tc.tools) {
t.Errorf("tools %v, want %v", got, tc.tools)
}
if tc.verbs == nil {
tc.verbs = map[string]string{}
}
if !reflect.DeepEqual(m.Verbs, tc.verbs) {
t.Errorf("verbs %v, want %v", m.Verbs, tc.verbs)
}
if got := keysOf(m.Subjects); !reflect.DeepEqual(got, tc.subjects) {
t.Errorf("subjects %v, want %v", got, tc.subjects)
}
if (m.Unread != "") != tc.unread || m.Revision != tc.cat.revision {
t.Errorf("unread %q, revision %q", m.Unread, m.Revision)
}
})
}
// §4: a bundle whose marks name a tool it does not register is not loaded, its finding mark-unserved raised; a bundle that
// serves it clears it, and a catalog without the mark too.
t.Run("mark-unserved raised and cleared", func(t *testing.T) {
cat := &fakeCatalog{revision: "7", byFp: map[string]catalogseat.CatalogEntry{"src2:1": v1}, newest: map[string]catalogseat.CatalogEntry{"shell": v1}, defs: login}
w := &findings{}
g := gate.New(gate.Deps{Node: "n1", Runner: "node-tools", Catalog: cat, Findings: Findings(w, "n1", "node-tools", nil)})
if got := g.Unserved("shell", "src2:1", []string{"other"}); !reflect.DeepEqual(got, []string{"wipe"}) || w.raised[keys.EscapeName("n1.shell")] != gate.KindMarkUnserved {
t.Fatalf("unserved %v, findings %v", got, w.raised)
}
if got := g.Unserved("shell", "src2:1", []string{"wipe"}); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a bundle serving the mark: unserved %v, findings %v", got, w.raised)
}
_ = g.Unserved("shell", "src2:1", nil)
cat.byFp["src2:1"], cat.newest["shell"] = entry("src2:1", nil, nil), entry("src2:1", nil, nil)
if got := g.Unserved("shell", "src2:1", nil); got != nil || w.raised[keys.EscapeName("n1.shell")] != "" {
t.Fatalf("a catalog without the mark: unserved %v, findings %v", got, w.raised)
}
})
}
// findings is the runner's held writer as a fixture: each finding raised now, by id.
type findings struct {
raised map[string]string
+35
View File
@@ -276,6 +276,41 @@ func (c *Conn) StreamFill(module string, r buscalls.StreamRequest) (buscalls.Str
return c.streamFill(b.Stream)
}
// StreamPublish says one event on the module's own bound stream, by an acknowledged publish on the binding's subject plus
// "." plus the event, the id as the message id (design 42 zfs-owns-zfs, A-42). The module's own binding alone: another
// module's, and a binding that is not a stream, are refused by Binding.
func (c *Conn) StreamPublish(module string, r buscalls.StreamPublishRequest) (buscalls.StreamPublished, error) {
b, err := c.streamBinding(module, r.Stream)
if err != nil {
return buscalls.StreamPublished{}, err
}
if b.Subject == "" {
return buscalls.StreamPublished{}, fmt.Errorf("%s's stream %q has no subject to say an event on", module, r.Stream)
}
if r.Event == "" || strings.ContainsAny(r.Event, " *>\t\r\n") || strings.HasPrefix(r.Event, ".") || strings.HasSuffix(r.Event, ".") {
return buscalls.StreamPublished{}, fmt.Errorf("the event %q is not a name to say on %s's stream %q", r.Event, module, r.Stream)
}
body := []byte(r.Body)
if len(body) == 0 {
body = []byte("null")
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
js, err := c.JetStream()
if err != nil {
return buscalls.StreamPublished{}, err
}
var opts []jetstream.PublishOpt
if r.ID != "" {
opts = append(opts, jetstream.WithMsgID(r.ID))
}
ack, err := js.Publish(ctx, b.Subject+"."+r.Event, body, opts...)
if err != nil {
return buscalls.StreamPublished{}, fmt.Errorf("the event %s was not acknowledged on %s's stream %q: %w", r.Event, module, r.Stream, err)
}
return buscalls.StreamPublished{Sequence: ack.Sequence}, nil
}
// StreamReadMost is the most messages one read answers.
const StreamReadMost = 1000
+26
View File
@@ -2,6 +2,7 @@ package bus_test
import (
"context"
"encoding/json"
"errors"
"io"
"strings"
@@ -99,6 +100,23 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
t.Fatalf("fill %+v, %v", fill, err)
}
// A-42: an event said on the module's own stream lands on the binding's subject plus the event, acknowledged, and a
// second say with the same id is the same message.
said, err := conn.StreamPublish("m", buscalls.StreamPublishRequest{Stream: "events", Event: "scrub.requested",
Body: json.RawMessage(`{"pool":"tank"}`), ID: "id-1"})
if err != nil || said.Sequence != 3 {
t.Fatalf("said %+v, %v", said, err)
}
if again, err := conn.StreamPublish("m", buscalls.StreamPublishRequest{Stream: "events", Event: "scrub.requested",
Body: json.RawMessage(`{"pool":"tank"}`), ID: "id-1"}); err != nil || again.Sequence != 3 {
t.Fatalf("said again %+v, %v", again, err)
}
if page, err := conn.StreamRead("m", buscalls.StreamReadRequest{Stream: "events", From: 3,
Subjects: "mesh.state.m.events.scrub.requested"}); err != nil || len(page.Messages) != 1 ||
string(page.Messages[0].Body) != `{"pool":"tank"}` {
t.Fatalf("read back %+v, %v", page, err)
}
for _, c := range []struct {
name, module, refusal string
call func(module string) error
@@ -115,6 +133,14 @@ func TestABundleReachesItsOwnBoundObjectsAndStreamsOnly(t *testing.T) {
_, err := conn.ObjectsList(module, buscalls.ObjectsRequest{Objects: "events"})
return err
}},
{"say on another module's stream", "other", `other was bound no stream called "events"`, func(module string) error {
_, err := conn.StreamPublish(module, buscalls.StreamPublishRequest{Stream: "events", Event: "x"})
return err
}},
{"say on a binding that is not a stream", "m", `"files" is bound with the storage type object-store, not stream`, func(module string) error {
_, err := conn.StreamPublish(module, buscalls.StreamPublishRequest{Stream: "files", Event: "x"})
return err
}},
{"a work queue not bound", "m", `bound no stream called "jobs"`, func(module string) error {
_, err := conn.WorkQueueSize(module, buscalls.WorkQueueRequest{Queue: "jobs"})
return err
+11 -2
View File
@@ -849,6 +849,12 @@ func answerOf(data []byte) (Answered, error) {
// PublishAs emits an event as a module: published into JetStream and awaited, de-duplicated by its
// own id (ADR 0042). The body is the payload; the metadata rides as headers.
func (c *Conn) PublishAs(module string, env Envelope) error {
_, err := c.PublishAsAcked(module, env)
return err
}
// PublishAsAcked is PublishAs that also answers the sequence the bus acknowledged the event at.
func (c *Conn) PublishAsAcked(module string, env Envelope) (uint64, error) {
// On the module's own stream when its membership names one carrying the event (ADR 0363 §2), on EVENTS
// otherwise — and only there, as before, for a membership that names no stream.
msg := nats.NewMsg(eventSubject(c.Membership(module), module, env.Key))
@@ -870,8 +876,11 @@ func (c *Conn) PublishAs(module string, env Envelope) error {
if id := env.Headers["x-event-id"]; id != "" {
opts = append(opts, nats.MsgId(id))
}
_, err := c.js.PublishMsg(msg, opts...)
return err
ack, err := c.js.PublishMsg(msg, opts...)
if err != nil {
return 0, err
}
return ack.Sequence, nil
}
// Say publishes one message on core NATS, kept by nobody: a heartbeat, whose loss is the next one.
+51 -3
View File
@@ -25,9 +25,12 @@ const FileField = "file"
// counts with buscalls.SeatChunks, each acknowledged before the next, and answers the verb's own answer with the whole
// file added as FileField. The asking bundle receives no chunk itself.
func (c *Conn) AskAsChunked(module, key string, body any) (Answered, error) {
seat, verb, ok := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
if !strings.HasPrefix(key, "seat:") || !ok || strings.Contains(verb, "@") {
return Answered{}, fmt.Errorf("%q is no seat:<seat>.<verb> of no one machine, which a chunked answer needs", key)
// The verb may be asked of the machine that ran the ask (`@<node>`, A-254): the chunk subjects name the seat, the verb and
// the call alone.
name, _, _ := strings.Cut(key, "@")
seat, verb, ok := strings.Cut(strings.TrimPrefix(name, "seat:"), ".")
if !strings.HasPrefix(key, "seat:") || !ok {
return Answered{}, fmt.Errorf("%q is no seat:<seat>.<verb>, which a chunked answer needs", key)
}
args := map[string]any{}
if raw, err := json.Marshal(body); err != nil {
@@ -93,3 +96,48 @@ func (c *Conn) AskAsChunked(module, key string, body any) (Answered, error) {
answered.Result = out
return answered, nil
}
// AnswerChunked is a seat verb's answer as its definition says `chunks` (design 47 G A-257, §681): a bundle answers the whole file
// as the member FileField (base64), and the runner that serves the verb sends it in chunks with buscalls.SeatChunks on the call
// token the body names, each acknowledged by the caller before the next, and answers the verb's own members with `chunks`, the
// count, in place of the file. The bundle sends no chunk itself. An answer with no file is answered as it is. The send runs after
// the answer is returned: the caller hears the chunks before it asks (AskAsChunked) and acknowledges each once it has the answer.
func (c *Conn) AnswerChunked(seat, verb string, body json.RawMessage, answer any) (any, error) {
raw, err := json.Marshal(answer)
if err != nil {
return nil, err
}
fields := map[string]json.RawMessage{}
if err := json.Unmarshal(raw, &fields); err != nil {
return answer, nil
}
enc, has := fields[FileField]
if !has {
return answer, nil
}
var text string
if err := json.Unmarshal(enc, &text); err != nil {
return nil, fmt.Errorf("%s.%s answers a %s that is no string: %w", seat, verb, FileField, err)
}
file, err := base64.StdEncoding.DecodeString(text)
if err != nil {
return nil, fmt.Errorf("%s.%s answers a %s in no base64: %w", seat, verb, FileField, err)
}
var asked struct {
Call string `json:"call"`
}
if err := json.Unmarshal(body, &asked); err != nil || asked.Call == "" {
return nil, fmt.Errorf("%s.%s is asked in chunks with no call token", seat, verb)
}
delete(fields, FileField)
count := int64((len(file) + buscalls.SeatChunkBytes - 1) / buscalls.SeatChunkBytes)
fields["chunks"], _ = json.Marshal(count)
send := buscalls.SeatChunks{Seat: seat, Verb: verb, Call: asked.Call}
go func() { _ = send.Send(c.nc, file, buscalls.SeatAckWithin, c.Logf) }()
return json.RawMessage(mustMarshal(fields)), nil
}
func mustMarshal(v any) []byte {
b, _ := json.Marshal(v)
return b
}
+42
View File
@@ -57,6 +57,7 @@ func TestAChunkedSeatAnswerIsReceivedByTheRunnerAndHandedWhole(t *testing.T) {
t.Fatal(err)
}
defer stop()
holder.Flush() // the bus holds the verb's subscription before it is asked: the asker is another connection
got, err := asker.AskAsChunked("mesh-build-go", "seat:mesh-catalog.input", map[string]any{"digest": "d1"})
if err != nil {
t.Fatal(err)
@@ -72,4 +73,45 @@ func TestAChunkedSeatAnswerIsReceivedByTheRunnerAndHandedWhole(t *testing.T) {
if body, err := base64.StdEncoding.DecodeString(ans.File); err != nil || string(body) != string(file) {
t.Errorf("the file handed is %d bytes, want the %d sent: %v", len(body), len(file), err)
}
// Design 47 G A-257, A-254: a bundle answers a chunked verb with the whole file in the member `file`; the runner that serves the
// verb sends the file in chunks (each acknowledged before the next) and answers the count, so the bundle sends none, and the asker
// of one node's verb receives it whole.
t.Run("ABundlesChunkedAnswerIsSentByTheRunner", func(t *testing.T) {
url := mt.URL(t)
holder := dial(t, url, "mesh-build-go", "anchor", "")
asker := dial(t, url, "mesh-delivery-indexer", "anchor", "")
file := make([]byte, buscalls.SeatChunkBytes*2+7)
for i := range file {
file[i] = byte(i * 7)
}
stop, err := holder.Handle("seat:node-build-go.output@anchor", func(body json.RawMessage) (any, error) {
answer := map[string]any{"ask": "a1", "file": base64.StdEncoding.EncodeToString(file)}
return holder.AnswerChunked("node-build-go", "output", body, answer)
})
if err != nil {
t.Fatal(err)
}
defer stop()
holder.Flush() // the bus holds the verb's subscription before it is asked: the asker is another connection
got, err := asker.AskAsChunked("mesh-delivery-indexer", "seat:node-build-go.output@anchor", map[string]any{"ask": "a1"})
if err != nil {
t.Fatal(err)
}
var ans struct {
Ask string `json:"ask"`
Chunks int `json:"chunks"`
File string `json:"file"`
}
if err := json.Unmarshal(got.Result, &ans); err != nil || ans.Ask != "a1" || ans.Chunks != 3 {
t.Fatalf("the answer %s: %v", got.Result, err)
}
if body, err := base64.StdEncoding.DecodeString(ans.File); err != nil || string(body) != string(file) {
t.Errorf("the file handed is %d bytes, want the %d sent: %v", len(body), len(file), err)
}
// an answer with no file is the verb's own: no chunks are sent
plain, err := holder.AnswerChunked("node-build-go", "output", json.RawMessage(`{"call":"c"}`), map[string]any{"ask": "x", "chunks": 0})
if m, _ := plain.(map[string]any); err != nil || m["chunks"] != 0 {
t.Fatalf("an answer with no file was %v, %v", plain, err)
}
})
}
+76
View File
@@ -10,6 +10,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/membership"
"github.com/nats-io/nats.go"
@@ -143,6 +144,76 @@ func (h *busHistory) Read(keys string, from uint64, limit int) ([]ledger.Kept, e
}
}
// Purge removes every operation of one instance from the history stream, on the module's ledger connection, the one
// granted the purge of its own ledger's streams.
func (h *busHistory) Purge(key string) error {
return purgeSubject(h.read, h.write, h.stream, h.prefix+"."+key)
}
// Fill is the history stream's fill against its cap, the cap the manifest gives the ledger.
func (h *busHistory) Fill() (buscalls.Fill, error) {
name := h.stream
if name == "" {
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
js, err := h.read.JetStream()
if err != nil {
return buscalls.Fill{}, err
}
if name, err = js.StreamNameBySubject(ctx, h.prefix+".>"); err != nil {
return buscalls.Fill{}, fmt.Errorf("no stream on the bus keeps %s: %w", h.prefix, err)
}
}
f, err := h.read.streamFill(name)
return f.Fill, err
}
// purgeSubject purges one subject's messages from the stream that keeps it: the stream named, or found on the runner's
// own connection by its subject, purged on the ledger connection.
func purgeSubject(read, write *Conn, stream, subject string) error {
if write == nil {
return errors.New("a ledger's streams are purged only on its module's ledger connection, and none is held")
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
if stream == "" {
js, err := read.JetStream()
if err != nil {
return err
}
if stream, err = js.StreamNameBySubject(ctx, subject); err != nil {
return fmt.Errorf("no stream on the bus keeps %s: %w", subject, err)
}
}
// The purge is asked of the stream by its name, with no read of its info first: the ledger account is granted the
// purge of its own ledger's two streams and nothing else (PurgeAPI).
body, err := json.Marshal(map[string]string{"filter": subject})
if err != nil {
return err
}
reply, err := write.nc.RequestWithContext(ctx, PurgeAPI(stream), body)
if err != nil {
return fmt.Errorf("purging %s from %s: %w", subject, stream, err)
}
var ack struct {
Success bool `json:"success"`
Error *struct {
Description string `json:"description"`
} `json:"error"`
}
if err := json.Unmarshal(reply.Data, &ack); err != nil || !ack.Success {
why := string(reply.Data)
if ack.Error != nil {
why = ack.Error.Description
}
return fmt.Errorf("the bus refused the purge of %s from %s: %s", subject, stream, why)
}
return nil
}
// PurgeAPI is the bus's subject that purges a stream, the one a ledger account is granted for its own ledger's streams.
func PurgeAPI(stream string) string { return "$JS.API.STREAM.PURGE." + stream }
type busState struct {
c *Conn
bucket string
@@ -231,6 +302,11 @@ func (s *busSnapshots) Get(key string) ([]byte, uint64, error) {
return lastOn(s.read, "", s.prefix+"."+key)
}
// Purge removes one instance's snapshot, on the module's ledger connection.
func (s *busSnapshots) Purge(key string) error {
return purgeSubject(s.read, s.write, "", s.prefix+"."+key)
}
func (s *busSnapshots) Put(key string, value []byte, expect uint64) error {
if s.write == nil {
return errors.New("a ledger's snapshot is published only on its module's ledger connection, and none is held")
-27
View File
@@ -1,27 +0,0 @@
package bus
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// Design 47 G §347, §59, §462: a sent object is read by the reference that carried it, from a sender's sent store only; a
// state's objects, and a reference that is none are refused before the bus is asked.
func TestSentGetRefusesWhatIsNoSentObjectsReference(t *testing.T) {
digest := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
for _, c := range []struct {
name string
req buscalls.ObjectsSentGetRequest
want string
}{
{"no digest", buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3}, "named by the reference"},
{"a state's objects", buscalls.ObjectsSentGetRequest{Store: "m_asks-objects", Object: "o", Size: 3, Digest: digest}, "no sent store"},
} {
_, err := (&Conn{}).SentGet(c.req)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: got %v, want an error containing %q", c.name, err, c.want)
}
}
}
+68 -69
View File
@@ -87,6 +87,74 @@ func TestAWatchHoldsWhileItsCurrentValuesKeepArriving(t *testing.T) {
}
return nil
})
// **A watch given up, or stopped, hands over nothing more** (issue 774). The runtime's watch stopped its watcher
// when the current values did not arrive within the bound, but the watcher still held values, and its goroutine
// went on handing them over: to a watch id the bundle no longer held (on the provisioner, 51 failed watches were
// followed by 1257 orphan changes). Each row blocks the bundle on its first value, gives the watch up, lets the
// bundle go and counts what it is still handed.
t.Run("AWatchGivenUpHandsOverNothingMore", func(t *testing.T) {
for _, tc := range []struct {
name string
refused bool // the bound gives it up, or the caller stops it
}{
{"refused at the bound", true},
{"stopped by its caller", false},
} {
t.Run(tc.name, func(t *testing.T) {
was := bus.StateTimeout
bus.StateTimeout = 300 * time.Millisecond
t.Cleanup(func() { bus.StateTimeout = was })
conn, kv := owedWatcher(t)
put := func(i int) {
if _, err := kv.Put(fmt.Sprintf("novox/hq.%d.%040x.status=mesh/check", i+1, i), []byte(`{"v":1}`)); err != nil {
t.Fatal(err)
}
}
if tc.refused {
for i := 0; i < 4; i++ {
put(i)
}
}
var calls atomic.Int32
first := make(chan struct{}, 1)
release := make(chan struct{})
deliver := func(ch bus.StateChange) error {
if calls.Add(1) == 1 {
first <- struct{}{}
<-release
}
return nil
}
var stop func()
if tc.refused {
if _, err := conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err == nil {
t.Fatal("the watch whose bundle took nothing was not refused")
}
} else {
var err error
if stop, err = conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err != nil {
t.Fatal(err)
}
for i := 0; i < 4; i++ {
put(i)
}
select {
case <-first:
case <-time.After(5 * time.Second):
t.Fatal("the bundle was handed no change")
}
stop()
}
// Let the watcher's server side deliver what it held, then free the bundle.
time.Sleep(300 * time.Millisecond)
close(release)
time.Sleep(500 * time.Millisecond)
if n := calls.Load(); n != 1 {
t.Fatalf("the bundle was handed %d changes of a watch given up, want 1 (the one in hand)", n)
}
})
}
})
}
// A watch whose bundle takes nothing for longer than the bound is still refused, so a bundle that hangs
@@ -187,72 +255,3 @@ func owedWatcher(t *testing.T) (*bus.Conn, interface {
})
return conn, kv
}
// **A watch given up, or stopped, hands over nothing more** (issue 774). The runtime's watch stopped its watcher
// when the current values did not arrive within the bound, but the watcher still held values, and its goroutine
// went on handing them over: to a watch id the bundle no longer held (on the provisioner, 51 failed watches were
// followed by 1257 orphan changes). Each row blocks the bundle on its first value, gives the watch up, lets the
// bundle go and counts what it is still handed.
func TestAWatchGivenUpHandsOverNothingMore(t *testing.T) {
for _, tc := range []struct {
name string
refused bool // the bound gives it up, or the caller stops it
}{
{"refused at the bound", true},
{"stopped by its caller", false},
} {
t.Run(tc.name, func(t *testing.T) {
was := bus.StateTimeout
bus.StateTimeout = 300 * time.Millisecond
t.Cleanup(func() { bus.StateTimeout = was })
conn, kv := owedWatcher(t)
put := func(i int) {
if _, err := kv.Put(fmt.Sprintf("novox/hq.%d.%040x.status=mesh/check", i+1, i), []byte(`{"v":1}`)); err != nil {
t.Fatal(err)
}
}
if tc.refused {
for i := 0; i < 4; i++ {
put(i)
}
}
var calls atomic.Int32
first := make(chan struct{}, 1)
release := make(chan struct{})
deliver := func(ch bus.StateChange) error {
if calls.Add(1) == 1 {
first <- struct{}{}
<-release
}
return nil
}
var stop func()
if tc.refused {
if _, err := conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err == nil {
t.Fatal("the watch whose bundle took nothing was not refused")
}
} else {
var err error
if stop, err = conn.StateWatch("gitea", "mesh-delivery-checker.owed", "", deliver); err != nil {
t.Fatal(err)
}
for i := 0; i < 4; i++ {
put(i)
}
select {
case <-first:
case <-time.After(5 * time.Second):
t.Fatal("the bundle was handed no change")
}
stop()
}
// Let the watcher's server side deliver what it held, then free the bundle.
time.Sleep(300 * time.Millisecond)
close(release)
time.Sleep(500 * time.Millisecond)
if n := calls.Load(); n != 1 {
t.Fatalf("the bundle was handed %d changes of a watch given up, want 1 (the one in hand)", n)
}
})
}
}
@@ -1,6 +1,11 @@
package bus
import "testing"
import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// novox/hq ADR 0363 §2, §3: an event goes to the stream whose pattern carries it, as the bus matches subjects.
func TestAnEventFallsUnderAStreamsPatternAsTheBusMatchesIt(t *testing.T) {
@@ -22,6 +27,24 @@ func TestAnEventFallsUnderAStreamsPatternAsTheBusMatchesIt(t *testing.T) {
t.Errorf("%q under %q: %v, want %v", c.event, c.pattern, got, c.want)
}
}
// Design 47 G §347, §59, §462: a sent object is read by the reference that carried it, from a sender's sent store only; a
// state's objects, and a reference that is none are refused before the bus is asked.
t.Run("SentGetRefusesWhatIsNoSentObjectsReference", func(t *testing.T) {
digest := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
for _, c := range []struct {
name string
req buscalls.ObjectsSentGetRequest
want string
}{
{"no digest", buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3}, "named by the reference"},
{"a state's objects", buscalls.ObjectsSentGetRequest{Store: "m_asks-objects", Object: "o", Size: 3, Digest: digest}, "no sent store"},
} {
_, err := (&Conn{}).SentGet(c.req)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: got %v, want an error containing %q", c.name, err, c.want)
}
}
})
}
// A membership naming no stream publishes every event on EVENTS, exactly as before (issue 725).
+87
View File
@@ -0,0 +1,87 @@
package bus
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"github.com/nats-io/nats.go/jetstream"
)
// SubscribeNew hears one event, named `<emitter>.<event>`, from the next message on, on an ordered consumer of its own on each
// stream that carries it, and hands every message to deliver as the envelope a module sees. The subjects the name lands on are
// mesh-sdk's one rule (buscompose.EventSubjects, design 47 G §559, A-153), so a name with a wildcard is refused, naming the rule;
// a name no stream on the bus carries is an error, never a subscription that hears nothing. stop ends every consumer.
func (c *Conn) SubscribeNew(name string, deliver func(json.RawMessage) error) (func(), error) {
subjects, err := buscompose.EventSubjects(name, "")
if err != nil {
return nil, err
}
js, err := c.JetStream()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), StateTimeout)
defer cancel()
// The streams that carry each subject, found by the bus: a namespace no stream carries hears nothing and is skipped.
byStream := map[string][]string{}
for _, subject := range subjects {
if strings.HasSuffix(subject, ".>") {
continue
}
stream, err := js.StreamNameBySubject(ctx, subject)
if err != nil || stream == "" {
continue
}
byStream[stream] = append(byStream[stream], subject, subject+".>")
}
if len(byStream) == 0 {
return nil, fmt.Errorf("no stream on the bus carries %s (%s): nothing would be heard", name, strings.Join(subjects, ", "))
}
var stops []func()
stopAll := func() {
for _, s := range stops {
s()
}
}
for _, stream := range slices.Sorted(mapKeys(byStream)) {
cons, err := js.OrderedConsumer(ctx, stream, jetstream.OrderedConsumerConfig{FilterSubjects: byStream[stream],
DeliverPolicy: jetstream.DeliverNewPolicy})
if err != nil {
stopAll()
return nil, fmt.Errorf("an ordered consumer of %s on %s: %w", name, stream, err)
}
cc, err := cons.Consume(func(m jetstream.Msg) {
env, ok := envelopeOf(m.Subject(), m.Headers(), m.Data())
if !ok {
return
}
raw, err := json.Marshal(env)
if err != nil {
return
}
if err := deliver(raw); err != nil && c.Logf != nil {
c.Logf("[mesh-tools] a subscription for new %s messages did not take one: %v", name, err)
}
})
if err != nil {
stopAll()
return nil, fmt.Errorf("consuming %s on %s: %w", name, stream, err)
}
stops = append(stops, cc.Stop)
}
return stopAll, nil
}
func mapKeys(m map[string][]string) func(func(string) bool) {
return func(yield func(string) bool) {
for k := range m {
if !yield(k) {
return
}
}
}
}
+17 -11
View File
@@ -19,7 +19,7 @@ package console
import (
assigner "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"context"
@@ -927,6 +927,21 @@ func schemaWithoutNode(raw json.RawMessage) map[string]any {
}
// answerText is a discovery verb's answer as MCP content: JSON, indented.
// callFailed is why a call failed, as mesh_call says it: a marked tool or verb that reaches no responder names `withheld`
// and the grants part's read (design 46 7-approval-mark §6), and a target known only from an earlier round says so.
func (s *Surface) callFailed(t target, err error) string {
why := whyItFailed(t.Address, err)
if ok, _ := s.markedTool(t); ok && errors.Is(err, bus.ErrNoResponders) {
// A marked subject withheld from a machine whose tool runner does not honour the mark (§6).
why += "; it needs the operator's approval, so it may be withheld where the machine's tool runner does not state " +
"approval-mark/1: mesh-grants.grants_user_list names it under withheld"
}
if t.Stale != nil {
why += " (" + t.Stale.say() + ")"
}
return why
}
func answerText(v any) map[string]any {
var b strings.Builder
enc := json.NewEncoder(&b)
@@ -1137,16 +1152,7 @@ func (s *Surface) discover(ctx context.Context, name string, args map[string]any
got, err = callTool(s.conn, t.Key, callArgs, nil, x.Listing)
}
if err != nil {
why := whyItFailed(t.Address, err)
if ok, _ := s.markedTool(t); ok && errors.Is(err, bus.ErrNoResponders) {
// A marked subject withheld from a machine whose tool runner does not honour the mark (§6).
why += "; it needs the operator's approval, so it may be withheld where the machine's tool runner does not state " +
"approval-mark/1: mesh-grants.grants_user_list names it under withheld"
}
if t.Stale != nil {
why += " (" + t.Stale.say() + ")"
}
return failure(why)
return failure(s.callFailed(t, err))
}
result, unread := resolveSent(s.conn, got.Result)
content := []map[string]any{{"type": "text", "text": pretty(result)}}
@@ -6,6 +6,7 @@ import (
"log"
"strings"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
)
@@ -20,6 +21,10 @@ type FirstViewer interface {
// state and since when.
type entryLister interface{ Entries() any }
// ownBuilder is a FirstViewer that also says the build the tool runner itself was started with (design 47 G §205, §424):
// mesh_first_views names it, or says `own build unknown` and why.
type ownBuilder interface{ OwnBuild() process.Build }
// firstViewsAnswer is mesh_first_views and mesh_first_view_again.
func (s *Surface) firstViewsAnswer(ctx context.Context, name string, args map[string]any) map[string]any {
fv, ok := s.src.(FirstViewer)
@@ -29,6 +34,9 @@ func (s *Surface) firstViewsAnswer(ctx context.Context, name string, args map[st
}
if name == verbFirstViews {
out := map[string]any{"first views": fv.Status()}
if ob, ok := fv.(ownBuilder); ok {
out["own build"] = ob.OwnBuild().String()
}
if el, ok := fv.(entryLister); ok {
out["entries"] = el.Entries()
}
@@ -7,6 +7,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
)
@@ -14,12 +15,16 @@ import (
type viewer struct {
busSource
asked []string
build process.Build
}
func (v *viewer) Status() []views.Status {
return []views.Status{{Owner: "mesh-delivery-assigner", Tool: "assignments", State: "assignments", Revision: 4, Asks: 1}}
}
// OwnBuild is the build the viewer was started with.
func (v *viewer) OwnBuild() process.Build { return v.build }
// Entries are the entries by fingerprint the viewer keeps (design 47 G A-103).
func (v *viewer) Entries() any { return []string{"fp:a asked"} }
@@ -41,6 +46,13 @@ func TestTheConsoleShowsAndAsksAgainTheRunnersFirstViews(t *testing.T) {
if got := text(s.discover(context.Background(), "mesh_first_views", nil)); !strings.Contains(got, "mesh-delivery-assigner") || !strings.Contains(got, "assignments") || !strings.Contains(got, "fp:a asked") {
t.Errorf("mesh_first_views shows %s", got)
}
// The tool runner's own build is named, or said unknown with the reason (design 47 G §205, §424).
for build, want := range map[string]string{"abcd1234": "abcd1234", "": "own build unknown"} {
v.build = process.ReadOwnBuild(func(string) string { return build })
if got := text(s.discover(context.Background(), "mesh_first_views", nil)); !strings.Contains(got, `own build`) || !strings.Contains(got, want) {
t.Errorf("mesh_first_views with the build %q shows %s", build, got)
}
}
got := s.discover(context.Background(), "mesh_first_view_again", map[string]any{"why": "doubted"})
if len(v.asked) != 1 || v.asked[0] != "doubted" || !strings.Contains(text(got), "asked again") {
t.Errorf("mesh_first_view_again asked %v and answered %s", v.asked, text(got))
@@ -70,8 +70,8 @@ func TestTheManifestNamesNoWaitsAndKeepsItsApprovals(t *testing.T) {
// `registration-changed`, and the gate's `source-moved`; never the indexer's `views.modules` or `views.manifests`), the two
// events of the seat that serves the seat list (ADR 0636 §4) and the keeper's `findings-asked`, and nothing else.
wantConsumes := []string{"mesh-catalog.catalog-revision", "mesh-catalog.registration-changed", "mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed", "mesh-conditions.findings-asked", "mesh-delivery-assigner.views.assignments",
"mesh-delivery-assigner.views.seat-holders", "mesh-delivery.source-moved"}
"mesh-catalog.seat-definition-removed", "mesh-conditions.findings-asked", "seat:mesh-module-assigner.views.assignments",
"seat:mesh-module-assigner.views.seat-holders", "mesh-delivery.source-moved"}
if !reflect.DeepEqual(m.Consumes, wantConsumes) {
t.Errorf("the tool runner consumes %v, want %v", m.Consumes, wantConsumes)
}
@@ -11,7 +11,8 @@ import (
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// A tool's own error that mentions 503 is the tool's answer, not a routing fault (issue 1529); a real no-responders still says
// A marked tool or verb that reaches no responder names withheld and the grants part's read (design 46 7-approval-mark §6);
// an unmarked one, or a marked one failing otherwise, does not. A tool's own error that mentions 503 is the tool's answer, not a routing fault (issue 1529); a real no-responders still says
// nothing serves.
func TestARealNoRespondersStillSaysNothingServes(t *testing.T) {
got := whyItFailed("scanner.scan", errors.New("the scanner refused the scan job: 503 Service Unavailable"))
@@ -23,4 +24,20 @@ func TestARealNoRespondersStillSaysNothingServes(t *testing.T) {
t.Fatalf("%v: got %q", err, got)
}
}
s := &Surface{Approvals: &fakeApprovals{marked: "shell.wipe"}}
for _, c := range []struct {
target target
err error
hinted bool
}{
{target{Address: "shell.wipe", Tool: Tool{Module: "shell", Name: "wipe"}}, bus.ErrNoResponders, true},
{target{Address: "node-login-shell.execute", Seat: true, Tool: Tool{Module: "node-login-shell", Name: "execute"}}, bus.ErrNoResponders, false},
{target{Address: "shell.tidy", Tool: Tool{Module: "shell", Name: "tidy"}}, bus.ErrNoResponders, false},
{target{Address: "shell.wipe", Tool: Tool{Module: "shell", Name: "wipe"}}, errors.New("its own failure"), false},
} {
got := s.callFailed(c.target, c.err)
if hinted := strings.Contains(got, "approval-mark/1: mesh-grants.grants_user_list names it under withheld"); hinted != c.hinted {
t.Errorf("%s (%v): %q", c.target.Address, c.err, got)
}
}
}
@@ -1,9 +1,18 @@
package console
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/stateobjects"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// A refusal is said only for the bus's own refusals; a tool's answer that mentions authorization is the
@@ -19,6 +28,45 @@ func TestARefusalIsSaidOnlyForTheBussOwn(t *testing.T) {
t.Errorf("%q read as a refusal: %v, want %v", msg, got, refusal)
}
}
// ADR 0666 §16: an answer a module sent as a reference is read from its sent store for the person asking; one the bus does
// not let the runner read is shown as its reference with its size and why it was not read.
t.Run("AnAnswerSentAsAReferenceIsReadForThePersonAsking", func(t *testing.T) {
nc, err := nats.Connect(meshtest.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
obs, err := js.CreateObjectStore(context.Background(), jetstream.ObjectStoreConfig{Bucket: "m_sent-objects"})
if err != nil {
t.Fatal(err)
}
conn, err := bus.Connect(bus.Credential{URL: meshtest.URL(t), Module: "console", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer conn.Close()
body := []byte(`{"lines":"` + strings.Repeat("x", 9<<20) + `"}`)
sum := sha256.Sum256(body)
ref := stateobjects.Ref{Store: "m_sent-objects", Object: "tool.lines." + hex.EncodeToString(sum[:]), Size: int64(len(body)),
Digest: hex.EncodeToString(sum[:]), Form: stateobjects.FormVersion}
if _, err := obs.PutBytes(context.Background(), ref.Object, body); err != nil {
t.Fatal(err)
}
if got, unread := resolveSent(conn, stateobjects.Wrap(ref)); unread != "" || string(got) != string(body) {
t.Fatalf("read %d bytes of %d, %q", len(got), len(body), unread)
}
missing := ref
missing.Store = "gone_sent-objects"
wrapped := stateobjects.Wrap(missing)
got, unread := resolveSent(conn, wrapped)
if string(got) != string(wrapped) || !strings.Contains(unread, "9437") || !strings.Contains(unread, "gone_sent-objects") {
t.Fatalf("an unread answer showed %q and %q", got, unread)
}
if got, unread := resolveSent(conn, []byte(`{"a":1}`)); unread != "" || string(got) != `{"a":1}` {
t.Fatalf("a plain answer showed %q, %q", got, unread)
}
})
}
// **A timeout says it is not a failure** (novox/hq issue 265): a push that outlasted the wait had
-56
View File
@@ -1,56 +0,0 @@
package console
import (
"context"
"crypto/sha256"
"encoding/hex"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/stateobjects"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
// ADR 0666 §16: an answer a module sent as a reference is read from its sent store for the person asking; one the bus does
// not let the runner read is shown as its reference with its size and why it was not read.
func TestAnAnswerSentAsAReferenceIsReadForThePersonAsking(t *testing.T) {
nc, err := nats.Connect(meshtest.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
obs, err := js.CreateObjectStore(context.Background(), jetstream.ObjectStoreConfig{Bucket: "m_sent-objects"})
if err != nil {
t.Fatal(err)
}
conn, err := bus.Connect(bus.Credential{URL: meshtest.URL(t), Module: "console", Node: "anchor"})
if err != nil {
t.Fatal(err)
}
defer conn.Close()
body := []byte(`{"lines":"` + strings.Repeat("x", 9<<20) + `"}`)
sum := sha256.Sum256(body)
ref := stateobjects.Ref{Store: "m_sent-objects", Object: "tool.lines." + hex.EncodeToString(sum[:]), Size: int64(len(body)),
Digest: hex.EncodeToString(sum[:]), Form: stateobjects.FormVersion}
if _, err := obs.PutBytes(context.Background(), ref.Object, body); err != nil {
t.Fatal(err)
}
if got, unread := resolveSent(conn, stateobjects.Wrap(ref)); unread != "" || string(got) != string(body) {
t.Fatalf("read %d bytes of %d, %q", len(got), len(body), unread)
}
missing := ref
missing.Store = "gone_sent-objects"
wrapped := stateobjects.Wrap(missing)
got, unread := resolveSent(conn, wrapped)
if string(got) != string(wrapped) || !strings.Contains(unread, "9437") || !strings.Contains(unread, "gone_sent-objects") {
t.Fatalf("an unread answer showed %q and %q", got, unread)
}
if got, unread := resolveSent(conn, []byte(`{"a":1}`)); unread != "" || string(got) != `{"a":1}` {
t.Fatalf("a plain answer showed %q, %q", got, unread)
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"encoding/json"
"sync"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
+1 -1
View File
@@ -17,7 +17,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/catalogue"
+106 -61
View File
@@ -16,8 +16,8 @@ import (
"git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
opseat "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/seats/operatorchannel"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
@@ -203,7 +203,7 @@ func TestMeshWaitsTimeoutIsBounded(t *testing.T) {
// a wildcard, of one token, more than 16 names, none, or a from of another word is refused at once naming the rule; a
// wait whose every subscription the bus refuses answers after its timeout, in words naming no wait grant.
func TestMeshWaitRefusesLoudlyAndNeverLoops(t *testing.T) {
s, mesh := waitingRuntime(t)
s, _ := waitingRuntime(t)
var seventeen []any
for i := 0; i < 17; i++ {
seventeen = append(seventeen, fmt.Sprintf("mesh-knowledge.reviewed%d", i))
@@ -230,63 +230,80 @@ func TestMeshWaitRefusesLoudlyAndNeverLoops(t *testing.T) {
}
}
// E §14: against the catalog half, a misspelt emitter is refused naming the half and its revision, a known emitter's
// undeclared event is waited on and its timeout says so, and a declared one says nothing of it.
half := NewSurface(s.conn, halfSource{busSource{s.conn}, map[string]meshcatalogclient.Half{"main": {
Revision: catalogseat.CatalogRevision{Revision: 7, Definitions: []catalogseat.CatalogRevisionDefinitionsItem{{Seat: "mesh-catalog"}}},
Entries: map[string]catalogseat.CatalogEntry{"mesh-delivery-view": {Module: "mesh-delivery-view",
Manifest: map[string]any{"emits": []any{"delivered"}}}},
}}}, "desk.node-tools")
for _, c := range []struct {
name, says string
refused bool
}{
{"mesh-delivry-view.delivered", "mesh-delivry-view is neither a seat nor a module of the catalog half (mesh-catalog.revision, revision 7 of main)", true},
{"mesh-delivery-view.planned", "no declaration names mesh-delivery-view.planned", false},
{"mesh-catalog.catalog-revision", "", false},
} {
text, isErr := callWait(half, map[string]any{"subjects": []any{c.name}, "timeout": 1})
if isErr != c.refused || c.says != "" && !strings.Contains(text, c.says) ||
c.says == "" && (strings.Contains(text, "undeclared") || strings.Contains(text, "unchecked")) {
t.Errorf("%s against the catalog half: %s", c.name, text)
t.Run("the catalog half", func(t *testing.T) {
t.Parallel()
// E §14: against the catalog half, a misspelt emitter is refused naming the half and its revision, a known emitter's
// undeclared event is waited on and its timeout says so, and a declared one says nothing of it.
half := NewSurface(s.conn, halfSource{busSource{s.conn}, map[string]meshcatalogclient.Half{"main": {
Revision: catalogseat.CatalogRevision{Revision: 7, Definitions: []catalogseat.CatalogRevisionDefinitionsItem{{Seat: "mesh-catalog"}}},
Entries: map[string]catalogseat.CatalogEntry{"mesh-delivery-view": {Module: "mesh-delivery-view",
Manifest: map[string]any{"emits": []any{"delivered"}}}},
}}}, "desk.node-tools")
for _, c := range []struct {
name, says string
refused bool
}{
{"mesh-delivry-view.delivered", "mesh-delivry-view is neither a seat nor a module of the catalog half (mesh-catalog.revision, revision 7 of main)", true},
{"mesh-delivery-view.planned", "no declaration names mesh-delivery-view.planned", false},
{"mesh-catalog.catalog-revision", "", false},
} {
text, isErr := callWait(half, map[string]any{"subjects": []any{c.name}, "timeout": 1})
if isErr != c.refused || c.says != "" && !strings.Contains(text, c.says) ||
c.says == "" && (strings.Contains(text, "undeclared") || strings.Contains(text, "unchecked")) {
t.Errorf("%s against the catalog half: %s", c.name, text)
}
}
}
})
t.Run("a bus that refuses every subscription", func(t *testing.T) {
t.Parallel()
// The bus refusing every subscription of the call: answered after the timeout, in §16's words.
b, err := server.NewServer(&server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, NoLog: true, NoSigs: true,
Users: []*server.User{{Username: "desk.node-tools", Password: "pw", Permissions: &server.Permissions{
Subscribe: &server.SubjectPermission{Allow: []string{"_INBOX.desk.node-tools.>", "mesh.assignment.desk.*"}},
}}}})
if err != nil {
t.Fatal(err)
}
go b.Start()
if !b.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(b.Shutdown)
conn, err := bus.Connect(bus.Credential{URL: b.ClientURL(), Module: "node-tools", Node: "desk", User: "desk.node-tools", Password: "pw"})
if err != nil {
t.Fatal(err)
}
conn.Logf = func(string, ...any) {}
t.Cleanup(conn.Close)
start := time.Now()
text, isErr := callWait(NewSurface(conn, nil, "desk.node-tools"), map[string]any{"subjects": []any{"mesh-delivery-view.delivered"}, "timeout": 1})
if took := time.Since(start); took < time.Second || took > 3*time.Second {
t.Errorf("a wait the bus refused whole answered after %s, want its timeout of 1s", took)
}
if !isErr || !strings.Contains(text, "the bus refused this account a subscription on mesh.seat.mesh-delivery-view.event.delivered") ||
!strings.Contains(text, "novox/hq ADR 0670 §1") || strings.Contains(text, "wait grant") || strings.Contains(text, "0367 §2") {
t.Errorf("a wait the bus refused whole: %s", text)
}
})
// The bus refusing every subscription of the call: answered after the timeout, in §16's words.
b, err := server.NewServer(&server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, NoLog: true, NoSigs: true,
Users: []*server.User{{Username: "desk.node-tools", Password: "pw", Permissions: &server.Permissions{
Subscribe: &server.SubjectPermission{Allow: []string{"_INBOX.desk.node-tools.>", "mesh.assignment.desk.*"}},
}}}})
if err != nil {
t.Fatal(err)
}
go b.Start()
if !b.ReadyForConnections(10 * time.Second) {
t.Fatal("the bus did not come up")
}
t.Cleanup(b.Shutdown)
conn, err := bus.Connect(bus.Credential{URL: b.ClientURL(), Module: "node-tools", Node: "desk", User: "desk.node-tools", Password: "pw"})
if err != nil {
t.Fatal(err)
}
conn.Logf = func(string, ...any) {}
t.Cleanup(conn.Close)
start := time.Now()
text, isErr := callWait(NewSurface(conn, nil, "desk.node-tools"), map[string]any{"subjects": []any{"mesh-delivery-view.delivered"}, "timeout": 1})
if took := time.Since(start); took < time.Second || took > 3*time.Second {
t.Errorf("a wait the bus refused whole answered after %s, want its timeout of 1s", took)
}
if !isErr || !strings.Contains(text, "the bus refused this account a subscription on mesh.seat.mesh-delivery-view.event.delivered") ||
!strings.Contains(text, "novox/hq ADR 0670 §1") || strings.Contains(text, "wait grant") || strings.Contains(text, "0367 §2") {
t.Errorf("a wait the bus refused whole: %s", text)
}
// Design 34 every-event §37: a wait whose subjects overlap a marked event's, or a stream's with a `grant`, runs only on the
// operator's warrant for that one wait; with none it subscribes nothing, asks once and answers at its timeout or the choice.
markedWaits(t, s, mesh)
// Each group of rows has a runtime of its own, so the waits of the groups run side by side (A-311: the one-second floor of
// a wait's timeout stands, and no clock is injected).
for name, group := range map[string]func(*testing.T, *Surface, *mt.Mesh){
"no warrant": markedWaitsWithNoWarrant, "a warrant": markedWaitsOnAWarrant, "the catalog half unread": markedWaitsBeforeTheHalfIsRead,
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
s, mesh := waitingRuntime(t)
group(t, s, mesh)
})
}
}
// markedWaits are §37's rows, against a gate on the test bus, the router's records written there as the router writes them.
func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
// markedWaitsWithNoWarrant are §37's rows of a wait with no warrant, against a gate on the test bus, the router's records written there as the router writes them.
func markedWaitsWithNoWarrant(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
@@ -321,6 +338,17 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
!strings.Contains(text, `"timed-out": true`) || gw.asks() != 3 {
t.Errorf("from a seat: %s, %d asks", text, gw.asks())
}
}
// markedWaitsOnAWarrant are §37's rows of a wait chosen while it waits and of one on a warrant.
func markedWaitsOnAWarrant(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
start := time.Now()
text, isErr := callWait(g, args)
return text, isErr, time.Since(start)
}
// The most timeout a wait takes, its ask chosen while it waits: answered at the choice; the same call made then
// subscribes and hears the event published after it, answered with its approval; a warrant for other names, another
// filter or another timeout is refused, asked anew.
@@ -328,7 +356,7 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
done := make(chan string, 1)
go func() { text, _, d := took(long); done <- fmt.Sprintf("%s after %s", text, d) }()
mt.Until(t, func() error {
if gw.asks() < 4 {
if gw.asks() < 1 {
return os.ErrNotExist
}
return nil
@@ -342,7 +370,7 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
case <-time.After(10 * time.Second):
t.Fatal("a long marked wait was not answered at the choice")
}
before = g.conn.Subscriptions()
before := g.conn.Subscriptions()
go func() {
listening(t, g, before, 6)
mesh.Emit(t, "mesh.mod.mail.event.message", map[string]any{"n": 2})
@@ -361,6 +389,17 @@ func markedWaits(t *testing.T, s *Surface, mesh *mt.Mesh) {
t.Errorf("another wait than the warrant's %v: %s", other, text)
}
}
}
// markedWaitsBeforeTheHalfIsRead are §37's rows of a wait before the catalog half is read, and of one whose runner build is unknown.
func markedWaitsBeforeTheHalfIsRead(t *testing.T, s *Surface, mesh *mt.Mesh) {
gw := newGatedWaits(t, s, mesh, "src2:runner")
g := gw.surface
took := func(args map[string]any) (string, bool, time.Duration) {
start := time.Now()
text, isErr := callWait(g, args)
return text, isErr, time.Since(start)
}
// Before the catalog half is read nothing is asked: judged when its first view comes within the timeout, refused at
// the timeout when it does not.
gw.cat.mu.Lock()
@@ -562,17 +601,23 @@ func TestMeshWaitIsAnnouncedAmongSeven(t *testing.T) {
func fmtAnswer(out map[string]any) string { raw, _ := json.Marshal(out); return string(raw) }
// fakeApprovals is a gate as mesh_approvals reads it.
type fakeApprovals struct{ cancelled string }
type fakeApprovals struct {
cancelled string
// marked is the one `<module>.<tool>` its Describe says is marked.
marked string
}
func (f *fakeApprovals) List() ([]approval.Listed, error) {
return []approval.Listed{{Ask: "a1", State: "asked"}}, nil
}
func (f *fakeApprovals) Read(id string) (approval.Whole, error) { return approval.Whole{}, nil }
func (f *fakeApprovals) Applied() []approval.Marks { return nil }
func (f *fakeApprovals) Describe(_, _, _ string) (bool, string) { return false, "" }
func (f *fakeApprovals) Again(string) error { return nil }
func (f *fakeApprovals) Cancel(id string) error { f.cancelled = id; return nil }
func (f *fakeApprovals) Fill() (uint64, int64, error) { return 10, 64 << 20, nil }
func (f *fakeApprovals) Describe(module, _, name string) (bool, string) {
return f.marked != "" && f.marked == module+"."+name, "entry " + module
}
func (f *fakeApprovals) Again(string) error { return nil }
func (f *fakeApprovals) Cancel(id string) error { f.cancelled = id; return nil }
func (f *fakeApprovals) Fill() (uint64, int64, error) { return 10, 64 << 20, nil }
// A client that hangs up or cancels ends its wait at once: the subscription is closed and nothing is left
// waiting for the timeout (the review of issue 745). Over the console's own HTTP, as an agent host calls it.
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
+113 -32
View File
@@ -20,39 +20,41 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/novox/mesh-tools/node-tools/internal/bus"
)
// The owner and its first-view tools (the tools' addresses are `<owner>.<tool>`).
const Assigner = "mesh-delivery-assigner"
// AssignerSeat is the seat whose holder's views the runner follows: the holder is read from the catalog half's claims of the
// seat, never named in code (design 47 G §616, §428), and the names the console reads the two states by are the seat's
// (`<seat>.<state>`; the manifest's `consumes` names their views families by the seat, as `seat:<seat>.views.<state>`).
const AssignerSeat = meshmoduleassigner.Seat
// The names the console reads the assigner's two states by (`<owner>.<state>`), as the manifest's `consumes` names their views
// families (no seat's client names a holder, novox/hq ADR 0625 §4; TestTheFirstViewsAreTheManifestsViews).
const (
AssignmentsRead = Assigner + ".assignments"
SeatHoldersRead = Assigner + ".seat-holders"
AssignmentsRead = AssignerSeat + ".assignments"
SeatHoldersRead = AssignerSeat + ".seat-holders"
)
// Spec is one first view the runner holds.
type Spec struct{ Owner, Tool, State string }
// Spec is one first view the runner holds of the seat's holder: its first-view tool and the state.
type Spec struct{ Tool, State string }
// Specs are the two owner states asked of an owner's tool; the catalog half's views are adopted by the branch follower.
// Specs are the two states asked of the holder's tool; the catalog half's views are adopted by the branch follower.
var Specs = []Spec{
{Assigner, "assignments", "assignments"},
{Assigner, "seat-holders", "seat-holders"},
{"assignments", "assignments"},
{"seat-holders", "seat-holders"},
}
// Name is the name a state is read by, `<owner>.<state>`: AssignmentsRead and SeatHoldersRead.
func (s Spec) Name() string { return s.Owner + "." + s.State }
// Name is the name a state is read by, `<seat>.<state>`: AssignmentsRead and SeatHoldersRead.
func (s Spec) Name() string { return AssignerSeat + "." + s.State }
// Views is the first views: the assigner's two, and the catalog half's per watched branch.
type Views struct {
@@ -63,6 +65,7 @@ type Views struct {
adopted map[string]*adoption // by `<owner>.<state>`, the views the branch follower adopted
hearers map[string][]*adoption // event key `<seat>.<event>` -> the adopted views that hear it
branches *meshcatalogclient.Branches
holder *holding // the module that claims the assigner seat, once the catalog half names it
ctx context.Context
cancel context.CancelFunc
wg sync.WaitGroup
@@ -89,6 +92,9 @@ type Deps struct {
Findings FindingWriter
// StartFindings starts that writer, and its stop ends it; nil starts none.
StartFindings func(ctx context.Context) (stop func(), err error)
// Build is the build this process was started with (process.OwnBuild, read once at the start): its fingerprint, or
// why it is unknown (design 47 G §205, §424).
Build process.Build
// Node is the machine the runner is on, named in its findings' subject.
Node string
// Bound and Wait are the follower's (zero: two minutes, five seconds).
@@ -111,7 +117,7 @@ func Start(parent context.Context, conn *bus.Conn, logf func(string, ...any)) (*
// state adapter, JetStream and node are this connection's, in this one place (design 47 G A-16, A-27).
func DepsOf(conn *bus.Conn, logf func(string, ...any)) Deps {
me := conn.Module()
d := Deps{Module: me, Node: conn.Node(),
d := Deps{Module: me, Node: conn.Node(), Build: process.OwnBuild(),
Ask: func(address string, body any) (json.RawMessage, error) {
got, err := conn.Ask(address, body, "")
return got.Result, err
@@ -168,22 +174,6 @@ func StartWith(parent context.Context, d Deps, logf func(string, ...any)) (*View
ctx, cancel := context.WithCancel(parent)
v := &Views{logf: logf, ctx: ctx, cancel: cancel, followers: map[string]*views.Follower{}, byState: map[string]*views.Follower{},
adopted: map[string]*adoption{}, hearers: map[string][]*adoption{}, deps: d}
for _, s := range Specs {
s := s
call := func(tool string, body any) (json.RawMessage, error) { return d.Ask(s.Owner+"."+tool, body) }
f := &views.Follower{Owner: s.Owner, Tool: s.Tool, State: s.State, Ask: views.AskVia(call, s.Tool), Logf: logf,
Bound: d.Bound, Wait: d.Wait,
Raise: func(st views.Status) {
v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, true)
},
Clear: func(st views.Status) {
v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, false)
}}
v.followers[s.Name()] = f
v.byState[s.State] = f
v.wg.Add(1)
go func() { defer v.wg.Done(); _ = f.Start(ctx) }()
}
stopFindings := func() {}
if d.StartFindings != nil {
if stop, err := d.StartFindings(ctx); err != nil {
@@ -199,6 +189,9 @@ func StartWith(parent context.Context, d Deps, logf func(string, ...any)) (*View
logf("[mesh-tools] the catalog half is not followed (%v): the console cannot say which modules exist", err)
} else {
branches.Logf = func(format string, args ...any) { logf("[mesh-tools] "+format, args...) }
branches.Changed = func(_, module string, entry json.RawMessage, deleted bool, _ time.Time) {
v.claimed(module, entry, deleted)
}
v.branches = branches
}
stop, err := d.Consume(v.hear)
@@ -228,7 +221,9 @@ func (v *Views) hear(env bus.Envelope) error {
if err != nil {
return nil // not a views event: nothing to hold, nothing to retry
}
v.mu.Lock()
f := v.byState[ev.State]
v.mu.Unlock()
if f == nil || f.Owner != ev.Owner {
return nil
}
@@ -261,6 +256,13 @@ func (v *Views) finding(f condseat.Finding, tool string, raised bool) {
// not answered yet, so nothing is claimed of it.
func (v *Views) State(name string) (map[string][]byte, error) {
v.mu.Lock()
if state, ok := strings.CutPrefix(name, AssignerSeat+"."); ok { // the seat's name stands for its holder's
if v.holder == nil {
v.mu.Unlock()
return nil, fmt.Errorf("the catalog half names no holder of the seat %s yet, so %s has no first view", AssignerSeat, name)
}
name = v.holder.module + "." + state
}
f := v.followers[name]
v.mu.Unlock()
if f == nil {
@@ -297,6 +299,10 @@ func (v *Views) Status() []views.Status {
return out
}
// OwnBuild is the build the tool runner was started with: the fingerprint of the want its node-engine applied, or why it is
// unknown (`own build unknown: started without MESH_FINGERPRINT`).
func (v *Views) OwnBuild() process.Build { return v.deps.Build }
// Entries are the entries by fingerprint the catalog half's follower keeps, with their state and since when (design 47 G A-103).
func (v *Views) Entries() any {
if v.branches == nil {
@@ -489,3 +495,78 @@ func (v *Views) Modules() (map[string]catalogseat.CatalogEntry, error) {
}
return out, nil
}
// holding is the views of the module that claims the assigner seat, and what stops them.
type holding struct {
module string
cancel context.CancelFunc
views []*views.Follower
}
// claimed is the branch follower's word of one entry changed: an entry that serves a verb of the assigner seat makes its module
// the holder whose views the runner follows (a holder before it is dropped), the deletion of the holder's entry drops them
// (design 47 G §616, §428). It acts on the event alone; nothing waits on a timer for the claim.
func (v *Views) claimed(module string, entry json.RawMessage, deleted bool) {
claims := false
if !deleted {
var e catalogseat.CatalogEntry
if json.Unmarshal(entry, &e) == nil {
claims = slices.ContainsFunc(e.Serves, func(s string) bool { return strings.HasPrefix(s, AssignerSeat+".") })
}
}
v.mu.Lock()
cur := v.holder
switch {
case claims && cur != nil && cur.module == module, !claims && (cur == nil || cur.module != module), v.ctx.Err() != nil:
v.mu.Unlock()
return
}
if cur != nil {
v.dropHolderLocked()
}
if claims {
v.startHolderLocked(module)
}
v.mu.Unlock()
}
// startHolderLocked follows the first views of module, the holder of the seat, on the runner's own consumer.
func (v *Views) startHolderLocked(module string) {
ctx, cancel := context.WithCancel(v.ctx)
h := &holding{module: module, cancel: cancel}
d := v.deps
for _, s := range Specs {
s := s
call := func(tool string, body any) (json.RawMessage, error) { return d.Ask(module+"."+tool, body) }
f := &views.Follower{Owner: module, Tool: s.Tool, State: s.State, Ask: views.AskVia(call, s.Tool), Logf: v.logf,
Bound: d.Bound, Wait: d.Wait,
Raise: func(st views.Status) {
v.finding(unansweredFinding(d.Module, d.Node, st, time.Now()), st.Tool, true)
},
Clear: func(st views.Status) {
v.finding(unansweredFinding(d.Module, d.Node, st, time.Now()), st.Tool, false)
}}
v.followers[module+"."+s.State] = f
v.byState[s.State] = f
h.views = append(h.views, f)
v.wg.Add(1)
go func() { defer v.wg.Done(); _ = f.Start(ctx) }()
}
v.holder = h
v.logf("[mesh-tools] the seat %s is held by %s: its views are followed", AssignerSeat, module)
}
// dropHolderLocked stops the views of the holder and clears the findings they raised.
func (v *Views) dropHolderLocked() {
h := v.holder
v.holder = nil
h.cancel()
for _, f := range h.views {
delete(v.followers, f.Owner+"."+f.State)
delete(v.byState, f.State)
if st := f.Status(); st.Raised {
go v.finding(unansweredFinding(v.deps.Module, v.deps.Node, st, time.Now()), st.Tool, false)
}
}
v.logf("[mesh-tools] %s no longer holds the seat %s: its views are no longer followed", h.module, AssignerSeat)
}
@@ -17,6 +17,10 @@ import (
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"github.com/novox/mesh-tools/node-tools/internal/bus"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
)
@@ -41,7 +45,9 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
if err := json.Unmarshal(raw, &manifest); err != nil {
t.Fatal(err)
}
streamOf := map[string]string{Assigner: catalogue.StreamBusName("assigner-views")}
// the manifest names the assigner's views by the seat; the module the catalog half says claims it is the fixture's
const holder = "assigner-fixture"
streamOf := map[string]string{"seat:" + AssignerSeat: catalogue.StreamBusName("assigner-views")}
filters := map[string][]string{}
families := 0
for _, c := range manifest.Consumes {
@@ -53,16 +59,16 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
if stream == "" {
t.Fatalf("the manifest consumes %s, whose views stream the test does not know", c)
}
filters[stream] = append(filters[stream], process.ModuleStreamSubject(emitter, views.EventPrefix+event))
filters[stream] = append(filters[stream], process.ModuleStreamSubject(holder, views.EventPrefix+event))
families++
}
if families != len(Specs) {
t.Fatalf("the manifest consumes %d families, the runner holds %d first views", families, len(Specs))
}
runner := mt.MembershipOf("node-tools", "anchor", false, nil)
for _, owner := range []string{Assigner} {
stream := streamOf[owner]
mesh.ModuleStream(t, owner, stream)
for _, seat := range []string{"seat:" + AssignerSeat} {
stream := streamOf[seat]
mesh.ModuleStream(t, holder, stream)
mesh.ConsumerOn(t, stream, "anchor", "node-tools", filters[stream], 2*time.Second)
runner.Hears = append(runner.Hears, bus.HeardIssued{Stream: stream, Consumer: "anchor_node-tools"})
}
@@ -93,13 +99,13 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
t.Cleanup(stop)
// The owners are down: no tool answers, nothing is claimed, and past the bound each view is a finding.
if _, err := v.State(Assigner + ".assignments"); err == nil {
if _, err := v.State(AssignmentsRead); err == nil {
t.Fatal("a state its owner has not answered is claimed")
}
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(findings) != len(Specs)+catalogViews {
if len(findings) != catalogViews {
return errFindings(len(findings))
}
return nil
@@ -111,10 +117,25 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
t.Fatal(err)
}
t.Cleanup(nc.Close)
respond := func(subject string, answer any) {
if _, err := nc.Subscribe(subject, func(m *nats.Msg) {
raw, _ := json.Marshal(answer)
body, _ := json.Marshal(map[string]json.RawMessage{"result": raw})
_ = m.Respond(body)
}); err != nil {
t.Fatal(err)
}
}
// the catalog half answers: the gate's source names main, whose entries say the fixture claims the assigner seat
respond("mesh.seat.mesh-delivery.tool.source", delivery.SourceAnswer{Branches: []string{"main"}})
respond("mesh.seat.mesh-catalog.tool.registrations", catalogseat.RegistrationsAnswer{Registrations: []catalogseat.Registration{}})
respond("mesh.seat.mesh-catalog.tool.revision", catalogseat.RevisionAnswer{
Revision: &catalogseat.CatalogRevision{Revision: 3, Kind: meshcatalogclient.KindTrunk, Branch: "main"},
Entries: []catalogseat.CatalogEntry{{Module: holder, Serves: []string{AssignerSeat + ".assignments"}}}})
for _, s := range Specs {
s := s
if _, err := nc.Subscribe("mesh.mod."+s.Owner+".tool."+s.Tool, func(m *nats.Msg) {
answer, _ := json.Marshal(views.Answer{V: 1, Owner: s.Owner, State: s.State, Revision: 4, Created: time.Unix(1000, 0).UTC(),
if _, err := nc.Subscribe("mesh.mod."+holder+".tool."+s.Tool, func(m *nats.Msg) {
answer, _ := json.Marshal(views.Answer{V: 1, Owner: holder, State: s.State, Revision: 4, Created: time.Unix(1000, 0).UTC(),
Values: map[string]json.RawMessage{"first": json.RawMessage(`{"n":1}`)}})
body, _ := json.Marshal(map[string]json.RawMessage{"result": answer})
_ = m.Respond(body)
@@ -130,7 +151,7 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(findings) != catalogViews { // the seats' verbs are not served in this test: the catalog half stays unanswered
if len(findings) != 0 { // every view answered clears its finding
return errFindings(len(findings))
}
return nil
@@ -138,9 +159,9 @@ func TestTheRunnersConsumerCarriesTheOwnersViewsFamiliesWhenTheOwnersAreDownThen
// Each owner's next event arrives on the runner's consumer: all four families are bound.
for _, s := range Specs {
ev, _ := json.Marshal(views.Event{V: 1, Owner: s.Owner, State: s.State, Key: "next", Revision: 5, Created: time.Unix(1000, 0).UTC(),
ev, _ := json.Marshal(views.Event{V: 1, Owner: holder, State: s.State, Key: "next", Revision: 5, Created: time.Unix(1000, 0).UTC(),
Value: json.RawMessage(`{"n":2}`)})
mesh.Emit(t, process.ModuleStreamSubject(s.Owner, views.EventPrefix+s.State), json.RawMessage(ev))
mesh.Emit(t, process.ModuleStreamSubject(holder, views.EventPrefix+s.State), json.RawMessage(ev))
}
for _, s := range Specs {
name := s.Name()
+71 -25
View File
@@ -12,9 +12,9 @@ import (
"time"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
catalogseat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
delivery "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery"
"git.novox.be/novox/mesh-sdk/go/views"
"github.com/novox/mesh-tools/node-tools/internal/bus"
@@ -32,12 +32,14 @@ type owners struct {
// catalog is the revision of the watched branch main the seat mesh-catalog answers, and modules its entries
catalog int64
modules []string
// holder is the module among them that claims the assigner seat
holder string
}
func newOwners() *owners {
return &owners{state: map[string]map[string]string{"assignments": {"anchor": `{"p":1}`}, "seat-holders": {"seat": `{"h":1}`}},
rev: map[string]uint64{"assignments": 4, "seat-holders": 2}, asked: map[string]int{}, finds: map[string]bool{},
catalog: 3, modules: []string{"web"}}
catalog: 3, modules: []string{"web", fixtureHolder}, holder: fixtureHolder}
}
// Write and Clear are the runner's finding writer in a test: the findings written and not cleared, by their key.
@@ -71,7 +73,11 @@ func (o *owners) ask(address string, _ any) (json.RawMessage, error) {
case "seat:mesh-catalog.revision":
a := catalogseat.RevisionAnswer{Revision: &catalogseat.CatalogRevision{Revision: o.catalog, Kind: meshcatalogclient.KindTrunk, Branch: "main"}}
for _, m := range o.modules {
a.Entries = append(a.Entries, catalogseat.CatalogEntry{Module: m, Manifest: map[string]any{"module": m, "tools": []string{m + "_check"}}})
e := catalogseat.CatalogEntry{Module: m, Manifest: map[string]any{"module": m, "tools": []string{m + "_check"}}}
if m == o.holder {
e.Serves = []string{AssignerSeat + ".assignments", AssignerSeat + ".seat-holders"}
}
a.Entries = append(a.Entries, e)
}
return json.Marshal(a)
}
@@ -94,14 +100,10 @@ func (o *owners) say(t *testing.T, state, key, value string, revision uint64, ga
}
}
func owner(state string) string {
for _, s := range Specs {
if s.State == state {
return s.Owner
}
}
return ""
}
// fixtureHolder is the module the fixture catalog half says claims the assigner seat: no name is in the code.
const fixtureHolder = "assigner-fixture"
func owner(state string) string { return fixtureHolder }
func eventually(t *testing.T, what string, ok func() bool) {
t.Helper()
@@ -127,39 +129,39 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
defer stop()
// Down: nothing is claimed, and past the bound each unanswered view is a finding.
if _, err := v.State("mesh-delivery-assigner.assignments"); err == nil || !strings.Contains(err.Error(), "not been answered") {
t.Fatalf("a state its owner has not answered is claimed: %v", err)
if _, err := v.State(AssignmentsRead); err == nil || !strings.Contains(err.Error(), "names no holder") {
t.Fatalf("a state whose holder is not known is claimed: %v", err)
}
eventually(t, "no finding past the bound", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 4 })
eventually(t, "no finding past the bound", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 2 })
// Up: answered, the findings are cleared, and the pictures hold the owners' values.
o.mu.Lock()
o.down = false
o.mu.Unlock()
eventually(t, "not answered", func() bool { _, err := v.State("mesh-delivery-assigner.seat-holders"); return err == nil })
eventually(t, "not answered", func() bool { _, err := v.State(SeatHoldersRead); return err == nil })
eventually(t, "findings not cleared", func() bool { o.mu.Lock(); defer o.mu.Unlock(); return len(o.finds) == 0 })
if got, _ := v.Entry("mesh-delivery-assigner.seat-holders", "none"); got != nil {
if got, _ := v.Entry(AssignerSeat+".seat-holders", "none"); got != nil {
t.Errorf("an absent key is %q", got)
}
// The catalog half is followed by the shared branch follower on the runner's own consumer: the gate's source names main,
// main's view is adopted, and its entries are the modules, each with its manifest (design 47 G §196, §614).
eventually(t, "the catalog half holds no modules", func() bool { m, err := v.Modules(); return err == nil && len(m) == 1 })
eventually(t, "the catalog half holds no modules", func() bool { m, err := v.Modules(); return err == nil && len(m) == 2 })
m, _ := v.Modules()
if m["web"].Manifest["module"] != "web" {
t.Errorf("the module web is %+v", m["web"])
}
// A catalog-revision of the branch is heard on the same consumer and asks the view once more.
o.mu.Lock()
o.catalog, o.modules = 4, []string{"web", "db"}
o.catalog, o.modules = 4, []string{"web", "db", fixtureHolder}
o.mu.Unlock()
ev, _ := json.Marshal(catalogseat.CatalogRevisionEvent{V: 1, Branch: "main", Kind: meshcatalogclient.KindTrunk, Revision: 4})
if err := o.deliver(bus.Envelope{Key: catalogseat.Seat + "." + catalogseat.EventCatalogRevision, Body: ev}); err != nil {
t.Fatal(err)
}
eventually(t, "the catalog-revision was not taken", func() bool { m, err := v.Modules(); return err == nil && len(m) == 2 })
eventually(t, "the catalog-revision was not taken", func() bool { m, err := v.Modules(); return err == nil && len(m) == 3 })
// And the halves by branch, with their revision, which mesh_wait names checking an emitter (design 34 every-event §14).
if h, err := v.Halves(); err != nil || h["main"].Revision.Revision != 4 || len(h["main"].Entries) != 2 {
if h, err := v.Halves(); err != nil || h["main"].Revision.Revision != 4 || len(h["main"].Entries) != 3 {
t.Errorf("the halves: %+v, %v", h, err)
}
@@ -170,7 +172,7 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
o.say(t, "assignments", "laptop", `{"p":0}`, 5, false)
o.say(t, "assignments", "anchor", "", 6, false)
// (a deleted mark carries no value: Deleted is the encoding's, set below)
if got, _ := v.Entry("mesh-delivery-assigner.assignments", "laptop"); string(got) != `{"p":2}` {
if got, _ := v.Entry(AssignerSeat+".assignments", "laptop"); string(got) != `{"p":2}` {
t.Errorf("the next revision is not applied: %q", got)
}
if n := asked("assignments"); n != before {
@@ -188,7 +190,7 @@ func TestTheRunnerHoldsTheFourStatesFromTheOwnersFirstViewsAndTheirEvents(t *tes
assignments = s
}
}
if assignments.Owner != "mesh-delivery-assigner" || assignments.Tool != "assignments" || assignments.Asks < 2 || assignments.Answered.IsZero() {
if assignments.Owner != fixtureHolder || assignments.Tool != "assignments" || assignments.Asks < 2 || assignments.Answered.IsZero() {
t.Errorf("the status of assignments is %+v", assignments)
}
manifests := asked("seat-holders")
@@ -228,13 +230,19 @@ func TestTheFirstViewsAreTheManifestsViews(t *testing.T) {
t.Errorf("the first view %s is read by no name here", s.Name())
}
delete(want, s.Name())
if view := s.Owner + ".views." + s.Tool; !slices.Contains(m.Consumes, view) {
// the family is named by the seat, never by its holder (design 47 G §616)
if view := "seat:" + AssignerSeat + ".views." + s.Tool; !slices.Contains(m.Consumes, view) {
t.Errorf("the first view %s has no family in the manifest's consumes %v", view, m.Consumes)
}
}
if len(want) > 0 {
t.Errorf("no first view reads %v", want)
}
for _, c := range m.Consumes {
if strings.Contains(c, "assigner") && !strings.HasPrefix(c, "seat:") {
t.Errorf("the manifest names the assigner by a module: %s", c)
}
}
// the catalog half's follower hears these events and asks these verbs, and the manifest says so (design 47 G §614)
for _, ev := range []string{"mesh-catalog.catalog-revision", "mesh-catalog.registration-changed", "mesh-delivery.source-moved"} {
if !slices.Contains(m.Consumes, ev) {
@@ -246,4 +254,42 @@ func TestTheFirstViewsAreTheManifestsViews(t *testing.T) {
t.Errorf("the manifest still consumes the indexer's %s", ev)
}
}
// design 47 G §616: the holder of the assigner seat is whichever module the catalog half says serves it; when another one
// claims it the views are of the new holder, and when none does the seat's names answer that the catalog half names no holder.
t.Run("TheAssignersViewsFollowTheModuleThatClaimsTheSeat", func(t *testing.T) {
o := newOwners()
deps := Deps{Module: "node-tools", Ask: o.ask, Bound: time.Second, Wait: 10 * time.Millisecond,
Consume: func(deliver func(bus.Envelope) error) (func(), error) { o.deliver = deliver; return func() {}, nil }, Findings: o}
v, stop := StartWith(context.Background(), deps, func(string, ...any) {})
defer stop()
eventually(t, "not answered", func() bool { _, err := v.State(AssignmentsRead); return err == nil })
ownerOf := func() string {
for _, s := range v.Status() {
if s.State == "assignments" {
return s.Owner
}
}
return ""
}
if ownerOf() != fixtureHolder {
t.Fatalf("the views are of %q", ownerOf())
}
revise := func(rev int64, holder string, modules ...string) {
o.mu.Lock()
o.catalog, o.modules, o.holder = rev, modules, holder
o.mu.Unlock()
ev, _ := json.Marshal(catalogseat.CatalogRevisionEvent{V: 1, Branch: "main", Kind: meshcatalogclient.KindTrunk, Revision: rev})
if err := o.deliver(bus.Envelope{Key: catalogseat.Seat + "." + catalogseat.EventCatalogRevision, Body: ev}); err != nil {
t.Fatal(err)
}
}
revise(4, "other-holder", "web", "other-holder")
eventually(t, "the views did not move to the new holder", func() bool { return ownerOf() == "other-holder" })
eventually(t, "not answered", func() bool { _, err := v.State(SeatHoldersRead); return err == nil })
revise(5, "", "web")
eventually(t, "the views stayed after the claim went", func() bool {
_, err := v.State(AssignmentsRead)
return err != nil && strings.Contains(err.Error(), "names no holder")
})
})
}
+56 -2
View File
@@ -121,7 +121,7 @@ type Bus interface {
// busStateCalls are the methods of mesh-sdk buscalls a bundle asks and BusState answers.
var busStateCalls = map[string]bool{
buscalls.ObjectsPut: true, buscalls.ObjectsGet: true, buscalls.ObjectsList: true, buscalls.ObjectsDelete: true,
buscalls.ObjectsInfo: true, buscalls.StreamRead: true, buscalls.StreamFill: true, buscalls.WorkQueueSize: true,
buscalls.ObjectsInfo: true, buscalls.StreamRead: true, buscalls.StreamFill: true, buscalls.StreamPublish: true, buscalls.WorkQueueSize: true,
buscalls.WorkQueueDead: true, buscalls.LedgerApply: true, buscalls.LedgerGet: true, buscalls.LedgerHistory: true,
buscalls.LedgerQuery: true, buscalls.LedgerRebuild: true, buscalls.LedgerReindex: true, buscalls.LedgerSnapshot: true,
buscalls.LedgerLag: true, buscalls.ObjectsSentGet: true, buscalls.ObjectsPutChunk: true, buscalls.ObjectsGetChunk: true,
@@ -387,7 +387,29 @@ func Start(module, entry string, env []string, mesh Bus, logf func(string, ...an
asked := time.Now()
switch m.Method {
case "mesh/publish":
err = mesh.Publish(m.Params)
var said struct {
Ack bool `json:"ack"`
}
_ = json.Unmarshal(m.Params, &said)
if !said.Ack {
err = mesh.Publish(m.Params)
break
}
// With ack the runtime answers only once the bus acknowledged the event: its sequence, or {error}
// (design 47 G §559). A refusal is an answer, so a writer says it as a finding's resend.
acker, ok := mesh.(interface {
PublishAcked(params json.RawMessage) (uint64, error)
})
switch {
case !ok:
result = map[string]any{"error": "this tool runner's bus publishes no event with an acknowledgement"}
default:
if seq, perr := acker.PublishAcked(m.Params); perr != nil {
result = map[string]any{"error": perr.Error()}
} else {
result = map[string]any{"sequence": seq}
}
}
case "mesh/ask":
var asked json.RawMessage
if asked, err = mesh.Ask(m.Params); err == nil {
@@ -404,6 +426,38 @@ func Start(module, entry string, env []string, mesh Bus, logf func(string, ...an
result = json.RawMessage(raw)
}
case "mesh/subscribe":
var said struct {
Pattern string `json:"pattern"`
Deliver string `json:"deliver"`
}
_ = json.Unmarshal(m.Params, &said)
if said.Deliver == "new" {
// An ordered consumer of its own from the next message on (design 47 G §559, A-153): each message
// is handed to this child as an event is, and the consumer ends with the child.
newer, ok := mesh.(interface {
SubscribeNew(pattern string, deliver func(json.RawMessage) error) (func(), error)
})
if !ok {
err = errors.New("this tool runner's bus makes no subscription for new messages")
break
}
var stop func()
stop, err = newer.SubscribeNew(said.Pattern, func(envelope json.RawMessage) error {
_, err := c.ask(module, "mesh/event", map[string]any{"envelope": envelope}, EventTimeout)
return err
})
if err == nil {
c.mu.Lock()
if c.why != nil {
c.mu.Unlock()
stop()
} else {
c.watches = append(c.watches, stop)
c.mu.Unlock()
}
}
break
}
mu.Lock()
subscriber = c
mu.Unlock()
+60
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
@@ -68,6 +69,65 @@ func TestABundleTakesASeatsWorkThroughItsOwnChannel(t *testing.T) {
if err != nil || string(got) != `{"taken":true}` {
t.Fatalf("the work was not handed to the bundle: %s %v", got, err)
}
// Design 47 G §576, §518: a bundle reads the runner's capability words with mesh/capabilities, then runs its put session
// through the runner's put-chunk, and the sessions of its channel are dropped when the channel closes.
t.Run("ABundleReadsTheCapabilityWordsRunsAPutSessionAndItsChannelsClosingDropsThem", func(t *testing.T) {
dir := t.TempDir()
entry := filepath.Join(dir, "bundle")
if err := os.WriteFile(entry, []byte(puttingBundle), 0o755); err != nil {
t.Fatal(err)
}
b := &sessionBus{dropped: make(chan struct{})}
l, err := Start("putter", entry, os.Environ(), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.dropped:
case <-time.After(5 * time.Second):
t.Fatal("the bundle's sessions were not dropped when its channel closed")
}
b.mu.Lock()
defer b.mu.Unlock()
if len(b.chunks) != 1 || !strings.HasPrefix(b.chunks[0], "mesh/objects.put-chunk ") || !strings.Contains(b.chunks[0], `"seq":1`) {
t.Fatalf("the runner took %v", b.chunks)
}
})
// Design 47 G §559: `mesh/publish` with `ack` is answered with its sequence once the bus acknowledged it, or `{error}`;
// `mesh/subscribe` with `deliver: new` makes a subscription for new messages that ends with the bundle's channel.
t.Run("APublishWithAckIsAnsweredWithItsSequenceAndASubscriptionForNewMessagesEndsWithTheChannel", func(t *testing.T) {
dir := t.TempDir()
entry, out := filepath.Join(dir, "bundle"), filepath.Join(dir, "out")
if err := os.WriteFile(entry, []byte(findingBundle), 0o755); err != nil {
t.Fatal(err)
}
b := &findingBus{stopped: make(chan struct{})}
l, err := Start("writer", entry, append(os.Environ(), "FINDING_OUT="+out), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.stopped:
case <-time.After(5 * time.Second):
t.Fatal("the subscription for new messages was not ended with the bundle's channel")
}
got, _ := os.ReadFile(out)
for _, want := range []string{`"sequence":42`, `"error":"the bus refused it"`} {
if !strings.Contains(string(got), want) {
t.Errorf("the bundle was answered %s, want it to hold %s", got, want)
}
}
b.mu.Lock()
defer b.mu.Unlock()
if b.pattern != "mesh-conditions.findings-asked" {
t.Errorf("the subscription was made for %q", b.pattern)
}
if len(b.published) != 0 {
t.Errorf("a publish with ack went the unacknowledged way: %v", b.published)
}
})
}
// A bundle that pulls one ask from its worker when it is listed.
+36 -26
View File
@@ -2,12 +2,9 @@ package launch
import (
"encoding/json"
"os"
"path/filepath"
"errors"
"strings"
"sync"
"testing"
"time"
)
// puttingBundle asks the capability words, and once they are answered with put-sessions sends the one chunk of a put
@@ -41,28 +38,41 @@ func (b *sessionBus) BusState(method string, params json.RawMessage) (json.RawMe
}
func (b *sessionBus) DropSessions() { close(b.dropped) }
// Design 47 G §576, §518: a bundle reads the runner's capability words with mesh/capabilities, then runs its put session
// through the runner's put-chunk, and the sessions of its channel are dropped when the channel closes.
func TestABundleReadsTheCapabilityWordsRunsAPutSessionAndItsChannelsClosingDropsThem(t *testing.T) {
dir := t.TempDir()
entry := filepath.Join(dir, "bundle")
if err := os.WriteFile(entry, []byte(puttingBundle), 0o755); err != nil {
t.Fatal(err)
// findingBundle asks, as mesh-sdk's finding writer does, a publish with ack that the bus takes, one it refuses, and a
// subscription for new messages; each answer it gets is written to $FINDING_OUT, and it ends after the subscription's.
const findingBundle = `#!/bin/sh
while IFS= read -r line; do
id=$(printf '%s' "$line" | sed -n 's/.*"id":\([0-9]*\)[,}].*/\1/p')
case "$line" in
*'"method":"initialize"'*) printf '{"jsonrpc":"2.0","id":%s,"result":{}}\n' "$id" ;;
*'"method":"tools/list"'*) printf '{"jsonrpc":"2.0","id":%s,"result":{"tools":[]}}\n' "$id"
printf '{"jsonrpc":"2.0","id":"a1","method":"mesh/publish","params":{"key":"taken","ack":true,"body":{}}}\n' ;;
*'"id":"a1"'*) printf '%s\n' "$line" >>"$FINDING_OUT"
printf '{"jsonrpc":"2.0","id":"a2","method":"mesh/publish","params":{"key":"refused","ack":true,"body":{}}}\n' ;;
*'"id":"a2"'*) printf '%s\n' "$line" >>"$FINDING_OUT"
printf '{"jsonrpc":"2.0","id":"s1","method":"mesh/subscribe","params":{"pattern":"mesh-conditions.findings-asked","deliver":"new"}}\n' ;;
*'"id":"s1"'*) printf '%s\n' "$line" >>"$FINDING_OUT"; exit 0 ;;
esac
done
`
type findingBus struct {
subscribing
mu sync.Mutex
pattern string
stopped chan struct{}
}
func (b *findingBus) PublishAcked(params json.RawMessage) (uint64, error) {
if strings.Contains(string(params), `"refused"`) {
return 0, errors.New("the bus refused it")
}
b := &sessionBus{dropped: make(chan struct{})}
l, err := Start("putter", entry, os.Environ(), b, func(string, ...any) {})
if err != nil {
t.Fatal(err)
}
t.Cleanup(l.Stop)
select {
case <-b.dropped:
case <-time.After(5 * time.Second):
t.Fatal("the bundle's sessions were not dropped when its channel closed")
}
return 42, nil
}
func (b *findingBus) SubscribeNew(pattern string, _ func(json.RawMessage) error) (func(), error) {
b.mu.Lock()
defer b.mu.Unlock()
if len(b.chunks) != 1 || !strings.HasPrefix(b.chunks[0], "mesh/objects.put-chunk ") || !strings.Contains(b.chunks[0], `"seq":1`) {
t.Fatalf("the runner took %v", b.chunks)
}
b.pattern = pattern
b.mu.Unlock()
return func() { close(b.stopped) }, nil
}
@@ -0,0 +1,14 @@
package launch
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// A-42: a launched bundle may ask mesh/stream.publish.
func TestABundleMayAskStreamPublish(t *testing.T) {
if !busStateCalls[buscalls.StreamPublish] {
t.Fatal("mesh/stream.publish is not among the methods a bundle may ask")
}
}
+79
View File
@@ -58,6 +58,10 @@ type History interface {
// Read is the operations of the instances keys names (a key, with `*` for one name and `>` for the rest) from a
// sequence on, oldest first, at most limit of them (every one when limit is 0).
Read(keys string, from uint64, limit int) ([]Kept, error)
// Purge removes every operation of the instance key names, and nothing of another (an instance's removal).
Purge(key string) error
// Fill is the history's fill against its cap, the ledger's cap (buscalls.LedgerFill; design 47 G §587).
Fill() (buscalls.Fill, error)
}
// State is a ledger's state bucket: each instance's record under its key.
@@ -69,6 +73,8 @@ type State interface {
Write(key string, value []byte, revision uint64) (uint64, error)
// Keys is every key the bucket holds.
Keys() ([]string, error)
// Delete removes the key; a key it does not hold is no fault.
Delete(key string) error
}
// Snapshots are a ledger's snapshots: the latest record of each instance, on a subject of its own, never expiring.
@@ -77,6 +83,8 @@ type Snapshots interface {
Get(key string) ([]byte, uint64, error)
// Put writes the key's snapshot, expecting its last to be at expect (0: none); ErrRaced when it is not.
Put(key string, value []byte, expect uint64) error
// Purge removes the key's snapshot; one it does not hold is no fault.
Purge(key string) error
}
// Ledger is one ledger of one module: its types and where it is kept.
@@ -495,3 +503,74 @@ func (l *Ledger) Apply(r buscalls.ApplyRequest, author buscalls.Author) (buscall
}
return buscalls.Applied{Taken: true, Sequence: seq, StateBehind: behind}, nil
}
// Fill is the fill of the ledger's history against the ledger's cap (buscalls.LedgerFill; design 47 G §587): its messages,
// its bytes and the cap in bytes. The history holds every operation, and the cap is its alone: neither the fullest of the
// stores nor their sum is answered. A history that cannot say its fill is an error naming the ledger, never a fill of
// nothing.
func (l *Ledger) Fill(buscalls.LedgerRequest) (buscalls.Fill, error) {
f, err := l.History.Fill()
if err != nil {
return buscalls.Fill{}, fmt.Errorf("%s's ledger %s cannot say the fill of its history: %w", l.Module, l.Name, err)
}
return f, nil
}
// Remove removes one instance with everything it kept (buscalls.LedgerRemove; design 47 G §587, §155): its operations, its
// snapshot, its key in the state bucket and its keys in every declared index, and answers how many operations and how many
// bytes of the history that was. An instance the ledger does not hold answers 0 and 0, and a removal that left something
// (said, and answered as an error) is made again by calling it again, which removes the rest.
func (l *Ledger) Remove(r buscalls.LedgerRemoveRequest) (buscalls.LedgerRemoved, error) {
key, err := KeyOf(r.Instance)
if err != nil {
return buscalls.LedgerRemoved{}, err
}
out := buscalls.LedgerRemoved{Instance: r.Instance}
before, err := l.History.Fill()
if err != nil {
return out, fmt.Errorf("%s's ledger %s cannot say the fill of its history, so a removal is not made: %w", l.Module, l.Name, err)
}
ops, err := l.History.Read(key, 1, 0)
if err != nil {
return out, err
}
for _, step := range []struct {
what string
remove func(string) error
}{{"operations", l.History.Purge}, {"snapshot", l.Snapshots.Purge}, {"state", l.State.Delete}, {"index entries", l.dropFromIndexes}} {
if err := step.remove(key); err != nil {
l.say("[mesh-tools] %s's ledger %s removes %s and its %s could not be removed: %v; removing it again removes what "+
"is left", l.Module, l.Name, r.Instance, step.what, err)
return out, fmt.Errorf("%s's ledger %s removed part of %s and its %s are left: %w", l.Module, l.Name, r.Instance, step.what, err)
}
}
after, err := l.History.Fill()
if err != nil {
return out, fmt.Errorf("%s's ledger %s removed %s and cannot say the fill of its history after: %w", l.Module, l.Name, r.Instance, err)
}
out.Operations = uint64(len(ops))
if before.Bytes > after.Bytes {
out.Bytes = before.Bytes - after.Bytes
}
return out, nil
}
// dropFromIndexes removes the entries of the instance key names from every index bucket the ledger is bound to.
func (l *Ledger) dropFromIndexes(key string) error {
instance := InstanceOf(key)
for name, ix := range l.Indexes {
held, err := ix.Entries("", 0)
if err != nil {
return fmt.Errorf("the index %s: %w", name, err)
}
for _, e := range held {
var v buscalls.IndexEntry
if json.Unmarshal(e.Value, &v) == nil && v.Instance == instance {
if err := ix.Delete(e.Key); err != nil {
return fmt.Errorf("the index %s: %w", name, err)
}
}
}
}
return nil
}
+46
View File
@@ -99,4 +99,50 @@ func TestApplyAppendsDerivesAndSnapshots(t *testing.T) {
c.run(t, l, f)
})
}
// design 47 G §587: a ledger's fill is its history's against the ledger's cap, a store that cannot say it is an error naming
// the ledger; a removal takes an instance's operations, snapshot, state and index entries and answers what that was, an
// unknown instance answers none, and nothing of another instance goes.
t.Run("TheLedgerSaysItsFillAndRemovesAnInstanceWhole", func(t *testing.T) {
l, f := newLedger()
f.history.MaxBytes = 1 << 20
for _, n := range []string{"a", "b"} {
applied(t, l, "decision/0290", "note", `{"says":"`+n+`"}`, owner)
}
applied(t, l, "decision/0291", "note", `{"says":"other"}`, owner)
entry, _ := json.Marshal(buscalls.IndexEntry{Instance: "decision/0290", Sequence: 2})
other, _ := json.Marshal(buscalls.IndexEntry{Instance: "decision/0291", Sequence: 1})
_ = f.index.Put("t.decision.0290", entry)
_ = f.index.Put("t.decision.0291", other)
fill, err := l.Fill(buscalls.LedgerRequest{Ledger: "records"})
if err != nil || fill.Count != 3 || fill.Bytes == 0 || fill.MaxBytes != 1<<20 {
t.Fatalf("the fill is %+v (%v), want 3 messages against 1 MiB", fill, err)
}
got, err := l.Remove(buscalls.LedgerRemoveRequest{Ledger: "records", Instance: "decision/0290"})
if err != nil || got.Instance != "decision/0290" || got.Operations != 2 || got.Bytes == 0 {
t.Fatalf("the removal answered %+v (%v), want 2 operations and their bytes", got, err)
}
if kept, _ := f.history.Read("decision.0290", 1, 0); len(kept) != 0 {
t.Errorf("operations are left: %d", len(kept))
}
if raw, _, _ := f.state.Get("decision.0290"); raw != nil {
t.Errorf("the state is left: %s", raw)
}
if raw, _, _ := f.snapshots.Get("decision.0290"); raw != nil {
t.Errorf("the snapshot is left: %s", raw)
}
if held, _ := f.index.Entries("", 0); len(held) != 1 || held[0].Key != "t.decision.0291" {
t.Errorf("the index holds %v, want the other instance's entry alone", held)
}
if kept, _ := f.history.Read("decision.0291", 1, 0); len(kept) != 1 {
t.Errorf("another instance lost operations: %d left", len(kept))
}
if again, err := l.Remove(buscalls.LedgerRemoveRequest{Ledger: "records", Instance: "decision/0290"}); err != nil || again.Operations != 0 || again.Bytes != 0 {
t.Errorf("a removal said twice answered %+v (%v), want none", again, err)
}
f.history.FillFails = true
if _, err := l.Fill(buscalls.LedgerRequest{}); err == nil || !strings.Contains(err.Error(), "ledger records cannot say the fill") {
t.Errorf("a history that cannot say its fill answered %v", err)
}
})
}
+50 -3
View File
@@ -1,9 +1,12 @@
package ledger
import (
"errors"
"sort"
"strings"
"sync"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// The memory stores keep what the bus keeps, as the bus answers it, for a test of a ledger without a bus: a stream whose
@@ -12,9 +15,13 @@ import (
// MemoryHistory is a history in memory.
type MemoryHistory struct {
mu sync.Mutex
seq uint64
kept []struct {
// MaxBytes is the cap its fill is said against.
MaxBytes uint64
// FillFails makes Fill an error, as a store that cannot say its fill.
FillFails bool
mu sync.Mutex
seq uint64
kept []struct {
key string
Kept
}
@@ -73,6 +80,32 @@ func (h *MemoryHistory) Read(keys string, from uint64, limit int) ([]Kept, error
return out, nil
}
func (h *MemoryHistory) Purge(key string) error {
h.mu.Lock()
defer h.mu.Unlock()
kept := h.kept[:0]
for _, k := range h.kept {
if k.key != key {
kept = append(kept, k)
}
}
h.kept = kept
return nil
}
func (h *MemoryHistory) Fill() (buscalls.Fill, error) {
h.mu.Lock()
defer h.mu.Unlock()
if h.FillFails {
return buscalls.Fill{}, errors.New("the history does not say its fill")
}
f := buscalls.Fill{Count: uint64(len(h.kept)), MaxBytes: h.MaxBytes}
for _, k := range h.kept {
f.Bytes += uint64(len(k.Data))
}
return f, nil
}
// Expire drops every operation of the history, as a ledger's keeping days end them.
func (h *MemoryHistory) Expire() {
h.mu.Lock()
@@ -129,6 +162,13 @@ func (s *MemoryState) Keys() ([]string, error) {
return out, nil
}
func (s *MemoryState) Delete(key string) error {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.kv, key)
return nil
}
// MemorySnapshots are snapshots in memory.
type MemorySnapshots struct {
mu sync.Mutex
@@ -157,6 +197,13 @@ func (s *MemorySnapshots) Put(key string, value []byte, expect uint64) error {
return nil
}
func (s *MemorySnapshots) Purge(key string) error {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.kv, key)
return nil
}
// MemoryIndex is an index's bucket in memory.
type MemoryIndex struct {
mu sync.Mutex
+2 -2
View File
@@ -23,7 +23,7 @@ import (
assigner "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
seatassigner "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
meshcatalog "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
meshcatalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/membership"
"github.com/nats-io/nats-server/v2/server"
@@ -156,7 +156,7 @@ func sdkMembership(t *testing.T, mem bus.Membership) membership.Membership {
// no holder).
const (
catalogHolder = "mesh-catalog"
assignerHolder = "mesh-delivery-assigner"
assignerHolder = "mesh-module-assigner" // the seat's name stands for its holder in the console's state names (firstview.AssignmentsRead)
// AssignmentsBucket and SeatHoldersBucket are the assigner's states assignments and seat-holders.
AssignmentsBucket = assignerHolder + "_assignments"
SeatHoldersBucket = assignerHolder + "_seat-holders"
+62 -1
View File
@@ -2,11 +2,15 @@ package runtime
import (
"encoding/json"
"os"
"slices"
"sort"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/micro"
@@ -121,6 +125,7 @@ func TestTheRuntimeAnnouncesWhatItServesInTheServicesProtocol(t *testing.T) {
// issue 1966 TLS-3 (ENG-2): the runner states the capabilities of the calls it answers in its PING, on its own instance, which
// is what its machine's node-engine asks and takes into the node's record; no bundle is needed for it.
func TestTheRunnerStatesTheCapabilitiesOfItsCallsInItsPing(t *testing.T) {
t.Setenv(FingerprintWord, "sha256:runner")
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
@@ -139,7 +144,63 @@ func TestTheRunnerStatesTheCapabilitiesOfItsCallsInItsPing(t *testing.T) {
t.Fatalf("ping of the instance: %v", err)
}
var ping micro.Ping
if err := json.Unmarshal(msg.Data, &ping); err != nil || ping.Metadata[announce.Capabilities] != "approval-mark/1,bus-state-calls,ledger-calls,put-sessions,seat-settle-delay" {
if err := json.Unmarshal(msg.Data, &ping); err != nil || ping.Metadata[announce.Capabilities] != "approval-mark/1,bus-state-calls,finding-calls,ledger-calls,ledger-upkeep-calls,put-sessions,seat-settle-delay,sent-objects-calls,stream-publish" {
t.Errorf("the runner states %q (%v), want its bus-state and ledger calls", ping.Metadata[announce.Capabilities], err)
}
// Its own fingerprint beside them (design 46 approval mark §6).
if ping.Metadata[announce.Fingerprint] != "sha256:runner" {
t.Errorf("the runner states its fingerprint as %q", ping.Metadata[announce.Fingerprint])
}
// Design 47 G §462, §488, §622: each call is stated under the capability word of its own family, the words mesh-sdk's buscalls
// names: the read of a sent object under sent-objects-calls (no longer under bus-state-calls), the ledger's fill and removal
// under ledger-upkeep-calls, the others where they were.
t.Run("EachCallIsStatedUnderTheWordOfItsFamily", func(t *testing.T) {
for method, word := range map[string]string{
buscalls.ObjectsSentGet: buscalls.SentObjectsCalls,
buscalls.LedgerFill: buscalls.LedgerUpkeepCalls,
buscalls.LedgerRemove: buscalls.LedgerUpkeepCalls,
buscalls.LedgerApply: buscalls.LedgerCalls,
buscalls.ObjectsPut: buscalls.BusStateCalls,
buscalls.StreamPublish: buscalls.StreamPublishWord,
} {
var in []string
for c, methods := range callFamilies {
if slices.Contains(methods, method) {
in = append(in, c)
}
}
if len(in) != 1 || in[0] != word {
t.Errorf("%s is stated under %v, want %s alone", method, in, word)
}
}
})
// Design 47 G §545: the words the tool runner states are exactly its manifest's artifact `states`, so the delivery plan's edge
// of §605 runs to it for each word a module's artifact `calls`.
t.Run("TheWordsTheRunnerStatesAreItsManifestsStates", func(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Build struct {
Artifacts []struct {
Name string `json:"name"`
States []string `json:"states"`
} `json:"artifacts"`
} `json:"build"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
var declared []string
for _, a := range m.Build.Artifacts {
if a.Name == "runtime" {
declared = a.States
}
}
slices.Sort(declared)
if got := CallCapabilities(); !slices.Equal(got, declared) {
t.Errorf("the runner states %v, its manifest's states are %v", got, declared)
}
})
}
+23 -7
View File
@@ -18,28 +18,39 @@ import (
// (announce.Capabilities) for its machine's node-engine to take into the node's record. A module whose manifest asks for one is
// held off a machine whose runner does not state it.
const (
CapBusStateCalls = "bus-state-calls"
CapLedgerCalls = "ledger-calls"
CapBusStateCalls = buscalls.BusStateCalls
CapLedgerCalls = buscalls.LedgerCalls
// CapLedgerUpkeepCalls is the ledger's fill and removal (§462, §488) and CapSentObjectsCalls the read of a sent object (§488):
// each is stated once the runner answers its family, the words mesh-sdk's buscalls names (§622).
CapLedgerUpkeepCalls = buscalls.LedgerUpkeepCalls
CapSentObjectsCalls = buscalls.SentObjectsCalls
// CapPutSessions is the put and get sessions' word (design 47 G §518, §622), mesh-sdk's.
CapPutSessions = stdio.CapPutSessions
// CapApprovalMark says the runner honours the approval mark: its gate sits on every handler it serves (design 46
// 7-approval-mark §6, §20), so the grants part composes no deny of its modules' marked subjects.
CapApprovalMark = "approval-mark/1"
// CapStreamPublish is the call that says an event on the module's own bound stream (design 42 zfs-owns-zfs, A-44), the
// word mesh-sdk's buscalls names (§622).
CapStreamPublish = buscalls.StreamPublishWord
)
// callFamilies are the methods each capability stands for, the ones BusState and BusStateWatch answer.
var callFamilies = map[string][]string{
CapBusStateCalls: {buscalls.ObjectsPut, buscalls.ObjectsGet, buscalls.ObjectsList, buscalls.ObjectsDelete, buscalls.ObjectsInfo,
buscalls.ObjectsWatch, buscalls.ObjectsSentGet, buscalls.StreamRead, buscalls.StreamFill, buscalls.WorkQueueSize, buscalls.WorkQueueDead},
buscalls.ObjectsWatch, buscalls.StreamRead, buscalls.StreamFill, buscalls.WorkQueueSize, buscalls.WorkQueueDead},
CapStreamPublish: {buscalls.StreamPublish},
CapLedgerCalls: {buscalls.LedgerApply, buscalls.LedgerGet, buscalls.LedgerHistory, buscalls.LedgerWatch, buscalls.LedgerQuery,
buscalls.LedgerRebuild, buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag},
CapPutSessions: {buscalls.ObjectsPutChunk, buscalls.ObjectsGetChunk},
CapLedgerUpkeepCalls: {buscalls.LedgerFill, buscalls.LedgerRemove},
CapSentObjectsCalls: {buscalls.ObjectsSentGet},
CapPutSessions: {buscalls.ObjectsPutChunk, buscalls.ObjectsGetChunk},
}
// CallCapabilities is every capability of the calls this runner answers, sorted. The delayed settle of a pulled seat
// message names a fate, not a method, so it stands beside the method families (novox/hq ADR 0664 §5).
// message names a fate, not a method, so it stands beside the method families (novox/hq ADR 0664 §5); so does `finding-calls`,
// the acknowledged `mesh/publish` and the `mesh/subscribe` for new messages the launch answers (design 47 G §559).
func CallCapabilities() []string {
out := []string{stdio.CapSeatSettleDelay, CapApprovalMark}
out := []string{stdio.CapSeatSettleDelay, stdio.CapFindingCalls, CapApprovalMark}
for c := range callFamilies {
out = append(out, c)
}
@@ -103,6 +114,11 @@ func (b *moduleBus) BusState(method string, params json.RawMessage) (json.RawMes
if err = decode(method, params, &r); err == nil {
answer, err = conn.StreamFill(b.module, r)
}
case buscalls.StreamPublish:
var r buscalls.StreamPublishRequest
if err = decode(method, params, &r); err == nil {
answer, err = conn.StreamPublish(b.module, r)
}
case buscalls.WorkQueueSize:
var r buscalls.WorkQueueRequest
if err = decode(method, params, &r); err == nil {
@@ -114,7 +130,7 @@ func (b *moduleBus) BusState(method string, params json.RawMessage) (json.RawMes
answer, err = conn.WorkQueueDead(b.module, r)
}
case buscalls.LedgerApply, buscalls.LedgerGet, buscalls.LedgerHistory, buscalls.LedgerQuery, buscalls.LedgerRebuild,
buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag:
buscalls.LedgerReindex, buscalls.LedgerSnapshot, buscalls.LedgerLag, buscalls.LedgerFill, buscalls.LedgerRemove:
answer, err = b.ledgerCall(method, params)
default:
return nil, fmt.Errorf("the runtime answers no %s", method)
+69 -1
View File
@@ -6,6 +6,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"github.com/nats-io/nats.go"
@@ -121,9 +122,76 @@ func TestTheSubjectThatNamesNoCallerStillWorksAndNamesNone(t *testing.T) {
{map[string]seatdef.Definition{"node-desk": {Seat: "node-desk", Verbs: []seatdef.Verb{{Name: "who", Class: seatdef.ClassAct}}}}, ""},
} {
defs := c.defs
err := (&consumers{definitions: func() map[string]seatdef.Definition { return defs }}).admitted("node-desk", "who", "shanks.node-tools")
err := (&consumers{definitions: func() map[string]seatdef.Definition { return defs }}).admitted("node-desk", "who", "workstation.node-tools")
if c.says == "" && err != nil || c.says != "" && (err == nil || !strings.Contains(err.Error(), c.says)) {
t.Errorf("a proven caller with the definitions %v: %v, want %q", c.defs, err, c.says)
}
}
// Design 47 G A-249: a bundle's seat call is received by the runner in chunks only for a verb whose definition says `chunks`,
// of any machine: a verb a node holds is asked of the node that ran the ask (A-254).
t.Run("OnlyAChunkedSeatVerbIsAskedInChunks", func(t *testing.T) {
c := &consumers{definitions: func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"mesh-catalog": {Verbs: []seatdef.Verb{{Name: "input", Chunks: true}, {Name: "entry"}}}}
}}
for key, want := range map[string]bool{
"seat:mesh-catalog.input": true,
"seat:mesh-catalog.entry": false,
"seat:mesh-catalog.input@home-server": true,
"seat:mesh-catalog.entry@home-server": false,
"seat:mesh-unread.input": false,
"mesh-delivery-indexer.indexer": false,
} {
if got := c.chunked(key); got != want {
t.Errorf("chunked(%s) = %v, want %v", key, got, want)
}
}
if (&consumers{}).chunked("seat:mesh-catalog.input") {
t.Error("a runner that holds no seat definitions asks no chunks")
}
})
// **The tool runner answers a held seat's marked verb by caller and serves the seat's `approvals`** (design 47 G §677; lead
// answers A-265, A-271), with zsh's node-login-shell `execute` as the case: the marked verb reaches the gate with the caller
// its subject names, and `approvals` is answered from the runner's own record of the calls its gate holds, with the
// arguments for the call's own caller only, though no bundle names the verb.
t.Run("TheRunnerServesApprovalsForTheSeatsItsBundlesHold", func(t *testing.T) {
mesh := mt.New(t)
mesh.Issue(t, mt.MembershipOf("zsh", "anchor", true, map[string][]string{"node-login-shell": {"execute", seatdef.ApprovalsVerb}}))
nodeTools := connect(t, "node-tools", "anchor")
gate := &gateHolding{}
defined := WithSeatDefinitions(func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"node-login-shell": {Seat: "node-login-shell", Verbs: []seatdef.Verb{
{Name: "execute", NeedsApproval: true}, {Name: seatdef.ApprovalsVerb}}}}
})
stop, err := Run(nodeTools, []Served{{"zsh", []string{mt.Fixture("login-shell-execute.py")}}}, nil, func(string, ...any) {}, defined, WithGate(gate))
if err != nil {
t.Fatal(err)
}
defer stop()
asker, err := bus.Connect(bus.Credential{URL: mt.URL(t), Module: "node-tools", Node: "workstation", User: "workstation.node-tools", Password: "unused"})
if err != nil {
t.Fatal(err)
}
asker.Logf = func(string, ...any) {}
defer asker.Close()
mt.Until(t, func() error {
_, err := asker.Ask("seat:node-login-shell.execute@anchor", map[string]any{"command": "ls"}, "")
return err
})
if len(gate.calls) != 1 || gate.calls[0].Seat != "node-login-shell" || gate.calls[0].Name != "execute" || gate.calls[0].Caller != "workstation.node-tools" {
t.Fatalf("the marked verb reached the gate as %+v, want the seat's execute by workstation.node-tools", gate.calls)
}
var got approval.Approvals
mt.Until(t, func() error {
answered, err := asker.Ask("seat:node-login-shell.approvals@anchor", map[string]any{}, "")
if err == nil {
err = json.Unmarshal(answered.Result, &got)
}
return err
})
if len(got.Calls) != 1 || got.Calls[0].Caller != "workstation.node-tools" || got.Calls[0].Verb != "node-login-shell.execute" ||
string(got.Calls[0].Arguments) != `{"command":"ls"}` {
t.Fatalf("approvals answered %+v, want the one call the gate holds, with its caller's arguments", got)
}
})
}
@@ -1,29 +0,0 @@
package runtime
import (
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Design 47 G A-249: a bundle's seat call is received by the runner in chunks only for a verb whose definition says `chunks`,
// of no one machine.
func TestOnlyAChunkedSeatVerbIsAskedInChunks(t *testing.T) {
c := &consumers{definitions: func() map[string]seatdef.Definition {
return map[string]seatdef.Definition{"mesh-catalog": {Verbs: []seatdef.Verb{{Name: "input", Chunks: true}, {Name: "entry"}}}}
}}
for key, want := range map[string]bool{
"seat:mesh-catalog.input": true,
"seat:mesh-catalog.entry": false,
"seat:mesh-catalog.input@ace": false,
"seat:mesh-unread.input": false,
"mesh-delivery-indexer.indexer": false,
} {
if got := c.chunked(key); got != want {
t.Errorf("chunked(%s) = %v, want %v", key, got, want)
}
}
if (&consumers{}).chunked("seat:mesh-catalog.input") {
t.Error("a runner that holds no seat definitions asks no chunks")
}
}
+13 -2
View File
@@ -19,6 +19,8 @@ type Gate interface {
Unserved(module, fingerprint string, registered []string) []string
// Run is a call through the gate: run is called only when the call may run now, once.
Run(call approval.Call, run func() (any, error)) (any, error)
// Approvals is the seat verb `approvals` for a caller: the calls the gate holds, its acts and claims (A-265, A-271).
Approvals(caller string) (approval.Approvals, error)
}
// gatedRun is how a handler calls a tool: through the gate where there is one.
@@ -33,6 +35,14 @@ func (c *consumers) gated(call approval.Call, t launch.Tool, caller string) (any
return c.gate.Run(call, run)
}
// approvalsOf is how the runner answers a held seat's verb `approvals`: from its gate's record, nil without a gate.
func (r *runner) approvalsOf() func(caller string) (any, error) {
if r.events.gate == nil {
return nil
}
return func(caller string) (any, error) { return r.events.gate.Approvals(caller) }
}
// WithGate serves every call through the approval gate.
func WithGate(g Gate) Option { return func(c *consumers) { c.gate = g } }
@@ -94,8 +104,9 @@ func (c *consumers) chunked(key string) bool {
if c.definitions == nil || !strings.HasPrefix(key, "seat:") {
return false
}
seat, verb, ok := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
if !ok || strings.Contains(verb, "@") {
name, _, _ := strings.Cut(key, "@") // a verb asked of the machine that ran the ask is chunked as the verb is (A-254)
seat, verb, ok := strings.Cut(strings.TrimPrefix(name, "seat:"), ".")
if !ok {
return false
}
for _, v := range c.definitions()[seat].Verbs {
@@ -0,0 +1,26 @@
package runtime
import (
"git.novox.be/novox/mesh-sdk/go/approval"
)
// gateHolding is the runner's gate as far as the runtime reaches it: it records the calls it is run for and answers
// `approvals` from them.
type gateHolding struct{ calls []approval.Call }
func (g *gateHolding) Unserved(string, string, []string) []string { return nil }
func (g *gateHolding) Run(c approval.Call, run func() (any, error)) (any, error) {
g.calls = append(g.calls, c)
return run()
}
func (g *gateHolding) Approvals(caller string) (approval.Approvals, error) {
out := approval.Approvals{Calls: []approval.ListedCall{}, Acts: []approval.ListedAct{}, Claims: []string{}}
for i, c := range g.calls {
l := approval.ListedCall{Call: string(rune('a' + i)), Caller: c.Caller, Verb: c.Seat + "." + c.Name, State: "waiting"}
if caller == c.Caller {
l.Arguments = c.Args
}
out.Calls = append(out.Calls, l)
}
return out, nil
}
@@ -8,6 +8,7 @@ import (
"errors"
"fmt"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
@@ -240,6 +241,35 @@ func TestTheLedgerKeepsItsPromisesOnTheBus(t *testing.T) {
t.Fatalf("queried %+v, %v", q, err)
}
}},
{"fill and removal are answered through the calls and remove an instance from the bus whole", "decision.gone", func(t *testing.T) {
must(t, open(t), "decision/gone", "title", `{"title":"Gone"}`)
must(t, open(t), "decision/gone", "note", `{"says":"two"}`)
must(t, open(t), "decision/stays", "note", `{"says":"kept"}`)
raw, err := m.BusState(buscalls.LedgerFill, json.RawMessage(`{"ledger":"records"}`))
var fill buscalls.Fill
if _ = json.Unmarshal(raw, &fill); err != nil || fill.Count < 3 || fill.Bytes == 0 {
t.Fatalf("the fill answered %s (%v)", raw, err)
}
raw, err = m.BusState(buscalls.LedgerRemove, json.RawMessage(`{"ledger":"records","instance":"decision/gone"}`))
var removed buscalls.LedgerRemoved
if _ = json.Unmarshal(raw, &removed); err != nil || removed.Operations != 2 || removed.Bytes == 0 {
t.Fatalf("the removal answered %s (%v)", raw, err)
}
l := open(t)
if last, _ := l.History.Last("decision.gone"); last != 0 {
t.Fatalf("operations are left, at %d", last)
}
if st, _, _ := l.State.Get("decision.gone"); st != nil {
t.Fatalf("the state is left: %s", st)
}
if last, _ := l.History.Last("decision.stays"); last == 0 {
t.Fatal("another instance lost its operations")
}
if raw, err = m.BusState(buscalls.LedgerRemove, json.RawMessage(`{"ledger":"records","instance":"decision/gone"}`)); err != nil ||
!strings.Contains(string(raw), `"operations":0`) {
t.Fatalf("a removal said twice answered %s (%v)", raw, err)
}
}},
{"a forged author is refused", "decision.forged", func(t *testing.T) {
_, err := m.BusState(buscalls.LedgerApply, json.RawMessage(`{"ledger":"records","instance":"decision/forged","operation":"title","body":{"title":"T","author":"op@shanks"},"call":"forged"}`))
if err == nil || !strings.Contains(err.Error(), "the call forged, which is not a tool call in flight") {
@@ -258,4 +288,70 @@ func TestTheLedgerKeepsItsPromisesOnTheBus(t *testing.T) {
}
})
}
// Design 47 G §559, A-153: against the real bus a publish with ack answers the sequence the bus gave it, and a subscription
// for new messages hears only what is published after it was made, on the subject mesh-sdk's one rule names for the event;
// a wildcard is refused.
t.Run("AnAckedPublishAnswersItsSequenceAndANewSubscriptionHearsOnlyWhatComesAfter", func(t *testing.T) {
mt.New(t)
nc, err := nats.Connect(mt.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(nc.Close)
js, _ := jetstream.New(nc)
ctx := context.Background()
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "SEATEVENTS", Subjects: []string{"mesh.seat.mesh-conditions.event.>"}}); err != nil {
t.Fatal(err)
}
conn := connect(t, "node-tools", "anchor")
publish := func(body string) {
t.Helper()
if _, err := js.Publish(ctx, "mesh.seat.mesh-conditions.event.findings-asked", []byte(body)); err != nil {
t.Fatal(err)
}
}
publish(`{"before":true}`)
var mu sync.Mutex
var heard []string
stop, err := conn.SubscribeNew("mesh-conditions.findings-asked", func(raw json.RawMessage) error {
var env bus.Envelope
_ = json.Unmarshal(raw, &env)
mu.Lock()
heard = append(heard, env.Key+" "+string(env.Body))
mu.Unlock()
return nil
})
if err != nil {
t.Fatal(err)
}
defer stop()
publish(`{"after":true}`)
publish(`{"after":2}`)
mt.Until(t, func() error {
mu.Lock()
defer mu.Unlock()
if len(heard) < 2 {
return context.DeadlineExceeded
}
return nil
})
mu.Lock()
if len(heard) != 2 || heard[0] != `mesh-conditions.findings-asked {"after":true}` {
t.Errorf("heard %v, want the messages after the subscription alone, in order", heard)
}
mu.Unlock()
for name, want := range map[string]string{"mesh-conditions.*": "[A-Za-z0-9_-]"} {
if _, err := conn.SubscribeNew(name, func(json.RawMessage) error { return nil }); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("a subscription for %s answered %v, want one naming %q", name, err, want)
}
}
first, err := conn.PublishAsAcked("node-tools", bus.Envelope{Key: "finding-written", Body: json.RawMessage(`{}`)})
second, err2 := conn.PublishAsAcked("node-tools", bus.Envelope{Key: "finding-written", Body: json.RawMessage(`{}`)})
if err != nil || err2 != nil || first == 0 || second != first+1 {
t.Errorf("acknowledged publishes answered %d (%v) and %d (%v), want consecutive sequences", first, err, second, err2)
}
})
}
+11
View File
@@ -171,6 +171,17 @@ func (b *moduleBus) ledgerCall(method string, params json.RawMessage) (any, erro
var r buscalls.LedgerRequest
_ = json.Unmarshal(params, &r)
return l.Lag(r)
case buscalls.LedgerFill:
var r buscalls.LedgerRequest
_ = json.Unmarshal(params, &r)
return l.Fill(r)
case buscalls.LedgerRemove:
// Never on an apply: a removal is no operation of the history and has no author (design 47 G §587).
var r buscalls.LedgerRemoveRequest
if err := decode(method, params, &r); err != nil {
return nil, err
}
return l.Remove(r)
}
return nil, fmt.Errorf("the runtime answers no %s", method)
}
+1 -6
View File
@@ -4,7 +4,6 @@ import (
"context"
"encoding/json"
"errors"
"slices"
"sort"
"strings"
"testing"
@@ -205,13 +204,9 @@ func TestABundleToolNamedAsALedgerRepairIsRefused(t *testing.T) {
// no other name; none answers "the runtime answers no ...".
func TestEveryCallOfAStatedCapabilityIsAnswered(t *testing.T) {
m, _, _ := aLedger(t)
if got := strings.Join(CallCapabilities(), ","); got != "approval-mark/1,bus-state-calls,ledger-calls,put-sessions,seat-settle-delay" {
if got := strings.Join(CallCapabilities(), ","); got != "approval-mark/1,bus-state-calls,finding-calls,ledger-calls,ledger-upkeep-calls,put-sessions,seat-settle-delay,sent-objects-calls,stream-publish" {
t.Fatalf("the runner states %s", got)
}
// Design 47 G §462: the call that reads a sent object is listed under the bus state capability.
if !slices.Contains(callFamilies[CapBusStateCalls], buscalls.ObjectsSentGet) {
t.Errorf("%s does not list %s", CapBusStateCalls, buscalls.ObjectsSentGet)
}
for capability, methods := range callFamilies {
for _, method := range methods {
t.Run(capability+" "+method, func(t *testing.T) {
@@ -36,6 +36,25 @@ func TestNoBusWordReachesABundle(t *testing.T) {
t.Errorf("%s did not reach the bundle", want)
}
}
// design 47 G §618, §519: the runtime's own fingerprint reaches no bundle; the fingerprint a module's composed words hold does.
t.Run("TheRuntimesOwnFingerprintReachesNoBundle", func(t *testing.T) {
base := []string{"MESH_FINGERPRINT=runner-build", "PATH=/usr/bin"}
for name, c := range map[string]struct {
given map[string]string
want string
}{
"composed with none sees none": {nil, ""},
"composed with one sees its own": {map[string]string{"MESH_FINGERPRINT": "module-build"}, "MESH_FINGERPRINT=module-build"},
} {
env := strings.Join(BundleEnv(base, c.given, "telegram", "anchor"), "\n")
if strings.Contains(env, "runner-build") {
t.Errorf("%s: the runtime's own fingerprint reached the bundle: %s", name, env)
}
if c.want == "" && strings.Contains(env, "MESH_FINGERPRINT") || !strings.Contains(env, c.want) {
t.Errorf("%s: the environment is %s", name, env)
}
}
})
}
// A tool call reaches only a tool's subject: whatever key a caller names, it is never a seat's event, accept
+58 -6
View File
@@ -62,6 +62,10 @@ type ListedTool struct {
// NodeRuntimeModule is the module that is a machine's runtime, serving every module on it.
const NodeRuntimeModule = "node-tools"
// fingerprintEnv is the word the node-engine starts each process with, the fingerprint of the want it applies (mesh-sdk's
// process.FingerprintEnv; this package imports no SDK): the runtime's own is no bundle's.
const fingerprintEnv = "MESH_FINGERPRINT"
// runtimeOnly are the words of the runtime's own environment no bundle is given.
var runtimeOnly = map[string]bool{ToolEnv: true, ToolModules: true, "MESH_BROKER_FILE": true,
"MESH_BROKER_URL": true, "MESH_CONSOLE_LISTEN": true, LedgersFile: true}
@@ -306,7 +310,8 @@ func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, log
}
logf("%s", line)
conn.OnMembership(func(bus.Membership) { go func() { r.refreshRepairs(); conn.Flush() }() })
stopSeats, again := serveSeats(conn, r.modules, r.registrations, r.events.gated, r.events.admitted, r.fingerprintOf, logf)
stopSeats, again := serveSeats(conn, r.modules, r.registrations, r.events.gated, r.events.admitted, r.events.chunked, r.approvalsOf(),
r.fingerprintOf, logf)
r.seatsAgain = again
r.stops = append(r.stops, stopSeats)
@@ -316,7 +321,10 @@ func Run(conn *bus.Conn, served []Served, envs map[string]map[string]string, log
announced, err := announce.Serve(conn, announce.Service{
Name: conn.Module(), ID: instanceOf(conn),
Description: "the mesh's tool runtime on " + node + ": every assigned module's tools and the seats they hold",
Metadata: map[string]string{"node": node, announce.Capabilities: strings.Join(CallCapabilities(), ",")},
// Its own fingerprint beside the capabilities, so a reading is tied to the runner that made it (design 46 approval
// mark §6, lead answer A-203).
Metadata: map[string]string{"node": node, announce.Capabilities: strings.Join(CallCapabilities(), ","),
announce.Fingerprint: os.Getenv(FingerprintWord)},
}, func() []announce.Endpoint { return endpointsOf(conn, r.own(), r.registrations(), r.modules) })
if err != nil {
logf("[mesh-tools] cannot announce what it serves: %v", err)
@@ -655,6 +663,9 @@ func endpointsOf(conn *bus.Conn, own []registration, registrations []registratio
}
for _, v := range m.Seats {
t, ok := impl[v.Seat][v.Verb]
if v.Verb == seatdef.ApprovalsVerb && !ok {
t, ok = launch.Tool{Name: v.Verb, Description: "The calls this seat's holder holds for the operator's approval, its acts and its claims."}, true
}
if !ok || have[v.Subject] {
continue
}
@@ -710,7 +721,8 @@ func subjectsOf(conn *bus.Conn, module, tool string) []string {
// whenever a membership changes, and when again is called because a bundle was loaded again. Whether
// this machine holds the seat is the bus's to decide. It answers its stop and again.
func serveSeats(conn *bus.Conn, modules []string, current func() []registration, gated gatedRun,
admitted func(seat, verb, caller string) error, fingerprintOf func(string) string, logf func(string, ...any)) (func(), func()) {
admitted func(seat, verb, caller string) error, chunked func(key string) bool, approvals func(caller string) (any, error),
fingerprintOf func(string) string, logf func(string, ...any)) (func(), func()) {
var mu sync.Mutex
var stops []func()
stopped := false
@@ -744,6 +756,24 @@ func serveSeats(conn *bus.Conn, modules []string, current func() []registration,
continue
}
have[v.Subject] = true
// The seat's verb `approvals` is the gate's own, served wherever a marked verb is: the runner answers it from its
// record of the calls its gate holds, by caller, and no bundle names it (lead answers A-265, A-266, A-271).
if v.Verb == seatdef.ApprovalsVerb && approvals != nil {
seat := v.Seat
stop, err := conn.HandleSubjectCalled(v.Subject, func(caller string, _ json.RawMessage) (any, error) {
if err := admitted(seat, seatdef.ApprovalsVerb, caller); err != nil {
return nil, err
}
return approvals(caller)
})
if err != nil {
logf("[mesh-tools] cannot serve %s's approvals on %s: %v", seat, v.Subject, err)
continue
}
stops = append(stops, stop)
logf("[mesh-tools] serving %s's approvals on %s, the gate's own verb", seat, v.Subject)
continue
}
t, ok := impl[v.Seat][v.Verb]
if !ok {
logf("[mesh-tools] %s claims %s and implements no %s, which that seat promises; not served", module, v.Seat, v.Verb)
@@ -756,8 +786,13 @@ func serveSeats(conn *bus.Conn, modules []string, current func() []registration,
if err := admitted(v.Seat, v.Verb, caller); err != nil {
return nil, err
}
return gated(approval.Call{Address: v.Seat + "." + v.Verb, Module: module, Seat: v.Seat, Name: v.Verb,
answer, err := gated(approval.Call{Address: v.Seat + "." + v.Verb, Module: module, Seat: v.Seat, Name: v.Verb,
Fingerprint: fingerprintOf(module), Caller: caller, Args: argsOf(body), Description: t.Description}, t, caller)
if err != nil || !chunked("seat:"+v.Seat+"."+v.Verb) {
return answer, err
}
// A verb marked chunks: the bundle answers the whole file, the runner sends it in chunks (design 47 G A-257).
return conn.AnswerChunked(v.Seat, v.Verb, body, answer)
})
if err != nil {
logf("[mesh-tools] cannot serve %s's %s on %s: %v", v.Seat, v.Verb, v.Subject, err)
@@ -902,6 +937,21 @@ func (b *moduleBus) Publish(params json.RawMessage) error {
return b.all.conn.PublishAs(b.module, env)
}
// PublishAcked is Publish that answers the sequence the bus acknowledged the event at (`mesh/publish` with `ack`, design 47 G §559).
func (b *moduleBus) PublishAcked(params json.RawMessage) (uint64, error) {
var env bus.Envelope
if err := json.Unmarshal(params, &env); err != nil {
return 0, fmt.Errorf("not an event envelope: %w", err)
}
return b.all.conn.PublishAsAcked(b.module, env)
}
// SubscribeNew hands the bundle one event's messages published from now on, on an ordered consumer of its own (`mesh/subscribe`
// with `deliver: new`, design 47 G §559); the consumer ends with the bundle's channel.
func (b *moduleBus) SubscribeNew(pattern string, deliver func(json.RawMessage) error) (func(), error) {
return b.all.conn.SubscribeNew(pattern, deliver)
}
// Ask calls a tool as the module: `{key, body}`, answered with the tool's result (ADR 0198).
func (b *moduleBus) Ask(params json.RawMessage) (json.RawMessage, error) {
var asked struct {
@@ -1403,11 +1453,13 @@ func (b *moduleBus) bind(key string, stop func()) {
// BundleEnv is what one module's bundle is started with: the runtime's environment without its own words,
// the module's composed words over it, and the module's and machine's names. **No bus word reaches a
// bundle** (security review of 2026-10-08): the credential and the bus's address are the runtime's, and a
// bundle reaches the bus only through the runtime.
// bundle reaches the bus only through the runtime. **The runtime's own fingerprint reaches none either**: a bundle reads
// only the `MESH_FINGERPRINT` its module's composed words hold, the build the node-engine applied for that module, and with
// none it has none (design 47 G §519, §618).
func BundleEnv(base []string, given map[string]string, module, node string) []string {
words := map[string]string{}
for _, kv := range base {
if k, v, ok := strings.Cut(kv, "="); ok && !runtimeOnly[k] {
if k, v, ok := strings.Cut(kv, "="); ok && !runtimeOnly[k] && k != fingerprintEnv {
words[k] = v
}
}
+44 -2
View File
@@ -4,10 +4,12 @@ import (
"encoding/json"
"errors"
"os"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/approval"
"github.com/novox/mesh-tools/node-tools/internal/bus"
"github.com/novox/mesh-tools/node-tools/internal/launch"
mt "github.com/novox/mesh-tools/node-tools/internal/meshtest"
@@ -192,10 +194,50 @@ func TestEachBundleIsGivenItsOwnEnvironment(t *testing.T) {
}
}
// An entrypoint that is not executable is refused by name, and the others serve (ADR 0193).
// unservingGate is an approval gate whose marks of each module name the tools given, so a bundle not registering them is
// refused at the load (design 46 7-approval-mark §4).
type unservingGate map[string][]string
func (g unservingGate) Unserved(module, _ string, registered []string) []string {
var missing []string
for _, tool := range g[module] {
if !slices.Contains(registered, tool) {
missing = append(missing, tool)
}
}
return missing
}
func (unservingGate) Run(_ approval.Call, run func() (any, error)) (any, error) { return run() }
func (unservingGate) Approvals(string) (approval.Approvals, error) { return approval.Approvals{}, nil }
// An entrypoint that is not executable is refused by name, and the others serve (ADR 0193). A bundle whose marks name a
// tool it does not register is refused at the load, serving no tool of its module, and one that registers it serves
// (design 46 7-approval-mark §4).
func TestAnEntrypointThatIsNotExecutableIsRefused(t *testing.T) {
mt.NeedBundles(t)
mesh := mt.New(t)
{
mesh.Issue(t, mt.MembershipOf("marked", "anchor", true, nil))
mesh.Issue(t, mt.MembershipOf("served", "anchor", true, nil))
nodeTools := connect(t, "node-tools", "anchor")
asker := connect(t, "console", "workstation")
logs := &mt.Logs{}
stop, err := Run(nodeTools, []Served{{"marked", []string{mt.Fixture("caller-echo.py")}}, {"served", []string{mt.Fixture("caller-echo.py")}}},
nil, logs.Logf, WithGate(unservingGate{"marked": {"wipe"}, "served": {"who"}}))
if err != nil {
t.Fatal(err)
}
if !logs.Has("marked's bundle", "failed to load: its marks name wipe, which it does not register (design 46 7-approval-mark §4)") {
t.Errorf("a bundle not serving its marked tool was not refused at the load:\n%s", logs.All())
}
if _, err := call(t, asker, "served.who@anchor", map[string]any{}); err != nil {
t.Errorf("a bundle serving its marked tool: %v", err)
}
if _, err := asker.Ask("marked.who@anchor", map[string]any{}, ""); !errors.Is(err, bus.ErrNoResponders) {
t.Errorf("a refused bundle's tool answered: %v", err)
}
stop()
}
mt.NeedBundles(t)
mesh.Issue(t, mt.MembershipOf("alpha", "anchor", false, nil))
mesh.Issue(t, mt.MembershipOf("plain", "anchor", false, nil))
nodeTools := connect(t, "node-tools", "anchor")
+58 -60
View File
@@ -64,66 +64,6 @@ func asked(t *testing.T, b *moduleBus, method string, req any, into any) {
}
}
// Design 47 G §518, A-21, A-25: a put session writes each chunk as it is taken, carries the description of its first chunk,
// and answers the put's own info for the last; a chunk out of order fails the put naming both sequences and leaves none.
func TestAPutSessionSealsOnTheLastChunkAndDropsOnADisorder(t *testing.T) {
f := &fakeObjects{put: map[string][]byte{}}
b := aSession(t, f)
whole := bytes.Repeat([]byte("x"), 2*buscalls.ChunkBytes+5)
var taken buscalls.ObjectsChunkTaken
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(whole))
req := buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c1", Seq: seq, Last: end == len(whole), Data: whole[at:end]}
if seq == 1 {
req.Description = "what it is"
}
if req.Last {
var info buscalls.ObjectInfo
asked(t, b, buscalls.ObjectsPutChunk, req, &info)
if info.Size != uint64(len(whole)) || info.Description != "what it is" {
t.Fatalf("the put answered %+v", info)
}
break
}
asked(t, b, buscalls.ObjectsPutChunk, req, &taken)
if taken.Seq != seq || taken.Error != "" {
t.Fatalf("chunk %d was answered %+v", seq, taken)
}
at = end
}
if !bytes.Equal(f.put["a"], whole) {
t.Fatalf("the store holds %d bytes, want %d", len(f.put["a"]), len(whole))
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 1, Data: []byte("one")}, &taken)
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 3, Data: []byte("three")}, &taken)
if !strings.Contains(taken.Error, "chunk 3") || !strings.Contains(taken.Error, "chunk 2") {
t.Fatalf("a disorder was answered %+v", taken)
}
if _, ok := f.put["b"]; ok || len(f.deleted) != 1 || f.deleted[0] != "b" {
t.Fatalf("a dropped put left %v, deleted %v", f.put, f.deleted)
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 2, Data: []byte("two")}, &taken)
if taken.Error == "" {
t.Fatal("a chunk of a dropped put was taken")
}
}
// §518 "The bound": a put not sealed within the bound is dropped, its object deleted.
func TestAPutNotSealedWithinTheBoundIsDropped(t *testing.T) {
was := SessionBound
SessionBound = time.Millisecond
t.Cleanup(func() { SessionBound = was })
f := &fakeObjects{put: map[string][]byte{}, ctxEnded: make(chan struct{})}
b := aSession(t, f)
var taken buscalls.ObjectsChunkTaken
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c", Seq: 1, Data: []byte("x")}, &taken)
<-f.ctxEnded
if len(f.deleted) != 1 {
t.Fatalf("the unsealed object was left: %v", f.deleted)
}
}
// §490, §518, §620: a value over one chunk is got as a session (the chunked answer, then each chunk by its call and sequence,
// the whole checked against the digest); a sent object over one chunk the same, with `chunked` true.
func TestAGetOverOneChunkRunsAsASession(t *testing.T) {
@@ -192,4 +132,62 @@ func TestAGetOverOneChunkRunsAsASession(t *testing.T) {
if string(small.Data) != "small" {
t.Fatalf("a small value was answered %+v", small)
}
// Design 47 G §518, A-21, A-25: a put session writes each chunk as it is taken, carries the description of its first chunk,
// and answers the put's own info for the last; a chunk out of order fails the put naming both sequences and leaves none.
t.Run("APutSessionSealsOnTheLastChunkAndDropsOnADisorder", func(t *testing.T) {
f := &fakeObjects{put: map[string][]byte{}}
b := aSession(t, f)
whole := bytes.Repeat([]byte("x"), 2*buscalls.ChunkBytes+5)
var taken buscalls.ObjectsChunkTaken
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(whole))
req := buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c1", Seq: seq, Last: end == len(whole), Data: whole[at:end]}
if seq == 1 {
req.Description = "what it is"
}
if req.Last {
var info buscalls.ObjectInfo
asked(t, b, buscalls.ObjectsPutChunk, req, &info)
if info.Size != uint64(len(whole)) || info.Description != "what it is" {
t.Fatalf("the put answered %+v", info)
}
break
}
asked(t, b, buscalls.ObjectsPutChunk, req, &taken)
if taken.Seq != seq || taken.Error != "" {
t.Fatalf("chunk %d was answered %+v", seq, taken)
}
at = end
}
if !bytes.Equal(f.put["a"], whole) {
t.Fatalf("the store holds %d bytes, want %d", len(f.put["a"]), len(whole))
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 1, Data: []byte("one")}, &taken)
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 3, Data: []byte("three")}, &taken)
if !strings.Contains(taken.Error, "chunk 3") || !strings.Contains(taken.Error, "chunk 2") {
t.Fatalf("a disorder was answered %+v", taken)
}
if _, ok := f.put["b"]; ok || len(f.deleted) != 1 || f.deleted[0] != "b" {
t.Fatalf("a dropped put left %v, deleted %v", f.put, f.deleted)
}
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "b", Call: "c2", Seq: 2, Data: []byte("two")}, &taken)
if taken.Error == "" {
t.Fatal("a chunk of a dropped put was taken")
}
})
// §518 "The bound": a put not sealed within the bound is dropped, its object deleted.
t.Run("APutNotSealedWithinTheBoundIsDropped", func(t *testing.T) {
was := SessionBound
SessionBound = time.Millisecond
t.Cleanup(func() { SessionBound = was })
f := &fakeObjects{put: map[string][]byte{}, ctxEnded: make(chan struct{})}
b := aSession(t, f)
var taken buscalls.ObjectsChunkTaken
asked(t, b, buscalls.ObjectsPutChunk, buscalls.ObjectsPutChunkRequest{Objects: "files", Name: "a", Call: "c", Seq: 1, Data: []byte("x")}, &taken)
<-f.ctxEnded
if len(f.deleted) != 1 {
t.Fatalf("the unsealed object was left: %v", f.deleted)
}
})
}
@@ -0,0 +1,18 @@
package runtime
import (
"slices"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// A-44: mesh/stream.publish is a family of its own, the capability stream-publish, and no longer of bus-state-calls.
func TestStreamPublishIsACapabilityOfItsOwn(t *testing.T) {
if !slices.Equal(callFamilies[CapStreamPublish], []string{buscalls.StreamPublish}) || CapStreamPublish != "stream-publish" {
t.Fatal("stream-publish does not state only mesh/stream.publish")
}
if slices.Contains(callFamilies[CapBusStateCalls], buscalls.StreamPublish) {
t.Fatal("bus-state-calls still states mesh/stream.publish")
}
}
+14 -3
View File
@@ -58,7 +58,18 @@
"language": "go",
"system": "arch",
"from": "cmd/node-tools",
"binary": "node-tools"
"binary": "node-tools",
"states": [
"approval-mark/1",
"bus-state-calls",
"finding-calls",
"ledger-calls",
"ledger-upkeep-calls",
"put-sessions",
"seat-settle-delay",
"sent-objects-calls",
"stream-publish"
]
}
]
},
@@ -77,8 +88,8 @@
"mesh-catalog.seat-definition-changed",
"mesh-catalog.seat-definition-removed",
"mesh-conditions.findings-asked",
"mesh-delivery-assigner.views.assignments",
"mesh-delivery-assigner.views.seat-holders",
"seat:mesh-module-assigner.views.assignments",
"seat:mesh-module-assigner.views.seat-holders",
"mesh-delivery.source-moved"
],
"streams": [
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env python3
# The login shell's bundle as the seat node-login-shell has it: MCP over stdio, no SDK, one seat verb, execute, which says
# the command it was given and the caller the call's `_meta` named.
import json, sys
def say(m):
m["jsonrpc"] = "2.0"; sys.stdout.write(json.dumps(m) + "\n"); sys.stdout.flush()
TOOLS = [
{"name": "node-login-shell.execute", "description": "run a command", "inputSchema": {"type": "object", "properties": {"command": {"type": "string"}}}},
]
for line in sys.stdin:
req = json.loads(line); rid = req.get("id"); m = req.get("method"); p = req.get("params") or {}
if m == "initialize":
say({"id": rid, "result": {"protocolVersion": "2025-03-26", "capabilities": {"tools": {}}, "serverInfo": {"name": "zsh", "version": "1"}}})
elif m == "tools/list":
say({"id": rid, "result": {"tools": TOOLS}})
elif m == "tools/call":
meta = p.get("_meta") or {}
said = {"ran": (p.get("arguments") or {}).get("command", ""), "caller": meta.get("mesh/caller", "")}
say({"id": rid, "result": {"content": [{"type": "text", "text": json.dumps(said)}]}})
elif rid is not None:
say({"id": rid, "result": {}})