Compare commits

...
5 Commits
Author SHA1 Message Date
mesh-admin 67dfeafd51 Merge pull request 'go/asks: an ask says what a channel that proves who answers shows whole, and what Details shows (issue 383)' (#17) from fix/383-an-ask-says-what-a-proving-channel-shows-whole into main 2026-10-10 13:50:56 +00:00
jochen 4f3efc98e0 go/asks: an ask says what a channel that proves who answers shows whole, and what Details shows (hq issue 383)
mesh/merge-gate pass: builds mesh-sdk, sent nowhere; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (typescript), go/merge-check.sh (go)
mesh/delivery delivered
A settings proposal reached the phone with its values masked, so the operator could not
tell what they were approving. An ask now carries Whole, shown in place of Explanation on
a kind that proves its sender and carries the ask's answers, and Details, what the
Details answer shows. Both enter the digest only when given, so an ask without them
digests as before and a router and an asker of different builds still agree on it.
2026-10-10 15:23:41 +02:00
mesh-admin 7605a3fa56 Merge pull request 'go: a counter by compare-and-set, so a number is handed out once (hq research 042)' (#16) from feat/a-counter-by-compare-and-set into main 2026-10-10 10:37:11 +00:00
jochen 9c548753b8 go: say an Increment error other than a lost write may have spent its number, refuse null, and prove the writers collided
mesh/repo-check pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (typescript), go/merge-check.sh (go)
mesh/delivery delivered
mesh/merge-gate pass: builds mesh-sdk, sent nowhere; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/delivery-group group feat/a-counter-by-compare-and-set merged: every member is merged
The review of 2026-10-10: a timeout can come after the write landed, so a caller tolerates gaps; a null value
is no counter; and a contention test where no write was lost would prove nothing.
2026-10-10 12:33:41 +02:00
jochen c58ce2c7fe go: a counter by compare-and-set, so a number is handed out once (hq research 042)
mesh/delivery-group group feat/a-counter-by-compare-and-set checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds mesh-sdk, sent nowhere; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (typescript), go/merge-check.sh (go)
The coming issues module hands out issue numbers from its own state; two instances must never hand out one
number twice. Create and Update already write by compare-and-set (ADR 0259), so State(...).Increment reads,
writes one more at the revision read, and on a lost write reads and tries again, bounded by the caller's
attempts. A lost write past the bound is ErrChanged, any other refusal and a value that is no counter are
returned at once: nothing is reported counted that was not written.

Part of the issue tracker decision, hq research 042.
2026-10-10 12:31:05 +02:00
5 changed files with 370 additions and 4 deletions
+5 -1
View File
@@ -96,7 +96,8 @@ an event under the module's own name or kind, `stdio.SeatTake` takes a held seat
and `stdio.SeatAnswer` carry a proof (a code typed by a person, by request and reply, never kept),
`stdio.SeatRecord` reads a record kept for the module, and `State(...).Create` and `.Update` write by
compare-and-set. The runtime does each only as far as the module's membership lists. `go/asks` is the
contract on top: the `Ask` an asker publishes and the checks it is held to, the `Warrant` it acts on
contract on top: the `Ask` an asker publishes and the checks it is held to (its `Whole` is what a channel that proves
who answers shows in place of its `Explanation`, and its `Details` what the Details answer shows, hq issue 383), the `Warrant` it acts on
(`For`, `Says`), and the `Message`, `Choice`, `LinkAsked` and `Code` a channel takes and says, with the
subjects each travels on.
`go/asker` is an asking module's client (ADR 0259 §10): `Ask` keeps an ask in the module's own state and then
@@ -104,6 +105,9 @@ publishes it under the module's name; `Cancel`; `Where` reads its state from the
`Take` hand the module each warrant once — marked acted on by compare-and-set before the module acts, and only
for an ask it made, an option it offered, at that option's level, before it expired. A lost compare-and-set is
`stdio.ErrChanged`, told by the runtime's error code.
`State(...).Increment(key, attempts)` is a counter on top of the two (for numbers handed out once, such as an
issue's): it reads the key, writes one more at the revision read, and on a lost write reads and tries again, at
most `attempts` times; past that it is `stdio.ErrChanged` and nothing was counted.
### `primitives` — sealing, semver, resolved-env
Seal/unseal for secrets, semantic version comparison, and the accessor a module uses to read its
+16 -1
View File
@@ -154,7 +154,9 @@ func canonical(b *strings.Builder, v string) {
//
// It is over the ask's named fields in a fixed order, each written canonically, and the expiry as UTC
// RFC 3339 to the nanosecond — never over a language's encoding of the struct, so a field added to Ask
// later changes no digest until it is added here, on purpose.
// later changes no digest until it is added here, on purpose. Whole and Details (novox/hq issue 383) follow
// the options only when the ask gives either: an ask without them digests as it did before they existed, so
// a router and an asker of different builds still agree on every such ask.
func (a Ask) Digest() string {
var b strings.Builder
b.WriteString("novox.ask.v1\n")
@@ -168,6 +170,10 @@ func (a Ask) Digest() string {
canonical(&b, v)
}
}
if a.Whole != "" || a.Details != "" {
canonical(&b, a.Whole)
canonical(&b, a.Details)
}
sum := sha256.Sum256([]byte(b.String()))
return "sha256:" + hex.EncodeToString(sum[:])
}
@@ -190,6 +196,15 @@ type Ask struct {
// About is what the ask is about (a condition's key): a newer ask about it replaces the older.
About string `json:"about,omitempty"`
Urgent bool `json:"urgent,omitempty"`
// Whole is the explanation with its exact values whole, shown in place of Explanation on a channel kind
// that proves who answers (verified-sender) and carries the ask's answers (novox/hq issue 383): what the
// person approves — a mount point, a share, a private address — must be readable where they approve it.
// The asker withholds a value shaped like a secret in it, and the router refuses the ask when one is left;
// Explanation stays under the whole content rule everywhere else. Empty, Explanation is shown everywhere.
Whole string `json:"whole,omitempty"`
// Details is what the Details answer on the ask's message shows, line by line under the content rule: a
// fingerprint, how to read the proposal whole at the terminal. Empty, an ask's own message offers no Details.
Details string `json:"details,omitempty"`
}
// The bounds of an ask (novox/hq ADR 0234 §8, ADR 0259 §4).
+24 -2
View File
@@ -230,9 +230,31 @@ func TestDigestsAreCanonicalOverNamedFields(t *testing.T) {
t.Errorf("changing %s left the digest the same", name)
}
}
// Whole and Details (issue 383) are in the digest once given, and change it; an ask without either digests
// exactly as before they existed — the encoding by hand, so no build can drift from it unseen.
plain := Ask{ID: "p", Headline: "h", Explanation: "e", Who: Operator, Expires: now, OnExpiry: "o", About: "a",
Options: []Option{{ID: "ok", Label: "OK", Does: "d", Level: Acknowledge}}}
byHand := sha256.Sum256([]byte("novox.ask.v1\n1:p\n1:h\n1:e\n8:operator\n20:2026-10-08T14:00:00Z\n1:o\n1:a\n5:false\n1:1\n" +
"2:ok\n2:OK\n1:d\n11:acknowledge\n0:\n"))
if plain.Digest() != "sha256:"+hex.EncodeToString(byHand[:]) {
t.Errorf("an ask without Whole and Details does not digest as it did before them: %s", plain.Digest())
}
withWhole, withDetails := plain, plain
withWhole.Whole = "e, whole"
withDetails.Details = "fingerprint 1234"
if withWhole.Digest() == plain.Digest() || withDetails.Digest() == plain.Digest() || withWhole.Digest() == withDetails.Digest() {
t.Error("Whole or Details left the digest the same")
}
both := withWhole
both.Details = "fingerprint 1234"
changed := both
changed.Whole += "!"
if changed.Digest() == both.Digest() {
t.Error("changing Whole beside Details left the digest the same")
}
// A field added to Ask or Option without being added to Digest fails here.
if n := reflect.TypeOf(Ask{}).NumField(); n != 9 {
t.Errorf("Ask has %d fields, and Digest covers 9: add the new one to Digest and to this test", n)
if n := reflect.TypeOf(Ask{}).NumField(); n != 11 {
t.Errorf("Ask has %d fields, and Digest covers 11: add the new one to Digest and to this test", n)
}
if n := reflect.TypeOf(Option{}).NumField(); n != 5 {
t.Errorf("Option has %d fields, and Digest covers 5: add the new one to Digest and to this test", n)
+47
View File
@@ -2,6 +2,8 @@ package stdio
import (
"encoding/json"
"errors"
"fmt"
"time"
)
@@ -74,6 +76,51 @@ func (s KeptState) Delete(key string) error {
return err
}
// Increment adds one to a counter kept at key and returns the number it now holds: one for a key with no
// value. Read, then written only if the key is still at the revision read (Create when it had none, Update
// when it had one), so of any number of writers each number is handed out once and no increment is lost.
// A write another writer won is read again and tried again, at most attempts times in all; past that the
// error is ErrChanged and nothing was counted. Any other error is returned at once, as is a value that is
// not a whole number (null included): nothing is ever reported counted that was not written. The reverse
// does not hold: an error that is not ErrChanged — a timeout above all — may come after the write landed, so
// that number may be spent and never handed out. A caller tolerates gaps, never a number handed out twice.
// Deleting the key starts the count again at one; a key whose numbers must stay unique is never deleted.
func (s KeptState) Increment(key string, attempts int) (uint64, error) {
if attempts < 1 {
return 0, fmt.Errorf("an increment of %s is tried at least once: %d attempts", key, attempts)
}
for i := 0; i < attempts; i++ {
e, err := s.Get(key)
if err != nil {
return 0, err
}
var next uint64 = 1
if e == nil {
_, err = s.Create(key, next)
} else {
var n uint64
if string(e.Value) == "null" {
return 0, fmt.Errorf("%s holds null, not a counter", key)
}
if err := json.Unmarshal(e.Value, &n); err != nil {
return 0, fmt.Errorf("%s holds %s, not a counter: %w", key, e.Value, err)
}
if n == ^uint64(0) {
return 0, fmt.Errorf("%s is at the largest number a counter holds", key)
}
next = n + 1
_, err = s.Update(key, next, e.Revision)
}
if err == nil {
return next, nil
}
if !errors.Is(err, ErrChanged) {
return 0, err
}
}
return 0, fmt.Errorf("%s was written by another writer first %d times: %w", key, attempts, ErrChanged)
}
// Keys is every key with a current value, sorted.
func (s KeptState) Keys() ([]string, error) {
raw, err := s.ask("keys", map[string]any{})
+278
View File
@@ -0,0 +1,278 @@
package stdio
import (
"bufio"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"sync"
"testing"
"time"
)
// casRuntime is the runtime's side of mesh/state.get, create and update as the bus does them (novox/hq ADR
// 0201, ADR 0259): one revision per write across the bucket, create refused when the key has a value,
// update refused unless the key is still at the revision named, each refusal answered with CodeChanged.
// refuse, when set, is asked before each write and may refuse it with an error of its own.
type casRuntime struct {
mu sync.Mutex
revision uint64
keys map[string]StateEntry
writes map[string]int
lost int
refuse func(verb string) (code int, message string, refused bool)
}
func (r *casRuntime) answer(m message) map[string]any {
var p struct {
Key string `json:"key"`
Value json.RawMessage `json:"value"`
Revision uint64 `json:"revision"`
}
_ = json.Unmarshal(m.Params, &p)
r.mu.Lock()
defer r.mu.Unlock()
verb := strings.TrimPrefix(m.Method, "mesh/state.")
lost := func() map[string]any {
r.lost++
return map[string]any{"error": map[string]any{"code": CodeChanged, "message": p.Key + ": the key was written by another writer first"}}
}
if verb != "get" && r.refuse != nil {
if code, msg, refused := r.refuse(verb); refused {
return map[string]any{"error": map[string]any{"code": code, "message": msg}}
}
}
e, has := r.keys[p.Key]
switch verb {
case "get":
if !has {
return map[string]any{"result": nil}
}
return map[string]any{"result": e}
case "create":
if has {
return lost()
}
case "update":
if p.Revision == 0 {
return map[string]any{"error": map[string]any{"code": -32000, "message": "an update names the revision it read"}}
}
if !has || e.Revision != p.Revision {
return lost()
}
case "put":
default:
return map[string]any{"error": map[string]any{"code": -32601, "message": "no " + m.Method}}
}
r.revision++
r.keys[p.Key] = StateEntry{Key: p.Key, Value: p.Value, Revision: r.revision}
r.writes[verb]++
return map[string]any{"result": map[string]any{"revision": r.revision}}
}
// counting serves a bundle whose one tool, "count", runs run, against r; it answers the bundle's asks each
// beside the loop, as the runtime does, so two writers' reads and writes interleave. It returns what the
// tool answered.
func counting(t *testing.T, r *casRuntime, run func() (any, error)) map[string]any {
t.Helper()
t.Setenv(ServedAs, "issues")
inR, inW := io.Pipe()
outR, outW := io.Pipe()
go func() {
_ = ServeOn("", []Tool{{Name: "count", Run: func(map[string]any) (any, error) { return run() }}}, inR, outW)
}()
var wmu sync.Mutex
send := func(m map[string]any) {
m["jsonrpc"] = "2.0"
b, _ := json.Marshal(m)
wmu.Lock()
defer wmu.Unlock()
_, _ = inW.Write(append(b, '\n'))
}
done := make(chan map[string]any, 1)
go func() {
lines := bufio.NewScanner(outR)
lines.Buffer(make([]byte, 1<<20), 1<<24)
for lines.Scan() {
var m message
if json.Unmarshal(lines.Bytes(), &m) != nil {
continue
}
if strings.HasPrefix(m.Method, "mesh/state.") {
go func(m message) {
a := r.answer(m)
a["id"] = m.ID
send(a)
}(m)
continue
}
var answer map[string]any
_ = json.Unmarshal(lines.Bytes(), &answer)
if answer["id"] == float64(1) {
done <- answer
return
}
}
}()
send(map[string]any{"id": 1, "method": "tools/call", "params": map[string]any{"name": "count"}})
t.Cleanup(func() { _ = inW.Close() })
select {
case a := <-done:
return a
case <-time.After(60 * time.Second):
t.Fatal("the bundle did not answer the count")
return nil
}
}
func text(t *testing.T, a map[string]any) (string, bool) {
t.Helper()
res, _ := a["result"].(map[string]any)
content, _ := res["content"].([]any)
if len(content) != 1 {
t.Fatalf("answered %v", a)
}
isError, _ := res["isError"].(bool)
return content[0].(map[string]any)["text"].(string), isError
}
// Two writers incrementing one counter a thousand times each end at two thousand: every number handed out
// once, none lost, each writer's numbers rising — read, then written only at the revision read.
func TestTwoWritersIncrementingOneCounterLoseNoUpdate(t *testing.T) {
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
const each = 1000
got := counting(t, r, func() (any, error) {
var wg sync.WaitGroup
numbers := make([][]uint64, 2)
errs := make([]error, 2)
for w := range numbers {
wg.Add(1)
go func(w int) {
defer wg.Done()
for i := 0; i < each; i++ {
n, err := State("numbers").Increment("issue", 1000)
if err != nil {
errs[w] = err
return
}
numbers[w] = append(numbers[w], n)
}
}(w)
}
wg.Wait()
if err := errors.Join(errs...); err != nil {
return nil, err
}
seen := map[uint64]bool{}
for w, ns := range numbers {
for i, n := range ns {
if seen[n] {
t.Errorf("the number %d was handed out twice", n)
}
seen[n] = true
if i > 0 && n <= ns[i-1] {
t.Errorf("writer %d was handed %d after %d", w, n, ns[i-1])
}
}
}
return len(seen), nil
})
if out, isError := text(t, got); isError || out != "2000" {
t.Fatalf("two writers counted %s (error: %v)", out, isError)
}
var final uint64
_ = json.Unmarshal(r.keys["issue"].Value, &final)
if final != 2*each {
t.Fatalf("the counter ended at %d, not %d: an increment was lost", final, 2*each)
}
if r.lost == 0 {
t.Fatal("the two writers never collided: the test proved nothing about contention")
}
t.Logf("2000 increments, %d writes lost to the other writer and tried again", r.lost)
if r.writes["create"] != 1 || r.writes["update"] != 2*each-1 || r.writes["put"] != 0 {
t.Fatalf("written by %v: one create, then updates at the revision read, never a put", r.writes)
}
}
// A lost write is read again and tried again, never reported counted; past its attempts it is ErrChanged
// with nothing counted; any other refusal, and a value that is no counter, is returned at once.
func TestAnIncrementThatLosesIsTriedAgainAndNeverReportedCounted(t *testing.T) {
t.Run("lost twice then won", func(t *testing.T) {
lost := 0
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`41`), Revision: 7}},
writes: map[string]int{}, revision: 7}
r.refuse = func(string) (int, string, bool) {
if lost < 2 {
lost++
// Another writer was first: the counter moved on by one, to 42 and then 43.
r.revision++
r.keys["issue"] = StateEntry{Key: "issue", Value: json.RawMessage(fmt.Sprint(41 + lost)), Revision: r.revision}
return CodeChanged, "issue: the key was written by another writer first", true
}
return 0, "", false
}
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 3) })
if out, isError := text(t, got); isError || out != "44" {
t.Fatalf("answered %s (error: %v), not 44", out, isError)
}
})
t.Run("lost every time", func(t *testing.T) {
tries := 0
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
r.refuse = func(string) (int, string, bool) {
tries++
return CodeChanged, "issue: the key was written by another writer first", true
}
var err error
got := counting(t, r, func() (any, error) {
n, e := State("numbers").Increment("issue", 5)
err = e
return n, e
})
if _, isError := text(t, got); !isError || !errors.Is(err, ErrChanged) || tries != 5 {
t.Fatalf("after %d tries: %v", tries, err)
}
})
t.Run("refused otherwise", func(t *testing.T) {
tries := 0
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
r.refuse = func(string) (int, string, bool) {
tries++
return -32000, "issues keeps no state numbers", true
}
var err error
got := counting(t, r, func() (any, error) {
n, e := State("numbers").Increment("issue", 5)
err = e
return n, e
})
if _, isError := text(t, got); !isError || errors.Is(err, ErrChanged) || tries != 1 {
t.Fatalf("after %d tries: %v", tries, err)
}
})
t.Run("no counter", func(t *testing.T) {
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`"x"`), Revision: 1}},
writes: map[string]int{}, revision: 1}
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 5) })
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
t.Fatalf("a value that is no counter was counted: %v, writes %v", got, r.writes)
}
})
t.Run("null", func(t *testing.T) {
r := &casRuntime{keys: map[string]StateEntry{"issue": {Key: "issue", Value: json.RawMessage(`null`), Revision: 1}},
writes: map[string]int{}, revision: 1}
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 5) })
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
t.Fatalf("a null was counted from zero: %v, writes %v", got, r.writes)
}
})
t.Run("no attempts", func(t *testing.T) {
r := &casRuntime{keys: map[string]StateEntry{}, writes: map[string]int{}}
got := counting(t, r, func() (any, error) { return State("numbers").Increment("issue", 0) })
if _, isError := text(t, got); !isError || len(r.writes) != 0 {
t.Fatalf("zero attempts counted: %v", got)
}
})
}