novox/mesh-sdk
18
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Refuse a key-value write too large for its bucket before it is put (issue 774)

Browse Source 
mesh-grants' sealed grants and the controller's merge record each met "message size exceeds maximum" (JetStream
10054) only after the put, and a writer that retries looped on a value that can never land; an update over the
limit was even answered as a revision mismatch, which every compare-and-set writer retries.

Quick fix: cap the two values in their modules. Thorough fix, chosen because the kind was met twice and every
module writes through the SDK: the package kvlimit checks a value against its bucket's largest message (read from
its stream and kept) before the put, naming the bucket, the key, the size and the limit, and turns the bus's own
refusal into the same *TooLarge (errors.Is kvlimit.ErrTooLarge), so no writer retries it. The SDK's own writers
use it: process.State (Put, Create, CreateTTL, Update), process.AwaitedBucket, the runtime state of a bundle
(stdio KeptState, against the module state's 256 KiB) and mesh-hand-compose. A value that can outgrow its bucket
is split over keys or kept in the object store with its name in the key.
This commit is contained in:
jschoubben
2026-10-30 18:02:18 +01:00
parent 59311407e2
commit 7b02f63daf
7 changed files with 247 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
go/cmd/mesh-hand-compose/main.go
+3 -2
View File
@@ -32,6 +32,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/kvlimit"
)
func main() {
@@ -190,7 +191,7 @@ func writeIt(url, creds, node, bucket, key string, encoded []byte, digest string
if err != nil {
return fmt.Errorf("the bucket %s: %w", bucket, err)
}
if _, err := composed.Put(ctx, key, encoded); err != nil {
if _, err := kvlimit.Put(ctx, composed, key, encoded); err != nil {
return fmt.Errorf("%s in %s: %w", key, bucket, err)
}
value, err := delivery.EncodeComposedDigest(delivery.ComposedDigest{Digest: digest, Judgement: delivery.JudgedCannot,
@@ -202,7 +203,7 @@ func writeIt(url, creds, node, bucket, key string, encoded []byte, digest string
if err != nil {
return fmt.Errorf("the bucket %s: %w", delivery.ControllerComposedBucket, err)
}
if _, err := digests.Put(ctx, key, value); err != nil {
if _, err := kvlimit.Put(ctx, digests, key, value); err != nil {
return fmt.Errorf("%s in %s: %w", key, delivery.ControllerComposedBucket, err)
}
return nil