novox/mesh-controller
18
0
Fork 0
Code Issues Pull Requests 4 Releases Wiki Activity

Move the route proxy into its module and run it from the trust image, on mesh-sdk's public membership (issues 1633, 1634, 1635)
mesh/delivery-group group feat/r4d rejected: a member's own check failed
Details
mesh/repo-check out of date: judged beside mesh-catalog#434 at c948f3a61c74, it is now at 7451e7edf4e5; judged again on a recheck or when it is merged
Details
mesh/merge-gate group feat/r4d: composed again for the group's new heads
Details
mesh/delivery superseded: a newer head of the same pull request
Details
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…

Browse Source 
A bundle is built only from its module's directory (ADR 0480 clause 4), so the proxy leaves
examples/ and the controller's internal/broker; the broker keeps no second copy of what the
proxy shares with mesh-sdk. The image and its Dockerfile go, and a test holds that only
build-agent declares one (clause 11).
This commit is contained in:
jschoubben
2026-10-22 09:53:56 +02:00
parent 3d4ad6fefb
commit 499ff5c218
313 changed files with 89380 additions and 419 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/route-proxy/Dockerfile
-29
View File
@@ -1,29 +0,0 @@
ARG ALPINE_BASE=alpine:3.20
ARG GO_BASE=golang:1.25
# The route-proxy module's runtime image: the reference reverse proxy compiled into a container.
#
# **The proxy source sits beside this module.** The canonical proxy, the contract written as something
# that runs, lives in this same repository, mesh-controller, at examples/route-proxy (novox/hq 08-connectivity
# §3), and the repository is the build context (a module lives whole in one repository, novox/hq ADR 0420),
# so this Dockerfile compiles ./examples/route-proxy from the repository root. The build seat hands it only
# that program's build source, its import closure, go.mod, go.sum and the vendored packages it uses
# (novox/hq ADR 0267, the artifact's `compiles`), so a merge elsewhere in the repository does not
# move this image, and a COPY of anything outside it fails by name.
#
# docker build -f modules/route-proxy/Dockerfile -t mesh-route-proxy:development .
#
# The mesh pins the digest of what this produces; the committed module.json carries the placeholder
# digest every mesh-built image does, replaced at publish.
FROM ${GO_BASE} AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -buildvcs=false -ldflags=-buildid= -o /mesh-route-proxy ./examples/route-proxy
# A small runtime with the public CA roots the ACME client needs to reach a real authority, and run
# as root so it can bind :80 and :443 — the two privileged ports a public front door listens on.
FROM ${ALPINE_BASE}
RUN apk add --no-cache ca-certificates
COPY --from=build /mesh-route-proxy /usr/local/bin/mesh-route-proxy
ENTRYPOINT ["/usr/local/bin/mesh-route-proxy"]
modules/route-proxy/README.md
+8 -8
View File
@@ -45,15 +45,15 @@ pinned image is not moved without a decision; mesh-controller's
## How it ships the Go proxy
The proxy is a Go program, unlike the TypeScript tool-runtime modules. Its source sits in this same
repository, mesh-controller, at `examples/route-proxy`, the contract written as something that runs. This
module ships the packaging: a multi-stage [`Dockerfile`](Dockerfile) whose build context is the repository
root and which compiles `./examples/route-proxy` into `mesh-route-proxy`. The committed `module.json` carries the
placeholder digest every mesh-built image does (`@sha256:0000…`); the mesh pins the real digest at
publish.
The proxy is a Go program, unlike the TypeScript tool-runtime modules. Its source sits in this module, at
`cmd/route-proxy`, in the module's own Go module (`go.mod`, with a `vendor/` directory), the contract written as
something that runs. No image is built for it (novox/hq ADR 0480): the `server` artifact is a Go bundle compiled
for alpine, an `archive` resource unpacks it onto the machine, and the `route-proxy` container runs the pinned
`trust` image (the same alpine digest whose system roots the public account directory is named after) with the
archive mounted read-only at `/opt/mesh` and `/opt/mesh/route-proxy` as its program. A new build of the program
restarts the container, because the archive is in its `restart-on`.
In the lab, `mesh-lab/scripts/build-route-proxy-image.sh` builds this into the local daemon as
`mesh-route-proxy:development`, and a scenario stocks it and serves it by digest.
In the lab, a scenario stocks the same alpine image and serves it by digest.
## Certificates, and the endpoint (resolves 04-ISSUES/004)
modules/route-proxy/cmd/route-provider/provider_test.go
+3 -2
View File
@@ -77,7 +77,8 @@ func TestAConsumerOfARouteIsToldItsNamesFromItsPlanEntry(t *testing.T) {
// route-proxy's own files and resources stay as they were (issue 792): the module answers its route through the
// helper but is not marked moved, so its acme.env, its broker secret and its containers are composed as today and
// the public ingress does not change. Everything of the manifest but its build, what it provides and the state it
// keeps is pinned by this digest; changing the ingress means changing it here, on purpose.
// keeps is pinned by this digest; changing the ingress means changing it here, on purpose. Re-pinned in R4d (novox/hq
// ADR 0480): the program is an archive mounted into the trust image, the container and its files are otherwise as before.
func TestRouteProxyIsNotMovedAndKeepsItsOwnFiles(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
@@ -102,7 +103,7 @@ func TestRouteProxyIsNotMovedAndKeepsItsOwnFiles(t *testing.T) {
t.Fatal(err)
}
sum := sha256.Sum256(pinned)
if got := hex.EncodeToString(sum[:]); got != "57a39de0943d8b33b9db6cb998ce97f8793602cca6cdfa5849c1c392390454a3" {
if got := hex.EncodeToString(sum[:]); got != "85cb955e5a4b2d8826f50aa76433064a576e8b15b71537032eb93c7388c0ee28" {
t.Fatalf("route-proxy's own manifest changed: digest %s", got)
}
// The two acme templates are pinned the same way (issue 849): they are what the proxy's certificates are asked
modules/route-proxy/cmd/route-proxy/authority_test.go
+73
View File
@@ -0,0 +1,73 @@
package main
import (
"path/filepath"
"strings"
"testing"
)
// The lab's first root, and the one a re-initialised CA generates in its place.
const (
firstRoot = "-----BEGIN CERTIFICATE-----\nMIIBeFIRST\n-----END CERTIFICATE-----\n"
secondRoot = "-----BEGIN CERTIFICATE-----\nMIIBeSECOND\n-----END CERTIFICATE-----\n"
)
// A re-initialised CA does not need somebody to delete the cache by hand.
//
// autocert keeps its account key at one fixed name and reuses it for ever. When an internal CA is
// re-initialised it has never heard of that account, rejects every use of it, and autocert has no
// path back: nothing is re-registered, no order reaches the CA, and issuance stops with nothing
// saying why. Naming the cache after the authority means the account is only ever found where it is
// still valid — the new root lands in a directory with no account in it, and autocert registers.
func TestANewCARootMeansANewAccountCache(t *testing.T) {
const directory = "https://anchor.internal/acme/acme/directory"
before := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot))
after := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(secondRoot))
if before == after {
t.Fatalf("a re-initialised CA reuses the account it was rejected for: %s", before)
}
}
// And the SAME authority keeps the account it registered, restart after restart.
//
// This is the whole reason ACME_CACHE is required in the first place: an account and its
// certificates that did not persist would be re-ordered on every restart, which works silently until
// a rate limit says it does not. Whitespace around the delivered root is not a new authority — the
// mesh writes that file, and a trailing newline coming or going must not throw away an account.
func TestTheSameAuthorityKeepsItsAccount(t *testing.T) {
const directory = "https://anchor.internal/acme/acme/directory"
first := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot))
again := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte("\n"+firstRoot+"\n\n"))
if first != again {
t.Errorf("the same authority was given two caches, so every restart orders again:\n%s\n%s",
first, again)
}
}
// Staging and production are different authorities, and were sharing one account.
//
// The latent fault of the same shape: pointing ACME_DIRECTORY at production after testing against
// staging reused the staging account, because the cache had no idea they were different.
func TestStagingAndProductionDoNotShareAnAccount(t *testing.T) {
staging := forThisAuthority("/acme", stagingDirectory, nil)
production := forThisAuthority("/acme", "https://acme-v02.api.letsencrypt.org/directory", nil)
if staging == production {
t.Errorf("two issuers share one account: %s", staging)
}
}
// It stays inside the directory the mesh gave it, and is a plain name.
//
// The mesh owns ACME_CACHE and mounts it; a name derived from a certificate that escaped it — or
// that carried a separator out of the PEM — would put an account somewhere nothing persists.
func TestTheAccountCacheStaysWhereTheMeshPutIt(t *testing.T) {
const cache = "/var/lib/route-proxy/acme"
got := forThisAuthority(cache, "https://anchor.internal/acme/acme/directory", []byte(firstRoot))
if !strings.HasPrefix(got, cache+"/") {
t.Fatalf("the account cache is not under %s: %s", cache, got)
}
name := strings.TrimPrefix(got, cache+"/")
if name != filepath.Base(got) || strings.ContainsAny(name, "/.") {
t.Errorf("the account cache is not a plain name: %q", name)
}
}
modules/route-proxy/cmd/route-proxy/bus.go
+85
View File
@@ -0,0 +1,85 @@
package main
import (
"encoding/json"
"fmt"
"log"
"strings"
"sync/atomic"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/process"
)
// What the mesh issued this proxy, read on the bus (novox/hq ADR 0160, ADR 0167).
//
// **The proxy is told, not left to work it out.** Its membership carries the routes it is given —
// the same contributions its file is written from — and every machine's address on the private
// network, which is who may be served an internal name. Read once at connect and followed, so a
// route added or a machine joining reaches a running proxy without a restart.
// followMembership connects with the credential in path and applies every membership the mesh
// issues this proxy. It retries the first connection for as long as it takes: a proxy that started
// before the bus keeps serving the file, and takes the bus when it answers.
func followMembership(path string, held *table, fromBus *atomic.Bool) {
for {
err := followOnce(path, held, fromBus)
if err == nil {
return
}
log.Printf("cannot follow this proxy's membership, serving the file meanwhile: %v", err)
time.Sleep(30 * time.Second)
}
}
func followOnce(path string, held *table, fromBus *atomic.Bool) error {
account, err := process.ReadAccount(path)
if err != nil {
return err
}
conn, err := process.Connect(account, nats.Name(account.Node+"."+account.Module))
if err != nil {
return fmt.Errorf("connecting to the bus at %s: %w", membership.BareAddress(account.URL), err)
}
subject := membership.Subject(account.Node, account.Module)
err = conn.FollowMembership(func(issued membership.Membership) {
if took := applyMembership(issued, held); took && !fromBus.Swap(true) {
log.Printf("routes now come from this proxy's membership on %s", subject)
}
}, func(err error) { log.Print(err) })
if err != nil {
conn.Close()
return err
}
return nil
}
// applyMembership serves what a membership says, and says whether it said anything about routes.
//
// A membership with no routes in it is one from a controller older than ADR 0167, and the file stays
// the source rather than every route being withdrawn because a field was absent.
func applyMembership(issued membership.Membership, held *table) bool {
raw, carries := issued.Receives["route"]
if !carries {
return false
}
var contributions []contribution
if err := json.Unmarshal(raw, &contributions); err != nil {
log.Printf("the routes in this proxy's membership are not contributions, keeping what is served: %v", err)
return false
}
inside, err := sourcesOf(issued.Mesh)
if err != nil {
log.Printf("the mesh in this proxy's membership is unreadable, keeping what is served: %v", err)
return false
}
routes, public := routesOf(contributions)
held.set(routes, public)
held.setInside(inside)
log.Printf("serving %d route(s) from the membership, internal names to %d machine(s): %s",
len(routes), len(inside), strings.Join(held.names(), ", "))
return true
}
modules/route-proxy/cmd/route-proxy/challenge_test.go
+109
View File
@@ -0,0 +1,109 @@
package main
// The challenge path falls through for real. autocert's own HTTPHandler answers 404 itself for a
// token it does not hold and never consults its fallback on the challenge path — the
// predecessor's fault, the edge owning /.well-known/acme-challenge outright, rediscovered live
// when Mailu's renewal died behind this proxy on cutover day (2026-09-26). These tests pin the
// three behaviours tokenOrRoute exists for.
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"golang.org/x/crypto/acme/autocert"
)
func routedTo(t *testing.T, marker string) http.Handler {
t.Helper()
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
if _, err := w.Write([]byte(marker)); err != nil {
t.Fatal(err)
}
})
}
func TestATokenNoAuthorityHoldsIsRoutedNot404d(t *testing.T) {
m := &autocert.Manager{Prompt: autocert.AcceptTOS, Cache: autocert.DirCache(t.TempDir())}
h := tokenOrRoute(routedTo(t, "the workload answered"), m)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "http://mail.example/.well-known/acme-challenge/somebody-elses-token", nil))
if rec.Code != http.StatusOK || rec.Body.String() != "the workload answered" {
t.Fatalf("a token no authority holds must reach plain routing; got %d %q", rec.Code, rec.Body.String())
}
}
func TestATokenAManagerHoldsIsAnsweredByIt(t *testing.T) {
// autocert reads a token it does not have in memory from its cache, under "<token>+http-01" —
// which is also how a token would survive the manager restarting mid-issuance.
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "held-token+http-01"), []byte("the-key-authorization"), 0o600); err != nil {
t.Fatal(err)
}
m := &autocert.Manager{Prompt: autocert.AcceptTOS, Cache: autocert.DirCache(dir)}
h := tokenOrRoute(routedTo(t, "must not be reached"), m)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "http://mail.example/.well-known/acme-challenge/held-token", nil))
if rec.Code != http.StatusOK || rec.Body.String() != "the-key-authorization" {
t.Fatalf("the manager holding a token answers it; got %d %q", rec.Code, rec.Body.String())
}
}
func TestASecondAuthorityIsProbedBeforeRouting(t *testing.T) {
first := &autocert.Manager{Prompt: autocert.AcceptTOS, Cache: autocert.DirCache(t.TempDir())}
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "internal-token+http-01"), []byte("internal-key"), 0o600); err != nil {
t.Fatal(err)
}
second := &autocert.Manager{Prompt: autocert.AcceptTOS, Cache: autocert.DirCache(dir)}
h := tokenOrRoute(routedTo(t, "must not be reached"), first, second)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "http://git.internal/.well-known/acme-challenge/internal-token", nil))
if rec.Code != http.StatusOK || rec.Body.String() != "internal-key" {
t.Fatalf("the second authority's token is found by probing past the first; got %d %q", rec.Code, rec.Body.String())
}
}
func TestAnAuthorityWhosePolicyRefusesTheNameIsProbedPast(t *testing.T) {
// autocert checks the host policy before the token and answers 403 — the internal authority
// does this for every public name. A policy refusal is as much "not mine" as a missing token:
// the request must still reach plain routing, where the workload's own ACME client answers.
refusing := &autocert.Manager{
Prompt: autocert.AcceptTOS,
Cache: autocert.DirCache(t.TempDir()),
HostPolicy: func(ctx context.Context, host string) error {
return fmt.Errorf("no internal-only route for %q in this mesh", host)
},
}
h := tokenOrRoute(routedTo(t, "the workload answered"), refusing)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "http://mail.example/.well-known/acme-challenge/mailus-token", nil))
if rec.Code != http.StatusOK || rec.Body.String() != "the workload answered" {
t.Fatalf("a policy refusal must fall through to routing; got %d %q", rec.Code, rec.Body.String())
}
}
func TestAnOrdinaryPathNeverTouchesTheChallengeMachinery(t *testing.T) {
m := &autocert.Manager{Prompt: autocert.AcceptTOS, Cache: autocert.DirCache(t.TempDir())}
h := tokenOrRoute(routedTo(t, "routed"), m)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "http://site.example/index.html", nil))
if rec.Code != http.StatusOK || rec.Body.String() != "routed" {
t.Fatalf("an ordinary path goes straight to routing; got %d %q", rec.Code, rec.Body.String())
}
}
modules/route-proxy/cmd/route-proxy/forwarded_test.go
+29
View File
@@ -0,0 +1,29 @@
package main
import (
"crypto/tls"
"net/http"
"net/http/httptest"
"net/url"
"testing"
)
// A backend behind the proxy learns the client used TLS and which name it asked for, so the addresses
// it writes into its own pages are the ones a client can use (2026-10-03: a forge's Go import tag
// named an http clone URL, and Go refused the module path).
func TestABackendIsToldTheRequestWasHTTPSAndForWhichName(t *testing.T) {
var proto, host, fwdHost, fwdFor string
backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proto, host, fwdHost, fwdFor = r.Header.Get("X-Forwarded-Proto"), r.Host, r.Header.Get("X-Forwarded-Host"), r.Header.Get("X-Forwarded-For")
}))
defer backend.Close()
where, _ := url.Parse(backend.URL)
req := httptest.NewRequest(http.MethodGet, "https://git.example.org/novox/mesh-sdk/go?go-get=1", nil)
req.TLS = &tls.ConnectionState{}
req.Host = "git.example.org"
req.RemoteAddr = "192.0.2.7:51000"
towards(where).ServeHTTP(httptest.NewRecorder(), req)
if proto != "https" || fwdHost != "git.example.org" || host != "git.example.org" || fwdFor != "192.0.2.7" {
t.Errorf("the backend was told proto=%q host=%q forwarded-host=%q for=%q", proto, host, fwdHost, fwdFor)
}
}
modules/route-proxy/cmd/route-proxy/main.go
+1076
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/cmd/route-proxy/policy_test.go
+273
View File
@@ -0,0 +1,273 @@
package main
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"golang.org/x/crypto/bcrypt"
)
// What a route carries about the requests arriving at it — novox/hq ADR 0108.
//
// Each test here is one of the four capabilities that record closed the set at, plus the negative
// case it promised would be refused. The negative case is the one that rots quietly: nothing fails
// if it stops working, so nothing tells you it has.
// served starts a workload and gives back the host and port the mesh would have recorded for it.
func served(t *testing.T, body string) (string, int) {
t.Helper()
workload := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(body))
}))
t.Cleanup(workload.Close)
host, port, _ := strings.Cut(strings.TrimPrefix(workload.URL, "http://"), ":")
n, err := strconv.Atoi(port)
if err != nil {
t.Fatal(err)
}
return host, n
}
// ask makes one request through the proxy for a given name and path, without following redirects.
func ask(t *testing.T, proxy, name, path string, auth [2]string) *http.Response {
t.Helper()
req, err := http.NewRequest(http.MethodGet, proxy+path, nil)
if err != nil {
t.Fatal(err)
}
req.Host = name
if auth[0] != "" {
req.SetBasicAuth(auth[0], auth[1])
}
client := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
}}
answer, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = answer.Body.Close() })
return answer
}
func proxyFor(t *testing.T, routesJSON string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "routes.json")
if err := os.WriteFile(path, []byte(routesJSON), 0o644); err != nil {
t.Fatal(err)
}
routes, public, err := routesFrom(path)
if err != nil {
t.Fatal(err)
}
held := newTable()
held.set(routes, public)
server := httptest.NewServer(handler(held))
t.Cleanup(server.Close)
return server.URL
}
// A refusal on a path shadows the ordinary route for that path and leaves every other path alone.
//
// **This is why path scoping is a prerequisite and not a sibling capability.** The rule being
// reproduced matches a path on a host that is already routed to a workload, so a table mapping a
// host to one target cannot express it at all — no amount of authentication or source filtering
// would have helped.
func TestARefusedPathShadowsTheRouteAndLeavesTheRestServed(t *testing.T) {
at, port := served(t, "the workload")
proxy := proxyFor(t, `{"given":[
{"from":"forge","node":"anchor","at":"`+at+`","values":{"name":"forge.example","port":`+strconv.Itoa(port)+`}},
{"from":"forge","node":"anchor","values":{"name":"forge.example","path":"/api/internal","priority":100,"deny":true}}
]}`)
if got := ask(t, proxy, "forge.example", "/api/internal/hook", [2]string{}).StatusCode; got != http.StatusForbidden {
t.Fatalf("the refused path answered %d, so the block that was put in front of it during an "+
"incident is not in front of it any more", got)
}
if got := ask(t, proxy, "forge.example", "/", [2]string{}).StatusCode; got != http.StatusOK {
t.Fatalf("refusing one path took the whole route with it: %d", got)
}
}
// A redirect answers with the redirect, and the request keeps its own path and query.
//
// Losing the path would turn canonicalising one name onto another into "every deep link now lands
// on the front page", which is the kind of breakage that produces no error anywhere.
func TestARedirectKeepsThePathAndQuery(t *testing.T) {
proxy := proxyFor(t, `{"given":[
{"from":"site","node":"anchor","values":{"name":"www.example","redirect":"https://example/"}}
]}`)
answer := ask(t, proxy, "www.example", "/deep/page?ref=1", [2]string{})
if answer.StatusCode != http.StatusMovedPermanently {
t.Fatalf("a declared redirect answered %d", answer.StatusCode)
}
where := answer.Header.Get("Location")
if !strings.Contains(where, "/deep/page") || !strings.Contains(where, "ref=1") {
t.Fatalf("the redirect dropped the path or the query: %q", where)
}
}
// Authentication refuses a request with no credentials, admits one with the right ones, and refuses
// the wrong ones — with the credentials read from the secret the declaration *named*.
func TestAuthenticationAdmitsOnlyWhatTheSecretSays(t *testing.T) {
at, port := served(t, "the console")
hash, err := bcrypt.GenerateFromPassword([]byte("correct horse"), bcrypt.MinCost)
if err != nil {
t.Fatal(err)
}
secret := filepath.Join(t.TempDir(), "console-auth")
if err := os.WriteFile(secret, []byte("# a comment\nadmin:"+string(hash)+"\n"), 0o600); err != nil {
t.Fatal(err)
}
proxy := proxyFor(t, `{"given":[
{"from":"console","node":"anchor","at":"`+at+`","values":{"name":"console.example","port":`+strconv.Itoa(port)+`,"auth":"`+secret+`"}}
]}`)
if got := ask(t, proxy, "console.example", "/", [2]string{}).StatusCode; got != http.StatusUnauthorized {
t.Fatalf("an admin surface with no login of its own answered %d without credentials", got)
}
if got := ask(t, proxy, "console.example", "/", [2]string{"admin", "wrong"}).StatusCode; got != http.StatusUnauthorized {
t.Fatalf("the wrong password answered %d", got)
}
if got := ask(t, proxy, "console.example", "/", [2]string{"admin", "correct horse"}).StatusCode; got != http.StatusOK {
t.Fatalf("the right password answered %d", got)
}
}
// The negative case ADR 0108 promised would be refused: a credential in the declaration.
//
// **Refused whole, not tolerated and not served unprotected.** A hash carried in a declaration was
// the rejected option; nothing in the running system should quietly accept it later, because the
// precedent is far easier to set than to withdraw. If this test is deleted the option returns and
// nothing else notices.
func TestACredentialInTheDeclarationIsRefusedRatherThanServed(t *testing.T) {
inline := []string{
`{"given":[{"from":"c","node":"n","at":"127.0.0.1","values":{"name":"c.example","port":8080,"auth":"admin:$2a$10$abcdefghijklmnopqrstuv"}}]}`,
`{"given":[{"from":"c","node":"n","at":"127.0.0.1","values":{"name":"c.example","port":8080,"auth":"$2a$10$abcdefghijklmnopqrstuv"}}]}`,
}
for _, body := range inline {
path := filepath.Join(t.TempDir(), "routes.json")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
routes, _, err := routesFrom(path)
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 {
t.Fatalf("a declaration carrying a credential was served anyway: %v", routes)
}
}
}
// Authentication declared, secret unreadable: the route refuses. It does not serve unprotected.
//
// **Fail closed.** The alternative turns a missing file into a silently public admin surface, which
// is the outcome the whole record exists to prevent. It answers rather than 404s, so an operator
// sees "cannot read the credentials" instead of concluding the route was withdrawn.
func TestAnUnreadableSecretFailsClosed(t *testing.T) {
at, port := served(t, "the console")
missing := filepath.Join(t.TempDir(), "not-mounted")
proxy := proxyFor(t, `{"given":[
{"from":"console","node":"anchor","at":"`+at+`","values":{"name":"console.example","port":`+strconv.Itoa(port)+`,"auth":"`+missing+`"}}
]}`)
answer := ask(t, proxy, "console.example", "/", [2]string{})
if answer.StatusCode == http.StatusOK {
t.Fatal("a route whose credentials could not be read served the workload unprotected")
}
if answer.StatusCode != http.StatusServiceUnavailable {
t.Fatalf("expected the route to say it cannot check, got %d", answer.StatusCode)
}
}
// Equal priorities resolve the same way every time, so the same declaration serves the same way
// after a restart.
//
// Sorting only by priority leaves rules that share one in whatever order the map produced. The
// proxy would still work, and would work differently between restarts — which is the hardest kind
// of fault to believe when it is reported.
func TestRulesThatShareAPriorityAreStillTotallyOrdered(t *testing.T) {
first := []rule{
{path: "/a", priority: 10, target: "http://x:1"},
{path: "/bb", priority: 10, target: "http://y:2"},
{path: "", priority: 10, target: "http://z:3"},
}
second := []rule{
{path: "", priority: 10, target: "http://z:3"},
{path: "/bb", priority: 10, target: "http://y:2"},
{path: "/a", priority: 10, target: "http://x:1"},
}
inOrder(first)
inOrder(second)
for i := range first {
if first[i].path != second[i].path || first[i].target != second[i].target {
t.Fatalf("two orderings of the same rules disagree at %d: %q vs %q",
i, first[i].path, second[i].path)
}
}
// And the more specific rule is matched first, which is the intuitive reading.
if first[0].path != "/bb" {
t.Fatalf("the longest path is not matched first: %q", first[0].path)
}
}
// Priority decides before path length does, so a rule can be made to win regardless of specificity.
func TestPriorityOutranksPathLength(t *testing.T) {
rules := []rule{
{path: "/very/long/path", priority: 1, target: "http://x:1"},
{path: "", priority: 100, target: "http://y:2"},
}
inOrder(rules)
if rules[0].priority != 100 {
t.Fatalf("a higher priority did not win: %+v", rules[0])
}
}
// A priority above a port number survives, because a priority is an ordering and not a port.
//
// **Found by review, and it was load-bearing.** Priority was first read with the port reader, which
// caps at 65535 — so a rule declared above that silently became priority 0 and stopped shadowing the
// route it exists to shadow. The one real rule this has to reproduce is declared at 100000, so the
// capability would have shipped looking complete and doing nothing.
func TestAPriorityAboveAPortNumberSurvives(t *testing.T) {
at, port := served(t, "the workload")
proxy := proxyFor(t, `{"given":[
{"from":"forge","node":"anchor","at":"`+at+`","values":{"name":"forge.example","port":`+strconv.Itoa(port)+`}},
{"from":"forge","node":"anchor","values":{"name":"forge.example","path":"/api/internal","priority":100000,"deny":true}}
]}`)
if got := ask(t, proxy, "forge.example", "/api/internal/hook", [2]string{}).StatusCode; got != http.StatusForbidden {
t.Fatalf("a rule declared at priority 100000 answered %d instead of refusing", got)
}
}
// A host routed only on some paths says so, rather than claiming the name is not served here.
//
// Saying "no route for this name" while listing that very name as served is a contradiction an
// operator has to disbelieve the proxy to get past — and path scoping makes it reachable, because a
// host can now have rules that none of this request's paths match.
func TestAHostRoutedOnlyOnSomePathsSaysSo(t *testing.T) {
proxy := proxyFor(t, `{"given":[
{"from":"forge","node":"anchor","values":{"name":"forge.example","path":"/api/internal","deny":true}}
]}`)
answer := ask(t, proxy, "forge.example", "/elsewhere", [2]string{})
if answer.StatusCode != http.StatusNotFound {
t.Fatalf("an uncovered path answered %d", answer.StatusCode)
}
body := make([]byte, 256)
n, _ := answer.Body.Read(body)
said := string(body[:n])
if !strings.Contains(said, "is served here") || !strings.Contains(said, "/elsewhere") {
t.Fatalf("the refusal does not distinguish an uncovered path from an unserved name: %q", said)
}
}
modules/route-proxy/cmd/route-proxy/reach_test.go
+206
View File
@@ -0,0 +1,206 @@
package main
import (
"crypto/tls"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/membership"
)
// behind is a workload the proxy can send to, and a table routing one public name and one
// internal-only name to it, with the mesh's machines as the membership would issue them.
func behind(t *testing.T, mesh ...string) *table {
t.Helper()
workload := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "the workload")
}))
t.Cleanup(workload.Close)
at, _ := url.Parse(workload.URL)
host, port, _ := net.SplitHostPort(at.Host)
routes, public, err := routesFrom(write(t, fmt.Sprintf(`{"given":[
{"from":"app","node":"anchor","at":%q,
"values":{"name":"app.example","internal-name":"app.anchor.internal","port":%s}},
{"from":"admin","node":"anchor","at":%q,
"values":{"internal-name":"admin.anchor.internal","port":%s}}
]}`, host, port, host, port)))
if err != nil {
t.Fatal(err)
}
held := newTable()
inside, err := sourcesOf(mesh)
if err != nil {
t.Fatal(err)
}
held.setInside(inside)
held.set(routes, public)
return held
}
// askFrom is what the proxy answers a request for host coming from remote.
func askFrom(held *table, host, remote string) (int, string) {
r := httptest.NewRequest(http.MethodGet, "http://"+host+"/", nil)
r.RemoteAddr = remote
w := httptest.NewRecorder()
handler(held).ServeHTTP(w, r)
return w.Code, w.Body.String()
}
// **An internal-only name is served to the private network and to nobody else** (novox/hq ADR
// 0138, issue 191). The proxy answers public names on the same listeners, so without this a name
// being internal kept nobody out: a request from the internet only had to carry it.
func TestAnInternalOnlyNameIsServedOnlyInsideThePrivateNetwork(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
if code, body := askFrom(held, "admin.anchor.internal", "10.10.0.7:51000"); code != http.StatusOK ||
body != "the workload" {
t.Errorf("a request from the private network was not served: %d %q", code, body)
}
if code, body := askFrom(held, "admin.anchor.internal", "127.0.0.1:51000"); code != http.StatusOK {
t.Errorf("a request from the machine itself was not served: %d %q", code, body)
}
code, body := askFrom(held, "admin.anchor.internal", "203.0.113.9:51000")
if code != http.StatusNotFound {
t.Fatalf("a request from outside the private network reached an internal-only name: %d %q",
code, body)
}
// Answered as a name never routed, and the list of what is served does not name it either —
// otherwise the refusal would tell an outsider exactly what to ask for from inside.
if strings.Contains(strings.SplitN(body, "\n", 2)[1], "admin.anchor.internal") {
t.Errorf("the refusal names the internal-only route to an outsider: %q", body)
}
if !strings.Contains(body, "app.example") {
t.Errorf("the refusal stopped listing the public names: %q", body)
}
}
// The internal name of a route that also has a public one is internal too: served inside, and to
// an outsider only under the public name. Nothing is lost — the outsider has the public name — and a
// name stays one thing whichever route it came from.
func TestAnInternalAliasOfAPublicRouteIsServedInsideOnly(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
if code, body := askFrom(held, "app.anchor.internal", "10.10.0.7:51000"); code != http.StatusOK {
t.Errorf("the internal alias stopped answering the private network: %d %q", code, body)
}
if code, _ := askFrom(held, "app.anchor.internal", "203.0.113.9:51000"); code != http.StatusNotFound {
t.Errorf("the internal alias was served to an outsider: %d", code)
}
if code, _ := askFrom(held, "app.example", "203.0.113.9:51000"); code != http.StatusOK {
t.Errorf("the public name was refused to an outsider: %d", code)
}
}
// Before a membership has said who the mesh is, only the machine itself is inside — refused to
// everyone else, never served to everyone.
func TestUntilTheMeshIsIssuedAnInternalOnlyNameIsServedToTheMachineAlone(t *testing.T) {
held := behind(t)
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:51000"); code != http.StatusNotFound {
t.Errorf("an internal-only name was served with no private network said: %d", code)
}
if code, _ := askFrom(held, "admin.anchor.internal", "[::1]:51000"); code != http.StatusOK {
t.Errorf("an internal-only name was refused to the machine itself: %d", code)
}
}
type from struct {
net.Conn
remote net.Addr
}
func (c from) RemoteAddr() net.Addr { return c.remote }
// The handshake refuses an internal-only name to an outsider too: the certificate would name it,
// and serving it would answer the question the routing refuses to.
func TestTheHandshakeRefusesAnInternalOnlyNameToAnOutsider(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
served := &tls.Certificate{}
pick := certificateFor(held, func(*tls.ClientHelloInfo) (*tls.Certificate, error) { return served, nil }, nil)
hello := func(name, remote string) *tls.ClientHelloInfo {
addr, _ := net.ResolveTCPAddr("tcp", remote)
return &tls.ClientHelloInfo{ServerName: name, Conn: from{remote: addr}}
}
if _, err := pick(hello("admin.anchor.internal", "203.0.113.9:443")); err == nil {
t.Error("an outsider was handed a certificate for an internal-only name")
}
if got, err := pick(hello("admin.anchor.internal", "10.10.0.7:443")); err != nil || got != served {
t.Errorf("a client on the private network was refused: %v", err)
}
if got, err := pick(hello("app.example", "203.0.113.9:443")); err != nil || got != served {
t.Errorf("a public name was refused to an outsider: %v", err)
}
}
// The mesh is issued as machines' addresses; a range is read as well. One that does not parse is
// refused rather than skipped, so a typo never quietly narrows or widens who is inside.
func TestTheMeshIsReadAsAddressesAndRanges(t *testing.T) {
if _, err := sourcesOf([]string{"10.10.0.1", "not-an-address"}); err == nil {
t.Error("an entry that is not an address was accepted")
}
inside, err := sourcesOf([]string{"10.10.0.1", "fd00::1", "10.20.0.0/24"})
if err != nil {
t.Fatal(err)
}
for remote, want := range map[string]bool{
"10.10.0.1:1": true,
"[::ffff:10.10.0.1]:1": true,
"[fd00::1]:1": true,
"10.20.0.200:1": true,
"10.10.0.2:1": false,
"192.168.1.10:1": false,
"not-an-address": false,
} {
if inside.holds(remote) != want {
t.Errorf("%s inside the mesh: got %v, want %v", remote, !want, want)
}
}
}
// What the mesh issues is what is served: the routes in the membership, internal names to the
// machines it names (novox/hq ADR 0167).
func TestAMembershipIsServedAsIssued(t *testing.T) {
held := newTable()
took := applyMembership(membership.Membership{
Receives: map[string]json.RawMessage{"route": json.RawMessage(`[
{"from":"admin","node":"anchor","at":"anchor.internal",
"values":{"internal-name":"admin.anchor.internal","port":8080}}]`)},
Mesh: []string{"10.10.0.7"},
}, held)
if !took {
t.Fatal("a membership carrying routes was not applied")
}
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound {
t.Error("a machine the membership names was refused the internal-only route")
}
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.9:1"); code != http.StatusNotFound {
t.Errorf("a machine the membership does not name was served the internal-only route: %d", code)
}
}
// A membership that says nothing about routes is one from a controller that does not issue them,
// and changes nothing: the file stays the source rather than every route being withdrawn.
func TestAMembershipWithoutRoutesLeavesTheFileServing(t *testing.T) {
held := behind(t, "10.10.0.7")
before := held.names()
if applyMembership(membership.Membership{Mesh: []string{"10.10.0.7"}}, held) {
t.Error("a membership without routes was taken as the source of routes")
}
if got := held.names(); strings.Join(got, ",") != strings.Join(before, ",") {
t.Errorf("a membership without routes changed what is served: %v, was %v", got, before)
}
if applyMembership(membership.Membership{
Receives: map[string]json.RawMessage{"route": json.RawMessage(`[]`)},
Mesh: []string{"not-an-address"},
}, held) {
t.Error("a membership whose mesh cannot be read was applied")
}
}
modules/route-proxy/cmd/route-proxy/routes_test.go
+534
View File
@@ -0,0 +1,534 @@
package main
import (
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
)
func write(t *testing.T, body string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "routes.json")
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return path
}
// plain is the table an ordinary set of routes makes: one host, one target, no policy.
func plain(routes map[string]string) map[string][]rule {
out := map[string][]rule{}
for host, target := range routes {
out[host] = []rule{{target: target}}
}
return out
}
// allPublic is every host in a routes map, ACME-eligible — the ordinary case for a test with no
// internal-name alias of its own to distinguish.
func allPublic(routes map[string][]rule) map[string]bool {
out := map[string]bool{}
for host := range routes {
out[host] = true
}
return out
}
// targetOf is where a host's first matching rule sends a request.
func targetOf(routes map[string][]rule, host string) string {
if rules := routes[host]; len(rules) > 0 {
return rules[0].target
}
return ""
}
// A route is a grant: the consumer supplies a target, and where that machine is comes from the
// mesh rather than from a naming convention the proxy has to know.
func TestARouteGoesToWhereTheMeshSaysTheConsumerIs(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"contributions":1,"requirement":"route","given":[
{"from":"app","node":"laptop","at":"laptop.internal","values":{"name":"App.Example","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
// Lower-cased, because a Host header is not case-sensitive and a route that only answers the
// spelling in the manifest answers half the requests made to it.
if targetOf(routes, "app.example") != "http://laptop.internal:8080" {
t.Fatalf("the route does not point at the consumer: %v", routes)
}
}
// A route with an internal-name alias is reachable under both hostnames, pointed at the same
// target — the same convenience a predecessor proxy gave for reaching a service over the VPN
// without a public TLS round trip.
func TestARouteWithAnInternalNameIsReachableUnderBoth(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal",
"values":{"name":"app.example","internal-name":"app.anchor.internal","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if targetOf(routes, "app.example") != "http://anchor.internal:8080" {
t.Fatalf("the public name does not point at the consumer: %v", routes)
}
if targetOf(routes, "app.anchor.internal") != "http://anchor.internal:8080" {
t.Fatalf("the internal alias does not point at the same consumer: %v", routes)
}
if !public["app.example"] {
t.Errorf("the public name is not eligible for a certificate: %v", public)
}
if public["app.anchor.internal"] {
t.Errorf("the internal alias is eligible for a certificate no public CA could ever issue: %v",
public)
}
}
// A route whose endpoint reaches only the private network carries an internal name and no public
// one (novox/hq ADR 0138), and is served under that name rather than skipped as naming nothing —
// skipping it left every internal-only module unreachable by name (novox/hq issue 191).
func TestARouteWithOnlyAnInternalNameIsServed(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal",
"values":{"internal-name":"App.Anchor.Internal","port":8443,"scheme":"https","insecure":true}}
]}`))
if err != nil {
t.Fatal(err)
}
if targetOf(routes, "app.anchor.internal") != "https://anchor.internal:8443" {
t.Fatalf("the internal-only route is not served: %v", routes)
}
if len(routes) != 1 {
t.Errorf("an internal-only route made hosts it never named: %v", routes)
}
if len(public) != 0 {
t.Errorf("an internal-only route made a name eligible for a public certificate: %v", public)
}
held := newTable()
held.set(routes, public)
if err := onlyInternalNamesTheMeshSaid(held)(context.Background(), "app.anchor.internal"); err != nil {
t.Errorf("the internal authority refused the internal-only route's name: %v", err)
}
if err := onlyWhatTheMeshSaid(held)(context.Background(), "app.anchor.internal"); err == nil {
t.Error("a public certificate was ordered for an internal-only name")
}
}
// A route with neither name has nothing to be served under, and is still skipped.
func TestARouteWithNeitherNameIsSkipped(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal","values":{"internal-name":" ","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 || len(public) != 0 {
t.Errorf("a route that named nothing was served: %v %v", routes, public)
}
}
// A route with no internal-name composed gets no second host — the ordinary case, unchanged.
func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","values":{"name":"app.example","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 1 {
t.Fatalf("a route with no internal-name grew a second host: %v", routes)
}
}
// A workload beside the proxy is ordinary, and reaching it over loopback is both correct and the
// only thing that works when there is no private network.
func TestAConsumerOnTheProxysOwnMachineIsReachedOverLoopback(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","values":{"name":"app.example","port":9000}}
]}`))
if err != nil {
t.Fatal(err)
}
if targetOf(routes, "app.example") != "http://127.0.0.1:9000" {
t.Fatalf("a workload on this machine was not reachable: %v", routes)
}
}
// Skipped rather than served wrongly. A route with no port would proxy to :0.
func TestAContributionMissingWhatARouteNeedsIsSkipped(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"a","node":"n","at":"n.internal","values":{"name":"no-port.example"}},
{"from":"b","node":"n","at":"n.internal","values":{"port":8080}},
{"from":"c","node":"n","at":"n.internal","values":{"name":"fine.example","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 1 || targetOf(routes, "fine.example") == "" {
t.Fatalf("an unusable contribution was served: %v", routes)
}
}
// A route may name a target reached over https, for a backend that terminates its own TLS — the
// shape Mailu's webmail front needs, which this proxy reaches as a plain workload otherwise.
func TestARouteMayTargetHttps(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"mail","node":"anchor","at":"anchor.internal",
"values":{"name":"mail.example","port":7443,"scheme":"https","insecure":true}}
]}`))
if err != nil {
t.Fatal(err)
}
if targetOf(routes, "mail.example") != "https://anchor.internal:7443" {
t.Fatalf("an https target was not built as one: %v", routes)
}
if !routes["mail.example"][0].insecure {
t.Fatal("insecure was declared and not carried onto the rule")
}
}
// A scheme that is neither http nor https is refused rather than guessed at.
func TestARouteWithAnUnknownSchemeIsSkipped(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"a","node":"n","at":"n.internal","values":{"name":"bad.example","port":80,"scheme":"ftp"}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 {
t.Fatalf("a route with an unusable scheme was served: %v", routes)
}
}
// End to end: a backend terminating TLS with a certificate nothing would ordinarily trust is still
// reached when the route declared `insecure`, and the response comes back through unmodified.
func TestTheProxyReachesAnInsecureHttpsBackend(t *testing.T) {
workload := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte("the workload, over its own TLS"))
}))
defer workload.Close()
target := strings.TrimPrefix(workload.URL, "https://")
held := newTable()
routes := map[string][]rule{"mail.example": {{target: "https://" + target, insecure: true}}}
held.set(routes, allPublic(routes))
proxy := httptest.NewServer(handler(held))
defer proxy.Close()
asked, err := http.NewRequest(http.MethodGet, proxy.URL, nil)
if err != nil {
t.Fatal(err)
}
asked.Host = "mail.example"
answer, err := http.DefaultClient.Do(asked)
if err != nil {
t.Fatal(err)
}
defer answer.Body.Close()
if answer.StatusCode != http.StatusOK {
t.Fatalf("an insecure https backend was not reached: %d", answer.StatusCode)
}
}
// End to end through the proxy itself: a request for the name reaches the workload, and a name
// nobody asked for is refused in a way that says what IS served.
func TestTheProxyReachesTheWorkloadAndNamesWhatItServes(t *testing.T) {
workload := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte("the workload"))
}))
defer workload.Close()
target := strings.TrimPrefix(workload.URL, "http://")
host, port, _ := strings.Cut(target, ":")
held := newTable()
held.set(plain(map[string]string{"app.example": "http://" + host + ":" + port}), allPublic(plain(map[string]string{"app.example": "http://" + host + ":" + port})))
proxy := httptest.NewServer(handler(held))
defer proxy.Close()
asked, err := http.NewRequest(http.MethodGet, proxy.URL, nil)
if err != nil {
t.Fatal(err)
}
asked.Host = "app.example"
answer, err := http.DefaultClient.Do(asked)
if err != nil {
t.Fatal(err)
}
defer answer.Body.Close()
if answer.StatusCode != http.StatusOK {
t.Fatalf("a request for a served name got %d", answer.StatusCode)
}
// And a name that is not served says which are — a route withdrawn and a name that never
// existed are different things, and a bare 404 makes an operator go and read the mesh.
other, _ := http.NewRequest(http.MethodGet, proxy.URL, nil)
other.Host = "nobody.example"
refused, err := http.DefaultClient.Do(other)
if err != nil {
t.Fatal(err)
}
defer refused.Body.Close()
if refused.StatusCode != http.StatusNotFound {
t.Fatalf("a name nobody asked for got %d", refused.StatusCode)
}
body := make([]byte, 256)
n, _ := refused.Body.Read(body)
if !strings.Contains(string(body[:n]), "app.example") {
t.Fatalf("the refusal does not say what is served: %s", body[:n])
}
}
// The file is the whole truth about who has a route, so the table replaces rather than merges.
//
// Merging would keep serving a name whose module was unassigned — the stale-route fault
// 08-connectivity lists as open, reintroduced one level down. A stale public name pointing at
// nothing fails more visibly than a stale grant, which is exactly why it must not survive.
func TestWithdrawingARouteStopsServingIt(t *testing.T) {
held := newTable()
initial := plain(map[string]string{
"going.example": "http://a.internal:80",
"staying.example": "http://b.internal:80",
})
held.set(initial, allPublic(initial))
held.set(plain(map[string]string{"staying.example": "http://b.internal:80"}), allPublic(plain(map[string]string{"staying.example": "http://b.internal:80"})))
if _, still := held.find("going.example", "/"); still {
t.Fatal("a route whose module was unassigned is still served")
}
if _, kept := held.find("staying.example", "/"); !kept {
t.Fatal("withdrawing one route took another with it")
}
}
// A Host header carries a port and the name does not.
func TestARequestNamingAPortStillFindsItsRoute(t *testing.T) {
held := newTable()
held.set(plain(map[string]string{"app.example": "http://a.internal:8080"}), allPublic(plain(map[string]string{"app.example": "http://a.internal:8080"})))
if _, found := held.find("app.example:8080", "/"); !found {
t.Fatal("a request to app.example:8080 did not find the route for app.example")
}
}
// Defends novox/hq 04-ISSUES/004: issuance targets staging unless something says otherwise.
//
// The failure this guards is not a broken proxy. It is a working one that quietly spends a
// production quota which does not replenish for a week, on exactly the work most likely to
// iterate.
func TestTheIssuerIsStagingUnlessNamed(t *testing.T) {
t.Setenv("ACME_DIRECTORY", "")
if got := issuer(); !strings.Contains(got, "staging") {
t.Fatalf("with nothing set the issuer is %q, and a default that spends production quota "+
"is a default nobody chose", got)
}
t.Setenv("ACME_DIRECTORY", "https://acme-v02.api.letsencrypt.org/directory")
if got := issuer(); strings.Contains(got, "staging") {
t.Fatalf("an issuer was named explicitly and %q was used instead", got)
}
}
// A certificate is only ever asked for on a name the mesh routes here.
//
// **Without this, anything that can reach the port spends the quota.** A scan sending arbitrary
// names, or one misconfigured client, becomes a stream of failed orders against the account's
// rate limit — and the proxy would look healthy throughout.
func TestNoCertificateIsAskedForOnAnUnroutedName(t *testing.T) {
held := newTable()
held.set(plain(map[string]string{"photos.example": "http://127.0.0.1:8080"}), allPublic(plain(map[string]string{"photos.example": "http://127.0.0.1:8080"})))
policy := onlyWhatTheMeshSaid(held)
if err := policy(context.Background(), "photos.example"); err != nil {
t.Errorf("a name the mesh routes here was refused a certificate: %v", err)
}
for _, name := range []string{"unknown.example", "", "photos.example.evil"} {
if err := policy(context.Background(), name); err == nil {
t.Errorf("a certificate would be ordered for %q, which the mesh never mentioned", name)
}
}
}
// A certificate is asked for on a route's public name, never on its internal-network alias — no
// public CA can validate a private name, and asking anyway would only spend the account's rate
// limit on an order that can never succeed.
func TestNoCertificateIsAskedForOnAnInternalAlias(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal",
"values":{"name":"app.example","internal-name":"app.anchor.internal","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
held := newTable()
held.set(routes, public)
policy := onlyWhatTheMeshSaid(held)
if err := policy(context.Background(), "app.example"); err != nil {
t.Errorf("the route's public name was refused a certificate: %v", err)
}
if err := policy(context.Background(), "app.anchor.internal"); err == nil {
t.Error("a certificate was ordered for the internal alias, which no public CA can validate")
}
}
// A certificate is asked of the *internal* authority only for a name that is routed here and is
// not a route's own public name — the internal-network alias, never the route it accompanies.
func TestTheInternalAuthorityOnlyCertifiesInternalOnlyAliases(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal",
"values":{"name":"app.example","internal-name":"app.anchor.internal","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
held := newTable()
held.set(routes, public)
policy := onlyInternalNamesTheMeshSaid(held)
if err := policy(context.Background(), "app.anchor.internal"); err != nil {
t.Errorf("the internal alias was refused by its own authority: %v", err)
}
if err := policy(context.Background(), "app.example"); err == nil {
t.Error("the internal authority certified a route's public name, which the public authority already covers")
}
if err := policy(context.Background(), "unrouted.internal"); err == nil {
t.Error("the internal authority certified a name nobody routed here")
}
}
// A route withdrawn stops being certifiable, without the proxy restarting.
func TestWithdrawingARouteWithdrawsItsCertificate(t *testing.T) {
held := newTable()
held.set(plain(map[string]string{"photos.example": "http://127.0.0.1:8080"}), allPublic(plain(map[string]string{"photos.example": "http://127.0.0.1:8080"})))
policy := onlyWhatTheMeshSaid(held)
if err := policy(context.Background(), "photos.example"); err != nil {
t.Fatal(err)
}
held.set(nil, nil)
if err := policy(context.Background(), "photos.example"); err == nil {
t.Fatal("a withdrawn route can still order certificates, so the policy read a copy taken " +
"once rather than what is served now")
}
}
// A route may say the largest body it carries, and the proxy holds requests to it.
//
// The registry is why: image layers arrive as single requests of gigabytes, and a proxy's own
// default refuses them long before the workload is reached. It is configuration beside `insecure`,
// not a fifth policy — novox/hq ADR 0108 closed that set at four.
func TestARouteMayLimitTheBodyItCarries(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"registry","node":"anchor","at":"anchor.internal",
"values":{"name":"images.example","port":5000,"max-request-body":21474836480}}
]}`))
if err != nil {
t.Fatal(err)
}
rules := routes["images.example"]
if len(rules) != 1 {
t.Fatalf("the route is not served once: %v", routes)
}
if rules[0].maxRequestBody != 21474836480 {
t.Fatalf("the limit did not survive the contribution: %d", rules[0].maxRequestBody)
}
}
// Saying nothing leaves the route unlimited, which is what every route already got.
func TestARouteThatSaysNothingCarriesAnySize(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal","values":{"name":"app.example","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if got := routes["app.example"][0].maxRequestBody; got != 0 {
t.Fatalf("a route that asked for no limit got one: %d", got)
}
}
// A limit that is not a whole positive number of bytes takes the route with it. Serving it without
// the limit would carry exactly what the module said not to carry, and report success doing it.
func TestARouteWithAnUnusableLimitIsSkipped(t *testing.T) {
for _, asked := range []string{`"lots"`, `-1`, `0`, `1.5`} {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"registry","node":"anchor","at":"anchor.internal",
"values":{"name":"images.example","port":5000,"max-request-body":`+asked+`}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes["images.example"]) != 0 {
t.Errorf("a route asking for a max-request-body of %s was served anyway: %v", asked, routes)
}
}
}
// A request larger than the route carries is refused by the proxy, with the limit named, and the
// workload never sees it. A request within it is proxied normally.
func TestABodyOverTheLimitIsRefusedAndOneUnderItIsCarried(t *testing.T) {
var reached int
workload := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
reached++
fmt.Fprintf(w, "carried %d bytes", len(body))
}))
defer workload.Close()
at := strings.TrimPrefix(workload.URL, "http://")
host, port, _ := strings.Cut(at, ":")
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"registry","node":"anchor","at":"`+host+`",
"values":{"name":"images.example","port":`+port+`,"max-request-body":8}}
]}`))
if err != nil {
t.Fatal(err)
}
held := newTable()
held.set(routes, map[string]bool{})
proxy := httptest.NewServer(handler(held))
defer proxy.Close()
over, err := post(proxy.URL, "images.example", "123456789")
if err != nil {
t.Fatal(err)
}
defer over.Body.Close()
if over.StatusCode != http.StatusRequestEntityTooLarge {
t.Fatalf("a body over the limit answered %d, not 413", over.StatusCode)
}
if reached != 0 {
t.Fatalf("the workload was reached by a request the route said it would not carry")
}
under, err := post(proxy.URL, "images.example", "1234")
if err != nil {
t.Fatal(err)
}
defer under.Body.Close()
if under.StatusCode != http.StatusOK {
t.Fatalf("a body within the limit answered %d, not 200", under.StatusCode)
}
if reached != 1 {
t.Fatalf("the workload was not reached by a request within the limit")
}
}
// post sends a body to the proxy as the named route, since a route is found by the Host header.
func post(url, host, body string) (*http.Response, error) {
asked, err := http.NewRequest(http.MethodPost, url, strings.NewReader(body))
if err != nil {
return nil, err
}
asked.Host = host
asked.Header.Set("Content-Type", "application/octet-stream")
return http.DefaultClient.Do(asked)
}
modules/route-proxy/go.mod
+14 -1
View File
@@ -2,4 +2,17 @@ module route-proxy
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021114014-66e8fdaa62e2
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022075101-c61212eba77c
github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0
)
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/net v0.58.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0 // indirect
)
modules/route-proxy/go.sum
+30 -2
View File
@@ -1,2 +1,30 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021114014-66e8fdaa62e2 h1:Cdl5N56QPfmFBoLHrDfvvyc5Q+PUnTjf3fgYD7imcco=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021114014-66e8fdaa62e2/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022075101-c61212eba77c h1:YdSbkI42rlVmClybH8BS717ut0vouKIMavO691fpdps=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022075101-c61212eba77c/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
modules/route-proxy/module.json
+20 -21
View File
@@ -149,11 +149,18 @@
"internal-acme-env"
]
},
{
"id": "program-route-proxy",
"type": "archive",
"path": "${dir:state}/route-proxy",
"artifact": "server",
"unknown": "program"
},
{
"id": "server",
"type": "container",
"name": "route-proxy",
"artifact": "server",
"artifact": "trust",
"network": "host",
"env-file": [
"${dir:state}/acme.env",
@@ -163,7 +170,8 @@
"${dir:routes-dir}:/routes:ro",
"${dir:acme-cache}:/acme",
"${dir:ca-dir}:/ca:ro",
"${dir:mesh-state}/broker:/run/secrets/broker:ro"
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:state}/route-proxy:/opt/mesh:ro"
],
"env": {
"ROUTES": "/routes/mesh.json",
@@ -174,11 +182,15 @@
"INTERNAL_ACME_CA_BUNDLE": "/ca/internal-root.crt",
"MESH_BROKER_FILE": "/run/secrets/broker"
},
"args": [
"/opt/mesh/route-proxy"
],
"restart-on": [
"trust",
"acme-env",
"internal-trust",
"internal-acme-env"
"internal-acme-env",
"program-route-proxy"
],
"logging": "journald",
"health": {
@@ -195,14 +207,11 @@
"artifacts": [
{
"name": "server",
"kind": "image",
"from": "Dockerfile",
"compiles": "examples/route-proxy",
"context": {
"seat": "git",
"repository": "novox/mesh-controller",
"ref": "main"
}
"kind": "bundle",
"language": "go",
"system": "alpine",
"from": "cmd/route-proxy",
"binary": "route-proxy"
},
{
"name": "trust",
@@ -225,16 +234,6 @@
"kind": "archive",
"from": "files"
}
],
"on": [
{
"arg": "GO_BASE",
"image": "golang@sha256:6c2a5538f964f1c82f97ad14988bf05de100d922d159d0e398b54c7b0ca0c6c9"
},
{
"arg": "ALPINE_BASE",
"image": "alpine@sha256:d9e853e87e55526f6b2917df91a2115c36dd7c696a35be12163d44e6e2a4b6bc"
}
]
},
"jails": [
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/membership/membership.go
Vendored
+165
View File
@@ -0,0 +1,165 @@
// Package membership is what the mesh issues an assignment to serve and to reach (novox/hq ADR 0160), as a
// process reads it from the bus: the Membership shape, the one subject it is published on, the stream that
// holds it, and the bus address helpers a reader needs. The controller composes it and the route proxy reads
// it; both use this one definition, so there is no second copy to drift (novox/hq issue 1633).
package membership
import (
"encoding/json"
"strings"
)
// AssignmentsStream holds every assignment's membership, the newest per subject.
const AssignmentsStream = "ASSIGNMENTS"
// Subject is the one address a runtime derives for itself: where its own membership is published, from
// the two names its credential carries. Everything else is in the membership.
func Subject(node, module string) string {
return "mesh.assignment." + node + "." + module
}
// DirectGetSubject is the subject-addressed direct get of a membership from its stream: the one request an
// account may make of the stream.
func DirectGetSubject(node, module string) string {
return "$JS.API.DIRECT.GET." + AssignmentsStream + "." + Subject(node, module)
}
// Membership is one assignment's subjects: what this instance of a module on this machine serves,
// and what it may reach.
type Membership struct {
Node string `json:"node"`
Module string `json:"module"`
// Serves is every address a tool of this instance answers on. `{tool}` stands for the tool's
// own name. An address with a queue is shared with the module's other instances.
Serves []Served `json:"serves"`
// Seats is every verb of a seat this instance holds, at the subject the seat's callers use.
Seats []SeatServed `json:"seats,omitempty"`
// Emits is where an event of this module lands; `{event}` stands for the event's name.
Emits string `json:"emits"`
// Reaches is each tool this module may call, `<module>.<tool>`, to the subjects that reach it.
Reaches map[string][]string `json:"reaches,omitempty"`
// Tools is where this instance answers what it serves.
Tools string `json:"tools"`
// Receives is what this assignment is given for each requirement it receives, by requirement (novox/hq
// ADR 0167): kept as JSON because the catalogue owns the shape of a contribution.
Receives map[string]json.RawMessage `json:"receives,omitempty"`
// Mesh is every machine's address on the private network (novox/hq ADR 0167).
Mesh []string `json:"mesh,omitempty"`
// State is every bucket this module's code may reach, and whether it may write it (novox/hq ADR 0201).
State []StateIssued `json:"state,omitempty"`
// Logs is every log this module's code may reach (novox/hq ADR 0297).
Logs []LogIssued `json:"logs,omitempty"`
// SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats
// that name their caller or their kind (novox/hq ADR 0259 §3).
SeatTraffic *SeatTraffic `json:"seat-traffic,omitempty"`
// Streams are this module's own streams, Hears the module streams it hears (novox/hq ADR 0363).
Streams []StreamIssued `json:"streams,omitempty"`
Hears []HeardIssued `json:"hears,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server (novox/hq ADR 0367 §2).
Waits []WaitIssued `json:"waits,omitempty"`
}
// Served is one address a tool is answered on.
type Served struct {
Subject string `json:"subject"`
Queue string `json:"queue,omitempty"`
}
// SeatServed is one verb of a held seat, where its callers ask.
type SeatServed struct {
Seat string `json:"seat"`
Verb string `json:"verb"`
Subject string `json:"subject"`
}
// StateIssued is one bucket an assignment may reach, by the name its module uses for it (novox/hq ADR 0201).
// Key, when set, is the one key it may reach (novox/hq ADR 0260): this machine's.
type StateIssued struct {
Name string `json:"name"`
Bucket string `json:"bucket"`
Writes bool `json:"writes,omitempty"`
Key string `json:"key,omitempty"`
}
// LogIssued is one log an assignment may reach, by the name its module uses for it (novox/hq ADR 0297).
type LogIssued struct {
Name string `json:"name"`
Stream string `json:"stream"`
Subject string `json:"subject"`
Writes bool `json:"writes"`
}
// Worker is one durable consumer a holder pulls a seat's work from.
type Worker struct {
Stream string
Consumer string
Filter string
}
// SeatTraffic is one module's seat traffic beyond tools. Publish and Subscribe are subject patterns, in the
// server's wildcards.
type SeatTraffic struct {
Publish []string `json:"publish,omitempty"`
Subscribe []string `json:"subscribe,omitempty"`
Answers []string `json:"answers,omitempty"`
Workers []Worker `json:"workers,omitempty"`
Records []string `json:"records,omitempty"`
Kinds []KindHeld `json:"kinds,omitempty"`
}
// KindHeld is one kind of a kinded bench and who holds it.
type KindHeld struct {
Seat string `json:"seat"`
Kind string `json:"kind"`
Module string `json:"module"`
Node string `json:"node"`
Capabilities []string `json:"capabilities,omitempty"`
}
// StreamIssued is one of a module's own streams as its membership names it (ADR 0363 §2).
type StreamIssued struct {
Stream string `json:"stream"`
Events []string `json:"events"`
Subject string `json:"subject"`
}
// HeardIssued is one module stream an assignment hears (ADR 0363 §8).
type HeardIssued struct {
Stream string `json:"stream"`
Consumer string `json:"consumer"`
}
// WaitIssued is one family as a membership issues it: the name a session asks for, and its subject.
type WaitIssued struct {
Event string `json:"event"`
Subject string `json:"subject"`
}
// CredentialIn reads the credential out of a bus address, `[scheme://]user:password@host:port`, splitting on
// the last at-sign so a password containing one stays whole, and returns the address without it.
func CredentialIn(address string) (user, password, bare string) {
at := strings.LastIndex(address, "@")
if at < 0 {
return "", "", address
}
scheme := ""
rest := address[:at]
if i := strings.Index(rest, "://"); i >= 0 {
scheme, rest = rest[:i+3], rest[i+3:]
}
user, password, _ = strings.Cut(rest, ":")
return user, password, scheme + address[at+1:]
}
// BareAddress is a bus address with any credential stripped, for something that only needs to know
// whether a server is answering there.
func BareAddress(address string) string {
_, _, bare := CredentialIn(address)
if bare == "" {
return address
}
if strings.Contains(bare, "://") {
return bare
}
return "nats://" + bare
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/account.go
Vendored
+54
View File
@@ -0,0 +1,54 @@
// Package process is a module's own process's connection to the bus (novox/hq ADR 0436 clause 4): it reads
// the account the controller delivers, connects as it with its pinned fingerprint and its inbox prefix, and
// offers emitting an event, the module's states with compare-and-set, consuming a work queue, reading the
// router's records of the module's own asks, and calling a tool or verb — each only as the account's grants
// allow. It offers no Serve and no way to serve a tool: a process never answers a call. A sibling of stdio,
// which it imports only for stdio's pure types (ErrChanged, CodeChanged, StateEntry, EmitOptions); it never
// touches stdio's current, channel or live(), which stay the bundle's own way to the bus.
package process
import (
"encoding/json"
"fmt"
"os"
)
// Account is a process's own bus account, exactly as the controller delivers it as the module's own secret
// broker-<process id> (mesh-controller cmd/mesh-controller/modules.go issueProcess, heldProcessAccount, and
// issueWith for a module's own broker account, which also carries claims). A process account carries no
// claims at all; this type reads either file without error.
type Account struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
// Claims rides on a module's own broker account only; a process holds no seat, so its own account
// carries none (novox/hq ADR 0436 §2, §3). Kept raw: this package never reads it.
Claims json.RawMessage `json:"claims,omitempty"`
}
// InboxPrefix is this account's own inbox, and nothing wider: `_INBOX.<user>`.
func (a Account) InboxPrefix() string { return "_INBOX." + a.User }
// ReadAccount reads and validates the account file the controller delivers: it refuses a file missing url,
// user or password, naming the field — never the password, which does not appear in an error.
func ReadAccount(path string) (Account, error) {
data, err := os.ReadFile(path)
if err != nil {
return Account{}, fmt.Errorf("reading the process account at %s: %w", path, err)
}
var a Account
if err := json.Unmarshal(data, &a); err != nil {
return Account{}, fmt.Errorf("the process account at %s is not JSON: %w", path, err)
}
for _, f := range []struct{ name, value string }{
{"url", a.URL}, {"user", a.User}, {"password", a.Password},
} {
if f.value == "" {
return Account{}, fmt.Errorf("the process account at %s has no %s", path, f.name)
}
}
return a, nil
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/asks.go
Vendored
+68
View File
@@ -0,0 +1,68 @@
package process
import (
"context"
"encoding/json"
"fmt"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Ask reads the router's record of one of this process's module's own asks, under its own module and no
// other's (novox/hq ADR 0436 clause 4, ADR 0259 §3): by direct get on bucket, the same subject the tool
// runner reads (mesh-tools node-tools/internal/bus/seat.go ~300-345):
//
// $JS.API.DIRECT.GET.KV_<bucket>.$KV.<bucket>.<module>.<id>
//
// bucket is the router's records bucket as the caller's own manifest names it (operator-channel's is
// messenger_asks in the controller's own test fixtures) — never hard-coded to one bucket here, since a
// process may read records of more than one seat's router. A missing, deleted or purged key is nil, not
// an error.
func (c *Conn) Ask(bucket, id string) (*stdio.StateEntry, error) {
if err := checkAskID(id); err != nil {
return nil, err
}
subject := "$JS.API.DIRECT.GET.KV_" + bucket + ".$KV." + bucket + "." + c.account.Module + "." + id
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
defer cancel()
reply, err := c.nc.RequestWithContext(ctx, subject, nil)
if err != nil {
return nil, fmt.Errorf("reading %s's ask %s from %s: %w", c.account.Module, id, bucket, err)
}
if status := reply.Header.Get("Status"); status == "404" {
return nil, nil
} else if status != "" {
return nil, fmt.Errorf("the record %s.%s could not be read: %s %s", c.account.Module, id, status,
reply.Header.Get("Description"))
}
if op := reply.Header.Get("KV-Operation"); op == "DEL" || op == "PURGE" {
return nil, nil
}
var revision uint64
if _, err := fmt.Sscan(reply.Header.Get("Nats-Sequence"), &revision); err != nil {
return nil, fmt.Errorf("the record %s.%s carries no readable revision: %w", c.account.Module, id, err)
}
return &stdio.StateEntry{Key: c.account.Module + "." + id, Value: askValueOf(reply.Data), Revision: revision}, nil
}
// checkAskID refuses an id holding a dot or a wildcard before anything is sent: either would name a
// different key under the direct-get subject's own dot-joined path, never this one (novox/hq issue 1189
// AC2).
func checkAskID(id string) error {
if id == "" || strings.ContainsAny(id, ".*>") {
return fmt.Errorf("%q is not an ask id: an ask id holds no dot and no wildcard, naming one key and "+
"no other", id)
}
return nil
}
// askValueOf is a record's raw bytes as JSON, quoting them when they are not already valid JSON
// (mesh-tools node-tools/internal/bus/state.go:351-357, valueOf).
func askValueOf(raw []byte) json.RawMessage {
if len(raw) == 0 || !json.Valid(raw) {
b, _ := json.Marshal(string(raw))
return b
}
return json.RawMessage(append([]byte(nil), raw...))
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/call.go
Vendored
+122
View File
@@ -0,0 +1,122 @@
package process
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"github.com/nats-io/nats.go"
)
// CallTimeout bounds a call to a tool or a seat verb (mesh-tools node-tools/internal/bus/bus.go:35,
// RequestTimeout): what a module already expects.
var CallTimeout = 30 * time.Second
// Call invokes a tool or a seat verb as this process's own account (novox/hq ADR 0436 clause 4), on the
// caller-named subject, falling back to the subject that names no caller when nothing answers it —
// exactly the provisioner client's own pattern (mesh-catalog modules/mesh-provisioner-client/cmd/
// mesh-provisioner-client/secrets.go:59-61, 93-97, 119, 125-133). key is "seat:<seat>.<verb>" for a seat's
// verb, or "<module>.<tool>" for a module's own tool, as a process's bus.invokes names it. The reply is
// {result, error, node} (mesh-tools node-tools/internal/bus/bus.go answerOf); a non-empty error is the
// call's failure. It offers no way to answer a call — a process reaches the bus only as itself, and
// never serves anything.
func (c *Conn) Call(key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjects(key, c.callerToken())
if err != nil {
return nil, err
}
return c.callOn(called, fallback, body)
}
// CallOn is Call to the tool or seat verb of one named machine (novox/hq ADR 0469 §3; issue 1470): the tool
// runner of that node alone answers, on mesh.mod.<module>.call.<tool>.<node>.<token> (a seat verb:
// mesh.seat.<seat>.call.<verb>.<node>.<token>), falling back to the subject that names no caller
// (mesh-tools node-tools/internal/bus/caller.go:15-19). node is one subject token: an empty one, or one
// holding '.', '*', '>' or white space, is refused before anything is published.
func (c *Conn) CallOn(node, key string, body any) (json.RawMessage, error) {
if err := checkNode(node); err != nil {
return nil, err
}
called, fallback, err := callSubjectsOn(key, node, c.callerToken())
if err != nil {
return nil, err
}
return c.callOn(called, fallback, body)
}
func (c *Conn) callOn(called, fallback string, body any) (json.RawMessage, error) {
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
result, err := c.request(called, data)
if errors.Is(err, nats.ErrNoResponders) {
result, err = c.request(fallback, data)
}
return result, err
}
// callerToken is this account's user name as the last token of a call it publishes, each dot written ~
// (mesh-tools node-tools/internal/bus/caller.go:43, CallerToken).
func (c *Conn) callerToken() string { return strings.ReplaceAll(c.account.User, ".", "~") }
// callSubjects is the caller-named subject key is called on, and its plain fallback (mesh-tools
// node-tools/internal/bus/caller.go CalledSubject, toolShape, calledPattern).
func callSubjects(key, token string) (called, fallback string, err error) {
return callSubjectsOn(key, "", token)
}
// checkNode refuses a node name that is not one subject token.
func checkNode(node string) error {
if node == "" || strings.ContainsAny(node, ".*> \t\r\n") {
return fmt.Errorf("%q is not a node name (one subject token: no '.', '*', '>' or space)", node)
}
return nil
}
// callSubjectsOn is callSubjects for one named node ("" names none): the node is a token between the tool and
// the caller on the called subject, and the last one of the fallback.
func callSubjectsOn(key, node, token string) (called, fallback string, err error) {
at := ""
if node != "" {
at = "." + node
}
if rest, ok := strings.CutPrefix(key, "seat:"); ok {
seat, verb, ok := strings.Cut(rest, ".")
if !ok || seat == "" || verb == "" {
return "", "", fmt.Errorf("%q names no seat and verb (seat:<seat>.<verb>)", key)
}
return "mesh.seat." + seat + ".call." + verb + at + "." + token, "mesh.seat." + seat + ".tool." + verb + at, nil
}
module, tool, ok := strings.Cut(key, ".")
if !ok || module == "" || tool == "" {
return "", "", fmt.Errorf("%q is not <module>.<tool>, and does not start seat:<seat>.<verb>", key)
}
return "mesh.mod." + module + ".call." + tool + at + "." + token, "mesh.mod." + module + ".tool." + tool + at, nil
}
// request asks one subject once and reads its reply ({result, error, node}); a reply naming an error is
// the call's failure, not a transport one.
func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
ctx, cancel := context.WithTimeout(context.Background(), CallTimeout)
defer cancel()
msg, err := c.nc.RequestWithContext(ctx, subject, data)
if err != nil {
return nil, err
}
var r struct {
Result json.RawMessage `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
}
if err := json.Unmarshal(msg.Data, &r); err != nil {
return nil, fmt.Errorf("%s answered something that is not a seat's answer: %w", subject, err)
}
if r.Error != "" {
return nil, errors.New(r.Error)
}
return r.Result, nil
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/conn.go
Vendored
+164
View File
@@ -0,0 +1,164 @@
package process
import (
"context"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"errors"
"fmt"
"regexp"
"sync"
"github.com/nats-io/nats.go"
)
// Conn is one process's own connection to the bus (novox/hq ADR 0436 clause 4): dialled with the account
// the controller delivered, pinned to its fingerprint, on its own inbox prefix. It offers no Serve and no
// way to serve a tool — a process reaches the bus only as itself, and never answers a call.
type Conn struct {
nc *nats.Conn
account Account
mu sync.Mutex
next uint64
pending map[string]map[uint64]context.CancelCauseFunc
}
// errPermissionRefused marks a context cancelled because the bus refused this account's publish of the
// subject a pending call was waiting an answer on: a denied publish is refused only as an async protocol
// error with no reply (mesh-tools node-tools/internal/bus/caller.go publishRefused), so without this a
// call would wait out its whole timeout instead of failing at once.
var errPermissionRefused = errors.New("the bus refused this account's publish")
// refusedPublish reads the subject out of the bus's "-ERR 'Permissions Violation for Publish to
// "<subject>"'" (mesh-tools node-tools/internal/bus/caller.go's own regexp).
var refusedPublish = regexp.MustCompile(`(?i)Publish to "(\S+)"`)
// Connect dials the bus with account: pinned to its fingerprint when it carries one (refused with any
// other certificate), authenticated as its user and password, on the inbox prefix _INBOX.<user> and no
// other, reconnecting without limit as the tool runner does (novox/hq issue 1192: a process outlives a
// bus restart, never giving up the way the client library's own default of 60 attempts would). Ported
// from mesh-controller internal/broker/jetstream.go (DialPinned, PinnedToFingerprint), which now uses this one since this
// package may not import the controller.
//
// An ErrorHandler among opts is chained behind this package's own, which watches every async error for a
// permission violation refusing a publish this connection is waiting an answer to, and cancels that wait
// at once.
func Connect(account Account, opts ...nats.Option) (*Conn, error) {
c := &Conn{account: account, pending: map[string]map[uint64]context.CancelCauseFunc{}}
nc, err := nats.Connect(account.URL, c.options(opts)...)
if err != nil {
return nil, fmt.Errorf("connecting the process account %s: %w", account.User, err)
}
c.nc = nc
return c, nil
}
// options is the full option list Connect dials with, apart from Connect so a unit test can check what
// it carries without a live bus: this package's own (the inbox prefix, the credentials, the pin,
// reconnecting without limit, and an ErrorHandler chaining any the caller passed), then opts, then this
// package's own ErrorHandler again last, so it is the one AsyncErrorCB nats.Connect keeps however opts
// ordered theirs.
func (c *Conn) options(opts []nats.Option) []nats.Option {
// opts are functions over a *nats.Options; applied to a scratch value, this reads off a caller's own
// ErrorHandler (if any) to chain, without needing nats.Connect's own URL/Servers parsing here.
var scratch nats.Options
for _, opt := range opts {
if opt != nil {
_ = opt(&scratch)
}
}
chained := scratch.AsyncErrorCB
all := []nats.Option{
nats.CustomInboxPrefix(c.account.InboxPrefix()),
nats.UserInfo(c.account.User, c.account.Password),
nats.MaxReconnects(-1),
}
if c.account.Fingerprint != "" {
all = append(all, nats.Secure(PinnedToFingerprint(c.account.Fingerprint)))
}
all = append(all, opts...)
return append(all, nats.ErrorHandler(func(nc *nats.Conn, sub *nats.Subscription, err error) {
c.publishRefused(err)
if chained != nil {
chained(nc, sub, err)
}
}))
}
// Close ends the connection.
func (c *Conn) Close() { c.nc.Close() }
// withRefusal derives a context from ctx that is also cancelled, with errPermissionRefused as its cause,
// the moment the bus refuses this account's publish of subject — before ctx's own deadline, which a
// denied publish would otherwise run out, refused only as an async error with no reply. Every call that
// requests on the bus registers the subject it is about to publish to here first.
func (c *Conn) withRefusal(ctx context.Context, subject string) (context.Context, func()) {
derived, cancel := context.WithCancelCause(ctx)
c.mu.Lock()
c.next++
id := c.next
if c.pending[subject] == nil {
c.pending[subject] = map[uint64]context.CancelCauseFunc{}
}
c.pending[subject][id] = cancel
c.mu.Unlock()
return derived, func() {
c.mu.Lock()
if list := c.pending[subject]; list != nil {
delete(list, id)
if len(list) == 0 {
delete(c.pending, subject)
}
}
c.mu.Unlock()
cancel(nil)
}
}
// publishRefused ends every call waiting on the subject the bus's error names, with errPermissionRefused
// as the cause (mesh-tools node-tools/internal/bus/caller.go publishRefused): true when there was one
// waiting.
func (c *Conn) publishRefused(err error) bool {
if !errors.Is(err, nats.ErrPermissionViolation) {
return false
}
m := refusedPublish.FindStringSubmatch(err.Error())
if len(m) < 2 {
return false
}
c.mu.Lock()
cancels := c.pending[m[1]]
delete(c.pending, m[1])
c.mu.Unlock()
for _, cancel := range cancels {
cancel(errPermissionRefused)
}
return len(cancels) > 0
}
// PinnedToFingerprint accepts exactly the certificate with this SHA-256 and no other — the leaf alone, its
// name and chain deliberately not consulted, as mesh-controller's own pin does (jetstream.go:116-132).
// PinnedToFingerprint is public so that a process dialling the bus on its own terms, and the controller,
// share this one pin (novox/hq issue 1633).
func PinnedToFingerprint(want string) *tls.Config {
return &tls.Config{
InsecureSkipVerify: true, //nolint:gosec // replaced by the pin below, which is stricter
MinVersion: tls.VersionTLS12,
VerifyPeerCertificate: func(rawCerts [][]byte, _ [][]*x509.Certificate) error {
if len(rawCerts) == 0 {
return errors.New("the bus presented no certificate")
}
sum := sha256.Sum256(rawCerts[0])
got := "sha256:" + hex.EncodeToString(sum[:])
if got != want {
return fmt.Errorf("the bus presented a certificate this mesh does not know (%s…), expected %s…",
got[:23], want[:23])
}
return nil
},
}
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/emit.go
Vendored
+69
View File
@@ -0,0 +1,69 @@
package process
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Emit publishes an event as this process's module (novox/hq ADR 0436 clause 4), under its module's own
// namespace (mesh.mod.<module>.event.<type>, the same subject the module's own principal would publish it
// on), with the SDK's own envelope and headers (stdio.go:498-509). A chosen message id
// (EmitOptions.MessageID) is sent as x-event-id, and as the bus's own dedup id (Nats-Msg-Id), so a second
// publish inside the stream's duplicate window is dropped; empty is a random one, as stdio.Emit on main.
func (c *Conn) Emit(eventType string, body any, opts ...stdio.EmitOptions) error {
data, err := json.Marshal(body)
if err != nil {
return err
}
id := ""
for _, o := range opts {
if o.MessageID != "" {
id = o.MessageID
}
}
if id == "" {
id = eventID()
}
msg := nats.NewMsg("mesh.mod." + c.account.Module + ".event." + eventType)
msg.Header.Set("x-event-id", id)
msg.Header.Set("x-source", c.account.Module)
msg.Header.Set("x-node", c.account.Node)
msg.Header.Set("x-time", time.Now().UTC().Format(time.RFC3339Nano))
msg.Header.Set("content-type", "application/json")
for _, o := range opts {
if o.CausationID != "" {
msg.Header.Set("x-causation-id", o.CausationID)
}
if o.Schema != "" {
msg.Header.Set("x-schema", o.Schema)
}
}
msg.Data = data
js, err := jetstream.New(c.nc)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
defer cancel()
_, err = js.PublishMsg(ctx, msg, jetstream.WithMsgID(id))
return err
}
// eventID is a random UUID, version 4, as stdio's own eventID (go/stdio.go:515-522) — duplicated rather
// than imported, since stdio's is unexported and this package touches none of stdio's bundle machinery.
func eventID() string {
var b [16]byte
_, _ = rand.Read(b[:])
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
h := hex.EncodeToString(b[:])
return h[0:8] + "-" + h[8:12] + "-" + h[12:16] + "-" + h[16:20] + "-" + h[20:32]
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/follow.go
Vendored
+53
View File
@@ -0,0 +1,53 @@
package process
import (
"encoding/json"
"fmt"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/membership"
)
// FollowMembership reads this account's own membership and follows it (novox/hq ADR 0160): apply is called
// with every membership the mesh issues this account's node and module, the one already issued first and
// each later one as it is published, so a change reaches a running process without a restart. A body that is
// not a membership is handed to onBad (nil to ignore it) and skipped.
//
// The subject is followed before it is read, so an issue landing between the two is applied, not missed. The
// read is the subject-addressed direct get, the one request this account may make of the stream; no
// membership issued yet, or the read failing, is not an error: the process follows the subject and is told
// when one arrives. The returned error is a refusal to follow at all.
func (c *Conn) FollowMembership(apply func(membership.Membership), onBad func(error)) error {
if c.account.Node == "" || c.account.Module == "" {
return fmt.Errorf("the account names no node or module, so it has no membership")
}
bad := func(err error) {
if onBad != nil {
onBad(err)
}
}
take := func(body []byte) {
var issued membership.Membership
if err := json.Unmarshal(body, &issued); err != nil {
bad(fmt.Errorf("a membership arrived that is not one: %w", err))
return
}
apply(issued)
}
subject := membership.Subject(c.account.Node, c.account.Module)
if _, err := c.nc.Subscribe(subject, func(m *nats.Msg) { take(m.Data) }); err != nil {
return fmt.Errorf("cannot follow %s: %w", subject, err)
}
got, err := c.nc.Request(membership.DirectGetSubject(c.account.Node, c.account.Module), nil, 5*time.Second)
switch {
case err != nil:
bad(fmt.Errorf("cannot read the membership issued on %s yet (%v); following it", subject, err))
case got.Header.Get("Status") != "" || len(got.Data) == 0:
// None issued yet: the follow above delivers it.
default:
take(got.Data)
}
return nil
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/queue.go
Vendored
+127
View File
@@ -0,0 +1,127 @@
package process
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// Message is one piece of work this process's queue consumer delivered, with how many times the bus has
// delivered it. Nothing acts on its behalf: the handler decides its fate exactly once, by calling Ack,
// NakWithDelay or Term (mesh-tools node-tools/internal/bus/seat.go's seat work Consume, with the same
// three outcomes, offered here explicitly instead of by a handler's return value, since the caller
// chooses its own delay before a message comes back).
type Message struct {
Subject string
Body []byte
Headers map[string]string
// Delivered is how many times the bus has handed this message over, this time included.
Delivered uint64
ack func() error
nak func(time.Duration) error
term func() error
}
// Ack acknowledges the message: the bus never delivers it again.
func (m *Message) Ack() error { return m.ack() }
// NakWithDelay asks for the message to be delivered again no sooner than delay, its delivery count
// raised by one.
func (m *Message) NakWithDelay(delay time.Duration) error { return m.nak(delay) }
// Term terminates the message: the bus never delivers it again, and does not count it a dead letter the
// way running out of deliveries does.
func (m *Message) Term() error { return m.term() }
// ProcessID is this account's own process id, the last segment of its user name <node>.<module>.<id>
// (mesh-controller internal/broker, Principal.Username for KindProcess).
func (a Account) ProcessID() string {
prefix := a.Node + "." + a.Module + "."
if !strings.HasPrefix(a.User, prefix) {
return ""
}
return strings.TrimPrefix(a.User, prefix)
}
// QueueConsumerName is the one durable consumer a process's bus.queues names it on its work-queue stream
// (novox/hq ADR 0436 clauses 1, 2, 6; mesh-controller internal/broker/modstreams.go QueueConsumer):
// <node>_<module>_<process id>.
func (a Account) QueueConsumerName() string { return a.Node + "_" + a.Module + "_" + a.ProcessID() }
// errNoMessage is one pull that fetched nothing within its wait: not an error to the caller, just a
// reason to pull again.
var errNoMessage = errors.New("no message this pull")
// Consume binds this process's one durable consumer on stream (QueueConsumerName) and pulls from it, one
// message at a time, handing each to handle until ctx ends: it then stops pulling and returns ctx.Err(),
// never acknowledging a message still in handle's hands — only handle decides a message's fate, by
// calling Ack, NakWithDelay or Term on it.
func (c *Conn) Consume(ctx context.Context, stream string, handle func(*Message)) error {
js, err := jetstream.New(c.nc)
if err != nil {
return err
}
name := c.account.QueueConsumerName()
cons, err := js.Consumer(ctx, stream, name)
if err != nil {
return fmt.Errorf("binding %s's queue consumer %s on %s: %w", c.account.Module, name, stream, err)
}
return runQueue(ctx, fetchOne(cons), handle)
}
// fetchOne is one pull of at most one message, non-blocking past ctx: Fetch itself does not take a
// context, so this selects on its message channel and ctx together, and returns promptly once ctx ends
// even while a fetch is still outstanding.
func fetchOne(cons jetstream.Consumer) func(ctx context.Context) (*Message, error) {
return func(ctx context.Context) (*Message, error) {
batch, err := cons.Fetch(1, jetstream.FetchMaxWait(5*time.Second))
if err != nil {
return nil, err
}
select {
case msg, ok := <-batch.Messages():
if !ok {
return nil, errNoMessage
}
headers := map[string]string{}
for k := range msg.Headers() {
headers[k] = msg.Headers().Get(k)
}
var delivered uint64 = 1
if meta, err := msg.Metadata(); err == nil && meta != nil {
delivered = meta.NumDelivered
}
return &Message{Subject: msg.Subject(), Body: append([]byte(nil), msg.Data()...), Headers: headers,
Delivered: delivered, ack: msg.Ack, nak: msg.NakWithDelay, term: msg.Term}, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
}
// runQueue is Consume's loop, apart from the bus so a unit test can drive it with a fake next
// (novox/hq issue 1190 AC4): it checks ctx before every pull, and stops as soon as next or the check
// says ctx has ended, without touching the message past handing it to handle.
func runQueue(ctx context.Context, next func(context.Context) (*Message, error), handle func(*Message)) error {
for {
if err := ctx.Err(); err != nil {
return err
}
msg, err := next(ctx)
if err != nil {
if ctx.Err() != nil {
return ctx.Err()
}
if errors.Is(err, errNoMessage) {
continue
}
return err
}
handle(msg)
}
}
modules/route-proxy/vendor/git.novox.be/novox/mesh-sdk/go/process/state.go
Vendored
+130
View File
@@ -0,0 +1,130 @@
package process
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// stateTimeout bounds a state call (Get, Create, Update) and an Emit (emit.go): a denied publish is
// caught at once by Conn's own ErrorHandler (conn.go withRefusal/publishRefused), never waited out, so
// this stays the ordinary request bound and is not exposed for a test to shrink.
const stateTimeout = 30 * time.Second
// State is one of this process's module's states, read and written on this connection (novox/hq ADR 0436
// clause 4), in the bucket <module>_<name> (mesh-controller internal/broker/state.go:49) — only as the
// account's grants allow. It satisfies go/asker's Store.
type State struct {
Module string
Name string
// kv binds this state's underlying key-value store: a live connection's own (Conn.State), or, in a
// unit test, a fake that never reaches a bus.
kv func(ctx context.Context) (jetstream.KeyValue, error)
}
// bucket is this state's name on the bus.
func (s State) bucket() string { return s.Module + "_" + s.Name }
// State is the named state this connection's module keeps, bound lazily to its bucket on each call so a
// bucket the account is not granted fails at the call that needed it, naming the bucket. The bind itself
// (js.KeyValue) is what asks $JS.API.STREAM.INFO.KV_<bucket>, so it is what withRefusal watches: a bucket
// not granted is refused there, promptly, as nats.ErrPermissionViolation.
func (c *Conn) State(name string) State {
bucket := c.account.Module + "_" + name
return State{Module: c.account.Module, Name: name, kv: func(ctx context.Context) (jetstream.KeyValue, error) {
js, err := jetstream.New(c.nc)
if err != nil {
return nil, err
}
subject := "$JS.API.STREAM.INFO.KV_" + bucket
ctx, stop := c.withRefusal(ctx, subject)
defer stop()
kv, err := js.KeyValue(ctx, bucket)
if err != nil {
if errors.Is(context.Cause(ctx), errPermissionRefused) {
return nil, fmt.Errorf("%w: publishing %s", nats.ErrPermissionViolation, subject)
}
return nil, err
}
return kv, nil
}}
}
func (s State) store(ctx context.Context) (jetstream.KeyValue, error) {
kv, err := s.kv(ctx)
if err != nil {
return nil, fmt.Errorf("%s's state %s (bucket %s): %w", s.Module, s.Name, s.bucket(), err)
}
return kv, nil
}
// Get is a key's current value, or nil when it has none.
func (s State) Get(key string) (*stdio.StateEntry, error) {
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
defer cancel()
kv, err := s.store(ctx)
if err != nil {
return nil, err
}
entry, err := kv.Get(ctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
}
return &stdio.StateEntry{Key: key, Value: json.RawMessage(entry.Value()), Revision: entry.Revision()}, nil
}
// Create writes a key only when it has none, and returns the revision; a key that has one is refused as
// stdio.ErrChanged.
func (s State) Create(key string, value any) (uint64, error) {
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
defer cancel()
kv, err := s.store(ctx)
if err != nil {
return 0, err
}
data, err := json.Marshal(value)
if err != nil {
return 0, err
}
rev, err := kv.Create(ctx, key, data)
if errors.Is(err, jetstream.ErrKeyExists) {
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
}
if err != nil {
return 0, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
}
return rev, nil
}
// Update writes a key only when it is still at revision, and returns the new one; a key changed meanwhile
// is refused as stdio.ErrChanged.
func (s State) Update(key string, value any, revision uint64) (uint64, error) {
ctx, cancel := context.WithTimeout(context.Background(), stateTimeout)
defer cancel()
kv, err := s.store(ctx)
if err != nil {
return 0, err
}
data, err := json.Marshal(value)
if err != nil {
return 0, err
}
rev, err := kv.Update(ctx, key, data, revision)
if errors.Is(err, jetstream.ErrKeyRevisionMismatch) {
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
}
if err != nil {
return 0, fmt.Errorf("%s's state %s, key %s: %w", s.Module, s.Name, key, err)
}
return rev, nil
}
modules/route-proxy/vendor/github.com/klauspost/compress/LICENSE
Generated Vendored
+304
View File
@@ -0,0 +1,304 @@
Copyright (c) 2012 The Go Authors. All rights reserved.
Copyright (c) 2019 Klaus Post. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
------------------
Files: gzhttp/*
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016-2017 The New York Times Company
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
------------------
Files: s2/cmd/internal/readahead/*
The MIT License (MIT)
Copyright (c) 2015 Klaus Post
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
---------------------
Files: snappy/*
Files: internal/snapref/*
Copyright (c) 2011 The Snappy-Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-----------------
Files: s2/cmd/internal/filepathx/*
Copyright 2016 The filepathx Authors
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
modules/route-proxy/vendor/github.com/klauspost/compress/flate/deflate.go
Generated Vendored
+997
View File
@@ -0,0 +1,997 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Copyright (c) 2015 Klaus Post
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
import (
"errors"
"fmt"
"io"
"math"
"github.com/klauspost/compress/internal/le"
)
const (
NoCompression = 0
BestSpeed = 1
BestCompression = 9
DefaultCompression = -1
// HuffmanOnly disables Lempel-Ziv match searching and only performs Huffman
// entropy encoding. This mode is useful in compressing data that has
// already been compressed with an LZ style algorithm (e.g. Snappy or LZ4)
// that lacks an entropy encoder. Compression gains are achieved when
// certain bytes in the input stream occur more frequently than others.
//
// Note that HuffmanOnly produces a compressed output that is
// RFC 1951 compliant. That is, any valid DEFLATE decompressor will
// continue to be able to decompress this output.
HuffmanOnly = -2
ConstantCompression = HuffmanOnly // compatibility alias.
logWindowSize = 15
windowSize = 1 << logWindowSize
windowMask = windowSize - 1
logMaxOffsetSize = 15 // Standard DEFLATE
minMatchLength = 4 // The smallest match that the compressor looks for
maxMatchLength = 258 // The longest match for the compressor
minOffsetSize = 1 // The shortest offset that makes any sense
// The maximum number of tokens we will encode at the time.
// Smaller sizes usually creates less optimal blocks.
// Bigger can make context switching slow.
// We use this for levels 7-9, so we make it big.
maxFlateBlockTokens = 1 << 15
maxStoreBlockSize = 65535
hashBits = 17 // After 17 performance degrades
hashSize = 1 << hashBits
hashMask = (1 << hashBits) - 1
hashShift = (hashBits + minMatchLength - 1) / minMatchLength
maxHashOffset = 1 << 28
skipNever = math.MaxInt32
debugDeflate = false
)
type compressionLevel struct {
good, lazy, nice, chain, fastSkipHashing, level int
}
// Compression levels have been rebalanced from zlib deflate defaults
// to give a bigger spread in speed and compression.
// See https://blog.klauspost.com/rebalancing-deflate-compression-levels/
var levels = []compressionLevel{
{}, // 0
// Level 1-6 uses specialized algorithm - values not used
{0, 0, 0, 0, 0, 1},
{0, 0, 0, 0, 0, 2},
{0, 0, 0, 0, 0, 3},
{0, 0, 0, 0, 0, 4},
{0, 0, 0, 0, 0, 5},
{0, 0, 0, 0, 0, 6},
// Levels 7-9 use increasingly more lazy matching
// and increasingly stringent conditions for "good enough".
{8, 12, 16, 24, skipNever, 7},
{16, 30, 40, 64, skipNever, 8},
{32, 258, 258, 1024, skipNever, 9},
}
// advancedState contains state for the advanced levels, with bigger hash tables, etc.
type advancedState struct {
// deflate state
length int
offset int
maxInsertIndex int
chainHead int
hashOffset int
ii uint16 // position of last match, intended to overflow to reset.
// input window: unprocessed data is window[index:windowEnd]
index int
hashMatch [maxMatchLength + minMatchLength]uint32
// Input hash chains
// hashHead[hashValue] contains the largest inputIndex with the specified hash value
// If hashHead[hashValue] is within the current window, then
// hashPrev[hashHead[hashValue] & windowMask] contains the previous index
// with the same hash value.
hashHead [hashSize]uint32
hashPrev [windowSize]uint32
}
type compressor struct {
compressionLevel
h *huffmanEncoder
w *huffmanBitWriter
// compression algorithm
fill func(*compressor, []byte) int // copy data to window
step func(*compressor) // process window
window []byte
windowEnd int
blockStart int // window index where current tokens start
err error
// queued output tokens
tokens tokens
fast fastEnc
state *advancedState
sync bool // requesting flush
byteAvailable bool // if true, still need to process window[index-1].
}
func (d *compressor) fillDeflate(b []byte) int {
s := d.state
if s.index >= 2*windowSize-(minMatchLength+maxMatchLength) {
// shift the window by windowSize
//copy(d.window[:], d.window[windowSize:2*windowSize])
*(*[windowSize]byte)(d.window) = *(*[windowSize]byte)(d.window[windowSize:])
s.index -= windowSize
d.windowEnd -= windowSize
if d.blockStart >= windowSize {
d.blockStart -= windowSize
} else {
d.blockStart = math.MaxInt32
}
s.hashOffset += windowSize
if s.hashOffset > maxHashOffset {
delta := s.hashOffset - 1
s.hashOffset -= delta
s.chainHead -= delta
// Iterate over slices instead of arrays to avoid copying
// the entire table onto the stack (Issue #18625).
for i, v := range s.hashPrev[:] {
if int(v) > delta {
s.hashPrev[i] = uint32(int(v) - delta)
} else {
s.hashPrev[i] = 0
}
}
for i, v := range s.hashHead[:] {
if int(v) > delta {
s.hashHead[i] = uint32(int(v) - delta)
} else {
s.hashHead[i] = 0
}
}
}
}
n := copy(d.window[d.windowEnd:], b)
d.windowEnd += n
return n
}
func (d *compressor) writeBlock(tok *tokens, index int, eof bool) error {
if index > 0 || eof {
var window []byte
if d.blockStart <= index {
window = d.window[d.blockStart:index]
}
d.blockStart = index
//d.w.writeBlock(tok, eof, window)
d.w.writeBlockDynamic(tok, eof, window, d.sync)
return d.w.err
}
return nil
}
// writeBlockSkip writes the current block and uses the number of tokens
// to determine if the block should be stored on no matches, or
// only huffman encoded.
func (d *compressor) writeBlockSkip(tok *tokens, index int, eof bool) error {
if index > 0 || eof {
if d.blockStart <= index {
window := d.window[d.blockStart:index]
// If we removed less than a 64th of all literals
// we huffman compress the block.
if int(tok.n) > len(window)-int(tok.n>>6) {
d.w.writeBlockHuff(eof, window, d.sync)
} else {
// Write a dynamic huffman block.
d.w.writeBlockDynamic(tok, eof, window, d.sync)
}
} else {
d.w.writeBlock(tok, eof, nil)
}
d.blockStart = index
return d.w.err
}
return nil
}
// fillWindow will fill the current window with the supplied
// dictionary and calculate all hashes.
// This is much faster than doing a full encode.
// Should only be used after a start/reset.
func (d *compressor) fillWindow(b []byte) {
// Do not fill window if we are in store-only or huffman mode.
if d.level <= 0 && d.level > -MinCustomWindowSize {
return
}
if d.fast != nil {
// encode the last data, but discard the result
if len(b) > maxMatchOffset {
b = b[len(b)-maxMatchOffset:]
}
d.fast.Encode(&d.tokens, b)
d.tokens.Reset()
return
}
s := d.state
// If we are given too much, cut it.
if len(b) > windowSize {
b = b[len(b)-windowSize:]
}
// Add all to window.
n := copy(d.window[d.windowEnd:], b)
// Calculate 256 hashes at the time (more L1 cache hits)
loops := (n + 256 - minMatchLength) / 256
for j := range loops {
startindex := j * 256
end := min(startindex+256+minMatchLength-1, n)
tocheck := d.window[startindex:end]
dstSize := len(tocheck) - minMatchLength + 1
if dstSize <= 0 {
continue
}
dst := s.hashMatch[:dstSize]
bulkHash4(tocheck, dst)
var newH uint32
for i, val := range dst {
di := i + startindex
newH = val & hashMask
// Get previous value with the same hash.
// Our chain should point to the previous value.
s.hashPrev[di&windowMask] = s.hashHead[newH]
// Set the head of the hash chain to us.
s.hashHead[newH] = uint32(di + s.hashOffset)
}
}
// Update window information.
d.windowEnd += n
d.blockStart = d.windowEnd
s.index = n
}
// Try to find a match starting at index whose length is greater than prevSize.
// We only look at chainCount possibilities before giving up.
// pos = s.index, prevHead = s.chainHead-s.hashOffset, prevLength=minMatchLength-1, lookahead
func (d *compressor) findMatch(pos int, prevHead int, lookahead int) (length, offset int, ok bool) {
minMatchLook := min(lookahead, maxMatchLength)
win := d.window[0 : pos+minMatchLook]
// We quit when we get a match that's at least nice long
nice := min(d.nice, len(win)-pos)
// If we've got a match that's good enough, only look in 1/4 the chain.
tries := d.chain
length = minMatchLength - 1
wEnd := win[pos+length]
wPos := win[pos:]
minIndex := max(pos-windowSize, 0)
offset = 0
if d.chain < 100 {
for i := prevHead; tries > 0; tries-- {
if wEnd == win[i+length] {
n := matchLen(win[i:i+minMatchLook], wPos)
if n > length {
length = n
offset = pos - i
ok = true
if n >= nice {
// The match is good enough that we don't try to find a better one.
break
}
wEnd = win[pos+n]
}
}
if i <= minIndex {
// hashPrev[i & windowMask] has already been overwritten, so stop now.
break
}
i = int(d.state.hashPrev[i&windowMask]) - d.state.hashOffset
if i < minIndex {
break
}
}
return
}
// Minimum gain to accept a match.
cGain := 4
// Some like it higher (CSV), some like it lower (JSON)
const baseCost = 3
// Base is 4 bytes at with an additional cost.
// Matches must be better than this.
for i := prevHead; tries > 0; tries-- {
if wEnd == win[i+length] {
n := matchLen(win[i:i+minMatchLook], wPos)
if n > length {
// Calculate gain. Estimate
newGain := d.h.bitLengthRaw(wPos[:n]) - int(offsetExtraBits[offsetCode(uint32(pos-i))]) - baseCost - int(lengthExtraBits[lengthCodes[(n-3)&255]])
//fmt.Println("gain:", newGain, "prev:", cGain, "raw:", d.h.bitLengthRaw(wPos[:n]), "this-len:", n, "prev-len:", length)
if newGain > cGain {
length = n
offset = pos - i
cGain = newGain
ok = true
if n >= nice {
// The match is good enough that we don't try to find a better one.
break
}
wEnd = win[pos+n]
}
}
}
if i <= minIndex {
// hashPrev[i & windowMask] has already been overwritten, so stop now.
break
}
i = int(d.state.hashPrev[i&windowMask]) - d.state.hashOffset
if i < minIndex {
break
}
}
return
}
func (d *compressor) writeStoredBlock(buf []byte) error {
if d.w.writeStoredHeader(len(buf), false); d.w.err != nil {
return d.w.err
}
d.w.writeBytes(buf)
return d.w.err
}
// hash4 returns a hash representation of the first 4 bytes
// of the supplied slice.
// The caller must ensure that len(b) >= 4.
func hash4(b []byte) uint32 {
return hash4u(le.Load32(b, 0), hashBits)
}
// hash4 returns the hash of u to fit in a hash table with h bits.
// Preferably h should be a constant and should always be <32.
func hash4u(u uint32, h uint8) uint32 {
return (u * prime4bytes) >> (32 - h)
}
// bulkHash4 will compute hashes using the same
// algorithm as hash4
func bulkHash4(b []byte, dst []uint32) {
if len(b) < 4 {
return
}
hb := le.Load32(b, 0)
dst[0] = hash4u(hb, hashBits)
end := len(b) - 4 + 1
for i := 1; i < end; i++ {
hb = (hb >> 8) | uint32(b[i+3])<<24
dst[i] = hash4u(hb, hashBits)
}
}
func (d *compressor) initDeflate() {
d.window = make([]byte, 2*windowSize)
d.byteAvailable = false
d.err = nil
if d.state == nil {
return
}
s := d.state
s.index = 0
s.hashOffset = 1
s.length = minMatchLength - 1
s.offset = 0
s.chainHead = -1
}
// deflateLazy is the same as deflate, but with d.fastSkipHashing == skipNever,
// meaning it always has lazy matching on.
func (d *compressor) deflateLazy() {
s := d.state
// Sanity enables additional runtime tests.
// It's intended to be used during development
// to supplement the currently ad-hoc unit tests.
const sanity = debugDeflate
if d.windowEnd-s.index < minMatchLength+maxMatchLength && !d.sync {
return
}
if d.windowEnd != s.index && d.chain > 100 {
// Get literal huffman coder.
if d.h == nil {
d.h = newHuffmanEncoder(maxFlateBlockTokens)
}
var tmp [256]uint16
toIndex := d.window[s.index:d.windowEnd]
toIndex = toIndex[:min(len(toIndex), maxFlateBlockTokens)]
for _, v := range toIndex {
tmp[v]++
}
d.h.generate(tmp[:], 15)
}
s.maxInsertIndex = d.windowEnd - (minMatchLength - 1)
for {
if sanity && s.index > d.windowEnd {
panic("index > windowEnd")
}
lookahead := d.windowEnd - s.index
if lookahead < minMatchLength+maxMatchLength {
if !d.sync {
return
}
if sanity && s.index > d.windowEnd {
panic("index > windowEnd")
}
if lookahead == 0 {
// Flush current output block if any.
if d.byteAvailable {
// There is still one pending token that needs to be flushed
d.tokens.AddLiteral(d.window[s.index-1])
d.byteAvailable = false
}
if d.tokens.n > 0 {
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
return
}
}
if s.index < s.maxInsertIndex {
// Update the hash
hash := hash4(d.window[s.index:])
ch := s.hashHead[hash]
s.chainHead = int(ch)
s.hashPrev[s.index&windowMask] = ch
s.hashHead[hash] = uint32(s.index + s.hashOffset)
}
prevLength := s.length
prevOffset := s.offset
s.length = minMatchLength - 1
s.offset = 0
minIndex := max(s.index-windowSize, 0)
if s.chainHead-s.hashOffset >= minIndex && lookahead > prevLength && prevLength < d.lazy {
if newLength, newOffset, ok := d.findMatch(s.index, s.chainHead-s.hashOffset, lookahead); ok {
s.length = newLength
s.offset = newOffset
}
}
if prevLength >= minMatchLength && s.length <= prevLength {
// No better match, but check for better match at end...
//
// Skip forward a number of bytes.
// Offset of 2 seems to yield best results. 3 is sometimes better.
const checkOff = 2
// Check all, except full length
if prevLength < maxMatchLength-checkOff {
prevIndex := s.index - 1
if prevIndex+prevLength < s.maxInsertIndex {
end := min(lookahead, maxMatchLength+checkOff)
end += prevIndex
// Hash at match end.
h := hash4(d.window[prevIndex+prevLength:])
ch2 := int(s.hashHead[h]) - s.hashOffset - prevLength
if prevIndex-ch2 != prevOffset && ch2 > minIndex+checkOff {
length := matchLen(d.window[prevIndex+checkOff:end], d.window[ch2+checkOff:])
// It seems like a pure length metric is best.
if length > prevLength {
prevLength = length
prevOffset = prevIndex - ch2
// Extend back...
for i := checkOff - 1; i >= 0; i-- {
if prevLength >= maxMatchLength || d.window[prevIndex+i] != d.window[ch2+i] {
// Emit tokens we "owe"
for j := 0; j <= i; j++ {
d.tokens.AddLiteral(d.window[prevIndex+j])
if d.tokens.n == maxFlateBlockTokens {
// The block includes the current character
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
s.index++
if s.index < s.maxInsertIndex {
h := hash4(d.window[s.index:])
ch := s.hashHead[h]
s.chainHead = int(ch)
s.hashPrev[s.index&windowMask] = ch
s.hashHead[h] = uint32(s.index + s.hashOffset)
}
}
break
} else {
prevLength++
}
}
} else if false {
// Check one further ahead.
// Only rarely better, disabled for now.
prevIndex++
h := hash4(d.window[prevIndex+prevLength:])
ch2 := int(s.hashHead[h]) - s.hashOffset - prevLength
if prevIndex-ch2 != prevOffset && ch2 > minIndex+checkOff {
length := matchLen(d.window[prevIndex+checkOff:end], d.window[ch2+checkOff:])
// It seems like a pure length metric is best.
if length > prevLength+checkOff {
prevLength = length
prevOffset = prevIndex - ch2
prevIndex--
// Extend back...
for i := checkOff; i >= 0; i-- {
if prevLength >= maxMatchLength || d.window[prevIndex+i] != d.window[ch2+i-1] {
// Emit tokens we "owe"
for j := 0; j <= i; j++ {
d.tokens.AddLiteral(d.window[prevIndex+j])
if d.tokens.n == maxFlateBlockTokens {
// The block includes the current character
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
s.index++
if s.index < s.maxInsertIndex {
h := hash4(d.window[s.index:])
ch := s.hashHead[h]
s.chainHead = int(ch)
s.hashPrev[s.index&windowMask] = ch
s.hashHead[h] = uint32(s.index + s.hashOffset)
}
}
break
} else {
prevLength++
}
}
}
}
}
}
}
}
// There was a match at the previous step, and the current match is
// not better. Output the previous match.
d.tokens.AddMatch(uint32(prevLength-3), uint32(prevOffset-minOffsetSize))
// Insert in the hash table all strings up to the end of the match.
// index and index-1 are already inserted. If there is not enough
// lookahead, the last two strings are not inserted into the hash
// table.
newIndex := s.index + prevLength - 1
// Calculate missing hashes
end := min(newIndex, s.maxInsertIndex)
end += minMatchLength - 1
startindex := min(s.index+1, s.maxInsertIndex)
tocheck := d.window[startindex:end]
dstSize := len(tocheck) - minMatchLength + 1
if dstSize > 0 {
dst := s.hashMatch[:dstSize]
bulkHash4(tocheck, dst)
var newH uint32
for i, val := range dst {
di := i + startindex
newH = val & hashMask
// Get previous value with the same hash.
// Our chain should point to the previous value.
s.hashPrev[di&windowMask] = s.hashHead[newH]
// Set the head of the hash chain to us.
s.hashHead[newH] = uint32(di + s.hashOffset)
}
}
s.index = newIndex
d.byteAvailable = false
s.length = minMatchLength - 1
if d.tokens.n == maxFlateBlockTokens {
// The block includes the current character
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
s.ii = 0
} else {
// Reset, if we got a match this run.
if s.length >= minMatchLength {
s.ii = 0
}
// We have a byte waiting. Emit it.
if d.byteAvailable {
s.ii++
d.tokens.AddLiteral(d.window[s.index-1])
if d.tokens.n == maxFlateBlockTokens {
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
s.index++
// If we have a long run of no matches, skip additional bytes
// Resets when s.ii overflows after 64KB.
if n := int(s.ii) - d.chain; n > 0 {
n = 1 + int(n>>6)
for j := 0; j < n; j++ {
if s.index >= d.windowEnd-1 {
break
}
d.tokens.AddLiteral(d.window[s.index-1])
if d.tokens.n == maxFlateBlockTokens {
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
// Index...
if s.index < s.maxInsertIndex {
h := hash4(d.window[s.index:])
ch := s.hashHead[h]
s.chainHead = int(ch)
s.hashPrev[s.index&windowMask] = ch
s.hashHead[h] = uint32(s.index + s.hashOffset)
}
s.index++
}
// Flush last byte
d.tokens.AddLiteral(d.window[s.index-1])
d.byteAvailable = false
// s.length = minMatchLength - 1 // not needed, since s.ii is reset above, so it should never be > minMatchLength
if d.tokens.n == maxFlateBlockTokens {
if d.err = d.writeBlock(&d.tokens, s.index, false); d.err != nil {
return
}
d.tokens.Reset()
}
}
} else {
s.index++
d.byteAvailable = true
}
}
}
}
func (d *compressor) store() {
if d.windowEnd > 0 && (d.windowEnd == maxStoreBlockSize || d.sync) {
d.err = d.writeStoredBlock(d.window[:d.windowEnd])
d.windowEnd = 0
}
}
// fillWindow will fill the buffer with data for huffman-only compression.
// The number of bytes copied is returned.
func (d *compressor) fillBlock(b []byte) int {
n := copy(d.window[d.windowEnd:], b)
d.windowEnd += n
return n
}
// storeHuff will compress and store the currently added data,
// if enough has been accumulated or we at the end of the stream.
// Any error that occurred will be in d.err
func (d *compressor) storeHuff() {
if d.windowEnd < len(d.window) && !d.sync || d.windowEnd == 0 {
return
}
d.w.writeBlockHuff(false, d.window[:d.windowEnd], d.sync)
d.err = d.w.err
d.windowEnd = 0
}
// storeFast will compress and store the currently added data,
// if enough has been accumulated or we at the end of the stream.
// Any error that occurred will be in d.err
func (d *compressor) storeFast() {
// We only compress if we have maxStoreBlockSize.
if d.windowEnd < len(d.window) {
if !d.sync {
return
}
// Handle extremely small sizes.
if d.windowEnd < 128 {
if d.windowEnd == 0 {
return
}
if d.windowEnd <= 32 {
d.err = d.writeStoredBlock(d.window[:d.windowEnd])
} else {
d.w.writeBlockHuff(false, d.window[:d.windowEnd], true)
d.err = d.w.err
}
d.tokens.Reset()
d.windowEnd = 0
d.fast.Reset()
return
}
}
d.fast.Encode(&d.tokens, d.window[:d.windowEnd])
// If we made zero matches, store the block as is.
if d.tokens.n == 0 {
d.err = d.writeStoredBlock(d.window[:d.windowEnd])
// If we removed less than 1/16th, huffman compress the block.
} else if int(d.tokens.n) > d.windowEnd-(d.windowEnd>>4) {
d.w.writeBlockHuff(false, d.window[:d.windowEnd], d.sync)
d.err = d.w.err
} else {
d.w.writeBlockDynamic(&d.tokens, false, d.window[:d.windowEnd], d.sync)
d.err = d.w.err
}
d.tokens.Reset()
d.windowEnd = 0
}
// write will add input byte to the stream.
// Unless an error occurs all bytes will be consumed.
func (d *compressor) write(b []byte) (n int, err error) {
if d.err != nil {
return 0, d.err
}
n = len(b)
for len(b) > 0 {
if d.windowEnd == len(d.window) || d.sync {
d.step(d)
}
b = b[d.fill(d, b):]
if d.err != nil {
return 0, d.err
}
}
return n, d.err
}
func (d *compressor) syncFlush() error {
d.sync = true
if d.err != nil {
return d.err
}
d.step(d)
if d.err == nil {
d.w.writeStoredHeader(0, false)
d.w.flush()
d.err = d.w.err
}
d.sync = false
return d.err
}
func (d *compressor) init(w io.Writer, level int) (err error) {
d.w = newHuffmanBitWriter(w)
switch {
case level == NoCompression:
d.window = make([]byte, maxStoreBlockSize)
d.fill = (*compressor).fillBlock
d.step = (*compressor).store
case level == ConstantCompression:
d.w.logNewTablePenalty = 10
d.window = make([]byte, 32<<10)
d.fill = (*compressor).fillBlock
d.step = (*compressor).storeHuff
case level == DefaultCompression:
level = 5
fallthrough
case level >= 1 && level <= 6:
d.w.logNewTablePenalty = 7
d.fast = newFastEnc(level)
d.window = make([]byte, maxStoreBlockSize)
d.fill = (*compressor).fillBlock
d.step = (*compressor).storeFast
case 7 <= level && level <= 9:
d.w.logNewTablePenalty = 8
d.state = &advancedState{}
d.compressionLevel = levels[level]
d.initDeflate()
d.fill = (*compressor).fillDeflate
d.step = (*compressor).deflateLazy
case -level >= MinCustomWindowSize && -level <= MaxCustomWindowSize:
d.w.logNewTablePenalty = 7
d.fast = &fastEncL5Window{maxOffset: int32(-level), cur: maxStoreBlockSize}
d.window = make([]byte, maxStoreBlockSize)
d.fill = (*compressor).fillBlock
d.step = (*compressor).storeFast
default:
return fmt.Errorf("flate: invalid compression level %d: want value in range [-2, 9]", level)
}
d.level = level
return nil
}
// reset the state of the compressor.
func (d *compressor) reset(w io.Writer) {
d.w.reset(w)
d.sync = false
d.err = nil
// We only need to reset a few things for Snappy.
if d.fast != nil {
d.fast.Reset()
d.windowEnd = 0
d.tokens.Reset()
return
}
switch d.compressionLevel.chain {
case 0:
// level was NoCompression or ConstantCompression.
d.windowEnd = 0
default:
s := d.state
s.chainHead = -1
for i := range s.hashHead {
s.hashHead[i] = 0
}
for i := range s.hashPrev {
s.hashPrev[i] = 0
}
s.hashOffset = 1
s.index, d.windowEnd = 0, 0
d.blockStart, d.byteAvailable = 0, false
d.tokens.Reset()
s.length = minMatchLength - 1
s.offset = 0
s.ii = 0
s.maxInsertIndex = 0
}
}
func (d *compressor) close() error {
if d.err != nil {
return d.err
}
d.sync = true
d.step(d)
if d.err != nil {
return d.err
}
if d.w.writeStoredHeader(0, true); d.w.err != nil {
return d.w.err
}
d.w.flush()
d.w.reset(nil)
return d.w.err
}
// NewWriter returns a new Writer compressing data at the given level.
// Following zlib, levels range from 1 (BestSpeed) to 9 (BestCompression);
// higher levels typically run slower but compress more.
// Level 0 (NoCompression) does not attempt any compression; it only adds the
// necessary DEFLATE framing.
// Level -1 (DefaultCompression) uses the default compression level.
// Level -2 (ConstantCompression) will use Huffman compression only, giving
// a very fast compression for all types of input, but sacrificing considerable
// compression efficiency.
//
// If level is in the range [-2, 9] then the error returned will be nil.
// Otherwise the error returned will be non-nil.
func NewWriter(w io.Writer, level int) (*Writer, error) {
var dw Writer
if err := dw.d.init(w, level); err != nil {
return nil, err
}
return &dw, nil
}
// NewWriterDict is like NewWriter but initializes the new
// Writer with a preset dictionary. The returned Writer behaves
// as if the dictionary had been written to it without producing
// any compressed output. The compressed data written to w
// can only be decompressed by a Reader initialized with the
// same dictionary.
func NewWriterDict(w io.Writer, level int, dict []byte) (*Writer, error) {
zw, err := NewWriter(w, level)
if err != nil {
return nil, err
}
zw.d.fillWindow(dict)
zw.dict = append(zw.dict, dict...) // duplicate dictionary for Reset method.
return zw, err
}
// MinCustomWindowSize is the minimum window size that can be sent to NewWriterWindow.
const MinCustomWindowSize = 32
// MaxCustomWindowSize is the maximum custom window that can be sent to NewWriterWindow.
const MaxCustomWindowSize = windowSize
// NewWriterWindow returns a new Writer compressing data with a custom window size.
// windowSize must be from MinCustomWindowSize to MaxCustomWindowSize.
func NewWriterWindow(w io.Writer, windowSize int) (*Writer, error) {
if windowSize < MinCustomWindowSize {
return nil, errors.New("flate: requested window size less than MinWindowSize")
}
if windowSize > MaxCustomWindowSize {
return nil, errors.New("flate: requested window size bigger than MaxCustomWindowSize")
}
var dw Writer
if err := dw.d.init(w, -windowSize); err != nil {
return nil, err
}
return &dw, nil
}
// A Writer takes data written to it and writes the compressed
// form of that data to an underlying writer (see NewWriter).
type Writer struct {
d compressor
dict []byte
}
// Write writes data to w, which will eventually write the
// compressed form of data to its underlying writer.
func (w *Writer) Write(data []byte) (n int, err error) {
return w.d.write(data)
}
// Flush flushes any pending data to the underlying writer.
// It is useful mainly in compressed network protocols, to ensure that
// a remote reader has enough data to reconstruct a packet.
// Flush does not return until the data has been written.
// Calling Flush when there is no pending data still causes the Writer
// to emit a sync marker of at least 4 bytes.
// If the underlying writer returns an error, Flush returns that error.
//
// In the terminology of the zlib library, Flush is equivalent to Z_SYNC_FLUSH.
func (w *Writer) Flush() error {
// For more about flushing:
// http://www.bolet.org/~pornin/deflate-flush.html
return w.d.syncFlush()
}
// Close flushes and closes the writer.
func (w *Writer) Close() error {
return w.d.close()
}
// Reset discards the writer's state and makes it equivalent to
// the result of NewWriter or NewWriterDict called with dst
// and w's level and dictionary.
func (w *Writer) Reset(dst io.Writer) {
if len(w.dict) > 0 {
// w was created with NewWriterDict
w.d.reset(dst)
if dst != nil {
w.d.fillWindow(w.dict)
}
} else {
// w was created with NewWriter
w.d.reset(dst)
}
}
// ResetDict discards the writer's state and makes it equivalent to
// the result of NewWriter or NewWriterDict called with dst
// and w's level, but sets a specific dictionary.
func (w *Writer) ResetDict(dst io.Writer, dict []byte) {
w.dict = dict
w.d.reset(dst)
w.d.fillWindow(w.dict)
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/dict_decoder.go
Generated Vendored
+199
View File
@@ -0,0 +1,199 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
// dictDecoder implements the LZ77 sliding dictionary as used in decompression.
// LZ77 decompresses data through sequences of two forms of commands:
//
// - Literal insertions: Runs of one or more symbols are inserted into the data
// stream as is. This is accomplished through the writeByte method for a
// single symbol, or combinations of writeSlice/writeMark for multiple symbols.
// Any valid stream must start with a literal insertion if no preset dictionary
// is used.
//
// - Backward copies: Runs of one or more symbols are copied from previously
// emitted data. Backward copies come as the tuple (dist, length) where dist
// determines how far back in the stream to copy from and length determines how
// many bytes to copy. Note that it is valid for the length to be greater than
// the distance. Since LZ77 uses forward copies, that situation is used to
// perform a form of run-length encoding on repeated runs of symbols.
// The writeCopy and tryWriteCopy are used to implement this command.
//
// For performance reasons, this implementation performs little to no sanity
// checks about the arguments. As such, the invariants documented for each
// method call must be respected.
type dictDecoder struct {
hist []byte // Sliding window history
// Invariant: 0 <= rdPos <= wrPos <= len(hist)
wrPos int // Current output position in buffer
rdPos int // Have emitted hist[:rdPos] already
flushed int64 // Total bytes returned by readFlush since init
full bool // Has a full window length been written yet?
}
// init initializes dictDecoder to have a sliding window dictionary of the given
// size. If a preset dict is provided, it will initialize the dictionary with
// the contents of dict.
func (dd *dictDecoder) init(size int, dict []byte) {
*dd = dictDecoder{hist: dd.hist}
if cap(dd.hist) < size {
dd.hist = make([]byte, size)
}
dd.hist = dd.hist[:size]
if len(dict) > len(dd.hist) {
dict = dict[len(dict)-len(dd.hist):]
}
dd.wrPos = copy(dd.hist, dict)
if dd.wrPos == len(dd.hist) {
dd.wrPos = 0
dd.full = true
}
dd.rdPos = dd.wrPos
}
// histSize reports the total amount of historical data in the dictionary.
func (dd *dictDecoder) histSize() int {
if dd.full {
return len(dd.hist)
}
return dd.wrPos
}
// availRead reports the number of bytes that can be flushed by readFlush.
func (dd *dictDecoder) availRead() int {
return dd.wrPos - dd.rdPos
}
// availWrite reports the available amount of output buffer space.
func (dd *dictDecoder) availWrite() int {
return len(dd.hist) - dd.wrPos
}
// writeSlice returns a slice of the available buffer to write data to.
//
// This invariant will be kept: len(s) <= availWrite()
func (dd *dictDecoder) writeSlice() []byte {
return dd.hist[dd.wrPos:]
}
// writeMark advances the writer pointer by cnt.
//
// This invariant must be kept: 0 <= cnt <= availWrite()
func (dd *dictDecoder) writeMark(cnt int) {
dd.wrPos += cnt
}
// writeByte writes a single byte to the dictionary.
//
// This invariant must be kept: 0 < availWrite()
func (dd *dictDecoder) writeByte(c byte) {
dd.hist[dd.wrPos] = c
dd.wrPos++
}
// writeCopy copies a string at a given (dist, length) to the output.
// This returns the number of bytes copied and may be less than the requested
// length if the available space in the output buffer is too small.
//
// This invariant must be kept: 0 < dist <= histSize()
func (dd *dictDecoder) writeCopy(dist, length int) int {
dstBase := dd.wrPos
dstPos := dstBase
srcPos := dstPos - dist
endPos := min(dstPos+length, len(dd.hist))
// Copy non-overlapping section after destination position.
//
// This section is non-overlapping in that the copy length for this section
// is always less than or equal to the backwards distance. This can occur
// if a distance refers to data that wraps-around in the buffer.
// Thus, a backwards copy is performed here; that is, the exact bytes in
// the source prior to the copy is placed in the destination.
if srcPos < 0 {
srcPos += len(dd.hist)
dstPos += copy(dd.hist[dstPos:endPos], dd.hist[srcPos:])
srcPos = 0
}
// Copy possibly overlapping section before destination position.
//
// This section can overlap if the copy length for this section is larger
// than the backwards distance. This is allowed by LZ77 so that repeated
// strings can be succinctly represented using (dist, length) pairs.
// Thus, a forwards copy is performed here; that is, the bytes copied is
// possibly dependent on the resulting bytes in the destination as the copy
// progresses along. This is functionally equivalent to the following:
//
// for i := 0; i < endPos-dstPos; i++ {
// dd.hist[dstPos+i] = dd.hist[srcPos+i]
// }
// dstPos = endPos
//
for dstPos < endPos {
dstPos += copy(dd.hist[dstPos:endPos], dd.hist[srcPos:dstPos])
}
dd.wrPos = dstPos
return dstPos - dstBase
}
// tryWriteCopy tries to copy a string at a given (distance, length) to the
// output. This specialized version is optimized for short distances.
//
// This method is designed to be inlined for performance reasons.
//
// This invariant must be kept: 0 < dist <= histSize()
func (dd *dictDecoder) tryWriteCopy(dist, length int) int {
dstPos := dd.wrPos
endPos := dstPos + length
if dstPos < dist || endPos > len(dd.hist) {
return 0
}
dstBase := dstPos
srcPos := dstPos - dist
// Copy possibly overlapping section before destination position.
loop:
dstPos += copy(dd.hist[dstPos:endPos], dd.hist[srcPos:dstPos])
if dstPos < endPos {
goto loop // Avoid for-loop so that this function can be inlined
}
dd.wrPos = dstPos
return dstPos - dstBase
}
// appendWindow appends the current sliding window (up to len(hist) most recent
// bytes, oldest first) to dst.
func (dd *dictDecoder) appendWindow(dst []byte) []byte {
if dd.full {
dst = append(dst, dd.hist[dd.wrPos:]...)
return append(dst, dd.hist[:dd.wrPos]...)
}
return append(dst, dd.hist[:dd.wrPos]...)
}
// readFlush returns a slice of the historical buffer that is ready to be
// emitted to the user. The data returned by readFlush must be fully consumed
// before calling any other dictDecoder methods.
func (dd *dictDecoder) readFlush() []byte {
toRead := dd.hist[dd.rdPos:dd.wrPos]
dd.flushed += int64(len(toRead))
dd.rdPos = dd.wrPos
if dd.wrPos == len(dd.hist) {
dd.wrPos, dd.rdPos = 0, 0
dd.full = true
}
return toRead
}
// decoded reports the total number of bytes written into the dictionary since
// init (i.e. excluding any preset dict bytes).
func (dd *dictDecoder) decoded() int64 {
return dd.flushed + int64(dd.wrPos-dd.rdPos)
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/fast_encoder.go
Generated Vendored
+190
View File
@@ -0,0 +1,190 @@
// Copyright 2011 The Snappy-Go Authors. All rights reserved.
// Modified for deflate by Klaus Post (c) 2015.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
import (
"fmt"
"github.com/klauspost/compress/internal/le"
"github.com/klauspost/compress/internal/regmask"
)
type fastEnc interface {
Encode(dst *tokens, src []byte)
Reset()
}
func newFastEnc(level int) fastEnc {
switch level {
case 1:
return &fastEncL1{fastGen: fastGen{cur: maxStoreBlockSize}}
case 2:
return &fastEncL2{fastGen: fastGen{cur: maxStoreBlockSize}}
case 3:
return &fastEncL3{fastGen: fastGen{cur: maxStoreBlockSize}}
case 4:
return &fastEncL4{fastGen: fastGen{cur: maxStoreBlockSize}}
case 5:
return &fastEncL5{fastGen: fastGen{cur: maxStoreBlockSize}}
case 6:
return &fastEncL6{fastGen: fastGen{cur: maxStoreBlockSize}}
default:
panic("invalid level specified")
}
}
const (
tableBits = 15 // Bits used in the table
tableSize = 1 << tableBits // Size of the table
tableShift = 32 - tableBits // Right-shift to get the tableBits most significant bits of a uint32.
baseMatchOffset = 1 // The smallest match offset
baseMatchLength = 3 // The smallest match length per the RFC section 3.2.5
maxMatchOffset = 1 << 15 // The largest match offset
bTableBits = 17 // Bits used in the big tables
bTableSize = 1 << bTableBits // Size of the table
allocHistory = maxStoreBlockSize * 5 // Size to preallocate for history.
bufferReset = (1 << 31) - allocHistory - maxStoreBlockSize - 1 // Reset the buffer offset when reaching this.
)
const (
prime3bytes = 506832829
prime4bytes = 2654435761
prime5bytes = 889523592379
prime6bytes = 227718039650203
prime7bytes = 58295818150454627
prime8bytes = 0xcf1bbcdcb7a56463
)
func load3232(b []byte, i int32) uint32 {
return le.Load32(b, i)
}
func load6432(b []byte, i int32) uint64 {
return le.Load64(b, i)
}
type tableEntry struct {
offset int32
}
// fastGen maintains the table for matches,
// and the previous byte block for level 2.
// This is the generic implementation.
type fastGen struct {
hist []byte
cur int32
}
func (e *fastGen) addBlock(src []byte) int32 {
// check if we have space already
if len(e.hist)+len(src) > cap(e.hist) {
if cap(e.hist) == 0 {
e.hist = make([]byte, 0, allocHistory)
} else {
if cap(e.hist) < maxMatchOffset*2 {
panic("unexpected buffer size")
}
// Move down
offset := int32(len(e.hist)) - maxMatchOffset
// copy(e.hist[0:maxMatchOffset], e.hist[offset:])
*(*[maxMatchOffset]byte)(e.hist) = *(*[maxMatchOffset]byte)(e.hist[offset:])
e.cur += offset
e.hist = e.hist[:maxMatchOffset]
}
}
s := int32(len(e.hist))
e.hist = append(e.hist, src...)
return s
}
type tableEntryPrev struct {
Cur tableEntry
Prev tableEntry
}
// hash7 returns the hash of the lowest 7 bytes of u to fit in a hash table with h bits.
// Preferably h should be a constant and should always be <64.
func hash7(u uint64, h uint8) uint32 {
return uint32(((u << (64 - 56)) * prime7bytes) >> ((64 - h) & regmask.Shift64ByUint8))
}
// hashLen returns a hash of the lowest mls bytes of with length output bits.
// mls must be >=3 and <=8. Any other value will return hash for 4 bytes.
// length should always be < 32.
// Preferably length and mls should be a constant for inlining.
func hashLen(u uint64, length, mls uint8) uint32 {
switch mls {
case 3:
return (uint32(u<<8) * prime3bytes) >> (32 - length)
case 5:
return uint32(((u << (64 - 40)) * prime5bytes) >> (64 - length))
case 6:
return uint32(((u << (64 - 48)) * prime6bytes) >> (64 - length))
case 7:
return uint32(((u << (64 - 56)) * prime7bytes) >> (64 - length))
case 8:
return uint32((u * prime8bytes) >> (64 - length))
default:
return (uint32(u) * prime4bytes) >> (32 - length)
}
}
// matchlen will return the match length between offsets and t in src.
// The maximum length returned is maxMatchLength - 4.
// It is assumed that s > t, that t >=0 and s < len(src).
func (e *fastGen) matchlen(s, t int, src []byte) int32 {
if debugDeflate {
if t >= s {
panic(fmt.Sprint("t >=s:", t, s))
}
if int(s) >= len(src) {
panic(fmt.Sprint("s >= len(src):", s, len(src)))
}
if t < 0 {
panic(fmt.Sprint("t < 0:", t))
}
if s-t > maxMatchOffset {
panic(fmt.Sprint(s, "-", t, "(", s-t, ") > maxMatchLength (", maxMatchOffset, ")"))
}
}
a := src[s:min(s+maxMatchLength-4, len(src))]
b := src[t:]
return int32(matchLen(a, b))
}
// matchlenLong will return the match length between offsets and t in src.
// It is assumed that s > t, that t >=0 and s < len(src).
func (e *fastGen) matchlenLong(s, t int, src []byte) int32 {
if debugDeflate {
if t >= s {
panic(fmt.Sprint("t >=s:", t, s))
}
if int(s) >= len(src) {
panic(fmt.Sprint("s >= len(src):", s, len(src)))
}
if t < 0 {
panic(fmt.Sprint("t < 0:", t))
}
if s-t > maxMatchOffset {
panic(fmt.Sprint(s, "-", t, "(", s-t, ") > maxMatchLength (", maxMatchOffset, ")"))
}
}
return int32(matchLen(src[s:], src[t:]))
}
// Reset the encoding table.
func (e *fastGen) Reset() {
if cap(e.hist) < allocHistory {
e.hist = make([]byte, 0, allocHistory)
}
// We offset current position so everything will be out of reach.
// If we are above the buffer reset it will be cleared anyway since len(hist) == 0.
if e.cur <= bufferReset {
e.cur += maxMatchOffset + int32(len(e.hist))
}
e.hist = e.hist[:0]
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/huffman_bit_writer.go
Generated Vendored
+1174
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/klauspost/compress/flate/huffman_code.go
Generated Vendored
+417
View File
@@ -0,0 +1,417 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
import (
"math"
"math/bits"
)
const (
maxBitsLimit = 16
// number of valid literals
literalCount = 286
)
// hcode is a huffman code with a bit code and bit length.
type hcode uint32
func (h hcode) len() uint8 {
return uint8(h)
}
func (h hcode) code64() uint64 {
return uint64(h >> 8)
}
func (h hcode) zero() bool {
return h == 0
}
type huffmanEncoder struct {
codes []hcode
bitCount [17]int32
// Allocate a reusable buffer with the longest possible frequency table.
// Possible lengths are codegenCodeCount, offsetCodeCount and literalCount.
// The largest of these is literalCount, so we allocate for that case.
freqcache [literalCount + 1]literalNode
}
type literalNode struct {
literal uint16
freq uint16
}
// A levelInfo describes the state of the constructed tree for a given depth.
type levelInfo struct {
// Our level. for better printing
level int32
// The frequency of the last node at this level
lastFreq int32
// The frequency of the next character to add to this level
nextCharFreq int32
// The frequency of the next pair (from level below) to add to this level.
// Only valid if the "needed" value of the next lower level is 0.
nextPairFreq int32
// The number of chains remaining to generate for this level before moving
// up to the next level
needed int32
}
// set sets the code and length of an hcode.
func (h *hcode) set(code uint16, length uint8) {
*h = hcode(length) | (hcode(code) << 8)
}
func newhcode(code uint16, length uint8) hcode {
return hcode(length) | (hcode(code) << 8)
}
func reverseBits(number uint16, bitLength byte) uint16 {
return bits.Reverse16(number << ((16 - bitLength) & 15))
}
func maxNode() literalNode { return literalNode{math.MaxUint16, math.MaxUint16} }
func newHuffmanEncoder(size int) *huffmanEncoder {
// Make capacity to next power of two.
c := uint(bits.Len32(uint32(size - 1)))
return &huffmanEncoder{codes: make([]hcode, size, 1<<c)}
}
// Generates a HuffmanCode corresponding to the fixed literal table
func generateFixedLiteralEncoding() *huffmanEncoder {
h := newHuffmanEncoder(literalCount)
codes := h.codes
var ch uint16
for ch = range uint16(literalCount) {
var bits uint16
var size uint8
switch {
case ch < 144:
// size 8, 000110000 .. 10111111
bits = ch + 48
size = 8
case ch < 256:
// size 9, 110010000 .. 111111111
bits = ch + 400 - 144
size = 9
case ch < 280:
// size 7, 0000000 .. 0010111
bits = ch - 256
size = 7
default:
// size 8, 11000000 .. 11000111
bits = ch + 192 - 280
size = 8
}
codes[ch] = newhcode(reverseBits(bits, size), size)
}
return h
}
func generateFixedOffsetEncoding() *huffmanEncoder {
h := newHuffmanEncoder(30)
codes := h.codes
for ch := range codes {
codes[ch] = newhcode(reverseBits(uint16(ch), 5), 5)
}
return h
}
var fixedLiteralEncoding = generateFixedLiteralEncoding()
var fixedOffsetEncoding = generateFixedOffsetEncoding()
func (h *huffmanEncoder) bitLength(freq []uint16) int {
var total int
for i, f := range freq {
if f != 0 {
total += int(f) * int(h.codes[i].len())
}
}
return total
}
func (h *huffmanEncoder) bitLengthRaw(b []byte) int {
var total int
for _, f := range b {
total += int(h.codes[f].len())
}
return total
}
// canReuseBits returns the number of bits or math.MaxInt32 if the encoder cannot be reused.
func (h *huffmanEncoder) canReuseBits(freq []uint16) int {
var total int
for i, f := range freq {
if f != 0 {
code := h.codes[i]
if code.zero() {
return math.MaxInt32
}
total += int(f) * int(code.len())
}
}
return total
}
// Return the number of literals assigned to each bit size in the Huffman encoding
//
// This method is only called when list.length >= 3
// The cases of 0, 1, and 2 literals are handled by special case code.
//
// list An array of the literals with non-zero frequencies
//
// and their associated frequencies. The array is in order of increasing
// frequency, and has as its last element a special element with frequency
// MaxInt32
//
// maxBits The maximum number of bits that should be used to encode any literal.
//
// Must be less than 16.
//
// return An integer array in which array[i] indicates the number of literals
//
// that should be encoded in i bits.
func (h *huffmanEncoder) bitCounts(list []literalNode, maxBits int32) []int32 {
if maxBits >= maxBitsLimit {
panic("flate: maxBits too large")
}
n := int32(len(list))
list = list[0 : n+1]
list[n] = maxNode()
// The tree can't have greater depth than n - 1, no matter what. This
// saves a little bit of work in some small cases
if maxBits > n-1 {
maxBits = n - 1
}
// Create information about each of the levels.
// A bogus "Level 0" whose sole purpose is so that
// level1.prev.needed==0. This makes level1.nextPairFreq
// be a legitimate value that never gets chosen.
var levels [maxBitsLimit]levelInfo
// leafCounts[i] counts the number of literals at the left
// of ancestors of the rightmost node at level i.
// leafCounts[i][j] is the number of literals at the left
// of the level j ancestor.
var leafCounts [maxBitsLimit][maxBitsLimit]int32
// Descending to only have 1 bounds check.
l2f := int32(list[2].freq)
l1f := int32(list[1].freq)
l0f := int32(list[0].freq) + int32(list[1].freq)
for level := int32(1); level <= maxBits; level++ {
// For every level, the first two items are the first two characters.
// We initialize the levels as if we had already figured this out.
levels[level] = levelInfo{
level: level,
lastFreq: l1f,
nextCharFreq: l2f,
nextPairFreq: l0f,
}
leafCounts[level][level] = 2
if level == 1 {
levels[level].nextPairFreq = math.MaxInt32
}
}
// We need a total of 2*n - 2 items at top level and have already generated 2.
levels[maxBits].needed = 2*n - 4
level := uint32(maxBits)
for level < 16 {
l := &levels[level]
if l.nextPairFreq == math.MaxInt32 && l.nextCharFreq == math.MaxInt32 {
// We've run out of both leafs and pairs.
// End all calculations for this level.
// To make sure we never come back to this level or any lower level,
// set nextPairFreq impossibly large.
l.needed = 0
levels[level+1].nextPairFreq = math.MaxInt32
level++
continue
}
prevFreq := l.lastFreq
if l.nextCharFreq < l.nextPairFreq {
// The next item on this row is a leaf node.
n := leafCounts[level][level] + 1
l.lastFreq = l.nextCharFreq
// Lower leafCounts are the same of the previous node.
leafCounts[level][level] = n
e := list[n]
if e.literal < math.MaxUint16 {
l.nextCharFreq = int32(e.freq)
} else {
l.nextCharFreq = math.MaxInt32
}
} else {
// The next item on this row is a pair from the previous row.
// nextPairFreq isn't valid until we generate two
// more values in the level below
l.lastFreq = l.nextPairFreq
// Take leaf counts from the lower level, except counts[level] remains the same.
if true {
save := leafCounts[level][level]
leafCounts[level] = leafCounts[level-1]
leafCounts[level][level] = save
} else {
copy(leafCounts[level][:level], leafCounts[level-1][:level])
}
levels[l.level-1].needed = 2
}
if l.needed--; l.needed == 0 {
// We've done everything we need to do for this level.
// Continue calculating one level up. Fill in nextPairFreq
// of that level with the sum of the two nodes we've just calculated on
// this level.
if l.level == maxBits {
// All done!
break
}
levels[l.level+1].nextPairFreq = prevFreq + l.lastFreq
level++
} else {
// If we stole from below, move down temporarily to replenish it.
for levels[level-1].needed > 0 {
level--
}
}
}
// Somethings is wrong if at the end, the top level is null or hasn't used
// all of the leaves.
if leafCounts[maxBits][maxBits] != n {
panic("leafCounts[maxBits][maxBits] != n")
}
bitCount := h.bitCount[:maxBits+1]
bits := 1
counts := &leafCounts[maxBits]
for level := maxBits; level > 0; level-- {
// chain.leafCount gives the number of literals requiring at least "bits"
// bits to encode.
bitCount[bits] = counts[level] - counts[level-1]
bits++
}
return bitCount
}
// Look at the leaves and assign them a bit count and an encoding as specified
// in RFC 1951 3.2.2
func (h *huffmanEncoder) assignEncodingAndSize(bitCount []int32, list []literalNode) {
code := uint16(0)
for n, bits := range bitCount {
code <<= 1
if n == 0 || bits == 0 {
continue
}
// The literals list[len(list)-bits] .. list[len(list)-bits]
// are encoded using "bits" bits, and get the values
// code, code + 1, .... The code values are
// assigned in literal order (not frequency order).
chunk := list[len(list)-int(bits):]
sortByLiteral(chunk)
for _, node := range chunk {
h.codes[node.literal] = newhcode(reverseBits(code, uint8(n)), uint8(n))
code++
}
list = list[0 : len(list)-int(bits)]
}
}
// Update this Huffman Code object to be the minimum code for the specified frequency count.
//
// freq An array of frequencies, in which frequency[i] gives the frequency of literal i.
// maxBits The maximum number of bits to use for any literal.
func (h *huffmanEncoder) generate(freq []uint16, maxBits int32) {
list := h.freqcache[:len(freq)+1]
codes := h.codes[:len(freq)]
// Number of non-zero literals
count := 0
// Set list to be the set of all non-zero literals and their frequencies
for i, f := range freq {
if f != 0 {
list[count] = literalNode{uint16(i), f}
count++
} else {
codes[i] = 0
}
}
list[count] = literalNode{}
list = list[:count]
if count <= 2 {
// Handle the small cases here, because they are awkward for the general case code. With
// two or fewer literals, everything has bit length 1.
for i, node := range list {
// "list" is in order of increasing literal value.
h.codes[node.literal].set(uint16(i), 1)
}
return
}
sortByFreq(list)
// Get the number of literals for each bit count
bitCount := h.bitCounts(list, maxBits)
// And do the assignment
h.assignEncodingAndSize(bitCount, list)
}
// atLeastOne clamps the result between 1 and 15.
func atLeastOne(v float32) float32 {
if v < 1 {
return 1
}
if v > 15 {
return 15
}
return v
}
func histogram(b []byte, h []uint16) {
if true && len(b) >= 8<<10 {
// Split for bigger inputs
histogramSplit(b, h)
} else {
h = h[:256]
for _, t := range b {
h[t]++
}
}
}
func histogramSplit(b []byte, h []uint16) {
// Tested, and slightly faster than 2-way.
// Writing to separate arrays and combining is also slightly slower.
h = h[:256]
for len(b)&3 != 0 {
h[b[0]]++
b = b[1:]
}
n := len(b) / 4
x, y, z, w := b[:n], b[n:], b[n+n:], b[n+n+n:]
y, z, w = y[:len(x)], z[:len(x)], w[:len(x)]
for i, t := range x {
v0 := &h[t]
v1 := &h[y[i]]
v2 := &h[z[i]]
v3 := &h[w[i]]
*v0++
*v1++
*v2++
*v3++
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/huffman_sortByFreq.go
Generated Vendored
+159
View File
@@ -0,0 +1,159 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
// Sort sorts data.
// It makes one call to data.Len to determine n, and O(n*log(n)) calls to
// data.Less and data.Swap. The sort is not guaranteed to be stable.
func sortByFreq(data []literalNode) {
n := len(data)
quickSortByFreq(data, 0, n, maxDepth(n))
}
func quickSortByFreq(data []literalNode, a, b, maxDepth int) {
for b-a > 12 { // Use ShellSort for slices <= 12 elements
if maxDepth == 0 {
heapSort(data, a, b)
return
}
maxDepth--
mlo, mhi := doPivotByFreq(data, a, b)
// Avoiding recursion on the larger subproblem guarantees
// a stack depth of at most lg(b-a).
if mlo-a < b-mhi {
quickSortByFreq(data, a, mlo, maxDepth)
a = mhi // i.e., quickSortByFreq(data, mhi, b)
} else {
quickSortByFreq(data, mhi, b, maxDepth)
b = mlo // i.e., quickSortByFreq(data, a, mlo)
}
}
if b-a > 1 {
// Do ShellSort pass with gap 6
// It could be written in this simplified form cause b-a <= 12
for i := a + 6; i < b; i++ {
if data[i].freq == data[i-6].freq && data[i].literal < data[i-6].literal || data[i].freq < data[i-6].freq {
data[i], data[i-6] = data[i-6], data[i]
}
}
insertionSortByFreq(data, a, b)
}
}
func doPivotByFreq(data []literalNode, lo, hi int) (midlo, midhi int) {
m := int(uint(lo+hi) >> 1) // Written like this to avoid integer overflow.
if hi-lo > 40 {
// Tukey's ``Ninther,'' median of three medians of three.
s := (hi - lo) / 8
medianOfThreeSortByFreq(data, lo, lo+s, lo+2*s)
medianOfThreeSortByFreq(data, m, m-s, m+s)
medianOfThreeSortByFreq(data, hi-1, hi-1-s, hi-1-2*s)
}
medianOfThreeSortByFreq(data, lo, m, hi-1)
// Invariants are:
// data[lo] = pivot (set up by ChoosePivot)
// data[lo < i < a] < pivot
// data[a <= i < b] <= pivot
// data[b <= i < c] unexamined
// data[c <= i < hi-1] > pivot
// data[hi-1] >= pivot
pivot := lo
a, c := lo+1, hi-1
for ; a < c && (data[a].freq == data[pivot].freq && data[a].literal < data[pivot].literal || data[a].freq < data[pivot].freq); a++ {
}
b := a
for {
for ; b < c && (data[pivot].freq == data[b].freq && data[pivot].literal > data[b].literal || data[pivot].freq > data[b].freq); b++ { // data[b] <= pivot
}
for ; b < c && (data[pivot].freq == data[c-1].freq && data[pivot].literal < data[c-1].literal || data[pivot].freq < data[c-1].freq); c-- { // data[c-1] > pivot
}
if b >= c {
break
}
// data[b] > pivot; data[c-1] <= pivot
data[b], data[c-1] = data[c-1], data[b]
b++
c--
}
// If hi-c<3 then there are duplicates (by property of median of nine).
// Let's be a bit more conservative, and set border to 5.
protect := hi-c < 5
if !protect && hi-c < (hi-lo)/4 {
// Lets test some points for equality to pivot
dups := 0
if data[pivot].freq == data[hi-1].freq && data[pivot].literal > data[hi-1].literal || data[pivot].freq > data[hi-1].freq { // data[hi-1] = pivot
data[c], data[hi-1] = data[hi-1], data[c]
c++
dups++
}
if data[b-1].freq == data[pivot].freq && data[b-1].literal > data[pivot].literal || data[b-1].freq > data[pivot].freq { // data[b-1] = pivot
b--
dups++
}
// m-lo = (hi-lo)/2 > 6
// b-lo > (hi-lo)*3/4-1 > 8
// ==> m < b ==> data[m] <= pivot
if data[m].freq == data[pivot].freq && data[m].literal > data[pivot].literal || data[m].freq > data[pivot].freq { // data[m] = pivot
data[m], data[b-1] = data[b-1], data[m]
b--
dups++
}
// if at least 2 points are equal to pivot, assume skewed distribution
protect = dups > 1
}
if protect {
// Protect against a lot of duplicates
// Add invariant:
// data[a <= i < b] unexamined
// data[b <= i < c] = pivot
for {
for ; a < b && (data[b-1].freq == data[pivot].freq && data[b-1].literal > data[pivot].literal || data[b-1].freq > data[pivot].freq); b-- { // data[b] == pivot
}
for ; a < b && (data[a].freq == data[pivot].freq && data[a].literal < data[pivot].literal || data[a].freq < data[pivot].freq); a++ { // data[a] < pivot
}
if a >= b {
break
}
// data[a] == pivot; data[b-1] < pivot
data[a], data[b-1] = data[b-1], data[a]
a++
b--
}
}
// Swap pivot into middle
data[pivot], data[b-1] = data[b-1], data[pivot]
return b - 1, c
}
// Insertion sort
func insertionSortByFreq(data []literalNode, a, b int) {
for i := a + 1; i < b; i++ {
for j := i; j > a && (data[j].freq == data[j-1].freq && data[j].literal < data[j-1].literal || data[j].freq < data[j-1].freq); j-- {
data[j], data[j-1] = data[j-1], data[j]
}
}
}
// quickSortByFreq, loosely following Bentley and McIlroy,
// ``Engineering a Sort Function,'' SP&E November 1993.
// medianOfThreeSortByFreq moves the median of the three values data[m0], data[m1], data[m2] into data[m1].
func medianOfThreeSortByFreq(data []literalNode, m1, m0, m2 int) {
// sort 3 elements
if data[m1].freq == data[m0].freq && data[m1].literal < data[m0].literal || data[m1].freq < data[m0].freq {
data[m1], data[m0] = data[m0], data[m1]
}
// data[m0] <= data[m1]
if data[m2].freq == data[m1].freq && data[m2].literal < data[m1].literal || data[m2].freq < data[m1].freq {
data[m2], data[m1] = data[m1], data[m2]
// data[m0] <= data[m2] && data[m1] < data[m2]
if data[m1].freq == data[m0].freq && data[m1].literal < data[m0].literal || data[m1].freq < data[m0].freq {
data[m1], data[m0] = data[m0], data[m1]
}
}
// now data[m0] <= data[m1] <= data[m2]
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/huffman_sortByLiteral.go
Generated Vendored
+201
View File
@@ -0,0 +1,201 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
// Sort sorts data.
// It makes one call to data.Len to determine n, and O(n*log(n)) calls to
// data.Less and data.Swap. The sort is not guaranteed to be stable.
func sortByLiteral(data []literalNode) {
n := len(data)
quickSort(data, 0, n, maxDepth(n))
}
func quickSort(data []literalNode, a, b, maxDepth int) {
for b-a > 12 { // Use ShellSort for slices <= 12 elements
if maxDepth == 0 {
heapSort(data, a, b)
return
}
maxDepth--
mlo, mhi := doPivot(data, a, b)
// Avoiding recursion on the larger subproblem guarantees
// a stack depth of at most lg(b-a).
if mlo-a < b-mhi {
quickSort(data, a, mlo, maxDepth)
a = mhi // i.e., quickSort(data, mhi, b)
} else {
quickSort(data, mhi, b, maxDepth)
b = mlo // i.e., quickSort(data, a, mlo)
}
}
if b-a > 1 {
// Do ShellSort pass with gap 6
// It could be written in this simplified form cause b-a <= 12
for i := a + 6; i < b; i++ {
if data[i].literal < data[i-6].literal {
data[i], data[i-6] = data[i-6], data[i]
}
}
insertionSort(data, a, b)
}
}
func heapSort(data []literalNode, a, b int) {
first := a
lo := 0
hi := b - a
// Build heap with greatest element at top.
for i := (hi - 1) / 2; i >= 0; i-- {
siftDown(data, i, hi, first)
}
// Pop elements, largest first, into end of data.
for i := hi - 1; i >= 0; i-- {
data[first], data[first+i] = data[first+i], data[first]
siftDown(data, lo, i, first)
}
}
// siftDown implements the heap property on data[lo, hi).
// first is an offset into the array where the root of the heap lies.
func siftDown(data []literalNode, lo, hi, first int) {
root := lo
for {
child := 2*root + 1
if child >= hi {
break
}
if child+1 < hi && data[first+child].literal < data[first+child+1].literal {
child++
}
if data[first+root].literal > data[first+child].literal {
return
}
data[first+root], data[first+child] = data[first+child], data[first+root]
root = child
}
}
func doPivot(data []literalNode, lo, hi int) (midlo, midhi int) {
m := int(uint(lo+hi) >> 1) // Written like this to avoid integer overflow.
if hi-lo > 40 {
// Tukey's ``Ninther,'' median of three medians of three.
s := (hi - lo) / 8
medianOfThree(data, lo, lo+s, lo+2*s)
medianOfThree(data, m, m-s, m+s)
medianOfThree(data, hi-1, hi-1-s, hi-1-2*s)
}
medianOfThree(data, lo, m, hi-1)
// Invariants are:
// data[lo] = pivot (set up by ChoosePivot)
// data[lo < i < a] < pivot
// data[a <= i < b] <= pivot
// data[b <= i < c] unexamined
// data[c <= i < hi-1] > pivot
// data[hi-1] >= pivot
pivot := lo
a, c := lo+1, hi-1
for ; a < c && data[a].literal < data[pivot].literal; a++ {
}
b := a
for {
for ; b < c && data[pivot].literal > data[b].literal; b++ { // data[b] <= pivot
}
for ; b < c && data[pivot].literal < data[c-1].literal; c-- { // data[c-1] > pivot
}
if b >= c {
break
}
// data[b] > pivot; data[c-1] <= pivot
data[b], data[c-1] = data[c-1], data[b]
b++
c--
}
// If hi-c<3 then there are duplicates (by property of median of nine).
// Let's be a bit more conservative, and set border to 5.
protect := hi-c < 5
if !protect && hi-c < (hi-lo)/4 {
// Lets test some points for equality to pivot
dups := 0
if data[pivot].literal > data[hi-1].literal { // data[hi-1] = pivot
data[c], data[hi-1] = data[hi-1], data[c]
c++
dups++
}
if data[b-1].literal > data[pivot].literal { // data[b-1] = pivot
b--
dups++
}
// m-lo = (hi-lo)/2 > 6
// b-lo > (hi-lo)*3/4-1 > 8
// ==> m < b ==> data[m] <= pivot
if data[m].literal > data[pivot].literal { // data[m] = pivot
data[m], data[b-1] = data[b-1], data[m]
b--
dups++
}
// if at least 2 points are equal to pivot, assume skewed distribution
protect = dups > 1
}
if protect {
// Protect against a lot of duplicates
// Add invariant:
// data[a <= i < b] unexamined
// data[b <= i < c] = pivot
for {
for ; a < b && data[b-1].literal > data[pivot].literal; b-- { // data[b] == pivot
}
for ; a < b && data[a].literal < data[pivot].literal; a++ { // data[a] < pivot
}
if a >= b {
break
}
// data[a] == pivot; data[b-1] < pivot
data[a], data[b-1] = data[b-1], data[a]
a++
b--
}
}
// Swap pivot into middle
data[pivot], data[b-1] = data[b-1], data[pivot]
return b - 1, c
}
// Insertion sort
func insertionSort(data []literalNode, a, b int) {
for i := a + 1; i < b; i++ {
for j := i; j > a && data[j].literal < data[j-1].literal; j-- {
data[j], data[j-1] = data[j-1], data[j]
}
}
}
// maxDepth returns a threshold at which quicksort should switch
// to heapsort. It returns 2*ceil(lg(n+1)).
func maxDepth(n int) int {
var depth int
for i := n; i > 0; i >>= 1 {
depth++
}
return depth * 2
}
// medianOfThree moves the median of the three values data[m0], data[m1], data[m2] into data[m1].
func medianOfThree(data []literalNode, m1, m0, m2 int) {
// sort 3 elements
if data[m1].literal < data[m0].literal {
data[m1], data[m0] = data[m0], data[m1]
}
// data[m0] <= data[m1]
if data[m2].literal < data[m1].literal {
data[m2], data[m1] = data[m1], data[m2]
// data[m0] <= data[m2] && data[m1] < data[m2]
if data[m1].literal < data[m0].literal {
data[m1], data[m0] = data[m0], data[m1]
}
}
// now data[m0] <= data[m1] <= data[m2]
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/inflate.go
Generated Vendored
+959
View File
@@ -0,0 +1,959 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package flate implements the DEFLATE compressed data format, described in
// RFC 1951. The gzip and zlib packages implement access to DEFLATE-based file
// formats.
package flate
import (
"bufio"
"compress/flate"
"fmt"
"io"
"math/bits"
"sync"
"github.com/klauspost/compress/internal/regmask"
)
const (
maxCodeLen = 16 // max length of Huffman code
maxCodeLenMask = 15 // mask for max length of Huffman code
// The next three numbers come from the RFC section 3.2.7, with the
// additional proviso in section 3.2.5 which implies that distance codes
// 30 and 31 should never occur in compressed data.
maxNumLit = 286
maxNumDist = 30
numCodes = 19 // number of codes in Huffman meta-code
debugDecode = false
)
// Value of length - 3 and extra bits.
type lengthExtra struct {
length, extra uint8
}
var decCodeToLen = [32]lengthExtra{{length: 0x0, extra: 0x0}, {length: 0x1, extra: 0x0}, {length: 0x2, extra: 0x0}, {length: 0x3, extra: 0x0}, {length: 0x4, extra: 0x0}, {length: 0x5, extra: 0x0}, {length: 0x6, extra: 0x0}, {length: 0x7, extra: 0x0}, {length: 0x8, extra: 0x1}, {length: 0xa, extra: 0x1}, {length: 0xc, extra: 0x1}, {length: 0xe, extra: 0x1}, {length: 0x10, extra: 0x2}, {length: 0x14, extra: 0x2}, {length: 0x18, extra: 0x2}, {length: 0x1c, extra: 0x2}, {length: 0x20, extra: 0x3}, {length: 0x28, extra: 0x3}, {length: 0x30, extra: 0x3}, {length: 0x38, extra: 0x3}, {length: 0x40, extra: 0x4}, {length: 0x50, extra: 0x4}, {length: 0x60, extra: 0x4}, {length: 0x70, extra: 0x4}, {length: 0x80, extra: 0x5}, {length: 0xa0, extra: 0x5}, {length: 0xc0, extra: 0x5}, {length: 0xe0, extra: 0x5}, {length: 0xff, extra: 0x0}, {length: 0x0, extra: 0x0}, {length: 0x0, extra: 0x0}, {length: 0x0, extra: 0x0}}
var bitMask32 = [32]uint32{
0, 1, 3, 7, 0xF, 0x1F, 0x3F, 0x7F, 0xFF,
0x1FF, 0x3FF, 0x7FF, 0xFFF, 0x1FFF, 0x3FFF, 0x7FFF, 0xFFFF,
0x1ffff, 0x3ffff, 0x7FFFF, 0xfFFFF, 0x1fFFFF, 0x3fFFFF, 0x7fFFFF, 0xffFFFF,
0x1ffFFFF, 0x3ffFFFF, 0x7ffFFFF, 0xfffFFFF, 0x1fffFFFF, 0x3fffFFFF, 0x7fffFFFF,
} // up to 32 bits
// Initialize the fixedHuffmanDecoder only once upon first use.
var fixedOnce sync.Once
var fixedHuffmanDecoder huffmanDecoder
// A CorruptInputError reports the presence of corrupt input at a given offset.
type CorruptInputError = flate.CorruptInputError
// An InternalError reports an error in the flate code itself.
type InternalError string
func (e InternalError) Error() string { return "flate: internal error: " + string(e) }
// A ReadError reports an error encountered while reading input.
//
// Deprecated: No longer returned.
type ReadError = flate.ReadError
// A WriteError reports an error encountered while writing output.
//
// Deprecated: No longer returned.
type WriteError = flate.WriteError
// Resetter resets a ReadCloser returned by NewReader or NewReaderDict to
// to switch to a new underlying Reader. This permits reusing a ReadCloser
// instead of allocating a new one.
type Resetter interface {
// Reset discards any buffered data and resets the Resetter as if it was
// newly initialized with the given reader.
Reset(r io.Reader, dict []byte) error
}
// The data structure for decoding Huffman tables is based on that of
// zlib. There is a lookup table of a fixed bit width (huffmanChunkBits),
// For codes smaller than the table width, there are multiple entries
// (each combination of trailing bits has the same value). For codes
// larger than the table width, the table contains a link to an overflow
// table. The width of each entry in the link table is the maximum code
// size minus the chunk width.
//
// Note that you can do a lookup in the table even without all bits
// filled. Since the extra bits are zero, and the DEFLATE Huffman codes
// have the property that shorter codes come before longer ones, the
// bit length estimate in the result is a lower bound on the actual
// number of bits.
//
// See the following:
// http://www.gzip.org/algorithm.txt
// chunk & 15 is number of bits
// chunk >> 4 is value, including table link
const (
huffmanChunkBits = 9
huffmanNumChunks = 1 << huffmanChunkBits
huffmanCountMask = 15
huffmanValueShift = 4
)
type huffmanDecoder struct {
maxRead int // the maximum number of bits we can read and not overread
chunks *[huffmanNumChunks]uint16 // chunks as described above
links [][]uint16 // overflow links
linkMask uint32 // mask the width of the link table
}
// Initialize Huffman decoding tables from array of code lengths.
// Following this function, h is guaranteed to be initialized into a complete
// tree (i.e., neither over-subscribed nor under-subscribed). The exception is a
// degenerate case where the tree has only a single symbol with length 1. Empty
// trees are permitted.
func (h *huffmanDecoder) init(lengths []int) bool {
// Sanity enables additional runtime tests during Huffman
// table construction. It's intended to be used during
// development to supplement the currently ad-hoc unit tests.
const sanity = false
if h.chunks == nil {
h.chunks = new([huffmanNumChunks]uint16)
}
if h.maxRead != 0 {
*h = huffmanDecoder{chunks: h.chunks, links: h.links}
}
// Count number of codes of each length,
// compute maxRead and max length.
var count [maxCodeLen]int
var min, max int
for _, n := range lengths {
if n == 0 {
continue
}
if min == 0 || n < min {
min = n
}
if n > max {
max = n
}
count[n&maxCodeLenMask]++
}
// Empty tree. The decompressor.huffSym function will fail later if the tree
// is used. Technically, an empty tree is only valid for the HDIST tree and
// not the HCLEN and HLIT tree. However, a stream with an empty HCLEN tree
// is guaranteed to fail since it will attempt to use the tree to decode the
// codes for the HLIT and HDIST trees. Similarly, an empty HLIT tree is
// guaranteed to fail later since the compressed data section must be
// composed of at least one symbol (the end-of-block marker).
if max == 0 {
return true
}
code := 0
var nextcode [maxCodeLen]int
for i := min; i <= max; i++ {
code <<= 1
nextcode[i&maxCodeLenMask] = code
code += count[i&maxCodeLenMask]
}
// Check that the coding is complete (i.e., that we've
// assigned all 2-to-the-max possible bit sequences).
// Exception: To be compatible with zlib, we also need to
// accept degenerate single-code codings. See also
// TestDegenerateHuffmanCoding.
if code != 1<<uint(max) && !(code == 1 && max == 1) {
if debugDecode {
fmt.Println("coding failed, code, max:", code, max, code == 1<<uint(max), code == 1 && max == 1, "(one should be true)")
}
return false
}
h.maxRead = min
chunks := h.chunks[:]
for i := range chunks {
chunks[i] = 0
}
if max > huffmanChunkBits {
numLinks := 1 << (uint(max) - huffmanChunkBits)
h.linkMask = uint32(numLinks - 1)
// create link tables
link := nextcode[huffmanChunkBits+1] >> 1
if cap(h.links) < huffmanNumChunks-link {
h.links = make([][]uint16, huffmanNumChunks-link)
} else {
h.links = h.links[:huffmanNumChunks-link]
}
for j := uint(link); j < huffmanNumChunks; j++ {
reverse := int(bits.Reverse16(uint16(j)))
reverse >>= uint(16 - huffmanChunkBits)
off := j - uint(link)
if sanity && h.chunks[reverse] != 0 {
panic("impossible: overwriting existing chunk")
}
h.chunks[reverse] = uint16(off<<huffmanValueShift | (huffmanChunkBits + 1))
if cap(h.links[off]) < numLinks {
h.links[off] = make([]uint16, numLinks)
} else {
h.links[off] = h.links[off][:numLinks]
}
}
} else {
h.links = h.links[:0]
}
for i, n := range lengths {
if n == 0 {
continue
}
code := nextcode[n]
nextcode[n]++
chunk := uint16(i<<huffmanValueShift | n)
reverse := int(bits.Reverse16(uint16(code)))
reverse >>= uint(16 - n)
if n <= huffmanChunkBits {
for off := reverse; off < len(h.chunks); off += 1 << uint(n) {
// We should never need to overwrite
// an existing chunk. Also, 0 is
// never a valid chunk, because the
// lower 4 "count" bits should be
// between 1 and 15.
if sanity && h.chunks[off] != 0 {
panic("impossible: overwriting existing chunk")
}
h.chunks[off] = chunk
}
} else {
j := reverse & (huffmanNumChunks - 1)
if sanity && h.chunks[j]&huffmanCountMask != huffmanChunkBits+1 {
// Longer codes should have been
// associated with a link table above.
panic("impossible: not an indirect chunk")
}
value := h.chunks[j] >> huffmanValueShift
linktab := h.links[value]
reverse >>= huffmanChunkBits
for off := reverse; off < len(linktab); off += 1 << uint(n-huffmanChunkBits) {
if sanity && linktab[off] != 0 {
panic("impossible: overwriting existing chunk")
}
linktab[off] = chunk
}
}
}
if sanity {
// Above we've sanity checked that we never overwrote
// an existing entry. Here we additionally check that
// we filled the tables completely.
for i, chunk := range h.chunks {
if chunk == 0 {
// As an exception, in the degenerate
// single-code case, we allow odd
// chunks to be missing.
if code == 1 && i%2 == 1 {
continue
}
panic("impossible: missing chunk")
}
}
for _, linktab := range h.links {
for _, chunk := range linktab {
if chunk == 0 {
panic("impossible: missing chunk")
}
}
}
}
return true
}
// Reader is the actual read interface needed by NewReader.
// If the passed in io.Reader does not also have ReadByte,
// the NewReader will introduce its own buffering.
type Reader interface {
io.Reader
io.ByteReader
}
type step uint8
const (
copyData step = iota + 1
nextBlock
huffmanBytesBuffer
huffmanBytesReader
huffmanBufioReader
huffmanStringsReader
huffmanGenericReader
)
// flushMode tells decompressor when to return data
type flushMode uint8
const (
syncFlush flushMode = iota // return data after sync flush block
partialFlush // return data after each block
)
// Decompress state.
type decompressor struct {
// Input source.
r Reader
roffset int64
// Huffman decoders for literal/length, distance.
h1, h2 huffmanDecoder
// Length arrays used to define Huffman codes.
bits *[maxNumLit + maxNumDist]int
codebits *[numCodes]int
// Output history, buffer.
dict dictDecoder
// Next step in the decompression,
// and decompression state.
step step
stepState int
err error
toRead []byte
hl, hd *huffmanDecoder
copyLen int
copyDist int
// Temporary buffer (avoids repeated allocation).
buf [4]byte
// Input bits, in top of b.
b uint32
nb uint
final bool
flushMode flushMode
cb func(InflateCheckpoint)
cp InflateCheckpoint
hasCP bool // WithResumeFrom was supplied
uncOffset int64 // baseline uncompressed offset (from a resume checkpoint)
cpBuf []byte
}
func (f *decompressor) nextBlock() {
for f.nb < 1+2 {
if f.err = f.moreBits(); f.err != nil {
return
}
}
f.final = f.b&1 == 1
f.b >>= 1
typ := f.b & 3
f.b >>= 2
f.nb -= 1 + 2
switch typ {
case 0:
f.dataBlock()
if debugDecode {
fmt.Println("stored block")
}
case 1:
// compressed, fixed Huffman tables
f.hl = &fixedHuffmanDecoder
f.hd = nil
f.huffmanBlockDecoder()
if debugDecode {
fmt.Println("predefinied huffman block")
}
case 2:
// compressed, dynamic Huffman tables
if f.err = f.readHuffman(); f.err != nil {
break
}
f.hl = &f.h1
f.hd = &f.h2
f.huffmanBlockDecoder()
if debugDecode {
fmt.Println("dynamic huffman block")
}
default:
// 3 is reserved.
if debugDecode {
fmt.Println("reserved data block encountered")
}
f.err = CorruptInputError(f.roffset)
}
}
func (f *decompressor) Read(b []byte) (int, error) {
for {
if len(f.toRead) > 0 {
n := copy(b, f.toRead)
f.toRead = f.toRead[n:]
if len(f.toRead) == 0 {
return n, f.err
}
return n, nil
}
if f.err != nil {
return 0, f.err
}
f.doStep()
if f.err != nil && len(f.toRead) == 0 {
f.toRead = f.dict.readFlush() // Flush what's left in case of error
}
}
}
// WriteTo implements the io.WriteTo interface for io.Copy and friends.
func (f *decompressor) WriteTo(w io.Writer) (int64, error) {
total := int64(0)
flushed := false
for {
if len(f.toRead) > 0 {
n, err := w.Write(f.toRead)
total += int64(n)
if err != nil {
f.err = err
return total, err
}
if n != len(f.toRead) {
return total, io.ErrShortWrite
}
f.toRead = f.toRead[:0]
}
if f.err != nil && flushed {
if f.err == io.EOF {
return total, nil
}
return total, f.err
}
if f.err == nil {
f.doStep()
}
if len(f.toRead) == 0 && f.err != nil && !flushed {
f.toRead = f.dict.readFlush() // Flush what's left in case of error
flushed = true
}
}
}
func (f *decompressor) Close() error {
if f.err == io.EOF {
return nil
}
return f.err
}
// RFC 1951 section 3.2.7.
// Compression with dynamic Huffman codes
var codeOrder = [...]int{16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}
func (f *decompressor) readHuffman() error {
// HLIT[5], HDIST[5], HCLEN[4].
for f.nb < 5+5+4 {
if err := f.moreBits(); err != nil {
return err
}
}
nlit := int(f.b&0x1F) + 257
if nlit > maxNumLit {
if debugDecode {
fmt.Println("nlit > maxNumLit", nlit)
}
return CorruptInputError(f.roffset)
}
f.b >>= 5
ndist := int(f.b&0x1F) + 1
if ndist > maxNumDist {
if debugDecode {
fmt.Println("ndist > maxNumDist", ndist)
}
return CorruptInputError(f.roffset)
}
f.b >>= 5
nclen := int(f.b&0xF) + 4
// numCodes is 19, so nclen is always valid.
f.b >>= 4
f.nb -= 5 + 5 + 4
// (HCLEN+4)*3 bits: code lengths in the magic codeOrder order.
for i := range nclen {
for f.nb < 3 {
if err := f.moreBits(); err != nil {
return err
}
}
f.codebits[codeOrder[i]] = int(f.b & 0x7)
f.b >>= 3
f.nb -= 3
}
for i := nclen; i < len(codeOrder); i++ {
f.codebits[codeOrder[i]] = 0
}
if !f.h1.init(f.codebits[0:]) {
if debugDecode {
fmt.Println("init codebits failed")
}
return CorruptInputError(f.roffset)
}
// HLIT + 257 code lengths, HDIST + 1 code lengths,
// using the code length Huffman code.
for i, n := 0, nlit+ndist; i < n; {
x, err := f.huffSym(&f.h1)
if err != nil {
return err
}
if x < 16 {
// Actual length.
f.bits[i] = x
i++
continue
}
// Repeat previous length or zero.
var rep int
var nb uint
var b int
switch x {
default:
return InternalError("unexpected length code")
case 16:
rep = 3
nb = 2
if i == 0 {
if debugDecode {
fmt.Println("i==0")
}
return CorruptInputError(f.roffset)
}
b = f.bits[i-1]
case 17:
rep = 3
nb = 3
b = 0
case 18:
rep = 11
nb = 7
b = 0
}
for f.nb < nb {
if err := f.moreBits(); err != nil {
if debugDecode {
fmt.Println("morebits:", err)
}
return err
}
}
rep += int(f.b & uint32(1<<(nb&regmask.Shift32ByUint)-1))
f.b >>= nb & regmask.Shift32ByUint
f.nb -= nb
if i+rep > n {
if debugDecode {
fmt.Println("i+rep > n", i, rep, n)
}
return CorruptInputError(f.roffset)
}
for j := 0; j < rep; j++ {
f.bits[i] = b
i++
}
}
if !f.h1.init(f.bits[0:nlit]) || !f.h2.init(f.bits[nlit:nlit+ndist]) {
if debugDecode {
fmt.Println("init2 failed")
}
return CorruptInputError(f.roffset)
}
// As an optimization, we can initialize the maxRead bits to read at a time
// for the HLIT tree to the length of the EOB marker since we know that
// every block must terminate with one. This preserves the property that
// we never read any extra bytes after the end of the DEFLATE stream.
if f.h1.maxRead < f.bits[endBlockMarker] {
f.h1.maxRead = f.bits[endBlockMarker]
}
if !f.final {
// If not the final block, the smallest block possible is
// a predefined table, BTYPE=01, with a single EOB marker.
// This will take up 3 + 7 bits.
f.h1.maxRead += 10
}
return nil
}
// Copy a single uncompressed data block from input to output.
func (f *decompressor) dataBlock() {
// Uncompressed.
// Discard current half-byte.
left := (f.nb) & 7
f.nb -= left
f.b >>= left
offBytes := f.nb >> 3
// Unfilled values will be overwritten.
f.buf[0] = uint8(f.b)
f.buf[1] = uint8(f.b >> 8)
f.buf[2] = uint8(f.b >> 16)
f.buf[3] = uint8(f.b >> 24)
f.roffset += int64(offBytes)
f.nb, f.b = 0, 0
// Length then ones-complement of length.
nr, err := io.ReadFull(f.r, f.buf[offBytes:4])
f.roffset += int64(nr)
if err != nil {
f.err = noEOF(err)
return
}
n := uint16(f.buf[0]) | uint16(f.buf[1])<<8
nn := uint16(f.buf[2]) | uint16(f.buf[3])<<8
if nn != ^n {
if debugDecode {
ncomp := ^n
fmt.Println("uint16(nn) != uint16(^n)", nn, ncomp)
}
f.err = CorruptInputError(f.roffset)
return
}
if n == 0 {
if f.flushMode == syncFlush {
f.toRead = f.dict.readFlush()
}
f.finishBlock()
return
}
f.copyLen = int(n)
f.copyData()
}
// copyData copies f.copyLen bytes from the underlying reader into f.hist.
// It pauses for reads when f.hist is full.
func (f *decompressor) copyData() {
buf := f.dict.writeSlice()
if len(buf) > f.copyLen {
buf = buf[:f.copyLen]
}
cnt, err := io.ReadFull(f.r, buf)
f.roffset += int64(cnt)
f.copyLen -= cnt
f.dict.writeMark(cnt)
if err != nil {
f.err = noEOF(err)
return
}
if f.dict.availWrite() == 0 || f.copyLen > 0 {
f.toRead = f.dict.readFlush()
f.step = copyData
return
}
f.finishBlock()
}
func (f *decompressor) finishBlock() {
if f.final {
if f.dict.availRead() > 0 {
f.toRead = f.dict.readFlush()
}
f.err = io.EOF
} else if f.flushMode == partialFlush && f.dict.availRead() > 0 {
f.toRead = f.dict.readFlush()
}
if f.cb != nil {
bitPos := f.roffset*8 - int64(f.nb)
f.cpBuf = f.dict.appendWindow(f.cpBuf[:0])
f.cb(InflateCheckpoint{
UncompressedOffset: f.uncOffset + f.dict.decoded(),
CompressedOffset: bitPos / 8,
Final: f.final,
BitOffset: uint8(bitPos & 7),
Window: f.cpBuf,
})
}
f.step = nextBlock
}
func (f *decompressor) doStep() {
switch f.step {
case copyData:
f.copyData()
case nextBlock:
f.nextBlock()
case huffmanBytesBuffer:
f.huffmanBytesBuffer()
case huffmanBytesReader:
f.huffmanBytesReader()
case huffmanBufioReader:
f.huffmanBufioReader()
case huffmanStringsReader:
f.huffmanStringsReader()
case huffmanGenericReader:
f.huffmanGenericReader()
default:
panic("BUG: unexpected step state")
}
}
// noEOF returns err, unless err == io.EOF, in which case it returns io.ErrUnexpectedEOF.
func noEOF(e error) error {
if e == io.EOF {
return io.ErrUnexpectedEOF
}
return e
}
func (f *decompressor) moreBits() error {
c, err := f.r.ReadByte()
if err != nil {
return noEOF(err)
}
f.roffset++
f.b |= uint32(c) << (f.nb & regmask.Shift32ByUint)
f.nb += 8
return nil
}
// Read the next Huffman-encoded symbol from f according to h.
func (f *decompressor) huffSym(h *huffmanDecoder) (int, error) {
// Since a huffmanDecoder can be empty or be composed of a degenerate tree
// with single element, huffSym must error on these two edge cases. In both
// cases, the chunks slice will be 0 for the invalid sequence, leading it
// satisfy the n == 0 check below.
n := uint(h.maxRead)
// Optimization. Compiler isn't smart enough to keep f.b,f.nb in registers,
// but is smart enough to keep local variables in registers, so use nb and b,
// inline call to moreBits and reassign b,nb back to f on return.
nb, b := f.nb, f.b
for {
for nb < n {
c, err := f.r.ReadByte()
if err != nil {
f.b = b
f.nb = nb
return 0, noEOF(err)
}
f.roffset++
b |= uint32(c) << (nb & regmask.Shift32ByUint)
nb += 8
}
chunk := h.chunks[b&(huffmanNumChunks-1)]
n = uint(chunk & huffmanCountMask)
if n > huffmanChunkBits {
chunk = h.links[chunk>>huffmanValueShift][(b>>huffmanChunkBits)&h.linkMask]
n = uint(chunk & huffmanCountMask)
}
if n <= nb {
if n == 0 {
f.b = b
f.nb = nb
if debugDecode {
fmt.Println("huffsym: n==0")
}
f.err = CorruptInputError(f.roffset)
return 0, f.err
}
f.b = b >> (n & regmask.Shift32ByUint)
f.nb = nb - n
return int(chunk >> huffmanValueShift), nil
}
}
}
func makeReader(r io.Reader) Reader {
if rr, ok := r.(Reader); ok {
return rr
}
return bufio.NewReader(r)
}
func fixedHuffmanDecoderInit() {
fixedOnce.Do(func() {
// These come from the RFC section 3.2.6.
var bits [288]int
for i := range 144 {
bits[i] = 8
}
for i := 144; i < 256; i++ {
bits[i] = 9
}
for i := 256; i < 280; i++ {
bits[i] = 7
}
for i := 280; i < 288; i++ {
bits[i] = 8
}
fixedHuffmanDecoder.init(bits[:])
})
}
func (f *decompressor) Reset(r io.Reader, dict []byte) error {
*f = decompressor{
r: makeReader(r),
bits: f.bits,
codebits: f.codebits,
h1: f.h1,
h2: f.h2,
dict: f.dict,
step: nextBlock,
}
f.dict.init(maxMatchOffset, dict)
return nil
}
// ResetCP will adjust the input to the provided checkpoint.
// It is assumed the input stream is forwarded to cp.CompressedOffset.
func (f *decompressor) ResetCP(r io.Reader, cp InflateCheckpoint) error {
*f = decompressor{
r: makeReader(r),
bits: f.bits,
codebits: f.codebits,
h1: f.h1,
h2: f.h2,
dict: f.dict,
step: nextBlock,
cpBuf: f.cpBuf,
}
return f.applyCP(cp)
}
// applyCP seeds the decompressor state from a resume checkpoint:
// loads the sliding window, sets the absolute compressed/uncompressed
// offsets, and skips cp.BitOffset bits into the first input byte so
// the next decode aligns with the start of a deflate block.
func (f *decompressor) applyCP(cp InflateCheckpoint) error {
f.dict.init(maxMatchOffset, cp.Window)
f.roffset = cp.CompressedOffset
f.uncOffset = cp.UncompressedOffset
f.final = cp.Final
f.b = 0
f.nb = 0
if cp.BitOffset > 0 {
c, err := f.r.ReadByte()
if err != nil {
return noEOF(err)
}
f.roffset++
f.b = uint32(c) >> cp.BitOffset
f.nb = 8 - uint(cp.BitOffset)
}
return nil
}
type ReaderOpt func(*decompressor)
// WithPartialBlock tells decompressor to return after each block,
// so it can read data written with partial flush
func WithPartialBlock() ReaderOpt {
return func(f *decompressor) {
f.flushMode = partialFlush
}
}
// WithDict initializes the reader with a preset dictionary
func WithDict(dict []byte) ReaderOpt {
return func(f *decompressor) {
f.dict.init(maxMatchOffset, dict)
}
}
// InflateCheckpoint provides a resumable checkpoint for inflate.
type InflateCheckpoint struct {
UncompressedOffset int64 // Byte offset in the decompressed stream
CompressedOffset int64 // Byte offset in the compressed stream
Final bool // True if this is the final block
BitOffset uint8 // 0-7 bits
Window []byte // 32KB sliding window dictionary
}
// WithEobCallback will call the provided function after each block
// with the current gzip checkpoint.
// After returning the provided window can no longer be referenced.
// The callback will not be triggered after a block is marked "final".
// The callback is not retained after Reset.
func WithEobCallback(cb func(InflateCheckpoint)) ReaderOpt {
return func(f *decompressor) {
f.cb = cb
}
}
// WithResumeFrom will adjust the input to the provided checkpoint.
// It is assumed the input stream is forwarded to the provided offset.
// The checkpoint is removed when Reset is called.
func WithResumeFrom(cp InflateCheckpoint) ReaderOpt {
return func(f *decompressor) {
f.cp = cp
f.hasCP = true
}
}
// NewReaderOpts returns new reader with provided options
func NewReaderOpts(r io.Reader, opts ...ReaderOpt) io.ReadCloser {
fixedHuffmanDecoderInit()
var f decompressor
f.r = makeReader(r)
f.bits = new([maxNumLit + maxNumDist]int)
f.codebits = new([numCodes]int)
f.step = nextBlock
f.dict.init(maxMatchOffset, nil)
for _, opt := range opts {
opt(&f)
}
if f.hasCP {
if err := f.applyCP(f.cp); err != nil {
f.err = err
}
}
return &f
}
// NewReader returns a new ReadCloser that can be used
// to read the uncompressed version of r.
// If r does not also implement io.ByteReader,
// the decompressor may read more data than necessary from r.
// It is the caller's responsibility to call Close on the ReadCloser
// when finished reading.
//
// The ReadCloser returned by NewReader also implements Resetter.
func NewReader(r io.Reader) io.ReadCloser {
return NewReaderOpts(r)
}
// NewReaderDict is like NewReader but initializes the reader
// with a preset dictionary. The returned Reader behaves as if
// the uncompressed data stream started with the given dictionary,
// which has already been read. NewReaderDict is typically used
// to read data compressed by NewWriterDict.
//
// The ReadCloser returned by NewReader also implements Resetter.
func NewReaderDict(r io.Reader, dict []byte) io.ReadCloser {
return NewReaderOpts(r, WithDict(dict))
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/inflate_gen.go
Generated Vendored
+1339
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level1.go
Generated Vendored
+215
View File
@@ -0,0 +1,215 @@
package flate
import (
"fmt"
"github.com/klauspost/compress/internal/le"
)
// fastGen maintains the table for matches,
// and the previous byte block for level 2.
// This is the generic implementation.
type fastEncL1 struct {
fastGen
table [tableSize]tableEntry
}
// EncodeL1 uses a similar algorithm to level 1
func (e *fastEncL1) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashBytes = 5
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
const skipLog = 5
const doEvery = 2
nextS := s
var candidate tableEntry
var t int32
for {
nextHash := hashLen(cv, tableBits, hashBytes)
candidate = e.table[nextHash]
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
now := load6432(src, nextS)
e.table[nextHash] = tableEntry{offset: s + e.cur}
nextHash = hashLen(now, tableBits, hashBytes)
t = candidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
e.table[nextHash] = tableEntry{offset: nextS + e.cur}
break
}
// Do one right away...
cv = now
s = nextS
nextS++
candidate = e.table[nextHash]
now >>= 8
e.table[nextHash] = tableEntry{offset: s + e.cur}
t = candidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
e.table[nextHash] = tableEntry{offset: nextS + e.cur}
break
}
cv = now
s = nextS
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
for {
// Invariant: we have a 4-byte match at s, and no need to emit any
// literal bytes prior to s.
// Extend the 4-byte match as long as possible.
l := e.matchlenLong(int(s+4), int(t+4), src) + 4
// Extend backwards
for t > 0 && s > nextEmit && le.Load8(src, t-1) == le.Load8(src, s-1) {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
// Save the match found
if false {
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
} else {
// Inlined...
xoffset := uint32(s - t - baseMatchOffset)
xlength := l
oc := offsetCode(xoffset)
xoffset |= oc << 16
for xlength > 0 {
xl := xlength
if xl > 258 {
if xl > 258+baseMatchLength {
xl = 258
} else {
xl = 258 - baseMatchLength
}
}
xlength -= xl
xl -= baseMatchLength
dst.extraHist[lengthCodes1[uint8(xl)]]++
dst.offHist[oc]++
dst.tokens[dst.n] = token(matchType | uint32(xl)<<lengthShift | xoffset)
dst.n++
}
}
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
// Index first pair after match end.
if int(s+l+8) < len(src) {
cv := load6432(src, s)
e.table[hashLen(cv, tableBits, hashBytes)] = tableEntry{offset: s + e.cur}
}
goto emitRemainder
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-2 and at s. If
// another emitCopy is not our next move, also calculate nextHash
// at s+1. At least on GOARCH=amd64, these three hash calculations
// are faster as one load64 call (with some shifts) instead of
// three load32 calls.
x := load6432(src, s-2)
o := e.cur + s - 2
prevHash := hashLen(x, tableBits, hashBytes)
e.table[prevHash] = tableEntry{offset: o}
x >>= 16
currHash := hashLen(x, tableBits, hashBytes)
candidate = e.table[currHash]
e.table[currHash] = tableEntry{offset: o + 2}
t = candidate.offset - e.cur
if s-t > maxMatchOffset || uint32(x) != load3232(src, t) {
cv = x >> 8
s++
break
}
}
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level2.go
Generated Vendored
+214
View File
@@ -0,0 +1,214 @@
package flate
import "fmt"
// fastGen maintains the table for matches,
// and the previous byte block for level 2.
// This is the generic implementation.
type fastEncL2 struct {
fastGen
table [bTableSize]tableEntry
}
// EncodeL2 uses a similar algorithm to level 1, but is capable
// of matching across blocks giving better compression at a small slowdown.
func (e *fastEncL2) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashBytes = 5
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
// When should we start skipping if we haven't found matches in a long while.
const skipLog = 5
const doEvery = 2
nextS := s
var candidate tableEntry
for {
nextHash := hashLen(cv, bTableBits, hashBytes)
s = nextS
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
candidate = e.table[nextHash]
now := load6432(src, nextS)
e.table[nextHash] = tableEntry{offset: s + e.cur}
nextHash = hashLen(now, bTableBits, hashBytes)
offset := s - (candidate.offset - e.cur)
if offset < maxMatchOffset && uint32(cv) == load3232(src, candidate.offset-e.cur) {
e.table[nextHash] = tableEntry{offset: nextS + e.cur}
break
}
// Do one right away...
cv = now
s = nextS
nextS++
candidate = e.table[nextHash]
now >>= 8
e.table[nextHash] = tableEntry{offset: s + e.cur}
offset = s - (candidate.offset - e.cur)
if offset < maxMatchOffset && uint32(cv) == load3232(src, candidate.offset-e.cur) {
break
}
cv = now
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
// Call emitCopy, and then see if another emitCopy could be our next
// move. Repeat until we find no match for the input immediately after
// what was consumed by the last emitCopy call.
//
// If we exit this loop normally then we need to call emitLiteral next,
// though we don't yet know how big the literal will be. We handle that
// by proceeding to the next iteration of the main loop. We also can
// exit this loop via goto if we get close to exhausting the input.
for {
// Invariant: we have a 4-byte match at s, and no need to emit any
// literal bytes prior to s.
// Extend the 4-byte match as long as possible.
t := candidate.offset - e.cur
l := e.matchlenLong(int(s+4), int(t+4), src) + 4
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
// Index first pair after match end.
if int(s+l+8) < len(src) {
cv := load6432(src, s)
e.table[hashLen(cv, bTableBits, hashBytes)] = tableEntry{offset: s + e.cur}
}
goto emitRemainder
}
// Store every second hash in-between, but offset by 1.
for i := s - l + 2; i < s-5; i += 7 {
x := load6432(src, i)
nextHash := hashLen(x, bTableBits, hashBytes)
e.table[nextHash] = tableEntry{offset: e.cur + i}
// Skip one
x >>= 16
nextHash = hashLen(x, bTableBits, hashBytes)
e.table[nextHash] = tableEntry{offset: e.cur + i + 2}
// Skip one
x >>= 16
nextHash = hashLen(x, bTableBits, hashBytes)
e.table[nextHash] = tableEntry{offset: e.cur + i + 4}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-2 to s. If
// another emitCopy is not our next move, also calculate nextHash
// at s+1. At least on GOARCH=amd64, these three hash calculations
// are faster as one load64 call (with some shifts) instead of
// three load32 calls.
x := load6432(src, s-2)
o := e.cur + s - 2
prevHash := hashLen(x, bTableBits, hashBytes)
prevHash2 := hashLen(x>>8, bTableBits, hashBytes)
e.table[prevHash] = tableEntry{offset: o}
e.table[prevHash2] = tableEntry{offset: o + 1}
currHash := hashLen(x>>16, bTableBits, hashBytes)
candidate = e.table[currHash]
e.table[currHash] = tableEntry{offset: o + 2}
offset := s - (candidate.offset - e.cur)
if offset > maxMatchOffset || uint32(x>>16) != load3232(src, candidate.offset-e.cur) {
cv = x >> 24
s++
break
}
}
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level3.go
Generated Vendored
+241
View File
@@ -0,0 +1,241 @@
package flate
import "fmt"
// fastEncL3
type fastEncL3 struct {
fastGen
table [1 << 16]tableEntryPrev
}
// Encode uses a similar algorithm to level 2, will check up to two candidates.
func (e *fastEncL3) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
tableBits = 16
tableSize = 1 << tableBits
hashBytes = 5
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntryPrev{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i]
if v.Cur.offset <= minOff {
v.Cur.offset = 0
} else {
v.Cur.offset = v.Cur.offset - e.cur + maxMatchOffset
}
if v.Prev.offset <= minOff {
v.Prev.offset = 0
} else {
v.Prev.offset = v.Prev.offset - e.cur + maxMatchOffset
}
e.table[i] = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// Skip if too small.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
const skipLog = 7
nextS := s
var candidate tableEntry
for {
nextHash := hashLen(cv, tableBits, hashBytes)
s = nextS
nextS = s + 1 + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
candidates := e.table[nextHash]
now := load6432(src, nextS)
// Safe offset distance until s + 4...
minOffset := e.cur + s - (maxMatchOffset - 4)
e.table[nextHash] = tableEntryPrev{Prev: candidates.Cur, Cur: tableEntry{offset: s + e.cur}}
// Check both candidates
candidate = candidates.Cur
if candidate.offset < minOffset {
cv = now
// Previous will also be invalid, we have nothing.
continue
}
if uint32(cv) == load3232(src, candidate.offset-e.cur) {
if candidates.Prev.offset < minOffset || uint32(cv) != load3232(src, candidates.Prev.offset-e.cur) {
break
}
// Both match and are valid, pick longest.
offset := s - (candidate.offset - e.cur)
o2 := s - (candidates.Prev.offset - e.cur)
l1, l2 := matchLen(src[s+4:], src[s-offset+4:]), matchLen(src[s+4:], src[s-o2+4:])
if l2 > l1 {
candidate = candidates.Prev
}
break
} else {
// We only check if value mismatches.
// Offset will always be invalid in other cases.
candidate = candidates.Prev
if candidate.offset > minOffset && uint32(cv) == load3232(src, candidate.offset-e.cur) {
break
}
}
cv = now
}
// Call emitCopy, and then see if another emitCopy could be our next
// move. Repeat until we find no match for the input immediately after
// what was consumed by the last emitCopy call.
//
// If we exit this loop normally then we need to call emitLiteral next,
// though we don't yet know how big the literal will be. We handle that
// by proceeding to the next iteration of the main loop. We also can
// exit this loop via goto if we get close to exhausting the input.
for {
// Invariant: we have a 4-byte match at s, and no need to emit any
// literal bytes prior to s.
// Extend the 4-byte match as long as possible.
//
t := candidate.offset - e.cur
l := e.matchlenLong(int(s+4), int(t+4), src) + 4
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
t += l
// Index first pair after match end.
if int(t+8) < len(src) && t > 0 {
cv = load6432(src, t)
nextHash := hashLen(cv, tableBits, hashBytes)
e.table[nextHash] = tableEntryPrev{
Prev: e.table[nextHash].Cur,
Cur: tableEntry{offset: e.cur + t},
}
}
goto emitRemainder
}
// Store every 5th hash in-between.
for i := s - l + 2; i < s-5; i += 6 {
nextHash := hashLen(load6432(src, i), tableBits, hashBytes)
e.table[nextHash] = tableEntryPrev{
Prev: e.table[nextHash].Cur,
Cur: tableEntry{offset: e.cur + i}}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-2 to s.
x := load6432(src, s-2)
prevHash := hashLen(x, tableBits, hashBytes)
e.table[prevHash] = tableEntryPrev{
Prev: e.table[prevHash].Cur,
Cur: tableEntry{offset: e.cur + s - 2},
}
x >>= 8
prevHash = hashLen(x, tableBits, hashBytes)
e.table[prevHash] = tableEntryPrev{
Prev: e.table[prevHash].Cur,
Cur: tableEntry{offset: e.cur + s - 1},
}
x >>= 8
currHash := hashLen(x, tableBits, hashBytes)
candidates := e.table[currHash]
cv = x
e.table[currHash] = tableEntryPrev{
Prev: candidates.Cur,
Cur: tableEntry{offset: s + e.cur},
}
// Check both candidates
candidate = candidates.Cur
minOffset := e.cur + s - (maxMatchOffset - 4)
if candidate.offset > minOffset {
if uint32(cv) == load3232(src, candidate.offset-e.cur) {
// Found a match...
continue
}
candidate = candidates.Prev
if candidate.offset > minOffset && uint32(cv) == load3232(src, candidate.offset-e.cur) {
// Match at prev...
continue
}
}
cv = x >> 8
s++
break
}
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level4.go
Generated Vendored
+221
View File
@@ -0,0 +1,221 @@
package flate
import "fmt"
type fastEncL4 struct {
fastGen
table [tableSize]tableEntry
bTable [tableSize]tableEntry
}
func (e *fastEncL4) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashShortBytes = 4
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
for i := range e.bTable[:] {
e.bTable[i] = tableEntry{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
for i := range e.bTable[:] {
v := e.bTable[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.bTable[i].offset = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
const skipLog = 6
const doEvery = 1
nextS := s
var t int32
for {
nextHashS := hashLen(cv, tableBits, hashShortBytes)
nextHashL := hash7(cv, tableBits)
s = nextS
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
// Fetch a short+long candidate
sCandidate := e.table[nextHashS]
lCandidate := e.bTable[nextHashL]
next := load6432(src, nextS)
entry := tableEntry{offset: s + e.cur}
e.table[nextHashS] = entry
e.bTable[nextHashL] = entry
t = lCandidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// We got a long match. Use that.
break
}
t = sCandidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Found a 4 match...
lCandidate = e.bTable[hash7(next, tableBits)]
// If the next long is a candidate, check if we should use that instead...
lOff := lCandidate.offset - e.cur
if nextS-lOff < maxMatchOffset && load3232(src, lOff) == uint32(next) {
l1, l2 := matchLen(src[s+4:], src[t+4:]), matchLen(src[nextS+4:], src[nextS-lOff+4:])
if l2 > l1 {
s = nextS
t = lCandidate.offset - e.cur
}
}
break
}
cv = next
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
// Extend the 4-byte match as long as possible.
l := e.matchlenLong(int(s+4), int(t+4), src) + 4
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
if debugDeflate {
if t >= s {
panic("s-t")
}
if (s - t) > maxMatchOffset {
panic(fmt.Sprintln("mmo", t))
}
if l < baseMatchLength {
panic("bml")
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
// Index first pair after match end.
if int(s+8) < len(src) {
cv := load6432(src, s)
e.table[hashLen(cv, tableBits, hashShortBytes)] = tableEntry{offset: s + e.cur}
e.bTable[hash7(cv, tableBits)] = tableEntry{offset: s + e.cur}
}
goto emitRemainder
}
// Store every 3rd hash in-between
if true {
i := nextS
if i < s-1 {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
t2 := tableEntry{offset: t.offset + 1}
e.bTable[hash7(cv, tableBits)] = t
e.bTable[hash7(cv>>8, tableBits)] = t2
e.table[hashLen(cv>>8, tableBits, hashShortBytes)] = t2
i += 3
for ; i < s-1; i += 3 {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
t2 := tableEntry{offset: t.offset + 1}
e.bTable[hash7(cv, tableBits)] = t
e.bTable[hash7(cv>>8, tableBits)] = t2
e.table[hashLen(cv>>8, tableBits, hashShortBytes)] = t2
}
}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-1 and at s.
x := load6432(src, s-1)
o := e.cur + s - 1
prevHashS := hashLen(x, tableBits, hashShortBytes)
prevHashL := hash7(x, tableBits)
e.table[prevHashS] = tableEntry{offset: o}
e.bTable[prevHashL] = tableEntry{offset: o}
cv = x >> 8
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level5.go
Generated Vendored
+705
View File
@@ -0,0 +1,705 @@
package flate
import "fmt"
type fastEncL5 struct {
fastGen
table [tableSize]tableEntry
bTable [tableSize]tableEntryPrev
}
func (e *fastEncL5) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashShortBytes = 4
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
for i := range e.bTable[:] {
e.bTable[i] = tableEntryPrev{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
for i := range e.bTable[:] {
v := e.bTable[i]
if v.Cur.offset <= minOff {
v.Cur.offset = 0
v.Prev.offset = 0
} else {
v.Cur.offset = v.Cur.offset - e.cur + maxMatchOffset
if v.Prev.offset <= minOff {
v.Prev.offset = 0
} else {
v.Prev.offset = v.Prev.offset - e.cur + maxMatchOffset
}
}
e.bTable[i] = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
const skipLog = 6
const doEvery = 1
nextS := s
var l int32
var t int32
for {
nextHashS := hashLen(cv, tableBits, hashShortBytes)
nextHashL := hash7(cv, tableBits)
s = nextS
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
// Fetch a short+long candidate
sCandidate := e.table[nextHashS]
lCandidate := e.bTable[nextHashL]
next := load6432(src, nextS)
entry := tableEntry{offset: s + e.cur}
e.table[nextHashS] = entry
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = entry, eLong.Cur
nextHashS = hashLen(next, tableBits, hashShortBytes)
nextHashL = hash7(next, tableBits)
t = lCandidate.Cur.offset - e.cur
if s-t < maxMatchOffset {
if uint32(cv) == load3232(src, t) {
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
t2 := lCandidate.Prev.offset - e.cur
if s-t2 < maxMatchOffset && uint32(cv) == load3232(src, t2) {
l = e.matchlen(int(s+4), int(t+4), src) + 4
ml1 := e.matchlen(int(s+4), int(t2+4), src) + 4
if ml1 > l {
t = t2
l = ml1
break
}
}
break
}
t = lCandidate.Prev.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
break
}
}
t = sCandidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Found a 4 match...
l = e.matchlen(int(s+4), int(t+4), src) + 4
lCandidate = e.bTable[nextHashL]
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
// If the next long is a candidate, use that...
t2 := lCandidate.Cur.offset - e.cur
if nextS-t2 < maxMatchOffset {
if load3232(src, t2) == uint32(next) {
ml := e.matchlen(int(nextS+4), int(t2+4), src) + 4
if ml > l {
t = t2
s = nextS
l = ml
break
}
}
// If the previous long is a candidate, use that...
t2 = lCandidate.Prev.offset - e.cur
if nextS-t2 < maxMatchOffset && load3232(src, t2) == uint32(next) {
ml := e.matchlen(int(nextS+4), int(t2+4), src) + 4
if ml > l {
t = t2
s = nextS
l = ml
break
}
}
}
break
}
cv = next
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
if l == 0 {
// Extend the 4-byte match as long as possible.
l = e.matchlenLong(int(s+4), int(t+4), src) + 4
} else if l == maxMatchLength {
l += e.matchlenLong(int(s+l), int(t+l), src)
}
// Try to locate a better match by checking the end of best match...
if sAt := s + l; l < 30 && sAt < sLimit {
// Allow some bytes at the beginning to mismatch.
// Sweet spot is 2/3 bytes depending on input.
// 3 is only a little better when it is but sometimes a lot worse.
// The skipped bytes are tested in Extend backwards,
// and still picked up as part of the match if they do.
const skipBeginning = 2
eLong := e.bTable[hash7(load6432(src, sAt), tableBits)].Cur.offset
t2 := eLong - e.cur - l + skipBeginning
s2 := s + skipBeginning
off := s2 - t2
if t2 >= 0 && off < maxMatchOffset && off > 0 {
if l2 := e.matchlenLong(int(s2), int(t2), src); l2 > l {
t = t2
l = l2
s = s2
}
}
}
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
if debugDeflate {
if t >= s {
panic(fmt.Sprintln("s-t", s, t))
}
if (s - t) > maxMatchOffset {
panic(fmt.Sprintln("mmo", s-t))
}
if l < baseMatchLength {
panic("bml")
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
goto emitRemainder
}
// Store every 3rd hash in-between.
if true {
const hashEvery = 3
i := s - l + 1
if i < s-1 {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
e.table[hashLen(cv, tableBits, hashShortBytes)] = t
eLong := &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
// Do an long at i+1
cv >>= 8
t = tableEntry{offset: t.offset + 1}
eLong = &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
// We only have enough bits for a short entry at i+2
cv >>= 8
t = tableEntry{offset: t.offset + 1}
e.table[hashLen(cv, tableBits, hashShortBytes)] = t
// Skip one - otherwise we risk hitting 's'
i += 4
for ; i < s-1; i += hashEvery {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
t2 := tableEntry{offset: t.offset + 1}
eLong := &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
e.table[hashLen(cv>>8, tableBits, hashShortBytes)] = t2
}
}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-1 and at s.
x := load6432(src, s-1)
o := e.cur + s - 1
prevHashS := hashLen(x, tableBits, hashShortBytes)
prevHashL := hash7(x, tableBits)
e.table[prevHashS] = tableEntry{offset: o}
eLong := &e.bTable[prevHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: o}, eLong.Cur
cv = x >> 8
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
// fastEncL5Window is a level 5 encoder,
// but with a custom window size.
type fastEncL5Window struct {
hist []byte
cur int32
maxOffset int32
table [tableSize]tableEntry
bTable [tableSize]tableEntryPrev
}
func (e *fastEncL5Window) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashShortBytes = 4
)
maxMatchOffset := e.maxOffset
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
for i := range e.bTable[:] {
e.bTable[i] = tableEntryPrev{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
for i := range e.bTable[:] {
v := e.bTable[i]
if v.Cur.offset <= minOff {
v.Cur.offset = 0
v.Prev.offset = 0
} else {
v.Cur.offset = v.Cur.offset - e.cur + maxMatchOffset
if v.Prev.offset <= minOff {
v.Prev.offset = 0
} else {
v.Prev.offset = v.Prev.offset - e.cur + maxMatchOffset
}
}
e.bTable[i] = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
for {
const skipLog = 6
const doEvery = 1
nextS := s
var l int32
var t int32
for {
nextHashS := hashLen(cv, tableBits, hashShortBytes)
nextHashL := hash7(cv, tableBits)
s = nextS
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
// Fetch a short+long candidate
sCandidate := e.table[nextHashS]
lCandidate := e.bTable[nextHashL]
next := load6432(src, nextS)
entry := tableEntry{offset: s + e.cur}
e.table[nextHashS] = entry
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = entry, eLong.Cur
nextHashS = hashLen(next, tableBits, hashShortBytes)
nextHashL = hash7(next, tableBits)
t = lCandidate.Cur.offset - e.cur
if s-t < maxMatchOffset {
if uint32(cv) == load3232(src, t) {
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
t2 := lCandidate.Prev.offset - e.cur
if s-t2 < maxMatchOffset && uint32(cv) == load3232(src, t2) {
l = e.matchlen(s+4, t+4, src) + 4
ml1 := e.matchlen(s+4, t2+4, src) + 4
if ml1 > l {
t = t2
l = ml1
break
}
}
break
}
t = lCandidate.Prev.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
break
}
}
t = sCandidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Found a 4 match...
l = e.matchlen(s+4, t+4, src) + 4
lCandidate = e.bTable[nextHashL]
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
// If the next long is a candidate, use that...
t2 := lCandidate.Cur.offset - e.cur
if nextS-t2 < maxMatchOffset {
if load3232(src, t2) == uint32(next) {
ml := e.matchlen(nextS+4, t2+4, src) + 4
if ml > l {
t = t2
s = nextS
l = ml
break
}
}
// If the previous long is a candidate, use that...
t2 = lCandidate.Prev.offset - e.cur
if nextS-t2 < maxMatchOffset && load3232(src, t2) == uint32(next) {
ml := e.matchlen(nextS+4, t2+4, src) + 4
if ml > l {
t = t2
s = nextS
l = ml
break
}
}
}
break
}
cv = next
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
if l == 0 {
// Extend the 4-byte match as long as possible.
l = e.matchlenLong(s+4, t+4, src) + 4
} else if l == maxMatchLength {
l += e.matchlenLong(s+l, t+l, src)
}
// Try to locate a better match by checking the end of best match...
if sAt := s + l; l < 30 && sAt < sLimit {
// Allow some bytes at the beginning to mismatch.
// Sweet spot is 2/3 bytes depending on input.
// 3 is only a little better when it is but sometimes a lot worse.
// The skipped bytes are tested in Extend backwards,
// and still picked up as part of the match if they do.
const skipBeginning = 2
eLong := e.bTable[hash7(load6432(src, sAt), tableBits)].Cur.offset
t2 := eLong - e.cur - l + skipBeginning
s2 := s + skipBeginning
off := s2 - t2
if t2 >= 0 && off < maxMatchOffset && off > 0 {
if l2 := e.matchlenLong(s2, t2, src); l2 > l {
t = t2
l = l2
s = s2
}
}
}
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
if debugDeflate {
if t >= s {
panic(fmt.Sprintln("s-t", s, t))
}
if (s - t) > maxMatchOffset {
panic(fmt.Sprintln("mmo", s-t))
}
if l < baseMatchLength {
panic("bml")
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
goto emitRemainder
}
// Store every 3rd hash in-between.
if true {
const hashEvery = 3
i := s - l + 1
if i < s-1 {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
e.table[hashLen(cv, tableBits, hashShortBytes)] = t
eLong := &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
// Do an long at i+1
cv >>= 8
t = tableEntry{offset: t.offset + 1}
eLong = &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
// We only have enough bits for a short entry at i+2
cv >>= 8
t = tableEntry{offset: t.offset + 1}
e.table[hashLen(cv, tableBits, hashShortBytes)] = t
// Skip one - otherwise we risk hitting 's'
i += 4
for ; i < s-1; i += hashEvery {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
t2 := tableEntry{offset: t.offset + 1}
eLong := &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = t, eLong.Cur
e.table[hashLen(cv>>8, tableBits, hashShortBytes)] = t2
}
}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-1 and at s.
x := load6432(src, s-1)
o := e.cur + s - 1
prevHashS := hashLen(x, tableBits, hashShortBytes)
prevHashL := hash7(x, tableBits)
e.table[prevHashS] = tableEntry{offset: o}
eLong := &e.bTable[prevHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: o}, eLong.Cur
cv = x >> 8
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
// Reset the encoding table.
func (e *fastEncL5Window) Reset() {
// We keep the same allocs, since we are compressing the same block sizes.
if cap(e.hist) < allocHistory {
e.hist = make([]byte, 0, allocHistory)
}
// We offset current position so everything will be out of reach.
// If we are above the buffer reset it will be cleared anyway since len(hist) == 0.
if e.cur <= int32(bufferReset) {
e.cur += e.maxOffset + int32(len(e.hist))
}
e.hist = e.hist[:0]
}
func (e *fastEncL5Window) addBlock(src []byte) int32 {
// check if we have space already
maxMatchOffset := e.maxOffset
if len(e.hist)+len(src) > cap(e.hist) {
if cap(e.hist) == 0 {
e.hist = make([]byte, 0, allocHistory)
} else {
if cap(e.hist) < int(maxMatchOffset*2) {
panic("unexpected buffer size")
}
// Move down
offset := int32(len(e.hist)) - maxMatchOffset
copy(e.hist[0:maxMatchOffset], e.hist[offset:])
e.cur += offset
e.hist = e.hist[:maxMatchOffset]
}
}
s := int32(len(e.hist))
e.hist = append(e.hist, src...)
return s
}
// matchlen will return the match length between offsets and t in src.
// The maximum length returned is maxMatchLength - 4.
// It is assumed that s > t, that t >=0 and s < len(src).
func (e *fastEncL5Window) matchlen(s, t int32, src []byte) int32 {
if debugDecode {
if t >= s {
panic(fmt.Sprint("t >=s:", t, s))
}
if int(s) >= len(src) {
panic(fmt.Sprint("s >= len(src):", s, len(src)))
}
if t < 0 {
panic(fmt.Sprint("t < 0:", t))
}
if s-t > e.maxOffset {
panic(fmt.Sprint(s, "-", t, "(", s-t, ") > maxMatchLength (", maxMatchOffset, ")"))
}
}
s1 := min(int(s)+maxMatchLength-4, len(src))
// Extend the match to be as long as possible.
return int32(matchLen(src[s:s1], src[t:]))
}
// matchlenLong will return the match length between offsets and t in src.
// It is assumed that s > t, that t >=0 and s < len(src).
func (e *fastEncL5Window) matchlenLong(s, t int32, src []byte) int32 {
if debugDeflate {
if t >= s {
panic(fmt.Sprint("t >=s:", t, s))
}
if int(s) >= len(src) {
panic(fmt.Sprint("s >= len(src):", s, len(src)))
}
if t < 0 {
panic(fmt.Sprint("t < 0:", t))
}
if s-t > e.maxOffset {
panic(fmt.Sprint(s, "-", t, "(", s-t, ") > maxMatchLength (", maxMatchOffset, ")"))
}
}
// Extend the match to be as long as possible.
return int32(matchLen(src[s:], src[t:]))
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/level6.go
Generated Vendored
+325
View File
@@ -0,0 +1,325 @@
package flate
import "fmt"
type fastEncL6 struct {
fastGen
table [tableSize]tableEntry
bTable [tableSize]tableEntryPrev
}
func (e *fastEncL6) Encode(dst *tokens, src []byte) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
hashShortBytes = 4
)
if debugDeflate && e.cur < 0 {
panic(fmt.Sprint("e.cur < 0: ", e.cur))
}
// Protect against e.cur wraparound.
for e.cur >= bufferReset {
if len(e.hist) == 0 {
for i := range e.table[:] {
e.table[i] = tableEntry{}
}
for i := range e.bTable[:] {
e.bTable[i] = tableEntryPrev{}
}
e.cur = maxMatchOffset
break
}
// Shift down everything in the table that isn't already too far away.
minOff := e.cur + int32(len(e.hist)) - maxMatchOffset
for i := range e.table[:] {
v := e.table[i].offset
if v <= minOff {
v = 0
} else {
v = v - e.cur + maxMatchOffset
}
e.table[i].offset = v
}
for i := range e.bTable[:] {
v := e.bTable[i]
if v.Cur.offset <= minOff {
v.Cur.offset = 0
v.Prev.offset = 0
} else {
v.Cur.offset = v.Cur.offset - e.cur + maxMatchOffset
if v.Prev.offset <= minOff {
v.Prev.offset = 0
} else {
v.Prev.offset = v.Prev.offset - e.cur + maxMatchOffset
}
}
e.bTable[i] = v
}
e.cur = maxMatchOffset
}
s := e.addBlock(src)
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = uint16(len(src))
return
}
// Override src
src = e.hist
nextEmit := s
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int32(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load6432(src, s)
// Repeat MUST be > 1 and within range
repeat := int32(1)
for {
const skipLog = 7
const doEvery = 1
nextS := s
var l int32
var t int32
for {
nextHashS := hashLen(cv, tableBits, hashShortBytes)
nextHashL := hash7(cv, tableBits)
s = nextS
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit {
goto emitRemainder
}
// Fetch a short+long candidate
sCandidate := e.table[nextHashS]
lCandidate := e.bTable[nextHashL]
next := load6432(src, nextS)
entry := tableEntry{offset: s + e.cur}
e.table[nextHashS] = entry
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = entry, eLong.Cur
// Calculate hashes of 'next'
nextHashS = hashLen(next, tableBits, hashShortBytes)
nextHashL = hash7(next, tableBits)
t = lCandidate.Cur.offset - e.cur
if s-t < maxMatchOffset {
if uint32(cv) == load3232(src, t) {
// Long candidate matches at least 4 bytes.
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
// Check the previous long candidate as well.
t2 := lCandidate.Prev.offset - e.cur
if s-t2 < maxMatchOffset && uint32(cv) == load3232(src, t2) {
l = e.matchlen(int(s+4), int(t+4), src) + 4
ml1 := e.matchlen(int(s+4), int(t2+4), src) + 4
if ml1 > l {
t = t2
l = ml1
break
}
}
break
}
// Current value did not match, but check if previous long value does.
t = lCandidate.Prev.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
break
}
}
t = sCandidate.offset - e.cur
if s-t < maxMatchOffset && uint32(cv) == load3232(src, t) {
// Found a 4 match...
l = e.matchlen(int(s+4), int(t+4), src) + 4
// Look up next long candidate (at nextS)
lCandidate = e.bTable[nextHashL]
// Store the next match
e.table[nextHashS] = tableEntry{offset: nextS + e.cur}
eLong := &e.bTable[nextHashL]
eLong.Cur, eLong.Prev = tableEntry{offset: nextS + e.cur}, eLong.Cur
// Check repeat at s + repOff
const repOff = 1
t2 := s - repeat + repOff
if load3232(src, t2) == uint32(cv>>(8*repOff)) {
ml := e.matchlen(int(s+4+repOff), int(t2+4), src) + 4
if ml > l {
t = t2
l = ml
s += repOff
// Not worth checking more.
break
}
}
// If the next long is a candidate, use that...
t2 = lCandidate.Cur.offset - e.cur
if nextS-t2 < maxMatchOffset {
if load3232(src, t2) == uint32(next) {
ml := e.matchlen(int(nextS+4), int(t2+4), src) + 4
if ml > l {
t = t2
s = nextS
l = ml
// This is ok, but check previous as well.
}
}
// If the previous long is a candidate, use that...
t2 = lCandidate.Prev.offset - e.cur
if nextS-t2 < maxMatchOffset && load3232(src, t2) == uint32(next) {
ml := e.matchlen(int(nextS+4), int(t2+4), src) + 4
if ml > l {
t = t2
s = nextS
l = ml
break
}
}
}
break
}
cv = next
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
// Extend the 4-byte match as long as possible.
if l == 0 {
l = e.matchlenLong(int(s+4), int(t+4), src) + 4
} else if l == maxMatchLength {
l += e.matchlenLong(int(s+l), int(t+l), src)
}
// Try to locate a better match by checking the end-of-match...
if sAt := s + l; sAt < sLimit {
// Allow some bytes at the beginning to mismatch.
// Sweet spot is 2/3 bytes depending on input.
// 3 is only a little better when it is but sometimes a lot worse.
// The skipped bytes are tested in Extend backwards,
// and still picked up as part of the match if they do.
const skipBeginning = 2
eLong := &e.bTable[hash7(load6432(src, sAt), tableBits)]
// Test current
t2 := eLong.Cur.offset - e.cur - l + skipBeginning
s2 := s + skipBeginning
off := s2 - t2
if off < maxMatchOffset {
if off > 0 && t2 >= 0 {
if l2 := e.matchlenLong(int(s2), int(t2), src); l2 > l {
t = t2
l = l2
s = s2
}
}
// Test next:
t2 = eLong.Prev.offset - e.cur - l + skipBeginning
off := s2 - t2
if off > 0 && off < maxMatchOffset && t2 >= 0 {
if l2 := e.matchlenLong(int(s2), int(t2), src); l2 > l {
t = t2
l = l2
s = s2
}
}
}
}
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
if false {
if t >= s {
panic(fmt.Sprintln("s-t", s, t))
}
if (s - t) > maxMatchOffset {
panic(fmt.Sprintln("mmo", s-t))
}
if l < baseMatchLength {
panic("bml")
}
}
dst.AddMatchLong(l, uint32(s-t-baseMatchOffset))
repeat = s - t
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
// Index after match end.
for i := nextS + 1; i < int32(len(src))-8; i += 2 {
cv := load6432(src, i)
e.table[hashLen(cv, tableBits, hashShortBytes)] = tableEntry{offset: i + e.cur}
eLong := &e.bTable[hash7(cv, tableBits)]
eLong.Cur, eLong.Prev = tableEntry{offset: i + e.cur}, eLong.Cur
}
goto emitRemainder
}
// Store every long hash in-between and every second short.
if true {
for i := nextS + 1; i < s-1; i += 2 {
cv := load6432(src, i)
t := tableEntry{offset: i + e.cur}
t2 := tableEntry{offset: t.offset + 1}
eLong := &e.bTable[hash7(cv, tableBits)]
eLong2 := &e.bTable[hash7(cv>>8, tableBits)]
e.table[hashLen(cv, tableBits, hashShortBytes)] = t
eLong.Cur, eLong.Prev = t, eLong.Cur
eLong2.Cur, eLong2.Prev = t2, eLong2.Cur
}
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-1 and at s.
cv = load6432(src, s)
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/matchlen_generic.go
Generated Vendored
+34
View File
@@ -0,0 +1,34 @@
// Copyright 2019+ Klaus Post. All rights reserved.
// License information can be found in the LICENSE file.
package flate
import (
"math/bits"
"github.com/klauspost/compress/internal/le"
)
// matchLen returns the maximum common prefix length of a and b.
// a must be the shortest of the two.
func matchLen(a, b []byte) (n int) {
left := len(a)
for left >= 8 {
diff := le.Load64(a, n) ^ le.Load64(b, n)
if diff != 0 {
return n + bits.TrailingZeros64(diff)>>3
}
n += 8
left -= 8
}
a = a[n:]
b = b[n:]
for i := range a {
if a[i] != b[i] {
break
}
n++
}
return n
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/stateless.go
Generated Vendored
+325
View File
@@ -0,0 +1,325 @@
package flate
import (
"io"
"math"
"sync"
"github.com/klauspost/compress/internal/le"
)
const (
maxStatelessBlock = math.MaxInt16
// dictionary will be taken from maxStatelessBlock, so limit it.
maxStatelessDict = 8 << 10
slTableBits = 13
slTableSize = 1 << slTableBits
slTableShift = 32 - slTableBits
)
type statelessWriter struct {
dst io.Writer
closed bool
}
func (s *statelessWriter) Close() error {
if s.closed {
return nil
}
s.closed = true
// Emit EOF block
return StatelessDeflate(s.dst, nil, true, nil)
}
func (s *statelessWriter) Write(p []byte) (n int, err error) {
err = StatelessDeflate(s.dst, p, false, nil)
if err != nil {
return 0, err
}
return len(p), nil
}
func (s *statelessWriter) Reset(w io.Writer) {
s.dst = w
s.closed = false
}
// NewStatelessWriter will do compression but without maintaining any state
// between Write calls.
// There will be no memory kept between Write calls,
// but compression and speed will be suboptimal.
// Because of this, the size of actual Write calls will affect output size.
func NewStatelessWriter(dst io.Writer) io.WriteCloser {
return &statelessWriter{dst: dst}
}
// bitWriterPool contains bit writers that can be reused.
var bitWriterPool = sync.Pool{
New: func() any {
return newHuffmanBitWriter(nil)
},
}
// tokensPool contains tokens struct objects that can be reused
var tokensPool = sync.Pool{
New: func() any {
return &tokens{}
},
}
// StatelessDeflate allows compressing directly to a Writer without retaining state.
// When returning everything will be flushed.
// Up to 8KB of an optional dictionary can be given which is presumed to precede the block.
// Longer dictionaries will be truncated and will still produce valid output.
// Sending nil dictionary is perfectly fine.
func StatelessDeflate(out io.Writer, in []byte, eof bool, dict []byte) error {
bw := bitWriterPool.Get().(*huffmanBitWriter)
bw.reset(out)
defer func() {
// don't keep a reference to our output
bw.reset(nil)
bitWriterPool.Put(bw)
}()
if eof && len(in) == 0 {
// Just write an EOF block.
// Could be faster...
bw.writeStoredHeader(0, true)
bw.flush()
return bw.err
}
// Truncate dict
if len(dict) > maxStatelessDict {
dict = dict[len(dict)-maxStatelessDict:]
}
// For subsequent loops, keep shallow dict reference to avoid alloc+copy.
var inDict []byte
dst := tokensPool.Get().(*tokens)
dst.Reset()
defer func() {
tokensPool.Put(dst)
}()
for len(in) > 0 {
todo := in
if len(inDict) > 0 {
if len(todo) > maxStatelessBlock-maxStatelessDict {
todo = todo[:maxStatelessBlock-maxStatelessDict]
}
} else if len(todo) > maxStatelessBlock-len(dict) {
todo = todo[:maxStatelessBlock-len(dict)]
}
inOrg := in
in = in[len(todo):]
uncompressed := todo
if len(dict) > 0 {
// combine dict and source
bufLen := len(todo) + len(dict)
combined := make([]byte, bufLen)
copy(combined, dict)
copy(combined[len(dict):], todo)
todo = combined
}
// Compress
if len(inDict) == 0 {
statelessEnc(dst, todo, int16(len(dict)))
} else {
statelessEnc(dst, inDict[:maxStatelessDict+len(todo)], maxStatelessDict)
}
isEof := eof && len(in) == 0
if dst.n == 0 {
bw.writeStoredHeader(len(uncompressed), isEof)
if bw.err != nil {
return bw.err
}
bw.writeBytes(uncompressed)
} else if int(dst.n) > len(uncompressed)-len(uncompressed)>>4 {
// If we removed less than 1/16th, huffman compress the block.
bw.writeBlockHuff(isEof, uncompressed, len(in) == 0)
} else {
bw.writeBlockDynamic(dst, isEof, uncompressed, len(in) == 0)
}
if len(in) > 0 {
// Retain a dict if we have more
inDict = inOrg[len(uncompressed)-maxStatelessDict:]
dict = nil
dst.Reset()
}
if bw.err != nil {
return bw.err
}
}
if !eof {
// Align, only a stored block can do that.
bw.writeStoredHeader(0, false)
}
bw.flush()
return bw.err
}
func hashSL(u uint32) uint32 {
return (u * 0x1e35a7bd) >> slTableShift
}
func load3216(b []byte, i int16) uint32 {
return le.Load32(b, i)
}
func load6416(b []byte, i int16) uint64 {
return le.Load64(b, i)
}
func statelessEnc(dst *tokens, src []byte, startAt int16) {
const (
inputMargin = 12 - 1
minNonLiteralBlockSize = 1 + 1 + inputMargin
)
type tableEntry struct {
offset int16
}
var table [slTableSize]tableEntry
// This check isn't in the Snappy implementation, but there, the caller
// instead of the callee handles this case.
if len(src)-int(startAt) < minNonLiteralBlockSize {
// We do not fill the token table.
// This will be picked up by caller.
dst.n = 0
return
}
// Index until startAt
if startAt > 0 {
cv := load3232(src, 0)
for i := range startAt {
table[hashSL(cv)] = tableEntry{offset: i}
cv = (cv >> 8) | (uint32(src[i+4]) << 24)
}
}
s := startAt + 1
nextEmit := startAt
// sLimit is when to stop looking for offset/length copies. The inputMargin
// lets us use a fast path for emitLiteral in the main loop, while we are
// looking for copies.
sLimit := int16(len(src) - inputMargin)
// nextEmit is where in src the next emitLiteral should start from.
cv := load3216(src, s)
for {
const skipLog = 5
const doEvery = 2
nextS := s
var candidate tableEntry
for {
nextHash := hashSL(cv)
candidate = table[nextHash]
nextS = s + doEvery + (s-nextEmit)>>skipLog
if nextS > sLimit || nextS <= 0 {
goto emitRemainder
}
now := load6416(src, nextS)
table[nextHash] = tableEntry{offset: s}
nextHash = hashSL(uint32(now))
if cv == load3216(src, candidate.offset) {
table[nextHash] = tableEntry{offset: nextS}
break
}
// Do one right away...
cv = uint32(now)
s = nextS
nextS++
candidate = table[nextHash]
now >>= 8
table[nextHash] = tableEntry{offset: s}
if cv == load3216(src, candidate.offset) {
table[nextHash] = tableEntry{offset: nextS}
break
}
cv = uint32(now)
s = nextS
}
// A 4-byte match has been found. We'll later see if more than 4 bytes
// match. But, prior to the match, src[nextEmit:s] are unmatched. Emit
// them as literal bytes.
for {
// Invariant: we have a 4-byte match at s, and no need to emit any
// literal bytes prior to s.
// Extend the 4-byte match as long as possible.
t := candidate.offset
l := int16(matchLen(src[s+4:], src[t+4:]) + 4)
// Extend backwards
for t > 0 && s > nextEmit && src[t-1] == src[s-1] {
s--
t--
l++
}
if nextEmit < s {
if false {
emitLiteral(dst, src[nextEmit:s])
} else {
for _, v := range src[nextEmit:s] {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
}
// Save the match found
dst.AddMatchLong(int32(l), uint32(s-t-baseMatchOffset))
s += l
nextEmit = s
if nextS >= s {
s = nextS + 1
}
if s >= sLimit {
goto emitRemainder
}
// We could immediately start working at s now, but to improve
// compression we first update the hash table at s-2 and at s. If
// another emitCopy is not our next move, also calculate nextHash
// at s+1. At least on GOARCH=amd64, these three hash calculations
// are faster as one load64 call (with some shifts) instead of
// three load32 calls.
x := load6416(src, s-2)
o := s - 2
prevHash := hashSL(uint32(x))
table[prevHash] = tableEntry{offset: o}
x >>= 16
currHash := hashSL(uint32(x))
candidate = table[currHash]
table[currHash] = tableEntry{offset: o + 2}
if uint32(x) != load3216(src, candidate.offset) {
cv = uint32(x >> 8)
s++
break
}
}
}
emitRemainder:
if int(nextEmit) < len(src) {
// If nothing was added, don't encode literals.
if dst.n == 0 {
return
}
emitLiteral(dst, src[nextEmit:])
}
}
modules/route-proxy/vendor/github.com/klauspost/compress/flate/token.go
Generated Vendored
+379
View File
@@ -0,0 +1,379 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package flate
import (
"bytes"
"encoding/binary"
"fmt"
"io"
"math"
)
const (
// bits 0-16 xoffset = offset - MIN_OFFSET_SIZE, or literal - 16 bits
// bits 16-22 offsetcode - 5 bits
// bits 22-30 xlength = length - MIN_MATCH_LENGTH - 8 bits
// bits 30-32 type 0 = literal 1=EOF 2=Match 3=Unused - 2 bits
lengthShift = 22
offsetMask = 1<<lengthShift - 1
typeMask = 3 << 30
literalType = 0 << 30
matchType = 1 << 30
matchOffsetOnlyMask = 0xffff
)
// The length code for length X (MIN_MATCH_LENGTH <= X <= MAX_MATCH_LENGTH)
// is lengthCodes[length - MIN_MATCH_LENGTH]
var lengthCodes = [256]uint8{
0, 1, 2, 3, 4, 5, 6, 7, 8, 8,
9, 9, 10, 10, 11, 11, 12, 12, 12, 12,
13, 13, 13, 13, 14, 14, 14, 14, 15, 15,
15, 15, 16, 16, 16, 16, 16, 16, 16, 16,
17, 17, 17, 17, 17, 17, 17, 17, 18, 18,
18, 18, 18, 18, 18, 18, 19, 19, 19, 19,
19, 19, 19, 19, 20, 20, 20, 20, 20, 20,
20, 20, 20, 20, 20, 20, 20, 20, 20, 20,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 22, 22, 22, 22,
22, 22, 22, 22, 22, 22, 22, 22, 22, 22,
22, 22, 23, 23, 23, 23, 23, 23, 23, 23,
23, 23, 23, 23, 23, 23, 23, 23, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 28,
}
// lengthCodes1 is length codes, but starting at 1.
var lengthCodes1 = [256]uint8{
1, 2, 3, 4, 5, 6, 7, 8, 9, 9,
10, 10, 11, 11, 12, 12, 13, 13, 13, 13,
14, 14, 14, 14, 15, 15, 15, 15, 16, 16,
16, 16, 17, 17, 17, 17, 17, 17, 17, 17,
18, 18, 18, 18, 18, 18, 18, 18, 19, 19,
19, 19, 19, 19, 19, 19, 20, 20, 20, 20,
20, 20, 20, 20, 21, 21, 21, 21, 21, 21,
21, 21, 21, 21, 21, 21, 21, 21, 21, 21,
22, 22, 22, 22, 22, 22, 22, 22, 22, 22,
22, 22, 22, 22, 22, 22, 23, 23, 23, 23,
23, 23, 23, 23, 23, 23, 23, 23, 23, 23,
23, 23, 24, 24, 24, 24, 24, 24, 24, 24,
24, 24, 24, 24, 24, 24, 24, 24, 25, 25,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 29,
}
var offsetCodes = [256]uint32{
0, 1, 2, 3, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7,
8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9,
10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10,
11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11,
12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13,
13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14,
15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15,
}
// offsetCodes14 are offsetCodes, but with 14 added.
var offsetCodes14 = [256]uint32{
14, 15, 16, 17, 18, 18, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21,
22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23,
24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,
25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28,
29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29,
29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29,
29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29,
29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29,
}
type token uint32
type tokens struct {
extraHist [32]uint16 // codes 256->maxnumlit
offHist [32]uint16 // offset codes
litHist [256]uint16 // codes 0->255
nFilled int
n uint16 // Must be able to contain maxStoreBlockSize
tokens [maxStoreBlockSize + 1]token
}
func (t *tokens) Reset() {
if t.n == 0 {
return
}
t.n = 0
t.nFilled = 0
for i := range t.litHist[:] {
t.litHist[i] = 0
}
for i := range t.extraHist[:] {
t.extraHist[i] = 0
}
for i := range t.offHist[:] {
t.offHist[i] = 0
}
}
func (t *tokens) Fill() {
if t.n == 0 {
return
}
for i, v := range t.litHist[:] {
if v == 0 {
t.litHist[i] = 1
t.nFilled++
}
}
for i, v := range t.extraHist[:literalCount-256] {
if v == 0 {
t.nFilled++
t.extraHist[i] = 1
}
}
for i, v := range t.offHist[:offsetCodeCount] {
if v == 0 {
t.offHist[i] = 1
}
}
}
func indexTokens(in []token) tokens {
var t tokens
t.indexTokens(in)
return t
}
func (t *tokens) indexTokens(in []token) {
t.Reset()
for _, tok := range in {
if tok < matchType {
t.AddLiteral(tok.literal())
continue
}
t.AddMatch(uint32(tok.length()), tok.offset()&matchOffsetOnlyMask)
}
}
// emitLiteral writes a literal chunk and returns the number of bytes written.
func emitLiteral(dst *tokens, lit []byte) {
for _, v := range lit {
dst.tokens[dst.n] = token(v)
dst.litHist[v]++
dst.n++
}
}
func (t *tokens) AddLiteral(lit byte) {
t.tokens[t.n] = token(lit)
t.litHist[lit]++
t.n++
}
// from https://stackoverflow.com/a/28730362
func mFastLog2(val float32) float32 {
ux := int32(math.Float32bits(val))
log2 := (float32)(((ux >> 23) & 255) - 128)
ux &= -0x7f800001
ux += 127 << 23
uval := math.Float32frombits(uint32(ux))
log2 += ((-0.34484843)*uval+2.02466578)*uval - 0.67487759
return log2
}
// EstimatedBits will return an minimum size estimated by an *optimal*
// compression of the block.
// The size of the block
func (t *tokens) EstimatedBits() int {
shannon := float32(0)
bits := int(0)
nMatches := 0
total := int(t.n) + t.nFilled
if total > 0 {
invTotal := 1.0 / float32(total)
for _, v := range t.litHist[:] {
if v > 0 {
n := float32(v)
shannon += atLeastOne(-mFastLog2(n*invTotal)) * n
}
}
// Just add 15 for EOB
shannon += 15
for i, v := range t.extraHist[1 : literalCount-256] {
if v > 0 {
n := float32(v)
shannon += atLeastOne(-mFastLog2(n*invTotal)) * n
bits += int(lengthExtraBits[i&31]) * int(v)
nMatches += int(v)
}
}
}
if nMatches > 0 {
invTotal := 1.0 / float32(nMatches)
for i, v := range t.offHist[:offsetCodeCount] {
if v > 0 {
n := float32(v)
shannon += atLeastOne(-mFastLog2(n*invTotal)) * n
bits += int(offsetExtraBits[i&31]) * int(v)
}
}
}
return int(shannon) + bits
}
// AddMatch adds a match to the tokens.
// This function is very sensitive to inlining and right on the border.
func (t *tokens) AddMatch(xlength uint32, xoffset uint32) {
if debugDeflate {
if xlength >= maxMatchLength+baseMatchLength {
panic(fmt.Errorf("invalid length: %v", xlength))
}
if xoffset >= maxMatchOffset+baseMatchOffset {
panic(fmt.Errorf("invalid offset: %v", xoffset))
}
}
oCode := offsetCode(xoffset)
xoffset |= oCode << 16
t.extraHist[lengthCodes1[uint8(xlength)]]++
t.offHist[oCode&31]++
t.tokens[t.n] = token(matchType | xlength<<lengthShift | xoffset)
t.n++
}
// AddMatchLong adds a match to the tokens, potentially longer than max match length.
// Length should NOT have the base subtracted, only offset should.
func (t *tokens) AddMatchLong(xlength int32, xoffset uint32) {
if debugDeflate {
if xoffset >= maxMatchOffset+baseMatchOffset {
panic(fmt.Errorf("invalid offset: %v", xoffset))
}
}
oc := offsetCode(xoffset)
xoffset |= oc << 16
for xlength > 0 {
xl := xlength
if xl > 258 {
// We need to have at least baseMatchLength left over for next loop.
if xl > 258+baseMatchLength {
xl = 258
} else {
xl = 258 - baseMatchLength
}
}
xlength -= xl
xl -= baseMatchLength
t.extraHist[lengthCodes1[uint8(xl)]]++
t.offHist[oc&31]++
t.tokens[t.n] = token(matchType | uint32(xl)<<lengthShift | xoffset)
t.n++
}
}
func (t *tokens) AddEOB() {
t.tokens[t.n] = token(endBlockMarker)
t.extraHist[0]++
t.n++
}
func (t *tokens) Slice() []token {
return t.tokens[:t.n]
}
// VarInt returns the tokens as varint encoded bytes.
func (t *tokens) VarInt() []byte {
var b = make([]byte, binary.MaxVarintLen32*int(t.n))
var off int
for _, v := range t.tokens[:t.n] {
off += binary.PutUvarint(b[off:], uint64(v))
}
return b[:off]
}
// FromVarInt restores t to the varint encoded tokens provided.
// Any data in t is removed.
func (t *tokens) FromVarInt(b []byte) error {
var buf = bytes.NewReader(b)
var toks []token
for {
r, err := binary.ReadUvarint(buf)
if err == io.EOF {
break
}
if err != nil {
return err
}
toks = append(toks, token(r))
}
t.indexTokens(toks)
return nil
}
// Returns the type of a token
func (t token) typ() uint32 { return uint32(t) & typeMask }
// Returns the literal of a literal token
func (t token) literal() uint8 { return uint8(t) }
// Returns the extra offset of a match token
func (t token) offset() uint32 { return uint32(t) & offsetMask }
func (t token) length() uint8 { return uint8(t >> lengthShift) }
// Convert length to code.
func lengthCode(len uint8) uint8 { return lengthCodes[len] }
// Returns the offset code corresponding to a specific offset
func offsetCode(off uint32) uint32 {
if false {
if off < uint32(len(offsetCodes)) {
return offsetCodes[off&255]
} else if off>>7 < uint32(len(offsetCodes)) {
return offsetCodes[(off>>7)&255] + 14
} else {
return offsetCodes[(off>>14)&255] + 28
}
}
if off < uint32(len(offsetCodes)) {
return offsetCodes[uint8(off)]
}
return offsetCodes14[uint8(off>>7)]
}
modules/route-proxy/vendor/github.com/klauspost/compress/internal/le/le.go
Generated Vendored
+5
View File
@@ -0,0 +1,5 @@
package le
type Indexer interface {
int | int8 | int16 | int32 | int64 | uint | uint8 | uint16 | uint32 | uint64
}
modules/route-proxy/vendor/github.com/klauspost/compress/internal/le/unsafe_disabled.go
Generated Vendored
+42
View File
@@ -0,0 +1,42 @@
//go:build !(amd64 || arm64 || ppc64le || riscv64) || nounsafe || purego || appengine
package le
import (
"encoding/binary"
)
// Load8 will load from b at index i.
func Load8[I Indexer](b []byte, i I) byte {
return b[i]
}
// Load16 will load from b at index i.
func Load16[I Indexer](b []byte, i I) uint16 {
return binary.LittleEndian.Uint16(b[i:])
}
// Load32 will load from b at index i.
func Load32[I Indexer](b []byte, i I) uint32 {
return binary.LittleEndian.Uint32(b[i:])
}
// Load64 will load from b at index i.
func Load64[I Indexer](b []byte, i I) uint64 {
return binary.LittleEndian.Uint64(b[i:])
}
// Store16 will store v at b.
func Store16(b []byte, v uint16) {
binary.LittleEndian.PutUint16(b, v)
}
// Store32 will store v at b.
func Store32(b []byte, v uint32) {
binary.LittleEndian.PutUint32(b, v)
}
// Store64 will store v at b.
func Store64[I Indexer](b []byte, i I, v uint64) {
binary.LittleEndian.PutUint64(b[i:], v)
}
modules/route-proxy/vendor/github.com/klauspost/compress/internal/le/unsafe_enabled.go
Generated Vendored
+52
View File
@@ -0,0 +1,52 @@
// We enable 64 bit LE platforms:
//go:build (amd64 || arm64 || ppc64le || riscv64) && !nounsafe && !purego && !appengine
package le
import (
"unsafe"
)
// Load8 will load from b at index i.
func Load8[I Indexer](b []byte, i I) byte {
//return binary.LittleEndian.Uint16(b[i:])
//return *(*uint16)(unsafe.Pointer(&b[i]))
return *(*byte)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(b)), i))
}
// Load16 will load from b at index i.
func Load16[I Indexer](b []byte, i I) uint16 {
//return binary.LittleEndian.Uint16(b[i:])
//return *(*uint16)(unsafe.Pointer(&b[i]))
return *(*uint16)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(b)), i))
}
// Load32 will load from b at index i.
func Load32[I Indexer](b []byte, i I) uint32 {
//return binary.LittleEndian.Uint32(b[i:])
//return *(*uint32)(unsafe.Pointer(&b[i]))
return *(*uint32)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(b)), i))
}
// Load64 will load from b at index i.
func Load64[I Indexer](b []byte, i I) uint64 {
//return binary.LittleEndian.Uint64(b[i:])
//return *(*uint64)(unsafe.Pointer(&b[i]))
return *(*uint64)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(b)), i))
}
// Store16 will store v at b.
func Store16(b []byte, v uint16) {
*(*uint16)(unsafe.Pointer(unsafe.SliceData(b))) = v
}
// Store32 will store v at b.
func Store32(b []byte, v uint32) {
*(*uint32)(unsafe.Pointer(unsafe.SliceData(b))) = v
}
// Store64 will store v at b[i:].
func Store64[I Indexer](b []byte, i I, v uint64) {
*(*uint64)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(b)), i)) = v
}
modules/route-proxy/vendor/github.com/klauspost/compress/internal/regmask/regmask.go
Generated Vendored
+50
View File
@@ -0,0 +1,50 @@
//go:build !386
// Package regmask provides masks for variable shift counts.
//
// Go defines a shift by more than the width of the shifted value as producing
// zero, so the compiler has to guard every variable shift against that: on
// amd64 with a CMP and an SBB, on arm64 with a CMP and a CSEL, and so on. When
// the count is known to be smaller than the width anyway, masking it with the
// width minus one lets the compiler drop the guard. The mask itself costs
// nothing on the architectures whose shift instructions already use only the
// low bits of the count, which the compiler knows about.
//
// Each constant is named for the width of the value being shifted and the type
// of the shift count, and may only be used where the count is always smaller
// than that width:
//
// x := y >> (n & regmask.Shift64ByUint) // y is 64 bits, n is a uint below 64
package regmask
const (
// ShiftNByUint8 - shifting an N bit value by a uint8
Shift8ByUint8 = 7
Shift16ByUint8 = 15
Shift32ByUint8 = 31
Shift64ByUint8 = 63
// ShiftNByUint16 - shifting an N bit value by a uint16
Shift8ByUint16 = Shift8ByUint8
Shift16ByUint16 = Shift16ByUint8
Shift32ByUint16 = Shift32ByUint8
Shift64ByUint16 = Shift64ByUint8
// ShiftNByUint32 - shifting an N bit value by a uint32
Shift8ByUint32 = Shift8ByUint8
Shift16ByUint32 = Shift16ByUint8
Shift32ByUint32 = Shift32ByUint8
Shift64ByUint32 = Shift64ByUint8
// ShiftNByUint64 - shifting an N bit value by a uint64
Shift8ByUint64 = Shift8ByUint8
Shift16ByUint64 = Shift16ByUint8
Shift32ByUint64 = Shift32ByUint8
Shift64ByUint64 = Shift64ByUint8
// ShiftNByUint - shifting an N bit value by a uint
Shift8ByUint = Shift8ByUint8
Shift16ByUint = Shift16ByUint8
Shift32ByUint = Shift32ByUint8
Shift64ByUint = Shift64ByUint8
)
modules/route-proxy/vendor/github.com/klauspost/compress/internal/regmask/regmask_386.go
Generated Vendored
+39
View File
@@ -0,0 +1,39 @@
//go:build 386
package regmask
// A 64 bit shift on 386 is synthesized from 32 bit instructions, and masking its
// count makes the compiler emit more code rather than less, so those masks are
// no-ops here: each is the largest value its count type can hold, which the
// compiler folds away. Shifts of narrower values behave as elsewhere.
const (
// ShiftNByUint8 - shifting an N bit value by a uint8
Shift8ByUint8 = 7
Shift16ByUint8 = 15
Shift32ByUint8 = 31
Shift64ByUint8 = 0xff
// ShiftNByUint16 - shifting an N bit value by a uint16
Shift8ByUint16 = Shift8ByUint8
Shift16ByUint16 = Shift16ByUint8
Shift32ByUint16 = Shift32ByUint8
Shift64ByUint16 = 0xffff
// ShiftNByUint32 - shifting an N bit value by a uint32
Shift8ByUint32 = Shift8ByUint8
Shift16ByUint32 = Shift16ByUint8
Shift32ByUint32 = Shift32ByUint8
Shift64ByUint32 = 0xffffffff
// ShiftNByUint64 - shifting an N bit value by a uint64
Shift8ByUint64 = Shift8ByUint8
Shift16ByUint64 = Shift16ByUint8
Shift32ByUint64 = Shift32ByUint8
Shift64ByUint64 = 0xffffffffffffffff
// ShiftNByUint - shifting an N bit value by a uint
Shift8ByUint = Shift8ByUint8
Shift16ByUint = Shift16ByUint8
Shift32ByUint = Shift32ByUint8
Shift64ByUint = ^uint(0)
)
modules/route-proxy/vendor/github.com/nats-io/nats.go/.gitignore
Generated
+55
View File
@@ -0,0 +1,55 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
# Git backup files
*.orig
# Emacs
*~
\#*\#
.\#*
# vi/vim
.??*.swp
# Mac
.DS_Store
# Eclipse
.project
.settings/
# bin
# Goland
.idea
# VS Code
.vscode
# Claude Code
.claude/
CLAUDE.local.md
# Git worktrees
.worktrees/
modules/route-proxy/vendor/github.com/nats-io/nats.go/.golangci.yaml
Generated Vendored
+43
View File
@@ -0,0 +1,43 @@
version: "2"
linters:
settings:
staticcheck:
# The merged staticcheck linter pulls in more check families than this
# repo has ever run; keep it to the set that was in force before.
checks:
- SA*
- S1*
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
rules:
- linters:
- errcheck
text: Unsubscribe
- linters:
- errcheck
text: Drain
- linters:
- errcheck
text: msg.Ack
- linters:
- errcheck
text: watcher.Stop
paths:
- third_party$
- builtin$
- examples$
issues:
max-issues-per-linter: 0
max-same-issues: 0
formatters:
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
modules/route-proxy/vendor/github.com/nats-io/nats.go/.words
Generated Vendored
+106
View File
@@ -0,0 +1,106 @@
1
derek
dlc
ivan
acknowledgement/SM
arity
deduplication/S
demarshal/SDG
durables
iff
observable/S
redelivery/S
retransmitting
retry/SB
SlowConsumer
AppendInt
ReadMIMEHeader
clientProtoZero
jetstream
v1
v2
ack/SGD
auth
authToken
chans
creds
config/S
cseq
impl
msgh
msgId
mux/S
nack
ptr
puback
scanf
stderr
stdout
structs
tm
todo
unsub/S
permessage
permessage-deflate
urlA
urlB
websocket
ws
wss
NKey
pList
backend/S
backoff/S
decompressor/CGS
inflight
inlined
lookups
reconnection/MS
redeliver/ADGS
responder/S
rewrap/S
rollup/S
unreceive/DRSZGB
variadic
wakeup/S
whitespace
wrap/AS
omitempty
apache
html
ietf
www
sum256
32bit/S
64bit/S
64k
128k
512k
hacky
handroll/D
rfc6455
rfc7692
0x00
0xff
20x
40x
50x
ErrXXX
atlanta
eu
modules/route-proxy/vendor/github.com/nats-io/nats.go/.words.readme
Generated Vendored
+25
View File
@@ -0,0 +1,25 @@
The .words file is used by gospel (v1.2+), which wraps the Hunspell libraries
but populates the dictionary with identifiers from the Go source.
<https://github.com/kortschak/gospel>
Alas, no comments are allowed in the .words file and newer versions of gospel
error out on seeing them. This is really a hunspell restriction.
We assume en_US hunspell dictionaries are installed and used.
The /AFFIXRULES are defined in en_US.aff (eg: /usr/share/hunspell/en_US.aff)
Invoke `hunspell -D` to see the actual locations.
Words which are in the base dictionary can't have extra affix rules added to
them, so we have to start with the affixed variant we want to add.
Thus `creds` rather than `cred/S` and so on.
So we can't use receive/DRSZGBU, adding 'U', to allow unreceive and variants,
we have to use unreceive as the stem.
We can't define our own affix or compound rules,
to capture rfc\d{3,} or 0x[0-9A-Fa-f]{2}
The spelling tokenizer doesn't take "permessage-deflate" as allowing for ...
"permessage-deflate", which is an RFC7692 registered extension for websockets.
We have to explicitly list "permessage".
modules/route-proxy/vendor/github.com/nats-io/nats.go/CLAUDE.md
Generated Vendored
+198
View File
@@ -0,0 +1,198 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Official Go client library for the NATS messaging system. Provides core pub/sub, request/reply, JetStream (streams, consumers, KV, object store), and a micro services framework. Module path: `github.com/nats-io/nats.go`.
## Build and Test Commands
This project uses a **dual module** setup: `go.mod` for production (minimal deps) and `go_test.mod` for testing (protobuf encoder + jwt + nkeys + nuid + the ntf tester). Always use `-modfile=go_test.mod` when running tests.
`go_test.mod` declares `go 1.26.0` while `go.mod` stays at `1.25.0`, because `github.com/synadia-io/orbit.go/ntf` requires 1.26 and that raises the floor for the whole test module. `go get`/`go mod tidy` will rewrite this line; leave it at 1.26.0. TESTING.md has the full derivation.
Integration tests (everything in `./test/`, `./jetstream/test/`, `./micro/test/`) run against real servers spawned by a **tester** service, driven through `github.com/synadia-io/orbit.go/ntf-client`. There are two ways to run it:
- **In-process, no docker (the default)** — a `TestMain` in each of the three test packages starts `github.com/synadia-io/orbit.go/ntf` inside the test binary. No build tag; this is what a plain `go test` does.
- **Docker** (`synadia/ntf-server`, tag pinned in `.github/workflows/ci.yaml`) — located via `TESTER_NATS_URL`. This is what CI uses.
`TESTER_NATS_URL` takes precedence: when set, it is used instead of starting an in-process tester. The two are **not** necessarily equivalent — in-process links whatever nats-server `orbit.go/ntf` depends on, the container ships its own — so `TestMain` hard-fails when `GITHUB_ACTIONS` is set without `TESTER_NATS_URL`, rather than silently changing what CI tests against. (`GITHUB_ACTIONS`, not `CI`: devcontainers and other runners set that one too.)
```bash
# Default local workflow: no docker, nothing to start or tear down.
make test # full race-enabled suite
make test T=TestName PKG=./test/... # single test, verbose
make test-norace # NoRace suite
# Equivalent raw command:
go test -modfile=go_test.mod -tags=internal_testing -race -p=1 ./... --failfast -vet=off
# Docker workflow (host-side mode publishes the tester's ports so `go test`
# from your terminal can reach the spawned NATS servers via localhost).
# Use the test-docker targets: plain `make test` ignores the container.
make tester-up-host
make test-docker T=TestName PKG=./test/...
# Run all tests against the tester (single command, covers both white-box
# tests at the repo root and integration tests in ./test/, ./jetstream/test/,
# ./micro/test/).
TESTER_NATS_URL=nats://localhost:4222 \
go test -modfile=go_test.mod -race -v -p=1 ./... --failfast -vet=off -tags=internal_testing
# Run NoRace tests (must be run separately, without -race flag)
TESTER_NATS_URL=nats://localhost:4222 \
go test -modfile=go_test.mod -v -run=TestNoRace -p=1 ./... --failfast -vet=off
# Run a specific test
TESTER_NATS_URL=nats://localhost:4222 \
go test -modfile=go_test.mod -race -tags=internal_testing -run TestName ./...
# Run tests for a specific package
TESTER_NATS_URL=nats://localhost:4222 go test -modfile=go_test.mod -race ./jetstream/test/... --failfast
TESTER_NATS_URL=nats://localhost:4222 go test -modfile=go_test.mod -race ./micro/test/... --failfast
# Makefile wrappers for the docker path (TESTER_NATS_URL defaults to nats://localhost:4222)
make test-docker # full race-enabled suite with internal_testing tag
make test-docker T=TestName PKG=./test/... # single test, verbose
# Test against a different nats-server (branch/tag/commit/local checkout).
# Adds a replace directive to go_test.mod — never commit it. Do NOT use
# `go get ...@main`, which can drop ntf from the module graph (why: TESTING.md).
make server-replace V=main
make server-replace-drop
# Stop the tester
make tester-down
# Alternative: run the full suite inside an alpine sibling container (matches CI).
# Doesn't need TESTER_NATS_URL or tester-up-host — the Makefile target handles it.
make test-tester
# Build
go build ./...
# Formatting
go fmt -modfile=go_test.mod ./...
# Vet
go vet -modfile=go_test.mod ./...
# Static analysis (as CI does it; staticcheck has no -modfile flag, so it goes
# through GOFLAGS)
GOFLAGS="-mod=mod -modfile=go_test.mod" staticcheck ./...
# Linting (golangci-lint runs only on jetstream/; the modfile GOFLAGS lets it
# typecheck jetstream/test, whose deps live in go_test.mod only)
GOFLAGS="-mod=mod -modfile=go_test.mod" golangci-lint run --timeout 5m0s ./jetstream/...
# Spell check
find . -type f -name "*.go" | xargs misspell -error -locale US
# Update test dependencies (never change go.mod for test deps)
go mod tidy -modfile=go_test.mod
```
A plain `go test -modfile=go_test.mod ./...` now runs the integration suites too, since `TestMain` starts a tester on its own. It takes minutes rather than seconds — that is expected, not a hang.
## Important Build Tags
- **`internal_testing`** -- Exposes internal test helpers (e.g., `AddMsgFilter`, `CloseTCPConn`) from `testing_internal.go`. Required for some tests in `./test/`.
- **`!race && !skip_no_race_tests`** -- NoRace tests in `test/norace_test.go` only run when the race detector is OFF.
- **`compat`** -- Compatibility tests in `test/compat_test.go` (connect to an external NATS server via `NATS_URL`).
- **`go1.23`** -- Iterator-based tests in `test/nats_iter_test.go` and `nats_iter.go`.
## CI Pipeline (ci.yaml)
1. **lint** -- `go fmt`, `go vet`, `staticcheck`, `misspell` (all packages), `golangci-lint` (jetstream only).
2. **test** -- Matrix of Go 1.26 and 1.27; 1.26 is the floor because `go_test.mod` declares it (a lower row would silently fetch 1.26 via `GOTOOLCHAIN=auto`). The 1.27 row runs coverage. Runs inside an `alpine` container on the same docker network as the `synadia/ntf-server` service, which is started with `command: serve --advertise nats` (the integration tests dial the spawned NATS servers by service name). CI sets `TESTER_NATS_URL` at job level so it uses the docker tester rather than the in-process default; `TestMain` fails the run if that variable ever goes missing under `GITHUB_ACTIONS`. Two steps: NoRace tests (without `-race`), then full race-enabled tests with `-tags=internal_testing` (`scripts/cov.sh` runs for coverage instead of the plain race run).
## Project Structure
```
nats.go # Core connection, pub/sub, request/reply (~6500 lines)
parser.go # Client-side protocol parser
ws.go # WebSocket transport support
js.go # Legacy JetStream API (deprecated, see jetstream/)
jsm.go # Legacy JetStream management
kv.go # Legacy KeyValue API
object.go # Legacy Object Store API
enc.go # EncodedConn (deprecated)
netchan.go # Go channel bindings
timer.go # Internal timer utilities
context.go # Context-aware request methods
nats_iter.go # Go 1.23+ iterator support (go:build go1.23)
testing_internal.go # Internal test hooks (go:build internal_testing)
jetstream/ # New JetStream API (preferred over legacy)
jetstream.go # Top-level JetStream interface
stream.go # Stream management
stream_config.go # Stream configuration types
consumer.go # Consumer management
consumer_config.go # Consumer configuration types
pull.go # Pull consumer implementation
push.go # Push consumer (deprecated)
ordered.go # Ordered consumer
publish.go # JetStream publish methods
kv.go # KeyValue store
object.go # Object store
message.go # JetStream message types
errors.go # JetStream error types
test/ # Integration tests (package test, uses testservice)
micro/ # Micro services framework
service.go # Service interface and implementation
request.go # Request handling
test/ # Integration tests
internal/
parser/ # NATS protocol parser (used by core client)
syncx/ # Concurrent map utility
encoders/
builtin/ # Default encoders (JSON, GOB, string)
protobuf/ # Protocol Buffers encoder
test/ # Integration tests for core package (package test)
testservice_helper_test.go # withServer / withServerInstance / newTester / dialInstance helpers
main_test.go # TestMain: starts the tester (in-process by default)
helper_test.go # Shared utility helpers (Wait, checkFor, getStableNumGoroutine, ...)
norace_test.go # Tests that cannot run with -race (build tag guarded)
js_internal_test.go # Tests requiring internal_testing tag
configs/certs/ # CA/server/key PEMs loaded by TLS tests (the *.conf files are unused)
bench/ # Benchmarking utilities
examples/ # Example command-line tools (nats-pub, nats-sub, etc.)
scripts/cov.sh # Coverage collection script (run by the CI coverage matrix row)
```
The tester client is an external test-only dependency: `github.com/synadia-io/orbit.go/ntf-client`
(package `ntf`, imported under the alias `testservice`). It lives in `go_test.mod` only.
## Test Architecture
- **Root `nats_test.go`** (package `nats`) -- White-box unit tests with access to unexported internals.
- **`test/`** (package `test`) -- Black-box integration tests. Tests bring up a NATS server via the testservice helpers (`withServer`, `withJSServer`, `withJSCluster`, ...) which talk to the tester over NATS — either the in-process one started by `TestMain` (the default) or the `synadia/ntf-server` docker service via `TESTER_NATS_URL`. Use the `testerURL` package var; never read `TESTER_NATS_URL` directly, or the test will bypass the in-process tester (this exact bug hit `headers_test.go`).
- **`jetstream/test/`** (package `test`) -- Integration tests for the new JetStream API, same testservice harness.
- **`micro/test/`** (package `micro_test`) -- Integration tests for the micro services framework, same testservice harness.
- **NoRace tests** -- Prefixed `TestNoRace*`, guarded by `//go:build !race && !skip_no_race_tests`. Must be run separately without `-race`.
- Tests always run with `-p=1` (no parallel packages) because the tester serializes some bookkeeping that doesn't tolerate concurrent CreateServer calls from independent test binaries.
## Code Conventions
- **License header** -- Every `.go` file starts with the Apache 2.0 license header (Copyright year range).
- **Error variables** -- Exported errors defined as `var Err... = errors.New("nats: ...")` in `nats.go`. JetStream errors in `jetstream/errors.go` follow the same pattern.
- **Options pattern** -- Connection options use functional options: `nats.Connect(url, nats.Name("myapp"), nats.MaxReconnects(5))`. JetStream and micro use similar patterns.
- **No external dependencies in production** -- Only `klauspost/compress`, `nkeys`, `nuid` in `go.mod`. Test deps (protobuf, jwt, etc.) are isolated in `go_test.mod`. PRs adding dependencies are scrutinized heavily.
- **Commits require sign-off** -- Use `git commit -s` (DCO: `Signed-off-by`).
- **US English spelling** -- Enforced by `misspell -locale US` in CI.
- **Interface-driven design** -- JetStream and micro packages define interfaces (`JetStream`, `Stream`, `Consumer`, `Service`) with concrete unexported implementations.
## Key Types
- `nats.Conn` -- Core connection, handles all NATS protocol operations.
- `nats.Msg` -- Message type for pub/sub and request/reply.
- `nats.Subscription` -- Represents a subscription (sync, async, or channel-based).
- `jetstream.JetStream` -- Entry point for new JetStream API (created via `jetstream.New(nc)`).
- `jetstream.Stream`, `jetstream.Consumer` -- Stream and consumer management.
- `micro.Service` -- Micro service instance (created via `micro.AddService(nc, config)`).
modules/route-proxy/vendor/github.com/nats-io/nats.go/CODE-OF-CONDUCT.md
Generated Vendored
+3
View File
@@ -0,0 +1,3 @@
## Community Code of Conduct
NATS follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
modules/route-proxy/vendor/github.com/nats-io/nats.go/CONTRIBUTING.md
Generated Vendored
+86
View File
@@ -0,0 +1,86 @@
# Contributing
Thanks for your interest in contributing! This document contains `nats-io/nats.go` specific contributing details. If you
are a first-time contributor, please refer to the general [NATS Contributor Guide](https://nats.io/contributing/) to get
a comprehensive overview of contributing to the NATS project.
## Getting started
There are three general ways you can contribute to this repo:
- Proposing an enhancement or new feature
- Reporting a bug or regression
- Contributing changes to the source code
For the first two, refer to the [GitHub Issues](https://github.com/nats-io/nats.go/issues/new/choose) which guides you
through the available options along with the needed information to collect.
## Contributing changes
_Prior to opening a pull request, it is recommended to open an issue first to ensure the maintainers can review intended
changes. Exceptions to this rule include fixing non-functional source such as code comments, documentation or other
supporting files._
Proposing source code changes is done through GitHub's standard pull request workflow.
If your branch is a work-in-progress then please start by creating your pull requests as draft, by clicking the
down-arrow next to the `Create pull request` button and instead selecting `Create draft pull request`.
This will defer the automatic process of requesting a review from the NATS team and significantly reduces noise until
you are ready. Once you are happy, you can click the `Ready for review` button.
### Guidelines
A good pull request includes:
- A high-level description of the changes, including links to any issues that are related by adding comments
like `Resolves #NNN` to your description.
See [Linking a Pull Request to an Issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue)
for more information.
- An up-to-date parent commit. Please make sure you are pulling in the latest `main` branch and rebasing your work on
top of it, i.e. `git rebase main`.
- Unit tests where appropriate. Bug fixes will benefit from the addition of regression tests. New features will not be
accepted without suitable test coverage!
- No more commits than necessary. Sometimes having multiple commits is useful for telling a story or isolating changes
from one another, but please squash down any unnecessary commits that may just be for clean-up, comments or small
changes.
- No additional external dependencies that aren't absolutely essential. Please do everything you can to avoid pulling in
additional libraries/dependencies into `go.mod` as we will be very critical of these.
### Sign-off
In order to accept a contribution, you will first need to certify that the contribution is your original work and that
you license the work to the project under
the [Apache-2.0 license](https://github.com/nats-io/nats.go/blob/main/LICENSE).
This is done by using `Signed-off-by` statements, which should appear in **both** your commit messages and your PR
description. Please note that we can only accept sign-offs under a legal name. Nicknames and aliases are not permitted.
To perform a sign-off with `git`, use `git commit -s` (or `--signoff`).
## Get help
If you have questions about the contribution process, please start
a [GitHub discussion](https://github.com/nats-io/nats.go/discussions), join the [NATS Slack](https://slack.nats.io/), or
send your question to the [NATS Google Group](https://groups.google.com/forum/#!forum/natsio).
## Testing
The integration tests run against real NATS servers spawned by a tester service. Locally it runs inside the test binary
(`github.com/synadia-io/orbit.go/ntf`), so no docker is needed; CI uses the
[synadia/ntf-server](https://hub.docker.com/r/synadia/ntf-server) docker image instead. See [TESTING.md](TESTING.md) for
the full workflow; the short version:
```shell
make test # full race-enabled suite, no docker
make test T=TestName PKG=./test/... # iterate on a single test
```
Nothing to install or start.
You should use `go_test.mod` to manage your testing dependencies. Please use the following command to update your
dependencies and avoid changing the main `go.mod` in a PR:
```shell
go mod tidy -modfile=go_test.mod
```
modules/route-proxy/vendor/github.com/nats-io/nats.go/GOVERNANCE.md
Generated Vendored
+3
View File
@@ -0,0 +1,3 @@
# NATS Go Client Governance
NATS Go Client (go-nats) is part of the NATS project and is subject to the [NATS Governance](https://github.com/nats-io/nats-general/blob/master/GOVERNANCE.md).
modules/route-proxy/vendor/github.com/nats-io/nats.go/LICENSE
Generated Vendored
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
modules/route-proxy/vendor/github.com/nats-io/nats.go/MAINTAINERS.md
Generated Vendored
+8
View File
@@ -0,0 +1,8 @@
# Maintainers
Maintainership is on a per project basis.
### Maintainers
- Derek Collison <derek@nats.io> [@derekcollison](https://github.com/derekcollison)
- Ivan Kozlovic <ivan@nats.io> [@kozlovic](https://github.com/kozlovic)
- Waldemar Quevedo <wally@nats.io> [@wallyqs](https://github.com/wallyqs)
modules/route-proxy/vendor/github.com/nats-io/nats.go/Makefile
Generated Vendored
+121
View File
@@ -0,0 +1,121 @@
# Local convenience targets. The default path needs no docker: a TestMain runs
# the ntf tester inside each test binary.
#
# make test # full suite
# make test T=TestName PKG=./test/... # one test
#
# The test-docker targets run against an external tester instead. Docker modes
# and the server-replace workflow: see TESTING.md.
# Tester image is pinned in ci.yaml (single source of truth); parsed from there.
TESTER_IMAGE ?= $(shell sed -n 's|^ *image: *\(synadia/ntf-server:[^ ]*\).*|\1|p' .github/workflows/ci.yaml)
ifeq ($(strip $(TESTER_IMAGE)),)
$(error could not parse the tester image from .github/workflows/ci.yaml)
endif
TESTER_NAME ?= nats-tester
TESTER_NETWORK ?= nats-tester-net
GO_IMAGE ?= golang:1.26-alpine
# T limits the run to a single test (-run, verbose); PKG limits the packages.
TESTER_NATS_URL ?= nats://localhost:4222
PKG ?= ./...
.PHONY: tester-net tester-up tester-up-host tester-down test-tester test test-norace test-docker server-replace server-replace-drop
# Points the in-process tester at another nats-server via a replace directive in
# go_test.mod. V is a version/branch/commit, or a path when it starts with / or .
# Never commit the directive. Prefer this over `go get`, which can drop ntf from
# the module graph entirely — see TESTING.md.
# make server-replace V=main | v2.13.0 | /path/to/nats-server
server-replace:
@test -n "$(V)" || { echo "usage: make server-replace V=main|v2.13.0|/path/to/nats-server"; exit 1; }
go mod edit -modfile=go_test.mod -replace=github.com/nats-io/nats-server/v2=$(if $(filter /% .%,$(V)),$(V),github.com/nats-io/nats-server/v2@$(V))
GOFLAGS=-mod=mod go mod tidy -modfile=go_test.mod
@echo "now building against:"
@go list -m -modfile=go_test.mod github.com/nats-io/nats-server/v2
# Removes the directive. Does not fully restore go_test.mod; see the warning it
# prints below.
server-replace-drop:
go mod edit -modfile=go_test.mod -dropreplace=github.com/nats-io/nats-server/v2
GOFLAGS=-mod=mod go mod tidy -modfile=go_test.mod
@echo "now building against:"
@go list -m -modfile=go_test.mod github.com/nats-io/nats-server/v2
@git diff --quiet -- go_test.mod go_test.sum || { \
echo ""; \
echo "WARNING: go_test.mod/go_test.sum still differ from HEAD — the replaced"; \
echo "module's dependencies stayed upgraded. Do not commit these. For an exact"; \
echo "restore (discards other local edits to both files):"; \
echo " git checkout -- go_test.mod go_test.sum"; \
}
# -count=1 everywhere: results depend on external state (which tester answers)
# that Go's cache key misses, so a hit can replay an in-process result for a
# test-docker run. -p=1: the shared container tester does not tolerate
# concurrent CreateServer calls from independent test binaries.
test:
TESTER_NATS_URL= go test -modfile=go_test.mod -tags=internal_testing -race -p=1 -count=1 $(if $(T),-v -run '$(T)') $(PKG) --failfast -vet=off
# test-docker is test against the container started by `make tester-up-host`.
test-docker:
TESTER_NATS_URL=$(TESTER_NATS_URL) go test -modfile=go_test.mod -tags=internal_testing -race -p=1 -count=1 $(if $(T),-v -run '$(T)') $(PKG) --failfast -vet=off
# TestNoRace* tests must run with the race detector off.
test-norace:
TESTER_NATS_URL= go test -modfile=go_test.mod -p=1 -count=1 $(if $(T),-v) -run '$(or $(T),TestNoRace)' $(PKG) --failfast -vet=off
tester-net:
@docker network inspect $(TESTER_NETWORK) >/dev/null 2>&1 || \
docker network create $(TESTER_NETWORK)
# No host port publishing, for sibling-container mode (make test-tester).
# Publishing here would break server bring-up: docker-proxy holds 0.0.0.0:<port>
# inside the container's netns, racing the tester's port-reservation handover
# ("bind: address already in use"). The ephemeral range is left at the kernel
# default so TIME_WAIT churn cannot crowd it; narrowing it only matters when
# ports must match a published range (see tester-up-host).
tester-up: tester-net
docker run -d \
--name $(TESTER_NAME) \
--network $(TESTER_NETWORK) \
--restart unless-stopped \
-e NATS_ADVERTISE=$(TESTER_NAME) \
$(TESTER_IMAGE) serve
@echo "Tester up as '$(TESTER_NAME)'. Run the suite: make test-tester"
# Publishes ports so `go test` on the host reaches the spawned servers;
# ip_local_port_range is narrowed to match the published range. On macOS
# docker-proxy races the port handover, failing ~5-10% of server creations in
# heavy suites — rerun, or use sibling-container mode, which is immune.
tester-up-host: tester-net
docker run -d \
--name $(TESTER_NAME) \
--network $(TESTER_NETWORK) \
--restart unless-stopped \
--sysctl net.ipv4.ip_local_port_range="30000 31000" \
-p 4222:4222 \
-p 30000-31000:30000-31000 \
-e NATS_ADVERTISE=localhost \
$(TESTER_IMAGE) serve
@echo "Tester up as '$(TESTER_NAME)'. Run: make test-docker [T=...] [PKG=...]"
@echo "(plain 'make test' ignores this container and uses the in-process tester)"
# Removes the container, discarding its logs. To keep it for debugging instead:
# docker logs -f $(TESTER_NAME) / docker restart $(TESTER_NAME).
tester-down:
-docker rm -f $(TESTER_NAME)
-docker network rm $(TESTER_NETWORK)
# Same two passes as the CI `test` job: NoRace first, then the full race suite.
test-tester: tester-net
docker run --rm \
--network $(TESTER_NETWORK) \
-v $(CURDIR):/src \
-w /src \
-e TESTER_NATS_URL=nats://$(TESTER_NAME):4222 \
-e CGO_ENABLED=1 \
$(GO_IMAGE) sh -c '\
apk add --no-cache gcc libc-dev git make >/dev/null && \
go test -modfile=go_test.mod -v -run=TestNoRace -p=1 -count=1 ./... --failfast -vet=off && \
go test -modfile=go_test.mod -tags=internal_testing -race -v -p=1 -count=1 ./... --failfast -vet=off'
modules/route-proxy/vendor/github.com/nats-io/nats.go/README.md
Generated Vendored
+397
View File
@@ -0,0 +1,397 @@
# NATS - Go Client
A [Go](http://golang.org) client for the [NATS messaging system](https://nats.io).
[![License Apache 2][License-Image]][License-Url] [![Go Report Card][ReportCard-Image]][ReportCard-Url] [![Build Status][Build-Status-Image]][Build-Status-Url] [![GoDoc][GoDoc-Image]][GoDoc-Url] [![Coverage Status][Coverage-image]][Coverage-Url]
[License-Url]: https://www.apache.org/licenses/LICENSE-2.0
[License-Image]: https://img.shields.io/badge/License-Apache2-blue.svg
[ReportCard-Url]: https://goreportcard.com/report/github.com/nats-io/nats.go
[ReportCard-Image]: https://goreportcard.com/badge/github.com/nats-io/nats.go
[Build-Status-Url]: https://github.com/nats-io/nats.go/actions/workflows/ci.yaml?query=event%3Arelease
[Build-Status-Image]: https://github.com/nats-io/nats.go/actions/workflows/ci.yaml/badge.svg?event=release
[GoDoc-Url]: https://pkg.go.dev/github.com/nats-io/nats.go
[GoDoc-Image]: https://img.shields.io/badge/GoDoc-reference-007d9c
[Coverage-Url]: https://coveralls.io/r/nats-io/nats.go?branch=main
[Coverage-image]: https://coveralls.io/repos/github/nats-io/nats.go/badge.svg?branch=main
**Check out [NATS by example](https://natsbyexample.com) - An evolving collection of runnable, cross-client reference examples for NATS.**
## Installation
```bash
# To get the latest released Go client:
go get github.com/nats-io/nats.go@latest
# To get a specific version:
go get github.com/nats-io/nats.go@v1.54.0
# Note that the latest major version for NATS Server is v2:
go get github.com/nats-io/nats-server/v2@latest
```
## Basic Usage
```go
import "github.com/nats-io/nats.go"
// Connect to a server
nc, _ := nats.Connect(nats.DefaultURL)
// Simple Publisher
nc.Publish("foo", []byte("Hello World"))
// Simple Async Subscriber
nc.Subscribe("foo", func(m *nats.Msg) {
fmt.Printf("Received a message: %s\n", string(m.Data))
})
// Responding to a request message
nc.Subscribe("request", func(m *nats.Msg) {
m.Respond([]byte("answer is 42"))
})
// Simple Sync Subscriber
sub, err := nc.SubscribeSync("foo")
m, err := sub.NextMsg(timeout)
// Channel Subscriber
ch := make(chan *nats.Msg, 64)
sub, err := nc.ChanSubscribe("foo", ch)
msg := <- ch
// Unsubscribe
sub.Unsubscribe()
// Drain
sub.Drain()
// Requests
msg, err := nc.Request("help", []byte("help me"), 10*time.Millisecond)
// Replies
nc.Subscribe("help", func(m *nats.Msg) {
nc.Publish(m.Reply, []byte("I can help!"))
})
// Drain connection (Preferred for responders)
// Close() not needed if this is called.
nc.Drain()
// Close connection
nc.Close()
```
## JetStream
[![JetStream API Reference](https://pkg.go.dev/badge/github.com/nats-io/nats.go/jetstream.svg)](https://pkg.go.dev/github.com/nats-io/nats.go/jetstream)
JetStream is the built-in NATS persistence system. `nats.go` provides a built-in
API enabling both managing JetStream assets as well as publishing/consuming
persistent messages.
### Basic usage
```go
// connect to nats server
nc, _ := nats.Connect(nats.DefaultURL)
// create jetstream context from nats connection
js, _ := jetstream.New(nc)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// get existing stream handle
stream, _ := js.Stream(ctx, "foo")
// retrieve consumer handle from a stream
cons, _ := stream.Consumer(ctx, "cons")
// consume messages from the consumer in callback
cc, _ := cons.Consume(func(msg jetstream.Msg) {
fmt.Println("Received jetstream message: ", string(msg.Data()))
msg.Ack()
})
defer cc.Stop()
```
To find more information on `nats.go` JetStream API, visit
[`jetstream/README.md`](jetstream/README.md)
> The current JetStream API replaces the [legacy JetStream API](legacy_jetstream.md)
## Service API
The service API (`micro`) allows you to [easily build NATS services](micro/README.md) The
services API is currently in beta release.
## New Authentication (Nkeys and User Credentials)
This requires server with version >= 2.0.0
NATS servers have a new security and authentication mechanism to authenticate with user credentials and Nkeys.
The simplest form is to use the helper method UserCredentials(credsFilepath).
```go
nc, err := nats.Connect(url, nats.UserCredentials("user.creds"))
```
The helper method creates two callback handlers to present the user JWT and sign the nonce challenge from the server.
The core client library never has direct access to your private key and simply performs the callback for signing the server challenge.
The helper will load and wipe and erase memory it uses for each connect or reconnect.
The helper also can take two entries, one for the JWT and one for the NKey seed file.
```go
nc, err := nats.Connect(url, nats.UserCredentials("user.jwt", "user.nk"))
```
You can also set the callback handlers directly and manage challenge signing directly.
```go
nc, err := nats.Connect(url, nats.UserJWT(jwtCB, sigCB))
```
Bare Nkeys are also supported. The nkey seed should be in a read only file, e.g. seed.txt
```bash
> cat seed.txt
# This is my seed nkey!
SUAGMJH5XLGZKQQWAWKRZJIGMOU4HPFUYLXJMXOO5NLFEO2OOQJ5LPRDPM
```
This is a helper function which will load and decode and do the proper signing for the server nonce.
It will clear memory in between invocations.
You can choose to use the low level option and provide the public key and a signature callback on your own.
```go
opt, err := nats.NkeyOptionFromSeed("seed.txt")
nc, err := nats.Connect(serverUrl, opt)
// Direct
nc, err := nats.Connect(serverUrl, nats.Nkey(pubNkey, sigCB))
```
## TLS
```go
// tls as a scheme will enable secure connections by default. This will also verify the server name.
nc, err := nats.Connect("tls://nats.demo.io:4443")
// If you are using a self-signed certificate, you need to have a tls.Config with RootCAs setup.
// We provide a helper method to make this case easier.
nc, err = nats.Connect("tls://localhost:4443", nats.RootCAs("./configs/certs/ca.pem"))
// If the server requires client certificate, there is a helper function for that too:
cert := nats.ClientCert("./configs/certs/client-cert.pem", "./configs/certs/client-key.pem")
nc, err = nats.Connect("tls://localhost:4443", cert)
// You can also supply a complete tls.Config
certFile := "./configs/certs/client-cert.pem"
keyFile := "./configs/certs/client-key.pem"
cert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
t.Fatalf("error parsing X509 certificate/key pair: %v", err)
}
config := &tls.Config{
ServerName: opts.Host,
Certificates: []tls.Certificate{cert},
RootCAs: pool,
MinVersion: tls.VersionTLS12,
}
nc, err = nats.Connect("nats://localhost:4443", nats.Secure(config))
if err != nil {
t.Fatalf("Got an error on Connect with Secure Options: %+v\n", err)
}
```
## Wildcard Subscriptions
```go
// "*" matches any token, at any level of the subject.
nc.Subscribe("foo.*.baz", func(m *Msg) {
fmt.Printf("Msg received on [%s] : %s\n", m.Subject, string(m.Data))
})
nc.Subscribe("foo.bar.*", func(m *Msg) {
fmt.Printf("Msg received on [%s] : %s\n", m.Subject, string(m.Data))
})
// ">" matches any length of the tail of a subject, and can only be the last token
// E.g. 'foo.>' will match 'foo.bar', 'foo.bar.baz', 'foo.foo.bar.bax.22'
nc.Subscribe("foo.>", func(m *Msg) {
fmt.Printf("Msg received on [%s] : %s\n", m.Subject, string(m.Data))
})
// Matches all of the above
nc.Publish("foo.bar.baz", []byte("Hello World"))
```
## Queue Groups
```go
// All subscriptions with the same queue name will form a queue group.
// Each message will be delivered to only one subscriber per queue group,
// using queuing semantics. You can have as many queue groups as you wish.
// Normal subscribers will continue to work as expected.
nc.QueueSubscribe("foo", "job_workers", func(_ *Msg) {
received += 1
})
```
## Advanced Usage
```go
// Normally, the library will return an error when trying to connect and
// there is no server running. The RetryOnFailedConnect option will set
// the connection in reconnecting state if it failed to connect right away.
nc, err := nats.Connect(nats.DefaultURL,
nats.RetryOnFailedConnect(true),
nats.MaxReconnects(10),
nats.ReconnectWait(time.Second),
nats.ReconnectHandler(func(_ *nats.Conn) {
// Note that this will be invoked for the first asynchronous connect.
}))
if err != nil {
// Should not return an error even if it can't connect, but you still
// need to check in case there are some configuration errors.
}
// Flush connection to server, returns when all messages have been processed.
nc.Flush()
fmt.Println("All clear!")
// FlushTimeout specifies a timeout value as well.
err := nc.FlushTimeout(1*time.Second)
if err != nil {
fmt.Println("Flushed timed out!")
} else {
fmt.Println("All clear!")
}
// Auto-unsubscribe after MAX_WANTED messages received
const MAX_WANTED = 10
sub, err := nc.Subscribe("foo")
sub.AutoUnsubscribe(MAX_WANTED)
// Multiple connections
nc1 := nats.Connect("nats://host1:4222")
nc2 := nats.Connect("nats://host2:4222")
nc1.Subscribe("foo", func(m *Msg) {
fmt.Printf("Received a message: %s\n", string(m.Data))
})
nc2.Publish("foo", []byte("Hello World!"))
```
## Clustered Usage
```go
var servers = "nats://localhost:1222, nats://localhost:1223, nats://localhost:1224"
nc, err := nats.Connect(servers)
// Optionally set ReconnectWait and MaxReconnect attempts.
// This example means 10 seconds total per backend.
nc, err = nats.Connect(servers, nats.MaxReconnects(5), nats.ReconnectWait(2 * time.Second))
// You can also add some jitter for the reconnection.
// This call will add up to 500 milliseconds for non TLS connections and 2 seconds for TLS connections.
// If not specified, the library defaults to 100 milliseconds and 1 second, respectively.
nc, err = nats.Connect(servers, nats.ReconnectJitter(500*time.Millisecond, 2*time.Second))
// You can also specify a custom reconnect delay handler. If set, the library will invoke it when it has tried
// all URLs in its list. The value returned will be used as the total sleep time, so add your own jitter.
// The library will pass the number of times it went through the whole list.
nc, err = nats.Connect(servers, nats.CustomReconnectDelay(func(attempts int) time.Duration {
return someBackoffFunction(attempts)
}))
// Optionally disable randomization of the server pool
nc, err = nats.Connect(servers, nats.DontRandomize())
// Setup callbacks to be notified on disconnects, reconnects and connection closed.
nc, err = nats.Connect(servers,
nats.DisconnectErrHandler(func(nc *nats.Conn, err error) {
fmt.Printf("Got disconnected! Reason: %q\n", err)
}),
nats.ReconnectHandler(func(nc *nats.Conn) {
fmt.Printf("Got reconnected to %v!\n", nc.ConnectedUrl())
}),
nats.ClosedHandler(func(nc *nats.Conn) {
fmt.Printf("Connection closed. Reason: %q\n", nc.LastError())
})
)
// When connecting to a mesh of servers with auto-discovery capabilities,
// you may need to provide a username/password or token in order to connect
// to any server in that mesh when authentication is required.
// Instead of providing the credentials in the initial URL, you will use
// new option setters:
nc, err = nats.Connect("nats://localhost:4222", nats.UserInfo("foo", "bar"))
// For token based authentication:
nc, err = nats.Connect("nats://localhost:4222", nats.Token("S3cretT0ken"))
// You can even pass the two at the same time in case one of the servers
// in the mesh requires token instead of user name and password.
nc, err = nats.Connect("nats://localhost:4222",
nats.UserInfo("foo", "bar"),
nats.Token("S3cretT0ken"))
// Note that if credentials are specified in the initial URLs, they take
// precedence on the credentials specified through the options.
// For instance, in the connect call below, the client library will use
// the user "my" and password "pwd" to connect to localhost:4222, however,
// it will use username "foo" and password "bar" when (re)connecting to
// a different server URL that it got as part of the auto-discovery.
nc, err = nats.Connect("nats://my:pwd@localhost:4222", nats.UserInfo("foo", "bar"))
```
## Context support (+Go 1.7)
```go
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
nc, err := nats.Connect(nats.DefaultURL)
// Request with context
msg, err := nc.RequestWithContext(ctx, "foo", []byte("bar"))
// Synchronous subscriber with context
sub, err := nc.SubscribeSync("foo")
msg, err := sub.NextMsgWithContext(ctx)
```
## Backward compatibility
In the development of nats.go, we are committed to maintaining backward compatibility and ensuring a stable and reliable experience for all users. In general, we follow the standard go compatibility guidelines.
However, it's important to clarify our stance on certain types of changes:
- **Expanding structures:**
Adding new fields to structs is not considered a breaking change.
- **Adding methods to exported interfaces:**
Extending public interfaces with new methods is also not viewed as a breaking change within the context of this project. It is important to note that no unexported methods will be added to interfaces allowing users to implement them.
Additionally, this library always supports at least 2 latest minor Go versions. For example, if the latest Go version is 1.22, the library will support Go 1.21 and 1.22.
## Testing
See [TESTING.md](TESTING.md) for how to run the test suite locally.
## License
Unless otherwise noted, the NATS source files are distributed
under the Apache Version 2.0 license found in the LICENSE file.
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fnats-io%2Fgo-nats.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fnats-io%2Fgo-nats?ref=badge_large)
modules/route-proxy/vendor/github.com/nats-io/nats.go/TESTING.md
Generated Vendored
+214
View File
@@ -0,0 +1,214 @@
# Testing
## TL;DR
```shell
make test # full race-enabled suite, no docker
make test T=TestName PKG=./test/... # iterate on a single test
```
## How the suite is set up
The repo uses two module files: `go.mod` for production (minimal dependencies)
and `go_test.mod` for testing. Always pass `-modfile=go_test.mod` to `go test`;
the make targets below do this for you.
Unit tests live at the repo root (package `nats`, white-box) and run with plain
`go test`. Integration tests live in `./test`, `./jetstream/test`, and
`./micro/test`; they bring up real NATS servers by talking to a **tester**
service, which spawns `nats-server` instances, clusters, and super-clusters on
demand. Tests drive it through the `github.com/synadia-io/orbit.go/ntf-client`
package.
There are two ways to run that tester, and tests reach both the same way:
- **In-process (the default)** — a `TestMain` starts
`github.com/synadia-io/orbit.go/ntf` inside each test binary. No docker, no
setup, nothing to tear down.
- **Out-of-process** — the `synadia/ntf-server` docker image, located via the
`TESTER_NATS_URL` environment variable. This is what CI uses.
`TESTER_NATS_URL` always wins: set it and the tests talk to that tester instead
of starting their own. The `make test` targets clear it so an exported value
left over in your shell cannot silently redirect them, but a raw `go test`
inherits it — unset it if a plain `go test` tries to reach a container you did
not intend.
The two paths do **not** necessarily run the same server build: in-process links
whatever `nats-server` `orbit.go/ntf` depends on, while the container ships its
own (pinned in `.github/workflows/ci.yaml`). To stop CI drifting onto the wrong
one, `TestMain` refuses to start the in-process tester when `GITHUB_ACTIONS` is
set but `TESTER_NATS_URL` is not. `GITHUB_ACTIONS` rather than the more general
`CI` because devcontainers and other runners set that one too, and they have no
reason to be forced onto a container.
That guard only catches a missing URL, not a version difference, so the first
test to reach the tester prints the server it got:
```text
tester at nats://localhost:62354: nats-server 2.14.5
```
`go test` shows this whenever the package is run with `-v` (all CI steps are) or
when a test in it fails. If a failure reproduces locally but not in CI, or the
reverse, compare that line against the image tag in `ci.yaml` first.
## In-process mode (default, no docker)
```shell
make test # everything
make test T=TestSubSubject PKG=./test/... # one test, verbose
make test PKG=./jetstream/test/... # one package
make test-norace # NoRace tests (race detector off)
```
Equivalent to:
```shell
go test -modfile=go_test.mod -tags=internal_testing -race -p=1 ./... --failfast -vet=off
```
Because the tester and the `nats-server` instances it spawns run inside the test
binary, `-race` instruments the server too: expect somewhat slower runs, and note
that a data race in `nats-server` surfaces here as a nats.go test failure.
This is also why `go_test.mod` declares `go 1.26.0` while `go.mod` stays at
`1.25.0` — `orbit.go/ntf` requires 1.26, and merely requiring a module raises the
floor for the whole test module. Production builds are unaffected; 1.26 is the
floor for the CI test matrix.
## Testing against a different nats-server
The in-process tester links `nats-server` as an ordinary Go dependency, so
pointing it at another branch, tag, commit, or local checkout is a module
operation — no docker, no image build:
```shell
make server-replace V=main # nats-server main
make server-replace V=v2.13.0 # a specific tag
make server-replace V=/path/to/your/nats-server # a local checkout
make test # run against it
make server-replace-drop # undo
```
Verify what you are actually running:
```shell
go list -m -modfile=go_test.mod github.com/nats-io/nats-server/v2
```
**Do not use `go get` for this.** A branch pseudo-version is derived from the
last reachable tag, so `@main` can sort *below* the release `orbit.go/ntf`
requires. `go get` resolves that conflict by removing ntf from the module graph,
which silently leaves you with no in-process tester:
```text
go: downgraded github.com/nats-io/nats-server/v2 => v2.14.1-0.2026...
go: removed github.com/synadia-io/orbit.go/ntf
```
A `replace` directive overrides the version outright and has no such failure
mode, which is what `make server-replace` uses.
`go_test.mod` is a tracked file, so the replace must not be committed. Note that
`make server-replace-drop` removes the directive but does **not** fully restore
the file: while the replacement was active, `go mod tidy` recorded the replaced
module's own newer dependencies, and MVS never downgrades them again. The target
warns when this happens; for an exact restore use
`git checkout -- go_test.mod go_test.sum`.
The docker path can also test a different server, but only versions Synadia
publishes as images — this is what the nightly `latest-server.yaml` workflow
uses:
```shell
make tester-up-host TESTER_IMAGE=synadia/ntf-server:nightly-latest-nats-main
make test-docker
```
## Host-side mode (docker, iterating on individual tests)
`make tester-up-host` starts the tester with its ports published, so `go test`
run from your terminal can reach the spawned servers via localhost.
```shell
make tester-up-host
make test-docker T=TestSubSubject PKG=./test/... # one test, verbose
make test-docker PKG=./jetstream/test/... # one package
make test-docker # everything
```
Use the `test-docker` targets, not `test` — plain `make test` ignores the
container and uses the in-process tester.
`make test-docker` wraps the full invocation, which is equivalent to:
```shell
TESTER_NATS_URL=nats://localhost:4222 \
go test -modfile=go_test.mod -tags=internal_testing -race -p=1 ./... --failfast -vet=off
```
`-p=1` is required here: a shared container tester does not tolerate concurrent
CreateServer calls from independent test binaries. The in-process tester is
per-binary and could run packages in parallel — measurably faster — but `-p=1`
is kept deliberately, since more concurrent server churn risks intermittent
connection failures and the wall-clock saving is not worth chasing flakes.
Known caveat: on macOS, docker-proxy races the tester's port handover, so in
heavy suites roughly 5-10% of server creations can fail with
`bind: address already in use`. Rerun the failing test, or use
sibling-container mode for full-suite runs.
## Sibling-container mode (full suite, matches CI)
```shell
make tester-up
make test-tester
make tester-down
```
`make test-tester` runs the whole suite (NoRace pass, then the race-enabled
pass) inside a Go container on the same docker network as the tester — no
published ports, so the docker-proxy race above does not apply. This is the
same shape CI uses, with the tester attached as a service container
(`.github/workflows/ci.yaml`).
## NoRace tests
Tests prefixed `TestNoRace` are guarded by `//go:build !race &&
!skip_no_race_tests` and must run with the race detector off:
`make test-norace`.
## Build tags
- `internal_testing` — exposes internal test hooks from `testing_internal.go`;
required by some tests in `./test`. The make targets set it.
- `skip_no_race_tests` — excludes the NoRace tests from a non-race run (used
by `scripts/cov.sh`).
- `compat` — compatibility tests in `test/compat_test.go`, which connect to an
external NATS server via `NATS_URL`.
- `go1.23` — iterator-based tests in `test/nats_iter_test.go`.
## Coverage
```shell
TESTER_NATS_URL=nats://localhost:4222 ./scripts/cov.sh
```
Merges unit and integration coverage into `acc.out` and opens the HTML report
(CI passes an argument to skip the browser).
## Troubleshooting
- `cannot reach the tester at ...` — `TESTER_NATS_URL` is set but nothing is
answering there. Either start the container (`make tester-up-host`) or unset
the variable to use the in-process tester.
- `TESTER_NATS_URL must be set in CI` — running under GitHub Actions without
`TESTER_NATS_URL`. This guard exists so CI cannot silently swap the pinned
container server for the in-process one, which is a different version. Point
`TESTER_NATS_URL` at a tester, or unset `GITHUB_ACTIONS`.
- Tester container misbehaving — `docker logs -f nats-tester` shows server spawn
and config errors; `docker restart nats-tester` restarts it while keeping
those logs. `make tester-down` removes the container and discards them.
- Updating test-only dependencies: `go mod tidy -modfile=go_test.mod` (never
change the main `go.mod` for test dependencies).
modules/route-proxy/vendor/github.com/nats-io/nats.go/context.go
Generated Vendored
+246
View File
@@ -0,0 +1,246 @@
// Copyright 2016-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package nats
import (
"context"
"reflect"
)
// RequestMsgWithContext takes a context, a subject and payload
// in bytes and request expecting a single response.
func (nc *Conn) RequestMsgWithContext(ctx context.Context, msg *Msg) (*Msg, error) {
if msg == nil {
return nil, ErrInvalidMsg
}
hdr, err := msg.headerBytes()
if err != nil {
return nil, err
}
return nc.requestWithContext(ctx, msg.Subject, hdr, msg.Data)
}
// RequestWithContext takes a context, a subject and payload
// in bytes and request expecting a single response.
func (nc *Conn) RequestWithContext(ctx context.Context, subj string, data []byte) (*Msg, error) {
return nc.requestWithContext(ctx, subj, nil, data)
}
func (nc *Conn) requestWithContext(ctx context.Context, subj string, hdr, data []byte) (*Msg, error) {
if ctx == nil {
return nil, ErrInvalidContext
}
if nc == nil {
return nil, ErrInvalidConnection
}
// Check whether the context is done already before making
// the request.
if ctx.Err() != nil {
return nil, ctx.Err()
}
var m *Msg
var err error
// If user wants the old style.
if nc.useOldRequestStyle() {
m, err = nc.oldRequestWithContext(ctx, subj, hdr, data)
} else {
mch, token, err := nc.createNewRequestAndSend(subj, hdr, data)
if err != nil {
return nil, err
}
var ok bool
select {
case m, ok = <-mch:
if !ok {
return nil, ErrConnectionClosed
}
case <-ctx.Done():
nc.mu.Lock()
delete(nc.respMap, token)
nc.mu.Unlock()
return nil, ctx.Err()
}
}
// Check for no responder status.
if err == nil && len(m.Data) == 0 && m.Header.Get(statusHdr) == noResponders {
m, err = nil, ErrNoResponders
}
return m, err
}
// oldRequestWithContext utilizes inbox and subscription per request.
func (nc *Conn) oldRequestWithContext(ctx context.Context, subj string, hdr, data []byte) (*Msg, error) {
inbox := nc.NewInbox()
ch := make(chan *Msg, RequestChanLen)
s, err := nc.subscribe(inbox, _EMPTY_, nil, ch, nil, true, nil)
if err != nil {
return nil, err
}
s.AutoUnsubscribe(1)
defer s.Unsubscribe()
err = nc.publish(subj, inbox, false, hdr, data)
if err != nil {
return nil, err
}
return s.NextMsgWithContext(ctx)
}
func (s *Subscription) nextMsgWithContext(ctx context.Context, pullSubInternal, waitIfNoMsg bool) (*Msg, error) {
if ctx == nil {
return nil, ErrInvalidContext
}
if s == nil {
return nil, ErrBadSubscription
}
if ctx.Err() != nil {
return nil, ctx.Err()
}
s.mu.Lock()
err := s.validateNextMsgState(pullSubInternal)
if err != nil {
s.mu.Unlock()
return nil, err
}
// snapshot
mch := s.mch
s.mu.Unlock()
var ok bool
var msg *Msg
// If something is available right away, let's optimize that case.
select {
case msg, ok = <-mch:
if !ok {
return nil, s.getNextMsgErr()
}
if err := s.processNextMsgDelivered(msg); err != nil {
return nil, err
}
return msg, nil
default:
// If internal and we don't want to wait, signal that there is no
// message in the internal queue.
if pullSubInternal && !waitIfNoMsg {
return nil, errNoMessages
}
}
select {
case msg, ok = <-mch:
if !ok {
return nil, s.getNextMsgErr()
}
if err := s.processNextMsgDelivered(msg); err != nil {
return nil, err
}
case <-ctx.Done():
return nil, ctx.Err()
}
return msg, nil
}
// NextMsgWithContext takes a context and returns the next message
// available to a synchronous subscriber, blocking until it is delivered
// or context gets canceled.
func (s *Subscription) NextMsgWithContext(ctx context.Context) (*Msg, error) {
return s.nextMsgWithContext(ctx, false, true)
}
// FlushWithContext will allow a context to control the duration
// of a Flush() call. This context should be non-nil and should
// have a deadline set. We will return an error if none is present.
func (nc *Conn) FlushWithContext(ctx context.Context) error {
if nc == nil {
return ErrInvalidConnection
}
if ctx == nil {
return ErrInvalidContext
}
_, ok := ctx.Deadline()
if !ok {
return ErrNoDeadlineContext
}
nc.mu.Lock()
if nc.isClosed() {
nc.mu.Unlock()
return ErrConnectionClosed
}
// Create a buffered channel to prevent chan send to block
// in processPong()
ch := make(chan struct{}, 1)
nc.sendPing(ch)
nc.mu.Unlock()
var err error
select {
case _, ok := <-ch:
if !ok {
err = ErrConnectionClosed
} else {
close(ch)
}
case <-ctx.Done():
err = ctx.Err()
}
if err != nil {
nc.removeFlushEntry(ch)
}
return err
}
// RequestWithContext will create an Inbox and perform a Request
// using the provided cancellation context with the Inbox reply
// for the data v. A response will be decoded into the vPtr last parameter.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) RequestWithContext(ctx context.Context, subject string, v any, vPtr any) error {
if ctx == nil {
return ErrInvalidContext
}
b, err := c.Enc.Encode(subject, v)
if err != nil {
return err
}
m, err := c.Conn.RequestWithContext(ctx, subject, b)
if err != nil {
return err
}
if reflect.TypeOf(vPtr) == emptyMsgType {
mPtr := vPtr.(*Msg)
*mPtr = *m
} else {
err := c.Enc.Decode(m.Subject, m.Data, vPtr)
if err != nil {
return err
}
}
return nil
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/dependencies.md
Generated Vendored
+15
View File
@@ -0,0 +1,15 @@
# External Dependencies
This file lists the dependencies used in this repository.
| Dependency | License |
|-------------------------------------------|--------------|
| Go | BSD 3-Clause |
| github.com/golang/protobuf/proto | BSD-3-Clause |
| github.com/klauspost/compress | BSD-3-Clause |
| github.com/nats-io/nkeys | Apache-2.0 |
| github.com/nats-io/nuid | Apache-2.0 |
| github.com/synadia-io/orbit.go/ntf-client | Apache-2.0 |
| go.uber.org/goleak | MIT |
| golang.org/x/text | BSD-3-Clause |
| google.golang.org/protobuf | BSD-3-Clause |
modules/route-proxy/vendor/github.com/nats-io/nats.go/dependencies.tpl
Generated Vendored
+9
View File
@@ -0,0 +1,9 @@
# External Dependencies
This file lists the dependencies used in this repository.
{{/* compress has actually a BSD 3-Clause license, but the License file in the repo confuses go-license tooling, hence the manual exception */}}
| Dependency | License |
|--------------------------------------------------|-----------------------------------------|
{{ range . }}| {{ .Name }} | {{ if eq .Name "github.com/klauspost/compress/flate" }}BSD 3-Clause{{ else }}{{ .LicenseName }}{{ end }} |
{{ end }}
modules/route-proxy/vendor/github.com/nats-io/nats.go/enc.go
Generated Vendored
+303
View File
@@ -0,0 +1,303 @@
// Copyright 2012-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package nats
import (
"errors"
"fmt"
"reflect"
"sync"
"time"
// Default Encoders
"github.com/nats-io/nats.go/encoders/builtin"
)
//lint:file-ignore SA1019 Ignore deprecation warnings for EncodedConn
// Encoder interface is for all register encoders
//
// Deprecated: Encoded connections are no longer supported.
type Encoder interface {
Encode(subject string, v any) ([]byte, error)
Decode(subject string, data []byte, vPtr any) error
}
var encMap map[string]Encoder
var encLock sync.Mutex
// Indexed names into the Registered Encoders.
const (
JSON_ENCODER = "json"
GOB_ENCODER = "gob"
DEFAULT_ENCODER = "default"
)
func init() {
encMap = make(map[string]Encoder)
// Register json, gob and default encoder
RegisterEncoder(JSON_ENCODER, &builtin.JsonEncoder{})
RegisterEncoder(GOB_ENCODER, &builtin.GobEncoder{})
RegisterEncoder(DEFAULT_ENCODER, &builtin.DefaultEncoder{})
}
// EncodedConn are the preferred way to interface with NATS. They wrap a bare connection to
// a nats server and have an extendable encoder system that will encode and decode messages
// from raw Go types.
//
// Deprecated: Encoded connections are no longer supported.
type EncodedConn struct {
Conn *Conn
Enc Encoder
}
// NewEncodedConn will wrap an existing Connection and utilize the appropriate registered
// encoder.
//
// Deprecated: Encoded connections are no longer supported.
func NewEncodedConn(c *Conn, encType string) (*EncodedConn, error) {
if c == nil {
return nil, errors.New("nats: Nil Connection")
}
if c.IsClosed() {
return nil, ErrConnectionClosed
}
ec := &EncodedConn{Conn: c, Enc: EncoderForType(encType)}
if ec.Enc == nil {
return nil, fmt.Errorf("no encoder registered for '%s'", encType)
}
return ec, nil
}
// RegisterEncoder will register the encType with the given Encoder. Useful for customization.
//
// Deprecated: Encoded connections are no longer supported.
func RegisterEncoder(encType string, enc Encoder) {
encLock.Lock()
defer encLock.Unlock()
encMap[encType] = enc
}
// EncoderForType will return the registered Encoder for the encType.
//
// Deprecated: Encoded connections are no longer supported.
func EncoderForType(encType string) Encoder {
encLock.Lock()
defer encLock.Unlock()
return encMap[encType]
}
// Publish publishes the data argument to the given subject. The data argument
// will be encoded using the associated encoder.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Publish(subject string, v any) error {
b, err := c.Enc.Encode(subject, v)
if err != nil {
return err
}
return c.Conn.publish(subject, _EMPTY_, false, nil, b)
}
// PublishRequest will perform a Publish() expecting a response on the
// reply subject. Use Request() for automatically waiting for a response
// inline.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) PublishRequest(subject, reply string, v any) error {
b, err := c.Enc.Encode(subject, v)
if err != nil {
return err
}
return c.Conn.publish(subject, reply, true, nil, b)
}
// Request will create an Inbox and perform a Request() call
// with the Inbox reply for the data v. A response will be
// decoded into the vPtr Response.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Request(subject string, v any, vPtr any, timeout time.Duration) error {
b, err := c.Enc.Encode(subject, v)
if err != nil {
return err
}
m, err := c.Conn.Request(subject, b, timeout)
if err != nil {
return err
}
if reflect.TypeOf(vPtr) == emptyMsgType {
mPtr := vPtr.(*Msg)
*mPtr = *m
} else {
err = c.Enc.Decode(m.Subject, m.Data, vPtr)
}
return err
}
// Handler is a specific callback used for Subscribe. It is generalized to
// an any, but we will discover its format and arguments at runtime
// and perform the correct callback, including demarshaling encoded data
// back into the appropriate struct based on the signature of the Handler.
//
// Handlers are expected to have one of four signatures.
//
// type person struct {
// Name string `json:"name,omitempty"`
// Age uint `json:"age,omitempty"`
// }
//
// handler := func(m *Msg)
// handler := func(p *person)
// handler := func(subject string, o *obj)
// handler := func(subject, reply string, o *obj)
//
// These forms allow a callback to request a raw Msg ptr, where the processing
// of the message from the wire is untouched. Process a JSON representation
// and demarshal it into the given struct, e.g. person.
// There are also variants where the callback wants either the subject, or the
// subject and the reply subject.
//
// Deprecated: Encoded connections are no longer supported.
type Handler any
// Dissect the cb Handler's signature
func argInfo(cb Handler) (reflect.Type, int) {
cbType := reflect.TypeOf(cb)
if cbType.Kind() != reflect.Func {
panic("nats: Handler needs to be a func")
}
numArgs := cbType.NumIn()
if numArgs == 0 {
return nil, numArgs
}
return cbType.In(numArgs - 1), numArgs
}
var emptyMsgType = reflect.TypeOf(&Msg{})
// Subscribe will create a subscription on the given subject and process incoming
// messages using the specified Handler. The Handler should be a func that matches
// a signature from the description of Handler from above.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Subscribe(subject string, cb Handler) (*Subscription, error) {
return c.subscribe(subject, _EMPTY_, cb)
}
// QueueSubscribe will create a queue subscription on the given subject and process
// incoming messages using the specified Handler. The Handler should be a func that
// matches a signature from the description of Handler from above.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) QueueSubscribe(subject, queue string, cb Handler) (*Subscription, error) {
return c.subscribe(subject, queue, cb)
}
// Internal implementation that all public functions will use.
func (c *EncodedConn) subscribe(subject, queue string, cb Handler) (*Subscription, error) {
if cb == nil {
return nil, errors.New("nats: Handler required for EncodedConn Subscription")
}
argType, numArgs := argInfo(cb)
if argType == nil {
return nil, errors.New("nats: Handler requires at least one argument")
}
cbValue := reflect.ValueOf(cb)
wantsRaw := (argType == emptyMsgType)
natsCB := func(m *Msg) {
var oV []reflect.Value
if wantsRaw {
oV = []reflect.Value{reflect.ValueOf(m)}
} else {
var oPtr reflect.Value
if argType.Kind() != reflect.Ptr {
oPtr = reflect.New(argType)
} else {
oPtr = reflect.New(argType.Elem())
}
if err := c.Enc.Decode(m.Subject, m.Data, oPtr.Interface()); err != nil {
if c.Conn.Opts.AsyncErrorCB != nil {
c.Conn.ach.push(func() {
c.Conn.Opts.AsyncErrorCB(c.Conn, m.Sub, errors.New("nats: Got an error trying to unmarshal: "+err.Error()))
})
}
return
}
if argType.Kind() != reflect.Ptr {
oPtr = reflect.Indirect(oPtr)
}
// Callback Arity
switch numArgs {
case 1:
oV = []reflect.Value{oPtr}
case 2:
subV := reflect.ValueOf(m.Subject)
oV = []reflect.Value{subV, oPtr}
case 3:
subV := reflect.ValueOf(m.Subject)
replyV := reflect.ValueOf(m.Reply)
oV = []reflect.Value{subV, replyV, oPtr}
}
}
cbValue.Call(oV)
}
return c.Conn.subscribe(subject, queue, natsCB, nil, nil, false, nil)
}
// FlushTimeout allows a Flush operation to have an associated timeout.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) FlushTimeout(timeout time.Duration) (err error) {
return c.Conn.FlushTimeout(timeout)
}
// Flush will perform a round trip to the server and return when it
// receives the internal reply.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Flush() error {
return c.Conn.Flush()
}
// Close will close the connection to the server. This call will release
// all blocking calls, such as Flush(), etc.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Close() {
c.Conn.Close()
}
// Drain will put a connection into a drain state. All subscriptions will
// immediately be put into a drain state. Upon completion, the publishers
// will be drained and can not publish any additional messages. Upon draining
// of the publishers, the connection will be closed. Use the ClosedCB()
// option to know when the connection has moved from draining to closed.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) Drain() error {
return c.Conn.Drain()
}
// LastError reports the last error encountered via the Connection.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) LastError() error {
return c.Conn.LastError()
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/encoders/builtin/default_enc.go
Generated Vendored
+123
View File
@@ -0,0 +1,123 @@
// Copyright 2012-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package builtin
import (
"bytes"
"fmt"
"reflect"
"strconv"
"unsafe"
)
// DefaultEncoder implementation for EncodedConn.
// This encoder will leave []byte and string untouched, but will attempt to
// turn numbers into appropriate strings that can be decoded. It will also
// properly encoded and decode bools. If will encode a struct, but if you want
// to properly handle structures you should use JsonEncoder.
//
// Deprecated: Encoded connections are no longer supported.
type DefaultEncoder struct {
// Empty
}
var trueB = []byte("true")
var falseB = []byte("false")
var nilB = []byte("")
// Encode
//
// Deprecated: Encoded connections are no longer supported.
func (je *DefaultEncoder) Encode(subject string, v any) ([]byte, error) {
switch arg := v.(type) {
case string:
bytes := *(*[]byte)(unsafe.Pointer(&arg))
return bytes, nil
case []byte:
return arg, nil
case bool:
if arg {
return trueB, nil
} else {
return falseB, nil
}
case nil:
return nilB, nil
default:
var buf bytes.Buffer
fmt.Fprintf(&buf, "%+v", arg)
return buf.Bytes(), nil
}
}
// Decode
//
// Deprecated: Encoded connections are no longer supported.
func (je *DefaultEncoder) Decode(subject string, data []byte, vPtr any) error {
// Figure out what it's pointing to...
sData := *(*string)(unsafe.Pointer(&data))
switch arg := vPtr.(type) {
case *string:
*arg = sData
return nil
case *[]byte:
*arg = data
return nil
case *int:
n, err := strconv.ParseInt(sData, 10, 64)
if err != nil {
return err
}
*arg = int(n)
return nil
case *int32:
n, err := strconv.ParseInt(sData, 10, 64)
if err != nil {
return err
}
*arg = int32(n)
return nil
case *int64:
n, err := strconv.ParseInt(sData, 10, 64)
if err != nil {
return err
}
*arg = int64(n)
return nil
case *float32:
n, err := strconv.ParseFloat(sData, 32)
if err != nil {
return err
}
*arg = float32(n)
return nil
case *float64:
n, err := strconv.ParseFloat(sData, 64)
if err != nil {
return err
}
*arg = float64(n)
return nil
case *bool:
b, err := strconv.ParseBool(sData)
if err != nil {
return err
}
*arg = b
return nil
default:
vt := reflect.TypeOf(arg).Elem()
return fmt.Errorf("nats: Default Encoder can't decode to type %s", vt)
}
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/encoders/builtin/gob_enc.go
Generated Vendored
+51
View File
@@ -0,0 +1,51 @@
// Copyright 2013-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package builtin
import (
"bytes"
"encoding/gob"
)
// GobEncoder is a Go specific GOB Encoder implementation for EncodedConn.
// This encoder will use the builtin encoding/gob to Marshal
// and Unmarshal most types, including structs.
//
// Deprecated: Encoded connections are no longer supported.
type GobEncoder struct {
// Empty
}
// FIXME(dlc) - This could probably be more efficient.
// Encode
//
// Deprecated: Encoded connections are no longer supported.
func (ge *GobEncoder) Encode(subject string, v any) ([]byte, error) {
b := new(bytes.Buffer)
enc := gob.NewEncoder(b)
if err := enc.Encode(v); err != nil {
return nil, err
}
return b.Bytes(), nil
}
// Decode
//
// Deprecated: Encoded connections are no longer supported.
func (ge *GobEncoder) Decode(subject string, data []byte, vPtr any) (err error) {
dec := gob.NewDecoder(bytes.NewBuffer(data))
err = dec.Decode(vPtr)
return
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/encoders/builtin/json_enc.go
Generated Vendored
+62
View File
@@ -0,0 +1,62 @@
// Copyright 2012-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package builtin
import (
"encoding/json"
"strings"
)
// JsonEncoder is a JSON Encoder implementation for EncodedConn.
// This encoder will use the builtin encoding/json to Marshal
// and Unmarshal most types, including structs.
//
// Deprecated: Encoded connections are no longer supported.
type JsonEncoder struct {
// Empty
}
// Encode
//
// Deprecated: Encoded connections are no longer supported.
func (je *JsonEncoder) Encode(subject string, v any) ([]byte, error) {
b, err := json.Marshal(v)
if err != nil {
return nil, err
}
return b, nil
}
// Decode
//
// Deprecated: Encoded connections are no longer supported.
func (je *JsonEncoder) Decode(subject string, data []byte, vPtr any) (err error) {
switch arg := vPtr.(type) {
case *string:
// If they want a string and it is a JSON string, strip quotes
// This allows someone to send a struct but receive as a plain string
// This cast should be efficient for Go 1.3 and beyond.
str := string(data)
if strings.HasPrefix(str, `"`) && strings.HasSuffix(str, `"`) {
*arg = str[1 : len(str)-1]
} else {
*arg = str
}
case *[]byte:
*arg = data
default:
err = json.Unmarshal(data, arg)
}
return
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/go_test.mod
Generated Vendored
+24
View File
@@ -0,0 +1,24 @@
module github.com/nats-io/nats.go
go 1.26.0
require (
github.com/golang/protobuf v1.5.4
github.com/klauspost/compress v1.20.0
github.com/nats-io/jwt/v2 v2.8.2
github.com/nats-io/nkeys v0.4.16
github.com/nats-io/nuid v1.0.1
github.com/synadia-io/orbit.go/ntf v0.0.1
github.com/synadia-io/orbit.go/ntf-client v0.0.3
google.golang.org/protobuf v1.36.12
)
require (
github.com/antithesishq/antithesis-sdk-go v0.8.0 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/nats-server/v2 v2.15.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
modules/route-proxy/vendor/github.com/nats-io/nats.go/go_test.sum
Generated Vendored
+33
View File
@@ -0,0 +1,33 @@
github.com/antithesishq/antithesis-sdk-go v0.8.0 h1:bVV5ZAuwaREn4klRSJusYaA+A7NXq8Qi9C/knCaQ2N4=
github.com/antithesishq/antithesis-sdk-go v0.8.0/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.15.0 h1:M99yf0y05rTr46/qc/Is6ZAowI58Ryp2SjufLCUeVJc=
github.com/nats-io/nats-server/v2 v2.15.0/go.mod h1:5qLF4CDGzZVFt//3fUrY1ePpwbi05r7QHPNroSUtolk=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/synadia-io/orbit.go/ntf v0.0.1 h1:eTL4Dln8ha8ELjVNVxnSpsVG1zHxr6g/eMwhFL98r1Q=
github.com/synadia-io/orbit.go/ntf v0.0.1/go.mod h1:a+ZZMkFv7dikn9hmE1PH/9FysauhZEFk6MBjCVwGn/s=
github.com/synadia-io/orbit.go/ntf-client v0.0.3 h1:FoFjSd+idhy10BqDadGpUk4OcpBu/7V/Y8sHWhbSjdc=
github.com/synadia-io/orbit.go/ntf-client v0.0.3/go.mod h1:P745h+KxJPtoHF6csj2PQUB+uf72XR49o2krTaTQ8Ls=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
modules/route-proxy/vendor/github.com/nats-io/nats.go/internal/parser/parse.go
Generated Vendored
+104
View File
@@ -0,0 +1,104 @@
// Copyright 2020-2022 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package parser
import (
"errors"
"fmt"
)
const (
AckDomainTokenPos = iota + 2
AckAccHashTokenPos
AckStreamTokenPos
AckConsumerTokenPos
AckNumDeliveredTokenPos
AckStreamSeqTokenPos
AckConsumerSeqTokenPos
AckTimestampSeqTokenPos
AckNumPendingTokenPos
)
var ErrInvalidSubjectFormat = errors.New("invalid format of ACK subject")
// Quick parser for positive numbers in ack reply encoding.
// NOTE: This parser does not detect uint64 overflow
func ParseNum(d string) (n uint64) {
if len(d) == 0 {
return 0
}
// ASCII numbers 0-9
const (
asciiZero = 48
asciiNine = 57
)
for _, dec := range d {
if dec < asciiZero || dec > asciiNine {
return 0
}
n = n*10 + uint64(dec) - asciiZero
}
return
}
func GetMetadataFields(subject string) ([]string, error) {
v1TokenCounts, v2TokenCounts := 9, 12
var start int
tokens := make([]string, 0, v2TokenCounts)
for i := 0; i < len(subject); i++ {
if subject[i] == '.' {
tokens = append(tokens, subject[start:i])
start = i + 1
}
}
tokens = append(tokens, subject[start:])
//
// Newer server will include the domain name and account hash in the subject,
// and a token at the end.
//
// Old subject was:
// $JS.ACK.<stream>.<consumer>.<delivered>.<sseq>.<cseq>.<tm>.<pending>
//
// New subject would be:
// $JS.ACK.<domain>.<account hash>.<stream>.<consumer>.<delivered>.<sseq>.<cseq>.<tm>.<pending>.<a token with a random value>
//
// v1 has 9 tokens, v2 has 12, but we must not be strict on the 12th since
// it may be removed in the future. Also, the library has no use for it.
// The point is that a v2 ACK subject is valid if it has at least 11 tokens.
//
tokensLen := len(tokens)
// If lower than 9 or more than 9 but less than 11, report an error
if tokensLen < v1TokenCounts || (tokensLen > v1TokenCounts && tokensLen < v2TokenCounts-1) {
return nil, ErrInvalidSubjectFormat
}
if tokens[0] != "$JS" || tokens[1] != "ACK" {
return nil, fmt.Errorf("%w: subject should start with $JS.ACK", ErrInvalidSubjectFormat)
}
// For v1 style, we insert 2 empty tokens (domain and hash) so that the
// rest of the library references known fields at a constant location.
if tokensLen == v1TokenCounts {
// Extend the array (we know the backend is big enough)
tokens = append(tokens[:AckDomainTokenPos+2], tokens[AckDomainTokenPos:]...)
// Clear the domain and hash tokens
tokens[AckDomainTokenPos], tokens[AckAccHashTokenPos] = "", ""
} else if tokens[AckDomainTokenPos] == "_" {
// If domain is "_", replace with empty value.
tokens[AckDomainTokenPos] = ""
}
return tokens, nil
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/internal/syncx/map.go
Generated Vendored
+73
View File
@@ -0,0 +1,73 @@
// Copyright 2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package syncx
import "sync"
// Map is a type-safe wrapper around sync.Map.
// It is safe for concurrent use.
// The zero value of Map is an empty map ready to use.
type Map[K comparable, V any] struct {
m sync.Map
}
func (m *Map[K, V]) Load(key K) (V, bool) {
v, ok := m.m.Load(key)
if !ok {
var empty V
return empty, false
}
return v.(V), true
}
func (m *Map[K, V]) Store(key K, value V) {
m.m.Store(key, value)
}
func (m *Map[K, V]) Delete(key K) {
m.m.Delete(key)
}
func (m *Map[K, V]) Range(f func(key K, value V) bool) {
m.m.Range(func(key, value any) bool {
return f(key.(K), value.(V))
})
}
func (m *Map[K, V]) LoadOrStore(key K, value V) (V, bool) {
v, loaded := m.m.LoadOrStore(key, value)
return v.(V), loaded
}
func (m *Map[K, V]) LoadAndDelete(key K) (V, bool) {
v, ok := m.m.LoadAndDelete(key)
if !ok {
var empty V
return empty, false
}
return v.(V), true
}
func (m *Map[K, V]) CompareAndSwap(key K, old, new V) bool {
return m.m.CompareAndSwap(key, old, new)
}
func (m *Map[K, V]) CompareAndDelete(key K, value V) bool {
return m.m.CompareAndDelete(key, value)
}
func (m *Map[K, V]) Swap(key K, value V) (V, bool) {
previous, loaded := m.m.Swap(key, value)
return previous.(V), loaded
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/MIGRATION.md
Generated Vendored
+749
View File
@@ -0,0 +1,749 @@
# Migrating from Legacy JetStream API to `jetstream` Package
This guide helps you migrate from the legacy JetStream API in the `nats` package
(`nats.JetStreamContext`) to the new `jetstream` package
(`github.com/nats-io/nats.go/jetstream`).
- [Why Migrate?](#why-migrate)
- [Getting Started](#getting-started)
- [Stream Management](#stream-management)
- [Consumer Management](#consumer-management)
- [Publishing](#publishing)
- [Consuming Messages](#consuming-messages)
- [Replacing js.Subscribe()](#replacing-jssubscribe)
- [Replacing js.PullSubscribe()](#replacing-jspullsubscribe)
- [Ordered Consumers](#ordered-consumers)
- [Push Consumers](#push-consumers)
- [Subscription Options Mapping](#subscription-options-mapping)
- [Error Handling in Consume/Messages](#error-handling-in-consumemessages)
- [Message Acknowledgement](#message-acknowledgement)
- [KeyValue Store](#keyvalue-store)
- [Object Store](#object-store)
## Why Migrate?
The legacy JetStream API (`nats.JetStreamContext`) is deprecated. The `jetstream`
package provides a cleaner, more predictable API with several key improvements:
- **Explicit resource management.** Streams and consumers are created and managed
explicitly. The legacy `js.Subscribe()` implicitly created consumers behind
the scenes, leading to surprising behavior.
- **Pull consumers as the default.** Pull consumers with `Consume()` and
`Messages()` provide the same continuous message delivery as the legacy push-based
`Subscribe()`, but with better flow control and no slow consumer issues.
- **`context.Context` throughout.** All API calls accept `context.Context` for
timeout and cancellation, replacing the mix of `MaxWait`, `AckWait`, and
`Context()` options.
- **Clear interface separation.** Instead of one large `JetStreamContext` interface,
functionality is split across focused interfaces: `JetStream`, `Stream` and
`Consumer`.
## Getting Started
The core NATS connection remains unchanged. Only the JetStream initialization
differs:
```go
import (
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
nc, _ := nats.Connect(nats.DefaultURL)
```
**Legacy:**
```go
js, _ := nc.JetStream()
// With domain
js, _ := nc.JetStream(nats.Domain("hub"))
// With custom API prefix
js, _ := nc.JetStream(nats.APIPrefix("myprefix"))
```
**New:**
```go
js, _ := jetstream.New(nc)
// With domain
js, _ := jetstream.NewWithDomain(nc, "hub")
// With custom API prefix
js, _ := jetstream.NewWithAPIPrefix(nc, "myprefix")
```
### Initialization Options
| Legacy | New |
|-----------------------------------|--------------------------------------------|
| `nats.Domain(domain)` | `jetstream.NewWithDomain(nc, domain)` |
| `nats.APIPrefix(prefix)` | `jetstream.NewWithAPIPrefix(nc, prefix)` |
| `nats.PublishAsyncMaxPending(n)` | `jetstream.WithPublishAsyncMaxPending(n)` |
| `nats.PublishAsyncErrHandler(cb)` | `jetstream.WithPublishAsyncErrHandler(cb)` |
## Stream Management
`StreamConfig` is essentially the same struct — it just lives in the `jetstream`
package now. The new API takes `StreamConfig` by value (not pointer) and
management methods return a `Stream` handle instead of `*StreamInfo`.
| Legacy | New | Notes |
|---------------------------------|-------------------------------------|-------------------------------------------------------------------|
| `js.AddStream(cfg)` | `js.CreateStream(ctx, cfg)` | Also: `CreateOrUpdateStream()` |
| `js.UpdateStream(cfg)` | `js.UpdateStream(ctx, cfg)` | |
| `js.DeleteStream(name)` | `js.DeleteStream(ctx, name)` | |
| `js.StreamInfo(name)` | `s.Info(ctx)` / `s.CachedInfo()` | Get stream handle first via `js.Stream(ctx, name)` |
| `js.PurgeStream(name, opts...)` | `s.Purge(ctx, opts...)` | Options: `WithPurgeSubject`, `WithPurgeSequence`, `WithPurgeKeep` |
| `js.GetMsg(name, seq)` | `s.GetMsg(ctx, seq)` | |
| `js.GetLastMsg(name, subj)` | `s.GetLastMsgForSubject(ctx, subj)` | |
| `js.DeleteMsg(name, seq)` | `s.DeleteMsg(ctx, seq)` | Also: `s.SecureDeleteMsg()` |
| `js.Streams()` | `js.ListStreams(ctx)` | Returns lister with `.Info()` channel and `.Err()` |
| `js.StreamNames()` | `js.StreamNames(ctx)` | Returns lister with `.Name()` channel and `.Err()` |
The key architectural difference is that stream-specific operations (purge, get/delete
messages) now live on the `Stream` interface instead of the top-level context. Get
a stream handle first, then operate on it:
```go
s, _ := js.Stream(ctx, "ORDERS")
s.Purge(ctx)
msg, _ := s.GetMsg(ctx, 100)
```
## Consumer Management
The biggest conceptual change: in the legacy API, `js.Subscribe()` would
implicitly create consumers. In the new API, consumer creation is always explicit
and separate from message consumption.
| Legacy | New | Notes |
|------------------------------------------|----------------------------------------|----------------------------------------------------------------|
| `js.AddConsumer(stream, cfg)` | `js.CreateConsumer(ctx, stream, cfg)` | Also: `CreateOrUpdateConsumer()`, `UpdateConsumer()` |
| `js.Subscribe(subj, handler)` (implicit) | No equivalent | Must create consumer explicitly first |
| `js.ConsumerInfo(stream, name)` | `cons.Info(ctx)` / `cons.CachedInfo()` | Get consumer handle first via `js.Consumer(ctx, stream, name)` |
| `js.DeleteConsumer(stream, name)` | `js.DeleteConsumer(ctx, stream, name)` | |
| `js.Consumers(stream)` | `s.ListConsumers(ctx)` | Returns lister with `.Info()` channel and `.Err()` |
| `js.ConsumerNames(stream)` | `s.ConsumerNames(ctx)` | Returns lister with `.Name()` channel and `.Err()` |
Consumer management is available at two levels:
- On `JetStream` — requires stream name as parameter (e.g. `js.CreateConsumer(ctx, "ORDERS", cfg)`), bypassing the need to fetch a stream
- On `Stream` — no stream name needed (e.g. `s.CreateConsumer(ctx, cfg)`)
The new API provides three creation methods:
- `CreateConsumer` — fails if the consumer already exists with different config
- `UpdateConsumer` - fails if the consumer does not exist
- `CreateOrUpdateConsumer` — creates or updates as needed
**Additional notes on consumer behavior:**
- The default ack policy changed between the APIs. In the legacy API,
`AddConsumer()` defaulted to `AckNone`. In the new API, the default is
`AckExplicit`.
- In the legacy API, `sub.Unsubscribe()` on an implicitly created
consumer would automatically delete that consumer on the server. The new API
does not perform any automatic cleanup - consumers must be deleted explicitly
via `DeleteConsumer()`, or via `InactiveThreshold` on the consumer
config to let the server remove it automatically after a period of inactivity.
Push consumers use separate methods: `CreatePushConsumer`, `CreateOrUpdatePushConsumer`,
`UpdatePushConsumer`, and `PushConsumer` (for getting a handle).
```go
s, _ := js.Stream(ctx, "ORDERS")
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
})
```
## Publishing
Publishing is largely the same, with the addition of `context.Context` for
synchronous operations.
### Synchronous Publish
**Legacy:**
```go
ack, _ := js.Publish("ORDERS.new", []byte("hello"))
ack, _ = js.PublishMsg(&nats.Msg{
Subject: "ORDERS.new",
Data: []byte("hello"),
})
```
**New:**
```go
ack, _ := js.Publish(ctx, "ORDERS.new", []byte("hello"))
ack, _ = js.PublishMsg(ctx, &nats.Msg{
Subject: "ORDERS.new",
Data: []byte("hello"),
})
```
### Async Publish
**Legacy:**
```go
ackF, _ := js.PublishAsync("ORDERS.new", []byte("hello"))
select {
case ack := <-ackF.Ok():
fmt.Println(ack.Sequence)
case err := <-ackF.Err():
fmt.Println(err)
}
// Wait for all pending acks
<-js.PublishAsyncComplete()
```
**New:**
```go
// Async publish does not take context (returns immediately)
ackF, _ := js.PublishAsync("ORDERS.new", []byte("hello"))
select {
case ack := <-ackF.Ok():
fmt.Println(ack.Sequence)
case err := <-ackF.Err():
fmt.Println(err)
}
<-js.PublishAsyncComplete()
```
### Publish Options
| Legacy | New |
|------------------------------------------|---------------------------------------------------|
| `nats.MsgId(id)` | `jetstream.WithMsgID(id)` |
| `nats.ExpectStream(name)` | `jetstream.WithExpectStream(name)` |
| `nats.ExpectLastSequence(seq)` | `jetstream.WithExpectLastSequence(seq)` |
| `nats.ExpectLastSequencePerSubject(seq)` | `jetstream.WithExpectLastSequencePerSubject(seq)` |
| `nats.ExpectLastMsgId(id)` | `jetstream.WithExpectLastMsgID(id)` |
| `nats.RetryWait(dur)` | `jetstream.WithRetryWait(dur)` |
| `nats.RetryAttempts(n)` | `jetstream.WithRetryAttempts(n)` |
| `nats.StallWait(dur)` | `jetstream.WithStallWait(dur)` |
## Consuming Messages
This is the most significant area of change. The legacy API offered many
subscription flavors (`Subscribe`, `SubscribeSync`, `QueueSubscribe`,
`ChanSubscribe`, `PullSubscribe`) that blurred the line between consumer
creation, stream lookup and message consumption. The new API separates these
concerns: first create a consumer, then choose how to receive messages.
With the exception of PullSubscribe, all legacy subscription flavors utilized push consumers under the hood. The new API recommends pull consumers for all use cases, as they provide better flow control and no risk of slow consumer issues. Pull-based consumption is available via `Consume()` and `Messages()`, which maintain persistent pull subscriptions with pre-buffering for efficient continuous delivery. Push consumers are still supported for users who prefer that model, but pull consumers are the recommended default.
### Replacing `js.Subscribe()`
The legacy `js.Subscribe()` created a push consumer behind the scenes (unless
explicitly specified otherwise via `nats.Bind()` or `nats.Durable()`) and
delivered messages either via a callback. In the new API, the recommended
replacement is a **pull consumer** with `Consume()` or `Messages()`. These
provide the same continuous delivery with better flow control.
#### Legacy: callback subscription
```go
sub, _ := js.Subscribe("ORDERS.*", func(msg *nats.Msg) {
fmt.Printf("Received: %s\n", string(msg.Data))
msg.Ack()
}, nats.Durable("processor"), nats.ManualAck)
defer sub.Unsubscribe()
```
#### New: callback with `Consume()`
`Consume()` is the closest equivalent to `js.Subscribe()` — it delivers messages
to a callback function continuously.
```go
s, _ := js.Stream(ctx, "ORDERS")
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
FilterSubject: "ORDERS.*",
})
cc, _ := cons.Consume(func(msg jetstream.Msg) {
fmt.Printf("Received: %s\n", string(msg.Data()))
msg.Ack()
})
defer cc.Stop()
```
> Note: `ManualAck()` is not needed — messages are never auto-acknowledged in
> the new API.
#### New: iterator with `Messages()`
`Messages()` provides an iterator-based approach, useful when you want explicit
control over when the next message is fetched.
```go
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
FilterSubject: "ORDERS.*",
})
iter, _ := cons.Messages()
for {
msg, err := iter.Next()
if err != nil {
// handle error
}
fmt.Printf("Received: %s\n", string(msg.Data()))
msg.Ack()
}
// Call iter.Stop() when done
```
Both `Consume()` and `Messages()` maintain overlapping pull requests to the
server, providing efficient continuous delivery without gaps.
#### Legacy: synchronous subscription
```go
sub, _ := js.SubscribeSync("ORDERS.*", nats.Durable("processor"))
msg, _ := sub.NextMsg(time.Second)
```
**New:** Use `Messages()` and call `Next()`:
```go
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
FilterSubject: "ORDERS.*",
})
iter, _ := cons.Messages()
msg, _ := iter.Next()
```
#### Legacy: queue subscription
```go
// Multiple instances share work via a queue group
sub, _ := js.QueueSubscribe("ORDERS.*", "workers", handler,
nats.Durable("processor"))
```
**New with pull consumers:** With pull consumers, there is no need for an
explicit queue group. Multiple application instances (or goroutines) calling
`Consume()` or `Messages()` on the same durable consumer will naturally
distribute messages among themselves — the server tracks pending acknowledgements
and avoids delivering the same message to multiple consumers:
```go
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
})
cc, _ := cons.Consume(handler)
defer cc.Stop()
```
**New with push consumers:** If you need push-based queue semantics, set
`DeliverGroup` on a push consumer — this is the direct equivalent of the legacy
queue group:
```go
cons, _ := s.CreateOrUpdatePushConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
DeliverSubject: "deliver.orders",
DeliverGroup: "workers",
})
cc, _ := cons.Consume(handler)
defer cc.Stop()
```
> **Note:** Push consumers with `DeliverGroup` cannot be flow controlled. If you
> experience slow consumer issues, consider using pull-based consumers instead —
> multiple instances on the same durable consumer achieve the same work
> distribution without the slow consumer risk.
#### Legacy: channel subscription
```go
ch := make(chan *nats.Msg, 64)
sub, _ := js.ChanSubscribe("ORDERS.*", ch, nats.Durable("processor"))
for msg := range ch {
msg.Ack()
}
```
**New:** There is no direct channel-based equivalent. Use `Consume()` or
`Messages()` instead.
### Replacing `js.PullSubscribe()`
The legacy pull subscription required creating a subscription and then calling
`Fetch()` in a loop.
#### Legacy: pull subscribe + fetch loop
```go
sub, _ := js.PullSubscribe("ORDERS.*", "processor")
for {
msgs, _ := sub.Fetch(10, nats.MaxWait(5*time.Second))
for _, msg := range msgs {
fmt.Printf("Received: %s\n", string(msg.Data))
msg.Ack()
}
}
```
**New with `Fetch()`/`FetchNoWait()` (one-off batch):**
If you specifically need one-off batch fetching, `Fetch()` is available directly
on the consumer — no separate subscription step:
```go
cons, _ := s.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
FilterSubject: "ORDERS.*",
})
// non-blocking, returns a `FetchResult` that provides messages and error
msgs, _ := cons.Fetch(10, jetstream.FetchMaxWait(5*time.Second))
for msg := range msgs.Messages() {
fmt.Printf("Received: %s\n", string(msg.Data()))
msg.Ack()
}
if msgs.Error() != nil {
// handle error
}
```
> **Warning:** `Fetch()`, `FetchNoWait()`, and `FetchBytes()` are one-off,
> single pull requests. They do not perform pre-buffering optimizations. For
> continuous message processing, always prefer `Consume()` or `Messages()`.
> When using `FetchBytes()`, the requested byte size must stay under the
> client's max pending bytes limit (64MB by default), otherwise it will trigger
> slow consumer errors on the underlying subscription.
### Ordered Consumers
Ordered consumers provide strictly ordered, gap-free message delivery. The library
automatically recreates the underlying consumer on sequence gaps or heartbeat
failures.
**Legacy:**
```go
sub, _ := js.Subscribe("ORDERS.*", handler, nats.OrderedConsumer())
```
**New:**
```go
cons, _ := js.OrderedConsumer(ctx, "ORDERS", jetstream.OrderedConsumerConfig{
FilterSubjects: []string{"ORDERS.*"},
})
// Use the same consumption methods as regular consumers
cc, _ := cons.Consume(func(msg jetstream.Msg) {
fmt.Printf("Received: %s\n", string(msg.Data()))
})
defer cc.Stop()
```
### Push Consumers
Pull consumers are recommended for most use cases, but push consumers are also
supported. Push consumers require `DeliverSubject` in their config and only
support `Consume()` (not `Fetch()` or `Messages()`).
**Legacy:**
```go
sub, _ := js.Subscribe("ORDERS.*", handler,
nats.Durable("processor"),
nats.DeliverSubject("deliver.orders"),
nats.IdleHeartbeat(30*time.Second),
)
```
**New:**
```go
cons, _ := s.CreateOrUpdatePushConsumer(ctx, jetstream.ConsumerConfig{
Durable: "processor",
FilterSubject: "ORDERS.*",
DeliverSubject: "deliver.orders",
IdleHeartbeat: 30 * time.Second,
})
cc, _ := cons.Consume(func(msg jetstream.Msg) {
fmt.Printf("Received: %s\n", string(msg.Data()))
msg.Ack()
})
defer cc.Stop()
```
### Subscription Options Mapping
Most legacy `SubOpt` options map directly to `ConsumerConfig` fields. Since
consumer creation is explicit, these are set at creation time rather than passed
as subscription options.
| Legacy SubOpt | New ConsumerConfig field |
|-------------------------------------|---------------------------------------------------------------------------|
| `nats.Durable("name")` | `Durable: "name"` |
| `nats.ConsumerName("name")` | `Name: "name"` |
| `nats.Description("desc")` | `Description: "desc"` |
| `nats.DeliverAll()` | `DeliverPolicy: jetstream.DeliverAllPolicy` |
| `nats.DeliverLast()` | `DeliverPolicy: jetstream.DeliverLastPolicy` |
| `nats.DeliverLastPerSubject()` | `DeliverPolicy: jetstream.DeliverLastPerSubjectPolicy` |
| `nats.DeliverNew()` | `DeliverPolicy: jetstream.DeliverNewPolicy` |
| `nats.StartSequence(seq)` | `DeliverPolicy: jetstream.DeliverByStartSequencePolicy, OptStartSeq: seq` |
| `nats.StartTime(t)` | `DeliverPolicy: jetstream.DeliverByStartTimePolicy, OptStartTime: &t` |
| `nats.AckExplicit()` | `AckPolicy: jetstream.AckExplicitPolicy` |
| `nats.AckAll()` | `AckPolicy: jetstream.AckAllPolicy` |
| `nats.AckNone()` | `AckPolicy: jetstream.AckNonePolicy` |
| `nats.ManualAck()` | Not needed (messages are never auto-acked) |
| `nats.MaxDeliver(n)` | `MaxDeliver: n` |
| `nats.MaxAckPending(n)` | `MaxAckPending: n` |
| `nats.BackOff(durations)` | `BackOff: durations` |
| `nats.ReplayOriginal()` | `ReplayPolicy: jetstream.ReplayOriginalPolicy` |
| `nats.ReplayInstant()` | `ReplayPolicy: jetstream.ReplayInstantPolicy` |
| `nats.RateLimit(bps)` | `RateLimit: bps` |
| `nats.HeadersOnly()` | `HeadersOnly: true` |
| `nats.InactiveThreshold(dur)` | `InactiveThreshold: dur` |
| `nats.ConsumerFilterSubjects(s...)` | `FilterSubjects: s` |
| `nats.ConsumerReplicas(n)` | `Replicas: n` |
| `nats.ConsumerMemoryStorage()` | `MemoryStorage: true` |
The following options have no direct equivalent — use the consumer handle
directly instead:
| Legacy SubOpt | New equivalent |
|-------------------------------|---------------------------------------------------------------------|
| `nats.Bind(stream, consumer)` | `js.Consumer(ctx, stream, consumer)` or `s.Consumer(ctx, consumer)` |
| `nats.BindStream(stream)` | Use `js.Stream(ctx, stream)` to get a stream handle |
| `nats.OrderedConsumer()` | `js.OrderedConsumer(ctx, stream, cfg)` |
### Consume/Messages Options
`Consume()` and `Messages()` accept options that control pull request behavior:
| Option | Description |
|----------------------------|--------------------------------------------------------------|
| `PullMaxMessages(n)` | Max messages buffered (default: 500) |
| `PullMaxBytes(n)` | Max bytes buffered (mutually exclusive with PullMaxMessages) |
| `PullExpiry(dur)` | Pull request timeout (default: 30s) |
| `PullHeartbeat(dur)` | Idle heartbeat interval |
| `PullThresholdMessages(n)` | Refill threshold (default: 50% of max) |
| `PullThresholdBytes(n)` | Byte-based refill threshold |
| `StopAfter(n)` | Auto-stop after N messages |
| `ConsumeErrHandler(fn)` | Custom error handler |
### Error Handling in Consume/Messages
Both `Consume()` and `Messages()` handle server-sent status messages internally.
Some errors are terminal (stop consumption), while others are recoverable
(consumption continues).
**Terminal errors** — consumption stops automatically:
- `ErrConsumerDeleted` — the consumer was deleted on the server
- `ErrBadRequest` — invalid request (e.g. misconfigured consumer)
- Connection closed — for `Consume()` this surfaces as `ErrConnectionClosed`;
for `Messages()`, `Next()` returns `ErrMsgIteratorClosed`
**Recoverable errors** — reported via error handler, consumption continues:
- `ErrNoHeartbeat` — missed idle heartbeats from server; a new pull request
is issued automatically
- `ErrConsumerLeadershipChanged` — consumer moved to a different server in the
cluster; pending counts are reset
- `nats.ErrNoResponders` — no JetStream service available (temporary)
#### Error handling with `Consume()`
Use `ConsumeErrHandler` to be notified about both terminal and recoverable errors:
```go
cc, _ := cons.Consume(func(msg jetstream.Msg) {
msg.Ack()
}, jetstream.ConsumeErrHandler(func(cc jetstream.ConsumeContext, err error) {
if errors.Is(err, jetstream.ErrConsumerDeleted) ||
errors.Is(err, jetstream.ErrBadRequest) {
log.Fatalf("terminal consumer error: %v", err)
}
log.Printf("recoverable consumer error: %v", err)
}))
defer cc.Stop()
```
#### Error handling with `Messages()`
With `Messages()`, terminal errors are returned directly by `Next()`. By default,
`ErrNoHeartbeat` is also returned by `Next()` (controlled by
`WithMessagesErrOnMissingHeartbeat`), but it is not terminal — you can continue
calling `Next()`:
```go
iter, _ := cons.Messages()
for {
msg, err := iter.Next()
if err != nil {
if errors.Is(err, jetstream.ErrMsgIteratorClosed) {
// iterator was stopped (either explicitly or due to connection close)
break
}
if errors.Is(err, jetstream.ErrNoHeartbeat) {
// recoverable — new pull request is issued, keep going
log.Println("missed heartbeat, re-pulling")
continue
}
// ErrConsumerDeleted, ErrBadRequest are terminal
log.Fatalf("terminal error: %v", err)
}
msg.Ack()
}
```
## Message Acknowledgement
Ack methods are similar, with minor naming changes. The main difference is that
message fields are accessed via methods instead of struct fields.
| Legacy | New |
|-------------------------|------------------------------|
| `msg.Ack()` | Unchanged |
| `msg.AckSync()` | `msg.DoubleAck(ctx)` |
| `msg.Nak()` | Unchanged |
| `msg.NakWithDelay(dur)` | Unchanged |
| `msg.InProgress()` | Unchanged |
| `msg.Term()` | Unchanged |
| N/A | `msg.TermWithReason(reason)` |
| `msg.Metadata()` | Unchanged |
### Accessing Message Data
**Legacy:** Direct struct fields on `*nats.Msg`:
```go
fmt.Println(string(msg.Data))
fmt.Println(msg.Subject)
fmt.Println(msg.Header.Get("key"))
```
**New:** Methods on `jetstream.Msg` interface:
```go
fmt.Println(string(msg.Data()))
fmt.Println(msg.Subject())
fmt.Println(msg.Headers().Get("key"))
```
## KeyValue Store
The KV API is nearly identical. The main changes are:
1. All methods take `context.Context` as the first parameter
2. New `CreateOrUpdateKeyValue()` and `UpdateKeyValue()` methods
3. Types live in the `jetstream` package
**Legacy:**
```go
js, _ := nc.JetStream()
kv, _ := js.CreateKeyValue(&nats.KeyValueConfig{
Bucket: "profiles",
})
kv.Put("sue.color", []byte("blue"))
entry, _ := kv.Get("sue.color")
fmt.Println(string(entry.Value()))
watcher, _ := kv.Watch("sue.*")
defer watcher.Stop()
```
**New:**
```go
js, _ := jetstream.New(nc)
kv, _ := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: "profiles",
})
kv.Put(ctx, "sue.color", []byte("blue"))
entry, _ := kv.Get(ctx, "sue.color")
fmt.Println(string(entry.Value()))
watcher, _ := kv.Watch(ctx, "sue.*")
defer watcher.Stop()
```
### KV Management Methods
| Legacy | New |
|-----------------------------|---------------------------------------|
| `js.KeyValue(bucket)` | `js.KeyValue(ctx, bucket)` |
| `js.CreateKeyValue(cfg)` | `js.CreateKeyValue(ctx, cfg)` |
| N/A | `js.UpdateKeyValue(ctx, cfg)` |
| N/A | `js.CreateOrUpdateKeyValue(ctx, cfg)` |
| `js.DeleteKeyValue(bucket)` | `js.DeleteKeyValue(ctx, bucket)` |
| `js.KeyValueStoreNames()` | `js.KeyValueStoreNames(ctx)` |
| `js.KeyValueStores()` | `js.KeyValueStores(ctx)` |
## Object Store
Same pattern as KV — all methods gain `context.Context`, types move to `jetstream`
package.
**Legacy:**
```go
js, _ := nc.JetStream()
os, _ := js.CreateObjectStore(&nats.ObjectStoreConfig{
Bucket: "configs",
})
os.PutString("config-1", "data")
result, _ := os.Get("config-1")
data, _ := io.ReadAll(result)
```
**New:**
```go
js, _ := jetstream.New(nc)
os, _ := js.CreateObjectStore(ctx, jetstream.ObjectStoreConfig{
Bucket: "configs",
})
os.PutString(ctx, "config-1", "data")
result, _ := os.Get(ctx, "config-1")
data, _ := io.ReadAll(result)
```
### Object Store Management Methods
| Legacy | New |
|--------------------------------|------------------------------------------|
| `js.ObjectStore(bucket)` | `js.ObjectStore(ctx, bucket)` |
| `js.CreateObjectStore(cfg)` | `js.CreateObjectStore(ctx, cfg)` |
| N/A | `js.UpdateObjectStore(ctx, cfg)` |
| N/A | `js.CreateOrUpdateObjectStore(ctx, cfg)` |
| `js.DeleteObjectStore(bucket)` | `js.DeleteObjectStore(ctx, bucket)` |
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/README.md
Generated Vendored
+1092
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/api.go
Generated Vendored
+161
View File
@@ -0,0 +1,161 @@
// Copyright 2022-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"encoding/json"
"strings"
)
type (
apiResponse struct {
Type string `json:"type"`
Error *APIError `json:"error,omitempty"`
}
// apiPaged includes variables used to create paged responses from the JSON API
apiPaged struct {
Total int `json:"total"`
Offset int `json:"offset"`
Limit int `json:"limit"`
}
)
// Request API subjects for JetStream.
const (
// DefaultAPIPrefix is the default prefix for the JetStream API.
DefaultAPIPrefix = "$JS.API."
// jsDomainT is used to create JetStream API prefix by specifying only Domain
jsDomainT = "$JS.%s.API."
// jsExtDomainT is used to create a StreamSource External APIPrefix
jsExtDomainT = "$JS.%s.API"
// apiAccountInfo is for obtaining general information about JetStream.
apiAccountInfo = "INFO"
// apiConsumerCreateT is used to create consumers.
apiConsumerCreateT = "CONSUMER.CREATE.%s.%s"
// apiConsumerCreateWithFilterSubjectT is used to create consumers with a filter subject.
// it accepts stream name, consumer name and filter subject
apiConsumerCreateWithFilterSubjectT = "CONSUMER.CREATE.%s.%s.%s"
// apiConsumerInfoT is used to retrieve consumer information.
apiConsumerInfoT = "CONSUMER.INFO.%s.%s"
// apiRequestNextT is the prefix for the request next message(s) for a consumer in worker/pull mode.
apiRequestNextT = "CONSUMER.MSG.NEXT.%s.%s"
// apiConsumerDeleteT is used to delete consumers.
apiConsumerDeleteT = "CONSUMER.DELETE.%s.%s"
// apiConsumerPauseT is used to pause a consumer.
apiConsumerPauseT = "CONSUMER.PAUSE.%s.%s"
// apiConsumerListT is used to return all detailed consumer information
apiConsumerListT = "CONSUMER.LIST.%s"
// apiConsumerNamesT is used to return a list with all consumer names for the stream.
apiConsumerNamesT = "CONSUMER.NAMES.%s"
// apiStreams can lookup a stream by subject.
apiStreams = "STREAM.NAMES"
// apiStreamCreateT is the endpoint to create new streams.
apiStreamCreateT = "STREAM.CREATE.%s"
// apiStreamInfoT is the endpoint to get information on a stream.
apiStreamInfoT = "STREAM.INFO.%s"
// apiStreamUpdateT is the endpoint to update existing streams.
apiStreamUpdateT = "STREAM.UPDATE.%s"
// apiStreamDeleteT is the endpoint to delete streams.
apiStreamDeleteT = "STREAM.DELETE.%s"
// apiStreamPurgeT is the endpoint to purge streams.
apiStreamPurgeT = "STREAM.PURGE.%s"
// apiStreamListT is the endpoint that will return all detailed stream information
apiStreamListT = "STREAM.LIST"
// apiMsgGetT is the endpoint to get a message.
apiMsgGetT = "STREAM.MSG.GET.%s"
// apiDirectMsgGetT is the endpoint to perform a direct get of a message.
apiDirectMsgGetT = "DIRECT.GET.%s"
// apiDirectMsgGetLastBySubjectT is the endpoint to perform a direct get of a message by subject.
apiDirectMsgGetLastBySubjectT = "DIRECT.GET.%s.%s"
// apiMsgDeleteT is the endpoint to remove a message.
apiMsgDeleteT = "STREAM.MSG.DELETE.%s"
// apiConsumerUnpinT is the endpoint to unpin a consumer.
apiConsumerUnpinT = "CONSUMER.UNPIN.%s.%s"
// apiConsumerResetT is the endpoint to reset a consumer's delivery state.
apiConsumerResetT = "CONSUMER.RESET.%s.%s"
)
func (js *jetStream) apiRequestJSON(ctx context.Context, subject string, resp any, data ...[]byte) (*jetStreamMsg, error) {
jsMsg, err := js.apiRequest(ctx, subject, data...)
if err != nil {
return nil, err
}
if err := json.Unmarshal(jsMsg.Data(), resp); err != nil {
return nil, err
}
return jsMsg, nil
}
// a RequestWithContext with tracing via TraceCB
func (js *jetStream) apiRequest(ctx context.Context, subj string, data ...[]byte) (*jetStreamMsg, error) {
subj = js.apiSubject(subj)
var req []byte
if len(data) > 0 {
req = data[0]
}
if js.opts.ClientTrace != nil {
ctrace := js.opts.ClientTrace
if ctrace.RequestSent != nil {
ctrace.RequestSent(subj, req)
}
}
resp, err := js.conn.RequestWithContext(ctx, subj, req)
if err != nil {
return nil, err
}
if js.opts.ClientTrace != nil {
ctrace := js.opts.ClientTrace
if ctrace.ResponseReceived != nil {
ctrace.ResponseReceived(subj, resp.Data, resp.Header)
}
}
return js.toJSMsg(resp), nil
}
func (js *jetStream) apiSubject(subj string) string {
if js.opts.apiPrefix == "" {
return subj
}
var b strings.Builder
b.WriteString(js.opts.apiPrefix)
b.WriteString(subj)
return b.String()
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/consumer.go
Generated Vendored
+572
View File
@@ -0,0 +1,572 @@
// Copyright 2022-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"crypto/sha256"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/nats-io/nats.go/internal/syncx"
"github.com/nats-io/nuid"
)
type (
// Consumer contains methods for fetching/processing messages from a stream,
// as well as fetching consumer info.
//
// This package provides two implementations of Consumer interface:
//
// - Standard named/ephemeral pull consumers. These consumers are created using
// CreateConsumer method on Stream or JetStream interface. They can be
// explicitly configured (using [ConsumerConfig]) and managed by the user,
// either from this package or externally.
//
// - Ordered consumers. These consumers are created using OrderedConsumer
// method on Stream or JetStream interface. They are managed by the library
// and provide a simple way to consume messages from a stream. Ordered
// consumers are ephemeral in-memory pull consumers and are resilient to
// deletes and restarts. They provide limited configuration options
// using [OrderedConsumerConfig].
//
// Consumer provides methods for optimized continuous consumption of messages
// using Consume and Messages methods, as well as simple one-off messages
// retrieval using Fetch and Next methods.
Consumer interface {
// Fetch is used to retrieve up to a provided number of messages from a
// stream. This method will send a single request and deliver either all
// requested messages unless the timeout is met earlier. Fetch timeout
// defaults to 30 seconds and can be configured using FetchMaxWait
// option.
//
// By default, Fetch uses a 5s idle heartbeat for requests longer than
// 10 seconds. For shorter requests, the idle heartbeat is disabled.
// This can be configured using FetchHeartbeat option. If a client does
// not receive a heartbeat message from a stream for more than 2 times
// the idle heartbeat setting, Fetch will return [ErrNoHeartbeat].
//
// Fetch is non-blocking and returns MessageBatch, exposing a channel
// for delivered messages.
//
// Messages channel is always closed, thus it is safe to range over it
// without additional checks. After the channel is closed,
// MessageBatch.Error() should be checked to see if there was an error
// during message delivery (e.g. missing heartbeat).
//
// NOTE: Fetch has worse performance when used to continuously retrieve
// messages in comparison to Messages or Consume methods, as it does not
// perform any optimizations (e.g. overlapping pull requests) and new
// subscription is created for each execution.
Fetch(batch int, opts ...FetchOpt) (MessageBatch, error)
// FetchBytes is used to retrieve up to a provided number of bytes from
// the stream. This method will send a single request and deliver the
// provided number of bytes unless the timeout is met earlier. FetchBytes
// timeout defaults to 30 seconds and can be configured using
// FetchMaxWait option.
//
// By default, FetchBytes uses a 5s idle heartbeat for requests longer than
// 10 seconds. For shorter requests, the idle heartbeat is disabled.
// This can be configured using FetchHeartbeat option. If a client does
// not receive a heartbeat message from a stream for more than 2 times
// the idle heartbeat setting, FetchBytes will return [ErrNoHeartbeat].
//
// FetchBytes is non-blocking and returns MessageBatch, exposing a channel
// for delivered messages.
//
// Messages channel is always closed, thus it is safe to range over it
// without additional checks. After the channel is closed,
// MessageBatch.Error() should be checked to see if there was an error
// during message delivery (e.g. missing heartbeat).
//
// NOTE: FetchBytes has worse performance when used to continuously
// retrieve messages in comparison to Messages or Consume methods, as it
// does not perform any optimizations (e.g. overlapping pull requests)
// and new subscription is created for each execution.
FetchBytes(maxBytes int, opts ...FetchOpt) (MessageBatch, error)
// FetchNoWait is used to retrieve up to a provided number of messages
// from a stream. Unlike Fetch, FetchNoWait will only deliver messages
// that are currently available in the stream and will not wait for new
// messages to arrive, even if batch size is not met.
//
// FetchNoWait is non-blocking and returns MessageBatch, exposing a
// channel for delivered messages.
//
// Messages channel is always closed, thus it is safe to range over it
// without additional checks. After the channel is closed,
// MessageBatch.Error() should be checked to see if there was an error
// during message delivery (e.g. missing heartbeat).
//
// NOTE: FetchNoWait has worse performance when used to continuously
// retrieve messages in comparison to Messages or Consume methods, as it
// does not perform any optimizations (e.g. overlapping pull requests)
// and new subscription is created for each execution.
FetchNoWait(batch int) (MessageBatch, error)
// Consume will continuously receive messages and handle them
// with the provided callback function. Consume can be configured using
// PullConsumeOpt options:
//
// - Error handling and monitoring can be configured using ConsumeErrHandler
// option, which provides information about errors encountered during
// consumption (both transient and terminal)
// - Consume can be configured to stop after a certain number of
// messages have been received using StopAfter option.
// - Consume can be optimized for throughput or memory usage using
// PullExpiry, PullMaxMessages, PullMaxBytes and PullHeartbeat options.
// Unless there is a specific use case, these options should not be used.
//
// Consume returns a ConsumeContext, which can be used to stop or drain
// the consumer.
Consume(handler MessageHandler, opts ...PullConsumeOpt) (ConsumeContext, error)
// Messages returns MessagesContext, allowing continuous iteration
// over messages in a stream. Messages can be configured using
// PullMessagesOpt options:
//
// - Messages can be optimized for throughput or memory usage using
// PullExpiry, PullMaxMessages, PullMaxBytes and PullHeartbeat options.
// Unless there is a specific use case, these options should not be used.
// - WithMessagesErrOnMissingHeartbeat can be used to enable/disable
// erroring out on MessagesContext.Next when a heartbeat is missing.
// This option is enabled by default.
Messages(opts ...PullMessagesOpt) (MessagesContext, error)
// Next is used to retrieve the next message from the consumer. This
// method will block until the message is retrieved or the timeout
// is reached.
Next(opts ...FetchOpt) (Msg, error)
// Info fetches current ConsumerInfo from the server.
Info(context.Context) (*ConsumerInfo, error)
// CachedInfo returns ConsumerInfo currently cached on this consumer.
// This method does not perform any network requests. The cached
// ConsumerInfo is updated on every call to Info and Update.
CachedInfo() *ConsumerInfo
}
PushConsumer interface {
// Consume will continuously receive messages and handle them
// with the provided callback function. Consume can be configured using
// PushConsumeOpt options:
//
// - Error handling and monitoring can be configured using ConsumeErrHandler.
Consume(handler MessageHandler, opts ...PushConsumeOpt) (ConsumeContext, error)
// Info fetches current ConsumerInfo from the server.
Info(context.Context) (*ConsumerInfo, error)
// CachedInfo returns ConsumerInfo currently cached on this consumer.
CachedInfo() *ConsumerInfo
}
createConsumerRequest struct {
Stream string `json:"stream_name"`
Config *ConsumerConfig `json:"config"`
Action string `json:"action"`
}
)
// Info fetches current ConsumerInfo from the server.
func (p *pullConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
ctx, cancel := p.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
infoSubject := fmt.Sprintf(apiConsumerInfoT, p.stream, p.name)
var resp consumerInfoResponse
if _, err := p.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return nil, ErrConsumerNotFound
}
return nil, resp.Error
}
if resp.Error == nil && resp.ConsumerInfo == nil {
return nil, ErrConsumerNotFound
}
p.info = resp.ConsumerInfo
return resp.ConsumerInfo, nil
}
// CachedInfo returns ConsumerInfo currently cached on this consumer.
// This method does not perform any network requests. The cached
// ConsumerInfo is updated on every call to Info and Update.
func (p *pullConsumer) CachedInfo() *ConsumerInfo {
return p.info
}
// Info fetches current ConsumerInfo from the server.
func (p *pushConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
ctx, cancel := p.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
infoSubject := fmt.Sprintf(apiConsumerInfoT, p.stream, p.name)
var resp consumerInfoResponse
if _, err := p.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return nil, ErrConsumerNotFound
}
return nil, resp.Error
}
if resp.Error == nil && resp.ConsumerInfo == nil {
return nil, ErrConsumerNotFound
}
p.info = resp.ConsumerInfo
return resp.ConsumerInfo, nil
}
// CachedInfo returns ConsumerInfo currently cached on this consumer.
// This method does not perform any network requests. The cached
// ConsumerInfo is updated on every call to Info and Update.
func (p *pushConsumer) CachedInfo() *ConsumerInfo {
return p.info
}
func upsertPullConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (Consumer, error) {
resp, err := upsertConsumer(ctx, js, stream, cfg, action)
if err != nil {
return nil, err
}
return &pullConsumer{
js: js,
stream: stream,
name: resp.Name,
durable: cfg.Durable != "",
info: resp.ConsumerInfo,
subs: syncx.Map[string, *pullSubscription]{},
}, nil
}
func upsertPushConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (PushConsumer, error) {
if cfg.DeliverSubject == "" {
return nil, ErrNotPushConsumer
}
resp, err := upsertConsumer(ctx, js, stream, cfg, action)
if err != nil {
return nil, err
}
return &pushConsumer{
js: js,
stream: stream,
name: resp.Name,
info: resp.ConsumerInfo,
}, nil
}
func upsertConsumer(ctx context.Context, js *jetStream, stream string, cfg ConsumerConfig, action string) (*consumerInfoResponse, error) {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
req := createConsumerRequest{
Stream: stream,
Config: &cfg,
Action: action,
}
reqJSON, err := json.Marshal(req)
if err != nil {
return nil, err
}
consumerName := cfg.Name
if consumerName == "" {
if cfg.Durable != "" {
consumerName = cfg.Durable
} else {
consumerName = generateConsName()
}
}
if err := validateConsumerName(consumerName); err != nil {
return nil, err
}
var ccSubj string
if cfg.FilterSubject != "" && len(cfg.FilterSubjects) == 0 {
if err := validateSubject(cfg.FilterSubject); err != nil {
return nil, err
}
ccSubj = fmt.Sprintf(apiConsumerCreateWithFilterSubjectT, stream, consumerName, cfg.FilterSubject)
} else {
ccSubj = fmt.Sprintf(apiConsumerCreateT, stream, consumerName)
}
var resp consumerInfoResponse
if _, err := js.apiRequestJSON(ctx, ccSubj, &resp, reqJSON); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeStreamNotFound {
return nil, ErrStreamNotFound
}
if resp.Error.ErrorCode == JSErrCodeMaximumConsumersLimit {
return nil, ErrMaximumConsumersLimit
}
return nil, resp.Error
}
if resp.Error == nil && resp.ConsumerInfo == nil {
return nil, ErrConsumerCreationResponseEmpty
}
// check whether multiple filter subjects (if used) are reflected in the returned ConsumerInfo
if len(cfg.FilterSubjects) != 0 && len(resp.Config.FilterSubjects) == 0 {
return nil, ErrConsumerMultipleFilterSubjectsNotSupported
}
return &resp, nil
}
const (
consumerActionCreate = "create"
consumerActionUpdate = "update"
consumerActionCreateOrUpdate = ""
)
func generateConsName() string {
name := nuid.Next()
sha := sha256.New()
sha.Write([]byte(name))
b := sha.Sum(nil)
for i := 0; i < 8; i++ {
b[i] = rdigits[int(b[i]%base)]
}
return string(b[:8])
}
func getConsumer(ctx context.Context, js *jetStream, stream, name string) (Consumer, error) {
info, err := fetchConsumerInfo(ctx, js, stream, name)
if err != nil {
return nil, err
}
if info.Config.DeliverSubject != "" {
return nil, ErrNotPullConsumer
}
cons := &pullConsumer{
js: js,
stream: stream,
name: name,
durable: info.Config.Durable != "",
info: info,
subs: syncx.Map[string, *pullSubscription]{},
}
return cons, nil
}
func getPushConsumer(ctx context.Context, js *jetStream, stream, name string) (PushConsumer, error) {
info, err := fetchConsumerInfo(ctx, js, stream, name)
if err != nil {
return nil, err
}
if info.Config.DeliverSubject == "" {
return nil, ErrNotPushConsumer
}
cons := &pushConsumer{
js: js,
stream: stream,
name: name,
info: info,
}
return cons, nil
}
func fetchConsumerInfo(ctx context.Context, js *jetStream, stream, name string) (*ConsumerInfo, error) {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
if err := validateConsumerName(name); err != nil {
return nil, err
}
infoSubject := fmt.Sprintf(apiConsumerInfoT, stream, name)
var resp consumerInfoResponse
if _, err := js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return nil, ErrConsumerNotFound
}
return nil, resp.Error
}
if resp.Error == nil && resp.ConsumerInfo == nil {
return nil, ErrConsumerNotFound
}
return resp.ConsumerInfo, nil
}
func deleteConsumer(ctx context.Context, js *jetStream, stream, consumer string) error {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
if err := validateConsumerName(consumer); err != nil {
return err
}
deleteSubject := fmt.Sprintf(apiConsumerDeleteT, stream, consumer)
var resp consumerDeleteResponse
if _, err := js.apiRequestJSON(ctx, deleteSubject, &resp); err != nil {
return err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return ErrConsumerNotFound
}
return resp.Error
}
return nil
}
func pauseConsumer(ctx context.Context, js *jetStream, stream, consumer string, pauseUntil *time.Time) (*ConsumerPauseResponse, error) {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
if err := validateConsumerName(consumer); err != nil {
return nil, err
}
subject := fmt.Sprintf(apiConsumerPauseT, stream, consumer)
var resp consumerPauseApiResponse
req, err := json.Marshal(consumerPauseRequest{
PauseUntil: pauseUntil,
})
if err != nil {
return nil, err
}
if _, err := js.apiRequestJSON(ctx, subject, &resp, req); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return nil, ErrConsumerNotFound
}
return nil, resp.Error
}
return &ConsumerPauseResponse{
Paused: resp.Paused,
PauseUntil: resp.PauseUntil,
PauseRemaining: resp.PauseRemaining,
}, nil
}
func resumeConsumer(ctx context.Context, js *jetStream, stream, consumer string) (*ConsumerPauseResponse, error) {
return pauseConsumer(ctx, js, stream, consumer, nil)
}
func resetConsumer(ctx context.Context, js *jetStream, stream, consumer string, seq uint64) (*ConsumerResetResponse, error) {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
if err := validateConsumerName(consumer); err != nil {
return nil, err
}
subject := fmt.Sprintf(apiConsumerResetT, stream, consumer)
req, err := json.Marshal(consumerResetRequest{Seq: seq})
if err != nil {
return nil, err
}
var resp consumerResetApiResponse
if _, err := js.apiRequestJSON(ctx, subject, &resp, req); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerInvalidReset {
return nil, ErrConsumerInvalidReset
}
return nil, resp.Error
}
if resp.ConsumerInfo == nil {
return nil, ErrConsumerResetResponseEmpty
}
return &resp.ConsumerResetResponse, nil
}
func validateConsumerName(name string) error {
if name == "" {
return fmt.Errorf("%w: name is required", ErrInvalidConsumerName)
}
if strings.ContainsAny(name, ">*. /\\\t\r\n") {
return fmt.Errorf("%w: %q", ErrInvalidConsumerName, name)
}
return nil
}
func unpinConsumer(ctx context.Context, js *jetStream, stream, consumer, group string) error {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
if err := validateConsumerName(consumer); err != nil {
return err
}
unpinSubject := fmt.Sprintf(apiConsumerUnpinT, stream, consumer)
var req = consumerUnpinRequest{
Group: group,
}
reqJSON, err := json.Marshal(req)
if err != nil {
return err
}
var resp apiResponse
if _, err := js.apiRequestJSON(ctx, unpinSubject, &resp, reqJSON); err != nil {
return err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return ErrConsumerNotFound
}
return resp.Error
}
return nil
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/consumer_config.go
Generated Vendored
+593
View File
@@ -0,0 +1,593 @@
// Copyright 2022-2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"encoding/json"
"fmt"
"time"
)
type (
// ConsumerInfo is the detailed information about a JetStream consumer.
ConsumerInfo struct {
// Stream specifies the name of the stream that the consumer is bound
// to.
Stream string `json:"stream_name"`
// Name represents the unique identifier for the consumer. This can be
// either set explicitly by the client or generated automatically if not
// set.
Name string `json:"name"`
// Created is the timestamp when the consumer was created.
Created time.Time `json:"created"`
// Config contains the configuration settings of the consumer, set when
// creating or updating the consumer.
Config ConsumerConfig `json:"config"`
// Delivered holds information about the most recently delivered
// message, including its sequence numbers and timestamp.
Delivered SequenceInfo `json:"delivered"`
// AckFloor indicates the message before the first unacknowledged
// message.
AckFloor SequenceInfo `json:"ack_floor"`
// NumAckPending is the number of messages that have been delivered but
// not yet acknowledged.
NumAckPending int `json:"num_ack_pending"`
// NumRedelivered counts the number of messages that have been
// redelivered and not yet acknowledged. Each message is counted only
// once, even if it has been redelivered multiple times. This count is
// reset when the message is eventually acknowledged.
NumRedelivered int `json:"num_redelivered"`
// NumWaiting is the count of active pull requests. It is only relevant
// for pull-based consumers.
NumWaiting int `json:"num_waiting"`
// NumPending is the number of messages that match the consumer's
// filter, but have not been delivered yet.
NumPending uint64 `json:"num_pending"`
// Cluster contains information about the cluster to which this consumer
// belongs (if applicable).
Cluster *ClusterInfo `json:"cluster,omitempty"`
// PushBound indicates whether at least one subscription exists for the
// delivery subject of this consumer. This is only applicable to
// push-based consumers.
PushBound bool `json:"push_bound,omitempty"`
// TimeStamp indicates when the info was gathered by the server.
TimeStamp time.Time `json:"ts"`
// PriorityGroups contains the information about the currently defined priority groups
PriorityGroups []PriorityGroupState `json:"priority_groups,omitempty"`
// Paused indicates whether the consumer is paused.
Paused bool `json:"paused,omitempty"`
// PauseRemaining contains the amount of time left until the consumer
// unpauses. It will only be non-zero if the consumer is currently paused.
PauseRemaining time.Duration `json:"pause_remaining,omitempty"`
}
PriorityGroupState struct {
// Group this status is for.
Group string `json:"group"`
// PinnedClientID is the generated ID of the pinned client.
PinnedClientID string `json:"pinned_client_id,omitempty"`
// PinnedTS is the timestamp when the client was pinned.
PinnedTS time.Time `json:"pinned_ts,omitempty"`
}
// ConsumerConfig represents the configuration of a JetStream consumer,
// encompassing both push and pull consumer settings
ConsumerConfig struct {
// Name is an optional name for the consumer. If not set, one is
// generated automatically.
//
// Name cannot contain whitespace, ., *, >, path separators (forward or
// backwards slash), and non-printable characters.
Name string `json:"name,omitempty"`
// Durable is an optional durable name for the consumer. If both Durable
// and Name are set, they have to be equal. Unless InactiveThreshold is set, a
// durable consumer will not be cleaned up automatically.
//
// Durable cannot contain whitespace, ., *, >, path separators (forward or
// backwards slash), and non-printable characters.
Durable string `json:"durable_name,omitempty"`
// Description provides an optional description of the consumer.
Description string `json:"description,omitempty"`
// DeliverPolicy defines from which point to start delivering messages
// from the stream. Defaults to DeliverAllPolicy.
DeliverPolicy DeliverPolicy `json:"deliver_policy"`
// OptStartSeq is an optional sequence number from which to start
// message delivery. Only applicable when DeliverPolicy is set to
// DeliverByStartSequencePolicy.
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
// OptStartTime is an optional time from which to start message
// delivery. Only applicable when DeliverPolicy is set to
// DeliverByStartTimePolicy.
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
// AckPolicy defines the acknowledgement policy for the consumer.
// Defaults to AckExplicitPolicy.
AckPolicy AckPolicy `json:"ack_policy"`
// AckWait defines how long the server will wait for an acknowledgement
// before resending a message. If not set, server default is 30 seconds.
AckWait time.Duration `json:"ack_wait,omitempty"`
// MaxDeliver defines the maximum number of delivery attempts for a
// message. Applies to any message that is re-sent due to ack policy.
// If not set, server default is -1 (unlimited).
MaxDeliver int `json:"max_deliver,omitempty"`
// BackOff specifies the optional back-off intervals for retrying
// message delivery after a failed acknowledgement. It overrides
// AckWait.
//
// BackOff only applies to messages not acknowledged in specified time,
// not messages that were nack'ed.
//
// The number of intervals specified must be lower or equal to
// MaxDeliver. If the number of intervals is lower, the last interval is
// used for all remaining attempts.
BackOff []time.Duration `json:"backoff,omitempty"`
// FilterSubject can be used to filter messages delivered from the
// stream. FilterSubject is exclusive with FilterSubjects.
FilterSubject string `json:"filter_subject,omitempty"`
// ReplayPolicy defines the rate at which messages are sent to the
// consumer. If ReplayOriginalPolicy is set, messages are sent in the
// same intervals in which they were stored on stream. This can be used
// e.g. to simulate production traffic in development environments. If
// ReplayInstantPolicy is set, messages are sent as fast as possible.
// Defaults to ReplayInstantPolicy.
ReplayPolicy ReplayPolicy `json:"replay_policy"`
// RateLimit specifies an optional maximum rate of message delivery in
// bits per second.
RateLimit uint64 `json:"rate_limit_bps,omitempty"`
// SampleFrequency is an optional frequency for sampling how often
// acknowledgements are sampled for observability. See
// https://docs.nats.io/running-a-nats-service/nats_admin/monitoring/monitoring_jetstream
SampleFrequency string `json:"sample_freq,omitempty"`
// MaxWaiting is a maximum number of pull requests waiting to be
// fulfilled. If not set, this will inherit settings from stream's
// ConsumerLimits or (if those are not set) from account settings. If
// neither are set, server default is 512.
MaxWaiting int `json:"max_waiting,omitempty"`
// MaxAckPending is a maximum number of outstanding unacknowledged
// messages. Once this limit is reached, the server will suspend sending
// messages to the consumer. If not set, server default is 1000.
// Set to -1 for unlimited.
MaxAckPending int `json:"max_ack_pending,omitempty"`
// HeadersOnly indicates whether only headers of messages should be sent
// (and no payload). Defaults to false.
HeadersOnly bool `json:"headers_only,omitempty"`
// MaxRequestBatch is the optional maximum batch size a single pull
// request can make. When set with MaxRequestMaxBytes, the batch size
// will be constrained by whichever limit is hit first.
MaxRequestBatch int `json:"max_batch,omitempty"`
// MaxRequestExpires is the maximum duration a single pull request will
// wait for messages to be available to pull.
MaxRequestExpires time.Duration `json:"max_expires,omitempty"`
// MaxRequestMaxBytes is the optional maximum total bytes that can be
// requested in a given batch. When set with MaxRequestBatch, the batch
// size will be constrained by whichever limit is hit first.
MaxRequestMaxBytes int `json:"max_bytes,omitempty"`
// InactiveThreshold is a duration which instructs the server to clean
// up the consumer if it has been inactive for the specified duration.
// Durable consumers will not be cleaned up by default, but if
// InactiveThreshold is set, they will be. If not set, this will inherit
// settings from stream's ConsumerLimits. If neither are set, server
// default is 5 seconds.
//
// A consumer is considered inactive if no pull requests are received by
// the server (for pull consumers), or no interest is detected on the
// deliver subject (for push consumers), not if there are no
// messages to be delivered.
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
// Replicas is the number of replicas for the consumer's state. By
// default, consumers inherit the number of replicas from the stream.
Replicas int `json:"num_replicas"`
// MemoryStorage is a flag to force the consumer to use memory storage
// rather than inherit the storage type from the stream.
MemoryStorage bool `json:"mem_storage,omitempty"`
// FilterSubjects allows filtering messages from a stream by subject.
// This field is exclusive with FilterSubject. Requires nats-server
// v2.10.0 or later.
FilterSubjects []string `json:"filter_subjects,omitempty"`
// Metadata is a set of application-defined key-value pairs for
// associating metadata on the consumer. This feature requires
// nats-server v2.10.0 or later.
Metadata map[string]string `json:"metadata,omitempty"`
// PauseUntil is for suspending the consumer until the deadline.
PauseUntil *time.Time `json:"pause_until,omitempty"`
// PriorityPolicy represents the priority policy the consumer is set to.
// Requires nats-server v2.11.0 or later.
PriorityPolicy PriorityPolicy `json:"priority_policy,omitempty"`
// PinnedTTL represents the time after which the client will be unpinned
// if no new pull requests are sent. Used with PriorityPolicyPinned.
// Requires nats-server v2.11.0 or later.
PinnedTTL time.Duration `json:"priority_timeout,omitempty"`
// PriorityGroups is a list of priority groups this consumer supports.
PriorityGroups []string `json:"priority_groups,omitempty"`
// Fields specific to push consumers:
// DeliverSubject is the subject to deliver messages to for push consumers
DeliverSubject string `json:"deliver_subject,omitempty"`
// DeliverGroup is the group name for push consumers
DeliverGroup string `json:"deliver_group,omitempty"`
// FlowControl is a flag to enable flow control for the consumer.
// When set, the server will regularly send an empty message with status
// header 100 and a reply subject. Consumers must reply to these
// messages to control the rate of message delivery.
FlowControl bool `json:"flow_control,omitempty"`
// IdleHeartbeat enables push consumer idle heartbeat messages.
// If the Consumer is idle for more than the set value, an empty message
// with Status header 100 will be sent indicating the consumer is still
// alive.
IdleHeartbeat time.Duration `json:"idle_heartbeat,omitempty"`
}
// OrderedConsumerConfig is the configuration of an ordered JetStream
// consumer. For more information, see [Ordered Consumers] in README
//
// [Ordered Consumers]: https://github.com/nats-io/nats.go/blob/main/jetstream/README.md#ordered-consumers
OrderedConsumerConfig struct {
// FilterSubjects allows filtering messages from a stream by subject.
// This field is exclusive with FilterSubject. Requires nats-server
// v2.10.0 or later.
FilterSubjects []string `json:"filter_subjects,omitempty"`
// DeliverPolicy defines from which point to start delivering messages
// from the stream. Defaults to DeliverAllPolicy.
DeliverPolicy DeliverPolicy `json:"deliver_policy"`
// OptStartSeq is an optional sequence number from which to start
// message delivery. Only applicable when DeliverPolicy is set to
// DeliverByStartSequencePolicy.
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
// OptStartTime is an optional time from which to start message
// delivery. Only applicable when DeliverPolicy is set to
// DeliverByStartTimePolicy.
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
// ReplayPolicy defines the rate at which messages are sent to the
// consumer. If ReplayOriginalPolicy is set, messages are sent in the
// same intervals in which they were stored on the stream. This can be
// used e.g. to simulate production traffic in development environments.
// If ReplayInstantPolicy is set, messages are sent as fast as possible.
// Defaults to ReplayInstantPolicy.
ReplayPolicy ReplayPolicy `json:"replay_policy"`
// InactiveThreshold is a duration which instructs the server to clean
// up the consumer if it has been inactive for the specified duration.
// Defaults to 5m.
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
// HeadersOnly indicates whether only headers of messages should be sent
// (and no payload). Defaults to false.
HeadersOnly bool `json:"headers_only,omitempty"`
// MaxResetAttempts is the maximum number of attempts to recreate the
// consumer in a single recovery cycle. Defaults to unlimited.
MaxResetAttempts int
// Metadata is a set of application-defined key-value pairs for
// associating metadata with the consumer. This feature requires
// nats-server v2.10.0 or later.
Metadata map[string]string `json:"metadata,omitempty"`
// NamePrefix is an optional custom prefix for the consumer name.
// If provided, ordered consumer names will be generated as:
// {NamePrefix}_{sequence_number} (e.g., "custom_1", "custom_2").
// If not provided, a unique ID (NUID) will be used as the prefix.
NamePrefix string `json:"-"`
}
// DeliverPolicy determines from which point to start delivering messages.
DeliverPolicy int
// AckPolicy determines how the consumer should acknowledge delivered
// messages.
AckPolicy int
// ReplayPolicy determines how the consumer should replay messages it
// already has queued in the stream.
ReplayPolicy int
// SequenceInfo has both the consumer and the stream sequence and last
// activity.
SequenceInfo struct {
Consumer uint64 `json:"consumer_seq"`
Stream uint64 `json:"stream_seq"`
Last *time.Time `json:"last_active,omitempty"`
}
// PriorityPolicy determines the priority policy the consumer is set to.
PriorityPolicy int
)
const (
// PriorityPolicyNone is the default priority policy.
PriorityPolicyNone PriorityPolicy = iota
// PriorityPolicyPinned is the priority policy that pins a consumer to a
// specific client.
PriorityPolicyPinned
// PriorityPolicyOverflow is the priority policy that allows for
// restricting when a consumer will receive messages based on the number of
// pending messages or acks.
PriorityPolicyOverflow
// PriorityPolicyPrioritized is the priority policy that allows for the
// server to deliver messages to clients based on their priority (instead
// of round-robin). Requires nats-server v2.12.0 or later.
PriorityPolicyPrioritized
)
func (p *PriorityPolicy) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString(""):
*p = PriorityPolicyNone
case jsonString("pinned_client"):
*p = PriorityPolicyPinned
case jsonString("overflow"):
*p = PriorityPolicyOverflow
case jsonString("prioritized"):
*p = PriorityPolicyPrioritized
default:
return fmt.Errorf("nats: cannot unmarshal %q", data)
}
return nil
}
func (p PriorityPolicy) MarshalJSON() ([]byte, error) {
switch p {
case PriorityPolicyNone:
return json.Marshal("")
case PriorityPolicyPinned:
return json.Marshal("pinned_client")
case PriorityPolicyOverflow:
return json.Marshal("overflow")
case PriorityPolicyPrioritized:
return json.Marshal("prioritized")
}
return nil, fmt.Errorf("nats: unknown priority policy %v", p)
}
const (
// DeliverAllPolicy starts delivering messages from the very beginning of a
// stream. This is the default.
DeliverAllPolicy DeliverPolicy = iota
// DeliverLastPolicy will start the consumer with the last sequence
// received.
DeliverLastPolicy
// DeliverNewPolicy will only deliver new messages that are sent after the
// consumer is created.
DeliverNewPolicy
// DeliverByStartSequencePolicy will deliver messages starting from a given
// sequence configured with OptStartSeq in ConsumerConfig.
DeliverByStartSequencePolicy
// DeliverByStartTimePolicy will deliver messages starting from a given time
// configured with OptStartTime in ConsumerConfig.
DeliverByStartTimePolicy
// DeliverLastPerSubjectPolicy will start the consumer with the last message
// for all subjects received.
DeliverLastPerSubjectPolicy
)
func (p *DeliverPolicy) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString("all"), jsonString("undefined"):
*p = DeliverAllPolicy
case jsonString("last"):
*p = DeliverLastPolicy
case jsonString("new"):
*p = DeliverNewPolicy
case jsonString("by_start_sequence"):
*p = DeliverByStartSequencePolicy
case jsonString("by_start_time"):
*p = DeliverByStartTimePolicy
case jsonString("last_per_subject"):
*p = DeliverLastPerSubjectPolicy
default:
return fmt.Errorf("nats: cannot unmarshal %q", data)
}
return nil
}
func (p DeliverPolicy) MarshalJSON() ([]byte, error) {
switch p {
case DeliverAllPolicy:
return json.Marshal("all")
case DeliverLastPolicy:
return json.Marshal("last")
case DeliverNewPolicy:
return json.Marshal("new")
case DeliverByStartSequencePolicy:
return json.Marshal("by_start_sequence")
case DeliverByStartTimePolicy:
return json.Marshal("by_start_time")
case DeliverLastPerSubjectPolicy:
return json.Marshal("last_per_subject")
}
return nil, fmt.Errorf("nats: unknown deliver policy %v", p)
}
func (p DeliverPolicy) String() string {
switch p {
case DeliverAllPolicy:
return "all"
case DeliverLastPolicy:
return "last"
case DeliverNewPolicy:
return "new"
case DeliverByStartSequencePolicy:
return "by_start_sequence"
case DeliverByStartTimePolicy:
return "by_start_time"
case DeliverLastPerSubjectPolicy:
return "last_per_subject"
}
return ""
}
const (
// AckExplicitPolicy requires ack or nack for all messages.
AckExplicitPolicy AckPolicy = iota
// AckAllPolicy when acking a sequence number, this implicitly acks all
// sequences below this one as well.
AckAllPolicy
// AckNonePolicy requires no acks for delivered messages.
AckNonePolicy
// AckFlowControlPolicy functions like AckAllPolicy, but acks based on
// responses to flow control. Used by durable stream sourcing and
// mirroring against an existing durable push consumer.
//
// This feature requires nats-server v2.14.0 or later.
AckFlowControlPolicy
)
func (p *AckPolicy) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString("none"):
*p = AckNonePolicy
case jsonString("all"):
*p = AckAllPolicy
case jsonString("explicit"):
*p = AckExplicitPolicy
case jsonString("flow_control"):
*p = AckFlowControlPolicy
default:
return fmt.Errorf("nats: cannot unmarshal %q", data)
}
return nil
}
func (p AckPolicy) MarshalJSON() ([]byte, error) {
switch p {
case AckNonePolicy:
return json.Marshal("none")
case AckAllPolicy:
return json.Marshal("all")
case AckExplicitPolicy:
return json.Marshal("explicit")
case AckFlowControlPolicy:
return json.Marshal("flow_control")
}
return nil, fmt.Errorf("nats: unknown acknowledgement policy %v", p)
}
func (p AckPolicy) String() string {
switch p {
case AckNonePolicy:
return "AckNone"
case AckAllPolicy:
return "AckAll"
case AckExplicitPolicy:
return "AckExplicit"
case AckFlowControlPolicy:
return "AckFlowControl"
}
return "Unknown AckPolicy"
}
const (
// ReplayInstantPolicy will replay messages as fast as possible.
ReplayInstantPolicy ReplayPolicy = iota
// ReplayOriginalPolicy will maintain the same timing as the messages were
// received.
ReplayOriginalPolicy
)
func (p *ReplayPolicy) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString("instant"):
*p = ReplayInstantPolicy
case jsonString("original"):
*p = ReplayOriginalPolicy
default:
return fmt.Errorf("nats: cannot unmarshal %q", data)
}
return nil
}
func (p ReplayPolicy) MarshalJSON() ([]byte, error) {
switch p {
case ReplayOriginalPolicy:
return json.Marshal("original")
case ReplayInstantPolicy:
return json.Marshal("instant")
}
return nil, fmt.Errorf("nats: unknown replay policy %v", p)
}
func (p ReplayPolicy) String() string {
switch p {
case ReplayOriginalPolicy:
return "original"
case ReplayInstantPolicy:
return "instant"
}
return ""
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/errors.go
Generated Vendored
+552
View File
@@ -0,0 +1,552 @@
// Copyright 2022-2026 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"errors"
"fmt"
)
type (
// JetStreamError is an error result that happens when using JetStream.
// In case of client-side error, [APIError] returns nil.
JetStreamError interface {
APIError() *APIError
error
}
jsError struct {
apiErr *APIError
message string
}
// APIError is included in all API responses if there was an error.
APIError struct {
Code int `json:"code"`
ErrorCode ErrorCode `json:"err_code"`
Description string `json:"description,omitempty"`
}
// ErrorCode represents error_code returned in response from JetStream API.
ErrorCode uint16
)
const (
JSErrCodeBadRequest ErrorCode = 10003
JSErrCodeConsumerCreate ErrorCode = 10012
JSErrCodeConsumerNameExists ErrorCode = 10013
JSErrCodeConsumerNotFound ErrorCode = 10014
JSErrCodeMaximumConsumersLimit ErrorCode = 10026
JSErrCodeMessageNotFound ErrorCode = 10037
JSErrCodeJetStreamNotEnabledForAccount ErrorCode = 10039
JSErrCodeStreamNameInUse ErrorCode = 10058
JSErrCodeStreamNotFound ErrorCode = 10059
JSErrCodeStreamWrongLastSequence ErrorCode = 10071
JSErrCodeJetStreamNotEnabled ErrorCode = 10076
// JSErrCodeStreamWrongLastSequenceConstant is returned instead of
// JSErrCodeStreamWrongLastSequence for CAS conflicts on replicated (R>1)
// streams. The two are equivalent "wrong last sequence" responses.
JSErrCodeStreamWrongLastSequenceConstant ErrorCode = 10164
JSErrCodeConsumerAlreadyExists ErrorCode = 10105
JSErrCodeDuplicateFilterSubjects ErrorCode = 10136
JSErrCodeOverlappingFilterSubjects ErrorCode = 10138
JSErrCodeConsumerEmptyFilter ErrorCode = 10139
JSErrCodeConsumerExists ErrorCode = 10148
JSErrCodeConsumerDoesNotExist ErrorCode = 10149
JSErrCodeMirrorWithMsgSchedules ErrorCode = 10186
JSErrCodeSourceWithMsgSchedules ErrorCode = 10187
JSErrCodeMessageSchedulesDisabled ErrorCode = 10188
JSErrCodeSchedulePatternInvalid ErrorCode = 10189
JSErrCodeScheduleTargetInvalid ErrorCode = 10190
JSErrCodeScheduleTTLInvalid ErrorCode = 10191
JSErrCodeScheduleRollupInvalid ErrorCode = 10192
JSErrCodeScheduleSourceInvalid ErrorCode = 10203
JSErrCodeConsumerInvalidReset ErrorCode = 10204
)
var (
// JetStream API errors
// ErrJetStreamNotEnabled is an error returned when JetStream is not
// enabled.
//
// Note: This error will not be returned in clustered mode, even if each
// server in the cluster does not have JetStream enabled. In clustered mode,
// requests will time out instead.
ErrJetStreamNotEnabled JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabled, Description: "jetstream not enabled", Code: 503}}
// ErrJetStreamNotEnabledForAccount is an error returned when JetStream is
// not enabled for an account.
ErrJetStreamNotEnabledForAccount JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabledForAccount, Description: "jetstream not enabled for account", Code: 503}}
// ErrStreamNotFound is an error returned when stream with given name does
// not exist.
ErrStreamNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNotFound, Description: "stream not found", Code: 404}}
// ErrStreamNameAlreadyInUse is returned when a stream with given name
// already exists and has a different configuration.
ErrStreamNameAlreadyInUse JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNameInUse, Description: "stream name already in use", Code: 400}}
// ErrStreamSubjectTransformNotSupported is returned when the connected
// nats-server version does not support setting the stream subject
// transform. If this error is returned when executing CreateStream(), the
// stream with invalid configuration was already created in the server.
ErrStreamSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
// ErrStreamSourceSubjectTransformNotSupported is returned when the
// connected nats-server version does not support setting the stream source
// subject transform. If this error is returned when executing
// CreateStream(), the stream with invalid configuration was already created
// in the server.
ErrStreamSourceSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
// ErrStreamSourceNotSupported is returned when the connected nats-server
// version does not support setting the stream sources. If this error is
// returned when executing CreateStream(), the stream with invalid
// configuration was already created in the server.
ErrStreamSourceNotSupported JetStreamError = &jsError{message: "stream sourcing is not supported by nats-server"}
// ErrStreamSourceMultipleFilterSubjectsNotSupported is returned when the
// connected nats-server version does not support setting the stream
// sources. If this error is returned when executing CreateStream(), the
// stream with invalid configuration was already created in the server.
ErrStreamSourceMultipleFilterSubjectsNotSupported JetStreamError = &jsError{message: "stream sourcing with multiple subject filters not supported by nats-server"}
// ErrConsumerNotFound is an error returned when consumer with given name
// does not exist.
ErrConsumerNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerNotFound, Description: "consumer not found", Code: 404}}
// ErrConsumerCreationResponseEmpty is an error returned when the response from the server
// when creating a consumer is empty. This means that the state of the consumer is unknown and
// the consumer may not have been created successfully.
ErrConsumerCreationResponseEmpty JetStreamError = &jsError{message: "consumer creation response is empty"}
// ErrInvalidJetStreamResponse is returned when the response from the server
// to a JetStream API call (stream CRUD, message get, etc.) does not contain
// the expected data payload. The operation may or may not have succeeded on
// the server side.
ErrInvalidJetStreamResponse JetStreamError = &jsError{message: "invalid jetstream api response"}
// ErrConsumerExists is returned when attempting to create a consumer with
// CreateConsumer but a consumer with given name already exists.
ErrConsumerExists JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerExists, Description: "consumer already exists", Code: 400}}
// ErrConsumerNameExists is returned when attempting to update a consumer
// with UpdateConsumer but a consumer with given name does not exist.
ErrConsumerDoesNotExist JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerDoesNotExist, Description: "consumer does not exist", Code: 400}}
// ErrConsumerResetResponseEmpty is returned when the response from the
// server to a consumer reset request is missing the ConsumerInfo
// payload. The reset may or may not have taken effect.
ErrConsumerResetResponseEmpty JetStreamError = &jsError{message: "consumer reset response is empty"}
// ErrConsumerInvalidReset is returned when ResetConsumerToSequence is
// called with a sequence that violates the consumer's DeliverPolicy
// constraints (e.g. seq below OptStartSeq, or non-zero seq with a
// DeliverPolicy other than all/by-start-sequence/by-start-time).
ErrConsumerInvalidReset JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerInvalidReset, Description: "invalid reset", Code: 400}}
// ErrMsgNotFound is returned when message with provided sequence number
// does not exist.
ErrMsgNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMessageNotFound, Description: "message not found", Code: 404}}
// ErrBadRequest is returned when invalid request is sent to JetStream API.
ErrBadRequest JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeBadRequest, Description: "bad request", Code: 400}}
// ErrConsumerCreate is returned when nats-server reports error when
// creating consumer (e.g. illegal update).
ErrConsumerCreate JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerCreate, Description: "could not create consumer", Code: 500}}
// ErrMaximumConsumersLimit is returned when user limit of allowed
// consumers for stream is reached
ErrMaximumConsumersLimit JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMaximumConsumersLimit, Description: "maximum consumers limit reached", Code: 400}}
// ErrDuplicateFilterSubjects is returned when both FilterSubject and
// FilterSubjects are specified when creating consumer.
ErrDuplicateFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeDuplicateFilterSubjects, Description: "consumer cannot have both FilterSubject and FilterSubjects specified", Code: 500}}
// ErrOverlappingFilterSubjects is returned when filter subjects overlap when
// creating consumer.
ErrOverlappingFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeOverlappingFilterSubjects, Description: "consumer subject filters cannot overlap", Code: 500}}
// ErrEmptyFilter is returned when a filter in FilterSubjects is empty.
ErrEmptyFilter JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerEmptyFilter, Description: "consumer filter in FilterSubjects cannot be empty", Code: 500}}
// ErrScheduleTargetInvalid is returned when publishing a scheduled
// message without a valid target subject.
ErrScheduleTargetInvalid JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeScheduleTargetInvalid, Description: "message schedules target is invalid", Code: 400}}
// ErrSchedulePatternInvalid is returned when the schedule expression
// is not a valid @at, @every, or cron pattern.
ErrSchedulePatternInvalid JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeSchedulePatternInvalid, Description: "message schedules pattern is invalid", Code: 400}}
// ErrMessageSchedulesDisabled is returned when publishing a scheduled
// message to a stream that does not have AllowMsgSchedules enabled.
ErrMessageSchedulesDisabled JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMessageSchedulesDisabled, Description: "message schedules is disabled", Code: 400}}
// ErrScheduleSourceInvalid is returned when the schedule source
// subject is invalid.
ErrScheduleSourceInvalid JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeScheduleSourceInvalid, Description: "message schedules source is invalid", Code: 400}}
// ErrScheduleTTLInvalid is returned when the schedule TTL value
// is not a valid duration or "never".
ErrScheduleTTLInvalid JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeScheduleTTLInvalid, Description: "message schedules invalid per-message TTL", Code: 400}}
// ErrScheduleRollupInvalid is returned when a scheduled message
// has an invalid rollup configuration.
ErrScheduleRollupInvalid JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeScheduleRollupInvalid, Description: "message schedules invalid rollup", Code: 400}}
// ErrMirrorWithMsgSchedules is returned when attempting to enable
// message scheduling on a mirror stream.
ErrMirrorWithMsgSchedules JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMirrorWithMsgSchedules, Description: "stream mirrors can not also schedule messages", Code: 400}}
// ErrSourceWithMsgSchedules is returned when attempting to enable
// message scheduling on a stream with sources.
ErrSourceWithMsgSchedules JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeSourceWithMsgSchedules, Description: "stream source can not also schedule messages", Code: 400}}
// Client errors
// ErrConsumerMultipleFilterSubjectsNotSupported is returned when the
// connected nats-server version does not support setting multiple filter
// subjects with filter_subjects field. If this error is returned when
// executing AddConsumer(), the consumer with invalid configuration was
// already created in the server.
ErrConsumerMultipleFilterSubjectsNotSupported JetStreamError = &jsError{message: "multiple consumer filter subjects not supported by nats-server"}
// ErrConsumerNameAlreadyInUse is an error returned when attempting to create
// a consumer with a name that is already in use.
ErrConsumerNameAlreadyInUse JetStreamError = &jsError{message: "consumer name already in use"}
// ErrNotPullConsumer is returned when attempting to fetch or create pull
// consumer and the returned consumer is a push consumer.
ErrNotPullConsumer JetStreamError = &jsError{message: "consumer is not a pull consumer"}
// ErrNotPushConsumer is returned when attempting to fetch or create push
// consumer and the returned consumer is a pull consumer.
ErrNotPushConsumer JetStreamError = &jsError{message: "consumer is not a push consumer"}
// ErrConsumerAlreadyConsuming is returned when attempting to consume from
// the same push consumer more than once.
ErrConsumerAlreadyConsuming JetStreamError = &jsError{message: "consumer is already consuming"}
// ErrInvalidJSAck is returned when JetStream ack from message publish is
// invalid.
ErrInvalidJSAck JetStreamError = &jsError{message: "invalid jetstream publish response"}
// ErrStreamNameRequired is returned when the provided stream name is empty.
ErrStreamNameRequired JetStreamError = &jsError{message: "stream name is required"}
// ErrMsgAlreadyAckd is returned when attempting to acknowledge message more
// than once.
ErrMsgAlreadyAckd JetStreamError = &jsError{message: "message was already acknowledged"}
// ErrNoStreamResponse is returned when there is no response from stream
// (e.g. no responders error).
ErrNoStreamResponse JetStreamError = &jsError{message: "no response from stream"}
// ErrNotJSMessage is returned when attempting to get metadata from non
// JetStream message.
ErrNotJSMessage JetStreamError = &jsError{message: "not a jetstream message"}
// ErrInvalidStreamName is returned when the provided stream name is invalid
// (contains '.').
ErrInvalidStreamName JetStreamError = &jsError{message: "invalid stream name"}
// ErrInvalidSubject is returned when the provided subject name is invalid.
ErrInvalidSubject JetStreamError = &jsError{message: "invalid subject name"}
// ErrInvalidConsumerName is returned when the provided consumer name is
// invalid (contains '.').
ErrInvalidConsumerName JetStreamError = &jsError{message: "invalid consumer name"}
// ErrNoMessages is returned when no messages are currently available for a
// consumer.
ErrNoMessages JetStreamError = &jsError{message: "no messages"}
// ErrPinIDMismatch is returned when Pin ID sent in the request does not match
// the currently pinned consumer subscriber ID on the server.
ErrPinIDMismatch JetStreamError = &jsError{message: "pin ID mismatch"}
// ErrMaxBytesExceeded is returned when a message would exceed MaxBytes set
// on a pull request.
ErrMaxBytesExceeded JetStreamError = &jsError{message: "message size exceeds max bytes"}
// ErrBatchCompleted is returned when a fetch request sent the whole batch,
// but there are still bytes left. This is applicable only when MaxBytes is
// set on a pull request.
ErrBatchCompleted JetStreamError = &jsError{message: "batch completed"}
// ErrConsumerDeleted is returned when attempting to send pull request to a
// consumer which does not exist.
ErrConsumerDeleted JetStreamError = &jsError{message: "consumer deleted"}
// ErrConsumerLeadershipChanged is returned when pending requests are no
// longer valid after leadership has changed.
ErrConsumerLeadershipChanged JetStreamError = &jsError{message: "leadership change"}
// ErrHandlerRequired is returned when no handler func is provided in
// Stream().
ErrHandlerRequired JetStreamError = &jsError{message: "handler cannot be empty"}
// ErrEndOfData is returned when iterating over paged API from JetStream
// reaches end of data.
ErrEndOfData JetStreamError = &jsError{message: "end of data reached"}
// ErrNoHeartbeat is received when no message is received in IdleHeartbeat
// time (if set).
ErrNoHeartbeat JetStreamError = &jsError{message: "no heartbeat received"}
// ErrConsumerHasActiveSubscription is returned when a consumer is already
// subscribed to a stream.
ErrConsumerHasActiveSubscription JetStreamError = &jsError{message: "consumer has active subscription"}
// ErrMsgNotBound is returned when given message is not bound to any
// subscription.
ErrMsgNotBound JetStreamError = &jsError{message: "message is not bound to subscription/connection"}
// ErrMsgNoReply is returned when attempting to reply to a message without a
// reply subject.
ErrMsgNoReply JetStreamError = &jsError{message: "message does not have a reply"}
// ErrMsgDeleteUnsuccessful is returned when an attempt to delete a message
// is unsuccessful.
ErrMsgDeleteUnsuccessful JetStreamError = &jsError{message: "message deletion unsuccessful"}
// ErrAsyncPublishReplySubjectSet is returned when reply subject is set on
// async message publish.
ErrAsyncPublishReplySubjectSet JetStreamError = &jsError{message: "reply subject should be empty"}
// ErrTooManyStalledMsgs is returned when too many outstanding async
// messages are waiting for ack.
ErrTooManyStalledMsgs JetStreamError = &jsError{message: "stalled with too many outstanding async published messages"}
// ErrInvalidOption is returned when there is a collision between options.
ErrInvalidOption JetStreamError = &jsError{message: "invalid jetstream option"}
// ErrMsgIteratorClosed is returned when attempting to get message from a
// closed iterator.
ErrMsgIteratorClosed JetStreamError = &jsError{message: "messages iterator closed"}
// ErrConnectionClosed is returned when JetStream operations fail due to
// underlying connection being closed.
ErrConnectionClosed JetStreamError = &jsError{message: "connection closed"}
// ErrServerShutdown is returned when pull request fails due to server
// shutdown.
ErrServerShutdown JetStreamError = &jsError{message: "server shutdown"}
// ErrOrderedConsumerReset indicates that the ordered consumer was
// automatically reset and recreated to preserve message ordering.
ErrOrderedConsumerReset JetStreamError = &jsError{message: "recreating ordered consumer"}
// ErrOrderConsumerUsedAsFetch is returned when ordered consumer was already
// used to process messages using Fetch (or FetchBytes).
ErrOrderConsumerUsedAsFetch JetStreamError = &jsError{message: "ordered consumer initialized as fetch"}
// ErrOrderConsumerUsedAsConsume is returned when ordered consumer was
// already used to process messages using Consume or Messages.
ErrOrderConsumerUsedAsConsume JetStreamError = &jsError{message: "ordered consumer initialized as consume"}
// ErrOrderedConsumerConcurrentRequests is returned when attempting to run
// concurrent operations on ordered consumers.
ErrOrderedConsumerConcurrentRequests JetStreamError = &jsError{message: "cannot run concurrent processing using ordered consumer"}
// ErrOrderedConsumerNotCreated is returned when trying to get consumer info
// of an ordered consumer which was not yet created.
ErrOrderedConsumerNotCreated JetStreamError = &jsError{message: "consumer instance not yet created"}
// ErrJetStreamPublisherClosed is returned for each unfinished ack future when JetStream.Cleanup is called.
ErrJetStreamPublisherClosed JetStreamError = &jsError{message: "jetstream context closed"}
// ErrAsyncPublishTimeout is returned when waiting for ack on async publish
ErrAsyncPublishTimeout JetStreamError = &jsError{message: "timeout waiting for ack"}
// KeyValue Errors
// ErrKeyExists is returned when attempting to create a key that already
// exists.
//
// Note: ErrKeyExists matches errors by code 10071, which CAS conflicts
// from Update/Delete/Purge also carry on non-replicated streams;
// replicated (R>1) streams report code 10164 instead and will not match.
// Do not use ErrKeyExists to detect revision conflicts - use
// ErrKeyRevisionMismatch.
ErrKeyExists JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamWrongLastSequence, Code: 400}, message: "key exists"}
// ErrKeyRevisionMismatch is returned by Update, and by Delete/Purge when
// the LastRevision option is used, if the provided revision does not
// match the key's current revision (an optimistic-concurrency conflict).
// Replicated (R>1) streams report this as error code 10164 instead of
// 10071; both map to this error.
ErrKeyRevisionMismatch JetStreamError = &jsError{message: "key revision mismatch"}
// ErrKeyValueConfigRequired is returned when attempting to create a bucket
// without a config.
ErrKeyValueConfigRequired JetStreamError = &jsError{message: "config required"}
// ErrInvalidBucketName is returned when attempting to create a bucket with
// an invalid name.
ErrInvalidBucketName JetStreamError = &jsError{message: "invalid bucket name"}
// ErrInvalidKey is returned when attempting to create a key with an invalid
// name.
ErrInvalidKey JetStreamError = &jsError{message: "invalid key"}
// ErrBucketExists is returned when attempting to create a bucket that
// already exists and has a different configuration.
ErrBucketExists JetStreamError = &jsError{message: "bucket name already in use"}
// ErrBucketNotFound is returned when attempting to access a bucket that
// does not exist.
ErrBucketNotFound JetStreamError = &jsError{message: "bucket not found"}
// ErrBadBucket is returned when attempting to access a bucket that is not a
// key-value store.
ErrBadBucket JetStreamError = &jsError{message: "bucket not valid key-value store"}
// ErrKeyNotFound is returned when attempting to access a key that does not
// exist.
ErrKeyNotFound JetStreamError = &jsError{message: "key not found"}
// ErrKeyDeleted is returned when attempting to access a key that was
// deleted.
ErrKeyDeleted JetStreamError = &jsError{message: "key was deleted"}
// ErrHistoryTooLarge is returned when provided history limit is larger than
// 64.
ErrHistoryTooLarge JetStreamError = &jsError{message: "history limited to a max of 64"}
// ErrNoKeysFound is returned when no keys are found.
ErrNoKeysFound JetStreamError = &jsError{message: "no keys found"}
// ErrTTLOnDeleteNotSupported is returned when attempting to set a TTL
// on a delete operation.
ErrTTLOnDeleteNotSupported JetStreamError = &jsError{message: "TTL is not supported on delete"}
// ErrLimitMarkerTTLNotSupported is returned when the connected jetstream API
// does not support setting the LimitMarkerTTL.
ErrLimitMarkerTTLNotSupported JetStreamError = &jsError{message: "limit marker TTLs not supported by server"}
// ErrObjectConfigRequired is returned when attempting to create an object
// without a config.
ErrObjectConfigRequired JetStreamError = &jsError{message: "object-store config required"}
// ErrBadObjectMeta is returned when the meta information of an object is
// invalid.
ErrBadObjectMeta JetStreamError = &jsError{message: "object-store meta information invalid"}
// ErrObjectNotFound is returned when an object is not found.
ErrObjectNotFound JetStreamError = &jsError{message: "object not found"}
// ErrInvalidStoreName is returned when the name of an object-store is
// invalid.
ErrInvalidStoreName JetStreamError = &jsError{message: "invalid object-store name"}
// ErrDigestMismatch is returned when the digests of an object do not match.
ErrDigestMismatch JetStreamError = &jsError{message: "received a corrupt object, digests do not match"}
// ErrInvalidDigestFormat is returned when the digest hash of an object has
// an invalid format.
ErrInvalidDigestFormat JetStreamError = &jsError{message: "object digest hash has invalid format"}
// ErrNoObjectsFound is returned when no objects are found.
ErrNoObjectsFound JetStreamError = &jsError{message: "no objects found"}
// ErrObjectAlreadyExists is returned when an object with the same name
// already exists.
ErrObjectAlreadyExists JetStreamError = &jsError{message: "an object already exists with that name"}
// ErrNameRequired is returned when a name is required.
ErrNameRequired JetStreamError = &jsError{message: "name is required"}
// ErrLinkNotAllowed is returned when a link cannot be set when putting the
// object in a bucket.
ErrLinkNotAllowed JetStreamError = &jsError{message: "link cannot be set when putting the object in bucket"}
// ErrObjectRequired is returned when an object is required.
ErrObjectRequired = &jsError{message: "object required"}
// ErrNoLinkToDeleted is returned when it is not allowed to link to a
// deleted object.
ErrNoLinkToDeleted JetStreamError = &jsError{message: "not allowed to link to a deleted object"}
// ErrNoLinkToLink is returned when it is not allowed to link to another
// link.
ErrNoLinkToLink JetStreamError = &jsError{message: "not allowed to link to another link"}
// ErrCantGetBucket is returned when an invalid Get is attempted on an
// object that is a link to a bucket.
ErrCantGetBucket JetStreamError = &jsError{message: "invalid Get, object is a link to a bucket"}
// ErrBucketRequired is returned when a bucket is required.
ErrBucketRequired JetStreamError = &jsError{message: "bucket required"}
// ErrBucketMalformed is returned when a bucket is malformed.
ErrBucketMalformed JetStreamError = &jsError{message: "bucket malformed"}
// ErrUpdateMetaDeleted is returned when the meta information of a deleted
// object cannot be updated.
ErrUpdateMetaDeleted JetStreamError = &jsError{message: "cannot update meta for a deleted object"}
)
// Error prints the JetStream API error code and description.
func (e *APIError) Error() string {
return fmt.Sprintf("nats: API error: code=%d err_code=%d description=%s", e.Code, e.ErrorCode, e.Description)
}
// APIError implements the JetStreamError interface.
func (e *APIError) APIError() *APIError {
return e
}
// Is matches against an APIError.
func (e *APIError) Is(err error) bool {
if e == nil {
return false
}
// Extract internal APIError to match against.
var aerr *APIError
ok := errors.As(err, &aerr)
if !ok {
return ok
}
return e.ErrorCode == aerr.ErrorCode
}
func (err *jsError) APIError() *APIError {
return err.apiErr
}
func (err *jsError) Error() string {
if err.apiErr != nil && err.apiErr.Description != "" {
return err.apiErr.Error()
}
return fmt.Sprintf("nats: %s", err.message)
}
func (err *jsError) Unwrap() error {
// Allow matching to embedded APIError in case there is one.
if err.apiErr == nil {
return nil
}
return err.apiErr
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/jetstream.go
Generated Vendored
+1324
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/jetstream_options.go
Generated Vendored
+864
View File
@@ -0,0 +1,864 @@
// Copyright 2022-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"fmt"
"time"
)
type pullOptFunc func(*consumeOpts) error
func (fn pullOptFunc) configureConsume(opts *consumeOpts) error {
return fn(opts)
}
func (fn pullOptFunc) configureMessages(opts *consumeOpts) error {
return fn(opts)
}
// WithClientTrace enables request/response API calls tracing.
func WithClientTrace(ct *ClientTrace) JetStreamOpt {
return func(opts *JetStreamOptions) error {
opts.ClientTrace = ct
return nil
}
}
// WithPublishAsyncErrHandler sets error handler for async message publish.
func WithPublishAsyncErrHandler(cb MsgErrHandler) JetStreamOpt {
return func(opts *JetStreamOptions) error {
opts.publisherOpts.aecb = cb
return nil
}
}
// WithPublishAsyncAckHandler sets an ack handler for async message publish,
// invoked when a publish is successfully acked by the server. See
// [MsgAckHandler] for constraints on the handler.
func WithPublishAsyncAckHandler(cb MsgAckHandler) JetStreamOpt {
return func(opts *JetStreamOptions) error {
opts.publisherOpts.ackcb = cb
return nil
}
}
// WithPublishAsyncMaxPending sets the maximum outstanding async publishes that
// can be inflight at one time.
func WithPublishAsyncMaxPending(max int) JetStreamOpt {
return func(opts *JetStreamOptions) error {
if max < 1 {
return fmt.Errorf("%w: max ack pending should be >= 1", ErrInvalidOption)
}
opts.publisherOpts.maxpa = max
return nil
}
}
// WithPublishAsyncTimeout sets the timeout for async message publish.
// If not provided, timeout is disabled.
func WithPublishAsyncTimeout(dur time.Duration) JetStreamOpt {
return func(opts *JetStreamOptions) error {
opts.publisherOpts.ackTimeout = dur
return nil
}
}
// WithDefaultTimeout sets the default timeout for JetStream API requests.
// It is used when context used for the request does not have a deadline set.
// If not provided, a default of 5 seconds will be used.
func WithDefaultTimeout(timeout time.Duration) JetStreamOpt {
return func(opts *JetStreamOptions) error {
if timeout <= 0 {
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
}
opts.DefaultTimeout = timeout
return nil
}
}
// WithPurgeSubject sets a specific subject for which messages on a stream will
// be purged
func WithPurgeSubject(subject string) StreamPurgeOpt {
return func(req *StreamPurgeRequest) error {
req.Subject = subject
return nil
}
}
// WithPurgeSequence is used to set a specific sequence number up to which (but
// not including) messages will be purged from a stream Can be combined with
// [WithPurgeSubject] option, but not with [WithPurgeKeep]
func WithPurgeSequence(sequence uint64) StreamPurgeOpt {
return func(req *StreamPurgeRequest) error {
if req.Keep != 0 {
return fmt.Errorf("%w: both 'keep' and 'sequence' cannot be provided in purge request", ErrInvalidOption)
}
req.Sequence = sequence
return nil
}
}
// WithPurgeKeep sets the number of messages to be kept in the stream after
// purge. Can be combined with [WithPurgeSubject] option, but not with
// [WithPurgeSequence]
func WithPurgeKeep(keep uint64) StreamPurgeOpt {
return func(req *StreamPurgeRequest) error {
if req.Sequence != 0 {
return fmt.Errorf("%w: both 'keep' and 'sequence' cannot be provided in purge request", ErrInvalidOption)
}
req.Keep = keep
return nil
}
}
// WithGetMsgSubject sets the stream subject from which the message should be
// retrieved. Server will return a first message with a seq >= to the input seq
// that has the specified subject.
func WithGetMsgSubject(subject string) GetMsgOpt {
return func(req *apiMsgGetRequest) error {
req.NextFor = subject
return nil
}
}
// PullMaxMessages limits the number of messages to be buffered in the client.
// If not provided, a default of 500 messages will be used.
// This option is exclusive with PullMaxBytes.
//
// PullMaxMessages implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullMaxMessages int
func (max PullMaxMessages) configureConsume(opts *consumeOpts) error {
if max <= 0 {
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
}
opts.MaxMessages = int(max)
return nil
}
func (max PullMaxMessages) configureMessages(opts *consumeOpts) error {
if max <= 0 {
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
}
opts.MaxMessages = int(max)
return nil
}
type pullMaxMessagesWithBytesLimit struct {
maxMessages int
maxBytes int
}
// PullMaxMessagesWithBytesLimit limits the number of messages to be buffered
// in the client. Additionally, it sets the maximum size a single fetch request
// can have. Note that this will not limit the total size of messages buffered
// in the client, but rather can serve as a way to limit what nats server will
// have to internally buffer for a single fetch request.
//
// The byte limit should never be set to a value lower than the maximum message
// size that can be expected from the server. If the byte limit is lower than
// the maximum message size, the consumer will stall and not be able to consume
// messages.
//
// This is an advanced option and should be used with caution. Most users should
// use [PullMaxMessages] or [PullMaxBytes] instead.
//
// PullMaxMessagesWithBytesLimit implements both PullConsumeOpt and
// PullMessagesOpt, allowing it to configure Consumer.Consume and Consumer.Messages.
func PullMaxMessagesWithBytesLimit(maxMessages, byteLimit int) pullMaxMessagesWithBytesLimit {
return pullMaxMessagesWithBytesLimit{maxMessages, byteLimit}
}
func (m pullMaxMessagesWithBytesLimit) configureConsume(opts *consumeOpts) error {
if m.maxMessages <= 0 {
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
}
if m.maxBytes <= 0 {
return fmt.Errorf("%w: maxBytes size must be at least 1", ErrInvalidOption)
}
if opts.MaxMessages > 0 {
return fmt.Errorf("%w: maxMessages already set", ErrInvalidOption)
}
opts.MaxMessages = m.maxMessages
opts.MaxBytes = m.maxBytes
opts.LimitSize = true
return nil
}
func (m pullMaxMessagesWithBytesLimit) configureMessages(opts *consumeOpts) error {
if m.maxMessages <= 0 {
return fmt.Errorf("%w: maxMessages size must be at least 1", ErrInvalidOption)
}
if m.maxBytes <= 0 {
return fmt.Errorf("%w: maxBytes size must be at least 1", ErrInvalidOption)
}
if opts.MaxMessages > 0 {
return fmt.Errorf("%w: maxMessages already set", ErrInvalidOption)
}
opts.MaxMessages = m.maxMessages
opts.MaxBytes = m.maxBytes
opts.LimitSize = true
return nil
}
// PullExpiry sets timeout on a single pull request, waiting until at least one
// message is available.
// If not provided, a default of 30 seconds will be used.
//
// PullExpiry implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullExpiry time.Duration
func (exp PullExpiry) configureConsume(opts *consumeOpts) error {
expiry := time.Duration(exp)
if expiry < time.Second {
return fmt.Errorf("%w: expires value must be at least 1s", ErrInvalidOption)
}
opts.Expires = expiry
return nil
}
func (exp PullExpiry) configureMessages(opts *consumeOpts) error {
expiry := time.Duration(exp)
if expiry < time.Second {
return fmt.Errorf("%w: expires value must be at least 1s", ErrInvalidOption)
}
opts.Expires = expiry
return nil
}
// PullMaxBytes limits the number of bytes to be buffered in the client.
// If not provided, the limit is not set (max messages will be used instead).
// This option is exclusive with PullMaxMessages.
//
// The value should be set to a high enough value to accommodate the largest
// message expected from the server. Note that it may not be sufficient to set
// this value to the maximum message size, as this setting controls the client
// buffer size, not the max bytes requested from the server within a single pull
// request. If the value is set too low, the consumer will stall and not be able
// to consume messages.
//
// PullMaxBytes implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullMaxBytes int
func (max PullMaxBytes) configureConsume(opts *consumeOpts) error {
if max <= 0 {
return fmt.Errorf("%w: max bytes must be greater than 0", ErrInvalidOption)
}
opts.MaxBytes = int(max)
return nil
}
func (max PullMaxBytes) configureMessages(opts *consumeOpts) error {
if max <= 0 {
return fmt.Errorf("%w: max bytes must be greater than 0", ErrInvalidOption)
}
opts.MaxBytes = int(max)
return nil
}
// PullThresholdMessages sets the message count on which consuming will trigger
// new pull request to the server. Defaults to 50% of MaxMessages.
//
// PullThresholdMessages implements both PullConsumeOpt and PullMessagesOpt,
// allowing it to configure Consumer.Consume and Consumer.Messages.
type PullThresholdMessages int
func (t PullThresholdMessages) configureConsume(opts *consumeOpts) error {
opts.ThresholdMessages = int(t)
return nil
}
func (t PullThresholdMessages) configureMessages(opts *consumeOpts) error {
opts.ThresholdMessages = int(t)
return nil
}
// PullThresholdBytes sets the byte count on which consuming will trigger
// new pull request to the server. Defaults to 50% of MaxBytes (if set).
//
// PullThresholdBytes implements both PullConsumeOpt and PullMessagesOpt,
// allowing it to configure Consumer.Consume and Consumer.Messages.
type PullThresholdBytes int
func (t PullThresholdBytes) configureConsume(opts *consumeOpts) error {
opts.ThresholdBytes = int(t)
return nil
}
func (t PullThresholdBytes) configureMessages(opts *consumeOpts) error {
opts.ThresholdBytes = int(t)
return nil
}
// PullMinPending sets the minimum number of messages that should be pending for
// a consumer with PriorityPolicyOverflow to be considered for delivery.
// If provided, PullPriorityGroup must be set as well and the consumer has to have
// PriorityPolicy set to PriorityPolicyOverflow.
//
// PullMinPending implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullMinPending int
func (min PullMinPending) configureConsume(opts *consumeOpts) error {
if min < 1 {
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
}
opts.MinPending = int64(min)
return nil
}
func (min PullMinPending) configureMessages(opts *consumeOpts) error {
if min < 1 {
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
}
opts.MinPending = int64(min)
return nil
}
// PullMinAckPending sets the minimum number of pending acks that should be
// present for a consumer with PriorityPolicyOverflow to be considered for
// delivery. If provided, PullPriorityGroup must be set as well and the consumer
// has to have PriorityPolicy set to PriorityPolicyOverflow.
//
// PullMinAckPending implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullMinAckPending int
func (min PullMinAckPending) configureConsume(opts *consumeOpts) error {
if min < 1 {
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
}
opts.MinAckPending = int64(min)
return nil
}
func (min PullMinAckPending) configureMessages(opts *consumeOpts) error {
if min < 1 {
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
}
opts.MinAckPending = int64(min)
return nil
}
// PullPrioritized sets the priority used when sending pull requests for consumer with
// PriorityPolicyPrioritized. Lower values indicate higher priority (0 is the
// highest priority). Maximum priority value is 9.
//
// If provided, PullPriorityGroup must be set as well and the consumer has to
// have PriorityPolicy set to PriorityPolicyPrioritized.
//
// PullPrioritized implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullPrioritized uint8
func (p PullPrioritized) configureConsume(opts *consumeOpts) error {
opts.Priority = uint8(p)
return nil
}
func (p PullPrioritized) configureMessages(opts *consumeOpts) error {
opts.Priority = uint8(p)
return nil
}
// PullPriorityGroup sets the priority group for a consumer.
// It has to match one of the priority groups set on the consumer.
//
// PullPriorityGroup implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullPriorityGroup string
func (g PullPriorityGroup) configureConsume(opts *consumeOpts) error {
opts.Group = string(g)
return nil
}
func (g PullPriorityGroup) configureMessages(opts *consumeOpts) error {
opts.Group = string(g)
return nil
}
// PullHeartbeat sets the idle heartbeat duration for a pull subscription
// If a client does not receive a heartbeat message from a stream for more
// than the idle heartbeat setting, the subscription will be removed
// and error will be passed to the message handler.
// If not provided, a default PullExpiry / 2 will be used (capped at 30 seconds)
//
// PullHeartbeat implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type PullHeartbeat time.Duration
func (hb PullHeartbeat) configureConsume(opts *consumeOpts) error {
hbTime := time.Duration(hb)
if hbTime < 500*time.Millisecond || hbTime > 30*time.Second {
return fmt.Errorf("%w: idle_heartbeat value must be within 500ms-30s range", ErrInvalidOption)
}
opts.Heartbeat = hbTime
return nil
}
func (hb PullHeartbeat) configureMessages(opts *consumeOpts) error {
hbTime := time.Duration(hb)
if hbTime < 500*time.Millisecond || hbTime > 30*time.Second {
return fmt.Errorf("%w: idle_heartbeat value must be within 500ms-30s range", ErrInvalidOption)
}
opts.Heartbeat = hbTime
return nil
}
// StopAfter sets the number of messages after which the consumer is
// automatically stopped and no more messages are pulled from the server.
//
// StopAfter implements both PullConsumeOpt and PullMessagesOpt, allowing
// it to configure Consumer.Consume and Consumer.Messages.
type StopAfter int
func (nMsgs StopAfter) configureConsume(opts *consumeOpts) error {
if nMsgs <= 0 {
return fmt.Errorf("%w: auto stop after value cannot be less than 1", ErrInvalidOption)
}
opts.StopAfter = int(nMsgs)
return nil
}
func (nMsgs StopAfter) configureMessages(opts *consumeOpts) error {
if nMsgs <= 0 {
return fmt.Errorf("%w: auto stop after value cannot be less than 1", ErrInvalidOption)
}
opts.StopAfter = int(nMsgs)
return nil
}
// ConsumeErrHandler sets custom error handler invoked when an error was
// encountered while consuming messages It will be invoked for both terminal
// (Consumer Deleted, invalid request body) and non-terminal (e.g. missing
// heartbeats) errors.
type ConsumeErrHandler ConsumeErrHandlerFunc
func (c ConsumeErrHandler) configureConsume(opts *consumeOpts) error {
opts.ErrHandler = c
return nil
}
func (c ConsumeErrHandler) configurePushConsume(opts *pushConsumeOpts) error {
opts.ErrHandler = c
return nil
}
// WithMessagesErrOnMissingHeartbeat sets whether a missing heartbeat error
// should be reported when calling [MessagesContext.Next] (Default: true).
func WithMessagesErrOnMissingHeartbeat(hbErr bool) PullMessagesOpt {
return pullOptFunc(func(cfg *consumeOpts) error {
cfg.ReportMissingHeartbeats = hbErr
return nil
})
}
// FetchMinPending sets the minimum number of messages that should be pending for
// a consumer with PriorityPolicyOverflow to be considered for delivery.
// If provided, FetchPriorityGroup must be set as well and the consumer has to have
// PriorityPolicy set to PriorityPolicyOverflow.
func FetchMinPending(min int64) FetchOpt {
return func(req *pullRequest) error {
if min < 1 {
return fmt.Errorf("%w: min pending should be more than 0", ErrInvalidOption)
}
req.MinPending = min
return nil
}
}
// FetchMinAckPending sets the minimum number of pending acks that should be
// present for a consumer with PriorityPolicyOverflow to be considered for
// delivery. If provided, FetchPriorityGroup must be set as well and the consumer
// has to have PriorityPolicy set to PriorityPolicyOverflow.
func FetchMinAckPending(min int64) FetchOpt {
return func(req *pullRequest) error {
if min < 1 {
return fmt.Errorf("%w: min ack pending should be more than 0", ErrInvalidOption)
}
req.MinAckPending = min
return nil
}
}
// FetchPrioritized sets the priority used when sending fetch requests for consumer with
// PriorityPolicyPrioritized. Lower values indicate higher priority (0 is the
// highest priority). Maximum priority value is 9.
//
// If provided, FetchPriorityGroup must be set as well and the consumer has to
// have PriorityPolicy set to PriorityPolicyPrioritized.
func FetchPrioritized(priority uint8) FetchOpt {
return func(req *pullRequest) error {
req.Priority = priority
return nil
}
}
// FetchPriorityGroup sets the priority group for a consumer.
// It has to match one of the priority groups set on the consumer.
func FetchPriorityGroup(group string) FetchOpt {
return func(req *pullRequest) error {
req.Group = group
return nil
}
}
// FetchMaxWait sets custom timeout for fetching predefined batch of messages.
//
// If not provided, a default of 30 seconds will be used.
func FetchMaxWait(timeout time.Duration) FetchOpt {
return func(req *pullRequest) error {
if timeout <= 0 {
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
}
req.Expires = timeout
req.maxWaitSet = true
return nil
}
}
// FetchHeartbeat sets custom heartbeat for individual fetch request. If a
// client does not receive a heartbeat message from a stream for more than 2
// times the idle heartbeat setting, Fetch will return [ErrNoHeartbeat].
//
// Heartbeat value has to be lower than FetchMaxWait / 2.
//
// If not provided, heartbeat will is set to 5s for requests with FetchMaxWait > 10s
// and disabled otherwise.
func FetchHeartbeat(hb time.Duration) FetchOpt {
return func(req *pullRequest) error {
if hb <= 0 {
return fmt.Errorf("%w: timeout value must be greater than 0", ErrInvalidOption)
}
req.Heartbeat = hb
return nil
}
}
// FetchContext sets a context for the Fetch operation.
// The Fetch operation will be canceled if the context is canceled.
// If the context has a deadline, it will be used to set expiry on pull request.
func FetchContext(ctx context.Context) FetchOpt {
return func(req *pullRequest) error {
req.ctx = ctx
// If context has a deadline, use it to set expiry
if deadline, ok := ctx.Deadline(); ok {
remaining := time.Until(deadline)
if remaining <= 0 {
return fmt.Errorf("%w: context deadline already exceeded", ErrInvalidOption)
}
// Use 90% of remaining time for server (capped at 1s)
buffer := time.Duration(float64(remaining) * 0.1)
if buffer > time.Second {
buffer = time.Second
}
req.Expires = remaining - buffer
}
return nil
}
}
// WithDeletedDetails can be used to display the information about messages
// deleted from a stream on a stream info request
func WithDeletedDetails(deletedDetails bool) StreamInfoOpt {
return func(req *streamInfoRequest) error {
req.DeletedDetails = deletedDetails
return nil
}
}
// WithSubjectFilter can be used to display the information about messages
// stored on given subjects.
// NOTE: if the subject filter matches over 100k
// subjects, this will result in multiple requests to the server to retrieve all
// the information, and all of the returned subjects will be kept in memory.
func WithSubjectFilter(subject string) StreamInfoOpt {
return func(req *streamInfoRequest) error {
req.SubjectFilter = subject
return nil
}
}
// WithStreamListSubject can be used to filter results of ListStreams and
// StreamNames requests to only streams that have given subject in their
// configuration.
func WithStreamListSubject(subject string) StreamListOpt {
return func(req *streamsRequest) error {
req.Subject = subject
return nil
}
}
// WithMsgID sets the message ID used for deduplication.
func WithMsgID(id string) PublishOpt {
return func(opts *pubOpts) error {
opts.id = id
return nil
}
}
// WithMsgTTL sets per msg TTL.
// Requires [StreamConfig.AllowMsgTTL] to be enabled.
func WithMsgTTL(dur time.Duration) PublishOpt {
return func(opts *pubOpts) error {
opts.ttl = dur
return nil
}
}
// WithExpectStream sets the expected stream the message should be published to.
// If the message is published to a different stream server will reject the
// message and publish will fail.
func WithExpectStream(stream string) PublishOpt {
return func(opts *pubOpts) error {
opts.stream = stream
return nil
}
}
// WithExpectLastSequence sets the expected sequence number the last message
// on a stream should have. If the last message has a different sequence number
// server will reject the message and publish will fail.
func WithExpectLastSequence(seq uint64) PublishOpt {
return func(opts *pubOpts) error {
opts.lastSeq = &seq
return nil
}
}
// WithExpectLastSequencePerSubject sets the expected sequence number the last
// message on a subject the message is published to. If the last message on a
// subject has a different sequence number server will reject the message and
// publish will fail.
func WithExpectLastSequencePerSubject(seq uint64) PublishOpt {
return func(opts *pubOpts) error {
opts.lastSubjectSeq = &seq
return nil
}
}
// WithExpectLastSequenceForSubject sets the sequence and subject for which the
// last sequence number should be checked. If the last message on a subject
// has a different sequence number server will reject the message and publish
// will fail.
func WithExpectLastSequenceForSubject(seq uint64, subject string) PublishOpt {
return func(opts *pubOpts) error {
if subject == "" {
return fmt.Errorf("%w: subject cannot be empty", ErrInvalidOption)
}
opts.lastSubjectSeq = &seq
opts.lastSubject = subject
return nil
}
}
// WithExpectLastMsgID sets the expected message ID the last message on a stream
// should have. If the last message has a different message ID server will
// reject the message and publish will fail.
func WithExpectLastMsgID(id string) PublishOpt {
return func(opts *pubOpts) error {
opts.lastMsgID = id
return nil
}
}
// WithRetryWait sets the retry wait time when ErrNoResponders is encountered.
// Defaults to 250ms.
func WithRetryWait(dur time.Duration) PublishOpt {
return func(opts *pubOpts) error {
if dur <= 0 {
return fmt.Errorf("%w: retry wait should be more than 0", ErrInvalidOption)
}
opts.retryWait = dur
return nil
}
}
// WithRetryAttempts sets the retry number of attempts when ErrNoResponders is
// encountered. Defaults to 2
func WithRetryAttempts(num int) PublishOpt {
return func(opts *pubOpts) error {
if num < 0 {
return fmt.Errorf("%w: retry attempts cannot be negative", ErrInvalidOption)
}
opts.retryAttempts = num
return nil
}
}
// WithStallWait sets the max wait when the producer becomes stall producing
// messages. If a publish call is blocked for this long, ErrTooManyStalledMsgs
// is returned.
func WithStallWait(ttl time.Duration) PublishOpt {
return func(opts *pubOpts) error {
if ttl <= 0 {
return fmt.Errorf("%w: stall wait should be more than 0", ErrInvalidOption)
}
opts.stallWait = ttl
return nil
}
}
// Predefined schedule constants for use with [WithScheduleCron].
const (
ScheduleYearly = "@yearly"
ScheduleMonthly = "@monthly"
ScheduleWeekly = "@weekly"
ScheduleDaily = "@daily"
ScheduleHourly = "@hourly"
)
// WithScheduleAt schedules a message for one-time delivery at a specific time.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleAt(t time.Time) PublishOpt {
return func(opts *pubOpts) error {
opts.schedule = "@at " + t.UTC().Format(time.RFC3339)
return nil
}
}
// WithScheduleEvery schedules a message for repeated delivery at the given
// interval. The minimum interval is 1 second.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleEvery(d time.Duration) PublishOpt {
return func(opts *pubOpts) error {
if d < time.Second {
return fmt.Errorf("%w: schedule interval must be at least 1s", ErrInvalidOption)
}
opts.schedule = "@every " + d.String()
return nil
}
}
// WithScheduleCron schedules a message using a cron expression.
// Accepts 6-field cron format (seconds minutes hours day-of-month month
// day-of-week) or predefined schedules (e.g. [ScheduleHourly],
// [ScheduleDaily]).
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleCron(expr string) PublishOpt {
return func(opts *pubOpts) error {
if expr == "" {
return fmt.Errorf("%w: cron expression cannot be empty", ErrInvalidOption)
}
opts.schedule = expr
return nil
}
}
// WithScheduleTarget sets the subject where scheduled messages will be
// delivered.
// This option is required for scheduled messages.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleTarget(subject string) PublishOpt {
return func(opts *pubOpts) error {
if subject == "" {
return fmt.Errorf("%w: schedule target subject cannot be empty", ErrInvalidOption)
}
opts.scheduleTarget = subject
return nil
}
}
// WithScheduleSource instructs the scheduler to read the latest message
// from the given subject and republish it on schedule. Used for data
// sampling use cases.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleSource(subject string) PublishOpt {
return func(opts *pubOpts) error {
if subject == "" {
return fmt.Errorf("%w: schedule source subject cannot be empty", ErrInvalidOption)
}
opts.scheduleSource = subject
return nil
}
}
// WithScheduleTTL sets the TTL on messages generated by the scheduler.
// Requires [StreamConfig.AllowMsgTTL] to be enabled on the stream.
func WithScheduleTTL(d time.Duration) PublishOpt {
return func(opts *pubOpts) error {
if d <= 0 {
return fmt.Errorf("%w: schedule TTL must be greater than 0", ErrInvalidOption)
}
opts.scheduleTTL = d.String()
return nil
}
}
// WithScheduleTTLNever marks messages generated by the scheduler as never
// expiring. This overrides the stream's MaxAge for delivered messages.
// Requires [StreamConfig.AllowMsgTTL] to be enabled on the stream.
func WithScheduleTTLNever() PublishOpt {
return func(opts *pubOpts) error {
opts.scheduleTTL = "never"
return nil
}
}
// WithScheduleTimeZone sets the time zone for cron schedule expressions.
// The zone must be a valid IANA time zone name (e.g., "America/New_York").
// Only valid with cron schedule expressions; the server will reject this
// header if used with @at or @every schedules.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleTimeZone(zone string) PublishOpt {
return func(opts *pubOpts) error {
if zone == "" {
return fmt.Errorf("%w: schedule time zone cannot be empty", ErrInvalidOption)
}
opts.scheduleTZ = zone
return nil
}
}
// WithScheduleRollup sets a per-subject rollup on messages generated by the
// scheduler, so each delivery purges the previous message on the target
// subject.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
func WithScheduleRollup() PublishOpt {
return func(opts *pubOpts) error {
opts.scheduleRollup = MsgRollupSubject
return nil
}
}
type nextOptFunc func(*nextOpts)
func (fn nextOptFunc) configureNext(opts *nextOpts) {
fn(opts)
}
// NextMaxWait sets a timeout for the Next operation.
// If the timeout is reached before a message is available, a timeout error is returned.
func NextMaxWait(timeout time.Duration) NextOpt {
return nextOptFunc(func(opts *nextOpts) {
opts.timeout = timeout
})
}
// NextContext sets a context for the Next operation.
// The Next operation will be canceled if the context is canceled.
func NextContext(ctx context.Context) NextOpt {
return nextOptFunc(func(opts *nextOpts) {
opts.ctx = ctx
})
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/kv.go
Generated Vendored
+1622
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/kv_options.go
Generated Vendored
+131
View File
@@ -0,0 +1,131 @@
// Copyright 2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"fmt"
"time"
)
type watchOptFn func(opts *watchOpts) error
func (opt watchOptFn) configureWatcher(opts *watchOpts) error {
return opt(opts)
}
// IncludeHistory instructs the key watcher to include historical values as
// well (up to KeyValueMaxHistory).
func IncludeHistory() WatchOpt {
return watchOptFn(func(opts *watchOpts) error {
if opts.updatesOnly {
return fmt.Errorf("%w: include history cannot be used with updates only", ErrInvalidOption)
}
opts.includeHistory = true
return nil
})
}
// UpdatesOnly instructs the key watcher to only include updates on values
// (without latest values when started).
func UpdatesOnly() WatchOpt {
return watchOptFn(func(opts *watchOpts) error {
if opts.includeHistory {
return fmt.Errorf("%w: updates only cannot be used with include history", ErrInvalidOption)
}
opts.updatesOnly = true
return nil
})
}
// IgnoreDeletes will prevent the key watcher from passing any deleted keys.
func IgnoreDeletes() WatchOpt {
return watchOptFn(func(opts *watchOpts) error {
opts.ignoreDeletes = true
return nil
})
}
// MetaOnly instructs the key watcher to retrieve only the entry metadata, not
// the entry value.
func MetaOnly() WatchOpt {
return watchOptFn(func(opts *watchOpts) error {
opts.metaOnly = true
return nil
})
}
// ResumeFromRevision instructs the key watcher to resume from a specific
// revision number.
func ResumeFromRevision(revision uint64) WatchOpt {
return watchOptFn(func(opts *watchOpts) error {
opts.resumeFromRevision = revision
return nil
})
}
// DeleteMarkersOlderThan indicates that delete or purge markers older than that
// will be deleted as part of [KeyValue.PurgeDeletes] operation, otherwise, only the data
// will be removed but markers that are recent will be kept.
// Note that if no option is specified, the default is 30 minutes. You can set
// this option to a negative value to instruct to always remove the markers,
// regardless of their age.
type DeleteMarkersOlderThan time.Duration
func (ttl DeleteMarkersOlderThan) configurePurge(opts *purgeOpts) error {
opts.dmthr = time.Duration(ttl)
return nil
}
type deleteOptFn func(opts *deleteOpts) error
func (opt deleteOptFn) configureDelete(opts *deleteOpts) error {
return opt(opts)
}
// LastRevision deletes if the latest revision matches the provided one. If the
// provided revision is not the latest, the delete will return an error.
func LastRevision(revision uint64) KVDeleteOpt {
return deleteOptFn(func(opts *deleteOpts) error {
opts.revision = revision
return nil
})
}
// PurgeTTL sets the TTL for the purge operation.
// After the TTL expires, the delete markers will be removed.
// This requires LimitMarkerTTL to be enabled on the bucket.
// Note that this is not the same as the TTL for the key itself, which is set
// using the KeyTTL option when creating the key.
func PurgeTTL(ttl time.Duration) KVDeleteOpt {
return deleteOptFn(func(opts *deleteOpts) error {
opts.ttl = ttl
return nil
})
}
type createOptFn func(opts *createOpts) error
func (opt createOptFn) configureCreate(opts *createOpts) error {
return opt(opts)
}
// KeyTTL sets the TTL for the key. This is the time after which the key will be
// automatically deleted. The TTL is set when the key is created and cannot be
// changed later. This requires LimitMarkerTTL to be enabled on the bucket.
func KeyTTL(ttl time.Duration) KVCreateOpt {
return createOptFn(func(opts *createOpts) error {
opts.ttl = ttl
return nil
})
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/message.go
Generated Vendored
+541
View File
@@ -0,0 +1,541 @@
// Copyright 2022-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"bytes"
"context"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/internal/parser"
)
type (
// Msg contains methods to operate on a JetStream message. Metadata, Data,
// Headers, Subject and Reply can be used to retrieve the specific parts of
// the underlying message. Ack, DoubleAck, Nak, NakWithDelay, InProgress and
// Term are various flavors of ack requests.
Msg interface {
// Metadata returns [MsgMetadata] for a JetStream message.
Metadata() (*MsgMetadata, error)
// Data returns the message body.
Data() []byte
// Headers returns a map of headers for a message.
Headers() nats.Header
// Subject returns a subject on which a message was published/received.
Subject() string
// Reply returns a reply subject for a message.
Reply() string
// Ack acknowledges a message. This tells the server that the message was
// successfully processed and it can move on to the next message.
Ack() error
// DoubleAck acknowledges a message and waits for ack reply from the server.
// While it impacts performance, it is useful for scenarios where
// message loss is not acceptable.
DoubleAck(context.Context) error
// Nak negatively acknowledges a message. This tells the server to
// redeliver the message.
//
// Nak does not adhere to AckWait or Backoff configured on the consumer
// and triggers instant redelivery. For a delayed redelivery, use
// NakWithDelay.
Nak() error
// NakWithDelay negatively acknowledges a message. This tells the server
// to redeliver the message after the given delay.
NakWithDelay(delay time.Duration) error
// InProgress tells the server that this message is being worked on. It
// resets the redelivery timer on the server.
InProgress() error
// Term tells the server to not redeliver this message, regardless of
// the value of MaxDeliver.
Term() error
// TermWithReason tells the server to not redeliver this message, regardless of
// the value of MaxDeliver. The provided reason will be included in JetStream
// advisory event sent by the server.
//
// Note: This will only work with JetStream servers >= 2.10.4.
// For older servers, TermWithReason will be ignored by the server and the message
// will not be terminated.
TermWithReason(reason string) error
}
// MsgMetadata is the JetStream metadata associated with received messages.
MsgMetadata struct {
// Sequence is the sequence information for the message.
Sequence SequencePair
// NumDelivered is the number of times this message was delivered to the
// consumer.
NumDelivered uint64
// NumPending is the number of messages that match the consumer's
// filter, but have not been delivered yet.
NumPending uint64
// Timestamp is the time the message was originally stored on a stream.
Timestamp time.Time
// Stream is the stream name this message is stored on.
Stream string
// Consumer is the consumer name this message was delivered to.
Consumer string
// Domain is the domain this message was received on.
Domain string
}
// SequencePair includes the consumer and stream sequence numbers for a
// message.
SequencePair struct {
// Consumer is the consumer sequence number for message deliveries. This
// is the total number of messages the consumer has seen (including
// redeliveries).
Consumer uint64 `json:"consumer_seq"`
// Stream is the stream sequence number for a message.
Stream uint64 `json:"stream_seq"`
}
jetStreamMsg struct {
msg *nats.Msg
ackd bool
js *jetStream
sync.Mutex
}
ackOpts struct {
nakDelay time.Duration
termReason string
}
ackType []byte
)
const (
statusControlMsg = "100"
statusBadRequest = "400"
statusNoMsgs = "404"
statusTimeout = "408"
statusConflict = "409"
statusNoResponders = "503"
statusPinIdMismatch = "423"
fcRequestDescr = "flowcontrol request"
idleHeartbeatDescr = "idle heartbeat"
consumerDeleted = "consumer deleted"
leadershipChange = "leadership change"
maxBytesExceeded = "message size exceeds maxbytes"
batchCompleted = "batch completed"
serverShutdown = "server shutdown"
)
// Headers used when publishing messages.
const (
// MsgIdHeader is used to specify a user-defined message ID. It can be used
// e.g. for deduplication in conjunction with the Duplicates duration on
// ConsumerConfig or to provide optimistic concurrency safety together with
// [ExpectedLastMsgIDHeader].
//
// This can be set when publishing messages using [WithMsgID] option.
MsgIDHeader = "Nats-Msg-Id"
// ExpectedStreamHeader contains stream name and is used to assure that the
// published message is received by expected stream. Server will reject the
// message if it is not the case.
//
// This can be set when publishing messages using [WithExpectStream] option.
ExpectedStreamHeader = "Nats-Expected-Stream"
// ExpectedLastSeqHeader contains the expected last sequence number of the
// stream and can be used to apply optimistic concurrency control at stream
// level. Server will reject the message if it is not the case.
//
// This can be set when publishing messages using [WithExpectLastSequence]
// option.
ExpectedLastSeqHeader = "Nats-Expected-Last-Sequence"
// ExpectedLastSubjSeqHeader contains the expected last sequence number on
// the subject and can be used to apply optimistic concurrency control at
// subject level. Server will reject the message if it is not the case.
//
// This can be set when publishing messages using
// [WithExpectLastSequencePerSubject] option.
ExpectedLastSubjSeqHeader = "Nats-Expected-Last-Subject-Sequence"
// ExpectedLastSubjSeqSubjHeader contains the subject for which the
// expected last sequence number is set. This is used together with
// [ExpectedLastSubjSeqHeader] to apply optimistic concurrency control at
// subject level. Server will reject the message if it is not the case.
ExpectedLastSubjSeqSubjHeader = "Nats-Expected-Last-Subject-Sequence-Subject"
// ExpectedLastMsgIDHeader contains the expected last message ID on the
// subject and can be used to apply optimistic concurrency control at
// stream level. Server will reject the message if it is not the case.
//
// This can be set when publishing messages using [WithExpectLastMsgID]
// option.
ExpectedLastMsgIDHeader = "Nats-Expected-Last-Msg-Id"
// MsgTTLHeader is used to specify the TTL for a specific message. This will
// override the default TTL for the stream.
MsgTTLHeader = "Nats-TTL"
// MsgRollup is used to apply a purge of all prior messages in the stream
// ("all") or at the subject ("sub") before this message.
MsgRollup = "Nats-Rollup"
// MarkerReasonHeader is used to specify a reason for message deletion.
MarkerReasonHeader = "Nats-Marker-Reason"
// ScheduleHeader is used to specify a schedule expression for a message.
// Valid values include "@at <RFC3339>", "@every <duration>", 6-field cron
// expressions, and predefined schedules (@yearly, @monthly, @weekly,
// @daily, @hourly).
//
// This can be set when publishing messages using [WithScheduleAt],
// [WithScheduleEvery], or [WithScheduleCron] options.
// Requires [StreamConfig.AllowMsgSchedules] to be enabled.
ScheduleHeader = "Nats-Schedule"
// ScheduleTargetHeader specifies the subject where scheduled messages
// will be delivered.
//
// This can be set when publishing messages using [WithScheduleTarget] option.
ScheduleTargetHeader = "Nats-Schedule-Target"
// ScheduleSourceHeader instructs the scheduler to read the latest message
// from the given subject and republish it on schedule. Used for data
// sampling use cases.
//
// This can be set when publishing messages using [WithScheduleSource] option.
ScheduleSourceHeader = "Nats-Schedule-Source"
// ScheduleTTLHeader sets a TTL on messages generated by the scheduler.
// The server default is "5m". Use "never" to disable TTL on generated
// messages.
//
// This can be set when publishing messages using [WithScheduleTTL] or
// [WithScheduleTTLNever] options.
// Requires [StreamConfig.AllowMsgTTL] to be enabled on the stream.
ScheduleTTLHeader = "Nats-Schedule-TTL"
// ScheduleTimeZoneHeader sets the time zone for cron schedule expressions.
// Must be a valid IANA time zone name (e.g., "America/New_York").
// Only valid with cron schedule expressions.
//
// This can be set when publishing messages using [WithScheduleTimeZone] option.
ScheduleTimeZoneHeader = "Nats-Schedule-Time-Zone"
// ScheduleRollupHeader sets a rollup on messages generated by the
// scheduler. Only [MsgRollupSubject] is a valid value.
//
// This can be set when publishing messages using [WithScheduleRollup] option.
ScheduleRollupHeader = "Nats-Schedule-Rollup"
)
// Headers for republished messages and direct gets. Those headers are set by
// the server and should not be set by the client.
const (
// StreamHeader contains the stream name the message was republished from or
// the stream name the message was retrieved from using direct get.
StreamHeader = "Nats-Stream"
// SequenceHeader contains the original sequence number of the message.
SequenceHeader = "Nats-Sequence"
// TimeStampHeaer contains the original timestamp of the message.
TimeStampHeaer = "Nats-Time-Stamp"
// SubjectHeader contains the original subject the message was published to.
SubjectHeader = "Nats-Subject"
// SchedulerHeader is set by the server on messages generated by a schedule.
// Contains the subject holding the schedule definition.
SchedulerHeader = "Nats-Scheduler"
// ScheduleNextHeader is set by the server on generated messages. Contains
// the timestamp of the next invocation, or "purge" for one-time schedules.
ScheduleNextHeader = "Nats-Schedule-Next"
// LastSequenceHeader contains the last sequence of the message having the
// same subject, otherwise zero if this is the first message for the
// subject.
LastSequenceHeader = "Nats-Last-Sequence"
)
// Rollups, can be subject only or all messages.
const (
// MsgRollupSubject is used to purge all messages before this message on the
// message subject.
MsgRollupSubject = "sub"
// MsgRollupAll is used to purge all messages before this message on the
// stream.
MsgRollupAll = "all"
)
var (
ackAck ackType = []byte("+ACK")
ackNak ackType = []byte("-NAK")
ackProgress ackType = []byte("+WPI")
ackTerm ackType = []byte("+TERM")
)
// Metadata returns [MsgMetadata] for a JetStream message.
func (m *jetStreamMsg) Metadata() (*MsgMetadata, error) {
if err := m.checkReply(); err != nil {
return nil, err
}
tokens, err := parser.GetMetadataFields(m.msg.Reply)
if err != nil {
return nil, fmt.Errorf("%w: %s", ErrNotJSMessage, err)
}
meta := &MsgMetadata{
Domain: tokens[parser.AckDomainTokenPos],
NumDelivered: parser.ParseNum(tokens[parser.AckNumDeliveredTokenPos]),
NumPending: parser.ParseNum(tokens[parser.AckNumPendingTokenPos]),
Timestamp: time.Unix(0, int64(parser.ParseNum(tokens[parser.AckTimestampSeqTokenPos]))),
Stream: tokens[parser.AckStreamTokenPos],
Consumer: tokens[parser.AckConsumerTokenPos],
}
meta.Sequence.Stream = parser.ParseNum(tokens[parser.AckStreamSeqTokenPos])
meta.Sequence.Consumer = parser.ParseNum(tokens[parser.AckConsumerSeqTokenPos])
return meta, nil
}
// Data returns the message body.
func (m *jetStreamMsg) Data() []byte {
return m.msg.Data
}
// Headers returns a map of headers for a message.
func (m *jetStreamMsg) Headers() nats.Header {
return m.msg.Header
}
// Subject returns a subject on which a message was published/received.
func (m *jetStreamMsg) Subject() string {
return m.msg.Subject
}
// Reply returns a reply subject for a JetStream message.
func (m *jetStreamMsg) Reply() string {
return m.msg.Reply
}
// Ack acknowledges a message. This tells the server that the message was
// successfully processed and it can move on to the next message.
func (m *jetStreamMsg) Ack() error {
return m.ackReply(context.Background(), ackAck, false, ackOpts{})
}
// DoubleAck acknowledges a message and waits for ack reply from the server.
// While it impacts performance, it is useful for scenarios where
// message loss is not acceptable.
func (m *jetStreamMsg) DoubleAck(ctx context.Context) error {
return m.ackReply(ctx, ackAck, true, ackOpts{})
}
// Nak negatively acknowledges a message. This tells the server to
// redeliver the message.
func (m *jetStreamMsg) Nak() error {
return m.ackReply(context.Background(), ackNak, false, ackOpts{})
}
// NakWithDelay negatively acknowledges a message. This tells the server
// to redeliver the message after the given delay.
func (m *jetStreamMsg) NakWithDelay(delay time.Duration) error {
return m.ackReply(context.Background(), ackNak, false, ackOpts{nakDelay: delay})
}
// InProgress tells the server that this message is being worked on. It
// resets the redelivery timer on the server.
func (m *jetStreamMsg) InProgress() error {
return m.ackReply(context.Background(), ackProgress, false, ackOpts{})
}
// Term tells the server to not redeliver this message, regardless of
// the value of MaxDeliver.
func (m *jetStreamMsg) Term() error {
return m.ackReply(context.Background(), ackTerm, false, ackOpts{})
}
// TermWithReason tells the server to not redeliver this message, regardless of
// the value of MaxDeliver. The provided reason will be included in JetStream
// advisory event sent by the server.
//
// Note: This will only work with JetStream servers >= 2.10.4.
// For older servers, TermWithReason will be ignored by the server and the message
// will not be terminated.
func (m *jetStreamMsg) TermWithReason(reason string) error {
return m.ackReply(context.Background(), ackTerm, false, ackOpts{termReason: reason})
}
func (m *jetStreamMsg) ackReply(ctx context.Context, ackType ackType, sync bool, opts ackOpts) error {
err := m.checkReply()
if err != nil {
return err
}
m.Lock()
if m.ackd {
m.Unlock()
return ErrMsgAlreadyAckd
}
m.Unlock()
if sync {
var cancel context.CancelFunc
ctx, cancel = m.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
}
var body []byte
if opts.nakDelay > 0 {
body = []byte(fmt.Sprintf("%s {\"delay\": %d}", ackType, opts.nakDelay.Nanoseconds()))
} else if opts.termReason != "" {
body = []byte(fmt.Sprintf("%s %s", ackType, opts.termReason))
} else {
body = ackType
}
if sync {
_, err = m.js.conn.RequestWithContext(ctx, m.msg.Reply, body)
} else {
err = m.js.conn.Publish(m.msg.Reply, body)
}
if err != nil {
return err
}
// Mark that the message has been acked unless it is ackProgress
// which can be sent many times.
if !bytes.Equal(ackType, ackProgress) {
m.Lock()
m.ackd = true
m.Unlock()
}
return nil
}
func (m *jetStreamMsg) checkReply() error {
if m == nil || m.msg.Sub == nil {
return ErrMsgNotBound
}
if m.msg.Reply == "" {
return ErrMsgNoReply
}
return nil
}
// checkMsg returns whether the given message is a user message or a control message.
// If the status header is present, it returns an appropriate error based
// on the status code (404, etc.)
func checkMsg(msg *nats.Msg) (bool, error) {
// If payload or no header, consider this a user message
if len(msg.Data) > 0 || len(msg.Header) == 0 {
return true, nil
}
// Look for status header
val := msg.Header.Get("Status")
descr := msg.Header.Get("Description")
// If not present, then this is considered a user message
if val == "" {
return true, nil
}
switch val {
case statusBadRequest:
return false, ErrBadRequest
case statusNoResponders:
return false, nats.ErrNoResponders
case statusNoMsgs:
// 404 indicates that there are no messages.
return false, ErrNoMessages
case statusTimeout:
return false, nats.ErrTimeout
case statusControlMsg:
return false, nil
case statusPinIdMismatch:
return false, ErrPinIDMismatch
case statusConflict:
if strings.Contains(strings.ToLower(descr), maxBytesExceeded) {
return false, ErrMaxBytesExceeded
}
if strings.Contains(strings.ToLower(descr), batchCompleted) {
return false, ErrBatchCompleted
}
if strings.Contains(strings.ToLower(descr), consumerDeleted) {
return false, ErrConsumerDeleted
}
if strings.Contains(strings.ToLower(descr), leadershipChange) {
return false, ErrConsumerLeadershipChanged
}
if strings.Contains(strings.ToLower(descr), serverShutdown) {
return false, ErrServerShutdown
}
}
return false, fmt.Errorf("nats: %s", msg.Header.Get("Description"))
}
func parsePending(msg *nats.Msg) (int, int, error) {
msgsLeftStr := msg.Header.Get("Nats-Pending-Messages")
var msgsLeft int
var err error
if msgsLeftStr != "" {
msgsLeft, err = strconv.Atoi(msgsLeftStr)
if err != nil {
return 0, 0, errors.New("nats: invalid format of Nats-Pending-Messages")
}
}
bytesLeftStr := msg.Header.Get("Nats-Pending-Bytes")
var bytesLeft int
if bytesLeftStr != "" {
bytesLeft, err = strconv.Atoi(bytesLeftStr)
if err != nil {
return 0, 0, errors.New("nats: invalid format of Nats-Pending-Bytes")
}
}
return msgsLeft, bytesLeft, nil
}
// toJSMsg converts core [nats.Msg] to [jetStreamMsg], exposing JetStream-specific operations
func (js *jetStream) toJSMsg(msg *nats.Msg) *jetStreamMsg {
return &jetStreamMsg{
msg: msg,
js: js,
}
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/object.go
Generated Vendored
+1633
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/object_options.go
Generated Vendored
+41
View File
@@ -0,0 +1,41 @@
// Copyright 2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
// GetObjectShowDeleted makes [ObjectStore.Get] return object even if it was
// marked as deleted.
func GetObjectShowDeleted() GetObjectOpt {
return func(opts *getObjectOpts) error {
opts.showDeleted = true
return nil
}
}
// GetObjectInfoShowDeleted makes [ObjectStore.GetInfo] return object info event
// if it was marked as deleted.
func GetObjectInfoShowDeleted() GetObjectInfoOpt {
return func(opts *getObjectInfoOpts) error {
opts.showDeleted = true
return nil
}
}
// ListObjectsShowDeleted makes [ObjectStore.ListObjects] also return deleted
// objects.
func ListObjectsShowDeleted() ListObjectsOpt {
return func(opts *listObjectOpts) error {
opts.showDeleted = true
return nil
}
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/ordered.go
Generated Vendored
+834
View File
@@ -0,0 +1,834 @@
// Copyright 2022-2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/nats-io/nats.go"
)
type (
orderedConsumer struct {
js *jetStream
cfg *OrderedConsumerConfig
stream string
currentConsumer *pullConsumer
currentSub *pullSubscription
cursor cursor
namePrefix string
serial int
consumerType consumerType
doReset chan struct{}
resetInProgress atomic.Uint32
userErrHandler ConsumeErrHandler
stopAfter int
stopAfterMsgsLeft chan int
withStopAfter bool
runningFetch *fetchResult
subscription *orderedSubscription
sync.Mutex
}
orderedSubscription struct {
consumer *orderedConsumer
opts []PullMessagesOpt
done chan struct{}
closed atomic.Uint32
}
cursor struct {
streamSeq uint64
deliverSeq uint64
}
consumerType int
)
const (
consumerTypeNotSet consumerType = iota
consumerTypeConsume
consumerTypeFetch
)
var (
errOrderedSequenceMismatch = errors.New("sequence mismatch")
errOrderedConsumerClosed = errors.New("ordered consumer closed")
)
// Consume can be used to continuously receive messages and handle them
// with the provided callback function. Consume cannot be used concurrently
// when using ordered consumer.
//
// See [Consumer.Consume] for more details.
func (c *orderedConsumer) Consume(handler MessageHandler, opts ...PullConsumeOpt) (ConsumeContext, error) {
c.Lock()
defer c.Unlock()
if (c.consumerType == consumerTypeNotSet || c.consumerType == consumerTypeConsume) && c.currentConsumer == nil {
err := c.reset()
if err != nil {
return nil, err
}
} else if c.consumerType == consumerTypeConsume && c.currentConsumer != nil {
return nil, ErrOrderedConsumerConcurrentRequests
}
if c.consumerType == consumerTypeFetch {
return nil, ErrOrderConsumerUsedAsFetch
}
c.consumerType = consumerTypeConsume
consumeOpts, err := parseConsumeOpts(true, opts...)
if err != nil {
return nil, fmt.Errorf("%w: %s", ErrInvalidOption, err)
}
c.userErrHandler = consumeOpts.ErrHandler
opts = append(opts, consumeReconnectNotify(),
ConsumeErrHandler(c.errHandler(c.serial)))
if consumeOpts.StopAfter > 0 {
c.withStopAfter = true
c.stopAfter = consumeOpts.StopAfter
}
c.stopAfterMsgsLeft = make(chan int, 1)
if c.stopAfter > 0 {
opts = append(opts, consumeStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
}
sub := &orderedSubscription{
consumer: c,
done: make(chan struct{}, 1),
}
c.subscription = sub
internalHandler := func(serial int) func(msg Msg) {
return func(msg Msg) {
c.Lock()
// handler is a noop if message was delivered for a consumer with different serial
if serial != c.serial {
c.Unlock()
return
}
meta, err := msg.Metadata()
if err != nil {
currentSub := c.currentSub
c.Unlock()
c.errHandler(serial)(currentSub, err)
return
}
dseq := meta.Sequence.Consumer
if dseq != c.cursor.deliverSeq+1 {
c.Unlock()
c.errHandler(serial)(sub, errOrderedSequenceMismatch)
return
}
c.cursor.deliverSeq = dseq
c.cursor.streamSeq = meta.Sequence.Stream
c.Unlock()
handler(msg)
}
}
cc, err := c.currentConsumer.Consume(internalHandler(c.serial), opts...)
if err != nil {
return nil, err
}
c.currentSub = cc.(*pullSubscription)
go func() {
for {
select {
case <-c.doReset:
if err := c.reset(); err != nil {
if errors.Is(err, errOrderedConsumerClosed) {
continue
}
c.errHandler(c.serial)(c.currentSub, err)
}
if c.withStopAfter {
select {
case c.stopAfter = <-c.stopAfterMsgsLeft:
default:
}
if c.stopAfter <= 0 {
sub.Stop()
return
}
}
if c.stopAfter > 0 {
opts = opts[:len(opts)-2]
} else {
opts = opts[:len(opts)-1]
}
// overwrite the previous err handler to use the new serial
opts = append(opts, ConsumeErrHandler(c.errHandler(c.serial)))
if c.withStopAfter {
opts = append(opts, consumeStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
}
if cc, err := c.currentConsumer.Consume(internalHandler(c.serial), opts...); err != nil {
c.errHandler(c.serial)(cc, err)
} else {
c.Lock()
c.currentSub = cc.(*pullSubscription)
c.Unlock()
}
case <-sub.done:
s := sub.consumer.currentSub
if s != nil {
sub.consumer.Lock()
s.Stop()
sub.consumer.Unlock()
}
return
case msgsLeft, ok := <-c.stopAfterMsgsLeft:
if !ok {
close(sub.done)
}
c.stopAfter = msgsLeft
return
}
}
}()
return sub, nil
}
func (c *orderedConsumer) errHandler(serial int) func(cc ConsumeContext, err error) {
return func(cc ConsumeContext, err error) {
c.Lock()
if c.userErrHandler != nil && !errors.Is(err, errOrderedSequenceMismatch) && !errors.Is(err, errConnected) {
c.userErrHandler(cc, err)
}
if errors.Is(err, ErrConnectionClosed) {
if c.subscription != nil {
c.Unlock()
c.subscription.Stop()
return
}
c.Unlock()
return
}
if errors.Is(err, ErrNoHeartbeat) ||
errors.Is(err, errOrderedSequenceMismatch) ||
errors.Is(err, ErrConsumerDeleted) ||
errors.Is(err, errConnected) ||
errors.Is(err, nats.ErrNoResponders) {
// only reset if serial matches the current consumer serial and there is no reset in progress
if serial == c.serial && c.resetInProgress.Load() == 0 {
c.resetInProgress.Store(1)
c.doReset <- struct{}{}
}
}
c.Unlock()
}
}
// Messages returns MessagesContext, allowing continuously iterating
// over messages on a stream. Messages cannot be used concurrently
// when using ordered consumer.
//
// See [Consumer.Messages] for more details.
func (c *orderedConsumer) Messages(opts ...PullMessagesOpt) (MessagesContext, error) {
if (c.consumerType == consumerTypeNotSet || c.consumerType == consumerTypeConsume) && c.currentConsumer == nil {
err := c.reset()
if err != nil {
return nil, err
}
} else if c.consumerType == consumerTypeConsume && c.currentConsumer != nil {
return nil, ErrOrderedConsumerConcurrentRequests
}
if c.consumerType == consumerTypeFetch {
return nil, ErrOrderConsumerUsedAsFetch
}
c.consumerType = consumerTypeConsume
consumeOpts, err := parseMessagesOpts(true, opts...)
if err != nil {
return nil, fmt.Errorf("%w: %s", ErrInvalidOption, err)
}
opts = append(opts,
WithMessagesErrOnMissingHeartbeat(true),
messagesReconnectNotify())
c.stopAfterMsgsLeft = make(chan int, 1)
if consumeOpts.StopAfter > 0 {
c.withStopAfter = true
c.stopAfter = consumeOpts.StopAfter
}
c.userErrHandler = consumeOpts.ErrHandler
if c.stopAfter > 0 {
opts = append(opts, messagesStopAfterNotify(c.stopAfter, c.stopAfterMsgsLeft))
}
cc, err := c.currentConsumer.Messages(opts...)
if err != nil {
return nil, err
}
c.currentSub = cc.(*pullSubscription)
sub := &orderedSubscription{
consumer: c,
opts: opts,
done: make(chan struct{}, 1),
}
c.subscription = sub
return sub, nil
}
func (s *orderedSubscription) Next(opts ...NextOpt) (Msg, error) {
for {
msg, err := s.consumer.currentSub.Next(opts...)
if err != nil {
// Check for errors which should be returned directly
// without resetting the consumer
if errors.Is(err, ErrInvalidOption) {
return nil, err
}
if errors.Is(err, nats.ErrTimeout) {
return nil, err
}
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return nil, err
}
if errors.Is(err, ErrMsgIteratorClosed) {
s.Stop()
return nil, err
}
if s.consumer.withStopAfter {
select {
case s.consumer.stopAfter = <-s.consumer.stopAfterMsgsLeft:
default:
}
if s.consumer.stopAfter <= 0 {
s.Stop()
return nil, ErrMsgIteratorClosed
}
s.opts[len(s.opts)-1] = StopAfter(s.consumer.stopAfter)
}
if err := s.consumer.reset(); err != nil {
if errors.Is(err, errOrderedConsumerClosed) {
return nil, ErrMsgIteratorClosed
}
return nil, err
}
cc, err := s.consumer.currentConsumer.Messages(s.opts...)
if err != nil {
return nil, err
}
s.consumer.currentSub = cc.(*pullSubscription)
continue
}
meta, err := msg.Metadata()
if err != nil {
return nil, err
}
serial := serialNumberFromConsumer(meta.Consumer)
if serial != s.consumer.serial {
continue
}
dseq := meta.Sequence.Consumer
if dseq != s.consumer.cursor.deliverSeq+1 {
if err := s.consumer.reset(); err != nil {
if errors.Is(err, errOrderedConsumerClosed) {
return nil, ErrMsgIteratorClosed
}
return nil, err
}
cc, err := s.consumer.currentConsumer.Messages(s.opts...)
if err != nil {
return nil, err
}
s.consumer.currentSub = cc.(*pullSubscription)
continue
}
s.consumer.cursor.deliverSeq = dseq
s.consumer.cursor.streamSeq = meta.Sequence.Stream
return msg, nil
}
}
func (s *orderedSubscription) Stop() {
if !s.closed.CompareAndSwap(0, 1) {
return
}
s.consumer.Lock()
defer s.consumer.Unlock()
if s.consumer.currentSub != nil {
s.consumer.currentSub.Stop()
}
close(s.done)
}
func (s *orderedSubscription) Drain() {
if !s.closed.CompareAndSwap(0, 1) {
return
}
s.consumer.Lock()
defer s.consumer.Unlock()
if s.consumer.currentSub != nil {
s.consumer.currentSub.Drain()
}
close(s.done)
}
// Closed returns a channel that is closed when the consuming is
// fully stopped/drained. When the channel is closed, no more messages
// will be received and processing is complete.
func (s *orderedSubscription) Closed() <-chan struct{} {
closedCh := make(chan struct{})
go func() {
// First wait for s.done to be closed
<-s.done
// Then ensure underlying consumer is also closed (it may still be draining)
s.consumer.Lock()
if s.consumer.currentSub != nil {
closed := s.consumer.currentSub.Closed()
s.consumer.Unlock()
<-closed
} else {
s.consumer.Unlock()
}
close(closedCh)
}()
return closedCh
}
// Fetch is used to retrieve up to a provided number of messages from a
// stream. This method will always send a single request and wait until
// either all messages are retrieved or request times out.
//
// It is not efficient to use Fetch with on an ordered consumer, as it will
// reset the consumer for each subsequent Fetch call.
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
func (c *orderedConsumer) Fetch(batch int, opts ...FetchOpt) (MessageBatch, error) {
c.Lock()
if c.consumerType == consumerTypeConsume {
c.Unlock()
return nil, ErrOrderConsumerUsedAsConsume
}
if c.runningFetch != nil {
if !c.runningFetch.closed() {
return nil, ErrOrderedConsumerConcurrentRequests
}
if c.runningFetch.sseq != 0 {
c.cursor.streamSeq = c.runningFetch.sseq
}
}
c.consumerType = consumerTypeFetch
sub := orderedSubscription{
consumer: c,
done: make(chan struct{}),
}
c.subscription = &sub
c.Unlock()
err := c.reset()
if err != nil {
return nil, err
}
msgs, err := c.currentConsumer.Fetch(batch, opts...)
if err != nil {
return nil, err
}
c.runningFetch = msgs.(*fetchResult)
return msgs, nil
}
// FetchBytes is used to retrieve up to a provided bytes from the
// stream. This method will always send a single request and wait until
// provided number of bytes is exceeded or request times out.
//
// It is not efficient to use FetchBytes with on an ordered consumer, as it will
// reset the consumer for each subsequent Fetch call.
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
func (c *orderedConsumer) FetchBytes(maxBytes int, opts ...FetchOpt) (MessageBatch, error) {
c.Lock()
if c.consumerType == consumerTypeConsume {
c.Unlock()
return nil, ErrOrderConsumerUsedAsConsume
}
if c.runningFetch != nil {
if !c.runningFetch.closed() {
return nil, ErrOrderedConsumerConcurrentRequests
}
if c.runningFetch.sseq != 0 {
c.cursor.streamSeq = c.runningFetch.sseq
}
}
c.consumerType = consumerTypeFetch
sub := orderedSubscription{
consumer: c,
done: make(chan struct{}),
}
c.subscription = &sub
c.Unlock()
err := c.reset()
if err != nil {
return nil, err
}
msgs, err := c.currentConsumer.FetchBytes(maxBytes, opts...)
if err != nil {
return nil, err
}
c.runningFetch = msgs.(*fetchResult)
return msgs, nil
}
// FetchNoWait is used to retrieve up to a provided number of messages
// from a stream. This method will always send a single request and
// immediately return up to a provided number of messages or wait until
// at least one message is available or request times out.
//
// It is not efficient to use FetchNoWait with on an ordered consumer, as it will
// reset the consumer for each subsequent Fetch call.
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
func (c *orderedConsumer) FetchNoWait(batch int) (MessageBatch, error) {
if c.consumerType == consumerTypeConsume {
return nil, ErrOrderConsumerUsedAsConsume
}
if c.runningFetch != nil && !c.runningFetch.done {
return nil, ErrOrderedConsumerConcurrentRequests
}
c.consumerType = consumerTypeFetch
sub := orderedSubscription{
consumer: c,
done: make(chan struct{}),
}
c.subscription = &sub
err := c.reset()
if err != nil {
return nil, err
}
return c.currentConsumer.FetchNoWait(batch)
}
// Next is used to retrieve the next message from the stream. This
// method will block until the message is retrieved or timeout is
// reached.
//
// It is not efficient to use Next with on an ordered consumer, as it will
// reset the consumer for each subsequent Fetch call.
// Consider using [Consumer.Consume] or [Consumer.Messages] instead.
func (c *orderedConsumer) Next(opts ...FetchOpt) (Msg, error) {
res, err := c.Fetch(1, opts...)
if err != nil {
return nil, err
}
msg := <-res.Messages()
if msg != nil {
return msg, nil
}
if res.Error() == nil {
return nil, nats.ErrTimeout
}
return nil, res.Error()
}
func serialNumberFromConsumer(name string) int {
if len(name) == 0 {
return 0
}
parts := strings.Split(name, "_")
if len(parts) < 2 {
return 0
}
serial, err := strconv.Atoi(parts[len(parts)-1])
if err != nil {
return 0
}
return serial
}
func (c *orderedConsumer) reset() error {
c.Lock()
defer c.Unlock()
defer c.resetInProgress.Store(0)
if c.currentConsumer != nil {
c.currentConsumer.Lock()
if c.currentSub != nil {
c.currentSub.Stop()
}
consName := c.currentConsumer.CachedInfo().Name
c.currentConsumer.Unlock()
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
_ = c.js.DeleteConsumer(ctx, c.stream, consName)
cancel()
}()
}
c.cursor.deliverSeq = 0
consumerConfig := c.getConsumerConfig()
var err error
var cons Consumer
backoffOpts := backoffOpts{
attempts: c.cfg.MaxResetAttempts,
initialInterval: time.Second,
factor: 2,
maxInterval: 10 * time.Second,
cancel: c.subscription.done,
}
err = retryWithBackoff(func(attempt int) (bool, error) {
isClosed := c.subscription.closed.Load() == 1
if isClosed {
return false, errOrderedConsumerClosed
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cons, err = c.js.CreateOrUpdateConsumer(ctx, c.stream, *consumerConfig)
if err != nil {
return true, err
}
return false, nil
}, backoffOpts)
if err != nil {
return err
}
c.currentConsumer = cons.(*pullConsumer)
return nil
}
func (c *orderedConsumer) getConsumerConfig() *ConsumerConfig {
c.serial++
var nextSeq uint64
// if stream sequence is not initialized, no message was consumed yet
// therefore, start from the beginning (either from 1 or from the provided sequence)
if c.cursor.streamSeq == 0 {
if c.cfg.OptStartSeq != 0 {
nextSeq = c.cfg.OptStartSeq
} else {
nextSeq = 1
}
} else {
// otherwise, start from the next sequence
nextSeq = c.cursor.streamSeq + 1
}
if c.cfg.MaxResetAttempts == 0 {
c.cfg.MaxResetAttempts = -1
}
name := fmt.Sprintf("%s_%d", c.namePrefix, c.serial)
cfg := &ConsumerConfig{
Name: name,
DeliverPolicy: DeliverByStartSequencePolicy,
OptStartSeq: nextSeq,
AckPolicy: AckNonePolicy,
InactiveThreshold: 5 * time.Minute,
Replicas: 1,
HeadersOnly: c.cfg.HeadersOnly,
MemoryStorage: true,
Metadata: c.cfg.Metadata,
}
if len(c.cfg.FilterSubjects) == 1 {
cfg.FilterSubject = c.cfg.FilterSubjects[0]
} else {
cfg.FilterSubjects = c.cfg.FilterSubjects
}
if c.cfg.InactiveThreshold != 0 {
cfg.InactiveThreshold = c.cfg.InactiveThreshold
}
// if the cursor is not yet set, use the provided deliver policy
if c.cursor.streamSeq != 0 {
return cfg
}
// initial request, some options may be modified at that point
cfg.DeliverPolicy = c.cfg.DeliverPolicy
if c.cfg.DeliverPolicy == DeliverLastPerSubjectPolicy ||
c.cfg.DeliverPolicy == DeliverLastPolicy ||
c.cfg.DeliverPolicy == DeliverNewPolicy ||
c.cfg.DeliverPolicy == DeliverAllPolicy {
cfg.OptStartSeq = 0
} else if c.cfg.DeliverPolicy == DeliverByStartTimePolicy {
cfg.OptStartSeq = 0
cfg.OptStartTime = c.cfg.OptStartTime
} else {
cfg.OptStartSeq = c.cfg.OptStartSeq
}
if cfg.DeliverPolicy == DeliverLastPerSubjectPolicy && len(c.cfg.FilterSubjects) == 0 {
cfg.FilterSubjects = []string{">"}
}
return cfg
}
func consumeStopAfterNotify(numMsgs int, msgsLeftAfterStop chan int) PullConsumeOpt {
return pullOptFunc(func(opts *consumeOpts) error {
opts.StopAfter = numMsgs
opts.stopAfterMsgsLeft = msgsLeftAfterStop
return nil
})
}
func messagesStopAfterNotify(numMsgs int, msgsLeftAfterStop chan int) PullMessagesOpt {
return pullOptFunc(func(opts *consumeOpts) error {
opts.StopAfter = numMsgs
opts.stopAfterMsgsLeft = msgsLeftAfterStop
return nil
})
}
func consumeReconnectNotify() PullConsumeOpt {
return pullOptFunc(func(opts *consumeOpts) error {
opts.notifyOnReconnect = true
return nil
})
}
func messagesReconnectNotify() PullMessagesOpt {
return pullOptFunc(func(opts *consumeOpts) error {
opts.notifyOnReconnect = true
return nil
})
}
// Info returns information about the ordered consumer.
// Note that this method will fetch the latest instance of the
// consumer from the server, which can be deleted by the library at any time.
func (c *orderedConsumer) Info(ctx context.Context) (*ConsumerInfo, error) {
c.Lock()
defer c.Unlock()
if c.currentConsumer == nil {
return nil, ErrOrderedConsumerNotCreated
}
infoSubject := fmt.Sprintf(apiConsumerInfoT, c.stream, c.currentConsumer.name)
var resp consumerInfoResponse
if _, err := c.js.apiRequestJSON(ctx, infoSubject, &resp); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeConsumerNotFound {
return nil, ErrConsumerNotFound
}
return nil, resp.Error
}
if resp.Error == nil && resp.ConsumerInfo == nil {
return nil, ErrConsumerNotFound
}
c.currentConsumer.info = resp.ConsumerInfo
return resp.ConsumerInfo, nil
}
// CachedInfo returns cached information about the consumer currently
// used by the ordered consumer. Cached info will be updated on every call
// to [Consumer.Info] or on consumer reset.
func (c *orderedConsumer) CachedInfo() *ConsumerInfo {
c.Lock()
defer c.Unlock()
if c.currentConsumer == nil {
return nil
}
return c.currentConsumer.info
}
type backoffOpts struct {
// total retry attempts
// -1 for unlimited
attempts int
// initial interval after which first retry will be performed
// defaults to 1s
initialInterval time.Duration
// determines whether first function execution should be performed immediately
disableInitialExecution bool
// multiplier on each attempt
// defaults to 2
factor float64
// max interval between retries
// after reaching this value, all subsequent
// retries will be performed with this interval
// defaults to 1 minute
maxInterval time.Duration
// custom backoff intervals
// if set, overrides all other options except attempts
// if attempts are set, then the last interval will be used
// for all subsequent retries after reaching the limit
customBackoff []time.Duration
// cancel channel
// if set, retry will be canceled when this channel is closed
cancel <-chan struct{}
}
func retryWithBackoff(f func(int) (bool, error), opts backoffOpts) error {
var err error
var shouldContinue bool
// if custom backoff is set, use it instead of other options
if len(opts.customBackoff) > 0 {
if opts.attempts != 0 {
return errors.New("cannot use custom backoff intervals when attempts are set")
}
for i, interval := range opts.customBackoff {
select {
case <-opts.cancel:
return nil
case <-time.After(interval):
}
shouldContinue, err = f(i)
if !shouldContinue {
return err
}
}
return err
}
// set default options
if opts.initialInterval == 0 {
opts.initialInterval = 1 * time.Second
}
if opts.factor == 0 {
opts.factor = 2
}
if opts.maxInterval == 0 {
opts.maxInterval = 1 * time.Minute
}
if opts.attempts == 0 {
return errors.New("retry attempts have to be set when not using custom backoff intervals")
}
interval := opts.initialInterval
for i := 0; ; i++ {
if i == 0 && opts.disableInitialExecution {
time.Sleep(interval)
continue
}
shouldContinue, err = f(i)
if !shouldContinue {
return err
}
if opts.attempts > 0 && i >= opts.attempts-1 {
break
}
select {
case <-opts.cancel:
return nil
case <-time.After(interval):
}
interval = time.Duration(float64(interval) * opts.factor)
if interval >= opts.maxInterval {
interval = opts.maxInterval
}
}
return err
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/publish.go
Generated Vendored
+731
View File
@@ -0,0 +1,731 @@
// Copyright 2022-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"math/rand"
"strconv"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nuid"
)
type (
asyncPublisherOpts struct {
// For async publish error handling.
aecb MsgErrHandler
// For async publish ack handling.
ackcb MsgAckHandler
// Max async pub ack in flight
maxpa int
// ackTimeout is the max time to wait for an ack.
ackTimeout time.Duration
}
// PublishOpt are the options that can be passed to Publish methods.
PublishOpt func(*pubOpts) error
pubOpts struct {
id string
lastMsgID string // Expected last msgId
stream string // Expected stream name
lastSeq *uint64 // Expected last sequence
lastSubjectSeq *uint64 // Expected last sequence for subject
lastSubject string // Expected subject for last sequence
ttl time.Duration // Message TTL
schedule string // Schedule expression
scheduleTarget string // Target subject for scheduled messages
scheduleSource string // Source subject for sampling
scheduleTTL string // TTL for generated messages
scheduleTZ string // Time zone for cron schedules
scheduleRollup string // Rollup for generated messages
// Publish retries for NoResponders err.
retryWait time.Duration // Retry wait between attempts
retryAttempts int // Retry attempts
// stallWait is the max wait of a async pub ack.
stallWait time.Duration
// internal option to re-use existing paf in case of retry.
pafRetry *pubAckFuture
}
// PubAckFuture is a future for a PubAck.
// It can be used to wait for a PubAck or an error after an async publish.
PubAckFuture interface {
// Ok returns a receive only channel that can be used to get a PubAck.
Ok() <-chan *PubAck
// Err returns a receive only channel that can be used to get the error from an async publish.
Err() <-chan error
// Msg returns the message that was sent to the server.
Msg() *nats.Msg
}
pubAckFuture struct {
jsClient *jetStreamClient
msg *nats.Msg
retries int
maxRetries int
retryWait time.Duration
ack *PubAck
err error
errCh chan error
doneCh chan *PubAck
reply string
timeout *time.Timer
}
jetStreamClient struct {
asyncPublishContext
asyncPublisherOpts
}
// MsgErrHandler is used to process asynchronous errors from JetStream
// PublishAsync. It will return the original message sent to the server for
// possible retransmitting and the error encountered.
MsgErrHandler func(JetStream, *nats.Msg, error)
// MsgAckHandler is used to process asynchronous acks from JetStream
// PublishAsync. It will return the original message sent to the server
// and the resulting PubAck.
//
// The handler is invoked synchronously on the ack-processing goroutine,
// so it must not block or call methods on the PubAckFuture returned by
// the corresponding PublishAsync/PublishMsgAsync call, as doing so may
// deadlock.
MsgAckHandler func(JetStream, *nats.Msg, *PubAck)
asyncPublishContext struct {
sync.RWMutex
replyPrefix string
replySub *nats.Subscription
acks map[string]*pubAckFuture
stallCh chan struct{}
doneCh chan struct{}
rr *rand.Rand
// channel to signal when server is disconnected or conn is closed
connStatusCh chan (nats.Status)
}
pubAckResponse struct {
apiResponse
*PubAck
}
// PubAck is an ack received after successfully publishing a message.
PubAck struct {
// Stream is the stream name the message was published to.
Stream string `json:"stream"`
// Sequence is the stream sequence number of the message.
Sequence uint64 `json:"seq"`
// Duplicate indicates whether the message was a duplicate.
// Duplicate can be detected using the [MsgIDHeader] and [StreamConfig.Duplicates].
Duplicate bool `json:"duplicate,omitempty"`
// Domain is the domain the message was published to.
Domain string `json:"domain,omitempty"`
Value string `json:"val,omitempty"`
}
)
const (
// Default time wait between retries on Publish if err is ErrNoResponders.
DefaultPubRetryWait = 250 * time.Millisecond
// Default number of retries
DefaultPubRetryAttempts = 2
)
const (
statusHdr = "Status"
rdigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
base = 62
)
// Publish performs a synchronous publish to a stream and waits for ack
// from server. It accepts subject name (which must be bound to a stream)
// and message payload.
func (js *jetStream) Publish(ctx context.Context, subj string, data []byte, opts ...PublishOpt) (*PubAck, error) {
return js.PublishMsg(ctx, &nats.Msg{Subject: subj, Data: data}, opts...)
}
// PublishMsg performs a synchronous publish to a stream and waits for
// ack from server. It accepts subject name (which must be bound to a
// stream) and nats.Message.
func (js *jetStream) PublishMsg(ctx context.Context, m *nats.Msg, opts ...PublishOpt) (*PubAck, error) {
ctx, cancel := js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
o := pubOpts{
retryWait: DefaultPubRetryWait,
retryAttempts: DefaultPubRetryAttempts,
}
if len(opts) > 0 {
if m.Header == nil {
m.Header = nats.Header{}
}
for _, opt := range opts {
if err := opt(&o); err != nil {
return nil, err
}
}
}
if o.stallWait > 0 {
return nil, fmt.Errorf("%w: stall wait cannot be set to sync publish", ErrInvalidOption)
}
if o.id != "" {
m.Header.Set(MsgIDHeader, o.id)
}
if o.lastMsgID != "" {
m.Header.Set(ExpectedLastMsgIDHeader, o.lastMsgID)
}
if o.stream != "" {
m.Header.Set(ExpectedStreamHeader, o.stream)
}
if o.lastSeq != nil {
m.Header.Set(ExpectedLastSeqHeader, strconv.FormatUint(*o.lastSeq, 10))
}
if o.lastSubjectSeq != nil {
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
}
if o.lastSubject != "" {
m.Header.Set(ExpectedLastSubjSeqSubjHeader, o.lastSubject)
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
}
if o.ttl > 0 {
m.Header.Set(MsgTTLHeader, o.ttl.String())
}
if o.schedule != "" {
m.Header.Set(ScheduleHeader, o.schedule)
}
if o.scheduleTarget != "" {
m.Header.Set(ScheduleTargetHeader, o.scheduleTarget)
}
if o.scheduleSource != "" {
m.Header.Set(ScheduleSourceHeader, o.scheduleSource)
}
if o.scheduleTTL != "" {
m.Header.Set(ScheduleTTLHeader, o.scheduleTTL)
}
if o.scheduleTZ != "" {
m.Header.Set(ScheduleTimeZoneHeader, o.scheduleTZ)
}
if o.scheduleRollup != "" {
m.Header.Set(ScheduleRollupHeader, o.scheduleRollup)
}
var resp *nats.Msg
var err error
resp, err = js.conn.RequestMsgWithContext(ctx, m)
if err != nil {
for r := 0; errors.Is(err, nats.ErrNoResponders) && (r < o.retryAttempts || o.retryAttempts < 0); r++ {
// To protect against small blips in leadership changes etc, if we get a no responders here retry.
select {
case <-ctx.Done():
case <-time.After(o.retryWait):
}
resp, err = js.conn.RequestMsgWithContext(ctx, m)
}
if err != nil {
if errors.Is(err, nats.ErrNoResponders) {
return nil, ErrNoStreamResponse
}
return nil, err
}
}
var ackResp pubAckResponse
if err := json.Unmarshal(resp.Data, &ackResp); err != nil {
return nil, ErrInvalidJSAck
}
if ackResp.Error != nil {
return nil, fmt.Errorf("nats: %w", ackResp.Error)
}
if ackResp.PubAck == nil || ackResp.PubAck.Stream == "" {
return nil, ErrInvalidJSAck
}
return ackResp.PubAck, nil
}
// PublishAsync performs an asynchronous publish to a stream and returns
// [PubAckFuture] interface. It accepts subject name (which must be bound
// to a stream) and message payload.
func (js *jetStream) PublishAsync(subj string, data []byte, opts ...PublishOpt) (PubAckFuture, error) {
return js.PublishMsgAsync(&nats.Msg{Subject: subj, Data: data}, opts...)
}
// PublishMsgAsync performs an asynchronous publish to a stream and
// returns [PubAckFuture] interface. It accepts subject name (which must
// be bound to a stream) and nats.Message.
func (js *jetStream) PublishMsgAsync(m *nats.Msg, opts ...PublishOpt) (PubAckFuture, error) {
o := pubOpts{
retryWait: DefaultPubRetryWait,
retryAttempts: DefaultPubRetryAttempts,
}
if len(opts) > 0 {
if m.Header == nil {
m.Header = nats.Header{}
}
for _, opt := range opts {
if err := opt(&o); err != nil {
return nil, err
}
}
}
defaultStallWait := 200 * time.Millisecond
stallWait := defaultStallWait
if o.stallWait > 0 {
stallWait = o.stallWait
}
if o.id != "" {
m.Header.Set(MsgIDHeader, o.id)
}
if o.lastMsgID != "" {
m.Header.Set(ExpectedLastMsgIDHeader, o.lastMsgID)
}
if o.stream != "" {
m.Header.Set(ExpectedStreamHeader, o.stream)
}
if o.lastSeq != nil {
m.Header.Set(ExpectedLastSeqHeader, strconv.FormatUint(*o.lastSeq, 10))
}
if o.lastSubjectSeq != nil {
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
}
if o.lastSubject != "" {
m.Header.Set(ExpectedLastSubjSeqSubjHeader, o.lastSubject)
m.Header.Set(ExpectedLastSubjSeqHeader, strconv.FormatUint(*o.lastSubjectSeq, 10))
}
if o.ttl > 0 {
m.Header.Set(MsgTTLHeader, o.ttl.String())
}
if o.schedule != "" {
m.Header.Set(ScheduleHeader, o.schedule)
}
if o.scheduleTarget != "" {
m.Header.Set(ScheduleTargetHeader, o.scheduleTarget)
}
if o.scheduleSource != "" {
m.Header.Set(ScheduleSourceHeader, o.scheduleSource)
}
if o.scheduleTTL != "" {
m.Header.Set(ScheduleTTLHeader, o.scheduleTTL)
}
if o.scheduleTZ != "" {
m.Header.Set(ScheduleTimeZoneHeader, o.scheduleTZ)
}
if o.scheduleRollup != "" {
m.Header.Set(ScheduleRollupHeader, o.scheduleRollup)
}
paf := o.pafRetry
if paf == nil && m.Reply != "" {
return nil, ErrAsyncPublishReplySubjectSet
}
var id string
var reply string
// register new paf if not retrying
if paf == nil {
var err error
reply, err = js.newAsyncReply()
if err != nil {
return nil, fmt.Errorf("nats: error creating async reply handler: %s", err)
}
id = reply[js.opts.replyPrefixLen:]
paf = &pubAckFuture{msg: m, jsClient: js.publisher, maxRetries: o.retryAttempts, retryWait: o.retryWait, reply: reply}
numPending, maxPending := js.registerPAF(id, paf)
if maxPending > 0 && numPending > maxPending {
select {
case <-js.asyncStall():
case <-time.After(stallWait):
js.clearPAF(id)
return nil, ErrTooManyStalledMsgs
}
}
if js.publisher.ackTimeout > 0 {
paf.timeout = time.AfterFunc(js.publisher.ackTimeout, func() {
js.publisher.Lock()
defer js.publisher.Unlock()
if _, ok := js.publisher.acks[id]; !ok {
// paf has already been resolved
// while waiting for the lock
return
}
// ack timed out, remove from pending acks
delete(js.publisher.acks, id)
// check on anyone stalled and waiting.
if js.publisher.stallCh != nil && len(js.publisher.acks) < js.publisher.maxpa {
close(js.publisher.stallCh)
js.publisher.stallCh = nil
}
// send error to user
paf.err = ErrAsyncPublishTimeout
if paf.errCh != nil {
paf.errCh <- paf.err
}
// call error callback if set
if js.publisher.asyncPublisherOpts.aecb != nil {
js.publisher.asyncPublisherOpts.aecb(js, paf.msg, ErrAsyncPublishTimeout)
}
// check on anyone one waiting on done status.
if js.publisher.doneCh != nil && len(js.publisher.acks) == 0 {
close(js.publisher.doneCh)
js.publisher.doneCh = nil
}
})
}
} else {
// when retrying, get the ID from existing reply subject
reply = paf.reply
if paf.timeout != nil {
paf.timeout.Reset(js.publisher.ackTimeout)
}
id = reply[js.opts.replyPrefixLen:]
}
pubMsg := &nats.Msg{
Subject: m.Subject,
Reply: reply,
Data: m.Data,
Header: m.Header,
}
if err := js.conn.PublishMsg(pubMsg); err != nil {
js.clearPAF(id)
return nil, err
}
return paf, nil
}
// For quick token lookup etc.
const (
aReplyTokensize = 6
)
func (js *jetStream) newAsyncReply() (string, error) {
js.publisher.Lock()
if js.publisher.replySub == nil {
// Create our wildcard reply subject.
sha := sha256.New()
sha.Write([]byte(nuid.Next()))
b := sha.Sum(nil)
for i := 0; i < aReplyTokensize; i++ {
b[i] = rdigits[int(b[i]%base)]
}
js.publisher.replyPrefix = fmt.Sprintf("%s%s.", js.opts.replyPrefix, b[:aReplyTokensize])
sub, err := js.conn.Subscribe(fmt.Sprintf("%s*", js.publisher.replyPrefix), js.handleAsyncReply)
if err != nil {
js.publisher.Unlock()
return "", err
}
js.publisher.replySub = sub
js.publisher.rr = rand.New(rand.NewSource(time.Now().UnixNano()))
}
if js.publisher.connStatusCh == nil {
js.publisher.connStatusCh = js.conn.StatusChanged(nats.RECONNECTING, nats.CLOSED)
go js.resetPendingAcksOnReconnect()
}
var sb strings.Builder
sb.Grow(len(js.publisher.replyPrefix) + aReplyTokensize)
sb.WriteString(js.publisher.replyPrefix)
for {
rn := js.publisher.rr.Int63()
var b [aReplyTokensize]byte
for i, l := 0, rn; i < len(b); i++ {
b[i] = rdigits[l%base]
l /= base
}
if _, ok := js.publisher.acks[string(b[:])]; ok {
continue
}
sb.Write(b[:])
break
}
js.publisher.Unlock()
return sb.String(), nil
}
// Handle an async reply from PublishAsync.
func (js *jetStream) handleAsyncReply(m *nats.Msg) {
if len(m.Subject) <= js.opts.replyPrefixLen {
return
}
id := m.Subject[js.opts.replyPrefixLen:]
js.publisher.Lock()
paf := js.getPAF(id)
if paf == nil {
js.publisher.Unlock()
return
}
closeStc := func() {
// Check on anyone stalled and waiting.
if js.publisher.stallCh != nil && len(js.publisher.acks) < js.publisher.maxpa {
close(js.publisher.stallCh)
js.publisher.stallCh = nil
}
}
closeDchFn := func() func() {
var dch chan struct{}
// Check on anyone one waiting on done status.
if js.publisher.doneCh != nil && len(js.publisher.acks) == 0 {
dch = js.publisher.doneCh
js.publisher.doneCh = nil
}
// Return function to close done channel which
// should be deferred so that error is processed and
// can be checked.
return func() {
if dch != nil {
close(dch)
}
}
}
if paf.timeout != nil {
paf.timeout.Stop()
}
// Process no responders etc.
if len(m.Data) == 0 && m.Header.Get(statusHdr) == statusNoResponders {
if paf.retries < paf.maxRetries {
paf.retries++
time.AfterFunc(paf.retryWait, func() {
js.publisher.Lock()
paf := js.getPAF(id)
js.publisher.Unlock()
if paf == nil {
return
}
_, err := js.PublishMsgAsync(paf.msg, func(po *pubOpts) error {
po.pafRetry = paf
return nil
})
if err != nil {
js.publisher.Lock()
js.handleAsyncErrAndUnlock(paf, err)
}
})
js.publisher.Unlock()
return
}
delete(js.publisher.acks, id)
closeStc()
defer closeDchFn()()
js.handleAsyncErrAndUnlock(paf, ErrNoStreamResponse)
return
}
// Remove
delete(js.publisher.acks, id)
closeStc()
defer closeDchFn()()
var pa pubAckResponse
if err := json.Unmarshal(m.Data, &pa); err != nil {
js.handleAsyncErrAndUnlock(paf, ErrInvalidJSAck)
return
}
if pa.Error != nil {
js.handleAsyncErrAndUnlock(paf, pa.Error)
return
}
if pa.PubAck == nil || pa.PubAck.Stream == "" {
js.handleAsyncErrAndUnlock(paf, ErrInvalidJSAck)
return
}
// So here we have received a proper puback.
paf.ack = pa.PubAck
if paf.doneCh != nil {
paf.doneCh <- paf.ack
}
cb := js.publisher.asyncPublisherOpts.ackcb
js.publisher.Unlock()
if cb != nil {
cb(js, paf.msg, paf.ack)
}
}
func (js *jetStream) handleAsyncErrAndUnlock(paf *pubAckFuture, err error) {
paf.err = err
if paf.errCh != nil {
paf.errCh <- paf.err
}
cb := js.publisher.asyncPublisherOpts.aecb
js.publisher.Unlock()
if cb != nil {
cb(js, paf.msg, err)
}
}
func (js *jetStream) resetPendingAcksOnReconnect() {
js.publisher.Lock()
connStatusCh := js.publisher.connStatusCh
js.publisher.Unlock()
for {
newStatus, ok := <-connStatusCh
if !ok || newStatus == nats.CLOSED {
return
}
js.publisher.Lock()
errCb := js.publisher.asyncPublisherOpts.aecb
for id, paf := range js.publisher.acks {
paf.err = nats.ErrDisconnected
if paf.errCh != nil {
paf.errCh <- paf.err
}
if errCb != nil {
defer errCb(js, paf.msg, nats.ErrDisconnected)
}
delete(js.publisher.acks, id)
}
if js.publisher.doneCh != nil {
close(js.publisher.doneCh)
js.publisher.doneCh = nil
}
js.publisher.Unlock()
}
}
// registerPAF will register for a PubAckFuture.
func (js *jetStream) registerPAF(id string, paf *pubAckFuture) (int, int) {
js.publisher.Lock()
if js.publisher.acks == nil {
js.publisher.acks = make(map[string]*pubAckFuture)
}
js.publisher.acks[id] = paf
np := len(js.publisher.acks)
maxpa := js.publisher.asyncPublisherOpts.maxpa
js.publisher.Unlock()
return np, maxpa
}
// Lock should be held.
func (js *jetStream) getPAF(id string) *pubAckFuture {
if js.publisher.acks == nil {
return nil
}
return js.publisher.acks[id]
}
// clearPAF will remove a PubAckFuture that was registered.
func (js *jetStream) clearPAF(id string) {
js.publisher.Lock()
delete(js.publisher.acks, id)
js.publisher.Unlock()
}
func (js *jetStream) asyncStall() <-chan struct{} {
js.publisher.Lock()
if js.publisher.stallCh == nil {
js.publisher.stallCh = make(chan struct{})
}
stc := js.publisher.stallCh
js.publisher.Unlock()
return stc
}
func (paf *pubAckFuture) Ok() <-chan *PubAck {
paf.jsClient.Lock()
defer paf.jsClient.Unlock()
if paf.doneCh == nil {
paf.doneCh = make(chan *PubAck, 1)
if paf.ack != nil {
paf.doneCh <- paf.ack
}
}
return paf.doneCh
}
func (paf *pubAckFuture) Err() <-chan error {
paf.jsClient.Lock()
defer paf.jsClient.Unlock()
if paf.errCh == nil {
paf.errCh = make(chan error, 1)
if paf.err != nil {
paf.errCh <- paf.err
}
}
return paf.errCh
}
func (paf *pubAckFuture) Msg() *nats.Msg {
paf.jsClient.RLock()
defer paf.jsClient.RUnlock()
return paf.msg
}
// PublishAsyncPending returns the number of async publishes outstanding
// for this context.
func (js *jetStream) PublishAsyncPending() int {
js.publisher.RLock()
defer js.publisher.RUnlock()
return len(js.publisher.acks)
}
// PublishAsyncComplete returns a channel that will be closed when all
// outstanding asynchronously published messages are acknowledged by the
// server.
func (js *jetStream) PublishAsyncComplete() <-chan struct{} {
js.publisher.Lock()
defer js.publisher.Unlock()
if js.publisher.doneCh == nil {
js.publisher.doneCh = make(chan struct{})
}
dch := js.publisher.doneCh
if len(js.publisher.acks) == 0 {
close(js.publisher.doneCh)
js.publisher.doneCh = nil
}
return dch
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/pull.go
Generated Vendored
+1242
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/push.go
Generated Vendored
+268
View File
@@ -0,0 +1,268 @@
package jetstream
import (
"errors"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nuid"
)
type (
pushConsumer struct {
sync.Mutex
js *jetStream
stream string
name string
info *ConsumerInfo
started atomic.Bool
}
pushSubscription struct {
sync.Mutex
id string
errs chan error
subscription *nats.Subscription
connStatusChanged chan nats.Status
closedCh chan struct{}
done chan struct{}
closed atomic.Bool
consumeOpts *pushConsumeOpts
hbMonitor *hbMonitor
idleHeartbeat time.Duration
}
pushConsumeOpts struct {
ErrHandler ConsumeErrHandler
}
PushConsumeOpt interface {
configurePushConsume(*pushConsumeOpts) error
}
)
func (p *pushConsumer) Consume(handler MessageHandler, opts ...PushConsumeOpt) (ConsumeContext, error) {
if handler == nil {
return nil, ErrHandlerRequired
}
consumeOpts := &pushConsumeOpts{}
for _, opt := range opts {
if err := opt.configurePushConsume(consumeOpts); err != nil {
return nil, err
}
}
p.Lock()
defer p.Unlock()
if p.info == nil {
return nil, ErrConsumerNotFound
}
if p.started.Load() {
return nil, ErrConsumerAlreadyConsuming
}
consumeID := nuid.Next()
sub := &pushSubscription{
id: consumeID,
errs: make(chan error, 1),
done: make(chan struct{}, 1),
consumeOpts: consumeOpts,
connStatusChanged: p.js.conn.StatusChanged(nats.CONNECTED, nats.RECONNECTING),
idleHeartbeat: p.info.Config.IdleHeartbeat,
}
sub.hbMonitor = sub.scheduleHeartbeatCheck(sub.idleHeartbeat)
internalHandler := func(msg *nats.Msg) {
if sub.hbMonitor != nil {
sub.hbMonitor.Stop()
}
defer func() {
if sub.hbMonitor != nil {
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
}
}()
status, descr := msg.Header.Get("Status"), msg.Header.Get("Description")
if status == "" {
jsMsg := p.js.toJSMsg(msg)
handler(jsMsg)
return
}
sub.Lock()
if err, terminate := sub.handleStatusMsg(msg, status, descr); err != nil {
if sub.consumeOpts.ErrHandler != nil {
sub.consumeOpts.ErrHandler(sub, err)
}
if terminate {
sub.Stop()
}
}
sub.Unlock()
}
var err error
if p.info.Config.DeliverGroup != "" {
sub.subscription, err = p.js.conn.QueueSubscribe(p.info.Config.DeliverSubject, p.info.Config.DeliverGroup, internalHandler)
} else {
sub.subscription, err = p.js.conn.Subscribe(p.info.Config.DeliverSubject, internalHandler)
}
if err != nil {
return nil, err
}
sub.subscription.SetClosedHandler(func(sid string) func(string) {
return func(subject string) {
p.started.Store(false)
sub.Lock()
defer sub.Unlock()
if sub.closedCh != nil {
close(sub.closedCh)
sub.closedCh = nil
}
}
}(sub.id))
go func() {
isConnected := true
for {
if sub.closed.Load() {
return
}
select {
case status, ok := <-sub.connStatusChanged:
if !ok {
continue
}
if status == nats.RECONNECTING {
if sub.hbMonitor != nil {
sub.hbMonitor.Stop()
}
isConnected = false
}
if status == nats.CONNECTED {
sub.Lock()
if !isConnected {
isConnected = true
if sub.hbMonitor != nil {
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
}
}
sub.Unlock()
}
case err := <-sub.errs:
sub.Lock()
if sub.consumeOpts.ErrHandler != nil {
sub.consumeOpts.ErrHandler(sub, err)
}
if errors.Is(err, ErrNoHeartbeat) {
if sub.hbMonitor != nil {
sub.hbMonitor.Reset(2 * sub.idleHeartbeat)
}
}
sub.Unlock()
case <-sub.done:
return
}
}
}()
p.started.Store(true)
return sub, nil
}
func (s *pushSubscription) handleStatusMsg(msg *nats.Msg, status, description string) (error, bool) {
switch status {
case statusControlMsg:
switch strings.ToLower(description) {
case idleHeartbeatDescr:
return nil, false
case fcRequestDescr:
if err := msg.Respond(nil); err != nil {
if s.consumeOpts.ErrHandler != nil {
s.consumeOpts.ErrHandler(s, err)
}
}
return nil, false
}
case statusConflict:
if description == consumerDeleted {
return ErrConsumerDeleted, true
}
if description == leadershipChange {
if s.consumeOpts.ErrHandler != nil {
s.consumeOpts.ErrHandler(s, ErrConsumerLeadershipChanged)
return ErrConsumerLeadershipChanged, false
}
}
}
return nil, false
}
// Stop unsubscribes from the stream and cancels subscription.
// No more messages will be received after calling this method.
// All messages that are already in the buffer are discarded.
func (s *pushSubscription) Stop() {
if !s.closed.CompareAndSwap(false, true) {
return
}
s.Lock()
defer s.Unlock()
close(s.done)
s.subscription.Unsubscribe()
if s.hbMonitor != nil {
s.hbMonitor.Stop()
}
}
// Drain unsubscribes from the stream and cancels subscription.
// All messages that are already in the buffer will be processed in callback function.
func (s *pushSubscription) Drain() {
if !s.closed.CompareAndSwap(false, true) {
return
}
s.Lock()
defer s.Unlock()
close(s.done)
s.subscription.Drain()
if s.hbMonitor != nil {
s.hbMonitor.Stop()
}
}
// Closed returns a channel that is closed when consuming is
// fully stopped/drained. When the channel is closed, no more messages
// will be received and processing is complete.
func (s *pushSubscription) Closed() <-chan struct{} {
s.Lock()
defer s.Unlock()
ch := s.closedCh
if ch == nil {
ch = make(chan struct{})
s.closedCh = ch
}
if !s.subscription.IsValid() {
close(s.closedCh)
s.closedCh = nil
}
return ch
}
func (s *pushSubscription) scheduleHeartbeatCheck(dur time.Duration) *hbMonitor {
if dur == 0 {
return nil
}
return &hbMonitor{
timer: time.AfterFunc(2*dur, func() {
s.Lock()
defer s.Unlock()
s.errs <- ErrNoHeartbeat
}),
}
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/stream.go
Generated Vendored
+879
View File
@@ -0,0 +1,879 @@
// Copyright 2022-2026 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"context"
"encoding/json"
"errors"
"fmt"
"strconv"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nuid"
)
type (
// Stream contains CRUD methods on a consumer via [ConsumerManager], as well
// as operations on an existing stream. It allows fetching and removing
// messages from a stream, as well as purging a stream.
Stream interface {
ConsumerManager
// Info returns StreamInfo from the server.
Info(ctx context.Context, opts ...StreamInfoOpt) (*StreamInfo, error)
// CachedInfo returns ConsumerInfo currently cached on this stream.
// This method does not perform any network requests. The cached
// StreamInfo is updated on every call to Info and Update.
CachedInfo() *StreamInfo
// Purge removes messages from a stream. It is a destructive operation.
// Use with caution. See StreamPurgeOpt for available options.
Purge(ctx context.Context, opts ...StreamPurgeOpt) error
// GetMsg retrieves a raw stream message stored in JetStream by sequence number.
GetMsg(ctx context.Context, seq uint64, opts ...GetMsgOpt) (*RawStreamMsg, error)
// GetLastMsgForSubject retrieves the last raw stream message stored in
// JetStream on a given subject.
GetLastMsgForSubject(ctx context.Context, subject string) (*RawStreamMsg, error)
// DeleteMsg deletes a message from a stream.
// On the server, the message is marked as erased, but not overwritten.
DeleteMsg(ctx context.Context, seq uint64) error
// SecureDeleteMsg deletes a message from a stream. The deleted message
// is overwritten with random data. As a result, this operation is slower
// than DeleteMsg.
SecureDeleteMsg(ctx context.Context, seq uint64) error
}
// ConsumerManager provides CRUD API for managing consumers. It is
// available as a part of [Stream] interface. CreateConsumer,
// UpdateConsumer, CreateOrUpdateConsumer and Consumer methods return a
// [Consumer] interface, allowing to operate on a consumer (e.g. consume
// messages).
ConsumerManager interface {
// CreateOrUpdateConsumer creates a pull consumer on a given stream with
// given config. If consumer already exists, it will be updated (if
// possible). Consumer interface is returned, allowing to operate on a
// consumer (e.g. fetch messages).
CreateOrUpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
// CreateConsumer creates a pull consumer on a given stream with given
// config. If consumer already exists and the provided configuration
// differs from its configuration, ErrConsumerExists is returned. If the
// provided configuration is the same as the existing consumer, the
// existing consumer is returned. Consumer interface is returned,
// allowing to operate on a consumer (e.g. fetch messages).
CreateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
// UpdateConsumer updates an existing pull consumer. If consumer does not
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
// returned, allowing to operate on a consumer (e.g. fetch messages).
UpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error)
// OrderedConsumer returns an OrderedConsumer instance. OrderedConsumer
// are managed by the library and provide a simple way to consume
// messages from a stream. Ordered consumers are ephemeral in-memory
// pull consumers and are resilient to deletes and restarts.
OrderedConsumer(ctx context.Context, cfg OrderedConsumerConfig) (Consumer, error)
// Consumer returns an interface to an existing consumer, allowing processing
// of messages. If consumer does not exist, ErrConsumerNotFound is
// returned.
//
// It returns ErrNotPullConsumer if the consumer is not a pull consumer (deliver subject is not set).
Consumer(ctx context.Context, consumer string) (Consumer, error)
// DeleteConsumer removes a consumer with given name from a stream.
// If consumer does not exist, ErrConsumerNotFound is returned.
DeleteConsumer(ctx context.Context, consumer string) error
// PauseConsumer pauses a consumer.
PauseConsumer(ctx context.Context, consumer string, pauseUntil time.Time) (*ConsumerPauseResponse, error)
// ResumeConsumer resumes a consumer.
ResumeConsumer(ctx context.Context, consumer string) (*ConsumerPauseResponse, error)
// ListConsumers returns ConsumerInfoLister enabling iterating over a
// channel of consumer infos.
ListConsumers(context.Context) ConsumerInfoLister
// ConsumerNames returns a ConsumerNameLister enabling iterating over a
// channel of consumer names.
ConsumerNames(context.Context) ConsumerNameLister
// UnpinConsumer unpins the currently pinned client for a consumer for the given group name.
// If consumer does not exist, ErrConsumerNotFound is returned.
UnpinConsumer(ctx context.Context, consumer string, group string) error
// ResetConsumer resets a consumer's delivery state without deleting
// and recreating it. The consumer is reset to deliver from
// ack_floor + 1.
ResetConsumer(ctx context.Context, consumer string) (*ConsumerResetResponse, error)
// ResetConsumerToSequence resets a consumer's delivery state to the
// given stream sequence. The seq must be compatible with the
// consumer's DeliverPolicy (e.g. for DeliverByStartSequencePolicy,
// seq must be >= OptStartSeq). If incompatible,
// ErrConsumerInvalidReset is returned.
ResetConsumerToSequence(ctx context.Context, consumer string, seq uint64) (*ConsumerResetResponse, error)
// CreateOrUpdatePushConsumer creates a push consumer on a given stream with
// given config. If consumer already exists, it will be updated (if
// possible). Consumer interface is returned, allowing to consume messages.
CreateOrUpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
// CreatePushConsumer creates a push consumer on a given stream with given
// config. If consumer already exists and the provided configuration
// differs from its configuration, ErrConsumerExists is returned. If the
// provided configuration is the same as the existing consumer, the
// existing consumer is returned. Consumer interface is returned,
// allowing to consume messages.
CreatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
// UpdatePushConsumer updates an existing push consumer. If consumer does not
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
// returned, allowing to consume messages.
UpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error)
// PushConsumer returns an interface to an existing push consumer, allowing processing
// of messages. If consumer does not exist, ErrConsumerNotFound is
// returned.
//
// It returns ErrNotPushConsumer if the consumer is not a push consumer (deliver subject is not set).
PushConsumer(ctx context.Context, consumer string) (PushConsumer, error)
}
RawStreamMsg struct {
Subject string
Sequence uint64
Header nats.Header
Data []byte
Time time.Time
}
stream struct {
name string
info *StreamInfo
js *jetStream
}
// StreamInfoOpt is a function setting options for [Stream.Info]
StreamInfoOpt func(*streamInfoRequest) error
streamInfoRequest struct {
apiPagedRequest
DeletedDetails bool `json:"deleted_details,omitempty"`
SubjectFilter string `json:"subjects_filter,omitempty"`
}
consumerInfoResponse struct {
apiResponse
*ConsumerInfo
}
// StreamPurgeOpt is a function setting options for [Stream.Purge]
StreamPurgeOpt func(*StreamPurgeRequest) error
// StreamPurgeRequest is an API request body to purge a stream.
StreamPurgeRequest struct {
// Purge up to but not including sequence.
Sequence uint64 `json:"seq,omitempty"`
// Subject to match against messages for the purge command.
Subject string `json:"filter,omitempty"`
// Number of messages to keep.
Keep uint64 `json:"keep,omitempty"`
}
streamPurgeResponse struct {
apiResponse
Success bool `json:"success,omitempty"`
Purged uint64 `json:"purged"`
}
consumerDeleteResponse struct {
apiResponse
Success bool `json:"success,omitempty"`
}
consumerPauseRequest struct {
PauseUntil *time.Time `json:"pause_until,omitempty"`
}
ConsumerPauseResponse struct {
// Paused is true if the consumer is paused.
Paused bool `json:"paused"`
// PauseUntil is the time until the consumer is paused.
PauseUntil time.Time `json:"pause_until"`
// PauseRemaining is the time remaining until the consumer is paused.
PauseRemaining time.Duration `json:"pause_remaining,omitempty"`
}
consumerPauseApiResponse struct {
apiResponse
ConsumerPauseResponse
}
consumerResetRequest struct {
Seq uint64 `json:"seq,omitempty"`
}
// ConsumerResetResponse contains the result of a consumer reset operation.
// It carries the updated ConsumerInfo together with the stream sequence the
// server actually reset the consumer to.
//
// The embedded *ConsumerInfo is guaranteed non-nil on a successful response;
// the helper returns ErrConsumerResetResponseEmpty if the server omits it.
ConsumerResetResponse struct {
*ConsumerInfo
// ResetSeq is the stream sequence the consumer was reset to. When
// ResetConsumer is called without an explicit sequence, this is the
// server-resolved value (typically ack_floor + 1).
ResetSeq uint64 `json:"reset_seq"`
}
consumerResetApiResponse struct {
apiResponse
ConsumerResetResponse
}
// GetMsgOpt is a function setting options for [Stream.GetMsg]
GetMsgOpt func(*apiMsgGetRequest) error
apiMsgGetRequest struct {
Seq uint64 `json:"seq,omitempty"`
LastFor string `json:"last_by_subj,omitempty"`
NextFor string `json:"next_by_subj,omitempty"`
}
// apiMsgGetResponse is the response for a Stream get request.
apiMsgGetResponse struct {
apiResponse
Message *storedMsg `json:"message,omitempty"`
}
// storedMsg is a raw message stored in JetStream.
storedMsg struct {
Subject string `json:"subject"`
Sequence uint64 `json:"seq"`
Header []byte `json:"hdrs,omitempty"`
Data []byte `json:"data,omitempty"`
Time time.Time `json:"time"`
}
msgDeleteRequest struct {
Seq uint64 `json:"seq"`
NoErase bool `json:"no_erase,omitempty"`
}
msgDeleteResponse struct {
apiResponse
Success bool `json:"success,omitempty"`
}
// ConsumerInfoLister is used to iterate over a channel of consumer infos.
// Err method can be used to check for errors encountered during iteration.
// Info channel is always closed and therefore can be used in a range loop.
ConsumerInfoLister interface {
Info() <-chan *ConsumerInfo
Err() error
}
// ConsumerNameLister is used to iterate over a channel of consumer names.
// Err method can be used to check for errors encountered during iteration.
// Name channel is always closed and therefore can be used in a range loop.
ConsumerNameLister interface {
Name() <-chan string
Err() error
}
consumerLister struct {
js *jetStream
offset int
pageInfo *apiPaged
consumers chan *ConsumerInfo
names chan string
err error
}
consumerListResponse struct {
apiResponse
apiPaged
Consumers []*ConsumerInfo `json:"consumers"`
}
consumerNamesResponse struct {
apiResponse
apiPaged
Consumers []string `json:"consumers"`
}
consumerUnpinRequest struct {
Group string `json:"group"`
}
)
// CreateOrUpdateConsumer creates a consumer on a given stream with
// given config. If consumer already exists, it will be updated (if
// possible). Consumer interface is returned, allowing to operate on a
// consumer (e.g. fetch messages).
func (s *stream) CreateOrUpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionCreateOrUpdate)
}
// CreateConsumer creates a consumer on a given stream with given
// config. If consumer already exists and the provided configuration
// differs from its configuration, ErrConsumerExists is returned. If the
// provided configuration is the same as the existing consumer, the
// existing consumer is returned. Consumer interface is returned,
// allowing to operate on a consumer (e.g. fetch messages).
func (s *stream) CreateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionCreate)
}
// UpdateConsumer updates an existing consumer. If consumer does not
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
// returned, allowing to operate on a consumer (e.g. fetch messages).
func (s *stream) UpdateConsumer(ctx context.Context, cfg ConsumerConfig) (Consumer, error) {
return upsertPullConsumer(ctx, s.js, s.name, cfg, consumerActionUpdate)
}
// CreateOrUpdatePushConsumer creates a consumer on a given stream with
// given config. If consumer already exists, it will be updated (if
// possible). Consumer interface is returned, allowing to consume messages.
func (s *stream) CreateOrUpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionCreateOrUpdate)
}
// CreatePushConsumer creates a consumer on a given stream with given
// config. If consumer already exists and the provided configuration
// differs from its configuration, ErrConsumerExists is returned. If the
// provided configuration is the same as the existing consumer, the
// existing consumer is returned. Consumer interface is returned,
// allowing to consume messages.
func (s *stream) CreatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionCreate)
}
// UpdatePushConsumer updates an existing consumer. If consumer does not
// exist, ErrConsumerDoesNotExist is returned. Consumer interface is
// returned, allowing to consume messages.
func (s *stream) UpdatePushConsumer(ctx context.Context, cfg ConsumerConfig) (PushConsumer, error) {
return upsertPushConsumer(ctx, s.js, s.name, cfg, consumerActionUpdate)
}
// OrderedConsumer returns an OrderedConsumer instance. OrderedConsumer
// are managed by the library and provide a simple way to consume
// messages from a stream. Ordered consumers are ephemeral in-memory
// pull consumers and are resilient to deletes and restarts.
func (s *stream) OrderedConsumer(ctx context.Context, cfg OrderedConsumerConfig) (Consumer, error) {
namePrefix := cfg.NamePrefix
if namePrefix == "" {
namePrefix = nuid.Next()
}
oc := &orderedConsumer{
js: s.js,
cfg: &cfg,
stream: s.name,
namePrefix: namePrefix,
doReset: make(chan struct{}, 1),
}
consCfg := oc.getConsumerConfig()
cons, err := s.CreateOrUpdateConsumer(ctx, *consCfg)
if err != nil {
return nil, err
}
oc.currentConsumer = cons.(*pullConsumer)
return oc, nil
}
// Consumer returns an interface to an existing consumer, allowing processing
// of messages. If consumer does not exist, ErrConsumerNotFound is
// returned.
func (s *stream) Consumer(ctx context.Context, name string) (Consumer, error) {
return getConsumer(ctx, s.js, s.name, name)
}
func (s *stream) PushConsumer(ctx context.Context, name string) (PushConsumer, error) {
return getPushConsumer(ctx, s.js, s.name, name)
}
// DeleteConsumer removes a consumer with given name from a stream.
// If consumer does not exist, ErrConsumerNotFound is returned.
func (s *stream) DeleteConsumer(ctx context.Context, name string) error {
return deleteConsumer(ctx, s.js, s.name, name)
}
// PauseConsumer pauses a consumer.
func (s *stream) PauseConsumer(ctx context.Context, name string, pauseUntil time.Time) (*ConsumerPauseResponse, error) {
return pauseConsumer(ctx, s.js, s.name, name, &pauseUntil)
}
// ResumeConsumer resumes a consumer.
func (s *stream) ResumeConsumer(ctx context.Context, name string) (*ConsumerPauseResponse, error) {
return resumeConsumer(ctx, s.js, s.name, name)
}
// Info returns StreamInfo from the server.
func (s *stream) Info(ctx context.Context, opts ...StreamInfoOpt) (*StreamInfo, error) {
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
var infoReq *streamInfoRequest
for _, opt := range opts {
if infoReq == nil {
infoReq = &streamInfoRequest{}
}
if err := opt(infoReq); err != nil {
return nil, err
}
}
var req []byte
var err error
var subjectMap map[string]uint64
var offset int
infoSubject := fmt.Sprintf(apiStreamInfoT, s.name)
var info *StreamInfo
for {
if infoReq != nil {
if infoReq.SubjectFilter != "" {
if subjectMap == nil {
subjectMap = make(map[string]uint64)
}
infoReq.Offset = offset
}
req, err = json.Marshal(infoReq)
if err != nil {
return nil, err
}
}
var resp streamInfoResponse
if _, err = s.js.apiRequestJSON(ctx, infoSubject, &resp, req); err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeStreamNotFound {
return nil, ErrStreamNotFound
}
return nil, resp.Error
}
if resp.StreamInfo == nil {
return nil, ErrInvalidJetStreamResponse
}
info = resp.StreamInfo
var total int
if resp.Total != 0 {
total = resp.Total
}
if len(resp.StreamInfo.State.Subjects) > 0 {
for subj, msgs := range resp.StreamInfo.State.Subjects {
subjectMap[subj] = msgs
}
offset = len(subjectMap)
}
if total == 0 || total <= offset {
info.State.Subjects = nil
// we don't want to store subjects in cache
cached := *info
s.info = &cached
info.State.Subjects = subjectMap
break
}
}
return info, nil
}
// CachedInfo returns ConsumerInfo currently cached on this stream.
// This method does not perform any network requests. The cached
// StreamInfo is updated on every call to Info and Update.
func (s *stream) CachedInfo() *StreamInfo {
return s.info
}
// Purge removes messages from a stream. It is a destructive operation.
// Use with caution. See StreamPurgeOpt for available options.
func (s *stream) Purge(ctx context.Context, opts ...StreamPurgeOpt) error {
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
var purgeReq StreamPurgeRequest
for _, opt := range opts {
if err := opt(&purgeReq); err != nil {
return err
}
}
var req []byte
var err error
req, err = json.Marshal(purgeReq)
if err != nil {
return err
}
purgeSubject := fmt.Sprintf(apiStreamPurgeT, s.name)
var resp streamPurgeResponse
if _, err = s.js.apiRequestJSON(ctx, purgeSubject, &resp, req); err != nil {
return err
}
if resp.Error != nil {
return resp.Error
}
return nil
}
// GetMsg retrieves a raw stream message stored in JetStream by sequence number.
func (s *stream) GetMsg(ctx context.Context, seq uint64, opts ...GetMsgOpt) (*RawStreamMsg, error) {
req := &apiMsgGetRequest{Seq: seq}
for _, opt := range opts {
if err := opt(req); err != nil {
return nil, err
}
}
return s.getMsg(ctx, req)
}
// GetLastMsgForSubject retrieves the last raw stream message stored in
// JetStream on a given subject subject.
func (s *stream) GetLastMsgForSubject(ctx context.Context, subject string) (*RawStreamMsg, error) {
return s.getMsg(ctx, &apiMsgGetRequest{LastFor: subject})
}
func (s *stream) getMsg(ctx context.Context, mreq *apiMsgGetRequest) (*RawStreamMsg, error) {
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
req, err := json.Marshal(mreq)
if err != nil {
return nil, err
}
var gmSubj string
// handle direct gets
if s.info.Config.AllowDirect {
if mreq.LastFor != "" {
gmSubj = fmt.Sprintf(apiDirectMsgGetLastBySubjectT, s.name, mreq.LastFor)
r, err := s.js.apiRequest(ctx, gmSubj, nil)
if err != nil {
return nil, err
}
return convertDirectGetMsgResponseToMsg(r.msg)
}
gmSubj = fmt.Sprintf(apiDirectMsgGetT, s.name)
r, err := s.js.apiRequest(ctx, gmSubj, req)
if err != nil {
return nil, err
}
return convertDirectGetMsgResponseToMsg(r.msg)
}
var resp apiMsgGetResponse
dsSubj := fmt.Sprintf(apiMsgGetT, s.name)
_, err = s.js.apiRequestJSON(ctx, dsSubj, &resp, req)
if err != nil {
return nil, err
}
if resp.Error != nil {
if resp.Error.ErrorCode == JSErrCodeMessageNotFound {
return nil, ErrMsgNotFound
}
return nil, resp.Error
}
if resp.Message == nil {
return nil, ErrInvalidJetStreamResponse
}
msg := resp.Message
var hdr nats.Header
if len(msg.Header) > 0 {
hdr, err = nats.DecodeHeadersMsg(msg.Header)
if err != nil {
return nil, err
}
}
return &RawStreamMsg{
Subject: msg.Subject,
Sequence: msg.Sequence,
Header: hdr,
Data: msg.Data,
Time: msg.Time,
}, nil
}
func convertDirectGetMsgResponseToMsg(r *nats.Msg) (*RawStreamMsg, error) {
// Check for 404/408. We would get a no-payload message and a "Status" header
if len(r.Data) == 0 {
val := r.Header.Get(statusHdr)
if val != "" {
switch val {
case statusNoMsgs:
return nil, ErrMsgNotFound
default:
desc := r.Header.Get("Description")
if desc == "" {
desc = "unable to get message"
}
return nil, fmt.Errorf("nats: %s", desc)
}
}
}
// Check for headers that give us the required information to
// reconstruct the message.
if len(r.Header) == 0 {
return nil, errors.New("nats: response should have headers")
}
stream := r.Header.Get(StreamHeader)
if stream == "" {
return nil, errors.New("nats: missing stream header")
}
seqStr := r.Header.Get(SequenceHeader)
if seqStr == "" {
return nil, errors.New("nats: missing sequence header")
}
seq, err := strconv.ParseUint(seqStr, 10, 64)
if err != nil {
return nil, fmt.Errorf("nats: invalid sequence header '%s': %v", seqStr, err)
}
timeStr := r.Header.Get(TimeStampHeaer)
if timeStr == "" {
return nil, errors.New("nats: missing timestamp header")
}
tm, err := time.Parse(time.RFC3339Nano, timeStr)
if err != nil {
return nil, fmt.Errorf("nats: invalid timestamp header '%s': %v", timeStr, err)
}
subj := r.Header.Get(SubjectHeader)
if subj == "" {
return nil, errors.New("nats: missing subject header")
}
return &RawStreamMsg{
Subject: subj,
Sequence: seq,
Header: r.Header,
Data: r.Data,
Time: tm,
}, nil
}
// DeleteMsg deletes a message from a stream.
// On the server, the message is marked as erased, but not overwritten.
func (s *stream) DeleteMsg(ctx context.Context, seq uint64) error {
return s.deleteMsg(ctx, &msgDeleteRequest{Seq: seq, NoErase: true})
}
// SecureDeleteMsg deletes a message from a stream. The deleted message
// is overwritten with random data. As a result, this operation is slower
// than DeleteMsg.
func (s *stream) SecureDeleteMsg(ctx context.Context, seq uint64) error {
return s.deleteMsg(ctx, &msgDeleteRequest{Seq: seq})
}
func (s *stream) deleteMsg(ctx context.Context, req *msgDeleteRequest) error {
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
r, err := json.Marshal(req)
if err != nil {
return err
}
subj := fmt.Sprintf(apiMsgDeleteT, s.name)
var resp msgDeleteResponse
if _, err = s.js.apiRequestJSON(ctx, subj, &resp, r); err != nil {
return err
}
if !resp.Success {
return fmt.Errorf("%w: %s", ErrMsgDeleteUnsuccessful, resp.Error.Error())
}
return nil
}
// ListConsumers returns ConsumerInfoLister enabling iterating over a
// channel of consumer infos.
func (s *stream) ListConsumers(ctx context.Context) ConsumerInfoLister {
l := &consumerLister{
js: s.js,
consumers: make(chan *ConsumerInfo),
}
go func() {
defer close(l.consumers)
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
for {
page, err := l.consumerInfos(ctx, s.name)
if err != nil && !errors.Is(err, ErrEndOfData) {
l.err = err
return
}
for _, info := range page {
select {
case <-ctx.Done():
l.err = ctx.Err()
return
default:
}
if info != nil {
l.consumers <- info
}
}
if errors.Is(err, ErrEndOfData) {
return
}
}
}()
return l
}
func (s *consumerLister) Info() <-chan *ConsumerInfo {
return s.consumers
}
func (s *consumerLister) Err() error {
return s.err
}
// ConsumerNames returns a ConsumerNameLister enabling iterating over a
// channel of consumer names.
func (s *stream) ConsumerNames(ctx context.Context) ConsumerNameLister {
l := &consumerLister{
js: s.js,
names: make(chan string),
}
go func() {
defer close(l.names)
ctx, cancel := s.js.wrapContextWithoutDeadline(ctx)
if cancel != nil {
defer cancel()
}
for {
page, err := l.consumerNames(ctx, s.name)
if err != nil && !errors.Is(err, ErrEndOfData) {
l.err = err
return
}
for _, info := range page {
select {
case l.names <- info:
case <-ctx.Done():
l.err = ctx.Err()
return
}
}
if errors.Is(err, ErrEndOfData) {
return
}
}
}()
return l
}
func (s *consumerLister) Name() <-chan string {
return s.names
}
// consumerInfos fetches the next ConsumerInfo page
func (s *consumerLister) consumerInfos(ctx context.Context, stream string) ([]*ConsumerInfo, error) {
if s.pageInfo != nil && s.offset >= s.pageInfo.Total {
return nil, ErrEndOfData
}
req, err := json.Marshal(
apiPagedRequest{Offset: s.offset},
)
if err != nil {
return nil, err
}
slSubj := fmt.Sprintf(apiConsumerListT, stream)
var resp consumerListResponse
_, err = s.js.apiRequestJSON(ctx, slSubj, &resp, req)
if err != nil {
return nil, err
}
if resp.Error != nil {
return nil, resp.Error
}
s.pageInfo = &resp.apiPaged
s.offset += len(resp.Consumers)
return resp.Consumers, nil
}
// consumerNames fetches the next consumer names page
func (s *consumerLister) consumerNames(ctx context.Context, stream string) ([]string, error) {
if s.pageInfo != nil && s.offset >= s.pageInfo.Total {
return nil, ErrEndOfData
}
req, err := json.Marshal(
apiPagedRequest{Offset: s.offset},
)
if err != nil {
return nil, err
}
slSubj := fmt.Sprintf(apiConsumerNamesT, stream)
var resp consumerNamesResponse
_, err = s.js.apiRequestJSON(ctx, slSubj, &resp, req)
if err != nil {
return nil, err
}
if resp.Error != nil {
return nil, resp.Error
}
s.pageInfo = &resp.apiPaged
s.offset += len(resp.Consumers)
return resp.Consumers, nil
}
// UnpinConsumer unpins the currently pinned client for a consumer for the given group name.
// If consumer does not exist, ErrConsumerNotFound is returned.
func (s *stream) UnpinConsumer(ctx context.Context, consumer string, group string) error {
return unpinConsumer(ctx, s.js, s.name, consumer, group)
}
// ResetConsumer resets a consumer's delivery state. See [Stream.ResetConsumer].
func (s *stream) ResetConsumer(ctx context.Context, consumer string) (*ConsumerResetResponse, error) {
return resetConsumer(ctx, s.js, s.name, consumer, 0)
}
// ResetConsumerToSequence resets a consumer's delivery state to the given
// stream sequence. See [Stream.ResetConsumerToSequence].
func (s *stream) ResetConsumerToSequence(ctx context.Context, consumer string, seq uint64) (*ConsumerResetResponse, error) {
return resetConsumer(ctx, s.js, s.name, consumer, seq)
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jetstream/stream_config.go
Generated Vendored
+822
View File
@@ -0,0 +1,822 @@
// Copyright 2022-2026 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jetstream
import (
"encoding/json"
"errors"
"fmt"
"strings"
"time"
)
type (
// StreamInfo shows config and current state for this stream.
StreamInfo struct {
// Config contains the configuration settings of the stream, set when
// creating or updating the stream.
Config StreamConfig `json:"config"`
// Created is the timestamp when the stream was created.
Created time.Time `json:"created"`
// State provides the state of the stream at the time of request,
// including metrics like the number of messages in the stream, total
// bytes, etc.
State StreamState `json:"state"`
// Cluster contains information about the cluster to which this stream
// belongs (if applicable).
Cluster *ClusterInfo `json:"cluster,omitempty"`
// Mirror contains information about another stream this one is
// mirroring. Mirroring is used to create replicas of another stream's
// data. This field is omitted if the stream is not mirroring another
// stream.
Mirror *StreamSourceInfo `json:"mirror,omitempty"`
// Sources is a list of source streams from which this stream collects
// data.
Sources []*StreamSourceInfo `json:"sources,omitempty"`
// TimeStamp indicates when the info was gathered by the server.
TimeStamp time.Time `json:"ts"`
}
// StreamConfig is the configuration of a JetStream stream.
StreamConfig struct {
// Name is the name of the stream. It is required and must be unique
// across the JetStream account.
//
// Name Names cannot contain whitespace, ., *, >, path separators
// (forward or backwards slash), and non-printable characters.
Name string `json:"name"`
// Description is an optional description of the stream.
Description string `json:"description,omitempty"`
// Subjects is a list of subjects that the stream is listening on.
// Wildcards are supported. Subjects cannot be set if the stream is
// created as a mirror.
Subjects []string `json:"subjects,omitempty"`
// Retention defines the message retention policy for the stream.
// Defaults to LimitsPolicy.
Retention RetentionPolicy `json:"retention"`
// MaxConsumers specifies the maximum number of consumers allowed for
// the stream.
MaxConsumers int `json:"max_consumers"`
// MaxMsgs is the maximum number of messages the stream will store.
// After reaching the limit, stream adheres to the discard policy.
// If not set, server default is -1 (unlimited).
MaxMsgs int64 `json:"max_msgs"`
// MaxBytes is the maximum total size of messages the stream will store.
// After reaching the limit, stream adheres to the discard policy.
// If not set, server default is -1 (unlimited).
MaxBytes int64 `json:"max_bytes"`
// Discard defines the policy for handling messages when the stream
// reaches its limits in terms of number of messages or total bytes.
Discard DiscardPolicy `json:"discard"`
// DiscardNewPerSubject is a flag to enable discarding new messages per
// subject when limits are reached. Requires DiscardPolicy to be
// DiscardNew and the MaxMsgsPerSubject to be set.
DiscardNewPerSubject bool `json:"discard_new_per_subject,omitempty"`
// MaxAge is the maximum age of messages that the stream will retain.
MaxAge time.Duration `json:"max_age"`
// MaxMsgsPerSubject is the maximum number of messages per subject that
// the stream will retain.
MaxMsgsPerSubject int64 `json:"max_msgs_per_subject"`
// MaxMsgSize is the maximum size of any single message in the stream.
MaxMsgSize int32 `json:"max_msg_size,omitempty"`
// Storage specifies the type of storage backend used for the stream
// (file or memory).
Storage StorageType `json:"storage"`
// Replicas is the number of stream replicas in clustered JetStream.
// Defaults to 1, maximum is 5.
Replicas int `json:"num_replicas"`
// NoAck is a flag to disable acknowledging messages received by this
// stream.
//
// If set to true, publish methods from the JetStream client will not
// work as expected, since they rely on acknowledgements. Core NATS
// publish methods should be used instead. Note that this will make
// message delivery less reliable.
NoAck bool `json:"no_ack,omitempty"`
// Duplicates is the window within which to track duplicate messages.
// If not set, server default is 2 minutes.
Duplicates time.Duration `json:"duplicate_window,omitempty"`
// Placement is used to declare where the stream should be placed via
// tags and/or an explicit cluster name.
Placement *Placement `json:"placement,omitempty"`
// Mirror defines the configuration for mirroring another stream.
Mirror *StreamSource `json:"mirror,omitempty"`
// Sources is a list of other streams this stream sources messages from.
Sources []*StreamSource `json:"sources,omitempty"`
// Sealed streams do not allow messages to be published or deleted via limits or API,
// sealed streams can not be unsealed via configuration update. Can only
// be set on already created streams via the Update API.
Sealed bool `json:"sealed,omitempty"`
// DenyDelete restricts the ability to delete messages from a stream via
// the API. Defaults to false.
DenyDelete bool `json:"deny_delete,omitempty"`
// DenyPurge restricts the ability to purge messages from a stream via
// the API. Defaults to false.
DenyPurge bool `json:"deny_purge,omitempty"`
// AllowRollup allows the use of the Nats-Rollup header to replace all
// contents of a stream, or subject in a stream, with a single new
// message.
AllowRollup bool `json:"allow_rollup_hdrs,omitempty"`
// Compression specifies the message storage compression algorithm.
// Defaults to NoCompression.
Compression StoreCompression `json:"compression"`
// FirstSeq is the initial sequence number of the first message in the
// stream.
FirstSeq uint64 `json:"first_seq,omitempty"`
// SubjectTransform allows applying a transformation to matching
// messages' subjects.
SubjectTransform *SubjectTransformConfig `json:"subject_transform,omitempty"`
// RePublish allows immediate republishing a message to the configured
// subject after it's stored.
RePublish *RePublish `json:"republish,omitempty"`
// AllowDirect enables direct access to individual messages using direct
// get API. Defaults to false.
AllowDirect bool `json:"allow_direct"`
// MirrorDirect enables direct access to individual messages from the
// origin stream using direct get API. Defaults to false.
MirrorDirect bool `json:"mirror_direct"`
// ConsumerLimits defines limits of certain values that consumers can
// set, defaults for those who don't set these settings
ConsumerLimits StreamConsumerLimits `json:"consumer_limits,omitempty"`
// Metadata is a set of application-defined key-value pairs for
// associating metadata on the stream. This feature requires nats-server
// v2.10.0 or later.
Metadata map[string]string `json:"metadata,omitempty"`
// Template identifies the template that manages the Stream.
// Deprecated: This feature is no longer supported.
Template string `json:"template_owner,omitempty"`
// AllowMsgTTL allows header initiated per-message TTLs.
// This feature requires nats-server v2.11.0 or later.
AllowMsgTTL bool `json:"allow_msg_ttl,omitempty"`
// Enables and sets a duration for adding server markers for delete, purge and max age limits.
// This feature requires nats-server v2.11.0 or later.
SubjectDeleteMarkerTTL time.Duration `json:"subject_delete_marker_ttl,omitempty"`
// AllowMsgCounter enables the feature
AllowMsgCounter bool `json:"allow_msg_counter,omitempty"`
// AllowAtomicPublish allows atomic batch publishing into the stream.
AllowAtomicPublish bool `json:"allow_atomic,omitempty"`
// AllowMsgSchedules enables the scheduling of messages
AllowMsgSchedules bool `json:"allow_msg_schedules,omitempty"`
// PersistMode allows to opt-in to different persistence mode settings.
PersistMode PersistModeType `json:"persist_mode,omitempty"`
// AllowBatchPublish allows fast batch publishing into the stream.
// This feature requires nats-server v2.14.0 or later.
AllowBatchPublish bool `json:"allow_batched,omitempty"`
}
// StreamSourceInfo shows information about an upstream stream
// source/mirror.
StreamSourceInfo struct {
// Name is the name of the stream that is being replicated.
Name string `json:"name"`
// Lag informs how many messages behind the source/mirror operation is.
// This will only show correctly if there is active communication
// with stream/mirror.
Lag uint64 `json:"lag"`
// Seq is the last message sequence seen from the source stream. Not
// reported for mirrors.
Seq uint64 `json:"seq,omitempty"`
// Active informs when last the mirror or sourced stream had activity.
// Value will be -1 when there has been no activity.
Active time.Duration `json:"active"`
// FilterSubject is the subject filter defined for this source/mirror.
FilterSubject string `json:"filter_subject,omitempty"`
// SubjectTransforms is a list of subject transforms defined for this
// source/mirror.
SubjectTransforms []SubjectTransformConfig `json:"subject_transforms,omitempty"`
}
// StreamState is the state of a JetStream stream at the time of request.
StreamState struct {
// Msgs is the number of messages stored in the stream.
Msgs uint64 `json:"messages"`
// Bytes is the number of bytes stored in the stream.
Bytes uint64 `json:"bytes"`
// FirstSeq is the sequence number of the first message in the stream.
FirstSeq uint64 `json:"first_seq"`
// FirstTime is the timestamp of the first message in the stream.
FirstTime time.Time `json:"first_ts"`
// LastSeq is the sequence number of the last message in the stream.
LastSeq uint64 `json:"last_seq"`
// LastTime is the timestamp of the last message in the stream.
LastTime time.Time `json:"last_ts"`
// Consumers is the number of consumers on the stream.
Consumers int `json:"consumer_count"`
// Deleted is a list of sequence numbers that have been removed from the
// stream. This field will only be returned if the stream has been
// fetched with the DeletedDetails option.
Deleted []uint64 `json:"deleted"`
// NumDeleted is the number of messages that have been removed from the
// stream. Only deleted messages causing a gap in stream sequence numbers
// are counted. Messages deleted at the beginning or end of the stream
// are not counted.
NumDeleted int `json:"num_deleted"`
// NumSubjects is the number of unique subjects the stream has received
// messages on.
NumSubjects uint64 `json:"num_subjects"`
// Subjects is a map of subjects the stream has received messages on
// with message count per subject. This field will only be returned if
// the stream has been fetched with the SubjectFilter option.
Subjects map[string]uint64 `json:"subjects"`
}
// ClusterInfo shows information about the underlying set of servers that
// make up the stream or consumer.
ClusterInfo struct {
// Name is the name of the cluster.
Name string `json:"name,omitempty"`
// RaftGroup is the name of the Raft group managing the asset (in
// clustered environments).
RaftGroup string `json:"raft_group,omitempty"`
// Leader is the server name of the RAFT leader.
Leader string `json:"leader,omitempty"`
// LeaderSince is the time that it was elected as leader in RFC3339
// format, absent when not the leader.
LeaderSince *time.Time `json:"leader_since,omitempty"`
// SystemAcc indicates if the traffic_account is the system account.
// When true, replication traffic goes over the system account.
SystemAcc bool `json:"system_account,omitempty"`
// TrafficAcc is the account where the replication traffic goes over.
TrafficAcc string `json:"traffic_account,omitempty"`
// Replicas is the list of members of the RAFT cluster.
Replicas []*PeerInfo `json:"replicas,omitempty"`
// Desired is the desired set of servers that should make up the
// stream or consumer. Absent when no reconfiguration is in progress.
Desired *DesiredClusterInfo `json:"desired,omitempty"`
}
// DesiredClusterInfo shows the desired set of servers that should make up
// the stream or consumer.
DesiredClusterInfo struct {
// Created is when the desired state was recorded on the assignment.
Created time.Time `json:"created"`
// Name is the name of the target cluster the group should end up in.
Name string `json:"name,omitempty"`
// Replicas are the peers chosen to be the final peer set, whereas
// ClusterInfo.Replicas holds the peers that currently host the stream
// or consumer. Absent while scaling down until the final peer set is
// selected.
Replicas []*DesiredPeerInfo `json:"replicas,omitempty"`
// Origin is the configuration the reconfiguration can be rolled back
// to if it is canceled.
Origin *DesiredClusterInfoOrigin `json:"origin,omitempty"`
// Status is what the group leader is currently doing to reach the
// desired state, or what it is waiting on.
Status *DesiredClusterInfoStatus `json:"status,omitempty"`
}
// DesiredClusterInfoOrigin is the configuration the reconfiguration can
// be rolled back to if it is canceled.
DesiredClusterInfoOrigin struct {
// Replicas is the original replicas before it was updated.
Replicas int `json:"replicas"`
// Placement is the original placement before it was updated.
Placement *Placement `json:"placement,omitempty"`
// Retention remains active while changing between retention
// policies, until unset.
Retention *RetentionPolicy `json:"retention,omitempty"`
}
// DesiredPeerInfo is a minimal version of PeerInfo describing a peer in
// the desired peer set.
DesiredPeerInfo struct {
// Name is the unique name for the peer.
Name string `json:"name"`
// Offline indicates the peer has not been seen recently.
Offline bool `json:"offline,omitempty"`
// Peer is the unique ID for the peer.
Peer string `json:"peer"`
}
// DesiredClusterInfoStatus is what the group leader is currently doing to
// reach the desired state, or what it is waiting on.
DesiredClusterInfoStatus struct {
// Description is a short status line describing what the group leader
// is currently doing to move this group toward its desired state, or
// what it is waiting on.
Description string `json:"description"`
// Type classifies the description by what has to change for the
// migration to advance, so it can be matched on without parsing the
// status line.
Type MigrationStatusType `json:"type"`
// Err is the underlying failure behind this status, if it had one.
// Only set for faults that persist across cycles, never for races
// that resolve themselves.
Err string `json:"err,omitempty"`
}
// PeerInfo shows information about the peers in the cluster that are
// supporting the stream or consumer.
PeerInfo struct {
// Name is the server name of the peer.
Name string `json:"name"`
// Current indicates if the peer is up to date and synchronized with the
// leader.
Current bool `json:"current"`
// Offline indicates if the peer is considered offline by the group.
Offline bool `json:"offline,omitempty"`
// Active it the duration since this peer was last seen.
Active time.Duration `json:"active"`
// Lag is the number of uncommitted operations this peer is behind the
// leader.
Lag uint64 `json:"lag,omitempty"`
// Peer is the unique ID for the peer.
Peer string `json:"peer"`
// Pending indicates the peer is part of the assignment, but is not a
// peer of the Raft group yet or is being removed.
Pending bool `json:"pending,omitempty"`
}
// SubjectTransformConfig is for applying a subject transform (to matching
// messages) before doing anything else when a new message is received.
SubjectTransformConfig struct {
// Source is the subject pattern to match incoming messages against.
Source string `json:"src"`
// Destination is the subject pattern to remap the subject to.
Destination string `json:"dest"`
}
// RePublish is for republishing messages once committed to a stream. The
// original subject is remapped from the subject pattern to the destination
// pattern.
RePublish struct {
// Source is the subject pattern to match incoming messages against.
Source string `json:"src,omitempty"`
// Destination is the subject pattern to republish the subject to.
Destination string `json:"dest"`
// HeadersOnly is a flag to indicate that only the headers should be
// republished.
HeadersOnly bool `json:"headers_only,omitempty"`
}
// Placement is used to guide placement of streams in clustered JetStream.
Placement struct {
// Cluster is the name of the cluster to which the stream should be
// assigned.
Cluster string `json:"cluster"`
// Tags are used to match streams to servers in the cluster. A stream
// will be assigned to a server with a matching tag.
Tags []string `json:"tags,omitempty"`
}
// StreamSource dictates how streams can source from other streams.
StreamSource struct {
// Name is the name of the stream to source from.
Name string `json:"name"`
// OptStartSeq is the sequence number to start sourcing from.
OptStartSeq uint64 `json:"opt_start_seq,omitempty"`
// OptStartTime is the timestamp of messages to start sourcing from.
OptStartTime *time.Time `json:"opt_start_time,omitempty"`
// FilterSubject is the subject filter used to only replicate messages
// with matching subjects.
FilterSubject string `json:"filter_subject,omitempty"`
// SubjectTransforms is a list of subject transforms to apply to
// matching messages.
//
// Subject transforms on sources and mirrors are also used as subject
// filters with optional transformations.
SubjectTransforms []SubjectTransformConfig `json:"subject_transforms,omitempty"`
// External is a configuration referencing a stream source in another
// account or JetStream domain.
External *ExternalStream `json:"external,omitempty"`
// Consumer is consumer information for durable sourcing.
//
// This feature requires nats-server v2.14.0 or later.
Consumer *StreamConsumerSource `json:"consumer,omitempty"`
// Domain is used to configure a stream source in another JetStream
// domain. This setting will set the External field with the appropriate
// APIPrefix.
Domain string `json:"-"`
}
// StreamConsumerSource is consumer information for durable sourcing.
//
// This feature requires nats-server v2.14.0 or later.
StreamConsumerSource struct {
// Name is the consumer name.
Name string `json:"name,omitempty"`
// DeliverSubject is the subject to deliver messages to.
DeliverSubject string `json:"deliver_subject,omitempty"`
}
// ExternalStream allows you to qualify access to a stream source in another
// account.
ExternalStream struct {
// APIPrefix is the subject prefix that imports the other account/domain
// $JS.API.CONSUMER.> subjects.
APIPrefix string `json:"api"`
// DeliverPrefix is the delivery subject to use for the push consumer.
DeliverPrefix string `json:"deliver"`
}
// StreamConsumerLimits are the limits for a consumer on a stream. These can
// be overridden on a per consumer basis.
StreamConsumerLimits struct {
// InactiveThreshold is a duration which instructs the server to clean
// up the consumer if it has been inactive for the specified duration.
InactiveThreshold time.Duration `json:"inactive_threshold,omitempty"`
// MaxAckPending is a maximum number of outstanding unacknowledged
// messages for a consumer.
MaxAckPending int `json:"max_ack_pending,omitempty"`
}
// DiscardPolicy determines how to proceed when limits of messages or bytes
// are reached.
DiscardPolicy int
// RetentionPolicy determines how messages in a stream are retained.
RetentionPolicy int
// StorageType determines how messages are stored for retention.
StorageType int
// StoreCompression determines how messages are compressed.
StoreCompression uint8
// PersistModeType determines what persistence mode the stream uses.
PersistModeType int
// MigrationStatusType classifies a migration status by what has to
// change for the migration to advance.
MigrationStatusType string
)
const (
// MigrationStatusMeta means the meta leader must record or advance the
// desired state.
MigrationStatusMeta MigrationStatusType = "meta"
// MigrationStatusMembership means a proposed membership change must
// commit.
MigrationStatusMembership MigrationStatusType = "membership"
// MigrationStatusSnapshot means a snapshot must be installed.
MigrationStatusSnapshot MigrationStatusType = "snapshot"
// MigrationStatusCatchup means more peers must catch up before one can
// be removed.
MigrationStatusCatchup MigrationStatusType = "catchup"
// MigrationStatusQuorum means more peers must come online before acting
// without losing quorum.
MigrationStatusQuorum MigrationStatusType = "quorum"
// MigrationStatusBlocked means another asset must move first.
MigrationStatusBlocked MigrationStatusType = "blocked"
// MigrationStatusUnavailable means there is nothing to do: shutting down,
// or the assignment is gone.
MigrationStatusUnavailable MigrationStatusType = "unavailable"
)
const (
// LimitsPolicy (default) means that messages are retained until any given
// limit is reached. This could be one of MaxMsgs, MaxBytes, or MaxAge.
LimitsPolicy RetentionPolicy = iota
// InterestPolicy specifies that when all known observables have
// acknowledged a message it can be removed.
InterestPolicy
// WorkQueuePolicy specifies that when the first worker or subscriber
// acknowledges the message it can be removed.
WorkQueuePolicy
)
const (
// DiscardOld will remove older messages to return to the limits. This is
// the default.
DiscardOld DiscardPolicy = iota
// DiscardNew will fail to store new messages once the limits are reached.
DiscardNew
)
const (
limitsPolicyString = "limits"
interestPolicyString = "interest"
workQueuePolicyString = "workqueue"
)
const (
// DefaultPersistMode specifies the default persist mode. Writes to the stream will immediately be flushed.
// The publish acknowledgement will be sent after the persisting completes.
DefaultPersistMode = PersistModeType(iota)
// AsyncPersistMode specifies writes to the stream will be flushed asynchronously.
// The publish acknowledgement may be sent before the persisting completes.
// This means writes could be lost if they weren't flushed prior to a hard kill of the server.
AsyncPersistMode
)
func (rp RetentionPolicy) String() string {
switch rp {
case LimitsPolicy:
return "Limits"
case InterestPolicy:
return "Interest"
case WorkQueuePolicy:
return "WorkQueue"
default:
return "Unknown Retention Policy"
}
}
func (rp RetentionPolicy) MarshalJSON() ([]byte, error) {
switch rp {
case LimitsPolicy:
return json.Marshal(limitsPolicyString)
case InterestPolicy:
return json.Marshal(interestPolicyString)
case WorkQueuePolicy:
return json.Marshal(workQueuePolicyString)
default:
return nil, fmt.Errorf("nats: can not marshal %v", rp)
}
}
func (rp *RetentionPolicy) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString(limitsPolicyString):
*rp = LimitsPolicy
case jsonString(interestPolicyString):
*rp = InterestPolicy
case jsonString(workQueuePolicyString):
*rp = WorkQueuePolicy
default:
return fmt.Errorf("nats: can not unmarshal %q", data)
}
return nil
}
func (dp DiscardPolicy) String() string {
switch dp {
case DiscardOld:
return "DiscardOld"
case DiscardNew:
return "DiscardNew"
default:
return "Unknown Discard Policy"
}
}
func (dp DiscardPolicy) MarshalJSON() ([]byte, error) {
switch dp {
case DiscardOld:
return json.Marshal("old")
case DiscardNew:
return json.Marshal("new")
default:
return nil, fmt.Errorf("nats: can not marshal %v", dp)
}
}
func (dp *DiscardPolicy) UnmarshalJSON(data []byte) error {
switch strings.ToLower(string(data)) {
case jsonString("old"):
*dp = DiscardOld
case jsonString("new"):
*dp = DiscardNew
default:
return fmt.Errorf("nats: can not unmarshal %q", data)
}
return nil
}
func (pm PersistModeType) String() string {
switch pm {
case DefaultPersistMode:
return "Default"
case AsyncPersistMode:
return "Async"
default:
return "Unknown Persist Mode"
}
}
func (pm PersistModeType) MarshalJSON() ([]byte, error) {
switch pm {
case DefaultPersistMode:
return json.Marshal("default")
case AsyncPersistMode:
return json.Marshal("async")
default:
return nil, fmt.Errorf("nats: can not marshal %v", pm)
}
}
func (pm *PersistModeType) UnmarshalJSON(data []byte) error {
switch strings.ToLower(string(data)) {
case jsonString("default"):
*pm = DefaultPersistMode
case jsonString("async"):
*pm = AsyncPersistMode
default:
return fmt.Errorf("nats: can not unmarshal %q", data)
}
return nil
}
const (
// FileStorage specifies on disk storage. It's the default.
FileStorage StorageType = iota
// MemoryStorage specifies in memory only.
MemoryStorage
)
const (
memoryStorageString = "memory"
fileStorageString = "file"
)
func (st StorageType) String() string {
switch st {
case MemoryStorage:
return "Memory"
case FileStorage:
return "File"
default:
return "Unknown Storage Type"
}
}
func (st StorageType) MarshalJSON() ([]byte, error) {
switch st {
case MemoryStorage:
return json.Marshal(memoryStorageString)
case FileStorage:
return json.Marshal(fileStorageString)
default:
return nil, fmt.Errorf("nats: can not marshal %v", st)
}
}
func (st *StorageType) UnmarshalJSON(data []byte) error {
switch string(data) {
case jsonString(memoryStorageString):
*st = MemoryStorage
case jsonString(fileStorageString):
*st = FileStorage
default:
return fmt.Errorf("nats: can not unmarshal %q", data)
}
return nil
}
func jsonString(s string) string {
return "\"" + s + "\""
}
const (
// NoCompression disables compression on the stream. This is the default.
NoCompression StoreCompression = iota
// S2Compression enables S2 compression on the stream.
S2Compression
)
func (alg StoreCompression) String() string {
switch alg {
case NoCompression:
return "None"
case S2Compression:
return "S2"
default:
return "Unknown StoreCompression"
}
}
func (alg StoreCompression) MarshalJSON() ([]byte, error) {
var str string
switch alg {
case S2Compression:
str = "s2"
case NoCompression:
str = "none"
default:
return nil, errors.New("unknown compression algorithm")
}
return json.Marshal(str)
}
func (alg *StoreCompression) UnmarshalJSON(b []byte) error {
var str string
if err := json.Unmarshal(b, &str); err != nil {
return err
}
switch str {
case "s2":
*alg = S2Compression
case "none":
*alg = NoCompression
default:
return errors.New("unknown compression algorithm")
}
return nil
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/js.go
Generated Vendored
+4297
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/jserrors.go
Generated Vendored
+273
View File
@@ -0,0 +1,273 @@
// Copyright 2020-2025 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package nats
import (
"errors"
"fmt"
)
var (
// API errors
// ErrJetStreamNotEnabled is an error returned when JetStream is not enabled for an account.
//
// Note: This error will not be returned in clustered mode, even if each
// server in the cluster does not have JetStream enabled. In clustered mode,
// requests will time out instead.
ErrJetStreamNotEnabled JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabled, Description: "jetstream not enabled", Code: 503}}
// ErrJetStreamNotEnabledForAccount is an error returned when JetStream is not enabled for an account.
ErrJetStreamNotEnabledForAccount JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeJetStreamNotEnabledForAccount, Description: "jetstream not enabled for account", Code: 503}}
// ErrStreamNotFound is an error returned when stream with given name does not exist.
ErrStreamNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNotFound, Description: "stream not found", Code: 404}}
// ErrStreamNameAlreadyInUse is returned when a stream with given name already exists and has a different configuration.
ErrStreamNameAlreadyInUse JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeStreamNameInUse, Description: "stream name already in use", Code: 400}}
// ErrStreamSubjectTransformNotSupported is returned when the connected nats-server version does not support setting
// the stream subject transform. If this error is returned when executing AddStream(), the stream with invalid
// configuration was already created in the server.
ErrStreamSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
// ErrStreamSourceSubjectTransformNotSupported is returned when the connected nats-server version does not support setting
// the stream source subject transform. If this error is returned when executing AddStream(), the stream with invalid
// configuration was already created in the server.
ErrStreamSourceSubjectTransformNotSupported JetStreamError = &jsError{message: "stream subject transformation not supported by nats-server"}
// ErrStreamSourceNotSupported is returned when the connected nats-server version does not support setting
// the stream sources. If this error is returned when executing AddStream(), the stream with invalid
// configuration was already created in the server.
ErrStreamSourceNotSupported JetStreamError = &jsError{message: "stream sourcing is not supported by nats-server"}
// ErrStreamSourceMultipleSubjectTransformsNotSupported is returned when the connected nats-server version does not support setting
// the stream sources. If this error is returned when executing AddStream(), the stream with invalid
// configuration was already created in the server.
ErrStreamSourceMultipleSubjectTransformsNotSupported JetStreamError = &jsError{message: "stream sourcing with multiple subject transforms not supported by nats-server"}
// ErrConsumerNotFound is an error returned when consumer with given name does not exist.
ErrConsumerNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerNotFound, Description: "consumer not found", Code: 404}}
// ErrConsumerCreationResponseEmpty is an error returned when the response from the server
// when creating a consumer is empty. This means that the state of the consumer is unknown and
// the consumer may not have been created successfully.
ErrConsumerCreationResponseEmpty JetStreamError = &jsError{message: "consumer creation response is empty"}
// ErrMsgNotFound is returned when message with provided sequence number does npt exist.
ErrMsgNotFound JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeMessageNotFound, Description: "message not found", Code: 404}}
// ErrBadRequest is returned when invalid request is sent to JetStream API.
ErrBadRequest JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeBadRequest, Description: "bad request", Code: 400}}
// ErrDuplicateFilterSubjects is returned when both FilterSubject and FilterSubjects are specified when creating consumer.
ErrDuplicateFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeDuplicateFilterSubjects, Description: "consumer cannot have both FilterSubject and FilterSubjects specified", Code: 500}}
// ErrDuplicateFilterSubjects is returned when filter subjects overlap when creating consumer.
ErrOverlappingFilterSubjects JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeOverlappingFilterSubjects, Description: "consumer subject filters cannot overlap", Code: 500}}
// ErrEmptyFilter is returned when a filter in FilterSubjects is empty.
ErrEmptyFilter JetStreamError = &jsError{apiErr: &APIError{ErrorCode: JSErrCodeConsumerEmptyFilter, Description: "consumer filter in FilterSubjects cannot be empty", Code: 500}}
// Client errors
// ErrConsumerNameAlreadyInUse is an error returned when consumer with given name already exists.
ErrConsumerNameAlreadyInUse JetStreamError = &jsError{message: "consumer name already in use"}
// ErrConsumerNotActive is an error returned when consumer is not active.
ErrConsumerNotActive JetStreamError = &jsError{message: "consumer not active"}
// ErrInvalidJSAck is returned when JetStream ack from message publish is invalid.
ErrInvalidJSAck JetStreamError = &jsError{message: "invalid jetstream publish response"}
// ErrStreamConfigRequired is returned when empty stream configuration is supplied to add/update stream.
ErrStreamConfigRequired JetStreamError = &jsError{message: "stream configuration is required"}
// ErrStreamNameRequired is returned when the provided stream name is empty.
ErrStreamNameRequired JetStreamError = &jsError{message: "stream name is required"}
// ErrConsumerNameRequired is returned when the provided consumer durable name is empty.
ErrConsumerNameRequired JetStreamError = &jsError{message: "consumer name is required"}
// ErrConsumerMultipleFilterSubjectsNotSupported is returned when the connected nats-server version does not support setting
// multiple filter subjects with filter_subjects field. If this error is returned when executing AddConsumer(), the consumer with invalid
// configuration was already created in the server.
ErrConsumerMultipleFilterSubjectsNotSupported JetStreamError = &jsError{message: "multiple consumer filter subjects not supported by nats-server"}
// ErrConsumerConfigRequired is returned when empty consumer consuguration is supplied to add/update consumer.
ErrConsumerConfigRequired JetStreamError = &jsError{message: "consumer configuration is required"}
// ErrPullSubscribeToPushConsumer is returned when attempting to use PullSubscribe on push consumer.
ErrPullSubscribeToPushConsumer JetStreamError = &jsError{message: "cannot pull subscribe to push based consumer"}
// ErrPullSubscribeRequired is returned when attempting to use subscribe methods not suitable for pull consumers for pull consumers.
ErrPullSubscribeRequired JetStreamError = &jsError{message: "must use pull subscribe to bind to pull based consumer"}
// ErrMsgAlreadyAckd is returned when attempting to acknowledge message more than once.
ErrMsgAlreadyAckd JetStreamError = &jsError{message: "message was already acknowledged"}
// ErrNoStreamResponse is returned when there is no response from stream (e.g. no responders error).
ErrNoStreamResponse JetStreamError = &jsError{message: "no response from stream"}
// ErrNotJSMessage is returned when attempting to get metadata from non JetStream message .
ErrNotJSMessage JetStreamError = &jsError{message: "not a jetstream message"}
// ErrInvalidStreamName is returned when the provided stream name is invalid (contains '.' or ' ').
ErrInvalidStreamName JetStreamError = &jsError{message: "invalid stream name"}
// ErrInvalidConsumerName is returned when the provided consumer name is invalid (contains '.' or ' ').
ErrInvalidConsumerName JetStreamError = &jsError{message: "invalid consumer name"}
// ErrInvalidFilterSubject is returned when the provided filter subject is invalid.
ErrInvalidFilterSubject JetStreamError = &jsError{message: "invalid filter subject"}
// ErrNoMatchingStream is returned when stream lookup by subject is unsuccessful.
ErrNoMatchingStream JetStreamError = &jsError{message: "no stream matches subject"}
// ErrSubjectMismatch is returned when the provided subject does not match consumer's filter subject.
ErrSubjectMismatch JetStreamError = &jsError{message: "subject does not match consumer"}
// ErrContextAndTimeout is returned when attempting to use both context and timeout.
ErrContextAndTimeout JetStreamError = &jsError{message: "context and timeout can not both be set"}
// ErrCantAckIfConsumerAckNone is returned when attempting to ack a message for consumer with AckNone policy set.
ErrCantAckIfConsumerAckNone JetStreamError = &jsError{message: "cannot acknowledge a message for a consumer with AckNone policy"}
// ErrConsumerDeleted is returned when attempting to send pull request to a consumer which does not exist
ErrConsumerDeleted JetStreamError = &jsError{message: "consumer deleted"}
// ErrConsumerLeadershipChanged is returned when pending requests are no longer valid after leadership has changed
ErrConsumerLeadershipChanged JetStreamError = &jsError{message: "Leadership Changed"}
// ErrConsumerInfoOnOrderedReset is returned when attempting to fetch consumer info for an ordered consumer that is currently being recreated.
ErrConsumerInfoOnOrderedReset JetStreamError = &jsError{message: "cannot fetch consumer info; ordered consumer is being reset"}
// ErrNoHeartbeat is returned when no heartbeat is received from server when sending requests with pull consumer.
ErrNoHeartbeat JetStreamError = &jsError{message: "no heartbeat received"}
// ErrSubscriptionClosed is returned when attempting to send pull request to a closed subscription
ErrSubscriptionClosed JetStreamError = &jsError{message: "subscription closed"}
// ErrJetStreamPublisherClosed is returned for each unfinished ack future when JetStream.Cleanup is called.
ErrJetStreamPublisherClosed JetStreamError = &jsError{message: "jetstream context closed"}
// Deprecated: ErrInvalidDurableName is no longer returned and will be removed in future releases.
// Use ErrInvalidConsumerName instead.
ErrInvalidDurableName = errors.New("nats: invalid durable name")
// ErrAsyncPublishTimeout is returned when waiting for ack on async publish
ErrAsyncPublishTimeout JetStreamError = &jsError{message: "timeout waiting for ack"}
// ErrTooManyStalledMsgs is returned when too many outstanding async
// messages are waiting for ack.
ErrTooManyStalledMsgs JetStreamError = &jsError{message: "stalled with too many outstanding async published messages"}
// ErrFetchDisconnected is returned when the connection to the server is lost
// while waiting for messages to be delivered on PullSubscribe.
ErrFetchDisconnected = &jsError{message: "disconnected during fetch"}
)
// Error code represents JetStream error codes returned by the API
type ErrorCode uint16
const (
JSErrCodeJetStreamNotEnabledForAccount ErrorCode = 10039
JSErrCodeJetStreamNotEnabled ErrorCode = 10076
JSErrCodeInsufficientResourcesErr ErrorCode = 10023
JSErrCodeJetStreamNotAvailable ErrorCode = 10008
JSErrCodeStreamNotFound ErrorCode = 10059
JSErrCodeStreamNameInUse ErrorCode = 10058
JSErrCodeConsumerNotFound ErrorCode = 10014
JSErrCodeConsumerNameExists ErrorCode = 10013
JSErrCodeConsumerAlreadyExists ErrorCode = 10105
JSErrCodeDuplicateFilterSubjects ErrorCode = 10136
JSErrCodeOverlappingFilterSubjects ErrorCode = 10138
JSErrCodeConsumerEmptyFilter ErrorCode = 10139
JSErrCodeMessageNotFound ErrorCode = 10037
JSErrCodeBadRequest ErrorCode = 10003
JSStreamInvalidConfig ErrorCode = 10052
JSErrCodeStreamWrongLastSequence ErrorCode = 10071
// JSErrCodeStreamWrongLastSequenceConstant is returned instead of
// JSErrCodeStreamWrongLastSequence for CAS conflicts on replicated (R>1)
// streams. The two are equivalent "wrong last sequence" responses.
JSErrCodeStreamWrongLastSequenceConstant ErrorCode = 10164
)
// APIError is included in all API responses if there was an error.
type APIError struct {
Code int `json:"code"`
ErrorCode ErrorCode `json:"err_code"`
Description string `json:"description,omitempty"`
}
// Error prints the JetStream API error code and description
func (e *APIError) Error() string {
return fmt.Sprintf("nats: %s", e.Description)
}
// APIError implements the JetStreamError interface.
func (e *APIError) APIError() *APIError {
return e
}
// Is matches against an APIError.
func (e *APIError) Is(err error) bool {
if e == nil {
return false
}
// Extract internal APIError to match against.
var aerr *APIError
ok := errors.As(err, &aerr)
if !ok {
return ok
}
return e.ErrorCode == aerr.ErrorCode
}
// JetStreamError is an error result that happens when using JetStream.
// In case of client-side error, `APIError()` returns nil
type JetStreamError interface {
APIError() *APIError
error
}
type jsError struct {
apiErr *APIError
message string
}
func (err *jsError) APIError() *APIError {
return err.apiErr
}
func (err *jsError) Error() string {
if err.apiErr != nil && err.apiErr.Description != "" {
return err.apiErr.Error()
}
return fmt.Sprintf("nats: %s", err.message)
}
func (err *jsError) Unwrap() error {
// Allow matching to embedded APIError in case there is one.
if err.apiErr == nil {
return nil
}
return err.apiErr
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/jsm.go
Generated Vendored
+1850
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/kv.go
Generated Vendored
+1376
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/legacy_jetstream.md
Generated Vendored
+83
View File
@@ -0,0 +1,83 @@
# Legacy JetStream API
This is a documentation for the legacy JetStream API. A README for the current
API can be found [here](jetstream/README.md)
## JetStream Basic Usage
```go
import "github.com/nats-io/nats.go"
// Connect to NATS
nc, _ := nats.Connect(nats.DefaultURL)
// Create JetStream Context
js, _ := nc.JetStream(nats.PublishAsyncMaxPending(256))
// Simple Stream Publisher
js.Publish("ORDERS.scratch", []byte("hello"))
// Simple Async Stream Publisher
for i := 0; i < 500; i++ {
js.PublishAsync("ORDERS.scratch", []byte("hello"))
}
select {
case <-js.PublishAsyncComplete():
case <-time.After(5 * time.Second):
fmt.Println("Did not resolve in time")
}
// Simple Async Ephemeral Consumer
js.Subscribe("ORDERS.*", func(m *nats.Msg) {
fmt.Printf("Received a JetStream message: %s\n", string(m.Data))
})
// Simple Sync Durable Consumer (optional SubOpts at the end)
sub, err := js.SubscribeSync("ORDERS.*", nats.Durable("MONITOR"), nats.MaxDeliver(3))
m, err := sub.NextMsg(timeout)
// Simple Pull Consumer
sub, err := js.PullSubscribe("ORDERS.*", "MONITOR")
msgs, err := sub.Fetch(10)
// Unsubscribe
sub.Unsubscribe()
// Drain
sub.Drain()
```
## JetStream Basic Management
```go
import "github.com/nats-io/nats.go"
// Connect to NATS
nc, _ := nats.Connect(nats.DefaultURL)
// Create JetStream Context
js, _ := nc.JetStream()
// Create a Stream
js.AddStream(&nats.StreamConfig{
Name: "ORDERS",
Subjects: []string{"ORDERS.*"},
})
// Update a Stream
js.UpdateStream(&nats.StreamConfig{
Name: "ORDERS",
MaxBytes: 8,
})
// Create a Consumer
js.AddConsumer("ORDERS", &nats.ConsumerConfig{
Durable: "MONITOR",
})
// Delete Consumer
js.DeleteConsumer("ORDERS", "MONITOR")
// Delete Stream
js.DeleteStream("ORDERS")
```
modules/route-proxy/vendor/github.com/nats-io/nats.go/nats.go
Generated Vendored
+6924
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/nats_iter.go
Generated Vendored
+73
View File
@@ -0,0 +1,73 @@
// Copyright 2012-2024 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build go1.23
package nats
import (
"errors"
"iter"
"time"
)
// Msgs returns an iter.Seq2[*Msg, error] that can be used to iterate over
// messages. It can only be used with a subscription that has been created with
// SubscribeSync or QueueSubscribeSync, otherwise it will return an error on the
// first iteration.
//
// The iterator will block until a message is available. The
// subscription will not be closed when the iterator is done.
func (sub *Subscription) Msgs() iter.Seq2[*Msg, error] {
return func(yield func(*Msg, error) bool) {
for {
msg, err := sub.nextMsgNoTimeout()
if err != nil {
yield(nil, err)
return
}
if !yield(msg, nil) {
return
}
}
}
}
// MsgsTimeout returns an iter.Seq2[*Msg, error] that can be used to iterate
// over messages. It can only be used with a subscription that has been created
// with SubscribeSync or QueueSubscribeSync, otherwise it will return an error
// on the first iteration.
//
// The iterator will block until a message is available or the timeout is
// reached. If the timeout is reached, the iterator will return nats.ErrTimeout
// but it will not be closed.
func (sub *Subscription) MsgsTimeout(timeout time.Duration) iter.Seq2[*Msg, error] {
return func(yield func(*Msg, error) bool) {
for {
msg, err := sub.NextMsg(timeout)
if err != nil {
if !yield(nil, err) {
return
}
if !errors.Is(err, ErrTimeout) {
return
}
continue
}
if !yield(msg, nil) {
return
}
}
}
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/netchan.go
Generated Vendored
+117
View File
@@ -0,0 +1,117 @@
// Copyright 2013-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package nats
import (
"errors"
"reflect"
)
// This allows the functionality for network channels by binding send and receive Go chans
// to subjects and optionally queue groups.
// Data will be encoded and decoded via the EncodedConn and its associated encoders.
// BindSendChan binds a channel for send operations to NATS.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) BindSendChan(subject string, channel any) error {
chVal := reflect.ValueOf(channel)
if chVal.Kind() != reflect.Chan {
return ErrChanArg
}
go chPublish(c, chVal, subject)
return nil
}
// Publish all values that arrive on the channel until it is closed or we
// encounter an error.
func chPublish(c *EncodedConn, chVal reflect.Value, subject string) {
for {
val, ok := chVal.Recv()
if !ok {
// Channel has most likely been closed.
return
}
if e := c.Publish(subject, val.Interface()); e != nil {
// Do this under lock.
c.Conn.mu.Lock()
defer c.Conn.mu.Unlock()
if c.Conn.Opts.AsyncErrorCB != nil {
// FIXME(dlc) - Not sure this is the right thing to do.
// FIXME(ivan) - If the connection is not yet closed, try to schedule the callback
if c.Conn.isClosed() {
go c.Conn.Opts.AsyncErrorCB(c.Conn, nil, e)
} else {
c.Conn.ach.push(func() { c.Conn.Opts.AsyncErrorCB(c.Conn, nil, e) })
}
}
return
}
}
}
// BindRecvChan binds a channel for receive operations from NATS.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) BindRecvChan(subject string, channel any) (*Subscription, error) {
return c.bindRecvChan(subject, _EMPTY_, channel)
}
// BindRecvQueueChan binds a channel for queue-based receive operations from NATS.
//
// Deprecated: Encoded connections are no longer supported.
func (c *EncodedConn) BindRecvQueueChan(subject, queue string, channel any) (*Subscription, error) {
return c.bindRecvChan(subject, queue, channel)
}
// Internal function to bind receive operations for a channel.
func (c *EncodedConn) bindRecvChan(subject, queue string, channel any) (*Subscription, error) {
chVal := reflect.ValueOf(channel)
if chVal.Kind() != reflect.Chan {
return nil, ErrChanArg
}
argType := chVal.Type().Elem()
cb := func(m *Msg) {
var oPtr reflect.Value
if argType.Kind() != reflect.Ptr {
oPtr = reflect.New(argType)
} else {
oPtr = reflect.New(argType.Elem())
}
if err := c.Enc.Decode(m.Subject, m.Data, oPtr.Interface()); err != nil {
c.Conn.err = errors.New("nats: Got an error trying to unmarshal: " + err.Error())
if c.Conn.Opts.AsyncErrorCB != nil {
c.Conn.ach.push(func() { c.Conn.Opts.AsyncErrorCB(c.Conn, m.Sub, c.Conn.err) })
}
return
}
if argType.Kind() != reflect.Ptr {
oPtr = reflect.Indirect(oPtr)
}
// This is a bit hacky, but in this instance we may be trying to send to a closed channel.
// and the user does not know when it is safe to close the channel.
defer func() {
// If we have panicked, recover and close the subscription.
if r := recover(); r != nil {
m.Sub.Unsubscribe()
}
}()
// Actually do the send to the channel.
chVal.Send(oPtr)
}
return c.Conn.subscribe(subject, queue, cb, nil, nil, false, nil)
}
modules/route-proxy/vendor/github.com/nats-io/nats.go/object.go
Generated Vendored
+1432
View File
File diff suppressed because it is too large Load Diff
modules/route-proxy/vendor/github.com/nats-io/nats.go/parser.go
Generated Vendored
+555
View File
@@ -0,0 +1,555 @@
// Copyright 2012-2023 The NATS Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package nats
import (
"fmt"
)
type msgArg struct {
subject []byte
reply []byte
sid int64
hdr int
size int
}
const MAX_CONTROL_LINE_SIZE = 4096
type parseState struct {
state int
as int
drop int
hdr int
ma msgArg
argBuf []byte
msgBuf []byte
msgCopied bool
scratch [MAX_CONTROL_LINE_SIZE]byte
}
const (
OP_START = iota
OP_PLUS
OP_PLUS_O
OP_PLUS_OK
OP_MINUS
OP_MINUS_E
OP_MINUS_ER
OP_MINUS_ERR
OP_MINUS_ERR_SPC
MINUS_ERR_ARG
OP_M
OP_MS
OP_MSG
OP_MSG_SPC
MSG_ARG
MSG_PAYLOAD
MSG_END
OP_H
OP_P
OP_PI
OP_PIN
OP_PING
OP_PO
OP_PON
OP_PONG
OP_I
OP_IN
OP_INF
OP_INFO
OP_INFO_SPC
INFO_ARG
)
// parse is the fast protocol parser engine.
func (nc *Conn) parse(buf []byte) error {
var i int
var b byte
// Move to loop instead of range syntax to allow jumping of i
for i = 0; i < len(buf); i++ {
b = buf[i]
switch nc.ps.state {
case OP_START:
switch b {
case 'M', 'm':
nc.ps.state = OP_M
nc.ps.hdr = -1
nc.ps.ma.hdr = -1
case 'H', 'h':
nc.ps.state = OP_H
nc.ps.hdr = 0
nc.ps.ma.hdr = 0
case 'P', 'p':
nc.ps.state = OP_P
case '+':
nc.ps.state = OP_PLUS
case '-':
nc.ps.state = OP_MINUS
case 'I', 'i':
nc.ps.state = OP_I
default:
goto parseErr
}
case OP_H:
switch b {
case 'M', 'm':
nc.ps.state = OP_M
default:
goto parseErr
}
case OP_M:
switch b {
case 'S', 's':
nc.ps.state = OP_MS
default:
goto parseErr
}
case OP_MS:
switch b {
case 'G', 'g':
nc.ps.state = OP_MSG
default:
goto parseErr
}
case OP_MSG:
switch b {
case ' ', '\t':
nc.ps.state = OP_MSG_SPC
default:
goto parseErr
}
case OP_MSG_SPC:
switch b {
case ' ', '\t':
continue
default:
nc.ps.state = MSG_ARG
nc.ps.as = i
}
case MSG_ARG:
switch b {
case '\r':
nc.ps.drop = 1
case '\n':
var arg []byte
if nc.ps.argBuf != nil {
arg = nc.ps.argBuf
} else {
arg = buf[nc.ps.as : i-nc.ps.drop]
}
if err := nc.processMsgArgs(arg); err != nil {
return err
}
nc.ps.drop, nc.ps.as, nc.ps.state = 0, i+1, MSG_PAYLOAD
// jump ahead with the index. If this overruns
// what is left we fall out and process a split buffer.
i = nc.ps.as + nc.ps.ma.size - 1
default:
if nc.ps.argBuf != nil {
nc.ps.argBuf = append(nc.ps.argBuf, b)
}
}
case MSG_PAYLOAD:
if nc.ps.msgBuf != nil {
if len(nc.ps.msgBuf) >= nc.ps.ma.size {
nc.processMsg(nc.ps.msgBuf)
nc.ps.argBuf, nc.ps.msgBuf, nc.ps.msgCopied, nc.ps.state = nil, nil, false, MSG_END
} else {
// copy as much as we can to the buffer and skip ahead.
toCopy := nc.ps.ma.size - len(nc.ps.msgBuf)
avail := len(buf) - i
if avail < toCopy {
toCopy = avail
}
if toCopy > 0 {
start := len(nc.ps.msgBuf)
// This is needed for copy to work.
nc.ps.msgBuf = nc.ps.msgBuf[:start+toCopy]
copy(nc.ps.msgBuf[start:], buf[i:i+toCopy])
// Update our index
i = (i + toCopy) - 1
} else {
nc.ps.msgBuf = append(nc.ps.msgBuf, b)
}
}
} else if i-nc.ps.as >= nc.ps.ma.size {
nc.processMsg(buf[nc.ps.as:i])
nc.ps.argBuf, nc.ps.msgBuf, nc.ps.msgCopied, nc.ps.state = nil, nil, false, MSG_END
}
case MSG_END:
switch b {
case '\n':
nc.ps.drop, nc.ps.as, nc.ps.state = 0, i+1, OP_START
default:
continue
}
case OP_PLUS:
switch b {
case 'O', 'o':
nc.ps.state = OP_PLUS_O
default:
goto parseErr
}
case OP_PLUS_O:
switch b {
case 'K', 'k':
nc.ps.state = OP_PLUS_OK
default:
goto parseErr
}
case OP_PLUS_OK:
switch b {
case '\n':
nc.processOK()
nc.ps.drop, nc.ps.state = 0, OP_START
}
case OP_MINUS:
switch b {
case 'E', 'e':
nc.ps.state = OP_MINUS_E
default:
goto parseErr
}
case OP_MINUS_E:
switch b {
case 'R', 'r':
nc.ps.state = OP_MINUS_ER
default:
goto parseErr
}
case OP_MINUS_ER:
switch b {
case 'R', 'r':
nc.ps.state = OP_MINUS_ERR
default:
goto parseErr
}
case OP_MINUS_ERR:
switch b {
case ' ', '\t':
nc.ps.state = OP_MINUS_ERR_SPC
default:
goto parseErr
}
case OP_MINUS_ERR_SPC:
switch b {
case ' ', '\t':
continue
default:
nc.ps.state = MINUS_ERR_ARG
nc.ps.as = i
}
case MINUS_ERR_ARG:
switch b {
case '\r':
nc.ps.drop = 1
case '\n':
var arg []byte
if nc.ps.argBuf != nil {
arg = nc.ps.argBuf
nc.ps.argBuf = nil
} else {
arg = buf[nc.ps.as : i-nc.ps.drop]
}
nc.processErr(string(arg))
nc.ps.drop, nc.ps.as, nc.ps.state = 0, i+1, OP_START
default:
if nc.ps.argBuf != nil {
nc.ps.argBuf = append(nc.ps.argBuf, b)
}
}
case OP_P:
switch b {
case 'I', 'i':
nc.ps.state = OP_PI
case 'O', 'o':
nc.ps.state = OP_PO
default:
goto parseErr
}
case OP_PO:
switch b {
case 'N', 'n':
nc.ps.state = OP_PON
default:
goto parseErr
}
case OP_PON:
switch b {
case 'G', 'g':
nc.ps.state = OP_PONG
default:
goto parseErr
}
case OP_PONG:
switch b {
case '\n':
nc.processPong()
nc.ps.drop, nc.ps.state = 0, OP_START
}
case OP_PI:
switch b {
case 'N', 'n':
nc.ps.state = OP_PIN
default:
goto parseErr
}
case OP_PIN:
switch b {
case 'G', 'g':
nc.ps.state = OP_PING
default:
goto parseErr
}
case OP_PING:
switch b {
case '\n':
nc.processPing()
nc.ps.drop, nc.ps.state = 0, OP_START
}
case OP_I:
switch b {
case 'N', 'n':
nc.ps.state = OP_IN
default:
goto parseErr
}
case OP_IN:
switch b {
case 'F', 'f':
nc.ps.state = OP_INF
default:
goto parseErr
}
case OP_INF:
switch b {
case 'O', 'o':
nc.ps.state = OP_INFO
default:
goto parseErr
}
case OP_INFO:
switch b {
case ' ', '\t':
nc.ps.state = OP_INFO_SPC
default:
goto parseErr
}
case OP_INFO_SPC:
switch b {
case ' ', '\t':
continue
default:
nc.ps.state = INFO_ARG
nc.ps.as = i
}
case INFO_ARG:
switch b {
case '\r':
nc.ps.drop = 1
case '\n':
var arg []byte
if nc.ps.argBuf != nil {
arg = nc.ps.argBuf
nc.ps.argBuf = nil
} else {
arg = buf[nc.ps.as : i-nc.ps.drop]
}
nc.processAsyncInfo(arg)
nc.ps.drop, nc.ps.as, nc.ps.state = 0, i+1, OP_START
default:
if nc.ps.argBuf != nil {
nc.ps.argBuf = append(nc.ps.argBuf, b)
}
}
default:
goto parseErr
}
}
// Check for split buffer scenarios
if (nc.ps.state == MSG_ARG || nc.ps.state == MINUS_ERR_ARG || nc.ps.state == INFO_ARG) && nc.ps.argBuf == nil {
nc.ps.argBuf = nc.ps.scratch[:0]
nc.ps.argBuf = append(nc.ps.argBuf, buf[nc.ps.as:i-nc.ps.drop]...)
// FIXME, check max len
}
// Check for split msg
if nc.ps.state == MSG_PAYLOAD && nc.ps.msgBuf == nil {
// We need to clone the msgArg if it is still referencing the
// read buffer and we are not able to process the msg.
if nc.ps.argBuf == nil {
nc.cloneMsgArg()
}
// If we will overflow the scratch buffer, just create a
// new buffer to hold the split message.
if nc.ps.ma.size > cap(nc.ps.scratch)-len(nc.ps.argBuf) {
lrem := len(buf[nc.ps.as:])
nc.ps.msgBuf = make([]byte, lrem, nc.ps.ma.size)
copy(nc.ps.msgBuf, buf[nc.ps.as:])
nc.ps.msgCopied = true
} else {
nc.ps.msgBuf = nc.ps.scratch[len(nc.ps.argBuf):len(nc.ps.argBuf)]
nc.ps.msgBuf = append(nc.ps.msgBuf, (buf[nc.ps.as:])...)
}
}
return nil
parseErr:
return fmt.Errorf("nats: Parse Error [%d]: '%s'", nc.ps.state, buf[i:])
}
// cloneMsgArg is used when the split buffer scenario has the pubArg in the existing read buffer, but
// we need to hold onto it into the next read.
func (nc *Conn) cloneMsgArg() {
nc.ps.argBuf = nc.ps.scratch[:0]
nc.ps.argBuf = append(nc.ps.argBuf, nc.ps.ma.subject...)
nc.ps.argBuf = append(nc.ps.argBuf, nc.ps.ma.reply...)
nc.ps.ma.subject = nc.ps.argBuf[:len(nc.ps.ma.subject)]
if nc.ps.ma.reply != nil {
nc.ps.ma.reply = nc.ps.argBuf[len(nc.ps.ma.subject):]
}
}
const argsLenMax = 4
const hdrArgsLenMax = 5
func (nc *Conn) processMsgArgs(arg []byte) error {
// Use separate function for header based messages.
if nc.ps.hdr >= 0 {
return nc.processHeaderMsgArgs(arg)
}
// Unroll splitArgs to avoid runtime/heap issues
a := [argsLenMax][]byte{}
args := a[:0]
start := -1
for i, b := range arg {
switch b {
case ' ', '\t', '\r', '\n':
if start >= 0 {
args = append(args, arg[start:i])
start = -1
}
default:
if start < 0 {
start = i
}
}
}
if start >= 0 {
args = append(args, arg[start:])
}
switch len(args) {
case 3:
nc.ps.ma.subject = args[0]
nc.ps.ma.sid = parseInt64(args[1])
nc.ps.ma.reply = nil
nc.ps.ma.size = int(parseInt64(args[2]))
case 4:
nc.ps.ma.subject = args[0]
nc.ps.ma.sid = parseInt64(args[1])
nc.ps.ma.reply = args[2]
nc.ps.ma.size = int(parseInt64(args[3]))
default:
return fmt.Errorf("nats: processMsgArgs Parse Error: '%s'", arg)
}
if nc.ps.ma.sid < 0 {
return fmt.Errorf("nats: processMsgArgs Bad or Missing Sid: '%s'", arg)
}
if nc.ps.ma.size < 0 {
return fmt.Errorf("nats: processMsgArgs Bad or Missing Size: '%s'", arg)
}
return nil
}
// processHeaderMsgArgs is for a header based message.
func (nc *Conn) processHeaderMsgArgs(arg []byte) error {
// Unroll splitArgs to avoid runtime/heap issues
a := [hdrArgsLenMax][]byte{}
args := a[:0]
start := -1
for i, b := range arg {
switch b {
case ' ', '\t', '\r', '\n':
if start >= 0 {
args = append(args, arg[start:i])
start = -1
}
default:
if start < 0 {
start = i
}
}
}
if start >= 0 {
args = append(args, arg[start:])
}
switch len(args) {
case 4:
nc.ps.ma.subject = args[0]
nc.ps.ma.sid = parseInt64(args[1])
nc.ps.ma.reply = nil
nc.ps.ma.hdr = int(parseInt64(args[2]))
nc.ps.ma.size = int(parseInt64(args[3]))
case 5:
nc.ps.ma.subject = args[0]
nc.ps.ma.sid = parseInt64(args[1])
nc.ps.ma.reply = args[2]
nc.ps.ma.hdr = int(parseInt64(args[3]))
nc.ps.ma.size = int(parseInt64(args[4]))
default:
return fmt.Errorf("nats: processHeaderMsgArgs Parse Error: '%s'", arg)
}
if nc.ps.ma.sid < 0 {
return fmt.Errorf("nats: processHeaderMsgArgs Bad or Missing Sid: '%s'", arg)
}
if nc.ps.ma.hdr < 0 || nc.ps.ma.hdr > nc.ps.ma.size {
return fmt.Errorf("nats: processHeaderMsgArgs Bad or Missing Header Size: '%s'", arg)
}
if nc.ps.ma.size < 0 {
return fmt.Errorf("nats: processHeaderMsgArgs Bad or Missing Size: '%s'", arg)
}
return nil
}
// ASCII numbers 0-9
const (
ascii_0 = 48
ascii_9 = 57
)
// parseInt64 expects decimal positive numbers. We
// return -1 to signal error
func parseInt64(d []byte) (n int64) {
if len(d) == 0 {
return -1
}
for _, dec := range d {
if dec < ascii_0 || dec > ascii_9 {
return -1
}
n = n*10 + (int64(dec) - ascii_0)
}
return n
}

Some files were not shown because too many files have changed in this diff Show More