Compare commits

...
Author SHA1 Message Date
mesh-tools 4c9be8efc8 Merge pull request 'The web front's route helper reads the seat's plans, so a route-only module's route is provided (issue 774)' (#486) from fix/route-proxy-reads-plans into main 2026-10-30 23:43:15 +00:00
jschoubben 62c05c681c Start route-proxy's provider helper only once Serve holds the runtime's channel, so its first watch is never refused and the helper never stops until a restart (issue 774, review)
mesh/delivery merged: source.main moved to novox/mesh-controller@4c9be8efc83a
mesh/check checking 62c05c681cfb on 4c9be8efc83a
2026-10-31 00:32:26 +01:00
jschoubben bfb728730f Answer route-proxy's routes from the seat mesh-provisioner's plans and keys, not the controller's assignments, so a route the seat keys for a module not handed is marked provided and its new builds are no longer withheld (issue 774) 2026-10-31 00:27:11 +01:00
mesh-tools 159db37618 Merge pull request 'The web front keeps what it serves, its certificates and what it refuses on the bus, with tools to read them (issue 774)' (#485) from feat/route-proxy-tools into main 2026-10-30 22:11:52 +00:00
jschoubben b7c3868980 Keep a scan out of route-proxy's unrouted state: only names under a served domain, a few a minute, no visitor address (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@159db37618c2
mesh/check being carried to b7c3868980f3 on 159db37618c2 | from a4426190b0c7
Review of ddae6729: a scanner's random Host or SNI names each became a key, bounded only by the
bucket's lifetime and size, and every failed write was retried at each change and logged each time.

A refused name is now kept by name only under the registrable domain of a name the proxy serves, and
at most five new names a minute; every other refusal is counted in memory and written once a minute
under <node>._other (last minute, since start), which route_proxy_unrouted shows. A refused write of
that state is dropped and logged once until a write succeeds; trimming memory to 4096 names moved
from the request to the writer. The visitor's address is no longer kept: the journal has it.

Quick: drop unrouted keys from strangers by a fixed allow-list. Thorough: bound by the proxy's own
domains, by rate, and by never retrying. Chose thorough: the scan is constant on a public machine.
2026-10-30 23:03:00 +01:00
jschoubben ddae6729c1 Give route-proxy its state on the bus and three tools, so a site's route, certificate and last answer are read from the mesh instead of with curl (issue 774)
The web front kept nothing on the bus: whether grafana or drive was routed, where to, with which
certificate, and what it last answered could only be learned with curl or the container's log.

The proxy now keeps `routes` and `unrouted` (announced readable, keyed <node>.<name>), written on
change by one writer: route set changes, certificates stored by autocert's cache or failing at a
handshake, upstream answers that differ from the last written (or are a minute old), and names
refused in a request or a handshake. The provider helper's bundle serves route_proxy_routes,
route_proxy_route and route_proxy_unrouted from those states.

Quick: log the routes at start and grep the journal. Thorough: the state on the bus and the tools
that read it, mesh-wide, in one call. Chose thorough: the operator's question is mesh-wide and
recurs, and the log says nothing about certificates or answers.

Tests: two table tests added (the ledger, the tools), both failing without the change; four
authority tests and four skipped-route tests merged into two, so the module has four fewer.
2026-10-30 22:58:04 +01:00
jschoubben a4426190b0 Merge pull request 'Accept the credential form none in a provisioner's plan, so design 47's release delivers (issue 774)' (#483) from fix/774-none-form into main
Reviewed-on: #483
2026-10-30 11:23:41 +00:00
jschoubben 2154ea8e3c Accept the credential form none in a provisioner's plan, so the release's catalogue, which states it on every offer without a credential, no longer has every plan refused and every module held (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@a4426190b0c7
mesh/check checking 2154ea8e3c4e on a4426190b0c7
2026-10-30 12:18:36 +01:00
mesh-tools 812c528479 Merge pull request 'Pin the release's mesh-sdk in the controller, so it reads design 47's catalog until its lease is handed away (issue 774)' (#482) from feat/774-l3-reland into main 2026-10-30 11:01:57 +00:00
jschoubben 8b32e04e2f Judge the controller's module check against the catalogue it is released with, captured at the group's catalogue head f3a57dcc, so a group that brings a word the trunk's catalogue lacks does not fail on the trunk (issue 774)
mesh/delivery merged: source.main moved to novox/mesh-controller@812c528479e1
mesh/delivery-group group feat/774-l3-reland merged: every member is merged
mesh/check checking 8b32e04e2f8b on 6fe9efa71219 | novox/mesh-controller#482 in group feat/774-l3-reland
The release's fourth check failed TestModuleCheckPassesTheCatalogue with 27 problems. The checker clones the catalogue
beside a controller's tests at its trunk, even in a group holding a catalogue pull request (mesh-delivery checker
internal/check/beside.go besideFor, ADR 0484 clause 4). The release's sdk refuses an offer stating no credential form,
and the trunk's catalogue states none until the release's catalogue merges.

- Quick fix (not chosen): recapture alone, which the test never reads while MESH_CHECK_BESIDE is set.
- Chosen: the test reads the captured copy, now at the group's catalogue f3a57dcc, through beside.CapturedDir. Every other
  beside test still reads the clone. The check itself is unchanged: every rule, every manifest. Moving CAPTURED to the
  catalogue a change is released with is part of that change.

Run as the checker runs it (catalogue at main, mesh-host at main, mesh-sdk at the pin, mesh-controller's trunk
beside): the whole unit and integration suite passes, 1706 tests, 8 skipped. The test also passes with the catalogue at
f3a57dcc beside, and with nothing beside.
2026-10-30 10:38:29 +01:00
jschoubben 79f44386d2 Merge main (delivery 1, the parse-only sdk) into L3's re-land, and pin mesh-sdk 2da0de7, the release's final head (issue 774)
mesh/delivery-group group feat/774-l3-reland rejected: composed together: the checker's verdict at the members' heads (8bad7efb3e7a): unit: github.com/novox/me…
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: unit: github.com/novox/mesh-controller/cmd/mesh-controller.TestModuleCheckPassesTheCatalogue: the catalogue does not pass its own c…
The conflicts were go.mod, go.sum and testdata/beside/CAPTURED alone, taken from the release. 2da0de7 is 6e4a301's code;
its conformance events are unchanged.
2026-10-30 01:13:41 +01:00
jschoubben 80df1b6361 Pin mesh-sdk 6e4a301, the release's final sdk (issue 774)
The sdk's conformance events are unchanged at 6e4a301 and captured so. The whole integration suite passes at this head
against a local store and bus: 1706 tests, 8 skipped for a registry, a container, a refusing bus or a lab's run.
2026-10-30 00:20:10 +01:00
jschoubben 816cf96459 Pin mesh-sdk 007b6fd, the release's final sdk, so the controller reads the release's manifests and composes as the group does (issue 774)
The sdk now refuses an offer that says no credential form (6b8c694, ADR 0570). So:
- the catalogue copy in testdata/beside is captured again at the release branch's catalogue 8ae8e88d, whose offers each
  say one, and the sdk's conformance events at 007b6fd, as CAPTURED asks;
- the fixtures of identity_bound_test and issued_test say the form their rows meant: pair for a login of each
  consumer's own, none for a keyless provision. No behaviour of the controller changed.
2026-10-30 00:07:04 +01:00
jschoubben 3f05df5d1a Pin mesh-sdk 2e5cc4f, so the controller reads the release's new manifest words (issue 774)
The controller on dbcaa29 parses manifests strictly and refuses `bus` on an artifact and `max-age-days` on a log, both
in the re-land group's catalogue. The SDK's conformance events are captured again at the pinned commit, as
testdata/beside/CAPTURED asks; they are unchanged.
2026-10-29 22:19:39 +01:00
83 changed files with 2624 additions and 332 deletions
+8 -2
View File
@@ -56,9 +56,15 @@ func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
} }
} }
// The real catalogue passes the command, the way it passes the test that used to be the only check. // The real catalogue passes the command, the way it passes the test that used to be the only check: the catalogue this
// controller is released with, captured in testdata/beside at the commit CAPTURED names. Not the clone a merge check puts
// beside, which is the catalogue's trunk even in a delivery group holding a catalogue pull request (ADR 0484 clause 4):
// a controller whose check reads a word that group's catalogue brings would fail on the trunk's catalogue before the
// group merges both (issue 774: the release's sdk refuses an offer that states no credential form, and the catalogue's
// trunk states none until the release's catalogue merges). Moving CAPTURED to the catalogue a change is released with is
// part of that change.
func TestModuleCheckPassesTheCatalogue(t *testing.T) { func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := beside.Catalogue(t) root := filepath.Join(beside.CapturedDir(t, "mesh-catalog"), "modules")
paths, err := manifestsUnder(root) paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 { if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err) t.Fatalf("no manifests under %s: %v", root, err)
+2 -2
View File
@@ -23,10 +23,10 @@ func TestModuleCheckRefusesAnIdentityThatOverflowsWhatItRequires(t *testing.T) {
return p return p
} }
objects := write("objects", `{"module":"objects","version":"1", objects := write("objects", `{"module":"objects","version":"1",
"provides":[{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"}}], "provides":[{"name":"s3-bucket","scope":"mesh","identity":{"max":20,"in":"an S3 access key"},"credential":{"form":"pair"}}],
"receives":{"s3-bucket":"/var/lib/mesh/objects/mesh.json"}}`) "receives":{"s3-bucket":"/var/lib/mesh/objects/mesh.json"}}`)
resolver := write("resolver", `{"module":"resolver","version":"1", resolver := write("resolver", `{"module":"resolver","version":"1",
"provides":[{"name":"wildcard-resolution","scope":"mesh","identity":false}]}`) "provides":[{"name":"wildcard-resolution","scope":"mesh","identity":false,"credential":{"form":"none"}}]}`)
album := write("photoalbum", `{"module":"photoalbum","version":"1","requires":["s3-bucket"]}`) album := write("photoalbum", `{"module":"photoalbum","version":"1","requires":["s3-bucket"]}`)
nm := write("networkmanager", `{"module":"networkmanager","version":"1","requires":["wildcard-resolution"]}`) nm := write("networkmanager", `{"module":"networkmanager","version":"1","requires":["wildcard-resolution"]}`)
+7 -2
View File
@@ -182,6 +182,9 @@ func TestANeedIsIssuedToItsHolderOnlyWhereTheProviderIsMovedAndItsSoftwareIssues
offer := catalogue.Offer{Name: "gitea-read", Scope: catalogue.ScopeMesh} offer := catalogue.Offer{Name: "gitea-read", Scope: catalogue.ScopeMesh}
if issued { if issued {
offer.Credential = &catalogue.OfferCredential{IssuedBy: catalogue.IssuedBySoftware, Tool: "gitea_agent_token"} offer.Credential = &catalogue.OfferCredential{IssuedBy: catalogue.IssuedBySoftware, Tool: "gitea_agent_token"}
} else {
// A login of each consumer's own, said: an offer naming no form has no credential (mesh-sdk 6b8c694).
offer.Credential = &catalogue.OfferCredential{Form: catalogue.CredentialFormPair}
} }
return catalogue.Manifest{Module: "gitea", Version: "1", Moved: moved, Provides: []catalogue.Offer{offer}} return catalogue.Manifest{Module: "gitea", Version: "1", Moved: moved, Provides: []catalogue.Offer{offer}}
} }
@@ -216,8 +219,10 @@ func TestANeedIsIssuedToItsHolderOnlyWhereTheProviderIsMovedAndItsSoftwareIssues
// marked moved; another module is not. // marked moved; another module is not.
func TestTheModuleHoldingTheVaultSeatCountsAsMovedForItsConsumers(t *testing.T) { func TestTheModuleHoldingTheVaultSeatCountsAsMovedForItsConsumers(t *testing.T) {
vault := catalogue.Manifest{Module: "mesh-vault", Version: "1", Claims: []catalogue.Claim{{Name: "mesh-vault", Scope: catalogue.ScopeMesh}}, vault := catalogue.Manifest{Module: "mesh-vault", Version: "1", Claims: []catalogue.Claim{{Name: "mesh-vault", Scope: catalogue.ScopeMesh}},
Provides: []catalogue.Offer{{Name: "secret", Scope: catalogue.ScopeMesh}}} Provides: []catalogue.Offer{{Name: "secret", Scope: catalogue.ScopeMesh,
other := catalogue.Manifest{Module: "mesh-vault", Version: "1", Provides: []catalogue.Offer{{Name: "secret", Scope: catalogue.ScopeMesh}}} Credential: &catalogue.OfferCredential{Form: catalogue.CredentialFormPair}}}}
other := catalogue.Manifest{Module: "mesh-vault", Version: "1", Provides: []catalogue.Offer{{Name: "secret", Scope: catalogue.ScopeMesh,
Credential: &catalogue.OfferCredential{Form: catalogue.CredentialFormPair}}}}
if got := needOf(t, map[string]catalogue.Manifest{"mesh-vault": vault}, "mesh-vault", "gitea", "secret"); !got.IssuedToHolder || !got.ProviderMoved { if got := needOf(t, map[string]catalogue.Manifest{"mesh-vault": vault}, "mesh-vault", "gitea", "secret"); !got.IssuedToHolder || !got.ProviderMoved {
t.Errorf("the vault's consumer is not answered by a moved provider: %+v", got) t.Errorf("the vault's consumer is not answered by a moved provider: %+v", got)
} }
+1 -1
View File
@@ -3,7 +3,7 @@ module github.com/novox/mesh-controller
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261029233535-78c9ec881771 git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0
github.com/jackc/pgx/v5 v5.10.0 github.com/jackc/pgx/v5 v5.10.0
github.com/nats-io/nats-server/v2 v2.11.17 github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
+2 -2
View File
@@ -1,7 +1,7 @@
git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5 h1:E5stfIeCN6uYgA9TCv+4K8JgaURg3zQppvUgJLaDMP4= git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5 h1:E5stfIeCN6uYgA9TCv+4K8JgaURg3zQppvUgJLaDMP4=
git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5/go.mod h1:4wyoCr6IIIIarEUuROThWDj7DAvJvIidqMjWcpSnOwE= git.novox.be/novox/mesh-host v0.0.0-20261018171808-9d9e3a46c4f5/go.mod h1:4wyoCr6IIIIarEUuROThWDj7DAvJvIidqMjWcpSnOwE=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261029233535-78c9ec881771 h1:Tt16HJk8JZRce1i3JzzDSNdCWOrv7dw8FQ/bckNQ5Ig= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0 h1:lKU3HjNXGiVbLW2hUgFjU/TMTUYMlNnnmZxG5T7biQA=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261029233535-78c9ec881771/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030001116-2da0de75c3f0/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+11 -2
View File
@@ -53,6 +53,16 @@ func Dir(t testing.TB, repository string) string {
t.Logf("judged against %s as cloned beside this check", dir) t.Logf("judged against %s as cloned beside this check", dir)
return dir return dir
} }
t.Logf("no %s beside this check: a merge check judges %s against its own clone", Env, repository)
return CapturedDir(t, repository)
}
// CapturedDir is the copy of the named repository captured in testdata/beside, laid out as Dir lays it out, whatever
// MESH_CHECK_BESIDE says: for a test that judges this repository against the commit it was released with, which a
// merge check does not clone (it clones the catalogue at its trunk even in a delivery group holding a catalogue pull
// request, ADR 0484 clause 4).
func CapturedDir(t testing.TB, repository string) string {
t.Helper()
from := filepath.Join(Captured(), repository) from := filepath.Join(Captured(), repository)
if _, err := os.Stat(from); err != nil { if _, err := os.Stat(from); err != nil {
t.Fatalf("no captured copy of %s at %s: %v", repository, from, err) t.Fatalf("no captured copy of %s at %s: %v", repository, from, err)
@@ -81,8 +91,7 @@ func Dir(t testing.TB, repository string) string {
if err != nil { if err != nil {
t.Fatalf("the captured copy of %s could not be laid out: %v", repository, err) t.Fatalf("the captured copy of %s could not be laid out: %v", repository, err)
} }
t.Logf("judged against the copy of %s captured in testdata/beside (see CAPTURED); a merge check "+ t.Logf("judged against the copy of %s captured in testdata/beside (see CAPTURED)", repository)
"judges it against the repository's own clone", repository)
return dir return dir
} }
+3 -2
View File
@@ -127,9 +127,10 @@ var (
// begin. // begin.
var PlacementKinds = []string{"dir", "access", "port", "seat", "machine", "backup"} var PlacementKinds = []string{"dir", "access", "port", "seat", "machine", "backup"}
// credentialForms are the forms a provider's offer may declare for its credential (issue 768). // credentialForms are the forms a provider's offer may declare for its credential (issue 768); "none" is an offer with
// no credential, which the release's catalogue states on every such offer (issue 774).
var credentialForms = []string{catalogue.CredentialFormShared, catalogue.CredentialFormPair, var credentialForms = []string{catalogue.CredentialFormShared, catalogue.CredentialFormPair,
catalogue.CredentialFormIssued, catalogue.CredentialFormGiven} catalogue.CredentialFormIssued, catalogue.CredentialFormGiven, catalogue.CredentialFormNone}
var ( var (
digestShape = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`) digestShape = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
+4 -2
View File
@@ -316,14 +316,16 @@ func TestAPlanNamesTheCredentialFormOfEachProvidersOffer(t *testing.T) {
p := aPlan() p := aPlan()
p.Provisions = []ProvisionedCredential{ p.Provisions = []ProvisionedCredential{
{Provision: "postgres-database", Provider: "novox", ProviderModule: "postgres", CredentialForm: "pair"}, {Provision: "postgres-database", Provider: "novox", ProviderModule: "postgres", CredentialForm: "pair"},
{Provision: "sonarr-api", Provider: "ace", ProviderModule: "sonarr", CredentialForm: "shared"}} {Provision: "sonarr-api", Provider: "ace", ProviderModule: "sonarr", CredentialForm: "shared"},
{Provision: "artifact-store", Provider: "novox", ProviderModule: "distribution", CredentialForm: "none"}}
values, err := AssignmentValues(assigned, map[string]Provisioning{"g14.dunst": p}) values, err := AssignmentValues(assigned, map[string]Provisioning{"g14.dunst": p})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if want := `{"provision":"postgres-database","provider":"novox","provider-module":"postgres",` + if want := `{"provision":"postgres-database","provider":"novox","provider-module":"postgres",` +
`"credential-form":"pair"},{"provision":"sonarr-api","provider":"ace","provider-module":"sonarr",` + `"credential-form":"pair"},{"provision":"sonarr-api","provider":"ace","provider-module":"sonarr",` +
`"credential-form":"shared"}`; !strings.Contains(string(values["g14.dunst"]), want) { `"credential-form":"shared"},{"provision":"artifact-store","provider":"novox","provider-module":"distribution",` +
`"credential-form":"none"}`; !strings.Contains(string(values["g14.dunst"]), want) {
t.Errorf("the plan's value is %s, want it to carry %s", values["g14.dunst"], want) t.Errorf("the plan's value is %s, want it to carry %s", values["g14.dunst"], want)
} }
bad := aPlan() bad := aPlan()
+14
View File
@@ -90,3 +90,17 @@ A small Go program beside the proxy (`cmd/route-provider`) runs the SDK's provid
and the offer says `"answered-by": "helper"`: a handed consumer (gitea's origins file) is filled by its machine's client and the offer says `"answered-by": "helper"`: a handed consumer (gitea's origins file) is filled by its machine's client
from this module's entry, which carries the public names the controller composed for it. The module is **not** marked from this module's entry, which carries the public names the controller composed for it. The module is **not** marked
moved: its own files, `acme.env` and the broker secret, are composed as before, and the proxy itself is unchanged. moved: its own files, `acme.env` and the broker secret, are composed as before, and the proxy itself is unchanged.
## What it says on the bus, and its tools (issue 774)
The proxy keeps two states, announced readable (`conn.Keeps`, novox/hq ADR 0495), each keyed `<node>.<name>`:
| state | one key per | holds | written when |
|---|---|---|---|
| `routes` | served name | the rules it is served by (path, module, node, target, refusal, redirect, authentication); its certificate (authority, issuer, not-after, last renewal, outcome, or why it failed); the last upstream answer (status, time, error, path) | a route arrives, changes or leaves; an authority stores a certificate; a handshake fails to find one or hands out one not yet recorded; an upstream answer differs from the last one written, or the same one is a minute old |
| `unrouted` | name answered "no route for", in a request or a TLS handshake, under a domain the proxy serves | when it was last asked, how often since the start, whether it is an internal-only name asked from outside; never who asked (the journal keeps that, for fail2ban) | first asked, at most five new names a minute, then at most once a minute; deleted when the name is served; kept a week. Every other refusal is only counted, in one key per proxy (`<node>._other`: the last minute's count and the total), written once a minute. A refused write is dropped, never retried |
Nothing is written on a timer. The provider helper's binary serves the module's tools from those states, one call each:
`route_proxy_routes` (every served name with its target, certificate expiry and last answer), `route_proxy_route`
(one name in full, or that it is not served and when it was last asked for) and `route_proxy_unrouted` (the names
refused, newest first).
+11 -3
View File
@@ -1,6 +1,6 @@
// route-provider: route-proxy's Go half, one binary the node's runtime launches and speaks MCP to over stdio // route-provider: route-proxy's Go half, one binary the node's runtime launches and speaks MCP to over stdio
// through the Go SDK. It serves no tools; it runs the SDK's provider helper that answers the consumers of `route` // through the Go SDK. It runs the SDK's provider helper that answers the consumers of `route` (issue 792), and serves
// (issue 792). The TypeScript adapter beside it keeps writing the proxy's files. // the module's tools, which read what the proxy keeps on the bus (issue 774, tools.go).
// //
// stdout is the MCP channel, so everything this module says, it says on stderr. // stdout is the MCP channel, so everything this module says, it says on stderr.
package main package main
@@ -9,9 +9,12 @@ import (
"context" "context"
"fmt" "fmt"
"os" "os"
"time"
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/provider" "git.novox.be/novox/mesh-sdk/go/provider"
"route-proxy/internal/routestate"
) )
func say(format string, args ...any) { func say(format string, args ...any) {
@@ -25,12 +28,17 @@ func main() {
} else { } else {
p := newProvider(node, provider.StateBus(ModuleName), say) p := newProvider(node, provider.StateBus(ModuleName), say)
go func() { go func() {
// The helper's watches go through the runtime's channel, which Serve holds: one asked before it does is
// refused ("before Serve"), Run returns and the helper stays stopped until the process restarts.
<-stdio.Served()
if err := p.Run(context.Background()); err != nil { if err := p.Run(context.Background()); err != nil {
say("the provider helper stopped: %v", err) say("the provider helper stopped: %v", err)
} }
}() }()
} }
if err := stdio.Serve("", nil); err != nil { tools := front{routes: stdio.State(routestate.RoutesState), unrouted: stdio.State(routestate.UnroutedState),
now: time.Now}.tools()
if err := stdio.Serve("", tools); err != nil {
say("%v", err) say("%v", err)
os.Exit(1) os.Exit(1)
} }
@@ -9,7 +9,9 @@ import (
"sync" "sync"
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/provider" "git.novox.be/novox/mesh-sdk/go/provider"
"git.novox.be/novox/mesh-sdk/go/provisions"
) )
// bucketBus is the helper's bus over in-memory buckets. // bucketBus is the helper's bus over in-memory buckets.
@@ -44,33 +46,60 @@ func (b *bucketBus) Delete(_ context.Context, bucket, key string) error {
return nil return nil
} }
// A consumer of a route is answered with the names its plan's entry carries, and nothing of a credential (issue 792): // A consumer of a route is answered with the names its plan's entry carries, and nothing of a credential (issue 792),
// the entry's values are the controller's composition, copied unchanged, and no vault is asked. // from the seat mesh-provisioner's plans and keys (novox/hq ADR 0570 clause 2), never the controller's assignments: a
// module not handed whose route the seat keys (grafana, issue 774) has a plan of provisions alone, and a helper that
// read only the assignments left its key asked for good, so the assigner withheld every new build of 38 such modules.
func TestAConsumerOfARouteIsToldItsNamesFromItsPlanEntry(t *testing.T) { func TestAConsumerOfARouteIsToldItsNamesFromItsPlanEntry(t *testing.T) {
bus := &bucketBus{buckets: map[string]map[string][]byte{}} bus := &bucketBus{buckets: map[string]map[string][]byte{}}
plan, _ := json.Marshal(map[string]any{"node": "g14", "module": "gitea", "provisioning": map[string]any{ for module, names := range map[string]map[string]string{
"provisions": []any{map[string]any{"provision": ProvisionName, "provider": "novox", "gitea": {"name": "git.example.org", "internal-name": "git.novox.internal", "name-web": "git.example.org"},
"provider-module": ModuleName, "at": "10.7.0.1", "from": "novox", "as": "mesh_g14_gitea", "grafana": {"name": "grafana.example.org"},
"names": map[string]any{"name": "git.example.org", "internal-name": "git.novox.internal", } {
"name-web": "git.example.org"}}}}}) plan, err := provisions.EncodePlan(provisions.Plan{Node: "g14", Module: module, Plan: catalogue.Plan{
if err := bus.Put(context.Background(), provider.AssignmentsBucket, "g14.gitea", plan); err != nil { Provisions: []catalogue.PlanProvision{{Provision: ProvisionName, Provider: "novox", ProviderModule: ModuleName,
At: "10.7.0.1", From: "novox", As: "mesh_g14_" + module, Names: names}}}})
if err != nil {
t.Fatal(err) t.Fatal(err)
} }
key := "g14." + module
if err := bus.Put(context.Background(), provisions.PlansBucket, key, plan); err != nil {
t.Fatal(err)
}
asked, err := json.Marshal(provisions.Provision{V: provisions.Version, Provision: ProvisionName,
Consumer: provisions.End{Node: "g14", Module: module}, Provider: provisions.End{Node: "novox", Module: ModuleName},
State: provisions.Asked})
if err != nil {
t.Fatal(err)
}
if err := bus.Put(context.Background(), provisions.ProvisionsBucket, key+"."+ProvisionName, asked); err != nil {
t.Fatal(err)
}
}
p := newProvider("novox", bus, t.Logf) p := newProvider("novox", bus, t.Logf)
out, err := p.Pass(context.Background()) out, err := p.Pass(context.Background())
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if len(out) != 1 || out[0].Status != provider.Answered { if len(out) != 2 || out[0].Status != provider.Answered || out[1].Status != provider.Answered {
t.Fatalf("the pass did %+v", out) t.Fatalf("the pass did %+v", out)
} }
for key, want := range map[string]map[string]string{
"g14.gitea.route": {"name-web": "git.example.org", "name": "git.example.org", "internal-name": "git.novox.internal"},
"g14.grafana.route": {"name": "grafana.example.org"},
} {
var e provider.Entry var e provider.Entry
if err := json.Unmarshal(bus.buckets[provider.BucketOf(ModuleName)]["g14.gitea.route"], &e); err != nil { if err := json.Unmarshal(bus.buckets[provider.BucketOf(ModuleName)][key], &e); err != nil {
t.Fatal(err) t.Fatalf("%s: %v", key, err)
}
for k, v := range want {
if e.Values[k] != v {
t.Errorf("%s says %s %v, want %s", key, k, e.Values[k], v)
}
}
if e.State != provisions.Provided || e.VaultName != "" || e.User != "" || e.Refused != "" {
t.Errorf("the entry %s is %+v", key, e)
} }
if e.Values["name-web"] != "git.example.org" || e.Values["name"] != "git.example.org" ||
e.Values["internal-name"] != "git.novox.internal" || e.VaultName != "" || e.User != "" || e.Refused != "" {
t.Fatalf("the entry is %+v", e)
} }
} }
@@ -95,7 +124,8 @@ func TestRouteProxyIsNotMovedAndKeepsItsOwnFiles(t *testing.T) {
if len(provides) != 1 || provides[0].(map[string]any)["answered-by"] != "helper" { if len(provides) != 1 || provides[0].(map[string]any)["answered-by"] != "helper" {
t.Fatalf("route is not said to be answered by the helper: %v", provides) t.Fatalf("route is not said to be answered by the helper: %v", provides)
} }
for _, k := range []string{"build", "provides", "state", "reads"} { // Its states and its tools are what it says on the bus, not what it runs or writes (issue 774).
for _, k := range []string{"build", "provides", "state", "reads", "tools", "replaces"} {
delete(m, k) delete(m, k)
} }
pinned, err := json.Marshal(m) // sorted keys, compact pinned, err := json.Marshal(m) // sorted keys, compact
@@ -0,0 +1,296 @@
package main
// route-proxy's tools (issue 774): the web front's command line. Each answers from what the proxy keeps on the bus
// (route-proxy_routes and route-proxy_unrouted, written on change by the proxy itself), in one call, for every
// machine running a proxy: which sites are served and where they go, whether their certificates are good, what the
// last request through each got back, and which names were asked for and are served nowhere.
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate"
)
// The tools' names, as the manifest declares them.
const (
RoutesTool = "route_proxy_routes"
RouteTool = "route_proxy_route"
UnroutedTool = "route_proxy_unrouted"
)
// reader is one of the module's states, as stdio.KeptState reads it.
type reader interface {
Keys() ([]string, error)
Get(key string) (*stdio.StateEntry, error)
}
// front is what the tools read: the routes and the refusals, and the clock a certificate's days left are counted by.
type front struct {
routes, unrouted reader
now func() time.Time
}
// RouteLine is one served name in the list: where it goes, how long its certificate holds, what it last answered.
type RouteLine struct {
Name string `json:"name"`
Proxy string `json:"proxy"`
Reach string `json:"reach"`
Targets []string `json:"targets"`
// Reachable is yes when the last answer upstream gave was below 500, no when it was 500 or more or none came,
// unknown before any request.
Reachable string `json:"reachable"`
// Certificate is its outcome, issuer and expiry in words; ExpiresInDays is the whole days until not-after.
Certificate string `json:"certificate"`
ExpiresInDays *int `json:"expires_in_days,omitempty"`
LastAnswer *routestate.Answer `json:"last_answer,omitempty"`
}
func (f front) tools() []stdio.Tool {
node := map[string]any{"type": "string", "description": "only the proxy on this machine; every machine when left out"}
return []stdio.Tool{
{Name: RoutesTool,
Description: "Every site (web name, host name) the mesh's web front serves through route-proxy, the reverse proxy " +
"on the public machine: for each route the module and node it forwards to, its TLS certificate's issuer and " +
"expiry (days left), and the last answer upstream gave (status, time, error), so is X reachable and is its " +
"certificate good is answered without curl. Read from the proxy's routes state on the bus. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{"node": node}},
Run: func(args map[string]any) (any, error) {
at, _ := args["node"].(string)
return f.list(at)
}},
{Name: RouteTool,
Description: "One site the mesh's web front (route-proxy) is asked about, in full: every rule it is served by " +
"(path, module, node, target, refusal, redirect, authentication), its certificate (issuer, not-after, last " +
"renewal and its outcome, or why it failed) and the last upstream answer. A name with no route says so, and " +
"when it was last asked for. Answers is X reachable, why does this site not load. Read-only.",
Input: map[string]any{"type": "object", "required": []string{"name"}, "properties": map[string]any{
"name": map[string]any{"type": "string", "description": "the site's name, such as app.example.org"},
"node": node}},
Run: func(args map[string]any) (any, error) {
name, _ := args["name"].(string)
at, _ := args["node"].(string)
return f.one(name, at)
}},
{Name: UnroutedTool,
Description: "Names the mesh's web front (route-proxy) answered \"no route for\": a site asked for in a request or " +
"a TLS handshake that no route serves, newest first, with when it was last asked and how often. Only names under a " +
"domain the proxy serves are kept by name, a few new ones a minute; every other refusal (a scan) is counted per " +
"proxy, the last minute's count and the total since its start. A site that should be served and is missing " +
"its route shows here. Kept a week. Read-only.",
Input: map[string]any{"type": "object", "properties": map[string]any{"node": node}},
Run: func(args map[string]any) (any, error) {
at, _ := args["node"].(string)
return f.refusals(at)
}},
}
}
// read is every value of a state, of node's proxy alone when node is named, decoded into a fresh T each.
func read[T any](state reader, node string) ([]T, error) {
keys, err := state.Keys()
if err != nil {
return nil, err
}
var out []T
for _, key := range keys {
at, _ := routestate.Split(key)
if node != "" && at != node {
continue
}
entry, err := state.Get(key)
if err != nil {
return nil, err
}
if entry == nil {
continue // deleted between the list and the read
}
var v T
if err := json.Unmarshal(entry.Value, &v); err != nil {
return nil, fmt.Errorf("%s holds what is not a route-proxy record: %w", key, err)
}
out = append(out, v)
}
return out, nil
}
func (f front) list(node string) (map[string]any, error) {
routes, err := read[routestate.Route](f.routes, node)
if err != nil {
return nil, err
}
lines := make([]RouteLine, 0, len(routes))
unreachable, expiring := 0, 0
for _, r := range routes {
line := f.line(r)
if line.Reachable == "no" {
unreachable++
}
if line.ExpiresInDays != nil && *line.ExpiresInDays < 14 || r.Certificate.Outcome == routestate.Failed {
expiring++
}
lines = append(lines, line)
}
sort.Slice(lines, func(i, j int) bool {
if lines[i].Name != lines[j].Name {
return lines[i].Name < lines[j].Name
}
return lines[i].Proxy < lines[j].Proxy
})
return map[string]any{"routes": lines, "served": len(lines), "unreachable": unreachable,
"certificates_failing_or_within_14_days": expiring}, nil
}
func (f front) line(r routestate.Route) RouteLine {
line := RouteLine{Name: r.Name, Proxy: r.Proxy, Reach: r.Reach, LastAnswer: r.LastAnswer, Reachable: "unknown",
Targets: []string{}}
for _, t := range r.Targets {
line.Targets = append(line.Targets, describe(t))
}
if a := r.LastAnswer; a != nil {
line.Reachable = "yes"
if a.Error != "" || a.Status >= 500 || a.Status == 0 {
line.Reachable = "no"
}
}
c := r.Certificate
words := []string{}
if c.Outcome != "" {
words = append(words, c.Outcome)
}
if c.Issuer != "" {
words = append(words, "by "+c.Issuer)
}
if c.NotAfter != nil {
days := int(c.NotAfter.Sub(f.now()).Hours() / 24)
line.ExpiresInDays = &days
words = append(words, "until "+c.NotAfter.UTC().Format("2006-01-02"))
}
if c.Error != "" {
words = append(words, "("+c.Error+")")
}
if len(words) == 0 {
words = append(words, "none seen yet")
}
line.Certificate = c.Authority + ": " + strings.Join(words, " ")
return line
}
// describe is one rule in a few words: path → module@node at target, or what it answers with.
func describe(t routestate.Target) string {
path := t.Path
if path == "" {
path = "/"
}
var what string
switch {
case t.Deny:
what = "refused"
case t.Redirect != "":
what = "redirected to " + t.Redirect
default:
what = fmt.Sprintf("%s@%s at %s", t.Module, t.Node, t.Target)
}
if t.Sealed != "" {
what += " (authentication cannot be checked: " + t.Sealed + ")"
} else if t.Auth {
what += " (authenticated)"
}
return path + " → " + what
}
func (f front) one(name, node string) (map[string]any, error) {
name = strings.ToLower(strings.TrimSpace(name))
if name == "" {
return nil, fmt.Errorf("name is the site to read, such as app.example.org")
}
routes, err := read[routestate.Route](f.routes, node)
if err != nil {
return nil, err
}
var found []routestate.Route
for _, r := range routes {
if r.Name == name {
found = append(found, r)
}
}
if len(found) > 0 {
out := map[string]any{"name": name, "routed": true, "routes": found}
if len(found) == 1 {
out["reachable"] = f.line(found[0]).Reachable
}
return out, nil
}
asked, _, err := f.asked(node)
if err != nil {
return nil, err
}
for _, a := range asked {
if a.Name == name {
return map[string]any{"name": name, "routed": false, "asked": a,
"says": fmt.Sprintf("no proxy serves %s; the proxy on %s answered \"no route\" for it, last at %s",
name, a.Proxy, a.AskedAt.UTC().Format(time.RFC3339))}, nil
}
}
return map[string]any{"name": name, "routed": false,
"says": fmt.Sprintf("no proxy serves %s, and none was asked for it this week", name)}, nil
}
func (f front) refusals(node string) (map[string]any, error) {
asked, others, err := f.asked(node)
if err != nil {
return nil, err
}
sort.Slice(asked, func(i, j int) bool { return asked[i].AskedAt.After(asked[j].AskedAt) })
if asked == nil {
asked = []routestate.Unrouted{}
}
return map[string]any{"unrouted": asked, "names": len(asked), "other_refusals": others}, nil
}
// asked is the names refused and kept by name, and each proxy's count of the refusals not kept by name.
func (f front) asked(node string) ([]routestate.Unrouted, []routestate.Other, error) {
keys, err := f.unrouted.Keys()
if err != nil {
return nil, nil, err
}
names, others := keySet{}, keySet{}
for _, key := range keys {
if _, name := routestate.Split(key); name == routestate.OtherName {
others[key] = true
} else {
names[key] = true
}
}
asked, err := read[routestate.Unrouted](only{f.unrouted, names}, node)
if err != nil {
return nil, nil, err
}
counted, err := read[routestate.Other](only{f.unrouted, others}, node)
if counted == nil {
counted = []routestate.Other{}
}
return asked, counted, err
}
// keySet is a set of keys; only is a state narrowed to them.
type keySet map[string]bool
type only struct {
reader
keys keySet
}
func (o only) Keys() ([]string, error) {
out := make([]string, 0, len(o.keys))
for k := range o.keys {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
@@ -0,0 +1,99 @@
package main
import (
"encoding/json"
"sort"
"strings"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate"
)
// held is a state as the bus holds it, read-only.
type held map[string]any
func (h held) Keys() ([]string, error) {
out := []string{}
for k := range h {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
func (h held) Get(key string) (*stdio.StateEntry, error) {
v, ok := h[key]
if !ok {
return nil, nil
}
raw, err := json.Marshal(v)
return &stdio.StateEntry{Key: key, Value: raw}, err
}
// The web front's tools answer from the bus, in one call each (issue 774): every site with its target, certificate
// expiry and last answer; one site in full, or that it is not served and when it was asked for; and the names the
// proxy answered "no route for".
func TestTheWebFrontsToolsAnswerFromTheBus(t *testing.T) {
now := time.Date(2026, 10, 30, 21, 0, 0, 0, time.UTC)
expires := now.Add(30 * 24 * time.Hour)
routes := held{
"anchor.grafana.example": routestate.Route{Name: "grafana.example", Proxy: "anchor", Reach: routestate.ReachPublic,
Targets: []routestate.Target{{Module: "grafana", Node: "home-server", Target: "http://10.0.0.2:3000"}},
Certificate: routestate.Certificate{Authority: routestate.ReachPublic, Issuer: "Let's Encrypt R11", NotAfter: &expires, Outcome: routestate.Issued},
LastAnswer: &routestate.Answer{Status: 502, At: now, Error: "dial tcp 10.0.0.2:3000: connect: connection refused"}},
"anchor.drive.example": routestate.Route{Name: "drive.example", Proxy: "anchor", Reach: routestate.ReachPublic,
Targets: []routestate.Target{{Module: "nextcloud", Node: "home-server", Target: "http://10.0.0.2:8080"}},
Certificate: routestate.Certificate{Authority: routestate.ReachPublic},
LastAnswer: &routestate.Answer{Status: 200, At: now}},
}
unrouted := held{
"anchor.old.example": routestate.Unrouted{Name: "old.example", Proxy: "anchor", AskedAt: now.Add(-time.Hour), Times: 3},
"anchor.wiki.example": routestate.Unrouted{Name: "wiki.example", Proxy: "anchor", AskedAt: now, Times: 1},
"anchor._other": routestate.Other{Proxy: "anchor", Minute: now, LastMinute: 40, SinceStart: 900},
}
tools := map[string]stdio.Tool{}
for _, tool := range (front{routes: routes, unrouted: unrouted, now: func() time.Time { return now }}).tools() {
tools[tool.Name] = tool
}
for _, row := range []struct {
tool string
args map[string]any
want []string // what the answer says, in its JSON
}{
{RoutesTool, nil, []string{`"served":2`, `"unreachable":1`,
`"name":"grafana.example"`, `"reachable":"no"`, `"expires_in_days":30`, `/ → grafana@home-server at http://10.0.0.2:3000`,
`"name":"drive.example"`, `"reachable":"yes"`, `public: none seen yet`}},
{RoutesTool, map[string]any{"node": "home-server"}, []string{`"served":0`}},
{RouteTool, map[string]any{"name": "Grafana.Example"}, []string{`"routed":true`, `"reachable":"no"`,
`connection refused`, `"issuer":"Let's Encrypt R11"`}},
{RouteTool, map[string]any{"name": "wiki.example"}, []string{`"routed":false`,
`answered \"no route\" for it, last at 2026-10-30T21:00:00Z`}},
{RouteTool, map[string]any{"name": "nothing.example"}, []string{`"routed":false`, `none was asked for it`}},
{UnroutedTool, nil, []string{`"names":2`, `"unrouted":[{"name":"wiki.example"`,
`"other_refusals":[{"proxy":"anchor"`, `"last_minute":40`, `"since_start":900`}},
} {
t.Run(row.tool, func(t *testing.T) {
tool, ok := tools[row.tool]
if !ok {
t.Fatalf("%s is not served", row.tool)
}
if !strings.Contains(tool.Description, "route-proxy") {
t.Errorf("%s's description does not name the web front, so a search does not find it", row.tool)
}
answer, err := tool.Run(row.args)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(answer)
for _, w := range row.want {
if !strings.Contains(string(raw), w) {
t.Errorf("%s(%v) does not say %s:\n%s", row.tool, row.args, w, raw)
}
}
})
}
}
@@ -19,14 +19,15 @@ const (
// path back: nothing is re-registered, no order reaches the CA, and issuance stops with nothing // path back: nothing is re-registered, no order reaches the CA, and issuance stops with nothing
// saying why. Naming the cache after the authority means the account is only ever found where it is // saying why. Naming the cache after the authority means the account is only ever found where it is
// still valid — the new root lands in a directory with no account in it, and autocert registers. // still valid — the new root lands in a directory with no account in it, and autocert registers.
func TestANewCARootMeansANewAccountCache(t *testing.T) { func TestTheAccountCacheIsNamedAfterItsAuthority(t *testing.T) {
t.Run("a new CA root means a new account cache", func(t *testing.T) {
const directory = "https://anchor.internal/acme/acme/directory" const directory = "https://anchor.internal/acme/acme/directory"
before := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot)) before := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot))
after := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(secondRoot)) after := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(secondRoot))
if before == after { if before == after {
t.Fatalf("a re-initialised CA reuses the account it was rejected for: %s", before) t.Fatalf("a re-initialised CA reuses the account it was rejected for: %s", before)
} }
} })
// And the SAME authority keeps the account it registered, restart after restart. // And the SAME authority keeps the account it registered, restart after restart.
// //
@@ -34,7 +35,7 @@ func TestANewCARootMeansANewAccountCache(t *testing.T) {
// certificates that did not persist would be re-ordered on every restart, which works silently until // certificates that did not persist would be re-ordered on every restart, which works silently until
// a rate limit says it does not. Whitespace around the delivered root is not a new authority — the // a rate limit says it does not. Whitespace around the delivered root is not a new authority — the
// mesh writes that file, and a trailing newline coming or going must not throw away an account. // mesh writes that file, and a trailing newline coming or going must not throw away an account.
func TestTheSameAuthorityKeepsItsAccount(t *testing.T) { t.Run("the same authority keeps its account", func(t *testing.T) {
const directory = "https://anchor.internal/acme/acme/directory" const directory = "https://anchor.internal/acme/acme/directory"
first := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot)) first := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte(firstRoot))
again := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte("\n"+firstRoot+"\n\n")) again := forThisAuthority("/var/lib/route-proxy/acme", directory, []byte("\n"+firstRoot+"\n\n"))
@@ -42,25 +43,25 @@ func TestTheSameAuthorityKeepsItsAccount(t *testing.T) {
t.Errorf("the same authority was given two caches, so every restart orders again:\n%s\n%s", t.Errorf("the same authority was given two caches, so every restart orders again:\n%s\n%s",
first, again) first, again)
} }
} })
// Staging and production are different authorities, and were sharing one account. // Staging and production are different authorities, and were sharing one account.
// //
// The latent fault of the same shape: pointing ACME_DIRECTORY at production after testing against // The latent fault of the same shape: pointing ACME_DIRECTORY at production after testing against
// staging reused the staging account, because the cache had no idea they were different. // staging reused the staging account, because the cache had no idea they were different.
func TestStagingAndProductionDoNotShareAnAccount(t *testing.T) { t.Run("staging and production do not share an account", func(t *testing.T) {
staging := forThisAuthority("/acme", stagingDirectory, nil) staging := forThisAuthority("/acme", stagingDirectory, nil)
production := forThisAuthority("/acme", "https://acme-v02.api.letsencrypt.org/directory", nil) production := forThisAuthority("/acme", "https://acme-v02.api.letsencrypt.org/directory", nil)
if staging == production { if staging == production {
t.Errorf("two issuers share one account: %s", staging) t.Errorf("two issuers share one account: %s", staging)
} }
} })
// It stays inside the directory the mesh gave it, and is a plain name. // It stays inside the directory the mesh gave it, and is a plain name.
// //
// The mesh owns ACME_CACHE and mounts it; a name derived from a certificate that escaped it — or // The mesh owns ACME_CACHE and mounts it; a name derived from a certificate that escaped it — or
// that carried a separator out of the PEM — would put an account somewhere nothing persists. // that carried a separator out of the PEM — would put an account somewhere nothing persists.
func TestTheAccountCacheStaysWhereTheMeshPutIt(t *testing.T) { t.Run("the account cache stays where the mesh put it", func(t *testing.T) {
const cache = "/var/lib/route-proxy/acme" const cache = "/var/lib/route-proxy/acme"
got := forThisAuthority(cache, "https://anchor.internal/acme/acme/directory", []byte(firstRoot)) got := forThisAuthority(cache, "https://anchor.internal/acme/acme/directory", []byte(firstRoot))
if !strings.HasPrefix(got, cache+"/") { if !strings.HasPrefix(got, cache+"/") {
@@ -70,4 +71,5 @@ func TestTheAccountCacheStaysWhereTheMeshPutIt(t *testing.T) {
if name != filepath.Base(got) || strings.ContainsAny(name, "/.") { if name != filepath.Base(got) || strings.ContainsAny(name, "/.") {
t.Errorf("the account cache is not a plain name: %q", name) t.Errorf("the account cache is not a plain name: %q", name)
} }
})
} }
@@ -10,10 +10,23 @@ import (
"github.com/nats-io/nats.go" "github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/announce"
"git.novox.be/novox/mesh-sdk/go/membership" "git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/process" "git.novox.be/novox/mesh-sdk/go/process"
"route-proxy/internal/routestate"
) )
// keptStates is every state route-proxy keeps, as both its halves write them: the routes and the refusals this
// proxy writes, readable by others, and the bound entries the provider helper writes.
func keptStates() []announce.State {
return []announce.State{
{Name: routestate.BoundState},
{Name: routestate.RoutesState, Readable: true},
{Name: routestate.UnroutedState, Readable: true},
}
}
// What the mesh issued this proxy, read on the bus (novox/hq ADR 0160, ADR 0167). // What the mesh issued this proxy, read on the bus (novox/hq ADR 0160, ADR 0167).
// //
// **The proxy is told, not left to work it out.** Its membership carries the routes it is given — // **The proxy is told, not left to work it out.** Its membership carries the routes it is given —
@@ -44,6 +57,14 @@ func followOnce(path string, held *table, fromBus *atomic.Bool) error {
if err != nil { if err != nil {
return fmt.Errorf("connecting to the bus at %s: %w", membership.BareAddress(account.URL), err) return fmt.Errorf("connecting to the bus at %s: %w", membership.BareAddress(account.URL), err)
} }
// What this proxy keeps, said on the bus (novox/hq ADR 0495; issue 774): the names it serves and the names it
// refused, readable by any module granted a read of them; the provider helper's entries, read by the machines'
// clients through their own grant. Said before the first write, so no write of an unannounced state is refused.
if err := conn.Keeps(keptStates()...); err != nil {
log.Printf("cannot say on the bus what this proxy keeps, so it keeps nothing there: %v", err)
} else {
held.ledger.attach(account.Node, conn.State(routestate.RoutesState), conn.State(routestate.UnroutedState))
}
subject := membership.Subject(account.Node, account.Module) subject := membership.Subject(account.Node, account.Module)
err = conn.FollowMembership(func(issued membership.Membership) { err = conn.FollowMembership(func(issued membership.Membership) {
if took := applyMembership(issued, held); took && !fromBus.Swap(true) { if took := applyMembership(issued, held); took && !fromBus.Swap(true) {
@@ -0,0 +1,552 @@
package main
import (
"context"
"crypto/x509"
"encoding/json"
"encoding/pem"
"log"
"net/http"
"reflect"
"sort"
"strings"
"sync"
"time"
"golang.org/x/crypto/acme/autocert"
stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate"
)
// What this proxy says on the bus about the names it serves and the names it refuses (issue 774).
//
// **On change, never on a timer.** A route arriving or leaving, a certificate issued or failing, an upstream answer
// that differs from the last one written, a name asked for that is not served: each marks its key, and one writer
// puts the marked keys. An answer that is the same as the last one written is written again only once it is older
// than refreshAfter, so a site answering 200 all day costs one write a minute, not one per request.
//
// Until the bus is reached nothing is written; what happened meanwhile is held and written when it is.
// refreshAfter is how old a written answer, refusal or certificate failure may be before the same one is written
// again with its new time.
const refreshAfter = time.Minute
// mostAsked bounds the names held in memory as asked and not served: a scan sends any name it likes.
const mostAsked = 4096
// newPerMinute is how many names not seen before are kept by name in a minute; past it they are only counted.
const newPerMinute = 5
// kvStore is one of the module's states, as process.State is one.
type kvStore interface {
Put(key string, value any) (uint64, error)
Delete(key string) error
Keys() ([]string, error)
Get(key string) (*stdio.StateEntry, error)
}
// ledger is what this proxy keeps of what it serves and refuses, and writes to the bus.
type ledger struct {
mu sync.Mutex
now func() time.Time
node string
routes, unrouted kvStore
served map[string]*routestate.Route
previous map[string]routestate.Route // what the bus held for this machine when it was reached
asked map[string]*routestate.Unrouted
answerWritten map[string]time.Time
askedWritten map[string]time.Time
dirtyRoutes map[string]bool // a name to put when served, to delete when not
dirtyAsked map[string]bool
// domains are the registrable domains of the names served: a refused name under one is kept by name.
domains map[string]bool
newMinute time.Time
newInMinute int
other routestate.Other
otherMinute time.Time
otherInMinute int
otherDirty bool
routesFailing, askFailed bool // a failed write said once, until one succeeds
kick chan struct{}
}
func newLedger() *ledger {
return &ledger{
now: time.Now,
served: map[string]*routestate.Route{},
previous: map[string]routestate.Route{},
asked: map[string]*routestate.Unrouted{},
answerWritten: map[string]time.Time{},
askedWritten: map[string]time.Time{},
dirtyRoutes: map[string]bool{},
dirtyAsked: map[string]bool{},
domains: map[string]bool{},
kick: make(chan struct{}, 1),
}
}
// loop writes whatever was marked, each time something was.
func (l *ledger) loop() {
for range l.kick {
l.flush()
}
}
func (l *ledger) wake() {
select {
case l.kick <- struct{}{}:
default:
}
}
// attach is the bus reached, as node: what the bus held for this machine is read back, so a restart keeps a name's
// certificate and last answer, and everything held is written.
func (l *ledger) attach(node string, routes, unrouted kvStore) {
if l == nil {
return
}
before := map[string]routestate.Route{}
if keys, err := routes.Keys(); err != nil {
log.Printf("cannot read back what this proxy said it serves: %v", err)
} else {
for _, key := range keys {
at, name := routestate.Split(key)
if at != node {
continue
}
entry, err := routes.Get(key)
if err != nil || entry == nil {
continue
}
var r routestate.Route
if json.Unmarshal(entry.Value, &r) == nil {
before[name] = r
}
}
}
l.mu.Lock()
l.node, l.routes, l.unrouted = node, routes, unrouted
for name, r := range before {
l.previous[name] = r
l.dirtyRoutes[name] = true // put again when served, deleted when no longer
if now := l.served[name]; now != nil {
keepWhatWasKnown(now, r)
}
}
for name := range l.served {
l.dirtyRoutes[name] = true
}
for name := range l.asked {
l.dirtyAsked[name] = true
}
l.mu.Unlock()
l.wake()
}
// keepWhatWasKnown carries a certificate and a last answer read back from the bus onto a route that knows none yet.
func keepWhatWasKnown(r *routestate.Route, before routestate.Route) {
if r.Certificate.Outcome == "" && r.Certificate.NotAfter == nil && before.Certificate.Authority == r.Certificate.Authority {
r.Certificate = before.Certificate
}
if r.LastAnswer == nil {
r.LastAnswer = before.LastAnswer
}
}
// serving is the table this proxy now serves: a name new or changed is written, a name gone is deleted, and a name
// that was asked for unserved and now is served leaves the unrouted state.
func (l *ledger) serving(routes map[string][]rule, public map[string]bool) {
if l == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
for name, rules := range routes {
reach := routestate.ReachInternal
if public[name] {
reach = routestate.ReachPublic
}
targets := targetsOf(rules)
r := l.served[name]
if r == nil {
r = &routestate.Route{Name: name}
if before, ok := l.previous[name]; ok {
r.Certificate.Authority = reach
keepWhatWasKnown(r, before)
}
l.served[name] = r
l.dirtyRoutes[name] = true
}
if r.Reach != reach || !reflect.DeepEqual(r.Targets, targets) {
r.Reach, r.Targets = reach, targets
l.dirtyRoutes[name] = true
}
if r.Certificate.Authority != reach {
r.Certificate = routestate.Certificate{Authority: reach}
l.dirtyRoutes[name] = true
}
if _, was := l.asked[name]; was {
delete(l.asked, name)
l.dirtyAsked[name] = true
}
}
l.domains = map[string]bool{}
for name := range routes {
if d := routestate.Registrable(name); d != "" {
l.domains[d] = true
}
}
for name := range l.served {
if _, still := routes[name]; !still {
delete(l.served, name)
delete(l.answerWritten, name)
l.dirtyRoutes[name] = true
}
}
l.wake()
}
func targetsOf(rules []rule) []routestate.Target {
out := make([]routestate.Target, 0, len(rules))
for _, r := range rules {
out = append(out, routestate.Target{Path: r.path, Module: r.from, Node: r.node, Target: r.target,
Deny: r.policy.deny, Redirect: r.policy.redirectTo, Auth: len(r.policy.users) > 0 || r.policy.sealed != "",
Sealed: r.policy.sealed})
}
return out
}
// answered is an upstream answer to a request for name: its status, or the error that stood in for one.
func (l *ledger) answered(name, path string, status int, failed error) {
if l == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
r := l.served[name]
if r == nil {
return
}
now := l.now()
why := ""
if failed != nil {
why = failed.Error()
}
before := r.LastAnswer
r.LastAnswer = &routestate.Answer{Status: status, At: now, Error: why, Path: path}
if before != nil && before.Status == status && before.Error == why && now.Sub(l.answerWritten[name]) < refreshAfter {
return
}
l.answerWritten[name] = now
l.dirtyRoutes[name] = true
l.wake()
}
// refused is a name this proxy answered "no route for": in a request, or in a handshake's server name.
//
// **Only a name under a domain this proxy serves is kept by name**, and at most newPerMinute new ones a minute: a scan
// sends any name it likes, and each kept name is a key on the bus. Every other refusal is counted, and the count is
// written once a minute under the proxy's OtherName key. Nothing here scans: forgetting old names is the writer's.
// Who asked is not kept: the journal line beside it is what the machine's intrusion prevention reads.
func (l *ledger) refused(name string, internalOnly, handshake bool) {
if l == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
if _, isServed := l.served[name]; isServed && !internalOnly {
return
}
now := l.now()
a := l.asked[name]
if a == nil {
if !routestate.Keyable(name) || !l.domains[routestate.Registrable(name)] || !l.mayAddName(now) {
l.countOther(now)
return
}
a = &routestate.Unrouted{Name: name}
l.asked[name] = a
}
a.AskedAt, a.InternalOnly, a.Handshake = now, internalOnly, handshake
a.Times++
if a.Times > 1 && now.Sub(l.askedWritten[name]) < refreshAfter {
return
}
l.askedWritten[name] = now
l.dirtyAsked[name] = true
l.wake()
}
// mayAddName says whether another new name may be kept this minute, and counts it.
func (l *ledger) mayAddName(now time.Time) bool {
if minute := now.Truncate(time.Minute); !minute.Equal(l.newMinute) {
l.newMinute, l.newInMinute = minute, 0
}
if l.newInMinute >= newPerMinute {
return false
}
l.newInMinute++
return true
}
// countOther counts a refusal not kept by name. A minute's count is written when the next minute's first one comes.
func (l *ledger) countOther(now time.Time) {
minute := now.Truncate(time.Minute)
if !minute.Equal(l.otherMinute) {
if l.otherInMinute > 0 {
l.other.Minute, l.other.LastMinute = l.otherMinute, l.otherInMinute
l.otherDirty = true
l.wake()
}
l.otherMinute, l.otherInMinute = minute, 0
}
l.otherInMinute++
l.other.SinceStart++
}
// forgetOldestAsked drops the names asked longest ago from memory until mostAsked are left; the bus keeps them until
// their time to live ends. Called by the writer, never on a request.
func (l *ledger) forgetOldestAsked() {
if len(l.asked) <= mostAsked {
return
}
names := make([]string, 0, len(l.asked))
for name := range l.asked {
names = append(names, name)
}
sort.Slice(names, func(i, j int) bool { return l.asked[names[i]].AskedAt.Before(l.asked[names[j]].AskedAt) })
for _, name := range names[:len(names)-mostAsked] {
delete(l.asked, name)
delete(l.askedWritten, name)
delete(l.dirtyAsked, name)
}
}
// certified is a certificate an authority issued for name, as autocert stored it.
func (l *ledger) certified(name string, leaf *x509.Certificate) {
if l == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
r := l.served[name]
if r == nil {
return
}
now := l.now()
notAfter := leaf.NotAfter.UTC()
r.Certificate.Issuer, r.Certificate.NotAfter = issuerOf(leaf), &notAfter
r.Certificate.RenewedAt, r.Certificate.At = &now, &now
r.Certificate.Outcome, r.Certificate.Error = routestate.Issued, ""
l.dirtyRoutes[name] = true
l.wake()
}
// handedOut is the certificate a handshake for name was answered with: written when it is not the one recorded.
func (l *ledger) handedOut(name string, leaf *x509.Certificate) {
if l == nil || leaf == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
r := l.served[name]
if r == nil {
return
}
notAfter := leaf.NotAfter.UTC()
if r.Certificate.NotAfter != nil && r.Certificate.NotAfter.Equal(notAfter) && r.Certificate.Outcome != routestate.Failed {
return
}
now := l.now()
r.Certificate.Issuer, r.Certificate.NotAfter, r.Certificate.At = issuerOf(leaf), &notAfter, &now
if r.Certificate.Outcome != routestate.Issued || r.Certificate.Error != "" {
r.Certificate.Outcome, r.Certificate.Error = routestate.Cached, ""
}
l.dirtyRoutes[name] = true
l.wake()
}
// certificateFailed is a handshake for name that found no certificate to answer with, and why.
func (l *ledger) certificateFailed(name string, failed error) {
if l == nil {
return
}
l.mu.Lock()
defer l.mu.Unlock()
r := l.served[name]
if r == nil {
return
}
now := l.now()
same := r.Certificate.Outcome == routestate.Failed && r.Certificate.Error == failed.Error()
if same && r.Certificate.At != nil && now.Sub(*r.Certificate.At) < refreshAfter {
return
}
r.Certificate.Outcome, r.Certificate.Error, r.Certificate.At = routestate.Failed, failed.Error(), &now
l.dirtyRoutes[name] = true
l.wake()
}
func issuerOf(leaf *x509.Certificate) string {
if cn := leaf.Issuer.CommonName; cn != "" {
if len(leaf.Issuer.Organization) > 0 {
return leaf.Issuer.Organization[0] + " " + cn
}
return cn
}
return leaf.Issuer.String()
}
// flush writes every marked key. A route's write that fails stays marked and is tried again at the next change; a
// refused name's write that fails is dropped, never tried again: the bucket refusing it is full, or the bus is away,
// and either way it is not worth a storm of retries. Each kind of failure is logged once, until a write succeeds.
func (l *ledger) flush() {
type put struct {
store kvStore
key string
value any // nil deletes
route bool
}
l.mu.Lock()
if l.routes == nil {
l.mu.Unlock()
return
}
l.forgetOldestAsked()
var puts []put
for name := range l.dirtyRoutes {
p := put{store: l.routes, key: routestate.Key(l.node, name), route: true}
if r := l.served[name]; r != nil {
c := *r
c.Proxy = l.node
p.value = c
}
puts = append(puts, p)
}
for name := range l.dirtyAsked {
p := put{store: l.unrouted, key: routestate.Key(l.node, name)}
if a := l.asked[name]; a != nil {
c := *a
c.Proxy = l.node
p.value = c
}
puts = append(puts, p)
}
if l.otherDirty {
o := l.other
o.Proxy = l.node
puts = append(puts, put{store: l.unrouted, key: routestate.Key(l.node, routestate.OtherName), value: o})
}
l.dirtyRoutes, l.dirtyAsked, l.otherDirty = map[string]bool{}, map[string]bool{}, false
l.mu.Unlock()
for _, p := range puts {
var err error
if p.value == nil {
err = p.store.Delete(p.key)
} else {
_, err = p.store.Put(p.key, p.value)
}
l.mu.Lock()
switch {
case err == nil && p.route:
l.routesFailing = false
case err == nil:
l.askFailed = false
case p.route:
if !l.routesFailing {
log.Printf("cannot write %s to the bus, kept for the next change: %v", p.key, err)
}
l.routesFailing = true
_, name := routestate.Split(p.key)
l.dirtyRoutes[name] = true
default:
if !l.askFailed {
log.Printf("cannot write %s to the bus, dropped with every refused name until a write succeeds: %v",
p.key, err)
}
l.askFailed = true
}
l.mu.Unlock()
}
}
// recordingCache is an authority's certificate cache that tells the ledger of every certificate stored in it: the
// one place autocert says it was issued one.
type recordingCache struct {
autocert.Cache
held *table
}
func (c recordingCache) Put(ctx context.Context, key string, data []byte) error {
if err := c.Cache.Put(ctx, key, data); err != nil {
return err
}
name := strings.TrimSuffix(key, "+rsa")
if strings.Contains(name, "+") || !c.held.routed(name) {
return nil // the account key, a challenge token
}
if leaf := leafOf(data); leaf != nil {
c.held.ledger.certified(bareHost(name), leaf)
}
return nil
}
// leafOf is the first certificate in what autocert stores: the private key, then the chain, in PEM.
func leafOf(data []byte) *x509.Certificate {
for {
var block *pem.Block
block, data = pem.Decode(data)
if block == nil {
return nil
}
if block.Type == "CERTIFICATE" {
leaf, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil
}
return leaf
}
}
}
// answerRecorder carries a proxied request's answer through and remembers its status and the proxy's error.
type answerRecorder struct {
http.ResponseWriter
status int
err error
}
func (a *answerRecorder) WriteHeader(status int) {
if a.status == 0 {
a.status = status
}
a.ResponseWriter.WriteHeader(status)
}
func (a *answerRecorder) Write(b []byte) (int, error) {
if a.status == 0 {
a.status = http.StatusOK
}
return a.ResponseWriter.Write(b)
}
// Unwrap lets http.ResponseController reach the connection beneath: a flush, an upgrade's hijack.
func (a *answerRecorder) Unwrap() http.ResponseWriter { return a.ResponseWriter }
// proxyFailed is the reverse proxy's answer when upstream gave none: the error kept for the ledger, and a 502.
func proxyFailed(w http.ResponseWriter, r *http.Request, err error) {
if a, ok := w.(*answerRecorder); ok {
a.err = err
}
log.Printf("http: proxy error for %s%s: %v", r.Host, r.URL.Path, err)
w.WriteHeader(http.StatusBadGateway)
}
@@ -0,0 +1,291 @@
package main
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"math/big"
"net"
"net/http"
"net/http/httptest"
"sort"
"strings"
"testing"
"time"
"golang.org/x/crypto/acme/autocert"
stdio "git.novox.be/novox/mesh-sdk/go"
"route-proxy/internal/routestate"
)
// fakeState is a state on the bus, in memory, counting what was written to it.
type fakeState struct {
values map[string]json.RawMessage
puts int
tries int // every Put asked, refused or not
fail error // what every Put is refused with, while set
}
func newFakeState() *fakeState { return &fakeState{values: map[string]json.RawMessage{}} }
func (f *fakeState) Put(key string, value any) (uint64, error) {
f.tries++
if f.fail != nil {
return 0, f.fail
}
raw, err := json.Marshal(value)
if err != nil {
return 0, err
}
f.values[key] = raw
f.puts++
return uint64(f.puts), nil
}
func (f *fakeState) Delete(key string) error { delete(f.values, key); return nil }
func (f *fakeState) Keys() ([]string, error) {
out := []string{}
for k := range f.values {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
func (f *fakeState) Get(key string) (*stdio.StateEntry, error) {
v, ok := f.values[key]
if !ok {
return nil, nil
}
return &stdio.StateEntry{Key: key, Value: v}, nil
}
func (f *fakeState) route(t *testing.T, key string) *routestate.Route {
t.Helper()
raw, ok := f.values[key]
if !ok {
return nil
}
var r routestate.Route
if err := json.Unmarshal(raw, &r); err != nil {
t.Fatal(err)
}
return &r
}
// issuedPEM is what autocert stores for a name: the key, then the certificate, signed by an issuer named here.
func issuedPEM(t *testing.T, name string, notAfter time.Time) []byte {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{SerialNumber: big.NewInt(1), DNSNames: []string{name},
Subject: pkix.Name{CommonName: name}, Issuer: pkix.Name{CommonName: "R11", Organization: []string{"Let's Encrypt"}},
NotBefore: notAfter.Add(-90 * 24 * time.Hour), NotAfter: notAfter}
issuer := &x509.Certificate{SerialNumber: big.NewInt(2), Subject: template.Issuer, NotBefore: template.NotBefore,
NotAfter: notAfter, IsCA: true, BasicConstraintsValid: true}
der, err := x509.CreateCertificate(rand.Reader, template, issuer, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
keyDER, _ := x509.MarshalECPrivateKey(key)
out := pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: keyDER})
return append(out, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})...)
}
// What the proxy tells the bus (issue 774): each served name with where it goes, its certificate and the last
// upstream answer; each name refused; and only on change. Without the ledger nothing reaches the bus, and the
// operator's question "is grafana reachable" is answered with curl.
func TestTheBusIsToldWhatIsServedAndRefused(t *testing.T) {
notAfter := time.Date(2027, 1, 28, 12, 0, 0, 0, time.UTC)
gone := func() string { // an address nothing listens on: the upstream that does not answer
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer l.Close()
return l.Addr().String()
}()
_, port, _ := net.SplitHostPort(gone)
contributions := []contribution{
{From: "grafana", Node: "anchor", At: "127.0.0.1", Values: map[string]any{"name": "grafana.example", "port": mustFloat(t, port)}},
{From: "drive", Node: "home-server", At: "10.0.0.2", Values: map[string]any{"internal-name": "drive.internal", "port": 8080.0}},
}
for _, row := range []struct {
name string
act func(t *testing.T, held *table)
check func(t *testing.T, routes, unrouted *fakeState)
}{
{"a served name is written with the module and node it forwards to",
func(t *testing.T, held *table) {},
func(t *testing.T, routes, _ *fakeState) {
r := routes.route(t, "anchor.grafana.example")
if r == nil || r.Proxy != "anchor" || r.Reach != routestate.ReachPublic || len(r.Targets) != 1 ||
r.Targets[0].Module != "grafana" || r.Targets[0].Node != "anchor" || r.Targets[0].Target != "http://127.0.0.1:"+port {
t.Fatalf("grafana's route is not on the bus as served: %+v", r)
}
if d := routes.route(t, "anchor.drive.internal"); d == nil || d.Reach != routestate.ReachInternal ||
d.Certificate.Authority != routestate.ReachInternal {
t.Fatalf("an internal-only name is not on the bus as internal: %+v", d)
}
}},
{"a withdrawn route is deleted",
func(t *testing.T, held *table) {
routes, public := routesOf(contributions[1:])
held.set(routes, public)
},
func(t *testing.T, routes, _ *fakeState) {
if routes.route(t, "anchor.grafana.example") != nil {
t.Fatal("a route no longer served is still on the bus")
}
}},
{"an upstream that does not answer is written as a 502 with why, once a minute",
func(t *testing.T, held *table) {
for range 3 {
req := httptest.NewRequest(http.MethodGet, "http://grafana.example/login", nil)
handler(held).ServeHTTP(httptest.NewRecorder(), req)
}
},
func(t *testing.T, routes, _ *fakeState) {
r := routes.route(t, "anchor.grafana.example")
if r == nil || r.LastAnswer == nil || r.LastAnswer.Status != http.StatusBadGateway ||
r.LastAnswer.Error == "" || r.LastAnswer.Path != "/login" {
t.Fatalf("the failed answer is not on the bus: %+v", r)
}
if routes.puts != 3 { // the two routes, then the first answer; the same answer again is not written
t.Fatalf("the same answer was written %d times in all, want 3 writes", routes.puts)
}
}},
{"a name under a served domain is written as asked, without who asked, and leaves when it is served",
func(t *testing.T, held *table) {
req := httptest.NewRequest(http.MethodGet, "http://wiki.grafana.example/", nil)
req.RemoteAddr = "192.0.2.7:51000"
handler(held).ServeHTTP(httptest.NewRecorder(), req)
held.ledger.flush()
var a routestate.Unrouted
raw := held.ledger.unrouted.(*fakeState).values["anchor.wiki.grafana.example"]
if err := json.Unmarshal(raw, &a); err != nil || a.Times != 1 || a.AskedAt.IsZero() {
t.Fatalf("the refused name is not on the bus as asked: %s", raw)
}
if strings.Contains(string(raw), "192.0.2.7") || strings.Contains(string(raw), `"from"`) {
t.Fatalf("who asked is on the bus: %s", raw)
}
routes, public := routesOf(append(contributions, contribution{From: "grafana", Node: "anchor",
Values: map[string]any{"name": "wiki.grafana.example", "port": 3000.0}}))
held.set(routes, public)
},
func(t *testing.T, _, unrouted *fakeState) {
if len(unrouted.values) != 0 {
t.Fatalf("a name now served is still written as unrouted: %v", unrouted.values)
}
}},
{"a handshake for a name not routed is written as asked",
func(t *testing.T, held *table) {
pick := certificateFor(held, func(*tls.ClientHelloInfo) (*tls.Certificate, error) {
return nil, errors.New("no public route for \"forgotten.grafana.example\" in this mesh, so no certificate is asked for")
}, nil)
_, _ = pick(&tls.ClientHelloInfo{ServerName: "forgotten.grafana.example"})
},
func(t *testing.T, _, unrouted *fakeState) {
var a routestate.Unrouted
if err := json.Unmarshal(unrouted.values["anchor.forgotten.grafana.example"], &a); err != nil || !a.Handshake {
t.Fatalf("a handshake refused for its name is not on the bus: %v", unrouted.values)
}
}},
{"a flood of names writes at most the limit by name, counts the rest once a minute, and never retries a refused write",
func(t *testing.T, held *table) {
clock := time.Date(2026, 10, 30, 21, 0, 0, 0, time.UTC)
held.ledger.now = func() time.Time { return clock }
full := held.ledger.unrouted.(*fakeState)
for i := range 40 {
held.ledger.refused(fmt.Sprintf("x%d.grafana.example", i), false, false) // under a served domain
held.ledger.refused(fmt.Sprintf("x%d.scan.test", i), false, true) // under none
}
held.ledger.flush()
if named := len(full.values); named != newPerMinute {
t.Fatalf("a flood wrote %d names, want the limit %d: %v", named, newPerMinute, full.values)
}
full.fail = errors.New("nats: maximum bytes exceeded")
clock = clock.Add(time.Minute)
held.ledger.refused("y.grafana.example", false, false) // a new minute: one more name,
held.ledger.refused("v.scan.test", false, false) // and the last minute's count
held.ledger.flush()
tries := full.tries
held.ledger.refused("y.grafana.example", false, false)
held.ledger.refused("z.scan.test", false, false)
held.ledger.flush()
if full.tries != tries {
t.Fatalf("writes the bucket refused were tried again: %d tries, then %d", tries, full.tries)
}
full.fail = nil
clock = clock.Add(time.Minute)
held.ledger.refused("w.scan.test", false, false)
},
func(t *testing.T, _, unrouted *fakeState) {
var o routestate.Other
if err := json.Unmarshal(unrouted.values["anchor._other"], &o); err != nil || o.LastMinute != 2 || o.SinceStart != 78 {
t.Fatalf("the refusals not kept by name are not counted on the bus: %s", unrouted.values["anchor._other"])
}
}},
{"a certificate an authority issued is written with its issuer, expiry and renewal",
func(t *testing.T, held *table) {
cache := recordingCache{Cache: autocert.DirCache(t.TempDir()), held: held}
if err := cache.Put(context.Background(), "grafana.example", issuedPEM(t, "grafana.example", notAfter)); err != nil {
t.Fatal(err)
}
_ = cache.Put(context.Background(), "acme_account+key", []byte("not a certificate"))
},
func(t *testing.T, routes, _ *fakeState) {
c := routes.route(t, "anchor.grafana.example").Certificate
if c.Outcome != routestate.Issued || c.Issuer != "Let's Encrypt R11" || c.NotAfter == nil ||
!c.NotAfter.Equal(notAfter) || c.RenewedAt == nil {
t.Fatalf("the issued certificate is not on the bus: %+v", c)
}
}},
{"a certificate that cannot be had is written with why",
func(t *testing.T, held *table) {
pick := certificateFor(held, func(*tls.ClientHelloInfo) (*tls.Certificate, error) {
return nil, errors.New("acme: urn:ietf:params:acme:error:rateLimited")
}, nil)
_, _ = pick(&tls.ClientHelloInfo{ServerName: "grafana.example"})
},
func(t *testing.T, routes, _ *fakeState) {
c := routes.route(t, "anchor.grafana.example").Certificate
if c.Outcome != routestate.Failed || c.Error != "acme: urn:ietf:params:acme:error:rateLimited" {
t.Fatalf("the failed certificate is not on the bus: %+v", c)
}
}},
} {
t.Run(row.name, func(t *testing.T) {
held := newTable()
held.ledger = newLedger()
routes, unrouted := newFakeState(), newFakeState()
held.ledger.attach("anchor", routes, unrouted)
table, public := routesOf(contributions)
held.set(table, public)
held.ledger.flush()
row.act(t, held)
held.ledger.flush()
row.check(t, routes, unrouted)
})
}
}
func mustFloat(t *testing.T, port string) float64 {
t.Helper()
var n float64
if err := json.Unmarshal([]byte(port), &n); err != nil {
t.Fatal(err)
}
return n
}
+48 -7
View File
@@ -179,6 +179,8 @@ type rule struct {
// registry is the case that needs it — image layers arrive as single requests of gigabytes, and // registry is the case that needs it — image layers arrive as single requests of gigabytes, and
// a proxy's own default refuses them long before the workload is reached. // a proxy's own default refuses them long before the workload is reached.
maxRequestBody int64 maxRequestBody int64
// from and node are the consumer that contributed the rule and its machine, which the bus is told (issue 774).
from, node string
} }
// table is what the proxy is currently serving, replaced whole whenever the file changes. // table is what the proxy is currently serving, replaced whole whenever the file changes.
@@ -202,6 +204,8 @@ type table struct {
// machine's address on the private network, as the mesh issued it in this proxy's membership // machine's address on the private network, as the mesh issued it in this proxy's membership
// (novox/hq ADR 0167). Empty until it is issued, and then only the machine itself is inside. // (novox/hq ADR 0167). Empty until it is issued, and then only the machine itself is inside.
inside sources inside sources
// ledger is what this proxy says on the bus about what it serves and refuses (issue 774); nil says nothing.
ledger *ledger
} }
// sources is who may be served an internal name: the private network's addresses as the mesh // sources is who may be served an internal name: the private network's addresses as the mesh
@@ -289,6 +293,7 @@ func (t *table) set(routes map[string][]rule, public map[string]bool) {
t.to = made t.to = made
t.public = public t.public = public
t.mu.Unlock() t.mu.Unlock()
t.ledger.serving(made, public)
} }
// inOrder puts the rules for one host into the order they are matched in, and does so totally. // inOrder puts the rules for one host into the order they are matched in, and does so totally.
@@ -437,6 +442,8 @@ func run() error {
} }
held := newTable() held := newTable()
held.ledger = newLedger()
go held.ledger.loop()
// **The bus first, the file until it has spoken** (novox/hq ADR 0167). The membership carries // **The bus first, the file until it has spoken** (novox/hq ADR 0167). The membership carries
// the routes and who the mesh is; the file carries the routes alone, so while the proxy reads // the routes and who the mesh is; the file carries the routes alone, so while the proxy reads
// it an internal name is served to this machine and to nobody else — refused, never opened. // it an internal name is served to this machine and to nobody else — refused, never opened.
@@ -487,7 +494,7 @@ func run() error {
"to persist, or every restart orders them again") "to persist, or every restart orders them again")
} }
publicManager, err := newManager(cache, issuer(), strings.TrimSpace(os.Getenv("ACME_CA_BUNDLE")), publicManager, err := newManager(cache, issuer(), strings.TrimSpace(os.Getenv("ACME_CA_BUNDLE")),
onlyWhatTheMeshSaid(held)) onlyWhatTheMeshSaid(held), held)
if err != nil { if err != nil {
return err return err
} }
@@ -499,7 +506,7 @@ func run() error {
var internalManager *autocert.Manager var internalManager *autocert.Manager
if directory := strings.TrimSpace(os.Getenv("INTERNAL_ACME_DIRECTORY")); directory != "" { if directory := strings.TrimSpace(os.Getenv("INTERNAL_ACME_DIRECTORY")); directory != "" {
internalManager, err = newManager(cache, directory, strings.TrimSpace(os.Getenv("INTERNAL_ACME_CA_BUNDLE")), internalManager, err = newManager(cache, directory, strings.TrimSpace(os.Getenv("INTERNAL_ACME_CA_BUNDLE")),
onlyInternalNamesTheMeshSaid(held)) onlyInternalNamesTheMeshSaid(held), held)
if err != nil { if err != nil {
return fmt.Errorf("internal certificate authority: %w", err) return fmt.Errorf("internal certificate authority: %w", err)
} }
@@ -619,7 +626,7 @@ func (p *probedResponse) replayTo(w http.ResponseWriter) {
// one of the public ones — the lab's, or the mesh's own step-ca — serves its own ACME API over TLS // one of the public ones — the lab's, or the mesh's own step-ca — serves its own ACME API over TLS
// with a certificate nothing trusts yet. *Skip* would also apply the day this points at a public // with a certificate nothing trusts yet. *Skip* would also apply the day this points at a public
// issuer, and nothing would say so; naming a bundle is a deliberate, visible act instead. // issuer, and nothing would say so; naming a bundle is a deliberate, visible act instead.
func newManager(cache, directory, bundle string, policy autocert.HostPolicy) (*autocert.Manager, error) { func newManager(cache, directory, bundle string, policy autocert.HostPolicy, held *table) (*autocert.Manager, error) {
client := &acme.Client{DirectoryURL: directory} client := &acme.Client{DirectoryURL: directory}
var root []byte var root []byte
if bundle != "" { if bundle != "" {
@@ -651,7 +658,7 @@ func newManager(cache, directory, bundle string, policy autocert.HostPolicy) (*a
// different names because their directories differ. // different names because their directories differ.
mine := forThisAuthority(cache, directory, root) mine := forThisAuthority(cache, directory, root)
return &autocert.Manager{ return &autocert.Manager{
Cache: autocert.DirCache(mine), Cache: recordingCache{Cache: autocert.DirCache(mine), held: held},
Prompt: autocert.AcceptTOS, Prompt: autocert.AcceptTOS,
HostPolicy: policy, HostPolicy: policy,
Client: client, Client: client,
@@ -700,9 +707,10 @@ func certificateFor(held *table, fromPublic func(*tls.ClientHelloInfo) (*tls.Cer
if internalManager != nil { if internalManager != nil {
fromInternal = internalManager.TLSConfig().GetCertificate fromInternal = internalManager.TLSConfig().GetCertificate
} }
return func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) { pick := func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
if held.eligibleForInternalACME(hello.ServerName) { if held.eligibleForInternalACME(hello.ServerName) {
if hello.Conn != nil && held.hiddenFrom(hello.ServerName, hello.Conn.RemoteAddr().String()) { if hello.Conn != nil && held.hiddenFrom(hello.ServerName, hello.Conn.RemoteAddr().String()) {
held.ledger.refused(bareHost(hello.ServerName), true, true)
return nil, fmt.Errorf("no public route for %q in this mesh, so no certificate is asked for", return nil, fmt.Errorf("no public route for %q in this mesh, so no certificate is asked for",
hello.ServerName) hello.ServerName)
} }
@@ -712,6 +720,28 @@ func certificateFor(held *table, fromPublic func(*tls.ClientHelloInfo) (*tls.Cer
} }
return fromPublic(hello) return fromPublic(hello)
} }
// What the bus is told of a handshake (issue 774): a routed name's certificate, or why it has none; a name not
// routed here, which a browser asking for a site the mesh forgot to route meets before any request is made.
return func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
cert, err := pick(hello)
name := bareHost(hello.ServerName)
switch {
case name == "":
case !held.routed(name):
held.ledger.refused(name, false, true)
case err != nil:
if !strings.HasPrefix(err.Error(), "no public route for") {
held.ledger.certificateFailed(name, err)
}
case cert != nil:
leaf := cert.Leaf
if leaf == nil && len(cert.Certificate) > 0 {
leaf, _ = x509.ParseCertificate(cert.Certificate[0])
}
held.ledger.handedOut(name, leaf)
}
return cert, err
}
} }
// newTable is an empty routing table. // newTable is an empty routing table.
@@ -737,6 +767,9 @@ func handler(held *table) http.Handler {
// this proxy's log for exactly that line (novox/hq ADR 0179): the address last, as the // this proxy's log for exactly that line (novox/hq ADR 0179): the address last, as the
// jail's filter expects it. // jail's filter expects it.
log.Printf("refused: no route for %q, asked from %s", r.Host, r.RemoteAddr) log.Printf("refused: no route for %q, asked from %s", r.Host, r.RemoteAddr)
if hidden || !held.routed(r.Host) {
held.ledger.refused(bareHost(r.Host), hidden, false)
}
w.Header().Set("Content-Type", "text/plain; charset=utf-8") w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusNotFound) w.WriteHeader(http.StatusNotFound)
if !hidden && held.routed(r.Host) { if !hidden && held.routed(r.Host) {
@@ -786,7 +819,14 @@ func handler(held *table) http.Handler {
r.Body = http.MaxBytesReader(w, r.Body, matched.maxRequestBody) r.Body = http.MaxBytesReader(w, r.Body, matched.maxRequestBody)
} }
matched.to.ServeHTTP(w, r) // The upstream's answer is what the bus is told of this name (issue 774): its status, or why there was none.
answer := &answerRecorder{ResponseWriter: w}
matched.to.ServeHTTP(answer, r)
status := answer.status
if status == 0 && answer.err == nil && r.Header.Get("Upgrade") != "" {
status = http.StatusSwitchingProtocols
}
held.ledger.answered(bareHost(r.Host), r.URL.Path, status, answer.err)
}) })
} }
@@ -955,6 +995,7 @@ func routesOf(contributions []contribution) (map[string][]rule, map[string]bool)
} }
made.target = fmt.Sprintf("%s://%s:%d", scheme, at, port) made.target = fmt.Sprintf("%s://%s:%d", scheme, at, port)
} }
made.from, made.node = c.From, c.Node
if name != "" { if name != "" {
host := strings.ToLower(name) host := strings.ToLower(name)
@@ -1072,5 +1113,5 @@ func towards(where *url.URL) *httputil.ReverseProxy {
pr.SetURL(where) pr.SetURL(where)
pr.Out.Host = pr.In.Host pr.Out.Host = pr.In.Host
pr.SetXForwarded() pr.SetXForwarded()
}} }, ErrorHandler: proxyFailed}
} }
@@ -121,19 +121,6 @@ func TestARouteWithOnlyAnInternalNameIsServed(t *testing.T) {
} }
} }
// A route with neither name has nothing to be served under, and is still skipped.
func TestARouteWithNeitherNameIsSkipped(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal","values":{"internal-name":" ","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 || len(public) != 0 {
t.Errorf("a route that named nothing was served: %v %v", routes, public)
}
}
// A route with no internal-name composed gets no second host — the ordinary case, unchanged. // A route with no internal-name composed gets no second host — the ordinary case, unchanged.
func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) { func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[ routes, _, err := routesFrom(write(t, `{"given":[
@@ -161,21 +148,6 @@ func TestAConsumerOnTheProxysOwnMachineIsReachedOverLoopback(t *testing.T) {
} }
} }
// Skipped rather than served wrongly. A route with no port would proxy to :0.
func TestAContributionMissingWhatARouteNeedsIsSkipped(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"a","node":"n","at":"n.internal","values":{"name":"no-port.example"}},
{"from":"b","node":"n","at":"n.internal","values":{"port":8080}},
{"from":"c","node":"n","at":"n.internal","values":{"name":"fine.example","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 1 || targetOf(routes, "fine.example") == "" {
t.Fatalf("an unusable contribution was served: %v", routes)
}
}
// A route may name a target reached over https, for a backend that terminates its own TLS — the // A route may name a target reached over https, for a backend that terminates its own TLS — the
// shape Mailu's webmail front needs, which this proxy reaches as a plain workload otherwise. // shape Mailu's webmail front needs, which this proxy reaches as a plain workload otherwise.
func TestARouteMayTargetHttps(t *testing.T) { func TestARouteMayTargetHttps(t *testing.T) {
@@ -194,19 +166,6 @@ func TestARouteMayTargetHttps(t *testing.T) {
} }
} }
// A scheme that is neither http nor https is refused rather than guessed at.
func TestARouteWithAnUnknownSchemeIsSkipped(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"a","node":"n","at":"n.internal","values":{"name":"bad.example","port":80,"scheme":"ftp"}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 {
t.Fatalf("a route with an unusable scheme was served: %v", routes)
}
}
// End to end: a backend terminating TLS with a certificate nothing would ordinarily trust is still // End to end: a backend terminating TLS with a certificate nothing would ordinarily trust is still
// reached when the route declared `insecure`, and the response comes back through unmodified. // reached when the route declared `insecure`, and the response comes back through unmodified.
func TestTheProxyReachesAnInsecureHttpsBackend(t *testing.T) { func TestTheProxyReachesAnInsecureHttpsBackend(t *testing.T) {
@@ -455,23 +414,6 @@ func TestARouteThatSaysNothingCarriesAnySize(t *testing.T) {
} }
} }
// A limit that is not a whole positive number of bytes takes the route with it. Serving it without
// the limit would carry exactly what the module said not to carry, and report success doing it.
func TestARouteWithAnUnusableLimitIsSkipped(t *testing.T) {
for _, asked := range []string{`"lots"`, `-1`, `0`, `1.5`} {
routes, _, err := routesFrom(write(t, `{"given":[
{"from":"registry","node":"anchor","at":"anchor.internal",
"values":{"name":"images.example","port":5000,"max-request-body":`+asked+`}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes["images.example"]) != 0 {
t.Errorf("a route asking for a max-request-body of %s was served anyway: %v", asked, routes)
}
}
}
// A request larger than the route carries is refused by the proxy, with the limit named, and the // A request larger than the route carries is refused by the proxy, with the limit named, and the
// workload never sees it. A request within it is proxied normally. // workload never sees it. A request within it is proxied normally.
func TestABodyOverTheLimitIsRefusedAndOneUnderItIsCarried(t *testing.T) { func TestABodyOverTheLimitIsRefusedAndOneUnderItIsCarried(t *testing.T) {
@@ -532,3 +474,32 @@ func post(url, host, body string) (*http.Response, error) {
asked.Header.Set("Content-Type", "application/octet-stream") asked.Header.Set("Content-Type", "application/octet-stream")
return http.DefaultClient.Do(asked) return http.DefaultClient.Do(asked)
} }
// A contribution that cannot be served is skipped rather than served wrongly: one that names nothing to be served
// under; one with no port, which would proxy to :0; a scheme that is neither http nor https, refused rather than
// guessed at; and a body limit that is not a whole positive number of bytes, since serving it without the limit would
// carry exactly what the module said not to carry, and report success doing it.
func TestAContributionThatCannotBeServedIsSkipped(t *testing.T) {
for _, row := range []struct{ why, given, served string }{
{"neither name", `{"from":"app","node":"anchor","at":"anchor.internal","values":{"internal-name":" ","port":8080}}`, ""},
{"no port", `{"from":"a","node":"n","at":"n.internal","values":{"name":"no-port.example"}},
{"from":"b","node":"n","at":"n.internal","values":{"port":8080}},
{"from":"c","node":"n","at":"n.internal","values":{"name":"fine.example","port":8080}}`, "fine.example"},
{"an unknown scheme", `{"from":"a","node":"n","at":"n.internal","values":{"name":"bad.example","port":80,"scheme":"ftp"}}`, ""},
{"a limit in words", `{"from":"r","node":"a","at":"a.internal","values":{"name":"images.example","port":5000,"max-request-body":"lots"}}`, ""},
{"a negative limit", `{"from":"r","node":"a","at":"a.internal","values":{"name":"images.example","port":5000,"max-request-body":-1}}`, ""},
{"a zero limit", `{"from":"r","node":"a","at":"a.internal","values":{"name":"images.example","port":5000,"max-request-body":0}}`, ""},
{"a fractional limit", `{"from":"r","node":"a","at":"a.internal","values":{"name":"images.example","port":5000,"max-request-body":1.5}}`, ""},
} {
routes, public, err := routesFrom(write(t, `{"given":[`+row.given+`]}`))
if err != nil {
t.Fatal(err)
}
switch {
case row.served == "" && (len(routes) != 0 || len(public) != 0):
t.Errorf("%s: an unusable contribution was served: %v %v", row.why, routes, public)
case row.served != "" && (len(routes) != 1 || targetOf(routes, row.served) == ""):
t.Errorf("%s: an unusable contribution was served beside the usable one: %v", row.why, routes)
}
}
}
+1 -1
View File
@@ -3,7 +3,7 @@ module route-proxy
go 1.26.0 go 1.26.0
require ( require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68
github.com/nats-io/nats.go v1.54.0 github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0 golang.org/x/crypto v0.57.0
) )
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9 h1:Ufmd2cYEXICQ817IGflccP9bexABE4WhIO49HPbaoSo= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68 h1:2rMJX0RMIBaN5NDIaEZVwbj5MFism1O3bRayEz+WlOs=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261022190740-104533e8c0c9/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY= git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261030231858-b9ef1e4fcb68/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI= github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo= github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
@@ -0,0 +1,151 @@
// Package routestate is what the web front keeps on the bus about the names it serves, and about the names it was asked
// for and does not serve (issue 774): the shape both halves of route-proxy agree on. The proxy writes it, on change,
// in its states `routes` and `unrouted`, announced readable; the module's tools read it from there, so the question
// "is this site reachable, and is its certificate good" is answered from the bus rather than with curl.
//
// A key is `<node>.<name>`: the machine whose proxy answers, then the name as asked, lower-cased. A node's name holds
// no dot, so the first dot splits them.
package routestate
import (
"net"
"regexp"
"strings"
"time"
)
// The states route-proxy keeps, by their local names (the buckets are route-proxy_<name>).
const (
// RoutesState holds one key per served name: where it forwards, its certificate, the last answer it got.
RoutesState = "routes"
// UnroutedState holds one key per name the proxy answered "no route for" with, and when it was last asked.
UnroutedState = "unrouted"
// BoundState is the provider helper's bucket of consumer entries (mesh-sdk go/provider).
BoundState = "bound"
)
// The words a route's reach and a certificate's authority are said in.
const (
ReachPublic = "public"
ReachInternal = "internal"
)
// The words a certificate's outcome is said in.
const (
// Issued: the authority issued it while this proxy ran, at RenewedAt.
Issued = "issued"
// Cached: this proxy served it from what it kept, and saw no issuance of it since it started.
Cached = "cached"
// Failed: the last attempt to have one failed, Error says why.
Failed = "failed"
)
// Target is one rule a name is served by: what a request matching it reaches, or what it is answered with.
type Target struct {
// Path is the path prefix the rule is scoped to; empty is every path.
Path string `json:"path,omitempty"`
// Module and Node are the consumer that contributed the route and the machine it runs on.
Module string `json:"module,omitempty"`
Node string `json:"node,omitempty"`
// Target is where the request is forwarded, scheme://address:port; empty for a rule that answers itself.
Target string `json:"target,omitempty"`
Deny bool `json:"deny,omitempty"`
Redirect string `json:"redirect,omitempty"`
// Auth says the rule asks for credentials; Sealed says why it cannot check them, and so refuses everything.
Auth bool `json:"auth,omitempty"`
Sealed string `json:"sealed,omitempty"`
}
// Certificate is what this proxy knows of a name's certificate.
type Certificate struct {
// Authority is which issuer certifies the name: public (the ACME directory the module names) or internal (the
// mesh's own).
Authority string `json:"authority"`
Issuer string `json:"issuer,omitempty"`
NotAfter *time.Time `json:"not_after,omitempty"`
// RenewedAt is when an authority last issued it to this proxy.
RenewedAt *time.Time `json:"renewed_at,omitempty"`
// Outcome is issued, cached or failed; empty until a handshake or an issuance said anything.
Outcome string `json:"outcome,omitempty"`
Error string `json:"error,omitempty"`
At *time.Time `json:"at,omitempty"`
}
// Answer is the last answer the proxy got from upstream for a name: the status it carried back, or the error that
// stood in for one (a 502).
type Answer struct {
Status int `json:"status"`
At time.Time `json:"at"`
Error string `json:"error,omitempty"`
Path string `json:"path,omitempty"`
}
// Route is one served name, the value of a key of RoutesState.
type Route struct {
Name string `json:"name"`
// Proxy is the machine whose proxy serves it.
Proxy string `json:"proxy"`
// Reach is public (a route's own name) or internal (only its internal-name, served inside the private network).
Reach string `json:"reach"`
Targets []Target `json:"targets"`
Certificate Certificate `json:"certificate"`
LastAnswer *Answer `json:"last_answer,omitempty"`
}
// Unrouted is one name the proxy answered "no route for", the value of a key of UnroutedState.
type Unrouted struct {
Name string `json:"name"`
Proxy string `json:"proxy"`
// AskedAt is when the name was last asked for. Who asked is not kept here: the journal keeps it, for the
// machine's intrusion prevention.
AskedAt time.Time `json:"asked_at"`
// Times is how often it was asked since the proxy last started.
Times int `json:"times"`
// InternalOnly says the name is served, but only inside the private network, and was asked from outside.
InternalOnly bool `json:"internal_only,omitempty"`
// Handshake says it was asked in a TLS handshake (the name a client sent), not in an HTTP request.
Handshake bool `json:"handshake,omitempty"`
}
// OtherName is the name of the one key per proxy that counts the refusals not kept by name: names under no domain
// the proxy serves (what a scan sends), and names past the limit of new names a minute. Its key is
// `<node>._other`; no host name begins with an underscore.
const OtherName = "_other"
// Other is the value of a proxy's OtherName key.
type Other struct {
Proxy string `json:"proxy"`
// Minute is the last whole minute counted, and LastMinute how many refusals it held.
Minute time.Time `json:"minute"`
LastMinute int `json:"last_minute"`
// SinceStart is every refusal not kept by name since the proxy started.
SinceStart int `json:"since_start"`
}
// Registrable is the domain a name is registered under, as its last two labels; "" for an address or a single
// label. Two labels is wrong for a public suffix of two (co.uk); it only decides which refused names are kept by name.
func Registrable(name string) string {
if net.ParseIP(strings.Trim(name, "[]")) != nil {
return ""
}
labels := strings.Split(name, ".")
if len(labels) < 2 {
return ""
}
return strings.Join(labels[len(labels)-2:], ".")
}
var keyable = regexp.MustCompile(`^[a-z0-9_]([a-z0-9_.-]{0,251}[a-z0-9_])?$`)
// Keyable says whether a name can be a key's name: what a host name or an address is. A name asked for in a request
// is whatever a client sent; one no key could hold is no site anybody runs.
func Keyable(name string) bool { return keyable.MatchString(name) && !strings.Contains(name, "..") }
// Key is the key of name served or asked at node's proxy.
func Key(node, name string) string { return node + "." + name }
// Split is a key's node and name.
func Split(key string) (node, name string) {
node, name, _ = strings.Cut(key, ".")
return node, name
}
+24 -2
View File
@@ -244,9 +244,31 @@
} }
], ],
"state": [ "state": [
"bound" "bound",
"routes",
{
"name": "unrouted",
"ttl-seconds": 604800
}
], ],
"tools": [
"route_proxy_routes",
"route_proxy_route",
"route_proxy_unrouted"
],
"replaces": {
"route_proxy_routes": [
"curl -I"
],
"route_proxy_route": [
"openssl s_client -connect"
],
"route_proxy_unrouted": [
"docker logs route-proxy"
]
},
"reads": [ "reads": [
"mesh-controller.assignments" "mesh-provisioner.plans",
"mesh-provisioner.provisions"
] ]
} }
+2 -2
View File
@@ -7,9 +7,9 @@ The copy carries the catalogue's private details: domains, first names in module
private addresses. That is acceptable while mesh-controller is private; if it ever goes public, this copy is private addresses. That is acceptable while mesh-controller is private; if it ever goes public, this copy is
a second place to clean besides the catalogue itself. a second place to clean besides the catalogue itself.
mesh-catalog 3a1017f5594217092a654dd1e2d72226766bc16a modules/*/module.json, modules/*/files, modules/*/templates, predefined.json mesh-catalog f3a57dcc56fb6c198095cafd11d0decff9f720c4 modules/*/module.json, modules/*/files, modules/*/templates, predefined.json
mesh-host 9d9e3a46c4f5943cfffb413d962ce48b75e2fd39 examples/foundation-first-node.lock mesh-host 9d9e3a46c4f5943cfffb413d962ce48b75e2fd39 examples/foundation-first-node.lock
mesh-sdk 78c9ec8817710eae7f1a068aef68924293fdef59 conformance/events/module-event.json mesh-sdk 2da0de75c3f0087cff9c2f5ef789ca6dc3c59045 conformance/events/module-event.json
To move them, from this repository's root, with the two repositories checked out beside it: To move them, from this repository's root, with the two repositories checked out beside it:
@@ -0,0 +1 @@
${seat:mesh-artifact-store:reach}
@@ -1,12 +1,38 @@
{ {
"module": "artifact-store-tools", "module": "artifact-store-tools",
"version": "1", "version": "1",
"claims": [
{
"name": "mesh-artifact-sweep",
"scope": "mesh",
"serves": [
"artifacts",
"held",
"collect",
"mirrors",
"images"
]
}
],
"requires": [
"artifact-store"
],
"invokes": [ "invokes": [
"seat:mesh-controller.artifacts", "seat:mesh-hand-acts.hand-act",
"seat:mesh-controller.collect",
"mesh-catalog.catalog_modules", "mesh-catalog.catalog_modules",
"mesh-catalog.catalog_module" "mesh-catalog.catalog_module"
], ],
"reads": [
"mesh-delivery-assigner.wants",
"mesh-delivery-builder.builds",
"mesh-delivery-builder.build-log",
"mesh-delivery-indexer.manifests",
"node-engine.have",
"mesh-controller.lease"
],
"state": [
"findings"
],
"tools": [ "tools": [
"artifact_store_repositories", "artifact_store_repositories",
"artifact_store_usage", "artifact_store_usage",
@@ -36,6 +62,14 @@
"mode": "0600", "mode": "0600",
"trusted": false, "trusted": false,
"template": "files/store-names" "template": "files/store-names"
},
{
"id": "store-address",
"type": "file",
"path": "${dir:state}/store-address",
"mode": "0600",
"trusted": false,
"template": "files/store-address"
} }
], ],
"build": { "build": {
@@ -52,7 +86,8 @@
], ],
"env": { "env": {
"MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry", "MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry",
"MESH_ARTIFACT_STORE_NAMES_FILE": "${dir:state}/store-names" "MESH_ARTIFACT_STORE_NAMES_FILE": "${dir:state}/store-names",
"MESH_ARTIFACT_STORE_ADDRESS_FILE": "${dir:state}/store-address"
} }
}, },
{ {
@@ -8,7 +8,10 @@
"provides": [ "provides": [
{ {
"name": "rootless-image-build", "name": "rootless-image-build",
"reach": "machine" "reach": "machine",
"credential": {
"form": "none"
}
} }
], ],
"resources": [ "resources": [
@@ -7,7 +7,9 @@
"disk_filesystems", "disk_filesystems",
"disk_folders", "disk_folders",
"disk_devices", "disk_devices",
"cpu_top" "cpu_top",
"disk_readings",
"disk_measure"
], ],
"build": { "build": {
"artifacts": [ "artifacts": [
@@ -23,5 +25,16 @@
] ]
} }
] ]
} },
"state": [
"readings"
],
"reads": [
"mesh-config.resolved",
"mesh-delivery-assigner.placement",
"mesh-delivery-builder.builds",
"mesh-delivery-indexer.manifests",
"mesh-delivery-indexer.modules",
"node-engine.have"
]
} }
@@ -4,7 +4,10 @@
"provides": [ "provides": [
{ {
"name": "artifact-store", "name": "artifact-store",
"scope": "mesh" "scope": "mesh",
"credential": {
"form": "none"
}
} }
], ],
"claims": [ "claims": [
@@ -1,14 +1,15 @@
# Written by the mesh (module dnsmasq, novox/hq issue 348). Replaced on every push. # Written by the mesh (module dnsmasq, novox/hq issue 348). Replaced on every push.
# #
# The resolver binds its addresses once, at start (bind-interfaces), so it starts only once this # The resolver binds its addresses once, at start (bind-interfaces), so it starts only once this
# machine's private address is there: it waits up to 80 seconds for it, and a start that found it # machine's private address is there: a start that finds it missing fails after at most two seconds and
# missing is tried again every ten seconds rather than left failed. # is tried again every ten seconds (Restart=on-failure, no start limit) until the tunnel has brought it.
# It does not order itself After= or Wants= network-online.target, and its wait is short: the packaged
# unit is Before= network-online.target and nss-lookup.target, and the tunnel is ordered after them, so
# a long wait would hold the tunnel back and a drop-in ordering would be a cycle (issue 1126).
[Unit] [Unit]
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=0 StartLimitIntervalSec=0
[Service] [Service]
ExecStartPre=/usr/bin/sh -c 'for i in $(seq 1 80); do ip -4 -o addr show to ${machine:address} | grep -q . && exit 0; sleep 1; done; echo "${machine:address} is not on this machine" >&2; exit 1' ExecStartPre=/usr/bin/sh -c 'for i in $(seq 1 2); do ip -4 -o addr show to ${machine:address} | grep -q . && exit 0; sleep 1; done; echo "${machine:address} is not on this machine" >&2; exit 1'
Restart=on-failure Restart=on-failure
RestartSec=10s RestartSec=10s
@@ -9,7 +9,10 @@
{ {
"name": "wildcard-resolution", "name": "wildcard-resolution",
"scope": "mesh", "scope": "mesh",
"identity": false "identity": false,
"credential": {
"form": "none"
}
} }
], ],
"requires": [ "requires": [
@@ -7,7 +7,7 @@
"privileged" "privileged"
], ],
"invokes": [ "invokes": [
"seat:mesh-controller.images" "seat:mesh-artifact-sweep.images"
], ],
"claims": [ "claims": [
{ {
@@ -40,6 +40,7 @@
], ],
"emits": [ "emits": [
"repo.created", "repo.created",
"repo.writable",
"issue.opened", "issue.opened",
"pull.merged", "pull.merged",
"pull.updated", "pull.updated",
@@ -181,6 +182,12 @@
"trusted": true, "trusted": true,
"default": {} "default": {}
}, },
"check-gates-merge": {
"type": "boolean",
"description": "Whether gitea_merge_pull_request judges mesh/check before it merges. True (the default): a head merges only on a passing mesh/check judged against the trunk's head (ADR 0447, ADR 0534 §4). False (the operator, 2026-10-25 22:56: \"run the check locally only, but not as a gate, it loses us too much time\"): a head the operator's Approve names in approved_heads merges whatever its mesh/check says (pending, failed, missing or judged on an older trunk); a head that moved, a branch no rule guards and a merge git refuses are still refused. Set together with mesh-delivery-gate's check-gates-merge.",
"trusted": true,
"default": true
},
"main-mergers": { "main-mergers": {
"type": "list", "type": "list",
"description": "The forge logins that may merge into main beside the tools' account, on every repository of the organisation (ADR 0534 §2): the operator's own, so the operator can always merge by hand at the forge. The tools' account is always on main's merge whitelist; no agent's account belongs here.", "description": "The forge logins that may merge into main beside the tools' account, on every repository of the organisation (ADR 0534 §2): the operator's own, so the operator can always merge by hand at the forge. The tools' account is always on main's merge whitelist; no agent's account belongs here.",
@@ -259,7 +266,8 @@
"env": { "env": {
"DB_TYPE": "postgres", "DB_TYPE": "postgres",
"USER_UID": "1000", "USER_UID": "1000",
"USER_GID": "1000" "USER_GID": "1000",
"SSH_MAX_STARTUPS": "100:30:200"
}, },
"env-file": [ "env-file": [
"${dir:state}/server.env" "${dir:state}/server.env"
@@ -316,7 +324,8 @@
"takes-config": [ "takes-config": [
"url", "url",
"keep-main-rules", "keep-main-rules",
"main-mergers" "main-mergers",
"check-gates-merge"
] ]
}, },
{ {
@@ -391,6 +400,7 @@
"provides": [ "provides": [
{ {
"name": "npm-package-registry", "name": "npm-package-registry",
"credential": {"form": "pair"},
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"max": 40, "max": 40,
@@ -399,6 +409,7 @@
}, },
{ {
"name": "git", "name": "git",
"credential": {"form": "none"},
"scope": "mesh" "scope": "mesh"
}, },
{ {
@@ -411,6 +422,7 @@
}, },
{ {
"name": "forge-account", "name": "forge-account",
"credential": {"form": "none"},
"scope": "mesh" "scope": "mesh"
} }
], ],
@@ -537,6 +549,7 @@
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret", "MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins", "MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins",
"MESH_GITEA_SSH_PORT": "222", "MESH_GITEA_SSH_PORT": "222",
"MESH_GITEA_HOST_KEY_FILE": "${dir:data}/ssh/ssh_host_ed25519_key.pub",
"MESH_FORGE_HOST_FILE": "/etc/gitea/forge-host" "MESH_FORGE_HOST_FILE": "/etc/gitea/forge-host"
} }
} }
@@ -554,11 +567,14 @@
"announced" "announced"
], ],
"reads": [ "reads": [
"mesh-controller.assignments", "mesh-provisioner.plans",
"mesh-provisioner.provisions",
"ssh-client.keys", "ssh-client.keys",
"mesh-build-go.keys", "mesh-build-go.keys",
"mesh-build-node.keys", "mesh-build-node.keys",
"mesh-build-image.keys", "mesh-build-image.keys",
"mesh-delivery-checker.owed" "mesh-delivery-checker.owed",
"mesh-delivery-indexer.modules",
"mesh-delivery-gate.source"
] ]
} }
@@ -7,7 +7,10 @@
"provides": [ "provides": [
{ {
"name": "go-toolchain", "name": "go-toolchain",
"reach": "machine" "reach": "machine",
"credential": {
"form": "none"
}
} }
], ],
"resources": [ "resources": [
@@ -7,6 +7,9 @@
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"in": "an InfluxDB v1 authorization" "in": "an InfluxDB v1 authorization"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -0,0 +1,4 @@
# Written by the mesh's kernel module (systemd-modules-load reads it at boot).
# tun: buildah's network for a RUN line (pasta) and every other user-space tunnel open /dev/net/tun.
# Loaded at boot, so an upgrade that removes the running kernel's modules cannot keep it out later.
tun
@@ -100,6 +100,25 @@
"id": "linux-lts-headers", "id": "linux-lts-headers",
"type": "package", "type": "package",
"package": "linux-lts-headers" "package": "linux-lts-headers"
},
{
"id": "modules-load",
"type": "file",
"path": "/etc/modules-load.d/mesh-kernel.conf",
"mode": "0644",
"template": "files/modules-load.conf"
},
{
"id": "modules-keep",
"type": "package",
"package": "kernel-modules-hook"
},
{
"id": "modules-cleanup",
"type": "service",
"unit": "linux-modules-cleanup.service",
"state": "stopped",
"boot": "enabled"
} }
], ],
"build": { "build": {
@@ -9,6 +9,7 @@
"provides": [ "provides": [
{ {
"name": "oidc-client", "name": "oidc-client",
"credential": {"form": "pair"},
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"max": 255, "max": 255,
@@ -240,6 +241,7 @@
"bound" "bound"
], ],
"reads": [ "reads": [
"mesh-controller.assignments" "mesh-provisioner.plans",
"mesh-provisioner.provisions"
] ]
} }
@@ -665,7 +665,14 @@
"path": "${dir:mesh-state}/config.json", "path": "${dir:mesh-state}/config.json",
"mode": "0600", "mode": "0600",
"template": "files/config.json", "template": "files/config.json",
"takes-config": ["domain", "proxy-address", "sitename", "website", "url", "container"], "takes-config": [
"domain",
"proxy-address",
"sitename",
"website",
"url",
"container"
],
"merge": "json" "merge": "json"
}, },
{ {
@@ -758,6 +765,9 @@
"identity": { "identity": {
"max": 64, "max": 64,
"in": "a mailbox's local part" "in": "a mailbox's local part"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -1,3 +1,5 @@
{ {
"capacity": "shanks=3,ace=2,g14=0,novox=0" "capacity": 0,
"cores-per-step": 6,
"build-hours": ""
} }
@@ -10,7 +10,7 @@
"state": [ "state": [
"toolchains", "toolchains",
"keys", "keys",
"steps" {"name": "steps", "ttl-seconds": 172800}, "pauses"
], ],
"claims": [ "claims": [
{ {
@@ -20,14 +20,27 @@
} }
], ],
"reads": [ "reads": [
"mesh-delivery-checker.kept" "mesh-delivery-checker.kept",
"mesh-delivery-indexer.manifests"
], ],
"config": { "config": {
"capacity": { "capacity": {
"type": "number",
"description": "How many asks of the Go build seat this module runs at once on this node (novox/hq ADR 0525 §5); 0 pulls nothing. It is set per node in the mesh's configuration; a node with no value builds nothing.",
"minimum": 0,
"default": 0
},
"build-hours": {
"type": "string", "type": "string",
"description": "How many asks of the Go build seat this module runs at once on each node, as node=asks pairs separated by commas, such as shanks=3,ace=2 (novox/hq ADR 0525 §5); a node set to 0 pulls nothing, a node not named runs one at a time. The control-node and the laptop build nothing.", "description": "When this node may take this module's build asks: one window HH:MM-HH:MM in the node's local time (an end before its start crosses midnight, 18:00-06:00). Outside it the node takes no new step and finishes the ones it runs; an empty value builds at any hour. It is set per node in the mesh's configuration.",
"pattern": "|[a-z0-9_-]+=[0-9]+(,[a-z0-9_-]+=[0-9]+)*", "pattern": "|([01][0-9]|2[0-3]):[0-5][0-9]-([01][0-9]|2[0-3]):[0-5][0-9]",
"default": "shanks=3,ace=2,g14=0,novox=0" "default": ""
},
"cores-per-step": {
"type": "number",
"description": "How many processors one Go step of this module needs (a full check of a large repository takes what it is given: three at about 4 cores each ran past the 30-minute step bound on 2026-10-26). The capacity above is an upper bound: on each node the module runs at most floor(the cores of its build slice's CPU quota / this) steps at once, the quota read live from its cgroup (issue 1806), so a lowered build CPU share never starves its steps. Above 0.",
"minimum": 0.1,
"default": 6
} }
}, },
"tools": [ "tools": [
@@ -38,6 +51,7 @@
"build_go_release", "build_go_release",
"build_go_step", "build_go_step",
"build_go_steps", "build_go_steps",
"build_go_step_end",
"build_go_toolchains", "build_go_toolchains",
"build_go_status", "build_go_status",
"build_go_health" "build_go_health"
@@ -105,7 +119,7 @@
"path": "${dir:state}/settings.json", "path": "${dir:state}/settings.json",
"mode": "0600", "mode": "0600",
"owner": "mesh-build-go", "owner": "mesh-build-go",
"takes-config": ["capacity"], "takes-config": ["capacity", "cores-per-step", "build-hours"],
"merge": "json", "merge": "json",
"template": "files/settings.json" "template": "files/settings.json"
}, },
@@ -1,3 +1,5 @@
{ {
"capacity": "shanks=3,ace=2,g14=0,novox=0" "capacity": 0,
"cores-per-step": 2,
"build-hours": ""
} }
@@ -9,7 +9,7 @@
], ],
"state": [ "state": [
"keys", "keys",
"steps" {"name": "steps", "ttl-seconds": 172800}, "pauses"
], ],
"claims": [ "claims": [
{ {
@@ -20,10 +20,22 @@
], ],
"config": { "config": {
"capacity": { "capacity": {
"type": "number",
"description": "How many asks of the image build seat this module runs at once on this node (novox/hq ADR 0525 §5); 0 pulls nothing. It is set per node in the mesh's configuration; a node with no value builds nothing.",
"minimum": 0,
"default": 0
},
"build-hours": {
"type": "string", "type": "string",
"description": "How many asks of the image build seat this module runs at once on each node, as node=asks pairs separated by commas, such as shanks=3,ace=2 (novox/hq ADR 0525 §5); a node set to 0 pulls nothing, a node not named runs one at a time. The control-node and the laptop build nothing.", "description": "When this node may take this module's build asks: one window HH:MM-HH:MM in the node's local time (an end before its start crosses midnight, 18:00-06:00). Outside it the node takes no new step and finishes the ones it runs; an empty value builds at any hour. It is set per node in the mesh's configuration.",
"pattern": "|[a-z0-9_-]+=[0-9]+(,[a-z0-9_-]+=[0-9]+)*", "pattern": "|([01][0-9]|2[0-3]):[0-5][0-9]-([01][0-9]|2[0-3]):[0-5][0-9]",
"default": "shanks=3,ace=2,g14=0,novox=0" "default": ""
},
"cores-per-step": {
"type": "number",
"description": "How many processors one image step of this module needs (buildah, whose RUN lines are mostly fetches). The capacity above is an upper bound: on each node the module runs at most floor(the cores of its build slice's CPU quota / this) steps at once, the quota read live from its cgroup (issue 1806), so a lowered build CPU share never starves its steps. Above 0.",
"minimum": 0.1,
"default": 2
} }
}, },
"tools": [ "tools": [
@@ -33,6 +45,7 @@
"build_image_release", "build_image_release",
"build_image_step", "build_image_step",
"build_image_steps", "build_image_steps",
"build_image_step_end",
"build_image_status", "build_image_status",
"build_image_health" "build_image_health"
], ],
@@ -77,7 +90,7 @@
"path": "${dir:state}/settings.json", "path": "${dir:state}/settings.json",
"mode": "0600", "mode": "0600",
"owner": "mesh-build-image", "owner": "mesh-build-image",
"takes-config": ["capacity"], "takes-config": ["capacity", "cores-per-step", "build-hours"],
"merge": "json", "merge": "json",
"template": "files/settings.json" "template": "files/settings.json"
}, },
@@ -1,3 +1,5 @@
{ {
"capacity": "shanks=3,ace=2,g14=0,novox=0" "capacity": 0,
"cores-per-step": 2,
"build-hours": ""
} }
@@ -10,7 +10,7 @@
"state": [ "state": [
"toolchains", "toolchains",
"keys", "keys",
"steps" {"name": "steps", "ttl-seconds": 172800}, "pauses"
], ],
"claims": [ "claims": [
{ {
@@ -21,10 +21,22 @@
], ],
"config": { "config": {
"capacity": { "capacity": {
"type": "number",
"description": "How many asks of the Node build seat this module runs at once on this node (novox/hq ADR 0525 §5); 0 pulls nothing. It is set per node in the mesh's configuration; a node with no value builds nothing.",
"minimum": 0,
"default": 0
},
"build-hours": {
"type": "string", "type": "string",
"description": "How many asks of the Node build seat this module runs at once on each node, as node=asks pairs separated by commas, such as shanks=3,ace=2 (novox/hq ADR 0525 §5); a node set to 0 pulls nothing, a node not named runs one at a time. The control-node and the laptop build nothing.", "description": "When this node may take this module's build asks: one window HH:MM-HH:MM in the node's local time (an end before its start crosses midnight, 18:00-06:00). Outside it the node takes no new step and finishes the ones it runs; an empty value builds at any hour. It is set per node in the mesh's configuration.",
"pattern": "|[a-z0-9_-]+=[0-9]+(,[a-z0-9_-]+=[0-9]+)*", "pattern": "|([01][0-9]|2[0-3]):[0-5][0-9]-([01][0-9]|2[0-3]):[0-5][0-9]",
"default": "shanks=3,ace=2,g14=0,novox=0" "default": ""
},
"cores-per-step": {
"type": "number",
"description": "How many processors one Node step of this module needs (tsc, esbuild and a test runner; a median step takes 15 s). The capacity above is an upper bound: on each node the module runs at most floor(the cores of its build slice's CPU quota / this) steps at once, the quota read live from its cgroup (issue 1806), so a lowered build CPU share never starves its steps. Above 0.",
"minimum": 0.1,
"default": 2
} }
}, },
"tools": [ "tools": [
@@ -34,6 +46,7 @@
"build_node_release", "build_node_release",
"build_node_step", "build_node_step",
"build_node_steps", "build_node_steps",
"build_node_step_end",
"build_node_toolchains", "build_node_toolchains",
"build_node_status", "build_node_status",
"build_node_health" "build_node_health"
@@ -99,7 +112,7 @@
"path": "${dir:state}/settings.json", "path": "${dir:state}/settings.json",
"mode": "0600", "mode": "0600",
"owner": "mesh-build-node", "owner": "mesh-build-node",
"takes-config": ["capacity"], "takes-config": ["capacity", "cores-per-step", "build-hours"],
"merge": "json", "merge": "json",
"template": "files/settings.json" "template": "files/settings.json"
}, },
@@ -25,9 +25,7 @@
"secrets": { "secrets": {
"postgres-database": "${dir:state}/database.secret" "postgres-database": "${dir:state}/database.secret"
}, },
"consumes": [ "consumes": [],
"mesh-controller.built-before"
],
"emits": [ "emits": [
"registered", "registered",
"upgraded", "upgraded",
@@ -92,5 +90,12 @@
"from": "files" "from": "files"
} }
] ]
} },
"invokes": [
"mesh-delivery-builder.builder_builds_since"
],
"reads": [
"mesh-delivery-builder.builds",
"mesh-delivery-indexer.manifests"
]
} }
@@ -0,0 +1,87 @@
{
"module": "mesh-conditions",
"version": "1",
"runs-as": "mesh-conditions",
"claims": [
{
"name": "mesh-conditions",
"scope": "mesh",
"serves": [
"conditions",
"silence",
"compare",
"take-over",
"hand-back",
"resync"
]
}
],
"uses": [
"operator-channel"
],
"reads": [
"mesh-delivery-watchman.findings",
"mesh-self-check.findings",
"mesh-healers.findings",
"mesh-delivery-assigner.findings",
"messenger.findings",
"mesh-provisioner.findings",
"artifact-store-tools.findings",
"mesh-healers.attempts",
"mesh-controller.conditions",
"mesh-controller.lease",
"mesh-delivery-assigner.wants",
"node-engine.have",
"mesh-delivery-assigner.placement"
],
"state": [
"conditions",
{
"name": "history",
"ttl-seconds": 7776000
},
"takeover",
"held",
{
"name": "bounds",
"key-ttl": true
},
"asked"
],
"own-secrets": {
"broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-conditions",
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-conditions",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-conditions"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-conditions",
"place": ".",
"unknown": "program"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-conditions",
"binary": "mesh-conditions",
"loads": [
"mesh-conditions"
]
}
]
}
}
@@ -7,7 +7,10 @@
"name": "mesh-config", "name": "mesh-config",
"scope": "mesh", "scope": "mesh",
"serves": [ "serves": [
"config" "config",
"compare",
"take-over",
"hand-back"
] ]
} }
], ],
@@ -23,14 +26,30 @@
"gitea.forge.pull-request.closed" "gitea.forge.pull-request.closed"
], ],
"invokes": [ "invokes": [
"seat:mesh-hand-acts.hand-act",
"seat:mesh-hand-acts.warranted",
"seat:mesh-controller.hand-act", "seat:mesh-controller.hand-act",
"seat:mesh-controller.warranted", "seat:mesh-controller.warranted",
"gitea.gitea_get_pull_request" "gitea.gitea_get_pull_request"
], ],
"reads": [ "reads": [
"mesh-controller.assignments", "mesh-delivery-assigner.placement",
"mesh-delivery-assigner.wants",
"mesh-delivery-builder.builds",
"mesh-delivery-indexer.modules",
"mesh-delivery-indexer.manifests",
"mesh-controller.predefined",
"mesh-controller.declared-config", "mesh-controller.declared-config",
"mesh-controller.predefined" "mesh-delivery-indexer.predefined",
"mesh-delivery-indexer.config-candidates",
"mesh-controller.lease"
],
"tools": [
"config_compare",
"config_take_over",
"config_hand_back",
"config_declared",
"config_resync"
], ],
"state": [ "state": [
"values", "values",
@@ -38,7 +57,9 @@
"predefined", "predefined",
"proposals", "proposals",
"pulls", "pulls",
"kept" "kept",
"declared",
"takeover"
], ],
"logs": [ "logs": [
"history" "history"
@@ -1,4 +1,5 @@
{ {
"refusal-stall-floor-minutes": 10, "refusal-stall-floor-minutes": 10,
"author-new-head-minutes": 120 "author-new-head-minutes": 120,
"check-gates-merge": true
} }
@@ -14,7 +14,8 @@
"table", "table",
"stalled", "stalled",
"recheck", "recheck",
"close" "close",
"settle"
] ]
} }
], ],
@@ -26,13 +27,17 @@
"gitea.forge.pull-request.edited", "gitea.forge.pull-request.edited",
"gitea.forge.pull-request.closed", "gitea.forge.pull-request.closed",
"mesh-delivery-forge-bridge.forge.**", "mesh-delivery-forge-bridge.forge.**",
"mesh-config.config-changed" "mesh-config.config-changed",
"gitea.repo.writable"
], ],
"emits": [ "emits": [
"transition", "transition",
"group", "group",
"merged" "merged"
], ],
"uses": [
"operator-channel"
],
"invokes": [ "invokes": [
"gitea.gitea_commit_statuses", "gitea.gitea_commit_statuses",
"gitea.gitea_contains", "gitea.gitea_contains",
@@ -57,14 +62,23 @@
{ {
"name": "source", "name": "source",
"history": 64 "history": 64
},
"conditions",
{
"name": "turns",
"history": 5
} }
], ],
"reads": [ "reads": [
"mesh-delivery-teller.answers", "mesh-delivery-teller.answers",
"mesh-delivery-checker.verdicts" "mesh-delivery-checker.verdicts",
"mesh-delivery-view.reach"
], ],
"tools": [ "tools": [
"delivery_status" "delivery_hand_now",
"delivery_merge_again",
"delivery_status",
"delivery_turns"
], ],
"config": { "config": {
"refusal-stall-floor-minutes": { "refusal-stall-floor-minutes": {
@@ -82,6 +96,12 @@
"maximum": 10080, "maximum": 10080,
"trusted": false, "trusted": false,
"default": 120 "default": 120
},
"check-gates-merge": {
"type": "boolean",
"description": "Whether mesh/check gates the merge. True (the default): the operator's Approve is asked, and a merge handed, only on a passing verdict judged against the trunk's head, and a trunk's move puts a verdict out of date (ADR 0447, ADR 0505 §4). False (the operator, 2026-10-25 22:56: \"run the check locally only, but not as a gate, it loses us too much time\"; developers run the unit and integration levels locally before a pull request): a pull request reviewed at its head is ready, and its Approve is asked once the check at its heads (a group: of all its heads together) has a verdict, a pass or a failure said in the ask, or after 60 minutes with none, said so (the operator, 2026-10-27 23:54: \"if you don't want me to approve it, don't send the approval request\"); an approved merge waits for no verdict; no merge is refused for a check judged on an older trunk; a later verdict is shown on the pull request and the delivery, and asks a pending Approve again in its words; a group still merges all its members in one pass. Set it false together with the gitea module's check-gates-merge, which lets its merge tool merge an approved head without a passing mesh/check.",
"trusted": false,
"default": true
} }
}, },
"resources": [ "resources": [
@@ -99,7 +119,8 @@
"mode": "0600", "mode": "0600",
"takes-config": [ "takes-config": [
"refusal-stall-floor-minutes", "refusal-stall-floor-minutes",
"author-new-head-minutes" "author-new-head-minutes",
"check-gates-merge"
], ],
"merge": "json", "merge": "json",
"template": "files/settings.json" "template": "files/settings.json"
@@ -0,0 +1,65 @@
{
"module": "mesh-hand-acts",
"version": "1",
"runs-as": "mesh-hand-acts",
"claims": [
{
"name": "mesh-hand-acts",
"scope": "mesh",
"serves": [
"hand-act",
"hand-acts",
"warranted"
]
}
],
"reads": [
"mesh-controller.hand-acts",
"messenger.asks"
],
"logs": [
{
"name": "acts",
"max-age-days": 400
}
],
"state": [
"warrants"
],
"own-secrets": {
"broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-hand-acts",
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-hand-acts",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-hand-acts"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-hand-acts",
"place": ".",
"unknown": "program"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-hand-acts",
"binary": "mesh-hand-acts",
"loads": [
"mesh-hand-acts"
]
}
]
}
}
@@ -0,0 +1,82 @@
{
"module": "mesh-healers",
"version": "1",
"runs-as": "mesh-healers",
"claims": [
{
"name": "mesh-healers",
"scope": "mesh",
"serves": [
"healers",
"heal",
"drill"
]
}
],
"consumes": [
"mesh-conditions.condition-raised",
"mesh-conditions.condition-changed",
"mesh-hand-acts.act-recorded"
],
"invokes": [
"mesh-grants.grants_assert",
"seat:mesh-hand-acts.hand-act",
"seat:mesh-controller.drill"
],
"reads": [
"mesh-controller.lease",
"mesh-conditions.conditions",
"mesh-controller.hand-acts"
],
"state": [
{
"name": "attempts",
"key-ttl": true
},
{
"name": "would",
"key-ttl": true
},
"findings",
{
"name": "wakes",
"key-ttl": true
}
],
"own-secrets": {
"broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-healers",
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-healers",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-healers"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-healers",
"place": ".",
"unknown": "program"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-healers",
"binary": "mesh-healers",
"loads": [
"mesh-healers"
]
}
]
}
}
@@ -62,10 +62,13 @@
"mesh-catalog modules/mesh-build-node", "mesh-catalog modules/mesh-build-node",
"mesh-catalog modules/mesh-catalog", "mesh-catalog modules/mesh-catalog",
"mesh-catalog modules/mesh-cli", "mesh-catalog modules/mesh-cli",
"mesh-catalog modules/mesh-conditions",
"mesh-catalog modules/mesh-config", "mesh-catalog modules/mesh-config",
"mesh-catalog modules/mesh-delivery", "mesh-catalog modules/mesh-delivery",
"mesh-catalog modules/mesh-delivery-gate", "mesh-catalog modules/mesh-delivery-gate",
"mesh-catalog modules/mesh-files", "mesh-catalog modules/mesh-files",
"mesh-catalog modules/mesh-hand-acts",
"mesh-catalog modules/mesh-healers",
"mesh-catalog modules/mesh-issues", "mesh-catalog modules/mesh-issues",
"mesh-catalog modules/mesh-knowledge", "mesh-catalog modules/mesh-knowledge",
"mesh-catalog modules/mesh-mail", "mesh-catalog modules/mesh-mail",
@@ -75,6 +78,7 @@
"mesh-catalog modules/mesh-provisioner", "mesh-catalog modules/mesh-provisioner",
"mesh-catalog modules/mesh-provisioner-client", "mesh-catalog modules/mesh-provisioner-client",
"mesh-catalog modules/mesh-reads", "mesh-catalog modules/mesh-reads",
"mesh-catalog modules/mesh-self-check",
"mesh-catalog modules/mesh-vault", "mesh-catalog modules/mesh-vault",
"mesh-catalog modules/mesh-watcher", "mesh-catalog modules/mesh-watcher",
"mesh-catalog modules/messenger", "mesh-catalog modules/messenger",
@@ -13,12 +13,17 @@
"move", "move",
"note", "note",
"link", "link",
"milestones",
"milestone-set",
"milestone-move",
"import", "import",
"groom", "groom",
"set", "set",
"tracking", "tracking",
"reserve", "reserve",
"fix-times" "fix-times",
"learn-fixes",
"changes"
] ]
} }
], ],
@@ -33,12 +38,17 @@
"move", "move",
"note", "note",
"link", "link",
"milestones",
"milestone-set",
"milestone-move",
"import", "import",
"groom", "groom",
"set", "set",
"tracking", "tracking",
"reserve", "reserve",
"fix-times" "fix-times",
"learn-fixes",
"changes"
] ]
} }
], ],
@@ -52,7 +62,9 @@
"triaged", "triaged",
"ready", "ready",
"ready-to-verify", "ready-to-verify",
"reserved" "reserved",
"milestone-set",
"milestone-moved"
], ],
"consumes": [ "consumes": [
"mesh-issues.opened", "mesh-issues.opened",
@@ -66,6 +78,7 @@
"seat:operator-channel.notify", "seat:operator-channel.notify",
"gitea.gitea_pull_reviews", "gitea.gitea_pull_reviews",
"gitea.gitea_contains", "gitea.gitea_contains",
"gitea.gitea_get_pull_request",
"records.records_status", "records.records_status",
"records.records_list", "records.records_list",
"records.records_read", "records.records_read",
@@ -81,7 +94,11 @@
}, },
"counters", "counters",
"grooming", "grooming",
"reservations" "reservations",
{
"name": "milestones",
"history": 5
}
], ],
"config": { "config": {
"first-number": { "first-number": {
@@ -157,10 +174,13 @@
"mesh-catalog modules/mesh-build-node", "mesh-catalog modules/mesh-build-node",
"mesh-catalog modules/mesh-catalog", "mesh-catalog modules/mesh-catalog",
"mesh-catalog modules/mesh-cli", "mesh-catalog modules/mesh-cli",
"mesh-catalog modules/mesh-conditions",
"mesh-catalog modules/mesh-config", "mesh-catalog modules/mesh-config",
"mesh-catalog modules/mesh-delivery", "mesh-catalog modules/mesh-delivery",
"mesh-catalog modules/mesh-delivery-gate", "mesh-catalog modules/mesh-delivery-gate",
"mesh-catalog modules/mesh-files", "mesh-catalog modules/mesh-files",
"mesh-catalog modules/mesh-hand-acts",
"mesh-catalog modules/mesh-healers",
"mesh-catalog modules/mesh-issues", "mesh-catalog modules/mesh-issues",
"mesh-catalog modules/mesh-knowledge", "mesh-catalog modules/mesh-knowledge",
"mesh-catalog modules/mesh-mail", "mesh-catalog modules/mesh-mail",
@@ -170,6 +190,7 @@
"mesh-catalog modules/mesh-provisioner", "mesh-catalog modules/mesh-provisioner",
"mesh-catalog modules/mesh-provisioner-client", "mesh-catalog modules/mesh-provisioner-client",
"mesh-catalog modules/mesh-reads", "mesh-catalog modules/mesh-reads",
"mesh-catalog modules/mesh-self-check",
"mesh-catalog modules/mesh-vault", "mesh-catalog modules/mesh-vault",
"mesh-catalog modules/mesh-watcher", "mesh-catalog modules/mesh-watcher",
"mesh-catalog modules/messenger", "mesh-catalog modules/messenger",
@@ -32,7 +32,8 @@
"propose", "propose",
"import", "import",
"return", "return",
"withdraw" "withdraw",
"progress"
] ]
} }
], ],
@@ -52,13 +53,15 @@
"propose", "propose",
"import", "import",
"return", "return",
"withdraw" "withdraw",
"progress"
] ]
} }
], ],
"tools": [ "tools": [
"knowledge_health", "knowledge_health",
"knowledge_highest" "knowledge_highest",
"knowledge_marks"
], ],
"emits": [ "emits": [
"reserved", "reserved",
@@ -81,7 +84,14 @@
"records.records_status", "records.records_status",
"records.records_list", "records.records_list",
"records.records_read", "records.records_read",
"seat:operator-channel.notify" "seat:operator-channel.notify",
"seat:issue-tracker.list",
"seat:issue-tracker.get",
"seat:issue-tracker.milestones",
"seat:issue-tracker.changes",
"seat:mesh-delivery.deliveries",
"seat:mesh-delivery.show",
"seat:mesh-delivery.groups"
], ],
"config": { "config": {
"judge-model": { "judge-model": {
@@ -11,11 +11,36 @@
] ]
} }
], ],
"uses": [
"operator-channel"
],
"reads": [
"mesh-delivery-assigner.placement",
"mesh-delivery-assigner.wants",
"mesh-delivery-indexer.modules",
"mesh-delivery-indexer.manifests",
"mesh-delivery-builder.builds",
"node-engine.have",
"mesh-config.resolved",
"gitea.bound",
"keycloak.bound",
"postgres.bound",
"route-adapter.bound",
"mesh-vault.bound"
],
"tools": [
"provisioner_provisions",
"provisioner_plans",
"provisioner_compose",
"provisioner_decide"
],
"state": [ "state": [
"registry", "registry",
"needs", "needs",
"applied", "applied",
"collect" "collect",
"provisions",
"plans"
], ],
"own-secrets": { "own-secrets": {
"broker": "${dir:state}/broker" "broker": "${dir:state}/broker"
@@ -0,0 +1,87 @@
{
"module": "mesh-self-check",
"version": "1",
"runs-as": "mesh-self-check",
"claims": [
{
"name": "mesh-self-check",
"scope": "mesh",
"serves": [
"doctor",
"probes"
]
}
],
"invokes": [
"nats.nats_streams",
"nats.nats_closed_connections",
"seat:mesh-watchdogs.signals",
"seat:node-intrusion-prevention.banned"
],
"reads": [
"mesh-delivery-assigner.placement",
"mesh-delivery-assigner.seat-holders",
"mesh-delivery-watchman.presence",
"mesh-host.health",
"disk-load.readings",
"restic.good",
"mesh-delivery-indexer.modules",
"mesh-delivery-indexer.manifests"
],
"state": [
"findings",
"verdict",
{
"name": "verdicts",
"ttl-seconds": 604800
}
],
"own-secrets": {
"broker": "${dir:state}/broker"
},
"secrets-owner": "mesh-self-check",
"resources": [
{
"id": "account",
"type": "user",
"name": "mesh-self-check",
"shell": "/usr/bin/nologin",
"home": "/var/lib/mesh-self-check"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "mesh-self-check",
"place": ".",
"unknown": "program"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/mesh-self-check",
"binary": "mesh-self-check",
"loads": [
"mesh-self-check"
]
}
]
},
"config": {
"probes": {
"type": "list",
"description": "The ids of the probes that run, each judging one thing of the mesh (mesh-self-check.probes lists them with what each asserts). A probe not listed runs nothing and raises nothing, and what it raised before is cleared. D8 and DA are in the code and off by default.",
"trusted": true,
"default": [
"D13",
"D-fill"
],
"runtime": true
}
}
}
@@ -5,6 +5,9 @@
"provides": [ "provides": [
{ {
"name": "secret", "name": "secret",
"credential": {
"form": "pair"
},
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"in": "a vault entry" "in": "a vault entry"
@@ -14,10 +17,27 @@
"emits": [ "emits": [
"provisioned", "provisioned",
"rotated", "rotated",
"secret-changed" "secret-changed",
"account-hashed",
"own-replaced",
"account-issued",
"signed"
], ],
"reads": [ "reads": [
"mesh-controller.assignments" "node-engine.have",
"mesh-grants.accounts",
"mesh-controller.lease",
"mesh-delivery-assigner.placement",
"mesh-provisioner.plans",
"mesh-provisioner.provisions"
],
"state": [
{
"name": "bus-accounts",
"history": 2
},
"signers",
"bound"
], ],
"receives": { "receives": {
"secret": "${dir:grants}/mesh.json" "secret": "${dir:grants}/mesh.json"
@@ -135,7 +155,8 @@
"MESH_VAULT_LEDGER": "${dir:ledger}", "MESH_VAULT_LEDGER": "${dir:ledger}",
"MESH_VAULT_ROOT": "${dir:root}", "MESH_VAULT_ROOT": "${dir:root}",
"MESH_VAULT_STORE": "${dir:store}", "MESH_VAULT_STORE": "${dir:store}",
"MESH_VAULT_STATE": "${dir:state}" "MESH_VAULT_STATE": "${dir:state}",
"MESH_VAULT_SECRET_ASK_CALLERS": "node-tools"
} }
} }
] ]
@@ -145,6 +166,10 @@
"secret_holders", "secret_holders",
"secret_verify", "secret_verify",
"secret_handed", "secret_handed",
"secret_own",
"account_hash",
"account_adopt",
"account_retire",
"secret_asks", "secret_asks",
"secret_vault_export", "secret_vault_export",
"provisioner_retirement", "provisioner_retirement",
@@ -152,7 +177,12 @@
"provisioner_retire_reject", "provisioner_retire_reject",
"provisioner_delete", "provisioner_delete",
"vault_status", "vault_status",
"vault_health" "vault_health",
"own_drift",
"own_take_switch",
"vault_signers",
"vault_signed",
"signers_reassert"
], ],
"claims": [ "claims": [
{ {
@@ -166,7 +196,10 @@
"rotate", "rotate",
"move-in", "move-in",
"node-key", "node-key",
"move-asks" "move-asks",
"own",
"issue-account",
"sign"
] ]
} }
] ]
@@ -3,11 +3,15 @@
"version": "1", "version": "1",
"slug": "watch", "slug": "watch",
"consumes": [ "consumes": [
"mesh-controller.doctor-heartbeat" "mesh-controller.doctor-heartbeat",
"mesh-self-check.checked"
],
"reads": [
"mesh-conditions.takeover",
"mesh-delivery-assigner.seat-holders"
], ],
"invokes": [ "invokes": [
"mesh-watcher.watcher_ping", "mesh-watcher.watcher_ping"
"seat:mesh-controller.seats"
], ],
"own-secrets": { "own-secrets": {
"telegram-token": "${dir:state}/telegram-token" "telegram-token": "${dir:state}/telegram-token"
@@ -39,7 +43,9 @@
"type": "file", "type": "file",
"path": "${dir:state}/settings.json", "path": "${dir:state}/settings.json",
"mode": "0600", "mode": "0600",
"takes-config": ["telegram-chat-id"], "takes-config": [
"telegram-chat-id"
],
"merge": "json", "merge": "json",
"template": "files/settings.json" "template": "files/settings.json"
} }
@@ -77,6 +77,9 @@
"mesh-controller.condition-raised", "mesh-controller.condition-raised",
"mesh-controller.condition-changed", "mesh-controller.condition-changed",
"mesh-controller.condition-cleared", "mesh-controller.condition-cleared",
"mesh-conditions.condition-raised",
"mesh-conditions.condition-changed",
"mesh-conditions.condition-cleared",
"mesh-delivery-view.moving", "mesh-delivery-view.moving",
"mesh-delivery-view.settled", "mesh-delivery-view.settled",
"mesh-delivery-view.stuck", "mesh-delivery-view.stuck",
@@ -90,9 +93,13 @@
], ],
"invokes": [ "invokes": [
"seat:mesh-controller.conditions", "seat:mesh-controller.conditions",
"seat:mesh-controller.root-free", "seat:mesh-conditions.conditions",
"seat:node-engine.root-free",
"seat:issue-tracker.tracking" "seat:issue-tracker.tracking"
], ],
"reads": [
"mesh-conditions.takeover"
],
"state": [ "state": [
"open", "open",
"sent", "sent",
@@ -12,6 +12,9 @@
"identity": { "identity": {
"max": 20, "max": 20,
"in": "an S3 access key" "in": "an S3 access key"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -36,8 +39,8 @@
] ]
} }
], ],
"invokes": [ "reads": [
"seat:mesh-controller.seats" "mesh-delivery-assigner.seat-holders"
], ],
"contributes": { "contributes": {
"route": { "route": {
@@ -12,6 +12,9 @@
"identity": { "identity": {
"max": 63, "max": 63,
"in": "a MongoDB database name" "in": "a MongoDB database name"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -8,6 +8,9 @@
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"in": "a Mosquitto client and topic prefix" "in": "a Mosquitto client and topic prefix"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -5,11 +5,9 @@
"nfs-share" "nfs-share"
], ],
"reads": [ "reads": [
"disk-load.readings",
"nfs-server.exports" "nfs-server.exports"
], ],
"invokes": [
"seat:mesh-controller.data"
],
"capabilities": [ "capabilities": [
"package-manager", "package-manager",
"service-manager" "service-manager"
@@ -12,6 +12,9 @@
"identity": { "identity": {
"max": 128, "max": 128,
"in": "a SQL Server login and database name" "in": "a SQL Server login and database name"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -8,7 +8,10 @@
"provides": [ "provides": [
{ {
"name": "mesh-bus", "name": "mesh-bus",
"scope": "mesh" "scope": "mesh",
"credential": {
"form": "none"
}
} }
], ],
"claims": [ "claims": [
@@ -19,12 +22,16 @@
], ],
"bus-users": "/var/lib/nats-module/conf/accounts.conf", "bus-users": "/var/lib/nats-module/conf/accounts.conf",
"own-secrets": { "own-secrets": {
"broker": "${dir:mesh-state}/broker" "broker": "${dir:mesh-state}/broker",
"broker-nats-tools": "${dir:tools-secret}/broker-nats-tools"
}, },
"capabilities": [ "capabilities": [
"container-runtime" "container-runtime"
], ],
"emits": [], "emits": [
"consumer-removed",
"bucket-keys-removed"
],
"consumes": [], "consumes": [],
"listens": [ "listens": [
{ {
@@ -45,7 +52,9 @@
"nats_buckets", "nats_buckets",
"nats_users", "nats_users",
"nats_user_can", "nats_user_can",
"nats_snapshot_verify" "nats_snapshot_verify",
"nats_consumer_remove",
"nats_bucket_remove"
], ],
"guards": [ "guards": [
8222 8222
@@ -86,6 +95,13 @@
"mode": "0700", "mode": "0700",
"unknown": "program" "unknown": "program"
}, },
{
"id": "tools-secret",
"type": "directory",
"path": "/var/lib/mesh/nats-tools",
"mode": "0711",
"unknown": "program"
},
{ {
"id": "snapshots", "id": "snapshots",
"type": "directory", "type": "directory",
@@ -190,11 +206,18 @@
"loads": [ "loads": [
"nats-tools" "nats-tools"
], ],
"bus": {
"removes": [
"DEAD_LETTER_NOTICES.controller",
"mesh-host_acts"
]
},
"env": { "env": {
"MESH_NATS_MONITOR": "http://127.0.0.1:8222", "MESH_NATS_MONITOR": "http://127.0.0.1:8222",
"MESH_NATS_CONTAINER": "mesh-broker-nats", "MESH_NATS_CONTAINER": "mesh-broker-nats",
"MESH_NATS_USERS_FILE": "/etc/nats/accounts.conf", "MESH_NATS_USERS_FILE": "/etc/nats/accounts.conf",
"MESH_NATS_SNAPSHOTS_DIR": "${dir:snapshots}" "MESH_NATS_SNAPSHOTS_DIR": "${dir:snapshots}",
"NATS_TOOLS_CREDENTIAL": "${dir:tools-secret}/broker-nats-tools"
} }
}, },
{ {
@@ -72,7 +72,7 @@
}, },
"user": { "user": {
"type": "string", "type": "string",
"description": "The Nextcloud account the module's tools act as; empty is admin.", "description": "The Nextcloud account the module's tools act as; empty is mesh-admin, the account the module makes and keeps for itself. Any other account is used as given and never repaired.",
"trusted": true, "trusted": true,
"default": "" "default": ""
} }
@@ -13,7 +13,10 @@
{ {
"name": "nfs-share", "name": "nfs-share",
"scope": "mesh", "scope": "mesh",
"identity": false "identity": false,
"credential": {
"form": "none"
}
} }
], ],
"data": { "data": {
@@ -45,11 +48,9 @@
} }
], ],
"reads": [ "reads": [
"disk-load.readings",
"mesh-delivery-assigner.seat-holders",
"mounts.shares" "mounts.shares"
],
"invokes": [
"seat:mesh-controller.seats",
"seat:mesh-controller.data"
], ],
"config": { "config": {
"shares": { "shares": {
@@ -7,7 +7,10 @@
"provides": [ "provides": [
{ {
"name": "node-toolchain", "name": "node-toolchain",
"reach": "machine" "reach": "machine",
"credential": {
"form": "none"
}
} }
], ],
"resources": [ "resources": [
@@ -7,7 +7,10 @@
"provides": [ "provides": [
{ {
"name": "model-access", "name": "model-access",
"scope": "node" "scope": "node",
"credential": {
"form": "none"
}
} }
], ],
"listens": [ "listens": [
@@ -9,6 +9,7 @@
"provides": [ "provides": [
{ {
"name": "postgres-database", "name": "postgres-database",
"credential": {"form": "pair"},
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"max": 63, "max": 63,
@@ -195,6 +196,7 @@
"bound" "bound"
], ],
"reads": [ "reads": [
"mesh-controller.assignments" "mesh-provisioner.plans",
"mesh-provisioner.provisions"
] ]
} }
@@ -314,6 +314,48 @@
"mode": "0644", "mode": "0644",
"template": "templates/systemd/system/suspend-then-hibernate.target.d/mesh-power.conf" "template": "templates/systemd/system/suspend-then-hibernate.target.d/mesh-power.conf"
}, },
{
"id": "reboot-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/reboot.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "reboot-bound",
"type": "file",
"path": "/etc/systemd/system/reboot.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/reboot.target.d/mesh-power.conf"
},
{
"id": "poweroff-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/poweroff.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "poweroff-bound",
"type": "file",
"path": "/etc/systemd/system/poweroff.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/poweroff.target.d/mesh-power.conf"
},
{
"id": "halt-drop-ins",
"type": "directory",
"path": "/etc/systemd/system/halt.target.d",
"mode": "0755",
"unknown": "program"
},
{
"id": "halt-bound",
"type": "file",
"path": "/etc/systemd/system/halt.target.d/mesh-power.conf",
"mode": "0644",
"template": "templates/systemd/system/halt.target.d/mesh-power.conf"
},
{ {
"id": "udev-rule", "id": "udev-rule",
"type": "file", "type": "file",
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min
@@ -0,0 +1,8 @@
# Written by the mesh (module power, novox/hq ADR 0211; issue 1126). Replaced on every push.
#
# A shutdown is bounded at ten minutes, not systemd's thirty. A unit started again during the shutdown
# (on ace, 2026-10-28, the bus and the authorization manager) is never stopped a second time, and the
# shutdown then waits on this job's timeout with nothing on the console. The target keeps its own
# forcing action: the last phase still ends every process, unmounts and syncs.
[Unit]
JobTimeoutSec=10min
@@ -7,6 +7,9 @@
"scope": "mesh", "scope": "mesh",
"identity": { "identity": {
"in": "a Redis ACL user and key prefix" "in": "a Redis ACL user and key prefix"
},
"credential": {
"form": "pair"
} }
} }
], ],
@@ -139,22 +139,17 @@
"MESH_BACKUP_DATA": "${dir:state}/data.conf" "MESH_BACKUP_DATA": "${dir:state}/data.conf"
}, },
"health": { "health": {
"kind": "tool", "kind": "unit"
"tool": "backup_health",
"interval": "60s",
"timeout": "20s",
"looks": 2,
"grace": "60s"
} }
}, },
{ {
"id": "measure", "id": "awake",
"type": "process", "type": "process",
"name": "restic-measure", "name": "restic-awake",
"artifact": "tools", "artifact": "tools",
"run": [ "run": [
"./restic-backups", "./restic-backups",
"measure" "awake"
], ],
"user": "${machine:account}", "user": "${machine:account}",
"env": { "env": {
@@ -165,7 +160,12 @@
"MESH_BACKUP_DATA": "${dir:state}/data.conf" "MESH_BACKUP_DATA": "${dir:state}/data.conf"
}, },
"health": { "health": {
"kind": "unit" "kind": "tool",
"tool": "backup_health",
"interval": "60s",
"timeout": "20s",
"looks": 2,
"grace": "60s"
} }
} }
], ],
@@ -195,5 +195,11 @@
"from": "files" "from": "files"
} }
] ]
} },
"reads": [
"disk-load.readings"
],
"state": [
"good"
]
} }
@@ -9,6 +9,7 @@
"provides": [ "provides": [
{ {
"name": "route", "name": "route",
"credential": {"form": "none"},
"scope": "mesh", "scope": "mesh",
"identity": false "identity": false
} }
@@ -129,6 +130,7 @@
"bound" "bound"
], ],
"reads": [ "reads": [
"mesh-controller.assignments" "mesh-provisioner.plans",
"mesh-provisioner.provisions"
] ]
} }
@@ -9,7 +9,10 @@
"provides": [ "provides": [
{ {
"name": "greeting", "name": "greeting",
"scope": "mesh" "scope": "mesh",
"credential": {
"form": "none"
}
} }
], ],
"serves": { "serves": {
@@ -7,7 +7,10 @@
"provides": [ "provides": [
{ {
"name": "internal-acme-ca", "name": "internal-acme-ca",
"scope": "mesh" "scope": "mesh",
"credential": {
"form": "none"
}
} }
], ],
"serves": { "serves": {
@@ -8,7 +8,10 @@
"provides": [ "provides": [
{ {
"name": "split-dns", "name": "split-dns",
"reach": "machine" "reach": "machine",
"credential": {
"form": "none"
}
} }
], ],
"requires": [ "requires": [
@@ -32,7 +32,10 @@
"provides": [ "provides": [
{ {
"name": "analytics", "name": "analytics",
"scope": "mesh" "scope": "mesh",
"credential": {
"form": "none"
}
} }
], ],
"serves": { "serves": {
@@ -9,7 +9,10 @@
"provides": [ "provides": [
{ {
"name": "x11-display", "name": "x11-display",
"reach": "machine" "reach": "machine",
"credential": {
"form": "none"
}
} }
], ],
"claims": [ "claims": [