Compare commits
39
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aed7e0c968 | ||
|
|
43df7ebc5a | ||
|
|
c5e1bc7bb4 | ||
|
|
6190b40175 | ||
|
|
edf39a3b74 | ||
|
|
15983e2f96 | ||
|
|
263c86d2e5 | ||
|
|
ec4ab7afe7 | ||
|
|
0e6f382bb7 | ||
|
|
d745a4d31e | ||
|
|
2772120758 | ||
|
|
13a7d40ac5 | ||
|
|
a3ad3c5061 | ||
|
|
5b75499ea6 | ||
|
|
0366208274 | ||
|
|
9f893b6305 | ||
|
|
d1baca9b94 | ||
|
|
3a5c86b5d6 | ||
|
|
befe451f6d | ||
|
|
7ad54356fd | ||
|
|
69f76a53e7 | ||
|
|
b82aa1691c | ||
|
|
7e27259a64 | ||
|
|
3fe19c9c0e | ||
|
|
424fc5a281 | ||
|
|
f067fab9db | ||
|
|
56a9a1b082 | ||
|
|
df94be0e60 | ||
|
|
8f5e8bcb6e | ||
|
|
ca3c3e6dfd | ||
|
|
2051d8b80b | ||
|
|
3e84581927 | ||
|
|
4d9cb0827f | ||
|
|
ac90b35b49 | ||
|
|
f2d765320a | ||
|
|
4fdb047cc8 | ||
|
|
a92c08fd5b | ||
|
|
991ffcfda0 | ||
|
|
2927cb2ceb |
@@ -1,7 +1,8 @@
|
|||||||
# artifact-store-tools
|
# artifact-store-tools
|
||||||
|
|
||||||
Tools that say what the mesh's artifact store holds, and what the controller's records say of it
|
Tools that say what the mesh's artifact store holds, and what the controller's records say of it
|
||||||
(novox/hq ADR 0251 §1–3, to-be 51). It declares no resources and changes nothing on its machine.
|
(novox/hq ADR 0251 §1–3, to-be 51). It changes nothing on its machine; its only resources are its state
|
||||||
|
directory and the file the mesh writes there from its setting `older-store-names`.
|
||||||
|
|
||||||
## Why a module beside the store
|
## Why a module beside the store
|
||||||
|
|
||||||
@@ -18,7 +19,7 @@ on that machine.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `artifact_store_repositories` | r | every repository: tags and what each names, how many manifests (tagged or not), size |
|
| `artifact_store_repositories` | r | every repository: tags and what each names, how many manifests (tagged or not), size |
|
||||||
| `artifact_store_usage` | r | the store's size, the largest repositories, bytes shared between repositories, bytes no manifest marks (what the nightly collector frees next) |
|
| `artifact_store_usage` | r | the store's size, the largest repositories, bytes shared between repositories, bytes no manifest marks (what the nightly collector frees next) |
|
||||||
| `artifact_store_references` | r | what the controller's records say of each manifest, counted and sized by state and repository; each manifest listed when one repository is asked; what the records keep that the store does not hold |
|
| `artifact_store_references` | r | what the controller's records and the catalogue's definitions say of each manifest, counted and sized by state and repository; each manifest listed when one repository is asked; what the records keep that the store does not hold |
|
||||||
| `artifact_store_collect` | a | a dry run unless `dry_run` is false: what the controller would let go of, and the bytes the nightly collector would free then and now. A real run needs `why` and asks the controller's `collect` |
|
| `artifact_store_collect` | a | a dry run unless `dry_run` is false: what the controller would let go of, and the bytes the nightly collector would free then and now. A real run needs `why` and asks the controller's `collect` |
|
||||||
|
|
||||||
### How the store is read
|
### How the store is read
|
||||||
@@ -43,6 +44,7 @@ it lists. The store's collector removes every blob no manifest marks, so these n
|
|||||||
| state | what | removable |
|
| state | what | removable |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `kept` | a definition names it, or one of the five most recent builds of a module the mesh holds; `why` says which | no |
|
| `kept` | a definition names it, or one of the five most recent builds of a module the mesh holds; `why` says which | no |
|
||||||
|
| `named-by-a-definition` | no record names it, and a definition in the catalogue pins it by digest, under the store's current name or an older one; `why` names the module (hq issue 323) | no |
|
||||||
| `holder-of-kept-archive` | the manifest that keeps a kept archive's blob (hq issue 253) | no |
|
| `holder-of-kept-archive` | the manifest that keeps a kept archive's blob (hq issue 253) | no |
|
||||||
| `eligible` | the mesh made it and keeps it for no reason | through the controller's `collect` |
|
| `eligible` | the mesh made it and keeps it for no reason | through the controller's `collect` |
|
||||||
| `holder-of-eligible-archive` | the manifest that keeps an eligible archive's blob | with its archive |
|
| `holder-of-eligible-archive` | the manifest that keeps an eligible archive's blob | with its archive |
|
||||||
@@ -56,6 +58,20 @@ its own right; the controller lets an index go with the platform manifests only
|
|||||||
counted as the index's. Two indexes naming one platform give it the more keeping state. An index no
|
counted as the index's. Two indexes naming one platform give it the more keeping state. An index no
|
||||||
record names leaves its platform manifests unrecorded with it.
|
record names leaves its platform manifests unrecorded with it.
|
||||||
|
|
||||||
|
**What the definitions pin** (hq issue 323). A definition may pin an image of the store the mesh did not
|
||||||
|
make: pushed before the mesh built it, and pinned under the store's current name or a name it had before.
|
||||||
|
No record of the controller names it, yet a machine runs it, so `artifact_store_references` reads every
|
||||||
|
definition the catalogue holds (`mesh-catalog.catalog_modules`, then `catalog_module` for each, eight at
|
||||||
|
a time and within twenty seconds) and finds every `<name>/<repository>@sha256:…` in any of its strings,
|
||||||
|
under each of the store's names: `artifact-store://`, the store's address as the controller answers it,
|
||||||
|
and the names it had before. Those are the mesh's setting `older-store-names` (space-separated, `none` by
|
||||||
|
default, set with the controller's `settings`), which the mesh writes to the module's `store-names` file: a
|
||||||
|
catalogue names no installation. When they cannot be read, the answer's first line says so. A manifest no record
|
||||||
|
names that a definition pins is `named-by-a-definition`; one the records name keeps the records' state. A
|
||||||
|
definition read at its commit is kept, so the next call asks only for the list. Definitions that could
|
||||||
|
not be read are listed under `definitions.unread`, and the answer's first line says that what they pin
|
||||||
|
reads as unrecorded until they are.
|
||||||
|
|
||||||
**Unrecorded copies of upstream images** are listed in `unrecorded_mirrors`: each manifest at the top of
|
**Unrecorded copies of upstream images** are listed in `unrecorded_mirrors`: each manifest at the top of
|
||||||
a repository only the build's mirror writes (`upstream/<host>/<path>`, or a module's `<module>/on-<argument>`
|
a repository only the build's mirror writes (`upstream/<host>/<path>`, or a module's `<module>/on-<argument>`
|
||||||
from before hq ADR 0257) that no record names, with what it and its platforms mark alone. They are the
|
from before hq ADR 0257) that no record names, with what it and its platforms mark alone. They are the
|
||||||
@@ -88,6 +104,8 @@ with slashes, a cut listing refused), manifests read with one absent or unreadab
|
|||||||
digest never becoming a path outside the blobs, the mark (an index, a shared blob, a blob nothing marks),
|
digest never becoming a path outside the blobs, the mark (an index, a shared blob, a blob nothing marks),
|
||||||
every state, what the records keep that the store lacks, bytes freed counting a blob shared with a kept
|
every state, what the records keep that the store lacks, bytes freed counting a blob shared with a kept
|
||||||
manifest as kept, a dry run never confirming, a real run without `why` refused before anything is asked,
|
manifest as kept, a dry run never confirming, a real run without `why` refused before anything is asked,
|
||||||
the fall-back when the references let go of cannot be carried, escalation through `sudo -n`, and that the
|
the fall-back when the references let go of cannot be carried, a manifest a definition pins under the
|
||||||
tools served are exactly the manifest's `tools` and its `invokes` exactly the two controller verbs.
|
store's current or an older name, a definition not read said first, escalation through `sudo -n`, and that the
|
||||||
|
tools served are exactly the manifest's `tools` and its `invokes` exactly the two controller verbs and the
|
||||||
|
catalogue's two tools.
|
||||||
Any command other than the two read-only scripts fails the test.
|
Any command other than the two read-only scripts fails the test.
|
||||||
|
|||||||
@@ -7,12 +7,15 @@ import (
|
|||||||
|
|
||||||
// The states a manifest the store holds can be in, as the records see it (novox/hq to-be 51).
|
// The states a manifest the store holds can be in, as the records see it (novox/hq to-be 51).
|
||||||
const (
|
const (
|
||||||
StateKept = "kept"
|
StateKept = "kept"
|
||||||
StateHolderKept = "holder-of-kept-archive"
|
StateHolderKept = "holder-of-kept-archive"
|
||||||
StateEligible = "eligible"
|
StateEligible = "eligible"
|
||||||
StateHolderEligible = "holder-of-eligible-archive"
|
StateHolderEligible = "holder-of-eligible-archive"
|
||||||
StateCollectedPresent = "let-go-yet-present"
|
StateCollectedPresent = "let-go-yet-present"
|
||||||
StateNamedDocument = "named-document"
|
StateNamedDocument = "named-document"
|
||||||
|
// StateNamedByDefinition is a manifest no record names that a definition pins by digest, under the
|
||||||
|
// store's current name or an older one (novox/hq issue 323): a machine may run it, so it is kept.
|
||||||
|
StateNamedByDefinition = "named-by-a-definition"
|
||||||
StateUnrecorded = "unrecorded"
|
StateUnrecorded = "unrecorded"
|
||||||
collectionRemovesThese = "eligible and holder-of-eligible-archive, through the controller's collect; nothing else"
|
collectionRemovesThese = "eligible and holder-of-eligible-archive, through the controller's collect; nothing else"
|
||||||
)
|
)
|
||||||
@@ -45,8 +48,8 @@ var mirrorRepository = regexp.MustCompile(`^(upstream/[a-z0-9][a-z0-9._/-]*|[a-z
|
|||||||
// keeping orders the states a platform manifest may take from the indexes naming it: the most keeping
|
// keeping orders the states a platform manifest may take from the indexes naming it: the most keeping
|
||||||
// wins, so a platform two indexes name is as kept as the more kept of them.
|
// wins, so a platform two indexes name is as kept as the more kept of them.
|
||||||
var keeping = map[string]int{
|
var keeping = map[string]int{
|
||||||
StateKept: 0, StateNamedDocument: 1, StateHolderKept: 2, StateEligible: 3, StateHolderEligible: 4,
|
StateKept: 0, StateNamedByDefinition: 1, StateNamedDocument: 2, StateHolderKept: 3, StateEligible: 4,
|
||||||
StateCollectedPresent: 5,
|
StateHolderEligible: 5, StateCollectedPresent: 6,
|
||||||
}
|
}
|
||||||
|
|
||||||
// recordIndex finds a record by repository and digest.
|
// recordIndex finds a record by repository and digest.
|
||||||
@@ -64,8 +67,9 @@ func indexRecords(r *Records) recordIndex {
|
|||||||
return idx
|
return idx
|
||||||
}
|
}
|
||||||
|
|
||||||
// Classify gives every manifest the store holds one state.
|
// Classify gives every manifest the store holds one state. A manifest no record names that a definition
|
||||||
func Classify(v *View, recs *Records) []Entry {
|
// pins is named by a definition; defs may be nil, when the definitions were not read.
|
||||||
|
func Classify(v *View, recs *Records, defs *Definitions) []Entry {
|
||||||
idx := indexRecords(recs)
|
idx := indexRecords(recs)
|
||||||
var out []Entry
|
var out []Entry
|
||||||
for _, name := range v.L.RepoNames() {
|
for _, name := range v.L.RepoNames() {
|
||||||
@@ -99,6 +103,12 @@ func Classify(v *View, recs *Records) []Entry {
|
|||||||
e.State = StateNamedDocument
|
e.State = StateNamedDocument
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if e.State == StateUnrecorded && defs != nil {
|
||||||
|
for _, p := range defs.Pins[Key(name, d)] {
|
||||||
|
e.State = StateNamedByDefinition
|
||||||
|
e.Why = append(e.Why, p.why())
|
||||||
|
}
|
||||||
|
}
|
||||||
out = append(out, e)
|
out = append(out, e)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,343 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// What the definitions pin in the store (novox/hq issue 323). A definition may pin an image of the store by
|
||||||
|
// digest that the mesh did not make: one pushed before the mesh built it, pinned under the store's current
|
||||||
|
// name or under a name the store had before. No record of the controller names such a manifest, and yet a
|
||||||
|
// machine runs it, so it is said to be named by a definition and never unrecorded.
|
||||||
|
|
||||||
|
// NamesFileEnv names the file the mesh writes from the setting older-store-names: a line `older=` with the
|
||||||
|
// names the store had before, as a definition may still write them — host names, or host:port, separated by
|
||||||
|
// spaces — or `none`. A catalogue names no installation, so the names are the mesh's setting, not the
|
||||||
|
// manifest's.
|
||||||
|
const NamesFileEnv = "MESH_ARTIFACT_STORE_NAMES_FILE"
|
||||||
|
|
||||||
|
// CatalogueModules and CatalogueModule are the catalogue's tools these definitions are read through.
|
||||||
|
const (
|
||||||
|
CatalogueModules = "mesh-catalog.catalog_modules"
|
||||||
|
CatalogueModule = "mesh-catalog.catalog_module"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DefinitionsTimeout bounds reading every definition, below the runtime's thirty-second call limit; what
|
||||||
|
// was read is kept, so a call that runs out reads only the rest the next time.
|
||||||
|
const DefinitionsTimeout = 20 * time.Second
|
||||||
|
|
||||||
|
// definitionReaders is how many definitions are asked at once.
|
||||||
|
const definitionReaders = 8
|
||||||
|
|
||||||
|
// StoreNames are the names a definition may pin an image of the store under: the scheme the mesh writes
|
||||||
|
// (artifact-store://), the store's address now, and the names it had before.
|
||||||
|
type StoreNames struct {
|
||||||
|
Current []string
|
||||||
|
Older []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// NamesOf is the store's names: its address as the controller answers it, and the older ones given.
|
||||||
|
func NamesOf(address, older string) StoreNames {
|
||||||
|
n := StoreNames{Current: []string{"artifact-store://"}}
|
||||||
|
if a := strings.Trim(strings.TrimSpace(address), "/"); a != "" {
|
||||||
|
n.Current = append(n.Current, a+"/")
|
||||||
|
}
|
||||||
|
for _, o := range strings.FieldsFunc(older, func(r rune) bool { return r == ',' || r == ' ' || r == '\n' || r == '\t' }) {
|
||||||
|
if o = strings.Trim(o, "/"); o != "" && o != "none" {
|
||||||
|
n.Older = append(n.Older, o+"/")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// OlderNames reads the older names from the file the mesh writes, and says why when it cannot.
|
||||||
|
func OlderNames() (string, string) {
|
||||||
|
path := os.Getenv(NamesFileEnv)
|
||||||
|
if path == "" {
|
||||||
|
return "", NamesFileEnv + " is not set, so no older name of the store was read"
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return "", "the store's older names could not be read: " + err.Error()
|
||||||
|
}
|
||||||
|
for _, line := range strings.Split(string(raw), "\n") {
|
||||||
|
if v, ok := strings.CutPrefix(strings.TrimSpace(line), "older="); ok {
|
||||||
|
return v, ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", path + " holds no line older=, so no older name of the store was read"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pin is one definition pinning a manifest of the store by digest.
|
||||||
|
type Pin struct {
|
||||||
|
Module string `json:"module"`
|
||||||
|
// Name is the store's name it is pinned under, as written: "artifact-store://", an address, or an older name.
|
||||||
|
Name string `json:"name"`
|
||||||
|
Older bool `json:"older,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// pinned is one pin found in a manifest: where in the store, and under which name.
|
||||||
|
type pinned struct {
|
||||||
|
Repository, Digest, Name string
|
||||||
|
Older bool
|
||||||
|
}
|
||||||
|
|
||||||
|
var pinShapes sync.Map // name -> *regexp.Regexp
|
||||||
|
|
||||||
|
// pinShape is `<name><repository>@sha256:<hex>`, the name standing on its own: not the tail of a longer host.
|
||||||
|
func pinShape(name string) *regexp.Regexp {
|
||||||
|
if re, ok := pinShapes.Load(name); ok {
|
||||||
|
return re.(*regexp.Regexp)
|
||||||
|
}
|
||||||
|
re := regexp.MustCompile(`(?:^|[^A-Za-z0-9.-])` + regexp.QuoteMeta(name) +
|
||||||
|
`([a-z0-9][a-z0-9._/-]*)@(sha256:[0-9a-f]{64})`)
|
||||||
|
pinShapes.Store(name, re)
|
||||||
|
return re
|
||||||
|
}
|
||||||
|
|
||||||
|
// PinsIn finds every image of the store a manifest pins by digest, in any string it holds: a resource's
|
||||||
|
// image, a build's base, a file's content.
|
||||||
|
func PinsIn(manifest json.RawMessage, names StoreNames) []pinned {
|
||||||
|
var doc any
|
||||||
|
if json.Unmarshal(manifest, &doc) != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
var out []pinned
|
||||||
|
var walk func(any)
|
||||||
|
walk = func(x any) {
|
||||||
|
switch v := x.(type) {
|
||||||
|
case map[string]any:
|
||||||
|
for _, e := range v {
|
||||||
|
walk(e)
|
||||||
|
}
|
||||||
|
case []any:
|
||||||
|
for _, e := range v {
|
||||||
|
walk(e)
|
||||||
|
}
|
||||||
|
case string:
|
||||||
|
for _, group := range []struct {
|
||||||
|
names []string
|
||||||
|
older bool
|
||||||
|
}{{names.Current, false}, {names.Older, true}} {
|
||||||
|
for _, name := range group.names {
|
||||||
|
for _, m := range pinShape(name).FindAllStringSubmatch(v, -1) {
|
||||||
|
p := pinned{Repository: strings.Trim(m[1], "/"), Digest: m[2], Name: name, Older: group.older}
|
||||||
|
if k := p.Repository + "@" + p.Digest + " " + p.Name; !seen[k] {
|
||||||
|
seen[k] = true
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
walk(doc)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Definitions is what the catalogue's definitions pin in the store, and what of them could not be read.
|
||||||
|
type Definitions struct {
|
||||||
|
// Pins are by Key(repository, digest).
|
||||||
|
Pins map[string][]Pin
|
||||||
|
// Read is how many definitions were read; Unread says, per module, why its definition was not.
|
||||||
|
Read int
|
||||||
|
Unread map[string]string
|
||||||
|
// NotAsked says why the catalogue could not be asked at all, when it could not.
|
||||||
|
NotAsked string
|
||||||
|
// NamesNotRead says why the store's older names could not be read, when they could not.
|
||||||
|
NamesNotRead string
|
||||||
|
Names StoreNames
|
||||||
|
}
|
||||||
|
|
||||||
|
// Catalogue reads the definitions through the catalogue's tools, and keeps each definition it read by its
|
||||||
|
// module and commit, which never change together.
|
||||||
|
type Catalogue struct {
|
||||||
|
Ask Asking
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
kept map[string][]pinned // "<module>@<commit>|<names>" -> its pins
|
||||||
|
}
|
||||||
|
|
||||||
|
type catalogueEntry struct {
|
||||||
|
Module string `json:"module"`
|
||||||
|
Commit string `json:"commit"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read asks the catalogue for every module and reads each definition, bounded by DefinitionsTimeout.
|
||||||
|
func (c *Catalogue) Read(ctx context.Context, names StoreNames) *Definitions {
|
||||||
|
defs := &Definitions{Pins: map[string][]Pin{}, Unread: map[string]string{}, Names: names}
|
||||||
|
if c == nil || c.Ask == nil {
|
||||||
|
defs.NotAsked = "this bundle has no way to ask the catalogue"
|
||||||
|
return defs
|
||||||
|
}
|
||||||
|
raw, err := c.Ask(CatalogueModules, map[string]any{})
|
||||||
|
if err != nil {
|
||||||
|
defs.NotAsked = "the catalogue's modules could not be asked: " + err.Error()
|
||||||
|
return defs
|
||||||
|
}
|
||||||
|
answer, output, ok := answerOf(raw)
|
||||||
|
if !ok {
|
||||||
|
defs.NotAsked = "the catalogue refused its modules: " + lastLine(output)
|
||||||
|
return defs
|
||||||
|
}
|
||||||
|
var list struct {
|
||||||
|
Modules []catalogueEntry `json:"modules"`
|
||||||
|
Error string `json:"error"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(answer, &list); err != nil || list.Error != "" || list.Modules == nil {
|
||||||
|
defs.NotAsked = "the catalogue's modules answer is not readable: " + firstOf(list.Error, errText(err), lastLine(output))
|
||||||
|
return defs
|
||||||
|
}
|
||||||
|
c.mu.Lock()
|
||||||
|
if c.kept == nil {
|
||||||
|
c.kept = map[string][]pinned{}
|
||||||
|
}
|
||||||
|
c.mu.Unlock()
|
||||||
|
nameKey := strings.Join(names.Current, ",") + ";" + strings.Join(names.Older, ",")
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, DefinitionsTimeout)
|
||||||
|
defer cancel()
|
||||||
|
type result struct {
|
||||||
|
module string
|
||||||
|
pins []pinned
|
||||||
|
why string
|
||||||
|
}
|
||||||
|
results := make(chan result, len(list.Modules))
|
||||||
|
work := make(chan catalogueEntry)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < definitionReaders; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
for m := range work {
|
||||||
|
pins, why := c.definition(m, names, nameKey)
|
||||||
|
results <- result{m.Module, pins, why}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
defer close(work)
|
||||||
|
for _, m := range list.Modules {
|
||||||
|
select {
|
||||||
|
case work <- m:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
go func() { wg.Wait(); close(results) }()
|
||||||
|
|
||||||
|
got := map[string]bool{}
|
||||||
|
collect:
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case r, open := <-results:
|
||||||
|
if !open {
|
||||||
|
break collect
|
||||||
|
}
|
||||||
|
got[r.module] = true
|
||||||
|
if r.why != "" {
|
||||||
|
defs.Unread[r.module] = r.why
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
defs.Read++
|
||||||
|
for _, p := range r.pins {
|
||||||
|
k := Key(p.Repository, p.Digest)
|
||||||
|
defs.Pins[k] = append(defs.Pins[k], Pin{Module: r.module, Name: p.Name, Older: p.Older})
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
break collect
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, m := range list.Modules {
|
||||||
|
if !got[m.Module] {
|
||||||
|
defs.Unread[m.Module] = fmt.Sprintf("not read within %d s", int(DefinitionsTimeout/time.Second))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for k := range defs.Pins {
|
||||||
|
sort.Slice(defs.Pins[k], func(i, j int) bool { return defs.Pins[k][i].Module < defs.Pins[k][j].Module })
|
||||||
|
}
|
||||||
|
return defs
|
||||||
|
}
|
||||||
|
|
||||||
|
// definition reads one module's definition, or answers it from what was read before.
|
||||||
|
func (c *Catalogue) definition(m catalogueEntry, names StoreNames, nameKey string) ([]pinned, string) {
|
||||||
|
key := m.Module + "@" + m.Commit + "|" + nameKey
|
||||||
|
if m.Commit != "" {
|
||||||
|
c.mu.Lock()
|
||||||
|
pins, had := c.kept[key]
|
||||||
|
c.mu.Unlock()
|
||||||
|
if had {
|
||||||
|
return pins, ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
args := map[string]any{"module": m.Module}
|
||||||
|
if m.Commit != "" {
|
||||||
|
args["commit"] = m.Commit
|
||||||
|
}
|
||||||
|
raw, err := c.Ask(CatalogueModule, args)
|
||||||
|
if err != nil {
|
||||||
|
return nil, "could not be asked: " + err.Error()
|
||||||
|
}
|
||||||
|
answer, output, ok := answerOf(raw)
|
||||||
|
if !ok {
|
||||||
|
return nil, "refused: " + lastLine(output)
|
||||||
|
}
|
||||||
|
var one struct {
|
||||||
|
Module *struct {
|
||||||
|
Manifest json.RawMessage `json:"manifest"`
|
||||||
|
} `json:"module"`
|
||||||
|
Error string `json:"error"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(answer, &one); err != nil || one.Error != "" || one.Module == nil || len(one.Module.Manifest) == 0 {
|
||||||
|
return nil, "not readable: " + firstOf(one.Error, errText(err), "no manifest in the answer")
|
||||||
|
}
|
||||||
|
pins := PinsIn(one.Module.Manifest, names)
|
||||||
|
if m.Commit != "" {
|
||||||
|
c.mu.Lock()
|
||||||
|
c.kept[key] = pins
|
||||||
|
c.mu.Unlock()
|
||||||
|
}
|
||||||
|
return pins, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// why says which definitions name a manifest, and under which of the store's names.
|
||||||
|
func (p Pin) why() string {
|
||||||
|
if p.Older {
|
||||||
|
return fmt.Sprintf("a definition names it: %s, under %s, an older name of the store", p.Module, shownName(p.Name))
|
||||||
|
}
|
||||||
|
return "a definition names it: " + p.Module
|
||||||
|
}
|
||||||
|
|
||||||
|
func firstOf(s ...string) string {
|
||||||
|
for _, x := range s {
|
||||||
|
if strings.TrimSpace(x) != "" {
|
||||||
|
return x
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func errText(err error) string {
|
||||||
|
if err == nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return err.Error()
|
||||||
|
}
|
||||||
|
|
||||||
|
// shownName is a name of the store as a person reads it: a host without the slash a pin puts after it, the
|
||||||
|
// scheme as written.
|
||||||
|
func shownName(name string) string {
|
||||||
|
if strings.HasSuffix(name, "://") {
|
||||||
|
return name
|
||||||
|
}
|
||||||
|
return strings.TrimSuffix(name, "/")
|
||||||
|
}
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A definition may pin an image of the store the mesh did not make, under a name the store had before
|
||||||
|
// (novox/hq issue 323): three apps the anchor runs are pinned so. No record names them; a definition does.
|
||||||
|
|
||||||
|
const oldName = "registry-api.example.org"
|
||||||
|
|
||||||
|
// container is a definition whose one container runs image.
|
||||||
|
func container(image string) string {
|
||||||
|
return fmt.Sprintf(`{"module":"x","resources":[{"id":"server","type":"container","image":%q}]}`, image)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAnImageADefinitionPinsUnderAnOlderNameOfTheStoreIsNamedByADefinition(t *testing.T) {
|
||||||
|
v := view(t, world())
|
||||||
|
names := NamesOf("store:5000", oldName)
|
||||||
|
defs := &Definitions{Pins: map[string][]Pin{}}
|
||||||
|
for module, manifest := range map[string]string{
|
||||||
|
// m-stranger: no record names it; the invoicing app pins it under the store's older name.
|
||||||
|
"invoicing": container(oldName + "/app/server@" + d("m-stranger")),
|
||||||
|
// m-old: the records say eligible; a definition pinning it does not change what the records say.
|
||||||
|
"website": container("store:5000/app/server@" + d("m-old")),
|
||||||
|
} {
|
||||||
|
for _, p := range PinsIn(json.RawMessage(manifest), names) {
|
||||||
|
defs.Pins[Key(p.Repository, p.Digest)] = append(defs.Pins[Key(p.Repository, p.Digest)], Pin{Module: module, Name: p.Name, Older: p.Older})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
got := stateByName(Classify(v, withFill(records()), defs))
|
||||||
|
stranger := got["app/server "+d("m-stranger")]
|
||||||
|
if stranger.State != StateNamedByDefinition {
|
||||||
|
t.Fatalf("a manifest a definition pins under an older name reads %q, want %q", stranger.State, StateNamedByDefinition)
|
||||||
|
}
|
||||||
|
if want := []string{"a definition names it: invoicing, under " + oldName + ", an older name of the store"}; !reflect.DeepEqual(stranger.Why, want) {
|
||||||
|
t.Errorf("why %q, want %q", stranger.Why, want)
|
||||||
|
}
|
||||||
|
if s := got["app/server "+d("m-old")].State; s != StateEligible {
|
||||||
|
t.Errorf("a recorded manifest a definition pins reads %q: the records decide what the mesh made", s)
|
||||||
|
}
|
||||||
|
// Without the definitions it is what it was: unrecorded.
|
||||||
|
if s := stateByName(Classify(v, withFill(records()), nil))["app/server "+d("m-stranger")].State; s != StateUnrecorded {
|
||||||
|
t.Errorf("without definitions %q", s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPinsAreFoundInEveryStringAndOnlyUnderTheStoresNames(t *testing.T) {
|
||||||
|
names := NamesOf("store:5000", oldName+", older.example/")
|
||||||
|
manifest := `{"resources":[
|
||||||
|
{"type":"container","image":"` + oldName + `/team/www@` + d("www") + `"},
|
||||||
|
{"type":"file","content":"IMAGE=older.example/team/tool@` + d("tool") + `\n"},
|
||||||
|
{"type":"container","image":"artifact-store://mod/art@` + d("built") + `"},
|
||||||
|
{"type":"container","image":"store:5000/mod/art@` + d("addressed") + `"},
|
||||||
|
{"type":"container","image":"not` + oldName + `/team/www@` + d("lookalike") + `"},
|
||||||
|
{"type":"container","image":"docker.io/library/nginx@` + d("upstream") + `"},
|
||||||
|
{"type":"container","image":"` + oldName + `/team/www:latest"}],
|
||||||
|
"build":{"on":{"base":"` + oldName + `/team/www@` + d("www") + `"}}}`
|
||||||
|
var got []string
|
||||||
|
for _, p := range PinsIn(json.RawMessage(manifest), names) {
|
||||||
|
got = append(got, fmt.Sprintf("%s@%s %s %v", p.Repository, p.Digest, p.Name, p.Older))
|
||||||
|
}
|
||||||
|
want := map[string]bool{
|
||||||
|
"team/www@" + d("www") + " " + oldName + "/ true": true,
|
||||||
|
"team/tool@" + d("tool") + " older.example/ true": true,
|
||||||
|
"mod/art@" + d("built") + " artifact-store:// false": true,
|
||||||
|
"mod/art@" + d("addressed") + " store:5000/ false": true,
|
||||||
|
}
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Fatalf("found %d pins, want %d (one per image and name; not a longer host's tail, not another registry, not a tag): %v", len(got), len(want), got)
|
||||||
|
}
|
||||||
|
for _, g := range got {
|
||||||
|
if !want[g] {
|
||||||
|
t.Errorf("found %s", g)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPlatformTakesTheStateOfAnIndexADefinitionNames(t *testing.T) {
|
||||||
|
v := mirrorWorld()
|
||||||
|
defs := &Definitions{Pins: map[string][]Pin{
|
||||||
|
Key("route-proxy/on-go_base", d("i-pre")): {{Module: "route-proxy", Name: oldName + "/", Older: true}},
|
||||||
|
}}
|
||||||
|
got := stateByName(Classify(v, withFill(mirrorRecords()), defs))
|
||||||
|
for _, name := range []string{"i-pre", "p-pre"} {
|
||||||
|
if s := got["route-proxy/on-go_base "+d(name)].State; s != StateNamedByDefinition {
|
||||||
|
t.Errorf("%s reads %q, want %q", name, s, StateNamedByDefinition)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if e := got["route-proxy/on-go_base "+d("p-pre")]; e.Index != d("i-pre") {
|
||||||
|
t.Errorf("p-pre takes its state from %q", e.Index)
|
||||||
|
}
|
||||||
|
if s := got["route-proxy/on-go_base "+d("p-lost")].State; s != StateUnrecorded {
|
||||||
|
t.Errorf("a platform nothing names reads %q", s)
|
||||||
|
}
|
||||||
|
for _, um := range UnrecordedMirrors(v, Classify(v, withFill(mirrorRecords()), defs)) {
|
||||||
|
if strings.Contains(um.Reference, hexOf(d("i-pre"))) {
|
||||||
|
t.Errorf("a copy a definition names is offered to the mirrors verb: %v", um)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// olderNames writes the file the mesh writes from the setting older-store-names.
|
||||||
|
func olderNames(t *testing.T, names string) {
|
||||||
|
t.Helper()
|
||||||
|
path := filepath.Join(t.TempDir(), "store-names")
|
||||||
|
if err := os.WriteFile(path, []byte("# written by the mesh\nolder="+names+"\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv(NamesFileEnv, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTheOlderNamesAreReadFromTheFileTheMeshWrites(t *testing.T) {
|
||||||
|
olderNames(t, oldName+" older.example")
|
||||||
|
got, why := OlderNames()
|
||||||
|
if why != "" || !reflect.DeepEqual(NamesOf("", got).Older, []string{oldName + "/", "older.example/"}) {
|
||||||
|
t.Errorf("read %q (%s)", got, why)
|
||||||
|
}
|
||||||
|
olderNames(t, "none")
|
||||||
|
if got, why := OlderNames(); why != "" || len(NamesOf("", got).Older) != 0 {
|
||||||
|
t.Errorf("none read as %q (%s)", got, why)
|
||||||
|
}
|
||||||
|
t.Setenv(NamesFileEnv, filepath.Join(t.TempDir(), "absent"))
|
||||||
|
if _, why := OlderNames(); why == "" {
|
||||||
|
t.Error("an absent file was not said")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOlderNamesNotReadAreSaidFirst(t *testing.T) {
|
||||||
|
t.Setenv(NamesFileEnv, "")
|
||||||
|
a := &asked{definitions: map[string]string{"invoicing": container(oldName + "/app/server@" + d("m-stranger"))}}
|
||||||
|
out, err := references(t, a)(map[string]any{"repository": "app/server"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
m := out.(map[string]any)
|
||||||
|
if w := m["said"].([]string)[0]; !strings.Contains(w, NamesFileEnv) {
|
||||||
|
t.Errorf("first said %q", w)
|
||||||
|
}
|
||||||
|
if s := manifestsOf(t, out)["app/server "+d("m-stranger")].State; s != StateUnrecorded {
|
||||||
|
t.Errorf("m-stranger reads %q without the older names", s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// references is one artifact_store_references tool, kept across calls as the runtime keeps the bundle.
|
||||||
|
func references(t *testing.T, a *asked) func(map[string]any) (any, error) {
|
||||||
|
f := world()
|
||||||
|
s := &Store{Container: "mesh-registry", Run: f.run(t)}
|
||||||
|
for _, tl := range Tools(s, Controller{Ask: a.ask}, &Catalogue{Ask: a.ask}) {
|
||||||
|
if tl.Name == "artifact_store_references" {
|
||||||
|
return tl.Run
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Fatal("no artifact_store_references")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func manifestsOf(t *testing.T, out any) map[string]Entry {
|
||||||
|
t.Helper()
|
||||||
|
entries, ok := out.(map[string]any)["manifests"].([]Entry)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("no manifests in %v", out)
|
||||||
|
}
|
||||||
|
return stateByName(entries)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReferencesReadTheDefinitionsUnderTheStoresOlderNames(t *testing.T) {
|
||||||
|
olderNames(t, oldName)
|
||||||
|
a := &asked{definitions: map[string]string{
|
||||||
|
"invoicing": container(oldName + "/app/server@" + d("m-stranger")),
|
||||||
|
"grafana": container("grafana/grafana@" + d("upstream")),
|
||||||
|
}}
|
||||||
|
run := references(t, a)
|
||||||
|
out, err := run(map[string]any{"repository": "app/server"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
e := manifestsOf(t, out)["app/server "+d("m-stranger")]
|
||||||
|
if e.State != StateNamedByDefinition || len(e.Why) != 1 || !strings.Contains(e.Why[0], "invoicing") || !strings.Contains(e.Why[0], oldName) {
|
||||||
|
t.Errorf("m-stranger: %+v", e)
|
||||||
|
}
|
||||||
|
said := out.(map[string]any)["definitions"].(map[string]any)
|
||||||
|
if said["read"] != 2 || !reflect.DeepEqual(said["older_store_names"], []string{oldName}) ||
|
||||||
|
!reflect.DeepEqual(said["store_names"], []string{"artifact-store://", "store:5000"}) {
|
||||||
|
t.Errorf("definitions said %v", said)
|
||||||
|
}
|
||||||
|
if w := out.(map[string]any)["said"].([]string)[0]; strings.Contains(w, "could not be read") {
|
||||||
|
t.Errorf("a warning with every definition read: %s", w)
|
||||||
|
}
|
||||||
|
// A definition read at its commit is kept: the next call asks only for the list.
|
||||||
|
before := 0
|
||||||
|
for _, c := range a.calls {
|
||||||
|
if c == CatalogueModule {
|
||||||
|
before++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := run(map[string]any{"repository": "app/server"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
after := 0
|
||||||
|
for _, c := range a.calls {
|
||||||
|
if c == CatalogueModule {
|
||||||
|
after++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if before != 2 || after != 2 {
|
||||||
|
t.Errorf("asked catalog_module %d times, then %d: each definition at its commit is read once", before, after)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestADefinitionNotReadIsSaidFirstAndLeavesWhatItPinsUnrecorded(t *testing.T) {
|
||||||
|
olderNames(t, oldName)
|
||||||
|
a := &asked{
|
||||||
|
definitions: map[string]string{"invoicing": container(oldName + "/app/server@" + d("m-stranger")), "grafana": container("x")},
|
||||||
|
failing: map[string]bool{"invoicing": true},
|
||||||
|
}
|
||||||
|
out, err := references(t, a)(map[string]any{"repository": "app/server"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
m := out.(map[string]any)
|
||||||
|
if s := manifestsOf(t, out)["app/server "+d("m-stranger")].State; s != StateUnrecorded {
|
||||||
|
t.Errorf("m-stranger reads %q with its definition unread", s)
|
||||||
|
}
|
||||||
|
if w := m["said"].([]string)[0]; !strings.Contains(w, "1 definitions could not be read") {
|
||||||
|
t.Errorf("first said %q", w)
|
||||||
|
}
|
||||||
|
if u := m["definitions"].(map[string]any)["unread"].(map[string]string); u["invoicing"] == "" || len(u) != 1 {
|
||||||
|
t.Errorf("unread %v", u)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The catalogue not answering at all: the records are still answered, and the warning stands first.
|
||||||
|
a = &asked{noCatalogue: true}
|
||||||
|
out, err = references(t, a)(map[string]any{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
m = out.(map[string]any)
|
||||||
|
if w := m["said"].([]string)[0]; !strings.Contains(w, "the definitions could not be read") || !strings.Contains(w, "no responders") {
|
||||||
|
t.Errorf("first said %q", w)
|
||||||
|
}
|
||||||
|
if m["definitions"].(map[string]any)["not_asked"] == nil {
|
||||||
|
t.Errorf("definitions %v", m["definitions"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoCatalogueIsSaidAndNothingIsAsked(t *testing.T) {
|
||||||
|
defs := (&Catalogue{}).Read(context.Background(), NamesOf("", ""))
|
||||||
|
if defs.NotAsked == "" || defs.Read != 0 {
|
||||||
|
t.Errorf("%+v", defs)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -24,7 +24,7 @@ func main() {
|
|||||||
UID: os.Getuid(),
|
UID: os.Getuid(),
|
||||||
}
|
}
|
||||||
// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): artifact-store-tools.
|
// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): artifact-store-tools.
|
||||||
if err := stdio.Serve("", Tools(s, Controller{Ask: stdio.Ask})); err != nil {
|
if err := stdio.Serve("", Tools(s, Controller{Ask: stdio.Ask}, &Catalogue{Ask: stdio.Ask})); err != nil {
|
||||||
fmt.Fprintln(os.Stderr, err)
|
fmt.Fprintln(os.Stderr, err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -65,7 +65,7 @@ func TestAPlatformManifestTakesTheStateOfTheIndexNamingIt(t *testing.T) {
|
|||||||
for i := range recs.References {
|
for i := range recs.References {
|
||||||
fill(&recs.References[i])
|
fill(&recs.References[i])
|
||||||
}
|
}
|
||||||
got := stateByName(Classify(v, recs))
|
got := stateByName(Classify(v, recs, nil))
|
||||||
up := "upstream/docker.io/library/golang "
|
up := "upstream/docker.io/library/golang "
|
||||||
for name, want := range map[string]string{
|
for name, want := range map[string]string{
|
||||||
up + d("p-amd"): StateEligible, // only the eligible index names it
|
up + d("p-amd"): StateEligible, // only the eligible index names it
|
||||||
@@ -92,7 +92,7 @@ func TestAPlatformManifestTakesTheStateOfTheIndexNamingIt(t *testing.T) {
|
|||||||
if len(mirrors) != 2 || !got["route-proxy/on-go_base "+d("i-pre")].Mirror || !got["route-proxy/on-go_base "+d("p-lost")].Mirror {
|
if len(mirrors) != 2 || !got["route-proxy/on-go_base "+d("i-pre")].Mirror || !got["route-proxy/on-go_base "+d("p-lost")].Mirror {
|
||||||
t.Errorf("offered as unrecorded copies: %v", mirrors)
|
t.Errorf("offered as unrecorded copies: %v", mirrors)
|
||||||
}
|
}
|
||||||
um := UnrecordedMirrors(v, Classify(v, recs))
|
um := UnrecordedMirrors(v, Classify(v, recs, nil))
|
||||||
if len(um) != 2 || um[0].Reference != "artifact-store://route-proxy/on-go_base@"+d("i-pre") || um[0].OnlyTheir != 1+1+1000 {
|
if len(um) != 2 || um[0].Reference != "artifact-store://route-proxy/on-go_base@"+d("i-pre") || um[0].OnlyTheir != 1+1+1000 {
|
||||||
t.Errorf("unrecorded copies: %+v", um)
|
t.Errorf("unrecorded copies: %+v", um)
|
||||||
}
|
}
|
||||||
@@ -104,7 +104,7 @@ func TestLettingAnIndexGoTakesItsOwnPlatformsAndSaysWhatStaysAndNamesOne(t *test
|
|||||||
for i := range recs.References {
|
for i := range recs.References {
|
||||||
fill(&recs.References[i])
|
fill(&recs.References[i])
|
||||||
}
|
}
|
||||||
entries := Classify(v, recs)
|
entries := Classify(v, recs, nil)
|
||||||
old := "artifact-store://upstream/docker.io/library/golang@" + d("i-old")
|
old := "artifact-store://upstream/docker.io/library/golang@" + d("i-old")
|
||||||
gone := LetGoKeys(v, []string{old}, Spared(v, entries))
|
gone := LetGoKeys(v, []string{old}, Spared(v, entries))
|
||||||
up := "upstream/docker.io/library/golang"
|
up := "upstream/docker.io/library/golang"
|
||||||
|
|||||||
@@ -215,7 +215,7 @@ func TestTheMarkIsTheCollectorsIncludingAnIndexAndSharedBlobs(t *testing.T) {
|
|||||||
func TestEveryStateIsGiven(t *testing.T) {
|
func TestEveryStateIsGiven(t *testing.T) {
|
||||||
v := view(t, world())
|
v := view(t, world())
|
||||||
got := map[string]string{}
|
got := map[string]string{}
|
||||||
for _, e := range Classify(v, withFill(records())) {
|
for _, e := range Classify(v, withFill(records()), nil) {
|
||||||
got[e.Repository+" "+e.Digest] = e.State
|
got[e.Repository+" "+e.Digest] = e.State
|
||||||
}
|
}
|
||||||
want := map[string]string{
|
want := map[string]string{
|
||||||
@@ -239,7 +239,7 @@ func TestALetGoReferenceStillPresentIsSaidSo(t *testing.T) {
|
|||||||
v := view(t, world())
|
v := view(t, world())
|
||||||
r := records()
|
r := records()
|
||||||
r.References = append(r.References, Record{Reference: "artifact-store://app/server@" + d("m-stranger"), State: "collected"})
|
r.References = append(r.References, Record{Reference: "artifact-store://app/server@" + d("m-stranger"), State: "collected"})
|
||||||
for _, e := range Classify(v, withFill(r)) {
|
for _, e := range Classify(v, withFill(r), nil) {
|
||||||
if e.Digest == d("m-stranger") && e.State != StateCollectedPresent {
|
if e.Digest == d("m-stranger") && e.State != StateCollectedPresent {
|
||||||
t.Errorf("state %s", e.State)
|
t.Errorf("state %s", e.State)
|
||||||
}
|
}
|
||||||
@@ -270,7 +270,7 @@ func TestBytesFreedCountASharedBlobAsKept(t *testing.T) {
|
|||||||
|
|
||||||
func TestOnlyTheirBytesLeaveOutWhatAnotherStateMarks(t *testing.T) {
|
func TestOnlyTheirBytesLeaveOutWhatAnotherStateMarks(t *testing.T) {
|
||||||
v := view(t, world())
|
v := view(t, world())
|
||||||
entries := Classify(v, withFill(records()))
|
entries := Classify(v, withFill(records()), nil)
|
||||||
sums := Summarise(v, entries, entries)
|
sums := Summarise(v, entries, entries)
|
||||||
el := sums[StateEligible]
|
el := sums[StateEligible]
|
||||||
if el.OnlyTheirBytes != 500+v.L.Blobs[d("m-old")] {
|
if el.OnlyTheirBytes != 500+v.L.Blobs[d("m-old")] {
|
||||||
@@ -284,6 +284,11 @@ type asked struct {
|
|||||||
calls []string
|
calls []string
|
||||||
args []map[string]any
|
args []map[string]any
|
||||||
fail map[string]bool
|
fail map[string]bool
|
||||||
|
// definitions are the catalogue's modules and their manifests; failing, the modules whose
|
||||||
|
// definition cannot be read; noCatalogue, the catalogue not answering at all.
|
||||||
|
definitions map[string]string
|
||||||
|
failing map[string]bool
|
||||||
|
noCatalogue bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func (a *asked) ask(key string, body any) (json.RawMessage, error) {
|
func (a *asked) ask(key string, body any) (json.RawMessage, error) {
|
||||||
@@ -299,6 +304,31 @@ func (a *asked) ask(key string, body any) (json.RawMessage, error) {
|
|||||||
switch key {
|
switch key {
|
||||||
case "seat:mesh-controller.artifacts":
|
case "seat:mesh-controller.artifacts":
|
||||||
answer = records()
|
answer = records()
|
||||||
|
case CatalogueModules:
|
||||||
|
if a.noCatalogue {
|
||||||
|
return nil, fmt.Errorf("no responders")
|
||||||
|
}
|
||||||
|
ms := []map[string]any{}
|
||||||
|
for name := range a.definitions {
|
||||||
|
ms = append(ms, map[string]any{"module": name, "commit": "c-" + name, "path": "modules/" + name})
|
||||||
|
}
|
||||||
|
answer = map[string]any{"modules": ms}
|
||||||
|
case CatalogueModule:
|
||||||
|
name, _ := args["module"].(string)
|
||||||
|
if a.failing[name] {
|
||||||
|
return nil, fmt.Errorf("no answer")
|
||||||
|
}
|
||||||
|
if args["commit"] != "c-"+name {
|
||||||
|
return nil, fmt.Errorf("asked %s at %v", name, args["commit"])
|
||||||
|
}
|
||||||
|
manifest, has := a.definitions[name]
|
||||||
|
if !has {
|
||||||
|
answer = map[string]any{"error": "the catalogue holds no " + name}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// The catalogue's tool answers through the runtime as the protocol's content list.
|
||||||
|
raw, _ := json.Marshal(map[string]any{"module": map[string]any{"module": name, "manifest": json.RawMessage(manifest)}})
|
||||||
|
return json.Marshal(map[string]any{"content": []map[string]any{{"type": "text", "text": string(raw)}}})
|
||||||
case "seat:mesh-controller.collect":
|
case "seat:mesh-controller.collect":
|
||||||
if args["confirm"] == "true" {
|
if args["confirm"] == "true" {
|
||||||
answer = CollectAnswer{LetGo: []string{"artifact-store://app/server@" + d("m-old")}}
|
answer = CollectAnswer{LetGo: []string{"artifact-store://app/server@" + d("m-old")}}
|
||||||
@@ -315,7 +345,7 @@ func (a *asked) ask(key string, body any) (json.RawMessage, error) {
|
|||||||
func tool(t *testing.T, a *asked, name string) func(map[string]any) (any, error) {
|
func tool(t *testing.T, a *asked, name string) func(map[string]any) (any, error) {
|
||||||
f := world()
|
f := world()
|
||||||
s := &Store{Container: "mesh-registry", Run: f.run(t)}
|
s := &Store{Container: "mesh-registry", Run: f.run(t)}
|
||||||
for _, tl := range Tools(s, Controller{Ask: a.ask}) {
|
for _, tl := range Tools(s, Controller{Ask: a.ask}, &Catalogue{Ask: a.ask}) {
|
||||||
if tl.Name == name {
|
if tl.Name == name {
|
||||||
return tl.Run
|
return tl.Run
|
||||||
}
|
}
|
||||||
@@ -386,7 +416,7 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
|
|||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
var served []string
|
var served []string
|
||||||
for _, tl := range Tools(&Store{}, Controller{}) {
|
for _, tl := range Tools(&Store{}, Controller{}, &Catalogue{}) {
|
||||||
if !strings.HasPrefix(tl.Name, "artifact_store_") || tl.Description == "" || tl.Run == nil {
|
if !strings.HasPrefix(tl.Name, "artifact_store_") || tl.Description == "" || tl.Run == nil {
|
||||||
t.Errorf("tool %q", tl.Name)
|
t.Errorf("tool %q", tl.Name)
|
||||||
}
|
}
|
||||||
@@ -398,7 +428,7 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
|
|||||||
if !reflect.DeepEqual(served, listed) {
|
if !reflect.DeepEqual(served, listed) {
|
||||||
t.Fatalf("served %v, manifest %v", served, listed)
|
t.Fatalf("served %v, manifest %v", served, listed)
|
||||||
}
|
}
|
||||||
want := []string{"seat:mesh-controller.artifacts", "seat:mesh-controller.collect"}
|
want := []string{"seat:mesh-controller.artifacts", "seat:mesh-controller.collect", CatalogueModules, CatalogueModule}
|
||||||
if !reflect.DeepEqual(m.Invokes, want) {
|
if !reflect.DeepEqual(m.Invokes, want) {
|
||||||
t.Errorf("invokes %v", m.Invokes)
|
t.Errorf("invokes %v", m.Invokes)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
|
|
||||||
// Tools are the artifact store's tools (novox/hq ADR 0251 §1). The first three only read; artifact_store_collect
|
// Tools are the artifact store's tools (novox/hq ADR 0251 §1). The first three only read; artifact_store_collect
|
||||||
// only reads too, unless it is a real run, and then it asks the controller, which decides and deletes.
|
// only reads too, unless it is a real run, and then it asks the controller, which decides and deletes.
|
||||||
func Tools(s *Store, c Controller) []stdio.Tool {
|
func Tools(s *Store, c Controller, cat *Catalogue) []stdio.Tool {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
repoArg := map[string]any{"type": "string", "description": "one repository, as the store names it (<module>/<artifact>)"}
|
repoArg := map[string]any{"type": "string", "description": "one repository, as the store names it (<module>/<artifact>)"}
|
||||||
return []stdio.Tool{
|
return []stdio.Tool{
|
||||||
@@ -53,7 +53,9 @@ func Tools(s *Store, c Controller) []stdio.Tool {
|
|||||||
{
|
{
|
||||||
Name: "artifact_store_references",
|
Name: "artifact_store_references",
|
||||||
Description: "What the controller's records say of each manifest the artifact store holds: kept (a definition " +
|
Description: "What the controller's records say of each manifest the artifact store holds: kept (a definition " +
|
||||||
"names it, or one of the five most recent builds of a module the mesh holds), the holder of a kept archive, " +
|
"names it, or one of the five most recent builds of a module the mesh holds), named by a definition (no " +
|
||||||
|
"record names it, and a definition in the catalogue pins it by digest, under the store's current name or " +
|
||||||
|
"an older one: the module is said), the holder of a kept archive, " +
|
||||||
"eligible (the mesh made it and keeps it for no reason), the holder of an eligible archive, let go yet present, " +
|
"eligible (the mesh made it and keeps it for no reason), the holder of an eligible archive, let go yet present, " +
|
||||||
"a named document the controller keeps under a tag, or unrecorded — no record names it. A platform manifest " +
|
"a named document the controller keeps under a tag, or unrecorded — no record names it. A platform manifest " +
|
||||||
"takes the state of the recorded index naming it. Counted and sized by " +
|
"takes the state of the recorded index naming it. Counted and sized by " +
|
||||||
@@ -67,11 +69,19 @@ func Tools(s *Store, c Controller) []stdio.Tool {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
// The definitions are read while the store is: each bounded below the runtime's call limit.
|
||||||
|
read := make(chan *Definitions, 1)
|
||||||
|
go func() {
|
||||||
|
older, why := OlderNames()
|
||||||
|
defs := cat.Read(ctx, NamesOf(recs.Store, older))
|
||||||
|
defs.NamesNotRead = why
|
||||||
|
read <- defs
|
||||||
|
}()
|
||||||
v, err := s.View(ctx)
|
v, err := s.View(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return References(v, recs, repo)
|
return References(v, recs, <-read, repo)
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -116,7 +126,7 @@ func Tools(s *Store, c Controller) []stdio.Tool {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return map[string]any{"controller": answer, "bytes_not_worked_out": "the records could not be read: " + err.Error()}, nil
|
return map[string]any{"controller": answer, "bytes_not_worked_out": "the records could not be read: " + err.Error()}, nil
|
||||||
}
|
}
|
||||||
return Collected(v, answer, dry, Classify(v, recs)), nil
|
return Collected(v, answer, dry, Classify(v, recs, nil)), nil
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -225,11 +235,11 @@ func Usage(v *View, top int) map[string]any {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// References answers artifact_store_references.
|
// References answers artifact_store_references.
|
||||||
func References(v *View, recs *Records, only string) (map[string]any, error) {
|
func References(v *View, recs *Records, defs *Definitions, only string) (map[string]any, error) {
|
||||||
if only != "" && v.L.Repos[only] == nil {
|
if only != "" && v.L.Repos[only] == nil {
|
||||||
return nil, fmt.Errorf("the store holds no repository %q", only)
|
return nil, fmt.Errorf("the store holds no repository %q", only)
|
||||||
}
|
}
|
||||||
entries := Classify(v, recs)
|
entries := Classify(v, recs, defs)
|
||||||
byRepo := map[string][]Entry{}
|
byRepo := map[string][]Entry{}
|
||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
byRepo[e.Repository] = append(byRepo[e.Repository], e)
|
byRepo[e.Repository] = append(byRepo[e.Repository], e)
|
||||||
@@ -243,7 +253,7 @@ func References(v *View, recs *Records, only string) (map[string]any, error) {
|
|||||||
}
|
}
|
||||||
whole := Summarise(v, entries, entries)
|
whole := Summarise(v, entries, entries)
|
||||||
said := []string{}
|
said := []string{}
|
||||||
for _, state := range []string{StateKept, StateHolderKept, StateEligible, StateHolderEligible, StateCollectedPresent, StateNamedDocument, StateUnrecorded} {
|
for _, state := range []string{StateKept, StateNamedByDefinition, StateHolderKept, StateEligible, StateHolderEligible, StateCollectedPresent, StateNamedDocument, StateUnrecorded} {
|
||||||
if s := whole[state]; s != nil {
|
if s := whole[state]; s != nil {
|
||||||
said = append(said, fmt.Sprintf("%s: %d manifests marking %s, %s of it marked by nothing else", state, s.Manifests, human(s.Bytes), human(s.OnlyTheirBytes)))
|
said = append(said, fmt.Sprintf("%s: %d manifests marking %s, %s of it marked by nothing else", state, s.Manifests, human(s.Bytes), human(s.OnlyTheirBytes)))
|
||||||
}
|
}
|
||||||
@@ -276,9 +286,57 @@ func References(v *View, recs *Records, only string) (map[string]any, error) {
|
|||||||
if u := unreadNote(v); u != nil {
|
if u := unreadNote(v); u != nil {
|
||||||
out["unread"] = u
|
out["unread"] = u
|
||||||
}
|
}
|
||||||
|
out["definitions"] = definitionsSaid(defs)
|
||||||
|
if warn := definitionsWarning(defs); warn != "" {
|
||||||
|
// Said first: what it leaves out is a manifest a machine may run, read as unrecorded.
|
||||||
|
out["said"] = append([]string{warn}, said...)
|
||||||
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// definitionsSaid is how the definitions were read: how many, under which names, and what was not read.
|
||||||
|
func definitionsSaid(defs *Definitions) map[string]any {
|
||||||
|
if defs == nil {
|
||||||
|
return map[string]any{"read": 0, "not_asked": "the definitions were not read"}
|
||||||
|
}
|
||||||
|
var current, older []string
|
||||||
|
for _, n := range defs.Names.Current {
|
||||||
|
current = append(current, shownName(n))
|
||||||
|
}
|
||||||
|
for _, n := range defs.Names.Older {
|
||||||
|
older = append(older, shownName(n))
|
||||||
|
}
|
||||||
|
out := map[string]any{"read": defs.Read, "store_names": nonNil(current), "older_store_names": nonNil(older)}
|
||||||
|
if defs.NotAsked != "" {
|
||||||
|
out["not_asked"] = defs.NotAsked
|
||||||
|
}
|
||||||
|
if defs.NamesNotRead != "" {
|
||||||
|
out["older_store_names_not_read"] = defs.NamesNotRead
|
||||||
|
}
|
||||||
|
if len(defs.Unread) > 0 {
|
||||||
|
out["unread"] = defs.Unread
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// definitionsWarning says, when any definition was not read, that a manifest it pins reads as unrecorded.
|
||||||
|
func definitionsWarning(defs *Definitions) string {
|
||||||
|
switch {
|
||||||
|
case defs == nil:
|
||||||
|
return ""
|
||||||
|
case defs.NotAsked != "":
|
||||||
|
return "the definitions could not be read (" + defs.NotAsked + "), so a manifest a definition pins " +
|
||||||
|
"and no record names reads as unrecorded here: ask again before reading unrecorded as unused"
|
||||||
|
case defs.NamesNotRead != "":
|
||||||
|
return defs.NamesNotRead + ", so a manifest a definition pins under an older name of the store and no " +
|
||||||
|
"record names reads as unrecorded here"
|
||||||
|
case len(defs.Unread) > 0:
|
||||||
|
return fmt.Sprintf("%d definitions could not be read (listed in definitions.unread), so a manifest one "+
|
||||||
|
"of them pins and no record names reads as unrecorded here: ask again — what was read is kept", len(defs.Unread))
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
// Collected answers artifact_store_collect: the controller's answer, and what the store's collector frees.
|
// Collected answers artifact_store_collect: the controller's answer, and what the store's collector frees.
|
||||||
func Collected(v *View, a *CollectAnswer, dry bool, entries []Entry) map[string]any {
|
func Collected(v *View, a *CollectAnswer, dry bool, entries []Entry) map[string]any {
|
||||||
refs := a.LetGo
|
refs := a.LetGo
|
||||||
|
|||||||
@@ -3,7 +3,9 @@
|
|||||||
"version": "1",
|
"version": "1",
|
||||||
"invokes": [
|
"invokes": [
|
||||||
"seat:mesh-controller.artifacts",
|
"seat:mesh-controller.artifacts",
|
||||||
"seat:mesh-controller.collect"
|
"seat:mesh-controller.collect",
|
||||||
|
"mesh-catalog.catalog_modules",
|
||||||
|
"mesh-catalog.catalog_module"
|
||||||
],
|
],
|
||||||
"tools": [
|
"tools": [
|
||||||
"artifact_store_repositories",
|
"artifact_store_repositories",
|
||||||
@@ -11,6 +13,29 @@
|
|||||||
"artifact_store_references",
|
"artifact_store_references",
|
||||||
"artifact_store_collect"
|
"artifact_store_collect"
|
||||||
],
|
],
|
||||||
|
"settings": {
|
||||||
|
"older-store-names": {
|
||||||
|
"kind": "preference",
|
||||||
|
"default": "none",
|
||||||
|
"why": "the names the artifact store had before, as a definition may still pin an image under them: host names or host:port, separated by spaces; none when it had no other. artifact_store_references reads every definition under these as well as under the store's current name, so an image a definition pins under one is said to be named by a definition (novox/hq issue 323)"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"resources": [
|
||||||
|
{
|
||||||
|
"id": "state",
|
||||||
|
"type": "directory",
|
||||||
|
"mode": "0700",
|
||||||
|
"place": "."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "store-names",
|
||||||
|
"type": "file",
|
||||||
|
"path": "${dir:state}/store-names",
|
||||||
|
"mode": "0600",
|
||||||
|
"trusted": false,
|
||||||
|
"content": "# The names the artifact store had before, from the setting older-store-names, written by the mesh.\n# artifact_store_references reads every definition under these too (novox/hq issue 323).\n# Replaced on every push; change the setting, never this file.\nolder=${setting:older-store-names}\n"
|
||||||
|
}
|
||||||
|
],
|
||||||
"build": {
|
"build": {
|
||||||
"artifacts": [
|
"artifacts": [
|
||||||
{
|
{
|
||||||
@@ -24,7 +49,8 @@
|
|||||||
"artifact-store-tools"
|
"artifact-store-tools"
|
||||||
],
|
],
|
||||||
"env": {
|
"env": {
|
||||||
"MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry"
|
"MESH_ARTIFACT_STORE_CONTAINER": "mesh-registry",
|
||||||
|
"MESH_ARTIFACT_STORE_NAMES_FILE": "${dir:state}/store-names"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
|||||||
+18
-8
@@ -45,28 +45,38 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
|
|||||||
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
|
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
|
||||||
|
|
||||||
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
|
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
|
||||||
never the repository's default branch, never a branch a protection rule names (a glob rule covers by the same
|
never the repository's default branch, never a branch a protection rule names. A rule covers a branch as the
|
||||||
matcher as `guard.ts`). The rules are read before anything is sent; rules that cannot be read refuse the
|
forge itself decides it (Gitea 1.27, novox/hq issue 539): a plain name covers that branch in any case; a glob
|
||||||
|
(`*`, `**`, `?`, `[abc]`, `[!abc]`, `{a,b}`, `\` escaping) covers case-sensitively with `/` as separator; a glob
|
||||||
|
the forge cannot compile covers its own text. `cmd/forge/rules.go` and `guard.ts` hold this one rule. The rules are read before anything is sent; rules that cannot be read refuse the
|
||||||
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
|
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
|
||||||
required checks green, through `gitea_merge_pull_request`.
|
required checks green, through `gitea_merge_pull_request`.
|
||||||
|
|
||||||
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
|
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
|
||||||
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
|
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
|
||||||
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. A `patch` goes
|
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. The blobs are read at
|
||||||
to the forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than
|
the commit the branch (or, for a new branch, its base) was read at, never at its name. A `patch` goes to the
|
||||||
the file. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than the
|
||||||
|
file. Neither endpoint takes a commit to build on, only a branch (Gitea 1.27), so the parent answered is the
|
||||||
|
one the forge says it made the commit on; when the branch moved between the read and the commit, the answer
|
||||||
|
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
||||||
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
|
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
|
||||||
branch not at the commit the forge named is a failure, never a success.
|
branch not at the commit the forge named is a failure, never a success.
|
||||||
|
|
||||||
**Code search** has no endpoint on the forge, so the ref's archive (`/archive/{ref}.tar.gz`, at most 64 MiB)
|
**Code search** has no endpoint on the forge, so the ref's archive (`/archive/{ref}.tar.gz`, at most 64 MiB)
|
||||||
is read once and searched in memory; a binary file is passed over, a file over 4 MiB is named under
|
is read once and searched in memory; a binary file is passed over, a file over 4 MiB is named under
|
||||||
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found.
|
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found. A
|
||||||
|
search that reaches the call's deadline stops there and answers what it found, with `stopped_at_deadline`.
|
||||||
|
|
||||||
It reaches the forge as the npm-registry bundle does (below), and one tool call takes at most 50 seconds.
|
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), and one
|
||||||
|
tool call takes at most 50 seconds.
|
||||||
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
|
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
|
||||||
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
|
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
|
||||||
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
|
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
|
||||||
commit failing; every malformed ask refused; and each read summarised.
|
commit failing; a branch or base that moved under the commit said, with the parent the forge used; a file
|
||||||
|
changed under the commit refused; a commit over its size limits refused; protection globs matched as the
|
||||||
|
forge matches them; a code search stopped at its deadline; every malformed ask refused; and each read
|
||||||
|
summarised.
|
||||||
|
|
||||||
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
||||||
|
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/url"
|
||||||
"strings"
|
"strings"
|
||||||
"unicode/utf8"
|
"unicode/utf8"
|
||||||
)
|
)
|
||||||
@@ -52,8 +53,14 @@ type CommittedFile struct {
|
|||||||
Operation string `json:"operation"`
|
Operation string `json:"operation"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// maxFiles is the most files one commit carries.
|
// What one commit carries at most (novox/hq issue 539): files, one file's content, one patch, and every
|
||||||
const maxFiles = 200
|
// file's content together, in bytes as committed (content_base64 decoded). Larger work is several commits.
|
||||||
|
const (
|
||||||
|
maxFiles = 200
|
||||||
|
maxFileBytes = 4 << 20
|
||||||
|
maxPatchBytes = 4 << 20
|
||||||
|
maxCommitBytes = 16 << 20
|
||||||
|
)
|
||||||
|
|
||||||
// readCommitAsk reads the tool's arguments.
|
// readCommitAsk reads the tool's arguments.
|
||||||
func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||||
@@ -109,8 +116,11 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
|||||||
return a, refuse("give files or patch, not both: one commit carries one of them. Nothing was sent")
|
return a, refuse("give files or patch, not both: one commit carries one of them. Nothing was sent")
|
||||||
case len(a.Files) > maxFiles:
|
case len(a.Files) > maxFiles:
|
||||||
return a, refuse("%d files is more than the %d one commit carries. Nothing was sent", len(a.Files), maxFiles)
|
return a, refuse("%d files is more than the %d one commit carries. Nothing was sent", len(a.Files), maxFiles)
|
||||||
|
case len(a.Patch) > maxPatchBytes:
|
||||||
|
return a, refuse("the patch is %s, more than the %s one commit carries: split it into several commits. Nothing was sent", mib(len(a.Patch)), mib(maxPatchBytes))
|
||||||
}
|
}
|
||||||
seen := map[string]bool{}
|
seen := map[string]bool{}
|
||||||
|
total := 0
|
||||||
for i, fc := range a.Files {
|
for i, fc := range a.Files {
|
||||||
if strings.HasPrefix(fc.Path, "/") {
|
if strings.HasPrefix(fc.Path, "/") {
|
||||||
return a, refuse("files[%d].path %q is relative to the repository's root, without a leading slash. Nothing was sent", i, fc.Path)
|
return a, refuse("files[%d].path %q is relative to the repository's root, without a leading slash. Nothing was sent", i, fc.Path)
|
||||||
@@ -134,15 +144,35 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
|||||||
if fc.Content != nil && !utf8.ValidString(*fc.Content) {
|
if fc.Content != nil && !utf8.ValidString(*fc.Content) {
|
||||||
return a, refuse("files[%d].content (%s) is not UTF-8 text: give it as content_base64. Nothing was sent", i, fc.Path)
|
return a, refuse("files[%d].content (%s) is not UTF-8 text: give it as content_base64. Nothing was sent", i, fc.Path)
|
||||||
}
|
}
|
||||||
if fc.ContentBase64 != nil {
|
size := 0
|
||||||
if _, err := base64.StdEncoding.DecodeString(*fc.ContentBase64); err != nil {
|
switch {
|
||||||
|
case fc.Content != nil:
|
||||||
|
size = len(*fc.Content)
|
||||||
|
case fc.ContentBase64 != nil:
|
||||||
|
if base64.StdEncoding.DecodedLen(len(*fc.ContentBase64)) > maxFileBytes+3 {
|
||||||
|
size = base64.StdEncoding.DecodedLen(len(*fc.ContentBase64))
|
||||||
|
break
|
||||||
|
}
|
||||||
|
raw, err := base64.StdEncoding.DecodeString(*fc.ContentBase64)
|
||||||
|
if err != nil {
|
||||||
return a, refuse("files[%d].content_base64 (%s) is not base64: %v. Nothing was sent", i, fc.Path, err)
|
return a, refuse("files[%d].content_base64 (%s) is not base64: %v. Nothing was sent", i, fc.Path, err)
|
||||||
}
|
}
|
||||||
|
size = len(raw)
|
||||||
}
|
}
|
||||||
|
if size > maxFileBytes {
|
||||||
|
return a, refuse("files[%d] (%s) is %s, more than the %s one file of a commit may be. Nothing was sent", i, fc.Path, mib(size), mib(maxFileBytes))
|
||||||
|
}
|
||||||
|
total += size
|
||||||
|
}
|
||||||
|
if total > maxCommitBytes {
|
||||||
|
return a, refuse("the files are %s together, more than the %s one commit carries: split them into several commits. Nothing was sent", mib(total), mib(maxCommitBytes))
|
||||||
}
|
}
|
||||||
return a, nil
|
return a, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// mib says a size in bytes in MiB, to a tenth.
|
||||||
|
func mib(n int) string { return fmt.Sprintf("%.1f MiB", float64(n)/(1<<20)) }
|
||||||
|
|
||||||
// targetRefused says why a commit may not go on a branch: the default branch, or one a protection rule names.
|
// targetRefused says why a commit may not go on a branch: the default branch, or one a protection rule names.
|
||||||
func targetRefused(target, defaultBranch string, rules []Protection) error {
|
func targetRefused(target, defaultBranch string, rules []Protection) error {
|
||||||
if target == defaultBranch {
|
if target == defaultBranch {
|
||||||
@@ -172,7 +202,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
var r Repo
|
var r Repo
|
||||||
if err := f.getJSON(ctx, repo, &r); err != nil {
|
if err := f.GetJSON(ctx, repo, &r); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if r.DefaultBranch == "" {
|
if r.DefaultBranch == "" {
|
||||||
@@ -180,7 +210,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
}
|
}
|
||||||
// Read the rules, or refuse: a branch the rules could not be read for is never written (fails closed).
|
// Read the rules, or refuse: a branch the rules could not be read for is never written (fails closed).
|
||||||
var rules []Protection
|
var rules []Protection
|
||||||
if err := f.getJSON(ctx, repo+"/branch_protections", &rules); err != nil {
|
if err := f.GetJSON(ctx, repo+"/branch_protections", &rules); err != nil {
|
||||||
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
|
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
|
||||||
}
|
}
|
||||||
if err := targetRefused(a.Branch, r.DefaultBranch, rules); err != nil {
|
if err := targetRefused(a.Branch, r.DefaultBranch, rules); err != nil {
|
||||||
@@ -192,9 +222,13 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
ref := a.Branch // where the files are read and the commit goes
|
// The files are judged at the commit just read, never at a branch name that could move between two
|
||||||
|
// reads (novox/hq issue 539): a file the branch changes meanwhile is refused by the forge, whose update
|
||||||
|
// and delete carry the blob read here.
|
||||||
|
var ref string
|
||||||
if existing != nil {
|
if existing != nil {
|
||||||
out.Parent = existing.Commit.ID
|
out.Parent = existing.Commit.ID
|
||||||
|
ref = existing.Commit.ID
|
||||||
if a.Base != "" {
|
if a.Base != "" {
|
||||||
out.Said = append(out.Said, fmt.Sprintf("%s exists, at %s: base %s was not used", a.Branch, short(existing.Commit.ID), a.Base))
|
out.Said = append(out.Said, fmt.Sprintf("%s exists, at %s: base %s was not used", a.Branch, short(existing.Commit.ID), a.Base))
|
||||||
}
|
}
|
||||||
@@ -204,7 +238,6 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
if out.Base == "" {
|
if out.Base == "" {
|
||||||
out.Base = r.DefaultBranch
|
out.Base = r.DefaultBranch
|
||||||
}
|
}
|
||||||
ref = out.Base
|
|
||||||
b, err := f.branch(ctx, repo, out.Base)
|
b, err := f.branch(ctx, repo, out.Base)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -213,8 +246,6 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
return nil, refuse("%s does not exist, and its base %s is not a branch of %s/%s. Nothing was sent", a.Branch, out.Base, a.Owner, a.Repo)
|
return nil, refuse("%s does not exist, and its base %s is not a branch of %s/%s. Nothing was sent", a.Branch, out.Base, a.Owner, a.Repo)
|
||||||
}
|
}
|
||||||
out.Parent = b.Commit.ID
|
out.Parent = b.Commit.ID
|
||||||
// The files are judged at the commit just read; a file the base changes meanwhile is refused by the
|
|
||||||
// forge, whose update and delete carry the blob read here.
|
|
||||||
ref = b.Commit.ID
|
ref = b.Commit.ID
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -227,14 +258,14 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
on["branch"] = a.Branch
|
on["branch"] = a.Branch
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The forge takes no commit to build on, only a branch (Gitea 1.27: neither /contents nor /diffpatch
|
||||||
|
// carries a last commit), so the parent is read from what it made, never assumed (novox/hq issue 539).
|
||||||
var made struct {
|
var made struct {
|
||||||
Commit struct {
|
Commit madeCommit `json:"commit"`
|
||||||
SHA string `json:"sha"`
|
|
||||||
} `json:"commit"`
|
|
||||||
}
|
}
|
||||||
if a.Patch != "" {
|
if a.Patch != "" {
|
||||||
on["content"] = a.Patch
|
on["content"] = a.Patch
|
||||||
if err := f.sendJSON(ctx, http.MethodPost, repo+"/diffpatch", on, &made); err != nil {
|
if err := f.SendJSON(ctx, http.MethodPost, repo+"/diffpatch", on, &made); err != nil {
|
||||||
return nil, fmt.Errorf("the forge did not take the patch: %w", err)
|
return nil, fmt.Errorf("the forge did not take the patch: %w", err)
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
@@ -266,7 +297,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
out.Files = append(out.Files, CommittedFile{Path: fc.Path, Operation: op["operation"].(string)})
|
out.Files = append(out.Files, CommittedFile{Path: fc.Path, Operation: op["operation"].(string)})
|
||||||
}
|
}
|
||||||
on["files"] = ops
|
on["files"] = ops
|
||||||
if err := f.sendJSON(ctx, http.MethodPost, repo+"/contents", on, &made); err != nil {
|
if err := f.SendJSON(ctx, http.MethodPost, repo+"/contents", on, &made); err != nil {
|
||||||
return nil, fmt.Errorf("the forge did not take the commit: %w", err)
|
return nil, fmt.Errorf("the forge did not take the commit: %w", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -274,6 +305,25 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
return nil, fmt.Errorf("the forge answered the commit on %s without saying the commit it made: read the branch (gitea_list_branches) before trying again", a.Branch)
|
return nil, fmt.Errorf("the forge answered the commit on %s without saying the commit it made: read the branch (gitea_list_branches) before trying again", a.Branch)
|
||||||
}
|
}
|
||||||
out.Commit = made.Commit.SHA
|
out.Commit = made.Commit.SHA
|
||||||
|
read := out.Parent
|
||||||
|
parent, err := f.parentOf(ctx, repo, made.Commit)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("the forge made %s on %s, but which commit it went on could not be read: %w", out.Commit, a.Branch, err)
|
||||||
|
}
|
||||||
|
out.Parent = parent
|
||||||
|
if parent != read {
|
||||||
|
moved := "branch " + a.Branch
|
||||||
|
if out.Created {
|
||||||
|
moved = "base " + out.Base
|
||||||
|
}
|
||||||
|
how := "each file it changes was checked against " + short(read) + " by its blob, so none of them changed in between"
|
||||||
|
if a.Patch != "" {
|
||||||
|
how = "the forge applied the patch there, and it refuses a patch whose context changed"
|
||||||
|
}
|
||||||
|
out.Said = append(out.Said, fmt.Sprintf("the %s moved from %s to %s between the read and the commit, so the "+
|
||||||
|
"commit went on %s: %s. What else changed in between: gitea_compare %s...%s",
|
||||||
|
moved, short(read), short(parent), short(parent), how, short(read), short(parent)))
|
||||||
|
}
|
||||||
|
|
||||||
// Evidence over assertion: the branch is read back, and it must be at the commit the forge said.
|
// Evidence over assertion: the branch is read back, and it must be at the commit the forge said.
|
||||||
after, err := f.branch(ctx, repo, a.Branch)
|
after, err := f.branch(ctx, repo, a.Branch)
|
||||||
@@ -294,6 +344,30 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
|||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// madeCommit is the commit the forge answers a write with.
|
||||||
|
type madeCommit struct {
|
||||||
|
SHA string `json:"sha"`
|
||||||
|
Parents []struct {
|
||||||
|
SHA string `json:"sha"`
|
||||||
|
} `json:"parents"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// parentOf is the commit the forge made its commit on: from its answer, or read from the commit when the
|
||||||
|
// answer does not say it.
|
||||||
|
func (f *Forge) parentOf(ctx context.Context, repo string, made madeCommit) (string, error) {
|
||||||
|
if len(made.Parents) == 0 {
|
||||||
|
var read madeCommit
|
||||||
|
if err := f.GetJSON(ctx, repo+"/git/commits/"+url.PathEscape(made.SHA)+"?stat=false&verification=false&files=false", &read); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
made = read
|
||||||
|
}
|
||||||
|
if len(made.Parents) != 1 || made.Parents[0].SHA == "" {
|
||||||
|
return "", fmt.Errorf("the forge says %s has %d parents, not one", made.SHA, len(made.Parents))
|
||||||
|
}
|
||||||
|
return made.Parents[0].SHA, nil
|
||||||
|
}
|
||||||
|
|
||||||
// blobAt is the blob sha of a file at a ref, "" when there is no file there.
|
// blobAt is the blob sha of a file at a ref, "" when there is no file there.
|
||||||
func (f *Forge) blobAt(ctx context.Context, repo, path, ref string) (string, error) {
|
func (f *Forge) blobAt(ctx context.Context, repo, path, ref string) (string, error) {
|
||||||
seg, err := segments("file path", path)
|
seg, err := segments("file path", path)
|
||||||
@@ -305,7 +379,7 @@ func (f *Forge) blobAt(ctx context.Context, repo, path, ref string) (string, err
|
|||||||
SHA string `json:"sha"`
|
SHA string `json:"sha"`
|
||||||
}
|
}
|
||||||
var list []any
|
var list []any
|
||||||
body, err := f.do(ctx, http.MethodGet, repo+"/contents/"+seg+"?ref="+queryEscape(ref), nil, maxBody)
|
body, err := f.Do(ctx, http.MethodGet, repo+"/contents/"+seg+"?ref="+queryEscape(ref), nil, maxBody)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, errNotFound) {
|
if errors.Is(err, errNotFound) {
|
||||||
return "", nil
|
return "", nil
|
||||||
|
|||||||
@@ -1,156 +1,43 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
|
||||||
"regexp"
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
|
||||||
|
"gitea/internal/forgehttp"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account,
|
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
|
||||||
// which the vault delivered into a file this module owns (novox/hq ADR 0086), by basic authentication — the
|
// (internal/forgehttp, shared with the npm-registry bundle beside it).
|
||||||
// same way the npm-registry bundle beside it does. The password is read when a call is made and never logged,
|
|
||||||
// answered or put in an error.
|
|
||||||
|
|
||||||
// Forge is the forge's HTTP interface.
|
// Forge is the forge's HTTP interface.
|
||||||
type Forge struct {
|
type Forge struct {
|
||||||
URL string // e.g. http://127.0.0.1:3000
|
forgehttp.Client
|
||||||
User string
|
|
||||||
PasswordFile string
|
|
||||||
HTTP *http.Client
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
||||||
func ForgeFromEnv() (*Forge, error) {
|
func ForgeFromEnv() (*Forge, error) {
|
||||||
f := &Forge{
|
c, missing := forgehttp.FromEnv()
|
||||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
|
||||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
|
||||||
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
|
||||||
}
|
|
||||||
var missing []string
|
|
||||||
for _, kv := range [][2]string{{"MESH_GITEA_URL", f.URL}, {"MESH_GITEA_ADMIN_USER", f.User},
|
|
||||||
{"MESH_GITEA_ADMIN_PASSWORD_FILE", f.PasswordFile}} {
|
|
||||||
if kv[1] == "" {
|
|
||||||
missing = append(missing, kv[0])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if len(missing) > 0 {
|
if len(missing) > 0 {
|
||||||
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
||||||
}
|
}
|
||||||
return f, nil
|
c.Timeout = deadline
|
||||||
|
return &Forge{Client: c}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// errNotFound is the forge answering 404.
|
// errNotFound is the forge answering 404.
|
||||||
var errNotFound = errors.New("not found")
|
var errNotFound = forgehttp.ErrNotFound
|
||||||
|
|
||||||
// statusError is an answer the forge gave that is not a success, said without the request's credential.
|
|
||||||
type statusError struct {
|
|
||||||
method, path string
|
|
||||||
status int
|
|
||||||
said string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *statusError) Error() string {
|
|
||||||
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.status, e.method, e.path, e.said)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) client() *http.Client {
|
|
||||||
if f.HTTP != nil {
|
|
||||||
return f.HTTP
|
|
||||||
}
|
|
||||||
return &http.Client{Timeout: 50 * time.Second}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) password() (string, error) {
|
|
||||||
raw, err := os.ReadFile(f.PasswordFile)
|
|
||||||
if err != nil {
|
|
||||||
// The path is the manifest's, not a secret; the content never appears.
|
|
||||||
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(string(raw)), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// maxBody is the largest JSON answer read from the forge; maxArchive the largest archive code search reads.
|
// maxBody is the largest JSON answer read from the forge; maxArchive the largest archive code search reads.
|
||||||
const (
|
const (
|
||||||
maxBody = 32 << 20
|
maxBody = forgehttp.MaxBody
|
||||||
maxArchive = 64 << 20
|
maxArchive = 64 << 20
|
||||||
)
|
)
|
||||||
|
|
||||||
// do sends one request and answers the body, read up to limit; 404 is errNotFound. A body the limit cut is
|
|
||||||
// refused, never read as if it were whole.
|
|
||||||
func (f *Forge) do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
|
|
||||||
password, err := f.password()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
var body io.Reader
|
|
||||||
if payload != nil {
|
|
||||||
raw, err := json.Marshal(payload)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
body = bytes.NewReader(raw)
|
|
||||||
}
|
|
||||||
req, err := http.NewRequestWithContext(ctx, method, f.URL+path, body)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
req.SetBasicAuth(f.User, password)
|
|
||||||
req.Header.Set("Accept", "application/json")
|
|
||||||
if payload != nil {
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
}
|
|
||||||
res, err := f.client().Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
|
||||||
}
|
|
||||||
defer res.Body.Close()
|
|
||||||
got, err := io.ReadAll(io.LimitReader(res.Body, limit+1))
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if int64(len(got)) > limit {
|
|
||||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
|
|
||||||
}
|
|
||||||
switch {
|
|
||||||
case res.StatusCode == http.StatusNotFound:
|
|
||||||
return nil, fmt.Errorf("%s %s: %w", method, path, errNotFound)
|
|
||||||
case res.StatusCode >= 300:
|
|
||||||
said := strings.TrimSpace(string(got))
|
|
||||||
if len(said) > 300 {
|
|
||||||
said = said[:300] + "…"
|
|
||||||
}
|
|
||||||
return nil, &statusError{method, path, res.StatusCode, said}
|
|
||||||
}
|
|
||||||
return got, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) getJSON(ctx context.Context, path string, into any) error {
|
|
||||||
return f.sendJSON(ctx, http.MethodGet, path, nil, into)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) sendJSON(ctx context.Context, method, path string, payload, into any) error {
|
|
||||||
body, err := f.do(ctx, method, path, payload, maxBody)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if into == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(body, into); err != nil {
|
|
||||||
return fmt.Errorf("the forge's answer to %s %s is not readable: %w", method, path, err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Refused is a refusal of policy or of an argument, in the words the caller sees.
|
// Refused is a refusal of policy or of an argument, in the words the caller sees.
|
||||||
type Refused struct{ msg string }
|
type Refused struct{ msg string }
|
||||||
|
|
||||||
@@ -204,36 +91,6 @@ func (p Protection) name() string {
|
|||||||
return p.BranchName
|
return p.BranchName
|
||||||
}
|
}
|
||||||
|
|
||||||
// ruleCovers is whether a protection rule's name — a branch, or a glob of them (`*`, `**`, `?`) — covers a
|
|
||||||
// branch: the TypeScript bundle's ruleCovers (guard.ts), which gitea_api's file writes are judged by.
|
|
||||||
func ruleCovers(rule, branch string) bool {
|
|
||||||
if rule == branch {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
if !strings.ContainsAny(rule, "*?[") {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
var re strings.Builder
|
|
||||||
re.WriteString("^")
|
|
||||||
for i := 0; i < len(rule); i++ {
|
|
||||||
c := rule[i]
|
|
||||||
switch {
|
|
||||||
case c == '*' && i+1 < len(rule) && rule[i+1] == '*':
|
|
||||||
re.WriteString(".*")
|
|
||||||
i++
|
|
||||||
case c == '*':
|
|
||||||
re.WriteString("[^/]*")
|
|
||||||
case c == '?':
|
|
||||||
re.WriteString("[^/]")
|
|
||||||
default:
|
|
||||||
re.WriteString(regexp.QuoteMeta(string(c)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
re.WriteString("$")
|
|
||||||
ok, err := regexp.MatchString(re.String(), branch)
|
|
||||||
return err == nil && ok
|
|
||||||
}
|
|
||||||
|
|
||||||
// Branch is one branch with the commit it points at.
|
// Branch is one branch with the commit it points at.
|
||||||
type Branch struct {
|
type Branch struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
@@ -249,7 +106,7 @@ func (f *Forge) branch(ctx context.Context, repo, name string) (*Branch, error)
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
var b Branch
|
var b Branch
|
||||||
if err := f.getJSON(ctx, repo+"/branches/"+seg, &b); err != nil {
|
if err := f.GetJSON(ctx, repo+"/branches/"+seg, &b); err != nil {
|
||||||
if errors.Is(err, errNotFound) {
|
if errors.Is(err, errNotFound) {
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,9 +13,12 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"gitea/internal/forgehttp"
|
||||||
)
|
)
|
||||||
|
|
||||||
// fakeForge is the forge's HTTP interface as far as these tools use it.
|
// fakeForge is the forge's HTTP interface as far as these tools use it.
|
||||||
@@ -37,6 +40,16 @@ type fakeForge struct {
|
|||||||
comments map[string][]map[string]any
|
comments map[string][]map[string]any
|
||||||
commits []map[string]any
|
commits []map[string]any
|
||||||
dir []map[string]any
|
dir []map[string]any
|
||||||
|
// moves: a branch moved by someone else right after it is first read — to a commit, with its tree.
|
||||||
|
moves map[string]move
|
||||||
|
noParents bool // the forge's answer to a write does not say the parents
|
||||||
|
parentsBroken bool // and GET git/commits fails
|
||||||
|
parents map[string]string // commit -> parent, as GET git/commits says it
|
||||||
|
}
|
||||||
|
|
||||||
|
type move struct {
|
||||||
|
to string
|
||||||
|
tree map[string]string
|
||||||
}
|
}
|
||||||
|
|
||||||
func newFake(t *testing.T) *fakeForge {
|
func newFake(t *testing.T) *fakeForge {
|
||||||
@@ -62,7 +75,7 @@ func (f *fakeForge) serve() *Forge {
|
|||||||
}
|
}
|
||||||
srv := httptest.NewServer(http.HandlerFunc(f.handle))
|
srv := httptest.NewServer(http.HandlerFunc(f.handle))
|
||||||
f.t.Cleanup(srv.Close)
|
f.t.Cleanup(srv.Close)
|
||||||
return &Forge{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}
|
return &Forge{Client: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -92,6 +105,11 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
reply(map[string]any{"name": name, "commit": map[string]any{"id": c}})
|
reply(map[string]any{"name": name, "commit": map[string]any{"id": c}})
|
||||||
|
if m, ok := f.moves[name]; ok {
|
||||||
|
delete(f.moves, name)
|
||||||
|
f.trees[m.to] = m.tree
|
||||||
|
f.branches[name] = m.to
|
||||||
|
}
|
||||||
case r.Method == "GET" && (path == repo+"/contents" || path == repo+"/contents/docs"):
|
case r.Method == "GET" && (path == repo+"/contents" || path == repo+"/contents/docs"):
|
||||||
reply(f.dir)
|
reply(f.dir)
|
||||||
case r.Method == "GET" && strings.HasPrefix(path, repo+"/contents/"):
|
case r.Method == "GET" && strings.HasPrefix(path, repo+"/contents/"):
|
||||||
@@ -148,19 +166,28 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !f.staleRead {
|
if !f.staleRead {
|
||||||
f.branches[target] = made
|
f.branches[target] = made
|
||||||
}
|
}
|
||||||
reply(map[string]any{"commit": map[string]any{"sha": made}})
|
reply(map[string]any{"commit": f.made(made, from)})
|
||||||
case r.Method == "POST" && path == repo+"/diffpatch":
|
case r.Method == "POST" && path == repo+"/diffpatch":
|
||||||
var body map[string]any
|
var body map[string]any
|
||||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||||
f.patches = append(f.patches, body)
|
f.patches = append(f.patches, body)
|
||||||
f.next++
|
f.next++
|
||||||
made := fmt.Sprintf("p%d", f.next)
|
made := fmt.Sprintf("p%d", f.next)
|
||||||
|
from := f.branches[body["branch"].(string)]
|
||||||
target := body["branch"].(string)
|
target := body["branch"].(string)
|
||||||
if nb, ok := body["new_branch"].(string); ok && nb != "" {
|
if nb, ok := body["new_branch"].(string); ok && nb != "" {
|
||||||
target = nb
|
target = nb
|
||||||
}
|
}
|
||||||
f.branches[target] = made
|
f.branches[target] = made
|
||||||
reply(map[string]any{"commit": map[string]any{"sha": made}})
|
reply(map[string]any{"commit": f.made(made, from)})
|
||||||
|
case r.Method == "GET" && strings.HasPrefix(path, repo+"/git/commits/"):
|
||||||
|
sha := strings.TrimPrefix(path, repo+"/git/commits/")
|
||||||
|
p, ok := f.parents[sha]
|
||||||
|
if !ok || f.parentsBroken {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
reply(map[string]any{"sha": sha, "parents": []any{map[string]any{"sha": p}}})
|
||||||
case r.Method == "GET" && path == repo+"/commits":
|
case r.Method == "GET" && path == repo+"/commits":
|
||||||
f.writes = append(f.writes, map[string]any{"query": r.URL.RawQuery})
|
f.writes = append(f.writes, map[string]any{"query": r.URL.RawQuery})
|
||||||
reply(f.commits)
|
reply(f.commits)
|
||||||
@@ -180,6 +207,18 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// made is the forge's answer about a commit it made on a parent, with the parents unless noParents.
|
||||||
|
func (f *fakeForge) made(sha, parent string) map[string]any {
|
||||||
|
if f.parents == nil {
|
||||||
|
f.parents = map[string]string{}
|
||||||
|
}
|
||||||
|
f.parents[sha] = parent
|
||||||
|
if f.noParents {
|
||||||
|
return map[string]any{"sha": sha}
|
||||||
|
}
|
||||||
|
return map[string]any{"sha": sha, "parents": []any{map[string]any{"sha": parent}}}
|
||||||
|
}
|
||||||
|
|
||||||
func filesArg(files ...map[string]any) []any {
|
func filesArg(files ...map[string]any) []any {
|
||||||
out := make([]any, len(files))
|
out := make([]any, len(files))
|
||||||
for i, f := range files {
|
for i, f := range files {
|
||||||
@@ -359,17 +398,27 @@ func TestACommitAskThatCannotBeReadIsRefused(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRuleCoversIsTheGuardsRule(t *testing.T) {
|
// The forge's own answers (Gitea 1.27's Match), one table for this bundle and guard.ts alike (novox/hq issue 539).
|
||||||
for _, c := range []struct {
|
// `forge` is the forge's answer, which this port must give; `guard` is where guard.ts deliberately covers more.
|
||||||
rule, branch string
|
func TestRuleCoversAnswersAsTheForgeDoes(t *testing.T) {
|
||||||
want bool
|
raw, err := os.ReadFile("../../test/rule-cases.json")
|
||||||
}{
|
if err != nil {
|
||||||
{"main", "main", true}, {"main", "mainx", false}, {"release/*", "release/1.0", true},
|
t.Fatal(err)
|
||||||
{"release/*", "release/a/b", false}, {"release/**", "release/a/b", true}, {"v?", "v1", true},
|
}
|
||||||
{"a.b", "axb", false},
|
var cases []struct {
|
||||||
} {
|
Rule string `json:"rule"`
|
||||||
if got := ruleCovers(c.rule, c.branch); got != c.want {
|
Branch string `json:"branch"`
|
||||||
t.Errorf("%s covers %s: %v", c.rule, c.branch, got)
|
Forge bool `json:"forge"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &cases); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(cases) < 100 {
|
||||||
|
t.Fatalf("only %d cases", len(cases))
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := ruleCovers(c.Rule, c.Branch); got != c.Forge {
|
||||||
|
t.Errorf("%q covers %q: %v, the forge says %v", c.Rule, c.Branch, got, c.Forge)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -581,3 +630,115 @@ func TestToolsAreTheOnesTheReadmeNamesAndTheManifestBuilds(t *testing.T) {
|
|||||||
t.Error("the manifest builds no cmd/forge bundle")
|
t.Error("the manifest builds no cmd/forge bundle")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// novox/hq issue 539: the forge takes a branch, never a commit to build on, so the answer's parent is the one
|
||||||
|
// the forge made the commit on, and a branch that moves between the read and the commit is said.
|
||||||
|
func TestAnExistingBranchMovedUnderTheCommitIsSaidAndItsParentIsTheOneTheForgeUsed(t *testing.T) {
|
||||||
|
f := newFake(t)
|
||||||
|
f.moves = map[string]move{"fix/1-old": {to: "c2", tree: map[string]string{"README.md": "hello\n", "docs/a.md": "a on fix\n", "other.md": "x\n"}}}
|
||||||
|
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/1-old",
|
||||||
|
"files": filesArg(map[string]any{"path": "docs/a.md", "content": "a fixed\n"})}))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.Parent != "c2" || f.branches["fix/1-old"] != got.Commit {
|
||||||
|
t.Fatalf("parent %s, want c2 (the commit the forge built on): %+v", got.Parent, got)
|
||||||
|
}
|
||||||
|
said := strings.Join(got.Said, " ")
|
||||||
|
if !strings.Contains(said, "moved from c1 to c2") || !strings.Contains(said, "after c2") {
|
||||||
|
t.Fatalf("the move was not said: %v", got.Said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAnExistingBranchsFilesAreJudgedAtTheCommitReadNotAtWhereTheBranchMovedTo(t *testing.T) {
|
||||||
|
f := newFake(t)
|
||||||
|
// Someone else changes docs/a.md on the branch between the read and the commit: that change is not
|
||||||
|
// overwritten by a blob read at the branch's new head.
|
||||||
|
f.moves = map[string]move{"fix/1-old": {to: "c2", tree: map[string]string{"README.md": "hello\n", "docs/a.md": "theirs\n"}}}
|
||||||
|
_, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/1-old",
|
||||||
|
"files": filesArg(map[string]any{"path": "docs/a.md", "content": "mine\n"})}))
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "sha does not match") {
|
||||||
|
t.Fatalf("a file changed under the commit was overwritten: %v", err)
|
||||||
|
}
|
||||||
|
if f.trees["c2"]["docs/a.md"] != "theirs\n" || f.branches["fix/1-old"] != "c2" {
|
||||||
|
t.Fatalf("the branch was changed: %v", f.branches)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPatchOnABaseThatMovedNamesTheCommitTheForgeAppliedItTo(t *testing.T) {
|
||||||
|
f := newFake(t)
|
||||||
|
f.moves = map[string]move{"main": {to: "c9", tree: map[string]string{"README.md": "hello\n"}}}
|
||||||
|
patch := "diff --git a/README.md b/README.md\n--- a/README.md\n+++ b/README.md\n@@ -1 +1 @@\n-hello\n+hi\n"
|
||||||
|
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/6", "patch": patch}))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.Parent != "c9" || !strings.Contains(strings.Join(got.Said, " "), "base main moved from c0 to c9") {
|
||||||
|
t.Fatalf("%+v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAParentTheAnswerDoesNotSayIsReadFromTheCommit(t *testing.T) {
|
||||||
|
f := newFake(t)
|
||||||
|
f.noParents = true
|
||||||
|
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/7",
|
||||||
|
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||||
|
if err != nil || got.Parent != "c0" {
|
||||||
|
t.Fatalf("%v %+v", err, got)
|
||||||
|
}
|
||||||
|
// A parent that cannot be read is a failure, said with the commit that was made.
|
||||||
|
f = newFake(t)
|
||||||
|
f.noParents, f.parentsBroken = true, true
|
||||||
|
_, err = commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/8",
|
||||||
|
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "the forge made n1 on fix/8, but which commit it went on could not be read") {
|
||||||
|
t.Fatalf("%v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestACommitLargerThanItsLimitsIsRefusedBeforeAnythingIsSent(t *testing.T) {
|
||||||
|
big := strings.Repeat("x", maxFileBytes+1)
|
||||||
|
quarter := strings.Repeat("y", maxFileBytes)
|
||||||
|
many := filesArg()
|
||||||
|
for i := 0; i < maxCommitBytes/maxFileBytes+1; i++ {
|
||||||
|
many = append(many, map[string]any{"path": fmt.Sprintf("f%d", i), "content": quarter})
|
||||||
|
}
|
||||||
|
for name, args := range map[string]map[string]any{
|
||||||
|
"one file": {"files": filesArg(map[string]any{"path": "x", "content": big})},
|
||||||
|
"one file b64": {"files": filesArg(map[string]any{"path": "x", "content_base64": base64.StdEncoding.EncodeToString([]byte(big))})},
|
||||||
|
"the patch": {"patch": "diff --git a/x b/x\n" + big},
|
||||||
|
"all files": {"files": many},
|
||||||
|
} {
|
||||||
|
full := map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b"}
|
||||||
|
for k, v := range args {
|
||||||
|
full[k] = v
|
||||||
|
}
|
||||||
|
_, err := readCommitAsk(full)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "MiB") || !strings.Contains(err.Error(), "Nothing was sent") {
|
||||||
|
t.Errorf("%s: %v", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// At the limit is taken.
|
||||||
|
if _, err := readCommitAsk(map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b",
|
||||||
|
"files": filesArg(map[string]any{"path": "x", "content_base64": base64.StdEncoding.EncodeToString([]byte(quarter))})}); err != nil {
|
||||||
|
t.Errorf("a file at the limit was refused: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCodeSearchStopsAtItsDeadlineAndSaysSo(t *testing.T) {
|
||||||
|
gz := archive(t, map[string]string{"a.md": "forge\n", "b.md": "forge\n"})
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
out := &Found{Matches: []Match{}}
|
||||||
|
if err := searchArchive(ctx, gz, "", regexp.MustCompile("forge"), 100, out); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !out.Stopped || out.Files != 0 {
|
||||||
|
t.Fatalf("a search past its deadline went on: %+v", out)
|
||||||
|
}
|
||||||
|
// Within its deadline, the same archive is searched whole.
|
||||||
|
out = &Found{Matches: []Match{}}
|
||||||
|
if err := searchArchive(context.Background(), gz, "", regexp.MustCompile("forge"), 100, out); err != nil || out.Stopped || len(out.Matches) != 2 {
|
||||||
|
t.Fatalf("%v %+v", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -53,7 +53,9 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
|
|||||||
"independent review approved and whose required checks passed (gitea_merge_pull_request). Carries " +
|
"independent review approved and whose required checks passed (gitea_merge_pull_request). Carries " +
|
||||||
"either files — each {path, content (text) | content_base64 | delete: true}, created or updated as " +
|
"either files — each {path, content (text) | content_base64 | delete: true}, created or updated as " +
|
||||||
"the branch holds them — or patch, a unified diff as git diff writes it, which the forge applies. " +
|
"the branch holds them — or patch, a unified diff as git diff writes it, which the forge applies. " +
|
||||||
"Answers the commit it made and its parent, read back from the branch. Replaces git commit and git " +
|
"At most 200 files, 4 MiB each and 16 MiB together, or a 4 MiB patch. Answers the commit it made and " +
|
||||||
|
"the parent the forge made it on, read back from the branch, and says when the branch moved under " +
|
||||||
|
"it. Replaces git commit and git " +
|
||||||
"push of a feature branch. (a)",
|
"push of a feature branch. (a)",
|
||||||
Input: map[string]any{
|
Input: map[string]any{
|
||||||
"type": "object",
|
"type": "object",
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ func history(ctx context.Context, f *Forge, owner, name, ref, path string, limit
|
|||||||
q.Set("verification", "false")
|
q.Set("verification", "false")
|
||||||
q.Set("files", "false")
|
q.Set("files", "false")
|
||||||
var got []forgeCommit
|
var got []forgeCommit
|
||||||
if err := f.getJSON(ctx, repo+"/commits?"+q.Encode(), &got); err != nil {
|
if err := f.GetJSON(ctx, repo+"/commits?"+q.Encode(), &got); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
h := &History{Repository: owner + "/" + name, Ref: ref, Path: path, Page: page, Commits: []CommitLine{}, More: len(got) >= limit}
|
h := &History{Repository: owner + "/" + name, Ref: ref, Path: path, Page: page, Commits: []CommitLine{}, More: len(got) >= limit}
|
||||||
@@ -121,7 +121,7 @@ func compare(ctx context.Context, f *Forge, owner, name, base, head string) (*Co
|
|||||||
Total int `json:"total_commits"`
|
Total int `json:"total_commits"`
|
||||||
Commits []forgeCommit `json:"commits"`
|
Commits []forgeCommit `json:"commits"`
|
||||||
}
|
}
|
||||||
if err := f.getJSON(ctx, repo+"/compare/"+b+"..."+h, &got); err != nil {
|
if err := f.GetJSON(ctx, repo+"/compare/"+b+"..."+h, &got); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
c := &Comparison{Repository: owner + "/" + name, Base: base, Head: head, Total: got.Total, Commits: []CommitLine{}, Files: []FileLine{}}
|
c := &Comparison{Repository: owner + "/" + name, Base: base, Head: head, Total: got.Total, Commits: []CommitLine{}, Files: []FileLine{}}
|
||||||
@@ -195,7 +195,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
|||||||
SHA string `json:"sha"`
|
SHA string `json:"sha"`
|
||||||
} `json:"head"`
|
} `json:"head"`
|
||||||
}
|
}
|
||||||
if err := f.getJSON(ctx, pr, &pull); err != nil {
|
if err := f.GetJSON(ctx, pr, &pull); err != nil {
|
||||||
if errors.Is(err, errNotFound) {
|
if errors.Is(err, errNotFound) {
|
||||||
return nil, refuse("%s/%s has no pull request %d", owner, name, number)
|
return nil, refuse("%s/%s has no pull request %d", owner, name, number)
|
||||||
}
|
}
|
||||||
@@ -218,7 +218,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
|||||||
var all []forgeReview
|
var all []forgeReview
|
||||||
for p := 1; ; p++ {
|
for p := 1; ; p++ {
|
||||||
var batch []forgeReview
|
var batch []forgeReview
|
||||||
if err := f.getJSON(ctx, fmt.Sprintf("%s/reviews?limit=%d&page=%d", pr, page, p), &batch); err != nil {
|
if err := f.GetJSON(ctx, fmt.Sprintf("%s/reviews?limit=%d&page=%d", pr, page, p), &batch); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
all = append(all, batch...)
|
all = append(all, batch...)
|
||||||
@@ -240,7 +240,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
|||||||
CommitID string `json:"commit_id"`
|
CommitID string `json:"commit_id"`
|
||||||
Resolver *forgeUser `json:"resolver"`
|
Resolver *forgeUser `json:"resolver"`
|
||||||
}
|
}
|
||||||
if err := f.getJSON(ctx, fmt.Sprintf("%s/reviews/%d/comments", pr, r.ID), &cs); err != nil {
|
if err := f.GetJSON(ctx, fmt.Sprintf("%s/reviews/%d/comments", pr, r.ID), &cs); err != nil {
|
||||||
return nil, fmt.Errorf("review %d's line comments could not be read: %w", r.ID, err)
|
return nil, fmt.Errorf("review %d's line comments could not be read: %w", r.ID, err)
|
||||||
}
|
}
|
||||||
for _, c := range cs {
|
for _, c := range cs {
|
||||||
@@ -292,7 +292,7 @@ func listDir(ctx context.Context, f *Forge, owner, name, path, ref string) (*Lis
|
|||||||
if ref != "" {
|
if ref != "" {
|
||||||
at += "?ref=" + queryEscape(ref)
|
at += "?ref=" + queryEscape(ref)
|
||||||
}
|
}
|
||||||
body, err := f.do(ctx, "GET", at, nil, maxBody)
|
body, err := f.Do(ctx, "GET", at, nil, maxBody)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, errNotFound) {
|
if errors.Is(err, errNotFound) {
|
||||||
return nil, refuse("%s/%s has no %q at %s", owner, name, path, orDefault(ref))
|
return nil, refuse("%s/%s has no %q at %s", owner, name, path, orDefault(ref))
|
||||||
|
|||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ruleCovers is whether a protection rule's name covers a branch, as the forge itself decides it (Gitea 1.27,
|
||||||
|
// models/git/protected_branch.go Match and modules/glob, novox/hq issue 539): a name without a glob character
|
||||||
|
// covers the branch of that name in any case; any other name is a glob with `/` as its separator — `*` any
|
||||||
|
// run within a segment, `**` any run across them, `?` one character of a segment, `[abc]`, `[a-z]` and `[!abc]`
|
||||||
|
// one character, `{a,b}` either, `\` the next character itself — matched case-sensitively; a glob the forge
|
||||||
|
// cannot compile covers its own text exactly. The TypeScript bundle's ruleCovers (guard.ts) is the same rule.
|
||||||
|
func ruleCovers(rule, branch string) bool {
|
||||||
|
if !strings.ContainsAny(rule, `*?\[]{}`) {
|
||||||
|
return strings.EqualFold(rule, branch)
|
||||||
|
}
|
||||||
|
re, err := globRegexp(rule)
|
||||||
|
if err != nil {
|
||||||
|
return rule == branch
|
||||||
|
}
|
||||||
|
return re.MatchString(branch)
|
||||||
|
}
|
||||||
|
|
||||||
|
// globRegexp is the forge's glob as a regular expression, compiled as modules/glob compiles it with `/` as
|
||||||
|
// the separator.
|
||||||
|
func globRegexp(glob string) (*regexp.Regexp, error) {
|
||||||
|
g := []rune(glob)
|
||||||
|
var b strings.Builder
|
||||||
|
pos := 0
|
||||||
|
var compile func(sub bool) error
|
||||||
|
chars := func() error {
|
||||||
|
b.WriteByte('[')
|
||||||
|
if pos < len(g) && g[pos] == '!' {
|
||||||
|
pos++
|
||||||
|
b.WriteByte('^')
|
||||||
|
}
|
||||||
|
for pos < len(g) {
|
||||||
|
c := g[pos]
|
||||||
|
pos++
|
||||||
|
switch {
|
||||||
|
case c == ']':
|
||||||
|
b.WriteByte(']')
|
||||||
|
return nil
|
||||||
|
case c == '\\':
|
||||||
|
if pos >= len(g) {
|
||||||
|
return errors.New("unterminated character class escape")
|
||||||
|
}
|
||||||
|
b.WriteByte('\\')
|
||||||
|
b.WriteRune(g[pos])
|
||||||
|
pos++
|
||||||
|
default:
|
||||||
|
b.WriteRune(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errors.New("unterminated character class")
|
||||||
|
}
|
||||||
|
compile = func(sub bool) error {
|
||||||
|
for pos < len(g) {
|
||||||
|
c := g[pos]
|
||||||
|
pos++
|
||||||
|
if sub && c == '}' {
|
||||||
|
b.WriteByte(')')
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
switch c {
|
||||||
|
case '*':
|
||||||
|
if pos < len(g) && g[pos] == '*' {
|
||||||
|
pos++
|
||||||
|
b.WriteString(".*")
|
||||||
|
} else {
|
||||||
|
b.WriteString("[^/]*")
|
||||||
|
}
|
||||||
|
case '?':
|
||||||
|
b.WriteString("[^/]")
|
||||||
|
case '+':
|
||||||
|
b.WriteString(`\+`)
|
||||||
|
case '[':
|
||||||
|
if err := chars(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
case '{':
|
||||||
|
b.WriteByte('(')
|
||||||
|
if err := compile(true); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
case ',':
|
||||||
|
if sub {
|
||||||
|
b.WriteByte('|')
|
||||||
|
} else {
|
||||||
|
b.WriteByte(',')
|
||||||
|
}
|
||||||
|
case '\\':
|
||||||
|
if pos >= len(g) {
|
||||||
|
return errors.New("no character to escape")
|
||||||
|
}
|
||||||
|
b.WriteByte('\\')
|
||||||
|
b.WriteRune(g[pos])
|
||||||
|
pos++
|
||||||
|
case '.', '^', '$', '(', ')', '|':
|
||||||
|
b.WriteByte('\\')
|
||||||
|
b.WriteRune(c)
|
||||||
|
default:
|
||||||
|
b.WriteRune(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
b.WriteByte('^')
|
||||||
|
if err := compile(false); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b.WriteByte('$')
|
||||||
|
return regexp.Compile(b.String())
|
||||||
|
}
|
||||||
@@ -32,6 +32,7 @@ type Found struct {
|
|||||||
Files int `json:"files_searched"`
|
Files int `json:"files_searched"`
|
||||||
Skipped []string `json:"skipped,omitempty"`
|
Skipped []string `json:"skipped,omitempty"`
|
||||||
Truncated bool `json:"truncated"`
|
Truncated bool `json:"truncated"`
|
||||||
|
Stopped bool `json:"stopped_at_deadline,omitempty"`
|
||||||
Said []string `json:"said"`
|
Said []string `json:"said"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,7 +70,7 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
|||||||
ref := a.Ref
|
ref := a.Ref
|
||||||
if ref == "" {
|
if ref == "" {
|
||||||
var r Repo
|
var r Repo
|
||||||
if err := f.getJSON(ctx, repo, &r); err != nil {
|
if err := f.GetJSON(ctx, repo, &r); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if r.Empty || r.DefaultBranch == "" {
|
if r.Empty || r.DefaultBranch == "" {
|
||||||
@@ -87,7 +88,7 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
body, err := f.do(ctx, "GET", repo+"/archive/"+seg+".tar.gz", nil, maxArchive)
|
body, err := f.Do(ctx, "GET", repo+"/archive/"+seg+".tar.gz", nil, maxArchive)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, errNotFound) {
|
if errors.Is(err, errNotFound) {
|
||||||
return nil, refuse("%s/%s has no ref %s", a.Owner, a.Repo, ref)
|
return nil, refuse("%s/%s has no ref %s", a.Owner, a.Repo, ref)
|
||||||
@@ -95,11 +96,15 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
out := &Found{Repository: a.Owner + "/" + a.Repo, Ref: ref, Query: a.Query, Matches: []Match{}}
|
out := &Found{Repository: a.Owner + "/" + a.Repo, Ref: ref, Query: a.Query, Matches: []Match{}}
|
||||||
if err := searchArchive(body, prefix, re, a.Max, out); err != nil {
|
if err := searchArchive(ctx, body, prefix, re, a.Max, out); err != nil {
|
||||||
return nil, fmt.Errorf("the archive of %s at %s could not be read: %w", out.Repository, ref, err)
|
return nil, fmt.Errorf("the archive of %s at %s could not be read: %w", out.Repository, ref, err)
|
||||||
}
|
}
|
||||||
said := fmt.Sprintf("%d matches in %d files searched", len(out.Matches), out.Files)
|
said := fmt.Sprintf("%d matches in %d files searched", len(out.Matches), out.Files)
|
||||||
if out.Truncated {
|
switch {
|
||||||
|
case out.Stopped:
|
||||||
|
said = fmt.Sprintf("the search stopped at its deadline of %s after %d files, with %d matches: the rest of the "+
|
||||||
|
"archive was not searched. Narrow with path", deadline, out.Files, len(out.Matches))
|
||||||
|
case out.Truncated:
|
||||||
said = fmt.Sprintf("the first %d matches, and more were found: narrow with path or the query", len(out.Matches))
|
said = fmt.Sprintf("the first %d matches, and more were found: narrow with path or the query", len(out.Matches))
|
||||||
}
|
}
|
||||||
out.Said = append(out.Said, said)
|
out.Said = append(out.Said, said)
|
||||||
@@ -110,18 +115,32 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// searchArchive reads a gzipped tar the forge made — every path under one top directory — and collects the
|
// searchArchive reads a gzipped tar the forge made — every path under one top directory — and collects the
|
||||||
// lines that match, from text files under the prefix.
|
// lines that match, from text files under the prefix. It stops at the call's deadline (novox/hq issue 539):
|
||||||
func searchArchive(gz []byte, prefix string, re *regexp.Regexp, max int, out *Found) error {
|
// what it found by then is kept, and out.Stopped says the rest was not searched.
|
||||||
|
func searchArchive(ctx context.Context, gz []byte, prefix string, re *regexp.Regexp, max int, out *Found) error {
|
||||||
zr, err := gzip.NewReader(bytes.NewReader(gz))
|
zr, err := gzip.NewReader(bytes.NewReader(gz))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
tr := tar.NewReader(zr)
|
tr := tar.NewReader(ctxReader{ctx, zr})
|
||||||
|
stopped := func(err error) bool {
|
||||||
|
if ctx.Err() != nil && (err == nil || errors.Is(err, ctx.Err())) {
|
||||||
|
out.Stopped = true
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
for {
|
for {
|
||||||
|
if stopped(nil) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
h, err := tr.Next()
|
h, err := tr.Next()
|
||||||
if err == io.EOF {
|
if err == io.EOF {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
if stopped(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -140,6 +159,9 @@ func searchArchive(gz []byte, prefix string, re *regexp.Regexp, max int, out *Fo
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
content, err := io.ReadAll(io.LimitReader(tr, maxFileSearched+1))
|
content, err := io.ReadAll(io.LimitReader(tr, maxFileSearched+1))
|
||||||
|
if stopped(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -169,3 +191,16 @@ func searchArchive(gz []byte, prefix string, re *regexp.Regexp, max int, out *Fo
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ctxReader is a reader that ends when its context does: decompressing is the long part of a search.
|
||||||
|
type ctxReader struct {
|
||||||
|
ctx context.Context
|
||||||
|
r io.Reader
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c ctxReader) Read(p []byte) (int, error) {
|
||||||
|
if err := c.ctx.Err(); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return c.r.Read(p)
|
||||||
|
}
|
||||||
|
|||||||
@@ -2,45 +2,33 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
"os"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea/internal/forgehttp"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin
|
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
|
||||||
// account, which the vault delivered into a file this module owns (novox/hq ADR 0086). Basic
|
// (internal/forgehttp, shared with the forge bundle beside it). Basic authentication, so this bundle mints
|
||||||
// authentication, so this bundle mints nothing and keeps nothing: the TypeScript bundle's token is its
|
// nothing and keeps nothing: the TypeScript bundle's token is its own.
|
||||||
// own. The password is read when a call is made and never logged, answered or put in an error.
|
|
||||||
|
|
||||||
// Forge is the forge's HTTP interface.
|
// Forge is the forge's HTTP interface.
|
||||||
type Forge struct {
|
type Forge struct {
|
||||||
URL string // e.g. http://127.0.0.1:3000
|
forgehttp.Client
|
||||||
User string
|
Owner string // the package registry's owner, as the seat serves it
|
||||||
PasswordFile string
|
|
||||||
Owner string // the package registry's owner, as the seat serves it
|
|
||||||
HTTP *http.Client
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
||||||
func ForgeFromEnv() (*Forge, error) {
|
func ForgeFromEnv() (*Forge, error) {
|
||||||
f := &Forge{
|
c, missing := forgehttp.FromEnv()
|
||||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
f := &Forge{Client: c, Owner: os.Getenv("MESH_NPM_OWNER")}
|
||||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
if f.Owner == "" {
|
||||||
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
missing = append(missing, "MESH_NPM_OWNER")
|
||||||
Owner: os.Getenv("MESH_NPM_OWNER"),
|
|
||||||
}
|
|
||||||
var missing []string
|
|
||||||
for name, v := range map[string]string{"MESH_GITEA_URL": f.URL, "MESH_GITEA_ADMIN_USER": f.User,
|
|
||||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": f.PasswordFile, "MESH_NPM_OWNER": f.Owner} {
|
|
||||||
if v == "" {
|
|
||||||
missing = append(missing, name)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if len(missing) > 0 {
|
if len(missing) > 0 {
|
||||||
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
||||||
@@ -49,86 +37,14 @@ func ForgeFromEnv() (*Forge, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// errNotFound is the forge answering 404.
|
// errNotFound is the forge answering 404.
|
||||||
var errNotFound = errors.New("not found")
|
var errNotFound = forgehttp.ErrNotFound
|
||||||
|
|
||||||
// statusError is an answer the forge gave that is not a success, said without the request's credential.
|
|
||||||
type statusError struct {
|
|
||||||
method, path string
|
|
||||||
status int
|
|
||||||
said string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *statusError) Error() string {
|
|
||||||
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.status, e.method, e.path, e.said)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) client() *http.Client {
|
|
||||||
if f.HTTP != nil {
|
|
||||||
return f.HTTP
|
|
||||||
}
|
|
||||||
return &http.Client{Timeout: 20 * time.Second}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) password() (string, error) {
|
|
||||||
raw, err := os.ReadFile(f.PasswordFile)
|
|
||||||
if err != nil {
|
|
||||||
// The path is the manifest's, not a secret; the content never appears.
|
|
||||||
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(string(raw)), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// maxBody is the largest answer read from the forge.
|
// maxBody is the largest answer read from the forge.
|
||||||
const maxBody = 32 << 20
|
const maxBody = forgehttp.MaxBody
|
||||||
|
|
||||||
// do sends one request and answers the body; 404 is errNotFound.
|
// do sends one request without a body and answers the body; 404 is errNotFound.
|
||||||
func (f *Forge) do(ctx context.Context, method, path string) ([]byte, error) {
|
func (f *Forge) do(ctx context.Context, method, path string) ([]byte, error) {
|
||||||
password, err := f.password()
|
return f.Do(ctx, method, path, nil, maxBody)
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
req, err := http.NewRequestWithContext(ctx, method, f.URL+path, nil)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
req.SetBasicAuth(f.User, password)
|
|
||||||
req.Header.Set("Accept", "application/json")
|
|
||||||
res, err := f.client().Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
|
||||||
}
|
|
||||||
defer res.Body.Close()
|
|
||||||
// One byte past the cap is read, so a body the cap cut is told from one that fits it exactly: a
|
|
||||||
// cut body is refused, never parsed as if it were whole.
|
|
||||||
body, err := io.ReadAll(io.LimitReader(res.Body, maxBody+1))
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if len(body) > maxBody {
|
|
||||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, maxBody>>20)
|
|
||||||
}
|
|
||||||
switch {
|
|
||||||
case res.StatusCode == http.StatusNotFound:
|
|
||||||
return nil, fmt.Errorf("%s %s: %w", method, path, errNotFound)
|
|
||||||
case res.StatusCode >= 300:
|
|
||||||
said := strings.TrimSpace(string(body))
|
|
||||||
if len(said) > 300 {
|
|
||||||
said = said[:300] + "…"
|
|
||||||
}
|
|
||||||
return nil, &statusError{method, path, res.StatusCode, said}
|
|
||||||
}
|
|
||||||
return body, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f *Forge) getJSON(ctx context.Context, path string, into any) error {
|
|
||||||
body, err := f.do(ctx, http.MethodGet, path)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(body, into); err != nil {
|
|
||||||
return fmt.Errorf("the forge's answer to %s is not readable: %w", path, err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// PackageVersion is one version of one package, as the forge lists it.
|
// PackageVersion is one version of one package, as the forge lists it.
|
||||||
@@ -146,7 +62,7 @@ func (f *Forge) Packages(ctx context.Context) ([]PackageVersion, error) {
|
|||||||
for page := 1; page <= 1000; page++ {
|
for page := 1; page <= 1000; page++ {
|
||||||
var batch []PackageVersion
|
var batch []PackageVersion
|
||||||
path := fmt.Sprintf("/api/v1/packages/%s?type=npm&page=%d&limit=%d", url.PathEscape(f.Owner), page, pageSize)
|
path := fmt.Sprintf("/api/v1/packages/%s?type=npm&page=%d&limit=%d", url.PathEscape(f.Owner), page, pageSize)
|
||||||
if err := f.getJSON(ctx, path, &batch); err != nil {
|
if err := f.GetJSON(ctx, path, &batch); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
all = append(all, batch...)
|
all = append(all, batch...)
|
||||||
@@ -166,7 +82,7 @@ func (f *Forge) Size(ctx context.Context, name, version string) (int64, error) {
|
|||||||
var files []struct {
|
var files []struct {
|
||||||
Size int64 `json:"size"`
|
Size int64 `json:"size"`
|
||||||
}
|
}
|
||||||
if err := f.getJSON(ctx, f.versionPath(name, version)+"/files", &files); err != nil {
|
if err := f.GetJSON(ctx, f.versionPath(name, version)+"/files", &files); err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
}
|
}
|
||||||
var total int64
|
var total int64
|
||||||
@@ -182,7 +98,7 @@ func (f *Forge) DistTags(ctx context.Context, name string) (map[string]string, e
|
|||||||
DistTags map[string]string `json:"dist-tags"`
|
DistTags map[string]string `json:"dist-tags"`
|
||||||
}
|
}
|
||||||
path := fmt.Sprintf("/api/packages/%s/npm/%s", url.PathEscape(f.Owner), url.PathEscape(name))
|
path := fmt.Sprintf("/api/packages/%s/npm/%s", url.PathEscape(f.Owner), url.PathEscape(name))
|
||||||
if err := f.getJSON(ctx, path, &meta); err != nil {
|
if err := f.GetJSON(ctx, path, &meta); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return meta.DistTags, nil
|
return meta.DistTags, nil
|
||||||
@@ -208,7 +124,7 @@ func (f *Forge) Repositories(ctx context.Context) ([]Repository, error) {
|
|||||||
var found struct {
|
var found struct {
|
||||||
Data []Repository `json:"data"`
|
Data []Repository `json:"data"`
|
||||||
}
|
}
|
||||||
if err := f.getJSON(ctx, fmt.Sprintf("/api/v1/repos/search?page=%d&limit=%d", page, pageSize), &found); err != nil {
|
if err := f.GetJSON(ctx, fmt.Sprintf("/api/v1/repos/search?page=%d&limit=%d", page, pageSize), &found); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
all = append(all, found.Data...)
|
all = append(all, found.Data...)
|
||||||
@@ -239,7 +155,7 @@ func (f *Forge) Files(ctx context.Context, repo Repository) ([]string, error) {
|
|||||||
}
|
}
|
||||||
path := fmt.Sprintf("/api/v1/repos/%s/git/trees/%s?recursive=true&page=%d&per_page=1000",
|
path := fmt.Sprintf("/api/v1/repos/%s/git/trees/%s?recursive=true&page=%d&per_page=1000",
|
||||||
repoPath(repo.FullName), url.PathEscape(repo.DefaultBranch), page)
|
repoPath(repo.FullName), url.PathEscape(repo.DefaultBranch), page)
|
||||||
if err := f.getJSON(ctx, path, &tree); err != nil {
|
if err := f.GetJSON(ctx, path, &tree); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
for _, e := range tree.Tree {
|
for _, e := range tree.Tree {
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ import (
|
|||||||
"sync"
|
"sync"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea/internal/forgehttp"
|
||||||
)
|
)
|
||||||
|
|
||||||
// fakeForge answers the forge's interface from maps, and records every delete.
|
// fakeForge answers the forge's interface from maps, and records every delete.
|
||||||
@@ -38,7 +40,7 @@ func (f *fakeForge) serve() *Forge {
|
|||||||
if err := os.WriteFile(file, []byte(f.password+"\n"), 0o600); err != nil {
|
if err := os.WriteFile(file, []byte(f.password+"\n"), 0o600); err != nil {
|
||||||
f.t.Fatal(err)
|
f.t.Fatal(err)
|
||||||
}
|
}
|
||||||
return &Forge{URL: srv.URL, User: "admin", PasswordFile: file, Owner: "acme"}
|
return &Forge{Client: forgehttp.Client{URL: srv.URL, User: "admin", PasswordFile: file}, Owner: "acme"}
|
||||||
}
|
}
|
||||||
|
|
||||||
var (
|
var (
|
||||||
@@ -501,7 +503,7 @@ func TestAnAnswerTheCapCutIsRefused(t *testing.T) {
|
|||||||
if err := os.WriteFile(file, []byte("x\n"), 0o600); err != nil {
|
if err := os.WriteFile(file, []byte("x\n"), 0o600); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
f := &Forge{URL: srv.URL, User: "admin", PasswordFile: file, Owner: "acme"}
|
f := &Forge{Client: forgehttp.Client{URL: srv.URL, User: "admin", PasswordFile: file}, Owner: "acme"}
|
||||||
if _, err := f.do(context.Background(), http.MethodGet, "/big"); err == nil || !strings.Contains(err.Error(), "not read whole") {
|
if _, err := f.do(context.Background(), http.MethodGet, "/big"); err == nil || !strings.Contains(err.Error(), "not read whole") {
|
||||||
t.Fatalf("a cut answer: %v", err)
|
t.Fatalf("a cut answer: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
+98
-13
@@ -164,21 +164,106 @@ export function judgeApiCall(method: string, rawPath: string, body: unknown): Ap
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Whether a protection rule's name — a branch, or a glob of them (`*`, `**`) — covers a branch. */
|
/**
|
||||||
|
* Whether a protection rule's name covers a branch, as the forge itself decides it (Gitea 1.27,
|
||||||
|
* models/git/protected_branch.go Match and modules/glob; novox/hq issue 539): a name without a glob character
|
||||||
|
* covers the branch of that name in any case; any other name is a glob with `/` as its separator — `*` any run
|
||||||
|
* within a segment, `**` any run across them, `?` one character of a segment, `[abc]`, `[a-z]` and `[!abc]` one
|
||||||
|
* character, `{a,b}` either, `\` the next character itself — matched case-sensitively; a glob the forge cannot
|
||||||
|
* compile covers its own text exactly. The forge bundle's ruleCovers (cmd/forge/rules.go) is the same rule, and
|
||||||
|
* both are held to one table of answers (test/rule-cases.json).
|
||||||
|
*
|
||||||
|
* The forge hands the compiled glob to Go's regexp, which reads some things JavaScript's reads otherwise: a
|
||||||
|
* backslash before a letter or digit (`\d`, `\w`, `\pL`, `\b`, `\A` are classes or anchors there), a class
|
||||||
|
* that opens with `]` (`[]a]` holds `]` and `a` there, an empty class here) or with `^` (a negation there),
|
||||||
|
* and a POSIX class (`[[:digit:]]`).
|
||||||
|
* A rule using any of them is taken to cover every branch: refusing a write the forge would take is safe,
|
||||||
|
* taking one it guards is not.
|
||||||
|
*/
|
||||||
export function ruleCovers(rule: string, branch: string): boolean {
|
export function ruleCovers(rule: string, branch: string): boolean {
|
||||||
if (rule === branch) return true;
|
if (!/[*?\\[\]{}]/.test(rule)) return rule.toLowerCase() === branch.toLowerCase();
|
||||||
if (!/[*?[]/.test(rule)) return false;
|
if (goReadsOtherwise(rule)) return true;
|
||||||
let re = "";
|
const re = globRegExp(rule);
|
||||||
for (let i = 0; i < rule.length; i++) {
|
return re ? re.test(branch) : rule === branch;
|
||||||
const c = rule[i];
|
}
|
||||||
if (c === "*" && rule[i + 1] === "*") {
|
|
||||||
re += ".*";
|
/** Whether a glob holds what Go's regexp reads otherwise than JavaScript's (see ruleCovers): fail closed. */
|
||||||
i++;
|
function goReadsOtherwise(rule: string): boolean {
|
||||||
} else if (c === "*") re += "[^/]*";
|
return /\\[\p{L}\p{N}]/u.test(rule) || /\[!?[\]^]/.test(rule) || /\[[^\]]*\[:/.test(rule);
|
||||||
else if (c === "?") re += "[^/]";
|
}
|
||||||
else re += c.replace(/[.+^${}()|[\]\\]/g, "\\$&");
|
|
||||||
|
/** The forge's glob as a regular expression, compiled as its modules/glob compiles it with `/` as separator. */
|
||||||
|
function globRegExp(glob: string): RegExp | null {
|
||||||
|
const g = Array.from(glob);
|
||||||
|
let out = "";
|
||||||
|
let pos = 0;
|
||||||
|
// A character as itself: escaped when it is syntax to a regular expression, and only then — the `u` flag
|
||||||
|
// refuses any other escape, and `-` may be escaped only inside a class.
|
||||||
|
const lit = (c: string) => c.replace(/[.*+?^${}()|[\]\\/]/g, "\\$&");
|
||||||
|
const litInClass = (c: string) => c.replace(/[\]\\[^-]/g, "\\$&");
|
||||||
|
const chars = (): boolean => {
|
||||||
|
out += "[";
|
||||||
|
if (g[pos] === "!") {
|
||||||
|
pos++;
|
||||||
|
out += "^";
|
||||||
|
}
|
||||||
|
while (pos < g.length) {
|
||||||
|
const c = g[pos++];
|
||||||
|
if (c === "]") {
|
||||||
|
out += "]";
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (c === "\\") {
|
||||||
|
if (pos >= g.length) return false;
|
||||||
|
out += litInClass(g[pos++]);
|
||||||
|
} else out += c === "[" || c === "^" ? "\\" + c : c;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
const compile = (sub: boolean): boolean => {
|
||||||
|
while (pos < g.length) {
|
||||||
|
const c = g[pos++];
|
||||||
|
if (sub && c === "}") {
|
||||||
|
out += ")";
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
switch (c) {
|
||||||
|
case "*":
|
||||||
|
if (g[pos] === "*") {
|
||||||
|
pos++;
|
||||||
|
out += ".*";
|
||||||
|
} else out += "[^/]*";
|
||||||
|
break;
|
||||||
|
case "?":
|
||||||
|
out += "[^/]";
|
||||||
|
break;
|
||||||
|
case "[":
|
||||||
|
if (!chars()) return false;
|
||||||
|
break;
|
||||||
|
case "{":
|
||||||
|
out += "(";
|
||||||
|
if (!compile(true)) return false;
|
||||||
|
break;
|
||||||
|
case ",":
|
||||||
|
out += sub ? "|" : ",";
|
||||||
|
break;
|
||||||
|
case "\\":
|
||||||
|
if (pos >= g.length) return false;
|
||||||
|
out += lit(g[pos++]);
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
out += lit(c);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// An alternative left open is a glob the forge cannot compile.
|
||||||
|
return !sub;
|
||||||
|
};
|
||||||
|
if (!compile(false)) return null;
|
||||||
|
try {
|
||||||
|
return new RegExp(`^${out}$`, "u");
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
}
|
}
|
||||||
return new RegExp(`^${re}$`).test(branch);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Refuses a file written to the default branch or to a branch a protection rule names. */
|
/** Refuses a file written to the default branch or to a branch a protection rule names. */
|
||||||
|
|||||||
@@ -0,0 +1,151 @@
|
|||||||
|
// Package forgehttp is the forge's HTTP interface as this module's Go bundles reach it (cmd/forge,
|
||||||
|
// cmd/npm-registry): the forge on its own machine, as its admin account, whose password the vault delivered
|
||||||
|
// into a file this module owns (novox/hq ADR 0086), by basic authentication. The password is read when a call
|
||||||
|
// is made and never logged, answered or put in an error. One copy for both bundles (novox/hq issue 539).
|
||||||
|
package forgehttp
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// MaxBody is the largest JSON answer read from the forge.
|
||||||
|
const MaxBody = 32 << 20
|
||||||
|
|
||||||
|
// Client is the forge's HTTP interface.
|
||||||
|
type Client struct {
|
||||||
|
URL string // e.g. http://127.0.0.1:3000
|
||||||
|
User string
|
||||||
|
PasswordFile string
|
||||||
|
HTTP *http.Client // when nil, a client with Timeout
|
||||||
|
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
|
||||||
|
}
|
||||||
|
|
||||||
|
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset.
|
||||||
|
func FromEnv() (Client, []string) {
|
||||||
|
c := Client{
|
||||||
|
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||||
|
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
||||||
|
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
||||||
|
}
|
||||||
|
var missing []string
|
||||||
|
for _, kv := range [][2]string{{"MESH_GITEA_URL", c.URL}, {"MESH_GITEA_ADMIN_USER", c.User},
|
||||||
|
{"MESH_GITEA_ADMIN_PASSWORD_FILE", c.PasswordFile}} {
|
||||||
|
if kv[1] == "" {
|
||||||
|
missing = append(missing, kv[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return c, missing
|
||||||
|
}
|
||||||
|
|
||||||
|
// ErrNotFound is the forge answering 404.
|
||||||
|
var ErrNotFound = errors.New("not found")
|
||||||
|
|
||||||
|
// StatusError is an answer the forge gave that is not a success, said without the request's credential.
|
||||||
|
type StatusError struct {
|
||||||
|
Method, Path string
|
||||||
|
Status int
|
||||||
|
Said string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *StatusError) Error() string {
|
||||||
|
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.Status, e.Method, e.Path, e.Said)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) client() *http.Client {
|
||||||
|
if c.HTTP != nil {
|
||||||
|
return c.HTTP
|
||||||
|
}
|
||||||
|
timeout := c.Timeout
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = 20 * time.Second
|
||||||
|
}
|
||||||
|
return &http.Client{Timeout: timeout}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) password() (string, error) {
|
||||||
|
raw, err := os.ReadFile(c.PasswordFile)
|
||||||
|
if err != nil {
|
||||||
|
// The path is the manifest's, not a secret; the content never appears.
|
||||||
|
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
||||||
|
}
|
||||||
|
return strings.TrimSpace(string(raw)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Do sends one request, with payload as its JSON body when not nil, and answers the body, read up to limit;
|
||||||
|
// 404 is ErrNotFound. One byte past the limit is read, so a body the limit cut is told from one that fits it
|
||||||
|
// exactly: a cut body is refused, never read as if it were whole.
|
||||||
|
func (c *Client) Do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
|
||||||
|
password, err := c.password()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var body io.Reader
|
||||||
|
if payload != nil {
|
||||||
|
raw, err := json.Marshal(payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
body = bytes.NewReader(raw)
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, method, c.URL+path, body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
req.SetBasicAuth(c.User, password)
|
||||||
|
req.Header.Set("Accept", "application/json")
|
||||||
|
if payload != nil {
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
}
|
||||||
|
res, err := c.client().Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
||||||
|
}
|
||||||
|
defer res.Body.Close()
|
||||||
|
got, err := io.ReadAll(io.LimitReader(res.Body, limit+1))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if int64(len(got)) > limit {
|
||||||
|
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case res.StatusCode == http.StatusNotFound:
|
||||||
|
return nil, fmt.Errorf("%s %s: %w", method, path, ErrNotFound)
|
||||||
|
case res.StatusCode >= 300:
|
||||||
|
said := strings.TrimSpace(string(got))
|
||||||
|
if len(said) > 300 {
|
||||||
|
said = said[:300] + "…"
|
||||||
|
}
|
||||||
|
return nil, &StatusError{method, path, res.StatusCode, said}
|
||||||
|
}
|
||||||
|
return got, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetJSON reads one answer into into.
|
||||||
|
func (c *Client) GetJSON(ctx context.Context, path string, into any) error {
|
||||||
|
return c.SendJSON(ctx, http.MethodGet, path, nil, into)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SendJSON sends one request and reads its answer into into, when into is not nil.
|
||||||
|
func (c *Client) SendJSON(ctx context.Context, method, path string, payload, into any) error {
|
||||||
|
body, err := c.Do(ctx, method, path, payload, MaxBody)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if into == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, into); err != nil {
|
||||||
|
return fmt.Errorf("the forge's answer to %s %s is not readable: %w", method, path, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@
|
|||||||
import { test, after, beforeEach } from "node:test";
|
import { test, after, beforeEach } from "node:test";
|
||||||
import assert from "node:assert/strict";
|
import assert from "node:assert/strict";
|
||||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||||
|
import { readFileSync } from "node:fs";
|
||||||
|
|
||||||
import { GiteaClient } from "../dist/client.js";
|
import { GiteaClient } from "../dist/client.js";
|
||||||
import { ConfiguredToken } from "../dist/token.js";
|
import { ConfiguredToken } from "../dist/token.js";
|
||||||
@@ -146,6 +147,25 @@ test("gitea_api refuses a file written to the default branch or one a rule guard
|
|||||||
assert.deepEqual(writes(), ["PUT /repos/novox/mesh-catalog/contents/modules/sudo/module.json"]);
|
assert.deepEqual(writes(), ["PUT /repos/novox/mesh-catalog/contents/modules/sudo/module.json"]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("a protection rule covers a branch as the forge decides it: classes, alternatives, a plain name in any case (issue 539)", async () => {
|
||||||
|
// One table with the forge bundle (cmd/forge/rules.go): `forge` is the forge's answer; `guard` is where this
|
||||||
|
// guard deliberately covers more — a rule Go's regexp reads otherwise than JavaScript's — and never less.
|
||||||
|
const cases = JSON.parse(readFileSync(new URL("./rule-cases.json", import.meta.url), "utf8")) as
|
||||||
|
Array<{ rule: string; branch: string; forge: boolean; guard?: boolean }>;
|
||||||
|
assert.ok(cases.length >= 100);
|
||||||
|
for (const c of cases) {
|
||||||
|
assert.ok(c.guard === undefined || (c.guard && !c.forge), `${c.rule}: the guard may only cover more than the forge`);
|
||||||
|
assert.equal(ruleCovers(c.rule, c.branch), c.guard ?? c.forge, `${c.rule} covers ${c.branch}`);
|
||||||
|
}
|
||||||
|
const put = (branch: string) => tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/contents/README.md",
|
||||||
|
method: "PUT", body: { content: "e30=", sha: "x", branch } });
|
||||||
|
state.rules.push({ rule_name: "release-[ab]" }, { rule_name: "{stable,lts}/*" });
|
||||||
|
await assert.rejects(put("release-a"), /protection rule release-\[ab\]/);
|
||||||
|
await assert.rejects(put("lts/2"), /protection rule \{stable,lts\}\/\*/);
|
||||||
|
await assert.rejects(put("MAIN"), /protection rule main/);
|
||||||
|
assert.deepEqual(writes(), []);
|
||||||
|
});
|
||||||
|
|
||||||
test("gitea_api still writes what an author writes", async () => {
|
test("gitea_api still writes what an author writes", async () => {
|
||||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/issues/3/comments", method: "POST", body: { body: "hi" } });
|
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/issues/3/comments", method: "POST", body: { body: "hi" } });
|
||||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/pulls/7/reviews", method: "POST", body: { event: "COMMENT" } });
|
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/pulls/7/reviews", method: "POST", body: { event: "COMMENT" } });
|
||||||
|
|||||||
@@ -0,0 +1,285 @@
|
|||||||
|
[
|
||||||
|
{"rule":"main","branch":"main","forge":true,"why":"a plain name covers its branch"},
|
||||||
|
{"rule":"main","branch":"mainx","forge":false},
|
||||||
|
{"rule":"main","branch":"MAIN","forge":true,"why":"a plain name covers in any case (EqualFold)"},
|
||||||
|
{"rule":"Main","branch":"main","forge":true},
|
||||||
|
{"rule":"release/*","branch":"release/1.0","forge":true,"why":"* stays within a segment"},
|
||||||
|
{"rule":"release/*","branch":"release/a/b","forge":false},
|
||||||
|
{"rule":"release/**","branch":"release/a/b","forge":true,"why":"** crosses segments"},
|
||||||
|
{"rule":"v?","branch":"v1","forge":true},
|
||||||
|
{"rule":"v?","branch":"v/","forge":false,"why":"? is not the separator"},
|
||||||
|
{"rule":"a.b","branch":"axb","forge":false,"why":". is itself"},
|
||||||
|
{"rule":"Feat/*","branch":"feat/x","forge":false,"why":"a glob is matched with case"},
|
||||||
|
{"rule":"release-[ab]","branch":"release-a","forge":true,"why":"a class"},
|
||||||
|
{"rule":"release-[ab]","branch":"release-c","forge":false},
|
||||||
|
{"rule":"release-[!ab]","branch":"release-c","forge":true,"why":"a negated class"},
|
||||||
|
{"rule":"release-[!ab]","branch":"release-a","forge":false},
|
||||||
|
{"rule":"v[0-9].x","branch":"v7.x","forge":true,"why":"a range"},
|
||||||
|
{"rule":"v[0-9].x","branch":"vA.x","forge":false},
|
||||||
|
{"rule":"{main,master}","branch":"master","forge":true,"why":"alternatives"},
|
||||||
|
{"rule":"{main,master}","branch":"mainline","forge":false},
|
||||||
|
{"rule":"rel/{a,b}/*","branch":"rel/b/1","forge":true},
|
||||||
|
{"rule":"rel/{a,b}/*","branch":"rel/c/1","forge":false},
|
||||||
|
{"rule":"a\\*","branch":"a*","forge":true,"why":"an escape"},
|
||||||
|
{"rule":"a\\*","branch":"ab","forge":false},
|
||||||
|
{"rule":"a+b","branch":"a+b","forge":true,"why":"+ is itself"},
|
||||||
|
{"rule":"a+b","branch":"aab","forge":false},
|
||||||
|
{"rule":"{open","branch":"{open","forge":true,"why":"a glob the forge cannot compile covers its own text"},
|
||||||
|
{"rule":"[open","branch":"[open","forge":true},
|
||||||
|
{"rule":"[open","branch":"o","forge":false},
|
||||||
|
{"rule":"release-\\d","branch":"release-1","forge":true,"why":"Go reads \\d as a digit"},
|
||||||
|
{"rule":"release-\\d","branch":"release-d","forge":false,"guard":true},
|
||||||
|
{"rule":"v\\w*","branch":"v2","forge":true},
|
||||||
|
{"rule":"main\\s","branch":"main","forge":false,"guard":true},
|
||||||
|
{"rule":"[]a]","branch":"a","forge":true,"why":"a class opening with ] holds ]"},
|
||||||
|
{"rule":"[]a]","branch":"]","forge":true},
|
||||||
|
{"rule":"[]a]","branch":"b","forge":false,"guard":true},
|
||||||
|
{"rule":"rel-[\\d]","branch":"rel-1","forge":true},
|
||||||
|
{"rule":"v[\\pL]","branch":"va","forge":true,"why":"\\p is a Unicode class"},
|
||||||
|
{"rule":"v\\pL","branch":"va","forge":true},
|
||||||
|
{"rule":"[[:alpha:]]","branch":"a","forge":true,"why":"a POSIX class"},
|
||||||
|
{"rule":"rel-[[:digit:]]","branch":"rel-1","forge":true},
|
||||||
|
{"rule":"rel-[[:digit:]]","branch":"rel-x","forge":false,"guard":true},
|
||||||
|
{"rule":"[^+]*","branch":"-[1","forge":true,"why":"a class opening with ^ is negated in Go"},
|
||||||
|
{"rule":"d[^](","branch":"d[^](","forge":true},
|
||||||
|
{"rule":"]?","branch":":,1},","forge":false},
|
||||||
|
{"rule":"Ab,","branch":"Ab,","forge":true},
|
||||||
|
{"rule":"\\:[}-|","branch":",{","forge":false},
|
||||||
|
{"rule":"^,*.d::","branch":"/x","forge":false},
|
||||||
|
{"rule":"","branch":"A*:","forge":false},
|
||||||
|
{"rule":"!|A{","branch":":.{]","forge":false},
|
||||||
|
{"rule":"","branch":"]éb","forge":false},
|
||||||
|
{"rule":"/{)w","branch":"/{)w","forge":true},
|
||||||
|
{"rule":"*^w]","branch":"}!.","forge":false},
|
||||||
|
{"rule":"*),^?*d","branch":"*),^?*d","forge":true},
|
||||||
|
{"rule":"A)^}!","branch":"","forge":false},
|
||||||
|
{"rule":"}^d?","branch":"1\\.","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"$?[","branch":"$?[","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"+","branch":"+","forge":true},
|
||||||
|
{"rule":"","branch":"},[","forge":false},
|
||||||
|
{"rule":"xA[d:a)","branch":"]1.!bx","forge":false},
|
||||||
|
{"rule":"^/}(","branch":"","forge":false},
|
||||||
|
{"rule":"-|x.x1?","branch":"-|x.x1?","forge":true},
|
||||||
|
{"rule":"x+)A]-A","branch":"x+)A]-A","forge":true},
|
||||||
|
{"rule":"x*a","branch":"x*a","forge":true},
|
||||||
|
{"rule":"$xb","branch":"}é","forge":false},
|
||||||
|
{"rule":"wAx?/a,","branch":"","forge":false},
|
||||||
|
{"rule":"[,","branch":"[,","forge":true},
|
||||||
|
{"rule":"}!","branch":"}!","forge":true},
|
||||||
|
{"rule":"*:|\\w","branch":"","forge":false,"guard":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"é","branch":"é","forge":true},
|
||||||
|
{"rule":"..(","branch":"{\\{+","forge":false},
|
||||||
|
{"rule":"*|{","branch":"1.}-","forge":false},
|
||||||
|
{"rule":"!+,","branch":"}++-*","forge":false},
|
||||||
|
{"rule":"x\\({","branch":"*","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"?-{+-","branch":"?-{+-","forge":true},
|
||||||
|
{"rule":"/-{é|$","branch":"b!.[","forge":false},
|
||||||
|
{"rule":"a-+dxA","branch":"1AA,xA","forge":false},
|
||||||
|
{"rule":"]|(","branch":"]|(","forge":true},
|
||||||
|
{"rule":"}","branch":"{!x{d","forge":false},
|
||||||
|
{"rule":"/{*](","branch":"!]","forge":false},
|
||||||
|
{"rule":"($+{-}|","branch":"/","forge":false},
|
||||||
|
{"rule":"(-b.-1}","branch":"*A*+,1","forge":false},
|
||||||
|
{"rule":"1[d|)wa","branch":"+}[!}}","forge":false},
|
||||||
|
{"rule":"w","branch":":.","forge":false},
|
||||||
|
{"rule":"?}\\*","branch":"","forge":false},
|
||||||
|
{"rule":"?$*","branch":"?$*","forge":true},
|
||||||
|
{"rule":"","branch":"!/{béa","forge":false},
|
||||||
|
{"rule":"","branch":"A1","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"!a$1/b{","branch":"!a$1/b{","forge":true},
|
||||||
|
{"rule":":dww/|d","branch":":dww/|d","forge":true},
|
||||||
|
{"rule":"!|*}","branch":"xéxé{b","forge":false},
|
||||||
|
{"rule":"1é||","branch":"a+.","forge":false},
|
||||||
|
{"rule":"/+],","branch":"!-\\[{é","forge":false},
|
||||||
|
{"rule":"a}(","branch":"a}(","forge":true},
|
||||||
|
{"rule":"|A?.","branch":"}.","forge":false},
|
||||||
|
{"rule":"d$a^d","branch":"*{","forge":false},
|
||||||
|
{"rule":"-ba$+A","branch":"-ba$+A","forge":true},
|
||||||
|
{"rule":"-.|,?a!","branch":"-.|,?a!","forge":true},
|
||||||
|
{"rule":"A","branch":"A","forge":true},
|
||||||
|
{"rule":"w","branch":"w","forge":true},
|
||||||
|
{"rule":".!1!+a","branch":".!1!+a","forge":true},
|
||||||
|
{"rule":"+$w:}","branch":"+$w:}","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":")]","branch":")]","forge":true},
|
||||||
|
{"rule":"a})^,}1","branch":"a})^,}1","forge":true},
|
||||||
|
{"rule":":[:!:","branch":":[:!:","forge":true},
|
||||||
|
{"rule":"w?w:]1!","branch":"\\.]-","forge":false},
|
||||||
|
{"rule":"*$/","branch":"*$/","forge":true},
|
||||||
|
{"rule":"é](","branch":"é](","forge":true},
|
||||||
|
{"rule":"x$a/","branch":"x$a/","forge":true},
|
||||||
|
{"rule":"*)b+[^","branch":"*)b+[^","forge":true},
|
||||||
|
{"rule":"/]/?","branch":"/]/?","forge":true},
|
||||||
|
{"rule":"{x,A","branch":"!Ab","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"-,","branch":"xé}A[.","forge":false},
|
||||||
|
{"rule":"w)Ad","branch":"w)Ad","forge":true},
|
||||||
|
{"rule":":??","branch":":??","forge":true},
|
||||||
|
{"rule":"x/xé|","branch":"x/xé|","forge":true},
|
||||||
|
{"rule":"","branch":"1!","forge":false},
|
||||||
|
{"rule":"!![é[","branch":"x","forge":false},
|
||||||
|
{"rule":"wb$","branch":"wb$","forge":true},
|
||||||
|
{"rule":")","branch":")","forge":true},
|
||||||
|
{"rule":"Aw:)(1","branch":"{b\\,","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"1*}?d","branch":"1*}?d","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"?-b.","branch":"?-b.","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"d","branch":",aé1","forge":false},
|
||||||
|
{"rule":"+","branch":"+{","forge":false},
|
||||||
|
{"rule":"]d","branch":"]}:/bA","forge":false},
|
||||||
|
{"rule":"*?+:|","branch":"!é.","forge":false},
|
||||||
|
{"rule":"}$d","branch":"}$d","forge":true},
|
||||||
|
{"rule":"/","branch":"/","forge":true},
|
||||||
|
{"rule":"a$+b)b","branch":"d}","forge":false},
|
||||||
|
{"rule":"?$[/A","branch":"","forge":false},
|
||||||
|
{"rule":"é){","branch":"é){","forge":true},
|
||||||
|
{"rule":"$(d","branch":"{é}","forge":false},
|
||||||
|
{"rule":"b(","branch":"b(","forge":true},
|
||||||
|
{"rule":"^^+]","branch":"^^+]","forge":true},
|
||||||
|
{"rule":",)","branch":",)","forge":true},
|
||||||
|
{"rule":"d-.A-","branch":"d-.A-","forge":true},
|
||||||
|
{"rule":"$","branch":"\\/","forge":false},
|
||||||
|
{"rule":"}+[\\|\\$","branch":"x[!b","forge":false},
|
||||||
|
{"rule":"(d1aé!é","branch":"A[-","forge":false},
|
||||||
|
{"rule":"x","branch":"1","forge":false},
|
||||||
|
{"rule":"x","branch":"dx","forge":false},
|
||||||
|
{"rule":"]|)]","branch":"]|)]","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"[.*?","branch":"[.*?","forge":true},
|
||||||
|
{"rule":"[?","branch":"[?","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"),/","branch":"a1,,1","forge":false},
|
||||||
|
{"rule":"}+","branch":"}+","forge":true},
|
||||||
|
{"rule":"$d\\,d","branch":"$d\\,d","forge":false},
|
||||||
|
{"rule":"d1","branch":"d1","forge":true},
|
||||||
|
{"rule":"-!^.:","branch":"-!^.:","forge":true},
|
||||||
|
{"rule":"x","branch":"+].A11","forge":false},
|
||||||
|
{"rule":"","branch":"-]","forge":false},
|
||||||
|
{"rule":"}]}.","branch":"","forge":false},
|
||||||
|
{"rule":"^{wx","branch":"]1a!","forge":false},
|
||||||
|
{"rule":"^","branch":"*]x,","forge":false},
|
||||||
|
{"rule":"(:x*x","branch":"","forge":false},
|
||||||
|
{"rule":"|[/*a/","branch":"}","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"1\\^","branch":"!é!x{","forge":false},
|
||||||
|
{"rule":"(.b,AA+","branch":"(.b,AA+","forge":true},
|
||||||
|
{"rule":"xx","branch":"","forge":false},
|
||||||
|
{"rule":"d)^):","branch":"/}!","forge":false},
|
||||||
|
{"rule":"|","branch":",é*]*[","forge":false},
|
||||||
|
{"rule":"d*d:*1","branch":"\\b,","forge":false},
|
||||||
|
{"rule":":]","branch":"[","forge":false},
|
||||||
|
{"rule":"([|!\\^é","branch":"([|!\\^é","forge":true},
|
||||||
|
{"rule":"$[w)x-*","branch":"","forge":false},
|
||||||
|
{"rule":")","branch":")","forge":true},
|
||||||
|
{"rule":".d^)?","branch":"x\\é","forge":false},
|
||||||
|
{"rule":"1|{w","branch":"1|{w","forge":true},
|
||||||
|
{"rule":"é$,(]$$","branch":"é$,(]$$","forge":true},
|
||||||
|
{"rule":"/","branch":"/","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"+","branch":"","forge":false},
|
||||||
|
{"rule":"]","branch":"","forge":false},
|
||||||
|
{"rule":"+:a:","branch":"+:a:","forge":true},
|
||||||
|
{"rule":"/*","branch":"d.}","forge":false},
|
||||||
|
{"rule":"|/ba1-","branch":"|/ba1-","forge":true},
|
||||||
|
{"rule":"","branch":"+","forge":false},
|
||||||
|
{"rule":"A","branch":"A","forge":true},
|
||||||
|
{"rule":",-!]","branch":",-!]","forge":true},
|
||||||
|
{"rule":"$*d!)","branch":"","forge":false},
|
||||||
|
{"rule":")é+x.d","branch":")é+x.d","forge":true},
|
||||||
|
{"rule":"éa$\\[1","branch":"1/,","forge":false},
|
||||||
|
{"rule":"}?1|]d","branch":"}?1|]d","forge":true},
|
||||||
|
{"rule":"[","branch":"[","forge":true},
|
||||||
|
{"rule":"-1.*+","branch":"-1.*+","forge":true},
|
||||||
|
{"rule":"Aw|{","branch":"Aw|{","forge":true},
|
||||||
|
{"rule":"adAAAa","branch":"A","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"d/^{:","branch":"d/^{:","forge":true},
|
||||||
|
{"rule":"é:(,^","branch":"é:(,^","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"!ébé({+","branch":"!ébé({+","forge":true},
|
||||||
|
{"rule":"x","branch":"x","forge":true},
|
||||||
|
{"rule":"|","branch":",:*","forge":false},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"x|","branch":"x|","forge":true},
|
||||||
|
{"rule":"a","branch":"aaxéb,","forge":false},
|
||||||
|
{"rule":"/w-),1é","branch":"/w-),1é","forge":true},
|
||||||
|
{"rule":"|*)^,","branch":"|*)^,","forge":true},
|
||||||
|
{"rule":"){é.!b.","branch":"a-}:","forge":false},
|
||||||
|
{"rule":"(\\","branch":"(\\","forge":true},
|
||||||
|
{"rule":".w(\\","branch":"+","forge":false},
|
||||||
|
{"rule":"/d[","branch":"/d[","forge":true},
|
||||||
|
{"rule":"(}$x!}","branch":"a,bb","forge":false},
|
||||||
|
{"rule":"w{","branch":"w{","forge":true},
|
||||||
|
{"rule":"[+a\\A(+","branch":"[+a\\A(+","forge":true},
|
||||||
|
{"rule":"(1^{)bb","branch":"(1^{)bb","forge":true},
|
||||||
|
{"rule":"^1","branch":"é-}","forge":false},
|
||||||
|
{"rule":"*,+}1):","branch":"*,+}1):","forge":true},
|
||||||
|
{"rule":"[A[(.x,","branch":"]db!*","forge":false},
|
||||||
|
{"rule":"","branch":"Ad}","forge":false},
|
||||||
|
{"rule":".","branch":".","forge":true},
|
||||||
|
{"rule":"w!é{x","branch":"é.{","forge":false},
|
||||||
|
{"rule":"$\\^","branch":"bx]+","forge":false},
|
||||||
|
{"rule":"1)","branch":"+*d","forge":false},
|
||||||
|
{"rule":"x)}/])","branch":"]","forge":false},
|
||||||
|
{"rule":",a}.1}","branch":",a}.1}","forge":true},
|
||||||
|
{"rule":"-a{(","branch":"-a{(","forge":true},
|
||||||
|
{"rule":"1","branch":"1","forge":true},
|
||||||
|
{"rule":"","branch":"","forge":true},
|
||||||
|
{"rule":"}[é$","branch":"]d-!","forge":false},
|
||||||
|
{"rule":"xé{é+?","branch":"xé{é+?","forge":true},
|
||||||
|
{"rule":"/A1!","branch":"xAA\\::","forge":false},
|
||||||
|
{"rule":"1][b-.(","branch":"{/-b![","forge":false},
|
||||||
|
{"rule":"(d}(","branch":"(d}(","forge":true},
|
||||||
|
{"rule":"{*}","branch":"!ébd","forge":true},
|
||||||
|
{"rule":"()^^)","branch":"()^^)","forge":true},
|
||||||
|
{"rule":"1(1A{$","branch":"*,[","forge":false},
|
||||||
|
{"rule":"A:!^\\","branch":"A:!^\\","forge":true},
|
||||||
|
{"rule":"-]:b]","branch":"-]:b]","forge":true},
|
||||||
|
{"rule":",","branch":",","forge":true},
|
||||||
|
{"rule":"[-:]","branch":"[-:]","forge":false},
|
||||||
|
{"rule":"","branch":"é:b1:d","forge":false},
|
||||||
|
{"rule":"1\\!]?d","branch":"","forge":false},
|
||||||
|
{"rule":"","branch":"bb!]+","forge":false},
|
||||||
|
{"rule":"/da","branch":"b]d[","forge":false},
|
||||||
|
{"rule":",]","branch":",]","forge":true},
|
||||||
|
{"rule":"[","branch":"[","forge":true},
|
||||||
|
{"rule":"a]{A-","branch":"-+","forge":false},
|
||||||
|
{"rule":"A?A^1","branch":"A?A^1","forge":true},
|
||||||
|
{"rule":"\\-$","branch":"","forge":false},
|
||||||
|
{"rule":"+","branch":"*[}[","forge":false},
|
||||||
|
{"rule":",b?{dw","branch":",b?{dw","forge":true},
|
||||||
|
{"rule":"d[$b1","branch":"/","forge":false},
|
||||||
|
{"rule":"})|xAA","branch":"é,!","forge":false},
|
||||||
|
{"rule":"]--","branch":"]--","forge":true},
|
||||||
|
{"rule":"],/b","branch":"],/b","forge":true},
|
||||||
|
{"rule":"/","branch":"/","forge":true},
|
||||||
|
{"rule":"*aé","branch":"*aé","forge":true},
|
||||||
|
{"rule":"{","branch":"{","forge":true},
|
||||||
|
{"rule":":aé","branch":"","forge":false},
|
||||||
|
{"rule":".\\}","branch":".\\}","forge":false},
|
||||||
|
{"rule":"wA]*","branch":"\\]!d{+","forge":false},
|
||||||
|
{"rule":"a/1x$/","branch":"[1!:/A","forge":false},
|
||||||
|
{"rule":"[.wb}d.","branch":"-!*..a","forge":false},
|
||||||
|
{"rule":".[{()a","branch":"+}","forge":false},
|
||||||
|
{"rule":"]/!:,","branch":"{Aéa","forge":false},
|
||||||
|
{"rule":"é{ba[-","branch":"],,1d/","forge":false},
|
||||||
|
{"rule":"(w*","branch":"1}.!","forge":false},
|
||||||
|
{"rule":"{?","branch":"1","forge":false},
|
||||||
|
{"rule":")","branch":",+!","forge":false},
|
||||||
|
{"rule":":","branch":"{/+A][","forge":false},
|
||||||
|
{"rule":".1{)","branch":"a\\bA","forge":false},
|
||||||
|
{"rule":"w-1+x1-","branch":"*]1}bb","forge":false},
|
||||||
|
{"rule":"xb$$","branch":"éx:]-:","forge":false},
|
||||||
|
{"rule":"++$!?*$","branch":"bA","forge":false},
|
||||||
|
{"rule":"]{","branch":"[xd*x.","forge":false}
|
||||||
|
]
|
||||||
@@ -65,6 +65,11 @@ type ChecksOf struct {
|
|||||||
Since string `json:"since"`
|
Since string `json:"since"`
|
||||||
Waits string `json:"waits,omitempty"`
|
Waits string `json:"waits,omitempty"`
|
||||||
Group string `json:"group,omitempty"`
|
Group string `json:"group,omitempty"`
|
||||||
|
// Class, Opened and MergedAt are what the issue tracker measures a fix's time by (novox/hq ADR 0313): the
|
||||||
|
// class of what it moves, when its pull request was first heard, and when it merged.
|
||||||
|
Class string `json:"class,omitempty"`
|
||||||
|
Opened string `json:"opened,omitempty"`
|
||||||
|
MergedAt string `json:"merged_at,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Checks is the `checks` verb: a repository and a pull request's number, or a repository and a commit.
|
// Checks is the `checks` verb: a repository and a pull request's number, or a repository and a commit.
|
||||||
@@ -104,7 +109,13 @@ func (h *Holder) Checks(repository string, number int, commit string) (*ChecksAn
|
|||||||
}
|
}
|
||||||
if d != nil {
|
if d != nil {
|
||||||
a.Delivery = &ChecksOf{ID: d.ID, State: d.State, Since: d.Since.UTC().Format(time.RFC3339), Waits: d.waits,
|
a.Delivery = &ChecksOf{ID: d.ID, State: d.State, Since: d.Since.UTC().Format(time.RFC3339), Waits: d.waits,
|
||||||
Group: d.Group}
|
Group: d.Group, Class: fixClass(&d.Delivery)}
|
||||||
|
if !d.opened.IsZero() {
|
||||||
|
a.Delivery.Opened = d.opened.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
if !d.MergedAt.IsZero() {
|
||||||
|
a.Delivery.MergedAt = d.MergedAt.UTC().Format(time.RFC3339)
|
||||||
|
}
|
||||||
if d.Check != nil && sameCommit(d.Commit, a.Commit) {
|
if d.Check != nil && sameCommit(d.Commit, a.Commit) {
|
||||||
a.Verdict = d.Check
|
a.Verdict = d.Check
|
||||||
}
|
}
|
||||||
@@ -139,6 +150,7 @@ type checksDelivery struct {
|
|||||||
Delivery
|
Delivery
|
||||||
waits string
|
waits string
|
||||||
groupSaid string
|
groupSaid string
|
||||||
|
opened time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// deliveryOf is the delivery of a commit — its head, or the commit it merged as — else, for a pull request,
|
// deliveryOf is the delivery of a commit — its head, or the commit it merged as — else, for a pull request,
|
||||||
@@ -167,7 +179,7 @@ func (h *Holder) deliveryOf(repository string, number int, commit string) *check
|
|||||||
if found == nil {
|
if found == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
c := &checksDelivery{Delivery: *found, waits: h.waitsFor(found)}
|
c := &checksDelivery{Delivery: *found, waits: h.waitsFor(found), opened: h.openedOf(found)}
|
||||||
if found.Check != nil {
|
if found.Check != nil {
|
||||||
v := *found.Check
|
v := *found.Check
|
||||||
c.Check = &v
|
c.Check = &v
|
||||||
|
|||||||
@@ -14,8 +14,10 @@ import (
|
|||||||
// the forge's holder, asked through its tools). Owed, never lost: kept with the delivery before anything is
|
// the forge's holder, asked through its tools). Owed, never lost: kept with the delivery before anything is
|
||||||
// tried, and tried again until done.
|
// tried, and tried again until done.
|
||||||
|
|
||||||
// transitionEvent is what `mesh-delivery.transition` says: no secret, no address.
|
// transitionEvent is what `mesh-delivery.transition` says: no secret, no address. opened is when its pull
|
||||||
func transitionEvent(d *Delivery, t Transition) map[string]any {
|
// request was first heard (zero when not known); with its class and its merge's moment it is what the issue
|
||||||
|
// tracker measures a fix's time by (novox/hq ADR 0313 §2, §4).
|
||||||
|
func transitionEvent(d *Delivery, t Transition, opened time.Time) map[string]any {
|
||||||
e := map[string]any{"id": d.ID, "repository": d.Repository, "commit": d.Commit, "from": t.From, "to": t.To,
|
e := map[string]any{"id": d.ID, "repository": d.Repository, "commit": d.Commit, "from": t.From, "to": t.To,
|
||||||
"event": t.Event, "why": t.Why, "at": t.At}
|
"event": t.Event, "why": t.Why, "at": t.At}
|
||||||
if t.By != "" {
|
if t.By != "" {
|
||||||
@@ -36,9 +38,57 @@ func transitionEvent(d *Delivery, t Transition) map[string]any {
|
|||||||
if d.Plan != nil {
|
if d.Plan != nil {
|
||||||
e["plan"] = d.Plan.Summary
|
e["plan"] = d.Plan.Summary
|
||||||
}
|
}
|
||||||
|
if c := fixClass(d); c != "" {
|
||||||
|
e["class"] = c
|
||||||
|
}
|
||||||
|
if !opened.IsZero() {
|
||||||
|
e["opened"] = opened.UTC()
|
||||||
|
}
|
||||||
|
if !d.MergedAt.IsZero() {
|
||||||
|
e["merged_at"] = d.MergedAt.UTC()
|
||||||
|
}
|
||||||
return e
|
return e
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// fixClass is the class of what a delivery moves (novox/hq ADR 0313 §2): its plan's, as the planner said it;
|
||||||
|
// none for a plan that moves no module, from a planner that did not say a class; else its walk's class once its
|
||||||
|
// delivery time is known; empty when nothing says it.
|
||||||
|
func fixClass(d *Delivery) string {
|
||||||
|
switch {
|
||||||
|
case d.Plan != nil && d.Plan.Class != "":
|
||||||
|
return d.Plan.Class
|
||||||
|
case d.Plan != nil && len(d.Plan.Moved) == 0 && len(d.Plan.Dependents) == 0 && len(d.Plan.New) == 0:
|
||||||
|
return classNone
|
||||||
|
case d.Time != nil:
|
||||||
|
return d.Time.Class
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// classNone is the class of a delivery that moves no module: nothing for a walk to move, and no fix budget.
|
||||||
|
const classNone = "none"
|
||||||
|
|
||||||
|
// openedOf is when a delivery's pull request was first heard: the earliest head of it this owner holds, never
|
||||||
|
// one made from the forge's word at its merge; zero when none is held or it has no pull request. Under the lock.
|
||||||
|
func (h *Holder) openedOf(d *Delivery) time.Time {
|
||||||
|
if d.Number <= 0 {
|
||||||
|
return time.Time{}
|
||||||
|
}
|
||||||
|
var first time.Time
|
||||||
|
for _, c := range h.deliveries {
|
||||||
|
if c.FromForge || c.Number != d.Number || !strings.EqualFold(c.Repository, d.Repository) || c.Created.IsZero() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if first.IsZero() || c.Created.Before(first) {
|
||||||
|
first = c.Created
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if first.IsZero() && !d.FromForge {
|
||||||
|
first = d.Created
|
||||||
|
}
|
||||||
|
return first
|
||||||
|
}
|
||||||
|
|
||||||
// noteLine is one transition as the commit's note keeps it.
|
// noteLine is one transition as the commit's note keeps it.
|
||||||
func noteLine(d *Delivery, t Transition) string {
|
func noteLine(d *Delivery, t Transition) string {
|
||||||
by := ""
|
by := ""
|
||||||
@@ -104,6 +154,9 @@ func statusOf(d *Delivery) (string, string) {
|
|||||||
case Superseded:
|
case Superseded:
|
||||||
return "warning", clip("superseded: "+why, 140)
|
return "warning", clip("superseded: "+why, 140)
|
||||||
case Stopped:
|
case Stopped:
|
||||||
|
if d.coveredStop() {
|
||||||
|
return "success", clip(d.coveredStopSaid(), 140) // its change reached the machines (novox/hq issue 480)
|
||||||
|
}
|
||||||
return "error", clip("stopped: "+why, 140)
|
return "error", clip("stopped: "+why, 140)
|
||||||
case Retired:
|
case Retired:
|
||||||
return "success", "history: merged before this owner delivered anything, never walked"
|
return "success", "history: merged before this owner delivered anything, never walked"
|
||||||
@@ -124,6 +177,9 @@ const viewMarker = "<!-- mesh-delivery:view -->"
|
|||||||
func ViewBody(d *Delivery, g *Group) string {
|
func ViewBody(d *Delivery, g *Group) string {
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
fmt.Fprintf(&b, "%s\n**Delivery** `%s` — **%s** since %s\n\n", viewMarker, d.ID, d.State, d.Since.UTC().Format(time.RFC3339))
|
fmt.Fprintf(&b, "%s\n**Delivery** `%s` — **%s** since %s\n\n", viewMarker, d.ID, d.State, d.Since.UTC().Format(time.RFC3339))
|
||||||
|
if d.coveredStop() {
|
||||||
|
fmt.Fprintf(&b, "**Covered** after its stop: a later merge that contains it was delivered: %s\n\n", d.CoveredBy)
|
||||||
|
}
|
||||||
if d.HeldWhy != "" && d.State == Held {
|
if d.HeldWhy != "" && d.State == Held {
|
||||||
fmt.Fprintf(&b, "Held for a person: %s — `mesh-delivery.release` with why lets it go on.\n\n", d.HeldWhy)
|
fmt.Fprintf(&b, "Held for a person: %s — `mesh-delivery.release` with why lets it go on.\n\n", d.HeldWhy)
|
||||||
}
|
}
|
||||||
@@ -278,6 +334,16 @@ func (h *Holder) Flush() {
|
|||||||
d.Owed = owed(d.Owed)
|
d.Owed = owed(d.Owed)
|
||||||
h.keep(d)
|
h.keep(d)
|
||||||
}
|
}
|
||||||
|
// Counted for delivery_status, so a window's writes can be judged against its tries (novox/hq issue 407).
|
||||||
|
h.tried.last = h.Now()
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
h.tried.done++
|
||||||
|
case dropped != "":
|
||||||
|
h.tried.givenUp++
|
||||||
|
default:
|
||||||
|
h.tried.failed++
|
||||||
|
}
|
||||||
if dropped != "" {
|
if dropped != "" {
|
||||||
h.refuse(fmt.Errorf("%s: its owed %s given up — %s: %v", j.id, j.e.Kind, dropped, err))
|
h.refuse(fmt.Errorf("%s: its owed %s given up — %s: %v", j.id, j.e.Kind, dropped, err))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -67,6 +67,16 @@ type Holder struct {
|
|||||||
// leaveRefused is, by walk, what the controller answered when it refused both to leave a stopped merge out and
|
// leaveRefused is, by walk, what the controller answered when it refused both to leave a stopped merge out and
|
||||||
// to stop the walk whole.
|
// to stop the walk whole.
|
||||||
leaveRefused map[string]string
|
leaveRefused map[string]string
|
||||||
|
// tried counts the owed effects Flush tried since this owner started, by how each ended (novox/hq issue 407).
|
||||||
|
tried owedTries
|
||||||
|
}
|
||||||
|
|
||||||
|
// owedTries is how many owed effects were tried since this owner started, and when the last was: three numbers
|
||||||
|
// and a time, however many were tried. Not kept: a restarted owner counts from its start, and says since when, so a
|
||||||
|
// window's count is the difference of two readings in the same run (novox/hq issue 407).
|
||||||
|
type owedTries struct {
|
||||||
|
done, failed, givenUp int
|
||||||
|
last time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
// unmatched is a walk no delivery is known for yet: its merge not heard, or merged before this owner held.
|
// unmatched is a walk no delivery is known for yet: its merge not heard, or merged before this owner held.
|
||||||
@@ -162,9 +172,10 @@ func (h *Holder) refuse(err error) {
|
|||||||
// moved records what a delivery's transitions owe outside, and keeps it.
|
// moved records what a delivery's transitions owe outside, and keeps it.
|
||||||
func (h *Holder) moved(d *Delivery, ts []Transition) {
|
func (h *Holder) moved(d *Delivery, ts []Transition) {
|
||||||
now := h.Now()
|
now := h.Now()
|
||||||
|
opened := h.openedOf(d)
|
||||||
for _, t := range ts {
|
for _, t := range ts {
|
||||||
h.Logf("[mesh-delivery] %s: %s → %s (%s): %s", d.ID, orNothing(t.From), t.To, t.Event, t.Why)
|
h.Logf("[mesh-delivery] %s: %s → %s (%s): %s", d.ID, orNothing(t.From), t.To, t.Event, t.Why)
|
||||||
d.Owed = append(d.Owed, Effect{Kind: EffectEmit, Line: string(mustJSON(transitionEvent(d, t))), Since: now})
|
d.Owed = append(d.Owed, Effect{Kind: EffectEmit, Line: string(mustJSON(transitionEvent(d, t, opened))), Since: now})
|
||||||
line := noteLine(d, t)
|
line := noteLine(d, t)
|
||||||
d.Owed = append(d.Owed, Effect{Kind: EffectNote, Commit: d.Commit, Line: line, Since: now})
|
d.Owed = append(d.Owed, Effect{Kind: EffectNote, Commit: d.Commit, Line: line, Since: now})
|
||||||
if d.MergedAs != "" && !sameCommit(d.MergedAs, d.Commit) && afterTheTrunk(t.To) {
|
if d.MergedAs != "" && !sameCommit(d.MergedAs, d.Commit) && afterTheTrunk(t.To) {
|
||||||
@@ -953,12 +964,16 @@ func (h *Holder) plan() []func() {
|
|||||||
// grace, it is stopped saying so — never left waiting for a start that cannot come.
|
// grace, it is stopped saying so — never left waiting for a start that cannot come.
|
||||||
for _, d := range h.deliveries {
|
for _, d := range h.deliveries {
|
||||||
ended := d.endedUnstarted()
|
ended := d.endedUnstarted()
|
||||||
|
if d.coveredStop() && !d.CoverTold {
|
||||||
|
h.tellCovered(d, now) // covered before this holder told it, or before a restart (novox/hq issue 480)
|
||||||
|
continue
|
||||||
|
}
|
||||||
if strandedUncovered(d) {
|
if strandedUncovered(d) {
|
||||||
// Stopped and final, but still read for a later merge that delivered its modules: until one did,
|
// Stopped and final, but still read for a later merge that delivered its modules: until one did,
|
||||||
// `stalled` keeps listing it (review of novox/hq issue 474).
|
// `stalled` keeps listing it (review of novox/hq issue 474).
|
||||||
var ask []string
|
var ask []string
|
||||||
if d.CoveredBy, ask = h.coverOf(d); d.CoveredBy != "" {
|
if d.CoveredBy, ask = h.coverOf(d); d.CoveredBy != "" {
|
||||||
h.keep(d)
|
h.tellCovered(d, now)
|
||||||
}
|
}
|
||||||
for _, in := range ask {
|
for _, in := range ask {
|
||||||
if a := h.askContains(d, in, now); a != nil {
|
if a := h.askContains(d, in, now); a != nil {
|
||||||
@@ -1103,6 +1118,25 @@ func (h *Holder) laterOnItsWay(d *Delivery) string {
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// tellCovered tells the forge that a stop is covered since, once: its status says covered and by which merge, its view
|
||||||
|
// says so, and its note keeps the line on its head and its merge commit (novox/hq issue 480). No transition: the stop
|
||||||
|
// stays final, and the bus's transition event is not said again.
|
||||||
|
func (h *Holder) tellCovered(d *Delivery, now time.Time) {
|
||||||
|
d.CoverTold = true
|
||||||
|
line := oneLine(fmt.Sprintf("%s delivery %s: stopped, %s", now.UTC().Format(time.RFC3339), d.ID, d.coveredStopSaid()))
|
||||||
|
d.Owed = append(d.Owed, Effect{Kind: EffectNote, Commit: d.Commit, Line: line, Since: now})
|
||||||
|
if d.MergedAs != "" && !sameCommit(d.MergedAs, d.Commit) {
|
||||||
|
d.Owed = append(d.Owed, Effect{Kind: EffectNote, Commit: d.MergedAs, Line: line, Since: now})
|
||||||
|
}
|
||||||
|
if d.shownOnItsPullRequest() {
|
||||||
|
d.owe(Effect{Kind: EffectView, Since: now})
|
||||||
|
state, desc := statusOf(d)
|
||||||
|
d.owe(Effect{Kind: EffectStatus, Commit: d.Commit, Context: "mesh/delivery", State: state, Line: desc, Since: now})
|
||||||
|
}
|
||||||
|
h.Logf("[mesh-delivery] %s: stopped, %s", d.ID, d.coveredStopSaid())
|
||||||
|
h.keep(d)
|
||||||
|
}
|
||||||
|
|
||||||
// strandedUncovered is whether a delivery stopped because its ended walk delivered nothing is still uncovered: a
|
// strandedUncovered is whether a delivery stopped because its ended walk delivered nothing is still uncovered: a
|
||||||
// merged change that never reached the machines, which `stalled` keeps listing until a later delivered merge
|
// merged change that never reached the machines, which `stalled` keeps listing until a later delivered merge
|
||||||
// carries its modules (review of novox/hq issue 474).
|
// carries its modules (review of novox/hq issue 474).
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// novox/hq issue 403, ADR 0313: the issue tracker measures a fix's time from the move to fixing to delivered and
|
||||||
|
// splits it at the pull request's opening and its merge. A transition says what it reads them by: when the pull
|
||||||
|
// request was first heard (its first head, not a later one), when it merged, and the class of what it moves —
|
||||||
|
// core, leaf, or none for a change that moves no module. `checks` says the same for a fix the tracker hears of late.
|
||||||
|
|
||||||
|
// transitionsTo are the transition events said for a delivery that moved to a state.
|
||||||
|
func transitionsTo(w *world, id string, to State) []map[string]any {
|
||||||
|
var out []map[string]any
|
||||||
|
for _, e := range w.said {
|
||||||
|
if e["event-kind"] == "transition" && e["id"] == id && e["to"] == string(to) {
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestATransitionSaysWhenItsPullRequestOpenedMergedAndItsClass(t *testing.T) {
|
||||||
|
w := newWorld(t)
|
||||||
|
repo := "novox/mesh-catalog"
|
||||||
|
opened := w.now
|
||||||
|
first := pr(repo, 9, "head9first", "fix/403-x", "modules/app/a.go")
|
||||||
|
w.h.PullUpdated(first)
|
||||||
|
w.later(3 * time.Minute)
|
||||||
|
second := pr(repo, 9, "head9second", "fix/403-x", "modules/app/a.go")
|
||||||
|
w.h.PullUpdated(second)
|
||||||
|
c := verdictMoving(repo, 9, "head9second", "app")
|
||||||
|
c.Plan.Class = "core"
|
||||||
|
w.h.Checked(c)
|
||||||
|
w.later(2 * time.Minute)
|
||||||
|
mergedWhen := w.now
|
||||||
|
w.h.PullMerged(mergedAt(second, "merge9aaaaaa", mergedWhen))
|
||||||
|
walk := aWalk("plan-merge9", repo, "merge9aaaaaa", false)
|
||||||
|
walk.Delivery = &WalkDelivery{Merges: []WalkMerge{{Repository: repo, Commit: "merge9aaaaaa"}}}
|
||||||
|
w.h.WalkMoved(w.ctl.put(walk))
|
||||||
|
w.settleAll()
|
||||||
|
walk = w.ctl.walk(walk.ID)
|
||||||
|
walk.State, walk.Revision = walkDone, walk.Revision+1
|
||||||
|
walk.Modules = map[string]*WalkModule{"app": {State: "built"}}
|
||||||
|
w.h.WalkMoved(w.ctl.put(walk))
|
||||||
|
w.settleAll()
|
||||||
|
|
||||||
|
id := IDOf(repo, "head9second")
|
||||||
|
if s := w.state(id); s != Delivered {
|
||||||
|
t.Fatalf("the merge is %s, not delivered", s)
|
||||||
|
}
|
||||||
|
said := transitionsTo(w, id, Delivered)
|
||||||
|
if len(said) == 0 {
|
||||||
|
t.Fatalf("no transition to delivered was said: %v", w.said)
|
||||||
|
}
|
||||||
|
e := said[len(said)-1]
|
||||||
|
if e["opened"] != opened.UTC().Format(time.RFC3339) {
|
||||||
|
t.Errorf("opened %v, want the first head's %s (not the second head's)", e["opened"], opened.Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
if e["merged_at"] != mergedWhen.UTC().Format(time.RFC3339) {
|
||||||
|
t.Errorf("merged_at %v, want %s", e["merged_at"], mergedWhen.Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
if e["class"] != "core" {
|
||||||
|
t.Errorf("class %v, want the plan's core", e["class"])
|
||||||
|
}
|
||||||
|
|
||||||
|
a, err := w.h.Checks(repo, 9, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if a.Delivery == nil || a.Delivery.Opened != opened.UTC().Format(time.RFC3339) || a.Delivery.Class != "core" ||
|
||||||
|
a.Delivery.MergedAt != mergedWhen.UTC().Format(time.RFC3339) {
|
||||||
|
t.Errorf("checks says %+v", a.Delivery)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A plan that moves no module is class none, so the tracker gives its fix no budget; a plan from a controller that
|
||||||
|
// says no class falls back to its walk's class once known, and says nothing before.
|
||||||
|
func TestAFixClassIsNoneWhenItMovesNothing(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
why string
|
||||||
|
d Delivery
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"the planner's class", Delivery{Plan: &DeliveryPlan{Moved: []string{"app"}, Class: "leaf"}}, "leaf"},
|
||||||
|
{"a plan that moves nothing, no class said", Delivery{Plan: planMoving("novox/x")}, "none"},
|
||||||
|
{"no class said, its walk's known", Delivery{Plan: planMoving("novox/x", "app"), Time: &DeliveryTime{Class: "core"}}, "core"},
|
||||||
|
{"no class said, nothing known", Delivery{Plan: planMoving("novox/x", "app")}, ""},
|
||||||
|
} {
|
||||||
|
if got := fixClass(&c.d); got != c.want {
|
||||||
|
t.Errorf("%s: %q, want %q", c.why, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// triedSaid is delivery_status's count of owed effects tried (novox/hq issue 407), read as a test reads it.
|
||||||
|
func triedSaid(t *testing.T, w *world) map[string]any {
|
||||||
|
t.Helper()
|
||||||
|
got, ok := w.h.Status("listening")["owed-tried"].(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("delivery_status says no owed-tried: %v", w.h.Status("listening"))
|
||||||
|
}
|
||||||
|
return got
|
||||||
|
}
|
||||||
|
|
||||||
|
func countOf(t *testing.T, m map[string]any, key string) int {
|
||||||
|
t.Helper()
|
||||||
|
n, ok := m[key].(int)
|
||||||
|
if !ok {
|
||||||
|
t.Fatalf("owed-tried has no count %q: %v", key, m)
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// novox/hq issue 407: delivery_status says how many owed effects were tried since this owner started, how
|
||||||
|
// many were done, failed to be tried again, or were given up, and when the last was tried — so a window's
|
||||||
|
// writes can be checked against the tries made in it (issue 326's rule) by reading it at both ends.
|
||||||
|
func TestDeliveryStatusCountsTheOwedEffectsTried(t *testing.T) {
|
||||||
|
w := newWorld(t)
|
||||||
|
start := w.now
|
||||||
|
tried := triedSaid(t, w)
|
||||||
|
if countOf(t, tried, "tried") != 0 || countOf(t, tried, "done") != 0 || countOf(t, tried, "failed") != 0 ||
|
||||||
|
countOf(t, tried, "given-up") != 0 {
|
||||||
|
t.Fatalf("a holder that tried nothing says %v", tried)
|
||||||
|
}
|
||||||
|
if tried["since"] != start.Format(time.RFC3339) {
|
||||||
|
t.Fatalf("its count is since %v, the holder started %s", tried["since"], start.Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
if _, there := tried["last"]; there {
|
||||||
|
t.Fatalf("nothing tried, yet a last try is said: %v", tried)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A head seen: its event and its note are owed and done.
|
||||||
|
w.h.PullUpdated(pr("novox/app", 7, head, "feat/x", "src/a.go"))
|
||||||
|
w.settleAll()
|
||||||
|
tried = triedSaid(t, w)
|
||||||
|
done := countOf(t, tried, "done")
|
||||||
|
if done < len(w.said)+len(w.forge.notesOn(head)) || done == 0 {
|
||||||
|
t.Fatalf("said %d and noted %d, yet done counts %d", len(w.said), len(w.forge.notesOn(head)), done)
|
||||||
|
}
|
||||||
|
if countOf(t, tried, "tried") != done || countOf(t, tried, "failed") != 0 {
|
||||||
|
t.Fatalf("every try succeeded, yet %v", tried)
|
||||||
|
}
|
||||||
|
if tried["last"] != w.now.Format(time.RFC3339) {
|
||||||
|
t.Fatalf("last try said %v, tried at %s", tried["last"], w.now.Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
|
||||||
|
// The forge away: its effects fail and are counted failed; quiet passes try nothing more.
|
||||||
|
w.forge.down = true
|
||||||
|
w.later(time.Minute)
|
||||||
|
w.h.Checked(verdict("novox/app", 7, head, "pass"))
|
||||||
|
w.h.Flush()
|
||||||
|
tried = triedSaid(t, w)
|
||||||
|
failed := countOf(t, tried, "failed")
|
||||||
|
if failed == 0 {
|
||||||
|
t.Fatalf("the forge away failed nothing: %v", tried)
|
||||||
|
}
|
||||||
|
if countOf(t, tried, "tried") != countOf(t, tried, "done")+failed+countOf(t, tried, "given-up") {
|
||||||
|
t.Fatalf("tries do not add up: %v", tried)
|
||||||
|
}
|
||||||
|
if tried["last"] != w.now.Format(time.RFC3339) {
|
||||||
|
t.Fatalf("last try said %v, tried at %s", tried["last"], w.now.Format(time.RFC3339))
|
||||||
|
}
|
||||||
|
before := countOf(t, tried, "tried")
|
||||||
|
w.h.Flush() // before each failure's next try: nothing is tried
|
||||||
|
if n := countOf(t, triedSaid(t, w), "tried"); n != before {
|
||||||
|
t.Fatalf("a pass before any next try tried %d", n-before)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The forge back, past every next try: the failed effects are done.
|
||||||
|
w.forge.down = false
|
||||||
|
w.later(time.Hour)
|
||||||
|
w.h.Flush()
|
||||||
|
tried = triedSaid(t, w)
|
||||||
|
if countOf(t, tried, "tried") != before+failed || countOf(t, tried, "done") <= done {
|
||||||
|
t.Fatalf("the forge back: tried %d (was %d, %d failed), %v", countOf(t, tried, "tried"), before, failed, tried)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The count is this run's: a restarted holder says it counts since its own start, from nothing.
|
||||||
|
func TestOwedTriedCountsSinceTheStart(t *testing.T) {
|
||||||
|
w := newWorld(t)
|
||||||
|
w.h.PullUpdated(pr("novox/app", 7, head, "feat/x", "src/a.go"))
|
||||||
|
w.settleAll()
|
||||||
|
if countOf(t, triedSaid(t, w), "tried") == 0 {
|
||||||
|
t.Fatal("nothing counted before the restart")
|
||||||
|
}
|
||||||
|
again := newWorldOver(t, w.store, w.ctl, w.forge, w.now.Add(time.Minute))
|
||||||
|
tried := triedSaid(t, again)
|
||||||
|
if countOf(t, tried, "tried") != 0 || tried["since"] != again.now.Format(time.RFC3339) {
|
||||||
|
t.Fatalf("a restarted holder says %v", tried)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// **An unanswered line says the state of the delivery it holds, as data** (novox/hq issue 451): for a delivery in
|
||||||
|
// ready or rejected, whose check answered once, the controller advises a recheck rather than the build queue, and
|
||||||
|
// it reads that state from the line, never from its words. A head with no delivery held says none.
|
||||||
|
func TestAnUnansweredLineNamesTheStateOfItsDelivery(t *testing.T) {
|
||||||
|
at := time.Date(2026, 10, 11, 0, 43, 0, 0, time.UTC)
|
||||||
|
for _, c := range []struct {
|
||||||
|
gate string
|
||||||
|
want State
|
||||||
|
}{{"pass", Ready}, {"fail", Rejected}} {
|
||||||
|
w := newWorldOver(t, newMemStore(), newFakeController(), newFakeForge(), at)
|
||||||
|
id := heldIn(t, w, c.gate, c.want)
|
||||||
|
w.forge.open = openPullsOf(t, readyRepoCheckPending)
|
||||||
|
w.h.SeeOpenPulls()
|
||||||
|
w.later(Bounds[Proposed].For + time.Minute)
|
||||||
|
w.h.SeeOpenPulls()
|
||||||
|
lines := w.h.Stalled()
|
||||||
|
if len(lines) != 1 || lines[0].ID != id || lines[0].State != Unanswered || lines[0].Held != c.want {
|
||||||
|
t.Fatalf("a %s delivery's unanswered line does not say its state as data: %+v", c.want, lines)
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(lines[0])
|
||||||
|
if !strings.Contains(string(raw), `"held":"`+string(c.want)+`"`) {
|
||||||
|
t.Fatalf("the line the controller reads carries no held state: %s", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The same head with no delivery held here: unanswered, and no state to say.
|
||||||
|
w := newWorldOver(t, newMemStore(), newFakeController(), newFakeForge(), at)
|
||||||
|
w.forge.open = openPullsOf(t, readyRepoCheckPending)
|
||||||
|
w.h.SeeOpenPulls()
|
||||||
|
w.later(Bounds[Proposed].For + time.Minute)
|
||||||
|
w.h.SeeOpenPulls()
|
||||||
|
lines := w.h.Stalled()
|
||||||
|
if len(lines) != 1 || lines[0].State != Unanswered || lines[0].Held != "" {
|
||||||
|
t.Fatalf("a head with no delivery held says a state: %+v", lines)
|
||||||
|
}
|
||||||
|
if raw, _ := json.Marshal(lines[0]); strings.Contains(string(raw), `"held"`) {
|
||||||
|
t.Fatalf("a line with no delivery held carries a held field: %s", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// novox/hq issue 480: a delivery stopped because its walk ended done before its word without building its modules
|
||||||
|
// (issue 474) is still read for a later merge that delivers them, and kept CoveredBy once one did. Its state stays
|
||||||
|
// stopped — a final state never moves, as for a covered superseded member (issue 442) — but the forge's status, its
|
||||||
|
// view, its note and `show` went on saying "stopped" for a change that reached every machine. Once covered they say
|
||||||
|
// covered, and by which merge. And while `stalled` lists such a stop, `deliveries` lists it too, past a day.
|
||||||
|
|
||||||
|
// strandedNight is theNight with #215 stopped, its walk past the grace with nothing later carrying build-agent.
|
||||||
|
func strandedNight(t *testing.T) (*world, string) {
|
||||||
|
w := theNight(t)
|
||||||
|
id := IDOf(ctl474, head215)
|
||||||
|
w.later(strandedAfter + 5*time.Minute)
|
||||||
|
w.settleAll()
|
||||||
|
if d := w.delivery(id); d.State != Stopped || d.Stranded == "" {
|
||||||
|
t.Fatalf("#215 is %s, stranded %q", d.State, d.Stranded)
|
||||||
|
}
|
||||||
|
return w, id
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAStopALaterMergeCoversReadsCoveredByThatMerge(t *testing.T) {
|
||||||
|
w, id := strandedNight(t)
|
||||||
|
if got := statusOn(w, ctl474, head215); !strings.HasPrefix(got, "error stopped: ") {
|
||||||
|
t.Fatalf("stopped, its status says %q", got)
|
||||||
|
}
|
||||||
|
w.forge.contains[merge215+" in "+merge219] = true
|
||||||
|
by := deliverLater(w, ctl474, 219, "main", head219, merge219, w.now, walkDone, "build-agent")
|
||||||
|
w.later(2 * time.Minute)
|
||||||
|
w.settleAll()
|
||||||
|
d := w.delivery(id)
|
||||||
|
if d.State != Stopped || !strings.Contains(d.CoveredBy, by) {
|
||||||
|
t.Fatalf("after #219 it is %s, covered by %q", d.State, d.CoveredBy)
|
||||||
|
}
|
||||||
|
got := statusOn(w, ctl474, head215)
|
||||||
|
if !strings.HasPrefix(got, "success covered") || !strings.Contains(got, by) {
|
||||||
|
t.Fatalf("covered by %s, its status still says %q", by, got)
|
||||||
|
}
|
||||||
|
shown, err := w.h.Show(id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if reads, _ := shown.(map[string]any)["reads"].(string); !strings.HasPrefix(reads, "covered") || !strings.Contains(reads, by) {
|
||||||
|
t.Fatalf("show reads %q", reads)
|
||||||
|
}
|
||||||
|
view := w.forge.views[ctl474+"#215"]
|
||||||
|
if !strings.Contains(view, "Covered") || !strings.Contains(view, by) {
|
||||||
|
t.Fatalf("its view does not say it was covered by %s:\n%s", by, view)
|
||||||
|
}
|
||||||
|
noted := false
|
||||||
|
for _, l := range w.forge.notesOn(merge215) {
|
||||||
|
noted = noted || strings.Contains(l, "covered") && strings.Contains(l, by)
|
||||||
|
}
|
||||||
|
if !noted {
|
||||||
|
t.Fatalf("its merge's note does not say it was covered: %v", w.forge.notesOn(merge215))
|
||||||
|
}
|
||||||
|
// Told once: further ticks owe nothing more.
|
||||||
|
w.later(time.Hour)
|
||||||
|
w.settleAll()
|
||||||
|
if d := w.delivery(id); len(d.Owed) != 0 || !d.CoverTold {
|
||||||
|
t.Fatalf("still owes %+v, told %v", d.Owed, d.CoverTold)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A stop covered before this was built (CoveredBy kept, nothing told) is told after a restart, from what was kept.
|
||||||
|
func TestACoveredStopKeptBeforeIsToldAfterARestart(t *testing.T) {
|
||||||
|
w, id := strandedNight(t)
|
||||||
|
w.h.mu.Lock()
|
||||||
|
d := w.h.deliveries[id]
|
||||||
|
d.CoveredBy = IDOf(ctl474, head219) + " (merged as " + merge219[:8] + ")"
|
||||||
|
if err := w.store.PutDelivery(d); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
w.h.mu.Unlock()
|
||||||
|
again := newWorldOver(t, w.store, w.ctl, w.forge, w.now.Add(time.Hour))
|
||||||
|
again.settleAll()
|
||||||
|
if got := statusOn(again, ctl474, head215); !strings.HasPrefix(got, "success covered") ||
|
||||||
|
!strings.Contains(got, IDOf(ctl474, head219)) {
|
||||||
|
t.Fatalf("after a restart its status says %q", got)
|
||||||
|
}
|
||||||
|
if d := again.delivery(id); d.State != Stopped || len(d.Transitions) != len(w.delivery(id).Transitions) {
|
||||||
|
t.Fatalf("a restart moved it: %s", d.State)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only a stop for a walk that never built its modules reads covered: a person's stop, or one the cover test has
|
||||||
|
// not answered, keeps its error.
|
||||||
|
func TestOnlyACoveredStrandedStopReadsCovered(t *testing.T) {
|
||||||
|
for _, d := range []*Delivery{
|
||||||
|
{State: Stopped, CoveredBy: "x"}, // a person's stop, with a cover found while it was held
|
||||||
|
{State: Stopped, Stranded: "never reached"}, // stranded, not covered
|
||||||
|
} {
|
||||||
|
if state, _ := statusOf(d); state != "error" {
|
||||||
|
t.Fatalf("%+v reads %s", d, state)
|
||||||
|
}
|
||||||
|
if d.coveredStop() {
|
||||||
|
t.Fatalf("%+v reads as a covered stop", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAStopStalledListsStaysInDeliveries(t *testing.T) {
|
||||||
|
w, id := strandedNight(t)
|
||||||
|
w.later(30 * time.Hour)
|
||||||
|
w.settleAll()
|
||||||
|
listed := func() *Line {
|
||||||
|
for _, l := range w.h.Deliveries("", "", "", false) {
|
||||||
|
if l.ID == id {
|
||||||
|
return &l
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if len(w.h.Stalled()) != 1 {
|
||||||
|
t.Fatalf("stalled lists %+v", w.h.Stalled())
|
||||||
|
}
|
||||||
|
l := listed()
|
||||||
|
if l == nil || !l.Stalled || !strings.Contains(l.Waits, "build-agent") {
|
||||||
|
t.Fatalf("listed by stalled, deliveries shows %+v", l)
|
||||||
|
}
|
||||||
|
// Covered, it ends like any final delivery: a day after its stop.
|
||||||
|
w.forge.contains[merge215+" in "+merge219] = true
|
||||||
|
deliverLater(w, ctl474, 219, "main", head219, merge219, w.now, walkDone, "build-agent")
|
||||||
|
w.later(2 * time.Minute)
|
||||||
|
w.settleAll()
|
||||||
|
if len(w.h.Stalled()) != 0 || listed() != nil {
|
||||||
|
t.Fatalf("covered, stalled lists %+v and deliveries %+v", w.h.Stalled(), listed())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -192,7 +192,7 @@ func tools(h *Holder, l *listening) []stdio.Tool {
|
|||||||
seat := "mesh-delivery."
|
seat := "mesh-delivery."
|
||||||
return []stdio.Tool{
|
return []stdio.Tool{
|
||||||
{Name: seat + "deliveries",
|
{Name: seat + "deliveries",
|
||||||
Description: "Every delivery not final, and those that ended in the last day, one line each: its id " +
|
Description: "Every delivery not final, those that ended in the last day, and any stop `stalled` still lists, one line each: its id " +
|
||||||
"(owner/repository@commit), its state, its group, what it waits for and since when. Narrowed by state, " +
|
"(owner/repository@commit), its state, its group, what it waits for and since when. Narrowed by state, " +
|
||||||
"repository or group.",
|
"repository or group.",
|
||||||
Input: map[string]any{"type": "object", "properties": map[string]any{"state": str("one state"),
|
Input: map[string]any{"type": "object", "properties": map[string]any{"state": str("one state"),
|
||||||
@@ -347,7 +347,8 @@ func tools(h *Holder, l *listening) []stdio.Tool {
|
|||||||
}},
|
}},
|
||||||
{Name: "delivery_status",
|
{Name: "delivery_status",
|
||||||
Description: "Whether this owner works: how many deliveries in each state, the groups, what is owed to " +
|
Description: "Whether this owner works: how many deliveries in each state, the groups, what is owed to " +
|
||||||
"the forge and the bus and not yet done, walks with no delivery yet, whether its events arrive, when " +
|
"the forge and the bus and not yet done, how many owed effects were tried since it started (done, failed " +
|
||||||
|
"and tried again, given up) and when the last was, walks with no delivery yet, whether its events arrive, when " +
|
||||||
"the controller's walks were last read or why they could not be, and the last refusals.",
|
"the controller's walks were last read or why they could not be, and the last refusals.",
|
||||||
Run: func(map[string]any) (any, error) { return h.Status(l.get()), nil }},
|
Run: func(map[string]any) (any, error) { return h.Status(l.get()), nil }},
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -68,6 +68,9 @@ type Delivery struct {
|
|||||||
// Stranded says why a merge whose walk ended done before its word, without moving every module it moves, is
|
// Stranded says why a merge whose walk ended done before its word, without moving every module it moves, is
|
||||||
// stopped: nothing later delivered them either (novox/hq issue 474). Set once the cover test says no.
|
// stopped: nothing later delivered them either (novox/hq issue 474). Set once the cover test says no.
|
||||||
Stranded string `json:"stranded,omitempty"`
|
Stranded string `json:"stranded,omitempty"`
|
||||||
|
// CoverTold is whether the forge was told that such a stop was covered since: its status, its view and its note
|
||||||
|
// (novox/hq issue 480). Kept, so a restart tells it once, and a stop covered before this was built is told too.
|
||||||
|
CoverTold bool `json:"cover_told,omitempty"`
|
||||||
// TurnAwaits says, for a group member whose turn waits for a later merge to carry a superseded member before it,
|
// TurnAwaits says, for a group member whose turn waits for a later merge to carry a superseded member before it,
|
||||||
// which merges and until when; TurnAwaitsUntil is that moment, which bounds the wait (ADR 0276 decision 9,
|
// which merges and until when; TurnAwaitsUntil is that moment, which bounds the wait (ADR 0276 decision 9,
|
||||||
// novox/hq issue 445). `show` shows both, and `stalled` says them past that moment.
|
// novox/hq issue 445). `show` shows both, and `stalled` says them past that moment.
|
||||||
@@ -162,6 +165,9 @@ type DeliveryPlan struct {
|
|||||||
Machines []MachinePlan `json:"machines,omitempty"`
|
Machines []MachinePlan `json:"machines,omitempty"`
|
||||||
Steps []string `json:"steps,omitempty"`
|
Steps []string `json:"steps,omitempty"`
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
|
// Class is the class of the slowest-budgeted module it moves, as the planner says it: core, leaf, or none when
|
||||||
|
// it moves no module (novox/hq ADR 0313 §2). Empty in a plan from a controller that did not say it.
|
||||||
|
Class string `json:"class,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// receiving are the machines the plan sends a module to, in the plan's order; none when the plan names no
|
// receiving are the machines the plan sends a module to, in the plan's order; none when the plan names no
|
||||||
@@ -248,6 +254,20 @@ func (d *Delivery) FromForgesWord() bool {
|
|||||||
|
|
||||||
// shownOnItsPullRequest is whether its view and status are kept on its pull request: not for history, whose
|
// shownOnItsPullRequest is whether its view and status are kept on its pull request: not for history, whose
|
||||||
// pull request the forge may no longer hold — or holds under its number another one.
|
// pull request the forge may no longer hold — or holds under its number another one.
|
||||||
|
// coveredStop is whether a delivery stopped because its walk ended done before its word without building its modules
|
||||||
|
// (novox/hq issue 474) was covered since: a later merge into its trunk that contains it was delivered, carrying every
|
||||||
|
// module it moves to every machine running it (ADR 0270's cover test, kept as CoveredBy). Its state stays stopped —
|
||||||
|
// final, as a covered superseded member stays superseded (issue 442) — and it reads covered, by that merge (novox/hq
|
||||||
|
// issue 480). A stop for any other reason never reads covered.
|
||||||
|
func (d *Delivery) coveredStop() bool {
|
||||||
|
return d.State == Stopped && d.Stranded != "" && d.CoveredBy != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// coveredStopSaid is how a covered stop reads: on the forge's status, in its view and note, and in `show`.
|
||||||
|
func (d *Delivery) coveredStopSaid() string {
|
||||||
|
return "covered after its stop: a later merge that contains it was delivered: " + d.CoveredBy
|
||||||
|
}
|
||||||
|
|
||||||
func (d *Delivery) shownOnItsPullRequest() bool {
|
func (d *Delivery) shownOnItsPullRequest() bool {
|
||||||
return d.Number > 0 && d.History == "" && d.State != Retired
|
return d.Number > 0 && d.History == "" && d.State != Retired
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -250,8 +250,8 @@ func (h *Holder) uncheckedLines(now time.Time) []StalledLine {
|
|||||||
h2 = "none: its delivery is " + string(u.held) + ", a state the operator moves — the operator's"
|
h2 = "none: its delivery is " + string(u.held) + ", a state the operator moves — the operator's"
|
||||||
}
|
}
|
||||||
out = append(out, StalledLine{ID: id, Number: u.pull.Number, State: state, For: now.Sub(u.first).Round(time.Second).String(),
|
out = append(out, StalledLine{ID: id, Number: u.pull.Number, State: state, For: now.Sub(u.first).Round(time.Second).String(),
|
||||||
Bound: u.bound().String(),
|
Bound: u.bound().String(), Held: u.held,
|
||||||
H2: h2, Says: says, Waiting: waiting})
|
H2: h2, Says: says, Waiting: waiting})
|
||||||
}
|
}
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -71,6 +71,11 @@ func (h *Holder) waitsFor(d *Delivery) string {
|
|||||||
if d.Walk != nil {
|
if d.Walk != nil {
|
||||||
return "its walk: " + clip(d.Walk.Note, 120)
|
return "its walk: " + clip(d.Walk.Note, 120)
|
||||||
}
|
}
|
||||||
|
case Stopped:
|
||||||
|
if strandedUncovered(d) {
|
||||||
|
return "a later delivered merge to carry " + strings.Join(d.notMovedBy(), ", ") + ": it is stopped, and " +
|
||||||
|
"listed by stalled until one does (novox/hq issue 480)"
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
@@ -86,7 +91,8 @@ func (h *Holder) Deliveries(state, repository, group string, all bool) []Line {
|
|||||||
group != "" && d.Group != group {
|
group != "" && d.Group != group {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if d.State.Final() && !all && now.Sub(d.Since) > 24*time.Hour {
|
// A final delivery `stalled` still lists stays, however old: the two never disagree (novox/hq issue 480).
|
||||||
|
if d.State.Final() && !all && now.Sub(d.Since) > 24*time.Hour && !strandedUncovered(d) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
l := Line{ID: d.ID, State: d.State, Since: d.Since.UTC().Format(time.RFC3339), For: now.Sub(d.Since).Round(time.Second).String(),
|
l := Line{ID: d.ID, State: d.State, Since: d.Since.UTC().Format(time.RFC3339), For: now.Sub(d.Since).Round(time.Second).String(),
|
||||||
@@ -97,7 +103,7 @@ func (h *Holder) Deliveries(state, repository, group string, all bool) []Line {
|
|||||||
if d.Walk != nil {
|
if d.Walk != nil {
|
||||||
l.Walk = d.Walk.ID
|
l.Walk = d.Walk.ID
|
||||||
}
|
}
|
||||||
l.Stalled = pastBound(d, now)
|
l.Stalled = pastBound(d, now) || strandedUncovered(d)
|
||||||
out = append(out, l)
|
out = append(out, l)
|
||||||
}
|
}
|
||||||
sort.Slice(out, func(i, j int) bool {
|
sort.Slice(out, func(i, j int) bool {
|
||||||
@@ -115,6 +121,9 @@ func (h *Holder) Show(id string) (any, error) {
|
|||||||
defer h.mu.Unlock()
|
defer h.mu.Unlock()
|
||||||
if d := h.deliveries[id]; d != nil {
|
if d := h.deliveries[id]; d != nil {
|
||||||
out := map[string]any{"delivery": d, "waits": h.waitsFor(d)}
|
out := map[string]any{"delivery": d, "waits": h.waitsFor(d)}
|
||||||
|
if d.coveredStop() {
|
||||||
|
out["reads"] = d.coveredStopSaid() // stopped, and its change reached the machines (novox/hq issue 480)
|
||||||
|
}
|
||||||
if g := h.groups[d.Group]; g != nil {
|
if g := h.groups[d.Group]; g != nil {
|
||||||
state, why := GroupState(g, h.membersOf(g))
|
state, why := GroupState(g, h.membersOf(g))
|
||||||
out["group"] = map[string]any{"group": g, "state": state, "why": why}
|
out["group"] = map[string]any{"group": g, "state": state, "why": why}
|
||||||
@@ -190,10 +199,14 @@ type StalledLine struct {
|
|||||||
// the operator's own zone, which it cannot do inside a finished sentence (novox/hq issue 443). Waiting stays
|
// the operator's own zone, which it cannot do inside a finished sentence (novox/hq issue 443). Waiting stays
|
||||||
// beside them for a controller that does not read them yet.
|
// beside them for a controller that does not read them yet.
|
||||||
Checks []WaitingCheck `json:"checks,omitempty"`
|
Checks []WaitingCheck `json:"checks,omitempty"`
|
||||||
For string `json:"for"`
|
// Held is the state of the delivery a line of the state Unanswered is about, ready or rejected, when one is held
|
||||||
Bound string `json:"bound"`
|
// here (novox/hq issues 441, 451): its check answered once, so the controller advises a recheck of it rather than
|
||||||
H2 string `json:"h2"`
|
// the build queue, reading the state here and never from Says. Empty for a head with no delivery.
|
||||||
Says string `json:"says"`
|
Held State `json:"held,omitempty"`
|
||||||
|
For string `json:"for"`
|
||||||
|
Bound string `json:"bound"`
|
||||||
|
H2 string `json:"h2"`
|
||||||
|
Says string `json:"says"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Stalled is the `stalled` verb: what the controller's self-check raises as `delivery.<id>.stalled`.
|
// Stalled is the `stalled` verb: what the controller's self-check raises as `delivery.<id>.stalled`.
|
||||||
@@ -635,6 +648,15 @@ func (h *Holder) Status(listening string) map[string]any {
|
|||||||
out := map[string]any{"deliveries": byState, "groups": len(h.groups), "owed": owed, "unkept": len(h.dirty),
|
out := map[string]any{"deliveries": byState, "groups": len(h.groups), "owed": owed, "unkept": len(h.dirty),
|
||||||
"walks-unmatched": len(h.unmatched), "events": listening, "started": h.started.Format(time.RFC3339),
|
"walks-unmatched": len(h.unmatched), "events": listening, "started": h.started.Format(time.RFC3339),
|
||||||
"seat-on-record": h.held}
|
"seat-on-record": h.held}
|
||||||
|
// The owed effects tried since this owner started (novox/hq issue 407): done, failed and to be tried again,
|
||||||
|
// or given up, and when the last was tried. A count since the start, said with that start: two readings in
|
||||||
|
// one run give a window's tries.
|
||||||
|
tried := map[string]any{"since": h.started.Format(time.RFC3339), "done": h.tried.done, "failed": h.tried.failed,
|
||||||
|
"given-up": h.tried.givenUp, "tried": h.tried.done + h.tried.failed + h.tried.givenUp}
|
||||||
|
if !h.tried.last.IsZero() {
|
||||||
|
tried["last"] = h.tried.last.Format(time.RFC3339)
|
||||||
|
}
|
||||||
|
out["owed-tried"] = tried
|
||||||
// Each held delivery by name, with why and the verb that releases it (novox/hq issue 529): a count alone
|
// Each held delivery by name, with why and the verb that releases it (novox/hq issue 529): a count alone
|
||||||
// pointed a reader at nothing.
|
// pointed a reader at nothing.
|
||||||
var held []string
|
var held []string
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ what it refuses.
|
|||||||
| `groom` | optionally `pass`, `confirm` | the triage over the backlog: without arguments a dry run in the background, answering its pass id; with `pass` its progress and findings; with `confirm` and `pass` exactly that pass's findings applied, each checked again; see below |
|
| `groom` | optionally `pass`, `confirm` | the triage over the backlog: without arguments a dry run in the background, answering its pass id; with `pass` its progress and findings; with `confirm` and `pass` exactly that pass's findings applied, each checked again; see below |
|
||||||
| `tracking` | optionally `conditions` (condition keys) | for every condition key an issue links (link kind `condition`), the issues that link it, those not closed first: status, severity, needs, worker with since, active and stale, and where the fix is (`fix`); the operator's channel says it in a condition's message (hq ADR 0285) |
|
| `tracking` | optionally `conditions` (condition keys) | for every condition key an issue links (link kind `condition`), the issues that link it, those not closed first: status, severity, needs, worker with since, active and stale, and where the fix is (`fix`); the operator's channel says it in a condition's message (hq ADR 0285) |
|
||||||
| `reserve` | `kind` (`decision` or `research`), optionally `for` (one line: what the record is about) | the next number of that kind, spent forever; answers `kind` and `number` (and `prefix`, the number as the record's name starts with it); see below |
|
| `reserve` | `kind` (`decision` or `research`), optionally `for` (one line: what the record is about) | the next number of that kind, spent forever; answers `kind` and `number` (and `prefix`, the number as the record's name starts with it); see below |
|
||||||
|
| `fix-times` | optionally `days` (1 to 90, default 7) | each class's fix budget, the median and worst fix time of the issues whose last fix was delivered in the window, within and over, and every issue over its budget with its longest phase; see *A fix's time* |
|
||||||
|
|
||||||
**The cycle's rules, at the write** (hq playbook 03, which `00-META/checks/cycle.py` enforced on the
|
**The cycle's rules, at the write** (hq playbook 03, which `00-META/checks/cycle.py` enforced on the
|
||||||
folders until the import, and hq ADR 0284): a status is one of `open`,
|
folders until the import, and hq ADR 0284): a status is one of `open`,
|
||||||
@@ -134,11 +135,46 @@ carries `abandoned` on `get` and its `list` row, beside its `worker`. A move by
|
|||||||
`take-over`'s reason, as any stale worker; the answer, the note and the `moved` event say it was abandoned
|
`take-over`'s reason, as any stale worker; the answer, the note and the `moved` event say it was abandoned
|
||||||
(`abandoned`).
|
(`abandoned`).
|
||||||
|
|
||||||
Every five minutes the module looks for a fix merged and not delivered after `stuck-after` (default `2h`): it is
|
Every five minutes the module looks for a fix merged and not delivered after `stuck-after` (default `2h`) of an
|
||||||
said once — a note on the issue and a warning on the operator's channel (`seat:operator-channel.notify`, key
|
issue not closed: it is said once — a note on the issue and a warning on the operator's channel
|
||||||
`issue-tracker.fix-stuck.<number>`), cleared there when every fix of the issue is delivered. A module that runs only
|
(`seat:operator-channel.notify`, key `issue-tracker.fix-stuck.<number>`), cleared there when every fix of the issue
|
||||||
|
is delivered or the issue is closed. Resolved is the end of the warning as wontfix is: its worker proved the
|
||||||
|
acceptance live, so a fix whose own walk failed and that a later merge carried is not waited for (hq issue 547). An
|
||||||
|
issue resolved or wontfix is never said stuck. A module that runs only
|
||||||
its bundle cannot raise a controller condition (hq ADR 0240 rule 8, ADR 0256), so the warning is the channel's, not the controller's.
|
its bundle cannot raise a controller condition (hq ADR 0240 rule 8, ADR 0256), so the warning is the channel's, not the controller's.
|
||||||
|
|
||||||
|
## A fix's time (hq ADR 0313)
|
||||||
|
|
||||||
|
An issue's **fix time** runs from its first move to `fixing` (kept as `fixing-from`; a take-over or a later move
|
||||||
|
back to `fixing` does not restart it) to the moment the last of its fixes is delivered. An issue moved before the
|
||||||
|
moment was kept has it read from its log: a move's own moment, or, for a value carried into the log after one at
|
||||||
|
another status, its worker's `active` (the move), never its worker's `since`, which a diagnosis first keeps. Such a
|
||||||
|
fix time says `from-log`: it is kept and shown, and never warned on, so the first fixes heard of after the tracker
|
||||||
|
starts measuring raise no warning for work done before it. Once every fix it names is delivered, `get` carries it as `fix-time`:
|
||||||
|
`from`, `end`, `took`, the issue's `class`, its **fix budget** (`budget`: five minutes for a leaf module, ten for a
|
||||||
|
core one, counted from the move to `fixing`), `over`, the fix delivered last (`fix`), the `worker`, and four
|
||||||
|
`phases` read from that fix, which add up to the fix time:
|
||||||
|
|
||||||
|
| phase | ends at | read from |
|
||||||
|
|---|---|---|
|
||||||
|
| implementing | its pull request opened | mesh-delivery's `opened` (the pull request's first head heard) |
|
||||||
|
| review | the first approval at its head | the forge's reviews (`gitea_pull_reviews`): the forge's own approval, or a comment review whose first line is `Verdict: approve at <sha>`, the sha the head it was posted at or a prefix of at least seven characters (the forge refuses an approval of one's own pull request, so the mesh's independent reviews are comments) |
|
||||||
|
| checks | its merge, required checks and any wait for the merge included | mesh-delivery's `merged_at` |
|
||||||
|
| merge to running | delivered | the delivery's `delivered` (its delivery time) |
|
||||||
|
|
||||||
|
A phase whose end came before the end of the phase before it counts zero; a phase whose end is not known (no
|
||||||
|
approval at the head on the forge, a delivery that did not say when the pull request opened) is `unknown`, and its
|
||||||
|
time counts in the next phase, which says so. The class is that of the slowest-budgeted module the fixes move, as
|
||||||
|
mesh-delivery says it from the delivery plan (`class`: `core`, `leaf`, or `none` for a fix that moves no module);
|
||||||
|
an issue whose fixes move no module, or whose class is not said, is measured and shown with no fix budget.
|
||||||
|
|
||||||
|
**Over its budget is loud.** When the last fix is delivered and the fix time exceeds the class's fix budget, the
|
||||||
|
operator's channel is told (`seat:operator-channel.notify`, key `issue-tracker.fix.<class>.over-budget`), naming the
|
||||||
|
issue, its worker and its longest phase. Nothing is said while a fix is undelivered: a merged fix not delivered is
|
||||||
|
loud as a stuck fix. The warning clears when the next issue of that class is delivered within its budget; the
|
||||||
|
newest fix time of each class decides, and every sweep says or clears what a failed ask left. The delivery budget
|
||||||
|
(mesh-delivery's `times`, from the merge) stays a measure inside the fix budget.
|
||||||
|
|
||||||
## The import of hq's issues (slice 3)
|
## The import of hq's issues (slice 3)
|
||||||
|
|
||||||
`issue-tracker.import` reads every `04-ISSUES/NNN-slug/00-report.md` of hq **through the records module
|
`issue-tracker.import` reads every `04-ISSUES/NNN-slug/00-report.md` of hq **through the records module
|
||||||
@@ -361,6 +397,13 @@ reserve spending nothing, the first reserve refused while the knowledge base doe
|
|||||||
set, a kind held none of being no answer, hq's folders never read, and a
|
set, a kind held none of being no answer, hq's folders never read, and a
|
||||||
reservation that cannot be kept leaving its number spent.
|
reservation that cannot be kept leaving its number spent.
|
||||||
|
|
||||||
|
A fix's time is tested in `fixtime_test.go`: the start at the first move to `fixing` through a diagnosis, a
|
||||||
|
take-over and a move away and back; phases that add up to the fix time, one ended early counting zero and one not
|
||||||
|
known counted in the next; the class of the slowest fix, none and not known with no budget; no warning while
|
||||||
|
undelivered, the warning once delivered over budget, cleared by the next of its class within budget and not by
|
||||||
|
another class, and said at the next sweep when the channel did not take it; `fix-times` over a window, an issue
|
||||||
|
before it not counted; the start read from the log; and the first approval at the head.
|
||||||
|
|
||||||
The import is tested over `testdata/hq`, copies of seven real reports of hq chosen for their shapes (a
|
The import is tested over `testdata/hq`, copies of seven real reports of hq chosen for their shapes (a
|
||||||
bracketed list with commas inside parentheses, quoted block items, an apostrophe in prose, a `resolved:`
|
bracketed list with commas inside parentheses, quoted block items, an apostrophe in prose, a `resolved:`
|
||||||
date, a replay, an empty list, a wontfix): the frontmatter read as written, each issue created at its own
|
date, a replay, an empty list, a wontfix): the frontmatter read as written, each issue created at its own
|
||||||
|
|||||||
@@ -145,7 +145,7 @@ func (t *Tracker) askUnheard() error {
|
|||||||
if since.IsZero() {
|
if since.IsZero() {
|
||||||
since = t.now()
|
since = t.now()
|
||||||
}
|
}
|
||||||
if _, err := t.recordFix(pr, d.State, d.Why, d.Merged, since, n); err != nil {
|
if _, err := t.recordFix(pr, d.heard(since), n); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
heard = true
|
heard = true
|
||||||
|
|||||||
@@ -37,6 +37,34 @@ type Fix struct {
|
|||||||
Since time.Time `json:"since"`
|
Since time.Time `json:"since"`
|
||||||
// StuckSaid says it was said stuck, on the issue and to the operator's channel.
|
// StuckSaid says it was said stuck, on the issue and to the operator's channel.
|
||||||
StuckSaid bool `json:"stuck-said,omitempty"`
|
StuckSaid bool `json:"stuck-said,omitempty"`
|
||||||
|
// Class is the class of what its delivery moves, as mesh-delivery says it: core, leaf, or none when it moves
|
||||||
|
// no module; Opened when its pull request was first heard (novox/hq ADR 0313 §2, §4).
|
||||||
|
Class string `json:"class,omitempty"`
|
||||||
|
Opened *time.Time `json:"opened,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// heardFix is what the delivery said of a pull request at one moment: its state, why, whether it is merged, and
|
||||||
|
// what a fix's time is read by — the class of what it moves, when its pull request opened and when it merged,
|
||||||
|
// each when said.
|
||||||
|
type heardFix struct {
|
||||||
|
State string
|
||||||
|
Why string
|
||||||
|
Merged bool
|
||||||
|
At time.Time
|
||||||
|
Class string
|
||||||
|
Opened *time.Time
|
||||||
|
MergedAt *time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// news says whether what was heard adds anything to what a fix keeps.
|
||||||
|
func (h heardFix) news(f *Fix, kept string) bool {
|
||||||
|
if f.State != h.State || f.Why != kept || (h.Merged && !f.Merged) || (h.Class != "" && h.Class != f.Class) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if h.Opened != nil && (f.Opened == nil || !f.Opened.Equal(*h.Opened)) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return h.MergedAt != nil && (f.MergedAt == nil || !f.MergedAt.Equal(*h.MergedAt))
|
||||||
}
|
}
|
||||||
|
|
||||||
// reached says whether a delivery's state says the change reached the machines (mesh-delivery's Reached).
|
// reached says whether a delivery's state says the change reached the machines (mesh-delivery's Reached).
|
||||||
@@ -147,7 +175,8 @@ func DeliveryLine(is *Issue) string {
|
|||||||
|
|
||||||
// recordFix keeps what the delivery said of a pull request on every issue whose fixed-by names it (only, when
|
// recordFix keeps what the delivery said of a pull request on every issue whose fixed-by names it (only, when
|
||||||
// given, on that one issue), and answers the issues it changed.
|
// given, on that one issue), and answers the issues it changed.
|
||||||
func (t *Tracker) recordFix(pr, state, why string, merged bool, at time.Time, only uint64) ([]uint64, error) {
|
func (t *Tracker) recordFix(pr string, h heardFix, only uint64) ([]uint64, error) {
|
||||||
|
state, why, merged, at := h.State, h.Why, h.Merged, h.At
|
||||||
all, err := t.allIssues()
|
all, err := t.allIssues()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -158,7 +187,7 @@ func (t *Tracker) recordFix(pr, state, why string, merged bool, at time.Time, on
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
kept := cut(why, MostReasonKept)
|
kept := cut(why, MostReasonKept)
|
||||||
if f := fixOf(is, pr); f != nil && (at.Before(f.Since) || (f.State == state && f.Why == kept && f.Merged == (merged || f.Merged))) {
|
if f := fixOf(is, pr); f != nil && (at.Before(f.Since) || !h.news(f, kept)) {
|
||||||
// Older than what is kept, or nothing new: transitions can arrive out of order.
|
// Older than what is kept, or nothing new: transitions can arrive out of order.
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -179,6 +208,17 @@ func (t *Tracker) recordFix(pr, state, why string, merged bool, at time.Time, on
|
|||||||
m := at
|
m := at
|
||||||
f.MergedAt = &m
|
f.MergedAt = &m
|
||||||
}
|
}
|
||||||
|
if h.MergedAt != nil && (merged || f.Merged) {
|
||||||
|
m := h.MergedAt.UTC()
|
||||||
|
f.MergedAt = &m // the delivery's own moment of the merge, not when it was heard
|
||||||
|
}
|
||||||
|
if h.Class != "" {
|
||||||
|
f.Class = h.Class
|
||||||
|
}
|
||||||
|
if h.Opened != nil {
|
||||||
|
o := h.Opened.UTC()
|
||||||
|
f.Opened = &o
|
||||||
|
}
|
||||||
if reached(state) && !contains(i.Delivered, pr) {
|
if reached(state) && !contains(i.Delivered, pr) {
|
||||||
i.Delivered = append(i.Delivered, pr)
|
i.Delivered = append(i.Delivered, pr)
|
||||||
}
|
}
|
||||||
@@ -207,6 +247,10 @@ func (t *Tracker) afterFix(n uint64) error {
|
|||||||
t.Logf("[mesh-issues] the warning that %d is delivered and not resolved could not be cleared on the operator's channel (tried again at the next sweep): %v", n, err)
|
t.Logf("[mesh-issues] the warning that %d is delivered and not resolved could not be cleared on the operator's channel (tried again at the next sweep): %v", n, err)
|
||||||
}
|
}
|
||||||
if allDelivered(is) {
|
if allDelivered(is) {
|
||||||
|
// Its fix time, final once its last fix is delivered (novox/hq ADR 0313).
|
||||||
|
if err := t.timeFix(n); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if unresolved(is) {
|
if unresolved(is) {
|
||||||
prs := liveFixes(is)
|
prs := liveFixes(is)
|
||||||
sort.Strings(prs)
|
sort.Strings(prs)
|
||||||
@@ -266,9 +310,11 @@ func (t *Tracker) afterFix(n uint64) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// stillStuck says whether an issue has a fix said stuck that is still named, not delivered, and the issue not
|
// stillStuck says whether an issue has a fix said stuck that is still named, not delivered, and the issue not
|
||||||
// closed as wontfix.
|
// closed. A closed issue is the end of its warning: resolved means its worker proved the acceptance live, so a fix
|
||||||
|
// the tracker never heard delivered is not waited for any more (a failed walk that a later merge carried stays
|
||||||
|
// failed, novox/hq issue 547), and wontfix waits for nothing.
|
||||||
func stillStuck(is *Issue) bool {
|
func stillStuck(is *Issue) bool {
|
||||||
if is.Status == StatusWontfix {
|
if closedStatus(is.Status) {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
named := liveFixes(is)
|
named := liveFixes(is)
|
||||||
@@ -281,19 +327,30 @@ func stillStuck(is *Issue) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// clearStuck clears the warning said for an issue on the operator's channel once nothing it was said for is
|
// clearStuck clears the warning said for an issue on the operator's channel once nothing it was said for is
|
||||||
// stuck — delivered, no longer named, or the issue closed as wontfix; one that fails is tried again at the next
|
// stuck — delivered, no longer named, or the issue closed (resolved or wontfix); one that fails is tried again at
|
||||||
// sweep.
|
// the next sweep.
|
||||||
func (t *Tracker) clearStuck(is *Issue) error {
|
func (t *Tracker) clearStuck(is *Issue) error {
|
||||||
if !is.StuckOpen || stillStuck(is) || t.Notify == nil {
|
if !is.StuckOpen || stillStuck(is) || t.Notify == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if err := t.Notify(stuckKey(is.Number), "", "", "", "", "the fix of issue "+keyOf(is.Number)+" is delivered or no longer waited for", true); err != nil {
|
if err := t.Notify(stuckKey(is.Number), "", "", "", "", stuckEnded(is), true); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_, err := t.change(is.Number, Act{Op: "warned", By: Seat}, func(i *Issue) error { i.StuckOpen = false; return nil })
|
_, err := t.change(is.Number, Act{Op: "warned", By: Seat}, func(i *Issue) error { i.StuckOpen = false; return nil })
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// stuckEnded is what the operator's channel says when a stuck warning ends.
|
||||||
|
func stuckEnded(is *Issue) string {
|
||||||
|
switch is.Status {
|
||||||
|
case StatusResolved:
|
||||||
|
return "resolved: issue " + keyOf(is.Number) + " was proven live by its worker"
|
||||||
|
case StatusWontfix:
|
||||||
|
return "issue " + keyOf(is.Number) + " was closed as wontfix: its fix is no longer waited for"
|
||||||
|
}
|
||||||
|
return "the fix of issue " + keyOf(is.Number) + " is delivered or no longer waited for"
|
||||||
|
}
|
||||||
|
|
||||||
func stuckKey(n uint64) string { return "issue-tracker.fix-stuck." + keyOf(n) }
|
func stuckKey(n uint64) string { return "issue-tracker.fix-stuck." + keyOf(n) }
|
||||||
|
|
||||||
func prKey(repository string, number int) string {
|
func prKey(repository string, number int) string {
|
||||||
@@ -308,12 +365,15 @@ func prKey(repository string, number int) string {
|
|||||||
// to is kept on each issue whose fixed-by names it.
|
// to is kept on each issue whose fixed-by names it.
|
||||||
func (t *Tracker) OnTransition(e stdio.Envelope) error {
|
func (t *Tracker) OnTransition(e stdio.Envelope) error {
|
||||||
var body struct {
|
var body struct {
|
||||||
Repository string `json:"repository"`
|
Repository string `json:"repository"`
|
||||||
Number int `json:"number"`
|
Number int `json:"number"`
|
||||||
To string `json:"to"`
|
To string `json:"to"`
|
||||||
Why string `json:"why"`
|
Why string `json:"why"`
|
||||||
MergedAs string `json:"merged_as"`
|
MergedAs string `json:"merged_as"`
|
||||||
At time.Time `json:"at"`
|
At time.Time `json:"at"`
|
||||||
|
Class string `json:"class"`
|
||||||
|
Opened *time.Time `json:"opened"`
|
||||||
|
MergedAt *time.Time `json:"merged_at"`
|
||||||
}
|
}
|
||||||
if err := json.Unmarshal(e.Body, &body); err != nil || body.To == "" || body.Number <= 0 {
|
if err := json.Unmarshal(e.Body, &body); err != nil || body.To == "" || body.Number <= 0 {
|
||||||
return nil
|
return nil
|
||||||
@@ -323,7 +383,8 @@ func (t *Tracker) OnTransition(e stdio.Envelope) error {
|
|||||||
at = t.now()
|
at = t.now()
|
||||||
}
|
}
|
||||||
merged := body.MergedAs != ""
|
merged := body.MergedAs != ""
|
||||||
changed, err := t.recordFix(prKey(body.Repository, body.Number), body.To, body.Why, merged, at, 0)
|
changed, err := t.recordFix(prKey(body.Repository, body.Number), heardFix{State: body.To, Why: body.Why, Merged: merged,
|
||||||
|
At: at, Class: body.Class, Opened: nonZero(body.Opened), MergedAt: nonZero(body.MergedAt)}, 0)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -343,6 +404,24 @@ type Delivery struct {
|
|||||||
Since time.Time
|
Since time.Time
|
||||||
// Merged is the pull request's own merged flag, as the forge says it.
|
// Merged is the pull request's own merged flag, as the forge says it.
|
||||||
Merged bool
|
Merged bool
|
||||||
|
// Class, Opened and MergedAt are what a fix's time is read by (novox/hq ADR 0313), when the delivery says them.
|
||||||
|
Class string
|
||||||
|
Opened *time.Time
|
||||||
|
MergedAt *time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// heard is what a delivery answered, as a fix keeps it.
|
||||||
|
func (d Delivery) heard(since time.Time) heardFix {
|
||||||
|
return heardFix{State: d.State, Why: d.Why, Merged: d.Merged, At: since, Class: d.Class, Opened: d.Opened, MergedAt: d.MergedAt}
|
||||||
|
}
|
||||||
|
|
||||||
|
// nonZero is a moment said, or nil for one not said.
|
||||||
|
func nonZero(t *time.Time) *time.Time {
|
||||||
|
if t == nil || t.IsZero() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
u := t.UTC()
|
||||||
|
return &u
|
||||||
}
|
}
|
||||||
|
|
||||||
// DeliveryState asks mesh-delivery for one pull request's delivery.
|
// DeliveryState asks mesh-delivery for one pull request's delivery.
|
||||||
@@ -357,9 +436,12 @@ func busDeliveryState(owner string) DeliveryState {
|
|||||||
}
|
}
|
||||||
var out struct {
|
var out struct {
|
||||||
Delivery struct {
|
Delivery struct {
|
||||||
State string `json:"state"`
|
State string `json:"state"`
|
||||||
Waits string `json:"waits"`
|
Waits string `json:"waits"`
|
||||||
Since time.Time `json:"since"`
|
Since time.Time `json:"since"`
|
||||||
|
Class string `json:"class"`
|
||||||
|
Opened *time.Time `json:"opened"`
|
||||||
|
MergedAt *time.Time `json:"merged_at"`
|
||||||
} `json:"delivery"`
|
} `json:"delivery"`
|
||||||
Pull struct {
|
Pull struct {
|
||||||
Merged bool `json:"merged"`
|
Merged bool `json:"merged"`
|
||||||
@@ -368,7 +450,8 @@ func busDeliveryState(owner string) DeliveryState {
|
|||||||
if err := json.Unmarshal(raw, &out); err != nil {
|
if err := json.Unmarshal(raw, &out); err != nil {
|
||||||
return Delivery{}, fmt.Errorf("mesh-delivery's checks answered no delivery: %.200s", raw)
|
return Delivery{}, fmt.Errorf("mesh-delivery's checks answered no delivery: %.200s", raw)
|
||||||
}
|
}
|
||||||
return Delivery{State: out.Delivery.State, Why: out.Delivery.Waits, Since: out.Delivery.Since.UTC(), Merged: out.Pull.Merged}, nil
|
return Delivery{State: out.Delivery.State, Why: out.Delivery.Waits, Since: out.Delivery.Since.UTC(), Merged: out.Pull.Merged,
|
||||||
|
Class: out.Delivery.Class, Opened: nonZero(out.Delivery.Opened), MergedAt: nonZero(out.Delivery.MergedAt)}, nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -417,7 +500,7 @@ func (t *Tracker) OnMoved(e stdio.Envelope) error {
|
|||||||
if since.IsZero() {
|
if since.IsZero() {
|
||||||
since = t.now()
|
since = t.now()
|
||||||
}
|
}
|
||||||
if _, err := t.recordFix(pr, state, d.Why, d.Merged, since, body.Number); err != nil {
|
if _, err := t.recordFix(pr, d.heard(since), body.Number); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -453,7 +536,7 @@ func (t *Tracker) stuckAfter() time.Duration {
|
|||||||
return DefaultStuckAfter
|
return DefaultStuckAfter
|
||||||
}
|
}
|
||||||
|
|
||||||
// Sweep says every fix merged and not delivered past the bound: a note on its issue and a warning on the
|
// Sweep says every fix of an issue not closed merged and not delivered past the bound: a note on its issue and a warning on the
|
||||||
// operator's channel, once; and every issue in fixing whose fixes were all delivered past its own bound and is not
|
// operator's channel, once; and every issue in fixing whose fixes were all delivered past its own bound and is not
|
||||||
// resolved (novox/hq issue 483). First it asks the delivery of a few fixes never heard of. It answers how many
|
// resolved (novox/hq issue 483). First it asks the delivery of a few fixes never heard of. It answers how many
|
||||||
// warnings it said. Run every few minutes.
|
// warnings it said. Run every few minutes.
|
||||||
@@ -468,21 +551,26 @@ func (t *Tracker) Sweep() (int, error) {
|
|||||||
bound, now := t.stuckAfter(), t.now()
|
bound, now := t.stuckAfter(), t.now()
|
||||||
said := 0
|
said := 0
|
||||||
for n, is := range all {
|
for n, is := range all {
|
||||||
|
// A fix time missed at its delivery (a failure, or a restart between) is kept now; one of an issue moved to
|
||||||
|
// fixing before the tracker kept the moment is kept only when its fix is heard of again.
|
||||||
|
if is.FixingFrom != nil && is.FixTime == nil && allDelivered(is) {
|
||||||
|
if err := t.timeFix(n); err != nil {
|
||||||
|
t.Logf("[mesh-issues] the fix time of %d could not be kept (tried again at the next sweep): %v", n, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
ok, err := t.sayNotResolved(is, now)
|
ok, err := t.sayNotResolved(is, now)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Logf("[mesh-issues] that %d is delivered and not resolved could not be said on the operator's channel (said at the next sweep): %v", n, err)
|
t.Logf("[mesh-issues] that %d is delivered and not resolved could not be said on the operator's channel (said at the next sweep): %v", n, err)
|
||||||
} else if ok {
|
} else if ok {
|
||||||
said++
|
said++
|
||||||
}
|
}
|
||||||
if is.Status == StatusWontfix {
|
|
||||||
if err := t.clearStuck(is); err != nil {
|
|
||||||
t.Logf("[mesh-issues] the stuck fix of %d could not be cleared on the operator's channel: %v", n, err)
|
|
||||||
}
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if err := t.clearStuck(is); err != nil {
|
if err := t.clearStuck(is); err != nil {
|
||||||
t.Logf("[mesh-issues] the stuck fix of %d could not be cleared on the operator's channel: %v", n, err)
|
t.Logf("[mesh-issues] the stuck fix of %d could not be cleared on the operator's channel: %v", n, err)
|
||||||
}
|
}
|
||||||
|
if closedStatus(is.Status) {
|
||||||
|
// A resolved or wontfix issue waits for no fix: never said stuck (novox/hq issue 547).
|
||||||
|
continue
|
||||||
|
}
|
||||||
named := liveFixes(is)
|
named := liveFixes(is)
|
||||||
for _, f := range is.Fixes {
|
for _, f := range is.Fixes {
|
||||||
if !contains(named, f.PullRequest) || contains(is.Delivered, f.PullRequest) || !f.Merged || f.MergedAt == nil ||
|
if !contains(named, f.PullRequest) || contains(is.Delivered, f.PullRequest) || !f.Merged || f.MergedAt == nil ||
|
||||||
@@ -526,5 +614,8 @@ func (t *Tracker) Sweep() (int, error) {
|
|||||||
said++
|
said++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if err := t.judgeFixBudgets(); err != nil {
|
||||||
|
t.Logf("[mesh-issues] a fix over its budget could not be said or cleared on the operator's channel (tried again at the next sweep): %v", err)
|
||||||
|
}
|
||||||
return said, nil
|
return said, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,10 +11,22 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
type notified struct {
|
type notified struct {
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
said []string
|
said []string
|
||||||
clear []string
|
clear []string
|
||||||
err error
|
resolved []string
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
// fixedAtFixing names a fix while the issue stays at fixing, as an agent does when the fix merges (ADR 0281 §3,
|
||||||
|
// ADR 0284): a fix stuck is said only for an issue not closed (issue 547).
|
||||||
|
func fixedAtFixing(t *testing.T, h *harness, n uint64, fixedBy ...string) {
|
||||||
|
t.Helper()
|
||||||
|
h.fixing(t, n)
|
||||||
|
if _, err := h.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", LocatedIn: []string{"mesh-catalog modules/records"},
|
||||||
|
FixedBy: fixedBy}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (n *notified) notify(key, severity, summary, headline, explanation, resolved string, clear bool) error {
|
func (n *notified) notify(key, severity, summary, headline, explanation, resolved string, clear bool) error {
|
||||||
@@ -25,6 +37,7 @@ func (n *notified) notify(key, severity, summary, headline, explanation, resolve
|
|||||||
}
|
}
|
||||||
if clear {
|
if clear {
|
||||||
n.clear = append(n.clear, key)
|
n.clear = append(n.clear, key)
|
||||||
|
n.resolved = append(n.resolved, resolved)
|
||||||
} else {
|
} else {
|
||||||
n.said = append(n.said, key+" "+severity+" "+summary)
|
n.said = append(n.said, key+" "+severity+" "+summary)
|
||||||
}
|
}
|
||||||
@@ -121,7 +134,7 @@ func TestAnIssueKnowsWhetherItsFixIsDelivered(t *testing.T) {
|
|||||||
func TestAStuckFixIsSaidOnceAndClearedWhenDelivered(t *testing.T) {
|
func TestAStuckFixIsSaidOnceAndClearedWhenDelivered(t *testing.T) {
|
||||||
h := newHarness(t)
|
h := newHarness(t)
|
||||||
n := h.open(t, "the counter skips a number")
|
n := h.open(t, "the counter skips a number")
|
||||||
resolve(t, h, n, "mesh-catalog#9")
|
fixedAtFixing(t, h, n, "mesh-catalog#9")
|
||||||
note := ¬ified{}
|
note := ¬ified{}
|
||||||
h.Notify = note.notify
|
h.Notify = note.notify
|
||||||
merged := time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC) // three hours before the harness's clock
|
merged := time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC) // three hours before the harness's clock
|
||||||
@@ -152,7 +165,7 @@ func TestAStuckFixIsSaidOnceAndClearedWhenDelivered(t *testing.T) {
|
|||||||
// Under the bound nothing is said.
|
// Under the bound nothing is said.
|
||||||
h2 := newHarness(t)
|
h2 := newHarness(t)
|
||||||
m := h2.open(t, "y")
|
m := h2.open(t, "y")
|
||||||
resolve(t, h2, m, "mesh-catalog#9")
|
fixedAtFixing(t, h2, m, "mesh-catalog#9")
|
||||||
n2 := ¬ified{}
|
n2 := ¬ified{}
|
||||||
h2.Notify = n2.notify
|
h2.Notify = n2.notify
|
||||||
transition(t, h2, 9, "delivering", "", "abc", time.Date(2026, 10, 10, 11, 30, 0, 0, time.UTC))
|
transition(t, h2, 9, "delivering", "", "abc", time.Date(2026, 10, 10, 11, 30, 0, 0, time.UTC))
|
||||||
@@ -194,7 +207,8 @@ func TestAFixIsNamedWhileTheIssueStaysLocated(t *testing.T) {
|
|||||||
func TestDeliveredStaysDeliveredAndClearsAreRetried(t *testing.T) {
|
func TestDeliveredStaysDeliveredAndClearsAreRetried(t *testing.T) {
|
||||||
h := newHarness(t)
|
h := newHarness(t)
|
||||||
n := h.open(t, "x")
|
n := h.open(t, "x")
|
||||||
resolve(t, h, n, "mesh-catalog#9")
|
fixedAtFixing(t, h, n, "mesh-catalog#9")
|
||||||
|
h.settings.DeliveredNotResolvedAfter = "24h" // only the stuck warning is under test here
|
||||||
note := ¬ified{}
|
note := ¬ified{}
|
||||||
h.Notify = note.notify
|
h.Notify = note.notify
|
||||||
merged := time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC)
|
merged := time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC)
|
||||||
@@ -244,3 +258,78 @@ func TestDeliveredStaysDeliveredAndClearsAreRetried(t *testing.T) {
|
|||||||
t.Fatalf("%d asks, %q", asks, v.Delivery)
|
t.Fatalf("%d asks, %q", asks, v.Delivery)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A resolved issue is the end of its stuck warning, as wontfix is: resolving clears it on the operator's channel,
|
||||||
|
// saying so, and the sweep never says a fix stuck for an issue resolved or wontfix, even one whose delivery failed
|
||||||
|
// and never reads delivered (issue 547).
|
||||||
|
func TestAResolvedIssueEndsItsStuckWarningAndIsNeverSaidStuck(t *testing.T) {
|
||||||
|
merged := time.Date(2026, 10, 10, 9, 0, 0, 0, time.UTC) // three hours before the harness's clock
|
||||||
|
moved := func(t *testing.T, h *harness, n uint64) {
|
||||||
|
t.Helper()
|
||||||
|
raw, _ := json.Marshal(map[string]any{"number": n})
|
||||||
|
if err := h.OnMoved(stdio.Envelope{Body: raw}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, to := range []string{StatusResolved, StatusWontfix} {
|
||||||
|
t.Run(to, func(t *testing.T) {
|
||||||
|
h := newHarness(t)
|
||||||
|
n := h.open(t, "the counter skips a number")
|
||||||
|
fixedAtFixing(t, h, n, "mesh-catalog#9")
|
||||||
|
note := ¬ified{}
|
||||||
|
h.Notify = note.notify
|
||||||
|
transition(t, h, 9, "failed", "the gate failed on one machine", "abc", merged)
|
||||||
|
if said, err := h.Sweep(); err != nil || said != 1 || len(note.said) != 1 {
|
||||||
|
t.Fatalf("not said stuck: %d %v %q", said, err, note.said)
|
||||||
|
}
|
||||||
|
ask := MoveAsk{Number: n, Status: to, FixedBy: []string{"mesh-catalog#9"}, ReplayNone: "proven live"}
|
||||||
|
if to == StatusWontfix {
|
||||||
|
ask = MoveAsk{Number: n, Status: to, Reason: "not worth it"}
|
||||||
|
}
|
||||||
|
if _, err := h.Move(ask); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
moved(t, h, n)
|
||||||
|
if len(note.clear) != 1 || note.clear[0] != stuckKey(n) || get(t, h, n).StuckOpen {
|
||||||
|
t.Fatalf("not cleared at the move: %q", note.clear)
|
||||||
|
}
|
||||||
|
if to == StatusResolved && !strings.HasPrefix(note.resolved[0], "resolved: ") {
|
||||||
|
t.Fatalf("%q", note.resolved)
|
||||||
|
}
|
||||||
|
if said, _ := h.Sweep(); said != 0 || len(note.said) != 1 || len(note.clear) != 1 {
|
||||||
|
t.Fatalf("said again after %s: %d %q, cleared %q", to, said, note.said, note.clear)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
// A warning left open by a tracker that kept it for a resolved issue is cleared at the first sweep, and an
|
||||||
|
// issue resolved before its fix was ever said stuck is never said stuck.
|
||||||
|
h := newHarness(t)
|
||||||
|
n := h.open(t, "x")
|
||||||
|
fixedAtFixing(t, h, n, "mesh-catalog#9")
|
||||||
|
note := ¬ified{err: errStuck}
|
||||||
|
h.Notify = note.notify
|
||||||
|
transition(t, h, 9, "failed", "the gate failed", "abc", merged)
|
||||||
|
if _, err := h.change(n, Act{Op: "warned", By: Seat}, func(i *Issue) error {
|
||||||
|
i.Fixes[0].StuckSaid, i.StuckOpen = true, true
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := h.Move(MoveAsk{Number: n, Status: StatusResolved, FixedBy: []string{"mesh-catalog#9"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
moved(t, h, n) // the channel does not answer: kept open, tried at the sweep
|
||||||
|
note.err = nil
|
||||||
|
if said, _ := h.Sweep(); said != 0 || len(note.clear) != 1 || get(t, h, n).StuckOpen {
|
||||||
|
t.Fatalf("said %d, cleared %q", said, note.clear)
|
||||||
|
}
|
||||||
|
m := h.open(t, "y")
|
||||||
|
fixedAtFixing(t, h, m, "mesh-catalog#10")
|
||||||
|
transition(t, h, 10, "failed", "the gate failed", "abc", merged)
|
||||||
|
if _, err := h.Move(MoveAsk{Number: m, Status: StatusResolved, FixedBy: []string{"mesh-catalog#10"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if said, _ := h.Sweep(); said != 0 || len(note.said) != 0 {
|
||||||
|
t.Fatalf("a resolved issue said stuck: %q", note.said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,647 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// A fix's time (novox/hq ADR 0313, issue 403). An issue's fix time runs from its first move to fixing to the moment
|
||||||
|
// the last of its fixes is delivered to every machine it should reach. It is held to the fix budget of its class —
|
||||||
|
// five minutes for a leaf module, ten for a core one — and split into four phases read from the fix delivered
|
||||||
|
// last: implementing (to its pull request opened), review (to an approval at its head), checks (to its merge) and
|
||||||
|
// merge to running (to delivered, its delivery time). A fix over its budget is said on the operator's channel once
|
||||||
|
// it is final, naming the issue, its worker and its longest phase; the warning clears when the next issue of that
|
||||||
|
// class is delivered within its budget. `fix-times` answers the last days' median and worst per class.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The classes of a fix (ADR 0282 §1, ADR 0313 §2) and the class of a fix that moves no module.
|
||||||
|
const (
|
||||||
|
ClassCore = "core"
|
||||||
|
ClassLeaf = "leaf"
|
||||||
|
ClassNone = "none"
|
||||||
|
)
|
||||||
|
|
||||||
|
// FixBudgets are the fix budgets by class, counted from the move to fixing (ADR 0313 §2).
|
||||||
|
var FixBudgets = map[string]time.Duration{ClassLeaf: 5 * time.Minute, ClassCore: 10 * time.Minute}
|
||||||
|
|
||||||
|
// fixClasses in the order `fix-times` says them.
|
||||||
|
var fixClasses = []string{ClassLeaf, ClassCore}
|
||||||
|
|
||||||
|
// The phases of a fix time, in order (ADR 0313 §4).
|
||||||
|
const (
|
||||||
|
PhaseImplementing = "implementing"
|
||||||
|
PhaseReview = "review"
|
||||||
|
PhaseChecks = "checks"
|
||||||
|
PhaseRunning = "merge to running"
|
||||||
|
)
|
||||||
|
|
||||||
|
// FixTime is an issue's fix time, kept on it once the last of its fixes is delivered.
|
||||||
|
type FixTime struct {
|
||||||
|
// From is the first move to fixing; End when the fix delivered last was delivered.
|
||||||
|
From time.Time `json:"from"`
|
||||||
|
End time.Time `json:"end"`
|
||||||
|
TookMS int64 `json:"took_ms"`
|
||||||
|
Took string `json:"took"`
|
||||||
|
// Class is core, leaf or none; empty when a fix's class is not known. A fix of class none, or one not known,
|
||||||
|
// has no budget: its time is shown and never warned on.
|
||||||
|
Class string `json:"class,omitempty"`
|
||||||
|
Budget string `json:"budget,omitempty"`
|
||||||
|
BudgetMS int64 `json:"budget_ms,omitempty"`
|
||||||
|
Over bool `json:"over,omitempty"`
|
||||||
|
// Fix is the pull request delivered last, which the phases are read from; Worker who worked on the issue.
|
||||||
|
Fix string `json:"fix"`
|
||||||
|
Worker string `json:"worker,omitempty"`
|
||||||
|
Phases []FixPhase `json:"phases"`
|
||||||
|
Longest string `json:"longest,omitempty"`
|
||||||
|
Said string `json:"said"`
|
||||||
|
// Warned says the over-budget warning was said for it on the operator's channel and not cleared since.
|
||||||
|
Warned bool `json:"warned,omitempty"`
|
||||||
|
// FromLog says its start was read back from the log, not kept by the move: an issue moved to fixing before the
|
||||||
|
// tracker measured fix times. Its fix time is kept and shown, and never warned on, so the first fixes heard of
|
||||||
|
// after the tracker starts measuring raise no warning for work done before it (novox/hq issue 403).
|
||||||
|
FromLog bool `json:"from-log,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FixPhase is one phase of a fix time: measured, or unknown when the moment that ends it is not known — its time is
|
||||||
|
// then counted in the phase after it, which says so.
|
||||||
|
type FixPhase struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
State string `json:"state"`
|
||||||
|
TookMS int64 `json:"took_ms"`
|
||||||
|
Took string `json:"took"`
|
||||||
|
Said string `json:"said,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Approval is when a pull request was first approved at its head, as the forge says it; a test stubs the asking.
|
||||||
|
// found is false when no approval at its head is on the forge.
|
||||||
|
type Approval func(repository string, number int) (at time.Time, found bool, err error)
|
||||||
|
|
||||||
|
// classOfFixes is the class of an issue's fixes: that of the slowest-budgeted module they move — core when any
|
||||||
|
// moves a core module, leaf when any moves a leaf one, none when none moves a module — and empty, with why, when
|
||||||
|
// a fix's class is not known and none is core.
|
||||||
|
func classOfFixes(is *Issue, prs []string) (string, string) {
|
||||||
|
var unknown []string
|
||||||
|
leaf := false
|
||||||
|
for _, pr := range prs {
|
||||||
|
f := fixOf(is, pr)
|
||||||
|
switch {
|
||||||
|
case f == nil || f.Class == "":
|
||||||
|
unknown = append(unknown, pr)
|
||||||
|
case f.Class == ClassCore:
|
||||||
|
return ClassCore, ""
|
||||||
|
case f.Class == ClassLeaf:
|
||||||
|
leaf = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(unknown) > 0 {
|
||||||
|
return "", "the delivery did not say what " + strings.Join(unknown, ", ") + " moves"
|
||||||
|
}
|
||||||
|
if leaf {
|
||||||
|
return ClassLeaf, ""
|
||||||
|
}
|
||||||
|
return ClassNone, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// fixPhases splits the span from start to the delivered moment at the moments that end each phase, in order. A
|
||||||
|
// phase whose end is not known is unknown, and its time counts in the next phase known; a phase whose end came
|
||||||
|
// before the end of the phase before it counts zero (ADR 0313 §4). The phases add up to the fix time.
|
||||||
|
func fixPhases(start time.Time, ends []struct {
|
||||||
|
name string
|
||||||
|
at *time.Time
|
||||||
|
why string
|
||||||
|
}) ([]FixPhase, string) {
|
||||||
|
var out []FixPhase
|
||||||
|
prev := start
|
||||||
|
var carried []string
|
||||||
|
for _, e := range ends {
|
||||||
|
if e.at == nil {
|
||||||
|
out = append(out, FixPhase{Name: e.name, State: "unknown", Took: "0s", Said: e.why})
|
||||||
|
carried = append(carried, e.name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
took := time.Duration(0)
|
||||||
|
if e.at.After(prev) {
|
||||||
|
took = e.at.Sub(prev)
|
||||||
|
prev = *e.at
|
||||||
|
}
|
||||||
|
p := FixPhase{Name: e.name, State: "measured", TookMS: took.Milliseconds(), Took: took.Round(time.Second).String()}
|
||||||
|
if len(carried) > 0 {
|
||||||
|
p.Said = "counts " + strings.Join(carried, " and ") + " too: its end is not known"
|
||||||
|
carried = nil
|
||||||
|
}
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
best := -1
|
||||||
|
for i, p := range out {
|
||||||
|
if p.State == "measured" && (best < 0 || p.TookMS > out[best].TookMS) {
|
||||||
|
best = i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
longest := ""
|
||||||
|
if best >= 0 {
|
||||||
|
longest = out[best].Name + " " + out[best].Took
|
||||||
|
}
|
||||||
|
return out, longest
|
||||||
|
}
|
||||||
|
|
||||||
|
// startOfFixing is when an issue was first moved to fixing (ADR 0313 §1), and whether that moment came from its
|
||||||
|
// log rather than from the move that kept it: the earliest of what it keeps (fixing-from, kept by the move since the
|
||||||
|
// tracker measures fix times) and what its log says. An issue moved before the moment was kept has it read from its
|
||||||
|
// log: a move's operation says its own moment. A value carried into the log, or imported, holds the issue whole and
|
||||||
|
// no moment of its own (it is stamped when it was carried); when the value before it was not at fixing it is the
|
||||||
|
// move's own write, and its worker's active is the move's moment (a move to fixing marks its worker active), not
|
||||||
|
// its worker's since, which a move from diagnosing by the same worker keeps (ADR 0284 §6). Only a first value
|
||||||
|
// already at fixing falls back to its worker's since, the best moment the log holds for it. Zero when it never was.
|
||||||
|
func (t *Tracker) startOfFixing(is *Issue) (time.Time, bool) {
|
||||||
|
var kept time.Time
|
||||||
|
if is.FixingFrom != nil {
|
||||||
|
kept = is.FixingFrom.UTC()
|
||||||
|
}
|
||||||
|
if t.Log == nil {
|
||||||
|
return kept, false
|
||||||
|
}
|
||||||
|
entries, err := t.readKey(is.Number, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Logf("[mesh-issues] the log of %d could not be read for its first move to fixing (its kept moment is used): %v", is.Number, err)
|
||||||
|
return kept, false
|
||||||
|
}
|
||||||
|
var logged time.Time
|
||||||
|
status, seen := "", false
|
||||||
|
for _, e := range entries {
|
||||||
|
var op Operation
|
||||||
|
if json.Unmarshal(e.Value, &op) != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
at, became := op.At, ""
|
||||||
|
switch {
|
||||||
|
case op.Issue != nil:
|
||||||
|
became = op.Issue.Status
|
||||||
|
if w := op.Issue.Worker; w != nil {
|
||||||
|
switch {
|
||||||
|
case seen && status != StatusFixing && !w.Active.IsZero():
|
||||||
|
at = w.Active
|
||||||
|
case !w.Since.IsZero():
|
||||||
|
at = w.Since
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case op.Set["status"] != nil:
|
||||||
|
_ = json.Unmarshal(op.Set["status"], &became)
|
||||||
|
default:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if became == StatusFixing && status != StatusFixing && !at.IsZero() {
|
||||||
|
logged = at.UTC()
|
||||||
|
break
|
||||||
|
}
|
||||||
|
status, seen = became, true
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case kept.IsZero():
|
||||||
|
return logged, !logged.IsZero()
|
||||||
|
case !logged.IsZero() && logged.Before(kept):
|
||||||
|
return logged, true
|
||||||
|
}
|
||||||
|
return kept, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// fixTimeOf is an issue's fix time once every fix it names is delivered; nil, with why, when it is not measured.
|
||||||
|
func (t *Tracker) fixTimeOf(is *Issue) (*FixTime, string) {
|
||||||
|
if !allDelivered(is) {
|
||||||
|
return nil, "not every fix is delivered"
|
||||||
|
}
|
||||||
|
start, fromLog := t.startOfFixing(is)
|
||||||
|
if start.IsZero() {
|
||||||
|
return nil, "it was never moved to fixing"
|
||||||
|
}
|
||||||
|
prs := liveFixes(is)
|
||||||
|
var last *Fix
|
||||||
|
for _, pr := range prs {
|
||||||
|
if f := fixOf(is, pr); f != nil && (last == nil || f.Since.After(last.Since)) {
|
||||||
|
last = f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if last == nil {
|
||||||
|
return nil, "no fix is heard of"
|
||||||
|
}
|
||||||
|
end := last.Since.UTC()
|
||||||
|
took := time.Duration(0)
|
||||||
|
if end.After(start) {
|
||||||
|
took = end.Sub(start)
|
||||||
|
}
|
||||||
|
ft := &FixTime{From: start, End: end, TookMS: took.Milliseconds(), Took: took.Round(time.Second).String(), Fix: last.PullRequest,
|
||||||
|
FromLog: fromLog}
|
||||||
|
if is.Worker != nil {
|
||||||
|
ft.Worker = is.Worker.By
|
||||||
|
} else if is.FixTime != nil {
|
||||||
|
ft.Worker = is.FixTime.Worker
|
||||||
|
}
|
||||||
|
class, why := classOfFixes(is, prs)
|
||||||
|
ft.Class = class
|
||||||
|
if b, ok := FixBudgets[class]; ok {
|
||||||
|
ft.Budget, ft.BudgetMS, ft.Over = b.String(), b.Milliseconds(), took > b
|
||||||
|
}
|
||||||
|
|
||||||
|
// The phases, from the fix delivered last.
|
||||||
|
type end_ = struct {
|
||||||
|
name string
|
||||||
|
at *time.Time
|
||||||
|
why string
|
||||||
|
}
|
||||||
|
var approved *time.Time
|
||||||
|
approvalWhy := "no approval at its head is on the forge: review and checks are counted together"
|
||||||
|
if t.Approval != nil {
|
||||||
|
repo, number, _ := strings.Cut(last.PullRequest, "#")
|
||||||
|
var n int
|
||||||
|
if _, err := fmt.Sscanf(number, "%d", &n); err == nil {
|
||||||
|
at, found, err := t.Approval(repo, n)
|
||||||
|
switch {
|
||||||
|
case err != nil:
|
||||||
|
approvalWhy = "the forge could not be read for its approval (" + cut(err.Error(), 200) + "): review and checks are counted together"
|
||||||
|
case found:
|
||||||
|
a := at.UTC()
|
||||||
|
approved = &a
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
approvalWhy = "the forge is not asked here: review and checks are counted together"
|
||||||
|
}
|
||||||
|
openedWhy := "the delivery did not say when its pull request opened: implementing is counted in review"
|
||||||
|
mergedWhy := "the delivery did not say when it merged: checks are counted in merge to running"
|
||||||
|
ft.Phases, ft.Longest = fixPhases(start, []end_{
|
||||||
|
{PhaseImplementing, last.Opened, openedWhy},
|
||||||
|
{PhaseReview, approved, approvalWhy},
|
||||||
|
{PhaseChecks, last.MergedAt, mergedWhy},
|
||||||
|
{PhaseRunning, &end, ""},
|
||||||
|
})
|
||||||
|
|
||||||
|
var said []string
|
||||||
|
said = append(said, fmt.Sprintf("fixed in %s, from the first move to fixing (%s) to %s delivered (%s)", ft.Took,
|
||||||
|
start.Format(time.RFC3339), last.PullRequest, end.Format(time.RFC3339)))
|
||||||
|
switch {
|
||||||
|
case class == ClassNone:
|
||||||
|
said = append(said, "its fixes move no module, so it has no fix budget")
|
||||||
|
case class == "":
|
||||||
|
said = append(said, "no fix budget: "+why)
|
||||||
|
case ft.Over:
|
||||||
|
said = append(said, fmt.Sprintf("over its %s fix budget of %s", class, ft.Budget))
|
||||||
|
default:
|
||||||
|
said = append(said, fmt.Sprintf("within its %s fix budget of %s", class, ft.Budget))
|
||||||
|
}
|
||||||
|
if ft.Longest != "" {
|
||||||
|
said = append(said, "longest: "+ft.Longest)
|
||||||
|
}
|
||||||
|
if fromLog {
|
||||||
|
said = append(said, "its start was read from its log (moved to fixing before fix times were measured), so it is never warned on")
|
||||||
|
}
|
||||||
|
ft.Said = strings.Join(said, "; ")
|
||||||
|
return ft, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// sameFixTime says whether two fix times say the same, the warning's mark aside.
|
||||||
|
func sameFixTime(a, b *FixTime) bool {
|
||||||
|
if a == nil || b == nil {
|
||||||
|
return a == b
|
||||||
|
}
|
||||||
|
x, y := *a, *b
|
||||||
|
x.Warned, y.Warned = false, false
|
||||||
|
ra, _ := json.Marshal(x)
|
||||||
|
rb, _ := json.Marshal(y)
|
||||||
|
return string(ra) == string(rb)
|
||||||
|
}
|
||||||
|
|
||||||
|
// timeFix keeps an issue's fix time once every fix it names is delivered, and again when a later fix moves its
|
||||||
|
// end; then the warnings of its class are judged.
|
||||||
|
func (t *Tracker) timeFix(n uint64) error {
|
||||||
|
is, _, err := read(t.Store, n)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ft, _ := t.fixTimeOf(is)
|
||||||
|
if ft == nil || sameFixTime(ft, is.FixTime) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := t.change(n, Act{Op: "fix-time", By: Seat}, func(i *Issue) error {
|
||||||
|
if i.FixTime != nil {
|
||||||
|
ft.Warned = i.FixTime.Warned
|
||||||
|
}
|
||||||
|
i.FixTime = ft
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := t.judgeFixBudgets(); err != nil {
|
||||||
|
t.Logf("[mesh-issues] a fix over its budget could not be said or cleared on the operator's channel (tried again at the next sweep): %v", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// fixOverKey is the warning a fix over its class's budget raises (ADR 0313 §6).
|
||||||
|
func fixOverKey(class string) string { return "issue-tracker.fix." + class + ".over-budget" }
|
||||||
|
|
||||||
|
// judgeFixBudgets says or clears each class's warning by the newest fix time of the class (ADR 0313 §6): the newest
|
||||||
|
// over its budget and not yet said is said; the newest within its budget clears what was said for the class. Run
|
||||||
|
// after every fix time kept and at every sweep, so a warning that failed is said, or cleared, at the next.
|
||||||
|
func (t *Tracker) judgeFixBudgets() error {
|
||||||
|
if t.Notify == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
all, err := t.allIssues()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var firstErr error
|
||||||
|
for _, class := range fixClasses {
|
||||||
|
var newest *Issue
|
||||||
|
var warned []uint64
|
||||||
|
for _, is := range all {
|
||||||
|
ft := is.FixTime
|
||||||
|
if ft == nil || ft.Class != class {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if ft.Warned {
|
||||||
|
warned = append(warned, is.Number)
|
||||||
|
}
|
||||||
|
if newest == nil || ft.End.After(newest.FixTime.End) || (ft.End.Equal(newest.FixTime.End) && is.Number > newest.Number) {
|
||||||
|
newest = is
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if newest == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ft := newest.FixTime
|
||||||
|
if ft.Over {
|
||||||
|
if ft.Warned || ft.FromLog {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
worker := ft.Worker
|
||||||
|
if worker == "" {
|
||||||
|
worker = "nobody named"
|
||||||
|
}
|
||||||
|
longest := ft.Longest
|
||||||
|
if longest == "" {
|
||||||
|
longest = "not known"
|
||||||
|
}
|
||||||
|
if err := t.Notify(fixOverKey(class), "warning",
|
||||||
|
fmt.Sprintf("issue %d: fixed in %s, over its %s fix budget of %s (longest: %s)", newest.Number, ft.Took, class, ft.Budget, longest),
|
||||||
|
fmt.Sprintf("A fix took longer than its budget (issue %d)", newest.Number),
|
||||||
|
fmt.Sprintf("Issue %d was fixed in %s, from %s's move to fixing to its last fix delivered, over the %s fix "+
|
||||||
|
"budget of %s. Its longest phase was %s. Nothing is needed from you: it says where fixes lose time. "+
|
||||||
|
"It clears when the next %s fix is delivered within its budget.", newest.Number, ft.Took, worker, class,
|
||||||
|
ft.Budget, longest, class), "", false); err != nil {
|
||||||
|
if firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := t.change(newest.Number, Act{Op: "warned", By: Seat}, func(i *Issue) error {
|
||||||
|
if i.FixTime != nil {
|
||||||
|
i.FixTime.Warned = true
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(warned) == 0 || ft.BudgetMS == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := t.Notify(fixOverKey(class), "", "", "", "", fmt.Sprintf("issue %d's fix was delivered within its %s fix budget (%s of %s)",
|
||||||
|
newest.Number, class, ft.Took, ft.Budget), true); err != nil {
|
||||||
|
if firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
sort.Slice(warned, func(i, j int) bool { return warned[i] < warned[j] })
|
||||||
|
for _, n := range warned {
|
||||||
|
if _, err := t.change(n, Act{Op: "warned", By: Seat}, func(i *Issue) error {
|
||||||
|
if i.FixTime != nil {
|
||||||
|
i.FixTime.Warned = false
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return firstErr
|
||||||
|
}
|
||||||
|
|
||||||
|
// FixTimesClass is one class's line of `fix-times`.
|
||||||
|
type FixTimesClass struct {
|
||||||
|
Class string `json:"class"`
|
||||||
|
Budget string `json:"budget"`
|
||||||
|
BudgetMS int64 `json:"budget_ms"`
|
||||||
|
Count int `json:"count"`
|
||||||
|
Within int `json:"within"`
|
||||||
|
Over int `json:"over"`
|
||||||
|
Median string `json:"median,omitempty"`
|
||||||
|
MedianMS int64 `json:"median_ms,omitempty"`
|
||||||
|
Worst string `json:"worst,omitempty"`
|
||||||
|
WorstMS int64 `json:"worst_ms,omitempty"`
|
||||||
|
// WorstIssue is the issue whose fix took the worst time.
|
||||||
|
WorstIssue uint64 `json:"worst_issue,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FixTimesRow is an issue as `fix-times` names it.
|
||||||
|
type FixTimesRow struct {
|
||||||
|
Number uint64 `json:"number"`
|
||||||
|
Title string `json:"title"`
|
||||||
|
Class string `json:"class,omitempty"`
|
||||||
|
Took string `json:"took"`
|
||||||
|
TookMS int64 `json:"took_ms"`
|
||||||
|
Budget string `json:"budget,omitempty"`
|
||||||
|
Longest string `json:"longest,omitempty"`
|
||||||
|
Worker string `json:"worker,omitempty"`
|
||||||
|
Delivered string `json:"delivered"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FixTimes is the answer of `fix-times`.
|
||||||
|
type FixTimes struct {
|
||||||
|
Days int `json:"days"`
|
||||||
|
Since string `json:"since"`
|
||||||
|
Line string `json:"line"`
|
||||||
|
Classes []FixTimesClass `json:"classes"`
|
||||||
|
// Over is every issue over its budget in the window, newest first, with its longest phase.
|
||||||
|
Over []FixTimesRow `json:"over,omitempty"`
|
||||||
|
// Unbudgeted are the issues fixed in the window with no fix budget: their fixes move no module, or what they
|
||||||
|
// move is not known.
|
||||||
|
Unbudgeted []FixTimesRow `json:"unbudgeted,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// MostFixTimesDays is the longest window `fix-times` reads.
|
||||||
|
const MostFixTimesDays = 90
|
||||||
|
|
||||||
|
// FixTimes is the `fix-times` verb: each class's fix budget, the last days' median and worst fix time, and every
|
||||||
|
// issue over its budget with its longest phase (ADR 0313 §5, §7). An issue counts in the window its last fix was
|
||||||
|
// delivered in.
|
||||||
|
func (t *Tracker) FixTimes(days int) (*FixTimes, error) {
|
||||||
|
if days <= 0 {
|
||||||
|
days = 7
|
||||||
|
}
|
||||||
|
if days > MostFixTimesDays {
|
||||||
|
return nil, fmt.Errorf("days %d is more than the %d fix-times reads", days, MostFixTimesDays)
|
||||||
|
}
|
||||||
|
all, err := t.allIssues()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
since := t.now().Add(-time.Duration(days) * 24 * time.Hour)
|
||||||
|
a := &FixTimes{Days: days, Since: since.Format(time.RFC3339)}
|
||||||
|
byClass := map[string][]*Issue{}
|
||||||
|
row := func(is *Issue) FixTimesRow {
|
||||||
|
ft := is.FixTime
|
||||||
|
return FixTimesRow{Number: is.Number, Title: is.Title, Class: ft.Class, Took: ft.Took, TookMS: ft.TookMS, Budget: ft.Budget,
|
||||||
|
Longest: ft.Longest, Worker: ft.Worker, Delivered: ft.End.Format(time.RFC3339)}
|
||||||
|
}
|
||||||
|
for _, is := range all {
|
||||||
|
ft := is.FixTime
|
||||||
|
if ft == nil || ft.End.Before(since) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := FixBudgets[ft.Class]; !ok {
|
||||||
|
a.Unbudgeted = append(a.Unbudgeted, row(is))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
byClass[ft.Class] = append(byClass[ft.Class], is)
|
||||||
|
if ft.Over {
|
||||||
|
a.Over = append(a.Over, row(is))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
newestFirst := func(rows []FixTimesRow) {
|
||||||
|
sort.Slice(rows, func(i, j int) bool {
|
||||||
|
if rows[i].Delivered != rows[j].Delivered {
|
||||||
|
return rows[i].Delivered > rows[j].Delivered
|
||||||
|
}
|
||||||
|
return rows[i].Number > rows[j].Number
|
||||||
|
})
|
||||||
|
}
|
||||||
|
newestFirst(a.Over)
|
||||||
|
newestFirst(a.Unbudgeted)
|
||||||
|
var line []string
|
||||||
|
for _, class := range fixClasses {
|
||||||
|
b := FixBudgets[class]
|
||||||
|
c := FixTimesClass{Class: class, Budget: b.String(), BudgetMS: b.Milliseconds()}
|
||||||
|
is := byClass[class]
|
||||||
|
sort.Slice(is, func(i, j int) bool {
|
||||||
|
if is[i].FixTime.TookMS != is[j].FixTime.TookMS {
|
||||||
|
return is[i].FixTime.TookMS < is[j].FixTime.TookMS
|
||||||
|
}
|
||||||
|
return is[i].Number < is[j].Number
|
||||||
|
})
|
||||||
|
c.Count = len(is)
|
||||||
|
for _, i := range is {
|
||||||
|
if i.FixTime.Over {
|
||||||
|
c.Over++
|
||||||
|
} else {
|
||||||
|
c.Within++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n := len(is); n > 0 {
|
||||||
|
ms := is[n/2].FixTime.TookMS
|
||||||
|
if n%2 == 0 {
|
||||||
|
ms = (is[n/2-1].FixTime.TookMS + is[n/2].FixTime.TookMS) / 2
|
||||||
|
}
|
||||||
|
med := time.Duration(ms) * time.Millisecond
|
||||||
|
c.MedianMS, c.Median = ms, med.Round(time.Second).String()
|
||||||
|
worst := is[n-1]
|
||||||
|
c.WorstMS, c.Worst, c.WorstIssue = worst.FixTime.TookMS, worst.FixTime.Took, worst.Number
|
||||||
|
line = append(line, fmt.Sprintf("%s: median %s, worst %s (issue %d), %d of %d within %s", class, c.Median,
|
||||||
|
c.Worst, c.WorstIssue, c.Within, c.Count, c.Budget))
|
||||||
|
} else {
|
||||||
|
line = append(line, fmt.Sprintf("%s: no issue fixed", class))
|
||||||
|
}
|
||||||
|
a.Classes = append(a.Classes, c)
|
||||||
|
}
|
||||||
|
a.Line = fmt.Sprintf("fix time over the last %d day(s), from the move to fixing to delivered — %s", days, strings.Join(line, "; "))
|
||||||
|
if len(a.Unbudgeted) > 0 {
|
||||||
|
a.Line += fmt.Sprintf("; %d fixed with no fix budget", len(a.Unbudgeted))
|
||||||
|
}
|
||||||
|
return a, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// firstApproval is when a pull request was first approved at its head among the forge's reviews: the forge's own
|
||||||
|
// approval, or — the forge refusing an approval of one's own pull request, so the mesh's independent reviews are
|
||||||
|
// posted as comments by the account that opened it — a comment review whose first line is "Verdict: approve at
|
||||||
|
// <sha>", the sha the head it was posted at or a prefix of it of at least seven characters. An approval dismissed,
|
||||||
|
// stale, or of an older head does not count.
|
||||||
|
func firstApproval(raw json.RawMessage) (time.Time, bool, error) {
|
||||||
|
var a struct {
|
||||||
|
Reviews []struct {
|
||||||
|
State string `json:"state"`
|
||||||
|
AtHead bool `json:"at_head"`
|
||||||
|
Commit string `json:"commit"`
|
||||||
|
Body string `json:"body"`
|
||||||
|
Submitted string `json:"submitted"`
|
||||||
|
Stale bool `json:"stale"`
|
||||||
|
Dismissed bool `json:"dismissed"`
|
||||||
|
} `json:"reviews"`
|
||||||
|
}
|
||||||
|
var s string
|
||||||
|
if json.Unmarshal(raw, &s) == nil {
|
||||||
|
raw = json.RawMessage(s)
|
||||||
|
}
|
||||||
|
var wrapped struct {
|
||||||
|
Content []struct {
|
||||||
|
Text string `json:"text"`
|
||||||
|
} `json:"content"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(raw, &wrapped) == nil && len(wrapped.Content) > 0 {
|
||||||
|
raw = json.RawMessage(wrapped.Content[0].Text)
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(raw, &a); err != nil || a.Reviews == nil {
|
||||||
|
return time.Time{}, false, fmt.Errorf("the forge's gitea_pull_reviews answered no reviews: %.200s", raw)
|
||||||
|
}
|
||||||
|
var first time.Time
|
||||||
|
for _, r := range a.Reviews {
|
||||||
|
if !r.AtHead || r.Stale || r.Dismissed {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch r.State {
|
||||||
|
case "APPROVED":
|
||||||
|
case "COMMENT":
|
||||||
|
if !verdictApproves(r.Body, r.Commit) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
at, err := time.Parse(time.RFC3339, r.Submitted)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if first.IsZero() || at.Before(first) {
|
||||||
|
first = at
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return first, !first.IsZero(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// verdictApproves says whether a review's body opens with the mesh's approving verdict on the commit it was posted
|
||||||
|
// at: "Verdict: approve at <sha>", the sha that commit or a prefix of it of at least seven characters.
|
||||||
|
func verdictApproves(body, commit string) bool {
|
||||||
|
line, _, _ := strings.Cut(strings.TrimLeft(body, " \t\r\n"), "\n")
|
||||||
|
rest, ok := strings.CutPrefix(strings.TrimSpace(line), "Verdict: approve at ")
|
||||||
|
if !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
sha := strings.ToLower(strings.TrimRight(strings.TrimSpace(rest), "."))
|
||||||
|
commit = strings.ToLower(strings.TrimSpace(commit))
|
||||||
|
return len(sha) >= 7 && commit != "" && strings.Trim(sha, "0123456789abcdef") == "" && strings.HasPrefix(commit, sha)
|
||||||
|
}
|
||||||
|
|
||||||
|
// busApproval asks the forge's reviews of a pull request over the bus (gitea_pull_reviews).
|
||||||
|
func busApproval(owner func() string) Approval {
|
||||||
|
return func(repository string, number int) (time.Time, bool, error) {
|
||||||
|
raw, err := stdio.Ask("gitea.gitea_pull_reviews", map[string]any{"owner": owner(), "repo": repository, "number": number})
|
||||||
|
if err != nil {
|
||||||
|
return time.Time{}, false, err
|
||||||
|
}
|
||||||
|
return firstApproval(raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,480 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// novox/hq ADR 0313 (issue 403): an issue's fix time runs from its first move to fixing to its last fix
|
||||||
|
// delivered; it is held to its class's fix budget, split into four phases that add up to it, shown in get and
|
||||||
|
// fix-times, and a fix over its budget is said on the operator's channel once final, cleared at the next of its
|
||||||
|
// class delivered within budget.
|
||||||
|
|
||||||
|
// fixClock is a harness whose clock a test moves.
|
||||||
|
type fixClock struct {
|
||||||
|
*harness
|
||||||
|
at time.Time
|
||||||
|
// approvals is when each pull request was approved at its head, as the forge would say it.
|
||||||
|
approvals map[int]time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
func newFixClock(t *testing.T) *fixClock {
|
||||||
|
c := &fixClock{harness: newHarness(t), at: time.Date(2026, 10, 12, 9, 0, 0, 0, time.UTC), approvals: map[int]time.Time{}}
|
||||||
|
c.Tracker.Now = func() time.Time { return c.at }
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *fixClock) approval(repository string, number int) (time.Time, bool, error) {
|
||||||
|
a, ok := c.approvals[number]
|
||||||
|
return a, ok, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *fixClock) later(d time.Duration) time.Time { c.at = c.at.Add(d); return c.at }
|
||||||
|
|
||||||
|
// delivery says one transition of a pull request as mesh-delivery says it, with what a fix's time is read by.
|
||||||
|
func (c *fixClock) delivery(t *testing.T, number int, to, class string, opened, mergedAt *time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
body := map[string]any{"repository": "novox/mesh-catalog", "number": number, "to": to, "at": c.at}
|
||||||
|
if mergedAt != nil {
|
||||||
|
body["merged_as"] = "m" + strings.Repeat("a", 11)
|
||||||
|
body["merged_at"] = *mergedAt
|
||||||
|
}
|
||||||
|
if class != "" {
|
||||||
|
body["class"] = class
|
||||||
|
}
|
||||||
|
if opened != nil {
|
||||||
|
body["opened"] = *opened
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(body)
|
||||||
|
if err := c.OnTransition(stdio.Envelope{Key: "mesh-delivery.transition", Body: raw}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func at(t time.Time) *time.Time { return &t }
|
||||||
|
|
||||||
|
func phaseTook(t *testing.T, ft *FixTime) map[string]time.Duration {
|
||||||
|
t.Helper()
|
||||||
|
out := map[string]time.Duration{}
|
||||||
|
var sum int64
|
||||||
|
for _, p := range ft.Phases {
|
||||||
|
out[p.Name] = time.Duration(p.TookMS) * time.Millisecond
|
||||||
|
sum += p.TookMS
|
||||||
|
}
|
||||||
|
if sum != ft.TookMS {
|
||||||
|
t.Fatalf("the phases add up to %dms, the fix time is %dms: %+v", sum, ft.TookMS, ft.Phases)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// fixOne takes an issue through a fix: opened, moved to fixing, a pull request opened, approved, merged and
|
||||||
|
// delivered after the given spans; its fix time as kept.
|
||||||
|
func (c *fixClock) fixOne(t *testing.T, title string, pr int, class string, implementing, review, checks, running time.Duration) uint64 {
|
||||||
|
t.Helper()
|
||||||
|
n := c.open(t, title)
|
||||||
|
c.fixing(t, n)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#" + itoa(pr)}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
opened := c.later(implementing)
|
||||||
|
c.delivery(t, pr, "proposed", "", &opened, nil)
|
||||||
|
approved := c.later(review)
|
||||||
|
c.approvals[pr] = approved
|
||||||
|
merged := c.later(checks)
|
||||||
|
c.delivery(t, pr, "published", class, &opened, &merged)
|
||||||
|
c.later(running)
|
||||||
|
c.delivery(t, pr, "delivered", class, &opened, &merged)
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
func itoa(n int) string { b, _ := json.Marshal(n); return string(b) }
|
||||||
|
|
||||||
|
func TestAFixTimeRunsFromTheFirstMoveToFixingAndAddsUpItsPhases(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
c.Tracker.Approval = c.approval
|
||||||
|
note := ¬ified{}
|
||||||
|
c.Notify = note.notify
|
||||||
|
|
||||||
|
n := c.open(t, "a fix measured")
|
||||||
|
c.later(time.Minute)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusDiagnosing, By: "fix@g14"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c.later(10 * time.Minute) // finding the cause is not the fix's time
|
||||||
|
start := c.at
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#31"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c.later(30 * time.Second)
|
||||||
|
// A take-over, and a move away and back, restart nothing.
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "other@shanks", TakeOver: "the first worker's session ended"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusOpen}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "other@shanks"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
opened := c.later(90 * time.Second) // implementing: 2m
|
||||||
|
c.delivery(t, 31, "proposed", "", &opened, nil)
|
||||||
|
c.approvals[31] = c.later(time.Minute) // review: 1m
|
||||||
|
merged := c.later(3 * time.Minute) // checks: 3m
|
||||||
|
c.delivery(t, 31, "published", "leaf", &opened, &merged)
|
||||||
|
if is := get(t, c.harness, n); is.FixTime != nil {
|
||||||
|
t.Fatalf("a fix merged and not delivered has a fix time: %+v", is.FixTime)
|
||||||
|
}
|
||||||
|
c.later(2 * time.Minute) // merge to running: 2m
|
||||||
|
c.delivery(t, 31, "delivered", "leaf", &opened, &merged)
|
||||||
|
|
||||||
|
v, err := c.View(n)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ft := v.FixTime
|
||||||
|
if ft == nil {
|
||||||
|
t.Fatalf("no fix time once the fix is delivered: %+v", v.Issue)
|
||||||
|
}
|
||||||
|
if !ft.From.Equal(start) || ft.TookMS != (8*time.Minute).Milliseconds() {
|
||||||
|
t.Fatalf("fix time from %s took %s, want from %s (the first move to fixing) and 8m0s", ft.From, ft.Took, start)
|
||||||
|
}
|
||||||
|
got := phaseTook(t, ft)
|
||||||
|
if got[PhaseImplementing] != 2*time.Minute || got[PhaseReview] != time.Minute || got[PhaseChecks] != 3*time.Minute ||
|
||||||
|
got[PhaseRunning] != 2*time.Minute {
|
||||||
|
t.Fatalf("phases %v", got)
|
||||||
|
}
|
||||||
|
if ft.Class != ClassLeaf || !ft.Over || ft.Budget != "5m0s" || ft.Longest != "checks 3m0s" || ft.Worker != "other@shanks" {
|
||||||
|
t.Fatalf("class %q over %v budget %q longest %q worker %q", ft.Class, ft.Over, ft.Budget, ft.Longest, ft.Worker)
|
||||||
|
}
|
||||||
|
// Over its budget: said once, naming the issue, its worker and its longest phase.
|
||||||
|
if len(note.said) != 1 || !strings.HasPrefix(note.said[0], "issue-tracker.fix.leaf.over-budget warning issue "+itoa(int(n))) ||
|
||||||
|
!strings.Contains(note.said[0], "checks 3m0s") {
|
||||||
|
t.Fatalf("said %q", note.said)
|
||||||
|
}
|
||||||
|
// Heard again: nothing new is kept or said.
|
||||||
|
c.delivery(t, 31, "delivered", "leaf", &opened, &merged)
|
||||||
|
if len(note.said) != 1 {
|
||||||
|
t.Fatalf("said again: %q", note.said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A phase whose end is not known is unknown and counted in the next; one that ended before the phase before it
|
||||||
|
// counts zero (a pull request opened before the move to fixing).
|
||||||
|
func TestAPhaseNotKnownIsCountedInTheNextAndOneEndedEarlyCountsZero(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
n := c.open(t, "a fix whose pull request came first")
|
||||||
|
opened := c.at
|
||||||
|
c.later(time.Minute)
|
||||||
|
c.fixing(t, n)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#32"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
merged := c.later(4 * time.Minute)
|
||||||
|
c.later(time.Minute)
|
||||||
|
c.delivery(t, 32, "delivered", "leaf", &opened, &merged)
|
||||||
|
ft := get(t, c.harness, n).FixTime
|
||||||
|
if ft == nil {
|
||||||
|
t.Fatal("no fix time")
|
||||||
|
}
|
||||||
|
got := phaseTook(t, ft)
|
||||||
|
if got[PhaseImplementing] != 0 || got[PhaseChecks] != 4*time.Minute || got[PhaseRunning] != time.Minute {
|
||||||
|
t.Fatalf("phases %v", got)
|
||||||
|
}
|
||||||
|
for _, p := range ft.Phases {
|
||||||
|
if p.Name == PhaseReview && (p.State != "unknown" || p.Said == "") {
|
||||||
|
t.Fatalf("review with no approval asked: %+v", p)
|
||||||
|
}
|
||||||
|
if p.Name == PhaseChecks && !strings.Contains(p.Said, "review") {
|
||||||
|
t.Fatalf("checks does not say it counts review: %+v", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ft.Over {
|
||||||
|
t.Fatalf("5m of a leaf's 5m is within it: %+v", ft)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The class is the slowest-budgeted of the fixes'; a fix that moves no module has no budget and is never warned
|
||||||
|
// on; one whose class is not said has none either.
|
||||||
|
func TestAnIssuesClassIsItsSlowestFixAndOneMovingNothingHasNoBudget(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
note := ¬ified{}
|
||||||
|
c.Notify = note.notify
|
||||||
|
|
||||||
|
n := c.open(t, "two fixes, one core")
|
||||||
|
c.fixing(t, n)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#41", "mesh-catalog#42"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
merged := c.later(time.Minute)
|
||||||
|
c.delivery(t, 41, "delivered", "leaf", nil, &merged)
|
||||||
|
if get(t, c.harness, n).FixTime != nil {
|
||||||
|
t.Fatal("a fix time before the last fix is delivered")
|
||||||
|
}
|
||||||
|
c.later(7 * time.Minute)
|
||||||
|
c.delivery(t, 42, "delivered", "core", nil, &merged)
|
||||||
|
ft := get(t, c.harness, n).FixTime
|
||||||
|
if ft == nil || ft.Class != ClassCore || ft.Over || ft.Fix != "mesh-catalog#42" {
|
||||||
|
t.Fatalf("8m of a core fix: %+v", ft)
|
||||||
|
}
|
||||||
|
|
||||||
|
none := c.fixOne(t, "a fix in hq alone", 43, ClassNone, time.Minute, time.Minute, 10*time.Minute, time.Second)
|
||||||
|
ft = get(t, c.harness, none).FixTime
|
||||||
|
if ft == nil || ft.Class != ClassNone || ft.Over || ft.BudgetMS != 0 || !strings.Contains(ft.Said, "no fix budget") {
|
||||||
|
t.Fatalf("a fix moving nothing: %+v", ft)
|
||||||
|
}
|
||||||
|
unknown := c.fixOne(t, "a fix whose class is not said", 44, "", time.Minute, time.Minute, 10*time.Minute, time.Second)
|
||||||
|
ft = get(t, c.harness, unknown).FixTime
|
||||||
|
if ft == nil || ft.Class != "" || ft.Over || !strings.Contains(ft.Said, "no fix budget") {
|
||||||
|
t.Fatalf("a fix of no known class: %+v", ft)
|
||||||
|
}
|
||||||
|
if len(note.said) != 0 {
|
||||||
|
t.Fatalf("warned on a fix with no budget: %q", note.said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nothing is said while a fix is undelivered, however long; once delivered over budget it is said; the next leaf
|
||||||
|
// fix delivered within budget clears it.
|
||||||
|
func TestAFixOverItsBudgetIsSaidOnceFinalAndClearedByTheNextWithin(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
c.Tracker.Approval = c.approval
|
||||||
|
note := ¬ified{}
|
||||||
|
c.Notify = note.notify
|
||||||
|
|
||||||
|
n := c.open(t, "a slow fix")
|
||||||
|
c.fixing(t, n)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#51"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
merged := c.later(20 * time.Minute)
|
||||||
|
c.delivery(t, 51, "published", "leaf", nil, &merged)
|
||||||
|
if _, err := c.Sweep(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(note.said) != 0 {
|
||||||
|
t.Fatalf("said while undelivered: %q", note.said)
|
||||||
|
}
|
||||||
|
c.later(time.Minute)
|
||||||
|
c.delivery(t, 51, "delivered", "leaf", nil, &merged)
|
||||||
|
if len(note.said) != 1 || !strings.HasPrefix(note.said[0], fixOverKey(ClassLeaf)+" warning") {
|
||||||
|
t.Fatalf("said %q", note.said)
|
||||||
|
}
|
||||||
|
if !get(t, c.harness, n).FixTime.Warned {
|
||||||
|
t.Fatal("not marked said")
|
||||||
|
}
|
||||||
|
// A core fix within its budget clears nothing of the leaf's.
|
||||||
|
c.fixOne(t, "a quick core fix", 52, ClassCore, time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
if len(note.clear) != 0 {
|
||||||
|
t.Fatalf("cleared by another class: %q", note.clear)
|
||||||
|
}
|
||||||
|
c.fixOne(t, "a quick leaf fix", 53, ClassLeaf, time.Minute, 30*time.Second, time.Minute, time.Minute)
|
||||||
|
if len(note.clear) != 1 || note.clear[0] != fixOverKey(ClassLeaf) {
|
||||||
|
t.Fatalf("cleared %q", note.clear)
|
||||||
|
}
|
||||||
|
if get(t, c.harness, n).FixTime.Warned {
|
||||||
|
t.Fatal("still marked said after the clear")
|
||||||
|
}
|
||||||
|
if _, err := c.Sweep(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(note.said) != 1 || len(note.clear) != 1 {
|
||||||
|
t.Fatalf("the sweep said again: %q, cleared %q", note.said, note.clear)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A warning the channel did not take is said at the next sweep.
|
||||||
|
func TestAWarningNotTakenIsSaidAtTheNextSweep(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
note := ¬ified{err: errFake("the channel is away")}
|
||||||
|
c.Notify = note.notify
|
||||||
|
n := c.fixOne(t, "slow", 61, ClassLeaf, 5*time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
if get(t, c.harness, n).FixTime == nil {
|
||||||
|
t.Fatal("a fix time is kept although the warning was not taken")
|
||||||
|
}
|
||||||
|
note.err = nil
|
||||||
|
if _, err := c.Sweep(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(note.said) != 1 || !get(t, c.harness, n).FixTime.Warned {
|
||||||
|
t.Fatalf("said %q", note.said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// fix-times answers each class's budget, the window's median and worst, and every issue over its budget with its
|
||||||
|
// longest phase; an issue delivered before the window does not count, so a lever is judged by the medians before
|
||||||
|
// and after it.
|
||||||
|
func TestFixTimesAnswersTheWindowsMedianAndWorst(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
old := c.fixOne(t, "long ago", 70, ClassLeaf, 30*time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
c.later(8 * 24 * time.Hour)
|
||||||
|
a := c.fixOne(t, "four minutes", 71, ClassLeaf, time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
b := c.fixOne(t, "six minutes", 72, ClassLeaf, 3*time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
c.fixOne(t, "eight minutes", 73, ClassLeaf, 5*time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
c.fixOne(t, "a core fix", 74, ClassCore, time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
c.fixOne(t, "hq alone", 75, ClassNone, time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
|
||||||
|
out, err := c.call(t, "fix-times", map[string]any{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ft := out.(*FixTimes)
|
||||||
|
leaf, core := ft.Classes[0], ft.Classes[1]
|
||||||
|
if ft.Days != 7 || leaf.Class != ClassLeaf || leaf.Count != 3 || leaf.Median != "6m0s" || leaf.Worst != "8m0s" ||
|
||||||
|
leaf.Within != 1 || leaf.Over != 2 || leaf.Budget != "5m0s" {
|
||||||
|
t.Fatalf("leaf %+v (issue %d, delivered before the window, must not count)", leaf, old)
|
||||||
|
}
|
||||||
|
if core.Count != 1 || core.Median != "4m0s" || core.Within != 1 {
|
||||||
|
t.Fatalf("core %+v", core)
|
||||||
|
}
|
||||||
|
if len(ft.Over) != 2 || ft.Over[0].Title != "eight minutes" || ft.Over[0].Longest != "implementing 5m0s" || ft.Over[1].Number != b {
|
||||||
|
t.Fatalf("over %+v", ft.Over)
|
||||||
|
}
|
||||||
|
if len(ft.Unbudgeted) != 1 || ft.Unbudgeted[0].Title != "hq alone" {
|
||||||
|
t.Fatalf("unbudgeted %+v", ft.Unbudgeted)
|
||||||
|
}
|
||||||
|
if !strings.Contains(ft.Line, "leaf: median 6m0s, worst 8m0s") {
|
||||||
|
t.Fatalf("line %q", ft.Line)
|
||||||
|
}
|
||||||
|
// Wider, the old one counts and moves the median.
|
||||||
|
out, _ = c.call(t, "fix-times", map[string]any{"days": 30})
|
||||||
|
if l := out.(*FixTimes).Classes[0]; l.Count != 4 || l.Median != "7m0s" {
|
||||||
|
t.Fatalf("over 30 days: %+v", l)
|
||||||
|
}
|
||||||
|
if _, err := c.call(t, "fix-times", map[string]any{"days": 91}); err == nil {
|
||||||
|
t.Fatal("days 91 not refused")
|
||||||
|
}
|
||||||
|
_ = a
|
||||||
|
}
|
||||||
|
|
||||||
|
// An issue moved to fixing before the tracker kept the moment has its start read from its log.
|
||||||
|
func TestTheStartOfAnIssueMovedBeforeItWasKeptIsReadFromItsLog(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
n := c.open(t, "moved to fixing long ago")
|
||||||
|
c.later(time.Minute)
|
||||||
|
start := c.at
|
||||||
|
c.fixing(t, n)
|
||||||
|
c.later(time.Hour)
|
||||||
|
is := get(t, c.harness, n)
|
||||||
|
if is.FixingFrom == nil || !is.FixingFrom.Equal(start) {
|
||||||
|
t.Fatalf("fixing-from %v, want %s", is.FixingFrom, start)
|
||||||
|
}
|
||||||
|
if got, fromLog := c.startOfFixing(is); !got.Equal(start) || fromLog {
|
||||||
|
t.Fatalf("kept: %s from the log %v, want %s kept by the move", got, fromLog, start)
|
||||||
|
}
|
||||||
|
is.FixingFrom = nil
|
||||||
|
if got, fromLog := c.startOfFixing(is); !got.Equal(start) || !fromLog {
|
||||||
|
t.Fatalf("from the log: %s (from the log %v), want %s", got, fromLog, start)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A value carried into the log holds no moment of its own: after a value at diagnosing, the first value at fixing
|
||||||
|
// is the move's own write, and its start is its worker's active (the move), not its worker's since (the start of
|
||||||
|
// the diagnosis, ADR 0313 §1). A first value already at fixing falls back to its worker's since.
|
||||||
|
func TestACarriedIssueStartsAtTheMoveNotTheDiagnosis(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
diagnosed := time.Date(2026, 10, 1, 8, 0, 0, 0, time.UTC)
|
||||||
|
moved := diagnosed.Add(40 * time.Minute)
|
||||||
|
carry := func(n uint64, is Issue) {
|
||||||
|
t.Helper()
|
||||||
|
if _, err := c.log.AppendAt(keyOf(n), Operation{Op: OpCarried, By: Seat, At: c.at, Issue: &is}, c.log.lastOf(keyOf(n))); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
carry(900, Issue{Number: 900, Title: "x", Status: StatusDiagnosing, Worker: &Worker{By: "fix@g14", Since: diagnosed, Active: diagnosed}})
|
||||||
|
carry(900, Issue{Number: 900, Title: "x", Status: StatusFixing, Worker: &Worker{By: "fix@g14", Since: diagnosed, Active: moved}})
|
||||||
|
if got, fromLog := c.startOfFixing(&Issue{Number: 900}); !got.Equal(moved) || !fromLog {
|
||||||
|
t.Fatalf("start %s (from the log %v), want the move at %s, not the diagnosis at %s", got, fromLog, moved, diagnosed)
|
||||||
|
}
|
||||||
|
carry(901, Issue{Number: 901, Title: "y", Status: StatusFixing, Worker: &Worker{By: "fix@g14", Since: diagnosed, Active: moved}})
|
||||||
|
if got, _ := c.startOfFixing(&Issue{Number: 901}); !got.Equal(diagnosed) {
|
||||||
|
t.Fatalf("a first value at fixing: %s, want its worker's since %s", got, diagnosed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// An issue moved to fixing before fix times were measured gets its fix time when its fix is next heard of, with its
|
||||||
|
// start from its log; it is kept and shown, never warned on, and leaves the class's warning to a fix measured whole.
|
||||||
|
func TestAFixWhoseStartCameFromTheLogIsNeverWarnedOn(t *testing.T) {
|
||||||
|
c := newFixClock(t)
|
||||||
|
note := ¬ified{}
|
||||||
|
c.Notify = note.notify
|
||||||
|
n := c.open(t, "fixed before the tracker measured")
|
||||||
|
c.fixing(t, n)
|
||||||
|
if _, err := c.Move(MoveAsk{Number: n, Status: StatusFixing, By: "fix@g14", FixedBy: []string{"mesh-catalog#81"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// As kept before this change: no fixing-from.
|
||||||
|
if _, err := c.change(n, Act{Op: "test", By: Seat}, func(i *Issue) error { i.FixingFrom = nil; return nil }); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
merged := c.later(3 * time.Hour)
|
||||||
|
c.delivery(t, 81, "delivered", ClassLeaf, nil, &merged)
|
||||||
|
ft := get(t, c.harness, n).FixTime
|
||||||
|
if ft == nil || !ft.Over || !ft.FromLog || ft.Warned || ft.TookMS != (3*time.Hour).Milliseconds() {
|
||||||
|
t.Fatalf("fix time %+v", ft)
|
||||||
|
}
|
||||||
|
if _, err := c.Sweep(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(note.said) != 0 {
|
||||||
|
t.Fatalf("warned on a fix whose start came from the log: %q", note.said)
|
||||||
|
}
|
||||||
|
// A fix measured whole after it is warned on as before.
|
||||||
|
c.fixOne(t, "measured whole", 82, ClassLeaf, 6*time.Minute, time.Minute, time.Minute, time.Minute)
|
||||||
|
if len(note.said) != 1 {
|
||||||
|
t.Fatalf("said %q", note.said)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The forge's reviews: the first approval at the head, never one dismissed, stale or of an older head.
|
||||||
|
func TestTheReviewEndsAtTheFirstApprovalAtItsHead(t *testing.T) {
|
||||||
|
raw := json.RawMessage(`{"reviews":[
|
||||||
|
{"state":"APPROVED","at_head":false,"submitted":"2026-10-12T09:01:00Z"},
|
||||||
|
{"state":"REQUEST_CHANGES","at_head":true,"submitted":"2026-10-12T09:02:00Z"},
|
||||||
|
{"state":"APPROVED","at_head":true,"dismissed":true,"submitted":"2026-10-12T09:03:00Z"},
|
||||||
|
{"state":"APPROVED","at_head":true,"submitted":"2026-10-12T09:05:00Z"},
|
||||||
|
{"state":"APPROVED","at_head":true,"submitted":"2026-10-12T09:04:00Z"}]}`)
|
||||||
|
at, found, err := firstApproval(raw)
|
||||||
|
if err != nil || !found || at.Format(time.RFC3339) != "2026-10-12T09:04:00Z" {
|
||||||
|
t.Fatalf("%s %v %v", at, found, err)
|
||||||
|
}
|
||||||
|
// The mesh's own reviews: comments by the account that opened the pull request, whose first line is the verdict.
|
||||||
|
head := "43df7ebc5aa21c0bceacd1580f01f67d83d3495d"
|
||||||
|
comments := json.RawMessage(`{"reviews":[
|
||||||
|
{"state":"COMMENT","at_head":true,"commit":"` + head + `","body":"Verdict: request changes at ` + head + `\n\nBlocking: …","submitted":"2026-10-12T09:01:00Z"},
|
||||||
|
{"state":"COMMENT","at_head":true,"commit":"` + head + `","body":"Looks fine. Verdict: approve at ` + head + `","submitted":"2026-10-12T09:02:00Z"},
|
||||||
|
{"state":"COMMENT","at_head":true,"commit":"` + head + `","body":"Verdict: approve at 43df7eb\n\nAll clauses hold.","submitted":"2026-10-12T09:07:00Z"},
|
||||||
|
{"state":"COMMENT","at_head":true,"commit":"` + head + `","body":"Verdict: approve at ` + head + `\n\nAgain.","submitted":"2026-10-12T09:08:00Z"}]}`)
|
||||||
|
at, found, err = firstApproval(comments)
|
||||||
|
if err != nil || !found || at.Format(time.RFC3339) != "2026-10-12T09:07:00Z" {
|
||||||
|
t.Fatalf("a comment's verdict: %s %v %v", at, found, err)
|
||||||
|
}
|
||||||
|
for _, c := range []struct {
|
||||||
|
body, commit string
|
||||||
|
approves bool
|
||||||
|
}{
|
||||||
|
{"Verdict: approve at " + head, head, true},
|
||||||
|
{"Verdict: approve at 43df7eb", head, true},
|
||||||
|
{"Verdict: approve at 43df7e", head, false}, // shorter than seven characters
|
||||||
|
{"Verdict: approve at 0123456789", head, false}, // another commit
|
||||||
|
{"Verdict: request changes at " + head, head, false}, // not an approval
|
||||||
|
{"Not yet.\nVerdict: approve at " + head, head, false}, // not the first line
|
||||||
|
{"Verdict: approve at 43df7ebc", "", false}, // no commit to hold it to
|
||||||
|
} {
|
||||||
|
if got := verdictApproves(c.body, c.commit); got != c.approves {
|
||||||
|
t.Errorf("%q at %q approves %v, want %v", c.body, c.commit, got, c.approves)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, found, err := firstApproval(json.RawMessage(`{"reviews":[]}`)); err != nil || found {
|
||||||
|
t.Fatalf("no reviews: %v %v", found, err)
|
||||||
|
}
|
||||||
|
if _, _, err := firstApproval(json.RawMessage(`"nothing"`)); err == nil {
|
||||||
|
t.Fatal("an answer with no reviews read as none")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type errFake string
|
||||||
|
|
||||||
|
func (e errFake) Error() string { return string(e) }
|
||||||
@@ -112,6 +112,12 @@ type Issue struct {
|
|||||||
Triage *Verdict `json:"triage,omitempty"`
|
Triage *Verdict `json:"triage,omitempty"`
|
||||||
// Worker is who works on it, while it is diagnosing or fixing (ADR 0284); nil at every other status.
|
// Worker is who works on it, while it is diagnosing or fixing (ADR 0284); nil at every other status.
|
||||||
Worker *Worker `json:"worker,omitempty"`
|
Worker *Worker `json:"worker,omitempty"`
|
||||||
|
// FixingFrom is when it was first moved to fixing: where its fix time starts (novox/hq ADR 0313 §1). A
|
||||||
|
// take-over or a later move back to fixing keeps it.
|
||||||
|
FixingFrom *time.Time `json:"fixing-from,omitempty"`
|
||||||
|
// FixTime is its fix time, phase by phase, once the last of its fixes is delivered (ADR 0313 §4): measured,
|
||||||
|
// never set by a verb.
|
||||||
|
FixTime *FixTime `json:"fix-time,omitempty"`
|
||||||
// Opening is the id of the call that opened it, so a write whose answer was lost is known as its own.
|
// Opening is the id of the call that opened it, so a write whose answer was lost is known as its own.
|
||||||
Opening string `json:"opening"`
|
Opening string `json:"opening"`
|
||||||
// Imported says that the issue, or its status, was taken from its folder in hq by the import of
|
// Imported says that the issue, or its status, was taken from its folder in hq by the import of
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ import (
|
|||||||
const Seat = "issue-tracker"
|
const Seat = "issue-tracker"
|
||||||
|
|
||||||
// Verbs are the seat's verbs, as the manifest declares and claims them.
|
// Verbs are the seat's verbs, as the manifest declares and claims them.
|
||||||
var Verbs = []string{"open", "get", "list", "move", "note", "link", "import", "groom", "set", "tracking", "reserve"}
|
var Verbs = []string{"open", "get", "list", "move", "note", "link", "import", "groom", "set", "tracking", "reserve", "fix-times"}
|
||||||
|
|
||||||
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, format+"\n", a...) }
|
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, format+"\n", a...) }
|
||||||
|
|
||||||
@@ -95,13 +95,16 @@ func main() {
|
|||||||
CallerOf: stdio.CallerOf,
|
CallerOf: stdio.CallerOf,
|
||||||
}
|
}
|
||||||
t.Notify = busNotifier()
|
t.Notify = busNotifier()
|
||||||
t.Deliveries = func(repository, number string) (Delivery, error) {
|
forgeOwner := func() string {
|
||||||
owner := "novox"
|
|
||||||
if s, err := t.Settings(); err == nil && strings.TrimSpace(s.ForgeOwner) != "" {
|
if s, err := t.Settings(); err == nil && strings.TrimSpace(s.ForgeOwner) != "" {
|
||||||
owner = strings.TrimSpace(s.ForgeOwner)
|
return strings.TrimSpace(s.ForgeOwner)
|
||||||
}
|
}
|
||||||
return busDeliveryState(owner)(repository, number)
|
return "novox"
|
||||||
}
|
}
|
||||||
|
t.Deliveries = func(repository, number string) (Delivery, error) {
|
||||||
|
return busDeliveryState(forgeOwner())(repository, number)
|
||||||
|
}
|
||||||
|
t.Approval = busApproval(forgeOwner)
|
||||||
if len(os.Args) > 1 && os.Args[1] == "dry-run" {
|
if len(os.Args) > 1 && os.Args[1] == "dry-run" {
|
||||||
os.Exit(dryRunAtTheTerminal(t, os.Args[2:]))
|
os.Exit(dryRunAtTheTerminal(t, os.Args[2:]))
|
||||||
}
|
}
|
||||||
@@ -381,8 +384,10 @@ func Tools(t *Tracker) []stdio.Tool {
|
|||||||
Description: "One issue whole, by its number: title, status, severity, what it needs, report, who opened it " +
|
Description: "One issue whole, by its number: title, status, severity, what it needs, report, who opened it " +
|
||||||
"and when, the owners it is located in, who works on it and since when (worker, and worker-stale when " +
|
"and when, the owners it is located in, who works on it and since when (worker, and worker-stale when " +
|
||||||
"nobody did anything on it for a day), what fixed it and whether that is delivered, its notes and its " +
|
"nobody did anything on it for a day), what fixed it and whether that is delivered, its notes and its " +
|
||||||
"links; the issues it waits for, those that depend on it (needed-by), and whether it is ready; and " +
|
"links; the issues it waits for, those that depend on it (needed-by), and whether it is ready; " +
|
||||||
"whether its fix is delivered, as the delivery last said it (merged, held and why, delivered).",
|
"whether its fix is delivered, as the delivery last said it (merged, held and why, delivered); and, once " +
|
||||||
|
"the last of its fixes is delivered, its fix time (fix-time) from its first move to fixing, its class and " +
|
||||||
|
"fix budget, and its phases: implementing, review, checks, merge to running.",
|
||||||
Input: schema([]string{"number"}, map[string]any{"number": number("the issue's number")}),
|
Input: schema([]string{"number"}, map[string]any{"number": number("the issue's number")}),
|
||||||
Run: func(m map[string]any) (any, error) {
|
Run: func(m map[string]any) (any, error) {
|
||||||
a, err := readArgs("get", m, "number")
|
a, err := readArgs("get", m, "number")
|
||||||
@@ -710,5 +715,26 @@ func Tools(t *Tracker) []stdio.Tool {
|
|||||||
}
|
}
|
||||||
return t.reserve(t.callerOf(m), ask)
|
return t.reserve(t.callerOf(m), ask)
|
||||||
}},
|
}},
|
||||||
|
{Name: Seat + ".fix-times",
|
||||||
|
Description: "How long fixes take, from the move to fixing to the last fix delivered (the fix time): each class's fix budget (leaf 5m, core 10m), the median and worst fix time of the issues whose last " +
|
||||||
|
"fix was delivered in the last days (7 unless days says otherwise), how many were within and over, and every " +
|
||||||
|
"issue over its budget with its longest phase (implementing, review, checks, merge to running), newest first; " +
|
||||||
|
"issues whose fixes move no module, or whose class is not known, are listed apart with no budget. One issue's " +
|
||||||
|
"fix time, phase by phase, is in its get. A lever meant to shorten fixes is judged by comparing this median " +
|
||||||
|
"over the seven days before it was delivered with the seven days after. Refused: days above 90.",
|
||||||
|
Input: schema(nil, map[string]any{
|
||||||
|
"days": map[string]any{"type": "integer", "minimum": 1, "maximum": MostFixTimesDays, "description": "the window, in days back from now (default 7)"},
|
||||||
|
}),
|
||||||
|
Run: func(m map[string]any) (any, error) {
|
||||||
|
a, err := readArgs("fix-times", m, "days")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
days, err := a.number("days", false)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return t.FixTimes(int(days))
|
||||||
|
}},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -106,7 +106,7 @@ func TestItDeclaresAndHoldsTheIssueTrackerSeatOnceForTheMesh(t *testing.T) {
|
|||||||
// the import and a reservation read the record through the records module (issue 424).
|
// the import and a reservation read the record through the records module (issue 424).
|
||||||
if !reflect.DeepEqual(m.Consumes, []string{"mesh-issues." + EventOpened, "mesh-issues." + EventMoved, "mesh-delivery.transition"}) ||
|
if !reflect.DeepEqual(m.Consumes, []string{"mesh-issues." + EventOpened, "mesh-issues." + EventMoved, "mesh-delivery.transition"}) ||
|
||||||
!reflect.DeepEqual(m.Invokes, []string{"claude-code.claude_code_judge", "seat:mesh-delivery.checks", "seat:operator-channel.notify",
|
!reflect.DeepEqual(m.Invokes, []string{"claude-code.claude_code_judge", "seat:mesh-delivery.checks", "seat:operator-channel.notify",
|
||||||
"records.records_status", "records.records_list", "records.records_read", "mesh-knowledge.knowledge_highest"}) {
|
"gitea.gitea_pull_reviews", "records.records_status", "records.records_list", "records.records_read", "mesh-knowledge.knowledge_highest"}) {
|
||||||
t.Fatalf("consumes %v, invokes %v", m.Consumes, m.Invokes)
|
t.Fatalf("consumes %v, invokes %v", m.Consumes, m.Invokes)
|
||||||
}
|
}
|
||||||
if len(m.Build.Artifacts) != 1 || m.Build.Artifacts[0].From != "cmd/mesh-issues" ||
|
if len(m.Build.Artifacts) != 1 || m.Build.Artifacts[0].From != "cmd/mesh-issues" ||
|
||||||
|
|||||||
@@ -36,8 +36,12 @@ type Tracker struct {
|
|||||||
Judge Judgement
|
Judge Judgement
|
||||||
// Deliveries answers a pull request's delivery (ADR 0279 §3, ADR 0281); nil asks nothing.
|
// Deliveries answers a pull request's delivery (ADR 0279 §3, ADR 0281); nil asks nothing.
|
||||||
Deliveries DeliveryState
|
Deliveries DeliveryState
|
||||||
// Notify tells the operator's channel of a stuck fix (ADR 0281); nil says nothing there.
|
// Notify tells the operator's channel of a stuck fix (ADR 0281) and of a fix over its budget (ADR 0313); nil
|
||||||
|
// says nothing there.
|
||||||
Notify Notifier
|
Notify Notifier
|
||||||
|
// Approval answers when a pull request was first approved at its head (ADR 0313 §4); nil asks nothing, and a
|
||||||
|
// fix's review is counted with its checks.
|
||||||
|
Approval Approval
|
||||||
// CallerOf is who called, for the arguments one call was handed (stdio.CallerOf); nil proves nobody.
|
// CallerOf is who called, for the arguments one call was handed (stdio.CallerOf); nil proves nobody.
|
||||||
CallerOf func(map[string]any) string
|
CallerOf func(map[string]any) string
|
||||||
// Log is the tracker's record, the log `changes` (novox/hq ADR 0300); the bucket `issues` is its fold.
|
// Log is the tracker's record, the log `changes` (novox/hq ADR 0300); the bucket `issues` is its fold.
|
||||||
@@ -456,6 +460,11 @@ func (t *Tracker) move(by string, a MoveAsk) (*Moved, error) {
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
is.Status = to
|
is.Status = to
|
||||||
|
if to == StatusFixing && from != StatusFixing && is.FixingFrom == nil {
|
||||||
|
// The first move to fixing starts the fix time; none after it restarts it (ADR 0313 §1).
|
||||||
|
start := at
|
||||||
|
is.FixingFrom = &start
|
||||||
|
}
|
||||||
if (to == StatusResolved || to == StatusWontfix) && from != to {
|
if (to == StatusResolved || to == StatusWontfix) && from != to {
|
||||||
is.Closed = &at
|
is.Closed = &at
|
||||||
} else if to != StatusResolved && to != StatusWontfix {
|
} else if to != StatusResolved && to != StatusWontfix {
|
||||||
|
|||||||
@@ -17,7 +17,8 @@
|
|||||||
"groom",
|
"groom",
|
||||||
"set",
|
"set",
|
||||||
"tracking",
|
"tracking",
|
||||||
"reserve"
|
"reserve",
|
||||||
|
"fix-times"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@@ -36,7 +37,8 @@
|
|||||||
"groom",
|
"groom",
|
||||||
"set",
|
"set",
|
||||||
"tracking",
|
"tracking",
|
||||||
"reserve"
|
"reserve",
|
||||||
|
"fix-times"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@@ -61,6 +63,7 @@
|
|||||||
"claude-code.claude_code_judge",
|
"claude-code.claude_code_judge",
|
||||||
"seat:mesh-delivery.checks",
|
"seat:mesh-delivery.checks",
|
||||||
"seat:operator-channel.notify",
|
"seat:operator-channel.notify",
|
||||||
|
"gitea.gitea_pull_reviews",
|
||||||
"records.records_status",
|
"records.records_status",
|
||||||
"records.records_list",
|
"records.records_list",
|
||||||
"records.records_read",
|
"records.records_read",
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ operator, it is accepted only on the operator's Approve, given on a channel that
|
|||||||
| `reserve` | `kind`, `for` | the next decision or research number, asked of the issue tracker (`issue-tracker.reserve`); spent whether or not its record is accepted |
|
| `reserve` | `kind`, `for` | the next decision or research number, asked of the issue tracker (`issue-tracker.reserve`); spent whether or not its record is accepted |
|
||||||
| `propose` | the record whole | proposed, or refused naming every rule it breaks; an accepted record changes only by a progressive insight (`insight`). Each version proposed is then reviewed by the knowledge base itself (below) |
|
| `propose` | the record whole | proposed, or refused naming every rule it breaks; an accepted record changes only by a progressive insight (`insight`). Each version proposed is then reviewed by the knowledge base itself (below) |
|
||||||
| `import` | `dry-run`, `commit` | hq's decision records and research efforts at the commit the records module stands at, each whole at its own number |
|
| `import` | `dry-run`, `commit` | hq's decision records and research efforts at the commit the records module stands at, each whole at its own number |
|
||||||
| `return` | `number`, `reason` | drafted clauses sent back to be drafted again |
|
| `return` | `number`, `reason` | drafted clauses sent back to be drafted again; refused for a record whose clauses another record changed or links by number |
|
||||||
|
|
||||||
`ask`, `get`, `holds` and `history` stay within `bound` bytes (4096 unless asked for, at most 65536) and say what they cut
|
`ask`, `get`, `holds` and `history` stay within `bound` bytes (4096 unless asked for, at most 65536) and say what they cut
|
||||||
and how to read the rest. The module's own tool `knowledge_health` answers its health (below).
|
and how to read the rest. The module's own tool `knowledge_health` answers its health (below).
|
||||||
@@ -78,7 +78,7 @@ as text. `get` serves each, and any other field a record of hq wrote under `fiel
|
|||||||
- 25 on the structure: kind, reserved number, title, topic, date, deciders, research status, at least one
|
- 25 on the structure: kind, reserved number, title, topic, date, deciders, research status, at least one
|
||||||
clause, clause numbering, bold lead, clause size, the Context, Consequences and "How it is checked" sections,
|
clause, clause numbering, bold lead, clause size, the Context, Consequences and "How it is checked" sections,
|
||||||
a research effort's overview, link kind, link target, link fit, no link to itself, what it extends, what it
|
a research effort's overview, link kind, link target, link fit, no link to itself, what it extends, what it
|
||||||
supersedes or changes is accepted, a changed clause is in force, a superseded record cited beside its
|
supersedes or changes is accepted, a changed clause is in force or changed (a drafted one too), a superseded record cited beside its
|
||||||
superseder, a graduated effort says what it became, and the links in its text resolve;
|
superseder, a graduated effort says what it became, and the links in its text resolve;
|
||||||
- 4 on the prose: no word the glossary retired, no daemon's line, a deferral names its tracking issue, and a
|
- 4 on the prose: no word the glossary retired, no daemon's line, a deferral names its tracking issue, and a
|
||||||
progressive insight in the dated, marked form.
|
progressive insight in the dated, marked form.
|
||||||
@@ -132,8 +132,16 @@ as text. `get` serves each, and any other field a record of hq wrote under `fiel
|
|||||||
- `partial`: both ends of a change agree;
|
- `partial`: both ends of a change agree;
|
||||||
- `rests-on-live`: no accepted record extends a superseded one.
|
- `rests-on-live`: no accepted record extends a superseded one.
|
||||||
|
|
||||||
A failure is kept as a finding and makes the module unhealthy, so the mesh raises a condition naming the rule
|
A failure is kept as a finding and told to the operator once, as a warning naming the rule and the records
|
||||||
and the records. The acceptance stands.
|
(`operator-channel.notify`, keyed `knowledge.finding.<rule>.<records>`, sent again at the next step when it
|
||||||
|
could not be sent, and cleared once the rule holds for them again). It is a fault of the records, which their
|
||||||
|
authors repair, not of the module: the module's health names it and stays healthy, so it never holds the
|
||||||
|
module's own rollouts (the operator's decision of 2026-10-12 on hq issue 504). The acceptance stands.
|
||||||
|
|
||||||
|
`cited` is also judged ahead, in the review: a decision that names no home and that no accepted (or
|
||||||
|
superseded) record links is not judged and not accepted. Its review, by the knowledge base itself, asks changes
|
||||||
|
naming the rule (hq issue 504). It is not refused at propose, because no record can link a number only
|
||||||
|
reserved: a record citing a new one is proposed after it and accepted, and the new one is proposed again.
|
||||||
|
|
||||||
## The import
|
## The import
|
||||||
|
|
||||||
@@ -170,6 +178,12 @@ the older clause marked `changed`. When every prose record of a topic has been t
|
|||||||
operator is told once (`operator-channel.notify`, a warning) with the counts. `return` sends a record back to be
|
operator is told once (`operator-channel.notify`, a warning) with the counts. `return` sends a record back to be
|
||||||
drafted again.
|
drafted again.
|
||||||
|
|
||||||
|
A drafted clause may be changed by a later record (`changes 0281 §9`, novox/hq issue 494, ADR 0321): on its acceptance the
|
||||||
|
clause is marked `changed` with the change named, keeps its number and stays `drafted`, so it is still never served
|
||||||
|
as in force, and `get` says both. A redraft numbers a record's clauses anew, so `return` refuses a record any of
|
||||||
|
whose clauses another record changed or links by number, served or proposed, naming each link: a link never comes
|
||||||
|
to point at another clause.
|
||||||
|
|
||||||
## Where it keeps its records
|
## Where it keeps its records
|
||||||
|
|
||||||
In a database in the store: the module requires a `postgres-database` provision named `knowledge`, as
|
In a database in the store: the module requires a `postgres-database` provision named `knowledge`, as
|
||||||
@@ -181,14 +195,14 @@ the consolidated schema for a fresh database is `schema.sql`. A test holds the t
|
|||||||
|
|
||||||
- **the bundle**: the verbs, `knowledge_health`, the operator's answers heard, and once a minute (and at once when
|
- **the bundle**: the verbs, `knowledge_health`, the operator's answers heard, and once a minute (and at once when
|
||||||
a record is proposed) the router's record read, reservations never proposed withdrawn, the reviews of proposals,
|
a record is proposed) the router's record read, reservations never proposed withdrawn, the reviews of proposals,
|
||||||
and the drafting worker. It never changes the schema, and
|
and the drafting worker; and it tells the whole-set findings. It never changes the schema, and
|
||||||
refuses to serve while the schema is behind.
|
refuses to serve while the schema is behind.
|
||||||
- **`prepare`** (run once): brings the schema to this version.
|
- **`prepare`** (run once): brings the schema to this version.
|
||||||
- **`keep`** (judged by its unit): judges the whole record set every five minutes and keeps the findings.
|
- **`keep`** (judged by its unit): judges the whole record set every five minutes and keeps the findings.
|
||||||
- **`watch`** (judged by `knowledge_health`, run as the machine's account, the one the runtime runs the bundle
|
- **`watch`** (judged by `knowledge_health`, run as the machine's account, the one the runtime runs the bundle
|
||||||
as, so the tool can read what it writes): every minute, reads whether the database serves and what the
|
as, so the tool can read what it writes): every minute, reads whether the database serves and what the
|
||||||
findings say, and leaves its verdict in `${dir:state}/health.json`. That is the file the health tool answers
|
findings say, and leaves its verdict in `${dir:state}/health.json`. That is the file the health tool answers
|
||||||
from. A verdict older than three minutes is unhealthy.
|
from. A verdict older than three minutes is unhealthy; a whole-set finding is named in it but leaves it healthy.
|
||||||
|
|
||||||
## Settings
|
## Settings
|
||||||
|
|
||||||
|
|||||||
@@ -70,6 +70,13 @@ func held(r *Record, c Clause) string {
|
|||||||
if c.Unconfirmed != "" {
|
if c.Unconfirmed != "" {
|
||||||
s += " (the second judgement did not confirm it: " + c.Unconfirmed + ")"
|
s += " (the second judgement did not confirm it: " + c.Unconfirmed + ")"
|
||||||
}
|
}
|
||||||
|
if len(c.ChangedBy) > 0 {
|
||||||
|
if c.Status == ClauseSuperseded {
|
||||||
|
s += "; superseded by " + strings.Join(c.ChangedBy, ", ")
|
||||||
|
} else {
|
||||||
|
s += "; changed by " + strings.Join(c.ChangedBy, ", ") + ": read both"
|
||||||
|
}
|
||||||
|
}
|
||||||
return s
|
return s
|
||||||
case r.Status == StatusProposed || c.Status == ClauseProposed:
|
case r.Status == StatusProposed || c.Status == ClauseProposed:
|
||||||
return "proposed: not in force"
|
return "proposed: not in force"
|
||||||
|
|||||||
@@ -503,6 +503,15 @@ func (k *KB) Return(caller string, key Key, reason string) (map[string]any, erro
|
|||||||
if r.Drafting == nil {
|
if r.Drafting == nil {
|
||||||
return nil, fmt.Errorf("%s's clauses were written by its author, not drafted: a written clause is changed by a new record", key)
|
return nil, fmt.Errorf("%s's clauses were written by its author, not drafted: a written clause is changed by a new record", key)
|
||||||
}
|
}
|
||||||
|
all, err := k.Store.All(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if held := clauseHolds(r, all); len(held) > 0 {
|
||||||
|
return nil, fmt.Errorf("%s is not returned: drafting it again would number its clauses anew, and these hold them to their numbers: %s. "+
|
||||||
|
"A clause keeps its number once another record links it (novox/hq issue 494). To say a drafted clause is wrong "+
|
||||||
|
"or no longer holds, propose a record that changes it (changes %s §N)", key, strings.Join(held, "; "), strings.TrimPrefix(key.String(), "ADR "))
|
||||||
|
}
|
||||||
r.Clauses = nil
|
r.Clauses = nil
|
||||||
r.Drafting = &Drafting{State: DraftWaiting, Returned: reason, At: k.now()}
|
r.Drafting = &Drafting{State: DraftWaiting, Returned: reason, At: k.now()}
|
||||||
why := "returned to be drafted again: " + reason
|
why := "returned to be drafted again: " + reason
|
||||||
@@ -512,3 +521,36 @@ func (k *KB) Return(caller string, key Key, reason string) (map[string]any, erro
|
|||||||
k.emit(EventReturned, r, caller, why, e.Versions+1)
|
k.emit(EventReturned, r, caller, why, e.Versions+1)
|
||||||
return map[string]any{"record": key.String(), "says": fmt.Sprintf("%s's clauses are taken back and will be drafted again, with your reason; until then it serves none.", key)}, nil
|
return map[string]any{"record": key.String(), "says": fmt.Sprintf("%s's clauses are taken back and will be drafted again, with your reason; until then it serves none.", key)}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// clauseHolds is what holds a record's clauses to their numbers: each clause another record changed or superseded,
|
||||||
|
// and each link of another record, served or proposed, that names one of its clauses. A redraft numbers the clauses
|
||||||
|
// anew, so a record any of these name is never returned: its links would point at other clauses.
|
||||||
|
func clauseHolds(r *Record, all []*Entry) []string {
|
||||||
|
var out []string
|
||||||
|
for _, c := range r.Clauses {
|
||||||
|
if len(c.ChangedBy) > 0 && (c.Status == ClauseChanged || c.Status == ClauseSuperseded) {
|
||||||
|
out = append(out, fmt.Sprintf("§%d is %s by %s", c.N, c.Status, strings.Join(c.ChangedBy, ", ")))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
key := r.Key()
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, e := range all {
|
||||||
|
for _, v := range []*Record{e.Current, e.Pending} {
|
||||||
|
if v == nil || v.Key() == key || v.Status == StatusWithdrawn {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, l := range v.Links {
|
||||||
|
to, ok := l.To.Record()
|
||||||
|
if !ok || to != key || l.To.Clause == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s := fmt.Sprintf("%s (%s) %s §%d", v.Key(), v.Status, l.Kind, l.To.Clause)
|
||||||
|
if !seen[s] {
|
||||||
|
seen[s] = true
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// novox/hq issue 494: a record may change a clause the knowledge base holds only as drafted, and a record whose
|
||||||
|
// clauses another record links by number is never drafted again, so no link ever points at another clause.
|
||||||
|
|
||||||
|
// draftedNotConfirmed drafts 0240 and has the second judgement reject its first clause: it stays drafted.
|
||||||
|
func draftedNotConfirmed(t *testing.T, w *world) Key {
|
||||||
|
t.Helper()
|
||||||
|
k := Key{KindDecision, 240}
|
||||||
|
draftUntil(t, w, k, DraftDrafted)
|
||||||
|
e := mustGet(t, w.k.Store, k)
|
||||||
|
r := e.Current.Clone()
|
||||||
|
r.Clauses[0].Text += " NOT EXACT"
|
||||||
|
if err := w.k.Store.Save(context.Background(), Version{By: "test", Why: "a draft the checker will reject", Record: r}, true, false, e.Revision); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r = draftUntil(t, w, k, DraftDone)
|
||||||
|
if r.Clauses[0].Source != SourceDrafted || !r.Accepted() {
|
||||||
|
t.Fatalf("0240 §1 is not a drafted clause of an accepted record: %s %+v", r.Status, r.Clauses[0])
|
||||||
|
}
|
||||||
|
return k
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestARecordMayChangeAClauseHeldOnlyAsDrafted(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s Store) {
|
||||||
|
w := imported(t, s)
|
||||||
|
k := draftedNotConfirmed(t, w)
|
||||||
|
n := w.reserved(t, "author@test")
|
||||||
|
p := valid(n)
|
||||||
|
p.Links = append(p.Links, LinkAsk{Kind: LinkChanges, To: "0240 §1", From: 2})
|
||||||
|
if _, err := w.k.Propose("author@test", p); err != nil {
|
||||||
|
t.Fatalf("a change of a drafted clause was refused: %v", err)
|
||||||
|
}
|
||||||
|
w.reviewAll(t)
|
||||||
|
if e := mustGet(t, s, Key{KindDecision, n}); !e.Current.Accepted() {
|
||||||
|
t.Fatalf("not accepted: %s", e.Current.Status)
|
||||||
|
}
|
||||||
|
// The drafted clause is changed by it, keeps its number and stays drafted: still never served as in force.
|
||||||
|
r := mustGet(t, s, k).Current
|
||||||
|
c := r.Clause(1)
|
||||||
|
want := itoa4(n) + " §2"
|
||||||
|
if c == nil || c.Status != ClauseChanged || c.Source != SourceDrafted || len(c.ChangedBy) != 1 || !strings.HasSuffix(c.ChangedBy[0], want) {
|
||||||
|
t.Fatalf("0240 §1 after the change: %+v", c)
|
||||||
|
}
|
||||||
|
if Served(r, *c) {
|
||||||
|
t.Fatal("a drafted clause that was changed is served as holding")
|
||||||
|
}
|
||||||
|
g, err := w.k.Get(GetAsk{Kind: KindDecision, Number: 240})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if h := g.Clauses[0].Holds; !strings.HasPrefix(h, "drafted, not confirmed") || !strings.Contains(h, "changed by ADR "+want) {
|
||||||
|
t.Fatalf("get says of 0240 §1: %q", h)
|
||||||
|
}
|
||||||
|
// The whole-set rules find nothing wrong with the change.
|
||||||
|
f, _ := s.Findings(context.Background())
|
||||||
|
for _, x := range f {
|
||||||
|
if x.Rule == "partial" {
|
||||||
|
t.Fatalf("a whole-set finding: %+v", x)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestARecordWhoseClausesAreLinkedByNumberIsNeverReturned(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s Store) {
|
||||||
|
w := imported(t, s)
|
||||||
|
k := draftedNotConfirmed(t, w)
|
||||||
|
// A proposed record cites 0240 §2: returning 0240 would point it at whatever is drafted second next time.
|
||||||
|
n := w.reserved(t, "author@test")
|
||||||
|
p := valid(n)
|
||||||
|
p.Links = append(p.Links, LinkAsk{Kind: LinkCites, To: "0240 §2", From: 1})
|
||||||
|
w.judge.review = `{"verdict":"changes","findings":["not yet"]}`
|
||||||
|
if _, err := w.k.Propose("author@test", p); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
w.reviewAll(t)
|
||||||
|
before := mustGet(t, s, k)
|
||||||
|
_, err := w.k.Return("operator@shanks", k, "clause 1 says more than the section")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("a record a proposed record links by clause was returned")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "ADR "+itoa4(n)+" (proposed) cites §2") || !strings.Contains(err.Error(), "changes 0240 §N") {
|
||||||
|
t.Fatalf("the refusal does not say what holds the clauses or the way forward: %v", err)
|
||||||
|
}
|
||||||
|
after := mustGet(t, s, k)
|
||||||
|
if after.Versions != before.Versions || len(after.Current.Clauses) != len(before.Current.Clauses) || len(w.said.of(EventReturned)) != 0 {
|
||||||
|
t.Fatalf("a refused return changed 0240: %d versions (was %d), %d clauses", after.Versions, before.Versions, len(after.Current.Clauses))
|
||||||
|
}
|
||||||
|
|
||||||
|
// A clause another record changed holds the record to its numbers too, with no link left to scan.
|
||||||
|
w2 := imported(t, newMemStore())
|
||||||
|
k2 := draftedNotConfirmed(t, w2)
|
||||||
|
n2 := w2.reserved(t, "author@test")
|
||||||
|
p2 := valid(n2)
|
||||||
|
p2.Links = append(p2.Links, LinkAsk{Kind: LinkChanges, To: "0240 §1", From: 2})
|
||||||
|
w2.accepted(t, "author@test", p2)
|
||||||
|
if _, err := w2.k.Return("operator@shanks", k2, "again"); err == nil || !strings.Contains(err.Error(), "§1 is changed by ADR "+itoa4(n2)+" §2") {
|
||||||
|
t.Fatalf("a record with a changed clause was returned, or the refusal does not name the change: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -514,6 +514,73 @@ func (k *KB) judgeSet(ctx context.Context) []Finding {
|
|||||||
return findings
|
return findings
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// findingPrefix keys the notice of one whole-set finding: the prefix, the rule, and the records it names.
|
||||||
|
const findingPrefix = "knowledge.finding."
|
||||||
|
|
||||||
|
func findingKey(f Finding) string {
|
||||||
|
names := make([]string, len(f.Records))
|
||||||
|
for i, r := range f.Records {
|
||||||
|
names[i] = strings.ReplaceAll(strings.ToLower(r), " ", "-")
|
||||||
|
}
|
||||||
|
return findingPrefix + f.Rule + "." + strings.Join(names, ".")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TellFindings, called by the background work and never under the write lock, tells the operator, once, each finding of the rules that judge the whole record set, as a warning
|
||||||
|
// naming the rule and the records, and clears the notice once the rule holds for them again. A finding is a fault
|
||||||
|
// of the records, which their authors repair; it is not the module failing, so it leaves the module's health as it
|
||||||
|
// is and never holds the module's own rollouts (the operator's decision of 2026-10-12 on novox/hq issue 504, as
|
||||||
|
// issue 508 decided for a proposal never reviewed). A notice that cannot be sent is not kept as told, and is sent
|
||||||
|
// again at the next step.
|
||||||
|
func (k *KB) TellFindings() {
|
||||||
|
if k.Notify == nil || k.Clear == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
findings, err := k.Store.Findings(ctx)
|
||||||
|
if err != nil {
|
||||||
|
k.logf("[mesh-knowledge] the whole-set findings cannot be read to tell them: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
now := map[string]bool{}
|
||||||
|
for _, f := range findings {
|
||||||
|
key := findingKey(f)
|
||||||
|
now[key] = true
|
||||||
|
first, err := k.Store.Told(ctx, key)
|
||||||
|
if err != nil || !first {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
records := strings.Join(f.Records, ", ")
|
||||||
|
summary := fmt.Sprintf("The rule %s, which judges the whole record set, fails: %s.", f.Rule, f.Says)
|
||||||
|
explanation := "The knowledge base works; its records break a rule. The acceptance stands. The author of " + records +
|
||||||
|
" repairs it: a decision cited by nothing names its home in a progressive insight (propose with homes and insight), " +
|
||||||
|
"or a record that cites it is accepted. get and links show the record."
|
||||||
|
resolved := fmt.Sprintf("The rule %s holds again for %s", f.Rule, records)
|
||||||
|
if err := k.Notify(key, summary, "A rule of the record set fails: "+records, explanation, resolved); err != nil {
|
||||||
|
k.logf("[mesh-knowledge] the operator could not be told that %s fails for %s: %v; told again at the next step", f.Rule, records, err)
|
||||||
|
if uerr := k.Store.Untold(ctx, key); uerr != nil {
|
||||||
|
k.logf("[mesh-knowledge] and that it was not told could not be kept: %v", uerr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
told, err := k.Store.ToldOf(ctx, findingPrefix)
|
||||||
|
if err != nil {
|
||||||
|
k.logf("[mesh-knowledge] the notices of whole-set findings cannot be read: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, key := range told {
|
||||||
|
if now[key] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := k.Clear(key, ""); err != nil {
|
||||||
|
k.logf("[mesh-knowledge] the notice %s could not be cleared: %v; it is cleared again at the next step", key, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := k.Store.Untold(ctx, key); err != nil {
|
||||||
|
k.logf("[mesh-knowledge] the notice %s was cleared, and that could not be kept: %v", key, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- reservations that were never proposed -------------------------------------------------------------------
|
// ---- reservations that were never proposed -------------------------------------------------------------------
|
||||||
|
|
||||||
// Withdraw withdraws every number reserved longer ago than the settings allow and never proposed: it stays spent,
|
// Withdraw withdraws every number reserved longer ago than the settings allow and never proposed: it stays spent,
|
||||||
|
|||||||
@@ -40,6 +40,9 @@ func TestMain(m *testing.M) {
|
|||||||
serve(k, os.Getenv("MESH_KNOWLEDGE_STATUS"))
|
serve(k, os.Getenv("MESH_KNOWLEDGE_STATUS"))
|
||||||
os.Exit(0)
|
os.Exit(0)
|
||||||
}
|
}
|
||||||
|
// Every test runs in a zone that is not UTC, so a time said locally is proven to be, wherever the tests run. It
|
||||||
|
// is set once, before any goroutine starts: a test that set it while a bus test's server read it raced.
|
||||||
|
time.Local = testZone
|
||||||
code := m.Run()
|
code := m.Run()
|
||||||
stopPostgres()
|
stopPostgres()
|
||||||
os.Exit(code)
|
os.Exit(code)
|
||||||
@@ -436,9 +439,13 @@ func valid(n int) ProposeAsk {
|
|||||||
{Name: "Consequences", Text: "Merges land sooner."},
|
{Name: "Consequences", Text: "Merges land sooner."},
|
||||||
{Name: "How it is checked", Text: "The controller's merge-window tests."},
|
{Name: "How it is checked", Text: "The controller's merge-window tests."},
|
||||||
},
|
},
|
||||||
Links: []LinkAsk{{Kind: LinkExtends, To: "0276"}, {Kind: LinkChanges, To: "0276 §1", From: 1}}}
|
Links: []LinkAsk{{Kind: LinkExtends, To: "0276"}, {Kind: LinkChanges, To: "0276 §1", From: 1}},
|
||||||
|
Homes: []string{testHome}}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// testHome is the hq design valid names as its home: a decision is accepted only cited (novox/hq issue 504).
|
||||||
|
const testHome = "03-DESIGN/01-to-be/55-the-knowledge-base.md"
|
||||||
|
|
||||||
// imported is a world whose knowledge base holds the fixture's records.
|
// imported is a world whose knowledge base holds the fixture's records.
|
||||||
func imported(t *testing.T, s Store) *world {
|
func imported(t *testing.T, s Store) *world {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|||||||
@@ -164,6 +164,8 @@ func background(k *KB) {
|
|||||||
logf("[mesh-knowledge] withdrew %d reservation(s) never proposed", n)
|
logf("[mesh-knowledge] withdrew %d reservation(s) never proposed", n)
|
||||||
}
|
}
|
||||||
review()
|
review()
|
||||||
|
// The keeper judges the record set in a process of its own, which tells nobody: its findings are told here.
|
||||||
|
k.TellFindings()
|
||||||
for {
|
for {
|
||||||
said, err := k.DraftStep()
|
said, err := k.DraftStep()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -552,10 +554,11 @@ type HealthAnswer struct {
|
|||||||
At time.Time `json:"at,omitempty"`
|
At time.Time `json:"at,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Health is whether the knowledge base works: unhealthy when its database cannot serve, or when a rule that judges
|
// Health is whether the knowledge base works: unhealthy when its database cannot serve. A rule that judges the whole
|
||||||
// the whole record set fails — the mesh raises a condition naming the rule and the records. A proposal that ran out
|
// record set and fails is a fault of the records, which their authors repair, not of the module: it is told to the
|
||||||
// of review tries waits on its author, not on the module: it is told to its author and the operator, and named
|
// operator as a warning naming the rule and the records (TellFindings), and named here, but leaves the module
|
||||||
// here, but leaves the module healthy, so it never holds the module's own rollouts (novox/hq issue 508).
|
// healthy (the operator's decision of 2026-10-12 on novox/hq issue 504). So does a proposal that ran out of review
|
||||||
|
// tries (novox/hq issue 508). Neither holds the module's own rollouts.
|
||||||
func (k *KB) Health(ready Healther) HealthAnswer {
|
func (k *KB) Health(ready Healther) HealthAnswer {
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 8*time.Second)
|
ctx, cancel := context.WithTimeout(context.Background(), 8*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
@@ -586,17 +589,14 @@ func (k *KB) Health(ready Healther) HealthAnswer {
|
|||||||
waiting = fmt.Sprintf("%d proposed record(s) could not be reviewed in %d tries and wait for their authors to propose them again: %s",
|
waiting = fmt.Sprintf("%d proposed record(s) could not be reviewed in %d tries and wait for their authors to propose them again: %s",
|
||||||
n, MostReviewTries, strings.Join(unreviewed, "; "))
|
n, MostReviewTries, strings.Join(unreviewed, "; "))
|
||||||
}
|
}
|
||||||
|
why := "the database serves and every rule that judges the whole record set holds"
|
||||||
if len(says) > 0 {
|
if len(says) > 0 {
|
||||||
if len(says) > 8 {
|
if len(says) > 8 {
|
||||||
says = append(says[:8], fmt.Sprintf("and %d more", len(says)-8))
|
says = append(says[:8], fmt.Sprintf("and %d more", len(says)-8))
|
||||||
}
|
}
|
||||||
why := fmt.Sprintf("%d rule(s) that judge the whole record set fail: %s", len(findings), strings.Join(says, "; "))
|
why = fmt.Sprintf("the database serves; %d rule(s) that judge the whole record set fail, told to the operator as a warning "+
|
||||||
if waiting != "" {
|
"for the records' authors to repair: %s", len(findings), strings.Join(says, "; "))
|
||||||
why += ". " + waiting
|
|
||||||
}
|
|
||||||
return HealthAnswer{State: "unhealthy", Why: why}
|
|
||||||
}
|
}
|
||||||
why := "the database serves and every rule that judges the whole record set holds"
|
|
||||||
if waiting != "" {
|
if waiting != "" {
|
||||||
why += "; " + waiting
|
why += "; " + waiting
|
||||||
}
|
}
|
||||||
@@ -605,10 +605,11 @@ func (k *KB) Health(ready Healther) HealthAnswer {
|
|||||||
|
|
||||||
func healthTool(status string, now func() time.Time) stdio.Tool {
|
func healthTool(status string, now func() time.Time) stdio.Tool {
|
||||||
return stdio.Tool{Name: HealthTool,
|
return stdio.Tool{Name: HealthTool,
|
||||||
Description: "How the knowledge base is, as its watch last saw it: healthy when its database serves this version and every " +
|
Description: "How the knowledge base is, as its watch last saw it: healthy when its database serves this version, unhealthy " +
|
||||||
"rule that judges the whole record set holds; unhealthy otherwise, naming the rule and the records — a decision accepted " +
|
"otherwise. A rule that judges the whole record set and fails — a decision accepted that nothing cites, a supersession " +
|
||||||
"that nothing cites, a supersession or a change said on one end only, a record extending a superseded one. A " +
|
"or a change said on one end only, a record extending a superseded one — is named but leaves it healthy: it is told " +
|
||||||
"proposed record that ran out of review tries is named but leaves it healthy: it waits on its author (novox/hq issue 508).",
|
"to the operator as a warning for the records' authors (novox/hq issue 504). So is a proposed record that ran out of " +
|
||||||
|
"review tries: it waits on its author (novox/hq issue 508).",
|
||||||
Run: func(map[string]any) (any, error) { return ReadStatus(status, now()), nil }}
|
Run: func(map[string]any) (any, error) { return ReadStatus(status, now()), nil }}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -918,7 +919,8 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
|
|||||||
"link to nothing, a clause without a bold lead, a superseded record cited alone, an unknown topic, a retired word. A " +
|
"link to nothing, a clause without a bold lead, a superseded record cited alone, an unknown topic, a retired word. A " +
|
||||||
"proposal is revised by proposing it again; an accepted record changes only by a progressive insight (insight: what " +
|
"proposal is revised by proposing it again; an accepted record changes only by a progressive insight (insight: what " +
|
||||||
"stood before). The knowledge base then reviews each version itself, in a judgement that sees only the record, the " +
|
"stood before). The knowledge base then reviews each version itself, in a judgement that sees only the record, the " +
|
||||||
"records it links and the rules: on approve it is accepted (a record that needs the operator waits for the operator's " +
|
"records it links and the rules. A decision that names no home and that no accepted record links is not judged: " +
|
||||||
|
"its review asks changes for the whole-set rule cited, which its acceptance would break. On approve it is accepted (a record that needs the operator waits for the operator's " +
|
||||||
"Approve), on changes it stays proposed with the findings for its author to revise, and when no judgement can be had " +
|
"Approve), on changes it stays proposed with the findings for its author to revise, and when no judgement can be had " +
|
||||||
"it stays proposed and says so. Nothing proposed is in force.",
|
"it stays proposed and says so. Nothing proposed is in force.",
|
||||||
Input: schema([]string{"number", "title"}, map[string]any{
|
Input: schema([]string{"number", "title"}, map[string]any{
|
||||||
@@ -933,7 +935,7 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
|
|||||||
"links": map[string]any{"type": "array", "description": "its links: kind, to (\"0276\", \"0276 §4\", \"research 045\", \"issue 365\", a design's path), from (the clause it goes out from)",
|
"links": map[string]any{"type": "array", "description": "its links: kind, to (\"0276\", \"0276 §4\", \"research 045\", \"issue 365\", a design's path), from (the clause it goes out from)",
|
||||||
"items": map[string]any{"type": "object", "properties": map[string]any{"kind": map[string]any{"type": "string", "enum": LinkKinds},
|
"items": map[string]any{"type": "object", "properties": map[string]any{"kind": map[string]any{"type": "string", "enum": LinkKinds},
|
||||||
"to": text("the target"), "from": integer("the clause of this record it goes out from")}, "required": []string{"kind", "to"}}},
|
"to": text("the target"), "from": integer("the clause of this record it goes out from")}, "required": []string{"kind", "to"}}},
|
||||||
"homes": texts("hq documents that cite it, such as a design's path"),
|
"homes": texts("hq documents that cite it, such as a design's path; a decision needs one unless an accepted record links it"),
|
||||||
"needs-operator": flag("it holds only once the operator approves it"),
|
"needs-operator": flag("it holds only once the operator approves it"),
|
||||||
"status": map[string]any{"type": "string", "enum": ResearchStatuses, "description": "a research effort's status"},
|
"status": map[string]any{"type": "string", "enum": ResearchStatuses, "description": "a research effort's status"},
|
||||||
"insight": text("for an accepted record: a progressive insight, saying what stood before"),
|
"insight": text("for an accepted record: a progressive insight, saying what stood before"),
|
||||||
@@ -1008,7 +1010,8 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
|
|||||||
{Name: Seat + ".return",
|
{Name: Seat + ".return",
|
||||||
Description: "Send the clauses drafted from a prose decision back to be drafted again, with your reason, which the next " +
|
Description: "Send the clauses drafted from a prose decision back to be drafted again, with your reason, which the next " +
|
||||||
"drafting reads. Until they are drafted and confirmed again the record serves none. Refused: a record whose clauses its " +
|
"drafting reads. Until they are drafted and confirmed again the record serves none. Refused: a record whose clauses its " +
|
||||||
"author wrote, an empty reason.",
|
"author wrote, an empty reason, and a record whose clauses another record changed or links by number (a redraft " +
|
||||||
|
"numbers them anew; a record that changes the drafted clause says it is wrong instead).",
|
||||||
Input: schema([]string{"number", "reason"}, map[string]any{"kind": kindArg(), "number": integer("the record's number"),
|
Input: schema([]string{"number", "reason"}, map[string]any{"kind": kindArg(), "number": integer("the record's number"),
|
||||||
"reason": text("what is wrong with the drafted clauses")}),
|
"reason": text("what is wrong with the drafted clauses")}),
|
||||||
Run: func(m map[string]any) (any, error) {
|
Run: func(m map[string]any) (any, error) {
|
||||||
|
|||||||
@@ -408,7 +408,10 @@ func (k *KB) reviewOne(key Key) (string, error) {
|
|||||||
k.mu.Lock()
|
k.mu.Lock()
|
||||||
e, err := k.Store.Get(ctx, key)
|
e, err := k.Store.Get(ctx, key)
|
||||||
var linked []*Record
|
var linked []*Record
|
||||||
|
var all []*Entry
|
||||||
|
var allErr error
|
||||||
if err == nil && e != nil {
|
if err == nil && e != nil {
|
||||||
|
all, allErr = k.Store.All(ctx)
|
||||||
seen := map[Key]bool{key: true}
|
seen := map[Key]bool{key: true}
|
||||||
for _, l := range e.Working().Links {
|
for _, l := range e.Working().Links {
|
||||||
to, ok := l.To.Record()
|
to, ok := l.To.Record()
|
||||||
@@ -430,9 +433,20 @@ func (k *KB) reviewOne(key Key) (string, error) {
|
|||||||
var verdict string
|
var verdict string
|
||||||
var findings []string
|
var findings []string
|
||||||
by := ReviewBy
|
by := ReviewBy
|
||||||
if len(question) > MostReviewQuestion {
|
uncited := ""
|
||||||
|
if allErr == nil {
|
||||||
|
uncited = UncitedOnAcceptance(w, all)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case allErr != nil:
|
||||||
|
err = fmt.Errorf("the record set cannot be read to judge whether a record cites it: %v", allErr)
|
||||||
|
case uncited != "":
|
||||||
|
// The whole-set rule cited, judged ahead: changes, asked by the knowledge base itself without a judgement,
|
||||||
|
// so its acceptance never turns the module's health red for it (novox/hq issue 504).
|
||||||
|
verdict, findings = VerdictChanges, []string{"the rule cited, which judges the whole record set: " + uncited}
|
||||||
|
case len(question) > MostReviewQuestion:
|
||||||
err = fmt.Errorf("the record is %d bytes as a question, and a judgement takes at most %d: shorten it", len(question), MostReviewQuestion)
|
err = fmt.Errorf("the record is %d bytes as a question, and a judgement takes at most %d: shorten it", len(question), MostReviewQuestion)
|
||||||
} else {
|
default:
|
||||||
var answer json.RawMessage
|
var answer json.RawMessage
|
||||||
var judge string
|
var judge string
|
||||||
answer, judge, err = k.Judge(question, reviewSchema, k.judgeModel())
|
answer, judge, err = k.Judge(question, reviewSchema, k.judgeModel())
|
||||||
|
|||||||
@@ -155,13 +155,16 @@ func TestProposedAgainAfterTheLastTryClearsTheNotice(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// inZone sets the local zone to one that is not UTC for one test, so a time said locally is proven to be, wherever
|
// testZone is the local zone every test runs in, set once by TestMain: not UTC.
|
||||||
// the test runs.
|
var testZone = time.FixedZone("CEST", 2*60*60)
|
||||||
|
|
||||||
|
// inZone says the test needs the local zone not to be UTC; TestMain set it before any goroutine started. Setting it
|
||||||
|
// here, under a running bus server's goroutines, was a data race the race detector failed.
|
||||||
func inZone(t *testing.T) {
|
func inZone(t *testing.T) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
was := time.Local
|
if time.Local != testZone {
|
||||||
time.Local = time.FixedZone("CEST", 2*60*60)
|
t.Fatal("the local zone is not the tests' own: TestMain sets it")
|
||||||
t.Cleanup(func() { time.Local = was })
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// novox/hq issue 497: a clear the operator's channel refused keeps its notice as told, and is sent again at the next
|
// novox/hq issue 497: a clear the operator's channel refused keeps its notice as told, and is sent again at the next
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ var PerRecordRules = []Rule{
|
|||||||
{"self-link", "no link points at the record itself", "records.py supersession"},
|
{"self-link", "no link points at the record itself", "records.py supersession"},
|
||||||
{"rests-on", "a record it extends is accepted or proposed, never superseded or withdrawn", "records.py rests-on"},
|
{"rests-on", "a record it extends is accepted or proposed, never superseded or withdrawn", "records.py rests-on"},
|
||||||
{"supersedes-accepted", "a record it supersedes or changes is accepted", "records.py supersession, partial-supersession"},
|
{"supersedes-accepted", "a record it supersedes or changes is accepted", "records.py supersession, partial-supersession"},
|
||||||
{"changes-in-force", "a clause it changes is in force or changed, never superseded, proposed or drafted", "records.py partial-supersession"},
|
{"changes-in-force", "a clause it changes is in force or changed, never superseded or proposed; a drafted one may be changed", "records.py partial-supersession"},
|
||||||
{"live-citation", "a superseded record it cites is cited beside the record that replaced it", "records.py live-citation"},
|
{"live-citation", "a superseded record it cites is cited beside the record that replaced it", "records.py live-citation"},
|
||||||
{"became", "a graduated research effort names what it became", "cycle.py research"},
|
{"became", "a graduated research effort names what it became", "cycle.py research"},
|
||||||
{"text-links", "every link its text writes to a decision record or research effort names one the knowledge base holds", "records.py links"},
|
{"text-links", "every link its text writes to a decision record or research effort names one the knowledge base holds", "records.py links"},
|
||||||
@@ -261,8 +261,10 @@ func JudgeRecord(r *Record, held *Entry, lookup Lookup, ref *Reference) []Refusa
|
|||||||
if !target.Accepted() || target.Status == StatusSuperseded {
|
if !target.Accepted() || target.Status == StatusSuperseded {
|
||||||
refuse("supersedes-accepted", "%s changes %s, which is %s, not accepted", self, to, target.Status)
|
refuse("supersedes-accepted", "%s changes %s, which is %s, not accepted", self, to, target.Status)
|
||||||
} else if c := target.Clause(l.To.Clause); c != nil {
|
} else if c := target.Clause(l.To.Clause); c != nil {
|
||||||
if (c.Status != ClauseInForce && c.Status != ClauseChanged) || c.Source == SourceDrafted {
|
// A drafted clause may be changed (novox/hq issue 494, ADR 0321): the change is recorded against the clause the
|
||||||
refuse("changes-in-force", "%s changes %s §%d, which is %s (%s)", self, to, l.To.Clause, c.Status, c.Source)
|
// knowledge base holds, by its number, which nothing renumbers once another record links it (Return).
|
||||||
|
if c.Status != ClauseInForce && c.Status != ClauseChanged {
|
||||||
|
refuse("changes-in-force", "%s changes %s §%d, which is %s", self, to, l.To.Clause, c.Status)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
case LinkCites:
|
case LinkCites:
|
||||||
@@ -546,6 +548,36 @@ func JudgeSet(records []*Record) []Finding {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UncitedOnAcceptance is the whole-set rule `cited` judged ahead, on a decision's working version as it would
|
||||||
|
// stand once accepted, over the records as JudgeSet reads them: "" when it names a home or a record accepted (or
|
||||||
|
// superseded) links it, else what the rule would say. hq held this rule at the merge of the record's own pull
|
||||||
|
// request, which carried the design citing it; the knowledge base holds it in the review, before the judgement, so
|
||||||
|
// an acceptance never leaves a decision cited by nothing (novox/hq issue 504). It is not refused at propose: no
|
||||||
|
// record can link a number only reserved, so a record that will cite a new one is proposed after it, accepted, and
|
||||||
|
// the new one proposed again. A citing record changed by a correction after the acceptance is what the rule, run
|
||||||
|
// after every acceptance, still says. An imported record was held to hq's checks, and its correction is not judged
|
||||||
|
// here, as JudgeSet does not judge it.
|
||||||
|
func UncitedOnAcceptance(r *Record, all []*Entry) string {
|
||||||
|
if r.Kind != KindDecision || len(r.Homes) > 0 || (r.Pending != nil && r.AcceptedAt.IsZero()) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
k := r.Key()
|
||||||
|
for _, e := range all {
|
||||||
|
from := e.Current
|
||||||
|
if from.Key() == k || !(from.Accepted() || from.Status == StatusSuperseded) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, l := range from.Links {
|
||||||
|
if to, ok := l.To.Record(); ok && to == k {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s would be accepted with nothing citing it: it names no home and no accepted record links it. Name "+
|
||||||
|
"its home (homes: the hq design or playbook that cites it), or have the record that links it accepted first; then "+
|
||||||
|
"propose this one again", k)
|
||||||
|
}
|
||||||
|
|
||||||
// RuleFrom is the day each dated rule began to hold a record, as hq's checks hold it: a record dated earlier was
|
// RuleFrom is the day each dated rule began to hold a record, as hq's checks hold it: a record dated earlier was
|
||||||
// never held to it, and a dated correction of it (a progressive insight, which keeps the record's date) is not.
|
// never held to it, and a dated correction of it (a progressive insight, which keeps the record's date) is not.
|
||||||
// words.py held the retired words from 2026-10-07 and the daemons' lines from 2026-10-09; cycle.py the deferrals
|
// words.py held the retired words from 2026-10-07 and the daemons' lines from 2026-10-09; cycle.py the deferrals
|
||||||
|
|||||||
@@ -1,38 +1,119 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// to-be 55 §10: "A whole-set rule that fails is loud — the module's test: an acceptance that leaves a decision
|
// A decision nothing would cite is not accepted: its review asks changes itself, without a judgement, naming the
|
||||||
// cited by nothing raises the condition." The condition is the mesh's, raised when the module's health answers
|
// rule cited, and the module stays healthy (novox/hq issue 504). Before, the review approved it, the acceptance
|
||||||
// unhealthy; the test reads the health answer the node-engine reads.
|
// broke the rule, and the module turned unhealthy for a record its author could only repair after the fact.
|
||||||
func TestAnAcceptanceThatLeavesADecisionCitedByNothingMakesTheModuleUnhealthy(t *testing.T) {
|
func TestADecisionNothingWouldCiteIsNotAccepted(t *testing.T) {
|
||||||
eachStore(t, func(t *testing.T, s Store) {
|
eachStore(t, func(t *testing.T, s Store) {
|
||||||
w := imported(t, s)
|
w := imported(t, s)
|
||||||
if h := w.k.Health(nil); !h.Healthy {
|
|
||||||
t.Fatalf("unhealthy after the import of real records: %s", h.Why)
|
|
||||||
}
|
|
||||||
n := w.reserved(t, "author@g14")
|
n := w.reserved(t, "author@g14")
|
||||||
w.accepted(t, "author@g14", valid(n))
|
p := valid(n)
|
||||||
h := w.k.Health(nil)
|
p.Homes = nil
|
||||||
if h.Healthy || h.State != "unhealthy" || !strings.Contains(h.Why, "the rule cited") || !strings.Contains(h.Why, itoaKey(n)) {
|
if _, err := w.k.Propose("author@g14", p); err != nil {
|
||||||
t.Fatalf("an accepted decision nothing cites left the module %+v", h)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
// The acceptance stands: the condition says it, and undoes nothing.
|
judged := len(w.judge.reviews)
|
||||||
|
w.reviewAll(t)
|
||||||
|
e := mustGet(t, s, Key{KindDecision, n})
|
||||||
|
if e.Current.Status != StatusProposed {
|
||||||
|
t.Fatalf("a decision nothing cites was %s by its review", e.Current.Status)
|
||||||
|
}
|
||||||
|
rv := e.Current.Reviews
|
||||||
|
if len(rv) == 0 || rv[len(rv)-1].Verdict != VerdictChanges || !strings.Contains(rv[len(rv)-1].Findings, "the rule cited") ||
|
||||||
|
rv[len(rv)-1].By != ReviewBy {
|
||||||
|
t.Fatalf("its review asked no changes for the rule cited: %+v", rv)
|
||||||
|
}
|
||||||
|
if len(w.judge.reviews) != judged {
|
||||||
|
t.Fatal("a judgement was spent on a record the rule cited refuses")
|
||||||
|
}
|
||||||
|
if h := w.k.Health(nil); !h.Healthy {
|
||||||
|
t.Fatalf("unhealthy for a record not accepted: %s", h.Why)
|
||||||
|
}
|
||||||
|
// Named with its home, it is accepted, and the module stays healthy.
|
||||||
|
p.Homes = []string{testHome}
|
||||||
|
w.accepted(t, "author@g14", p)
|
||||||
|
if h := w.k.Health(nil); !h.Healthy {
|
||||||
|
t.Fatalf("unhealthy after a decision with its home was accepted: %s", h.Why)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// A new decision is cited by a record proposed after it: no record can link a number only reserved. Once the
|
||||||
|
// citing record is accepted, the new one proposed again is accepted. A correction of the citing record that drops
|
||||||
|
// the link still leaves the decision cited by nothing, and that is loud: told to the operator once, as a warning
|
||||||
|
// naming the rule and the record, while the module stays healthy and its rollouts are not held (the operator's
|
||||||
|
// decision of 2026-10-12 on novox/hq issue 504; to-be 55 §10). A record that cites it clears the notice.
|
||||||
|
func TestARecordAcceptedCitingANewDecisionLetsItBeAcceptedAndDroppingTheLinkIsLoud(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s Store) {
|
||||||
|
w := imported(t, s)
|
||||||
|
n := w.reserved(t, "author@g14")
|
||||||
|
a := valid(n)
|
||||||
|
a.Homes = nil
|
||||||
|
if _, err := w.k.Propose("author@g14", a); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
w.reviewAll(t)
|
||||||
|
m := w.reserved(t, "author@g14")
|
||||||
|
b := valid(m)
|
||||||
|
b.Links = append(b.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
||||||
|
w.accepted(t, "author@g14", b)
|
||||||
|
w.accepted(t, "author@g14", a) // its revision: an accepted record links it now
|
||||||
|
if h := w.k.Health(nil); !h.Healthy {
|
||||||
|
t.Fatalf("unhealthy with both accepted, one citing the other: %s", h.Why)
|
||||||
|
}
|
||||||
|
w.k.TellFindings()
|
||||||
|
if len(w.told) != 0 {
|
||||||
|
t.Fatalf("told with every rule holding: %v", w.told)
|
||||||
|
}
|
||||||
|
// A correction of the citing record drops its link: the rule, run after the acceptance, says so.
|
||||||
|
b.Links = b.Links[:len(b.Links)-1]
|
||||||
|
b.Insight = "it cited " + itoaKey(n)
|
||||||
|
w.accepted(t, "author@g14", b)
|
||||||
|
h := w.k.Health(nil)
|
||||||
|
if !h.Healthy || !strings.Contains(h.Why, "the rule cited") || !strings.Contains(h.Why, itoaKey(n)) {
|
||||||
|
t.Fatalf("a decision left cited by nothing: the health should stay healthy and name it, not %+v", h)
|
||||||
|
}
|
||||||
|
w.k.TellFindings()
|
||||||
|
w.k.TellFindings()
|
||||||
|
if len(w.told) != 1 || !strings.Contains(w.told[0], "rule cited") || !strings.Contains(w.told[0], itoaKey(n)) ||
|
||||||
|
!strings.HasPrefix(w.toldKeys[0], findingPrefix) {
|
||||||
|
t.Fatalf("the operator is not told once, naming the rule and the record: %v %v", w.told, w.toldKeys)
|
||||||
|
}
|
||||||
|
// A restart tells nothing again: what was told is kept in the store.
|
||||||
|
told := w.toldKeys[0]
|
||||||
|
restarted := newWorld(t, s, fixture())
|
||||||
|
restarted.clock = w.clock
|
||||||
|
restarted.k.TellFindings()
|
||||||
|
if len(restarted.told) != 0 {
|
||||||
|
t.Fatalf("told again after a restart: %v", restarted.told)
|
||||||
|
}
|
||||||
|
w.told = nil
|
||||||
if mustGet(t, s, Key{KindDecision, n}).Current.Status != StatusAccepted {
|
if mustGet(t, s, Key{KindDecision, n}).Current.Status != StatusAccepted {
|
||||||
t.Fatal("the acceptance was undone")
|
t.Fatal("the acceptance was undone")
|
||||||
}
|
}
|
||||||
// A record that cites it heals it.
|
// A record that cites it heals it.
|
||||||
m := w.reserved(t, "author@g14")
|
k := w.reserved(t, "author@g14")
|
||||||
p := valid(m)
|
c := valid(k)
|
||||||
p.Links = append(p.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
c.Links = append(c.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
||||||
w.accepted(t, "author@g14", p)
|
w.accepted(t, "author@g14", c)
|
||||||
h = w.k.Health(nil)
|
if h := w.k.Health(nil); strings.Contains(h.Why, itoaKey(n)+" is accepted and nothing cites it") {
|
||||||
if strings.Contains(h.Why, itoaKey(n)+" is accepted and nothing cites it") {
|
|
||||||
t.Fatalf("still said after a record cited it: %s", h.Why)
|
t.Fatalf("still said after a record cited it: %s", h.Why)
|
||||||
}
|
}
|
||||||
|
w.k.TellFindings()
|
||||||
|
if len(w.cleared) != 1 || w.cleared[0][0] != told {
|
||||||
|
t.Fatalf("the notice is not cleared once the rule holds again: %v", w.cleared)
|
||||||
|
}
|
||||||
|
w.k.TellFindings()
|
||||||
|
if len(w.cleared) != 1 || len(w.told) != 0 {
|
||||||
|
t.Fatalf("after the clear: cleared %v, told %v", w.cleared, w.told)
|
||||||
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -83,3 +164,29 @@ func TestEachWholeSetRuleFails(t *testing.T) {
|
|||||||
t.Errorf("a consistent set fails: %+v", f)
|
t.Errorf("a consistent set fails: %+v", f)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A finding's notice the operator's channel refused is not kept as told: it is sent again at the next step.
|
||||||
|
func TestAFindingsNoticeThatFailsIsSentAgain(t *testing.T) {
|
||||||
|
eachStore(t, func(t *testing.T, s Store) {
|
||||||
|
w := newWorld(t, s, fixture())
|
||||||
|
f := Finding{Rule: "cited", Records: []string{"ADR 0300"}, Says: "ADR 0300 is accepted and nothing cites it"}
|
||||||
|
if err := s.SetFindings(context.Background(), []Finding{f}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
tries := 0
|
||||||
|
w.k.Notify = func(key, summary, headline, explanation, resolved string) error {
|
||||||
|
tries++
|
||||||
|
if tries == 1 {
|
||||||
|
return errors.New("nothing answers operator-channel.notify")
|
||||||
|
}
|
||||||
|
w.told = append(w.told, summary)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
w.k.TellFindings()
|
||||||
|
w.k.TellFindings()
|
||||||
|
w.k.TellFindings()
|
||||||
|
if tries != 2 || len(w.told) != 1 {
|
||||||
|
t.Fatalf("a refused notice: %d tries, told %v", tries, w.told)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
@@ -16,15 +16,20 @@ channel's client and no channel's secret.
|
|||||||
its sender) and `desk-channel` (kind `desktop`, the machines' notifiers, proves nothing). Which kinds exist
|
its sender) and `desk-channel` (kind `desktop`, the machines' notifiers, proves nothing). Which kinds exist
|
||||||
and what each promises is the controller's word, in this module's membership; whether each can send now
|
and what each promises is the controller's word, in this module's membership; whether each can send now
|
||||||
is the channel's own, said on `intake.standing`.
|
is the channel's own, said on `intake.standing`.
|
||||||
- **Keeps its state on the bus** (ADR 0201): `open` and `sent` (the messages), `asks` (every ask, closed ones
|
- **Keeps its state on the bus** (ADR 0201): `open` and `sent` (the messages, and the restart passes made),
|
||||||
for 30 days), `identities` (the operator's linked accounts, one per kind, and a link asked for).
|
`asks` (every ask, closed ones for 30 days), `identities` (the operator's linked accounts, one per kind, and a
|
||||||
|
link asked for), `walks` (the delivery walks open now, each with what it holds).
|
||||||
- **Lost answers are said again from here, never from a channel's memory** (issue 384,
|
- **Lost answers are said again from here, never from a channel's memory** (issue 384,
|
||||||
`cmd/messenger/restart.go`): a channel whose messages' answers were lost — the desk's notifier started
|
`cmd/messenger/restart.go`): a channel whose messages' answers were lost — the desk's notifier started
|
||||||
again — says when in its standing (`answers-lost-at`). The router says again on that kind, directly and not
|
again — says when in its standing (`answers-lost-at`). The router says again on that kind, directly and not
|
||||||
through the outbox, each message that is still open, asked there and last said before that time: one per
|
through the outbox, each message that is still open, asked there, never answered and last said before that
|
||||||
problem, under its handle, with the words and answers of now, so it replaces the message in place. What is
|
time: one per problem, under its handle, with the words and answers of now, so it replaces the message in
|
||||||
silenced is edited to say so, with no answers and no urgency; what ended never comes back. A kind that cannot
|
place. **A restart says nothing it has said** (novox/hq ADR 0308 §6, issue 531): a silenced message, one
|
||||||
send waits; it never goes to another channel. Telegram's answers are never lost and it says no time.
|
withheld for its content, one answered and one a walk holds are left as they are, and what ended never comes
|
||||||
|
back. The standings of one restart (a notifier on each machine, seconds apart) are one pass: a loss is acted
|
||||||
|
on once it stood 10 s, and the pass made is kept in the state (`sent`, key `passes`), so a router that starts
|
||||||
|
again does not make it again. A kind that cannot send waits; it never goes to another channel. Telegram's
|
||||||
|
answers are never lost and it says no time.
|
||||||
|
|
||||||
## Asks, answers and warrants (novox/hq ADR 0259)
|
## Asks, answers and warrants (novox/hq ADR 0259)
|
||||||
|
|
||||||
@@ -143,7 +148,10 @@ new message is shown, so it rings once and no second message stands beside the f
|
|||||||
notification by its id; Telegram deletes the earlier message). A message said again goes alone, never as a
|
notification by its id; Telegram deletes the earlier message). A message said again goes alone, never as a
|
||||||
line of a digest beside the message it follows; one first said inside a digest gets a message of its own the
|
line of a digest beside the message it follows; one first said inside a digest gets a message of its own the
|
||||||
next time, and later words replace or edit that one. An ask's buttons live on that one message, so an ask that
|
next time, and later words replace or edit that one. An ask's buttons live on that one message, so an ask that
|
||||||
ends takes them away everywhere; its expiry is said with its day when it is not today.
|
ends takes them away everywhere; its expiry is said with its day when it is not today. An ask replaced by a
|
||||||
|
newer one about the same thing ends its own message too, which then says "Replaced by a newer question"; the
|
||||||
|
newer ask shows a message of its own (issue 548). A message left open by an ask that already ended is ended on
|
||||||
|
the next minute's sweep.
|
||||||
|
|
||||||
- **Routing:** urgent to every kind that can send; a warning to the desk where it can send, otherwise to
|
- **Routing:** urgent to every kind that can send; a warning to the desk where it can send, otherwise to
|
||||||
the channel that reaches the operator away; an ask to every kind that carries one of its answers.
|
the channel that reaches the operator away; an ask to every kind that carries one of its answers.
|
||||||
@@ -165,10 +173,37 @@ ends takes them away everywhere; its expiry is said with its day when it is not
|
|||||||
history. An event that cannot be read
|
history. An event that cannot be read
|
||||||
is refused by name, counted, and told to the operator once an hour.
|
is refused by name, counted, and told to the operator once an hour.
|
||||||
|
|
||||||
|
## While a delivery walks (novox/hq ADR 0308)
|
||||||
|
|
||||||
|
The controller's `plan-moved` says each walk's start and end: `walk`, `phase` (`started`, `ended`), `touches`
|
||||||
|
(`[{machine, modules}]`: what it moves and what a moved tool runner or node-engine carries), `outcome`
|
||||||
|
(`delivered`, `failed`, `stopped`), `bound_seconds` and `budget_class` (`cmd/messenger/walks.go`).
|
||||||
|
|
||||||
|
- **Held:** while a walk is open and within its bound, a message whose subject is a machine it touches, or a
|
||||||
|
module it touches (on a machine it touches, when the subject names one), is held: kept on the walk with when
|
||||||
|
and what, and the history says `held`. A message several open walks touch is told when the last ends or passes
|
||||||
|
its bound. At a walk's end, a later open walk that touches a message takes it over only as a first hold would:
|
||||||
|
still open, not silenced, not said since, and not one that is never held.
|
||||||
|
- **Never held:** an ask waiting for the operator (and a condition an open ask is about), a secret given or
|
||||||
|
replaced, a subject no open walk touches, and a walk's own failure (a plan, walk, delivery or build).
|
||||||
|
- **The end is one message:** the outcome, the modules and machines reached, how long; each held message still
|
||||||
|
open, one line each; the ones that cleared counted. Delivered with nothing held and nothing open: one line —
|
||||||
|
and nothing on a kind listed in the setting `only-what-needs-me`. What it carries is marked told on a kind
|
||||||
|
only once the message was sent there. Until then each is kept owed in the state (`owed`): a message lost with a
|
||||||
|
restart before it was sent leaves them untold and owed, and the restarted router says each on its own. A
|
||||||
|
record that clears owes nothing more.
|
||||||
|
- **The bound:** at its start plus `bound_seconds` (900 when not given), what is held and still open is said,
|
||||||
|
with "The walk … is still running, since HH:MM", and the cleared counted; the walk holds nothing more. A walk
|
||||||
|
whose end never comes is forgotten a day after it started. A start heard after its end opens nothing: the
|
||||||
|
controller's moments are compared on its own clock, never with one capped at the router's now.
|
||||||
|
- **Kept in the state** (`walks`): a router that starts again mid-walk holds and tells as before.
|
||||||
|
|
||||||
## What the operator gives
|
## What the operator gives
|
||||||
|
|
||||||
1. **The time zone**, as a setting: `{"time-zone": "<an IANA zone name>"}`. Every time a message says is
|
1. **The time zone**, as a setting: `{"time-zone": "<an IANA zone name>"}`. Every time a message says is
|
||||||
said in it; not given, the zone of the machine this runs on.
|
said in it; not given, the zone of the machine this runs on.
|
||||||
|
**The kinds that say only what needs the operator**, as a setting: `{"only-what-needs-me": ["<kind>"]}`. A
|
||||||
|
walk that delivered with nothing held and nothing open is not told on them (ADR 0308 §4); empty by default.
|
||||||
2. **The channels**: assign `telegram` and `desk-channel` (each says in its README what it needs). Linking
|
2. **The channels**: assign `telegram` and `desk-channel` (each says in its README what it needs). Linking
|
||||||
an account as the operator is done on the channel, never by a setting.
|
an account as the operator is done on the channel, never by a setting.
|
||||||
|
|
||||||
|
|||||||
@@ -691,30 +691,58 @@ func (h *Holder) showClosed(rec AskRecord) {
|
|||||||
}
|
}
|
||||||
told := h.toldLine(rec) // read before the lock: it reads the ask's record
|
told := h.toldLine(rec) // read before the lock: it reads the ask's record
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
|
ownMessage := r.Origin == "ask"
|
||||||
// The first line is how it ended, which an ask's own message shows as its title (sayOn); the rest follows.
|
// The first line is how it ended, which an ask's own message shows as its title (sayOn); the rest follows.
|
||||||
switch {
|
switch {
|
||||||
case rec.Warrant != nil && rec.Warrant.Outcome == asks.OutcomeChosen:
|
case rec.Warrant != nil && rec.Warrant.Outcome == asks.OutcomeChosen:
|
||||||
r.Answered = chosenLine(rec.Warrant, rec.Closed.In(h.zone())) + "\n" + told
|
r.Answered = chosenLine(rec.Warrant, rec.Closed.In(h.zone())) + "\n" + told
|
||||||
case rec.State == string(asks.OutcomeExpired):
|
case rec.State == string(asks.OutcomeExpired):
|
||||||
r.Answered = "No longer answerable: nobody answered in time.\n" + strings.TrimSuffix(capital(rec.Ask.OnExpiry), ".") + "."
|
r.Answered = "No longer answerable: nobody answered in time.\n" + strings.TrimSuffix(capital(rec.Ask.OnExpiry), ".") + "."
|
||||||
case rec.State == string(asks.OutcomeReplaced):
|
case rec.State == string(asks.OutcomeReplaced) && !ownMessage:
|
||||||
|
// On a condition's message, which the newer ask takes over and shows in place.
|
||||||
r.Answered = ""
|
r.Answered = ""
|
||||||
|
case rec.State == string(asks.OutcomeReplaced):
|
||||||
|
// Its own message: the newer ask looks for its message under its own key and opens one of its own, so
|
||||||
|
// this one ends here, on every channel that showed it (issue 548).
|
||||||
|
r.Answered = "Replaced by a newer question.\n" + rec.Asker + " asked again about the same thing; that question is the one to answer."
|
||||||
default:
|
default:
|
||||||
r.Answered = "No longer needed."
|
r.Answered = "No longer needed."
|
||||||
}
|
}
|
||||||
ownMessage := r.Origin == "ask"
|
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
if r.told() {
|
if r.told() {
|
||||||
h.edit(r, "answered", false)
|
h.edit(r, "answered", false)
|
||||||
}
|
}
|
||||||
if ownMessage && rec.State != string(asks.OutcomeReplaced) {
|
if ownMessage {
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
r.Cleared = h.Now()
|
r.Cleared = h.Now()
|
||||||
|
r.Owed = "" // an ended question is owed to nobody (the review of PR 238)
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
}
|
}
|
||||||
h.persist(r)
|
h.persist(r)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// endLeftovers ends the own message of an ask that already ended and whose message stayed open: one left by a
|
||||||
|
// router before issue 548 was fixed, whose replaced asks kept their messages for good. Its copies say how the
|
||||||
|
// ask ended, as its end would have.
|
||||||
|
func (h *Holder) endLeftovers() {
|
||||||
|
h.mu.Lock()
|
||||||
|
var ended []AskRecord
|
||||||
|
for _, r := range h.open {
|
||||||
|
if r.Origin != "ask" || r.Ask == "" || !r.Cleared.IsZero() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if e := h.asks[r.Ask]; e != nil && e.rec.State != StateOpen && e.rec.Record == r.Key {
|
||||||
|
ended = append(ended, e.rec)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
sort.Slice(ended, func(i, j int) bool { return ended[i].Record < ended[j].Record })
|
||||||
|
for _, rec := range ended {
|
||||||
|
h.Logf("[messenger] the message of %s stayed open after its ask ended (%s): ended now", rec.Record, rec.State)
|
||||||
|
h.showClosed(rec)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// chosenLine is how a chosen ask ended, in the operator's words: "You chose Approve at 13:30 on Telegram."
|
// chosenLine is how a chosen ask ended, in the operator's words: "You chose Approve at 13:30 on Telegram."
|
||||||
func chosenLine(w *asks.Warrant, at time.Time) string {
|
func chosenLine(w *asks.Warrant, at time.Time) string {
|
||||||
who, where := "Somebody", ""
|
who, where := "Somebody", ""
|
||||||
@@ -775,6 +803,7 @@ func (h *Holder) sweepAsks(now time.Time) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
|
h.endLeftovers()
|
||||||
sort.Strings(expired)
|
sort.Strings(expired)
|
||||||
for _, k := range expired {
|
for _, k := range expired {
|
||||||
e := h.askOf(k)
|
e := h.askOf(k)
|
||||||
|
|||||||
@@ -154,6 +154,11 @@ type Record struct {
|
|||||||
Subject string `json:"subject"`
|
Subject string `json:"subject"`
|
||||||
Severity string `json:"severity"`
|
Severity string `json:"severity"`
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
|
// Scope, SubjectID and Machine are the condition's subject as the controller gave it, which a walk's touches
|
||||||
|
// are matched against (ADR 0308 §2); empty for a record kept before, whose key names them.
|
||||||
|
Scope string `json:"scope,omitempty"`
|
||||||
|
SubjectID string `json:"subject_id,omitempty"`
|
||||||
|
Machine string `json:"machine,omitempty"`
|
||||||
// Headline, Explanation and Resolved are the plain words the operator reads (novox/hq ADR 0253):
|
// Headline, Explanation and Resolved are the plain words the operator reads (novox/hq ADR 0253):
|
||||||
// the controller's, or a module's through notify; empty, the message is said from the summary.
|
// the controller's, or a module's through notify; empty, the message is said from the summary.
|
||||||
Headline string `json:"headline,omitempty"`
|
Headline string `json:"headline,omitempty"`
|
||||||
@@ -195,14 +200,17 @@ type Record struct {
|
|||||||
// ClearHeld is what was waiting to be said of it when it cleared (a reminder, an escalation): said after
|
// ClearHeld is what was waiting to be said of it when it cleared (a reminder, an escalation): said after
|
||||||
// all if it comes back before its clearing was said. Blips counts those comebacks, never said, so a
|
// all if it comes back before its clearing was said. Blips counts those comebacks, never said, so a
|
||||||
// condition that flaps is known as one and its message's "again, Nth time" counts only what was said.
|
// condition that flaps is known as one and its message's "again, Nth time" counts only what was said.
|
||||||
ClearHeld string `json:"clear_held,omitempty"`
|
ClearHeld string `json:"clear_held,omitempty"`
|
||||||
Blips int `json:"blips,omitempty"`
|
Blips int `json:"blips,omitempty"`
|
||||||
Reminded bool `json:"reminded,omitempty"`
|
Reminded bool `json:"reminded,omitempty"`
|
||||||
Pending string `json:"pending,omitempty"` // what could not be said yet: raised, reminder, …
|
Pending string `json:"pending,omitempty"` // what could not be said yet: raised, reminder, …
|
||||||
Folded bool `json:"folded,omitempty"` // held by the cap
|
// Owed is what a walk held of it and has not told yet (ADR 0308 §4, §5): kept in the state until a message
|
||||||
Cleared time.Time `json:"cleared,omitempty"`
|
// carrying it was sent, so a router that restarts before says it on its own (issue 538), never history.
|
||||||
Count int `json:"count"`
|
Owed string `json:"owed,omitempty"`
|
||||||
Refused string `json:"refused,omitempty"`
|
Folded bool `json:"folded,omitempty"` // held by the cap
|
||||||
|
Cleared time.Time `json:"cleared,omitempty"`
|
||||||
|
Count int `json:"count"`
|
||||||
|
Refused string `json:"refused,omitempty"`
|
||||||
// History: known from the state or from an old event, and never said — not news.
|
// History: known from the state or from an old event, and never said — not news.
|
||||||
History bool `json:"history,omitempty"`
|
History bool `json:"history,omitempty"`
|
||||||
// Seen is when this holder last took word of it; the open set read after it is authoritative.
|
// Seen is when this holder last took word of it; the open set read after it is authoritative.
|
||||||
@@ -211,7 +219,8 @@ type Record struct {
|
|||||||
|
|
||||||
// recordOf is a condition as this holder keeps it.
|
// recordOf is a condition as this holder keeps it.
|
||||||
func recordOf(c Condition) Record {
|
func recordOf(c Condition) Record {
|
||||||
return Record{Key: c.Key, Kind: c.Kind, Subject: c.SubjectWords(), Severity: c.Severity, Summary: c.Summary,
|
return Record{Key: c.Key, Kind: c.Kind, Subject: c.SubjectWords(), Scope: c.Scope, SubjectID: c.ID, Machine: c.Machine,
|
||||||
|
Severity: c.Severity, Summary: c.Summary,
|
||||||
Headline: c.Headline, Explanation: c.Explanation, Resolved: c.Resolved, Needs: c.Needs, Actions: c.Actions,
|
Headline: c.Headline, Explanation: c.Explanation, Resolved: c.Resolved, Needs: c.Needs, Actions: c.Actions,
|
||||||
Origin: "condition",
|
Origin: "condition",
|
||||||
More: "conditions show " + c.Key, Raised: c.Raised, SilencedTill: c.SilencedTill, SilencedWhy: c.SilencedWhy}
|
More: "conditions show " + c.Key, Raised: c.Raised, SilencedTill: c.SilencedTill, SilencedWhy: c.SilencedWhy}
|
||||||
@@ -232,6 +241,13 @@ func (r *Record) takeWords(from Record) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// takeSubject keeps the subject as the controller says it now, when it says one.
|
||||||
|
func (r *Record) takeSubject(from Record) {
|
||||||
|
if from.Scope != "" || from.SubjectID != "" || from.Machine != "" {
|
||||||
|
r.Scope, r.SubjectID, r.Machine = from.Scope, from.SubjectID, from.Machine
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (r *Record) silenced(now time.Time) bool {
|
func (r *Record) silenced(now time.Time) bool {
|
||||||
return !r.SilencedTill.IsZero() && now.Before(r.SilencedTill)
|
return !r.SilencedTill.IsZero() && now.Before(r.SilencedTill)
|
||||||
}
|
}
|
||||||
@@ -292,6 +308,9 @@ type Holder struct {
|
|||||||
Zone func() *time.Location
|
Zone func() *time.Location
|
||||||
// Tracking asks the issue tracker which issues track which conditions; nil asks it through Ask (tracking.go).
|
// Tracking asks the issue tracker which issues track which conditions; nil asks it through Ask (tracking.go).
|
||||||
Tracking func() (map[string][]TrackedIssue, error)
|
Tracking func() (map[string][]TrackedIssue, error)
|
||||||
|
// NeedsOnly says whether a kind is set to say only what needs the operator (the setting
|
||||||
|
// `only-what-needs-me`, ADR 0308 §4); nil, none is.
|
||||||
|
NeedsOnly func(kind string) bool
|
||||||
|
|
||||||
work sync.Mutex // one event, call or tick at a time
|
work sync.Mutex // one event, call or tick at a time
|
||||||
mu sync.Mutex // what the status reads
|
mu sync.Mutex // what the status reads
|
||||||
@@ -329,8 +348,14 @@ type Holder struct {
|
|||||||
lastUnlink map[string]Unlinked
|
lastUnlink map[string]Unlinked
|
||||||
// rootWord is the controller's last word on whether an agent can become root, by the machines asked.
|
// rootWord is the controller's last word on whether an agent can become root, by the machines asked.
|
||||||
rootWord map[string]rootWord
|
rootWord map[string]rootWord
|
||||||
// shows is, by kind, the AnswersLostAt this router last acted on in full (restart.go).
|
// passes is, by kind, the restart pass this router last made in full, kept in its state (restart.go); lost
|
||||||
shows map[string]time.Time
|
// the loss of answers it has yet to act on, by kind.
|
||||||
|
passes map[string]Pass
|
||||||
|
lost map[string]time.Time
|
||||||
|
// walks are the walks open now, by id, each with what it holds (walks.go, ADR 0308).
|
||||||
|
walks map[string]*Walk
|
||||||
|
// ended are the walks whose end was heard, by id: what a start heard after its end is matched against.
|
||||||
|
ended map[string]*Walk
|
||||||
// track is the issue tracker's last word on which issues track which conditions (tracking.go).
|
// track is the issue tracker's last word on which issues track which conditions (tracking.go).
|
||||||
track trackWord
|
track trackWord
|
||||||
}
|
}
|
||||||
@@ -348,7 +373,10 @@ func (h *Holder) init() {
|
|||||||
h.chanOK = map[string]time.Time{}
|
h.chanOK = map[string]time.Time{}
|
||||||
h.saidOnce = map[string]time.Time{}
|
h.saidOnce = map[string]time.Time{}
|
||||||
h.heard = map[string]int{}
|
h.heard = map[string]int{}
|
||||||
h.shows = map[string]time.Time{}
|
h.passes = map[string]Pass{}
|
||||||
|
h.lost = map[string]time.Time{}
|
||||||
|
h.walks = map[string]*Walk{}
|
||||||
|
h.ended = map[string]*Walk{}
|
||||||
}
|
}
|
||||||
if h.Now == nil {
|
if h.Now == nil {
|
||||||
h.Now = time.Now
|
h.Now = time.Now
|
||||||
@@ -382,11 +410,20 @@ func (h *Holder) Load() error {
|
|||||||
r.History = true
|
r.History = true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if r.Owed != "" && r.Cleared.IsZero() {
|
||||||
|
// Held from the operator on purpose and not told yet: said on its own, as the next tick tries what is
|
||||||
|
// unsent (issue 538). What the operator could have heard is history; this they could not.
|
||||||
|
r.Pending, r.History = r.Owed, false
|
||||||
|
}
|
||||||
h.open[r.Key] = &r
|
h.open[r.Key] = &r
|
||||||
}
|
}
|
||||||
if sent, err := h.Store.Recent(); err == nil {
|
if sent, err := h.Store.Recent(); err == nil {
|
||||||
h.recent = sent
|
h.recent = sent
|
||||||
}
|
}
|
||||||
|
if err := h.loadWalks(); err != nil {
|
||||||
|
h.storeErr = err.Error()
|
||||||
|
return err
|
||||||
|
}
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
err = h.loadAsks()
|
err = h.loadAsks()
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
@@ -485,6 +522,7 @@ func (h *Holder) raised(rec Record, quiet bool) {
|
|||||||
wasQuiet := needsNothing(old)
|
wasQuiet := needsNothing(old)
|
||||||
wasSilenced := old.silenced(now)
|
wasSilenced := old.silenced(now)
|
||||||
old.Summary, old.Subject, old.Kind = rec.Summary, rec.Subject, rec.Kind
|
old.Summary, old.Subject, old.Kind = rec.Summary, rec.Subject, rec.Kind
|
||||||
|
old.takeSubject(rec)
|
||||||
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
||||||
old.takeWords(rec)
|
old.takeWords(rec)
|
||||||
if rec.Severity != "" {
|
if rec.Severity != "" {
|
||||||
@@ -584,6 +622,7 @@ func (h *Holder) changed(rec Record, quiet bool) {
|
|||||||
wasQuiet := needsNothing(old)
|
wasQuiet := needsNothing(old)
|
||||||
wasSilenced := old.silenced(h.Now())
|
wasSilenced := old.silenced(h.Now())
|
||||||
old.Summary, old.Subject = rec.Summary, rec.Subject
|
old.Summary, old.Subject = rec.Summary, rec.Subject
|
||||||
|
old.takeSubject(rec)
|
||||||
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
||||||
old.takeWords(rec)
|
old.takeWords(rec)
|
||||||
if rec.Severity != "" {
|
if rec.Severity != "" {
|
||||||
@@ -648,6 +687,7 @@ func (h *Holder) cleared(key string, quiet bool) {
|
|||||||
}
|
}
|
||||||
old.Cleared = now
|
old.Cleared = now
|
||||||
old.Pending = ""
|
old.Pending = ""
|
||||||
|
old.Owed = "" // what a walk held of it is owed no more: it ended (the review of PR 238)
|
||||||
old.Folded = false
|
old.Folded = false
|
||||||
old.Seen = now
|
old.Seen = now
|
||||||
told := old.told()
|
told := old.told()
|
||||||
@@ -710,6 +750,8 @@ func (h *Holder) Tick() {
|
|||||||
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
||||||
h.sweepAsks(now)
|
h.sweepAsks(now)
|
||||||
h.breakGlass()
|
h.breakGlass()
|
||||||
|
h.boundWalks(now)
|
||||||
|
h.duePasses(now)
|
||||||
for _, r := range recs {
|
for _, r := range recs {
|
||||||
switch {
|
switch {
|
||||||
case !r.Cleared.IsZero():
|
case !r.Cleared.IsZero():
|
||||||
@@ -775,6 +817,7 @@ func (h *Holder) Flush() {
|
|||||||
h.init()
|
h.init()
|
||||||
now := h.Now()
|
now := h.Now()
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
|
h.duePasses(now)
|
||||||
h.flush(now, false)
|
h.flush(now, false)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -66,6 +66,9 @@ func (f *fakeChannel) Edit(id string, m Message) error {
|
|||||||
type memStore struct {
|
type memStore struct {
|
||||||
recs map[string]Record
|
recs map[string]Record
|
||||||
recent []Sent
|
recent []Sent
|
||||||
|
// walks and passes are what ADR 0308 keeps: the open walks and the restart passes.
|
||||||
|
walks map[string]Walk
|
||||||
|
passes map[string]Pass
|
||||||
// puts counts the writes, so a write of bytes identical to what is stored is still seen (issue 422's review).
|
// puts counts the writes, so a write of bytes identical to what is stored is still seen (issue 422's review).
|
||||||
puts int
|
puts int
|
||||||
}
|
}
|
||||||
@@ -87,7 +90,24 @@ func (m *memStore) All() ([]Record, error) {
|
|||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
func (m *memStore) PutRecent(s []Sent) error { m.recent = s; return nil }
|
func (m *memStore) PutRecent(s []Sent) error { m.recent = s; return nil }
|
||||||
func (m *memStore) Recent() ([]Sent, error) { return m.recent, nil }
|
func (m *memStore) PutWalk(w Walk) error {
|
||||||
|
if m.walks == nil {
|
||||||
|
m.walks = map[string]Walk{}
|
||||||
|
}
|
||||||
|
m.walks[w.ID] = w
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (m *memStore) DeleteWalk(id string) error { delete(m.walks, id); return nil }
|
||||||
|
func (m *memStore) Walks() ([]Walk, error) {
|
||||||
|
var out []Walk
|
||||||
|
for _, w := range m.walks {
|
||||||
|
out = append(out, w)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
func (m *memStore) PutPasses(p map[string]Pass) error { m.passes = p; return nil }
|
||||||
|
func (m *memStore) Passes() (map[string]Pass, error) { return m.passes, nil }
|
||||||
|
func (m *memStore) Recent() ([]Sent, error) { return m.recent, nil }
|
||||||
|
|
||||||
type clock struct{ t time.Time }
|
type clock struct{ t time.Time }
|
||||||
|
|
||||||
|
|||||||
@@ -29,6 +29,9 @@ type Settings struct {
|
|||||||
// TimeZone is the operator's, an IANA name such as Europe/Brussels: every time a message says is said
|
// TimeZone is the operator's, an IANA name such as Europe/Brussels: every time a message says is said
|
||||||
// in it. Empty is this machine's own zone.
|
// in it. Empty is this machine's own zone.
|
||||||
TimeZone string `json:"time-zone"`
|
TimeZone string `json:"time-zone"`
|
||||||
|
// OnlyWhatNeedsMe are the channel kinds set to say only what needs the operator: a walk that delivered
|
||||||
|
// with nothing held and nothing open is not told there at all (novox/hq ADR 0308 §4).
|
||||||
|
OnlyWhatNeedsMe []string `json:"only-what-needs-me"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// zoneOf is the location a time-zone setting names, or this machine's own when it names none or one
|
// zoneOf is the location a time-zone setting names, or this machine's own when it names none or one
|
||||||
@@ -60,6 +63,19 @@ func readSettings(path string) (Settings, error) {
|
|||||||
if tz, ok := loose["time-zone"].(string); ok {
|
if tz, ok := loose["time-zone"].(string); ok {
|
||||||
s.TimeZone = strings.TrimSpace(tz)
|
s.TimeZone = strings.TrimSpace(tz)
|
||||||
}
|
}
|
||||||
|
switch v := loose["only-what-needs-me"].(type) {
|
||||||
|
case nil:
|
||||||
|
case []any:
|
||||||
|
for _, k := range v {
|
||||||
|
kind, ok := k.(string)
|
||||||
|
if !ok || strings.TrimSpace(kind) == "" {
|
||||||
|
return s, fmt.Errorf("only-what-needs-me lists something that is not a channel kind: %v", k)
|
||||||
|
}
|
||||||
|
s.OnlyWhatNeedsMe = append(s.OnlyWhatNeedsMe, strings.TrimSpace(kind))
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return s, fmt.Errorf("only-what-needs-me is not a list of channel kinds")
|
||||||
|
}
|
||||||
return s, nil
|
return s, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -109,6 +125,50 @@ func (stateStore) All() ([]Record, error) {
|
|||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The walks open now, one key each in the state `walks`, and the restart passes under one key of `sent`
|
||||||
|
// (novox/hq ADR 0308): a router that starts again holds, tells and passes as if it had not.
|
||||||
|
func (stateStore) PutWalk(w Walk) error {
|
||||||
|
_, err := stdio.State("walks").Put(kvKey(w.ID), w)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (stateStore) DeleteWalk(id string) error { return stdio.State("walks").Delete(kvKey(id)) }
|
||||||
|
func (stateStore) Walks() ([]Walk, error) {
|
||||||
|
keys, err := stdio.State("walks").Keys()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var out []Walk
|
||||||
|
for _, k := range keys {
|
||||||
|
e, err := stdio.State("walks").Get(k)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if e == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var w Walk
|
||||||
|
if err := json.Unmarshal(e.Value, &w); err != nil {
|
||||||
|
logf("[messenger] the walk kept as %s cannot be read (%v); left as it is", k, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, w)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
func (stateStore) PutPasses(p map[string]Pass) error {
|
||||||
|
_, err := stdio.State("sent").Put("passes", p)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (stateStore) Passes() (map[string]Pass, error) {
|
||||||
|
e, err := stdio.State("sent").Get("passes")
|
||||||
|
if err != nil || e == nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var p map[string]Pass
|
||||||
|
return p, json.Unmarshal(e.Value, &p)
|
||||||
|
}
|
||||||
|
|
||||||
func (stateStore) PutRecent(s []Sent) error {
|
func (stateStore) PutRecent(s []Sent) error {
|
||||||
_, err := stdio.State("sent").Put("recent", s)
|
_, err := stdio.State("sent").Put("recent", s)
|
||||||
return err
|
return err
|
||||||
@@ -271,6 +331,14 @@ func main() {
|
|||||||
return loc
|
return loc
|
||||||
},
|
},
|
||||||
Emit: func(event string, body any) error { return stdio.Emit(event, body) },
|
Emit: func(event string, body any) error { return stdio.Emit(event, body) },
|
||||||
|
NeedsOnly: func(kind string) bool {
|
||||||
|
for _, k := range settings().OnlyWhatNeedsMe {
|
||||||
|
if k == kind {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
},
|
||||||
}
|
}
|
||||||
bench.To = func(kind string) string {
|
bench.To = func(kind string) string {
|
||||||
if ids := h.identitiesOf(kind); len(ids) > 0 {
|
if ids := h.identitiesOf(kind); len(ids) > 0 {
|
||||||
@@ -326,6 +394,18 @@ func run(h *Holder, l *listening) {
|
|||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
handle := func(e stdio.Envelope) error {
|
handle := func(e stdio.Envelope) error {
|
||||||
|
if e.Key == ControllerSeat+"."+EventPlanMoved || e.Key == ControllerSeat+"."+EventWalk {
|
||||||
|
// A walk's start or end (novox/hq ADR 0308 §1); any other move of a plan is not read here.
|
||||||
|
ev, isWalk, err := DecodeWalk(e.Body)
|
||||||
|
if err != nil {
|
||||||
|
h.Unreadable(e.Key, err)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if isWalk {
|
||||||
|
h.Walked(ev)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
event, ok := eventOf(e.Key)
|
event, ok := eventOf(e.Key)
|
||||||
if !ok {
|
if !ok {
|
||||||
return nil
|
return nil
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ func TestItDeclaresTheSeatsOfAnAskAndHoldsTheOperatorChannel(t *testing.T) {
|
|||||||
}
|
}
|
||||||
if !reflect.DeepEqual(m.Consumes, []string{
|
if !reflect.DeepEqual(m.Consumes, []string{
|
||||||
ControllerSeat + "." + EventRaised, ControllerSeat + "." + EventChanged, ControllerSeat + "." + EventCleared,
|
ControllerSeat + "." + EventRaised, ControllerSeat + "." + EventChanged, ControllerSeat + "." + EventCleared,
|
||||||
"intake.choice", "intake.link", "intake.failed", "intake.standing"}) {
|
ControllerSeat + "." + EventPlanMoved, "intake.choice", "intake.link", "intake.failed", "intake.standing"}) {
|
||||||
t.Fatalf("consumes %v", m.Consumes)
|
t.Fatalf("consumes %v", m.Consumes)
|
||||||
}
|
}
|
||||||
// It performs nothing: the verbs it calls read the open conditions, whether root is free, and which issues
|
// It performs nothing: the verbs it calls read the open conditions, whether root is free, and which issues
|
||||||
@@ -100,7 +100,7 @@ func TestItDeclaresTheSeatsOfAnAskAndHoldsTheOperatorChannel(t *testing.T) {
|
|||||||
}
|
}
|
||||||
state = append(state, name)
|
state = append(state, name)
|
||||||
}
|
}
|
||||||
if !reflect.DeepEqual(state, []string{"open", "sent", "asks", "identities"}) {
|
if !reflect.DeepEqual(state, []string{"open", "sent", "asks", "identities", "walks"}) {
|
||||||
t.Fatalf("state %v", state)
|
t.Fatalf("state %v", state)
|
||||||
}
|
}
|
||||||
// It holds no channel's secret: only its own bus account, read by its own account alone (ADR 0259 §8).
|
// It holds no channel's secret: only its own bus account, read by its own account alone (ADR 0259 §8).
|
||||||
|
|||||||
@@ -18,6 +18,9 @@ type item struct {
|
|||||||
Severity string
|
Severity string
|
||||||
At time.Time
|
At time.Time
|
||||||
Msg *Message
|
Msg *Message
|
||||||
|
// Carries are the records a walk's end or bound message tells (ADR 0308 §4): each is marked told on a
|
||||||
|
// channel only once that message was sent there (issue 538 A), never when it is held to be sent.
|
||||||
|
Carries []string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (i item) urgent() bool { return i.Severity == Urgent && i.What != "cleared" }
|
func (i item) urgent() bool { return i.Severity == Urgent && i.What != "cleared" }
|
||||||
@@ -32,6 +35,10 @@ func (h *Holder) deliver(r *Record, what string) {
|
|||||||
h.record(Sent{At: h.Now(), Channel: "-", Key: r.Key, What: what, Outcome: "quiet: nothing for the operator to do"})
|
h.record(Sent{At: h.Now(), Channel: "-", Key: r.Key, What: what, Outcome: "quiet: nothing for the operator to do"})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if h.hold(r, what) {
|
||||||
|
// An open walk touches it: told when the walk ends or passes its bound (ADR 0308 §2).
|
||||||
|
return
|
||||||
|
}
|
||||||
var to []Channel
|
var to []Channel
|
||||||
if kinds := h.askKinds(r); len(kinds) > 0 {
|
if kinds := h.askKinds(r); len(kinds) > 0 {
|
||||||
// An ask goes to every channel that carries one of its answers (novox/hq ADR 0259): the desk for an
|
// An ask goes to every channel that carries one of its answers (novox/hq ADR 0259): the desk for an
|
||||||
@@ -118,6 +125,8 @@ func (h *Holder) queuedWhat(key string) string {
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// queued says whether something is held to be said of a record: its own message, or a walk's message that
|
||||||
|
// carries it.
|
||||||
func (h *Holder) queued(key string) bool {
|
func (h *Holder) queued(key string) bool {
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
defer h.mu.Unlock()
|
defer h.mu.Unlock()
|
||||||
@@ -126,6 +135,11 @@ func (h *Holder) queued(key string) bool {
|
|||||||
if it.Msg == nil && it.Key == key {
|
if it.Msg == nil && it.Key == key {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
for _, k := range it.Carries {
|
||||||
|
if k == key {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
@@ -135,6 +149,9 @@ func (h *Holder) queued(key string) bool {
|
|||||||
// edit fails, or that said it inside a digest, is given a new message instead — held like any other.
|
// edit fails, or that said it inside a digest, is given a new message instead — held like any other.
|
||||||
// silentOnly: only where an edit notifies nobody; elsewhere nothing.
|
// silentOnly: only where an edit notifies nobody; elsewhere nothing.
|
||||||
func (h *Holder) edit(r *Record, what string, silentOnly bool) {
|
func (h *Holder) edit(r *Record, what string, silentOnly bool) {
|
||||||
|
if h.hold(r, what) {
|
||||||
|
return // an open walk touches it (ADR 0308 §2); a clearing or a silence is never held
|
||||||
|
}
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
sent := map[string]string{}
|
sent := map[string]string{}
|
||||||
for k, v := range r.Sent {
|
for k, v := range r.Sent {
|
||||||
@@ -225,12 +242,23 @@ func (h *Holder) due(ch Channel, list []item, now time.Time) bool {
|
|||||||
if now.Sub(oldest) < BurstWindow {
|
if now.Sub(oldest) < BurstWindow {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
if ch == h.Desk && !lastWarn.IsZero() && now.Sub(lastWarn) < DesktopWarningEvery {
|
if ch == h.Desk && !lastWarn.IsZero() && now.Sub(lastWarn) < DesktopWarningEvery && !carriesWalk(list) {
|
||||||
|
// A walk's message is what the rhythm would have spared the operator many of: not held to it.
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// carriesWalk says whether held items carry a walk's end or bound message.
|
||||||
|
func carriesWalk(list []item) bool {
|
||||||
|
for _, it := range list {
|
||||||
|
if it.What == WalkEndWhat {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
// line is what an item says, resolved against the record now: nil when it is no longer to be said.
|
// line is what an item says, resolved against the record now: nil when it is no longer to be said.
|
||||||
func (h *Holder) resolve(it item, now time.Time, kind string) (*Record, *Message) {
|
func (h *Holder) resolve(it item, now time.Time, kind string) (*Record, *Message) {
|
||||||
if it.Msg != nil {
|
if it.Msg != nil {
|
||||||
@@ -310,7 +338,9 @@ func (h *Holder) flushChannel(ch Channel, now time.Time, force bool) {
|
|||||||
for _, e := range entries {
|
for _, e := range entries {
|
||||||
// A message said again about a record shown here goes alone too: it replaces that record's message,
|
// A message said again about a record shown here goes alone too: it replaces that record's message,
|
||||||
// so a reminder is never a digest line beside the message it follows (issue 373).
|
// so a reminder is never a digest line beside the message it follows (issue 373).
|
||||||
if len(e.msg.Actions) > 1 || (len(e.msg.Actions) > 0 && e.r != nil && e.r.Ask != "") || h.replacing(ch, e) != "" {
|
// A walk's end or bound message goes alone too: its lines are what it says (ADR 0308 §4).
|
||||||
|
if len(e.msg.Actions) > 1 || (len(e.msg.Actions) > 0 && e.r != nil && e.r.Ask != "") || h.replacing(ch, e) != "" ||
|
||||||
|
e.it.What == WalkEndWhat {
|
||||||
alone = append(alone, e)
|
alone = append(alone, e)
|
||||||
} else {
|
} else {
|
||||||
rest = append(rest, e)
|
rest = append(rest, e)
|
||||||
@@ -400,6 +430,7 @@ func (h *Holder) sendEntries(ch Channel, entries []entry, now time.Time) bool {
|
|||||||
}
|
}
|
||||||
r.Pending, r.Folded, r.History = "", false, false
|
r.Pending, r.Folded, r.History = "", false, false
|
||||||
if e.it.What != "cleared" {
|
if e.it.What != "cleared" {
|
||||||
|
r.Owed = ""
|
||||||
if r.Sent == nil {
|
if r.Sent == nil {
|
||||||
r.Sent = map[string]string{}
|
r.Sent = map[string]string{}
|
||||||
}
|
}
|
||||||
@@ -415,7 +446,39 @@ func (h *Holder) sendEntries(ch Channel, entries []entry, now time.Time) bool {
|
|||||||
r.LastSaid = now
|
r.LastSaid = now
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
type carried struct {
|
||||||
|
r *Record
|
||||||
|
walk string
|
||||||
|
}
|
||||||
|
var told []carried
|
||||||
|
for _, e := range entries {
|
||||||
|
for _, key := range e.it.Carries {
|
||||||
|
r := h.open[key]
|
||||||
|
if r == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if r.Sent == nil {
|
||||||
|
r.Sent = map[string]string{}
|
||||||
|
}
|
||||||
|
if _, has := r.Sent[name]; !has {
|
||||||
|
r.Sent[name] = ""
|
||||||
|
}
|
||||||
|
if r.FirstSent.IsZero() {
|
||||||
|
r.FirstSent = now
|
||||||
|
}
|
||||||
|
r.LastSaid, r.Pending, r.History, r.Owed = now, "", false, ""
|
||||||
|
if !r.Cleared.IsZero() && !r.ClearUnsaid {
|
||||||
|
// It cleared while the message saying it open waited: its clearing is said too.
|
||||||
|
r.ClearUnsaid = true
|
||||||
|
}
|
||||||
|
told = append(told, carried{r, strings.TrimPrefix(e.it.Key, "walk.")})
|
||||||
|
}
|
||||||
|
}
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
|
for _, c := range told {
|
||||||
|
h.record(Sent{At: now, Channel: name, Key: c.r.Key, What: "released", Outcome: "told in the message of walk " + c.walk})
|
||||||
|
h.persist(c.r)
|
||||||
|
}
|
||||||
key, what := "(digest)", fmt.Sprintf("digest of %d", len(entries))
|
key, what := "(digest)", fmt.Sprintf("digest of %d", len(entries))
|
||||||
if len(entries) == 1 {
|
if len(entries) == 1 {
|
||||||
key, what = entries[0].it.Key, entries[0].it.What
|
key, what = entries[0].it.Key, entries[0].it.What
|
||||||
|
|||||||
@@ -0,0 +1,111 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.novox.be/novox/mesh-sdk/go/asks"
|
||||||
|
)
|
||||||
|
|
||||||
|
// openAskRecords are the records still open that show an ask as their own message.
|
||||||
|
func openAskRecords(h *Holder) []string {
|
||||||
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
|
var out []string
|
||||||
|
for k, r := range h.open {
|
||||||
|
if r.Origin == "ask" && r.Cleared.IsZero() {
|
||||||
|
out = append(out, k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Issue 548: an ask replaced by a newer one about the same thing kept its own message open for good, its
|
||||||
|
// buttons on the phone answering nothing. Two asks with the same About from one asker leave exactly one open
|
||||||
|
// record, and the older one's message says it was replaced, its buttons gone, on every channel that showed it.
|
||||||
|
func TestAReplacedAsksMessageEndsWithIt(t *testing.T) {
|
||||||
|
h, tg, dt, c, store, _ := newRouter(t)
|
||||||
|
link(h, store, c, "42", c.now().Add(-2*time.Hour))
|
||||||
|
a := ackAsk(c, "d-1")
|
||||||
|
a.About = "delivery.hq"
|
||||||
|
submit(t, h, "mesh-delivery", a)
|
||||||
|
settle(h, c)
|
||||||
|
if len(tg.sends) != 1 || len(dt.sends) != 1 {
|
||||||
|
t.Fatalf("the first ask was not shown: telegram %d, desktop %d", len(tg.sends), len(dt.sends))
|
||||||
|
}
|
||||||
|
b := ackAsk(c, "d-2")
|
||||||
|
b.About = "delivery.hq"
|
||||||
|
b.Headline = "Delivery of hq held, asked again"
|
||||||
|
submit(t, h, "mesh-delivery", b)
|
||||||
|
settle(h, c)
|
||||||
|
if got := openAskRecords(h); len(got) != 1 || got[0] != "ask.mesh-delivery.d-2" {
|
||||||
|
t.Fatalf("open ask records: %v; want only the newer ask's", got)
|
||||||
|
}
|
||||||
|
for _, ch := range []*fakeChannel{tg, dt} {
|
||||||
|
id := map[string]string{"telegram": "1", "desktop": "g14=1"}[ch.name]
|
||||||
|
m, edited := ch.edits[id]
|
||||||
|
for _, r := range ch.replaced {
|
||||||
|
if r.id == id {
|
||||||
|
m, edited = r.m, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !edited || len(m.Actions) != 0 || !strings.Contains(m.Title, "Replaced by a newer question") {
|
||||||
|
t.Errorf("%s: the replaced ask's message still stands: edited %v, %q %v", ch.name, edited, m.Text(), labelsOf(m))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if store.asks["mesh-delivery.d-1"].State != string(asks.OutcomeReplaced) {
|
||||||
|
t.Errorf("the older ask was not replaced: %s", store.asks["mesh-delivery.d-1"].State)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The leftover of issue 548: a record from before the fix whose ask already ended keeps its message open. The
|
||||||
|
// next tick ends it, as the ask's end would have.
|
||||||
|
func TestAnAsksMessageLeftOpenAfterItsAskEndedIsEndedOnTheNextTick(t *testing.T) {
|
||||||
|
h, tg, _, c, store, _ := newRouter(t)
|
||||||
|
link(h, store, c, "42", c.now().Add(-2*time.Hour))
|
||||||
|
a := ackAsk(c, "d-1")
|
||||||
|
a.About = "delivery.hq"
|
||||||
|
submit(t, h, "mesh-delivery", a)
|
||||||
|
settle(h, c)
|
||||||
|
// Ended as replaced the way the router did before the fix: the ask's record only, its message untouched.
|
||||||
|
key := "mesh-delivery.d-1"
|
||||||
|
rec := store.asks[key]
|
||||||
|
rec.State, rec.Closed, rec.Told = string(asks.OutcomeReplaced), c.now(), true
|
||||||
|
rec.Warrant = &asks.Warrant{Ask: "d-1", Asker: "mesh-delivery", About: a.About, Outcome: asks.OutcomeReplaced,
|
||||||
|
Words: "a newer ask about the same thing replaced it", At: c.now()}
|
||||||
|
store.asks[key] = rec
|
||||||
|
h.keepAsk(key, rec, 99)
|
||||||
|
if got := openAskRecords(h); len(got) != 1 {
|
||||||
|
t.Fatalf("the leftover is not set up: %v", got)
|
||||||
|
}
|
||||||
|
c.pass(time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
h.Tick()
|
||||||
|
h.Flush()
|
||||||
|
if got := openAskRecords(h); len(got) != 0 {
|
||||||
|
t.Fatalf("the message of an ended ask is still open: %v", got)
|
||||||
|
}
|
||||||
|
if m, ok := tg.edits["1"]; !ok || len(m.Actions) != 0 || !strings.Contains(m.Title, "Replaced by a newer question") {
|
||||||
|
t.Errorf("the leftover's copy was not ended: %v %q %v", ok, m.Text(), labelsOf(m))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The review of PR 238: what a walk owed of a record is dropped when the record clears, so a router that
|
||||||
|
// restarts never says a held message of something that already ended.
|
||||||
|
func TestAClearedRecordOwesNothing(t *testing.T) {
|
||||||
|
h, _, _, c, st := newHolder(t)
|
||||||
|
k := cond("machine.shanks.units", Warning, "2 failed services on shanks")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
h.mu.Lock()
|
||||||
|
h.open[k.Key].Owed = "raised"
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.cleared(k.Key, false)
|
||||||
|
if r := h.open[k.Key]; r.Owed != "" {
|
||||||
|
t.Errorf("a cleared record still owes %q", r.Owed)
|
||||||
|
}
|
||||||
|
if r := st.recs[k.Key]; r.Owed != "" {
|
||||||
|
t.Errorf("a cleared record still owes %q in the state", r.Owed)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -18,6 +18,22 @@ package main
|
|||||||
// Whether a record is said again is its own: last said before the answers were lost. So a router that started
|
// Whether a record is said again is its own: last said before the answers were lost. So a router that started
|
||||||
// after the loss still says it, and a standing that repeats the time says nothing more (the record was said
|
// after the loss still says it, and a standing that repeats the time says nothing more (the record was said
|
||||||
// since). A kind that cannot send waits: what is open there is said once it can.
|
// since). A kind that cannot send waits: what is open there is said once it can.
|
||||||
|
//
|
||||||
|
// **A restart says nothing it has said** (novox/hq ADR 0308 §6, issue 531). On 2026-10-12 every delivery of the
|
||||||
|
// tool runner made this pass twice, a second apart, each time editing the four silenced messages on the desk,
|
||||||
|
// which showed them again — one as "A mesh message was withheld". Three causes, each closed here:
|
||||||
|
//
|
||||||
|
// - **Twice:** the desk's standing says the latest start of any machine's notifier, and the notifiers of two
|
||||||
|
// machines started a second apart, so two standings with two times came, and each was acted on at once.
|
||||||
|
// Now a loss is acted on once it stands for LostSettle: the standings of one restart are one pass.
|
||||||
|
// - **Again after the router's own restart:** which loss was acted on was kept in memory, so a router that
|
||||||
|
// started again acted on the standing it had already acted on. Now it is kept in the module's state
|
||||||
|
// (`sent`, key `passes`), read back on start.
|
||||||
|
// - **What it said:** a silenced message was edited to its silenced words, and a message withheld for its
|
||||||
|
// content was said again as withheld. Now the pass says again only what is open, asked there and never
|
||||||
|
// answered, whose words may leave; a silenced, a withheld or an answered message is left as it is. Being
|
||||||
|
// held by a walk is no reason: a message with an open ask is never held (ADR 0308 §3), even one a walk took
|
||||||
|
// before its ask came, so its answers are made to work again like any other's.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -25,7 +41,17 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Stand takes what a kind said of itself, and acts on answers lost there since this router last acted.
|
// LostSettle: a loss of answers is acted on once no later loss came for this long — the standings of one
|
||||||
|
// restart, a notifier on each machine, are one pass.
|
||||||
|
const LostSettle = 10 * time.Second
|
||||||
|
|
||||||
|
// Pass is the restart pass made in full on a kind: the loss it answered and when it was made.
|
||||||
|
type Pass struct {
|
||||||
|
Lost time.Time `json:"lost"`
|
||||||
|
At time.Time `json:"at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stand takes what a kind said of itself, and notes answers lost there since this router last acted.
|
||||||
func (h *Holder) Stand(kind string, s Standing) {
|
func (h *Holder) Stand(kind string, s Standing) {
|
||||||
if h.Bench != nil {
|
if h.Bench != nil {
|
||||||
h.Bench.Stand(kind, s)
|
h.Bench.Stand(kind, s)
|
||||||
@@ -33,28 +59,73 @@ func (h *Holder) Stand(kind string, s Standing) {
|
|||||||
if s.AnswersLostAt.IsZero() {
|
if s.AnswersLostAt.IsZero() {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
h.work.Lock()
|
|
||||||
defer h.work.Unlock()
|
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
h.init()
|
h.init()
|
||||||
done := h.shows[kind].Equal(s.AnswersLostAt)
|
if !s.AnswersLostAt.After(h.passes[kind].Lost) || !s.AnswersLostAt.After(h.lost[kind]) {
|
||||||
|
return // acted on, or noted already
|
||||||
|
}
|
||||||
|
h.lost[kind] = s.AnswersLostAt
|
||||||
|
}
|
||||||
|
|
||||||
|
// duePasses makes the pass of each kind whose loss of answers stood for LostSettle, once per loss.
|
||||||
|
func (h *Holder) duePasses(now time.Time) {
|
||||||
|
h.mu.Lock()
|
||||||
|
due := map[string]time.Time{}
|
||||||
|
for kind, lost := range h.lost {
|
||||||
|
if now.Sub(lost) >= LostSettle {
|
||||||
|
due[kind] = lost
|
||||||
|
}
|
||||||
|
}
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
if done {
|
for kind, lost := range due {
|
||||||
return
|
ch := h.channelOf(kind)
|
||||||
}
|
if ch == nil {
|
||||||
ch := h.channelOf(kind)
|
continue // the controller's word on the kinds is not read yet: acted on later
|
||||||
if ch == nil {
|
}
|
||||||
return // the controller's word on the kinds is not read yet: acted on at a later standing
|
if err := ch.Ready(); err != nil {
|
||||||
}
|
h.mu.Lock()
|
||||||
if err := ch.Ready(); err != nil {
|
said := h.saidOnce["lost."+kind]
|
||||||
h.Logf("[messenger] %s lost its answers at %s and cannot send (%v): what is open there is said again once it can",
|
h.mu.Unlock()
|
||||||
kind, s.AnswersLostAt.UTC().Format(time.RFC3339), err)
|
if said.IsZero() || now.Sub(said) >= 10*time.Minute {
|
||||||
return
|
h.Logf("[messenger] %s lost its answers at %s and cannot send (%v): what is open there is said again once it can",
|
||||||
}
|
kind, lost.UTC().Format(time.RFC3339), err)
|
||||||
if h.reoffer(ch, s.AnswersLostAt) {
|
h.mu.Lock()
|
||||||
|
h.saidOnce["lost."+kind] = now
|
||||||
|
h.mu.Unlock()
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
h.shows[kind] = s.AnswersLostAt
|
tried := h.saidOnce["lost-try."+kind]
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
|
if !tried.IsZero() && now.Sub(tried) < RetryEvery {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !h.reoffer(ch, lost) {
|
||||||
|
h.mu.Lock()
|
||||||
|
h.saidOnce["lost-try."+kind] = now // what failed is tried again in a minute
|
||||||
|
h.mu.Unlock()
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
delete(h.saidOnce, "lost-try."+kind)
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.mu.Lock()
|
||||||
|
h.passes[kind] = Pass{Lost: lost, At: now}
|
||||||
|
if h.lost[kind].Equal(lost) {
|
||||||
|
delete(h.lost, kind)
|
||||||
|
}
|
||||||
|
passes := map[string]Pass{}
|
||||||
|
for k, p := range h.passes {
|
||||||
|
passes[k] = p
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
if k, ok := h.Store.(walkKeeper); ok {
|
||||||
|
if err := k.PutPasses(passes); err != nil {
|
||||||
|
h.noteStore(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -68,9 +139,9 @@ func (h *Holder) channelOf(kind string) Channel {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// reoffer says again on one channel, whose answers were lost at `lost`, what is still open and asked there,
|
// reoffer says again on one channel, whose answers were lost at `lost`, what is still open, asked there and
|
||||||
// edits what is silenced to say so, and leaves what ended alone. It answers whether every message to be
|
// never answered, and leaves everything else as it is: what ended, what is silenced, what was withheld for its
|
||||||
// said again was.
|
// content, and what was answered (ADR 0308 §6). It answers whether every message to be said again was.
|
||||||
func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
||||||
kind := ch.Name()
|
kind := ch.Name()
|
||||||
rp, canReplace := ch.(replacer)
|
rp, canReplace := ch.(replacer)
|
||||||
@@ -82,13 +153,14 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
|||||||
}
|
}
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
||||||
again, silenced, ended, failed := 0, 0, 0, 0
|
again, left, ended, failed := 0, 0, 0, 0
|
||||||
for _, r := range recs {
|
for _, r := range recs {
|
||||||
h.mu.Lock()
|
h.mu.Lock()
|
||||||
id, shown := r.Sent[kind]
|
id, shown := r.Sent[kind]
|
||||||
open := r.Cleared.IsZero()
|
open := r.Cleared.IsZero()
|
||||||
before := lastSaid(r).Before(lost)
|
before := lastSaid(r).Before(lost)
|
||||||
reminded := r.Reminded
|
reminded := r.Reminded
|
||||||
|
withheld, answered := r.Refused != "", r.Answered != ""
|
||||||
h.mu.Unlock()
|
h.mu.Unlock()
|
||||||
if !shown || id == "" {
|
if !shown || id == "" {
|
||||||
continue // never a message of its own there: nothing to say again under
|
continue // never a message of its own there: nothing to say again under
|
||||||
@@ -96,13 +168,8 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
|||||||
switch {
|
switch {
|
||||||
case !open:
|
case !open:
|
||||||
ended++ // its end is said where its message is; nothing comes back
|
ended++ // its end is said where its message is; nothing comes back
|
||||||
case r.silenced(now):
|
case r.silenced(now), withheld, answered:
|
||||||
h.mu.Lock()
|
left++ // a silence, a withheld message or an answer is never said again for a restart
|
||||||
r.SilenceSaid = true
|
|
||||||
h.mu.Unlock()
|
|
||||||
h.editOn(ch, r, "silenced")
|
|
||||||
h.persist(r)
|
|
||||||
silenced++
|
|
||||||
case h.askKinds(r)[kind] && before && canReplace:
|
case h.askKinds(r)[kind] && before && canReplace:
|
||||||
what := "raised"
|
what := "raised"
|
||||||
if reminded {
|
if reminded {
|
||||||
@@ -125,28 +192,10 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
h.record(Sent{At: now, Channel: kind, Key: "(answers lost)", What: "at " + lost.UTC().Format(time.RFC3339),
|
h.record(Sent{At: now, Channel: kind, Key: "(answers lost)", What: "at " + lost.UTC().Format(time.RFC3339),
|
||||||
Outcome: fmt.Sprintf("%d open and asked said again, %d silenced said so, %d ended left ended, %d failed",
|
Outcome: fmt.Sprintf("%d open and asked said again, %d silenced, withheld or answered left as they are, "+
|
||||||
again, silenced, ended, failed)})
|
"%d ended left ended, %d failed", again, left, ended, failed)})
|
||||||
h.Logf("[messenger] %s lost its answers at %s: %d open and asked said again in place, %d silenced said so, %d ended "+
|
h.Logf("[messenger] %s lost its answers at %s: %d open and asked said again in place, %d silenced, withheld or "+
|
||||||
"left ended, %d failed; nothing shown from the channel's memory", kind, lost.UTC().Format(time.RFC3339), again,
|
"answered left as they are, %d ended left ended, %d failed; nothing shown from the channel's memory", kind,
|
||||||
silenced, ended, failed)
|
lost.UTC().Format(time.RFC3339), again, left, ended, failed)
|
||||||
return failed == 0
|
return failed == 0
|
||||||
}
|
}
|
||||||
|
|
||||||
// editOn edits a record's message on one channel, where it is shown as a message of its own.
|
|
||||||
func (h *Holder) editOn(ch Channel, r *Record, what string) {
|
|
||||||
kind := ch.Name()
|
|
||||||
h.mu.Lock()
|
|
||||||
id := r.Sent[kind]
|
|
||||||
h.mu.Unlock()
|
|
||||||
if id == "" || !ch.CanEdit() {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
err := ch.Edit(id, h.sayOn(r, what, kind))
|
|
||||||
h.noteChannel(kind, err)
|
|
||||||
if err != nil {
|
|
||||||
h.Logf("[messenger] could not edit the message for %s on %s (%v)", r.Key, kind, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
h.record(Sent{At: h.Now(), Channel: kind, Key: r.Key, What: what, Outcome: "edited"})
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ import (
|
|||||||
// resolved, silenced and cancelled ones among them. Now the desk says in its standing when the answers were
|
// resolved, silenced and cancelled ones among them. Now the desk says in its standing when the answers were
|
||||||
// lost, and this router says again only what is still open and unanswered: with the desk showing one open
|
// lost, and this router says again only what is still open and unanswered: with the desk showing one open
|
||||||
// message with an answer, one silenced, one open asking nothing and one resolved, one is said again under its
|
// message with an answer, one silenced, one open asking nothing and one resolved, one is said again under its
|
||||||
// handle (in place of its message), with its answers; one is edited to the silenced words, with no answers
|
// handle (in place of its message), with its answers; nothing is said for the others — the silenced one
|
||||||
// and no urgency; nothing is said for the others. A standing that repeats the time says nothing more; its
|
// included (novox/hq ADR 0308 §6, issue 531: its edit showed it on the desk again). A standing that repeats the time says nothing more; its
|
||||||
// ticket is one-time still, and the channel that reaches the operator away hears nothing of it.
|
// ticket is one-time still, and the channel that reaches the operator away hears nothing of it.
|
||||||
func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
||||||
h, tg, dt, c, _, w := newRouter(t)
|
h, tg, dt, c, _, w := newRouter(t)
|
||||||
@@ -77,8 +77,7 @@ func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
|||||||
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
||||||
t.Fatalf("the clearing was not said: %+v", e)
|
t.Fatalf("the clearing was not said: %+v", e)
|
||||||
}
|
}
|
||||||
silencedBefore := dt.edits["g14=2"]
|
delete(dt.edits, "g14=2") // so an edit said again would be seen as such
|
||||||
delete(dt.edits, "g14=2") // so the edit said again is seen as such
|
|
||||||
sends, edits, replaced, tgSaid := len(dt.sends), len(dt.edits), len(dt.replaced), saidOn(tg)
|
sends, edits, replaced, tgSaid := len(dt.sends), len(dt.edits), len(dt.replaced), saidOn(tg)
|
||||||
// The desk's notifier started again: its standing says when the answers were lost.
|
// The desk's notifier started again: its standing says when the answers were lost.
|
||||||
c.pass(time.Minute)
|
c.pass(time.Minute)
|
||||||
@@ -95,12 +94,11 @@ func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
|||||||
if ticketFor(again, "Silence for a week") == "" || !strings.Contains(again.Title, headlineOf(open.Key)) {
|
if ticketFor(again, "Silence for a week") == "" || !strings.Contains(again.Title, headlineOf(open.Key)) {
|
||||||
t.Fatalf("said again without its answer or its words: %+v", again)
|
t.Fatalf("said again without its answer or its words: %+v", again)
|
||||||
}
|
}
|
||||||
if e, said := dt.edits["g14=2"]; !said || !strings.HasPrefix(e.Title, "Silenced until") || len(e.Actions) != 0 || e.Urgent ||
|
if e, said := dt.edits["g14=2"]; said {
|
||||||
!e.Quiet || e.Title != silencedBefore.Title {
|
t.Fatalf("the silenced one was said again for the restart: %+v", e)
|
||||||
t.Fatalf("the silenced one was not edited to the silenced words again: %+v (said %v)", e, said)
|
|
||||||
}
|
}
|
||||||
if len(dt.edits) != edits+1 {
|
if len(dt.edits) != edits {
|
||||||
t.Fatalf("an edit reached a message that is not silenced: %+v", dt.edits)
|
t.Fatalf("an edit reached the desk for the restart: %+v", dt.edits)
|
||||||
}
|
}
|
||||||
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
||||||
t.Fatalf("the resolved one was changed: %+v", e)
|
t.Fatalf("the resolved one was changed: %+v", e)
|
||||||
@@ -179,8 +177,13 @@ func TestLostAnswersAreJudgedPerMessageAndWaitForTheDesk(t *testing.T) {
|
|||||||
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
||||||
t.Fatalf("once the desk can send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
t.Fatalf("once the desk can send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||||
}
|
}
|
||||||
// The router started again, remembering no standing it acted on: the message was said since, so nothing.
|
// The router started again, reading back the pass it made from its state: nothing.
|
||||||
h.shows = map[string]time.Time{}
|
h.mu.Lock()
|
||||||
|
h.passes, h.lost = map[string]Pass{}, map[string]time.Time{}
|
||||||
|
if err := h.loadWalks(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
||||||
runFor(h, c, time.Minute)
|
runFor(h, c, time.Minute)
|
||||||
if len(dt.replaced) != replaced+1 {
|
if len(dt.replaced) != replaced+1 {
|
||||||
@@ -191,12 +194,12 @@ func TestLostAnswersAreJudgedPerMessageAndWaitForTheDesk(t *testing.T) {
|
|||||||
lost = c.now()
|
lost = c.now()
|
||||||
dt.fail = errorf("no session answers")
|
dt.fail = errorf("no session answers")
|
||||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
||||||
|
runFor(h, c, 30*time.Second)
|
||||||
dt.fail = nil
|
dt.fail = nil
|
||||||
runFor(h, c, time.Minute)
|
|
||||||
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
||||||
t.Fatalf("after a failed send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
t.Fatalf("after a failed send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||||
}
|
}
|
||||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
runFor(h, c, time.Minute)
|
||||||
if len(dt.replaced) != replaced+2 || saidOn(tg) != tgSaid {
|
if len(dt.replaced) != replaced+2 || saidOn(tg) != tgSaid {
|
||||||
t.Fatalf("the failed send was not tried again on the desk: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
t.Fatalf("the failed send was not tried again on the desk: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -113,6 +113,9 @@ func (h *Holder) Status(listening string) Status {
|
|||||||
"deduplicated by the condition's key; reminded once after 1 h (urgent) or 12 h (warning); cleared by editing the first message",
|
"deduplicated by the condition's key; reminded once after 1 h (urgent) or 12 h (warning); cleared by editing the first message",
|
||||||
"at most 20 messages an hour per channel, a last line of defence: reached, it is said once and the rest held for one digest",
|
"at most 20 messages an hour per channel, a last line of defence: reached, it is said once and the rest held for one digest",
|
||||||
"refused: anything carrying an address, a path or a secret; channel-refused is sent with the words withheld",
|
"refused: anything carrying an address, a path or a secret; channel-refused is sent with the words withheld",
|
||||||
|
"while a delivery walks, a message about a machine or module it touches is held, and the walk's end is one message: its outcome, what it reached, how long, what is still open; past the walk's bound (twice its largest delivery budget, 15 min with none) what is held and open is said (novox/hq ADR 0308)",
|
||||||
|
"an ask, a secret given or replaced, and a walk's own failure are never held",
|
||||||
|
"a channel that lost its answers is said again only what is open, asked there and never answered — never a silenced or withheld message — once per restart, and the pass is kept in this module's state",
|
||||||
"a silenced condition's message says it is silenced, until when and why, and nothing more is said of it while the silence lasts; a silence is the controller's, asked of the operator or set through the mesh",
|
"a silenced condition's message says it is silenced, until when and why, and nothing more is said of it while the silence lasts; a silence is the controller's, asked of the operator or set through the mesh",
|
||||||
}
|
}
|
||||||
anyCan := false
|
anyCan := false
|
||||||
@@ -152,9 +155,11 @@ type OpenMessage struct {
|
|||||||
Reminded bool `json:"reminded"`
|
Reminded bool `json:"reminded"`
|
||||||
Unsent string `json:"unsent,omitempty"`
|
Unsent string `json:"unsent,omitempty"`
|
||||||
Held bool `json:"held,omitempty"`
|
Held bool `json:"held,omitempty"`
|
||||||
History bool `json:"never_said,omitempty"`
|
// HeldBy are the walks holding it until they end or pass their bound (novox/hq ADR 0308).
|
||||||
Refused string `json:"refused,omitempty"`
|
HeldBy []string `json:"held_by_walks,omitempty"`
|
||||||
Count int `json:"times_opened"`
|
History bool `json:"never_said,omitempty"`
|
||||||
|
Refused string `json:"refused,omitempty"`
|
||||||
|
Count int `json:"times_opened"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Open is what is unresolved now, urgent first, oldest first.
|
// Open is what is unresolved now, urgent first, oldest first.
|
||||||
@@ -180,6 +185,7 @@ func (h *Holder) Open() []OpenMessage {
|
|||||||
o.SentOn = append(o.SentOn, ch)
|
o.SentOn = append(o.SentOn, ch)
|
||||||
}
|
}
|
||||||
sort.Strings(o.SentOn)
|
sort.Strings(o.SentOn)
|
||||||
|
o.HeldBy = h.heldBy(r.Key, now)
|
||||||
if r.silenced(now) {
|
if r.silenced(now) {
|
||||||
o.Silenced = r.SilencedTill.UTC().Format(time.RFC3339)
|
o.Silenced = r.SilencedTill.UTC().Format(time.RFC3339)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -103,7 +103,9 @@ func (h *Holder) Sync(open []Condition, asked time.Time) SyncResult {
|
|||||||
switch {
|
switch {
|
||||||
case told || silenced || h.queued(r.Key):
|
case told || silenced || h.queued(r.Key):
|
||||||
case r.Severity == Urgent:
|
case r.Severity == Urgent:
|
||||||
urgent = append(urgent, r)
|
if !h.hold(r, "raised") { // an open walk touches it: told at its end or bound (ADR 0308 §2)
|
||||||
|
urgent = append(urgent, r)
|
||||||
|
}
|
||||||
case !known && now.Sub(r.Raised) <= FreshFor:
|
case !known && now.Sub(r.Raised) <= FreshFor:
|
||||||
fresh = append(fresh, r)
|
fresh = append(fresh, r)
|
||||||
case wasQuiet && !needsNothing(r):
|
case wasQuiet && !needsNothing(r):
|
||||||
|
|||||||
@@ -0,0 +1,906 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// While a delivery walks, what it touches is held from the operator and told once, with the walk's result,
|
||||||
|
// when it ends (novox/hq ADR 0308, issue 532).
|
||||||
|
//
|
||||||
|
// **What this reads.** The controller says each walk's start and end on its `plan-moved` event (the plan
|
||||||
|
// whole, ADR 0282 §6), and on those two moments the body carries beside the plan (ADR 0308 §1):
|
||||||
|
//
|
||||||
|
// - `walk`: the plan's id (`id` is read when it is missing);
|
||||||
|
// - `phase`: `started` or `ended`; a plan-moved without it is a move inside a walk and is not read here;
|
||||||
|
// - `touches`: `[{machine, modules}]`, every module the walk moves on every machine it sends it to, and every
|
||||||
|
// module a moved tool runner or node-engine carries there;
|
||||||
|
// - `outcome`: `delivered`, `failed` or `stopped`, on `ended`;
|
||||||
|
// - `bound_seconds`: twice the largest delivery budget among its deliveries, or 900 when none has one (§5),
|
||||||
|
// on `started`; missing, 900 is taken;
|
||||||
|
// - `budget_class`: `core`, `leaf` or `none`;
|
||||||
|
// - `started_at` on the start and `ended_at` on the end (else `at`, else the plan's `updated`): when it
|
||||||
|
// happened. A start older than its bound is history and opens nothing; an end older than FreshFor with
|
||||||
|
// nothing held closes quietly.
|
||||||
|
//
|
||||||
|
// A touch may name `carried` beside `modules` (what a moved tool runner or node-engine carries); the
|
||||||
|
// controller lists those in `modules` too, which is what is read.
|
||||||
|
//
|
||||||
|
// An event of its own named `walk`, with the same body, is read the same way, should the controller say it
|
||||||
|
// there instead.
|
||||||
|
//
|
||||||
|
// **What is held (§2).** While a walk is open and within its bound, a message whose subject is a machine the
|
||||||
|
// walk touches, or a module it touches, is held instead of sent: kept on the walk, with when it was held and
|
||||||
|
// what was to be said, and the channel's history records it with the outcome `held`. A message touched by
|
||||||
|
// several open walks is kept on each, and told when the last of them ends or passes its bound.
|
||||||
|
//
|
||||||
|
// **What never is (§3):** an ask waiting for the operator's answer (an ask's own message, and a condition an
|
||||||
|
// open ask is about), a secret given or replaced, a message about a machine or module no open walk touches,
|
||||||
|
// and a walk's own failure: anything about a plan, a walk, a delivery or a build.
|
||||||
|
//
|
||||||
|
// **The end (§4).** One message: its outcome, the modules and machines it reached and how long it took; then
|
||||||
|
// each held message whose condition is still open, one line each; the ones that cleared meanwhile only
|
||||||
|
// counted. A walk that delivered with nothing held and nothing open is one line, and nothing at all on a kind
|
||||||
|
// the operator set to say only what needs them (the setting `only-what-needs-me`).
|
||||||
|
//
|
||||||
|
// **The bound (§5).** At the walk's start plus its bound, what it holds that is still open is said, with a line
|
||||||
|
// that the walk is still running and since when, and the cleared ones counted; it holds nothing more after
|
||||||
|
// that. A walk whose end never comes is forgotten a day after it started.
|
||||||
|
//
|
||||||
|
// **Kept in the module's state** (the bucket `walks`, one key per walk), so a router that restarts mid-walk
|
||||||
|
// holds and tells as if it had not.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The walk events, by their local names under the controller's seat.
|
||||||
|
const (
|
||||||
|
EventPlanMoved = "plan-moved"
|
||||||
|
EventWalk = "walk"
|
||||||
|
|
||||||
|
WalkStarted = "started"
|
||||||
|
WalkEnded = "ended"
|
||||||
|
|
||||||
|
WalkDelivered = "delivered"
|
||||||
|
WalkFailed = "failed"
|
||||||
|
WalkStopped = "stopped"
|
||||||
|
|
||||||
|
// DefaultWalkBound is a walk's bound when the controller gives none: the bound of a walk none of whose
|
||||||
|
// deliveries has a budget (ADR 0308 §5).
|
||||||
|
DefaultWalkBound = 15 * time.Minute
|
||||||
|
// WalkForget: a walk whose end never came is forgotten this long after it started.
|
||||||
|
WalkForget = 24 * time.Hour
|
||||||
|
// WalkEndWhat is the outbox's name for a walk's end or bound message: it goes alone, never in a digest.
|
||||||
|
WalkEndWhat = "walk-end"
|
||||||
|
// MostNamed: an end message names at most this many modules or machines, then counts the rest.
|
||||||
|
MostNamed = 6
|
||||||
|
)
|
||||||
|
|
||||||
|
// Touch is what a walk moves on one machine.
|
||||||
|
type Touch struct {
|
||||||
|
Machine string `json:"machine"`
|
||||||
|
Modules []string `json:"modules"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WalkEvent is a walk's start or end as the controller says it.
|
||||||
|
type WalkEvent struct {
|
||||||
|
Walk string
|
||||||
|
Phase string
|
||||||
|
Outcome string
|
||||||
|
Class string
|
||||||
|
Touches []Touch
|
||||||
|
BoundSeconds int
|
||||||
|
At time.Time
|
||||||
|
// StartedAt is the walk's own start, on both events (`started_at`): the bound runs from it, so a router
|
||||||
|
// that hears a start late, or starts again, ends the hold on time.
|
||||||
|
StartedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// HeldMessage is one message a walk holds: when it was first held and what was to be said.
|
||||||
|
type HeldMessage struct {
|
||||||
|
At time.Time `json:"at"`
|
||||||
|
What string `json:"what"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Walk is one open walk as this router keeps it.
|
||||||
|
type Walk struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Started time.Time `json:"started"`
|
||||||
|
BoundSeconds int `json:"bound_seconds"`
|
||||||
|
Class string `json:"budget_class,omitempty"`
|
||||||
|
Touches []Touch `json:"touches"`
|
||||||
|
Held map[string]HeldMessage `json:"held,omitempty"`
|
||||||
|
// BoundSaid: its bound passed and what it held was said; it holds nothing more.
|
||||||
|
BoundSaid bool `json:"bound_said,omitempty"`
|
||||||
|
// Ended: its end was heard, at this moment. Kept a day, so a start heard after its end opens nothing.
|
||||||
|
Ended time.Time `json:"ended,omitempty"`
|
||||||
|
// EndedSaid is the end's moment as the controller said it, by the controller's clock and never capped at
|
||||||
|
// this router's now: a start heard after its end is compared with it on that one clock (issue 538 C).
|
||||||
|
EndedSaid time.Time `json:"ended_said,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *Walk) bound() time.Duration {
|
||||||
|
if w.BoundSeconds <= 0 {
|
||||||
|
return DefaultWalkBound
|
||||||
|
}
|
||||||
|
return time.Duration(w.BoundSeconds) * time.Second
|
||||||
|
}
|
||||||
|
|
||||||
|
// holding says whether the walk still holds at a moment: open, and within its bound.
|
||||||
|
func (w *Walk) holding(now time.Time) bool {
|
||||||
|
return !w.BoundSaid && now.Before(w.Started.Add(w.bound()))
|
||||||
|
}
|
||||||
|
|
||||||
|
// touches says whether the walk touches a subject: a machine it sends to, or — when the subject names no
|
||||||
|
// machine of its own — a module it moves anywhere.
|
||||||
|
func (w *Walk) touches(s subject) bool {
|
||||||
|
for _, t := range w.Touches {
|
||||||
|
for _, m := range s.machines {
|
||||||
|
if m != "" && m == t.Machine {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.machineNamed {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, t := range w.Touches {
|
||||||
|
for _, mod := range t.Modules {
|
||||||
|
for _, m := range s.modules {
|
||||||
|
if m != "" && m == mod {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// walkKeeper is a store that keeps the walks and the restart passes too (the module's state, ADR 0201).
|
||||||
|
type walkKeeper interface {
|
||||||
|
PutWalk(w Walk) error
|
||||||
|
DeleteWalk(id string) error
|
||||||
|
Walks() ([]Walk, error)
|
||||||
|
PutPasses(map[string]Pass) error
|
||||||
|
Passes() (map[string]Pass, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeWalk reads a plan-moved or walk event's body: false when it says no walk's start or end.
|
||||||
|
func DecodeWalk(body []byte) (WalkEvent, bool, error) {
|
||||||
|
var raw map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(body, &raw); err != nil || raw == nil {
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("a walk event's body is not a JSON object")
|
||||||
|
}
|
||||||
|
str := func(names ...string) string {
|
||||||
|
for _, n := range names {
|
||||||
|
var s string
|
||||||
|
if v, ok := raw[n]; ok && json.Unmarshal(v, &s) == nil && strings.TrimSpace(s) != "" {
|
||||||
|
return strings.TrimSpace(s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
var ev WalkEvent
|
||||||
|
ev.Phase = str("phase")
|
||||||
|
if ev.Phase == "" {
|
||||||
|
return WalkEvent{}, false, nil
|
||||||
|
}
|
||||||
|
ev.Walk = str("walk", "id")
|
||||||
|
if ev.Walk == "" {
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("a walk's %s names no walk", ev.Phase)
|
||||||
|
}
|
||||||
|
switch ev.Phase {
|
||||||
|
case WalkStarted, WalkEnded:
|
||||||
|
default:
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("walk %s: phase %q is neither started nor ended", ev.Walk, ev.Phase)
|
||||||
|
}
|
||||||
|
ev.Outcome = str("outcome")
|
||||||
|
if ev.Phase == WalkEnded {
|
||||||
|
switch ev.Outcome {
|
||||||
|
case WalkDelivered, WalkFailed, WalkStopped:
|
||||||
|
default:
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("walk %s ended with outcome %q: neither delivered, failed nor stopped", ev.Walk, ev.Outcome)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ev.Class = str("budget_class", "budget-class")
|
||||||
|
for _, n := range []string{"bound_seconds", "bound-seconds"} {
|
||||||
|
if v, ok := raw[n]; ok && string(v) != "null" {
|
||||||
|
var f float64
|
||||||
|
if json.Unmarshal(v, &f) != nil || f < 0 {
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("walk %s: %s is not a number of seconds", ev.Walk, n)
|
||||||
|
}
|
||||||
|
ev.BoundSeconds = int(f)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v, ok := raw["touches"]; ok && string(v) != "null" {
|
||||||
|
var list []struct {
|
||||||
|
Machine string `json:"machine"`
|
||||||
|
Node string `json:"node"`
|
||||||
|
Modules []string `json:"modules"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(v, &list) != nil {
|
||||||
|
return WalkEvent{}, false, fmt.Errorf("walk %s: touches is not a list of {machine, modules}", ev.Walk)
|
||||||
|
}
|
||||||
|
for _, t := range list {
|
||||||
|
m := t.Machine
|
||||||
|
if m == "" {
|
||||||
|
m = t.Node
|
||||||
|
}
|
||||||
|
if m != "" {
|
||||||
|
ev.Touches = append(ev.Touches, Touch{Machine: m, Modules: t.Modules})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s := str("started_at"); s != "" {
|
||||||
|
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||||
|
ev.StartedAt = t
|
||||||
|
}
|
||||||
|
}
|
||||||
|
when := []string{"started_at", "at", "updated"}
|
||||||
|
if ev.Phase == WalkEnded {
|
||||||
|
when = []string{"ended_at", "at", "updated"}
|
||||||
|
}
|
||||||
|
for _, n := range when {
|
||||||
|
if s := str(n); s != "" {
|
||||||
|
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||||
|
ev.At = t
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ev, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// mergeTouches is what two sayings of a walk's touches name together.
|
||||||
|
func mergeTouches(a, b []Touch) []Touch {
|
||||||
|
by := map[string]map[string]bool{}
|
||||||
|
var order []string
|
||||||
|
for _, list := range [][]Touch{a, b} {
|
||||||
|
for _, t := range list {
|
||||||
|
if by[t.Machine] == nil {
|
||||||
|
by[t.Machine] = map[string]bool{}
|
||||||
|
order = append(order, t.Machine)
|
||||||
|
}
|
||||||
|
for _, m := range t.Modules {
|
||||||
|
by[t.Machine][m] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(order)
|
||||||
|
var out []Touch
|
||||||
|
for _, machine := range order {
|
||||||
|
var mods []string
|
||||||
|
for m := range by[machine] {
|
||||||
|
mods = append(mods, m)
|
||||||
|
}
|
||||||
|
sort.Strings(mods)
|
||||||
|
out = append(out, Touch{Machine: machine, Modules: mods})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Walked takes a walk's start or end.
|
||||||
|
func (h *Holder) Walked(ev WalkEvent) {
|
||||||
|
h.work.Lock()
|
||||||
|
defer h.work.Unlock()
|
||||||
|
h.mu.Lock()
|
||||||
|
h.init()
|
||||||
|
now := h.Now()
|
||||||
|
h.mu.Unlock()
|
||||||
|
at := ev.At
|
||||||
|
if at.IsZero() || at.After(now) {
|
||||||
|
at = now
|
||||||
|
}
|
||||||
|
switch ev.Phase {
|
||||||
|
case WalkStarted:
|
||||||
|
w := &Walk{ID: ev.Walk, Started: at, BoundSeconds: ev.BoundSeconds, Class: ev.Class}
|
||||||
|
h.mu.Lock()
|
||||||
|
end := h.ended[ev.Walk]
|
||||||
|
h.mu.Unlock()
|
||||||
|
if end != nil && startOfEnded(ev, at, end) {
|
||||||
|
// A short walk's two events can come in either order: this start is of the walk that ended.
|
||||||
|
h.Logf("[messenger] walk %s: its start came after its end, which was heard at %s: nothing held", ev.Walk,
|
||||||
|
end.Ended.UTC().Format(time.RFC3339))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if now.Sub(at) >= w.bound() {
|
||||||
|
h.Logf("[messenger] walk %s started %s ago, past its bound of %s: history, nothing held", ev.Walk,
|
||||||
|
roughly(now.Sub(at)), w.bound())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
if known := h.walks[ev.Walk]; known != nil {
|
||||||
|
w = known // said again: the same walk, its touches as both sayings name them
|
||||||
|
} else {
|
||||||
|
w.Held = map[string]HeldMessage{}
|
||||||
|
h.walks[ev.Walk] = w
|
||||||
|
}
|
||||||
|
w.Touches = mergeTouches(w.Touches, ev.Touches)
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.keepWalk(w)
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: "walk." + ev.Walk, What: "started: " + touchWords(w.Touches),
|
||||||
|
Outcome: "holding what it touches for at most " + roughly(w.bound())})
|
||||||
|
case WalkEnded:
|
||||||
|
h.mu.Lock()
|
||||||
|
w := h.walks[ev.Walk]
|
||||||
|
if w != nil && !ev.StartedAt.IsZero() && w.Started.After(ev.StartedAt) {
|
||||||
|
// The end of an earlier walk under the same id, heard after the next one started: that one stays open.
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.Logf("[messenger] walk %s: an end of its earlier walk (started %s) came late: the open one stays", ev.Walk,
|
||||||
|
ev.StartedAt.UTC().Format(time.RFC3339))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if w != nil {
|
||||||
|
delete(h.walks, ev.Walk)
|
||||||
|
w.Touches = mergeTouches(w.Touches, ev.Touches)
|
||||||
|
if !ev.StartedAt.IsZero() {
|
||||||
|
w.Started = ev.StartedAt
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tomb := &Walk{ID: ev.Walk, Started: ev.StartedAt, Ended: at, EndedSaid: ev.At}
|
||||||
|
if prev := h.ended[ev.Walk]; prev != nil && prev.Ended.After(at) {
|
||||||
|
tomb = prev
|
||||||
|
}
|
||||||
|
h.ended[ev.Walk] = tomb
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.keepWalk(tomb) // in place of the open walk: a start heard after this end opens nothing
|
||||||
|
if w == nil {
|
||||||
|
h.Logf("[messenger] walk %s %s, and its start was not heard here: nothing held, nothing said", ev.Walk, ev.Outcome)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.endWalk(w, ev.Outcome, at, now)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// startOfEnded says whether a start is of a walk whose end was heard already: of that walk, or of an earlier one
|
||||||
|
// under its id. The controller's own moments are compared on its clock, never one capped at this router's now
|
||||||
|
// against one that is not (issue 538 C: a controller's clock ahead of the router's would open a walk after its
|
||||||
|
// end). Only when the controller said neither moment are the moments as heard here compared.
|
||||||
|
func startOfEnded(ev WalkEvent, at time.Time, end *Walk) bool {
|
||||||
|
said := ev.StartedAt
|
||||||
|
if said.IsZero() {
|
||||||
|
said = ev.At
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case !said.IsZero() && !end.Started.IsZero():
|
||||||
|
return !said.After(end.Started)
|
||||||
|
case !said.IsZero() && !end.EndedSaid.IsZero():
|
||||||
|
return !said.After(end.EndedSaid)
|
||||||
|
}
|
||||||
|
return !at.After(end.Ended)
|
||||||
|
}
|
||||||
|
|
||||||
|
// release is what a walk held that no other open walk holds or touches: each open message, oldest first, with
|
||||||
|
// what was to be said of it, and how many cleared meanwhile. A message said meanwhile on a path that is never
|
||||||
|
// held, or silenced meanwhile, is neither. One that another open walk touches — even one that started after it
|
||||||
|
// was held — is held on by that walk, and told at its end or bound (ADR 0308 §2).
|
||||||
|
//
|
||||||
|
// A message is handed on under the rules of its first hold (issue 538 B): still open, not silenced, not said
|
||||||
|
// since it was held, and not one ADR 0308 §3 never holds (an open ask about it, a walk's own failure).
|
||||||
|
func (h *Holder) release(w *Walk, now time.Time) (open []*Record, whats map[string]string, cleared int) {
|
||||||
|
// What may be handed on is judged first: neverHeld asks the asks and the walks under the lock itself.
|
||||||
|
h.mu.Lock()
|
||||||
|
recs := map[string]*Record{}
|
||||||
|
for key := range w.Held {
|
||||||
|
if r := h.open[key]; r != nil && r.Cleared.IsZero() && !r.silenced(now) && !lastSaid(r).After(w.Held[key].At) {
|
||||||
|
recs[key] = r
|
||||||
|
}
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
handable := map[string]bool{}
|
||||||
|
for key, r := range recs {
|
||||||
|
handable[key] = !h.neverHeld(r)
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
whats = map[string]string{}
|
||||||
|
taken := map[string]*Walk{}
|
||||||
|
var handed []string
|
||||||
|
for key, hm := range w.Held {
|
||||||
|
r := h.open[key]
|
||||||
|
live := r != nil && r.Cleared.IsZero() && handable[key]
|
||||||
|
var s subject
|
||||||
|
if live {
|
||||||
|
s = subjectOf(r)
|
||||||
|
}
|
||||||
|
elsewhere := false
|
||||||
|
for _, o := range h.walks {
|
||||||
|
if o.ID == w.ID || !o.holding(now) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := o.Held[key]; ok {
|
||||||
|
elsewhere = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if live && o.touches(s) {
|
||||||
|
if o.Held == nil {
|
||||||
|
o.Held = map[string]HeldMessage{}
|
||||||
|
}
|
||||||
|
o.Held[key] = hm
|
||||||
|
taken[o.ID] = o
|
||||||
|
handed = append(handed, key+" to walk "+o.ID)
|
||||||
|
elsewhere = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if elsewhere {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case r == nil || !r.Cleared.IsZero():
|
||||||
|
cleared++
|
||||||
|
case r.silenced(now) || lastSaid(r).After(hm.At):
|
||||||
|
default:
|
||||||
|
open = append(open, r)
|
||||||
|
whats[key] = hm.What
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(open, func(i, j int) bool {
|
||||||
|
if !open[i].Raised.Equal(open[j].Raised) {
|
||||||
|
return open[i].Raised.Before(open[j].Raised)
|
||||||
|
}
|
||||||
|
return open[i].Key < open[j].Key
|
||||||
|
})
|
||||||
|
h.mu.Unlock()
|
||||||
|
for _, o := range taken {
|
||||||
|
h.keepWalk(o)
|
||||||
|
}
|
||||||
|
sort.Strings(handed)
|
||||||
|
for _, x := range handed {
|
||||||
|
key, _, _ := strings.Cut(x, " to walk ")
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: key, What: "held on: walk " + w.ID + " ended, " + x[len(key)+1:] + " touches it",
|
||||||
|
Outcome: "held"})
|
||||||
|
}
|
||||||
|
return open, whats, cleared
|
||||||
|
}
|
||||||
|
|
||||||
|
// endWalk tells a walk's end once (ADR 0308 §4).
|
||||||
|
func (h *Holder) endWalk(w *Walk, outcome string, ended, now time.Time) {
|
||||||
|
open, whats, cleared := h.release(w, now)
|
||||||
|
if now.Sub(ended) > FreshFor && len(open) == 0 {
|
||||||
|
h.Logf("[messenger] walk %s %s %s ago, heard late: history, nothing said (%d held cleared meanwhile)",
|
||||||
|
w.ID, outcome, roughly(now.Sub(ended)), cleared)
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: "walk." + w.ID, What: "ended " + outcome, Outcome: "quiet: heard late"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
took := ended.Sub(w.Started)
|
||||||
|
head := map[string]string{WalkDelivered: "Delivered", WalkFailed: "Delivery failed", WalkStopped: "Delivery stopped"}[outcome]
|
||||||
|
if head == "" {
|
||||||
|
head = "Delivery ended"
|
||||||
|
}
|
||||||
|
reached := touchWords(w.Touches)
|
||||||
|
oneLine := outcome == WalkDelivered && len(w.Held) == 0 && len(open) == 0
|
||||||
|
var m Message
|
||||||
|
if oneLine {
|
||||||
|
m = Message{Title: head + ": " + reached + ", in " + roughly(took), Quiet: true}
|
||||||
|
} else {
|
||||||
|
lines := []string{"It took " + roughly(took) + "."}
|
||||||
|
lines = append(lines, h.heldLines(open, whats, cleared)...)
|
||||||
|
m = Message{Title: head + ": " + reached, Body: strings.Join(lines, "\n")}
|
||||||
|
}
|
||||||
|
h.tellWalk(w, m, open, whats, oneLine, "ended "+outcome, now)
|
||||||
|
}
|
||||||
|
|
||||||
|
// heldLines are the lines a walk's end or bound says of what it held: each still open, then the cleared count.
|
||||||
|
func (h *Holder) heldLines(open []*Record, whats map[string]string, cleared int) []string {
|
||||||
|
var lines []string
|
||||||
|
if len(open) > 0 {
|
||||||
|
lines = append(lines, "Still open after it:")
|
||||||
|
for _, r := range open {
|
||||||
|
what := whats[r.Key]
|
||||||
|
if what == "" {
|
||||||
|
what = "raised"
|
||||||
|
}
|
||||||
|
lines = append(lines, "- "+h.say(r, what).Title)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case cleared == 1:
|
||||||
|
lines = append(lines, "1 problem it raised resolved during it.")
|
||||||
|
case cleared > 1:
|
||||||
|
lines = append(lines, fmt.Sprintf("%d problems it raised resolved during it.", cleared))
|
||||||
|
}
|
||||||
|
return lines
|
||||||
|
}
|
||||||
|
|
||||||
|
// boundWalks says, of every walk past its bound, what it holds that is still open (ADR 0308 §5), then forgets
|
||||||
|
// the walks whose end never came, and the ends heard a day ago. What a walk holds is said before it is
|
||||||
|
// forgotten: a router back after a long while says it, then forgets the walk.
|
||||||
|
func (h *Holder) boundWalks(now time.Time) {
|
||||||
|
h.mu.Lock()
|
||||||
|
var due, gone []*Walk
|
||||||
|
var stale []string
|
||||||
|
for _, w := range h.walks {
|
||||||
|
if !w.BoundSaid && !now.Before(w.Started.Add(w.bound())) {
|
||||||
|
due = append(due, w)
|
||||||
|
}
|
||||||
|
if now.Sub(w.Started) >= WalkForget {
|
||||||
|
gone = append(gone, w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for id, e := range h.ended {
|
||||||
|
if now.Sub(e.Ended) >= WalkForget {
|
||||||
|
stale = append(stale, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, id := range stale {
|
||||||
|
delete(h.ended, id)
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
for _, id := range stale {
|
||||||
|
h.mu.Lock()
|
||||||
|
reopened := h.walks[id] != nil
|
||||||
|
h.mu.Unlock()
|
||||||
|
if !reopened {
|
||||||
|
h.forgetWalk(id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
for _, w := range gone {
|
||||||
|
h.mu.Lock()
|
||||||
|
delete(h.walks, w.ID)
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.Logf("[messenger] walk %s started %s ago and its end never came: forgotten", w.ID, roughly(now.Sub(w.Started)))
|
||||||
|
h.forgetWalk(w.ID)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
sort.Slice(due, func(i, j int) bool { return due[i].Started.Before(due[j].Started) })
|
||||||
|
for _, w := range due {
|
||||||
|
h.mu.Lock()
|
||||||
|
w.BoundSaid = true
|
||||||
|
h.mu.Unlock()
|
||||||
|
open, whats, cleared := h.release(w, now)
|
||||||
|
h.mu.Lock()
|
||||||
|
w.Held = map[string]HeldMessage{}
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.keepWalk(w)
|
||||||
|
if len(open) == 0 && cleared == 0 {
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: "walk." + w.ID, What: "past its bound", Outcome: "nothing held"})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
since := w.Started.In(h.zone()).Format("15:04")
|
||||||
|
lines := []string{"The walk " + w.ID + " is still running, since " + since + "."}
|
||||||
|
if _, ok := Check(lines[0]); !ok {
|
||||||
|
lines[0] = "The walk is still running, since " + since + "."
|
||||||
|
}
|
||||||
|
lines = append(lines, h.heldLines(open, whats, cleared)...)
|
||||||
|
m := Message{Title: "Still running: the delivery to " + touchWords(w.Touches), Body: strings.Join(lines, "\n")}
|
||||||
|
h.tellWalk(w, m, open, whats, false, "past its bound", now)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// tellWalk holds a walk's message on the kinds it goes to, alone, and marks what it carries as said there. When
|
||||||
|
// no channel can send, or the message cannot carry what it held, each held message is left to its own path:
|
||||||
|
// unsent, and tried again every minute as any other (outbox.go), or said on its own.
|
||||||
|
func (h *Holder) tellWalk(w *Walk, m Message, open []*Record, whats map[string]string, oneLine bool, what string, now time.Time) {
|
||||||
|
for _, r := range open {
|
||||||
|
m.Urgent = m.Urgent || r.Severity == Urgent
|
||||||
|
}
|
||||||
|
var alone []*Record // said on their own, after the walk's message
|
||||||
|
if _, ok := Check(m.Text()); !ok {
|
||||||
|
m = Message{Title: "A delivery ended", Body: "What it reached could not be said here: the delivery's walks say it.", Urgent: m.Urgent}
|
||||||
|
alone, open = open, nil
|
||||||
|
}
|
||||||
|
// Owed until a message carrying it was sent, in the state: a router that restarts first says it on its own
|
||||||
|
// (issue 538), whichever path below it takes.
|
||||||
|
h.mu.Lock()
|
||||||
|
for _, r := range append(append([]*Record(nil), open...), alone...) {
|
||||||
|
say := whats[r.Key]
|
||||||
|
if say == "" {
|
||||||
|
say = "raised"
|
||||||
|
}
|
||||||
|
r.Owed = say
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
var to []Channel
|
||||||
|
if m.Urgent {
|
||||||
|
for _, ch := range h.channels() {
|
||||||
|
if ch.Ready() == nil {
|
||||||
|
to = append(to, ch)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
switch {
|
||||||
|
case h.Desk != nil && h.Desk.Ready() == nil:
|
||||||
|
to = []Channel{h.Desk}
|
||||||
|
case h.Away != nil && h.Away.Ready() == nil:
|
||||||
|
to = []Channel{h.Away}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var kept []Channel
|
||||||
|
for _, ch := range to {
|
||||||
|
if oneLine && h.NeedsOnly != nil && h.NeedsOnly(ch.Name()) {
|
||||||
|
h.record(Sent{At: now, Channel: ch.Name(), Key: "walk." + w.ID, What: what,
|
||||||
|
Outcome: "quiet: this kind says only what needs the operator"})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
kept = append(kept, ch)
|
||||||
|
}
|
||||||
|
if len(to) == 0 {
|
||||||
|
h.Logf("[messenger] walk %s %s: no channel can send now; what it held is tried again every minute: %s", w.ID, what, h.whyNot())
|
||||||
|
for _, r := range append(open, alone...) {
|
||||||
|
say := whats[r.Key]
|
||||||
|
if say == "" {
|
||||||
|
say = "raised"
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
r.Pending = say
|
||||||
|
h.mu.Unlock()
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released by walk " + w.ID + " " + what,
|
||||||
|
Outcome: "unsent: no channel can send; tried again every minute"})
|
||||||
|
h.persist(r)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sev := Warning
|
||||||
|
if m.Urgent {
|
||||||
|
sev = Urgent
|
||||||
|
}
|
||||||
|
// What it carries is marked told on a channel once the message was sent there (outbox.go), not now: a
|
||||||
|
// message lost with a restart before it was sent leaves them untold and owed, and the restarted router says
|
||||||
|
// each on its own (issue 538 A).
|
||||||
|
var carries []string
|
||||||
|
for _, r := range open {
|
||||||
|
carries = append(carries, r.Key)
|
||||||
|
}
|
||||||
|
for _, ch := range kept {
|
||||||
|
msg := m
|
||||||
|
h.enqueue(ch.Name(), item{Key: "walk." + w.ID, What: WalkEndWhat, Severity: sev, At: now, Msg: &msg, Carries: carries})
|
||||||
|
}
|
||||||
|
for _, r := range open {
|
||||||
|
if len(kept) > 0 {
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released",
|
||||||
|
Outcome: "to be told in the message of walk " + w.ID + " " + what})
|
||||||
|
}
|
||||||
|
h.persist(r) // owed, in the state, until the message was sent
|
||||||
|
}
|
||||||
|
h.flush(now, false)
|
||||||
|
for _, r := range alone {
|
||||||
|
say := whats[r.Key]
|
||||||
|
if say == "" {
|
||||||
|
say = "raised"
|
||||||
|
}
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released", Outcome: "said on its own: walk " + w.ID + "'s message could not carry it"})
|
||||||
|
h.deliver(r, say)
|
||||||
|
h.persist(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// touchWords names what a walk reaches: its modules, then the machines.
|
||||||
|
func touchWords(touches []Touch) string {
|
||||||
|
mods := map[string]bool{}
|
||||||
|
var machines []string
|
||||||
|
for _, t := range touches {
|
||||||
|
machines = append(machines, t.Machine)
|
||||||
|
for _, m := range t.Modules {
|
||||||
|
mods[m] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var names []string
|
||||||
|
for m := range mods {
|
||||||
|
names = append(names, m)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
sort.Strings(machines)
|
||||||
|
if len(names) == 0 {
|
||||||
|
return "nothing moved"
|
||||||
|
}
|
||||||
|
if len(machines) == 0 {
|
||||||
|
return listed(names)
|
||||||
|
}
|
||||||
|
return listed(names) + " on " + listed(machines)
|
||||||
|
}
|
||||||
|
|
||||||
|
// listed is at most MostNamed names, then how many more.
|
||||||
|
func listed(names []string) string {
|
||||||
|
if len(names) <= MostNamed {
|
||||||
|
return strings.Join(names, ", ")
|
||||||
|
}
|
||||||
|
return strings.Join(names[:MostNamed], ", ") + fmt.Sprintf(" and %d more", len(names)-MostNamed)
|
||||||
|
}
|
||||||
|
|
||||||
|
// subject is what a message is about, as a walk's touches are matched against it.
|
||||||
|
type subject struct {
|
||||||
|
machines, modules []string
|
||||||
|
// machineNamed: the subject names its machine, so a module it names is matched on that machine alone.
|
||||||
|
machineNamed bool
|
||||||
|
// tokens are every name it carries, to tell a message about a walk itself.
|
||||||
|
tokens []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// subjectOf is a record's subject: a condition's scope and id (`<scope>.<id>.<kind>` when the record kept no
|
||||||
|
// subject of its own), a notice's subject words.
|
||||||
|
func subjectOf(r *Record) subject {
|
||||||
|
scope, id := r.Scope, r.SubjectID
|
||||||
|
parts := strings.Split(r.Key, ".")
|
||||||
|
if scope == "" && len(parts) > 0 {
|
||||||
|
scope = parts[0]
|
||||||
|
}
|
||||||
|
if id == "" && len(parts) >= 3 {
|
||||||
|
id = strings.Join(parts[1:len(parts)-1], ".")
|
||||||
|
}
|
||||||
|
var s subject
|
||||||
|
tokens := strings.Split(id, ".")
|
||||||
|
if r.Origin == "notify" {
|
||||||
|
tokens = append(strings.FieldsFunc(r.Subject, func(c rune) bool { return c == ' ' || c == ',' || c == '/' }), parts[1:]...)
|
||||||
|
scope = "notify"
|
||||||
|
}
|
||||||
|
s.tokens = tokens
|
||||||
|
if r.Machine != "" {
|
||||||
|
s.machines, s.machineNamed = append(s.machines, r.Machine), true
|
||||||
|
}
|
||||||
|
switch scope {
|
||||||
|
case "machine", "node", "bus":
|
||||||
|
s.machines, s.machineNamed = append(s.machines, tokens[0]), true
|
||||||
|
if len(tokens) > 1 {
|
||||||
|
s.modules = append(s.modules, tokens[1])
|
||||||
|
}
|
||||||
|
case "module", "core", "seat":
|
||||||
|
s.modules = append(s.modules, tokens[0])
|
||||||
|
if len(tokens) > 1 {
|
||||||
|
s.machines, s.machineNamed = append(s.machines, tokens[1]), true
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
// Its names, each matched as a machine and as a module.
|
||||||
|
s.machines = append(s.machines, tokens...)
|
||||||
|
s.modules = append(s.modules, tokens...)
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// neverHeldScopes are the subjects of a walk's own failure (ADR 0308 §3): a plan, a walk, a delivery, a build.
|
||||||
|
var neverHeldScopes = map[string]bool{"plan": true, "walk": true, "delivery": true, "build": true, "release": true}
|
||||||
|
|
||||||
|
// neverHeldKinds are what is sent at once whatever a walk touches: a secret given or replaced (ADR 0308 §3).
|
||||||
|
var neverHeldKinds = map[string]bool{"secret-given": true, "secret-replaced": true}
|
||||||
|
|
||||||
|
// neverHeld says whether a record is one ADR 0308 §3 sends at once.
|
||||||
|
func (h *Holder) neverHeld(r *Record) bool {
|
||||||
|
if r.Origin == "ask" || h.openAsk(r) != nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
|
parts := strings.Split(r.Key, ".")
|
||||||
|
kind := r.Kind
|
||||||
|
if kind == "" || kind == "notice" {
|
||||||
|
kind = parts[len(parts)-1]
|
||||||
|
}
|
||||||
|
scope := r.Scope
|
||||||
|
if scope == "" {
|
||||||
|
scope = parts[0]
|
||||||
|
}
|
||||||
|
if neverHeldKinds[kind] || neverHeldScopes[scope] {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
for _, t := range subjectOf(r).tokens {
|
||||||
|
if h.walks[t] != nil {
|
||||||
|
return true // about a walk itself
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// hold holds a record's message when an open walk touches its subject (ADR 0308 §2): kept on every such walk,
|
||||||
|
// with when it was first held and what was to be said, and said held in the channel's history.
|
||||||
|
func (h *Holder) hold(r *Record, what string) bool {
|
||||||
|
switch what {
|
||||||
|
case "cleared", "silenced", "asked":
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
h.init()
|
||||||
|
walking := len(h.walks) > 0
|
||||||
|
h.mu.Unlock()
|
||||||
|
if !walking || h.neverHeld(r) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
now := h.Now()
|
||||||
|
h.mu.Lock()
|
||||||
|
s := subjectOf(r)
|
||||||
|
var by []*Walk
|
||||||
|
var ids []string
|
||||||
|
fresh := false
|
||||||
|
for _, w := range h.walks {
|
||||||
|
if !w.holding(now) || !w.touches(s) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
prev, had := w.Held[r.Key]
|
||||||
|
hm := HeldMessage{At: now, What: what}
|
||||||
|
if had {
|
||||||
|
hm.At = prev.At
|
||||||
|
if !r.told() && prev.What == "raised" {
|
||||||
|
hm.What = prev.What // never said yet: it is a new problem to the operator, whatever came since
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !had || hm.What != prev.What {
|
||||||
|
fresh = true
|
||||||
|
}
|
||||||
|
if w.Held == nil {
|
||||||
|
w.Held = map[string]HeldMessage{}
|
||||||
|
}
|
||||||
|
w.Held[r.Key] = hm
|
||||||
|
by = append(by, w)
|
||||||
|
ids = append(ids, w.ID)
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
if len(by) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
sort.Strings(ids)
|
||||||
|
for _, w := range by {
|
||||||
|
h.keepWalk(w)
|
||||||
|
}
|
||||||
|
if fresh {
|
||||||
|
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: what + ", while walk " + strings.Join(ids, ", ") + " touches it",
|
||||||
|
Outcome: "held"})
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// heldBy is the walks holding a record now.
|
||||||
|
func (h *Holder) heldBy(key string, now time.Time) []string {
|
||||||
|
var ids []string
|
||||||
|
for _, w := range h.walks {
|
||||||
|
if _, ok := w.Held[key]; ok && w.holding(now) {
|
||||||
|
ids = append(ids, w.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(ids)
|
||||||
|
return ids
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Holder) keepWalk(w *Walk) {
|
||||||
|
k, ok := h.Store.(walkKeeper)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
h.mu.Lock()
|
||||||
|
c := *w
|
||||||
|
c.Held = map[string]HeldMessage{}
|
||||||
|
for key, v := range w.Held {
|
||||||
|
c.Held[key] = v
|
||||||
|
}
|
||||||
|
h.mu.Unlock()
|
||||||
|
if err := k.PutWalk(c); err != nil {
|
||||||
|
h.noteStore(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h *Holder) forgetWalk(id string) {
|
||||||
|
if k, ok := h.Store.(walkKeeper); ok {
|
||||||
|
if err := k.DeleteWalk(id); err != nil {
|
||||||
|
h.noteStore(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// loadWalks reads back the open walks and the restart passes; called with h.mu held.
|
||||||
|
func (h *Holder) loadWalks() error {
|
||||||
|
k, ok := h.Store.(walkKeeper)
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
walks, err := k.Walks()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for i := range walks {
|
||||||
|
w := walks[i]
|
||||||
|
if !w.Ended.IsZero() {
|
||||||
|
h.ended[w.ID] = &w
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if w.Held == nil {
|
||||||
|
w.Held = map[string]HeldMessage{}
|
||||||
|
}
|
||||||
|
h.walks[w.ID] = &w
|
||||||
|
}
|
||||||
|
passes, err := k.Passes()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for kind, p := range passes {
|
||||||
|
h.passes[kind] = p
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,770 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The tests of novox/hq ADR 0308 "How it is checked" §2–§6, with the controller's walk events given by a fake
|
||||||
|
// source: the body the controller's plan-moved carries at a walk's start and end (ADR 0308 §1).
|
||||||
|
|
||||||
|
// walkBody is a plan-moved body as the controller says a walk's start or end (mesh-controller pull request 247).
|
||||||
|
func walkBody(t *testing.T, id, phase, outcome string, at time.Time, bound int, touches ...Touch) []byte {
|
||||||
|
t.Helper()
|
||||||
|
b := map[string]any{"id": id, "state": "walking", "walk": id, "phase": phase, "touches": touches,
|
||||||
|
"updated": at.Format(time.RFC3339Nano), "budget_class": "leaf"}
|
||||||
|
if phase == WalkStarted {
|
||||||
|
b["started_at"] = at.Format(time.RFC3339Nano)
|
||||||
|
} else {
|
||||||
|
b["ended_at"] = at.Format(time.RFC3339Nano)
|
||||||
|
}
|
||||||
|
if outcome != "" {
|
||||||
|
b["outcome"] = outcome
|
||||||
|
}
|
||||||
|
if bound > 0 {
|
||||||
|
b["bound_seconds"] = bound
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(b)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
|
||||||
|
// walk says a walk's start or end to the router, through the decoder the bus's events go through.
|
||||||
|
func walk(t *testing.T, h *Holder, id, phase, outcome string, at time.Time, bound int, touches ...Touch) {
|
||||||
|
t.Helper()
|
||||||
|
ev, ok, err := DecodeWalk(walkBody(t, id, phase, outcome, at, bound, touches...))
|
||||||
|
if err != nil || !ok {
|
||||||
|
t.Fatalf("the walk event was not read: %v %v", ok, err)
|
||||||
|
}
|
||||||
|
h.Walked(ev)
|
||||||
|
}
|
||||||
|
|
||||||
|
func novox(modules ...string) Touch { return Touch{Machine: "novox", Modules: modules} }
|
||||||
|
|
||||||
|
// heard says whether any message on the channels carries the words: sent, edited or said again in place.
|
||||||
|
func heard(words string, chs ...*fakeChannel) bool {
|
||||||
|
for _, ch := range chs {
|
||||||
|
for _, m := range ch.sends {
|
||||||
|
if strings.Contains(m.Text(), words) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, m := range ch.edits {
|
||||||
|
if strings.Contains(m.Text(), words) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, r := range ch.replaced {
|
||||||
|
if strings.Contains(r.m.Text(), words) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// outcomes are the history's outcomes for a key, oldest first.
|
||||||
|
func outcomes(h *Holder, key string) []string {
|
||||||
|
var out []string
|
||||||
|
h.mu.Lock()
|
||||||
|
defer h.mu.Unlock()
|
||||||
|
for _, s := range h.recent {
|
||||||
|
if s.Key == key {
|
||||||
|
out = append(out, s.Outcome)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func has(list []string, want string) bool {
|
||||||
|
for _, s := range list {
|
||||||
|
if s == want {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// §2: during a walk, a message about a machine or a module it touches is held, not sent, and the channel's
|
||||||
|
// history says it was held — a module's condition, a machine's, and a module's notice alike.
|
||||||
|
func TestDuringAWalkWhatItTouchesIsHeldAndTheHistorySaysSo(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-1", WalkStarted, "", c.now(), 600, novox("mesh-tools", "node-tools"))
|
||||||
|
mod := cond("module.node-tools.novox.unhealthy", Urgent, "the tool runner on novox does not answer")
|
||||||
|
mod.Machine, mod.Raised = "novox", c.now()
|
||||||
|
h.Condition(EventRaised, mod)
|
||||||
|
machine := cond("machine.novox.claude-code.agent-home.data-shrank", Urgent, "the agent's home shrank")
|
||||||
|
machine.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, machine)
|
||||||
|
if _, err := h.Notify("tools.slow", Urgent, "tools answer slowly", "node-tools", false, "tools slow",
|
||||||
|
"Needs you: look at the tools."); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
if saidOn(tg, dt) != 0 {
|
||||||
|
t.Fatalf("held messages were said: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
for _, key := range []string{mod.Key, machine.Key, "notify.tools.slow"} {
|
||||||
|
if !has(outcomes(h, key), "held") {
|
||||||
|
t.Errorf("the history does not say %s was held: %v", key, outcomes(h, key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(h.Open()) != 3 {
|
||||||
|
t.Fatalf("held messages are not kept open: %+v", h.Open())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §2: a message touched by two overlapping walks is held until the second of them ends; the first's end says
|
||||||
|
// nothing of it.
|
||||||
|
func TestAMessageTouchedByTwoOverlappingWalksIsHeldUntilTheSecondEnds(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
c.pass(time.Minute)
|
||||||
|
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"), Touch{Machine: "ace", Modules: []string{"node-tools"}})
|
||||||
|
k := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0, novox("mesh-tools"))
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if heard(headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("said at the first walk's end, while the second still holds it: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
if !heard("Delivered: mesh-tools on novox", tg, dt) {
|
||||||
|
t.Fatalf("the first walk's end was not told: %+v", dt.sends)
|
||||||
|
}
|
||||||
|
walk(t, h, "plan-b", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("the second walk's end did not tell it: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §3: an ask, a secret given, a message about a machine no open walk touches and a walk's own failure are
|
||||||
|
// sent at once, while a walk touches the machine they are on.
|
||||||
|
func TestWhatAWalkNeverHoldsIsSentAtOnce(t *testing.T) {
|
||||||
|
h, tg, dt, c, _, _ := newRouter(t)
|
||||||
|
walk(t, h, "plan-9", WalkStarted, "", c.now(), 1200, novox("telegram", "mesh-tools"))
|
||||||
|
for _, key := range []string{
|
||||||
|
"machine.novox.telegram.telegram-token.secret-given", // a secret given, on a touched machine
|
||||||
|
"machine.novox.mounts.smb.secret-replaced", // a secret replaced
|
||||||
|
"machine.ace.silent", // a machine no walk touches
|
||||||
|
"delivery.leaf.over-budget", // the walk's own delivery
|
||||||
|
"plan.plan-9.failed", // the walk itself
|
||||||
|
} {
|
||||||
|
k := cond(key, Urgent, "it happened")
|
||||||
|
k.Raised = c.now()
|
||||||
|
if !strings.HasPrefix(key, "machine.") {
|
||||||
|
k.Machine = "novox" // a walk's own failure, on a machine it touches
|
||||||
|
}
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
if !heard(headlineOf(key), tg, dt) {
|
||||||
|
t.Errorf("%s was not sent at once during the walk: %v", key, outcomes(h, key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// An ask's own message, and an ask about a held condition: both offered at once.
|
||||||
|
submit(t, h, ConditionOwner, ackAsk(c, "own"))
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !has(outcomes(h, "ask."+ConditionOwner+".own"), "sent") {
|
||||||
|
t.Errorf("an ask waiting for the operator was held: %v", outcomes(h, "ask."+ConditionOwner+".own"))
|
||||||
|
}
|
||||||
|
held := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||||
|
held.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, held)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if heard(headlineOf(held.Key), tg, dt) {
|
||||||
|
t.Fatalf("a condition about a touched machine was not held")
|
||||||
|
}
|
||||||
|
about := ackAsk(c, "about")
|
||||||
|
about.About = held.Key
|
||||||
|
submit(t, h, ConditionOwner, about)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !heard(headlineOf(held.Key), dt) {
|
||||||
|
t.Fatalf("the ask about a held condition was held: %v", outcomes(h, held.Key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §4: a walk's end is one message — its outcome, what it reached and how long it took — then each held message
|
||||||
|
// still open, one line each; the one that cleared is only counted, and nothing more is said of them after.
|
||||||
|
func TestAWalksEndIsToldOnceWithWhatIsStillOpen(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-3", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||||
|
still := cond("machine.novox.tools-silent", Warning, "the tools on novox are silent")
|
||||||
|
still.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, still)
|
||||||
|
gone := cond("module.mesh-tools.novox.unhealthy", Warning, "mesh-tools is unhealthy on novox")
|
||||||
|
gone.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, gone)
|
||||||
|
c.pass(time.Second)
|
||||||
|
other := cond("machine.novox.core-behind", Urgent, "novox runs an older core")
|
||||||
|
other.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, other)
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
h.Condition(EventCleared, Condition{Key: gone.Key, At: c.now()})
|
||||||
|
runFor(h, c, 5*time.Minute)
|
||||||
|
if saidOn(tg, dt) != 0 {
|
||||||
|
t.Fatalf("said during the walk: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
walk(t, h, "plan-3", WalkEnded, WalkFailed, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
// Urgent among what it holds: on every kind, one message each.
|
||||||
|
if len(dt.sends) != 1 || len(tg.sends) != 1 {
|
||||||
|
t.Fatalf("the end was not one message per kind: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
m := dt.sends[0]
|
||||||
|
if m.Title != "Delivery failed: mesh-tools on novox" || !strings.Contains(m.Body, "It took 7 min.") {
|
||||||
|
t.Fatalf("the end does not say its outcome, what it reached and how long: %q", m.Text())
|
||||||
|
}
|
||||||
|
want := "Still open after it:\n- " + headlineOf(still.Key) + "\n- Urgent: " + headlineOf(other.Key) +
|
||||||
|
"\n1 problem it raised resolved during it."
|
||||||
|
if !strings.Contains(m.Body, want) || strings.Contains(m.Text(), headlineOf(gone.Key)) {
|
||||||
|
t.Fatalf("the end message %q does not say %q", m.Body, want)
|
||||||
|
}
|
||||||
|
keyless(t, m)
|
||||||
|
runFor(h, c, 20*time.Minute)
|
||||||
|
if len(dt.sends) != 1 || len(tg.sends) != 1 || len(dt.replaced)+len(tg.replaced) != 0 {
|
||||||
|
t.Fatalf("what the end told was said again: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §4: a walk that delivered with nothing held and nothing open is one line, and nothing at all on a kind set
|
||||||
|
// to say only what needs the operator.
|
||||||
|
func TestADeliveredWalkWithNothingHeldIsOneLineOrNothing(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-4", WalkStarted, "", c.now(), 600, novox("mesh-tools"), Touch{Machine: "ace", Modules: []string{"mesh-tools"}})
|
||||||
|
runFor(h, c, 3*time.Minute)
|
||||||
|
walk(t, h, "plan-4", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if len(dt.sends) != 1 || len(tg.sends) != 0 {
|
||||||
|
t.Fatalf("desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
if m := dt.sends[0]; m.Title != "Delivered: mesh-tools on ace, novox, in 3 min" || m.Body != "" {
|
||||||
|
t.Fatalf("not one line: %q", m.Text())
|
||||||
|
}
|
||||||
|
h.NeedsOnly = func(kind string) bool { return kind == "desktop" }
|
||||||
|
walk(t, h, "plan-5", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||||
|
runFor(h, c, 3*time.Minute)
|
||||||
|
walk(t, h, "plan-5", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if len(dt.sends) != 1 || len(tg.sends) != 0 {
|
||||||
|
t.Fatalf("told on a kind that says only what needs the operator: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
if !has(outcomes(h, "walk.plan-5"), "quiet: this kind says only what needs the operator") {
|
||||||
|
t.Fatalf("the history does not say why nothing was told: %v", outcomes(h, "walk.plan-5"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §5: the bound is the controller's (bound_seconds: twice the largest budget of the walk's deliveries; none
|
||||||
|
// given, 15 minutes). At it, what is held and still open is sent with the still-running line, and what
|
||||||
|
// cleared is counted; before it, nothing; after it, the walk holds nothing more.
|
||||||
|
func TestAHoldEndsAtTheWalksBound(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
given int
|
||||||
|
bound time.Duration
|
||||||
|
}{
|
||||||
|
{"core: twice ten minutes", 1200, 20 * time.Minute},
|
||||||
|
{"leaf at five minutes", 600, 10 * time.Minute},
|
||||||
|
{"leaf at three minutes", 360, 6 * time.Minute},
|
||||||
|
{"no budget", 0, 15 * time.Minute},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
started := c.now()
|
||||||
|
walk(t, h, "plan-7", WalkStarted, "", started, tc.given, novox("node-tools"))
|
||||||
|
open := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
open.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, open)
|
||||||
|
gone := cond("module.node-tools.novox.unhealthy", Urgent, "node-tools is unhealthy")
|
||||||
|
gone.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, gone)
|
||||||
|
c.pass(30 * time.Second)
|
||||||
|
h.Condition(EventCleared, Condition{Key: gone.Key, At: c.now()})
|
||||||
|
runFor(h, c, tc.bound-time.Minute-30*time.Second)
|
||||||
|
if saidOn(tg, dt) != 0 {
|
||||||
|
t.Fatalf("said before the bound: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
if len(dt.sends) != 1 || len(tg.sends) != 1 {
|
||||||
|
t.Fatalf("at the bound: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
m := dt.sends[0]
|
||||||
|
since := started.In(h.zone()).Format("15:04")
|
||||||
|
for _, want := range []string{"is still running, since " + since + ".", "- Urgent: " + headlineOf(open.Key),
|
||||||
|
"1 problem it raised resolved during it."} {
|
||||||
|
if !strings.Contains(m.Body, want) {
|
||||||
|
t.Fatalf("the bound's message %q does not say %q", m.Text(), want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Past its bound the walk holds nothing: a new message about what it touches goes at once.
|
||||||
|
late := cond("machine.novox.disk", Urgent, "a disk fails on novox")
|
||||||
|
late.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, late)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !heard(headlineOf(late.Key), dt) {
|
||||||
|
t.Fatalf("held past the bound: %v", outcomes(h, late.Key))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §2, kept: what a walk holds survives the router's restart, and is told at the walk's end.
|
||||||
|
func TestHeldMessagesSurviveARouterRestart(t *testing.T) {
|
||||||
|
h, tg, dt, c, st := newHolder(t)
|
||||||
|
walk(t, h, "plan-8", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||||
|
if err := again.Load(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again.Start()
|
||||||
|
c.pass(2 * time.Minute)
|
||||||
|
walk(t, again, "plan-8", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(again, c, time.Minute)
|
||||||
|
if !heard("- Urgent: "+headlineOf(k.Key), dt) || saidOn(tg, dt) != 2 {
|
||||||
|
t.Fatalf("what was held before the restart was not told at the walk's end: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// §6, issue 531: the desk's notifiers on two machines start again a second apart, with four silenced messages
|
||||||
|
// shown there — one of them withheld for its content, a file path — and a fifth open, asked there, and
|
||||||
|
// withheld. Nothing is said again: no silenced, no withheld message; the pass runs once; and a router that
|
||||||
|
// starts again reads it back from its state and makes it no more.
|
||||||
|
func TestARestartSaysNothingItHasSaid(t *testing.T) {
|
||||||
|
h, tg, dt, c, _, _ := newRouter(t)
|
||||||
|
h.Start()
|
||||||
|
c.pass(2 * time.Minute)
|
||||||
|
keys := []string{"machine.novox.telegram.telegram-token.secret-given", "machine.novox.claude-code.agent-home.data-shrank",
|
||||||
|
"machine.shanks.mounts.smb-password-recalbox.secret-given", "machine.ace.nzbgeek.api-key.secret-given"}
|
||||||
|
var silenced []Condition
|
||||||
|
for i, key := range keys {
|
||||||
|
k := cond(key, Urgent, "it happened")
|
||||||
|
if i == 1 {
|
||||||
|
k.Explanation = "Needs you: look at /var/lib/agent-home, which shrank."
|
||||||
|
}
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
silenced = append(silenced, k)
|
||||||
|
}
|
||||||
|
withheldOpen := cond("machine.novox.units", Urgent, "2 failed services")
|
||||||
|
withheldOpen.Explanation = "Needs you: read /etc/systemd/system/x.service."
|
||||||
|
withheldOpen.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, withheldOpen)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
ask := ackAsk(c, "units")
|
||||||
|
ask.About = withheldOpen.Key
|
||||||
|
submit(t, h, ConditionOwner, ask)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
for _, k := range silenced {
|
||||||
|
k.SilencedTill, k.SilencedWhy, k.At = c.now().Add(7*24*time.Hour), "known", c.now()
|
||||||
|
h.Condition(EventChanged, k)
|
||||||
|
}
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
for _, key := range []string{silenced[1].Key, withheldOpen.Key} {
|
||||||
|
if r := h.open[key]; r == nil || r.Refused == "" {
|
||||||
|
t.Fatalf("the test's %s was not withheld for its content: %+v", key, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sends, edits, replaced, tgSaid := len(dt.sends), len(dt.edits), len(dt.replaced), saidOn(tg)
|
||||||
|
dt.edits = map[string]Message{}
|
||||||
|
passes := func(h *Holder) int { return len(outcomes(h, "(answers lost)")) }
|
||||||
|
// The notifiers on two machines started again a second apart: two standings, two times.
|
||||||
|
c.pass(time.Minute)
|
||||||
|
first := c.now()
|
||||||
|
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: first})
|
||||||
|
c.pass(time.Second)
|
||||||
|
h.Flush() // the runtime's tick, between the two standings
|
||||||
|
second := c.now()
|
||||||
|
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: second})
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if len(dt.sends) != sends || len(dt.edits) != 0 || len(dt.replaced) != replaced || saidOn(tg) != tgSaid {
|
||||||
|
t.Fatalf("the restart said again: sends %+v, edits %+v (was %d), replaced %+v", dt.sends[sends:], dt.edits, edits,
|
||||||
|
dt.replaced[replaced:])
|
||||||
|
}
|
||||||
|
if passes(h) != 1 {
|
||||||
|
t.Fatalf("the pass ran %d times for one restart", passes(h))
|
||||||
|
}
|
||||||
|
// The router starts again; the desk repeats its standing.
|
||||||
|
again := &Holder{Away: tg, Desk: dt, Store: h.Store, Asks: h.Asks, Bench: h.Bench, Publish: h.Publish, Now: c.now,
|
||||||
|
Logf: t.Logf, Zone: h.Zone, RootFree: h.RootFree, Node: h.Node}
|
||||||
|
if err := again.Load(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again.Start()
|
||||||
|
again.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: second})
|
||||||
|
runFor(again, c, 2*time.Minute)
|
||||||
|
if len(dt.sends) != sends || len(dt.edits) != 0 || len(dt.replaced) != replaced || saidOn(tg) != tgSaid || passes(again) != 1 {
|
||||||
|
t.Fatalf("the router's restart made the pass again: %d passes, sends %+v, edits %+v", passes(again),
|
||||||
|
dt.sends[sends:], dt.edits)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The controller's walk events are read as ADR 0308 §1 says them; any other plan-moved is not a walk's.
|
||||||
|
func TestTheWalkEventsAreReadAsTheControllerSaysThem(t *testing.T) {
|
||||||
|
at := time.Date(2026, 10, 12, 11, 12, 0, 0, time.UTC)
|
||||||
|
ev, ok, err := DecodeWalk(walkBody(t, "plan-1", WalkStarted, "", at, 1200, novox("mesh-tools", "node-tools")))
|
||||||
|
if err != nil || !ok || ev.Walk != "plan-1" || ev.BoundSeconds != 1200 || ev.Class != "leaf" || !ev.At.Equal(at) ||
|
||||||
|
len(ev.Touches) != 1 || ev.Touches[0].Machine != "novox" || len(ev.Touches[0].Modules) != 2 {
|
||||||
|
t.Fatalf("%+v %v %v", ev, ok, err)
|
||||||
|
}
|
||||||
|
// As mesh-controller's walk events (its pull request 247) say an end: ended_at, and carried beside modules.
|
||||||
|
end, ok, err := DecodeWalk([]byte(`{"id":"plan-1","walk":"plan-1","phase":"ended","outcome":"stopped",` +
|
||||||
|
`"touches":[{"machine":"novox","modules":["node-tools","messenger"],"carried":["messenger"]}],` +
|
||||||
|
`"started_at":"2026-10-12T11:12:00Z","ended_at":"2026-10-12T11:19:00Z","updated":"2026-10-12T11:19:01Z"}`))
|
||||||
|
if err != nil || !ok || end.Outcome != WalkStopped || !end.At.Equal(at.Add(7*time.Minute)) || len(end.Touches[0].Modules) != 2 {
|
||||||
|
t.Fatalf("%+v %v %v", end, ok, err)
|
||||||
|
}
|
||||||
|
if _, ok, err := DecodeWalk([]byte(`{"id":"plan-1","state":"walking","tier":1}`)); ok || err != nil {
|
||||||
|
t.Fatalf("a move inside a walk read as its start or end: %v %v", ok, err)
|
||||||
|
}
|
||||||
|
if _, _, err := DecodeWalk([]byte(`{"walk":"plan-1","phase":"ended"}`)); err == nil {
|
||||||
|
t.Fatalf("an end without its outcome was read")
|
||||||
|
}
|
||||||
|
w := Walk{}
|
||||||
|
if w.bound() != DefaultWalkBound {
|
||||||
|
t.Fatalf("a walk without a bound is bounded at %s", w.bound())
|
||||||
|
}
|
||||||
|
// A start older than its bound is history: nothing is held.
|
||||||
|
h, _, _, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-old", WalkStarted, "", c.now().Add(-time.Hour), 1200, novox("mesh-tools"))
|
||||||
|
if len(h.walks) != 0 {
|
||||||
|
t.Fatalf("an old start opened a walk: %+v", h.walks)
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, 1: no channel can send at a walk's end or bound. What it held is not lost with the
|
||||||
|
// walk: it is unsent, and said on its own once a channel can send.
|
||||||
|
func TestWhatAWalkHeldIsSaidOnceAChannelCanSendAgain(t *testing.T) {
|
||||||
|
for _, at := range []string{"end", "bound"} {
|
||||||
|
t.Run(at, func(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-c", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
down := errors.New("no session")
|
||||||
|
tg.notReady, dt.notReady = down, down
|
||||||
|
if at == "end" {
|
||||||
|
walk(t, h, "plan-c", WalkEnded, WalkFailed, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
} else {
|
||||||
|
runFor(h, c, 10*time.Minute)
|
||||||
|
}
|
||||||
|
if saidOn(tg, dt) != 0 {
|
||||||
|
t.Fatalf("said on a channel that cannot send: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
if r := h.open[k.Key]; r == nil || r.Pending == "" {
|
||||||
|
t.Fatalf("what the walk held is not left unsent: %+v", r)
|
||||||
|
}
|
||||||
|
tg.notReady, dt.notReady = nil, nil
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
if !heard(headlineOf(k.Key), dt) {
|
||||||
|
t.Fatalf("what the walk held was never said: %v", outcomes(h, k.Key))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, 2: a message held by walk A, about a machine walk B also touches, is held on at
|
||||||
|
// A's end while B — started after it was held — is still open, and told at B's end.
|
||||||
|
func TestAMessageIsHeldOnByALaterWalkThatTouchesIt(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"))
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if heard(headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("told at the first walk's end while the second touches it: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
if got := h.heldBy(k.Key, c.now()); len(got) != 1 || got[0] != "plan-b" {
|
||||||
|
t.Fatalf("the second walk does not hold it on: %v", got)
|
||||||
|
}
|
||||||
|
walk(t, h, "plan-b", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("the second walk's end did not tell it: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, 3: a router back after more than a day reads back a walk whose end never came;
|
||||||
|
// what it held is said, with the still-running line, before the walk is forgotten.
|
||||||
|
func TestAWalkReadBackADayLateSaysWhatItHeldBeforeItIsForgotten(t *testing.T) {
|
||||||
|
h, tg, dt, c, st := newHolder(t)
|
||||||
|
walk(t, h, "plan-d", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
c.pass(25 * time.Hour)
|
||||||
|
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||||
|
if err := again.Load(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again.Start()
|
||||||
|
runFor(again, c, 2*time.Minute)
|
||||||
|
if !heard("is still running", dt) || !heard("- Urgent: "+headlineOf(k.Key), dt) {
|
||||||
|
t.Fatalf("what the walk held was forgotten unsaid: desk %+v, %v", dt.sends, outcomes(again, k.Key))
|
||||||
|
}
|
||||||
|
if len(again.walks) != 0 {
|
||||||
|
t.Fatalf("the walk was not forgotten: %+v", again.walks)
|
||||||
|
}
|
||||||
|
if ws, _ := st.Walks(); len(ws) != 0 {
|
||||||
|
t.Fatalf("the walk is still kept: %+v", ws)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, 4 (and of mesh-controller pull request 247): a short walk's end can come before
|
||||||
|
// its start. The start of a walk already ended opens nothing — also after the router's restart — while the
|
||||||
|
// next walk under the same id, started after that end, does.
|
||||||
|
func TestAStartHeardAfterItsEndOpensNothing(t *testing.T) {
|
||||||
|
h, tg, dt, c, st := newHolder(t)
|
||||||
|
started := c.now()
|
||||||
|
c.pass(20 * time.Second)
|
||||||
|
end, _, err := DecodeWalk([]byte(`{"walk":"plan-e","phase":"ended","outcome":"delivered","touches":[{"machine":"novox","modules":["mesh-tools"]}],` +
|
||||||
|
`"started_at":"` + started.Format(time.RFC3339Nano) + `","ended_at":"` + c.now().Format(time.RFC3339Nano) + `"}`))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h.Walked(end)
|
||||||
|
c.pass(time.Second)
|
||||||
|
walk(t, h, "plan-e", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||||
|
if len(h.walks) != 0 {
|
||||||
|
t.Fatalf("a start heard after its end opened a walk: %+v", h.walks)
|
||||||
|
}
|
||||||
|
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||||
|
if err := again.Load(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again.Start()
|
||||||
|
walk(t, again, "plan-e", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||||
|
if len(again.walks) != 0 {
|
||||||
|
t.Fatalf("after a restart, a start heard after its end opened a walk: %+v", again.walks)
|
||||||
|
}
|
||||||
|
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
again.Condition(EventRaised, k)
|
||||||
|
runFor(again, c, time.Minute)
|
||||||
|
if !heard(headlineOf(k.Key), dt) {
|
||||||
|
t.Fatalf("held by a walk that had ended: %v", outcomes(again, k.Key))
|
||||||
|
}
|
||||||
|
// Retried: the same id walks again, started after the end.
|
||||||
|
walk(t, again, "plan-e", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||||
|
if len(again.walks) != 1 {
|
||||||
|
t.Fatalf("a walk started after the last end did not open: %+v", again.walks)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, non-blocking: a message whose answers became asked while a walk held it is never
|
||||||
|
// held (ADR 0308 §3); when the desk's answers are lost, it is said again in place like any other, so its
|
||||||
|
// buttons work again.
|
||||||
|
func TestARestartSaysAgainAnAskedMessageAWalkTookBeforeItsAsk(t *testing.T) {
|
||||||
|
h, _, dt, c, _, _ := newRouter(t)
|
||||||
|
h.Start()
|
||||||
|
stand(h, c)
|
||||||
|
walk(t, h, "plan-f", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
ask := ackAsk(c, "units")
|
||||||
|
ask.About = k.Key
|
||||||
|
submit(t, h, ConditionOwner, ask)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
if !heard(headlineOf(k.Key), dt) {
|
||||||
|
t.Fatalf("the ask about it was not offered: %v", outcomes(h, k.Key))
|
||||||
|
}
|
||||||
|
replaced := len(dt.replaced)
|
||||||
|
c.pass(time.Minute)
|
||||||
|
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: c.now()})
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if len(dt.replaced) != replaced+1 {
|
||||||
|
t.Fatalf("its answers were not made to work again: replaced %+v, %v", dt.replaced[replaced:], outcomes(h, "(answers lost)"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Review of pull request 230, non-blocking: a walk's end that cannot be said as it is (here a module named by a
|
||||||
|
// file path) is said in its plain form, and what it held, still open, is said on its own, not marked told.
|
||||||
|
func TestWhatAWalksPlainEndCannotCarryIsSaidOnItsOwn(t *testing.T) {
|
||||||
|
h, tg, dt, c, _ := newHolder(t)
|
||||||
|
walk(t, h, "plan-g", WalkStarted, "", c.now(), 600, novox("/var/lib/mesh-tools"))
|
||||||
|
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
walk(t, h, "plan-g", WalkEnded, WalkFailed, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if !heard("A delivery ended", dt) {
|
||||||
|
t.Fatalf("the plain end was not said: %+v", dt.sends)
|
||||||
|
}
|
||||||
|
if !heard(headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("what the walk held was marked told and never said: %v", outcomes(h, k.Key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Issue 538 A: what a walk held is marked told only once the message carrying it was sent. When that message
|
||||||
|
// could not be sent and the router restarts before its retry, the message is gone with it: what it held is not
|
||||||
|
// marked told, it stays owed in the state, and the restarted router says it — an urgent one and a warning alike,
|
||||||
|
// once each, never left as history.
|
||||||
|
func TestAHeldMessageIsToldOnlyOnceTheWalksMessageWasSent(t *testing.T) {
|
||||||
|
for _, then := range []string{"sent", "restart urgent", "restart warning"} {
|
||||||
|
t.Run(then, func(t *testing.T) {
|
||||||
|
h, tg, dt, c, st := newHolder(t)
|
||||||
|
sev := Urgent
|
||||||
|
if then == "restart warning" {
|
||||||
|
sev = Warning
|
||||||
|
}
|
||||||
|
walk(t, h, "plan-h", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
k := cond("machine.novox.tools-silent", sev, "the tools on novox are silent")
|
||||||
|
k.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, k)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
down := errors.New("cannot connect")
|
||||||
|
tg.fail, dt.fail = down, down
|
||||||
|
walk(t, h, "plan-h", WalkEnded, WalkFailed, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
if saidOn(tg, dt) != 0 {
|
||||||
|
t.Fatalf("said on channels that cannot send: %+v %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
if r := h.open[k.Key]; r == nil || r.told() {
|
||||||
|
t.Fatalf("marked told while the message carrying it was not sent: %+v", r)
|
||||||
|
}
|
||||||
|
tg.fail, dt.fail = nil, nil
|
||||||
|
if then == "sent" {
|
||||||
|
runFor(h, c, 2*time.Minute)
|
||||||
|
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) || saidOn(tg, dt) != 2 {
|
||||||
|
t.Fatalf("not told once in the walk's message: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
if r := h.open[k.Key]; r == nil || !r.told() || r.Owed != "" || r.Sent["telegram"] != "" || r.Sent["desktop"] != "" {
|
||||||
|
t.Fatalf("not marked told on both channels once sent: %+v", r)
|
||||||
|
}
|
||||||
|
runFor(h, c, 10*time.Minute)
|
||||||
|
if saidOn(tg, dt) != 2 {
|
||||||
|
t.Fatalf("said again after the walk's message: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||||
|
if err := again.Load(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
again.Start()
|
||||||
|
again.Sync([]Condition{k}, c.now())
|
||||||
|
runFor(again, c, 5*time.Minute)
|
||||||
|
if !heard(headlineOf(k.Key), tg, dt) {
|
||||||
|
t.Fatalf("what the lost message carried was not said after the restart: %v", outcomes(again, k.Key))
|
||||||
|
}
|
||||||
|
said := saidOn(tg, dt)
|
||||||
|
if r := again.open[k.Key]; r == nil || !r.told() || r.Owed != "" || r.Pending != "" {
|
||||||
|
t.Fatalf("said, and still owed or not marked told: %+v", r)
|
||||||
|
}
|
||||||
|
runFor(again, c, 20*time.Minute) // within its reminder's bound: nothing more is due
|
||||||
|
again.Sync([]Condition{k}, c.now())
|
||||||
|
runFor(again, c, time.Minute)
|
||||||
|
if saidOn(tg, dt) != said {
|
||||||
|
t.Fatalf("said again: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Issue 538 B: at a walk's end, a later walk that touches what it held takes over only what a first hold would
|
||||||
|
// take: not a message silenced meanwhile, nor one an ask waiting for the operator is about (ADR 0308 §3).
|
||||||
|
func TestAWalksEndHandsOnOnlyWhatALaterWalkWouldHold(t *testing.T) {
|
||||||
|
h, tg, dt, c, _, _ := newRouter(t)
|
||||||
|
h.Start()
|
||||||
|
stand(h, c)
|
||||||
|
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||||
|
quiet := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||||
|
quiet.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, quiet)
|
||||||
|
asked := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||||
|
asked.Raised = c.now()
|
||||||
|
h.Condition(EventRaised, asked)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
if heard(headlineOf(quiet.Key), tg, dt) || heard(headlineOf(asked.Key), tg, dt) {
|
||||||
|
t.Fatalf("the test's messages were not held")
|
||||||
|
}
|
||||||
|
quiet.SilencedTill, quiet.SilencedWhy, quiet.At = c.now().Add(7*24*time.Hour), "known", c.now()
|
||||||
|
h.Condition(EventChanged, quiet)
|
||||||
|
ask := ackAsk(c, "units")
|
||||||
|
ask.About = asked.Key
|
||||||
|
submit(t, h, ConditionOwner, ask)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"))
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
stand(h, c)
|
||||||
|
helds := func(key string) int {
|
||||||
|
n := 0
|
||||||
|
for _, o := range outcomes(h, key) {
|
||||||
|
if o == "held" {
|
||||||
|
n++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
before := map[string]int{quiet.Key: helds(quiet.Key), asked.Key: helds(asked.Key)}
|
||||||
|
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0)
|
||||||
|
runFor(h, c, time.Minute)
|
||||||
|
for _, key := range []string{quiet.Key, asked.Key} {
|
||||||
|
if got := h.heldBy(key, c.now()); len(got) != 0 {
|
||||||
|
t.Errorf("%s was handed on to %v, which a first hold would never hold", key, got)
|
||||||
|
}
|
||||||
|
if helds(key) != before[key] {
|
||||||
|
t.Errorf("the history says %s held again: %v", key, outcomes(h, key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Issue 538 C: the controller's clock ahead of the router's. A short walk's end, heard before its start, opens
|
||||||
|
// nothing when its start comes, whether the end said the walk's start or only its own moment; the next walk
|
||||||
|
// under its id, started after that end, still opens.
|
||||||
|
func TestAStartHeardAfterItsEndOpensNothingWhenTheControllersClockIsAhead(t *testing.T) {
|
||||||
|
for _, said := range []string{"both", "end only"} {
|
||||||
|
t.Run(said, func(t *testing.T) {
|
||||||
|
h, _, _, c, _ := newHolder(t)
|
||||||
|
ended := c.now().Add(10 * time.Minute) // by the controller's clock
|
||||||
|
started := ended.Add(-20 * time.Second)
|
||||||
|
body := `{"walk":"plan-k","phase":"ended","outcome":"delivered","touches":[{"machine":"novox","modules":["mesh-tools"]}],` +
|
||||||
|
`"ended_at":"` + ended.Format(time.RFC3339Nano) + `"`
|
||||||
|
if said == "both" {
|
||||||
|
body += `,"started_at":"` + started.Format(time.RFC3339Nano) + `"`
|
||||||
|
}
|
||||||
|
end, _, err := DecodeWalk([]byte(body + `}`))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h.Walked(end)
|
||||||
|
c.pass(time.Second)
|
||||||
|
walk(t, h, "plan-k", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||||
|
if len(h.walks) != 0 {
|
||||||
|
t.Fatalf("a start heard after its end opened a walk: %+v", h.walks)
|
||||||
|
}
|
||||||
|
walk(t, h, "plan-k", WalkStarted, "", ended.Add(time.Minute), 600, novox("mesh-tools"))
|
||||||
|
if len(h.walks) != 1 {
|
||||||
|
t.Fatalf("the next walk under its id, started after the end, did not open: %+v", h.walks)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -73,6 +73,7 @@
|
|||||||
"mesh-controller.condition-raised",
|
"mesh-controller.condition-raised",
|
||||||
"mesh-controller.condition-changed",
|
"mesh-controller.condition-changed",
|
||||||
"mesh-controller.condition-cleared",
|
"mesh-controller.condition-cleared",
|
||||||
|
"mesh-controller.plan-moved",
|
||||||
"intake.choice",
|
"intake.choice",
|
||||||
"intake.link",
|
"intake.link",
|
||||||
"intake.failed",
|
"intake.failed",
|
||||||
@@ -93,7 +94,8 @@
|
|||||||
"name": "asks",
|
"name": "asks",
|
||||||
"ttl-seconds": 2592000
|
"ttl-seconds": 2592000
|
||||||
},
|
},
|
||||||
"identities"
|
"identities",
|
||||||
|
"walks"
|
||||||
],
|
],
|
||||||
"own-secrets": {
|
"own-secrets": {
|
||||||
"broker": "${dir:state}/broker"
|
"broker": "${dir:state}/broker"
|
||||||
@@ -129,7 +131,7 @@
|
|||||||
"path": "${dir:state}/settings.json",
|
"path": "${dir:state}/settings.json",
|
||||||
"mode": "0600",
|
"mode": "0600",
|
||||||
"merge": "json",
|
"merge": "json",
|
||||||
"content": "{\n \"time-zone\": \"\"\n}\n",
|
"content": "{\n \"time-zone\": \"\",\n \"only-what-needs-me\": []\n}\n",
|
||||||
"owner": "messenger"
|
"owner": "messenger"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ func limitOf(a map[string]any, fallback int) int {
|
|||||||
return fallback
|
return fallback
|
||||||
}
|
}
|
||||||
|
|
||||||
func tools(monitor Monitor, users func(context.Context) ([]byte, error)) []stdio.Tool {
|
func tools(monitor Monitor, users func(context.Context) ([]byte, error), written func(context.Context) (time.Time, error)) []stdio.Tool {
|
||||||
get := func(path string) ([]byte, error) {
|
get := func(path string) ([]byte, error) {
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
@@ -123,20 +123,28 @@ func tools(monitor Monitor, users func(context.Context) ([]byte, error)) []stdio
|
|||||||
return Users(c, arg(a, "user")), nil
|
return Users(c, arg(a, "user")), nil
|
||||||
}},
|
}},
|
||||||
{Name: "nats_user_can",
|
{Name: "nats_user_can",
|
||||||
Description: "Whether a bus user may publish or subscribe to a subject, and the grant that allows it — the question behind a request that timed out because the bus refused it.",
|
Description: "Whether a bus user may publish or subscribe to a subject, and the grant that allows it — the question behind a request that timed out because the bus refused it. Also whether the running server holds it: `in_force` is true only when the user list on the machine allows it and the server reloaded its configuration after that list was written (novox/hq issue 353: a list sent is not a list the bus enforces until it reloads).",
|
||||||
Input: map[string]any{"user": str("the user exactly, e.g. g14.node-tools"), "action": str("publish or subscribe"), "subject": str("a concrete subject, e.g. $KV.claude-code_servers.all.x")},
|
Input: map[string]any{"user": str("the user exactly, e.g. g14.node-tools"), "action": str("publish or subscribe"), "subject": str("a concrete subject, e.g. $KV.claude-code_servers.all.x")},
|
||||||
Run: func(a map[string]any) (any, error) {
|
Run: func(a map[string]any) (any, error) {
|
||||||
c, err := conf()
|
c, err := conf()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return UserCan(c, arg(a, "user"), arg(a, "action"), arg(a, "subject"))
|
answer, err := UserCan(c, arg(a, "user"), arg(a, "action"), arg(a, "subject"))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
at, werr := written(ctx)
|
||||||
|
varz, verr := get("/varz")
|
||||||
|
return InForce(answer, at, werr, varz, verr), nil
|
||||||
}},
|
}},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
if err := stdio.Serve("", tools(LiveMonitor(), ReadUsers)); err != nil {
|
if err := stdio.Serve("", tools(LiveMonitor(), ReadUsers, UsersWritten)); err != nil {
|
||||||
fmt.Fprintln(os.Stderr, err)
|
fmt.Fprintln(os.Stderr, err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -65,3 +65,24 @@ func ReadUsers(ctx context.Context) ([]byte, error) {
|
|||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UsersWritten is when the composed user list the server includes was last written, read in the container
|
||||||
|
// to the nanosecond (`stat -c %y`): what the server's last reload must come after for the list to be in force.
|
||||||
|
func UsersWritten(ctx context.Context) (time.Time, error) {
|
||||||
|
container := env("MESH_NATS_CONTAINER", "mesh-broker-nats")
|
||||||
|
path := env("MESH_NATS_USERS_FILE", "/etc/nats/accounts.conf")
|
||||||
|
out, err := exec.CommandContext(ctx, "docker", "exec", container, "stat", "-c", "%y", path).Output()
|
||||||
|
if err != nil {
|
||||||
|
return time.Time{}, fmt.Errorf("when the bus's user list was written could not be read inside %s: %w", container, err)
|
||||||
|
}
|
||||||
|
return ParseWritten(string(out))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseWritten reads `stat -c %y`: "2026-10-09 17:38:13.422211788 +0000".
|
||||||
|
func ParseWritten(s string) (time.Time, error) {
|
||||||
|
t, err := time.Parse("2006-01-02 15:04:05.999999999 -0700", strings.TrimSpace(s))
|
||||||
|
if err != nil {
|
||||||
|
return time.Time{}, fmt.Errorf("when the bus's user list was written is not a time: %q", strings.TrimSpace(s))
|
||||||
|
}
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -536,3 +536,39 @@ func UserCan(conf []byte, user, action, subject string) (map[string]any, error)
|
|||||||
}
|
}
|
||||||
return nil, fmt.Errorf("the bus has no user %q; nats_users lists them", user)
|
return nil, fmt.Errorf("the bus has no user %q; nats_users lists them", user)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// InForce adds to a UserCan answer whether the running server holds what the file says (novox/hq issue 353):
|
||||||
|
// `in_force` only when the grant is allowed in the file and the server's last configuration load
|
||||||
|
// (`config_load_time` of /varz) came after the file was last written. Anything not known is not in force, said
|
||||||
|
// in `why`: an asker waiting for its grant must not take a list sent for a list the bus enforces.
|
||||||
|
func InForce(answer map[string]any, written time.Time, writtenErr error, varz []byte, varzErr error) map[string]any {
|
||||||
|
answer["in_force"] = false
|
||||||
|
switch {
|
||||||
|
case writtenErr != nil:
|
||||||
|
answer["why"] = writtenErr.Error()
|
||||||
|
return answer
|
||||||
|
case varzErr != nil:
|
||||||
|
answer["why"] = "the server's last configuration load could not be read: " + varzErr.Error()
|
||||||
|
return answer
|
||||||
|
}
|
||||||
|
var v struct {
|
||||||
|
ConfigLoadTime time.Time `json:"config_load_time"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(varz, &v); err != nil || v.ConfigLoadTime.IsZero() {
|
||||||
|
answer["why"] = "the server does not say when it last loaded its configuration"
|
||||||
|
return answer
|
||||||
|
}
|
||||||
|
answer["file_written"] = written.UTC().Format(time.RFC3339Nano)
|
||||||
|
answer["server_loaded"] = v.ConfigLoadTime.UTC().Format(time.RFC3339Nano)
|
||||||
|
loaded := !v.ConfigLoadTime.Before(written)
|
||||||
|
allowed, _ := answer["allowed"].(bool)
|
||||||
|
switch {
|
||||||
|
case !allowed:
|
||||||
|
answer["why"] = "the user list on the machine does not allow it"
|
||||||
|
case !loaded:
|
||||||
|
answer["why"] = "the user list on the machine allows it, and the server has not reloaded since it was written"
|
||||||
|
default:
|
||||||
|
answer["in_force"] = true
|
||||||
|
}
|
||||||
|
return answer
|
||||||
|
}
|
||||||
|
|||||||
@@ -2,6 +2,8 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -170,3 +172,39 @@ func TestClosedConnectionsArePerUserAndNameAndSayWhichWereDropped(t *testing.T)
|
|||||||
t.Errorf("narrowed: %+v", got)
|
t.Errorf("narrowed: %+v", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// novox/hq issue 353: a grant written to the user list is in force only once the server reloaded after the
|
||||||
|
// write; anything not known is not in force.
|
||||||
|
func TestAGrantIsInForceOnlyWhenTheServerReloadedAfterTheListWasWritten(t *testing.T) {
|
||||||
|
written, err := ParseWritten("2026-10-09 17:55:56.422211788 +0200\n")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
varz := func(at string) []byte { return []byte(`{"config_load_time":"` + at + `"}`) }
|
||||||
|
allowed := func() map[string]any { return map[string]any{"allowed": true} }
|
||||||
|
cases := []struct {
|
||||||
|
name string
|
||||||
|
answer map[string]any
|
||||||
|
werr error
|
||||||
|
varz []byte
|
||||||
|
verr error
|
||||||
|
want bool
|
||||||
|
says string
|
||||||
|
}{
|
||||||
|
{"reloaded after the write", allowed(), nil, varz("2026-10-09T15:56:05.1Z"), nil, true, ""},
|
||||||
|
{"reloaded before the write", allowed(), nil, varz("2026-10-09T15:55:56.4Z"), nil, false, "has not reloaded"},
|
||||||
|
{"not allowed in the file", map[string]any{"allowed": false}, nil, varz("2026-10-09T15:56:05Z"), nil, false, "does not allow"},
|
||||||
|
{"the write time unknown", allowed(), errors.New("no container"), varz("2026-10-09T15:56:05Z"), nil, false, "no container"},
|
||||||
|
{"the server unknown", allowed(), nil, nil, errors.New("refused"), false, "could not be read"},
|
||||||
|
{"a server that says no load time", allowed(), nil, []byte(`{}`), nil, false, "does not say"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got := InForce(c.answer, written, c.werr, c.varz, c.verr)
|
||||||
|
if got["in_force"] != c.want || (c.says != "" && !strings.Contains(fmt.Sprint(got["why"]), c.says)) {
|
||||||
|
t.Errorf("%s: %v", c.name, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := ParseWritten("yesterday"); err == nil {
|
||||||
|
t.Error("a write time that is not one was read")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user