Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
46c46c955b | ||
|
|
1ca26c944a | ||
|
|
96e1792881 | ||
|
|
cb9b66e628 |
+147
-32
@@ -1,8 +1,10 @@
|
||||
# betterbird
|
||||
|
||||
Betterbird, the mail and calendar client of g14 and shanks (a Thunderbird fork, with the operator's
|
||||
Exchange calendar through the Owl add-on), as a module (novox/hq issue 345, ADR 0208). It requires
|
||||
`x11-display`, so it is assigned only where a display server is held on the same machine.
|
||||
Exchange calendar through the Owl add-on), as a module (novox/hq issue 345, ADR 0208, ADR 0316, ADR 0317,
|
||||
to-be 56). It requires `x11-display`, so it is assigned only where a display server is held on the same
|
||||
machine. It installs Betterbird from a pinned upstream release, ships Owl with it through Betterbird's
|
||||
enterprise policy, and declares the operator's mail accounts without their credentials.
|
||||
|
||||
It exists because a symptom had no tools: on a workstation, a calendar reminder appeared as a
|
||||
transparent window with nothing in it (2026-10-09). Nothing in the mesh could say which window that
|
||||
@@ -11,24 +13,57 @@ issue 345.
|
||||
|
||||
## Owns
|
||||
|
||||
| what | where |
|
||||
|---|---|
|
||||
| the module's directory | `~/.config/betterbird/` |
|
||||
| the module's `user.js`, filled from the setting `prefs` | `~/.config/betterbird/user.js` |
|
||||
| what | where | class |
|
||||
|---|---|---|
|
||||
| Betterbird, the upstream release (`archive`, pinned by URL and sha256) | `/opt/betterbird` | owned, whole |
|
||||
| the enterprise policy (`install/distribution/policies.json`) | `/opt/betterbird/distribution` | owned, whole |
|
||||
| Owl's xpi (`owl`, pinned by the vendor's sha256) | `/opt/betterbird/owl` | owned, whole |
|
||||
| the spelling dictionaries' default (`install/defaults-pref/vendor-prefs.js`) | `/opt/betterbird/defaults/pref` | owned: its one file |
|
||||
| the tools' program, for the start's preparation | `/usr/local/lib/mesh-betterbird` | owned, whole |
|
||||
| the start | `/usr/local/bin/betterbird` | owned, whole |
|
||||
| the launcher entry | `/usr/local/share/applications/eu.betterbird.Betterbird.desktop` | owned, whole |
|
||||
| the module's directory, with its `user.js` (setting `prefs`), `accounts.json` (setting `accounts`) and the record of adopted and created accounts | `~/.config/betterbird/` | owned, whole |
|
||||
| the profile's `user.js`, placed by the tools | the profile | the module's, or never touched |
|
||||
| the account lists in the profile's `prefs.js` | the profile | written into: the module's keys only |
|
||||
| the profile: mail, address books, calendars, logins | `~/.thunderbird/` | found (ADR 0182) |
|
||||
|
||||
Nothing else. It holds no seat and adds no start.
|
||||
**The community package `betterbird-bin` is declared absent** (ADR 0180), before anything is placed in
|
||||
`/opt/betterbird`: the node-engine removes it with `pacman -R`, then unpacks the release where the package
|
||||
had it. Betterbird chooses its profile by its installation's path, so the same profile opens after the
|
||||
switch. A package left in place (its removal failed) holds the path, and the release is refused there by
|
||||
the node-engine rather than mixed in. The module creates no link: the start, a launcher entry naming the
|
||||
icon in the installation, and the dictionaries as a default preference replace the package's links.
|
||||
|
||||
**Every new build waits for the operator** (`upgrade: record`): the first push to a node removes the
|
||||
package and replaces what a running Betterbird has open, so the operator quits Betterbird, then pushes.
|
||||
An upgrade is a change to the pins, reviewed and delivered like any other (ADR 0316 §5).
|
||||
|
||||
**The pins.** The release is `betterbird-153.4.0esr-bb10.en-US.linux-x86_64.tar.xz` (81,620,684 bytes) and
|
||||
Owl is 1.5.6.1 (430,201 bytes), each pinned by sha256 in `module.json`. The build seat fetches each once,
|
||||
refuses it when the digest differs, keeps it under its sha256, and packs it into the artifact store; a node
|
||||
receives the mesh's archive by digest and never asks the vendor. **Owl is never in this repository**: its
|
||||
licence forbids copying it to third parties, and its vendor serves only its newest version, at one
|
||||
address. Keeping the pinned copy in the mesh's private artifact store for the operator's own machines was
|
||||
the operator's call (2026-10-12, ADR 0316 §6).
|
||||
|
||||
**Owl's range.** Owl declares which Betterbird versions it supports (`strict_min_version` to
|
||||
`strict_max_version`: 115 to 156 for 1.5.6.1). `owlSupports` in `manifest_test.go` holds the range read
|
||||
from each pinned xpi, and the module's tests refuse a Betterbird pin outside it: moving Owl's pin means
|
||||
reading the new range there (ADR 0316 §8).
|
||||
|
||||
**The policy** turns Betterbird's own updater off (`DisableAppUpdate`, which Betterbird's update check
|
||||
honours) and installs Owl into every profile from `file:///opt/betterbird/owl/owl.xpi` as
|
||||
`force_installed`, with `updates_disabled`. A profile that already runs another Owl keeps it: the policy
|
||||
installs the pinned one where Owl is missing. A newer Owl in the profile (g14 runs 1.5.7) is said as a
|
||||
fact by `betterbird_status`; Owl installed by policy cannot be removed in Betterbird, so the remedy is
|
||||
moving the pin. An older one is a finding of `betterbird_check`, with the same remedy. **Owl's licence stays the person's**: the mesh holds no licence, key or payment (ADR 0316 §9).
|
||||
|
||||
**No start of its own.** Betterbird is started by the operator from the launcher. The start
|
||||
(`/usr/local/bin/betterbird`) first runs `betterbird-tools prepare`, which, with no Betterbird of the
|
||||
account running, places the user.js and writes the created accounts into the account lists; then it
|
||||
starts Betterbird either way. `betterbird_restart` starts it as the transient user unit
|
||||
`mesh-betterbird`, so its output then goes to the journal.
|
||||
|
||||
- **No package.** g14 and shanks both run `betterbird-bin` 153.1.0esr-1, installed explicitly from the
|
||||
AUR: `pacman` counts it as foreign. The node-engine installs packages from the official
|
||||
repositories only, so the module does not declare it, and **installing and upgrading Betterbird stay
|
||||
the operator's** (an AUR helper). `betterbird_status` says the version and that it is from the AUR;
|
||||
`betterbird_check` says when it is missing.
|
||||
- **No start.** Betterbird is started by the operator from the launcher (on the workstation measured,
|
||||
it runs in the login session's scope, started by hand, with no autostart entry and no line in i3's
|
||||
configuration). `betterbird_restart` starts it as the transient user unit `mesh-betterbird`, so its
|
||||
output then goes to the journal.
|
||||
- **The profile is Betterbird's** (ADR 0182): `~/.thunderbird/` holds the mail, the address books,
|
||||
the calendars' caches, the logins and the keys. The module declares none of it.
|
||||
|
||||
## The user.js: how a preference is declared
|
||||
|
||||
@@ -52,33 +87,104 @@ default, the one in use, is the other.
|
||||
what to do. The tools refuse a declared file that holds anything but comments and `user_pref`
|
||||
statements, or that sets a preference whose name can hold a credential.
|
||||
|
||||
The file is **trusted** (it carries no `"trusted": false`): a preference can weaken how the mail client
|
||||
checks a server, so the setting is changed only at the controller's terminal, never through a verb
|
||||
(novox/hq issue 339).
|
||||
The file is **trusted** (`"trusted": true`): a preference can weaken how the mail client checks a server,
|
||||
so the setting is changed only at the controller's terminal or proposed and set on the operator's warrant
|
||||
(novox/hq issue 339, ADR 0277). An account's preferences (`mail.account.`, `mail.identity.`, `mail.server.`,
|
||||
`mail.smtpserver.` and the account lists) are refused in `prefs`: an account is declared in `accounts`, so
|
||||
it has one owner (ADR 0316 §11).
|
||||
## The accounts: declared without credentials (ADR 0316 §10–§16, ADR 0317)
|
||||
|
||||
The setting `accounts` reaches the machine as `~/.config/betterbird/accounts.json`, a trusted mergeable
|
||||
file: a map of accounts by name, each with `address`, `from-name` and `label` (optional), `incoming` (`kind`
|
||||
imap or pop3, `host`, `port`, `security` tls, starttls or none, `sign-in` password, encrypted-password,
|
||||
oauth, kerberos, ntlm or tls-certificate, `user`) and `outgoing` (the same, without a kind; its sign-in may
|
||||
be none), and, only in a node's own layer, `key`. **Never a password or a token**: the whole file is
|
||||
refused when any key in it is shaped like one, and an account signs in at its first use, in Betterbird's
|
||||
own window, where Betterbird keeps the result in its login store. The work account through Owl is never
|
||||
declared: the person creates it in Owl's setup (ADR 0316 §13).
|
||||
|
||||
At every apply (the start from the launcher, `betterbird_user_js` with apply, `betterbird_restart`), each
|
||||
declared account, in its place in the setting, is (ADR 0317 §1):
|
||||
|
||||
1. **kept by its recorded keys**, when the module recorded it on this node for this profile;
|
||||
2. **its creation**, when it is under the module's own keys (`account_mesh_<name>`, `id_mesh_<name>`,
|
||||
`server_mesh_<name>`, `smtp_mesh_<name>`);
|
||||
3. **adopted by the key a node's layer names** (`key`), when that account is in the list, its default
|
||||
identity has the address and it is not Owl's;
|
||||
4. **adopted by its address**, when exactly one account's default identity has it: its keys (account,
|
||||
default identity, incoming server, and the outgoing server that identity names) become the module's, and
|
||||
the account lists are left as they are;
|
||||
5. **created** under the module's own keys when no account has it: its keys are added at the end of
|
||||
`mail.accountmanager.accounts` and `mail.smtpservers` in `prefs.js`, every other key kept in its place,
|
||||
only while Betterbird is not running.
|
||||
|
||||
**Refused, for that account alone and with the remedy** (§7): two accounts with the address (remove the
|
||||
duplicate in Betterbird, or name the `key`), Owl's account at the address, a named key that does not
|
||||
qualify, an account of another kind than declared (never converted), an identity that names no outgoing
|
||||
server, and an outgoing server that another identity, neither adopted nor the module's, also uses
|
||||
(Betterbird lets identities share one, and the declared values would change that other account's sending;
|
||||
ADR 0317 §2 leaves the identity's server and the lists as they are, so the module refuses rather than give
|
||||
it a server of its own). A recorded adoption follows the identity's current outgoing server at every apply,
|
||||
under the same rule. The other accounts go ahead; the start and `betterbird_check` say it.
|
||||
`betterbird_user_js` answers an account's preferences by name with their values withheld.
|
||||
|
||||
The declared fields are written in the user.js under the account's keys, so they win at every start; only
|
||||
the fields the setting names, never the account's directory, folders, filters, caches, mail or kept
|
||||
sign-in, so an adopted account keeps its sign-in (§5). The apply names, by preference name only, each one
|
||||
whose value it changed (§4). What was adopted or created is recorded in
|
||||
`~/.config/betterbird/accounts-record.json`. **Leaving the setting** releases an adopted account (its
|
||||
lines and its record go, the account stays the person's) and takes a created account's keys out of the
|
||||
lists (its mail stays in the profile).
|
||||
|
||||
**The value is derived, not retyped** (§9): `betterbird_accounts` with `propose` builds `accounts` from the
|
||||
profile (every account but Owl's, no credential) and proposes it through the controller's settings verb;
|
||||
the operator reads it whole at their desk (`settings proposals <id>`) and approves on a channel that proves
|
||||
who answers. Its answer to its caller holds counts and kinds only.
|
||||
|
||||
**A node with no profile yet.** The preparation needs the profile Betterbird makes at its first start: on a
|
||||
node set up from nothing, the declared accounts are written at the second start from the launcher (or at
|
||||
`betterbird_restart`).
|
||||
|
||||
|
||||
## Tools
|
||||
|
||||
They are served by the node's runtime as the operator account (ADR 0175). **No answer carries mail, a
|
||||
calendar entry, an account, an address or a credential.**
|
||||
calendar entry, an address, a host, an account's name or a credential**: accounts are said by count, place
|
||||
and key.
|
||||
|
||||
| tool | does |
|
||||
|---|---|
|
||||
| `betterbird_status` (r) | whether it runs (pid, since, scope, children), the package and version and that it is from the AUR, where its output goes, the profile and how it was found, the user.js (declared, placed, in effect), the add-ons the operator installed (ids), and the preferences that decide how its windows and notifications look |
|
||||
| `betterbird_status` (r) | whether it runs (pid, since, scope, children); what is installed and from where (the mesh's release, or the community package before the switch), its version and build beside the newest published build (Betterbird's own `versions.txt`); the policy; Owl's version against the pin and whether it is active; where its output goes; the profile and how it was found; the user.js (declared, placed, in effect); the declared accounts by count and, per place, adopted, created or refused with their keys; the add-ons in the profile (ids, versions); the preferences that decide how its windows and notifications look |
|
||||
| `betterbird_prefs` (r) | named preferences or prefixes (ending in `.` or `*`, at most 200; an empty name, `*` or `.` is refused): value and source (user.js, prefs.js or the default). Withheld without its value: a name that can hold a credential (`password`, `token`, `oauth`, `signon`, …); one that names an account, server, calendar, recent folder or person (`mail.server.`, `mail.identity.`, `mail.last_msg…`, `messenger.account.`, `mail.cloud_files.`, `calendar.registry.`, …); and every `extensions.*` but the theme's id (`extensions.activeThemeID`). In every value answered, the home is `~`, and a mail address, a `user%40domain@host` URI and a host name are masked |
|
||||
| `betterbird_user_js` (r; a with `apply`) | the declared user.js against the profile's; `apply` places it |
|
||||
| `betterbird_log` (r) | the last lines of the journal of `mesh-betterbird` (or of the reminder test), addresses and credentials masked (the value after `password=`, `token:` and the like, and the token after `Bearer`/`Basic`). A Betterbird the module did not start prints elsewhere, and the answer says where |
|
||||
| `betterbird_restart` (a) | ends Betterbird (asked to quit, forced after 8 s), places the user.js where allowed, starts it as `mesh-betterbird`. **Refused while a composer window is open** (X instance `Msgcompose`, whatever its class), because the unsent message would be lost, unless `discard_unsent` is true, which only the operator's yes justifies. It answers within the runtime's 30 s: a start not yet seen after 4 s is said in `note`, and `betterbird_status` shows it |
|
||||
| `betterbird_check` (r) | installed; one running; profile found; the user.js on the machine, valid, placed, in effect; its output read |
|
||||
| `betterbird_check` (r) | installed from the mesh's release and not behind the newest published build; the policy placed; Owl at its pin and active; one running; profile found; the user.js on the machine, valid, placed, in effect; the declared accounts in the list, none refused, each signed in once; its output read |
|
||||
| `betterbird_accounts` (r; a with `propose`) | the profile's accounts by count and kind (declarable, Owl's, others); with `propose`, the setting `accounts` built from them and proposed to the operator (ADR 0317 §9) |
|
||||
| `betterbird_reminder_test` (a) | a reminder on demand without the operator's calendar (below) |
|
||||
|
||||
### A reminder on demand
|
||||
|
||||
**It opens windows on the operator's screen**, and its description says so first, so a caller tells the
|
||||
operator before calling it. On 2026-10-12 at 12:05 a test Betterbird with an empty profile carrying the
|
||||
operator's theme looked like the operator's configuration lost; so the test is **visibly a test**: its
|
||||
profile carries a `userChrome.css` that frames every window of the test Betterbird in red and runs a red
|
||||
band reading MESH TEST across its top, and its event is called "MESH TEST reminder (a throw-away
|
||||
Betterbird of the mesh, not your calendar)". The reminder window itself is left unstyled, because how
|
||||
Betterbird draws it is what the test looks at; its title is the event's. A window title cannot be set by a
|
||||
preference, so the marking is the band, the frame and the event's name.
|
||||
|
||||
`betterbird_reminder_test` starts a **second** Betterbird with `--no-remote` in a throw-away profile
|
||||
under `/run/user/<uid>/mesh-betterbird-reminder-test/`, as the user unit
|
||||
`mesh-betterbird-reminder-test` (ended after 10 minutes at most). The profile holds the module's
|
||||
user.js and one calendar: a local `.ics` file with one event, "Mesh reminder test", whose reminder is
|
||||
user.js (the setting `prefs`; never a declared account) and one calendar: a local `.ics` file with one
|
||||
event, "MESH TEST reminder", whose reminder is
|
||||
due `after` seconds from the call (default 40). `carry` copies named preferences of the operator's
|
||||
profile into it (a theme, a colour switch), under the same limits as `betterbird_prefs`. Then:
|
||||
profile into it (a theme, a colour switch), under the same limits as `betterbird_prefs`. The test's own
|
||||
preferences come last in its user.js, so nothing declared or carried overrides them; one of them turns
|
||||
Betterbird's update check off (`betterbird.update.checkIntervalDays` 0), because a fresh profile checks
|
||||
at its start and, when a newer build is published, opens the About window on the operator's screen
|
||||
(seen 2026-10-12). Then:
|
||||
|
||||
- `xorg_windows` with class `Calendar` answers the reminder window: role `AlarmWindow`, type `Dialog`,
|
||||
its depth (32: a visual with alpha, which the compositor blends), its map state;
|
||||
@@ -106,13 +212,19 @@ A window of depth 32 that its program has not drawn is transparent under a compo
|
||||
that is why the symptom looks the way it does. Why Betterbird did not draw it is hq issue 345's
|
||||
question.
|
||||
|
||||
## What changes when it is assigned
|
||||
## What changes when it is pushed
|
||||
|
||||
| | g14 | shanks |
|
||||
|---|---|---|
|
||||
| files | `~/.config/betterbird/` and its `user.js` (no preference) | the same |
|
||||
| the profile | untouched until `betterbird_user_js` with `apply` or `betterbird_restart` | the same; the profile has no user.js today, so `apply` may place one |
|
||||
| Betterbird | keeps running as started | keeps running as started (since 2026-10-04, its output on the session's pipe nobody reads: `betterbird_check` names it until `betterbird_restart`) |
|
||||
The module is assigned to shanks today (look-only, from issue 345's first part); g14 runs Betterbird
|
||||
outside the mesh. With this version, a push to a node (each waits for the operator: `upgrade: record`):
|
||||
|
||||
| | on a node with the community package (g14, shanks) |
|
||||
|---|---|
|
||||
| the package | `betterbird-bin` removed (quit Betterbird first) |
|
||||
| `/opt/betterbird` | the mesh's 153.4.0esr-bb10, in the same place, so the same profile opens |
|
||||
| the policy, Owl | placed; Owl installed into the profile by Betterbird at its next start where it is missing |
|
||||
| `/usr/local` | the start, the launcher entry, the tools' program |
|
||||
| `~/.config/betterbird/` | `user.js`, `accounts.json` (no account until the setting `accounts` is approved) |
|
||||
| the profile | untouched until the next start from the launcher, `betterbird_user_js` with apply, or `betterbird_restart` |
|
||||
|
||||
## Relies on
|
||||
|
||||
@@ -120,3 +232,6 @@ question.
|
||||
- **`xorg`** for `xorg_windows` and `xorg_screenshot`, which look at Betterbird's windows. This module
|
||||
carries no window code of its own.
|
||||
- `systemd-run` and `journalctl` of the account's user manager.
|
||||
- The build seat's archive pinned by URL and sha256 (mesh-controller, ADR 0316 §2): a controller that does
|
||||
not know it refuses this manifest.
|
||||
- The controller's `settings` verb, for `betterbird_accounts` with `propose` (`invokes`).
|
||||
|
||||
@@ -0,0 +1,811 @@
|
||||
package main
|
||||
|
||||
// The operator's mail accounts, declared without credentials (novox/hq ADR 0316 §10–§14, to-be 56 §4).
|
||||
//
|
||||
// The setting `accounts` reaches the machine as ~/.config/betterbird/accounts.json, a file the node-engine
|
||||
// writes from the module's settings layers. Per account it names the address, the incoming server and the
|
||||
// outgoing server; never a password or a token, and the whole file is refused when any key in it is shaped
|
||||
// like one. Betterbird's accounts are preferences, so a declared account becomes:
|
||||
//
|
||||
// - its declared fields as preferences in the user.js the module places, where they win at every start
|
||||
// (§11): under the keys of the account the profile already holds at that address, which the module adopts
|
||||
// (novox/hq ADR 0317), or, where the profile holds none, under keys of the module's own that end in
|
||||
// mesh_<name> (account_mesh_<name>, id_mesh_<name>, server_mesh_<name>, smtp_mesh_<name>);
|
||||
// - for an account the module created, its key in the account list (mail.accountmanager.accounts) and in the
|
||||
// list of outgoing servers (mail.smtpservers) of the profile's prefs.js: written into, never owned, and
|
||||
// only while Betterbird is not running, because it rewrites prefs.js when it quits (§12). Every other key
|
||||
// keeps its place, and an adopted account's keys are in the lists already.
|
||||
//
|
||||
// Which account is adopted, created or refused is adoption.go's.
|
||||
//
|
||||
// Nobody's password passes through here: an account signs in at its first use, in Betterbird's own window,
|
||||
// and Betterbird keeps the result in its login store (§14). Of that store the module reads only the server
|
||||
// names each entry is kept for, which Betterbird writes in the clear, to count the declared accounts that
|
||||
// have not signed in yet (§16); never a user name or a password, which are encrypted there anyway.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// accountsFile is the setting accounts on the machine, written by the node-engine.
|
||||
const accountsFile = ".config/betterbird/accounts.json"
|
||||
|
||||
// Account is one declared mail account, in the shape of the setting accounts: keyed by its name.
|
||||
type Account struct {
|
||||
Address string `json:"address"`
|
||||
// FromName is the name the account's mail is sent under. Absent: Betterbird's, none.
|
||||
FromName string `json:"from-name,omitempty"`
|
||||
// Label is what Betterbird shows the account as. Absent: its address.
|
||||
Label string `json:"label,omitempty"`
|
||||
Incoming Server `json:"incoming"`
|
||||
Outgoing Server `json:"outgoing"`
|
||||
// Key is the profile's account key to adopt, given only in a node's own layer to settle an address the
|
||||
// profile holds more than once (novox/hq ADR 0317 §8).
|
||||
Key string `json:"key,omitempty"`
|
||||
}
|
||||
|
||||
// Server is an account's incoming or outgoing server: never a password or a token.
|
||||
type Server struct {
|
||||
// Kind is imap or pop3, for the incoming server; the outgoing one is smtp and says nothing.
|
||||
Kind string `json:"kind,omitempty"`
|
||||
Host string `json:"host"`
|
||||
Port int `json:"port,omitempty"`
|
||||
Security string `json:"security"`
|
||||
SignIn string `json:"sign-in"`
|
||||
User string `json:"user,omitempty"`
|
||||
}
|
||||
|
||||
// The words a server's connection security and sign-in method are said in, and Betterbird's numbers for
|
||||
// them (socketType / try_ssl, authMethod).
|
||||
var (
|
||||
securities = map[string]int{"none": 0, "starttls": 2, "tls": 3}
|
||||
signIns = map[string]int{"none": 1, "password": 3, "encrypted-password": 4, "kerberos": 5, "ntlm": 6,
|
||||
"tls-certificate": 7, "oauth": 10}
|
||||
)
|
||||
|
||||
func wordFor(words map[string]int, n int) string {
|
||||
for w, v := range words {
|
||||
if v == n {
|
||||
return w
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
var (
|
||||
accountName = regexp.MustCompile(`^[a-z][a-z0-9]{0,30}$`)
|
||||
addressShape = regexp.MustCompile(`^[^\s@"\\]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}$`)
|
||||
hostShape = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9.\-]*[A-Za-z0-9])?$`)
|
||||
profileKey = regexp.MustCompile(`^[A-Za-z0-9_]{1,64}$`)
|
||||
// plainText is a value written into a preference: no quote, backslash or line break.
|
||||
plainText = regexp.MustCompile(`^[^"\\\n\r]*$`)
|
||||
// secretKey is a key the accounts file may never carry, at any depth: the setting holds no credential.
|
||||
secretKey = regexp.MustCompile(`(?i)(pass|secret|token|oauth.?refresh|refresh|credential|api.?key|private.?key|cookie|bearer)`)
|
||||
)
|
||||
|
||||
// keysOf are the module's own keys for a declared account: each ends in mesh_<name>.
|
||||
func keysOf(name string) (account, identity, server, smtp string) {
|
||||
return "account_mesh_" + name, "id_mesh_" + name, "server_mesh_" + name, "smtp_mesh_" + name
|
||||
}
|
||||
|
||||
// ownKey is a key of an account list the module wrote, and the name it was declared under.
|
||||
var ownKey = regexp.MustCompile(`^(?:account|smtp)_mesh_([a-z][a-z0-9]{0,30})$`)
|
||||
|
||||
// readAccounts reads the declared accounts. A missing file is no account (the module is older here); a file
|
||||
// that carries a key shaped like a credential, an unknown field or a value that cannot be written is refused
|
||||
// whole, naming the key and never a value.
|
||||
func (m *Machine) readAccounts() (map[string]Account, bool, error) {
|
||||
raw, err := readBounded(m.home(accountsFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return map[string]Account{}, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
accounts, err := parseAccounts(raw)
|
||||
return accounts, true, err
|
||||
}
|
||||
|
||||
func parseAccounts(raw []byte) (map[string]Account, error) {
|
||||
var tree any
|
||||
if err := json.Unmarshal(raw, &tree); err != nil {
|
||||
return nil, fmt.Errorf("~/%s is not JSON: %v", accountsFile, err)
|
||||
}
|
||||
if where := secretKeyIn(tree, ""); where != "" {
|
||||
// An account's name is said by nothing: its segment of the path is withheld.
|
||||
if parts := strings.SplitN(where, ".", 3); len(parts) == 3 && parts[0] == "accounts" {
|
||||
where = "accounts.<an account>." + parts[2]
|
||||
}
|
||||
return nil, fmt.Errorf("the setting accounts carries %s, a key shaped like a credential: an account is declared "+
|
||||
"without its password or token, which it signs in with at its first use (novox/hq ADR 0316 §10, §14). "+
|
||||
"Take it out at the controller's terminal", where)
|
||||
}
|
||||
var top map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &top); err != nil {
|
||||
return nil, fmt.Errorf("~/%s is not an object: %v", accountsFile, err)
|
||||
}
|
||||
accounts := map[string]Account{}
|
||||
if body, ok := top["accounts"]; ok && string(body) != "null" {
|
||||
dec := json.NewDecoder(bytes.NewReader(body))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&accounts); err != nil {
|
||||
return nil, fmt.Errorf("the setting accounts is not a map of accounts by name: %v", err)
|
||||
}
|
||||
}
|
||||
var problems []string
|
||||
names := sortedNames(accounts)
|
||||
for i, name := range names {
|
||||
problems = append(problems, accountProblems(fmt.Sprintf("declared account %d of %d", i+1, len(names)),
|
||||
name, accounts[name])...)
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return nil, fmt.Errorf("the setting accounts is refused: %s", strings.Join(problems, "; "))
|
||||
}
|
||||
return accounts, nil
|
||||
}
|
||||
|
||||
// secretKeyIn is the path of the first key shaped like a credential in a JSON tree, or "".
|
||||
func secretKeyIn(node any, at string) string {
|
||||
switch v := node.(type) {
|
||||
case map[string]any:
|
||||
keys := make([]string, 0, len(v))
|
||||
for k := range v {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, k := range keys {
|
||||
here := k
|
||||
if at != "" {
|
||||
here = at + "." + k
|
||||
}
|
||||
if secretKey.MatchString(k) {
|
||||
return here
|
||||
}
|
||||
if w := secretKeyIn(v[k], here); w != "" {
|
||||
return w
|
||||
}
|
||||
}
|
||||
case []any:
|
||||
for i, x := range v {
|
||||
if w := secretKeyIn(x, fmt.Sprintf("%s[%d]", at, i)); w != "" {
|
||||
return w
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func sortedNames(accounts map[string]Account) []string {
|
||||
names := make([]string, 0, len(accounts))
|
||||
for n := range accounts {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
// accountProblems is what is wrong with one declared account, said of it by its place (label) and never by a
|
||||
// value of it, its name included (novox/hq ADR 0316 §16).
|
||||
func accountProblems(label, name string, a Account) []string {
|
||||
var out []string
|
||||
say := func(format string, args ...any) { out = append(out, label+": "+fmt.Sprintf(format, args...)) }
|
||||
if !accountName.MatchString(name) {
|
||||
say("an account's name is lower-case letters and digits, starting with a letter (it names the module's keys, mesh_<name>)")
|
||||
}
|
||||
if !addressShape.MatchString(a.Address) {
|
||||
say("the address is not a mail address")
|
||||
}
|
||||
if a.Key != "" && !profileKey.MatchString(a.Key) {
|
||||
say("key is an account key of the profile, such as account2")
|
||||
}
|
||||
for field, v := range map[string]string{"from-name": a.FromName, "label": a.Label} {
|
||||
if !plainText.MatchString(v) {
|
||||
say("%s holds a quote, a backslash or a line break", field)
|
||||
}
|
||||
}
|
||||
for side, s := range map[string]Server{"incoming": a.Incoming, "outgoing": a.Outgoing} {
|
||||
switch {
|
||||
case side == "incoming" && s.Kind != "imap" && s.Kind != "pop3":
|
||||
say("the incoming server's kind is imap or pop3")
|
||||
case side == "outgoing" && s.Kind != "" && s.Kind != "smtp":
|
||||
say("the outgoing server is smtp")
|
||||
}
|
||||
if !hostShape.MatchString(s.Host) {
|
||||
say("the %s server's host is not a host name", side)
|
||||
}
|
||||
if s.Port < 0 || s.Port > 65535 {
|
||||
say("the %s server's port is not a port", side)
|
||||
}
|
||||
if _, ok := securities[s.Security]; !ok {
|
||||
say("the %s server's security is tls, starttls or none", side)
|
||||
}
|
||||
if _, ok := signIns[s.SignIn]; !ok || (side == "incoming" && s.SignIn == "none") {
|
||||
say("the %s server's sign-in is password, encrypted-password, oauth, kerberos, ntlm or tls-certificate%s",
|
||||
side, map[bool]string{true: ", or none", false: ""}[side == "outgoing"])
|
||||
}
|
||||
if !plainText.MatchString(s.User) {
|
||||
say("the %s server's user holds a quote, a backslash or a line break", side)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// quoted is a preference's string value as user.js writes it.
|
||||
func quoted(s string) string { return strconv.Quote(s) }
|
||||
|
||||
// declaredPrefs are a declared account's fields as preferences under the keys it resolved to, name to value
|
||||
// as user.js writes it, in a fixed order. Only the fields the setting names (novox/hq ADR 0317 §5): never
|
||||
// the account's directory, folders, filters or caches. An adopted account keeps its kind, the outgoing server
|
||||
// its identity names, and its own name and sender's name where the declaration says none; a created one
|
||||
// gets every field.
|
||||
func declaredPrefs(k keySet, a Account, adopted bool) [][2]string {
|
||||
label := a.Label
|
||||
if label == "" && !adopted {
|
||||
label = a.Address
|
||||
}
|
||||
inPort := a.Incoming.Port
|
||||
if inPort == 0 {
|
||||
inPort = defaultPort(a.Incoming.Kind, a.Incoming.Security)
|
||||
}
|
||||
outPort := a.Outgoing.Port
|
||||
if outPort == 0 {
|
||||
outPort = defaultPort("smtp", a.Outgoing.Security)
|
||||
}
|
||||
var out [][2]string
|
||||
add := func(key, value string) { out = append(out, [2]string{key, value}) }
|
||||
if !adopted {
|
||||
add("mail.account."+k.Account+".identities", quoted(k.Identity))
|
||||
add("mail.account."+k.Account+".server", quoted(k.Server))
|
||||
}
|
||||
if a.FromName != "" || !adopted {
|
||||
add("mail.identity."+k.Identity+".fullName", quoted(a.FromName))
|
||||
}
|
||||
add("mail.identity."+k.Identity+".useremail", quoted(a.Address))
|
||||
if !adopted {
|
||||
add("mail.identity."+k.Identity+".smtpServer", quoted(k.SMTP))
|
||||
add("mail.identity."+k.Identity+".valid", "true")
|
||||
add("mail.server."+k.Server+".type", quoted(a.Incoming.Kind))
|
||||
}
|
||||
add("mail.server."+k.Server+".hostname", quoted(a.Incoming.Host))
|
||||
add("mail.server."+k.Server+".port", strconv.Itoa(inPort))
|
||||
add("mail.server."+k.Server+".socketType", strconv.Itoa(securities[a.Incoming.Security]))
|
||||
add("mail.server."+k.Server+".authMethod", strconv.Itoa(signIns[a.Incoming.SignIn]))
|
||||
add("mail.server."+k.Server+".userName", quoted(a.Incoming.User))
|
||||
if label != "" {
|
||||
add("mail.server."+k.Server+".name", quoted(label))
|
||||
}
|
||||
add("mail.smtpserver."+k.SMTP+".hostname", quoted(a.Outgoing.Host))
|
||||
add("mail.smtpserver."+k.SMTP+".port", strconv.Itoa(outPort))
|
||||
add("mail.smtpserver."+k.SMTP+".try_ssl", strconv.Itoa(securities[a.Outgoing.Security]))
|
||||
add("mail.smtpserver."+k.SMTP+".authMethod", strconv.Itoa(signIns[a.Outgoing.SignIn]))
|
||||
add("mail.smtpserver."+k.SMTP+".username", quoted(a.Outgoing.User))
|
||||
if label != "" {
|
||||
add("mail.smtpserver."+k.SMTP+".description", quoted(label))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// defaultPort is the port a server listens on when the declaration says none.
|
||||
func defaultPort(kind, security string) int {
|
||||
switch {
|
||||
case kind == "imap" && security == "tls":
|
||||
return 993
|
||||
case kind == "imap":
|
||||
return 143
|
||||
case kind == "pop3" && security == "tls":
|
||||
return 995
|
||||
case kind == "pop3":
|
||||
return 110
|
||||
case security == "tls":
|
||||
return 465
|
||||
case security == "starttls":
|
||||
return 587
|
||||
}
|
||||
return 25
|
||||
}
|
||||
|
||||
// composeUserJS is the user.js the module places in the profile: the declared file (the setting prefs), then
|
||||
// each adopted or created account's declared fields under its keys. A refused account has none.
|
||||
func composeUserJS(declaredText string, accounts map[string]Account, resolved []resolvedAccount) string {
|
||||
var lines []string
|
||||
for _, r := range resolved {
|
||||
if r.How == refused {
|
||||
continue
|
||||
}
|
||||
lines = append(lines, fmt.Sprintf("// declared account %d: %s under %s", r.Place, r.How, r.Keys.Account))
|
||||
for _, p := range declaredPrefs(r.Keys, accounts[r.name], r.How == adopted) {
|
||||
lines = append(lines, fmt.Sprintf("user_pref(%s, %s);", quoted(p[0]), p[1]))
|
||||
}
|
||||
}
|
||||
if len(lines) == 0 {
|
||||
return declaredText
|
||||
}
|
||||
return strings.TrimRight(declaredText, "\n") + "\n\n" +
|
||||
"// The accounts the setting accounts declares (novox/hq ADR 0316 §11, ADR 0317): the fields it names, under\n" +
|
||||
"// the keys of the account adopted or created. No password or token: an account signs in at its first use.\n" +
|
||||
strings.Join(lines, "\n") + "\n"
|
||||
}
|
||||
|
||||
// profileAccount is one account the profile holds, as its preferences say it.
|
||||
type profileAccount struct {
|
||||
Key, Server, Kind string
|
||||
// Identities are its identities, the default first, and Addresses their addresses in that order.
|
||||
Identities []string
|
||||
Addresses []string
|
||||
Owl, Ours bool
|
||||
}
|
||||
|
||||
// profileAccounts reads the accounts the profile's list names, from prefs.js and user.js (user.js wins).
|
||||
func profileAccounts(prefs map[string]string) []profileAccount {
|
||||
var out []profileAccount
|
||||
for _, key := range listOf(prefs["mail.accountmanager.accounts"]) {
|
||||
a := profileAccount{Key: key, Server: unquote(prefs["mail.account."+key+".server"])}
|
||||
a.Kind = unquote(prefs["mail.server."+a.Server+".type"])
|
||||
a.Owl = strings.HasPrefix(a.Kind, "owl")
|
||||
a.Ours = ownKey.MatchString(key)
|
||||
for _, id := range listOf(prefs["mail.account."+key+".identities"]) {
|
||||
a.Identities = append(a.Identities, id)
|
||||
a.Addresses = append(a.Addresses, unquote(prefs["mail.identity."+id+".useremail"]))
|
||||
}
|
||||
out = append(out, a)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// mergedPrefs are a profile's preferences as Betterbird starts with them: prefs.js, then user.js over it.
|
||||
func mergedPrefs(profile string) map[string]string {
|
||||
prefsJS, _ := readBounded(filepath.Join(profile, "prefs.js"))
|
||||
userJS, _ := readBounded(filepath.Join(profile, "user.js"))
|
||||
out := readPrefs(prefsJS)
|
||||
for k, v := range readPrefs(userJS) {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// listOf reads a comma-separated list preference's value.
|
||||
func listOf(v string) []string {
|
||||
var out []string
|
||||
for _, k := range strings.Split(unquote(v), ",") {
|
||||
if k = strings.TrimSpace(k); k != "" {
|
||||
out = append(out, k)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// unquote is a preference's string value without its quotes; any other value as written.
|
||||
func unquote(v string) string {
|
||||
if s, err := strconv.Unquote(v); err == nil {
|
||||
return s
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// writeAccountLists writes the module's keys into the profile's account list and list of outgoing servers
|
||||
// (§12): the declared accounts' keys added at the end, the module's keys of accounts no longer declared taken
|
||||
// out, every other key and line kept as it was. Only while Betterbird is not running. Says what changed.
|
||||
func (m *Machine) writeAccountLists(profile string, resolved []resolvedAccount) (string, error) {
|
||||
if ps := m.mainProcs(); len(ps) > 0 {
|
||||
return "", fmt.Errorf("Betterbird is running (pid %d), and it rewrites prefs.js when it quits: the account "+
|
||||
"list is written at its next start from the launcher, or by betterbird_restart", ps[0].PID)
|
||||
}
|
||||
path := filepath.Join(profile, "prefs.js")
|
||||
raw, err := readBounded(path)
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return "", err
|
||||
}
|
||||
var wantAccounts, wantSMTP []string
|
||||
for _, r := range resolved {
|
||||
if r.How == created {
|
||||
wantAccounts, wantSMTP = append(wantAccounts, r.Keys.Account), append(wantSMTP, r.Keys.SMTP)
|
||||
}
|
||||
}
|
||||
out, changed := withListed(string(raw), map[string][]string{
|
||||
"mail.accountmanager.accounts": wantAccounts, "mail.smtpservers": wantSMTP})
|
||||
if !changed {
|
||||
return "unchanged", nil
|
||||
}
|
||||
mode := os.FileMode(0o600)
|
||||
if st, err := os.Stat(path); err == nil {
|
||||
mode = st.Mode().Perm()
|
||||
}
|
||||
tmp := path + ".mesh-new"
|
||||
if err := os.WriteFile(tmp, []byte(out), mode); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := os.Rename(tmp, path); err != nil {
|
||||
os.Remove(tmp)
|
||||
return "", err
|
||||
}
|
||||
return fmt.Sprintf("written: %d account(s) the module created in the account list", len(wantAccounts)), nil
|
||||
}
|
||||
|
||||
// withListed is prefs.js with each list preference holding the module's wanted keys: its own keys that are
|
||||
// not wanted taken out, the wanted ones not there added at its end, everything else as it was. A list the
|
||||
// file does not hold yet is added as a line, when anything is wanted in it.
|
||||
func withListed(text string, want map[string][]string) (string, bool) {
|
||||
changed := false
|
||||
done := map[string]bool{}
|
||||
var lines []string
|
||||
s := bufio.NewScanner(strings.NewReader(text))
|
||||
s.Buffer(make([]byte, 64<<10), 4<<20)
|
||||
for s.Scan() {
|
||||
line := s.Text()
|
||||
if mm := prefLine.FindStringSubmatch(line); mm != nil {
|
||||
if keys, ok := want[mm[1]]; ok && !done[mm[1]] {
|
||||
done[mm[1]] = true
|
||||
next := listedWith(listOf(mm[2]), keys)
|
||||
if next != strings.Join(listOf(mm[2]), ",") {
|
||||
line = fmt.Sprintf("user_pref(%s, %s);", quoted(mm[1]), quoted(next))
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
}
|
||||
lines = append(lines, line)
|
||||
}
|
||||
names := make([]string, 0, len(want))
|
||||
for n := range want {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, n := range names {
|
||||
if !done[n] && len(want[n]) > 0 {
|
||||
lines = append(lines, fmt.Sprintf("user_pref(%s, %s);", quoted(n), quoted(strings.Join(want[n], ","))))
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
out := strings.Join(lines, "\n")
|
||||
if text == "" || strings.HasSuffix(text, "\n") {
|
||||
out += "\n"
|
||||
}
|
||||
return out, changed
|
||||
}
|
||||
|
||||
// listedWith is a list with the module's own keys made the wanted ones, every other key in its place.
|
||||
func listedWith(have, wanted []string) string {
|
||||
want := map[string]bool{}
|
||||
for _, k := range wanted {
|
||||
want[k] = true
|
||||
}
|
||||
var out []string
|
||||
there := map[string]bool{}
|
||||
for _, k := range have {
|
||||
if ownKey.MatchString(k) && !want[k] {
|
||||
continue
|
||||
}
|
||||
out = append(out, k)
|
||||
there[k] = true
|
||||
}
|
||||
for _, k := range wanted {
|
||||
if !there[k] {
|
||||
out = append(out, k)
|
||||
}
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
// AccountCount is what status and check say of the accounts: numbers, places and keys, never an address, a
|
||||
// host or a name (novox/hq ADR 0316 §16, ADR 0317 §3).
|
||||
type AccountCount struct {
|
||||
Declared int `json:"declared"`
|
||||
// InList are the declared accounts the profile's account list names, adopted or created.
|
||||
InList int `json:"declared_in_the_list"`
|
||||
// Others are the accounts the profile holds that no declaration names (the work account in Owl among them).
|
||||
Others int `json:"other_accounts"`
|
||||
// WithoutSignIn are the declared accounts with no sign-in kept in Betterbird's login store yet.
|
||||
WithoutSignIn int `json:"declared_without_a_sign_in,omitempty"`
|
||||
// Each is every declared account by its place in the setting: adopted, created or refused, and its keys.
|
||||
Each []resolvedAccount `json:"each,omitempty"`
|
||||
Problem string `json:"problem,omitempty"`
|
||||
}
|
||||
|
||||
// countAccounts counts the declared accounts against the profile's.
|
||||
func (m *Machine) countAccounts(profile, profileDir string) AccountCount {
|
||||
var c AccountCount
|
||||
accounts, _, err := m.readAccounts()
|
||||
if err != nil {
|
||||
c.Problem = err.Error()
|
||||
return c
|
||||
}
|
||||
c.Declared = len(accounts)
|
||||
prefs := mergedPrefs(profile)
|
||||
have := profileAccounts(prefs)
|
||||
c.Each = resolveAccounts(accounts, prefs, m.readRecord(profileDir))
|
||||
mine := map[string]bool{}
|
||||
refusedN := 0
|
||||
for _, r := range c.Each {
|
||||
if r.How == refused {
|
||||
refusedN++
|
||||
continue
|
||||
}
|
||||
mine[r.Keys.Account] = true
|
||||
}
|
||||
for _, a := range have {
|
||||
switch {
|
||||
case mine[a.Key]:
|
||||
c.InList++
|
||||
case a.Kind != "none":
|
||||
c.Others++
|
||||
}
|
||||
}
|
||||
if refusedN > 0 {
|
||||
c.Problem = fmt.Sprintf("%d declared account(s) are refused: each says why and what to do", refusedN)
|
||||
}
|
||||
c.WithoutSignIn = len(withoutSignIn(accounts, keptSignIns(profile)))
|
||||
return c
|
||||
}
|
||||
|
||||
// keptSignIns are the server names Betterbird's login store keeps a sign-in for: the `hostname` of each
|
||||
// entry of logins.json, which Betterbird writes in the clear. Nothing else of the file is read.
|
||||
func keptSignIns(profile string) map[string]bool {
|
||||
raw, err := readBounded(filepath.Join(profile, "logins.json"))
|
||||
out := map[string]bool{}
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
var store struct {
|
||||
Logins []struct {
|
||||
Hostname string `json:"hostname"`
|
||||
} `json:"logins"`
|
||||
}
|
||||
if json.Unmarshal(raw, &store) != nil {
|
||||
return out
|
||||
}
|
||||
for _, l := range store.Logins {
|
||||
out[strings.ToLower(l.Hostname)] = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// oauthIssuers are the sign-in services of the providers whose servers sign in through OAuth, by a suffix of
|
||||
// the server's host: their kept sign-in is under the service's name, not the server's.
|
||||
var oauthIssuers = map[string]string{"gmail.com": "oauth://accounts.google.com",
|
||||
"googlemail.com": "oauth://accounts.google.com", "office365.com": "oauth://login.microsoftonline.com",
|
||||
"outlook.com": "oauth://login.microsoftonline.com"}
|
||||
|
||||
// withoutSignIn are the declared accounts whose incoming server has no sign-in kept yet. A server signed in
|
||||
// to without a password (none, kerberos, ntlm, a certificate) keeps nothing and is never counted.
|
||||
func withoutSignIn(accounts map[string]Account, kept map[string]bool) []string {
|
||||
var out []string
|
||||
for _, name := range sortedNames(accounts) {
|
||||
in := accounts[name].Incoming
|
||||
switch in.SignIn {
|
||||
case "password", "encrypted-password":
|
||||
if !kept[strings.ToLower(in.Kind+"://"+in.Host)] {
|
||||
out = append(out, name)
|
||||
}
|
||||
case "oauth":
|
||||
issuer := ""
|
||||
for suffix, i := range oauthIssuers {
|
||||
if strings.HasSuffix(strings.ToLower(in.Host), suffix) {
|
||||
issuer = i
|
||||
}
|
||||
}
|
||||
found := false
|
||||
for k := range kept {
|
||||
if (issuer != "" && k == issuer) || (issuer == "" && strings.HasPrefix(k, "oauth://")) {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
out = append(out, name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// AccountsAnswer is what betterbird_accounts answers its caller: counts and kinds, never an address, a host or
|
||||
// a name (novox/hq ADR 0317 §9). The value itself goes only to the operator, in a proposal they read at
|
||||
// their own desk.
|
||||
type AccountsAnswer struct {
|
||||
Profile string `json:"profile"`
|
||||
// Declarable are the profile's mail accounts a declaration can hold (IMAP and POP3), by kind.
|
||||
Declarable map[string]int `json:"declarable"`
|
||||
// KeptByOwl are the accounts Owl keeps (the work account through Exchange): never declared (ADR 0316 §13).
|
||||
KeptByOwl int `json:"kept_by_owl"`
|
||||
// NotDeclarable are accounts of another kind (local folders, feeds, news, chat), by kind.
|
||||
NotDeclarable map[string]int `json:"not_declarable,omitempty"`
|
||||
// AlreadyDeclared are the declarable accounts under the module's own keys.
|
||||
AlreadyDeclared int `json:"under_the_modules_keys"`
|
||||
// Proposed is the controller's answer to the proposal, with every address and host masked.
|
||||
Proposed string `json:"proposed,omitempty"`
|
||||
Note string `json:"note"`
|
||||
}
|
||||
|
||||
// profileDeclarable reads the profile's accounts: the value a declaration of them would be, by name, and
|
||||
// what is answered of them.
|
||||
func (m *Machine) profileDeclarable() (map[string]Account, AccountsAnswer, error) {
|
||||
p, err := m.profile()
|
||||
if err != nil {
|
||||
return nil, AccountsAnswer{}, err
|
||||
}
|
||||
prefs := mergedPrefs(p.path)
|
||||
a := AccountsAnswer{Profile: p.Dir, Declarable: map[string]int{}, NotDeclarable: map[string]int{}}
|
||||
value := map[string]Account{}
|
||||
used := map[string]bool{}
|
||||
for _, pa := range profileAccounts(prefs) {
|
||||
switch {
|
||||
case pa.Owl:
|
||||
a.KeptByOwl++
|
||||
continue
|
||||
case pa.Kind != "imap" && pa.Kind != "pop3":
|
||||
kind := pa.Kind
|
||||
if kind == "none" {
|
||||
kind = "local folders"
|
||||
}
|
||||
a.NotDeclarable[kind]++
|
||||
continue
|
||||
}
|
||||
a.Declarable[pa.Kind]++
|
||||
if pa.Ours {
|
||||
a.AlreadyDeclared++
|
||||
}
|
||||
address := ""
|
||||
if len(pa.Addresses) > 0 {
|
||||
address = pa.Addresses[0]
|
||||
}
|
||||
name := nameFor(pa, address, used)
|
||||
used[name] = true
|
||||
value[name] = readAccount(prefs, pa)
|
||||
}
|
||||
if len(a.NotDeclarable) == 0 {
|
||||
a.NotDeclarable = nil
|
||||
}
|
||||
return value, a, nil
|
||||
}
|
||||
|
||||
// asker calls a verb of the mesh as this module, through the runtime: a variable so a test answers.
|
||||
var asker = stdio.Ask
|
||||
|
||||
// ProfileAccounts says, by count and kind, which accounts the profile in use holds; with propose it builds
|
||||
// the setting accounts from them (every account but Owl's, without any credential) and proposes it through
|
||||
// the controller's settings verb for the operator's Approve (novox/hq ADR 0317 §9, ADR 0277). The value
|
||||
// never comes back to the caller: the operator reads it whole at their desk (`settings proposals <id>`) and
|
||||
// checks the fingerprint the ask shows.
|
||||
func (m *Machine) ProfileAccounts(propose bool) (AccountsAnswer, error) {
|
||||
value, a, err := m.profileDeclarable()
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
a.Note = "counts and kinds only: no address, host or name leaves through this answer. With propose, the " +
|
||||
"setting accounts is built from these accounts (never Owl's, never a password or token) and proposed to " +
|
||||
"the operator, who reads it whole at their desk before approving (novox/hq ADR 0317 §9)"
|
||||
if !propose {
|
||||
return a, nil
|
||||
}
|
||||
if len(value) == 0 {
|
||||
return a, errors.New("the profile holds no IMAP or POP3 account to declare: nothing was proposed")
|
||||
}
|
||||
var problems []string
|
||||
for i, name := range sortedNames(value) {
|
||||
if p := accountProblems("", name, value[name]); len(p) > 0 {
|
||||
problems = append(problems, fmt.Sprintf("account %d of %d: %d field(s) cannot be declared as the profile "+
|
||||
"holds them", i+1, len(value), len(p)))
|
||||
}
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
return a, fmt.Errorf("nothing was proposed: %s; betterbird_prefs shows the account's preferences by name",
|
||||
strings.Join(problems, "; "))
|
||||
}
|
||||
body, err := json.Marshal(map[string]any{"accounts": value})
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
raw, err := asker("seat:mesh-controller.settings", map[string]any{"module": "betterbird", "values": string(body),
|
||||
"propose": "true"})
|
||||
if err != nil {
|
||||
return a, fmt.Errorf("the controller's settings verb did not answer: %v", err)
|
||||
}
|
||||
said, ok := verbSaid(raw)
|
||||
a.Proposed = m.maskValue(said)
|
||||
if !ok {
|
||||
return a, fmt.Errorf("the controller did not take the proposal: %s", a.Proposed)
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// verbSaid is a verb's words and whether it went ahead, whatever wraps them: the controller's {ok, output},
|
||||
// or the protocol's content list.
|
||||
func verbSaid(raw json.RawMessage) (string, bool) {
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
return verbSaid(json.RawMessage(s))
|
||||
}
|
||||
var m map[string]json.RawMessage
|
||||
if json.Unmarshal(raw, &m) != nil {
|
||||
return string(raw), true
|
||||
}
|
||||
if content, has := m["content"]; has {
|
||||
var items []struct {
|
||||
Text string `json:"text"`
|
||||
}
|
||||
var e struct {
|
||||
IsError bool `json:"isError"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &e)
|
||||
if json.Unmarshal(content, &items) == nil && len(items) > 0 {
|
||||
out, ok := verbSaid(json.RawMessage(items[0].Text))
|
||||
return out, ok && !e.IsError
|
||||
}
|
||||
}
|
||||
if okRaw, has := m["ok"]; has {
|
||||
var ok bool
|
||||
_ = json.Unmarshal(okRaw, &ok)
|
||||
var output string
|
||||
_ = json.Unmarshal(m["output"], &output)
|
||||
return output, ok
|
||||
}
|
||||
return string(raw), true
|
||||
}
|
||||
|
||||
// readAccount reads one profile account's preferences into the shape of the setting.
|
||||
func readAccount(prefs map[string]string, pa profileAccount) Account {
|
||||
num := func(key string, def int) int {
|
||||
if n, err := strconv.Atoi(prefs[key]); err == nil {
|
||||
return n
|
||||
}
|
||||
return def
|
||||
}
|
||||
text := func(key string) string { return unquote(prefs[key]) }
|
||||
srv := "mail.server." + pa.Server + "."
|
||||
acc := Account{Incoming: Server{Kind: pa.Kind, Host: text(srv + "hostname"), User: text(srv + "userName"),
|
||||
Security: wordFor(securities, num(srv+"socketType", 0)), SignIn: wordFor(signIns, num(srv+"authMethod", 3))}}
|
||||
acc.Incoming.Port = num(srv+"port", defaultPort(pa.Kind, acc.Incoming.Security))
|
||||
ids := listOf(prefs["mail.account."+pa.Key+".identities"])
|
||||
if len(ids) > 0 {
|
||||
id := "mail.identity." + ids[0] + "."
|
||||
acc.Address, acc.FromName = text(id+"useremail"), text(id+"fullName")
|
||||
smtp := text(id + "smtpServer")
|
||||
if smtp == "" {
|
||||
smtp = text("mail.smtp.defaultserver")
|
||||
}
|
||||
if smtp != "" {
|
||||
s := "mail.smtpserver." + smtp + "."
|
||||
acc.Outgoing = Server{Host: text(s + "hostname"), User: text(s + "username"),
|
||||
Security: wordFor(securities, num(s+"try_ssl", 0)), SignIn: wordFor(signIns, num(s+"authMethod", 3))}
|
||||
acc.Outgoing.Port = num(s+"port", defaultPort("smtp", acc.Outgoing.Security))
|
||||
}
|
||||
}
|
||||
if label := text(srv + "name"); label != "" && label != acc.Address {
|
||||
acc.Label = label
|
||||
}
|
||||
return acc
|
||||
}
|
||||
|
||||
// nameFor is the name a profile account is answered under: its own name when the module declared it, else
|
||||
// the first label of its address's domain, made unique.
|
||||
func nameFor(pa profileAccount, address string, used map[string]bool) string {
|
||||
if mm := ownKey.FindStringSubmatch(pa.Key); mm != nil {
|
||||
return mm[1]
|
||||
}
|
||||
base := "account"
|
||||
if _, domain, ok := strings.Cut(address, "@"); ok {
|
||||
label, _, _ := strings.Cut(strings.ToLower(domain), ".")
|
||||
label = regexp.MustCompile(`[^a-z0-9]`).ReplaceAllString(label, "")
|
||||
if label != "" && label[0] >= 'a' && label[0] <= 'z' {
|
||||
base = label
|
||||
}
|
||||
}
|
||||
if len(base) > 28 {
|
||||
base = base[:28]
|
||||
}
|
||||
name := base
|
||||
for i := 2; used[name]; i++ {
|
||||
name = base + strconv.Itoa(i)
|
||||
}
|
||||
return name
|
||||
}
|
||||
@@ -0,0 +1,518 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The setting accounts (novox/hq ADR 0316 §10–§16, ADR 0317 §9 for betterbird_accounts): declared without
|
||||
// credentials, written under the module's own keys, the account list written into and never owned, refused
|
||||
// rather than duplicated, and said by count.
|
||||
|
||||
const personal = `{"accounts": {
|
||||
"personal": {"address": "me@mail.example.org", "from-name": "Me",
|
||||
"incoming": {"kind": "imap", "host": "imap.example.org", "security": "tls", "sign-in": "password", "user": "me"},
|
||||
"outgoing": {"host": "smtp.example.org", "port": 587, "security": "starttls", "sign-in": "password", "user": "me"}}
|
||||
}}`
|
||||
|
||||
// profileWithAccounts is a profile holding an account of the person's at another address, Owl's work account,
|
||||
// local folders, and an account list in that order.
|
||||
func (f *fake) profileWithAccounts() {
|
||||
f.betterbird()
|
||||
f.write(profileDir+"/prefs.js", "// Mozilla User Preferences\n"+
|
||||
`user_pref("mail.account.account1.server", "server1");`+"\n"+
|
||||
`user_pref("mail.account.account1.identities", "id1");`+"\n"+
|
||||
`user_pref("mail.account.account2.server", "server2");`+"\n"+
|
||||
`user_pref("mail.account.account3.server", "server3");`+"\n"+
|
||||
`user_pref("mail.account.account3.identities", "id3");`+"\n"+
|
||||
`user_pref("mail.accountmanager.accounts", "account1,account3,account2");`+"\n"+
|
||||
`user_pref("mail.identity.id1.useremail", "other@mail.example.org");`+"\n"+
|
||||
`user_pref("mail.identity.id1.fullName", "Other");`+"\n"+
|
||||
`user_pref("mail.identity.id1.smtpServer", "smtp1");`+"\n"+
|
||||
`user_pref("mail.identity.id3.useremail", "work@corp.example.com");`+"\n"+
|
||||
`user_pref("mail.server.server1.type", "imap");`+"\n"+
|
||||
`user_pref("mail.server.server1.hostname", "imap.example.org");`+"\n"+
|
||||
`user_pref("mail.server.server1.socketType", 3);`+"\n"+
|
||||
`user_pref("mail.server.server1.authMethod", 10);`+"\n"+
|
||||
`user_pref("mail.server.server1.userName", "other");`+"\n"+
|
||||
`user_pref("mail.server.server1.directory", "/home/operator/.thunderbird/x/ImapMail/imap.example.org");`+"\n"+
|
||||
`user_pref("mail.server.server2.type", "none");`+"\n"+
|
||||
`user_pref("mail.server.server3.type", "owl-ews");`+"\n"+
|
||||
`user_pref("mail.smtpserver.smtp1.hostname", "smtp.example.org");`+"\n"+
|
||||
`user_pref("mail.smtpserver.smtp1.try_ssl", 3);`+"\n"+
|
||||
`user_pref("mail.smtpserver.smtp1.authMethod", 10);`+"\n"+
|
||||
`user_pref("mail.smtpserver.smtp1.username", "other");`+"\n"+
|
||||
`user_pref("mail.smtpservers", "smtp1");`+"\n"+
|
||||
`user_pref("mailnews.tags.version", 2);`+"\n")
|
||||
}
|
||||
|
||||
func (f *fake) declareAccounts(body string) {
|
||||
f.write(testHome+"/"+accountsFile, body)
|
||||
}
|
||||
|
||||
func (f *fake) read(path string) string {
|
||||
f.t.Helper()
|
||||
raw, err := os.ReadFile(filepath.Join(f.Root, path))
|
||||
if err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
func TestAnAccountsValueCarryingAPasswordLikeFieldIsRefusedWithoutItsValue(t *testing.T) {
|
||||
for _, body := range []string{
|
||||
`{"accounts": {"personal": {"address": "me@mail.example.org", "password": "hunter2"}}}`,
|
||||
`{"accounts": {"personal": {"incoming": {"oauth_refresh": "1//hunter2"}}}}`,
|
||||
`{"token": "hunter2", "accounts": {}}`,
|
||||
} {
|
||||
_, err := parseAccounts([]byte(body))
|
||||
if err == nil || !strings.Contains(err.Error(), "shaped like a credential") || strings.Contains(err.Error(), "hunter2") {
|
||||
t.Errorf("%s: %v", body, err)
|
||||
}
|
||||
}
|
||||
// An unknown field is refused too, and a value that cannot be declared, said by the account's place.
|
||||
if _, err := parseAccounts([]byte(`{"accounts": {"personal": {"address": "me@mail.example.org", "keychain": 1}}}`)); err == nil {
|
||||
t.Error("an unknown field was taken")
|
||||
}
|
||||
_, err := parseAccounts([]byte(strings.Replace(personal, `"security": "tls"`, `"security": "maybe"`, 1)))
|
||||
if err == nil || !strings.Contains(err.Error(), "declared account 1 of 1") || strings.Contains(err.Error(), "personal") ||
|
||||
strings.Contains(err.Error(), "example.org") {
|
||||
t.Errorf("a bad value is said by the account's place, never its name or a value: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheSettingPrefsRefusesAnAccountsPreferences(t *testing.T) {
|
||||
for _, name := range []string{"mail.account.account1.server", "mail.identity.id1.useremail", "mail.server.server1.hostname",
|
||||
"mail.smtpserver.smtp1.hostname", "mail.accountmanager.accounts", "mail.smtpservers"} {
|
||||
err := validUserJS(mark + ")\nuser_pref(\"" + name + "\", \"x\");\n")
|
||||
if err == nil || !strings.Contains(err.Error(), "setting accounts") {
|
||||
t.Errorf("%s was taken in prefs: %v", name, err)
|
||||
}
|
||||
}
|
||||
if err := validUserJS(mark + ")\nuser_pref(\"calendar.alarms.show\", true);\n"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// noPackage answers pacman as a machine without the community package.
|
||||
func (f *fake) noPackage() {
|
||||
f.answer = func(name string, args []string) Output {
|
||||
if name == "pacman" {
|
||||
return Output{Code: 1}
|
||||
}
|
||||
return Output{}
|
||||
}
|
||||
}
|
||||
|
||||
// userPrefLines are the user_pref lines of a user.js, by preference name.
|
||||
func userPrefLines(t *testing.T, text string) map[string]string {
|
||||
t.Helper()
|
||||
out := map[string]string{}
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
if !strings.HasPrefix(line, "user_pref(") {
|
||||
continue
|
||||
}
|
||||
if !statement.MatchString(line) {
|
||||
t.Errorf("not a statement Betterbird reads as one: %s", line)
|
||||
}
|
||||
out[line[len(`user_pref("`):strings.Index(line, `",`)]] = line
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestADeclaredAccountTheProfileDoesNotHoldIsCreatedUnderTheModulesKeysAndListed(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
f.declareAccounts(personal)
|
||||
// Betterbird is running: the user.js is placed, the list waits for its next start.
|
||||
u, err := f.userJS(true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(u.Lists, "not written") || !strings.Contains(u.Lists, "running") {
|
||||
t.Fatalf("the list is never written while Betterbird runs: %+v", u)
|
||||
}
|
||||
if len(u.Each) != 1 || u.Each[0].How != created || u.Each[0].Keys != (keySet{"account_mesh_personal", "id_mesh_personal",
|
||||
"server_mesh_personal", "smtp_mesh_personal"}) {
|
||||
t.Fatalf("%+v", u.Each)
|
||||
}
|
||||
placed := f.read(profileDir + "/user.js")
|
||||
for name := range userPrefLines(t, placed) {
|
||||
if key := strings.Split(name, ".")[2]; !strings.HasSuffix(key, "mesh_personal") {
|
||||
t.Errorf("a created account's preference outside the module's keys: %s", name)
|
||||
}
|
||||
if strings.Contains(strings.ToLower(name), "password") || strings.Contains(name, "directory") {
|
||||
t.Errorf("the module writes no credential and none of Betterbird's own preferences: %s", name)
|
||||
}
|
||||
}
|
||||
for _, want := range []string{`user_pref("mail.server.server_mesh_personal.port", 993);`,
|
||||
`user_pref("mail.server.server_mesh_personal.socketType", 3);`,
|
||||
`user_pref("mail.smtpserver.smtp_mesh_personal.try_ssl", 2);`,
|
||||
`user_pref("mail.identity.id_mesh_personal.useremail", "me@mail.example.org");`} {
|
||||
if !strings.Contains(placed, want) {
|
||||
t.Errorf("the user.js lacks %s:\n%s", want, placed)
|
||||
}
|
||||
}
|
||||
|
||||
// Quit: the list gains the module's keys at its end, every other line as it was.
|
||||
before := f.read(profileDir + "/prefs.js")
|
||||
os.RemoveAll(filepath.Join(f.Root, "proc", "4000"))
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after := f.read(profileDir + "/prefs.js")
|
||||
if !strings.Contains(after, `user_pref("mail.accountmanager.accounts", "account1,account3,account2,account_mesh_personal");`) ||
|
||||
!strings.Contains(after, `user_pref("mail.smtpservers", "smtp1,smtp_mesh_personal");`) {
|
||||
t.Fatalf("the lists:\n%s", after)
|
||||
}
|
||||
bl, al := strings.Split(before, "\n"), strings.Split(after, "\n")
|
||||
if len(bl) != len(al) {
|
||||
t.Fatalf("lines were added or lost: %d then %d", len(bl), len(al))
|
||||
}
|
||||
for i := range bl {
|
||||
if bl[i] != al[i] && !strings.Contains(bl[i], "mail.accountmanager.accounts") && !strings.Contains(bl[i], "mail.smtpservers") {
|
||||
t.Errorf("line %d changed: %q to %q", i+1, bl[i], al[i])
|
||||
}
|
||||
}
|
||||
|
||||
// The record lost: an account under the module's own keys is still its creation, never adopted.
|
||||
os.Remove(filepath.Join(f.Root, testHome, recordFile))
|
||||
u, err = f.userJS(true)
|
||||
if err != nil || u.Each[0].How != created {
|
||||
t.Fatalf("without the record, the module's keys still say created: %+v %v", u.Each, err)
|
||||
}
|
||||
|
||||
// The account leaves the setting: only the module's keys go, the mail stays in the profile.
|
||||
f.declareAccounts(`{"accounts": {}}`)
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := f.read(profileDir + "/prefs.js"); got != before {
|
||||
t.Fatalf("removing the module's keys left the file otherwise as it was:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// adoptable is a declaration of the profile's account1 (other@mail.example.org): equal but for its outgoing
|
||||
// port, said explicitly, and its incoming user.
|
||||
const adoptable = `{"accounts": {
|
||||
"mail": {"address": "Other@mail.example.org",
|
||||
"incoming": {"kind": "imap", "host": "imap.example.org", "security": "tls", "sign-in": "oauth", "user": "other2"},
|
||||
"outgoing": {"host": "smtp.example.org", "port": 465, "security": "tls", "sign-in": "oauth", "user": "other"}}}}`
|
||||
|
||||
func TestADeclaredAccountTheProfileHoldsIsAdoptedUnderItsOwnKeysAndTheListIsLeftAlone(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
f.declareAccounts(adoptable)
|
||||
os.RemoveAll(filepath.Join(f.Root, "proc", "4000"))
|
||||
before := f.read(profileDir + "/prefs.js")
|
||||
u, err := f.userJS(true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(u.Each) != 1 || u.Each[0].How != adopted || u.Each[0].Keys != (keySet{"account1", "id1", "server1", "smtp1"}) {
|
||||
t.Fatalf("adopted under the profile's keys: %+v", u.Each)
|
||||
}
|
||||
// The address's case and the user name differ; the default port declared explicitly is no change. Names
|
||||
// only, no value.
|
||||
if strings.Join(u.Each[0].Changed, ",") != "mail.identity.id1.useremail,mail.server.server1.userName" {
|
||||
t.Fatalf("the changed preferences, by name: %v", u.Each[0].Changed)
|
||||
}
|
||||
if f.read(profileDir+"/prefs.js") != before || u.Lists != "unchanged" {
|
||||
t.Fatalf("an adopted account's keys are in the lists already: nothing written to prefs.js (%s)", u.Lists)
|
||||
}
|
||||
lines := userPrefLines(t, f.read(profileDir+"/user.js"))
|
||||
for name := range lines {
|
||||
key := strings.Split(name, ".")[2]
|
||||
if key != "account1" && key != "id1" && key != "server1" && key != "smtp1" {
|
||||
t.Errorf("an adopted account's preference outside its keys: %s", name)
|
||||
}
|
||||
for _, never := range []string{"directory", ".type", "smtpServer", "identities", ".server"} {
|
||||
if strings.HasSuffix(name, never) {
|
||||
t.Errorf("the module wrote %s, which is Betterbird's or the account's own", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !strings.Contains(lines["mail.server.server1.userName"], `"other2"`) {
|
||||
t.Fatalf("the declared value wins at every start: %v", lines)
|
||||
}
|
||||
// The status says it by place and keys, never by address, host or name.
|
||||
s, err := f.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := json.Marshal(s.Accounts)
|
||||
if s.Accounts.InList != 1 || s.Accounts.Others != 1 || strings.Contains(string(raw), "example") ||
|
||||
strings.Contains(string(raw), `"mail"`) || !strings.Contains(string(raw), `"account1"`) {
|
||||
t.Fatalf("%s", raw)
|
||||
}
|
||||
|
||||
// Leaving the setting releases it: its lines and its record go, the account and the lists stay.
|
||||
f.declareAccounts(`{"accounts": {}}`)
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.read(profileDir+"/prefs.js") != before || strings.Contains(f.read(profileDir+"/user.js"), "server1") ||
|
||||
strings.Contains(f.read(testHome+"/"+recordFile), "account1") {
|
||||
t.Fatal("a released account stays the person's, with no line or record of the module's")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAmbiguousOwlsOrOtherKindOfMatchIsRefusedForThatAccountAlone(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
os.RemoveAll(filepath.Join(f.Root, "proc", "4000"))
|
||||
// A second account with account1's address.
|
||||
prefs := f.read(profileDir + "/prefs.js")
|
||||
prefs = strings.Replace(prefs, `"account1,account3,account2"`, `"account1,account3,account2,account4"`, 1) +
|
||||
`user_pref("mail.account.account4.server", "server4");` + "\n" +
|
||||
`user_pref("mail.account.account4.identities", "id4");` + "\n" +
|
||||
`user_pref("mail.identity.id4.useremail", "other@mail.example.org");` + "\n" +
|
||||
`user_pref("mail.identity.id4.smtpServer", "smtp4");` + "\n" +
|
||||
`user_pref("mail.smtpserver.smtp4.hostname", "smtp.example.org");` + "\n" +
|
||||
`user_pref("mail.server.server4.type", "imap");` + "\n"
|
||||
f.write(profileDir+"/prefs.js", prefs)
|
||||
both := strings.Replace(adoptable, `{"accounts": {`, `{"accounts": {
|
||||
"zz": {"address": "new@mail.example.org",
|
||||
"incoming": {"kind": "imap", "host": "imap.example.org", "security": "tls", "sign-in": "password"},
|
||||
"outgoing": {"host": "smtp.example.org", "security": "tls", "sign-in": "password"}},`, 1)
|
||||
f.declareAccounts(both)
|
||||
err := f.Prepare()
|
||||
if err == nil || !strings.Contains(err.Error(), "declared account 1 of 2: 2 accounts in the profile have its address (account1, account4)") ||
|
||||
!strings.Contains(err.Error(), "key") || strings.Contains(err.Error(), "example.org") {
|
||||
t.Fatalf("refused by place, with the matches' keys and the remedy, never the address: %v", err)
|
||||
}
|
||||
u, _ := f.userJS(false)
|
||||
if u.Each[0].How != refused || u.Each[1].How != created {
|
||||
t.Fatalf("the other declared account goes ahead: %+v", u.Each)
|
||||
}
|
||||
if strings.Contains(f.read(profileDir+"/user.js"), "server1") || strings.Contains(f.read(profileDir+"/user.js"), "server4") {
|
||||
t.Fatal("a refused account gets no line")
|
||||
}
|
||||
|
||||
// The node's layer names the key: that one is adopted.
|
||||
f.declareAccounts(strings.Replace(both, `"address": "Other@mail.example.org",`, `"address": "Other@mail.example.org", "key": "account4",`, 1))
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
if u, _ := f.userJS(false); u.Each[0].How != adopted || u.Each[0].Keys.Account != "account4" {
|
||||
t.Fatalf("%+v", u.Each)
|
||||
}
|
||||
os.Remove(filepath.Join(f.Root, testHome, recordFile))
|
||||
for key, want := range map[string]string{"account9": "not in the profile's account list", "account3": "Owl's",
|
||||
"account2": "another address"} {
|
||||
f.declareAccounts(strings.Replace(adoptable, `"address": "Other@mail.example.org",`, `"address": "Other@mail.example.org", "key": "`+key+`",`, 1))
|
||||
if err := f.Prepare(); err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Errorf("key %s: %v", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Owl's account at the address, and an account of another kind than declared.
|
||||
f.declareAccounts(`{"accounts": {"work": {"address": "work@corp.example.com",
|
||||
"incoming": {"kind": "imap", "host": "outlook.example.com", "security": "tls", "sign-in": "oauth"},
|
||||
"outgoing": {"host": "smtp.example.com", "security": "tls", "sign-in": "oauth"}}}}`)
|
||||
if err := f.Prepare(); err == nil || !strings.Contains(err.Error(), "Owl's") {
|
||||
t.Errorf("Owl's account is never adopted: %v", err)
|
||||
}
|
||||
f.write(profileDir+"/prefs.js", strings.Replace(prefs, `"mail.server.server4.type", "imap"`, `"mail.server.server4.type", "pop3"`, 1))
|
||||
f.declareAccounts(strings.Replace(adoptable, `"address": "Other@mail.example.org",`, `"address": "Other@mail.example.org", "key": "account4",`, 1))
|
||||
if err := f.Prepare(); err == nil || !strings.Contains(err.Error(), "never converts") {
|
||||
t.Errorf("an account of another kind is refused, not converted: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeclaredAccountsWithoutASignInAreCountedFromTheServerNamesAlone(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
links(t, nil)
|
||||
f.declareAccounts(`{"accounts": {
|
||||
"personal": {"address": "me@mail.example.org", "incoming": {"kind": "imap", "host": "imap.example.org", "security": "tls", "sign-in": "password"},
|
||||
"outgoing": {"host": "smtp.example.org", "security": "tls", "sign-in": "password"}},
|
||||
"google": {"address": "me@gmail.com", "incoming": {"kind": "imap", "host": "imap.gmail.com", "security": "tls", "sign-in": "oauth"},
|
||||
"outgoing": {"host": "smtp.gmail.com", "security": "tls", "sign-in": "oauth"}},
|
||||
"lan": {"address": "me@lan.example.org", "incoming": {"kind": "pop3", "host": "pop.example.org", "security": "tls", "sign-in": "kerberos"},
|
||||
"outgoing": {"host": "smtp.example.org", "security": "tls", "sign-in": "none"}}}}`)
|
||||
// The username and password fields are encrypted and never read; a fixture holding junk there proves it.
|
||||
f.write(profileDir+"/logins.json", `{"logins": [
|
||||
{"hostname": "imap://imap.example.org", "httpRealm": "imap://imap.example.org", "encryptedUsername": "MEIE...", "encryptedPassword": "MDoE..."},
|
||||
{"hostname": "smtp://smtp.example.org", "encryptedUsername": "x", "encryptedPassword": "y"}]}`)
|
||||
c := f.countAccounts(filepath.Join(f.Root, profileDir), "~/.thunderbird/60016jsp.default-default")
|
||||
if c.Declared != 3 || c.WithoutSignIn != 1 || c.Others != 2 || c.InList != 0 {
|
||||
t.Fatalf("three declared, the OAuth one not signed in, the person's and Owl's other (local folders not counted): %+v", c)
|
||||
}
|
||||
raw, _ := json.Marshal(c)
|
||||
if strings.Contains(string(raw), "example") || strings.Contains(string(raw), "gmail") {
|
||||
t.Fatalf("a count names nothing: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBetterbirdAccountsAnswersCountsAndProposesTheValueWithoutOwlOrACredential(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
links(t, nil)
|
||||
a, err := f.ProfileAccounts(false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a.Declarable["imap"] != 1 || a.KeptByOwl != 1 || a.NotDeclarable["local folders"] != 1 || a.Proposed != "" {
|
||||
t.Fatalf("%+v", a)
|
||||
}
|
||||
raw, _ := json.Marshal(a)
|
||||
for _, never := range []string{"example", "other", "Other", "work@", "smtp1"} {
|
||||
if strings.Contains(string(raw), never) {
|
||||
t.Errorf("the answer to the caller names %q: %s", never, raw)
|
||||
}
|
||||
}
|
||||
|
||||
var asked map[string]any
|
||||
asker = func(key string, body any) (json.RawMessage, error) {
|
||||
if key != "seat:mesh-controller.settings" {
|
||||
t.Fatalf("asked %s", key)
|
||||
}
|
||||
asked = body.(map[string]any)
|
||||
return json.RawMessage(`{"ok": true, "output": "proposed p-17 for betterbird on the whole mesh: accounts (other@mail.example.org); fingerprint ab12"}`), nil
|
||||
}
|
||||
t.Cleanup(func() { asker = nil })
|
||||
a, err = f.ProfileAccounts(true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if asked["module"] != "betterbird" || asked["propose"] != "true" || asked["node"] != nil {
|
||||
t.Fatalf("proposed for the whole mesh, never set: %v", asked)
|
||||
}
|
||||
value, err := parseAccounts([]byte(asked["values"].(string)))
|
||||
if err != nil {
|
||||
t.Fatalf("the proposed value is one the module takes: %v", err)
|
||||
}
|
||||
other, ok := value["mail"]
|
||||
if len(value) != 1 || !ok || other.Address != "other@mail.example.org" || other.Incoming.SignIn != "oauth" ||
|
||||
other.Incoming.Port != 993 || other.Outgoing.Security != "tls" || other.Outgoing.Port != 465 {
|
||||
t.Fatalf("the value holds the person's account and not Owl's: %+v", value)
|
||||
}
|
||||
if strings.Contains(a.Proposed, "other@") || !strings.Contains(a.Proposed, "p-17") {
|
||||
t.Fatalf("the controller's words come back with every address masked: %s", a.Proposed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAdoptedAccountNeverWritesIntoAnOutgoingServerAnotherIdentityUses(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
os.RemoveAll(filepath.Join(f.Root, "proc", "4000"))
|
||||
// A second account of the person's, not declared, sends through smtp1 too.
|
||||
prefs := strings.Replace(f.read(profileDir+"/prefs.js"), `"account1,account3,account2"`, `"account1,account3,account2,account5"`, 1) +
|
||||
`user_pref("mail.account.account5.server", "server5");` + "\n" +
|
||||
`user_pref("mail.account.account5.identities", "id5");` + "\n" +
|
||||
`user_pref("mail.identity.id5.useremail", "fifth@mail.example.org");` + "\n" +
|
||||
`user_pref("mail.identity.id5.smtpServer", "smtp1");` + "\n" +
|
||||
`user_pref("mail.server.server5.type", "imap");` + "\n"
|
||||
f.write(profileDir+"/prefs.js", prefs)
|
||||
f.declareAccounts(adoptable)
|
||||
err := f.Prepare()
|
||||
if err == nil || !strings.Contains(err.Error(), "also used by identity id5") || strings.Contains(err.Error(), "example.org") {
|
||||
t.Fatalf("refused with the remedy, by keys: %v", err)
|
||||
}
|
||||
if user, _ := os.ReadFile(filepath.Join(f.Root, profileDir, "user.js")); strings.Contains(string(user), "smtpserver.smtp1") {
|
||||
t.Fatal("the shared outgoing server was written")
|
||||
}
|
||||
|
||||
// Recorded adoption, then the person moves the identity to a server of its own: the module follows it.
|
||||
f.write(profileDir+"/prefs.js", strings.Replace(prefs, `"mail.identity.id5.smtpServer", "smtp1"`, `"mail.identity.id5.smtpServer", "smtp5"`, 1))
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
moved := strings.Replace(f.read(profileDir+"/prefs.js"), `"mail.identity.id1.smtpServer", "smtp1"`, `"mail.identity.id1.smtpServer", "smtp9"`, 1) +
|
||||
`user_pref("mail.smtpserver.smtp9.hostname", "smtp.example.org");` + "\n"
|
||||
f.write(profileDir+"/prefs.js", moved)
|
||||
u, err := f.userJS(true)
|
||||
if err != nil || u.Each[0].How != adopted || u.Each[0].Keys.SMTP != "smtp9" {
|
||||
t.Fatalf("the identity's current outgoing server is followed: %+v %v", u.Each, err)
|
||||
}
|
||||
user := f.read(profileDir + "/user.js")
|
||||
if strings.Contains(user, "smtpserver.smtp1.") || !strings.Contains(user, "smtpserver.smtp9.hostname") {
|
||||
t.Fatalf("never the old one:\n%s", user)
|
||||
}
|
||||
// And the move back onto a shared server is refused again.
|
||||
f.write(profileDir+"/prefs.js", strings.Replace(moved, `"mail.identity.id5.smtpServer", "smtp5"`, `"mail.identity.id5.smtpServer", "smtp9"`, 1))
|
||||
if err := f.Prepare(); err == nil || !strings.Contains(err.Error(), "also used by identity id5") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheUserJSAnswerWithholdsTheAccountsValues(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare(`user_pref("mail.compose.default_to", "x");`)
|
||||
f.declareAccounts(personal)
|
||||
u, err := f.userJS(false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, never := range []string{"example.org", `"me"`, `"Me"`} {
|
||||
if strings.Contains(u.Content, never) {
|
||||
t.Errorf("the answer carries %s:\n%s", never, u.Content)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(u.Content, `user_pref("mail.server.server_mesh_personal.hostname", "<withheld>");`) {
|
||||
t.Fatalf("an account's preference is named and its value withheld:\n%s", u.Content)
|
||||
}
|
||||
// What is placed in the profile keeps the values: only the answer withholds them.
|
||||
if f.read(testHome+"/"+accountsFile) == "" {
|
||||
t.Fatal("no declaration")
|
||||
}
|
||||
}
|
||||
|
||||
func TestACredentialRefusalNamesNoAccount(t *testing.T) {
|
||||
_, err := parseAccounts([]byte(`{"accounts": {"personal": {"incoming": {"password": "hunter2"}}}}`))
|
||||
if err == nil || strings.Contains(err.Error(), "personal") || !strings.Contains(err.Error(), "accounts.<an account>.incoming.password") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAccountsOwnAliasesAndNonMailIdentitiesDoNotShareItsOutgoingServer(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.profileWithAccounts()
|
||||
f.noPackage()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
os.RemoveAll(filepath.Join(f.Root, "proc", "4000"))
|
||||
// account1 has an alias identity on smtp1; smtp1 is the default server; Owl's account and local folders
|
||||
// have identities that name no outgoing server.
|
||||
prefs := strings.Replace(f.read(profileDir+"/prefs.js"), `"mail.account.account1.identities", "id1"`, `"mail.account.account1.identities", "id1,id1b"`, 1)
|
||||
prefs = strings.Replace(prefs, `"mail.account.account2.server", "server2");`, `"mail.account.account2.server", "server2");`+"\n"+
|
||||
`user_pref("mail.account.account2.identities", "id2");`, 1) +
|
||||
`user_pref("mail.identity.id1b.useremail", "alias@mail.example.org");` + "\n" +
|
||||
`user_pref("mail.identity.id1b.smtpServer", "smtp1");` + "\n" +
|
||||
`user_pref("mail.identity.id2.useremail", "local@mail.example.org");` + "\n" +
|
||||
`user_pref("mail.smtp.defaultserver", "smtp1");` + "\n"
|
||||
f.write(profileDir+"/prefs.js", prefs)
|
||||
f.declareAccounts(adoptable)
|
||||
if err := f.Prepare(); err != nil {
|
||||
t.Fatalf("an account's own alias, and identities that do not send by SMTP, are no sharing: %v", err)
|
||||
}
|
||||
// An identity of another mail account naming no server does use the default one.
|
||||
f.write(profileDir+"/prefs.js", strings.Replace(prefs, `"account1,account3,account2"`, `"account1,account3,account2,account6"`, 1)+
|
||||
`user_pref("mail.account.account6.server", "server6");`+"\n"+
|
||||
`user_pref("mail.account.account6.identities", "id6");`+"\n"+
|
||||
`user_pref("mail.identity.id6.useremail", "six@mail.example.org");`+"\n"+
|
||||
`user_pref("mail.server.server6.type", "pop3");`+"\n")
|
||||
os.Remove(filepath.Join(f.Root, testHome, recordFile))
|
||||
if err := f.Prepare(); err == nil || !strings.Contains(err.Error(), "also used by identity id6") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,330 @@
|
||||
package main
|
||||
|
||||
// Which account of the profile a declared account is (novox/hq ADR 0317): adopted, created or refused.
|
||||
//
|
||||
// At every apply, per declared account in its place in the setting (§1):
|
||||
//
|
||||
// 1. an account the module recorded on this node for this profile, adopted or created, is kept by its
|
||||
// recorded keys without matching again, while its account key is still in the profile's account list;
|
||||
// 2. otherwise an account under the module's own mesh_<name> keys is the module's creation;
|
||||
// 3. otherwise a node's layer may name the key to adopt (§8): adopted only when it is in the account list,
|
||||
// its default identity has the declared address and it is not Owl's;
|
||||
// 4. otherwise the profile's accounts whose default identity has the declared address: one is adopted, none
|
||||
// means the account is created under mesh_<name> keys (ADR 0316 §11).
|
||||
//
|
||||
// Refused, with the remedy, for that account alone (§7): two or more accounts with the address, the one that
|
||||
// has it being Owl's, a named key that does not qualify, an account of another kind than the declaration says
|
||||
// (the module never converts one: the review of ADR 0317), and an identity that names no outgoing server. The
|
||||
// other declared accounts go ahead.
|
||||
//
|
||||
// Adopting takes the account's keys as the module's (§2): its account key, its default identity, its incoming
|
||||
// server and the outgoing server that identity names. The account lists are left as they are, because the keys
|
||||
// are in them already; the declared fields are written under those keys (§4, §5). What was adopted or created
|
||||
// is recorded in the module's directory in the home, by the tools and never by a setting (§3); leaving the
|
||||
// setting releases an adopted account (its lines and its record go, the account stays the person's, §6).
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// recordFile is where the module records, per profile, what it adopted or created (ADR 0317 §3).
|
||||
const recordFile = ".config/betterbird/accounts-record.json"
|
||||
|
||||
// How a declared account resolved.
|
||||
const (
|
||||
adopted = "adopted"
|
||||
created = "created"
|
||||
refused = "refused"
|
||||
)
|
||||
|
||||
// keySet are an account's four keys.
|
||||
type keySet struct {
|
||||
Account string `json:"account"`
|
||||
Identity string `json:"identity"`
|
||||
Server string `json:"server"`
|
||||
SMTP string `json:"smtp"`
|
||||
}
|
||||
|
||||
// resolvedAccount is one declared account as it resolved, said by its place and keys, never its name, address
|
||||
// or host (ADR 0316 §16, ADR 0317 §3).
|
||||
type resolvedAccount struct {
|
||||
Place int `json:"place"`
|
||||
How string `json:"how"`
|
||||
Keys keySet `json:"keys,omitempty"`
|
||||
// Why is a refusal's reason and its remedy.
|
||||
Why string `json:"why,omitempty"`
|
||||
// Changed are the preferences an apply changed for it, by name only (ADR 0317 §4).
|
||||
Changed []string `json:"changed,omitempty"`
|
||||
name string
|
||||
}
|
||||
|
||||
// record is what the module adopted or created, per profile (its directory as the tools say it), per
|
||||
// declared account's name.
|
||||
type record map[string]map[string]recorded
|
||||
|
||||
type recorded struct {
|
||||
How string `json:"how"`
|
||||
Keys keySet `json:"keys"`
|
||||
}
|
||||
|
||||
func (m *Machine) readRecord(profileDir string) map[string]recorded {
|
||||
raw, err := readBounded(m.home(recordFile))
|
||||
if err != nil {
|
||||
return map[string]recorded{}
|
||||
}
|
||||
var r record
|
||||
if json.Unmarshal(raw, &r) != nil || r[profileDir] == nil {
|
||||
return map[string]recorded{}
|
||||
}
|
||||
return r[profileDir]
|
||||
}
|
||||
|
||||
// writeRecord keeps what this apply resolved for the profile: a refused account is not recorded, and an
|
||||
// account no longer declared is dropped (§6).
|
||||
func (m *Machine) writeRecord(profileDir string, resolved []resolvedAccount) error {
|
||||
r := record{}
|
||||
if raw, err := readBounded(m.home(recordFile)); err == nil {
|
||||
_ = json.Unmarshal(raw, &r)
|
||||
}
|
||||
mine := map[string]recorded{}
|
||||
for _, a := range resolved {
|
||||
if a.How != refused {
|
||||
mine[a.name] = recorded{How: a.How, Keys: a.Keys}
|
||||
}
|
||||
}
|
||||
if len(mine) == 0 {
|
||||
delete(r, profileDir)
|
||||
} else {
|
||||
r[profileDir] = mine
|
||||
}
|
||||
body, err := json.MarshalIndent(r, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path := m.home(recordFile)
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp := path + ".mesh-new"
|
||||
if err := os.WriteFile(tmp, append(body, '\n'), 0o600); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.Rename(tmp, path); err != nil {
|
||||
os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveAccounts resolves every declared account, in its place in the setting (the file's order, by name).
|
||||
func resolveAccounts(accounts map[string]Account, prefs map[string]string, rec map[string]recorded) []resolvedAccount {
|
||||
have := profileAccounts(prefs)
|
||||
inList := map[string]profileAccount{}
|
||||
for _, a := range have {
|
||||
inList[a.Key] = a
|
||||
}
|
||||
names := sortedNames(accounts)
|
||||
taken := map[string]bool{}
|
||||
var out []resolvedAccount
|
||||
for i, name := range names {
|
||||
r := resolveOne(i+1, len(names), name, accounts[name], prefs, have, inList, rec, taken)
|
||||
r.name = name
|
||||
if r.How != refused {
|
||||
taken[r.Keys.Account] = true
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return refuseSharedOutgoing(out, prefs, have)
|
||||
}
|
||||
|
||||
// refuseSharedOutgoing refuses an adopted account whose outgoing server another identity uses that is neither
|
||||
// adopted nor the module's (review 34 of mesh-catalog #232): Betterbird lets identities share an outgoing
|
||||
// server, and the declared values written into it would change that other account's sending at every start.
|
||||
// ADR 0317 §2 adopts the outgoing server the identity names and leaves the lists as they are, so the module
|
||||
// does not give the identity a server of its own; it refuses, with the remedy.
|
||||
func refuseSharedOutgoing(resolved []resolvedAccount, prefs map[string]string, have []profileAccount) []resolvedAccount {
|
||||
// Every identity of an adopted or created account is its own, its aliases too (review 36).
|
||||
accountOf := map[string]profileAccount{}
|
||||
kindOf := map[string]string{}
|
||||
for _, a := range have {
|
||||
accountOf[a.Key] = a
|
||||
for _, id := range a.Identities {
|
||||
kindOf[id] = a.Kind
|
||||
}
|
||||
}
|
||||
ours := map[string]bool{}
|
||||
for _, r := range resolved {
|
||||
if r.How == refused {
|
||||
continue
|
||||
}
|
||||
ours[r.Keys.Identity] = true
|
||||
for _, id := range accountOf[r.Keys.Account].Identities {
|
||||
ours[id] = true
|
||||
}
|
||||
}
|
||||
defaultServer := unquote(prefs["mail.smtp.defaultserver"])
|
||||
users := map[string][]string{}
|
||||
seen := map[string]bool{}
|
||||
use := func(id string) {
|
||||
if seen[id] || ours[id] {
|
||||
return
|
||||
}
|
||||
seen[id] = true
|
||||
server := unquote(prefs["mail.identity."+id+".smtpServer"])
|
||||
// Only a mail account's identity sends through the default server when it names none: Owl's, local
|
||||
// folders and feeds do not send by SMTP (review 36).
|
||||
if server == "" && (kindOf[id] == "imap" || kindOf[id] == "pop3") {
|
||||
server = defaultServer
|
||||
}
|
||||
if server != "" {
|
||||
users[server] = append(users[server], id)
|
||||
}
|
||||
}
|
||||
for _, a := range have {
|
||||
for _, id := range a.Identities {
|
||||
use(id)
|
||||
}
|
||||
}
|
||||
for k := range prefs {
|
||||
if strings.HasPrefix(k, "mail.identity.") && strings.HasSuffix(k, ".smtpServer") {
|
||||
use(strings.TrimSuffix(strings.TrimPrefix(k, "mail.identity."), ".smtpServer"))
|
||||
}
|
||||
}
|
||||
for i, r := range resolved {
|
||||
if r.How != adopted {
|
||||
continue
|
||||
}
|
||||
if others := users[r.Keys.SMTP]; len(others) > 0 {
|
||||
sort.Strings(others)
|
||||
resolved[i] = resolvedAccount{Place: r.Place, How: refused, name: r.name, Why: fmt.Sprintf(
|
||||
"declared account %d of %d: the outgoing server %s of the profile's account %s is also used by "+
|
||||
"identity %s, whose sending the declared values would change: give the account an outgoing "+
|
||||
"server of its own in Betterbird's account settings, then start Betterbird again",
|
||||
r.Place, len(resolved), r.Keys.SMTP, r.Keys.Account, strings.Join(others, ", "))}
|
||||
}
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
func resolveOne(place, of int, name string, a Account, prefs map[string]string, have []profileAccount,
|
||||
inList map[string]profileAccount, rec map[string]recorded, taken map[string]bool) resolvedAccount {
|
||||
r := resolvedAccount{Place: place}
|
||||
refuse := func(format string, args ...any) resolvedAccount {
|
||||
r.How, r.Keys = refused, keySet{}
|
||||
r.Why = fmt.Sprintf("declared account %d of %d: ", place, of) + fmt.Sprintf(format, args...)
|
||||
return r
|
||||
}
|
||||
// 1. Recorded, while its account is still in the list.
|
||||
if was, ok := rec[name]; ok {
|
||||
if _, there := inList[was.Keys.Account]; there || was.How == created {
|
||||
r.How, r.Keys = was.How, was.Keys
|
||||
if was.How == adopted {
|
||||
// The identity's outgoing server as it is now: a person may have moved it since.
|
||||
smtpKey := unquote(prefs["mail.identity."+was.Keys.Identity+".smtpServer"])
|
||||
if smtpKey == "" {
|
||||
return refuse("the profile's account %s has an identity that names no outgoing server: choose one "+
|
||||
"for it in Betterbird's account settings, then start Betterbird again", was.Keys.Account)
|
||||
}
|
||||
r.Keys.SMTP = smtpKey
|
||||
}
|
||||
return r
|
||||
}
|
||||
}
|
||||
// 2. Under the module's own keys: its creation.
|
||||
account, identity, server, smtp := keysOf(name)
|
||||
own := keySet{account, identity, server, smtp}
|
||||
if _, there := inList[account]; there {
|
||||
r.How, r.Keys = created, own
|
||||
return r
|
||||
}
|
||||
adopt := func(pa profileAccount) resolvedAccount {
|
||||
if pa.Kind != a.Incoming.Kind {
|
||||
return refuse("the profile's account %s is of kind %s and the declaration says %s: the module never "+
|
||||
"converts an account. Declare the kind the account has, or remove the account in Betterbird",
|
||||
pa.Key, pa.Kind, a.Incoming.Kind)
|
||||
}
|
||||
smtpKey := unquote(prefs["mail.identity."+pa.Identities[0]+".smtpServer"])
|
||||
if smtpKey == "" {
|
||||
return refuse("the profile's account %s has an identity that names no outgoing server: choose one for it "+
|
||||
"in Betterbird's account settings, then start Betterbird again", pa.Key)
|
||||
}
|
||||
r.How, r.Keys = adopted, keySet{pa.Key, pa.Identities[0], pa.Server, smtpKey}
|
||||
return r
|
||||
}
|
||||
matchesDefault := func(pa profileAccount) bool {
|
||||
return len(pa.Addresses) > 0 && strings.EqualFold(pa.Addresses[0], a.Address)
|
||||
}
|
||||
// 3. The key a node's layer names.
|
||||
if a.Key != "" {
|
||||
pa, there := inList[a.Key]
|
||||
switch {
|
||||
case !there:
|
||||
return refuse("the key %s this node's layer names is not in the profile's account list", a.Key)
|
||||
case pa.Owl:
|
||||
return refuse("the key %s this node's layer names is Owl's account, which is never declared "+
|
||||
"(novox/hq ADR 0316 §13)", a.Key)
|
||||
case !matchesDefault(pa):
|
||||
return refuse("the key %s this node's layer names is an account whose default identity has another "+
|
||||
"address", a.Key)
|
||||
case taken[pa.Key]:
|
||||
return refuse("the key %s is already another declared account's", a.Key)
|
||||
}
|
||||
return adopt(pa)
|
||||
}
|
||||
// 4. By the default identity's address.
|
||||
var matches []profileAccount
|
||||
for _, pa := range have {
|
||||
if !pa.Ours && !taken[pa.Key] && matchesDefault(pa) {
|
||||
matches = append(matches, pa)
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case len(matches) == 0:
|
||||
r.How, r.Keys = created, own
|
||||
return r
|
||||
case len(matches) > 1:
|
||||
var keys []string
|
||||
for _, pa := range matches {
|
||||
keys = append(keys, pa.Key)
|
||||
}
|
||||
return refuse("%d accounts in the profile have its address (%s): remove the duplicate in Betterbird, or "+
|
||||
"give the one to adopt as key in this node's layer of the setting accounts (novox/hq ADR 0317 §7, §8)",
|
||||
len(matches), strings.Join(keys, ", "))
|
||||
case matches[0].Owl:
|
||||
return refuse("the profile's account with its address, %s, is Owl's, which is never declared (novox/hq "+
|
||||
"ADR 0316 §13): take it out of the setting accounts", matches[0].Key)
|
||||
}
|
||||
return adopt(matches[0])
|
||||
}
|
||||
|
||||
// changedPrefs are the preferences of a resolved account whose declared value differs from the profile's
|
||||
// saved one (prefs.js), by name only (ADR 0317 §4). A preference prefs.js does not hold is changed unless the
|
||||
// declared value is what Betterbird reads in its place.
|
||||
func changedPrefs(r resolvedAccount, a Account, saved map[string]string) []string {
|
||||
defaults := map[string]string{}
|
||||
if r.How == adopted {
|
||||
// What Betterbird reads when prefs.js holds no value: the same defaults betterbird_accounts reads with.
|
||||
pa := profileAccount{Key: r.Keys.Account, Server: r.Keys.Server, Kind: a.Incoming.Kind,
|
||||
Identities: []string{r.Keys.Identity}}
|
||||
view := readAccount(saved, pa)
|
||||
for _, p := range declaredPrefs(r.Keys, view, true) {
|
||||
defaults[p[0]] = p[1]
|
||||
}
|
||||
}
|
||||
var out []string
|
||||
for _, p := range declaredPrefs(r.Keys, a, r.How == adopted) {
|
||||
have, ok := saved[p[0]]
|
||||
if !ok {
|
||||
have, ok = defaults[p[0]]
|
||||
}
|
||||
if !ok || have != p[1] {
|
||||
out = append(out, p[0])
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -10,19 +10,23 @@ package main
|
||||
// (passwordSecret, accountPrefix);
|
||||
// - the profile's extensions.json, for the add-ons' ids and whether each is on;
|
||||
// - the module's declared user.js (~/.config/betterbird/user.js), which the node-engine writes from
|
||||
// the module's setting `prefs`.
|
||||
// the module's setting `prefs`, and its declared accounts (~/.config/betterbird/accounts.json, the
|
||||
// setting `accounts`, accounts.go);
|
||||
// - of logins.json, only the server name each kept sign-in is for, which Betterbird keeps in the clear,
|
||||
// to count the declared accounts that have not signed in yet (novox/hq ADR 0316 §16).
|
||||
//
|
||||
// Never read: the mail store, the address books, logins.json, key4.db, cert9.db, cookies, the
|
||||
// calendar's caches. Nothing of the operator's mail, calendar or accounts is answered.
|
||||
// Never read: the mail store, the address books, a user name or password in logins.json, key4.db,
|
||||
// cert9.db, cookies, the calendar's caches. Nothing of the operator's mail or calendar is answered, and of
|
||||
// their accounts only counts and kinds.
|
||||
//
|
||||
// The one file a tool writes in the profile is user.js, and only the module's own: the declared file,
|
||||
// copied whole (betterbird_user_js with apply, betterbird_restart). A user.js that is not the module's
|
||||
// is never overwritten.
|
||||
// The files a tool writes in the profile are user.js, and only the module's own (the declared file and
|
||||
// the declared accounts' preferences, betterbird_user_js with apply, betterbird_restart, and the start
|
||||
// from the launcher), and the module's own keys in the account lists of prefs.js, only while Betterbird is
|
||||
// not running (accounts.go). A user.js that is not the module's is never overwritten.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -35,8 +39,12 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
mainComm = "betterbird-bin"
|
||||
startBin = "/usr/bin/betterbird"
|
||||
mainComm = "betterbird-bin"
|
||||
// startBin is Betterbird itself, where the mesh's release and the community package both put it. The
|
||||
// module's start (/usr/local/bin/betterbird) runs the preparation first; a tool that starts Betterbird
|
||||
// prepares itself and starts this.
|
||||
startBin = installDir + "/betterbird"
|
||||
// packageFor is the community package the mesh's release replaces (novox/hq ADR 0316 §4).
|
||||
packageFor = "betterbird-bin"
|
||||
appIni = "/opt/betterbird/application.ini"
|
||||
restartAs = "mesh-betterbird"
|
||||
@@ -262,6 +270,11 @@ var (
|
||||
allowedExtensions = map[string]bool{"extensions.activeThemeID": true}
|
||||
)
|
||||
|
||||
// accountOwned are the preferences of an account and of the account lists: the setting accounts' alone, so
|
||||
// the setting prefs refuses them and an account has one owner (novox/hq ADR 0316 §11).
|
||||
var accountOwned = []string{"mail.account.", "mail.identity.", "mail.server.", "mail.smtpserver.",
|
||||
"mail.accountmanager.", "mail.smtpservers", "mail.smtp.defaultserver"}
|
||||
|
||||
// withheld says why a preference's value is never answered, or "" when it may be.
|
||||
func withheld(name string) string {
|
||||
if strings.HasPrefix(name, "extensions.") && !allowedExtensions[name] {
|
||||
@@ -392,6 +405,12 @@ func validUserJS(text string) error {
|
||||
if passwordSecret.MatchString(name) {
|
||||
return fmt.Errorf("line %d sets %s, a preference that can hold a credential: not through the mesh", i+1, name)
|
||||
}
|
||||
for _, p := range accountOwned {
|
||||
if strings.HasPrefix(name, p) {
|
||||
return fmt.Errorf("line %d sets %s, an account's preference: an account is declared in the setting "+
|
||||
"accounts, never in prefs (novox/hq ADR 0316 §11)", i+1, name)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
@@ -433,13 +452,35 @@ type UserJS struct {
|
||||
Effective string `json:"effective,omitempty"`
|
||||
Content string `json:"content,omitempty"`
|
||||
Applied bool `json:"applied,omitempty"`
|
||||
// Accounts are how many accounts the setting accounts declares, and what is wrong with them.
|
||||
Accounts int `json:"declared_accounts"`
|
||||
AccountsProblem string `json:"accounts_problem,omitempty"`
|
||||
// Each is every declared account by its place: adopted, created or refused (with why), its keys, and the
|
||||
// preferences an apply changed for it, by name (novox/hq ADR 0317 §3, §4).
|
||||
Each []resolvedAccount `json:"accounts,omitempty"`
|
||||
// Lists says whether the keys of the accounts the module created were written into the account lists.
|
||||
Lists string `json:"account_lists,omitempty"`
|
||||
}
|
||||
|
||||
// userJS compares the declared user.js with the profile's; apply places the declared one when the
|
||||
// profile has none or the module's.
|
||||
// refusedAccounts are the reasons of the declared accounts that were refused.
|
||||
func (u UserJS) refusedAccounts() []string {
|
||||
var out []string
|
||||
for _, r := range u.Each {
|
||||
if r.How == refused {
|
||||
out = append(out, r.Why)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// userJS compares the user.js the module places — the declared file, then the declared accounts' fields under
|
||||
// the keys each was adopted or created under (adoption.go) — with the profile's; apply places it when the
|
||||
// profile has none or the module's, records what was adopted or created, and writes the keys of the accounts
|
||||
// it created into the profile's account lists when Betterbird is not running. A refused account gets nothing;
|
||||
// the others go ahead (novox/hq ADR 0317 §7).
|
||||
func (m *Machine) userJS(apply bool) (UserJS, error) {
|
||||
a := UserJS{Declared: "~/" + declared}
|
||||
want, err := readBounded(m.home(declared))
|
||||
declaredText, err := readBounded(m.home(declared))
|
||||
a.Present = err == nil
|
||||
p, perr := m.profile()
|
||||
if perr != nil {
|
||||
@@ -447,11 +488,21 @@ func (m *Machine) userJS(apply bool) (UserJS, error) {
|
||||
}
|
||||
target := filepath.Join(p.path, "user.js")
|
||||
a.Profile = m.tilde(strings.TrimPrefix(target, m.Root))
|
||||
accounts, _, aerr := m.readAccounts()
|
||||
if aerr != nil {
|
||||
a.AccountsProblem = aerr.Error()
|
||||
accounts = nil
|
||||
} else {
|
||||
a.Accounts = len(accounts)
|
||||
a.Each = resolveAccounts(accounts, mergedPrefs(p.path), m.readRecord(p.Dir))
|
||||
}
|
||||
var want []byte
|
||||
if a.Present {
|
||||
if verr := validUserJS(string(want)); verr != nil {
|
||||
if verr := validUserJS(string(declaredText)); verr != nil {
|
||||
a.Valid = verr.Error()
|
||||
}
|
||||
a.Content = string(want)
|
||||
want = []byte(composeUserJS(string(declaredText), accounts, a.Each))
|
||||
a.Content = m.answeredUserJS(string(want))
|
||||
for _, line := range strings.Split(string(want), "\n") {
|
||||
for _, st := range splitStatements(strings.TrimSpace(line)) {
|
||||
if statement.MatchString(st) {
|
||||
@@ -477,6 +528,17 @@ func (m *Machine) userJS(apply bool) (UserJS, error) {
|
||||
return a, fmt.Errorf("the module's user.js is not on this machine (~/%s): the module is not pushed here yet", declared)
|
||||
case a.Valid != "":
|
||||
return a, fmt.Errorf("the module's user.js is refused: %s; change the setting prefs", a.Valid)
|
||||
case a.AccountsProblem != "":
|
||||
return a, fmt.Errorf("nothing was placed: %s", a.AccountsProblem)
|
||||
}
|
||||
saved, _ := readBounded(filepath.Join(p.path, "prefs.js"))
|
||||
savedPrefs := readPrefs(saved)
|
||||
for i, r := range a.Each {
|
||||
if r.How != refused {
|
||||
a.Each[i].Changed = changedPrefs(r, accounts[r.name], savedPrefs)
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case herr == nil && !bytes.HasPrefix(have, []byte(mark)):
|
||||
return a, fmt.Errorf("%s is not the module's: move it aside (its preferences belong in the setting prefs), then apply again", a.Profile)
|
||||
}
|
||||
@@ -491,6 +553,14 @@ func (m *Machine) userJS(apply bool) (UserJS, error) {
|
||||
}
|
||||
}
|
||||
a.Applied, a.Placed = true, "the module's, as declared"
|
||||
if err := m.writeRecord(p.Dir, a.Each); err != nil {
|
||||
return a, fmt.Errorf("the user.js is placed, and what was adopted or created could not be recorded: %v", err)
|
||||
}
|
||||
if lists, err := m.writeAccountLists(p.path, a.Each); err != nil {
|
||||
a.Lists = "not written: " + err.Error()
|
||||
} else {
|
||||
a.Lists = lists
|
||||
}
|
||||
}
|
||||
if ps := m.mainProcs(); len(ps) > 0 {
|
||||
if st, err := os.Stat(target); err == nil {
|
||||
@@ -504,27 +574,74 @@ func (m *Machine) userJS(apply bool) (UserJS, error) {
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// answeredUserJS is the user.js as a tool answers it (novox/hq ADR 0316 §16, ADR 0317 §9): an account's
|
||||
// preferences with their values withheld, every other value masked as betterbird_prefs masks it.
|
||||
func (m *Machine) answeredUserJS(text string) string {
|
||||
lines := strings.Split(text, "\n")
|
||||
for i, line := range lines {
|
||||
for _, st := range splitStatements(strings.TrimSpace(line)) {
|
||||
mm := prefLine.FindStringSubmatch(st)
|
||||
if mm == nil {
|
||||
continue
|
||||
}
|
||||
value := m.maskValue(mm[2])
|
||||
if why := withheld(mm[1]); why != "" {
|
||||
value = `"<withheld>"`
|
||||
}
|
||||
lines[i] = fmt.Sprintf("user_pref(%s, %s);", quoted(mm[1]), value)
|
||||
}
|
||||
}
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
// Prepare is what the module's start runs before Betterbird (`betterbird-tools prepare`): with no Betterbird
|
||||
// of this account running, the user.js placed and the declared accounts written into the account lists. With
|
||||
// one running it does nothing, since the start then only hands over to it. Its error is said by the start,
|
||||
// which starts Betterbird either way.
|
||||
func (m *Machine) Prepare() error {
|
||||
if len(m.mainProcs()) > 0 {
|
||||
return nil
|
||||
}
|
||||
u, err := m.userJS(true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var problems []string
|
||||
if strings.HasPrefix(u.Lists, "not written") {
|
||||
problems = append(problems, "the account lists were "+u.Lists)
|
||||
}
|
||||
problems = append(problems, u.refusedAccounts()...)
|
||||
if len(problems) > 0 {
|
||||
return fmt.Errorf("%s", strings.Join(problems, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Status is what betterbird_status answers.
|
||||
type Status struct {
|
||||
Running []Proc `json:"running"`
|
||||
Children int `json:"children"`
|
||||
Installed string `json:"installed,omitempty"`
|
||||
Running []Proc `json:"running"`
|
||||
Children int `json:"children"`
|
||||
// Installation is what is installed and from where, the policy and Owl (novox/hq ADR 0316 §16).
|
||||
Installation Installation `json:"installation"`
|
||||
// Installed is the community package's version, while it is still there.
|
||||
Installed string `json:"community_package,omitempty"`
|
||||
Foreign bool `json:"from_the_aur,omitempty"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Build string `json:"build,omitempty"`
|
||||
Profile *Profile `json:"profile,omitempty"`
|
||||
Problem string `json:"profile_problem,omitempty"`
|
||||
Output []Stream `json:"output,omitempty"`
|
||||
UserJS *UserJS `json:"user_js,omitempty"`
|
||||
AddOns []AddOn `json:"add_ons,omitempty"`
|
||||
// Accounts are counts: the declared, those in the profile's list, the others, those without a sign-in.
|
||||
Accounts *AccountCount `json:"accounts,omitempty"`
|
||||
AddOns []AddOn `json:"add_ons,omitempty"`
|
||||
// Display are the preferences that decide how its windows and notifications look.
|
||||
Display []Pref `json:"display_prefs,omitempty"`
|
||||
}
|
||||
|
||||
// AddOn is one add-on the operator installed: its id, and whether it is on.
|
||||
type AddOn struct {
|
||||
ID string `json:"id"`
|
||||
Active bool `json:"active"`
|
||||
ID string `json:"id"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Active bool `json:"active"`
|
||||
}
|
||||
|
||||
// displayPrefs are the preferences betterbird_status answers: the reminder dialog, the theme, the
|
||||
@@ -550,39 +667,24 @@ func (m *Machine) Status() (Status, error) {
|
||||
if v, err := m.installed(packageFor); err == nil && v != "" {
|
||||
s.Installed, s.Foreign = v, m.foreign(packageFor)
|
||||
}
|
||||
if raw, err := readBounded(m.path(appIni)); err == nil {
|
||||
_, sec := iniSections(raw)
|
||||
s.Version, s.Build = sec["App"]["Version"], sec["App"]["BuildID"]
|
||||
}
|
||||
if len(s.Running) > 0 {
|
||||
s.Output = m.streams(s.Running[0].PID)
|
||||
}
|
||||
p, err := m.profile()
|
||||
if err != nil {
|
||||
s.Installation = m.installation("")
|
||||
s.Problem = err.Error()
|
||||
return s, nil
|
||||
}
|
||||
s.Profile = &p
|
||||
s.Installation = m.installation(p.path)
|
||||
if u, err := m.userJS(false); err == nil {
|
||||
u.Content = ""
|
||||
s.UserJS = &u
|
||||
}
|
||||
if raw, err := readBounded(filepath.Join(p.path, "extensions.json")); err == nil {
|
||||
var ext struct {
|
||||
Addons []struct {
|
||||
ID string `json:"id"`
|
||||
Active bool `json:"active"`
|
||||
Location string `json:"location"`
|
||||
} `json:"addons"`
|
||||
}
|
||||
if json.Unmarshal(raw, &ext) == nil {
|
||||
for _, a := range ext.Addons {
|
||||
if a.Location == "app-profile" {
|
||||
s.AddOns = append(s.AddOns, AddOn{a.ID, a.Active})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
c := m.countAccounts(p.path, p.Dir)
|
||||
s.Accounts = &c
|
||||
s.AddOns = addOns(p.path)
|
||||
if got, err := m.Prefs(displayPrefs); err == nil {
|
||||
s.Display = got["prefs"].([]Pref)
|
||||
}
|
||||
@@ -711,7 +813,8 @@ func (m *Machine) Restart(discard bool) (RestartAnswer, error) {
|
||||
if u, err := m.userJS(true); err != nil {
|
||||
a.UserJS = "not placed: " + err.Error()
|
||||
} else {
|
||||
a.UserJS = fmt.Sprintf("placed: %d preference(s) from ~/%s", u.Statements, declared)
|
||||
a.UserJS = fmt.Sprintf("placed: %d preference(s) from ~/%s, %d declared account(s); account lists %s",
|
||||
u.Statements, declared, u.Accounts, u.Lists)
|
||||
}
|
||||
// As detach does, with shorter bounds: the worst case of the whole call (3 s for the window tree,
|
||||
// 8 s to quit, 3 + 5 s to start, 4 s to see it) stays under the runtime's 30 s.
|
||||
@@ -750,12 +853,56 @@ type CheckAnswer struct {
|
||||
func (m *Machine) Check() (CheckAnswer, error) {
|
||||
a := CheckAnswer{Findings: []Finding{}}
|
||||
add := func(what, do string) { a.Findings = append(a.Findings, Finding{what, do}) }
|
||||
v, err := m.installed(packageFor)
|
||||
if err != nil {
|
||||
if _, err := m.installed(packageFor); err != nil {
|
||||
return a, err
|
||||
}
|
||||
if v == "" {
|
||||
add("Betterbird ("+packageFor+") is not installed", "install it from the AUR: the node-engine installs packages from the official repositories only, so the module does not")
|
||||
p, perr := m.profile()
|
||||
profilePath := ""
|
||||
if perr == nil {
|
||||
profilePath = p.path
|
||||
}
|
||||
in := m.installation(profilePath)
|
||||
switch {
|
||||
case in.From == "not installed":
|
||||
add("Betterbird is not installed in "+installDir, "the module's push places the mesh's release; it waits for the operator's push (upgrade policy record)")
|
||||
case in.From != "the mesh":
|
||||
add("Betterbird is "+in.From, "quit Betterbird, then push the module to this node: the push removes the community package and places the mesh's release")
|
||||
}
|
||||
if in.Behind {
|
||||
add("Betterbird "+in.Version+" (build "+in.Build+") is behind the newest published build "+in.Newest,
|
||||
"move the module's pin in a reviewed change (novox/hq ADR 0316 §5)")
|
||||
}
|
||||
if in.From == "the mesh" && !strings.HasPrefix(in.Policy, "placed:") {
|
||||
add("the enterprise policy is "+in.Policy, "push the module to this node")
|
||||
}
|
||||
switch {
|
||||
case in.From != "the mesh":
|
||||
case in.Owl == nil || in.Owl.Pinned == "":
|
||||
add("Owl's xpi is not placed ("+owlFile+")", "push the module to this node")
|
||||
case in.Owl.InProfile == "":
|
||||
add("Owl is not installed in the profile yet", "Betterbird installs it from the policy at its next start")
|
||||
case !in.Owl.AtThePin && !newerVersion(in.Owl.InProfile, in.Owl.Pinned):
|
||||
add("Owl "+in.Owl.InProfile+" runs in the profile, older than the mesh's pin "+in.Owl.Pinned,
|
||||
"Owl is installed by policy and cannot be removed in Betterbird: move the module's Owl pin in a reviewed change (novox/hq ADR 0316 §6)")
|
||||
case !in.Owl.Active:
|
||||
add("Owl is installed and not active", "turn it on in Betterbird's add-ons")
|
||||
}
|
||||
if perr == nil {
|
||||
c := m.countAccounts(p.path, p.Dir)
|
||||
for _, r := range c.Each {
|
||||
if r.How == refused {
|
||||
add("refused: "+r.Why, "as it says; the other declared accounts go ahead")
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case c.InList < c.Declared:
|
||||
add(fmt.Sprintf("%d of %d declared account(s) are not in the profile's account list yet", c.Declared-c.InList, c.Declared),
|
||||
"quit Betterbird and start it from the launcher, or betterbird_restart")
|
||||
}
|
||||
if c.WithoutSignIn > 0 {
|
||||
add(fmt.Sprintf("%d declared account(s) have not signed in yet", c.WithoutSignIn),
|
||||
"a person signs in once, in Betterbird's own window, when it asks (novox/hq ADR 0316 §14)")
|
||||
}
|
||||
}
|
||||
ps := m.mainProcs()
|
||||
switch {
|
||||
@@ -772,6 +919,8 @@ func (m *Machine) Check() (CheckAnswer, error) {
|
||||
add("the module's user.js is not on this machine (~/"+declared+")", "the module is not pushed here: the controller's push")
|
||||
case u.Valid != "":
|
||||
add("the module's user.js is refused: "+u.Valid, "change the module's setting prefs")
|
||||
case u.AccountsProblem != "":
|
||||
add("the module's user.js is not placed: "+u.AccountsProblem, "change the module's setting accounts at the controller's terminal, or propose it (betterbird_accounts)")
|
||||
case !strings.HasPrefix(u.Placed, "the module's, as declared"):
|
||||
add("the profile's user.js is "+u.Placed, "betterbird_user_js with apply, then betterbird_restart")
|
||||
case strings.HasPrefix(u.Effective, "not yet"):
|
||||
|
||||
@@ -161,7 +161,7 @@ func TestStatusSaysWhatRunsTheProfileTheAddOnsAndTheDisplayPrefs(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(s.Running) != 1 || s.Running[0].PID != 4000 || s.Children != 1 || s.Installed != "153.1.0esr-1" || !s.Foreign ||
|
||||
s.Version != "153.1.0" || s.Profile == nil || len(s.AddOns) != 1 || s.AddOns[0].ID != "owl@beonex.com" {
|
||||
s.Installation.Version != "153.1.0" || s.Profile == nil || len(s.AddOns) != 1 || s.AddOns[0].ID != "owl@beonex.com" {
|
||||
t.Fatalf("%+v", s)
|
||||
}
|
||||
if !s.Output[0].Dead {
|
||||
@@ -238,6 +238,13 @@ func TestTheReminderTestTouchesNothingOfTheOperatorsProfile(t *testing.T) {
|
||||
!strings.Contains(string(user), "file:///run/user/1000/"+testUnit+"/test.ics") || strings.Contains(string(user), "someone@") {
|
||||
t.Fatalf("%s", user)
|
||||
}
|
||||
// Betterbird's update check stays off, so no About window opens on the operator's screen, and the
|
||||
// test's own preferences come last, so nothing carried or declared overrides them.
|
||||
if !strings.Contains(string(user), `"betterbird.update.checkIntervalDays", 0`) ||
|
||||
strings.Index(string(user), "extensions.activeThemeID") > strings.Index(string(user), "calendar.registry.meshtest.uri") ||
|
||||
strings.Index(string(user), `"calendar.alarms.show"`) > strings.Index(string(user), "calendar.registry.meshtest.uri") {
|
||||
t.Fatalf("%s", user)
|
||||
}
|
||||
// The event starts 10 minutes after the reminder, which is due 40 s from now.
|
||||
if !strings.Contains(string(cal), "DTSTART:20270115T081040Z") || !strings.Contains(string(cal), "TRIGGER:-PT10M") ||
|
||||
a.Due != "2027-01-15T08:00:40Z" {
|
||||
|
||||
@@ -0,0 +1,253 @@
|
||||
package main
|
||||
|
||||
// What the mesh installs (novox/hq ADR 0316 §1–§9, to-be 56 §1–§3): Betterbird from its pinned upstream release
|
||||
// in /opt/betterbird, where the community package betterbird-bin put it, so Betterbird keeps choosing the same
|
||||
// profile; the enterprise policy in its distribution directory, which turns Betterbird's own updater off and
|
||||
// installs Owl into every profile from the xpi the mesh placed beside it; and a defaults file naming the
|
||||
// system's spelling dictionaries. The tools read these files and say what they find: the version against the
|
||||
// pin and the newest published build, whether the policy is placed, Owl's version against its pin.
|
||||
//
|
||||
// **The pin on the machine is what the mesh placed.** The manifest pins the release and the xpi by sha256; the
|
||||
// node-engine places exactly those bytes, so the version in /opt/betterbird/application.ini (when the mesh
|
||||
// placed it) and the version in the placed xpi are the pins. The community package's own copy is said as
|
||||
// such, and the switch from it waits for the operator (ADR 0316 §4).
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
installDir = "/opt/betterbird"
|
||||
// policyFile is the enterprise policy the module places (ADR 0316 §7).
|
||||
policyFile = installDir + "/distribution/policies.json"
|
||||
// owlFile is Owl's xpi as the mesh placed it, and the URL the policy installs it from.
|
||||
owlFile = installDir + "/owl/owl.xpi"
|
||||
owlID = "owl@beonex.com"
|
||||
// preparation is the tools' program as the module places it for its start (betterbird-tools prepare).
|
||||
preparation = "/usr/local/lib/mesh-betterbird/betterbird-tools"
|
||||
// versionsURL is where Betterbird publishes its newest build per major version: what its own updater
|
||||
// reads (MailGlue's BetterbirdUpdateCheck), and what the status compares the installed build with.
|
||||
versionsURL = "https://www.betterbird.eu/start/versions.txt"
|
||||
)
|
||||
|
||||
// publishedBuilds fetches Betterbird's list of newest builds: a variable so a test answers.
|
||||
var publishedBuilds = func() (string, error) {
|
||||
client := &http.Client{Timeout: 5 * time.Second}
|
||||
resp, err := client.Get(versionsURL)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return "", fmt.Errorf("%s answered %s", versionsURL, resp.Status)
|
||||
}
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
return string(body), err
|
||||
}
|
||||
|
||||
// Installation is what is installed, and from where.
|
||||
type Installation struct {
|
||||
// From is the mesh, the community package (betterbird-bin, before the switch), or nothing.
|
||||
From string `json:"from"`
|
||||
Version string `json:"version,omitempty"`
|
||||
Build string `json:"build,omitempty"`
|
||||
// Newest is the newest build Betterbird published for this major version, and whether this one is behind.
|
||||
Newest string `json:"newest_published_build,omitempty"`
|
||||
Behind bool `json:"behind,omitempty"`
|
||||
// Policy says whether the module's enterprise policy is placed, and what it does.
|
||||
Policy string `json:"policy"`
|
||||
// Owl is the add-on's version as the mesh placed it (the pin), and as the profile runs it.
|
||||
Owl *OwlState `json:"owl,omitempty"`
|
||||
}
|
||||
|
||||
// OwlState is Owl against its pin.
|
||||
type OwlState struct {
|
||||
Pinned string `json:"pinned,omitempty"`
|
||||
InProfile string `json:"in_profile,omitempty"`
|
||||
Active bool `json:"active"`
|
||||
AtThePin bool `json:"at_the_pin"`
|
||||
// Note says, as a fact, a profile running an Owl newer than the pin: the policy's install keeps a newer
|
||||
// version, and Owl installed by policy cannot be removed in Betterbird.
|
||||
Note string `json:"note,omitempty"`
|
||||
}
|
||||
|
||||
// newerVersion says whether version a is newer than b, comparing dotted numbers.
|
||||
func newerVersion(a, b string) bool {
|
||||
if a == "" || b == "" {
|
||||
return false
|
||||
}
|
||||
pa, pb := strings.Split(a, "."), strings.Split(b, ".")
|
||||
for i := 0; i < len(pa) || i < len(pb); i++ {
|
||||
var x, y int
|
||||
if i < len(pa) {
|
||||
fmt.Sscanf(pa[i], "%d", &x)
|
||||
}
|
||||
if i < len(pb) {
|
||||
fmt.Sscanf(pb[i], "%d", &y)
|
||||
}
|
||||
if x != y {
|
||||
return x > y
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// installation reads what is installed. profile may be empty when none is found yet.
|
||||
func (m *Machine) installation(profile string) Installation {
|
||||
var in Installation
|
||||
community, _ := m.installed(packageFor)
|
||||
raw, err := readBounded(m.path(appIni))
|
||||
switch {
|
||||
case community != "":
|
||||
in.From = "the community package " + packageFor + " " + community + ": the switch to the mesh's release waits for the operator's push, with Betterbird quit (novox/hq ADR 0316 §4)"
|
||||
case err == nil:
|
||||
in.From = "the mesh"
|
||||
default:
|
||||
in.From = "not installed"
|
||||
}
|
||||
if err == nil {
|
||||
_, sec := iniSections(raw)
|
||||
in.Version, in.Build = sec["App"]["Version"], sec["App"]["BuildID"]
|
||||
in.Newest, in.Behind = newestBuild(in.Version, in.Build)
|
||||
}
|
||||
in.Policy = m.policyState()
|
||||
pinned := xpiVersion(m.path(owlFile))
|
||||
owl := &OwlState{Pinned: pinned}
|
||||
if profile != "" {
|
||||
for _, a := range addOns(profile) {
|
||||
if a.ID == owlID {
|
||||
owl.InProfile, owl.Active = a.Version, a.Active
|
||||
}
|
||||
}
|
||||
}
|
||||
owl.AtThePin = owl.Pinned != "" && owl.Pinned == owl.InProfile
|
||||
if newerVersion(owl.InProfile, owl.Pinned) {
|
||||
owl.Note = "the profile runs Owl " + owl.InProfile + ", newer than the mesh's pin " + owl.Pinned + ": the policy " +
|
||||
"keeps a newer version and installs the pin only where Owl is missing. To run the same Owl everywhere, " +
|
||||
"move the module's pin in a reviewed change"
|
||||
}
|
||||
if owl.Pinned != "" || owl.InProfile != "" {
|
||||
in.Owl = owl
|
||||
}
|
||||
return in
|
||||
}
|
||||
|
||||
// newestBuild is the newest build Betterbird published for a version's major, and whether build is older.
|
||||
func newestBuild(version, build string) (string, bool) {
|
||||
major, _, _ := strings.Cut(version, ".")
|
||||
if major == "" || build == "" {
|
||||
return "", false
|
||||
}
|
||||
list, err := publishedBuilds()
|
||||
if err != nil {
|
||||
return "unknown: " + err.Error(), false
|
||||
}
|
||||
for _, l := range strings.Split(list, "\n") {
|
||||
f := strings.Split(strings.TrimSpace(l), ",")
|
||||
if len(f) == 3 && f[0] == "Linux" && f[1] == major {
|
||||
return f[2], f[2] > build
|
||||
}
|
||||
}
|
||||
return "unknown: " + versionsURL + " names no build for " + major, false
|
||||
}
|
||||
|
||||
// policyState says whether the module's policy is placed and does what ADR 0316 §5 and §7 say.
|
||||
func (m *Machine) policyState() string {
|
||||
raw, err := readBounded(m.path(policyFile))
|
||||
if err != nil {
|
||||
return "not placed: " + policyFile + " is missing"
|
||||
}
|
||||
if why := policyProblem(raw); why != "" {
|
||||
return "placed, and " + why
|
||||
}
|
||||
return "placed: Betterbird's updater off, Owl installed into every profile from the mesh's xpi, its updates off"
|
||||
}
|
||||
|
||||
// policyProblem is what the policy fails to say of what ADR 0316 asks, or "".
|
||||
func policyProblem(raw []byte) string {
|
||||
var p struct {
|
||||
Policies struct {
|
||||
DisableAppUpdate bool `json:"DisableAppUpdate"`
|
||||
ExtensionSettings map[string]struct {
|
||||
Mode string `json:"installation_mode"`
|
||||
URL string `json:"install_url"`
|
||||
Disabled bool `json:"updates_disabled"`
|
||||
} `json:"ExtensionSettings"`
|
||||
} `json:"policies"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &p); err != nil {
|
||||
return "it is not JSON: " + err.Error()
|
||||
}
|
||||
owl, ok := p.Policies.ExtensionSettings[owlID]
|
||||
switch {
|
||||
case !p.Policies.DisableAppUpdate:
|
||||
return "it does not turn Betterbird's updater off (DisableAppUpdate)"
|
||||
case !ok || owl.Mode != "force_installed":
|
||||
return "it does not install Owl into every profile (force_installed)"
|
||||
case owl.URL != "file://"+owlFile:
|
||||
return "it installs Owl from " + owl.URL + ", not from the mesh's xpi"
|
||||
case !owl.Disabled:
|
||||
return "it leaves Owl's updates on"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// xpiVersion is the version an add-on's manifest.json says, or "" when it cannot be read.
|
||||
func xpiVersion(path string) string {
|
||||
r, err := zip.OpenReader(path)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer r.Close()
|
||||
for _, f := range r.File {
|
||||
if f.Name != "manifest.json" {
|
||||
continue
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer rc.Close()
|
||||
var mf struct {
|
||||
Version string `json:"version"`
|
||||
}
|
||||
if json.NewDecoder(io.LimitReader(rc, 1<<20)).Decode(&mf) != nil {
|
||||
return ""
|
||||
}
|
||||
return mf.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// addOns reads a profile's extensions.json: each add-on's id, version, whether it is on, and where from.
|
||||
func addOns(profile string) []AddOn {
|
||||
raw, err := readBounded(filepath.Join(profile, "extensions.json"))
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var ext struct {
|
||||
Addons []struct {
|
||||
ID string `json:"id"`
|
||||
Version string `json:"version"`
|
||||
Active bool `json:"active"`
|
||||
Location string `json:"location"`
|
||||
} `json:"addons"`
|
||||
}
|
||||
if json.Unmarshal(raw, &ext) != nil {
|
||||
return nil
|
||||
}
|
||||
var out []AddOn
|
||||
for _, a := range ext.Addons {
|
||||
if a.Location == "app-profile" {
|
||||
out = append(out, AddOn{ID: a.ID, Version: a.Version, Active: a.Active})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// What the mesh installs, as the tools say it (novox/hq ADR 0316 §16): the version against the pin and the
|
||||
// newest published build, the policy, Owl against its pin; and the reminder test, visibly a test.
|
||||
|
||||
func (f *fake) xpi(path, version string) {
|
||||
f.t.Helper()
|
||||
full := filepath.Join(f.Root, path)
|
||||
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
out, err := os.Create(full)
|
||||
if err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
w := zip.NewWriter(out)
|
||||
m, _ := w.Create("manifest.json")
|
||||
m.Write([]byte(`{"version": "` + version + `", "browser_specific_settings": {"gecko": {"id": "owl@beonex.com"}}}`))
|
||||
w.Close()
|
||||
out.Close()
|
||||
}
|
||||
|
||||
// placedByTheMesh is a node after the switch: the release, the policy and Owl's xpi in /opt/betterbird, and
|
||||
// the profile running Owl at the pin.
|
||||
func (f *fake) placedByTheMesh() {
|
||||
f.noPackage()
|
||||
f.write(appIni, "[App]\nVendor=Betterbird Project\nVersion=153.4.0\nBuildID=20260929222610\n")
|
||||
raw, err := os.ReadFile(filepath.Join("..", "..", "install", "distribution", "policies.json"))
|
||||
if err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
f.write(policyFile, string(raw))
|
||||
f.xpi(owlFile, "1.5.6.1")
|
||||
f.write(profileDir+"/extensions.json", `{"addons":[{"id":"owl@beonex.com","version":"1.5.6.1","active":true,"location":"app-profile"}]}`)
|
||||
}
|
||||
|
||||
func TestTheStatusSaysTheInstallationAgainstThePinTheNewestBuildThePolicyAndOwl(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.betterbird()
|
||||
f.placedByTheMesh()
|
||||
links(t, nil)
|
||||
publishedBuilds = func() (string, error) {
|
||||
return "blocklist,{}\nLinux,153,20261101000000\nLinux,140,20260829111240\n", nil
|
||||
}
|
||||
t.Cleanup(func() { publishedBuilds = func() (string, error) { return "", os.ErrDeadlineExceeded } })
|
||||
s, err := f.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
in := s.Installation
|
||||
if in.From != "the mesh" || in.Version != "153.4.0" || in.Newest != "20261101000000" || !in.Behind ||
|
||||
!strings.HasPrefix(in.Policy, "placed:") || in.Owl == nil || in.Owl.Pinned != "1.5.6.1" || !in.Owl.AtThePin || !in.Owl.Active {
|
||||
t.Fatalf("%+v %+v", in, in.Owl)
|
||||
}
|
||||
c, err := f.Check()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := false
|
||||
for _, x := range c.Findings {
|
||||
found = found || strings.Contains(x.What, "behind the newest published build 20261101000000")
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("a release left behind is said: %+v", c.Findings)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheCommunityPackageAPolicyThatIsNotTheModulesAndAnOwlOffItsPinAreSaid(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.betterbird()
|
||||
f.placedByTheMesh()
|
||||
links(t, nil)
|
||||
f.write(policyFile, `{"policies": {"ExtensionSettings": {"owl@beonex.com": {"installation_mode": "force_installed",
|
||||
"install_url": "https://vendor.example/owl.xpi", "updates_disabled": true}}}}`)
|
||||
f.write(profileDir+"/extensions.json", `{"addons":[{"id":"owl@beonex.com","version":"1.5.7","active":true,"location":"app-profile"}]}`)
|
||||
in := f.installation(filepath.Join(f.Root, profileDir))
|
||||
if !strings.Contains(in.Policy, "DisableAppUpdate") || in.Owl.AtThePin || in.Owl.InProfile != "1.5.7" {
|
||||
t.Fatalf("%+v %+v", in, in.Owl)
|
||||
}
|
||||
// A newer Owl than the pin is a fact with the real remedy, never a finding to remove it.
|
||||
if !strings.Contains(in.Owl.Note, "move the module's pin") {
|
||||
t.Fatalf("%+v", in.Owl)
|
||||
}
|
||||
f.write(policyFile, `{"policies": {"DisableAppUpdate": true, "ExtensionSettings": {"owl@beonex.com": {"installation_mode": "force_installed",
|
||||
"install_url": "file:///opt/betterbird/owl/owl.xpi", "updates_disabled": true}}}}`)
|
||||
c, _ := f.Check()
|
||||
for _, x := range c.Findings {
|
||||
if strings.Contains(x.What, "Owl") {
|
||||
t.Errorf("a newer Owl is not a finding: %+v", x)
|
||||
}
|
||||
}
|
||||
f.write(profileDir+"/extensions.json", `{"addons":[{"id":"owl@beonex.com","version":"1.5.5","active":true,"location":"app-profile"}]}`)
|
||||
c, _ = f.Check()
|
||||
found := false
|
||||
for _, x := range c.Findings {
|
||||
found = found || (strings.Contains(x.What, "older than the mesh's pin") && strings.Contains(x.Do, "move the module's Owl pin"))
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("an older Owl is a finding whose remedy is the pin: %+v", c.Findings)
|
||||
}
|
||||
f.write(policyFile, `{"policies": {"DisableAppUpdate": true, "ExtensionSettings": {"owl@beonex.com": {"installation_mode": "force_installed",
|
||||
"install_url": "https://vendor.example/owl.xpi", "updates_disabled": true}}}}`)
|
||||
if in := f.installation(""); !strings.Contains(in.Policy, "not from the mesh's xpi") {
|
||||
t.Fatalf("%s", in.Policy)
|
||||
}
|
||||
f.answer = func(name string, args []string) Output {
|
||||
if name == "pacman" && args[0] == "-Q" {
|
||||
return Output{Stdout: "betterbird-bin 153.1.0esr-1\n"}
|
||||
}
|
||||
return Output{}
|
||||
}
|
||||
in = f.installation("")
|
||||
if !strings.HasPrefix(in.From, "the community package betterbird-bin 153.1.0esr-1") || !strings.Contains(in.From, "operator") {
|
||||
t.Fatalf("%s", in.From)
|
||||
}
|
||||
raw, _ := json.Marshal(in)
|
||||
if strings.Contains(string(raw), "example.org") {
|
||||
t.Fatalf("%s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheReminderTestIsVisiblyATestAndSaysItOpensAWindow(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.desktopSession()
|
||||
f.betterbird()
|
||||
links(t, nil)
|
||||
f.declare("// none")
|
||||
f.declareAccounts(personal)
|
||||
a, err := f.ReminderTest(40*time.Second, nil, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dir := filepath.Join("/run/user/1000", testUnit)
|
||||
css := f.read(dir + "/profile/chrome/userChrome.css")
|
||||
user := f.read(dir + "/profile/user.js")
|
||||
cal := f.read(dir + "/test.ics")
|
||||
if !strings.Contains(css, "MESH TEST") || !strings.Contains(css, `:root:not([windowtype="Calendar:AlarmWindow"])`) {
|
||||
t.Fatalf("the test's windows but the reminder carry a MESH TEST band: %s", css)
|
||||
}
|
||||
if !strings.Contains(user, `user_pref("toolkit.legacyUserProfileCustomizations.stylesheets", true);`) ||
|
||||
strings.Contains(user, "mesh_personal") || strings.Contains(user, "example.org") {
|
||||
t.Fatalf("the test profile reads its userChrome.css and holds no declared account: %s", user)
|
||||
}
|
||||
if !strings.Contains(cal, "SUMMARY:MESH TEST reminder") || !strings.Contains(a.OnScreen, "operator's screen") {
|
||||
t.Fatalf("the event and the answer say it is a test on the operator's screen: %s %+v", cal, a)
|
||||
}
|
||||
if !f.called("systemd-run --user --collect --quiet --unit="+testUnit) || !strings.Contains(strings.Join(f.calls, "\n"), startBin+" --no-remote") {
|
||||
t.Fatalf("the test starts Betterbird itself, never the module's start: %v", f.calls)
|
||||
}
|
||||
for _, tool := range tools() {
|
||||
if tool.Name == "betterbird_reminder_test" && !strings.HasPrefix(tool.Description, "OPENS WINDOWS ON THE OPERATOR'S SCREEN") {
|
||||
t.Fatalf("the description says first that a window opens on the operator's screen: %s", tool.Description)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheUserJSDescriptionSaysAdoption(t *testing.T) {
|
||||
for _, tool := range tools() {
|
||||
if tool.Name == "betterbird_user_js" && (!strings.Contains(tool.Description, "adopted under its own keys") ||
|
||||
strings.Contains(tool.Description, "refuses the apply")) {
|
||||
t.Fatalf("%s", tool.Description)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -16,6 +16,15 @@ import (
|
||||
)
|
||||
|
||||
func main() {
|
||||
// The module's start runs this before Betterbird (novox/hq ADR 0316 §11, §12): the user.js placed and the
|
||||
// declared accounts written into the account lists, while no Betterbird of this account runs.
|
||||
if len(os.Args) == 2 && os.Args[1] == "prepare" {
|
||||
if err := machine.Prepare(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "betterbird: "+err.Error())
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err := stdio.Serve("", tools()); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
@@ -29,10 +38,13 @@ func tools() []stdio.Tool {
|
||||
{
|
||||
Name: "betterbird_status",
|
||||
Description: "Betterbird: whether it runs (the main process's pid, since, and the scope it runs in; how " +
|
||||
"many child processes), the installed package and version and whether it is from the AUR, where its " +
|
||||
"output goes, the profile it uses and how that was found, the module's user.js (declared, placed, in " +
|
||||
"effect), the add-ons the operator installed, and the preferences that decide how its windows and " +
|
||||
"notifications look (reminder dialog, theme, renderer, system notifications). (r)",
|
||||
"many child processes); what is installed in /opt/betterbird and from where (the mesh's pinned release, " +
|
||||
"or the community package before the switch), its version and build beside the newest published build; " +
|
||||
"whether the enterprise policy is placed; Owl's version against the mesh's pin and whether it is active; " +
|
||||
"where its output goes; the profile it uses and how that was found; the module's user.js (declared, " +
|
||||
"placed, in effect); the declared accounts by count (in the profile's list, other accounts, not signed " +
|
||||
"in yet), never an address, host or name; the add-ons in the profile; and the preferences that decide " +
|
||||
"how its windows and notifications look. (r)",
|
||||
Run: func(map[string]any) (any, error) { return machine.Status() },
|
||||
},
|
||||
{
|
||||
@@ -55,11 +67,15 @@ func tools() []stdio.Tool {
|
||||
},
|
||||
{
|
||||
Name: "betterbird_user_js",
|
||||
Description: "The module's user.js — the preferences the module declares, from its setting prefs, written by " +
|
||||
"the mesh to ~/.config/betterbird/user.js — against the profile's user.js: whether it is placed, and " +
|
||||
"whether the running Betterbird started after it was. With apply, place it in the profile (only where " +
|
||||
"the profile has no user.js or the module's own; an operator's file is never overwritten). Betterbird " +
|
||||
"reads it at its next start: betterbird_restart. (r; a with apply)",
|
||||
Description: "The module's user.js — the preferences the module declares (setting prefs), then the declared " +
|
||||
"accounts' preferences under the module's own keys (setting accounts) — against the profile's user.js: " +
|
||||
"whether it is placed, and whether the running Betterbird started after it was. With apply, place it in " +
|
||||
"the profile (only where the profile has no user.js or the module's own; an operator's file is never " +
|
||||
"overwritten). A declared account the profile already holds at its address is adopted under its own keys; " +
|
||||
"one it does not hold is created under the module's keys, which are added to the account lists when " +
|
||||
"Betterbird is not running; an ambiguous one is refused alone, with the remedy (novox/hq ADR 0317). " +
|
||||
"Accounts' values are withheld in the answer. Betterbird reads it at its next start: betterbird_restart. " +
|
||||
"(r; a with apply)",
|
||||
Input: map[string]any{
|
||||
"apply": map[string]any{"type": "boolean", "description": "place the declared user.js in the profile (default false)"},
|
||||
},
|
||||
@@ -118,19 +134,43 @@ func tools() []stdio.Tool {
|
||||
},
|
||||
{
|
||||
Name: "betterbird_check",
|
||||
Description: "Check what the module promises: Betterbird is installed; one runs; its profile is found; the " +
|
||||
"module's user.js is on the machine, valid, placed in the profile and in effect; its output is read " +
|
||||
"(a pipe nobody reads loses what it prints). Answers ok and each finding with what to do. (r)",
|
||||
Description: "Check what the module promises: Betterbird installed from the mesh's pinned release and not " +
|
||||
"behind the newest published build; the enterprise policy placed; Owl at its pin and active; one " +
|
||||
"Betterbird runs; its profile is found; the module's user.js on the machine, valid, placed and in " +
|
||||
"effect; the declared accounts in the profile's list, none refused, each signed in once; its output " +
|
||||
"read. Answers ok and each finding with what to do, never an address, host or name. (r)",
|
||||
Run: func(map[string]any) (any, error) { return machine.Check() },
|
||||
},
|
||||
{
|
||||
Name: "betterbird_accounts",
|
||||
Description: "The accounts of the profile Betterbird uses, by count and kind: the IMAP and POP3 accounts a " +
|
||||
"declaration can hold, the accounts Owl keeps (never declared), and the rest. Never an address, a host " +
|
||||
"or a name, and never anything of the login store. With propose, it builds the module's setting " +
|
||||
"accounts from them (every account but Owl's, without any password or token) and proposes it through " +
|
||||
"the controller's settings verb for the operator's Approve on a channel that proves who answers " +
|
||||
"(novox/hq ADR 0317, ADR 0277): nothing changes until then, and the operator reads the value whole at " +
|
||||
"their desk. (r; a with propose)",
|
||||
Input: map[string]any{
|
||||
"propose": map[string]any{"type": "boolean", "description": "build the setting accounts from the profile and propose it to the operator (default false)"},
|
||||
},
|
||||
Run: func(args map[string]any) (any, error) {
|
||||
propose, err := flag(args, "propose", false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return machine.ProfileAccounts(propose)
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "betterbird_reminder_test",
|
||||
Description: "Show a calendar reminder without touching the operator's calendar: a second Betterbird in a " +
|
||||
"throw-away profile under the runtime directory (--no-remote, unit mesh-betterbird-reminder-test, at most " +
|
||||
"10 minutes), with the module's user.js and one local calendar file holding one event whose reminder is " +
|
||||
"due after `after` seconds. carry copies named preferences of the operator's profile (a theme, a " +
|
||||
"colour switch) into it, never a credential or an account. Then xorg_windows (class Calendar) and " +
|
||||
"xorg_screenshot show the reminder window. stop ends it and removes the profile. (a)",
|
||||
Description: "OPENS WINDOWS ON THE OPERATOR'S SCREEN: tell the operator before calling it. Shows a calendar " +
|
||||
"reminder without touching the operator's calendar: a second Betterbird in a throw-away profile under " +
|
||||
"the runtime directory (--no-remote, unit mesh-betterbird-reminder-test, at most 10 minutes), its main " +
|
||||
"window marked MESH TEST in red, with the module's user.js (no accounts) and one local calendar file " +
|
||||
"holding one event, \"MESH TEST reminder\", whose reminder is due after `after` seconds. carry copies " +
|
||||
"named preferences of the operator's profile (a theme, a colour switch) into it, never a credential or " +
|
||||
"an account. Then xorg_windows (class Calendar) and xorg_screenshot show the reminder window. stop ends " +
|
||||
"it and removes the profile. (a)",
|
||||
Input: map[string]any{
|
||||
"after": map[string]any{"type": "integer", "description": "seconds until the reminder is due (default 40, 15 to 300)"},
|
||||
"carry": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
|
||||
|
||||
@@ -2,25 +2,32 @@ package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// betterbird's shape (novox/hq issue 345, ADR 0208): no package (Betterbird comes from the AUR, and the
|
||||
// node-engine installs from the official repositories only), no seat, no start of its own (the operator
|
||||
// starts it from the launcher); the X display on its own machine; one directory and one file of its own,
|
||||
// the user.js the setting prefs fills; and the Go bundle serving exactly the listed betterbird_ tools.
|
||||
// betterbird's shape (novox/hq issue 345, ADR 0208, ADR 0316): the X display on its own machine; its directory
|
||||
// in the home with the user.js the setting prefs fills and the accounts file the setting accounts fills, both
|
||||
// trusted; the community package declared absent before anything is placed in /opt/betterbird; Betterbird from
|
||||
// its pinned upstream release, Owl pinned by its vendor's digest, the policy and the dictionaries' default
|
||||
// unpacked into the installation; the tools' program for the start's preparation, the start and the launcher;
|
||||
// every new build waiting for the operator's push; and the Go bundle serving exactly the listed tools.
|
||||
|
||||
type manifest struct {
|
||||
Module string `json:"module"`
|
||||
Version string `json:"version"`
|
||||
Requires []string `json:"requires"`
|
||||
Settings map[string]any `json:"settings"`
|
||||
Tools []string `json:"tools"`
|
||||
Resources []map[string]any `json:"resources"`
|
||||
Module string `json:"module"`
|
||||
Version string `json:"version"`
|
||||
Upgrade map[string]string `json:"upgrade"`
|
||||
Requires []string `json:"requires"`
|
||||
Invokes []string `json:"invokes"`
|
||||
Settings map[string]any `json:"settings"`
|
||||
Tools []string `json:"tools"`
|
||||
Resources []map[string]any `json:"resources"`
|
||||
Build struct {
|
||||
Artifacts []map[string]any `json:"artifacts"`
|
||||
} `json:"build"`
|
||||
@@ -41,43 +48,216 @@ func readManifest(t *testing.T) (manifest, string) {
|
||||
return m, string(raw)
|
||||
}
|
||||
|
||||
func TestItOwnsTheUserJSAndNothingOfTheProfile(t *testing.T) {
|
||||
func resource(t *testing.T, m manifest, id string) map[string]any {
|
||||
t.Helper()
|
||||
for _, r := range m.Resources {
|
||||
if r["id"] == id {
|
||||
return r
|
||||
}
|
||||
}
|
||||
t.Fatalf("module.json has no resource %s", id)
|
||||
return nil
|
||||
}
|
||||
|
||||
func artifact(t *testing.T, m manifest, name string) map[string]any {
|
||||
t.Helper()
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a["name"] == name {
|
||||
return a
|
||||
}
|
||||
}
|
||||
t.Fatalf("module.json builds no artifact %s", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestItPlacesBetterbirdWhereTheCommunityPackageDidAndOnlyAfterItIsGone(t *testing.T) {
|
||||
m, raw := readManifest(t)
|
||||
if m.Module != "betterbird" || !reflect.DeepEqual(m.Requires, []string{"x11-display"}) {
|
||||
t.Fatalf("%+v", m)
|
||||
}
|
||||
// Every new build waits for a person's push: the switch removes a package and replaces what a running
|
||||
// Betterbird has open (ADR 0316 §4).
|
||||
if m.Upgrade["policy"] != "record" || strings.TrimSpace(m.Upgrade["why"]) == "" {
|
||||
t.Fatalf("the module is marked for a person's approval, with why: %v", m.Upgrade)
|
||||
}
|
||||
var ids []string
|
||||
for _, r := range m.Resources {
|
||||
ids = append(ids, r["id"].(string))
|
||||
if r["type"] == "package" {
|
||||
t.Errorf("betterbird-bin is the AUR's: a package resource asks the node-engine for what it cannot install")
|
||||
}
|
||||
if p, _ := r["path"].(string); !strings.HasPrefix(p, "${machine:account-home}/.config/betterbird") {
|
||||
t.Errorf("a path outside the module's directory: %s", p)
|
||||
}
|
||||
}
|
||||
if strings.Join(ids, ",") != "configuration-dir,user-js" {
|
||||
t.Fatalf("%v", ids)
|
||||
want := "configuration-dir,user-js,accounts,community-package,installation,owl,policies,spelling,preparation,start,launcher"
|
||||
if strings.Join(ids, ",") != want {
|
||||
t.Fatalf("the resources, in the order the node-engine applies them: %v, want %s", ids, want)
|
||||
}
|
||||
pkg := resource(t, m, "community-package")
|
||||
if pkg["type"] != "package" || pkg["package"] != packageFor || pkg["absent"] != true {
|
||||
t.Fatalf("the community package is declared absent (ADR 0180): %v", pkg)
|
||||
}
|
||||
places := map[string]string{"installation": installDir, "owl": filepath.Dir(owlFile),
|
||||
"policies": filepath.Dir(policyFile), "spelling": installDir + "/defaults/pref", "preparation": filepath.Dir(preparation)}
|
||||
arts := map[string]string{"installation": "archive", "owl": "owl", "policies": "distribution", "spelling": "defaults", "preparation": "tools"}
|
||||
for id, path := range places {
|
||||
r := resource(t, m, id)
|
||||
if r["type"] != "archive" || r["path"] != path || r["artifact"] != arts[id] || r["owner"] != nil {
|
||||
t.Errorf("%s is the %s archive unpacked at %s, root's: %v", id, arts[id], path, r)
|
||||
}
|
||||
}
|
||||
for _, never := range []string{".thunderbird", "prefs.js", "logins.json", "key4.db"} {
|
||||
if strings.Contains(raw, never) {
|
||||
t.Errorf("module.json names %q: the profile is Betterbird's", never)
|
||||
}
|
||||
}
|
||||
s := strings.ToLower(raw)
|
||||
for _, never := range []string{"/home/", "jochen", "g14", "shanks", "novox.be", "http://", "mediahuis"} {
|
||||
if strings.Contains(s, never) {
|
||||
t.Errorf("module.json names %q", never)
|
||||
}
|
||||
}
|
||||
file := m.Resources[1]
|
||||
}
|
||||
|
||||
func TestTheUserJSAndTheAccountsAreTrustedFilesOfTheModulesOwn(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
file := resource(t, m, "user-js")
|
||||
content := file["content"].(string)
|
||||
if !strings.HasPrefix(content, mark) || !strings.Contains(content, "${setting:prefs}") || file["trusted"] != nil {
|
||||
t.Fatalf("the user.js begins with the mark, holds the setting, and stays trusted (only the terminal sets it): %v", file)
|
||||
if !strings.HasPrefix(content, mark) || !strings.Contains(content, "${setting:prefs}") || file["trusted"] != true {
|
||||
t.Fatalf("the user.js begins with the mark, holds the setting, and is trusted (only the terminal or the operator's warrant sets it): %v", file)
|
||||
}
|
||||
// The default is a comment, so a machine without the setting places a valid, empty user.js.
|
||||
def := m.Settings["prefs"].(map[string]any)["default"].(string)
|
||||
if err := validUserJS(strings.ReplaceAll(content, "${setting:prefs}", def)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
acc := resource(t, m, "accounts")
|
||||
if acc["path"] != "${machine:account-home}/"+accountsFile || acc["merge"] != "json" || acc["trusted"] != true ||
|
||||
acc["mode"] != "0600" || acc["owner"] != "${machine:account}" {
|
||||
t.Fatalf("the setting accounts reaches the machine as a trusted mergeable file of the account's (ADR 0316 §10, §15): %v", acc)
|
||||
}
|
||||
got, err := parseAccounts([]byte(acc["content"].(string)))
|
||||
if err != nil || len(got) != 0 {
|
||||
t.Fatalf("its default declares no account: %v %v", got, err)
|
||||
}
|
||||
if _, declared := m.Settings["accounts"]; declared {
|
||||
t.Fatal("accounts is the operator's value and has no default in the definition (ADR 0262)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBothPinsAreAURLAVersionAndASha256AndOwlIsNeverInTheRepository(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
sha := regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
for _, name := range []string{"archive", "owl"} {
|
||||
a := artifact(t, m, name)
|
||||
u, _ := a["url"].(string)
|
||||
if a["kind"] != "archive" || !strings.HasPrefix(u, "https://") || !sha.MatchString(fmt.Sprint(a["sha256"])) ||
|
||||
strings.TrimSpace(fmt.Sprint(a["version"])) == "" || a["from"] != nil {
|
||||
t.Errorf("%s is pinned by URL, version and sha256, and copied from nowhere in the repository: %v", name, a)
|
||||
}
|
||||
}
|
||||
if a := artifact(t, m, "archive"); a["within"] != "betterbird" {
|
||||
t.Errorf("the release unpacks into betterbird/, whose content is the installation: %v", a)
|
||||
}
|
||||
if a := artifact(t, m, "owl"); a["file"] != filepath.Base(owlFile) {
|
||||
t.Errorf("Owl's xpi is kept whole as %s: %v", filepath.Base(owlFile), a)
|
||||
}
|
||||
// Owl's licence forbids copying it to third parties: no xpi anywhere in the module's tree.
|
||||
filepath.Walk(filepath.Join("..", ".."), func(p string, info os.FileInfo, err error) error {
|
||||
if err == nil && strings.HasSuffix(strings.ToLower(p), ".xpi") {
|
||||
t.Errorf("%s is in the repository", p)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// owlSupports is the range of Betterbird versions each pinned Owl declares in its manifest.json
|
||||
// (browser_specific_settings.gecko strict_min_version, strict_max_version), read from the vendor's xpi
|
||||
// when its pin was taken. Moving Owl's pin means reading the new xpi's range into here.
|
||||
var owlSupports = map[string][2]string{
|
||||
"1.5.6.1": {"115.0", "156.*"},
|
||||
}
|
||||
|
||||
// owlAccepts says why a Betterbird version is outside the range an Owl version declares, or nil.
|
||||
func owlAccepts(owl, betterbird string) error {
|
||||
r, known := owlSupports[owl]
|
||||
if !known {
|
||||
return fmt.Errorf("Owl %s's supported range is not recorded in owlSupports: read it from the xpi's manifest.json", owl)
|
||||
}
|
||||
major := func(v string) int {
|
||||
head, _, _ := strings.Cut(v, ".")
|
||||
n, err := strconv.Atoi(head)
|
||||
if err != nil {
|
||||
return -1
|
||||
}
|
||||
return n
|
||||
}
|
||||
b, lo, hi := major(betterbird), major(r[0]), major(r[1])
|
||||
if b < 0 || b < lo || b > hi {
|
||||
return fmt.Errorf("Betterbird %s is outside the range Owl %s supports (%s to %s): a Betterbird Owl does not support "+
|
||||
"waits for Owl's pin to move (novox/hq ADR 0316 §8)", betterbird, owl, r[0], r[1])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestABetterbirdOwlDoesNotSupportIsRefused(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
owl, bb := fmt.Sprint(artifact(t, m, "owl")["version"]), fmt.Sprint(artifact(t, m, "archive")["version"])
|
||||
if err := owlAccepts(owl, bb); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, outside := range []string{"157.0esr-bb1", "102.15.1esr-bb4"} {
|
||||
if err := owlAccepts(owl, outside); err == nil {
|
||||
t.Errorf("Betterbird %s was accepted with Owl %s", outside, owl)
|
||||
}
|
||||
}
|
||||
if err := owlAccepts("9.9", bb); err == nil {
|
||||
t.Error("an Owl whose range nobody read was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestThePolicyTurnsTheUpdaterOffAndInstallsTheMeshsOwlEverywhere(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
if a := artifact(t, m, "distribution"); a["kind"] != "archive" || a["from"] != "install/distribution" {
|
||||
t.Fatalf("%v", a)
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join("..", "..", "install", "distribution", "policies.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if why := policyProblem(raw); why != "" {
|
||||
t.Fatalf("the module's policies.json: %s", why)
|
||||
}
|
||||
entries, _ := os.ReadDir(filepath.Join("..", "..", "install", "distribution"))
|
||||
if len(entries) != 1 {
|
||||
t.Fatalf("the distribution directory holds the policy alone: %v", entries)
|
||||
}
|
||||
if a := artifact(t, m, "defaults"); a["kind"] != "archive" || a["from"] != "install/defaults-pref" {
|
||||
t.Fatalf("%v", a)
|
||||
}
|
||||
prefs, err := os.ReadFile(filepath.Join("..", "..", "install", "defaults-pref", "vendor-prefs.js"))
|
||||
if err != nil || !strings.Contains(string(prefs), `pref("spellchecker.dictionary_path", "/usr/share/hunspell");`) {
|
||||
t.Fatalf("the dictionaries are named as a default, in place of a link: %s %v", prefs, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheStartPreparesThenStartsBetterbirdAndTheLauncherUsesIt(t *testing.T) {
|
||||
m, _ := readManifest(t)
|
||||
start := resource(t, m, "start")
|
||||
content := start["content"].(string)
|
||||
if start["path"] != "/usr/local/bin/betterbird" || start["mode"] != "0755" ||
|
||||
!strings.Contains(content, "\n"+preparation+" prepare ||") || !strings.HasSuffix(content, "\nexec "+startBin+" \"$@\"\n") {
|
||||
t.Fatalf("the start runs the preparation, says when it did not complete, and starts Betterbird either way: %v", start)
|
||||
}
|
||||
launcher := resource(t, m, "launcher")
|
||||
lc := launcher["content"].(string)
|
||||
if launcher["path"] != "/usr/local/share/applications/eu.betterbird.Betterbird.desktop" ||
|
||||
!strings.Contains(lc, "\nExec=/usr/local/bin/betterbird %u\n") || !strings.Contains(lc, "\nIcon="+installDir+"/") ||
|
||||
!strings.Contains(lc, "\nStartupWMClass=eu.betterbird.Betterbird\n") {
|
||||
t.Fatalf("the launcher starts the module's start and names the icon in the installation: %v", launcher)
|
||||
}
|
||||
if b := artifact(t, m, "tools"); filepath.Base(preparation) != b["binary"] {
|
||||
t.Fatalf("the preparation runs the tools' binary: %v", b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheToolsAgreeWithTheManifest(t *testing.T) {
|
||||
m, raw := readManifest(t)
|
||||
m, _ := readManifest(t)
|
||||
served := map[string]bool{}
|
||||
for _, tool := range tools() {
|
||||
served[tool.Name] = true
|
||||
@@ -94,17 +274,12 @@ func TestTheToolsAgreeWithTheManifest(t *testing.T) {
|
||||
for name := range served {
|
||||
t.Errorf("the bundle serves %s, which module.json does not list", name)
|
||||
}
|
||||
if len(m.Build.Artifacts) != 1 {
|
||||
t.Fatalf("%v", m.Build.Artifacts)
|
||||
}
|
||||
b := m.Build.Artifacts[0]
|
||||
b := artifact(t, m, "tools")
|
||||
if b["kind"] != "bundle" || b["language"] != "go" || b["from"] != "cmd/betterbird-tools" || b["binary"] != "betterbird-tools" {
|
||||
t.Errorf("the Go tools bundle: %v", b)
|
||||
}
|
||||
s := strings.ToLower(raw)
|
||||
for _, never := range []string{"/home/", "jochen", "g14", "shanks", "novox.be", "http"} {
|
||||
if strings.Contains(s, never) {
|
||||
t.Errorf("module.json names %q", never)
|
||||
}
|
||||
// betterbird_accounts proposes the setting accounts through the controller's settings verb (ADR 0317 §9).
|
||||
if !reflect.DeepEqual(m.Invokes, []string{"seat:mesh-controller.settings"}) {
|
||||
t.Errorf("the module invokes the controller's settings verb and nothing else: %v", m.Invokes)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -10,6 +10,13 @@ package main
|
||||
// module declares. A theme or another preference of the operator's profile can be carried over by
|
||||
// name (carry), within the same limits as betterbird_prefs: never a credential, never an account.
|
||||
//
|
||||
// **It is visibly a test** (the operator, 2026-10-12): on 2026-10-12 at 12:05 a test Betterbird with an empty
|
||||
// profile carrying the operator's theme looked like the operator's configuration lost. So the test profile
|
||||
// carries a userChrome.css that marks its main window with a red band reading MESH TEST, and the event is
|
||||
// called "MESH TEST reminder". The reminder window itself is left as Betterbird draws it, because how it is
|
||||
// drawn is what the test looks at. The tool's description says a window opens on the operator's screen, so
|
||||
// a caller tells the operator first.
|
||||
//
|
||||
// It lives for at most ten minutes (the unit's RuntimeMaxSec), and stop ends it early and removes the
|
||||
// profile. The reminder window is then found with xorg_windows (class Calendar, role AlarmWindow) and
|
||||
// looked at with xorg_screenshot.
|
||||
@@ -41,6 +48,7 @@ type ReminderAnswer struct {
|
||||
Carried []string `json:"carried,omitempty"`
|
||||
Withheld []string `json:"withheld,omitempty"`
|
||||
Look string `json:"look"`
|
||||
OnScreen string `json:"on_screen,omitempty"`
|
||||
Stopped bool `json:"stopped,omitempty"`
|
||||
}
|
||||
|
||||
@@ -50,11 +58,38 @@ func ics(now time.Time, after time.Duration, stamp string) string {
|
||||
start := now.Add(after + 10*time.Minute)
|
||||
return strings.Join([]string{"BEGIN:VCALENDAR", "VERSION:2.0", "PRODID:-//novox mesh//betterbird reminder test//EN",
|
||||
"BEGIN:VEVENT", "UID:mesh-reminder-test-" + stamp + "@mesh", "DTSTAMP:" + f(now), "DTSTART:" + f(start),
|
||||
"DTEND:" + f(start.Add(30*time.Minute)), "SUMMARY:Mesh reminder test", "BEGIN:VALARM", "ACTION:DISPLAY",
|
||||
"DESCRIPTION:Mesh reminder test", "TRIGGER:-PT10M", "END:VALARM", "END:VEVENT", "END:VCALENDAR", ""}, "\r\n")
|
||||
"DTEND:" + f(start.Add(30*time.Minute)), "SUMMARY:" + testTitle, "BEGIN:VALARM", "ACTION:DISPLAY",
|
||||
"DESCRIPTION:" + testTitle, "TRIGGER:-PT10M", "END:VALARM", "END:VEVENT", "END:VCALENDAR", ""}, "\r\n")
|
||||
}
|
||||
|
||||
// testTitle is the test event's title: what the reminder window shows.
|
||||
const testTitle = "MESH TEST reminder (a throw-away Betterbird of the mesh, not your calendar)"
|
||||
|
||||
// testChrome is the test profile's userChrome.css: a red band reading MESH TEST across the top of every
|
||||
// Betterbird window but the reminder (windowtype Calendar:AlarmWindow), and a red frame around it, so the test
|
||||
// Betterbird is never taken for the operator's.
|
||||
const testChrome = `/* The mesh's reminder test (module betterbird, novox/hq issue 345): a throw-away Betterbird, not yours. */
|
||||
:root:not([windowtype="Calendar:AlarmWindow"]) {
|
||||
outline: 6px solid #d00000 !important;
|
||||
outline-offset: -6px !important;
|
||||
}
|
||||
:root:not([windowtype="Calendar:AlarmWindow"]) > body::before {
|
||||
content: "MESH TEST \2014 a throw-away Betterbird started by the mesh for a reminder test; your mail and calendar are not here. It ends by itself within 10 minutes.";
|
||||
display: block !important;
|
||||
background: #d00000 !important;
|
||||
color: #ffffff !important;
|
||||
font: bold 15px sans-serif !important;
|
||||
padding: 6px 12px !important;
|
||||
text-align: center !important;
|
||||
}
|
||||
`
|
||||
|
||||
// testPrefs are the test profile's own preferences: its one calendar, and no first-run questions.
|
||||
//
|
||||
// betterbird.update.checkIntervalDays 0 is Betterbird's own switch for its update check (MailGlue): a
|
||||
// fresh profile has never checked, so Betterbird checks at its start and, when betterbird.eu publishes a
|
||||
// newer build than the installed one, opens the About window on the operator's screen and gives it the
|
||||
// focus (seen on the workstation 2026-10-12, novox/hq issue 345). The test is about the reminder only.
|
||||
func testPrefs(calendar string) string {
|
||||
return strings.Join([]string{
|
||||
`user_pref("calendar.registry.meshtest.type", "ics");`,
|
||||
@@ -68,7 +103,9 @@ func testPrefs(calendar string) string {
|
||||
`user_pref("datareporting.policy.dataSubmissionEnabled", false);`,
|
||||
`user_pref("toolkit.telemetry.reportingpolicy.firstRun", false);`,
|
||||
`user_pref("app.update.auto", false);`,
|
||||
`user_pref("betterbird.update.checkIntervalDays", 0);`,
|
||||
`user_pref("mail.rights.version", 1);`,
|
||||
`user_pref("toolkit.legacyUserProfileCustomizations.stylesheets", true);`,
|
||||
"",
|
||||
}, "\n")
|
||||
}
|
||||
@@ -125,14 +162,22 @@ func (m *Machine) ReminderTest(after time.Duration, carry []string, stop bool) (
|
||||
if err := os.WriteFile(calendar, []byte(ics(now, after, strconv.FormatInt(now.Unix(), 10))), 0o600); err != nil {
|
||||
return a, err
|
||||
}
|
||||
user := testPrefs(strings.TrimPrefix(calendar, m.Root))
|
||||
// The module's user.js first, then what is carried, then the test's own preferences: the last
|
||||
// statement wins, so neither can take the test's calendar or its quiet start away.
|
||||
user := ""
|
||||
if mine, err := readBounded(m.home(declared)); err == nil && validUserJS(string(mine)) == nil {
|
||||
user = string(mine) + "\n" + user
|
||||
user = string(mine) + "\n"
|
||||
}
|
||||
user += strings.Join(carried, "\n") + "\n"
|
||||
user += strings.Join(carried, "\n") + "\n" + testPrefs(strings.TrimPrefix(calendar, m.Root))
|
||||
if err := os.WriteFile(filepath.Join(dir, "profile", "user.js"), []byte(user), 0o600); err != nil {
|
||||
return a, err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Join(dir, "profile", "chrome"), 0o700); err != nil {
|
||||
return a, err
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "profile", "chrome", "userChrome.css"), []byte(testChrome), 0o600); err != nil {
|
||||
return a, err
|
||||
}
|
||||
// The unit removes the profile when it ends, however it ends: at its 10 minutes, on stop, or when the
|
||||
// test Betterbird is closed.
|
||||
call := []string{"--user", "--collect", "--quiet", "--unit=" + testUnit, "-p", "RuntimeMaxSec=600",
|
||||
@@ -146,7 +191,9 @@ func (m *Machine) ReminderTest(after time.Duration, carry []string, stop bool) (
|
||||
if err := failed(m.cmd(8*time.Second, s.Env(), "systemd-run", call...), "systemd-run", call...); err != nil {
|
||||
return a, err
|
||||
}
|
||||
a.Event = "Mesh reminder test, in a calendar file of the test profile only"
|
||||
a.Event = testTitle + ", in a calendar file of the test profile only"
|
||||
a.OnScreen = "a second Betterbird's window is now on the operator's screen, marked MESH TEST in red; its reminder " +
|
||||
"window follows when the reminder is due. Both go when the test ends (stop, or 10 minutes)"
|
||||
a.Due = now.Add(after).UTC().Format(time.RFC3339)
|
||||
return a, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// No test reaches the network or the mesh: Betterbird's list of published builds and the controller's
|
||||
// settings verb answer from here, and a test that needs them says what they answer.
|
||||
func TestMain(m *testing.M) {
|
||||
publishedBuilds = func() (string, error) { return "", errors.New("no network in tests") }
|
||||
asker = func(string, any) (json.RawMessage, error) { return nil, errors.New("no mesh in tests") }
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
// Written by the mesh (module betterbird, novox/hq ADR 0316 §3): the system's spelling dictionaries, named
|
||||
// as a default preference in place of the link the community package made.
|
||||
pref("spellchecker.dictionary_path", "/usr/share/hunspell");
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"policies": {
|
||||
"DisableAppUpdate": true,
|
||||
"ExtensionSettings": {
|
||||
"owl@beonex.com": {
|
||||
"installation_mode": "force_installed",
|
||||
"install_url": "file:///opt/betterbird/owl/owl.xpi",
|
||||
"updates_disabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,14 +1,21 @@
|
||||
{
|
||||
"module": "betterbird",
|
||||
"version": "1",
|
||||
"upgrade": {
|
||||
"policy": "record",
|
||||
"why": "a new build may remove the community package betterbird-bin and replaces the installation in /opt/betterbird, which a running Betterbird has open: the switch on each node and every upgrade wait for the operator, who quits Betterbird first and then pushes (novox/hq ADR 0316 §4, §5)"
|
||||
},
|
||||
"requires": [
|
||||
"x11-display"
|
||||
],
|
||||
"invokes": [
|
||||
"seat:mesh-controller.settings"
|
||||
],
|
||||
"settings": {
|
||||
"prefs": {
|
||||
"kind": "preference",
|
||||
"default": "// none",
|
||||
"why": "Betterbird runs with its own defaults until a machine's assignment names a preference: one line of user_pref(\"name\", value); statements, which the module's user.js holds and Betterbird reads at its start (novox/hq issue 345)"
|
||||
"why": "Betterbird runs with its own defaults until a machine's assignment names a preference: one line of user_pref(\"name\", value); statements, which the module's user.js holds and Betterbird reads at its start (novox/hq issue 345). An account's preferences (mail.account., mail.identity., mail.server., mail.smtpserver. and the account lists) are refused here: an account is declared in the setting accounts, so it has one owner (novox/hq ADR 0316 §11)"
|
||||
}
|
||||
},
|
||||
"tools": [
|
||||
@@ -18,6 +25,7 @@
|
||||
"betterbird_log",
|
||||
"betterbird_restart",
|
||||
"betterbird_check",
|
||||
"betterbird_accounts",
|
||||
"betterbird_reminder_test"
|
||||
],
|
||||
"resources": [
|
||||
@@ -34,7 +42,68 @@
|
||||
"path": "${machine:account-home}/.config/betterbird/user.js",
|
||||
"owner": "${machine:account}",
|
||||
"mode": "0644",
|
||||
"content": "// Betterbird's user.js (module betterbird, novox/hq issue 345). Owned by the mesh: the node-engine\n// writes it here at every push, and the module's tools copy it whole into Betterbird's profile\n// (betterbird_user_js with apply, betterbird_restart), where Betterbird reads it at its start and lets it\n// win over the preferences Betterbird saves itself. Change the module's setting prefs, never this file or the profile's copy.\n//\n// Only comments and user_pref(\"name\", value); statements; the tools refuse anything else, and any\n// preference whose name can hold a credential.\n${setting:prefs}\n"
|
||||
"content": "// Betterbird's user.js (module betterbird, novox/hq issue 345). Owned by the mesh: the node-engine\n// writes it here at every push, and the module's tools copy it whole into Betterbird's profile\n// (betterbird_user_js with apply, betterbird_restart), where Betterbird reads it at its start and lets it\n// win over the preferences Betterbird saves itself. Change the module's setting prefs, never this file or the profile's copy.\n//\n// Only comments and user_pref(\"name\", value); statements; the tools refuse anything else, and any\n// preference whose name can hold a credential.\n${setting:prefs}\n",
|
||||
"trusted": true
|
||||
},
|
||||
{
|
||||
"id": "accounts",
|
||||
"type": "file",
|
||||
"path": "${machine:account-home}/.config/betterbird/accounts.json",
|
||||
"owner": "${machine:account}",
|
||||
"mode": "0600",
|
||||
"merge": "json",
|
||||
"content": "{\n \"accounts\": {}\n}\n",
|
||||
"trusted": true
|
||||
},
|
||||
{
|
||||
"id": "community-package",
|
||||
"type": "package",
|
||||
"package": "betterbird-bin",
|
||||
"absent": true
|
||||
},
|
||||
{
|
||||
"id": "installation",
|
||||
"type": "archive",
|
||||
"path": "/opt/betterbird",
|
||||
"artifact": "archive"
|
||||
},
|
||||
{
|
||||
"id": "owl",
|
||||
"type": "archive",
|
||||
"path": "/opt/betterbird/owl",
|
||||
"artifact": "owl"
|
||||
},
|
||||
{
|
||||
"id": "policies",
|
||||
"type": "archive",
|
||||
"path": "/opt/betterbird/distribution",
|
||||
"artifact": "distribution"
|
||||
},
|
||||
{
|
||||
"id": "spelling",
|
||||
"type": "archive",
|
||||
"path": "/opt/betterbird/defaults/pref",
|
||||
"artifact": "defaults"
|
||||
},
|
||||
{
|
||||
"id": "preparation",
|
||||
"type": "archive",
|
||||
"path": "/usr/local/lib/mesh-betterbird",
|
||||
"artifact": "tools"
|
||||
},
|
||||
{
|
||||
"id": "start",
|
||||
"type": "file",
|
||||
"path": "/usr/local/bin/betterbird",
|
||||
"mode": "0755",
|
||||
"content": "#!/bin/sh\n# Betterbird's start, written by the mesh (module betterbird, novox/hq ADR 0316). Replaced at every push.\n#\n# Before Betterbird starts, the module's tools place its user.js in the profile (the setting prefs, and the\n# accounts the setting accounts declares) and add the declared accounts to the profile's account list. They do\n# it only while no Betterbird of this account runs, because Betterbird rewrites its saved preferences when it quits; a\n# second start while one runs only hands over to it. What the preparation could not do it says here, and\n# betterbird_check says it too. Betterbird starts either way: its configuration never holds the mail back.\n/usr/local/lib/mesh-betterbird/betterbird-tools prepare || echo \"betterbird: the mesh's preparation did not complete; betterbird_check says why\" >&2\nexec /opt/betterbird/betterbird \"$@\"\n"
|
||||
},
|
||||
{
|
||||
"id": "launcher",
|
||||
"type": "file",
|
||||
"path": "/usr/local/share/applications/eu.betterbird.Betterbird.desktop",
|
||||
"mode": "0644",
|
||||
"content": "[Desktop Entry]\nVersion=1.0\nType=Application\nName=Betterbird\nGenericName=Mail Client\nComment=Send and receive mail with Betterbird\nExec=/usr/local/bin/betterbird %u\nIcon=/opt/betterbird/chrome/icons/default/default128.png\nTerminal=false\nCategories=Network;Email;\nMimeType=message/rfc822;x-scheme-handler/mailto;application/x-xpinstall;application/x-extension-ics;text/calendar;text/vcard;text/x-vcard;x-scheme-handler/webcal;x-scheme-handler/webcals;x-scheme-handler/mid;\nStartupNotify=true\nStartupWMClass=eu.betterbird.Betterbird\nKeywords=Email;E-mail;Calendar;Thunderbird;\nActions=ComposeMessage;OpenAddressBook;\n# Written by the mesh (module betterbird, novox/hq ADR 0316 §3), in place of the community package's entry:\n# it starts Betterbird through the module's start, and names the icon in the installation, with no link.\n\n[Desktop Action ComposeMessage]\nName=Write new message\nExec=/usr/local/bin/betterbird -compose\n\n[Desktop Action OpenAddressBook]\nName=Open address book\nExec=/usr/local/bin/betterbird -addressbook\n"
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
@@ -49,6 +118,38 @@
|
||||
"loads": [
|
||||
"betterbird-tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "archive",
|
||||
"kind": "archive",
|
||||
"url": "https://www.betterbird.eu/downloads/LinuxArchive/betterbird-153.4.0esr-bb10.en-US.linux-x86_64.tar.xz",
|
||||
"sha256": "38dd24890a66b4cf9a45bc24e15192e00c26b929e1bb8df7974c16a4539129e2",
|
||||
"version": "153.4.0esr-bb10",
|
||||
"within": "betterbird",
|
||||
"names-on-purpose": {
|
||||
"www.betterbird.eu": "Betterbird's project publishes its Linux release there; the build seat fetches it once, by its pinned sha256 (novox/hq ADR 0316 §2)"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "owl",
|
||||
"kind": "archive",
|
||||
"url": "https://www.beonex.com/owl/owl.xpi",
|
||||
"sha256": "a54cc3d52113b34f97942ee19190947dee23561e2217f18db3d80ac023991221",
|
||||
"version": "1.5.6.1",
|
||||
"file": "owl.xpi",
|
||||
"names-on-purpose": {
|
||||
"www.beonex.com": "Owl's vendor publishes its xpi there; the build seat fetches it once, by its pinned sha256 (novox/hq ADR 0316 §6)"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "distribution",
|
||||
"kind": "archive",
|
||||
"from": "install/distribution"
|
||||
},
|
||||
{
|
||||
"name": "defaults",
|
||||
"kind": "archive",
|
||||
"from": "install/defaults-pref"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
+8
-18
@@ -45,38 +45,28 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
|
||||
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
|
||||
|
||||
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
|
||||
never the repository's default branch, never a branch a protection rule names. A rule covers a branch as the
|
||||
forge itself decides it (Gitea 1.27, novox/hq issue 539): a plain name covers that branch in any case; a glob
|
||||
(`*`, `**`, `?`, `[abc]`, `[!abc]`, `{a,b}`, `\` escaping) covers case-sensitively with `/` as separator; a glob
|
||||
the forge cannot compile covers its own text. `cmd/forge/rules.go` and `guard.ts` hold this one rule. The rules are read before anything is sent; rules that cannot be read refuse the
|
||||
never the repository's default branch, never a branch a protection rule names (a glob rule covers by the same
|
||||
matcher as `guard.ts`). The rules are read before anything is sent; rules that cannot be read refuse the
|
||||
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
|
||||
required checks green, through `gitea_merge_pull_request`.
|
||||
|
||||
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
|
||||
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
|
||||
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. The blobs are read at
|
||||
the commit the branch (or, for a new branch, its base) was read at, never at its name. A `patch` goes to the
|
||||
forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than the
|
||||
file. Neither endpoint takes a commit to build on, only a branch (Gitea 1.27), so the parent answered is the
|
||||
one the forge says it made the commit on; when the branch moved between the read and the commit, the answer
|
||||
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
||||
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. A `patch` goes
|
||||
to the forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than
|
||||
the file. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
||||
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
|
||||
branch not at the commit the forge named is a failure, never a success.
|
||||
|
||||
**Code search** has no endpoint on the forge, so the ref's archive (`/archive/{ref}.tar.gz`, at most 64 MiB)
|
||||
is read once and searched in memory; a binary file is passed over, a file over 4 MiB is named under
|
||||
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found. A
|
||||
search that reaches the call's deadline stops there and answers what it found, with `stopped_at_deadline`.
|
||||
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found.
|
||||
|
||||
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), and one
|
||||
tool call takes at most 50 seconds.
|
||||
It reaches the forge as the npm-registry bundle does (below), and one tool call takes at most 50 seconds.
|
||||
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
|
||||
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
|
||||
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
|
||||
commit failing; a branch or base that moved under the commit said, with the parent the forge used; a file
|
||||
changed under the commit refused; a commit over its size limits refused; protection globs matched as the
|
||||
forge matches them; a code search stopped at its deadline; every malformed ask refused; and each read
|
||||
summarised.
|
||||
commit failing; every malformed ask refused; and each read summarised.
|
||||
|
||||
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
@@ -53,14 +52,8 @@ type CommittedFile struct {
|
||||
Operation string `json:"operation"`
|
||||
}
|
||||
|
||||
// What one commit carries at most (novox/hq issue 539): files, one file's content, one patch, and every
|
||||
// file's content together, in bytes as committed (content_base64 decoded). Larger work is several commits.
|
||||
const (
|
||||
maxFiles = 200
|
||||
maxFileBytes = 4 << 20
|
||||
maxPatchBytes = 4 << 20
|
||||
maxCommitBytes = 16 << 20
|
||||
)
|
||||
// maxFiles is the most files one commit carries.
|
||||
const maxFiles = 200
|
||||
|
||||
// readCommitAsk reads the tool's arguments.
|
||||
func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
@@ -116,11 +109,8 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
return a, refuse("give files or patch, not both: one commit carries one of them. Nothing was sent")
|
||||
case len(a.Files) > maxFiles:
|
||||
return a, refuse("%d files is more than the %d one commit carries. Nothing was sent", len(a.Files), maxFiles)
|
||||
case len(a.Patch) > maxPatchBytes:
|
||||
return a, refuse("the patch is %s, more than the %s one commit carries: split it into several commits. Nothing was sent", mib(len(a.Patch)), mib(maxPatchBytes))
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
total := 0
|
||||
for i, fc := range a.Files {
|
||||
if strings.HasPrefix(fc.Path, "/") {
|
||||
return a, refuse("files[%d].path %q is relative to the repository's root, without a leading slash. Nothing was sent", i, fc.Path)
|
||||
@@ -144,35 +134,15 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
if fc.Content != nil && !utf8.ValidString(*fc.Content) {
|
||||
return a, refuse("files[%d].content (%s) is not UTF-8 text: give it as content_base64. Nothing was sent", i, fc.Path)
|
||||
}
|
||||
size := 0
|
||||
switch {
|
||||
case fc.Content != nil:
|
||||
size = len(*fc.Content)
|
||||
case fc.ContentBase64 != nil:
|
||||
if base64.StdEncoding.DecodedLen(len(*fc.ContentBase64)) > maxFileBytes+3 {
|
||||
size = base64.StdEncoding.DecodedLen(len(*fc.ContentBase64))
|
||||
break
|
||||
}
|
||||
raw, err := base64.StdEncoding.DecodeString(*fc.ContentBase64)
|
||||
if err != nil {
|
||||
if fc.ContentBase64 != nil {
|
||||
if _, err := base64.StdEncoding.DecodeString(*fc.ContentBase64); err != nil {
|
||||
return a, refuse("files[%d].content_base64 (%s) is not base64: %v. Nothing was sent", i, fc.Path, err)
|
||||
}
|
||||
size = len(raw)
|
||||
}
|
||||
if size > maxFileBytes {
|
||||
return a, refuse("files[%d] (%s) is %s, more than the %s one file of a commit may be. Nothing was sent", i, fc.Path, mib(size), mib(maxFileBytes))
|
||||
}
|
||||
total += size
|
||||
}
|
||||
if total > maxCommitBytes {
|
||||
return a, refuse("the files are %s together, more than the %s one commit carries: split them into several commits. Nothing was sent", mib(total), mib(maxCommitBytes))
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// mib says a size in bytes in MiB, to a tenth.
|
||||
func mib(n int) string { return fmt.Sprintf("%.1f MiB", float64(n)/(1<<20)) }
|
||||
|
||||
// targetRefused says why a commit may not go on a branch: the default branch, or one a protection rule names.
|
||||
func targetRefused(target, defaultBranch string, rules []Protection) error {
|
||||
if target == defaultBranch {
|
||||
@@ -202,7 +172,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
return nil, err
|
||||
}
|
||||
var r Repo
|
||||
if err := f.GetJSON(ctx, repo, &r); err != nil {
|
||||
if err := f.getJSON(ctx, repo, &r); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r.DefaultBranch == "" {
|
||||
@@ -210,7 +180,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
}
|
||||
// Read the rules, or refuse: a branch the rules could not be read for is never written (fails closed).
|
||||
var rules []Protection
|
||||
if err := f.GetJSON(ctx, repo+"/branch_protections", &rules); err != nil {
|
||||
if err := f.getJSON(ctx, repo+"/branch_protections", &rules); err != nil {
|
||||
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
|
||||
}
|
||||
if err := targetRefused(a.Branch, r.DefaultBranch, rules); err != nil {
|
||||
@@ -222,13 +192,9 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The files are judged at the commit just read, never at a branch name that could move between two
|
||||
// reads (novox/hq issue 539): a file the branch changes meanwhile is refused by the forge, whose update
|
||||
// and delete carry the blob read here.
|
||||
var ref string
|
||||
ref := a.Branch // where the files are read and the commit goes
|
||||
if existing != nil {
|
||||
out.Parent = existing.Commit.ID
|
||||
ref = existing.Commit.ID
|
||||
if a.Base != "" {
|
||||
out.Said = append(out.Said, fmt.Sprintf("%s exists, at %s: base %s was not used", a.Branch, short(existing.Commit.ID), a.Base))
|
||||
}
|
||||
@@ -238,6 +204,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
if out.Base == "" {
|
||||
out.Base = r.DefaultBranch
|
||||
}
|
||||
ref = out.Base
|
||||
b, err := f.branch(ctx, repo, out.Base)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -246,6 +213,8 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
return nil, refuse("%s does not exist, and its base %s is not a branch of %s/%s. Nothing was sent", a.Branch, out.Base, a.Owner, a.Repo)
|
||||
}
|
||||
out.Parent = b.Commit.ID
|
||||
// The files are judged at the commit just read; a file the base changes meanwhile is refused by the
|
||||
// forge, whose update and delete carry the blob read here.
|
||||
ref = b.Commit.ID
|
||||
}
|
||||
|
||||
@@ -258,14 +227,14 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
on["branch"] = a.Branch
|
||||
}
|
||||
|
||||
// The forge takes no commit to build on, only a branch (Gitea 1.27: neither /contents nor /diffpatch
|
||||
// carries a last commit), so the parent is read from what it made, never assumed (novox/hq issue 539).
|
||||
var made struct {
|
||||
Commit madeCommit `json:"commit"`
|
||||
Commit struct {
|
||||
SHA string `json:"sha"`
|
||||
} `json:"commit"`
|
||||
}
|
||||
if a.Patch != "" {
|
||||
on["content"] = a.Patch
|
||||
if err := f.SendJSON(ctx, http.MethodPost, repo+"/diffpatch", on, &made); err != nil {
|
||||
if err := f.sendJSON(ctx, http.MethodPost, repo+"/diffpatch", on, &made); err != nil {
|
||||
return nil, fmt.Errorf("the forge did not take the patch: %w", err)
|
||||
}
|
||||
} else {
|
||||
@@ -297,7 +266,7 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
out.Files = append(out.Files, CommittedFile{Path: fc.Path, Operation: op["operation"].(string)})
|
||||
}
|
||||
on["files"] = ops
|
||||
if err := f.SendJSON(ctx, http.MethodPost, repo+"/contents", on, &made); err != nil {
|
||||
if err := f.sendJSON(ctx, http.MethodPost, repo+"/contents", on, &made); err != nil {
|
||||
return nil, fmt.Errorf("the forge did not take the commit: %w", err)
|
||||
}
|
||||
}
|
||||
@@ -305,25 +274,6 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
return nil, fmt.Errorf("the forge answered the commit on %s without saying the commit it made: read the branch (gitea_list_branches) before trying again", a.Branch)
|
||||
}
|
||||
out.Commit = made.Commit.SHA
|
||||
read := out.Parent
|
||||
parent, err := f.parentOf(ctx, repo, made.Commit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the forge made %s on %s, but which commit it went on could not be read: %w", out.Commit, a.Branch, err)
|
||||
}
|
||||
out.Parent = parent
|
||||
if parent != read {
|
||||
moved := "branch " + a.Branch
|
||||
if out.Created {
|
||||
moved = "base " + out.Base
|
||||
}
|
||||
how := "each file it changes was checked against " + short(read) + " by its blob, so none of them changed in between"
|
||||
if a.Patch != "" {
|
||||
how = "the forge applied the patch there, and it refuses a patch whose context changed"
|
||||
}
|
||||
out.Said = append(out.Said, fmt.Sprintf("the %s moved from %s to %s between the read and the commit, so the "+
|
||||
"commit went on %s: %s. What else changed in between: gitea_compare %s...%s",
|
||||
moved, short(read), short(parent), short(parent), how, short(read), short(parent)))
|
||||
}
|
||||
|
||||
// Evidence over assertion: the branch is read back, and it must be at the commit the forge said.
|
||||
after, err := f.branch(ctx, repo, a.Branch)
|
||||
@@ -344,30 +294,6 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// madeCommit is the commit the forge answers a write with.
|
||||
type madeCommit struct {
|
||||
SHA string `json:"sha"`
|
||||
Parents []struct {
|
||||
SHA string `json:"sha"`
|
||||
} `json:"parents"`
|
||||
}
|
||||
|
||||
// parentOf is the commit the forge made its commit on: from its answer, or read from the commit when the
|
||||
// answer does not say it.
|
||||
func (f *Forge) parentOf(ctx context.Context, repo string, made madeCommit) (string, error) {
|
||||
if len(made.Parents) == 0 {
|
||||
var read madeCommit
|
||||
if err := f.GetJSON(ctx, repo+"/git/commits/"+url.PathEscape(made.SHA)+"?stat=false&verification=false&files=false", &read); err != nil {
|
||||
return "", err
|
||||
}
|
||||
made = read
|
||||
}
|
||||
if len(made.Parents) != 1 || made.Parents[0].SHA == "" {
|
||||
return "", fmt.Errorf("the forge says %s has %d parents, not one", made.SHA, len(made.Parents))
|
||||
}
|
||||
return made.Parents[0].SHA, nil
|
||||
}
|
||||
|
||||
// blobAt is the blob sha of a file at a ref, "" when there is no file there.
|
||||
func (f *Forge) blobAt(ctx context.Context, repo, path, ref string) (string, error) {
|
||||
seg, err := segments("file path", path)
|
||||
@@ -379,7 +305,7 @@ func (f *Forge) blobAt(ctx context.Context, repo, path, ref string) (string, err
|
||||
SHA string `json:"sha"`
|
||||
}
|
||||
var list []any
|
||||
body, err := f.Do(ctx, http.MethodGet, repo+"/contents/"+seg+"?ref="+queryEscape(ref), nil, maxBody)
|
||||
body, err := f.do(ctx, http.MethodGet, repo+"/contents/"+seg+"?ref="+queryEscape(ref), nil, maxBody)
|
||||
if err != nil {
|
||||
if errors.Is(err, errNotFound) {
|
||||
return "", nil
|
||||
|
||||
@@ -1,43 +1,156 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
|
||||
// (internal/forgehttp, shared with the npm-registry bundle beside it).
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account,
|
||||
// which the vault delivered into a file this module owns (novox/hq ADR 0086), by basic authentication — the
|
||||
// same way the npm-registry bundle beside it does. The password is read when a call is made and never logged,
|
||||
// answered or put in an error.
|
||||
|
||||
// Forge is the forge's HTTP interface.
|
||||
type Forge struct {
|
||||
forgehttp.Client
|
||||
URL string // e.g. http://127.0.0.1:3000
|
||||
User string
|
||||
PasswordFile string
|
||||
HTTP *http.Client
|
||||
}
|
||||
|
||||
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
||||
func ForgeFromEnv() (*Forge, error) {
|
||||
c, missing := forgehttp.FromEnv()
|
||||
f := &Forge{
|
||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
||||
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
||||
}
|
||||
var missing []string
|
||||
for _, kv := range [][2]string{{"MESH_GITEA_URL", f.URL}, {"MESH_GITEA_ADMIN_USER", f.User},
|
||||
{"MESH_GITEA_ADMIN_PASSWORD_FILE", f.PasswordFile}} {
|
||||
if kv[1] == "" {
|
||||
missing = append(missing, kv[0])
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
||||
}
|
||||
c.Timeout = deadline
|
||||
return &Forge{Client: c}, nil
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// errNotFound is the forge answering 404.
|
||||
var errNotFound = forgehttp.ErrNotFound
|
||||
var errNotFound = errors.New("not found")
|
||||
|
||||
// statusError is an answer the forge gave that is not a success, said without the request's credential.
|
||||
type statusError struct {
|
||||
method, path string
|
||||
status int
|
||||
said string
|
||||
}
|
||||
|
||||
func (e *statusError) Error() string {
|
||||
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.status, e.method, e.path, e.said)
|
||||
}
|
||||
|
||||
func (f *Forge) client() *http.Client {
|
||||
if f.HTTP != nil {
|
||||
return f.HTTP
|
||||
}
|
||||
return &http.Client{Timeout: 50 * time.Second}
|
||||
}
|
||||
|
||||
func (f *Forge) password() (string, error) {
|
||||
raw, err := os.ReadFile(f.PasswordFile)
|
||||
if err != nil {
|
||||
// The path is the manifest's, not a secret; the content never appears.
|
||||
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
||||
}
|
||||
return strings.TrimSpace(string(raw)), nil
|
||||
}
|
||||
|
||||
// maxBody is the largest JSON answer read from the forge; maxArchive the largest archive code search reads.
|
||||
const (
|
||||
maxBody = forgehttp.MaxBody
|
||||
maxBody = 32 << 20
|
||||
maxArchive = 64 << 20
|
||||
)
|
||||
|
||||
// do sends one request and answers the body, read up to limit; 404 is errNotFound. A body the limit cut is
|
||||
// refused, never read as if it were whole.
|
||||
func (f *Forge) do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
|
||||
password, err := f.password()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var body io.Reader
|
||||
if payload != nil {
|
||||
raw, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body = bytes.NewReader(raw)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, f.URL+path, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.SetBasicAuth(f.User, password)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
if payload != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
res, err := f.client().Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
got, err := io.ReadAll(io.LimitReader(res.Body, limit+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if int64(len(got)) > limit {
|
||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
|
||||
}
|
||||
switch {
|
||||
case res.StatusCode == http.StatusNotFound:
|
||||
return nil, fmt.Errorf("%s %s: %w", method, path, errNotFound)
|
||||
case res.StatusCode >= 300:
|
||||
said := strings.TrimSpace(string(got))
|
||||
if len(said) > 300 {
|
||||
said = said[:300] + "…"
|
||||
}
|
||||
return nil, &statusError{method, path, res.StatusCode, said}
|
||||
}
|
||||
return got, nil
|
||||
}
|
||||
|
||||
func (f *Forge) getJSON(ctx context.Context, path string, into any) error {
|
||||
return f.sendJSON(ctx, http.MethodGet, path, nil, into)
|
||||
}
|
||||
|
||||
func (f *Forge) sendJSON(ctx context.Context, method, path string, payload, into any) error {
|
||||
body, err := f.do(ctx, method, path, payload, maxBody)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if into == nil {
|
||||
return nil
|
||||
}
|
||||
if err := json.Unmarshal(body, into); err != nil {
|
||||
return fmt.Errorf("the forge's answer to %s %s is not readable: %w", method, path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Refused is a refusal of policy or of an argument, in the words the caller sees.
|
||||
type Refused struct{ msg string }
|
||||
|
||||
@@ -91,6 +204,36 @@ func (p Protection) name() string {
|
||||
return p.BranchName
|
||||
}
|
||||
|
||||
// ruleCovers is whether a protection rule's name — a branch, or a glob of them (`*`, `**`, `?`) — covers a
|
||||
// branch: the TypeScript bundle's ruleCovers (guard.ts), which gitea_api's file writes are judged by.
|
||||
func ruleCovers(rule, branch string) bool {
|
||||
if rule == branch {
|
||||
return true
|
||||
}
|
||||
if !strings.ContainsAny(rule, "*?[") {
|
||||
return false
|
||||
}
|
||||
var re strings.Builder
|
||||
re.WriteString("^")
|
||||
for i := 0; i < len(rule); i++ {
|
||||
c := rule[i]
|
||||
switch {
|
||||
case c == '*' && i+1 < len(rule) && rule[i+1] == '*':
|
||||
re.WriteString(".*")
|
||||
i++
|
||||
case c == '*':
|
||||
re.WriteString("[^/]*")
|
||||
case c == '?':
|
||||
re.WriteString("[^/]")
|
||||
default:
|
||||
re.WriteString(regexp.QuoteMeta(string(c)))
|
||||
}
|
||||
}
|
||||
re.WriteString("$")
|
||||
ok, err := regexp.MatchString(re.String(), branch)
|
||||
return err == nil && ok
|
||||
}
|
||||
|
||||
// Branch is one branch with the commit it points at.
|
||||
type Branch struct {
|
||||
Name string `json:"name"`
|
||||
@@ -106,7 +249,7 @@ func (f *Forge) branch(ctx context.Context, repo, name string) (*Branch, error)
|
||||
return nil, err
|
||||
}
|
||||
var b Branch
|
||||
if err := f.GetJSON(ctx, repo+"/branches/"+seg, &b); err != nil {
|
||||
if err := f.getJSON(ctx, repo+"/branches/"+seg, &b); err != nil {
|
||||
if errors.Is(err, errNotFound) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
@@ -13,12 +13,9 @@ import (
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// fakeForge is the forge's HTTP interface as far as these tools use it.
|
||||
@@ -40,16 +37,6 @@ type fakeForge struct {
|
||||
comments map[string][]map[string]any
|
||||
commits []map[string]any
|
||||
dir []map[string]any
|
||||
// moves: a branch moved by someone else right after it is first read — to a commit, with its tree.
|
||||
moves map[string]move
|
||||
noParents bool // the forge's answer to a write does not say the parents
|
||||
parentsBroken bool // and GET git/commits fails
|
||||
parents map[string]string // commit -> parent, as GET git/commits says it
|
||||
}
|
||||
|
||||
type move struct {
|
||||
to string
|
||||
tree map[string]string
|
||||
}
|
||||
|
||||
func newFake(t *testing.T) *fakeForge {
|
||||
@@ -75,7 +62,7 @@ func (f *fakeForge) serve() *Forge {
|
||||
}
|
||||
srv := httptest.NewServer(http.HandlerFunc(f.handle))
|
||||
f.t.Cleanup(srv.Close)
|
||||
return &Forge{Client: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}}
|
||||
return &Forge{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}
|
||||
}
|
||||
|
||||
func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -105,11 +92,6 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
reply(map[string]any{"name": name, "commit": map[string]any{"id": c}})
|
||||
if m, ok := f.moves[name]; ok {
|
||||
delete(f.moves, name)
|
||||
f.trees[m.to] = m.tree
|
||||
f.branches[name] = m.to
|
||||
}
|
||||
case r.Method == "GET" && (path == repo+"/contents" || path == repo+"/contents/docs"):
|
||||
reply(f.dir)
|
||||
case r.Method == "GET" && strings.HasPrefix(path, repo+"/contents/"):
|
||||
@@ -166,28 +148,19 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
||||
if !f.staleRead {
|
||||
f.branches[target] = made
|
||||
}
|
||||
reply(map[string]any{"commit": f.made(made, from)})
|
||||
reply(map[string]any{"commit": map[string]any{"sha": made}})
|
||||
case r.Method == "POST" && path == repo+"/diffpatch":
|
||||
var body map[string]any
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
f.patches = append(f.patches, body)
|
||||
f.next++
|
||||
made := fmt.Sprintf("p%d", f.next)
|
||||
from := f.branches[body["branch"].(string)]
|
||||
target := body["branch"].(string)
|
||||
if nb, ok := body["new_branch"].(string); ok && nb != "" {
|
||||
target = nb
|
||||
}
|
||||
f.branches[target] = made
|
||||
reply(map[string]any{"commit": f.made(made, from)})
|
||||
case r.Method == "GET" && strings.HasPrefix(path, repo+"/git/commits/"):
|
||||
sha := strings.TrimPrefix(path, repo+"/git/commits/")
|
||||
p, ok := f.parents[sha]
|
||||
if !ok || f.parentsBroken {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
reply(map[string]any{"sha": sha, "parents": []any{map[string]any{"sha": p}}})
|
||||
reply(map[string]any{"commit": map[string]any{"sha": made}})
|
||||
case r.Method == "GET" && path == repo+"/commits":
|
||||
f.writes = append(f.writes, map[string]any{"query": r.URL.RawQuery})
|
||||
reply(f.commits)
|
||||
@@ -207,18 +180,6 @@ func (f *fakeForge) handle(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
// made is the forge's answer about a commit it made on a parent, with the parents unless noParents.
|
||||
func (f *fakeForge) made(sha, parent string) map[string]any {
|
||||
if f.parents == nil {
|
||||
f.parents = map[string]string{}
|
||||
}
|
||||
f.parents[sha] = parent
|
||||
if f.noParents {
|
||||
return map[string]any{"sha": sha}
|
||||
}
|
||||
return map[string]any{"sha": sha, "parents": []any{map[string]any{"sha": parent}}}
|
||||
}
|
||||
|
||||
func filesArg(files ...map[string]any) []any {
|
||||
out := make([]any, len(files))
|
||||
for i, f := range files {
|
||||
@@ -398,27 +359,17 @@ func TestACommitAskThatCannotBeReadIsRefused(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// The forge's own answers (Gitea 1.27's Match), one table for this bundle and guard.ts alike (novox/hq issue 539).
|
||||
// `forge` is the forge's answer, which this port must give; `guard` is where guard.ts deliberately covers more.
|
||||
func TestRuleCoversAnswersAsTheForgeDoes(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../test/rule-cases.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var cases []struct {
|
||||
Rule string `json:"rule"`
|
||||
Branch string `json:"branch"`
|
||||
Forge bool `json:"forge"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &cases); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(cases) < 100 {
|
||||
t.Fatalf("only %d cases", len(cases))
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := ruleCovers(c.Rule, c.Branch); got != c.Forge {
|
||||
t.Errorf("%q covers %q: %v, the forge says %v", c.Rule, c.Branch, got, c.Forge)
|
||||
func TestRuleCoversIsTheGuardsRule(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
rule, branch string
|
||||
want bool
|
||||
}{
|
||||
{"main", "main", true}, {"main", "mainx", false}, {"release/*", "release/1.0", true},
|
||||
{"release/*", "release/a/b", false}, {"release/**", "release/a/b", true}, {"v?", "v1", true},
|
||||
{"a.b", "axb", false},
|
||||
} {
|
||||
if got := ruleCovers(c.rule, c.branch); got != c.want {
|
||||
t.Errorf("%s covers %s: %v", c.rule, c.branch, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -630,115 +581,3 @@ func TestToolsAreTheOnesTheReadmeNamesAndTheManifestBuilds(t *testing.T) {
|
||||
t.Error("the manifest builds no cmd/forge bundle")
|
||||
}
|
||||
}
|
||||
|
||||
// novox/hq issue 539: the forge takes a branch, never a commit to build on, so the answer's parent is the one
|
||||
// the forge made the commit on, and a branch that moves between the read and the commit is said.
|
||||
func TestAnExistingBranchMovedUnderTheCommitIsSaidAndItsParentIsTheOneTheForgeUsed(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.moves = map[string]move{"fix/1-old": {to: "c2", tree: map[string]string{"README.md": "hello\n", "docs/a.md": "a on fix\n", "other.md": "x\n"}}}
|
||||
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/1-old",
|
||||
"files": filesArg(map[string]any{"path": "docs/a.md", "content": "a fixed\n"})}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Parent != "c2" || f.branches["fix/1-old"] != got.Commit {
|
||||
t.Fatalf("parent %s, want c2 (the commit the forge built on): %+v", got.Parent, got)
|
||||
}
|
||||
said := strings.Join(got.Said, " ")
|
||||
if !strings.Contains(said, "moved from c1 to c2") || !strings.Contains(said, "after c2") {
|
||||
t.Fatalf("the move was not said: %v", got.Said)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnExistingBranchsFilesAreJudgedAtTheCommitReadNotAtWhereTheBranchMovedTo(t *testing.T) {
|
||||
f := newFake(t)
|
||||
// Someone else changes docs/a.md on the branch between the read and the commit: that change is not
|
||||
// overwritten by a blob read at the branch's new head.
|
||||
f.moves = map[string]move{"fix/1-old": {to: "c2", tree: map[string]string{"README.md": "hello\n", "docs/a.md": "theirs\n"}}}
|
||||
_, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/1-old",
|
||||
"files": filesArg(map[string]any{"path": "docs/a.md", "content": "mine\n"})}))
|
||||
if err == nil || !strings.Contains(err.Error(), "sha does not match") {
|
||||
t.Fatalf("a file changed under the commit was overwritten: %v", err)
|
||||
}
|
||||
if f.trees["c2"]["docs/a.md"] != "theirs\n" || f.branches["fix/1-old"] != "c2" {
|
||||
t.Fatalf("the branch was changed: %v", f.branches)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPatchOnABaseThatMovedNamesTheCommitTheForgeAppliedItTo(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.moves = map[string]move{"main": {to: "c9", tree: map[string]string{"README.md": "hello\n"}}}
|
||||
patch := "diff --git a/README.md b/README.md\n--- a/README.md\n+++ b/README.md\n@@ -1 +1 @@\n-hello\n+hi\n"
|
||||
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/6", "patch": patch}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Parent != "c9" || !strings.Contains(strings.Join(got.Said, " "), "base main moved from c0 to c9") {
|
||||
t.Fatalf("%+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAParentTheAnswerDoesNotSayIsReadFromTheCommit(t *testing.T) {
|
||||
f := newFake(t)
|
||||
f.noParents = true
|
||||
got, err := commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/7",
|
||||
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||
if err != nil || got.Parent != "c0" {
|
||||
t.Fatalf("%v %+v", err, got)
|
||||
}
|
||||
// A parent that cannot be read is a failure, said with the commit that was made.
|
||||
f = newFake(t)
|
||||
f.noParents, f.parentsBroken = true, true
|
||||
_, err = commit(context.Background(), f.serve(), ask(t, map[string]any{"branch": "fix/8",
|
||||
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||
if err == nil || !strings.Contains(err.Error(), "the forge made n1 on fix/8, but which commit it went on could not be read") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestACommitLargerThanItsLimitsIsRefusedBeforeAnythingIsSent(t *testing.T) {
|
||||
big := strings.Repeat("x", maxFileBytes+1)
|
||||
quarter := strings.Repeat("y", maxFileBytes)
|
||||
many := filesArg()
|
||||
for i := 0; i < maxCommitBytes/maxFileBytes+1; i++ {
|
||||
many = append(many, map[string]any{"path": fmt.Sprintf("f%d", i), "content": quarter})
|
||||
}
|
||||
for name, args := range map[string]map[string]any{
|
||||
"one file": {"files": filesArg(map[string]any{"path": "x", "content": big})},
|
||||
"one file b64": {"files": filesArg(map[string]any{"path": "x", "content_base64": base64.StdEncoding.EncodeToString([]byte(big))})},
|
||||
"the patch": {"patch": "diff --git a/x b/x\n" + big},
|
||||
"all files": {"files": many},
|
||||
} {
|
||||
full := map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b"}
|
||||
for k, v := range args {
|
||||
full[k] = v
|
||||
}
|
||||
_, err := readCommitAsk(full)
|
||||
if err == nil || !strings.Contains(err.Error(), "MiB") || !strings.Contains(err.Error(), "Nothing was sent") {
|
||||
t.Errorf("%s: %v", name, err)
|
||||
}
|
||||
}
|
||||
// At the limit is taken.
|
||||
if _, err := readCommitAsk(map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b",
|
||||
"files": filesArg(map[string]any{"path": "x", "content_base64": base64.StdEncoding.EncodeToString([]byte(quarter))})}); err != nil {
|
||||
t.Errorf("a file at the limit was refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodeSearchStopsAtItsDeadlineAndSaysSo(t *testing.T) {
|
||||
gz := archive(t, map[string]string{"a.md": "forge\n", "b.md": "forge\n"})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
out := &Found{Matches: []Match{}}
|
||||
if err := searchArchive(ctx, gz, "", regexp.MustCompile("forge"), 100, out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !out.Stopped || out.Files != 0 {
|
||||
t.Fatalf("a search past its deadline went on: %+v", out)
|
||||
}
|
||||
// Within its deadline, the same archive is searched whole.
|
||||
out = &Found{Matches: []Match{}}
|
||||
if err := searchArchive(context.Background(), gz, "", regexp.MustCompile("forge"), 100, out); err != nil || out.Stopped || len(out.Matches) != 2 {
|
||||
t.Fatalf("%v %+v", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,9 +53,7 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
|
||||
"independent review approved and whose required checks passed (gitea_merge_pull_request). Carries " +
|
||||
"either files — each {path, content (text) | content_base64 | delete: true}, created or updated as " +
|
||||
"the branch holds them — or patch, a unified diff as git diff writes it, which the forge applies. " +
|
||||
"At most 200 files, 4 MiB each and 16 MiB together, or a 4 MiB patch. Answers the commit it made and " +
|
||||
"the parent the forge made it on, read back from the branch, and says when the branch moved under " +
|
||||
"it. Replaces git commit and git " +
|
||||
"Answers the commit it made and its parent, read back from the branch. Replaces git commit and git " +
|
||||
"push of a feature branch. (a)",
|
||||
Input: map[string]any{
|
||||
"type": "object",
|
||||
|
||||
@@ -72,7 +72,7 @@ func history(ctx context.Context, f *Forge, owner, name, ref, path string, limit
|
||||
q.Set("verification", "false")
|
||||
q.Set("files", "false")
|
||||
var got []forgeCommit
|
||||
if err := f.GetJSON(ctx, repo+"/commits?"+q.Encode(), &got); err != nil {
|
||||
if err := f.getJSON(ctx, repo+"/commits?"+q.Encode(), &got); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
h := &History{Repository: owner + "/" + name, Ref: ref, Path: path, Page: page, Commits: []CommitLine{}, More: len(got) >= limit}
|
||||
@@ -121,7 +121,7 @@ func compare(ctx context.Context, f *Forge, owner, name, base, head string) (*Co
|
||||
Total int `json:"total_commits"`
|
||||
Commits []forgeCommit `json:"commits"`
|
||||
}
|
||||
if err := f.GetJSON(ctx, repo+"/compare/"+b+"..."+h, &got); err != nil {
|
||||
if err := f.getJSON(ctx, repo+"/compare/"+b+"..."+h, &got); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &Comparison{Repository: owner + "/" + name, Base: base, Head: head, Total: got.Total, Commits: []CommitLine{}, Files: []FileLine{}}
|
||||
@@ -195,7 +195,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
||||
SHA string `json:"sha"`
|
||||
} `json:"head"`
|
||||
}
|
||||
if err := f.GetJSON(ctx, pr, &pull); err != nil {
|
||||
if err := f.getJSON(ctx, pr, &pull); err != nil {
|
||||
if errors.Is(err, errNotFound) {
|
||||
return nil, refuse("%s/%s has no pull request %d", owner, name, number)
|
||||
}
|
||||
@@ -218,7 +218,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
||||
var all []forgeReview
|
||||
for p := 1; ; p++ {
|
||||
var batch []forgeReview
|
||||
if err := f.GetJSON(ctx, fmt.Sprintf("%s/reviews?limit=%d&page=%d", pr, page, p), &batch); err != nil {
|
||||
if err := f.getJSON(ctx, fmt.Sprintf("%s/reviews?limit=%d&page=%d", pr, page, p), &batch); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
all = append(all, batch...)
|
||||
@@ -240,7 +240,7 @@ func reviews(ctx context.Context, f *Forge, owner, name string, number int) (*Re
|
||||
CommitID string `json:"commit_id"`
|
||||
Resolver *forgeUser `json:"resolver"`
|
||||
}
|
||||
if err := f.GetJSON(ctx, fmt.Sprintf("%s/reviews/%d/comments", pr, r.ID), &cs); err != nil {
|
||||
if err := f.getJSON(ctx, fmt.Sprintf("%s/reviews/%d/comments", pr, r.ID), &cs); err != nil {
|
||||
return nil, fmt.Errorf("review %d's line comments could not be read: %w", r.ID, err)
|
||||
}
|
||||
for _, c := range cs {
|
||||
@@ -292,7 +292,7 @@ func listDir(ctx context.Context, f *Forge, owner, name, path, ref string) (*Lis
|
||||
if ref != "" {
|
||||
at += "?ref=" + queryEscape(ref)
|
||||
}
|
||||
body, err := f.Do(ctx, "GET", at, nil, maxBody)
|
||||
body, err := f.do(ctx, "GET", at, nil, maxBody)
|
||||
if err != nil {
|
||||
if errors.Is(err, errNotFound) {
|
||||
return nil, refuse("%s/%s has no %q at %s", owner, name, path, orDefault(ref))
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ruleCovers is whether a protection rule's name covers a branch, as the forge itself decides it (Gitea 1.27,
|
||||
// models/git/protected_branch.go Match and modules/glob, novox/hq issue 539): a name without a glob character
|
||||
// covers the branch of that name in any case; any other name is a glob with `/` as its separator — `*` any
|
||||
// run within a segment, `**` any run across them, `?` one character of a segment, `[abc]`, `[a-z]` and `[!abc]`
|
||||
// one character, `{a,b}` either, `\` the next character itself — matched case-sensitively; a glob the forge
|
||||
// cannot compile covers its own text exactly. The TypeScript bundle's ruleCovers (guard.ts) is the same rule.
|
||||
func ruleCovers(rule, branch string) bool {
|
||||
if !strings.ContainsAny(rule, `*?\[]{}`) {
|
||||
return strings.EqualFold(rule, branch)
|
||||
}
|
||||
re, err := globRegexp(rule)
|
||||
if err != nil {
|
||||
return rule == branch
|
||||
}
|
||||
return re.MatchString(branch)
|
||||
}
|
||||
|
||||
// globRegexp is the forge's glob as a regular expression, compiled as modules/glob compiles it with `/` as
|
||||
// the separator.
|
||||
func globRegexp(glob string) (*regexp.Regexp, error) {
|
||||
g := []rune(glob)
|
||||
var b strings.Builder
|
||||
pos := 0
|
||||
var compile func(sub bool) error
|
||||
chars := func() error {
|
||||
b.WriteByte('[')
|
||||
if pos < len(g) && g[pos] == '!' {
|
||||
pos++
|
||||
b.WriteByte('^')
|
||||
}
|
||||
for pos < len(g) {
|
||||
c := g[pos]
|
||||
pos++
|
||||
switch {
|
||||
case c == ']':
|
||||
b.WriteByte(']')
|
||||
return nil
|
||||
case c == '\\':
|
||||
if pos >= len(g) {
|
||||
return errors.New("unterminated character class escape")
|
||||
}
|
||||
b.WriteByte('\\')
|
||||
b.WriteRune(g[pos])
|
||||
pos++
|
||||
default:
|
||||
b.WriteRune(c)
|
||||
}
|
||||
}
|
||||
return errors.New("unterminated character class")
|
||||
}
|
||||
compile = func(sub bool) error {
|
||||
for pos < len(g) {
|
||||
c := g[pos]
|
||||
pos++
|
||||
if sub && c == '}' {
|
||||
b.WriteByte(')')
|
||||
return nil
|
||||
}
|
||||
switch c {
|
||||
case '*':
|
||||
if pos < len(g) && g[pos] == '*' {
|
||||
pos++
|
||||
b.WriteString(".*")
|
||||
} else {
|
||||
b.WriteString("[^/]*")
|
||||
}
|
||||
case '?':
|
||||
b.WriteString("[^/]")
|
||||
case '+':
|
||||
b.WriteString(`\+`)
|
||||
case '[':
|
||||
if err := chars(); err != nil {
|
||||
return err
|
||||
}
|
||||
case '{':
|
||||
b.WriteByte('(')
|
||||
if err := compile(true); err != nil {
|
||||
return err
|
||||
}
|
||||
case ',':
|
||||
if sub {
|
||||
b.WriteByte('|')
|
||||
} else {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
case '\\':
|
||||
if pos >= len(g) {
|
||||
return errors.New("no character to escape")
|
||||
}
|
||||
b.WriteByte('\\')
|
||||
b.WriteRune(g[pos])
|
||||
pos++
|
||||
case '.', '^', '$', '(', ')', '|':
|
||||
b.WriteByte('\\')
|
||||
b.WriteRune(c)
|
||||
default:
|
||||
b.WriteRune(c)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
b.WriteByte('^')
|
||||
if err := compile(false); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.WriteByte('$')
|
||||
return regexp.Compile(b.String())
|
||||
}
|
||||
@@ -32,7 +32,6 @@ type Found struct {
|
||||
Files int `json:"files_searched"`
|
||||
Skipped []string `json:"skipped,omitempty"`
|
||||
Truncated bool `json:"truncated"`
|
||||
Stopped bool `json:"stopped_at_deadline,omitempty"`
|
||||
Said []string `json:"said"`
|
||||
}
|
||||
|
||||
@@ -70,7 +69,7 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
||||
ref := a.Ref
|
||||
if ref == "" {
|
||||
var r Repo
|
||||
if err := f.GetJSON(ctx, repo, &r); err != nil {
|
||||
if err := f.getJSON(ctx, repo, &r); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r.Empty || r.DefaultBranch == "" {
|
||||
@@ -88,7 +87,7 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
body, err := f.Do(ctx, "GET", repo+"/archive/"+seg+".tar.gz", nil, maxArchive)
|
||||
body, err := f.do(ctx, "GET", repo+"/archive/"+seg+".tar.gz", nil, maxArchive)
|
||||
if err != nil {
|
||||
if errors.Is(err, errNotFound) {
|
||||
return nil, refuse("%s/%s has no ref %s", a.Owner, a.Repo, ref)
|
||||
@@ -96,15 +95,11 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
||||
return nil, err
|
||||
}
|
||||
out := &Found{Repository: a.Owner + "/" + a.Repo, Ref: ref, Query: a.Query, Matches: []Match{}}
|
||||
if err := searchArchive(ctx, body, prefix, re, a.Max, out); err != nil {
|
||||
if err := searchArchive(body, prefix, re, a.Max, out); err != nil {
|
||||
return nil, fmt.Errorf("the archive of %s at %s could not be read: %w", out.Repository, ref, err)
|
||||
}
|
||||
said := fmt.Sprintf("%d matches in %d files searched", len(out.Matches), out.Files)
|
||||
switch {
|
||||
case out.Stopped:
|
||||
said = fmt.Sprintf("the search stopped at its deadline of %s after %d files, with %d matches: the rest of the "+
|
||||
"archive was not searched. Narrow with path", deadline, out.Files, len(out.Matches))
|
||||
case out.Truncated:
|
||||
if out.Truncated {
|
||||
said = fmt.Sprintf("the first %d matches, and more were found: narrow with path or the query", len(out.Matches))
|
||||
}
|
||||
out.Said = append(out.Said, said)
|
||||
@@ -115,32 +110,18 @@ func searchCode(ctx context.Context, f *Forge, a SearchAsk) (*Found, error) {
|
||||
}
|
||||
|
||||
// searchArchive reads a gzipped tar the forge made — every path under one top directory — and collects the
|
||||
// lines that match, from text files under the prefix. It stops at the call's deadline (novox/hq issue 539):
|
||||
// what it found by then is kept, and out.Stopped says the rest was not searched.
|
||||
func searchArchive(ctx context.Context, gz []byte, prefix string, re *regexp.Regexp, max int, out *Found) error {
|
||||
// lines that match, from text files under the prefix.
|
||||
func searchArchive(gz []byte, prefix string, re *regexp.Regexp, max int, out *Found) error {
|
||||
zr, err := gzip.NewReader(bytes.NewReader(gz))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tr := tar.NewReader(ctxReader{ctx, zr})
|
||||
stopped := func(err error) bool {
|
||||
if ctx.Err() != nil && (err == nil || errors.Is(err, ctx.Err())) {
|
||||
out.Stopped = true
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
tr := tar.NewReader(zr)
|
||||
for {
|
||||
if stopped(nil) {
|
||||
return nil
|
||||
}
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
return nil
|
||||
}
|
||||
if stopped(err) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -159,9 +140,6 @@ func searchArchive(ctx context.Context, gz []byte, prefix string, re *regexp.Reg
|
||||
continue
|
||||
}
|
||||
content, err := io.ReadAll(io.LimitReader(tr, maxFileSearched+1))
|
||||
if stopped(err) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -191,16 +169,3 @@ func searchArchive(ctx context.Context, gz []byte, prefix string, re *regexp.Reg
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ctxReader is a reader that ends when its context does: decompressing is the long part of a search.
|
||||
type ctxReader struct {
|
||||
ctx context.Context
|
||||
r io.Reader
|
||||
}
|
||||
|
||||
func (c ctxReader) Read(p []byte) (int, error) {
|
||||
if err := c.ctx.Err(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return c.r.Read(p)
|
||||
}
|
||||
|
||||
@@ -2,33 +2,45 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
|
||||
// (internal/forgehttp, shared with the forge bundle beside it). Basic authentication, so this bundle mints
|
||||
// nothing and keeps nothing: the TypeScript bundle's token is its own.
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin
|
||||
// account, which the vault delivered into a file this module owns (novox/hq ADR 0086). Basic
|
||||
// authentication, so this bundle mints nothing and keeps nothing: the TypeScript bundle's token is its
|
||||
// own. The password is read when a call is made and never logged, answered or put in an error.
|
||||
|
||||
// Forge is the forge's HTTP interface.
|
||||
type Forge struct {
|
||||
forgehttp.Client
|
||||
Owner string // the package registry's owner, as the seat serves it
|
||||
URL string // e.g. http://127.0.0.1:3000
|
||||
User string
|
||||
PasswordFile string
|
||||
Owner string // the package registry's owner, as the seat serves it
|
||||
HTTP *http.Client
|
||||
}
|
||||
|
||||
// ForgeFromEnv is the forge as the manifest hands it to this bundle.
|
||||
func ForgeFromEnv() (*Forge, error) {
|
||||
c, missing := forgehttp.FromEnv()
|
||||
f := &Forge{Client: c, Owner: os.Getenv("MESH_NPM_OWNER")}
|
||||
if f.Owner == "" {
|
||||
missing = append(missing, "MESH_NPM_OWNER")
|
||||
f := &Forge{
|
||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
||||
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
||||
Owner: os.Getenv("MESH_NPM_OWNER"),
|
||||
}
|
||||
var missing []string
|
||||
for name, v := range map[string]string{"MESH_GITEA_URL": f.URL, "MESH_GITEA_ADMIN_USER": f.User,
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": f.PasswordFile, "MESH_NPM_OWNER": f.Owner} {
|
||||
if v == "" {
|
||||
missing = append(missing, name)
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest", strings.Join(missing, ", "))
|
||||
@@ -37,14 +49,86 @@ func ForgeFromEnv() (*Forge, error) {
|
||||
}
|
||||
|
||||
// errNotFound is the forge answering 404.
|
||||
var errNotFound = forgehttp.ErrNotFound
|
||||
var errNotFound = errors.New("not found")
|
||||
|
||||
// statusError is an answer the forge gave that is not a success, said without the request's credential.
|
||||
type statusError struct {
|
||||
method, path string
|
||||
status int
|
||||
said string
|
||||
}
|
||||
|
||||
func (e *statusError) Error() string {
|
||||
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.status, e.method, e.path, e.said)
|
||||
}
|
||||
|
||||
func (f *Forge) client() *http.Client {
|
||||
if f.HTTP != nil {
|
||||
return f.HTTP
|
||||
}
|
||||
return &http.Client{Timeout: 20 * time.Second}
|
||||
}
|
||||
|
||||
func (f *Forge) password() (string, error) {
|
||||
raw, err := os.ReadFile(f.PasswordFile)
|
||||
if err != nil {
|
||||
// The path is the manifest's, not a secret; the content never appears.
|
||||
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
||||
}
|
||||
return strings.TrimSpace(string(raw)), nil
|
||||
}
|
||||
|
||||
// maxBody is the largest answer read from the forge.
|
||||
const maxBody = forgehttp.MaxBody
|
||||
const maxBody = 32 << 20
|
||||
|
||||
// do sends one request without a body and answers the body; 404 is errNotFound.
|
||||
// do sends one request and answers the body; 404 is errNotFound.
|
||||
func (f *Forge) do(ctx context.Context, method, path string) ([]byte, error) {
|
||||
return f.Do(ctx, method, path, nil, maxBody)
|
||||
password, err := f.password()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, f.URL+path, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.SetBasicAuth(f.User, password)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
res, err := f.client().Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
// One byte past the cap is read, so a body the cap cut is told from one that fits it exactly: a
|
||||
// cut body is refused, never parsed as if it were whole.
|
||||
body, err := io.ReadAll(io.LimitReader(res.Body, maxBody+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(body) > maxBody {
|
||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, maxBody>>20)
|
||||
}
|
||||
switch {
|
||||
case res.StatusCode == http.StatusNotFound:
|
||||
return nil, fmt.Errorf("%s %s: %w", method, path, errNotFound)
|
||||
case res.StatusCode >= 300:
|
||||
said := strings.TrimSpace(string(body))
|
||||
if len(said) > 300 {
|
||||
said = said[:300] + "…"
|
||||
}
|
||||
return nil, &statusError{method, path, res.StatusCode, said}
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func (f *Forge) getJSON(ctx context.Context, path string, into any) error {
|
||||
body, err := f.do(ctx, http.MethodGet, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(body, into); err != nil {
|
||||
return fmt.Errorf("the forge's answer to %s is not readable: %w", path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// PackageVersion is one version of one package, as the forge lists it.
|
||||
@@ -62,7 +146,7 @@ func (f *Forge) Packages(ctx context.Context) ([]PackageVersion, error) {
|
||||
for page := 1; page <= 1000; page++ {
|
||||
var batch []PackageVersion
|
||||
path := fmt.Sprintf("/api/v1/packages/%s?type=npm&page=%d&limit=%d", url.PathEscape(f.Owner), page, pageSize)
|
||||
if err := f.GetJSON(ctx, path, &batch); err != nil {
|
||||
if err := f.getJSON(ctx, path, &batch); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
all = append(all, batch...)
|
||||
@@ -82,7 +166,7 @@ func (f *Forge) Size(ctx context.Context, name, version string) (int64, error) {
|
||||
var files []struct {
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
if err := f.GetJSON(ctx, f.versionPath(name, version)+"/files", &files); err != nil {
|
||||
if err := f.getJSON(ctx, f.versionPath(name, version)+"/files", &files); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var total int64
|
||||
@@ -98,7 +182,7 @@ func (f *Forge) DistTags(ctx context.Context, name string) (map[string]string, e
|
||||
DistTags map[string]string `json:"dist-tags"`
|
||||
}
|
||||
path := fmt.Sprintf("/api/packages/%s/npm/%s", url.PathEscape(f.Owner), url.PathEscape(name))
|
||||
if err := f.GetJSON(ctx, path, &meta); err != nil {
|
||||
if err := f.getJSON(ctx, path, &meta); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return meta.DistTags, nil
|
||||
@@ -124,7 +208,7 @@ func (f *Forge) Repositories(ctx context.Context) ([]Repository, error) {
|
||||
var found struct {
|
||||
Data []Repository `json:"data"`
|
||||
}
|
||||
if err := f.GetJSON(ctx, fmt.Sprintf("/api/v1/repos/search?page=%d&limit=%d", page, pageSize), &found); err != nil {
|
||||
if err := f.getJSON(ctx, fmt.Sprintf("/api/v1/repos/search?page=%d&limit=%d", page, pageSize), &found); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
all = append(all, found.Data...)
|
||||
@@ -155,7 +239,7 @@ func (f *Forge) Files(ctx context.Context, repo Repository) ([]string, error) {
|
||||
}
|
||||
path := fmt.Sprintf("/api/v1/repos/%s/git/trees/%s?recursive=true&page=%d&per_page=1000",
|
||||
repoPath(repo.FullName), url.PathEscape(repo.DefaultBranch), page)
|
||||
if err := f.GetJSON(ctx, path, &tree); err != nil {
|
||||
if err := f.getJSON(ctx, path, &tree); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, e := range tree.Tree {
|
||||
|
||||
@@ -15,8 +15,6 @@ import (
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// fakeForge answers the forge's interface from maps, and records every delete.
|
||||
@@ -40,7 +38,7 @@ func (f *fakeForge) serve() *Forge {
|
||||
if err := os.WriteFile(file, []byte(f.password+"\n"), 0o600); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
return &Forge{Client: forgehttp.Client{URL: srv.URL, User: "admin", PasswordFile: file}, Owner: "acme"}
|
||||
return &Forge{URL: srv.URL, User: "admin", PasswordFile: file, Owner: "acme"}
|
||||
}
|
||||
|
||||
var (
|
||||
@@ -503,7 +501,7 @@ func TestAnAnswerTheCapCutIsRefused(t *testing.T) {
|
||||
if err := os.WriteFile(file, []byte("x\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f := &Forge{Client: forgehttp.Client{URL: srv.URL, User: "admin", PasswordFile: file}, Owner: "acme"}
|
||||
f := &Forge{URL: srv.URL, User: "admin", PasswordFile: file, Owner: "acme"}
|
||||
if _, err := f.do(context.Background(), http.MethodGet, "/big"); err == nil || !strings.Contains(err.Error(), "not read whole") {
|
||||
t.Fatalf("a cut answer: %v", err)
|
||||
}
|
||||
|
||||
+13
-98
@@ -164,106 +164,21 @@ export function judgeApiCall(method: string, rawPath: string, body: unknown): Ap
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a protection rule's name covers a branch, as the forge itself decides it (Gitea 1.27,
|
||||
* models/git/protected_branch.go Match and modules/glob; novox/hq issue 539): a name without a glob character
|
||||
* covers the branch of that name in any case; any other name is a glob with `/` as its separator — `*` any run
|
||||
* within a segment, `**` any run across them, `?` one character of a segment, `[abc]`, `[a-z]` and `[!abc]` one
|
||||
* character, `{a,b}` either, `\` the next character itself — matched case-sensitively; a glob the forge cannot
|
||||
* compile covers its own text exactly. The forge bundle's ruleCovers (cmd/forge/rules.go) is the same rule, and
|
||||
* both are held to one table of answers (test/rule-cases.json).
|
||||
*
|
||||
* The forge hands the compiled glob to Go's regexp, which reads some things JavaScript's reads otherwise: a
|
||||
* backslash before a letter or digit (`\d`, `\w`, `\pL`, `\b`, `\A` are classes or anchors there), a class
|
||||
* that opens with `]` (`[]a]` holds `]` and `a` there, an empty class here) or with `^` (a negation there),
|
||||
* and a POSIX class (`[[:digit:]]`).
|
||||
* A rule using any of them is taken to cover every branch: refusing a write the forge would take is safe,
|
||||
* taking one it guards is not.
|
||||
*/
|
||||
/** Whether a protection rule's name — a branch, or a glob of them (`*`, `**`) — covers a branch. */
|
||||
export function ruleCovers(rule: string, branch: string): boolean {
|
||||
if (!/[*?\\[\]{}]/.test(rule)) return rule.toLowerCase() === branch.toLowerCase();
|
||||
if (goReadsOtherwise(rule)) return true;
|
||||
const re = globRegExp(rule);
|
||||
return re ? re.test(branch) : rule === branch;
|
||||
}
|
||||
|
||||
/** Whether a glob holds what Go's regexp reads otherwise than JavaScript's (see ruleCovers): fail closed. */
|
||||
function goReadsOtherwise(rule: string): boolean {
|
||||
return /\\[\p{L}\p{N}]/u.test(rule) || /\[!?[\]^]/.test(rule) || /\[[^\]]*\[:/.test(rule);
|
||||
}
|
||||
|
||||
/** The forge's glob as a regular expression, compiled as its modules/glob compiles it with `/` as separator. */
|
||||
function globRegExp(glob: string): RegExp | null {
|
||||
const g = Array.from(glob);
|
||||
let out = "";
|
||||
let pos = 0;
|
||||
// A character as itself: escaped when it is syntax to a regular expression, and only then — the `u` flag
|
||||
// refuses any other escape, and `-` may be escaped only inside a class.
|
||||
const lit = (c: string) => c.replace(/[.*+?^${}()|[\]\\/]/g, "\\$&");
|
||||
const litInClass = (c: string) => c.replace(/[\]\\[^-]/g, "\\$&");
|
||||
const chars = (): boolean => {
|
||||
out += "[";
|
||||
if (g[pos] === "!") {
|
||||
pos++;
|
||||
out += "^";
|
||||
}
|
||||
while (pos < g.length) {
|
||||
const c = g[pos++];
|
||||
if (c === "]") {
|
||||
out += "]";
|
||||
return true;
|
||||
}
|
||||
if (c === "\\") {
|
||||
if (pos >= g.length) return false;
|
||||
out += litInClass(g[pos++]);
|
||||
} else out += c === "[" || c === "^" ? "\\" + c : c;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
const compile = (sub: boolean): boolean => {
|
||||
while (pos < g.length) {
|
||||
const c = g[pos++];
|
||||
if (sub && c === "}") {
|
||||
out += ")";
|
||||
return true;
|
||||
}
|
||||
switch (c) {
|
||||
case "*":
|
||||
if (g[pos] === "*") {
|
||||
pos++;
|
||||
out += ".*";
|
||||
} else out += "[^/]*";
|
||||
break;
|
||||
case "?":
|
||||
out += "[^/]";
|
||||
break;
|
||||
case "[":
|
||||
if (!chars()) return false;
|
||||
break;
|
||||
case "{":
|
||||
out += "(";
|
||||
if (!compile(true)) return false;
|
||||
break;
|
||||
case ",":
|
||||
out += sub ? "|" : ",";
|
||||
break;
|
||||
case "\\":
|
||||
if (pos >= g.length) return false;
|
||||
out += lit(g[pos++]);
|
||||
break;
|
||||
default:
|
||||
out += lit(c);
|
||||
}
|
||||
}
|
||||
// An alternative left open is a glob the forge cannot compile.
|
||||
return !sub;
|
||||
};
|
||||
if (!compile(false)) return null;
|
||||
try {
|
||||
return new RegExp(`^${out}$`, "u");
|
||||
} catch {
|
||||
return null;
|
||||
if (rule === branch) return true;
|
||||
if (!/[*?[]/.test(rule)) return false;
|
||||
let re = "";
|
||||
for (let i = 0; i < rule.length; i++) {
|
||||
const c = rule[i];
|
||||
if (c === "*" && rule[i + 1] === "*") {
|
||||
re += ".*";
|
||||
i++;
|
||||
} else if (c === "*") re += "[^/]*";
|
||||
else if (c === "?") re += "[^/]";
|
||||
else re += c.replace(/[.+^${}()|[\]\\]/g, "\\$&");
|
||||
}
|
||||
return new RegExp(`^${re}$`).test(branch);
|
||||
}
|
||||
|
||||
/** Refuses a file written to the default branch or to a branch a protection rule names. */
|
||||
|
||||
@@ -1,151 +0,0 @@
|
||||
// Package forgehttp is the forge's HTTP interface as this module's Go bundles reach it (cmd/forge,
|
||||
// cmd/npm-registry): the forge on its own machine, as its admin account, whose password the vault delivered
|
||||
// into a file this module owns (novox/hq ADR 0086), by basic authentication. The password is read when a call
|
||||
// is made and never logged, answered or put in an error. One copy for both bundles (novox/hq issue 539).
|
||||
package forgehttp
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// MaxBody is the largest JSON answer read from the forge.
|
||||
const MaxBody = 32 << 20
|
||||
|
||||
// Client is the forge's HTTP interface.
|
||||
type Client struct {
|
||||
URL string // e.g. http://127.0.0.1:3000
|
||||
User string
|
||||
PasswordFile string
|
||||
HTTP *http.Client // when nil, a client with Timeout
|
||||
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
|
||||
}
|
||||
|
||||
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset.
|
||||
func FromEnv() (Client, []string) {
|
||||
c := Client{
|
||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
||||
PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
||||
}
|
||||
var missing []string
|
||||
for _, kv := range [][2]string{{"MESH_GITEA_URL", c.URL}, {"MESH_GITEA_ADMIN_USER", c.User},
|
||||
{"MESH_GITEA_ADMIN_PASSWORD_FILE", c.PasswordFile}} {
|
||||
if kv[1] == "" {
|
||||
missing = append(missing, kv[0])
|
||||
}
|
||||
}
|
||||
return c, missing
|
||||
}
|
||||
|
||||
// ErrNotFound is the forge answering 404.
|
||||
var ErrNotFound = errors.New("not found")
|
||||
|
||||
// StatusError is an answer the forge gave that is not a success, said without the request's credential.
|
||||
type StatusError struct {
|
||||
Method, Path string
|
||||
Status int
|
||||
Said string
|
||||
}
|
||||
|
||||
func (e *StatusError) Error() string {
|
||||
return fmt.Sprintf("the forge answered %d to %s %s: %s", e.Status, e.Method, e.Path, e.Said)
|
||||
}
|
||||
|
||||
func (c *Client) client() *http.Client {
|
||||
if c.HTTP != nil {
|
||||
return c.HTTP
|
||||
}
|
||||
timeout := c.Timeout
|
||||
if timeout <= 0 {
|
||||
timeout = 20 * time.Second
|
||||
}
|
||||
return &http.Client{Timeout: timeout}
|
||||
}
|
||||
|
||||
func (c *Client) password() (string, error) {
|
||||
raw, err := os.ReadFile(c.PasswordFile)
|
||||
if err != nil {
|
||||
// The path is the manifest's, not a secret; the content never appears.
|
||||
return "", fmt.Errorf("the forge's admin password cannot be read from its file: %w", err)
|
||||
}
|
||||
return strings.TrimSpace(string(raw)), nil
|
||||
}
|
||||
|
||||
// Do sends one request, with payload as its JSON body when not nil, and answers the body, read up to limit;
|
||||
// 404 is ErrNotFound. One byte past the limit is read, so a body the limit cut is told from one that fits it
|
||||
// exactly: a cut body is refused, never read as if it were whole.
|
||||
func (c *Client) Do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
|
||||
password, err := c.password()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var body io.Reader
|
||||
if payload != nil {
|
||||
raw, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body = bytes.NewReader(raw)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, c.URL+path, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.SetBasicAuth(c.User, password)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
if payload != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
res, err := c.client().Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the forge did not answer %s %s: %w", method, path, err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
got, err := io.ReadAll(io.LimitReader(res.Body, limit+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if int64(len(got)) > limit {
|
||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
|
||||
}
|
||||
switch {
|
||||
case res.StatusCode == http.StatusNotFound:
|
||||
return nil, fmt.Errorf("%s %s: %w", method, path, ErrNotFound)
|
||||
case res.StatusCode >= 300:
|
||||
said := strings.TrimSpace(string(got))
|
||||
if len(said) > 300 {
|
||||
said = said[:300] + "…"
|
||||
}
|
||||
return nil, &StatusError{method, path, res.StatusCode, said}
|
||||
}
|
||||
return got, nil
|
||||
}
|
||||
|
||||
// GetJSON reads one answer into into.
|
||||
func (c *Client) GetJSON(ctx context.Context, path string, into any) error {
|
||||
return c.SendJSON(ctx, http.MethodGet, path, nil, into)
|
||||
}
|
||||
|
||||
// SendJSON sends one request and reads its answer into into, when into is not nil.
|
||||
func (c *Client) SendJSON(ctx context.Context, method, path string, payload, into any) error {
|
||||
body, err := c.Do(ctx, method, path, payload, MaxBody)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if into == nil {
|
||||
return nil
|
||||
}
|
||||
if err := json.Unmarshal(body, into); err != nil {
|
||||
return fmt.Errorf("the forge's answer to %s %s is not readable: %w", method, path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -7,7 +7,6 @@
|
||||
import { test, after, beforeEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
import { GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken } from "../dist/token.js";
|
||||
@@ -147,25 +146,6 @@ test("gitea_api refuses a file written to the default branch or one a rule guard
|
||||
assert.deepEqual(writes(), ["PUT /repos/novox/mesh-catalog/contents/modules/sudo/module.json"]);
|
||||
});
|
||||
|
||||
test("a protection rule covers a branch as the forge decides it: classes, alternatives, a plain name in any case (issue 539)", async () => {
|
||||
// One table with the forge bundle (cmd/forge/rules.go): `forge` is the forge's answer; `guard` is where this
|
||||
// guard deliberately covers more — a rule Go's regexp reads otherwise than JavaScript's — and never less.
|
||||
const cases = JSON.parse(readFileSync(new URL("./rule-cases.json", import.meta.url), "utf8")) as
|
||||
Array<{ rule: string; branch: string; forge: boolean; guard?: boolean }>;
|
||||
assert.ok(cases.length >= 100);
|
||||
for (const c of cases) {
|
||||
assert.ok(c.guard === undefined || (c.guard && !c.forge), `${c.rule}: the guard may only cover more than the forge`);
|
||||
assert.equal(ruleCovers(c.rule, c.branch), c.guard ?? c.forge, `${c.rule} covers ${c.branch}`);
|
||||
}
|
||||
const put = (branch: string) => tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/contents/README.md",
|
||||
method: "PUT", body: { content: "e30=", sha: "x", branch } });
|
||||
state.rules.push({ rule_name: "release-[ab]" }, { rule_name: "{stable,lts}/*" });
|
||||
await assert.rejects(put("release-a"), /protection rule release-\[ab\]/);
|
||||
await assert.rejects(put("lts/2"), /protection rule \{stable,lts\}\/\*/);
|
||||
await assert.rejects(put("MAIN"), /protection rule main/);
|
||||
assert.deepEqual(writes(), []);
|
||||
});
|
||||
|
||||
test("gitea_api still writes what an author writes", async () => {
|
||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/issues/3/comments", method: "POST", body: { body: "hi" } });
|
||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/pulls/7/reviews", method: "POST", body: { event: "COMMENT" } });
|
||||
|
||||
@@ -1,285 +0,0 @@
|
||||
[
|
||||
{"rule":"main","branch":"main","forge":true,"why":"a plain name covers its branch"},
|
||||
{"rule":"main","branch":"mainx","forge":false},
|
||||
{"rule":"main","branch":"MAIN","forge":true,"why":"a plain name covers in any case (EqualFold)"},
|
||||
{"rule":"Main","branch":"main","forge":true},
|
||||
{"rule":"release/*","branch":"release/1.0","forge":true,"why":"* stays within a segment"},
|
||||
{"rule":"release/*","branch":"release/a/b","forge":false},
|
||||
{"rule":"release/**","branch":"release/a/b","forge":true,"why":"** crosses segments"},
|
||||
{"rule":"v?","branch":"v1","forge":true},
|
||||
{"rule":"v?","branch":"v/","forge":false,"why":"? is not the separator"},
|
||||
{"rule":"a.b","branch":"axb","forge":false,"why":". is itself"},
|
||||
{"rule":"Feat/*","branch":"feat/x","forge":false,"why":"a glob is matched with case"},
|
||||
{"rule":"release-[ab]","branch":"release-a","forge":true,"why":"a class"},
|
||||
{"rule":"release-[ab]","branch":"release-c","forge":false},
|
||||
{"rule":"release-[!ab]","branch":"release-c","forge":true,"why":"a negated class"},
|
||||
{"rule":"release-[!ab]","branch":"release-a","forge":false},
|
||||
{"rule":"v[0-9].x","branch":"v7.x","forge":true,"why":"a range"},
|
||||
{"rule":"v[0-9].x","branch":"vA.x","forge":false},
|
||||
{"rule":"{main,master}","branch":"master","forge":true,"why":"alternatives"},
|
||||
{"rule":"{main,master}","branch":"mainline","forge":false},
|
||||
{"rule":"rel/{a,b}/*","branch":"rel/b/1","forge":true},
|
||||
{"rule":"rel/{a,b}/*","branch":"rel/c/1","forge":false},
|
||||
{"rule":"a\\*","branch":"a*","forge":true,"why":"an escape"},
|
||||
{"rule":"a\\*","branch":"ab","forge":false},
|
||||
{"rule":"a+b","branch":"a+b","forge":true,"why":"+ is itself"},
|
||||
{"rule":"a+b","branch":"aab","forge":false},
|
||||
{"rule":"{open","branch":"{open","forge":true,"why":"a glob the forge cannot compile covers its own text"},
|
||||
{"rule":"[open","branch":"[open","forge":true},
|
||||
{"rule":"[open","branch":"o","forge":false},
|
||||
{"rule":"release-\\d","branch":"release-1","forge":true,"why":"Go reads \\d as a digit"},
|
||||
{"rule":"release-\\d","branch":"release-d","forge":false,"guard":true},
|
||||
{"rule":"v\\w*","branch":"v2","forge":true},
|
||||
{"rule":"main\\s","branch":"main","forge":false,"guard":true},
|
||||
{"rule":"[]a]","branch":"a","forge":true,"why":"a class opening with ] holds ]"},
|
||||
{"rule":"[]a]","branch":"]","forge":true},
|
||||
{"rule":"[]a]","branch":"b","forge":false,"guard":true},
|
||||
{"rule":"rel-[\\d]","branch":"rel-1","forge":true},
|
||||
{"rule":"v[\\pL]","branch":"va","forge":true,"why":"\\p is a Unicode class"},
|
||||
{"rule":"v\\pL","branch":"va","forge":true},
|
||||
{"rule":"[[:alpha:]]","branch":"a","forge":true,"why":"a POSIX class"},
|
||||
{"rule":"rel-[[:digit:]]","branch":"rel-1","forge":true},
|
||||
{"rule":"rel-[[:digit:]]","branch":"rel-x","forge":false,"guard":true},
|
||||
{"rule":"[^+]*","branch":"-[1","forge":true,"why":"a class opening with ^ is negated in Go"},
|
||||
{"rule":"d[^](","branch":"d[^](","forge":true},
|
||||
{"rule":"]?","branch":":,1},","forge":false},
|
||||
{"rule":"Ab,","branch":"Ab,","forge":true},
|
||||
{"rule":"\\:[}-|","branch":",{","forge":false},
|
||||
{"rule":"^,*.d::","branch":"/x","forge":false},
|
||||
{"rule":"","branch":"A*:","forge":false},
|
||||
{"rule":"!|A{","branch":":.{]","forge":false},
|
||||
{"rule":"","branch":"]éb","forge":false},
|
||||
{"rule":"/{)w","branch":"/{)w","forge":true},
|
||||
{"rule":"*^w]","branch":"}!.","forge":false},
|
||||
{"rule":"*),^?*d","branch":"*),^?*d","forge":true},
|
||||
{"rule":"A)^}!","branch":"","forge":false},
|
||||
{"rule":"}^d?","branch":"1\\.","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"$?[","branch":"$?[","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"+","branch":"+","forge":true},
|
||||
{"rule":"","branch":"},[","forge":false},
|
||||
{"rule":"xA[d:a)","branch":"]1.!bx","forge":false},
|
||||
{"rule":"^/}(","branch":"","forge":false},
|
||||
{"rule":"-|x.x1?","branch":"-|x.x1?","forge":true},
|
||||
{"rule":"x+)A]-A","branch":"x+)A]-A","forge":true},
|
||||
{"rule":"x*a","branch":"x*a","forge":true},
|
||||
{"rule":"$xb","branch":"}é","forge":false},
|
||||
{"rule":"wAx?/a,","branch":"","forge":false},
|
||||
{"rule":"[,","branch":"[,","forge":true},
|
||||
{"rule":"}!","branch":"}!","forge":true},
|
||||
{"rule":"*:|\\w","branch":"","forge":false,"guard":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"é","branch":"é","forge":true},
|
||||
{"rule":"..(","branch":"{\\{+","forge":false},
|
||||
{"rule":"*|{","branch":"1.}-","forge":false},
|
||||
{"rule":"!+,","branch":"}++-*","forge":false},
|
||||
{"rule":"x\\({","branch":"*","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"?-{+-","branch":"?-{+-","forge":true},
|
||||
{"rule":"/-{é|$","branch":"b!.[","forge":false},
|
||||
{"rule":"a-+dxA","branch":"1AA,xA","forge":false},
|
||||
{"rule":"]|(","branch":"]|(","forge":true},
|
||||
{"rule":"}","branch":"{!x{d","forge":false},
|
||||
{"rule":"/{*](","branch":"!]","forge":false},
|
||||
{"rule":"($+{-}|","branch":"/","forge":false},
|
||||
{"rule":"(-b.-1}","branch":"*A*+,1","forge":false},
|
||||
{"rule":"1[d|)wa","branch":"+}[!}}","forge":false},
|
||||
{"rule":"w","branch":":.","forge":false},
|
||||
{"rule":"?}\\*","branch":"","forge":false},
|
||||
{"rule":"?$*","branch":"?$*","forge":true},
|
||||
{"rule":"","branch":"!/{béa","forge":false},
|
||||
{"rule":"","branch":"A1","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"!a$1/b{","branch":"!a$1/b{","forge":true},
|
||||
{"rule":":dww/|d","branch":":dww/|d","forge":true},
|
||||
{"rule":"!|*}","branch":"xéxé{b","forge":false},
|
||||
{"rule":"1é||","branch":"a+.","forge":false},
|
||||
{"rule":"/+],","branch":"!-\\[{é","forge":false},
|
||||
{"rule":"a}(","branch":"a}(","forge":true},
|
||||
{"rule":"|A?.","branch":"}.","forge":false},
|
||||
{"rule":"d$a^d","branch":"*{","forge":false},
|
||||
{"rule":"-ba$+A","branch":"-ba$+A","forge":true},
|
||||
{"rule":"-.|,?a!","branch":"-.|,?a!","forge":true},
|
||||
{"rule":"A","branch":"A","forge":true},
|
||||
{"rule":"w","branch":"w","forge":true},
|
||||
{"rule":".!1!+a","branch":".!1!+a","forge":true},
|
||||
{"rule":"+$w:}","branch":"+$w:}","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":")]","branch":")]","forge":true},
|
||||
{"rule":"a})^,}1","branch":"a})^,}1","forge":true},
|
||||
{"rule":":[:!:","branch":":[:!:","forge":true},
|
||||
{"rule":"w?w:]1!","branch":"\\.]-","forge":false},
|
||||
{"rule":"*$/","branch":"*$/","forge":true},
|
||||
{"rule":"é](","branch":"é](","forge":true},
|
||||
{"rule":"x$a/","branch":"x$a/","forge":true},
|
||||
{"rule":"*)b+[^","branch":"*)b+[^","forge":true},
|
||||
{"rule":"/]/?","branch":"/]/?","forge":true},
|
||||
{"rule":"{x,A","branch":"!Ab","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"-,","branch":"xé}A[.","forge":false},
|
||||
{"rule":"w)Ad","branch":"w)Ad","forge":true},
|
||||
{"rule":":??","branch":":??","forge":true},
|
||||
{"rule":"x/xé|","branch":"x/xé|","forge":true},
|
||||
{"rule":"","branch":"1!","forge":false},
|
||||
{"rule":"!![é[","branch":"x","forge":false},
|
||||
{"rule":"wb$","branch":"wb$","forge":true},
|
||||
{"rule":")","branch":")","forge":true},
|
||||
{"rule":"Aw:)(1","branch":"{b\\,","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"1*}?d","branch":"1*}?d","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"?-b.","branch":"?-b.","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"d","branch":",aé1","forge":false},
|
||||
{"rule":"+","branch":"+{","forge":false},
|
||||
{"rule":"]d","branch":"]}:/bA","forge":false},
|
||||
{"rule":"*?+:|","branch":"!é.","forge":false},
|
||||
{"rule":"}$d","branch":"}$d","forge":true},
|
||||
{"rule":"/","branch":"/","forge":true},
|
||||
{"rule":"a$+b)b","branch":"d}","forge":false},
|
||||
{"rule":"?$[/A","branch":"","forge":false},
|
||||
{"rule":"é){","branch":"é){","forge":true},
|
||||
{"rule":"$(d","branch":"{é}","forge":false},
|
||||
{"rule":"b(","branch":"b(","forge":true},
|
||||
{"rule":"^^+]","branch":"^^+]","forge":true},
|
||||
{"rule":",)","branch":",)","forge":true},
|
||||
{"rule":"d-.A-","branch":"d-.A-","forge":true},
|
||||
{"rule":"$","branch":"\\/","forge":false},
|
||||
{"rule":"}+[\\|\\$","branch":"x[!b","forge":false},
|
||||
{"rule":"(d1aé!é","branch":"A[-","forge":false},
|
||||
{"rule":"x","branch":"1","forge":false},
|
||||
{"rule":"x","branch":"dx","forge":false},
|
||||
{"rule":"]|)]","branch":"]|)]","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"[.*?","branch":"[.*?","forge":true},
|
||||
{"rule":"[?","branch":"[?","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"),/","branch":"a1,,1","forge":false},
|
||||
{"rule":"}+","branch":"}+","forge":true},
|
||||
{"rule":"$d\\,d","branch":"$d\\,d","forge":false},
|
||||
{"rule":"d1","branch":"d1","forge":true},
|
||||
{"rule":"-!^.:","branch":"-!^.:","forge":true},
|
||||
{"rule":"x","branch":"+].A11","forge":false},
|
||||
{"rule":"","branch":"-]","forge":false},
|
||||
{"rule":"}]}.","branch":"","forge":false},
|
||||
{"rule":"^{wx","branch":"]1a!","forge":false},
|
||||
{"rule":"^","branch":"*]x,","forge":false},
|
||||
{"rule":"(:x*x","branch":"","forge":false},
|
||||
{"rule":"|[/*a/","branch":"}","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"1\\^","branch":"!é!x{","forge":false},
|
||||
{"rule":"(.b,AA+","branch":"(.b,AA+","forge":true},
|
||||
{"rule":"xx","branch":"","forge":false},
|
||||
{"rule":"d)^):","branch":"/}!","forge":false},
|
||||
{"rule":"|","branch":",é*]*[","forge":false},
|
||||
{"rule":"d*d:*1","branch":"\\b,","forge":false},
|
||||
{"rule":":]","branch":"[","forge":false},
|
||||
{"rule":"([|!\\^é","branch":"([|!\\^é","forge":true},
|
||||
{"rule":"$[w)x-*","branch":"","forge":false},
|
||||
{"rule":")","branch":")","forge":true},
|
||||
{"rule":".d^)?","branch":"x\\é","forge":false},
|
||||
{"rule":"1|{w","branch":"1|{w","forge":true},
|
||||
{"rule":"é$,(]$$","branch":"é$,(]$$","forge":true},
|
||||
{"rule":"/","branch":"/","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"+","branch":"","forge":false},
|
||||
{"rule":"]","branch":"","forge":false},
|
||||
{"rule":"+:a:","branch":"+:a:","forge":true},
|
||||
{"rule":"/*","branch":"d.}","forge":false},
|
||||
{"rule":"|/ba1-","branch":"|/ba1-","forge":true},
|
||||
{"rule":"","branch":"+","forge":false},
|
||||
{"rule":"A","branch":"A","forge":true},
|
||||
{"rule":",-!]","branch":",-!]","forge":true},
|
||||
{"rule":"$*d!)","branch":"","forge":false},
|
||||
{"rule":")é+x.d","branch":")é+x.d","forge":true},
|
||||
{"rule":"éa$\\[1","branch":"1/,","forge":false},
|
||||
{"rule":"}?1|]d","branch":"}?1|]d","forge":true},
|
||||
{"rule":"[","branch":"[","forge":true},
|
||||
{"rule":"-1.*+","branch":"-1.*+","forge":true},
|
||||
{"rule":"Aw|{","branch":"Aw|{","forge":true},
|
||||
{"rule":"adAAAa","branch":"A","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"d/^{:","branch":"d/^{:","forge":true},
|
||||
{"rule":"é:(,^","branch":"é:(,^","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"!ébé({+","branch":"!ébé({+","forge":true},
|
||||
{"rule":"x","branch":"x","forge":true},
|
||||
{"rule":"|","branch":",:*","forge":false},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"x|","branch":"x|","forge":true},
|
||||
{"rule":"a","branch":"aaxéb,","forge":false},
|
||||
{"rule":"/w-),1é","branch":"/w-),1é","forge":true},
|
||||
{"rule":"|*)^,","branch":"|*)^,","forge":true},
|
||||
{"rule":"){é.!b.","branch":"a-}:","forge":false},
|
||||
{"rule":"(\\","branch":"(\\","forge":true},
|
||||
{"rule":".w(\\","branch":"+","forge":false},
|
||||
{"rule":"/d[","branch":"/d[","forge":true},
|
||||
{"rule":"(}$x!}","branch":"a,bb","forge":false},
|
||||
{"rule":"w{","branch":"w{","forge":true},
|
||||
{"rule":"[+a\\A(+","branch":"[+a\\A(+","forge":true},
|
||||
{"rule":"(1^{)bb","branch":"(1^{)bb","forge":true},
|
||||
{"rule":"^1","branch":"é-}","forge":false},
|
||||
{"rule":"*,+}1):","branch":"*,+}1):","forge":true},
|
||||
{"rule":"[A[(.x,","branch":"]db!*","forge":false},
|
||||
{"rule":"","branch":"Ad}","forge":false},
|
||||
{"rule":".","branch":".","forge":true},
|
||||
{"rule":"w!é{x","branch":"é.{","forge":false},
|
||||
{"rule":"$\\^","branch":"bx]+","forge":false},
|
||||
{"rule":"1)","branch":"+*d","forge":false},
|
||||
{"rule":"x)}/])","branch":"]","forge":false},
|
||||
{"rule":",a}.1}","branch":",a}.1}","forge":true},
|
||||
{"rule":"-a{(","branch":"-a{(","forge":true},
|
||||
{"rule":"1","branch":"1","forge":true},
|
||||
{"rule":"","branch":"","forge":true},
|
||||
{"rule":"}[é$","branch":"]d-!","forge":false},
|
||||
{"rule":"xé{é+?","branch":"xé{é+?","forge":true},
|
||||
{"rule":"/A1!","branch":"xAA\\::","forge":false},
|
||||
{"rule":"1][b-.(","branch":"{/-b![","forge":false},
|
||||
{"rule":"(d}(","branch":"(d}(","forge":true},
|
||||
{"rule":"{*}","branch":"!ébd","forge":true},
|
||||
{"rule":"()^^)","branch":"()^^)","forge":true},
|
||||
{"rule":"1(1A{$","branch":"*,[","forge":false},
|
||||
{"rule":"A:!^\\","branch":"A:!^\\","forge":true},
|
||||
{"rule":"-]:b]","branch":"-]:b]","forge":true},
|
||||
{"rule":",","branch":",","forge":true},
|
||||
{"rule":"[-:]","branch":"[-:]","forge":false},
|
||||
{"rule":"","branch":"é:b1:d","forge":false},
|
||||
{"rule":"1\\!]?d","branch":"","forge":false},
|
||||
{"rule":"","branch":"bb!]+","forge":false},
|
||||
{"rule":"/da","branch":"b]d[","forge":false},
|
||||
{"rule":",]","branch":",]","forge":true},
|
||||
{"rule":"[","branch":"[","forge":true},
|
||||
{"rule":"a]{A-","branch":"-+","forge":false},
|
||||
{"rule":"A?A^1","branch":"A?A^1","forge":true},
|
||||
{"rule":"\\-$","branch":"","forge":false},
|
||||
{"rule":"+","branch":"*[}[","forge":false},
|
||||
{"rule":",b?{dw","branch":",b?{dw","forge":true},
|
||||
{"rule":"d[$b1","branch":"/","forge":false},
|
||||
{"rule":"})|xAA","branch":"é,!","forge":false},
|
||||
{"rule":"]--","branch":"]--","forge":true},
|
||||
{"rule":"],/b","branch":"],/b","forge":true},
|
||||
{"rule":"/","branch":"/","forge":true},
|
||||
{"rule":"*aé","branch":"*aé","forge":true},
|
||||
{"rule":"{","branch":"{","forge":true},
|
||||
{"rule":":aé","branch":"","forge":false},
|
||||
{"rule":".\\}","branch":".\\}","forge":false},
|
||||
{"rule":"wA]*","branch":"\\]!d{+","forge":false},
|
||||
{"rule":"a/1x$/","branch":"[1!:/A","forge":false},
|
||||
{"rule":"[.wb}d.","branch":"-!*..a","forge":false},
|
||||
{"rule":".[{()a","branch":"+}","forge":false},
|
||||
{"rule":"]/!:,","branch":"{Aéa","forge":false},
|
||||
{"rule":"é{ba[-","branch":"],,1d/","forge":false},
|
||||
{"rule":"(w*","branch":"1}.!","forge":false},
|
||||
{"rule":"{?","branch":"1","forge":false},
|
||||
{"rule":")","branch":",+!","forge":false},
|
||||
{"rule":":","branch":"{/+A][","forge":false},
|
||||
{"rule":".1{)","branch":"a\\bA","forge":false},
|
||||
{"rule":"w-1+x1-","branch":"*]1}bb","forge":false},
|
||||
{"rule":"xb$$","branch":"éx:]-:","forge":false},
|
||||
{"rule":"++$!?*$","branch":"bA","forge":false},
|
||||
{"rule":"]{","branch":"[xd*x.","forge":false}
|
||||
]
|
||||
@@ -132,16 +132,8 @@ as text. `get` serves each, and any other field a record of hq wrote under `fiel
|
||||
- `partial`: both ends of a change agree;
|
||||
- `rests-on-live`: no accepted record extends a superseded one.
|
||||
|
||||
A failure is kept as a finding and told to the operator once, as a warning naming the rule and the records
|
||||
(`operator-channel.notify`, keyed `knowledge.finding.<rule>.<records>`, sent again at the next step when it
|
||||
could not be sent, and cleared once the rule holds for them again). It is a fault of the records, which their
|
||||
authors repair, not of the module: the module's health names it and stays healthy, so it never holds the
|
||||
module's own rollouts (the operator's decision of 2026-10-12 on hq issue 504). The acceptance stands.
|
||||
|
||||
`cited` is also judged ahead, in the review: a decision that names no home and that no accepted (or
|
||||
superseded) record links is not judged and not accepted. Its review, by the knowledge base itself, asks changes
|
||||
naming the rule (hq issue 504). It is not refused at propose, because no record can link a number only
|
||||
reserved: a record citing a new one is proposed after it and accepted, and the new one is proposed again.
|
||||
A failure is kept as a finding and makes the module unhealthy, so the mesh raises a condition naming the rule
|
||||
and the records. The acceptance stands.
|
||||
|
||||
## The import
|
||||
|
||||
@@ -189,14 +181,14 @@ the consolidated schema for a fresh database is `schema.sql`. A test holds the t
|
||||
|
||||
- **the bundle**: the verbs, `knowledge_health`, the operator's answers heard, and once a minute (and at once when
|
||||
a record is proposed) the router's record read, reservations never proposed withdrawn, the reviews of proposals,
|
||||
and the drafting worker; and it tells the whole-set findings. It never changes the schema, and
|
||||
and the drafting worker. It never changes the schema, and
|
||||
refuses to serve while the schema is behind.
|
||||
- **`prepare`** (run once): brings the schema to this version.
|
||||
- **`keep`** (judged by its unit): judges the whole record set every five minutes and keeps the findings.
|
||||
- **`watch`** (judged by `knowledge_health`, run as the machine's account, the one the runtime runs the bundle
|
||||
as, so the tool can read what it writes): every minute, reads whether the database serves and what the
|
||||
findings say, and leaves its verdict in `${dir:state}/health.json`. That is the file the health tool answers
|
||||
from. A verdict older than three minutes is unhealthy; a whole-set finding is named in it but leaves it healthy.
|
||||
from. A verdict older than three minutes is unhealthy.
|
||||
|
||||
## Settings
|
||||
|
||||
|
||||
@@ -514,73 +514,6 @@ func (k *KB) judgeSet(ctx context.Context) []Finding {
|
||||
return findings
|
||||
}
|
||||
|
||||
// findingPrefix keys the notice of one whole-set finding: the prefix, the rule, and the records it names.
|
||||
const findingPrefix = "knowledge.finding."
|
||||
|
||||
func findingKey(f Finding) string {
|
||||
names := make([]string, len(f.Records))
|
||||
for i, r := range f.Records {
|
||||
names[i] = strings.ReplaceAll(strings.ToLower(r), " ", "-")
|
||||
}
|
||||
return findingPrefix + f.Rule + "." + strings.Join(names, ".")
|
||||
}
|
||||
|
||||
// TellFindings, called by the background work and never under the write lock, tells the operator, once, each finding of the rules that judge the whole record set, as a warning
|
||||
// naming the rule and the records, and clears the notice once the rule holds for them again. A finding is a fault
|
||||
// of the records, which their authors repair; it is not the module failing, so it leaves the module's health as it
|
||||
// is and never holds the module's own rollouts (the operator's decision of 2026-10-12 on novox/hq issue 504, as
|
||||
// issue 508 decided for a proposal never reviewed). A notice that cannot be sent is not kept as told, and is sent
|
||||
// again at the next step.
|
||||
func (k *KB) TellFindings() {
|
||||
if k.Notify == nil || k.Clear == nil {
|
||||
return
|
||||
}
|
||||
ctx := context.Background()
|
||||
findings, err := k.Store.Findings(ctx)
|
||||
if err != nil {
|
||||
k.logf("[mesh-knowledge] the whole-set findings cannot be read to tell them: %v", err)
|
||||
return
|
||||
}
|
||||
now := map[string]bool{}
|
||||
for _, f := range findings {
|
||||
key := findingKey(f)
|
||||
now[key] = true
|
||||
first, err := k.Store.Told(ctx, key)
|
||||
if err != nil || !first {
|
||||
continue
|
||||
}
|
||||
records := strings.Join(f.Records, ", ")
|
||||
summary := fmt.Sprintf("The rule %s, which judges the whole record set, fails: %s.", f.Rule, f.Says)
|
||||
explanation := "The knowledge base works; its records break a rule. The acceptance stands. The author of " + records +
|
||||
" repairs it: a decision cited by nothing names its home in a progressive insight (propose with homes and insight), " +
|
||||
"or a record that cites it is accepted. get and links show the record."
|
||||
resolved := fmt.Sprintf("The rule %s holds again for %s", f.Rule, records)
|
||||
if err := k.Notify(key, summary, "A rule of the record set fails: "+records, explanation, resolved); err != nil {
|
||||
k.logf("[mesh-knowledge] the operator could not be told that %s fails for %s: %v; told again at the next step", f.Rule, records, err)
|
||||
if uerr := k.Store.Untold(ctx, key); uerr != nil {
|
||||
k.logf("[mesh-knowledge] and that it was not told could not be kept: %v", uerr)
|
||||
}
|
||||
}
|
||||
}
|
||||
told, err := k.Store.ToldOf(ctx, findingPrefix)
|
||||
if err != nil {
|
||||
k.logf("[mesh-knowledge] the notices of whole-set findings cannot be read: %v", err)
|
||||
return
|
||||
}
|
||||
for _, key := range told {
|
||||
if now[key] {
|
||||
continue
|
||||
}
|
||||
if err := k.Clear(key, ""); err != nil {
|
||||
k.logf("[mesh-knowledge] the notice %s could not be cleared: %v; it is cleared again at the next step", key, err)
|
||||
continue
|
||||
}
|
||||
if err := k.Store.Untold(ctx, key); err != nil {
|
||||
k.logf("[mesh-knowledge] the notice %s was cleared, and that could not be kept: %v", key, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- reservations that were never proposed -------------------------------------------------------------------
|
||||
|
||||
// Withdraw withdraws every number reserved longer ago than the settings allow and never proposed: it stays spent,
|
||||
|
||||
@@ -40,9 +40,6 @@ func TestMain(m *testing.M) {
|
||||
serve(k, os.Getenv("MESH_KNOWLEDGE_STATUS"))
|
||||
os.Exit(0)
|
||||
}
|
||||
// Every test runs in a zone that is not UTC, so a time said locally is proven to be, wherever the tests run. It
|
||||
// is set once, before any goroutine starts: a test that set it while a bus test's server read it raced.
|
||||
time.Local = testZone
|
||||
code := m.Run()
|
||||
stopPostgres()
|
||||
os.Exit(code)
|
||||
@@ -439,13 +436,9 @@ func valid(n int) ProposeAsk {
|
||||
{Name: "Consequences", Text: "Merges land sooner."},
|
||||
{Name: "How it is checked", Text: "The controller's merge-window tests."},
|
||||
},
|
||||
Links: []LinkAsk{{Kind: LinkExtends, To: "0276"}, {Kind: LinkChanges, To: "0276 §1", From: 1}},
|
||||
Homes: []string{testHome}}
|
||||
Links: []LinkAsk{{Kind: LinkExtends, To: "0276"}, {Kind: LinkChanges, To: "0276 §1", From: 1}}}
|
||||
}
|
||||
|
||||
// testHome is the hq design valid names as its home: a decision is accepted only cited (novox/hq issue 504).
|
||||
const testHome = "03-DESIGN/01-to-be/55-the-knowledge-base.md"
|
||||
|
||||
// imported is a world whose knowledge base holds the fixture's records.
|
||||
func imported(t *testing.T, s Store) *world {
|
||||
t.Helper()
|
||||
|
||||
@@ -164,8 +164,6 @@ func background(k *KB) {
|
||||
logf("[mesh-knowledge] withdrew %d reservation(s) never proposed", n)
|
||||
}
|
||||
review()
|
||||
// The keeper judges the record set in a process of its own, which tells nobody: its findings are told here.
|
||||
k.TellFindings()
|
||||
for {
|
||||
said, err := k.DraftStep()
|
||||
if err != nil {
|
||||
@@ -554,11 +552,10 @@ type HealthAnswer struct {
|
||||
At time.Time `json:"at,omitempty"`
|
||||
}
|
||||
|
||||
// Health is whether the knowledge base works: unhealthy when its database cannot serve. A rule that judges the whole
|
||||
// record set and fails is a fault of the records, which their authors repair, not of the module: it is told to the
|
||||
// operator as a warning naming the rule and the records (TellFindings), and named here, but leaves the module
|
||||
// healthy (the operator's decision of 2026-10-12 on novox/hq issue 504). So does a proposal that ran out of review
|
||||
// tries (novox/hq issue 508). Neither holds the module's own rollouts.
|
||||
// Health is whether the knowledge base works: unhealthy when its database cannot serve, or when a rule that judges
|
||||
// the whole record set fails — the mesh raises a condition naming the rule and the records. A proposal that ran out
|
||||
// of review tries waits on its author, not on the module: it is told to its author and the operator, and named
|
||||
// here, but leaves the module healthy, so it never holds the module's own rollouts (novox/hq issue 508).
|
||||
func (k *KB) Health(ready Healther) HealthAnswer {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 8*time.Second)
|
||||
defer cancel()
|
||||
@@ -589,14 +586,17 @@ func (k *KB) Health(ready Healther) HealthAnswer {
|
||||
waiting = fmt.Sprintf("%d proposed record(s) could not be reviewed in %d tries and wait for their authors to propose them again: %s",
|
||||
n, MostReviewTries, strings.Join(unreviewed, "; "))
|
||||
}
|
||||
why := "the database serves and every rule that judges the whole record set holds"
|
||||
if len(says) > 0 {
|
||||
if len(says) > 8 {
|
||||
says = append(says[:8], fmt.Sprintf("and %d more", len(says)-8))
|
||||
}
|
||||
why = fmt.Sprintf("the database serves; %d rule(s) that judge the whole record set fail, told to the operator as a warning "+
|
||||
"for the records' authors to repair: %s", len(findings), strings.Join(says, "; "))
|
||||
why := fmt.Sprintf("%d rule(s) that judge the whole record set fail: %s", len(findings), strings.Join(says, "; "))
|
||||
if waiting != "" {
|
||||
why += ". " + waiting
|
||||
}
|
||||
return HealthAnswer{State: "unhealthy", Why: why}
|
||||
}
|
||||
why := "the database serves and every rule that judges the whole record set holds"
|
||||
if waiting != "" {
|
||||
why += "; " + waiting
|
||||
}
|
||||
@@ -605,11 +605,10 @@ func (k *KB) Health(ready Healther) HealthAnswer {
|
||||
|
||||
func healthTool(status string, now func() time.Time) stdio.Tool {
|
||||
return stdio.Tool{Name: HealthTool,
|
||||
Description: "How the knowledge base is, as its watch last saw it: healthy when its database serves this version, unhealthy " +
|
||||
"otherwise. A rule that judges the whole record set and fails — a decision accepted that nothing cites, a supersession " +
|
||||
"or a change said on one end only, a record extending a superseded one — is named but leaves it healthy: it is told " +
|
||||
"to the operator as a warning for the records' authors (novox/hq issue 504). So is a proposed record that ran out of " +
|
||||
"review tries: it waits on its author (novox/hq issue 508).",
|
||||
Description: "How the knowledge base is, as its watch last saw it: healthy when its database serves this version and every " +
|
||||
"rule that judges the whole record set holds; unhealthy otherwise, naming the rule and the records — a decision accepted " +
|
||||
"that nothing cites, a supersession or a change said on one end only, a record extending a superseded one. A " +
|
||||
"proposed record that ran out of review tries is named but leaves it healthy: it waits on its author (novox/hq issue 508).",
|
||||
Run: func(map[string]any) (any, error) { return ReadStatus(status, now()), nil }}
|
||||
}
|
||||
|
||||
@@ -919,8 +918,7 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
|
||||
"link to nothing, a clause without a bold lead, a superseded record cited alone, an unknown topic, a retired word. A " +
|
||||
"proposal is revised by proposing it again; an accepted record changes only by a progressive insight (insight: what " +
|
||||
"stood before). The knowledge base then reviews each version itself, in a judgement that sees only the record, the " +
|
||||
"records it links and the rules. A decision that names no home and that no accepted record links is not judged: " +
|
||||
"its review asks changes for the whole-set rule cited, which its acceptance would break. On approve it is accepted (a record that needs the operator waits for the operator's " +
|
||||
"records it links and the rules: on approve it is accepted (a record that needs the operator waits for the operator's " +
|
||||
"Approve), on changes it stays proposed with the findings for its author to revise, and when no judgement can be had " +
|
||||
"it stays proposed and says so. Nothing proposed is in force.",
|
||||
Input: schema([]string{"number", "title"}, map[string]any{
|
||||
@@ -935,7 +933,7 @@ func Tools(k *KB, caller func(map[string]any) string) []stdio.Tool {
|
||||
"links": map[string]any{"type": "array", "description": "its links: kind, to (\"0276\", \"0276 §4\", \"research 045\", \"issue 365\", a design's path), from (the clause it goes out from)",
|
||||
"items": map[string]any{"type": "object", "properties": map[string]any{"kind": map[string]any{"type": "string", "enum": LinkKinds},
|
||||
"to": text("the target"), "from": integer("the clause of this record it goes out from")}, "required": []string{"kind", "to"}}},
|
||||
"homes": texts("hq documents that cite it, such as a design's path; a decision needs one unless an accepted record links it"),
|
||||
"homes": texts("hq documents that cite it, such as a design's path"),
|
||||
"needs-operator": flag("it holds only once the operator approves it"),
|
||||
"status": map[string]any{"type": "string", "enum": ResearchStatuses, "description": "a research effort's status"},
|
||||
"insight": text("for an accepted record: a progressive insight, saying what stood before"),
|
||||
|
||||
@@ -408,10 +408,7 @@ func (k *KB) reviewOne(key Key) (string, error) {
|
||||
k.mu.Lock()
|
||||
e, err := k.Store.Get(ctx, key)
|
||||
var linked []*Record
|
||||
var all []*Entry
|
||||
var allErr error
|
||||
if err == nil && e != nil {
|
||||
all, allErr = k.Store.All(ctx)
|
||||
seen := map[Key]bool{key: true}
|
||||
for _, l := range e.Working().Links {
|
||||
to, ok := l.To.Record()
|
||||
@@ -433,20 +430,9 @@ func (k *KB) reviewOne(key Key) (string, error) {
|
||||
var verdict string
|
||||
var findings []string
|
||||
by := ReviewBy
|
||||
uncited := ""
|
||||
if allErr == nil {
|
||||
uncited = UncitedOnAcceptance(w, all)
|
||||
}
|
||||
switch {
|
||||
case allErr != nil:
|
||||
err = fmt.Errorf("the record set cannot be read to judge whether a record cites it: %v", allErr)
|
||||
case uncited != "":
|
||||
// The whole-set rule cited, judged ahead: changes, asked by the knowledge base itself without a judgement,
|
||||
// so its acceptance never turns the module's health red for it (novox/hq issue 504).
|
||||
verdict, findings = VerdictChanges, []string{"the rule cited, which judges the whole record set: " + uncited}
|
||||
case len(question) > MostReviewQuestion:
|
||||
if len(question) > MostReviewQuestion {
|
||||
err = fmt.Errorf("the record is %d bytes as a question, and a judgement takes at most %d: shorten it", len(question), MostReviewQuestion)
|
||||
default:
|
||||
} else {
|
||||
var answer json.RawMessage
|
||||
var judge string
|
||||
answer, judge, err = k.Judge(question, reviewSchema, k.judgeModel())
|
||||
|
||||
@@ -155,16 +155,13 @@ func TestProposedAgainAfterTheLastTryClearsTheNotice(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// testZone is the local zone every test runs in, set once by TestMain: not UTC.
|
||||
var testZone = time.FixedZone("CEST", 2*60*60)
|
||||
|
||||
// inZone says the test needs the local zone not to be UTC; TestMain set it before any goroutine started. Setting it
|
||||
// here, under a running bus server's goroutines, was a data race the race detector failed.
|
||||
// inZone sets the local zone to one that is not UTC for one test, so a time said locally is proven to be, wherever
|
||||
// the test runs.
|
||||
func inZone(t *testing.T) {
|
||||
t.Helper()
|
||||
if time.Local != testZone {
|
||||
t.Fatal("the local zone is not the tests' own: TestMain sets it")
|
||||
}
|
||||
was := time.Local
|
||||
time.Local = time.FixedZone("CEST", 2*60*60)
|
||||
t.Cleanup(func() { time.Local = was })
|
||||
}
|
||||
|
||||
// novox/hq issue 497: a clear the operator's channel refused keeps its notice as told, and is sent again at the next
|
||||
|
||||
@@ -546,36 +546,6 @@ func JudgeSet(records []*Record) []Finding {
|
||||
return out
|
||||
}
|
||||
|
||||
// UncitedOnAcceptance is the whole-set rule `cited` judged ahead, on a decision's working version as it would
|
||||
// stand once accepted, over the records as JudgeSet reads them: "" when it names a home or a record accepted (or
|
||||
// superseded) links it, else what the rule would say. hq held this rule at the merge of the record's own pull
|
||||
// request, which carried the design citing it; the knowledge base holds it in the review, before the judgement, so
|
||||
// an acceptance never leaves a decision cited by nothing (novox/hq issue 504). It is not refused at propose: no
|
||||
// record can link a number only reserved, so a record that will cite a new one is proposed after it, accepted, and
|
||||
// the new one proposed again. A citing record changed by a correction after the acceptance is what the rule, run
|
||||
// after every acceptance, still says. An imported record was held to hq's checks, and its correction is not judged
|
||||
// here, as JudgeSet does not judge it.
|
||||
func UncitedOnAcceptance(r *Record, all []*Entry) string {
|
||||
if r.Kind != KindDecision || len(r.Homes) > 0 || (r.Pending != nil && r.AcceptedAt.IsZero()) {
|
||||
return ""
|
||||
}
|
||||
k := r.Key()
|
||||
for _, e := range all {
|
||||
from := e.Current
|
||||
if from.Key() == k || !(from.Accepted() || from.Status == StatusSuperseded) {
|
||||
continue
|
||||
}
|
||||
for _, l := range from.Links {
|
||||
if to, ok := l.To.Record(); ok && to == k {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
}
|
||||
return fmt.Sprintf("%s would be accepted with nothing citing it: it names no home and no accepted record links it. Name "+
|
||||
"its home (homes: the hq design or playbook that cites it), or have the record that links it accepted first; then "+
|
||||
"propose this one again", k)
|
||||
}
|
||||
|
||||
// RuleFrom is the day each dated rule began to hold a record, as hq's checks hold it: a record dated earlier was
|
||||
// never held to it, and a dated correction of it (a progressive insight, which keeps the record's date) is not.
|
||||
// words.py held the retired words from 2026-10-07 and the daemons' lines from 2026-10-09; cycle.py the deferrals
|
||||
|
||||
@@ -1,119 +1,38 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A decision nothing would cite is not accepted: its review asks changes itself, without a judgement, naming the
|
||||
// rule cited, and the module stays healthy (novox/hq issue 504). Before, the review approved it, the acceptance
|
||||
// broke the rule, and the module turned unhealthy for a record its author could only repair after the fact.
|
||||
func TestADecisionNothingWouldCiteIsNotAccepted(t *testing.T) {
|
||||
// to-be 55 §10: "A whole-set rule that fails is loud — the module's test: an acceptance that leaves a decision
|
||||
// cited by nothing raises the condition." The condition is the mesh's, raised when the module's health answers
|
||||
// unhealthy; the test reads the health answer the node-engine reads.
|
||||
func TestAnAcceptanceThatLeavesADecisionCitedByNothingMakesTheModuleUnhealthy(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s Store) {
|
||||
w := imported(t, s)
|
||||
if h := w.k.Health(nil); !h.Healthy {
|
||||
t.Fatalf("unhealthy after the import of real records: %s", h.Why)
|
||||
}
|
||||
n := w.reserved(t, "author@g14")
|
||||
p := valid(n)
|
||||
p.Homes = nil
|
||||
if _, err := w.k.Propose("author@g14", p); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
judged := len(w.judge.reviews)
|
||||
w.reviewAll(t)
|
||||
e := mustGet(t, s, Key{KindDecision, n})
|
||||
if e.Current.Status != StatusProposed {
|
||||
t.Fatalf("a decision nothing cites was %s by its review", e.Current.Status)
|
||||
}
|
||||
rv := e.Current.Reviews
|
||||
if len(rv) == 0 || rv[len(rv)-1].Verdict != VerdictChanges || !strings.Contains(rv[len(rv)-1].Findings, "the rule cited") ||
|
||||
rv[len(rv)-1].By != ReviewBy {
|
||||
t.Fatalf("its review asked no changes for the rule cited: %+v", rv)
|
||||
}
|
||||
if len(w.judge.reviews) != judged {
|
||||
t.Fatal("a judgement was spent on a record the rule cited refuses")
|
||||
}
|
||||
if h := w.k.Health(nil); !h.Healthy {
|
||||
t.Fatalf("unhealthy for a record not accepted: %s", h.Why)
|
||||
}
|
||||
// Named with its home, it is accepted, and the module stays healthy.
|
||||
p.Homes = []string{testHome}
|
||||
w.accepted(t, "author@g14", p)
|
||||
if h := w.k.Health(nil); !h.Healthy {
|
||||
t.Fatalf("unhealthy after a decision with its home was accepted: %s", h.Why)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// A new decision is cited by a record proposed after it: no record can link a number only reserved. Once the
|
||||
// citing record is accepted, the new one proposed again is accepted. A correction of the citing record that drops
|
||||
// the link still leaves the decision cited by nothing, and that is loud: told to the operator once, as a warning
|
||||
// naming the rule and the record, while the module stays healthy and its rollouts are not held (the operator's
|
||||
// decision of 2026-10-12 on novox/hq issue 504; to-be 55 §10). A record that cites it clears the notice.
|
||||
func TestARecordAcceptedCitingANewDecisionLetsItBeAcceptedAndDroppingTheLinkIsLoud(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s Store) {
|
||||
w := imported(t, s)
|
||||
n := w.reserved(t, "author@g14")
|
||||
a := valid(n)
|
||||
a.Homes = nil
|
||||
if _, err := w.k.Propose("author@g14", a); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
w.reviewAll(t)
|
||||
m := w.reserved(t, "author@g14")
|
||||
b := valid(m)
|
||||
b.Links = append(b.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
||||
w.accepted(t, "author@g14", b)
|
||||
w.accepted(t, "author@g14", a) // its revision: an accepted record links it now
|
||||
if h := w.k.Health(nil); !h.Healthy {
|
||||
t.Fatalf("unhealthy with both accepted, one citing the other: %s", h.Why)
|
||||
}
|
||||
w.k.TellFindings()
|
||||
if len(w.told) != 0 {
|
||||
t.Fatalf("told with every rule holding: %v", w.told)
|
||||
}
|
||||
// A correction of the citing record drops its link: the rule, run after the acceptance, says so.
|
||||
b.Links = b.Links[:len(b.Links)-1]
|
||||
b.Insight = "it cited " + itoaKey(n)
|
||||
w.accepted(t, "author@g14", b)
|
||||
w.accepted(t, "author@g14", valid(n))
|
||||
h := w.k.Health(nil)
|
||||
if !h.Healthy || !strings.Contains(h.Why, "the rule cited") || !strings.Contains(h.Why, itoaKey(n)) {
|
||||
t.Fatalf("a decision left cited by nothing: the health should stay healthy and name it, not %+v", h)
|
||||
if h.Healthy || h.State != "unhealthy" || !strings.Contains(h.Why, "the rule cited") || !strings.Contains(h.Why, itoaKey(n)) {
|
||||
t.Fatalf("an accepted decision nothing cites left the module %+v", h)
|
||||
}
|
||||
w.k.TellFindings()
|
||||
w.k.TellFindings()
|
||||
if len(w.told) != 1 || !strings.Contains(w.told[0], "rule cited") || !strings.Contains(w.told[0], itoaKey(n)) ||
|
||||
!strings.HasPrefix(w.toldKeys[0], findingPrefix) {
|
||||
t.Fatalf("the operator is not told once, naming the rule and the record: %v %v", w.told, w.toldKeys)
|
||||
}
|
||||
// A restart tells nothing again: what was told is kept in the store.
|
||||
told := w.toldKeys[0]
|
||||
restarted := newWorld(t, s, fixture())
|
||||
restarted.clock = w.clock
|
||||
restarted.k.TellFindings()
|
||||
if len(restarted.told) != 0 {
|
||||
t.Fatalf("told again after a restart: %v", restarted.told)
|
||||
}
|
||||
w.told = nil
|
||||
// The acceptance stands: the condition says it, and undoes nothing.
|
||||
if mustGet(t, s, Key{KindDecision, n}).Current.Status != StatusAccepted {
|
||||
t.Fatal("the acceptance was undone")
|
||||
}
|
||||
// A record that cites it heals it.
|
||||
k := w.reserved(t, "author@g14")
|
||||
c := valid(k)
|
||||
c.Links = append(c.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
||||
w.accepted(t, "author@g14", c)
|
||||
if h := w.k.Health(nil); strings.Contains(h.Why, itoaKey(n)+" is accepted and nothing cites it") {
|
||||
m := w.reserved(t, "author@g14")
|
||||
p := valid(m)
|
||||
p.Links = append(p.Links, LinkAsk{Kind: LinkCites, To: itoa4(n)})
|
||||
w.accepted(t, "author@g14", p)
|
||||
h = w.k.Health(nil)
|
||||
if strings.Contains(h.Why, itoaKey(n)+" is accepted and nothing cites it") {
|
||||
t.Fatalf("still said after a record cited it: %s", h.Why)
|
||||
}
|
||||
w.k.TellFindings()
|
||||
if len(w.cleared) != 1 || w.cleared[0][0] != told {
|
||||
t.Fatalf("the notice is not cleared once the rule holds again: %v", w.cleared)
|
||||
}
|
||||
w.k.TellFindings()
|
||||
if len(w.cleared) != 1 || len(w.told) != 0 {
|
||||
t.Fatalf("after the clear: cleared %v, told %v", w.cleared, w.told)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -164,29 +83,3 @@ func TestEachWholeSetRuleFails(t *testing.T) {
|
||||
t.Errorf("a consistent set fails: %+v", f)
|
||||
}
|
||||
}
|
||||
|
||||
// A finding's notice the operator's channel refused is not kept as told: it is sent again at the next step.
|
||||
func TestAFindingsNoticeThatFailsIsSentAgain(t *testing.T) {
|
||||
eachStore(t, func(t *testing.T, s Store) {
|
||||
w := newWorld(t, s, fixture())
|
||||
f := Finding{Rule: "cited", Records: []string{"ADR 0300"}, Says: "ADR 0300 is accepted and nothing cites it"}
|
||||
if err := s.SetFindings(context.Background(), []Finding{f}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tries := 0
|
||||
w.k.Notify = func(key, summary, headline, explanation, resolved string) error {
|
||||
tries++
|
||||
if tries == 1 {
|
||||
return errors.New("nothing answers operator-channel.notify")
|
||||
}
|
||||
w.told = append(w.told, summary)
|
||||
return nil
|
||||
}
|
||||
w.k.TellFindings()
|
||||
w.k.TellFindings()
|
||||
w.k.TellFindings()
|
||||
if tries != 2 || len(w.told) != 1 {
|
||||
t.Fatalf("a refused notice: %d tries, told %v", tries, w.told)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,52 +0,0 @@
|
||||
# mesh-reads
|
||||
|
||||
A node's files and accounts, read through the mesh (novox/hq ADR 0310, extending ADR 0268 §5; hq to-be 41
|
||||
WP8; issue 418). It holds the node seat `node-reads` on every node, the control-node included, so reading a
|
||||
file, searching one, listing a directory, the owner and mode of each component of a path and looking up an
|
||||
account or a group never go through the login shell's `execute`, and an approval is asked for an act, never
|
||||
for a read.
|
||||
|
||||
## Who it reads as
|
||||
|
||||
The account `mesh-reads`, which this module makes on every node (`runs-as`, ADR 0259 §8): no login shell, a
|
||||
home of its own under `/var/lib`, and no group beside its own. The tool runner serves it in a runtime of its
|
||||
own, on its own bus credential, never the machine's runtime, which runs as the operator's account. So the
|
||||
node's file modes bound what it sees: a file only the operator, root or a module's own account may read is
|
||||
refused by its mode, and the answer says so. It refuses every read when it finds itself running as root.
|
||||
|
||||
## What it refuses by name
|
||||
|
||||
Before any read, whatever the file modes would allow (ADR 0310 §5):
|
||||
|
||||
- **The places the mesh keeps a secret on this node**, which the controller composes from the manifests
|
||||
and gives it as `MESH_SECRET_PLACES`: the directory of every module's own secret (its bus credential
|
||||
among them), every credential file and grant directory, the vault's directories, and where the
|
||||
operator-sealed secrets are kept. Without that list the module does not start.
|
||||
- **The known credential files of a machine's programs** (`refuse.go`): an account's `.ssh`, `.gnupg`,
|
||||
`.aws`, `.kube`; the agent's `.credentials.json`, `.netrc`, `.pgpass`, `.git-credentials`; private key
|
||||
names (`id_ed25519`, `*.key`, `ssh_host_*_key`); `/etc/shadow`; `/etc/ssl/private`, `/etc/wireguard`
|
||||
and the network manager's connections.
|
||||
|
||||
A link is judged by where it leads as well as by its name. `stat` and `readlink` say a secret place's owner
|
||||
and mode, and mark it, because those are not its content; `list`, `read` and `search` refuse it.
|
||||
|
||||
## The content rule
|
||||
|
||||
Every answer passes the content rule's secret class, from `mesh-sdk`'s `outward` package (ADR 0310 §4): a
|
||||
line of a file or a matching line shaped like a secret is withheld and its number said, as is a link target
|
||||
or an account's words. A path and an entry's name are not judged: they are what the caller named or what the
|
||||
file system calls a thing, and the rule would take a temporary directory's or a container layer's name for a
|
||||
token.
|
||||
|
||||
## Its verbs and their bounds
|
||||
|
||||
| verb | replaces | bounds |
|
||||
|---|---|---|
|
||||
| `stat` | `stat`, `ls -ld`, `namei -l`, `findmnt -T` | 200 paths, 64 components each; links followed only with `follow` |
|
||||
| `readlink` | `readlink`, `readlink -f`, `realpath` | one path |
|
||||
| `list` | `ls`, `ls -la` | 2000 entries, cut and said |
|
||||
| `read` | `cat`, `head`, `tail`, `less` | 64 KiB, or up to 256 KiB when asked; cut at a line and said; text only |
|
||||
| `search` | `grep`, `grep -r`, `grep -F` | a fixed string; 100 matches (500 asked), 4 deep (8 asked), 5000 files, 4 MiB a file, 20 seconds; the bound met is said |
|
||||
| `account` | `getent passwd`, `getent group`, `id`, `groups` | 2000 accounts and groups; never a password field |
|
||||
|
||||
No answer is larger than 1 MiB. Nothing here writes a file.
|
||||
@@ -1,204 +0,0 @@
|
||||
// mesh-reads (novox/hq ADR 0310, to-be 41 WP8): the holder of node-reads. A node's files and accounts read
|
||||
// through the mesh — the owner and mode of each component of a path, a link, a directory, a text file, a search
|
||||
// by fixed string, an account or a group — so these reads never go through the login shell's execute.
|
||||
//
|
||||
// It runs as an account of its own, mesh-reads, served by the tool runner in a runtime of its own (ADR 0259
|
||||
// §8): in no group that holds a secret, with no login shell and no home of the operator's, so the node's file
|
||||
// modes bound what it sees. On top, it refuses by name the places a secret is kept (refuse.go) and withholds
|
||||
// any piece of an answer shaped like a secret (mesh-sdk's outward). Read-only: nothing here writes a file.
|
||||
//
|
||||
// stdout is the MCP channel; what this module says, it says on stderr.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// Seat is the role this module holds.
|
||||
const Seat = "node-reads"
|
||||
|
||||
// PlacesWord is where the controller names this node's secret places (novox/hq ADR 0310 §5).
|
||||
const PlacesWord = "MESH_SECRET_PLACES"
|
||||
|
||||
func main() {
|
||||
places, err := placesFrom(os.Getenv(PlacesWord))
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "[mesh-reads] %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := stdio.Serve("", tools(newReader(places))); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "[mesh-reads] %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// placesFrom reads the controller's list. Absent is refused: without it, a module's own-secret directory
|
||||
// would be read wherever the account's modes allow, and a refusal the decision promises would be silent.
|
||||
func placesFrom(raw string) ([]string, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return nil, fmt.Errorf("%s is not given, so this node's secret places are unknown and nothing would be "+
|
||||
"refused by them (novox/hq ADR 0310 §5): a controller that knows node-reads composes it", PlacesWord)
|
||||
}
|
||||
var places []string
|
||||
if err := json.Unmarshal([]byte(raw), &places); err != nil {
|
||||
return nil, fmt.Errorf("%s is not a list of paths: %v", PlacesWord, err)
|
||||
}
|
||||
return places, nil
|
||||
}
|
||||
|
||||
func str(description string) map[string]any {
|
||||
return map[string]any{"type": "string", "description": description}
|
||||
}
|
||||
|
||||
func arg(a map[string]any, k string) string {
|
||||
switch v := a[k].(type) {
|
||||
case string:
|
||||
return strings.TrimSpace(v)
|
||||
case float64:
|
||||
return strconv.FormatFloat(v, 'f', -1, 64)
|
||||
case bool:
|
||||
return strconv.FormatBool(v)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func number(a map[string]any, k string) (int, error) {
|
||||
s := arg(a, k)
|
||||
if s == "" {
|
||||
return 0, nil
|
||||
}
|
||||
n, err := strconv.Atoi(s)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("%s is a whole number, and %q was given", k, s)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// pathsArg is the paths given, one per line — or, from a caller that sends a list, the list.
|
||||
func pathsArg(a map[string]any) []string {
|
||||
var raw []string
|
||||
switch v := a["paths"].(type) {
|
||||
case string:
|
||||
raw = strings.Split(v, "\n")
|
||||
case []any:
|
||||
for _, p := range v {
|
||||
if s, ok := p.(string); ok {
|
||||
raw = append(raw, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
var out []string
|
||||
for _, p := range raw {
|
||||
if p = strings.TrimSpace(p); p != "" {
|
||||
out = append(out, p)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// bounded is the last bound (novox/hq ADR 0310): whatever a verb made, no answer is larger than MaxAnswer.
|
||||
func bounded(answer any, err error) (any, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body, err := json.Marshal(answer)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(body) > MaxAnswer {
|
||||
return nil, fmt.Errorf("the answer is %d bytes, and an answer of node-reads is at most %d: ask for less", len(body), MaxAnswer)
|
||||
}
|
||||
return answer, nil
|
||||
}
|
||||
|
||||
// verb is one of the seat's verbs: listed as `<seat>.<verb>`, so the runtime serves it on the seat's subject.
|
||||
// The descriptions are the controller's (internal/catalogue/reads.go), said again here for a caller of the
|
||||
// module's own address.
|
||||
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
|
||||
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input,
|
||||
Run: func(a map[string]any) (any, error) { return bounded(run(a)) }}
|
||||
}
|
||||
|
||||
func object(props map[string]any, required ...string) map[string]any {
|
||||
if required == nil {
|
||||
required = []string{}
|
||||
}
|
||||
return map[string]any{"type": "object", "properties": props, "required": required}
|
||||
}
|
||||
|
||||
func tools(r Reader) []stdio.Tool {
|
||||
return []stdio.Tool{
|
||||
verb("stat", "Each component of one path or several, from / down: its type, owner and group (name and "+
|
||||
"number), mode, whether it is a link and to what, and whether it is a mount point and of which file "+
|
||||
"system. A link is not followed unless follow is \"true\". Read as the account mesh-reads, which holds no "+
|
||||
"secret: a component it may not reach says so. At most 200 paths in one call.",
|
||||
object(map[string]any{"paths": str("one absolute path, or several separated by line breaks"),
|
||||
"follow": str("\"true\": also say where the path leads once every link is followed")}, "paths"),
|
||||
func(a map[string]any) (any, error) { return r.Stat(pathsArg(a), arg(a, "follow") == "true") }),
|
||||
verb("readlink", "What one link points to, as written, and where the path leads once every link is "+
|
||||
"followed, and whether that exists.",
|
||||
object(map[string]any{"path": str("an absolute path")}, "path"),
|
||||
func(a map[string]any) (any, error) { return r.Readlink(arg(a, "path")) }),
|
||||
verb("list", "Every entry of one directory with its type, owner, group, mode, size and, for a link, its "+
|
||||
"target. At most 2000 entries; a longer directory is cut and says so. A place a secret is kept is "+
|
||||
"refused by name.",
|
||||
object(map[string]any{"path": str("an absolute path to a directory")}, "path"),
|
||||
func(a map[string]any) (any, error) { return r.List(arg(a, "path")) }),
|
||||
verb("read", "One text file, or a part of it by line: at most 256 KiB in one answer (64 KiB unless more "+
|
||||
"is asked), cut at a line and said so. A negative from counts from the end, as tail does. A binary file "+
|
||||
"is refused; a place a secret is kept, or a known credential file, is refused by name; a line shaped "+
|
||||
"like a secret is withheld and its number said. /proc and /sys are read as far as mesh-reads may.",
|
||||
object(map[string]any{"path": str("an absolute path to a file"),
|
||||
"from": str("the first line to answer, counted from 1; negative: that many lines from the end (optional)"),
|
||||
"lines": str("how many lines to answer (optional)"),
|
||||
"bytes": str("the most bytes to answer, up to 262144 (optional, 65536 unless given)")}, "path"),
|
||||
func(a map[string]any) (any, error) {
|
||||
var ask ReadAsk
|
||||
var err error
|
||||
if ask.From, err = number(a, "from"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ask.Lines, err = number(a, "lines"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ask.Bytes, err = number(a, "bytes"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return r.Read(arg(a, "path"), ask)
|
||||
}),
|
||||
verb("search", "Find a fixed string (never a pattern) in one file or under one directory: every matching "+
|
||||
"line with its file and number. Bounded: at most 500 matches (100 unless more is asked), 8 directories "+
|
||||
"deep (4 unless more is asked), 5000 files, 4 MiB per file and 20 seconds; links are not followed and "+
|
||||
"binary files are skipped. A place a secret is kept is skipped and named; a matching line shaped like "+
|
||||
"a secret is withheld. The answer says which bound it met.",
|
||||
object(map[string]any{"text": str("the fixed string to find"),
|
||||
"path": str("an absolute path to a file or a directory"),
|
||||
"depth": str("how many directories deep, up to 8 (optional)"),
|
||||
"matches": str("the most matches to answer, up to 500 (optional)"),
|
||||
"case": str("\"false\": match whatever the case (optional)")}, "text", "path"),
|
||||
func(a map[string]any) (any, error) {
|
||||
text, _ := a["text"].(string) // as given: a space at either end is part of what is found
|
||||
ask := SearchAsk{Text: text, Path: arg(a, "path"), CaseInsensitive: arg(a, "case") == "false"}
|
||||
var err error
|
||||
if ask.Depth, err = number(a, "depth"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ask.Matches, err = number(a, "matches"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return r.Search(ask)
|
||||
}),
|
||||
verb("account", "One account or group by name or number, as this node's account files say: an account's "+
|
||||
"number, group, home, shell and every group it is in; a group's number and members. With neither, every "+
|
||||
"account and group. Never a password field.",
|
||||
object(map[string]any{"name": str("an account's name or number (optional)"),
|
||||
"group": str("a group's name or number (optional)")}),
|
||||
func(a map[string]any) (any, error) { return r.Account(arg(a, "name"), arg(a, "group")) }),
|
||||
}
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// ADR 0310 §3: the account the reads run as is the module's own, made by the mesh, with no login shell, no
|
||||
// home of the operator's and no group beside its own, and the module runs as it.
|
||||
func TestTheAccountHoldsNoSecretsGroupAndNoLogin(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m struct {
|
||||
RunsAs string `json:"runs-as"`
|
||||
SecretsOwner string `json:"secrets-owner"`
|
||||
Claims []map[string]any `json:"claims"`
|
||||
Resources []map[string]any `json:"resources"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m.RunsAs != "mesh-reads" || m.SecretsOwner != "mesh-reads" {
|
||||
t.Errorf("runs as %q, secrets owned by %q", m.RunsAs, m.SecretsOwner)
|
||||
}
|
||||
if len(m.Claims) != 1 || m.Claims[0]["name"] != Seat {
|
||||
t.Errorf("claims %v", m.Claims)
|
||||
}
|
||||
found := false
|
||||
for _, r := range m.Resources {
|
||||
if r["type"] != "user" {
|
||||
continue
|
||||
}
|
||||
found = true
|
||||
if r["name"] != m.RunsAs {
|
||||
t.Errorf("a user other than its own: %v", r)
|
||||
}
|
||||
if r["shell"] != "/usr/bin/nologin" {
|
||||
t.Errorf("the account has a login shell: %v", r["shell"])
|
||||
}
|
||||
if home, _ := r["home"].(string); !strings.HasPrefix(home, "/var/lib/") {
|
||||
t.Errorf("the account's home is %q, not one of the mesh's", home)
|
||||
}
|
||||
if _, grouped := r["groups"]; grouped {
|
||||
t.Errorf("the account is put in groups: %v", r["groups"])
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("the module makes no account of its own")
|
||||
}
|
||||
for _, r := range m.Resources {
|
||||
if r["type"] != "user" && r["type"] != "directory" {
|
||||
t.Errorf("a read-only holder declares a %v: %v", r["type"], r)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,926 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The bounds (novox/hq ADR 0310, Consequences: set by the build and stated in each verb's description). Every
|
||||
// answer stays under MaxAnswer, which the tools check once more after an answer is made.
|
||||
const (
|
||||
MaxAnswer = 1 << 20 // no answer of this module is larger, whatever the bounds below allow
|
||||
MaxPaths = 200 // paths one stat answers
|
||||
MaxComponents = 64 // components of one path
|
||||
MaxEntries = 2000 // entries one list answers
|
||||
DefaultReadSize = 64 << 10 // bytes a read answers unless more is asked
|
||||
MaxReadSize = 256 << 10 // bytes a read answers at most
|
||||
MaxScan = 64 << 20 // bytes a read passes over to reach the lines asked
|
||||
DefaultMatches = 100
|
||||
MaxMatches = 500
|
||||
DefaultDepth = 4
|
||||
MaxDepth = 8
|
||||
MaxFiles = 5000 // files one search opens
|
||||
MaxFileScan = 4 << 20 // bytes one search reads of one file
|
||||
MaxLine = 300 // characters of one matching line answered
|
||||
SearchTime = 20 * time.Second
|
||||
MaxAccounts = 2000
|
||||
)
|
||||
|
||||
// Reader reads one node's files and accounts as the account it runs as.
|
||||
type Reader struct {
|
||||
Refuser
|
||||
// Passwd, Group and MountInfo are the files accounts, groups and mount points are read from.
|
||||
Passwd, Group, MountInfo string
|
||||
// Euid is the account this runs as; never root (ADR 0310 §3).
|
||||
Euid func() int
|
||||
// Now is the clock a search is bounded by.
|
||||
Now func() time.Time
|
||||
}
|
||||
|
||||
func newReader(places []string) Reader {
|
||||
return Reader{Refuser: Refuser{Places: places}, Passwd: "/etc/passwd", Group: "/etc/group",
|
||||
MountInfo: "/proc/self/mountinfo", Euid: os.Geteuid, Now: time.Now}
|
||||
}
|
||||
|
||||
// asItsOwn refuses every read when this runs as root: what it may see is bounded by its account's file
|
||||
// modes, and root's are none (novox/hq ADR 0310 §3). Loud, never a silent widening.
|
||||
func (r Reader) asItsOwn() error {
|
||||
if r.Euid() == 0 {
|
||||
return errors.New("mesh-reads runs as root here, and its reads are bounded by an account of its own that " +
|
||||
"holds no secret (novox/hq ADR 0310 §3): it answers nothing until it runs as mesh-reads")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func absolute(path string) (string, error) {
|
||||
path = strings.TrimSpace(path)
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
return "", fmt.Errorf("%q is not an absolute path: name it from /", path)
|
||||
}
|
||||
return filepath.Clean(path), nil
|
||||
}
|
||||
|
||||
// why turns a file error into the words an answer says: what stopped the read, never a Go error's text.
|
||||
func why(err error) string {
|
||||
switch {
|
||||
case errors.Is(err, fs.ErrNotExist):
|
||||
return "it does not exist"
|
||||
case errors.Is(err, fs.ErrPermission):
|
||||
return "the account mesh-reads may not reach it (its file modes refuse it)"
|
||||
case errors.Is(err, syscall.ELOOP):
|
||||
return "its links loop"
|
||||
case errors.Is(err, syscall.ENOTDIR):
|
||||
return "a component of it is not a directory"
|
||||
}
|
||||
var pe *fs.PathError
|
||||
if errors.As(err, &pe) {
|
||||
return pe.Err.Error()
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
|
||||
// ---- stat ----
|
||||
|
||||
// Component is one component of a path as lstat sees it.
|
||||
type Component struct {
|
||||
Path string `json:"path"`
|
||||
Type string `json:"type,omitempty"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
UID *int `json:"uid,omitempty"`
|
||||
Group string `json:"group,omitempty"`
|
||||
GID *int `json:"gid,omitempty"`
|
||||
Mode string `json:"mode,omitempty"` // octal, with the setuid, setgid and sticky bits
|
||||
Perms string `json:"perms,omitempty"` // as ls shows them
|
||||
Size *int64 `json:"size,omitempty"`
|
||||
Link string `json:"link,omitempty"` // what a link points to, as written
|
||||
Mount *Mount `json:"mount,omitempty"`
|
||||
Secret string `json:"secret-place,omitempty"` // why a read under it is refused
|
||||
Unreachd string `json:"unreachable,omitempty"`
|
||||
}
|
||||
|
||||
// Mount is a mount point: its file system and source.
|
||||
type Mount struct {
|
||||
FS string `json:"fs"`
|
||||
Source string `json:"source"`
|
||||
}
|
||||
|
||||
// PathStat is one path's components, from / down, and where it leads once every link is followed.
|
||||
type PathStat struct {
|
||||
Path string `json:"path"`
|
||||
Components []Component `json:"components"`
|
||||
LeadsTo string `json:"leads-to,omitempty"`
|
||||
Leads *Component `json:"there,omitempty"`
|
||||
Stopped string `json:"stopped,omitempty"`
|
||||
}
|
||||
|
||||
// StatAnswer is what stat answers.
|
||||
type StatAnswer struct {
|
||||
Paths []PathStat `json:"paths"`
|
||||
Withheld []Withheld `json:"withheld,omitempty"`
|
||||
}
|
||||
|
||||
func (r Reader) Stat(paths []string, follow bool) (StatAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return StatAnswer{}, err
|
||||
}
|
||||
if len(paths) == 0 {
|
||||
return StatAnswer{}, errors.New("name a path to stat")
|
||||
}
|
||||
if len(paths) > MaxPaths {
|
||||
return StatAnswer{}, fmt.Errorf("%d paths asked, and one stat answers at most %d: ask in parts", len(paths), MaxPaths)
|
||||
}
|
||||
names := r.names()
|
||||
mounts := r.mounts()
|
||||
var out StatAnswer
|
||||
for _, raw := range paths {
|
||||
path, err := absolute(raw)
|
||||
if err != nil {
|
||||
return StatAnswer{}, err
|
||||
}
|
||||
ps := PathStat{Path: path}
|
||||
walk := []string{"/"}
|
||||
acc := ""
|
||||
for _, part := range strings.Split(strings.TrimPrefix(path, "/"), "/") {
|
||||
if part == "" {
|
||||
continue
|
||||
}
|
||||
acc += "/" + part
|
||||
walk = append(walk, acc)
|
||||
}
|
||||
if len(walk) > MaxComponents {
|
||||
return StatAnswer{}, fmt.Errorf("%s has %d components, and a stat answers at most %d", path, len(walk), MaxComponents)
|
||||
}
|
||||
for _, p := range walk {
|
||||
c := r.component(p, names, mounts)
|
||||
ps.Components = append(ps.Components, c)
|
||||
if c.Unreachd != "" {
|
||||
ps.Stopped = c.Path + ": " + c.Unreachd
|
||||
break
|
||||
}
|
||||
}
|
||||
if follow && ps.Stopped == "" {
|
||||
if leads, err := filepath.EvalSymlinks(path); err != nil {
|
||||
ps.Stopped = "following its links: " + why(err)
|
||||
} else {
|
||||
ps.LeadsTo = leads
|
||||
if leads != path {
|
||||
c := r.component(leads, names, mounts)
|
||||
ps.Leads = &c
|
||||
}
|
||||
}
|
||||
}
|
||||
out.Paths = append(out.Paths, ps)
|
||||
}
|
||||
out.Withheld = cleanStat(&out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r Reader) component(p string, names accounts, mounts map[string]Mount) Component {
|
||||
c := Component{Path: p}
|
||||
info, err := os.Lstat(p)
|
||||
if err != nil {
|
||||
c.Unreachd = why(err)
|
||||
return c
|
||||
}
|
||||
c.Type = typeOf(info.Mode())
|
||||
c.Perms = info.Mode().String()
|
||||
if st, ok := info.Sys().(*syscall.Stat_t); ok {
|
||||
uid, gid := int(st.Uid), int(st.Gid)
|
||||
c.UID, c.GID = &uid, &gid
|
||||
c.Owner, c.Group = names.user(uid), names.group(gid)
|
||||
c.Mode = fmt.Sprintf("%04o", st.Mode&07777)
|
||||
}
|
||||
if info.Mode().IsRegular() {
|
||||
size := info.Size()
|
||||
c.Size = &size
|
||||
}
|
||||
if info.Mode()&fs.ModeSymlink != 0 {
|
||||
if target, err := os.Readlink(p); err == nil {
|
||||
c.Link = target
|
||||
}
|
||||
}
|
||||
if m, ok := mounts[p]; ok {
|
||||
c.Mount = &m
|
||||
}
|
||||
if why := r.Refused(p); why != "" {
|
||||
c.Secret = why
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
func typeOf(m fs.FileMode) string {
|
||||
switch {
|
||||
case m.IsDir():
|
||||
return "directory"
|
||||
case m&fs.ModeSymlink != 0:
|
||||
return "link"
|
||||
case m.IsRegular():
|
||||
return "file"
|
||||
case m&fs.ModeNamedPipe != 0:
|
||||
return "fifo"
|
||||
case m&fs.ModeSocket != 0:
|
||||
return "socket"
|
||||
case m&fs.ModeCharDevice != 0:
|
||||
return "character device"
|
||||
case m&fs.ModeDevice != 0:
|
||||
return "block device"
|
||||
}
|
||||
return "other"
|
||||
}
|
||||
|
||||
func cleanStat(a *StatAnswer) []Withheld {
|
||||
var w []Withheld
|
||||
cleanC := func(c *Component) {
|
||||
for _, f := range []*string{&c.Link} {
|
||||
if s, what := clean(*f); what != "" {
|
||||
w = append(w, Withheld{At: "a link's target", Class: what})
|
||||
*f = s
|
||||
}
|
||||
}
|
||||
if c.Mount != nil {
|
||||
if s, what := clean(c.Mount.Source); what != "" {
|
||||
w = append(w, Withheld{At: "a mount's source", Class: what})
|
||||
c.Mount.Source = s
|
||||
}
|
||||
}
|
||||
}
|
||||
for i := range a.Paths {
|
||||
p := &a.Paths[i]
|
||||
for j := range p.Components {
|
||||
cleanC(&p.Components[j])
|
||||
}
|
||||
if p.Leads != nil {
|
||||
cleanC(p.Leads)
|
||||
}
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
// ---- readlink ----
|
||||
|
||||
// LinkAnswer is what readlink answers.
|
||||
type LinkAnswer struct {
|
||||
Path string `json:"path"`
|
||||
Link string `json:"link,omitempty"` // as written; empty when the path is not a link
|
||||
IsLink bool `json:"is-link"`
|
||||
LeadsTo string `json:"leads-to,omitempty"`
|
||||
Exists bool `json:"exists"`
|
||||
Stopped string `json:"stopped,omitempty"`
|
||||
Withheld []Withheld `json:"withheld,omitempty"`
|
||||
}
|
||||
|
||||
func (r Reader) Readlink(raw string) (LinkAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return LinkAnswer{}, err
|
||||
}
|
||||
path, err := absolute(raw)
|
||||
if err != nil {
|
||||
return LinkAnswer{}, err
|
||||
}
|
||||
a := LinkAnswer{Path: path}
|
||||
info, err := os.Lstat(path)
|
||||
if err != nil {
|
||||
a.Stopped = why(err)
|
||||
return a, nil
|
||||
}
|
||||
if info.Mode()&fs.ModeSymlink != 0 {
|
||||
a.IsLink = true
|
||||
if a.Link, err = os.Readlink(path); err != nil {
|
||||
a.Stopped = why(err)
|
||||
}
|
||||
}
|
||||
if leads, err := filepath.EvalSymlinks(path); err == nil {
|
||||
a.LeadsTo, a.Exists = leads, true
|
||||
} else if a.Stopped == "" {
|
||||
a.Stopped = "following its links: " + why(err)
|
||||
}
|
||||
for _, f := range []*string{&a.Link} {
|
||||
if s, what := clean(*f); what != "" {
|
||||
a.Withheld = append(a.Withheld, Withheld{At: "a link's target", Class: what})
|
||||
*f = s
|
||||
}
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// ---- list ----
|
||||
|
||||
// Entry is one entry of a directory.
|
||||
type Entry struct {
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
Group string `json:"group,omitempty"`
|
||||
Mode string `json:"mode,omitempty"`
|
||||
Size *int64 `json:"size,omitempty"`
|
||||
Link string `json:"link,omitempty"`
|
||||
}
|
||||
|
||||
// ListAnswer is what list answers.
|
||||
type ListAnswer struct {
|
||||
Path string `json:"path"`
|
||||
Entries []Entry `json:"entries"`
|
||||
Count int `json:"count"`
|
||||
Cut string `json:"cut,omitempty"`
|
||||
Withheld []Withheld `json:"withheld,omitempty"`
|
||||
}
|
||||
|
||||
// refusedRead is a refusal by name, before any read: the path asked and where it leads are both judged, so a
|
||||
// link to a secret place is refused as the place is.
|
||||
func (r Reader) refusedRead(path string) error {
|
||||
if why := r.Refused(path); why != "" {
|
||||
return fmt.Errorf("%s is refused by name: %s (novox/hq ADR 0310 §5)", path, why)
|
||||
}
|
||||
if leads, err := filepath.EvalSymlinks(path); err == nil && leads != path {
|
||||
if why := r.Refused(leads); why != "" {
|
||||
return fmt.Errorf("%s leads to %s, which is refused by name: %s (novox/hq ADR 0310 §5)", path, leads, why)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r Reader) List(raw string) (ListAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return ListAnswer{}, err
|
||||
}
|
||||
path, err := absolute(raw)
|
||||
if err != nil {
|
||||
return ListAnswer{}, err
|
||||
}
|
||||
if err := r.refusedRead(path); err != nil {
|
||||
return ListAnswer{}, err
|
||||
}
|
||||
entries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
return ListAnswer{}, fmt.Errorf("%s cannot be listed: %s", path, why(err))
|
||||
}
|
||||
names := r.names()
|
||||
a := ListAnswer{Path: path, Count: len(entries), Entries: []Entry{}}
|
||||
if len(entries) > MaxEntries {
|
||||
a.Cut = fmt.Sprintf("cut at %d of %d entries (the bound of one list), in name order", MaxEntries, len(entries))
|
||||
entries = entries[:MaxEntries]
|
||||
}
|
||||
for _, e := range entries {
|
||||
en := Entry{Name: e.Name(), Type: typeOf(e.Type())}
|
||||
if info, err := e.Info(); err == nil {
|
||||
en.Type = typeOf(info.Mode())
|
||||
if st, ok := info.Sys().(*syscall.Stat_t); ok {
|
||||
en.Owner, en.Group = names.user(int(st.Uid)), names.group(int(st.Gid))
|
||||
en.Mode = fmt.Sprintf("%04o", st.Mode&07777)
|
||||
}
|
||||
if info.Mode().IsRegular() {
|
||||
size := info.Size()
|
||||
en.Size = &size
|
||||
}
|
||||
if info.Mode()&fs.ModeSymlink != 0 {
|
||||
en.Link, _ = os.Readlink(filepath.Join(path, e.Name()))
|
||||
}
|
||||
}
|
||||
for _, f := range []*string{&en.Link} {
|
||||
if s, what := clean(*f); what != "" {
|
||||
a.Withheld = append(a.Withheld, Withheld{At: "a link's target", Class: what})
|
||||
*f = s
|
||||
}
|
||||
}
|
||||
a.Entries = append(a.Entries, en)
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// ---- read ----
|
||||
|
||||
// ReadAnswer is what read answers.
|
||||
type ReadAnswer struct {
|
||||
Path string `json:"path"`
|
||||
Size int64 `json:"size"` // as the file says; 0 for most of /proc and /sys
|
||||
From int `json:"from"` // the first line answered, counted from 1
|
||||
Lines int `json:"lines"`
|
||||
Text string `json:"text"`
|
||||
Cut string `json:"cut,omitempty"`
|
||||
Withheld []int `json:"withheld-lines,omitempty"`
|
||||
Why string `json:"withheld-because,omitempty"`
|
||||
}
|
||||
|
||||
// ReadAsk is what a read asks: the lines from From (1 is the first; a negative From counts from the end, as
|
||||
// tail does), at most Lines of them (0: as many as fit), and at most Bytes bytes.
|
||||
type ReadAsk struct {
|
||||
From, Lines, Bytes int
|
||||
}
|
||||
|
||||
func (r Reader) Read(raw string, ask ReadAsk) (ReadAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return ReadAnswer{}, err
|
||||
}
|
||||
path, err := absolute(raw)
|
||||
if err != nil {
|
||||
return ReadAnswer{}, err
|
||||
}
|
||||
if err := r.refusedRead(path); err != nil {
|
||||
return ReadAnswer{}, err
|
||||
}
|
||||
bound := ask.Bytes
|
||||
switch {
|
||||
case bound <= 0:
|
||||
bound = DefaultReadSize
|
||||
case bound > MaxReadSize:
|
||||
return ReadAnswer{}, fmt.Errorf("%d bytes asked, and one read answers at most %d", bound, MaxReadSize)
|
||||
}
|
||||
if ask.Lines < 0 {
|
||||
return ReadAnswer{}, errors.New("lines is a count: name how many, or none for as many as fit")
|
||||
}
|
||||
// Non-blocking, so a fifo answers rather than waits; and judged by what was opened, not by its name.
|
||||
f, err := os.OpenFile(path, os.O_RDONLY|syscall.O_NONBLOCK, 0)
|
||||
if err != nil {
|
||||
return ReadAnswer{}, fmt.Errorf("%s cannot be read: %s", path, why(err))
|
||||
}
|
||||
defer f.Close()
|
||||
info, err := f.Stat()
|
||||
if err != nil {
|
||||
return ReadAnswer{}, fmt.Errorf("%s cannot be read: %s", path, why(err))
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return ReadAnswer{}, fmt.Errorf("%s is a %s, not a file: list a directory, stat anything else", path, typeOf(info.Mode()))
|
||||
}
|
||||
a := ReadAnswer{Path: path, Size: info.Size()}
|
||||
lines, cut, err := pick(io.LimitReader(f, MaxScan), ask, bound)
|
||||
if err != nil {
|
||||
return ReadAnswer{}, fmt.Errorf("%s: %v", path, err)
|
||||
}
|
||||
a.Cut = cut
|
||||
if len(lines) > 0 {
|
||||
a.From = lines[0].n
|
||||
}
|
||||
var text strings.Builder
|
||||
for _, l := range lines {
|
||||
s, what := clean(l.text)
|
||||
if what != "" {
|
||||
a.Withheld = append(a.Withheld, l.n)
|
||||
a.Why = "a line shaped like a secret is withheld by the content rule (novox/hq ADR 0310 §4)"
|
||||
}
|
||||
text.WriteString(s)
|
||||
if l.newline {
|
||||
text.WriteByte('\n')
|
||||
}
|
||||
}
|
||||
a.Text, a.Lines = text.String(), len(lines)
|
||||
return a, nil
|
||||
}
|
||||
|
||||
type line struct {
|
||||
n int
|
||||
text string
|
||||
newline bool
|
||||
size int
|
||||
}
|
||||
|
||||
// pick reads lines and keeps the ones asked, within bound bytes. A NUL byte is a binary file, refused.
|
||||
func pick(in io.Reader, ask ReadAsk, bound int) ([]line, string, error) {
|
||||
br := bufio.NewReaderSize(in, 64<<10)
|
||||
var kept []line
|
||||
size, n := 0, 0
|
||||
from := ask.From
|
||||
if from == 0 {
|
||||
from = 1
|
||||
}
|
||||
for {
|
||||
raw, err := br.ReadString('\n')
|
||||
if raw == "" && err != nil {
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
return nil, "", err
|
||||
}
|
||||
n++
|
||||
if strings.IndexByte(raw, 0) >= 0 {
|
||||
return nil, "", errors.New("it is a binary file, and read answers text")
|
||||
}
|
||||
l := line{n: n, text: strings.TrimSuffix(raw, "\n"), newline: strings.HasSuffix(raw, "\n"), size: len(raw)}
|
||||
if from > 0 {
|
||||
if n < from {
|
||||
continue
|
||||
}
|
||||
if ask.Lines > 0 && len(kept) == ask.Lines {
|
||||
break
|
||||
}
|
||||
if size+len(raw) > bound {
|
||||
return kept, fmt.Sprintf("cut at line %d: the answer holds at most %d bytes", n, bound), nil
|
||||
}
|
||||
kept = append(kept, l)
|
||||
size += len(raw)
|
||||
} else {
|
||||
// From the end, as tail does: the last -from lines, kept as they pass.
|
||||
kept = append(kept, l)
|
||||
size += len(raw)
|
||||
for len(kept) > -from || size > bound {
|
||||
size -= kept[0].size
|
||||
kept = kept[1:]
|
||||
}
|
||||
}
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
}
|
||||
if from > 0 && n < from && n > 0 {
|
||||
return nil, "", fmt.Errorf("it has %d lines, and line %d was asked", n, from)
|
||||
}
|
||||
if from < 0 && ask.Lines > 0 && len(kept) > ask.Lines {
|
||||
kept = kept[:ask.Lines]
|
||||
}
|
||||
return kept, "", nil
|
||||
}
|
||||
|
||||
// ---- search ----
|
||||
|
||||
// Match is one matching line.
|
||||
type Match struct {
|
||||
File string `json:"file"`
|
||||
Line int `json:"line"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
// SearchAnswer is what search answers.
|
||||
type SearchAnswer struct {
|
||||
Text string `json:"text"`
|
||||
Path string `json:"path"`
|
||||
Matches []Match `json:"matches"`
|
||||
Files int `json:"files-read"`
|
||||
Bound string `json:"bound-met,omitempty"`
|
||||
Skipped []string `json:"refused-by-name,omitempty"`
|
||||
Unreadable int `json:"unreadable,omitempty"`
|
||||
Binary int `json:"binary-skipped,omitempty"`
|
||||
Withheld []Withheld `json:"withheld,omitempty"`
|
||||
}
|
||||
|
||||
// SearchAsk is what a search asks.
|
||||
type SearchAsk struct {
|
||||
Text, Path string
|
||||
Depth, Matches int
|
||||
CaseInsensitive bool
|
||||
}
|
||||
|
||||
var errBound = errors.New("bound met")
|
||||
|
||||
func (r Reader) Search(ask SearchAsk) (SearchAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return SearchAnswer{}, err
|
||||
}
|
||||
if ask.Text == "" {
|
||||
return SearchAnswer{}, errors.New("name the text to find")
|
||||
}
|
||||
if strings.ContainsAny(ask.Text, "\n\x00") {
|
||||
return SearchAnswer{}, errors.New("the text is found within one line: it may not hold a line break")
|
||||
}
|
||||
root, err := absolute(ask.Path)
|
||||
if err != nil {
|
||||
return SearchAnswer{}, err
|
||||
}
|
||||
if err := r.refusedRead(root); err != nil {
|
||||
return SearchAnswer{}, err
|
||||
}
|
||||
depth, matches := ask.Depth, ask.Matches
|
||||
switch {
|
||||
case depth == 0:
|
||||
depth = DefaultDepth
|
||||
case depth < 0 || depth > MaxDepth:
|
||||
return SearchAnswer{}, fmt.Errorf("depth %d asked; a search goes at most %d directories deep", depth, MaxDepth)
|
||||
}
|
||||
switch {
|
||||
case matches == 0:
|
||||
matches = DefaultMatches
|
||||
case matches < 0 || matches > MaxMatches:
|
||||
return SearchAnswer{}, fmt.Errorf("%d matches asked; a search answers at most %d", matches, MaxMatches)
|
||||
}
|
||||
needle := ask.Text
|
||||
if ask.CaseInsensitive {
|
||||
needle = strings.ToLower(needle)
|
||||
}
|
||||
a := SearchAnswer{Text: ask.Text, Path: root, Matches: []Match{}}
|
||||
deadline := r.Now().Add(SearchTime)
|
||||
base := strings.Count(root, "/")
|
||||
if root == "/" {
|
||||
base = 0
|
||||
}
|
||||
err = filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
|
||||
if r.Now().After(deadline) {
|
||||
a.Bound = fmt.Sprintf("time: a search runs at most %s", SearchTime)
|
||||
return errBound
|
||||
}
|
||||
if err != nil {
|
||||
a.Unreadable++
|
||||
if d != nil && d.IsDir() {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if p != root {
|
||||
if why := r.Refused(p); why != "" {
|
||||
a.Skipped = append(a.Skipped, p)
|
||||
if d.IsDir() {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if d.IsDir() {
|
||||
if p != root && strings.Count(p, "/")-base > depth {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !d.Type().IsRegular() {
|
||||
return nil // links are not followed; a device, a fifo or a socket is not text
|
||||
}
|
||||
if a.Files >= MaxFiles {
|
||||
a.Bound = fmt.Sprintf("files: a search reads at most %d files", MaxFiles)
|
||||
return errBound
|
||||
}
|
||||
a.Files++
|
||||
return r.searchFile(p, needle, ask.CaseInsensitive, matches, &a)
|
||||
})
|
||||
if err != nil && !errors.Is(err, errBound) {
|
||||
return SearchAnswer{}, fmt.Errorf("%s cannot be searched: %s", root, why(err))
|
||||
}
|
||||
return a, nil
|
||||
}
|
||||
|
||||
func (r Reader) searchFile(p, needle string, fold bool, bound int, a *SearchAnswer) error {
|
||||
f, err := os.OpenFile(p, os.O_RDONLY|syscall.O_NONBLOCK, 0)
|
||||
if err != nil {
|
||||
a.Unreadable++
|
||||
return nil
|
||||
}
|
||||
defer f.Close()
|
||||
br := bufio.NewReaderSize(io.LimitReader(f, MaxFileScan), 64<<10)
|
||||
if head, _ := br.Peek(8 << 10); bytes.IndexByte(head, 0) >= 0 {
|
||||
a.Binary++
|
||||
return nil
|
||||
}
|
||||
n := 0
|
||||
for {
|
||||
raw, err := br.ReadString('\n')
|
||||
if raw == "" && err != nil {
|
||||
return nil
|
||||
}
|
||||
n++
|
||||
hay := raw
|
||||
if fold {
|
||||
hay = strings.ToLower(hay)
|
||||
}
|
||||
if strings.Contains(hay, needle) {
|
||||
if len(a.Matches) >= bound {
|
||||
a.Bound = fmt.Sprintf("matches: %d answered, as asked or the default", bound)
|
||||
return errBound
|
||||
}
|
||||
text := strings.TrimRight(raw, "\r\n")
|
||||
if r := []rune(text); len(r) > MaxLine {
|
||||
text = string(r[:MaxLine]) + "…"
|
||||
}
|
||||
file := p
|
||||
for _, f := range []*string{&text} {
|
||||
if s, what := clean(*f); what != "" {
|
||||
a.Withheld = append(a.Withheld, Withheld{At: "a matching line", Class: what})
|
||||
*f = s
|
||||
}
|
||||
}
|
||||
a.Matches = append(a.Matches, Match{File: file, Line: n, Text: text})
|
||||
}
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- accounts ----
|
||||
|
||||
// Account is one account as the account file says it, never its password field.
|
||||
type Account struct {
|
||||
Name string `json:"name"`
|
||||
UID int `json:"uid"`
|
||||
GID int `json:"gid"`
|
||||
Group string `json:"group,omitempty"`
|
||||
Gecos string `json:"gecos,omitempty"`
|
||||
Home string `json:"home"`
|
||||
Shell string `json:"shell"`
|
||||
Groups []string `json:"groups,omitempty"`
|
||||
}
|
||||
|
||||
// GroupEntry is one group.
|
||||
type GroupEntry struct {
|
||||
Name string `json:"name"`
|
||||
GID int `json:"gid"`
|
||||
Members []string `json:"members"`
|
||||
}
|
||||
|
||||
// AccountAnswer is what account answers.
|
||||
type AccountAnswer struct {
|
||||
Accounts []Account `json:"accounts,omitempty"`
|
||||
Groups []GroupEntry `json:"groups,omitempty"`
|
||||
Source string `json:"source"`
|
||||
Cut string `json:"cut,omitempty"`
|
||||
Withheld []Withheld `json:"withheld,omitempty"`
|
||||
}
|
||||
|
||||
type accounts struct {
|
||||
users []Account
|
||||
groups []GroupEntry
|
||||
}
|
||||
|
||||
func (a accounts) user(uid int) string {
|
||||
for _, u := range a.users {
|
||||
if u.UID == uid {
|
||||
return u.Name
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (a accounts) group(gid int) string {
|
||||
for _, g := range a.groups {
|
||||
if g.GID == gid {
|
||||
return g.Name
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// names reads the account files; a file it cannot read gives no names, and a component says only numbers.
|
||||
func (r Reader) names() accounts {
|
||||
var a accounts
|
||||
if raw, err := os.ReadFile(r.Passwd); err == nil {
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
f := strings.Split(l, ":")
|
||||
if len(f) < 7 || strings.HasPrefix(l, "#") {
|
||||
continue
|
||||
}
|
||||
uid, e1 := strconv.Atoi(f[2])
|
||||
gid, e2 := strconv.Atoi(f[3])
|
||||
if e1 != nil || e2 != nil {
|
||||
continue
|
||||
}
|
||||
a.users = append(a.users, Account{Name: f[0], UID: uid, GID: gid, Gecos: f[4], Home: f[5], Shell: f[6]})
|
||||
}
|
||||
}
|
||||
if raw, err := os.ReadFile(r.Group); err == nil {
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
f := strings.Split(l, ":")
|
||||
if len(f) < 4 || strings.HasPrefix(l, "#") {
|
||||
continue
|
||||
}
|
||||
gid, err := strconv.Atoi(f[2])
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
members := []string{}
|
||||
for _, m := range strings.Split(f[3], ",") {
|
||||
if m = strings.TrimSpace(m); m != "" {
|
||||
members = append(members, m)
|
||||
}
|
||||
}
|
||||
a.groups = append(a.groups, GroupEntry{Name: f[0], GID: gid, Members: members})
|
||||
}
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
func (r Reader) Account(name, group string) (AccountAnswer, error) {
|
||||
if err := r.asItsOwn(); err != nil {
|
||||
return AccountAnswer{}, err
|
||||
}
|
||||
all := r.names()
|
||||
if len(all.users) == 0 {
|
||||
return AccountAnswer{}, fmt.Errorf("%s gave no account: it is unreadable or empty", r.Passwd)
|
||||
}
|
||||
out := AccountAnswer{Source: r.Passwd + ", " + r.Group + " (an account kept elsewhere, such as a directory " +
|
||||
"service or systemd's dynamic accounts, is not here)"}
|
||||
complete := func(u Account) Account {
|
||||
u.Group = all.group(u.GID)
|
||||
for _, g := range all.groups {
|
||||
if g.GID == u.GID {
|
||||
u.Groups = append(u.Groups, g.Name)
|
||||
continue
|
||||
}
|
||||
for _, m := range g.Members {
|
||||
if m == u.Name {
|
||||
u.Groups = append(u.Groups, g.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return u
|
||||
}
|
||||
name, group = strings.TrimSpace(name), strings.TrimSpace(group)
|
||||
switch {
|
||||
case name != "":
|
||||
for _, u := range all.users {
|
||||
if u.Name == name || strconv.Itoa(u.UID) == name {
|
||||
out.Accounts = append(out.Accounts, complete(u))
|
||||
}
|
||||
}
|
||||
if len(out.Accounts) == 0 {
|
||||
return AccountAnswer{}, fmt.Errorf("no account %q on this node", name)
|
||||
}
|
||||
case group != "":
|
||||
for _, g := range all.groups {
|
||||
if g.Name == group || strconv.Itoa(g.GID) == group {
|
||||
// Its members as getent says them, and every account whose own group it is.
|
||||
for _, u := range all.users {
|
||||
if u.GID == g.GID && !contains(g.Members, u.Name) {
|
||||
g.Members = append(g.Members, u.Name)
|
||||
}
|
||||
}
|
||||
out.Groups = append(out.Groups, g)
|
||||
}
|
||||
}
|
||||
if len(out.Groups) == 0 {
|
||||
return AccountAnswer{}, fmt.Errorf("no group %q on this node", group)
|
||||
}
|
||||
default:
|
||||
for _, u := range all.users {
|
||||
out.Accounts = append(out.Accounts, complete(u))
|
||||
}
|
||||
out.Groups = all.groups
|
||||
if len(out.Accounts) > MaxAccounts {
|
||||
out.Cut = fmt.Sprintf("cut at %d of %d accounts", MaxAccounts, len(out.Accounts))
|
||||
out.Accounts = out.Accounts[:MaxAccounts]
|
||||
}
|
||||
if len(out.Groups) > MaxAccounts {
|
||||
out.Cut = strings.TrimPrefix(out.Cut+"; ", "; ") + fmt.Sprintf("cut at %d of %d groups", MaxAccounts, len(out.Groups))
|
||||
out.Groups = out.Groups[:MaxAccounts]
|
||||
}
|
||||
}
|
||||
for i := range out.Accounts {
|
||||
u := &out.Accounts[i]
|
||||
for _, f := range []*string{&u.Gecos, &u.Home, &u.Shell} {
|
||||
if s, what := clean(*f); what != "" {
|
||||
out.Withheld = append(out.Withheld, Withheld{At: "account " + u.Name, Class: what})
|
||||
*f = s
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.SliceStable(out.Accounts, func(i, j int) bool { return out.Accounts[i].UID < out.Accounts[j].UID })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func contains(list []string, s string) bool {
|
||||
for _, x := range list {
|
||||
if x == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ---- mounts ----
|
||||
|
||||
// mounts reads the mount points this process sees, by path. /proc/self/mountinfo is readable by every account.
|
||||
func (r Reader) mounts() map[string]Mount {
|
||||
out := map[string]Mount{}
|
||||
raw, err := os.ReadFile(r.MountInfo)
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
f := strings.Fields(l)
|
||||
dash := -1
|
||||
for i, x := range f {
|
||||
if x == "-" {
|
||||
dash = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if len(f) < 5 || dash < 0 || dash+2 >= len(f) {
|
||||
continue
|
||||
}
|
||||
out[unescape(f[4])] = Mount{FS: f[dash+1], Source: unescape(f[dash+2])}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// unescape reads mountinfo's octal escapes (\040 for a space).
|
||||
func unescape(s string) string {
|
||||
if !strings.Contains(s, `\`) {
|
||||
return s
|
||||
}
|
||||
var b strings.Builder
|
||||
for i := 0; i < len(s); i++ {
|
||||
if s[i] == '\\' && i+3 < len(s) {
|
||||
if v, err := strconv.ParseUint(s[i+1:i+4], 8, 8); err == nil {
|
||||
b.WriteByte(byte(v))
|
||||
i += 3
|
||||
continue
|
||||
}
|
||||
}
|
||||
b.WriteByte(s[i])
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
@@ -1,452 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A token shaped as a secret, written into files the tests read.
|
||||
const aToken = "ghp_abcdefghijklmnopqrstuvwxyz0123456789"
|
||||
|
||||
func testReader(t *testing.T, places ...string) (Reader, string) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
passwd := filepath.Join(dir, "passwd")
|
||||
group := filepath.Join(dir, "group")
|
||||
mountinfo := filepath.Join(dir, "mountinfo")
|
||||
write(t, passwd, "root:x:0:0::/root:/usr/bin/bash\nops:x:1000:1000:The Operator:/home/ops:/usr/bin/zsh\n"+
|
||||
"mesh-reads:x:961:961::/var/lib/mesh-reads:/usr/bin/nologin\n")
|
||||
write(t, group, "root:x:0:\nwheel:x:998:ops\nops:x:1000:\nmesh-reads:x:961:\ndocker:x:970:ops,builder\n")
|
||||
data := filepath.Join(dir, "data")
|
||||
write(t, mountinfo, "22 1 0:21 / / rw,relatime - btrfs /dev/nvme0n1p2 rw\n"+
|
||||
"40 22 0:40 / "+strings.ReplaceAll(data, " ", `\040`)+" rw - nfs4 home-server:/tank/data rw\n")
|
||||
r := Reader{Refuser: Refuser{Places: places}, Passwd: passwd, Group: group, MountInfo: mountinfo,
|
||||
Euid: func() int { return 961 }, Now: time.Now}
|
||||
return r, dir
|
||||
}
|
||||
|
||||
func write(t *testing.T, path, content string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §3: run as root, every read is refused, loudly — its bound is an account of its own.
|
||||
func TestEveryReadIsRefusedAsRoot(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
r.Euid = func() int { return 0 }
|
||||
write(t, filepath.Join(dir, "f"), "x\n")
|
||||
if _, err := r.Read(filepath.Join(dir, "f"), ReadAsk{}); err == nil || !strings.Contains(err.Error(), "root") {
|
||||
t.Errorf("a read as root: %v", err)
|
||||
}
|
||||
if _, err := r.Stat([]string{dir}, false); err == nil {
|
||||
t.Error("a stat as root was answered")
|
||||
}
|
||||
if _, err := r.Account("ops", ""); err == nil {
|
||||
t.Error("an account lookup as root was answered")
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §5: without the controller's list of secret places it does not start: a refusal it promises would be
|
||||
// silent.
|
||||
func TestItDoesNotStartWithoutTheNodesSecretPlaces(t *testing.T) {
|
||||
if _, err := placesFrom(""); err == nil || !strings.Contains(err.Error(), PlacesWord) {
|
||||
t.Errorf("no places: %v", err)
|
||||
}
|
||||
if p, err := placesFrom(`["/var/lib/mesh-vault"]`); err != nil || len(p) != 1 {
|
||||
t.Errorf("a list: %v %v", p, err)
|
||||
}
|
||||
if p, err := placesFrom(`[]`); err != nil || len(p) != 0 {
|
||||
t.Errorf("an empty list is a node with no secret place: %v %v", p, err)
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §5: a path under the vault's directory, a module's own-secret directory and a known credential file
|
||||
// are each refused by name before any read — though the account's modes would allow it — and a link to one is
|
||||
// refused as the place is.
|
||||
func TestSecretPlacesAreRefusedByNameBeforeAnyRead(t *testing.T) {
|
||||
r0, dir := testReader(t)
|
||||
vault := filepath.Join(dir, "var/lib/mesh-vault")
|
||||
own := filepath.Join(dir, "var/lib/desk-channel")
|
||||
r, _ := r0, dir
|
||||
r.Places = []string{vault, own}
|
||||
for _, p := range []string{
|
||||
filepath.Join(vault, "ledger/entries.json"),
|
||||
filepath.Join(own, "broker"),
|
||||
filepath.Join(own, "settings.json"),
|
||||
filepath.Join(dir, "home/ops/.claude/.credentials.json"),
|
||||
filepath.Join(dir, "home/ops/.ssh/config"),
|
||||
filepath.Join(dir, "home/ops/.pgpass"),
|
||||
filepath.Join(dir, "etc/ssh/ssh_host_ed25519_key"),
|
||||
filepath.Join(dir, "etc/shadow"),
|
||||
filepath.Join(dir, "srv/tls/server.key"),
|
||||
} {
|
||||
write(t, p, "harmless words\n")
|
||||
if _, err := r.Read(p, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "refused by name") {
|
||||
t.Errorf("%s was read: %v", p, err)
|
||||
}
|
||||
}
|
||||
if _, err := r.List(vault); err == nil || !strings.Contains(err.Error(), "refused by name") {
|
||||
t.Errorf("the vault's directory was listed: %v", err)
|
||||
}
|
||||
link := filepath.Join(dir, "tmp/innocent")
|
||||
write(t, filepath.Join(dir, "tmp/x"), "")
|
||||
if err := os.Symlink(filepath.Join(own, "broker"), link); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := r.Read(link, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "leads to") {
|
||||
t.Errorf("a link to a secret place was read: %v", err)
|
||||
}
|
||||
// A search under a directory holding a secret place skips it and names it.
|
||||
write(t, filepath.Join(dir, "home/ops/notes.txt"), "harmless words\n")
|
||||
a, err := r.Search(SearchAsk{Text: "harmless", Path: filepath.Join(dir, "home"), Depth: 8})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(a.Matches) != 1 || !strings.HasSuffix(a.Matches[0].File, "notes.txt") || len(a.Skipped) != 3 {
|
||||
t.Errorf("search: matches %v, skipped %v", a.Matches, a.Skipped)
|
||||
}
|
||||
// What a public key or a certificate is, is not refused.
|
||||
for _, p := range []string{filepath.Join(dir, "etc/ssh/ssh_host_ed25519_key.pub"), filepath.Join(dir, "etc/ssl/certs/ca.pem")} {
|
||||
write(t, p, "harmless words\n")
|
||||
if _, err := r.Read(p, ReadAsk{}); err != nil {
|
||||
t.Errorf("%s was refused: %v", p, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §3: what the account's file modes refuse is refused — the first bound, below the names.
|
||||
func TestAFileTheAccountMayNotReadIsRefusedByItsMode(t *testing.T) {
|
||||
if os.Geteuid() == 0 {
|
||||
t.Skip("run as root, a file's mode refuses nothing")
|
||||
}
|
||||
r, dir := testReader(t)
|
||||
p := filepath.Join(dir, "operators-only")
|
||||
write(t, p, "the operator's words\n")
|
||||
if err := os.Chmod(p, 0o000); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := r.Read(p, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "may not reach it") {
|
||||
t.Errorf("a file of mode 0000: %v", err)
|
||||
}
|
||||
closed := filepath.Join(dir, "closed")
|
||||
write(t, filepath.Join(closed, "inside"), "x")
|
||||
if err := os.Chmod(closed, 0o700&^0o100); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer os.Chmod(closed, 0o755)
|
||||
a, err := r.Stat([]string{filepath.Join(closed, "inside")}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := a.Paths[0]; !strings.Contains(got.Stopped, "may not reach") {
|
||||
t.Errorf("a path through a directory it may not search: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §2 and the bounds: a read over the size bound is cut at a line and says so; more than the most is
|
||||
// refused; lines and a tail are answered as asked.
|
||||
func TestAReadIsBounded(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
p := filepath.Join(dir, "big")
|
||||
var b strings.Builder
|
||||
for i := 1; i <= 20000; i++ {
|
||||
fmt.Fprintf(&b, "line %d of the file\n", i)
|
||||
}
|
||||
write(t, p, b.String())
|
||||
a, err := r.Read(p, ReadAsk{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(a.Text) > DefaultReadSize || !strings.Contains(a.Cut, fmt.Sprint(DefaultReadSize)) || a.From != 1 {
|
||||
t.Errorf("a default read: %d bytes, cut %q", len(a.Text), a.Cut)
|
||||
}
|
||||
if _, err := r.Read(p, ReadAsk{Bytes: MaxReadSize + 1}); err == nil {
|
||||
t.Error("more than the most a read answers was answered")
|
||||
}
|
||||
a, err = r.Read(p, ReadAsk{From: 10, Lines: 2})
|
||||
if err != nil || a.Text != "line 10 of the file\nline 11 of the file\n" || a.From != 10 || a.Cut != "" {
|
||||
t.Errorf("lines 10 and 11: %+v %v", a, err)
|
||||
}
|
||||
a, err = r.Read(p, ReadAsk{From: -2})
|
||||
if err != nil || a.Text != "line 19999 of the file\nline 20000 of the file\n" || a.From != 19999 {
|
||||
t.Errorf("the last two lines: %q from %d, %v", a.Text, a.From, err)
|
||||
}
|
||||
if _, err := r.Read(p, ReadAsk{From: 30000}); err == nil || !strings.Contains(err.Error(), "20000 lines") {
|
||||
t.Errorf("a line past the end: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A binary file, a directory and a fifo are refused, and the fifo without waiting for a writer.
|
||||
func TestOnlyATextFileIsRead(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
bin := filepath.Join(dir, "bin")
|
||||
write(t, bin, "ELF\x00\x01\x02")
|
||||
if _, err := r.Read(bin, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "binary") {
|
||||
t.Errorf("a binary file: %v", err)
|
||||
}
|
||||
if _, err := r.Read(dir, ReadAsk{}); err == nil || !strings.Contains(err.Error(), "directory") {
|
||||
t.Errorf("a directory: %v", err)
|
||||
}
|
||||
fifo := filepath.Join(dir, "fifo")
|
||||
if err := syscall.Mkfifo(fifo, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
done := make(chan error, 1)
|
||||
go func() { _, err := r.Read(fifo, ReadAsk{}); done <- err }()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err == nil || !strings.Contains(err.Error(), "fifo") {
|
||||
t.Errorf("a fifo: %v", err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("a read of a fifo waits for a writer")
|
||||
}
|
||||
}
|
||||
|
||||
// ADR 0310 §4: a line shaped like a secret is withheld, its number said, and the rest is answered.
|
||||
func TestALineShapedLikeASecretIsWithheld(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
p := filepath.Join(dir, "app.conf")
|
||||
write(t, p, "listen = /run/app.sock\ntoken = "+aToken+"\nuser = app\n")
|
||||
a, err := r.Read(p, ReadAsk{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(a.Text, aToken) || !strings.Contains(a.Text, "user = app") ||
|
||||
!strings.Contains(a.Text, "listen = /run/app.sock") || fmt.Sprint(a.Withheld) != "[2]" {
|
||||
t.Errorf("read: %q withheld %v", a.Text, a.Withheld)
|
||||
}
|
||||
m, err := r.Search(SearchAsk{Text: "token", Path: dir})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, _ := json.Marshal(m)
|
||||
if strings.Contains(string(body), aToken) || len(m.Matches) != 1 || len(m.Withheld) != 1 {
|
||||
t.Errorf("search: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
// The bounds of a search: matches, depth, the most asked, and time — each met is said.
|
||||
func TestASearchIsBounded(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
for i := 0; i < 30; i++ {
|
||||
write(t, filepath.Join(dir, fmt.Sprintf("f%02d", i)), "needle\nneedle\n")
|
||||
}
|
||||
a, err := r.Search(SearchAsk{Text: "needle", Path: dir, Matches: 7})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(a.Matches) != 7 || !strings.HasPrefix(a.Bound, "matches") {
|
||||
t.Errorf("seven matches asked: %d, bound %q", len(a.Matches), a.Bound)
|
||||
}
|
||||
if _, err := r.Search(SearchAsk{Text: "needle", Path: dir, Matches: MaxMatches + 1}); err == nil {
|
||||
t.Error("more matches than the most were asked and answered")
|
||||
}
|
||||
if _, err := r.Search(SearchAsk{Text: "needle", Path: dir, Depth: MaxDepth + 1}); err == nil {
|
||||
t.Error("deeper than the most was asked and answered")
|
||||
}
|
||||
deep := t.TempDir()
|
||||
write(t, filepath.Join(deep, "a/b/c/shallow"), "needle\n")
|
||||
write(t, filepath.Join(deep, "a/b/c/d/e/deep"), "needle\n")
|
||||
a, err = r.Search(SearchAsk{Text: "needle", Path: deep, Depth: 3})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(a.Matches) != 1 || !strings.HasSuffix(a.Matches[0].File, "shallow") {
|
||||
t.Errorf("three deep: %+v", a.Matches)
|
||||
}
|
||||
// Case: as asked.
|
||||
write(t, filepath.Join(deep, "case"), "NEEDLE\n")
|
||||
if a, _ := r.Search(SearchAsk{Text: "needle", Path: filepath.Join(deep, "case")}); len(a.Matches) != 0 {
|
||||
t.Error("a match whatever the case, unasked")
|
||||
}
|
||||
if a, _ := r.Search(SearchAsk{Text: "needle", Path: filepath.Join(deep, "case"), CaseInsensitive: true}); len(a.Matches) != 1 {
|
||||
t.Error("no match whatever the case, asked")
|
||||
}
|
||||
// Time: a clock past the bound stops the walk and says so.
|
||||
start := time.Now()
|
||||
calls := 0
|
||||
r.Now = func() time.Time { calls++; return start.Add(time.Duration(calls) * SearchTime / 3) }
|
||||
a, err = r.Search(SearchAsk{Text: "needle", Path: dir})
|
||||
if err != nil || !strings.HasPrefix(a.Bound, "time") {
|
||||
t.Errorf("the time bound: %q %v", a.Bound, err)
|
||||
}
|
||||
// A fixed string: a pattern's characters are themselves.
|
||||
write(t, filepath.Join(deep, "pattern"), "a.c\nabc\n")
|
||||
if a, _ := r.Search(SearchAsk{Text: "a.c", Path: filepath.Join(deep, "pattern")}); len(a.Matches) != 1 || a.Matches[0].Line != 1 {
|
||||
t.Errorf("a.c as a fixed string: %+v", a.Matches)
|
||||
}
|
||||
}
|
||||
|
||||
// The survey issue 418 was asked for: each component of a path with its owner, group, mode, whether it is a
|
||||
// link and to what, and whether it is a mount point — the link not followed unless asked.
|
||||
func TestStatSaysEachComponent(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
data := filepath.Join(dir, "data")
|
||||
write(t, filepath.Join(data, "real/file"), "x")
|
||||
if err := os.Chmod(filepath.Join(data, "real"), os.ModeSetgid|0o750); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink("real", filepath.Join(data, "link")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a, err := r.Stat([]string{filepath.Join(data, "link"), filepath.Join(data, "real/file")}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
link := a.Paths[0]
|
||||
last := link.Components[len(link.Components)-1]
|
||||
if link.Components[0].Path != "/" || last.Type != "link" || last.Link != "real" || link.LeadsTo != "" {
|
||||
t.Errorf("the link, not followed: %+v", link)
|
||||
}
|
||||
var dataC, realC Component
|
||||
for _, c := range a.Paths[1].Components {
|
||||
switch c.Path {
|
||||
case data:
|
||||
dataC = c
|
||||
case filepath.Join(data, "real"):
|
||||
realC = c
|
||||
}
|
||||
}
|
||||
if dataC.Mount == nil || dataC.Mount.FS != "nfs4" {
|
||||
t.Errorf("the mount point is not said: %+v", dataC)
|
||||
}
|
||||
if realC.Mode != "2750" || realC.Type != "directory" || realC.UID == nil || realC.Perms == "" {
|
||||
t.Errorf("a component's mode: %+v", realC)
|
||||
}
|
||||
if uid := *realC.UID; uid == 1000 && realC.Owner != "ops" {
|
||||
t.Errorf("owner %d named %q", uid, realC.Owner)
|
||||
}
|
||||
a, err = r.Stat([]string{filepath.Join(data, "link")}, true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a.Paths[0].LeadsTo != filepath.Join(data, "real") || a.Paths[0].Leads == nil || a.Paths[0].Leads.Type != "directory" {
|
||||
t.Errorf("followed: %+v", a.Paths[0])
|
||||
}
|
||||
a, err = r.Stat([]string{filepath.Join(dir, "missing/deeper")}, false)
|
||||
if err != nil || !strings.Contains(a.Paths[0].Stopped, "does not exist") {
|
||||
t.Errorf("a missing path: %+v %v", a, err)
|
||||
}
|
||||
if _, err := r.Stat([]string{"relative/path"}, false); err == nil {
|
||||
t.Error("a relative path was answered")
|
||||
}
|
||||
many := make([]string, MaxPaths+1)
|
||||
for i := range many {
|
||||
many[i] = "/"
|
||||
}
|
||||
if _, err := r.Stat(many, false); err == nil {
|
||||
t.Error("more paths than the most were answered")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadlinkSaysTheTargetAndWhereItLeads(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
write(t, filepath.Join(dir, "target"), "")
|
||||
if err := os.Symlink("target", filepath.Join(dir, "l1")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink("l1", filepath.Join(dir, "l2")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink("nowhere", filepath.Join(dir, "dangling")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a, err := r.Readlink(filepath.Join(dir, "l2"))
|
||||
if err != nil || !a.IsLink || a.Link != "l1" || a.LeadsTo != filepath.Join(dir, "target") || !a.Exists {
|
||||
t.Errorf("l2: %+v %v", a, err)
|
||||
}
|
||||
a, err = r.Readlink(filepath.Join(dir, "dangling"))
|
||||
if err != nil || !a.IsLink || a.Exists || !strings.Contains(a.Stopped, "does not exist") {
|
||||
t.Errorf("dangling: %+v %v", a, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListSaysEveryEntry(t *testing.T) {
|
||||
r, dir := testReader(t)
|
||||
write(t, filepath.Join(dir, "d/a"), "abc")
|
||||
if err := os.Symlink("a", filepath.Join(dir, "d/b")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a, err := r.List(filepath.Join(dir, "d"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a.Count != 2 || a.Entries[0].Name != "a" || *a.Entries[0].Size != 3 || a.Entries[1].Type != "link" || a.Entries[1].Link != "a" {
|
||||
t.Errorf("list: %+v", a)
|
||||
}
|
||||
for i := 0; i < MaxEntries+5; i++ {
|
||||
write(t, filepath.Join(dir, "many", fmt.Sprintf("e%05d", i)), "")
|
||||
}
|
||||
a, err = r.List(filepath.Join(dir, "many"))
|
||||
if err != nil || len(a.Entries) != MaxEntries || a.Count != MaxEntries+5 || a.Cut == "" {
|
||||
t.Errorf("a long directory: %d of %d, cut %q, %v", len(a.Entries), a.Count, a.Cut, err)
|
||||
}
|
||||
}
|
||||
|
||||
// getent passwd, getent group and id: an account with every group it is in, a group with its members, never a
|
||||
// password field.
|
||||
func TestAccountAndGroupLookup(t *testing.T) {
|
||||
r, _ := testReader(t)
|
||||
a, err := r.Account("ops", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
u := a.Accounts[0]
|
||||
sort.Strings(u.Groups)
|
||||
if u.UID != 1000 || u.Group != "ops" || u.Shell != "/usr/bin/zsh" || fmt.Sprint(u.Groups) != "[docker ops wheel]" {
|
||||
t.Errorf("ops: %+v", u)
|
||||
}
|
||||
if a, err := r.Account("961", ""); err != nil || a.Accounts[0].Name != "mesh-reads" {
|
||||
t.Errorf("by number: %+v %v", a, err)
|
||||
}
|
||||
g, err := r.Account("", "docker")
|
||||
if err != nil || g.Groups[0].GID != 970 || fmt.Sprint(g.Groups[0].Members) != "[ops builder]" {
|
||||
t.Errorf("docker: %+v %v", g, err)
|
||||
}
|
||||
if _, err := r.Account("nobody-here", ""); err == nil {
|
||||
t.Error("a missing account was answered")
|
||||
}
|
||||
all, err := r.Account("", "")
|
||||
if err != nil || len(all.Accounts) != 3 || len(all.Groups) != 5 {
|
||||
t.Errorf("all: %+v %v", all, err)
|
||||
}
|
||||
body, _ := json.Marshal(all)
|
||||
if strings.Contains(string(body), `"x"`) || strings.Contains(string(body), "password") {
|
||||
t.Errorf("a password field is answered: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
// No answer is larger than the bound, whatever a verb made.
|
||||
func TestNoAnswerIsLargerThanTheBound(t *testing.T) {
|
||||
if _, err := bounded(strings.Repeat("x", MaxAnswer+1), nil); err == nil {
|
||||
t.Error("an answer over the bound was given")
|
||||
}
|
||||
if _, err := bounded("small", nil); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}
|
||||
|
||||
// The seat's six verbs, as the controller defines them (mesh-controller internal/catalogue/reads.go).
|
||||
func TestItServesTheSeatsVerbs(t *testing.T) {
|
||||
r, _ := testReader(t)
|
||||
var names []string
|
||||
for _, tool := range tools(r) {
|
||||
names = append(names, tool.Name)
|
||||
}
|
||||
if fmt.Sprint(names) != "[node-reads.stat node-reads.readlink node-reads.list node-reads.read node-reads.search node-reads.account]" {
|
||||
t.Errorf("verbs: %v", names)
|
||||
}
|
||||
}
|
||||
@@ -1,109 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"git.novox.be/novox/mesh-sdk/go/outward"
|
||||
)
|
||||
|
||||
// What is refused by name, before any read (novox/hq ADR 0310 §5), and what an answer withholds (§4).
|
||||
//
|
||||
// **Two lists of places.** The places the mesh knows a secret is kept on this node — every module's own-secret
|
||||
// directory, every credential file and grant directory, the vault's directories — come from the controller,
|
||||
// which composes them from the manifests (MESH_SECRET_PLACES). The known credential files of the programs
|
||||
// on a machine are this module's list, below: the mesh does not write them, so nothing else names them. Both
|
||||
// hold whatever the account's file modes would allow: the file modes are the first bound, the names the second.
|
||||
|
||||
// knownDirs are directories that hold nothing but credentials, wherever they are: an account's keys.
|
||||
var knownDirs = map[string]bool{
|
||||
".ssh": true, ".gnupg": true, ".password-store": true, ".aws": true, ".kube": true, ".azure": true,
|
||||
}
|
||||
|
||||
// knownNames are file names that are a credential whatever directory they are in.
|
||||
var knownNames = map[string]bool{
|
||||
".credentials.json": true, // the agent's credentials (Claude Code)
|
||||
".netrc": true, ".pgpass": true, ".git-credentials": true, ".npmrc": true, ".pypirc": true,
|
||||
".my.cnf": true, ".vault-token": true, "credentials": true, "credentials.json": true,
|
||||
"id_rsa": true, "id_dsa": true, "id_ecdsa": true, "id_ed25519": true, "id_ecdsa_sk": true, "id_ed25519_sk": true,
|
||||
"shadow": true, "gshadow": true, "shadow-": true, "gshadow-": true, "opasswd": true,
|
||||
"broker": true, // the mesh's name for a module's bus credential
|
||||
}
|
||||
|
||||
// knownSuffixes are file name endings a credential has.
|
||||
var knownSuffixes = []string{".key", ".p12", ".pfx", ".jks", ".keystore", ".creds", ".nk", ".kdbx", "_key"}
|
||||
|
||||
// knownPaths are the places on a machine that hold credentials and are not a module's.
|
||||
var knownPaths = []string{
|
||||
"/etc/ssl/private", "/etc/wireguard", "/etc/NetworkManager/system-connections", "/etc/ipsec.d/private",
|
||||
"/var/lib/iwd", "/etc/docker/key.json", "/root/.docker/config.json", "/var/lib/sss/secrets",
|
||||
}
|
||||
|
||||
// Refuser says whether a path is a place a secret is kept.
|
||||
type Refuser struct {
|
||||
// Places are the mesh's, from the controller: refused with everything beneath them.
|
||||
Places []string
|
||||
}
|
||||
|
||||
// Refused says why a path may not be read, or "" when it may. The path is clean and absolute.
|
||||
func (r Refuser) Refused(path string) string {
|
||||
path = filepath.Clean(path)
|
||||
for _, p := range r.Places {
|
||||
if within(path, p) {
|
||||
return "it is a place the mesh keeps a secret (" + p + ")"
|
||||
}
|
||||
}
|
||||
for _, p := range knownPaths {
|
||||
if within(path, p) {
|
||||
return "it is a place a machine keeps credentials (" + p + ")"
|
||||
}
|
||||
}
|
||||
parts := strings.Split(strings.TrimPrefix(path, "/"), "/")
|
||||
for i, part := range parts {
|
||||
if knownDirs[part] {
|
||||
return "it is in " + part + ", which holds an account's credentials"
|
||||
}
|
||||
if i == len(parts)-1 {
|
||||
if knownNames[part] {
|
||||
return part + " is a known credential file"
|
||||
}
|
||||
for _, s := range knownSuffixes {
|
||||
if strings.HasSuffix(part, s) && len(part) > len(s) {
|
||||
return part + " is named as a key or a credential (" + s + ")"
|
||||
}
|
||||
}
|
||||
if strings.HasSuffix(part, ".pem") && (strings.Contains(part, "key") || strings.Contains(part, "private")) {
|
||||
return part + " is named as a private key"
|
||||
}
|
||||
if part == "config.json" && i > 0 && parts[i-1] == ".docker" {
|
||||
return "a container registry login is kept there"
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func within(path, place string) bool {
|
||||
return path == place || strings.HasPrefix(path, strings.TrimSuffix(place, "/")+"/")
|
||||
}
|
||||
|
||||
// Withheld is one piece of an answer the content rule kept back: where it was, and the class it carried.
|
||||
type Withheld struct {
|
||||
At string `json:"at"`
|
||||
Class string `json:"why"`
|
||||
}
|
||||
|
||||
// clean is the content rule's secret class (mesh-sdk outward, novox/hq ADR 0310 §4) over one piece of an
|
||||
// answer: the piece, or a mark in its place when it is shaped like a secret.
|
||||
//
|
||||
// **What a file holds, not what the file system names.** It judges a file's lines, a matching line, a link's
|
||||
// target, a mount's source and an account's words. A path and an entry's name are not judged: they are what
|
||||
// the caller named or the file system calls a thing, and the rule's random-run test takes a temporary
|
||||
// directory's or a container layer's name for a token, which would withhold the very survey asked for. A
|
||||
// place where a secret sits by name is refused before any read instead (Refuser).
|
||||
func clean(s string) (string, string) {
|
||||
if refusal, ok := outward.CheckSecret(s); !ok {
|
||||
return "‹withheld: " + refusal.What + "›", refusal.What
|
||||
}
|
||||
return s, ""
|
||||
}
|
||||
@@ -1,5 +0,0 @@
|
||||
module mesh-reads
|
||||
|
||||
go 1.25.0
|
||||
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012115257-4dc5f29d3e23
|
||||
@@ -1,2 +0,0 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012115257-4dc5f29d3e23 h1:/F4NwABPAuFZlBx1yK4rdj6sVL5K1XvRlOnZZfNnlsQ=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261012115257-4dc5f29d3e23/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
@@ -1,54 +0,0 @@
|
||||
{
|
||||
"module": "mesh-reads",
|
||||
"version": "1",
|
||||
"runs-as": "mesh-reads",
|
||||
"claims": [
|
||||
{
|
||||
"name": "node-reads",
|
||||
"scope": "node",
|
||||
"serves": [
|
||||
"stat",
|
||||
"readlink",
|
||||
"list",
|
||||
"read",
|
||||
"search",
|
||||
"account"
|
||||
]
|
||||
}
|
||||
],
|
||||
"own-secrets": {
|
||||
"broker": "${dir:state}/broker"
|
||||
},
|
||||
"secrets-owner": "mesh-reads",
|
||||
"resources": [
|
||||
{
|
||||
"id": "account",
|
||||
"type": "user",
|
||||
"name": "mesh-reads",
|
||||
"shell": "/usr/bin/nologin",
|
||||
"home": "/var/lib/mesh-reads"
|
||||
},
|
||||
{
|
||||
"id": "state",
|
||||
"type": "directory",
|
||||
"mode": "0700",
|
||||
"owner": "mesh-reads",
|
||||
"place": "."
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
"artifacts": [
|
||||
{
|
||||
"name": "tools",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/mesh-reads",
|
||||
"binary": "mesh-reads",
|
||||
"loads": [
|
||||
"mesh-reads"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -16,20 +16,15 @@ channel's client and no channel's secret.
|
||||
its sender) and `desk-channel` (kind `desktop`, the machines' notifiers, proves nothing). Which kinds exist
|
||||
and what each promises is the controller's word, in this module's membership; whether each can send now
|
||||
is the channel's own, said on `intake.standing`.
|
||||
- **Keeps its state on the bus** (ADR 0201): `open` and `sent` (the messages, and the restart passes made),
|
||||
`asks` (every ask, closed ones for 30 days), `identities` (the operator's linked accounts, one per kind, and a
|
||||
link asked for), `walks` (the delivery walks open now, each with what it holds).
|
||||
- **Keeps its state on the bus** (ADR 0201): `open` and `sent` (the messages), `asks` (every ask, closed ones
|
||||
for 30 days), `identities` (the operator's linked accounts, one per kind, and a link asked for).
|
||||
- **Lost answers are said again from here, never from a channel's memory** (issue 384,
|
||||
`cmd/messenger/restart.go`): a channel whose messages' answers were lost — the desk's notifier started
|
||||
again — says when in its standing (`answers-lost-at`). The router says again on that kind, directly and not
|
||||
through the outbox, each message that is still open, asked there, never answered and last said before that
|
||||
time: one per problem, under its handle, with the words and answers of now, so it replaces the message in
|
||||
place. **A restart says nothing it has said** (novox/hq ADR 0308 §6, issue 531): a silenced message, one
|
||||
withheld for its content, one answered and one a walk holds are left as they are, and what ended never comes
|
||||
back. The standings of one restart (a notifier on each machine, seconds apart) are one pass: a loss is acted
|
||||
on once it stood 10 s, and the pass made is kept in the state (`sent`, key `passes`), so a router that starts
|
||||
again does not make it again. A kind that cannot send waits; it never goes to another channel. Telegram's
|
||||
answers are never lost and it says no time.
|
||||
through the outbox, each message that is still open, asked there and last said before that time: one per
|
||||
problem, under its handle, with the words and answers of now, so it replaces the message in place. What is
|
||||
silenced is edited to say so, with no answers and no urgency; what ended never comes back. A kind that cannot
|
||||
send waits; it never goes to another channel. Telegram's answers are never lost and it says no time.
|
||||
|
||||
## Asks, answers and warrants (novox/hq ADR 0259)
|
||||
|
||||
@@ -170,36 +165,10 @@ ends takes them away everywhere; its expiry is said with its day when it is not
|
||||
history. An event that cannot be read
|
||||
is refused by name, counted, and told to the operator once an hour.
|
||||
|
||||
## While a delivery walks (novox/hq ADR 0308)
|
||||
|
||||
The controller's `plan-moved` says each walk's start and end: `walk`, `phase` (`started`, `ended`), `touches`
|
||||
(`[{machine, modules}]`: what it moves and what a moved tool runner or node-engine carries), `outcome`
|
||||
(`delivered`, `failed`, `stopped`), `bound_seconds` and `budget_class` (`cmd/messenger/walks.go`).
|
||||
|
||||
- **Held:** while a walk is open and within its bound, a message whose subject is a machine it touches, or a
|
||||
module it touches (on a machine it touches, when the subject names one), is held: kept on the walk with when
|
||||
and what, and the history says `held`. A message several open walks touch is told when the last ends or passes
|
||||
its bound. At a walk's end, a later open walk that touches a message takes it over only as a first hold would:
|
||||
still open, not silenced, not said since, and not one that is never held.
|
||||
- **Never held:** an ask waiting for the operator (and a condition an open ask is about), a secret given or
|
||||
replaced, a subject no open walk touches, and a walk's own failure (a plan, walk, delivery or build).
|
||||
- **The end is one message:** the outcome, the modules and machines reached, how long; each held message still
|
||||
open, one line each; the ones that cleared counted. Delivered with nothing held and nothing open: one line —
|
||||
and nothing on a kind listed in the setting `only-what-needs-me`. What it carries is marked told on a kind
|
||||
only once the message was sent there. Until then each is kept owed in the state (`owed`): a message lost with a
|
||||
restart before it was sent leaves them untold and owed, and the restarted router says each on its own.
|
||||
- **The bound:** at its start plus `bound_seconds` (900 when not given), what is held and still open is said,
|
||||
with "The walk … is still running, since HH:MM", and the cleared counted; the walk holds nothing more. A walk
|
||||
whose end never comes is forgotten a day after it started. A start heard after its end opens nothing: the
|
||||
controller's moments are compared on its own clock, never with one capped at the router's now.
|
||||
- **Kept in the state** (`walks`): a router that starts again mid-walk holds and tells as before.
|
||||
|
||||
## What the operator gives
|
||||
|
||||
1. **The time zone**, as a setting: `{"time-zone": "<an IANA zone name>"}`. Every time a message says is
|
||||
said in it; not given, the zone of the machine this runs on.
|
||||
**The kinds that say only what needs the operator**, as a setting: `{"only-what-needs-me": ["<kind>"]}`. A
|
||||
walk that delivered with nothing held and nothing open is not told on them (ADR 0308 §4); empty by default.
|
||||
2. **The channels**: assign `telegram` and `desk-channel` (each says in its README what it needs). Linking
|
||||
an account as the operator is done on the channel, never by a setting.
|
||||
|
||||
|
||||
@@ -154,11 +154,6 @@ type Record struct {
|
||||
Subject string `json:"subject"`
|
||||
Severity string `json:"severity"`
|
||||
Summary string `json:"summary"`
|
||||
// Scope, SubjectID and Machine are the condition's subject as the controller gave it, which a walk's touches
|
||||
// are matched against (ADR 0308 §2); empty for a record kept before, whose key names them.
|
||||
Scope string `json:"scope,omitempty"`
|
||||
SubjectID string `json:"subject_id,omitempty"`
|
||||
Machine string `json:"machine,omitempty"`
|
||||
// Headline, Explanation and Resolved are the plain words the operator reads (novox/hq ADR 0253):
|
||||
// the controller's, or a module's through notify; empty, the message is said from the summary.
|
||||
Headline string `json:"headline,omitempty"`
|
||||
@@ -200,17 +195,14 @@ type Record struct {
|
||||
// ClearHeld is what was waiting to be said of it when it cleared (a reminder, an escalation): said after
|
||||
// all if it comes back before its clearing was said. Blips counts those comebacks, never said, so a
|
||||
// condition that flaps is known as one and its message's "again, Nth time" counts only what was said.
|
||||
ClearHeld string `json:"clear_held,omitempty"`
|
||||
Blips int `json:"blips,omitempty"`
|
||||
Reminded bool `json:"reminded,omitempty"`
|
||||
Pending string `json:"pending,omitempty"` // what could not be said yet: raised, reminder, …
|
||||
// Owed is what a walk held of it and has not told yet (ADR 0308 §4, §5): kept in the state until a message
|
||||
// carrying it was sent, so a router that restarts before says it on its own (issue 538), never history.
|
||||
Owed string `json:"owed,omitempty"`
|
||||
Folded bool `json:"folded,omitempty"` // held by the cap
|
||||
Cleared time.Time `json:"cleared,omitempty"`
|
||||
Count int `json:"count"`
|
||||
Refused string `json:"refused,omitempty"`
|
||||
ClearHeld string `json:"clear_held,omitempty"`
|
||||
Blips int `json:"blips,omitempty"`
|
||||
Reminded bool `json:"reminded,omitempty"`
|
||||
Pending string `json:"pending,omitempty"` // what could not be said yet: raised, reminder, …
|
||||
Folded bool `json:"folded,omitempty"` // held by the cap
|
||||
Cleared time.Time `json:"cleared,omitempty"`
|
||||
Count int `json:"count"`
|
||||
Refused string `json:"refused,omitempty"`
|
||||
// History: known from the state or from an old event, and never said — not news.
|
||||
History bool `json:"history,omitempty"`
|
||||
// Seen is when this holder last took word of it; the open set read after it is authoritative.
|
||||
@@ -219,8 +211,7 @@ type Record struct {
|
||||
|
||||
// recordOf is a condition as this holder keeps it.
|
||||
func recordOf(c Condition) Record {
|
||||
return Record{Key: c.Key, Kind: c.Kind, Subject: c.SubjectWords(), Scope: c.Scope, SubjectID: c.ID, Machine: c.Machine,
|
||||
Severity: c.Severity, Summary: c.Summary,
|
||||
return Record{Key: c.Key, Kind: c.Kind, Subject: c.SubjectWords(), Severity: c.Severity, Summary: c.Summary,
|
||||
Headline: c.Headline, Explanation: c.Explanation, Resolved: c.Resolved, Needs: c.Needs, Actions: c.Actions,
|
||||
Origin: "condition",
|
||||
More: "conditions show " + c.Key, Raised: c.Raised, SilencedTill: c.SilencedTill, SilencedWhy: c.SilencedWhy}
|
||||
@@ -241,13 +232,6 @@ func (r *Record) takeWords(from Record) {
|
||||
}
|
||||
}
|
||||
|
||||
// takeSubject keeps the subject as the controller says it now, when it says one.
|
||||
func (r *Record) takeSubject(from Record) {
|
||||
if from.Scope != "" || from.SubjectID != "" || from.Machine != "" {
|
||||
r.Scope, r.SubjectID, r.Machine = from.Scope, from.SubjectID, from.Machine
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Record) silenced(now time.Time) bool {
|
||||
return !r.SilencedTill.IsZero() && now.Before(r.SilencedTill)
|
||||
}
|
||||
@@ -308,9 +292,6 @@ type Holder struct {
|
||||
Zone func() *time.Location
|
||||
// Tracking asks the issue tracker which issues track which conditions; nil asks it through Ask (tracking.go).
|
||||
Tracking func() (map[string][]TrackedIssue, error)
|
||||
// NeedsOnly says whether a kind is set to say only what needs the operator (the setting
|
||||
// `only-what-needs-me`, ADR 0308 §4); nil, none is.
|
||||
NeedsOnly func(kind string) bool
|
||||
|
||||
work sync.Mutex // one event, call or tick at a time
|
||||
mu sync.Mutex // what the status reads
|
||||
@@ -348,14 +329,8 @@ type Holder struct {
|
||||
lastUnlink map[string]Unlinked
|
||||
// rootWord is the controller's last word on whether an agent can become root, by the machines asked.
|
||||
rootWord map[string]rootWord
|
||||
// passes is, by kind, the restart pass this router last made in full, kept in its state (restart.go); lost
|
||||
// the loss of answers it has yet to act on, by kind.
|
||||
passes map[string]Pass
|
||||
lost map[string]time.Time
|
||||
// walks are the walks open now, by id, each with what it holds (walks.go, ADR 0308).
|
||||
walks map[string]*Walk
|
||||
// ended are the walks whose end was heard, by id: what a start heard after its end is matched against.
|
||||
ended map[string]*Walk
|
||||
// shows is, by kind, the AnswersLostAt this router last acted on in full (restart.go).
|
||||
shows map[string]time.Time
|
||||
// track is the issue tracker's last word on which issues track which conditions (tracking.go).
|
||||
track trackWord
|
||||
}
|
||||
@@ -373,10 +348,7 @@ func (h *Holder) init() {
|
||||
h.chanOK = map[string]time.Time{}
|
||||
h.saidOnce = map[string]time.Time{}
|
||||
h.heard = map[string]int{}
|
||||
h.passes = map[string]Pass{}
|
||||
h.lost = map[string]time.Time{}
|
||||
h.walks = map[string]*Walk{}
|
||||
h.ended = map[string]*Walk{}
|
||||
h.shows = map[string]time.Time{}
|
||||
}
|
||||
if h.Now == nil {
|
||||
h.Now = time.Now
|
||||
@@ -410,20 +382,11 @@ func (h *Holder) Load() error {
|
||||
r.History = true
|
||||
}
|
||||
}
|
||||
if r.Owed != "" && r.Cleared.IsZero() {
|
||||
// Held from the operator on purpose and not told yet: said on its own, as the next tick tries what is
|
||||
// unsent (issue 538). What the operator could have heard is history; this they could not.
|
||||
r.Pending, r.History = r.Owed, false
|
||||
}
|
||||
h.open[r.Key] = &r
|
||||
}
|
||||
if sent, err := h.Store.Recent(); err == nil {
|
||||
h.recent = sent
|
||||
}
|
||||
if err := h.loadWalks(); err != nil {
|
||||
h.storeErr = err.Error()
|
||||
return err
|
||||
}
|
||||
h.mu.Unlock()
|
||||
err = h.loadAsks()
|
||||
h.mu.Lock()
|
||||
@@ -522,7 +485,6 @@ func (h *Holder) raised(rec Record, quiet bool) {
|
||||
wasQuiet := needsNothing(old)
|
||||
wasSilenced := old.silenced(now)
|
||||
old.Summary, old.Subject, old.Kind = rec.Summary, rec.Subject, rec.Kind
|
||||
old.takeSubject(rec)
|
||||
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
||||
old.takeWords(rec)
|
||||
if rec.Severity != "" {
|
||||
@@ -622,7 +584,6 @@ func (h *Holder) changed(rec Record, quiet bool) {
|
||||
wasQuiet := needsNothing(old)
|
||||
wasSilenced := old.silenced(h.Now())
|
||||
old.Summary, old.Subject = rec.Summary, rec.Subject
|
||||
old.takeSubject(rec)
|
||||
old.SilencedTill, old.SilencedWhy = rec.SilencedTill, rec.SilencedWhy
|
||||
old.takeWords(rec)
|
||||
if rec.Severity != "" {
|
||||
@@ -749,8 +710,6 @@ func (h *Holder) Tick() {
|
||||
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
||||
h.sweepAsks(now)
|
||||
h.breakGlass()
|
||||
h.boundWalks(now)
|
||||
h.duePasses(now)
|
||||
for _, r := range recs {
|
||||
switch {
|
||||
case !r.Cleared.IsZero():
|
||||
@@ -816,7 +775,6 @@ func (h *Holder) Flush() {
|
||||
h.init()
|
||||
now := h.Now()
|
||||
h.mu.Unlock()
|
||||
h.duePasses(now)
|
||||
h.flush(now, false)
|
||||
}
|
||||
|
||||
|
||||
@@ -66,9 +66,6 @@ func (f *fakeChannel) Edit(id string, m Message) error {
|
||||
type memStore struct {
|
||||
recs map[string]Record
|
||||
recent []Sent
|
||||
// walks and passes are what ADR 0308 keeps: the open walks and the restart passes.
|
||||
walks map[string]Walk
|
||||
passes map[string]Pass
|
||||
// puts counts the writes, so a write of bytes identical to what is stored is still seen (issue 422's review).
|
||||
puts int
|
||||
}
|
||||
@@ -90,24 +87,7 @@ func (m *memStore) All() ([]Record, error) {
|
||||
return out, nil
|
||||
}
|
||||
func (m *memStore) PutRecent(s []Sent) error { m.recent = s; return nil }
|
||||
func (m *memStore) PutWalk(w Walk) error {
|
||||
if m.walks == nil {
|
||||
m.walks = map[string]Walk{}
|
||||
}
|
||||
m.walks[w.ID] = w
|
||||
return nil
|
||||
}
|
||||
func (m *memStore) DeleteWalk(id string) error { delete(m.walks, id); return nil }
|
||||
func (m *memStore) Walks() ([]Walk, error) {
|
||||
var out []Walk
|
||||
for _, w := range m.walks {
|
||||
out = append(out, w)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
func (m *memStore) PutPasses(p map[string]Pass) error { m.passes = p; return nil }
|
||||
func (m *memStore) Passes() (map[string]Pass, error) { return m.passes, nil }
|
||||
func (m *memStore) Recent() ([]Sent, error) { return m.recent, nil }
|
||||
func (m *memStore) Recent() ([]Sent, error) { return m.recent, nil }
|
||||
|
||||
type clock struct{ t time.Time }
|
||||
|
||||
|
||||
@@ -29,9 +29,6 @@ type Settings struct {
|
||||
// TimeZone is the operator's, an IANA name such as Europe/Brussels: every time a message says is said
|
||||
// in it. Empty is this machine's own zone.
|
||||
TimeZone string `json:"time-zone"`
|
||||
// OnlyWhatNeedsMe are the channel kinds set to say only what needs the operator: a walk that delivered
|
||||
// with nothing held and nothing open is not told there at all (novox/hq ADR 0308 §4).
|
||||
OnlyWhatNeedsMe []string `json:"only-what-needs-me"`
|
||||
}
|
||||
|
||||
// zoneOf is the location a time-zone setting names, or this machine's own when it names none or one
|
||||
@@ -63,19 +60,6 @@ func readSettings(path string) (Settings, error) {
|
||||
if tz, ok := loose["time-zone"].(string); ok {
|
||||
s.TimeZone = strings.TrimSpace(tz)
|
||||
}
|
||||
switch v := loose["only-what-needs-me"].(type) {
|
||||
case nil:
|
||||
case []any:
|
||||
for _, k := range v {
|
||||
kind, ok := k.(string)
|
||||
if !ok || strings.TrimSpace(kind) == "" {
|
||||
return s, fmt.Errorf("only-what-needs-me lists something that is not a channel kind: %v", k)
|
||||
}
|
||||
s.OnlyWhatNeedsMe = append(s.OnlyWhatNeedsMe, strings.TrimSpace(kind))
|
||||
}
|
||||
default:
|
||||
return s, fmt.Errorf("only-what-needs-me is not a list of channel kinds")
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
@@ -125,50 +109,6 @@ func (stateStore) All() ([]Record, error) {
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// The walks open now, one key each in the state `walks`, and the restart passes under one key of `sent`
|
||||
// (novox/hq ADR 0308): a router that starts again holds, tells and passes as if it had not.
|
||||
func (stateStore) PutWalk(w Walk) error {
|
||||
_, err := stdio.State("walks").Put(kvKey(w.ID), w)
|
||||
return err
|
||||
}
|
||||
func (stateStore) DeleteWalk(id string) error { return stdio.State("walks").Delete(kvKey(id)) }
|
||||
func (stateStore) Walks() ([]Walk, error) {
|
||||
keys, err := stdio.State("walks").Keys()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []Walk
|
||||
for _, k := range keys {
|
||||
e, err := stdio.State("walks").Get(k)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if e == nil {
|
||||
continue
|
||||
}
|
||||
var w Walk
|
||||
if err := json.Unmarshal(e.Value, &w); err != nil {
|
||||
logf("[messenger] the walk kept as %s cannot be read (%v); left as it is", k, err)
|
||||
continue
|
||||
}
|
||||
out = append(out, w)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
func (stateStore) PutPasses(p map[string]Pass) error {
|
||||
_, err := stdio.State("sent").Put("passes", p)
|
||||
return err
|
||||
}
|
||||
func (stateStore) Passes() (map[string]Pass, error) {
|
||||
e, err := stdio.State("sent").Get("passes")
|
||||
if err != nil || e == nil {
|
||||
return nil, err
|
||||
}
|
||||
var p map[string]Pass
|
||||
return p, json.Unmarshal(e.Value, &p)
|
||||
}
|
||||
|
||||
func (stateStore) PutRecent(s []Sent) error {
|
||||
_, err := stdio.State("sent").Put("recent", s)
|
||||
return err
|
||||
@@ -331,14 +271,6 @@ func main() {
|
||||
return loc
|
||||
},
|
||||
Emit: func(event string, body any) error { return stdio.Emit(event, body) },
|
||||
NeedsOnly: func(kind string) bool {
|
||||
for _, k := range settings().OnlyWhatNeedsMe {
|
||||
if k == kind {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
},
|
||||
}
|
||||
bench.To = func(kind string) string {
|
||||
if ids := h.identitiesOf(kind); len(ids) > 0 {
|
||||
@@ -394,18 +326,6 @@ func run(h *Holder, l *listening) {
|
||||
}
|
||||
}()
|
||||
handle := func(e stdio.Envelope) error {
|
||||
if e.Key == ControllerSeat+"."+EventPlanMoved || e.Key == ControllerSeat+"."+EventWalk {
|
||||
// A walk's start or end (novox/hq ADR 0308 §1); any other move of a plan is not read here.
|
||||
ev, isWalk, err := DecodeWalk(e.Body)
|
||||
if err != nil {
|
||||
h.Unreadable(e.Key, err)
|
||||
return nil
|
||||
}
|
||||
if isWalk {
|
||||
h.Walked(ev)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
event, ok := eventOf(e.Key)
|
||||
if !ok {
|
||||
return nil
|
||||
|
||||
@@ -81,7 +81,7 @@ func TestItDeclaresTheSeatsOfAnAskAndHoldsTheOperatorChannel(t *testing.T) {
|
||||
}
|
||||
if !reflect.DeepEqual(m.Consumes, []string{
|
||||
ControllerSeat + "." + EventRaised, ControllerSeat + "." + EventChanged, ControllerSeat + "." + EventCleared,
|
||||
ControllerSeat + "." + EventPlanMoved, "intake.choice", "intake.link", "intake.failed", "intake.standing"}) {
|
||||
"intake.choice", "intake.link", "intake.failed", "intake.standing"}) {
|
||||
t.Fatalf("consumes %v", m.Consumes)
|
||||
}
|
||||
// It performs nothing: the verbs it calls read the open conditions, whether root is free, and which issues
|
||||
@@ -100,7 +100,7 @@ func TestItDeclaresTheSeatsOfAnAskAndHoldsTheOperatorChannel(t *testing.T) {
|
||||
}
|
||||
state = append(state, name)
|
||||
}
|
||||
if !reflect.DeepEqual(state, []string{"open", "sent", "asks", "identities", "walks"}) {
|
||||
if !reflect.DeepEqual(state, []string{"open", "sent", "asks", "identities"}) {
|
||||
t.Fatalf("state %v", state)
|
||||
}
|
||||
// It holds no channel's secret: only its own bus account, read by its own account alone (ADR 0259 §8).
|
||||
|
||||
@@ -18,9 +18,6 @@ type item struct {
|
||||
Severity string
|
||||
At time.Time
|
||||
Msg *Message
|
||||
// Carries are the records a walk's end or bound message tells (ADR 0308 §4): each is marked told on a
|
||||
// channel only once that message was sent there (issue 538 A), never when it is held to be sent.
|
||||
Carries []string
|
||||
}
|
||||
|
||||
func (i item) urgent() bool { return i.Severity == Urgent && i.What != "cleared" }
|
||||
@@ -35,10 +32,6 @@ func (h *Holder) deliver(r *Record, what string) {
|
||||
h.record(Sent{At: h.Now(), Channel: "-", Key: r.Key, What: what, Outcome: "quiet: nothing for the operator to do"})
|
||||
return
|
||||
}
|
||||
if h.hold(r, what) {
|
||||
// An open walk touches it: told when the walk ends or passes its bound (ADR 0308 §2).
|
||||
return
|
||||
}
|
||||
var to []Channel
|
||||
if kinds := h.askKinds(r); len(kinds) > 0 {
|
||||
// An ask goes to every channel that carries one of its answers (novox/hq ADR 0259): the desk for an
|
||||
@@ -125,8 +118,6 @@ func (h *Holder) queuedWhat(key string) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// queued says whether something is held to be said of a record: its own message, or a walk's message that
|
||||
// carries it.
|
||||
func (h *Holder) queued(key string) bool {
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
@@ -135,11 +126,6 @@ func (h *Holder) queued(key string) bool {
|
||||
if it.Msg == nil && it.Key == key {
|
||||
return true
|
||||
}
|
||||
for _, k := range it.Carries {
|
||||
if k == key {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
@@ -149,9 +135,6 @@ func (h *Holder) queued(key string) bool {
|
||||
// edit fails, or that said it inside a digest, is given a new message instead — held like any other.
|
||||
// silentOnly: only where an edit notifies nobody; elsewhere nothing.
|
||||
func (h *Holder) edit(r *Record, what string, silentOnly bool) {
|
||||
if h.hold(r, what) {
|
||||
return // an open walk touches it (ADR 0308 §2); a clearing or a silence is never held
|
||||
}
|
||||
h.mu.Lock()
|
||||
sent := map[string]string{}
|
||||
for k, v := range r.Sent {
|
||||
@@ -242,23 +225,12 @@ func (h *Holder) due(ch Channel, list []item, now time.Time) bool {
|
||||
if now.Sub(oldest) < BurstWindow {
|
||||
return false
|
||||
}
|
||||
if ch == h.Desk && !lastWarn.IsZero() && now.Sub(lastWarn) < DesktopWarningEvery && !carriesWalk(list) {
|
||||
// A walk's message is what the rhythm would have spared the operator many of: not held to it.
|
||||
if ch == h.Desk && !lastWarn.IsZero() && now.Sub(lastWarn) < DesktopWarningEvery {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// carriesWalk says whether held items carry a walk's end or bound message.
|
||||
func carriesWalk(list []item) bool {
|
||||
for _, it := range list {
|
||||
if it.What == WalkEndWhat {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// line is what an item says, resolved against the record now: nil when it is no longer to be said.
|
||||
func (h *Holder) resolve(it item, now time.Time, kind string) (*Record, *Message) {
|
||||
if it.Msg != nil {
|
||||
@@ -338,9 +310,7 @@ func (h *Holder) flushChannel(ch Channel, now time.Time, force bool) {
|
||||
for _, e := range entries {
|
||||
// A message said again about a record shown here goes alone too: it replaces that record's message,
|
||||
// so a reminder is never a digest line beside the message it follows (issue 373).
|
||||
// A walk's end or bound message goes alone too: its lines are what it says (ADR 0308 §4).
|
||||
if len(e.msg.Actions) > 1 || (len(e.msg.Actions) > 0 && e.r != nil && e.r.Ask != "") || h.replacing(ch, e) != "" ||
|
||||
e.it.What == WalkEndWhat {
|
||||
if len(e.msg.Actions) > 1 || (len(e.msg.Actions) > 0 && e.r != nil && e.r.Ask != "") || h.replacing(ch, e) != "" {
|
||||
alone = append(alone, e)
|
||||
} else {
|
||||
rest = append(rest, e)
|
||||
@@ -430,7 +400,6 @@ func (h *Holder) sendEntries(ch Channel, entries []entry, now time.Time) bool {
|
||||
}
|
||||
r.Pending, r.Folded, r.History = "", false, false
|
||||
if e.it.What != "cleared" {
|
||||
r.Owed = ""
|
||||
if r.Sent == nil {
|
||||
r.Sent = map[string]string{}
|
||||
}
|
||||
@@ -446,39 +415,7 @@ func (h *Holder) sendEntries(ch Channel, entries []entry, now time.Time) bool {
|
||||
r.LastSaid = now
|
||||
}
|
||||
}
|
||||
type carried struct {
|
||||
r *Record
|
||||
walk string
|
||||
}
|
||||
var told []carried
|
||||
for _, e := range entries {
|
||||
for _, key := range e.it.Carries {
|
||||
r := h.open[key]
|
||||
if r == nil {
|
||||
continue
|
||||
}
|
||||
if r.Sent == nil {
|
||||
r.Sent = map[string]string{}
|
||||
}
|
||||
if _, has := r.Sent[name]; !has {
|
||||
r.Sent[name] = ""
|
||||
}
|
||||
if r.FirstSent.IsZero() {
|
||||
r.FirstSent = now
|
||||
}
|
||||
r.LastSaid, r.Pending, r.History, r.Owed = now, "", false, ""
|
||||
if !r.Cleared.IsZero() && !r.ClearUnsaid {
|
||||
// It cleared while the message saying it open waited: its clearing is said too.
|
||||
r.ClearUnsaid = true
|
||||
}
|
||||
told = append(told, carried{r, strings.TrimPrefix(e.it.Key, "walk.")})
|
||||
}
|
||||
}
|
||||
h.mu.Unlock()
|
||||
for _, c := range told {
|
||||
h.record(Sent{At: now, Channel: name, Key: c.r.Key, What: "released", Outcome: "told in the message of walk " + c.walk})
|
||||
h.persist(c.r)
|
||||
}
|
||||
key, what := "(digest)", fmt.Sprintf("digest of %d", len(entries))
|
||||
if len(entries) == 1 {
|
||||
key, what = entries[0].it.Key, entries[0].it.What
|
||||
|
||||
@@ -18,22 +18,6 @@ package main
|
||||
// Whether a record is said again is its own: last said before the answers were lost. So a router that started
|
||||
// after the loss still says it, and a standing that repeats the time says nothing more (the record was said
|
||||
// since). A kind that cannot send waits: what is open there is said once it can.
|
||||
//
|
||||
// **A restart says nothing it has said** (novox/hq ADR 0308 §6, issue 531). On 2026-10-12 every delivery of the
|
||||
// tool runner made this pass twice, a second apart, each time editing the four silenced messages on the desk,
|
||||
// which showed them again — one as "A mesh message was withheld". Three causes, each closed here:
|
||||
//
|
||||
// - **Twice:** the desk's standing says the latest start of any machine's notifier, and the notifiers of two
|
||||
// machines started a second apart, so two standings with two times came, and each was acted on at once.
|
||||
// Now a loss is acted on once it stands for LostSettle: the standings of one restart are one pass.
|
||||
// - **Again after the router's own restart:** which loss was acted on was kept in memory, so a router that
|
||||
// started again acted on the standing it had already acted on. Now it is kept in the module's state
|
||||
// (`sent`, key `passes`), read back on start.
|
||||
// - **What it said:** a silenced message was edited to its silenced words, and a message withheld for its
|
||||
// content was said again as withheld. Now the pass says again only what is open, asked there and never
|
||||
// answered, whose words may leave; a silenced, a withheld or an answered message is left as it is. Being
|
||||
// held by a walk is no reason: a message with an open ask is never held (ADR 0308 §3), even one a walk took
|
||||
// before its ask came, so its answers are made to work again like any other's.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
@@ -41,17 +25,7 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// LostSettle: a loss of answers is acted on once no later loss came for this long — the standings of one
|
||||
// restart, a notifier on each machine, are one pass.
|
||||
const LostSettle = 10 * time.Second
|
||||
|
||||
// Pass is the restart pass made in full on a kind: the loss it answered and when it was made.
|
||||
type Pass struct {
|
||||
Lost time.Time `json:"lost"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
|
||||
// Stand takes what a kind said of itself, and notes answers lost there since this router last acted.
|
||||
// Stand takes what a kind said of itself, and acts on answers lost there since this router last acted.
|
||||
func (h *Holder) Stand(kind string, s Standing) {
|
||||
if h.Bench != nil {
|
||||
h.Bench.Stand(kind, s)
|
||||
@@ -59,73 +33,28 @@ func (h *Holder) Stand(kind string, s Standing) {
|
||||
if s.AnswersLostAt.IsZero() {
|
||||
return
|
||||
}
|
||||
h.work.Lock()
|
||||
defer h.work.Unlock()
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
h.init()
|
||||
if !s.AnswersLostAt.After(h.passes[kind].Lost) || !s.AnswersLostAt.After(h.lost[kind]) {
|
||||
return // acted on, or noted already
|
||||
}
|
||||
h.lost[kind] = s.AnswersLostAt
|
||||
}
|
||||
|
||||
// duePasses makes the pass of each kind whose loss of answers stood for LostSettle, once per loss.
|
||||
func (h *Holder) duePasses(now time.Time) {
|
||||
h.mu.Lock()
|
||||
due := map[string]time.Time{}
|
||||
for kind, lost := range h.lost {
|
||||
if now.Sub(lost) >= LostSettle {
|
||||
due[kind] = lost
|
||||
}
|
||||
}
|
||||
done := h.shows[kind].Equal(s.AnswersLostAt)
|
||||
h.mu.Unlock()
|
||||
for kind, lost := range due {
|
||||
ch := h.channelOf(kind)
|
||||
if ch == nil {
|
||||
continue // the controller's word on the kinds is not read yet: acted on later
|
||||
}
|
||||
if err := ch.Ready(); err != nil {
|
||||
h.mu.Lock()
|
||||
said := h.saidOnce["lost."+kind]
|
||||
h.mu.Unlock()
|
||||
if said.IsZero() || now.Sub(said) >= 10*time.Minute {
|
||||
h.Logf("[messenger] %s lost its answers at %s and cannot send (%v): what is open there is said again once it can",
|
||||
kind, lost.UTC().Format(time.RFC3339), err)
|
||||
h.mu.Lock()
|
||||
h.saidOnce["lost."+kind] = now
|
||||
h.mu.Unlock()
|
||||
}
|
||||
continue
|
||||
}
|
||||
if done {
|
||||
return
|
||||
}
|
||||
ch := h.channelOf(kind)
|
||||
if ch == nil {
|
||||
return // the controller's word on the kinds is not read yet: acted on at a later standing
|
||||
}
|
||||
if err := ch.Ready(); err != nil {
|
||||
h.Logf("[messenger] %s lost its answers at %s and cannot send (%v): what is open there is said again once it can",
|
||||
kind, s.AnswersLostAt.UTC().Format(time.RFC3339), err)
|
||||
return
|
||||
}
|
||||
if h.reoffer(ch, s.AnswersLostAt) {
|
||||
h.mu.Lock()
|
||||
tried := h.saidOnce["lost-try."+kind]
|
||||
h.shows[kind] = s.AnswersLostAt
|
||||
h.mu.Unlock()
|
||||
if !tried.IsZero() && now.Sub(tried) < RetryEvery {
|
||||
continue
|
||||
}
|
||||
if !h.reoffer(ch, lost) {
|
||||
h.mu.Lock()
|
||||
h.saidOnce["lost-try."+kind] = now // what failed is tried again in a minute
|
||||
h.mu.Unlock()
|
||||
continue
|
||||
}
|
||||
h.mu.Lock()
|
||||
delete(h.saidOnce, "lost-try."+kind)
|
||||
h.mu.Unlock()
|
||||
h.mu.Lock()
|
||||
h.passes[kind] = Pass{Lost: lost, At: now}
|
||||
if h.lost[kind].Equal(lost) {
|
||||
delete(h.lost, kind)
|
||||
}
|
||||
passes := map[string]Pass{}
|
||||
for k, p := range h.passes {
|
||||
passes[k] = p
|
||||
}
|
||||
h.mu.Unlock()
|
||||
if k, ok := h.Store.(walkKeeper); ok {
|
||||
if err := k.PutPasses(passes); err != nil {
|
||||
h.noteStore(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -139,9 +68,9 @@ func (h *Holder) channelOf(kind string) Channel {
|
||||
return nil
|
||||
}
|
||||
|
||||
// reoffer says again on one channel, whose answers were lost at `lost`, what is still open, asked there and
|
||||
// never answered, and leaves everything else as it is: what ended, what is silenced, what was withheld for its
|
||||
// content, and what was answered (ADR 0308 §6). It answers whether every message to be said again was.
|
||||
// reoffer says again on one channel, whose answers were lost at `lost`, what is still open and asked there,
|
||||
// edits what is silenced to say so, and leaves what ended alone. It answers whether every message to be
|
||||
// said again was.
|
||||
func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
||||
kind := ch.Name()
|
||||
rp, canReplace := ch.(replacer)
|
||||
@@ -153,14 +82,13 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
||||
}
|
||||
h.mu.Unlock()
|
||||
sort.Slice(recs, func(i, j int) bool { return recs[i].Raised.Before(recs[j].Raised) })
|
||||
again, left, ended, failed := 0, 0, 0, 0
|
||||
again, silenced, ended, failed := 0, 0, 0, 0
|
||||
for _, r := range recs {
|
||||
h.mu.Lock()
|
||||
id, shown := r.Sent[kind]
|
||||
open := r.Cleared.IsZero()
|
||||
before := lastSaid(r).Before(lost)
|
||||
reminded := r.Reminded
|
||||
withheld, answered := r.Refused != "", r.Answered != ""
|
||||
h.mu.Unlock()
|
||||
if !shown || id == "" {
|
||||
continue // never a message of its own there: nothing to say again under
|
||||
@@ -168,8 +96,13 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
||||
switch {
|
||||
case !open:
|
||||
ended++ // its end is said where its message is; nothing comes back
|
||||
case r.silenced(now), withheld, answered:
|
||||
left++ // a silence, a withheld message or an answer is never said again for a restart
|
||||
case r.silenced(now):
|
||||
h.mu.Lock()
|
||||
r.SilenceSaid = true
|
||||
h.mu.Unlock()
|
||||
h.editOn(ch, r, "silenced")
|
||||
h.persist(r)
|
||||
silenced++
|
||||
case h.askKinds(r)[kind] && before && canReplace:
|
||||
what := "raised"
|
||||
if reminded {
|
||||
@@ -192,10 +125,28 @@ func (h *Holder) reoffer(ch Channel, lost time.Time) bool {
|
||||
}
|
||||
}
|
||||
h.record(Sent{At: now, Channel: kind, Key: "(answers lost)", What: "at " + lost.UTC().Format(time.RFC3339),
|
||||
Outcome: fmt.Sprintf("%d open and asked said again, %d silenced, withheld or answered left as they are, "+
|
||||
"%d ended left ended, %d failed", again, left, ended, failed)})
|
||||
h.Logf("[messenger] %s lost its answers at %s: %d open and asked said again in place, %d silenced, withheld or "+
|
||||
"answered left as they are, %d ended left ended, %d failed; nothing shown from the channel's memory", kind,
|
||||
lost.UTC().Format(time.RFC3339), again, left, ended, failed)
|
||||
Outcome: fmt.Sprintf("%d open and asked said again, %d silenced said so, %d ended left ended, %d failed",
|
||||
again, silenced, ended, failed)})
|
||||
h.Logf("[messenger] %s lost its answers at %s: %d open and asked said again in place, %d silenced said so, %d ended "+
|
||||
"left ended, %d failed; nothing shown from the channel's memory", kind, lost.UTC().Format(time.RFC3339), again,
|
||||
silenced, ended, failed)
|
||||
return failed == 0
|
||||
}
|
||||
|
||||
// editOn edits a record's message on one channel, where it is shown as a message of its own.
|
||||
func (h *Holder) editOn(ch Channel, r *Record, what string) {
|
||||
kind := ch.Name()
|
||||
h.mu.Lock()
|
||||
id := r.Sent[kind]
|
||||
h.mu.Unlock()
|
||||
if id == "" || !ch.CanEdit() {
|
||||
return
|
||||
}
|
||||
err := ch.Edit(id, h.sayOn(r, what, kind))
|
||||
h.noteChannel(kind, err)
|
||||
if err != nil {
|
||||
h.Logf("[messenger] could not edit the message for %s on %s (%v)", r.Key, kind, err)
|
||||
return
|
||||
}
|
||||
h.record(Sent{At: h.Now(), Channel: kind, Key: r.Key, What: what, Outcome: "edited"})
|
||||
}
|
||||
|
||||
@@ -12,8 +12,8 @@ import (
|
||||
// resolved, silenced and cancelled ones among them. Now the desk says in its standing when the answers were
|
||||
// lost, and this router says again only what is still open and unanswered: with the desk showing one open
|
||||
// message with an answer, one silenced, one open asking nothing and one resolved, one is said again under its
|
||||
// handle (in place of its message), with its answers; nothing is said for the others — the silenced one
|
||||
// included (novox/hq ADR 0308 §6, issue 531: its edit showed it on the desk again). A standing that repeats the time says nothing more; its
|
||||
// handle (in place of its message), with its answers; one is edited to the silenced words, with no answers
|
||||
// and no urgency; nothing is said for the others. A standing that repeats the time says nothing more; its
|
||||
// ticket is one-time still, and the channel that reaches the operator away hears nothing of it.
|
||||
func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
||||
h, tg, dt, c, _, w := newRouter(t)
|
||||
@@ -77,7 +77,8 @@ func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
||||
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
||||
t.Fatalf("the clearing was not said: %+v", e)
|
||||
}
|
||||
delete(dt.edits, "g14=2") // so an edit said again would be seen as such
|
||||
silencedBefore := dt.edits["g14=2"]
|
||||
delete(dt.edits, "g14=2") // so the edit said again is seen as such
|
||||
sends, edits, replaced, tgSaid := len(dt.sends), len(dt.edits), len(dt.replaced), saidOn(tg)
|
||||
// The desk's notifier started again: its standing says when the answers were lost.
|
||||
c.pass(time.Minute)
|
||||
@@ -94,11 +95,12 @@ func TestLostAnswersGetOnlyWhatIsOpenAndUnansweredSaidAgain(t *testing.T) {
|
||||
if ticketFor(again, "Silence for a week") == "" || !strings.Contains(again.Title, headlineOf(open.Key)) {
|
||||
t.Fatalf("said again without its answer or its words: %+v", again)
|
||||
}
|
||||
if e, said := dt.edits["g14=2"]; said {
|
||||
t.Fatalf("the silenced one was said again for the restart: %+v", e)
|
||||
if e, said := dt.edits["g14=2"]; !said || !strings.HasPrefix(e.Title, "Silenced until") || len(e.Actions) != 0 || e.Urgent ||
|
||||
!e.Quiet || e.Title != silencedBefore.Title {
|
||||
t.Fatalf("the silenced one was not edited to the silenced words again: %+v (said %v)", e, said)
|
||||
}
|
||||
if len(dt.edits) != edits {
|
||||
t.Fatalf("an edit reached the desk for the restart: %+v", dt.edits)
|
||||
if len(dt.edits) != edits+1 {
|
||||
t.Fatalf("an edit reached a message that is not silenced: %+v", dt.edits)
|
||||
}
|
||||
if e := dt.edits["g14=4"]; !strings.HasPrefix(e.Title, "Resolved: ") {
|
||||
t.Fatalf("the resolved one was changed: %+v", e)
|
||||
@@ -177,13 +179,8 @@ func TestLostAnswersAreJudgedPerMessageAndWaitForTheDesk(t *testing.T) {
|
||||
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
||||
t.Fatalf("once the desk can send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||
}
|
||||
// The router started again, reading back the pass it made from its state: nothing.
|
||||
h.mu.Lock()
|
||||
h.passes, h.lost = map[string]Pass{}, map[string]time.Time{}
|
||||
if err := h.loadWalks(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.mu.Unlock()
|
||||
// The router started again, remembering no standing it acted on: the message was said since, so nothing.
|
||||
h.shows = map[string]time.Time{}
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.replaced) != replaced+1 {
|
||||
@@ -194,12 +191,12 @@ func TestLostAnswersAreJudgedPerMessageAndWaitForTheDesk(t *testing.T) {
|
||||
lost = c.now()
|
||||
dt.fail = errorf("no session answers")
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
||||
runFor(h, c, 30*time.Second)
|
||||
dt.fail = nil
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.replaced) != replaced+1 || saidOn(tg) != tgSaid {
|
||||
t.Fatalf("after a failed send: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||
}
|
||||
runFor(h, c, time.Minute)
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: lost})
|
||||
if len(dt.replaced) != replaced+2 || saidOn(tg) != tgSaid {
|
||||
t.Fatalf("the failed send was not tried again on the desk: desk %+v, telegram %+v", dt.replaced[replaced:], tg.sends)
|
||||
}
|
||||
|
||||
@@ -113,9 +113,6 @@ func (h *Holder) Status(listening string) Status {
|
||||
"deduplicated by the condition's key; reminded once after 1 h (urgent) or 12 h (warning); cleared by editing the first message",
|
||||
"at most 20 messages an hour per channel, a last line of defence: reached, it is said once and the rest held for one digest",
|
||||
"refused: anything carrying an address, a path or a secret; channel-refused is sent with the words withheld",
|
||||
"while a delivery walks, a message about a machine or module it touches is held, and the walk's end is one message: its outcome, what it reached, how long, what is still open; past the walk's bound (twice its largest delivery budget, 15 min with none) what is held and open is said (novox/hq ADR 0308)",
|
||||
"an ask, a secret given or replaced, and a walk's own failure are never held",
|
||||
"a channel that lost its answers is said again only what is open, asked there and never answered — never a silenced or withheld message — once per restart, and the pass is kept in this module's state",
|
||||
"a silenced condition's message says it is silenced, until when and why, and nothing more is said of it while the silence lasts; a silence is the controller's, asked of the operator or set through the mesh",
|
||||
}
|
||||
anyCan := false
|
||||
@@ -155,11 +152,9 @@ type OpenMessage struct {
|
||||
Reminded bool `json:"reminded"`
|
||||
Unsent string `json:"unsent,omitempty"`
|
||||
Held bool `json:"held,omitempty"`
|
||||
// HeldBy are the walks holding it until they end or pass their bound (novox/hq ADR 0308).
|
||||
HeldBy []string `json:"held_by_walks,omitempty"`
|
||||
History bool `json:"never_said,omitempty"`
|
||||
Refused string `json:"refused,omitempty"`
|
||||
Count int `json:"times_opened"`
|
||||
History bool `json:"never_said,omitempty"`
|
||||
Refused string `json:"refused,omitempty"`
|
||||
Count int `json:"times_opened"`
|
||||
}
|
||||
|
||||
// Open is what is unresolved now, urgent first, oldest first.
|
||||
@@ -185,7 +180,6 @@ func (h *Holder) Open() []OpenMessage {
|
||||
o.SentOn = append(o.SentOn, ch)
|
||||
}
|
||||
sort.Strings(o.SentOn)
|
||||
o.HeldBy = h.heldBy(r.Key, now)
|
||||
if r.silenced(now) {
|
||||
o.Silenced = r.SilencedTill.UTC().Format(time.RFC3339)
|
||||
}
|
||||
|
||||
@@ -103,9 +103,7 @@ func (h *Holder) Sync(open []Condition, asked time.Time) SyncResult {
|
||||
switch {
|
||||
case told || silenced || h.queued(r.Key):
|
||||
case r.Severity == Urgent:
|
||||
if !h.hold(r, "raised") { // an open walk touches it: told at its end or bound (ADR 0308 §2)
|
||||
urgent = append(urgent, r)
|
||||
}
|
||||
urgent = append(urgent, r)
|
||||
case !known && now.Sub(r.Raised) <= FreshFor:
|
||||
fresh = append(fresh, r)
|
||||
case wasQuiet && !needsNothing(r):
|
||||
|
||||
@@ -1,906 +0,0 @@
|
||||
package main
|
||||
|
||||
// While a delivery walks, what it touches is held from the operator and told once, with the walk's result,
|
||||
// when it ends (novox/hq ADR 0308, issue 532).
|
||||
//
|
||||
// **What this reads.** The controller says each walk's start and end on its `plan-moved` event (the plan
|
||||
// whole, ADR 0282 §6), and on those two moments the body carries beside the plan (ADR 0308 §1):
|
||||
//
|
||||
// - `walk`: the plan's id (`id` is read when it is missing);
|
||||
// - `phase`: `started` or `ended`; a plan-moved without it is a move inside a walk and is not read here;
|
||||
// - `touches`: `[{machine, modules}]`, every module the walk moves on every machine it sends it to, and every
|
||||
// module a moved tool runner or node-engine carries there;
|
||||
// - `outcome`: `delivered`, `failed` or `stopped`, on `ended`;
|
||||
// - `bound_seconds`: twice the largest delivery budget among its deliveries, or 900 when none has one (§5),
|
||||
// on `started`; missing, 900 is taken;
|
||||
// - `budget_class`: `core`, `leaf` or `none`;
|
||||
// - `started_at` on the start and `ended_at` on the end (else `at`, else the plan's `updated`): when it
|
||||
// happened. A start older than its bound is history and opens nothing; an end older than FreshFor with
|
||||
// nothing held closes quietly.
|
||||
//
|
||||
// A touch may name `carried` beside `modules` (what a moved tool runner or node-engine carries); the
|
||||
// controller lists those in `modules` too, which is what is read.
|
||||
//
|
||||
// An event of its own named `walk`, with the same body, is read the same way, should the controller say it
|
||||
// there instead.
|
||||
//
|
||||
// **What is held (§2).** While a walk is open and within its bound, a message whose subject is a machine the
|
||||
// walk touches, or a module it touches, is held instead of sent: kept on the walk, with when it was held and
|
||||
// what was to be said, and the channel's history records it with the outcome `held`. A message touched by
|
||||
// several open walks is kept on each, and told when the last of them ends or passes its bound.
|
||||
//
|
||||
// **What never is (§3):** an ask waiting for the operator's answer (an ask's own message, and a condition an
|
||||
// open ask is about), a secret given or replaced, a message about a machine or module no open walk touches,
|
||||
// and a walk's own failure: anything about a plan, a walk, a delivery or a build.
|
||||
//
|
||||
// **The end (§4).** One message: its outcome, the modules and machines it reached and how long it took; then
|
||||
// each held message whose condition is still open, one line each; the ones that cleared meanwhile only
|
||||
// counted. A walk that delivered with nothing held and nothing open is one line, and nothing at all on a kind
|
||||
// the operator set to say only what needs them (the setting `only-what-needs-me`).
|
||||
//
|
||||
// **The bound (§5).** At the walk's start plus its bound, what it holds that is still open is said, with a line
|
||||
// that the walk is still running and since when, and the cleared ones counted; it holds nothing more after
|
||||
// that. A walk whose end never comes is forgotten a day after it started.
|
||||
//
|
||||
// **Kept in the module's state** (the bucket `walks`, one key per walk), so a router that restarts mid-walk
|
||||
// holds and tells as if it had not.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The walk events, by their local names under the controller's seat.
|
||||
const (
|
||||
EventPlanMoved = "plan-moved"
|
||||
EventWalk = "walk"
|
||||
|
||||
WalkStarted = "started"
|
||||
WalkEnded = "ended"
|
||||
|
||||
WalkDelivered = "delivered"
|
||||
WalkFailed = "failed"
|
||||
WalkStopped = "stopped"
|
||||
|
||||
// DefaultWalkBound is a walk's bound when the controller gives none: the bound of a walk none of whose
|
||||
// deliveries has a budget (ADR 0308 §5).
|
||||
DefaultWalkBound = 15 * time.Minute
|
||||
// WalkForget: a walk whose end never came is forgotten this long after it started.
|
||||
WalkForget = 24 * time.Hour
|
||||
// WalkEndWhat is the outbox's name for a walk's end or bound message: it goes alone, never in a digest.
|
||||
WalkEndWhat = "walk-end"
|
||||
// MostNamed: an end message names at most this many modules or machines, then counts the rest.
|
||||
MostNamed = 6
|
||||
)
|
||||
|
||||
// Touch is what a walk moves on one machine.
|
||||
type Touch struct {
|
||||
Machine string `json:"machine"`
|
||||
Modules []string `json:"modules"`
|
||||
}
|
||||
|
||||
// WalkEvent is a walk's start or end as the controller says it.
|
||||
type WalkEvent struct {
|
||||
Walk string
|
||||
Phase string
|
||||
Outcome string
|
||||
Class string
|
||||
Touches []Touch
|
||||
BoundSeconds int
|
||||
At time.Time
|
||||
// StartedAt is the walk's own start, on both events (`started_at`): the bound runs from it, so a router
|
||||
// that hears a start late, or starts again, ends the hold on time.
|
||||
StartedAt time.Time
|
||||
}
|
||||
|
||||
// HeldMessage is one message a walk holds: when it was first held and what was to be said.
|
||||
type HeldMessage struct {
|
||||
At time.Time `json:"at"`
|
||||
What string `json:"what"`
|
||||
}
|
||||
|
||||
// Walk is one open walk as this router keeps it.
|
||||
type Walk struct {
|
||||
ID string `json:"id"`
|
||||
Started time.Time `json:"started"`
|
||||
BoundSeconds int `json:"bound_seconds"`
|
||||
Class string `json:"budget_class,omitempty"`
|
||||
Touches []Touch `json:"touches"`
|
||||
Held map[string]HeldMessage `json:"held,omitempty"`
|
||||
// BoundSaid: its bound passed and what it held was said; it holds nothing more.
|
||||
BoundSaid bool `json:"bound_said,omitempty"`
|
||||
// Ended: its end was heard, at this moment. Kept a day, so a start heard after its end opens nothing.
|
||||
Ended time.Time `json:"ended,omitempty"`
|
||||
// EndedSaid is the end's moment as the controller said it, by the controller's clock and never capped at
|
||||
// this router's now: a start heard after its end is compared with it on that one clock (issue 538 C).
|
||||
EndedSaid time.Time `json:"ended_said,omitempty"`
|
||||
}
|
||||
|
||||
func (w *Walk) bound() time.Duration {
|
||||
if w.BoundSeconds <= 0 {
|
||||
return DefaultWalkBound
|
||||
}
|
||||
return time.Duration(w.BoundSeconds) * time.Second
|
||||
}
|
||||
|
||||
// holding says whether the walk still holds at a moment: open, and within its bound.
|
||||
func (w *Walk) holding(now time.Time) bool {
|
||||
return !w.BoundSaid && now.Before(w.Started.Add(w.bound()))
|
||||
}
|
||||
|
||||
// touches says whether the walk touches a subject: a machine it sends to, or — when the subject names no
|
||||
// machine of its own — a module it moves anywhere.
|
||||
func (w *Walk) touches(s subject) bool {
|
||||
for _, t := range w.Touches {
|
||||
for _, m := range s.machines {
|
||||
if m != "" && m == t.Machine {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
if s.machineNamed {
|
||||
return false
|
||||
}
|
||||
for _, t := range w.Touches {
|
||||
for _, mod := range t.Modules {
|
||||
for _, m := range s.modules {
|
||||
if m != "" && m == mod {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// walkKeeper is a store that keeps the walks and the restart passes too (the module's state, ADR 0201).
|
||||
type walkKeeper interface {
|
||||
PutWalk(w Walk) error
|
||||
DeleteWalk(id string) error
|
||||
Walks() ([]Walk, error)
|
||||
PutPasses(map[string]Pass) error
|
||||
Passes() (map[string]Pass, error)
|
||||
}
|
||||
|
||||
// DecodeWalk reads a plan-moved or walk event's body: false when it says no walk's start or end.
|
||||
func DecodeWalk(body []byte) (WalkEvent, bool, error) {
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(body, &raw); err != nil || raw == nil {
|
||||
return WalkEvent{}, false, fmt.Errorf("a walk event's body is not a JSON object")
|
||||
}
|
||||
str := func(names ...string) string {
|
||||
for _, n := range names {
|
||||
var s string
|
||||
if v, ok := raw[n]; ok && json.Unmarshal(v, &s) == nil && strings.TrimSpace(s) != "" {
|
||||
return strings.TrimSpace(s)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
var ev WalkEvent
|
||||
ev.Phase = str("phase")
|
||||
if ev.Phase == "" {
|
||||
return WalkEvent{}, false, nil
|
||||
}
|
||||
ev.Walk = str("walk", "id")
|
||||
if ev.Walk == "" {
|
||||
return WalkEvent{}, false, fmt.Errorf("a walk's %s names no walk", ev.Phase)
|
||||
}
|
||||
switch ev.Phase {
|
||||
case WalkStarted, WalkEnded:
|
||||
default:
|
||||
return WalkEvent{}, false, fmt.Errorf("walk %s: phase %q is neither started nor ended", ev.Walk, ev.Phase)
|
||||
}
|
||||
ev.Outcome = str("outcome")
|
||||
if ev.Phase == WalkEnded {
|
||||
switch ev.Outcome {
|
||||
case WalkDelivered, WalkFailed, WalkStopped:
|
||||
default:
|
||||
return WalkEvent{}, false, fmt.Errorf("walk %s ended with outcome %q: neither delivered, failed nor stopped", ev.Walk, ev.Outcome)
|
||||
}
|
||||
}
|
||||
ev.Class = str("budget_class", "budget-class")
|
||||
for _, n := range []string{"bound_seconds", "bound-seconds"} {
|
||||
if v, ok := raw[n]; ok && string(v) != "null" {
|
||||
var f float64
|
||||
if json.Unmarshal(v, &f) != nil || f < 0 {
|
||||
return WalkEvent{}, false, fmt.Errorf("walk %s: %s is not a number of seconds", ev.Walk, n)
|
||||
}
|
||||
ev.BoundSeconds = int(f)
|
||||
break
|
||||
}
|
||||
}
|
||||
if v, ok := raw["touches"]; ok && string(v) != "null" {
|
||||
var list []struct {
|
||||
Machine string `json:"machine"`
|
||||
Node string `json:"node"`
|
||||
Modules []string `json:"modules"`
|
||||
}
|
||||
if json.Unmarshal(v, &list) != nil {
|
||||
return WalkEvent{}, false, fmt.Errorf("walk %s: touches is not a list of {machine, modules}", ev.Walk)
|
||||
}
|
||||
for _, t := range list {
|
||||
m := t.Machine
|
||||
if m == "" {
|
||||
m = t.Node
|
||||
}
|
||||
if m != "" {
|
||||
ev.Touches = append(ev.Touches, Touch{Machine: m, Modules: t.Modules})
|
||||
}
|
||||
}
|
||||
}
|
||||
if s := str("started_at"); s != "" {
|
||||
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||
ev.StartedAt = t
|
||||
}
|
||||
}
|
||||
when := []string{"started_at", "at", "updated"}
|
||||
if ev.Phase == WalkEnded {
|
||||
when = []string{"ended_at", "at", "updated"}
|
||||
}
|
||||
for _, n := range when {
|
||||
if s := str(n); s != "" {
|
||||
if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||
ev.At = t
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return ev, true, nil
|
||||
}
|
||||
|
||||
// mergeTouches is what two sayings of a walk's touches name together.
|
||||
func mergeTouches(a, b []Touch) []Touch {
|
||||
by := map[string]map[string]bool{}
|
||||
var order []string
|
||||
for _, list := range [][]Touch{a, b} {
|
||||
for _, t := range list {
|
||||
if by[t.Machine] == nil {
|
||||
by[t.Machine] = map[string]bool{}
|
||||
order = append(order, t.Machine)
|
||||
}
|
||||
for _, m := range t.Modules {
|
||||
by[t.Machine][m] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Strings(order)
|
||||
var out []Touch
|
||||
for _, machine := range order {
|
||||
var mods []string
|
||||
for m := range by[machine] {
|
||||
mods = append(mods, m)
|
||||
}
|
||||
sort.Strings(mods)
|
||||
out = append(out, Touch{Machine: machine, Modules: mods})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Walked takes a walk's start or end.
|
||||
func (h *Holder) Walked(ev WalkEvent) {
|
||||
h.work.Lock()
|
||||
defer h.work.Unlock()
|
||||
h.mu.Lock()
|
||||
h.init()
|
||||
now := h.Now()
|
||||
h.mu.Unlock()
|
||||
at := ev.At
|
||||
if at.IsZero() || at.After(now) {
|
||||
at = now
|
||||
}
|
||||
switch ev.Phase {
|
||||
case WalkStarted:
|
||||
w := &Walk{ID: ev.Walk, Started: at, BoundSeconds: ev.BoundSeconds, Class: ev.Class}
|
||||
h.mu.Lock()
|
||||
end := h.ended[ev.Walk]
|
||||
h.mu.Unlock()
|
||||
if end != nil && startOfEnded(ev, at, end) {
|
||||
// A short walk's two events can come in either order: this start is of the walk that ended.
|
||||
h.Logf("[messenger] walk %s: its start came after its end, which was heard at %s: nothing held", ev.Walk,
|
||||
end.Ended.UTC().Format(time.RFC3339))
|
||||
return
|
||||
}
|
||||
if now.Sub(at) >= w.bound() {
|
||||
h.Logf("[messenger] walk %s started %s ago, past its bound of %s: history, nothing held", ev.Walk,
|
||||
roughly(now.Sub(at)), w.bound())
|
||||
return
|
||||
}
|
||||
h.mu.Lock()
|
||||
if known := h.walks[ev.Walk]; known != nil {
|
||||
w = known // said again: the same walk, its touches as both sayings name them
|
||||
} else {
|
||||
w.Held = map[string]HeldMessage{}
|
||||
h.walks[ev.Walk] = w
|
||||
}
|
||||
w.Touches = mergeTouches(w.Touches, ev.Touches)
|
||||
h.mu.Unlock()
|
||||
h.keepWalk(w)
|
||||
h.record(Sent{At: now, Channel: "-", Key: "walk." + ev.Walk, What: "started: " + touchWords(w.Touches),
|
||||
Outcome: "holding what it touches for at most " + roughly(w.bound())})
|
||||
case WalkEnded:
|
||||
h.mu.Lock()
|
||||
w := h.walks[ev.Walk]
|
||||
if w != nil && !ev.StartedAt.IsZero() && w.Started.After(ev.StartedAt) {
|
||||
// The end of an earlier walk under the same id, heard after the next one started: that one stays open.
|
||||
h.mu.Unlock()
|
||||
h.Logf("[messenger] walk %s: an end of its earlier walk (started %s) came late: the open one stays", ev.Walk,
|
||||
ev.StartedAt.UTC().Format(time.RFC3339))
|
||||
return
|
||||
}
|
||||
if w != nil {
|
||||
delete(h.walks, ev.Walk)
|
||||
w.Touches = mergeTouches(w.Touches, ev.Touches)
|
||||
if !ev.StartedAt.IsZero() {
|
||||
w.Started = ev.StartedAt
|
||||
}
|
||||
}
|
||||
tomb := &Walk{ID: ev.Walk, Started: ev.StartedAt, Ended: at, EndedSaid: ev.At}
|
||||
if prev := h.ended[ev.Walk]; prev != nil && prev.Ended.After(at) {
|
||||
tomb = prev
|
||||
}
|
||||
h.ended[ev.Walk] = tomb
|
||||
h.mu.Unlock()
|
||||
h.keepWalk(tomb) // in place of the open walk: a start heard after this end opens nothing
|
||||
if w == nil {
|
||||
h.Logf("[messenger] walk %s %s, and its start was not heard here: nothing held, nothing said", ev.Walk, ev.Outcome)
|
||||
return
|
||||
}
|
||||
h.endWalk(w, ev.Outcome, at, now)
|
||||
}
|
||||
}
|
||||
|
||||
// startOfEnded says whether a start is of a walk whose end was heard already: of that walk, or of an earlier one
|
||||
// under its id. The controller's own moments are compared on its clock, never one capped at this router's now
|
||||
// against one that is not (issue 538 C: a controller's clock ahead of the router's would open a walk after its
|
||||
// end). Only when the controller said neither moment are the moments as heard here compared.
|
||||
func startOfEnded(ev WalkEvent, at time.Time, end *Walk) bool {
|
||||
said := ev.StartedAt
|
||||
if said.IsZero() {
|
||||
said = ev.At
|
||||
}
|
||||
switch {
|
||||
case !said.IsZero() && !end.Started.IsZero():
|
||||
return !said.After(end.Started)
|
||||
case !said.IsZero() && !end.EndedSaid.IsZero():
|
||||
return !said.After(end.EndedSaid)
|
||||
}
|
||||
return !at.After(end.Ended)
|
||||
}
|
||||
|
||||
// release is what a walk held that no other open walk holds or touches: each open message, oldest first, with
|
||||
// what was to be said of it, and how many cleared meanwhile. A message said meanwhile on a path that is never
|
||||
// held, or silenced meanwhile, is neither. One that another open walk touches — even one that started after it
|
||||
// was held — is held on by that walk, and told at its end or bound (ADR 0308 §2).
|
||||
//
|
||||
// A message is handed on under the rules of its first hold (issue 538 B): still open, not silenced, not said
|
||||
// since it was held, and not one ADR 0308 §3 never holds (an open ask about it, a walk's own failure).
|
||||
func (h *Holder) release(w *Walk, now time.Time) (open []*Record, whats map[string]string, cleared int) {
|
||||
// What may be handed on is judged first: neverHeld asks the asks and the walks under the lock itself.
|
||||
h.mu.Lock()
|
||||
recs := map[string]*Record{}
|
||||
for key := range w.Held {
|
||||
if r := h.open[key]; r != nil && r.Cleared.IsZero() && !r.silenced(now) && !lastSaid(r).After(w.Held[key].At) {
|
||||
recs[key] = r
|
||||
}
|
||||
}
|
||||
h.mu.Unlock()
|
||||
handable := map[string]bool{}
|
||||
for key, r := range recs {
|
||||
handable[key] = !h.neverHeld(r)
|
||||
}
|
||||
h.mu.Lock()
|
||||
whats = map[string]string{}
|
||||
taken := map[string]*Walk{}
|
||||
var handed []string
|
||||
for key, hm := range w.Held {
|
||||
r := h.open[key]
|
||||
live := r != nil && r.Cleared.IsZero() && handable[key]
|
||||
var s subject
|
||||
if live {
|
||||
s = subjectOf(r)
|
||||
}
|
||||
elsewhere := false
|
||||
for _, o := range h.walks {
|
||||
if o.ID == w.ID || !o.holding(now) {
|
||||
continue
|
||||
}
|
||||
if _, ok := o.Held[key]; ok {
|
||||
elsewhere = true
|
||||
continue
|
||||
}
|
||||
if live && o.touches(s) {
|
||||
if o.Held == nil {
|
||||
o.Held = map[string]HeldMessage{}
|
||||
}
|
||||
o.Held[key] = hm
|
||||
taken[o.ID] = o
|
||||
handed = append(handed, key+" to walk "+o.ID)
|
||||
elsewhere = true
|
||||
}
|
||||
}
|
||||
if elsewhere {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case r == nil || !r.Cleared.IsZero():
|
||||
cleared++
|
||||
case r.silenced(now) || lastSaid(r).After(hm.At):
|
||||
default:
|
||||
open = append(open, r)
|
||||
whats[key] = hm.What
|
||||
}
|
||||
}
|
||||
sort.Slice(open, func(i, j int) bool {
|
||||
if !open[i].Raised.Equal(open[j].Raised) {
|
||||
return open[i].Raised.Before(open[j].Raised)
|
||||
}
|
||||
return open[i].Key < open[j].Key
|
||||
})
|
||||
h.mu.Unlock()
|
||||
for _, o := range taken {
|
||||
h.keepWalk(o)
|
||||
}
|
||||
sort.Strings(handed)
|
||||
for _, x := range handed {
|
||||
key, _, _ := strings.Cut(x, " to walk ")
|
||||
h.record(Sent{At: now, Channel: "-", Key: key, What: "held on: walk " + w.ID + " ended, " + x[len(key)+1:] + " touches it",
|
||||
Outcome: "held"})
|
||||
}
|
||||
return open, whats, cleared
|
||||
}
|
||||
|
||||
// endWalk tells a walk's end once (ADR 0308 §4).
|
||||
func (h *Holder) endWalk(w *Walk, outcome string, ended, now time.Time) {
|
||||
open, whats, cleared := h.release(w, now)
|
||||
if now.Sub(ended) > FreshFor && len(open) == 0 {
|
||||
h.Logf("[messenger] walk %s %s %s ago, heard late: history, nothing said (%d held cleared meanwhile)",
|
||||
w.ID, outcome, roughly(now.Sub(ended)), cleared)
|
||||
h.record(Sent{At: now, Channel: "-", Key: "walk." + w.ID, What: "ended " + outcome, Outcome: "quiet: heard late"})
|
||||
return
|
||||
}
|
||||
took := ended.Sub(w.Started)
|
||||
head := map[string]string{WalkDelivered: "Delivered", WalkFailed: "Delivery failed", WalkStopped: "Delivery stopped"}[outcome]
|
||||
if head == "" {
|
||||
head = "Delivery ended"
|
||||
}
|
||||
reached := touchWords(w.Touches)
|
||||
oneLine := outcome == WalkDelivered && len(w.Held) == 0 && len(open) == 0
|
||||
var m Message
|
||||
if oneLine {
|
||||
m = Message{Title: head + ": " + reached + ", in " + roughly(took), Quiet: true}
|
||||
} else {
|
||||
lines := []string{"It took " + roughly(took) + "."}
|
||||
lines = append(lines, h.heldLines(open, whats, cleared)...)
|
||||
m = Message{Title: head + ": " + reached, Body: strings.Join(lines, "\n")}
|
||||
}
|
||||
h.tellWalk(w, m, open, whats, oneLine, "ended "+outcome, now)
|
||||
}
|
||||
|
||||
// heldLines are the lines a walk's end or bound says of what it held: each still open, then the cleared count.
|
||||
func (h *Holder) heldLines(open []*Record, whats map[string]string, cleared int) []string {
|
||||
var lines []string
|
||||
if len(open) > 0 {
|
||||
lines = append(lines, "Still open after it:")
|
||||
for _, r := range open {
|
||||
what := whats[r.Key]
|
||||
if what == "" {
|
||||
what = "raised"
|
||||
}
|
||||
lines = append(lines, "- "+h.say(r, what).Title)
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case cleared == 1:
|
||||
lines = append(lines, "1 problem it raised resolved during it.")
|
||||
case cleared > 1:
|
||||
lines = append(lines, fmt.Sprintf("%d problems it raised resolved during it.", cleared))
|
||||
}
|
||||
return lines
|
||||
}
|
||||
|
||||
// boundWalks says, of every walk past its bound, what it holds that is still open (ADR 0308 §5), then forgets
|
||||
// the walks whose end never came, and the ends heard a day ago. What a walk holds is said before it is
|
||||
// forgotten: a router back after a long while says it, then forgets the walk.
|
||||
func (h *Holder) boundWalks(now time.Time) {
|
||||
h.mu.Lock()
|
||||
var due, gone []*Walk
|
||||
var stale []string
|
||||
for _, w := range h.walks {
|
||||
if !w.BoundSaid && !now.Before(w.Started.Add(w.bound())) {
|
||||
due = append(due, w)
|
||||
}
|
||||
if now.Sub(w.Started) >= WalkForget {
|
||||
gone = append(gone, w)
|
||||
}
|
||||
}
|
||||
for id, e := range h.ended {
|
||||
if now.Sub(e.Ended) >= WalkForget {
|
||||
stale = append(stale, id)
|
||||
}
|
||||
}
|
||||
for _, id := range stale {
|
||||
delete(h.ended, id)
|
||||
}
|
||||
h.mu.Unlock()
|
||||
for _, id := range stale {
|
||||
h.mu.Lock()
|
||||
reopened := h.walks[id] != nil
|
||||
h.mu.Unlock()
|
||||
if !reopened {
|
||||
h.forgetWalk(id)
|
||||
}
|
||||
}
|
||||
defer func() {
|
||||
for _, w := range gone {
|
||||
h.mu.Lock()
|
||||
delete(h.walks, w.ID)
|
||||
h.mu.Unlock()
|
||||
h.Logf("[messenger] walk %s started %s ago and its end never came: forgotten", w.ID, roughly(now.Sub(w.Started)))
|
||||
h.forgetWalk(w.ID)
|
||||
}
|
||||
}()
|
||||
sort.Slice(due, func(i, j int) bool { return due[i].Started.Before(due[j].Started) })
|
||||
for _, w := range due {
|
||||
h.mu.Lock()
|
||||
w.BoundSaid = true
|
||||
h.mu.Unlock()
|
||||
open, whats, cleared := h.release(w, now)
|
||||
h.mu.Lock()
|
||||
w.Held = map[string]HeldMessage{}
|
||||
h.mu.Unlock()
|
||||
h.keepWalk(w)
|
||||
if len(open) == 0 && cleared == 0 {
|
||||
h.record(Sent{At: now, Channel: "-", Key: "walk." + w.ID, What: "past its bound", Outcome: "nothing held"})
|
||||
continue
|
||||
}
|
||||
since := w.Started.In(h.zone()).Format("15:04")
|
||||
lines := []string{"The walk " + w.ID + " is still running, since " + since + "."}
|
||||
if _, ok := Check(lines[0]); !ok {
|
||||
lines[0] = "The walk is still running, since " + since + "."
|
||||
}
|
||||
lines = append(lines, h.heldLines(open, whats, cleared)...)
|
||||
m := Message{Title: "Still running: the delivery to " + touchWords(w.Touches), Body: strings.Join(lines, "\n")}
|
||||
h.tellWalk(w, m, open, whats, false, "past its bound", now)
|
||||
}
|
||||
}
|
||||
|
||||
// tellWalk holds a walk's message on the kinds it goes to, alone, and marks what it carries as said there. When
|
||||
// no channel can send, or the message cannot carry what it held, each held message is left to its own path:
|
||||
// unsent, and tried again every minute as any other (outbox.go), or said on its own.
|
||||
func (h *Holder) tellWalk(w *Walk, m Message, open []*Record, whats map[string]string, oneLine bool, what string, now time.Time) {
|
||||
for _, r := range open {
|
||||
m.Urgent = m.Urgent || r.Severity == Urgent
|
||||
}
|
||||
var alone []*Record // said on their own, after the walk's message
|
||||
if _, ok := Check(m.Text()); !ok {
|
||||
m = Message{Title: "A delivery ended", Body: "What it reached could not be said here: the delivery's walks say it.", Urgent: m.Urgent}
|
||||
alone, open = open, nil
|
||||
}
|
||||
// Owed until a message carrying it was sent, in the state: a router that restarts first says it on its own
|
||||
// (issue 538), whichever path below it takes.
|
||||
h.mu.Lock()
|
||||
for _, r := range append(append([]*Record(nil), open...), alone...) {
|
||||
say := whats[r.Key]
|
||||
if say == "" {
|
||||
say = "raised"
|
||||
}
|
||||
r.Owed = say
|
||||
}
|
||||
h.mu.Unlock()
|
||||
var to []Channel
|
||||
if m.Urgent {
|
||||
for _, ch := range h.channels() {
|
||||
if ch.Ready() == nil {
|
||||
to = append(to, ch)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
switch {
|
||||
case h.Desk != nil && h.Desk.Ready() == nil:
|
||||
to = []Channel{h.Desk}
|
||||
case h.Away != nil && h.Away.Ready() == nil:
|
||||
to = []Channel{h.Away}
|
||||
}
|
||||
}
|
||||
var kept []Channel
|
||||
for _, ch := range to {
|
||||
if oneLine && h.NeedsOnly != nil && h.NeedsOnly(ch.Name()) {
|
||||
h.record(Sent{At: now, Channel: ch.Name(), Key: "walk." + w.ID, What: what,
|
||||
Outcome: "quiet: this kind says only what needs the operator"})
|
||||
continue
|
||||
}
|
||||
kept = append(kept, ch)
|
||||
}
|
||||
if len(to) == 0 {
|
||||
h.Logf("[messenger] walk %s %s: no channel can send now; what it held is tried again every minute: %s", w.ID, what, h.whyNot())
|
||||
for _, r := range append(open, alone...) {
|
||||
say := whats[r.Key]
|
||||
if say == "" {
|
||||
say = "raised"
|
||||
}
|
||||
h.mu.Lock()
|
||||
r.Pending = say
|
||||
h.mu.Unlock()
|
||||
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released by walk " + w.ID + " " + what,
|
||||
Outcome: "unsent: no channel can send; tried again every minute"})
|
||||
h.persist(r)
|
||||
}
|
||||
return
|
||||
}
|
||||
sev := Warning
|
||||
if m.Urgent {
|
||||
sev = Urgent
|
||||
}
|
||||
// What it carries is marked told on a channel once the message was sent there (outbox.go), not now: a
|
||||
// message lost with a restart before it was sent leaves them untold and owed, and the restarted router says
|
||||
// each on its own (issue 538 A).
|
||||
var carries []string
|
||||
for _, r := range open {
|
||||
carries = append(carries, r.Key)
|
||||
}
|
||||
for _, ch := range kept {
|
||||
msg := m
|
||||
h.enqueue(ch.Name(), item{Key: "walk." + w.ID, What: WalkEndWhat, Severity: sev, At: now, Msg: &msg, Carries: carries})
|
||||
}
|
||||
for _, r := range open {
|
||||
if len(kept) > 0 {
|
||||
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released",
|
||||
Outcome: "to be told in the message of walk " + w.ID + " " + what})
|
||||
}
|
||||
h.persist(r) // owed, in the state, until the message was sent
|
||||
}
|
||||
h.flush(now, false)
|
||||
for _, r := range alone {
|
||||
say := whats[r.Key]
|
||||
if say == "" {
|
||||
say = "raised"
|
||||
}
|
||||
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: "released", Outcome: "said on its own: walk " + w.ID + "'s message could not carry it"})
|
||||
h.deliver(r, say)
|
||||
h.persist(r)
|
||||
}
|
||||
}
|
||||
|
||||
// touchWords names what a walk reaches: its modules, then the machines.
|
||||
func touchWords(touches []Touch) string {
|
||||
mods := map[string]bool{}
|
||||
var machines []string
|
||||
for _, t := range touches {
|
||||
machines = append(machines, t.Machine)
|
||||
for _, m := range t.Modules {
|
||||
mods[m] = true
|
||||
}
|
||||
}
|
||||
var names []string
|
||||
for m := range mods {
|
||||
names = append(names, m)
|
||||
}
|
||||
sort.Strings(names)
|
||||
sort.Strings(machines)
|
||||
if len(names) == 0 {
|
||||
return "nothing moved"
|
||||
}
|
||||
if len(machines) == 0 {
|
||||
return listed(names)
|
||||
}
|
||||
return listed(names) + " on " + listed(machines)
|
||||
}
|
||||
|
||||
// listed is at most MostNamed names, then how many more.
|
||||
func listed(names []string) string {
|
||||
if len(names) <= MostNamed {
|
||||
return strings.Join(names, ", ")
|
||||
}
|
||||
return strings.Join(names[:MostNamed], ", ") + fmt.Sprintf(" and %d more", len(names)-MostNamed)
|
||||
}
|
||||
|
||||
// subject is what a message is about, as a walk's touches are matched against it.
|
||||
type subject struct {
|
||||
machines, modules []string
|
||||
// machineNamed: the subject names its machine, so a module it names is matched on that machine alone.
|
||||
machineNamed bool
|
||||
// tokens are every name it carries, to tell a message about a walk itself.
|
||||
tokens []string
|
||||
}
|
||||
|
||||
// subjectOf is a record's subject: a condition's scope and id (`<scope>.<id>.<kind>` when the record kept no
|
||||
// subject of its own), a notice's subject words.
|
||||
func subjectOf(r *Record) subject {
|
||||
scope, id := r.Scope, r.SubjectID
|
||||
parts := strings.Split(r.Key, ".")
|
||||
if scope == "" && len(parts) > 0 {
|
||||
scope = parts[0]
|
||||
}
|
||||
if id == "" && len(parts) >= 3 {
|
||||
id = strings.Join(parts[1:len(parts)-1], ".")
|
||||
}
|
||||
var s subject
|
||||
tokens := strings.Split(id, ".")
|
||||
if r.Origin == "notify" {
|
||||
tokens = append(strings.FieldsFunc(r.Subject, func(c rune) bool { return c == ' ' || c == ',' || c == '/' }), parts[1:]...)
|
||||
scope = "notify"
|
||||
}
|
||||
s.tokens = tokens
|
||||
if r.Machine != "" {
|
||||
s.machines, s.machineNamed = append(s.machines, r.Machine), true
|
||||
}
|
||||
switch scope {
|
||||
case "machine", "node", "bus":
|
||||
s.machines, s.machineNamed = append(s.machines, tokens[0]), true
|
||||
if len(tokens) > 1 {
|
||||
s.modules = append(s.modules, tokens[1])
|
||||
}
|
||||
case "module", "core", "seat":
|
||||
s.modules = append(s.modules, tokens[0])
|
||||
if len(tokens) > 1 {
|
||||
s.machines, s.machineNamed = append(s.machines, tokens[1]), true
|
||||
}
|
||||
default:
|
||||
// Its names, each matched as a machine and as a module.
|
||||
s.machines = append(s.machines, tokens...)
|
||||
s.modules = append(s.modules, tokens...)
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// neverHeldScopes are the subjects of a walk's own failure (ADR 0308 §3): a plan, a walk, a delivery, a build.
|
||||
var neverHeldScopes = map[string]bool{"plan": true, "walk": true, "delivery": true, "build": true, "release": true}
|
||||
|
||||
// neverHeldKinds are what is sent at once whatever a walk touches: a secret given or replaced (ADR 0308 §3).
|
||||
var neverHeldKinds = map[string]bool{"secret-given": true, "secret-replaced": true}
|
||||
|
||||
// neverHeld says whether a record is one ADR 0308 §3 sends at once.
|
||||
func (h *Holder) neverHeld(r *Record) bool {
|
||||
if r.Origin == "ask" || h.openAsk(r) != nil {
|
||||
return true
|
||||
}
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
parts := strings.Split(r.Key, ".")
|
||||
kind := r.Kind
|
||||
if kind == "" || kind == "notice" {
|
||||
kind = parts[len(parts)-1]
|
||||
}
|
||||
scope := r.Scope
|
||||
if scope == "" {
|
||||
scope = parts[0]
|
||||
}
|
||||
if neverHeldKinds[kind] || neverHeldScopes[scope] {
|
||||
return true
|
||||
}
|
||||
for _, t := range subjectOf(r).tokens {
|
||||
if h.walks[t] != nil {
|
||||
return true // about a walk itself
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// hold holds a record's message when an open walk touches its subject (ADR 0308 §2): kept on every such walk,
|
||||
// with when it was first held and what was to be said, and said held in the channel's history.
|
||||
func (h *Holder) hold(r *Record, what string) bool {
|
||||
switch what {
|
||||
case "cleared", "silenced", "asked":
|
||||
return false
|
||||
}
|
||||
h.mu.Lock()
|
||||
h.init()
|
||||
walking := len(h.walks) > 0
|
||||
h.mu.Unlock()
|
||||
if !walking || h.neverHeld(r) {
|
||||
return false
|
||||
}
|
||||
now := h.Now()
|
||||
h.mu.Lock()
|
||||
s := subjectOf(r)
|
||||
var by []*Walk
|
||||
var ids []string
|
||||
fresh := false
|
||||
for _, w := range h.walks {
|
||||
if !w.holding(now) || !w.touches(s) {
|
||||
continue
|
||||
}
|
||||
prev, had := w.Held[r.Key]
|
||||
hm := HeldMessage{At: now, What: what}
|
||||
if had {
|
||||
hm.At = prev.At
|
||||
if !r.told() && prev.What == "raised" {
|
||||
hm.What = prev.What // never said yet: it is a new problem to the operator, whatever came since
|
||||
}
|
||||
}
|
||||
if !had || hm.What != prev.What {
|
||||
fresh = true
|
||||
}
|
||||
if w.Held == nil {
|
||||
w.Held = map[string]HeldMessage{}
|
||||
}
|
||||
w.Held[r.Key] = hm
|
||||
by = append(by, w)
|
||||
ids = append(ids, w.ID)
|
||||
}
|
||||
h.mu.Unlock()
|
||||
if len(by) == 0 {
|
||||
return false
|
||||
}
|
||||
sort.Strings(ids)
|
||||
for _, w := range by {
|
||||
h.keepWalk(w)
|
||||
}
|
||||
if fresh {
|
||||
h.record(Sent{At: now, Channel: "-", Key: r.Key, What: what + ", while walk " + strings.Join(ids, ", ") + " touches it",
|
||||
Outcome: "held"})
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// heldBy is the walks holding a record now.
|
||||
func (h *Holder) heldBy(key string, now time.Time) []string {
|
||||
var ids []string
|
||||
for _, w := range h.walks {
|
||||
if _, ok := w.Held[key]; ok && w.holding(now) {
|
||||
ids = append(ids, w.ID)
|
||||
}
|
||||
}
|
||||
sort.Strings(ids)
|
||||
return ids
|
||||
}
|
||||
|
||||
func (h *Holder) keepWalk(w *Walk) {
|
||||
k, ok := h.Store.(walkKeeper)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
h.mu.Lock()
|
||||
c := *w
|
||||
c.Held = map[string]HeldMessage{}
|
||||
for key, v := range w.Held {
|
||||
c.Held[key] = v
|
||||
}
|
||||
h.mu.Unlock()
|
||||
if err := k.PutWalk(c); err != nil {
|
||||
h.noteStore(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Holder) forgetWalk(id string) {
|
||||
if k, ok := h.Store.(walkKeeper); ok {
|
||||
if err := k.DeleteWalk(id); err != nil {
|
||||
h.noteStore(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// loadWalks reads back the open walks and the restart passes; called with h.mu held.
|
||||
func (h *Holder) loadWalks() error {
|
||||
k, ok := h.Store.(walkKeeper)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
walks, err := k.Walks()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := range walks {
|
||||
w := walks[i]
|
||||
if !w.Ended.IsZero() {
|
||||
h.ended[w.ID] = &w
|
||||
continue
|
||||
}
|
||||
if w.Held == nil {
|
||||
w.Held = map[string]HeldMessage{}
|
||||
}
|
||||
h.walks[w.ID] = &w
|
||||
}
|
||||
passes, err := k.Passes()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for kind, p := range passes {
|
||||
h.passes[kind] = p
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,770 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The tests of novox/hq ADR 0308 "How it is checked" §2–§6, with the controller's walk events given by a fake
|
||||
// source: the body the controller's plan-moved carries at a walk's start and end (ADR 0308 §1).
|
||||
|
||||
// walkBody is a plan-moved body as the controller says a walk's start or end (mesh-controller pull request 247).
|
||||
func walkBody(t *testing.T, id, phase, outcome string, at time.Time, bound int, touches ...Touch) []byte {
|
||||
t.Helper()
|
||||
b := map[string]any{"id": id, "state": "walking", "walk": id, "phase": phase, "touches": touches,
|
||||
"updated": at.Format(time.RFC3339Nano), "budget_class": "leaf"}
|
||||
if phase == WalkStarted {
|
||||
b["started_at"] = at.Format(time.RFC3339Nano)
|
||||
} else {
|
||||
b["ended_at"] = at.Format(time.RFC3339Nano)
|
||||
}
|
||||
if outcome != "" {
|
||||
b["outcome"] = outcome
|
||||
}
|
||||
if bound > 0 {
|
||||
b["bound_seconds"] = bound
|
||||
}
|
||||
raw, err := json.Marshal(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
// walk says a walk's start or end to the router, through the decoder the bus's events go through.
|
||||
func walk(t *testing.T, h *Holder, id, phase, outcome string, at time.Time, bound int, touches ...Touch) {
|
||||
t.Helper()
|
||||
ev, ok, err := DecodeWalk(walkBody(t, id, phase, outcome, at, bound, touches...))
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("the walk event was not read: %v %v", ok, err)
|
||||
}
|
||||
h.Walked(ev)
|
||||
}
|
||||
|
||||
func novox(modules ...string) Touch { return Touch{Machine: "novox", Modules: modules} }
|
||||
|
||||
// heard says whether any message on the channels carries the words: sent, edited or said again in place.
|
||||
func heard(words string, chs ...*fakeChannel) bool {
|
||||
for _, ch := range chs {
|
||||
for _, m := range ch.sends {
|
||||
if strings.Contains(m.Text(), words) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, m := range ch.edits {
|
||||
if strings.Contains(m.Text(), words) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, r := range ch.replaced {
|
||||
if strings.Contains(r.m.Text(), words) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// outcomes are the history's outcomes for a key, oldest first.
|
||||
func outcomes(h *Holder, key string) []string {
|
||||
var out []string
|
||||
h.mu.Lock()
|
||||
defer h.mu.Unlock()
|
||||
for _, s := range h.recent {
|
||||
if s.Key == key {
|
||||
out = append(out, s.Outcome)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func has(list []string, want string) bool {
|
||||
for _, s := range list {
|
||||
if s == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// §2: during a walk, a message about a machine or a module it touches is held, not sent, and the channel's
|
||||
// history says it was held — a module's condition, a machine's, and a module's notice alike.
|
||||
func TestDuringAWalkWhatItTouchesIsHeldAndTheHistorySaysSo(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-1", WalkStarted, "", c.now(), 600, novox("mesh-tools", "node-tools"))
|
||||
mod := cond("module.node-tools.novox.unhealthy", Urgent, "the tool runner on novox does not answer")
|
||||
mod.Machine, mod.Raised = "novox", c.now()
|
||||
h.Condition(EventRaised, mod)
|
||||
machine := cond("machine.novox.claude-code.agent-home.data-shrank", Urgent, "the agent's home shrank")
|
||||
machine.Raised = c.now()
|
||||
h.Condition(EventRaised, machine)
|
||||
if _, err := h.Notify("tools.slow", Urgent, "tools answer slowly", "node-tools", false, "tools slow",
|
||||
"Needs you: look at the tools."); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runFor(h, c, 2*time.Minute)
|
||||
if saidOn(tg, dt) != 0 {
|
||||
t.Fatalf("held messages were said: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||
}
|
||||
for _, key := range []string{mod.Key, machine.Key, "notify.tools.slow"} {
|
||||
if !has(outcomes(h, key), "held") {
|
||||
t.Errorf("the history does not say %s was held: %v", key, outcomes(h, key))
|
||||
}
|
||||
}
|
||||
if len(h.Open()) != 3 {
|
||||
t.Fatalf("held messages are not kept open: %+v", h.Open())
|
||||
}
|
||||
}
|
||||
|
||||
// §2: a message touched by two overlapping walks is held until the second of them ends; the first's end says
|
||||
// nothing of it.
|
||||
func TestAMessageTouchedByTwoOverlappingWalksIsHeldUntilTheSecondEnds(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
c.pass(time.Minute)
|
||||
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"), Touch{Machine: "ace", Modules: []string{"node-tools"}})
|
||||
k := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0, novox("mesh-tools"))
|
||||
runFor(h, c, time.Minute)
|
||||
if heard(headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("said at the first walk's end, while the second still holds it: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
if !heard("Delivered: mesh-tools on novox", tg, dt) {
|
||||
t.Fatalf("the first walk's end was not told: %+v", dt.sends)
|
||||
}
|
||||
walk(t, h, "plan-b", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("the second walk's end did not tell it: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||
}
|
||||
}
|
||||
|
||||
// §3: an ask, a secret given, a message about a machine no open walk touches and a walk's own failure are
|
||||
// sent at once, while a walk touches the machine they are on.
|
||||
func TestWhatAWalkNeverHoldsIsSentAtOnce(t *testing.T) {
|
||||
h, tg, dt, c, _, _ := newRouter(t)
|
||||
walk(t, h, "plan-9", WalkStarted, "", c.now(), 1200, novox("telegram", "mesh-tools"))
|
||||
for _, key := range []string{
|
||||
"machine.novox.telegram.telegram-token.secret-given", // a secret given, on a touched machine
|
||||
"machine.novox.mounts.smb.secret-replaced", // a secret replaced
|
||||
"machine.ace.silent", // a machine no walk touches
|
||||
"delivery.leaf.over-budget", // the walk's own delivery
|
||||
"plan.plan-9.failed", // the walk itself
|
||||
} {
|
||||
k := cond(key, Urgent, "it happened")
|
||||
k.Raised = c.now()
|
||||
if !strings.HasPrefix(key, "machine.") {
|
||||
k.Machine = "novox" // a walk's own failure, on a machine it touches
|
||||
}
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
if !heard(headlineOf(key), tg, dt) {
|
||||
t.Errorf("%s was not sent at once during the walk: %v", key, outcomes(h, key))
|
||||
}
|
||||
}
|
||||
// An ask's own message, and an ask about a held condition: both offered at once.
|
||||
submit(t, h, ConditionOwner, ackAsk(c, "own"))
|
||||
runFor(h, c, time.Minute)
|
||||
if !has(outcomes(h, "ask."+ConditionOwner+".own"), "sent") {
|
||||
t.Errorf("an ask waiting for the operator was held: %v", outcomes(h, "ask."+ConditionOwner+".own"))
|
||||
}
|
||||
held := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||
held.Raised = c.now()
|
||||
h.Condition(EventRaised, held)
|
||||
runFor(h, c, time.Minute)
|
||||
if heard(headlineOf(held.Key), tg, dt) {
|
||||
t.Fatalf("a condition about a touched machine was not held")
|
||||
}
|
||||
about := ackAsk(c, "about")
|
||||
about.About = held.Key
|
||||
submit(t, h, ConditionOwner, about)
|
||||
runFor(h, c, time.Minute)
|
||||
if !heard(headlineOf(held.Key), dt) {
|
||||
t.Fatalf("the ask about a held condition was held: %v", outcomes(h, held.Key))
|
||||
}
|
||||
}
|
||||
|
||||
// §4: a walk's end is one message — its outcome, what it reached and how long it took — then each held message
|
||||
// still open, one line each; the one that cleared is only counted, and nothing more is said of them after.
|
||||
func TestAWalksEndIsToldOnceWithWhatIsStillOpen(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-3", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||
still := cond("machine.novox.tools-silent", Warning, "the tools on novox are silent")
|
||||
still.Raised = c.now()
|
||||
h.Condition(EventRaised, still)
|
||||
gone := cond("module.mesh-tools.novox.unhealthy", Warning, "mesh-tools is unhealthy on novox")
|
||||
gone.Raised = c.now()
|
||||
h.Condition(EventRaised, gone)
|
||||
c.pass(time.Second)
|
||||
other := cond("machine.novox.core-behind", Urgent, "novox runs an older core")
|
||||
other.Raised = c.now()
|
||||
h.Condition(EventRaised, other)
|
||||
runFor(h, c, 2*time.Minute)
|
||||
h.Condition(EventCleared, Condition{Key: gone.Key, At: c.now()})
|
||||
runFor(h, c, 5*time.Minute)
|
||||
if saidOn(tg, dt) != 0 {
|
||||
t.Fatalf("said during the walk: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
walk(t, h, "plan-3", WalkEnded, WalkFailed, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
// Urgent among what it holds: on every kind, one message each.
|
||||
if len(dt.sends) != 1 || len(tg.sends) != 1 {
|
||||
t.Fatalf("the end was not one message per kind: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
m := dt.sends[0]
|
||||
if m.Title != "Delivery failed: mesh-tools on novox" || !strings.Contains(m.Body, "It took 7 min.") {
|
||||
t.Fatalf("the end does not say its outcome, what it reached and how long: %q", m.Text())
|
||||
}
|
||||
want := "Still open after it:\n- " + headlineOf(still.Key) + "\n- Urgent: " + headlineOf(other.Key) +
|
||||
"\n1 problem it raised resolved during it."
|
||||
if !strings.Contains(m.Body, want) || strings.Contains(m.Text(), headlineOf(gone.Key)) {
|
||||
t.Fatalf("the end message %q does not say %q", m.Body, want)
|
||||
}
|
||||
keyless(t, m)
|
||||
runFor(h, c, 20*time.Minute)
|
||||
if len(dt.sends) != 1 || len(tg.sends) != 1 || len(dt.replaced)+len(tg.replaced) != 0 {
|
||||
t.Fatalf("what the end told was said again: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
}
|
||||
|
||||
// §4: a walk that delivered with nothing held and nothing open is one line, and nothing at all on a kind set
|
||||
// to say only what needs the operator.
|
||||
func TestADeliveredWalkWithNothingHeldIsOneLineOrNothing(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-4", WalkStarted, "", c.now(), 600, novox("mesh-tools"), Touch{Machine: "ace", Modules: []string{"mesh-tools"}})
|
||||
runFor(h, c, 3*time.Minute)
|
||||
walk(t, h, "plan-4", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.sends) != 1 || len(tg.sends) != 0 {
|
||||
t.Fatalf("desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
if m := dt.sends[0]; m.Title != "Delivered: mesh-tools on ace, novox, in 3 min" || m.Body != "" {
|
||||
t.Fatalf("not one line: %q", m.Text())
|
||||
}
|
||||
h.NeedsOnly = func(kind string) bool { return kind == "desktop" }
|
||||
walk(t, h, "plan-5", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||
runFor(h, c, 3*time.Minute)
|
||||
walk(t, h, "plan-5", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.sends) != 1 || len(tg.sends) != 0 {
|
||||
t.Fatalf("told on a kind that says only what needs the operator: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
if !has(outcomes(h, "walk.plan-5"), "quiet: this kind says only what needs the operator") {
|
||||
t.Fatalf("the history does not say why nothing was told: %v", outcomes(h, "walk.plan-5"))
|
||||
}
|
||||
}
|
||||
|
||||
// §5: the bound is the controller's (bound_seconds: twice the largest budget of the walk's deliveries; none
|
||||
// given, 15 minutes). At it, what is held and still open is sent with the still-running line, and what
|
||||
// cleared is counted; before it, nothing; after it, the walk holds nothing more.
|
||||
func TestAHoldEndsAtTheWalksBound(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
given int
|
||||
bound time.Duration
|
||||
}{
|
||||
{"core: twice ten minutes", 1200, 20 * time.Minute},
|
||||
{"leaf at five minutes", 600, 10 * time.Minute},
|
||||
{"leaf at three minutes", 360, 6 * time.Minute},
|
||||
{"no budget", 0, 15 * time.Minute},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
started := c.now()
|
||||
walk(t, h, "plan-7", WalkStarted, "", started, tc.given, novox("node-tools"))
|
||||
open := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
open.Raised = c.now()
|
||||
h.Condition(EventRaised, open)
|
||||
gone := cond("module.node-tools.novox.unhealthy", Urgent, "node-tools is unhealthy")
|
||||
gone.Raised = c.now()
|
||||
h.Condition(EventRaised, gone)
|
||||
c.pass(30 * time.Second)
|
||||
h.Condition(EventCleared, Condition{Key: gone.Key, At: c.now()})
|
||||
runFor(h, c, tc.bound-time.Minute-30*time.Second)
|
||||
if saidOn(tg, dt) != 0 {
|
||||
t.Fatalf("said before the bound: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
runFor(h, c, 2*time.Minute)
|
||||
if len(dt.sends) != 1 || len(tg.sends) != 1 {
|
||||
t.Fatalf("at the bound: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
m := dt.sends[0]
|
||||
since := started.In(h.zone()).Format("15:04")
|
||||
for _, want := range []string{"is still running, since " + since + ".", "- Urgent: " + headlineOf(open.Key),
|
||||
"1 problem it raised resolved during it."} {
|
||||
if !strings.Contains(m.Body, want) {
|
||||
t.Fatalf("the bound's message %q does not say %q", m.Text(), want)
|
||||
}
|
||||
}
|
||||
// Past its bound the walk holds nothing: a new message about what it touches goes at once.
|
||||
late := cond("machine.novox.disk", Urgent, "a disk fails on novox")
|
||||
late.Raised = c.now()
|
||||
h.Condition(EventRaised, late)
|
||||
runFor(h, c, time.Minute)
|
||||
if !heard(headlineOf(late.Key), dt) {
|
||||
t.Fatalf("held past the bound: %v", outcomes(h, late.Key))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// §2, kept: what a walk holds survives the router's restart, and is told at the walk's end.
|
||||
func TestHeldMessagesSurviveARouterRestart(t *testing.T) {
|
||||
h, tg, dt, c, st := newHolder(t)
|
||||
walk(t, h, "plan-8", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||
if err := again.Load(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again.Start()
|
||||
c.pass(2 * time.Minute)
|
||||
walk(t, again, "plan-8", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(again, c, time.Minute)
|
||||
if !heard("- Urgent: "+headlineOf(k.Key), dt) || saidOn(tg, dt) != 2 {
|
||||
t.Fatalf("what was held before the restart was not told at the walk's end: desk %+v, telegram %+v", dt.sends, tg.sends)
|
||||
}
|
||||
}
|
||||
|
||||
// §6, issue 531: the desk's notifiers on two machines start again a second apart, with four silenced messages
|
||||
// shown there — one of them withheld for its content, a file path — and a fifth open, asked there, and
|
||||
// withheld. Nothing is said again: no silenced, no withheld message; the pass runs once; and a router that
|
||||
// starts again reads it back from its state and makes it no more.
|
||||
func TestARestartSaysNothingItHasSaid(t *testing.T) {
|
||||
h, tg, dt, c, _, _ := newRouter(t)
|
||||
h.Start()
|
||||
c.pass(2 * time.Minute)
|
||||
keys := []string{"machine.novox.telegram.telegram-token.secret-given", "machine.novox.claude-code.agent-home.data-shrank",
|
||||
"machine.shanks.mounts.smb-password-recalbox.secret-given", "machine.ace.nzbgeek.api-key.secret-given"}
|
||||
var silenced []Condition
|
||||
for i, key := range keys {
|
||||
k := cond(key, Urgent, "it happened")
|
||||
if i == 1 {
|
||||
k.Explanation = "Needs you: look at /var/lib/agent-home, which shrank."
|
||||
}
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
silenced = append(silenced, k)
|
||||
}
|
||||
withheldOpen := cond("machine.novox.units", Urgent, "2 failed services")
|
||||
withheldOpen.Explanation = "Needs you: read /etc/systemd/system/x.service."
|
||||
withheldOpen.Raised = c.now()
|
||||
h.Condition(EventRaised, withheldOpen)
|
||||
runFor(h, c, time.Minute)
|
||||
ask := ackAsk(c, "units")
|
||||
ask.About = withheldOpen.Key
|
||||
submit(t, h, ConditionOwner, ask)
|
||||
runFor(h, c, time.Minute)
|
||||
for _, k := range silenced {
|
||||
k.SilencedTill, k.SilencedWhy, k.At = c.now().Add(7*24*time.Hour), "known", c.now()
|
||||
h.Condition(EventChanged, k)
|
||||
}
|
||||
runFor(h, c, time.Minute)
|
||||
for _, key := range []string{silenced[1].Key, withheldOpen.Key} {
|
||||
if r := h.open[key]; r == nil || r.Refused == "" {
|
||||
t.Fatalf("the test's %s was not withheld for its content: %+v", key, r)
|
||||
}
|
||||
}
|
||||
sends, edits, replaced, tgSaid := len(dt.sends), len(dt.edits), len(dt.replaced), saidOn(tg)
|
||||
dt.edits = map[string]Message{}
|
||||
passes := func(h *Holder) int { return len(outcomes(h, "(answers lost)")) }
|
||||
// The notifiers on two machines started again a second apart: two standings, two times.
|
||||
c.pass(time.Minute)
|
||||
first := c.now()
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: first})
|
||||
c.pass(time.Second)
|
||||
h.Flush() // the runtime's tick, between the two standings
|
||||
second := c.now()
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: second})
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.sends) != sends || len(dt.edits) != 0 || len(dt.replaced) != replaced || saidOn(tg) != tgSaid {
|
||||
t.Fatalf("the restart said again: sends %+v, edits %+v (was %d), replaced %+v", dt.sends[sends:], dt.edits, edits,
|
||||
dt.replaced[replaced:])
|
||||
}
|
||||
if passes(h) != 1 {
|
||||
t.Fatalf("the pass ran %d times for one restart", passes(h))
|
||||
}
|
||||
// The router starts again; the desk repeats its standing.
|
||||
again := &Holder{Away: tg, Desk: dt, Store: h.Store, Asks: h.Asks, Bench: h.Bench, Publish: h.Publish, Now: c.now,
|
||||
Logf: t.Logf, Zone: h.Zone, RootFree: h.RootFree, Node: h.Node}
|
||||
if err := again.Load(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again.Start()
|
||||
again.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: second})
|
||||
runFor(again, c, 2*time.Minute)
|
||||
if len(dt.sends) != sends || len(dt.edits) != 0 || len(dt.replaced) != replaced || saidOn(tg) != tgSaid || passes(again) != 1 {
|
||||
t.Fatalf("the router's restart made the pass again: %d passes, sends %+v, edits %+v", passes(again),
|
||||
dt.sends[sends:], dt.edits)
|
||||
}
|
||||
}
|
||||
|
||||
// The controller's walk events are read as ADR 0308 §1 says them; any other plan-moved is not a walk's.
|
||||
func TestTheWalkEventsAreReadAsTheControllerSaysThem(t *testing.T) {
|
||||
at := time.Date(2026, 10, 12, 11, 12, 0, 0, time.UTC)
|
||||
ev, ok, err := DecodeWalk(walkBody(t, "plan-1", WalkStarted, "", at, 1200, novox("mesh-tools", "node-tools")))
|
||||
if err != nil || !ok || ev.Walk != "plan-1" || ev.BoundSeconds != 1200 || ev.Class != "leaf" || !ev.At.Equal(at) ||
|
||||
len(ev.Touches) != 1 || ev.Touches[0].Machine != "novox" || len(ev.Touches[0].Modules) != 2 {
|
||||
t.Fatalf("%+v %v %v", ev, ok, err)
|
||||
}
|
||||
// As mesh-controller's walk events (its pull request 247) say an end: ended_at, and carried beside modules.
|
||||
end, ok, err := DecodeWalk([]byte(`{"id":"plan-1","walk":"plan-1","phase":"ended","outcome":"stopped",` +
|
||||
`"touches":[{"machine":"novox","modules":["node-tools","messenger"],"carried":["messenger"]}],` +
|
||||
`"started_at":"2026-10-12T11:12:00Z","ended_at":"2026-10-12T11:19:00Z","updated":"2026-10-12T11:19:01Z"}`))
|
||||
if err != nil || !ok || end.Outcome != WalkStopped || !end.At.Equal(at.Add(7*time.Minute)) || len(end.Touches[0].Modules) != 2 {
|
||||
t.Fatalf("%+v %v %v", end, ok, err)
|
||||
}
|
||||
if _, ok, err := DecodeWalk([]byte(`{"id":"plan-1","state":"walking","tier":1}`)); ok || err != nil {
|
||||
t.Fatalf("a move inside a walk read as its start or end: %v %v", ok, err)
|
||||
}
|
||||
if _, _, err := DecodeWalk([]byte(`{"walk":"plan-1","phase":"ended"}`)); err == nil {
|
||||
t.Fatalf("an end without its outcome was read")
|
||||
}
|
||||
w := Walk{}
|
||||
if w.bound() != DefaultWalkBound {
|
||||
t.Fatalf("a walk without a bound is bounded at %s", w.bound())
|
||||
}
|
||||
// A start older than its bound is history: nothing is held.
|
||||
h, _, _, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-old", WalkStarted, "", c.now().Add(-time.Hour), 1200, novox("mesh-tools"))
|
||||
if len(h.walks) != 0 {
|
||||
t.Fatalf("an old start opened a walk: %+v", h.walks)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
// Review of pull request 230, 1: no channel can send at a walk's end or bound. What it held is not lost with the
|
||||
// walk: it is unsent, and said on its own once a channel can send.
|
||||
func TestWhatAWalkHeldIsSaidOnceAChannelCanSendAgain(t *testing.T) {
|
||||
for _, at := range []string{"end", "bound"} {
|
||||
t.Run(at, func(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-c", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, 2*time.Minute)
|
||||
down := errors.New("no session")
|
||||
tg.notReady, dt.notReady = down, down
|
||||
if at == "end" {
|
||||
walk(t, h, "plan-c", WalkEnded, WalkFailed, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
} else {
|
||||
runFor(h, c, 10*time.Minute)
|
||||
}
|
||||
if saidOn(tg, dt) != 0 {
|
||||
t.Fatalf("said on a channel that cannot send: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
if r := h.open[k.Key]; r == nil || r.Pending == "" {
|
||||
t.Fatalf("what the walk held is not left unsent: %+v", r)
|
||||
}
|
||||
tg.notReady, dt.notReady = nil, nil
|
||||
runFor(h, c, 2*time.Minute)
|
||||
if !heard(headlineOf(k.Key), dt) {
|
||||
t.Fatalf("what the walk held was never said: %v", outcomes(h, k.Key))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Review of pull request 230, 2: a message held by walk A, about a machine walk B also touches, is held on at
|
||||
// A's end while B — started after it was held — is still open, and told at B's end.
|
||||
func TestAMessageIsHeldOnByALaterWalkThatTouchesIt(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
k := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"))
|
||||
runFor(h, c, time.Minute)
|
||||
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if heard(headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("told at the first walk's end while the second touches it: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
if got := h.heldBy(k.Key, c.now()); len(got) != 1 || got[0] != "plan-b" {
|
||||
t.Fatalf("the second walk does not hold it on: %v", got)
|
||||
}
|
||||
walk(t, h, "plan-b", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("the second walk's end did not tell it: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
}
|
||||
|
||||
// Review of pull request 230, 3: a router back after more than a day reads back a walk whose end never came;
|
||||
// what it held is said, with the still-running line, before the walk is forgotten.
|
||||
func TestAWalkReadBackADayLateSaysWhatItHeldBeforeItIsForgotten(t *testing.T) {
|
||||
h, tg, dt, c, st := newHolder(t)
|
||||
walk(t, h, "plan-d", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
c.pass(25 * time.Hour)
|
||||
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||
if err := again.Load(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again.Start()
|
||||
runFor(again, c, 2*time.Minute)
|
||||
if !heard("is still running", dt) || !heard("- Urgent: "+headlineOf(k.Key), dt) {
|
||||
t.Fatalf("what the walk held was forgotten unsaid: desk %+v, %v", dt.sends, outcomes(again, k.Key))
|
||||
}
|
||||
if len(again.walks) != 0 {
|
||||
t.Fatalf("the walk was not forgotten: %+v", again.walks)
|
||||
}
|
||||
if ws, _ := st.Walks(); len(ws) != 0 {
|
||||
t.Fatalf("the walk is still kept: %+v", ws)
|
||||
}
|
||||
}
|
||||
|
||||
// Review of pull request 230, 4 (and of mesh-controller pull request 247): a short walk's end can come before
|
||||
// its start. The start of a walk already ended opens nothing — also after the router's restart — while the
|
||||
// next walk under the same id, started after that end, does.
|
||||
func TestAStartHeardAfterItsEndOpensNothing(t *testing.T) {
|
||||
h, tg, dt, c, st := newHolder(t)
|
||||
started := c.now()
|
||||
c.pass(20 * time.Second)
|
||||
end, _, err := DecodeWalk([]byte(`{"walk":"plan-e","phase":"ended","outcome":"delivered","touches":[{"machine":"novox","modules":["mesh-tools"]}],` +
|
||||
`"started_at":"` + started.Format(time.RFC3339Nano) + `","ended_at":"` + c.now().Format(time.RFC3339Nano) + `"}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.Walked(end)
|
||||
c.pass(time.Second)
|
||||
walk(t, h, "plan-e", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||
if len(h.walks) != 0 {
|
||||
t.Fatalf("a start heard after its end opened a walk: %+v", h.walks)
|
||||
}
|
||||
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||
if err := again.Load(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again.Start()
|
||||
walk(t, again, "plan-e", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||
if len(again.walks) != 0 {
|
||||
t.Fatalf("after a restart, a start heard after its end opened a walk: %+v", again.walks)
|
||||
}
|
||||
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
again.Condition(EventRaised, k)
|
||||
runFor(again, c, time.Minute)
|
||||
if !heard(headlineOf(k.Key), dt) {
|
||||
t.Fatalf("held by a walk that had ended: %v", outcomes(again, k.Key))
|
||||
}
|
||||
// Retried: the same id walks again, started after the end.
|
||||
walk(t, again, "plan-e", WalkStarted, "", c.now(), 600, novox("mesh-tools"))
|
||||
if len(again.walks) != 1 {
|
||||
t.Fatalf("a walk started after the last end did not open: %+v", again.walks)
|
||||
}
|
||||
}
|
||||
|
||||
// Review of pull request 230, non-blocking: a message whose answers became asked while a walk held it is never
|
||||
// held (ADR 0308 §3); when the desk's answers are lost, it is said again in place like any other, so its
|
||||
// buttons work again.
|
||||
func TestARestartSaysAgainAnAskedMessageAWalkTookBeforeItsAsk(t *testing.T) {
|
||||
h, _, dt, c, _, _ := newRouter(t)
|
||||
h.Start()
|
||||
stand(h, c)
|
||||
walk(t, h, "plan-f", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
k := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
ask := ackAsk(c, "units")
|
||||
ask.About = k.Key
|
||||
submit(t, h, ConditionOwner, ask)
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
if !heard(headlineOf(k.Key), dt) {
|
||||
t.Fatalf("the ask about it was not offered: %v", outcomes(h, k.Key))
|
||||
}
|
||||
replaced := len(dt.replaced)
|
||||
c.pass(time.Minute)
|
||||
h.Stand("desktop", Standing{Ready: true, At: c.now(), AnswersLostAt: c.now()})
|
||||
runFor(h, c, time.Minute)
|
||||
if len(dt.replaced) != replaced+1 {
|
||||
t.Fatalf("its answers were not made to work again: replaced %+v, %v", dt.replaced[replaced:], outcomes(h, "(answers lost)"))
|
||||
}
|
||||
}
|
||||
|
||||
// Review of pull request 230, non-blocking: a walk's end that cannot be said as it is (here a module named by a
|
||||
// file path) is said in its plain form, and what it held, still open, is said on its own, not marked told.
|
||||
func TestWhatAWalksPlainEndCannotCarryIsSaidOnItsOwn(t *testing.T) {
|
||||
h, tg, dt, c, _ := newHolder(t)
|
||||
walk(t, h, "plan-g", WalkStarted, "", c.now(), 600, novox("/var/lib/mesh-tools"))
|
||||
k := cond("machine.novox.tools-silent", Urgent, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
walk(t, h, "plan-g", WalkEnded, WalkFailed, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if !heard("A delivery ended", dt) {
|
||||
t.Fatalf("the plain end was not said: %+v", dt.sends)
|
||||
}
|
||||
if !heard(headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("what the walk held was marked told and never said: %v", outcomes(h, k.Key))
|
||||
}
|
||||
}
|
||||
|
||||
// Issue 538 A: what a walk held is marked told only once the message carrying it was sent. When that message
|
||||
// could not be sent and the router restarts before its retry, the message is gone with it: what it held is not
|
||||
// marked told, it stays owed in the state, and the restarted router says it — an urgent one and a warning alike,
|
||||
// once each, never left as history.
|
||||
func TestAHeldMessageIsToldOnlyOnceTheWalksMessageWasSent(t *testing.T) {
|
||||
for _, then := range []string{"sent", "restart urgent", "restart warning"} {
|
||||
t.Run(then, func(t *testing.T) {
|
||||
h, tg, dt, c, st := newHolder(t)
|
||||
sev := Urgent
|
||||
if then == "restart warning" {
|
||||
sev = Warning
|
||||
}
|
||||
walk(t, h, "plan-h", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
k := cond("machine.novox.tools-silent", sev, "the tools on novox are silent")
|
||||
k.Raised = c.now()
|
||||
h.Condition(EventRaised, k)
|
||||
runFor(h, c, time.Minute)
|
||||
down := errors.New("cannot connect")
|
||||
tg.fail, dt.fail = down, down
|
||||
walk(t, h, "plan-h", WalkEnded, WalkFailed, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
if saidOn(tg, dt) != 0 {
|
||||
t.Fatalf("said on channels that cannot send: %+v %+v", tg.sends, dt.sends)
|
||||
}
|
||||
if r := h.open[k.Key]; r == nil || r.told() {
|
||||
t.Fatalf("marked told while the message carrying it was not sent: %+v", r)
|
||||
}
|
||||
tg.fail, dt.fail = nil, nil
|
||||
if then == "sent" {
|
||||
runFor(h, c, 2*time.Minute)
|
||||
if !heard("Still open after it:\n- Urgent: "+headlineOf(k.Key), tg, dt) || saidOn(tg, dt) != 2 {
|
||||
t.Fatalf("not told once in the walk's message: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||
}
|
||||
if r := h.open[k.Key]; r == nil || !r.told() || r.Owed != "" || r.Sent["telegram"] != "" || r.Sent["desktop"] != "" {
|
||||
t.Fatalf("not marked told on both channels once sent: %+v", r)
|
||||
}
|
||||
runFor(h, c, 10*time.Minute)
|
||||
if saidOn(tg, dt) != 2 {
|
||||
t.Fatalf("said again after the walk's message: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||
}
|
||||
return
|
||||
}
|
||||
again := &Holder{Away: tg, Desk: dt, Store: st, Now: c.now, Logf: t.Logf}
|
||||
if err := again.Load(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again.Start()
|
||||
again.Sync([]Condition{k}, c.now())
|
||||
runFor(again, c, 5*time.Minute)
|
||||
if !heard(headlineOf(k.Key), tg, dt) {
|
||||
t.Fatalf("what the lost message carried was not said after the restart: %v", outcomes(again, k.Key))
|
||||
}
|
||||
said := saidOn(tg, dt)
|
||||
if r := again.open[k.Key]; r == nil || !r.told() || r.Owed != "" || r.Pending != "" {
|
||||
t.Fatalf("said, and still owed or not marked told: %+v", r)
|
||||
}
|
||||
runFor(again, c, 20*time.Minute) // within its reminder's bound: nothing more is due
|
||||
again.Sync([]Condition{k}, c.now())
|
||||
runFor(again, c, time.Minute)
|
||||
if saidOn(tg, dt) != said {
|
||||
t.Fatalf("said again: telegram %+v, desk %+v", tg.sends, dt.sends)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Issue 538 B: at a walk's end, a later walk that touches what it held takes over only what a first hold would
|
||||
// take: not a message silenced meanwhile, nor one an ask waiting for the operator is about (ADR 0308 §3).
|
||||
func TestAWalksEndHandsOnOnlyWhatALaterWalkWouldHold(t *testing.T) {
|
||||
h, tg, dt, c, _, _ := newRouter(t)
|
||||
h.Start()
|
||||
stand(h, c)
|
||||
walk(t, h, "plan-a", WalkStarted, "", c.now(), 1200, novox("mesh-tools"))
|
||||
quiet := cond("machine.novox.network", Urgent, "novox lost its default route")
|
||||
quiet.Raised = c.now()
|
||||
h.Condition(EventRaised, quiet)
|
||||
asked := cond("machine.novox.units", Urgent, "2 failed services on novox")
|
||||
asked.Raised = c.now()
|
||||
h.Condition(EventRaised, asked)
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
if heard(headlineOf(quiet.Key), tg, dt) || heard(headlineOf(asked.Key), tg, dt) {
|
||||
t.Fatalf("the test's messages were not held")
|
||||
}
|
||||
quiet.SilencedTill, quiet.SilencedWhy, quiet.At = c.now().Add(7*24*time.Hour), "known", c.now()
|
||||
h.Condition(EventChanged, quiet)
|
||||
ask := ackAsk(c, "units")
|
||||
ask.About = asked.Key
|
||||
submit(t, h, ConditionOwner, ask)
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
walk(t, h, "plan-b", WalkStarted, "", c.now(), 1200, novox("node-tools"))
|
||||
runFor(h, c, time.Minute)
|
||||
stand(h, c)
|
||||
helds := func(key string) int {
|
||||
n := 0
|
||||
for _, o := range outcomes(h, key) {
|
||||
if o == "held" {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
before := map[string]int{quiet.Key: helds(quiet.Key), asked.Key: helds(asked.Key)}
|
||||
walk(t, h, "plan-a", WalkEnded, WalkDelivered, c.now(), 0)
|
||||
runFor(h, c, time.Minute)
|
||||
for _, key := range []string{quiet.Key, asked.Key} {
|
||||
if got := h.heldBy(key, c.now()); len(got) != 0 {
|
||||
t.Errorf("%s was handed on to %v, which a first hold would never hold", key, got)
|
||||
}
|
||||
if helds(key) != before[key] {
|
||||
t.Errorf("the history says %s held again: %v", key, outcomes(h, key))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Issue 538 C: the controller's clock ahead of the router's. A short walk's end, heard before its start, opens
|
||||
// nothing when its start comes, whether the end said the walk's start or only its own moment; the next walk
|
||||
// under its id, started after that end, still opens.
|
||||
func TestAStartHeardAfterItsEndOpensNothingWhenTheControllersClockIsAhead(t *testing.T) {
|
||||
for _, said := range []string{"both", "end only"} {
|
||||
t.Run(said, func(t *testing.T) {
|
||||
h, _, _, c, _ := newHolder(t)
|
||||
ended := c.now().Add(10 * time.Minute) // by the controller's clock
|
||||
started := ended.Add(-20 * time.Second)
|
||||
body := `{"walk":"plan-k","phase":"ended","outcome":"delivered","touches":[{"machine":"novox","modules":["mesh-tools"]}],` +
|
||||
`"ended_at":"` + ended.Format(time.RFC3339Nano) + `"`
|
||||
if said == "both" {
|
||||
body += `,"started_at":"` + started.Format(time.RFC3339Nano) + `"`
|
||||
}
|
||||
end, _, err := DecodeWalk([]byte(body + `}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.Walked(end)
|
||||
c.pass(time.Second)
|
||||
walk(t, h, "plan-k", WalkStarted, "", started, 600, novox("mesh-tools"))
|
||||
if len(h.walks) != 0 {
|
||||
t.Fatalf("a start heard after its end opened a walk: %+v", h.walks)
|
||||
}
|
||||
walk(t, h, "plan-k", WalkStarted, "", ended.Add(time.Minute), 600, novox("mesh-tools"))
|
||||
if len(h.walks) != 1 {
|
||||
t.Fatalf("the next walk under its id, started after the end, did not open: %+v", h.walks)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -73,7 +73,6 @@
|
||||
"mesh-controller.condition-raised",
|
||||
"mesh-controller.condition-changed",
|
||||
"mesh-controller.condition-cleared",
|
||||
"mesh-controller.plan-moved",
|
||||
"intake.choice",
|
||||
"intake.link",
|
||||
"intake.failed",
|
||||
@@ -94,8 +93,7 @@
|
||||
"name": "asks",
|
||||
"ttl-seconds": 2592000
|
||||
},
|
||||
"identities",
|
||||
"walks"
|
||||
"identities"
|
||||
],
|
||||
"own-secrets": {
|
||||
"broker": "${dir:state}/broker"
|
||||
@@ -131,7 +129,7 @@
|
||||
"path": "${dir:state}/settings.json",
|
||||
"mode": "0600",
|
||||
"merge": "json",
|
||||
"content": "{\n \"time-zone\": \"\",\n \"only-what-needs-me\": []\n}\n",
|
||||
"content": "{\n \"time-zone\": \"\"\n}\n",
|
||||
"owner": "messenger"
|
||||
}
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user