pacman keeps one mirror list for the mesh, ranked in the write that opens each round and written by every node (ADR 0585, issue 1876)

Each node's reflector ranked its own list on its own timer, and on 2026-10-31 one node's
lagging first mirror left it a Node.js version behind the rest. The list is now the
module's state mirrors, key list: the node that opens a week's round ranks it with the
module's reflector options, keeps only mirrors whose last sync is no older than the
newest found, and the round's key names the list's revision. Every keeper writes
/etc/pacman.d/mirrorlist whole from the key through the root helper's new verb
mirrorlist, when the key changes and when the file differs, and never writes an empty
or unreadable list. reflector.timer is declared stopped and disabled, and reflector.conf
saves nothing. pacman_mirrors answers the shared list, its ranker and age and whether
this node's file matches; with refresh it asks this node's keeper to rank again,
refused while a round runs, naming it. The round's integration test proves the opening
write and both nodes' files.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
This commit is contained in:
2026-11-01 16:18:22 +01:00
parent 37613ec6a9
commit 2b495c4818
19 changed files with 770 additions and 111 deletions
+16 -9
View File
@@ -17,10 +17,15 @@ that seat (ADR 0207).
- The manifest test runs `pacman-conf` on the rendered file and checks the repository list and the
options as pacman reads them. It skips that check where `pacman-conf` is absent.
- The host keeps the machine's previous file once, the first time it writes over it (ADR 0102).
- **Mirrors.** The `reflector` package, `/etc/xdg/reflector/reflector.conf` written whole (https,
Belgium, the Netherlands, Luxembourg, Germany, France, the 20 most recently synced, sorted by
rate, saved to `/etc/pacman.d/mirrorlist`), and `reflector.timer` running and enabled. The mirror
list stays reflector's to write, not the mesh's.
- **One mirror list for the mesh (ADR 0585).** The `reflector` package as the ranking tool,
`/etc/xdg/reflector/reflector.conf` written whole with its options (https, Belgium, the Netherlands,
Luxembourg, Germany, France, the 20 most recently synced, sorted by rate; nothing saved), and
`reflector.timer` declared stopped and disabled. The list is the module's state `mirrors`, one key
`list`: the node that opens a week's round ranks it with those options inside the write that opens
the round, keeps only the mirrors whose last sync is no older than the newest found, and the
round's key names the list's revision. Every node's keeper writes `/etc/pacman.d/mirrorlist` whole
from it through the root helper when the key changes and when the file differs, so a hand edit
comes back; an empty or unreadable list is never written.
- **Cache cleaning.** The `pacman-contrib` package and `paccache.timer` running and enabled. Each
week it keeps the last three versions of each package.
- **The package view on the bus (ADR 0507).** A process of its own, the keeper (`pacman-tools keeper`, account
@@ -65,15 +70,17 @@ that seat (ADR 0207).
## What it improves
- Mirrors were generated once and never again: in 2022, 2023 and 2024, and on one machine by its
hosting provider's installer. The list is now refreshed weekly. The timer's first run is at the
next weekly boundary; `pacman_mirrors` with `refresh: true` runs it at once.
hosting provider's installer. Then each node's reflector ranked its own list weekly, and on
2026-10-31 one node's lagging first mirror left it a version behind the rest. Now every node of a
round upgrades from one list ranked when the round opened; `pacman_mirrors` with `refresh: true`
ranks it again for the mesh, refused while a round runs.
- Package caches were never cleaned. One workstation held 48 GB, of which paccache would free 33 GB.
- Every machine has the same options. Only one had parallel downloads.
## What it leaves found
- `/etc/pacman.d/mirrorlist`, which reflector rewrites, and the stale `mirrorlist.pacnew`,
`.bak`, `.original` and similar copies beside it.
- The stale `mirrorlist.pacnew`, `.bak`, `.original` and similar copies beside the mirror list.
Until the first round opens (or a refresh), each node keeps the mirror list it has.
- `/etc/pacman.d/hooks`, the keyring, and the AUR helper. Packages from outside the repositories
are research 027 question 1.
@@ -97,7 +104,7 @@ that seat (ADR 0207).
| `pacman_remove_orphans` | a | `pacman -Rs` on named orphans, or all of them, as a unit of its own; a name that is not an orphan is refused |
| `pacman_cache` | r/a | size, interrupted downloads, what paccache would free keeping N; `clean: true` removes them |
| `pacman_history` | r | `/var/log/pacman.log`: installs, upgrades, downgrades, reinstalls and removals since a day, and the last full upgrade |
| `pacman_mirrors` | r/a | the list, its generator and age, reflector's options, timer and last run; `refresh: true` starts reflector |
| `pacman_mirrors` | r/a | the mesh's list, who ranked it and its age, and whether this node's file matches; `refresh: true` ranks it again for the mesh, refused during a round |
| `pacman_foreign` | r | the foreign packages, from `installed` on the bus |
| `pacman_news` | r | the distribution's news since the last full upgrade (or a day), over https; no network is `reachable: false` |
| `pacman_config` | r | `pacman-conf`: options, repositories, and whether `/etc/pacman.conf` is the module's |
@@ -34,7 +34,8 @@ var busFields = map[string][]string{
"RollbackItem": {"name", "from", "to", "source", "location"},
"Record": {"node", "run", "at", "installed", "kernel", "failed_units", "dkms"},
"DKMSModule": {"module", "version", "kernels"},
"Round": {"round", "opened", "opened_by", "order"},
"Round": {"round", "opened", "opened_by", "order", "mirrors", "mirrors_note"},
"MirrorList": {"servers", "ranked_at", "ranked_by", "options", "newest_sync", "round"},
"RoundNode": {"round", "node", "run", "pass"},
"Pass": {"reason", "ask", "asked", "answer", "answered"},
"Ask": {"id", "node", "act", "at", "of", "remove_added", "round", "pass", "reason"},
@@ -50,7 +51,7 @@ var busFields = map[string][]string{
}
// The value of each state's keys.
var stateValues = []any{InstalledValue{}, NotCurrent{}, UpdatesValue{}, Run{}, Record{}, Round{}, RoundNode{}, Ask{}}
var stateValues = []any{InstalledValue{}, NotCurrent{}, UpdatesValue{}, Run{}, Record{}, Round{}, RoundNode{}, Ask{}, MirrorList{}}
var stamp = time.Date(2026, 10, 24, 4, 0, 12, 0, time.UTC)
@@ -64,6 +65,8 @@ func fill(v reflect.Value) {
v.SetBool(true)
case reflect.Int, reflect.Int64:
v.SetInt(1)
case reflect.Uint64:
v.SetUint(1)
case reflect.Pointer:
v.Set(reflect.New(v.Type().Elem()))
fill(v.Elem())
+38 -3
View File
@@ -8,7 +8,8 @@ package main
//
// - pacman's database lock released (an inotify watch on its directory: the lock file deleted) ends a
// transaction, whoever ran it: `installed` and then `updates` are read again;
// - the mirror list rewritten (reflector writes /etc/pacman.d/mirrorlist): `updates` is read again;
// - the mirror list rewritten (the keeper writes /etc/pacman.d/mirrorlist from the mesh's list, ADR 0585): it is
// written again when it differs from that list, and `updates` is read again;
// - a day after `updates` was last read: it is read again, since the repositories change what an upgrade
// would do without any transaction on the node.
//
@@ -138,6 +139,8 @@ type Hooks struct {
AfterTransaction []func(k *Keeper, t Transaction) error
Changes []StateHook
Wakes []Wake
// MirrorsRewritten: after the mirror list file was written or replaced, before `updates` is read again.
MirrorsRewritten []func(k *Keeper) error
}
// keeperHooks is the hooks the keeper runs with, built once the keeper is connected: the one place a part of
@@ -159,7 +162,15 @@ func keeperHooks(k *Keeper) Hooks {
u := &Upgrades{Node: k.Node, Config: k.Config, Machine: k.Machine, Root: root, Events: k.Events,
Installed: k.State(StateInstalled), Updates: k.State(StateUpdates), Runs: k.State(StateRuns),
Records: k.State(StateRecords), Rounds: k.State(StateRounds), Asks: k.State(StateAsks), AskRecord: k.AskRecord,
Check: k.Check}
Check: k.Check, Mirrors: k.State(StateMirrors), Rank: k.Rank}
// writeMirrors writes this node's mirror list file from the shared list when they differ (ADR 0585 §3).
writeMirrors := func(*Keeper) error {
wrote, err := u.WriteMirrors()
if wrote {
k.logf("[pacman] keeper: %s written from the mesh's mirror list", Mirrorlist)
}
return err
}
k.upgrades = u
const roundWake = 1 // the round's step is the second wake
roundStep := func(k *Keeper, now time.Time) time.Time {
@@ -199,7 +210,16 @@ func keeperHooks(k *Keeper) Hooks {
}},
{State: StateRounds, Key: "**", Handle: onRoundChange},
{State: StateRuns, Key: "**", Handle: onRoundChange},
// The mesh's mirror list: written to this node's file at the keeper's start and on every change (ADR 0585 §3).
{State: StateMirrors, Key: MirrorsKey, Handle: func(k *Keeper, c stdio.StateChange) error {
if c.Op != "put" {
return nil
}
return writeMirrors(k)
}},
},
// And the file written again when it differs from the list: a hand edit, or reflector run by hand.
MirrorsRewritten: []func(*Keeper) error{writeMirrors},
Wakes: []Wake{
func(k *Keeper, now time.Time) time.Time {
if !now.Equal(k.started) {
@@ -239,6 +259,10 @@ func (noRoot) Rollback(run string, plan []RollbackItem) error {
return errors.New(noRootSaid + ": no rollback can start")
}
func (noRoot) Mirrorlist([]string) error {
return errors.New(noRootSaid + ": no mirror list can be written")
}
func (noRoot) Maps() ([]Mapped, error) {
return nil, errors.New(noRootSaid + ": no process's maps can be read")
}
@@ -267,9 +291,11 @@ type Keeper struct {
// AskRecord reads the operator channel's record of one of the module's asks (process.Conn.Ask).
AskRecord func(id string) (*stdio.StateEntry, error)
// Root carries out the acts that need root; nil is the root helper's client (newRootActs). Check is the
// after-upgrade check; nil is the machine's (AfterCheck). A test sets both.
// after-upgrade check; nil is the machine's (AfterCheck). Rank ranks the mesh's mirrors (RankMirrors, set by the
// process); nil ranks none. A test sets them.
Root RootActs
Check func(run Run, record *Record) Check
Rank func(conf []byte, round string) (MirrorList, error)
upgrades *Upgrades
@@ -390,6 +416,11 @@ func (k *Keeper) signal(s Signal) {
}
}
case MirrorsRewritten:
for _, h := range k.Hooks.MirrorsRewritten {
if err := h(k); err != nil {
k.logf("[pacman] keeper: the mirror list rewritten: %v", err)
}
}
k.refreshUpdates()
}
}
@@ -606,6 +637,10 @@ func keeper() int {
DBChanged: func() time.Time { return modified(filepath.Join(PacmanDBDir, "local")) },
MakeHooks: keeperHooks,
}
// It ranks the mesh's mirrors with reflector when it opens a round or is asked a refresh (ADR 0585 §2, §4).
k.Rank = func(conf []byte, round string) (MirrorList, error) {
return RankMirrors(conf, k.Node, k.Clock.Now(), round)
}
logf("[pacman] keeper of %s is starting as %s", account.Node, account.User)
if err := runKeeper(ctx, account, k); err != nil && ctx.Err() == nil {
logf("[pacman] keeper stopped: %v", err)
@@ -118,8 +118,26 @@ func roundNodeOf(s *fakeStore, round, node string) RoundNode {
// 1841/AC6 through the keeper's loop: the first of the order opens the week's round at its start on Saturday
// 04:00 and starts its run in it; the round's parts get the operator channel's reader and the rounds state.
//
// ADR 0585 §2, §3: it ranks the mesh's mirrors inside the write that opens the round, before the round's key, which names
// the list's revision; and its keeper writes the list to its node's file.
func TestTheFirstNodesKeeperOpensTheRoundAndTakesItsTurn(t *testing.T) {
k, states, _ := roundKeeper(t, "node-a", []string{"node-a", "node-b"})
root := &fakeRoot{}
k.Root = root
servers := []string{"https://nl.mirror.example/archlinux/$repo/os/$arch"}
k.Rank = func(conf []byte, round string) (MirrorList, error) {
if string(conf) != "--sort rate\n" {
t.Errorf("ranked with %q", conf)
}
return MirrorList{Servers: servers, RankedAt: t0, RankedBy: "node-a", Round: round}, nil
}
k.Machine.ReadFile = func(path string) ([]byte, error) {
if path == ReflectorConf {
return []byte("--sort rate\n"), nil
}
return nil, errNoFile
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go func() { _ = k.Run(ctx) }()
@@ -130,6 +148,14 @@ func TestTheFirstNodesKeeperOpensTheRoundAndTakesItsTurn(t *testing.T) {
if rn := roundNodeOf(rounds, round, "node-b"); rn.Run != "" {
t.Errorf("node-b started in node-a's keeper: %+v", rn)
}
var opened Round
e, _ := rounds.Get(round)
_ = json.Unmarshal(e.Value, &opened)
list, rev, err := readMirrorList(states.store(StateMirrors))
if err != nil || list == nil || list.Round != round || opened.Mirrors != rev || rev == 0 || opened.MirrorsNote != "" {
t.Errorf("the round opened with mirrors %d (%q), the list is %+v at %d (%v)", opened.Mirrors, opened.MirrorsNote, list, rev, err)
}
until(t, "the list written to the node's file", func() bool { w := root.written(); return len(w) == 1 && w[0][0] == servers[0] })
}
// 1841/AC6 through the keeper's loop: a node waits until the node before it passed its check, and starts on
@@ -169,4 +195,64 @@ func TestTheKeeperHandsTheOperatorChannelsReaderToTheRound(t *testing.T) {
if len(h.Wakes) < 2 {
t.Fatalf("no wake for the round: %d wakes", len(h.Wakes))
}
// ADR 0585 §3: the node's file is written from the shared list when they differ, and never from a list that is empty
// or unreadable; the file is written again when it differs (the watch's MirrorsRewritten).
if len(h.MirrorsRewritten) != 1 {
t.Fatalf("the mirror list file is not written again when it changes: %d hooks", len(h.MirrorsRewritten))
}
root := &fakeRoot{}
u := k.upgrades
u.Root = root
file := []byte(nil)
u.Machine.ReadFile = func(path string) ([]byte, error) {
if path == Mirrorlist && file != nil {
return file, nil
}
return nil, errNoFile
}
mirrors := states.store(StateMirrors)
good := []string{"https://a.example/$repo/os/$arch", "https://b.example/archlinux/$repo/os/$arch"}
for _, step := range []struct {
name string
put any
file []byte
wrote bool
failed bool
}{
{name: "no shared list yet writes nothing"},
{name: "an empty list is never written", put: MirrorList{Servers: []string{}}, failed: true},
{name: "an unreadable list is never written", put: "not a list", failed: true},
{name: "a list the file differs from is written whole", put: MirrorList{Servers: good}, file: []byte(mirrorlistReflector), wrote: true},
{name: "a file holding the list is not written again", put: MirrorList{Servers: good}, file: RenderMirrorlist(good)},
} {
if step.put != nil {
mirrors.Put(MirrorsKey, step.put)
}
file = step.file
before := len(root.written())
wrote, err := u.WriteMirrors()
if wrote != step.wrote || (err != nil) != step.failed || len(root.written()) != before+map[bool]int{true: 1}[step.wrote] {
t.Errorf("%s: wrote %v (%v), the root helper wrote %v", step.name, wrote, err, root.written())
}
}
// ADR 0585 §4: a refresh asked of the keeper is refused while a round runs, and ranks the list when none does; the
// ask is taken once either way.
asks, rounds := states.store(StateAsks), states.store(StateRounds)
ranked := 0
u.Rank = func([]byte, string) (MirrorList, error) { ranked++; return MirrorList{Servers: good}, nil }
u.Machine.ReadFile = func(string) ([]byte, error) { return []byte("--sort rate\n"), nil }
rounds.Put("2026-W43", Round{Round: "2026-W43", Opened: t0, OpenedBy: "node-a", Order: []string{"node-a"}})
for _, running := range []bool{true, false} {
if !running {
rounds.Delete("2026-W43")
}
key := NodeKey("node-a", "ask-m")
asks.Put(key, Ask{ID: "ask-m", Node: "node-a", Act: ActMirrors, At: t0})
e, _ := asks.Get(key)
err := u.TakeAsk(stdio.StateChange{Op: "put", Key: key, Value: e.Value})
if left, _ := asks.Get(key); left != nil || (err != nil) != running || ranked != map[bool]int{true: 0, false: 1}[running] ||
(running && !strings.Contains(err.Error(), "round 2026-W43 runs")) {
t.Errorf("a refresh with a round running %v: %v, ranked %d, ask left %v", running, err, ranked, left)
}
}
}
+8 -3
View File
@@ -64,16 +64,21 @@ func (b *bounded) Write(p []byte) (int, error) {
// ExecRunner runs a command on this machine, in the C locale so what is parsed is one language.
func ExecRunner(ctx context.Context, name string, args ...string) Ran {
ctx, cancel := context.WithTimeout(ctx, CallTimeout)
return execWithin(ctx, CallTimeout, nil, name, args...)
}
// execWithin runs a command on this machine within timeout, in the C locale, with env added to this process's.
func execWithin(ctx context.Context, timeout time.Duration, env []string, name string, args ...string) Ran {
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
cmd := exec.CommandContext(ctx, name, args...)
cmd.Env = append(os.Environ(), "LC_ALL=C")
cmd.Env = append(append(os.Environ(), "LC_ALL=C"), env...)
var out, errb bounded
cmd.Stdout, cmd.Stderr = &out, &errb
err := cmd.Run()
r := Ran{Stdout: out.String(), Stderr: errb.String()}
if ctx.Err() == context.DeadlineExceeded {
r.Status, r.Err = 124, fmt.Sprintf("no answer within %d s", int(CallTimeout.Seconds()))
r.Status, r.Err = 124, fmt.Sprintf("no answer within %d s", int(timeout.Seconds()))
return r
}
var exit *exec.ExitError
+7 -4
View File
@@ -433,15 +433,18 @@ func tools(m *Machine) []stdio.Tool {
},
},
{
Name: "pacman_mirrors",
Description: "The mirror list in force (servers, commented ones, who generated it and when), reflector's options, its timer and its last run. With refresh: true, start reflector now (sudo -n), without waiting.",
Input: schema(map[string]any{"refresh": map[string]any{"type": "boolean", "description": "rank and rewrite the list now"}}),
Name: "pacman_mirrors",
Description: "The mesh's one mirror list from the bus (ADR 0585): its servers, when and by which node it was ranked, its " +
"age and reflector's options, and whether this node's /etc/pacman.d/mirrorlist matches it (the file as found when " +
"not), with reflector's timer, which stays disabled. With refresh: true, this node's keeper ranks the list again " +
"for the whole mesh and every node writes it; refused while an upgrade round runs, naming the round.",
Input: schema(map[string]any{"refresh": map[string]any{"type": "boolean", "description": "rank the mesh's list again now"}}),
Run: func(args map[string]any) (any, error) {
refresh, err := flag(args, "refresh")
if err != nil {
return nil, err
}
return m.MirrorList(refresh)
return m.MirrorsView(refresh)
},
},
{
@@ -24,9 +24,11 @@ func TestItHoldsThePackageManagerSeatAndDeclaresItsPackage(t *testing.T) {
t.Errorf("%s: %v", id, r)
}
}
for id, unit := range map[string]string{"mirror-refresh": "reflector.timer", "cache-cleaning": "paccache.timer"} {
// ADR 0585 §3: reflector's weekly timer is declared stopped and disabled, the mesh's list ranked in the round instead.
for id, unit := range map[string][3]string{"mirror-refresh": {"reflector.timer", "stopped", "disabled"},
"cache-cleaning": {"paccache.timer", "running", "enabled"}} {
r := m.resource(t, id)
if r["unit"] != unit || r["state"] != "running" || r["boot"] != "enabled" {
if r["unit"] != unit[0] || r["state"] != unit[1] || r["boot"] != unit[2] {
t.Errorf("%s: %v", id, r)
}
}
@@ -103,7 +105,7 @@ func provenConf(t *testing.T, content, dir string) string {
return strings.ReplaceAll(content, "Include = "+Mirrorlist, "Include = "+list)
}
func TestReflectorWritesTheListPacmanReads(t *testing.T) {
func TestReflectorRanksWithTheModulesOptionsAndSavesNoList(t *testing.T) {
content := templateText(t, manifest(t).resource(t, "mirrors"))
opts := map[string]string{}
for _, l := range strings.Split(content, "\n") {
@@ -113,7 +115,8 @@ func TestReflectorWritesTheListPacmanReads(t *testing.T) {
k, v, _ := strings.Cut(l, " ")
opts[k] = v
}
if opts["--save"] != Mirrorlist || opts["--protocol"] != "https" || opts["--latest"] != "20" || opts["--sort"] != "rate" || opts["--country"] == "" {
// ADR 0585 §3: nothing saves a list but the keeper, from the mesh's.
if _, saves := opts["--save"]; saves || opts["--protocol"] != "https" || opts["--latest"] != "20" || opts["--sort"] != "rate" || opts["--country"] == "" {
t.Fatalf("%v", opts)
}
if manifest(t).resource(t, "mirrors")["path"] != ReflectorConf {
+340 -41
View File
@@ -1,36 +1,62 @@
package main
// The mirror list and its refresher (novox/hq to-be 42 Phase 1). On 2026-10-04 every machine's list
// had been generated once — by a tool no longer installed, or by a hosting provider's installer — and
// never again. The module installs reflector, owns its configuration and enables its weekly timer;
// this reads the list and the refresher's last run, and starts a refresh on demand.
// The mesh's one mirror list (novox/hq ADR 0585). Before it each node's reflector ranked its own list on its own weekly
// timer (to-be 42 Phase 1), and on 2026-10-31 one node's first mirror lagged, so its upgrade found nothing to install
// while the others moved on. Now:
//
// - the list is one key of the module's own state, `mirrors.list` (§1): the servers in order, when and by which node
// they were ranked, and the options;
// - it is ranked once per weekly round, by the node that opens it, inside the write that opens it (§2): reflector
// ranks with the module's options (reflector.conf, its `--save` left out), only the mirrors whose last sync is no
// older than the newest found are kept, and the round's key names the list's revision;
// - each node's keeper writes /etc/pacman.d/mirrorlist whole from the key, through the root helper, when the key
// changes and when the file differs (the keeper watches the file), so a hand edit comes back; a list that is
// empty or cannot be read is never written, and the node keeps its last (§3); reflector stays installed as the
// ranking tool, and its weekly timer is declared stopped and disabled;
// - `pacman_mirrors` answers the shared list, its age, its ranker and whether this node's file matches; with
// refresh it asks this node's keeper to rank again for the mesh, refused while a round runs, naming it (§4).
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"os"
"regexp"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Where the list is and how reflector is told to write it.
// Where the list is and where the module's reflector options are.
const (
Mirrorlist = "/etc/pacman.d/mirrorlist"
ReflectorConf = "/etc/xdg/reflector/reflector.conf"
)
// Mirrors is the mirror list and its refresher.
// RankTimeout bounds one ranking: reflector downloads from every candidate mirror to rate it, which takes longer than
// a tool call may (CallTimeout), so the keeper ranks, never a call.
const RankTimeout = 3 * time.Minute
// mostMirrors bounds a list the root helper writes: far more than a ranking keeps (the module's --latest).
const mostMirrors = 100
// Mirrors is a mirror list file as found: the servers in force, those commented out, and the generator's header.
type Mirrors struct {
Servers []string `json:"servers"`
Commented int `json:"commented_servers"`
GeneratedBy string `json:"generated_by,omitempty"`
When string `json:"generated_when,omitempty"`
Reflector []string `json:"reflector_options"`
Timer map[string]string `json:"reflector_timer,omitempty"`
LastRun map[string]string `json:"reflector_last_run,omitempty"`
Refreshing bool `json:"refresh_started"`
Note string `json:"note,omitempty"`
Servers []string `json:"servers"`
Commented int `json:"commented_servers"`
GeneratedBy string `json:"generated_by,omitempty"`
When string `json:"generated_when,omitempty"`
}
// mirrorlistHeader is the first line of the file the mesh writes, by which it is known as the mesh's.
const mirrorlistHeader = "# The mesh's mirror list (module pacman, novox/hq ADR 0585)"
// ParseMirrorlist reads the servers in force, those commented out, and the generator's header.
func ParseMirrorlist(text string) Mirrors {
m := Mirrors{Servers: []string{}, Reflector: []string{}}
m := Mirrors{Servers: []string{}}
for _, l := range lines(text) {
l = strings.TrimSpace(l)
switch {
@@ -40,6 +66,8 @@ func ParseMirrorlist(text string) Mirrors {
}
case strings.HasPrefix(strings.TrimLeft(l, "# "), "Server"):
m.Commented++
case l == mirrorlistHeader:
m.GeneratedBy = "the mesh"
case strings.Contains(l, "generated by Reflector"):
m.GeneratedBy = "reflector"
case strings.HasPrefix(l, "# When:"):
@@ -51,36 +79,307 @@ func ParseMirrorlist(text string) Mirrors {
return m
}
// MirrorList answers the list, reflector's options, its timer and its last run; refresh starts
// reflector now, without waiting, since ranking mirrors by rate takes longer than a call may.
func (m *Machine) MirrorList(refresh bool) (Mirrors, error) {
text, err := m.ReadFile(Mirrorlist)
if err != nil {
return Mirrors{}, err
// mirrorServer is a server line's value pacman takes from the mesh: http(s), a host, a path, and the repository and
// architecture left to pacman. Nothing else reaches the file: not a newline, not a second option.
var mirrorServer = regexp.MustCompile(`^https?://[A-Za-z0-9.-]+(:[0-9]+)?(/[A-Za-z0-9._~%+@-]+)*/\$repo/os/\$arch$`)
// checkServers refuses a list that is empty, too long, or holds anything but a server.
func checkServers(servers []string) error {
if len(servers) == 0 {
return errors.New("the list is empty")
}
out := ParseMirrorlist(string(text))
if conf, err := m.ReadFile(ReflectorConf); err == nil {
for _, l := range lines(string(conf)) {
if l = strings.TrimSpace(l); !strings.HasPrefix(l, "#") {
out.Reflector = append(out.Reflector, l)
if len(servers) > mostMirrors {
return fmt.Errorf("the list holds %d servers, more than %d", len(servers), mostMirrors)
}
for _, s := range servers {
if !mirrorServer.MatchString(s) {
return fmt.Errorf("%q is not a mirror's server", s)
}
}
return nil
}
// RenderMirrorlist is the mirror list file whole, as every node writes it from the shared list: the same servers are
// the same bytes, so a node compares its file with the list by its bytes.
func RenderMirrorlist(servers []string) []byte {
var b bytes.Buffer
b.WriteString(mirrorlistHeader + "\n")
b.WriteString("# Written whole from the bus (the state mirrors of the module pacman): a hand edit is written back.\n")
b.WriteString("# pacman_mirrors says when it was ranked and by which node.\n")
for _, s := range servers {
b.WriteString("Server = " + s + "\n")
}
return b.Bytes()
}
// ReflectorOptions is the options reflector.conf sets, each line split into its flag and its value as reflector reads
// them: a `--save` is left out, since the keeper writes the file (ADR 0585 §3).
func ReflectorOptions(conf string) []string {
out := []string{}
for _, l := range lines(conf) {
l = strings.TrimSpace(l)
if l == "" || strings.HasPrefix(l, "#") {
continue
}
flag, value, _ := strings.Cut(l, " ")
if flag == "--save" {
continue
}
out = append(out, flag)
if value = strings.Trim(strings.TrimSpace(value), `"'`); value != "" {
out = append(out, value)
}
}
return out
}
// rankedMirror is one mirror reflector ranked, in its order, with its last sync (zero when the status says none).
type rankedMirror struct {
URL string
LastSync time.Time
}
// reflectorTime is how reflector's --info writes a last sync.
const reflectorTime = "2006-01-02 15:04:05 MST"
// ParseReflectorInfo reads reflector's --info: one block per mirror in the order it ranked them, its server first and
// then `field : value` lines, of which last_sync is read.
func ParseReflectorInfo(text string) []rankedMirror {
var out []rankedMirror
open := false
for _, l := range strings.Split(text, "\n") {
l = strings.TrimSpace(l)
switch {
case l == "":
open = false
case !open && strings.Contains(l, "://") && !strings.Contains(l, " "):
out, open = append(out, rankedMirror{URL: l}), true
case open:
if k, v, ok := strings.Cut(l, ":"); ok && strings.TrimSpace(k) == "last_sync" {
if t, err := time.Parse(reflectorTime, strings.TrimSpace(v)); err == nil {
out[len(out)-1].LastSync = t.UTC()
}
}
}
}
if t, err := m.unitProps("reflector.timer", "LoadState", "ActiveState", "UnitFileState", "LastTriggerUSec", "NextElapseUSecRealtime"); err == nil {
out.Timer = t
}
if s, err := m.unitProps("reflector.service", "LoadState", "ActiveState", "Result", "ExecMainExitTimestamp", "ExecMainStatus"); err == nil {
out.LastRun = s
}
if out.Timer["LoadState"] == "not-found" {
out.Note = "reflector is not installed here; the module installs it"
}
if refresh {
if _, err := m.Root("systemctl", "start", "--no-block", "reflector.service"); err != nil {
return out, err
return out
}
// Freshest keeps, in their order, only the mirrors whose last sync is no older than the newest found (ADR 0585 §2), so
// a mirror lagging behind the others is never in the list; one whose last sync is unknown is left out.
func Freshest(ranked []rankedMirror) (servers []string, newest time.Time) {
for _, m := range ranked {
if m.LastSync.After(newest) {
newest = m.LastSync
}
out.Refreshing = true
out.Note = "reflector is ranking mirrors now; call again in a minute for the new list"
}
for _, m := range ranked {
if !m.LastSync.IsZero() && !m.LastSync.Before(newest) {
servers = append(servers, m.URL)
}
}
return servers, newest
}
// reflectorRun runs reflector with args on this machine, within RankTimeout; its cache in the temporary directory, since
// the keeper's account has no home of its own. A test replaces it.
var reflectorRun = func(ctx context.Context, args ...string) Ran {
return execWithin(ctx, RankTimeout, []string{"XDG_CACHE_HOME=" + os.TempDir()}, "reflector", args...)
}
// RankMirrors ranks the mirrors with reflector and the module's options (conf, reflector.conf's text) and keeps the
// freshest: the mesh's list, ranked by node at now for round (none for a refresh). A ranking that fails or keeps no
// mirror is an error, never an empty list (ADR 0585 §3).
func RankMirrors(conf []byte, node string, now time.Time, round string) (MirrorList, error) {
opts := ReflectorOptions(string(conf))
r := reflectorRun(context.Background(), append(append([]string(nil), opts...), "--info")...)
if r.Status != 0 || r.Err != "" {
return MirrorList{}, failure("reflector", "reflector", r)
}
ranked := ParseReflectorInfo(r.Stdout)
servers, newest := Freshest(ranked)
if err := checkServers(servers); err != nil {
return MirrorList{}, fmt.Errorf("reflector ranked no list the mesh can use (%d mirrors read): %w", len(ranked), err)
}
return MirrorList{Servers: servers, RankedAt: now.UTC(), RankedBy: node, Options: opts, Newest: newest, Round: round}, nil
}
// readMirrorList is the shared list and its revision; nil when there is none yet.
func readMirrorList(r interface {
Get(key string) (*stdio.StateEntry, error)
}) (*MirrorList, uint64, error) {
e, err := r.Get(MirrorsKey)
if err != nil || e == nil {
return nil, 0, err
}
var l MirrorList
if err := json.Unmarshal(e.Value, &l); err != nil {
return nil, e.Revision, fmt.Errorf("%s.%s is not readable: %w", StateMirrors, MirrorsKey, err)
}
return &l, e.Revision, nil
}
// ---- the keeper's part -------------------------------------------------------------------------------------------
// rankMirrors ranks the mesh's list and writes it to the shared key, answering its revision.
func (u *Upgrades) rankMirrors(round string) (uint64, error) {
if u.Rank == nil || u.Mirrors == nil {
return 0, errors.New("this keeper ranks no mirrors")
}
conf, err := u.Machine.ReadFile(ReflectorConf)
if err != nil {
return 0, fmt.Errorf("the module's reflector options cannot be read: %w", err)
}
list, err := u.Rank(conf, round)
if err != nil {
return 0, err
}
return u.Mirrors.Put(MirrorsKey, list)
}
// mirrorsForRound ranks the list in the write that opens round (ADR 0585 §2): the revision the round opens with, and
// why it was not ranked again when it was not, the round then opening with the list there is.
func (u *Upgrades) mirrorsForRound(round string) (uint64, string) {
rev, err := u.rankMirrors(round)
if err == nil {
return rev, ""
}
note := "the mirrors were not ranked again: " + err.Error()
if u.Mirrors != nil {
if _, kept, rerr := readMirrorList(u.Mirrors); rerr == nil && kept > 0 {
return kept, note + fmt.Sprintf("; the round opens with the list at revision %d", kept)
}
}
return 0, note + "; there is no shared list yet, and every node keeps its own"
}
// runningRound is the round that runs now, none when every node of the latest round passed: a refresh is refused
// while one runs, so every node of a round upgrades from the list it opened with (ADR 0585 §4).
func runningRound(rounds, runs Bucket, now time.Time) (string, error) {
id, err := latestRound(rounds)
if err != nil || id == "" {
return "", err
}
v, err := readRound(rounds, runs, id)
if err != nil || v == nil {
return "", err
}
for _, n := range RoundState(v, now, 0) {
if n.State != InRoundPassed {
return id, nil
}
}
return "", nil
}
// refreshRefused is the words of a refresh refused during round.
func refreshRefused(round string) error {
return fmt.Errorf("round %s runs: every node of it upgrades from the list it opened with, so the mirrors are ranked "+
"again only once every node of it passed (pacman_round says where it stands)", round)
}
// takeMirrorsAsk ranks the list again for the mesh on a refresh, unless a round runs.
func (u *Upgrades) takeMirrorsAsk() error {
if round, err := runningRound(u.Rounds, u.Runs, u.Machine.Now()); err != nil {
return err
} else if round != "" {
return refreshRefused(round)
}
_, err := u.rankMirrors("")
return err
}
// WriteMirrors writes this node's mirror list file whole from the shared list when they differ, through the root
// helper, answering whether it wrote. No list yet writes nothing; a list that is empty or cannot be read is never
// written, and the node keeps its last (ADR 0585 §3).
func (u *Upgrades) WriteMirrors() (bool, error) {
if u.Mirrors == nil {
return false, nil
}
list, _, err := readMirrorList(u.Mirrors)
if err != nil {
return false, fmt.Errorf("%v; this node keeps its mirror list", err)
}
if list == nil {
return false, nil
}
if err := checkServers(list.Servers); err != nil {
return false, fmt.Errorf("the shared mirror list is not written: %v; this node keeps its mirror list", err)
}
want := RenderMirrorlist(list.Servers)
if have, err := u.Machine.ReadFile(Mirrorlist); err == nil && bytes.Equal(have, want) {
return false, nil
}
if err := u.Root.Mirrorlist(list.Servers); err != nil {
return false, fmt.Errorf("writing %s: %w", Mirrorlist, err)
}
return true, nil
}
// ---- the tool -----------------------------------------------------------------------------------------------------
// MirrorsView answers pacman_mirrors (ADR 0585 §4): the shared list, its revision and age and who ranked it, and this
// node's file beside it; with refresh, the ask of this node's keeper to rank again for the mesh, refused while a round
// runs.
func (m *Machine) MirrorsView(refresh bool) (map[string]any, error) {
if err := m.states(); err != nil {
return nil, err
}
out := map[string]any{"node": m.Node}
list, rev, err := readMirrorList(m.States(StateMirrors))
if err != nil {
return nil, err
}
out["shared"], out["revision"] = list, rev
text, ferr := m.ReadFile(Mirrorlist)
if ferr != nil {
out["file_error"] = ferr.Error()
}
file := ParseMirrorlist(string(text))
if list == nil {
out["note"] = "no shared list yet: it is ranked when the next round opens, or by pacman_mirrors with refresh"
out["file"] = file
} else {
out["age"] = m.Now().Sub(list.RankedAt).Round(time.Minute).String()
out["ranked_by"] = list.RankedBy
matches := ferr == nil && bytes.Equal(text, RenderMirrorlist(list.Servers))
out["file_matches"] = matches
if !matches {
out["file"] = file
}
}
if t, err := m.unitProps("reflector.timer", "LoadState", "UnitFileState", "ActiveState"); err == nil {
out["reflector_timer"] = t
}
if !refresh {
return out, nil
}
now := m.Now()
round, err := runningRound(upgradeState(StateRounds), upgradeState(StateRuns), now)
if err != nil {
return nil, err
}
if round != "" {
return nil, refreshRefused(round)
}
a, err := AskMirrors(upgradeState(StateAsks), thisNode(), now)
if err != nil {
return nil, err
}
out["asked"] = a.ID
out["next"] = "the keeper of " + a.Node + " ranks the mirrors for the mesh now, which takes a minute or two; every " +
"node writes the new list once it lands, and pacman_mirrors says it"
return out, nil
}
// AskMirrors writes an ask for node's keeper to rank the mesh's mirrors again.
func AskMirrors(asks Bucket, node string, now time.Time) (Ask, error) {
if node == "" {
return Ask{}, errors.New("no node to ask: the runtime names none (MESH_NODE)")
}
a := Ask{ID: AskID(now), Node: node, Act: ActMirrors, At: now.UTC()}
if _, err := asks.Create(NodeKey(node, a.ID), a); err != nil {
return Ask{}, fmt.Errorf("writing the ask: %w", err)
}
return a, nil
}
+84 -21
View File
@@ -1,6 +1,7 @@
package main
import (
"context"
"strings"
"testing"
"time"
@@ -287,29 +288,91 @@ Server = https://mirror2.example.org/$repo/os/$arch
#Server = https://old.example.org/$repo/os/$arch
`
func TestMirrorsReadTheListAndRefreshWithoutWaiting(t *testing.T) {
var calls []call
m := machine(fake(func(c call) Ran {
if c.name == "systemctl" && c.args[0] == "show" {
return Ran{Stdout: "LoadState=loaded\nActiveState=active\nUnitFileState=enabled\n"}
}
return Ran{}
}, &calls), 1000)
m.ReadFile = func(p string) ([]byte, error) {
switch p {
case Mirrorlist:
return []byte(mirrorlistReflector), nil
case ReflectorConf:
return []byte("# comment\n--save /etc/pacman.d/mirrorlist\n--sort rate\n"), nil
}
return nil, errNoFile
// reflectorInfo is reflector's --info as it answered on 2026-11-01 (reflector 2023, its fields cut to those
// read and one more), the ranked order kept: two mirrors at the newest last sync, one behind, one with none.
const reflectorInfo = `https://nl.mirror.example/archlinux/$repo/os/$arch
active : True
country : Netherlands
last_sync : 2026-11-01 14:32:01 UTC
https://be.mirror.example/archlinux/$repo/os/$arch
active : True
last_sync : 2026-11-01 14:01:45 UTC
https://de.mirror.example/$repo/os/$arch
last_sync : 2026-11-01 14:32:01 UTC
https://fr.mirror.example/archlinux/$repo/os/$arch
last_sync : None
`
// ADR 0585 §2, §4: the ranking keeps only mirrors as fresh as the newest, in reflector's order, with the module's options
// and never its --save; a ranking that fails or keeps none is no list. pacman_mirrors answers the shared list, its
// ranker and age and whether this node's file matches it (the file as found when not); with refresh it asks this node's
// keeper to rank again, refused while a round runs, naming it.
func TestTheMeshsMirrorListIsRankedFreshAndAnsweredWithThisNodesFile(t *testing.T) {
conf := "# comment\n--save /etc/pacman.d/mirrorlist\n--country Belgium,Netherlands\n--sort rate\n"
if o := strings.Join(ReflectorOptions(conf), " "); o != "--country Belgium,Netherlands --sort rate" {
t.Errorf("the options are %q", o)
}
r, err := m.MirrorList(true)
if err != nil || len(r.Servers) != 2 || r.Commented != 1 || r.GeneratedBy != "reflector" || r.When != "2024-06-12 21:26:34 UTC" || len(r.Reflector) != 2 || !r.Refreshing {
t.Fatalf("%+v %v", r, err)
var asked []string
answer := Ran{Stdout: reflectorInfo}
saved := reflectorRun
reflectorRun = func(_ context.Context, args ...string) Ran { asked = args; return answer }
defer func() { reflectorRun = saved }()
l, err := RankMirrors([]byte(conf), "node-a", t0, "2026-W43")
if err != nil || strings.Join(l.Servers, " ") != "https://nl.mirror.example/archlinux/$repo/os/$arch https://de.mirror.example/$repo/os/$arch" ||
!l.Newest.Equal(time.Date(2026, 11, 1, 14, 32, 1, 0, time.UTC)) || l.RankedBy != "node-a" || l.Round != "2026-W43" ||
strings.Join(asked, " ") != "--country Belgium,Netherlands --sort rate --info" {
t.Fatalf("ranked %+v (%v), reflector asked %v", l, err, asked)
}
if calls[len(calls)-1].String() != "sudo -n systemctl start --no-block reflector.service" {
t.Fatalf("%v", calls[len(calls)-1])
for _, a := range []Ran{{Status: 1, Stderr: "error: no mirrors"}, {Stdout: "https://x.example/$repo/os/$arch\nlast_sync : None\n"}} {
answer = a
if _, err := RankMirrors([]byte(conf), "node-a", t0, ""); err == nil {
t.Errorf("a ranking from %+v is a list", a)
}
}
if checkServers([]string{"https://ok.example/$repo/os/$arch\nServer = https://evil.example/$repo/os/$arch"}) == nil {
t.Error("a server carrying a second line is taken")
}
states := &fakeStates{}
states.store(StateMirrors).Put(MirrorsKey, l)
file := RenderMirrorlist(l.Servers)
m := machine(func(context.Context, string, ...string) Ran {
return Ran{Stdout: "LoadState=loaded\nUnitFileState=disabled\nActiveState=inactive\n"}
}, 1000)
m.Node, m.States = "node-a", readerOf(states)
m.Now = func() time.Time { return t0.Add(90 * time.Minute) }
m.ReadFile = func(string) ([]byte, error) { return file, nil }
v, err := m.MirrorsView(false)
if err != nil || v["file_matches"] != true || v["ranked_by"] != "node-a" || v["age"] != "1h30m0s" || v["file"] != nil ||
v["reflector_timer"].(map[string]string)["UnitFileState"] != "disabled" {
t.Fatalf("the view is %v (%v)", v, err)
}
file = []byte(mirrorlistReflector)
if v, _ := m.MirrorsView(false); v["file_matches"] != false || v["file"].(Mirrors).GeneratedBy != "reflector" {
t.Errorf("a file other than the list is %v", v)
}
rounds, runs, asks := newBucket(), newBucket(), newBucket()
savedState, savedNode := upgradeState, thisNode
upgradeState = func(name string) Bucket {
return map[string]Bucket{StateRounds: rounds, StateRuns: runs, StateAsks: asks}[name]
}
thisNode = func() string { return "node-a" }
defer func() { upgradeState, thisNode = savedState, savedNode }()
rounds.Put("2026-W43", Round{Round: "2026-W43", Opened: t0, OpenedBy: "node-a", Order: []string{"node-a", "node-b"}})
if _, err := m.MirrorsView(true); err == nil || !strings.Contains(err.Error(), "round 2026-W43 runs") {
t.Errorf("a refresh during a round: %v", err)
}
if keys, _ := asks.Keys(); len(keys) != 0 {
t.Errorf("a refused refresh asked %v", keys)
}
rounds.Delete("2026-W43")
v, err = m.MirrorsView(true)
if keys, _ := asks.Keys(); err != nil || len(keys) != 1 || !strings.HasPrefix(keys[0], "node-a.") || v["asked"] == nil {
t.Errorf("a refresh asked %v (%v): %v", keys, err, v)
}
}
+40 -3
View File
@@ -51,6 +51,7 @@ const (
VerbRollback = "rollback"
VerbMaps = "maps"
VerbJournal = "journal"
VerbMirrorlist = "mirrorlist"
)
// The helper's environment, and the keeper's word for where the helper listens.
@@ -78,6 +79,8 @@ type RootRequest struct {
Plan []RollbackItem `json:"plan,omitempty"`
Unit string `json:"unit,omitempty"`
Lines int `json:"lines,omitempty"`
// Servers are the mirror list's servers, in order, for the verb mirrorlist.
Servers []string `json:"servers,omitempty"`
}
// RootAnswer is the helper's answer: an error, or what the verb read.
@@ -95,7 +98,9 @@ type RootHelper struct {
ProcRoot string
PlanDir string
Self string
Peer int
// MirrorlistPath is the mirror list file the verb mirrorlist writes: Mirrorlist, a test's own elsewhere.
MirrorlistPath string
Peer int
// PeerOf is the account at the other end of a connection; nil reads it from the socket (SO_PEERCRED).
PeerOf func(net.Conn) (int, error)
Logf func(format string, a ...any)
@@ -221,8 +226,17 @@ func (h *RootHelper) Handle(req RootRequest) RootAnswer {
return RootAnswer{Error: failure("journalctl", "journalctl", r).Error()}
}
return RootAnswer{Lines: lines(r.Stdout)}
case VerbMirrorlist:
if err := checkServers(req.Servers); err != nil {
return refused("%v", err)
}
if err := writeWhole(h.MirrorlistPath, RenderMirrorlist(req.Servers), 0o644); err != nil {
return RootAnswer{Error: "writing the mirror list: " + err.Error()}
}
return RootAnswer{}
}
return refused("%q is not a verb of the root helper (%s, %s, %s, %s)", req.Verb, VerbTransaction, VerbRollback, VerbMaps, VerbJournal)
return refused("%q is not a verb of the root helper (%s, %s, %s, %s, %s)", req.Verb, VerbTransaction, VerbRollback, VerbMaps,
VerbJournal, VerbMirrorlist)
}
// startUnit starts command as a transient unit of the service manager and answers once it started.
@@ -341,6 +355,28 @@ func (c rootClient) Rollback(run string, plan []RollbackItem) error {
return err
}
func (c rootClient) Mirrorlist(servers []string) error {
_, err := c.ask(RootRequest{Verb: VerbMirrorlist, Servers: servers})
return err
}
// writeWhole writes a file whole: into a file beside it, then renamed over it, so a reader never meets half of it.
func writeWhole(path string, content []byte, mode os.FileMode) error {
tmp := filepath.Join(filepath.Dir(path), "."+filepath.Base(path)+".mesh")
if err := os.WriteFile(tmp, content, mode); err != nil {
return err
}
if err := os.Chmod(tmp, mode); err != nil {
_ = os.Remove(tmp)
return err
}
if err := os.Rename(tmp, path); err != nil {
_ = os.Remove(tmp)
return err
}
return nil
}
func (c rootClient) Journal(unit string, n int) ([]string, error) {
a, err := c.ask(RootRequest{Verb: VerbJournal, Unit: unit, Lines: n})
return a.Lines, err
@@ -398,7 +434,8 @@ func rootHelper() int {
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
h := &RootHelper{Run: ExecRunner, Proc: osProc{}, ProcRoot: "/proc", PlanDir: filepath.Join(dir, "plans"), Self: self, Peer: uid, Logf: logf}
h := &RootHelper{Run: ExecRunner, Proc: osProc{}, ProcRoot: "/proc", PlanDir: filepath.Join(dir, "plans"), Self: self,
MirrorlistPath: Mirrorlist, Peer: uid, Logf: logf}
logf("[pacman] root helper listening on %s for account %s (%d)", sock, peerName, uid)
if err := h.Serve(ctx, l); err != nil {
logf("[pacman] root helper stopped: %v", err)
+18 -4
View File
@@ -34,10 +34,11 @@ func newRootBed(t *testing.T) *rootBed {
}
return Ran{}
},
Proc: stubProc{"/proc": {"1"}, "/proc/1/maps": {"7f00 r-xp 0 0:1 1 /usr/lib/libc.so.6 (deleted)"}, "/proc/1/cgroup": {"0::/system.slice/sshd.service"}, "/proc/1/exe": {"/usr/bin/sshd"}},
ProcRoot: "/proc",
PlanDir: t.TempDir(),
Self: "/var/lib/mesh/daemons/pacman-root/pacman-tools",
Proc: stubProc{"/proc": {"1"}, "/proc/1/maps": {"7f00 r-xp 0 0:1 1 /usr/lib/libc.so.6 (deleted)"}, "/proc/1/cgroup": {"0::/system.slice/sshd.service"}, "/proc/1/exe": {"/usr/bin/sshd"}},
ProcRoot: "/proc",
PlanDir: t.TempDir(),
Self: "/var/lib/mesh/daemons/pacman-root/pacman-tools",
MirrorlistPath: filepath.Join(t.TempDir(), "mirrorlist"),
}
return b
}
@@ -113,8 +114,16 @@ func TestTheRootHelperStartsARollbackWithThePlanInTheRequest(t *testing.T) {
}
}
// And (ADR 0585 §3) the verb mirrorlist writes the mirror list file whole from the servers it is sent, running nothing.
func TestTheRootHelperReadsTheReplacedFilesMappings(t *testing.T) {
b := newRootBed(t)
servers := []string{"https://a.example/$repo/os/$arch", "https://b.example/archlinux/$repo/os/$arch"}
if a := b.h.Handle(RootRequest{Verb: VerbMirrorlist, Servers: servers}); a.Error != "" {
t.Fatalf("the mirror list refused: %s", a.Error)
}
if got, err := os.ReadFile(b.h.MirrorlistPath); err != nil || string(got) != string(RenderMirrorlist(servers)) || len(b.ran) != 0 {
t.Fatalf("the mirror list file holds %q (%v), ran %v", got, err, b.ran)
}
a := b.h.Handle(RootRequest{Verb: VerbMaps})
want := []Mapped{{Unit: "sshd.service", Executable: "/usr/bin/sshd"}}
if a.Error != "" || !reflect.DeepEqual(a.Mapped, want) {
@@ -169,6 +178,11 @@ func TestTheRootHelperRefusesEverythingOffItsList(t *testing.T) {
"a removal with a location": {Verb: VerbRollback, Run: aRun, Plan: with(func(i *RollbackItem) {
i.Source, i.To = SourceRemoved, ""
})},
// ADR 0585 §3: never an empty mirror list, nor anything but a server in it.
"an empty mirror list": {Verb: VerbMirrorlist},
"a mirror list carrying a line": {Verb: VerbMirrorlist, Servers: []string{"https://a.example/$repo/os/$arch\nInclude = /tmp/x"}},
"a mirror list of a local file": {Verb: VerbMirrorlist, Servers: []string{"file:///tmp/$repo/os/$arch"}},
"a mirror list of another shape": {Verb: VerbMirrorlist, Servers: []string{"https://a.example/x"}},
}
for name, req := range cases {
b := newRootBed(t)
@@ -25,6 +25,7 @@ import (
type countingRoot struct {
mu sync.Mutex
started []string
mirrors []string
}
func (r *countingRoot) Transaction(kind, run string, ignore []string) error {
@@ -36,6 +37,20 @@ func (r *countingRoot) Transaction(kind, run string, ignore []string) error {
func (r *countingRoot) Rollback(string, []RollbackItem) error { return nil }
func (r *countingRoot) Journal(string, int) ([]string, error) { return nil, nil }
func (r *countingRoot) Maps() ([]Mapped, error) { return []Mapped{}, nil }
// Mirrorlist keeps the mirror list written last (ADR 0585 §3).
func (r *countingRoot) Mirrorlist(servers []string) error {
r.mu.Lock()
defer r.mu.Unlock()
r.mirrors = append([]string(nil), servers...)
return nil
}
func (r *countingRoot) mirrorlist() []string {
r.mu.Lock()
defer r.mu.Unlock()
return r.mirrors
}
func (r *countingRoot) count() int {
r.mu.Lock()
defer r.mu.Unlock()
@@ -69,20 +84,30 @@ func startRoundNode(t *testing.T, ctx context.Context, url, node string, order [
began := time.Now()
m.Now = func() time.Time { return t0.Add(time.Since(began)) }
m.ReadFile = func(path string) ([]byte, error) {
if path == OSRelease {
switch path {
case OSRelease:
return []byte("7.2.8-arch1-2\n"), nil
case ReflectorConf:
return []byte("--sort rate\n"), nil
}
return nil, errNoFile
}
n := &roundNode{root: &countingRoot{}, signals: make(chan Signal)}
k := &Keeper{Config: KeeperConfig{UpgradeOrder: order, RecordDays: 180, RoundStartMinutes: 120}, Machine: m, Clock: realClock{},
Watcher: fakeWatcher{n.signals}, Logf: func(f string, a ...any) { t.Logf(node+": "+f, a...) }, MakeHooks: keeperHooks,
Root: n.root, Check: func(Run, *Record) Check { return Check{Passed: passes} }}
Root: n.root, Check: func(Run, *Record) Check { return Check{Passed: passes} },
// The mesh's mirrors, ranked by the node that opens the round (ADR 0585 §2): reflector stood in for.
Rank: func(_ []byte, round string) (MirrorList, error) {
return MirrorList{Servers: roundMirrors, RankedAt: t0, RankedBy: node, Round: round}, nil
}}
account := process.Account{URL: url, Node: node, Module: "pacman", User: node + ".pacman.keeper", Password: "pw"}
go func() { _ = runKeeper(ctx, account, k) }()
return n
}
// roundMirrors is the mirror list the round's opener ranks.
var roundMirrors = []string{"https://nl.mirror.example/archlinux/$repo/os/$arch", "https://de.mirror.example/$repo/os/$arch"}
func roundBus(t *testing.T) (string, map[string]jetstream.KeyValue) {
t.Helper()
url := newTestBus(t)
@@ -152,6 +177,22 @@ func TestTheRoundGoesNodeByNodeOnARealBus(t *testing.T) {
if a.root.count() != 1 {
t.Errorf("node-a ran again: %v", a.root.started)
}
// ADR 0585 §2, §3: the round opened naming the revision of the list node-a ranked in the write that opened it, and
// both nodes wrote that list whole to their mirror list file, from the bus.
e, err := buckets[StateRounds].Get(context.Background(), round)
var opened Round
if err == nil {
_ = json.Unmarshal(e.Value(), &opened)
}
l, lerr := buckets[StateMirrors].Get(context.Background(), MirrorsKey)
if err != nil || lerr != nil || opened.Mirrors == 0 || opened.Mirrors != l.Revision() || !strings.Contains(string(l.Value()), `"round":"`+round+`"`) {
t.Errorf("the round opened with mirrors %d (%q), the list %v (%v, %v)", opened.Mirrors, opened.MirrorsNote, l, err, lerr)
}
for _, n := range []*roundNode{a, b} {
eventually(t, "each node wrote the mesh's mirror list", 10*time.Second, func() bool {
return strings.Join(n.root.mirrorlist(), " ") == strings.Join(roundMirrors, " ")
})
}
}
func TestAFailedCheckStopsTheRoundOnARealBus(t *testing.T) {
+9 -2
View File
@@ -238,8 +238,15 @@ func (u *Upgrades) RoundStep() (time.Time, error) {
soonest(opening.AddDate(0, 0, 7))
id := RoundID(now)
round := Round{Round: id, Opened: now.UTC(), OpenedBy: u.Node, Order: append([]string(nil), order...)}
if _, err := u.Rounds.Create(id, round); err != nil && !errors.Is(err, stdio.ErrChanged) {
errs = append(errs, fmt.Errorf("opening round %s: %w", id, err))
// The mirrors are ranked inside the write that opens the round, before its key, which names the list's
// revision (ADR 0585 §2): every node of the round upgrades from that list.
if e, err := u.Rounds.Get(id); err != nil {
errs = append(errs, fmt.Errorf("reading round %s: %w", id, err))
} else if e == nil {
round.Mirrors, round.MirrorsNote = u.mirrorsForRound(id)
if _, err := u.Rounds.Create(id, round); err != nil && !errors.Is(err, stdio.ErrChanged) {
errs = append(errs, fmt.Errorf("opening round %s: %w", id, err))
}
}
}
}
+26 -1
View File
@@ -35,10 +35,17 @@ const (
// StateAsks holds the acts asked of a node: key `<node>.<id>` an Ask, taken by that node's keeper alone
// (ADR 0508 §8).
StateAsks = "asks"
// StateMirrors holds the mesh's one mirror list: key `list` (MirrorsKey) a MirrorList, written by the keeper that
// ranks it, in the write that opens a round or on a refresh, and written to every node's mirror list file by that
// node's keeper (novox/hq ADR 0585).
StateMirrors = "mirrors"
)
// MirrorsKey is StateMirrors' one key.
const MirrorsKey = "list"
// States are the module's states, in the manifest's order.
var States = []string{StateInstalled, StateUpdates, StateRuns, StateRecords, StateRounds, StateAsks}
var States = []string{StateInstalled, StateUpdates, StateRuns, StateRecords, StateRounds, StateAsks, StateMirrors}
// The events the keeper emits (the keeper's `bus` `emits`, and the manifest's `emits`).
const (
@@ -319,6 +326,23 @@ type Round struct {
Opened time.Time `json:"opened"`
OpenedBy string `json:"opened_by"`
Order []string `json:"order"`
// Mirrors is the revision of `mirrors.list` the round opened with, ranked in the write that opened it (ADR 0585
// §2); MirrorsNote says why the list was not ranked again, when it was not (the round then opens with the list
// there is, at revision Mirrors, zero when there is none).
Mirrors uint64 `json:"mirrors,omitempty"`
MirrorsNote string `json:"mirrors_note,omitempty"`
}
// MirrorList is `mirrors.list`, the mesh's one mirror list (ADR 0585 §1): the servers in order, when and by which
// node they were ranked, the options reflector ranked them with, the newest last sync found among them (only mirrors as
// fresh as it are kept, §2), and the round it was ranked for (none for a refresh).
type MirrorList struct {
Servers []string `json:"servers"`
RankedAt time.Time `json:"ranked_at"`
RankedBy string `json:"ranked_by"`
Options []string `json:"options"`
Newest time.Time `json:"newest_sync"`
Round string `json:"round,omitempty"`
}
// RoundNode is `rounds.<round>.<node>`: the node's place in the round. Run is the run it started in its turn,
@@ -354,6 +378,7 @@ const (
ActUpgrade = "upgrade" // pacman_upgrade
ActRollback = "rollback" // pacman_rollback with apply
ActPass = "pass" // pacman_round with pass
ActMirrors = "mirrors" // pacman_mirrors with refresh: the mesh's list ranked again (ADR 0585 §4)
)
// Ask is `asks.<node>.<id>`: an act asked of a node, taken by that node's keeper alone, which answers in `runs`
@@ -403,6 +403,13 @@ func (u *Upgrades) TakeAsk(c stdio.StateChange) error {
return err
}
return u.Asks.Delete(c.Key)
case ActMirrors:
// Ranked once, whatever came of it: a refresh refused during a round or failed is said and not tried again.
err := u.takeMirrorsAsk()
if derr := u.Asks.Delete(c.Key); derr != nil {
err = errors.Join(err, fmt.Errorf("deleting the ask %s: %w", c.Key, derr))
}
return err
default:
return fmt.Errorf("ask %s asks %q, which this keeper does not carry out yet", c.Key, a.Act)
}
@@ -7,6 +7,7 @@ import (
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"time"
@@ -20,6 +21,25 @@ type fakeRoot struct {
journal []string
mapped []Mapped
mapsErr error
// mirrors is each mirror list written, whole (ADR 0585 §3), under mu: the keeper's loop writes it.
mu sync.Mutex
mirrors [][]string
}
func (f *fakeRoot) Mirrorlist(servers []string) error {
if f.fail != nil {
return f.fail
}
f.mu.Lock()
defer f.mu.Unlock()
f.mirrors = append(f.mirrors, append([]string(nil), servers...))
return nil
}
func (f *fakeRoot) written() [][]string {
f.mu.Lock()
defer f.mu.Unlock()
return append([][]string(nil), f.mirrors...)
}
func (f *fakeRoot) Transaction(kind, run string, ignore []string) error {
+6 -1
View File
@@ -43,6 +43,8 @@ type RootActs interface {
// run), carrying out the plan in the request (ADR 0510: the plan travels with the verb), and answers once it
// started.
Rollback(run string, plan []RollbackItem) error
// Mirrorlist writes the node's mirror list file whole with these servers (ADR 0585 §3).
Mirrorlist(servers []string) error
// Journal is the last n lines a transaction unit logged.
Journal(unit string, n int) ([]string, error)
// Maps is every process still mapping a replaced file, by its unit or container and its executable.
@@ -62,7 +64,10 @@ type Upgrades struct {
Root RootActs
Events EventSink
// The module's states.
Installed, Updates, Runs, Records, Rounds, Asks Bucket
Installed, Updates, Runs, Records, Rounds, Asks, Mirrors Bucket
// Rank ranks the mesh's mirrors with the module's reflector options (ADR 0585 §2); nil ranks none, and a round
// then opens with the list there is.
Rank func(conf []byte, round string) (MirrorList, error)
// AskRecord reads the operator channel's record of one of this module's asks (process.Conn.Ask on
// MessengerAsksBucket): how a pass is answered.
AskRecord func(id string) (*stdio.StateEntry, error)
+4 -4
View File
@@ -1,7 +1,7 @@
# The mesh's (module pacman, novox/hq to-be 42): how reflector refreshes the mirror list, weekly,
# through reflector.timer. Written whole at every push. Before the module, every machine's list was
# generated once and never again.
--save /etc/pacman.d/mirrorlist
# The mesh's (module pacman, novox/hq ADR 0585): the options reflector ranks the mesh's one mirror list with.
# The keeper of the node that opens a week's upgrade round runs reflector with them (and --info) and keeps only the
# mirrors as fresh as the newest; every node writes /etc/pacman.d/mirrorlist from that list. reflector's own timer
# stays disabled, and nothing here saves a list. Written whole at every push.
--protocol https
--country Belgium,Netherlands,Luxembourg,Germany,France
--latest 20
+6 -7
View File
@@ -53,7 +53,8 @@
"runs",
"records",
"rounds",
"asks"
"asks",
"mirrors"
],
"emits": [
"package.changed",
@@ -122,11 +123,8 @@
"id": "mirror-refresh",
"type": "service",
"unit": "reflector.timer",
"state": "running",
"boot": "enabled",
"health": {
"kind": "unit"
}
"state": "stopped",
"boot": "disabled"
},
{
"id": "cache-cleaning",
@@ -207,7 +205,8 @@
"runs",
"records",
"rounds",
"asks"
"asks",
"mirrors"
],
"emits": [
"package.changed",