Feature 57 (config and provisioning): modules declare config and provisions, their files rendered from templates; synced with main (issues of feature 57, 874)
This commit is contained in:
+58
-5
@@ -44,7 +44,7 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
|
||||
|
||||
| tool | | what |
|
||||
|---|---|---|
|
||||
| `gitea_commit` | a | one commit on a branch of its own, created from `base` (default the default branch) when absent, carrying either `files` (each `{path, content \| content_base64 \| delete: true}`) or `patch` (a unified diff, as `git diff` writes it); answers the commit and its parent |
|
||||
| `gitea_commit` | a | one commit on a branch of its own, created from `base` (default the default branch) when absent, carrying either `files` (each `{path, content \| content_base64 \| delete: true}`) `patch` (a unified diff, as `git diff` writes it), or `patch_file` with `patch_sha256` (a patch too large to type, see below); answers the commit and its parent |
|
||||
| `gitea_commits` | r | a ref's history, newest first, optionally only commits touching a path: sha, author, date, subject |
|
||||
| `gitea_compare` | r | what head holds that base does not: its commits and the files they touch |
|
||||
| `gitea_pull_reviews` | r | a pull request's reviews — state, the commit reviewed and whether it is the head now, stale, dismissed — with their line comments |
|
||||
@@ -52,13 +52,28 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
|
||||
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
|
||||
|
||||
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
|
||||
never the repository's default branch, never a branch a protection rule names. A rule covers a branch as the
|
||||
never the repository's default branch, never a branch a protection rule names. The tools' account may not read the
|
||||
rules themselves (the forge keeps them for a repository's administrator, novox/hq ADR 0365): for it an existing branch
|
||||
is judged by the protection the forge says it applies there, and a branch not made yet by the forge itself, which
|
||||
refuses the commit that would make a branch a rule guards. A rule covers a branch as the
|
||||
forge itself decides it (Gitea 1.27, novox/hq issue 539): a plain name covers that branch in any case; a glob
|
||||
(`*`, `**`, `?`, `[abc]`, `[!abc]`, `{a,b}`, `\` escaping) covers case-sensitively with `/` as separator; a glob
|
||||
the forge cannot compile covers its own text. `cmd/forge/rules.go` and `guard.ts` hold this one rule. The rules are read before anything is sent; rules that cannot be read refuse the
|
||||
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
|
||||
required checks green, through `gitea_merge_pull_request`.
|
||||
|
||||
**A patch too large to type** (novox/hq issue 852: a feature's diff is megabytes) goes as `patch_file`, with
|
||||
`patch_sha256`. The caller writes the patch into the module's spool directory `forge-patches`
|
||||
(`/var/lib/mesh/forge-patches`, mode 0733 and owned by the machine's account, which the tools run as: anyone writes,
|
||||
nobody lists; the bundle is given its path as `MESH_GITEA_PATCH_DIR`) and names it: `patch_file` is the file's NAME,
|
||||
32 to 64 characters of `0-9a-f` that the caller chooses at random, never a path. The tool then reads that one file,
|
||||
applies it exactly as it applies `patch` (same size limit, same answer) and removes it once it has found it, whether
|
||||
the commit was made or not. A file refused before that (a bad name, a missing hash or spool directory, a name with no
|
||||
file) stays in the spool. It refuses, sending nothing, a name of another shape, a file that is missing, empty, over
|
||||
4 MiB, with more than one hard link or not a regular file (a symlink is never followed), and a file whose SHA-256 is not `patch_sha256`: the directory has no sticky bit
|
||||
(the manifest allows none), so another writer could replace a file, and the hash is how the caller knows the tool read
|
||||
the bytes it wrote. It reads nothing outside the directory.
|
||||
|
||||
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
|
||||
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
|
||||
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. The blobs are read at
|
||||
@@ -66,7 +81,7 @@ the commit the branch (or, for a new branch, its base) was read at, never at its
|
||||
forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than the
|
||||
file. Neither endpoint takes a commit to build on, only a branch (Gitea 1.27), so the parent answered is the
|
||||
one the forge says it made the commit on; when the branch moved between the read and the commit, the answer
|
||||
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
||||
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the tools' account, so its sha is not the
|
||||
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
|
||||
branch not at the commit the forge named is a failure, never a success.
|
||||
|
||||
@@ -75,8 +90,10 @@ is read once and searched in memory; a binary file is passed over, a file over 4
|
||||
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found. A
|
||||
search that reaches the call's deadline stops there and answers what it found, with `stopped_at_deadline`.
|
||||
|
||||
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), and one
|
||||
tool call takes at most 50 seconds.
|
||||
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), as the
|
||||
tools' own account (novox/hq ADR 0365 §2, issue 469): the token the TypeScript bundle mints for `mesh-tools` and
|
||||
keeps (`MESH_GITEA_TOKEN_FILE`), sent only when its scopes line names that account, never the admin login. One tool
|
||||
call takes at most 50 seconds.
|
||||
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
|
||||
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
|
||||
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
|
||||
@@ -85,6 +102,42 @@ changed under the commit refused; a commit over its size limits refused; protect
|
||||
forge matches them; a code search stopped at its deadline; every malformed ask refused; and each read
|
||||
summarised.
|
||||
|
||||
## Each agent's read credential (novox/hq ADR 0365 §1, §3–§5, issue 737)
|
||||
|
||||
An agent clones and fetches the repositories it may read as a forge account of its own, `agent-<node>`. The
|
||||
module offers the provision `gitea-read`, whose credential its software issues (ADR 0305) one per holder (issue
|
||||
749): the Go bundle `cmd/agent-token` serves `gitea_agent_token`, which the vault alone asks (ADR 0349 clause 10;
|
||||
any other caller is refused and nothing is minted). For each holder the vault names that is the claude-code module
|
||||
on a machine, it
|
||||
|
||||
- makes the account `agent-<node>` with the administrator's login, when there is none: no administrator, a random
|
||||
password sent once and kept nowhere, and **in no team** — it reads public repositories until the operator adds it
|
||||
to a team that reads more (§3). An account found made an administrator, inactive or barred from logging in is
|
||||
made a plain account again;
|
||||
- deletes the account's token by the name `mesh-agent-read` (by its id, every one of them) and mints one whose one
|
||||
scope is `read:repository`: whatever team it is put in, the forge refuses it a push, a comment and any token or
|
||||
administration of its own (§5). Every ask mints anew; that is its rotation;
|
||||
- answers git's credential-store lines for it — the forge's public origin, read from `origins` (the route's name),
|
||||
and on the forge's own machine its address there too — sealed to the key the vault made for the call. A holder of
|
||||
another module is not answered and said under `skipped`.
|
||||
|
||||
The vault keeps each as `provision/<forge's node>/gitea-read/<node>/claude-code` and hands it to that holder alone;
|
||||
the claude-code module places it in the agent's home (ADR 0365 §4). The token is never logged, answered in the clear
|
||||
or put in an error. `go test ./cmd/agent-token` runs it against a fake forge.
|
||||
|
||||
## The agents' read teams (novox/hq ADR 0365 §3, issue 758)
|
||||
|
||||
An agent's account reads more than public repositories only through a team, per repository, and only to read. Which
|
||||
teams exist and who is in them is the operator's, and it is one trusted config value, `agent-read-teams`:
|
||||
`{"<team>": {"repos": ["<owner>/<repo>", ...], "members": ["<node>", ...]}}`, a member `<node>` being the account
|
||||
`agent-<node>`. The module renders it into `agent-teams.json` and runs `cmd/agent-teams` when it starts and
|
||||
whenever that file changes. For each team it makes `agent-read-<team>` in the organisation of the first repository's
|
||||
owner, with permission read and no other right, its repositories exactly the listed ones and its members exactly the
|
||||
listed accounts (an account that does not exist yet is skipped and said). A team named `agent-read-*` that has left
|
||||
the value is emptied, never deleted. A team without that prefix is never touched. There are no verbs: a change is a
|
||||
change of the value. It says each change in one line on stdout, and never a secret. `go test ./cmd/agent-teams`
|
||||
runs it against a fake forge.
|
||||
|
||||
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
||||
|
||||
Every publish adds a version to the registry, and until this nothing removed one. The Go bundle
|
||||
|
||||
+199
-14
@@ -8,6 +8,7 @@ import { ConfiguredToken, MintedToken, type TokenSource } from "./token.js";
|
||||
import { protectionBody, type BranchProtection, type ProtectionWanted } from "./protection.js";
|
||||
import { Refused, checkSentPath, pathSegment, pathSegments } from "./guard.js";
|
||||
import { isCommit } from "./checks.js";
|
||||
import { TOOLS_ACCOUNT } from "./tools-account.js";
|
||||
|
||||
/** A repository's path on the forge, its owner and name each one segment (guard.ts pathSegment): uniform for every
|
||||
* call, whether the names came from a caller or from the forge's own answer. */
|
||||
@@ -101,6 +102,94 @@ export interface OpenPull {
|
||||
/** The head's statuses by context, the newest of each, with its state (novox/hq issue 438): what tells a check
|
||||
* that started and never answered — pending — from one that answered. */
|
||||
checks: Record<string, HeadCheck>;
|
||||
/** The pull request's labels, by name. */
|
||||
labels?: string[];
|
||||
/** Its ready mark (novox/hq issue 722): the label `mesh/ready` and the newest comment "ready at <full head sha>", the
|
||||
* head a reviewer approved. Absent without the label; `head` empty when the label names no head in a comment. */
|
||||
ready?: ReadyMark;
|
||||
}
|
||||
|
||||
/** The label that marks a pull request ready to merge at the head a comment names (novox/hq issue 722). */
|
||||
export const READY_LABEL = "mesh/ready";
|
||||
|
||||
/** A comment that names the head a pull request is marked ready at: "ready at <full sha>", on a line of its own. */
|
||||
const READY_AT = /^\s*ready at ([0-9a-f]{40}|[0-9a-f]{64})\b/im;
|
||||
|
||||
/** A pull request's ready mark: the head it names, who said it and when, and the label's id, to take it off. */
|
||||
export interface ReadyMark {
|
||||
head: string;
|
||||
by?: string;
|
||||
at?: string;
|
||||
label_id: number;
|
||||
}
|
||||
|
||||
/** **The accounts whose "ready at" comment counts** (novox/hq issue 722): the trusted setting ready-authors, which
|
||||
* the mesh writes to the file MESH_GITEA_READY_AUTHORS_FILE names as a line `authors=<account> <account> …`. A file
|
||||
* that cannot be read leaves the mesh's own forge accounts alone, said; never an empty list read as everyone: the
|
||||
* account the tools act as (novox/hq ADR 0365) and the admin account, whose marks were made before it. */
|
||||
export function readyAuthors(file = process.env.MESH_GITEA_READY_AUTHORS_FILE, own = process.env.MESH_GITEA_ADMIN_USER,
|
||||
read: (f: string) => string = (f) => readFileSync(f, "utf8"),
|
||||
tools = process.env.MESH_GITEA_TOOLS_USER || TOOLS_ACCOUNT): string[] {
|
||||
const fallback = [...new Set([tools, own].filter((a): a is string => !!a))];
|
||||
if (!file) return fallback;
|
||||
try {
|
||||
const line = read(file).split("\n").find((l) => l.startsWith("authors="));
|
||||
const authors = (line ?? "").slice("authors=".length).split(/[\s,]+/).filter(Boolean);
|
||||
if (authors.length) return authors;
|
||||
console.error(`[gitea] ${file} names no account that may mark a pull request ready; only ${fallback.join(", ") || "none"} may`);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${file} cannot be read (${err instanceof Error ? err.message : err}); only ` +
|
||||
`${fallback.join(", ") || "none"} may mark a pull request ready`);
|
||||
}
|
||||
return fallback;
|
||||
}
|
||||
|
||||
/** One event of an issue's or pull request's timeline, as the forge keeps it, oldest first: a comment, a push to a
|
||||
* pull request's branch (`pull_push`), a label added (body "1") or taken off (body ""), and the rest. */
|
||||
export interface TimelineEvent {
|
||||
id: number;
|
||||
type: string;
|
||||
body: string;
|
||||
created_at?: string;
|
||||
user?: string;
|
||||
label?: { id: number; name: string };
|
||||
}
|
||||
|
||||
/** The comments that wrote "ready at" without counting, by id: each said once in the log (novox/hq issue 722). */
|
||||
const ignoredSaid = new Set<number>();
|
||||
|
||||
/** **The ready mark a pull request's timeline makes** (novox/hq issue 722), read in the forge's order — every event is
|
||||
* one row of one table, numbered as it happened, so the order is exact where times share a second. A "ready at"
|
||||
* comment counts only when its author is one of the accounts given, and only when it was written after the label
|
||||
* mesh/ready was last added and after the last push to the pull request's branch: a push or the label taken off
|
||||
* undoes every mark before it, so a mark older than the latest push never counts, even with the label still on. A
|
||||
* comment by any other account is ignored, and said once. */
|
||||
export function readyMarkOf(labelID: number, events: TimelineEvent[], authors: string[],
|
||||
say: (line: string) => void = (l) => console.error(l)): ReadyMark {
|
||||
const listed = new Set(authors.map((a) => a.toLowerCase()));
|
||||
let labelled = false;
|
||||
let mark: ReadyMark | undefined;
|
||||
for (const e of [...events].sort((x, y) => x.id - y.id)) {
|
||||
if (e.type === "pull_push") {
|
||||
mark = undefined;
|
||||
} else if (e.type === "label" && e.label?.name === READY_LABEL) {
|
||||
labelled = e.body === "1";
|
||||
mark = undefined;
|
||||
} else if (e.type === "comment") {
|
||||
const m = READY_AT.exec(e.body);
|
||||
if (!m) continue;
|
||||
if (!listed.has(String(e.user ?? "").toLowerCase())) {
|
||||
if (!ignoredSaid.has(e.id)) {
|
||||
ignoredSaid.add(e.id);
|
||||
say(`[gitea] a "ready at ${m[1].slice(0, 12)}" comment (${e.id}) by ${e.user ?? "an unknown account"} is ignored: ` +
|
||||
`only ${authors.join(", ") || "no account"} may mark a pull request ready (novox/hq issue 722)`);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (labelled) mark = { head: m[1].toLowerCase(), by: e.user, at: e.created_at, label_id: labelID };
|
||||
}
|
||||
}
|
||||
return mark ?? { head: "", label_id: labelID };
|
||||
}
|
||||
|
||||
/** One status on a head, the newest of its context: its state, what it said, and when it was set. */
|
||||
@@ -332,8 +421,9 @@ export class GiteaClient {
|
||||
|
||||
/** The open pull requests of every repository the mesh's account sees, updated since a moment when one is given,
|
||||
* page by page (novox/hq issue 347): each with its repository and its own update time. */
|
||||
async searchOpenPulls(since?: string): Promise<{ owner: string; name: string; number: number; updated_at?: string }[]> {
|
||||
const out: { owner: string; name: string; number: number; updated_at?: string }[] = [];
|
||||
async searchOpenPulls(since?: string): Promise<{ owner: string; name: string; number: number; updated_at?: string;
|
||||
labels?: { id: number; name: string }[] }[]> {
|
||||
const out: { owner: string; name: string; number: number; updated_at?: string; labels?: { id: number; name: string }[] }[] = [];
|
||||
for (let page = 1; page <= 20; page++) {
|
||||
const qs = new URLSearchParams({ type: "pulls", state: "open", limit: "50", page: String(page) });
|
||||
if (since) qs.set("since", since);
|
||||
@@ -343,7 +433,8 @@ export class GiteaClient {
|
||||
const [fo, fn] = String(r.full_name ?? "").split("/");
|
||||
const owner = String(r.owner ?? fo ?? "");
|
||||
const name = String(r.name ?? fn ?? "");
|
||||
if (owner && name && i?.number) out.push({ owner, name, number: Number(i.number), updated_at: i.updated_at });
|
||||
const labels = (Array.isArray(i?.labels) ? i.labels : []).map((l: any) => ({ id: Number(l?.id ?? 0), name: String(l?.name ?? "") }));
|
||||
if (owner && name && i?.number) out.push({ owner, name, number: Number(i.number), updated_at: i.updated_at, labels });
|
||||
}
|
||||
if (!found || found.length < 50) break;
|
||||
}
|
||||
@@ -354,19 +445,26 @@ export class GiteaClient {
|
||||
* requires and the contexts its head has a status of, in any state. What mesh-delivery reads to say a pull
|
||||
* request the mesh was never asked to check; asked of the forge, never of what the announcer remembers, so a
|
||||
* fault in the announcer cannot hide itself. Reads only. */
|
||||
async openPulls(): Promise<OpenPull[]> {
|
||||
async openPulls(readyAuthors: string[] = []): Promise<OpenPull[]> {
|
||||
const out: OpenPull[] = [];
|
||||
const rules = new Map<string, BranchProtection[] | undefined>();
|
||||
const rules = new Map<string, BranchProtection | null | undefined>();
|
||||
for (const found of await this.searchOpenPulls()) {
|
||||
const full = `${found.owner}/${found.name}`;
|
||||
const pull = await this.getPullRequest(found.owner, found.name, found.number);
|
||||
if (pull.state !== "open" || !pull.head_sha) continue;
|
||||
if (!rules.has(full)) rules.set(full, await this.branchProtections(found.owner, found.name).catch(() => undefined));
|
||||
const rule = rules.get(full)?.find((p) => (p.rule_name ?? p.branch_name) === (pull.base || "main"));
|
||||
// The base's protection as the forge applies it to that branch (appliedProtection): readable without administration.
|
||||
const base = `${full}:${pull.base || "main"}`;
|
||||
if (!rules.has(base)) rules.set(base, await this.appliedProtection(found.owner, found.name, pull.base || "main").catch(() => undefined));
|
||||
const rule = rules.get(base);
|
||||
const checks = await this.headChecks(found.owner, found.name, pull.head_sha);
|
||||
const labels = found.labels ?? [];
|
||||
// The ready mark (novox/hq issue 722): its timeline is read, whole, only for a pull request that carries the label.
|
||||
const label = labels.find((l) => l.name === READY_LABEL);
|
||||
const ready = label ? readyMarkOf(label.id, await this.timeline(found.owner, found.name, pull.number), readyAuthors)
|
||||
: undefined;
|
||||
out.push({ owner: found.owner, repo: found.name, number: pull.number, base: pull.base ?? "", head_sha: pull.head_sha,
|
||||
html_url: pull.html_url, required: rule?.enable_status_check ? (rule.status_check_contexts ?? []) : [],
|
||||
statuses: Object.keys(checks), checks });
|
||||
statuses: Object.keys(checks), checks, labels: labels.map((l) => l.name), ...(ready ? { ready } : {}) });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -580,16 +678,44 @@ export class GiteaClient {
|
||||
|
||||
// ---- Branch protection ----
|
||||
|
||||
/** A repository's branch protection rules. */
|
||||
/** A repository's branch protection rules. The forge answers them to a repository's administrator alone, which the
|
||||
* tools' account is not (novox/hq ADR 0365 §2): for it this is refused (403), and what a branch's protection asks
|
||||
* is read with appliedProtection. */
|
||||
async branchProtections(owner: string, repo: string): Promise<BranchProtection[]> {
|
||||
return (await this.request<BranchProtection[]>(`${repoPath(owner, repo)}/branch_protections`)) ?? [];
|
||||
}
|
||||
|
||||
/** The rule named for a branch, null when it has none. */
|
||||
/** The rule named for a branch, null when it has none — from the rules' list, so a repository administrator's. */
|
||||
async branchProtection(owner: string, repo: string, rule: string): Promise<BranchProtection | null> {
|
||||
return (await this.branchProtections(owner, repo)).find((p) => (p.rule_name ?? p.branch_name) === rule) ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The protection that covers an existing branch, as the forge applies it, or null when none does. Read from the
|
||||
* branch itself, which anyone who reads the code may read — the rules' own list is a repository administrator's
|
||||
* (novox/hq ADR 0365 §2) — and as the forge matches it, a glob rule included (its name is the rule's). The
|
||||
* branch's `user_can_push` stands for `enable_push`: whether this account may push there. A branch that does not
|
||||
* exist is null: nothing covers it yet, and the forge judges a push that makes it.
|
||||
*/
|
||||
async appliedProtection(owner: string, repo: string, branch: string): Promise<BranchProtection | null> {
|
||||
let b: any;
|
||||
try {
|
||||
b = await this.request<any>(`${repoPath(owner, repo)}/branches/${pathSegments("branch", branch)}`);
|
||||
} catch (err) {
|
||||
if (/: 404 /.test(String((err as Error)?.message))) return null;
|
||||
throw err;
|
||||
}
|
||||
if (!b?.protected) return null;
|
||||
return {
|
||||
rule_name: String(b.effective_branch_protection_name || branch),
|
||||
branch_name: branch,
|
||||
enable_push: Boolean(b.user_can_push),
|
||||
enable_status_check: Boolean(b.enable_status_check),
|
||||
status_check_contexts: Array.isArray(b.status_check_contexts) ? b.status_check_contexts.map(String) : [],
|
||||
required_approvals: Number(b.required_approvals ?? 0),
|
||||
};
|
||||
}
|
||||
|
||||
/** Make a branch require these commit statuses to merge (novox/hq ADR 0237): the rule named for the
|
||||
* branch is edited — its required statuses replaced by these, everything else it says kept unless
|
||||
* asked — or created, refusing direct pushes unless asked otherwise. Answers the rule as it is now. */
|
||||
@@ -639,6 +765,45 @@ export class GiteaClient {
|
||||
return this.requestText(`${repoPath(owner, repo)}/pulls/${index}.diff`);
|
||||
}
|
||||
|
||||
/** **Whether a pull request carries an approving review at a head** (novox/hq ADR 0382 §6): a review of the forge that
|
||||
* approves, is not dismissed and was made at that very head; or the mesh's ready mark naming that head (issue 722), which is
|
||||
* how an agent's approval is read where the forge refuses an agent's approve. Reads only. */
|
||||
async approvedAt(owner: string, repo: string, index: number, head: string, authors?: string[]): Promise<boolean> {
|
||||
indexOf(index);
|
||||
const want = head.toLowerCase();
|
||||
for (let page = 1; page <= 20; page++) {
|
||||
const raw = await this.request<any[]>(`${repoPath(owner, repo)}/pulls/${index}/reviews?page=${page}&limit=50`);
|
||||
for (const r of raw ?? []) {
|
||||
if (String(r?.state ?? "").toUpperCase() === "APPROVED" && !r?.dismissed && !r?.stale &&
|
||||
String(r?.commit_id ?? "").toLowerCase() === want) return true;
|
||||
}
|
||||
if (!raw || raw.length < 50) break;
|
||||
}
|
||||
const labels = await this.request<any[]>(`${repoPath(owner, repo)}/issues/${index}/labels`);
|
||||
const label = (labels ?? []).find((l) => l?.name === READY_LABEL);
|
||||
if (!label) return false;
|
||||
const ready = readyMarkOf(Number(label.id ?? 0), await this.timeline(owner, repo, index), authors ?? readyAuthors());
|
||||
return !!ready && ready.head.toLowerCase() === want;
|
||||
}
|
||||
|
||||
/** **Every event of an issue's or pull request's timeline**, oldest first, page by page (novox/hq issue 722): a long
|
||||
* thread hides nothing. Bounded at 100 pages of 50; a longer one is said, and its first 5000 events read. */
|
||||
async timeline(owner: string, repo: string, index: number): Promise<TimelineEvent[]> {
|
||||
indexOf(index);
|
||||
const out: TimelineEvent[] = [];
|
||||
for (let page = 1; page <= 100; page++) {
|
||||
const raw = await this.request<any[]>(`${repoPath(owner, repo)}/issues/${index}/timeline?page=${page}&limit=50`);
|
||||
for (const e of raw ?? []) {
|
||||
out.push({ id: Number(e?.id ?? 0), type: String(e?.type ?? ""), body: String(e?.body ?? ""),
|
||||
created_at: e?.created_at, user: e?.user?.login,
|
||||
...(e?.label ? { label: { id: Number(e.label.id ?? 0), name: String(e.label.name ?? "") } } : {}) });
|
||||
}
|
||||
if (!raw || raw.length < 50) return out;
|
||||
}
|
||||
console.error(`[gitea] ${owner}/${repo}#${index} has more than 5000 timeline events; only the first 5000 are read`);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Every comment on an issue or pull request, oldest first. */
|
||||
async listComments(owner: string, repo: string, index: number): Promise<GiteaComment[]> {
|
||||
indexOf(index);
|
||||
@@ -803,12 +968,26 @@ export class GiteaAdmin {
|
||||
return { status: res.status, body };
|
||||
}
|
||||
|
||||
/** One admin-API call with a JSON body, answering the status and body (the system webhook, novox/hq issue 729). */
|
||||
call(method: string, path: string, body?: unknown): Promise<AdminResponse> {
|
||||
return this.request(path, { method, ...(body === undefined ? {} : { body: JSON.stringify(body) }) });
|
||||
}
|
||||
|
||||
/** Fail with the forge's own message when a status the caller did not expect comes back. */
|
||||
private static fail(path: string, res: AdminResponse): never {
|
||||
const detail = typeof res.body === "string" ? res.body : JSON.stringify(res.body);
|
||||
throw new Error(`Gitea admin ${path}: ${res.status} ${detail}`);
|
||||
}
|
||||
|
||||
/** A repository's branch protection rules, read with the administrator account — the forge answers them to a
|
||||
* repository's administrator alone, which the tools' account is not (novox/hq ADR 0365 §2, ADR 0375). Only a read. */
|
||||
async branchProtections(owner: string, repo: string): Promise<BranchProtection[]> {
|
||||
const path = `${repoPath(owner, repo)}/branch_protections`;
|
||||
const res = await this.request(path);
|
||||
if (res.status !== 200) GiteaAdmin.fail(path, res);
|
||||
return Array.isArray(res.body) ? res.body : [];
|
||||
}
|
||||
|
||||
/** Ensure the npm-owner org exists. 201 created, 2xx/404-then-created, and 422/409 (a concurrent
|
||||
* create won the race) are all success. */
|
||||
async ensureOrg(name: string): Promise<void> {
|
||||
@@ -832,12 +1011,16 @@ export class GiteaAdmin {
|
||||
// unit the team does not have — so code read must be said here: without it gitea answers a
|
||||
// member's clone of a private repository with "not found", which is how the builder's first
|
||||
// credentialed clone failed against a team that named only packages.
|
||||
const units = {
|
||||
return this.ensureTeamWith(org, team, {
|
||||
permission: "read",
|
||||
units_map: { "repo.code": "read", "repo.packages": packageWrite ? "write" : "read" },
|
||||
includes_all_repositories: true,
|
||||
can_create_org_repo: false,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/** Ensure a team exists with exactly these units, and return its id (ensureTeam; the tools' team, tools-account.ts). */
|
||||
async ensureTeamWith(org: string, team: string, units: Record<string, unknown>): Promise<number> {
|
||||
const found = await this.findTeam(org, team);
|
||||
if (found !== null) {
|
||||
const patch = await this.request(`/teams/${found}`, {
|
||||
@@ -872,7 +1055,7 @@ export class GiteaAdmin {
|
||||
* The edit path is taken only when the user actually exists. A 422 from the create is also what
|
||||
* a plain validation failure returns, and reading it as "already there" made the follow-up edit
|
||||
* 404 — burying the create's own message, which is the one that says what is actually wrong. */
|
||||
async ensureUser(username: string, password: string, email: string): Promise<void> {
|
||||
async ensureUser(username: string, password: string, email: string, plain = false): Promise<void> {
|
||||
const res = await this.request("/admin/users", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ username, email, password, must_change_password: false }),
|
||||
@@ -886,7 +1069,9 @@ export class GiteaAdmin {
|
||||
// login_name is required by the admin edit endpoint; for a local user it is the username.
|
||||
// active and prohibit_login: a deactivated or login-prohibited user is refused like a wrong
|
||||
// password, so the provisioner's check reports it lost; applying again must undo both.
|
||||
body: JSON.stringify({ login_name: username, password, must_change_password: false, active: true, prohibit_login: false }),
|
||||
// plain: never a site administrator — one made so by hand is made a plain account again (novox/hq ADR 0365).
|
||||
body: JSON.stringify({ login_name: username, password, must_change_password: false, active: true, prohibit_login: false,
|
||||
...(plain ? { admin: false } : {}) }),
|
||||
});
|
||||
if (patch.status === 200) return;
|
||||
GiteaAdmin.fail(`/admin/users/${username}`, patch);
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
// agent-teams: the agents' read teams on the forge (novox/hq ADR 0365 clause 3, issue 758). A program the node's
|
||||
// runtime runs once when the module starts and again whenever the module's config value `agent-read-teams` changes:
|
||||
// it reads the file the mesh renders from that value, and makes the forge's teams named agent-read-* match it. Each
|
||||
// team reads only; its repositories and its members (agent accounts, agent-<node>) are exactly the value's. It
|
||||
// touches nothing outside the value: no team without the prefix, no repository or account of another kind.
|
||||
//
|
||||
// It says each change on stdout in one line, and never a secret.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
func main() {
|
||||
forge, missing := forgehttp.FromEnv()
|
||||
file := os.Getenv("MESH_AGENT_TEAMS_FILE")
|
||||
if file == "" {
|
||||
missing = append(missing, "MESH_AGENT_TEAMS_FILE")
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
fmt.Fprintf(os.Stderr, "agent-teams: %v not set by the manifest; nothing was changed\n", missing)
|
||||
os.Exit(1)
|
||||
}
|
||||
value, err := ReadValue(file)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "agent-teams:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
s := &Syncer{Forge: forge, Log: func(format string, a ...any) { fmt.Printf(format+"\n", a...) },
|
||||
WaitFor: 3 * time.Minute, Poll: 5 * time.Second}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
if err := s.Sync(ctx, value); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "agent-teams:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,339 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
const (
|
||||
// ConfigKey is the module's config value this program applies.
|
||||
ConfigKey = "agent-read-teams"
|
||||
// TeamPrefix names every team this program owns; a team without it is never touched (ADR 0365 §3).
|
||||
TeamPrefix = "agent-read-"
|
||||
// AccountPrefix names an agent's account for its node (cmd/agent-token AccountPrefix, ADR 0365 §1).
|
||||
AccountPrefix = "agent-"
|
||||
page = 50
|
||||
)
|
||||
|
||||
var (
|
||||
teamName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]{0,40}$`)
|
||||
nodeName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,31}$`)
|
||||
repoName = regexp.MustCompile(`^([A-Za-z0-9][A-Za-z0-9_.-]*)/([A-Za-z0-9_.-]+)$`)
|
||||
)
|
||||
|
||||
// readUnits are the units a read team has, each at read: the forge's rights of a team that reads and nothing more.
|
||||
var readUnits = map[string]string{
|
||||
"repo.code": "read", "repo.issues": "read", "repo.pulls": "read", "repo.releases": "read", "repo.wiki": "read",
|
||||
}
|
||||
|
||||
// Team is one read team of the value: its repositories (owner/repo, read only) and its members (nodes whose agent
|
||||
// account, agent-<node>, is in the team).
|
||||
type Team struct {
|
||||
Repos []string `json:"repos"`
|
||||
Members []string `json:"members"`
|
||||
}
|
||||
|
||||
// Value is the config value: each team by the name it is given without the prefix.
|
||||
type Value map[string]Team
|
||||
|
||||
// ReadValue reads the file the mesh renders from the config value. A file without the key is the empty value,
|
||||
// which empties every read team.
|
||||
func ReadValue(path string) (Value, error) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the teams file cannot be read (%s): %w", path, err)
|
||||
}
|
||||
var file map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &file); err != nil {
|
||||
return nil, fmt.Errorf("the teams file %s is not JSON: %w", path, err)
|
||||
}
|
||||
var v Value
|
||||
if r, ok := file[ConfigKey]; ok {
|
||||
dec := json.NewDecoder(strings.NewReader(string(r)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return nil, fmt.Errorf("%s in %s is not {team: {repos, members}}: %w", ConfigKey, path, err)
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// Syncer makes the forge's agent-read-* teams the value's.
|
||||
type Syncer struct {
|
||||
Forge forgehttp.Client
|
||||
Log func(format string, a ...any)
|
||||
WaitFor time.Duration // how long to wait for the forge to answer; none when zero
|
||||
Poll time.Duration
|
||||
}
|
||||
|
||||
type forgeTeam struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Permission string `json:"permission"`
|
||||
}
|
||||
|
||||
type forgeRepo struct {
|
||||
FullName string `json:"full_name"`
|
||||
}
|
||||
|
||||
type forgeMember struct {
|
||||
Login string `json:"login"`
|
||||
}
|
||||
|
||||
func (s *Syncer) say(format string, a ...any) {
|
||||
if s.Log != nil {
|
||||
s.Log(format, a...)
|
||||
}
|
||||
}
|
||||
|
||||
// Sync applies the value. A team that cannot be applied is said and the others still are; the error names them.
|
||||
func (s *Syncer) Sync(ctx context.Context, v Value) error {
|
||||
if err := s.waitForForge(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
orgs, err := s.orgs(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
wanted := map[string]bool{}
|
||||
var failed []string
|
||||
names := make([]string, 0, len(v))
|
||||
for n := range v {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, n := range names {
|
||||
wanted[TeamPrefix+n] = true
|
||||
if err := s.apply(ctx, n, v[n]); err != nil {
|
||||
s.say("not applied: team %s%s: %v", TeamPrefix, n, err)
|
||||
failed = append(failed, TeamPrefix+n)
|
||||
}
|
||||
}
|
||||
for _, org := range orgs {
|
||||
teams, err := s.teams(ctx, org)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, t := range teams {
|
||||
if !strings.HasPrefix(t.Name, TeamPrefix) || wanted[t.Name] {
|
||||
continue
|
||||
}
|
||||
if err := s.sync(ctx, org, t, nil, nil); err != nil {
|
||||
s.say("not applied: team %s in %s: %v", t.Name, org, err)
|
||||
failed = append(failed, t.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(failed) > 0 {
|
||||
return fmt.Errorf("not applied: %s", strings.Join(failed, ", "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Syncer) waitForForge(ctx context.Context) error {
|
||||
deadline := time.Now().Add(s.WaitFor)
|
||||
for {
|
||||
err := s.Forge.GetJSON(ctx, "/api/v1/version", nil)
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
var se *forgehttp.StatusError
|
||||
if errors.As(err, &se) || time.Now().After(deadline) || ctx.Err() != nil {
|
||||
return fmt.Errorf("the forge is not reachable: %w", err)
|
||||
}
|
||||
time.Sleep(s.Poll)
|
||||
}
|
||||
}
|
||||
|
||||
// apply makes one team of the value.
|
||||
func (s *Syncer) apply(ctx context.Context, name string, t Team) error {
|
||||
if !teamName.MatchString(name) {
|
||||
return fmt.Errorf("%q is not a team name", name)
|
||||
}
|
||||
if len(t.Repos) == 0 {
|
||||
return errors.New("it lists no repository, so the organisation to make it in is not known")
|
||||
}
|
||||
org := ""
|
||||
repos := map[string]bool{}
|
||||
for _, r := range t.Repos {
|
||||
m := repoName.FindStringSubmatch(r)
|
||||
if m == nil {
|
||||
return fmt.Errorf("%q is not owner/repo", r)
|
||||
}
|
||||
if org == "" {
|
||||
org = m[1]
|
||||
}
|
||||
if m[1] != org {
|
||||
return fmt.Errorf("%q is not in %s: a team's repositories are of one owner, the first's", r, org)
|
||||
}
|
||||
repos[r] = true
|
||||
}
|
||||
members := map[string]bool{}
|
||||
for _, n := range t.Members {
|
||||
if !nodeName.MatchString(n) {
|
||||
return fmt.Errorf("%q is not a node's name", n)
|
||||
}
|
||||
members[AccountPrefix+n] = true
|
||||
}
|
||||
full := TeamPrefix + name
|
||||
teams, err := s.teams(ctx, org)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var team *forgeTeam
|
||||
for i := range teams {
|
||||
if teams[i].Name == full {
|
||||
team = &teams[i]
|
||||
}
|
||||
}
|
||||
if team == nil {
|
||||
var made forgeTeam
|
||||
err := s.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/orgs/"+org+"/teams", map[string]any{
|
||||
"name": full, "description": "Read-only team of agents' forge accounts (ADR 0365 §3); set by the gitea config value " + ConfigKey,
|
||||
"permission": "read", "includes_all_repositories": false, "can_create_org_repo": false, "units_map": readUnits,
|
||||
}, &made)
|
||||
if err != nil {
|
||||
return fmt.Errorf("the team could not be made: %w", err)
|
||||
}
|
||||
s.say("made the team %s in %s, reading only", full, org)
|
||||
team = &made
|
||||
} else if team.Permission != "read" {
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPatch, fmt.Sprintf("/api/v1/teams/%d", team.ID), map[string]any{
|
||||
"name": full, "permission": "read", "includes_all_repositories": false, "can_create_org_repo": false, "units_map": readUnits,
|
||||
}, nil); err != nil {
|
||||
return fmt.Errorf("the team could not be made read-only: %w", err)
|
||||
}
|
||||
s.say("made the team %s in %s read-only (it was %s)", full, org, team.Permission)
|
||||
}
|
||||
return s.sync(ctx, org, *team, repos, members)
|
||||
}
|
||||
|
||||
// sync makes the team's repositories and members exactly the sets (nil is none).
|
||||
func (s *Syncer) sync(ctx context.Context, org string, t forgeTeam, repos, members map[string]bool) error {
|
||||
var errs []string
|
||||
var haveRepos []forgeRepo
|
||||
if err := s.list(ctx, fmt.Sprintf("/api/v1/teams/%d/repos", t.ID), &haveRepos); err != nil {
|
||||
return err
|
||||
}
|
||||
held := map[string]bool{}
|
||||
for _, r := range haveRepos {
|
||||
held[r.FullName] = true
|
||||
if !repos[r.FullName] {
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/teams/%d/repos/%s", t.ID, r.FullName), nil, nil); err != nil {
|
||||
errs = append(errs, err.Error())
|
||||
continue
|
||||
}
|
||||
s.say("took %s from the team %s", r.FullName, t.Name)
|
||||
}
|
||||
}
|
||||
for _, r := range sortedKeys(repos) {
|
||||
if held[r] {
|
||||
continue
|
||||
}
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPut, fmt.Sprintf("/api/v1/teams/%d/repos/%s", t.ID, r), nil, nil); err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s could not be given to the team: %v", r, err))
|
||||
continue
|
||||
}
|
||||
s.say("gave %s to the team %s, to read", r, t.Name)
|
||||
}
|
||||
var haveMembers []forgeMember
|
||||
if err := s.list(ctx, fmt.Sprintf("/api/v1/teams/%d/members", t.ID), &haveMembers); err != nil {
|
||||
return err
|
||||
}
|
||||
in := map[string]bool{}
|
||||
for _, m := range haveMembers {
|
||||
in[m.Login] = true
|
||||
if !members[m.Login] {
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/teams/%d/members/%s", t.ID, url.PathEscape(m.Login)), nil, nil); err != nil {
|
||||
errs = append(errs, err.Error())
|
||||
continue
|
||||
}
|
||||
s.say("took %s out of the team %s", m.Login, t.Name)
|
||||
}
|
||||
}
|
||||
for _, m := range sortedKeys(members) {
|
||||
if in[m] {
|
||||
continue
|
||||
}
|
||||
if err := s.Forge.GetJSON(ctx, "/api/v1/users/"+url.PathEscape(m), nil); err != nil {
|
||||
if errors.Is(err, forgehttp.ErrNotFound) {
|
||||
s.say("skipped %s for the team %s: the account does not exist yet", m, t.Name)
|
||||
continue
|
||||
}
|
||||
errs = append(errs, err.Error())
|
||||
continue
|
||||
}
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPut, fmt.Sprintf("/api/v1/teams/%d/members/%s", t.ID, url.PathEscape(m)), nil, nil); err != nil {
|
||||
errs = append(errs, fmt.Sprintf("%s could not be put in the team: %v", m, err))
|
||||
continue
|
||||
}
|
||||
s.say("put %s in the team %s", m, t.Name)
|
||||
}
|
||||
if len(errs) > 0 {
|
||||
return errors.New(strings.Join(errs, "; "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func sortedKeys(m map[string]bool) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// list reads every page of a list answer into into (a pointer to a slice).
|
||||
func (s *Syncer) list(ctx context.Context, path string, into any) error {
|
||||
var all []json.RawMessage
|
||||
for p := 1; ; p++ {
|
||||
var got []json.RawMessage
|
||||
if err := s.Forge.GetJSON(ctx, fmt.Sprintf("%s?limit=%d&page=%d", path, page, p), &got); err != nil {
|
||||
return err
|
||||
}
|
||||
all = append(all, got...)
|
||||
if len(got) < page {
|
||||
break
|
||||
}
|
||||
}
|
||||
raw, _ := json.Marshal(all)
|
||||
return json.Unmarshal(raw, into)
|
||||
}
|
||||
|
||||
func (s *Syncer) teams(ctx context.Context, org string) ([]forgeTeam, error) {
|
||||
var out []forgeTeam
|
||||
if err := s.list(ctx, "/api/v1/orgs/"+org+"/teams", &out); err != nil {
|
||||
if errors.Is(err, forgehttp.ErrNotFound) {
|
||||
return nil, fmt.Errorf("the organisation %s does not exist on the forge", org)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// orgs are the organisations on the forge, where a read team no longer in the value is found.
|
||||
func (s *Syncer) orgs(ctx context.Context) ([]string, error) {
|
||||
var out []struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if err := s.list(ctx, "/api/v1/admin/orgs", &out); err != nil {
|
||||
return nil, fmt.Errorf("the forge's organisations could not be read: %w", err)
|
||||
}
|
||||
names := make([]string, len(out))
|
||||
for i, o := range out {
|
||||
names[i] = o.Name
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
@@ -0,0 +1,333 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
type fTeam struct {
|
||||
ID int64
|
||||
Org string
|
||||
Name string
|
||||
Perm string
|
||||
Units map[string]string
|
||||
Repos map[string]bool
|
||||
Members map[string]bool
|
||||
}
|
||||
|
||||
type fakeForge struct {
|
||||
mu sync.Mutex
|
||||
orgs []string
|
||||
users map[string]bool
|
||||
teams []*fTeam
|
||||
next int64
|
||||
}
|
||||
|
||||
func newFake(orgs []string, users ...string) *fakeForge {
|
||||
f := &fakeForge{orgs: orgs, users: map[string]bool{}, next: 1}
|
||||
for _, u := range users {
|
||||
f.users[u] = true
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
func (f *fakeForge) add(org, name, perm string, repos, members []string) *fTeam {
|
||||
t := &fTeam{ID: f.next, Org: org, Name: name, Perm: perm, Repos: map[string]bool{}, Members: map[string]bool{}}
|
||||
f.next++
|
||||
for _, r := range repos {
|
||||
t.Repos[r] = true
|
||||
}
|
||||
for _, m := range members {
|
||||
t.Members[m] = true
|
||||
}
|
||||
f.teams = append(f.teams, t)
|
||||
return t
|
||||
}
|
||||
|
||||
func (f *fakeForge) byName(org, name string) *fTeam {
|
||||
for _, t := range f.teams {
|
||||
if t.Org == org && t.Name == name {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var (
|
||||
orgTeams = regexp.MustCompile(`^/api/v1/orgs/([^/]+)/teams$`)
|
||||
teamPath = regexp.MustCompile(`^/api/v1/teams/(\d+)$`)
|
||||
teamRepos = regexp.MustCompile(`^/api/v1/teams/(\d+)/repos(?:/(.+))?$`)
|
||||
teamMems = regexp.MustCompile(`^/api/v1/teams/(\d+)/members(?:/(.+))?$`)
|
||||
)
|
||||
|
||||
func (f *fakeForge) find(id string) *fTeam {
|
||||
n, _ := strconv.ParseInt(id, 10, 64)
|
||||
for _, t := range f.teams {
|
||||
if t.ID == n {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func keys(m map[string]bool) []string {
|
||||
out := []string{}
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func (f *fakeForge) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if u, p, ok := r.BasicAuth(); !ok || u != "mesh-admin" || p != "pw" {
|
||||
http.Error(w, "no", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
reply := func(status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
if v != nil {
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
}
|
||||
paged := r.URL.Query().Get("page") != "" && r.URL.Query().Get("page") != "1"
|
||||
var in map[string]any
|
||||
_ = json.NewDecoder(r.Body).Decode(&in)
|
||||
p := r.URL.Path
|
||||
switch {
|
||||
case p == "/api/v1/version":
|
||||
reply(200, map[string]string{"version": "1"})
|
||||
case p == "/api/v1/admin/orgs":
|
||||
out := []map[string]string{}
|
||||
for _, o := range f.orgs {
|
||||
out = append(out, map[string]string{"name": o})
|
||||
}
|
||||
if paged {
|
||||
out = nil
|
||||
}
|
||||
reply(200, out)
|
||||
case strings.HasPrefix(p, "/api/v1/users/"):
|
||||
if !f.users[strings.TrimPrefix(p, "/api/v1/users/")] {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
reply(200, map[string]string{})
|
||||
case orgTeams.MatchString(p):
|
||||
org := orgTeams.FindStringSubmatch(p)[1]
|
||||
known := false
|
||||
for _, o := range f.orgs {
|
||||
known = known || o == org
|
||||
}
|
||||
if !known {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
if r.Method == http.MethodPost {
|
||||
units, _ := in["units_map"].(map[string]any)
|
||||
t := f.add(org, in["name"].(string), in["permission"].(string), nil, nil)
|
||||
t.Units = map[string]string{}
|
||||
for k, v := range units {
|
||||
t.Units[k] = v.(string)
|
||||
}
|
||||
reply(201, map[string]any{"id": t.ID, "name": t.Name, "permission": t.Perm})
|
||||
return
|
||||
}
|
||||
out := []map[string]any{}
|
||||
for _, t := range f.teams {
|
||||
if t.Org == org && !paged {
|
||||
out = append(out, map[string]any{"id": t.ID, "name": t.Name, "permission": t.Perm})
|
||||
}
|
||||
}
|
||||
reply(200, out)
|
||||
case teamPath.MatchString(p):
|
||||
t := f.find(teamPath.FindStringSubmatch(p)[1])
|
||||
t.Perm = in["permission"].(string)
|
||||
reply(200, nil)
|
||||
case teamRepos.MatchString(p):
|
||||
m := teamRepos.FindStringSubmatch(p)
|
||||
t := f.find(m[1])
|
||||
switch {
|
||||
case m[2] == "" && r.Method == http.MethodGet:
|
||||
out := []map[string]string{}
|
||||
for _, k := range keys(t.Repos) {
|
||||
out = append(out, map[string]string{"full_name": k})
|
||||
}
|
||||
if paged {
|
||||
out = nil
|
||||
}
|
||||
reply(200, out)
|
||||
case r.Method == http.MethodPut:
|
||||
t.Repos[m[2]] = true
|
||||
reply(204, nil)
|
||||
case r.Method == http.MethodDelete:
|
||||
delete(t.Repos, m[2])
|
||||
reply(204, nil)
|
||||
}
|
||||
case teamMems.MatchString(p):
|
||||
m := teamMems.FindStringSubmatch(p)
|
||||
t := f.find(m[1])
|
||||
switch {
|
||||
case m[2] == "" && r.Method == http.MethodGet:
|
||||
out := []map[string]string{}
|
||||
for _, k := range keys(t.Members) {
|
||||
out = append(out, map[string]string{"login": k})
|
||||
}
|
||||
if paged {
|
||||
out = nil
|
||||
}
|
||||
reply(200, out)
|
||||
case r.Method == http.MethodPut:
|
||||
t.Members[m[2]] = true
|
||||
reply(204, nil)
|
||||
case r.Method == http.MethodDelete:
|
||||
delete(t.Members, m[2])
|
||||
reply(204, nil)
|
||||
}
|
||||
default:
|
||||
reply(404, nil)
|
||||
}
|
||||
}
|
||||
|
||||
func syncer(t *testing.T, f *fakeForge) (*Syncer, *[]string) {
|
||||
t.Helper()
|
||||
srv := httptest.NewServer(f)
|
||||
t.Cleanup(srv.Close)
|
||||
pw := filepath.Join(t.TempDir(), "pw")
|
||||
if err := os.WriteFile(pw, []byte("pw\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var said []string
|
||||
return &Syncer{Forge: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw},
|
||||
Log: func(format string, a ...any) { said = append(said, sprintf(format, a...)) }}, &said
|
||||
}
|
||||
|
||||
func sprintf(format string, a ...any) string { return fmt.Sprintf(format, a...) }
|
||||
|
||||
func equal(a, b []string) bool { return strings.Join(a, ",") == strings.Join(b, ",") }
|
||||
|
||||
// The first test of issue 758: a value makes a read-only team with exactly its repositories and members; a changed
|
||||
// value takes a repository and a member away; a team that is not named agent-read-* is never touched.
|
||||
func TestValueMakesReadTeamAndChangeRemoves(t *testing.T) {
|
||||
f := newFake([]string{"novox"}, "agent-g14", "agent-ace")
|
||||
other := f.add("novox", "Owners", "owner", []string{"novox/hq"}, []string{"agent-ace"})
|
||||
s, said := syncer(t, f)
|
||||
ctx := context.Background()
|
||||
|
||||
err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq", "novox/mesh-catalog"}, Members: []string{"g14"}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
team := f.byName("novox", "agent-read-all")
|
||||
if team == nil || team.Perm != "read" {
|
||||
t.Fatalf("team = %+v, want a read team", team)
|
||||
}
|
||||
for _, u := range team.Units {
|
||||
if u != "read" {
|
||||
t.Errorf("unit %q, want read only", u)
|
||||
}
|
||||
}
|
||||
if !equal(keys(team.Repos), []string{"novox/hq", "novox/mesh-catalog"}) || !equal(keys(team.Members), []string{"agent-g14"}) {
|
||||
t.Fatalf("repos %v members %v", keys(team.Repos), keys(team.Members))
|
||||
}
|
||||
|
||||
before := len(*said)
|
||||
if err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq", "novox/mesh-catalog"}, Members: []string{"g14"}}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(*said) != before {
|
||||
t.Errorf("a second run changed something: %v", (*said)[before:])
|
||||
}
|
||||
|
||||
if err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq"}, Members: nil}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !equal(keys(team.Repos), []string{"novox/hq"}) || len(team.Members) != 0 {
|
||||
t.Fatalf("after change: repos %v members %v", keys(team.Repos), keys(team.Members))
|
||||
}
|
||||
|
||||
// the team leaves the value: emptied, never deleted
|
||||
if err := s.Sync(ctx, Value{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.byName("novox", "agent-read-all") == nil || len(team.Repos) != 0 || len(team.Members) != 0 {
|
||||
t.Fatalf("removed team not emptied: %+v", team)
|
||||
}
|
||||
if other.Perm != "owner" || !equal(keys(other.Repos), []string{"novox/hq"}) || !equal(keys(other.Members), []string{"agent-ace"}) {
|
||||
t.Fatalf("a team without the prefix was touched: %+v", other)
|
||||
}
|
||||
for _, line := range *said {
|
||||
if strings.Contains(line, "pw") {
|
||||
t.Errorf("a line names the password: %q", line)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPermissionIsCorrectedAndMissingAccountSkipped(t *testing.T) {
|
||||
f := newFake([]string{"novox"}, "agent-g14")
|
||||
team := f.add("novox", "agent-read-all", "write", nil, nil)
|
||||
s, said := syncer(t, f)
|
||||
err := s.Sync(context.Background(), Value{"all": {Repos: []string{"novox/hq"}, Members: []string{"g14", "later"}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if team.Perm != "read" || !equal(keys(team.Members), []string{"agent-g14"}) {
|
||||
t.Fatalf("team = %+v", team)
|
||||
}
|
||||
if !strings.Contains(strings.Join(*said, "\n"), "agent-later") {
|
||||
t.Errorf("the missing account was not said: %v", *said)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRefusesWhatIsNotAReadTeam(t *testing.T) {
|
||||
f := newFake([]string{"novox", "other"})
|
||||
s, _ := syncer(t, f)
|
||||
for name, v := range map[string]Value{
|
||||
"no repo": {"a": {Members: []string{"g14"}}},
|
||||
"two owners": {"a": {Repos: []string{"novox/hq", "other/x"}}},
|
||||
"bad repo": {"a": {Repos: []string{"hq"}}},
|
||||
"bad member": {"a": {Repos: []string{"novox/hq"}, Members: []string{"../x"}}},
|
||||
"unknown org": {"a": {Repos: []string{"nowhere/hq"}}},
|
||||
"bad teamname": {"a b": {Repos: []string{"novox/hq"}}},
|
||||
} {
|
||||
if err := s.Sync(context.Background(), v); err == nil {
|
||||
t.Errorf("%s: no error", name)
|
||||
}
|
||||
}
|
||||
if len(f.teams) != 0 {
|
||||
t.Errorf("teams made: %+v", f.teams)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadValue(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
p := filepath.Join(dir, "t.json")
|
||||
_ = os.WriteFile(p, []byte(`{"agent-read-teams":{"a":{"repos":["o/r"],"members":["g14"]}}}`), 0o600)
|
||||
v, err := ReadValue(p)
|
||||
if err != nil || len(v["a"].Repos) != 1 || v["a"].Members[0] != "g14" {
|
||||
t.Fatalf("%v %v", v, err)
|
||||
}
|
||||
_ = os.WriteFile(p, []byte(`{}`), 0o600)
|
||||
if v, err := ReadValue(p); err != nil || len(v) != 0 {
|
||||
t.Fatalf("%v %v", v, err)
|
||||
}
|
||||
_ = os.WriteFile(p, []byte(`{"agent-read-teams":{"a":{"repos":["o/r"],"write":true}}}`), 0o600)
|
||||
if _, err := ReadValue(p); err == nil {
|
||||
t.Fatal("an unknown field was accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
// agent-token: the forge's read credential for each node's agent (novox/hq ADR 0365 clauses 1, 3, 4 and 5; issue
|
||||
// 737). A process the node's runtime launches and speaks MCP over stdio to, beside the module's other bundles.
|
||||
//
|
||||
// It serves one tool, gitea_agent_token, which the vault alone asks (ADR 0349 clause 10, issue 749): for each holder
|
||||
// of the provision `gitea-read` the vault names — the claude-code module on one machine — it makes that machine's
|
||||
// agent a forge account of its own, `agent-<node>`, no administrator and in no team, mints it a token whose one
|
||||
// scope is `read:repository`, and answers the git credential lines for it, each holder's sealed to the key the vault
|
||||
// made for the call. Asked again, it mints again and deletes the account's token from before: that is its rotation.
|
||||
//
|
||||
// stdout is the protocol; it says what it says on stderr, and never a token or the admin's password.
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
stdio "git.novox.be/novox/mesh-sdk/go"
|
||||
)
|
||||
|
||||
// ToolNames are the tools this bundle serves. The module's manifest lists no `tools` (see the npm-registry bundle
|
||||
// for why); the README names this one, and a test holds the three together.
|
||||
var ToolNames = []string{ToolName}
|
||||
|
||||
func main() {
|
||||
if err := stdio.Serve("", Tools(MinterFromEnv)); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// Tools are the bundle's tools over the minter the function gives, asked per call, so a manifest that does not set
|
||||
// the environment is a refusal naming what is missing, not a bundle that will not start.
|
||||
func Tools(minter func() (*Minter, error)) []stdio.Tool {
|
||||
return []stdio.Tool{{
|
||||
Name: ToolName,
|
||||
Description: "The forge's read credential for each node's agent (novox/hq ADR 0365), answered to the vault " +
|
||||
"alone: for each holder named, a forge account of the node's agent's own (agent-<node>), no administrator " +
|
||||
"and in no team, and a token of scope read:repository minted for it anew, its git credential lines " +
|
||||
"sealed to seal_to. Any other caller is refused and nothing is minted.",
|
||||
Input: map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"provision": map[string]any{"type": "string", "description": "the provision, " + Provision},
|
||||
"seal_to": map[string]any{"type": "string", "description": "the key made for this call, base64"},
|
||||
"holders": map[string]any{
|
||||
"type": "array",
|
||||
"description": "the provision's holders: each a consuming module on one machine",
|
||||
"items": map[string]any{"type": "object", "properties": map[string]any{
|
||||
"node": map[string]any{"type": "string"},
|
||||
"module": map[string]any{"type": "string"},
|
||||
}},
|
||||
},
|
||||
},
|
||||
"required": []string{"provision", "seal_to", "holders"},
|
||||
},
|
||||
Run: func(args map[string]any) (any, error) {
|
||||
caller := stdio.CallerOf(args)
|
||||
if err := fromTheVault(caller); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ask, err := readAsk(args)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m, err := minter()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return m.Answer(ask)
|
||||
},
|
||||
}}
|
||||
}
|
||||
@@ -0,0 +1,348 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/nacl/box"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
const (
|
||||
// ToolName is the issuing tool the vault asks (the offer's credential tool in module.json).
|
||||
ToolName = "gitea_agent_token"
|
||||
// Provision is what the tool issues: the forge's read credential, one per agent.
|
||||
Provision = "gitea-read"
|
||||
// Holder is the one consuming module answered: the agent's own module, which places the credential in the
|
||||
// agent's home (ADR 0365 §4, ADR 0266). A holder of another module is not answered and keeps nothing.
|
||||
Holder = "claude-code"
|
||||
// TokenName is the token's name in the agent account's list, found and deleted by it at the next mint.
|
||||
TokenName = "mesh-agent-read"
|
||||
// AccountPrefix names an agent's account for its node (ADR 0365 §1).
|
||||
AccountPrefix = "agent-"
|
||||
// vaultModule is the vault's module: its account is `<node>.mesh-vault` (ADR 0259 §8's runs-as).
|
||||
vaultModule = "mesh-vault"
|
||||
// deadline bounds one call, inside the vault's ask.
|
||||
deadline = 45 * time.Second
|
||||
)
|
||||
|
||||
// TokenScopes are the only scopes an agent's token carries (ADR 0365 §4, §5): it reads repositories and nothing
|
||||
// else, so the forge refuses it a push, a comment, a token of its own and any administration — whatever team the
|
||||
// account is in.
|
||||
var TokenScopes = []string{"read:repository"}
|
||||
|
||||
var nodeName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,31}$`)
|
||||
|
||||
// fromTheVault refuses every caller but the vault's own account (ADR 0349 clause 10): whoever else could call this
|
||||
// tool would hold every agent's credential.
|
||||
func fromTheVault(caller string) error {
|
||||
node, module, ok := strings.Cut(caller, ".")
|
||||
if ok && module == vaultModule && nodeName.MatchString(node) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%s answers the vault alone (novox/hq ADR 0349 clause 10, ADR 0365); %q is not the vault, and "+
|
||||
"nothing was minted", ToolName, caller)
|
||||
}
|
||||
|
||||
// HolderAsk is one holder the vault named.
|
||||
type HolderAsk struct {
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
}
|
||||
|
||||
// Ask is one call: the holders, and the key to seal each answer to.
|
||||
type Ask struct {
|
||||
SealTo [32]byte
|
||||
Holders []HolderAsk
|
||||
}
|
||||
|
||||
// readAsk reads the call's arguments, refusing anything that is not the vault's per-holder ask.
|
||||
func readAsk(args map[string]any) (Ask, error) {
|
||||
if p, _ := args["provision"].(string); p != Provision {
|
||||
return Ask{}, fmt.Errorf("%s issues %s alone, not %q; nothing was minted", ToolName, Provision, p)
|
||||
}
|
||||
key, _ := args["seal_to"].(string)
|
||||
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(key))
|
||||
if err != nil || len(raw) != 32 {
|
||||
return Ask{}, errors.New("seal_to is not the key of a call; nothing was minted")
|
||||
}
|
||||
var out Ask
|
||||
copy(out.SealTo[:], raw)
|
||||
list, ok := args["holders"].([]any)
|
||||
if !ok || len(list) == 0 {
|
||||
return Ask{}, fmt.Errorf("%s answers a value for each holder, and none was named; nothing was minted", ToolName)
|
||||
}
|
||||
seen := map[HolderAsk]bool{}
|
||||
for _, item := range list {
|
||||
m, _ := item.(map[string]any)
|
||||
node, _ := m["node"].(string)
|
||||
module, _ := m["module"].(string)
|
||||
h := HolderAsk{Node: strings.TrimSpace(node), Module: strings.TrimSpace(module)}
|
||||
if h.Node == "" || h.Module == "" || seen[h] {
|
||||
return Ask{}, errors.New("each holder is a node and a module, named once; nothing was minted")
|
||||
}
|
||||
seen[h] = true
|
||||
out.Holders = append(out.Holders, h)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Minter mints the agents' tokens with the administrator's login, which sets the forge up and nothing else (ADR
|
||||
// 0365 §2).
|
||||
type Minter struct {
|
||||
Forge forgehttp.Client
|
||||
// Node is the forge's own machine: a holder there also reaches the forge at Forge.URL.
|
||||
Node string
|
||||
// OriginsFile holds the forge's public origins, one per line (https://<its route's name>): the origins every
|
||||
// holder's credential is for.
|
||||
OriginsFile string
|
||||
// Password makes the password an agent's account is created with: random, sent once and kept nowhere.
|
||||
Password func() string
|
||||
Log func(format string, a ...any)
|
||||
}
|
||||
|
||||
// MinterFromEnv is the minter as the manifest hands it to this bundle. It builds the administrator's client itself:
|
||||
// the module's words reach all its bundles, and a word naming another bundle's token must not make this one act as
|
||||
// another account.
|
||||
func MinterFromEnv() (*Minter, error) {
|
||||
m := &Minter{
|
||||
Forge: forgehttp.Client{URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"), PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
|
||||
Timeout: 20 * time.Second},
|
||||
Node: os.Getenv("MESH_NODE"),
|
||||
OriginsFile: os.Getenv("MESH_GITEA_ORIGINS_FILE"),
|
||||
}
|
||||
var missing []string
|
||||
for _, kv := range [][2]string{{"MESH_GITEA_URL", m.Forge.URL}, {"MESH_GITEA_ADMIN_USER", m.Forge.User},
|
||||
{"MESH_GITEA_ADMIN_PASSWORD_FILE", m.Forge.PasswordFile}, {"MESH_GITEA_ORIGINS_FILE", m.OriginsFile}} {
|
||||
if kv[1] == "" {
|
||||
missing = append(missing, kv[0])
|
||||
}
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest; nothing was minted", strings.Join(missing, ", "))
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Sealed is one holder's credential as the vault takes it.
|
||||
type Sealed struct {
|
||||
Node string `json:"node"`
|
||||
Module string `json:"module"`
|
||||
Sealed string `json:"sealed"`
|
||||
}
|
||||
|
||||
// Answer is what the tool answers: a sealed credential for each holder answered, and why each other was not. Never
|
||||
// a token.
|
||||
type Answer struct {
|
||||
Holders []Sealed `json:"holders"`
|
||||
Skipped map[string]string `json:"skipped,omitempty"`
|
||||
}
|
||||
|
||||
// one call at a time: a mint deletes the account's token from before, and two at once would delete each other's.
|
||||
var minting sync.Mutex
|
||||
|
||||
// Answer mints each holder's token and seals its credential lines. A holder that is not an agent's module is not
|
||||
// answered; a failure for one holder fails the call, so the vault keeps nothing rather than a part.
|
||||
func (m *Minter) Answer(ask Ask) (Answer, error) {
|
||||
minting.Lock()
|
||||
defer minting.Unlock()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), deadline)
|
||||
defer cancel()
|
||||
origins, err := m.origins()
|
||||
if err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
out := Answer{Holders: []Sealed{}}
|
||||
skip := func(h HolderAsk, why string) {
|
||||
if out.Skipped == nil {
|
||||
out.Skipped = map[string]string{}
|
||||
}
|
||||
out.Skipped[h.Module+" on "+h.Node] = why
|
||||
}
|
||||
for _, h := range ask.Holders {
|
||||
if h.Module != Holder {
|
||||
skip(h, fmt.Sprintf("only %s places the forge's read credential, in its agent's home (ADR 0365 §4)", Holder))
|
||||
continue
|
||||
}
|
||||
if !nodeName.MatchString(h.Node) {
|
||||
skip(h, "not a machine's name an account can be named for")
|
||||
continue
|
||||
}
|
||||
reach := origins
|
||||
if h.Node == m.Node {
|
||||
reach = append(append([]string{}, origins...), m.Forge.URL)
|
||||
}
|
||||
if len(reach) == 0 {
|
||||
skip(h, "the forge names no origin it is reached at from that machine")
|
||||
continue
|
||||
}
|
||||
account := AccountPrefix + h.Node
|
||||
if err := m.ensureAccount(ctx, account); err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
token, err := m.mint(ctx, account)
|
||||
if err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
lines, err := credentialLines(account, token, reach)
|
||||
token = ""
|
||||
if err != nil {
|
||||
return Answer{}, err
|
||||
}
|
||||
sealed, err := box.SealAnonymous(nil, []byte(lines), &ask.SealTo, rand.Reader)
|
||||
lines = ""
|
||||
if err != nil {
|
||||
return Answer{}, fmt.Errorf("the credential for %s could not be sealed: %w", account, err)
|
||||
}
|
||||
out.Holders = append(out.Holders, Sealed{Node: h.Node, Module: h.Module, Sealed: base64.StdEncoding.EncodeToString(sealed)})
|
||||
m.say("minted a read token for %s (%s on %s); its token from before is deleted", account, h.Module, h.Node)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// origins are the forge's public origins, as its route names them.
|
||||
func (m *Minter) origins() ([]string, error) {
|
||||
raw, err := os.ReadFile(m.OriginsFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the forge's origins cannot be read (%s); nothing was minted: %w", m.OriginsFile, err)
|
||||
}
|
||||
var out []string
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
line = strings.TrimRight(strings.TrimSpace(line), "/")
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
u, err := url.Parse(line)
|
||||
if err != nil || (u.Scheme != "https" && u.Scheme != "http") || u.Host == "" || u.User != nil || u.Path != "" {
|
||||
return nil, fmt.Errorf("%s names %q, which is not an origin (scheme://host); nothing was minted", m.OriginsFile, line)
|
||||
}
|
||||
out = append(out, line)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// credentialLines are the lines of git's credential store for one account at each origin: what the agent's module
|
||||
// places, and git reads by protocol and host.
|
||||
func credentialLines(account, token string, origins []string) (string, error) {
|
||||
seen := map[string]bool{}
|
||||
var lines []string
|
||||
for _, o := range origins {
|
||||
u, err := url.Parse(strings.TrimRight(o, "/"))
|
||||
if err != nil || u.Host == "" {
|
||||
return "", fmt.Errorf("%q is not an origin", o)
|
||||
}
|
||||
u.Path, u.RawQuery, u.Fragment = "", "", ""
|
||||
u.User = url.UserPassword(account, token)
|
||||
if line := u.String(); !seen[line] {
|
||||
seen[line] = true
|
||||
lines = append(lines, line)
|
||||
}
|
||||
}
|
||||
sort.Strings(lines)
|
||||
return strings.Join(lines, "\n") + "\n", nil
|
||||
}
|
||||
|
||||
// gitea's account as the admin reads it.
|
||||
type forgeUser struct {
|
||||
IsAdmin bool `json:"is_admin"`
|
||||
Active bool `json:"active"`
|
||||
ProhibitLogin bool `json:"prohibit_login"`
|
||||
Restricted bool `json:"restricted"`
|
||||
}
|
||||
|
||||
// ensureAccount makes the agent's account, plain, when there is none, and puts back what was changed by hand: never
|
||||
// an administrator, active, able to use its token. It adds it to no team: what it may read beyond public
|
||||
// repositories is the operator's to grant (ADR 0365 §3).
|
||||
func (m *Minter) ensureAccount(ctx context.Context, account string) error {
|
||||
var u forgeUser
|
||||
err := m.Forge.GetJSON(ctx, "/api/v1/users/"+account, &u)
|
||||
switch {
|
||||
case errors.Is(err, forgehttp.ErrNotFound):
|
||||
password := m.password()
|
||||
err := m.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/admin/users", map[string]any{
|
||||
"username": account, "email": account + "@noreply.localhost", "password": password,
|
||||
"must_change_password": false, "visibility": "private",
|
||||
}, nil)
|
||||
password = ""
|
||||
if err != nil {
|
||||
return fmt.Errorf("the account %s could not be made: %w", account, err)
|
||||
}
|
||||
m.say("made the forge account %s for that machine's agent, in no team", account)
|
||||
return nil
|
||||
case err != nil:
|
||||
return fmt.Errorf("the account %s could not be read: %w", account, err)
|
||||
}
|
||||
if u.IsAdmin || !u.Active || u.ProhibitLogin {
|
||||
if err := m.Forge.SendJSON(ctx, http.MethodPatch, "/api/v1/admin/users/"+account, map[string]any{
|
||||
"login_name": account, "source_id": 0, "admin": false, "active": true, "prohibit_login": false,
|
||||
}, nil); err != nil {
|
||||
return fmt.Errorf("the account %s could not be made a plain account again: %w", account, err)
|
||||
}
|
||||
m.say("the account %s was changed by hand (administrator %t, active %t, login prohibited %t); made a plain account again",
|
||||
account, u.IsAdmin, u.Active, u.ProhibitLogin)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type forgeToken struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
SHA1 string `json:"sha1"`
|
||||
}
|
||||
|
||||
// mint deletes every token of the account by this module's name, by its id (a name held twice is cleared, not
|
||||
// refused), and mints one with the read scope alone.
|
||||
func (m *Minter) mint(ctx context.Context, account string) (string, error) {
|
||||
var held []forgeToken
|
||||
if err := m.Forge.GetJSON(ctx, "/api/v1/users/"+account+"/tokens?limit=50", &held); err != nil {
|
||||
return "", fmt.Errorf("the tokens of %s could not be read: %w", account, err)
|
||||
}
|
||||
for _, t := range held {
|
||||
if t.Name != TokenName {
|
||||
continue
|
||||
}
|
||||
if err := m.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/users/%s/tokens/%d", account, t.ID), nil, nil); err != nil &&
|
||||
!errors.Is(err, forgehttp.ErrNotFound) {
|
||||
return "", fmt.Errorf("the token of %s from before could not be deleted: %w", account, err)
|
||||
}
|
||||
}
|
||||
var made forgeToken
|
||||
if err := m.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/users/"+account+"/tokens",
|
||||
map[string]any{"name": TokenName, "scopes": TokenScopes}, &made); err != nil {
|
||||
return "", fmt.Errorf("no token could be minted for %s: %w", account, err)
|
||||
}
|
||||
if strings.TrimSpace(made.SHA1) == "" {
|
||||
return "", fmt.Errorf("the forge answered no token for %s", account)
|
||||
}
|
||||
return strings.TrimSpace(made.SHA1), nil
|
||||
}
|
||||
|
||||
func (m *Minter) password() string {
|
||||
if m.Password != nil {
|
||||
return m.Password()
|
||||
}
|
||||
b := make([]byte, 24)
|
||||
_, _ = rand.Read(b)
|
||||
return base64.RawURLEncoding.EncodeToString(b)
|
||||
}
|
||||
|
||||
func (m *Minter) say(format string, a ...any) {
|
||||
if m.Log != nil {
|
||||
m.Log(format, a...)
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, format+"\n", a...)
|
||||
}
|
||||
@@ -0,0 +1,376 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/crypto/nacl/box"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
const adminPassword = "admin-password-never-said"
|
||||
|
||||
// fakeForge is gitea's API as far as this bundle uses it, as the administrator's login reaches it.
|
||||
type fakeForge struct {
|
||||
mu sync.Mutex
|
||||
users map[string]*forgeUser
|
||||
tokens map[string][]forgeToken
|
||||
next int64
|
||||
calls []string
|
||||
made map[string]map[string]any
|
||||
minted map[string][]string // account -> scopes of its last mint
|
||||
failMint bool
|
||||
}
|
||||
|
||||
func newFakeForge() *fakeForge {
|
||||
return &fakeForge{users: map[string]*forgeUser{}, tokens: map[string][]forgeToken{}, made: map[string]map[string]any{},
|
||||
minted: map[string][]string{}, next: 1}
|
||||
}
|
||||
|
||||
var tokenPath = regexp.MustCompile(`^/api/v1/users/([^/]+)/tokens(?:/(\d+))?$`)
|
||||
|
||||
func (f *fakeForge) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
user, pass, ok := r.BasicAuth()
|
||||
if !ok || user != "mesh-admin" || pass != adminPassword {
|
||||
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
f.calls = append(f.calls, r.Method+" "+r.URL.Path)
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
var in map[string]any
|
||||
_ = json.Unmarshal(body, &in)
|
||||
reply := func(status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
if v != nil {
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, "/api/v1/users/") && strings.Count(r.URL.Path, "/") == 4:
|
||||
u, ok := f.users[strings.TrimPrefix(r.URL.Path, "/api/v1/users/")]
|
||||
if !ok {
|
||||
reply(404, map[string]any{"message": "user does not exist"})
|
||||
return
|
||||
}
|
||||
reply(200, map[string]any{"is_admin": u.IsAdmin, "active": u.Active, "prohibit_login": u.ProhibitLogin})
|
||||
case r.Method == http.MethodPost && r.URL.Path == "/api/v1/admin/users":
|
||||
name, _ := in["username"].(string)
|
||||
f.users[name] = &forgeUser{Active: true}
|
||||
f.made[name] = in
|
||||
reply(201, map[string]any{"login": name})
|
||||
case r.Method == http.MethodPatch && strings.HasPrefix(r.URL.Path, "/api/v1/admin/users/"):
|
||||
u := f.users[strings.TrimPrefix(r.URL.Path, "/api/v1/admin/users/")]
|
||||
u.IsAdmin, _ = in["admin"].(bool)
|
||||
u.Active, _ = in["active"].(bool)
|
||||
u.ProhibitLogin, _ = in["prohibit_login"].(bool)
|
||||
reply(200, map[string]any{})
|
||||
case tokenPath.MatchString(r.URL.Path):
|
||||
m := tokenPath.FindStringSubmatch(r.URL.Path)
|
||||
account := m[1]
|
||||
switch r.Method {
|
||||
case http.MethodGet:
|
||||
reply(200, f.tokens[account])
|
||||
case http.MethodDelete:
|
||||
kept := f.tokens[account][:0]
|
||||
for _, t := range f.tokens[account] {
|
||||
if fmt.Sprint(t.ID) != m[2] {
|
||||
kept = append(kept, t)
|
||||
}
|
||||
}
|
||||
f.tokens[account] = kept
|
||||
reply(204, nil)
|
||||
case http.MethodPost:
|
||||
if f.failMint {
|
||||
reply(500, map[string]any{"message": "the forge is unwell"})
|
||||
return
|
||||
}
|
||||
name, _ := in["name"].(string)
|
||||
for _, t := range f.tokens[account] {
|
||||
if t.Name == name {
|
||||
reply(400, map[string]any{"message": "access token name has been used already"})
|
||||
return
|
||||
}
|
||||
}
|
||||
var scopes []string
|
||||
for _, s := range in["scopes"].([]any) {
|
||||
scopes = append(scopes, s.(string))
|
||||
}
|
||||
f.minted[account] = scopes
|
||||
b := make([]byte, 20)
|
||||
_, _ = rand.Read(b)
|
||||
t := forgeToken{ID: f.next, Name: name, SHA1: fmt.Sprintf("%x", b)}
|
||||
f.next++
|
||||
f.tokens[account] = append(f.tokens[account], forgeToken{ID: t.ID, Name: t.Name})
|
||||
reply(201, t)
|
||||
}
|
||||
default:
|
||||
reply(404, map[string]any{"message": "not here: " + r.URL.Path})
|
||||
}
|
||||
}
|
||||
|
||||
type harness struct {
|
||||
forge *fakeForge
|
||||
server *httptest.Server
|
||||
minter *Minter
|
||||
public *[32]byte
|
||||
private *[32]byte
|
||||
logged *[]string
|
||||
}
|
||||
|
||||
func newHarness(t *testing.T) harness {
|
||||
t.Helper()
|
||||
f := newFakeForge()
|
||||
srv := httptest.NewServer(f)
|
||||
t.Cleanup(srv.Close)
|
||||
dir := t.TempDir()
|
||||
pw := filepath.Join(dir, "admin.secret")
|
||||
origins := filepath.Join(dir, "origins")
|
||||
if err := os.WriteFile(pw, []byte(adminPassword+"\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(origins, []byte("https://git.example.test\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pub, priv, err := box.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var logged []string
|
||||
m := &Minter{Forge: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}, Node: "anchor",
|
||||
OriginsFile: origins, Log: func(format string, a ...any) { logged = append(logged, fmt.Sprintf(format, a...)) }}
|
||||
return harness{forge: f, server: srv, minter: m, public: pub, private: priv, logged: &logged}
|
||||
}
|
||||
|
||||
func (h harness) open(t *testing.T, sealed string) string {
|
||||
t.Helper()
|
||||
raw, err := base64.StdEncoding.DecodeString(sealed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, ok := box.OpenAnonymous(nil, raw, h.public, h.private)
|
||||
if !ok {
|
||||
t.Fatal("a holder's value is not sealed to the call's key")
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
|
||||
func (h harness) args(holders ...HolderAsk) map[string]any {
|
||||
list := []any{}
|
||||
for _, x := range holders {
|
||||
list = append(list, map[string]any{"node": x.Node, "module": x.Module})
|
||||
}
|
||||
return map[string]any{"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:]), "holders": list}
|
||||
}
|
||||
|
||||
func TestOnlyTheVaultIsAnsweredAndNothingIsMintedForAnyoneElse(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
for _, caller := range []string{"controller", "novox.node-tools", "provisioner.anchor", "anchor.claude-code",
|
||||
"mesh-vault", "anchor.mesh-vault.x", "", "unproven"} {
|
||||
if err := fromTheVault(caller); err == nil {
|
||||
t.Errorf("%q was taken for the vault", caller)
|
||||
}
|
||||
}
|
||||
if err := fromTheVault("novox.mesh-vault"); err != nil {
|
||||
t.Errorf("the vault was refused: %v", err)
|
||||
}
|
||||
// Through the tool as the runtime serves it: a call no runtime named a caller for is the vault's to nobody.
|
||||
tools := Tools(func() (*Minter, error) { return h.minter, nil })
|
||||
if _, err := tools[0].Run(h.args(HolderAsk{"anchor", "claude-code"})); err == nil || !strings.Contains(err.Error(), "vault alone") {
|
||||
t.Fatalf("an unproven caller was answered: %v", err)
|
||||
}
|
||||
if len(h.forge.calls) != 0 {
|
||||
t.Fatalf("the forge was reached for a caller that is not the vault: %v", h.forge.calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEachAgentGetsAnAccountOfItsOwnAndAReadTokenSealedForItAlone(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
ask, err := readAsk(h.args(HolderAsk{"anchor", "claude-code"}, HolderAsk{"laptop", "claude-code"}, HolderAsk{"laptop", "lab"}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := h.minter.Answer(ask)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got.Holders) != 2 {
|
||||
t.Fatalf("answered %d holders, want the two agents: %+v", len(got.Holders), got)
|
||||
}
|
||||
if !strings.Contains(got.Skipped["lab on laptop"], Holder) {
|
||||
t.Errorf("a holder that is not an agent's module was not said: %v", got.Skipped)
|
||||
}
|
||||
lines := map[string]string{}
|
||||
for _, s := range got.Holders {
|
||||
lines[s.Node] = h.open(t, s.Sealed)
|
||||
}
|
||||
anchorTok := tokenIn(t, lines["anchor"], "agent-anchor")
|
||||
laptopTok := tokenIn(t, lines["laptop"], "agent-laptop")
|
||||
if anchorTok == laptopTok {
|
||||
t.Fatal("two agents were handed one token")
|
||||
}
|
||||
// The forge's own machine reaches it at its own address too; another machine only at its public origin.
|
||||
if !strings.Contains(lines["anchor"], "https://agent-anchor:"+anchorTok+"@git.example.test\n") ||
|
||||
!strings.Contains(lines["anchor"], strings.Replace(h.server.URL, "http://", "http://agent-anchor:"+anchorTok+"@", 1)) {
|
||||
t.Errorf("anchor's credential lines: %q", strings.ReplaceAll(lines["anchor"], anchorTok, "<token>"))
|
||||
}
|
||||
if lines["laptop"] != "https://agent-laptop:"+laptopTok+"@git.example.test\n" {
|
||||
t.Errorf("laptop's credential lines: %q", strings.ReplaceAll(lines["laptop"], laptopTok, "<token>"))
|
||||
}
|
||||
for _, account := range []string{"agent-anchor", "agent-laptop"} {
|
||||
u := h.forge.users[account]
|
||||
if u == nil || u.IsAdmin {
|
||||
t.Errorf("%s: %+v, want a plain account", account, u)
|
||||
}
|
||||
if made := h.forge.made[account]; made["must_change_password"] != false || made["password"] == "" {
|
||||
t.Errorf("%s was made with %v", account, made)
|
||||
}
|
||||
if !slices.Equal(h.forge.minted[account], []string{"read:repository"}) {
|
||||
t.Errorf("%s's token carries %v, want read:repository alone", account, h.forge.minted[account])
|
||||
}
|
||||
}
|
||||
for _, c := range h.forge.calls {
|
||||
if strings.Contains(c, "/teams") || strings.Contains(c, "/orgs/") || strings.Contains(c, "collaborators") {
|
||||
t.Errorf("an agent's account was given access (%s): it starts in no team, the operator grants (ADR 0365 §3)", c)
|
||||
}
|
||||
}
|
||||
if _, made := h.forge.users["agent-laptop-lab"]; made || len(h.forge.users) != 2 {
|
||||
t.Errorf("accounts made: %v", h.forge.users)
|
||||
}
|
||||
said := strings.Join(*h.logged, "\n")
|
||||
if strings.Contains(said, anchorTok) || strings.Contains(said, laptopTok) {
|
||||
t.Fatal("a token was said in the log")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAskedAgainTheTokenIsMintedAnewAndTheOneFromBeforeDeleted(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
ask, _ := readAsk(h.args(HolderAsk{"laptop", "claude-code"}))
|
||||
first, err := h.minter.Answer(ask)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.forge.tokens["agent-laptop"] = append(h.forge.tokens["agent-laptop"], forgeToken{ID: 99, Name: "a-persons-own"})
|
||||
again, err := h.minter.Answer(ask)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if h.open(t, first.Holders[0].Sealed) == h.open(t, again.Holders[0].Sealed) {
|
||||
t.Fatal("asked again, the same token was answered: it was not rotated")
|
||||
}
|
||||
var names []string
|
||||
for _, tok := range h.forge.tokens["agent-laptop"] {
|
||||
names = append(names, tok.Name)
|
||||
}
|
||||
slices.Sort(names)
|
||||
if !slices.Equal(names, []string{"a-persons-own", TokenName}) {
|
||||
t.Fatalf("the account holds %v: the token from before is not deleted, or another was", names)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAccountMadeAnAdministratorByHandIsMadePlainAgain(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
h.forge.users["agent-laptop"] = &forgeUser{IsAdmin: true, Active: true}
|
||||
ask, _ := readAsk(h.args(HolderAsk{"laptop", "claude-code"}))
|
||||
if _, err := h.minter.Answer(ask); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if h.forge.users["agent-laptop"].IsAdmin {
|
||||
t.Fatal("the agent's account is still an administrator")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAFailureSaysNeitherTheAdminsPasswordNorATokenAndAnswersNothing(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
h.forge.failMint = true
|
||||
ask, _ := readAsk(h.args(HolderAsk{"anchor", "claude-code"}, HolderAsk{"laptop", "claude-code"}))
|
||||
got, err := h.minter.Answer(ask)
|
||||
if err == nil || len(got.Holders) != 0 {
|
||||
t.Fatalf("a failed mint answered %+v, %v", got, err)
|
||||
}
|
||||
if strings.Contains(err.Error(), adminPassword) {
|
||||
t.Fatal("the admin's password was said in an error")
|
||||
}
|
||||
// A wrong password is a refusal, said without it.
|
||||
if err := os.WriteFile(h.minter.Forge.PasswordFile, []byte("wrong"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h.forge.failMint = false
|
||||
if _, err := h.minter.Answer(ask); err == nil || strings.Contains(err.Error(), "wrong") {
|
||||
t.Fatalf("a refused login: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheAskIsReadStrictly(t *testing.T) {
|
||||
h := newHarness(t)
|
||||
for name, args := range map[string]map[string]any{
|
||||
"another provision": {"provision": "git", "seal_to": base64.StdEncoding.EncodeToString(h.public[:]),
|
||||
"holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
|
||||
"no key": {"provision": Provision, "holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
|
||||
"short key": {"provision": Provision, "seal_to": "AAAA", "holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
|
||||
"no holders": {"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:])},
|
||||
"a holder twice": {"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:]),
|
||||
"holders": []any{map[string]any{"node": "a", "module": "claude-code"}, map[string]any{"node": "a", "module": "claude-code"}}},
|
||||
} {
|
||||
if _, err := readAsk(args); err == nil {
|
||||
t.Errorf("%s was taken", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheReadmeNamesWhatTheBundleServes(t *testing.T) {
|
||||
var served []string
|
||||
for _, tool := range Tools(MinterFromEnv) {
|
||||
served = append(served, tool.Name)
|
||||
}
|
||||
if !slices.Equal(served, ToolNames) {
|
||||
t.Fatalf("served %v, named %v", served, ToolNames)
|
||||
}
|
||||
readme, err := os.ReadFile("../../README.md")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, n := range ToolNames {
|
||||
if !strings.Contains(string(readme), "`"+n+"`") {
|
||||
t.Errorf("the README does not name %s", n)
|
||||
}
|
||||
}
|
||||
manifest, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(manifest), `"tool": "`+ToolName+`"`) || !strings.Contains(string(manifest), `"name": "`+Provision+`"`) {
|
||||
t.Error("the manifest does not offer the provision with this tool as its issuer")
|
||||
}
|
||||
}
|
||||
|
||||
// tokenIn is the token of account's credential lines, all one.
|
||||
func tokenIn(t *testing.T, lines, account string) string {
|
||||
t.Helper()
|
||||
re := regexp.MustCompile(regexp.QuoteMeta(account) + `:([0-9a-f]{40})@`)
|
||||
all := re.FindAllStringSubmatch(lines, -1)
|
||||
if len(all) == 0 {
|
||||
t.Fatalf("no credential for %s in the lines", account)
|
||||
}
|
||||
for _, m := range all {
|
||||
if m[1] != all[0][1] {
|
||||
t.Fatalf("%s's lines carry two tokens", account)
|
||||
}
|
||||
}
|
||||
return all[0][1]
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
package main
|
||||
|
||||
// The commit and the reads act as the tools' own account (novox/hq ADR 0365 §2, issue 469): the token the module
|
||||
// keeps for `mesh-tools`, never the admin login. That account is no repository's administrator, so the forge
|
||||
// refuses it the protection rules (403, as gitea's reqAdmin does); the commit then judges an existing branch by the
|
||||
// protection the forge says it applies, and leaves a branch not made yet to the forge's own judgement.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
const toolsToken = "tools-token-never-shown"
|
||||
|
||||
// asTools serves the fake forge to a bundle holding the tools' token: only that token is taken, the rules are
|
||||
// refused as to a non-administrator, and the branches named protected say so as the forge does.
|
||||
func asTools(t *testing.T, f *fakeForge, protected map[string]string, scopes string) (*Forge, *[]string) {
|
||||
dir := t.TempDir()
|
||||
file := filepath.Join(dir, "token")
|
||||
if err := os.WriteFile(file, []byte(toolsToken+"\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(file+".scopes", []byte(scopes+"\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var seen []string
|
||||
const repo = "/api/v1/repos/acme/app"
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
seen = append(seen, r.Method+" "+r.URL.Path)
|
||||
if _, _, basic := r.BasicAuth(); basic || r.Header.Get("Authorization") != "token "+toolsToken {
|
||||
http.Error(w, "unauthorized", 401)
|
||||
return
|
||||
}
|
||||
if r.URL.Path == repo+"/branch_protections" {
|
||||
http.Error(w, `{"message":"token does not have the administration of this repository"}`, 403)
|
||||
return
|
||||
}
|
||||
if name := strings.TrimPrefix(r.URL.Path, repo+"/branches/"); r.Method == "GET" && name != r.URL.Path {
|
||||
if rule, ok := protected[name]; ok {
|
||||
_, _ = w.Write([]byte(`{"name":"` + name + `","commit":{"id":"c0"},"protected":true,"effective_branch_protection_name":"` + rule + `"}`))
|
||||
return
|
||||
}
|
||||
}
|
||||
r.SetBasicAuth("mesh-admin", f.password) // the fake's own door, past the token checked above
|
||||
f.handle(w, r)
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
return &Forge{Client: forgehttp.Client{URL: srv.URL, TokenFile: file, Account: "mesh-tools"}}, &seen
|
||||
}
|
||||
|
||||
func TestTheToolsAccountCommitsOnANewBranchWithItsTokenThoughTheRulesAreNotItsToRead(t *testing.T) {
|
||||
f := newFake(t)
|
||||
forge, _ := asTools(t, f, nil, "read:user,write:issue,write:repository for mesh-tools")
|
||||
got, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": "fix/365",
|
||||
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(f.writes) != 1 || !got.Created {
|
||||
t.Fatalf("%d writes, %+v", len(f.writes), got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheToolsAccountIsRefusedAProtectedBranchTheForgeSaysItGuardsAndTheDefaultBranch(t *testing.T) {
|
||||
for branch, rule := range map[string]string{"release/1.0": "release/*", "fix/1-old": "fix/1-old", "main": ""} {
|
||||
f := newFake(t)
|
||||
protected := map[string]string{}
|
||||
if rule != "" {
|
||||
protected[branch] = rule
|
||||
}
|
||||
forge, _ := asTools(t, f, protected, "read:user,write:issue,write:repository for mesh-tools")
|
||||
_, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": branch,
|
||||
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||
if err == nil || !strings.Contains(err.Error(), "refused") || !strings.Contains(err.Error(), "ADR 0266") {
|
||||
t.Errorf("%s: %v", branch, err)
|
||||
}
|
||||
if rule != "" && !strings.Contains(err.Error(), rule) {
|
||||
t.Errorf("%s: the rule %s is not named: %v", branch, rule, err)
|
||||
}
|
||||
if len(f.writes)+len(f.patches) != 0 {
|
||||
t.Errorf("%s: something was written", branch)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestATokenKeptForTheAdministratorIsNeverSent(t *testing.T) {
|
||||
f := newFake(t)
|
||||
// What the module kept before ADR 0365: the admin's token, its scopes line naming no account.
|
||||
forge, seen := asTools(t, f, nil, "read:user,write:issue,write:repository")
|
||||
_, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": "fix/365",
|
||||
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
|
||||
if err == nil || !strings.Contains(err.Error(), "not mesh-tools's") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
if len(*seen) != 0 {
|
||||
t.Fatalf("the forge was asked %v", *seen)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheBundleHandedATokenFileNeedsNoAdminLogin(t *testing.T) {
|
||||
t.Setenv("MESH_GITEA_URL", "http://127.0.0.1:3000")
|
||||
t.Setenv("MESH_GITEA_TOKEN_FILE", "/state/token")
|
||||
t.Setenv("MESH_GITEA_TOOLS_USER", "")
|
||||
t.Setenv("MESH_GITEA_ADMIN_USER", "")
|
||||
t.Setenv("MESH_GITEA_ADMIN_PASSWORD_FILE", "")
|
||||
f, err := ForgeFromEnv()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.TokenFile != "/state/token" || f.Account != "mesh-tools" || f.User != "" || f.PasswordFile != "" {
|
||||
t.Fatalf("%+v", f.Client)
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// gitea_commit (novox/hq issue 375): one commit on a branch of its own, so an agent follows the branch / pull
|
||||
@@ -33,6 +35,11 @@ type CommitAsk struct {
|
||||
Message string
|
||||
Files []FileChange
|
||||
Patch string // a unified diff, as git diff writes it, instead of Files
|
||||
// PatchFile is the name of a file in the spool directory holding the patch (novox/hq issue 852), with the
|
||||
// SHA-256 its caller says it has; commitAsk reads it into Patch.
|
||||
PatchFile string
|
||||
PatchSHA256 string
|
||||
patchFileGiven bool
|
||||
}
|
||||
|
||||
// Committed is what gitea_commit made.
|
||||
@@ -67,6 +74,11 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
a := CommitAsk{Owner: text(args, "owner"), Repo: text(args, "repo"), Branch: text(args, "branch"),
|
||||
Base: text(args, "base"), Message: strings.TrimSpace(raw(args, "message"))}
|
||||
a.Patch = raw(args, "patch")
|
||||
if v, ok := args["patch_file"]; ok && v != nil {
|
||||
a.patchFileGiven = true
|
||||
a.PatchFile = raw(args, "patch_file")
|
||||
}
|
||||
a.PatchSHA256 = strings.ToLower(strings.TrimSpace(raw(args, "patch_sha256")))
|
||||
if a.Branch == "" {
|
||||
return a, refuse("branch is required: the branch the commit goes on, a branch of its own (created from base when absent). Nothing was sent")
|
||||
}
|
||||
@@ -109,11 +121,23 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
a.Files = append(a.Files, fc)
|
||||
}
|
||||
}
|
||||
sources := 0
|
||||
for _, g := range []bool{len(a.Files) > 0, strings.TrimSpace(a.Patch) != "", a.patchFileGiven} {
|
||||
if g {
|
||||
sources++
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case len(a.Files) == 0 && strings.TrimSpace(a.Patch) == "":
|
||||
return a, refuse("give files (each {path, content | content_base64 | delete: true}) or patch (a unified diff, as git diff writes it). Nothing was sent")
|
||||
case len(a.Files) > 0 && strings.TrimSpace(a.Patch) != "":
|
||||
return a, refuse("give files or patch, not both: one commit carries one of them. Nothing was sent")
|
||||
case sources == 0:
|
||||
return a, refuse("give files (each {path, content | content_base64 | delete: true}), patch (a unified diff, as git diff writes it) or patch_file (the name of a patch in the spool directory). Nothing was sent")
|
||||
case sources > 1:
|
||||
return a, refuse("give files, patch or patch_file, not more than one: one commit carries one of them. Nothing was sent")
|
||||
case a.PatchSHA256 != "" && !a.patchFileGiven:
|
||||
return a, refuse("patch_sha256 goes with patch_file only. Nothing was sent")
|
||||
case a.patchFileGiven:
|
||||
if err := checkPatchFileAsk(a); err != nil {
|
||||
return a, err
|
||||
}
|
||||
case len(a.Files) > maxFiles:
|
||||
return a, refuse("%d files is more than the %d one commit carries. Nothing was sent", len(a.Files), maxFiles)
|
||||
case len(a.Patch) > maxPatchBytes:
|
||||
@@ -132,13 +156,13 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
|
||||
return a, refuse("%s is named twice in files. Nothing was sent", fc.Path)
|
||||
}
|
||||
seen[fc.Path] = true
|
||||
given := 0
|
||||
have := 0
|
||||
for _, g := range []bool{fc.Content != nil, fc.ContentBase64 != nil, fc.Delete} {
|
||||
if g {
|
||||
given++
|
||||
have++
|
||||
}
|
||||
}
|
||||
if given != 1 {
|
||||
if have != 1 {
|
||||
return a, refuse("files[%d] (%s) needs exactly one of content, content_base64 or delete: true. Nothing was sent", i, fc.Path)
|
||||
}
|
||||
if fc.Content != nil && !utf8.ValidString(*fc.Content) {
|
||||
@@ -211,7 +235,32 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
|
||||
// Read the rules, or refuse: a branch the rules could not be read for is never written (fails closed).
|
||||
var rules []Protection
|
||||
if err := f.GetJSON(ctx, repo+"/branch_protections", &rules); err != nil {
|
||||
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
|
||||
var status *forgehttp.StatusError
|
||||
if !errors.As(err, &status) || status.Status != http.StatusForbidden {
|
||||
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
|
||||
}
|
||||
// The rules are a repository administrator's to read, and the tools' account is none (novox/hq ADR 0365
|
||||
// §2). What it may read is the protection the forge applies to an existing branch: one protected is
|
||||
// refused here. A branch not made yet is covered by no rule this account can see, and the forge itself
|
||||
// judges the commit that makes it against every rule, refusing one a rule guards.
|
||||
if err := targetRefused(a.Branch, r.DefaultBranch, nil); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b, err := f.branch(ctx, repo, a.Branch)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("whether a protection rule guards %s of %s/%s could not be read, so nothing was sent: %w", a.Branch, a.Owner, a.Repo, err)
|
||||
}
|
||||
if b != nil && b.Protected {
|
||||
rule := b.Rule
|
||||
if rule == "" {
|
||||
rule = a.Branch
|
||||
}
|
||||
rules = []Protection{{RuleName: rule}}
|
||||
if !ruleCovers(rule, a.Branch) {
|
||||
return nil, refuse("a commit on %s is refused: the protection rule %s guards it (novox/hq ADR 0266). Put it on a "+
|
||||
"branch of its own. Nothing was sent", a.Branch, rule)
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := targetRefused(a.Branch, r.DefaultBranch, rules); err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -11,8 +11,8 @@ import (
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
|
||||
// (internal/forgehttp, shared with the npm-registry bundle beside it).
|
||||
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the tools' own account (novox/hq
|
||||
// ADR 0365 §2; internal/forgehttp, shared with the npm-registry bundle beside it).
|
||||
|
||||
// Forge is the forge's HTTP interface.
|
||||
type Forge struct {
|
||||
@@ -91,12 +91,15 @@ func (p Protection) name() string {
|
||||
return p.BranchName
|
||||
}
|
||||
|
||||
// Branch is one branch with the commit it points at.
|
||||
// Branch is one branch with the commit it points at, and the protection the forge applies to it: what an account
|
||||
// that may not read the rules themselves (the tools' account, novox/hq ADR 0365) knows of them.
|
||||
type Branch struct {
|
||||
Name string `json:"name"`
|
||||
Commit struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"commit"`
|
||||
Protected bool `json:"protected"`
|
||||
Rule string `json:"effective_branch_protection_name"`
|
||||
}
|
||||
|
||||
// branch is the named branch, nil when the forge has none of that name.
|
||||
|
||||
@@ -619,11 +619,17 @@ func TestToolsAreTheOnesTheReadmeNamesAndTheManifestBuilds(t *testing.T) {
|
||||
if len(a.Loads) != 1 || a.Loads[0] != a.Binary {
|
||||
t.Errorf("the bundle loads %v, not its binary %s", a.Loads, a.Binary)
|
||||
}
|
||||
for _, k := range []string{"MESH_GITEA_URL", "MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD_FILE"} {
|
||||
// The tools' own account, never the admin login (novox/hq ADR 0365 §2).
|
||||
for _, k := range []string{"MESH_GITEA_URL", "MESH_GITEA_TOOLS_USER", "MESH_GITEA_TOKEN_FILE"} {
|
||||
if a.Env[k] == "" {
|
||||
t.Errorf("the bundle is not given %s", k)
|
||||
}
|
||||
}
|
||||
for _, k := range []string{"MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD_FILE"} {
|
||||
if a.Env[k] != "" {
|
||||
t.Errorf("the bundle is given %s: it acts as the tools' account, not the administrator", k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
|
||||
@@ -55,7 +55,9 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
|
||||
"the branch holds them — or patch, a unified diff as git diff writes it, which the forge applies. " +
|
||||
"At most 200 files, 4 MiB each and 16 MiB together, or a 4 MiB patch. Answers the commit it made and " +
|
||||
"the parent the forge made it on, read back from the branch, and says when the branch moved under " +
|
||||
"it. Replaces git commit and git " +
|
||||
"it. A patch too large to type goes as patch_file with patch_sha256: the name of a file in the " +
|
||||
"spool directory (read once, applied as patch is, removed afterwards, never read from anywhere " +
|
||||
"else). Replaces git commit and git " +
|
||||
"push of a feature branch. (a)",
|
||||
Input: map[string]any{
|
||||
"type": "object",
|
||||
@@ -79,7 +81,9 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
|
||||
"required": []string{"path"},
|
||||
},
|
||||
},
|
||||
"patch": str("a unified diff of the change, as git diff or git format-patch writes it; give this or files"),
|
||||
"patch": str("a unified diff of the change, as git diff or git format-patch writes it; give this, patch_file or files"),
|
||||
"patch_file": str("for a patch too large to type: the NAME (32 to 64 characters of 0-9a-f, chosen at random by the caller, no path) of a file the caller wrote into the forge patch spool directory (the module's forge-patches directory); needs patch_sha256; give this, patch or files"),
|
||||
"patch_sha256": str("with patch_file: the file's SHA-256, in hex; the commit is refused, and nothing is sent, when the file's differs"),
|
||||
},
|
||||
"required": []string{"owner", "repo", "branch", "message"},
|
||||
},
|
||||
@@ -88,7 +92,7 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return commit(ctx, f, a)
|
||||
return commitAsk(ctx, f, os.Getenv(patchDirEnv), a)
|
||||
}),
|
||||
},
|
||||
{
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// A patch too large to type into a call is handed over as a file (novox/hq issue 852). The caller writes it into
|
||||
// one spool directory the module declares (mode 0733: anyone writes, nobody lists) and names it in patch_file,
|
||||
// with the file's SHA-256. The tool runs as the machine's account, so it reads nothing but that directory: the
|
||||
// name is 32 to 64 characters of [0-9a-f], so it holds no separator and no dot; the file must be a regular file
|
||||
// (never a symlink); and its hash must be the caller's, because without the sticky bit another writer could have
|
||||
// replaced it. The file is removed once taken, whether the commit was made or not.
|
||||
|
||||
const patchDirEnv = "MESH_GITEA_PATCH_DIR"
|
||||
|
||||
var (
|
||||
patchFileName = regexp.MustCompile(`^[0-9a-f]{32,64}$`)
|
||||
patchSHA = regexp.MustCompile(`^[0-9a-f]{64}$`)
|
||||
)
|
||||
|
||||
// checkPatchFileAsk judges the arguments of a patch_file ask, before any file is looked at.
|
||||
func checkPatchFileAsk(a CommitAsk) error {
|
||||
if !patchFileName.MatchString(a.PatchFile) {
|
||||
return refuse("patch_file %q is not a name: 32 to 64 characters of 0-9a-f, no path (the file's name in the spool directory, chosen at random). Nothing was sent", a.PatchFile)
|
||||
}
|
||||
if !patchSHA.MatchString(a.PatchSHA256) {
|
||||
return refuse("patch_file needs patch_sha256: the file's SHA-256, 64 hex characters. Nothing was sent")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// commitAsk makes the commit the ask names; a patch_file is first read from the spool directory into the patch
|
||||
// and removed afterwards.
|
||||
func commitAsk(ctx context.Context, f *Forge, spool string, a CommitAsk) (*Committed, error) {
|
||||
if !a.patchFileGiven {
|
||||
return commit(ctx, f, a)
|
||||
}
|
||||
if err := checkPatchFileAsk(a); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if spool == "" {
|
||||
return nil, refuse("patch_file is not available: %s is not set, so there is no spool directory. Nothing was sent", patchDirEnv)
|
||||
}
|
||||
path := filepath.Join(spool, a.PatchFile)
|
||||
defer func() {
|
||||
// Whatever happened, the file is not left: removing a symlink removes the link, never its target.
|
||||
if st, err := os.Lstat(path); err == nil && !st.IsDir() {
|
||||
_ = os.Remove(path)
|
||||
}
|
||||
}()
|
||||
body, err := readSpooled(path, a.PatchFile)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
got := sha256.Sum256(body)
|
||||
if hex.EncodeToString(got[:]) != a.PatchSHA256 {
|
||||
return nil, refuse("the file %s is not the patch the caller hashed: its SHA-256 is %s, patch_sha256 says %s. Nothing was sent",
|
||||
a.PatchFile, hex.EncodeToString(got[:]), a.PatchSHA256)
|
||||
}
|
||||
a.Patch = string(body)
|
||||
return commit(ctx, f, a)
|
||||
}
|
||||
|
||||
// readSpooled reads a regular file of at most maxPatchBytes, never through a symlink.
|
||||
func readSpooled(path, name string) ([]byte, error) {
|
||||
st, err := os.Lstat(path)
|
||||
switch {
|
||||
case errors.Is(err, os.ErrNotExist):
|
||||
return nil, refuse("patch_file %s: no such file in the spool directory. Nothing was sent", name)
|
||||
case err != nil:
|
||||
return nil, refuse("patch_file %s cannot be read: %v. Nothing was sent", name, err)
|
||||
case !st.Mode().IsRegular():
|
||||
return nil, refuse("patch_file %s is not a regular file (a symlink or a directory is never followed). Nothing was sent", name)
|
||||
case st.Size() > maxPatchBytes:
|
||||
return nil, refuse("patch_file %s is %s, more than the %s one commit carries: split it into several commits. Nothing was sent", name, mib(int(st.Size())), mib(maxPatchBytes))
|
||||
}
|
||||
fh, err := os.OpenFile(path, os.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0)
|
||||
if err != nil {
|
||||
return nil, refuse("patch_file %s cannot be opened: %v. Nothing was sent", name, err)
|
||||
}
|
||||
defer fh.Close()
|
||||
fst, err := fh.Stat()
|
||||
if err != nil || !fst.Mode().IsRegular() {
|
||||
return nil, refuse("patch_file %s is not a regular file. Nothing was sent", name)
|
||||
}
|
||||
// A hard link to a file elsewhere is no symlink, but it is not the caller's own file either.
|
||||
if sys, ok := fst.Sys().(*syscall.Stat_t); !ok || sys.Nlink > 1 {
|
||||
return nil, refuse("patch_file %s has more than one hard link: it must be a file of its own. Nothing was sent", name)
|
||||
}
|
||||
body, err := io.ReadAll(io.LimitReader(fh, maxPatchBytes+1))
|
||||
switch {
|
||||
case err != nil:
|
||||
return nil, refuse("patch_file %s cannot be read: %v. Nothing was sent", name, err)
|
||||
case len(body) > maxPatchBytes:
|
||||
return nil, refuse("patch_file %s is more than the %s one commit carries: split it into several commits. Nothing was sent", name, mib(maxPatchBytes))
|
||||
case strings.TrimSpace(string(body)) == "":
|
||||
return nil, refuse("patch_file %s is empty. Nothing was sent", name)
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
@@ -0,0 +1,183 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// novox/hq issue 852: a patch too large to type into a call is handed over as a file in one spool directory.
|
||||
|
||||
const spoolName = "0123456789abcdef0123456789abcdef"
|
||||
const spoolPatch = "diff --git a/README.md b/README.md\n--- a/README.md\n+++ b/README.md\n@@ -1 +1 @@\n-hello\n+hi\n"
|
||||
|
||||
func sum(s string) string { h := sha256.Sum256([]byte(s)); return hex.EncodeToString(h[:]) }
|
||||
|
||||
func spoolWith(t *testing.T, name, content string) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
if name != "" {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return dir
|
||||
}
|
||||
|
||||
func fileAsk(t *testing.T, name, sha string) CommitAsk {
|
||||
t.Helper()
|
||||
return ask(t, map[string]any{"branch": "fix/9", "patch_file": name, "patch_sha256": sha})
|
||||
}
|
||||
|
||||
func TestAPatchFileIsAppliedAsThePatchWouldBeAndTheFileIsGone(t *testing.T) {
|
||||
f := newFake(t)
|
||||
dir := spoolWith(t, spoolName, spoolPatch)
|
||||
got, err := commitAsk(context.Background(), f.serve(), dir, fileAsk(t, spoolName, sum(spoolPatch)))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(f.patches) != 1 || f.patches[0]["content"] != spoolPatch || f.patches[0]["new_branch"] != "fix/9" || got.Commit != "p1" {
|
||||
t.Fatalf("%v %+v", f.patches, got)
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(dir, spoolName)); !os.IsNotExist(err) {
|
||||
t.Fatalf("the file is still there: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPatchFileIsRemovedWhenTheForgeRefusesIt(t *testing.T) {
|
||||
f := newFake(t)
|
||||
dir := spoolWith(t, spoolName, spoolPatch)
|
||||
// the default branch is refused after the file was read
|
||||
a := ask(t, map[string]any{"branch": "main", "patch_file": spoolName, "patch_sha256": sum(spoolPatch)})
|
||||
if _, err := commitAsk(context.Background(), f.serve(), dir, a); err == nil {
|
||||
t.Fatal("taken")
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(dir, spoolName)); !os.IsNotExist(err) {
|
||||
t.Fatalf("the file is still there: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPatchFileThatCannotBeTakenIsRefusedBeforeAnythingIsSent(t *testing.T) {
|
||||
outside := filepath.Join(t.TempDir(), "secret")
|
||||
if err := os.WriteFile(outside, []byte(spoolPatch), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type tc struct {
|
||||
name string
|
||||
setup func(dir string)
|
||||
file string
|
||||
sha string
|
||||
want string
|
||||
}
|
||||
for _, c := range []tc{
|
||||
{"wrong sha", func(d string) { _ = os.WriteFile(filepath.Join(d, spoolName), []byte(spoolPatch), 0o600) }, spoolName, sum("other"), "sha256"},
|
||||
{"not found", func(string) {}, spoolName, sum(spoolPatch), "no such file"},
|
||||
{"symlink", func(d string) { _ = os.Symlink(outside, filepath.Join(d, spoolName)) }, spoolName, sum(spoolPatch), "regular file"},
|
||||
{"hard link", func(d string) { _ = os.Link(outside, filepath.Join(d, spoolName)) }, spoolName, sum(spoolPatch), "hard link"},
|
||||
{"directory", func(d string) { _ = os.Mkdir(filepath.Join(d, spoolName), 0o700) }, spoolName, sum(spoolPatch), "regular file"},
|
||||
{"too large", func(d string) {
|
||||
_ = os.WriteFile(filepath.Join(d, spoolName), []byte(strings.Repeat("x", maxPatchBytes+1)), 0o600)
|
||||
}, spoolName, sum(spoolPatch), "MiB"},
|
||||
{"empty", func(d string) { _ = os.WriteFile(filepath.Join(d, spoolName), nil, 0o600) }, spoolName, sum(""), "empty"},
|
||||
} {
|
||||
f := newFake(t)
|
||||
dir := t.TempDir()
|
||||
c.setup(dir)
|
||||
_, err := commitAsk(context.Background(), f.serve(), dir, fileAsk(t, c.file, c.sha))
|
||||
if err == nil || !strings.Contains(err.Error(), c.want) || !strings.Contains(err.Error(), "Nothing was sent") {
|
||||
t.Errorf("%s: %v", c.name, err)
|
||||
}
|
||||
if len(f.patches)+len(f.writes) != 0 {
|
||||
t.Errorf("%s: something was sent", c.name)
|
||||
}
|
||||
}
|
||||
if b, err := os.ReadFile(outside); err != nil || string(b) != spoolPatch {
|
||||
t.Errorf("the file outside the spool was touched: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPatchFileAskThatCannotBeReadIsRefused(t *testing.T) {
|
||||
ok := sum(spoolPatch)
|
||||
for name, args := range map[string]map[string]any{
|
||||
"no sha": {"patch_file": spoolName},
|
||||
"sha not hex": {"patch_file": spoolName, "patch_sha256": strings.Repeat("z", 64)},
|
||||
"sha too short": {"patch_file": spoolName, "patch_sha256": ok[:63]},
|
||||
"empty name": {"patch_file": "", "patch_sha256": ok, "files": filesArg()},
|
||||
"dotdot": {"patch_file": "../" + spoolName, "patch_sha256": ok},
|
||||
"slash": {"patch_file": "a/" + spoolName, "patch_sha256": ok},
|
||||
"absolute": {"patch_file": "/etc/passwd", "patch_sha256": ok},
|
||||
"too short": {"patch_file": strings.Repeat("a", 31), "patch_sha256": ok},
|
||||
"too long": {"patch_file": strings.Repeat("a", 65), "patch_sha256": ok},
|
||||
"upper case": {"patch_file": strings.ToUpper(spoolName), "patch_sha256": ok},
|
||||
"not hex": {"patch_file": strings.Repeat("g", 32), "patch_sha256": ok},
|
||||
"with patch": {"patch_file": spoolName, "patch_sha256": ok, "patch": "diff"},
|
||||
"with files": {"patch_file": spoolName, "patch_sha256": ok, "files": filesArg(map[string]any{"path": "x", "content": "x"})},
|
||||
"sha without it": {"patch": "diff", "patch_sha256": ok},
|
||||
} {
|
||||
full := map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b"}
|
||||
for k, v := range args {
|
||||
full[k] = v
|
||||
}
|
||||
_, err := readCommitAsk(full)
|
||||
if err == nil || !strings.Contains(err.Error(), "Nothing was sent") {
|
||||
t.Errorf("%s: %v", name, err)
|
||||
}
|
||||
}
|
||||
// 64 characters is taken; so is an upper-case sha.
|
||||
for _, n := range []string{spoolName, strings.Repeat("ab", 32)} {
|
||||
if _, err := readCommitAsk(map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b",
|
||||
"patch_file": n, "patch_sha256": strings.ToUpper(sum(spoolPatch))}); err != nil {
|
||||
t.Errorf("%s: %v", n, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPatchFileWithoutASpoolDirectoryIsRefused(t *testing.T) {
|
||||
f := newFake(t)
|
||||
_, err := commitAsk(context.Background(), f.serve(), "", fileAsk(t, spoolName, sum(spoolPatch)))
|
||||
if err == nil || !strings.Contains(err.Error(), "MESH_GITEA_PATCH_DIR") || !strings.Contains(err.Error(), "Nothing was sent") {
|
||||
t.Fatalf("%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheManifestDeclaresTheSpoolDirectoryAndGivesItToTheBundle(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m struct {
|
||||
Resources []struct {
|
||||
ID, Type, Path, Mode, Owner string
|
||||
} `json:"resources"`
|
||||
Build struct {
|
||||
Artifacts []struct {
|
||||
From string `json:"from"`
|
||||
Env map[string]string `json:"env"`
|
||||
} `json:"artifacts"`
|
||||
} `json:"build"`
|
||||
}
|
||||
if err := jsonInto(raw, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := false
|
||||
for _, r := range m.Resources {
|
||||
if r.ID == "forge-patches" {
|
||||
found = true
|
||||
if r.Type != "directory" || r.Mode != "0733" || r.Owner != "${machine:account}" || r.Path == "" {
|
||||
t.Errorf("%+v", r)
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("no forge-patches directory")
|
||||
}
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a.From == "cmd/forge" && a.Env["MESH_GITEA_PATCH_DIR"] != "${dir:forge-patches}" {
|
||||
t.Errorf("env %v", a.Env)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The module's directory, where its manifest and its files/ are.
|
||||
const moduleDir = "../.."
|
||||
|
||||
// templateText is the text a file resource places on the machine: its template's, in the module's files/
|
||||
// (novox/hq ADR 0349 §2). A resource that names no template, or a template that is not there, fails the test.
|
||||
func templateText(t *testing.T, r map[string]any) string {
|
||||
t.Helper()
|
||||
name, _ := r["template"].(string)
|
||||
if name == "" {
|
||||
t.Fatalf("resource %v names no template", r["id"])
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(moduleDir, filepath.FromSlash(name)))
|
||||
if err != nil {
|
||||
t.Fatalf("resource %v names %s: %v", r["id"], name, err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
// No file resource carries its text inline: each names its template in files/, and the build packs files/
|
||||
// into the one archive the controller reads the templates back from (novox/hq ADR 0349 §2).
|
||||
func TestEveryFileTakesItsTextFromATemplate(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join(moduleDir, "module.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m struct {
|
||||
Resources []map[string]any `json:"resources"`
|
||||
Build struct {
|
||||
Artifacts []map[string]any `json:"artifacts"`
|
||||
} `json:"build"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, r := range m.Resources {
|
||||
if _, inline := r["content"]; inline {
|
||||
t.Errorf("resource %v keeps its text inline: keep it in files/ and name it as the template", r["id"])
|
||||
}
|
||||
if r["type"] == "file" {
|
||||
templateText(t, r)
|
||||
}
|
||||
}
|
||||
packs := 0
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a["kind"] == "archive" && a["from"] == "files" {
|
||||
packs++
|
||||
if a["name"] != "files" {
|
||||
t.Errorf("the templates archive is named %v, not files", a["name"])
|
||||
}
|
||||
}
|
||||
}
|
||||
if packs != 1 {
|
||||
t.Errorf("the build packs files/ into %d archives, not one", packs)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{}
|
||||
@@ -0,0 +1 @@
|
||||
{}
|
||||
@@ -0,0 +1 @@
|
||||
https://${bound:route:name-web}
|
||||
@@ -0,0 +1,3 @@
|
||||
# The accounts whose "ready at <sha>" comment marks a pull request ready (novox/hq issue 722), from the setting
|
||||
# ready-authors, written by the mesh. Replaced on every push; change the setting, never this file.
|
||||
authors=${config:ready-authors}
|
||||
@@ -0,0 +1,7 @@
|
||||
GITEA__security__INTERNAL_TOKEN=${secret:internal-token}
|
||||
GITEA__database__DB_TYPE=postgres
|
||||
GITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}
|
||||
GITEA__database__NAME=${bound:postgres-database:as}
|
||||
GITEA__database__USER=${bound:postgres-database:as}
|
||||
GITEA__database__PASSWD=${secret:postgres-database}
|
||||
GITEA__webhook__ALLOWED_HOST_LIST=172.17.0.1
|
||||
@@ -1,5 +1,10 @@
|
||||
module gitea
|
||||
|
||||
go 1.22
|
||||
go 1.25.0
|
||||
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.7
|
||||
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009
|
||||
|
||||
require (
|
||||
golang.org/x/crypto v0.50.0
|
||||
golang.org/x/sys v0.43.0 // indirect
|
||||
)
|
||||
|
||||
@@ -1,2 +1,6 @@
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009 h1:I9rm8HhYRJ0+mzQQz+VAH0kgv2YJWkvctEc67Yd0PX8=
|
||||
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
|
||||
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
|
||||
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
|
||||
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
|
||||
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
|
||||
+205
-84
@@ -40,10 +40,12 @@ import { emit, on } from "@novox/mesh-sdk/events";
|
||||
import { registerModuleTools } from "@novox/mesh-sdk/tools";
|
||||
import { commitStatusTool } from "./delivery.js";
|
||||
import { mergeTool } from "./merge.js";
|
||||
import { GiteaClient, movedSince } from "./client.js";
|
||||
import { GiteaAdmin, GiteaClient, movedSince } from "./client.js";
|
||||
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT, commentFor, headsToAnnounce, statusesFor, type Announced, type Checked,
|
||||
type RepoCheckFacts, reposToLook } from "./pulls.js";
|
||||
repoCheckFacts, reposToLook } from "./pulls.js";
|
||||
import { GateWriter, clonedAs, type BuildStarted } from "./gate.js";
|
||||
import { HOOK_PATH, HOOK_PORT, Pace, claimHead, ensureSystemHook, eventsTool, keptSecret, receiver, type ForgeEvent } from "./webhook.js";
|
||||
import { keepSiblings, noteVerdict, readSiblings, siblingsFile, siblingsOf, weighSiblings } from "./siblings.js";
|
||||
import { announcementOf, keepTook, lateVerdictComment, outOfDateWords, sameCommit, tookFile, weighTrunks } from "./trunk.js";
|
||||
|
||||
// Without a way to a token — configured, or mintable with the admin account (token.ts) — there is
|
||||
@@ -135,6 +137,7 @@ async function pollMerged(client: GiteaClient): Promise<void> {
|
||||
await emit("pull.closed", { owner: repo.owner, repo: repo.name, number: pull.number, title: pull.title,
|
||||
head: pull.head, head_sha: pull.head_sha, base: pull.base, html_url: pull.html_url });
|
||||
console.log(`[gitea] announced ${repo.full_name}#${pull.number} closed unmerged`);
|
||||
await siblingMoves(client, repo.full_name, pull.number);
|
||||
}
|
||||
announced.add(closedKey);
|
||||
changed = true;
|
||||
@@ -145,57 +148,9 @@ async function pollMerged(client: GiteaClient): Promise<void> {
|
||||
// at once.
|
||||
const fresh = !!pull.merged_at && !!since && pull.merged_at > since;
|
||||
if (primedMerges && fresh) {
|
||||
// What it changed, asked for only now: a module is rebuilt because a file inside its own
|
||||
// directory moved, and without this every module built from a repository is rebuilt for a
|
||||
// change to any of them (novox/hq 04-ISSUES/131).
|
||||
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
|
||||
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
|
||||
// change inside one is that module's, held or not, and only a file in none is shared code. Not
|
||||
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
|
||||
let moduleDirs: string[] | null = null;
|
||||
if (!changed.truncated) {
|
||||
try {
|
||||
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
|
||||
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
|
||||
// from the forge's word when its owner never heard the head.
|
||||
let headChecks: Record<string, string> | null = null;
|
||||
if (pull.head_sha) {
|
||||
try {
|
||||
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
await emit("pull.merged", {
|
||||
owner: repo.owner,
|
||||
repo: repo.name,
|
||||
number: pull.number,
|
||||
title: pull.title,
|
||||
body: pull.body,
|
||||
head_sha: pull.head_sha,
|
||||
...(headChecks ? { head_checks: headChecks } : {}),
|
||||
head: pull.head,
|
||||
base: pull.base,
|
||||
merge_commit_sha: pull.merge_commit_sha,
|
||||
merged_at: pull.merged_at,
|
||||
clone_url: repo.clone_url,
|
||||
html_url: pull.html_url,
|
||||
paths: changed.paths,
|
||||
paths_truncated: changed.truncated,
|
||||
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
|
||||
// not built.
|
||||
removed: changed.removed,
|
||||
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
|
||||
});
|
||||
// Said, because a trigger that fires silently is indistinguishable from one that did not
|
||||
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
|
||||
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
|
||||
if (await announceMerge(client, repo, pull)) pace.miss(`the merge of ${repo.full_name}#${pull.number}`);
|
||||
}
|
||||
else announced.add(pull.merge_commit_sha);
|
||||
announced.add(pull.merge_commit_sha);
|
||||
changed = true;
|
||||
}
|
||||
@@ -206,6 +161,75 @@ async function pollMerged(client: GiteaClient): Promise<void> {
|
||||
lastLook = began;
|
||||
}
|
||||
|
||||
type RepoOf = { owner: string; name: string; full_name: string; clone_url?: string };
|
||||
type PullOf = Awaited<ReturnType<GiteaClient["listPullRequests"]>>[number];
|
||||
|
||||
/** **A merge, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
|
||||
async function announceMerge(client: GiteaClient, repo: RepoOf, pull: PullOf): Promise<boolean> {
|
||||
if (!pull.merged || !pull.merge_commit_sha || announced.has(pull.merge_commit_sha)) return false;
|
||||
// Recorded before the first wait, so the other path finds it taken while this one is still asking the forge.
|
||||
announced.add(pull.merge_commit_sha);
|
||||
try {
|
||||
// What it changed, asked for only now: a module is rebuilt because a file inside its own
|
||||
// directory moved, and without this every module built from a repository is rebuilt for a
|
||||
// change to any of them (novox/hq 04-ISSUES/131).
|
||||
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
|
||||
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
|
||||
// change inside one is that module's, held or not, and only a file in none is shared code. Not
|
||||
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
|
||||
let moduleDirs: string[] | null = null;
|
||||
if (!changed.truncated) {
|
||||
try {
|
||||
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
|
||||
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
|
||||
// from the forge's word when its owner never heard the head.
|
||||
let headChecks: Record<string, string> | null = null;
|
||||
if (pull.head_sha) {
|
||||
try {
|
||||
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
await emit("pull.merged", {
|
||||
owner: repo.owner,
|
||||
repo: repo.name,
|
||||
number: pull.number,
|
||||
title: pull.title,
|
||||
body: pull.body,
|
||||
head_sha: pull.head_sha,
|
||||
...(headChecks ? { head_checks: headChecks } : {}),
|
||||
head: pull.head,
|
||||
base: pull.base,
|
||||
merge_commit_sha: pull.merge_commit_sha,
|
||||
merged_at: pull.merged_at,
|
||||
clone_url: repo.clone_url,
|
||||
html_url: pull.html_url,
|
||||
paths: changed.paths,
|
||||
paths_truncated: changed.truncated,
|
||||
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
|
||||
// not built.
|
||||
removed: changed.removed,
|
||||
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
|
||||
});
|
||||
// Said, because a trigger that fires silently is indistinguishable from one that did not
|
||||
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
|
||||
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
|
||||
// Its own merge of the head that was read is no sibling's move; any other is (ADR 0382 §5).
|
||||
await siblingMoves(client, repo.full_name, pull.number);
|
||||
} catch (err) {
|
||||
announced.delete(pull.merge_commit_sha);
|
||||
throw err;
|
||||
}
|
||||
keepAnnounced();
|
||||
return true;
|
||||
}
|
||||
|
||||
// **Every new head of an open pull request is announced, once** (novox/hq to-be 45 §9): the mesh checks it
|
||||
// against every machine of its facts before it merges. Kept beside the merges' record, so a restart
|
||||
// announces nothing twice; the first look on a machine with no record announces only what moved in the
|
||||
@@ -228,6 +252,33 @@ function keepHeads(): void {
|
||||
writeFileSync(tmp, JSON.stringify({ heads }));
|
||||
renameSync(tmp, pullsRecord);
|
||||
}
|
||||
/** **A head, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
|
||||
async function announceHead(client: GiteaClient, repo: RepoOf, pull: OpenOf): Promise<boolean> {
|
||||
const key = `${repo.full_name}#${pull.number}`;
|
||||
const sha = String(pull.head_sha ?? "");
|
||||
const before = heads[key];
|
||||
// Claimed before the first wait, so the other path finds it taken while this one is still announcing.
|
||||
if (!claimHead(heads, key, sha)) return false;
|
||||
try {
|
||||
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
|
||||
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url ?? "", pull));
|
||||
} catch (err) {
|
||||
if (before === undefined) delete heads[key];
|
||||
else heads[key] = before;
|
||||
throw err;
|
||||
}
|
||||
keepHeads();
|
||||
// A sibling's new head puts out of date every verdict that read the sibling at another head (ADR 0382 §5).
|
||||
await siblingMoves(client, repo.full_name, pull.number);
|
||||
// Said, so an operator knows the mesh was asked to check it.
|
||||
console.log(`[gitea] announced ${key} at ${sha.slice(0, 8)} to be checked before it merges`);
|
||||
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
|
||||
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
|
||||
await gates.pending(repo.owner, repo.name, sha);
|
||||
return true;
|
||||
}
|
||||
type OpenOf = Awaited<ReturnType<GiteaClient["listOpenPullsSince"]>>[number];
|
||||
|
||||
let lastPullLook = "";
|
||||
// **A sweep now and then looks at every repository** (novox/hq issue 347), whatever moved: the floor is a
|
||||
// heuristic, and a heuristic that misses is caught by the next sweep rather than never.
|
||||
@@ -259,15 +310,8 @@ async function pollPulls(client: GiteaClient): Promise<void> {
|
||||
const key = `${repo.full_name}#${pull.number}`;
|
||||
const fresh = primedPulls || (!!pull.updated_at && pull.updated_at > dayAgo);
|
||||
if (fresh) {
|
||||
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
|
||||
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url, pull));
|
||||
// Said, so an operator knows the mesh was asked to check it.
|
||||
console.log(`[gitea] announced ${key} at ${String(pull.head_sha).slice(0, 8)} to be checked before it merges`);
|
||||
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
|
||||
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
|
||||
await gates.pending(repo.owner, repo.name, String(pull.head_sha));
|
||||
}
|
||||
heads[key] = String(pull.head_sha);
|
||||
if (await announceHead(client, repo, pull)) pace.miss(`${key} at ${String(pull.head_sha).slice(0, 8)}`);
|
||||
} else heads[key] = String(pull.head_sha);
|
||||
changed = true;
|
||||
}
|
||||
if (await trunksMoved(client, repo.owner, repo.name)) changedTrunks = true;
|
||||
@@ -303,7 +347,22 @@ function keepTrunks(): void {
|
||||
|
||||
/** Whether a trunk of one repository moved, weighed with this module's forge and its one writer (trunk.ts). */
|
||||
function trunksMoved(client: GiteaClient, owner: string, name: string): Promise<boolean> {
|
||||
return weighTrunks(client, gates, trunks, owner, name);
|
||||
return weighTrunks(client, gates, trunks, owner, name, () => siblingMoves(client, `${owner}/${name}`));
|
||||
}
|
||||
|
||||
// **The siblings each open pull request's latest verdict names** (novox/hq ADR 0382 §5, siblings.ts), by member, kept beside
|
||||
// the other records. A sibling's new head, close unmerged, merge, or a move of its base puts the merge check of every
|
||||
// member whose verdict names it back to pending, with the reason; nothing is run for it.
|
||||
const siblingsRecord = siblingsFile(process.env.MESH_GITEA_STATE_DIR);
|
||||
const siblingNamed = readSiblings(siblingsRecord);
|
||||
|
||||
/** A sibling moved, weighed (ADR 0382 §5): never fatal to what heard it. */
|
||||
async function siblingMoves(client: GiteaClient, repository: string, number?: number): Promise<void> {
|
||||
try {
|
||||
if (await weighSiblings(client, gates, siblingNamed, repository, number)) keepSiblings(siblingsRecord, siblingNamed);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] whether a sibling of ${repository}${number ? "#" + number : ""} moved could not be weighed — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
|
||||
// **The verdict, set where the pull request shows it.** Every verdict is the head commit's status; one
|
||||
@@ -325,6 +384,17 @@ async function setVerdict(client: GiteaClient, event: { body: unknown }): Promis
|
||||
console.error("[gitea] a merge check's verdict named no repository, commit or verdict; ignored");
|
||||
return;
|
||||
}
|
||||
// The siblings this verdict read are kept for the member (ADR 0382 §5); a verdict that read none removes the record.
|
||||
const siblings = siblingsOf(c.siblings);
|
||||
if (c.number && noteVerdict(siblingNamed, `${c.owner}/${c.repo}#${c.number}`.toLowerCase(), c.commit, siblings)) {
|
||||
keepSiblings(siblingsRecord, siblingNamed);
|
||||
}
|
||||
// A sibling that moved while the check ran is weighed once this verdict has landed, in the head's own turn after it.
|
||||
if (siblings.length) {
|
||||
for (const repository of new Set(siblings.map((s) => s.repository))) {
|
||||
void Promise.resolve().then(() => siblingMoves(client, repository));
|
||||
}
|
||||
}
|
||||
// Each status links to the pull request, where the delivery's view is kept (novox/hq ADR 0239).
|
||||
let target: string | undefined;
|
||||
let base: string | undefined;
|
||||
@@ -362,22 +432,6 @@ async function setVerdict(client: GiteaClient, event: { body: unknown }): Promis
|
||||
console.log(`[gitea] ${c.owner}/${c.repo}#${c.number ?? "?"} at ${c.commit.slice(0, 8)}: merge check ${c.verdict}`);
|
||||
}
|
||||
|
||||
// **What only the forge knows of a repository check said as a warning** (novox/hq issue 293): whether the
|
||||
// head holds a merge-check.sh at all, and — when it does not — whether the base branch's protection
|
||||
// requires mesh/repo-check. Asked only for a warning; unknown is left undefined, which statusesFor reads
|
||||
// as possibly required: a failure on a status nothing requires blocks nothing, a success on one that is
|
||||
// required would let an untested repository merge.
|
||||
async function repoCheckFacts(client: GiteaClient, c: Checked, base: string | undefined): Promise<RepoCheckFacts> {
|
||||
if (c["repo-check"]?.verdict !== "warning") return {};
|
||||
const defined = await client.holdsFile(c.owner, c.repo, c.commit, "merge-check.sh").catch(() => undefined);
|
||||
if (defined !== false) return { defined };
|
||||
const rules = await client.branchProtections(c.owner, c.repo).catch(() => undefined);
|
||||
if (!rules) return { defined };
|
||||
const rule = rules.find((p) => (p.rule_name ?? p.branch_name) === (base || "main"));
|
||||
const required = !!rule?.enable_status_check && (rule.status_check_contexts ?? []).includes(REPO_CHECK_CONTEXT);
|
||||
return { defined, required };
|
||||
}
|
||||
|
||||
// **This module is the one writer of a head's mesh/merge-gate** (novox/hq issue 446): the announcement's pending words,
|
||||
// where the check is when a build machine takes it, and the verdict — each head's writes in one chain, the verdict last
|
||||
// (gate.ts). A second writer elsewhere raced the verdict and could leave a pass reading pending.
|
||||
@@ -391,10 +445,16 @@ const gates = new GateWriter({
|
||||
// **The status tool is served here, beside the writer** (novox/hq issue 456): the runtime starts each bundle as its own
|
||||
// process, so a recheck's pending written from the tools' bundle bypassed this writer's chain. Without a forge to
|
||||
// reach, nothing is served, as the tools' bundle serves nothing then.
|
||||
registerModuleTools("gitea", () => (gitea ? [commitStatusTool({
|
||||
/** How fast the polls look: 30 s until the forge's events are heard, five minutes while they are; and what
|
||||
* gitea_events says of them (webhook.ts). */
|
||||
const pace = new Pace();
|
||||
if (!gitea) pace.unheard("the forge is not watched: no way to a token", false);
|
||||
|
||||
// gitea_events is served whether or not the forge is watched: it says so (novox/hq issue 729).
|
||||
registerModuleTools("gitea", () => [eventsTool(pace), ...(gitea ? [commitStatusTool({
|
||||
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
|
||||
gates,
|
||||
}), mergeTool(gitea, { gates })] : []));
|
||||
}), mergeTool(gitea, { gates })] : [])]);
|
||||
|
||||
// **A build machine took an ask** (the build seat's `started`, novox/hq ADR 0157): said on the head it checks, when
|
||||
// the head is an open pull request's this module announced. Any other build — a branch, a merge's — says nothing.
|
||||
@@ -413,12 +473,70 @@ async function buildStarted(event: { body: unknown }): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
/** What the forge's event asks of the announcer at once (issue 729): a new head is announced, and a merge. */
|
||||
async function actOn(client: GiteaClient, event: ForgeEvent): Promise<void> {
|
||||
const b = event.body as { owner?: string; repo?: string; number?: number };
|
||||
if (!b.owner || !b.repo || !b.number) return;
|
||||
if (event.name === "forge.pull-request.closed") {
|
||||
// Closed unmerged is a sibling's move; a merge is weighed with its announcement below (ADR 0382 §5).
|
||||
if (!(event.body as { merged?: boolean }).merged) await siblingMoves(client, `${b.owner}/${b.repo}`, b.number);
|
||||
return;
|
||||
}
|
||||
const head = ["forge.pull-request.opened", "forge.pull-request.synchronized", "forge.pull-request.reopened"].includes(event.name);
|
||||
const merged = event.name === "forge.pull-request.merged";
|
||||
if (!head && !merged) return;
|
||||
const [repo, pull] = await Promise.all([client.getRepo(b.owner, b.repo), client.getPullRequest(b.owner, b.repo, b.number)]);
|
||||
if (head && pull.state === "open") await announceHead(client, repo, pull);
|
||||
if (merged) await announceMerge(client, repo, pull);
|
||||
}
|
||||
|
||||
/**
|
||||
* **The receiver, and the forge's one system webhook pointing at it** (issue 729). The secret is minted and kept beside
|
||||
* the token; the webhook is made or corrected with the admin account (ADR 0266: the tools' token holds no admin scope).
|
||||
* Answers whether events are heard; a forge that refuses is said, and the polls keep their pace.
|
||||
*/
|
||||
async function listenForForge(client: GiteaClient): Promise<boolean> {
|
||||
const stateDir = process.env.MESH_GITEA_STATE_DIR;
|
||||
const url = process.env.MESH_GITEA_HOOK_URL;
|
||||
if (!stateDir || !url) {
|
||||
console.log("[gitea] no webhook receiver: MESH_GITEA_STATE_DIR or MESH_GITEA_HOOK_URL unset, so the polls alone watch the forge");
|
||||
pace.unheard("MESH_GITEA_STATE_DIR or MESH_GITEA_HOOK_URL unset", false);
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
const secret = keptSecret(join(stateDir, "webhook.secret"));
|
||||
const port = Number(process.env.MESH_GITEA_HOOK_PORT ?? HOOK_PORT);
|
||||
const host = process.env.MESH_GITEA_HOOK_HOST ?? "0.0.0.0";
|
||||
const server = receiver({ secret, port, heard: (kind, delivery) => pace.heard(kind, delivery),
|
||||
refused: (reason, from) => pace.refused(reason, from),
|
||||
publish: (event) => emit(event.name, event.body),
|
||||
act: (event) => actOn(client, event) });
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(port, host, () => resolve());
|
||||
});
|
||||
const hook = await ensureSystemHook(GiteaAdmin.fromEnv(), url.replace(/\/+$/, "") + HOOK_PATH, secret);
|
||||
pace.listening(host, port, hook.url, hook);
|
||||
console.log(`[gitea] listening for the forge's events on ${host}:${port}; its system webhook ${hook.id} ${hook.did} ` +
|
||||
`at ${hook.url} (in place of ${hook.removed.system} system, ${hook.removed.default} default and ` +
|
||||
`${hook.removed.repository} repository webhook(s)${hook.removed.repositories.length ? ` in ${hook.removed.repositories.join(", ")}` : ""}), ` +
|
||||
`${hook.events.length} event families, signed`);
|
||||
return true;
|
||||
} catch (err) {
|
||||
const why = err instanceof Error ? err.message : String(err);
|
||||
pace.unheard(why);
|
||||
console.error(`[gitea] the forge's events are not heard, so the polls keep their pace — ${why}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (gitea) {
|
||||
const client = gitea;
|
||||
// A poll that fails says so once, not once a minute: the same reason repeating (the forge not up
|
||||
// yet, the admin account refused on a restored forge) is one fact, and a recovery is worth a line.
|
||||
let failing: string | null = null;
|
||||
const tick = (fn: () => Promise<void>, everyMs: number): void => {
|
||||
const tick = (fn: () => Promise<void>, every: number | (() => number)): void => {
|
||||
const everyMs = (): number => (typeof every === "number" ? every : every());
|
||||
// **One pass at a time** (novox/hq issue 250): the next pass is scheduled when this one has ended, so a
|
||||
// pass that outlasts its interval delays the next instead of running beside it — two passes at once
|
||||
// could each announce the same merge before either recorded it.
|
||||
@@ -433,12 +551,15 @@ if (gitea) {
|
||||
if (why !== failing) console.error(`[gitea] not watching until this clears — ${why}`);
|
||||
failing = why;
|
||||
})
|
||||
.finally(() => setTimeout(run, everyMs));
|
||||
.finally(() => setTimeout(run, everyMs()));
|
||||
run();
|
||||
};
|
||||
// **The forge's events first, the polls as the safety net** (novox/hq issue 729): while verified deliveries arrive the
|
||||
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
|
||||
await listenForForge(client);
|
||||
tick(() => pollRepos(client), 60_000);
|
||||
tick(() => pollMerged(client), 30_000);
|
||||
tick(() => pollPulls(client), 30_000);
|
||||
tick(() => pollMerged(client), () => pace.every());
|
||||
tick(() => pollPulls(client), () => pace.every());
|
||||
await on("mesh-controller.checked", (event) => {
|
||||
const c = (event.body ?? {}) as Checked;
|
||||
// A head's own verdict takes its turn after every write of that head's gate already asked for (gate.ts).
|
||||
|
||||
@@ -1,7 +1,17 @@
|
||||
// Package forgehttp is the forge's HTTP interface as this module's Go bundles reach it (cmd/forge,
|
||||
// cmd/npm-registry): the forge on its own machine, as its admin account, whose password the vault delivered
|
||||
// into a file this module owns (novox/hq ADR 0086), by basic authentication. The password is read when a call
|
||||
// is made and never logged, answered or put in an error. One copy for both bundles (novox/hq issue 539).
|
||||
// cmd/npm-registry), on the forge's own machine. One copy for both bundles (novox/hq issue 539).
|
||||
//
|
||||
// Two ways in, by what the manifest hands the bundle:
|
||||
//
|
||||
// - **as the tools' account** (novox/hq ADR 0365 §2), when MESH_GITEA_TOKEN_FILE names the token the module
|
||||
// keeps for it (token.ts mints it, for `mesh-tools`, and keeps it at 0600 with its scopes line beside it). A
|
||||
// token whose scopes line does not say it is the tools' account's — one kept from before, the administrator's —
|
||||
// is never sent: the call fails, saying so, until the module has minted the account's own. This bundle mints
|
||||
// nothing; a token the forge rejects is renewed by the module's next call, and said here.
|
||||
// - **as the admin account**, whose password the vault delivered into a file this module owns (novox/hq ADR
|
||||
// 0086), by basic authentication, when no token file is named.
|
||||
//
|
||||
// A credential is read when a call is made and never logged, answered or put in an error.
|
||||
package forgehttp
|
||||
|
||||
import (
|
||||
@@ -25,12 +35,31 @@ type Client struct {
|
||||
URL string // e.g. http://127.0.0.1:3000
|
||||
User string
|
||||
PasswordFile string
|
||||
HTTP *http.Client // when nil, a client with Timeout
|
||||
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
|
||||
// TokenFile, when set, is the tools' account's token (MESH_GITEA_TOKEN_FILE), sent instead of the admin login;
|
||||
// Account the account its scopes line must name (MESH_GITEA_TOOLS_USER, default mesh-tools).
|
||||
TokenFile string
|
||||
Account string
|
||||
HTTP *http.Client // when nil, a client with Timeout
|
||||
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
|
||||
}
|
||||
|
||||
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset.
|
||||
// ToolsAccount is the account the forge's tools act as when the manifest names none (token.ts, tools-account.ts).
|
||||
const ToolsAccount = "mesh-tools"
|
||||
|
||||
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset. A bundle
|
||||
// handed MESH_GITEA_TOKEN_FILE acts as the tools' account and needs no admin login.
|
||||
func FromEnv() (Client, []string) {
|
||||
if file := os.Getenv("MESH_GITEA_TOKEN_FILE"); file != "" {
|
||||
c := Client{URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"), TokenFile: file,
|
||||
Account: os.Getenv("MESH_GITEA_TOOLS_USER")}
|
||||
if c.Account == "" {
|
||||
c.Account = ToolsAccount
|
||||
}
|
||||
if c.URL == "" {
|
||||
return c, []string{"MESH_GITEA_URL"}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
c := Client{
|
||||
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
|
||||
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
|
||||
@@ -80,14 +109,45 @@ func (c *Client) password() (string, error) {
|
||||
return strings.TrimSpace(string(raw)), nil
|
||||
}
|
||||
|
||||
// token is the tools' account's token, refused unless its scopes line names that account.
|
||||
func (c *Client) token() (string, error) {
|
||||
raw, err := os.ReadFile(c.TokenFile)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("the tools' token is not kept yet (the module mints it for %s at its first call): %w", c.Account, err)
|
||||
}
|
||||
line, err := os.ReadFile(c.TokenFile + ".scopes")
|
||||
if err != nil || !strings.HasSuffix(strings.TrimSpace(string(line)), " for "+c.Account) {
|
||||
return "", fmt.Errorf("the token kept at %s is not %s's (novox/hq ADR 0365): the module mints that account's own at its next call; nothing was sent", c.TokenFile, c.Account)
|
||||
}
|
||||
token := strings.TrimSpace(string(raw))
|
||||
if token == "" {
|
||||
return "", fmt.Errorf("the token kept at %s is empty; nothing was sent", c.TokenFile)
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
|
||||
// authorize puts the bundle's credential on a request: the tools' token, or the admin login.
|
||||
func (c *Client) authorize(req *http.Request) error {
|
||||
if c.TokenFile != "" {
|
||||
token, err := c.token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "token "+token)
|
||||
return nil
|
||||
}
|
||||
password, err := c.password()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.SetBasicAuth(c.User, password)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Do sends one request, with payload as its JSON body when not nil, and answers the body, read up to limit;
|
||||
// 404 is ErrNotFound. One byte past the limit is read, so a body the limit cut is told from one that fits it
|
||||
// exactly: a cut body is refused, never read as if it were whole.
|
||||
func (c *Client) Do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
|
||||
password, err := c.password()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var body io.Reader
|
||||
if payload != nil {
|
||||
raw, err := json.Marshal(payload)
|
||||
@@ -100,7 +160,9 @@ func (c *Client) Do(ctx context.Context, method, path string, payload any, limit
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.SetBasicAuth(c.User, password)
|
||||
if err := c.authorize(req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", "application/json")
|
||||
if payload != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
@@ -118,6 +180,9 @@ func (c *Client) Do(ctx context.Context, method, path string, payload any, limit
|
||||
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
|
||||
}
|
||||
switch {
|
||||
case res.StatusCode == http.StatusUnauthorized && c.TokenFile != "":
|
||||
return nil, fmt.Errorf("the forge rejected the tools' token for %s (401): the module mints it afresh at its next call; nothing was done: %w",
|
||||
method+" "+path, &StatusError{method, path, res.StatusCode, "unauthorized"})
|
||||
case res.StatusCode == http.StatusNotFound:
|
||||
return nil, fmt.Errorf("%s %s: %w", method, path, ErrNotFound)
|
||||
case res.StatusCode >= 300:
|
||||
|
||||
+32
-7
@@ -11,7 +11,8 @@ import type { CommitStatus, GiteaClient } from "./client.js";
|
||||
import { mergeableUnder, newestByContext } from "./checks.js";
|
||||
import { MERGE_METHODS, Refused, checkName } from "./guard.js";
|
||||
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT } from "./pulls.js";
|
||||
import { announcementOf, askedWords, outOfDateWords, tookFile, tookWords, twelve, verdictAtTrunk } from "./trunk.js";
|
||||
import { groupRefusal, memberKey, namesSiblings, readSiblings, siblingOutOfDate, siblingsFile } from "./siblings.js";
|
||||
import { announcementOf, askedWords, outOfDateWords, sameCommit, tookFile, tookWords, twelve, verdictAtTrunk } from "./trunk.js";
|
||||
|
||||
/** A head named for a merge: at least 12 hex characters of its sha (novox/hq issue 381), up to a sha-256 commit's 64. */
|
||||
const REVIEWED_HEAD = /^[0-9a-f]{12,64}$/;
|
||||
@@ -54,9 +55,10 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
|
||||
const method = args.method ? String(args.method) : "merge";
|
||||
if (!MERGE_METHODS.has(method)) throw new Refused(`${method} is not a merge method: ${[...MERGE_METHODS].join(", ")}`);
|
||||
const deleteBranch = args.delete_branch === undefined ? true : Boolean(args.delete_branch);
|
||||
// **Only a head whose required statuses all succeeded** (novox/hq ADR 0266): read here rather than left
|
||||
// to the forge, whose administrator — the account this tool's token is minted for — may merge past a
|
||||
// rule that lets it. A rule that cannot be read is no rule passed.
|
||||
// **Only a head whose required statuses all succeeded** (novox/hq ADR 0266): read here as well as by the
|
||||
// forge, which holds the tools' own account to it (novox/hq ADR 0365 §2). The protection is the one the forge
|
||||
// applies to the base, read from the branch: the rules' own list is a repository administrator's. A rule that
|
||||
// cannot be read is no rule passed.
|
||||
// **Only the head that was reviewed** (novox/hq issue 381): mesh-controller#197 was reviewed at 065cfa0d,
|
||||
// force-pushed to 3ced96f, checked green there, and merged at 3ced96f, because this tool merged whatever the
|
||||
// head was and nothing told it which head the review had approved. The caller names it; a head that moved
|
||||
@@ -80,7 +82,7 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
|
||||
throw new Refused(`${owner}/${repo}#${number} is not merged: its head is ${pull.head_sha.slice(0, 12)}, not the ` +
|
||||
`${named} you named. If you reviewed an earlier head, review the one it has now, then name it. Nothing was merged`);
|
||||
}
|
||||
const rule = await gitea.branchProtection(owner, repo, pull.base);
|
||||
const rule = await gitea.appliedProtection(owner, repo, pull.base);
|
||||
// Only into a branch whose protection requires the checks: a pull request whose base was moved to a
|
||||
// branch nothing guards would merge past every check (novox/hq ADR 0266).
|
||||
if (!rule || !rule.enable_status_check || !(rule.status_check_contexts ?? []).filter(Boolean).length) {
|
||||
@@ -96,7 +98,11 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
|
||||
const checks = await gitea.headChecks(owner, repo, pull.head_sha);
|
||||
const at = verdictAtTrunk(checks, trunkHead);
|
||||
if (!at.current) {
|
||||
const judged = at.judged ? `was judged against trunk ${twelve(at.judged)}` : "names no trunk commit it was judged against";
|
||||
// A verdict put out of date by a sibling's move says so in its own words (ADR 0382 §5); it is asked again as a
|
||||
// trunk's move is, and a failed ask puts those words back, not the trunk's.
|
||||
const siblingWords = siblingOutOfDate(checks[CHECK_CONTEXT]?.description) ? checks[CHECK_CONTEXT]?.description : undefined;
|
||||
const judged = siblingWords ? `was put out of date by a sibling's move (${siblingWords})`
|
||||
: at.judged ? `was judged against trunk ${twelve(at.judged)}` : "names no trunk commit it was judged against";
|
||||
const took = tookWords(tookFile(deps.stateDir), `${owner}/${repo}`);
|
||||
const head = twelve(pull.head_sha), now = twelve(trunkHead);
|
||||
if (at.onItsWay) {
|
||||
@@ -122,7 +128,7 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
|
||||
const cloneURL = (await gitea.getRepo(owner, repo)).clone_url;
|
||||
await deps.emit("pull.updated", await announcementOf(gitea, owner, repo, cloneURL, pull));
|
||||
} catch (err) {
|
||||
await say(outOfDateWords(at.judged, trunkHead)).catch(() => {});
|
||||
await say(siblingWords ?? outOfDateWords(at.judged, trunkHead)).catch(() => {});
|
||||
throw new Error(`${owner}/${repo}#${number} is not merged: its verdict ${judged}, and ${pull.base} is now ` +
|
||||
`${now}, and its check could not be asked: ${err instanceof Error ? err.message : err}. Nothing was merged`);
|
||||
}
|
||||
@@ -138,6 +144,25 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
|
||||
throw new Refused(`${owner}/${repo}#${number} is not merged: ${verdict.says}. A pull request merges once ` +
|
||||
`its checks succeed; no tool merges past them (novox/hq ADR 0266)`);
|
||||
}
|
||||
// **A member judged beside its siblings merges only with its group passed and reviewed** (novox/hq ADR 0382 §6): the
|
||||
// siblings its verdict names are the holder's record (siblings.ts); a status that says it read siblings with no record
|
||||
// of them is no verdict to merge on, never read as none.
|
||||
const heard = readSiblings(siblingsFile(deps.stateDir))[memberKey(owner, repo, number)];
|
||||
const siblings = heard && sameCommit(heard.head, pull.head_sha) ? heard.siblings : [];
|
||||
if (!siblings.length && namesSiblings(statuses.find((s) => s.context === CHECK_CONTEXT)?.description)) {
|
||||
throw new Refused(`${owner}/${repo}#${number} is not merged: its verdict read siblings, and which ones is not held ` +
|
||||
`here, so the group cannot be weighed. Ask its check again at ${twelve(pull.head_sha)}. Nothing was merged`);
|
||||
}
|
||||
if (siblings.length) {
|
||||
const missing = await groupRefusal({ headChecks: (o, r, sha) => gitea.headChecks(o, r, sha),
|
||||
approvedAt: (o, r, n, h) => gitea.approvedAt(o, r, n, h) },
|
||||
{ owner, repo, number, head: String(pull.head_sha) }, siblings);
|
||||
if (missing) {
|
||||
throw new Refused(`${owner}/${repo}#${number} is not merged: its verdict was judged beside ${siblings.length} ` +
|
||||
`sibling(s), and ${missing}. A member judged beside its siblings merges once its group passed at the heads that ` +
|
||||
`verdict read and each of them is approved there (novox/hq ADR 0382). Nothing was merged`);
|
||||
}
|
||||
}
|
||||
await gitea.mergePullRequest(owner, repo, number, method, deleteBranch, pull.head_sha);
|
||||
// The merge commit only exists now; answered so the caller can follow what is built from it.
|
||||
// pull.merged is the events entrypoint's to announce (index.ts), once, within its poll.
|
||||
|
||||
+185
-8
@@ -1,6 +1,7 @@
|
||||
{
|
||||
"module": "gitea",
|
||||
"version": "1",
|
||||
"moved": true,
|
||||
"requires": [
|
||||
"postgres-database",
|
||||
"route",
|
||||
@@ -42,7 +43,47 @@
|
||||
"issue.opened",
|
||||
"pull.merged",
|
||||
"pull.updated",
|
||||
"pull.closed"
|
||||
"pull.closed",
|
||||
"forge.push",
|
||||
"forge.create",
|
||||
"forge.delete",
|
||||
"forge.fork",
|
||||
"forge.status",
|
||||
"forge.pull-request.opened",
|
||||
"forge.pull-request.reopened",
|
||||
"forge.pull-request.synchronized",
|
||||
"forge.pull-request.edited",
|
||||
"forge.pull-request.closed",
|
||||
"forge.pull-request.merged",
|
||||
"forge.pull-request.labeled",
|
||||
"forge.pull-request.unlabeled",
|
||||
"forge.pull-request.assigned",
|
||||
"forge.pull-request.unassigned",
|
||||
"forge.pull-request.reviewed",
|
||||
"forge.pull-request.review-requested",
|
||||
"forge.pull-request.commented",
|
||||
"forge.pull-request.changed",
|
||||
"forge.issue-comment.created",
|
||||
"forge.issue-comment.edited",
|
||||
"forge.issue-comment.deleted",
|
||||
"forge.issues.opened",
|
||||
"forge.issues.reopened",
|
||||
"forge.issues.closed",
|
||||
"forge.issues.edited",
|
||||
"forge.issues.labeled",
|
||||
"forge.issues.unlabeled",
|
||||
"forge.issues.assigned",
|
||||
"forge.issues.unassigned",
|
||||
"forge.issues.changed",
|
||||
"forge.release.published",
|
||||
"forge.release.updated",
|
||||
"forge.release.deleted",
|
||||
"forge.release.changed",
|
||||
"forge.repository.created",
|
||||
"forge.repository.deleted",
|
||||
"forge.repository.changed",
|
||||
"forge.wiki.changed",
|
||||
"forge.other"
|
||||
],
|
||||
"consumes": [
|
||||
"mesh-controller.checked",
|
||||
@@ -61,7 +102,14 @@
|
||||
"port": 22,
|
||||
"protocol": "tcp",
|
||||
"from": "mesh",
|
||||
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention \u2014 not 22, which the machine's own daemon holds and a module does not take"
|
||||
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention — not 22, which the machine's own daemon holds and a module does not take"
|
||||
},
|
||||
{
|
||||
"name": "forge-events",
|
||||
"port": 3047,
|
||||
"protocol": "tcp",
|
||||
"from": "machine",
|
||||
"why": "the forge's system webhook, every event signed with a secret the module keeps, from the forge's container on the docker bridge to the module's receiver at the bridge's gateway 172.17.0.1 (novox/hq issue 729)"
|
||||
}
|
||||
],
|
||||
"serves": {
|
||||
@@ -109,6 +157,26 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"config": {
|
||||
"url": {
|
||||
"type": "string",
|
||||
"description": "The address the module's tools reach the forge at, as a URL; unset, the code falls back to its environment, else this machine's gitea on its port.",
|
||||
"trusted": true,
|
||||
"default": ""
|
||||
},
|
||||
"agent-read-teams": {
|
||||
"type": "object",
|
||||
"description": "The read teams of the agents' forge accounts (ADR 0365 clause 3): each team by name, {\"<team>\": {\"repos\": [\"<owner>/<repo>\", ...], \"members\": [\"<node>\", ...]}}. The forge gets a team agent-read-<team> that reads only, with exactly those repositories and the accounts agent-<node>; a team taken out of the value is emptied. Nothing else on the forge is touched; an agent account is in no team until it is listed here.",
|
||||
"trusted": true,
|
||||
"default": {}
|
||||
},
|
||||
"ready-authors": {
|
||||
"type": "string",
|
||||
"description": "the forge accounts whose comment \"ready at <full head sha>\" marks a pull request ready for mesh-delivery to merge, separated by spaces: the account the forge's tools act as (novox/hq ADR 0365), the admin account for marks made before the tools acted as their own, and the operator's (novox/hq issue 722). A comment by any other account is ignored and said in the forge's log. Trusted: who may have a change merged is the operator's word",
|
||||
"trusted": true,
|
||||
"default": "mesh-tools mesh-admin jschoubben"
|
||||
}
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"id": "mesh-state",
|
||||
@@ -122,6 +190,21 @@
|
||||
"path": "${dir:mesh-state}/state",
|
||||
"mode": "0700"
|
||||
},
|
||||
{
|
||||
"id": "ready-authors",
|
||||
"type": "file",
|
||||
"path": "${dir:mesh-state}/ready-authors",
|
||||
"mode": "0600",
|
||||
"trusted": true,
|
||||
"template": "files/ready-authors"
|
||||
},
|
||||
{
|
||||
"id": "forge-patches",
|
||||
"type": "directory",
|
||||
"path": "/var/lib/mesh/forge-patches",
|
||||
"mode": "0733",
|
||||
"owner": "${machine:account}"
|
||||
},
|
||||
{
|
||||
"id": "state",
|
||||
"type": "directory",
|
||||
@@ -138,7 +221,7 @@
|
||||
"type": "file",
|
||||
"path": "${dir:state}/server.env",
|
||||
"mode": "0600",
|
||||
"content": "GITEA__security__INTERNAL_TOKEN=${secret:internal-token}\nGITEA__database__DB_TYPE=postgres\nGITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nGITEA__database__NAME=${bound:postgres-database:as}\nGITEA__database__USER=${bound:postgres-database:as}\nGITEA__database__PASSWD=${secret:postgres-database}\n"
|
||||
"template": "files/server.env"
|
||||
},
|
||||
{
|
||||
"id": "data",
|
||||
@@ -149,6 +232,13 @@
|
||||
{
|
||||
"id": "server",
|
||||
"type": "container",
|
||||
"health": {
|
||||
"kind": "http",
|
||||
"endpoint": "web",
|
||||
"path": "/api/healthz",
|
||||
"status": 200,
|
||||
"needs": "postgres-database"
|
||||
},
|
||||
"name": "gitea",
|
||||
"image": "gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef",
|
||||
"env": {
|
||||
@@ -194,13 +284,53 @@
|
||||
],
|
||||
"secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it"
|
||||
},
|
||||
{
|
||||
"id": "origins",
|
||||
"type": "file",
|
||||
"path": "${dir:mesh-state}/origins",
|
||||
"mode": "0644",
|
||||
"template": "files/origins"
|
||||
},
|
||||
{
|
||||
"id": "runtime-config",
|
||||
"type": "file",
|
||||
"path": "${dir:mesh-state}/config.json",
|
||||
"mode": "0600",
|
||||
"content": "{}\n",
|
||||
"merge": "json"
|
||||
"template": "files/config.json",
|
||||
"merge": "json",
|
||||
"takes-config": [
|
||||
"url"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "agent-teams-file",
|
||||
"type": "file",
|
||||
"path": "${dir:mesh-state}/agent-teams.json",
|
||||
"mode": "0600",
|
||||
"template": "files/agent-teams.json",
|
||||
"merge": "json",
|
||||
"takes-config": [
|
||||
"agent-read-teams"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "agent-teams",
|
||||
"type": "process",
|
||||
"name": "gitea-agent-teams",
|
||||
"artifact": "agent-teams",
|
||||
"run": [
|
||||
"./agent-teams"
|
||||
],
|
||||
"run-once": true,
|
||||
"env": {
|
||||
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
|
||||
"MESH_GITEA_ADMIN_USER": "mesh-admin",
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
|
||||
"MESH_AGENT_TEAMS_FILE": "${dir:mesh-state}/agent-teams.json"
|
||||
},
|
||||
"restart-on": [
|
||||
"agent-teams-file"
|
||||
]
|
||||
}
|
||||
],
|
||||
"provides": [
|
||||
@@ -215,6 +345,14 @@
|
||||
{
|
||||
"name": "git",
|
||||
"scope": "mesh"
|
||||
},
|
||||
{
|
||||
"name": "gitea-read",
|
||||
"scope": "mesh",
|
||||
"credential": {
|
||||
"issued-by": "software",
|
||||
"tool": "gitea_agent_token"
|
||||
}
|
||||
}
|
||||
],
|
||||
"build": {
|
||||
@@ -238,8 +376,13 @@
|
||||
"MESH_GITEA_CONFIG_FILE": "${dir:mesh-state}/config.json",
|
||||
"MESH_GITEA_ADMIN_USER": "mesh-admin",
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
|
||||
"MESH_GITEA_TOOLS_USER": "mesh-tools",
|
||||
"MESH_GITEA_STATE_DIR": "${dir:runtime-state}",
|
||||
"MESH_RECEIVES": "${dir:grants}/npm.json"
|
||||
"MESH_RECEIVES": "${dir:grants}/npm.json",
|
||||
"MESH_GITEA_READY_AUTHORS_FILE": "${dir:mesh-state}/ready-authors",
|
||||
"MESH_GITEA_HOOK_URL": "http://172.17.0.1:3047",
|
||||
"MESH_GITEA_HOOK_PORT": "3047",
|
||||
"MESH_GITEA_HOOK_HOST": "172.17.0.1"
|
||||
}
|
||||
},
|
||||
{
|
||||
@@ -271,9 +414,43 @@
|
||||
],
|
||||
"env": {
|
||||
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
|
||||
"MESH_GITEA_ADMIN_USER": "mesh-admin",
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret"
|
||||
"MESH_GITEA_TOOLS_USER": "mesh-tools",
|
||||
"MESH_GITEA_TOKEN_FILE": "${dir:runtime-state}/token",
|
||||
"MESH_GITEA_PATCH_DIR": "${dir:forge-patches}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "files",
|
||||
"kind": "archive",
|
||||
"from": "files"
|
||||
},
|
||||
{
|
||||
"name": "agent-token",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/agent-token",
|
||||
"binary": "agent-token",
|
||||
"loads": [
|
||||
"agent-token"
|
||||
],
|
||||
"env": {
|
||||
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
|
||||
"MESH_GITEA_ADMIN_USER": "mesh-admin",
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
|
||||
"MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "agent-teams",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/agent-teams",
|
||||
"binary": "agent-teams",
|
||||
"loads": [
|
||||
"agent-teams"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
+42
-4
@@ -9,8 +9,9 @@
|
||||
//
|
||||
// Pure functions here, so they are tested without a forge; index.ts does the asking and the setting.
|
||||
|
||||
import type { CommitStatus, GiteaPull, GiteaRepo } from "./client.js";
|
||||
import type { CommitStatus, GiteaClient, GiteaPull, GiteaRepo } from "./client.js";
|
||||
import { movedSince } from "./client.js";
|
||||
import { UNKNOWN_MARK, besideWords, siblingsOf } from "./siblings.js";
|
||||
|
||||
/** The status context a merge check's gate is kept under: one per commit, the newest replacing the last. */
|
||||
export const CHECK_CONTEXT = "mesh/merge-gate";
|
||||
@@ -65,6 +66,13 @@ export interface Checked {
|
||||
trunk?: string;
|
||||
beside?: Record<string, string>;
|
||||
took?: string;
|
||||
/** The other members of its delivery group the check read beside the head, each at its head merged onto its base
|
||||
* commit (novox/hq ADR 0382 §4); absent when it read none, and from a controller from before the rule. */
|
||||
siblings?: { repository: string; number: number; head: string; base: string; base_commit: string }[];
|
||||
/** Why the siblings could not be learned (mesh-delivery did not answer), when that is so. */
|
||||
siblings_unknown?: string;
|
||||
/** Where the judge came from, when it came from the group (ADR 0382 §3): said in the account. */
|
||||
judge_from?: string;
|
||||
}
|
||||
|
||||
/** A change plan, as the controller says it (mesh-controller internal/link, ChangePlan). */
|
||||
@@ -163,7 +171,7 @@ function described(verdict: string, summary: string, modules?: string[], depende
|
||||
return clipped(d);
|
||||
}
|
||||
|
||||
function clipped(d: string): string {
|
||||
export function clipped(d: string): string {
|
||||
d = d.replace(/\s+/g, " ").trim();
|
||||
return d.length > 140 ? d.slice(0, 139) + "…" : d;
|
||||
}
|
||||
@@ -176,6 +184,24 @@ export interface RepoCheckFacts {
|
||||
required?: boolean;
|
||||
}
|
||||
|
||||
// **What only the forge knows of a repository check said as a warning** (novox/hq issue 293): whether the
|
||||
// head holds a merge-check.sh at all, and — when it does not — whether the base branch's protection
|
||||
// requires mesh/repo-check. Asked only for a warning; unknown is left undefined, which statusesFor reads
|
||||
// as possibly required: a failure on a status nothing requires blocks nothing, a success on one that is
|
||||
// required would let an untested repository merge. The base's protection is the one the forge applies, read
|
||||
// from the branch: the rules' own list is a repository administrator's, which the tools' account is not
|
||||
// (novox/hq ADR 0365).
|
||||
export async function repoCheckFacts(
|
||||
forge: Pick<GiteaClient, "holdsFile" | "appliedProtection">, c: Checked, base: string | undefined): Promise<RepoCheckFacts> {
|
||||
if (c["repo-check"]?.verdict !== "warning") return {};
|
||||
const defined = await forge.holdsFile(c.owner, c.repo, c.commit, "merge-check.sh").catch(() => undefined);
|
||||
if (defined !== false) return { defined };
|
||||
const rule = await forge.appliedProtection(c.owner, c.repo, base || "main").catch(() => undefined);
|
||||
if (rule === undefined) return { defined };
|
||||
const required = !!rule?.enable_status_check && (rule.status_check_contexts ?? []).includes(REPO_CHECK_CONTEXT);
|
||||
return { defined, required };
|
||||
}
|
||||
|
||||
/** The description of a required repository check that is not defined. */
|
||||
export const UNDEFINED_REQUIRED = "fail: mesh/repo-check is required here and this head defines no merge-check.sh — " +
|
||||
"a person adds one, or lifts the requirement from the branch's protection";
|
||||
@@ -220,8 +246,12 @@ export function statusesFor(c: Checked, facts: RepoCheckFacts = {}): CommitStatu
|
||||
* 7a8b9c0d1e2f — pass: …", so the forge's 140 characters never clip it away; the trunk's move is weighed by it. A
|
||||
* verdict that names no trunk commit is said as before. */
|
||||
function judgedFirst(c: Checked, status: CommitStatus): CommitStatus {
|
||||
if (!c.trunk) return status;
|
||||
return { ...status, description: clipped(`head ${c.commit.slice(0, 12)} merged into trunk ${c.trunk.slice(0, 12)} — ${status.description}`) };
|
||||
// The siblings it read, named after the commits (ADR 0382 §4), so the clip never takes them either.
|
||||
const siblings = siblingsOf(c.siblings);
|
||||
// A verdict that could not learn them says so on the status too (clause 1: it is still checked, and merges).
|
||||
const mark = siblings.length ? besideWords(siblings) : c.siblings_unknown ? UNKNOWN_MARK : "";
|
||||
if (!c.trunk) return mark ? { ...status, description: clipped(`${mark} — ${status.description}`) } : status;
|
||||
return { ...status, description: clipped(`head ${c.commit.slice(0, 12)} merged into trunk ${c.trunk.slice(0, 12)}${mark ? " " + mark : ""} — ${status.description}`) };
|
||||
}
|
||||
|
||||
/** The comment a verdict leaves on its pull request, with the check's own account: the change plan of a
|
||||
@@ -240,6 +270,14 @@ export function commentFor(c: Checked, facts: RepoCheckFacts = {}): string | nul
|
||||
lines.splice(1, 0, `Judged: head \`${c.commit.slice(0, 12)}\` merged into trunk \`${c.trunk.slice(0, 12)}\`` +
|
||||
(beside ? `; read beside it: ${beside}` : ""));
|
||||
}
|
||||
const siblings = siblingsOf(c.siblings);
|
||||
if (siblings.length) {
|
||||
lines.splice(c.trunk ? 2 : 1, 0, `Siblings read beside it: ` + siblings.map((s) =>
|
||||
`${s.repository}#${s.number} \`${s.head.slice(0, 12)}\` merged onto \`${s.base_commit.slice(0, 12)}\` (${s.base})`).join(", ") +
|
||||
(c.judge_from ? `; judged by ${c.judge_from}` : ""));
|
||||
} else if (c.siblings_unknown) {
|
||||
lines.splice(c.trunk ? 2 : 1, 0, `Siblings could not be learned: ${c.siblings_unknown}`);
|
||||
}
|
||||
lines.push(`- \`${CHECK_CONTEXT}\`: **${saidAs(gate.verdict).toUpperCase()}** — ${gate.summary}` +
|
||||
(gate.modules?.length ? ` (modules: ${gate.modules.join(", ")}` +
|
||||
(gate.dependents?.length ? `; built after them: ${gate.dependents.join(", ")}` : "") + ")" : ""));
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
// **A delivery group's member is checked beside the other members' heads, and a sibling's move puts that verdict out of
|
||||
// date** (novox/hq ADR 0382 §5, §6; issue 867).
|
||||
//
|
||||
// The controller's verdict (`checked`) names the siblings its check read: each as the sibling's repository, pull
|
||||
// request, head, base branch and the base commit the sibling's head was merged onto. This module, the forge's holder,
|
||||
// keeps them per open pull request, beside its other records, and:
|
||||
//
|
||||
// - on its own hearing of a sibling's new head, of its close unmerged, or of a move of its base, sets
|
||||
// mesh/merge-gate and mesh/repo-check of every member whose verdict names that sibling back to pending, with the
|
||||
// reason. Nothing is run for it. The one move that does not count is the sibling's own merge of the head that was
|
||||
// read onto the base commit that was read (in either order of the two merges);
|
||||
// - lets the merge tool refuse a verdict that read siblings unless mesh/delivery-group passed at the very heads it
|
||||
// read, and every sibling head it read carries an approving review at that head (merge.ts).
|
||||
//
|
||||
// A verdict without siblings (a controller from before ADR 0382) is none of this: nothing is kept and nothing changes.
|
||||
// Pure functions and one small file of the module's own state, so this is tested without a forge.
|
||||
|
||||
import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
||||
import { dirname } from "node:path";
|
||||
import type { CommitStatus, GiteaPull, HeadCheck } from "./client.js";
|
||||
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT, clipped } from "./pulls.js";
|
||||
import { sameCommit, twelve, type OutOfDateWriter } from "./trunk.js";
|
||||
|
||||
/** The status the delivery group's owner sets on each member's head (novox/hq ADR 0239). */
|
||||
export const GROUP_CONTEXT = "mesh/delivery-group";
|
||||
|
||||
/** One sibling a verdict read, as the controller names it (contract of ADR 0382 §4). */
|
||||
export interface SiblingNamed {
|
||||
repository: string;
|
||||
number: number;
|
||||
head: string;
|
||||
base: string;
|
||||
base_commit: string;
|
||||
/** Set here, never by the controller: the sibling merged its head onto its base commit read, so it is no sibling's
|
||||
* move any more. */
|
||||
merged?: boolean;
|
||||
}
|
||||
|
||||
/** What is kept of one member's latest verdict that read siblings: the head it was of and the siblings it named. */
|
||||
export interface SiblingEntry {
|
||||
head: string;
|
||||
siblings: SiblingNamed[];
|
||||
}
|
||||
|
||||
/** The records, by `owner/repo#number` of the member. */
|
||||
export type SiblingRecord = Record<string, SiblingEntry>;
|
||||
|
||||
/** The member's key. */
|
||||
export function memberKey(owner: string, repo: string, number: number): string {
|
||||
return `${owner}/${repo}#${number}`.toLowerCase();
|
||||
}
|
||||
|
||||
/** The siblings a verdict names, trimmed to what is kept; none when it names none or names one badly. */
|
||||
export function siblingsOf(named: unknown): SiblingNamed[] {
|
||||
if (!Array.isArray(named)) return [];
|
||||
const out: SiblingNamed[] = [];
|
||||
for (const s of named) {
|
||||
const r = s as Record<string, unknown>;
|
||||
const repository = String(r?.repository ?? ""), head = String(r?.head ?? ""), base = String(r?.base ?? "");
|
||||
const number = Number(r?.number);
|
||||
if (!repository || !head || !base || !Number.isInteger(number) || number <= 0) continue;
|
||||
out.push({ repository, number, head, base, base_commit: String(r?.base_commit ?? "") });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The siblings as a status names them: "mesh-host#114@abcdef123456, …". */
|
||||
export function siblingWords(siblings: SiblingNamed[]): string {
|
||||
return siblings.map((s) => `${s.repository.split("/").pop()}#${s.number}@${twelve(s.head)}`).join(", ");
|
||||
}
|
||||
|
||||
/** What a status of a verdict that read siblings carries, after the commits it judged. */
|
||||
export function besideWords(siblings: SiblingNamed[]): string {
|
||||
// Short, so that the verdict word and the start of its reason survive the forge's 140 characters: the first sibling
|
||||
// by its pull request and 8 characters of its head, and how many more. The whole list is the holder's record and the comment's.
|
||||
const first = siblings[0];
|
||||
const more = siblings.length > 1 ? ` +${siblings.length - 1} more` : "";
|
||||
return `beside ${first.repository.split("/").pop()}#${first.number}@${twelve(first.head).slice(0, 8)}${more}`;
|
||||
}
|
||||
|
||||
/** The mark a status carries when the siblings could not be learned (ADR 0382 §1): said, never silent. */
|
||||
export const UNKNOWN_MARK = "siblings unknown";
|
||||
|
||||
/** Whether a status names that its verdict read siblings (so that a record lost does not read as none). */
|
||||
export function namesSiblings(description: string | undefined): boolean {
|
||||
return / beside [\w.-]+#\d+@[0-9a-f]{8}\b/.test(description ?? "");
|
||||
}
|
||||
|
||||
/** Whether a status is a verdict put out of date by a sibling's move. */
|
||||
export function siblingOutOfDate(description: string | undefined): boolean {
|
||||
return /^out of date: judged beside /.test(description ?? "");
|
||||
}
|
||||
|
||||
// --- the record on disk ----------------------------------------------------------------------------------------
|
||||
|
||||
/** The file the holder keeps the siblings its verdicts name in, beside its other records. */
|
||||
export function siblingsFile(stateDir: string | undefined): string | null {
|
||||
return stateDir ? `${stateDir.replace(/\/+$/, "")}/siblings-named.json` : null;
|
||||
}
|
||||
|
||||
export function readSiblings(file: string | null): SiblingRecord {
|
||||
if (!file || !existsSync(file)) return {};
|
||||
try {
|
||||
return (JSON.parse(readFileSync(file, "utf8")) as { members?: SiblingRecord }).members ?? {};
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
export function keepSiblings(file: string | null, record: SiblingRecord): void {
|
||||
if (!file) return;
|
||||
try {
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
writeFileSync(file + ".tmp", JSON.stringify({ members: record }));
|
||||
renameSync(file + ".tmp", file);
|
||||
} catch (err) {
|
||||
console.error(`[gitea] the siblings a verdict named could not be kept — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** A member's latest verdict, kept: its siblings when it read some, else the entry removed. Answers whether it changed. */
|
||||
export function noteVerdict(record: SiblingRecord, key: string, head: string, siblings: SiblingNamed[]): boolean {
|
||||
if (siblings.length === 0) {
|
||||
if (!(key in record)) return false;
|
||||
delete record[key];
|
||||
return true;
|
||||
}
|
||||
record[key] = { head, siblings };
|
||||
return true;
|
||||
}
|
||||
|
||||
// --- what a sibling's move is ---------------------------------------------------------------------------------
|
||||
|
||||
/** What weighing a sibling reads of the forge. */
|
||||
export interface SiblingForge {
|
||||
getPullRequest(owner: string, repo: string, index: number): Promise<GiteaPull>;
|
||||
branchHead(owner: string, repo: string, branch: string): Promise<string>;
|
||||
commitParents(owner: string, repo: string, sha: string): Promise<string[]>;
|
||||
}
|
||||
|
||||
/** The reason a verdict that named this sibling is out of date, from what the forge holds now, or nothing; and whether
|
||||
* the sibling has merged its head read onto the base commit read, which ends it as a sibling. */
|
||||
export async function siblingMoved(forge: SiblingForge, s: SiblingNamed): Promise<{ reason?: string; merged?: boolean }> {
|
||||
const [owner, repo] = s.repository.split("/");
|
||||
const who = `${repo}#${s.number}`;
|
||||
const pull = await forge.getPullRequest(owner, repo, s.number);
|
||||
if (pull.merged) {
|
||||
// Its own merge of the head that was read onto the base commit that was read: the head the merge joined is the head
|
||||
// read, and the commit it was made onto is the base commit read. Any other merge is a move of the base.
|
||||
const parents = pull.merge_commit_sha ? await forge.commitParents(owner, repo, pull.merge_commit_sha).catch(() => []) : [];
|
||||
const own = sameCommit(pull.head_sha, s.head) && parents.length >= 1 && sameCommit(parents[0], s.base_commit) &&
|
||||
(parents.length === 1 || sameCommit(parents[1], s.head));
|
||||
if (own) return { merged: true };
|
||||
return { reason: `${who} was merged otherwise than its head ${twelve(s.head)} onto ${twelve(s.base_commit)}` };
|
||||
}
|
||||
if (pull.state !== "open") return { reason: `${who} was closed unmerged` };
|
||||
if (!sameCommit(pull.head_sha, s.head)) {
|
||||
return { reason: `${who} at ${twelve(s.head)}, it is now at ${twelve(pull.head_sha)}` };
|
||||
}
|
||||
const base = await forge.branchHead(owner, repo, s.base);
|
||||
if (!sameCommit(base, s.base_commit)) {
|
||||
return { reason: `${who} merged onto ${twelve(s.base_commit)}, its ${s.base} is now ${twelve(base)}` };
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
/** A verdict out of date because a sibling moved, as its statuses say it (ADR 0382 §5; the wording of ADR 0323 §4). */
|
||||
export function siblingOutOfDateWords(reason: string): string {
|
||||
return clipped(`out of date: judged beside ${reason}; judged again on a recheck or when it is merged`);
|
||||
}
|
||||
|
||||
/** The statuses a sibling's move puts back to pending on one member's head, from what the forge holds on it now: none
|
||||
* when it has no merge check, or its gate is pending (a check on its way, or already out of date). */
|
||||
export function siblingStatuses(checks: Record<string, HeadCheck>, reason: string): CommitStatus[] {
|
||||
const gate = checks[CHECK_CONTEXT];
|
||||
if (!gate || gate.state === "pending") return [];
|
||||
const description = siblingOutOfDateWords(reason);
|
||||
const out: CommitStatus[] = [{ state: "pending", context: CHECK_CONTEXT, description }];
|
||||
if (checks[REPO_CHECK_CONTEXT]) out.push({ state: "pending", context: REPO_CHECK_CONTEXT, description });
|
||||
return out;
|
||||
}
|
||||
|
||||
/** What weighing a sibling's move writes with, beside the forge's reads. */
|
||||
export interface SiblingWeighing extends SiblingForge {}
|
||||
|
||||
/**
|
||||
* A sibling moved (a new head, a close, a base move, a merge): every member whose latest verdict names it has its
|
||||
* merge check put back to pending with the reason. `repository` and, when the event names one, `number` say which
|
||||
* sibling. A member whose head is no longer the one its verdict was of is dropped from the record: its new head is
|
||||
* checked afresh. Nothing is run. Answers whether the record changed. A read that fails leaves that member for the next
|
||||
* move to weigh, and is said.
|
||||
*/
|
||||
export async function weighSiblings(forge: SiblingWeighing, gates: OutOfDateWriter, record: SiblingRecord,
|
||||
repository: string, number?: number): Promise<boolean> {
|
||||
let changed = false;
|
||||
const repo = repository.toLowerCase();
|
||||
for (const [key, entry] of Object.entries(record)) {
|
||||
const [path] = key.split("#");
|
||||
const [owner, name] = path.split("/");
|
||||
const memberNumber = Number(key.split("#")[1]);
|
||||
// A member that is merged or no longer open has no verdict to put out of date: its entry goes (the record must not
|
||||
// grow without bound, and a merged head is never re-pended by a sibling's later push). Weighed when the event is the
|
||||
// member's own, or names one of its siblings.
|
||||
const own = path === repo && (number === undefined || number === memberNumber);
|
||||
if (own || entry.siblings.some((s) => !s.merged && s.repository.toLowerCase() === repo && (number === undefined || s.number === number))) {
|
||||
try {
|
||||
const member = await forge.getPullRequest(owner, name, memberNumber);
|
||||
if (member.merged || member.state !== "open") {
|
||||
delete record[key];
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${key}: whether it is still open could not be read — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
for (const s of entry.siblings) {
|
||||
if (s.merged || s.repository.toLowerCase() !== repo || (number !== undefined && s.number !== number)) continue;
|
||||
try {
|
||||
const moved = await siblingMoved(forge, s);
|
||||
if (moved.merged) {
|
||||
s.merged = true;
|
||||
changed = true;
|
||||
continue;
|
||||
}
|
||||
if (!moved.reason) continue;
|
||||
const member = await forge.getPullRequest(owner, name, memberNumber);
|
||||
if (!sameCommit(member.head_sha, entry.head)) {
|
||||
delete record[key];
|
||||
changed = true;
|
||||
break;
|
||||
}
|
||||
const written = await gates.outOfDate(owner, name, entry.head, (checks) => siblingStatuses(checks, moved.reason!),
|
||||
member.html_url || undefined);
|
||||
if (written.length) {
|
||||
console.log(`[gitea] ${key}: its verdict was judged beside ${moved.reason}: out of date, nothing run (novox/hq ADR 0382)`);
|
||||
}
|
||||
break;
|
||||
} catch (err) {
|
||||
console.error(`[gitea] ${key}: whether ${s.repository}#${s.number} moved could not be weighed, so it is weighed again ` +
|
||||
`on its next move — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
return changed;
|
||||
}
|
||||
|
||||
// --- the merge tool's rule (ADR 0382 §6) ----------------------------------------------------------------------
|
||||
|
||||
/** What the rule reads beside the pull request being merged. */
|
||||
export interface GroupForge {
|
||||
headChecks(owner: string, repo: string, sha: string): Promise<Record<string, HeadCheck>>;
|
||||
/** Whether the sibling's head carries an approving review at that head. */
|
||||
approvedAt(owner: string, repo: string, number: number, head: string): Promise<boolean>;
|
||||
}
|
||||
|
||||
/**
|
||||
* What stops a pull request whose verdict read siblings from merging, in words, or nothing: mesh/delivery-group must be
|
||||
* a success at its own head and at every sibling head the verdict read, and every sibling head read must carry an
|
||||
* approving review at that head. Every missing thing is named.
|
||||
*/
|
||||
export async function groupRefusal(forge: GroupForge, self: { owner: string; repo: string; number: number; head: string },
|
||||
siblings: SiblingNamed[]): Promise<string | undefined> {
|
||||
const missing: string[] = [];
|
||||
const members = [{ repository: `${self.owner}/${self.repo}`, number: self.number, head: self.head, sibling: false },
|
||||
...siblings.map((s) => ({ repository: s.repository, number: s.number, head: s.head, sibling: true }))];
|
||||
for (const m of members) {
|
||||
const [owner, repo] = m.repository.split("/");
|
||||
const at = `${repo}#${m.number}@${twelve(m.head)}`;
|
||||
let group: HeadCheck | undefined;
|
||||
try {
|
||||
group = (await forge.headChecks(owner, repo, m.head))[GROUP_CONTEXT];
|
||||
} catch (err) {
|
||||
missing.push(`${GROUP_CONTEXT} at ${at} could not be read (${err instanceof Error ? err.message : err})`);
|
||||
continue;
|
||||
}
|
||||
if (group?.state !== "success") missing.push(`${GROUP_CONTEXT} is ${group?.state ?? "missing"} at ${at}`);
|
||||
if (!m.sibling) continue;
|
||||
try {
|
||||
if (!(await forge.approvedAt(owner, repo, m.number, m.head))) missing.push(`${at} carries no approving review at that head`);
|
||||
} catch (err) {
|
||||
missing.push(`whether ${at} carries an approving review could not be read (${err instanceof Error ? err.message : err})`);
|
||||
}
|
||||
}
|
||||
return missing.length ? missing.join("; ") : undefined;
|
||||
}
|
||||
@@ -47,11 +47,18 @@ test("every open pull request is read with what its base requires and what its h
|
||||
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/pulls/14")) {
|
||||
return send({ number: 14, state: "open", base: { ref: "main" }, head: { ref: "y", sha: "bbb2222" }, html_url: "s" });
|
||||
}
|
||||
if (url.startsWith("/api/v1/repos/novox/hq/branch_protections")) {
|
||||
return send([{ rule_name: "main", enable_status_check: true, status_check_contexts: ["mesh/repo-check"] }]);
|
||||
// The base's protection, read from the branch as the tools' account may (novox/hq ADR 0365); the rules' list is refused it.
|
||||
if (url.includes("/branch_protections")) {
|
||||
res.statusCode = 403;
|
||||
return res.end(JSON.stringify({ message: "the rules are a repository administrator's" }));
|
||||
}
|
||||
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/branch_protections")) {
|
||||
return send([{ rule_name: "main", enable_status_check: true, status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"] }]);
|
||||
if (url.startsWith("/api/v1/repos/novox/hq/branches/main")) {
|
||||
return send({ name: "main", protected: true, effective_branch_protection_name: "main", enable_status_check: true,
|
||||
status_check_contexts: ["mesh/repo-check"] });
|
||||
}
|
||||
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/branches/main")) {
|
||||
return send({ name: "main", protected: true, effective_branch_protection_name: "main", enable_status_check: true,
|
||||
status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"] });
|
||||
}
|
||||
if (url.includes("/commits/aaa1111/status")) return send({ statuses: [] });
|
||||
if (url.includes("/commits/bbb2222/status")) return send({ statuses: [{ context: "mesh/merge-gate", status: "pending",
|
||||
@@ -137,3 +144,122 @@ test("a merged pull request whose branch was deleted is announced with its branc
|
||||
assert.equal(headBranchOf({ ref: "refs/pull/7/head" }), "refs/pull/7/head");
|
||||
assert.equal(headBranchOf(undefined), undefined);
|
||||
});
|
||||
|
||||
// **A pull request marked ready says the head it is marked at** (novox/hq issue 722): the label mesh/ready and the
|
||||
// newest comment "ready at <full head sha>" by a listed account, written after the label was added and after the last
|
||||
// push. Read from the pull request's whole timeline, page by page; without the label nothing is read.
|
||||
const head = "c".repeat(40), older = "d".repeat(40);
|
||||
const ev = (id: number, type: string, body: string, user = "mesh-admin", label?: { id: number; name: string }) =>
|
||||
({ id, type, body, user: { login: user }, created_at: `2026-10-14T10:${String(id % 60).padStart(2, "0")}:00Z`,
|
||||
...(label ? { label } : {}) });
|
||||
const readyLabel = { id: 31, name: "mesh/ready" };
|
||||
|
||||
function forgeWith(timeline: unknown[]) {
|
||||
const asked: string[] = [];
|
||||
const server = createServer((req, res) => {
|
||||
const url = req.url ?? "";
|
||||
asked.push(url);
|
||||
const send = (v: unknown) => res.end(JSON.stringify(v));
|
||||
if (url.startsWith("/api/v1/repos/issues/search")) {
|
||||
return send(url.includes("page=1") ? [
|
||||
{ number: 7, repository: { owner: "novox", name: "hq", full_name: "novox/hq" },
|
||||
labels: [{ id: 31, name: "mesh/ready" }, { id: 2, name: "bug" }] },
|
||||
{ number: 8, repository: { owner: "novox", name: "hq", full_name: "novox/hq" }, labels: [] },
|
||||
] : []);
|
||||
}
|
||||
if (url.startsWith("/api/v1/repos/novox/hq/pulls/7")) {
|
||||
return send({ number: 7, state: "open", base: { ref: "main" }, head: { ref: "x", sha: head }, html_url: "h" });
|
||||
}
|
||||
if (url.startsWith("/api/v1/repos/novox/hq/pulls/8")) {
|
||||
return send({ number: 8, state: "open", base: { ref: "main" }, head: { ref: "y", sha: older }, html_url: "h" });
|
||||
}
|
||||
const t = /\/issues\/7\/timeline\?page=(\d+)&limit=50/.exec(url);
|
||||
if (t) {
|
||||
const page = Number(t[1]);
|
||||
return send(timeline.slice((page - 1) * 50, page * 50));
|
||||
}
|
||||
if (url.includes("/branch_protections")) return send([]);
|
||||
if (url.includes("/status")) return send({ statuses: [] });
|
||||
res.statusCode = 404;
|
||||
res.end("{}");
|
||||
});
|
||||
return { server, asked };
|
||||
}
|
||||
|
||||
async function readOpen(timeline: unknown[], authors = ["mesh-admin", "jschoubben"]) {
|
||||
const { GiteaClient } = await import("../client.ts");
|
||||
const { server, asked } = forgeWith(timeline);
|
||||
await new Promise<void>((r) => server.listen(0, r));
|
||||
try {
|
||||
const client = new GiteaClient(`http://127.0.0.1:${(server.address() as any).port}`, "t");
|
||||
return { open: await client.openPulls(authors), asked };
|
||||
} finally {
|
||||
server.close();
|
||||
}
|
||||
}
|
||||
|
||||
test("a pull request's ready mark is read: the label and the head the newest listed ready comment names", async () => {
|
||||
const { open, asked } = await readOpen([
|
||||
ev(1, "pull_push", JSON.stringify({ commit_ids: [head] })),
|
||||
ev(2, "label", "1", "mesh-admin", readyLabel),
|
||||
ev(3, "comment", `ready at ${older}`),
|
||||
ev(4, "comment", `Reviewed: approve.\nready at ${head}\n`),
|
||||
ev(5, "comment", "thanks", "jschoubben"),
|
||||
]);
|
||||
assert.deepEqual(open[0].labels, ["mesh/ready", "bug"]);
|
||||
assert.deepEqual(open[0].ready, { head, by: "mesh-admin", at: "2026-10-14T10:04:00Z", label_id: 31 });
|
||||
assert.equal(open[1].ready, undefined, "no label, no mark");
|
||||
assert.ok(!asked.some((u) => u.includes("/issues/8/")), "no timeline read without the label");
|
||||
});
|
||||
|
||||
test("a ready comment by an account not listed is ignored, and said", async () => {
|
||||
const { readyMarkOf } = await import("../client.ts");
|
||||
const said: string[] = [];
|
||||
const mark = readyMarkOf(31, [
|
||||
ev(1, "pull_push", "{}") as any, ev(2, "label", "1", "mesh-admin", readyLabel) as any,
|
||||
{ ...ev(3, "comment", `ready at ${head}`, "outsider"), user: "outsider" },
|
||||
].map((e: any) => ({ ...e, user: typeof e.user === "string" ? e.user : e.user.login })), ["mesh-admin"], (l) => said.push(l));
|
||||
assert.equal(mark.head, "", "an outsider's mark counts");
|
||||
assert.equal(said.length, 1);
|
||||
assert.match(said[0], /by outsider is ignored/);
|
||||
});
|
||||
|
||||
test("a ready comment older than the latest push, or than the label, never counts", async () => {
|
||||
const pushedAfter = await readOpen([
|
||||
ev(1, "label", "1", "mesh-admin", readyLabel),
|
||||
ev(2, "comment", `ready at ${head}`),
|
||||
ev(3, "pull_push", JSON.stringify({ commit_ids: [head] })),
|
||||
]);
|
||||
assert.equal(pushedAfter.open[0].ready?.head, "", "a mark written before the latest push counts, the label still on");
|
||||
const beforeLabel = await readOpen([
|
||||
ev(1, "pull_push", "{}"),
|
||||
ev(2, "comment", `ready at ${head}`),
|
||||
ev(3, "label", "1", "mesh-admin", readyLabel),
|
||||
]);
|
||||
assert.equal(beforeLabel.open[0].ready?.head, "", "a mark written before the label was added counts");
|
||||
const takenOff = await readOpen([
|
||||
ev(1, "label", "1", "mesh-admin", readyLabel),
|
||||
ev(2, "comment", `ready at ${head}`),
|
||||
ev(3, "label", "", "mesh-admin", readyLabel),
|
||||
]);
|
||||
assert.equal(takenOff.open[0].ready?.head, "", "a mark whose label was taken off since counts");
|
||||
});
|
||||
|
||||
test("a long thread hides nothing: every page of the timeline is read", async () => {
|
||||
const events: unknown[] = [ev(1, "pull_push", "{}"), ev(2, "label", "1", "mesh-admin", readyLabel),
|
||||
ev(3, "comment", `ready at ${older}`)];
|
||||
for (let id = 4; id < 130; id++) events.push(ev(id, "comment", "a long discussion", "jschoubben"));
|
||||
events.push(ev(130, "comment", `ready at ${head}`, "jschoubben"));
|
||||
const { open, asked } = await readOpen(events);
|
||||
assert.equal(open[0].ready?.head, head, "the newest mark, on the third page, is not read");
|
||||
assert.equal(asked.filter((u) => u.includes("/issues/7/timeline")).length, 3);
|
||||
});
|
||||
|
||||
test("the accounts that may mark are the setting's, and only the mesh's own when it cannot be read", async () => {
|
||||
const { readyAuthors } = await import("../client.ts");
|
||||
assert.deepEqual(readyAuthors("f", "mesh-admin", () => "# x\nauthors=mesh-admin jschoubben\n"), ["mesh-admin", "jschoubben"]);
|
||||
// The tools' account and the admin's, whose marks were made before ADR 0365 (issue 469).
|
||||
assert.deepEqual(readyAuthors("f", "mesh-admin", () => { throw new Error("gone"); }, "mesh-tools"), ["mesh-tools", "mesh-admin"]);
|
||||
assert.deepEqual(readyAuthors("f", "mesh-admin", () => "authors=\n", "mesh-tools"), ["mesh-tools", "mesh-admin"]);
|
||||
assert.deepEqual(readyAuthors(undefined, "mesh-admin", undefined, "mesh-tools"), ["mesh-tools", "mesh-admin"]);
|
||||
});
|
||||
|
||||
@@ -30,6 +30,9 @@ const state = {
|
||||
// What the tools said on the bus.
|
||||
emitted: [] as { type: string; body: any }[],
|
||||
rules: [] as any[],
|
||||
// Whether the token is a repository administrator's. The tools' account is not (novox/hq ADR 0365): the forge
|
||||
// refuses it the rules' list, as gitea's reqAdmin does, and it reads a branch's protection from the branch.
|
||||
admin: false,
|
||||
lastCommit: "2026-01-01T00:00:00Z",
|
||||
// The pull request's head; the head the merge commit joined (the same, unless a test says otherwise) — the forge's
|
||||
// record of the pull request keeps saying `head` either way, as it would in the race issue 381 is about; and the
|
||||
@@ -41,8 +44,13 @@ const state = {
|
||||
// When set, the merge commit's parents as the forge reports them, or the status it answers them with.
|
||||
parents: null as string[] | null,
|
||||
commitsFail: 0,
|
||||
// The sibling beside which a verdict was judged (novox/hq ADR 0382 §6): its group status at its head, and its reviews.
|
||||
groupStatus: "",
|
||||
siblingGroup: "",
|
||||
reviews: [] as { state: string; commit_id: string; dismissed?: boolean }[],
|
||||
};
|
||||
const REVIEWED = "abc1234def5678901234567890abcdef12345678";
|
||||
const SIBLING_HEAD = "bbbbbbbbbbbb5678901234567890abcdef12345678".slice(0, 40);
|
||||
const TRUNK = "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b";
|
||||
const MOVED = "3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d";
|
||||
// A verdict judged against a trunk commit, as both statuses say it (novox/hq ADR 0323 §3).
|
||||
@@ -59,12 +67,16 @@ beforeEach(() => {
|
||||
state.trunk = TRUNK;
|
||||
state.emitted = [];
|
||||
state.rules = [protectedMain()];
|
||||
state.admin = false;
|
||||
state.head = REVIEWED;
|
||||
state.landed = "";
|
||||
state.merged = false;
|
||||
state.mergeCommit = "feedface00000000000000000000000000000000";
|
||||
state.parents = null;
|
||||
state.commitsFail = 0;
|
||||
state.groupStatus = "";
|
||||
state.siblingGroup = "";
|
||||
state.reviews = [];
|
||||
});
|
||||
|
||||
const json = (res: ServerResponse, status: number, body: unknown): void => {
|
||||
@@ -87,8 +99,18 @@ const server = createServer(async (req, res) => {
|
||||
if (method !== "GET") state.sent.push({ method, path, body });
|
||||
const repo = "/repos/novox/mesh-catalog";
|
||||
if (method === "GET" && path === repo) return json(res, 200, { full_name: "novox/mesh-catalog", default_branch: "main" });
|
||||
if (path.startsWith(`${repo}/branch_protections`) && !state.admin) {
|
||||
return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/branch_protections`) return json(res, 200, state.rules);
|
||||
if (method === "GET" && path === `${repo}/branches/main`) return json(res, 200, { name: "main", commit: { id: state.trunk } });
|
||||
const branch = method === "GET" && path.startsWith(`${repo}/branches/`) ? decodeURIComponent(path.slice(`${repo}/branches/`.length)) : null;
|
||||
if (branch !== null) {
|
||||
// As the forge says it to any reader of the code: the rule that covers the branch, as the forge matches it.
|
||||
const rule = (state.rules as any[]).find((r) => ruleCovers(String(r.rule_name ?? r.branch_name), branch));
|
||||
return json(res, 200, { name: branch, commit: { id: branch === "main" ? state.trunk : "c0ffee" }, protected: !!rule,
|
||||
...(rule ? { effective_branch_protection_name: rule.rule_name ?? rule.branch_name, enable_status_check: !!rule.enable_status_check,
|
||||
status_check_contexts: rule.status_check_contexts ?? [], required_approvals: rule.required_approvals ?? 0, user_can_push: false } : {}) });
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/pulls/7/files`) {
|
||||
return json(res, 200, url.searchParams.get("page") === "1" ? [{ filename: "modules/gitea/x.ts", status: "modified" }] : []);
|
||||
}
|
||||
@@ -113,8 +135,17 @@ const server = createServer(async (req, res) => {
|
||||
return json(res, 200, { sha: state.mergeCommit, parents: parents.map((sha) => ({ sha })) });
|
||||
}
|
||||
if (method === "GET" && path === `${repo}/commits/${state.head}/status`) {
|
||||
return json(res, 200, { sha: state.head, state: "success", statuses: state.statuses });
|
||||
const group = state.groupStatus ? [{ context: "mesh/delivery-group", status: state.groupStatus }] : [];
|
||||
return json(res, 200, { sha: state.head, state: "success", statuses: [...state.statuses, ...group] });
|
||||
}
|
||||
if (method === "GET" && path === `/repos/novox/mesh-host/commits/${SIBLING_HEAD}/status`) {
|
||||
return json(res, 200, { sha: SIBLING_HEAD, state: "success",
|
||||
statuses: state.siblingGroup ? [{ context: "mesh/delivery-group", status: state.siblingGroup }] : [] });
|
||||
}
|
||||
if (method === "GET" && path === "/repos/novox/mesh-host/pulls/114/reviews") {
|
||||
return json(res, 200, url.searchParams.get("page") === "1" ? state.reviews : []);
|
||||
}
|
||||
if (method === "GET" && path === "/repos/novox/mesh-host/issues/114/labels") return json(res, 200, []);
|
||||
if (method === "POST" && path === `${repo}/pulls/7/merge`) {
|
||||
state.merged = true;
|
||||
return json(res, 200, null);
|
||||
@@ -199,6 +230,7 @@ test("a protection rule covers a branch as the forge decides it: classes, altern
|
||||
});
|
||||
|
||||
test("gitea_api still writes what an author writes", async () => {
|
||||
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
|
||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/issues/3/comments", method: "POST", body: { body: "hi" } });
|
||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/pulls/7/reviews", method: "POST", body: { event: "COMMENT" } });
|
||||
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/branch_protections" });
|
||||
@@ -206,6 +238,7 @@ test("gitea_api still writes what an author writes", async () => {
|
||||
});
|
||||
|
||||
test("gitea_branch_protection_set only strengthens", async () => {
|
||||
state.admin = true; // a repository administrator's token: the tools' account is refused the rules by the forge
|
||||
const set = (args: Record<string, unknown>) =>
|
||||
tool("gitea_branch_protection_set").run({ owner: "novox", repo: "mesh-catalog", branch: "main", ...args });
|
||||
await assert.rejects(set({ status_checks: "" }), /drops the required status mesh\/merge-gate, mesh\/repo-check/);
|
||||
@@ -396,6 +429,7 @@ test("gitea_merge_pull_request reads back a fast-forward, and says it cannot rea
|
||||
});
|
||||
|
||||
test("gitea_delete_repo deletes only the tools' own user's repository, unguarded, untouched for 30 days, named in full", async () => {
|
||||
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
|
||||
const del = (owner: string, name: string, confirm: unknown) => tool("gitea_delete_repo").run({ owner, name, confirm });
|
||||
await assert.rejects(del("novox", "mesh-catalog", true), /confirm must be the repository's full name/);
|
||||
await assert.rejects(del("novox", "mesh-catalog", "novox/mesh-catalog"), /not a repository of the tools' own user/);
|
||||
@@ -427,6 +461,7 @@ test("gitea_merge_pull_request merges only into a branch whose protection requir
|
||||
});
|
||||
|
||||
test("a rule named for a branch is not made weaker than the glob rule that guards it now", async () => {
|
||||
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
|
||||
state.rules.push({ rule_name: "release/*", enable_push: false, enable_status_check: true,
|
||||
status_check_contexts: ["mesh/merge-gate"], block_admin_merge_override: true });
|
||||
const set = (args: Record<string, unknown>) =>
|
||||
@@ -511,3 +546,64 @@ test("no tool's caller text becomes another path or another user: `..` and `?sud
|
||||
assert.deepEqual(state.asked, ["DELETE /repos/novox/mesh-catalog/branches/feat/x",
|
||||
"GET /repos/novox/mesh-catalog/contents/modules/gitea/module.json"]);
|
||||
});
|
||||
|
||||
|
||||
// **A member judged beside its siblings merges only with its group passed and reviewed** (novox/hq ADR 0382 §6, issue 867):
|
||||
// the verdict names the sibling; mesh/delivery-group must be a success at the member's head and the sibling's head that
|
||||
// verdict read, and the sibling's head must carry an approving review. A verdict that read none merges as before.
|
||||
test("gitea_merge_pull_request merges a verdict that read siblings only with its group passed and each sibling head reviewed", async () => {
|
||||
const { keepSiblings, siblingsFile } = await import("../dist/siblings.js");
|
||||
const merge = () => tool("gitea_merge_pull_request").run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED });
|
||||
const beside = `head ${REVIEWED.slice(0, 12)} merged into trunk ${TRUNK.slice(0, 12)} beside mesh-host#114@${SIBLING_HEAD.slice(0, 8)}`;
|
||||
const named = (siblings: unknown[]) => keepSiblings(siblingsFile(stateDir),
|
||||
{ "novox/mesh-catalog#7": { head: REVIEWED, siblings: siblings as any } });
|
||||
state.statuses = [{ context: "mesh/merge-gate", state: "success", description: `${beside} — pass: x` },
|
||||
{ context: "mesh/repo-check", state: "success", description: `${beside} — pass: y` }];
|
||||
named([{ repository: "novox/mesh-host", number: 114, head: SIBLING_HEAD, base: "main", base_commit: "c".repeat(40) }]);
|
||||
|
||||
// Nothing passed: every missing thing is named, nothing merged.
|
||||
await assert.rejects(merge(), (err: Error) => {
|
||||
assert.match(err.message, /is not merged: its verdict was judged beside 1 sibling\(s\), and mesh\/delivery-group is missing at mesh-catalog#7@abc1234def56; mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review at that head/);
|
||||
return true;
|
||||
});
|
||||
assert.equal(state.merged, false);
|
||||
|
||||
// The group passed at the member's head only: the sibling's head still lacks it.
|
||||
state.groupStatus = "success";
|
||||
await assert.rejects(merge(), /mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review/);
|
||||
// Passed at both heads, but the only approval is at another head, or dismissed: not an approval at the head read.
|
||||
state.siblingGroup = "success";
|
||||
state.reviews = [{ state: "APPROVED", commit_id: "d".repeat(40) }, { state: "APPROVED", commit_id: SIBLING_HEAD, dismissed: true },
|
||||
{ state: "COMMENT", commit_id: SIBLING_HEAD }];
|
||||
await assert.rejects(merge(), /judged beside 1 sibling\(s\), and mesh-host#114@bbbbbbbbbbbb carries no approving review at that head\. A member judged beside/);
|
||||
assert.equal(state.merged, false);
|
||||
|
||||
// Passed and approved at the very heads read: it merges.
|
||||
state.reviews.push({ state: "APPROVED", commit_id: SIBLING_HEAD });
|
||||
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
|
||||
|
||||
// A status that says it read siblings, with no record of them, is no verdict to merge on.
|
||||
state.merged = false;
|
||||
keepSiblings(siblingsFile(stateDir), {});
|
||||
await assert.rejects(merge(), /its verdict read siblings, and which ones is not held here/);
|
||||
assert.equal(state.merged, false);
|
||||
|
||||
// A verdict that read none (an older controller, or a group of one) merges exactly as before.
|
||||
state.statuses = judgedAt(TRUNK);
|
||||
state.groupStatus = "";
|
||||
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
|
||||
});
|
||||
|
||||
test("a verdict put out of date by a sibling's move is refused in its own words, and asked again once", async () => {
|
||||
const merge = () => tool("gitea_merge_pull_request").run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED });
|
||||
const words = "out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd; judged again on a recheck or when it is merged";
|
||||
state.statuses = judgedAt(TRUNK).map((s) => ({ ...s, state: "pending", description: words }));
|
||||
await assert.rejects(merge(), (err: Error) => {
|
||||
assert.match(err.message, /its verdict was put out of date by a sibling's move \(out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd/);
|
||||
assert.match(err.message, /was asked/);
|
||||
assert.match(err.message, /never merges by itself/);
|
||||
return true;
|
||||
});
|
||||
assert.equal(state.merged, false);
|
||||
assert.equal(state.emitted.length, 1, "the check is asked, once");
|
||||
});
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// The controller's build-source check asks gitea_branch_protection_get for a repository's rules, which the forge
|
||||
// answers to a repository's administrator alone. The tools' account is not one (novox/hq ADR 0365 §2), so the read
|
||||
// goes with the forge's administrator account, and only the read (ADR 0375, issue 772): a protection edit through the
|
||||
// tools still gets the forge's refusal.
|
||||
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
|
||||
import { GiteaAdmin, GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken } from "../dist/token.js";
|
||||
import { getGiteaTools } from "../dist/tools/index.js";
|
||||
|
||||
const adminBasic = "Basic " + Buffer.from("mesh-admin:secret").toString("base64");
|
||||
const rules = [
|
||||
{ rule_name: "main", enable_push: false, enable_status_check: true, status_check_contexts: ["mesh/merge-gate"], block_admin_merge_override: true },
|
||||
{ branch_name: "release", enable_push: false },
|
||||
];
|
||||
const seen: string[] = [];
|
||||
|
||||
const server = createServer((req, res) => {
|
||||
const path = (req.url ?? "").replace(/^\/api\/v1/, "");
|
||||
const as = req.headers.authorization === adminBasic ? "admin" : "tools";
|
||||
seen.push(`${req.method} ${path} as ${as}`);
|
||||
const send = (status: number, body: unknown) => { res.writeHead(status, { "Content-Type": "application/json" }); res.end(JSON.stringify(body)); };
|
||||
if (path.includes("/branch_protections")) {
|
||||
if (as !== "admin") return send(403, { message: "user should be an owner or a collaborator with admin write of a repository" });
|
||||
if (req.method === "GET") return send(200, rules);
|
||||
}
|
||||
send(404, { message: "not found" });
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
after(() => new Promise<void>((r) => server.close(() => r())));
|
||||
const { port } = server.address() as { port: number };
|
||||
const url = `http://127.0.0.1:${port}`;
|
||||
const tools = getGiteaTools(new GiteaClient(url, new ConfiguredToken("tools-token")), () => new GiteaAdmin(url, "mesh-admin", "secret"));
|
||||
const tool = (name: string) => tools.find((t) => t.name === name)!;
|
||||
|
||||
test("gitea_branch_protection_get answers every rule, read with the administrator account the tools' token cannot read with", async () => {
|
||||
const out: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller" });
|
||||
assert.deepEqual(out.rules.map((r: any) => r.rule), ["main", "release"]);
|
||||
});
|
||||
|
||||
test("gitea_branch_protection_get answers the rule for a branch, null when it has none", async () => {
|
||||
const main: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller", branch: "main" });
|
||||
assert.equal(main.branch, "main");
|
||||
assert.equal(main.rule.rule, "main");
|
||||
assert.equal(main.rule.admin_may_override, false);
|
||||
const none: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller", branch: "dev" });
|
||||
assert.equal(none.rule, null);
|
||||
});
|
||||
|
||||
test("gitea_branch_protection_set still gets the forge's refusal, as the tools' account", async () => {
|
||||
await assert.rejects(
|
||||
tool("gitea_branch_protection_set").run({ owner: "novox", repo: "mesh-controller", branch: "main", status_checks: "mesh/merge-gate" }),
|
||||
/403/,
|
||||
);
|
||||
assert.ok(!seen.some((s) => /^(PATCH|POST|PUT|DELETE) .*branch_protections.* as admin$/.test(s)), seen.join("\n"));
|
||||
});
|
||||
@@ -0,0 +1,269 @@
|
||||
// **A delivery group's member is judged beside its siblings, and a sibling's move puts that verdict out of date**
|
||||
// (novox/hq ADR 0382 §4-§6, issue 867): the holder keeps the siblings each verdict names; a sibling's new head, close
|
||||
// unmerged or base move (other than its own merge of the head that was read) sets both statuses of every member whose
|
||||
// verdict names it back to pending, and runs nothing; a member merges only with its group passed and every sibling head
|
||||
// read approved.
|
||||
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { besideWords, groupRefusal, keepSiblings, namesSiblings, noteVerdict, readSiblings, siblingOutOfDate, siblingsFile,
|
||||
siblingsOf, siblingStatuses, weighSiblings, type SiblingRecord } from "../dist/siblings.js";
|
||||
import { weighTrunks } from "../dist/trunk.js";
|
||||
import { commentFor, statusesFor } from "../dist/pulls.js";
|
||||
|
||||
const h = (c: string) => c.repeat(40);
|
||||
const MEMBER_HEAD = h("a"), HOST_HEAD = h("b"), HOST_BASE = h("c"), HOST_NEW = h("d"), HOST_MOVED = h("e"), MERGE = h("f");
|
||||
const sibling = { repository: "novox/mesh-host", number: 114, head: HOST_HEAD, base: "main", base_commit: HOST_BASE };
|
||||
const verdict = { owner: "novox", repo: "mesh-controller", number: 346, commit: MEMBER_HEAD, verdict: "pass", summary: "x", id: "build-1",
|
||||
trunk: h("1").slice(0, 12), siblings: [sibling], judge_from: "sibling novox/mesh-controller#346 merge 123456789abc",
|
||||
gate: { verdict: "pass", summary: "every machine composes ".repeat(10) }, "repo-check": { verdict: "pass", summary: "ok" } };
|
||||
|
||||
test("both statuses name the siblings the verdict read, keeping the verdict word and the start of its reason within 140 characters", () => {
|
||||
// A realistic controller summary for a failure, long enough that the forge clips it.
|
||||
const reason = "2 of 4 machines do not compose: ace: mesh-host.service needs seat node-reads, which no module on ace claims after this change";
|
||||
const c = { ...verdict, gate: { verdict: "fail", summary: reason }, "repo-check": { verdict: "fail", summary: "its merge-check.sh failed: internal/link/check.go:145:2: undefined" } };
|
||||
const [gate, repo] = statusesFor(c);
|
||||
for (const s of [gate, repo]) {
|
||||
assert.ok(s.description.startsWith(`head ${MEMBER_HEAD.slice(0, 12)} merged into trunk 111111111111 beside mesh-host#114@bbbbbbbb — fail: `), s.description);
|
||||
assert.ok(s.description.length <= 140, `${s.description.length} characters`);
|
||||
assert.ok(namesSiblings(s.description));
|
||||
}
|
||||
assert.match(gate.description, /fail: 2 of 4 machines do not compose: ace: mesh-host/, gate.description);
|
||||
assert.match(repo.description, /fail: its merge-check\.sh failed: internal/, repo.description);
|
||||
assert.equal(besideWords([...siblingsOf([sibling]), ...siblingsOf([{ ...sibling, number: 115 }])]), "beside mesh-host#114@bbbbbbbb +1 more");
|
||||
const comment = commentFor({ ...verdict, gate: { verdict: "fail", summary: "no" } })!;
|
||||
assert.match(comment, /Siblings read beside it: novox\/mesh-host#114 `bbbbbbbbbbbb` merged onto `cccccccccccc` \(main\); judged by sibling novox\/mesh-controller#346 merge/);
|
||||
});
|
||||
|
||||
test("a verdict that could not learn its siblings says so on the status and in the comment, and names none", () => {
|
||||
const { siblings, ...unknown } = { ...verdict, siblings_unknown: "mesh-delivery did not answer" };
|
||||
const reason = "2 of 4 machines do not compose: ace: mesh-host.service needs seat node-reads, which no module claims";
|
||||
for (const s of statusesFor({ ...unknown, gate: { verdict: "fail", summary: reason } })) {
|
||||
assert.match(s.description, /^head aaaaaaaaaaaa merged into trunk 111111111111 siblings unknown — (fail|pass): /, s.description);
|
||||
assert.ok(s.description.length <= 140);
|
||||
assert.equal(namesSiblings(s.description), false, "it read no siblings, so the merge tool asks for none");
|
||||
}
|
||||
assert.match(commentFor({ ...unknown, gate: { verdict: "fail", summary: "no" } })!, /Siblings could not be learned: mesh-delivery did not answer/);
|
||||
});
|
||||
|
||||
test("a verdict without siblings is said exactly as before (an older controller)", () => {
|
||||
const { siblings, judge_from, ...old } = verdict;
|
||||
const [gate] = statusesFor(old);
|
||||
assert.equal(namesSiblings(gate.description), false);
|
||||
assert.match(gate.description, /^head aaaaaaaaaaaa merged into trunk 111111111111 — pass: /);
|
||||
assert.deepEqual(siblingsOf(undefined), []);
|
||||
assert.deepEqual(siblingsOf([{ repository: "novox/x" }]), [], "a sibling named without its head is none");
|
||||
});
|
||||
|
||||
test("the siblings a verdict names are kept per member, replaced by its next verdict, and removed by one that read none", () => {
|
||||
const file = siblingsFile(mkdtempSync(join(tmpdir(), "gitea-siblings-")));
|
||||
const record: SiblingRecord = {};
|
||||
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, []), false);
|
||||
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, siblingsOf([sibling])), true);
|
||||
keepSiblings(file, record);
|
||||
assert.deepEqual(readSiblings(file), record, "kept where the holder keeps its state");
|
||||
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, []), true);
|
||||
assert.deepEqual(record, {});
|
||||
assert.deepEqual(readSiblings(null), {});
|
||||
});
|
||||
|
||||
// --- a sibling's move -----------------------------------------------------------------------------------------
|
||||
|
||||
interface World {
|
||||
host: { merged: boolean; state: string; head_sha: string; merge_commit_sha?: string };
|
||||
hostBase: string;
|
||||
parents: string[];
|
||||
member: { head_sha: string; state: string; merged: boolean };
|
||||
checks: Record<string, { state: string; description?: string }>;
|
||||
written: { sha: string; context: string; state: string; description: string }[];
|
||||
reads: string[];
|
||||
}
|
||||
const world = (): World => ({ host: { merged: false, state: "open", head_sha: HOST_HEAD }, hostBase: HOST_BASE, parents: [],
|
||||
member: { head_sha: MEMBER_HEAD, state: "open", merged: false },
|
||||
checks: { "mesh/merge-gate": { state: "success", description: "head aaaaaaaaaaaa merged into trunk 111111111111 beside mesh-host#114@bbbbbbbb — pass: x" },
|
||||
"mesh/repo-check": { state: "success", description: "pass: y" } },
|
||||
written: [], reads: [] });
|
||||
const forgeOf = (w: World) => ({
|
||||
async getPullRequest(owner: string, repo: string, n: number) {
|
||||
w.reads.push(`${owner}/${repo}#${n}`);
|
||||
return repo === "mesh-host" ? { number: n, title: "", html_url: "", ...w.host } : { number: n, title: "", html_url: "", merged: false, ...w.member };
|
||||
},
|
||||
async branchHead(_o: string, _r: string, _b: string) { return w.hostBase; },
|
||||
async commitParents() { return w.parents; },
|
||||
});
|
||||
const gatesOf = (w: World) => ({
|
||||
async outOfDate(_o: string, _r: string, sha: string, decide: (c: any) => any[] | Promise<any[]>) {
|
||||
const put = await decide(w.checks);
|
||||
for (const s of put) { w.written.push({ sha, ...s }); w.checks[s.context] = { state: s.state, description: s.description }; }
|
||||
return put;
|
||||
},
|
||||
});
|
||||
const recordOf = (): SiblingRecord => ({ "novox/mesh-controller#346": { head: MEMBER_HEAD, siblings: siblingsOf([sibling]) } });
|
||||
const weigh = (w: World, record: SiblingRecord, repository = "novox/mesh-host", number?: number) =>
|
||||
weighSiblings(forgeOf(w) as any, gatesOf(w) as any, record, repository, number);
|
||||
|
||||
test("a sibling's new head puts both statuses of the member back to pending, with the reason, and runs nothing", async () => {
|
||||
const w = world();
|
||||
w.host.head_sha = HOST_NEW;
|
||||
await weigh(w, recordOf(), "novox/mesh-host", 114);
|
||||
assert.deepEqual(w.written.map((s) => [s.sha, s.context, s.state]),
|
||||
[[MEMBER_HEAD, "mesh/merge-gate", "pending"], [MEMBER_HEAD, "mesh/repo-check", "pending"]]);
|
||||
assert.equal(w.written[0].description,
|
||||
"out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd; judged again on a recheck or when it is merged");
|
||||
assert.ok(siblingOutOfDate(w.written[0].description));
|
||||
// Already pending (out of date, or a check on its way): not written again.
|
||||
w.written.length = 0;
|
||||
await weigh(w, recordOf(), "novox/mesh-host", 114);
|
||||
assert.deepEqual(w.written, []);
|
||||
});
|
||||
|
||||
test("a sibling closed unmerged puts the member's verdict out of date", async () => {
|
||||
const w = world();
|
||||
w.host.state = "closed";
|
||||
await weigh(w, recordOf(), "novox/mesh-host", 114);
|
||||
assert.equal(w.written.length, 2);
|
||||
assert.match(w.written[0].description, /^out of date: judged beside mesh-host#114 was closed unmerged/);
|
||||
});
|
||||
|
||||
test("a move of the sibling's base puts it out of date, naming both commits", async () => {
|
||||
const w = world();
|
||||
w.hostBase = HOST_MOVED;
|
||||
await weigh(w, recordOf());
|
||||
assert.equal(w.written.length, 2);
|
||||
assert.match(w.written[0].description, /mesh-host#114 merged onto cccccccccccc, its main is now eeeeeeeeeeee/);
|
||||
});
|
||||
|
||||
test("the sibling's own merge of the head read onto the base read does not, in either order of the two merges", async () => {
|
||||
// The sibling merges first (a merge commit of the base commit read and the head read).
|
||||
const w = world();
|
||||
Object.assign(w.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
|
||||
w.hostBase = MERGE;
|
||||
w.parents = [HOST_BASE, HOST_HEAD];
|
||||
const record = recordOf();
|
||||
assert.equal(await weigh(w, record), true, "the record changes: the sibling is marked merged");
|
||||
assert.deepEqual(w.written, []);
|
||||
assert.equal(record["novox/mesh-controller#346"].siblings[0].merged, true);
|
||||
// A move of that base afterwards is no sibling's move: the sibling is merged.
|
||||
w.hostBase = HOST_MOVED;
|
||||
await weigh(w, record);
|
||||
assert.deepEqual(w.written, []);
|
||||
|
||||
// The member merges first: its own repository moves, which no member's verdict names as a sibling.
|
||||
const m = world();
|
||||
m.member.merged = true;
|
||||
await weigh(m, recordOf(), "novox/mesh-controller");
|
||||
assert.deepEqual(m.written, []);
|
||||
// And the sibling then merges: still not a move.
|
||||
Object.assign(m.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
|
||||
m.parents = [HOST_BASE, HOST_HEAD];
|
||||
await weigh(m, recordOf());
|
||||
assert.deepEqual(m.written, []);
|
||||
|
||||
// A squash is one parent: the base commit read.
|
||||
const q = world();
|
||||
Object.assign(q.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
|
||||
q.parents = [HOST_BASE];
|
||||
await weigh(q, recordOf());
|
||||
assert.deepEqual(q.written, []);
|
||||
});
|
||||
|
||||
test("a merge of another head, or onto another base, is a move", async () => {
|
||||
const other = world();
|
||||
Object.assign(other.host, { merged: true, state: "closed", merge_commit_sha: MERGE, head_sha: HOST_NEW });
|
||||
other.parents = [HOST_BASE, HOST_NEW];
|
||||
await weigh(other, recordOf());
|
||||
assert.equal(other.written.length, 2);
|
||||
const onto = world();
|
||||
Object.assign(onto.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
|
||||
onto.parents = [HOST_MOVED, HOST_HEAD];
|
||||
await weigh(onto, recordOf());
|
||||
assert.equal(onto.written.length, 2);
|
||||
});
|
||||
|
||||
test("only the members whose verdict names the sibling are touched, and a member that moved on is dropped", async () => {
|
||||
// Another sibling's move, or another repository's: nothing read, nothing written.
|
||||
const w = world();
|
||||
w.host.head_sha = HOST_NEW;
|
||||
await weigh(w, recordOf(), "novox/mesh-lab");
|
||||
await weigh(w, recordOf(), "novox/mesh-host", 999);
|
||||
assert.deepEqual(w.written, []);
|
||||
// A verdict that is none (a gate not yet judged) is left alone.
|
||||
const none = world();
|
||||
none.host.head_sha = HOST_NEW;
|
||||
none.checks = {};
|
||||
await weigh(none, recordOf());
|
||||
assert.deepEqual(none.written, []);
|
||||
// The member has a new head: its old verdict is no verdict; the record is dropped, nothing written to the old head.
|
||||
const moved = world();
|
||||
moved.host.head_sha = HOST_NEW;
|
||||
moved.member.head_sha = h("9");
|
||||
const record = recordOf();
|
||||
assert.equal(await weigh(moved, record), true);
|
||||
assert.deepEqual(moved.written, []);
|
||||
assert.deepEqual(record, {});
|
||||
// A repository without a repo-check gets no pending one a verdict would never set.
|
||||
assert.deepEqual(siblingStatuses({ "mesh/merge-gate": { state: "success" } }, "x").map((s) => s.context), ["mesh/merge-gate"]);
|
||||
});
|
||||
|
||||
test("a trunk that moved tells the siblings' weighing; one first seen does not", async () => {
|
||||
const told: string[] = [];
|
||||
const forge = { listOpenPulls: async () => [{ number: 1, title: "", state: "open", merged: false, head_sha: h("2"), base: "main", html_url: "" }],
|
||||
branchHead: async () => h("3") };
|
||||
const gates = { outOfDate: async () => [] };
|
||||
const trunks: Record<string, string> = {};
|
||||
await weighTrunks(forge as any, gates as any, trunks, "novox", "mesh-host", async (b) => { told.push(b); });
|
||||
assert.deepEqual(told, [], "first seen");
|
||||
trunks["novox/mesh-host:main"] = h("4");
|
||||
await weighTrunks(forge as any, gates as any, trunks, "novox", "mesh-host", async (b) => { told.push(b); });
|
||||
assert.deepEqual(told, ["main"]);
|
||||
});
|
||||
|
||||
// --- the merge rule -------------------------------------------------------------------------------------------
|
||||
|
||||
test("the group rule names every missing thing: the group status at each head read, and each sibling head's review", async () => {
|
||||
const self = { owner: "novox", repo: "mesh-controller", number: 346, head: MEMBER_HEAD };
|
||||
const statuses: Record<string, string> = {};
|
||||
const approved = new Set<string>();
|
||||
const forge = {
|
||||
headChecks: async (_o: string, r: string, sha: string) => (statuses[`${r}@${sha}`] ? { "mesh/delivery-group": { state: statuses[`${r}@${sha}`] } } : {}),
|
||||
approvedAt: async (_o: string, r: string, n: number, head: string) => approved.has(`${r}#${n}@${head}`),
|
||||
};
|
||||
const siblings = siblingsOf([sibling]);
|
||||
const none = await groupRefusal(forge, self, siblings);
|
||||
assert.match(none!, /mesh\/delivery-group is missing at mesh-controller#346@aaaaaaaaaaaa; mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review at that head/);
|
||||
statuses[`mesh-controller@${MEMBER_HEAD}`] = "success";
|
||||
statuses[`mesh-host@${HOST_HEAD}`] = "pending";
|
||||
assert.match((await groupRefusal(forge, self, siblings))!, /^mesh\/delivery-group is pending at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review/);
|
||||
statuses[`mesh-host@${HOST_HEAD}`] = "success";
|
||||
assert.match((await groupRefusal(forge, self, siblings))!, /^mesh-host#114@bbbbbbbbbbbb carries no approving review at that head$/);
|
||||
approved.add(`mesh-host#114@${HOST_HEAD}`);
|
||||
assert.equal(await groupRefusal(forge, self, siblings), undefined);
|
||||
// A read that fails is named, never read as a pass.
|
||||
const broken = { ...forge, approvedAt: async () => { throw new Error("the forge is away"); } };
|
||||
assert.match((await groupRefusal(broken, self, siblings))!, /could not be read \(the forge is away\)/);
|
||||
assert.equal(besideWords(siblings), "beside mesh-host#114@bbbbbbbb");
|
||||
});
|
||||
|
||||
test("a merged or closed member's entry is dropped, so a sibling's later push never re-pends its merged head", async () => {
|
||||
// mesh-host#114 judged beside mesh-controller#346; #114 then merged; a push to #346 follows.
|
||||
const w = world();
|
||||
w.host.merged = true; // the member (mesh-host#114) is merged
|
||||
w.member.head_sha = h("9"); // the sibling (mesh-controller#346) has a new head
|
||||
const record: SiblingRecord = { "novox/mesh-host#114": { head: HOST_HEAD, siblings: siblingsOf([
|
||||
{ repository: "novox/mesh-controller", number: 346, head: MEMBER_HEAD, base: "main", base_commit: h("2") }]) } };
|
||||
assert.equal(await weigh(w, record, "novox/mesh-controller", 346), true);
|
||||
assert.deepEqual(w.written, [], "the merged head is not put back to pending");
|
||||
assert.deepEqual(record, {}, "and its entry is gone");
|
||||
// On its own close or merge the member's entry goes too, whatever the siblings do.
|
||||
const own = world();
|
||||
own.host.state = "closed";
|
||||
const kept = recordOf();
|
||||
const mine: SiblingRecord = { "novox/mesh-host#114": { head: HOST_HEAD, siblings: kept["novox/mesh-controller#346"].siblings } };
|
||||
assert.equal(await weigh(own, mine, "novox/mesh-host", 114), true);
|
||||
assert.deepEqual(mine, {});
|
||||
assert.deepEqual(own.written, []);
|
||||
});
|
||||
@@ -10,7 +10,7 @@
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
|
||||
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
@@ -29,6 +29,12 @@ interface Forge {
|
||||
mints: number;
|
||||
lastScopes: string[] | null;
|
||||
tokens: Map<string, string>;
|
||||
/** Tokens beside `tokens` that carry a name already held there: what two mints at once leave behind. */
|
||||
twins: { id: number; name: string; sha1: string }[];
|
||||
/** How each DELETE named its token: "id" or "name". */
|
||||
deletedBy: string[];
|
||||
/** Plant a token by name, beside any other of the same name, as a raced mint would. */
|
||||
plant(name: string): void;
|
||||
scopesOf: Map<string, string[]>;
|
||||
admins: Map<string, string>;
|
||||
pullState: string;
|
||||
@@ -39,9 +45,14 @@ interface Forge {
|
||||
|
||||
function fakeForge(): Promise<Forge> {
|
||||
const forge = {
|
||||
mintedFor: "",
|
||||
mints: 0,
|
||||
lastScopes: null as string[] | null,
|
||||
tokens: new Map<string, string>(), // name -> value
|
||||
twins: [] as { id: number; name: string; sha1: string }[],
|
||||
deletedBy: [] as string[],
|
||||
ids: new Map<string, number>(), // value -> id, as gitea numbers its tokens
|
||||
nextId: 0,
|
||||
scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does
|
||||
admins: new Map([[ADMIN, PASSWORD]]),
|
||||
pullState: "open",
|
||||
@@ -73,7 +84,14 @@ function fakeForge(): Promise<Forge> {
|
||||
const tokens = url.pathname.match(/^\/api\/v1\/users\/([^/]+)\/tokens(?:\/([^/]+))?$/);
|
||||
if (tokens) {
|
||||
const user = basic(req);
|
||||
if (user === null || user !== decodeURIComponent(tokens[1])) return json(res, 401, { message: "auth required" });
|
||||
// gitea's reqSelfOrAdmin: the account itself, or the administrator for another (the tools' account, ADR 0365).
|
||||
if (user === null || (user !== decodeURIComponent(tokens[1]) && user !== ADMIN)) return json(res, 401, { message: "auth required" });
|
||||
// This fake keeps one account's tokens: the tools' account's when the module mints for it (ADR 0365), and the
|
||||
// admin's own list is then empty — nothing of it is retired.
|
||||
if (decodeURIComponent(tokens[1]) === ADMIN && forge.mintedFor === "mesh-tools") {
|
||||
return req.method === "GET" ? json(res, 200, []) : json(res, 404, { message: "token not found" });
|
||||
}
|
||||
if (req.method === "POST") forge.mintedFor = decodeURIComponent(tokens[1]);
|
||||
if (req.method === "POST") {
|
||||
const { name, scopes } = await body(req);
|
||||
if (forge.tokens.has(name)) return json(res, 400, { message: "token name has already been used" });
|
||||
@@ -82,12 +100,35 @@ function fakeForge(): Promise<Forge> {
|
||||
const sha1 = `minted-${forge.mints}-${Math.random().toString(36).slice(2)}`;
|
||||
forge.tokens.set(name, sha1);
|
||||
forge.scopesOf.set(sha1, scopes);
|
||||
return json(res, 201, { id: forge.mints, name, sha1, scopes, token_last_eight: sha1.slice(-8) });
|
||||
forge.ids.set(sha1, ++forge.nextId);
|
||||
return json(res, 201, { id: forge.ids.get(sha1), name, sha1, scopes, token_last_eight: sha1.slice(-8) });
|
||||
}
|
||||
// gitea's list: every token of the account, a page at a time (at most three to a page here, so a
|
||||
// client that reads only the first page misses some), the values never shown.
|
||||
const listed = (): { id: number; name: string; sha1: string }[] => [
|
||||
...[...forge.tokens].map(([name, sha1]) => ({ id: forge.ids.get(sha1) ?? 0, name, sha1 })),
|
||||
...forge.twins,
|
||||
];
|
||||
if (req.method === "GET" && !tokens[2]) {
|
||||
const page = Math.max(1, Number(url.searchParams.get("page") ?? "1"));
|
||||
const limit = Math.min(3, Math.max(1, Number(url.searchParams.get("limit") ?? "3")));
|
||||
const all = listed();
|
||||
res.setHeader("X-Total-Count", String(all.length));
|
||||
return json(res, 200, all.slice((page - 1) * limit, page * limit)
|
||||
.map((t) => ({ id: t.id, name: t.name, sha1: "", token_last_eight: t.sha1.slice(-8), scopes: [] })));
|
||||
}
|
||||
if (req.method === "DELETE" && tokens[2]) {
|
||||
const name = decodeURIComponent(tokens[2]);
|
||||
if (!forge.tokens.has(name)) return json(res, 404, { message: "token not found" });
|
||||
forge.tokens.delete(name);
|
||||
// gitea: "identified by ID and if not available by name" — and a name held twice is refused.
|
||||
const key = decodeURIComponent(tokens[2]);
|
||||
const byId = /^\d+$/.test(key);
|
||||
const matches = listed().filter((t) => (byId ? String(t.id) === key : t.name === key));
|
||||
if (matches.length === 0) return json(res, 404, { message: "token not found" });
|
||||
if (matches.length > 1) return json(res, 422, { message: `multiple matches for token name '${key}'` });
|
||||
const t = matches[0];
|
||||
forge.deletedBy.push(byId ? "id" : "name");
|
||||
const twin = forge.twins.findIndex((w) => w.id === t.id);
|
||||
if (twin >= 0) forge.twins.splice(twin, 1);
|
||||
else forge.tokens.delete(t.name);
|
||||
return json(res, 204, null);
|
||||
}
|
||||
return json(res, 405, { message: "method not allowed" });
|
||||
@@ -153,6 +194,17 @@ function fakeForge(): Promise<Forge> {
|
||||
{ full_name: "novox/hq", name: "hq", owner: { login: "novox" }, private: true, html_url: "http://fake/novox/hq" },
|
||||
]);
|
||||
}
|
||||
// What setting up the tools' account asks with the admin login (tools-account.ts, ADR 0365): answered plainly.
|
||||
if (basic(req) === ADMIN) {
|
||||
const p = url.pathname;
|
||||
if (req.method === "GET" && p === "/api/v1/user") return json(res, 200, { login: ADMIN, is_admin: true });
|
||||
if (req.method === "GET" && p === "/api/v1/users/mesh-tools") return json(res, 200, { login: "mesh-tools", is_admin: false });
|
||||
if (req.method === "POST" && p === "/api/v1/admin/users") return json(res, 422, { message: "user already exists" });
|
||||
if (req.method === "PATCH" && p === "/api/v1/admin/users/mesh-tools") return json(res, 200, { login: "mesh-tools" });
|
||||
if (req.method === "GET" && p === "/api/v1/orgs/novox/teams") return json(res, 200, [{ id: 7, name: "mesh-tools" }]);
|
||||
if (req.method === "PATCH" && p === "/api/v1/teams/7") return json(res, 200, { id: 7 });
|
||||
if (req.method === "PUT" && p === "/api/v1/teams/7/members/mesh-tools") return json(res, 204, null);
|
||||
}
|
||||
const adminUser = url.pathname.match(/^\/api\/v1\/admin\/users\/([^/]+)$/);
|
||||
if (adminUser && req.method === "PATCH") {
|
||||
const h = req.headers.authorization ?? "";
|
||||
@@ -181,6 +233,14 @@ function fakeForge(): Promise<Forge> {
|
||||
get pullTitle() { return forge.pullTitle; },
|
||||
get branchDeleted() { return forge.branchDeleted; },
|
||||
tokens: forge.tokens,
|
||||
twins: forge.twins,
|
||||
deletedBy: forge.deletedBy,
|
||||
plant: (name: string) => {
|
||||
const sha1 = `planted-${++forge.nextId}`;
|
||||
forge.ids.set(sha1, forge.nextId);
|
||||
if (forge.tokens.has(name)) forge.twins.push({ id: forge.nextId, name, sha1 });
|
||||
else forge.tokens.set(name, sha1);
|
||||
},
|
||||
scopesOf: forge.scopesOf,
|
||||
admins: forge.admins,
|
||||
close: () => new Promise((r) => server.close(() => r())),
|
||||
@@ -335,6 +395,56 @@ test("concurrent first calls share one mint", async () => {
|
||||
assert.equal(forge.mints, before + 1);
|
||||
});
|
||||
|
||||
test("two tokens hold the name (two mints raced): both are deleted by their id, and one fresh token is minted", async () => {
|
||||
const { env, file, logs } = await delivered(forge);
|
||||
forge.tokens.clear();
|
||||
forge.twins.length = 0;
|
||||
for (const other of ["laptop", "backup", "ci"]) forge.plant(other); // the account's other tokens: kept
|
||||
forge.plant("mesh-tools");
|
||||
forge.plant("mesh-tools"); // gitea refuses to delete either by name from here on
|
||||
forge.deletedBy.length = 0;
|
||||
const before = forge.mints;
|
||||
|
||||
const repos = await minted(env, logs).listRepos();
|
||||
|
||||
assert.equal(repos.length, 1);
|
||||
assert.equal(forge.mints, before + 1);
|
||||
assert.deepEqual(forge.deletedBy, ["id", "id"]);
|
||||
assert.equal(forge.twins.length, 0);
|
||||
assert.deepEqual([...forge.tokens.keys()].sort(), ["backup", "ci", "laptop", "mesh-tools"]);
|
||||
assert.equal(await readFile(file, "utf8"), forge.tokens.get("mesh-tools") + "\n");
|
||||
assert.ok(logs.some((l) => l.includes('holds 2 tokens named "mesh-tools"')), logs.join("\n"));
|
||||
});
|
||||
|
||||
test("two processes on one kept file, both without a token, mint once: the second waits for the first and reuses it", async () => {
|
||||
const { env, logs } = await delivered(forge);
|
||||
forge.tokens.clear();
|
||||
forge.twins.length = 0;
|
||||
const before = forge.mints;
|
||||
// Two sources over one state directory, as the tools bundle and the watcher are two processes.
|
||||
const first = minted(env, logs);
|
||||
const second = minted(env, logs);
|
||||
|
||||
await Promise.all([first.listRepos(), second.listRepos()]);
|
||||
|
||||
assert.equal(forge.mints, before + 1);
|
||||
assert.equal(forge.twins.length, 0);
|
||||
assert.ok(logs.some((l) => l.includes("another process minted it")), logs.join("\n"));
|
||||
});
|
||||
|
||||
test("a lock left by a process that died mid-mint is taken over, not waited on for ever", async () => {
|
||||
const { env, file, logs } = await delivered(forge);
|
||||
forge.tokens.clear();
|
||||
forge.twins.length = 0;
|
||||
await mkdir(join(env.MESH_GITEA_STATE_DIR!), { recursive: true });
|
||||
await writeFile(`${file}.lock`, `2147483646 ${Date.now()} gone\n`); // a pid no process holds
|
||||
const before = forge.mints;
|
||||
|
||||
assert.equal((await minted(env, logs).listRepos()).length, 1);
|
||||
assert.equal(forge.mints, before + 1);
|
||||
await assert.rejects(stat(`${file}.lock`), /ENOENT/);
|
||||
});
|
||||
|
||||
test("the admin account is refused: said plainly, nothing kept, and the next call fails the same way rather than crashing", async () => {
|
||||
const { env, file, logs } = await delivered(forge);
|
||||
forge.admins.delete(ADMIN); // the forge's data came from a predecessor; mesh-admin was never created there
|
||||
|
||||
@@ -0,0 +1,332 @@
|
||||
// The forge's tools act as an account of their own, not the site administrator (novox/hq ADR 0365 §2, issue 469):
|
||||
// the token the tools and the watcher use is minted for `mesh-tools`, a plain account made by the module with the
|
||||
// administrator's login and given write on the mesh's repositories through one team; the administrator's own
|
||||
// token by that name goes; a token kept from before, the administrator's, is minted again for the account.
|
||||
//
|
||||
// The forge is a fake that keeps accounts, their tokens and the organisation's teams apart, and answers as gitea
|
||||
// does: an administrator may mint for another account (reqSelfOrAdmin), anyone else only for itself; a protection
|
||||
// edit needs the repository's administration, which neither a plain account nor its write team holds.
|
||||
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
import { mkdtemp, readFile, writeFile, mkdir } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { MintedToken } from "../dist/token.js";
|
||||
import { GiteaClient } from "../dist/client.js";
|
||||
import { TOOLS_ACCOUNT, TOOLS_TEAM_UNITS } from "../dist/tools-account.js";
|
||||
import { repoCheckFacts } from "../dist/pulls.js";
|
||||
|
||||
const ADMIN = "mesh-admin";
|
||||
const PASSWORD = "the-vault-minted-this";
|
||||
|
||||
interface Account { admin: boolean; password: string; active: boolean }
|
||||
interface Token { id: number; name: string; sha1: string; user: string; scopes: string[] }
|
||||
interface Team { id: number; name: string; units: any; members: Set<string> }
|
||||
|
||||
function fakeForge() {
|
||||
const users = new Map<string, Account>([[ADMIN, { admin: true, password: PASSWORD, active: true }]]);
|
||||
const tokens: Token[] = [];
|
||||
const teams: Team[] = [];
|
||||
const calls: string[] = [];
|
||||
// What the forge does wrong when a test says so: a token mint or delete it fails, a team another process made
|
||||
// between this one's look and its create (gitea answers the create 422).
|
||||
const faults = { mint: false, deleteToken: false, teamRace: false };
|
||||
let next = 0;
|
||||
const json = (res: ServerResponse, status: number, body: unknown): void => {
|
||||
res.writeHead(status, { "Content-Type": "application/json" });
|
||||
res.end(body === null ? "" : JSON.stringify(body));
|
||||
};
|
||||
const read = (req: IncomingMessage): Promise<any> =>
|
||||
new Promise((resolve) => {
|
||||
let t = "";
|
||||
req.on("data", (c) => (t += c));
|
||||
req.on("end", () => resolve(t ? JSON.parse(t) : null));
|
||||
});
|
||||
const who = (req: IncomingMessage): string | null => {
|
||||
const h = req.headers.authorization ?? "";
|
||||
if (h.startsWith("Basic ")) {
|
||||
const [u, p] = Buffer.from(h.slice(6), "base64").toString().split(":");
|
||||
return users.get(u)?.password === p ? u : null;
|
||||
}
|
||||
if (h.startsWith("token ")) return tokens.find((t) => t.sha1 === h.slice(6))?.user ?? null;
|
||||
return null;
|
||||
};
|
||||
const isAdmin = (u: string | null) => u !== null && users.get(u)?.admin === true;
|
||||
|
||||
const server = createServer(async (req, res) => {
|
||||
const url = new URL(req.url ?? "/", "http://fake");
|
||||
const p = url.pathname.replace(/^\/api\/v1/, "");
|
||||
const me = who(req);
|
||||
calls.push(`${req.method} ${p} as ${me}`);
|
||||
if (me === null) return json(res, 401, { message: "unauthorized" });
|
||||
|
||||
let m = p.match(/^\/users\/([^/]+)\/tokens(?:\/(\d+))?$/);
|
||||
if (m) {
|
||||
const owner = decodeURIComponent(m[1]);
|
||||
if (!(req.headers.authorization ?? "").startsWith("Basic ")) return json(res, 401, { message: "basic auth required" });
|
||||
if (owner !== me && !isAdmin(me)) return json(res, 403, { message: "reqSelfOrAdmin" });
|
||||
if (!users.has(owner)) return json(res, 404, { message: "user does not exist" });
|
||||
if (req.method === "POST") {
|
||||
if (faults.mint) return json(res, 500, { message: "the forge is not well" });
|
||||
const { name, scopes } = await read(req);
|
||||
if (tokens.some((t) => t.user === owner && t.name === name)) return json(res, 400, { message: "token name has already been used" });
|
||||
const t = { id: ++next, name, sha1: `sha-${owner}-${next}`, user: owner, scopes };
|
||||
tokens.push(t);
|
||||
return json(res, 201, { id: t.id, name, sha1: t.sha1, scopes });
|
||||
}
|
||||
if (req.method === "GET") {
|
||||
const page = Number(url.searchParams.get("page") ?? "1");
|
||||
const mine = tokens.filter((t) => t.user === owner);
|
||||
return json(res, 200, page === 1 ? mine.map((t) => ({ id: t.id, name: t.name, sha1: "" })) : []);
|
||||
}
|
||||
if (req.method === "DELETE" && m[2]) {
|
||||
if (faults.deleteToken && owner === ADMIN) return json(res, 500, { message: "the forge is not well" });
|
||||
const i = tokens.findIndex((t) => t.user === owner && t.id === Number(m![2]));
|
||||
if (i < 0) return json(res, 404, { message: "not found" });
|
||||
tokens.splice(i, 1);
|
||||
return json(res, 204, null);
|
||||
}
|
||||
}
|
||||
if (req.method === "GET" && p === "/user") return json(res, 200, { login: me, is_admin: isAdmin(me) });
|
||||
m = p.match(/^\/users\/([^/]+)$/);
|
||||
if (m && req.method === "GET") {
|
||||
const u = users.get(decodeURIComponent(m[1]));
|
||||
if (!u) return json(res, 404, { message: "user does not exist" });
|
||||
return json(res, 200, { login: m[1], is_admin: u.admin, active: u.active, prohibit_login: false });
|
||||
}
|
||||
if (p.startsWith("/admin/") && !isAdmin(me)) return json(res, 403, { message: "must be a site administrator" });
|
||||
if (req.method === "POST" && p === "/admin/users") {
|
||||
const b = await read(req);
|
||||
if (users.has(b.username)) return json(res, 422, { message: "user already exists" });
|
||||
if (!b.password || b.password.length < 8) return json(res, 422, { message: "password is too short" });
|
||||
users.set(b.username, { admin: b.admin === true, password: b.password, active: true });
|
||||
return json(res, 201, { login: b.username, is_admin: b.admin === true });
|
||||
}
|
||||
m = p.match(/^\/admin\/users\/([^/]+)$/);
|
||||
if (m && req.method === "PATCH") {
|
||||
const u = users.get(decodeURIComponent(m[1]));
|
||||
if (!u) return json(res, 404, { message: "user does not exist" });
|
||||
const b = await read(req);
|
||||
if (b.admin !== undefined) u.admin = b.admin;
|
||||
if (b.active !== undefined) u.active = b.active;
|
||||
return json(res, 200, { login: m[1], is_admin: u.admin });
|
||||
}
|
||||
if (p === "/orgs/novox/teams") {
|
||||
if (!isAdmin(me)) return json(res, 403, { message: "must be an owner of the organisation" });
|
||||
if (req.method === "GET") return json(res, 200, teams.map((t) => ({ id: t.id, name: t.name })));
|
||||
const b = await read(req);
|
||||
const t = { id: ++next, name: b.name, units: b, members: new Set<string>() };
|
||||
teams.push(t);
|
||||
if (faults.teamRace) return json(res, 422, { message: "team already exists" });
|
||||
return json(res, 201, { id: t.id, name: t.name });
|
||||
}
|
||||
m = p.match(/^\/teams\/(\d+)(?:\/members\/([^/]+))?$/);
|
||||
if (m) {
|
||||
if (!isAdmin(me)) return json(res, 403, { message: "must be an owner of the organisation" });
|
||||
const t = teams.find((x) => x.id === Number(m![1]));
|
||||
if (!t) return json(res, 404, { message: "team does not exist" });
|
||||
if (req.method === "PATCH") { t.units = await read(req); return json(res, 200, { id: t.id }); }
|
||||
if (req.method === "PUT" && m[2]) { t.members.add(decodeURIComponent(m[2])); return json(res, 204, null); }
|
||||
}
|
||||
// Repository administration: a site administrator, or a team member whose team administers the repository.
|
||||
m = p.match(/^\/repos\/novox\/([^/]+)\/branches\/(.+)$/);
|
||||
if (m && req.method === "GET") {
|
||||
if (m[2] === "main") return json(res, 200, { name: "main", protected: true, effective_branch_protection_name: "main",
|
||||
enable_status_check: true, status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"], user_can_push: false });
|
||||
if (m[2] === "feat/unguarded") return json(res, 200, { name: m[2], protected: false });
|
||||
return json(res, 404, { message: "branch does not exist" });
|
||||
}
|
||||
// gitea's reqAdmin on the whole group: reading the rules is administration too.
|
||||
m = p.match(/^\/repos\/novox\/([^/]+)\/branch_protections(?:\/.*)?$/);
|
||||
if (m) {
|
||||
const admins = isAdmin(me) || teams.some((t) => t.members.has(me) && t.units.permission === "admin");
|
||||
if (!admins) return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
|
||||
return json(res, 201, { rule_name: "main" });
|
||||
}
|
||||
m = p.match(/^\/repos\/novox\/([^/]+)\/statuses\/([0-9a-f]+)$/);
|
||||
if (m && req.method === "POST") {
|
||||
const writes = isAdmin(me) || teams.some((t) => t.members.has(me) && t.units.units_map?.["repo.code"] === "write");
|
||||
if (!writes) return json(res, 403, { message: "write on code required" });
|
||||
return json(res, 201, { context: (await read(req)).context, creator: { login: me } });
|
||||
}
|
||||
if (req.method === "GET" && p === "/repos/search") return json(res, 200, { data: [] });
|
||||
return json(res, 404, { message: "no such route in the fake" });
|
||||
});
|
||||
return new Promise<{ url: string; users: typeof users; tokens: typeof tokens; teams: typeof teams; calls: string[];
|
||||
faults: typeof faults; close(): Promise<void> }>((resolve) => {
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
const { port } = server.address() as { port: number };
|
||||
resolve({ url: `http://127.0.0.1:${port}`, users, tokens, teams, calls, faults,
|
||||
close: () => new Promise((r) => server.close(() => r())) });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
const forge = await fakeForge();
|
||||
after(() => forge.close());
|
||||
|
||||
async function environment(): Promise<NodeJS.ProcessEnv & { MESH_GITEA_STATE_DIR: string }> {
|
||||
const dir = await mkdtemp(join(tmpdir(), "gitea-tools-account-"));
|
||||
const passwordFile = join(dir, "admin.secret");
|
||||
await writeFile(passwordFile, PASSWORD + "\n", { mode: 0o600 });
|
||||
return {
|
||||
MESH_GITEA_URL: forge.url,
|
||||
MESH_GITEA_ADMIN_USER: ADMIN,
|
||||
MESH_GITEA_ADMIN_PASSWORD_FILE: passwordFile,
|
||||
MESH_GITEA_STATE_DIR: join(dir, "state"),
|
||||
};
|
||||
}
|
||||
|
||||
/** A source as a fresh process builds it from the manifest's environment (fromEnv shares one per file). */
|
||||
function fresh(env: NodeJS.ProcessEnv): GiteaClient {
|
||||
return GiteaClient.fromEnv(env);
|
||||
}
|
||||
|
||||
test("the tools' token is minted for an account of their own, which is no administrator and writes through its team", async () => {
|
||||
const env = await environment();
|
||||
await fresh(env).listRepos();
|
||||
|
||||
const account = forge.users.get(TOOLS_ACCOUNT);
|
||||
assert.ok(account, "the module made the tools' account");
|
||||
assert.equal(account!.admin, false, "the tools' account is no administrator");
|
||||
const held = forge.tokens.filter((t) => t.name === "mesh-tools");
|
||||
assert.deepEqual(held.map((t) => t.user), [TOOLS_ACCOUNT], "the one token is the tools' account's, none is the administrator's");
|
||||
assert.equal(await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8"), held[0].sha1 + "\n");
|
||||
|
||||
const team = forge.teams.find((t) => t.name === "mesh-tools");
|
||||
assert.ok(team, "the team is made in the mesh's organisation");
|
||||
assert.ok(team!.members.has(TOOLS_ACCOUNT));
|
||||
assert.equal(team!.units.permission, "write");
|
||||
assert.equal(team!.units.includes_all_repositories, true);
|
||||
assert.equal(team!.units.can_create_org_repo, false, "a repository's maker would administer it");
|
||||
assert.deepEqual(team!.units.units_map, TOOLS_TEAM_UNITS.units_map);
|
||||
assert.ok(!Object.values(team!.units.units_map).includes("admin"));
|
||||
});
|
||||
|
||||
test("with the tools' token, the forge writes a status and refuses a protection edit — the forge itself, not a guard", async () => {
|
||||
const env = await environment();
|
||||
const client = fresh(env);
|
||||
await client.setCommitStatus("novox", "hq", "abc123def4567890abc123def4567890abc12345", { state: "success", context: "mesh/merge-gate", description: "ok" } as any);
|
||||
await assert.rejects(
|
||||
client.setBranchProtection("novox", "hq", "main", { statusChecks: ["mesh/merge-gate"] }),
|
||||
/403/,
|
||||
);
|
||||
});
|
||||
|
||||
test("what a base's protection asks is read from the branch, which the tools' account may read, not from the rules' list it may not", async () => {
|
||||
const env = await environment();
|
||||
const client = fresh(env);
|
||||
await assert.rejects(client.branchProtections("novox", "hq"), /403/);
|
||||
const rule = await client.appliedProtection("novox", "hq", "main");
|
||||
assert.deepEqual(rule?.status_check_contexts, ["mesh/merge-gate", "mesh/repo-check"]);
|
||||
assert.equal(rule?.enable_status_check, true);
|
||||
assert.equal(rule?.rule_name, "main");
|
||||
assert.equal(await client.appliedProtection("novox", "hq", "feat/unguarded"), null);
|
||||
assert.equal(await client.appliedProtection("novox", "hq", "no/such-branch"), null);
|
||||
});
|
||||
|
||||
test("the administrator's token from before goes: a kept admin token is minted again for the tools' account, and the forge drops the old one", async () => {
|
||||
const env = await environment();
|
||||
// What the module kept before ADR 0365: a token of the administrator's own, by the same name, its scopes beside it.
|
||||
const old = { id: 900, name: "mesh-tools", sha1: "sha-admin-old", user: ADMIN, scopes: ["write:repository", "write:issue", "read:user"] };
|
||||
forge.tokens.push(old);
|
||||
await mkdir(env.MESH_GITEA_STATE_DIR, { recursive: true });
|
||||
await writeFile(join(env.MESH_GITEA_STATE_DIR, "token"), old.sha1 + "\n", { mode: 0o600 });
|
||||
await writeFile(join(env.MESH_GITEA_STATE_DIR, "token.scopes"), "read:user,write:issue,write:repository\n", { mode: 0o600 });
|
||||
|
||||
await fresh(env).listRepos();
|
||||
|
||||
assert.ok(!forge.tokens.some((t) => t.user === ADMIN), "the administrator holds no tools' token any more");
|
||||
const kept = (await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8")).trim();
|
||||
assert.notEqual(kept, old.sha1);
|
||||
assert.equal(forge.tokens.find((t) => t.sha1 === kept)?.user, TOOLS_ACCOUNT);
|
||||
assert.match(await readFile(join(env.MESH_GITEA_STATE_DIR, "token.scopes"), "utf8"), / for mesh-tools\n$/);
|
||||
});
|
||||
|
||||
test("a tools' account someone made an administrator is made a plain account again at the next mint", async () => {
|
||||
const env = await environment();
|
||||
forge.users.get(TOOLS_ACCOUNT)!.admin = true;
|
||||
await fresh(env).listRepos();
|
||||
assert.equal(forge.users.get(TOOLS_ACCOUNT)!.admin, false);
|
||||
});
|
||||
|
||||
test("the tools never act as the administrator: MESH_GITEA_TOOLS_USER naming it is refused", async () => {
|
||||
const env = await environment();
|
||||
assert.throws(() => MintedToken.fromEnv(forge.url, { ...env, MESH_GITEA_TOOLS_USER: ADMIN }), /names the admin account/);
|
||||
});
|
||||
|
||||
test("the manifest hands the tools their account, and the merge-ready authors name it rather than the administrator", async () => {
|
||||
const manifest = JSON.parse(await readFile(new URL("../module.json", import.meta.url), "utf8"));
|
||||
const code = manifest.build.artifacts.find((a: any) => a.name === "code");
|
||||
assert.equal(code.env.MESH_GITEA_TOOLS_USER, TOOLS_ACCOUNT);
|
||||
const authors = String(manifest.config["ready-authors"].default).split(/\s+/);
|
||||
assert.ok(authors.includes(TOOLS_ACCOUNT));
|
||||
// Kept for now (review of issue 469): a pull request the admin account marked ready before the tools acted as
|
||||
// their own keeps counting as ready after the delivery.
|
||||
assert.ok(authors.includes(ADMIN), "marks made by the admin account before ADR 0365 keep counting");
|
||||
// The Go forge bundle reads the token the tools' account holds, never the administrator's password.
|
||||
const forgeBundle = manifest.build.artifacts.find((a: any) => a.name === "forge");
|
||||
assert.ok(forgeBundle.env.MESH_GITEA_TOKEN_FILE);
|
||||
assert.equal(forgeBundle.env.MESH_GITEA_ADMIN_PASSWORD_FILE, undefined);
|
||||
});
|
||||
|
||||
// Review of issue 469: what reads the rules' list is moved to the branch, and nothing leaves the tools without a token.
|
||||
|
||||
test("a repository check said as a warning on a head without merge-check.sh reads the base's protection from the branch", async () => {
|
||||
const client = fresh(await environment());
|
||||
const forgeOf = { holdsFile: async () => false, appliedProtection: client.appliedProtection.bind(client) };
|
||||
const c = { owner: "novox", repo: "hq", commit: "abc123def4567890abc123def4567890abc12345", "repo-check": { verdict: "warning" } } as any;
|
||||
// main requires mesh/repo-check: required, so a head that defines no check fails it for a person to see.
|
||||
assert.deepEqual(await repoCheckFacts(forgeOf, c, "main"), { defined: false, required: true });
|
||||
// A base nothing guards: not required, so the head's repository check passes rather than fails.
|
||||
assert.deepEqual(await repoCheckFacts(forgeOf, c, "feat/unguarded"), { defined: false, required: false });
|
||||
});
|
||||
|
||||
test("a mint that fails leaves the administrator's old token in place: the tools are never left without one", async () => {
|
||||
const env = await environment();
|
||||
const old = { id: 901, name: "mesh-tools", sha1: "sha-admin-kept", user: ADMIN, scopes: ["read:user"] };
|
||||
forge.tokens.push(old);
|
||||
forge.faults.mint = true;
|
||||
try {
|
||||
await assert.rejects(fresh(env).listRepos(), /500/);
|
||||
} finally {
|
||||
forge.faults.mint = false;
|
||||
}
|
||||
assert.ok(forge.tokens.some((t) => t.sha1 === old.sha1), "the admin's token was deleted before a new one was kept");
|
||||
forge.tokens.splice(forge.tokens.indexOf(old), 1);
|
||||
});
|
||||
|
||||
test("an old token that cannot be deleted after the mint is said, the new token kept, and deleted at the next start", async () => {
|
||||
const env = await environment();
|
||||
const old = { id: 902, name: "mesh-tools", sha1: "sha-admin-stuck", user: ADMIN, scopes: ["read:user"] };
|
||||
forge.tokens.push(old);
|
||||
forge.faults.deleteToken = true;
|
||||
try {
|
||||
await fresh(env).listRepos();
|
||||
} finally {
|
||||
forge.faults.deleteToken = false;
|
||||
}
|
||||
const kept = (await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8")).trim();
|
||||
assert.equal(forge.tokens.find((t) => t.sha1 === kept)?.user, TOOLS_ACCOUNT, "the account's token is kept and works");
|
||||
assert.ok(forge.tokens.some((t) => t.sha1 === old.sha1));
|
||||
// The next process reuses the kept token and retires the old one it could not before.
|
||||
const next = new MintedToken({ url: forge.url, admin: ADMIN, passwordFile: env.MESH_GITEA_ADMIN_PASSWORD_FILE!,
|
||||
file: join(env.MESH_GITEA_STATE_DIR, "token"), account: TOOLS_ACCOUNT, log: () => {} });
|
||||
assert.equal(await next.current(), kept);
|
||||
assert.ok(!forge.tokens.some((t) => t.sha1 === old.sha1), "the admin's old token was deleted at the next start");
|
||||
});
|
||||
|
||||
test("a team another process made between the look and the create is found, not a failure", async () => {
|
||||
forge.teams.splice(0);
|
||||
forge.faults.teamRace = true;
|
||||
try {
|
||||
await fresh(await environment()).listRepos();
|
||||
} finally {
|
||||
forge.faults.teamRace = false;
|
||||
}
|
||||
const team = forge.teams.find((t) => t.name === "mesh-tools");
|
||||
assert.ok(team?.members.has(TOOLS_ACCOUNT));
|
||||
});
|
||||
@@ -173,3 +173,29 @@ test("a verdict that landed out of date keeps its account on the pull request",
|
||||
const bare = lateVerdictComment({ commit: HEAD, trunk: TRUNK.slice(0, 12), verdict: "pass" }, null, words);
|
||||
assert.match(bare, /head dfb1415aaaaa merged into trunk a2c2c35bbbbb, pass/);
|
||||
});
|
||||
|
||||
test("a release branch's move puts every open pull request into it out of date, as a trunk's does (ADR 0358 §5)", async () => {
|
||||
// novox/hq ADR 0358 §5: for a pull request into a release branch, the release branch takes the trunk's place in ADR
|
||||
// 0323 — the check merges the head onto the release branch's head and names it as the trunk commit, so the release
|
||||
// branch's move puts its open verdicts out of date, and leaves those into main alone.
|
||||
const release = "feat/57-config-and-provisioning";
|
||||
const heads: Record<string, string> = { main: TRUNK, [release]: "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee" };
|
||||
const pulls = [{ number: 315, base: release, head: "3".repeat(40) }, { number: 1, base: "main", head: "1".repeat(40) }];
|
||||
const { forge, statuses } = aForge(pulls, heads);
|
||||
const gates = new GateWriter(forge);
|
||||
const trunks: Record<string, string> = {};
|
||||
await weighTrunks(forge, gates, trunks, "novox", "mesh-controller");
|
||||
|
||||
heads[release] = MOVED;
|
||||
assert.equal(await weighTrunks(forge, gates, trunks, "novox", "mesh-controller"), true);
|
||||
const s = statuses.get("3".repeat(40))!;
|
||||
assert.equal(s["mesh/merge-gate"].state, "pending");
|
||||
assert.match(s["mesh/merge-gate"].description!, /judged against trunk eeeeeeeeeeee, the trunk is now d28ca95ccccc/);
|
||||
assert.equal(s["mesh/repo-check"].state, "pending");
|
||||
assert.equal(statuses.get("1".repeat(40))!["mesh/merge-gate"].state, "success", "a pull request into main is left");
|
||||
assert.equal(trunks[`novox/mesh-controller:${release}`], MOVED);
|
||||
|
||||
// The merge tool's rule reads the same: a verdict judged against the release branch's old head is not current.
|
||||
assert.equal(verdictAtTrunk({ "mesh/merge-gate": { state: "success",
|
||||
description: "head x merged into trunk eeeeeeeeeeee — pass" } }, MOVED).current, false);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,335 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "node:test";
|
||||
import { createHmac } from "node:crypto";
|
||||
import type { AddressInfo } from "node:net";
|
||||
|
||||
// **The forge says every event on the bus** (novox/hq issue 729): a signed delivery is published once and acted on,
|
||||
// an unsigned or unverified one is refused, and an event and the poll seeing the same head announce it once.
|
||||
|
||||
const secret = "s".repeat(64);
|
||||
const sign = (body: string, key = secret) => createHmac("sha256", key).update(body).digest("hex");
|
||||
const opened = JSON.stringify({ action: "opened", number: 7, pull_request: { number: 7, title: "t", user: { login: "jo" },
|
||||
head: { ref: "feat/x", sha: "a".repeat(40) }, base: { ref: "main" }, merged: false },
|
||||
repository: { name: "hq", full_name: "novox/hq", owner: { login: "novox" } }, sender: { login: "jo" } });
|
||||
|
||||
async function serve(published: { name: string }[], acted: { name: string }[]) {
|
||||
const { receiver, HOOK_PATH } = await import("../webhook.ts");
|
||||
const server = receiver({ secret, port: 0, log: () => {},
|
||||
publish: async (e) => { published.push(e); }, act: async (e) => { acted.push(e); } });
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}${HOOK_PATH}`;
|
||||
return { server, url };
|
||||
}
|
||||
|
||||
test("a signed event is published on the bus and acted on once", async () => {
|
||||
const published: { name: string }[] = [], acted: { name: string }[] = [];
|
||||
const { server, url } = await serve(published, acted);
|
||||
try {
|
||||
const headers = { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened), "X-Gitea-Delivery": "d-1" };
|
||||
const res = await fetch(url, { method: "POST", body: opened, headers });
|
||||
assert.equal(res.status, 200);
|
||||
// The same delivery sent again is answered and published nowhere.
|
||||
const again = await fetch(url, { method: "POST", body: opened, headers });
|
||||
assert.equal(again.status, 200);
|
||||
await server.idle();
|
||||
assert.deepEqual(published.map((e) => e.name), ["forge.pull-request.opened"]);
|
||||
assert.deepEqual(acted.map((e) => e.name), ["forge.pull-request.opened"]);
|
||||
} finally { server.close(); }
|
||||
});
|
||||
|
||||
test("an unsigned event, or one whose signature does not verify, is refused and published nowhere", async () => {
|
||||
const published: { name: string }[] = [], acted: { name: string }[] = [];
|
||||
const { server, url } = await serve(published, acted);
|
||||
try {
|
||||
const unsigned = await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request" } });
|
||||
assert.equal(unsigned.status, 401);
|
||||
assert.match(await unsigned.text(), /unsigned/);
|
||||
const forged = await fetch(url, { method: "POST", body: opened,
|
||||
headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened, "another secret") } });
|
||||
assert.equal(forged.status, 401);
|
||||
assert.match(await forged.text(), /does not verify/);
|
||||
const big = "x".repeat(5 * 1024 * 1024 + 1);
|
||||
const large = await fetch(url, { method: "POST", body: big,
|
||||
headers: { "X-Gitea-Event": "push", "X-Gitea-Signature": sign(big) } });
|
||||
assert.equal(large.status, 413);
|
||||
await server.idle();
|
||||
assert.deepEqual(published, []);
|
||||
assert.deepEqual(acted, []);
|
||||
} finally { server.close(); }
|
||||
});
|
||||
|
||||
test("an event and the poll seeing the same head announce it once", async () => {
|
||||
const { claimHead } = await import("../webhook.ts");
|
||||
const heads: Record<string, string> = {};
|
||||
const sha = "a".repeat(40);
|
||||
assert.equal(claimHead(heads, "novox/hq#7", sha), true, "the event claims the head and announces it");
|
||||
assert.equal(claimHead(heads, "novox/hq#7", sha), false, "the poll finds it taken");
|
||||
assert.equal(claimHead(heads, "novox/hq#7", "b".repeat(40)), true, "a new head is announced again");
|
||||
});
|
||||
|
||||
test("each forge event is named in the module's stream and trimmed", async () => {
|
||||
const { eventOf } = await import("../webhook.ts");
|
||||
const merged = eventOf("pull_request", { action: "closed", pull_request: { number: 3, merged: true, merge_commit_sha: "c".repeat(40),
|
||||
head: { sha: "a".repeat(40) }, base: { ref: "main" } }, repository: { name: "hq", owner: { login: "novox" } } });
|
||||
assert.equal(merged.name, "forge.pull-request.merged");
|
||||
assert.equal(merged.body.merge_commit_sha, "c".repeat(40));
|
||||
assert.equal(eventOf("pull_request", { action: "label_updated", pull_request: { number: 3 }, label: { name: "mesh/ready" } }).name,
|
||||
"forge.pull-request.labeled");
|
||||
const comment = eventOf("issue_comment", { action: "created", is_pull: true, issue: { number: 3 },
|
||||
comment: { id: 9, body: "ready at abc", user: { login: "jo" } } });
|
||||
assert.equal(comment.name, "forge.pull-request.commented");
|
||||
assert.equal(comment.body.comment, "ready at abc");
|
||||
assert.equal(eventOf("push", { ref: "refs/heads/main", after: "d".repeat(40) }).name, "forge.push");
|
||||
assert.equal(eventOf("pull_request_review_approved", {}).name, "forge.other");
|
||||
});
|
||||
|
||||
// A forge's admin hooks as 1.27.3 keeps them: POST /admin/hooks makes a system webhook only when config.is_system_webhook
|
||||
// is "true", and a default webhook otherwise; the listing names one kind at a time (type=system, the default, or default).
|
||||
// Repositories hold their own webhooks (a default webhook's copies), listed a page at a time.
|
||||
function fakeForge(start: { id: number; url: string; system: boolean }[] = [],
|
||||
repoHooks: Record<string, { id: number; url: string }[]> = {}, repoCount = 0) {
|
||||
const hooks = start.map((h) => ({ ...h, events: ["push"], active: true }));
|
||||
const repos = [...Object.keys(repoHooks)];
|
||||
for (let i = 0; repos.length < repoCount; i++) repos.push(`novox/r${i}`);
|
||||
let next = 100;
|
||||
const calls: string[] = [];
|
||||
const made: any[] = [];
|
||||
const page = (items: any[], q: URLSearchParams) => {
|
||||
const p = Number(q.get("page") ?? 1), l = Number(q.get("limit") ?? 50);
|
||||
return items.slice((p - 1) * l, p * l);
|
||||
};
|
||||
const admin = { call: async (method: string, path: string, body?: any) => {
|
||||
calls.push(`${method} ${path}`);
|
||||
const u = new URL(path, "http://f");
|
||||
const repoHook = u.pathname.match(/^\/repos\/([^/]+\/[^/]+)\/hooks(?:\/(\d+))?$/);
|
||||
if (method === "GET" && u.pathname === "/repos/search") {
|
||||
return { status: 200, body: { ok: true, data: page(repos.map((full_name) => ({ full_name })), u.searchParams) } };
|
||||
}
|
||||
if (repoHook) {
|
||||
const list = repoHooks[repoHook[1]] ?? [];
|
||||
if (method === "GET") return { status: 200, body: page(list.map((h) => ({ id: h.id, config: { url: h.url } })), u.searchParams) };
|
||||
const i = list.findIndex((h) => h.id === Number(repoHook[2]));
|
||||
if (i < 0) return { status: 404, body: {} };
|
||||
list.splice(i, 1);
|
||||
return { status: 204, body: null };
|
||||
}
|
||||
if (method === "GET") {
|
||||
const type = u.searchParams.get("type") ?? "system";
|
||||
const listed = hooks.filter((h) => type === "all" || h.system === (type === "system"));
|
||||
return { status: 200, body: listed.map((h) => ({ id: h.id, active: h.active, events: h.events,
|
||||
config: { url: h.url, content_type: "json" } })) };
|
||||
}
|
||||
if (method === "POST") {
|
||||
made.push(body);
|
||||
hooks.push({ id: next++, url: body.config.url, system: body.config.is_system_webhook === "true", events: body.events,
|
||||
active: body.active });
|
||||
return { status: 201, body: {} };
|
||||
}
|
||||
if (method === "DELETE") {
|
||||
const id = Number(path.split("/").pop());
|
||||
const i = hooks.findIndex((h) => h.id === id);
|
||||
if (i < 0) return { status: 404, body: {} };
|
||||
hooks.splice(i, 1);
|
||||
return { status: 204, body: null };
|
||||
}
|
||||
return { status: 405, body: {} };
|
||||
} };
|
||||
return { admin, hooks, calls, made, repoHooks };
|
||||
}
|
||||
|
||||
test("the webhook is made a system webhook, which the forge sends every repository's events to", async () => {
|
||||
const { ensureSystemHook } = await import("../webhook.ts");
|
||||
const forge = fakeForge();
|
||||
const url = "http://h:3047/forge-event";
|
||||
const hook = await ensureSystemHook(forge.admin, url, secret);
|
||||
assert.equal(forge.made.length, 1);
|
||||
assert.equal(forge.made[0].config.is_system_webhook, "true", "without it the forge makes a default webhook, which fires for no repository");
|
||||
assert.equal(forge.made[0].config.secret, secret);
|
||||
assert.ok(forge.made[0].events.includes("pull_request") && forge.made[0].events.includes("push"));
|
||||
assert.deepEqual(forge.hooks.map((h) => h.system), [true]);
|
||||
assert.equal(hook.kind, "system");
|
||||
assert.equal(hook.did, "created");
|
||||
assert.equal(hook.url, url);
|
||||
assert.ok(hook.events.includes("pull_request"));
|
||||
});
|
||||
|
||||
test("default webhooks left at the receiver by an earlier start are replaced by one system webhook", async () => {
|
||||
const { ensureSystemHook } = await import("../webhook.ts");
|
||||
const url = "http://h:3047/forge-event";
|
||||
// What the start at 23:05 left: a default webhook, invisible to the system listing, made again by every start.
|
||||
const forge = fakeForge([{ id: 4, url, system: false }, { id: 5, url, system: false }, { id: 6, url: "http://other/x", system: true }]);
|
||||
const hook = await ensureSystemHook(forge.admin, url, secret);
|
||||
assert.deepEqual(hook.removed, { system: 0, default: 2, repository: 0, repositories: [] });
|
||||
assert.equal(hook.did, "replaced");
|
||||
assert.deepEqual(forge.hooks.map((h) => [h.url, h.system]), [["http://other/x", true], [url, true]], "another url is kept");
|
||||
// A second start replaces the system webhook, so the secret kept now is the one the forge signs with (its edit never
|
||||
// changes a secret).
|
||||
const again = await ensureSystemHook(forge.admin, url, secret);
|
||||
assert.deepEqual(again.removed, { system: 1, default: 0, repository: 0, repositories: [] });
|
||||
assert.equal(forge.hooks.filter((h) => h.url === url).length, 1);
|
||||
assert.ok(forge.calls.includes("DELETE /admin/hooks/4") && forge.calls.includes("DELETE /admin/hooks/5"));
|
||||
// The new one is made before any old one is deleted, so no event falls in a gap.
|
||||
const post = forge.calls.indexOf("POST /admin/hooks");
|
||||
assert.ok(post >= 0 && forge.calls.findIndex((c) => c.startsWith("DELETE")) > post);
|
||||
});
|
||||
|
||||
test("a repository's copy of a default webhook at the receiver is deleted, on every page, and said", async () => {
|
||||
const { ensureSystemHook } = await import("../webhook.ts");
|
||||
const url = "http://h:3047/forge-event";
|
||||
// A repository made after the default webhook got its copy, which would send its events a second time; 120
|
||||
// repositories, so the copies sit beyond the first page.
|
||||
const repoHooks: Record<string, { id: number; url: string }[]> = {};
|
||||
for (let i = 0; i < 118; i++) repoHooks[`novox/r${i}`] = [];
|
||||
repoHooks["novox/late"] = [{ id: 31, url }, { id: 32, url: "http://ci/hook" }];
|
||||
repoHooks["novox/later"] = [{ id: 33, url }];
|
||||
const forge = fakeForge([{ id: 4, url, system: false }], repoHooks);
|
||||
const hook = await ensureSystemHook(forge.admin, url, secret);
|
||||
assert.deepEqual(hook.removed, { system: 0, default: 1, repository: 2, repositories: ["novox/late", "novox/later"] });
|
||||
assert.deepEqual(forge.repoHooks["novox/late"].map((h) => h.id), [32], "a repository's other webhook is kept");
|
||||
assert.deepEqual(forge.repoHooks["novox/later"], []);
|
||||
assert.ok(forge.calls.some((c) => /\/repos\/search\?.*page=3/.test(c)), "every page of repositories is read");
|
||||
});
|
||||
|
||||
test("a forge that makes no system webhook is said, never taken as heard", async () => {
|
||||
const { ensureSystemHook } = await import("../webhook.ts");
|
||||
const forge = fakeForge();
|
||||
const admin = { call: async (method: string, path: string, body?: any) => {
|
||||
// A forge that ignores the flag: what 23:05 met, had the module read back.
|
||||
if (method === "POST") body = { ...body, config: { ...body.config, is_system_webhook: undefined } };
|
||||
return forge.admin.call(method, path, body);
|
||||
} };
|
||||
await assert.rejects(ensureSystemHook(admin, "http://h:3047/forge-event", secret), /names 0 new system webhooks/);
|
||||
});
|
||||
|
||||
test("the receiver tells what it verified and what it refused", async () => {
|
||||
const { receiver, HOOK_PATH } = await import("../webhook.ts");
|
||||
const heard: string[] = [], refused: string[] = [];
|
||||
const server = receiver({ secret, port: 0, log: () => {}, publish: async () => {},
|
||||
heard: (kind, delivery) => heard.push(`${kind} ${delivery}`), refused: (reason) => refused.push(reason) });
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}${HOOK_PATH}`;
|
||||
try {
|
||||
await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened), "X-Gitea-Delivery": "d-9" } });
|
||||
await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened, "x") } });
|
||||
await server.idle();
|
||||
assert.deepEqual(heard, ["pull_request d-9"]);
|
||||
assert.deepEqual(refused, ["its signature does not verify"]);
|
||||
} finally { server.close(); }
|
||||
});
|
||||
|
||||
test("a forge that never delivers is said loudly once, and unhealthy, when the poll finds what no event said", async () => {
|
||||
const { Pace } = await import("../webhook.ts");
|
||||
let now = 1_000_000;
|
||||
const said: string[] = [];
|
||||
const pace = new Pace(() => now, (l) => said.push(l));
|
||||
const hook = { kind: "system" as const, id: 7, url: "http://h:3047/forge-event", events: ["push"], active: true,
|
||||
did: "created" as const, removed: { system: 0, default: 0, repository: 0, repositories: [] } };
|
||||
pace.listening("172.17.0.1", 3047, hook.url, hook);
|
||||
now += 5 * 60_000;
|
||||
pace.miss("novox/hq#7 at aaaaaaaa");
|
||||
assert.equal(said.length, 0, "five minutes after the start is not yet silence");
|
||||
assert.equal((pace.report() as any).healthy, true);
|
||||
pace.refused("it is unsigned", "172.17.0.3");
|
||||
assert.equal((pace.report() as any).healthy, false, "refused, and none verified");
|
||||
now += 6 * 60_000;
|
||||
pace.miss("the merge of novox/hq#7");
|
||||
pace.miss("novox/hq#8 at bbbbbbbb");
|
||||
assert.equal(said.filter((l) => /NOT HEARD: no delivery ever verified/.test(l)).length, 1, "said once");
|
||||
let r = pace.report() as any;
|
||||
assert.equal(r.healthy, false);
|
||||
assert.equal(r.state, "unhealthy");
|
||||
assert.match(r.why, /never delivered/);
|
||||
assert.equal(r.polls.found_unsaid, 3);
|
||||
assert.equal(r.refused.count, 1);
|
||||
assert.equal(r.refused.last.reason, "it is unsigned");
|
||||
assert.equal(r.hook.kind, "system");
|
||||
assert.equal(r.receiver.port, 3047);
|
||||
// The first delivery clears it.
|
||||
now += 1000;
|
||||
pace.heard("push", "d-1");
|
||||
r = pace.report() as any;
|
||||
assert.equal(r.healthy, true);
|
||||
assert.equal(r.verified.count, 1);
|
||||
assert.equal(r.verified.last.event, "push");
|
||||
assert.equal(r.polls.every_seconds, 300);
|
||||
assert.match(said.join("\n"), /heard again/);
|
||||
});
|
||||
|
||||
test("a forge whose events stop after one delivery is said loudly once per silent spell, and refusals are unhealthy", async () => {
|
||||
const { Pace } = await import("../webhook.ts");
|
||||
let now = 1_000_000;
|
||||
const said: string[] = [];
|
||||
const pace = new Pace(() => now, (l) => said.push(l));
|
||||
const hook = { kind: "system" as const, id: 7, url: "http://h:3047/forge-event", events: ["push"], active: true,
|
||||
did: "created" as const, removed: { system: 0, default: 0, repository: 0, repositories: [] } };
|
||||
pace.listening("172.17.0.1", 3047, hook.url, hook);
|
||||
now += 60_000;
|
||||
pace.heard("push", "d-1");
|
||||
// The hook is deleted by hand: nothing more comes, and the poll finds what no event said.
|
||||
now += 11 * 60_000;
|
||||
pace.miss("novox/hq#7 at aaaaaaaa");
|
||||
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 0, "one find alone may be the poll beating an event");
|
||||
pace.miss("the merge of novox/hq#7");
|
||||
pace.miss("novox/hq#8 at bbbbbbbb");
|
||||
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 1, "said once");
|
||||
assert.match(said.find((l) => /NOT HEARD/.test(l)) ?? "", /since the last at/);
|
||||
assert.equal((pace.report() as any).healthy, false);
|
||||
// A delivery ends the spell; the next spell is said again.
|
||||
now += 1000;
|
||||
pace.heard("push", "d-2");
|
||||
assert.equal((pace.report() as any).healthy, true);
|
||||
now += 11 * 60_000;
|
||||
pace.miss("novox/hq#9 at cccccccc");
|
||||
pace.miss("novox/hq#10 at dddddddd");
|
||||
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 2, "a second silent spell is said again");
|
||||
// Every delivery refused after a verified one (a secret that no longer matches) is unhealthy at once.
|
||||
now += 1000;
|
||||
pace.heard("push", "d-3");
|
||||
now += 1000;
|
||||
pace.refused("its signature does not verify", "172.17.0.3");
|
||||
pace.refused("its signature does not verify", "172.17.0.3");
|
||||
const r = pace.report() as any;
|
||||
assert.equal(r.healthy, false);
|
||||
assert.equal(r.refused.since_verified, 2);
|
||||
assert.match(r.why, /2 deliveries refused since the last verified one.*does not verify/);
|
||||
now += 1000;
|
||||
pace.heard("push", "d-4");
|
||||
assert.equal((pace.report() as any).healthy, true, "a verified delivery clears it");
|
||||
});
|
||||
|
||||
test("without a receiver by configuration the polls alone watch, healthy; a receiver that failed is unhealthy", async () => {
|
||||
const { Pace } = await import("../webhook.ts");
|
||||
let now = 1_000_000;
|
||||
const said: string[] = [];
|
||||
const quiet = new Pace(() => now, (l) => said.push(l));
|
||||
quiet.unheard("MESH_GITEA_HOOK_URL unset", false);
|
||||
now += 20 * 60_000;
|
||||
quiet.miss("novox/hq#7 at aaaaaaaa");
|
||||
assert.equal(said.length, 0, "no deliveries were expected");
|
||||
assert.equal((quiet.report() as any).healthy, true);
|
||||
const failed = new Pace(() => now, () => {});
|
||||
failed.unheard("the forge's system webhook could not be made: 403");
|
||||
const r = failed.report() as any;
|
||||
assert.equal(r.healthy, false);
|
||||
assert.match(r.why, /403/);
|
||||
});
|
||||
|
||||
test("the polls look every 30 s until events are heard, every five minutes while they are, and 30 s again on a miss", async () => {
|
||||
const { Pace } = await import("../webhook.ts");
|
||||
let now = 1_000_000;
|
||||
const said: string[] = [];
|
||||
const pace = new Pace(() => now, (l) => said.push(l));
|
||||
assert.equal(pace.every(), 30_000, "no delivery yet");
|
||||
pace.heard();
|
||||
assert.equal(pace.every(), 300_000, "a delivery heard");
|
||||
now += 11 * 60_000;
|
||||
assert.equal(pace.every(), 30_000, "ten minutes without a delivery");
|
||||
pace.heard();
|
||||
now += 1000;
|
||||
pace.miss("novox/hq#7 at aaaaaaaa");
|
||||
assert.equal(pace.every(), 30_000, "the poll found what no event said");
|
||||
assert.match(said.join("\n"), /MISSED novox\/hq#7/);
|
||||
now += 1000;
|
||||
pace.heard();
|
||||
assert.equal(pace.every(), 300_000, "the next delivery");
|
||||
});
|
||||
+216
-30
@@ -12,7 +12,20 @@
|
||||
// hands a token's value out exactly once, so a token not kept is a token lost;
|
||||
// - re-minted when the forge rejects it (401) or the kept file is gone. The one case that is not
|
||||
// a fault: the forge's data was restored from a predecessor and the token the file names never
|
||||
// existed there.
|
||||
// existed there;
|
||||
// - minted by one process at a time. The runtime starts each of the bundle's entrypoints as a process
|
||||
// of its own (novox/hq ADR 0193): the watcher (index.js) and the tools (tools/index.js) are two
|
||||
// processes over one kept file. Two that minted at once left the forge two tokens by one name, which
|
||||
// gitea then refuses to delete by name — for ever, and every forge tool with it (the 2026-10-14
|
||||
// outage). So a mint holds a lock file beside the kept token, and reads the kept file again once it
|
||||
// holds it: what another process minted meanwhile is reused, not minted over;
|
||||
// - replaced by id: an old token by this name is found in the account's list and deleted by its id,
|
||||
// every one of them, so a name held twice is cleared rather than refused.
|
||||
//
|
||||
// - minted for the tools' account, not the administrator (novox/hq ADR 0365 §2, issue 469): the administrator's
|
||||
// login mints it for `mesh-tools` (tools-account.ts), after making that account and its team, and deletes any
|
||||
// token by this name the administrator itself still holds — the token the tools used before. A token kept
|
||||
// for another account than the one asked (the scopes line says whose) is minted again at the next start.
|
||||
//
|
||||
// An explicitly configured token still wins, and is never minted over: if it is rejected, that is
|
||||
// reported, not repaired — somebody chose it.
|
||||
@@ -20,8 +33,10 @@
|
||||
// The token is never logged. Lines say that one was minted, reused or renewed, and where it is
|
||||
// kept; never what it is.
|
||||
|
||||
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
||||
import { chmodSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, renameSync, rmSync, statSync, unlinkSync, writeFileSync, writeSync } from "node:fs";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { dirname, join } from "node:path";
|
||||
import { setUpToolsAccount, TOOLS_ACCOUNT, type AdminCall } from "./tools-account.js";
|
||||
|
||||
/** The name the token carries in the forge's own list — one per mesh runtime, found by name. */
|
||||
export const TOKEN_NAME = "mesh-tools";
|
||||
@@ -39,7 +54,7 @@ export const TOKEN_NAME = "mesh-tools";
|
||||
* `gitea_api` (novox/hq 229); whoever may call a tool includes agents, and the forge's administration is the
|
||||
* operator's. The provisioner settles users with the admin account's own login (GiteaAdmin), never this token.
|
||||
* A token kept with a different list is minted again with this one at the next start (the scopes are kept
|
||||
* beside it), and the forge drops the old one by name — so the scope a kept token had does not outlive this.
|
||||
* beside it), and the old one is deleted from the forge by its id — so the scope a kept token had does not outlive this.
|
||||
*
|
||||
* A token kept from before a scope was added lacks it: the forge answers such a call with
|
||||
* `403 token does not have at least one of required scope(s)`, and the client treats that like a
|
||||
@@ -95,6 +110,14 @@ export interface MintedTokenOptions {
|
||||
readonly file: string;
|
||||
readonly name?: string;
|
||||
readonly scopes?: readonly string[];
|
||||
/**
|
||||
* The account the token is minted for: the tools' own (novox/hq ADR 0365 §2). The admin login mints it, after
|
||||
* `setUp` made the account, and the admin's own tokens by this name are deleted. Left out, the token is the
|
||||
* admin's own, as it was before.
|
||||
*/
|
||||
readonly account?: string;
|
||||
/** Run with the admin's password before each mint for `account`: makes the account and its team (tools-account.ts). */
|
||||
readonly setUp?: (password: string) => Promise<void>;
|
||||
readonly log?: (line: string) => void;
|
||||
readonly fetch?: typeof fetch;
|
||||
}
|
||||
@@ -106,12 +129,14 @@ export class MintedToken implements TokenSource {
|
||||
private inflight: Promise<string> | null = null;
|
||||
private readonly name: string;
|
||||
private readonly scopes: readonly string[];
|
||||
private readonly account: string;
|
||||
private readonly log: (line: string) => void;
|
||||
private readonly fetchImpl: typeof fetch;
|
||||
|
||||
constructor(private readonly opts: MintedTokenOptions) {
|
||||
this.name = opts.name ?? TOKEN_NAME;
|
||||
this.scopes = opts.scopes ?? TOKEN_SCOPES;
|
||||
this.account = opts.account ?? opts.admin;
|
||||
this.log = opts.log ?? ((line) => console.log(`[gitea] ${line}`));
|
||||
this.fetchImpl = opts.fetch ?? fetch;
|
||||
}
|
||||
@@ -124,9 +149,9 @@ export class MintedToken implements TokenSource {
|
||||
*/
|
||||
static fromEnv(url: string, env: NodeJS.ProcessEnv = process.env): MintedToken {
|
||||
const opts = MintedToken.optionsFromEnv(url, env);
|
||||
// One source per kept file in a process. The watcher and the tools entrypoint both build a
|
||||
// client in the same runtime; two sources over one file would each renew on a 401 and drop the
|
||||
// other's token by name, forever. Shared, a renewal is one renewal.
|
||||
// One source per kept file in a process: two sources in one process would each renew on a 401.
|
||||
// This covers one process only. The watcher and the tools are two processes (the runtime starts
|
||||
// each entrypoint as its own), and those are kept apart by the mint's lock file (mint, below).
|
||||
const shared = MintedToken.shared.get(opts.file);
|
||||
if (shared) return shared;
|
||||
const source = new MintedToken(opts);
|
||||
@@ -148,7 +173,17 @@ export class MintedToken implements TokenSource {
|
||||
if (missing.length) {
|
||||
throw new Error(`no Gitea token, and nothing to mint one with — set MESH_GITEA_TOKEN, or ${missing.join(", ")}`);
|
||||
}
|
||||
return { url, admin: admin!, passwordFile: passwordFile!, file: join(stateDir!, "token") };
|
||||
// The tools' own account, never the administrator (novox/hq ADR 0365 §2): MESH_GITEA_TOOLS_USER names it.
|
||||
const account = env.MESH_GITEA_TOOLS_USER || TOOLS_ACCOUNT;
|
||||
if (account === admin) throw new Error(`MESH_GITEA_TOOLS_USER names the admin account "${admin}"; the tools act as an account of their own (novox/hq ADR 0365)`);
|
||||
return {
|
||||
url, admin: admin!, passwordFile: passwordFile!, file: join(stateDir!, "token"), account,
|
||||
// GiteaAdmin is loaded when first used: client.ts imports this file.
|
||||
setUp: async (password) => {
|
||||
const { GiteaAdmin } = await import("./client.js");
|
||||
await setUpToolsAccount(new GiteaAdmin(url, admin!, password), account, () => randomBytes(24).toString("base64url"));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async current(): Promise<string> {
|
||||
@@ -159,11 +194,13 @@ export class MintedToken implements TokenSource {
|
||||
if (kept !== null && this.keptScopes() === this.scopesLine()) {
|
||||
this.held = kept;
|
||||
this.log(`reusing the token kept at ${this.opts.file}`);
|
||||
// A retirement of the admin's old token that failed after the mint is tried again, once a process.
|
||||
if (existsSync(this.retireFile)) await this.locked(() => this.retireAdmin()).catch((err) => this.log(`${err}`));
|
||||
return kept;
|
||||
}
|
||||
if (kept !== null) return this.mint("the kept token was minted with other scopes than the tools take now — minting one with them");
|
||||
if (kept !== null) return this.mint("the kept token was minted with other scopes than the tools take now — minting one with them", kept);
|
||||
}
|
||||
return this.mint("no token kept — minting one");
|
||||
return this.mint("no token kept — minting one", null);
|
||||
}
|
||||
|
||||
async renew(rejected: string): Promise<string> {
|
||||
@@ -177,7 +214,7 @@ export class MintedToken implements TokenSource {
|
||||
return kept;
|
||||
}
|
||||
this.held = null;
|
||||
return this.mint("the forge rejected the kept token — minting a fresh one");
|
||||
return this.mint("the forge rejected the kept token — minting a fresh one", rejected);
|
||||
}
|
||||
|
||||
/** What the forge's scoped tokens say when a kept token predates a scope the tools now need. */
|
||||
@@ -185,17 +222,95 @@ export class MintedToken implements TokenSource {
|
||||
return status === 403 && /required scope/i.test(body);
|
||||
}
|
||||
|
||||
/** One mint at a time: concurrent first calls share it, rather than each minting its own. */
|
||||
private mint(why: string): Promise<string> {
|
||||
/**
|
||||
* One mint at a time: concurrent first calls in this process share it, and a mint in another process
|
||||
* on this machine is waited for (the lock file). Once the lock is held, the kept file is read again:
|
||||
* a token there that is not `stale` (the one rejected, or kept with other scopes) and carries the
|
||||
* scopes asked for was minted by the process that held the lock before — it is reused.
|
||||
*/
|
||||
private mint(why: string, stale: string | null): Promise<string> {
|
||||
if (this.inflight === null) {
|
||||
this.log(why);
|
||||
this.inflight = this.doMint().finally(() => {
|
||||
this.inflight = this.locked(async () => {
|
||||
const kept = this.read();
|
||||
if (kept !== null && kept !== stale && this.keptScopes() === this.scopesLine()) {
|
||||
this.held = kept;
|
||||
this.log(`another process minted it meanwhile — reusing the token kept at ${this.opts.file}`);
|
||||
return kept;
|
||||
}
|
||||
return this.doMint();
|
||||
}).finally(() => {
|
||||
this.inflight = null;
|
||||
});
|
||||
}
|
||||
return this.inflight;
|
||||
}
|
||||
|
||||
/** How long a mint may hold the lock before another process takes it as left behind, and how long one waits. */
|
||||
static readonly LOCK_STALE_MS = 120_000;
|
||||
static readonly LOCK_WAIT_MS = 150_000;
|
||||
|
||||
/**
|
||||
* Run `work` holding the lock file beside the kept token: created exclusively, holding this process's
|
||||
* pid, the time and a nonce; removed after, only when it is still ours. A lock whose process is gone,
|
||||
* or older than LOCK_STALE_MS, was left by a process that died mid-mint and is taken over — never
|
||||
* waited on for ever, which would be the same lock-out by another road.
|
||||
*/
|
||||
private async locked<T>(work: () => Promise<T>): Promise<T> {
|
||||
const lock = `${this.opts.file}.lock`;
|
||||
mkdirSync(dirname(this.opts.file), { recursive: true, mode: 0o700 });
|
||||
const mine = `${process.pid} ${Date.now()} ${randomBytes(8).toString("hex")}\n`;
|
||||
const deadline = Date.now() + MintedToken.LOCK_WAIT_MS;
|
||||
for (;;) {
|
||||
try {
|
||||
const fd = openSync(lock, "wx", 0o600);
|
||||
try { writeSync(fd, mine); } finally { closeSync(fd); }
|
||||
break;
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code !== "EEXIST") {
|
||||
throw new Error(`cannot take the mint lock at ${lock}: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
if (this.leftBehind(lock)) {
|
||||
this.log(`the mint lock at ${lock} was left by a process that is gone — taking it over`);
|
||||
try { unlinkSync(lock); } catch { /* another waiter took it over first */ }
|
||||
continue;
|
||||
}
|
||||
if (Date.now() > deadline) {
|
||||
throw new Error(`another process has held the mint lock at ${lock} for over ${MintedToken.LOCK_WAIT_MS / 1000}s — not minting beside it`);
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 50 + Math.floor(Math.random() * 50)));
|
||||
}
|
||||
try {
|
||||
return await work();
|
||||
} finally {
|
||||
try {
|
||||
if (readFileSync(lock, "utf8") === mine) unlinkSync(lock);
|
||||
} catch { /* already gone */ }
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether the lock's holder is gone: its pid answers no signal, or it is older than LOCK_STALE_MS. */
|
||||
private leftBehind(lock: string): boolean {
|
||||
let text: string;
|
||||
let age: number;
|
||||
try {
|
||||
text = readFileSync(lock, "utf8");
|
||||
age = Date.now() - statSync(lock).mtimeMs;
|
||||
} catch {
|
||||
return false; // released meanwhile: the next attempt takes it
|
||||
}
|
||||
if (age > MintedToken.LOCK_STALE_MS) return true;
|
||||
const pid = Number(text.split(" ")[0]);
|
||||
if (!Number.isInteger(pid) || pid <= 0) return age > 1_000; // half-written, or not ours: give it a moment
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return false;
|
||||
} catch (err) {
|
||||
return (err as NodeJS.ErrnoException).code === "ESRCH";
|
||||
}
|
||||
}
|
||||
|
||||
private read(): string | null {
|
||||
try {
|
||||
const token = readFileSync(this.opts.file, "utf8").replace(/\n$/, "");
|
||||
@@ -206,9 +321,10 @@ export class MintedToken implements TokenSource {
|
||||
}
|
||||
}
|
||||
|
||||
/** The scopes as kept beside the token: sorted, one line. */
|
||||
/** The scopes as kept beside the token: sorted, one line, and whose it is when it is not the admin's. */
|
||||
private scopesLine(): string {
|
||||
return [...this.scopes].sort().join(",");
|
||||
const scopes = [...this.scopes].sort().join(",");
|
||||
return this.account === this.opts.admin ? scopes : `${scopes} for ${this.account}`;
|
||||
}
|
||||
|
||||
/** The scopes the kept token was minted with, "" when not kept (a token kept before they were). */
|
||||
@@ -231,17 +347,20 @@ export class MintedToken implements TokenSource {
|
||||
renameSync(`${this.opts.file}.scopes.tmp`, `${this.opts.file}.scopes`);
|
||||
}
|
||||
|
||||
private async doMint(): Promise<string> {
|
||||
let password: string;
|
||||
/** One call to the forge's API with the admin login, the password read now. */
|
||||
private password(): string {
|
||||
try {
|
||||
password = readFileSync(this.opts.passwordFile, "utf8").replace(/\n$/, "");
|
||||
return readFileSync(this.opts.passwordFile, "utf8").replace(/\n$/, "");
|
||||
} catch (err) {
|
||||
throw new Error(`cannot read the admin password at ${this.opts.passwordFile}: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
private adminCall(password = this.password()): AdminCall {
|
||||
const authorization = "Basic " + Buffer.from(`${this.opts.admin}:${password}`).toString("base64");
|
||||
const tokens = `${this.opts.url.replace(/\/+$/, "")}/api/v1/users/${encodeURIComponent(this.opts.admin)}/tokens`;
|
||||
const call = async (method: string, path = "", body?: unknown): Promise<{ status: number; body: any }> => {
|
||||
const res = await this.fetchImpl(tokens + path, {
|
||||
const api = `${this.opts.url.replace(/\/+$/, "")}/api/v1`;
|
||||
const admin: AdminCall = async (method, path, body) => {
|
||||
const res = await this.fetchImpl(api + path, {
|
||||
method,
|
||||
headers: { "Content-Type": "application/json", Authorization: authorization },
|
||||
...(body === undefined ? {} : { body: JSON.stringify(body) }),
|
||||
@@ -253,30 +372,97 @@ export class MintedToken implements TokenSource {
|
||||
}
|
||||
return { status: res.status, body: parsed };
|
||||
};
|
||||
return admin;
|
||||
}
|
||||
|
||||
/** Where a retirement of the admin's old token that failed is remembered, to be tried again (retireAdmin). */
|
||||
private get retireFile(): string {
|
||||
return `${this.opts.file}.retire-admin`;
|
||||
}
|
||||
|
||||
/**
|
||||
* The admin's own token by this name goes: what the tools used before must not outlive the move (novox/hq ADR
|
||||
* 0365 §2). Run after the account's token is minted and kept, so a failure here never leaves the tools without
|
||||
* a token: it is said, remembered beside the kept token, and tried again when a process next reads that token.
|
||||
*/
|
||||
private async retireAdmin(admin: AdminCall = this.adminCall()): Promise<void> {
|
||||
if (this.account === this.opts.admin) return;
|
||||
try {
|
||||
const retired = await this.dropNamed(admin, `/users/${encodeURIComponent(this.opts.admin)}/tokens`, this.opts.admin);
|
||||
if (retired > 0) this.log(`deleted ${retired} token(s) named "${this.name}" the admin account "${this.opts.admin}" held: the tools act as "${this.account}" now`);
|
||||
rmSync(this.retireFile, { force: true });
|
||||
} catch (err) {
|
||||
writeFileSync(this.retireFile, `${new Date().toISOString()}\n`, { mode: 0o600 });
|
||||
this.log(`FAILED to delete the token(s) named "${this.name}" the admin account "${this.opts.admin}" holds — tried again at the next start: ${(err as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
private async doMint(): Promise<string> {
|
||||
const password = this.password();
|
||||
const admin = this.adminCall(password);
|
||||
const tokensOf = (account: string) => `/users/${encodeURIComponent(account)}/tokens`;
|
||||
const call = (method: string, path = "", body?: unknown) => admin(method, tokensOf(this.account) + path, body);
|
||||
|
||||
if (this.account !== this.opts.admin) {
|
||||
// The tools' account and its team first (novox/hq ADR 0365 §2): a failure here fails the mint, tried again
|
||||
// at the next call.
|
||||
const me = await admin("GET", "/user");
|
||||
if (me.status === 401 || me.status === 403) throw new AdminRefused(this.opts.admin, me.status);
|
||||
await this.opts.setUp?.(password);
|
||||
}
|
||||
|
||||
let res = await call("POST", "", { name: this.name, scopes: this.scopes });
|
||||
if (res.status === 401 || res.status === 403) throw new AdminRefused(this.opts.admin, res.status);
|
||||
if (res.status === 400 || res.status === 422) {
|
||||
// The forge still holds a token by this name whose value we no longer have — the kept file
|
||||
// went while the forge's data stayed. It is ours to replace: drop it by name and mint again.
|
||||
this.log(`the forge already holds a token named "${this.name}" — replacing it`);
|
||||
const dropped = await call("DELETE", `/${encodeURIComponent(this.name)}`);
|
||||
if (dropped.status !== 204 && dropped.status !== 404) {
|
||||
throw new Error(`Gitea DELETE /users/${this.opts.admin}/tokens/${this.name}: ${dropped.status} ${detail(dropped.body)}`);
|
||||
}
|
||||
// went while the forge's data stayed, or two processes once minted at once and left two. It is
|
||||
// ours to replace: find every one by this name in the account's list and delete each by its id
|
||||
// (gitea refuses a delete by a name held twice: "multiple matches for token name"), then mint.
|
||||
await this.dropNamed(admin, tokensOf(this.account), this.account, true);
|
||||
res = await call("POST", "", { name: this.name, scopes: this.scopes });
|
||||
}
|
||||
if (res.status !== 201 && res.status !== 200) {
|
||||
throw new Error(`Gitea POST /users/${this.opts.admin}/tokens: ${res.status} ${detail(res.body)}`);
|
||||
throw new Error(`Gitea POST /users/${this.account}/tokens: ${res.status} ${detail(res.body)}`);
|
||||
}
|
||||
const token = typeof res.body?.sha1 === "string" ? res.body.sha1 : null;
|
||||
if (!token) throw new Error(`Gitea POST /users/${this.opts.admin}/tokens: ${res.status} but no token in the reply`);
|
||||
if (!token) throw new Error(`Gitea POST /users/${this.account}/tokens: ${res.status} but no token in the reply`);
|
||||
|
||||
this.keep(token);
|
||||
this.held = token;
|
||||
this.log(`minted a token for "${this.opts.admin}" (${this.scopes.join(", ")}), kept at ${this.opts.file}`);
|
||||
this.log(`minted a token for "${this.account}" (${this.scopes.join(", ")}), kept at ${this.opts.file}`);
|
||||
// Only now, with the account's own token kept, does the admin's old one go.
|
||||
await this.retireAdmin(admin);
|
||||
return token;
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete every token by this name an account holds, each by its id (gitea refuses a delete by a name held twice:
|
||||
* "multiple matches for token name"); answers how many. `replacing` says the line for a mint that found its name
|
||||
* taken.
|
||||
*/
|
||||
private async dropNamed(admin: AdminCall, tokens: string, account: string, replacing = false): Promise<number> {
|
||||
const held: number[] = [];
|
||||
for (let page = 1; page <= 100; page++) {
|
||||
const listed = await admin("GET", `${tokens}?page=${page}&limit=50`);
|
||||
if (listed.status !== 200 || !Array.isArray(listed.body)) {
|
||||
throw new Error(`Gitea GET /users/${account}/tokens: ${listed.status} ${detail(listed.body)}`);
|
||||
}
|
||||
if (listed.body.length === 0) break;
|
||||
for (const t of listed.body) if (t?.name === this.name && typeof t.id === "number") held.push(t.id);
|
||||
}
|
||||
if (replacing) {
|
||||
this.log(held.length === 1
|
||||
? `the forge already holds a token named "${this.name}" — replacing it`
|
||||
: `the forge holds ${held.length} tokens named "${this.name}" — deleting each by its id and minting one`);
|
||||
}
|
||||
for (const id of held) {
|
||||
const dropped = await admin("DELETE", `${tokens}/${id}`);
|
||||
if (dropped.status !== 204 && dropped.status !== 404) {
|
||||
throw new Error(`Gitea DELETE /users/${account}/tokens/${id} (named ${this.name}): ${dropped.status} ${detail(dropped.body)}`);
|
||||
}
|
||||
}
|
||||
return held.length;
|
||||
}
|
||||
}
|
||||
|
||||
function detail(body: unknown): string {
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
// The forge's tools act as an account of their own, not as the site administrator (novox/hq ADR 0365 §2,
|
||||
// issue 469).
|
||||
//
|
||||
// The administrator's token reached everything the forge holds, so the tools' boundary rested on their own guards
|
||||
// (guard.ts): a protection edit, say, the forge itself would have taken. Now the tools' token is minted for a plain
|
||||
// account, `mesh-tools`, which is no administrator and writes to the mesh's repositories through one team of the
|
||||
// mesh's organisation:
|
||||
//
|
||||
// - **the account**: made by the module with the administrator's login (that login sets the forge up and does
|
||||
// nothing else), never an administrator — one found made an administrator is made a plain account again. Its
|
||||
// password is random, sent once and kept nowhere: the account is reached only by the tokens the administrator
|
||||
// mints for it (gitea lets an administrator mint for another account: `reqSelfOrAdmin`).
|
||||
// - **the team**: `mesh-tools` in the organisation `novox`, on every repository of it, writing code (pushes to a
|
||||
// branch, commit statuses, merges), issues (notes, comments, labels) and pull requests. No administration of a
|
||||
// repository, so the forge refuses the account a protection edit, a repository's settings and its deletion;
|
||||
// no repository made in the organisation, since its maker would be that repository's administrator.
|
||||
//
|
||||
// Every step is idempotent (GiteaAdmin's, client.ts), and run with the administrator's login before each mint of the tools' token, so a team
|
||||
// or a membership lost behind the module's back is made again at the next mint.
|
||||
|
||||
/** The account the tools act as. */
|
||||
export const TOOLS_ACCOUNT = "mesh-tools";
|
||||
/** The organisation whose repositories are the mesh's, and the team through which the tools' account writes there. */
|
||||
export const TOOLS_ORG = "novox";
|
||||
export const TOOLS_TEAM = "mesh-tools";
|
||||
|
||||
/** What the team grants: write on code, issues and pull requests of every repository of the organisation, and no more. */
|
||||
export const TOOLS_TEAM_UNITS = {
|
||||
permission: "write",
|
||||
units_map: { "repo.code": "write", "repo.issues": "write", "repo.pulls": "write" },
|
||||
includes_all_repositories: true,
|
||||
can_create_org_repo: false,
|
||||
} as const;
|
||||
|
||||
/** One call to the forge's API with the administrator's login: a path under /api/v1, the status and body answered. */
|
||||
export type AdminCall = (method: string, path: string, body?: unknown) => Promise<{ status: number; body: any }>;
|
||||
|
||||
/** What setting the account up needs of the forge's admin API: GiteaAdmin's own (client.ts), shared with the
|
||||
* package provisioner, so a create race (422) and a user made by hand are handled once. */
|
||||
export interface ToolsForge {
|
||||
ensureUser(username: string, password: string, email: string, plain?: boolean): Promise<void>;
|
||||
ensureTeamWith(org: string, team: string, units: Record<string, unknown>): Promise<number>;
|
||||
addUserToTeam(teamId: number, username: string): Promise<void>;
|
||||
}
|
||||
|
||||
/** Make the tools' account, plain, a member of its team on every repository of the organisation. Its password is
|
||||
* set afresh each time and kept nowhere: the account is reached only by the tokens the administrator mints. */
|
||||
export async function setUpToolsAccount(forge: ToolsForge, account = TOOLS_ACCOUNT, password: () => string): Promise<void> {
|
||||
await forge.ensureUser(account, password(), `${account}@noreply.localhost`, true);
|
||||
const team = await forge.ensureTeamWith(TOOLS_ORG, TOOLS_TEAM, { name: TOOLS_TEAM, ...TOOLS_TEAM_UNITS });
|
||||
await forge.addUserToTeam(team, account);
|
||||
}
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
|
||||
import { emit } from "@novox/mesh-sdk/events";
|
||||
import { GiteaClient, type BranchProtection } from "../client.js";
|
||||
import { GiteaAdmin, GiteaClient, readyAuthors, type BranchProtection } from "../client.js";
|
||||
import { appendNote, run, viewBody, viewComment } from "../delivery.js";
|
||||
import { containedFrom, isCommit, mergeableUnder, newestByContext } from "../checks.js";
|
||||
import {
|
||||
@@ -59,13 +59,13 @@ function namesChecked(tools: ToolDefinition[]): ToolDefinition[] {
|
||||
}));
|
||||
}
|
||||
|
||||
export function getGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
return namesChecked(allGiteaTools(gitea));
|
||||
export function getGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
|
||||
return namesChecked(allGiteaTools(gitea, admin));
|
||||
}
|
||||
|
||||
// gitea_merge_pull_request is served from the watcher's bundle (merge.ts, index.ts), beside the merge check's one
|
||||
// writer (novox/hq issue 456, ADR 0323).
|
||||
function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
|
||||
return [
|
||||
// ---- Repositories ----
|
||||
{
|
||||
@@ -401,8 +401,11 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
},
|
||||
run: async (args) => {
|
||||
const owner = String(args.owner), repo = String(args.repo);
|
||||
if (!args.branch) return { rules: (await gitea.branchProtections(owner, repo)).map(summarised) };
|
||||
const rule = await gitea.branchProtection(owner, repo, String(args.branch));
|
||||
// The one read made as the forge's administrator (novox/hq ADR 0375, issue 772): the rules' list is the
|
||||
// repository administrator's, which the tools' account is not. No other tool uses this account.
|
||||
const rules = await (admin ? admin() : gitea).branchProtections(owner, repo);
|
||||
if (!args.branch) return { rules: rules.map(summarised) };
|
||||
const rule = rules.find((p) => (p.rule_name ?? p.branch_name) === String(args.branch)) ?? null;
|
||||
return { branch: String(args.branch), rule: rule ? summarised(rule) : null };
|
||||
},
|
||||
},
|
||||
@@ -504,7 +507,7 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
base = base || "main";
|
||||
let rule: BranchProtection | null = null, unreadable: string | undefined;
|
||||
try {
|
||||
rule = await gitea.branchProtection(owner, repo, base);
|
||||
rule = await gitea.appliedProtection(owner, repo, base);
|
||||
} catch (err) {
|
||||
unreadable = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
@@ -514,9 +517,9 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
},
|
||||
{
|
||||
name: "gitea_open_pulls",
|
||||
description: "Every open pull request of every repository, from the forge itself: its head, its base, the statuses the base's protection requires and the ones its head has, each with its state, description and when it was set. What mesh-delivery reads to say a pull request the mesh was never asked to check (novox/hq issue 347), or whose check started and never answered (issue 438). Reads only.",
|
||||
description: "Every open pull request of every repository, from the forge itself: its head, its base, the statuses the base's protection requires and the ones its head has, each with its state, description and when it was set. What mesh-delivery reads to say a pull request the mesh was never asked to check (novox/hq issue 347), or whose check started and never answered (issue 438), and each pull request's labels and ready mark — the label mesh/ready and the head named by the newest comment \"ready at <full head sha>\" that a listed account (the setting ready-authors) wrote after the label was added and after the last push — which mesh-delivery merges on (issue 722). Reads only.",
|
||||
input: {},
|
||||
run: async () => ({ pulls: await gitea.openPulls() }),
|
||||
run: async () => ({ pulls: await gitea.openPulls(readyAuthors()) }),
|
||||
},
|
||||
{
|
||||
name: "gitea_contains",
|
||||
@@ -589,8 +592,12 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
const { owner, repo, branch, newBranch } = judged.contents;
|
||||
const found = await gitea.api<{ default_branch?: string }>(`/repos/${checkName("owner", owner)}/${checkName("name", repo)}`);
|
||||
const defaultBranch = String(found?.default_branch ?? "main");
|
||||
const refused = judgeContentsWrite(newBranch ?? branch ?? defaultBranch, defaultBranch,
|
||||
await gitea.branchProtections(owner, repo));
|
||||
// The protection the forge applies to the branch written, read from the branch: the rules' own list is a
|
||||
// repository administrator's, which the tools' account is not (novox/hq ADR 0365). A branch not made yet is
|
||||
// left to the forge, which refuses a write that would make a branch a rule guards.
|
||||
const target = newBranch ?? branch ?? defaultBranch;
|
||||
const applied = target === defaultBranch ? null : await gitea.appliedProtection(owner, repo, target);
|
||||
const refused = judgeContentsWrite(target, defaultBranch, applied ? [applied] : []);
|
||||
if (refused) throw new Refused(refused);
|
||||
}
|
||||
// The path judged is the path sent: normalised, with only the allow-listed query keys (guard.ts).
|
||||
@@ -611,7 +618,7 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
|
||||
// way, gitea contributes none rather than failing the whole runtime, and says why.
|
||||
registerModuleTools("gitea", (env) => {
|
||||
try {
|
||||
return getGiteaTools(GiteaClient.fromEnv(env));
|
||||
return getGiteaTools(GiteaClient.fromEnv(env), () => GiteaAdmin.fromEnv(env));
|
||||
} catch (err) {
|
||||
console.log(`[gitea] no tools — ${err instanceof Error ? err.message : String(err)}`);
|
||||
return [];
|
||||
|
||||
@@ -128,10 +128,10 @@ export interface OutOfDateWriter {
|
||||
* out of date (ADR 0323 §4). A trunk first seen is recorded as it is and nothing is put out of date. Bounded however
|
||||
* many are open: one listing of the open pull requests (paged), one read of each trunk, then per open pull request into
|
||||
* a trunk that moved one read and at most two writes, one at a time. A trunk that could not be weighed whole is not
|
||||
* recorded, so the next pass weighs it again. Answers whether the record changed.
|
||||
* recorded, so the next pass weighs it again. Answers whether the record changed. `moved` is told of each base that moved, for the siblings whose base it is.
|
||||
*/
|
||||
export async function weighTrunks(forge: TrunkForge, gates: OutOfDateWriter, trunks: Record<string, string>, owner: string,
|
||||
name: string): Promise<boolean> {
|
||||
name: string, moved?: (base: string) => Promise<void>): Promise<boolean> {
|
||||
let changed = false;
|
||||
try {
|
||||
const open = (await forge.listOpenPulls(owner, name)).filter((p) => p.state === "open" && !!p.head_sha && !!p.base);
|
||||
@@ -151,6 +151,8 @@ export async function weighTrunks(forge: TrunkForge, gates: OutOfDateWriter, tru
|
||||
}
|
||||
console.log(`[gitea] ${owner}/${name}'s ${base} moved from ${twelve(before)} to ${twelve(head)}: ${put} open pull ` +
|
||||
`request(s) into it put out of date, nothing run (novox/hq ADR 0323)`);
|
||||
// A verdict that read a pull request into this trunk as a sibling is out of date by its move too (ADR 0382 §5).
|
||||
if (moved) await moved(base);
|
||||
}
|
||||
trunks[key] = head;
|
||||
changed = true;
|
||||
|
||||
@@ -0,0 +1,519 @@
|
||||
// **The forge says every event on the bus** (novox/hq issue 729; to-be 10: an event makes it fast, nothing makes it
|
||||
// necessary). The module keeps one system webhook on its forge for every event, pointing at its own receiver, signed
|
||||
// with a secret it mints and keeps beside its token; the receiver refuses whatever is unsigned or does not verify, and
|
||||
// publishes each event, trimmed, on the module's events as forge.<kind>.<action> (forge.pull-request.opened, forge.push, …),
|
||||
// on the shared EVENTS path its other events take; its own stream (ADR 0363) is a follow-up once issue 725 ships.
|
||||
// The polls stay as the safety net, slowed; the announced-heads and announced-merges records keep anything from
|
||||
// being announced twice, whichever path saw it first.
|
||||
//
|
||||
// The secret is never logged: lines say it was minted or reused and where it is kept, never what it is.
|
||||
|
||||
import { createHmac, randomBytes, timingSafeEqual } from "node:crypto";
|
||||
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
||||
import { createServer, type Server } from "node:http";
|
||||
import { dirname } from "node:path";
|
||||
|
||||
/** The name the receiver answers on, and the port it listens on unless the manifest says otherwise. */
|
||||
export const HOOK_PATH = "/forge-event";
|
||||
export const HOOK_PORT = 3047;
|
||||
|
||||
/** The secret the forge signs with: read from its 0600 file, or minted and kept there. */
|
||||
export function keptSecret(file: string, log: (line: string) => void = (l) => console.log(`[gitea] ${l}`)): string {
|
||||
try {
|
||||
const kept = readFileSync(file, "utf8").trim();
|
||||
if (kept.length >= 32) {
|
||||
log(`reusing the webhook secret kept at ${file}`);
|
||||
return kept;
|
||||
}
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err;
|
||||
}
|
||||
const secret = randomBytes(32).toString("hex");
|
||||
mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
|
||||
const tmp = `${file}.tmp`;
|
||||
writeFileSync(tmp, secret + "\n", { mode: 0o600 });
|
||||
chmodSync(tmp, 0o600);
|
||||
renameSync(tmp, file);
|
||||
log(`minted a webhook secret, kept at ${file}`);
|
||||
return secret;
|
||||
}
|
||||
|
||||
/** Whether a body carries the forge's HMAC-SHA256 signature (X-Gitea-Signature, hex) under the secret. */
|
||||
export function verified(secret: string, body: Buffer | string, signature: string | undefined): boolean {
|
||||
if (!secret || !signature || !/^[0-9a-f]{64}$/i.test(signature.trim())) return false;
|
||||
const want = createHmac("sha256", secret).update(body).digest();
|
||||
const got = Buffer.from(signature.trim(), "hex");
|
||||
return got.length === want.length && timingSafeEqual(got, want);
|
||||
}
|
||||
|
||||
/** One forge event as the bus carries it: its name inside the stream, and what is needed of it — no secrets. */
|
||||
export interface ForgeEvent {
|
||||
name: string;
|
||||
body: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/** Every event name the module emits for the forge, without its forge. prefix (module.json `emits`). */
|
||||
export const EVENTS: readonly string[] = ["push", "create", "delete", "fork", "status", "pull-request.opened", "pull-request.reopened", "pull-request.synchronized", "pull-request.edited", "pull-request.closed", "pull-request.merged", "pull-request.labeled", "pull-request.unlabeled", "pull-request.assigned", "pull-request.unassigned", "pull-request.reviewed", "pull-request.review-requested", "pull-request.commented", "pull-request.changed", "issue-comment.created", "issue-comment.edited", "issue-comment.deleted", "issues.opened", "issues.reopened", "issues.closed", "issues.edited", "issues.labeled", "issues.unlabeled", "issues.assigned", "issues.unassigned", "issues.changed", "release.published", "release.updated", "release.deleted", "release.changed", "repository.created", "repository.deleted", "repository.changed", "wiki.changed", "other"];
|
||||
|
||||
/** The families the module's stream carries (module.json `streams`); any other kind is said as other.<kind>; underscores become hyphens. */
|
||||
export const FAMILIES = ["push", "create", "delete", "fork", "status", "pull_request", "issue_comment", "issues",
|
||||
"release", "repository", "wiki"] as const;
|
||||
|
||||
const ACTIONS: Record<string, string> = { label_updated: "labeled", label_cleared: "unlabeled", synchronized: "synchronized" };
|
||||
|
||||
function str(v: unknown): string | undefined {
|
||||
return typeof v === "string" && v ? v : undefined;
|
||||
}
|
||||
function num(v: unknown): number | undefined {
|
||||
return typeof v === "number" ? v : undefined;
|
||||
}
|
||||
function login(u: any): string | undefined {
|
||||
return str(u?.login) ?? str(u?.username);
|
||||
}
|
||||
|
||||
/**
|
||||
* The event a forge delivery is: its kind (X-Gitea-Event) and the payload's action, trimmed to ids, repository, refs,
|
||||
* shas, author, labels and a comment's body (a ready mark is one, ADR 0364). A merge is pull-request.merged, a comment on
|
||||
* a pull request pull-request.commented.
|
||||
*/
|
||||
export function eventOf(kind: string, payload: any): ForgeEvent {
|
||||
// The bus's event names take lower-case letters, digits and hyphens: the forge's underscores become hyphens.
|
||||
const event = forgeEventOf(kind, payload);
|
||||
const name = event.name.toLowerCase().replace(/_/g, "-").replace(/[^a-z0-9.-]/g, "-");
|
||||
// Published on the module's events as it publishes its others, each name declared in its `emits`: one the manifest
|
||||
// does not name is forge.other, saying what the forge called it. Its own stream (ADR 0363) follows issue 725.
|
||||
if (EVENTS.includes(name)) return { name: `forge.${name}`, body: event.body };
|
||||
return { name: "forge.other", body: { ...event.body, kind, said: name } };
|
||||
}
|
||||
|
||||
function forgeEventOf(said: string, payload: any): ForgeEvent {
|
||||
// The forge names some kinds finer than their family (pull_request_label, pull_request_sync): read by the payload.
|
||||
let kind = said;
|
||||
if (said.startsWith("pull_request") && payload?.pull_request) kind = "pull_request";
|
||||
if ((said.startsWith("issue") || said === "pull_request_comment") && payload?.comment) kind = "issue_comment";
|
||||
if (said.startsWith("issue") && !payload?.comment && payload?.issue) kind = "issues";
|
||||
const repo = payload?.repository ?? {};
|
||||
const base: Record<string, unknown> = {
|
||||
owner: login(repo.owner), repo: str(repo.name), full_name: str(repo.full_name), clone_url: str(repo.clone_url),
|
||||
sender: login(payload?.sender),
|
||||
};
|
||||
const family = (FAMILIES as readonly string[]).includes(kind) ? kind : `other`;
|
||||
let action = str(payload?.action);
|
||||
const pull = payload?.pull_request;
|
||||
if (kind === "push") {
|
||||
return { name: "push", body: { ...base, ref: str(payload.ref), before: str(payload.before), after: str(payload.after),
|
||||
commits: Array.isArray(payload.commits) ? payload.commits.length : undefined, pusher: login(payload.pusher) } };
|
||||
}
|
||||
if (kind === "pull_request" && pull) {
|
||||
if (action === "closed" && pull.merged) action = "merged";
|
||||
action = ACTIONS[action ?? ""] ?? action ?? "changed";
|
||||
return { name: `pull_request.${action}`, body: { ...base, number: num(pull.number ?? payload.number),
|
||||
title: str(pull.title), author: login(pull.user), head: str(pull.head?.ref), head_sha: str(pull.head?.sha),
|
||||
base: str(pull.base?.ref), merged: !!pull.merged, merge_commit_sha: str(pull.merge_commit_sha),
|
||||
merged_at: str(pull.merged_at), html_url: str(pull.html_url),
|
||||
labels: Array.isArray(pull.labels) ? pull.labels.map((l: any) => str(l?.name)).filter(Boolean) : undefined,
|
||||
label: str(payload.label?.name) } };
|
||||
}
|
||||
if (kind === "issue_comment") {
|
||||
const issue = payload.issue ?? {};
|
||||
const onPull = !!payload.is_pull || !!issue.pull_request;
|
||||
return { name: onPull ? "pull_request.commented" : `issue_comment.${action ?? "created"}`, body: { ...base,
|
||||
number: num(issue.number), comment_id: num(payload.comment?.id), author: login(payload.comment?.user),
|
||||
comment: str(payload.comment?.body), action } };
|
||||
}
|
||||
if (kind === "create" || kind === "delete") {
|
||||
return { name: kind, body: { ...base, ref: str(payload.ref), ref_type: str(payload.ref_type), sha: str(payload.sha) } };
|
||||
}
|
||||
if (kind === "status") {
|
||||
return { name: "status", body: { ...base, sha: str(payload.sha), context: str(payload.context), state: str(payload.state) } };
|
||||
}
|
||||
const name = family === "other" ? `other.${kind.replace(/[^a-z0-9_-]/gi, "_") || "unknown"}` :
|
||||
["fork"].includes(family) ? family : `${family}.${action ?? "changed"}`;
|
||||
return { name, body: { ...base, number: num(payload?.number ?? payload?.issue?.number), action } };
|
||||
}
|
||||
|
||||
export interface ReceiverOptions {
|
||||
secret: string;
|
||||
port: number;
|
||||
host?: string;
|
||||
/** Publishes one event on the bus. */
|
||||
publish: (event: ForgeEvent) => Promise<void>;
|
||||
/** Acts on it at once, beside publishing it (the announcer). */
|
||||
act?: (event: ForgeEvent) => Promise<void>;
|
||||
/** Told of every verified delivery — its kind (X-Gitea-Event) and its delivery id — so the polls know events are
|
||||
* heard, and the module can say when it last heard one. */
|
||||
heard?: (kind: string, delivery: string) => void;
|
||||
/** Told of every refused delivery, with the reason and where it came from. */
|
||||
refused?: (reason: string, from: string) => void;
|
||||
log?: (line: string) => void;
|
||||
}
|
||||
|
||||
/** The most a delivery may carry: the forge's push of many commits stays well under it. */
|
||||
export const MAX_BODY = 5 * 1024 * 1024;
|
||||
/** How many deliveries' ids are remembered, so a delivery sent again is published once. */
|
||||
export const REMEMBERED = 2000;
|
||||
|
||||
/**
|
||||
* The receiver. A request without a signature is refused before its body is read, a body over MAX_BODY is refused, and
|
||||
* one whose signature does not verify is refused, each with 401 or 413 and a line saying so. A verified delivery is
|
||||
* answered 200 at once and queued: it is published and acted on after the reply, one at a time, so the forge's
|
||||
* delivery timeout is never reached. A delivery whose X-Gitea-Delivery id was already heard is answered and dropped.
|
||||
*/
|
||||
export function receiver(opts: ReceiverOptions): Server & { idle(): Promise<void> } {
|
||||
const log = opts.log ?? ((l) => console.log(`[gitea] ${l}`));
|
||||
const refuse = (from: string, reason: string): void => {
|
||||
log(`refused a forge event from ${from}: ${reason}`);
|
||||
opts.refused?.(reason, from);
|
||||
};
|
||||
const seen = new Set<string>();
|
||||
let queue: Promise<void> = Promise.resolve();
|
||||
const server = createServer((req, res) => {
|
||||
if (req.method !== "POST" || (req.url ?? "").split("?")[0] !== HOOK_PATH) {
|
||||
res.statusCode = 404;
|
||||
return res.end();
|
||||
}
|
||||
const from = req.socket.remoteAddress ?? "?";
|
||||
const signature = String(req.headers["x-gitea-signature"] ?? "").trim();
|
||||
if (!signature) {
|
||||
refuse(from, "it is unsigned");
|
||||
res.writeHead(401, { Connection: "close" });
|
||||
res.end("unsigned\n");
|
||||
req.destroy();
|
||||
return;
|
||||
}
|
||||
if (Number(req.headers["content-length"] ?? 0) > MAX_BODY) {
|
||||
refuse(from, `its body is over ${MAX_BODY} bytes`);
|
||||
res.writeHead(413, { Connection: "close" });
|
||||
res.end("too large\n");
|
||||
req.destroy();
|
||||
return;
|
||||
}
|
||||
const chunks: Buffer[] = [];
|
||||
let size = 0;
|
||||
let over = false;
|
||||
req.on("data", (c: Buffer) => {
|
||||
size += c.length;
|
||||
if (size > MAX_BODY) {
|
||||
if (!over) {
|
||||
over = true;
|
||||
refuse(from, `its body is over ${MAX_BODY} bytes`);
|
||||
res.writeHead(413, { Connection: "close" });
|
||||
res.end("too large\n");
|
||||
req.destroy();
|
||||
}
|
||||
} else chunks.push(c);
|
||||
});
|
||||
req.on("end", () => {
|
||||
if (over) return;
|
||||
const body = Buffer.concat(chunks);
|
||||
if (!verified(opts.secret, body, signature)) {
|
||||
refuse(from, "its signature does not verify");
|
||||
res.statusCode = 401;
|
||||
return res.end("signature does not verify\n");
|
||||
}
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = JSON.parse(body.toString("utf8"));
|
||||
} catch {
|
||||
refuse(from, "it is signed but not JSON");
|
||||
res.statusCode = 400;
|
||||
return res.end("not JSON\n");
|
||||
}
|
||||
const delivery = String(req.headers["x-gitea-delivery"] ?? "");
|
||||
const kind = String(req.headers["x-gitea-event"] ?? "other");
|
||||
opts.heard?.(kind, delivery);
|
||||
if (delivery && seen.has(delivery)) {
|
||||
res.statusCode = 200;
|
||||
return res.end("already heard\n");
|
||||
}
|
||||
if (delivery) {
|
||||
seen.add(delivery);
|
||||
if (seen.size > REMEMBERED) seen.delete(seen.values().next().value as string);
|
||||
}
|
||||
const event = eventOf(kind, payload);
|
||||
res.statusCode = 200;
|
||||
res.end("queued\n");
|
||||
queue = queue.then(async () => {
|
||||
try {
|
||||
await opts.publish(event);
|
||||
if (opts.act) await opts.act(event);
|
||||
} catch (err) {
|
||||
log(`a forge event ${event.name} could not be published or acted on; the polls catch it — ${err instanceof Error ? err.message : err}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
return Object.assign(server, { idle: () => queue });
|
||||
}
|
||||
|
||||
/** What ensuring the system webhook needs of the forge's administration (GiteaAdmin's login). */
|
||||
export interface HookAdmin {
|
||||
call(method: string, path: string, body?: unknown): Promise<{ status: number; body: any }>;
|
||||
}
|
||||
|
||||
/** The forge's one system webhook as ensured at start: read back from the forge after it was made. */
|
||||
export interface HookState {
|
||||
/** Always "system": a hook the forge's listing of system webhooks names, so it fires for every repository. */
|
||||
kind: "system";
|
||||
id: number;
|
||||
url: string;
|
||||
events: string[];
|
||||
active: boolean;
|
||||
content_type?: string;
|
||||
/** Made where none was, or made in place of the ones there (system, default, or a repository's) at the receiver's url. */
|
||||
did: "created" | "replaced";
|
||||
/** The hooks at the receiver's url it took the place of, by kind: a repository's is a copy of a default webhook, which
|
||||
* would have sent that repository's events a second time. */
|
||||
removed: { system: number; default: number; repository: number; repositories: string[] };
|
||||
}
|
||||
|
||||
/** How long without a verified delivery — since the start, or since the last one — a forge is said silent once the poll
|
||||
* finds what no event said. */
|
||||
export const SILENT_AFTER = 10 * 60_000;
|
||||
|
||||
/**
|
||||
* **How fast the polls look, and whether the forge's events are heard** (issue 729): every five minutes only while the
|
||||
* forge's events are heard — a verified delivery within the last ten minutes, and none missed since. Until the first
|
||||
* delivery, after ten minutes without one, or once the poll found something no event said, every thirty seconds.
|
||||
*
|
||||
* It also keeps what `gitea_events` says: where the receiver listens, the hook as ensured, the verified and refused
|
||||
* deliveries, and the polls' finds no event said. **A forge whose events stop is said** (issue 729): once the receiver
|
||||
* listens and the hook is ensured, when nothing verified for ten minutes — since the start, or since the last delivery —
|
||||
* and the poll has found at least two heads or merges no event said in that quiet (one could be the poll beating an
|
||||
* event by a second), it is logged loudly once per silent spell, and the module's events are said unhealthy until a
|
||||
* delivery verifies. Deliveries refused since the last verified one are said unhealthy too.
|
||||
*/
|
||||
export class Pace {
|
||||
private last = 0;
|
||||
private missed = 0;
|
||||
private readonly started: number;
|
||||
private readonly now: () => number;
|
||||
private readonly log: (l: string) => void;
|
||||
private where: { host: string; port: number; url: string } | null = null;
|
||||
private hook: HookState | null = null;
|
||||
private notHeard: string | null = null;
|
||||
private notHeardFails = false;
|
||||
private verifiedCount = 0;
|
||||
private lastVerified: { at: number; event: string; delivery: string } | null = null;
|
||||
private refusedCount = 0;
|
||||
private refusedSpell = 0;
|
||||
private lastRefused: { at: number; reason: string; from: string } | null = null;
|
||||
private missCount = 0;
|
||||
private lastMiss: { at: number; what: string } | null = null;
|
||||
private silentSince = 0;
|
||||
private quietMisses = 0;
|
||||
constructor(now: () => number = Date.now, log: (l: string) => void = (l) => console.error(`[gitea] ${l}`)) {
|
||||
this.now = now;
|
||||
this.log = log;
|
||||
this.started = now();
|
||||
}
|
||||
/** The receiver listens, and the hook at it was ensured: from now on, deliveries are expected. */
|
||||
listening(host: string, port: number, url: string, hook: HookState): void {
|
||||
this.where = { host, port, url };
|
||||
this.hook = hook;
|
||||
this.notHeard = null;
|
||||
}
|
||||
/** No events are listened for, and why: by configuration (`fails` false), or because the receiver could not listen
|
||||
* or the hook could not be ensured (`fails` true, said unhealthy). The polls alone watch. */
|
||||
unheard(why: string, fails = true): void {
|
||||
this.notHeard = why;
|
||||
this.notHeardFails = fails;
|
||||
}
|
||||
heard(event = "", delivery = ""): void {
|
||||
this.last = this.now();
|
||||
this.verifiedCount++;
|
||||
this.lastVerified = { at: this.last, event, delivery };
|
||||
this.refusedSpell = 0;
|
||||
this.quietMisses = 0;
|
||||
if (this.silentSince) this.log(`the forge's events are heard again: ${event || "a delivery"} verified`);
|
||||
this.silentSince = 0;
|
||||
}
|
||||
refused(reason: string, from: string): void {
|
||||
this.refusedCount++;
|
||||
this.refusedSpell++;
|
||||
this.lastRefused = { at: this.now(), reason, from };
|
||||
}
|
||||
/** The poll found a head or merge no forge event announced. */
|
||||
miss(what: string): void {
|
||||
const t = this.now();
|
||||
if (this.hearing()) this.log(`THE FORGE'S EVENTS MISSED ${what}: the poll found it, so the polls look every 30 s again until the next event`);
|
||||
this.missed = t;
|
||||
this.missCount++;
|
||||
this.lastMiss = { at: t, what };
|
||||
// Silent: nothing verified for SILENT_AFTER — never since the start, or not since the last delivery (a hook deleted by
|
||||
// hand, every delivery refused) — while the poll finds what no event said. Said once per silent spell.
|
||||
const quietSince = this.last || this.started;
|
||||
this.quietMisses++;
|
||||
if (this.expecting() && t - quietSince >= SILENT_AFTER && this.quietMisses >= 2 && !this.silentSince) {
|
||||
this.silentSince = t;
|
||||
const minutes = Math.round((t - quietSince) / 60_000);
|
||||
const refusedSince = this.refusedSinceHeard();
|
||||
this.log(`THE FORGE'S EVENTS ARE NOT HEARD: ${this.last ? `no delivery verified since the last at ` +
|
||||
`${new Date(this.last).toISOString()}` : "no delivery ever verified since the start"}, ${minutes} min, from the ` +
|
||||
`system webhook at ${this.hook?.url}${refusedSince ? `; ${refusedSince} refused since, the last because ` +
|
||||
`${this.lastRefused?.reason}` : ""}; the poll found ${what} — the polls alone watch the forge, every 30 s ` +
|
||||
`(novox/hq issue 729)`);
|
||||
}
|
||||
}
|
||||
/** Whether a delivery was refused after the last verified one (or with none verified). */
|
||||
private refusedSinceHeard(): number {
|
||||
return this.lastRefused && this.lastRefused.at >= this.last ? this.refusedSpell : 0;
|
||||
}
|
||||
hearing(): boolean {
|
||||
const t = this.now();
|
||||
return this.last > 0 && t - this.last < 10 * 60_000 && this.last > this.missed;
|
||||
}
|
||||
every(): number {
|
||||
return this.hearing() ? 5 * 60_000 : 30_000;
|
||||
}
|
||||
private expecting(): boolean {
|
||||
return this.where !== null && this.hook !== null;
|
||||
}
|
||||
/** What `gitea_events` answers: healthy, unhealthy or waits (ADR 0283's shape), with every fact behind it. */
|
||||
report(): Record<string, unknown> {
|
||||
const t = this.now();
|
||||
const iso = (ms: number): string => new Date(ms).toISOString();
|
||||
let state: "healthy" | "unhealthy" | "waits" = "healthy";
|
||||
let why: string;
|
||||
const refusedSince = this.refusedSinceHeard();
|
||||
if (this.silentSince) {
|
||||
state = "unhealthy";
|
||||
why = `${this.last ? `no delivery verified since the last at ${iso(this.last)}` : `the forge has never delivered an ` +
|
||||
`event since the start at ${iso(this.started)}`}, while the poll found ${this.missCount} head(s) or merge(s) no ` +
|
||||
`event said, the last ${this.lastMiss?.what} at ${iso(this.lastMiss?.at ?? t)}`;
|
||||
} else if (refusedSince) {
|
||||
state = "unhealthy";
|
||||
why = `${refusedSince} deliveries refused ${this.last ? `since the last verified one at ${iso(this.last)}` :
|
||||
"and none verified"}, the last at ${iso(this.lastRefused?.at ?? t)} because ${this.lastRefused?.reason}`;
|
||||
} else if (!this.expecting()) {
|
||||
why = this.notHeard ? `no events are listened for, so the polls alone watch the forge: ${this.notHeard}` :
|
||||
"the receiver is not listening yet";
|
||||
if (this.notHeard && this.notHeardFails) state = "unhealthy";
|
||||
} else if (this.hearing()) {
|
||||
why = `the forge's events are heard: ${this.verifiedCount} verified, the last at ${iso(this.last)}`;
|
||||
} else if (this.last === 0) {
|
||||
why = `no delivery heard yet; the forge is said silent if the poll finds what no event said ${SILENT_AFTER / 60_000} min after the start`;
|
||||
} else {
|
||||
why = `the last delivery was at ${iso(this.last)}; the polls look every 30 s until the next`;
|
||||
}
|
||||
return {
|
||||
healthy: state !== "unhealthy", state, why,
|
||||
started: iso(this.started),
|
||||
receiver: this.where ? { listening: true, ...this.where } : { listening: false, why: this.notHeard ?? "not started" },
|
||||
hook: this.hook,
|
||||
verified: { count: this.verifiedCount, last: this.lastVerified && { ...this.lastVerified, at: iso(this.lastVerified.at) } },
|
||||
refused: { count: this.refusedCount, since_verified: refusedSince, last: this.lastRefused && { ...this.lastRefused, at: iso(this.lastRefused.at) } },
|
||||
polls: { every_seconds: this.every() / 1000, hearing: this.hearing(), found_unsaid: this.missCount,
|
||||
last_found: this.lastMiss && { ...this.lastMiss, at: iso(this.lastMiss.at) } },
|
||||
silent: this.silentSince ? { since: iso(this.silentSince) } : null,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** Every event family the forge (1.27) can send a webhook. */
|
||||
export const HOOK_EVENTS = ["create", "delete", "fork", "push", "issues", "issue_comment", "pull_request",
|
||||
"pull_request_review_request", "wiki", "repository", "release", "package", "status", "workflow_run", "workflow_job"];
|
||||
|
||||
async function hooksAt(admin: HookAdmin, type: "system" | "default", url: string): Promise<any[]> {
|
||||
const listed = await admin.call("GET", `/admin/hooks?type=${type}&limit=50`);
|
||||
if (listed.status !== 200) throw new Error(`the forge's ${type} webhooks could not be listed: ${listed.status}`);
|
||||
return (Array.isArray(listed.body) ? listed.body : []).filter((h) => h?.config?.url === url);
|
||||
}
|
||||
|
||||
/** Every page of a listing that answers an array (or `{ data: [] }`, the repository search), PAGE at a time. */
|
||||
const PAGE = 50;
|
||||
async function everyPage(admin: HookAdmin, path: string, what: string): Promise<any[]> {
|
||||
const all: any[] = [];
|
||||
for (let page = 1; ; page++) {
|
||||
const r = await admin.call("GET", `${path}${path.includes("?") ? "&" : "?"}page=${page}&limit=${PAGE}`);
|
||||
if (r.status !== 200) throw new Error(`${what} could not be listed: ${r.status}`);
|
||||
const items: any[] = Array.isArray(r.body) ? r.body : Array.isArray(r.body?.data) ? r.body.data : [];
|
||||
all.push(...items);
|
||||
if (items.length < PAGE) return all;
|
||||
}
|
||||
}
|
||||
|
||||
/** Every repository's own webhooks at the receiver's url: copies of a default webhook made at it. */
|
||||
async function repoHooksAt(admin: HookAdmin, url: string): Promise<{ repo: string; id: number }[]> {
|
||||
const found: { repo: string; id: number }[] = [];
|
||||
for (const repo of await everyPage(admin, "/repos/search?sort=id", "the forge's repositories")) {
|
||||
const full = String(repo?.full_name ?? "");
|
||||
if (!full.includes("/")) continue;
|
||||
const hooks = await everyPage(admin, `/repos/${full}/hooks`, `${full}'s webhooks`);
|
||||
for (const h of hooks) if (h?.config?.url === url) found.push({ repo: full, id: Number(h.id) });
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
/**
|
||||
* **One system webhook, for every event, at the receiver** (issue 729), made each start and read back.
|
||||
*
|
||||
* The forge (1.27.3, routers/api/v1/utils/hook.go addHook) makes a *default* webhook from POST /admin/hooks unless
|
||||
* `config.is_system_webhook` is "true": a default webhook fires for no repository and is only copied into repositories
|
||||
* made after it, and the listing of system webhooks does not name it — so without the flag every start made another
|
||||
* one and no event came. Its edit (editHook) never changes a webhook's secret nor its kind, so a hook already there is
|
||||
* not corrected but replaced: the new system webhook is made first, with the secret kept now, so no event falls in a
|
||||
* gap; then every other hook at the receiver's url is deleted — system, default, and each repository's copy of a
|
||||
* default one, which would send that repository's events twice. The forge's listing of system webhooks must then name
|
||||
* exactly one at the url, or this fails, saying so.
|
||||
*/
|
||||
export async function ensureSystemHook(admin: HookAdmin, url: string, secret: string): Promise<HookState> {
|
||||
// The forge has no "every event" word: each family is named, and a family name (issues, pull_request) takes all of it.
|
||||
const want = { type: "gitea", active: true, events: HOOK_EVENTS, branch_filter: "*",
|
||||
config: { url, content_type: "json", secret, is_system_webhook: "true" } };
|
||||
const before = new Set((await hooksAt(admin, "system", url)).map((h) => Number(h.id)));
|
||||
const made = await admin.call("POST", "/admin/hooks", want);
|
||||
if (made.status !== 201 && made.status !== 200) throw new Error(`the forge's system webhook could not be made: ${made.status}`);
|
||||
const fresh = (await hooksAt(admin, "system", url)).filter((h) => !before.has(Number(h.id)));
|
||||
if (fresh.length !== 1) {
|
||||
throw new Error(`the forge names ${fresh.length} new system webhooks at ${url} after one was made, not one: its events ` +
|
||||
`are not known to be sent`);
|
||||
}
|
||||
const ours = fresh[0];
|
||||
const defaults = await hooksAt(admin, "default", url);
|
||||
const gone = async (path: string): Promise<void> => {
|
||||
const r = await admin.call("DELETE", path);
|
||||
if (r.status !== 204 && r.status !== 200 && r.status !== 404) throw new Error(`the forge's webhook ${path} could not be deleted: ${r.status}`);
|
||||
};
|
||||
for (const id of before) await gone(`/admin/hooks/${id}`);
|
||||
for (const h of defaults) await gone(`/admin/hooks/${h.id}`);
|
||||
const copies = await repoHooksAt(admin, url);
|
||||
for (const c of copies) await gone(`/repos/${c.repo}/hooks/${c.id}`);
|
||||
const kept = await hooksAt(admin, "system", url);
|
||||
if (kept.length !== 1 || Number(kept[0].id) !== Number(ours.id)) {
|
||||
throw new Error(`the forge names ${kept.length} system webhooks at ${url} once the others were deleted, not the one ` +
|
||||
`made (${ours.id})`);
|
||||
}
|
||||
const removed = { system: before.size, default: defaults.length, repository: copies.length,
|
||||
repositories: [...new Set(copies.map((c) => c.repo))] };
|
||||
return { kind: "system", id: Number(ours.id), url, events: Array.isArray(ours.events) ? ours.events : [],
|
||||
active: !!ours.active, content_type: ours.config?.content_type,
|
||||
did: removed.system + removed.default + removed.repository > 0 ? "replaced" : "created", removed };
|
||||
}
|
||||
|
||||
/** The `gitea_events` tool: what the module knows of the forge's events, from its Pace. */
|
||||
export function eventsTool(pace: Pace): { name: string; description: string; input: Record<string, unknown>;
|
||||
run: (args: Readonly<Record<string, unknown>>) => Promise<unknown> } {
|
||||
return {
|
||||
name: "gitea_events",
|
||||
description: "Whether the forge's events reach the mesh (novox/hq issue 729): where the receiver listens, the " +
|
||||
"system webhook as ensured at the start (its id, url, events, kind, and what it replaced), how many deliveries " +
|
||||
"verified and the last one's time and event, how many were refused and the last reason, how often the polls look " +
|
||||
"and whether events are heard, and how many heads or merges the poll found that no event said. Unhealthy when the " +
|
||||
"forge has delivered nothing that verified for ten minutes (since the start or since the last delivery) while the " +
|
||||
"poll found what no event said, when deliveries were refused since the last verified one, or when the receiver " +
|
||||
"could not listen or the webhook could not be made.",
|
||||
input: {},
|
||||
run: async () => pace.report(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* **Claim a head for announcing, once** (issue 729): true when `sha` is not already the head recorded for `key`, which
|
||||
* it then is. The poll and the forge's event both claim before they announce, so whichever comes second finds it taken.
|
||||
*/
|
||||
export function claimHead(heads: Record<string, string>, key: string, sha: string): boolean {
|
||||
if (!sha || heads[key] === sha) return false;
|
||||
heads[key] = sha;
|
||||
return true;
|
||||
}
|
||||
Reference in New Issue
Block a user