Feature 57 (config and provisioning): modules declare config and provisions, their files rendered from templates; synced with main (issues of feature 57, 874)

This commit is contained in:
2026-10-16 07:37:43 +00:00
parent 1a2c917abf
commit 9652b2d5ce
771 changed files with 52063 additions and 1774 deletions
+58 -5
View File
@@ -44,7 +44,7 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
| tool | | what |
|---|---|---|
| `gitea_commit` | a | one commit on a branch of its own, created from `base` (default the default branch) when absent, carrying either `files` (each `{path, content \| content_base64 \| delete: true}`) or `patch` (a unified diff, as `git diff` writes it); answers the commit and its parent |
| `gitea_commit` | a | one commit on a branch of its own, created from `base` (default the default branch) when absent, carrying either `files` (each `{path, content \| content_base64 \| delete: true}`) `patch` (a unified diff, as `git diff` writes it), or `patch_file` with `patch_sha256` (a patch too large to type, see below); answers the commit and its parent |
| `gitea_commits` | r | a ref's history, newest first, optionally only commits touching a path: sha, author, date, subject |
| `gitea_compare` | r | what head holds that base does not: its commits and the files they touch |
| `gitea_pull_reviews` | r | a pull request's reviews — state, the commit reviewed and whether it is the head now, stale, dismissed — with their line comments |
@@ -52,13 +52,28 @@ base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an ag
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
never the repository's default branch, never a branch a protection rule names. A rule covers a branch as the
never the repository's default branch, never a branch a protection rule names. The tools' account may not read the
rules themselves (the forge keeps them for a repository's administrator, novox/hq ADR 0365): for it an existing branch
is judged by the protection the forge says it applies there, and a branch not made yet by the forge itself, which
refuses the commit that would make a branch a rule guards. A rule covers a branch as the
forge itself decides it (Gitea 1.27, novox/hq issue 539): a plain name covers that branch in any case; a glob
(`*`, `**`, `?`, `[abc]`, `[!abc]`, `{a,b}`, `\` escaping) covers case-sensitively with `/` as separator; a glob
the forge cannot compile covers its own text. `cmd/forge/rules.go` and `guard.ts` hold this one rule. The rules are read before anything is sent; rules that cannot be read refuse the
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
required checks green, through `gitea_merge_pull_request`.
**A patch too large to type** (novox/hq issue 852: a feature's diff is megabytes) goes as `patch_file`, with
`patch_sha256`. The caller writes the patch into the module's spool directory `forge-patches`
(`/var/lib/mesh/forge-patches`, mode 0733 and owned by the machine's account, which the tools run as: anyone writes,
nobody lists; the bundle is given its path as `MESH_GITEA_PATCH_DIR`) and names it: `patch_file` is the file's NAME,
32 to 64 characters of `0-9a-f` that the caller chooses at random, never a path. The tool then reads that one file,
applies it exactly as it applies `patch` (same size limit, same answer) and removes it once it has found it, whether
the commit was made or not. A file refused before that (a bad name, a missing hash or spool directory, a name with no
file) stays in the spool. It refuses, sending nothing, a name of another shape, a file that is missing, empty, over
4 MiB, with more than one hard link or not a regular file (a symlink is never followed), and a file whose SHA-256 is not `patch_sha256`: the directory has no sticky bit
(the manifest allows none), so another writer could replace a file, and the hash is how the caller knows the tool read
the bytes it wrote. It reads nothing outside the directory.
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. The blobs are read at
@@ -66,7 +81,7 @@ the commit the branch (or, for a new branch, its base) was read at, never at its
forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than the
file. Neither endpoint takes a commit to build on, only a branch (Gitea 1.27), so the parent answered is the
one the forge says it made the commit on; when the branch moved between the read and the commit, the answer
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the tools' account, so its sha is not the
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
branch not at the commit the forge named is a failure, never a success.
@@ -75,8 +90,10 @@ is read once and searched in memory; a binary file is passed over, a file over 4
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found. A
search that reaches the call's deadline stops there and answers what it found, with `stopped_at_deadline`.
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), and one
tool call takes at most 50 seconds.
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), as the
tools' own account (novox/hq ADR 0365 §2, issue 469): the token the TypeScript bundle mints for `mesh-tools` and
keeps (`MESH_GITEA_TOKEN_FILE`), sent only when its scopes line names that account, never the admin login. One tool
call takes at most 50 seconds.
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
@@ -85,6 +102,42 @@ changed under the commit refused; a commit over its size limits refused; protect
forge matches them; a code search stopped at its deadline; every malformed ask refused; and each read
summarised.
## Each agent's read credential (novox/hq ADR 0365 §1, §3–§5, issue 737)
An agent clones and fetches the repositories it may read as a forge account of its own, `agent-<node>`. The
module offers the provision `gitea-read`, whose credential its software issues (ADR 0305) one per holder (issue
749): the Go bundle `cmd/agent-token` serves `gitea_agent_token`, which the vault alone asks (ADR 0349 clause 10;
any other caller is refused and nothing is minted). For each holder the vault names that is the claude-code module
on a machine, it
- makes the account `agent-<node>` with the administrator's login, when there is none: no administrator, a random
password sent once and kept nowhere, and **in no team** — it reads public repositories until the operator adds it
to a team that reads more (§3). An account found made an administrator, inactive or barred from logging in is
made a plain account again;
- deletes the account's token by the name `mesh-agent-read` (by its id, every one of them) and mints one whose one
scope is `read:repository`: whatever team it is put in, the forge refuses it a push, a comment and any token or
administration of its own (§5). Every ask mints anew; that is its rotation;
- answers git's credential-store lines for it — the forge's public origin, read from `origins` (the route's name),
and on the forge's own machine its address there too — sealed to the key the vault made for the call. A holder of
another module is not answered and said under `skipped`.
The vault keeps each as `provision/<forge's node>/gitea-read/<node>/claude-code` and hands it to that holder alone;
the claude-code module places it in the agent's home (ADR 0365 §4). The token is never logged, answered in the clear
or put in an error. `go test ./cmd/agent-token` runs it against a fake forge.
## The agents' read teams (novox/hq ADR 0365 §3, issue 758)
An agent's account reads more than public repositories only through a team, per repository, and only to read. Which
teams exist and who is in them is the operator's, and it is one trusted config value, `agent-read-teams`:
`{"<team>": {"repos": ["<owner>/<repo>", ...], "members": ["<node>", ...]}}`, a member `<node>` being the account
`agent-<node>`. The module renders it into `agent-teams.json` and runs `cmd/agent-teams` when it starts and
whenever that file changes. For each team it makes `agent-read-<team>` in the organisation of the first repository's
owner, with permission read and no other right, its repositories exactly the listed ones and its members exactly the
listed accounts (an account that does not exist yet is skipped and said). A team named `agent-read-*` that has left
the value is emptied, never deleted. A team without that prefix is never touched. There are no verbs: a change is a
change of the value. It says each change in one line on stdout, and never a secret. `go test ./cmd/agent-teams`
runs it against a fake forge.
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
Every publish adds a version to the registry, and until this nothing removed one. The Go bundle
+199 -14
View File
@@ -8,6 +8,7 @@ import { ConfiguredToken, MintedToken, type TokenSource } from "./token.js";
import { protectionBody, type BranchProtection, type ProtectionWanted } from "./protection.js";
import { Refused, checkSentPath, pathSegment, pathSegments } from "./guard.js";
import { isCommit } from "./checks.js";
import { TOOLS_ACCOUNT } from "./tools-account.js";
/** A repository's path on the forge, its owner and name each one segment (guard.ts pathSegment): uniform for every
* call, whether the names came from a caller or from the forge's own answer. */
@@ -101,6 +102,94 @@ export interface OpenPull {
/** The head's statuses by context, the newest of each, with its state (novox/hq issue 438): what tells a check
* that started and never answered — pending — from one that answered. */
checks: Record<string, HeadCheck>;
/** The pull request's labels, by name. */
labels?: string[];
/** Its ready mark (novox/hq issue 722): the label `mesh/ready` and the newest comment "ready at <full head sha>", the
* head a reviewer approved. Absent without the label; `head` empty when the label names no head in a comment. */
ready?: ReadyMark;
}
/** The label that marks a pull request ready to merge at the head a comment names (novox/hq issue 722). */
export const READY_LABEL = "mesh/ready";
/** A comment that names the head a pull request is marked ready at: "ready at <full sha>", on a line of its own. */
const READY_AT = /^\s*ready at ([0-9a-f]{40}|[0-9a-f]{64})\b/im;
/** A pull request's ready mark: the head it names, who said it and when, and the label's id, to take it off. */
export interface ReadyMark {
head: string;
by?: string;
at?: string;
label_id: number;
}
/** **The accounts whose "ready at" comment counts** (novox/hq issue 722): the trusted setting ready-authors, which
* the mesh writes to the file MESH_GITEA_READY_AUTHORS_FILE names as a line `authors=<account> <account> …`. A file
* that cannot be read leaves the mesh's own forge accounts alone, said; never an empty list read as everyone: the
* account the tools act as (novox/hq ADR 0365) and the admin account, whose marks were made before it. */
export function readyAuthors(file = process.env.MESH_GITEA_READY_AUTHORS_FILE, own = process.env.MESH_GITEA_ADMIN_USER,
read: (f: string) => string = (f) => readFileSync(f, "utf8"),
tools = process.env.MESH_GITEA_TOOLS_USER || TOOLS_ACCOUNT): string[] {
const fallback = [...new Set([tools, own].filter((a): a is string => !!a))];
if (!file) return fallback;
try {
const line = read(file).split("\n").find((l) => l.startsWith("authors="));
const authors = (line ?? "").slice("authors=".length).split(/[\s,]+/).filter(Boolean);
if (authors.length) return authors;
console.error(`[gitea] ${file} names no account that may mark a pull request ready; only ${fallback.join(", ") || "none"} may`);
} catch (err) {
console.error(`[gitea] ${file} cannot be read (${err instanceof Error ? err.message : err}); only ` +
`${fallback.join(", ") || "none"} may mark a pull request ready`);
}
return fallback;
}
/** One event of an issue's or pull request's timeline, as the forge keeps it, oldest first: a comment, a push to a
* pull request's branch (`pull_push`), a label added (body "1") or taken off (body ""), and the rest. */
export interface TimelineEvent {
id: number;
type: string;
body: string;
created_at?: string;
user?: string;
label?: { id: number; name: string };
}
/** The comments that wrote "ready at" without counting, by id: each said once in the log (novox/hq issue 722). */
const ignoredSaid = new Set<number>();
/** **The ready mark a pull request's timeline makes** (novox/hq issue 722), read in the forge's order — every event is
* one row of one table, numbered as it happened, so the order is exact where times share a second. A "ready at"
* comment counts only when its author is one of the accounts given, and only when it was written after the label
* mesh/ready was last added and after the last push to the pull request's branch: a push or the label taken off
* undoes every mark before it, so a mark older than the latest push never counts, even with the label still on. A
* comment by any other account is ignored, and said once. */
export function readyMarkOf(labelID: number, events: TimelineEvent[], authors: string[],
say: (line: string) => void = (l) => console.error(l)): ReadyMark {
const listed = new Set(authors.map((a) => a.toLowerCase()));
let labelled = false;
let mark: ReadyMark | undefined;
for (const e of [...events].sort((x, y) => x.id - y.id)) {
if (e.type === "pull_push") {
mark = undefined;
} else if (e.type === "label" && e.label?.name === READY_LABEL) {
labelled = e.body === "1";
mark = undefined;
} else if (e.type === "comment") {
const m = READY_AT.exec(e.body);
if (!m) continue;
if (!listed.has(String(e.user ?? "").toLowerCase())) {
if (!ignoredSaid.has(e.id)) {
ignoredSaid.add(e.id);
say(`[gitea] a "ready at ${m[1].slice(0, 12)}" comment (${e.id}) by ${e.user ?? "an unknown account"} is ignored: ` +
`only ${authors.join(", ") || "no account"} may mark a pull request ready (novox/hq issue 722)`);
}
continue;
}
if (labelled) mark = { head: m[1].toLowerCase(), by: e.user, at: e.created_at, label_id: labelID };
}
}
return mark ?? { head: "", label_id: labelID };
}
/** One status on a head, the newest of its context: its state, what it said, and when it was set. */
@@ -332,8 +421,9 @@ export class GiteaClient {
/** The open pull requests of every repository the mesh's account sees, updated since a moment when one is given,
* page by page (novox/hq issue 347): each with its repository and its own update time. */
async searchOpenPulls(since?: string): Promise<{ owner: string; name: string; number: number; updated_at?: string }[]> {
const out: { owner: string; name: string; number: number; updated_at?: string }[] = [];
async searchOpenPulls(since?: string): Promise<{ owner: string; name: string; number: number; updated_at?: string;
labels?: { id: number; name: string }[] }[]> {
const out: { owner: string; name: string; number: number; updated_at?: string; labels?: { id: number; name: string }[] }[] = [];
for (let page = 1; page <= 20; page++) {
const qs = new URLSearchParams({ type: "pulls", state: "open", limit: "50", page: String(page) });
if (since) qs.set("since", since);
@@ -343,7 +433,8 @@ export class GiteaClient {
const [fo, fn] = String(r.full_name ?? "").split("/");
const owner = String(r.owner ?? fo ?? "");
const name = String(r.name ?? fn ?? "");
if (owner && name && i?.number) out.push({ owner, name, number: Number(i.number), updated_at: i.updated_at });
const labels = (Array.isArray(i?.labels) ? i.labels : []).map((l: any) => ({ id: Number(l?.id ?? 0), name: String(l?.name ?? "") }));
if (owner && name && i?.number) out.push({ owner, name, number: Number(i.number), updated_at: i.updated_at, labels });
}
if (!found || found.length < 50) break;
}
@@ -354,19 +445,26 @@ export class GiteaClient {
* requires and the contexts its head has a status of, in any state. What mesh-delivery reads to say a pull
* request the mesh was never asked to check; asked of the forge, never of what the announcer remembers, so a
* fault in the announcer cannot hide itself. Reads only. */
async openPulls(): Promise<OpenPull[]> {
async openPulls(readyAuthors: string[] = []): Promise<OpenPull[]> {
const out: OpenPull[] = [];
const rules = new Map<string, BranchProtection[] | undefined>();
const rules = new Map<string, BranchProtection | null | undefined>();
for (const found of await this.searchOpenPulls()) {
const full = `${found.owner}/${found.name}`;
const pull = await this.getPullRequest(found.owner, found.name, found.number);
if (pull.state !== "open" || !pull.head_sha) continue;
if (!rules.has(full)) rules.set(full, await this.branchProtections(found.owner, found.name).catch(() => undefined));
const rule = rules.get(full)?.find((p) => (p.rule_name ?? p.branch_name) === (pull.base || "main"));
// The base's protection as the forge applies it to that branch (appliedProtection): readable without administration.
const base = `${full}:${pull.base || "main"}`;
if (!rules.has(base)) rules.set(base, await this.appliedProtection(found.owner, found.name, pull.base || "main").catch(() => undefined));
const rule = rules.get(base);
const checks = await this.headChecks(found.owner, found.name, pull.head_sha);
const labels = found.labels ?? [];
// The ready mark (novox/hq issue 722): its timeline is read, whole, only for a pull request that carries the label.
const label = labels.find((l) => l.name === READY_LABEL);
const ready = label ? readyMarkOf(label.id, await this.timeline(found.owner, found.name, pull.number), readyAuthors)
: undefined;
out.push({ owner: found.owner, repo: found.name, number: pull.number, base: pull.base ?? "", head_sha: pull.head_sha,
html_url: pull.html_url, required: rule?.enable_status_check ? (rule.status_check_contexts ?? []) : [],
statuses: Object.keys(checks), checks });
statuses: Object.keys(checks), checks, labels: labels.map((l) => l.name), ...(ready ? { ready } : {}) });
}
return out;
}
@@ -580,16 +678,44 @@ export class GiteaClient {
// ---- Branch protection ----
/** A repository's branch protection rules. */
/** A repository's branch protection rules. The forge answers them to a repository's administrator alone, which the
* tools' account is not (novox/hq ADR 0365 §2): for it this is refused (403), and what a branch's protection asks
* is read with appliedProtection. */
async branchProtections(owner: string, repo: string): Promise<BranchProtection[]> {
return (await this.request<BranchProtection[]>(`${repoPath(owner, repo)}/branch_protections`)) ?? [];
}
/** The rule named for a branch, null when it has none. */
/** The rule named for a branch, null when it has none — from the rules' list, so a repository administrator's. */
async branchProtection(owner: string, repo: string, rule: string): Promise<BranchProtection | null> {
return (await this.branchProtections(owner, repo)).find((p) => (p.rule_name ?? p.branch_name) === rule) ?? null;
}
/**
* The protection that covers an existing branch, as the forge applies it, or null when none does. Read from the
* branch itself, which anyone who reads the code may read — the rules' own list is a repository administrator's
* (novox/hq ADR 0365 §2) — and as the forge matches it, a glob rule included (its name is the rule's). The
* branch's `user_can_push` stands for `enable_push`: whether this account may push there. A branch that does not
* exist is null: nothing covers it yet, and the forge judges a push that makes it.
*/
async appliedProtection(owner: string, repo: string, branch: string): Promise<BranchProtection | null> {
let b: any;
try {
b = await this.request<any>(`${repoPath(owner, repo)}/branches/${pathSegments("branch", branch)}`);
} catch (err) {
if (/: 404 /.test(String((err as Error)?.message))) return null;
throw err;
}
if (!b?.protected) return null;
return {
rule_name: String(b.effective_branch_protection_name || branch),
branch_name: branch,
enable_push: Boolean(b.user_can_push),
enable_status_check: Boolean(b.enable_status_check),
status_check_contexts: Array.isArray(b.status_check_contexts) ? b.status_check_contexts.map(String) : [],
required_approvals: Number(b.required_approvals ?? 0),
};
}
/** Make a branch require these commit statuses to merge (novox/hq ADR 0237): the rule named for the
* branch is edited — its required statuses replaced by these, everything else it says kept unless
* asked — or created, refusing direct pushes unless asked otherwise. Answers the rule as it is now. */
@@ -639,6 +765,45 @@ export class GiteaClient {
return this.requestText(`${repoPath(owner, repo)}/pulls/${index}.diff`);
}
/** **Whether a pull request carries an approving review at a head** (novox/hq ADR 0382 §6): a review of the forge that
* approves, is not dismissed and was made at that very head; or the mesh's ready mark naming that head (issue 722), which is
* how an agent's approval is read where the forge refuses an agent's approve. Reads only. */
async approvedAt(owner: string, repo: string, index: number, head: string, authors?: string[]): Promise<boolean> {
indexOf(index);
const want = head.toLowerCase();
for (let page = 1; page <= 20; page++) {
const raw = await this.request<any[]>(`${repoPath(owner, repo)}/pulls/${index}/reviews?page=${page}&limit=50`);
for (const r of raw ?? []) {
if (String(r?.state ?? "").toUpperCase() === "APPROVED" && !r?.dismissed && !r?.stale &&
String(r?.commit_id ?? "").toLowerCase() === want) return true;
}
if (!raw || raw.length < 50) break;
}
const labels = await this.request<any[]>(`${repoPath(owner, repo)}/issues/${index}/labels`);
const label = (labels ?? []).find((l) => l?.name === READY_LABEL);
if (!label) return false;
const ready = readyMarkOf(Number(label.id ?? 0), await this.timeline(owner, repo, index), authors ?? readyAuthors());
return !!ready && ready.head.toLowerCase() === want;
}
/** **Every event of an issue's or pull request's timeline**, oldest first, page by page (novox/hq issue 722): a long
* thread hides nothing. Bounded at 100 pages of 50; a longer one is said, and its first 5000 events read. */
async timeline(owner: string, repo: string, index: number): Promise<TimelineEvent[]> {
indexOf(index);
const out: TimelineEvent[] = [];
for (let page = 1; page <= 100; page++) {
const raw = await this.request<any[]>(`${repoPath(owner, repo)}/issues/${index}/timeline?page=${page}&limit=50`);
for (const e of raw ?? []) {
out.push({ id: Number(e?.id ?? 0), type: String(e?.type ?? ""), body: String(e?.body ?? ""),
created_at: e?.created_at, user: e?.user?.login,
...(e?.label ? { label: { id: Number(e.label.id ?? 0), name: String(e.label.name ?? "") } } : {}) });
}
if (!raw || raw.length < 50) return out;
}
console.error(`[gitea] ${owner}/${repo}#${index} has more than 5000 timeline events; only the first 5000 are read`);
return out;
}
/** Every comment on an issue or pull request, oldest first. */
async listComments(owner: string, repo: string, index: number): Promise<GiteaComment[]> {
indexOf(index);
@@ -803,12 +968,26 @@ export class GiteaAdmin {
return { status: res.status, body };
}
/** One admin-API call with a JSON body, answering the status and body (the system webhook, novox/hq issue 729). */
call(method: string, path: string, body?: unknown): Promise<AdminResponse> {
return this.request(path, { method, ...(body === undefined ? {} : { body: JSON.stringify(body) }) });
}
/** Fail with the forge's own message when a status the caller did not expect comes back. */
private static fail(path: string, res: AdminResponse): never {
const detail = typeof res.body === "string" ? res.body : JSON.stringify(res.body);
throw new Error(`Gitea admin ${path}: ${res.status} ${detail}`);
}
/** A repository's branch protection rules, read with the administrator account — the forge answers them to a
* repository's administrator alone, which the tools' account is not (novox/hq ADR 0365 §2, ADR 0375). Only a read. */
async branchProtections(owner: string, repo: string): Promise<BranchProtection[]> {
const path = `${repoPath(owner, repo)}/branch_protections`;
const res = await this.request(path);
if (res.status !== 200) GiteaAdmin.fail(path, res);
return Array.isArray(res.body) ? res.body : [];
}
/** Ensure the npm-owner org exists. 201 created, 2xx/404-then-created, and 422/409 (a concurrent
* create won the race) are all success. */
async ensureOrg(name: string): Promise<void> {
@@ -832,12 +1011,16 @@ export class GiteaAdmin {
// unit the team does not have — so code read must be said here: without it gitea answers a
// member's clone of a private repository with "not found", which is how the builder's first
// credentialed clone failed against a team that named only packages.
const units = {
return this.ensureTeamWith(org, team, {
permission: "read",
units_map: { "repo.code": "read", "repo.packages": packageWrite ? "write" : "read" },
includes_all_repositories: true,
can_create_org_repo: false,
};
});
}
/** Ensure a team exists with exactly these units, and return its id (ensureTeam; the tools' team, tools-account.ts). */
async ensureTeamWith(org: string, team: string, units: Record<string, unknown>): Promise<number> {
const found = await this.findTeam(org, team);
if (found !== null) {
const patch = await this.request(`/teams/${found}`, {
@@ -872,7 +1055,7 @@ export class GiteaAdmin {
* The edit path is taken only when the user actually exists. A 422 from the create is also what
* a plain validation failure returns, and reading it as "already there" made the follow-up edit
* 404 — burying the create's own message, which is the one that says what is actually wrong. */
async ensureUser(username: string, password: string, email: string): Promise<void> {
async ensureUser(username: string, password: string, email: string, plain = false): Promise<void> {
const res = await this.request("/admin/users", {
method: "POST",
body: JSON.stringify({ username, email, password, must_change_password: false }),
@@ -886,7 +1069,9 @@ export class GiteaAdmin {
// login_name is required by the admin edit endpoint; for a local user it is the username.
// active and prohibit_login: a deactivated or login-prohibited user is refused like a wrong
// password, so the provisioner's check reports it lost; applying again must undo both.
body: JSON.stringify({ login_name: username, password, must_change_password: false, active: true, prohibit_login: false }),
// plain: never a site administrator — one made so by hand is made a plain account again (novox/hq ADR 0365).
body: JSON.stringify({ login_name: username, password, must_change_password: false, active: true, prohibit_login: false,
...(plain ? { admin: false } : {}) }),
});
if (patch.status === 200) return;
GiteaAdmin.fail(`/admin/users/${username}`, patch);
+42
View File
@@ -0,0 +1,42 @@
// agent-teams: the agents' read teams on the forge (novox/hq ADR 0365 clause 3, issue 758). A program the node's
// runtime runs once when the module starts and again whenever the module's config value `agent-read-teams` changes:
// it reads the file the mesh renders from that value, and makes the forge's teams named agent-read-* match it. Each
// team reads only; its repositories and its members (agent accounts, agent-<node>) are exactly the value's. It
// touches nothing outside the value: no team without the prefix, no repository or account of another kind.
//
// It says each change on stdout in one line, and never a secret.
package main
import (
"context"
"fmt"
"os"
"time"
"gitea/internal/forgehttp"
)
func main() {
forge, missing := forgehttp.FromEnv()
file := os.Getenv("MESH_AGENT_TEAMS_FILE")
if file == "" {
missing = append(missing, "MESH_AGENT_TEAMS_FILE")
}
if len(missing) > 0 {
fmt.Fprintf(os.Stderr, "agent-teams: %v not set by the manifest; nothing was changed\n", missing)
os.Exit(1)
}
value, err := ReadValue(file)
if err != nil {
fmt.Fprintln(os.Stderr, "agent-teams:", err)
os.Exit(1)
}
s := &Syncer{Forge: forge, Log: func(format string, a ...any) { fmt.Printf(format+"\n", a...) },
WaitFor: 3 * time.Minute, Poll: 5 * time.Second}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
if err := s.Sync(ctx, value); err != nil {
fmt.Fprintln(os.Stderr, "agent-teams:", err)
os.Exit(1)
}
}
+339
View File
@@ -0,0 +1,339 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"regexp"
"sort"
"strings"
"time"
"gitea/internal/forgehttp"
)
const (
// ConfigKey is the module's config value this program applies.
ConfigKey = "agent-read-teams"
// TeamPrefix names every team this program owns; a team without it is never touched (ADR 0365 §3).
TeamPrefix = "agent-read-"
// AccountPrefix names an agent's account for its node (cmd/agent-token AccountPrefix, ADR 0365 §1).
AccountPrefix = "agent-"
page = 50
)
var (
teamName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]{0,40}$`)
nodeName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,31}$`)
repoName = regexp.MustCompile(`^([A-Za-z0-9][A-Za-z0-9_.-]*)/([A-Za-z0-9_.-]+)$`)
)
// readUnits are the units a read team has, each at read: the forge's rights of a team that reads and nothing more.
var readUnits = map[string]string{
"repo.code": "read", "repo.issues": "read", "repo.pulls": "read", "repo.releases": "read", "repo.wiki": "read",
}
// Team is one read team of the value: its repositories (owner/repo, read only) and its members (nodes whose agent
// account, agent-<node>, is in the team).
type Team struct {
Repos []string `json:"repos"`
Members []string `json:"members"`
}
// Value is the config value: each team by the name it is given without the prefix.
type Value map[string]Team
// ReadValue reads the file the mesh renders from the config value. A file without the key is the empty value,
// which empties every read team.
func ReadValue(path string) (Value, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("the teams file cannot be read (%s): %w", path, err)
}
var file map[string]json.RawMessage
if err := json.Unmarshal(raw, &file); err != nil {
return nil, fmt.Errorf("the teams file %s is not JSON: %w", path, err)
}
var v Value
if r, ok := file[ConfigKey]; ok {
dec := json.NewDecoder(strings.NewReader(string(r)))
dec.DisallowUnknownFields()
if err := dec.Decode(&v); err != nil {
return nil, fmt.Errorf("%s in %s is not {team: {repos, members}}: %w", ConfigKey, path, err)
}
}
return v, nil
}
// Syncer makes the forge's agent-read-* teams the value's.
type Syncer struct {
Forge forgehttp.Client
Log func(format string, a ...any)
WaitFor time.Duration // how long to wait for the forge to answer; none when zero
Poll time.Duration
}
type forgeTeam struct {
ID int64 `json:"id"`
Name string `json:"name"`
Permission string `json:"permission"`
}
type forgeRepo struct {
FullName string `json:"full_name"`
}
type forgeMember struct {
Login string `json:"login"`
}
func (s *Syncer) say(format string, a ...any) {
if s.Log != nil {
s.Log(format, a...)
}
}
// Sync applies the value. A team that cannot be applied is said and the others still are; the error names them.
func (s *Syncer) Sync(ctx context.Context, v Value) error {
if err := s.waitForForge(ctx); err != nil {
return err
}
orgs, err := s.orgs(ctx)
if err != nil {
return err
}
wanted := map[string]bool{}
var failed []string
names := make([]string, 0, len(v))
for n := range v {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
wanted[TeamPrefix+n] = true
if err := s.apply(ctx, n, v[n]); err != nil {
s.say("not applied: team %s%s: %v", TeamPrefix, n, err)
failed = append(failed, TeamPrefix+n)
}
}
for _, org := range orgs {
teams, err := s.teams(ctx, org)
if err != nil {
return err
}
for _, t := range teams {
if !strings.HasPrefix(t.Name, TeamPrefix) || wanted[t.Name] {
continue
}
if err := s.sync(ctx, org, t, nil, nil); err != nil {
s.say("not applied: team %s in %s: %v", t.Name, org, err)
failed = append(failed, t.Name)
}
}
}
if len(failed) > 0 {
return fmt.Errorf("not applied: %s", strings.Join(failed, ", "))
}
return nil
}
func (s *Syncer) waitForForge(ctx context.Context) error {
deadline := time.Now().Add(s.WaitFor)
for {
err := s.Forge.GetJSON(ctx, "/api/v1/version", nil)
if err == nil {
return nil
}
var se *forgehttp.StatusError
if errors.As(err, &se) || time.Now().After(deadline) || ctx.Err() != nil {
return fmt.Errorf("the forge is not reachable: %w", err)
}
time.Sleep(s.Poll)
}
}
// apply makes one team of the value.
func (s *Syncer) apply(ctx context.Context, name string, t Team) error {
if !teamName.MatchString(name) {
return fmt.Errorf("%q is not a team name", name)
}
if len(t.Repos) == 0 {
return errors.New("it lists no repository, so the organisation to make it in is not known")
}
org := ""
repos := map[string]bool{}
for _, r := range t.Repos {
m := repoName.FindStringSubmatch(r)
if m == nil {
return fmt.Errorf("%q is not owner/repo", r)
}
if org == "" {
org = m[1]
}
if m[1] != org {
return fmt.Errorf("%q is not in %s: a team's repositories are of one owner, the first's", r, org)
}
repos[r] = true
}
members := map[string]bool{}
for _, n := range t.Members {
if !nodeName.MatchString(n) {
return fmt.Errorf("%q is not a node's name", n)
}
members[AccountPrefix+n] = true
}
full := TeamPrefix + name
teams, err := s.teams(ctx, org)
if err != nil {
return err
}
var team *forgeTeam
for i := range teams {
if teams[i].Name == full {
team = &teams[i]
}
}
if team == nil {
var made forgeTeam
err := s.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/orgs/"+org+"/teams", map[string]any{
"name": full, "description": "Read-only team of agents' forge accounts (ADR 0365 §3); set by the gitea config value " + ConfigKey,
"permission": "read", "includes_all_repositories": false, "can_create_org_repo": false, "units_map": readUnits,
}, &made)
if err != nil {
return fmt.Errorf("the team could not be made: %w", err)
}
s.say("made the team %s in %s, reading only", full, org)
team = &made
} else if team.Permission != "read" {
if err := s.Forge.SendJSON(ctx, http.MethodPatch, fmt.Sprintf("/api/v1/teams/%d", team.ID), map[string]any{
"name": full, "permission": "read", "includes_all_repositories": false, "can_create_org_repo": false, "units_map": readUnits,
}, nil); err != nil {
return fmt.Errorf("the team could not be made read-only: %w", err)
}
s.say("made the team %s in %s read-only (it was %s)", full, org, team.Permission)
}
return s.sync(ctx, org, *team, repos, members)
}
// sync makes the team's repositories and members exactly the sets (nil is none).
func (s *Syncer) sync(ctx context.Context, org string, t forgeTeam, repos, members map[string]bool) error {
var errs []string
var haveRepos []forgeRepo
if err := s.list(ctx, fmt.Sprintf("/api/v1/teams/%d/repos", t.ID), &haveRepos); err != nil {
return err
}
held := map[string]bool{}
for _, r := range haveRepos {
held[r.FullName] = true
if !repos[r.FullName] {
if err := s.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/teams/%d/repos/%s", t.ID, r.FullName), nil, nil); err != nil {
errs = append(errs, err.Error())
continue
}
s.say("took %s from the team %s", r.FullName, t.Name)
}
}
for _, r := range sortedKeys(repos) {
if held[r] {
continue
}
if err := s.Forge.SendJSON(ctx, http.MethodPut, fmt.Sprintf("/api/v1/teams/%d/repos/%s", t.ID, r), nil, nil); err != nil {
errs = append(errs, fmt.Sprintf("%s could not be given to the team: %v", r, err))
continue
}
s.say("gave %s to the team %s, to read", r, t.Name)
}
var haveMembers []forgeMember
if err := s.list(ctx, fmt.Sprintf("/api/v1/teams/%d/members", t.ID), &haveMembers); err != nil {
return err
}
in := map[string]bool{}
for _, m := range haveMembers {
in[m.Login] = true
if !members[m.Login] {
if err := s.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/teams/%d/members/%s", t.ID, url.PathEscape(m.Login)), nil, nil); err != nil {
errs = append(errs, err.Error())
continue
}
s.say("took %s out of the team %s", m.Login, t.Name)
}
}
for _, m := range sortedKeys(members) {
if in[m] {
continue
}
if err := s.Forge.GetJSON(ctx, "/api/v1/users/"+url.PathEscape(m), nil); err != nil {
if errors.Is(err, forgehttp.ErrNotFound) {
s.say("skipped %s for the team %s: the account does not exist yet", m, t.Name)
continue
}
errs = append(errs, err.Error())
continue
}
if err := s.Forge.SendJSON(ctx, http.MethodPut, fmt.Sprintf("/api/v1/teams/%d/members/%s", t.ID, url.PathEscape(m)), nil, nil); err != nil {
errs = append(errs, fmt.Sprintf("%s could not be put in the team: %v", m, err))
continue
}
s.say("put %s in the team %s", m, t.Name)
}
if len(errs) > 0 {
return errors.New(strings.Join(errs, "; "))
}
return nil
}
func sortedKeys(m map[string]bool) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// list reads every page of a list answer into into (a pointer to a slice).
func (s *Syncer) list(ctx context.Context, path string, into any) error {
var all []json.RawMessage
for p := 1; ; p++ {
var got []json.RawMessage
if err := s.Forge.GetJSON(ctx, fmt.Sprintf("%s?limit=%d&page=%d", path, page, p), &got); err != nil {
return err
}
all = append(all, got...)
if len(got) < page {
break
}
}
raw, _ := json.Marshal(all)
return json.Unmarshal(raw, into)
}
func (s *Syncer) teams(ctx context.Context, org string) ([]forgeTeam, error) {
var out []forgeTeam
if err := s.list(ctx, "/api/v1/orgs/"+org+"/teams", &out); err != nil {
if errors.Is(err, forgehttp.ErrNotFound) {
return nil, fmt.Errorf("the organisation %s does not exist on the forge", org)
}
return nil, err
}
return out, nil
}
// orgs are the organisations on the forge, where a read team no longer in the value is found.
func (s *Syncer) orgs(ctx context.Context) ([]string, error) {
var out []struct {
Name string `json:"name"`
}
if err := s.list(ctx, "/api/v1/admin/orgs", &out); err != nil {
return nil, fmt.Errorf("the forge's organisations could not be read: %w", err)
}
names := make([]string, len(out))
for i, o := range out {
names[i] = o.Name
}
return names, nil
}
+333
View File
@@ -0,0 +1,333 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
"sync"
"testing"
"gitea/internal/forgehttp"
)
type fTeam struct {
ID int64
Org string
Name string
Perm string
Units map[string]string
Repos map[string]bool
Members map[string]bool
}
type fakeForge struct {
mu sync.Mutex
orgs []string
users map[string]bool
teams []*fTeam
next int64
}
func newFake(orgs []string, users ...string) *fakeForge {
f := &fakeForge{orgs: orgs, users: map[string]bool{}, next: 1}
for _, u := range users {
f.users[u] = true
}
return f
}
func (f *fakeForge) add(org, name, perm string, repos, members []string) *fTeam {
t := &fTeam{ID: f.next, Org: org, Name: name, Perm: perm, Repos: map[string]bool{}, Members: map[string]bool{}}
f.next++
for _, r := range repos {
t.Repos[r] = true
}
for _, m := range members {
t.Members[m] = true
}
f.teams = append(f.teams, t)
return t
}
func (f *fakeForge) byName(org, name string) *fTeam {
for _, t := range f.teams {
if t.Org == org && t.Name == name {
return t
}
}
return nil
}
var (
orgTeams = regexp.MustCompile(`^/api/v1/orgs/([^/]+)/teams$`)
teamPath = regexp.MustCompile(`^/api/v1/teams/(\d+)$`)
teamRepos = regexp.MustCompile(`^/api/v1/teams/(\d+)/repos(?:/(.+))?$`)
teamMems = regexp.MustCompile(`^/api/v1/teams/(\d+)/members(?:/(.+))?$`)
)
func (f *fakeForge) find(id string) *fTeam {
n, _ := strconv.ParseInt(id, 10, 64)
for _, t := range f.teams {
if t.ID == n {
return t
}
}
return nil
}
func keys(m map[string]bool) []string {
out := []string{}
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
func (f *fakeForge) ServeHTTP(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if u, p, ok := r.BasicAuth(); !ok || u != "mesh-admin" || p != "pw" {
http.Error(w, "no", http.StatusUnauthorized)
return
}
reply := func(status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if v != nil {
_ = json.NewEncoder(w).Encode(v)
}
}
paged := r.URL.Query().Get("page") != "" && r.URL.Query().Get("page") != "1"
var in map[string]any
_ = json.NewDecoder(r.Body).Decode(&in)
p := r.URL.Path
switch {
case p == "/api/v1/version":
reply(200, map[string]string{"version": "1"})
case p == "/api/v1/admin/orgs":
out := []map[string]string{}
for _, o := range f.orgs {
out = append(out, map[string]string{"name": o})
}
if paged {
out = nil
}
reply(200, out)
case strings.HasPrefix(p, "/api/v1/users/"):
if !f.users[strings.TrimPrefix(p, "/api/v1/users/")] {
reply(404, nil)
return
}
reply(200, map[string]string{})
case orgTeams.MatchString(p):
org := orgTeams.FindStringSubmatch(p)[1]
known := false
for _, o := range f.orgs {
known = known || o == org
}
if !known {
reply(404, nil)
return
}
if r.Method == http.MethodPost {
units, _ := in["units_map"].(map[string]any)
t := f.add(org, in["name"].(string), in["permission"].(string), nil, nil)
t.Units = map[string]string{}
for k, v := range units {
t.Units[k] = v.(string)
}
reply(201, map[string]any{"id": t.ID, "name": t.Name, "permission": t.Perm})
return
}
out := []map[string]any{}
for _, t := range f.teams {
if t.Org == org && !paged {
out = append(out, map[string]any{"id": t.ID, "name": t.Name, "permission": t.Perm})
}
}
reply(200, out)
case teamPath.MatchString(p):
t := f.find(teamPath.FindStringSubmatch(p)[1])
t.Perm = in["permission"].(string)
reply(200, nil)
case teamRepos.MatchString(p):
m := teamRepos.FindStringSubmatch(p)
t := f.find(m[1])
switch {
case m[2] == "" && r.Method == http.MethodGet:
out := []map[string]string{}
for _, k := range keys(t.Repos) {
out = append(out, map[string]string{"full_name": k})
}
if paged {
out = nil
}
reply(200, out)
case r.Method == http.MethodPut:
t.Repos[m[2]] = true
reply(204, nil)
case r.Method == http.MethodDelete:
delete(t.Repos, m[2])
reply(204, nil)
}
case teamMems.MatchString(p):
m := teamMems.FindStringSubmatch(p)
t := f.find(m[1])
switch {
case m[2] == "" && r.Method == http.MethodGet:
out := []map[string]string{}
for _, k := range keys(t.Members) {
out = append(out, map[string]string{"login": k})
}
if paged {
out = nil
}
reply(200, out)
case r.Method == http.MethodPut:
t.Members[m[2]] = true
reply(204, nil)
case r.Method == http.MethodDelete:
delete(t.Members, m[2])
reply(204, nil)
}
default:
reply(404, nil)
}
}
func syncer(t *testing.T, f *fakeForge) (*Syncer, *[]string) {
t.Helper()
srv := httptest.NewServer(f)
t.Cleanup(srv.Close)
pw := filepath.Join(t.TempDir(), "pw")
if err := os.WriteFile(pw, []byte("pw\n"), 0o600); err != nil {
t.Fatal(err)
}
var said []string
return &Syncer{Forge: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw},
Log: func(format string, a ...any) { said = append(said, sprintf(format, a...)) }}, &said
}
func sprintf(format string, a ...any) string { return fmt.Sprintf(format, a...) }
func equal(a, b []string) bool { return strings.Join(a, ",") == strings.Join(b, ",") }
// The first test of issue 758: a value makes a read-only team with exactly its repositories and members; a changed
// value takes a repository and a member away; a team that is not named agent-read-* is never touched.
func TestValueMakesReadTeamAndChangeRemoves(t *testing.T) {
f := newFake([]string{"novox"}, "agent-g14", "agent-ace")
other := f.add("novox", "Owners", "owner", []string{"novox/hq"}, []string{"agent-ace"})
s, said := syncer(t, f)
ctx := context.Background()
err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq", "novox/mesh-catalog"}, Members: []string{"g14"}}})
if err != nil {
t.Fatal(err)
}
team := f.byName("novox", "agent-read-all")
if team == nil || team.Perm != "read" {
t.Fatalf("team = %+v, want a read team", team)
}
for _, u := range team.Units {
if u != "read" {
t.Errorf("unit %q, want read only", u)
}
}
if !equal(keys(team.Repos), []string{"novox/hq", "novox/mesh-catalog"}) || !equal(keys(team.Members), []string{"agent-g14"}) {
t.Fatalf("repos %v members %v", keys(team.Repos), keys(team.Members))
}
before := len(*said)
if err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq", "novox/mesh-catalog"}, Members: []string{"g14"}}}); err != nil {
t.Fatal(err)
}
if len(*said) != before {
t.Errorf("a second run changed something: %v", (*said)[before:])
}
if err := s.Sync(ctx, Value{"all": {Repos: []string{"novox/hq"}, Members: nil}}); err != nil {
t.Fatal(err)
}
if !equal(keys(team.Repos), []string{"novox/hq"}) || len(team.Members) != 0 {
t.Fatalf("after change: repos %v members %v", keys(team.Repos), keys(team.Members))
}
// the team leaves the value: emptied, never deleted
if err := s.Sync(ctx, Value{}); err != nil {
t.Fatal(err)
}
if f.byName("novox", "agent-read-all") == nil || len(team.Repos) != 0 || len(team.Members) != 0 {
t.Fatalf("removed team not emptied: %+v", team)
}
if other.Perm != "owner" || !equal(keys(other.Repos), []string{"novox/hq"}) || !equal(keys(other.Members), []string{"agent-ace"}) {
t.Fatalf("a team without the prefix was touched: %+v", other)
}
for _, line := range *said {
if strings.Contains(line, "pw") {
t.Errorf("a line names the password: %q", line)
}
}
}
func TestPermissionIsCorrectedAndMissingAccountSkipped(t *testing.T) {
f := newFake([]string{"novox"}, "agent-g14")
team := f.add("novox", "agent-read-all", "write", nil, nil)
s, said := syncer(t, f)
err := s.Sync(context.Background(), Value{"all": {Repos: []string{"novox/hq"}, Members: []string{"g14", "later"}}})
if err != nil {
t.Fatal(err)
}
if team.Perm != "read" || !equal(keys(team.Members), []string{"agent-g14"}) {
t.Fatalf("team = %+v", team)
}
if !strings.Contains(strings.Join(*said, "\n"), "agent-later") {
t.Errorf("the missing account was not said: %v", *said)
}
}
func TestRefusesWhatIsNotAReadTeam(t *testing.T) {
f := newFake([]string{"novox", "other"})
s, _ := syncer(t, f)
for name, v := range map[string]Value{
"no repo": {"a": {Members: []string{"g14"}}},
"two owners": {"a": {Repos: []string{"novox/hq", "other/x"}}},
"bad repo": {"a": {Repos: []string{"hq"}}},
"bad member": {"a": {Repos: []string{"novox/hq"}, Members: []string{"../x"}}},
"unknown org": {"a": {Repos: []string{"nowhere/hq"}}},
"bad teamname": {"a b": {Repos: []string{"novox/hq"}}},
} {
if err := s.Sync(context.Background(), v); err == nil {
t.Errorf("%s: no error", name)
}
}
if len(f.teams) != 0 {
t.Errorf("teams made: %+v", f.teams)
}
}
func TestReadValue(t *testing.T) {
dir := t.TempDir()
p := filepath.Join(dir, "t.json")
_ = os.WriteFile(p, []byte(`{"agent-read-teams":{"a":{"repos":["o/r"],"members":["g14"]}}}`), 0o600)
v, err := ReadValue(p)
if err != nil || len(v["a"].Repos) != 1 || v["a"].Members[0] != "g14" {
t.Fatalf("%v %v", v, err)
}
_ = os.WriteFile(p, []byte(`{}`), 0o600)
if v, err := ReadValue(p); err != nil || len(v) != 0 {
t.Fatalf("%v %v", v, err)
}
_ = os.WriteFile(p, []byte(`{"agent-read-teams":{"a":{"repos":["o/r"],"write":true}}}`), 0o600)
if _, err := ReadValue(p); err == nil {
t.Fatal("an unknown field was accepted")
}
}
+72
View File
@@ -0,0 +1,72 @@
// agent-token: the forge's read credential for each node's agent (novox/hq ADR 0365 clauses 1, 3, 4 and 5; issue
// 737). A process the node's runtime launches and speaks MCP over stdio to, beside the module's other bundles.
//
// It serves one tool, gitea_agent_token, which the vault alone asks (ADR 0349 clause 10, issue 749): for each holder
// of the provision `gitea-read` the vault names — the claude-code module on one machine — it makes that machine's
// agent a forge account of its own, `agent-<node>`, no administrator and in no team, mints it a token whose one
// scope is `read:repository`, and answers the git credential lines for it, each holder's sealed to the key the vault
// made for the call. Asked again, it mints again and deletes the account's token from before: that is its rotation.
//
// stdout is the protocol; it says what it says on stderr, and never a token or the admin's password.
package main
import (
"fmt"
"os"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// ToolNames are the tools this bundle serves. The module's manifest lists no `tools` (see the npm-registry bundle
// for why); the README names this one, and a test holds the three together.
var ToolNames = []string{ToolName}
func main() {
if err := stdio.Serve("", Tools(MinterFromEnv)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// Tools are the bundle's tools over the minter the function gives, asked per call, so a manifest that does not set
// the environment is a refusal naming what is missing, not a bundle that will not start.
func Tools(minter func() (*Minter, error)) []stdio.Tool {
return []stdio.Tool{{
Name: ToolName,
Description: "The forge's read credential for each node's agent (novox/hq ADR 0365), answered to the vault " +
"alone: for each holder named, a forge account of the node's agent's own (agent-<node>), no administrator " +
"and in no team, and a token of scope read:repository minted for it anew, its git credential lines " +
"sealed to seal_to. Any other caller is refused and nothing is minted.",
Input: map[string]any{
"type": "object",
"properties": map[string]any{
"provision": map[string]any{"type": "string", "description": "the provision, " + Provision},
"seal_to": map[string]any{"type": "string", "description": "the key made for this call, base64"},
"holders": map[string]any{
"type": "array",
"description": "the provision's holders: each a consuming module on one machine",
"items": map[string]any{"type": "object", "properties": map[string]any{
"node": map[string]any{"type": "string"},
"module": map[string]any{"type": "string"},
}},
},
},
"required": []string{"provision", "seal_to", "holders"},
},
Run: func(args map[string]any) (any, error) {
caller := stdio.CallerOf(args)
if err := fromTheVault(caller); err != nil {
return nil, err
}
ask, err := readAsk(args)
if err != nil {
return nil, err
}
m, err := minter()
if err != nil {
return nil, err
}
return m.Answer(ask)
},
}}
}
+348
View File
@@ -0,0 +1,348 @@
package main
import (
"context"
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"regexp"
"sort"
"strings"
"sync"
"time"
"golang.org/x/crypto/nacl/box"
"gitea/internal/forgehttp"
)
const (
// ToolName is the issuing tool the vault asks (the offer's credential tool in module.json).
ToolName = "gitea_agent_token"
// Provision is what the tool issues: the forge's read credential, one per agent.
Provision = "gitea-read"
// Holder is the one consuming module answered: the agent's own module, which places the credential in the
// agent's home (ADR 0365 §4, ADR 0266). A holder of another module is not answered and keeps nothing.
Holder = "claude-code"
// TokenName is the token's name in the agent account's list, found and deleted by it at the next mint.
TokenName = "mesh-agent-read"
// AccountPrefix names an agent's account for its node (ADR 0365 §1).
AccountPrefix = "agent-"
// vaultModule is the vault's module: its account is `<node>.mesh-vault` (ADR 0259 §8's runs-as).
vaultModule = "mesh-vault"
// deadline bounds one call, inside the vault's ask.
deadline = 45 * time.Second
)
// TokenScopes are the only scopes an agent's token carries (ADR 0365 §4, §5): it reads repositories and nothing
// else, so the forge refuses it a push, a comment, a token of its own and any administration — whatever team the
// account is in.
var TokenScopes = []string{"read:repository"}
var nodeName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,31}$`)
// fromTheVault refuses every caller but the vault's own account (ADR 0349 clause 10): whoever else could call this
// tool would hold every agent's credential.
func fromTheVault(caller string) error {
node, module, ok := strings.Cut(caller, ".")
if ok && module == vaultModule && nodeName.MatchString(node) {
return nil
}
return fmt.Errorf("%s answers the vault alone (novox/hq ADR 0349 clause 10, ADR 0365); %q is not the vault, and "+
"nothing was minted", ToolName, caller)
}
// HolderAsk is one holder the vault named.
type HolderAsk struct {
Node string `json:"node"`
Module string `json:"module"`
}
// Ask is one call: the holders, and the key to seal each answer to.
type Ask struct {
SealTo [32]byte
Holders []HolderAsk
}
// readAsk reads the call's arguments, refusing anything that is not the vault's per-holder ask.
func readAsk(args map[string]any) (Ask, error) {
if p, _ := args["provision"].(string); p != Provision {
return Ask{}, fmt.Errorf("%s issues %s alone, not %q; nothing was minted", ToolName, Provision, p)
}
key, _ := args["seal_to"].(string)
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(key))
if err != nil || len(raw) != 32 {
return Ask{}, errors.New("seal_to is not the key of a call; nothing was minted")
}
var out Ask
copy(out.SealTo[:], raw)
list, ok := args["holders"].([]any)
if !ok || len(list) == 0 {
return Ask{}, fmt.Errorf("%s answers a value for each holder, and none was named; nothing was minted", ToolName)
}
seen := map[HolderAsk]bool{}
for _, item := range list {
m, _ := item.(map[string]any)
node, _ := m["node"].(string)
module, _ := m["module"].(string)
h := HolderAsk{Node: strings.TrimSpace(node), Module: strings.TrimSpace(module)}
if h.Node == "" || h.Module == "" || seen[h] {
return Ask{}, errors.New("each holder is a node and a module, named once; nothing was minted")
}
seen[h] = true
out.Holders = append(out.Holders, h)
}
return out, nil
}
// Minter mints the agents' tokens with the administrator's login, which sets the forge up and nothing else (ADR
// 0365 §2).
type Minter struct {
Forge forgehttp.Client
// Node is the forge's own machine: a holder there also reaches the forge at Forge.URL.
Node string
// OriginsFile holds the forge's public origins, one per line (https://<its route's name>): the origins every
// holder's credential is for.
OriginsFile string
// Password makes the password an agent's account is created with: random, sent once and kept nowhere.
Password func() string
Log func(format string, a ...any)
}
// MinterFromEnv is the minter as the manifest hands it to this bundle. It builds the administrator's client itself:
// the module's words reach all its bundles, and a word naming another bundle's token must not make this one act as
// another account.
func MinterFromEnv() (*Minter, error) {
m := &Minter{
Forge: forgehttp.Client{URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
User: os.Getenv("MESH_GITEA_ADMIN_USER"), PasswordFile: os.Getenv("MESH_GITEA_ADMIN_PASSWORD_FILE"),
Timeout: 20 * time.Second},
Node: os.Getenv("MESH_NODE"),
OriginsFile: os.Getenv("MESH_GITEA_ORIGINS_FILE"),
}
var missing []string
for _, kv := range [][2]string{{"MESH_GITEA_URL", m.Forge.URL}, {"MESH_GITEA_ADMIN_USER", m.Forge.User},
{"MESH_GITEA_ADMIN_PASSWORD_FILE", m.Forge.PasswordFile}, {"MESH_GITEA_ORIGINS_FILE", m.OriginsFile}} {
if kv[1] == "" {
missing = append(missing, kv[0])
}
}
if len(missing) > 0 {
return nil, fmt.Errorf("the forge cannot be reached: %s not set by the manifest; nothing was minted", strings.Join(missing, ", "))
}
return m, nil
}
// Sealed is one holder's credential as the vault takes it.
type Sealed struct {
Node string `json:"node"`
Module string `json:"module"`
Sealed string `json:"sealed"`
}
// Answer is what the tool answers: a sealed credential for each holder answered, and why each other was not. Never
// a token.
type Answer struct {
Holders []Sealed `json:"holders"`
Skipped map[string]string `json:"skipped,omitempty"`
}
// one call at a time: a mint deletes the account's token from before, and two at once would delete each other's.
var minting sync.Mutex
// Answer mints each holder's token and seals its credential lines. A holder that is not an agent's module is not
// answered; a failure for one holder fails the call, so the vault keeps nothing rather than a part.
func (m *Minter) Answer(ask Ask) (Answer, error) {
minting.Lock()
defer minting.Unlock()
ctx, cancel := context.WithTimeout(context.Background(), deadline)
defer cancel()
origins, err := m.origins()
if err != nil {
return Answer{}, err
}
out := Answer{Holders: []Sealed{}}
skip := func(h HolderAsk, why string) {
if out.Skipped == nil {
out.Skipped = map[string]string{}
}
out.Skipped[h.Module+" on "+h.Node] = why
}
for _, h := range ask.Holders {
if h.Module != Holder {
skip(h, fmt.Sprintf("only %s places the forge's read credential, in its agent's home (ADR 0365 §4)", Holder))
continue
}
if !nodeName.MatchString(h.Node) {
skip(h, "not a machine's name an account can be named for")
continue
}
reach := origins
if h.Node == m.Node {
reach = append(append([]string{}, origins...), m.Forge.URL)
}
if len(reach) == 0 {
skip(h, "the forge names no origin it is reached at from that machine")
continue
}
account := AccountPrefix + h.Node
if err := m.ensureAccount(ctx, account); err != nil {
return Answer{}, err
}
token, err := m.mint(ctx, account)
if err != nil {
return Answer{}, err
}
lines, err := credentialLines(account, token, reach)
token = ""
if err != nil {
return Answer{}, err
}
sealed, err := box.SealAnonymous(nil, []byte(lines), &ask.SealTo, rand.Reader)
lines = ""
if err != nil {
return Answer{}, fmt.Errorf("the credential for %s could not be sealed: %w", account, err)
}
out.Holders = append(out.Holders, Sealed{Node: h.Node, Module: h.Module, Sealed: base64.StdEncoding.EncodeToString(sealed)})
m.say("minted a read token for %s (%s on %s); its token from before is deleted", account, h.Module, h.Node)
}
return out, nil
}
// origins are the forge's public origins, as its route names them.
func (m *Minter) origins() ([]string, error) {
raw, err := os.ReadFile(m.OriginsFile)
if err != nil {
return nil, fmt.Errorf("the forge's origins cannot be read (%s); nothing was minted: %w", m.OriginsFile, err)
}
var out []string
for _, line := range strings.Split(string(raw), "\n") {
line = strings.TrimRight(strings.TrimSpace(line), "/")
if line == "" {
continue
}
u, err := url.Parse(line)
if err != nil || (u.Scheme != "https" && u.Scheme != "http") || u.Host == "" || u.User != nil || u.Path != "" {
return nil, fmt.Errorf("%s names %q, which is not an origin (scheme://host); nothing was minted", m.OriginsFile, line)
}
out = append(out, line)
}
return out, nil
}
// credentialLines are the lines of git's credential store for one account at each origin: what the agent's module
// places, and git reads by protocol and host.
func credentialLines(account, token string, origins []string) (string, error) {
seen := map[string]bool{}
var lines []string
for _, o := range origins {
u, err := url.Parse(strings.TrimRight(o, "/"))
if err != nil || u.Host == "" {
return "", fmt.Errorf("%q is not an origin", o)
}
u.Path, u.RawQuery, u.Fragment = "", "", ""
u.User = url.UserPassword(account, token)
if line := u.String(); !seen[line] {
seen[line] = true
lines = append(lines, line)
}
}
sort.Strings(lines)
return strings.Join(lines, "\n") + "\n", nil
}
// gitea's account as the admin reads it.
type forgeUser struct {
IsAdmin bool `json:"is_admin"`
Active bool `json:"active"`
ProhibitLogin bool `json:"prohibit_login"`
Restricted bool `json:"restricted"`
}
// ensureAccount makes the agent's account, plain, when there is none, and puts back what was changed by hand: never
// an administrator, active, able to use its token. It adds it to no team: what it may read beyond public
// repositories is the operator's to grant (ADR 0365 §3).
func (m *Minter) ensureAccount(ctx context.Context, account string) error {
var u forgeUser
err := m.Forge.GetJSON(ctx, "/api/v1/users/"+account, &u)
switch {
case errors.Is(err, forgehttp.ErrNotFound):
password := m.password()
err := m.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/admin/users", map[string]any{
"username": account, "email": account + "@noreply.localhost", "password": password,
"must_change_password": false, "visibility": "private",
}, nil)
password = ""
if err != nil {
return fmt.Errorf("the account %s could not be made: %w", account, err)
}
m.say("made the forge account %s for that machine's agent, in no team", account)
return nil
case err != nil:
return fmt.Errorf("the account %s could not be read: %w", account, err)
}
if u.IsAdmin || !u.Active || u.ProhibitLogin {
if err := m.Forge.SendJSON(ctx, http.MethodPatch, "/api/v1/admin/users/"+account, map[string]any{
"login_name": account, "source_id": 0, "admin": false, "active": true, "prohibit_login": false,
}, nil); err != nil {
return fmt.Errorf("the account %s could not be made a plain account again: %w", account, err)
}
m.say("the account %s was changed by hand (administrator %t, active %t, login prohibited %t); made a plain account again",
account, u.IsAdmin, u.Active, u.ProhibitLogin)
}
return nil
}
type forgeToken struct {
ID int64 `json:"id"`
Name string `json:"name"`
SHA1 string `json:"sha1"`
}
// mint deletes every token of the account by this module's name, by its id (a name held twice is cleared, not
// refused), and mints one with the read scope alone.
func (m *Minter) mint(ctx context.Context, account string) (string, error) {
var held []forgeToken
if err := m.Forge.GetJSON(ctx, "/api/v1/users/"+account+"/tokens?limit=50", &held); err != nil {
return "", fmt.Errorf("the tokens of %s could not be read: %w", account, err)
}
for _, t := range held {
if t.Name != TokenName {
continue
}
if err := m.Forge.SendJSON(ctx, http.MethodDelete, fmt.Sprintf("/api/v1/users/%s/tokens/%d", account, t.ID), nil, nil); err != nil &&
!errors.Is(err, forgehttp.ErrNotFound) {
return "", fmt.Errorf("the token of %s from before could not be deleted: %w", account, err)
}
}
var made forgeToken
if err := m.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/users/"+account+"/tokens",
map[string]any{"name": TokenName, "scopes": TokenScopes}, &made); err != nil {
return "", fmt.Errorf("no token could be minted for %s: %w", account, err)
}
if strings.TrimSpace(made.SHA1) == "" {
return "", fmt.Errorf("the forge answered no token for %s", account)
}
return strings.TrimSpace(made.SHA1), nil
}
func (m *Minter) password() string {
if m.Password != nil {
return m.Password()
}
b := make([]byte, 24)
_, _ = rand.Read(b)
return base64.RawURLEncoding.EncodeToString(b)
}
func (m *Minter) say(format string, a ...any) {
if m.Log != nil {
m.Log(format, a...)
return
}
fmt.Fprintf(os.Stderr, format+"\n", a...)
}
+376
View File
@@ -0,0 +1,376 @@
package main
import (
"crypto/rand"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"regexp"
"slices"
"strings"
"sync"
"testing"
"golang.org/x/crypto/nacl/box"
"gitea/internal/forgehttp"
)
const adminPassword = "admin-password-never-said"
// fakeForge is gitea's API as far as this bundle uses it, as the administrator's login reaches it.
type fakeForge struct {
mu sync.Mutex
users map[string]*forgeUser
tokens map[string][]forgeToken
next int64
calls []string
made map[string]map[string]any
minted map[string][]string // account -> scopes of its last mint
failMint bool
}
func newFakeForge() *fakeForge {
return &fakeForge{users: map[string]*forgeUser{}, tokens: map[string][]forgeToken{}, made: map[string]map[string]any{},
minted: map[string][]string{}, next: 1}
}
var tokenPath = regexp.MustCompile(`^/api/v1/users/([^/]+)/tokens(?:/(\d+))?$`)
func (f *fakeForge) ServeHTTP(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
user, pass, ok := r.BasicAuth()
if !ok || user != "mesh-admin" || pass != adminPassword {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
f.calls = append(f.calls, r.Method+" "+r.URL.Path)
body, _ := io.ReadAll(r.Body)
var in map[string]any
_ = json.Unmarshal(body, &in)
reply := func(status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if v != nil {
_ = json.NewEncoder(w).Encode(v)
}
}
switch {
case r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, "/api/v1/users/") && strings.Count(r.URL.Path, "/") == 4:
u, ok := f.users[strings.TrimPrefix(r.URL.Path, "/api/v1/users/")]
if !ok {
reply(404, map[string]any{"message": "user does not exist"})
return
}
reply(200, map[string]any{"is_admin": u.IsAdmin, "active": u.Active, "prohibit_login": u.ProhibitLogin})
case r.Method == http.MethodPost && r.URL.Path == "/api/v1/admin/users":
name, _ := in["username"].(string)
f.users[name] = &forgeUser{Active: true}
f.made[name] = in
reply(201, map[string]any{"login": name})
case r.Method == http.MethodPatch && strings.HasPrefix(r.URL.Path, "/api/v1/admin/users/"):
u := f.users[strings.TrimPrefix(r.URL.Path, "/api/v1/admin/users/")]
u.IsAdmin, _ = in["admin"].(bool)
u.Active, _ = in["active"].(bool)
u.ProhibitLogin, _ = in["prohibit_login"].(bool)
reply(200, map[string]any{})
case tokenPath.MatchString(r.URL.Path):
m := tokenPath.FindStringSubmatch(r.URL.Path)
account := m[1]
switch r.Method {
case http.MethodGet:
reply(200, f.tokens[account])
case http.MethodDelete:
kept := f.tokens[account][:0]
for _, t := range f.tokens[account] {
if fmt.Sprint(t.ID) != m[2] {
kept = append(kept, t)
}
}
f.tokens[account] = kept
reply(204, nil)
case http.MethodPost:
if f.failMint {
reply(500, map[string]any{"message": "the forge is unwell"})
return
}
name, _ := in["name"].(string)
for _, t := range f.tokens[account] {
if t.Name == name {
reply(400, map[string]any{"message": "access token name has been used already"})
return
}
}
var scopes []string
for _, s := range in["scopes"].([]any) {
scopes = append(scopes, s.(string))
}
f.minted[account] = scopes
b := make([]byte, 20)
_, _ = rand.Read(b)
t := forgeToken{ID: f.next, Name: name, SHA1: fmt.Sprintf("%x", b)}
f.next++
f.tokens[account] = append(f.tokens[account], forgeToken{ID: t.ID, Name: t.Name})
reply(201, t)
}
default:
reply(404, map[string]any{"message": "not here: " + r.URL.Path})
}
}
type harness struct {
forge *fakeForge
server *httptest.Server
minter *Minter
public *[32]byte
private *[32]byte
logged *[]string
}
func newHarness(t *testing.T) harness {
t.Helper()
f := newFakeForge()
srv := httptest.NewServer(f)
t.Cleanup(srv.Close)
dir := t.TempDir()
pw := filepath.Join(dir, "admin.secret")
origins := filepath.Join(dir, "origins")
if err := os.WriteFile(pw, []byte(adminPassword+"\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(origins, []byte("https://git.example.test\n"), 0o644); err != nil {
t.Fatal(err)
}
pub, priv, err := box.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
var logged []string
m := &Minter{Forge: forgehttp.Client{URL: srv.URL, User: "mesh-admin", PasswordFile: pw}, Node: "anchor",
OriginsFile: origins, Log: func(format string, a ...any) { logged = append(logged, fmt.Sprintf(format, a...)) }}
return harness{forge: f, server: srv, minter: m, public: pub, private: priv, logged: &logged}
}
func (h harness) open(t *testing.T, sealed string) string {
t.Helper()
raw, err := base64.StdEncoding.DecodeString(sealed)
if err != nil {
t.Fatal(err)
}
out, ok := box.OpenAnonymous(nil, raw, h.public, h.private)
if !ok {
t.Fatal("a holder's value is not sealed to the call's key")
}
return string(out)
}
func (h harness) args(holders ...HolderAsk) map[string]any {
list := []any{}
for _, x := range holders {
list = append(list, map[string]any{"node": x.Node, "module": x.Module})
}
return map[string]any{"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:]), "holders": list}
}
func TestOnlyTheVaultIsAnsweredAndNothingIsMintedForAnyoneElse(t *testing.T) {
h := newHarness(t)
for _, caller := range []string{"controller", "novox.node-tools", "provisioner.anchor", "anchor.claude-code",
"mesh-vault", "anchor.mesh-vault.x", "", "unproven"} {
if err := fromTheVault(caller); err == nil {
t.Errorf("%q was taken for the vault", caller)
}
}
if err := fromTheVault("novox.mesh-vault"); err != nil {
t.Errorf("the vault was refused: %v", err)
}
// Through the tool as the runtime serves it: a call no runtime named a caller for is the vault's to nobody.
tools := Tools(func() (*Minter, error) { return h.minter, nil })
if _, err := tools[0].Run(h.args(HolderAsk{"anchor", "claude-code"})); err == nil || !strings.Contains(err.Error(), "vault alone") {
t.Fatalf("an unproven caller was answered: %v", err)
}
if len(h.forge.calls) != 0 {
t.Fatalf("the forge was reached for a caller that is not the vault: %v", h.forge.calls)
}
}
func TestEachAgentGetsAnAccountOfItsOwnAndAReadTokenSealedForItAlone(t *testing.T) {
h := newHarness(t)
ask, err := readAsk(h.args(HolderAsk{"anchor", "claude-code"}, HolderAsk{"laptop", "claude-code"}, HolderAsk{"laptop", "lab"}))
if err != nil {
t.Fatal(err)
}
got, err := h.minter.Answer(ask)
if err != nil {
t.Fatal(err)
}
if len(got.Holders) != 2 {
t.Fatalf("answered %d holders, want the two agents: %+v", len(got.Holders), got)
}
if !strings.Contains(got.Skipped["lab on laptop"], Holder) {
t.Errorf("a holder that is not an agent's module was not said: %v", got.Skipped)
}
lines := map[string]string{}
for _, s := range got.Holders {
lines[s.Node] = h.open(t, s.Sealed)
}
anchorTok := tokenIn(t, lines["anchor"], "agent-anchor")
laptopTok := tokenIn(t, lines["laptop"], "agent-laptop")
if anchorTok == laptopTok {
t.Fatal("two agents were handed one token")
}
// The forge's own machine reaches it at its own address too; another machine only at its public origin.
if !strings.Contains(lines["anchor"], "https://agent-anchor:"+anchorTok+"@git.example.test\n") ||
!strings.Contains(lines["anchor"], strings.Replace(h.server.URL, "http://", "http://agent-anchor:"+anchorTok+"@", 1)) {
t.Errorf("anchor's credential lines: %q", strings.ReplaceAll(lines["anchor"], anchorTok, "<token>"))
}
if lines["laptop"] != "https://agent-laptop:"+laptopTok+"@git.example.test\n" {
t.Errorf("laptop's credential lines: %q", strings.ReplaceAll(lines["laptop"], laptopTok, "<token>"))
}
for _, account := range []string{"agent-anchor", "agent-laptop"} {
u := h.forge.users[account]
if u == nil || u.IsAdmin {
t.Errorf("%s: %+v, want a plain account", account, u)
}
if made := h.forge.made[account]; made["must_change_password"] != false || made["password"] == "" {
t.Errorf("%s was made with %v", account, made)
}
if !slices.Equal(h.forge.minted[account], []string{"read:repository"}) {
t.Errorf("%s's token carries %v, want read:repository alone", account, h.forge.minted[account])
}
}
for _, c := range h.forge.calls {
if strings.Contains(c, "/teams") || strings.Contains(c, "/orgs/") || strings.Contains(c, "collaborators") {
t.Errorf("an agent's account was given access (%s): it starts in no team, the operator grants (ADR 0365 §3)", c)
}
}
if _, made := h.forge.users["agent-laptop-lab"]; made || len(h.forge.users) != 2 {
t.Errorf("accounts made: %v", h.forge.users)
}
said := strings.Join(*h.logged, "\n")
if strings.Contains(said, anchorTok) || strings.Contains(said, laptopTok) {
t.Fatal("a token was said in the log")
}
}
func TestAskedAgainTheTokenIsMintedAnewAndTheOneFromBeforeDeleted(t *testing.T) {
h := newHarness(t)
ask, _ := readAsk(h.args(HolderAsk{"laptop", "claude-code"}))
first, err := h.minter.Answer(ask)
if err != nil {
t.Fatal(err)
}
h.forge.tokens["agent-laptop"] = append(h.forge.tokens["agent-laptop"], forgeToken{ID: 99, Name: "a-persons-own"})
again, err := h.minter.Answer(ask)
if err != nil {
t.Fatal(err)
}
if h.open(t, first.Holders[0].Sealed) == h.open(t, again.Holders[0].Sealed) {
t.Fatal("asked again, the same token was answered: it was not rotated")
}
var names []string
for _, tok := range h.forge.tokens["agent-laptop"] {
names = append(names, tok.Name)
}
slices.Sort(names)
if !slices.Equal(names, []string{"a-persons-own", TokenName}) {
t.Fatalf("the account holds %v: the token from before is not deleted, or another was", names)
}
}
func TestAnAccountMadeAnAdministratorByHandIsMadePlainAgain(t *testing.T) {
h := newHarness(t)
h.forge.users["agent-laptop"] = &forgeUser{IsAdmin: true, Active: true}
ask, _ := readAsk(h.args(HolderAsk{"laptop", "claude-code"}))
if _, err := h.minter.Answer(ask); err != nil {
t.Fatal(err)
}
if h.forge.users["agent-laptop"].IsAdmin {
t.Fatal("the agent's account is still an administrator")
}
}
func TestAFailureSaysNeitherTheAdminsPasswordNorATokenAndAnswersNothing(t *testing.T) {
h := newHarness(t)
h.forge.failMint = true
ask, _ := readAsk(h.args(HolderAsk{"anchor", "claude-code"}, HolderAsk{"laptop", "claude-code"}))
got, err := h.minter.Answer(ask)
if err == nil || len(got.Holders) != 0 {
t.Fatalf("a failed mint answered %+v, %v", got, err)
}
if strings.Contains(err.Error(), adminPassword) {
t.Fatal("the admin's password was said in an error")
}
// A wrong password is a refusal, said without it.
if err := os.WriteFile(h.minter.Forge.PasswordFile, []byte("wrong"), 0o600); err != nil {
t.Fatal(err)
}
h.forge.failMint = false
if _, err := h.minter.Answer(ask); err == nil || strings.Contains(err.Error(), "wrong") {
t.Fatalf("a refused login: %v", err)
}
}
func TestTheAskIsReadStrictly(t *testing.T) {
h := newHarness(t)
for name, args := range map[string]map[string]any{
"another provision": {"provision": "git", "seal_to": base64.StdEncoding.EncodeToString(h.public[:]),
"holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
"no key": {"provision": Provision, "holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
"short key": {"provision": Provision, "seal_to": "AAAA", "holders": []any{map[string]any{"node": "a", "module": "claude-code"}}},
"no holders": {"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:])},
"a holder twice": {"provision": Provision, "seal_to": base64.StdEncoding.EncodeToString(h.public[:]),
"holders": []any{map[string]any{"node": "a", "module": "claude-code"}, map[string]any{"node": "a", "module": "claude-code"}}},
} {
if _, err := readAsk(args); err == nil {
t.Errorf("%s was taken", name)
}
}
}
func TestTheReadmeNamesWhatTheBundleServes(t *testing.T) {
var served []string
for _, tool := range Tools(MinterFromEnv) {
served = append(served, tool.Name)
}
if !slices.Equal(served, ToolNames) {
t.Fatalf("served %v, named %v", served, ToolNames)
}
readme, err := os.ReadFile("../../README.md")
if err != nil {
t.Fatal(err)
}
for _, n := range ToolNames {
if !strings.Contains(string(readme), "`"+n+"`") {
t.Errorf("the README does not name %s", n)
}
}
manifest, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(manifest), `"tool": "`+ToolName+`"`) || !strings.Contains(string(manifest), `"name": "`+Provision+`"`) {
t.Error("the manifest does not offer the provision with this tool as its issuer")
}
}
// tokenIn is the token of account's credential lines, all one.
func tokenIn(t *testing.T, lines, account string) string {
t.Helper()
re := regexp.MustCompile(regexp.QuoteMeta(account) + `:([0-9a-f]{40})@`)
all := re.FindAllStringSubmatch(lines, -1)
if len(all) == 0 {
t.Fatalf("no credential for %s in the lines", account)
}
for _, m := range all {
if m[1] != all[0][1] {
t.Fatalf("%s's lines carry two tokens", account)
}
}
return all[0][1]
}
+120
View File
@@ -0,0 +1,120 @@
package main
// The commit and the reads act as the tools' own account (novox/hq ADR 0365 §2, issue 469): the token the module
// keeps for `mesh-tools`, never the admin login. That account is no repository's administrator, so the forge
// refuses it the protection rules (403, as gitea's reqAdmin does); the commit then judges an existing branch by the
// protection the forge says it applies, and leaves a branch not made yet to the forge's own judgement.
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"gitea/internal/forgehttp"
)
const toolsToken = "tools-token-never-shown"
// asTools serves the fake forge to a bundle holding the tools' token: only that token is taken, the rules are
// refused as to a non-administrator, and the branches named protected say so as the forge does.
func asTools(t *testing.T, f *fakeForge, protected map[string]string, scopes string) (*Forge, *[]string) {
dir := t.TempDir()
file := filepath.Join(dir, "token")
if err := os.WriteFile(file, []byte(toolsToken+"\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(file+".scopes", []byte(scopes+"\n"), 0o600); err != nil {
t.Fatal(err)
}
var seen []string
const repo = "/api/v1/repos/acme/app"
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
seen = append(seen, r.Method+" "+r.URL.Path)
if _, _, basic := r.BasicAuth(); basic || r.Header.Get("Authorization") != "token "+toolsToken {
http.Error(w, "unauthorized", 401)
return
}
if r.URL.Path == repo+"/branch_protections" {
http.Error(w, `{"message":"token does not have the administration of this repository"}`, 403)
return
}
if name := strings.TrimPrefix(r.URL.Path, repo+"/branches/"); r.Method == "GET" && name != r.URL.Path {
if rule, ok := protected[name]; ok {
_, _ = w.Write([]byte(`{"name":"` + name + `","commit":{"id":"c0"},"protected":true,"effective_branch_protection_name":"` + rule + `"}`))
return
}
}
r.SetBasicAuth("mesh-admin", f.password) // the fake's own door, past the token checked above
f.handle(w, r)
}))
t.Cleanup(srv.Close)
return &Forge{Client: forgehttp.Client{URL: srv.URL, TokenFile: file, Account: "mesh-tools"}}, &seen
}
func TestTheToolsAccountCommitsOnANewBranchWithItsTokenThoughTheRulesAreNotItsToRead(t *testing.T) {
f := newFake(t)
forge, _ := asTools(t, f, nil, "read:user,write:issue,write:repository for mesh-tools")
got, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": "fix/365",
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
if err != nil {
t.Fatal(err)
}
if len(f.writes) != 1 || !got.Created {
t.Fatalf("%d writes, %+v", len(f.writes), got)
}
}
func TestTheToolsAccountIsRefusedAProtectedBranchTheForgeSaysItGuardsAndTheDefaultBranch(t *testing.T) {
for branch, rule := range map[string]string{"release/1.0": "release/*", "fix/1-old": "fix/1-old", "main": ""} {
f := newFake(t)
protected := map[string]string{}
if rule != "" {
protected[branch] = rule
}
forge, _ := asTools(t, f, protected, "read:user,write:issue,write:repository for mesh-tools")
_, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": branch,
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
if err == nil || !strings.Contains(err.Error(), "refused") || !strings.Contains(err.Error(), "ADR 0266") {
t.Errorf("%s: %v", branch, err)
}
if rule != "" && !strings.Contains(err.Error(), rule) {
t.Errorf("%s: the rule %s is not named: %v", branch, rule, err)
}
if len(f.writes)+len(f.patches) != 0 {
t.Errorf("%s: something was written", branch)
}
}
}
func TestATokenKeptForTheAdministratorIsNeverSent(t *testing.T) {
f := newFake(t)
// What the module kept before ADR 0365: the admin's token, its scopes line naming no account.
forge, seen := asTools(t, f, nil, "read:user,write:issue,write:repository")
_, err := commit(context.Background(), forge, ask(t, map[string]any{"branch": "fix/365",
"files": filesArg(map[string]any{"path": "x", "content": "x"})}))
if err == nil || !strings.Contains(err.Error(), "not mesh-tools's") {
t.Fatalf("%v", err)
}
if len(*seen) != 0 {
t.Fatalf("the forge was asked %v", *seen)
}
}
func TestTheBundleHandedATokenFileNeedsNoAdminLogin(t *testing.T) {
t.Setenv("MESH_GITEA_URL", "http://127.0.0.1:3000")
t.Setenv("MESH_GITEA_TOKEN_FILE", "/state/token")
t.Setenv("MESH_GITEA_TOOLS_USER", "")
t.Setenv("MESH_GITEA_ADMIN_USER", "")
t.Setenv("MESH_GITEA_ADMIN_PASSWORD_FILE", "")
f, err := ForgeFromEnv()
if err != nil {
t.Fatal(err)
}
if f.TokenFile != "/state/token" || f.Account != "mesh-tools" || f.User != "" || f.PasswordFile != "" {
t.Fatalf("%+v", f.Client)
}
}
+57 -8
View File
@@ -9,6 +9,8 @@ import (
"net/url"
"strings"
"unicode/utf8"
"gitea/internal/forgehttp"
)
// gitea_commit (novox/hq issue 375): one commit on a branch of its own, so an agent follows the branch / pull
@@ -33,6 +35,11 @@ type CommitAsk struct {
Message string
Files []FileChange
Patch string // a unified diff, as git diff writes it, instead of Files
// PatchFile is the name of a file in the spool directory holding the patch (novox/hq issue 852), with the
// SHA-256 its caller says it has; commitAsk reads it into Patch.
PatchFile string
PatchSHA256 string
patchFileGiven bool
}
// Committed is what gitea_commit made.
@@ -67,6 +74,11 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
a := CommitAsk{Owner: text(args, "owner"), Repo: text(args, "repo"), Branch: text(args, "branch"),
Base: text(args, "base"), Message: strings.TrimSpace(raw(args, "message"))}
a.Patch = raw(args, "patch")
if v, ok := args["patch_file"]; ok && v != nil {
a.patchFileGiven = true
a.PatchFile = raw(args, "patch_file")
}
a.PatchSHA256 = strings.ToLower(strings.TrimSpace(raw(args, "patch_sha256")))
if a.Branch == "" {
return a, refuse("branch is required: the branch the commit goes on, a branch of its own (created from base when absent). Nothing was sent")
}
@@ -109,11 +121,23 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
a.Files = append(a.Files, fc)
}
}
sources := 0
for _, g := range []bool{len(a.Files) > 0, strings.TrimSpace(a.Patch) != "", a.patchFileGiven} {
if g {
sources++
}
}
switch {
case len(a.Files) == 0 && strings.TrimSpace(a.Patch) == "":
return a, refuse("give files (each {path, content | content_base64 | delete: true}) or patch (a unified diff, as git diff writes it). Nothing was sent")
case len(a.Files) > 0 && strings.TrimSpace(a.Patch) != "":
return a, refuse("give files or patch, not both: one commit carries one of them. Nothing was sent")
case sources == 0:
return a, refuse("give files (each {path, content | content_base64 | delete: true}), patch (a unified diff, as git diff writes it) or patch_file (the name of a patch in the spool directory). Nothing was sent")
case sources > 1:
return a, refuse("give files, patch or patch_file, not more than one: one commit carries one of them. Nothing was sent")
case a.PatchSHA256 != "" && !a.patchFileGiven:
return a, refuse("patch_sha256 goes with patch_file only. Nothing was sent")
case a.patchFileGiven:
if err := checkPatchFileAsk(a); err != nil {
return a, err
}
case len(a.Files) > maxFiles:
return a, refuse("%d files is more than the %d one commit carries. Nothing was sent", len(a.Files), maxFiles)
case len(a.Patch) > maxPatchBytes:
@@ -132,13 +156,13 @@ func readCommitAsk(args map[string]any) (CommitAsk, error) {
return a, refuse("%s is named twice in files. Nothing was sent", fc.Path)
}
seen[fc.Path] = true
given := 0
have := 0
for _, g := range []bool{fc.Content != nil, fc.ContentBase64 != nil, fc.Delete} {
if g {
given++
have++
}
}
if given != 1 {
if have != 1 {
return a, refuse("files[%d] (%s) needs exactly one of content, content_base64 or delete: true. Nothing was sent", i, fc.Path)
}
if fc.Content != nil && !utf8.ValidString(*fc.Content) {
@@ -211,7 +235,32 @@ func commit(ctx context.Context, f *Forge, a CommitAsk) (*Committed, error) {
// Read the rules, or refuse: a branch the rules could not be read for is never written (fails closed).
var rules []Protection
if err := f.GetJSON(ctx, repo+"/branch_protections", &rules); err != nil {
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
var status *forgehttp.StatusError
if !errors.As(err, &status) || status.Status != http.StatusForbidden {
return nil, fmt.Errorf("the protection rules of %s/%s could not be read, so the commit was not judged and nothing was sent: %w", a.Owner, a.Repo, err)
}
// The rules are a repository administrator's to read, and the tools' account is none (novox/hq ADR 0365
// §2). What it may read is the protection the forge applies to an existing branch: one protected is
// refused here. A branch not made yet is covered by no rule this account can see, and the forge itself
// judges the commit that makes it against every rule, refusing one a rule guards.
if err := targetRefused(a.Branch, r.DefaultBranch, nil); err != nil {
return nil, err
}
b, err := f.branch(ctx, repo, a.Branch)
if err != nil {
return nil, fmt.Errorf("whether a protection rule guards %s of %s/%s could not be read, so nothing was sent: %w", a.Branch, a.Owner, a.Repo, err)
}
if b != nil && b.Protected {
rule := b.Rule
if rule == "" {
rule = a.Branch
}
rules = []Protection{{RuleName: rule}}
if !ruleCovers(rule, a.Branch) {
return nil, refuse("a commit on %s is refused: the protection rule %s guards it (novox/hq ADR 0266). Put it on a "+
"branch of its own. Nothing was sent", a.Branch, rule)
}
}
}
if err := targetRefused(a.Branch, r.DefaultBranch, rules); err != nil {
return nil, err
+6 -3
View File
@@ -11,8 +11,8 @@ import (
"gitea/internal/forgehttp"
)
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the forge's admin account
// (internal/forgehttp, shared with the npm-registry bundle beside it).
// The forge as this bundle reaches it: its own HTTP interface on the machine, as the tools' own account (novox/hq
// ADR 0365 §2; internal/forgehttp, shared with the npm-registry bundle beside it).
// Forge is the forge's HTTP interface.
type Forge struct {
@@ -91,12 +91,15 @@ func (p Protection) name() string {
return p.BranchName
}
// Branch is one branch with the commit it points at.
// Branch is one branch with the commit it points at, and the protection the forge applies to it: what an account
// that may not read the rules themselves (the tools' account, novox/hq ADR 0365) knows of them.
type Branch struct {
Name string `json:"name"`
Commit struct {
ID string `json:"id"`
} `json:"commit"`
Protected bool `json:"protected"`
Rule string `json:"effective_branch_protection_name"`
}
// branch is the named branch, nil when the forge has none of that name.
+7 -1
View File
@@ -619,11 +619,17 @@ func TestToolsAreTheOnesTheReadmeNamesAndTheManifestBuilds(t *testing.T) {
if len(a.Loads) != 1 || a.Loads[0] != a.Binary {
t.Errorf("the bundle loads %v, not its binary %s", a.Loads, a.Binary)
}
for _, k := range []string{"MESH_GITEA_URL", "MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD_FILE"} {
// The tools' own account, never the admin login (novox/hq ADR 0365 §2).
for _, k := range []string{"MESH_GITEA_URL", "MESH_GITEA_TOOLS_USER", "MESH_GITEA_TOKEN_FILE"} {
if a.Env[k] == "" {
t.Errorf("the bundle is not given %s", k)
}
}
for _, k := range []string{"MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD_FILE"} {
if a.Env[k] != "" {
t.Errorf("the bundle is given %s: it acts as the tools' account, not the administrator", k)
}
}
}
}
if !found {
+7 -3
View File
@@ -55,7 +55,9 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
"the branch holds them — or patch, a unified diff as git diff writes it, which the forge applies. " +
"At most 200 files, 4 MiB each and 16 MiB together, or a 4 MiB patch. Answers the commit it made and " +
"the parent the forge made it on, read back from the branch, and says when the branch moved under " +
"it. Replaces git commit and git " +
"it. A patch too large to type goes as patch_file with patch_sha256: the name of a file in the " +
"spool directory (read once, applied as patch is, removed afterwards, never read from anywhere " +
"else). Replaces git commit and git " +
"push of a feature branch. (a)",
Input: map[string]any{
"type": "object",
@@ -79,7 +81,9 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
"required": []string{"path"},
},
},
"patch": str("a unified diff of the change, as git diff or git format-patch writes it; give this or files"),
"patch": str("a unified diff of the change, as git diff or git format-patch writes it; give this, patch_file or files"),
"patch_file": str("for a patch too large to type: the NAME (32 to 64 characters of 0-9a-f, chosen at random by the caller, no path) of a file the caller wrote into the forge patch spool directory (the module's forge-patches directory); needs patch_sha256; give this, patch or files"),
"patch_sha256": str("with patch_file: the file's SHA-256, in hex; the commit is refused, and nothing is sent, when the file's differs"),
},
"required": []string{"owner", "repo", "branch", "message"},
},
@@ -88,7 +92,7 @@ func Tools(forge func() (*Forge, error)) []stdio.Tool {
if err != nil {
return nil, err
}
return commit(ctx, f, a)
return commitAsk(ctx, f, os.Getenv(patchDirEnv), a)
}),
},
{
+109
View File
@@ -0,0 +1,109 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"io"
"os"
"path/filepath"
"regexp"
"strings"
"syscall"
)
// A patch too large to type into a call is handed over as a file (novox/hq issue 852). The caller writes it into
// one spool directory the module declares (mode 0733: anyone writes, nobody lists) and names it in patch_file,
// with the file's SHA-256. The tool runs as the machine's account, so it reads nothing but that directory: the
// name is 32 to 64 characters of [0-9a-f], so it holds no separator and no dot; the file must be a regular file
// (never a symlink); and its hash must be the caller's, because without the sticky bit another writer could have
// replaced it. The file is removed once taken, whether the commit was made or not.
const patchDirEnv = "MESH_GITEA_PATCH_DIR"
var (
patchFileName = regexp.MustCompile(`^[0-9a-f]{32,64}$`)
patchSHA = regexp.MustCompile(`^[0-9a-f]{64}$`)
)
// checkPatchFileAsk judges the arguments of a patch_file ask, before any file is looked at.
func checkPatchFileAsk(a CommitAsk) error {
if !patchFileName.MatchString(a.PatchFile) {
return refuse("patch_file %q is not a name: 32 to 64 characters of 0-9a-f, no path (the file's name in the spool directory, chosen at random). Nothing was sent", a.PatchFile)
}
if !patchSHA.MatchString(a.PatchSHA256) {
return refuse("patch_file needs patch_sha256: the file's SHA-256, 64 hex characters. Nothing was sent")
}
return nil
}
// commitAsk makes the commit the ask names; a patch_file is first read from the spool directory into the patch
// and removed afterwards.
func commitAsk(ctx context.Context, f *Forge, spool string, a CommitAsk) (*Committed, error) {
if !a.patchFileGiven {
return commit(ctx, f, a)
}
if err := checkPatchFileAsk(a); err != nil {
return nil, err
}
if spool == "" {
return nil, refuse("patch_file is not available: %s is not set, so there is no spool directory. Nothing was sent", patchDirEnv)
}
path := filepath.Join(spool, a.PatchFile)
defer func() {
// Whatever happened, the file is not left: removing a symlink removes the link, never its target.
if st, err := os.Lstat(path); err == nil && !st.IsDir() {
_ = os.Remove(path)
}
}()
body, err := readSpooled(path, a.PatchFile)
if err != nil {
return nil, err
}
got := sha256.Sum256(body)
if hex.EncodeToString(got[:]) != a.PatchSHA256 {
return nil, refuse("the file %s is not the patch the caller hashed: its SHA-256 is %s, patch_sha256 says %s. Nothing was sent",
a.PatchFile, hex.EncodeToString(got[:]), a.PatchSHA256)
}
a.Patch = string(body)
return commit(ctx, f, a)
}
// readSpooled reads a regular file of at most maxPatchBytes, never through a symlink.
func readSpooled(path, name string) ([]byte, error) {
st, err := os.Lstat(path)
switch {
case errors.Is(err, os.ErrNotExist):
return nil, refuse("patch_file %s: no such file in the spool directory. Nothing was sent", name)
case err != nil:
return nil, refuse("patch_file %s cannot be read: %v. Nothing was sent", name, err)
case !st.Mode().IsRegular():
return nil, refuse("patch_file %s is not a regular file (a symlink or a directory is never followed). Nothing was sent", name)
case st.Size() > maxPatchBytes:
return nil, refuse("patch_file %s is %s, more than the %s one commit carries: split it into several commits. Nothing was sent", name, mib(int(st.Size())), mib(maxPatchBytes))
}
fh, err := os.OpenFile(path, os.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0)
if err != nil {
return nil, refuse("patch_file %s cannot be opened: %v. Nothing was sent", name, err)
}
defer fh.Close()
fst, err := fh.Stat()
if err != nil || !fst.Mode().IsRegular() {
return nil, refuse("patch_file %s is not a regular file. Nothing was sent", name)
}
// A hard link to a file elsewhere is no symlink, but it is not the caller's own file either.
if sys, ok := fst.Sys().(*syscall.Stat_t); !ok || sys.Nlink > 1 {
return nil, refuse("patch_file %s has more than one hard link: it must be a file of its own. Nothing was sent", name)
}
body, err := io.ReadAll(io.LimitReader(fh, maxPatchBytes+1))
switch {
case err != nil:
return nil, refuse("patch_file %s cannot be read: %v. Nothing was sent", name, err)
case len(body) > maxPatchBytes:
return nil, refuse("patch_file %s is more than the %s one commit carries: split it into several commits. Nothing was sent", name, mib(maxPatchBytes))
case strings.TrimSpace(string(body)) == "":
return nil, refuse("patch_file %s is empty. Nothing was sent", name)
}
return body, nil
}
+183
View File
@@ -0,0 +1,183 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"os"
"path/filepath"
"strings"
"testing"
)
// novox/hq issue 852: a patch too large to type into a call is handed over as a file in one spool directory.
const spoolName = "0123456789abcdef0123456789abcdef"
const spoolPatch = "diff --git a/README.md b/README.md\n--- a/README.md\n+++ b/README.md\n@@ -1 +1 @@\n-hello\n+hi\n"
func sum(s string) string { h := sha256.Sum256([]byte(s)); return hex.EncodeToString(h[:]) }
func spoolWith(t *testing.T, name, content string) string {
t.Helper()
dir := t.TempDir()
if name != "" {
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
return dir
}
func fileAsk(t *testing.T, name, sha string) CommitAsk {
t.Helper()
return ask(t, map[string]any{"branch": "fix/9", "patch_file": name, "patch_sha256": sha})
}
func TestAPatchFileIsAppliedAsThePatchWouldBeAndTheFileIsGone(t *testing.T) {
f := newFake(t)
dir := spoolWith(t, spoolName, spoolPatch)
got, err := commitAsk(context.Background(), f.serve(), dir, fileAsk(t, spoolName, sum(spoolPatch)))
if err != nil {
t.Fatal(err)
}
if len(f.patches) != 1 || f.patches[0]["content"] != spoolPatch || f.patches[0]["new_branch"] != "fix/9" || got.Commit != "p1" {
t.Fatalf("%v %+v", f.patches, got)
}
if _, err := os.Lstat(filepath.Join(dir, spoolName)); !os.IsNotExist(err) {
t.Fatalf("the file is still there: %v", err)
}
}
func TestAPatchFileIsRemovedWhenTheForgeRefusesIt(t *testing.T) {
f := newFake(t)
dir := spoolWith(t, spoolName, spoolPatch)
// the default branch is refused after the file was read
a := ask(t, map[string]any{"branch": "main", "patch_file": spoolName, "patch_sha256": sum(spoolPatch)})
if _, err := commitAsk(context.Background(), f.serve(), dir, a); err == nil {
t.Fatal("taken")
}
if _, err := os.Lstat(filepath.Join(dir, spoolName)); !os.IsNotExist(err) {
t.Fatalf("the file is still there: %v", err)
}
}
func TestAPatchFileThatCannotBeTakenIsRefusedBeforeAnythingIsSent(t *testing.T) {
outside := filepath.Join(t.TempDir(), "secret")
if err := os.WriteFile(outside, []byte(spoolPatch), 0o600); err != nil {
t.Fatal(err)
}
type tc struct {
name string
setup func(dir string)
file string
sha string
want string
}
for _, c := range []tc{
{"wrong sha", func(d string) { _ = os.WriteFile(filepath.Join(d, spoolName), []byte(spoolPatch), 0o600) }, spoolName, sum("other"), "sha256"},
{"not found", func(string) {}, spoolName, sum(spoolPatch), "no such file"},
{"symlink", func(d string) { _ = os.Symlink(outside, filepath.Join(d, spoolName)) }, spoolName, sum(spoolPatch), "regular file"},
{"hard link", func(d string) { _ = os.Link(outside, filepath.Join(d, spoolName)) }, spoolName, sum(spoolPatch), "hard link"},
{"directory", func(d string) { _ = os.Mkdir(filepath.Join(d, spoolName), 0o700) }, spoolName, sum(spoolPatch), "regular file"},
{"too large", func(d string) {
_ = os.WriteFile(filepath.Join(d, spoolName), []byte(strings.Repeat("x", maxPatchBytes+1)), 0o600)
}, spoolName, sum(spoolPatch), "MiB"},
{"empty", func(d string) { _ = os.WriteFile(filepath.Join(d, spoolName), nil, 0o600) }, spoolName, sum(""), "empty"},
} {
f := newFake(t)
dir := t.TempDir()
c.setup(dir)
_, err := commitAsk(context.Background(), f.serve(), dir, fileAsk(t, c.file, c.sha))
if err == nil || !strings.Contains(err.Error(), c.want) || !strings.Contains(err.Error(), "Nothing was sent") {
t.Errorf("%s: %v", c.name, err)
}
if len(f.patches)+len(f.writes) != 0 {
t.Errorf("%s: something was sent", c.name)
}
}
if b, err := os.ReadFile(outside); err != nil || string(b) != spoolPatch {
t.Errorf("the file outside the spool was touched: %v", err)
}
}
func TestAPatchFileAskThatCannotBeReadIsRefused(t *testing.T) {
ok := sum(spoolPatch)
for name, args := range map[string]map[string]any{
"no sha": {"patch_file": spoolName},
"sha not hex": {"patch_file": spoolName, "patch_sha256": strings.Repeat("z", 64)},
"sha too short": {"patch_file": spoolName, "patch_sha256": ok[:63]},
"empty name": {"patch_file": "", "patch_sha256": ok, "files": filesArg()},
"dotdot": {"patch_file": "../" + spoolName, "patch_sha256": ok},
"slash": {"patch_file": "a/" + spoolName, "patch_sha256": ok},
"absolute": {"patch_file": "/etc/passwd", "patch_sha256": ok},
"too short": {"patch_file": strings.Repeat("a", 31), "patch_sha256": ok},
"too long": {"patch_file": strings.Repeat("a", 65), "patch_sha256": ok},
"upper case": {"patch_file": strings.ToUpper(spoolName), "patch_sha256": ok},
"not hex": {"patch_file": strings.Repeat("g", 32), "patch_sha256": ok},
"with patch": {"patch_file": spoolName, "patch_sha256": ok, "patch": "diff"},
"with files": {"patch_file": spoolName, "patch_sha256": ok, "files": filesArg(map[string]any{"path": "x", "content": "x"})},
"sha without it": {"patch": "diff", "patch_sha256": ok},
} {
full := map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b"}
for k, v := range args {
full[k] = v
}
_, err := readCommitAsk(full)
if err == nil || !strings.Contains(err.Error(), "Nothing was sent") {
t.Errorf("%s: %v", name, err)
}
}
// 64 characters is taken; so is an upper-case sha.
for _, n := range []string{spoolName, strings.Repeat("ab", 32)} {
if _, err := readCommitAsk(map[string]any{"owner": "acme", "repo": "app", "message": "m", "branch": "b",
"patch_file": n, "patch_sha256": strings.ToUpper(sum(spoolPatch))}); err != nil {
t.Errorf("%s: %v", n, err)
}
}
}
func TestAPatchFileWithoutASpoolDirectoryIsRefused(t *testing.T) {
f := newFake(t)
_, err := commitAsk(context.Background(), f.serve(), "", fileAsk(t, spoolName, sum(spoolPatch)))
if err == nil || !strings.Contains(err.Error(), "MESH_GITEA_PATCH_DIR") || !strings.Contains(err.Error(), "Nothing was sent") {
t.Fatalf("%v", err)
}
}
func TestTheManifestDeclaresTheSpoolDirectoryAndGivesItToTheBundle(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Resources []struct {
ID, Type, Path, Mode, Owner string
} `json:"resources"`
Build struct {
Artifacts []struct {
From string `json:"from"`
Env map[string]string `json:"env"`
} `json:"artifacts"`
} `json:"build"`
}
if err := jsonInto(raw, &m); err != nil {
t.Fatal(err)
}
found := false
for _, r := range m.Resources {
if r.ID == "forge-patches" {
found = true
if r.Type != "directory" || r.Mode != "0733" || r.Owner != "${machine:account}" || r.Path == "" {
t.Errorf("%+v", r)
}
}
}
if !found {
t.Error("no forge-patches directory")
}
for _, a := range m.Build.Artifacts {
if a.From == "cmd/forge" && a.Env["MESH_GITEA_PATCH_DIR"] != "${dir:forge-patches}" {
t.Errorf("env %v", a.Env)
}
}
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"testing"
)
// The module's directory, where its manifest and its files/ are.
const moduleDir = "../.."
// templateText is the text a file resource places on the machine: its template's, in the module's files/
// (novox/hq ADR 0349 §2). A resource that names no template, or a template that is not there, fails the test.
func templateText(t *testing.T, r map[string]any) string {
t.Helper()
name, _ := r["template"].(string)
if name == "" {
t.Fatalf("resource %v names no template", r["id"])
}
raw, err := os.ReadFile(filepath.Join(moduleDir, filepath.FromSlash(name)))
if err != nil {
t.Fatalf("resource %v names %s: %v", r["id"], name, err)
}
return string(raw)
}
// No file resource carries its text inline: each names its template in files/, and the build packs files/
// into the one archive the controller reads the templates back from (novox/hq ADR 0349 §2).
func TestEveryFileTakesItsTextFromATemplate(t *testing.T) {
raw, err := os.ReadFile(filepath.Join(moduleDir, "module.json"))
if err != nil {
t.Fatal(err)
}
var m struct {
Resources []map[string]any `json:"resources"`
Build struct {
Artifacts []map[string]any `json:"artifacts"`
} `json:"build"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
for _, r := range m.Resources {
if _, inline := r["content"]; inline {
t.Errorf("resource %v keeps its text inline: keep it in files/ and name it as the template", r["id"])
}
if r["type"] == "file" {
templateText(t, r)
}
}
packs := 0
for _, a := range m.Build.Artifacts {
if a["kind"] == "archive" && a["from"] == "files" {
packs++
if a["name"] != "files" {
t.Errorf("the templates archive is named %v, not files", a["name"])
}
}
}
if packs != 1 {
t.Errorf("the build packs files/ into %d archives, not one", packs)
}
}
+1
View File
@@ -0,0 +1 @@
{}
+1
View File
@@ -0,0 +1 @@
{}
+1
View File
@@ -0,0 +1 @@
https://${bound:route:name-web}
+3
View File
@@ -0,0 +1,3 @@
# The accounts whose "ready at <sha>" comment marks a pull request ready (novox/hq issue 722), from the setting
# ready-authors, written by the mesh. Replaced on every push; change the setting, never this file.
authors=${config:ready-authors}
+7
View File
@@ -0,0 +1,7 @@
GITEA__security__INTERNAL_TOKEN=${secret:internal-token}
GITEA__database__DB_TYPE=postgres
GITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}
GITEA__database__NAME=${bound:postgres-database:as}
GITEA__database__USER=${bound:postgres-database:as}
GITEA__database__PASSWD=${secret:postgres-database}
GITEA__webhook__ALLOWED_HOST_LIST=172.17.0.1
+7 -2
View File
@@ -1,5 +1,10 @@
module gitea
go 1.22
go 1.25.0
require git.novox.be/novox/mesh-sdk/go v0.1.7
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009
require (
golang.org/x/crypto v0.50.0
golang.org/x/sys v0.43.0 // indirect
)
+6 -2
View File
@@ -1,2 +1,6 @@
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009 h1:I9rm8HhYRJ0+mzQQz+VAH0kgv2YJWkvctEc67Yd0PX8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013235212-c8b07242a009/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+205 -84
View File
@@ -40,10 +40,12 @@ import { emit, on } from "@novox/mesh-sdk/events";
import { registerModuleTools } from "@novox/mesh-sdk/tools";
import { commitStatusTool } from "./delivery.js";
import { mergeTool } from "./merge.js";
import { GiteaClient, movedSince } from "./client.js";
import { GiteaAdmin, GiteaClient, movedSince } from "./client.js";
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT, commentFor, headsToAnnounce, statusesFor, type Announced, type Checked,
type RepoCheckFacts, reposToLook } from "./pulls.js";
repoCheckFacts, reposToLook } from "./pulls.js";
import { GateWriter, clonedAs, type BuildStarted } from "./gate.js";
import { HOOK_PATH, HOOK_PORT, Pace, claimHead, ensureSystemHook, eventsTool, keptSecret, receiver, type ForgeEvent } from "./webhook.js";
import { keepSiblings, noteVerdict, readSiblings, siblingsFile, siblingsOf, weighSiblings } from "./siblings.js";
import { announcementOf, keepTook, lateVerdictComment, outOfDateWords, sameCommit, tookFile, weighTrunks } from "./trunk.js";
// Without a way to a token — configured, or mintable with the admin account (token.ts) — there is
@@ -135,6 +137,7 @@ async function pollMerged(client: GiteaClient): Promise<void> {
await emit("pull.closed", { owner: repo.owner, repo: repo.name, number: pull.number, title: pull.title,
head: pull.head, head_sha: pull.head_sha, base: pull.base, html_url: pull.html_url });
console.log(`[gitea] announced ${repo.full_name}#${pull.number} closed unmerged`);
await siblingMoves(client, repo.full_name, pull.number);
}
announced.add(closedKey);
changed = true;
@@ -145,57 +148,9 @@ async function pollMerged(client: GiteaClient): Promise<void> {
// at once.
const fresh = !!pull.merged_at && !!since && pull.merged_at > since;
if (primedMerges && fresh) {
// What it changed, asked for only now: a module is rebuilt because a file inside its own
// directory moved, and without this every module built from a repository is rebuilt for a
// change to any of them (novox/hq 04-ISSUES/131).
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
// change inside one is that module's, held or not, and only a file in none is shared code. Not
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
let moduleDirs: string[] | null = null;
if (!changed.truncated) {
try {
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
}
}
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
// from the forge's word when its owner never heard the head.
let headChecks: Record<string, string> | null = null;
if (pull.head_sha) {
try {
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
}
}
await emit("pull.merged", {
owner: repo.owner,
repo: repo.name,
number: pull.number,
title: pull.title,
body: pull.body,
head_sha: pull.head_sha,
...(headChecks ? { head_checks: headChecks } : {}),
head: pull.head,
base: pull.base,
merge_commit_sha: pull.merge_commit_sha,
merged_at: pull.merged_at,
clone_url: repo.clone_url,
html_url: pull.html_url,
paths: changed.paths,
paths_truncated: changed.truncated,
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
// not built.
removed: changed.removed,
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
});
// Said, because a trigger that fires silently is indistinguishable from one that did not
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
if (await announceMerge(client, repo, pull)) pace.miss(`the merge of ${repo.full_name}#${pull.number}`);
}
else announced.add(pull.merge_commit_sha);
announced.add(pull.merge_commit_sha);
changed = true;
}
@@ -206,6 +161,75 @@ async function pollMerged(client: GiteaClient): Promise<void> {
lastLook = began;
}
type RepoOf = { owner: string; name: string; full_name: string; clone_url?: string };
type PullOf = Awaited<ReturnType<GiteaClient["listPullRequests"]>>[number];
/** **A merge, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
async function announceMerge(client: GiteaClient, repo: RepoOf, pull: PullOf): Promise<boolean> {
if (!pull.merged || !pull.merge_commit_sha || announced.has(pull.merge_commit_sha)) return false;
// Recorded before the first wait, so the other path finds it taken while this one is still asking the forge.
announced.add(pull.merge_commit_sha);
try {
// What it changed, asked for only now: a module is rebuilt because a file inside its own
// directory moved, and without this every module built from a repository is rebuilt for a
// change to any of them (novox/hq 04-ISSUES/131).
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
// change inside one is that module's, held or not, and only a file in none is shared code. Not
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
let moduleDirs: string[] | null = null;
if (!changed.truncated) {
try {
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
}
}
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
// from the forge's word when its owner never heard the head.
let headChecks: Record<string, string> | null = null;
if (pull.head_sha) {
try {
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
}
}
await emit("pull.merged", {
owner: repo.owner,
repo: repo.name,
number: pull.number,
title: pull.title,
body: pull.body,
head_sha: pull.head_sha,
...(headChecks ? { head_checks: headChecks } : {}),
head: pull.head,
base: pull.base,
merge_commit_sha: pull.merge_commit_sha,
merged_at: pull.merged_at,
clone_url: repo.clone_url,
html_url: pull.html_url,
paths: changed.paths,
paths_truncated: changed.truncated,
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
// not built.
removed: changed.removed,
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
});
// Said, because a trigger that fires silently is indistinguishable from one that did not
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
// Its own merge of the head that was read is no sibling's move; any other is (ADR 0382 §5).
await siblingMoves(client, repo.full_name, pull.number);
} catch (err) {
announced.delete(pull.merge_commit_sha);
throw err;
}
keepAnnounced();
return true;
}
// **Every new head of an open pull request is announced, once** (novox/hq to-be 45 §9): the mesh checks it
// against every machine of its facts before it merges. Kept beside the merges' record, so a restart
// announces nothing twice; the first look on a machine with no record announces only what moved in the
@@ -228,6 +252,33 @@ function keepHeads(): void {
writeFileSync(tmp, JSON.stringify({ heads }));
renameSync(tmp, pullsRecord);
}
/** **A head, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
async function announceHead(client: GiteaClient, repo: RepoOf, pull: OpenOf): Promise<boolean> {
const key = `${repo.full_name}#${pull.number}`;
const sha = String(pull.head_sha ?? "");
const before = heads[key];
// Claimed before the first wait, so the other path finds it taken while this one is still announcing.
if (!claimHead(heads, key, sha)) return false;
try {
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url ?? "", pull));
} catch (err) {
if (before === undefined) delete heads[key];
else heads[key] = before;
throw err;
}
keepHeads();
// A sibling's new head puts out of date every verdict that read the sibling at another head (ADR 0382 §5).
await siblingMoves(client, repo.full_name, pull.number);
// Said, so an operator knows the mesh was asked to check it.
console.log(`[gitea] announced ${key} at ${sha.slice(0, 8)} to be checked before it merges`);
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
await gates.pending(repo.owner, repo.name, sha);
return true;
}
type OpenOf = Awaited<ReturnType<GiteaClient["listOpenPullsSince"]>>[number];
let lastPullLook = "";
// **A sweep now and then looks at every repository** (novox/hq issue 347), whatever moved: the floor is a
// heuristic, and a heuristic that misses is caught by the next sweep rather than never.
@@ -259,15 +310,8 @@ async function pollPulls(client: GiteaClient): Promise<void> {
const key = `${repo.full_name}#${pull.number}`;
const fresh = primedPulls || (!!pull.updated_at && pull.updated_at > dayAgo);
if (fresh) {
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url, pull));
// Said, so an operator knows the mesh was asked to check it.
console.log(`[gitea] announced ${key} at ${String(pull.head_sha).slice(0, 8)} to be checked before it merges`);
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
await gates.pending(repo.owner, repo.name, String(pull.head_sha));
}
heads[key] = String(pull.head_sha);
if (await announceHead(client, repo, pull)) pace.miss(`${key} at ${String(pull.head_sha).slice(0, 8)}`);
} else heads[key] = String(pull.head_sha);
changed = true;
}
if (await trunksMoved(client, repo.owner, repo.name)) changedTrunks = true;
@@ -303,7 +347,22 @@ function keepTrunks(): void {
/** Whether a trunk of one repository moved, weighed with this module's forge and its one writer (trunk.ts). */
function trunksMoved(client: GiteaClient, owner: string, name: string): Promise<boolean> {
return weighTrunks(client, gates, trunks, owner, name);
return weighTrunks(client, gates, trunks, owner, name, () => siblingMoves(client, `${owner}/${name}`));
}
// **The siblings each open pull request's latest verdict names** (novox/hq ADR 0382 §5, siblings.ts), by member, kept beside
// the other records. A sibling's new head, close unmerged, merge, or a move of its base puts the merge check of every
// member whose verdict names it back to pending, with the reason; nothing is run for it.
const siblingsRecord = siblingsFile(process.env.MESH_GITEA_STATE_DIR);
const siblingNamed = readSiblings(siblingsRecord);
/** A sibling moved, weighed (ADR 0382 §5): never fatal to what heard it. */
async function siblingMoves(client: GiteaClient, repository: string, number?: number): Promise<void> {
try {
if (await weighSiblings(client, gates, siblingNamed, repository, number)) keepSiblings(siblingsRecord, siblingNamed);
} catch (err) {
console.error(`[gitea] whether a sibling of ${repository}${number ? "#" + number : ""} moved could not be weighed — ${err instanceof Error ? err.message : err}`);
}
}
// **The verdict, set where the pull request shows it.** Every verdict is the head commit's status; one
@@ -325,6 +384,17 @@ async function setVerdict(client: GiteaClient, event: { body: unknown }): Promis
console.error("[gitea] a merge check's verdict named no repository, commit or verdict; ignored");
return;
}
// The siblings this verdict read are kept for the member (ADR 0382 §5); a verdict that read none removes the record.
const siblings = siblingsOf(c.siblings);
if (c.number && noteVerdict(siblingNamed, `${c.owner}/${c.repo}#${c.number}`.toLowerCase(), c.commit, siblings)) {
keepSiblings(siblingsRecord, siblingNamed);
}
// A sibling that moved while the check ran is weighed once this verdict has landed, in the head's own turn after it.
if (siblings.length) {
for (const repository of new Set(siblings.map((s) => s.repository))) {
void Promise.resolve().then(() => siblingMoves(client, repository));
}
}
// Each status links to the pull request, where the delivery's view is kept (novox/hq ADR 0239).
let target: string | undefined;
let base: string | undefined;
@@ -362,22 +432,6 @@ async function setVerdict(client: GiteaClient, event: { body: unknown }): Promis
console.log(`[gitea] ${c.owner}/${c.repo}#${c.number ?? "?"} at ${c.commit.slice(0, 8)}: merge check ${c.verdict}`);
}
// **What only the forge knows of a repository check said as a warning** (novox/hq issue 293): whether the
// head holds a merge-check.sh at all, and — when it does not — whether the base branch's protection
// requires mesh/repo-check. Asked only for a warning; unknown is left undefined, which statusesFor reads
// as possibly required: a failure on a status nothing requires blocks nothing, a success on one that is
// required would let an untested repository merge.
async function repoCheckFacts(client: GiteaClient, c: Checked, base: string | undefined): Promise<RepoCheckFacts> {
if (c["repo-check"]?.verdict !== "warning") return {};
const defined = await client.holdsFile(c.owner, c.repo, c.commit, "merge-check.sh").catch(() => undefined);
if (defined !== false) return { defined };
const rules = await client.branchProtections(c.owner, c.repo).catch(() => undefined);
if (!rules) return { defined };
const rule = rules.find((p) => (p.rule_name ?? p.branch_name) === (base || "main"));
const required = !!rule?.enable_status_check && (rule.status_check_contexts ?? []).includes(REPO_CHECK_CONTEXT);
return { defined, required };
}
// **This module is the one writer of a head's mesh/merge-gate** (novox/hq issue 446): the announcement's pending words,
// where the check is when a build machine takes it, and the verdict — each head's writes in one chain, the verdict last
// (gate.ts). A second writer elsewhere raced the verdict and could leave a pass reading pending.
@@ -391,10 +445,16 @@ const gates = new GateWriter({
// **The status tool is served here, beside the writer** (novox/hq issue 456): the runtime starts each bundle as its own
// process, so a recheck's pending written from the tools' bundle bypassed this writer's chain. Without a forge to
// reach, nothing is served, as the tools' bundle serves nothing then.
registerModuleTools("gitea", () => (gitea ? [commitStatusTool({
/** How fast the polls look: 30 s until the forge's events are heard, five minutes while they are; and what
* gitea_events says of them (webhook.ts). */
const pace = new Pace();
if (!gitea) pace.unheard("the forge is not watched: no way to a token", false);
// gitea_events is served whether or not the forge is watched: it says so (novox/hq issue 729).
registerModuleTools("gitea", () => [eventsTool(pace), ...(gitea ? [commitStatusTool({
setCommitStatus: (owner, repo, sha, status) => (gitea as GiteaClient).setCommitStatus(owner, repo, sha, status),
gates,
}), mergeTool(gitea, { gates })] : []));
}), mergeTool(gitea, { gates })] : [])]);
// **A build machine took an ask** (the build seat's `started`, novox/hq ADR 0157): said on the head it checks, when
// the head is an open pull request's this module announced. Any other build — a branch, a merge's — says nothing.
@@ -413,12 +473,70 @@ async function buildStarted(event: { body: unknown }): Promise<void> {
}
}
/** What the forge's event asks of the announcer at once (issue 729): a new head is announced, and a merge. */
async function actOn(client: GiteaClient, event: ForgeEvent): Promise<void> {
const b = event.body as { owner?: string; repo?: string; number?: number };
if (!b.owner || !b.repo || !b.number) return;
if (event.name === "forge.pull-request.closed") {
// Closed unmerged is a sibling's move; a merge is weighed with its announcement below (ADR 0382 §5).
if (!(event.body as { merged?: boolean }).merged) await siblingMoves(client, `${b.owner}/${b.repo}`, b.number);
return;
}
const head = ["forge.pull-request.opened", "forge.pull-request.synchronized", "forge.pull-request.reopened"].includes(event.name);
const merged = event.name === "forge.pull-request.merged";
if (!head && !merged) return;
const [repo, pull] = await Promise.all([client.getRepo(b.owner, b.repo), client.getPullRequest(b.owner, b.repo, b.number)]);
if (head && pull.state === "open") await announceHead(client, repo, pull);
if (merged) await announceMerge(client, repo, pull);
}
/**
* **The receiver, and the forge's one system webhook pointing at it** (issue 729). The secret is minted and kept beside
* the token; the webhook is made or corrected with the admin account (ADR 0266: the tools' token holds no admin scope).
* Answers whether events are heard; a forge that refuses is said, and the polls keep their pace.
*/
async function listenForForge(client: GiteaClient): Promise<boolean> {
const stateDir = process.env.MESH_GITEA_STATE_DIR;
const url = process.env.MESH_GITEA_HOOK_URL;
if (!stateDir || !url) {
console.log("[gitea] no webhook receiver: MESH_GITEA_STATE_DIR or MESH_GITEA_HOOK_URL unset, so the polls alone watch the forge");
pace.unheard("MESH_GITEA_STATE_DIR or MESH_GITEA_HOOK_URL unset", false);
return false;
}
try {
const secret = keptSecret(join(stateDir, "webhook.secret"));
const port = Number(process.env.MESH_GITEA_HOOK_PORT ?? HOOK_PORT);
const host = process.env.MESH_GITEA_HOOK_HOST ?? "0.0.0.0";
const server = receiver({ secret, port, heard: (kind, delivery) => pace.heard(kind, delivery),
refused: (reason, from) => pace.refused(reason, from),
publish: (event) => emit(event.name, event.body),
act: (event) => actOn(client, event) });
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, host, () => resolve());
});
const hook = await ensureSystemHook(GiteaAdmin.fromEnv(), url.replace(/\/+$/, "") + HOOK_PATH, secret);
pace.listening(host, port, hook.url, hook);
console.log(`[gitea] listening for the forge's events on ${host}:${port}; its system webhook ${hook.id} ${hook.did} ` +
`at ${hook.url} (in place of ${hook.removed.system} system, ${hook.removed.default} default and ` +
`${hook.removed.repository} repository webhook(s)${hook.removed.repositories.length ? ` in ${hook.removed.repositories.join(", ")}` : ""}), ` +
`${hook.events.length} event families, signed`);
return true;
} catch (err) {
const why = err instanceof Error ? err.message : String(err);
pace.unheard(why);
console.error(`[gitea] the forge's events are not heard, so the polls keep their pace — ${why}`);
return false;
}
}
if (gitea) {
const client = gitea;
// A poll that fails says so once, not once a minute: the same reason repeating (the forge not up
// yet, the admin account refused on a restored forge) is one fact, and a recovery is worth a line.
let failing: string | null = null;
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const tick = (fn: () => Promise<void>, every: number | (() => number)): void => {
const everyMs = (): number => (typeof every === "number" ? every : every());
// **One pass at a time** (novox/hq issue 250): the next pass is scheduled when this one has ended, so a
// pass that outlasts its interval delays the next instead of running beside it — two passes at once
// could each announce the same merge before either recorded it.
@@ -433,12 +551,15 @@ if (gitea) {
if (why !== failing) console.error(`[gitea] not watching until this clears — ${why}`);
failing = why;
})
.finally(() => setTimeout(run, everyMs));
.finally(() => setTimeout(run, everyMs()));
run();
};
// **The forge's events first, the polls as the safety net** (novox/hq issue 729): while verified deliveries arrive the
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
await listenForForge(client);
tick(() => pollRepos(client), 60_000);
tick(() => pollMerged(client), 30_000);
tick(() => pollPulls(client), 30_000);
tick(() => pollMerged(client), () => pace.every());
tick(() => pollPulls(client), () => pace.every());
await on("mesh-controller.checked", (event) => {
const c = (event.body ?? {}) as Checked;
// A head's own verdict takes its turn after every write of that head's gate already asked for (gate.ts).
+76 -11
View File
@@ -1,7 +1,17 @@
// Package forgehttp is the forge's HTTP interface as this module's Go bundles reach it (cmd/forge,
// cmd/npm-registry): the forge on its own machine, as its admin account, whose password the vault delivered
// into a file this module owns (novox/hq ADR 0086), by basic authentication. The password is read when a call
// is made and never logged, answered or put in an error. One copy for both bundles (novox/hq issue 539).
// cmd/npm-registry), on the forge's own machine. One copy for both bundles (novox/hq issue 539).
//
// Two ways in, by what the manifest hands the bundle:
//
// - **as the tools' account** (novox/hq ADR 0365 §2), when MESH_GITEA_TOKEN_FILE names the token the module
// keeps for it (token.ts mints it, for `mesh-tools`, and keeps it at 0600 with its scopes line beside it). A
// token whose scopes line does not say it is the tools' account's — one kept from before, the administrator's —
// is never sent: the call fails, saying so, until the module has minted the account's own. This bundle mints
// nothing; a token the forge rejects is renewed by the module's next call, and said here.
// - **as the admin account**, whose password the vault delivered into a file this module owns (novox/hq ADR
// 0086), by basic authentication, when no token file is named.
//
// A credential is read when a call is made and never logged, answered or put in an error.
package forgehttp
import (
@@ -25,12 +35,31 @@ type Client struct {
URL string // e.g. http://127.0.0.1:3000
User string
PasswordFile string
HTTP *http.Client // when nil, a client with Timeout
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
// TokenFile, when set, is the tools' account's token (MESH_GITEA_TOKEN_FILE), sent instead of the admin login;
// Account the account its scopes line must name (MESH_GITEA_TOOLS_USER, default mesh-tools).
TokenFile string
Account string
HTTP *http.Client // when nil, a client with Timeout
Timeout time.Duration // one request's limit when HTTP is nil; 20 s when zero
}
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset.
// ToolsAccount is the account the forge's tools act as when the manifest names none (token.ts, tools-account.ts).
const ToolsAccount = "mesh-tools"
// FromEnv is the forge as the manifest hands it to a bundle, and the names of the variables it left unset. A bundle
// handed MESH_GITEA_TOKEN_FILE acts as the tools' account and needs no admin login.
func FromEnv() (Client, []string) {
if file := os.Getenv("MESH_GITEA_TOKEN_FILE"); file != "" {
c := Client{URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"), TokenFile: file,
Account: os.Getenv("MESH_GITEA_TOOLS_USER")}
if c.Account == "" {
c.Account = ToolsAccount
}
if c.URL == "" {
return c, []string{"MESH_GITEA_URL"}
}
return c, nil
}
c := Client{
URL: strings.TrimRight(os.Getenv("MESH_GITEA_URL"), "/"),
User: os.Getenv("MESH_GITEA_ADMIN_USER"),
@@ -80,14 +109,45 @@ func (c *Client) password() (string, error) {
return strings.TrimSpace(string(raw)), nil
}
// token is the tools' account's token, refused unless its scopes line names that account.
func (c *Client) token() (string, error) {
raw, err := os.ReadFile(c.TokenFile)
if err != nil {
return "", fmt.Errorf("the tools' token is not kept yet (the module mints it for %s at its first call): %w", c.Account, err)
}
line, err := os.ReadFile(c.TokenFile + ".scopes")
if err != nil || !strings.HasSuffix(strings.TrimSpace(string(line)), " for "+c.Account) {
return "", fmt.Errorf("the token kept at %s is not %s's (novox/hq ADR 0365): the module mints that account's own at its next call; nothing was sent", c.TokenFile, c.Account)
}
token := strings.TrimSpace(string(raw))
if token == "" {
return "", fmt.Errorf("the token kept at %s is empty; nothing was sent", c.TokenFile)
}
return token, nil
}
// authorize puts the bundle's credential on a request: the tools' token, or the admin login.
func (c *Client) authorize(req *http.Request) error {
if c.TokenFile != "" {
token, err := c.token()
if err != nil {
return err
}
req.Header.Set("Authorization", "token "+token)
return nil
}
password, err := c.password()
if err != nil {
return err
}
req.SetBasicAuth(c.User, password)
return nil
}
// Do sends one request, with payload as its JSON body when not nil, and answers the body, read up to limit;
// 404 is ErrNotFound. One byte past the limit is read, so a body the limit cut is told from one that fits it
// exactly: a cut body is refused, never read as if it were whole.
func (c *Client) Do(ctx context.Context, method, path string, payload any, limit int64) ([]byte, error) {
password, err := c.password()
if err != nil {
return nil, err
}
var body io.Reader
if payload != nil {
raw, err := json.Marshal(payload)
@@ -100,7 +160,9 @@ func (c *Client) Do(ctx context.Context, method, path string, payload any, limit
if err != nil {
return nil, err
}
req.SetBasicAuth(c.User, password)
if err := c.authorize(req); err != nil {
return nil, err
}
req.Header.Set("Accept", "application/json")
if payload != nil {
req.Header.Set("Content-Type", "application/json")
@@ -118,6 +180,9 @@ func (c *Client) Do(ctx context.Context, method, path string, payload any, limit
return nil, fmt.Errorf("the forge's answer to %s %s is larger than %d MiB and was not read whole", method, path, limit>>20)
}
switch {
case res.StatusCode == http.StatusUnauthorized && c.TokenFile != "":
return nil, fmt.Errorf("the forge rejected the tools' token for %s (401): the module mints it afresh at its next call; nothing was done: %w",
method+" "+path, &StatusError{method, path, res.StatusCode, "unauthorized"})
case res.StatusCode == http.StatusNotFound:
return nil, fmt.Errorf("%s %s: %w", method, path, ErrNotFound)
case res.StatusCode >= 300:
+32 -7
View File
@@ -11,7 +11,8 @@ import type { CommitStatus, GiteaClient } from "./client.js";
import { mergeableUnder, newestByContext } from "./checks.js";
import { MERGE_METHODS, Refused, checkName } from "./guard.js";
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT } from "./pulls.js";
import { announcementOf, askedWords, outOfDateWords, tookFile, tookWords, twelve, verdictAtTrunk } from "./trunk.js";
import { groupRefusal, memberKey, namesSiblings, readSiblings, siblingOutOfDate, siblingsFile } from "./siblings.js";
import { announcementOf, askedWords, outOfDateWords, sameCommit, tookFile, tookWords, twelve, verdictAtTrunk } from "./trunk.js";
/** A head named for a merge: at least 12 hex characters of its sha (novox/hq issue 381), up to a sha-256 commit's 64. */
const REVIEWED_HEAD = /^[0-9a-f]{12,64}$/;
@@ -54,9 +55,10 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
const method = args.method ? String(args.method) : "merge";
if (!MERGE_METHODS.has(method)) throw new Refused(`${method} is not a merge method: ${[...MERGE_METHODS].join(", ")}`);
const deleteBranch = args.delete_branch === undefined ? true : Boolean(args.delete_branch);
// **Only a head whose required statuses all succeeded** (novox/hq ADR 0266): read here rather than left
// to the forge, whose administrator — the account this tool's token is minted for — may merge past a
// rule that lets it. A rule that cannot be read is no rule passed.
// **Only a head whose required statuses all succeeded** (novox/hq ADR 0266): read here as well as by the
// forge, which holds the tools' own account to it (novox/hq ADR 0365 §2). The protection is the one the forge
// applies to the base, read from the branch: the rules' own list is a repository administrator's. A rule that
// cannot be read is no rule passed.
// **Only the head that was reviewed** (novox/hq issue 381): mesh-controller#197 was reviewed at 065cfa0d,
// force-pushed to 3ced96f, checked green there, and merged at 3ced96f, because this tool merged whatever the
// head was and nothing told it which head the review had approved. The caller names it; a head that moved
@@ -80,7 +82,7 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
throw new Refused(`${owner}/${repo}#${number} is not merged: its head is ${pull.head_sha.slice(0, 12)}, not the ` +
`${named} you named. If you reviewed an earlier head, review the one it has now, then name it. Nothing was merged`);
}
const rule = await gitea.branchProtection(owner, repo, pull.base);
const rule = await gitea.appliedProtection(owner, repo, pull.base);
// Only into a branch whose protection requires the checks: a pull request whose base was moved to a
// branch nothing guards would merge past every check (novox/hq ADR 0266).
if (!rule || !rule.enable_status_check || !(rule.status_check_contexts ?? []).filter(Boolean).length) {
@@ -96,7 +98,11 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
const checks = await gitea.headChecks(owner, repo, pull.head_sha);
const at = verdictAtTrunk(checks, trunkHead);
if (!at.current) {
const judged = at.judged ? `was judged against trunk ${twelve(at.judged)}` : "names no trunk commit it was judged against";
// A verdict put out of date by a sibling's move says so in its own words (ADR 0382 §5); it is asked again as a
// trunk's move is, and a failed ask puts those words back, not the trunk's.
const siblingWords = siblingOutOfDate(checks[CHECK_CONTEXT]?.description) ? checks[CHECK_CONTEXT]?.description : undefined;
const judged = siblingWords ? `was put out of date by a sibling's move (${siblingWords})`
: at.judged ? `was judged against trunk ${twelve(at.judged)}` : "names no trunk commit it was judged against";
const took = tookWords(tookFile(deps.stateDir), `${owner}/${repo}`);
const head = twelve(pull.head_sha), now = twelve(trunkHead);
if (at.onItsWay) {
@@ -122,7 +128,7 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
const cloneURL = (await gitea.getRepo(owner, repo)).clone_url;
await deps.emit("pull.updated", await announcementOf(gitea, owner, repo, cloneURL, pull));
} catch (err) {
await say(outOfDateWords(at.judged, trunkHead)).catch(() => {});
await say(siblingWords ?? outOfDateWords(at.judged, trunkHead)).catch(() => {});
throw new Error(`${owner}/${repo}#${number} is not merged: its verdict ${judged}, and ${pull.base} is now ` +
`${now}, and its check could not be asked: ${err instanceof Error ? err.message : err}. Nothing was merged`);
}
@@ -138,6 +144,25 @@ export function mergeTool(gitea: GiteaClient, given: MergeDeps): ToolDefinition
throw new Refused(`${owner}/${repo}#${number} is not merged: ${verdict.says}. A pull request merges once ` +
`its checks succeed; no tool merges past them (novox/hq ADR 0266)`);
}
// **A member judged beside its siblings merges only with its group passed and reviewed** (novox/hq ADR 0382 §6): the
// siblings its verdict names are the holder's record (siblings.ts); a status that says it read siblings with no record
// of them is no verdict to merge on, never read as none.
const heard = readSiblings(siblingsFile(deps.stateDir))[memberKey(owner, repo, number)];
const siblings = heard && sameCommit(heard.head, pull.head_sha) ? heard.siblings : [];
if (!siblings.length && namesSiblings(statuses.find((s) => s.context === CHECK_CONTEXT)?.description)) {
throw new Refused(`${owner}/${repo}#${number} is not merged: its verdict read siblings, and which ones is not held ` +
`here, so the group cannot be weighed. Ask its check again at ${twelve(pull.head_sha)}. Nothing was merged`);
}
if (siblings.length) {
const missing = await groupRefusal({ headChecks: (o, r, sha) => gitea.headChecks(o, r, sha),
approvedAt: (o, r, n, h) => gitea.approvedAt(o, r, n, h) },
{ owner, repo, number, head: String(pull.head_sha) }, siblings);
if (missing) {
throw new Refused(`${owner}/${repo}#${number} is not merged: its verdict was judged beside ${siblings.length} ` +
`sibling(s), and ${missing}. A member judged beside its siblings merges once its group passed at the heads that ` +
`verdict read and each of them is approved there (novox/hq ADR 0382). Nothing was merged`);
}
}
await gitea.mergePullRequest(owner, repo, number, method, deleteBranch, pull.head_sha);
// The merge commit only exists now; answered so the caller can follow what is built from it.
// pull.merged is the events entrypoint's to announce (index.ts), once, within its poll.
+185 -8
View File
@@ -1,6 +1,7 @@
{
"module": "gitea",
"version": "1",
"moved": true,
"requires": [
"postgres-database",
"route",
@@ -42,7 +43,47 @@
"issue.opened",
"pull.merged",
"pull.updated",
"pull.closed"
"pull.closed",
"forge.push",
"forge.create",
"forge.delete",
"forge.fork",
"forge.status",
"forge.pull-request.opened",
"forge.pull-request.reopened",
"forge.pull-request.synchronized",
"forge.pull-request.edited",
"forge.pull-request.closed",
"forge.pull-request.merged",
"forge.pull-request.labeled",
"forge.pull-request.unlabeled",
"forge.pull-request.assigned",
"forge.pull-request.unassigned",
"forge.pull-request.reviewed",
"forge.pull-request.review-requested",
"forge.pull-request.commented",
"forge.pull-request.changed",
"forge.issue-comment.created",
"forge.issue-comment.edited",
"forge.issue-comment.deleted",
"forge.issues.opened",
"forge.issues.reopened",
"forge.issues.closed",
"forge.issues.edited",
"forge.issues.labeled",
"forge.issues.unlabeled",
"forge.issues.assigned",
"forge.issues.unassigned",
"forge.issues.changed",
"forge.release.published",
"forge.release.updated",
"forge.release.deleted",
"forge.release.changed",
"forge.repository.created",
"forge.repository.deleted",
"forge.repository.changed",
"forge.wiki.changed",
"forge.other"
],
"consumes": [
"mesh-controller.checked",
@@ -61,7 +102,14 @@
"port": 22,
"protocol": "tcp",
"from": "mesh",
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention \u2014 not 22, which the machine's own daemon holds and a module does not take"
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention — not 22, which the machine's own daemon holds and a module does not take"
},
{
"name": "forge-events",
"port": 3047,
"protocol": "tcp",
"from": "machine",
"why": "the forge's system webhook, every event signed with a secret the module keeps, from the forge's container on the docker bridge to the module's receiver at the bridge's gateway 172.17.0.1 (novox/hq issue 729)"
}
],
"serves": {
@@ -109,6 +157,26 @@
}
}
},
"config": {
"url": {
"type": "string",
"description": "The address the module's tools reach the forge at, as a URL; unset, the code falls back to its environment, else this machine's gitea on its port.",
"trusted": true,
"default": ""
},
"agent-read-teams": {
"type": "object",
"description": "The read teams of the agents' forge accounts (ADR 0365 clause 3): each team by name, {\"<team>\": {\"repos\": [\"<owner>/<repo>\", ...], \"members\": [\"<node>\", ...]}}. The forge gets a team agent-read-<team> that reads only, with exactly those repositories and the accounts agent-<node>; a team taken out of the value is emptied. Nothing else on the forge is touched; an agent account is in no team until it is listed here.",
"trusted": true,
"default": {}
},
"ready-authors": {
"type": "string",
"description": "the forge accounts whose comment \"ready at <full head sha>\" marks a pull request ready for mesh-delivery to merge, separated by spaces: the account the forge's tools act as (novox/hq ADR 0365), the admin account for marks made before the tools acted as their own, and the operator's (novox/hq issue 722). A comment by any other account is ignored and said in the forge's log. Trusted: who may have a change merged is the operator's word",
"trusted": true,
"default": "mesh-tools mesh-admin jschoubben"
}
},
"resources": [
{
"id": "mesh-state",
@@ -122,6 +190,21 @@
"path": "${dir:mesh-state}/state",
"mode": "0700"
},
{
"id": "ready-authors",
"type": "file",
"path": "${dir:mesh-state}/ready-authors",
"mode": "0600",
"trusted": true,
"template": "files/ready-authors"
},
{
"id": "forge-patches",
"type": "directory",
"path": "/var/lib/mesh/forge-patches",
"mode": "0733",
"owner": "${machine:account}"
},
{
"id": "state",
"type": "directory",
@@ -138,7 +221,7 @@
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "GITEA__security__INTERNAL_TOKEN=${secret:internal-token}\nGITEA__database__DB_TYPE=postgres\nGITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nGITEA__database__NAME=${bound:postgres-database:as}\nGITEA__database__USER=${bound:postgres-database:as}\nGITEA__database__PASSWD=${secret:postgres-database}\n"
"template": "files/server.env"
},
{
"id": "data",
@@ -149,6 +232,13 @@
{
"id": "server",
"type": "container",
"health": {
"kind": "http",
"endpoint": "web",
"path": "/api/healthz",
"status": 200,
"needs": "postgres-database"
},
"name": "gitea",
"image": "gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef",
"env": {
@@ -194,13 +284,53 @@
],
"secrets-in-environment": "gitea honours GITEA__database__PASSWD__FILE and GITEA__security__INTERNAL_TOKEN__FILE; convertible, awaiting a bed that proves it"
},
{
"id": "origins",
"type": "file",
"path": "${dir:mesh-state}/origins",
"mode": "0644",
"template": "files/origins"
},
{
"id": "runtime-config",
"type": "file",
"path": "${dir:mesh-state}/config.json",
"mode": "0600",
"content": "{}\n",
"merge": "json"
"template": "files/config.json",
"merge": "json",
"takes-config": [
"url"
]
},
{
"id": "agent-teams-file",
"type": "file",
"path": "${dir:mesh-state}/agent-teams.json",
"mode": "0600",
"template": "files/agent-teams.json",
"merge": "json",
"takes-config": [
"agent-read-teams"
]
},
{
"id": "agent-teams",
"type": "process",
"name": "gitea-agent-teams",
"artifact": "agent-teams",
"run": [
"./agent-teams"
],
"run-once": true,
"env": {
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_AGENT_TEAMS_FILE": "${dir:mesh-state}/agent-teams.json"
},
"restart-on": [
"agent-teams-file"
]
}
],
"provides": [
@@ -215,6 +345,14 @@
{
"name": "git",
"scope": "mesh"
},
{
"name": "gitea-read",
"scope": "mesh",
"credential": {
"issued-by": "software",
"tool": "gitea_agent_token"
}
}
],
"build": {
@@ -238,8 +376,13 @@
"MESH_GITEA_CONFIG_FILE": "${dir:mesh-state}/config.json",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_TOOLS_USER": "mesh-tools",
"MESH_GITEA_STATE_DIR": "${dir:runtime-state}",
"MESH_RECEIVES": "${dir:grants}/npm.json"
"MESH_RECEIVES": "${dir:grants}/npm.json",
"MESH_GITEA_READY_AUTHORS_FILE": "${dir:mesh-state}/ready-authors",
"MESH_GITEA_HOOK_URL": "http://172.17.0.1:3047",
"MESH_GITEA_HOOK_PORT": "3047",
"MESH_GITEA_HOOK_HOST": "172.17.0.1"
}
},
{
@@ -271,9 +414,43 @@
],
"env": {
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret"
"MESH_GITEA_TOOLS_USER": "mesh-tools",
"MESH_GITEA_TOKEN_FILE": "${dir:runtime-state}/token",
"MESH_GITEA_PATCH_DIR": "${dir:forge-patches}"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
},
{
"name": "agent-token",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/agent-token",
"binary": "agent-token",
"loads": [
"agent-token"
],
"env": {
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
"MESH_GITEA_ADMIN_USER": "mesh-admin",
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_ORIGINS_FILE": "${dir:mesh-state}/origins"
}
},
{
"name": "agent-teams",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/agent-teams",
"binary": "agent-teams",
"loads": [
"agent-teams"
]
}
]
},
+42 -4
View File
@@ -9,8 +9,9 @@
//
// Pure functions here, so they are tested without a forge; index.ts does the asking and the setting.
import type { CommitStatus, GiteaPull, GiteaRepo } from "./client.js";
import type { CommitStatus, GiteaClient, GiteaPull, GiteaRepo } from "./client.js";
import { movedSince } from "./client.js";
import { UNKNOWN_MARK, besideWords, siblingsOf } from "./siblings.js";
/** The status context a merge check's gate is kept under: one per commit, the newest replacing the last. */
export const CHECK_CONTEXT = "mesh/merge-gate";
@@ -65,6 +66,13 @@ export interface Checked {
trunk?: string;
beside?: Record<string, string>;
took?: string;
/** The other members of its delivery group the check read beside the head, each at its head merged onto its base
* commit (novox/hq ADR 0382 §4); absent when it read none, and from a controller from before the rule. */
siblings?: { repository: string; number: number; head: string; base: string; base_commit: string }[];
/** Why the siblings could not be learned (mesh-delivery did not answer), when that is so. */
siblings_unknown?: string;
/** Where the judge came from, when it came from the group (ADR 0382 §3): said in the account. */
judge_from?: string;
}
/** A change plan, as the controller says it (mesh-controller internal/link, ChangePlan). */
@@ -163,7 +171,7 @@ function described(verdict: string, summary: string, modules?: string[], depende
return clipped(d);
}
function clipped(d: string): string {
export function clipped(d: string): string {
d = d.replace(/\s+/g, " ").trim();
return d.length > 140 ? d.slice(0, 139) + "…" : d;
}
@@ -176,6 +184,24 @@ export interface RepoCheckFacts {
required?: boolean;
}
// **What only the forge knows of a repository check said as a warning** (novox/hq issue 293): whether the
// head holds a merge-check.sh at all, and — when it does not — whether the base branch's protection
// requires mesh/repo-check. Asked only for a warning; unknown is left undefined, which statusesFor reads
// as possibly required: a failure on a status nothing requires blocks nothing, a success on one that is
// required would let an untested repository merge. The base's protection is the one the forge applies, read
// from the branch: the rules' own list is a repository administrator's, which the tools' account is not
// (novox/hq ADR 0365).
export async function repoCheckFacts(
forge: Pick<GiteaClient, "holdsFile" | "appliedProtection">, c: Checked, base: string | undefined): Promise<RepoCheckFacts> {
if (c["repo-check"]?.verdict !== "warning") return {};
const defined = await forge.holdsFile(c.owner, c.repo, c.commit, "merge-check.sh").catch(() => undefined);
if (defined !== false) return { defined };
const rule = await forge.appliedProtection(c.owner, c.repo, base || "main").catch(() => undefined);
if (rule === undefined) return { defined };
const required = !!rule?.enable_status_check && (rule.status_check_contexts ?? []).includes(REPO_CHECK_CONTEXT);
return { defined, required };
}
/** The description of a required repository check that is not defined. */
export const UNDEFINED_REQUIRED = "fail: mesh/repo-check is required here and this head defines no merge-check.sh — " +
"a person adds one, or lifts the requirement from the branch's protection";
@@ -220,8 +246,12 @@ export function statusesFor(c: Checked, facts: RepoCheckFacts = {}): CommitStatu
* 7a8b9c0d1e2f — pass: …", so the forge's 140 characters never clip it away; the trunk's move is weighed by it. A
* verdict that names no trunk commit is said as before. */
function judgedFirst(c: Checked, status: CommitStatus): CommitStatus {
if (!c.trunk) return status;
return { ...status, description: clipped(`head ${c.commit.slice(0, 12)} merged into trunk ${c.trunk.slice(0, 12)} — ${status.description}`) };
// The siblings it read, named after the commits (ADR 0382 §4), so the clip never takes them either.
const siblings = siblingsOf(c.siblings);
// A verdict that could not learn them says so on the status too (clause 1: it is still checked, and merges).
const mark = siblings.length ? besideWords(siblings) : c.siblings_unknown ? UNKNOWN_MARK : "";
if (!c.trunk) return mark ? { ...status, description: clipped(`${mark} — ${status.description}`) } : status;
return { ...status, description: clipped(`head ${c.commit.slice(0, 12)} merged into trunk ${c.trunk.slice(0, 12)}${mark ? " " + mark : ""} — ${status.description}`) };
}
/** The comment a verdict leaves on its pull request, with the check's own account: the change plan of a
@@ -240,6 +270,14 @@ export function commentFor(c: Checked, facts: RepoCheckFacts = {}): string | nul
lines.splice(1, 0, `Judged: head \`${c.commit.slice(0, 12)}\` merged into trunk \`${c.trunk.slice(0, 12)}\`` +
(beside ? `; read beside it: ${beside}` : ""));
}
const siblings = siblingsOf(c.siblings);
if (siblings.length) {
lines.splice(c.trunk ? 2 : 1, 0, `Siblings read beside it: ` + siblings.map((s) =>
`${s.repository}#${s.number} \`${s.head.slice(0, 12)}\` merged onto \`${s.base_commit.slice(0, 12)}\` (${s.base})`).join(", ") +
(c.judge_from ? `; judged by ${c.judge_from}` : ""));
} else if (c.siblings_unknown) {
lines.splice(c.trunk ? 2 : 1, 0, `Siblings could not be learned: ${c.siblings_unknown}`);
}
lines.push(`- \`${CHECK_CONTEXT}\`: **${saidAs(gate.verdict).toUpperCase()}** — ${gate.summary}` +
(gate.modules?.length ? ` (modules: ${gate.modules.join(", ")}` +
(gate.dependents?.length ? `; built after them: ${gate.dependents.join(", ")}` : "") + ")" : ""));
+286
View File
@@ -0,0 +1,286 @@
// **A delivery group's member is checked beside the other members' heads, and a sibling's move puts that verdict out of
// date** (novox/hq ADR 0382 §5, §6; issue 867).
//
// The controller's verdict (`checked`) names the siblings its check read: each as the sibling's repository, pull
// request, head, base branch and the base commit the sibling's head was merged onto. This module, the forge's holder,
// keeps them per open pull request, beside its other records, and:
//
// - on its own hearing of a sibling's new head, of its close unmerged, or of a move of its base, sets
// mesh/merge-gate and mesh/repo-check of every member whose verdict names that sibling back to pending, with the
// reason. Nothing is run for it. The one move that does not count is the sibling's own merge of the head that was
// read onto the base commit that was read (in either order of the two merges);
// - lets the merge tool refuse a verdict that read siblings unless mesh/delivery-group passed at the very heads it
// read, and every sibling head it read carries an approving review at that head (merge.ts).
//
// A verdict without siblings (a controller from before ADR 0382) is none of this: nothing is kept and nothing changes.
// Pure functions and one small file of the module's own state, so this is tested without a forge.
import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { dirname } from "node:path";
import type { CommitStatus, GiteaPull, HeadCheck } from "./client.js";
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT, clipped } from "./pulls.js";
import { sameCommit, twelve, type OutOfDateWriter } from "./trunk.js";
/** The status the delivery group's owner sets on each member's head (novox/hq ADR 0239). */
export const GROUP_CONTEXT = "mesh/delivery-group";
/** One sibling a verdict read, as the controller names it (contract of ADR 0382 §4). */
export interface SiblingNamed {
repository: string;
number: number;
head: string;
base: string;
base_commit: string;
/** Set here, never by the controller: the sibling merged its head onto its base commit read, so it is no sibling's
* move any more. */
merged?: boolean;
}
/** What is kept of one member's latest verdict that read siblings: the head it was of and the siblings it named. */
export interface SiblingEntry {
head: string;
siblings: SiblingNamed[];
}
/** The records, by `owner/repo#number` of the member. */
export type SiblingRecord = Record<string, SiblingEntry>;
/** The member's key. */
export function memberKey(owner: string, repo: string, number: number): string {
return `${owner}/${repo}#${number}`.toLowerCase();
}
/** The siblings a verdict names, trimmed to what is kept; none when it names none or names one badly. */
export function siblingsOf(named: unknown): SiblingNamed[] {
if (!Array.isArray(named)) return [];
const out: SiblingNamed[] = [];
for (const s of named) {
const r = s as Record<string, unknown>;
const repository = String(r?.repository ?? ""), head = String(r?.head ?? ""), base = String(r?.base ?? "");
const number = Number(r?.number);
if (!repository || !head || !base || !Number.isInteger(number) || number <= 0) continue;
out.push({ repository, number, head, base, base_commit: String(r?.base_commit ?? "") });
}
return out;
}
/** The siblings as a status names them: "mesh-host#114@abcdef123456, …". */
export function siblingWords(siblings: SiblingNamed[]): string {
return siblings.map((s) => `${s.repository.split("/").pop()}#${s.number}@${twelve(s.head)}`).join(", ");
}
/** What a status of a verdict that read siblings carries, after the commits it judged. */
export function besideWords(siblings: SiblingNamed[]): string {
// Short, so that the verdict word and the start of its reason survive the forge's 140 characters: the first sibling
// by its pull request and 8 characters of its head, and how many more. The whole list is the holder's record and the comment's.
const first = siblings[0];
const more = siblings.length > 1 ? ` +${siblings.length - 1} more` : "";
return `beside ${first.repository.split("/").pop()}#${first.number}@${twelve(first.head).slice(0, 8)}${more}`;
}
/** The mark a status carries when the siblings could not be learned (ADR 0382 §1): said, never silent. */
export const UNKNOWN_MARK = "siblings unknown";
/** Whether a status names that its verdict read siblings (so that a record lost does not read as none). */
export function namesSiblings(description: string | undefined): boolean {
return / beside [\w.-]+#\d+@[0-9a-f]{8}\b/.test(description ?? "");
}
/** Whether a status is a verdict put out of date by a sibling's move. */
export function siblingOutOfDate(description: string | undefined): boolean {
return /^out of date: judged beside /.test(description ?? "");
}
// --- the record on disk ----------------------------------------------------------------------------------------
/** The file the holder keeps the siblings its verdicts name in, beside its other records. */
export function siblingsFile(stateDir: string | undefined): string | null {
return stateDir ? `${stateDir.replace(/\/+$/, "")}/siblings-named.json` : null;
}
export function readSiblings(file: string | null): SiblingRecord {
if (!file || !existsSync(file)) return {};
try {
return (JSON.parse(readFileSync(file, "utf8")) as { members?: SiblingRecord }).members ?? {};
} catch {
return {};
}
}
export function keepSiblings(file: string | null, record: SiblingRecord): void {
if (!file) return;
try {
mkdirSync(dirname(file), { recursive: true });
writeFileSync(file + ".tmp", JSON.stringify({ members: record }));
renameSync(file + ".tmp", file);
} catch (err) {
console.error(`[gitea] the siblings a verdict named could not be kept — ${err instanceof Error ? err.message : err}`);
}
}
/** A member's latest verdict, kept: its siblings when it read some, else the entry removed. Answers whether it changed. */
export function noteVerdict(record: SiblingRecord, key: string, head: string, siblings: SiblingNamed[]): boolean {
if (siblings.length === 0) {
if (!(key in record)) return false;
delete record[key];
return true;
}
record[key] = { head, siblings };
return true;
}
// --- what a sibling's move is ---------------------------------------------------------------------------------
/** What weighing a sibling reads of the forge. */
export interface SiblingForge {
getPullRequest(owner: string, repo: string, index: number): Promise<GiteaPull>;
branchHead(owner: string, repo: string, branch: string): Promise<string>;
commitParents(owner: string, repo: string, sha: string): Promise<string[]>;
}
/** The reason a verdict that named this sibling is out of date, from what the forge holds now, or nothing; and whether
* the sibling has merged its head read onto the base commit read, which ends it as a sibling. */
export async function siblingMoved(forge: SiblingForge, s: SiblingNamed): Promise<{ reason?: string; merged?: boolean }> {
const [owner, repo] = s.repository.split("/");
const who = `${repo}#${s.number}`;
const pull = await forge.getPullRequest(owner, repo, s.number);
if (pull.merged) {
// Its own merge of the head that was read onto the base commit that was read: the head the merge joined is the head
// read, and the commit it was made onto is the base commit read. Any other merge is a move of the base.
const parents = pull.merge_commit_sha ? await forge.commitParents(owner, repo, pull.merge_commit_sha).catch(() => []) : [];
const own = sameCommit(pull.head_sha, s.head) && parents.length >= 1 && sameCommit(parents[0], s.base_commit) &&
(parents.length === 1 || sameCommit(parents[1], s.head));
if (own) return { merged: true };
return { reason: `${who} was merged otherwise than its head ${twelve(s.head)} onto ${twelve(s.base_commit)}` };
}
if (pull.state !== "open") return { reason: `${who} was closed unmerged` };
if (!sameCommit(pull.head_sha, s.head)) {
return { reason: `${who} at ${twelve(s.head)}, it is now at ${twelve(pull.head_sha)}` };
}
const base = await forge.branchHead(owner, repo, s.base);
if (!sameCommit(base, s.base_commit)) {
return { reason: `${who} merged onto ${twelve(s.base_commit)}, its ${s.base} is now ${twelve(base)}` };
}
return {};
}
/** A verdict out of date because a sibling moved, as its statuses say it (ADR 0382 §5; the wording of ADR 0323 §4). */
export function siblingOutOfDateWords(reason: string): string {
return clipped(`out of date: judged beside ${reason}; judged again on a recheck or when it is merged`);
}
/** The statuses a sibling's move puts back to pending on one member's head, from what the forge holds on it now: none
* when it has no merge check, or its gate is pending (a check on its way, or already out of date). */
export function siblingStatuses(checks: Record<string, HeadCheck>, reason: string): CommitStatus[] {
const gate = checks[CHECK_CONTEXT];
if (!gate || gate.state === "pending") return [];
const description = siblingOutOfDateWords(reason);
const out: CommitStatus[] = [{ state: "pending", context: CHECK_CONTEXT, description }];
if (checks[REPO_CHECK_CONTEXT]) out.push({ state: "pending", context: REPO_CHECK_CONTEXT, description });
return out;
}
/** What weighing a sibling's move writes with, beside the forge's reads. */
export interface SiblingWeighing extends SiblingForge {}
/**
* A sibling moved (a new head, a close, a base move, a merge): every member whose latest verdict names it has its
* merge check put back to pending with the reason. `repository` and, when the event names one, `number` say which
* sibling. A member whose head is no longer the one its verdict was of is dropped from the record: its new head is
* checked afresh. Nothing is run. Answers whether the record changed. A read that fails leaves that member for the next
* move to weigh, and is said.
*/
export async function weighSiblings(forge: SiblingWeighing, gates: OutOfDateWriter, record: SiblingRecord,
repository: string, number?: number): Promise<boolean> {
let changed = false;
const repo = repository.toLowerCase();
for (const [key, entry] of Object.entries(record)) {
const [path] = key.split("#");
const [owner, name] = path.split("/");
const memberNumber = Number(key.split("#")[1]);
// A member that is merged or no longer open has no verdict to put out of date: its entry goes (the record must not
// grow without bound, and a merged head is never re-pended by a sibling's later push). Weighed when the event is the
// member's own, or names one of its siblings.
const own = path === repo && (number === undefined || number === memberNumber);
if (own || entry.siblings.some((s) => !s.merged && s.repository.toLowerCase() === repo && (number === undefined || s.number === number))) {
try {
const member = await forge.getPullRequest(owner, name, memberNumber);
if (member.merged || member.state !== "open") {
delete record[key];
changed = true;
continue;
}
} catch (err) {
console.error(`[gitea] ${key}: whether it is still open could not be read — ${err instanceof Error ? err.message : err}`);
}
}
for (const s of entry.siblings) {
if (s.merged || s.repository.toLowerCase() !== repo || (number !== undefined && s.number !== number)) continue;
try {
const moved = await siblingMoved(forge, s);
if (moved.merged) {
s.merged = true;
changed = true;
continue;
}
if (!moved.reason) continue;
const member = await forge.getPullRequest(owner, name, memberNumber);
if (!sameCommit(member.head_sha, entry.head)) {
delete record[key];
changed = true;
break;
}
const written = await gates.outOfDate(owner, name, entry.head, (checks) => siblingStatuses(checks, moved.reason!),
member.html_url || undefined);
if (written.length) {
console.log(`[gitea] ${key}: its verdict was judged beside ${moved.reason}: out of date, nothing run (novox/hq ADR 0382)`);
}
break;
} catch (err) {
console.error(`[gitea] ${key}: whether ${s.repository}#${s.number} moved could not be weighed, so it is weighed again ` +
`on its next move — ${err instanceof Error ? err.message : err}`);
}
}
}
return changed;
}
// --- the merge tool's rule (ADR 0382 §6) ----------------------------------------------------------------------
/** What the rule reads beside the pull request being merged. */
export interface GroupForge {
headChecks(owner: string, repo: string, sha: string): Promise<Record<string, HeadCheck>>;
/** Whether the sibling's head carries an approving review at that head. */
approvedAt(owner: string, repo: string, number: number, head: string): Promise<boolean>;
}
/**
* What stops a pull request whose verdict read siblings from merging, in words, or nothing: mesh/delivery-group must be
* a success at its own head and at every sibling head the verdict read, and every sibling head read must carry an
* approving review at that head. Every missing thing is named.
*/
export async function groupRefusal(forge: GroupForge, self: { owner: string; repo: string; number: number; head: string },
siblings: SiblingNamed[]): Promise<string | undefined> {
const missing: string[] = [];
const members = [{ repository: `${self.owner}/${self.repo}`, number: self.number, head: self.head, sibling: false },
...siblings.map((s) => ({ repository: s.repository, number: s.number, head: s.head, sibling: true }))];
for (const m of members) {
const [owner, repo] = m.repository.split("/");
const at = `${repo}#${m.number}@${twelve(m.head)}`;
let group: HeadCheck | undefined;
try {
group = (await forge.headChecks(owner, repo, m.head))[GROUP_CONTEXT];
} catch (err) {
missing.push(`${GROUP_CONTEXT} at ${at} could not be read (${err instanceof Error ? err.message : err})`);
continue;
}
if (group?.state !== "success") missing.push(`${GROUP_CONTEXT} is ${group?.state ?? "missing"} at ${at}`);
if (!m.sibling) continue;
try {
if (!(await forge.approvedAt(owner, repo, m.number, m.head))) missing.push(`${at} carries no approving review at that head`);
} catch (err) {
missing.push(`whether ${at} carries an approving review could not be read (${err instanceof Error ? err.message : err})`);
}
}
return missing.length ? missing.join("; ") : undefined;
}
+130 -4
View File
@@ -47,11 +47,18 @@ test("every open pull request is read with what its base requires and what its h
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/pulls/14")) {
return send({ number: 14, state: "open", base: { ref: "main" }, head: { ref: "y", sha: "bbb2222" }, html_url: "s" });
}
if (url.startsWith("/api/v1/repos/novox/hq/branch_protections")) {
return send([{ rule_name: "main", enable_status_check: true, status_check_contexts: ["mesh/repo-check"] }]);
// The base's protection, read from the branch as the tools' account may (novox/hq ADR 0365); the rules' list is refused it.
if (url.includes("/branch_protections")) {
res.statusCode = 403;
return res.end(JSON.stringify({ message: "the rules are a repository administrator's" }));
}
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/branch_protections")) {
return send([{ rule_name: "main", enable_status_check: true, status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"] }]);
if (url.startsWith("/api/v1/repos/novox/hq/branches/main")) {
return send({ name: "main", protected: true, effective_branch_protection_name: "main", enable_status_check: true,
status_check_contexts: ["mesh/repo-check"] });
}
if (url.startsWith("/api/v1/repos/novox/mesh-sdk/branches/main")) {
return send({ name: "main", protected: true, effective_branch_protection_name: "main", enable_status_check: true,
status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"] });
}
if (url.includes("/commits/aaa1111/status")) return send({ statuses: [] });
if (url.includes("/commits/bbb2222/status")) return send({ statuses: [{ context: "mesh/merge-gate", status: "pending",
@@ -137,3 +144,122 @@ test("a merged pull request whose branch was deleted is announced with its branc
assert.equal(headBranchOf({ ref: "refs/pull/7/head" }), "refs/pull/7/head");
assert.equal(headBranchOf(undefined), undefined);
});
// **A pull request marked ready says the head it is marked at** (novox/hq issue 722): the label mesh/ready and the
// newest comment "ready at <full head sha>" by a listed account, written after the label was added and after the last
// push. Read from the pull request's whole timeline, page by page; without the label nothing is read.
const head = "c".repeat(40), older = "d".repeat(40);
const ev = (id: number, type: string, body: string, user = "mesh-admin", label?: { id: number; name: string }) =>
({ id, type, body, user: { login: user }, created_at: `2026-10-14T10:${String(id % 60).padStart(2, "0")}:00Z`,
...(label ? { label } : {}) });
const readyLabel = { id: 31, name: "mesh/ready" };
function forgeWith(timeline: unknown[]) {
const asked: string[] = [];
const server = createServer((req, res) => {
const url = req.url ?? "";
asked.push(url);
const send = (v: unknown) => res.end(JSON.stringify(v));
if (url.startsWith("/api/v1/repos/issues/search")) {
return send(url.includes("page=1") ? [
{ number: 7, repository: { owner: "novox", name: "hq", full_name: "novox/hq" },
labels: [{ id: 31, name: "mesh/ready" }, { id: 2, name: "bug" }] },
{ number: 8, repository: { owner: "novox", name: "hq", full_name: "novox/hq" }, labels: [] },
] : []);
}
if (url.startsWith("/api/v1/repos/novox/hq/pulls/7")) {
return send({ number: 7, state: "open", base: { ref: "main" }, head: { ref: "x", sha: head }, html_url: "h" });
}
if (url.startsWith("/api/v1/repos/novox/hq/pulls/8")) {
return send({ number: 8, state: "open", base: { ref: "main" }, head: { ref: "y", sha: older }, html_url: "h" });
}
const t = /\/issues\/7\/timeline\?page=(\d+)&limit=50/.exec(url);
if (t) {
const page = Number(t[1]);
return send(timeline.slice((page - 1) * 50, page * 50));
}
if (url.includes("/branch_protections")) return send([]);
if (url.includes("/status")) return send({ statuses: [] });
res.statusCode = 404;
res.end("{}");
});
return { server, asked };
}
async function readOpen(timeline: unknown[], authors = ["mesh-admin", "jschoubben"]) {
const { GiteaClient } = await import("../client.ts");
const { server, asked } = forgeWith(timeline);
await new Promise<void>((r) => server.listen(0, r));
try {
const client = new GiteaClient(`http://127.0.0.1:${(server.address() as any).port}`, "t");
return { open: await client.openPulls(authors), asked };
} finally {
server.close();
}
}
test("a pull request's ready mark is read: the label and the head the newest listed ready comment names", async () => {
const { open, asked } = await readOpen([
ev(1, "pull_push", JSON.stringify({ commit_ids: [head] })),
ev(2, "label", "1", "mesh-admin", readyLabel),
ev(3, "comment", `ready at ${older}`),
ev(4, "comment", `Reviewed: approve.\nready at ${head}\n`),
ev(5, "comment", "thanks", "jschoubben"),
]);
assert.deepEqual(open[0].labels, ["mesh/ready", "bug"]);
assert.deepEqual(open[0].ready, { head, by: "mesh-admin", at: "2026-10-14T10:04:00Z", label_id: 31 });
assert.equal(open[1].ready, undefined, "no label, no mark");
assert.ok(!asked.some((u) => u.includes("/issues/8/")), "no timeline read without the label");
});
test("a ready comment by an account not listed is ignored, and said", async () => {
const { readyMarkOf } = await import("../client.ts");
const said: string[] = [];
const mark = readyMarkOf(31, [
ev(1, "pull_push", "{}") as any, ev(2, "label", "1", "mesh-admin", readyLabel) as any,
{ ...ev(3, "comment", `ready at ${head}`, "outsider"), user: "outsider" },
].map((e: any) => ({ ...e, user: typeof e.user === "string" ? e.user : e.user.login })), ["mesh-admin"], (l) => said.push(l));
assert.equal(mark.head, "", "an outsider's mark counts");
assert.equal(said.length, 1);
assert.match(said[0], /by outsider is ignored/);
});
test("a ready comment older than the latest push, or than the label, never counts", async () => {
const pushedAfter = await readOpen([
ev(1, "label", "1", "mesh-admin", readyLabel),
ev(2, "comment", `ready at ${head}`),
ev(3, "pull_push", JSON.stringify({ commit_ids: [head] })),
]);
assert.equal(pushedAfter.open[0].ready?.head, "", "a mark written before the latest push counts, the label still on");
const beforeLabel = await readOpen([
ev(1, "pull_push", "{}"),
ev(2, "comment", `ready at ${head}`),
ev(3, "label", "1", "mesh-admin", readyLabel),
]);
assert.equal(beforeLabel.open[0].ready?.head, "", "a mark written before the label was added counts");
const takenOff = await readOpen([
ev(1, "label", "1", "mesh-admin", readyLabel),
ev(2, "comment", `ready at ${head}`),
ev(3, "label", "", "mesh-admin", readyLabel),
]);
assert.equal(takenOff.open[0].ready?.head, "", "a mark whose label was taken off since counts");
});
test("a long thread hides nothing: every page of the timeline is read", async () => {
const events: unknown[] = [ev(1, "pull_push", "{}"), ev(2, "label", "1", "mesh-admin", readyLabel),
ev(3, "comment", `ready at ${older}`)];
for (let id = 4; id < 130; id++) events.push(ev(id, "comment", "a long discussion", "jschoubben"));
events.push(ev(130, "comment", `ready at ${head}`, "jschoubben"));
const { open, asked } = await readOpen(events);
assert.equal(open[0].ready?.head, head, "the newest mark, on the third page, is not read");
assert.equal(asked.filter((u) => u.includes("/issues/7/timeline")).length, 3);
});
test("the accounts that may mark are the setting's, and only the mesh's own when it cannot be read", async () => {
const { readyAuthors } = await import("../client.ts");
assert.deepEqual(readyAuthors("f", "mesh-admin", () => "# x\nauthors=mesh-admin jschoubben\n"), ["mesh-admin", "jschoubben"]);
// The tools' account and the admin's, whose marks were made before ADR 0365 (issue 469).
assert.deepEqual(readyAuthors("f", "mesh-admin", () => { throw new Error("gone"); }, "mesh-tools"), ["mesh-tools", "mesh-admin"]);
assert.deepEqual(readyAuthors("f", "mesh-admin", () => "authors=\n", "mesh-tools"), ["mesh-tools", "mesh-admin"]);
assert.deepEqual(readyAuthors(undefined, "mesh-admin", undefined, "mesh-tools"), ["mesh-tools", "mesh-admin"]);
});
+98 -2
View File
@@ -30,6 +30,9 @@ const state = {
// What the tools said on the bus.
emitted: [] as { type: string; body: any }[],
rules: [] as any[],
// Whether the token is a repository administrator's. The tools' account is not (novox/hq ADR 0365): the forge
// refuses it the rules' list, as gitea's reqAdmin does, and it reads a branch's protection from the branch.
admin: false,
lastCommit: "2026-01-01T00:00:00Z",
// The pull request's head; the head the merge commit joined (the same, unless a test says otherwise) — the forge's
// record of the pull request keeps saying `head` either way, as it would in the race issue 381 is about; and the
@@ -41,8 +44,13 @@ const state = {
// When set, the merge commit's parents as the forge reports them, or the status it answers them with.
parents: null as string[] | null,
commitsFail: 0,
// The sibling beside which a verdict was judged (novox/hq ADR 0382 §6): its group status at its head, and its reviews.
groupStatus: "",
siblingGroup: "",
reviews: [] as { state: string; commit_id: string; dismissed?: boolean }[],
};
const REVIEWED = "abc1234def5678901234567890abcdef12345678";
const SIBLING_HEAD = "bbbbbbbbbbbb5678901234567890abcdef12345678".slice(0, 40);
const TRUNK = "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b";
const MOVED = "3c4d5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d";
// A verdict judged against a trunk commit, as both statuses say it (novox/hq ADR 0323 §3).
@@ -59,12 +67,16 @@ beforeEach(() => {
state.trunk = TRUNK;
state.emitted = [];
state.rules = [protectedMain()];
state.admin = false;
state.head = REVIEWED;
state.landed = "";
state.merged = false;
state.mergeCommit = "feedface00000000000000000000000000000000";
state.parents = null;
state.commitsFail = 0;
state.groupStatus = "";
state.siblingGroup = "";
state.reviews = [];
});
const json = (res: ServerResponse, status: number, body: unknown): void => {
@@ -87,8 +99,18 @@ const server = createServer(async (req, res) => {
if (method !== "GET") state.sent.push({ method, path, body });
const repo = "/repos/novox/mesh-catalog";
if (method === "GET" && path === repo) return json(res, 200, { full_name: "novox/mesh-catalog", default_branch: "main" });
if (path.startsWith(`${repo}/branch_protections`) && !state.admin) {
return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
}
if (method === "GET" && path === `${repo}/branch_protections`) return json(res, 200, state.rules);
if (method === "GET" && path === `${repo}/branches/main`) return json(res, 200, { name: "main", commit: { id: state.trunk } });
const branch = method === "GET" && path.startsWith(`${repo}/branches/`) ? decodeURIComponent(path.slice(`${repo}/branches/`.length)) : null;
if (branch !== null) {
// As the forge says it to any reader of the code: the rule that covers the branch, as the forge matches it.
const rule = (state.rules as any[]).find((r) => ruleCovers(String(r.rule_name ?? r.branch_name), branch));
return json(res, 200, { name: branch, commit: { id: branch === "main" ? state.trunk : "c0ffee" }, protected: !!rule,
...(rule ? { effective_branch_protection_name: rule.rule_name ?? rule.branch_name, enable_status_check: !!rule.enable_status_check,
status_check_contexts: rule.status_check_contexts ?? [], required_approvals: rule.required_approvals ?? 0, user_can_push: false } : {}) });
}
if (method === "GET" && path === `${repo}/pulls/7/files`) {
return json(res, 200, url.searchParams.get("page") === "1" ? [{ filename: "modules/gitea/x.ts", status: "modified" }] : []);
}
@@ -113,8 +135,17 @@ const server = createServer(async (req, res) => {
return json(res, 200, { sha: state.mergeCommit, parents: parents.map((sha) => ({ sha })) });
}
if (method === "GET" && path === `${repo}/commits/${state.head}/status`) {
return json(res, 200, { sha: state.head, state: "success", statuses: state.statuses });
const group = state.groupStatus ? [{ context: "mesh/delivery-group", status: state.groupStatus }] : [];
return json(res, 200, { sha: state.head, state: "success", statuses: [...state.statuses, ...group] });
}
if (method === "GET" && path === `/repos/novox/mesh-host/commits/${SIBLING_HEAD}/status`) {
return json(res, 200, { sha: SIBLING_HEAD, state: "success",
statuses: state.siblingGroup ? [{ context: "mesh/delivery-group", status: state.siblingGroup }] : [] });
}
if (method === "GET" && path === "/repos/novox/mesh-host/pulls/114/reviews") {
return json(res, 200, url.searchParams.get("page") === "1" ? state.reviews : []);
}
if (method === "GET" && path === "/repos/novox/mesh-host/issues/114/labels") return json(res, 200, []);
if (method === "POST" && path === `${repo}/pulls/7/merge`) {
state.merged = true;
return json(res, 200, null);
@@ -199,6 +230,7 @@ test("a protection rule covers a branch as the forge decides it: classes, altern
});
test("gitea_api still writes what an author writes", async () => {
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/issues/3/comments", method: "POST", body: { body: "hi" } });
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/pulls/7/reviews", method: "POST", body: { event: "COMMENT" } });
await tool("gitea_api").run({ path: "/repos/novox/mesh-catalog/branch_protections" });
@@ -206,6 +238,7 @@ test("gitea_api still writes what an author writes", async () => {
});
test("gitea_branch_protection_set only strengthens", async () => {
state.admin = true; // a repository administrator's token: the tools' account is refused the rules by the forge
const set = (args: Record<string, unknown>) =>
tool("gitea_branch_protection_set").run({ owner: "novox", repo: "mesh-catalog", branch: "main", ...args });
await assert.rejects(set({ status_checks: "" }), /drops the required status mesh\/merge-gate, mesh\/repo-check/);
@@ -396,6 +429,7 @@ test("gitea_merge_pull_request reads back a fast-forward, and says it cannot rea
});
test("gitea_delete_repo deletes only the tools' own user's repository, unguarded, untouched for 30 days, named in full", async () => {
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
const del = (owner: string, name: string, confirm: unknown) => tool("gitea_delete_repo").run({ owner, name, confirm });
await assert.rejects(del("novox", "mesh-catalog", true), /confirm must be the repository's full name/);
await assert.rejects(del("novox", "mesh-catalog", "novox/mesh-catalog"), /not a repository of the tools' own user/);
@@ -427,6 +461,7 @@ test("gitea_merge_pull_request merges only into a branch whose protection requir
});
test("a rule named for a branch is not made weaker than the glob rule that guards it now", async () => {
state.admin = true; // reads the rules' list, a repository administrator's (novox/hq ADR 0365)
state.rules.push({ rule_name: "release/*", enable_push: false, enable_status_check: true,
status_check_contexts: ["mesh/merge-gate"], block_admin_merge_override: true });
const set = (args: Record<string, unknown>) =>
@@ -511,3 +546,64 @@ test("no tool's caller text becomes another path or another user: `..` and `?sud
assert.deepEqual(state.asked, ["DELETE /repos/novox/mesh-catalog/branches/feat/x",
"GET /repos/novox/mesh-catalog/contents/modules/gitea/module.json"]);
});
// **A member judged beside its siblings merges only with its group passed and reviewed** (novox/hq ADR 0382 §6, issue 867):
// the verdict names the sibling; mesh/delivery-group must be a success at the member's head and the sibling's head that
// verdict read, and the sibling's head must carry an approving review. A verdict that read none merges as before.
test("gitea_merge_pull_request merges a verdict that read siblings only with its group passed and each sibling head reviewed", async () => {
const { keepSiblings, siblingsFile } = await import("../dist/siblings.js");
const merge = () => tool("gitea_merge_pull_request").run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED });
const beside = `head ${REVIEWED.slice(0, 12)} merged into trunk ${TRUNK.slice(0, 12)} beside mesh-host#114@${SIBLING_HEAD.slice(0, 8)}`;
const named = (siblings: unknown[]) => keepSiblings(siblingsFile(stateDir),
{ "novox/mesh-catalog#7": { head: REVIEWED, siblings: siblings as any } });
state.statuses = [{ context: "mesh/merge-gate", state: "success", description: `${beside} — pass: x` },
{ context: "mesh/repo-check", state: "success", description: `${beside} — pass: y` }];
named([{ repository: "novox/mesh-host", number: 114, head: SIBLING_HEAD, base: "main", base_commit: "c".repeat(40) }]);
// Nothing passed: every missing thing is named, nothing merged.
await assert.rejects(merge(), (err: Error) => {
assert.match(err.message, /is not merged: its verdict was judged beside 1 sibling\(s\), and mesh\/delivery-group is missing at mesh-catalog#7@abc1234def56; mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review at that head/);
return true;
});
assert.equal(state.merged, false);
// The group passed at the member's head only: the sibling's head still lacks it.
state.groupStatus = "success";
await assert.rejects(merge(), /mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review/);
// Passed at both heads, but the only approval is at another head, or dismissed: not an approval at the head read.
state.siblingGroup = "success";
state.reviews = [{ state: "APPROVED", commit_id: "d".repeat(40) }, { state: "APPROVED", commit_id: SIBLING_HEAD, dismissed: true },
{ state: "COMMENT", commit_id: SIBLING_HEAD }];
await assert.rejects(merge(), /judged beside 1 sibling\(s\), and mesh-host#114@bbbbbbbbbbbb carries no approving review at that head\. A member judged beside/);
assert.equal(state.merged, false);
// Passed and approved at the very heads read: it merges.
state.reviews.push({ state: "APPROVED", commit_id: SIBLING_HEAD });
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
// A status that says it read siblings, with no record of them, is no verdict to merge on.
state.merged = false;
keepSiblings(siblingsFile(stateDir), {});
await assert.rejects(merge(), /its verdict read siblings, and which ones is not held here/);
assert.equal(state.merged, false);
// A verdict that read none (an older controller, or a group of one) merges exactly as before.
state.statuses = judgedAt(TRUNK);
state.groupStatus = "";
assert.equal(((await merge()) as Record<string, unknown>).merged, true);
});
test("a verdict put out of date by a sibling's move is refused in its own words, and asked again once", async () => {
const merge = () => tool("gitea_merge_pull_request").run({ owner: "novox", repo: "mesh-catalog", number: 7, head: REVIEWED });
const words = "out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd; judged again on a recheck or when it is merged";
state.statuses = judgedAt(TRUNK).map((s) => ({ ...s, state: "pending", description: words }));
await assert.rejects(merge(), (err: Error) => {
assert.match(err.message, /its verdict was put out of date by a sibling's move \(out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd/);
assert.match(err.message, /was asked/);
assert.match(err.message, /never merges by itself/);
return true;
});
assert.equal(state.merged, false);
assert.equal(state.emitted.length, 1, "the check is asked, once");
});
@@ -0,0 +1,59 @@
// The controller's build-source check asks gitea_branch_protection_get for a repository's rules, which the forge
// answers to a repository's administrator alone. The tools' account is not one (novox/hq ADR 0365 §2), so the read
// goes with the forge's administrator account, and only the read (ADR 0375, issue 772): a protection edit through the
// tools still gets the forge's refusal.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { createServer } from "node:http";
import { GiteaAdmin, GiteaClient } from "../dist/client.js";
import { ConfiguredToken } from "../dist/token.js";
import { getGiteaTools } from "../dist/tools/index.js";
const adminBasic = "Basic " + Buffer.from("mesh-admin:secret").toString("base64");
const rules = [
{ rule_name: "main", enable_push: false, enable_status_check: true, status_check_contexts: ["mesh/merge-gate"], block_admin_merge_override: true },
{ branch_name: "release", enable_push: false },
];
const seen: string[] = [];
const server = createServer((req, res) => {
const path = (req.url ?? "").replace(/^\/api\/v1/, "");
const as = req.headers.authorization === adminBasic ? "admin" : "tools";
seen.push(`${req.method} ${path} as ${as}`);
const send = (status: number, body: unknown) => { res.writeHead(status, { "Content-Type": "application/json" }); res.end(JSON.stringify(body)); };
if (path.includes("/branch_protections")) {
if (as !== "admin") return send(403, { message: "user should be an owner or a collaborator with admin write of a repository" });
if (req.method === "GET") return send(200, rules);
}
send(404, { message: "not found" });
});
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
after(() => new Promise<void>((r) => server.close(() => r())));
const { port } = server.address() as { port: number };
const url = `http://127.0.0.1:${port}`;
const tools = getGiteaTools(new GiteaClient(url, new ConfiguredToken("tools-token")), () => new GiteaAdmin(url, "mesh-admin", "secret"));
const tool = (name: string) => tools.find((t) => t.name === name)!;
test("gitea_branch_protection_get answers every rule, read with the administrator account the tools' token cannot read with", async () => {
const out: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller" });
assert.deepEqual(out.rules.map((r: any) => r.rule), ["main", "release"]);
});
test("gitea_branch_protection_get answers the rule for a branch, null when it has none", async () => {
const main: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller", branch: "main" });
assert.equal(main.branch, "main");
assert.equal(main.rule.rule, "main");
assert.equal(main.rule.admin_may_override, false);
const none: any = await tool("gitea_branch_protection_get").run({ owner: "novox", repo: "mesh-controller", branch: "dev" });
assert.equal(none.rule, null);
});
test("gitea_branch_protection_set still gets the forge's refusal, as the tools' account", async () => {
await assert.rejects(
tool("gitea_branch_protection_set").run({ owner: "novox", repo: "mesh-controller", branch: "main", status_checks: "mesh/merge-gate" }),
/403/,
);
assert.ok(!seen.some((s) => /^(PATCH|POST|PUT|DELETE) .*branch_protections.* as admin$/.test(s)), seen.join("\n"));
});
+269
View File
@@ -0,0 +1,269 @@
// **A delivery group's member is judged beside its siblings, and a sibling's move puts that verdict out of date**
// (novox/hq ADR 0382 §4-§6, issue 867): the holder keeps the siblings each verdict names; a sibling's new head, close
// unmerged or base move (other than its own merge of the head that was read) sets both statuses of every member whose
// verdict names it back to pending, and runs nothing; a member merges only with its group passed and every sibling head
// read approved.
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { besideWords, groupRefusal, keepSiblings, namesSiblings, noteVerdict, readSiblings, siblingOutOfDate, siblingsFile,
siblingsOf, siblingStatuses, weighSiblings, type SiblingRecord } from "../dist/siblings.js";
import { weighTrunks } from "../dist/trunk.js";
import { commentFor, statusesFor } from "../dist/pulls.js";
const h = (c: string) => c.repeat(40);
const MEMBER_HEAD = h("a"), HOST_HEAD = h("b"), HOST_BASE = h("c"), HOST_NEW = h("d"), HOST_MOVED = h("e"), MERGE = h("f");
const sibling = { repository: "novox/mesh-host", number: 114, head: HOST_HEAD, base: "main", base_commit: HOST_BASE };
const verdict = { owner: "novox", repo: "mesh-controller", number: 346, commit: MEMBER_HEAD, verdict: "pass", summary: "x", id: "build-1",
trunk: h("1").slice(0, 12), siblings: [sibling], judge_from: "sibling novox/mesh-controller#346 merge 123456789abc",
gate: { verdict: "pass", summary: "every machine composes ".repeat(10) }, "repo-check": { verdict: "pass", summary: "ok" } };
test("both statuses name the siblings the verdict read, keeping the verdict word and the start of its reason within 140 characters", () => {
// A realistic controller summary for a failure, long enough that the forge clips it.
const reason = "2 of 4 machines do not compose: ace: mesh-host.service needs seat node-reads, which no module on ace claims after this change";
const c = { ...verdict, gate: { verdict: "fail", summary: reason }, "repo-check": { verdict: "fail", summary: "its merge-check.sh failed: internal/link/check.go:145:2: undefined" } };
const [gate, repo] = statusesFor(c);
for (const s of [gate, repo]) {
assert.ok(s.description.startsWith(`head ${MEMBER_HEAD.slice(0, 12)} merged into trunk 111111111111 beside mesh-host#114@bbbbbbbb — fail: `), s.description);
assert.ok(s.description.length <= 140, `${s.description.length} characters`);
assert.ok(namesSiblings(s.description));
}
assert.match(gate.description, /fail: 2 of 4 machines do not compose: ace: mesh-host/, gate.description);
assert.match(repo.description, /fail: its merge-check\.sh failed: internal/, repo.description);
assert.equal(besideWords([...siblingsOf([sibling]), ...siblingsOf([{ ...sibling, number: 115 }])]), "beside mesh-host#114@bbbbbbbb +1 more");
const comment = commentFor({ ...verdict, gate: { verdict: "fail", summary: "no" } })!;
assert.match(comment, /Siblings read beside it: novox\/mesh-host#114 `bbbbbbbbbbbb` merged onto `cccccccccccc` \(main\); judged by sibling novox\/mesh-controller#346 merge/);
});
test("a verdict that could not learn its siblings says so on the status and in the comment, and names none", () => {
const { siblings, ...unknown } = { ...verdict, siblings_unknown: "mesh-delivery did not answer" };
const reason = "2 of 4 machines do not compose: ace: mesh-host.service needs seat node-reads, which no module claims";
for (const s of statusesFor({ ...unknown, gate: { verdict: "fail", summary: reason } })) {
assert.match(s.description, /^head aaaaaaaaaaaa merged into trunk 111111111111 siblings unknown — (fail|pass): /, s.description);
assert.ok(s.description.length <= 140);
assert.equal(namesSiblings(s.description), false, "it read no siblings, so the merge tool asks for none");
}
assert.match(commentFor({ ...unknown, gate: { verdict: "fail", summary: "no" } })!, /Siblings could not be learned: mesh-delivery did not answer/);
});
test("a verdict without siblings is said exactly as before (an older controller)", () => {
const { siblings, judge_from, ...old } = verdict;
const [gate] = statusesFor(old);
assert.equal(namesSiblings(gate.description), false);
assert.match(gate.description, /^head aaaaaaaaaaaa merged into trunk 111111111111 — pass: /);
assert.deepEqual(siblingsOf(undefined), []);
assert.deepEqual(siblingsOf([{ repository: "novox/x" }]), [], "a sibling named without its head is none");
});
test("the siblings a verdict names are kept per member, replaced by its next verdict, and removed by one that read none", () => {
const file = siblingsFile(mkdtempSync(join(tmpdir(), "gitea-siblings-")));
const record: SiblingRecord = {};
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, []), false);
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, siblingsOf([sibling])), true);
keepSiblings(file, record);
assert.deepEqual(readSiblings(file), record, "kept where the holder keeps its state");
assert.equal(noteVerdict(record, "novox/mesh-controller#346", MEMBER_HEAD, []), true);
assert.deepEqual(record, {});
assert.deepEqual(readSiblings(null), {});
});
// --- a sibling's move -----------------------------------------------------------------------------------------
interface World {
host: { merged: boolean; state: string; head_sha: string; merge_commit_sha?: string };
hostBase: string;
parents: string[];
member: { head_sha: string; state: string; merged: boolean };
checks: Record<string, { state: string; description?: string }>;
written: { sha: string; context: string; state: string; description: string }[];
reads: string[];
}
const world = (): World => ({ host: { merged: false, state: "open", head_sha: HOST_HEAD }, hostBase: HOST_BASE, parents: [],
member: { head_sha: MEMBER_HEAD, state: "open", merged: false },
checks: { "mesh/merge-gate": { state: "success", description: "head aaaaaaaaaaaa merged into trunk 111111111111 beside mesh-host#114@bbbbbbbb — pass: x" },
"mesh/repo-check": { state: "success", description: "pass: y" } },
written: [], reads: [] });
const forgeOf = (w: World) => ({
async getPullRequest(owner: string, repo: string, n: number) {
w.reads.push(`${owner}/${repo}#${n}`);
return repo === "mesh-host" ? { number: n, title: "", html_url: "", ...w.host } : { number: n, title: "", html_url: "", merged: false, ...w.member };
},
async branchHead(_o: string, _r: string, _b: string) { return w.hostBase; },
async commitParents() { return w.parents; },
});
const gatesOf = (w: World) => ({
async outOfDate(_o: string, _r: string, sha: string, decide: (c: any) => any[] | Promise<any[]>) {
const put = await decide(w.checks);
for (const s of put) { w.written.push({ sha, ...s }); w.checks[s.context] = { state: s.state, description: s.description }; }
return put;
},
});
const recordOf = (): SiblingRecord => ({ "novox/mesh-controller#346": { head: MEMBER_HEAD, siblings: siblingsOf([sibling]) } });
const weigh = (w: World, record: SiblingRecord, repository = "novox/mesh-host", number?: number) =>
weighSiblings(forgeOf(w) as any, gatesOf(w) as any, record, repository, number);
test("a sibling's new head puts both statuses of the member back to pending, with the reason, and runs nothing", async () => {
const w = world();
w.host.head_sha = HOST_NEW;
await weigh(w, recordOf(), "novox/mesh-host", 114);
assert.deepEqual(w.written.map((s) => [s.sha, s.context, s.state]),
[[MEMBER_HEAD, "mesh/merge-gate", "pending"], [MEMBER_HEAD, "mesh/repo-check", "pending"]]);
assert.equal(w.written[0].description,
"out of date: judged beside mesh-host#114 at bbbbbbbbbbbb, it is now at dddddddddddd; judged again on a recheck or when it is merged");
assert.ok(siblingOutOfDate(w.written[0].description));
// Already pending (out of date, or a check on its way): not written again.
w.written.length = 0;
await weigh(w, recordOf(), "novox/mesh-host", 114);
assert.deepEqual(w.written, []);
});
test("a sibling closed unmerged puts the member's verdict out of date", async () => {
const w = world();
w.host.state = "closed";
await weigh(w, recordOf(), "novox/mesh-host", 114);
assert.equal(w.written.length, 2);
assert.match(w.written[0].description, /^out of date: judged beside mesh-host#114 was closed unmerged/);
});
test("a move of the sibling's base puts it out of date, naming both commits", async () => {
const w = world();
w.hostBase = HOST_MOVED;
await weigh(w, recordOf());
assert.equal(w.written.length, 2);
assert.match(w.written[0].description, /mesh-host#114 merged onto cccccccccccc, its main is now eeeeeeeeeeee/);
});
test("the sibling's own merge of the head read onto the base read does not, in either order of the two merges", async () => {
// The sibling merges first (a merge commit of the base commit read and the head read).
const w = world();
Object.assign(w.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
w.hostBase = MERGE;
w.parents = [HOST_BASE, HOST_HEAD];
const record = recordOf();
assert.equal(await weigh(w, record), true, "the record changes: the sibling is marked merged");
assert.deepEqual(w.written, []);
assert.equal(record["novox/mesh-controller#346"].siblings[0].merged, true);
// A move of that base afterwards is no sibling's move: the sibling is merged.
w.hostBase = HOST_MOVED;
await weigh(w, record);
assert.deepEqual(w.written, []);
// The member merges first: its own repository moves, which no member's verdict names as a sibling.
const m = world();
m.member.merged = true;
await weigh(m, recordOf(), "novox/mesh-controller");
assert.deepEqual(m.written, []);
// And the sibling then merges: still not a move.
Object.assign(m.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
m.parents = [HOST_BASE, HOST_HEAD];
await weigh(m, recordOf());
assert.deepEqual(m.written, []);
// A squash is one parent: the base commit read.
const q = world();
Object.assign(q.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
q.parents = [HOST_BASE];
await weigh(q, recordOf());
assert.deepEqual(q.written, []);
});
test("a merge of another head, or onto another base, is a move", async () => {
const other = world();
Object.assign(other.host, { merged: true, state: "closed", merge_commit_sha: MERGE, head_sha: HOST_NEW });
other.parents = [HOST_BASE, HOST_NEW];
await weigh(other, recordOf());
assert.equal(other.written.length, 2);
const onto = world();
Object.assign(onto.host, { merged: true, state: "closed", merge_commit_sha: MERGE });
onto.parents = [HOST_MOVED, HOST_HEAD];
await weigh(onto, recordOf());
assert.equal(onto.written.length, 2);
});
test("only the members whose verdict names the sibling are touched, and a member that moved on is dropped", async () => {
// Another sibling's move, or another repository's: nothing read, nothing written.
const w = world();
w.host.head_sha = HOST_NEW;
await weigh(w, recordOf(), "novox/mesh-lab");
await weigh(w, recordOf(), "novox/mesh-host", 999);
assert.deepEqual(w.written, []);
// A verdict that is none (a gate not yet judged) is left alone.
const none = world();
none.host.head_sha = HOST_NEW;
none.checks = {};
await weigh(none, recordOf());
assert.deepEqual(none.written, []);
// The member has a new head: its old verdict is no verdict; the record is dropped, nothing written to the old head.
const moved = world();
moved.host.head_sha = HOST_NEW;
moved.member.head_sha = h("9");
const record = recordOf();
assert.equal(await weigh(moved, record), true);
assert.deepEqual(moved.written, []);
assert.deepEqual(record, {});
// A repository without a repo-check gets no pending one a verdict would never set.
assert.deepEqual(siblingStatuses({ "mesh/merge-gate": { state: "success" } }, "x").map((s) => s.context), ["mesh/merge-gate"]);
});
test("a trunk that moved tells the siblings' weighing; one first seen does not", async () => {
const told: string[] = [];
const forge = { listOpenPulls: async () => [{ number: 1, title: "", state: "open", merged: false, head_sha: h("2"), base: "main", html_url: "" }],
branchHead: async () => h("3") };
const gates = { outOfDate: async () => [] };
const trunks: Record<string, string> = {};
await weighTrunks(forge as any, gates as any, trunks, "novox", "mesh-host", async (b) => { told.push(b); });
assert.deepEqual(told, [], "first seen");
trunks["novox/mesh-host:main"] = h("4");
await weighTrunks(forge as any, gates as any, trunks, "novox", "mesh-host", async (b) => { told.push(b); });
assert.deepEqual(told, ["main"]);
});
// --- the merge rule -------------------------------------------------------------------------------------------
test("the group rule names every missing thing: the group status at each head read, and each sibling head's review", async () => {
const self = { owner: "novox", repo: "mesh-controller", number: 346, head: MEMBER_HEAD };
const statuses: Record<string, string> = {};
const approved = new Set<string>();
const forge = {
headChecks: async (_o: string, r: string, sha: string) => (statuses[`${r}@${sha}`] ? { "mesh/delivery-group": { state: statuses[`${r}@${sha}`] } } : {}),
approvedAt: async (_o: string, r: string, n: number, head: string) => approved.has(`${r}#${n}@${head}`),
};
const siblings = siblingsOf([sibling]);
const none = await groupRefusal(forge, self, siblings);
assert.match(none!, /mesh\/delivery-group is missing at mesh-controller#346@aaaaaaaaaaaa; mesh\/delivery-group is missing at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review at that head/);
statuses[`mesh-controller@${MEMBER_HEAD}`] = "success";
statuses[`mesh-host@${HOST_HEAD}`] = "pending";
assert.match((await groupRefusal(forge, self, siblings))!, /^mesh\/delivery-group is pending at mesh-host#114@bbbbbbbbbbbb; mesh-host#114@bbbbbbbbbbbb carries no approving review/);
statuses[`mesh-host@${HOST_HEAD}`] = "success";
assert.match((await groupRefusal(forge, self, siblings))!, /^mesh-host#114@bbbbbbbbbbbb carries no approving review at that head$/);
approved.add(`mesh-host#114@${HOST_HEAD}`);
assert.equal(await groupRefusal(forge, self, siblings), undefined);
// A read that fails is named, never read as a pass.
const broken = { ...forge, approvedAt: async () => { throw new Error("the forge is away"); } };
assert.match((await groupRefusal(broken, self, siblings))!, /could not be read \(the forge is away\)/);
assert.equal(besideWords(siblings), "beside mesh-host#114@bbbbbbbb");
});
test("a merged or closed member's entry is dropped, so a sibling's later push never re-pends its merged head", async () => {
// mesh-host#114 judged beside mesh-controller#346; #114 then merged; a push to #346 follows.
const w = world();
w.host.merged = true; // the member (mesh-host#114) is merged
w.member.head_sha = h("9"); // the sibling (mesh-controller#346) has a new head
const record: SiblingRecord = { "novox/mesh-host#114": { head: HOST_HEAD, siblings: siblingsOf([
{ repository: "novox/mesh-controller", number: 346, head: MEMBER_HEAD, base: "main", base_commit: h("2") }]) } };
assert.equal(await weigh(w, record, "novox/mesh-controller", 346), true);
assert.deepEqual(w.written, [], "the merged head is not put back to pending");
assert.deepEqual(record, {}, "and its entry is gone");
// On its own close or merge the member's entry goes too, whatever the siblings do.
const own = world();
own.host.state = "closed";
const kept = recordOf();
const mine: SiblingRecord = { "novox/mesh-host#114": { head: HOST_HEAD, siblings: kept["novox/mesh-controller#346"].siblings } };
assert.equal(await weigh(own, mine, "novox/mesh-host", 114), true);
assert.deepEqual(mine, {});
assert.deepEqual(own.written, []);
});
+116 -6
View File
@@ -10,7 +10,7 @@
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
@@ -29,6 +29,12 @@ interface Forge {
mints: number;
lastScopes: string[] | null;
tokens: Map<string, string>;
/** Tokens beside `tokens` that carry a name already held there: what two mints at once leave behind. */
twins: { id: number; name: string; sha1: string }[];
/** How each DELETE named its token: "id" or "name". */
deletedBy: string[];
/** Plant a token by name, beside any other of the same name, as a raced mint would. */
plant(name: string): void;
scopesOf: Map<string, string[]>;
admins: Map<string, string>;
pullState: string;
@@ -39,9 +45,14 @@ interface Forge {
function fakeForge(): Promise<Forge> {
const forge = {
mintedFor: "",
mints: 0,
lastScopes: null as string[] | null,
tokens: new Map<string, string>(), // name -> value
twins: [] as { id: number; name: string; sha1: string }[],
deletedBy: [] as string[],
ids: new Map<string, number>(), // value -> id, as gitea numbers its tokens
nextId: 0,
scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does
admins: new Map([[ADMIN, PASSWORD]]),
pullState: "open",
@@ -73,7 +84,14 @@ function fakeForge(): Promise<Forge> {
const tokens = url.pathname.match(/^\/api\/v1\/users\/([^/]+)\/tokens(?:\/([^/]+))?$/);
if (tokens) {
const user = basic(req);
if (user === null || user !== decodeURIComponent(tokens[1])) return json(res, 401, { message: "auth required" });
// gitea's reqSelfOrAdmin: the account itself, or the administrator for another (the tools' account, ADR 0365).
if (user === null || (user !== decodeURIComponent(tokens[1]) && user !== ADMIN)) return json(res, 401, { message: "auth required" });
// This fake keeps one account's tokens: the tools' account's when the module mints for it (ADR 0365), and the
// admin's own list is then empty — nothing of it is retired.
if (decodeURIComponent(tokens[1]) === ADMIN && forge.mintedFor === "mesh-tools") {
return req.method === "GET" ? json(res, 200, []) : json(res, 404, { message: "token not found" });
}
if (req.method === "POST") forge.mintedFor = decodeURIComponent(tokens[1]);
if (req.method === "POST") {
const { name, scopes } = await body(req);
if (forge.tokens.has(name)) return json(res, 400, { message: "token name has already been used" });
@@ -82,12 +100,35 @@ function fakeForge(): Promise<Forge> {
const sha1 = `minted-${forge.mints}-${Math.random().toString(36).slice(2)}`;
forge.tokens.set(name, sha1);
forge.scopesOf.set(sha1, scopes);
return json(res, 201, { id: forge.mints, name, sha1, scopes, token_last_eight: sha1.slice(-8) });
forge.ids.set(sha1, ++forge.nextId);
return json(res, 201, { id: forge.ids.get(sha1), name, sha1, scopes, token_last_eight: sha1.slice(-8) });
}
// gitea's list: every token of the account, a page at a time (at most three to a page here, so a
// client that reads only the first page misses some), the values never shown.
const listed = (): { id: number; name: string; sha1: string }[] => [
...[...forge.tokens].map(([name, sha1]) => ({ id: forge.ids.get(sha1) ?? 0, name, sha1 })),
...forge.twins,
];
if (req.method === "GET" && !tokens[2]) {
const page = Math.max(1, Number(url.searchParams.get("page") ?? "1"));
const limit = Math.min(3, Math.max(1, Number(url.searchParams.get("limit") ?? "3")));
const all = listed();
res.setHeader("X-Total-Count", String(all.length));
return json(res, 200, all.slice((page - 1) * limit, page * limit)
.map((t) => ({ id: t.id, name: t.name, sha1: "", token_last_eight: t.sha1.slice(-8), scopes: [] })));
}
if (req.method === "DELETE" && tokens[2]) {
const name = decodeURIComponent(tokens[2]);
if (!forge.tokens.has(name)) return json(res, 404, { message: "token not found" });
forge.tokens.delete(name);
// gitea: "identified by ID and if not available by name" — and a name held twice is refused.
const key = decodeURIComponent(tokens[2]);
const byId = /^\d+$/.test(key);
const matches = listed().filter((t) => (byId ? String(t.id) === key : t.name === key));
if (matches.length === 0) return json(res, 404, { message: "token not found" });
if (matches.length > 1) return json(res, 422, { message: `multiple matches for token name '${key}'` });
const t = matches[0];
forge.deletedBy.push(byId ? "id" : "name");
const twin = forge.twins.findIndex((w) => w.id === t.id);
if (twin >= 0) forge.twins.splice(twin, 1);
else forge.tokens.delete(t.name);
return json(res, 204, null);
}
return json(res, 405, { message: "method not allowed" });
@@ -153,6 +194,17 @@ function fakeForge(): Promise<Forge> {
{ full_name: "novox/hq", name: "hq", owner: { login: "novox" }, private: true, html_url: "http://fake/novox/hq" },
]);
}
// What setting up the tools' account asks with the admin login (tools-account.ts, ADR 0365): answered plainly.
if (basic(req) === ADMIN) {
const p = url.pathname;
if (req.method === "GET" && p === "/api/v1/user") return json(res, 200, { login: ADMIN, is_admin: true });
if (req.method === "GET" && p === "/api/v1/users/mesh-tools") return json(res, 200, { login: "mesh-tools", is_admin: false });
if (req.method === "POST" && p === "/api/v1/admin/users") return json(res, 422, { message: "user already exists" });
if (req.method === "PATCH" && p === "/api/v1/admin/users/mesh-tools") return json(res, 200, { login: "mesh-tools" });
if (req.method === "GET" && p === "/api/v1/orgs/novox/teams") return json(res, 200, [{ id: 7, name: "mesh-tools" }]);
if (req.method === "PATCH" && p === "/api/v1/teams/7") return json(res, 200, { id: 7 });
if (req.method === "PUT" && p === "/api/v1/teams/7/members/mesh-tools") return json(res, 204, null);
}
const adminUser = url.pathname.match(/^\/api\/v1\/admin\/users\/([^/]+)$/);
if (adminUser && req.method === "PATCH") {
const h = req.headers.authorization ?? "";
@@ -181,6 +233,14 @@ function fakeForge(): Promise<Forge> {
get pullTitle() { return forge.pullTitle; },
get branchDeleted() { return forge.branchDeleted; },
tokens: forge.tokens,
twins: forge.twins,
deletedBy: forge.deletedBy,
plant: (name: string) => {
const sha1 = `planted-${++forge.nextId}`;
forge.ids.set(sha1, forge.nextId);
if (forge.tokens.has(name)) forge.twins.push({ id: forge.nextId, name, sha1 });
else forge.tokens.set(name, sha1);
},
scopesOf: forge.scopesOf,
admins: forge.admins,
close: () => new Promise((r) => server.close(() => r())),
@@ -335,6 +395,56 @@ test("concurrent first calls share one mint", async () => {
assert.equal(forge.mints, before + 1);
});
test("two tokens hold the name (two mints raced): both are deleted by their id, and one fresh token is minted", async () => {
const { env, file, logs } = await delivered(forge);
forge.tokens.clear();
forge.twins.length = 0;
for (const other of ["laptop", "backup", "ci"]) forge.plant(other); // the account's other tokens: kept
forge.plant("mesh-tools");
forge.plant("mesh-tools"); // gitea refuses to delete either by name from here on
forge.deletedBy.length = 0;
const before = forge.mints;
const repos = await minted(env, logs).listRepos();
assert.equal(repos.length, 1);
assert.equal(forge.mints, before + 1);
assert.deepEqual(forge.deletedBy, ["id", "id"]);
assert.equal(forge.twins.length, 0);
assert.deepEqual([...forge.tokens.keys()].sort(), ["backup", "ci", "laptop", "mesh-tools"]);
assert.equal(await readFile(file, "utf8"), forge.tokens.get("mesh-tools") + "\n");
assert.ok(logs.some((l) => l.includes('holds 2 tokens named "mesh-tools"')), logs.join("\n"));
});
test("two processes on one kept file, both without a token, mint once: the second waits for the first and reuses it", async () => {
const { env, logs } = await delivered(forge);
forge.tokens.clear();
forge.twins.length = 0;
const before = forge.mints;
// Two sources over one state directory, as the tools bundle and the watcher are two processes.
const first = minted(env, logs);
const second = minted(env, logs);
await Promise.all([first.listRepos(), second.listRepos()]);
assert.equal(forge.mints, before + 1);
assert.equal(forge.twins.length, 0);
assert.ok(logs.some((l) => l.includes("another process minted it")), logs.join("\n"));
});
test("a lock left by a process that died mid-mint is taken over, not waited on for ever", async () => {
const { env, file, logs } = await delivered(forge);
forge.tokens.clear();
forge.twins.length = 0;
await mkdir(join(env.MESH_GITEA_STATE_DIR!), { recursive: true });
await writeFile(`${file}.lock`, `2147483646 ${Date.now()} gone\n`); // a pid no process holds
const before = forge.mints;
assert.equal((await minted(env, logs).listRepos()).length, 1);
assert.equal(forge.mints, before + 1);
await assert.rejects(stat(`${file}.lock`), /ENOENT/);
});
test("the admin account is refused: said plainly, nothing kept, and the next call fails the same way rather than crashing", async () => {
const { env, file, logs } = await delivered(forge);
forge.admins.delete(ADMIN); // the forge's data came from a predecessor; mesh-admin was never created there
+332
View File
@@ -0,0 +1,332 @@
// The forge's tools act as an account of their own, not the site administrator (novox/hq ADR 0365 §2, issue 469):
// the token the tools and the watcher use is minted for `mesh-tools`, a plain account made by the module with the
// administrator's login and given write on the mesh's repositories through one team; the administrator's own
// token by that name goes; a token kept from before, the administrator's, is minted again for the account.
//
// The forge is a fake that keeps accounts, their tokens and the organisation's teams apart, and answers as gitea
// does: an administrator may mint for another account (reqSelfOrAdmin), anyone else only for itself; a protection
// edit needs the repository's administration, which neither a plain account nor its write team holds.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import { mkdtemp, readFile, writeFile, mkdir } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { MintedToken } from "../dist/token.js";
import { GiteaClient } from "../dist/client.js";
import { TOOLS_ACCOUNT, TOOLS_TEAM_UNITS } from "../dist/tools-account.js";
import { repoCheckFacts } from "../dist/pulls.js";
const ADMIN = "mesh-admin";
const PASSWORD = "the-vault-minted-this";
interface Account { admin: boolean; password: string; active: boolean }
interface Token { id: number; name: string; sha1: string; user: string; scopes: string[] }
interface Team { id: number; name: string; units: any; members: Set<string> }
function fakeForge() {
const users = new Map<string, Account>([[ADMIN, { admin: true, password: PASSWORD, active: true }]]);
const tokens: Token[] = [];
const teams: Team[] = [];
const calls: string[] = [];
// What the forge does wrong when a test says so: a token mint or delete it fails, a team another process made
// between this one's look and its create (gitea answers the create 422).
const faults = { mint: false, deleteToken: false, teamRace: false };
let next = 0;
const json = (res: ServerResponse, status: number, body: unknown): void => {
res.writeHead(status, { "Content-Type": "application/json" });
res.end(body === null ? "" : JSON.stringify(body));
};
const read = (req: IncomingMessage): Promise<any> =>
new Promise((resolve) => {
let t = "";
req.on("data", (c) => (t += c));
req.on("end", () => resolve(t ? JSON.parse(t) : null));
});
const who = (req: IncomingMessage): string | null => {
const h = req.headers.authorization ?? "";
if (h.startsWith("Basic ")) {
const [u, p] = Buffer.from(h.slice(6), "base64").toString().split(":");
return users.get(u)?.password === p ? u : null;
}
if (h.startsWith("token ")) return tokens.find((t) => t.sha1 === h.slice(6))?.user ?? null;
return null;
};
const isAdmin = (u: string | null) => u !== null && users.get(u)?.admin === true;
const server = createServer(async (req, res) => {
const url = new URL(req.url ?? "/", "http://fake");
const p = url.pathname.replace(/^\/api\/v1/, "");
const me = who(req);
calls.push(`${req.method} ${p} as ${me}`);
if (me === null) return json(res, 401, { message: "unauthorized" });
let m = p.match(/^\/users\/([^/]+)\/tokens(?:\/(\d+))?$/);
if (m) {
const owner = decodeURIComponent(m[1]);
if (!(req.headers.authorization ?? "").startsWith("Basic ")) return json(res, 401, { message: "basic auth required" });
if (owner !== me && !isAdmin(me)) return json(res, 403, { message: "reqSelfOrAdmin" });
if (!users.has(owner)) return json(res, 404, { message: "user does not exist" });
if (req.method === "POST") {
if (faults.mint) return json(res, 500, { message: "the forge is not well" });
const { name, scopes } = await read(req);
if (tokens.some((t) => t.user === owner && t.name === name)) return json(res, 400, { message: "token name has already been used" });
const t = { id: ++next, name, sha1: `sha-${owner}-${next}`, user: owner, scopes };
tokens.push(t);
return json(res, 201, { id: t.id, name, sha1: t.sha1, scopes });
}
if (req.method === "GET") {
const page = Number(url.searchParams.get("page") ?? "1");
const mine = tokens.filter((t) => t.user === owner);
return json(res, 200, page === 1 ? mine.map((t) => ({ id: t.id, name: t.name, sha1: "" })) : []);
}
if (req.method === "DELETE" && m[2]) {
if (faults.deleteToken && owner === ADMIN) return json(res, 500, { message: "the forge is not well" });
const i = tokens.findIndex((t) => t.user === owner && t.id === Number(m![2]));
if (i < 0) return json(res, 404, { message: "not found" });
tokens.splice(i, 1);
return json(res, 204, null);
}
}
if (req.method === "GET" && p === "/user") return json(res, 200, { login: me, is_admin: isAdmin(me) });
m = p.match(/^\/users\/([^/]+)$/);
if (m && req.method === "GET") {
const u = users.get(decodeURIComponent(m[1]));
if (!u) return json(res, 404, { message: "user does not exist" });
return json(res, 200, { login: m[1], is_admin: u.admin, active: u.active, prohibit_login: false });
}
if (p.startsWith("/admin/") && !isAdmin(me)) return json(res, 403, { message: "must be a site administrator" });
if (req.method === "POST" && p === "/admin/users") {
const b = await read(req);
if (users.has(b.username)) return json(res, 422, { message: "user already exists" });
if (!b.password || b.password.length < 8) return json(res, 422, { message: "password is too short" });
users.set(b.username, { admin: b.admin === true, password: b.password, active: true });
return json(res, 201, { login: b.username, is_admin: b.admin === true });
}
m = p.match(/^\/admin\/users\/([^/]+)$/);
if (m && req.method === "PATCH") {
const u = users.get(decodeURIComponent(m[1]));
if (!u) return json(res, 404, { message: "user does not exist" });
const b = await read(req);
if (b.admin !== undefined) u.admin = b.admin;
if (b.active !== undefined) u.active = b.active;
return json(res, 200, { login: m[1], is_admin: u.admin });
}
if (p === "/orgs/novox/teams") {
if (!isAdmin(me)) return json(res, 403, { message: "must be an owner of the organisation" });
if (req.method === "GET") return json(res, 200, teams.map((t) => ({ id: t.id, name: t.name })));
const b = await read(req);
const t = { id: ++next, name: b.name, units: b, members: new Set<string>() };
teams.push(t);
if (faults.teamRace) return json(res, 422, { message: "team already exists" });
return json(res, 201, { id: t.id, name: t.name });
}
m = p.match(/^\/teams\/(\d+)(?:\/members\/([^/]+))?$/);
if (m) {
if (!isAdmin(me)) return json(res, 403, { message: "must be an owner of the organisation" });
const t = teams.find((x) => x.id === Number(m![1]));
if (!t) return json(res, 404, { message: "team does not exist" });
if (req.method === "PATCH") { t.units = await read(req); return json(res, 200, { id: t.id }); }
if (req.method === "PUT" && m[2]) { t.members.add(decodeURIComponent(m[2])); return json(res, 204, null); }
}
// Repository administration: a site administrator, or a team member whose team administers the repository.
m = p.match(/^\/repos\/novox\/([^/]+)\/branches\/(.+)$/);
if (m && req.method === "GET") {
if (m[2] === "main") return json(res, 200, { name: "main", protected: true, effective_branch_protection_name: "main",
enable_status_check: true, status_check_contexts: ["mesh/merge-gate", "mesh/repo-check"], user_can_push: false });
if (m[2] === "feat/unguarded") return json(res, 200, { name: m[2], protected: false });
return json(res, 404, { message: "branch does not exist" });
}
// gitea's reqAdmin on the whole group: reading the rules is administration too.
m = p.match(/^\/repos\/novox\/([^/]+)\/branch_protections(?:\/.*)?$/);
if (m) {
const admins = isAdmin(me) || teams.some((t) => t.members.has(me) && t.units.permission === "admin");
if (!admins) return json(res, 403, { message: "user should be an owner or a collaborator with admin write of a repository" });
return json(res, 201, { rule_name: "main" });
}
m = p.match(/^\/repos\/novox\/([^/]+)\/statuses\/([0-9a-f]+)$/);
if (m && req.method === "POST") {
const writes = isAdmin(me) || teams.some((t) => t.members.has(me) && t.units.units_map?.["repo.code"] === "write");
if (!writes) return json(res, 403, { message: "write on code required" });
return json(res, 201, { context: (await read(req)).context, creator: { login: me } });
}
if (req.method === "GET" && p === "/repos/search") return json(res, 200, { data: [] });
return json(res, 404, { message: "no such route in the fake" });
});
return new Promise<{ url: string; users: typeof users; tokens: typeof tokens; teams: typeof teams; calls: string[];
faults: typeof faults; close(): Promise<void> }>((resolve) => {
server.listen(0, "127.0.0.1", () => {
const { port } = server.address() as { port: number };
resolve({ url: `http://127.0.0.1:${port}`, users, tokens, teams, calls, faults,
close: () => new Promise((r) => server.close(() => r())) });
});
});
}
const forge = await fakeForge();
after(() => forge.close());
async function environment(): Promise<NodeJS.ProcessEnv & { MESH_GITEA_STATE_DIR: string }> {
const dir = await mkdtemp(join(tmpdir(), "gitea-tools-account-"));
const passwordFile = join(dir, "admin.secret");
await writeFile(passwordFile, PASSWORD + "\n", { mode: 0o600 });
return {
MESH_GITEA_URL: forge.url,
MESH_GITEA_ADMIN_USER: ADMIN,
MESH_GITEA_ADMIN_PASSWORD_FILE: passwordFile,
MESH_GITEA_STATE_DIR: join(dir, "state"),
};
}
/** A source as a fresh process builds it from the manifest's environment (fromEnv shares one per file). */
function fresh(env: NodeJS.ProcessEnv): GiteaClient {
return GiteaClient.fromEnv(env);
}
test("the tools' token is minted for an account of their own, which is no administrator and writes through its team", async () => {
const env = await environment();
await fresh(env).listRepos();
const account = forge.users.get(TOOLS_ACCOUNT);
assert.ok(account, "the module made the tools' account");
assert.equal(account!.admin, false, "the tools' account is no administrator");
const held = forge.tokens.filter((t) => t.name === "mesh-tools");
assert.deepEqual(held.map((t) => t.user), [TOOLS_ACCOUNT], "the one token is the tools' account's, none is the administrator's");
assert.equal(await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8"), held[0].sha1 + "\n");
const team = forge.teams.find((t) => t.name === "mesh-tools");
assert.ok(team, "the team is made in the mesh's organisation");
assert.ok(team!.members.has(TOOLS_ACCOUNT));
assert.equal(team!.units.permission, "write");
assert.equal(team!.units.includes_all_repositories, true);
assert.equal(team!.units.can_create_org_repo, false, "a repository's maker would administer it");
assert.deepEqual(team!.units.units_map, TOOLS_TEAM_UNITS.units_map);
assert.ok(!Object.values(team!.units.units_map).includes("admin"));
});
test("with the tools' token, the forge writes a status and refuses a protection edit — the forge itself, not a guard", async () => {
const env = await environment();
const client = fresh(env);
await client.setCommitStatus("novox", "hq", "abc123def4567890abc123def4567890abc12345", { state: "success", context: "mesh/merge-gate", description: "ok" } as any);
await assert.rejects(
client.setBranchProtection("novox", "hq", "main", { statusChecks: ["mesh/merge-gate"] }),
/403/,
);
});
test("what a base's protection asks is read from the branch, which the tools' account may read, not from the rules' list it may not", async () => {
const env = await environment();
const client = fresh(env);
await assert.rejects(client.branchProtections("novox", "hq"), /403/);
const rule = await client.appliedProtection("novox", "hq", "main");
assert.deepEqual(rule?.status_check_contexts, ["mesh/merge-gate", "mesh/repo-check"]);
assert.equal(rule?.enable_status_check, true);
assert.equal(rule?.rule_name, "main");
assert.equal(await client.appliedProtection("novox", "hq", "feat/unguarded"), null);
assert.equal(await client.appliedProtection("novox", "hq", "no/such-branch"), null);
});
test("the administrator's token from before goes: a kept admin token is minted again for the tools' account, and the forge drops the old one", async () => {
const env = await environment();
// What the module kept before ADR 0365: a token of the administrator's own, by the same name, its scopes beside it.
const old = { id: 900, name: "mesh-tools", sha1: "sha-admin-old", user: ADMIN, scopes: ["write:repository", "write:issue", "read:user"] };
forge.tokens.push(old);
await mkdir(env.MESH_GITEA_STATE_DIR, { recursive: true });
await writeFile(join(env.MESH_GITEA_STATE_DIR, "token"), old.sha1 + "\n", { mode: 0o600 });
await writeFile(join(env.MESH_GITEA_STATE_DIR, "token.scopes"), "read:user,write:issue,write:repository\n", { mode: 0o600 });
await fresh(env).listRepos();
assert.ok(!forge.tokens.some((t) => t.user === ADMIN), "the administrator holds no tools' token any more");
const kept = (await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8")).trim();
assert.notEqual(kept, old.sha1);
assert.equal(forge.tokens.find((t) => t.sha1 === kept)?.user, TOOLS_ACCOUNT);
assert.match(await readFile(join(env.MESH_GITEA_STATE_DIR, "token.scopes"), "utf8"), / for mesh-tools\n$/);
});
test("a tools' account someone made an administrator is made a plain account again at the next mint", async () => {
const env = await environment();
forge.users.get(TOOLS_ACCOUNT)!.admin = true;
await fresh(env).listRepos();
assert.equal(forge.users.get(TOOLS_ACCOUNT)!.admin, false);
});
test("the tools never act as the administrator: MESH_GITEA_TOOLS_USER naming it is refused", async () => {
const env = await environment();
assert.throws(() => MintedToken.fromEnv(forge.url, { ...env, MESH_GITEA_TOOLS_USER: ADMIN }), /names the admin account/);
});
test("the manifest hands the tools their account, and the merge-ready authors name it rather than the administrator", async () => {
const manifest = JSON.parse(await readFile(new URL("../module.json", import.meta.url), "utf8"));
const code = manifest.build.artifacts.find((a: any) => a.name === "code");
assert.equal(code.env.MESH_GITEA_TOOLS_USER, TOOLS_ACCOUNT);
const authors = String(manifest.config["ready-authors"].default).split(/\s+/);
assert.ok(authors.includes(TOOLS_ACCOUNT));
// Kept for now (review of issue 469): a pull request the admin account marked ready before the tools acted as
// their own keeps counting as ready after the delivery.
assert.ok(authors.includes(ADMIN), "marks made by the admin account before ADR 0365 keep counting");
// The Go forge bundle reads the token the tools' account holds, never the administrator's password.
const forgeBundle = manifest.build.artifacts.find((a: any) => a.name === "forge");
assert.ok(forgeBundle.env.MESH_GITEA_TOKEN_FILE);
assert.equal(forgeBundle.env.MESH_GITEA_ADMIN_PASSWORD_FILE, undefined);
});
// Review of issue 469: what reads the rules' list is moved to the branch, and nothing leaves the tools without a token.
test("a repository check said as a warning on a head without merge-check.sh reads the base's protection from the branch", async () => {
const client = fresh(await environment());
const forgeOf = { holdsFile: async () => false, appliedProtection: client.appliedProtection.bind(client) };
const c = { owner: "novox", repo: "hq", commit: "abc123def4567890abc123def4567890abc12345", "repo-check": { verdict: "warning" } } as any;
// main requires mesh/repo-check: required, so a head that defines no check fails it for a person to see.
assert.deepEqual(await repoCheckFacts(forgeOf, c, "main"), { defined: false, required: true });
// A base nothing guards: not required, so the head's repository check passes rather than fails.
assert.deepEqual(await repoCheckFacts(forgeOf, c, "feat/unguarded"), { defined: false, required: false });
});
test("a mint that fails leaves the administrator's old token in place: the tools are never left without one", async () => {
const env = await environment();
const old = { id: 901, name: "mesh-tools", sha1: "sha-admin-kept", user: ADMIN, scopes: ["read:user"] };
forge.tokens.push(old);
forge.faults.mint = true;
try {
await assert.rejects(fresh(env).listRepos(), /500/);
} finally {
forge.faults.mint = false;
}
assert.ok(forge.tokens.some((t) => t.sha1 === old.sha1), "the admin's token was deleted before a new one was kept");
forge.tokens.splice(forge.tokens.indexOf(old), 1);
});
test("an old token that cannot be deleted after the mint is said, the new token kept, and deleted at the next start", async () => {
const env = await environment();
const old = { id: 902, name: "mesh-tools", sha1: "sha-admin-stuck", user: ADMIN, scopes: ["read:user"] };
forge.tokens.push(old);
forge.faults.deleteToken = true;
try {
await fresh(env).listRepos();
} finally {
forge.faults.deleteToken = false;
}
const kept = (await readFile(join(env.MESH_GITEA_STATE_DIR, "token"), "utf8")).trim();
assert.equal(forge.tokens.find((t) => t.sha1 === kept)?.user, TOOLS_ACCOUNT, "the account's token is kept and works");
assert.ok(forge.tokens.some((t) => t.sha1 === old.sha1));
// The next process reuses the kept token and retires the old one it could not before.
const next = new MintedToken({ url: forge.url, admin: ADMIN, passwordFile: env.MESH_GITEA_ADMIN_PASSWORD_FILE!,
file: join(env.MESH_GITEA_STATE_DIR, "token"), account: TOOLS_ACCOUNT, log: () => {} });
assert.equal(await next.current(), kept);
assert.ok(!forge.tokens.some((t) => t.sha1 === old.sha1), "the admin's old token was deleted at the next start");
});
test("a team another process made between the look and the create is found, not a failure", async () => {
forge.teams.splice(0);
forge.faults.teamRace = true;
try {
await fresh(await environment()).listRepos();
} finally {
forge.faults.teamRace = false;
}
const team = forge.teams.find((t) => t.name === "mesh-tools");
assert.ok(team?.members.has(TOOLS_ACCOUNT));
});
+26
View File
@@ -173,3 +173,29 @@ test("a verdict that landed out of date keeps its account on the pull request",
const bare = lateVerdictComment({ commit: HEAD, trunk: TRUNK.slice(0, 12), verdict: "pass" }, null, words);
assert.match(bare, /head dfb1415aaaaa merged into trunk a2c2c35bbbbb, pass/);
});
test("a release branch's move puts every open pull request into it out of date, as a trunk's does (ADR 0358 §5)", async () => {
// novox/hq ADR 0358 §5: for a pull request into a release branch, the release branch takes the trunk's place in ADR
// 0323 — the check merges the head onto the release branch's head and names it as the trunk commit, so the release
// branch's move puts its open verdicts out of date, and leaves those into main alone.
const release = "feat/57-config-and-provisioning";
const heads: Record<string, string> = { main: TRUNK, [release]: "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee" };
const pulls = [{ number: 315, base: release, head: "3".repeat(40) }, { number: 1, base: "main", head: "1".repeat(40) }];
const { forge, statuses } = aForge(pulls, heads);
const gates = new GateWriter(forge);
const trunks: Record<string, string> = {};
await weighTrunks(forge, gates, trunks, "novox", "mesh-controller");
heads[release] = MOVED;
assert.equal(await weighTrunks(forge, gates, trunks, "novox", "mesh-controller"), true);
const s = statuses.get("3".repeat(40))!;
assert.equal(s["mesh/merge-gate"].state, "pending");
assert.match(s["mesh/merge-gate"].description!, /judged against trunk eeeeeeeeeeee, the trunk is now d28ca95ccccc/);
assert.equal(s["mesh/repo-check"].state, "pending");
assert.equal(statuses.get("1".repeat(40))!["mesh/merge-gate"].state, "success", "a pull request into main is left");
assert.equal(trunks[`novox/mesh-controller:${release}`], MOVED);
// The merge tool's rule reads the same: a verdict judged against the release branch's old head is not current.
assert.equal(verdictAtTrunk({ "mesh/merge-gate": { state: "success",
description: "head x merged into trunk eeeeeeeeeeee — pass" } }, MOVED).current, false);
});
+335
View File
@@ -0,0 +1,335 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { createHmac } from "node:crypto";
import type { AddressInfo } from "node:net";
// **The forge says every event on the bus** (novox/hq issue 729): a signed delivery is published once and acted on,
// an unsigned or unverified one is refused, and an event and the poll seeing the same head announce it once.
const secret = "s".repeat(64);
const sign = (body: string, key = secret) => createHmac("sha256", key).update(body).digest("hex");
const opened = JSON.stringify({ action: "opened", number: 7, pull_request: { number: 7, title: "t", user: { login: "jo" },
head: { ref: "feat/x", sha: "a".repeat(40) }, base: { ref: "main" }, merged: false },
repository: { name: "hq", full_name: "novox/hq", owner: { login: "novox" } }, sender: { login: "jo" } });
async function serve(published: { name: string }[], acted: { name: string }[]) {
const { receiver, HOOK_PATH } = await import("../webhook.ts");
const server = receiver({ secret, port: 0, log: () => {},
publish: async (e) => { published.push(e); }, act: async (e) => { acted.push(e); } });
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}${HOOK_PATH}`;
return { server, url };
}
test("a signed event is published on the bus and acted on once", async () => {
const published: { name: string }[] = [], acted: { name: string }[] = [];
const { server, url } = await serve(published, acted);
try {
const headers = { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened), "X-Gitea-Delivery": "d-1" };
const res = await fetch(url, { method: "POST", body: opened, headers });
assert.equal(res.status, 200);
// The same delivery sent again is answered and published nowhere.
const again = await fetch(url, { method: "POST", body: opened, headers });
assert.equal(again.status, 200);
await server.idle();
assert.deepEqual(published.map((e) => e.name), ["forge.pull-request.opened"]);
assert.deepEqual(acted.map((e) => e.name), ["forge.pull-request.opened"]);
} finally { server.close(); }
});
test("an unsigned event, or one whose signature does not verify, is refused and published nowhere", async () => {
const published: { name: string }[] = [], acted: { name: string }[] = [];
const { server, url } = await serve(published, acted);
try {
const unsigned = await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request" } });
assert.equal(unsigned.status, 401);
assert.match(await unsigned.text(), /unsigned/);
const forged = await fetch(url, { method: "POST", body: opened,
headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened, "another secret") } });
assert.equal(forged.status, 401);
assert.match(await forged.text(), /does not verify/);
const big = "x".repeat(5 * 1024 * 1024 + 1);
const large = await fetch(url, { method: "POST", body: big,
headers: { "X-Gitea-Event": "push", "X-Gitea-Signature": sign(big) } });
assert.equal(large.status, 413);
await server.idle();
assert.deepEqual(published, []);
assert.deepEqual(acted, []);
} finally { server.close(); }
});
test("an event and the poll seeing the same head announce it once", async () => {
const { claimHead } = await import("../webhook.ts");
const heads: Record<string, string> = {};
const sha = "a".repeat(40);
assert.equal(claimHead(heads, "novox/hq#7", sha), true, "the event claims the head and announces it");
assert.equal(claimHead(heads, "novox/hq#7", sha), false, "the poll finds it taken");
assert.equal(claimHead(heads, "novox/hq#7", "b".repeat(40)), true, "a new head is announced again");
});
test("each forge event is named in the module's stream and trimmed", async () => {
const { eventOf } = await import("../webhook.ts");
const merged = eventOf("pull_request", { action: "closed", pull_request: { number: 3, merged: true, merge_commit_sha: "c".repeat(40),
head: { sha: "a".repeat(40) }, base: { ref: "main" } }, repository: { name: "hq", owner: { login: "novox" } } });
assert.equal(merged.name, "forge.pull-request.merged");
assert.equal(merged.body.merge_commit_sha, "c".repeat(40));
assert.equal(eventOf("pull_request", { action: "label_updated", pull_request: { number: 3 }, label: { name: "mesh/ready" } }).name,
"forge.pull-request.labeled");
const comment = eventOf("issue_comment", { action: "created", is_pull: true, issue: { number: 3 },
comment: { id: 9, body: "ready at abc", user: { login: "jo" } } });
assert.equal(comment.name, "forge.pull-request.commented");
assert.equal(comment.body.comment, "ready at abc");
assert.equal(eventOf("push", { ref: "refs/heads/main", after: "d".repeat(40) }).name, "forge.push");
assert.equal(eventOf("pull_request_review_approved", {}).name, "forge.other");
});
// A forge's admin hooks as 1.27.3 keeps them: POST /admin/hooks makes a system webhook only when config.is_system_webhook
// is "true", and a default webhook otherwise; the listing names one kind at a time (type=system, the default, or default).
// Repositories hold their own webhooks (a default webhook's copies), listed a page at a time.
function fakeForge(start: { id: number; url: string; system: boolean }[] = [],
repoHooks: Record<string, { id: number; url: string }[]> = {}, repoCount = 0) {
const hooks = start.map((h) => ({ ...h, events: ["push"], active: true }));
const repos = [...Object.keys(repoHooks)];
for (let i = 0; repos.length < repoCount; i++) repos.push(`novox/r${i}`);
let next = 100;
const calls: string[] = [];
const made: any[] = [];
const page = (items: any[], q: URLSearchParams) => {
const p = Number(q.get("page") ?? 1), l = Number(q.get("limit") ?? 50);
return items.slice((p - 1) * l, p * l);
};
const admin = { call: async (method: string, path: string, body?: any) => {
calls.push(`${method} ${path}`);
const u = new URL(path, "http://f");
const repoHook = u.pathname.match(/^\/repos\/([^/]+\/[^/]+)\/hooks(?:\/(\d+))?$/);
if (method === "GET" && u.pathname === "/repos/search") {
return { status: 200, body: { ok: true, data: page(repos.map((full_name) => ({ full_name })), u.searchParams) } };
}
if (repoHook) {
const list = repoHooks[repoHook[1]] ?? [];
if (method === "GET") return { status: 200, body: page(list.map((h) => ({ id: h.id, config: { url: h.url } })), u.searchParams) };
const i = list.findIndex((h) => h.id === Number(repoHook[2]));
if (i < 0) return { status: 404, body: {} };
list.splice(i, 1);
return { status: 204, body: null };
}
if (method === "GET") {
const type = u.searchParams.get("type") ?? "system";
const listed = hooks.filter((h) => type === "all" || h.system === (type === "system"));
return { status: 200, body: listed.map((h) => ({ id: h.id, active: h.active, events: h.events,
config: { url: h.url, content_type: "json" } })) };
}
if (method === "POST") {
made.push(body);
hooks.push({ id: next++, url: body.config.url, system: body.config.is_system_webhook === "true", events: body.events,
active: body.active });
return { status: 201, body: {} };
}
if (method === "DELETE") {
const id = Number(path.split("/").pop());
const i = hooks.findIndex((h) => h.id === id);
if (i < 0) return { status: 404, body: {} };
hooks.splice(i, 1);
return { status: 204, body: null };
}
return { status: 405, body: {} };
} };
return { admin, hooks, calls, made, repoHooks };
}
test("the webhook is made a system webhook, which the forge sends every repository's events to", async () => {
const { ensureSystemHook } = await import("../webhook.ts");
const forge = fakeForge();
const url = "http://h:3047/forge-event";
const hook = await ensureSystemHook(forge.admin, url, secret);
assert.equal(forge.made.length, 1);
assert.equal(forge.made[0].config.is_system_webhook, "true", "without it the forge makes a default webhook, which fires for no repository");
assert.equal(forge.made[0].config.secret, secret);
assert.ok(forge.made[0].events.includes("pull_request") && forge.made[0].events.includes("push"));
assert.deepEqual(forge.hooks.map((h) => h.system), [true]);
assert.equal(hook.kind, "system");
assert.equal(hook.did, "created");
assert.equal(hook.url, url);
assert.ok(hook.events.includes("pull_request"));
});
test("default webhooks left at the receiver by an earlier start are replaced by one system webhook", async () => {
const { ensureSystemHook } = await import("../webhook.ts");
const url = "http://h:3047/forge-event";
// What the start at 23:05 left: a default webhook, invisible to the system listing, made again by every start.
const forge = fakeForge([{ id: 4, url, system: false }, { id: 5, url, system: false }, { id: 6, url: "http://other/x", system: true }]);
const hook = await ensureSystemHook(forge.admin, url, secret);
assert.deepEqual(hook.removed, { system: 0, default: 2, repository: 0, repositories: [] });
assert.equal(hook.did, "replaced");
assert.deepEqual(forge.hooks.map((h) => [h.url, h.system]), [["http://other/x", true], [url, true]], "another url is kept");
// A second start replaces the system webhook, so the secret kept now is the one the forge signs with (its edit never
// changes a secret).
const again = await ensureSystemHook(forge.admin, url, secret);
assert.deepEqual(again.removed, { system: 1, default: 0, repository: 0, repositories: [] });
assert.equal(forge.hooks.filter((h) => h.url === url).length, 1);
assert.ok(forge.calls.includes("DELETE /admin/hooks/4") && forge.calls.includes("DELETE /admin/hooks/5"));
// The new one is made before any old one is deleted, so no event falls in a gap.
const post = forge.calls.indexOf("POST /admin/hooks");
assert.ok(post >= 0 && forge.calls.findIndex((c) => c.startsWith("DELETE")) > post);
});
test("a repository's copy of a default webhook at the receiver is deleted, on every page, and said", async () => {
const { ensureSystemHook } = await import("../webhook.ts");
const url = "http://h:3047/forge-event";
// A repository made after the default webhook got its copy, which would send its events a second time; 120
// repositories, so the copies sit beyond the first page.
const repoHooks: Record<string, { id: number; url: string }[]> = {};
for (let i = 0; i < 118; i++) repoHooks[`novox/r${i}`] = [];
repoHooks["novox/late"] = [{ id: 31, url }, { id: 32, url: "http://ci/hook" }];
repoHooks["novox/later"] = [{ id: 33, url }];
const forge = fakeForge([{ id: 4, url, system: false }], repoHooks);
const hook = await ensureSystemHook(forge.admin, url, secret);
assert.deepEqual(hook.removed, { system: 0, default: 1, repository: 2, repositories: ["novox/late", "novox/later"] });
assert.deepEqual(forge.repoHooks["novox/late"].map((h) => h.id), [32], "a repository's other webhook is kept");
assert.deepEqual(forge.repoHooks["novox/later"], []);
assert.ok(forge.calls.some((c) => /\/repos\/search\?.*page=3/.test(c)), "every page of repositories is read");
});
test("a forge that makes no system webhook is said, never taken as heard", async () => {
const { ensureSystemHook } = await import("../webhook.ts");
const forge = fakeForge();
const admin = { call: async (method: string, path: string, body?: any) => {
// A forge that ignores the flag: what 23:05 met, had the module read back.
if (method === "POST") body = { ...body, config: { ...body.config, is_system_webhook: undefined } };
return forge.admin.call(method, path, body);
} };
await assert.rejects(ensureSystemHook(admin, "http://h:3047/forge-event", secret), /names 0 new system webhooks/);
});
test("the receiver tells what it verified and what it refused", async () => {
const { receiver, HOOK_PATH } = await import("../webhook.ts");
const heard: string[] = [], refused: string[] = [];
const server = receiver({ secret, port: 0, log: () => {}, publish: async () => {},
heard: (kind, delivery) => heard.push(`${kind} ${delivery}`), refused: (reason) => refused.push(reason) });
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}${HOOK_PATH}`;
try {
await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened), "X-Gitea-Delivery": "d-9" } });
await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened, "x") } });
await server.idle();
assert.deepEqual(heard, ["pull_request d-9"]);
assert.deepEqual(refused, ["its signature does not verify"]);
} finally { server.close(); }
});
test("a forge that never delivers is said loudly once, and unhealthy, when the poll finds what no event said", async () => {
const { Pace } = await import("../webhook.ts");
let now = 1_000_000;
const said: string[] = [];
const pace = new Pace(() => now, (l) => said.push(l));
const hook = { kind: "system" as const, id: 7, url: "http://h:3047/forge-event", events: ["push"], active: true,
did: "created" as const, removed: { system: 0, default: 0, repository: 0, repositories: [] } };
pace.listening("172.17.0.1", 3047, hook.url, hook);
now += 5 * 60_000;
pace.miss("novox/hq#7 at aaaaaaaa");
assert.equal(said.length, 0, "five minutes after the start is not yet silence");
assert.equal((pace.report() as any).healthy, true);
pace.refused("it is unsigned", "172.17.0.3");
assert.equal((pace.report() as any).healthy, false, "refused, and none verified");
now += 6 * 60_000;
pace.miss("the merge of novox/hq#7");
pace.miss("novox/hq#8 at bbbbbbbb");
assert.equal(said.filter((l) => /NOT HEARD: no delivery ever verified/.test(l)).length, 1, "said once");
let r = pace.report() as any;
assert.equal(r.healthy, false);
assert.equal(r.state, "unhealthy");
assert.match(r.why, /never delivered/);
assert.equal(r.polls.found_unsaid, 3);
assert.equal(r.refused.count, 1);
assert.equal(r.refused.last.reason, "it is unsigned");
assert.equal(r.hook.kind, "system");
assert.equal(r.receiver.port, 3047);
// The first delivery clears it.
now += 1000;
pace.heard("push", "d-1");
r = pace.report() as any;
assert.equal(r.healthy, true);
assert.equal(r.verified.count, 1);
assert.equal(r.verified.last.event, "push");
assert.equal(r.polls.every_seconds, 300);
assert.match(said.join("\n"), /heard again/);
});
test("a forge whose events stop after one delivery is said loudly once per silent spell, and refusals are unhealthy", async () => {
const { Pace } = await import("../webhook.ts");
let now = 1_000_000;
const said: string[] = [];
const pace = new Pace(() => now, (l) => said.push(l));
const hook = { kind: "system" as const, id: 7, url: "http://h:3047/forge-event", events: ["push"], active: true,
did: "created" as const, removed: { system: 0, default: 0, repository: 0, repositories: [] } };
pace.listening("172.17.0.1", 3047, hook.url, hook);
now += 60_000;
pace.heard("push", "d-1");
// The hook is deleted by hand: nothing more comes, and the poll finds what no event said.
now += 11 * 60_000;
pace.miss("novox/hq#7 at aaaaaaaa");
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 0, "one find alone may be the poll beating an event");
pace.miss("the merge of novox/hq#7");
pace.miss("novox/hq#8 at bbbbbbbb");
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 1, "said once");
assert.match(said.find((l) => /NOT HEARD/.test(l)) ?? "", /since the last at/);
assert.equal((pace.report() as any).healthy, false);
// A delivery ends the spell; the next spell is said again.
now += 1000;
pace.heard("push", "d-2");
assert.equal((pace.report() as any).healthy, true);
now += 11 * 60_000;
pace.miss("novox/hq#9 at cccccccc");
pace.miss("novox/hq#10 at dddddddd");
assert.equal(said.filter((l) => /NOT HEARD/.test(l)).length, 2, "a second silent spell is said again");
// Every delivery refused after a verified one (a secret that no longer matches) is unhealthy at once.
now += 1000;
pace.heard("push", "d-3");
now += 1000;
pace.refused("its signature does not verify", "172.17.0.3");
pace.refused("its signature does not verify", "172.17.0.3");
const r = pace.report() as any;
assert.equal(r.healthy, false);
assert.equal(r.refused.since_verified, 2);
assert.match(r.why, /2 deliveries refused since the last verified one.*does not verify/);
now += 1000;
pace.heard("push", "d-4");
assert.equal((pace.report() as any).healthy, true, "a verified delivery clears it");
});
test("without a receiver by configuration the polls alone watch, healthy; a receiver that failed is unhealthy", async () => {
const { Pace } = await import("../webhook.ts");
let now = 1_000_000;
const said: string[] = [];
const quiet = new Pace(() => now, (l) => said.push(l));
quiet.unheard("MESH_GITEA_HOOK_URL unset", false);
now += 20 * 60_000;
quiet.miss("novox/hq#7 at aaaaaaaa");
assert.equal(said.length, 0, "no deliveries were expected");
assert.equal((quiet.report() as any).healthy, true);
const failed = new Pace(() => now, () => {});
failed.unheard("the forge's system webhook could not be made: 403");
const r = failed.report() as any;
assert.equal(r.healthy, false);
assert.match(r.why, /403/);
});
test("the polls look every 30 s until events are heard, every five minutes while they are, and 30 s again on a miss", async () => {
const { Pace } = await import("../webhook.ts");
let now = 1_000_000;
const said: string[] = [];
const pace = new Pace(() => now, (l) => said.push(l));
assert.equal(pace.every(), 30_000, "no delivery yet");
pace.heard();
assert.equal(pace.every(), 300_000, "a delivery heard");
now += 11 * 60_000;
assert.equal(pace.every(), 30_000, "ten minutes without a delivery");
pace.heard();
now += 1000;
pace.miss("novox/hq#7 at aaaaaaaa");
assert.equal(pace.every(), 30_000, "the poll found what no event said");
assert.match(said.join("\n"), /MISSED novox\/hq#7/);
now += 1000;
pace.heard();
assert.equal(pace.every(), 300_000, "the next delivery");
});
+216 -30
View File
@@ -12,7 +12,20 @@
// hands a token's value out exactly once, so a token not kept is a token lost;
// - re-minted when the forge rejects it (401) or the kept file is gone. The one case that is not
// a fault: the forge's data was restored from a predecessor and the token the file names never
// existed there.
// existed there;
// - minted by one process at a time. The runtime starts each of the bundle's entrypoints as a process
// of its own (novox/hq ADR 0193): the watcher (index.js) and the tools (tools/index.js) are two
// processes over one kept file. Two that minted at once left the forge two tokens by one name, which
// gitea then refuses to delete by name — for ever, and every forge tool with it (the 2026-10-14
// outage). So a mint holds a lock file beside the kept token, and reads the kept file again once it
// holds it: what another process minted meanwhile is reused, not minted over;
// - replaced by id: an old token by this name is found in the account's list and deleted by its id,
// every one of them, so a name held twice is cleared rather than refused.
//
// - minted for the tools' account, not the administrator (novox/hq ADR 0365 §2, issue 469): the administrator's
// login mints it for `mesh-tools` (tools-account.ts), after making that account and its team, and deletes any
// token by this name the administrator itself still holds — the token the tools used before. A token kept
// for another account than the one asked (the scopes line says whose) is minted again at the next start.
//
// An explicitly configured token still wins, and is never minted over: if it is rejected, that is
// reported, not repaired — somebody chose it.
@@ -20,8 +33,10 @@
// The token is never logged. Lines say that one was minted, reused or renewed, and where it is
// kept; never what it is.
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { chmodSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, renameSync, rmSync, statSync, unlinkSync, writeFileSync, writeSync } from "node:fs";
import { randomBytes } from "node:crypto";
import { dirname, join } from "node:path";
import { setUpToolsAccount, TOOLS_ACCOUNT, type AdminCall } from "./tools-account.js";
/** The name the token carries in the forge's own list — one per mesh runtime, found by name. */
export const TOKEN_NAME = "mesh-tools";
@@ -39,7 +54,7 @@ export const TOKEN_NAME = "mesh-tools";
* `gitea_api` (novox/hq 229); whoever may call a tool includes agents, and the forge's administration is the
* operator's. The provisioner settles users with the admin account's own login (GiteaAdmin), never this token.
* A token kept with a different list is minted again with this one at the next start (the scopes are kept
* beside it), and the forge drops the old one by name — so the scope a kept token had does not outlive this.
* beside it), and the old one is deleted from the forge by its id — so the scope a kept token had does not outlive this.
*
* A token kept from before a scope was added lacks it: the forge answers such a call with
* `403 token does not have at least one of required scope(s)`, and the client treats that like a
@@ -95,6 +110,14 @@ export interface MintedTokenOptions {
readonly file: string;
readonly name?: string;
readonly scopes?: readonly string[];
/**
* The account the token is minted for: the tools' own (novox/hq ADR 0365 §2). The admin login mints it, after
* `setUp` made the account, and the admin's own tokens by this name are deleted. Left out, the token is the
* admin's own, as it was before.
*/
readonly account?: string;
/** Run with the admin's password before each mint for `account`: makes the account and its team (tools-account.ts). */
readonly setUp?: (password: string) => Promise<void>;
readonly log?: (line: string) => void;
readonly fetch?: typeof fetch;
}
@@ -106,12 +129,14 @@ export class MintedToken implements TokenSource {
private inflight: Promise<string> | null = null;
private readonly name: string;
private readonly scopes: readonly string[];
private readonly account: string;
private readonly log: (line: string) => void;
private readonly fetchImpl: typeof fetch;
constructor(private readonly opts: MintedTokenOptions) {
this.name = opts.name ?? TOKEN_NAME;
this.scopes = opts.scopes ?? TOKEN_SCOPES;
this.account = opts.account ?? opts.admin;
this.log = opts.log ?? ((line) => console.log(`[gitea] ${line}`));
this.fetchImpl = opts.fetch ?? fetch;
}
@@ -124,9 +149,9 @@ export class MintedToken implements TokenSource {
*/
static fromEnv(url: string, env: NodeJS.ProcessEnv = process.env): MintedToken {
const opts = MintedToken.optionsFromEnv(url, env);
// One source per kept file in a process. The watcher and the tools entrypoint both build a
// client in the same runtime; two sources over one file would each renew on a 401 and drop the
// other's token by name, forever. Shared, a renewal is one renewal.
// One source per kept file in a process: two sources in one process would each renew on a 401.
// This covers one process only. The watcher and the tools are two processes (the runtime starts
// each entrypoint as its own), and those are kept apart by the mint's lock file (mint, below).
const shared = MintedToken.shared.get(opts.file);
if (shared) return shared;
const source = new MintedToken(opts);
@@ -148,7 +173,17 @@ export class MintedToken implements TokenSource {
if (missing.length) {
throw new Error(`no Gitea token, and nothing to mint one with — set MESH_GITEA_TOKEN, or ${missing.join(", ")}`);
}
return { url, admin: admin!, passwordFile: passwordFile!, file: join(stateDir!, "token") };
// The tools' own account, never the administrator (novox/hq ADR 0365 §2): MESH_GITEA_TOOLS_USER names it.
const account = env.MESH_GITEA_TOOLS_USER || TOOLS_ACCOUNT;
if (account === admin) throw new Error(`MESH_GITEA_TOOLS_USER names the admin account "${admin}"; the tools act as an account of their own (novox/hq ADR 0365)`);
return {
url, admin: admin!, passwordFile: passwordFile!, file: join(stateDir!, "token"), account,
// GiteaAdmin is loaded when first used: client.ts imports this file.
setUp: async (password) => {
const { GiteaAdmin } = await import("./client.js");
await setUpToolsAccount(new GiteaAdmin(url, admin!, password), account, () => randomBytes(24).toString("base64url"));
},
};
}
async current(): Promise<string> {
@@ -159,11 +194,13 @@ export class MintedToken implements TokenSource {
if (kept !== null && this.keptScopes() === this.scopesLine()) {
this.held = kept;
this.log(`reusing the token kept at ${this.opts.file}`);
// A retirement of the admin's old token that failed after the mint is tried again, once a process.
if (existsSync(this.retireFile)) await this.locked(() => this.retireAdmin()).catch((err) => this.log(`${err}`));
return kept;
}
if (kept !== null) return this.mint("the kept token was minted with other scopes than the tools take now — minting one with them");
if (kept !== null) return this.mint("the kept token was minted with other scopes than the tools take now — minting one with them", kept);
}
return this.mint("no token kept — minting one");
return this.mint("no token kept — minting one", null);
}
async renew(rejected: string): Promise<string> {
@@ -177,7 +214,7 @@ export class MintedToken implements TokenSource {
return kept;
}
this.held = null;
return this.mint("the forge rejected the kept token — minting a fresh one");
return this.mint("the forge rejected the kept token — minting a fresh one", rejected);
}
/** What the forge's scoped tokens say when a kept token predates a scope the tools now need. */
@@ -185,17 +222,95 @@ export class MintedToken implements TokenSource {
return status === 403 && /required scope/i.test(body);
}
/** One mint at a time: concurrent first calls share it, rather than each minting its own. */
private mint(why: string): Promise<string> {
/**
* One mint at a time: concurrent first calls in this process share it, and a mint in another process
* on this machine is waited for (the lock file). Once the lock is held, the kept file is read again:
* a token there that is not `stale` (the one rejected, or kept with other scopes) and carries the
* scopes asked for was minted by the process that held the lock before — it is reused.
*/
private mint(why: string, stale: string | null): Promise<string> {
if (this.inflight === null) {
this.log(why);
this.inflight = this.doMint().finally(() => {
this.inflight = this.locked(async () => {
const kept = this.read();
if (kept !== null && kept !== stale && this.keptScopes() === this.scopesLine()) {
this.held = kept;
this.log(`another process minted it meanwhile — reusing the token kept at ${this.opts.file}`);
return kept;
}
return this.doMint();
}).finally(() => {
this.inflight = null;
});
}
return this.inflight;
}
/** How long a mint may hold the lock before another process takes it as left behind, and how long one waits. */
static readonly LOCK_STALE_MS = 120_000;
static readonly LOCK_WAIT_MS = 150_000;
/**
* Run `work` holding the lock file beside the kept token: created exclusively, holding this process's
* pid, the time and a nonce; removed after, only when it is still ours. A lock whose process is gone,
* or older than LOCK_STALE_MS, was left by a process that died mid-mint and is taken over — never
* waited on for ever, which would be the same lock-out by another road.
*/
private async locked<T>(work: () => Promise<T>): Promise<T> {
const lock = `${this.opts.file}.lock`;
mkdirSync(dirname(this.opts.file), { recursive: true, mode: 0o700 });
const mine = `${process.pid} ${Date.now()} ${randomBytes(8).toString("hex")}\n`;
const deadline = Date.now() + MintedToken.LOCK_WAIT_MS;
for (;;) {
try {
const fd = openSync(lock, "wx", 0o600);
try { writeSync(fd, mine); } finally { closeSync(fd); }
break;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "EEXIST") {
throw new Error(`cannot take the mint lock at ${lock}: ${(err as Error).message}`);
}
}
if (this.leftBehind(lock)) {
this.log(`the mint lock at ${lock} was left by a process that is gone — taking it over`);
try { unlinkSync(lock); } catch { /* another waiter took it over first */ }
continue;
}
if (Date.now() > deadline) {
throw new Error(`another process has held the mint lock at ${lock} for over ${MintedToken.LOCK_WAIT_MS / 1000}s — not minting beside it`);
}
await new Promise((r) => setTimeout(r, 50 + Math.floor(Math.random() * 50)));
}
try {
return await work();
} finally {
try {
if (readFileSync(lock, "utf8") === mine) unlinkSync(lock);
} catch { /* already gone */ }
}
}
/** Whether the lock's holder is gone: its pid answers no signal, or it is older than LOCK_STALE_MS. */
private leftBehind(lock: string): boolean {
let text: string;
let age: number;
try {
text = readFileSync(lock, "utf8");
age = Date.now() - statSync(lock).mtimeMs;
} catch {
return false; // released meanwhile: the next attempt takes it
}
if (age > MintedToken.LOCK_STALE_MS) return true;
const pid = Number(text.split(" ")[0]);
if (!Number.isInteger(pid) || pid <= 0) return age > 1_000; // half-written, or not ours: give it a moment
try {
process.kill(pid, 0);
return false;
} catch (err) {
return (err as NodeJS.ErrnoException).code === "ESRCH";
}
}
private read(): string | null {
try {
const token = readFileSync(this.opts.file, "utf8").replace(/\n$/, "");
@@ -206,9 +321,10 @@ export class MintedToken implements TokenSource {
}
}
/** The scopes as kept beside the token: sorted, one line. */
/** The scopes as kept beside the token: sorted, one line, and whose it is when it is not the admin's. */
private scopesLine(): string {
return [...this.scopes].sort().join(",");
const scopes = [...this.scopes].sort().join(",");
return this.account === this.opts.admin ? scopes : `${scopes} for ${this.account}`;
}
/** The scopes the kept token was minted with, "" when not kept (a token kept before they were). */
@@ -231,17 +347,20 @@ export class MintedToken implements TokenSource {
renameSync(`${this.opts.file}.scopes.tmp`, `${this.opts.file}.scopes`);
}
private async doMint(): Promise<string> {
let password: string;
/** One call to the forge's API with the admin login, the password read now. */
private password(): string {
try {
password = readFileSync(this.opts.passwordFile, "utf8").replace(/\n$/, "");
return readFileSync(this.opts.passwordFile, "utf8").replace(/\n$/, "");
} catch (err) {
throw new Error(`cannot read the admin password at ${this.opts.passwordFile}: ${(err as Error).message}`);
}
}
private adminCall(password = this.password()): AdminCall {
const authorization = "Basic " + Buffer.from(`${this.opts.admin}:${password}`).toString("base64");
const tokens = `${this.opts.url.replace(/\/+$/, "")}/api/v1/users/${encodeURIComponent(this.opts.admin)}/tokens`;
const call = async (method: string, path = "", body?: unknown): Promise<{ status: number; body: any }> => {
const res = await this.fetchImpl(tokens + path, {
const api = `${this.opts.url.replace(/\/+$/, "")}/api/v1`;
const admin: AdminCall = async (method, path, body) => {
const res = await this.fetchImpl(api + path, {
method,
headers: { "Content-Type": "application/json", Authorization: authorization },
...(body === undefined ? {} : { body: JSON.stringify(body) }),
@@ -253,30 +372,97 @@ export class MintedToken implements TokenSource {
}
return { status: res.status, body: parsed };
};
return admin;
}
/** Where a retirement of the admin's old token that failed is remembered, to be tried again (retireAdmin). */
private get retireFile(): string {
return `${this.opts.file}.retire-admin`;
}
/**
* The admin's own token by this name goes: what the tools used before must not outlive the move (novox/hq ADR
* 0365 §2). Run after the account's token is minted and kept, so a failure here never leaves the tools without
* a token: it is said, remembered beside the kept token, and tried again when a process next reads that token.
*/
private async retireAdmin(admin: AdminCall = this.adminCall()): Promise<void> {
if (this.account === this.opts.admin) return;
try {
const retired = await this.dropNamed(admin, `/users/${encodeURIComponent(this.opts.admin)}/tokens`, this.opts.admin);
if (retired > 0) this.log(`deleted ${retired} token(s) named "${this.name}" the admin account "${this.opts.admin}" held: the tools act as "${this.account}" now`);
rmSync(this.retireFile, { force: true });
} catch (err) {
writeFileSync(this.retireFile, `${new Date().toISOString()}\n`, { mode: 0o600 });
this.log(`FAILED to delete the token(s) named "${this.name}" the admin account "${this.opts.admin}" holds — tried again at the next start: ${(err as Error).message}`);
}
}
private async doMint(): Promise<string> {
const password = this.password();
const admin = this.adminCall(password);
const tokensOf = (account: string) => `/users/${encodeURIComponent(account)}/tokens`;
const call = (method: string, path = "", body?: unknown) => admin(method, tokensOf(this.account) + path, body);
if (this.account !== this.opts.admin) {
// The tools' account and its team first (novox/hq ADR 0365 §2): a failure here fails the mint, tried again
// at the next call.
const me = await admin("GET", "/user");
if (me.status === 401 || me.status === 403) throw new AdminRefused(this.opts.admin, me.status);
await this.opts.setUp?.(password);
}
let res = await call("POST", "", { name: this.name, scopes: this.scopes });
if (res.status === 401 || res.status === 403) throw new AdminRefused(this.opts.admin, res.status);
if (res.status === 400 || res.status === 422) {
// The forge still holds a token by this name whose value we no longer have — the kept file
// went while the forge's data stayed. It is ours to replace: drop it by name and mint again.
this.log(`the forge already holds a token named "${this.name}" — replacing it`);
const dropped = await call("DELETE", `/${encodeURIComponent(this.name)}`);
if (dropped.status !== 204 && dropped.status !== 404) {
throw new Error(`Gitea DELETE /users/${this.opts.admin}/tokens/${this.name}: ${dropped.status} ${detail(dropped.body)}`);
}
// went while the forge's data stayed, or two processes once minted at once and left two. It is
// ours to replace: find every one by this name in the account's list and delete each by its id
// (gitea refuses a delete by a name held twice: "multiple matches for token name"), then mint.
await this.dropNamed(admin, tokensOf(this.account), this.account, true);
res = await call("POST", "", { name: this.name, scopes: this.scopes });
}
if (res.status !== 201 && res.status !== 200) {
throw new Error(`Gitea POST /users/${this.opts.admin}/tokens: ${res.status} ${detail(res.body)}`);
throw new Error(`Gitea POST /users/${this.account}/tokens: ${res.status} ${detail(res.body)}`);
}
const token = typeof res.body?.sha1 === "string" ? res.body.sha1 : null;
if (!token) throw new Error(`Gitea POST /users/${this.opts.admin}/tokens: ${res.status} but no token in the reply`);
if (!token) throw new Error(`Gitea POST /users/${this.account}/tokens: ${res.status} but no token in the reply`);
this.keep(token);
this.held = token;
this.log(`minted a token for "${this.opts.admin}" (${this.scopes.join(", ")}), kept at ${this.opts.file}`);
this.log(`minted a token for "${this.account}" (${this.scopes.join(", ")}), kept at ${this.opts.file}`);
// Only now, with the account's own token kept, does the admin's old one go.
await this.retireAdmin(admin);
return token;
}
/**
* Delete every token by this name an account holds, each by its id (gitea refuses a delete by a name held twice:
* "multiple matches for token name"); answers how many. `replacing` says the line for a mint that found its name
* taken.
*/
private async dropNamed(admin: AdminCall, tokens: string, account: string, replacing = false): Promise<number> {
const held: number[] = [];
for (let page = 1; page <= 100; page++) {
const listed = await admin("GET", `${tokens}?page=${page}&limit=50`);
if (listed.status !== 200 || !Array.isArray(listed.body)) {
throw new Error(`Gitea GET /users/${account}/tokens: ${listed.status} ${detail(listed.body)}`);
}
if (listed.body.length === 0) break;
for (const t of listed.body) if (t?.name === this.name && typeof t.id === "number") held.push(t.id);
}
if (replacing) {
this.log(held.length === 1
? `the forge already holds a token named "${this.name}" — replacing it`
: `the forge holds ${held.length} tokens named "${this.name}" — deleting each by its id and minting one`);
}
for (const id of held) {
const dropped = await admin("DELETE", `${tokens}/${id}`);
if (dropped.status !== 204 && dropped.status !== 404) {
throw new Error(`Gitea DELETE /users/${account}/tokens/${id} (named ${this.name}): ${dropped.status} ${detail(dropped.body)}`);
}
}
return held.length;
}
}
function detail(body: unknown): string {
+52
View File
@@ -0,0 +1,52 @@
// The forge's tools act as an account of their own, not as the site administrator (novox/hq ADR 0365 §2,
// issue 469).
//
// The administrator's token reached everything the forge holds, so the tools' boundary rested on their own guards
// (guard.ts): a protection edit, say, the forge itself would have taken. Now the tools' token is minted for a plain
// account, `mesh-tools`, which is no administrator and writes to the mesh's repositories through one team of the
// mesh's organisation:
//
// - **the account**: made by the module with the administrator's login (that login sets the forge up and does
// nothing else), never an administrator — one found made an administrator is made a plain account again. Its
// password is random, sent once and kept nowhere: the account is reached only by the tokens the administrator
// mints for it (gitea lets an administrator mint for another account: `reqSelfOrAdmin`).
// - **the team**: `mesh-tools` in the organisation `novox`, on every repository of it, writing code (pushes to a
// branch, commit statuses, merges), issues (notes, comments, labels) and pull requests. No administration of a
// repository, so the forge refuses the account a protection edit, a repository's settings and its deletion;
// no repository made in the organisation, since its maker would be that repository's administrator.
//
// Every step is idempotent (GiteaAdmin's, client.ts), and run with the administrator's login before each mint of the tools' token, so a team
// or a membership lost behind the module's back is made again at the next mint.
/** The account the tools act as. */
export const TOOLS_ACCOUNT = "mesh-tools";
/** The organisation whose repositories are the mesh's, and the team through which the tools' account writes there. */
export const TOOLS_ORG = "novox";
export const TOOLS_TEAM = "mesh-tools";
/** What the team grants: write on code, issues and pull requests of every repository of the organisation, and no more. */
export const TOOLS_TEAM_UNITS = {
permission: "write",
units_map: { "repo.code": "write", "repo.issues": "write", "repo.pulls": "write" },
includes_all_repositories: true,
can_create_org_repo: false,
} as const;
/** One call to the forge's API with the administrator's login: a path under /api/v1, the status and body answered. */
export type AdminCall = (method: string, path: string, body?: unknown) => Promise<{ status: number; body: any }>;
/** What setting the account up needs of the forge's admin API: GiteaAdmin's own (client.ts), shared with the
* package provisioner, so a create race (422) and a user made by hand are handled once. */
export interface ToolsForge {
ensureUser(username: string, password: string, email: string, plain?: boolean): Promise<void>;
ensureTeamWith(org: string, team: string, units: Record<string, unknown>): Promise<number>;
addUserToTeam(teamId: number, username: string): Promise<void>;
}
/** Make the tools' account, plain, a member of its team on every repository of the organisation. Its password is
* set afresh each time and kept nowhere: the account is reached only by the tokens the administrator mints. */
export async function setUpToolsAccount(forge: ToolsForge, account = TOOLS_ACCOUNT, password: () => string): Promise<void> {
await forge.ensureUser(account, password(), `${account}@noreply.localhost`, true);
const team = await forge.ensureTeamWith(TOOLS_ORG, TOOLS_TEAM, { name: TOOLS_TEAM, ...TOOLS_TEAM_UNITS });
await forge.addUserToTeam(team, account);
}
+19 -12
View File
@@ -10,7 +10,7 @@
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { emit } from "@novox/mesh-sdk/events";
import { GiteaClient, type BranchProtection } from "../client.js";
import { GiteaAdmin, GiteaClient, readyAuthors, type BranchProtection } from "../client.js";
import { appendNote, run, viewBody, viewComment } from "../delivery.js";
import { containedFrom, isCommit, mergeableUnder, newestByContext } from "../checks.js";
import {
@@ -59,13 +59,13 @@ function namesChecked(tools: ToolDefinition[]): ToolDefinition[] {
}));
}
export function getGiteaTools(gitea: GiteaClient): ToolDefinition[] {
return namesChecked(allGiteaTools(gitea));
export function getGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
return namesChecked(allGiteaTools(gitea, admin));
}
// gitea_merge_pull_request is served from the watcher's bundle (merge.ts, index.ts), beside the merge check's one
// writer (novox/hq issue 456, ADR 0323).
function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefinition[] {
return [
// ---- Repositories ----
{
@@ -401,8 +401,11 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
},
run: async (args) => {
const owner = String(args.owner), repo = String(args.repo);
if (!args.branch) return { rules: (await gitea.branchProtections(owner, repo)).map(summarised) };
const rule = await gitea.branchProtection(owner, repo, String(args.branch));
// The one read made as the forge's administrator (novox/hq ADR 0375, issue 772): the rules' list is the
// repository administrator's, which the tools' account is not. No other tool uses this account.
const rules = await (admin ? admin() : gitea).branchProtections(owner, repo);
if (!args.branch) return { rules: rules.map(summarised) };
const rule = rules.find((p) => (p.rule_name ?? p.branch_name) === String(args.branch)) ?? null;
return { branch: String(args.branch), rule: rule ? summarised(rule) : null };
},
},
@@ -504,7 +507,7 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
base = base || "main";
let rule: BranchProtection | null = null, unreadable: string | undefined;
try {
rule = await gitea.branchProtection(owner, repo, base);
rule = await gitea.appliedProtection(owner, repo, base);
} catch (err) {
unreadable = err instanceof Error ? err.message : String(err);
}
@@ -514,9 +517,9 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
},
{
name: "gitea_open_pulls",
description: "Every open pull request of every repository, from the forge itself: its head, its base, the statuses the base's protection requires and the ones its head has, each with its state, description and when it was set. What mesh-delivery reads to say a pull request the mesh was never asked to check (novox/hq issue 347), or whose check started and never answered (issue 438). Reads only.",
description: "Every open pull request of every repository, from the forge itself: its head, its base, the statuses the base's protection requires and the ones its head has, each with its state, description and when it was set. What mesh-delivery reads to say a pull request the mesh was never asked to check (novox/hq issue 347), or whose check started and never answered (issue 438), and each pull request's labels and ready mark — the label mesh/ready and the head named by the newest comment \"ready at <full head sha>\" that a listed account (the setting ready-authors) wrote after the label was added and after the last push — which mesh-delivery merges on (issue 722). Reads only.",
input: {},
run: async () => ({ pulls: await gitea.openPulls() }),
run: async () => ({ pulls: await gitea.openPulls(readyAuthors()) }),
},
{
name: "gitea_contains",
@@ -589,8 +592,12 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
const { owner, repo, branch, newBranch } = judged.contents;
const found = await gitea.api<{ default_branch?: string }>(`/repos/${checkName("owner", owner)}/${checkName("name", repo)}`);
const defaultBranch = String(found?.default_branch ?? "main");
const refused = judgeContentsWrite(newBranch ?? branch ?? defaultBranch, defaultBranch,
await gitea.branchProtections(owner, repo));
// The protection the forge applies to the branch written, read from the branch: the rules' own list is a
// repository administrator's, which the tools' account is not (novox/hq ADR 0365). A branch not made yet is
// left to the forge, which refuses a write that would make a branch a rule guards.
const target = newBranch ?? branch ?? defaultBranch;
const applied = target === defaultBranch ? null : await gitea.appliedProtection(owner, repo, target);
const refused = judgeContentsWrite(target, defaultBranch, applied ? [applied] : []);
if (refused) throw new Refused(refused);
}
// The path judged is the path sent: normalised, with only the allow-listed query keys (guard.ts).
@@ -611,7 +618,7 @@ function allGiteaTools(gitea: GiteaClient): ToolDefinition[] {
// way, gitea contributes none rather than failing the whole runtime, and says why.
registerModuleTools("gitea", (env) => {
try {
return getGiteaTools(GiteaClient.fromEnv(env));
return getGiteaTools(GiteaClient.fromEnv(env), () => GiteaAdmin.fromEnv(env));
} catch (err) {
console.log(`[gitea] no tools — ${err instanceof Error ? err.message : String(err)}`);
return [];
+4 -2
View File
@@ -128,10 +128,10 @@ export interface OutOfDateWriter {
* out of date (ADR 0323 §4). A trunk first seen is recorded as it is and nothing is put out of date. Bounded however
* many are open: one listing of the open pull requests (paged), one read of each trunk, then per open pull request into
* a trunk that moved one read and at most two writes, one at a time. A trunk that could not be weighed whole is not
* recorded, so the next pass weighs it again. Answers whether the record changed.
* recorded, so the next pass weighs it again. Answers whether the record changed. `moved` is told of each base that moved, for the siblings whose base it is.
*/
export async function weighTrunks(forge: TrunkForge, gates: OutOfDateWriter, trunks: Record<string, string>, owner: string,
name: string): Promise<boolean> {
name: string, moved?: (base: string) => Promise<void>): Promise<boolean> {
let changed = false;
try {
const open = (await forge.listOpenPulls(owner, name)).filter((p) => p.state === "open" && !!p.head_sha && !!p.base);
@@ -151,6 +151,8 @@ export async function weighTrunks(forge: TrunkForge, gates: OutOfDateWriter, tru
}
console.log(`[gitea] ${owner}/${name}'s ${base} moved from ${twelve(before)} to ${twelve(head)}: ${put} open pull ` +
`request(s) into it put out of date, nothing run (novox/hq ADR 0323)`);
// A verdict that read a pull request into this trunk as a sibling is out of date by its move too (ADR 0382 §5).
if (moved) await moved(base);
}
trunks[key] = head;
changed = true;
+519
View File
@@ -0,0 +1,519 @@
// **The forge says every event on the bus** (novox/hq issue 729; to-be 10: an event makes it fast, nothing makes it
// necessary). The module keeps one system webhook on its forge for every event, pointing at its own receiver, signed
// with a secret it mints and keeps beside its token; the receiver refuses whatever is unsigned or does not verify, and
// publishes each event, trimmed, on the module's events as forge.<kind>.<action> (forge.pull-request.opened, forge.push, …),
// on the shared EVENTS path its other events take; its own stream (ADR 0363) is a follow-up once issue 725 ships.
// The polls stay as the safety net, slowed; the announced-heads and announced-merges records keep anything from
// being announced twice, whichever path saw it first.
//
// The secret is never logged: lines say it was minted or reused and where it is kept, never what it is.
import { createHmac, randomBytes, timingSafeEqual } from "node:crypto";
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { createServer, type Server } from "node:http";
import { dirname } from "node:path";
/** The name the receiver answers on, and the port it listens on unless the manifest says otherwise. */
export const HOOK_PATH = "/forge-event";
export const HOOK_PORT = 3047;
/** The secret the forge signs with: read from its 0600 file, or minted and kept there. */
export function keptSecret(file: string, log: (line: string) => void = (l) => console.log(`[gitea] ${l}`)): string {
try {
const kept = readFileSync(file, "utf8").trim();
if (kept.length >= 32) {
log(`reusing the webhook secret kept at ${file}`);
return kept;
}
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err;
}
const secret = randomBytes(32).toString("hex");
mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
const tmp = `${file}.tmp`;
writeFileSync(tmp, secret + "\n", { mode: 0o600 });
chmodSync(tmp, 0o600);
renameSync(tmp, file);
log(`minted a webhook secret, kept at ${file}`);
return secret;
}
/** Whether a body carries the forge's HMAC-SHA256 signature (X-Gitea-Signature, hex) under the secret. */
export function verified(secret: string, body: Buffer | string, signature: string | undefined): boolean {
if (!secret || !signature || !/^[0-9a-f]{64}$/i.test(signature.trim())) return false;
const want = createHmac("sha256", secret).update(body).digest();
const got = Buffer.from(signature.trim(), "hex");
return got.length === want.length && timingSafeEqual(got, want);
}
/** One forge event as the bus carries it: its name inside the stream, and what is needed of it — no secrets. */
export interface ForgeEvent {
name: string;
body: Record<string, unknown>;
}
/** Every event name the module emits for the forge, without its forge. prefix (module.json `emits`). */
export const EVENTS: readonly string[] = ["push", "create", "delete", "fork", "status", "pull-request.opened", "pull-request.reopened", "pull-request.synchronized", "pull-request.edited", "pull-request.closed", "pull-request.merged", "pull-request.labeled", "pull-request.unlabeled", "pull-request.assigned", "pull-request.unassigned", "pull-request.reviewed", "pull-request.review-requested", "pull-request.commented", "pull-request.changed", "issue-comment.created", "issue-comment.edited", "issue-comment.deleted", "issues.opened", "issues.reopened", "issues.closed", "issues.edited", "issues.labeled", "issues.unlabeled", "issues.assigned", "issues.unassigned", "issues.changed", "release.published", "release.updated", "release.deleted", "release.changed", "repository.created", "repository.deleted", "repository.changed", "wiki.changed", "other"];
/** The families the module's stream carries (module.json `streams`); any other kind is said as other.<kind>; underscores become hyphens. */
export const FAMILIES = ["push", "create", "delete", "fork", "status", "pull_request", "issue_comment", "issues",
"release", "repository", "wiki"] as const;
const ACTIONS: Record<string, string> = { label_updated: "labeled", label_cleared: "unlabeled", synchronized: "synchronized" };
function str(v: unknown): string | undefined {
return typeof v === "string" && v ? v : undefined;
}
function num(v: unknown): number | undefined {
return typeof v === "number" ? v : undefined;
}
function login(u: any): string | undefined {
return str(u?.login) ?? str(u?.username);
}
/**
* The event a forge delivery is: its kind (X-Gitea-Event) and the payload's action, trimmed to ids, repository, refs,
* shas, author, labels and a comment's body (a ready mark is one, ADR 0364). A merge is pull-request.merged, a comment on
* a pull request pull-request.commented.
*/
export function eventOf(kind: string, payload: any): ForgeEvent {
// The bus's event names take lower-case letters, digits and hyphens: the forge's underscores become hyphens.
const event = forgeEventOf(kind, payload);
const name = event.name.toLowerCase().replace(/_/g, "-").replace(/[^a-z0-9.-]/g, "-");
// Published on the module's events as it publishes its others, each name declared in its `emits`: one the manifest
// does not name is forge.other, saying what the forge called it. Its own stream (ADR 0363) follows issue 725.
if (EVENTS.includes(name)) return { name: `forge.${name}`, body: event.body };
return { name: "forge.other", body: { ...event.body, kind, said: name } };
}
function forgeEventOf(said: string, payload: any): ForgeEvent {
// The forge names some kinds finer than their family (pull_request_label, pull_request_sync): read by the payload.
let kind = said;
if (said.startsWith("pull_request") && payload?.pull_request) kind = "pull_request";
if ((said.startsWith("issue") || said === "pull_request_comment") && payload?.comment) kind = "issue_comment";
if (said.startsWith("issue") && !payload?.comment && payload?.issue) kind = "issues";
const repo = payload?.repository ?? {};
const base: Record<string, unknown> = {
owner: login(repo.owner), repo: str(repo.name), full_name: str(repo.full_name), clone_url: str(repo.clone_url),
sender: login(payload?.sender),
};
const family = (FAMILIES as readonly string[]).includes(kind) ? kind : `other`;
let action = str(payload?.action);
const pull = payload?.pull_request;
if (kind === "push") {
return { name: "push", body: { ...base, ref: str(payload.ref), before: str(payload.before), after: str(payload.after),
commits: Array.isArray(payload.commits) ? payload.commits.length : undefined, pusher: login(payload.pusher) } };
}
if (kind === "pull_request" && pull) {
if (action === "closed" && pull.merged) action = "merged";
action = ACTIONS[action ?? ""] ?? action ?? "changed";
return { name: `pull_request.${action}`, body: { ...base, number: num(pull.number ?? payload.number),
title: str(pull.title), author: login(pull.user), head: str(pull.head?.ref), head_sha: str(pull.head?.sha),
base: str(pull.base?.ref), merged: !!pull.merged, merge_commit_sha: str(pull.merge_commit_sha),
merged_at: str(pull.merged_at), html_url: str(pull.html_url),
labels: Array.isArray(pull.labels) ? pull.labels.map((l: any) => str(l?.name)).filter(Boolean) : undefined,
label: str(payload.label?.name) } };
}
if (kind === "issue_comment") {
const issue = payload.issue ?? {};
const onPull = !!payload.is_pull || !!issue.pull_request;
return { name: onPull ? "pull_request.commented" : `issue_comment.${action ?? "created"}`, body: { ...base,
number: num(issue.number), comment_id: num(payload.comment?.id), author: login(payload.comment?.user),
comment: str(payload.comment?.body), action } };
}
if (kind === "create" || kind === "delete") {
return { name: kind, body: { ...base, ref: str(payload.ref), ref_type: str(payload.ref_type), sha: str(payload.sha) } };
}
if (kind === "status") {
return { name: "status", body: { ...base, sha: str(payload.sha), context: str(payload.context), state: str(payload.state) } };
}
const name = family === "other" ? `other.${kind.replace(/[^a-z0-9_-]/gi, "_") || "unknown"}` :
["fork"].includes(family) ? family : `${family}.${action ?? "changed"}`;
return { name, body: { ...base, number: num(payload?.number ?? payload?.issue?.number), action } };
}
export interface ReceiverOptions {
secret: string;
port: number;
host?: string;
/** Publishes one event on the bus. */
publish: (event: ForgeEvent) => Promise<void>;
/** Acts on it at once, beside publishing it (the announcer). */
act?: (event: ForgeEvent) => Promise<void>;
/** Told of every verified delivery — its kind (X-Gitea-Event) and its delivery id — so the polls know events are
* heard, and the module can say when it last heard one. */
heard?: (kind: string, delivery: string) => void;
/** Told of every refused delivery, with the reason and where it came from. */
refused?: (reason: string, from: string) => void;
log?: (line: string) => void;
}
/** The most a delivery may carry: the forge's push of many commits stays well under it. */
export const MAX_BODY = 5 * 1024 * 1024;
/** How many deliveries' ids are remembered, so a delivery sent again is published once. */
export const REMEMBERED = 2000;
/**
* The receiver. A request without a signature is refused before its body is read, a body over MAX_BODY is refused, and
* one whose signature does not verify is refused, each with 401 or 413 and a line saying so. A verified delivery is
* answered 200 at once and queued: it is published and acted on after the reply, one at a time, so the forge's
* delivery timeout is never reached. A delivery whose X-Gitea-Delivery id was already heard is answered and dropped.
*/
export function receiver(opts: ReceiverOptions): Server & { idle(): Promise<void> } {
const log = opts.log ?? ((l) => console.log(`[gitea] ${l}`));
const refuse = (from: string, reason: string): void => {
log(`refused a forge event from ${from}: ${reason}`);
opts.refused?.(reason, from);
};
const seen = new Set<string>();
let queue: Promise<void> = Promise.resolve();
const server = createServer((req, res) => {
if (req.method !== "POST" || (req.url ?? "").split("?")[0] !== HOOK_PATH) {
res.statusCode = 404;
return res.end();
}
const from = req.socket.remoteAddress ?? "?";
const signature = String(req.headers["x-gitea-signature"] ?? "").trim();
if (!signature) {
refuse(from, "it is unsigned");
res.writeHead(401, { Connection: "close" });
res.end("unsigned\n");
req.destroy();
return;
}
if (Number(req.headers["content-length"] ?? 0) > MAX_BODY) {
refuse(from, `its body is over ${MAX_BODY} bytes`);
res.writeHead(413, { Connection: "close" });
res.end("too large\n");
req.destroy();
return;
}
const chunks: Buffer[] = [];
let size = 0;
let over = false;
req.on("data", (c: Buffer) => {
size += c.length;
if (size > MAX_BODY) {
if (!over) {
over = true;
refuse(from, `its body is over ${MAX_BODY} bytes`);
res.writeHead(413, { Connection: "close" });
res.end("too large\n");
req.destroy();
}
} else chunks.push(c);
});
req.on("end", () => {
if (over) return;
const body = Buffer.concat(chunks);
if (!verified(opts.secret, body, signature)) {
refuse(from, "its signature does not verify");
res.statusCode = 401;
return res.end("signature does not verify\n");
}
let payload: unknown;
try {
payload = JSON.parse(body.toString("utf8"));
} catch {
refuse(from, "it is signed but not JSON");
res.statusCode = 400;
return res.end("not JSON\n");
}
const delivery = String(req.headers["x-gitea-delivery"] ?? "");
const kind = String(req.headers["x-gitea-event"] ?? "other");
opts.heard?.(kind, delivery);
if (delivery && seen.has(delivery)) {
res.statusCode = 200;
return res.end("already heard\n");
}
if (delivery) {
seen.add(delivery);
if (seen.size > REMEMBERED) seen.delete(seen.values().next().value as string);
}
const event = eventOf(kind, payload);
res.statusCode = 200;
res.end("queued\n");
queue = queue.then(async () => {
try {
await opts.publish(event);
if (opts.act) await opts.act(event);
} catch (err) {
log(`a forge event ${event.name} could not be published or acted on; the polls catch it — ${err instanceof Error ? err.message : err}`);
}
});
});
});
return Object.assign(server, { idle: () => queue });
}
/** What ensuring the system webhook needs of the forge's administration (GiteaAdmin's login). */
export interface HookAdmin {
call(method: string, path: string, body?: unknown): Promise<{ status: number; body: any }>;
}
/** The forge's one system webhook as ensured at start: read back from the forge after it was made. */
export interface HookState {
/** Always "system": a hook the forge's listing of system webhooks names, so it fires for every repository. */
kind: "system";
id: number;
url: string;
events: string[];
active: boolean;
content_type?: string;
/** Made where none was, or made in place of the ones there (system, default, or a repository's) at the receiver's url. */
did: "created" | "replaced";
/** The hooks at the receiver's url it took the place of, by kind: a repository's is a copy of a default webhook, which
* would have sent that repository's events a second time. */
removed: { system: number; default: number; repository: number; repositories: string[] };
}
/** How long without a verified delivery — since the start, or since the last one — a forge is said silent once the poll
* finds what no event said. */
export const SILENT_AFTER = 10 * 60_000;
/**
* **How fast the polls look, and whether the forge's events are heard** (issue 729): every five minutes only while the
* forge's events are heard — a verified delivery within the last ten minutes, and none missed since. Until the first
* delivery, after ten minutes without one, or once the poll found something no event said, every thirty seconds.
*
* It also keeps what `gitea_events` says: where the receiver listens, the hook as ensured, the verified and refused
* deliveries, and the polls' finds no event said. **A forge whose events stop is said** (issue 729): once the receiver
* listens and the hook is ensured, when nothing verified for ten minutes — since the start, or since the last delivery —
* and the poll has found at least two heads or merges no event said in that quiet (one could be the poll beating an
* event by a second), it is logged loudly once per silent spell, and the module's events are said unhealthy until a
* delivery verifies. Deliveries refused since the last verified one are said unhealthy too.
*/
export class Pace {
private last = 0;
private missed = 0;
private readonly started: number;
private readonly now: () => number;
private readonly log: (l: string) => void;
private where: { host: string; port: number; url: string } | null = null;
private hook: HookState | null = null;
private notHeard: string | null = null;
private notHeardFails = false;
private verifiedCount = 0;
private lastVerified: { at: number; event: string; delivery: string } | null = null;
private refusedCount = 0;
private refusedSpell = 0;
private lastRefused: { at: number; reason: string; from: string } | null = null;
private missCount = 0;
private lastMiss: { at: number; what: string } | null = null;
private silentSince = 0;
private quietMisses = 0;
constructor(now: () => number = Date.now, log: (l: string) => void = (l) => console.error(`[gitea] ${l}`)) {
this.now = now;
this.log = log;
this.started = now();
}
/** The receiver listens, and the hook at it was ensured: from now on, deliveries are expected. */
listening(host: string, port: number, url: string, hook: HookState): void {
this.where = { host, port, url };
this.hook = hook;
this.notHeard = null;
}
/** No events are listened for, and why: by configuration (`fails` false), or because the receiver could not listen
* or the hook could not be ensured (`fails` true, said unhealthy). The polls alone watch. */
unheard(why: string, fails = true): void {
this.notHeard = why;
this.notHeardFails = fails;
}
heard(event = "", delivery = ""): void {
this.last = this.now();
this.verifiedCount++;
this.lastVerified = { at: this.last, event, delivery };
this.refusedSpell = 0;
this.quietMisses = 0;
if (this.silentSince) this.log(`the forge's events are heard again: ${event || "a delivery"} verified`);
this.silentSince = 0;
}
refused(reason: string, from: string): void {
this.refusedCount++;
this.refusedSpell++;
this.lastRefused = { at: this.now(), reason, from };
}
/** The poll found a head or merge no forge event announced. */
miss(what: string): void {
const t = this.now();
if (this.hearing()) this.log(`THE FORGE'S EVENTS MISSED ${what}: the poll found it, so the polls look every 30 s again until the next event`);
this.missed = t;
this.missCount++;
this.lastMiss = { at: t, what };
// Silent: nothing verified for SILENT_AFTER — never since the start, or not since the last delivery (a hook deleted by
// hand, every delivery refused) — while the poll finds what no event said. Said once per silent spell.
const quietSince = this.last || this.started;
this.quietMisses++;
if (this.expecting() && t - quietSince >= SILENT_AFTER && this.quietMisses >= 2 && !this.silentSince) {
this.silentSince = t;
const minutes = Math.round((t - quietSince) / 60_000);
const refusedSince = this.refusedSinceHeard();
this.log(`THE FORGE'S EVENTS ARE NOT HEARD: ${this.last ? `no delivery verified since the last at ` +
`${new Date(this.last).toISOString()}` : "no delivery ever verified since the start"}, ${minutes} min, from the ` +
`system webhook at ${this.hook?.url}${refusedSince ? `; ${refusedSince} refused since, the last because ` +
`${this.lastRefused?.reason}` : ""}; the poll found ${what} — the polls alone watch the forge, every 30 s ` +
`(novox/hq issue 729)`);
}
}
/** Whether a delivery was refused after the last verified one (or with none verified). */
private refusedSinceHeard(): number {
return this.lastRefused && this.lastRefused.at >= this.last ? this.refusedSpell : 0;
}
hearing(): boolean {
const t = this.now();
return this.last > 0 && t - this.last < 10 * 60_000 && this.last > this.missed;
}
every(): number {
return this.hearing() ? 5 * 60_000 : 30_000;
}
private expecting(): boolean {
return this.where !== null && this.hook !== null;
}
/** What `gitea_events` answers: healthy, unhealthy or waits (ADR 0283's shape), with every fact behind it. */
report(): Record<string, unknown> {
const t = this.now();
const iso = (ms: number): string => new Date(ms).toISOString();
let state: "healthy" | "unhealthy" | "waits" = "healthy";
let why: string;
const refusedSince = this.refusedSinceHeard();
if (this.silentSince) {
state = "unhealthy";
why = `${this.last ? `no delivery verified since the last at ${iso(this.last)}` : `the forge has never delivered an ` +
`event since the start at ${iso(this.started)}`}, while the poll found ${this.missCount} head(s) or merge(s) no ` +
`event said, the last ${this.lastMiss?.what} at ${iso(this.lastMiss?.at ?? t)}`;
} else if (refusedSince) {
state = "unhealthy";
why = `${refusedSince} deliveries refused ${this.last ? `since the last verified one at ${iso(this.last)}` :
"and none verified"}, the last at ${iso(this.lastRefused?.at ?? t)} because ${this.lastRefused?.reason}`;
} else if (!this.expecting()) {
why = this.notHeard ? `no events are listened for, so the polls alone watch the forge: ${this.notHeard}` :
"the receiver is not listening yet";
if (this.notHeard && this.notHeardFails) state = "unhealthy";
} else if (this.hearing()) {
why = `the forge's events are heard: ${this.verifiedCount} verified, the last at ${iso(this.last)}`;
} else if (this.last === 0) {
why = `no delivery heard yet; the forge is said silent if the poll finds what no event said ${SILENT_AFTER / 60_000} min after the start`;
} else {
why = `the last delivery was at ${iso(this.last)}; the polls look every 30 s until the next`;
}
return {
healthy: state !== "unhealthy", state, why,
started: iso(this.started),
receiver: this.where ? { listening: true, ...this.where } : { listening: false, why: this.notHeard ?? "not started" },
hook: this.hook,
verified: { count: this.verifiedCount, last: this.lastVerified && { ...this.lastVerified, at: iso(this.lastVerified.at) } },
refused: { count: this.refusedCount, since_verified: refusedSince, last: this.lastRefused && { ...this.lastRefused, at: iso(this.lastRefused.at) } },
polls: { every_seconds: this.every() / 1000, hearing: this.hearing(), found_unsaid: this.missCount,
last_found: this.lastMiss && { ...this.lastMiss, at: iso(this.lastMiss.at) } },
silent: this.silentSince ? { since: iso(this.silentSince) } : null,
};
}
}
/** Every event family the forge (1.27) can send a webhook. */
export const HOOK_EVENTS = ["create", "delete", "fork", "push", "issues", "issue_comment", "pull_request",
"pull_request_review_request", "wiki", "repository", "release", "package", "status", "workflow_run", "workflow_job"];
async function hooksAt(admin: HookAdmin, type: "system" | "default", url: string): Promise<any[]> {
const listed = await admin.call("GET", `/admin/hooks?type=${type}&limit=50`);
if (listed.status !== 200) throw new Error(`the forge's ${type} webhooks could not be listed: ${listed.status}`);
return (Array.isArray(listed.body) ? listed.body : []).filter((h) => h?.config?.url === url);
}
/** Every page of a listing that answers an array (or `{ data: [] }`, the repository search), PAGE at a time. */
const PAGE = 50;
async function everyPage(admin: HookAdmin, path: string, what: string): Promise<any[]> {
const all: any[] = [];
for (let page = 1; ; page++) {
const r = await admin.call("GET", `${path}${path.includes("?") ? "&" : "?"}page=${page}&limit=${PAGE}`);
if (r.status !== 200) throw new Error(`${what} could not be listed: ${r.status}`);
const items: any[] = Array.isArray(r.body) ? r.body : Array.isArray(r.body?.data) ? r.body.data : [];
all.push(...items);
if (items.length < PAGE) return all;
}
}
/** Every repository's own webhooks at the receiver's url: copies of a default webhook made at it. */
async function repoHooksAt(admin: HookAdmin, url: string): Promise<{ repo: string; id: number }[]> {
const found: { repo: string; id: number }[] = [];
for (const repo of await everyPage(admin, "/repos/search?sort=id", "the forge's repositories")) {
const full = String(repo?.full_name ?? "");
if (!full.includes("/")) continue;
const hooks = await everyPage(admin, `/repos/${full}/hooks`, `${full}'s webhooks`);
for (const h of hooks) if (h?.config?.url === url) found.push({ repo: full, id: Number(h.id) });
}
return found;
}
/**
* **One system webhook, for every event, at the receiver** (issue 729), made each start and read back.
*
* The forge (1.27.3, routers/api/v1/utils/hook.go addHook) makes a *default* webhook from POST /admin/hooks unless
* `config.is_system_webhook` is "true": a default webhook fires for no repository and is only copied into repositories
* made after it, and the listing of system webhooks does not name it — so without the flag every start made another
* one and no event came. Its edit (editHook) never changes a webhook's secret nor its kind, so a hook already there is
* not corrected but replaced: the new system webhook is made first, with the secret kept now, so no event falls in a
* gap; then every other hook at the receiver's url is deleted — system, default, and each repository's copy of a
* default one, which would send that repository's events twice. The forge's listing of system webhooks must then name
* exactly one at the url, or this fails, saying so.
*/
export async function ensureSystemHook(admin: HookAdmin, url: string, secret: string): Promise<HookState> {
// The forge has no "every event" word: each family is named, and a family name (issues, pull_request) takes all of it.
const want = { type: "gitea", active: true, events: HOOK_EVENTS, branch_filter: "*",
config: { url, content_type: "json", secret, is_system_webhook: "true" } };
const before = new Set((await hooksAt(admin, "system", url)).map((h) => Number(h.id)));
const made = await admin.call("POST", "/admin/hooks", want);
if (made.status !== 201 && made.status !== 200) throw new Error(`the forge's system webhook could not be made: ${made.status}`);
const fresh = (await hooksAt(admin, "system", url)).filter((h) => !before.has(Number(h.id)));
if (fresh.length !== 1) {
throw new Error(`the forge names ${fresh.length} new system webhooks at ${url} after one was made, not one: its events ` +
`are not known to be sent`);
}
const ours = fresh[0];
const defaults = await hooksAt(admin, "default", url);
const gone = async (path: string): Promise<void> => {
const r = await admin.call("DELETE", path);
if (r.status !== 204 && r.status !== 200 && r.status !== 404) throw new Error(`the forge's webhook ${path} could not be deleted: ${r.status}`);
};
for (const id of before) await gone(`/admin/hooks/${id}`);
for (const h of defaults) await gone(`/admin/hooks/${h.id}`);
const copies = await repoHooksAt(admin, url);
for (const c of copies) await gone(`/repos/${c.repo}/hooks/${c.id}`);
const kept = await hooksAt(admin, "system", url);
if (kept.length !== 1 || Number(kept[0].id) !== Number(ours.id)) {
throw new Error(`the forge names ${kept.length} system webhooks at ${url} once the others were deleted, not the one ` +
`made (${ours.id})`);
}
const removed = { system: before.size, default: defaults.length, repository: copies.length,
repositories: [...new Set(copies.map((c) => c.repo))] };
return { kind: "system", id: Number(ours.id), url, events: Array.isArray(ours.events) ? ours.events : [],
active: !!ours.active, content_type: ours.config?.content_type,
did: removed.system + removed.default + removed.repository > 0 ? "replaced" : "created", removed };
}
/** The `gitea_events` tool: what the module knows of the forge's events, from its Pace. */
export function eventsTool(pace: Pace): { name: string; description: string; input: Record<string, unknown>;
run: (args: Readonly<Record<string, unknown>>) => Promise<unknown> } {
return {
name: "gitea_events",
description: "Whether the forge's events reach the mesh (novox/hq issue 729): where the receiver listens, the " +
"system webhook as ensured at the start (its id, url, events, kind, and what it replaced), how many deliveries " +
"verified and the last one's time and event, how many were refused and the last reason, how often the polls look " +
"and whether events are heard, and how many heads or merges the poll found that no event said. Unhealthy when the " +
"forge has delivered nothing that verified for ten minutes (since the start or since the last delivery) while the " +
"poll found what no event said, when deliveries were refused since the last verified one, or when the receiver " +
"could not listen or the webhook could not be made.",
input: {},
run: async () => pace.report(),
};
}
/**
* **Claim a head for announcing, once** (issue 729): true when `sha` is not already the head recorded for `key`, which
* it then is. The poll and the forge's event both claim before they announce, so whichever comes second finds it taken.
*/
export function claimHead(heads: Record<string, string>, key: string, sha: string): boolean {
if (!sha || heads[key] === sha) return false;
heads[key] = sha;
return true;
}