jochen
f2d765320a
The forge's contents and diffpatch endpoints take a branch, never a commit to
build on, so the parent is read from the commit the forge made, and a branch
that moved under the commit is said. Blobs are read at the commit read first,
not at the branch name. Protection rules match as Gitea 1.27 matches them
([...], {a,b}, escapes, a plain name in any case), in the Go bundle and in
guard.ts alike. One commit carries at most 4 MiB per file or patch and 16 MiB
together; code search stops at its deadline and says so. The HTTP client the
two Go bundles copied is one package, internal/forgehttp.
152 lines
11 KiB
Markdown
152 lines
11 KiB
Markdown
# gitea
|
|
|
|
The forge, and the mesh's npm package registry: it claims the `git` and `npm-package-registry` seats.
|
|
Its tools, its events and the provisioner of registry accounts are the TypeScript bundle (`index.ts`,
|
|
`tools/`, `provisioner/`). This README covers the two Go bundles beside it.
|
|
|
|
## Announcing pull requests to the merge check (novox/hq to-be 45 §9, issue 347)
|
|
|
|
Every new head of an open pull request is announced as `pull.updated`, once, and the controller has it checked.
|
|
|
|
- **Which repositories a pass looks at.** Each that moved since the last pass, less a minute's margin (issue 250);
|
|
each holding a pull request updated since then, found by the forge's own search — opening, reopening or
|
|
retargeting a pull request moves its own update time and not its repository's, and a pull request opened more
|
|
than a minute after its push was never announced (issue 347); and every repository on a sweep, every ten
|
|
minutes, or when the search fails.
|
|
- **A pull request nobody checks is said — by the delivery's owner, not here.** `gitea_open_pulls` answers every
|
|
open pull request from the forge itself, with the statuses its base requires and the ones its head has. The
|
|
`mesh-delivery` module asks it every five minutes, and a head on a base requiring `mesh/merge-gate` or
|
|
`mesh/repo-check` with neither for ten minutes since it first saw it so is a delivery held past its bound
|
|
(`delivery.<id>.stalled`, state `unannounced`), which the controller raises like any other (issue 347).
|
|
Each of the head's statuses comes with its state and when it was set, so a head whose check started and never
|
|
answered — a required check pending, or never set beside one that is — is said as a stalled line in the state
|
|
`unanswered` once that has lasted the delivery table's bound for a check asked and not answered, an hour, counted
|
|
from its first sight and again only when a check is asked anew (issue 438).
|
|
- **How to see it work.** A pull request opened more than a minute after its branch was pushed — this fix's own
|
|
was one, and the announcer it fixes never saw it — has `mesh/merge-gate` pending within a pass of the next sweep
|
|
at the latest, and `mesh-controller.conditions` names none for it.
|
|
- **A pull request or issue number that is not one is refused** before the forge is asked: a tool called with a
|
|
misnamed argument sent `pulls/NaN`.
|
|
|
|
## An agent's commit and the reads around it (novox/hq issue 375)
|
|
|
|
An agent that finished a change on a branch had no way to put it on the forge: the module had no commit or
|
|
push tool, `gitea_api` refused the forge's patch endpoint, and its only file write took one whole file
|
|
base64-encoded. The Go bundle `cmd/forge` serves the commit, and the reads an agent otherwise made through
|
|
`gitea_api` as raw JSON, summarised.
|
|
|
|
| tool | | what |
|
|
|---|---|---|
|
|
| `gitea_commit` | a | one commit on a branch of its own, created from `base` (default the default branch) when absent, carrying either `files` (each `{path, content \| content_base64 \| delete: true}`) or `patch` (a unified diff, as `git diff` writes it); answers the commit and its parent |
|
|
| `gitea_commits` | r | a ref's history, newest first, optionally only commits touching a path: sha, author, date, subject |
|
|
| `gitea_compare` | r | what head holds that base does not: its commits and the files they touch |
|
|
| `gitea_pull_reviews` | r | a pull request's reviews — state, the commit reviewed and whether it is the head now, stale, dismissed — with their line comments |
|
|
| `gitea_list_dir` | r | a directory's entries at a ref: name, type, a file's size |
|
|
| `gitea_search_code` | r | the lines of a ref's text files matching a literal string or a regular expression, optionally under a path |
|
|
|
|
**Where a commit may go** is the rule `gitea_api` already holds for a file written (novox/hq ADR 0266):
|
|
never the repository's default branch, never a branch a protection rule names. A rule covers a branch as the
|
|
forge itself decides it (Gitea 1.27, novox/hq issue 539): a plain name covers that branch in any case; a glob
|
|
(`*`, `**`, `?`, `[abc]`, `[!abc]`, `{a,b}`, `\` escaping) covers case-sensitively with `/` as separator; a glob
|
|
the forge cannot compile covers its own text. `cmd/forge/rules.go` and `guard.ts` hold this one rule. The rules are read before anything is sent; rules that cannot be read refuse the
|
|
commit. Nothing about merging changes: a merge still needs an independent review at the head and both
|
|
required checks green, through `gitea_merge_pull_request`.
|
|
|
|
**How the commit is made.** `files` become one call to the forge's multi-file contents endpoint: a path the
|
|
ref holds is updated, one it does not is created, and an update or delete carries the blob read just
|
|
before, so a file someone changed meanwhile is refused by the forge rather than overwritten. The blobs are read at
|
|
the commit the branch (or, for a new branch, its base) was read at, never at its name. A `patch` goes to the
|
|
forge's diff endpoint, which applies it; for a small change to a large file this is far shorter than the
|
|
file. Neither endpoint takes a commit to build on, only a branch (Gitea 1.27), so the parent answered is the
|
|
one the forge says it made the commit on; when the branch moved between the read and the commit, the answer
|
|
says so. One commit carries at most 200 files, 4 MiB per file and 16 MiB together, or a patch of 4 MiB. Either way the commit is the forge's own, made as the forge's admin account, so its sha is not the
|
|
local commit's: the merge check judges the head on the forge. After the write the branch is read back, and a
|
|
branch not at the commit the forge named is a failure, never a success.
|
|
|
|
**Code search** has no endpoint on the forge, so the ref's archive (`/archive/{ref}.tar.gz`, at most 64 MiB)
|
|
is read once and searched in memory; a binary file is passed over, a file over 4 MiB is named under
|
|
`skipped`, and at most `max` matches (default 100) are answered, with `truncated` when more were found. A
|
|
search that reaches the call's deadline stops there and answers what it found, with `stopped_at_deadline`.
|
|
|
|
It reaches the forge through the same HTTP client as the npm-registry bundle (`internal/forgehttp`), and one
|
|
tool call takes at most 50 seconds.
|
|
`go test ./cmd/forge` runs it against a fake forge: the files of one commit created, updated and deleted on
|
|
a new branch from its base; an existing branch committed on; the default branch, a protected branch and
|
|
unreadable rules refused before anything is sent; a patch sent to the forge; a branch read back at another
|
|
commit failing; a branch or base that moved under the commit said, with the parent the forge used; a file
|
|
changed under the commit refused; a commit over its size limits refused; protection globs matched as the
|
|
forge matches them; a code search stopped at its deadline; every malformed ask refused; and each read
|
|
summarised.
|
|
|
|
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
|
|
|
Every publish adds a version to the registry, and until this nothing removed one. The Go bundle
|
|
`cmd/npm-registry` says which versions each package holds, what keeps each one, and — asked with a why —
|
|
deletes the versions nothing keeps.
|
|
|
|
| tool | | what |
|
|
|---|---|---|
|
|
| `npm_packages` | r | every package of the registry's owner, each version newest first with its publish time, its size and why it is kept |
|
|
| `npm_retention` | a | the versions retention would delete, and the bytes. **A dry run unless `dry_run` is false**; a real run needs `why` |
|
|
|
|
**A version is kept when:**
|
|
|
|
- a lockfile on the default branch of any repository on the forge names it — `package-lock.json`
|
|
(every lockfile version), `npm-shrinkwrap.json`, `yarn.lock` (classic and berry) and `pnpm-lock.yaml`
|
|
(by each resolved `name@version` or `name/version`), whatever the version's age;
|
|
- it is the highest published version satisfying a range that a `package.json` there names in its
|
|
dependencies, development, peer or optional dependencies (an `npm:` alias counts for the package it
|
|
stands for; a path, a URL, git or a workspace is not a range on this registry). npm itself installs
|
|
the `latest` dist-tag when it satisfies the range; that version is kept by its dist-tag;
|
|
- a dist-tag names it — and a range that *is* a dist-tag's name keeps that tag's version;
|
|
- it is among the newest `keep` of its package, five by default (the artifact store's five builds,
|
|
ADR 0189 §3), by semantic-version order. **Pre-releases count** among the newest.
|
|
|
|
A version that is not a semantic version is never ordered, so it is kept. A range that cannot be read,
|
|
or a package whose dist-tags cannot be read, keeps **every** version of its package, and the answer says
|
|
why under `keeps_all`.
|
|
|
|
**Nothing is deleted on partial knowledge.** A repository whose files could not be listed, or a
|
|
manifest or lockfile that could not be fetched, is named under `repositories_unread` in every answer,
|
|
and a real run then deletes nothing at all. A file that was fetched but is not readable (broken JSON) is
|
|
named under `files_not_read`, and so is a lockfile entry of a shape the reader does not know (the rest of
|
|
that file is still read); what it would have named is not known, so a real run then deletes nothing at all
|
|
either. Under `node_modules` nothing is read: that is what a lockfile already says. An empty
|
|
repository, or one without its default branch, holds nothing to read and is not a failure.
|
|
|
|
A real run deletes each version through the forge's own interface (`DELETE
|
|
/api/v1/packages/{owner}/npm/{name}/{version}`), says what it deleted and what the forge refused, and a
|
|
version already gone counts as deleted.
|
|
|
|
### How it reaches the forge
|
|
|
|
Over the forge's HTTP interface on the machine (`MESH_GITEA_URL`), as the admin account the vault
|
|
delivered (`MESH_GITEA_ADMIN_USER`, the password in `MESH_GITEA_ADMIN_PASSWORD_FILE` — the same file
|
|
the TypeScript bundle mints its token with), by basic authentication: this bundle mints no token and
|
|
keeps nothing. The password is read per call and never logged, answered or put in an error. The
|
|
registry's owner is `MESH_NPM_OWNER`, the owner in the seat's `npm-path`.
|
|
|
|
Calls to the forge run eight at a time, and one tool call reads for at most 50 seconds, inside a
|
|
module's 60-second ask.
|
|
|
|
### Why the manifest lists no `tools`
|
|
|
|
The TypeScript bundle's tools are served without a `tools` list, and a claim without `serves` offers
|
|
the module's `tools` as the seat's verbs — so a list here would make these two tools verbs of both the
|
|
`git` and the `npm-package-registry` seats. The two names are kept in `ToolNames`, and a test holds them
|
|
to this README and to what the bundle serves.
|
|
|
|
### Tests
|
|
|
|
```
|
|
go test ./...
|
|
```
|
|
|
|
Against a fake forge (`httptest`): what keeps a version (a lockfile whatever its age, the highest
|
|
version satisfying a range, a dist-tag, the newest `keep`); a dry run deleting nothing; a real run
|
|
without why refused; a real run deleting only what nothing keeps; an unread repository stopping a real
|
|
run before anything is deleted; an unreadable range keeping its whole package; a wrong password failing
|
|
without saying the password. And the range matcher against npm's rules — exact, `^`, `~`, `x` and `*`,
|
|
comparisons with partial versions, hyphen ranges, `||`, and the pre-release rule — and each lockfile
|
|
format.
|