jochen
24ce702360
A module that provides the artifact store cannot build an artifact: building publishes to the store, so the store would be needed to create itself, and the module check refuses it. The tools become artifact-store-tools, assigned beside the store, and read manifests from the store's files through its container as they already read the listing, so they need no address for the store's door. Named for the artifact store, because "store" alone is the database server (hq glossary).
4.8 KiB
artifact-store-tools
Tools that say what the mesh's artifact store holds, and what the controller's records say of it (novox/hq ADR 0251 §1–3, to-be 51). It declares no resources and changes nothing on its machine.
Why a module beside the store
The store is the distribution module. A module that provides the artifact store cannot also build an
artifact: building publishes to the store, so the store would be needed to create itself, and the
controller's module check refuses that manifest. So the store's tools are a second module, as the
controller's refusal says to do. Assign it to the machine that holds the store: its tools read the
store's files through the store's own container, and anywhere else they answer that the container is not
on that machine.
Tools
| tool | what | |
|---|---|---|
artifact_store_repositories |
r | every repository: tags and what each names, how many manifests (tagged or not), size |
artifact_store_usage |
r | the store's size, the largest repositories, bytes shared between repositories, bytes no manifest marks (what the nightly collector frees next) |
artifact_store_references |
r | what the controller's records say of each manifest, counted and sized by state and repository; each manifest listed when one repository is asked; what the records keep that the store does not hold |
artifact_store_collect |
a | a dry run unless dry_run is false: what the controller would let go of, and the bytes the nightly collector would free then and now. A real run needs why and asks the controller's collect |
How the store is read
The store's door lists repositories and tags, but not a manifest no tag names, and the mesh pins every
machine by digest, so that is most of them. So the bundle reads the store's own files through its own
container (docker exec mesh-registry, busybox find, stat and cat), and only reads them: every
blob with its size, every manifest each repository holds, what each tag names, and each manifest's
content, four hundred to an exec. What it read it keeps (a manifest is named by its content, so it never
changes), and the next call reads only what is new. A manifest that could not be read is counted and
said; what it marks beyond itself is then unknown, so sizes may read low and freed bytes high, and the
answer says so.
docker runs as the tool runner's account; a socket that refuses it is asked again through sudo -n,
never with a prompt, as the container runtime's own tools do.
A manifest marks its own content, its configuration and its layers, and an index marks the manifests it lists. The store's collector removes every blob no manifest marks, so these numbers are its own.
The states of a manifest
| state | what | removable |
|---|---|---|
kept |
a definition names it, or one of the five most recent builds of a module the mesh holds; why says which |
no |
holder-of-kept-archive |
the manifest that keeps a kept archive's blob (hq issue 253) | no |
eligible |
the mesh made it and keeps it for no reason | through the controller's collect |
holder-of-eligible-archive |
the manifest that keeps an eligible archive's blob | with its archive |
let-go-yet-present |
the controller recorded letting go of it, and the store still holds it | no — a finding |
named-document |
a one-layer manifest the controller keeps under a tag (hq to-be 45 §9) | no |
unrecorded |
no record names it | never, by any tool (ADR 0189 §3) |
The records come from the controller's artifacts verb, asked with the references already let go of;
when that answer is too large to carry, it is asked without them and the answer says so.
Collecting
artifact_store_collect never deletes anything itself. The controller decides and records what it
lets go of (ADR 0189 §2), so a real run asks its collect with why and confirm; the controller holds
every kept archive first and records the run as a hand-act. Bytes come back at the store's nightly
collection, which runs with the store held still.
Tests
go test ./...
Against a fake container that prints what the two scripts would: the listing parsed (a repository name
with slashes, a cut listing refused), manifests read with one absent or unreadable said as unread, a
digest never becoming a path outside the blobs, the mark (an index, a shared blob, a blob nothing marks),
every state, what the records keep that the store lacks, bytes freed counting a blob shared with a kept
manifest as kept, a dry run never confirming, a real run without why refused before anything is asked,
the fall-back when the references let go of cannot be carried, escalation through sudo -n, and that the
tools served are exactly the manifest's tools and its invokes exactly the two controller verbs.
Any command other than the two read-only scripts fails the test.