docker: the container runtime as a module, with its tools in Go
Claims node-container-runtime (ADR 0207). Owns the packages, the socket and a weekly prune of dangling images and unused build cache. Serves 18 tools over every container, marking the mesh's. daemon.json, docker.service and the docker group are left to a proposed change: dnsmasq and zsh declare them today, and the controller refuses a second declaration (README).
This commit is contained in:
@@ -0,0 +1,166 @@
|
||||
# docker
|
||||
|
||||
The container runtime as a module (novox/hq to-be 42 phase 1, item 8; research 027/01–02; ADR 0166,
|
||||
ADR 0207). It claims the node seat `node-container-runtime`. That seat carries no verbs yet: its verbs,
|
||||
and the host creating containers through its holder, wait on ADR 0166's acceptance. Until then the
|
||||
tools below are the module's own.
|
||||
|
||||
## What it declares
|
||||
|
||||
| resource | what | the host's rule |
|
||||
|---|---|---|
|
||||
| `package` | `docker` | installed if absent; never uninstalled when the module goes |
|
||||
| `buildx` | `docker-buildx` | the same. Only the build machine has it today; `docker build` needs it for BuildKit everywhere |
|
||||
| `socket` | `docker.socket` running, enabled at boot | given back as found when the module goes (ADR 0118) |
|
||||
| `prune-service`, `prune-timer` | `/etc/systemd/system/docker-prune.{service,timer}`, written whole | removed with the module |
|
||||
| `prune` | `docker-prune.timer` running, enabled at boot; restarted when either file changes | stopped and disabled with the module (the mesh made the unit) |
|
||||
|
||||
The weekly prune takes **dangling images and build cache unused for a week, and nothing else**. It
|
||||
takes no volume, no container and no image a container uses, so it never touches a container the mesh
|
||||
holds. It runs at idle priority, at a random point in the hour after the weekly mark. A run missed
|
||||
while the machine was off happens at the next boot.
|
||||
|
||||
**Capabilities:** `package-manager`, `service-manager`, `privileged`. It does not declare
|
||||
`container-runtime`: under ADR 0165, which is still proposed, that word means a running daemon, and
|
||||
the module that installs the daemon cannot require it.
|
||||
|
||||
## What it does not declare yet, and why
|
||||
|
||||
Three things this module should own are already declared by other modules on every machine. The
|
||||
controller refuses two modules on one node that declare the same `path`, `unit`, `name` or `package`
|
||||
(`checkResources`, mesh-controller `internal/catalogue/resolve.go`). Declaring any of them here would
|
||||
make the module unassignable everywhere. The refusals were checked against the controller's own
|
||||
check:
|
||||
|
||||
```
|
||||
zsh and docker both declare the name "${machine:account}"
|
||||
dnsmasq and docker both declare the path "/etc/docker/daemon.json"
|
||||
dnsmasq and docker both declare the unit "docker.service"
|
||||
```
|
||||
|
||||
### 1. `/etc/docker/daemon.json` and `docker.service` (issue 190)
|
||||
|
||||
Today the file has three writers. Each writes into it (`into: json`, ADR 0102) and reloads the
|
||||
service:
|
||||
|
||||
- **`dnsmasq`** writes `dns` and `live-restore`, through `dnsmasq.runtime-dns` and `dnsmasq.runtime`.
|
||||
- **The private network**, generated by the controller (`internal/overlay/generator.go`), writes
|
||||
`insecure-registries`. The collision check does not see generated resources.
|
||||
- **Nobody** writes log rotation. One machine has `log-driver` and `log-opts` by hand.
|
||||
|
||||
**The change proposed, in one merge:**
|
||||
|
||||
1. `dnsmasq` drops its `runtime-dns` and `runtime` resources.
|
||||
2. `docker` adds the two resources below:
|
||||
|
||||
```json
|
||||
{"id": "daemon", "type": "file", "path": "/etc/docker/daemon.json", "mode": "0644", "into": "json",
|
||||
"content": "{\"dns\": [\"${machine:address}\"], \"live-restore\": true, \"log-driver\": \"json-file\", \"log-opts\": {\"max-size\": \"100m\", \"max-file\": \"5\"}}\n"},
|
||||
{"id": "runtime", "type": "service", "unit": "docker.service", "state": "running", "boot": "enabled", "reload-on": ["daemon"]}
|
||||
```
|
||||
|
||||
The service is **reloaded, never restarted**: a restart stops every container. The daemon reads
|
||||
`live-restore` on a reload. It reads `dns`, `log-driver` and `log-opts` only at its next start, so
|
||||
they apply then (to containers created afterwards, for the log keys). With `live-restore` on, that
|
||||
start keeps every container running.
|
||||
|
||||
**Why one merge, and only after this module is on every machine:**
|
||||
|
||||
- In one apply, the host first gives back the resources that are no longer declared, then applies
|
||||
the new ones (mesh-host `apply.go`).
|
||||
- `dnsmasq` gives back `dns` and `live-restore` to what they held before it, and `docker` sets them
|
||||
again in the same apply. The daemon is reloaded once, after both steps.
|
||||
- A machine pushed the new `dnsmasq` *without* this module would keep its pre-mesh values for both
|
||||
keys. On one machine that is `live-restore: false`, and the next daemon restart there would stop
|
||||
every container.
|
||||
|
||||
**Later:** the controller hands the registry to this module as a value, and the overlay stops
|
||||
generating its two resources (issue 190, steps 2 and 5). Until then the overlay keeps writing its one
|
||||
key beside this module's. The host merges disjoint keys correctly; the mesh-host `into.go` record is
|
||||
per resource.
|
||||
|
||||
### 2. The operator account's membership of the `docker` group
|
||||
|
||||
The right shape is the host's `user` shape. Its `groups` are additive: the host runs
|
||||
`usermod --append` and never takes a group away.
|
||||
|
||||
```json
|
||||
{"id": "group", "type": "user", "name": "${machine:account}", "groups": ["docker"]}
|
||||
```
|
||||
|
||||
`zsh` already declares a `user` resource for the same account (its login shell). The controller
|
||||
compares `name` across modules, so the two collide.
|
||||
|
||||
**The change proposed (mesh-controller, `checkResources`):** judge a `user` resource by the fields it
|
||||
sets, not by its name:
|
||||
|
||||
- `shell` and `home` stay single-owner;
|
||||
- `groups` may be declared by any number of modules, because the host only adds them.
|
||||
|
||||
Then this module declares the resource above, and no module has to carry another's group.
|
||||
|
||||
Today the operator account is in the group on every machine, by hand. Nothing is lost while it waits.
|
||||
|
||||
## The bootstrap's runtime
|
||||
|
||||
On the machine the mesh was first installed on, the foundation bundle declared `package docker`
|
||||
(`container-runtime`) and `docker.service` running and enabled (`container-runtime-running`). ADR 0207
|
||||
§5 exempts them.
|
||||
|
||||
- The host records them under their bare ids, with origin *carried*. A mesh declaration's orphan pass
|
||||
never sees them (mesh-host `store.go`).
|
||||
- So `docker.package` here is a **second record of the same package**. The apply says "already
|
||||
installed", and neither record ever uninstalls it.
|
||||
- This module does not declare `docker.service` today, so nothing overlaps there. The proposed step
|
||||
1 would add a second record of that unit. Its found state is *running*, because genesis started
|
||||
it, so undeclaring this module would leave the daemon running.
|
||||
|
||||
## Tools
|
||||
|
||||
The tools run as the operator account. If the daemon's socket refuses that account, a call is asked
|
||||
again through `sudo -n` (a process keeps the groups it started with). Every call has a 20 s bound.
|
||||
A failure is an error naming how it failed, never an empty answer.
|
||||
|
||||
**Every container on the machine is in scope.** A container the mesh holds carries the host's label
|
||||
`mesh-host.id` (its value names the assignment), and every answer says `mesh_held`.
|
||||
|
||||
| tool | | what |
|
||||
|---|---|---|
|
||||
| `docker_list` | r | every container: image, state, health, restarts, ports, mounts, compose project, `mesh_held`; filter by owner, state or name |
|
||||
| `docker_inspect` | r | one container whole, **environment values left out** (names kept) |
|
||||
| `docker_logs` | r | the last lines of both streams, merged in order, with timestamps (default 200, at most 2000) |
|
||||
| `docker_stats` | r | CPU, memory, I/O and process count per running container, heaviest first |
|
||||
| `docker_start` / `docker_stop` / `docker_restart` | a | one container. On a mesh-held one, the answer says the host restores its declared state at its next apply |
|
||||
| `docker_top` | r | the processes inside one container |
|
||||
| `docker_images` | r | images, largest first, with the containers using each; `dangling`, `unused` or `used` |
|
||||
| `docker_prune` | a | dangling images and build cache, and stopped containers the mesh does not hold if `containers` is true. **A dry run unless `dry_run` is false. Never a volume** |
|
||||
| `docker_disk_usage` | r | `docker system df -v`: total, active and reclaimable per kind, with the largest of each |
|
||||
| `docker_networks` | r | networks, subnets, and the containers on each |
|
||||
| `docker_volumes` | r | volumes, who mounts each, whether the mesh holds one of them, anonymous or not, and sizes if asked |
|
||||
| `docker_events` | r | the runtime's events over a window ending now (default 60 min, at most 24 h), without exec noise |
|
||||
| `docker_daemon_config` | r | `daemon.json` as on disk, `docker info`'s essentials, and keys the daemon has not taken yet |
|
||||
| `docker_unlabelled` | r | the containers the mesh does not hold: the cleanup list |
|
||||
| `docker_problems` | r | unhealthy, restarting, dead, killed for memory, failed, or restarted five times or more |
|
||||
| `docker_ports` | r | every published port, and the containers on the host's network |
|
||||
|
||||
## Tests
|
||||
|
||||
```
|
||||
go test ./...
|
||||
```
|
||||
|
||||
The tests run against a fake runner and cover:
|
||||
|
||||
- escalation through `sudo -n` on a refused socket, and never as root;
|
||||
- each failure named by its cause;
|
||||
- a name or id never read as an option;
|
||||
- mesh-held marking;
|
||||
- the environment left out of `inspect`;
|
||||
- the restore note on a mesh-held act;
|
||||
- prune being a dry run by default and never reaching a volume, a mesh container or `--volumes`;
|
||||
- the log merge;
|
||||
- size parsing;
|
||||
- what the daemon has not yet taken;
|
||||
- event filtering;
|
||||
- volume ownership;
|
||||
- that the tools served are exactly the manifest's `tools`.
|
||||
Reference in New Issue
Block a user