Compare commits

...
Author SHA1 Message Date
jochen add923c74a ssh-client: the mesh's region first in ~/.ssh/config, its hosts in config.d, tools in Go
The region at the end let earlier Host lines win over the mesh's (research 027/03). A
roster fact cannot be placed at the start, so the region holds one Include of config.d,
and the hosts are config.d/00-mesh, read first. Eight tools; authorized_keys and
known_hosts stay found until the controller holds those facts.
2026-10-04 12:37:44 +02:00
jochen d4a6008bd6 docker: the container runtime as a module, with its tools in Go
Claims node-container-runtime (ADR 0207). Owns the packages, the socket and a weekly
prune of dangling images and unused build cache. Serves 18 tools over every container,
marking the mesh's. daemon.json, docker.service and the docker group are left to a
proposed change: dnsmasq and zsh declare them today, and the controller refuses a
second declaration (README).
2026-10-04 12:37:44 +02:00
mesh-admin 83a51832d7 Merge pull request 'claude-code writes its managed files from a staged file, not /dev/stdin' (#258) from fix/claude-code-writes-managed-from-a-file into main 2026-10-04 10:01:17 +00:00
mesh-admin 9e63a258d0 Merge pull request 'zsh: keep each PATH directory once' (#260) from fix/zsh-unique-path into main 2026-10-04 09:34:43 +00:00
jochen 328d90fb88 zsh: keep each PATH directory once
Every nested shell, and every sourced file that prepends, added the same
directories again; a workstation's PATH carried each of several entries three
times. typeset -U in the .zshenv block applies to every zsh.
2026-10-04 11:34:36 +02:00
mesh-admin ca5ab288f6 Merge pull request 'zsh: save history and initialise completion' (#259) from fix/zsh-completion-and-history into main 2026-10-04 09:33:28 +00:00
jochen 5fd0f72221 zsh: save history and initialise completion
zsh saves no history by default (SAVEHIST=0) and nothing called compinit, so
every machine had 30 lines of unsaved history and only basic completion.
Found reviewing the shell on its first machine (hq to-be 41).
2026-10-04 11:33:17 +02:00
jochen 5003dc0377 claude-code writes its managed files from a staged file, not /dev/stdin
Node hands a child its input over a socket, which /dev/stdin cannot open
(ENXIO): on the first assignment nothing under /etc/claude-code was written.
2026-10-04 11:31:44 +02:00
mesh-admin 0a78d130e5 Merge pull request 'claude-code watches its MCP servers beside the handshake, and retries' (#257) from fix/claude-code-watches-without-blocking into main 2026-10-04 09:21:26 +00:00
jochen 4295aad88e claude-code watches its MCP servers beside the handshake, and asks again until the state answers (novox/hq ADR 0201)
Awaited at import, a bucket not yet on the bus — or a grant the bus had not
reloaded — answered after the runtime's 10s handshake, and the module was left
unserved on its first assignment. Also cites module state as ADR 0201, as hq
main numbers it (folds #256).
2026-10-04 11:13:33 +02:00
mesh-admin 9dfd3b1105 Merge pull request 'claude-code: the operator's agent, its managed configuration and the licence consumer side (hq design 36, to-be 40 WP2)' (#244) from feat/claude-code-agent into main 2026-10-04 09:01:31 +00:00
mesh-admin 22e8714040 Merge pull request 'The shell and the account's environment as modules: node-env, zsh, powerlevel10k, two plugins, and systemd finished (hq to-be 41 WP3, WP4)' (#255) from feat/the-shell-and-its-environment into main 2026-10-04 08:55:11 +00:00
jochen 11e7ede8a4 Merge remote-tracking branch 'origin/main' into feat/the-shell-and-its-environment 2026-10-04 10:31:03 +02:00
mesh-admin 27315d35cf Merge pull request 'The store does not collect until every controller composes the window (hq ADR 0189)' (#254) from fix/the-store-collects-once-the-window-is-understood into main 2026-10-04 02:26:57 +00:00
jschoubben 525c639041 The store does not collect until every controller composes the window (hq ADR 0189)
mesh-controller#259 fixes while-stopped to name the container as the machine
knows it — `distribution.store`, not `store`. Until that controller is the
one composing, novox refuses its whole declaration and takes nothing at all.

The step comes out; deletion stays on, already applied and harmless on its
own. A collect step without its window would be worse than none: garbage
collection against a live registry can sweep a blob a build is pushing.

Put back once the fixed controller is deployed and stays.
2026-10-04 04:26:38 +02:00
jochen 42c80fa9e1 zsh: no doubled blank line in the block when the first slot is empty 2026-10-04 04:05:30 +02:00
jochen 56e0830700 zsh-autosuggestions, zsh-syntax-highlighting: the plugins as packages and one line each
The distribution packages both, so they are installed as packages rather than cloned or
vendored (novox/hq ADR 0205). Each contributes the line that loads the package's own
copy, from the path the Arch package installs, to a slot of the login shell's block
(ADR 0204). Syntax highlighting goes in last, as its upstream asks.
2026-10-04 04:04:35 +02:00
jochen 0844b35ebb powerlevel10k: the prompt as a pinned archive of the module's own, loaded from a slot
The distribution does not package the theme, and the predecessor cloned whatever
upstream's default branch held the day a hook ran (novox/hq ADR 0205). So upstream's
v1.20.0 release is vendored verbatim, with its licence, and shipped as an archive the
host unpacks under the account's home and checks by digest.

The prompt's configuration is today's ~/.p10k.zsh byte for byte, as a second archive.
Inline, its 86 KB would ride in every declaration and be unreviewable JSON. The zsh code
that loads both is a contribution to the normal slot (ADR 0204). Instant prompt stays off,
as it is today.
2026-10-04 04:04:10 +02:00
jochen 566739e02c zsh: hold the mesh's login-shell seat, source the environment, and leave the rest to slots
The seat is now the mesh's node-login-shell, which a shell module claims rather than
declares (novox/hq ADR 0204), and the environment is one module's that every module
contributes to (ADR 0203). Per hq to-be 41 WP3:

- no seat declaration; the claim is node-login-shell serving execute;
- EDITOR, VISUAL, XDG_CONFIG_HOME and the three PATH entries are an environment
  contribution, not exports in the block;
- a ~/.zshenv block sources ~/.config/mesh/environment.sh, so a script, a login and
  execute all see the environment;
- the ~/.zshrc block goes at the start, so the operator's lines run after it, and holds
  today's shared defaults between the first, normal and last slots. The prompt, the
  plugins and the operator's own lines are no longer in it;
- execute is bounded below the runtime's call limit (20 s default, 25 s at most), kills its
  whole process group on timeout, cuts each stream at 256 KiB and says so, runs in the
  account's home without the mesh's words, with the account's session words. The dead
  runuser branch is gone, because the runtime is the account;
- zsh_config shows both files with their block line counts;
- the README lists the one-off migration (ADR 0182).
2026-10-04 04:02:54 +02:00
jochen 38b56a7877 node-env: the account's environment as one module's two files
Every module contributes variables and PATH entries as facts, and one holder of
node-environment places them (novox/hq ADR 0203, to-be 41 WP3). This is that holder: no
package, no process, no tools — the directories it owns under the home and two files the
controller fills, the POSIX file at the path the seat fixes (~/.config/mesh/environment.sh,
sourced by the login shell) and environment.d's 50-mesh.conf for the account's service
manager and graphical session.
2026-10-04 03:59:50 +02:00
jochen 0596503db5 systemd: act as the runtime's account can, and never read a failure as an answer
The node tools runtime runs as the operator account, not root, and gives its bundles no
session words (novox/hq ADR 0175, 0188, 0193). So, per hq to-be 41 WP4:

- system-scope start/stop/restart/enable/disable go through sudo -n when not root, as the
  packet filter and intrusion prevention do, and a refusal is named by how it failed;
- user scope is plain --user with XDG_RUNTIME_DIR and the session bus of /run/user/<uid>;
  the dead --machine branches are gone;
- a failed systemctl or journalctl is an error, and an unreachable user manager is said
  even when systemctl exits 0; systemd_failed reports it beside the other manager's answer
  instead of claiming nothing failed;
- status says whether the mesh declares the unit: its loaded unit file begins with the
  header the host writes for a module's process. Only such a unit carries the restore note;
- the package resource goes: the service manager is always present, and it collided with
  systemd-networkd's identical declaration;
- calls are bounded below the runtime's call limit, a unit name is never an option, and
  the runner is injected so the tests use a fake one.
2026-10-04 03:58:19 +02:00
jochen 3668b02b94 claude-code keeps its MCP servers in state, not events (novox/hq ADR 0202)
One key per registration in the module's servers bucket — all.<server> or
<node>.<server> — watched by every node, so a node assigned after a
registration takes it at start, which the mcp.registered event could not do.
Also narrows apply()'s refusal by hand: the builder compiles without strict,
where the discriminated union does not narrow and the build failed.
2026-10-04 03:48:41 +02:00
mesh-admin c0159ca0a1 Merge pull request 'Group 8: minio declares the bucket it derives (hq ADR 0201), and the store collects nightly (hq ADR 0189)' (#229) from feat/the-store-keeps-what-the-records-name into main 2026-10-04 01:47:48 +00:00
jschoubben 159ed53103 Rebased onto main: ADR 0188 renumbered to 0201, and minio takes the sdk at 0.1.7
The bundles refactor took ADR 0188 on main, so minio's comments cite 0201.
The sdk is 0.1.7 after the same rebase, and minio needs the `derived` field
it carries.
2026-10-04 02:45:13 +02:00
jschoubben 723e676b75 The store enables deletion and collects nightly (hq ADR 0189)
REGISTRY_STORAGE_DELETE_ENABLED on the server — the door already accepts a
push — and a scheduled step running the registry's own collector over the
volume at 03:30 with the server held still. Plain garbage-collect: what the
mesh keeps is still a manifest, so --delete-untagged is not needed and would
delete images machines are running.
2026-10-04 02:34:06 +02:00
jschoubben 661114370f minio declares the bucket it derives; its consumers stop transcribing it (hq ADR 0188)
serves.s3-bucket.bucket is ${consumer:as:dns}; the provisioner uses what it
is given. nextcloud, invoicing and photos ask for ${bound:s3-bucket:bucket}
instead of naming mesh-novox-* literals, which also named this node.
bucketFor and the long-dead accessKeyFor are gone.
2026-10-04 02:34:06 +02:00
jochen a1d7b9ad5a systemd: the service manager as a module — holds node-service-manager and answers for the units in both scopes
The holder of the seat the controller seeds under novox/hq ADR 0177. Eight
verbs under the seat's name — units, status, start, stop, restart, enable,
disable, journal — each taking an optional scope, "system" by default or
"user" for the operator account's own manager, reached as
`systemctl --user --machine=<account>@` when the runtime is not that account.
One tool of its own, systemd_failed, for every failed unit in both scopes.
A package, a claim and a bundle; no container, no process: served by the node
tools runtime (ADR 0175) once it exists. `module check` passes against a
controller that carries the seat; the tools type-check against the SDK.
2026-10-04 02:32:34 +02:00
jochen 5548b0f4e9 zsh: the shell as a module — package, the mesh's ~/.zshrc block, the login-shell seat and execute
The first module of the operator's environment (novox/hq to-be 37 §1, ADR 0173,
0176). A package, the mesh's default configuration as a block inside the
account's ~/.zshrc so the operator's own lines around it survive every push
(ADR 0174 as the host's `into: block` realises it), a `user` shape that makes
zsh the account's login shell, the `login-shell` seat declared with its one
verb, and a tools bundle: `execute` under the seat's name, `zsh_config` under
the module's. No container, no process: the tools are served by the node tools
runtime (ADR 0175), which does not exist yet — the bundle builds and the
manifest registers ahead of it. `module check` passes; the tools type-check
against the SDK.

Two things the manifest cannot yet say, left for the controller: the `user`
shape applies wherever the module is assigned, not only where it holds the
seat; and the runtime learns the account from MESH_OPERATOR_ACCOUNT, which
nothing sets yet.
2026-10-04 02:32:34 +02:00
jochen 295cc59e1e claude-code: declare using the licence manager's seat when that module exists; until then the mesh refuses a seat no module declares 2026-10-04 02:23:34 +02:00
jochen 6a7e4ebd5e claude-code over NATS: licence events, a token by request, a login pushed to the manager, MCP servers registered per node or mesh-wide
Events carry what happened and no secret; tokens travel on requests (design 32 §10). The manager's
licence.rotated/switched events make the module ask anthropic-licence-manager.current; at start it
asks once to catch up. A refresh token appearing in the credentials file is a login: it is pushed to
the manager's adopt at once, sealed to the manager's key — the one time a refresh token travels. A
switch replaces the old licence's grant whole, removes the API key and its helper, and rewrites
oauthAccount in ~/.claude.json. New tools register and unregister MCP servers on this node, or with
nodes: all / a list via an mcp.registered event every node consumes; called for one node, the
answer names the other nodes running claude-code. 26 tests.
2026-10-04 02:23:23 +02:00
mesh-admin 9e8146192c Merge pull request 'mssql: give TLS a host name when the server is an address' (#252) from fix/mssql-tls-names-the-host into main 2026-10-03 23:31:40 +00:00
jochen 6171d747db mssql: give TLS a host name when the server is an address
Node 25 refuses an IP address as the TLS server name, and the module reaches its server on
loopback, so every connection failed on the live machines. The certificate is trusted either way.
2026-10-04 01:31:31 +02:00
mesh-admin 84609c0373 Merge pull request 'The last three: mesh-catalog, mongodb and mssql code moves into bundles the node's runtime serves (hq ADR 0198, to-be 38 WP4c)' (#250) from feat/0198-the-last-three-module-code-moves into main 2026-10-03 23:28:58 +00:00
mesh-admin 28d5e7f939 Merge pull request 'audit-logger: the test subscribes as the module does' (#251) from fix/audit-logger-test-hears-every-event into main 2026-10-03 23:21:52 +00:00
jochen 4128380a3d audit-logger: its test subscribes as the module does and expects local event names
The test subscribed '**', which its in-memory broker never matched, while the module subscribes
'#'; and it still expected the module-qualified type from before event names became local.
2026-10-04 01:21:40 +02:00
jochen cf57d3fd8f mssql: its handlers, tools and provisioner run in the node's runtime, through the driver in its bundle (hq ADR 0198)
The mesh-mssql container goes with its Dockerfile (and the sqlcmd it fetched), build bases and bus credential. The client speaks TDS through the mssql driver its package.json names, inlined into the bundle by the builder (ADR 0198 §4): one session per call as one sqlcmd invocation was, FOR JSON rendering rows exactly as before, the consumer's password checked as a bound parameter. A caller's statement still runs only as the reader login (issue 193); the one-line rule and -x guarded against sqlcmd's own commands and variable substitution, which no longer stand between the caller and the server. The server is reached on loopback at the port the machine published (${port:1433}). The reader test drives a fake session in place of a fake sqlcmd.
2026-10-04 01:17:41 +02:00
jochen da8a46cfe8 mongodb: its handlers, tools and provisioner run in the node's runtime, through the driver in its bundle (hq ADR 0198)
The mesh-mongodb container goes with its Dockerfile, build bases and bus credential. Its client shelled out to mongosh, which no machine's system carries, so it now speaks to the server through the official mongodb driver its package.json names, inlined into the bundle by the builder (ADR 0198 §4); the tools answer exactly as before (relaxed Extended JSON). The server is reached on loopback at the port the machine published (${port:27017}). The root secret was owned by the mongo image's user (secrets-owner 999:999), which the runtime's account cannot read; the module's own copy is now the runtime's, and the server is given its own 999-owned copy rendered from the same secret.
2026-10-04 01:17:41 +02:00
jochen ed50130a6a mesh-catalog: its consumer and tools run in the node's runtime, and its preparation is a run-once process (hq ADR 0198)
The mesh-catalog container goes with its Dockerfile, build bases, bus credential and mesh-state directory. Its words are the database URL file where the mesh writes it. `pg` is a dependency in its package.json, which the builder now installs and inlines into the bundle (mesh-controller: a TypeScript bundle installs its module's own packages). `prepares: true` needs a container running the module's own artifact, so it becomes what ADR 0198 §3 says it is: prepare/index.js run by node as a run-once process, with the same words and no bus, before the runtime is started with the version that needs it, and again when the database URL changes.
2026-10-04 01:17:41 +02:00
mesh-admin bd2123166f Merge pull request 'Waves 2-3: nine modules' code moves into bundles the node's runtime serves (hq ADR 0198, to-be 38 WP4c)' (#248) from feat/0198-waves-2-3-module-code-moves into main 2026-10-03 23:01:18 +00:00
mesh-admin d9db931bd0 Merge pull request 'mosquitto: run mosquitto_ctrl inside the broker's container' (#249) from fix/mosquitto-ctrl-from-its-container into main 2026-10-03 23:01:03 +00:00
jochen 69f2efb591 mosquitto: run mosquitto_ctrl inside the broker's container
The module's code moved out of its container and took mosquitto_ctrl from a host package. A
machine whose package index is stale cannot install it (hq issue 205), so the tools failed. The
broker's own image carries the tool at the broker's version: the tools exec into the running
broker, and the bootstrap seeds from a throwaway container of the same image.
2026-10-04 01:00:51 +02:00
jochen 03e729d103 claude-code owns /etc/claude-code and ~/.claude as declared directories
So the controller's ownership check refuses a second module owning either. ~/.claude is the
operator's at 0700 (it was 0755 on the workstations); of what is inside, the module owns only what it
writes, and the host keeps a directory that is not empty when the module goes (hq ADR 0182).
2026-10-03 23:49:16 +02:00
jochen eab335b755 claude-code: launched over stdio (ADR 0193), the console's five tools in its instructions (ADR 0195)
Every bundle is now a child speaking MCP over stdio, so stdout is the channel: the module logs on
stderr. The managed CLAUDE.md teaches mesh_search, mesh_describe, mesh_call, mesh_overview and
mesh_machine with addresses (<seat>.<verb>, <node>/<module>.<tool>) instead of flat tool names.
A hand-over is applied whatever the trailing render says; a failed render is reported beside it.
Proven over stdio as the runtime drives it: five tools listed, a key made on first use, a sealed
switch writing an access-token-only 0600 credentials file that keeps unknown keys.
2026-10-03 23:41:01 +02:00
jochen f42b58f789 claude-code: the manifest, the managed directory and the tools (hq design 36, to-be 40 WP2)
The module owns /etc/claude-code: managed-mcp.json lists the console as `mesh` over HTTP on
loopback plus the servers in its mcp_servers setting (exclusive, by the operator's choice — the
https rule of managedMcpServers refuses a loopback console); managed-settings.json carries the
attribution convention, keeps claude.ai connectors, and adds the key-helper only for an API-key
licence; CLAUDE.md says how a session here works. Rendered whenever the runtime collects the
tools, written only on change, through the operator account's sudo. Under the home, only the
credentials file, only on a hand-over. Nothing declared under a home or /etc; the console's
port comes from node-tools' mcp-endpoint (mesh-tools #34).
2026-10-03 23:40:21 +02:00
jochen 5737752744 claude-code: the sealed hand-over, the credentials write with the lineage rule, the identity read (hq to-be 40 WP2, in progress)
The parts of the agent module that hold whichever way the console is registered: X25519 +
HKDF + AES-GCM from Node's own library so the bundle carries no dependency; the predecessor's
lineage rule (rotation only if newer, a re-issue adopted, a switch regardless) with its
incidents as tests; an atomic 0600 write that strips any refresh token and keeps keys it does not
know; the account read from the agent's own state file. Manifest and renderer follow.
2026-10-03 23:40:21 +02:00
173 changed files with 42509 additions and 561 deletions
+4 -2
View File
@@ -15,7 +15,7 @@ test("audit-logger records every event to the trail as one line each", async ()
const path = join(dir, "audit.log");
// The audit-logger's whole behaviour: consume everything, record it.
await on("**", async (event) => record(event, path));
await on("#", async (event) => record(event, path)); // the pattern index.ts subscribes
process.env.MESH_MODULE = "umami";
process.env.MESH_NODE = "anchor";
@@ -24,7 +24,9 @@ test("audit-logger records every event to the trail as one line each", async ()
const lines = (await readFile(path, "utf8")).trim().split("\n").map((l) => JSON.parse(l));
assert.equal(lines.length, 2);
assert.deepEqual(lines.map((l) => l.type), ["umami.site.created", "node.anchor.joined"]);
// A module names its events locally (design 29); the module is the `source`, which together with
// the type says whose event it was. This broker does no namespacing, so the type is as emitted.
assert.deepEqual(lines.map((l) => l.type), ["site.created", "node.anchor.joined"]);
assert.equal(lines[0].source, "umami");
assert.equal(lines[0].node, "anchor");
assert.equal(lines[0].body.domain, "my-app");
+74
View File
@@ -0,0 +1,74 @@
# claude-code
The operator's agent on a machine (novox/hq design 36): its package, its machine-wide managed
configuration, and the consumer side of the Anthropic licence manager (design 39, ADR 0183).
## What it owns
Two directories, declared, so the mesh refuses a second module owning either:
- `/etc/claude-code`, the agent's machine-wide managed directory, root's, `0755`.
- `~/.claude` under the operator account's home, the operator's, `0700`. The module owns the directory —
that it exists, who owns it, its mode — and of what is inside only what it writes. Everything else
in it (memory, history, projects, local settings, a person's own rules and skills) is the person's
and is never read or written (hq ADR 0182). Unassigned, the module leaves the directory: the host
removes a directory only when it is empty.
## What it writes
Under the agent's managed directory, `/etc/claude-code`, owned whole by this module and rewritten
whenever the node's tool runtime collects the module's tools:
| file | holds |
|---|---|
| `managed-mcp.json` | the tool servers every session loads: the mesh's console as `mesh`, and the servers set in this module's `mcp_servers` setting. **Exclusive**: a server not listed here does not load — not one added with `claude mcp add`, not a project's `.mcp.json`, not a plugin's |
| `managed-settings.json` | the repositories' attribution convention, the claude.ai connectors kept beside the managed servers, and the key-helper while the node holds an API-key licence |
| `CLAUDE.md` | how a session on this mesh works, this node's name and role, the conventions |
Under the operator's home, only `~/.claude/.credentials.json`, and only when the licence manager hands
this node a subscription token. Nothing else under the home is read or written.
## Over NATS
Everything between this module and the rest of the mesh is NATS, in three kinds: an **event** says that
something happened and carries no secret, because a stream keeps it; a **request** carries a token,
because nothing keeps it (hq design 32 §10); and **state** is the current value of something every node
must see, a node that joins later included — kept, so it carries no secret either (hq ADR 0201).
| what | how |
|---|---|
| the licence manager rotated a licence, or switched this node | its `licence.rotated` / `licence.switched` event; this module then asks `anthropic-licence-manager.current` for its token, sealed to the key it sends |
| this node starts | it asks `current` once, so a node that was off catches up |
| a person ran `/login` here | the credentials file gains a refresh token this module never writes; it asks `anthropic-licence-manager.adopt` at once with the grant sealed to the manager's key — the one time a refresh token travels, because the login made the manager's stale |
| an MCP server registered through this module | a key in the module's `servers` state — `all.<server>` for every node, `<node>.<server>` for one; every node watches it and renders what applies to it, a node's own entry over the one for every node. A node that joins later, or was off, reads the whole current set at start; unregistering is a delete. An entry with a secret in its `env` or `headers` is refused by the runtime |
## Tools
`claude_code_status`, `claude_code_render`, `claude_code_pull`, `claude_code_mcp_list`,
`claude_code_mcp_register` (this node by default; `nodes: "all"` or a list for more — called for this
node alone, its answer names the other nodes running claude-code), `claude_code_mcp_unregister`.
## Settings
Per node or for the whole mesh, through `mesh-controller.settings module=claude-code`:
- `role` — what this node is, in a few words; shown to every session.
- `mcp_servers` — extra tool servers, set by the operator for the mesh or a node, beside the ones
registered through the tools; keyed by name, in the vendor's `.mcp.json` entry shape
(`{"type":"http","url":…}` or `{"type":"stdio","command":…,"args":[…]}`). The name `mesh` is the
module's own and cannot be set. Put a person's own servers here, or they stop loading.
## On a machine that carried the predecessor
Remove these by hand, once; the mesh removes nothing it did not make (ADR 0182):
- `~/.claude/CLAUDE.md`
- `~/.claude/rules/00-hal-mesh.md`, `~/.claude/rules/conventions.md`
- `~/.claude/skills/cleanup/`, `~/.claude/skills/hal-switch-license/`
- the hand-made console entry in `~/.claude.json` under `mcpServers` — it is ignored now anyway
## Escalation
Writing `/etc/claude-code` needs root. The runtime runs as the operator account, and the module uses
that account's passwordless `sudo`; on a machine without it, `claude_code_render` says so and nothing
is written.
+112
View File
@@ -0,0 +1,112 @@
// The agent's credentials file, and whether an offered grant may replace what it holds (novox/hq
// ADR 0183, design 36 §5). Pure where it decides, so the rules are tested without a file.
//
// The file is the vendor's: `{ claudeAiOauth: { accessToken, expiresAt, refreshTokenExpiresAt?,
// scopes?, subscriptionType?, rateLimitTier? }, ... }`. A node never holds a refresh token, so the
// one this module writes never carries one, and a full grant a login left behind is stripped the
// moment the manager hands the node its own.
//
// The lineage rule is the predecessor's, with the incidents that earned it: a rotation of the same
// licence is applied only if newer; a grant re-issued by a login is adopted whatever its expiry; a
// switch to another licence is applied regardless, because across licences the expiries are
// unrelated numbers.
import { readFileSync, renameSync, writeFileSync, mkdirSync } from "node:fs";
import { dirname } from "node:path";
export interface Grant {
readonly accessToken: string;
readonly expiresAt: number;
readonly refreshTokenExpiresAt?: number | null;
readonly scopes?: readonly string[] | null;
readonly subscriptionType?: string | null;
readonly rateLimitTier?: string | null;
}
export type ApplySource = "rotation" | "switch";
export type ApplyDecision =
| { apply: true; reissued?: boolean }
| { apply: false; reason: "already-current" }
| { apply: false; reason: "not-newer"; localExpiresAt: number };
/** Two refresh-token expiries within a day are one lineage; a login starts a fresh window weeks away. */
export const GENERATION_TOLERANCE_MS = 24 * 60 * 60 * 1000;
export function sameGeneration(a?: number | null, b?: number | null): boolean {
if (a == null || b == null) return true;
return Math.abs(Number(a) - Number(b)) <= GENERATION_TOLERANCE_MS;
}
export function decideApply(local: Grant | null | undefined, offered: Grant, source: ApplySource): ApplyDecision {
if (!local?.accessToken) return { apply: true };
if (local.accessToken === offered.accessToken) return { apply: false, reason: "already-current" };
const reissued = !sameGeneration(local.refreshTokenExpiresAt, offered.refreshTokenExpiresAt);
if (source === "rotation" && !reissued && Number(local.expiresAt) >= Number(offered.expiresAt)) {
return { apply: false, reason: "not-newer", localExpiresAt: Number(local.expiresAt) };
}
return reissued ? { apply: true, reissued: true } : { apply: true };
}
type Oauth = Record<string, unknown> & { accessToken?: string; refreshToken?: string; expiresAt?: number };
type Credentials = Record<string, unknown> & { claudeAiOauth?: Oauth };
export function readCredentials(path: string): Credentials | null {
try {
const parsed = JSON.parse(readFileSync(path, "utf8")) as Credentials;
return parsed && typeof parsed === "object" ? parsed : null;
} catch {
return null;
}
}
/** The grant the file holds, or null. */
export function grantOf(creds: Credentials | null): Grant | null {
const o = creds?.claudeAiOauth;
if (!o?.accessToken) return null;
return {
accessToken: o.accessToken,
expiresAt: Number(o.expiresAt ?? 0),
refreshTokenExpiresAt: o.refreshTokenExpiresAt == null ? null : Number(o.refreshTokenExpiresAt),
};
}
/** Does the file hold a full grant — a refresh token this module never writes, so a person's login? */
export function holdsLogin(creds: Credentials | null): boolean {
return typeof creds?.claudeAiOauth?.refreshToken === "string" && creds.claudeAiOauth.refreshToken.length > 0;
}
/**
* The handed grant laid over what is there — a rotation of the licence the node already holds — or,
* for a switch, in place of it: the old licence's grant goes whole, scopes and subscription included,
* and only keys outside the grant (another kind of credential the vendor keeps in the file) stay.
* Either way, no refresh token survives.
*/
export function replacedBy(local: Credentials | null, grant: Grant): Credentials {
const next: Credentials = { ...(local ?? {}) };
delete next.claudeAiOauth;
return withGrant(next, grant);
}
/** Overlay the handed grant on what is there, and delete any refresh token. */
export function withGrant(local: Credentials | null, grant: Grant): Credentials {
const next: Credentials = { ...(local ?? {}) };
const oauth: Oauth = { ...(local?.claudeAiOauth ?? {}) };
oauth.accessToken = grant.accessToken;
oauth.expiresAt = grant.expiresAt;
for (const k of ["refreshTokenExpiresAt", "scopes", "subscriptionType", "rateLimitTier"] as const) {
const v = grant[k];
if (v != null) oauth[k] = v as unknown;
}
delete oauth.refreshToken;
next.claudeAiOauth = oauth;
return next;
}
/** Write atomically at 0600: a partial credentials file must never be read as a whole one. */
export function writeCredentials(path: string, creds: Credentials): void {
mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
const tmp = `${path}.mesh-tmp`;
writeFileSync(tmp, JSON.stringify(creds, null, 2) + "\n", { mode: 0o600 });
renameSync(tmp, path);
}
+50
View File
@@ -0,0 +1,50 @@
// Which account the agent is logged in as (novox/hq ADR 0183): not in the token, but in the agent's
// own state file beside the home, `~/.claude.json` → `oauthAccount`. Read to attribute a login; written,
// three keys and nothing else, when a licence is switched, so the file Claude Code shows the account from
// names the account whose token it now holds (as the predecessor learned: two files that disagree make
// a later login look like the wrong account).
import { readFileSync, renameSync, writeFileSync } from "node:fs";
export interface Identity {
readonly accountUuid: string;
readonly emailAddress?: string;
readonly organizationUuid?: string;
}
export function readIdentity(stateFile: string): Identity | null {
try {
const raw = JSON.parse(readFileSync(stateFile, "utf8")) as { oauthAccount?: Record<string, unknown> };
const a = raw.oauthAccount;
if (!a || typeof a.accountUuid !== "string") return null;
return {
accountUuid: a.accountUuid,
emailAddress: typeof a.emailAddress === "string" ? a.emailAddress : undefined,
organizationUuid: typeof a.organizationUuid === "string" ? a.organizationUuid : undefined,
};
} catch {
return null;
}
}
/**
* Point the state file's account at `id`, keeping every other key as found. Returns whether the file
* changed; a file that cannot be read as an object is left alone rather than replaced.
*/
export function writeIdentity(stateFile: string, id: Identity): boolean {
let raw: Record<string, unknown>;
try {
raw = JSON.parse(readFileSync(stateFile, "utf8")) as Record<string, unknown>;
if (!raw || typeof raw !== "object") return false;
} catch {
raw = {};
}
const current = (raw.oauthAccount ?? {}) as Record<string, unknown>;
if (current.accountUuid === id.accountUuid && current.emailAddress === id.emailAddress
&& current.organizationUuid === id.organizationUuid) return false;
raw.oauthAccount = { ...current, accountUuid: id.accountUuid, emailAddress: id.emailAddress, organizationUuid: id.organizationUuid };
const tmp = `${stateFile}.mesh-tmp`;
writeFileSync(tmp, JSON.stringify(raw, null, 2), { mode: 0o600 });
renameSync(tmp, stateFile);
return true;
}
+90
View File
@@ -0,0 +1,90 @@
{
"module": "claude-code",
"version": "1",
"slug": "agent",
"capabilities": [
"package-manager"
],
"requires": [
"mcp-endpoint"
],
"binds": {
"mcp-endpoint": "${dir:state}/mcp-endpoint.json"
},
"consumes": [
"claude-licence-manager.licence.rotated",
"claude-licence-manager.licence.switched"
],
"state": [
"servers"
],
"tools": [
"claude_code_status",
"claude_code_render",
"claude_code_pull",
"claude_code_mcp_list",
"claude_code_mcp_register",
"claude_code_mcp_unregister"
],
"resources": [
{
"id": "package",
"type": "package",
"package": "claude-code"
},
{
"id": "managed",
"type": "directory",
"path": "/etc/claude-code",
"mode": "0755"
},
{
"id": "agent-home",
"type": "directory",
"path": "${machine:account-home}/.claude",
"mode": "0700",
"owner": "${machine:account}"
},
{
"id": "state",
"type": "directory",
"mode": "0700",
"owner": "${machine:account}",
"place": "."
},
{
"id": "facts",
"type": "file",
"path": "${dir:state}/facts.json",
"mode": "0600",
"owner": "${machine:account}",
"content": "{\n \"node\": \"${machine:name}\",\n \"console\": \"http://127.0.0.1:${bound:mcp-endpoint:port}/mcp\"\n}\n"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"mode": "0600",
"owner": "${machine:account}",
"merge": "json",
"content": "{\n \"role\": \"\",\n \"mcp_servers\": {}\n}\n"
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"tools/index.js"
],
"env": {
"MESH_CLAUDE_CODE_STATE": "${dir:state}",
"MESH_CLAUDE_CODE_FACTS": "${dir:state}/facts.json",
"MESH_CLAUDE_CODE_SETTINGS": "${dir:state}/settings.json"
}
}
]
}
}
+271
View File
@@ -0,0 +1,271 @@
// What claude-code does on a node, written against two things it is handed — a way to ask a tool on the
// bus and a way to emit an event — so every path is tested without a bus (novox/hq design 36 §4–§5,
// ADR 0183, ADR 0198).
//
// **Over NATS, in two kinds** (design 32 §10): an event says that something happened and carries no
// secret, because a stream keeps it; a token travels on a request, which nothing keeps. So:
// - the licence manager's `licence.rotated` and `licence.switched` events tell this module to ask the
// seat for its current token, sealed to the key it sends with the request;
// - a login a person made here — a refresh token this module never writes — is offered to the seat at
// once, sealed to the seat's key: the one moment a refresh token travels, because the login made the
// manager's stale;
// - an MCP server registered through this module is **state, not an event** (novox/hq ADR 0201): one
// key per server in the module's `servers` bucket — `all.<server>` for every node, `<node>.<server>`
// for one — which every node watches. A node that joins later, or was off, reads the whole current set
// at start; unregistering is a delete. A secret never goes in an entry: the runtime refuses one.
import { chmodSync, existsSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { render, entryProblem, MANAGED_DIR, type Binding, type Facts, type Settings, type Servers } from "./render.js";
import { generateKeyPair, open, seal, type SealedBox } from "./seal.js";
import { decideApply, grantOf, holdsLogin, readCredentials, replacedBy, withGrant, writeCredentials, type Grant } from "./grant.js";
import { readIdentity, writeIdentity, type Identity } from "./identity.js";
export const SEAT = "anthropic-licence-manager";
export interface Paths {
state: string;
facts: string;
settings: string;
home: string;
node: string;
}
/** A tool on the bus: its address and arguments in, its JSON answer out. */
export type Ask = (address: string, args: Record<string, unknown>) => Promise<unknown>;
/** An event of this module's, by its local name. */
export type Emit = (type: string, body: unknown) => Promise<void>;
/** Write one managed file; answers what happened. */
export type WriteManaged = (name: string, content: string) => string;
export const readJson = <T>(p: string, fallback: T): T => {
try {
return JSON.parse(readFileSync(p, "utf8")) as T;
} catch {
return fallback;
}
};
const credentialsPath = (p: Paths) => join(p.home, ".claude", ".credentials.json");
const accountPath = (p: Paths) => join(p.home, ".claude.json");
const bindingPath = (p: Paths) => join(p.state, "licence.json");
const apiKeyPath = (p: Paths) => join(p.state, "api-key");
export const helperPath = (p: Paths) => join(p.state, "api-key-helper");
const keyPath = (p: Paths) => join(p.state, "key.pem");
const pubPath = (p: Paths) => join(p.state, "key.pub.pem");
const registryPath = (p: Paths) => join(p.state, "mcp-servers.json");
export function keypair(p: Paths): { publicKey: string; privateKey: string } {
if (!existsSync(keyPath(p))) {
const k = generateKeyPair();
writeFileSync(keyPath(p), k.privateKey, { mode: 0o600 });
writeFileSync(pubPath(p), k.publicKey, { mode: 0o644 });
}
return { privateKey: readFileSync(keyPath(p), "utf8"), publicKey: readFileSync(pubPath(p), "utf8") };
}
export function registered(p: Paths): Servers {
return readJson<Servers>(registryPath(p), {});
}
export function renderNow(p: Paths, write: WriteManaged): string[] {
const facts = readJson<Facts | null>(p.facts, null);
if (!facts?.console) throw new Error(`the mesh has not rendered ${p.facts} yet; nothing to write`);
const files = render(facts, readJson<Settings>(p.settings, {}), readJson<Binding | null>(bindingPath(p), null),
helperPath(p), registered(p));
return Object.entries(files).map(([name, content]) => write(name, content));
}
// ---- the licence ----------------------------------------------------------------------------------
/** What the seat answers to `current`: the licence this node is bound to and its token, sealed. */
export interface Current {
licence: string;
kind: "subscription" | "api-key";
sealed: SealedBox;
identity?: Identity | null;
}
/** Ask the seat for this node's current token and apply it. */
export async function pull(p: Paths, ask: Ask, write: WriteManaged): Promise<Record<string, unknown>> {
const answer = (await ask(`${SEAT}.current`, { node: p.node, public_key: keypair(p).publicKey })) as Current | null;
if (!answer?.sealed) return { applied: false, reason: "the seat holds no licence for this node" };
return apply(p, answer, write);
}
/** Apply what the seat handed over. A switch replaces the grant whole and cleans up after the old licence. */
export function apply(p: Paths, handed: Current, write: WriteManaged): Record<string, unknown> {
const plain = open(handed.sealed, keypair(p).privateKey);
const previous = readJson<Binding | null>(bindingPath(p), null);
const switched = previous?.licence !== handed.licence;
let outcome: Record<string, unknown> = { applied: true, licence: handed.licence, kind: handed.kind, switched };
if (handed.kind === "api-key") {
writeFileSync(apiKeyPath(p), plain.trim() + "\n", { mode: 0o600 });
writeFileSync(helperPath(p), `#!/bin/sh\nexec cat '${apiKeyPath(p)}'\n`, { mode: 0o700 });
chmodSync(helperPath(p), 0o700);
} else {
const grant = JSON.parse(plain) as Grant;
const local = readCredentials(credentialsPath(p));
const d = decideApply(grantOf(local), grant, switched ? "switch" : "rotation");
if (d.apply) writeCredentials(credentialsPath(p), switched ? replacedBy(local, grant) : withGrant(local, grant));
else outcome = { applied: false, licence: handed.licence, reason: "reason" in d ? d.reason : undefined }; // narrowed by hand: the build compiles without strict
// Away from the API key: it goes, with its helper.
rmSync(apiKeyPath(p), { force: true });
rmSync(helperPath(p), { force: true });
}
if (switched && handed.identity?.accountUuid) {
outcome.account = writeIdentity(accountPath(p), handed.identity) ? "updated" : "unchanged";
}
writeFileSync(bindingPath(p), JSON.stringify({ licence: handed.licence, kind: handed.kind }) + "\n", { mode: 0o600 });
try {
outcome.rendered = renderNow(p, write); // the key-helper comes or goes with the licence's kind
} catch (err) {
outcome.rendered = { failed: err instanceof Error ? err.message : String(err) };
}
return outcome;
}
/** A licence event from the manager: is it for this node? */
export function concerns(p: Paths, type: string, body: { licence?: string; node?: string }): boolean {
if (type.endsWith("licence.switched")) return body.node === p.node;
if (type.endsWith("licence.rotated")) return body.licence === readJson<Binding | null>(bindingPath(p), null)?.licence;
return false;
}
/** A refresh token in the credentials file is a login: this module never writes one. Offer it to the seat. */
export async function offerLogin(p: Paths, ask: Ask): Promise<Record<string, unknown> | null> {
const creds = readCredentials(credentialsPath(p));
if (!holdsLogin(creds)) return null;
const key = (await ask(`${SEAT}.public_key`, {})) as { public_key?: string } | null;
if (!key?.public_key) throw new Error("the licence manager did not say what key to seal a login to");
return (await ask(`${SEAT}.adopt`, {
node: p.node,
identity: readIdentity(accountPath(p)),
sealed: seal(JSON.stringify(creds!.claudeAiOauth), key.public_key),
})) as Record<string, unknown>;
}
// ---- MCP servers ----------------------------------------------------------------------------------
export interface Registration {
name: string;
entry?: Record<string, unknown>;
/** Which nodes: this one (absent), every node running the module ("all"), or a list. */
nodes?: "all" | string[];
}
/** The `servers` state, as this module reaches it through the runtime (`state("servers")` in the SDK). */
export interface ServerState {
put(key: string, value: Record<string, unknown>): Promise<number>;
delete(key: string): Promise<void>;
keys(): Promise<string[]>;
}
/** One change to the `servers` state, as a watch hands it over. */
export interface ServerChange {
key: string;
op: "put" | "delete";
value?: Record<string, unknown>;
}
/** The key a registration lives at: `all.<server>` for every node, `<node>.<server>` for one. */
export const keyOf = (scope: string, name: string) => `${scope}.${name}`;
/**
* What this node takes from the `servers` state: the entries for every node and for this one, by key —
* kept in memory from the watch, and written through to the module's own file whenever what applies here
* changes, so the managed directory can be rendered without the bus.
*/
export class ServerView {
private readonly entries = new Map<string, Record<string, unknown>>();
constructor(private readonly p: Paths) {}
/** Take one change; answers whether what applies to this node changed. */
take(c: ServerChange): boolean {
const dot = c.key.indexOf(".");
const scope = c.key.slice(0, dot), name = c.key.slice(dot + 1);
if (dot <= 0 || (scope !== "all" && scope !== this.p.node)) return false;
if (c.op === "put" && c.value && entryProblem(name, c.value) === null) this.entries.set(c.key, c.value);
else this.entries.delete(c.key);
return this.writeThrough();
}
/** What applies here: every node's entries, with this node's own laid over them by server name. */
effective(): Servers {
const out: Record<string, Record<string, unknown>> = {};
for (const scope of ["all", this.p.node]) {
for (const [key, entry] of [...this.entries].sort(([a], [b]) => a.localeCompare(b))) {
if (key.startsWith(scope + ".")) out[key.slice(scope.length + 1)] = entry;
}
}
return out;
}
private writeThrough(): boolean {
const now = JSON.stringify(this.effective(), null, 2) + "\n";
let before = "";
try {
before = readFileSync(registryPath(this.p), "utf8");
} catch {
/* none yet */
}
if (now === before) return false;
writeFileSync(registryPath(this.p), now, { mode: 0o600 });
return true;
}
}
/** A change from the watch: take it, and render when what applies here changed. */
export function onServerChange(view: ServerView, c: ServerChange, p: Paths, write: WriteManaged): string | null {
if (!view.take(c)) return null;
renderNow(p, write);
return `${c.op === "put" ? "registered" : "unregistered"} ${c.key}`;
}
const scopesOf = (p: Paths, nodes: Registration["nodes"]): string[] =>
nodes === undefined ? [p.node] : nodes === "all" ? ["all"] : nodes;
/**
* Register (or with no entry, unregister) a server: a put (or delete) per scope in the `servers` state.
* Taken into this node's view at once, so the answer says what it did here; every other node takes it
* from its watch, and a node that joins later from the current state.
*/
export async function registerServer(p: Paths, r: Registration, servers: ServerState, view: ServerView,
write: WriteManaged, others: () => Promise<string[]>): Promise<Record<string, unknown>> {
if (r.entry) {
const problem = entryProblem(r.name, r.entry);
if (problem) return { registered: false, reason: problem };
}
const scopes = scopesOf(p, r.nodes);
// Compared before and after rather than read from take(): this node's own watch may hand the view the
// same change first, and then take() here finds nothing new although this call made it.
const before = JSON.stringify(view.effective());
for (const scope of scopes) {
const key = keyOf(scope, r.name);
if (r.entry) await servers.put(key, r.entry);
else await servers.delete(key);
view.take({ key, op: r.entry ? "put" : "delete", value: r.entry });
}
const changedHere = JSON.stringify(view.effective()) !== before;
const here = scopes.includes("all") || scopes.includes(p.node);
const answer: Record<string, unknown> = {
[r.entry ? "registered" : "unregistered"]: r.name,
on: r.nodes === undefined ? [p.node] : r.nodes,
here: here ? (changedHere ? "changed" : "already so") : "not this node",
rendered: changedHere ? renderNow(p, write) : [],
};
if (!r.entry && view.effective()[r.name]) {
answer.still = `${r.name} still applies here from another registration (for every node, or for this one); unregister that too`;
}
if (r.nodes === undefined) {
// The question the operator wanted asked: here only, or more?
const elsewhere = (await others().catch(() => [] as string[])).filter((n) => n !== p.node);
answer.also = elsewhere.length
? `claude-code also runs on ${elsewhere.join(", ")}. To ${r.entry ? "register" : "unregister"} it there too, call again with nodes: "all" or a list of those nodes.`
: `To do the same on every node running claude-code, call again with nodes: "all".`;
}
return answer;
}
export { MANAGED_DIR };
+18
View File
@@ -0,0 +1,18 @@
{
"name": "@novox/module-claude-code",
"version": "0.1.0",
"description": "claude-code — the operator's agent on a machine: its managed configuration, and the consumer side of the Anthropic licence manager (novox/hq design 36).",
"type": "module",
"private": true,
"scripts": {
"build": "tsc seal.ts grant.ts identity.ts render.ts node.ts tools/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --rootDir . --outDir dist",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.7"
},
"devDependencies": {
"@types/node": "^22.0.0",
"typescript": "^5.6.0"
}
}
+135
View File
@@ -0,0 +1,135 @@
// What the module writes into the agent's machine-wide managed directory (novox/hq design 36 §1–§4).
// Pure: composed from the facts the mesh rendered, the settings the operator set and the licence the
// node holds, so what lands under /etc is tested without a machine.
//
// Three files, owned whole by this module:
// managed-mcp.json the tool servers every session loads: the mesh's console as `mesh`, and the
// servers the operator declared for the mesh or this node. Exclusive by the
// vendor's rule — a server not listed here does not load — which is why the
// list is the module's settings and nothing else (operator's choice, 2026-10-03).
// managed-settings.json the mesh's keys only: the repositories' attribution convention, the
// claude.ai connectors kept beside the managed servers, and — for an API-key
// licence only — the key-helper. A person's preferences are theirs.
// CLAUDE.md how a session on this mesh works, who this node is, the conventions.
export const MANAGED_DIR = "/etc/claude-code";
export interface Facts {
readonly node: string;
readonly console: string;
}
export interface Settings {
readonly role?: string;
/** Extra tool servers, in the vendor's `.mcp.json` entry shape, keyed by name. */
readonly mcp_servers?: Readonly<Record<string, Record<string, unknown>>>;
}
export interface Binding {
readonly licence: string;
readonly kind: "subscription" | "api-key";
}
export interface Rendered {
readonly [file: string]: string;
}
const MESH_ENTRY = "mesh";
export type Servers = Readonly<Record<string, Record<string, unknown>>>;
/** Whether an entry is one the vendor's managed file takes: a name of letters, digits, `-` and `_`, and
* an http/sse server with a url or a stdio server with a command. Returns why not, or null. */
export function entryProblem(name: string, entry: Record<string, unknown>): string | null {
if (!/^[A-Za-z0-9_-]+$/.test(name)) return `"${name}" is not a name the agent takes: letters, digits, - and _`;
if (name === MESH_ENTRY) return `"${MESH_ENTRY}" is the mesh's own entry`;
const type = entry?.type ?? "stdio";
if (type === "http" || type === "sse" || type === "streamable-http") {
return typeof entry.url === "string" && entry.url ? null : `an ${type} server needs a url`;
}
if (type === "stdio") return typeof entry.command === "string" && entry.command ? null : "a stdio server needs a command";
return `"${String(type)}" is not a server type the agent knows (http, sse, stdio)`;
}
/**
* Compose the three files. `registered` is the module's own list on this node — what was registered
* through its tools — laid over the servers the operator set in its settings.
*/
export function render(facts: Facts, settings: Settings, binding: Binding | null, helperPath: string,
registered: Servers = {}): Rendered {
const servers: Record<string, unknown> = {};
for (const [name, entry] of Object.entries({ ...(settings.mcp_servers ?? {}), ...registered })) {
if (entryProblem(name, entry) !== null) continue; // the mesh's own entry, or one the agent would refuse
servers[name] = entry;
}
servers[MESH_ENTRY] = { type: "http", url: facts.console };
const managed: Record<string, unknown> = {
attribution: { commit: "", pr: "" },
allowAllClaudeAiMcps: true,
};
if (binding?.kind === "api-key") managed.apiKeyHelper = helperPath;
return {
"managed-mcp.json": json({ mcpServers: sortKeys(servers) }),
"managed-settings.json": json(managed),
"CLAUDE.md": instructions(facts, settings),
};
}
function json(v: unknown): string {
return JSON.stringify(v, null, 2) + "\n";
}
function sortKeys(o: Record<string, unknown>): Record<string, unknown> {
return Object.fromEntries(Object.keys(o).sort().map((k) => [k, o[k]]));
}
export function instructions(facts: Facts, settings: Settings): string {
const role = settings.role?.trim() ? settings.role.trim() : "not stated — set it in this module's settings for the node";
return `# This machine is a node of a Novox mesh
Written by the mesh's \`claude-code\` module. Edit the module's settings or the catalogue, never this file:
it is rewritten whenever the module renders.
## Who this node is
- **Node:** \`${facts.node}\`
- **Role:** ${role}
- The other nodes, their roles and what runs where: ask the controller (\`mesh-controller.nodes\`,
\`mesh-controller.node\`). Nothing here lists them, because a copy drifts.
## How a session on this mesh works
The console is the only way to the mesh: the MCP server named \`mesh\`. It offers five tools, and
everything else is an address you find and call through them:
- \`mesh_search\` — words in, matching addresses out. \`mesh_describe\` — one address's arguments.
- \`mesh_call\` — call an address. A seat the mesh holds once is \`<seat>.<verb>\` (the mesh's own verbs
are \`mesh-controller.<verb>\`: \`status\`, \`plan\`, \`node\`, \`assign\`, \`push\`, \`settings\`);
a module on a machine is \`<node>/<module>.<tool>\`.
- \`mesh_overview\` and \`mesh_machine\` — the mesh's seats and machines, and what one machine runs.
- **Symptom first.** For an error, a failing service or anything unexpected, search the record with the
literal text before forming a hypothesis: the records module's \`records_search\`, then
\`records_read\`.
- **Ask the mesh before changing it**, and change it through the controller's verbs or the catalogue.
- **A licence** through the \`anthropic-licence-manager\` seat's verbs. Never edit the agent's credentials
file by hand, never print or ask for a token.
## Hard rules
- A file the mesh manages is changed through the verb or the catalogue that owns it, never on disk. If
unsure, \`mesh-controller.plan\` for the node says what the mesh writes there.
- Never write to a store's database by hand; schema changes are numbered migrations.
- Never push to a main branch: a branch, a pull request, and a human approval for every merge.
- The mesh creates no symlinks, and nobody else does either.
- A package is declared in a module, never installed by hand.
## Conventions
- Commit messages are concise, in the imperative, about why.
- Test before pushing: nodes update unattended.
- The playbooks in the record say how research, decisions, designs, issues and hand-offs are done.
`;
}
+77
View File
@@ -0,0 +1,77 @@
// Sealing a token to one recipient (novox/hq ADR 0183): the manager seals what it hands a node to that
// node's agent module key, and a node seals a waiting login to the key the manager names. X25519 for
// the agreement, HKDF-SHA256 for the key, AES-256-GCM for the box — all from Node's own library, so a
// bundle carries no dependency and no secret ever crosses the bus in the clear.
//
// A sealed box is `{ v: 1, eph, iv, tag, ct }`, every field base64. `eph` is a one-time public key, so
// two boxes of one value to one recipient share nothing, and only the recipient's private key opens it.
import {
createCipheriv, createDecipheriv, createPrivateKey, createPublicKey, diffieHellman,
generateKeyPairSync, hkdfSync, randomBytes, type KeyObject,
} from "node:crypto";
export interface SealedBox {
readonly v: 1;
readonly eph: string;
readonly iv: string;
readonly tag: string;
readonly ct: string;
}
/** A recipient's keypair, as the two PEM strings it is kept and published as. */
export interface KeyPairPem {
readonly publicKey: string;
readonly privateKey: string;
}
const INFO = Buffer.from("novox-mesh sealed box v1");
export function generateKeyPair(): KeyPairPem {
const { publicKey, privateKey } = generateKeyPairSync("x25519");
return {
publicKey: publicKey.export({ type: "spki", format: "pem" }).toString(),
privateKey: privateKey.export({ type: "pkcs8", format: "pem" }).toString(),
};
}
function keyFor(secret: Buffer, eph: Buffer, recipient: Buffer): Buffer {
// The ephemeral and the recipient's public halves are bound into the key, so a box cannot be
// re-addressed to another recipient by swapping its `eph`.
return Buffer.from(hkdfSync("sha256", secret, Buffer.concat([eph, recipient]), INFO, 32));
}
function rawPublic(key: KeyObject): Buffer {
return key.export({ type: "spki", format: "der" }).subarray(-32);
}
export function seal(plaintext: string, recipientPublicPem: string): SealedBox {
const recipient = createPublicKey(recipientPublicPem);
const eph = generateKeyPairSync("x25519");
const secret = diffieHellman({ privateKey: eph.privateKey, publicKey: recipient });
const ephRaw = eph.publicKey.export({ type: "spki", format: "der" });
const key = keyFor(secret, ephRaw, rawPublic(recipient));
const iv = randomBytes(12);
const cipher = createCipheriv("aes-256-gcm", key, iv);
const ct = Buffer.concat([cipher.update(plaintext, "utf8"), cipher.final()]);
return {
v: 1,
eph: ephRaw.toString("base64"),
iv: iv.toString("base64"),
tag: cipher.getAuthTag().toString("base64"),
ct: ct.toString("base64"),
};
}
/** Open a box with the recipient's private key. Throws on a box for another key or one tampered with. */
export function open(box: SealedBox, privateKeyPem: string): string {
if (!box || box.v !== 1) throw new Error("not a sealed box this module can open");
const priv = createPrivateKey(privateKeyPem);
const ephRaw = Buffer.from(box.eph, "base64");
const eph = createPublicKey({ key: ephRaw, format: "der", type: "spki" });
const secret = diffieHellman({ privateKey: priv, publicKey: eph });
const key = keyFor(secret, ephRaw, rawPublic(createPublicKey(priv)));
const decipher = createDecipheriv("aes-256-gcm", key, Buffer.from(box.iv, "base64"));
decipher.setAuthTag(Buffer.from(box.tag, "base64"));
return Buffer.concat([decipher.update(Buffer.from(box.ct, "base64")), decipher.final()]).toString("utf8");
}
+54
View File
@@ -0,0 +1,54 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, readFileSync, statSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
decideApply, grantOf, holdsLogin, readCredentials, withGrant, writeCredentials, type Grant,
} from "../dist/grant.js";
const NOW = 1_700_000_000_000;
const HOUR = 3_600_000;
const g = (over: Partial<Grant> = {}): Grant => ({
accessToken: "tok-A", expiresAt: NOW + HOUR, refreshTokenExpiresAt: NOW + 30 * 24 * HOUR, ...over,
});
test("a rotation applies a newer grant of the same licence", () => {
assert.deepEqual(decideApply(g(), g({ accessToken: "tok-B", expiresAt: NOW + 2 * HOUR }), "rotation"), { apply: true });
});
test("a rotation refuses a grant that arrived late and is older", () => {
const d = decideApply(g({ accessToken: "new", expiresAt: NOW + 2 * HOUR }), g({ accessToken: "old" }), "rotation");
assert.equal(d.apply === false && d.reason, "not-newer");
});
test("a grant re-issued by a login is adopted even though it expires sooner (2026-09-05)", () => {
const local = g({ expiresAt: NOW + 8 * HOUR, refreshTokenExpiresAt: NOW + 30 * 24 * HOUR });
const offered = g({ accessToken: "reissued", expiresAt: NOW + HOUR, refreshTokenExpiresAt: NOW + 5 * 24 * HOUR });
assert.deepEqual(decideApply(local, offered, "rotation"), { apply: true, reissued: true });
});
test("a switch to another licence applies whatever the expiries say", () => {
const local = g({ expiresAt: NOW + 8 * HOUR });
assert.equal(decideApply(local, g({ accessToken: "other", expiresAt: NOW + HOUR }), "switch").apply, true);
});
test("the same token is not rewritten", () => {
assert.deepEqual(decideApply(g(), g(), "switch"), { apply: false, reason: "already-current" });
});
test("a full grant left by a login is seen as a login, and stripped when the node's own is written", () => {
const dir = mkdtempSync(join(tmpdir(), "claude-code-"));
const path = join(dir, ".claude", ".credentials.json");
writeFileSync(join(dir, "x"), "");
const login = { claudeAiOauth: { accessToken: "at-login", refreshToken: "rt-login", expiresAt: NOW }, other: 1 };
assert.equal(holdsLogin(login), true);
writeCredentials(path, withGrant(login, g({ accessToken: "at-mesh", scopes: ["user:inference"] })));
const back = readCredentials(path)!;
assert.equal(holdsLogin(back), false);
assert.equal(grantOf(back)!.accessToken, "at-mesh");
assert.deepEqual(back.claudeAiOauth!.scopes, ["user:inference"]);
assert.equal(back.other, 1, "a key the module does not know was lost");
assert.ok(!readFileSync(path, "utf8").includes("rt-login"));
assert.equal(statSync(path).mode & 0o777, 0o600);
});
+19
View File
@@ -0,0 +1,19 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { readIdentity } from "../dist/identity.js";
test("the account is read from the agent's state file", () => {
const p = join(mkdtempSync(join(tmpdir(), "cc-id-")), ".claude.json");
writeFileSync(p, JSON.stringify({ oauthAccount: { accountUuid: "u-1", emailAddress: "a@example.org" }, other: 2 }));
assert.deepEqual(readIdentity(p), { accountUuid: "u-1", emailAddress: "a@example.org", organizationUuid: undefined });
});
test("no state file, or no account in it, is no identity rather than a guess", () => {
assert.equal(readIdentity("/nonexistent/.claude.json"), null);
const p = join(mkdtempSync(join(tmpdir(), "cc-id-")), ".claude.json");
writeFileSync(p, "{}");
assert.equal(readIdentity(p), null);
});
+173
View File
@@ -0,0 +1,173 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync, mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
apply, concerns, keypair, offerLogin, onServerChange, pull, registerServer, registered, ServerView, type Paths,
type ServerChange, type ServerState,
} from "../dist/node.js";
import { generateKeyPair, open, seal } from "../dist/seal.js";
const NOW = Date.now();
function node(name = "laptop"): { p: Paths; written: Record<string, string> } {
const root = mkdtempSync(join(tmpdir(), "cc-node-"));
const p = { state: join(root, "state"), facts: join(root, "state", "facts.json"), settings: join(root, "state", "settings.json"), home: join(root, "home"), node: name };
mkdirSync(p.state, { recursive: true });
mkdirSync(join(p.home, ".claude"), { recursive: true });
writeFileSync(p.facts, JSON.stringify({ node: name, console: "http://127.0.0.1:4270/mcp" }));
writeFileSync(p.settings, JSON.stringify({ role: "", mcp_servers: {} }));
return { p, written: {} };
}
const writer = (w: Record<string, string>) => (name: string, content: string) => { w[name] = content; return `${name}: written`; };
const creds = (p: Paths) => JSON.parse(readFileSync(join(p.home, ".claude", ".credentials.json"), "utf8"));
const grantFor = (p: Paths, licence: string, token: string, kind: "subscription" | "api-key" = "subscription", identity?: object) => ({
licence, kind, identity,
sealed: seal(kind === "api-key" ? token : JSON.stringify({ accessToken: token, expiresAt: NOW + 3_600_000, refreshTokenExpiresAt: NOW + 86_400_000, subscriptionType: licence }), keypair(p).publicKey),
});
test("a pull asks the seat with this node's key and applies what it answers", async () => {
const { p, written } = node();
let asked: [string, Record<string, unknown>] | null = null;
const r = await pull(p, async (address, args) => { asked = [address, args]; return grantFor(p, "personal", "at-1"); }, writer(written));
assert.equal(asked![0], "anthropic-licence-manager.current");
assert.equal(asked![1].node, "laptop");
assert.match(String(asked![1].public_key), /BEGIN PUBLIC KEY/);
assert.equal(r.applied, true);
assert.equal(creds(p).claudeAiOauth.accessToken, "at-1");
assert.ok(written["managed-mcp.json"]);
});
test("a switch replaces the old licence's grant whole and points the account at the new one", () => {
const { p, written } = node();
writeFileSync(join(p.home, ".claude.json"), JSON.stringify({ oauthAccount: { accountUuid: "old" }, projects: { keep: 1 } }));
apply(p, grantFor(p, "personal", "at-1"), writer(written));
const r = apply(p, grantFor(p, "work", "at-2", "subscription", { accountUuid: "new", emailAddress: "w@example.org" }), writer(written));
assert.equal(r.switched, true);
assert.equal(creds(p).claudeAiOauth.accessToken, "at-2");
assert.equal(creds(p).claudeAiOauth.subscriptionType, "work", "the old licence's subscription type survived the switch");
const account = JSON.parse(readFileSync(join(p.home, ".claude.json"), "utf8"));
assert.equal(account.oauthAccount.accountUuid, "new");
assert.deepEqual(account.projects, { keep: 1 });
});
test("switching to the API key adds the key-helper; switching away removes the key and the helper", () => {
const { p, written } = node();
apply(p, grantFor(p, "api", "sk-key", "api-key"), writer(written));
assert.ok(JSON.parse(written["managed-settings.json"]).apiKeyHelper);
assert.ok(existsSync(join(p.state, "api-key")));
apply(p, grantFor(p, "personal", "at-1"), writer(written));
assert.ok(!("apiKeyHelper" in JSON.parse(written["managed-settings.json"])));
assert.ok(!existsSync(join(p.state, "api-key")) && !existsSync(join(p.state, "api-key-helper")));
});
test("a rotation event concerns the node bound to that licence; a switch event the node it names", () => {
const { p, written } = node();
apply(p, grantFor(p, "personal", "at-1"), writer(written));
assert.equal(concerns(p, "claude-licence-manager.licence.rotated", { licence: "personal" }), true);
assert.equal(concerns(p, "claude-licence-manager.licence.rotated", { licence: "work" }), false);
assert.equal(concerns(p, "claude-licence-manager.licence.switched", { node: "laptop", licence: "work" }), true);
assert.equal(concerns(p, "claude-licence-manager.licence.switched", { node: "server" }), false);
});
test("a login is offered to the seat sealed to the seat's key, with the account it belongs to", async () => {
const { p } = node();
const manager = generateKeyPair();
writeFileSync(join(p.home, ".claude", ".credentials.json"), JSON.stringify({ claudeAiOauth: { accessToken: "at-login", refreshToken: "rt-login", expiresAt: NOW } }));
writeFileSync(join(p.home, ".claude.json"), JSON.stringify({ oauthAccount: { accountUuid: "u-9" } }));
const calls: [string, Record<string, unknown>][] = [];
await offerLogin(p, async (address, args) => { calls.push([address, args]); return address.endsWith("public_key") ? { public_key: manager.publicKey } : { adopted: true }; });
assert.deepEqual(calls.map((c) => c[0]), ["anthropic-licence-manager.public_key", "anthropic-licence-manager.adopt"]);
const adopt = calls[1][1] as { identity: { accountUuid: string }; sealed: never };
assert.equal(adopt.identity.accountUuid, "u-9");
assert.equal(JSON.parse(open(adopt.sealed, manager.privateKey)).refreshToken, "rt-login");
assert.ok(!JSON.stringify(adopt).includes("rt-login"), "the refresh token crossed in the clear");
});
test("no refresh token in the file is no login, and nothing is asked", async () => {
const { p } = node();
writeFileSync(join(p.home, ".claude", ".credentials.json"), JSON.stringify({ claudeAiOauth: { accessToken: "at", expiresAt: NOW } }));
assert.equal(await offerLogin(p, async () => { throw new Error("asked"); }), null);
});
/** The `servers` state as the bus holds it, shared by every node in a test, with each node's watch. */
function bus() {
const kept = new Map<string, Record<string, unknown>>();
const watchers: ((c: ServerChange) => void)[] = [];
const state: ServerState = {
put: async (key, value) => { kept.set(key, value); watchers.forEach((w) => w({ key, op: "put", value })); return kept.size; },
delete: async (key) => { kept.delete(key); watchers.forEach((w) => w({ key, op: "delete" })); },
keys: async () => [...kept.keys()].sort(),
};
/** A node joining: its view takes the current state, then every change. */
const join = (n: { p: Paths; written: Record<string, string> }) => {
const view = new ServerView(n.p);
for (const [key, value] of kept) onServerChange(view, { key, op: "put", value }, n.p, writer(n.written));
watchers.push((c) => onServerChange(view, c, n.p, writer(n.written)));
return view;
};
return { state, join, kept };
}
test("registering a server here puts it under this node's key, renders it, and asks about the other nodes", async () => {
const n = node();
const b = bus();
const view = b.join(n);
const r = await registerServer(n.p, { name: "search", entry: { type: "http", url: "https://s.example/mcp" } },
b.state, view, writer(n.written), async () => ["laptop", "server", "desktop"]);
assert.equal(r.here, "changed");
assert.match(String(r.also), /server, desktop/);
assert.deepEqual([...b.kept.keys()], ["laptop.search"]);
assert.ok(JSON.parse(n.written["managed-mcp.json"]).mcpServers.search);
});
test("registering for every node reaches the others through their watch, and a node joining later reads it", async () => {
const a = node("laptop"), s = node("server");
const b = bus();
const va = b.join(a);
b.join(s);
await registerServer(a.p, { name: "docs", entry: { type: "stdio", command: "docs-mcp" }, nodes: "all" },
b.state, va, writer(a.written), async () => []);
assert.deepEqual([...b.kept.keys()], ["all.docs"]);
assert.deepEqual(registered(s.p).docs, { type: "stdio", command: "docs-mcp" });
assert.ok(JSON.parse(s.written["managed-mcp.json"]).mcpServers.docs);
// The gap events left: a node assigned after the registration takes the whole current set at start.
const late = node("desktop");
b.join(late);
assert.deepEqual(registered(late.p).docs, { type: "stdio", command: "docs-mcp" });
// Unregistering is a delete, and every node's view drops it.
await registerServer(a.p, { name: "docs", nodes: "all" }, b.state, va, writer(a.written), async () => []);
assert.equal(registered(s.p).docs, undefined);
assert.equal(registered(late.p).docs, undefined);
});
test("a node's own registration overrides the one for every node; other nodes' keys leave this one alone", async () => {
const a = node("laptop"), s = node("server");
const b = bus();
const va = b.join(a);
const vs = b.join(s);
await registerServer(a.p, { name: "x", entry: { type: "http", url: "https://all" }, nodes: "all" }, b.state, va, writer(a.written), async () => []);
await registerServer(a.p, { name: "x", entry: { type: "http", url: "https://laptop" } }, b.state, va, writer(a.written), async () => []);
assert.equal(registered(a.p).x.url, "https://laptop");
assert.equal(registered(s.p).x.url, "https://all");
await registerServer(a.p, { name: "only", entry: { type: "http", url: "https://o" }, nodes: ["server"] }, b.state, va, writer(a.written), async () => []);
assert.equal(registered(a.p).only, undefined);
assert.equal(registered(s.p).only.url, "https://o");
// Unregistering here leaves the every-node one applying, and says so.
const r = await registerServer(a.p, { name: "x" }, b.state, va, writer(a.written), async () => []);
assert.match(String(r.still), /still applies here/);
assert.equal(registered(a.p).x.url, "https://all");
assert.equal(vs.effective().x.url, "https://all");
});
test("a bad entry is refused before anything is put; a repeated change changes nothing", async () => {
const n = node();
const b = bus();
const view = b.join(n);
const r = await registerServer(n.p, { name: "mesh", entry: { type: "http", url: "https://x" } }, b.state, view, writer(n.written), async () => []);
assert.equal(r.registered, false);
assert.equal(b.kept.size, 0);
assert.equal(onServerChange(view, { key: "all.a", op: "put", value: { type: "http", url: "https://a" } }, n.p, writer(n.written)), "registered all.a");
assert.equal(onServerChange(view, { key: "all.a", op: "put", value: { type: "http", url: "https://a" } }, n.p, writer(n.written)), null);
assert.equal(onServerChange(view, { key: "server.b", op: "put", value: { type: "http", url: "https://b" } }, n.p, writer(n.written)), null);
});
+40
View File
@@ -0,0 +1,40 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { render } from "../dist/render.js";
const facts = { node: "workstation", console: "http://127.0.0.1:4270/mcp" };
test("the console is the `mesh` server, and an operator's servers are listed beside it", () => {
const out = render(facts, { mcp_servers: { search: { type: "http", url: "https://s.example/mcp" } } }, null, "/h");
const mcp = JSON.parse(out["managed-mcp.json"]);
assert.deepEqual(Object.keys(mcp.mcpServers), ["mesh", "search"]);
assert.deepEqual(mcp.mcpServers.mesh, { type: "http", url: facts.console });
});
test("a setting cannot replace the mesh's own entry, and a name the vendor refuses is left out", () => {
const out = render(facts, { mcp_servers: { mesh: { type: "http", url: "http://evil" }, "bad name": {} } }, null, "/h");
const mcp = JSON.parse(out["managed-mcp.json"]);
assert.equal(mcp.mcpServers.mesh.url, facts.console);
assert.ok(!("bad name" in mcp.mcpServers));
});
test("managed settings carry the mesh's keys only, and the key-helper only for an API-key licence", () => {
const sub = JSON.parse(render(facts, {}, { licence: "personal", kind: "subscription" }, "/h")["managed-settings.json"]);
assert.deepEqual(sub, { attribution: { commit: "", pr: "" }, allowAllClaudeAiMcps: true });
const key = JSON.parse(render(facts, {}, { licence: "api", kind: "api-key" }, "/state/api-key-helper")["managed-settings.json"]);
assert.equal(key.apiKeyHelper, "/state/api-key-helper");
assert.ok(!("model" in key), "a preference is the person's");
});
test("the instruction file names the node and its role, and no other node", () => {
const md = render(facts, { role: "the laptop" }, null, "/h")["CLAUDE.md"];
assert.match(md, /\*\*Node:\*\* `workstation`/);
assert.match(md, /\*\*Role:\*\* the laptop/);
assert.match(md, /mesh_call/);
assert.match(md, /records_search/);
});
test("rendering is deterministic, so an unchanged input writes nothing", () => {
const s = { mcp_servers: { b: { type: "http", url: "https://b" }, a: { type: "http", url: "https://a" } } };
assert.deepEqual(render(facts, s, null, "/h"), render(facts, s, null, "/h"));
});
+31
View File
@@ -0,0 +1,31 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { generateKeyPair, open, seal } from "../dist/seal.js";
test("a box opens with its recipient's key and yields the value", () => {
const k = generateKeyPair();
assert.equal(open(seal("at-secret", k.publicKey), k.privateKey), "at-secret");
});
test("a box sealed for one node does not open with another node's key", () => {
const a = generateKeyPair();
const b = generateKeyPair();
assert.throws(() => open(seal("at-secret", a.publicKey), b.privateKey));
});
test("a tampered box is refused, not opened to garbage", () => {
const k = generateKeyPair();
const box = seal("at-secret", k.publicKey);
const ct = Buffer.from(box.ct, "base64");
ct[0] ^= 0xff;
assert.throws(() => open({ ...box, ct: ct.toString("base64") }, k.privateKey));
});
test("two boxes of one value share nothing a reader could compare", () => {
const k = generateKeyPair();
const x = seal("at-secret", k.publicKey);
const y = seal("at-secret", k.publicKey);
assert.notEqual(x.ct, y.ct);
assert.notEqual(x.eph, y.eph);
assert.ok(!JSON.stringify(x).includes("at-secret"));
});
+224
View File
@@ -0,0 +1,224 @@
// claude-code's bundle (novox/hq design 36, ADR 0183). The node's runtime launches it over stdio, as the
// operator account (ADR 0193), and is its bus (ADR 0198): it asks tools, emits and consumes through the
// runtime. It is given its state directory and two files the mesh renders into it (ADR 0192), beside the
// runtime's own words. **stdout is the MCP channel**: everything this module says, it says on stderr.
//
// At start it renders the agent's managed directory, asks the licence manager for this node's token,
// begins watching the credentials file for a login, takes the manager's licence events, and watches the
// module's `servers` state — every node's MCP server registrations (novox/hq ADR 0201). node.ts holds the
// logic.
import { mkdtempSync, readFileSync, rmSync, watchFile, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { spawnSync } from "node:child_process";
import { join } from "node:path";
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { broker } from "@novox/mesh-sdk/messaging";
import { on } from "@novox/mesh-sdk/events";
import { state } from "@novox/mesh-sdk/state";
import {
MANAGED_DIR, SEAT, ServerView, concerns, keypair, offerLogin, onServerChange, pull, readJson, registerServer,
registered, renderNow, type Ask, type Paths, type Registration, type ServerChange, type ServerState, type WriteManaged,
} from "../node.js";
import { grantOf, holdsLogin, readCredentials } from "../grant.js";
import { createHash } from "node:crypto";
const say = (line: string) => console.error(`[claude-code] ${line}`);
const fingerprint = (s: string) => "sha256:" + createHash("sha256").update(s).digest("hex").slice(0, 16);
function pathsFrom(env: NodeJS.ProcessEnv): Paths | null {
const state = env.MESH_CLAUDE_CODE_STATE, facts = env.MESH_CLAUDE_CODE_FACTS;
const settings = env.MESH_CLAUDE_CODE_SETTINGS, home = env.MESH_OPERATOR_HOME, node = env.MESH_NODE;
if (!state || !facts || !settings || !home || !node) return null;
return { state, facts, settings, home, node };
}
/** Write one managed file as root, only when its content changed. */
const writeManaged: WriteManaged = (name, content) => {
const path = join(MANAGED_DIR, name);
try {
if (readFileSync(path, "utf8") === content) return `${name}: unchanged`;
} catch {
/* absent */
}
// From a file, never /dev/stdin: Node hands a child its input over a socket, which /dev/stdin cannot
// open (ENXIO) — found on the first assignment, where nothing under /etc/claude-code was ever written.
const staged = mkdtempSync(join(tmpdir(), "claude-code-"));
const source = join(staged, name);
writeFileSync(source, content, { mode: 0o644 });
const asRoot = process.getuid?.() === 0;
const cmd = asRoot ? ["install", "-D", "-m", "0644", source, path] : ["sudo", "-n", "install", "-D", "-m", "0644", source, path];
const r = spawnSync(cmd[0], cmd.slice(1), { encoding: "utf8" });
rmSync(staged, { recursive: true, force: true });
if (r.status !== 0) {
throw new Error(`${name}: could not be written to ${MANAGED_DIR} (${(r.stderr || r.error?.message || "").trim()}); ` +
`the module writes there through the operator account's passwordless sudo`);
}
return `${name}: written`;
};
/** A tool on the bus, through the runtime; its MCP answer read back as JSON where it is JSON. */
const ask: Ask = async (address, args) => {
const answer = (await broker().request<Record<string, unknown>, { content?: { text?: string }[]; isError?: boolean }>(address, args)) ?? {};
const text = answer.content?.map((c) => c.text ?? "").join("") ?? "";
if (answer.isError) throw new Error(`${address}: ${text}`);
try {
return JSON.parse(text);
} catch {
return text;
}
};
/** The nodes claude-code runs on, from the controller's list of modules — for the register tool's question. */
async function nodesRunningMe(): Promise<string[]> {
const out = await ask("mesh-controller.modules", {});
const text = typeof out === "string" ? out : String((out as { output?: string })?.output ?? "");
const line = text.split("\n").find((l) => /^claude-code\s/.test(l)) ?? "";
const on = line.split(" on ")[1] ?? "";
return on.trim() === "nothing" ? [] : on.split(",").map((s) => s.trim()).filter(Boolean);
}
function status(p: Paths): Record<string, unknown> {
const creds = readCredentials(join(p.home, ".claude", ".credentials.json"));
const grant = grantOf(creds);
const managed = ["managed-mcp.json", "managed-settings.json", "CLAUDE.md"].map((f) => {
try {
return { file: join(MANAGED_DIR, f), fingerprint: fingerprint(readFileSync(join(MANAGED_DIR, f), "utf8")) };
} catch {
return { file: join(MANAGED_DIR, f), fingerprint: null };
}
});
return {
node: p.node,
licence: readJson(join(p.state, "licence.json"), null),
token: grant ? { fingerprint: fingerprint(grant.accessToken), expiresAt: new Date(grant.expiresAt).toISOString(),
loginWaiting: holdsLogin(creds) } : null,
managed,
registered: Object.keys(registered(p)),
};
}
/** The module's MCP servers on the bus (ADR 0201): its own state, which every node of it watches. */
const servers = () => state<Record<string, unknown>>("servers") as unknown as ServerState;
/** What this node takes from that state, kept from the watch. One per process. */
let view: ServerView | null = null;
const viewOf = (p: Paths) => (view ??= new ServerView(p));
function tools(p: Paths): ToolDefinition[] {
const nodesArg = { type: "string", description: 'more nodes: "all" for every node running claude-code, or a comma-separated list; absent is this node only' };
const nodesOf = (v: unknown): Registration["nodes"] =>
v === undefined || v === "" ? undefined : v === "all" ? "all" : String(v).split(",").map((s) => s.trim()).filter(Boolean);
return [
{
name: "claude_code_status",
description: "Claude Code on this machine as the mesh configured it: the licence it holds and when its token expires, the managed files, the MCP servers registered here. Fingerprints only, never a token.",
input: {},
run: async () => status(p),
},
{
name: "claude_code_render",
description: "Write Claude Code's managed directory now, from the mesh's facts, this module's settings and the servers registered here.",
input: {},
run: async () => ({ rendered: renderNow(p, writeManaged) }),
},
{
name: "claude_code_pull",
description: "Ask the licence manager for this node's current token now and apply it, rather than waiting for its next event.",
input: {},
run: async () => pull(p, ask, writeManaged),
},
{
name: "claude_code_mcp_list",
description: "The MCP servers registered through this module: those that apply on this node (beside the console, `mesh`, and those set in the module's settings), and every registration on the mesh, by key — `all.<server>` for every node, `<node>.<server>` for one.",
input: {},
run: async () => ({ here: registered(p), everywhere: await servers().keys() }),
},
{
name: "claude_code_mcp_register",
description: "Register an MCP server with Claude Code on this node, every node, or a list — an http/sse server by url, or a stdio server by command. Kept on the bus, so a node that joins later takes it too. Never put a secret in env or headers: the mesh refuses one.",
input: {
name: { type: "string", description: "the server's name: letters, digits, - and _" },
type: { type: "string", description: "http, sse or stdio (default stdio when a command is given, http when a url is)" },
url: { type: "string", description: "an http or sse server's url" },
command: { type: "string", description: "a stdio server's program" },
args: { type: "array", description: "a stdio server's arguments" },
env: { type: "object", description: "a stdio server's environment" },
headers: { type: "object", description: "an http server's headers" },
nodes: nodesArg,
},
run: async (a) => {
const entry: Record<string, unknown> = { type: a.type ?? (a.url ? "http" : "stdio") };
for (const k of ["url", "command", "args", "env", "headers"]) if (a[k] !== undefined) entry[k] = a[k];
return registerServer(p, { name: String(a.name ?? ""), entry, nodes: nodesOf(a.nodes) }, servers(), viewOf(p), writeManaged, nodesRunningMe);
},
},
{
name: "claude_code_mcp_unregister",
description: "Remove an MCP server registered through this module, on this node or more.",
input: { name: { type: "string", description: "the server's name" }, nodes: nodesArg },
run: async (a) => registerServer(p, { name: String(a.name ?? ""), nodes: nodesOf(a.nodes) }, servers(), viewOf(p), writeManaged, nodesRunningMe),
},
];
}
registerModuleTools("claude-code", (env) => {
const p = pathsFrom(env);
if (!p) return [];
try {
keypair(p);
for (const line of renderNow(p, writeManaged)) if (!line.endsWith("unchanged")) say(line);
} catch (err) {
say(err instanceof Error ? err.message : String(err));
}
return tools(p);
});
// Launched by the runtime: the bus is there from the first line (ADR 0198). Outside it — a test, a
// build — nothing below runs.
const p = process.env.MESH_SERVED_MODULE ? pathsFrom(process.env) : null;
if (p) {
const loud = (what: string) => (err: unknown) => say(`${what}: ${err instanceof Error ? err.message : String(err)}`);
void on<{ licence?: string; node?: string }>("claude-licence-manager.licence.*", async (event) => {
if (!concerns(p, event.type, event.body ?? {})) return;
say(`${event.type} — asking ${SEAT} for this node's token`);
say(JSON.stringify(await pull(p, ask, writeManaged).catch((e) => ({ failed: String(e) }))));
}).catch(loud("the licence events"));
// Every node's MCP servers: the whole current set first, then each change (ADR 0201). **Not awaited
// where the module is imported**: the runtime waits on the handshake, and a bucket that is not on the
// bus yet — or a grant the bus has not reloaded — answers late; awaited here, that left the bundle
// unable to answer `initialize` in time and the module unserved (found on its first assignment). So it
// watches beside the handshake and asks again until the state answers; until then the managed
// directory holds what the file kept from the last run.
const watchServers = (attempt = 0): void => {
state<Record<string, unknown>>("servers").watch((c) => {
try {
const done = onServerChange(viewOf(p), c as ServerChange, p, writeManaged);
if (done) say(done);
} catch (err) {
loud(`taking ${c.op} ${c.key}`)(err); // the view took it; the next render writes it
}
}).then(
() => say(`watching the MCP servers${attempt ? ` (after ${attempt} refusal(s))` : ""}`),
(err) => {
const wait = [2, 5, 10, 30][attempt] ?? 60;
say(`the MCP servers cannot be watched yet (${err instanceof Error ? err.message : String(err)}); asking again in ${wait}s`);
setTimeout(() => watchServers(attempt + 1), wait * 1000);
});
};
watchServers();
// Catch up once at start: a node that was off takes its current token now.
void pull(p, ask, writeManaged).then((r) => say(`at start: ${JSON.stringify(r)}`), loud("asking for this node's token at start"));
// A login: a refresh token appears in the credentials file. Polled, because the file is replaced by
// rename and a watch on the old inode would go quiet.
const credentials = join(p.home, ".claude", ".credentials.json");
watchFile(credentials, { interval: 5000 }, () => {
void offerLogin(p, ask).then((r) => { if (r) say(`a login here was offered to ${SEAT}: ${JSON.stringify(r)}`); },
loud("offering a login to the licence manager"));
});
}
+12
View File
@@ -0,0 +1,12 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["seal.ts", "grant.ts", "identity.ts", "render.ts", "node.ts", "tools/index.ts"]
}
+4 -1
View File
@@ -59,7 +59,10 @@
],
"volumes": [
"/var/lib/mesh-registry:/var/lib/registry"
]
],
"env": {
"REGISTRY_STORAGE_DELETE_ENABLED": "true"
}
}
]
}
+166
View File
@@ -0,0 +1,166 @@
# docker
The container runtime as a module (novox/hq to-be 42 phase 1, item 8; research 027/01–02; ADR 0166,
ADR 0207). It claims the node seat `node-container-runtime`. That seat carries no verbs yet: its verbs,
and the host creating containers through its holder, wait on ADR 0166's acceptance. Until then the
tools below are the module's own.
## What it declares
| resource | what | the host's rule |
|---|---|---|
| `package` | `docker` | installed if absent; never uninstalled when the module goes |
| `buildx` | `docker-buildx` | the same. Only the build machine has it today; `docker build` needs it for BuildKit everywhere |
| `socket` | `docker.socket` running, enabled at boot | given back as found when the module goes (ADR 0118) |
| `prune-service`, `prune-timer` | `/etc/systemd/system/docker-prune.{service,timer}`, written whole | removed with the module |
| `prune` | `docker-prune.timer` running, enabled at boot; restarted when either file changes | stopped and disabled with the module (the mesh made the unit) |
The weekly prune takes **dangling images and build cache unused for a week, and nothing else**. It
takes no volume, no container and no image a container uses, so it never touches a container the mesh
holds. It runs at idle priority, at a random point in the hour after the weekly mark. A run missed
while the machine was off happens at the next boot.
**Capabilities:** `package-manager`, `service-manager`, `privileged`. It does not declare
`container-runtime`: under ADR 0165, which is still proposed, that word means a running daemon, and
the module that installs the daemon cannot require it.
## What it does not declare yet, and why
Three things this module should own are already declared by other modules on every machine. The
controller refuses two modules on one node that declare the same `path`, `unit`, `name` or `package`
(`checkResources`, mesh-controller `internal/catalogue/resolve.go`). Declaring any of them here would
make the module unassignable everywhere. The refusals were checked against the controller's own
check:
```
zsh and docker both declare the name "${machine:account}"
dnsmasq and docker both declare the path "/etc/docker/daemon.json"
dnsmasq and docker both declare the unit "docker.service"
```
### 1. `/etc/docker/daemon.json` and `docker.service` (issue 190)
Today the file has three writers. Each writes into it (`into: json`, ADR 0102) and reloads the
service:
- **`dnsmasq`** writes `dns` and `live-restore`, through `dnsmasq.runtime-dns` and `dnsmasq.runtime`.
- **The private network**, generated by the controller (`internal/overlay/generator.go`), writes
`insecure-registries`. The collision check does not see generated resources.
- **Nobody** writes log rotation. One machine has `log-driver` and `log-opts` by hand.
**The change proposed, in one merge:**
1. `dnsmasq` drops its `runtime-dns` and `runtime` resources.
2. `docker` adds the two resources below:
```json
{"id": "daemon", "type": "file", "path": "/etc/docker/daemon.json", "mode": "0644", "into": "json",
"content": "{\"dns\": [\"${machine:address}\"], \"live-restore\": true, \"log-driver\": \"json-file\", \"log-opts\": {\"max-size\": \"100m\", \"max-file\": \"5\"}}\n"},
{"id": "runtime", "type": "service", "unit": "docker.service", "state": "running", "boot": "enabled", "reload-on": ["daemon"]}
```
The service is **reloaded, never restarted**: a restart stops every container. The daemon reads
`live-restore` on a reload. It reads `dns`, `log-driver` and `log-opts` only at its next start, so
they apply then (to containers created afterwards, for the log keys). With `live-restore` on, that
start keeps every container running.
**Why one merge, and only after this module is on every machine:**
- In one apply, the host first gives back the resources that are no longer declared, then applies
the new ones (mesh-host `apply.go`).
- `dnsmasq` gives back `dns` and `live-restore` to what they held before it, and `docker` sets them
again in the same apply. The daemon is reloaded once, after both steps.
- A machine pushed the new `dnsmasq` *without* this module would keep its pre-mesh values for both
keys. On one machine that is `live-restore: false`, and the next daemon restart there would stop
every container.
**Later:** the controller hands the registry to this module as a value, and the overlay stops
generating its two resources (issue 190, steps 2 and 5). Until then the overlay keeps writing its one
key beside this module's. The host merges disjoint keys correctly; the mesh-host `into.go` record is
per resource.
### 2. The operator account's membership of the `docker` group
The right shape is the host's `user` shape. Its `groups` are additive: the host runs
`usermod --append` and never takes a group away.
```json
{"id": "group", "type": "user", "name": "${machine:account}", "groups": ["docker"]}
```
`zsh` already declares a `user` resource for the same account (its login shell). The controller
compares `name` across modules, so the two collide.
**The change proposed (mesh-controller, `checkResources`):** judge a `user` resource by the fields it
sets, not by its name:
- `shell` and `home` stay single-owner;
- `groups` may be declared by any number of modules, because the host only adds them.
Then this module declares the resource above, and no module has to carry another's group.
Today the operator account is in the group on every machine, by hand. Nothing is lost while it waits.
## The bootstrap's runtime
On the machine the mesh was first installed on, the foundation bundle declared `package docker`
(`container-runtime`) and `docker.service` running and enabled (`container-runtime-running`). ADR 0207
§5 exempts them.
- The host records them under their bare ids, with origin *carried*. A mesh declaration's orphan pass
never sees them (mesh-host `store.go`).
- So `docker.package` here is a **second record of the same package**. The apply says "already
installed", and neither record ever uninstalls it.
- This module does not declare `docker.service` today, so nothing overlaps there. The proposed step
1 would add a second record of that unit. Its found state is *running*, because genesis started
it, so undeclaring this module would leave the daemon running.
## Tools
The tools run as the operator account. If the daemon's socket refuses that account, a call is asked
again through `sudo -n` (a process keeps the groups it started with). Every call has a 20 s bound.
A failure is an error naming how it failed, never an empty answer.
**Every container on the machine is in scope.** A container the mesh holds carries the host's label
`mesh-host.id` (its value names the assignment), and every answer says `mesh_held`.
| tool | | what |
|---|---|---|
| `docker_list` | r | every container: image, state, health, restarts, ports, mounts, compose project, `mesh_held`; filter by owner, state or name |
| `docker_inspect` | r | one container whole, **environment values left out** (names kept) |
| `docker_logs` | r | the last lines of both streams, merged in order, with timestamps (default 200, at most 2000) |
| `docker_stats` | r | CPU, memory, I/O and process count per running container, heaviest first |
| `docker_start` / `docker_stop` / `docker_restart` | a | one container. On a mesh-held one, the answer says the host restores its declared state at its next apply |
| `docker_top` | r | the processes inside one container |
| `docker_images` | r | images, largest first, with the containers using each; `dangling`, `unused` or `used` |
| `docker_prune` | a | dangling images and build cache, and stopped containers the mesh does not hold if `containers` is true. **A dry run unless `dry_run` is false. Never a volume** |
| `docker_disk_usage` | r | `docker system df -v`: total, active and reclaimable per kind, with the largest of each |
| `docker_networks` | r | networks, subnets, and the containers on each |
| `docker_volumes` | r | volumes, who mounts each, whether the mesh holds one of them, anonymous or not, and sizes if asked |
| `docker_events` | r | the runtime's events over a window ending now (default 60 min, at most 24 h), without exec noise |
| `docker_daemon_config` | r | `daemon.json` as on disk, `docker info`'s essentials, and keys the daemon has not taken yet |
| `docker_unlabelled` | r | the containers the mesh does not hold: the cleanup list |
| `docker_problems` | r | unhealthy, restarting, dead, killed for memory, failed, or restarted five times or more |
| `docker_ports` | r | every published port, and the containers on the host's network |
## Tests
```
go test ./...
```
The tests run against a fake runner and cover:
- escalation through `sudo -n` on a refused socket, and never as root;
- each failure named by its cause;
- a name or id never read as an option;
- mesh-held marking;
- the environment left out of `inspect`;
- the restore note on a mesh-held act;
- prune being a dry run by default and never reaching a volume, a mesh container or `--volumes`;
- the log merge;
- size parsing;
- what the daemon has not yet taken;
- event filtering;
- volume ownership;
- that the tools served are exactly the manifest's `tools`.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,360 @@
package main
import (
"context"
"encoding/json"
"errors"
"io/fs"
"os"
"reflect"
"strings"
"testing"
"time"
)
type call struct {
name string
args []string
}
// fake answers each command by the first rule whose prefix matches "name arg arg…".
type fake struct {
rules []rule
calls []call
}
type rule struct {
prefix string
ran Ran
}
func (f *fake) on(prefix string, r Ran) *fake { f.rules = append(f.rules, rule{prefix, r}); return f }
func (f *fake) run(_ context.Context, name string, args ...string) Ran {
f.calls = append(f.calls, call{name, args})
line := strings.Join(append([]string{name}, args...), " ")
for _, r := range f.rules {
if strings.HasPrefix(line, r.prefix) {
return r.ran
}
}
return Ran{Status: 1, Stderr: "unexpected: " + line}
}
func (f *fake) ran(prefix string) bool {
for _, c := range f.calls {
if strings.HasPrefix(strings.Join(append([]string{c.name}, c.args...), " "), prefix) {
return true
}
}
return false
}
func client(f *fake, uid int) *Client {
return &Client{Run: f.run, UID: uid, ReadFile: func(string) ([]byte, error) { return nil, fs.ErrNotExist },
Now: func() time.Time { return time.Date(2026, 10, 4, 12, 0, 0, 0, time.UTC) }}
}
const held = `{"Id":"aaaaaaaaaaaaaaaa","Name":"/mesh-web","Created":"2026-10-01T00:00:00Z","Image":"sha256:img1",
"Config":{"Image":"web:1","Labels":{"mesh-host.id":"hello-web.server","mesh-host.spec":"x"},"Env":["PASSWORD=hunter2","PATH=/bin"]},
"State":{"Status":"running","Running":true,"StartedAt":"2026-10-01T00:00:01Z","FinishedAt":"0001-01-01T00:00:00Z","Health":{"Status":"healthy"}},
"HostConfig":{"RestartPolicy":{"Name":"unless-stopped"},"NetworkMode":"bridge"},
"NetworkSettings":{"Ports":{"80/tcp":[{"HostIp":"0.0.0.0","HostPort":"8080"}]}},
"Mounts":[{"Type":"volume","Name":"webdata","Destination":"/data","RW":true}]}`
const stray = `{"Id":"bbbbbbbbbbbbbbbb","Name":"/dev-db","Created":"2026-09-01T00:00:00Z","Image":"sha256:img2",
"Config":{"Image":"postgres:16","Labels":{"com.docker.compose.project":"dev","com.docker.compose.project.working_dir":"/home/op/dev"}},
"State":{"Status":"exited","ExitCode":1,"FinishedAt":"2026-09-02T00:00:00Z"},
"HostConfig":{"RestartPolicy":{"Name":"no"}},"NetworkSettings":{"Ports":{}},
"Mounts":[{"Type":"volume","Name":"dbdata","Destination":"/var/lib/postgresql/data","RW":true}]}`
func machine() *fake {
return (&fake{}).
on("docker ps --all --quiet --no-trunc", Ran{Stdout: "aaaaaaaaaaaaaaaa\nbbbbbbbbbbbbbbbb\n"}).
on("docker container inspect aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb", Ran{Stdout: "[" + held + "," + stray + "]"}).
on("docker container inspect mesh-web", Ran{Stdout: "[" + held + "]"}).
on("docker container inspect dev-db", Ran{Stdout: "[" + stray + "]"})
}
func TestARefusedSocketIsAskedAgainThroughSudoWithoutAPromptUnlessThisIsRoot(t *testing.T) {
denied := Ran{Status: 1, Stderr: "permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get ...: dial unix /var/run/docker.sock: connect: permission denied\n"}
f := (&fake{}).on("docker ", denied).on("sudo -n docker info", Ran{Stdout: "{}"})
if _, err := client(f, 1000).docker(context.Background(), "info", "--format", "{{json .}}"); err != nil {
t.Fatal(err)
}
if !f.ran("sudo -n docker info --format") {
t.Fatalf("not escalated: %+v", f.calls)
}
f = (&fake{}).on("docker ", denied)
if _, err := client(f, 0).docker(context.Background(), "info"); err == nil || f.ran("sudo") {
t.Fatalf("root escalated or answered: %v %+v", err, f.calls)
}
}
func TestFailuresAreNamedByHowTheyFailed(t *testing.T) {
denied := Ran{Status: 1, Stderr: "permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock\n"}
cases := map[string]*fake{
"may not escalate without a prompt": (&fake{}).on("docker ", denied).on("sudo ", Ran{Status: 1, Stderr: "sudo: a password is required\n"}),
"sudo is not installed": (&fake{}).on("docker ", denied).on("sudo ", Ran{Status: 127, Err: "ENOENT"}),
"docker is not installed": (&fake{}).on("docker ", Ran{Status: 127, Err: "ENOENT"}),
"daemon is not answering": (&fake{}).on("docker ", Ran{Status: 1, Stderr: "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\n"}),
"did not answer: no answer within": (&fake{}).on("docker ", Ran{Status: 124, Err: "no answer within 20 s"}),
"docker info failed (3): boom": (&fake{}).on("docker ", Ran{Status: 3, Stderr: "boom\n"}),
}
for want, f := range cases {
_, err := client(f, 1000).docker(context.Background(), "info")
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("want %q, got %v", want, err)
}
}
}
func TestANameIsNeverAnOption(t *testing.T) {
for _, bad := range []string{"--help", "-v", "", "a b", "x;y"} {
if _, err := Ref(bad); err == nil {
t.Errorf("%q accepted", bad)
}
}
for _, good := range []string{"mesh-web", "aaaaaaaaaaaa", "registry.mesh.internal:5100/x@sha256:abc", "dev_db.1"} {
if _, err := Ref(good); err != nil {
t.Errorf("%q refused: %v", good, err)
}
}
f := machine()
for _, verb := range []string{"start", "stop", "restart"} {
if _, err := client(f, 1000).Act(context.Background(), verb, "--rm"); err == nil {
t.Errorf("%s took an option", verb)
}
}
if len(f.calls) != 0 {
t.Fatalf("docker was called: %+v", f.calls)
}
}
func TestEveryContainerIsListedAndTheMeshsAreMarked(t *testing.T) {
c := client(machine(), 1000)
all, err := c.Containers(context.Background(), "", "", "")
if err != nil || len(all) != 2 {
t.Fatalf("%v %+v", err, all)
}
web, db := all[1], all[0]
if !web.MeshHeld || web.HeldBy != "hello-web.server" || web.Module != "hello-web" || web.Health != "healthy" {
t.Errorf("held: %+v", web)
}
if !reflect.DeepEqual(web.Ports, []string{"0.0.0.0:8080->80/tcp"}) || web.Mounts[0].Name != "webdata" {
t.Errorf("ports/mounts: %+v", web)
}
if db.MeshHeld || db.Compose != "dev" || db.ComposeDir != "/home/op/dev" || db.FinishedAt == "" {
t.Errorf("stray: %+v", db)
}
mesh, _ := c.Containers(context.Background(), "mesh", "", "")
other, _ := c.Containers(context.Background(), "other", "", "")
if len(mesh) != 1 || mesh[0].Name != "mesh-web" || len(other) != 1 || other[0].Name != "dev-db" {
t.Errorf("held filter: %+v / %+v", mesh, other)
}
if _, err := c.Containers(context.Background(), "mine", "", ""); err == nil {
t.Error("an unknown held filter was accepted")
}
}
func TestNoContainersIsAnEmptyListAndAFailureIsAnError(t *testing.T) {
got, err := client((&fake{}).on("docker ps", Ran{}), 1000).Containers(context.Background(), "", "", "")
if err != nil || got == nil || len(got) != 0 {
t.Fatalf("%v %v", got, err)
}
if _, err := client((&fake{}).on("docker ps", Ran{Status: 1, Stderr: "Cannot connect to the Docker daemon\n"}), 1000).Containers(context.Background(), "", "", ""); err == nil {
t.Fatal("a daemon that does not answer read as no containers")
}
}
func TestInspectLeavesTheEnvironmentsValuesOut(t *testing.T) {
got, err := client(machine(), 1000).Inspect(context.Background(), "mesh-web")
if err != nil {
t.Fatal(err)
}
b, _ := json.Marshal(got)
if strings.Contains(string(b), "hunter2") || !strings.Contains(string(b), `"PASSWORD"`) || got["mesh_held"] != true {
t.Fatalf("%s", b)
}
}
func TestActingOnAMeshContainerSaysTheHostRestoresIt(t *testing.T) {
f := machine().on("docker stop", Ran{}).on("docker start", Ran{})
got, err := client(f, 1000).Act(context.Background(), "stop", "mesh-web")
if err != nil {
t.Fatal(err)
}
if !f.ran("docker stop --time 10 mesh-web") || got["mesh_held"] != true || !strings.Contains(got["note"].(string), "host restores") {
t.Fatalf("%v %+v", got, f.calls)
}
got, _ = client(f, 1000).Act(context.Background(), "start", "dev-db")
if _, noted := got["note"]; noted || got["mesh_held"] != false {
t.Fatalf("a stray was noted: %v", got)
}
}
func TestPruneIsADryRunByDefaultAndNeverTouchesAVolumeOrAMeshContainer(t *testing.T) {
f := machine().
on("docker image ls --no-trunc --filter dangling=true", Ran{Stdout: `{"ID":"sha256:dead","Size":"1.5GB"}` + "\n"}).
on("docker system df --format", Ran{Stdout: `{"Type":"Build Cache","TotalCount":"3","Size":"2GB","Reclaimable":"1GB"}` + "\n"}).
on("docker image prune", Ran{Stdout: "Deleted Images:\nx\n\nTotal reclaimed space: 1.5GB\n"}).
on("docker builder prune", Ran{Stdout: "Total:\t1GB\n"}).
on("docker container rm", Ran{})
c := client(f, 1000)
got, err := c.Prune(context.Background(), PruneAsk{Images: true, BuildCache: true, Containers: true, DryRun: true})
if err != nil {
t.Fatal(err)
}
if f.ran("docker image prune") || f.ran("docker builder prune") || f.ran("docker container rm") {
t.Fatalf("a dry run removed something: %+v", f.calls)
}
if got["images"].(map[string]any)["dangling"] != 1 || !reflect.DeepEqual(got["containers"].(map[string]any)["stopped_not_held"], []string{"dev-db"}) {
t.Fatalf("%v", got)
}
got, err = c.Prune(context.Background(), PruneAsk{Images: true, BuildCache: true, Containers: true, OlderThanH: 24})
if err != nil {
t.Fatal(err)
}
if !f.ran("docker image prune --force --filter until=24h") || !f.ran("docker builder prune --force --filter until=24h") || !f.ran("docker container rm dev-db") {
t.Fatalf("not pruned: %+v", f.calls)
}
for _, c := range f.calls {
line := strings.Join(c.args, " ")
if strings.Contains(line, "volume") || strings.Contains(line, "mesh-web") && c.args[0] != "container" || strings.Contains(line, "--volumes") || strings.Contains(line, "--all") && c.args[0] != "ps" {
t.Errorf("prune reached too far: %s", line)
}
}
if got["images"].(map[string]any)["reclaimed"] != "1.5GB" || got["build_cache"].(map[string]any)["reclaimed"] != "1GB" {
t.Errorf("reclaimed: %v", got)
}
}
func TestLogsMergeBothStreamsInOrderAndKeepTheTail(t *testing.T) {
f := (&fake{}).on("docker logs", Ran{Stdout: "2026-10-04T10:00:01Z out one\n2026-10-04T10:00:03Z out two\n", Stderr: "2026-10-04T10:00:02Z err one\n"})
got, err := client(f, 1000).Logs(context.Background(), "web", 2, "30m")
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got["lines"], []string{"2026-10-04T10:00:02Z err one", "2026-10-04T10:00:03Z out two"}) {
t.Fatalf("%v", got["lines"])
}
if !f.ran("docker logs --timestamps --tail 2 --since 30m web") {
t.Fatalf("%+v", f.calls)
}
if _, err := client(f, 1000).Logs(context.Background(), "web", 2, "--follow"); err == nil {
t.Fatal("since took an option")
}
f = (&fake{}).on("docker logs", Ran{Status: 1, Stderr: "Error response from daemon: No such container: nope\n"})
if _, err := client(f, 1000).Logs(context.Background(), "nope", 2, ""); err == nil {
t.Fatal("a missing container read as no lines")
}
}
func TestSizesAreReadAsDockerPrintsThem(t *testing.T) {
for in, want := range map[string]int64{"0B": 0, "55.63GB": 55630000000, "33.2MiB": 34812723, "1.5kB": 1500, "12MB (34%)": 12000000, "N/A": -1} {
if got := Bytes(in); got != want {
t.Errorf("%s: %d, want %d", in, got, want)
}
}
}
func TestDaemonConfigSaysWhatTheDaemonHasNotTakenYet(t *testing.T) {
f := (&fake{}).on("docker info", Ran{Stdout: `{"ServerVersion":"29.8.2","LiveRestoreEnabled":false,"LoggingDriver":"json-file","RegistryConfig":{"IndexConfigs":{"docker.io":{"Secure":true},"registry.mesh.internal:5100":{"Secure":false}}}}`})
c := client(f, 1000)
c.ReadFile = func(string) ([]byte, error) {
return []byte(`{"live-restore": true, "dns": ["10.0.0.1"], "log-driver": "local"}`), nil
}
got, err := c.DaemonConfig(context.Background())
if err != nil {
t.Fatal(err)
}
pending := strings.Join(got["pending"].([]string), "\n")
if !strings.Contains(pending, "live-restore is true in the file and false") || !strings.Contains(pending, "log-driver is local") {
t.Errorf("pending: %s", pending)
}
if !reflect.DeepEqual(got["daemon"].(map[string]any)["InsecureRegistries"], []string{"registry.mesh.internal:5100"}) {
t.Errorf("registries: %v", got["daemon"])
}
if !reflect.DeepEqual(got["read_only_at_start"], []string{"dns", "log-driver"}) {
t.Errorf("start-only: %v", got["read_only_at_start"])
}
c.ReadFile = func(string) ([]byte, error) { return nil, os.ErrNotExist }
got, _ = c.DaemonConfig(context.Background())
if !strings.HasPrefix(got["file_state"].(string), "absent") {
t.Errorf("absent: %v", got["file_state"])
}
c.ReadFile = func(string) ([]byte, error) { return nil, errors.New("permission denied") }
got, _ = c.DaemonConfig(context.Background())
if !strings.HasPrefix(got["file_state"].(string), "unreadable") {
t.Errorf("unreadable: %v", got["file_state"])
}
}
func TestEventsAreABoundedWindowWithoutExecNoise(t *testing.T) {
out := `{"Type":"container","Action":"exec_start: pg_isready","Actor":{"ID":"aaaaaaaaaaaaaaaa","Attributes":{"name":"db"}},"timeNano":1}
{"Type":"container","Action":"die","Actor":{"ID":"aaaaaaaaaaaaaaaa","Attributes":{"name":"web","mesh-host.id":"hello-web.server","exitCode":"137"}},"timeNano":2}
`
f := (&fake{}).on("docker events", Ran{Stdout: out})
got, err := client(f, 1000).Events(context.Background(), 30, "container", 10, false)
if err != nil {
t.Fatal(err)
}
evs := got["events"].([]map[string]any)
if len(evs) != 1 || evs[0]["action"] != "die" || evs[0]["mesh_held"] != true || evs[0]["exit_code"] != "137" {
t.Fatalf("%v", evs)
}
if !f.ran("docker events --since 30m --until 0s --format {{json .}} --filter type=container") {
t.Fatalf("%+v", f.calls)
}
if _, err := client(f, 1000).Events(context.Background(), 30, "secret", 10, false); err == nil {
t.Fatal("an unknown type was accepted")
}
}
func TestVolumesSayWhoMountsThemAndWhetherTheMeshDoes(t *testing.T) {
f := machine().
on("docker volume ls --quiet", Ran{Stdout: "webdata\ndbdata\nloose\n"}).
on("docker volume inspect", Ran{Stdout: `[{"Name":"webdata","Driver":"local"},{"Name":"dbdata","Driver":"local"},{"Name":"loose","Driver":"local","Labels":{"com.docker.volume.anonymous":""}}]`})
got, err := client(f, 1000).Volumes(context.Background(), false, false)
if err != nil {
t.Fatal(err)
}
vols := got["volumes"].([]map[string]any)
if vols[0]["mesh_held"] != true || vols[1]["mesh_held"] != false || len(vols[2]["mounted_by"].([]map[string]any)) != 0 || vols[2]["anonymous"] != true {
t.Fatalf("%v", vols)
}
got, _ = client(f, 1000).Volumes(context.Background(), true, false)
if got["count"] != 1 {
t.Fatalf("unmounted: %v", got)
}
}
func TestImagesNameTheirUsers(t *testing.T) {
f := machine().on("docker image ls", Ran{Stdout: `{"ID":"sha256:img1","Repository":"web","Tag":"1","Size":"100MB"}
{"ID":"sha256:img3","Repository":"<none>","Tag":"<none>","Size":"2GB"}
`})
got, err := client(f, 1000).Images(context.Background(), "", "", 10)
if err != nil {
t.Fatal(err)
}
imgs := got["images"].([]Image)
if imgs[0].ID != "sha256:img3" || !imgs[0].Dangling || imgs[1].UsedBy[0] != "mesh-web" || !imgs[1].MeshUsed {
t.Fatalf("%+v", imgs)
}
got, _ = client(f, 1000).Images(context.Background(), "unused", "", 10)
if got["count"] != 1 {
t.Fatalf("unused: %v", got)
}
}
func TestProblemsNameWhyAndUnlabelledIsTheCleanupList(t *testing.T) {
c := client(machine(), 1000)
p, err := c.Problems(context.Background())
if err != nil || len(p) != 1 || p[0]["name"] != "dev-db" || p[0]["why"].([]string)[0] != "exited 1" {
t.Fatalf("%v %v", p, err)
}
u, err := c.Unlabelled(context.Background())
if err != nil || u["count"] != 1 {
t.Fatalf("%v %v", u, err)
}
}
+279
View File
@@ -0,0 +1,279 @@
// docker's Go tools bundle (novox/hq ADR 0188, ADR 0193): a process the node's tool runtime launches
// and speaks MCP over stdio to, through the Go SDK. It answers for every container on this machine —
// the mesh's and every other — and for the runtime's images, networks, volumes, events and
// configuration. It runs as the operator account (ADR 0175 §4); docker.go says how it reaches the
// daemon's socket. The host applies the module's resources; these tools answer about the runtime.
package main
import (
"context"
"fmt"
"math"
"os"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
)
func main() {
// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): docker.
if err := stdio.Serve("", tools(NewClient())); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
var containerArg = map[string]any{"type": "string", "description": "the container's name or id"}
func tools(c *Client) []stdio.Tool {
ctx := context.Background()
act := func(verb, description string) stdio.Tool {
return stdio.Tool{
Name: "docker_" + verb, Description: description,
Input: map[string]any{"container": containerArg},
Run: func(args map[string]any) (any, error) {
ref, err := text(args, "container")
if err != nil {
return nil, err
}
return c.Act(ctx, verb, ref)
},
}
}
return []stdio.Tool{
{
Name: "docker_list",
Description: "Every container on this machine — the mesh's and every other — with its image, state, health, restarts, " +
"published ports, mounts, compose project, and mesh_held/held_by (the assignment that holds it).",
Input: map[string]any{
"held": map[string]any{"type": "string", "enum": []string{"all", "mesh", "other"}, "description": "whose: all (default), the mesh's, or the others"},
"state": map[string]any{"type": "string", "description": "only containers in this state (running, exited, created, restarting, paused, dead)"},
"match": map[string]any{"type": "string", "description": "only containers whose name or image contains this"},
},
Run: func(args map[string]any) (any, error) {
list, err := c.Containers(ctx, optional(args, "held"), optional(args, "state"), optional(args, "match"))
if err != nil {
return nil, err
}
return map[string]any{"count": len(list), "containers": list}, nil
},
},
{
Name: "docker_inspect",
Description: "One container whole, as docker inspects it, with mesh_held; its environment's values are left out (names kept), because that is where a container's secrets are.",
Input: map[string]any{"container": containerArg},
Run: func(args map[string]any) (any, error) {
ref, err := text(args, "container")
if err != nil {
return nil, err
}
return c.Inspect(ctx, ref)
},
},
{
Name: "docker_logs",
Description: "The last lines one container wrote, both streams merged in order, each with its timestamp (default 200, at most 2000 lines; a line is cut at 4 KiB).",
Input: map[string]any{
"container": containerArg,
"lines": map[string]any{"type": "integer", "description": "how many lines from the end (default 200, at most 2000)"},
"since": map[string]any{"type": "string", "description": "only lines since then: a duration such as 30m or 2h, or a time"},
},
Run: func(args map[string]any) (any, error) {
ref, err := text(args, "container")
if err != nil {
return nil, err
}
n, err := bounded(args, "lines", 200, 2000)
if err != nil {
return nil, err
}
return c.Logs(ctx, ref, n, optional(args, "since"))
},
},
{
Name: "docker_stats",
Description: "What the running containers use now — CPU, memory, network and disk I/O, processes — the heaviest by memory first; or one container's.",
Input: map[string]any{"container": map[string]any{"type": "string", "description": "one container (optional)"}},
Run: func(args map[string]any) (any, error) {
stats, err := c.Stats(ctx, optional(args, "container"))
if err != nil {
return nil, err
}
return map[string]any{"count": len(stats), "containers": stats}, nil
},
},
act("start", "Start one container. A container the mesh holds is started too, and the answer says the host restores what its declaration says at its next apply."),
act("stop", "Stop one container (ten seconds, then killed). For a container the mesh holds, the answer says the host will start it again at its next apply if its declaration says running."),
act("restart", "Restart one container (ten seconds to stop, then killed); the answer says whether the mesh holds it."),
{
Name: "docker_top",
Description: "The processes running inside one container: pid, user, elapsed time, CPU, resident memory and command.",
Input: map[string]any{"container": containerArg},
Run: func(args map[string]any) (any, error) {
ref, err := text(args, "container")
if err != nil {
return nil, err
}
return c.Top(ctx, ref)
},
},
{
Name: "docker_images",
Description: "The images on this machine, the largest first, each with its size and the containers using it (and whether one of them is the mesh's). " +
"filter: all, dangling, unused or used.",
Input: map[string]any{
"filter": map[string]any{"type": "string", "enum": []string{"all", "dangling", "unused", "used"}, "description": "which images (default all)"},
"match": map[string]any{"type": "string", "description": "only images whose repository:tag contains this"},
"limit": map[string]any{"type": "integer", "description": "how many to show (default 100, at most 1000); count says how many matched"},
},
Run: func(args map[string]any) (any, error) {
n, err := bounded(args, "limit", 100, 1000)
if err != nil {
return nil, err
}
return c.Images(ctx, optional(args, "filter"), optional(args, "match"), n)
},
},
{
Name: "docker_prune",
Description: "Reclaim space: dangling images and unused build cache, and — only when containers is true — stopped containers the mesh does not hold. " +
"Never a volume, never a container the mesh holds, never an image a container uses. A dry run by default: it lists what would go; dry_run false removes it.",
Input: map[string]any{
"dry_run": map[string]any{"type": "boolean", "description": "list only (default true)"},
"images": map[string]any{"type": "boolean", "description": "dangling images (default true)"},
"build_cache": map[string]any{"type": "boolean", "description": "build cache nothing refers to (default true)"},
"containers": map[string]any{"type": "boolean", "description": "stopped containers the mesh does not hold (default false); what they mounted is kept"},
"older_than_hours": map[string]any{"type": "integer", "description": "only what is older than this many hours (default 0: any age)"},
},
Run: func(args map[string]any) (any, error) {
older := 0
if v, ok := args["older_than_hours"]; ok && v != nil && v != float64(0) {
n, err := bounded(args, "older_than_hours", 0, 24*365)
if err != nil {
return nil, err
}
older = n
}
return c.Prune(ctx, PruneAsk{
DryRun: flag(args, "dry_run", true), Images: flag(args, "images", true), BuildCache: flag(args, "build_cache", true),
Containers: flag(args, "containers", false), OlderThanH: older,
})
},
},
{
Name: "docker_disk_usage",
Description: "What the runtime takes on disk (docker system df -v): per kind — images, containers, volumes, build cache — the total, the active and the reclaimable, and the largest of each.",
Input: map[string]any{"top": map[string]any{"type": "integer", "description": "how many of the largest per kind (default 10, at most 100)"}},
Run: func(args map[string]any) (any, error) {
n, err := bounded(args, "top", 10, 100)
if err != nil {
return nil, err
}
return c.DiskUsage(ctx, n)
},
},
{
Name: "docker_networks",
Description: "Every network the runtime has: driver, scope, subnets and gateway, and the running containers on it with their addresses and whether the mesh holds them.",
Run: func(map[string]any) (any, error) { return c.Networks(ctx) },
},
{
Name: "docker_volumes",
Description: "Every volume with the containers mounting it, whether the mesh holds any of them, whether it is anonymous, its compose project, and — when sizes is true (slower) — its size.",
Input: map[string]any{
"unmounted": map[string]any{"type": "boolean", "description": "only volumes no container mounts (default false)"},
"sizes": map[string]any{"type": "boolean", "description": "measure each volume (default false: it walks every volume)"},
},
Run: func(args map[string]any) (any, error) {
return c.Volumes(ctx, flag(args, "unmounted", false), flag(args, "sizes", false))
},
},
{
Name: "docker_events",
Description: "What the runtime did in a window ending now (default the last 60 minutes, at most 24 hours): containers created, started, died, health changes, images pulled — with mesh_held. Exec events are left out unless asked.",
Input: map[string]any{
"minutes": map[string]any{"type": "integer", "description": "how far back (default 60, at most 1440)"},
"type": map[string]any{"type": "string", "description": "only one kind: container, image, network, volume, daemon, plugin or builder"},
"limit": map[string]any{"type": "integer", "description": "the latest this many (default 200, at most 2000)"},
"execs": map[string]any{"type": "boolean", "description": "include exec_* events (default false: health checks make many)"},
},
Run: func(args map[string]any) (any, error) {
minutes, err := bounded(args, "minutes", 60, 1440)
if err != nil {
return nil, err
}
limit, err := bounded(args, "limit", 200, 2000)
if err != nil {
return nil, err
}
return c.Events(ctx, minutes, optional(args, "type"), limit, flag(args, "execs", false))
},
},
{
Name: "docker_daemon_config",
Description: "The runtime's configuration: /etc/docker/daemon.json as it is on disk, the daemon's essentials as it runs now (docker info: version, storage and logging drivers, " +
"live restore, root directory, insecure registries, warnings), and where the two differ — keys a reload or only a restart would take.",
Run: func(map[string]any) (any, error) { return c.DaemonConfig(ctx) },
},
{
Name: "docker_unlabelled",
Description: "The containers the mesh does not hold — the cleanup list — each with its image, state, compose project and directory, ports and mounts.",
Run: func(map[string]any) (any, error) { return c.Unlabelled(ctx) },
},
{
Name: "docker_problems",
Description: "Every container that is not well: unhealthy, restarting, dead, killed for memory, exited with a failure, or restarted five times or more — with whether the mesh holds it.",
Run: func(map[string]any) (any, error) {
p, err := c.Problems(ctx)
if err != nil {
return nil, err
}
return map[string]any{"count": len(p), "containers": p}, nil
},
},
{
Name: "docker_ports",
Description: "Every port the containers publish on this machine (address:port -> container port), and the containers on the host's network, which publish whatever they listen on.",
Run: func(map[string]any) (any, error) {
p, err := c.Ports(ctx)
if err != nil {
return nil, err
}
return map[string]any{"count": len(p), "ports": p}, nil
},
},
}
}
func text(args map[string]any, key string) (string, error) {
s, _ := args[key].(string)
if s = strings.TrimSpace(s); s == "" {
return "", fmt.Errorf("%s is required", key)
}
return s, nil
}
func optional(args map[string]any, key string) string {
s, _ := args[key].(string)
return strings.TrimSpace(s)
}
func flag(args map[string]any, key string, def bool) bool {
if b, ok := args[key].(bool); ok {
return b
}
return def
}
// bounded is a whole number argument, defaulted when absent and held to a ceiling.
func bounded(args map[string]any, key string, def, most int) (int, error) {
v, ok := args[key]
if !ok || v == nil {
return def, nil
}
f, ok := v.(float64)
if !ok || f != math.Trunc(f) || f < 1 {
return 0, fmt.Errorf("%s must be a whole number of at least 1", key)
}
return int(math.Min(f, float64(most))), nil
}
+59
View File
@@ -0,0 +1,59 @@
package main
import (
"bytes"
"context"
"errors"
"fmt"
"os/exec"
"strings"
"time"
)
// Ran is what a command did: its output, its exit status, and why it never ran to an answer.
type Ran struct {
Stdout string
Stderr string
Status int
// Err is "ENOENT" when the program is not installed, or says it was ended for taking too long.
Err string
}
// Runner runs one command, so every tool can be tested without a daemon.
type Runner func(ctx context.Context, name string, args ...string) Ran
// CallTimeout is how long one docker command may take: below the runtime's thirty-second call
// limit, so a daemon that hangs is answered as such rather than as a call the runtime gave up on.
const CallTimeout = 20 * time.Second
// ExecRunner runs a command on this machine, bounded by CallTimeout.
func ExecRunner(ctx context.Context, name string, args ...string) Ran {
ctx, cancel := context.WithTimeout(ctx, CallTimeout)
defer cancel()
cmd := exec.CommandContext(ctx, name, args...)
var out, errb bytes.Buffer
cmd.Stdout, cmd.Stderr = &out, &errb
err := cmd.Run()
r := Ran{Stdout: out.String(), Stderr: errb.String()}
var exitErr *exec.ExitError
switch {
case errors.Is(ctx.Err(), context.DeadlineExceeded):
r.Status, r.Err = 124, fmt.Sprintf("no answer within %d s", int(CallTimeout/time.Second))
case errors.Is(err, exec.ErrNotFound):
r.Status, r.Err = 127, "ENOENT"
case errors.As(err, &exitErr):
r.Status = exitErr.ExitCode()
case err != nil:
r.Status, r.Err = 1, err.Error()
}
return r
}
func firstLine(s string) string {
for _, l := range strings.Split(s, "\n") {
if l = strings.TrimSpace(l); l != "" {
return l
}
}
return ""
}
@@ -0,0 +1,60 @@
package main
import (
"encoding/json"
"os"
"reflect"
"sort"
"strings"
"testing"
)
func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Tools []string `json:"tools"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
served := []string{}
for _, tool := range tools(client(&fake{}, 1000)) {
if !strings.HasPrefix(tool.Name, "docker_") || tool.Description == "" || tool.Run == nil {
t.Errorf("tool %q", tool.Name)
}
served = append(served, tool.Name)
}
sort.Strings(served)
listed := append([]string{}, m.Tools...)
sort.Strings(listed)
if !reflect.DeepEqual(served, listed) {
t.Fatalf("served %v, manifest %v", served, listed)
}
}
func TestNumbersAreDefaultedAndBounded(t *testing.T) {
if n, _ := bounded(map[string]any{}, "lines", 200, 2000); n != 200 {
t.Error(n)
}
if n, _ := bounded(map[string]any{"lines": float64(99999)}, "lines", 200, 2000); n != 2000 {
t.Error(n)
}
for _, bad := range []any{float64(0), float64(-1), float64(1.5), "10"} {
if _, err := bounded(map[string]any{"lines": bad}, "lines", 200, 2000); err == nil {
t.Errorf("%v accepted", bad)
}
}
}
func TestAStopFromTheToolNeedsAContainer(t *testing.T) {
for _, tool := range tools(client(&fake{}, 1000)) {
if tool.Name == "docker_stop" {
if _, err := tool.Run(map[string]any{}); err == nil {
t.Fatal("a stop without a container was accepted")
}
}
}
}
+5
View File
@@ -0,0 +1,5 @@
module docker
go 1.22
require git.novox.be/novox/mesh-sdk/go v0.1.6
+2
View File
@@ -0,0 +1,2 @@
git.novox.be/novox/mesh-sdk/go v0.1.6 h1:9qzdYONYbJdWcu6sxQcq9v1LI0JxcfkiKYkMUzJSkVQ=
git.novox.be/novox/mesh-sdk/go v0.1.6/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
+94
View File
@@ -0,0 +1,94 @@
{
"module": "docker",
"version": "1",
"capabilities": [
"package-manager",
"service-manager",
"privileged"
],
"claims": [
{
"name": "node-container-runtime",
"scope": "node"
}
],
"tools": [
"docker_list",
"docker_inspect",
"docker_logs",
"docker_stats",
"docker_start",
"docker_stop",
"docker_restart",
"docker_top",
"docker_images",
"docker_prune",
"docker_disk_usage",
"docker_networks",
"docker_volumes",
"docker_events",
"docker_daemon_config",
"docker_unlabelled",
"docker_problems",
"docker_ports"
],
"resources": [
{
"id": "package",
"type": "package",
"package": "docker"
},
{
"id": "buildx",
"type": "package",
"package": "docker-buildx"
},
{
"id": "socket",
"type": "service",
"unit": "docker.socket",
"state": "running",
"boot": "enabled"
},
{
"id": "prune-service",
"type": "file",
"path": "/etc/systemd/system/docker-prune.service",
"mode": "0644",
"content": "# Generated by the mesh. Do not edit — module docker writes this file and replaces it at every push.\n[Unit]\nDescription=Prune dangling images and unused build cache (the mesh's docker module)\n# Never volumes, never a container, never an image a container uses: dangling\n# images and build cache nothing refers to, unused for a week. What a person\n# prunes beyond that is docker_prune's, by hand.\nAfter=docker.service\nConditionPathExists=/run/docker.sock\n\n[Service]\nType=oneshot\nNice=19\nIOSchedulingClass=idle\nExecStart=/usr/bin/docker image prune --force --filter until=168h\nExecStart=/usr/bin/docker builder prune --force --filter until=168h\n"
},
{
"id": "prune-timer",
"type": "file",
"path": "/etc/systemd/system/docker-prune.timer",
"mode": "0644",
"content": "# Generated by the mesh. Do not edit — module docker writes this file and replaces it at every push.\n[Unit]\nDescription=Weekly prune of dangling images and unused build cache (the mesh's docker module)\n\n[Timer]\nOnCalendar=weekly\nRandomizedDelaySec=1h\nPersistent=true\n\n[Install]\nWantedBy=timers.target\n"
},
{
"id": "prune",
"type": "service",
"unit": "docker-prune.timer",
"state": "running",
"boot": "enabled",
"restart-on": [
"prune-service",
"prune-timer"
]
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/docker-tools",
"binary": "docker-tools",
"loads": [
"docker-tools"
]
}
]
}
}
+1 -1
View File
@@ -68,7 +68,7 @@
"type": "file",
"path": "${dir:state}/api.env",
"mode": "0600",
"content": "NODE_ENV=production\nPORT=9000\nMONGO_URL=mongodb://${bound:mongodb-database:as}:${secret:mongodb-database}@${bound:mongodb-database:at}:${bound:mongodb-database:port}/${bound:mongodb-database:as}?authSource=${bound:mongodb-database:as}\nMONGO_DB=${bound:mongodb-database:as}\nMINIO_BUCKET=mesh-novox-invoice\nMINIO_ENDPOINT=${bound:s3-bucket:at}\nMINIO_PORT=${bound:s3-bucket:port}\nMINIO_ACCESSKEY=${bound:s3-bucket:as}\nMINIO_SECRET=${secret:s3-bucket}\n"
"content": "NODE_ENV=production\nPORT=9000\nMONGO_URL=mongodb://${bound:mongodb-database:as}:${secret:mongodb-database}@${bound:mongodb-database:at}:${bound:mongodb-database:port}/${bound:mongodb-database:as}?authSource=${bound:mongodb-database:as}\nMONGO_DB=${bound:mongodb-database:as}\nMINIO_BUCKET=${bound:s3-bucket:bucket}\nMINIO_ENDPOINT=${bound:s3-bucket:at}\nMINIO_PORT=${bound:s3-bucket:port}\nMINIO_ACCESSKEY=${bound:s3-bucket:as}\nMINIO_SECRET=${secret:s3-bucket}\n"
},
{
"id": "net",
-55
View File
@@ -1,55 +0,0 @@
# mesh-catalog's runtime: the tool runtime, carrying the catalogue's compiled graph, its consumer
# of what the builder announces, and its tools.
#
# **Built from this module's own directory and nothing else.** The sdk is in the base image, so
# nothing is copied out of a neighbouring checkout — which is what lets the mesh build this from a
# repository and a path (novox/hq ADR 0069) rather than only on a workstation with the siblings.
#
# Two bases, named rather than pinned: the image this is COMPILED in, and the image it RUNS in.
# They are different images on purpose — the first carries a compiler and the second must not, or
# every running container would carry one it never invokes. The mesh answers both with the copies it
# holds, because a fingerprint written here would name one particular copy and no other mesh has it
# (novox/hq issue 044). Declared in module.json's `build.on`; deliberately no defaults, so a build
# nobody told stops here and says which module to build first.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the sdk it will run against.
WORKDIR /app/modules/mesh-catalog
COPY . .
# The compiler is invoked by its real path rather than through node_modules/.bin, whose entries are
# symlinks to a launcher that requires its library relatively — resolved away when the base image
# was assembled.
RUN node /app/node_modules/typescript/bin/tsc pg.d.ts store.ts index.ts tools/index.ts prepare/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
# **A module may need something the base image does not carry.** The base holds what every module
# needs — the sdk, the broker client — and a postgres driver is not that: the one other module that
# reaches a database shells out to psql instead. So the catalogue brings its own.
#
# Installed into an empty directory rather than into the module's, because the module's package.json
# also names `@novox/mesh-sdk`, which is not on any registry — it is in the base image. Asking npm to
# resolve this module's dependencies would therefore fail on the one it already has.
RUN mkdir -p /deps && cd /deps && \
npm install --omit=dev --no-audit --no-fund --no-package-lock pg@8
FROM ${RUNTIME_BASE}
COPY --from=build /app/modules/mesh-catalog/dist /app/modules/mesh-catalog/dist
# Beside the compiled code, so `pg` resolves from it while `@novox/mesh-sdk` keeps walking up to the
# base image's own node_modules — the module gets its extra dependency without shadowing the sdk it
# was compiled against.
COPY --from=build /deps/node_modules /app/modules/mesh-catalog/node_modules
# Both entrypoints, loaded in serve mode.
#
# **A consumer cannot be started with `run`.** That mode imports an entrypoint without binding a
# broker — it is for a step that does its work offline and exits — and the catalogue's whole job is
# to listen for what the builder announces. Serve binds the broker first, then imports these, so
# `on()` has something to subscribe to.
ENV MESH_TOOL_MODULES=/app/modules/mesh-catalog/dist/index.js,/app/modules/mesh-catalog/dist/tools/index.js
# And what prepares this module's state, for the runtime's `prepare` mode (novox/hq ADR 0135). Named
# here, beside the entrypoints above, because the module knows which of its files prepares its state
# and nothing else could: the mesh asks one word and this says what answers it.
ENV MESH_PREPARE=/app/modules/mesh-catalog/dist/prepare/index.js
+24 -36
View File
@@ -25,9 +25,6 @@
"secrets": {
"postgres-database": "${dir:state}/database.secret"
},
"own-secrets": {
"broker": "${dir:mesh-state}/broker"
},
"consumes": [
"mesh-build-machine.built",
"mesh-controller.built-before"
@@ -38,14 +35,7 @@
"rebuild-needed",
"catching-up"
],
"prepares": true,
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -60,43 +50,41 @@
"content": "postgresql://${bound:postgres-database:as}:${secret:postgres-database}@${bound:postgres-database:at}:${bound:postgres-database:port}/${bound:postgres-database:as}\n"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-catalog",
"network": "host",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:state}:/run/state",
"${dir:state}/database.url:/run/secrets/database-url:ro"
"id": "prepare",
"type": "process",
"name": "mesh-catalog-prepare",
"artifact": "code",
"run": [
"node",
"prepare/index.js"
],
"run-once": true,
"env": {
"MESH_BROKER_FILE": "/run/secrets/broker",
"DATABASE_URL_FILE": "/run/secrets/database-url"
"DATABASE_URL_FILE": "${dir:state}/database.url"
},
"artifact": "runtime",
"restart-on": [
"database-url"
]
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "code",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"index.js",
"tools/index.js",
"prepare/index.js"
],
"loads": [
"index.js",
"tools/index.js"
],
"env": {
"DATABASE_URL_FILE": "${dir:state}/database.url"
}
}
]
}
+3 -3
View File
@@ -1,9 +1,9 @@
// Ambient types for `pg` (node-postgres), which ships its types only via the separate `@types/pg`
// package. Rather than pull that in at tsc time, this declares the exact slice model-usage uses —
// the same precedent anthropic-manager sets for `tweetnacl-sealedbox-js` (a local ambient .d.ts,
// listed in tsconfig `include`, default-imported). The real `pg` is installed into the module's
// runtime image (package.json `dependencies`; novox/hq ADR 0052), so this types the code without
// deciding what runs.
// listed in tsconfig `include`, default-imported). The real `pg` is the package.json dependency the
// builder installs and inlines into the module's bundle (novox/hq ADR 0198 §4), so this types the
// code without deciding what runs.
declare module "pg" {
/** One checked-out connection. Needed because registering a module-version and its edges is one
* act: a half-written registration is a graph that lies about what something was built against. */
+3 -2
View File
@@ -7,8 +7,9 @@
// nothing anywhere said so.
//
// Nothing here connects to the broker. Preparation runs before the version that would use it, so
// there is nothing yet to talk to; the runtime's `prepare` mode imports this and awaits it, and this
// process exiting non-zero is how the host knows not to start the runtime.
// there is nothing yet to talk to: the host runs this file as a run-once process, with the module's
// words and no bus (novox/hq ADR 0198 §3), before the node's runtime is started with the version
// that needs it, and this process exiting non-zero is how the host knows the step did not complete.
import { Graph } from "../store.js";
const graph = Graph.fromEnv();
+5 -15
View File
@@ -303,21 +303,11 @@ export class MinioClient {
// --- module-scoped helpers -------------------------------------------------
/** A deterministic 20-char access key id from a consumer name, so removal needs no stored state:
* the provisioner recomputes the same id at teardown that it minted at creation. */
export function accessKeyFor(consumer: string): string {
const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
const digest = createHash("sha256").update(consumer).digest();
let out = "";
for (let i = 0; i < 20; i++) out += chars[digest[i] % chars.length];
return out;
}
/** A DNS-safe bucket name derived from a consumer — the removable identity of its storage. */
export function bucketFor(consumer: string): string {
const name = consumer.toLowerCase().replace(/[^a-z0-9-]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 63);
return name.length >= 3 ? name : `mesh-${name}`;
}
// **Neither the access key nor the bucket is derived here any more.** `accessKeyFor` minted an id
// of its own until the mesh took that over (ADR 0048: the login is the mesh's, handed to both
// ends), and `bucketFor` derived the bucket until the mesh took that over too (ADR 0201: the rule
// is a line of this module's manifest, filled per consumer and delivered to both ends). Both
// survived with no callers, which is the state a rule comes back from; they are gone.
function bucketPolicy(bucket: string): string {
return JSON.stringify({
+2 -1
View File
@@ -49,7 +49,8 @@
"s3-bucket": {
"scheme": "http",
"region": "eu-west",
"port": 9000
"port": 9000,
"bucket": "${consumer:as:dns}"
}
},
"receives": {
+1 -1
View File
@@ -5,7 +5,7 @@
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
"@novox/mesh-sdk": "^0.1.7"
},
"devDependencies": {
"@types/node": "^22.0.0",
+31 -7
View File
@@ -10,18 +10,24 @@
// **The access key and its secret are the mesh's, not the provisioner's (ADR 0048).** The mesh
// derives the login (the access-key id) and hands it to both ends, and mints the secret key. minio
// creates the service account under exactly that access key with exactly that secret — a credential
// the provisioner invented is one the consumer could never present. The bucket is derived from the
// login, so teardown recomputes it with nothing to persist.
// the provisioner invented is one the consumer could never present.
//
// **The bucket name is the mesh's too (ADR 0201).** It used to be computed here, from the login,
// and every consumer transcribed the same rule into its own definition by hand — two copies of
// one rule with nothing comparing them, and one of three was wrong for months. Now the rule is a
// line of this module's manifest (`serves.s3-bucket.bucket: ${consumer:as:dns}`), the mesh fills
// it per consumer, and the same filled value reaches this provisioner and the consumer's own
// configuration. There is no second computation to disagree with.
import { runProvisioner, type Provision } from "@novox/mesh-sdk/provisioner";
import { emit } from "@novox/mesh-sdk/events";
import { MinioClient, bucketFor } from "../client.js";
import { MinioClient } from "../client.js";
const minio = MinioClient.fromEnv();
runProvisioner("s3-bucket", {
async create(p: Provision): Promise<void> {
const bucket = bucketFor(p.as);
const bucket = bucketNamed(p.derived);
const accessKeyId = p.as;
if (!(await minio.bucketExists(bucket))) await minio.createBucket(bucket);
@@ -38,8 +44,8 @@ runProvisioner("s3-bucket", {
});
},
async remove(p: { as: string }): Promise<void> {
const bucket = bucketFor(p.as);
async remove(p: { as: string; derived: Readonly<Record<string, unknown>> }): Promise<void> {
const bucket = bucketNamed(p.derived);
// Revoking the key is what cuts the consumer's access. The bucket is emptied-then-dropped only if
// empty; a bucket that still holds objects is left for an operator rather than erroring on every
@@ -57,10 +63,28 @@ runProvisioner("s3-bucket", {
// Asked every minute by the harness: whether the backend still holds this consumer exactly as
// the mesh gave it, so a login lost behind the provisioner's back is made again (novox/hq issue 120).
async holds(p: Provision): Promise<boolean> {
return minio.canReachAs(bucketFor(p.as), p.as, p.password);
return minio.canReachAs(bucketNamed(p.derived), p.as, p.password);
},
});
/** The bucket the mesh derived for this consumer.
*
* Absent means this module is running against a control plane that does not fill `${consumer:…}`
* yet, or a manifest whose `serves` block lost the line. Both are the same mistake from here —
* nobody said which bucket — and both are said rather than guessed: a provisioner that fell back
* to deriving one would restore the second rule and hide the fault behind a bucket that happens
* to be right. */
function bucketNamed(derived: Readonly<Record<string, unknown>>): string {
const bucket = derived.bucket;
if (typeof bucket !== "string" || bucket === "") {
throw new Error(
"the mesh did not say which bucket this consumer gets: minio's manifest must serve " +
"`bucket` under s3-bucket (novox/hq ADR 0201)",
);
}
return bucket;
}
/** Emit best-effort: a broker hiccup is logged and dropped, never allowed to throw back and fail a
* bucket that was made. */
async function announce(type: string, body: unknown): Promise<void> {
-37
View File
@@ -1,37 +0,0 @@
# mongodb's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the sdk it will run against. The compiler
# is invoked by its real path: node_modules/.bin entries are launcher symlinks the base image
# resolved away.
WORKDIR /app/modules/mongodb
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts provisioner/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
FROM ${RUNTIME_BASE}
# mongodb's client shells out to `mongosh`, installed from MongoDB's own apt repo so its shared
# libraries come with it — copying the bare binary out of the mongo image leaves it unable to load.
RUN apt-get update && apt-get install -y --no-install-recommends gnupg curl ca-certificates \
&& curl -fsSL https://pgp.mongodb.com/server-7.0.asc | gpg --dearmor -o /usr/share/keyrings/mongodb.gpg \
&& echo "deb [signed-by=/usr/share/keyrings/mongodb.gpg] https://repo.mongodb.org/apt/debian bookworm/mongodb-org/7.0 main" > /etc/apt/sources.list.d/mongodb.list \
&& apt-get update && apt-get install -y --no-install-recommends mongodb-mongosh \
&& rm -rf /var/lib/apt/lists/*
COPY --from=build /app/modules/mongodb/dist /app/modules/mongodb/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled. A container that instead ran only its
# provisioner (`run`) served no tools and emitted no events; a container that named no command
# ran no provisioner at all.
ENV MESH_TOOL_MODULES=/app/modules/mongodb/dist/index.js,/app/modules/mongodb/dist/tools/index.js,/app/modules/mongodb/dist/provisioner/index.js
+70 -79
View File
@@ -1,19 +1,16 @@
// mongodb's admin client — mongodb's own code, living in the module (novox/hq ADR 0039). Both this
// module's tools and its provisioner import it, and nothing outside mongodb does.
//
// Commands run through `mongosh`, not a wire-protocol driver: the module may take NO npm dependency
// beyond @novox/mesh-sdk, and hand-rolling the MongoDB wire protocol + SCRAM auth is more surface
// than this should carry — so it shells out to the shell the mongodb image ships, the same way
// postgres drives itself through `psql`, minio through `mc` and mailu through doveadm. One boundary,
// `evalJs()`, and every method is built on it: a snippet of JavaScript is evaluated server-side and
// its result comes back as EJSON on stdout.
// **The backend's own driver, inside the bundle** (novox/hq ADR 0198 §4). This used to shell out to
// `mongosh`, which the module's container installed from MongoDB's apt repository; the module's code
// now runs in the node's runtime, on machines whose system carries no mongosh, so it speaks to the
// server through the official `mongodb` driver its package.json names — installed and inlined into
// the bundle by the builder. One connection per call, as one mongosh invocation was: the module is
// called rarely, and a pool held open across calls would hold a credential the mesh may rotate.
import { randomBytes } from "node:crypto";
import { readFileSync } from "node:fs";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
const run = promisify(execFile);
import { MongoClient as Driver, MongoServerError, BSON, type Document } from "mongodb";
export interface DatabaseInfo {
readonly name: string;
@@ -59,32 +56,26 @@ export class MongoClient {
return this.conn.port;
}
/** The admin connection URI mongosh authenticates with, credentials percent-encoded. */
/** The admin connection URI, credentials percent-encoded. */
private uri(): string {
const u = encodeURIComponent(this.conn.user);
const p = encodeURIComponent(this.conn.password);
const a = encodeURIComponent(this.conn.authSource);
return `mongodb://${u}:${p}@${this.conn.host}:${this.conn.port}/?authSource=${a}`;
return `mongodb://${u}:${p}@${this.conn.host}:${this.conn.port}/?authSource=${a}&directConnection=true`;
}
/**
* Evaluate a JavaScript snippet server-side through `mongosh` and parse the JSON it prints (see
* header). The snippet MUST `print()` exactly one JSON document as its only stdout — every method
* below ends in `print(EJSON.stringify(...))`. `--quiet` suppresses the shell banner so stdout is
* the JSON alone; a non-zero exit (auth failure, bad command) rejects here rather than returning
* a partial success.
* The one execution boundary: connect as the administrator, do `work`, and close — a failure to
* connect or to authenticate rejects here rather than returning a partial success.
*/
async evalJs<T>(js: string): Promise<T> {
const { stdout } = await run(
"mongosh",
[this.uri(), "--quiet", "--eval", js],
{ maxBuffer: 16 << 20 },
);
const text = stdout.trim();
if (text.length === 0) {
throw new Error("mongosh returned no output — the eval printed nothing");
private async admin<T>(work: (client: Driver) => Promise<T>): Promise<T> {
const client = new Driver(this.uri(), { serverSelectionTimeoutMS: 10_000 });
try {
await client.connect();
return await work(client);
} finally {
await client.close();
}
return JSON.parse(text) as T;
}
/**
@@ -94,19 +85,16 @@ export class MongoClient {
* password and roles, so a rotated credential converges.
*/
async createDatabaseAndUser(database: string, user: string, password: string): Promise<void> {
const js = `
const target = db.getSiblingDB(${lit(database)});
let existing = null;
try { existing = target.getUser(${lit(user)}); } catch (e) { existing = null; }
const roles = [{ role: "dbOwner", db: ${lit(database)} }];
if (existing) {
target.updateUser(${lit(user)}, { pwd: ${lit(password)}, roles: roles });
} else {
target.createUser({ user: ${lit(user)}, pwd: ${lit(password)}, roles: roles });
}
print(EJSON.stringify({ ok: 1 }));
`;
await this.evalJs<{ ok: number }>(js);
await this.admin(async (client) => {
const target = client.db(database);
const roles = [{ role: "dbOwner", db: database }];
const found = await target.command({ usersInfo: user });
if (Array.isArray(found.users) && found.users.length > 0) {
await target.command({ updateUser: user, pwd: password, roles });
} else {
await target.command({ createUser: user, pwd: password, roles });
}
});
}
/**
@@ -115,45 +103,43 @@ print(EJSON.stringify({ ok: 1 }));
* authentication failure or a missing role; an unreachable server rejects (novox/hq issue 120).
*/
async canAuthenticateAs(database: string, user: string, password: string): Promise<boolean> {
// Connected without credentials, then authenticated inside the eval from the environment, so
// the consumer's password is neither on argv nor in the message of a failed command.
const uri = `mongodb://${this.conn.host}:${this.conn.port}/?serverSelectionTimeoutMS=10000`;
const js =
"const t = db.getSiblingDB(process.env.MESH_HOLDS_DB);" +
"t.auth(process.env.MESH_HOLDS_USER, process.env.MESH_HOLDS_PW);" +
"print(EJSON.stringify(t.runCommand({ connectionStatus: 1 }).authInfo.authenticatedUserRoles))";
let stdout: string;
// Credentials as options, never in a URI, so the consumer's password is in no message a failed
// connection prints.
const client = new Driver(`mongodb://${this.conn.host}:${this.conn.port}/?directConnection=true`, {
auth: { username: user, password },
authSource: database,
serverSelectionTimeoutMS: 10_000,
});
try {
({ stdout } = await run("mongosh", [uri, "--quiet", "--eval", js], {
env: { ...process.env, MESH_HOLDS_DB: database, MESH_HOLDS_USER: user, MESH_HOLDS_PW: password },
timeout: 30_000,
}));
await client.connect();
const status = await client.db(database).command({ connectionStatus: 1 });
const roles = (status.authInfo?.authenticatedUserRoles ?? []) as { role: string; db: string }[];
return roles.some((r) => r.role === "dbOwner" && r.db === database);
} catch (err) {
const text = `${(err as { stderr?: string }).stderr ?? ""}${(err as { stdout?: string }).stdout ?? ""}`;
if (/Authentication failed|AuthenticationFailed/i.test(text)) return false;
throw new Error(`mongosh could not check ${user}: ${text.trim().slice(0, 500) || String((err as Error).message).split("\n")[0]}`);
if (isAuthFailure(err)) return false;
throw new Error(`mongodb could not check ${user}: ${String((err as Error).message).split("\n")[0]}`);
} finally {
await client.close();
}
const roles = JSON.parse(stdout.trim()) as { role: string; db: string }[];
return roles.some((r) => r.role === "dbOwner" && r.db === database);
}
/** Drop a database and its owning user, idempotently. Dropping the database evicts its data; the
* user is removed first so a re-grant of the same login starts clean. */
async dropDatabaseAndUser(database: string, user: string): Promise<void> {
const js = `
const target = db.getSiblingDB(${lit(database)});
try { target.dropUser(${lit(user)}); } catch (e) {}
target.dropDatabase();
print(EJSON.stringify({ ok: 1 }));
`;
await this.evalJs<{ ok: number }>(js);
await this.admin(async (client) => {
const target = client.db(database);
try {
await target.command({ dropUser: user });
} catch (err) {
if (!(err instanceof MongoServerError && err.code === 11)) throw err; // 11: UserNotFound
}
await target.dropDatabase();
});
}
/** List the databases on the server, with on-disk size, for the mongodb_list_databases tool. */
async listDatabases(): Promise<DatabaseInfo[]> {
const res = await this.evalJs<{ databases: { name: string; sizeOnDisk?: number }[] }>(
`print(EJSON.stringify(db.adminCommand({ listDatabases: 1 })));`,
);
const res = await this.admin((client) => client.db("admin").admin().listDatabases());
return (res.databases ?? [])
.map((d) => ({ name: String(d.name), sizeBytes: Number(d.sizeOnDisk ?? 0) }))
.sort((a, b) => a.name.localeCompare(b.name));
@@ -162,6 +148,8 @@ print(EJSON.stringify({ ok: 1 }));
/**
* Run a read-only `find` against a collection in a named database, for the mongodb_query tool.
* `find` mutates nothing; the limit is capped so a tool call cannot stream an unbounded result.
* Documents come back as relaxed Extended JSON — an ObjectId as `{"$oid": …}` — exactly as the
* shell's `EJSON.stringify` rendered them before.
*/
async find(
database: string,
@@ -170,26 +158,29 @@ print(EJSON.stringify({ ok: 1 }));
limit: number,
): Promise<Record<string, unknown>[]> {
const capped = Math.max(1, Math.min(limit, 1000));
const js =
`print(EJSON.stringify(` +
`db.getSiblingDB(${lit(database)}).getCollection(${lit(collection)})` +
`.find(${JSON.stringify(filter)}).limit(${capped}).toArray()` +
`));`;
return this.evalJs<Record<string, unknown>[]>(js);
const docs = await this.admin((client) =>
client
.db(database)
.collection(collection)
.find(BSON.EJSON.deserialize(filter as Document, { relaxed: true }) as Document)
.limit(capped)
.toArray(),
);
return BSON.EJSON.serialize(docs, { relaxed: true }) as Record<string, unknown>[];
}
}
/** An authentication failure, as the server or the driver reports it. */
function isAuthFailure(err: unknown): boolean {
if (err instanceof MongoServerError && err.code === 18) return true; // 18: AuthenticationFailed
return /Authentication failed|AuthenticationFailed/i.test(String((err as Error)?.message ?? ""));
}
/** Generate a URL-safe password. */
export function generatePassword(): string {
return randomBytes(24).toString("base64url");
}
/** Embed a value as a JavaScript literal inside a mongosh snippet — JSON.stringify escapes quotes,
* backslashes and control characters, so a string cannot break out of the snippet. */
function lit(val: unknown): string {
return JSON.stringify(val);
}
function readSecretFile(path: string | undefined): string | undefined {
if (!path) return undefined;
try {
+3 -3
View File
@@ -1,9 +1,9 @@
// mongodb's events entrypoint, loaded by the per-node tool host (the provisioner container runs
// ./provisioner separately). The database lifecycle events are EMITTED from the provisioner, where
// mongodb's events entrypoint, launched by the node's runtime beside its tools and provisioner
// (novox/hq ADR 0198). The database lifecycle events are EMITTED from the provisioner, where
// the lifecycle actually happens (novox/hq ADR 0041/0042):
// module.mongodb.database.provisioned — a consumer's database + owning user was created
// module.mongodb.database.deprovisioned — that database was removed
// Here in the tool host we react to them, keeping a lightweight audit trail of who was granted a
// Here in the runtime we react to them, keeping a lightweight audit trail of who was granted a
// database and who lost one — observability the provider itself is best placed to log.
import { on } from "@novox/mesh-sdk/events";
+28 -43
View File
@@ -39,17 +39,9 @@
"mongodb-database": "${dir:grants}"
},
"own-secrets": {
"root": "${dir:state}/root.secret",
"broker": "${dir:mesh-state}/broker"
"root": "${dir:state}/root.secret"
},
"secrets-owner": "999:999",
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -71,6 +63,14 @@
"type": "network",
"name": "mongodb"
},
{
"id": "server-root",
"type": "file",
"path": "${dir:state}/server-root.secret",
"mode": "0400",
"owner": "999:999",
"content": "${secret:root}"
},
{
"id": "server",
"type": "container",
@@ -86,46 +86,31 @@
],
"volumes": [
"${dir:data}:/data/db",
"${dir:state}/root.secret:/run/secrets/root:ro"
"${dir:state}/server-root.secret:/run/secrets/root:ro"
]
},
{
"id": "runtime",
"type": "container",
"name": "mesh-mongodb",
"network": "mongodb",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:grants}:${dir:grants}:ro",
"${dir:state}/root.secret:/run/secrets/root:ro"
],
"env": {
"MESH_PROVISION_MONGODB": "mongodb://root@mongodb-server:27017/admin?authSource=admin",
"MESH_PROVISION_PASSWORD_FILE": "/run/secrets/root",
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_RECEIVES": "${dir:grants}/mesh.json"
},
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "code",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"loads": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"env": {
"MESH_PROVISION_MONGODB": "mongodb://root@127.0.0.1:${port:27017}/admin?authSource=admin",
"MESH_PROVISION_PASSWORD_FILE": "${dir:state}/root.secret",
"MESH_RECEIVES": "${dir:grants}/mesh.json"
}
}
]
}
+2 -1
View File
@@ -5,7 +5,8 @@
"type": "module",
"private": true,
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
"@novox/mesh-sdk": "^0.1.1",
"mongodb": "^6.21.0"
},
"devDependencies": {
"@types/node": "^22.0.0",
+1 -2
View File
@@ -11,8 +11,7 @@
// same-named database under exactly that login — a name the consumer cannot learn is a database it
// cannot reach.
//
// The commands run through MongoClient.evalJs(), which is the module's one execution boundary (see
// client.ts).
// The commands run through MongoClient, the official driver inside this bundle (see client.ts).
import { runProvisioner, type Provision } from "@novox/mesh-sdk/provisioner";
import { emit } from "@novox/mesh-sdk/events";
+2 -2
View File
@@ -1,6 +1,6 @@
// mongodb's tools — mongodb's own code (novox/hq ADR 0039), importing mongodb's own client. They
// return structured data; the mesh serves them through the sdk's tool harness. Both call through
// MongoClient.evalJs(), the module's one execution boundary (see client.ts).
// return structured data; the mesh serves them through the sdk's tool harness. Both call the server
// through MongoClient, the driver inside this bundle (see client.ts).
import { registerModuleTools, type ToolDefinition } from "@novox/mesh-sdk/tools";
import { MongoClient } from "../client.js";
+36 -4
View File
@@ -18,6 +18,7 @@ import { randomBytes } from "node:crypto";
import { connect as tcpConnect } from "node:net";
import { readFileSync } from "node:fs";
import { execFile } from "node:child_process";
import { basename, dirname } from "node:path";
import { promisify } from "node:util";
import { missingAcls, parseRoleAcls, staleAcls, wantedAcls } from "./topics.js";
@@ -30,6 +31,14 @@ export interface MqttConn {
/** The Dynamic Security admin client the runtime authenticates as. */
readonly adminUser: string;
readonly adminPassword: string;
/**
* The broker's own container, when `mosquitto_ctrl` is run inside it rather than from this
* machine's packages. The broker's image carries the tool at the broker's version, and inside it
* the broker listens on 127.0.0.1:1883 whatever port the machine publishes.
*/
readonly container?: string;
/** The broker's image, to seed the security file before the broker has ever started. */
readonly image?: string;
}
export class MosquittoClient {
@@ -53,7 +62,11 @@ export class MosquittoClient {
"mosquitto host or admin password is not set — mosquitto's own code cannot reach the broker",
);
}
return new MosquittoClient({ host, port, adminUser, adminPassword: adminPassword ?? "" });
return new MosquittoClient({
host, port, adminUser, adminPassword: adminPassword ?? "",
container: env.MESH_MQTT_CTRL_CONTAINER || undefined,
image: env.MESH_MQTT_CTRL_IMAGE || undefined,
});
}
get host(): string {
@@ -84,16 +97,18 @@ export class MosquittoClient {
* to this single-purpose runtime container; see the module README.
*/
async ctl(...args: string[]): Promise<string> {
const inside = this.conn.container !== undefined;
const base = [
"-h", this.conn.host,
"-p", String(this.conn.port),
"-h", inside ? "127.0.0.1" : this.conn.host,
"-p", inside ? "1883" : String(this.conn.port),
"-u", this.conn.adminUser,
"-P", this.conn.adminPassword,
];
let stdout: string;
let stderr: string;
try {
({ stdout, stderr } = await run("mosquitto_ctrl", [...base, "dynsec", ...args], {
const [command, argv] = this.ctrl([...base, "dynsec", ...args]);
({ stdout, stderr } = await run(command, argv, {
maxBuffer: 16 << 20,
timeout: 30_000,
}));
@@ -240,10 +255,27 @@ export class MosquittoClient {
async initBootstrapFile(configFile: string): Promise<void> {
// `dynsec init <file> <admin-username> [admin-password]` is an offline file operation — it does
// not connect to the broker. The password is a positional argument (omitting it prompts).
if (this.conn.image) {
// Before the broker has ever started there is no container to enter: a throwaway one from the
// broker's own image writes the file into the directory the broker will mount.
await run("docker", [
"run", "--rm", "--entrypoint", "mosquitto_ctrl",
"-v", `${dirname(configFile)}:/mosquitto/data`,
this.conn.image,
"dynsec", "init", `/mosquitto/data/${basename(configFile)}`, this.conn.adminUser, this.conn.adminPassword,
], { maxBuffer: 16 << 20 });
return;
}
await run("mosquitto_ctrl", ["dynsec", "init", configFile, this.conn.adminUser, this.conn.adminPassword], {
maxBuffer: 16 << 20,
});
}
/** How `mosquitto_ctrl` is run here: inside the broker's container when one is named. */
ctrl(argv: string[]): [string, string[]] {
if (this.conn.container) return ["docker", ["exec", this.conn.container, "mosquitto_ctrl", ...argv]];
return ["mosquitto_ctrl", argv];
}
}
/** Generate a URL-safe password with no argv- or MQTT-hostile characters. */
+3 -7
View File
@@ -92,7 +92,7 @@
"type": "file",
"path": "${dir:state}/bootstrap.env",
"mode": "0600",
"content": "MESH_PROVISION_MQTT=127.0.0.1:${port:1883}\nMESH_PROVISION_ADMIN_USER=mesh-admin\nMESH_PROVISION_PASSWORD_FILE=${dir:mesh-state}/admin\nMESH_DYNSEC_FILE=${dir:data}/dynamic-security.json\n"
"content": "MESH_PROVISION_MQTT=127.0.0.1:${port:1883}\nMESH_PROVISION_ADMIN_USER=mesh-admin\nMESH_PROVISION_PASSWORD_FILE=${dir:mesh-state}/admin\nMESH_DYNSEC_FILE=${dir:data}/dynamic-security.json\nMESH_MQTT_CTRL_IMAGE=eclipse-mosquitto@sha256:38c0da4f2ef84284d47b3b3eeea1cb3bdeabe81ee10caf0cd5c5ff61ee3ea408\n"
},
{
"id": "bootstrap",
@@ -125,11 +125,6 @@
"${dir:data}:/mosquitto/data",
"${dir:state}/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro"
]
},
{
"id": "client",
"type": "package",
"package": "mosquitto"
}
],
"build": {
@@ -153,7 +148,8 @@
"MESH_RECEIVES": "${dir:grants}/mesh.json",
"MESH_PROVISION_MQTT": "127.0.0.1:${port:1883}",
"MESH_PROVISION_ADMIN_USER": "mesh-admin",
"MESH_PROVISION_PASSWORD_FILE": "${dir:mesh-state}/admin"
"MESH_PROVISION_PASSWORD_FILE": "${dir:mesh-state}/admin",
"MESH_MQTT_CTRL_CONTAINER": "mosquitto"
}
}
]
+19
View File
@@ -0,0 +1,19 @@
// Run after `npm run build`.
// mosquitto_ctrl runs inside the broker's own container when the manifest names it, so a machine
// needs no mosquitto package (whose index may be too stale to install from) and the tool always
// matches the broker's version.
import assert from "node:assert/strict";
import { test } from "node:test";
import { MosquittoClient } from "../dist/client.js"; // compiled: client.ts uses parameter properties, which type stripping cannot run
const env = { MESH_PROVISION_MQTT: "127.0.0.1:21883", MESH_MQTT_PASSWORD: "pw" };
test("named, the broker's container runs mosquitto_ctrl", () => {
const c = MosquittoClient.fromEnv({ ...env, MESH_MQTT_CTRL_CONTAINER: "mosquitto" });
assert.deepEqual(c.ctrl(["dynsec", "listClients"]), ["docker", ["exec", "mosquitto", "mosquitto_ctrl", "dynsec", "listClients"]]);
});
test("unnamed, this machine's mosquitto_ctrl runs", () => {
const c = MosquittoClient.fromEnv(env);
assert.deepEqual(c.ctrl(["dynsec", "listClients"]), ["mosquitto_ctrl", ["dynsec", "listClients"]]);
});
-43
View File
@@ -1,43 +0,0 @@
# mssql's runtime: the tool runtime, carrying this module's compiled code.
#
# **Built from this module's own directory and nothing else.** The sdk and the tool runtime are in
# the base images, published like any other artifact — which is what makes this buildable by the
# mesh from a repository and a path (novox/hq ADR 0069) rather than only on a workstation that
# happens to have the siblings.
#
# Two bases, named rather than pinned (novox/hq issue 044): the image this is COMPILED in and the
# image it RUNS in — the second must not carry a compiler. Declared in module.json's `build.on`.
ARG BUILD_BASE
ARG RUNTIME_BASE
FROM ${BUILD_BASE} AS build
# Compiled under /app/modules so `@novox/mesh-sdk` resolves upward into the base's own
# node_modules — the module is compiled against exactly the sdk it will run against. The compiler
# is invoked by its real path: node_modules/.bin entries are launcher symlinks the base image
# resolved away.
WORKDIR /app/modules/mssql
COPY . .
RUN node /app/node_modules/typescript/bin/tsc client.ts index.ts tools/index.ts provisioner/index.ts \
--module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist
# **sqlcmd, which this module's client drives, has to be here** — it never was, so every tool failed
# with `spawn sqlcmd ENOENT`. go-sqlcmd is one static binary; fetched at a pinned release and checked
# against its digest, so a build that receives anything else stops here.
FROM ${BUILD_BASE} AS sqlcmd
ARG SQLCMD_VERSION=v1.10.0
ARG SQLCMD_SHA256=92516d98c63d99b0994de5b61350c91f6915f9b76f139a59039fbcb225c2e987
RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates bzip2 \
&& curl -fsSL -o /tmp/sqlcmd.tar.bz2 \
"https://github.com/microsoft/go-sqlcmd/releases/download/${SQLCMD_VERSION}/sqlcmd-linux-amd64.tar.bz2" \
&& echo "${SQLCMD_SHA256} /tmp/sqlcmd.tar.bz2" | sha256sum -c - \
&& tar -xjf /tmp/sqlcmd.tar.bz2 -C /usr/local/bin sqlcmd
FROM ${RUNTIME_BASE}
COPY --from=sqlcmd /usr/local/bin/sqlcmd /usr/local/bin/sqlcmd
COPY --from=build /app/modules/mssql/dist /app/modules/mssql/dist
# Every serve-time entrypoint, loaded by the runtime in serve mode: tools and events serve, and a
# provider's provisioner runs its reconcile loop in the same process, with the broker connected —
# the convention novox/hq issues 060/061 settled. A container that instead ran only its
# provisioner (`run`) served no tools and emitted no events; a container that named no command
# ran no provisioner at all.
ENV MESH_TOOL_MODULES=/app/modules/mssql/dist/index.js,/app/modules/mssql/dist/tools/index.js,/app/modules/mssql/dist/provisioner/index.js
+111 -98
View File
@@ -1,22 +1,74 @@
// mssql's admin client — mssql's own code, living in the module (novox/hq ADR 0039). Both this
// module's tools and its provisioner import it, and nothing outside mssql does.
//
// SQL is executed through `sqlcmd`, not a wire-protocol driver: the module may take NO npm
// dependency beyond @novox/mesh-sdk, and hand-rolling the TDS handshake, pre-login and query
// protocol is more surface than this should carry — so it shells out to the client the mssql
// tools ship, the same way postgres drives itself through `psql`, minio through `mc`, and mailu
// through doveadm. One boundary, `run()`, and every method is built on it.
// **The backend's own driver, inside the bundle** (novox/hq ADR 0198 §4). This used to shell out to
// `sqlcmd`, a binary the module's container fetched; the module's code now runs in the node's
// runtime, on machines whose system carries no SQL Server client, so it speaks TDS through the
// `mssql` driver its package.json names — installed and inlined into the bundle by the builder. One
// boundary, `session()`, and every method is built on it: a connection as one login to one database,
// opened for one call and closed after, as one sqlcmd invocation was.
//
// Structured rows come back as JSON: SQL Server itself renders the result with `FOR JSON`, and
// this parses the single JSON document sqlcmd prints — far more robust than parsing sqlcmd's
// column-aligned text, since SQL Server owns the quoting and typing.
// Structured rows still come back as JSON rendered by SQL Server itself (`FOR JSON`), so a tool's
// answer is shaped exactly as it was: SQL Server owns the quoting and typing.
import { isIP } from "node:net";
import { randomBytes } from "node:crypto";
import { readFileSync } from "node:fs";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import sql from "mssql";
const run = promisify(execFile);
/** Where a session connects, and as whom. */
export interface Target {
readonly host: string;
readonly port: number;
readonly user: string;
readonly password: string;
readonly database: string;
}
/** One login's connection to one database: run a batch, answer the rows of its last result set. */
export interface Session {
/** `params` are bound as NVARCHAR parameters (`@name`), never written into the text. */
run(text: string, params?: Record<string, string>): Promise<Record<string, unknown>[]>;
close(): Promise<void>;
}
/** How a session is opened — the driver, or a test's fake. */
export type Connect = (to: Target) => Promise<Session>;
/**
* The driver's session: TLS, trusting the self-signed certificate the mssql image ships with (what
* sqlcmd's `-C` did), one connection, closed with the session.
*/
export const connectWithDriver: Connect = async (to) => {
const pool = new sql.ConnectionPool({
server: to.host,
port: to.port,
user: to.user,
password: to.password,
database: to.database,
// TLS names a host, never an address: Node refuses an IP as the server name (DEP0123, an error
// since Node 25), and the module reaches its server on loopback. The certificate is trusted
// either way, so the name only has to be one TLS accepts.
options: { encrypt: true, trustServerCertificate: true, ...(isIP(to.host) ? { serverName: "localhost" } : {}) },
pool: { min: 0, max: 1 },
connectionTimeout: 15_000,
requestTimeout: 60_000,
});
await pool.connect();
return {
async run(text, params = {}) {
const request = pool.request();
const names = Object.keys(params);
for (const name of names) request.input(name, sql.NVarChar, params[name]);
// A batch when nothing is bound — CREATE DATABASE must stand alone in its batch, which a
// parameterised query (sp_executesql) is not.
const result = names.length > 0 ? await request.query(text) : await request.batch(text);
const sets = (result.recordsets ?? []) as Record<string, unknown>[][];
return sets.length > 0 ? sets[sets.length - 1] : [];
},
close: () => pool.close(),
};
};
export interface QueryResult {
/** The leading keyword of the statement, e.g. "SELECT", "CREATE". */
@@ -45,20 +97,17 @@ export interface MssqlConn {
*/
export const READER = "mesh_mssql_reader";
/** Who a sqlcmd invocation logs in as, and whether the text is a caller's rather than the module's. */
/** Who a session logs in as. */
interface Invocation {
readonly user: string;
readonly password: string;
/**
* A caller's text: sqlcmd substitutes no `$(NAME)` in it, which would read this process's
* environment — the administrator's password among it. (Its own commands are kept out by the
* caller's text never beginning a line; see readOnlyQuery.)
*/
readonly caller: boolean;
}
export class MssqlClient {
constructor(private readonly conn: MssqlConn) {}
constructor(
private readonly conn: MssqlConn,
private readonly connect: Connect = connectWithDriver,
) {}
/** The reader is made once per process: idempotent, and repeating it re-sets a rotated password. */
private readerReady?: Promise<void>;
@@ -90,63 +139,41 @@ export class MssqlClient {
return this.conn.port;
}
/**
* Execute a batch that returns no rows (DDL and the like), through `sqlcmd`. The password is
* passed by SQLCMDPASSWORD, never on argv, the way postgres passes PGPASSWORD; `-b` makes a
* failed statement an error here rather than a success with a warning, and `-C` trusts the
* server's self-signed certificate the mssql image ships with.
*/
async exec(sql: string, database = "master"): Promise<void> {
await this.sqlcmd(sql, database);
/** Execute a batch that returns no rows (DDL and the like). A failed statement rejects. */
async exec(text: string, database = "master"): Promise<void> {
await this.session(text, database);
}
/**
* Run a SELECT and return its rows as objects. The caller's SQL must be a single SELECT; it is
* wrapped so SQL Server renders the result with `FOR JSON PATH`, and the JSON document sqlcmd
* prints (split across output lines for a large result, and reassembled here) is parsed. An
* empty result yields no output at all — an empty array.
* wrapped so SQL Server renders the result with `FOR JSON PATH`, and the JSON document it answers
* (split across rows for a large result, and reassembled here) is parsed. An empty result yields
* no rows — an empty array. `params` are bound as `@name`, never written into the text.
*/
async query(
select: string,
database = "master",
variables: Record<string, string> = {},
params: Record<string, string> = {},
): Promise<Record<string, unknown>[]> {
const wrapped = `SET NOCOUNT ON;\n${stripTrailingSemis(select)}\nFOR JSON PATH, INCLUDE_NULL_VALUES;`;
const stdout = await this.sqlcmd(wrapped, database, variables);
return parseJsonRows(stdout);
return parseJsonRows(await this.session(wrapped, database, params));
}
/** The one execution boundary: invoke `sqlcmd` and return its concatenated stdout. */
private async sqlcmd(
sql: string,
/** The one execution boundary: open a session as `as`, run `text`, close it. */
private async session(
text: string,
database: string,
variables: Record<string, string> = {},
as: Invocation = { user: this.conn.user, password: this.conn.password, caller: false },
): Promise<string> {
// `-h -1` drops the column-header rule; `-y 0`/`-Y 0` lift the display-width cap so a long
// JSON document is not truncated; `-W` trims trailing whitespace so the JSON chunks rejoin
// cleanly. sqlcmd from the mssql-tools ships in the runtime container, the way `psql` ships
// with postgres's — the module owns its own code (ADR 0039) and shells out to it.
const { stdout } = await run(
"sqlcmd",
[
"-S", `${this.conn.host},${this.conn.port}`,
"-U", as.user,
"-d", database,
...(as.caller ? ["-x"] : []),
"-C",
"-b",
"-h", "-1",
"-y", "0",
"-Y", "0",
"-W",
"-Q", sql,
],
// `variables` reach sqlcmd as environment variables, which it substitutes as `$(NAME)` scripting
// variables: a value that must not appear on argv, or in the message of a failed command.
{ env: { ...process.env, ...variables, SQLCMDPASSWORD: as.password }, maxBuffer: 16 << 20 },
);
return stdout;
params: Record<string, string> = {},
as: Invocation = { user: this.conn.user, password: this.conn.password },
): Promise<Record<string, unknown>[]> {
const session = await this.connect({
host: this.conn.host, port: this.conn.port, user: as.user, password: as.password, database,
});
try {
return await session.run(text, params);
} finally {
await session.close();
}
}
/**
@@ -173,7 +200,7 @@ export class MssqlClient {
`SELECT 1 AS ok FROM sys.databases WHERE name = ${literal(database)}`,
);
if (dbs.length === 0) {
// CREATE DATABASE must stand alone in its batch; it runs as its own sqlcmd invocation.
// CREATE DATABASE must stand alone in its batch; it runs as its own session.
await this.exec(`CREATE DATABASE ${ident(database)}`);
}
@@ -206,15 +233,14 @@ export class MssqlClient {
* nothing logs in and no failed-login is recorded (novox/hq issue 120).
*/
async holdsLogin(database: string, login: string, password: string): Promise<boolean> {
// The password reaches sqlcmd as a scripting variable from the environment, never inside the
// query text, so it is neither on argv nor in the message of a failed command. It is the mesh's
// minted value, which carries no quote.
// The password is a bound parameter, never inside the query text, so it is in no message of a
// failed statement.
const server = await this.query(
`SELECT CAST(CASE WHEN EXISTS (SELECT 1 FROM sys.sql_logins WHERE name = ${literal(login)} ` +
`AND is_disabled = 0 AND PWDCOMPARE(N'$(MESHHOLDSPW)', password_hash) = 1) ` +
`AND is_disabled = 0 AND PWDCOMPARE(@meshholdspw, password_hash) = 1) ` +
`AND DB_ID(${literal(database)}) IS NOT NULL THEN 1 ELSE 0 END AS int) AS ok`,
"master",
{ MESHHOLDSPW: password },
{ meshholdspw: password },
);
if (Number(server[0]?.ok) !== 1) return false;
// The user must be this login's, by SID, and a db_owner. A user orphaned by a restore has the
@@ -294,35 +320,27 @@ export class MssqlClient {
* (novox/hq issue 193). Read-only by the login, not by a transaction wrapped around the text; the
* rows are rendered by FOR JSON. Never as the administrator: without the reader's password the call
* is refused.
*
* The text goes to the server as it is, over the driver: there is no client between that reads a
* line of its own (sqlcmd's `:!!`, which could start a program) or substitutes `$(NAME)` from this
* process's environment, so neither the one-line rule nor `-x` has anything left to guard.
*/
async readOnlyQuery(database: string, sql: string): Promise<QueryResult> {
async readOnlyQuery(database: string, text: string): Promise<QueryResult> {
const password = this.conn.readerPassword;
if (!password) throw readerMissing();
// **One line, refused otherwise.** sqlcmd reads a line that BEGINS with `:` or `!!` as its own
// command rather than SQL, and `:!!` starts a program in this container, which holds the
// administrator's password. Its switch for refusing those (-X) makes it ignore -Q in the
// version shipped here, so instead no line of a caller's text can begin one: the text follows
// this module's own on the first line, and a line break in it is refused. Proven on a throwaway
// server: the same text at the start of a line ran a program; mid-line it is a syntax error.
if (/[\r\n]/.test(sql)) {
throw new Error(
"mssql_query: the statement must be one line — sqlcmd takes a line beginning with ':' or " +
"'!!' as a command of its own, which can start a program (novox/hq issue 193)",
);
}
this.readerReady ??= this.ensureReader().catch((err) => {
this.readerReady = undefined; // asked again next call, not failed for the process's life
throw err;
});
await this.readerReady;
const stdout = await this.sqlcmd(
`SET NOCOUNT ON; ${stripTrailingSemis(sql)}\nFOR JSON PATH, INCLUDE_NULL_VALUES;`,
const rows = await this.session(
`SET NOCOUNT ON; ${stripTrailingSemis(text)}\nFOR JSON PATH, INCLUDE_NULL_VALUES;`,
database,
{},
{ user: READER, password, caller: true },
{ user: READER, password },
);
const command = /^\s*([A-Za-z]+)/.exec(sql)?.[1]?.toUpperCase() ?? "";
return { command, rows: parseJsonRows(stdout) };
const command = /^\s*([A-Za-z]+)/.exec(text)?.[1]?.toUpperCase() ?? "";
return { command, rows: parseJsonRows(rows) };
}
}
@@ -372,18 +390,13 @@ function safeUrl(raw: string): URL | undefined {
}
/**
* Parse the JSON a FOR JSON query prints through sqlcmd. SQL Server splits a large FOR JSON result
* into ~2033-character chunks, one per output row; with `-h -1 -W` each lands on its own line, so
* the document is reassembled by concatenating the non-empty lines. No output (an empty result, or
* a pure DDL batch) means no rows.
* Parse the JSON a FOR JSON query answers. SQL Server splits a large FOR JSON result into
* ~2033-character chunks, one per row of a single column, so the document is reassembled by
* concatenating that column in order. No rows (an empty result, or a pure DDL batch) means none.
*/
function parseJsonRows(stdout: string): Record<string, unknown>[] {
const joined = stdout
.split(/\r?\n/)
.map((l) => l.trimEnd())
.filter((l) => l.length > 0)
.join("");
if (joined.length === 0) return [];
function parseJsonRows(rows: Record<string, unknown>[]): Record<string, unknown>[] {
const joined = rows.map((row) => String(Object.values(row)[0] ?? "")).join("");
if (joined.trim().length === 0) return [];
const parsed = JSON.parse(joined);
return Array.isArray(parsed) ? (parsed as Record<string, unknown>[]) : [parsed as Record<string, unknown>];
}
+3 -3
View File
@@ -1,9 +1,9 @@
// mssql's events entrypoint, loaded by the per-node tool host (the provisioner container runs
// ./provisioner separately). The database lifecycle events are EMITTED from the provisioner, where
// mssql's events entrypoint, launched by the node's runtime beside its tools and provisioner
// (novox/hq ADR 0198). The database lifecycle events are EMITTED from the provisioner, where
// the lifecycle actually happens (novox/hq ADR 0041/0042):
// module.mssql.database.provisioned — a consumer's database + login/user was created
// module.mssql.database.deprovisioned — that database was removed
// Here in the tool host we react to them, keeping a lightweight audit trail of who was granted a
// Here in the runtime we react to them, keeping a lightweight audit trail of who was granted a
// database and who lost one — observability the provider itself is best placed to log.
import { on } from "@novox/mesh-sdk/events";
+19 -42
View File
@@ -38,16 +38,9 @@
},
"own-secrets": {
"sa": "${dir:state}/sa.secret",
"broker": "${dir:mesh-state}/broker",
"reader": "${dir:state}/reader.secret"
},
"resources": [
{
"id": "mesh-state",
"type": "directory",
"mode": "0700",
"place": "mesh"
},
{
"id": "state",
"type": "directory",
@@ -93,46 +86,30 @@
"${dir:data}:/var/opt/mssql"
],
"secrets-in-environment": "the image documents only MSSQL_SA_PASSWORD, no _FILE and no configuration field; not convertible without a wrapper entrypoint"
},
{
"id": "runtime",
"type": "container",
"name": "mesh-mssql",
"network": "mssql",
"volumes": [
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:grants}:/var/lib/mssql/grants:ro",
"${dir:state}/sa.secret:/run/secrets/sa:ro",
"${dir:state}/reader.secret:/run/secrets/reader:ro"
],
"env": {
"MESH_PROVISION_MSSQL": "mssql://sa@mssql:1433/master",
"MESH_PROVISION_PASSWORD_FILE": "/run/secrets/sa",
"MESH_BROKER_FILE": "/run/secrets/broker",
"MESH_RECEIVES": "/var/lib/mssql/grants/mesh.json",
"MESH_MSSQL_READER_PASSWORD_FILE": "/run/secrets/reader"
},
"artifact": "runtime"
}
],
"build": {
"on": [
{
"arg": "BUILD_BASE",
"module": "mesh-tools",
"artifact": "build"
},
{
"arg": "RUNTIME_BASE",
"module": "mesh-tools",
"artifact": "runtime"
}
],
"artifacts": [
{
"name": "runtime",
"kind": "image",
"from": "Dockerfile"
"name": "code",
"kind": "bundle",
"language": "typescript",
"entrypoints": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"loads": [
"index.js",
"tools/index.js",
"provisioner/index.js"
],
"env": {
"MESH_PROVISION_MSSQL": "mssql://sa@127.0.0.1:${port:1433}/master",
"MESH_PROVISION_PASSWORD_FILE": "${dir:state}/sa.secret",
"MESH_RECEIVES": "${dir:grants}/mesh.json",
"MESH_MSSQL_READER_PASSWORD_FILE": "${dir:state}/reader.secret"
}
}
]
}
+32
View File
@@ -0,0 +1,32 @@
// Ambient types for `mssql`, which ships its types only in the separate `@types/mssql` package. This
// declares the slice client.ts uses — the precedent mesh-catalog's pg.d.ts sets — so the module
// type-checks without deciding what runs: the real `mssql` is the package.json dependency the
// builder installs and inlines into the bundle (novox/hq ADR 0198 §4).
declare module "mssql" {
interface Result {
recordsets: unknown;
}
interface Request {
input(name: string, type: unknown, value: unknown): Request;
query(text: string): Promise<Result>;
batch(text: string): Promise<Result>;
}
class ConnectionPool {
constructor(config: {
server: string;
port?: number;
user?: string;
password?: string;
database?: string;
options?: { encrypt?: boolean; trustServerCertificate?: boolean; serverName?: string };
pool?: { min?: number; max?: number };
connectionTimeout?: number;
requestTimeout?: number;
});
connect(): Promise<ConnectionPool>;
request(): Request;
close(): Promise<void>;
}
const sql: { ConnectionPool: typeof ConnectionPool; NVarChar: unknown };
export default sql;
}
+3 -2
View File
@@ -5,11 +5,12 @@
"type": "module",
"private": true,
"scripts": {
"build": "tsc client.ts index.ts tools/index.ts provisioner/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"build": "tsc mssql.d.ts client.ts index.ts tools/index.ts provisioner/index.ts --module NodeNext --moduleResolution NodeNext --target ES2022 --outDir dist",
"test": "npm run build && node --test --experimental-strip-types 'test/*.test.ts'"
},
"dependencies": {
"@novox/mesh-sdk": "^0.1.1"
"@novox/mesh-sdk": "^0.1.1",
"mssql": "^11.0.2"
},
"devDependencies": {
"@types/node": "^22.0.0",
+51 -64
View File
@@ -1,96 +1,83 @@
// What holds mssql_query to being read-only (novox/hq issue 193): a caller's statement runs as the
// reader login and never as the administrator, with sqlcmd's variable substitution off, on one line
// that follows the module's own — a line break is refused before sqlcmd starts — and with no
// transaction wrapped around it as text. Without the reader's password the statement is refused.
// reader login and never as the administrator, with no transaction wrapped around it as text, and
// without the reader's password the statement is refused.
//
// sqlcmd is a fake on PATH that records each call's login, flags and text. That the reader cannot
// write is the server's to enforce and was proven against a real server; this holds the module to
// asking for it. Run against the compiled module (npm test builds first), the way the runtime loads it.
// The driver is a fake session that records each call's login, database, text and bound
// parameters. That the reader cannot write is the server's to enforce and was proven against a real
// server; this holds the module to asking for it. Run against the compiled module (npm test builds
// first), the way the runtime loads it.
import { test, before, after } from "node:test";
import { test } from "node:test";
import assert from "node:assert/strict";
import { chmod, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { MssqlClient, READER } from "../dist/client.js";
import { MssqlClient, READER, type Connect, type Target } from "../dist/client.js";
let dir: string;
let log: string;
const originalPath = process.env.PATH;
interface Call extends Target {
text: string;
params: Record<string, string>;
}
before(async () => {
dir = await mkdtemp(join(tmpdir(), "mssql-reader-"));
log = join(dir, "calls.jsonl");
await writeFile(join(dir, "sqlcmd"), `#!/usr/bin/env node
const fs = require("node:fs");
const args = process.argv.slice(2);
const at = (flag) => args[args.indexOf(flag) + 1];
fs.appendFileSync(${JSON.stringify(log)}, JSON.stringify({
user: at("-U"), database: at("-d"), sql: at("-Q"), noVariables: args.includes("-x"),
password: process.env.SQLCMDPASSWORD,
}) + "\\n");
const sql = at("-Q");
if (/FROM sys.server_principals/.test(sql)) process.stdout.write("");
else if (/FOR JSON/.test(sql)) process.stdout.write('[{"name":"alpha","n":1}]\\n');
`);
await chmod(join(dir, "sqlcmd"), 0o755);
process.env.PATH = `${dir}:${originalPath}`;
});
after(async () => {
process.env.PATH = originalPath;
await rm(dir, { recursive: true, force: true });
});
async function calls(): Promise<Record<string, unknown>[]> {
const text = await readFile(log, "utf8").catch(() => "");
await writeFile(log, "");
return text.split("\n").filter(Boolean).map((line) => JSON.parse(line));
function recording(): { connect: Connect; calls: Call[] } {
const calls: Call[] = [];
const connect: Connect = async (to) => ({
async run(text, params = {}) {
calls.push({ ...to, text, params });
if (/FROM sys.server_principals/.test(text)) return [];
// FOR JSON answers its document split across rows of one column.
if (/FOR JSON/.test(text)) return [{ JSON_F52E: '[{"name":"al' }, { JSON_F52E: 'pha","n":1}]' }];
return [];
},
async close() {},
});
return { connect, calls };
}
const conn = { host: "127.0.0.1", port: 1433, user: "sa", password: "admin-secret" };
test("a caller's statement runs as the reader, without variables, on the module's first line", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
test("a caller's statement runs as the reader, as it was written, on the database it names", async () => {
const { connect, calls } = recording();
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" }, connect);
const result = await client.readOnlyQuery("inventory", "SELECT '$(SQLCMDPASSWORD)' AS p");
const asked = (await calls()).at(-1)!;
const asked = calls.at(-1)!;
assert.equal(asked.user, READER, "the statement never runs as the administrator");
assert.equal(asked.password, "reader-secret");
assert.equal(asked.noVariables, true, "no $(NAME) is substituted in a caller's text");
const [first] = String(asked.sql).split("\n");
assert.ok(first.startsWith("SET NOCOUNT ON; SELECT '$(SQLCMDPASSWORD)'"), "the caller's text never begins a line");
assert.doesNotMatch(String(asked.sql), /BEGIN TRANSACTION|ROLLBACK/, "no transaction wrapped around it as text");
assert.deepEqual(result.rows, [{ name: "alpha", n: 1 }]);
assert.equal(asked.database, "inventory");
assert.ok(asked.text.startsWith("SET NOCOUNT ON; SELECT '$(SQLCMDPASSWORD)' AS p\nFOR JSON PATH"),
"the caller's text reaches the server unaltered");
assert.doesNotMatch(asked.text, /BEGIN TRANSACTION|ROLLBACK/, "no transaction wrapped around it as text");
assert.deepEqual(result.rows, [{ name: "alpha", n: 1 }], "a FOR JSON document split across rows is reassembled");
assert.equal(result.command, "SELECT");
});
test("a line break in a caller's statement is refused before sqlcmd starts", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
for (const sql of ["SELECT 1\n:!! id", "SELECT 1\r\n:!! id", "SELECT 1\r:!! id"]) {
await assert.rejects(client.readOnlyQuery("inventory", sql), /must be one line/);
}
assert.deepEqual(await calls(), []);
});
test("the reader is made as the administrator, kept out of sysadmin, and granted only reading", async () => {
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" });
const { connect, calls } = recording();
const client = new MssqlClient({ ...conn, readerPassword: "reader-secret" }, connect);
await client.readOnlyQuery("inventory", "SELECT 1 AS x");
await client.readOnlyQuery("inventory", "SELECT 2 AS x");
const made = await calls();
const asAdmin = made.filter((c) => c.user === "sa").map((c) => String(c.sql));
const asAdmin = calls.filter((c) => c.user === "sa").map((c) => c.text);
assert.ok(asAdmin.some((s) => s.startsWith(`CREATE LOGIN [${READER}]`)));
assert.ok(asAdmin.some((s) => /ALTER SERVER ROLE sysadmin DROP MEMBER/.test(s)));
assert.ok(asAdmin.includes(`GRANT CONNECT ANY DATABASE TO [${READER}]`));
assert.ok(asAdmin.includes(`GRANT SELECT ALL USER SECURABLES TO [${READER}]`));
assert.equal(asAdmin.filter((s) => s.startsWith("CREATE LOGIN")).length, 1, "made once, not per call");
assert.equal(made.filter((c) => c.user === READER).length, 2);
assert.equal(calls.filter((c) => c.user === READER).length, 2);
});
test("without the reader's password the statement is refused, and nothing runs as the administrator", async () => {
const client = new MssqlClient(conn);
const { connect, calls } = recording();
const client = new MssqlClient(conn, connect);
await assert.rejects(client.readOnlyQuery("inventory", "SELECT 1"), /refused rather than run as the administrator/);
assert.deepEqual(await calls(), []);
assert.deepEqual(calls, []);
});
test("a consumer's password is checked as a bound parameter, never in the text", async () => {
const { connect, calls } = recording();
const client = new MssqlClient(conn, connect);
await client.holdsLogin("shop", "shop_login", "minted-secret");
const asked = calls[0];
assert.equal(asked.params.meshholdspw, "minted-secret");
assert.doesNotMatch(asked.text, /minted-secret/);
assert.match(asked.text, /PWDCOMPARE\(@meshholdspw, password_hash\)/);
});
+1 -1
View File
@@ -8,5 +8,5 @@
"skipLibCheck": true,
"noEmit": true
},
"include": ["client.ts", "index.ts", "provisioner/index.ts", "tools/index.ts"]
"include": ["mssql.d.ts", "client.ts", "index.ts", "provisioner/index.ts", "tools/index.ts"]
}
+1 -1
View File
@@ -62,7 +62,7 @@
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "POSTGRES_HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nPOSTGRES_DB=${bound:postgres-database:as}\nPOSTGRES_USER=${bound:postgres-database:as}\nPOSTGRES_PASSWORD=${secret:postgres-database}\nNEXTCLOUD_ADMIN_USER=mesh-admin\nNEXTCLOUD_ADMIN_PASSWORD=${secret:admin}\nOBJECTSTORE_S3_HOST=${bound:s3-bucket:at}\nOBJECTSTORE_S3_PORT=${bound:s3-bucket:port}\nOBJECTSTORE_S3_BUCKET=mesh-novox-ncloud\nOBJECTSTORE_S3_KEY=${bound:s3-bucket:as}\nOBJECTSTORE_S3_SECRET=${secret:s3-bucket}\nOBJECTSTORE_S3_SSL=false\nOBJECTSTORE_S3_USEPATH_STYLE=true\nOBJECTSTORE_S3_REGION=${bound:s3-bucket:region}\n"
"content": "POSTGRES_HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nPOSTGRES_DB=${bound:postgres-database:as}\nPOSTGRES_USER=${bound:postgres-database:as}\nPOSTGRES_PASSWORD=${secret:postgres-database}\nNEXTCLOUD_ADMIN_USER=mesh-admin\nNEXTCLOUD_ADMIN_PASSWORD=${secret:admin}\nOBJECTSTORE_S3_HOST=${bound:s3-bucket:at}\nOBJECTSTORE_S3_PORT=${bound:s3-bucket:port}\nOBJECTSTORE_S3_BUCKET=${bound:s3-bucket:bucket}\nOBJECTSTORE_S3_KEY=${bound:s3-bucket:as}\nOBJECTSTORE_S3_SECRET=${secret:s3-bucket}\nOBJECTSTORE_S3_SSL=false\nOBJECTSTORE_S3_USEPATH_STYLE=true\nOBJECTSTORE_S3_REGION=${bound:s3-bucket:region}\n"
},
{
"id": "html",
+42
View File
@@ -0,0 +1,42 @@
{
"module": "node-env",
"version": "1",
"claims": [
{
"name": "node-environment",
"scope": "node"
}
],
"resources": [
{
"id": "mesh-config-dir",
"type": "directory",
"path": "${machine:account-home}/.config/mesh",
"owner": "${machine:account}",
"mode": "0755"
},
{
"id": "environment-d",
"type": "directory",
"path": "${machine:account-home}/.config/environment.d",
"owner": "${machine:account}",
"mode": "0755"
},
{
"id": "posix",
"type": "file",
"path": "${machine:account-home}/.config/mesh/environment.sh",
"owner": "${machine:account}",
"mode": "0644",
"content": "# The operator account's environment, generated by the mesh (module node-env, novox/hq ADR 0203).\n# Do not edit: this file is replaced at every push. Every line names the module that contributed it.\n# Sourced by the login shell from its always-read startup file (for zsh, ~/.zshenv), so a script, a\n# login and the shell's execute verb all see it. Your own variables belong in your shell's own lines.\n${environment:posix}"
},
{
"id": "systemd",
"type": "file",
"path": "${machine:account-home}/.config/environment.d/50-mesh.conf",
"owner": "${machine:account}",
"mode": "0644",
"content": "# The operator account's environment for its service manager and graphical session, generated by\n# the mesh (module node-env, novox/hq ADR 0203). Do not edit: this file is replaced at every push.\n# The same facts as ~/.config/mesh/environment.sh, in environment.d(5) syntax.\n${environment:systemd}"
}
]
}
+1 -1
View File
@@ -58,7 +58,7 @@
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "NODE_ENV=production\nPORT=9000\nMONGO_URL=mongodb://${bound:mongodb-database:as}:${secret:mongodb-database}@${bound:mongodb-database:at}:${bound:mongodb-database:port}/${bound:mongodb-database:as}?authSource=admin\nMONGO_DB=${bound:mongodb-database:as}\nMINIO_ENDPOINT=${bound:s3-bucket:at}\nMINIO_PORT=${bound:s3-bucket:port}\nMINIO_BUCKET=mesh-novox-photos\nMINIO_ACCESSKEY=${bound:s3-bucket:as}\nMINIO_SECRET=${secret:s3-bucket}\nMINIO_USE_SSL=false\n"
"content": "NODE_ENV=production\nPORT=9000\nMONGO_URL=mongodb://${bound:mongodb-database:as}:${secret:mongodb-database}@${bound:mongodb-database:at}:${bound:mongodb-database:port}/${bound:mongodb-database:as}?authSource=admin\nMONGO_DB=${bound:mongodb-database:as}\nMINIO_ENDPOINT=${bound:s3-bucket:at}\nMINIO_PORT=${bound:s3-bucket:port}\nMINIO_BUCKET=${bound:s3-bucket:bucket}\nMINIO_ACCESSKEY=${bound:s3-bucket:as}\nMINIO_SECRET=${secret:s3-bucket}\nMINIO_USE_SSL=false\n"
},
{
"id": "net",
+46
View File
@@ -0,0 +1,46 @@
# powerlevel10k
The zsh prompt as a module (novox/hq ADR 0204, ADR 0205, to-be 41).
- **Upstream:** https://github.com/romkatv/powerlevel10k
- **Version:** v1.20.0, vendored verbatim from the release archive
(`archive/refs/tags/v1.20.0.tar.gz`, sha256
`d8187d44b697b3a37a8c4896678b4380e717cbf2850179529358348780a2d3d7`) into `theme/`. It is 86
files and 1,427,736 bytes.
- **Licence:** MIT, in `theme/LICENSE`, which travels in the archive. gitstatus's own licence is
`theme/gitstatus/LICENSE`.
The distribution does not package the theme, so the module carries a pinned release (ADR 0205). An
upgrade is a change to this directory, reviewed like any other: replace `theme/` with the new
release's contents, and update the version here and in the shell code's comment.
## What it places
| path under the account's home | what | class (ADR 0182) |
|---|---|---|
| `~/.local/share/powerlevel10k/` | the theme, unpacked from the `theme` archive | owned, whole |
| `~/.config/powerlevel10k/p10k.zsh` | the prompt's configuration, unpacked from the `configuration` archive | owned, whole |
The configuration is today's `~/.p10k.zsh`, byte for byte. It is the predecessor's file, and was
identical on every machine. It ships as an archive of one file rather than as an inline file. At
86 KB, inline content would ride in every declaration the node receives, and would be unreadable
JSON in review. As its own file it is reviewed as a diff, and pinned by digest like the theme. The
directory is the module's, so `p10k configure` writing `~/.p10k.zsh` does not touch it: to change
the prompt, change `config/p10k.zsh` here.
The module contributes zsh code to the `normal` slot of the login shell's block. That code sources
the theme, then the configuration, each only if present. Instant prompt is not turned on: the
operator's `.zshrc` has its cache line commented out today, and the configuration's own
`POWERLEVEL9K_INSTANT_PROMPT` setting does nothing without that line.
## What it does not do
- **gitstatus downloads its binary on first use.** The theme's git status helper fetches
`gitstatusd` from upstream's releases into `~/.cache/gitstatus` the first time a prompt runs in a
git repository. ADR 0205 pins what the mesh ships, not what the software fetches for itself. A
machine without a route to upstream shows the prompt without git status.
- **Fonts are not this module's.** The configuration uses Nerd Font icons. The terminal's font is the
desktop's concern.
- **Moving from the predecessor:** once this module is assigned, `~/.zsh/themes/powerlevel10k` and
`~/.p10k.zsh` are no longer read, and the operator removes them, once (ADR 0182; the zsh module's
README lists the lines to delete from `.zshrc`).
File diff suppressed because it is too large Load Diff
+41
View File
@@ -0,0 +1,41 @@
{
"module": "powerlevel10k",
"version": "1",
"shell": [
{
"for": "zsh",
"slot": "normal",
"code": "# The prompt: powerlevel10k v1.20.0, pinned in this module (novox/hq ADR 0205), and its configuration.\n[[ ! -f ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme ]] || source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme\n[[ ! -f ~/.config/powerlevel10k/p10k.zsh ]] || source ~/.config/powerlevel10k/p10k.zsh\n"
}
],
"resources": [
{
"id": "theme",
"type": "archive",
"path": "${machine:account-home}/.local/share/powerlevel10k",
"owner": "${machine:account}",
"artifact": "theme"
},
{
"id": "configuration",
"type": "archive",
"path": "${machine:account-home}/.config/powerlevel10k",
"owner": "${machine:account}",
"artifact": "configuration"
}
],
"build": {
"artifacts": [
{
"name": "theme",
"kind": "archive",
"from": "theme"
},
{
"name": "configuration",
"kind": "archive",
"from": "config"
}
]
}
}
+13
View File
@@ -0,0 +1,13 @@
{
"name": "@novox/module-powerlevel10k",
"version": "0.1.0",
"description": "powerlevel10k \u2014 the zsh prompt as a module: upstream v1.20.0 vendored and shipped as a pinned archive, its configuration as a second, and the zsh code that loads both in the normal slot (novox/hq ADR 0204, ADR 0205).",
"type": "module",
"private": true,
"scripts": {
"test": "node --test --experimental-strip-types 'test/*.test.ts'"
},
"devDependencies": {
"@types/node": "^22.0.0"
}
}
+45
View File
@@ -0,0 +1,45 @@
// The prompt module's shape (novox/hq ADR 0204, ADR 0205): the vendored release is pinned, carries
// its licence in the archive, and is loaded with its configuration from the normal slot.
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync, readFileSync, statSync } from "node:fs";
const at = (p: string) => new URL(`../${p}`, import.meta.url);
const m = JSON.parse(readFileSync(at("module.json"), "utf8"));
const artifact = (name: string) => m.build.artifacts.find((a: { name: string }) => a.name === name);
test("the theme archive is built from the vendored release and carries its licence", () => {
assert.deepEqual(artifact("theme"), { name: "theme", kind: "archive", from: "theme" });
assert.match(readFileSync(at("theme/LICENSE"), "utf8"), /Permission is hereby granted, free of charge/);
assert.ok(existsSync(at("theme/powerlevel10k.zsh-theme")));
assert.ok(existsSync(at("theme/gitstatus/LICENSE")));
});
test("the configuration archive holds the prompt's configuration and nothing else", () => {
assert.deepEqual(artifact("configuration"), { name: "configuration", kind: "archive", from: "config" });
assert.ok(statSync(at("config/p10k.zsh")).size > 0);
});
test("both are unpacked under the account's home, owned by the account", () => {
const byId = Object.fromEntries(m.resources.map((r: { id: string }) => [r.id, r]));
assert.deepEqual(byId.theme, { id: "theme", type: "archive", path: "${machine:account-home}/.local/share/powerlevel10k", owner: "${machine:account}", artifact: "theme" });
assert.deepEqual(byId.configuration, { id: "configuration", type: "archive", path: "${machine:account-home}/.config/powerlevel10k", owner: "${machine:account}", artifact: "configuration" });
});
test("the zsh code in the normal slot sources the theme, then the configuration, and turns on no instant prompt", () => {
assert.equal(m.shell.length, 1);
const [c] = m.shell;
assert.equal(c.for, "zsh");
assert.equal(c.slot, "normal");
const sourced = [...(c.code as string).matchAll(/\|\| source (\S+)/g)].map((x) => x[1]);
assert.deepEqual(sourced, ["~/.local/share/powerlevel10k/powerlevel10k.zsh-theme", "~/.config/powerlevel10k/p10k.zsh"]);
assert.doesNotMatch(c.code, /instant/);
});
test("the README names the upstream, the version and the licence", () => {
const readme = readFileSync(at("README.md"), "utf8");
assert.match(readme, /github\.com\/romkatv\/powerlevel10k/);
assert.match(readme, /v1\.20\.0/);
assert.match(readme, /MIT/);
assert.match(m.shell[0].code, /v1\.20\.0/, "the version in the shell code's comment matches");
});
@@ -0,0 +1,5 @@
* text=auto
*.zsh text eol=lf
*.zsh-theme text eol=lf
/prompt_powerlevel9k_setup text eol=lf
/prompt_powerlevel10k_setup text eol=lf
+1
View File
@@ -0,0 +1 @@
*.zwc
+22
View File
@@ -0,0 +1,22 @@
Copyright (c) 2009-2014 Robby Russell and contributors (see https://github.com/robbyrussell/oh-my-zsh/contributors)
Copyright (c) 2014-2017 Ben Hilburn <bhilburn@gmail.com>
Copyright (c) 2019 Roman Perepelitsa <roman.perepelitsa@gmail.com> and contributors (see https://github.com/romkatv/powerlevel10k/contributors)
MIT LICENSE
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+14
View File
@@ -0,0 +1,14 @@
ZSH := $(shell command -v zsh 2> /dev/null)
all:
zwc:
$(MAKE) -C gitstatus zwc
$(or $(ZSH),:) -fc 'for f in *.zsh-theme internal/*.zsh; do zcompile -R -- $$f.zwc $$f || exit; done'
minify:
$(MAKE) -C gitstatus minify
rm -rf -- .git .gitattributes .gitignore LICENSE Makefile README.md font.md powerlevel10k.png
pkg: zwc
$(MAKE) -C gitstatus pkg
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,193 @@
# Config file for Powerlevel10k with the style of Pure (https://github.com/sindresorhus/pure).
#
# Differences from Pure:
#
# - Git:
# - `@c4d3ec2c` instead of something like `v1.4.0~11` when in detached HEAD state.
# - No automatic `git fetch` (the same as in Pure with `PURE_GIT_PULL=0`).
#
# Apart from the differences listed above, the replication of Pure prompt is exact. This includes
# even the questionable parts. For example, just like in Pure, there is no indication of Git status
# being stale; prompt symbol is the same in command, visual and overwrite vi modes; when prompt
# doesn't fit on one line, it wraps around with no attempt to shorten it.
#
# If you like the general style of Pure but not particularly attached to all its quirks, type
# `p10k configure` and pick "Lean" style. This will give you slick minimalist prompt while taking
# advantage of Powerlevel10k features that aren't present in Pure.
# Temporarily change options.
'builtin' 'local' '-a' 'p10k_config_opts'
[[ ! -o 'aliases' ]] || p10k_config_opts+=('aliases')
[[ ! -o 'sh_glob' ]] || p10k_config_opts+=('sh_glob')
[[ ! -o 'no_brace_expand' ]] || p10k_config_opts+=('no_brace_expand')
'builtin' 'setopt' 'no_aliases' 'no_sh_glob' 'brace_expand'
() {
emulate -L zsh -o extended_glob
# Unset all configuration options.
unset -m '(POWERLEVEL9K_*|DEFAULT_USER)~POWERLEVEL9K_GITSTATUS_DIR'
# Zsh >= 5.1 is required.
[[ $ZSH_VERSION == (5.<1->*|<6->.*) ]] || return
# Prompt colors.
local grey=242
local red=1
local yellow=3
local blue=4
local magenta=5
local cyan=6
local white=7
# Left prompt segments.
typeset -g POWERLEVEL9K_LEFT_PROMPT_ELEMENTS=(
# =========================[ Line #1 ]=========================
context # user@host
dir # current directory
vcs # git status
command_execution_time # previous command duration
# =========================[ Line #2 ]=========================
newline # \n
virtualenv # python virtual environment
prompt_char # prompt symbol
)
# Right prompt segments.
typeset -g POWERLEVEL9K_RIGHT_PROMPT_ELEMENTS=(
# =========================[ Line #1 ]=========================
# command_execution_time # previous command duration
# virtualenv # python virtual environment
# context # user@host
# time # current time
# =========================[ Line #2 ]=========================
newline # \n
)
# Basic style options that define the overall prompt look.
typeset -g POWERLEVEL9K_BACKGROUND= # transparent background
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_{LEFT,RIGHT}_WHITESPACE= # no surrounding whitespace
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_SUBSEGMENT_SEPARATOR=' ' # separate segments with a space
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_SEGMENT_SEPARATOR= # no end-of-line symbol
typeset -g POWERLEVEL9K_VISUAL_IDENTIFIER_EXPANSION= # no segment icons
# Add an empty line before each prompt except the first. This doesn't emulate the bug
# in Pure that makes prompt drift down whenever you use the Alt-C binding from fzf or similar.
typeset -g POWERLEVEL9K_PROMPT_ADD_NEWLINE=true
# Magenta prompt symbol if the last command succeeded.
typeset -g POWERLEVEL9K_PROMPT_CHAR_OK_{VIINS,VICMD,VIVIS}_FOREGROUND=$magenta
# Red prompt symbol if the last command failed.
typeset -g POWERLEVEL9K_PROMPT_CHAR_ERROR_{VIINS,VICMD,VIVIS}_FOREGROUND=$red
# Default prompt symbol.
typeset -g POWERLEVEL9K_PROMPT_CHAR_{OK,ERROR}_VIINS_CONTENT_EXPANSION='❯'
# Prompt symbol in command vi mode.
typeset -g POWERLEVEL9K_PROMPT_CHAR_{OK,ERROR}_VICMD_CONTENT_EXPANSION='❮'
# Prompt symbol in visual vi mode is the same as in command mode.
typeset -g POWERLEVEL9K_PROMPT_CHAR_{OK,ERROR}_VIVIS_CONTENT_EXPANSION='❮'
# Prompt symbol in overwrite vi mode is the same as in command mode.
typeset -g POWERLEVEL9K_PROMPT_CHAR_OVERWRITE_STATE=false
# Grey Python Virtual Environment.
typeset -g POWERLEVEL9K_VIRTUALENV_FOREGROUND=$grey
# Don't show Python version.
typeset -g POWERLEVEL9K_VIRTUALENV_SHOW_PYTHON_VERSION=false
typeset -g POWERLEVEL9K_VIRTUALENV_{LEFT,RIGHT}_DELIMITER=
# Blue current directory.
typeset -g POWERLEVEL9K_DIR_FOREGROUND=$blue
# Context format when root: user@host. The first part white, the rest grey.
typeset -g POWERLEVEL9K_CONTEXT_ROOT_TEMPLATE="%F{$white}%n%f%F{$grey}@%m%f"
# Context format when not root: user@host. The whole thing grey.
typeset -g POWERLEVEL9K_CONTEXT_TEMPLATE="%F{$grey}%n@%m%f"
# Don't show context unless root or in SSH.
typeset -g POWERLEVEL9K_CONTEXT_{DEFAULT,SUDO}_CONTENT_EXPANSION=
# Show previous command duration only if it's >= 5s.
typeset -g POWERLEVEL9K_COMMAND_EXECUTION_TIME_THRESHOLD=5
# Don't show fractional seconds. Thus, 7s rather than 7.3s.
typeset -g POWERLEVEL9K_COMMAND_EXECUTION_TIME_PRECISION=0
# Duration format: 1d 2h 3m 4s.
typeset -g POWERLEVEL9K_COMMAND_EXECUTION_TIME_FORMAT='d h m s'
# Yellow previous command duration.
typeset -g POWERLEVEL9K_COMMAND_EXECUTION_TIME_FOREGROUND=$yellow
# Grey Git prompt. This makes stale prompts indistinguishable from up-to-date ones.
typeset -g POWERLEVEL9K_VCS_FOREGROUND=$grey
# Disable async loading indicator to make directories that aren't Git repositories
# indistinguishable from large Git repositories without known state.
typeset -g POWERLEVEL9K_VCS_LOADING_TEXT=
# Don't wait for Git status even for a millisecond, so that prompt always updates
# asynchronously when Git state changes.
typeset -g POWERLEVEL9K_VCS_MAX_SYNC_LATENCY_SECONDS=0
# Cyan ahead/behind arrows.
typeset -g POWERLEVEL9K_VCS_{INCOMING,OUTGOING}_CHANGESFORMAT_FOREGROUND=$cyan
# Don't show remote branch, current tag or stashes.
typeset -g POWERLEVEL9K_VCS_GIT_HOOKS=(vcs-detect-changes git-untracked git-aheadbehind)
# Don't show the branch icon.
typeset -g POWERLEVEL9K_VCS_BRANCH_ICON=
# When in detached HEAD state, show @commit where branch normally goes.
typeset -g POWERLEVEL9K_VCS_COMMIT_ICON='@'
# Don't show staged, unstaged, untracked indicators.
typeset -g POWERLEVEL9K_VCS_{STAGED,UNSTAGED,UNTRACKED}_ICON=
# Show '*' when there are staged, unstaged or untracked files.
typeset -g POWERLEVEL9K_VCS_DIRTY_ICON='*'
# Show '⇣' if local branch is behind remote.
typeset -g POWERLEVEL9K_VCS_INCOMING_CHANGES_ICON=':⇣'
# Show '⇡' if local branch is ahead of remote.
typeset -g POWERLEVEL9K_VCS_OUTGOING_CHANGES_ICON=':⇡'
# Don't show the number of commits next to the ahead/behind arrows.
typeset -g POWERLEVEL9K_VCS_{COMMITS_AHEAD,COMMITS_BEHIND}_MAX_NUM=1
# Remove space between '⇣' and '⇡' and all trailing spaces.
typeset -g POWERLEVEL9K_VCS_CONTENT_EXPANSION='${${${P9K_CONTENT/⇣* :⇡/⇣⇡}// }//:/ }'
# Grey current time.
typeset -g POWERLEVEL9K_TIME_FOREGROUND=$grey
# Format for the current time: 09:51:02. See `man 3 strftime`.
typeset -g POWERLEVEL9K_TIME_FORMAT='%D{%H:%M:%S}'
# If set to true, time will update when you hit enter. This way prompts for the past
# commands will contain the start times of their commands rather than the end times of
# their preceding commands.
typeset -g POWERLEVEL9K_TIME_UPDATE_ON_COMMAND=false
# Transient prompt works similarly to the builtin transient_rprompt option. It trims down prompt
# when accepting a command line. Supported values:
#
# - off: Don't change prompt when accepting a command line.
# - always: Trim down prompt when accepting a command line.
# - same-dir: Trim down prompt when accepting a command line unless this is the first command
# typed after changing current working directory.
typeset -g POWERLEVEL9K_TRANSIENT_PROMPT=off
# Instant prompt mode.
#
# - off: Disable instant prompt. Choose this if you've tried instant prompt and found
# it incompatible with your zsh configuration files.
# - quiet: Enable instant prompt and don't print warnings when detecting console output
# during zsh initialization. Choose this if you've read and understood
# https://github.com/romkatv/powerlevel10k/blob/master/README.md#instant-prompt.
# - verbose: Enable instant prompt and print a warning when detecting console output during
# zsh initialization. Choose this if you've never tried instant prompt, haven't
# seen the warning, or if you are unsure what this all means.
typeset -g POWERLEVEL9K_INSTANT_PROMPT=verbose
# Hot reload allows you to change POWERLEVEL9K options after Powerlevel10k has been initialized.
# For example, you can type POWERLEVEL9K_BACKGROUND=red and see your prompt turn red. Hot reload
# can slow down prompt by 1-2 milliseconds, so it's better to keep it turned off unless you
# really need it.
typeset -g POWERLEVEL9K_DISABLE_HOT_RELOAD=true
# If p10k is already loaded, reload configuration.
# This works even with POWERLEVEL9K_DISABLE_HOT_RELOAD=true.
(( ! $+functions[p10k] )) || p10k reload
}
# Tell `p10k configure` which file it should overwrite.
typeset -g POWERLEVEL9K_CONFIG_FILE=${${(%):-%x}:a}
(( ${#p10k_config_opts} )) && setopt ${p10k_config_opts[@]}
'builtin' 'unset' 'p10k_config_opts'
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,111 @@
# Config file for Powerlevel10k with the style of robbyrussell theme from Oh My Zsh.
#
# Original: https://github.com/ohmyzsh/ohmyzsh/wiki/Themes#robbyrussell.
#
# Replication of robbyrussell theme is exact. The only observable difference is in
# performance. Powerlevel10k prompt is very fast everywhere, even in large Git repositories.
#
# Usage: Source this file either before or after loading Powerlevel10k.
#
# source ~/powerlevel10k/config/p10k-robbyrussell.zsh
# source ~/powerlevel10k/powerlevel10k.zsh-theme
# Temporarily change options.
'builtin' 'local' '-a' 'p10k_config_opts'
[[ ! -o 'aliases' ]] || p10k_config_opts+=('aliases')
[[ ! -o 'sh_glob' ]] || p10k_config_opts+=('sh_glob')
[[ ! -o 'no_brace_expand' ]] || p10k_config_opts+=('no_brace_expand')
'builtin' 'setopt' 'no_aliases' 'no_sh_glob' 'brace_expand'
() {
emulate -L zsh -o extended_glob
# Unset all configuration options.
unset -m '(POWERLEVEL9K_*|DEFAULT_USER)~POWERLEVEL9K_GITSTATUS_DIR'
# Zsh >= 5.1 is required.
[[ $ZSH_VERSION == (5.<1->*|<6->.*) ]] || return
# Left prompt segments.
typeset -g POWERLEVEL9K_LEFT_PROMPT_ELEMENTS=(prompt_char dir vcs)
# Right prompt segments.
typeset -g POWERLEVEL9K_RIGHT_PROMPT_ELEMENTS=()
# Basic style options that define the overall prompt look.
typeset -g POWERLEVEL9K_BACKGROUND= # transparent background
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_{LEFT,RIGHT}_WHITESPACE= # no surrounding whitespace
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_SUBSEGMENT_SEPARATOR=' ' # separate segments with a space
typeset -g POWERLEVEL9K_{LEFT,RIGHT}_SEGMENT_SEPARATOR= # no end-of-line symbol
typeset -g POWERLEVEL9K_VISUAL_IDENTIFIER_EXPANSION= # no segment icons
# Green prompt symbol if the last command succeeded.
typeset -g POWERLEVEL9K_PROMPT_CHAR_OK_{VIINS,VICMD,VIVIS}_FOREGROUND=green
# Red prompt symbol if the last command failed.
typeset -g POWERLEVEL9K_PROMPT_CHAR_ERROR_{VIINS,VICMD,VIVIS}_FOREGROUND=red
# Prompt symbol: bold arrow.
typeset -g POWERLEVEL9K_PROMPT_CHAR_CONTENT_EXPANSION='%B➜ '
# Cyan current directory.
typeset -g POWERLEVEL9K_DIR_FOREGROUND=cyan
# Show only the last segment of the current directory.
typeset -g POWERLEVEL9K_SHORTEN_STRATEGY=truncate_to_last
# Bold directory.
typeset -g POWERLEVEL9K_DIR_CONTENT_EXPANSION='%B$P9K_CONTENT'
# Git status formatter.
function my_git_formatter() {
emulate -L zsh
if [[ -n $P9K_CONTENT ]]; then
# If P9K_CONTENT is not empty, it's either "loading" or from vcs_info (not from
# gitstatus plugin). VCS_STATUS_* parameters are not available in this case.
typeset -g my_git_format=$P9K_CONTENT
else
# Use VCS_STATUS_* parameters to assemble Git status. See reference:
# https://github.com/romkatv/gitstatus/blob/master/gitstatus.plugin.zsh.
typeset -g my_git_format="${1+%B%4F}git:(${1+%1F}"
my_git_format+=${${VCS_STATUS_LOCAL_BRANCH:-${VCS_STATUS_COMMIT[1,8]}}//\%/%%}
my_git_format+="${1+%4F})"
if (( VCS_STATUS_NUM_CONFLICTED || VCS_STATUS_NUM_STAGED ||
VCS_STATUS_NUM_UNSTAGED || VCS_STATUS_NUM_UNTRACKED )); then
my_git_format+=" ${1+%3F}✗"
fi
fi
}
functions -M my_git_formatter 2>/dev/null
# Disable the default Git status formatting.
typeset -g POWERLEVEL9K_VCS_DISABLE_GITSTATUS_FORMATTING=true
# Install our own Git status formatter.
typeset -g POWERLEVEL9K_VCS_CONTENT_EXPANSION='${$((my_git_formatter(1)))+${my_git_format}}'
typeset -g POWERLEVEL9K_VCS_LOADING_CONTENT_EXPANSION='${$((my_git_formatter()))+${my_git_format}}'
# Grey Git status when loading.
typeset -g POWERLEVEL9K_VCS_LOADING_FOREGROUND=246
# Instant prompt mode.
#
# - off: Disable instant prompt. Choose this if you've tried instant prompt and found
# it incompatible with your zsh configuration files.
# - quiet: Enable instant prompt and don't print warnings when detecting console output
# during zsh initialization. Choose this if you've read and understood
# https://github.com/romkatv/powerlevel10k/blob/master/README.md#instant-prompt.
# - verbose: Enable instant prompt and print a warning when detecting console output during
# zsh initialization. Choose this if you've never tried instant prompt, haven't
# seen the warning, or if you are unsure what this all means.
typeset -g POWERLEVEL9K_INSTANT_PROMPT=verbose
# Hot reload allows you to change POWERLEVEL9K options after Powerlevel10k has been initialized.
# For example, you can type POWERLEVEL9K_BACKGROUND=red and see your prompt turn red. Hot reload
# can slow down prompt by 1-2 milliseconds, so it's better to keep it turned off unless you
# really need it.
typeset -g POWERLEVEL9K_DISABLE_HOT_RELOAD=true
# If p10k is already loaded, reload configuration.
# This works even with POWERLEVEL9K_DISABLE_HOT_RELOAD=true.
(( ! $+functions[p10k] )) || p10k reload
}
# Tell `p10k configure` which file it should overwrite.
typeset -g POWERLEVEL9K_CONFIG_FILE=${${(%):-%x}:a}
(( ${#p10k_config_opts} )) && setopt ${p10k_config_opts[@]}
'builtin' 'unset' 'p10k_config_opts'
+164
View File
@@ -0,0 +1,164 @@
# Recommended font: Meslo Nerd Font patched for Powerlevel10k
Gorgeous monospace font designed by Jim Lyles for Bitstream, customized by the same for Apple,
further customized by André Berg, and finally patched by yours truly with customized scripts
originally developed by Ryan L McIntyre of Nerd Fonts. Contains all glyphs and symbols that
Powerlevel10k may need. Battle-tested in dozens of different terminals on all major operating
systems.
*FAQ*: [How was the recommended font created?](README.md#how-was-the-recommended-font-created)
## Automatic font installation
If you are using iTerm2 or Termux, `p10k configure` can install the recommended font for you.
Simply answer `Yes` when asked whether to install *Meslo Nerd Font*.
If you are using a different terminal, proceed with manual font installation. 👇
## Manual font installation
1. Download these four ttf files:
- [MesloLGS NF Regular.ttf](
https://github.com/romkatv/powerlevel10k-media/raw/master/MesloLGS%20NF%20Regular.ttf)
- [MesloLGS NF Bold.ttf](
https://github.com/romkatv/powerlevel10k-media/raw/master/MesloLGS%20NF%20Bold.ttf)
- [MesloLGS NF Italic.ttf](
https://github.com/romkatv/powerlevel10k-media/raw/master/MesloLGS%20NF%20Italic.ttf)
- [MesloLGS NF Bold Italic.ttf](
https://github.com/romkatv/powerlevel10k-media/raw/master/MesloLGS%20NF%20Bold%20Italic.ttf)
1. Double-click on each file and click "Install". This will make `MesloLGS NF` font available to all
applications on your system.
1. Configure your terminal to use this font:
- **iTerm2**: Type `p10k configure` and answer `Yes` when asked whether to install
*Meslo Nerd Font*. Alternatively, open *iTerm2 → Preferences → Profiles → Text* and set *Font* to
`MesloLGS NF`.
- **Apple Terminal**: Open *Terminal → Preferences → Profiles → Text*, click *Change* under *Font*
and select `MesloLGS NF` family.
- **Hyper**: Open *Hyper → Edit → Preferences* and change the value of `fontFamily` under
`module.exports.config` to `MesloLGS NF`.
- **Visual Studio Code**: Open *File → Preferences → Settings* (PC) or
*Code → Preferences → Settings* (Mac), enter `terminal.integrated.fontFamily` in the search box at
the top of *Settings* tab and set the value below to `MesloLGS NF`.
Consult [this screenshot](
https://raw.githubusercontent.com/romkatv/powerlevel10k-media/389133fb8c9a2347929a23702ce3039aacc46c3d/visual-studio-code-font-settings.jpg)
to see how it should look like or see [this issue](
https://github.com/romkatv/powerlevel10k/issues/671) for extra information.
- **GNOME Terminal** (the default Ubuntu terminal): Open *Terminal → Preferences* and click on the
selected profile under *Profiles*. Check *Custom font* under *Text Appearance* and select
`MesloLGS NF Regular`.
- **Konsole**: Open *Settings → Edit Current Profile → Appearance*, click *Select Font* and select
`MesloLGS NF Regular`.
- **Tilix**: Open *Tilix → Preferences* and click on the selected profile under *Profiles*. Check
*Custom font* under *Text Appearance* and select `MesloLGS NF Regular`.
- **Windows Console Host** (the old thing): Click the icon in the top left corner, then
*Properties → Font* and set *Font* to `MesloLGS NF`.
- **Windows Terminal** by Microsoft (the new thing): Open *Settings* (<kbd>Ctrl+,</kbd>), click
either on the selected profile under *Profiles* or on *Defaults*, click *Appearance* and set
*Font face* to `MesloLGS NF`.
- **IntelliJ** (and other IDEs by Jet Brains): Open *IDE → Edit → Preferences → Editor →
Color Scheme → Console Font*. Select *Use console font instead of the default* and set the font
name to `MesloLGS NF`.
- **Termux**: Type `p10k configure` and answer `Yes` when asked whether to install
*Meslo Nerd Font*.
- **Blink**: Type `config`, go to *Appearance*, tap *Add a new font*, tap *Open Gallery*, select
*MesloLGS NF.css*, tap *import* and type `exit` in the home view to reload the font.
- **Tabby** (formerly **Terminus**): Open *Settings → Appearance* and set *Font* to `MesloLGS NF`.
- **Terminator**: Open *Preferences* using the context menu. Under *Profiles* select the *General*
tab (should be selected already), uncheck *Use the system fixed width font* (if not already)
and select `MesloLGS NF Regular`. Exit the Preferences dialog by clicking *Close*.
- **Guake**: Right Click on an open terminal and open *Preferences*. Under *Appearance*
tab, uncheck *Use the system fixed width font* (if not already) and select `MesloLGS NF Regular`.
Exit the Preferences dialog by clicking *Close*.
- **MobaXterm**: Open *Settings* → *Configuration* → *Terminal* → (under *Terminal look and feel*)
and change *Font* to `MesloLGS NF`.
- **Asbrú Connection Manager**: Open *Preferences → Local Shell Options → Look and Feel*, enable
*Use these personal options* and change *Font:* under *Terminal UI* to `MesloLGS NF Regular`.
To change the font for the remote host connections, go to *Preferences → Terminal Options →
Look and Feel* and change *Font:* under *Terminal UI* to `MesloLGS NF Regular`.
- **WSLtty**: Right click on an open terminal and then on *Options*. In the *Text* section, under
*Font*, click *"Select..."* and set Font to `MesloLGS NF Regular`.
- **Yakuake**: Click *≡* → *Manage Profiles* → *New* → *Appearance*. Click *Choose* next to the
*Font* dropdown, select `MesloLGS NF` and click *OK*. Click *OK* to save the profile. Select the
new profile and click *Set as Default*.
- **Alacritty**: Create or open `~/.config/alacritty/alacritty.toml` and add the following
section to it:
```toml
[font.normal]
family = "MesloLGS NF"
```
- **foot**: Create or open `~/.config/foot/foot.ini` and add the following section to it:
```ini
font=MesloLGS NF:size=12
```
- **kitty**: Create or open `~/.config/kitty/kitty.conf` and add the following line to it:
```text
font_family MesloLGS NF
```
Restart kitty by closing all sessions and opening a new session.
- **puTTY**: Set *Window* → *Appearance* → *Font* to `MesloLGS NF`. Requires puTTY
version >= 0.75.
- **WezTerm**: Create or open `$HOME/.config/wezterm/wezterm.lua` and add the following:
```lua
local wezterm = require 'wezterm';
return {
font = wezterm.font("MesloLGS NF"),
}
```
If the file already exists, only add the line with the font to the existing return.
Also add the first line if it is not already present.
- **urxvt**: Create or open `~/.Xresources` and add the following line to it:
```text
URxvt.font: xft:MesloLGS NF:size=11
```
You can adjust the font size to your preference. After changing the config run
`xrdb ~/.Xresources` to reload it. The new config is applied to all new terminals.
- **xterm**: Create or open `~/.Xresources` and add the following line to it:
```text
xterm*faceName: MesloLGS NF
```
After changing the config run `xrdb ~/.Xresources` to reload it. The new config is applied to
all new terminals.
- **Zed**: Open `~/.config/zed/settings.json` and set `terminal.font_family` to `"MesloLGS NF"`.
```jsonc
{
"terminal": {
"font_family": "MesloLGS NF"
},
// Other settings.
}
```
- Crostini (Linux on Chrome OS): Open
chrome-untrusted://terminal/html/nassh_preferences_editor.html, set *Text font family* to
`'MesloLGS NF'` (including the quotes) and *Custom CSS (inline text)* to the following:
```css
@font-face {
font-family: "MesloLGS NF";
src: url("https://raw.githubusercontent.com/romkatv/powerlevel10k-media/master/MesloLGS%20NF%20Regular.ttf");
font-weight: normal;
font-style: normal;
}
@font-face {
font-family: "MesloLGS NF";
src: url("https://raw.githubusercontent.com/romkatv/powerlevel10k-media/master/MesloLGS%20NF%20Bold.ttf");
font-weight: bold;
font-style: normal;
}
@font-face {
font-family: "MesloLGS NF";
src: url("https://raw.githubusercontent.com/romkatv/powerlevel10k-media/master/MesloLGS%20NF%20Italic.ttf");
font-weight: normal;
font-style: italic;
}
@font-face {
font-family: "MesloLGS NF";
src: url("https://raw.githubusercontent.com/romkatv/powerlevel10k-media/master/MesloLGS%20NF%20Bold%20Italic.ttf");
font-weight: bold;
font-style: italic;
}
```
**_CAVEAT_**: If you open the normal terminal preferences these settings will be overwritten.
1. Run `p10k configure` to generate a new `~/.p10k.zsh`. The old config may work
incorrectly with the new font.
_Using a different terminal and know how to set the font for it? Share your knowledge by sending a
PR to expand the list!_
@@ -0,0 +1,4 @@
BasedOnStyle: Google
ColumnLimit: 100
DerivePointerAlignment: false
PointerAlignment: Left
@@ -0,0 +1,16 @@
* text=auto
*.cc text eol=lf
*.h text eol=lf
*.info text eol=lf
*.json text eol=lf
*.md text eol=lf
*.sh text eol=lf
*.zsh text eol=lf
/.clang-format text eol=lf
/LICENSE text eol=lf
/Makefile text eol=lf
/build text eol=lf
/install text eol=lf
/mbuild text eol=lf
@@ -0,0 +1,8 @@
*.zwc
/core
/deps/libgit2-*.tar.gz
/locks
/logs
/obj
/usrbin/gitstatusd*
/.vscode/ipch
@@ -0,0 +1,17 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/src"
],
"defines": [
],
"compilerPath": "/usr/bin/g++",
"cStandard": "c11",
"cppStandard": "c++17",
"intelliSenseMode": "gcc-x64"
}
],
"version": 4
}
@@ -0,0 +1,72 @@
{
"files.exclude": {
"*.zwc": true,
"core": true,
"locks/": true,
"logs/": true,
"obj/": true,
"usrbin/": true,
},
"files.associations": {
"array": "cpp",
"atomic": "cpp",
"*.tcc": "cpp",
"cctype": "cpp",
"chrono": "cpp",
"clocale": "cpp",
"cmath": "cpp",
"complex": "cpp",
"condition_variable": "cpp",
"cstddef": "cpp",
"cstdint": "cpp",
"cstdio": "cpp",
"cstdlib": "cpp",
"cstring": "cpp",
"ctime": "cpp",
"cwchar": "cpp",
"cwctype": "cpp",
"deque": "cpp",
"unordered_map": "cpp",
"unordered_set": "cpp",
"vector": "cpp",
"exception": "cpp",
"fstream": "cpp",
"functional": "cpp",
"future": "cpp",
"initializer_list": "cpp",
"iomanip": "cpp",
"iosfwd": "cpp",
"iostream": "cpp",
"istream": "cpp",
"limits": "cpp",
"memory": "cpp",
"mutex": "cpp",
"new": "cpp",
"numeric": "cpp",
"optional": "cpp",
"ostream": "cpp",
"ratio": "cpp",
"sstream": "cpp",
"stdexcept": "cpp",
"streambuf": "cpp",
"string_view": "cpp",
"system_error": "cpp",
"thread": "cpp",
"type_traits": "cpp",
"tuple": "cpp",
"typeinfo": "cpp",
"utility": "cpp",
"variant": "cpp",
"cstdarg": "cpp",
"charconv": "cpp",
"algorithm": "cpp",
"cinttypes": "cpp",
"iterator": "cpp",
"map": "cpp",
"memory_resource": "cpp",
"random": "cpp",
"string": "cpp",
"bit": "cpp",
"netfwd": "cpp"
}
}
@@ -0,0 +1,674 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
@@ -0,0 +1,57 @@
APPNAME ?= gitstatusd
OBJDIR ?= obj
CXX ?= g++
ZSH := $(shell command -v zsh 2> /dev/null)
VERSION ?= $(shell . ./build.info && printf "%s" "$$gitstatus_version")
# Note: -fsized-deallocation is not used to avoid binary compatibility issues on macOS.
#
# Sized delete is implemented as __ZdlPvm in /usr/lib/libc++.1.dylib but this symbol is
# missing in macOS prior to 10.13.
CXXFLAGS += -std=c++14 -funsigned-char -O3 -DNDEBUG -DGITSTATUS_VERSION=$(VERSION) -Wall -Werror # -g -fsanitize=thread
LDFLAGS += -pthread # -fsanitize=thread
LDLIBS += -lgit2 # -lprofiler -lunwind
SRCS := $(shell find src -name "*.cc")
OBJS := $(patsubst src/%.cc, $(OBJDIR)/%.o, $(SRCS))
all: $(APPNAME)
$(APPNAME): usrbin/$(APPNAME)
usrbin/$(APPNAME): $(OBJS)
$(CXX) $(OBJS) $(LDFLAGS) $(LDLIBS) -o $@
$(OBJDIR):
mkdir -p -- $(OBJDIR)
$(OBJDIR)/%.o: src/%.cc Makefile build.info | $(OBJDIR)
$(CXX) $(CXXFLAGS) -MM -MT $@ src/$*.cc >$(OBJDIR)/$*.dep
$(CXX) $(CXXFLAGS) -Wall -c -o $@ src/$*.cc
clean:
rm -rf -- $(OBJDIR)
zwc:
$(or $(ZSH),:) -fc 'for f in *.zsh install; do zcompile -R -- $$f.zwc $$f || exit; done'
minify:
rm -rf -- .clang-format .git .gitattributes .gitignore .vscode deps docs src usrbin/.gitkeep LICENSE Makefile README.md build mbuild
pkg: zwc
GITSTATUS_DAEMON= GITSTATUS_CACHE_DIR=$(shell pwd)/usrbin ./install -f
-include $(OBJS:.o=.dep)
.PHONY: help
help:
@echo "Usage: make [TARGET]"
@echo "Available targets:"
@echo " all Build $(APPNAME) (default target)"
@echo " clean Remove generated files and directories"
@echo " zwc Compile Zsh files"
@echo " minify Remove unnecessary files and folders"
@echo " pkg Create a package"
@@ -0,0 +1,530 @@
# gitstatus
**gitstatus** is a 10x faster alternative to `git status` and `git describe`. Its primary use
case is to enable fast git prompt in interactive shells.
Heavy lifting is done by **gitstatusd** -- a custom binary written in C++. It comes with Zsh and
Bash bindings for integration with shell.
## Table of Contents
1. [Using from Zsh](#using-from-zsh)
1. [Using from Bash](#using-from-bash)
2. [Using from other shells](#using-from-other-shells)
1. [How it works](#how-it-works)
1. [Benchmarks](#benchmarks)
1. [Why fast](#why-fast)
1. [Requirements](#requirements)
1. [Compiling](#compiling)
1. [License](#license)
## Using from Zsh
The easiest way to take advantage of gitstatus from Zsh is to use a theme that's already integrated
with it. For example, [Powerlevel10k](https://github.com/romkatv/powerlevel10k) is a flexible and
fast theme with first-class gitstatus integration. If you install Powerlevel10k, you don't need to
install gitstatus.
![Powerlevel10k Zsh Theme](
https://raw.githubusercontent.com/romkatv/powerlevel10k-media/master/prompt-styles-high-contrast.png)
For those who wish to use gitstatus without a theme, there is
[gitstatus.prompt.zsh](gitstatus.prompt.zsh). Install it as follows:
```zsh
git clone --depth=1 https://github.com/romkatv/gitstatus.git ~/gitstatus
echo 'source ~/gitstatus/gitstatus.prompt.zsh' >>! ~/.zshrc
```
Users in China can use the official mirror on gitee.com for faster download.<br>
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.
```zsh
git clone --depth=1 https://gitee.com/romkatv/gitstatus.git ~/gitstatus
echo 'source ~/gitstatus/gitstatus.prompt.zsh' >>! ~/.zshrc
```
Alternatively, if you have Homebrew installed:
```zsh
brew install romkatv/gitstatus/gitstatus
echo "source $(brew --prefix)/opt/gitstatus/gitstatus.prompt.zsh" >>! ~/.zshrc
```
(If you choose this option, replace `~/gitstatus` with `$(brew --prefix)/opt/gitstatus/gitstatus`
in all code snippets below.)
_Make sure to disable your current theme if you have one._
This will give you a basic yet functional prompt with git status in it. It's
[over 10x faster](#benchmarks) than any alternative that can give you comparable prompt. In order
to customize it, set `PROMPT` and/or `RPROMPT` at the end of `~/.zshrc` after sourcing
`gitstatus.prompt.zsh`. Insert `${GITSTATUS_PROMPT}` where you want git status to go. For example:
```zsh
source ~/gitstatus/gitstatus.prompt.zsh
PROMPT='%~%# ' # left prompt: directory followed by %/# (normal/root)
RPROMPT='$GITSTATUS_PROMPT' # right prompt: git status
```
The expansion of `${GITSTATUS_PROMPT}` can contain the following bits:
| segment | meaning |
|-------------|-------------------------------------------------------|
| `master` | current branch |
| `#v1` | HEAD is tagged with `v1`; not shown when on a branch |
| `@5fc6fca4` | current commit; not shown when on a branch or tag |
| `⇣1` | local branch is behind the remote by 1 commit |
| `⇡2` | local branch is ahead of the remote by 2 commits |
| `⇠3` | local branch is behind the push remote by 3 commits |
| `⇢4` | local branch is ahead of the push remote by 4 commits |
| `*5` | there are 5 stashes |
| `merge` | merge is in progress (could be some other action) |
| `~6` | there are 6 merge conflicts |
| `+7` | there are 7 staged changes |
| `!8` | there are 8 unstaged changes |
| `?9` | there are 9 untracked files |
`$GITSTATUS_PROMPT_LEN` tells you how long `$GITSTATUS_PROMPT` is when printed to the console.
[gitstatus.prompt.zsh](gitstatus.prompt.zsh) has an example of using it to truncate the current
directory.
If you'd like to change the format of git status, or want to have greater control over the
process of assembling `PROMPT`, you can copy and modify parts of
[gitstatus.prompt.zsh](gitstatus.prompt.zsh) instead of sourcing the script. Your `~/.zshrc`
might look something like this:
```zsh
source ~/gitstatus/gitstatus.plugin.zsh
function my_set_prompt() {
PROMPT='%~%# '
RPROMPT=''
if gitstatus_query MY && [[ $VCS_STATUS_RESULT == ok-sync ]]; then
RPROMPT=${${VCS_STATUS_LOCAL_BRANCH:-@${VCS_STATUS_COMMIT}}//\%/%%} # escape %
(( VCS_STATUS_NUM_STAGED )) && RPROMPT+='+'
(( VCS_STATUS_NUM_UNSTAGED )) && RPROMPT+='!'
(( VCS_STATUS_NUM_UNTRACKED )) && RPROMPT+='?'
fi
setopt no_prompt_{bang,subst} prompt_percent # enable/disable correct prompt expansions
}
gitstatus_stop 'MY' && gitstatus_start -s -1 -u -1 -c -1 -d -1 'MY'
autoload -Uz add-zsh-hook
add-zsh-hook precmd my_set_prompt
```
This snippet is sourcing `gitstatus.plugin.zsh` rather than `gitstatus.prompt.zsh`. The former
defines low-level bindings that communicate with gitstatusd over pipes. The latter is a simple
script that uses these bindings to assemble git prompt.
Unlike [Powerlevel10k](https://github.com/romkatv/powerlevel10k), code based on
[gitstatus.prompt.zsh](gitstatus.prompt.zsh) is communicating with gitstatusd synchronously. This
can make your prompt slow when working in a large git repository or on a slow machine. To avoid
this problem, call `gitstatus_query` asynchronously as documented in
[gitstatus.plugin.zsh](gitstatus.plugin.zsh). This can be quite challenging.
## Using from Bash
The easiest way to take advantage of gitstatus from Bash is via
[gitstatus.prompt.sh](gitstatus.prompt.sh). Install it as follows:
```bash
git clone --depth=1 https://github.com/romkatv/gitstatus.git ~/gitstatus
echo 'source ~/gitstatus/gitstatus.prompt.sh' >> ~/.bashrc
```
Users in China can use the official mirror on gitee.com for faster download.<br>
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.
```bash
git clone --depth=1 https://gitee.com/romkatv/gitstatus.git ~/gitstatus
echo 'source ~/gitstatus/gitstatus.prompt.sh' >> ~/.bashrc
```
Alternatively, if you have Homebrew installed:
```zsh
brew install romkatv/gitstatus/gitstatus
echo "source $(brew --prefix)/opt/gitstatus/gitstatus.prompt.sh" >> ~/.bashrc
```
(If you choose this option, replace `~/gitstatus` with `$(brew --prefix)/opt/gitstatus/gitstatus`
in all code snippets below.)
This will give you a basic yet functional prompt with git status in it. It's
[over 10x faster](#benchmarks) than any alternative that can give you comparable prompt.
![Bash Prompt with GitStatus](
https://raw.githubusercontent.com/romkatv/gitstatus/1ac366952366d89980b3f3484f270b4fa5ae4293/bash-prompt.png)
In order to customize your prompt, set `PS1` at the end of `~/.bashrc` after sourcing
`gitstatus.prompt.sh`. Insert `${GITSTATUS_PROMPT}` where you want git status to go. For example:
```bash
source ~/gitstatus/gitstatus.prompt.sh
PS1='\w ${GITSTATUS_PROMPT}\n\$ ' # directory followed by git status and $/# (normal/root)
```
The expansion of `${GITSTATUS_PROMPT}` can contain the following bits:
| segment | meaning |
|-------------|-------------------------------------------------------|
| `master` | current branch |
| `#v1` | HEAD is tagged with `v1`; not shown when on a branch |
| `@5fc6fca4` | current commit; not shown when on a branch or tag |
| `⇣1` | local branch is behind the remote by 1 commit |
| `⇡2` | local branch is ahead of the remote by 2 commits |
| `⇠3` | local branch is behind the push remote by 3 commits |
| `⇢4` | local branch is ahead of the push remote by 4 commits |
| `*5` | there are 5 stashes |
| `merge` | merge is in progress (could be some other action) |
| `~6` | there are 6 merge conflicts |
| `+7` | there are 7 staged changes |
| `!8` | there are 8 unstaged changes |
| `?9` | there are 9 untracked files |
If you'd like to change the format of git status, or want to have greater control over the
process of assembling `PS1`, you can copy and modify parts of
[gitstatus.prompt.sh](gitstatus.prompt.sh) instead of sourcing the script. Your `~/.bashrc` might
look something like this:
```bash
source ~/gitstatus/gitstatus.plugin.sh
function my_set_prompt() {
PS1='\w'
if gitstatus_query && [[ "$VCS_STATUS_RESULT" == ok-sync ]]; then
if [[ -n "$VCS_STATUS_LOCAL_BRANCH" ]]; then
PS1+=" ${VCS_STATUS_LOCAL_BRANCH//\\/\\\\}" # escape backslash
else
PS1+=" @${VCS_STATUS_COMMIT//\\/\\\\}" # escape backslash
fi
(( VCS_STATUS_HAS_STAGED" )) && PS1+='+'
(( VCS_STATUS_HAS_UNSTAGED" )) && PS1+='!'
(( VCS_STATUS_HAS_UNTRACKED" )) && PS1+='?'
fi
PS1+='\n\$ '
shopt -u promptvars # disable expansion of '$(...)' and the like
}
gitstatus_stop && gitstatus_start
PROMPT_COMMAND=my_set_prompt
```
This snippet is sourcing `gitstatus.plugin.sh` rather than `gitstatus.prompt.sh`. The former
defines low-level bindings that communicate with gitstatusd over pipes. The latter is a simple
script that uses these bindings to assemble git prompt.
Note: Bash bindings, unlike Zsh bindings, don't support asynchronous calls.
## Using from other shells
If there are no gitstatusd bindings for your shell, you'll need to get your hands dirty.
Use the existing bindings for inspiration; run `gitstatusd --help` or read the same thing in
[options.cc](src/options.cc).
## How it works
gitstatusd reads requests from stdin and prints responses to stdout. Requests contain an ID and
a directory. Responses contain the same ID and machine-readable git status for the directory.
gitstatusd keeps some state in memory for the directories it has seen in order to serve future
requests faster.
[Zsh bindings](gitstatus.plugin.zsh) and [Bash bindings](gitstatus.plugin.sh) start gitstatusd in
the background and communicate with it via pipes. Themes such as
[Powerlevel10k](https://github.com/romkatv/powerlevel10k) use these bindings to put git status in
`PROMPT`.
Note that gitstatus cannot be used as a drop-in replacement for `git status` command as it doesn't
produce output in the same format. It does perform the same computation though.
## Benchmarks
The following benchmark results were obtained on Intel i9-7900X running Ubuntu 18.04 in
a clean [chromium](https://github.com/chromium/chromium) repository synced to `9394e49a`. The
repository was checked out to an ext4 filesystem on M.2 SSD.
Three functionally equivalent tools for computing git status were benchmarked:
* `gitstatusd`
* `git` with `core.untrackedcache` enabled and `core.fsmonitor` disabled
* `lg2` -- a demo/example executable from [libgit2](https://github.com/romkatv/libgit2) that
implements a subset of `git` functionality on top of libgit2 API; for the purposes of this
benchmark the subset is sufficient to generate the same data as the other tools
Every tool was benchmark in cold and hot conditions. For `git` the first run in a repository was
considered cold, with the following runs considered hot. `lg2` was patched to compute results twice
in a single invocation without freeing the repository in between; the second run was considered hot.
The same patching was not done for `git` because `git` cannot be easily modified to refresh inmemory
index state between invocations; in fact, this limitation is one of the primary reasons developers
use libgit2. `gitstatusd` was benchmarked similarly to `lg2` with two result computations in the
same invocation.
Two commands were benchmarked: `status` and `describe`.
### Status
In this benchmark all tools were computing the equivalent of `git status`. Lower numbers are better.
| Tool | Cold | Hot |
|---------------|-----------:|------------:|
| **gitstatus** | **291 ms** | **30.9 ms** |
| git | 876 ms | 295 ms |
| lg2 | 1730 ms | 1310 ms |
gitstatusd is substantially faster than the alternatives, especially on hot runs. Note that hot runs
are of primary importance to the main use case of gitstatus in interactive shells.
The performance of `git status` fluctuated wildly in this benchmarks for reasons unknown to the
author. Moreover, performance is sticky -- once `git status` settles around a number, it stays
there for a long time. Numbers as diverse as 295, 352, 663 and 730 had been observed on hot runs on
the same repository. The number in the table is the lowest (fastest or best) that `git status` had
shown.
### Describe
In this benchmark all tools were computing the equivalent of `git describe --tags --exact-match`
to find tags that resolve to the same commit as `HEAD`. Lower numbers are better.
| Tool | Cold | Hot |
|---------------|------------:|--------------:|
| **gitstatus** | **4.04 ms** | **0.0345 ms** |
| git | 18.0 ms | 14.5 ms |
| lg2 | 185 ms | 45.2 ms |
gitstatusd is once again faster than the alternatives, more so on hot runs.
## Why fast
Since gitstatusd doesn't have to print all staged/unstaged/untracked files but only report
whether there are any, it can terminate repository scan early. It can also remember which files
were dirty on the previous run and check them first on the next run to avoid the scan entirely if
the files are still dirty. However, the benchmarks above were performed in a clean repository where
these shortcuts do not trigger. All benchmarked tools had to do the same work -- check the status
of every file in the index to see if it has changed, check every directory for newly created files,
etc. And yet, gitstatusd came ahead by a large margin. This section describes what it does that
makes it so fast.
Most of the following comparisons are done against libgit2 rather than git because of the author's
familiarity with the former but not the with latter. libgit2 has clean, well-documented APIs and an
elegant implementation, which makes it so much easier to work with and to analyze performance
bottlenecks.
### Summary for the impatient
Under the benchmark conditions described above, the equivalent of libgit2's
`git_diff_index_to_workdir` (the most expensive part of `status` command) is 46.3 times faster in
gitstatusd. The speedup comes from the following sources.
* gitstatusd uses more efficient data structures and algorithms and employs performance-conscious
coding style throughout the codebase. This reduces CPU time in userspace by 32x compared to libgit2.
* gitstatusd uses less expensive system calls and makes fewer of them. This reduces CPU time spent
in kernel by 1.9x.
* gitstatusd can utilize multiple cores to scan index and workdir in parallel with almost perfect
scaling. This reduces total run time by 12.4x while having virtually no effect on total CPU time.
### Problem statement
The most resource-intensive part of the `status` command is finding the difference between _index_
and _workdir_ (`git_diff_index_to_workdir` in libgit2). Index is a list of all files in the git
repository with their last modification times. This is an obvious simplification but it suffices for
this exposition. On disk, index is stored sorted by file path. Here's an example of git index:
| File | Last modification time |
|-------------|-----------------------:|
| Makefile | 2019-04-01T14:12:32Z |
| src/hello.c | 2019-04-01T14:12:00Z |
| src/hello.h | 2019-04-01T14:12:32Z |
This list needs to be compared to the list of files in the working directory. If any of the files
listed in the index are missing from the workdir or have different last modification time, they are
"unstaged" in gitstatusd parlance. If you run `git status`, they'll be shown as "changes not staged
for commit". Thus, any implementation of `status` command has to call `stat()` or one of its
variants on every file in the index.
In addition, all files in the working directory for which there is no entry in the index at all are
"untracked". `git status` will show them as "untracked files". Finding untracked files requires some
form of work directory traversal.
### Single-threaded scan
Let's see how `git_diff_index_to_workdir` from libgit2 accomplishes these tasks. Here's its CPU
profile from 200 hot runs over chromium repository.
![libgit2 CPU profile (hot)](
https://raw.githubusercontent.com/romkatv/gitstatus/1ac366952366d89980b3f3484f270b4fa5ae4293/cpu-profile-libgit2.png)
(The CPU profile was created with [gperftools](https://github.com/gperftools/gperftools) and
rendered with [pprof](https://github.com/google/pprof)).
We can see `__GI__lxstat` taking a lot of time. This is the `stat()` call for every file in the
index. We can also identify `__opendir`, `__readdir` and `__GI___close_nocancel` -- glibc wrappers
for reading the contents of a directory. This is for finding untracked files. Out of the total 232
seconds, 111 seconds -- or 47.7% -- was spent on these calls. The rest is computation -- comparing
strings, sorting arrays, etc.
Now let's take a look at the CPU profile of gitstatusd on the same task.
![gitstatusd CPU profile (hot)](
https://raw.githubusercontent.com/romkatv/gitstatus/1ac366952366d89980b3f3484f270b4fa5ae4293/cpu-profile-gitstatusd-hot.png)
The first impression is that this profile looks pruned. This isn't an artifact. The profile was
generated with the same tools and the same flags as the profile of libgit2.
Since both profiles were generated from the same workload, absolute numbers can be compared. We can
see that gitstatusd took 62 seconds in total compared to libgit2's 232 seconds. System calls at the
core of the algorithm are clearly visible. `__GI___fxstatat` is a flavor of `stat()`, and the other
three calls -- `__libc_openat64`, `__libc_close` and `__GI___fxstat` are responsible for opening
directories and finding untracked files. Notice that there is almost nothing else in the profile
apart from these calls. The rest of the code accounts for 3.77 seconds of CPU time -- 32 times less
than in libgit2.
So, one reason gitstatusd is fast is that it has efficient diffing code -- very little time is spent
outside of kernel. However, if we look closely, we can notice that system calls in gitstatusd are
_also_ faster than in libgit2. For example, libgit2 spent 72.07 seconds in `__GI__lxstat` while
gitstatusd spent only 48.82 seconds in `__GI___fxstatat`. There are two reasons for this difference.
First, libgit2 makes more `stat()` calls than is strictly required. It's not necessary to stat
directories because index only has files. There are 25k directories in chromium repository (and 300k
files) -- that's 25k `stat()` calls that could be avoided. The second reason is that libgit2 and
gitstatusd use different flavors of `stat()`. libgit2 uses `lstat()`, which takes a path to the file
as input. Its performance is linear in the number of subdirectories in the path because it needs to
perform a lookup for every one of them and to check permissions. gitstatusd uses `fstatat()`, which
takes a file descriptor to the parent directory and a name of the file. Just a single lookup, less
CPU time.
Similarly to `lstat()` vs `fstatat()`, it's faster to open files and directories with `openat()`
from the parent directory file descriptor than with regular `open()` that accepts full file path.
gitstatusd takes advantage of `openat()` to open directories as fast as possible. It opens about 90%
of the directories (this depends on the actual directory structure of the repository) from the
immediate parent -- the most efficient way -- and the remaining 10% it opens from the repository's
root directory. The reason it's done this way is to keep the maximum number of simultaneously open
file descriptors bounded. libgit2 can have O(repository depth) simultaneously open file descriptors,
which may be OK for a single-threaded application but can balloon to a large number when scans are
done by many threads simultaneously, like in gitstatusd.
There is no equivalent to `__opendir` or `__readdir` in the gitstatusd profile because it uses the
equivalent of [untracked cache](https://git-scm.com/docs/git-update-index#_untracked_cache) from
git. On the first scan of the workdir gitstatusd lists all files just like libgit2. But, unlike
libgit2, it remembers the last modification time of every directory along with the list of
untracked files under it. On the next scan, gitstatusd can skip listing files in directories whose
last modification time hasn't changed.
To summarize, here's what gitstatusd was doing when the CPU profile was captured:
1. `__libc_openat64`: Open every directory for which there are files in the index.
2. `__GI___fxstat`: Check last modification time of the directory. Since it's the same as on the
last scan, this directory has the same list of untracked files as before, which is empty (the
repository is clean).
3. `__GI___fxstatat`: Check last modification time for every file in the index that belongs to this
directory.
4. `__libc_close`: Close the file descriptor to the directory.
Here's how the very first scan of a repository looks like in gitstatusd:
![gitstatusd CPU profile (cold)](
https://raw.githubusercontent.com/romkatv/gitstatus/1ac366952366d89980b3f3484f270b4fa5ae4293/cpu-profile-gitstatusd-cold.png)
(Some glibc functions are mislabel on this profile. `explicit_bzero` and `__nss_passwd_lookup` are
in reality `strcmp` and `memcmp`.)
This is a superset of the previous -- hot -- profile, with an extra `syscall` and string sorting for
directory listing. gitstatusd uses `getdents64` Linux system call directly, bypassing the glibc
wrapper that libgit2 uses. This is 23% faster. The details of this optimization can be found in a
[separate document](docs/listdir.md).
### Multithreading
The diffing algorithm in gitstatusd was designed from the ground up with the intention of using it
concurrently from multiple threads. With a fast SSD, `status` is CPU bound, so taking advantage of
all available CPU cores is an obvious way to yield results faster.
gitstatusd exhibits almost perfect scaling from multithreading. Engaging all cores allows it to
produce results 12.4 times faster than in single-threaded execution. This is on Intel i9-7900X with
10 cores (20 with hyperthreading) with single-core frequency of 4.3GHz and all-core frequency of
4.0GHz.
Note: `git status` also uses all available cores in some parts of its algorithm while `lg2` does
everything in a single thread.
### Postprocessing
Once the difference between the index and the workdir is found, we have a list of _candidates_ --
files that may be unstaged or untracked. To make the final judgement, these files need to be checked
against `.gitignore` rules and a few other things.
gitstatusd uses [patched libgit2](https://github.com/romkatv/libgit2) for this step. This fork
adds several optimizations that make libgit2 faster. The patched libgit2 performs more than twice
as fast in the benchmark as the original even without changes in the user code (that is, in the
code that uses the libgit2 APIs). The fork also adds several API extensions, most notable of which
is the support for multi-threaded scans. If `lg2 status` is modified to take advantage of these
extensions, it outperforms the original libgit2 by a factor of 18. Lastly, the fork fixes a score of
bugs, most of which become apparent only when using libgit2 from multiple threads.
_WARNING: Changes to libgit2 are extensive but the testing they underwent isn't. It is
**not recommended** to use the patched libgit2 in production._
## Requirements
* To compile: binutils, cmake, gcc, g++, git and GNU make.
* To run: Linux, macOS, FreeBSD, Android, WSL, Cygwin or MSYS2.
## Compiling
There are prebuilt `gitstatusd` binaries in [releases](
https://github.com/romkatv/gitstatus/releases). When using the official shell bindings
provided by gitstatus, the right binary for your architecture gets downloaded automatically.
If prebuilt binaries don't work for you, you'll need to get your hands dirty.
### Compiling for personal use
```zsh
git clone --depth=1 https://github.com/romkatv/gitstatus.git
cd gitstatus
./build -w -s -d docker
```
Users in China can use the official mirror on gitee.com for faster download.<br>
中国大陆用户可以使用 gitee.com 上的官方镜像加速下载.
```zsh
git clone --depth=1 https://gitee.com/romkatv/gitstatus.git
cd gitstatus
./build -w -s -d docker
```
- If it says that `-d docker` is not supported on your OS, remove this flag.
- If it says that `-s` is not supported on your OS, remove this flag.
- If it tell you to install docker but you cannot or don't want to, remove `-d docker`.
- If it says that some command is missing, install it.
If everything goes well, the newly built binary will appear in `./usrbin`. It'll be picked up
by shell bindings automatically.
When you update shell bindings, they may refuse to work with the binary you've built earlier. In
this case you'll need to rebuild.
If you are using gitstatus through [Powerlevel10k](https://github.com/romkatv/powerlevel10k), the
instructions are the same except that you don't need to clone gitstatus. Instead, change your
current directory to `/path/to/powerlevel10k/gitstatus` (`/path/to/powerlevel10k` is the directory
where you've installed Powerlevel10k) and run `./build -w -s -d docker` from there as described
above.
### Compiling for distribution
It's currently neither easy nor recommended to package and distribute gitstatus. There are no
instructions you can follow that would allow you to easily update your package when new versions of
gitstatus are released. This may change in the future but not soon.
## License
GNU General Public License v3.0. See [LICENSE](LICENSE). Contributions are covered by the same
license.
+656
View File
@@ -0,0 +1,656 @@
#!/bin/sh
#
# Type `build -h` for help and see https://github.com/romkatv/gitstatus
# for full documentation.
set -ue
if [ -n "${ZSH_VERSION:-}" ]; then
emulate sh -o err_exit -o no_unset
fi
export LC_ALL=C
if [ -z "${ZSH_VERSION-}" ] && command -v zsh >/dev/null 2>&1; then
# Avoid bash 3.*.
case "${BASH_VERSION-}" in
[0-3].*) exec zsh "$0" "$@";;
esac
fi
# Avoid ksh: https://github.com/romkatv/gitstatus/issues/282.
if [ -n "${KSH_VERSION-}" ]; then
if [ -z "${ZSH_VERSION-}" ] && command -v zsh >/dev/null 2>&1; then
exec zsh "$0" "$@"
elif [ -z "${BASH_VERSION-}" ] && command -v bash >/dev/null 2>&1 &&
bash_version="$(bash --version 2>&1)"; then
case "$bash_version" in
*version\ [4-9]*|*version\ [1-9][0-9]*) exec bash "$0" "$@";;
esac
fi
fi
usage="$(command cat <<\END
Usage: build [-m ARCH] [-c CPU] [-d CMD] [-i IMAGE] [-s] [-w]
Options:
-m ARCH `uname -m` from the target machine; defaults to `uname -m`
from the local machine
-c CPU generate machine instructions for CPU of this type; this
value gets passed as `-march` (or `-mcpu` for ppc64le) to gcc;
inferred from ARCH if not set explicitly
-d CMD build in a Docker container and use CMD as the `docker`
command; e.g., `-d docker` or `-d podman`
-i IMAGE build in this Docker image; inferred from ARCH if not set
explicitly
-s install whatever software is necessary for build to
succeed; on some operating systems this option is not
supported; on others it can have partial effect
-w automatically download tarballs for dependencies if they
do not already exist in ./deps; dependencies are described
in ./build.info
END
)"
build="$(command cat <<\END
outdir="$(command pwd)"
if command -v mktemp >/dev/null 2>&1; then
workdir="$(command mktemp -d "${TMPDIR:-/tmp}"/gitstatus-build.XXXXXXXXXX)"
else
workdir="${TMPDIR:-/tmp}/gitstatus-build.tmp.$$"
command mkdir -- "$workdir"
fi
cd -- "$workdir"
workdir="$(command pwd)"
narg() { echo $#; }
if [ "$(narg $workdir)" != 1 -o -z "${workdir##*:*}" -o -z "${workdir##*=*}" ]; then
>&2 echo "[error] cannot build in this directory: $workdir"
exit 1
fi
appname=gitstatusd
libgit2_tmp="$outdir"/deps/"$appname".libgit2.tmp
cleanup() {
trap - INT QUIT TERM ILL PIPE
cd /
if ! command rm -rf -- "$workdir" "$outdir"/usrbin/"$appname".tmp "$libgit2_tmp"; then
command sleep 5
command rm -rf -- "$workdir" "$outdir"/usrbin/"$appname".tmp "$libgit2_tmp"
fi
}
trap cleanup INT QUIT TERM ILL PIPE
if [ -n "$gitstatus_install_tools" ]; then
case "$gitstatus_kernel" in
linux)
if command -v apk >/dev/null 2>&1; then
command apk update
command apk add binutils cmake gcc g++ git make musl-dev perl-utils
elif command -v apt-get >/dev/null 2>&1; then
apt-get update
apt-get install -y binutils cmake gcc g++ make wget
else
>&2 echo "[error] -s is not supported on this system"
exit 1
fi
;;
freebsd|dragonfly)
command pkg install -y cmake gmake binutils git perl5 wget
;;
openbsd)
command pkg_add cmake gmake gcc g++ git wget
;;
netbsd)
command pkgin -y install cmake gmake binutils git
;;
darwin)
if ! command -v make >/dev/null 2>&1 || ! command -v gcc >/dev/null 2>&1; then
>&2 echo "[error] please run 'xcode-select --install' and retry"
exit 1
fi
if command -v port >/dev/null 2>&1; then
sudo port -N install libiconv cmake wget
elif command -v brew >/dev/null 2>&1; then
for formula in libiconv cmake git wget; do
if command brew ls --version "$formula" &>/dev/null; then
command brew upgrade "$formula"
else
command brew install "$formula"
fi
done
else
>&2 echo "[error] please install MacPorts or Homebrew and retry"
exit 1
fi
;;
msys*|mingw*)
command pacman -Syu --noconfirm
command pacman -S --needed --noconfirm binutils cmake gcc git make perl
;;
*)
>&2 echo "[internal error] unhandled kernel: $gitstatus_kernel"
exit 1
;;
esac
fi
cpus="$(command getconf _NPROCESSORS_ONLN 2>/dev/null)" ||
cpus="$(command sysctl -n hw.ncpu 2>/dev/null)" ||
cpus=8
case "$gitstatus_cpu" in
powerpc64|powerpc64le)
archflag="-mcpu"
;;
*)
archflag="-march"
;;
esac
cflags="$archflag=$gitstatus_cpu -fno-plt -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fpie"
ldflags=
static_pie=
if [ -z "${CC-}" ]; then
case "$gitstatus_kernel" in
freebsd) export CC=clang;;
*) export CC=cc;;
esac
fi
printf 'int main() {}\n' >"$workdir"/cc-test.c
if 2>/dev/null "$CC" \
-ffile-prefix-map=x=y \
-Werror \
-c "$workdir"/cc-test.c \
-o "$workdir"/cc-test.o; then
cflags="$cflags -ffile-prefix-map=$workdir/="
fi
command rm -f -- "$workdir"/cc-test "$workdir"/cc-test.o
if 2>/dev/null "$CC" \
-fstack-clash-protection \
-Werror \
-c "$workdir"/cc-test.c \
-o "$workdir"/cc-test.o; then
cflags="$cflags -fstack-clash-protection"
fi
command rm -f -- "$workdir"/cc-test "$workdir"/cc-test.o
if 2>/dev/null "$CC" \
-fcf-protection \
-Werror \
-c "$workdir"/cc-test.c \
-o "$workdir"/cc-test.o; then
cflags="$cflags -fcf-protection"
fi
command rm -f -- "$workdir"/cc-test "$workdir"/cc-test.o
if 2>/dev/null "$CC" \
-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now \
-Werror \
"$workdir"/cc-test.c \
-o "$workdir"/cc-test; then
ldflags="$ldflags -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"
fi
command rm -f -- "$workdir"/cc-test "$workdir"/cc-test.o
if 2>/dev/null "$CC" \
-fpie -static-pie \
-Werror \
"$workdir"/cc-test.c \
-o "$workdir"/cc-test; then
static_pie='-static-pie'
fi
if [ "$gitstatus_cpu" = x86-64 ]; then
cflags="$cflags -mtune=generic"
fi
libgit2_cmake_flags=
libgit2_cflags="${CFLAGS-} $cflags -O3 -DNDEBUG"
gitstatus_cxx=g++
gitstatus_cxxflags="${CXXFLAGS-} $cflags -I${workdir}/libgit2/include -DGITSTATUS_ZERO_NSEC -D_GNU_SOURCE -D_GLIBCXX_ASSERTIONS"
gitstatus_ldflags="${LDFLAGS-} $ldflags -L${workdir}/libgit2/build"
gitstatus_ldlibs=
gitstatus_make=make
case "$gitstatus_kernel" in
linux)
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
freebsd)
gitstatus_cxx=clang++
gitstatus_make=gmake
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
dragonfly)
gitstatus_cxx=clang++12
gitstatus_make=gmake
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
openbsd)
gitstatus_cxx=eg++
gitstatus_make=gmake
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
netbsd)
gitstatus_make=gmake
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
darwin)
command mkdir -- "$workdir"/lib
if [ -e /opt/local/lib/libiconv.a ]; then
command ln -s -- /opt/local/lib/libiconv.a "$workdir"/lib
libgit2_cflags="$libgit2_cflags -I/opt/local/include"
gitstatus_cxxflags="$gitstatus_cxxflags -I/opt/local/include"
else
brew_prefix="$(command brew --prefix)"
command ln -s -- "$brew_prefix"/opt/libiconv/lib/libiconv.a "$workdir"/lib
libgit2_cflags="$libgit2_cflags -I"$brew_prefix"/opt/libiconv/include"
gitstatus_cxxflags="$gitstatus_cxxflags -I"$brew_prefix"/opt/libiconv/include"
fi
libgit2_cmake_flags="$libgit2_cmake_flags -DUSE_ICONV=ON"
gitstatus_ldlibs="$gitstatus_ldlibs -liconv"
gitstatus_ldflags="$gitstatus_ldflags -L${workdir}/lib"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=OFF"
;;
msys*|mingw*)
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
cygwin*)
gitstatus_ldflags="$gitstatus_ldflags ${static_pie:--static}"
libgit2_cmake_flags="$libgit2_cmake_flags -DENABLE_REPRODUCIBLE_BUILDS=ON"
;;
*)
>&2 echo "[internal error] unhandled kernel: $gitstatus_kernel"
exit 1
;;
esac
for cmd in cat cmake git ld ln mkdir rm strip tar "$gitstatus_make"; do
if ! command -v "$cmd" >/dev/null 2>&1; then
if [ -n "$gitstatus_install_tools" ]; then
>&2 echo "[internal error] $cmd not found"
exit 1
else
>&2 echo "[error] command not found: $cmd"
exit 1
fi
fi
done
. "$outdir"/build.info
if [ -z "${libgit2_version:-}" ]; then
>&2 echo "[internal error] libgit2_version not set"
exit 1
fi
if [ -z "${libgit2_sha256:-}" ]; then
>&2 echo "[internal error] libgit2_sha256 not set"
exit 1
fi
libgit2_tarball="$outdir"/deps/libgit2-"$libgit2_version".tar.gz
if [ ! -e "$libgit2_tarball" ]; then
if [ -n "$gitstatus_download_deps" ]; then
if ! command -v wget >/dev/null 2>&1; then
if [ -n "$gitstatus_install_tools" ]; then
>&2 echo "[internal error] wget not found"
exit 1
else
>&2 echo "[error] command not found: wget"
exit 1
fi
fi
libgit2_url=https://github.com/romkatv/libgit2/archive/"$libgit2_version".tar.gz
if ! >"$libgit2_tmp" command wget --no-config -qO- -- "$libgit2_url" &&
! >"$libgit2_tmp" command wget -qO- -- "$libgit2_url"; then
set -x
>&2 command which wget
>&2 command ls -lAd -- "$(command which wget)"
>&2 command ls -lAd -- "$outdir"
>&2 command ls -lA -- "$outdir"
>&2 command ls -lAd -- "$outdir"/deps
>&2 command ls -lA -- "$outdir"/deps
set +x
exit 1
fi
command mv -f -- "$libgit2_tmp" "$libgit2_tarball"
else
>&2 echo "[error] file not found: deps/libgit2-"$libgit2_version".tar.gz"
exit 1
fi
fi
libgit2_actual_sha256=
if command -v shasum >/dev/null 2>/dev/null; then
libgit2_actual_sha256="$(command shasum -b -a 256 -- "$libgit2_tarball")"
libgit2_actual_sha256="${libgit2_actual_sha256%% *}"
elif command -v sha256sum >/dev/null 2>/dev/null; then
libgit2_actual_sha256="$(command sha256sum -b -- "$libgit2_tarball")"
libgit2_actual_sha256="${libgit2_actual_sha256%% *}"
elif command -v sha256 >/dev/null 2>/dev/null; then
libgit2_actual_sha256="$(command sha256 -- "$libgit2_tarball" </dev/null)"
# Ignore sha256 output if it's from hashalot. It's incompatible.
if [ ${#libgit2_actual_sha256} -lt 64 ]; then
libgit2_actual_sha256=
else
libgit2_actual_sha256="${libgit2_actual_sha256##* }"
fi
fi
if [ -z "$libgit2_actual_sha256" ]; then
>&2 echo "[error] command not found: shasum or sha256sum"
exit 1
fi
if [ "$libgit2_actual_sha256" != "$libgit2_sha256" ]; then
>&2 echo "[error] sha256 mismatch"
>&2 echo ""
>&2 echo " file : deps/libgit2-$libgit2_version.tar.gz"
>&2 echo " expected: $libgit2_sha256"
>&2 echo " actual : $libgit2_actual_sha256"
exit 1
fi
cd -- "$workdir"
command tar -xzf "$libgit2_tarball"
command mv -- libgit2-"$libgit2_version" libgit2
command mkdir libgit2/build
cd libgit2/build
CFLAGS="$libgit2_cflags" command cmake \
-DCMAKE_BUILD_TYPE=None \
-DZERO_NSEC=ON \
-DTHREADSAFE=ON \
-DUSE_BUNDLED_ZLIB=ON \
-DREGEX_BACKEND=builtin \
-DUSE_HTTP_PARSER=builtin \
-DUSE_SSH=OFF \
-DUSE_HTTPS=OFF \
-DBUILD_CLAR=OFF \
-DUSE_GSSAPI=OFF \
-DUSE_NTLMCLIENT=OFF \
-DBUILD_SHARED_LIBS=OFF \
$libgit2_cmake_flags \
..
command make -j "$cpus" VERBOSE=1
APPNAME="$appname".tmp \
OBJDIR="$workdir"/gitstatus \
CXX="${CXX:-$gitstatus_cxx}" \
CXXFLAGS="$gitstatus_cxxflags" \
LDFLAGS="$gitstatus_ldflags" \
LDLIBS="$gitstatus_ldlibs" \
command "$gitstatus_make" -C "$outdir" -j "$cpus"
app="$outdir"/usrbin/"$appname"
command strip "$app".tmp
command mkdir -- "$workdir"/repo
printf '[init]\n defaultBranch = master\n' >"$workdir"/.gitconfig
(
cd -- "$workdir"/repo
GIT_CONFIG_NOSYSTEM=1 HOME="$workdir" command git init
GIT_CONFIG_NOSYSTEM=1 HOME="$workdir" command git config user.name "Your Name"
GIT_CONFIG_NOSYSTEM=1 HOME="$workdir" command git config user.email "you@example.com"
GIT_CONFIG_NOSYSTEM=1 HOME="$workdir" command git commit \
--allow-empty --allow-empty-message --no-gpg-sign -m ''
)
resp="$(printf "hello\037$workdir/repo\036" | "$app".tmp)"
case "$resp" in
hello*1*/repo*master*);;
*)
>&2 echo 'error: invalid gitstatusd response for a git repo'
exit 1
;;
esac
resp="$(printf 'hello\037\036' | "$app".tmp)"
case "$resp" in
hello*0*);;
*)
>&2 echo 'error: invalid gitstatusd response for a non-repo'
exit 1
;;
esac
command mv -f -- "$app".tmp "$app"
cleanup
command cat >&2 <<-END
-------------------------------------------------
SUCCESS: created usrbin/$appname
END
END
)"
docker_image=
docker_cmd=
gitstatus_arch=
gitstatus_cpu=
gitstatus_install_tools=
gitstatus_download_deps=
while getopts ':m:c:i:d:swh' opt "$@"; do
case "$opt" in
h)
printf '%s\n' "$usage"
exit
;;
m)
if [ -n "$gitstatus_arch" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
exit 1
fi
gitstatus_arch="$OPTARG"
;;
c)
if [ -n "$gitstatus_cpu" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
exit 1
fi
gitstatus_cpu="$OPTARG"
;;
i)
if [ -n "$docker_image" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
exit 1
fi
docker_image="$OPTARG"
;;
d)
if [ -n "$docker_cmd" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
exit 1
fi
docker_cmd="$OPTARG"
;;
s)
if [ -n "$gitstatus_install_tools" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
gitstatus_install_tools=1
;;
w)
if [ -n "$gitstatus_download_deps" ]; then
>&2 echo "[error] duplicate option: -$opt"
exit 1
fi
gitstatus_download_deps=1
;;
\?) >&2 echo "[error] invalid option: -$OPTARG" ; exit 1;;
:) >&2 echo "[error] missing required argument: -$OPTARG"; exit 1;;
*) >&2 echo "[internal error] unhandled option: -$opt" ; exit 1;;
esac
done
if [ "$OPTIND" -le $# ]; then
>&2 echo "[error] unexpected positional argument"
exit 1
fi
if [ -n "$docker_image" -a -z "$docker_cmd" ]; then
>&2 echo "[error] cannot use -i without -d"
exit 1
fi
if [ -z "$gitstatus_arch" ]; then
gitstatus_arch="$(uname -m)"
gitstatus_arch="$(printf '%s' "$gitstatus_arch" | tr '[A-Z]' '[a-z]')"
fi
if [ -z "$gitstatus_cpu" ]; then
case "$gitstatus_arch" in
armel) gitstatus_cpu=armv5;;
armv6l|armhf) gitstatus_cpu=armv6;;
armv7l) gitstatus_cpu=armv7;;
arm64|aarch64) gitstatus_cpu=armv8-a;;
ppc64|ppc64le) gitstatus_cpu=powerpc64le;;
riscv64) gitstatus_cpu=rv64imafdc;;
loongarch64) gitstatus_cpu=loongarch64;;
x86_64|amd64) gitstatus_cpu=x86-64;;
x86) gitstatus_cpu=i586;;
s390x) gitstatus_cpu=z900;;
i386|i586|i686) gitstatus_cpu="$gitstatus_arch";;
*)
>&2 echo '[error] unable to infer target CPU architecture'
>&2 echo 'Please specify explicitly with `-c CPU`.'
exit 1
;;
esac
fi
gitstatus_kernel="$(uname -s)"
gitstatus_kernel="$(printf '%s' "$gitstatus_kernel" | tr '[A-Z]' '[a-z]')"
case "$gitstatus_kernel" in
linux)
if [ -n "$docker_cmd" ]; then
if [ -z "${docker_cmd##*/*}" ]; then
if [ ! -x "$docker_cmd" ]; then
>&2 echo "[error] not an executable file: $docker_cmd"
exit 1
fi
else
if ! command -v "$docker_cmd" >/dev/null 2>&1; then
>&2 echo "[error] command not found: $docker_cmd"
exit 1
fi
fi
if [ -z "$docker_image" ]; then
case "$gitstatus_arch" in
x86_64) docker_image=alpine:3.11.6;;
x86|i386|i586|i686) docker_image=i386/alpine:3.11.6;;
armv6l|armhf) docker_image=arm32v6/alpine:3.11.6;;
armv7l) docker_image=arm32v7/alpine:3.11.6;;
aarch64) docker_image=arm64v8/alpine:3.11.6;;
ppc64|ppc64le) docker_image=ppc64le/alpine:3.11.6;;
s390x) docker_image=s390x/alpine:3.11.6;;
*)
>&2 echo '[error] unable to infer docker image'
>&2 echo 'Please specify explicitly with `-i IMAGE`.'
exit 1
;;
esac
fi
fi
;;
freebsd|openbsd|netbsd|darwin|dragonfly)
if [ -n "$docker_cmd" ]; then
>&2 echo "[error] docker (-d) is not supported on $gitstatus_kernel"
exit 1
fi
;;
msys_nt-*|mingw32_nt-*|mingw64_nt-*|cygwin_nt-*)
if ! printf '%s' "$gitstatus_kernel" | grep -Eqx '[^-]+-[0-9]+\.[0-9]+(-.*)?'; then
>&2 echo '[error] unsupported kernel, sorry!'
exit 1
fi
gitstatus_kernel="$(printf '%s' "$gitstatus_kernel" | sed 's/^\([^-]*-[0-9]*\.[0-9]*\).*/\1/')"
if [ -n "$docker_cmd" ]; then
>&2 echo '[error] docker (-d) is not supported on windows'
exit 1
fi
if [ -n "$gitstatus_install_tools" -a -z "${gitstatus_kernel##cygwin_nt-*}" ]; then
>&2 echo '[error] -s is not supported on cygwin'
exit 1
fi
;;
*)
>&2 echo '[error] unsupported kernel, sorry!'
exit 1
;;
esac
dir="$(dirname -- "$0")"
cd -- "$dir"
dir="$(pwd)"
>&2 echo "Building gitstatusd..."
>&2 echo ""
>&2 echo " kernel := $gitstatus_kernel"
>&2 echo " arch := $gitstatus_arch"
>&2 echo " cpu := $gitstatus_cpu"
[ -z "$docker_cmd" ] || >&2 echo " docker command := $docker_cmd"
[ -z "$docker_image" ] || >&2 echo " docker image := $docker_image"
if [ -n "$gitstatus_install_tools" ]; then
>&2 echo " install tools := yes"
else
>&2 echo " install tools := no"
fi
if [ -n "$gitstatus_download_deps" ]; then
>&2 echo " download deps := yes"
else
>&2 echo " download deps := no"
fi
if [ -n "$docker_cmd" ]; then
"$docker_cmd" run \
-e docker_cmd="$docker_cmd" \
-e docker_image="$docker_image" \
-e gitstatus_kernel="$gitstatus_kernel" \
-e gitstatus_arch="$gitstatus_arch" \
-e gitstatus_cpu="$gitstatus_cpu" \
-e gitstatus_install_tools="$gitstatus_install_tools" \
-e gitstatus_download_deps="$gitstatus_download_deps" \
-v "$dir":/out \
-w /out \
--rm \
-- "$docker_image" /bin/sh -uexc "$build"
else
eval "$build"
fi
@@ -0,0 +1,22 @@
# This value gets embedded in gitstatusd at build time. It is
# read by ./Makefile. `gitstatusd --version` reports it back.
#
# This value is also read by shell bindings (indirectly, through
# ./install) when using GITSTATUS_DAEMON or usrbin/gitstatusd.
gitstatus_version="v1.5.4"
# libgit2 is a build time dependency of gitstatusd. The values of
# libgit2_version and libgit2_sha256 are read by ./build.
#
# If ./deps/libgit2-${libgit2_version}.tar.gz doesn't exist, build
# downloads it from the following location:
#
# https://github.com/romkatv/libgit2/archive/${libgit2_version}.tar.gz
#
# Once downloaded, the tarball is stored at the path indicated
# above so that repeated builds don't consume network bandwidth.
#
# If sha256 of ./deps/libgit2-${libgit2_version}.tar.gz doesn't match,
# build gets aborted.
libgit2_version="tag-2ecf33948a4df9ef45a66c68b8ef24a5e60eaac6"
libgit2_sha256="4ce11d71ee576dbbc410b9fa33a9642809cc1fa687b315f7c23eeb825b251e93"
@@ -0,0 +1,330 @@
# Fast directory listing
In order to find untracked files in a git repository, [gitstatusd](../README.md) needs to list the
contents of every directory. gitstatusd does it 27% faster than a reasonable implementation that a
seasoned C/C++ practitioner might write. This document explains the optimizations that went into it.
As directory listing is a common operation, many other projects can benefit from applying these
optimizations.
## v1
Given a path to a directory, `ListDir()` must produce the list of files in that directory. Moreover,
the list must be sorted lexicographically to enable fast comparison with Git index.
The following C++ implementation gets the job done. For simplicity, it returns an empty list on
error.
```c++
vector<string> ListDir(const char* dirname) {
vector<string> entries;
if (DIR* dir = opendir(dirname)) {
while (struct dirent* ent = (errno = 0, readdir(dir))) {
if (!Dots(ent->d_name)) entries.push_back(ent->d_name);
}
if (errno) entries.clear();
sort(entries.begin(), entries.end());
closedir(dir);
}
return entries;
}
```
Every directory has entries `"."` and `".."`, which we aren't interested in. We filter them out with
a helper function `Dots()`.
```c++
bool Dots(const char* s) { return s[0] == '.' && (!s[1] || (s[1] == '.' && !s[2])); }
```
To check how fast `ListDir()` performs, we can run it many times on a typical directory. One million
runs on a directory with 32 files with 16-character names takes 12.7 seconds.
## v2
Experienced C++ practitioners will scoff at our implementation of `ListDir()`. If it's meant to be
efficient, returning `vector<string>` is an unaffordable convenience. To avoid heap allocations we
can use a simple arena that will allow us to reuse memory between different `ListDir()` calls.
(Changed and added lines are marked with comments.)
```c++
void ListDir(const char* dirname, string& arena, vector<char*>& entries) { // +
entries.clear(); // +
if (DIR* dir = opendir(dirname)) {
arena.clear(); // +
while (struct dirent* ent = (errno = 0, readdir(dir))) {
if (!Dots(ent->d_name)) {
entries.push_back(reinterpret_cast<char*>(arena.size())); // +
arena.append(ent->d_name, strlen(ent->d_name) + 1); // +
}
}
if (errno) entries.clear();
for (char*& p : entries) p = &arena[reinterpret_cast<size_t>(p)]; // +
sort(entries.begin(), entries.end(), // +
[](const char* a, const char* b) { return strcmp(a, b) < 0; }); // +
closedir(dir);
}
}
```
To make performance comparison easier, we can normalize them relative to the baseline. v1 will get
performance score of 100. A twice-as-fast alternative will be 200.
| version | optimization | score |
|---------|----------------------------|----------:|
| v1 | baseline | 100.0 |
| **v2** | **avoid heap allocations** | **112.7** |
Avoiding heap allocations makes `ListDir()` 12.7% faster. Not bad. As an added bonus, those casts
will fend off the occasional frontend developer who accidentally wanders into the codebase.
## v3
`opendir()` is an expensive call whose performance is linear in the number of subdirectories in the
path because it needs to perform a lookup for every one of them. We can replace it with `openat()`,
which takes a file descriptor to the parent directory and a name of the subdirectory. Just a single
lookup, less CPU time. This optimization assumes that callers already have a descriptor to the
parent directory, which is indeed the case for gitstatusd, and is often the case in other
applications that traverse filesystem.
```c++
void ListDir(int parent_fd, const char* dirname, string& arena, vector<char*>& entries) { // +
entries.clear();
int dir_fd = openat(parent_fd, dirname, O_NOATIME | O_RDONLY | O_DIRECTORY | O_CLOEXEC); // +
if (dir_fd < 0) return; // +
if (DIR* dir = fdopendir(dir_fd)) {
arena.clear();
while (struct dirent* ent = (errno = 0, readdir(dir))) {
if (!Dots(ent->d_name)) {
entries.push_back(reinterpret_cast<char*>(arena.size()));
arena.append(ent->d_name, strlen(ent->d_name) + 1);
}
}
if (errno) entries.clear();
for (char*& p : entries) p = &arena[reinterpret_cast<size_t>(p)];
sort(entries.begin(), entries.end(),
[](const char* a, const char* b) { return strcmp(a, b) < 0; });
closedir(dir);
} else { // +
close(dir_fd); // +
} // +
}
```
This is worth about 3.5% in speed.
| version | optimization | score |
|---------|--------------------------------------|----------:|
| v1 | baseline | 100.0 |
| v2 | avoid heap allocations | 112.7 |
| **v3** | **open directories with `openat()`** | **116.2** |
## v4
Copying file names to the arena isn't free but it doesn't seem like we can avoid it. Poking around
we can see that the POSIX API we are using is implemented on Linux on top of `getdents64` system
call. Its documentation isn't very encouraging:
```text
These are not the interfaces you are interested in. Look at
readdir(3) for the POSIX-conforming C library interface. This page
documents the bare kernel system call interfaces.
Note: There are no glibc wrappers for these system calls.
```
Hmm... The API looks like something we can take advantage of, so let's try it anyway.
First, we'll need a simple `Arena` class that can allocate 8KB blocks of memory.
```c++
class Arena {
public:
enum { kBlockSize = 8 << 10 };
char* Alloc() {
if (cur_ == blocks_.size()) blocks_.emplace_back(kBlockSize, 0);
return blocks_[cur_++].data();
}
void Clear() { cur_ = 0; }
private:
size_t cur_ = 0;
vector<string> blocks_;
};
```
Next, we need to define `struct dirent64_t` ourselves because there is no wrapper for the system
call we are about to use.
```c++
struct dirent64_t {
ino64_t d_ino;
off64_t d_off;
unsigned short d_reclen;
unsigned char d_type;
char d_name[];
};
```
Finally we can get to the implementation of `ListDir()`.
```c++
void ListDir(int parent_fd, Arena& arena, vector<char*>& entries) { // +
entries.clear();
int dir_fd = openat(parent_fd, dirname, O_NOATIME | O_RDONLY | O_DIRECTORY | O_CLOEXEC);
if (dir_fd < 0) return;
arena.Clear(); // +
while (true) { // +
char* buf = arena.Alloc(); // +
int n = syscall(SYS_getdents64, dir_fd, buf, Arena::kBlockSize); // +
if (n <= 0) { // +
if (n) entries.clear(); // +
break; // +
} // +
for (int pos = 0; pos < n;) { // +
auto* ent = reinterpret_cast<dirent64_t*>(buf + pos); // +
if (!Dots(ent->d_name)) entries.push_back(ent->d_name); // +
pos += ent->d_reclen; // +
} // +
} // +
sort(entries.begin(), entries.end(),
[](const char* a, const char* b) { return strcmp(a, b) < 0; });
close(dir_fd);
}
```
How are we doing with this one?
| version | optimization | score |
|---------|----------------------------------|----------:|
| v1 | baseline | 100.0 |
| v2 | avoid heap allocations | 112.7 |
| v3 | open directories with `openat()` | 116.2 |
| **v4** | **call `getdents64()` directly** | **137.8** |
Solid 20% speedup. Worth the trouble. Unfortunately, we now have just one `reinterpret_cast` instead
of two, and it's not nearly as scary-looking. Hopefully with the next iteration we can get back some
of that evil vibe of low-level code.
As a bonus, every element in `entries` has `d_type` at offset -1. This can be useful to the callers
that need to distinguish between regular files and directories (gitstatusd, in fact, needs this).
Note how `ListDir()` implements this feature at zero cost, as a lucky accident of `dirent64_t`
memory layout.
## v5
The CPU profile of `ListDir()` reveals that almost all userspace CPU time is spent in `strcmp()`.
Digging into the source code of `std::sort()` we can see that it uses Insertion Sort for short
collections. Our 32-element vector falls under the threshold. Insertion Sort makes `O(N^2)`
comparisons, hence a lot of CPU time in `strcmp()`. Switching to `qsort()` or
[Timsort](https://en.wikipedia.org/wiki/Timsort) is of no use as all good sorting algorithms fall
back to Insertion Sort.
If we cannot make fewer comparisons, perhaps we can make each of them faster? `strcmp()` compares
characters one at a time. It cannot read ahead as it can be illegal to touch memory past the first
null byte. But _we_ know that it's safe to read a few extra bytes past the end of `d_name` for every
entry except the last in the buffer. And since we own the buffer, we can overallocate it so that
reading past the end of the last entry is also safe.
Combining these ideas with the fact that file names on Linux are at most 255 bytes long, we can
invoke `getdents64()` like this:
```c++
int n = syscall(SYS_getdents64, dir_fd, buf, Arena::kBlockSize - 256);
```
And then compare entries like this:
```c++
[](const char* a, const char* b) { return memcmp(a, b, 255) < 0; }
```
This version doesn't give any speedup compared to the previous but it opens an avenue for another
optimization. The pointers we pass to `memcmp()` aren't aligned. To be more specific, their
numerical values are `N * 8 + 3` for some `N`. When given such a pointer, `memcmp()` will check the
first 5 bytes one by one, and only then switch to comparing 8 bytes at a time. If we can handle the
first 5 bytes ourselves, we can pass aligned memory to `memcmp()` and take full advantage of its
vectorized loop.
Here's the implementation:
```c++
uint64_t Read64(const void* p) { // +
uint64_t x; // +
memcpy(&x, p, sizeof(x)); // +
return x; // +
} // +
void ByteSwap64(void* p) { // +
uint64_t x = __builtin_bswap64(Read64(p)); // +
memcpy(p, &x, sizeof(x)); // +
} // +
void ListDir(int parent_fd, Arena& arena, vector<char*>& entries) {
entries.clear();
int dir_fd = openat(parent_fd, dirname, O_NOATIME | O_RDONLY | O_DIRECTORY | O_CLOEXEC);
if (dir_fd < 0) return;
arena.Clear();
while (true) {
char* buf = arena.Alloc();
int n = syscall(SYS_getdents64, dir_fd, buf, Arena::kBlockSize - 256); // +
if (n <= 0) {
if (n) entries.clear();
break;
}
for (int pos = 0; pos < n;) {
auto* ent = reinterpret_cast<dirent64_t*>(buf + pos);
if (!Dots(ent->d_name)) {
ByteSwap64(ent->d_name); // +
entries.push_back(ent->d_name);
}
pos += ent->d_reclen;
}
}
sort(entries.begin(), entries.end(), [](const char* a, const char* b) {
uint64_t x = Read64(a); // +
uint64_t y = Read64(b); // +
return x < y || (x == y && a != b && memcmp(a + 5, b + 5, 256) < 0); // +
});
for (char* p : entries) ByteSwap64(p); // +
close(dir_fd);
}
```
This is for Little Endian architecture. Big Endian doesn't need `ByteSwap64()`, so it'll be a bit
faster.
| version | optimization | score |
|---------|----------------------------------|----------:|
| v1 | baseline | 100.0 |
| v2 | avoid heap allocations | 112.7 |
| v3 | open directories with `openat()` | 116.2 |
| v4 | call `getdents64()` directly | 137.8 |
| **v5** | **hand-optimize `strcmp()`** | **143.3** |
Fast and respectably arcane.
## Conclusion
Through a series of incremental improvements we've sped up directory listing by 43.3% compared to a
naive implementation (v1) and 27.2% compared to a reasonable implementation that a seasoned C/C++
practitioner might write (v2).
However, these numbers are based on an artificial benchmark while the real judge is always the real
code. Our goal was to speed up gitstatusd. Benchmark was just a tool. Thankfully, the different
versions of `ListDir()` have the same comparative performance within gitstatusd as in the benchmark.
In truth, the directory chosen for the benchmark wasn't arbitrary. It was picked by sampling
gitstatusd when it runs on [chromium](https://github.com/chromium/chromium) git repository.
The final version of `ListDir()` spends 97% of its CPU time in the kernel. If we assume that it
makes the minimum possible number of system calls and these calls are optimal (true to the best
of my knowledge), it puts the upper bound on possible future performance improvements at just 3%.
There is almost nothing left in `ListDir()` to optimize.
![ListDir() CPU profile](
https://raw.githubusercontent.com/romkatv/gitstatus/1ac366952366d89980b3f3484f270b4fa5ae4293/cpu-profile-listdir.png)
(The CPU profile was created with [gperftools](https://github.com/gperftools/gperftools) and
rendered with [pprof](https://github.com/google/pprof)).
@@ -0,0 +1,474 @@
# Bash bindings for gitstatus.
[[ $- == *i* ]] || return # non-interactive shell
# Starts gitstatusd in the background. Does nothing and succeeds if gitstatusd
# is already running.
#
# Usage: gitstatus_start [OPTION]...
#
# -t FLOAT Fail the self-check on initialization if not getting a response from
# gitstatusd for this this many seconds. Defaults to 5.
#
# -s INT Report at most this many staged changes; negative value means infinity.
# Defaults to 1.
#
# -u INT Report at most this many unstaged changes; negative value means infinity.
# Defaults to 1.
#
# -c INT Report at most this many conflicted changes; negative value means infinity.
# Defaults to 1.
#
# -d INT Report at most this many untracked files; negative value means infinity.
# Defaults to 1.
#
# -m INT Report -1 unstaged, untracked and conflicted if there are more than this many
# files in the index. Negative value means infinity. Defaults to -1.
#
# -e Count files within untracked directories like `git status --untracked-files`.
#
# -U Unless this option is specified, report zero untracked files for repositories
# with status.showUntrackedFiles = false.
#
# -W Unless this option is specified, report zero untracked files for repositories
# with bash.showUntrackedFiles = false.
#
# -D Unless this option is specified, report zero staged, unstaged and conflicted
# changes for repositories with bash.showDirtyState = false.
#
# -r INT Close git repositories that haven't been used for this many seconds. This is
# meant to release resources such as memory and file descriptors. The next request
# for a repo that's been closed is much slower than for a repo that hasn't been.
# Negative value means infinity. The default is 3600 (one hour).
function gitstatus_start() {
if [[ "$BASH_VERSION" < 4 ]]; then
>&2 printf 'gitstatus_start: need bash version >= 4.0, found %s\n' "$BASH_VERSION"
>&2 printf '\n'
>&2 printf 'To see the version of the current shell, type:\n'
>&2 printf '\n'
>&2 printf ' \033[32mecho\033[0m \033[33m"$BASH_VERSION"\033[0m\n'
>&2 printf '\n'
>&2 printf 'The output of `\033[32mbash\033[0m --version` may be different and is not relevant.\n'
return 1
fi
unset OPTIND
local opt timeout=5 max_dirty=-1 ttl=3600 extra_flags=
local max_num_staged=1 max_num_unstaged=1 max_num_conflicted=1 max_num_untracked=1
while getopts "t:s:u:c:d:m:r:eUWD" opt; do
case "$opt" in
t) timeout=$OPTARG;;
s) max_num_staged=$OPTARG;;
u) max_num_unstaged=$OPTARG;;
c) max_num_conflicted=$OPTARG;;
d) max_num_untracked=$OPTARG;;
m) max_dirty=$OPTARG;;
r) ttl=$OPTARG;;
e) extra_flags+='--recurse-untracked-dirs ';;
U) extra_flags+='--ignore-status-show-untracked-files ';;
W) extra_flags+='--ignore-bash-show-untracked-files ';;
D) extra_flags+='--ignore-bash-show-dirty-state ';;
*) return 1;;
esac
done
(( OPTIND == $# + 1 )) || { echo "usage: gitstatus_start [OPTION]..." >&2; return 1; }
[[ -z "${GITSTATUS_DAEMON_PID:-}" ]] || return 0 # already started
if [[ "${BASH_SOURCE[0]}" == */* ]]; then
local gitstatus_plugin_dir="${BASH_SOURCE[0]%/*}"
if [[ "$gitstatus_plugin_dir" != /* ]]; then
gitstatus_plugin_dir="$PWD"/"$gitstatus_plugin_dir"
fi
else
local gitstatus_plugin_dir="$PWD"
fi
local tmpdir req_fifo resp_fifo culprit
function gitstatus_start_impl() {
local log_level="${GITSTATUS_LOG_LEVEL:-}"
[[ -n "$log_level" || "${GITSTATUS_ENABLE_LOGGING:-0}" != 1 ]] || log_level=INFO
local uname_sm
uname_sm="$(command uname -sm)" || return
uname_sm="${uname_sm,,}"
local uname_s="${uname_sm% *}"
local uname_m="${uname_sm#* }"
if [[ "${GITSTATUS_NUM_THREADS:-0}" -gt 0 ]]; then
local threads="$GITSTATUS_NUM_THREADS"
else
local cpus
if ! command -v sysctl &>/dev/null || [[ "$uname_s" == linux ]] ||
! cpus="$(command sysctl -n hw.ncpu)"; then
if ! command -v getconf &>/dev/null || ! cpus="$(command getconf _NPROCESSORS_ONLN)"; then
cpus=8
fi
fi
local threads=$((cpus > 16 ? 32 : cpus > 0 ? 2 * cpus : 16))
fi
local daemon_args=(
--parent-pid="$$"
--num-threads="$threads"
--max-num-staged="$max_num_staged"
--max-num-unstaged="$max_num_unstaged"
--max-num-conflicted="$max_num_conflicted"
--max-num-untracked="$max_num_untracked"
--dirty-max-index-size="$max_dirty"
--repo-ttl-seconds="$ttl"
$extra_flags)
if [[ -n "$TMPDIR" && ( ( -d "$TMPDIR" && -w "$TMPDIR" ) || ! ( -d /tmp && -w /tmp ) ) ]]; then
local tmpdir=$TMPDIR
else
local tmpdir=/tmp
fi
tmpdir="$(command mktemp -d "$tmpdir"/gitstatus.bash.$$.XXXXXXXXXX)" || return
if [[ -n "$log_level" ]]; then
GITSTATUS_DAEMON_LOG="$tmpdir"/daemon.log
[[ "$log_level" == INFO ]] || daemon_args+=(--log-level="$log_level")
else
GITSTATUS_DAEMON_LOG=/dev/null
fi
req_fifo="$tmpdir"/req.fifo
resp_fifo="$tmpdir"/resp.fifo
command mkfifo -- "$req_fifo" "$resp_fifo" || return
{
(
trap '' INT QUIT TSTP
[[ "$GITSTATUS_DAEMON_LOG" == /dev/null ]] || set -x
builtin cd /
(
local fd_in fd_out
exec {fd_in}<"$req_fifo" {fd_out}>>"$resp_fifo" || exit
echo "$BASHPID" >&"$fd_out"
local _gitstatus_bash_daemon _gitstatus_bash_version _gitstatus_bash_downloaded
function _gitstatus_set_daemon() {
_gitstatus_bash_daemon="$1"
_gitstatus_bash_version="$2"
_gitstatus_bash_downloaded="$3"
}
set -- -d "$gitstatus_plugin_dir" -s "$uname_s" -m "$uname_m" \
-p "printf '.\036' >&$fd_out" -e "$fd_out" -- _gitstatus_set_daemon
[[ "${GITSTATUS_AUTO_INSTALL:-1}" -ne 0 ]] || set -- -n "$@"
source "$gitstatus_plugin_dir"/install || return
[[ -n "$_gitstatus_bash_daemon" ]] || return
[[ -n "$_gitstatus_bash_version" ]] || return
[[ "$_gitstatus_bash_downloaded" == [01] ]] || return
local sig=(TERM ILL PIPE)
if (( UID == EUID )); then
local home=~
else
local user
user="$(command id -un)" || return
[[ "$user" =~ ^[a-zA-Z0-9_,.-]+$ ]] || return
eval "local home=~$user"
[[ -n "$home" ]] || return
fi
if [[ -x "$_gitstatus_bash_daemon" ]]; then
HOME="$home" "$_gitstatus_bash_daemon" \
-G "$_gitstatus_bash_version" "${daemon_args[@]}" <&"$fd_in" >&"$fd_out" &
local pid=$!
trap "trap - ${sig[*]}; kill $pid &>/dev/null" ${sig[@]}
wait "$pid"
local ret=$?
trap - ${sig[@]}
case "$ret" in
0|129|130|131|137|141|143|159)
echo -nE $'}bye\x1f0\x1e' >&"$fd_out"
exit "$ret"
;;
esac
fi
(( ! _gitstatus_bash_downloaded )) || return
[[ "${GITSTATUS_AUTO_INSTALL:-1}" -ne 0 ]] || return
[[ "$_gitstatus_bash_daemon" == \
"${GITSTATUS_CACHE_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/gitstatus}"/* ]] || return
set -- -f "$@"
_gitstatus_bash_daemon=
_gitstatus_bash_version=
_gitstatus_bash_downloaded=
source "$gitstatus_plugin_dir"/install || return
[[ -n "$_gitstatus_bash_daemon" ]] || return
[[ -n "$_gitstatus_bash_version" ]] || return
[[ "$_gitstatus_bash_downloaded" == 1 ]] || return
HOME="$home" "$_gitstatus_bash_daemon" \
-G "$_gitstatus_bash_version" "${daemon_args[@]}" <&"$fd_in" >&"$fd_out" &
local pid=$!
trap "trap - ${sig[*]}; kill $pid &>/dev/null" ${sig[@]}
wait "$pid"
trap - ${sig[@]}
echo -nE $'}bye\x1f0\x1e' >&"$fd_out"
) & disown
) & disown
} 0</dev/null &>"$GITSTATUS_DAEMON_LOG"
exec {_GITSTATUS_REQ_FD}>>"$req_fifo" {_GITSTATUS_RESP_FD}<"$resp_fifo" || return
command rm -f -- "$req_fifo" "$resp_fifo" || return
[[ "$GITSTATUS_DAEMON_LOG" != /dev/null ]] || command rmdir -- "$tmpdir" 2>/dev/null
IFS='' read -r -u $_GITSTATUS_RESP_FD GITSTATUS_DAEMON_PID || return
[[ "$GITSTATUS_DAEMON_PID" == [1-9]* ]] || return
local reply
echo -nE $'}hello\x1f\x1e' >&$_GITSTATUS_REQ_FD || return
local dl=
while true; do
reply=
if ! IFS='' read -rd $'\x1e' -u $_GITSTATUS_RESP_FD -t "$timeout" reply; then
culprit="$reply"
return 1
fi
[[ "$reply" == $'}hello\x1f0' ]] && break
if [[ -z "$dl" ]]; then
dl=1
if [[ -t 2 ]]; then
local spinner=('\b\033[33m-\033[0m' '\b\033[33m\\\033[0m' '\b\033[33m|\033[0m' '\b\033[33m/\033[0m')
>&2 printf '[\033[33mgitstatus\033[0m] fetching \033[32mgitstatusd\033[0m .. '
else
local spinner=('.')
>&2 printf '[gitstatus] fetching gitstatusd ..'
fi
fi
>&2 printf "${spinner[0]}"
spinner=("${spinner[@]:1}" "${spinner[0]}")
done
if [[ -n "$dl" ]]; then
if [[ -t 2 ]]; then
>&2 printf '\b[\033[32mok\033[0m]\n'
else
>&2 echo ' [ok]'
fi
fi
_GITSTATUS_DIRTY_MAX_INDEX_SIZE=$max_dirty
_GITSTATUS_CLIENT_PID="$BASHPID"
}
if ! gitstatus_start_impl; then
>&2 printf '\n'
>&2 printf '[\033[31mERROR\033[0m]: gitstatus failed to initialize.\n'
if [[ -n "${culprit-}" ]]; then
>&2 printf '\n%s\n' "$culprit"
fi
[[ -z "${req_fifo:-}" ]] || command rm -f "$req_fifo"
[[ -z "${resp_fifo:-}" ]] || command rm -f "$resp_fifo"
unset -f gitstatus_start_impl
gitstatus_stop
return 1
fi
export _GITSTATUS_CLIENT_PID _GITSTATUS_REQ_FD _GITSTATUS_RESP_FD GITSTATUS_DAEMON_PID
unset -f gitstatus_start_impl
}
# Stops gitstatusd if it's running.
function gitstatus_stop() {
if [[ "${_GITSTATUS_CLIENT_PID:-$BASHPID}" == "$BASHPID" ]]; then
[[ -z "${_GITSTATUS_REQ_FD:-}" ]] || exec {_GITSTATUS_REQ_FD}>&- || true
[[ -z "${_GITSTATUS_RESP_FD:-}" ]] || exec {_GITSTATUS_RESP_FD}>&- || true
[[ -z "${GITSTATUS_DAEMON_PID:-}" ]] || kill "$GITSTATUS_DAEMON_PID" &>/dev/null || true
fi
unset _GITSTATUS_REQ_FD _GITSTATUS_RESP_FD GITSTATUS_DAEMON_PID
unset _GITSTATUS_DIRTY_MAX_INDEX_SIZE _GITSTATUS_CLIENT_PID
}
# Retrieves status of a git repository from a directory under its working tree.
#
# Usage: gitstatus_query [OPTION]...
#
# -d STR Directory to query. Defaults to $PWD. Has no effect if GIT_DIR is set.
# -t FLOAT Timeout in seconds. Will block for at most this long. If no results
# are available by then, will return error.
# -p Don't compute anything that requires reading Git index. If this option is used,
# the following parameters will be 0: VCS_STATUS_INDEX_SIZE,
# VCS_STATUS_{NUM,HAS}_{STAGED,UNSTAGED,UNTRACKED,CONFLICTED}.
#
# On success sets VCS_STATUS_RESULT to one of the following values:
#
# norepo-sync The directory doesn't belong to a git repository.
# ok-sync The directory belongs to a git repository.
#
# If VCS_STATUS_RESULT is ok-sync, additional variables are set:
#
# VCS_STATUS_WORKDIR Git repo working directory. Not empty.
# VCS_STATUS_COMMIT Commit hash that HEAD is pointing to. Either 40 hex digits or
# empty if there is no HEAD (empty repo).
# VCS_STATUS_COMMIT_ENCODING Encoding of the HEAD's commit message. Empty value means UTF-8.
# VCS_STATUS_COMMIT_SUMMARY The first paragraph of the HEAD's commit message as one line.
# VCS_STATUS_LOCAL_BRANCH Local branch name or empty if not on a branch.
# VCS_STATUS_REMOTE_NAME The remote name, e.g. "upstream" or "origin".
# VCS_STATUS_REMOTE_BRANCH Upstream branch name. Can be empty.
# VCS_STATUS_REMOTE_URL Remote URL. Can be empty.
# VCS_STATUS_ACTION Repository state, A.K.A. action. Can be empty.
# VCS_STATUS_INDEX_SIZE The number of files in the index.
# VCS_STATUS_NUM_STAGED The number of staged changes.
# VCS_STATUS_NUM_CONFLICTED The number of conflicted changes.
# VCS_STATUS_NUM_UNSTAGED The number of unstaged changes.
# VCS_STATUS_NUM_UNTRACKED The number of untracked files.
# VCS_STATUS_HAS_STAGED 1 if there are staged changes, 0 otherwise.
# VCS_STATUS_HAS_CONFLICTED 1 if there are conflicted changes, 0 otherwise.
# VCS_STATUS_HAS_UNSTAGED 1 if there are unstaged changes, 0 if there aren't, -1 if
# unknown.
# VCS_STATUS_NUM_STAGED_NEW The number of staged new files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_NUM_STAGED_DELETED The number of staged deleted files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_NUM_UNSTAGED_DELETED The number of unstaged deleted files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_HAS_UNTRACKED 1 if there are untracked files, 0 if there aren't, -1 if
# unknown.
# VCS_STATUS_COMMITS_AHEAD Number of commits the current branch is ahead of upstream.
# Non-negative integer.
# VCS_STATUS_COMMITS_BEHIND Number of commits the current branch is behind upstream.
# Non-negative integer.
# VCS_STATUS_STASHES Number of stashes. Non-negative integer.
# VCS_STATUS_TAG The last tag (in lexicographical order) that points to the same
# commit as HEAD.
# VCS_STATUS_PUSH_REMOTE_NAME The push remote name, e.g. "upstream" or "origin".
# VCS_STATUS_PUSH_REMOTE_URL Push remote URL. Can be empty.
# VCS_STATUS_PUSH_COMMITS_AHEAD Number of commits the current branch is ahead of push remote.
# Non-negative integer.
# VCS_STATUS_PUSH_COMMITS_BEHIND Number of commits the current branch is behind push remote.
# Non-negative integer.
# VCS_STATUS_NUM_SKIP_WORKTREE The number of files in the index with skip-worktree bit set.
# Non-negative integer.
# VCS_STATUS_NUM_ASSUME_UNCHANGED The number of files in the index with assume-unchanged bit set.
# Non-negative integer.
#
# The point of reporting -1 via VCS_STATUS_HAS_* is to allow the command to skip scanning files in
# large repos. See -m flag of gitstatus_start.
#
# gitstatus_query returns an error if gitstatus_start hasn't been called in the same
# shell or the call had failed.
function gitstatus_query() {
unset OPTIND
local opt dir= timeout=() no_diff=0
while getopts "d:c:t:p" opt "$@"; do
case "$opt" in
d) dir=$OPTARG;;
t) timeout=(-t "$OPTARG");;
p) no_diff=1;;
*) return 1;;
esac
done
(( OPTIND == $# + 1 )) || { echo "usage: gitstatus_query [OPTION]..." >&2; return 1; }
[[ -n "${GITSTATUS_DAEMON_PID-}" ]] || return # not started
local req_id="$RANDOM.$RANDOM.$RANDOM.$RANDOM"
if [[ -z "${GIT_DIR:-}" ]]; then
[[ "$dir" == /* ]] || dir="$(pwd -P)/$dir" || return
elif [[ "$GIT_DIR" == /* ]]; then
dir=:"$GIT_DIR"
else
dir=:"$(pwd -P)/$GIT_DIR" || return
fi
echo -nE "$req_id"$'\x1f'"$dir"$'\x1f'"$no_diff"$'\x1e' >&$_GITSTATUS_REQ_FD || return
local -a resp
while true; do
IFS=$'\x1f' read -rd $'\x1e' -a resp -u $_GITSTATUS_RESP_FD "${timeout[@]}" || return
[[ "${resp[0]}" == "$req_id" ]] && break
done
if [[ "${resp[1]}" == 1 ]]; then
VCS_STATUS_RESULT=ok-sync
VCS_STATUS_WORKDIR="${resp[2]}"
VCS_STATUS_COMMIT="${resp[3]}"
VCS_STATUS_LOCAL_BRANCH="${resp[4]}"
VCS_STATUS_REMOTE_BRANCH="${resp[5]}"
VCS_STATUS_REMOTE_NAME="${resp[6]}"
VCS_STATUS_REMOTE_URL="${resp[7]}"
VCS_STATUS_ACTION="${resp[8]}"
VCS_STATUS_INDEX_SIZE="${resp[9]}"
VCS_STATUS_NUM_STAGED="${resp[10]}"
VCS_STATUS_NUM_UNSTAGED="${resp[11]}"
VCS_STATUS_NUM_CONFLICTED="${resp[12]}"
VCS_STATUS_NUM_UNTRACKED="${resp[13]}"
VCS_STATUS_COMMITS_AHEAD="${resp[14]}"
VCS_STATUS_COMMITS_BEHIND="${resp[15]}"
VCS_STATUS_STASHES="${resp[16]}"
VCS_STATUS_TAG="${resp[17]}"
VCS_STATUS_NUM_UNSTAGED_DELETED="${resp[18]}"
VCS_STATUS_NUM_STAGED_NEW="${resp[19]:-0}"
VCS_STATUS_NUM_STAGED_DELETED="${resp[20]:-0}"
VCS_STATUS_PUSH_REMOTE_NAME="${resp[21]:-}"
VCS_STATUS_PUSH_REMOTE_URL="${resp[22]:-}"
VCS_STATUS_PUSH_COMMITS_AHEAD="${resp[23]:-0}"
VCS_STATUS_PUSH_COMMITS_BEHIND="${resp[24]:-0}"
VCS_STATUS_NUM_SKIP_WORKTREE="${resp[25]:-0}"
VCS_STATUS_NUM_ASSUME_UNCHANGED="${resp[26]:-0}"
VCS_STATUS_COMMIT_ENCODING="${resp[27]-}"
VCS_STATUS_COMMIT_SUMMARY="${resp[28]-}"
VCS_STATUS_HAS_STAGED=$((VCS_STATUS_NUM_STAGED > 0))
if (( _GITSTATUS_DIRTY_MAX_INDEX_SIZE >= 0 &&
VCS_STATUS_INDEX_SIZE > _GITSTATUS_DIRTY_MAX_INDEX_SIZE_ )); then
VCS_STATUS_HAS_UNSTAGED=-1
VCS_STATUS_HAS_CONFLICTED=-1
VCS_STATUS_HAS_UNTRACKED=-1
else
VCS_STATUS_HAS_UNSTAGED=$((VCS_STATUS_NUM_UNSTAGED > 0))
VCS_STATUS_HAS_CONFLICTED=$((VCS_STATUS_NUM_CONFLICTED > 0))
VCS_STATUS_HAS_UNTRACKED=$((VCS_STATUS_NUM_UNTRACKED > 0))
fi
else
VCS_STATUS_RESULT=norepo-sync
unset VCS_STATUS_WORKDIR
unset VCS_STATUS_COMMIT
unset VCS_STATUS_LOCAL_BRANCH
unset VCS_STATUS_REMOTE_BRANCH
unset VCS_STATUS_REMOTE_NAME
unset VCS_STATUS_REMOTE_URL
unset VCS_STATUS_ACTION
unset VCS_STATUS_INDEX_SIZE
unset VCS_STATUS_NUM_STAGED
unset VCS_STATUS_NUM_UNSTAGED
unset VCS_STATUS_NUM_CONFLICTED
unset VCS_STATUS_NUM_UNTRACKED
unset VCS_STATUS_HAS_STAGED
unset VCS_STATUS_HAS_UNSTAGED
unset VCS_STATUS_HAS_CONFLICTED
unset VCS_STATUS_HAS_UNTRACKED
unset VCS_STATUS_COMMITS_AHEAD
unset VCS_STATUS_COMMITS_BEHIND
unset VCS_STATUS_STASHES
unset VCS_STATUS_TAG
unset VCS_STATUS_NUM_UNSTAGED_DELETED
unset VCS_STATUS_NUM_STAGED_NEW
unset VCS_STATUS_NUM_STAGED_DELETED
unset VCS_STATUS_PUSH_REMOTE_NAME
unset VCS_STATUS_PUSH_REMOTE_URL
unset VCS_STATUS_PUSH_COMMITS_AHEAD
unset VCS_STATUS_PUSH_COMMITS_BEHIND
unset VCS_STATUS_NUM_SKIP_WORKTREE
unset VCS_STATUS_NUM_ASSUME_UNCHANGED
unset VCS_STATUS_COMMIT_ENCODING
unset VCS_STATUS_COMMIT_SUMMARY
fi
}
# Usage: gitstatus_check.
#
# Returns 0 if and only if gitstatus_start has succeeded previously.
# If it returns non-zero, gitstatus_query is guaranteed to return non-zero.
function gitstatus_check() {
[[ -n "$GITSTATUS_DAEMON_PID" ]]
}
@@ -0,0 +1,908 @@
# Zsh bindings for gitstatus.
#
# ------------------------------------------------------------------
#
# Example: Start gitstatusd, send it a request, wait for response and print it.
#
# source ~/gitstatus/gitstatus.plugin.zsh
# gitstatus_start MY
# gitstatus_query -d $PWD MY
# typeset -m 'VCS_STATUS_*'
#
# Output:
#
# VCS_STATUS_ACTION=''
# VCS_STATUS_COMMIT=c000eddcff0fb38df2d0137efe24d9d2d900f209
# VCS_STATUS_COMMITS_AHEAD=0
# VCS_STATUS_COMMITS_BEHIND=0
# VCS_STATUS_COMMIT_ENCODING=''
# VCS_STATUS_COMMIT_SUMMARY='pull upstream changes from gitstatus'
# VCS_STATUS_HAS_CONFLICTED=0
# VCS_STATUS_HAS_STAGED=0
# VCS_STATUS_HAS_UNSTAGED=1
# VCS_STATUS_HAS_UNTRACKED=1
# VCS_STATUS_INDEX_SIZE=33
# VCS_STATUS_LOCAL_BRANCH=master
# VCS_STATUS_NUM_ASSUME_UNCHANGED=0
# VCS_STATUS_NUM_CONFLICTED=0
# VCS_STATUS_NUM_STAGED=0
# VCS_STATUS_NUM_UNSTAGED=1
# VCS_STATUS_NUM_SKIP_WORKTREE=0
# VCS_STATUS_NUM_STAGED_NEW=0
# VCS_STATUS_NUM_STAGED_DELETED=0
# VCS_STATUS_NUM_UNSTAGED_DELETED=0
# VCS_STATUS_NUM_UNTRACKED=1
# VCS_STATUS_PUSH_COMMITS_AHEAD=0
# VCS_STATUS_PUSH_COMMITS_BEHIND=0
# VCS_STATUS_PUSH_REMOTE_NAME=''
# VCS_STATUS_PUSH_REMOTE_URL=''
# VCS_STATUS_REMOTE_BRANCH=master
# VCS_STATUS_REMOTE_NAME=origin
# VCS_STATUS_REMOTE_URL=git@github.com:romkatv/powerlevel10k.git
# VCS_STATUS_RESULT=ok-sync
# VCS_STATUS_STASHES=0
# VCS_STATUS_TAG=''
# VCS_STATUS_WORKDIR=/home/romka/powerlevel10k
[[ -o 'interactive' ]] || 'return'
# Temporarily change options.
'builtin' 'local' '-a' '_gitstatus_opts'
[[ ! -o 'aliases' ]] || _gitstatus_opts+=('aliases')
[[ ! -o 'sh_glob' ]] || _gitstatus_opts+=('sh_glob')
[[ ! -o 'no_brace_expand' ]] || _gitstatus_opts+=('no_brace_expand')
'builtin' 'setopt' 'no_aliases' 'no_sh_glob' 'brace_expand'
autoload -Uz add-zsh-hook || return
zmodload zsh/datetime zsh/system || return
zmodload -F zsh/files b:zf_rm || return
typeset -g _gitstatus_plugin_dir"${1:-}"="${${(%):-%x}:A:h}"
# Retrieves status of a git repo from a directory under its working tree.
#
## Usage: gitstatus_query [OPTION]... NAME
#
# -d STR Directory to query. Defaults to the current directory. Has no effect if GIT_DIR
# is set.
# -c STR Callback function to call once the results are available. Called only after
# gitstatus_query returns 0 with VCS_STATUS_RESULT=tout.
# -t FLOAT Timeout in seconds. Negative value means infinity. Will block for at most this long.
# If no results are available by then: if -c isn't specified, will return 1; otherwise
# will set VCS_STATUS_RESULT=tout and return 0.
# -p Don't compute anything that requires reading Git index. If this option is used,
# the following parameters will be 0: VCS_STATUS_INDEX_SIZE,
# VCS_STATUS_{NUM,HAS}_{STAGED,UNSTAGED,UNTRACKED,CONFLICTED}.
#
# On success sets VCS_STATUS_RESULT to one of the following values:
#
# tout Timed out waiting for data; will call the user-specified callback later.
# norepo-sync The directory isn't a git repo.
# ok-sync The directory is a git repo.
#
# When the callback is called, VCS_STATUS_RESULT is set to one of the following values:
#
# norepo-async The directory isn't a git repo.
# ok-async The directory is a git repo.
#
# If VCS_STATUS_RESULT is ok-sync or ok-async, additional variables are set:
#
# VCS_STATUS_WORKDIR Git repo working directory. Not empty.
# VCS_STATUS_COMMIT Commit hash that HEAD is pointing to. Either 40 hex digits or
# empty if there is no HEAD (empty repo).
# VCS_STATUS_COMMIT_ENCODING Encoding of the HEAD's commit message. Empty value means UTF-8.
# VCS_STATUS_COMMIT_SUMMARY The first paragraph of the HEAD's commit message as one line.
# VCS_STATUS_LOCAL_BRANCH Local branch name or empty if not on a branch.
# VCS_STATUS_REMOTE_NAME The remote name, e.g. "upstream" or "origin".
# VCS_STATUS_REMOTE_BRANCH Upstream branch name. Can be empty.
# VCS_STATUS_REMOTE_URL Remote URL. Can be empty.
# VCS_STATUS_ACTION Repository state, A.K.A. action. Can be empty.
# VCS_STATUS_INDEX_SIZE The number of files in the index.
# VCS_STATUS_NUM_STAGED The number of staged changes.
# VCS_STATUS_NUM_CONFLICTED The number of conflicted changes.
# VCS_STATUS_NUM_UNSTAGED The number of unstaged changes.
# VCS_STATUS_NUM_UNTRACKED The number of untracked files.
# VCS_STATUS_HAS_STAGED 1 if there are staged changes, 0 otherwise.
# VCS_STATUS_HAS_CONFLICTED 1 if there are conflicted changes, 0 otherwise.
# VCS_STATUS_HAS_UNSTAGED 1 if there are unstaged changes, 0 if there aren't, -1 if
# unknown.
# VCS_STATUS_NUM_STAGED_NEW The number of staged new files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_NUM_STAGED_DELETED The number of staged deleted files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_NUM_UNSTAGED_DELETED The number of unstaged deleted files. Note that renamed files
# are reported as deleted plus new.
# VCS_STATUS_HAS_UNTRACKED 1 if there are untracked files, 0 if there aren't, -1 if
# unknown.
# VCS_STATUS_COMMITS_AHEAD Number of commits the current branch is ahead of upstream.
# Non-negative integer.
# VCS_STATUS_COMMITS_BEHIND Number of commits the current branch is behind upstream.
# Non-negative integer.
# VCS_STATUS_STASHES Number of stashes. Non-negative integer.
# VCS_STATUS_TAG The last tag (in lexicographical order) that points to the same
# commit as HEAD.
# VCS_STATUS_PUSH_REMOTE_NAME The push remote name, e.g. "upstream" or "origin".
# VCS_STATUS_PUSH_REMOTE_URL Push remote URL. Can be empty.
# VCS_STATUS_PUSH_COMMITS_AHEAD Number of commits the current branch is ahead of push remote.
# Non-negative integer.
# VCS_STATUS_PUSH_COMMITS_BEHIND Number of commits the current branch is behind push remote.
# Non-negative integer.
# VCS_STATUS_NUM_SKIP_WORKTREE The number of files in the index with skip-worktree bit set.
# Non-negative integer.
# VCS_STATUS_NUM_ASSUME_UNCHANGED The number of files in the index with assume-unchanged bit set.
# Non-negative integer.
#
# The point of reporting -1 via VCS_STATUS_HAS_* is to allow the command to skip scanning files in
# large repos. See -m flag of gitstatus_start.
#
# gitstatus_query returns an error if gitstatus_start hasn't been called in the same shell or
# the call had failed.
#
# !!!!! WARNING: CONCURRENT CALLS WITH THE SAME NAME ARE NOT ALLOWED !!!!!
#
# It's illegal to call gitstatus_query if the last asynchronous call with the same NAME hasn't
# completed yet. If you need to issue concurrent requests, use different NAME arguments.
function gitstatus_query"${1:-}"() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local fsuf=${${(%):-%N}#gitstatus_query}
unset VCS_STATUS_RESULT
local opt dir callback OPTARG
local -i no_diff OPTIND
local -F timeout=-1
while getopts ":d:c:t:p" opt; do
case $opt in
+p) no_diff=0;;
p) no_diff=1;;
d) dir=$OPTARG;;
c) callback=$OPTARG;;
t)
if [[ $OPTARG != (|+|-)<->(|.<->)(|[eE](|-|+)<->) ]]; then
print -ru2 -- "gitstatus_query: invalid -t argument: $OPTARG"
return 1
fi
timeout=OPTARG
;;
\?) print -ru2 -- "gitstatus_query: invalid option: $OPTARG" ; return 1;;
:) print -ru2 -- "gitstatus_query: missing required argument: $OPTARG"; return 1;;
*) print -ru2 -- "gitstatus_query: invalid option: $opt" ; return 1;;
esac
done
if (( OPTIND != ARGC )); then
print -ru2 -- "gitstatus_query: exactly one positional argument is required"
return 1
fi
local name=$*[OPTIND]
if [[ $name != [[:IDENT:]]## ]]; then
print -ru2 -- "gitstatus_query: invalid positional argument: $name"
return 1
fi
(( _GITSTATUS_STATE_$name == 2 )) || return
if [[ -z $GIT_DIR ]]; then
if [[ $dir != /* ]]; then
if [[ $PWD == /* && $PWD -ef . ]]; then
dir=$PWD/$dir
else
dir=${dir:a}
fi
fi
else
if [[ $GIT_DIR == /* ]]; then
dir=:$GIT_DIR
elif [[ $PWD == /* && $PWD -ef . ]]; then
dir=:$PWD/$GIT_DIR
else
dir=:${GIT_DIR:a}
fi
fi
if [[ $dir != (|:)/* ]]; then
typeset -g VCS_STATUS_RESULT=norepo-sync
_gitstatus_clear$fsuf
return 0
fi
local -i req_fd=${(P)${:-_GITSTATUS_REQ_FD_$name}}
local req_id=$EPOCHREALTIME
print -rnu $req_fd -- $req_id' '$callback$'\x1f'$dir$'\x1f'$no_diff$'\x1e' || return
(( ++_GITSTATUS_NUM_INFLIGHT_$name ))
if (( timeout == 0 )); then
typeset -g VCS_STATUS_RESULT=tout
_gitstatus_clear$fsuf
else
while true; do
_gitstatus_process_response$fsuf $name $timeout $req_id || return
[[ $VCS_STATUS_RESULT == *-async ]] || break
done
fi
[[ $VCS_STATUS_RESULT != tout || -n $callback ]]
}
# If the last call to gitstatus_query timed out (VCS_STATUS_RESULT=tout), wait for the callback
# to be called. Otherwise do nothing.
#
# Usage: gitstatus_process_results [OPTION]... NAME
#
# -t FLOAT Timeout in seconds. Negative value means infinity. Will block for at most this long.
#
# Returns an error only when invoked with incorrect arguments and when gitstatusd isn't running or
# broken.
#
# If a callback gets called, VCS_STATUS_* parameters are set as in gitstatus_query.
# VCS_STATUS_RESULT is either norepo-async or ok-async.
function gitstatus_process_results"${1:-}"() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local fsuf=${${(%):-%N}#gitstatus_process_results}
local opt OPTARG
local -i OPTIND
local -F timeout=-1
while getopts ":t:" opt; do
case $opt in
t)
if [[ $OPTARG != (|+|-)<->(|.<->)(|[eE](|-|+)<->) ]]; then
print -ru2 -- "gitstatus_process_results: invalid -t argument: $OPTARG"
return 1
fi
timeout=OPTARG
;;
\?) print -ru2 -- "gitstatus_process_results: invalid option: $OPTARG" ; return 1;;
:) print -ru2 -- "gitstatus_process_results: missing required argument: $OPTARG"; return 1;;
*) print -ru2 -- "gitstatus_process_results: invalid option: $opt" ; return 1;;
esac
done
if (( OPTIND != ARGC )); then
print -ru2 -- "gitstatus_process_results: exactly one positional argument is required"
return 1
fi
local name=$*[OPTIND]
if [[ $name != [[:IDENT:]]## ]]; then
print -ru2 -- "gitstatus_process_results: invalid positional argument: $name"
return 1
fi
(( _GITSTATUS_STATE_$name == 2 )) || return
while (( _GITSTATUS_NUM_INFLIGHT_$name )); do
_gitstatus_process_response$fsuf $name $timeout '' || return
[[ $VCS_STATUS_RESULT == *-async ]] || break
done
return 0
}
function _gitstatus_clear"${1:-}"() {
unset VCS_STATUS_{WORKDIR,COMMIT,LOCAL_BRANCH,REMOTE_BRANCH,REMOTE_NAME,REMOTE_URL,ACTION,INDEX_SIZE,NUM_STAGED,NUM_UNSTAGED,NUM_CONFLICTED,NUM_UNTRACKED,HAS_STAGED,HAS_UNSTAGED,HAS_CONFLICTED,HAS_UNTRACKED,COMMITS_AHEAD,COMMITS_BEHIND,STASHES,TAG,NUM_UNSTAGED_DELETED,NUM_STAGED_NEW,NUM_STAGED_DELETED,PUSH_REMOTE_NAME,PUSH_REMOTE_URL,PUSH_COMMITS_AHEAD,PUSH_COMMITS_BEHIND,NUM_SKIP_WORKTREE,NUM_ASSUME_UNCHANGED}
}
function _gitstatus_process_response"${1:-}"() {
local name=$1 timeout req_id=$3 buf
local -i resp_fd=_GITSTATUS_RESP_FD_$name
local -i dirty_max_index_size=_GITSTATUS_DIRTY_MAX_INDEX_SIZE_$name
(( $2 >= 0 )) && timeout=-t$2 && [[ -t $resp_fd ]]
sysread $timeout -i $resp_fd 'buf[$#buf+1]' || {
if (( $? == 4 )); then
if [[ -n $req_id ]]; then
typeset -g VCS_STATUS_RESULT=tout
_gitstatus_clear$fsuf
fi
return 0
else
gitstatus_stop$fsuf $name
return 1
fi
}
while [[ $buf != *$'\x1e' ]]; do
if ! sysread -i $resp_fd 'buf[$#buf+1]'; then
gitstatus_stop$fsuf $name
return 1
fi
done
local s
for s in ${(ps:\x1e:)buf}; do
local -a resp=("${(@ps:\x1f:)s}")
if (( resp[2] )); then
if [[ $resp[1] == $req_id' '* ]]; then
typeset -g VCS_STATUS_RESULT=ok-sync
else
typeset -g VCS_STATUS_RESULT=ok-async
fi
for VCS_STATUS_WORKDIR \
VCS_STATUS_COMMIT \
VCS_STATUS_LOCAL_BRANCH \
VCS_STATUS_REMOTE_BRANCH \
VCS_STATUS_REMOTE_NAME \
VCS_STATUS_REMOTE_URL \
VCS_STATUS_ACTION \
VCS_STATUS_INDEX_SIZE \
VCS_STATUS_NUM_STAGED \
VCS_STATUS_NUM_UNSTAGED \
VCS_STATUS_NUM_CONFLICTED \
VCS_STATUS_NUM_UNTRACKED \
VCS_STATUS_COMMITS_AHEAD \
VCS_STATUS_COMMITS_BEHIND \
VCS_STATUS_STASHES \
VCS_STATUS_TAG \
VCS_STATUS_NUM_UNSTAGED_DELETED \
VCS_STATUS_NUM_STAGED_NEW \
VCS_STATUS_NUM_STAGED_DELETED \
VCS_STATUS_PUSH_REMOTE_NAME \
VCS_STATUS_PUSH_REMOTE_URL \
VCS_STATUS_PUSH_COMMITS_AHEAD \
VCS_STATUS_PUSH_COMMITS_BEHIND \
VCS_STATUS_NUM_SKIP_WORKTREE \
VCS_STATUS_NUM_ASSUME_UNCHANGED \
VCS_STATUS_COMMIT_ENCODING \
VCS_STATUS_COMMIT_SUMMARY in "${(@)resp[3,29]}"; do
done
typeset -gi VCS_STATUS_{INDEX_SIZE,NUM_STAGED,NUM_UNSTAGED,NUM_CONFLICTED,NUM_UNTRACKED,COMMITS_AHEAD,COMMITS_BEHIND,STASHES,NUM_UNSTAGED_DELETED,NUM_STAGED_NEW,NUM_STAGED_DELETED,PUSH_COMMITS_AHEAD,PUSH_COMMITS_BEHIND,NUM_SKIP_WORKTREE,NUM_ASSUME_UNCHANGED}
typeset -gi VCS_STATUS_HAS_STAGED=$((VCS_STATUS_NUM_STAGED > 0))
if (( dirty_max_index_size >= 0 && VCS_STATUS_INDEX_SIZE > dirty_max_index_size )); then
typeset -gi \
VCS_STATUS_HAS_UNSTAGED=-1 \
VCS_STATUS_HAS_CONFLICTED=-1 \
VCS_STATUS_HAS_UNTRACKED=-1
else
typeset -gi \
VCS_STATUS_HAS_UNSTAGED=$((VCS_STATUS_NUM_UNSTAGED > 0)) \
VCS_STATUS_HAS_CONFLICTED=$((VCS_STATUS_NUM_CONFLICTED > 0)) \
VCS_STATUS_HAS_UNTRACKED=$((VCS_STATUS_NUM_UNTRACKED > 0))
fi
else
if [[ $resp[1] == $req_id' '* ]]; then
typeset -g VCS_STATUS_RESULT=norepo-sync
else
typeset -g VCS_STATUS_RESULT=norepo-async
fi
_gitstatus_clear$fsuf
fi
(( --_GITSTATUS_NUM_INFLIGHT_$name ))
[[ $VCS_STATUS_RESULT == *-async ]] && emulate zsh -c "${resp[1]#* }"
done
return 0
}
function _gitstatus_daemon"${1:-}"() {
local -i pipe_fd
exec 0<&- {pipe_fd}>&1 1>>$daemon_log 2>&1 || return
local pgid=$sysparams[pid]
[[ $pgid == <1-> ]] || return
builtin cd -q / || return
{
{
trap '' PIPE
local uname_sm
uname_sm="${${(L)$(command uname -sm)}//ı/i}" || return
[[ $uname_sm == [^' ']##' '[^' ']## ]] || return
local uname_s=${uname_sm% *}
local uname_m=${uname_sm#* }
if [[ $GITSTATUS_NUM_THREADS == <1-> ]]; then
args+=(-t $GITSTATUS_NUM_THREADS)
else
local cpus
if (( ! $+commands[sysctl] )) || [[ $uname_s == linux ]] ||
! cpus="$(command sysctl -n hw.ncpu)"; then
if (( ! $+commands[getconf] )) || ! cpus="$(command getconf _NPROCESSORS_ONLN)"; then
cpus=8
fi
fi
args+=(-t $((cpus > 16 ? 32 : cpus > 0 ? 2 * cpus : 16)))
fi
command mkfifo -- $file_prefix.fifo || return
print -rnu $pipe_fd -- ${(l:20:)pgid} || return
exec <$file_prefix.fifo || return
zf_rm -- $file_prefix.fifo || return
local _gitstatus_zsh_daemon _gitstatus_zsh_version _gitstatus_zsh_downloaded
function _gitstatus_set_daemon$fsuf() {
_gitstatus_zsh_daemon="$1"
_gitstatus_zsh_version="$2"
_gitstatus_zsh_downloaded="$3"
}
local gitstatus_plugin_dir_var=_gitstatus_plugin_dir$fsuf
local gitstatus_plugin_dir=${(P)gitstatus_plugin_dir_var}
builtin set -- -d $gitstatus_plugin_dir -s $uname_s -m $uname_m \
-p "printf '\\001' >&$pipe_fd" -e $pipe_fd -- _gitstatus_set_daemon$fsuf
[[ ${GITSTATUS_AUTO_INSTALL:-1} == (|-|+)<1-> ]] || builtin set -- -n "$@"
builtin source $gitstatus_plugin_dir/install || return
[[ -n $_gitstatus_zsh_daemon ]] || return
[[ -n $_gitstatus_zsh_version ]] || return
[[ $_gitstatus_zsh_downloaded == [01] ]] || return
if (( UID == EUID )); then
local home=~
else
local user
user="$(command id -un)" || return
local home=${userdirs[$user]}
[[ -n $home ]] || return
fi
if [[ -x $_gitstatus_zsh_daemon ]]; then
HOME=$home $_gitstatus_zsh_daemon -G $_gitstatus_zsh_version "${(@)args}" >&$pipe_fd
local -i ret=$?
[[ $ret == (0|129|130|131|137|141|143|159) ]] && return ret
fi
(( ! _gitstatus_zsh_downloaded )) || return
[[ ${GITSTATUS_AUTO_INSTALL:-1} == (|-|+)<1-> ]] || return
[[ $_gitstatus_zsh_daemon == \
${GITSTATUS_CACHE_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/gitstatus}/* ]] || return
builtin set -- -f "$@"
_gitstatus_zsh_daemon=
_gitstatus_zsh_version=
_gitstatus_zsh_downloaded=
builtin source $gitstatus_plugin_dir/install || return
[[ -n $_gitstatus_zsh_daemon ]] || return
[[ -n $_gitstatus_zsh_version ]] || return
[[ $_gitstatus_zsh_downloaded == 1 ]] || return
HOME=$home $_gitstatus_zsh_daemon -G $_gitstatus_zsh_version "${(@)args}" >&$pipe_fd
} always {
local -i ret=$?
zf_rm -f -- $file_prefix.lock $file_prefix.fifo
kill -- -$pgid
}
} &!
(( lock_fd == -1 )) && return
{
if zsystem flock -- $file_prefix.lock && command sleep 5 && [[ -e $file_prefix.lock ]]; then
zf_rm -f -- $file_prefix.lock $file_prefix.fifo
kill -- -$pgid
fi
} &!
}
# Starts gitstatusd in the background. Does nothing and succeeds if gitstatusd is already running.
#
# Usage: gitstatus_start [OPTION]... NAME
#
# -t FLOAT Fail the self-check on initialization if not getting a response from gitstatusd for
# this this many seconds. Defaults to 5.
#
# -s INT Report at most this many staged changes; negative value means infinity.
# Defaults to 1.
#
# -u INT Report at most this many unstaged changes; negative value means infinity.
# Defaults to 1.
#
# -c INT Report at most this many conflicted changes; negative value means infinity.
# Defaults to 1.
#
# -d INT Report at most this many untracked files; negative value means infinity.
# Defaults to 1.
#
# -m INT Report -1 unstaged, untracked and conflicted if there are more than this many
# files in the index. Negative value means infinity. Defaults to -1.
#
# -e Count files within untracked directories like `git status --untracked-files`.
#
# -U Unless this option is specified, report zero untracked files for repositories
# with status.showUntrackedFiles = false.
#
# -W Unless this option is specified, report zero untracked files for repositories
# with bash.showUntrackedFiles = false.
#
# -D Unless this option is specified, report zero staged, unstaged and conflicted
# changes for repositories with bash.showDirtyState = false.
function gitstatus_start"${1:-}"() {
emulate -L zsh -o no_aliases -o no_bg_nice -o extended_glob -o typeset_silent || return
print -rnu2 || return
local fsuf=${${(%):-%N}#gitstatus_start}
local opt OPTARG
local -i OPTIND
local -F timeout=5
local -i async=0
local -a args=()
local -i dirty_max_index_size=-1
while getopts ":t:s:u:c:d:m:eaUWD" opt; do
case $opt in
a) async=1;;
+a) async=0;;
t)
if [[ $OPTARG != (|+)<->(|.<->)(|[eE](|-|+)<->) ]] || (( ${timeout::=OPTARG} <= 0 )); then
print -ru2 -- "gitstatus_start: invalid -t argument: $OPTARG"
return 1
fi
;;
s|u|c|d|m)
if [[ $OPTARG != (|-|+)<-> ]]; then
print -ru2 -- "gitstatus_start: invalid -$opt argument: $OPTARG"
return 1
fi
args+=(-$opt $OPTARG)
[[ $opt == m ]] && dirty_max_index_size=OPTARG
;;
e|U|W|D) args+=-$opt;;
+(e|U|W|D)) args=(${(@)args:#-$opt});;
\?) print -ru2 -- "gitstatus_start: invalid option: $OPTARG" ; return 1;;
:) print -ru2 -- "gitstatus_start: missing required argument: $OPTARG"; return 1;;
*) print -ru2 -- "gitstatus_start: invalid option: $opt" ; return 1;;
esac
done
if (( OPTIND != ARGC )); then
print -ru2 -- "gitstatus_start: exactly one positional argument is required"
return 1
fi
local name=$*[OPTIND]
if [[ $name != [[:IDENT:]]## ]]; then
print -ru2 -- "gitstatus_start: invalid positional argument: $name"
return 1
fi
local -i lock_fd resp_fd stderr_fd
local file_prefix xtrace=/dev/null daemon_log=/dev/null culprit
{
if (( _GITSTATUS_STATE_$name )); then
(( async )) && return
(( _GITSTATUS_STATE_$name == 2 )) && return
lock_fd=_GITSTATUS_LOCK_FD_$name
resp_fd=_GITSTATUS_RESP_FD_$name
xtrace=${(P)${:-GITSTATUS_XTRACE_$name}}
daemon_log=${(P)${:-GITSTATUS_DAEMON_LOG_$name}}
file_prefix=${(P)${:-_GITSTATUS_FILE_PREFIX_$name}}
else
typeset -gi _GITSTATUS_START_COUNTER
local log_level=$GITSTATUS_LOG_LEVEL
if [[ -n "$TMPDIR" && ( ( -d "$TMPDIR" && -w "$TMPDIR" ) || ! ( -d /tmp && -w /tmp ) ) ]]; then
local tmpdir=$TMPDIR
else
local tmpdir=/tmp
fi
local file_prefix=${tmpdir:A}/gitstatus.$name.$EUID
file_prefix+=.$sysparams[pid].$EPOCHSECONDS.$((++_GITSTATUS_START_COUNTER))
(( GITSTATUS_ENABLE_LOGGING )) && : ${log_level:=INFO}
if [[ -n $log_level ]]; then
xtrace=$file_prefix.xtrace.log
daemon_log=$file_prefix.daemon.log
fi
args+=(-v ${log_level:-FATAL})
typeset -g GITSTATUS_XTRACE_$name=$xtrace
typeset -g GITSTATUS_DAEMON_LOG_$name=$daemon_log
typeset -g _GITSTATUS_FILE_PREFIX_$name=$file_prefix
typeset -gi _GITSTATUS_CLIENT_PID_$name="sysparams[pid]"
typeset -gi _GITSTATUS_DIRTY_MAX_INDEX_SIZE_$name=dirty_max_index_size
fi
() {
if [[ $xtrace != /dev/null && -o no_xtrace ]]; then
exec {stderr_fd}>&2 || return
exec 2>>$xtrace || return
setopt xtrace
fi
setopt monitor || return
if (( ! _GITSTATUS_STATE_$name )); then
if [[ -r /proc/version && "$(</proc/version)" == *Microsoft* ]]; then
lock_fd=-1
else
print -rn >$file_prefix.lock || return
zsystem flock -f lock_fd $file_prefix.lock || return
[[ $lock_fd == <1-> ]] || return
fi
typeset -gi _GITSTATUS_LOCK_FD_$name=lock_fd
if [[ $OSTYPE == cygwin* && -d /proc/self/fd ]]; then
# Work around bugs in Cygwin 32-bit.
#
# This hangs:
#
# emulate -L zsh
# () { exec {fd}< $1 } <(:)
# =true # hangs here
#
# This hangs:
#
# sysopen -r -u fd <(:)
local -i fd
exec {fd}< <(_gitstatus_daemon$fsuf) || return
{
[[ -r /proc/self/fd/$fd ]] || return
sysopen -r -o cloexec -u resp_fd /proc/self/fd/$fd || return
} always {
exec {fd} >&- || return
}
else
sysopen -r -o cloexec -u resp_fd <(_gitstatus_daemon$fsuf) || return
fi
typeset -gi GITSTATUS_DAEMON_PID_$name="${sysparams[procsubstpid]:--1}"
[[ $resp_fd == <1-> ]] || return
typeset -gi _GITSTATUS_RESP_FD_$name=resp_fd
typeset -gi _GITSTATUS_STATE_$name=1
fi
if (( ! async )); then
(( _GITSTATUS_CLIENT_PID_$name == sysparams[pid] )) || return
local pgid
while (( $#pgid < 20 )); do
[[ -t $resp_fd ]]
sysread -s $((20 - $#pgid)) -t $timeout -i $resp_fd 'pgid[$#pgid+1]' || return
done
[[ $pgid == ' '#<1-> ]] || return
typeset -gi GITSTATUS_DAEMON_PID_$name=pgid
sysopen -w -o cloexec -u req_fd -- $file_prefix.fifo || return
[[ $req_fd == <1-> ]] || return
typeset -gi _GITSTATUS_REQ_FD_$name=req_fd
print -nru $req_fd -- $'}hello\x1f\x1e' || return
local expected=$'}hello\x1f0\x1e' actual
if (( $+functions[p10k] )) && [[ ! -t 1 && ! -t 0 ]]; then
local -F deadline='EPOCHREALTIME + 4'
else
local -F deadline='1'
fi
while true; do
[[ -t $resp_fd ]]
sysread -s 1 -t $timeout -i $resp_fd actual || return
[[ $expected == $actual* ]] && break
if [[ $actual != $'\1' ]]; then
[[ -t $resp_fd ]]
while sysread -t $timeout -i $resp_fd 'actual[$#actual+1]'; do
[[ -t $resp_fd ]]
done
culprit=$actual
return 1
fi
(( EPOCHREALTIME < deadline )) && continue
if (( deadline > 0 )); then
deadline=0
if (( stderr_fd )); then
unsetopt xtrace
exec 2>&$stderr_fd {stderr_fd}>&-
stderr_fd=0
fi
if (( $+functions[p10k] )); then
p10k clear-instant-prompt || return
fi
if [[ $name == POWERLEVEL9K ]]; then
local label=powerlevel10k
else
local label=gitstatus
fi
if [[ -t 2 ]]; then
local spinner=($'\b%3F-%f' $'\b%3F\\%f' $'\b%3F|%f' $'\b%3F/%f')
print -Prnu2 -- "[%3F$label%f] fetching %2Fgitstatusd%f .. "
else
local spinner=('.')
print -rnu2 -- "[$label] fetching gitstatusd .."
fi
fi
print -Prnu2 -- $spinner[1]
spinner=($spinner[2,-1] $spinner[1])
done
if (( deadline == 0 )); then
if [[ -t 2 ]]; then
print -Pru2 -- $'\b[%2Fok%f]'
else
print -ru2 -- ' [ok]'
fi
if [[ $xtrace != /dev/null && -o no_xtrace ]]; then
exec {stderr_fd}>&2 || return
exec 2>>$xtrace || return
setopt xtrace
fi
fi
while (( $#actual < $#expected )); do
[[ -t $resp_fd ]]
sysread -s $(($#expected - $#actual)) -t $timeout -i $resp_fd 'actual[$#actual+1]' || return
done
[[ $actual == $expected ]] || return
function _gitstatus_process_response_$name-$fsuf() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local pair=${${(%):-%N}#_gitstatus_process_response_}
local name=${pair%%-*}
local fsuf=${pair#*-}
[[ $name == POWERLEVEL9K && $fsuf == _p9k_ ]] && eval $__p9k_intro_base
if (( ARGC == 1 )); then
_gitstatus_process_response$fsuf $name 0 ''
else
gitstatus_stop$fsuf $name
fi
}
if ! zle -F $resp_fd _gitstatus_process_response_$name-$fsuf; then
unfunction _gitstatus_process_response_$name-$fsuf
return 1
fi
function _gitstatus_cleanup_$name-$fsuf() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local pair=${${(%):-%N}#_gitstatus_cleanup_}
local name=${pair%%-*}
local fsuf=${pair#*-}
(( _GITSTATUS_CLIENT_PID_$name == sysparams[pid] )) || return
gitstatus_stop$fsuf $name
}
if ! add-zsh-hook zshexit _gitstatus_cleanup_$name-$fsuf; then
unfunction _gitstatus_cleanup_$name-$fsuf
return 1
fi
if (( lock_fd != -1 )); then
zf_rm -- $file_prefix.lock || return
zsystem flock -u $lock_fd || return
fi
unset _GITSTATUS_LOCK_FD_$name
typeset -gi _GITSTATUS_STATE_$name=2
fi
}
} always {
local -i err=$?
(( stderr_fd )) && exec 2>&$stderr_fd {stderr_fd}>&-
(( err == 0 )) && return
gitstatus_stop$fsuf $name
setopt prompt_percent no_prompt_subst no_prompt_bang
(( $+functions[p10k] )) && p10k clear-instant-prompt
print -ru2 -- ''
print -Pru2 -- '[%F{red}ERROR%f]: gitstatus failed to initialize.'
print -ru2 -- ''
if [[ -n $culprit ]]; then
print -ru2 -- $culprit
return err
fi
if [[ -s $xtrace ]]; then
print -ru2 -- ''
print -Pru2 -- " Zsh log (%U${xtrace//\%/%%}%u):"
print -Pru2 -- '%F{yellow}'
print -lru2 -- "${(@)${(@f)$(<$xtrace)}/#/ }"
print -Pnru2 -- '%f'
fi
if [[ -s $daemon_log ]]; then
print -ru2 -- ''
print -Pru2 -- " Daemon log (%U${daemon_log//\%/%%}%u):"
print -Pru2 -- '%F{yellow}'
print -lru2 -- "${(@)${(@f)$(<$daemon_log)}/#/ }"
print -Pnru2 -- '%f'
fi
if [[ $GITSTATUS_LOG_LEVEL == DEBUG ]]; then
print -ru2 -- ''
print -ru2 -- ' System information:'
print -Pru2 -- '%F{yellow}'
print -ru2 -- " zsh: $ZSH_VERSION"
print -ru2 -- " uname -a: $(command uname -a)"
print -Pru2 -- '%f'
print -ru2 -- ' If you need help, open an issue and attach this whole error message to it:'
print -ru2 -- ''
print -Pru2 -- ' %Uhttps://github.com/romkatv/gitstatus/issues/new%u'
else
print -ru2 -- ''
local home=~
local zshrc=${${${(q)${ZDOTDIR:-~}}/#${(q)home}/'~'}//\%/%%}/.zshrc
print -Pru2 -- " Add the following parameter to %U$zshrc%u for extra diagnostics on error:"
print -ru2 -- ''
print -Pru2 -- ' %BGITSTATUS_LOG_LEVEL=DEBUG%b'
print -ru2 -- ''
print -ru2 -- ' Restart Zsh to retry gitstatus initialization:'
print -ru2 -- ''
print -Pru2 -- ' %F{green}%Uexec%u zsh%f'
fi
}
}
# Stops gitstatusd if it's running.
#
# Usage: gitstatus_stop NAME.
function gitstatus_stop"${1:-}"() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local fsuf=${${(%):-%N}#gitstatus_stop}
if (( ARGC != 1 )); then
print -ru2 -- "gitstatus_stop: exactly one positional argument is required"
return 1
fi
local name=$1
if [[ $name != [[:IDENT:]]## ]]; then
print -ru2 -- "gitstatus_stop: invalid positional argument: $name"
return 1
fi
local state_var=_GITSTATUS_STATE_$name
local req_fd_var=_GITSTATUS_REQ_FD_$name
local resp_fd_var=_GITSTATUS_RESP_FD_$name
local lock_fd_var=_GITSTATUS_LOCK_FD_$name
local client_pid_var=_GITSTATUS_CLIENT_PID_$name
local daemon_pid_var=GITSTATUS_DAEMON_PID_$name
local inflight_var=_GITSTATUS_NUM_INFLIGHT_$name
local file_prefix_var=_GITSTATUS_FILE_PREFIX_$name
local dirty_max_index_size_var=_GITSTATUS_DIRTY_MAX_INDEX_SIZE_$name
local req_fd=${(P)req_fd_var}
local resp_fd=${(P)resp_fd_var}
local lock_fd=${(P)lock_fd_var}
local daemon_pid=${(P)daemon_pid_var}
local file_prefix=${(P)file_prefix_var}
local cleanup=_gitstatus_cleanup_$name-$fsuf
local process=_gitstatus_process_response_$name-$fsuf
if (( $+functions[$cleanup] )); then
add-zsh-hook -d zshexit $cleanup
unfunction -- $cleanup
fi
if (( $+functions[$process] )); then
[[ -n $resp_fd ]] && zle -F $resp_fd
unfunction -- $process
fi
[[ $daemon_pid == <1-> ]] && kill -- -$daemon_pid 2>/dev/null
[[ $file_prefix == /* ]] && zf_rm -f -- $file_prefix.lock $file_prefix.fifo
[[ $lock_fd == <1-> ]] && zsystem flock -u $lock_fd
[[ $req_fd == <1-> ]] && exec {req_fd}>&-
[[ $resp_fd == <1-> ]] && exec {resp_fd}>&-
unset $state_var $req_fd_var $lock_fd_var $resp_fd_var $client_pid_var $daemon_pid_var
unset $inflight_var $file_prefix_var $dirty_max_index_size_var
unset VCS_STATUS_RESULT
_gitstatus_clear$fsuf
}
# Usage: gitstatus_check NAME.
#
# Returns 0 if and only if `gitstatus_start NAME` has succeeded previously.
# If it returns non-zero, gitstatus_query NAME is guaranteed to return non-zero.
function gitstatus_check"${1:-}"() {
emulate -L zsh -o no_aliases -o extended_glob -o typeset_silent
local fsuf=${${(%):-%N}#gitstatus_check}
if (( ARGC != 1 )); then
print -ru2 -- "gitstatus_check: exactly one positional argument is required"
return 1
fi
local name=$1
if [[ $name != [[:IDENT:]]## ]]; then
print -ru2 -- "gitstatus_check: invalid positional argument: $name"
return 1
fi
(( _GITSTATUS_STATE_$name == 2 ))
}
(( ${#_gitstatus_opts} )) && setopt ${_gitstatus_opts[@]}
'builtin' 'unset' '_gitstatus_opts'
@@ -0,0 +1,111 @@
# Simple Bash prompt with Git status.
# Source gitstatus.plugin.sh from $GITSTATUS_DIR or from the same directory
# in which the current script resides if the variable isn't set.
if [[ -n "${GITSTATUS_DIR-}" ]]; then
source "$GITSTATUS_DIR" || return
elif [[ "${BASH_SOURCE[0]}" == */* ]]; then
source "${BASH_SOURCE[0]%/*}/gitstatus.plugin.sh" || return
else
source gitstatus.plugin.sh || return
fi
# Sets GITSTATUS_PROMPT to reflect the state of the current git repository.
# The value is empty if not in a git repository. Forwards all arguments to
# gitstatus_query.
#
# Example value of GITSTATUS_PROMPT: master ⇣42⇡42 ⇠42⇢42 *42 merge ~42 +42 !42 ?42
#
# master current branch
# ⇣42 local branch is 42 commits behind the remote
# ⇡42 local branch is 42 commits ahead of the remote
# ⇠42 local branch is 42 commits behind the push remote
# ⇢42 local branch is 42 commits ahead of the push remote
# *42 42 stashes
# merge merge in progress
# ~42 42 merge conflicts
# +42 42 staged changes
# !42 42 unstaged changes
# ?42 42 untracked files
function gitstatus_prompt_update() {
GITSTATUS_PROMPT=""
gitstatus_query "$@" || return 1 # error
[[ "$VCS_STATUS_RESULT" == ok-sync ]] || return 0 # not a git repo
local reset=$'\001\e[0m\002' # no color
local clean=$'\001\e[38;5;076m\002' # green foreground
local untracked=$'\001\e[38;5;014m\002' # teal foreground
local modified=$'\001\e[38;5;011m\002' # yellow foreground
local conflicted=$'\001\e[38;5;196m\002' # red foreground
local p
local where # branch name, tag or commit
if [[ -n "$VCS_STATUS_LOCAL_BRANCH" ]]; then
where="$VCS_STATUS_LOCAL_BRANCH"
elif [[ -n "$VCS_STATUS_TAG" ]]; then
p+="${reset}#"
where="$VCS_STATUS_TAG"
else
p+="${reset}@"
where="${VCS_STATUS_COMMIT:0:8}"
fi
(( ${#where} > 32 )) && where="${where:0:12}…${where: -12}" # truncate long branch names and tags
p+="${clean}${where}"
# ⇣42 if behind the remote.
(( VCS_STATUS_COMMITS_BEHIND )) && p+=" ${clean}⇣${VCS_STATUS_COMMITS_BEHIND}"
# ⇡42 if ahead of the remote; no leading space if also behind the remote: ⇣42⇡42.
(( VCS_STATUS_COMMITS_AHEAD && !VCS_STATUS_COMMITS_BEHIND )) && p+=" "
(( VCS_STATUS_COMMITS_AHEAD )) && p+="${clean}⇡${VCS_STATUS_COMMITS_AHEAD}"
# ⇠42 if behind the push remote.
(( VCS_STATUS_PUSH_COMMITS_BEHIND )) && p+=" ${clean}⇠${VCS_STATUS_PUSH_COMMITS_BEHIND}"
(( VCS_STATUS_PUSH_COMMITS_AHEAD && !VCS_STATUS_PUSH_COMMITS_BEHIND )) && p+=" "
# ⇢42 if ahead of the push remote; no leading space if also behind: ⇠42⇢42.
(( VCS_STATUS_PUSH_COMMITS_AHEAD )) && p+="${clean}⇢${VCS_STATUS_PUSH_COMMITS_AHEAD}"
# *42 if have stashes.
(( VCS_STATUS_STASHES )) && p+=" ${clean}*${VCS_STATUS_STASHES}"
# 'merge' if the repo is in an unusual state.
[[ -n "$VCS_STATUS_ACTION" ]] && p+=" ${conflicted}${VCS_STATUS_ACTION}"
# ~42 if have merge conflicts.
(( VCS_STATUS_NUM_CONFLICTED )) && p+=" ${conflicted}~${VCS_STATUS_NUM_CONFLICTED}"
# +42 if have staged changes.
(( VCS_STATUS_NUM_STAGED )) && p+=" ${modified}+${VCS_STATUS_NUM_STAGED}"
# !42 if have unstaged changes.
(( VCS_STATUS_NUM_UNSTAGED )) && p+=" ${modified}!${VCS_STATUS_NUM_UNSTAGED}"
# ?42 if have untracked files. It's really a question mark, your font isn't broken.
(( VCS_STATUS_NUM_UNTRACKED )) && p+=" ${untracked}?${VCS_STATUS_NUM_UNTRACKED}"
GITSTATUS_PROMPT="${p}${reset}"
}
# Start gitstatusd in the background.
gitstatus_stop && gitstatus_start -s -1 -u -1 -c -1 -d -1
# On every prompt, fetch git status and set GITSTATUS_PROMPT.
if [[ -z "${PROMPT_COMMAND[*]}" ]]; then
PROMPT_COMMAND=gitstatus_prompt_update
elif [[ ! "${PROMPT_COMMAND[*]}" =~ [[:space:]\;]?gitstatus_prompt_update[[:space:]\;]? ]]; then
# Note: If PROMPT_COMMAND is an array, this will modify its first element.
PROMPT_COMMAND=$'gitstatus_prompt_update\n'"$PROMPT_COMMAND"
fi
# Retain 3 trailing components of the current directory.
PROMPT_DIRTRIM=3
# Enable promptvars so that ${GITSTATUS_PROMPT} in PS1 is expanded.
shopt -s promptvars
# Customize prompt. Put $GITSTATUS_PROMPT in it reflect git status.
#
# Example:
#
# user@host ~/projects/skynet master ⇡42
# $ █
PS1='\[\033[01;32m\]\u@\h\[\033[00m\] ' # green user@host
PS1+='\[\033[01;34m\]\w\[\033[00m\]' # blue current working directory
PS1+='${GITSTATUS_PROMPT:+ $GITSTATUS_PROMPT}' # git status (requires promptvars option)
PS1+='\n\[\033[01;$((31+!$?))m\]\$\[\033[00m\] ' # green/red (success/error) $/# (normal/root)
PS1+='\[\e]0;\u@\h: \w\a\]' # terminal title: user@host: dir
@@ -0,0 +1,111 @@
# Simple Zsh prompt with Git status.
# Source gitstatus.plugin.zsh from $GITSTATUS_DIR or from the same directory
# in which the current script resides if the variable isn't set.
source "${GITSTATUS_DIR:-${${(%):-%x}:h}}/gitstatus.plugin.zsh" || return
# Sets GITSTATUS_PROMPT to reflect the state of the current git repository. Empty if not
# in a git repository. In addition, sets GITSTATUS_PROMPT_LEN to the number of columns
# $GITSTATUS_PROMPT will occupy when printed.
#
# Example:
#
# GITSTATUS_PROMPT='master ⇣42⇡42 ⇠42⇢42 *42 merge ~42 +42 !42 ?42'
# GITSTATUS_PROMPT_LEN=39
#
# master current branch
# ⇣42 local branch is 42 commits behind the remote
# ⇡42 local branch is 42 commits ahead of the remote
# ⇠42 local branch is 42 commits behind the push remote
# ⇢42 local branch is 42 commits ahead of the push remote
# *42 42 stashes
# merge merge in progress
# ~42 42 merge conflicts
# +42 42 staged changes
# !42 42 unstaged changes
# ?42 42 untracked files
function gitstatus_prompt_update() {
emulate -L zsh
typeset -g GITSTATUS_PROMPT=''
typeset -gi GITSTATUS_PROMPT_LEN=0
# Call gitstatus_query synchronously. Note that gitstatus_query can also be called
# asynchronously; see documentation in gitstatus.plugin.zsh.
gitstatus_query 'MY' || return 1 # error
[[ $VCS_STATUS_RESULT == 'ok-sync' ]] || return 0 # not a git repo
local clean='%76F' # green foreground
local modified='%178F' # yellow foreground
local untracked='%39F' # blue foreground
local conflicted='%196F' # red foreground
local p
local where # branch name, tag or commit
if [[ -n $VCS_STATUS_LOCAL_BRANCH ]]; then
where=$VCS_STATUS_LOCAL_BRANCH
elif [[ -n $VCS_STATUS_TAG ]]; then
p+='%f#'
where=$VCS_STATUS_TAG
else
p+='%f@'
where=${VCS_STATUS_COMMIT[1,8]}
fi
(( $#where > 32 )) && where[13,-13]="…" # truncate long branch names and tags
p+="${clean}${where//\%/%%}" # escape %
# ⇣42 if behind the remote.
(( VCS_STATUS_COMMITS_BEHIND )) && p+=" ${clean}⇣${VCS_STATUS_COMMITS_BEHIND}"
# ⇡42 if ahead of the remote; no leading space if also behind the remote: ⇣42⇡42.
(( VCS_STATUS_COMMITS_AHEAD && !VCS_STATUS_COMMITS_BEHIND )) && p+=" "
(( VCS_STATUS_COMMITS_AHEAD )) && p+="${clean}⇡${VCS_STATUS_COMMITS_AHEAD}"
# ⇠42 if behind the push remote.
(( VCS_STATUS_PUSH_COMMITS_BEHIND )) && p+=" ${clean}⇠${VCS_STATUS_PUSH_COMMITS_BEHIND}"
(( VCS_STATUS_PUSH_COMMITS_AHEAD && !VCS_STATUS_PUSH_COMMITS_BEHIND )) && p+=" "
# ⇢42 if ahead of the push remote; no leading space if also behind: ⇠42⇢42.
(( VCS_STATUS_PUSH_COMMITS_AHEAD )) && p+="${clean}⇢${VCS_STATUS_PUSH_COMMITS_AHEAD}"
# *42 if have stashes.
(( VCS_STATUS_STASHES )) && p+=" ${clean}*${VCS_STATUS_STASHES}"
# 'merge' if the repo is in an unusual state.
[[ -n $VCS_STATUS_ACTION ]] && p+=" ${conflicted}${VCS_STATUS_ACTION}"
# ~42 if have merge conflicts.
(( VCS_STATUS_NUM_CONFLICTED )) && p+=" ${conflicted}~${VCS_STATUS_NUM_CONFLICTED}"
# +42 if have staged changes.
(( VCS_STATUS_NUM_STAGED )) && p+=" ${modified}+${VCS_STATUS_NUM_STAGED}"
# !42 if have unstaged changes.
(( VCS_STATUS_NUM_UNSTAGED )) && p+=" ${modified}!${VCS_STATUS_NUM_UNSTAGED}"
# ?42 if have untracked files. It's really a question mark, your font isn't broken.
(( VCS_STATUS_NUM_UNTRACKED )) && p+=" ${untracked}?${VCS_STATUS_NUM_UNTRACKED}"
GITSTATUS_PROMPT="${p}%f"
# The length of GITSTATUS_PROMPT after removing %f and %F.
GITSTATUS_PROMPT_LEN="${(m)#${${GITSTATUS_PROMPT//\%\%/x}//\%(f|<->F)}}"
}
# Start gitstatusd instance with name "MY". The same name is passed to
# gitstatus_query in gitstatus_prompt_update. The flags with -1 as values
# enable staged, unstaged, conflicted and untracked counters.
gitstatus_stop 'MY' && gitstatus_start -s -1 -u -1 -c -1 -d -1 'MY'
# On every prompt, fetch git status and set GITSTATUS_PROMPT.
autoload -Uz add-zsh-hook
add-zsh-hook precmd gitstatus_prompt_update
# Enable/disable the right prompt options.
setopt no_prompt_bang prompt_percent prompt_subst
# Customize prompt. Put $GITSTATUS_PROMPT in it to reflect git status.
#
# Example:
#
# user@host ~/projects/skynet master ⇡42
# % █
#
# The current directory gets truncated from the left if the whole prompt doesn't fit on the line.
PROMPT='%70F%n@%m%f ' # green user@host
PROMPT+='%39F%$((-GITSTATUS_PROMPT_LEN-1))<…<%~%<<%f' # blue current working directory
PROMPT+='${GITSTATUS_PROMPT:+ $GITSTATUS_PROMPT}' # git status
PROMPT+=$'\n' # new line
PROMPT+='%F{%(?.76.196)}%#%f ' # %/# (normal/root); green/red (ok/error)
+476
View File
@@ -0,0 +1,476 @@
#!/bin/sh
#
# This script does not have a stable API.
_gitstatus_install_daemon_found() {
local installed="$1"
shift
[ $# = 0 ] || "$@" "$daemon" "$version" "$installed"
}
_gitstatus_install_main() {
if [ -n "${ZSH_VERSION:-}" ]; then
emulate -L sh -o no_unset
else
set -u
fi
local argv1="$1"
shift
local no_check= no_install= uname_s= uname_m= gitstatus_dir= dl_status= e=
local opt= OPTARG= OPTIND=1
while getopts ':s:m:d:p:e:fnh' opt "$@"; do
case "$opt" in
h)
command cat <<\END
Usage: install [-s KERNEL] [-m ARCH] [-d DIR] [-p CMD] [-e ERRFD] [-f|-n] [-- CMD [ARG]...]
If positional arguments are specified, call this on success:
CMD [ARG]... DAEMON VERSION INSTALLED
DAEMON is path to gitstatusd. VERSION is a glob pattern for the
version this daemon should support; it's supposed to be passed as
-G to gitstatusd. INSTALLED is 1 if gitstatusd has just been
downloaded and 0 otherwise.
Options:
-s KERNEL use this instead of lowercase `uname -s`
-m ARCH use this instead of lowercase `uname -m`
-d DIR use this instead of `dirname "$0"`
-p CMD eval this every second while downloading gitstatusd
-e ERRFD write error messages to this file descriptor
-f download gitstatusd even if there is one locally
-n do not download gitstatusd (fail instead)
END
return
;;
n)
if [ -n "$no_install" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
no_install=1
;;
f)
if [ -n "$no_check" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
no_check=1
;;
d)
if [ -n "$gitstatus_dir" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
return 1
fi
gitstatus_dir="$OPTARG"
;;
p)
if [ -n "$dl_status" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
return 1
fi
dl_status="$OPTARG"
;;
e)
if [ -n "$e" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
return 1
fi
e="$OPTARG"
;;
m)
if [ -n "$uname_m" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
return 1
fi
uname_m="$OPTARG"
;;
s)
if [ -n "$uname_s" ]; then
>&2 echo "[gitstatus] error: duplicate option: -$opt"
return 1
fi
if [ -z "$OPTARG" ]; then
>&2 echo "[error] incorrect value of -$opt: $OPTARG"
return 1
fi
uname_s="$OPTARG"
;;
\?) >&2 echo "[gitstatus] error: invalid option: -$OPTARG" ; return 1;;
:) >&2 echo "[gitstatus] error: missing required argument: -$OPTARG"; return 1;;
*) >&2 echo "[gitstatus] internal error: unhandled option: -$opt" ; return 1;;
esac
done
shift "$((OPTIND - 1))"
: "${e:=2}"
: "${gitstatus_dir:=$argv1}"
if [ -n "$no_check" -a -n "$no_install" ]; then
>&2 echo "[gitstatus] error: incompatible options: -f, -n"
return 1
fi
if [ -z "$uname_s" ]; then
uname_s="$(command uname -s)" || return
uname_s="$(printf '%s' "$uname_s" | command tr '[A-Z]' '[a-z]')" || return
fi
if [ -z "$uname_m" ]; then
uname_m="$(command uname -m)" || return
uname_m="$(printf '%s' "$uname_m" | command tr '[A-Z]' '[a-z]')" || return
fi
local daemon="${GITSTATUS_DAEMON:-}"
local cache_dir="${GITSTATUS_CACHE_DIR:-${XDG_CACHE_HOME:-$HOME/.cache}/gitstatus}"
if [ -z "$no_check" ]; then
if [ -n "${daemon##/*}" ]; then
>&2 echo "[gitstatus] error: GITSTATUS_DAEMON is not absolute path: $daemon"
return 1
fi
if [ -z "$daemon" -a -e "$gitstatus_dir"/usrbin/gitstatusd ]; then
daemon="$gitstatus_dir"/usrbin/gitstatusd
fi
if [ -n "$daemon" ]; then
local gitstatus_version= libgit2_version=
if ! . "$gitstatus_dir"/build.info; then
>&2 echo "[gitstatus] internal error: failed to source build.info"
return 1
fi
if [ -z "$gitstatus_version" ]; then
>&2 echo "[gitstatus] internal error: empty gitstatus_version in build.info"
return 1
fi
local version="$gitstatus_version"
_gitstatus_install_daemon_found 0 "$@"
return
fi
fi
while IFS= read -r line; do
line="${line###*}"
[ -n "$line" ] || continue
local uname_s_glob= uname_m_glob= file= version= sha256=
eval "$line" || return
if [ -z "$uname_s_glob" -o \
-z "$uname_m_glob" -o \
-z "$file" -o \
-z "$version" -o \
-z "$sha256" ]; then
>&2 echo "[gitstatus] internal error: invalid install.info line: $line"
return 1
fi
case "$uname_s" in
$uname_s_glob) ;;
*) continue;;
esac
case "$uname_m" in
$uname_m_glob) ;;
*) continue;;
esac
# Found a match. The while loop will terminate during this iteration.
if [ -z "$no_check" ]; then
# Check if a suitable gitstatusd already exists.
local daemon="$gitstatus_dir"/usrbin/"$file"
if [ ! -e "$daemon" ]; then
daemon="$cache_dir"/"$file"
[ -e "$daemon" ] || daemon=
fi
if [ -n "$daemon" ]; then
_gitstatus_install_daemon_found 0 "$@"
return
fi
fi
# No suitable gitstatusd exists. Need to download.
if [ -n "$no_install" ]; then
>&2 echo "[gitstatus] error: no gitstatusd found and installation is disabled"
return 1
fi
local daemon="$cache_dir"/"$file"
if [ -n "${cache_dir##/*}" ]; then
>&2 echo "[gitstatus] error: GITSTATUS_CACHE_DIR is not absolute: $cache_dir"
return 1
fi
if [ ! -d "$cache_dir" ] && ! mkdir -p -- "$cache_dir" || [ ! -w "$cache_dir" ]; then
local dir="$cache_dir"
while true; do
if [ -e "$dir" ]; then
if [ ! -d "$dir" ]; then
>&"$e" printf 'Not a directory: \033[4;31m%s\033[0m\n' "$dir"
>&"$e" printf '\n'
>&"$e" printf 'Delete it, then restart your shell.\n'
elif [ ! -w "$dir" ]; then
>&"$e" printf 'Directory is not writable: \033[4;31m%s\033[0m\n' "$dir"
>&"$e" printf '\n'
>&"$e" printf 'Make it writable, then restart your shell.\n'
fi
break
fi
if [ "$dir" = / ] || [ "$dir" = . ]; then
break
fi
dir="$(dirname -- "$dir")"
done
return 1
fi
if [ -n "${TMPDIR-}" -a '(' '(' -d "${TMPDIR-}" -a -w "${TMPDIR-}" ')' -o '!' '(' -d /tmp -a -w /tmp ')' ')' ]; then
local tmp="$TMPDIR"
else
local tmp=/tmp
fi
if ! command -v mktemp >/dev/null 2>&1 ||
! tmpdir="$(command mktemp -d "$tmp"/gitstatus-install.XXXXXXXXXX)"; then
tmpdir="$tmp/gitstatus-install.tmp.$$"
if ! mkdir -p -- "$tmpdir"; then
if [ "$tmp" = /tmp ]; then
local label='directory'
else
local label='directory (\033[1mTMPDIR\033[m)'
fi
if [ ! -e "$tmp" ]; then
>&"$e" printf 'Temporary '"$label"' does not exist: \033[4;31m%s\033[0m\n' "$tmp"
>&"$e" printf '\n'
>&"$e" printf 'Create it, then restart your shell.\n'
elif [ ! -d "$tmp" ]; then
>&"$e" printf 'Not a '"$label"': \033[4;31m%s\033[0m\n' "$tmp"
>&"$e" printf '\n'
>&"$e" printf 'Make it a directory, then restart your shell.\n'
elif [ ! -w "$tmp" ]; then
>&"$e" printf 'Temporary '"$label"' is not writable: \033[4;31m%s\033[0m\n' "$tmp"
>&"$e" printf '\n'
>&"$e" printf 'Make it writable, then restart your shell.\n'
fi
return 1
fi
fi
if ! command -v curl >/dev/null 2>&1 && ! command -v wget >/dev/null 2>&1; then
>&"$e" printf 'Please install \033[32mcurl\033[0m or \033[32mwget\033[0m, then restart your shell.\n'
return 1
fi
(
run_cmd() {
command -v "$1" >/dev/null 2>/dev/null || return 127
local trapped= pid die ret
trap 'trapped=1' $sig
# The only reason for suppressing stderr is that `curl -f` cannot be silenced:
# `-s` doesn't work despite what the docs say.
command "$@" 2>/dev/null &
ret="$?"
if [ "$ret" = 0 ]; then
pid="$!"
die="trap - $sig; kill -- $pid 2>/dev/null; wait -- $pid 2>/dev/null; exit 1"
trap "$die" $sig
[ -z "$trapped" ] || eval "$die"
wait -- "$pid" 2>/dev/null
ret="$?"
fi
trap - $sig
[ -z "$trapped" ] || exit
return "$ret"
}
check_sha256() {
local data_file="$tmpdir"/"$1".tar.gz
local hash_file="$tmpdir"/"$1".tar.gz.sha256
local hash=
{
command -v shasum >/dev/null 2>/dev/null &&
run_cmd shasum -b -a 256 -- "$data_file" >"$hash_file" </dev/null &&
IFS= read -r hash <"$hash_file" &&
hash="${hash%% *}" &&
[ ${#hash} -eq 64 ]
} || {
command -v sha256sum >/dev/null 2>/dev/null &&
run_cmd sha256sum -b -- "$data_file" >"$hash_file" </dev/null &&
IFS= read -r hash <"$hash_file" &&
hash="${hash%% *}" &&
[ ${#hash} -eq 64 ]
} || {
# Note: sha256 can be from hashalot. It's incompatible.
# Thankfully, it produces shorter output.
command -v sha256 >/dev/null 2>/dev/null &&
run_cmd sha256 -- "$data_file" >"$hash_file" </dev/null &&
IFS= read -r hash <"$hash_file" &&
hash="${hash##* }" &&
[ ${#hash} -eq 64 ]
} || {
hash=
}
[ "$1" = 1 -a -z "$hash" -o "$hash" = "$sha256" ]
}
local url1="https://github.com/romkatv/gitstatus/releases/download/$version/$file.tar.gz"
local url2="https://gitee.com/romkatv/gitstatus/raw/release-$version/release/$file.tar.gz"
local sig='INT QUIT TERM ILL PIPE'
fetch() {
if [ "$1" != 1 ] && command -v sleep >/dev/null 2>/dev/null; then
if ! run_cmd sleep "$1"; then
echo -n >"$tmpdir"/"$1".status
return 1
fi
fi
local cmd part url ret
for cmd in 'curl -kfsSL' 'wget -qO-' 'curl -q -kfsSL' 'wget --no-config -qO-'; do
part=0
while true; do
if [ "$part" = 2 ]; then
ret=1
break
elif [ "$part" = 0 ]; then
url="$2"
else
url="$2"."$part"
fi
run_cmd $cmd -- "$url" >>"$tmpdir"/"$1".tar.gz
ret="$?"
[ "$ret" = 0 ] || break
check_sha256 "$1" && break
part=$((part+1))
done
[ "$ret" = 0 ] && break
run_cmd rm -f -- "$tmpdir"/"$1".tar.gz && continue
ret="$?"
break
done
echo -n >"$tmpdir"/"$1".status
return "$ret"
}
local trapped=
trap 'trapped=1' $sig
fetch 1 "$url1" &
local pid1="$!"
fetch 2 "$url2" &
local pid2="$!"
local die="trap - $sig; kill -- $pid1 $pid2 2>/dev/null; wait -- $pid1 $pid2 2>/dev/null; exit 1"
trap "$die" $sig
[ -z "$trapped" ] || eval "$die"
local n=
while true; do
[ -z "$dl_status" ] || eval "$dl_status" || eval "$die"
if command -v sleep >/dev/null 2>/dev/null; then
command sleep 1
elif command -v true >/dev/null 2>/dev/null; then
command true
fi
if [ -n "$pid1" -a -e "$tmpdir"/1.status ]; then
wait -- "$pid1" 2>/dev/null
local ret="$?"
pid1=
if [ "$ret" = 0 ]; then
if [ -n "$pid2" ]; then
kill -- "$pid2" 2>/dev/null
wait -- "$pid2" 2>/dev/null
fi
n=1
break
elif [ -z "$pid2" ]; then
break
else
die="trap - $sig; kill -- $pid2 2>/dev/null; wait -- $pid2 2>/dev/null; exit 1"
trap "$die" $sig
fi
elif [ -n "$pid2" -a -e "$tmpdir"/2.status ]; then
wait -- "$pid2" 2>/dev/null
local ret="$?"
pid2=
if [ "$ret" = 0 ]; then
if [ -n "$pid1" ]; then
kill -- "$pid1" 2>/dev/null
wait -- "$pid1" 2>/dev/null
fi
n=2
break
elif [ -z "$pid1" ]; then
break
else
die="trap - $sig; kill -- $pid1 2>/dev/null; wait -- $pid1 2>/dev/null; exit 1"
trap "$die" $sig
fi
fi
done
trap - $sig
if [ -z "$n" ]; then
>&"$e" printf 'Failed to download \033[32m%s\033[0m from any mirror:\n' "$file"
>&"$e" printf '\n'
>&"$e" printf ' 1. \033[4m%s\033[0m\n' "$url1"
>&"$e" printf ' 2. \033[4m%s\033[0m\n' "$url2"
>&"$e" printf '\n'
>&"$e" printf 'Check your internet connection, then restart your shell.\n'
exit 1
fi
command tar -C "$tmpdir" -xzf "$tmpdir"/"$n".tar.gz || exit
local tmpfile
if ! command -v mktemp >/dev/null 2>&1 ||
! tmpfile="$(command mktemp "$cache_dir"/gitstatusd.XXXXXXXXXX)"; then
tmpfile="$cache_dir"/gitstatusd.tmp.$$
fi
command mv -f -- "$tmpdir"/"$file" "$tmpfile" || exit
command mv -f -- "$tmpfile" "$cache_dir"/"$file" && exit
command rm -f -- "$cache_dir"/"$file"
command mv -f -- "$tmpfile" "$cache_dir"/"$file" && exit
command rm -f -- "$tmpfile"
exit 1
)
local ret="$?"
command rm -rf -- "$tmpdir"
[ "$ret" = 0 ] || return
_gitstatus_install_daemon_found 1 "$@"
return
done <"$gitstatus_dir"/install.info
>&"$e" printf 'There is no prebuilt \033[32mgitstatusd\033[0m for \033[1m%s\033[0m.\n' "$uname_s $uname_m"
>&"$e" printf '\n'
>&"$e" printf 'See: \033[4mhttps://github.com/romkatv/gitstatus#compiling\033[0m\n'
return 1
}
if [ -z "${0##*/*}" ]; then
_gitstatus_install_main "${0%/*}" "$@"
else
_gitstatus_install_main . "$@"
fi
@@ -0,0 +1,34 @@
# 3
#
# This file is used by ./install and indirectly by shell bindings.
#
# The first line is read by powerlevel10k instant prompt. It must
# be updated whenever the content of this file changes. The actual
# value doesn't matter as long as it's unique. Consecutive integers
# work fine.
# Official gitstatusd binaries.
uname_s_glob="cygwin_nt-10.0"; uname_m_glob="i686"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="5a8a809dcebdb6aa9b47d37e086c0485424a9d9c136770eec3c26cedf5bb75e3";
uname_s_glob="cygwin_nt-10.0"; uname_m_glob="x86_64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.1"; sha256="c84cade0d6b86e04c27a6055f45851f6b46d6b88ba58772f7ca8ef4d295c800f";
uname_s_glob="darwin"; uname_m_glob="arm64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="eae979e990ca37c56ee39fadd0c3f392cbbd0c6bdfb9a603010be60d9e48910a";
uname_s_glob="darwin"; uname_m_glob="x86_64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="9fd3913ec1b6b856ab6e08a99a2343f0e8e809eb6b62ca4b0963163656c668e6";
uname_s_glob="freebsd"; uname_m_glob="amd64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="8e57ad642251e5acfa430aed82cd4ffe103db0bfadae4a15ccaf462c455d0442";
uname_s_glob="linux"; uname_m_glob="aarch64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="32b57eb28bf6d80b280e4020a0045184f8ca897b20b570c12948aa6838673225";
uname_s_glob="linux"; uname_m_glob="armv6l"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.1"; sha256="4bf5a0d0a082f544a48536ad3675930d5d2cc6a8cf906710045e0788f51192b3";
uname_s_glob="linux"; uname_m_glob="armv7l"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.1"; sha256="2b9deb29f86c8209114b71b94fc2e1ed936a1658808a1bee46f4a82fd6a1f8cc";
uname_s_glob="linux"; uname_m_glob="armv8l"; file="gitstatusd-${uname_s}-aarch64"; version="v1.5.4"; sha256="32b57eb28bf6d80b280e4020a0045184f8ca897b20b570c12948aa6838673225";
uname_s_glob="linux"; uname_m_glob="i686"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="56d55e2e9a202d3072fa612d8fa1faa61243ffc86418a7fa64c2c9d9a82e0f64";
uname_s_glob="linux"; uname_m_glob="ppc64le"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="1afd072c8c26ef6ec2d9ac11cef96c84cd6f10e859665a6ffcfb6112c758547e";
uname_s_glob="linux"; uname_m_glob="x86_64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.4"; sha256="9633816e7832109e530c9e2532b11a1edae08136d63aa7e40246c0339b7db304";
uname_s_glob="msys_nt-10.0"; uname_m_glob="i686"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.1"; sha256="7f9b849fc52e7a95b9b933e25121ad5ae990a1871aad6616922ad7bcf1eebf20";
uname_s_glob="msys_nt-10.0"; uname_m_glob="x86_64"; file="gitstatusd-${uname_s}-${uname_m}"; version="v1.5.1"; sha256="5d3c626b5ee564dbc13ddba89752dc58b0efe925b26dbd8b2304849d9ba01732";
# Fallbacks to official gitstatusd binaries.
uname_s_glob="cygwin_nt-*"; uname_m_glob="i686"; file="gitstatusd-cygwin_nt-10.0-${uname_m}"; version="v1.5.2"; sha256="5a8a809dcebdb6aa9b47d37e086c0485424a9d9c136770eec3c26cedf5bb75e3";
uname_s_glob="cygwin_nt-*"; uname_m_glob="x86_64"; file="gitstatusd-cygwin_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="c84cade0d6b86e04c27a6055f45851f6b46d6b88ba58772f7ca8ef4d295c800f";
uname_s_glob="mingw32_nt-*"; uname_m_glob="i686"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="7f9b849fc52e7a95b9b933e25121ad5ae990a1871aad6616922ad7bcf1eebf20";
uname_s_glob="mingw32_nt-*"; uname_m_glob="x86_64"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="5d3c626b5ee564dbc13ddba89752dc58b0efe925b26dbd8b2304849d9ba01732";
uname_s_glob="mingw64_nt-*"; uname_m_glob="i686"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="7f9b849fc52e7a95b9b933e25121ad5ae990a1871aad6616922ad7bcf1eebf20";
uname_s_glob="mingw64_nt-*"; uname_m_glob="x86_64"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="5d3c626b5ee564dbc13ddba89752dc58b0efe925b26dbd8b2304849d9ba01732";
uname_s_glob="msys_nt-*"; uname_m_glob="i686"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="7f9b849fc52e7a95b9b933e25121ad5ae990a1871aad6616922ad7bcf1eebf20";
uname_s_glob="msys_nt-*"; uname_m_glob="x86_64"; file="gitstatusd-msys_nt-10.0-${uname_m}"; version="v1.5.1"; sha256="5d3c626b5ee564dbc13ddba89752dc58b0efe925b26dbd8b2304849d9ba01732";
+406
View File
@@ -0,0 +1,406 @@
#!/usr/bin/env zsh
#
# This script does not have a stable API.
#
# Usage: mbuild [-b git-ref] [kernel-arch]...
#
# Builds a bunch of gitstatusd-* binaries. Without arguments builds binaries
# for all platforms. git-ref defaults to master.
#
# Before using this script you need to set up build servers and list them
# in ~/.ssh/config. There should be a Host entry for every value of `assets`
# association defined below. VMs and cloud instances work as well as physical
# machines, including localhost. As long as the machine has been set up as
# described below and you can SSH to it without password, it should work.
#
# ===[ Build Server Setup ]===
#
# Linux
#
# - Install docker.
# $ apt install docker.io # adjust appropriately if there is no `apt`
# $ usermod -aG docker $USER # not needed if going to build as root
# - Install git.
# $ apt install git # adjust appropriately if there is no `apt`
#
# macOS
#
# - Install compiler tools:
# $ xcode-select --install
# - Install homebrew: https://brew.sh/.
# $ bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
#
# FreeBSD
#
# - Install git.
# $ pkg install git
#
# Windows
#
# - Disable Windows Defender (optional).
# ps> Set-MpPreference -DisableRealtimeMonitoring $true
# - Install 64-bit and 32-bit msys2: https://www.msys2.org/wiki/MSYS2-installation/.
# - Open each of them after installation, type `pacman -Syu --noconfirm` and close the window.
# - Then run in powershell while having no msys2 or cygwin windows open:
# ps> C:\msys32\autorebase.bat
# ps> C:\msys64\autorebase.bat
# - Install 64-bit and 32-bit cygwin: https://cygwin.com/install.html.
# - Choose to install 32-bit to c:/cygwin32 instead of the default c:/cygwin.
# - Select these packages: binutils, cmake, gcc-core, gcc-g++, git, make, perl, wget.
#
# IMPORTANT: Install msys2 and cygwin one at a time.
#
# IMPORTANT: msys2 builder can reboot the build machine.
#
# Option 1: OpenSSH for Windows
#
# - Install OpenSSH: https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse.
# ps> Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
# ps> Start-Service sshd
# ps> Set-Service -Name sshd -StartupType 'Automatic'
# - Enable publickey authentication: https://stackoverflow.com/a/50502015/1095235.
# ps> cd $env:USERPROFILE
# ps> mkdir .ssh
# ps> notepad.exe .ssh/authorized_keys
# - Paste your public key, save, close.
# ps> icacls .ssh/authorized_keys /inheritance:r
# ps> notepad.exe C:\ProgramData\ssh\sshd_config
# - Comment out these two lines, save, close:
# # Match Group administrators
# # AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys
# ps> Restart-Service sshd
#
# Option 2: OpenSSH from WSL
#
# - Install WSL.
# - Install Ubuntu.
# - Install sshd.
# $ apt install openssh-server
# $ dpkg-reconfigure openssh-server
# $ cat >/etc/ssh/sshd_config <<\END
# ClientAliveInterval 60
# AcceptEnv TERM LANG LC_*
# PermitRootLogin no
# AllowTcpForwarding no
# AllowAgentForwarding no
# AllowStreamLocalForwarding no
# AuthenticationMethods publickey
# END
# service ssh --full-restart
# - Add your public ssh key to ~/.ssh/authorized_keys.
# - Make `sshd` start when Windows boots.
'emulate' '-L' 'zsh' '-o' 'no_aliases' '-o' 'err_return'
setopt no_unset extended_glob pipe_fail prompt_percent typeset_silent \
no_prompt_subst no_prompt_bang pushd_silent warn_create_global
if [[ $ZSH_VERSION != (5.<1->*|<6->.*) || $ZSH_VERSION == 5.4(|.*) ]]; then
print -ru2 -- "[error] unsupported zsh version: $ZSH_VERSION"
return 1
fi
zmodload zsh/system
local -r git_url='https://github.com/romkatv/gitstatus.git'
local -rA assets=(
# target kernel-arch hostname of the build machine
cygwin_nt-10.0-i686 build-windows-x86_64
cygwin_nt-10.0-x86_64 build-windows-x86_64
msys_nt-10.0-i686 build-windows-x86_64
msys_nt-10.0-x86_64 build-windows-x86_64
darwin-arm64 build-macos-arm64
darwin-x86_64 build-macos-x86_64
freebsd-amd64 build-freebsd-amd64
linux-aarch64 build-linux-aarch64
linux-armv6l build-linux-armv7l
linux-armv7l build-linux-armv7l
linux-i686 build-linux-x86_64
linux-ppc64le build-linux-ppc64le
linux-x86_64 build-linux-x86_64
)
local -rA protocol=(
'cygwin_nt-10.0-*' windows
'msys_nt-10.0-*' windows
'darwin-*' unix
'freebsd-*' unix
'linux-*' unix
)
local -r rootdir=${ZSH_SCRIPT:h}
local -r logs=$rootdir/logs
local -r locks=$rootdir/locks
local -r binaries=$rootdir/usrbin
function usage() {
print -r -- 'usage: mbuild [-b REF] [KERNEL-ARCH]...'
}
local OPTARG opt git_ref=master
local -i OPTIND
while getopts ":b:h" opt; do
case $opt in
h) usage; return 0;;
b) [[ -n $OPTARG ]]; git_ref=$OPTARG;;
\?) print -ru2 -- "mbuild: invalid option: -$OPTARG" ; return 1;;
:) print -ru2 -- "mbuild: missing required argument: -$OPTARG"; return 1;;
*) print -ru2 -- "mbuild: invalid option: -$opt" ; return 1;;
esac
done
shift $((OPTIND - 1))
(( $# )) || set -- ${(ko)assets}
set -- ${(u)@}
local platform
for platform; do
if (( ! $+assets[$platform] )); then
print -ru2 -- "mbuild: invalid platform: $platform"
return 1
fi
done
local build='
rm -rf gitstatus
git clone --recursive --shallow-submodules --depth=1 -b '$git_ref' '$git_url'
cd gitstatus
if command -v zsh >/dev/null 2>&1; then
sh=zsh
elif command -v dash >/dev/null 2>&1; then
sh=dash
elif command -v ash >/dev/null 2>&1; then
sh=ash
else
sh=sh
fi
$sh -x ./build -m '
function build-unix() {
local intro flags=(-sw)
case $2 in
linux-ppc64le) ;;
linux-*) flags+=(-d docker);;
darwin-arm64) intro='PATH="/opt/homebrew/bin:$PATH"';;
darwin-*) intro='PATH="/usr/local/bin:$PATH"';;
esac
ssh $1 -- /bin/sh -uex <<<"
$intro
cd /tmp
$build ${2##*-} ${(j: :)${(@q)flags}}"
scp $1:/tmp/gitstatus/usrbin/gitstatusd $binaries/gitstatusd-$2
}
function build-windows() {
local shell=$(ssh $1 'echo $0')
if [[ $shell == '$0'* ]]; then
local c='c:'
else
local c='/mnt/c'
fi
local tmp env bin intro flags=(-w)
case $2 in
cygwin_nt-10.0-i686) bin='cygwin32/bin' ;|
cygwin_nt-10.0-x86_64) bin='cygwin64/bin' ;|
msys_nt-10.0-i686) bin='msys32/usr/bin';|
msys_nt-10.0-x86_64) bin='msys64/usr/bin';|
cygwin_nt-10.0-*)
tmp='/cygdrive/c/tmp'
;|
msys_nt-10.0-*)
tmp='/c/tmp'
env='MSYSTEM=MSYS'
# TODO: fix this (some errors about PGP keys).
# flags+=(-s)
# intro='pacman -S --needed --noconfirm git; '
intro+='PATH="$PATH:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"'
while true; do
# TODO: run autorebase only when getting an error that can be fixed by autorebasing.
break
local out
out="$(ssh $1 cmd.exe "$c/${bin%%/*}/autorebase.bat" 2>&1)"
[[ $out == *"The following DLLs couldn't be rebased"* ]] || break
# Reboot to get rid of whatever is using those DLLs.
ssh $1 powershell.exe <<<'Restart-Computer -Force' || true
sleep 30
while ! ssh $1 <<<''; do sleep 5; done
done
() {
while true; do
# TODO: fix this (some errors about PGP keys).
break
local -i fd
exec {fd}< <(
ssh $1 $c/$bin/env.exe $env c:/$bin/bash.exe -l 2>&1 <<<"
pacman -Syu --noconfirm
exit")
{
local line
while true; do
IFS= read -u $fd -r line || return 0
if [[ $line == *"warning: terminate MSYS2"* ]]; then
# At this point the machine is hosed. A rogue process with a corrupted name
# is eating all CPU. The top SSH connection won't terminate on its own.
ssh $1 powershell.exe <<<'Restart-Computer -Force' || true
sleep 30
while ! ssh $1 <<<''; do sleep 5; done
break
fi
done
} always {
exec {fd}<&-
kill -- -$sysparams[procsubstpid] 2>/dev/null || true
}
done
} "$@"
;|
esac
ssh $1 $c/$bin/env.exe $env c:/$bin/bash.exe -l <<<"
set -uex
$intro
mkdir -p -- $tmp
cd -- $tmp
$build ${2##*-} ${(j: :)${(@q)flags}}
exit"
scp $1:$c/tmp/gitstatus/usrbin/gitstatusd $binaries/gitstatusd-$2
chmod +x $binaries/gitstatusd-$2
}
if [[ -r /proc/version && "$(</proc/version)" == *Microsoft* ]]; then
() {
(( $# )) || return 0
print -ru2 -- "WARNING: lock files exist: $@"
(( $# )) && rm -- $@
} $locks/*(N)
function flock() {
local fd
sysopen -ro cloexec -u fd <(
exec </dev/null 2>/dev/null
(
trap '' TERM PIPE
local fd
while true; do
sysopen -wo create,excl -u fd -- $1 && break
sleep 1
done
exec {fd}>&-
while true; do
print || break
done
rm -- $1
) &!
)
local REPLY
IFS= read -ru $fd
}
else
function flock() {
: >>$1
zsystem flock $1
}
fi
function build() (
setopt xtrace
local platform=$1
local machine=$assets[$platform]
flock $locks/$machine
build-${protocol[(k)$platform]} $machine $platform
local tmp=gitstatusd-$platform.tmp.$$.tar.gz
( cd -q -- $binaries; tar --owner=0 --group=0 -I 'gzip -9' -cf $tmp gitstatusd-$platform )
mv -f -- $binaries/$tmp $binaries/gitstatusd-$platform.tar.gz
# Make sure the last command is a built-in (important for flock).
:
)
function mbuild() {
local platform pid pids=()
for platform; do
build $platform &>$logs/$platform &
print -r -- "starting build for $platform on $assets[$platform] (pid $!)"
pids+=($platform $!)
done
local failed=()
for platform pid in $pids; do
print -rn -- "$platform => "
if wait $pid; then
print -r -- "ok"
else
print -r -- "error"
failed+=$platform
fi
done
(( $#failed )) || return 0
print
print -r -- "Error logs:"
print
for platform in $failed; do
print -r -- " $platform => $logs/$platform"
done
return 1
}
# Copied from https://github.com/romkatv/run-process-tree.
function run-process-tree() {
zmodload zsh/parameter zsh/param/private || return
local -P opt=(${(kv)options[@]}) || return
local -P pat=(${patchars[@]}) || return
local -P dis_pat=(${dis_patchars[@]}) || return
emulate -L zsh -o err_return || return
setopt monitor traps_async pipe_fail no_unset
zmodload zsh/system
if (( $# == 0 )); then
print -ru2 -- 'usage: run-process-tree command [arg]...'
return 1
fi
local -P stdout REPLY
exec {stdout}>&1
{
{
local -Pi pipe
local -P gid=$sysparams[pid]
local -P sig=(ABRT EXIT HUP ILL INT PIPE QUIT TERM ZERR)
local -P trap=(trap "trap - $sig; kill -- -$sysparams[pid]" $sig)
exec {pipe}>&1 1>&$stdout
$trap
{
$trap
while sleep 1 && print -u $pipe .; do; done
} 2>/dev/null &
local -Pi watchdog=$!
{
trap - ZERR
exec {pipe}>&-
enable -p -- $pat
disable -p -- $dis_pat
options=($opt zle off monitor off)
"$@"
} &
local -Pi ret
wait $! || ret=$?
trap "exit $ret" TERM
kill $watchdog
wait $watchdog
return ret
} | while read; do; done || return
} always {
exec {stdout}>&-
}
}
mkdir -p -- $logs $locks $binaries
() {
run-process-tree mbuild $@
exit
} "$@"
@@ -0,0 +1,37 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#ifndef ROMKATV_GITSTATUS_ALGORITHM_H_
#define ROMKATV_GITSTATUS_ALGORITHM_H_
#include <algorithm>
namespace gitstatus {
// Requires: Iter is a BidirectionalIterator.
//
// Returns iterator pointing to the last value in [begin, end) that compares equal to the value, or
// begin if none compare equal.
template <class Iter, class T>
Iter FindLast(Iter begin, Iter end, const T& val) {
while (begin != end && !(*--end == val)) {}
return end;
}
} // namespace gitstatus
#endif // ROMKATV_GITSTATUS_ALGORITHM_H_
@@ -0,0 +1,118 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#include "arena.h"
#include <algorithm>
#include <type_traits>
#include "bits.h"
#include "check.h"
namespace gitstatus {
namespace {
size_t Clamp(size_t min, size_t val, size_t max) { return std::min(max, std::max(min, val)); }
static const uintptr_t kSingularity = reinterpret_cast<uintptr_t>(&kSingularity);
} // namespace
// Triple singularity. We are all fucked.
Arena::Block Arena::g_empty_block = {kSingularity, kSingularity, kSingularity};
Arena::Arena(Arena::Options opt) : opt_(std::move(opt)), top_(&g_empty_block) {
CHECK(opt_.min_block_size <= opt_.max_block_size);
}
Arena::Arena(Arena&& other) : Arena() { *this = std::move(other); }
Arena::~Arena() {
// See comments in Makefile for the reason sized deallocation is not used.
for (const Block& b : blocks_) ::operator delete(reinterpret_cast<void*>(b.start));
}
Arena& Arena::operator=(Arena&& other) {
if (this != &other) {
// In case std::vector ever gets small object optimization.
size_t idx = other.reusable_ ? other.top_ - other.blocks_.data() : 0;
opt_ = other.opt_;
blocks_ = std::move(other.blocks_);
reusable_ = other.reusable_;
top_ = reusable_ ? blocks_.data() + idx : &g_empty_block;
other.blocks_.clear();
other.reusable_ = 0;
other.top_ = &g_empty_block;
}
return *this;
}
void Arena::Reuse(size_t num_blocks) {
reusable_ = std::min(reusable_, num_blocks);
for (size_t i = reusable_; i != blocks_.size(); ++i) {
const Block& b = blocks_[i];
// See comments in Makefile for the reason sized deallocation is not used.
::operator delete(reinterpret_cast<void*>(b.start));
}
blocks_.resize(reusable_);
if (reusable_) {
top_ = blocks_.data();
top_->tip = top_->start;
} else {
top_ = &g_empty_block;
}
}
void Arena::AddBlock(size_t size, size_t alignment) {
if (alignment > alignof(std::max_align_t)) {
size += alignment - 1;
} else {
size = std::max(size, alignment);
}
if (size <= top_->size() && top_ < blocks_.data() + reusable_ - 1) {
assert(blocks_.front().size() == top_->size());
++top_;
top_->tip = top_->start;
return;
}
if (size <= opt_.max_alloc_threshold) {
size =
std::max(size, Clamp(opt_.min_block_size, NextPow2(top_->size() + 1), opt_.max_block_size));
}
auto p = reinterpret_cast<uintptr_t>(::operator new(size));
blocks_.push_back(Block{p, p, p + size});
if (reusable_) {
if (size < blocks_.front().size()) {
top_ = &blocks_.back();
return;
}
if (size > blocks_.front().size()) reusable_ = 0;
}
std::swap(blocks_.back(), blocks_[reusable_]);
top_ = &blocks_[reusable_++];
}
void* Arena::AllocateSlow(size_t size, size_t alignment) {
assert(alignment && !(alignment & (alignment - 1)));
AddBlock(size, alignment);
assert(Align(top_->tip, alignment) + size <= top_->end);
return Allocate(size, alignment);
}
} // namespace gitstatus
@@ -0,0 +1,273 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#ifndef ROMKATV_GITSTATUS_ARENA_H_
#define ROMKATV_GITSTATUS_ARENA_H_
#include <cassert>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <limits>
#include <new>
#include <type_traits>
#include <vector>
#include "string_view.h"
namespace gitstatus {
// Thread-compatible. Very fast and very flexible w.r.t. allocation size and alignment.
//
// Natural API extensions:
//
// // Donates a block to the arena. When the time comes, it'll be freed with
// // free(p, size, userdata).
// void Donate(void* p, size_t size, void* userdata, void(*free)(void*, size_t, void*));
class Arena {
public:
struct Options {
// The first call to Allocate() will allocate a block of this size. There is one exception when
// the first requested allocation size is larger than this limit. Subsequent blocks will be
// twice as large as the last until they saturate at max_block_size.
size_t min_block_size = 64;
// Allocate blocks at most this large. There is one exception when the requested allocation
// size is larger than this limit.
size_t max_block_size = 8 << 10;
// When the size of the first allocation in a block is larger than this threshold, the block
// size will be equal to the allocation size. This is meant to reduce memory waste when making
// many allocations with sizes slightly over max_block_size / 2. With max_alloc_threshold equal
// to max_block_size / N, the upper bound on wasted memory when making many equally-sized
// allocations is 100.0 / (N + 1) percent. When making allocations of different sizes, the upper
// bound on wasted memory is 50%.
size_t max_alloc_threshold = 1 << 10;
// Natural extensions:
//
// void* userdata;
// void (*alloc)(size_t size, size_t alignment, void* userdata);
// void (*free)(void* p, size_t size, void* userdata);
};
// Requires: opt.min_block_size <= opt.max_block_size.
//
// Doesn't allocate any memory.
Arena(Options opt);
Arena() : Arena(Options()) {}
Arena(Arena&&);
~Arena();
Arena& operator=(Arena&& other);
// Requires: alignment is a power of 2.
//
// Result is never null and always aligned. If size is zero, the result may be equal to the last.
// Alignment above alignof(std::max_align_t) is supported. There is no requirement for alignment
// to be less than size or to divide it.
inline void* Allocate(size_t size, size_t alignment) {
assert(alignment && !(alignment & (alignment - 1)));
uintptr_t p = Align(top_->tip, alignment);
uintptr_t e = p + size;
if (e <= top_->end) {
top_->tip = e;
return reinterpret_cast<void*>(p);
}
return AllocateSlow(size, alignment);
}
template <class T>
inline T* Allocate(size_t n) {
static_assert(!std::is_reference<T>(), "");
return static_cast<T*>(Allocate(n * sizeof(T), alignof(T)));
}
template <class T>
inline T* Allocate() {
return Allocate<T>(1);
}
inline char* MemDup(const char* p, size_t len) {
char* res = Allocate<char>(len);
std::memcpy(res, p, len);
return res;
}
// Copies the null-terminated string (including the trailing null character) to the arena and
// returns a pointer to the copy.
inline char* StrDup(const char* s) {
size_t len = std::strlen(s);
return MemDup(s, len + 1);
}
// Guarantees: !StrDup(p, len)[len].
inline char* StrDup(const char* p, size_t len) {
char* res = Allocate<char>(len + 1);
std::memcpy(res, p, len);
res[len] = 0;
return res;
}
// Guarantees: !StrDup(s)[s.len].
inline char* StrDup(StringView s) {
return StrDup(s.ptr, s.len);
}
template <class... Ts>
inline char* StrCat(const Ts&... ts) {
return [&](std::initializer_list<StringView> ss) {
size_t len = 0;
for (StringView s : ss) len += s.len;
char* p = Allocate<char>(len + 1);
for (StringView s : ss) {
std::memcpy(p, s.ptr, s.len);
p += s.len;
}
*p = 0;
return p - len;
}({ts...});
}
// Copies/moves `val` to the arena and returns a pointer to it.
template <class T>
inline std::remove_const_t<std::remove_reference_t<T>>* Dup(T&& val) {
return DirectInit<std::remove_const_t<std::remove_reference_t<T>>>(std::forward<T>(val));
}
// The same as `new T{args...}` but on the arena.
template <class T, class... Args>
inline T* DirectInit(Args&&... args) {
T* res = Allocate<T>();
::new (const_cast<void*>(static_cast<const void*>(res))) T(std::forward<Args>(args)...);
return res;
}
// The same as `new T(args...)` but on the arena.
template <class T, class... Args>
inline T* BraceInit(Args&&... args) {
T* res = Allocate<T>();
::new (const_cast<void*>(static_cast<const void*>(res))) T{std::forward<Args>(args)...};
return res;
}
// Tip() and TipSize() allow you to allocate the remainder of the current block. They can be
// useful if you are flexible w.r.t. the allocation size.
//
// Invariant:
//
// const void* tip = Tip();
// void* p = Allocate(TipSize(), 1); // grab the remainder of the current block
// assert(p == tip);
const void* Tip() const { return reinterpret_cast<const void*>(top_->tip); }
size_t TipSize() const { return top_->end - top_->tip; }
// Invalidates all allocations (without running destructors of allocated objects) and frees all
// blocks except at most the specified number of blocks. The retained blocks will be used to
// fulfil future allocation requests.
void Reuse(size_t num_blocks = std::numeric_limits<size_t>::max());
private:
struct Block {
size_t size() const { return end - start; }
uintptr_t start;
uintptr_t tip;
uintptr_t end;
};
inline static size_t Align(size_t n, size_t m) { return (n + m - 1) & ~(m - 1); };
void AddBlock(size_t size, size_t alignment);
bool ReuseBlock(size_t size, size_t alignment);
__attribute__((noinline)) void* AllocateSlow(size_t size, size_t alignment);
Options opt_;
std::vector<Block> blocks_;
// Invariant: !blocks_.empty() <= reusable_ && reusable_ <= blocks_.size().
size_t reusable_ = 0;
// Invariant: (top_ == &g_empty_block) == blocks_.empty().
// Invariant: blocks_.empty() || top_ == &blocks_.back() || top_ < blocks_.data() + reusable_.
Block* top_;
static Block g_empty_block;
};
// Copies of ArenaAllocator use the same thread-compatible Arena without synchronization.
template <class T>
class ArenaAllocator {
public:
using value_type = T;
using pointer = T*;
using const_pointer = const T*;
using reference = T&;
using const_reference = const T&;
using size_type = size_t;
using difference_type = ptrdiff_t;
using propagate_on_container_move_assignment = std::true_type;
template <class U>
struct rebind {
using other = ArenaAllocator<U>;
};
using is_always_equal = std::false_type;
ArenaAllocator(Arena* arena = nullptr) : arena_(*arena) {}
Arena& arena() const { return arena_; }
pointer address(reference x) const { return &x; }
const_pointer address(const_reference x) const { return &x; }
pointer allocate(size_type n, const void* hint = nullptr) { return arena_.Allocate<T>(n); }
void deallocate(T* p, std::size_t n) {}
size_type max_size() const { return std::numeric_limits<size_type>::max() / sizeof(value_type); }
template <class U, class... Args>
void construct(U* p, Args&&... args) {
::new (const_cast<void*>(static_cast<const void*>(p))) U(std::forward<Args>(args)...);
}
template <class U>
void destroy(U* p) {
p->~U();
}
bool operator==(const ArenaAllocator& other) const { return &arena_ == &other.arena_; }
bool operator!=(const ArenaAllocator& other) const { return &arena_ != &other.arena_; }
private:
Arena& arena_;
};
template <class C>
struct LazyWithArena;
template <template <class, class> class C, class T1, class A>
struct LazyWithArena<C<T1, A>> {
using type = C<T1, ArenaAllocator<typename C<T1, A>::value_type>>;
};
template <template <class, class, class> class C, class T1, class T2, class A>
struct LazyWithArena<C<T1, T2, A>> {
using type = C<T1, T2, ArenaAllocator<typename C<T1, T2, A>::value_type>>;
};
template <class C>
using WithArena = typename LazyWithArena<C>::type;
} // namespace gitstatus
#endif // ROMKATV_GITSTATUS_DIR_H_
@@ -0,0 +1,29 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#ifndef ROMKATV_GITSTATUS_BITS_H_
#define ROMKATV_GITSTATUS_BITS_H_
#include <cstddef>
namespace gitstatus {
inline size_t NextPow2(size_t n) { return n < 2 ? 1 : (~size_t{0} >> __builtin_clzll(n - 1)) + 1; }
} // namespace gitstatus
#endif // ROMKATV_GITSTATUS_BITS_H_
@@ -0,0 +1,61 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#ifndef ROMKATV_GITSTATUS_CHECK_H_
#define ROMKATV_GITSTATUS_CHECK_H_
#include "logging.h"
#include <stdexcept>
// The argument must be an expression convertible to bool.
// Does nothing if the expression evaluates to true. Otherwise
// it's equivalent to LOG(FATAL).
#define CHECK(cond...) \
static_cast<void>(0), (!!(cond)) ? static_cast<void>(0) : LOG(FATAL) << #cond << ": "
#define VERIFY(cond...) \
static_cast<void>(0), ::gitstatus::internal_check::Thrower(!(cond)) \
? static_cast<void>(0) \
: LOG(ERROR) << #cond << ": "
namespace gitstatus {
struct Exception : std::exception {
const char* what() const noexcept override { return "Exception"; }
};
namespace internal_check {
class Thrower {
public:
Thrower(bool should_throw) : throw_(should_throw) {}
Thrower(Thrower&&) = delete;
explicit operator bool() const { return !throw_; }
~Thrower() noexcept(false) {
if (throw_) throw Exception();
}
private:
bool throw_;
};
} // namespace internal_check
} // namespace gitstatus
#endif // ROMKATV_GITSTATUS_CHECK_H_
@@ -0,0 +1,157 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#include "check_dir_mtime.h"
#include <fcntl.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <time.h>
#include <unistd.h>
#include <cerrno>
#include <cstring>
#include <ctime>
#include <string>
#include <vector>
#include "check.h"
#include "dir.h"
#include "logging.h"
#include "print.h"
#include "scope_guard.h"
#include "stat.h"
namespace gitstatus {
namespace {
constexpr char kDirPrefix[] = ".gitstatus.";
void Touch(const char* path) {
int fd = creat(path, 0444);
VERIFY(fd >= 0) << Errno();
CHECK(!close(fd)) << Errno();
}
bool StatChanged(const char* path, const struct stat& prev) {
struct stat cur;
VERIFY(!lstat(path, &cur)) << Errno();
return !StatEq(prev, cur);
}
void RemoveStaleDirs(const char* root_dir) {
int dir_fd = open(root_dir, O_DIRECTORY | O_CLOEXEC);
if (dir_fd < 0) return;
ON_SCOPE_EXIT(&) { CHECK(!close(dir_fd)) << Errno(); };
Arena arena;
std::vector<char*> entries;
const std::time_t now = std::time(nullptr);
if (!ListDir(dir_fd, arena, entries,
/* precompose_unicode = */ false,
/* case_sensitive = */ true)) {
return;
}
std::string path = root_dir;
const size_t root_dir_len = path.size();
for (const char* entry : entries) {
if (std::strlen(entry) < std::strlen(kDirPrefix)) continue;
if (std::memcmp(entry, kDirPrefix, std::strlen(kDirPrefix))) continue;
struct stat st;
if (fstatat(dir_fd, entry, &st, AT_SYMLINK_NOFOLLOW)) {
LOG(WARN) << "Cannot stat " << Print(entry) << " in " << Print(root_dir) << ": " << Errno();
continue;
}
if (MTim(st).tv_sec + 10 > now) continue;
path.resize(root_dir_len);
path += entry;
size_t dir_len = path.size();
path += "/b/1";
if (unlink(path.c_str()) && errno != ENOENT) {
LOG(WARN) << "Cannot unlink " << Print(path) << ": " << Errno();
continue;
}
for (const char* d : {"/a/1", "/a", "/b", ""}) {
path.resize(dir_len);
path += d;
if (rmdir(path.c_str()) && errno != ENOENT) {
LOG(WARN) << "Cannot remove " << Print(path) << ": " << Errno();
break;
}
}
}
}
} // namespace
bool CheckDirMtime(const char* root_dir) {
try {
RemoveStaleDirs(root_dir);
std::string tmp = std::string() + root_dir + kDirPrefix + "XXXXXX";
VERIFY(mkdtemp(&tmp[0])) << Errno();
ON_SCOPE_EXIT(&) { rmdir(tmp.c_str()); };
std::string a_dir = tmp + "/a";
VERIFY(!mkdir(a_dir.c_str(), 0755)) << Errno();
ON_SCOPE_EXIT(&) { rmdir(a_dir.c_str()); };
struct stat a_st;
VERIFY(!lstat(a_dir.c_str(), &a_st)) << Errno();
std::string b_dir = tmp + "/b";
VERIFY(!mkdir(b_dir.c_str(), 0755)) << Errno();
ON_SCOPE_EXIT(&) { rmdir(b_dir.c_str()); };
struct stat b_st;
VERIFY(!lstat(b_dir.c_str(), &b_st)) << Errno();
while (sleep(1)) {
// zzzz
}
std::string a1 = a_dir + "/1";
VERIFY(!mkdir(a1.c_str(), 0755)) << Errno();
ON_SCOPE_EXIT(&) { rmdir(a1.c_str()); };
if (!StatChanged(a_dir.c_str(), a_st)) {
LOG(WARN) << "Creating a directory doesn't change mtime of the parent: " << Print(root_dir);
return false;
}
std::string b1 = b_dir + "/1";
Touch(b1.c_str());
ON_SCOPE_EXIT(&) { unlink(b1.c_str()); };
if (!StatChanged(b_dir.c_str(), b_st)) {
LOG(WARN) << "Creating a file doesn't change mtime of the parent: " << Print(root_dir);
return false;
}
LOG(INFO) << "All mtime checks have passes. Enabling untracked cache: " << Print(root_dir);
return true;
} catch (const Exception&) {
LOG(WARN) << "Error while testing for mtime capability: " << Print(root_dir);
return false;
}
}
} // namespace gitstatus
@@ -0,0 +1,31 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#ifndef ROMKATV_GITSTATUS_CHECK_DIR_MTIME_H_
#define ROMKATV_GITSTATUS_CHECK_DIR_MTIME_H_
namespace gitstatus {
// Similar to `git update-index --test-untracked-cache` but performs all tests
// in parallel, so the total testing time is one second regardless of the number
// of tests. It also performs fewer tests because gitstatus imposes fewer
// requirements on the filesystem in order to take advantage of untracked cache.
bool CheckDirMtime(const char* root_dir);
} // namespace gitstatus
#endif // ROMKATV_GITSTATUS_CHECK_DIR_MTIME_H_
@@ -0,0 +1,237 @@
// Copyright 2019 Roman Perepelitsa.
//
// This file is part of GitStatus.
//
// GitStatus is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// GitStatus is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with GitStatus. If not, see <https://www.gnu.org/licenses/>.
#include "dir.h"
#include <algorithm>
#include <atomic>
#include <cerrno>
#include <cstring>
#include <dirent.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <unistd.h>
#ifdef __linux__
#include <endian.h>
#include <sys/syscall.h>
#endif
#ifdef __APPLE__
#include <iconv.h>
#endif
#include "bits.h"
#include "check.h"
#include "scope_guard.h"
#include "string_cmp.h"
#include "tribool.h"
namespace gitstatus {
namespace {
bool Dots(const char* name) {
if (name[0] == '.') {
if (name[1] == 0) return true;
if (name[1] == '.' && name[2] == 0) return true;
}
return false;
}
} // namespace
// The linux-specific implementation is about 20% faster than the generic (posix) implementation.
#ifdef __linux__
uint64_t Read64(const void* p) {
uint64_t res;
std::memcpy(&res, p, 8);
return res;
}
void Write64(uint64_t x, void* p) { std::memcpy(p, &x, 8); }
void SwapBytes(char** begin, char** end) {
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
for (; begin != end; ++begin) Write64(__builtin_bswap64(Read64(*begin)), *begin);
#elif __BYTE_ORDER__ != __ORDER_BIG_ENDIAN__
#error "sorry, not implemented"
#endif
}
template <bool kCaseSensitive>
void SortEntries(char** begin, char** end) {
static_assert(kCaseSensitive, "");
SwapBytes(begin, end);
std::sort(begin, end, [](const char* a, const char* b) {
uint64_t x = Read64(a);
uint64_t y = Read64(b);
// Add 5 for good luck.
return x < y || (x == y && std::memcmp(a + 5, b + 5, 256) < 0);
});
SwapBytes(begin, end);
}
template <>
void SortEntries<false>(char** begin, char** end) {
std::sort(begin, end, StrLt<false>());
}
bool ListDir(int dir_fd, Arena& arena, std::vector<char*>& entries, bool precompose_unicode,
bool case_sensitive) {
struct linux_dirent64 {
ino64_t d_ino;
off64_t d_off;
unsigned short d_reclen;
unsigned char d_type;
char d_name[];
};
constexpr size_t kBufSize = 8 << 10;
const size_t orig_size = entries.size();
while (true) {
char* buf = static_cast<char*>(arena.Allocate(kBufSize, alignof(linux_dirent64)));
// Save 256 bytes for the rainy day.
int n = syscall(SYS_getdents64, dir_fd, buf, kBufSize - 256);
if (n < 0) {
entries.resize(orig_size);
return false;
}
for (int pos = 0; pos < n;) {
auto* ent = reinterpret_cast<linux_dirent64*>(buf + pos);
if (!Dots(ent->d_name)) entries.push_back(ent->d_name);
pos += ent->d_reclen;
}
if (n == 0) break;
// The following optimization relies on SYS_getdents64 always returning as many
// entries as would fit. This is not guaranteed by the specification and I don't
// know if this is true in practice. The optimization has no measurable effect on
// gitstatus performance, so it's turned off.
//
// if (n + sizeof(linux_dirent64) + 512 <= kBufSize) break;
}
if (case_sensitive) {
SortEntries<true>(entries.data() + orig_size, entries.data() + entries.size());
} else {
SortEntries<false>(entries.data() + orig_size, entries.data() + entries.size());
}
return true;
}
#else // __linux__
namespace {
char* DirentDup(Arena& arena, const struct dirent& ent, size_t len) {
char* p = arena.Allocate<char>(len + 2);
*p++ = ent.d_type;
std::memcpy(p, ent.d_name, len + 1);
return p;
}
#ifdef __APPLE__
std::atomic<bool> g_iconv_error(true);
Tribool IConvTry(char* inp, size_t ins, char* outp, size_t outs) {
if (outs == 0) return Tribool::kUnknown;
iconv_t ic = iconv_open("UTF-8", "UTF-8-MAC");
if (ic == (iconv_t)-1) {
if (g_iconv_error.load(std::memory_order_relaxed) &&
g_iconv_error.exchange(false, std::memory_order_relaxed)) {
LOG(ERROR) << "iconv_open(\"UTF-8\", \"UTF-8-MAC\") failed";
}
return Tribool::kFalse;
}
ON_SCOPE_EXIT(&) { CHECK(iconv_close(ic) == 0) << Errno(); };
--outs;
if (iconv(ic, &inp, &ins, &outp, &outs) >= 0) {
*outp = 0;
return Tribool::kTrue;
}
return errno == E2BIG ? Tribool::kUnknown : Tribool::kFalse;
}
char* DirenvConvert(Arena& arena, struct dirent& ent, bool do_convert) {
if (!do_convert) return DirentDup(arena, ent, std::strlen(ent.d_name));
size_t len = 0;
do_convert = false;
for (unsigned char c; (c = ent.d_name[len]); ++len) {
if (c & 0x80) do_convert = true;
}
if (!do_convert) return DirentDup(arena, ent, len);
size_t n = NextPow2(len + 2);
while (true) {
char* p = arena.Allocate<char>(n);
switch (IConvTry(ent.d_name, len, p + 1, n - 1)) {
case Tribool::kFalse:
return DirentDup(arena, ent, len);
case Tribool::kTrue:
*p = ent.d_type;
return p + 1;
case Tribool::kUnknown:
break;
}
n *= 2;
}
}
#else // __APPLE__
char* DirenvConvert(Arena& arena, struct dirent& ent, bool do_convert) {
return DirentDup(arena, ent, std::strlen(ent.d_name));
}
#endif // __APPLE__
} // namespace
bool ListDir(int dir_fd, Arena& arena, std::vector<char*>& entries, bool precompose_unicode,
bool case_sensitive) {
const size_t orig_size = entries.size();
dir_fd = dup(dir_fd);
if (dir_fd < 0) return false;
DIR* dir = fdopendir(dir_fd);
if (!dir) {
CHECK(!close(dir_fd)) << Errno();
return false;
}
ON_SCOPE_EXIT(&) { CHECK(!closedir(dir)) << Errno(); };
while (struct dirent* ent = (errno = 0, readdir(dir))) {
if (Dots(ent->d_name)) continue;
entries.push_back(DirenvConvert(arena, *ent, precompose_unicode));
}
if (errno) {
entries.resize(orig_size);
return false;
}
StrSort(entries.data() + orig_size, entries.data() + entries.size(), case_sensitive);
return true;
}
#endif // __linux__
} // namespace gitstatus

Some files were not shown because too many files have changed in this diff Show More