mesh-catalog/modules/ssh-client

ssh-client

The operator account's ~/.ssh as a module (novox/hq research 027/03, ADR 0182; to-be 42 phase 1, item 9). sshd is the machine's side.

What it declares, by ADR 0182's classes

path	class	how
~/.ssh/	owned	directory, the account's, mode 0700
~/.ssh/config.d/	owned	directory, the account's, mode 0700. Apart from keys, it holds only the files this module declares (ADR 0415 §7). As every module's owned path, ssh-client is the only module that writes under ~/.ssh: another module that needs a host there provides it through a provision, and this module writes it (ADR 0349 §1; the forge's host comes from forge-account). A host of the operator's own goes in ~/.ssh/config below the mesh's region
~/.ssh/config.d/10-forge	owned	the forge's host, with the machine's own key (ADR 0399)
~/.ssh/mesh_ed25519, .pub	written by the module's own process	the machine's account's key for the forge, made once (keykeeper.go)
~/.ssh/config.d/00-mesh	owned	a Host block per other machine of the mesh (a roster fact), regenerated whenever a machine joins, leaves or is renamed. It begins with the mesh's header
~/.ssh/config	written into, at the start	the mesh's region, # BEGIN mesh ssh-client.config … # END …. It holds one Include ~/.ssh/config.d/*. Every line below the region is the operator's, kept byte for byte
~/.ssh/authorized_keys	found	see The gap
~/.ssh/known_hosts	found	see The gap
~/.ssh/authorized_keys2, known_hosts2, rc, environment	kept	the files ssh and sshd read by themselves; this module writes none of them and never moves them (ADR 0415 §1, ADR 0417 §1)
private keys, public keys, certificates	kept	never read, never written, never moved, at any depth and whatever their name. ssh_client_keys and ssh_client_check read a file's first line to recognise a key, and ask ssh-keygen about it
anything else	swept	moved out of ~/.ssh by the sweep (below)

Why an include, not Host blocks in the region

ssh takes the first value it finds for each option. Research 027/03 asks that the mesh's hosts come before anything else in ~/.ssh/config. Before this change, the region was written at the end: a predecessor's hosts above it won, for the same machines.

A roster fact cannot be placed at the start. The controller gives facts no at, and the host leaves an existing region where it is. So the region at the start holds only the include, and the hosts are a whole file in config.d that the include reads first. 00- sorts it before any other drop-in.

ssh restores the including file's section after an Include (OpenSSH readconf.c). So an operator line just below the region is global again, as it was at the top of the file. This module's tests parse the declared region and check it, and ssh -G was compared before and after on every machine.

The old region, ssh-client.fact-ssh-config at the end of ~/.ssh/config, is no longer declared, so the host removes it, and only it, at the first push.

The gap: authorized keys and known hosts

Research 027/03 gives the mesh a region in authorized_keys (the operator's keys as the mesh records them) and one in known_hosts (every machine's host key). The controller holds neither fact.

• A roster template sees each machine's name, mesh name, address and account, and nothing else (mesh-controller roster.go).
• No machine fact carries an ssh host key.
• The only key the controller knows for the operator is a sealing key for secrets, not an ssh key.

So both files stay found, and this module invents nothing.

What would close it:

1. The host reports its machine's public host keys in its profile, and the roster gains a field for them.
2. The operator's public keys become a mesh record: per account, with a comment saying whose and where.

Each region is then one more into: block resource here. Until then, ssh_client_check reads the files as they are, and ssh_client_revoke / ssh_client_known_host act on them by hand.

The sweep: ~/.ssh holds only what this module knows (ADR 0415, issue 1017)

The module owns ~/.ssh, so it keeps it to what it knows. Its process sweeps when it starts and every ten minutes. It knows the paths in the table above, every key (a private key by its first line, a public key or certificate by its line, at any depth and whatever its name) and sockets (ssh's own runtime). Every other path is moved, whole, into ~/.local/state/ssh-client/retired/<UTC time>/, keeping its path relative to ~/.ssh, and the list of what moved is written beside it as <UTC time>.json. Nothing is deleted. A folder that holds a key is gone into rather than moved whole, so the key stays where it is. A file or folder that cannot be read to tell whether it holds a key is left and named in the log.

ADR 0417 (issue 1020) adds:

• the files ssh and sshd read by themselves (authorized_keys2, known_hosts2, rc, environment) stay;
• a symbolic link to a key, or to a folder holding one, stays; a link whose target cannot be checked is left and named; any other link is moved as the link;
• a path changed less than a minute ago is being written (ssh-keygen's or ssh_client_revoke's temporary file) and waits for the next sweep;
• the retired folder must be on ~/.ssh's file system: otherwise the sweep moves nothing, makes no folder and fails. A sweep that moved nothing leaves no folder;
• the list names each move before it is made, and a list that cannot be written stops the sweep;
• each sweep writes what it did, or why it failed, to ~/.local/state/ssh-client/last-sweep.json. ssh_client_check reports a failed last sweep, and a listed move with nothing at its destination, as a problem.

No other module declares a path under ~/.ssh (ADR 0415 §6): the catalogue's checks/owned (ADR 0422 §8) fails one that does.

So the predecessor's config.d/mesh, config.bak-*, known_hosts.old, removed-* and authorized_keys.revoked-* leave ~/.ssh at the first sweep. Each move is logged by the process, and ssh_client_check lists every sweep that moved something, with where each path lies now.

The predecessor's lines inside ~/.ssh/config, below the mesh's region (its header comment and its Include ~/.ssh/config.d/mesh), are the operator's part of a written-into file and stay. ssh skips an Include that matches no file.

Tools

They run as the operator account and never escalate. No answer carries a key: fingerprints only.

tool		what
ssh_client_hosts	r	every Host and Match section in the order ssh reads it, each with file, line and source (mesh, drop-in, operator); duplicates; what is set outside any section
ssh_client_resolve	r	ssh -G for one host: what ssh would use, and which sections matched
ssh_client_check	r	modes of the directory and every file; keys with no passphrase (ssh-keygen -y -P "", output used only to compare with the .pub), weak, old, unused, or whose .pub is another key's; one key under several names; the mesh's region first with its include; duplicate hosts; known_hosts for the mesh's machines, missing or stale (scanned live, 5 s each in parallel, unless scan is false); authorized keys without a comment; debris; what the sweep moved out of ~/.ssh and where each lies now. It says what it did not check
ssh_client_keys	r	every private key, by its public half: type, size, fingerprint, comment, mode, age, passphrase, whether ssh offers it by itself
ssh_client_authorized	r	authorized_keys by fingerprint, type, size, comment and options
ssh_client_revoke	a	removes every line with one fingerprint, keeping the file first as authorized_keys.revoked-<time>. It refuses the last key (a lockout) and a key in a mesh region (a push would write it back)
ssh_client_known_host	r/a	known entries against what the host offers now: matches, changed, not known or unreachable. With refresh, it replaces the host's entries through ssh-keygen -R (which keeps known_hosts.old) with what was scanned. That trusts whatever answers
ssh_client_test	r	one batch-mode connection that runs only true: the address reached, the method and key that authenticated, or why not and which keys were offered. A key with a passphrase works only through an agent. The runtime has none in its environment, so the tool looks for the account's agent socket under /run/user/<uid> and names the one it used, or says there was none

Tests

go test ./...

The tests use a temporary home and a fake runner, with public keys made for the tests only. They cover:

• reading order and source across includes, with an operator line after the include being global again;
• the declared region parsed as ssh reads it;
• duplicates;
• keys: fingerprints computed as ssh-keygen -l does, RSA size, passphrase asked and never prompted, a mismatched .pub;
• authorized never printing a key;
• revoke: the backup, the mode kept, a lockout refused, a mesh region refused;
• known_host matching, changed and refreshed, and an unreachable host never refreshed;
• test: success, and failure with the agent named;
• check's findings;
• the sweep: what it keeps (keys at any depth, links to keys and to key folders, sockets, what ssh reads itself), what it moves, what it leaves and names, a file being written, another file system, and the list written before each move;
• that the tools served are the manifest's tools.

Files it does not know

Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).

• ~/.ssh: the module's own sweep moves everything it does not know into a kept folder, listing each move first and failing loudly when one cannot be made (ADR 0415, ADR 0417); value module (ADR 0422 §3).
• ~/.ssh/config.d: the same sweep covers it; value module (ADR 0415, ADR 0417).