A consumer contributes a key prefix and gets an ACL user; the test is
that the grant means exactly what the manifest said, in both
directions: its own keys usable, anyone else's refused by the store
itself, and the flush a tenant must never have refused with them.
Waited for through the store rather than through logs: the user list,
asked with the password the host wrote into the server's own conf file
on the machine — nothing invented, both ends reading what the mesh
delivered.
And the forge is asked on the port the mesh assigned, not the one the
module declared. The old curl aimed at 3000, which was right until
ADR 0038 moved the machine side — a latent break that would have fired
on the first run to get past the settling that used to fail first.
The scenario stocks redis and its provisioner, and the rebuild builds
the provisioner image with the others.
Three changes, found by one failing test.
The forge failed three runs in a row as "status hangs", and it was
diagnosed twice as contention — real defects, fixed, and not the cause.
The heartbeats told the truth in the end: every exec on anchor crawled
from 15s to 105s, because eleven containers plus a database pull were
running in a 1GiB machine. Starvation presents as whatever you were
doing when the page-outs start, which is why it wore two other bugs'
clothes first.
So machine size is now the scenario's to declare — memory and cpus per
machine, default unchanged. The anchor that carries the whole substrate
is bigger than the laptop that joins it, and the comment on the
scenario says why in terms of what lands there.
`egress: true` gives a machine one extra interface on a lab-supplied
NAT network, addressed by DHCP because the one address a scenario has
no business choosing is on the host's side of the fence. Declared
per machine and off by default: a closed scenario stays the rule
(novox/hq ADR 0016), and the exception exists because a first node
fetches its images before any mesh can serve them — which is now the
tested path (04-ISSUES/029), and a lab that can never reach upstream
cannot prove the bootstrap it exists to prove. The uplink route is
metric-4096, so it never shadows a route the scenario declared. A
detached machine declaring egress is refused, not ignored.
And settled() treats a poll that threw as a poll that missed. An exec
timeout at minute four of a wait is "could not ask", not a verdict on
the machine.
**A password beginning with a dash broke the search for it.** The
credential test greps the machine's own files for the delivered
password; this run's password started `-S`, so grep read it as an option
and refused the whole invocation. The test compared the usage message
against "0" and reported the password as leaked. That is the worst way
for a search to fail — it says it found something. Fixed with `-e` and
`--`, which is what those exist for.
**The planning test could not redirect what the scenario does not
serve.** Rewriting an image reference only works for repositories the
scenario's registry actually holds, and the object store's provisioner
was not stocked — so that module kept its placeholder and the refusal
fired, correctly. It is stocked now, so all five are planned again. The
skip path stays for anything genuinely unserved, and says which module
and why: a planning test quietly covering four instead of five is the
false coverage this suite exists to prevent.
**The forge failed because of the one above it.** The planning test
threw before its cleanup could run, leaving a module assigned that
refused the next push, so no database container was ever created.
Yesterday's fix moved that cleanup where a failure cannot skip it — but
`after` still only unassigns what was assigned, so it now tracks what
actually got added rather than what was intended.
Everything up to now stopped at composing a declaration. That proves the
control plane and the host agree, and proves nothing about whether the
thing described works — which is how five modules sat pinned to images
that did not exist while parsing and resolving perfectly.
The forge is the right one to run first. It needs a database from
another module, a password it did not choose, and a connection string it
could not have written itself: the address and port come from what the
database serves, the user name from what the mesh decided both ends
would call it. If any of that is wrong it cannot start, and nothing else
in this suite would notice.
The test checks the chain in the order it has to happen — the login
exists, the database it owns exists, the forge answers, and its log does
not say authentication failed. That last one matters: a forge that
started and could not reach its database would still answer on its port.
Rewriting image references is now shared rather than copied from the
bundle, which had the same problem first. A digest is not knowable until
something is built, and when it is, it belongs to whichever registry
served it — so the text says which image and the scenario says which
copy. Matching is on the repository, with a test that a repository
ending in another one is not half-replaced.
Also makes the planning test put the machine back. Tests here share one
mesh, so the five modules it assigned were inherited by whatever ran
next; harmless while nothing pushed, and not harmless now.
**The adoption.** Software nobody here wrote, taking its credentials the
way such software does — from its environment — and needing two
containers that reach each other by name. The first module that could
not have been declared this morning: it needs the network shape and it
needs a sealed value to reach a container's environment.
Its password is accepted rather than generated, which is the whole shape
of an adoption: a service that already exists keeps the credential it
already has. Asserted properly — a wrong password is refused by the same
database, so the passing case means something.
**The warm scenario.** A mesh kept between runs and returned to, which
turned twelve minutes of bootstrap into thirty seconds of restore. Off
unless asked for: a run that is meant to mean something raises from
nothing.
Its guard fired for real during this work, unprompted — a mesh-host
commit landed and it refused the stale base, naming both commits, rather
than passing tests against yesterday's binary. That is 04-ISSUES/005's
rule one level down.
Three things the guard learned the hard way and now handles: a snapshot
captures disk and not memory, so the host is restarted after a restore
and asserted to have come back; the stocked image digests are worked out
while raising and a restored instance never raises, so they are kept;
and comparing only the repositories this run can see clears the ones it
cannot, so both directions are compared.
The one real bug behind five failed attempts was in mesh-host and it
reported itself precisely: a network shape the language had and no host
implemented. Everything else was scaffolding of mine.
The request goes to the name, across the private network, and returns the
workload's own answer. Then the module is unassigned and the same request must
stop working — a stale public name pointing at nothing fails more visibly than
a stale grant.
The workload declares its port as well as its route, because they are different
questions and the earlier test leaves this machine filtering: a module that
asked for a route and not for the port would be unreachable by the proxy it
just asked for.
Two ends holding a matching string proves they agree, not that either is right.
So the check is three logins over the private network from the consumer's own
machine: the delivered credential works, the rotated one works, and the one
that was rotated away does not. Without the last, the test passes against a
provider that added a password without replacing one.
Not over loopback: pg_hba trusts anything there, and a deliberately wrong
password returned a row for a whole afternoon once.
Two setup faults, each of which looked like the thing being tested failing.
The builder module was assigned without its artifact ever being built, so
nothing could start — and the build has to happen while the hand-started
builder is still alive. Same chicken-and-egg as the registry, resolved the same
way: the builder that exists builds the one that replaces it.
The firewall test's listeners were squeezed through three levels of shell
quoting and never started, so the test failed on its own setup — which reads
exactly like the firewall working.
Written and loaded are different things, and loaded and enforcing are different
again. The test opens two ports on a machine, declares one of them, and checks
from the other machine that the declared one answers and the undeclared one
does not — then removes the module and checks the port closes with nobody
editing a rule.
The base image gains nftables, read back through `nft --version` like the other
three: a machine that cannot load a rule set applies the mesh's filtering,
reports success and filters nothing, which is the exact fault the derivation
exists to remove.
Two earlier tests were asking for things that are not there. The lab's registry
drops tags when it stocks, so `registry:2` is not served and the mirror test
failed with "not found" — it now uses the pinned digest, which is what a
declaration carries anyway.
The first raises everything from the bundle its host carries and joins the mesh
it made; the second has a host and nothing else, and a person carries it a
token. This is the first scenario where the mesh is a mesh -- everything before
it proved a machine could talk to a control plane on its own loopback, which
proves less than it looks.
jschoubben