The canary walked one path on one machine — a mesh comes up, a module
lands, a consumer gets a credential — and stopped the run if it broke.
That path is exactly what the first three tests of the long run walk,
and the long run finishes them about 160 seconds in.
So the gate cost a whole scenario on every passing run to save roughly
45 seconds on a failing one. A scenario is three machines, one of them a
registry that boots a kernel in order to serve files, which is where the
two minutes went.
The test file stays and still runs when it is named. What is gone is
raising it on the way to everything else.
Measured rather than argued: the canary's scenario took 116s of which
60s was standing up a registry, and the run reached the same assertions
without it.
jschoubben
2b3a30619b