Merge pull request 'The gitea module makes the organisation's repositories from its config, protected before any team reaches them; no tool creates in the organisation (issue 1124, ADR 0428)' (#363) from fix/1124-repositories-config into main
This commit was merged in pull request #363.
This commit is contained in:
@@ -140,6 +140,43 @@ the value is emptied, never deleted. A team without that prefix is never touched
|
||||
change of the value. It says each change in one line on stdout, and never a secret. `go test ./cmd/agent-teams`
|
||||
runs it against a fake forge.
|
||||
|
||||
## The organisation's repositories (novox/hq ADR 0428, issue 1124)
|
||||
|
||||
Which repositories the organisation `novox` holds is one trusted config value, `repositories`:
|
||||
`{"<name>": {"private": true, "description": "..."}}`, changed only at the controller's terminal or on an approval
|
||||
that proves who answered, as `agent-read-teams` is. The module renders it into `repositories.json` and runs
|
||||
`cmd/repositories sync` when it starts and whenever that file changes. With the setup login (`mesh-admin`), for each
|
||||
listed repository:
|
||||
|
||||
- one that is not in the organisation and not under the setup account is made there, **empty and under the setup
|
||||
account**, where neither the tools' team nor the agents' team (both on every repository of the organisation)
|
||||
reaches it. Main gets a rule letting the setup login alone push; main's first commit is written with it: a README
|
||||
naming the description, and a `merge-check.sh` declaring `# mesh-check-toolchain: go` that vets and runs the unit
|
||||
tests, then the integration tests (`-tags integration`), of every Go module in the tree, and says so and passes on
|
||||
a tree with none. The rule is then narrowed to main's rule of ADR 0398 §2 (no push, only `mesh-tools` merges),
|
||||
requiring `mesh/repo-check` and `mesh/merge-gate`, and only then is the repository moved into the organisation.
|
||||
There main's rule is read back and put right once more (the step `rule-after-move`): the forge drops from a merge
|
||||
list a user without access to the repository, and under the setup account `mesh-tools` has none;
|
||||
- one found still under the setup account is finished: the steps it lacks are done, main is never written twice;
|
||||
- one in the organisation is only put right: main's rule narrowed, a missing one of the two statuses added and every
|
||||
status already there kept.
|
||||
|
||||
A step the forge refuses stops that repository, and the others go on. A repository taken out of the value is never
|
||||
asked about: nothing is deleted, renamed, moved out or unprotected. The pass keeps what it did per repository in
|
||||
`repositories-outcome.json`.
|
||||
|
||||
**How a refusal is seen.** `repositories sync` is a run-once process, declared as the module's last resource, so
|
||||
the node-engine's gate on a step that did not complete (ADR 0136) stops nothing of the module after it. When any
|
||||
step was refused it exits 1, and its last stderr lines are one per refused repository,
|
||||
`repositories: <name>: <step> refused: <the forge's answer>`, then, when there were several, a summary line naming
|
||||
them. The node-engine reports the module's step failed and raises apply-failing, and runs it again on every apply
|
||||
until it passes: that is the next pass. The condition says that the `gitea-repositories` step did not complete;
|
||||
the lines naming the repository, the step and the forge's answer are in that unit's journal
|
||||
(`node-service-manager.journal`).
|
||||
No tool creates in the organisation: `gitea_create_repo` creates under the tools' own user only, the tools' team
|
||||
keeps `can_create_org_repo` false and its token has no `write:organization` (ADR 0428 §8).
|
||||
`go test ./cmd/repositories` runs it against a fake forge.
|
||||
|
||||
## The package registry's retention (novox/hq ADR 0251 §4, to-be 51)
|
||||
|
||||
Every publish adds a version to the registry, and until this nothing removed one. The Go bundle
|
||||
|
||||
+5
-12
@@ -244,23 +244,16 @@ export class GiteaClient {
|
||||
return all;
|
||||
}
|
||||
|
||||
/** A repository, created under the tools' own user, or with `owner` in that organisation (novox/hq issue 1124).
|
||||
* The forge's refusal to create in an organisation (403 not allowed, 404 no such organisation, 409 exists) is
|
||||
* thrown with its status and its reason, naming the organisation. */
|
||||
/** A repository, created under the tools' own user and nowhere else (novox/hq ADR 0428 §8): the forge makes whoever
|
||||
* creates an organisation's repository its administrator, so the organisation's repositories are made by the
|
||||
* module's setup login from its config value `repositories` (cmd/repositories), never through a tool. */
|
||||
async createRepo(data: {
|
||||
name: string;
|
||||
description?: string;
|
||||
private?: boolean;
|
||||
auto_init?: boolean;
|
||||
}, owner?: string): Promise<GiteaRepo> {
|
||||
const options = { method: "POST", body: JSON.stringify(data) };
|
||||
if (owner === undefined) return GiteaClient.mapRepo(await this.request<any>("/user/repos", options));
|
||||
const path = `/orgs/${pathSegment("organisation", owner)}/repos`;
|
||||
try {
|
||||
return GiteaClient.mapRepo(await this.request<any>(path, options));
|
||||
} catch (e) {
|
||||
throw new Error(`the forge did not create ${owner}/${data.name} in the organisation ${owner}: ${(e as Error).message}`);
|
||||
}
|
||||
}): Promise<GiteaRepo> {
|
||||
return GiteaClient.mapRepo(await this.request<any>("/user/repos", { method: "POST", body: JSON.stringify(data) }));
|
||||
}
|
||||
|
||||
async deleteRepo(owner: string, repo: string): Promise<void> {
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
// The fake forge of issue 1124 / ADR 0428: it keeps repositories by owner/name with their branch protection rules,
|
||||
// their branches and their files, records every call in order, and refuses a call a test names. Like the forge, it
|
||||
// refuses a commit onto a branch whose rule does not let the caller push, and it answers only the setup login.
|
||||
|
||||
const (
|
||||
admin = "mesh-admin"
|
||||
tools = "mesh-tools"
|
||||
org = "novox"
|
||||
)
|
||||
|
||||
type fRepo struct {
|
||||
Private bool
|
||||
Description string
|
||||
Rules map[string]map[string]any // by rule_name
|
||||
Branches map[string]bool
|
||||
Files map[string]string
|
||||
Commits int
|
||||
}
|
||||
|
||||
type fakeForge struct {
|
||||
mu sync.Mutex
|
||||
repos map[string]*fRepo // "owner/name"
|
||||
calls []string // "METHOD /path"
|
||||
// refuse answers a call with a status and a message when it returns a status above zero.
|
||||
refuse func(method, path string) (int, string)
|
||||
}
|
||||
|
||||
func newFake() *fakeForge { return &fakeForge{repos: map[string]*fRepo{}} }
|
||||
|
||||
func (f *fakeForge) add(full string, rules map[string]map[string]any, branches ...string) *fRepo {
|
||||
r := &fRepo{Rules: map[string]map[string]any{}, Branches: map[string]bool{}, Files: map[string]string{}}
|
||||
for k, v := range rules {
|
||||
r.Rules[k] = v
|
||||
}
|
||||
for _, b := range branches {
|
||||
r.Branches[b] = true
|
||||
}
|
||||
f.repos[full] = r
|
||||
return r
|
||||
}
|
||||
|
||||
// writes are the calls that change something, in order.
|
||||
func (f *fakeForge) writes() []string {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
var out []string
|
||||
for _, c := range f.calls {
|
||||
if !strings.HasPrefix(c, "GET ") {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (f *fakeForge) all() []string {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
return append([]string(nil), f.calls...)
|
||||
}
|
||||
|
||||
var (
|
||||
repoPath = regexp.MustCompile(`^/api/v1/repos/([^/]+)/([^/]+)$`)
|
||||
rulesPath = regexp.MustCompile(`^/api/v1/repos/([^/]+)/([^/]+)/branch_protections(?:/([^/]+))?$`)
|
||||
branchPath = regexp.MustCompile(`^/api/v1/repos/([^/]+)/([^/]+)/branches/([^/]+)$`)
|
||||
filesPath = regexp.MustCompile(`^/api/v1/repos/([^/]+)/([^/]+)/contents$`)
|
||||
movePath = regexp.MustCompile(`^/api/v1/repos/([^/]+)/([^/]+)/transfer$`)
|
||||
)
|
||||
|
||||
func strs(v any) []string {
|
||||
var out []string
|
||||
if l, ok := v.([]any); ok {
|
||||
for _, s := range l {
|
||||
out = append(out, fmt.Sprint(s))
|
||||
}
|
||||
}
|
||||
if l, ok := v.([]string); ok {
|
||||
out = append(out, l...)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// lets says whether a rule lets the login push.
|
||||
func lets(rule map[string]any, login string) bool {
|
||||
if rule == nil {
|
||||
return true
|
||||
}
|
||||
if rule["enable_push"] != true {
|
||||
return false
|
||||
}
|
||||
if rule["enable_push_whitelist"] != true {
|
||||
return true
|
||||
}
|
||||
for _, u := range strs(rule["push_whitelist_usernames"]) {
|
||||
if u == login {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// reaches says whether a login has access to a repository, as the forge judges a merge whitelist: its owner does,
|
||||
// and in the organisation the tools' team does (it is on every repository there); nobody else here.
|
||||
func reaches(full, login string) bool {
|
||||
owner := strings.SplitN(full, "/", 2)[0]
|
||||
return owner == login || (owner == org && login == tools)
|
||||
}
|
||||
|
||||
// keepReaching drops from a rule's merge whitelist every user without access to the repository, as gitea's
|
||||
// updateUserWhitelist does on creating or editing a branch protection.
|
||||
func keepReaching(full string, rule map[string]any) {
|
||||
if _, ok := rule["merge_whitelist_usernames"]; !ok {
|
||||
return
|
||||
}
|
||||
kept := []any{}
|
||||
for _, u := range strs(rule["merge_whitelist_usernames"]) {
|
||||
if reaches(full, u) {
|
||||
kept = append(kept, u)
|
||||
}
|
||||
}
|
||||
rule["merge_whitelist_usernames"] = kept
|
||||
}
|
||||
|
||||
func (f *fakeForge) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if u, p, ok := r.BasicAuth(); !ok || u != admin || p != "pw" {
|
||||
http.Error(w, "no", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
reply := func(status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
if v != nil {
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
}
|
||||
p := r.URL.Path
|
||||
f.calls = append(f.calls, r.Method+" "+p)
|
||||
if f.refuse != nil {
|
||||
if status, said := f.refuse(r.Method, p); status > 0 {
|
||||
reply(status, map[string]string{"message": said})
|
||||
return
|
||||
}
|
||||
}
|
||||
var in map[string]any
|
||||
_ = json.NewDecoder(r.Body).Decode(&in)
|
||||
switch {
|
||||
case p == "/api/v1/version":
|
||||
reply(200, map[string]string{"version": "1.27"})
|
||||
case p == "/api/v1/user/repos" && r.Method == http.MethodPost:
|
||||
name := fmt.Sprint(in["name"])
|
||||
if f.repos[admin+"/"+name] != nil {
|
||||
reply(409, map[string]string{"message": "The repository with the same name already exists."})
|
||||
return
|
||||
}
|
||||
if in["auto_init"] == true {
|
||||
reply(400, map[string]string{"message": "this fake takes only an empty repository"})
|
||||
return
|
||||
}
|
||||
repo := f.add(admin+"/"+name, nil)
|
||||
repo.Private, _ = in["private"].(bool)
|
||||
repo.Description, _ = in["description"].(string)
|
||||
reply(201, map[string]any{"full_name": admin + "/" + name, "empty": true})
|
||||
case repoPath.MatchString(p) && r.Method == http.MethodGet:
|
||||
m := repoPath.FindStringSubmatch(p)
|
||||
repo := f.repos[m[1]+"/"+m[2]]
|
||||
if repo == nil {
|
||||
reply(404, map[string]string{"message": "not found"})
|
||||
return
|
||||
}
|
||||
reply(200, map[string]any{"full_name": m[1] + "/" + m[2], "empty": len(repo.Branches) == 0})
|
||||
case rulesPath.MatchString(p):
|
||||
m := rulesPath.FindStringSubmatch(p)
|
||||
repo := f.repos[m[1]+"/"+m[2]]
|
||||
if repo == nil {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
switch {
|
||||
case r.Method == http.MethodGet && m[3] == "":
|
||||
names := make([]string, 0, len(repo.Rules))
|
||||
for n := range repo.Rules {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
out := []map[string]any{}
|
||||
for _, n := range names {
|
||||
rule := map[string]any{"rule_name": n}
|
||||
for k, v := range repo.Rules[n] {
|
||||
rule[k] = v
|
||||
}
|
||||
out = append(out, rule)
|
||||
}
|
||||
reply(200, out)
|
||||
case r.Method == http.MethodPost && m[3] == "":
|
||||
name := fmt.Sprint(in["rule_name"])
|
||||
if repo.Rules[name] != nil {
|
||||
reply(403, map[string]string{"message": "branch protection already exist"})
|
||||
return
|
||||
}
|
||||
delete(in, "rule_name")
|
||||
keepReaching(m[1]+"/"+m[2], in)
|
||||
repo.Rules[name] = in
|
||||
reply(201, in)
|
||||
case r.Method == http.MethodPatch && m[3] != "":
|
||||
rule := repo.Rules[m[3]]
|
||||
if rule == nil {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
for k, v := range in {
|
||||
rule[k] = v
|
||||
}
|
||||
keepReaching(m[1]+"/"+m[2], rule)
|
||||
reply(200, rule)
|
||||
default:
|
||||
reply(405, nil)
|
||||
}
|
||||
case branchPath.MatchString(p) && r.Method == http.MethodGet:
|
||||
m := branchPath.FindStringSubmatch(p)
|
||||
repo := f.repos[m[1]+"/"+m[2]]
|
||||
if repo == nil || !repo.Branches[m[3]] {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
reply(200, map[string]any{"name": m[3]})
|
||||
case filesPath.MatchString(p) && r.Method == http.MethodPost:
|
||||
m := filesPath.FindStringSubmatch(p)
|
||||
repo := f.repos[m[1]+"/"+m[2]]
|
||||
branch := fmt.Sprint(in["branch"])
|
||||
if repo == nil {
|
||||
reply(404, nil)
|
||||
return
|
||||
}
|
||||
if !lets(repo.Rules[branch], admin) {
|
||||
reply(403, map[string]string{"message": "branch " + branch + " is protected"})
|
||||
return
|
||||
}
|
||||
files, _ := in["files"].([]any)
|
||||
for _, x := range files {
|
||||
op, _ := x.(map[string]any)
|
||||
raw, _ := base64.StdEncoding.DecodeString(fmt.Sprint(op["content"]))
|
||||
repo.Files[fmt.Sprint(op["path"])] = string(raw)
|
||||
}
|
||||
repo.Branches[branch] = true
|
||||
repo.Commits++
|
||||
reply(201, map[string]any{"commit": map[string]any{"sha": fmt.Sprintf("%040d", repo.Commits)}})
|
||||
case movePath.MatchString(p) && r.Method == http.MethodPost:
|
||||
m := movePath.FindStringSubmatch(p)
|
||||
from := m[1] + "/" + m[2]
|
||||
repo := f.repos[from]
|
||||
to := fmt.Sprint(in["new_owner"]) + "/" + m[2]
|
||||
if repo == nil || f.repos[to] != nil {
|
||||
reply(422, map[string]string{"message": "cannot transfer"})
|
||||
return
|
||||
}
|
||||
delete(f.repos, from)
|
||||
f.repos[to] = repo
|
||||
reply(202, map[string]any{"full_name": to})
|
||||
default:
|
||||
reply(404, map[string]string{"message": "not found"})
|
||||
}
|
||||
}
|
||||
|
||||
func syncer(t *testing.T, f *fakeForge) (*Syncer, *[]string) {
|
||||
t.Helper()
|
||||
srv := httptest.NewServer(f)
|
||||
t.Cleanup(srv.Close)
|
||||
pw := filepath.Join(t.TempDir(), "pw")
|
||||
if err := os.WriteFile(pw, []byte("pw\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var said []string
|
||||
return &Syncer{Forge: forgehttp.Client{URL: srv.URL, User: admin, PasswordFile: pw}, Org: org, Admin: admin, Tools: tools,
|
||||
Log: func(format string, a ...any) { said = append(said, fmt.Sprintf(format, a...)) }}, &said
|
||||
}
|
||||
|
||||
func run(t *testing.T, s *Syncer, v Value) Outcome {
|
||||
t.Helper()
|
||||
o, err := s.Sync(context.Background(), v)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
func result(o Outcome, name string) Result {
|
||||
for _, r := range o.Repositories {
|
||||
if r.Name == name {
|
||||
return r
|
||||
}
|
||||
}
|
||||
return Result{}
|
||||
}
|
||||
|
||||
func has(list []string, want string) bool {
|
||||
for _, s := range list {
|
||||
if s == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package main
|
||||
|
||||
import "strings"
|
||||
|
||||
// MergeCheck is the first commit's merge-check.sh (ADR 0428 §4): it declares the Go toolchain (ADR 0238 §4, read
|
||||
// among a script's first lines by the build seat), and runs the unit tests, then with -tags integration the
|
||||
// integration tests, of every Go module in the tree; on a tree with none it says so and passes.
|
||||
const MergeCheck = `#!/bin/sh
|
||||
# mesh-check-toolchain: go
|
||||
#
|
||||
# This repository's own check, mesh/repo-check (novox/hq ADR 0238 §4), written with its first commit (ADR 0428 §4):
|
||||
# every Go module in the tree is vetted, its unit tests run, and then its integration tests (-tags integration).
|
||||
# A tree with no Go module has nothing to test, and the check says so and passes.
|
||||
set -eu
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
modules=$(find . -name go.mod -not -path '*/vendor/*' -not -path './.git/*' | sort)
|
||||
if [ -z "$modules" ]; then
|
||||
echo "This tree holds no Go module, so there are no tests to run; the check passes."
|
||||
exit 0
|
||||
fi
|
||||
for mod in $modules; do
|
||||
dir=$(dirname "$mod")
|
||||
echo "== $dir: go vet and the unit tests"
|
||||
(cd "$dir" && go vet ./... && go test ./...)
|
||||
echo "== $dir: the integration tests (-tags integration)"
|
||||
(cd "$dir" && go vet -tags integration ./... && go test -tags integration ./...)
|
||||
done
|
||||
`
|
||||
|
||||
// FirstCommit is main's first commit of a listed repository: its README, naming the repository's purpose, and
|
||||
// its own check.
|
||||
func FirstCommit(name string, r Repo) map[string]string {
|
||||
purpose := strings.TrimSpace(r.Description)
|
||||
if purpose == "" {
|
||||
purpose = "(its purpose is not described yet: the gitea module's config value `repositories` describes it)"
|
||||
}
|
||||
readme := "# " + name + "\n\n" + purpose + "\n\n" +
|
||||
"Made by the mesh from the gitea module's config value `repositories` (novox/hq ADR 0428). Main takes no push: " +
|
||||
"a change reaches it by a pull request whose `mesh/repo-check` (this repository's `merge-check.sh`) and " +
|
||||
"`mesh/merge-gate` passed.\n"
|
||||
return map[string]string{"README.md": readme, "merge-check.sh": MergeCheck}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
// repositories: the organisation's repositories from the gitea module's config value `repositories` (novox/hq
|
||||
// ADR 0428, issue 1124). `repositories sync` is a run-once process the node-engine runs when the module starts and
|
||||
// again whenever the value changes: it reads the file the mesh renders from that value and, with the setup login,
|
||||
// makes every listed repository that does not exist under that login's own account, protects its main, writes
|
||||
// main's first commit, narrows main's rule and only then moves it into the organisation; one that exists only has
|
||||
// its main rule put right. A repository the value does not list is never touched. It keeps what it did, per
|
||||
// repository, in the outcome file, says each change on stdout in one line and never a secret, and exits 1 when a
|
||||
// step was refused, its last stderr lines naming each (outcome.go Report): the node-engine's gate on a run-once
|
||||
// step (ADR 0136) reports it and runs it again on every apply until it passes.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) != 2 || os.Args[1] != "sync" {
|
||||
fmt.Fprintln(os.Stderr, "usage: repositories sync")
|
||||
os.Exit(2)
|
||||
}
|
||||
os.Exit(runSync())
|
||||
}
|
||||
|
||||
func runSync() int {
|
||||
forge, missing := forgehttp.FromEnv()
|
||||
file, outcome := os.Getenv("MESH_REPOSITORIES_FILE"), os.Getenv("MESH_REPOSITORIES_OUTCOME")
|
||||
for k, v := range map[string]string{"MESH_REPOSITORIES_FILE": file, "MESH_REPOSITORIES_OUTCOME": outcome} {
|
||||
if v == "" {
|
||||
missing = append(missing, k)
|
||||
}
|
||||
}
|
||||
if forge.TokenFile != "" {
|
||||
// The tools' account cannot make or protect a repository, and must not (ADR 0428 §8).
|
||||
missing = append(missing, "the setup login (MESH_GITEA_TOKEN_FILE is set instead)")
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
fmt.Fprintf(os.Stderr, "repositories: %v not set by the manifest; nothing was changed\n", missing)
|
||||
return 1
|
||||
}
|
||||
toolsAccount := os.Getenv("MESH_GITEA_TOOLS_USER")
|
||||
if toolsAccount == "" {
|
||||
toolsAccount = forgehttp.ToolsAccount
|
||||
}
|
||||
s := &Syncer{Forge: forge, Org: "novox", Admin: forge.User, Tools: toolsAccount,
|
||||
Log: func(format string, a ...any) { fmt.Printf(format+"\n", a...) },
|
||||
WaitFor: 3 * time.Minute, Poll: 5 * time.Second}
|
||||
value, err := ReadValue(file)
|
||||
if err != nil {
|
||||
o := Outcome{At: time.Now().UTC(), Org: s.Org, Failed: err.Error()}
|
||||
_ = WriteOutcome(outcome, o)
|
||||
return Report(o, os.Stderr)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
o, _ := s.Sync(ctx, value) // a forge not reachable is in o.Failed
|
||||
werr := WriteOutcome(outcome, o)
|
||||
if werr != nil {
|
||||
fmt.Fprintln(os.Stderr, "repositories: the outcome could not be kept:", werr)
|
||||
}
|
||||
if code := Report(o, os.Stderr); code != 0 || werr == nil {
|
||||
return code
|
||||
}
|
||||
return 1
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// 1124/AC1: the manifest declares `repositories` under config as a trusted object value with an empty default, as
|
||||
// it declares agent-read-teams (ADR 0365 §3, ADR 0348), renders it into a file of its own, and runs this program
|
||||
// at the module's start and whenever that file changes, as a run-once process (1124/AC6: its failed exit is the
|
||||
// node-engine's gate, ADR 0136) declared last, so that gate stops nothing of the module after it.
|
||||
func TestManifestDeclaresTheValueAndItsTrigger(t *testing.T) {
|
||||
raw, err := os.ReadFile("../../module.json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m struct {
|
||||
Config map[string]map[string]any `json:"config"`
|
||||
Resources []map[string]any `json:"resources"`
|
||||
Build struct {
|
||||
Artifacts []map[string]any `json:"artifacts"`
|
||||
} `json:"build"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v := m.Config[ConfigKey]
|
||||
if v == nil || v["type"] != "object" || v["trusted"] != true {
|
||||
t.Fatalf("config %s = %v, want a trusted object", ConfigKey, v)
|
||||
}
|
||||
if d, ok := v["default"].(map[string]any); !ok || len(d) != 0 {
|
||||
t.Fatalf("its default is %v, want {}", v["default"])
|
||||
}
|
||||
var file, process map[string]any
|
||||
for _, r := range m.Resources {
|
||||
for _, k := range strs(r["takes-config"]) {
|
||||
if k == ConfigKey {
|
||||
file = r
|
||||
}
|
||||
}
|
||||
}
|
||||
if file == nil || file["type"] != "file" {
|
||||
t.Fatalf("no file takes the value %s", ConfigKey)
|
||||
}
|
||||
for _, r := range m.Resources {
|
||||
if r["type"] == "process" && has(strs(r["restart-on"]), file["id"].(string)) {
|
||||
process = r
|
||||
}
|
||||
}
|
||||
if process == nil {
|
||||
t.Fatalf("no process runs again when %s changes", file["id"])
|
||||
}
|
||||
if process["run-once"] != true {
|
||||
t.Errorf("the process is not run-once: %v", process)
|
||||
}
|
||||
if last := m.Resources[len(m.Resources)-1]; last["id"] != process["id"] {
|
||||
t.Errorf("the last resource is %v, want %v: a refused step gates whatever of the module comes after it", last["id"], process["id"])
|
||||
}
|
||||
if process["health"] != nil {
|
||||
t.Errorf("the process declares a health check: %v", process["health"])
|
||||
}
|
||||
if strings.Join(strs(process["run"]), " ") != "./repositories sync" || process["artifact"] != "repositories" {
|
||||
t.Errorf("the process runs %v from %v", process["run"], process["artifact"])
|
||||
}
|
||||
env, _ := process["env"].(map[string]any)
|
||||
for _, k := range []string{"MESH_GITEA_URL", "MESH_GITEA_ADMIN_USER", "MESH_GITEA_ADMIN_PASSWORD_FILE", "MESH_REPOSITORIES_FILE", "MESH_REPOSITORIES_OUTCOME"} {
|
||||
if env[k] == nil {
|
||||
t.Errorf("the process is not given %s", k)
|
||||
}
|
||||
}
|
||||
found := false
|
||||
for _, a := range m.Build.Artifacts {
|
||||
if a["name"] == "repositories" && a["from"] == "cmd/repositories" && a["binary"] == "repositories" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("no bundle builds cmd/repositories")
|
||||
}
|
||||
// Like agent-read-teams: the same kind of value, the same way of applying it.
|
||||
teams := m.Config["agent-read-teams"]
|
||||
if teams["trusted"] != v["trusted"] || teams["type"] != v["type"] {
|
||||
t.Errorf("repositories is declared unlike agent-read-teams: %v vs %v", v, teams)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package main
|
||||
|
||||
// What a pass left (ADR 0428 §6): the outcome file, and the pass's exit. A refused step is not judged by a health
|
||||
// check of its own: the sync is a run-once process, so its non-zero exit is the node-engine's gate (ADR 0136). The
|
||||
// module's step is reported failed, apply-failing is raised, and every apply runs it again until it passes, which is
|
||||
// the "next pass" of §6. Its last stderr lines name each refused repository, its step and the forge's answer.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// WriteOutcome keeps what the pass did per repository, whole or not at all.
|
||||
func WriteOutcome(file string, o Outcome) error {
|
||||
raw, err := json.MarshalIndent(o, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tmp, err := os.CreateTemp(filepath.Dir(file), ".repositories-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(tmp.Name())
|
||||
if _, err := tmp.Write(append(raw, '\n')); err != nil {
|
||||
tmp.Close()
|
||||
return err
|
||||
}
|
||||
if err := tmp.Chmod(0o644); err != nil {
|
||||
tmp.Close()
|
||||
return err
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp.Name(), file)
|
||||
}
|
||||
|
||||
// ReadOutcome reads what the last pass left.
|
||||
func ReadOutcome(file string) (*Outcome, error) {
|
||||
raw, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var o Outcome
|
||||
if err := json.Unmarshal(raw, &o); err != nil {
|
||||
return nil, fmt.Errorf("the outcome %s is not readable: %w", file, err)
|
||||
}
|
||||
return &o, nil
|
||||
}
|
||||
|
||||
// Report says on stderr what the pass refused and is the pass's exit: 0 when every listed repository was made, put
|
||||
// right or already right; 1 otherwise. One line per refused repository, "repositories: <name>: <step> refused:
|
||||
// <forge answer>", then a summary line, so the line the node-engine keeps last names what to look at.
|
||||
func Report(o Outcome, stderr io.Writer) int {
|
||||
if o.Failed != "" {
|
||||
fmt.Fprintf(stderr, "repositories: the pass did not run: %s; nothing was made\n", o.Failed)
|
||||
return 1
|
||||
}
|
||||
refused := o.Refused()
|
||||
if len(refused) == 0 {
|
||||
return 0
|
||||
}
|
||||
for _, r := range refused {
|
||||
fmt.Fprintf(stderr, "repositories: %s: %s refused: %s\n", r.Name, r.Step, r.Said)
|
||||
}
|
||||
if len(refused) > 1 {
|
||||
names := make([]string, 0, len(refused))
|
||||
for _, r := range refused {
|
||||
names = append(names, r.Name)
|
||||
}
|
||||
fmt.Fprintf(stderr, "repositories: %d of %d repositories had a step refused (%s); nothing further of them was done, the rest were\n",
|
||||
len(refused), len(o.Repositories), strings.Join(names, ", "))
|
||||
}
|
||||
return 1
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gitea/internal/forgehttp"
|
||||
)
|
||||
|
||||
const (
|
||||
// ConfigKey is the module's config value this program applies (ADR 0428 §1).
|
||||
ConfigKey = "repositories"
|
||||
// MainBranch is the branch whose rule takes no push and lets only the tools' account merge (ADR 0398 §2,
|
||||
// protection.ts MAIN_BRANCH).
|
||||
MainBranch = "main"
|
||||
)
|
||||
|
||||
// RequiredStatuses are what main requires in every listed repository from its first commit (ADR 0428 §5): the
|
||||
// repository's own check and the module graph's gate. A status already on the rule is kept beside them.
|
||||
var RequiredStatuses = []string{"mesh/merge-gate", "mesh/repo-check"}
|
||||
|
||||
// The steps of one repository, as an outcome names a refused one (ADR 0428 §6).
|
||||
const (
|
||||
StepRead = "read"
|
||||
StepCreate = "create"
|
||||
StepAdminRule = "admin-rule"
|
||||
StepFirstCommit = "first-commit"
|
||||
StepNarrow = "narrow"
|
||||
StepMove = "move"
|
||||
StepPutRight = "put-right"
|
||||
// StepRuleAfterMove reads main's rule back once the repository is in the organisation and puts it right: the
|
||||
// forge drops from a merge whitelist a user without access, and under the setup account the tools' have none.
|
||||
StepRuleAfterMove = "rule-after-move"
|
||||
)
|
||||
|
||||
// What a pass did with one repository.
|
||||
const (
|
||||
DidMade = "made"
|
||||
DidPutRight = "put right"
|
||||
DidUnchanged = "unchanged"
|
||||
DidRefused = "refused"
|
||||
)
|
||||
|
||||
// repoName is a repository name the forge takes and that is never another path.
|
||||
var repoName = regexp.MustCompile(`^[A-Za-z0-9_][A-Za-z0-9_.-]{0,99}$`)
|
||||
|
||||
// Repo is one listed repository: whether it is private, and its description, which its README names as its purpose.
|
||||
type Repo struct {
|
||||
Private bool `json:"private"`
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
// Value is the config value: each repository of the organisation by name.
|
||||
type Value map[string]Repo
|
||||
|
||||
// ReadValue reads the file the mesh renders from the config value. A file without the key is the empty value,
|
||||
// which makes nothing and touches nothing.
|
||||
func ReadValue(file string) (Value, error) {
|
||||
raw, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("the repositories file cannot be read (%s): %w", file, err)
|
||||
}
|
||||
var whole map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &whole); err != nil {
|
||||
return nil, fmt.Errorf("the repositories file %s is not JSON: %w", file, err)
|
||||
}
|
||||
v := Value{}
|
||||
if r, ok := whole[ConfigKey]; ok {
|
||||
dec := json.NewDecoder(strings.NewReader(string(r)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&v); err != nil {
|
||||
return nil, fmt.Errorf("%s in %s is not {name: {private, description}}: %w", ConfigKey, file, err)
|
||||
}
|
||||
}
|
||||
for name := range v {
|
||||
if !repoName.MatchString(name) || strings.Contains(name, "..") || strings.HasSuffix(name, ".git") {
|
||||
return nil, fmt.Errorf("%s in %s lists %q, which is not a repository name", ConfigKey, file, name)
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// Result is what a pass did with one repository; a refusal names its step and the forge's answer.
|
||||
type Result struct {
|
||||
Name string `json:"name"`
|
||||
Did string `json:"did"`
|
||||
Step string `json:"step,omitempty"`
|
||||
Said string `json:"said,omitempty"`
|
||||
}
|
||||
|
||||
// Outcome is what a pass left, kept in the outcome file and said on stderr (outcome.go).
|
||||
type Outcome struct {
|
||||
At time.Time `json:"at"`
|
||||
Org string `json:"org"`
|
||||
Failed string `json:"failed,omitempty"` // the pass did not run: why
|
||||
Repositories []Result `json:"repositories"`
|
||||
}
|
||||
|
||||
// Refused names the repositories a step of was refused.
|
||||
func (o Outcome) Refused() []Result {
|
||||
var out []Result
|
||||
for _, r := range o.Repositories {
|
||||
if r.Did == DidRefused {
|
||||
out = append(out, r)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Syncer makes every listed repository of the organisation, with the setup login, and puts main's rule right in
|
||||
// those that exist. A repository the value does not list is never asked about (ADR 0428 §7).
|
||||
type Syncer struct {
|
||||
Forge forgehttp.Client // the setup login's
|
||||
Org string // the organisation, novox
|
||||
Admin string // the setup login, under whose own account a repository is made
|
||||
Tools string // the tools' account, the only one that merges into main
|
||||
Log func(format string, a ...any)
|
||||
// WaitFor is how long to wait for the forge to answer; none when zero.
|
||||
WaitFor, Poll time.Duration
|
||||
}
|
||||
|
||||
func (s *Syncer) say(format string, a ...any) {
|
||||
if s.Log != nil {
|
||||
s.Log(format, a...)
|
||||
}
|
||||
}
|
||||
|
||||
// Sync applies the value. The error is only the forge not being reachable; a refused step is in the outcome.
|
||||
func (s *Syncer) Sync(ctx context.Context, v Value) (Outcome, error) {
|
||||
out := Outcome{At: time.Now().UTC(), Org: s.Org, Repositories: []Result{}}
|
||||
if err := s.waitForForge(ctx); err != nil {
|
||||
out.Failed = err.Error()
|
||||
return out, err
|
||||
}
|
||||
names := make([]string, 0, len(v))
|
||||
for n := range v {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, n := range names {
|
||||
r := s.one(ctx, n, v[n])
|
||||
if r.Did == DidRefused {
|
||||
s.say("refused: %s/%s at the step %s: %s", s.Org, n, r.Step, r.Said)
|
||||
} else if r.Did != DidUnchanged {
|
||||
s.say("%s: %s/%s", r.Did, s.Org, n)
|
||||
}
|
||||
out.Repositories = append(out.Repositories, r)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// waitForForge is cmd/agent-teams' (teams.go waitForForge): the forge starting with the module answers late.
|
||||
func (s *Syncer) waitForForge(ctx context.Context) error {
|
||||
deadline := time.Now().Add(s.WaitFor)
|
||||
for {
|
||||
err := s.Forge.GetJSON(ctx, "/api/v1/version", nil)
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
var se *forgehttp.StatusError
|
||||
if errors.As(err, &se) || time.Now().After(deadline) || ctx.Err() != nil {
|
||||
return fmt.Errorf("the forge is not reachable: %w", err)
|
||||
}
|
||||
time.Sleep(s.Poll)
|
||||
}
|
||||
}
|
||||
|
||||
func refused(name, step string, err error) Result {
|
||||
return Result{Name: name, Did: DidRefused, Step: step, Said: err.Error()}
|
||||
}
|
||||
|
||||
func repoAPI(owner, name string) string {
|
||||
return "/api/v1/repos/" + url.PathEscape(owner) + "/" + url.PathEscape(name)
|
||||
}
|
||||
|
||||
// exists says whether owner/name is on the forge.
|
||||
func (s *Syncer) exists(ctx context.Context, owner, name string) (bool, error) {
|
||||
err := s.Forge.GetJSON(ctx, repoAPI(owner, name), nil)
|
||||
if errors.Is(err, forgehttp.ErrNotFound) {
|
||||
return false, nil
|
||||
}
|
||||
return err == nil, err
|
||||
}
|
||||
|
||||
// one is one listed repository (ADR 0428 §2, §3): put right when it is in the organisation; else made, or finished
|
||||
// when found under the setup account, and moved in only once main's rule is narrowed. A refused step stops it.
|
||||
func (s *Syncer) one(ctx context.Context, name string, r Repo) Result {
|
||||
inOrg, err := s.exists(ctx, s.Org, name)
|
||||
if err != nil {
|
||||
return refused(name, StepRead, err)
|
||||
}
|
||||
if inOrg {
|
||||
changed, err := s.putRight(ctx, s.Org, name)
|
||||
if err != nil {
|
||||
return refused(name, StepPutRight, err)
|
||||
}
|
||||
if changed {
|
||||
return Result{Name: name, Did: DidPutRight}
|
||||
}
|
||||
return Result{Name: name, Did: DidUnchanged}
|
||||
}
|
||||
held, err := s.exists(ctx, s.Admin, name)
|
||||
if err != nil {
|
||||
return refused(name, StepRead, err)
|
||||
}
|
||||
if !held {
|
||||
// Empty, so main exists only once the rule below is on it (ADR 0428 §3).
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPost, "/api/v1/user/repos", map[string]any{
|
||||
"name": name, "private": r.Private, "description": r.Description, "auto_init": false, "default_branch": MainBranch,
|
||||
}, nil); err != nil {
|
||||
return refused(name, StepCreate, err)
|
||||
}
|
||||
}
|
||||
rules, err := s.rules(ctx, s.Admin, name)
|
||||
if err != nil {
|
||||
return refused(name, StepRead, err)
|
||||
}
|
||||
rule := ruleNamed(rules, MainBranch)
|
||||
if rule == nil {
|
||||
body := map[string]any{"rule_name": MainBranch, "enable_push": true, "enable_push_whitelist": true,
|
||||
"push_whitelist_usernames": []string{s.Admin}}
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPost, repoAPI(s.Admin, name)+"/branch_protections", body, nil); err != nil {
|
||||
return refused(name, StepAdminRule, err)
|
||||
}
|
||||
}
|
||||
hasMain, err := s.branchExists(ctx, s.Admin, name, MainBranch)
|
||||
if err != nil {
|
||||
return refused(name, StepRead, err)
|
||||
}
|
||||
if !hasMain {
|
||||
if err := s.firstCommit(ctx, name, r); err != nil {
|
||||
return refused(name, StepFirstCommit, err)
|
||||
}
|
||||
}
|
||||
if rule == nil || !isRight(rule, s.Tools) {
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPatch, repoAPI(s.Admin, name)+"/branch_protections/"+MainBranch,
|
||||
narrowed(rule, s.Tools), nil); err != nil {
|
||||
return refused(name, StepNarrow, err)
|
||||
}
|
||||
}
|
||||
if err := s.Forge.SendJSON(ctx, http.MethodPost, repoAPI(s.Admin, name)+"/transfer",
|
||||
map[string]any{"new_owner": s.Org}, nil); err != nil {
|
||||
return refused(name, StepMove, err)
|
||||
}
|
||||
if _, err := s.putRight(ctx, s.Org, name); err != nil {
|
||||
return refused(name, StepRuleAfterMove, err)
|
||||
}
|
||||
return Result{Name: name, Did: DidMade}
|
||||
}
|
||||
|
||||
// firstCommit writes main's first commit: the README and the repository's own check (ADR 0428 §4).
|
||||
func (s *Syncer) firstCommit(ctx context.Context, name string, r Repo) error {
|
||||
files := FirstCommit(name, r)
|
||||
paths := make([]string, 0, len(files))
|
||||
for p := range files {
|
||||
paths = append(paths, p)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
var ops []map[string]any
|
||||
for _, p := range paths {
|
||||
ops = append(ops, map[string]any{"operation": "create", "path": p,
|
||||
"content": base64.StdEncoding.EncodeToString([]byte(files[p]))})
|
||||
}
|
||||
return s.Forge.SendJSON(ctx, http.MethodPost, repoAPI(s.Admin, name)+"/contents", map[string]any{
|
||||
"branch": MainBranch, "message": "Begin " + name + " with its README and its own check (ADR 0428)", "files": ops,
|
||||
}, nil)
|
||||
}
|
||||
|
||||
// rule is a branch protection rule as the forge answers it; only the fields read here are typed.
|
||||
type rule map[string]any
|
||||
|
||||
func ruleNamed(rules []rule, name string) rule {
|
||||
for _, r := range rules {
|
||||
if ruleName(r) == name {
|
||||
return r
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func ruleName(r rule) string {
|
||||
if n, _ := r["rule_name"].(string); n != "" {
|
||||
return n
|
||||
}
|
||||
n, _ := r["branch_name"].(string)
|
||||
return n
|
||||
}
|
||||
|
||||
func strList(v any) []string {
|
||||
var out []string
|
||||
if l, ok := v.([]any); ok {
|
||||
for _, s := range l {
|
||||
if t, ok := s.(string); ok && strings.TrimSpace(t) != "" {
|
||||
out = append(out, t)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// statuses is what a rule requires, with RequiredStatuses added and none dropped.
|
||||
func statuses(rules ...rule) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
add := func(s string) {
|
||||
if !seen[s] {
|
||||
seen[s] = true
|
||||
out = append(out, s)
|
||||
}
|
||||
}
|
||||
for _, r := range rules {
|
||||
if r != nil && r["enable_status_check"] == true {
|
||||
for _, s := range strList(r["status_check_contexts"]) {
|
||||
add(s)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, s := range RequiredStatuses {
|
||||
add(s)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// narrowed is main's rule of ADR 0398 §2 (protection.ts mainRuleBody): no push, only the tools' account merges,
|
||||
// requiring the statuses (ADR 0428 §5). A field the forge keeps beside these is left as it is.
|
||||
func narrowed(r rule, toolsAccount string, also ...rule) map[string]any {
|
||||
return map[string]any{
|
||||
"enable_push": false, "enable_push_whitelist": false, "push_whitelist_usernames": []string{},
|
||||
"enable_merge_whitelist": true, "merge_whitelist_usernames": []string{toolsAccount},
|
||||
"enable_status_check": true, "status_check_contexts": statuses(append([]rule{r}, also...)...),
|
||||
}
|
||||
}
|
||||
|
||||
// isRight says whether a rule already is narrowed and requires both statuses.
|
||||
func isRight(r rule, toolsAccount string) bool {
|
||||
if r["enable_push"] != false || r["enable_push_whitelist"] == true || r["enable_merge_whitelist"] != true {
|
||||
return false
|
||||
}
|
||||
if m := strList(r["merge_whitelist_usernames"]); len(m) != 1 || m[0] != toolsAccount {
|
||||
return false
|
||||
}
|
||||
return r["enable_status_check"] == true && hasAll(strList(r["status_check_contexts"]))
|
||||
}
|
||||
|
||||
func hasAll(have []string) bool {
|
||||
for _, want := range RequiredStatuses {
|
||||
found := false
|
||||
for _, h := range have {
|
||||
found = found || h == want
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// putRight makes main's rule in an organisation's repository the narrowed one, keeping every status it requires.
|
||||
// A repository with no rule named main gets one, carrying what a glob rule covering main asks (client.ts
|
||||
// ensureMainMerges, issue 948). It never takes a status away and never unprotects (ADR 0428 §5, §7).
|
||||
func (s *Syncer) putRight(ctx context.Context, owner, name string) (bool, error) {
|
||||
rules, err := s.rules(ctx, owner, name)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if own := ruleNamed(rules, MainBranch); own != nil {
|
||||
if isRight(own, s.Tools) {
|
||||
return false, nil
|
||||
}
|
||||
return true, s.Forge.SendJSON(ctx, http.MethodPatch, repoAPI(owner, name)+"/branch_protections/"+MainBranch,
|
||||
narrowed(own, s.Tools), nil)
|
||||
}
|
||||
var covering []rule
|
||||
for _, r := range rules {
|
||||
if ok, _ := path.Match(ruleName(r), MainBranch); ok {
|
||||
covering = append(covering, r)
|
||||
}
|
||||
}
|
||||
body := narrowed(nil, s.Tools, covering...)
|
||||
body["rule_name"] = MainBranch
|
||||
approvals := 0.0
|
||||
for _, r := range covering {
|
||||
if n, _ := r["required_approvals"].(float64); n > approvals {
|
||||
approvals = n
|
||||
}
|
||||
if r["block_admin_merge_override"] == true {
|
||||
body["block_admin_merge_override"] = true
|
||||
}
|
||||
}
|
||||
if approvals > 0 {
|
||||
body["required_approvals"] = approvals
|
||||
}
|
||||
return true, s.Forge.SendJSON(ctx, http.MethodPost, repoAPI(owner, name)+"/branch_protections", body, nil)
|
||||
}
|
||||
|
||||
func (s *Syncer) rules(ctx context.Context, owner, name string) ([]rule, error) {
|
||||
var out []rule
|
||||
if err := s.Forge.GetJSON(ctx, repoAPI(owner, name)+"/branch_protections", &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *Syncer) branchExists(ctx context.Context, owner, name, branch string) (bool, error) {
|
||||
err := s.Forge.GetJSON(ctx, repoAPI(owner, name)+"/branches/"+url.PathEscape(branch), nil)
|
||||
if errors.Is(err, forgehttp.ErrNotFound) {
|
||||
return false, nil
|
||||
}
|
||||
return err == nil, err
|
||||
}
|
||||
@@ -0,0 +1,480 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The tests of issue 1124 / ADR 0428: the organisation's repositories are the gitea module's config value
|
||||
// `repositories`, made and protected by the setup login, never by a tool.
|
||||
|
||||
const mainRule = "main"
|
||||
|
||||
// 1124/AC2: a listed repository that does not exist is made under the setup account, given main's rule letting that
|
||||
// login alone push, given its first commit, narrowed to no push and the tools' merge requiring mesh/repo-check (and
|
||||
// mesh/merge-gate, 1124/AC5), and only then moved into the organisation. Nothing reaches the organisation before the
|
||||
// narrowed rule, and nothing else is pushed to main.
|
||||
func TestNewRepositoryIsMadeInOrder(t *testing.T) {
|
||||
f := newFake()
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"mesh-delivery": {Private: false, Description: "the mesh's delivery"}})
|
||||
|
||||
want := []string{
|
||||
"POST /api/v1/user/repos",
|
||||
"POST /api/v1/repos/mesh-admin/mesh-delivery/branch_protections",
|
||||
"POST /api/v1/repos/mesh-admin/mesh-delivery/contents",
|
||||
"PATCH /api/v1/repos/mesh-admin/mesh-delivery/branch_protections/main",
|
||||
"POST /api/v1/repos/mesh-admin/mesh-delivery/transfer",
|
||||
"PATCH /api/v1/repos/novox/mesh-delivery/branch_protections/main",
|
||||
}
|
||||
if got := f.writes(); strings.Join(got, "\n") != strings.Join(want, "\n") {
|
||||
t.Fatalf("writes, in order:\n%s\nwant:\n%s", strings.Join(got, "\n"), strings.Join(want, "\n"))
|
||||
}
|
||||
// Nothing names the organisation's copy before the narrowed rule but the read that found it missing.
|
||||
narrowed := false
|
||||
for _, c := range f.all() {
|
||||
if strings.HasPrefix(c, "PATCH ") {
|
||||
narrowed = true
|
||||
}
|
||||
if strings.Contains(c, "/repos/novox/mesh-delivery") && !narrowed && c != "GET /api/v1/repos/novox/mesh-delivery" {
|
||||
t.Errorf("%s before the narrowed rule", c)
|
||||
}
|
||||
}
|
||||
repo := f.repos["novox/mesh-delivery"]
|
||||
if repo == nil || f.repos["mesh-admin/mesh-delivery"] != nil {
|
||||
t.Fatalf("not moved into the organisation: %v", f.repos)
|
||||
}
|
||||
if repo.Private || repo.Description != "the mesh's delivery" {
|
||||
t.Errorf("made as %+v, want public with its description", repo)
|
||||
}
|
||||
if repo.Commits != 1 || len(repo.Files) != 2 || repo.Files["README.md"] == "" || repo.Files["merge-check.sh"] == "" {
|
||||
t.Errorf("first commit: %d commits, files %v", repo.Commits, keysOf(repo.Files))
|
||||
}
|
||||
if !strings.Contains(repo.Files["README.md"], "the mesh's delivery") {
|
||||
t.Errorf("the README does not name its purpose: %q", repo.Files["README.md"])
|
||||
}
|
||||
rule := repo.Rules[mainRule]
|
||||
if rule["enable_push"] != false || rule["enable_push_whitelist"] == true || len(strs(rule["push_whitelist_usernames"])) != 0 {
|
||||
t.Errorf("main still takes a push: %v", rule)
|
||||
}
|
||||
if rule["enable_merge_whitelist"] != true || strings.Join(strs(rule["merge_whitelist_usernames"]), ",") != tools {
|
||||
t.Errorf("main's merge whitelist is not the tools' account alone: %v", rule)
|
||||
}
|
||||
checks := strs(rule["status_check_contexts"])
|
||||
if rule["enable_status_check"] != true || !has(checks, "mesh/repo-check") || !has(checks, "mesh/merge-gate") {
|
||||
t.Errorf("main does not require mesh/repo-check and mesh/merge-gate (1124/AC5): %v", rule)
|
||||
}
|
||||
if r := result(o, "mesh-delivery"); r.Did != DidMade || r.Step != "" {
|
||||
t.Errorf("outcome %+v, want made", r)
|
||||
}
|
||||
|
||||
// A second pass changes nothing.
|
||||
before := len(f.writes())
|
||||
o = run(t, s, Value{"mesh-delivery": {Private: false, Description: "the mesh's delivery"}})
|
||||
if got := f.writes(); len(got) != before {
|
||||
t.Errorf("a second pass wrote %v", got[before:])
|
||||
}
|
||||
if r := result(o, "mesh-delivery"); r.Did != DidUnchanged {
|
||||
t.Errorf("second outcome %+v, want unchanged", r)
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC2: the forge drops from a merge whitelist a user without access to the repository, and under the setup
|
||||
// account the tools' account has none; so once the repository is in the organisation its main rule is read back and
|
||||
// put right: the tools' account the only one that merges, both statuses required.
|
||||
func TestOrganisationRuleListsTheToolsAfterTheMove(t *testing.T) {
|
||||
f := newFake()
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"mesh-delivery": {}})
|
||||
repo := f.repos["novox/mesh-delivery"]
|
||||
if repo == nil {
|
||||
t.Fatal("not moved into the organisation")
|
||||
}
|
||||
rule := repo.Rules[mainRule]
|
||||
if rule["enable_merge_whitelist"] != true || strings.Join(strs(rule["merge_whitelist_usernames"]), ",") != tools {
|
||||
t.Errorf("after the move main's merge whitelist is %v, want %s alone", rule["merge_whitelist_usernames"], tools)
|
||||
}
|
||||
if rule["enable_push"] != false {
|
||||
t.Errorf("after the move main takes a push: %v", rule)
|
||||
}
|
||||
checks := strs(rule["status_check_contexts"])
|
||||
if rule["enable_status_check"] != true || !has(checks, "mesh/repo-check") || !has(checks, "mesh/merge-gate") {
|
||||
t.Errorf("after the move main does not require both statuses: %v", rule)
|
||||
}
|
||||
if r := result(o, "mesh-delivery"); r.Did != DidMade {
|
||||
t.Errorf("outcome %+v, want made", r)
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC2: the setup login's rule lets that login alone push, and it is what the first commit lands through.
|
||||
func TestAdminRuleLetsTheSetupLoginAlonePush(t *testing.T) {
|
||||
f := newFake()
|
||||
var seen map[string]any
|
||||
f.refuse = func(method, path string) (int, string) {
|
||||
if method == "POST" && strings.HasSuffix(path, "/contents") {
|
||||
seen = map[string]any{}
|
||||
for k, v := range f.repos["mesh-admin/x"].Rules[mainRule] {
|
||||
seen[k] = v
|
||||
}
|
||||
}
|
||||
return 0, ""
|
||||
}
|
||||
s, _ := syncer(t, f)
|
||||
run(t, s, Value{"x": {Private: true}})
|
||||
if seen["enable_push"] != true || seen["enable_push_whitelist"] != true || strings.Join(strs(seen["push_whitelist_usernames"]), ",") != admin {
|
||||
t.Fatalf("the rule at the first commit was %v, want a push by %s alone", seen, admin)
|
||||
}
|
||||
if !f.repos["novox/x"].Private {
|
||||
t.Error("made public, want private")
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC3: a listed repository that exists in the organisation is not made again, only its main rule put right;
|
||||
// a rule still letting the setup login push is narrowed, its statuses kept (1124/AC5).
|
||||
func TestExistingOrganisationRepositoryIsOnlyPutRight(t *testing.T) {
|
||||
f := newFake()
|
||||
hq := f.add("novox/hq", map[string]map[string]any{mainRule: {
|
||||
"enable_push": true, "enable_push_whitelist": true, "push_whitelist_usernames": []any{admin},
|
||||
"enable_status_check": true, "status_check_contexts": []any{"other/x", "mesh/merge-gate"},
|
||||
}}, "main")
|
||||
bare := f.add("novox/bare", nil, "main")
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"hq": {}, "bare": {}})
|
||||
|
||||
for _, c := range f.writes() {
|
||||
if strings.HasSuffix(c, "/user/repos") || strings.HasSuffix(c, "/contents") || strings.HasSuffix(c, "/transfer") {
|
||||
t.Errorf("an existing repository got %s", c)
|
||||
}
|
||||
}
|
||||
rule := hq.Rules[mainRule]
|
||||
if rule["enable_push"] != false || strings.Join(strs(rule["merge_whitelist_usernames"]), ",") != tools {
|
||||
t.Errorf("hq's rule not narrowed: %v", rule)
|
||||
}
|
||||
checks := strs(rule["status_check_contexts"])
|
||||
for _, c := range []string{"other/x", "mesh/merge-gate", "mesh/repo-check"} {
|
||||
if !has(checks, c) {
|
||||
t.Errorf("hq's rule lacks %s: %v", c, checks)
|
||||
}
|
||||
}
|
||||
made := bare.Rules[mainRule]
|
||||
if made == nil || made["enable_push"] != false || !has(strs(made["status_check_contexts"]), "mesh/repo-check") ||
|
||||
!has(strs(made["status_check_contexts"]), "mesh/merge-gate") {
|
||||
t.Errorf("bare's rule: %v", made)
|
||||
}
|
||||
if hq.Commits != 0 || bare.Commits != 0 {
|
||||
t.Error("something was pushed to an existing main")
|
||||
}
|
||||
if r := result(o, "hq"); r.Did != DidPutRight {
|
||||
t.Errorf("hq outcome %+v", r)
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC3: a repository found still under the setup account is finished: with main already written it is not
|
||||
// written again; without a rule it gets the steps it lacks.
|
||||
func TestRepositoryUnderTheSetupAccountIsFinished(t *testing.T) {
|
||||
f := newFake()
|
||||
f.add("mesh-admin/half", map[string]map[string]any{mainRule: {
|
||||
"enable_push": true, "enable_push_whitelist": true, "push_whitelist_usernames": []any{admin},
|
||||
}}, "main")
|
||||
f.add("mesh-admin/empty", nil)
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"half": {}, "empty": {}})
|
||||
|
||||
got := strings.Join(f.writes(), "\n")
|
||||
want := strings.Join([]string{
|
||||
"POST /api/v1/repos/mesh-admin/empty/branch_protections",
|
||||
"POST /api/v1/repos/mesh-admin/empty/contents",
|
||||
"PATCH /api/v1/repos/mesh-admin/empty/branch_protections/main",
|
||||
"POST /api/v1/repos/mesh-admin/empty/transfer",
|
||||
"PATCH /api/v1/repos/novox/empty/branch_protections/main",
|
||||
"PATCH /api/v1/repos/mesh-admin/half/branch_protections/main",
|
||||
"POST /api/v1/repos/mesh-admin/half/transfer",
|
||||
"PATCH /api/v1/repos/novox/half/branch_protections/main",
|
||||
}, "\n")
|
||||
if got != want {
|
||||
t.Fatalf("writes:\n%s\nwant:\n%s", got, want)
|
||||
}
|
||||
for _, n := range []string{"half", "empty"} {
|
||||
repo := f.repos["novox/"+n]
|
||||
if repo == nil || repo.Rules[mainRule]["enable_push"] != false {
|
||||
t.Errorf("%s not finished: %+v", n, repo)
|
||||
}
|
||||
if r := result(o, n); r.Did != DidMade {
|
||||
t.Errorf("%s outcome %+v", n, r)
|
||||
}
|
||||
}
|
||||
if f.repos["novox/half"].Commits != 0 {
|
||||
t.Error("main of half was written again")
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC5: putting a rule right adds a missing one of mesh/repo-check and mesh/merge-gate and keeps every status
|
||||
// already there; no pass drops one.
|
||||
func TestStatusesAreAddedAndNeverDropped(t *testing.T) {
|
||||
f := newFake()
|
||||
only := f.add("novox/only", map[string]map[string]any{mainRule: {
|
||||
"enable_push": false, "enable_merge_whitelist": true, "merge_whitelist_usernames": []any{tools},
|
||||
"enable_status_check": true, "status_check_contexts": []any{"mesh/repo-check"},
|
||||
}}, "main")
|
||||
more := f.add("novox/more", map[string]map[string]any{mainRule: {
|
||||
"enable_push": false, "enable_merge_whitelist": true, "merge_whitelist_usernames": []any{tools},
|
||||
"enable_status_check": true, "status_check_contexts": []any{"mesh/merge-gate", "mesh/repo-check", "mesh/delivery"},
|
||||
}}, "main")
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"only": {}, "more": {}})
|
||||
if c := strs(only.Rules[mainRule]["status_check_contexts"]); !has(c, "mesh/merge-gate") || !has(c, "mesh/repo-check") {
|
||||
t.Errorf("only: %v", c)
|
||||
}
|
||||
if c := strs(more.Rules[mainRule]["status_check_contexts"]); len(c) != 3 || !has(c, "mesh/delivery") {
|
||||
t.Errorf("more lost a status: %v", c)
|
||||
}
|
||||
for _, w := range f.writes() {
|
||||
if strings.Contains(w, "/more/") {
|
||||
t.Errorf("a right rule was written: %s", w)
|
||||
}
|
||||
}
|
||||
if result(o, "more").Did != DidUnchanged || result(o, "only").Did != DidPutRight {
|
||||
t.Errorf("outcome %+v", o)
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC6: a refused step is said with the repository, the step and the forge's answer, nothing further of that
|
||||
// repository is sent, and the next listed repository is still made; the pass exits 1, its stderr naming them.
|
||||
func TestRefusedStepStopsThatRepositoryOnly(t *testing.T) {
|
||||
steps := map[string]func(method, path string) bool{
|
||||
StepCreate: func(m, p string) bool { return m == "POST" && p == "/api/v1/user/repos" },
|
||||
StepAdminRule: func(m, p string) bool { return m == "POST" && strings.HasSuffix(p, "/a/branch_protections") },
|
||||
StepFirstCommit: func(m, p string) bool { return m == "POST" && strings.HasSuffix(p, "/a/contents") },
|
||||
StepNarrow: func(m, p string) bool { return m == "PATCH" && strings.Contains(p, "/a/branch_protections/") },
|
||||
StepMove: func(m, p string) bool { return m == "POST" && strings.HasSuffix(p, "/a/transfer") },
|
||||
StepRuleAfterMove: func(m, p string) bool {
|
||||
return m == "PATCH" && strings.HasPrefix(p, "/api/v1/repos/novox/a/branch_protections/")
|
||||
},
|
||||
}
|
||||
for step, match := range steps {
|
||||
t.Run(step, func(t *testing.T) {
|
||||
f := newFake()
|
||||
refused := false
|
||||
f.refuse = func(method, path string) (int, string) {
|
||||
if refused {
|
||||
return 0, ""
|
||||
}
|
||||
// The create of a is refused, not b's.
|
||||
if step == StepCreate && match(method, path) && !strings.Contains(strings.Join(f.calls, "\n"), "/b") {
|
||||
refused = true
|
||||
return 403, "the forge says no to " + step
|
||||
}
|
||||
if step != StepCreate && match(method, path) {
|
||||
refused = true
|
||||
return 403, "the forge says no to " + step
|
||||
}
|
||||
return 0, ""
|
||||
}
|
||||
s, _ := syncer(t, f)
|
||||
o := run(t, s, Value{"a": {}, "b": {}})
|
||||
r := result(o, "a")
|
||||
if r.Did != DidRefused || r.Step != step || !strings.Contains(r.Said, "the forge says no to "+step) {
|
||||
t.Fatalf("a: %+v, want refused at %s with the forge's answer", r, step)
|
||||
}
|
||||
// Nothing of a after the refused call.
|
||||
calls := f.all()
|
||||
at := -1
|
||||
for i, c := range calls {
|
||||
m := strings.SplitN(c, " ", 2)
|
||||
if match(m[0], m[1]) {
|
||||
at = i
|
||||
break
|
||||
}
|
||||
}
|
||||
for _, c := range calls[at+1:] {
|
||||
if strings.Contains(c, "/a/") || strings.HasSuffix(c, "/a") {
|
||||
t.Errorf("after the refused %s: %s", step, c)
|
||||
}
|
||||
}
|
||||
if f.repos["novox/b"] == nil || result(o, "b").Did != DidMade {
|
||||
t.Errorf("b was not made after a's refusal: %+v", result(o, "b"))
|
||||
}
|
||||
var stderr bytes.Buffer
|
||||
if code := Report(o, &stderr); code == 0 {
|
||||
t.Errorf("a refused %s exits 0", step)
|
||||
}
|
||||
lines := strings.Split(strings.TrimRight(stderr.String(), "\n"), "\n")
|
||||
if want := "repositories: a: " + step + " refused: "; len(lines) != 1 || !strings.HasPrefix(lines[0], want) ||
|
||||
!strings.Contains(lines[0], "the forge says no to "+step) {
|
||||
t.Errorf("stderr %q, want one line %q<the forge's answer>", stderr.String(), want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC7: a repository taken out of the value gets no call at all.
|
||||
func TestRepositoryOutOfTheValueIsNotTouched(t *testing.T) {
|
||||
f := newFake()
|
||||
f.add("novox/old", map[string]map[string]any{mainRule: {"enable_push": true}}, "main")
|
||||
f.add("mesh-admin/older", nil)
|
||||
s, _ := syncer(t, f)
|
||||
run(t, s, Value{})
|
||||
run(t, s, Value{"new": {}})
|
||||
for _, c := range f.all() {
|
||||
if strings.Contains(c, "old") {
|
||||
t.Errorf("a repository out of the value was asked: %s", c)
|
||||
}
|
||||
}
|
||||
if f.repos["novox/old"].Rules[mainRule]["enable_push"] != true || f.repos["mesh-admin/older"] == nil {
|
||||
t.Error("a repository out of the value was changed")
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC6: the pass's exit is the node-engine's gate on a run-once step (ADR 0136): 0 when every repository was
|
||||
// made, put right or unchanged and nothing on stderr; 1 with one line per refused repository, then a summary line
|
||||
// naming them all, when several were; 1 naming why when the pass did not run.
|
||||
func TestReportIsTheExitAndTheStderrLines(t *testing.T) {
|
||||
now := time.Now()
|
||||
var stderr bytes.Buffer
|
||||
ok := Outcome{At: now, Repositories: []Result{{Name: "hq", Did: DidUnchanged}, {Name: "x", Did: DidMade}, {Name: "y", Did: DidPutRight}}}
|
||||
if code := Report(ok, &stderr); code != 0 || stderr.Len() != 0 {
|
||||
t.Errorf("all made: exit %d, stderr %q", code, stderr.String())
|
||||
}
|
||||
|
||||
stderr.Reset()
|
||||
two := Outcome{At: now, Repositories: []Result{
|
||||
{Name: "a", Did: DidRefused, Step: StepNarrow, Said: "403 Forbidden: user does not have permission"},
|
||||
{Name: "b", Did: DidMade},
|
||||
{Name: "c", Did: DidRefused, Step: StepMove, Said: "422: cannot transfer"},
|
||||
}}
|
||||
if code := Report(two, &stderr); code != 1 {
|
||||
t.Errorf("two refused: exit %d", code)
|
||||
}
|
||||
lines := strings.Split(strings.TrimRight(stderr.String(), "\n"), "\n")
|
||||
want := []string{
|
||||
"repositories: a: narrow refused: 403 Forbidden: user does not have permission",
|
||||
"repositories: c: move refused: 422: cannot transfer",
|
||||
}
|
||||
if len(lines) != 3 || lines[0] != want[0] || lines[1] != want[1] {
|
||||
t.Fatalf("stderr:\n%s\nwant the lines:\n%s\nthen a summary", stderr.String(), strings.Join(want, "\n"))
|
||||
}
|
||||
if !strings.Contains(lines[2], "2 of 3") || !strings.Contains(lines[2], "a, c") {
|
||||
t.Errorf("summary %q, want the count and the names", lines[2])
|
||||
}
|
||||
|
||||
stderr.Reset()
|
||||
if code := Report(Outcome{At: now, Failed: "the forge is not reachable"}, &stderr); code != 1 ||
|
||||
!strings.Contains(stderr.String(), "the forge is not reachable") {
|
||||
t.Errorf("a pass that did not run: exit %d, stderr %q", code, stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC6: the outcome is written whole, and read back whole.
|
||||
func TestOutcomeRoundTrip(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "outcome.json")
|
||||
o := Outcome{At: time.Now().UTC().Truncate(time.Second), Repositories: []Result{{Name: "a", Did: DidRefused, Step: StepMove, Said: "no"}}}
|
||||
if err := WriteOutcome(p, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := ReadOutcome(p)
|
||||
if err != nil || got.Repositories[0] != o.Repositories[0] || !got.At.Equal(o.At) {
|
||||
t.Fatalf("%+v %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC1: the value is read from the file the mesh renders; a missing key is the empty value; an unknown field
|
||||
// or a name that is no repository name is refused.
|
||||
func TestReadValue(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "r.json")
|
||||
_ = os.WriteFile(p, []byte(`{"repositories":{"mesh-delivery":{"private":true,"description":"d"}}}`), 0o600)
|
||||
v, err := ReadValue(p)
|
||||
if err != nil || !v["mesh-delivery"].Private || v["mesh-delivery"].Description != "d" {
|
||||
t.Fatalf("%v %v", v, err)
|
||||
}
|
||||
_ = os.WriteFile(p, []byte(`{}`), 0o600)
|
||||
if v, err := ReadValue(p); err != nil || len(v) != 0 {
|
||||
t.Fatalf("%v %v", v, err)
|
||||
}
|
||||
_ = os.WriteFile(p, []byte(`{"repositories":{"a":{"owner":"novox"}}}`), 0o600)
|
||||
if _, err := ReadValue(p); err == nil {
|
||||
t.Fatal("an unknown field was accepted")
|
||||
}
|
||||
for _, bad := range []string{"..", "a/b", "", "a b", "-x"} {
|
||||
raw, _ := json.Marshal(map[string]any{"repositories": map[string]any{bad: map[string]any{}}})
|
||||
_ = os.WriteFile(p, raw, 0o600)
|
||||
if _, err := ReadValue(p); err == nil {
|
||||
t.Errorf("%q was accepted as a repository name", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1124/AC4: the first commit's merge-check.sh declares its toolchain in its first lines, runs the tests of a Go
|
||||
// module in the tree, and on a tree with none says so and passes.
|
||||
func TestFirstCommitMergeCheck(t *testing.T) {
|
||||
files := FirstCommit("mesh-delivery", Repo{Description: "the mesh's delivery"})
|
||||
script := files["merge-check.sh"]
|
||||
lines := strings.SplitN(script, "\n", 4)
|
||||
if lines[0] != "#!/bin/sh" || lines[1] != "# mesh-check-toolchain: go" {
|
||||
t.Fatalf("first lines %q", lines[:2])
|
||||
}
|
||||
if !strings.Contains(files["README.md"], "mesh-delivery") || !strings.Contains(files["README.md"], "the mesh's delivery") {
|
||||
t.Errorf("README %q", files["README.md"])
|
||||
}
|
||||
goBin, err := exec.LookPath("go")
|
||||
if err != nil {
|
||||
t.Skip("no go on PATH to run the script with")
|
||||
}
|
||||
|
||||
tree := t.TempDir()
|
||||
mod := filepath.Join(tree, "sub")
|
||||
_ = os.MkdirAll(mod, 0o755)
|
||||
_ = os.WriteFile(filepath.Join(tree, "merge-check.sh"), []byte(script), 0o755)
|
||||
_ = os.WriteFile(filepath.Join(mod, "go.mod"), []byte("module tiny\n\ngo 1.21\n"), 0o644)
|
||||
_ = os.WriteFile(filepath.Join(mod, "tiny_test.go"), []byte(`package tiny
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRan(t *testing.T) { _ = os.WriteFile("ran", []byte("yes"), 0o644) }
|
||||
`), 0o644)
|
||||
cmd := exec.Command("sh", "merge-check.sh")
|
||||
cmd.Dir = tree
|
||||
cmd.Env = append(os.Environ(), "GOFLAGS=", "GOTOOLCHAIN=local", "PATH="+filepath.Dir(goBin)+":"+os.Getenv("PATH"))
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("on a tree with a Go module: %v\n%s", err, out)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(mod, "ran")); err != nil || !strings.Contains(string(out), "integration") {
|
||||
t.Fatalf("its tests did not run, or not with -tags integration:\n%s", out)
|
||||
}
|
||||
|
||||
// A failing test fails the check.
|
||||
_ = os.WriteFile(filepath.Join(mod, "fail_test.go"), []byte("package tiny\n\nimport \"testing\"\n\nfunc TestFails(t *testing.T) { t.Fatal(\"no\") }\n"), 0o644)
|
||||
if out, err := cmd2(tree, cmd.Env).CombinedOutput(); err == nil {
|
||||
t.Fatalf("a failing test passed the check:\n%s", out)
|
||||
}
|
||||
|
||||
empty := t.TempDir()
|
||||
_ = os.WriteFile(filepath.Join(empty, "merge-check.sh"), []byte(script), 0o755)
|
||||
out, err = cmd2(empty, cmd.Env).CombinedOutput()
|
||||
if err != nil || !strings.Contains(string(out), "no Go module") {
|
||||
t.Fatalf("on an empty tree: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
func cmd2(dir string, env []string) *exec.Cmd {
|
||||
c := exec.Command("sh", "merge-check.sh")
|
||||
c.Dir, c.Env = dir, env
|
||||
return c
|
||||
}
|
||||
|
||||
func keysOf(m map[string]string) []string {
|
||||
out := []string{}
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{}
|
||||
@@ -169,6 +169,12 @@
|
||||
"description": "The read teams of the agents' forge accounts (ADR 0365 clause 3): each team by name, {\"<team>\": {\"repos\": [\"<owner>/<repo>\", ...], \"members\": [\"<node>\", ...]}}. The forge gets a team agent-read-<team> that reads only, with exactly those repositories and the accounts agent-<node>; a team taken out of the value is emptied. Nothing else on the forge is touched; an agent account is in no team until it is listed here.",
|
||||
"trusted": true,
|
||||
"default": {}
|
||||
},
|
||||
"repositories": {
|
||||
"type": "object",
|
||||
"description": "The repositories of the organisation novox the mesh holds (ADR 0428): each by name, {\"<name>\": {\"private\": true, \"description\": \"...\"}}. One that does not exist is made by the module's setup login under its own account, its main protected and given a first commit (a README naming the description and a merge-check.sh), its rule narrowed to no push and the tools' merge requiring mesh/repo-check and mesh/merge-gate, and only then moved into the organisation; one that exists only has main's rule put right, its statuses kept. A repository taken out of the value is never touched.",
|
||||
"trusted": true,
|
||||
"default": {}
|
||||
}
|
||||
},
|
||||
"facts": {
|
||||
@@ -311,6 +317,17 @@
|
||||
"agent-read-teams"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "repositories-file",
|
||||
"type": "file",
|
||||
"path": "${dir:mesh-state}/repositories.json",
|
||||
"mode": "0600",
|
||||
"template": "files/repositories.json",
|
||||
"merge": "json",
|
||||
"takes-config": [
|
||||
"repositories"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "consumers",
|
||||
"type": "file",
|
||||
@@ -336,6 +353,28 @@
|
||||
"restart-on": [
|
||||
"agent-teams-file"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "repositories",
|
||||
"type": "process",
|
||||
"name": "gitea-repositories",
|
||||
"artifact": "repositories",
|
||||
"run": [
|
||||
"./repositories",
|
||||
"sync"
|
||||
],
|
||||
"run-once": true,
|
||||
"env": {
|
||||
"MESH_GITEA_URL": "http://127.0.0.1:${port:3000}",
|
||||
"MESH_GITEA_ADMIN_USER": "mesh-admin",
|
||||
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
|
||||
"MESH_GITEA_TOOLS_USER": "mesh-tools",
|
||||
"MESH_REPOSITORIES_FILE": "${dir:mesh-state}/repositories.json",
|
||||
"MESH_REPOSITORIES_OUTCOME": "${dir:runtime-state}/repositories-outcome.json"
|
||||
},
|
||||
"restart-on": [
|
||||
"repositories-file"
|
||||
]
|
||||
}
|
||||
],
|
||||
"provides": [
|
||||
@@ -460,6 +499,17 @@
|
||||
"agent-teams"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "repositories",
|
||||
"kind": "bundle",
|
||||
"language": "go",
|
||||
"system": "arch",
|
||||
"from": "cmd/repositories",
|
||||
"binary": "repositories",
|
||||
"loads": [
|
||||
"repositories"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "forge-account-provider",
|
||||
"kind": "bundle",
|
||||
|
||||
@@ -1,117 +0,0 @@
|
||||
// A repository is created where it is asked for (novox/hq issue 1124): gitea_create_repo created under the tools'
|
||||
// own user alone, so a repository of the novox organisation could not be made through it. With `owner` it is created
|
||||
// in that organisation (POST /orgs/{org}/repos), the forge's refusal said with its status and its reason; without
|
||||
// it, under the tools' own user as before.
|
||||
|
||||
import { test, after, beforeEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
|
||||
import { GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken } from "../dist/token.js";
|
||||
import { getGiteaTools } from "../dist/tools/index.js";
|
||||
|
||||
interface Sent { method: string; path: string; body: any }
|
||||
|
||||
const state = {
|
||||
sent: [] as Sent[],
|
||||
// When set, what the forge answers a create in an organisation: its status and its message.
|
||||
refuse: null as { status: number; message: string } | null,
|
||||
};
|
||||
|
||||
function json(res: ServerResponse, status: number, body: unknown) {
|
||||
res.statusCode = status;
|
||||
res.setHeader("Content-Type", "application/json");
|
||||
res.end(JSON.stringify(body));
|
||||
}
|
||||
|
||||
const server = createServer(async (req: IncomingMessage, res: ServerResponse) => {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const c of req) chunks.push(c as Buffer);
|
||||
const raw = Buffer.concat(chunks).toString();
|
||||
const path = new URL(req.url ?? "", "http://x").pathname.replace(/^\/api\/v1/, "");
|
||||
const method = req.method ?? "GET";
|
||||
state.sent.push({ method, path, body: raw ? JSON.parse(raw) : undefined });
|
||||
const created = (owner: string) => {
|
||||
const body = raw ? JSON.parse(raw) : {};
|
||||
return json(res, 201, { full_name: `${owner}/${body.name}`, name: body.name, owner: { login: owner },
|
||||
private: body.private, description: body.description, html_url: `http://forge/${owner}/${body.name}`,
|
||||
clone_url: `http://forge/${owner}/${body.name}.git`, default_branch: "main" });
|
||||
};
|
||||
if (method === "POST" && path === "/user/repos") return created("agent");
|
||||
const org = /^\/orgs\/([^/]+)\/repos$/.exec(path);
|
||||
if (method === "POST" && org) {
|
||||
if (state.refuse) return json(res, state.refuse.status, { message: state.refuse.message, url: "http://forge/api/swagger" });
|
||||
return created(decodeURIComponent(org[1]));
|
||||
}
|
||||
return json(res, 404, { message: "not found" });
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
after(() => new Promise<void>((r) => server.close(() => r())));
|
||||
const { port } = server.address() as { port: number };
|
||||
const client = new GiteaClient(`http://127.0.0.1:${port}`, new ConfiguredToken("t"));
|
||||
const tool = (name: string) => getGiteaTools(client).find((t) => t.name === name)!;
|
||||
const body = { name: "mesh-delivery", description: "the delivery's own repository", private: true, auto_init: true };
|
||||
|
||||
beforeEach(() => {
|
||||
state.sent = [];
|
||||
state.refuse = null;
|
||||
});
|
||||
|
||||
test("with an organisation the repository is created there, with the same body, and answered (1124/AC1)", async () => {
|
||||
// Fails before the fix: createRepo takes no owner and posts to /user/repos, so the repository is the tools' user's.
|
||||
const repo = await client.createRepo({ ...body }, "novox");
|
||||
assert.deepEqual(state.sent, [{ method: "POST", path: "/orgs/novox/repos", body }]);
|
||||
assert.equal(repo.full_name, "novox/mesh-delivery");
|
||||
assert.equal(repo.owner, "novox");
|
||||
|
||||
// The tool passes its owner through; before the fix it has no owner argument and posts to /user/repos.
|
||||
state.sent = [];
|
||||
const answered = await tool("gitea_create_repo").run({ owner: "novox", ...body }) as { repo: { full_name: string } };
|
||||
assert.deepEqual(state.sent, [{ method: "POST", path: "/orgs/novox/repos", body }]);
|
||||
assert.equal(answered.repo.full_name, "novox/mesh-delivery");
|
||||
assert.match(tool("gitea_create_repo").description, /organisation/, "the description says where it creates");
|
||||
});
|
||||
|
||||
test("without an owner the repository is created under the tools' own user, as before (1124/AC2)", async () => {
|
||||
// Holds before the fix: it is the behaviour the fix must leave unchanged.
|
||||
const repo = await client.createRepo({ ...body });
|
||||
assert.deepEqual(state.sent, [{ method: "POST", path: "/user/repos", body }]);
|
||||
assert.equal(repo.full_name, "agent/mesh-delivery");
|
||||
|
||||
state.sent = [];
|
||||
await tool("gitea_create_repo").run({ name: "mesh-delivery" });
|
||||
assert.deepEqual(state.sent, [{ method: "POST", path: "/user/repos", body: { name: "mesh-delivery", private: true, auto_init: true } }]);
|
||||
});
|
||||
|
||||
test("the forge's refusal to create in an organisation is said with its status, its reason and the organisation (1124/AC3)", async () => {
|
||||
// Fails before the fix: the create goes to /user/repos, which the fake forge answers 201, so nothing is refused.
|
||||
for (const [status, message] of [
|
||||
[403, "user should be an owner or a member of the organisation"],
|
||||
[404, "GetOrgByName: org does not exist [name: novox]"],
|
||||
[409, "The repository with the same name already exists."],
|
||||
] as const) {
|
||||
state.refuse = { status, message };
|
||||
for (const run of [
|
||||
() => client.createRepo({ ...body }, "novox"),
|
||||
() => tool("gitea_create_repo").run({ owner: "novox", ...body }),
|
||||
]) {
|
||||
await assert.rejects(run(), (e: Error) => {
|
||||
assert.match(e.message, new RegExp(`\\b${status}\\b`), `the status ${status} is said`);
|
||||
assert.ok(e.message.includes(message), `the forge's reason is said: ${e.message}`);
|
||||
assert.match(e.message, /organisation novox/, "the organisation is named");
|
||||
return true;
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("an owner that is not a name is refused before anything is sent (1124/AC4)", async () => {
|
||||
// Holds before the fix too: namesChecked (tools/index.ts) checks args.owner for every tool. It stays so the guard
|
||||
// keeps covering this tool's new argument.
|
||||
for (const owner of ["..", "novox/../agent", "", "no vox", "%2e%2e", "-x"]) {
|
||||
await assert.rejects(tool("gitea_create_repo").run({ owner, name: "mesh-delivery" }), /not a repository owner/,
|
||||
JSON.stringify(owner));
|
||||
}
|
||||
assert.deepEqual(state.sent, [], "nothing reached the forge");
|
||||
});
|
||||
@@ -0,0 +1,56 @@
|
||||
// No tool and no agent creates a repository in an organisation (novox/hq issue 1124, ADR 0428 §8): the forge makes
|
||||
// whoever creates an organisation's repository its administrator, and an administrator can take its protection off.
|
||||
// So gitea_create_repo creates only under the tools' own user, with no owner argument; the tools' team keeps
|
||||
// can_create_org_repo false; and the tools' token has no write:organization. The organisation's repositories are
|
||||
// made by the module's setup login from its config value `repositories` (cmd/repositories).
|
||||
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||
|
||||
import { GiteaClient } from "../dist/client.js";
|
||||
import { ConfiguredToken, TOKEN_SCOPES } from "../dist/token.js";
|
||||
import { getGiteaTools } from "../dist/tools/index.js";
|
||||
import { TOOLS_TEAM_UNITS } from "../dist/tools-account.js";
|
||||
|
||||
const sent: { method: string; path: string; body: any }[] = [];
|
||||
|
||||
const server = createServer(async (req: IncomingMessage, res: ServerResponse) => {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const c of req) chunks.push(c as Buffer);
|
||||
const raw = Buffer.concat(chunks).toString();
|
||||
const path = new URL(req.url ?? "", "http://x").pathname.replace(/^\/api\/v1/, "");
|
||||
const body = raw ? JSON.parse(raw) : undefined;
|
||||
sent.push({ method: req.method ?? "GET", path, body });
|
||||
res.setHeader("Content-Type", "application/json");
|
||||
if (req.method === "POST" && path === "/user/repos") {
|
||||
res.statusCode = 201;
|
||||
res.end(JSON.stringify({ full_name: `mesh-tools/${body.name}`, name: body.name, owner: { login: "mesh-tools" },
|
||||
private: body.private, html_url: "", clone_url: "", default_branch: "main" }));
|
||||
return;
|
||||
}
|
||||
res.statusCode = 404;
|
||||
res.end(JSON.stringify({ message: "not found" }));
|
||||
});
|
||||
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
|
||||
after(() => new Promise<void>((r) => server.close(() => r())));
|
||||
const { port } = server.address() as { port: number };
|
||||
const client = new GiteaClient(`http://127.0.0.1:${port}`, new ConfiguredToken("t"));
|
||||
const tool = getGiteaTools(client).find((t) => t.name === "gitea_create_repo")!;
|
||||
|
||||
test("gitea_create_repo has no owner argument and posts only to the tools' own user's repositories (1124/AC8)", async () => {
|
||||
assert.equal("owner" in tool.input, false, "the tool takes no owner");
|
||||
assert.equal(GiteaClient.prototype.createRepo.length, 1, "createRepo takes the repository alone, no owner");
|
||||
// An owner given anyway is not an argument of the tool: the repository is the tools' user's all the same.
|
||||
await tool.run({ owner: "novox", name: "mesh-delivery" });
|
||||
await client.createRepo({ name: "x" });
|
||||
assert.deepEqual(sent.map((s) => `${s.method} ${s.path}`), ["POST /user/repos", "POST /user/repos"]);
|
||||
assert.ok(sent.every((s) => !s.path.startsWith("/orgs/")), "nothing is sent to an organisation");
|
||||
assert.doesNotMatch(tool.description, /organisation/i, "the description offers no organisation");
|
||||
});
|
||||
|
||||
test("the tools' team cannot create in the organisation and its token holds no write:organization (1124/AC8)", () => {
|
||||
assert.equal(TOOLS_TEAM_UNITS.can_create_org_repo, false);
|
||||
assert.ok(!TOKEN_SCOPES.includes("write:organization"), `scopes ${TOKEN_SCOPES.join(", ")}`);
|
||||
assert.ok(TOKEN_SCOPES.every((s) => !/organization|admin/.test(s)), `scopes ${TOKEN_SCOPES.join(", ")}`);
|
||||
});
|
||||
@@ -82,10 +82,9 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
|
||||
},
|
||||
{
|
||||
name: "gitea_create_repo",
|
||||
description: "Create a repository: with owner, in that organisation (the forge's refusal said with its status " +
|
||||
"and reason); without owner, under the tools' own user.",
|
||||
description: "Create a repository under the tools' own user. A repository of the mesh's own is made by the gitea " +
|
||||
"module from its config value repositories, never by a tool (novox/hq ADR 0428 §8).",
|
||||
input: {
|
||||
owner: { type: "string", description: "an organisation to create the repository in (default: the tools' own user)" },
|
||||
name: { type: "string", description: "the repository name" },
|
||||
description: { type: "string", description: "an optional description" },
|
||||
private: { type: "boolean", description: "private repo (default true)" },
|
||||
@@ -97,7 +96,7 @@ function allGiteaTools(gitea: GiteaClient, admin?: () => GiteaAdmin): ToolDefini
|
||||
description: args.description ? String(args.description) : undefined,
|
||||
private: args.private === undefined ? true : Boolean(args.private),
|
||||
auto_init: args.auto_init === undefined ? true : Boolean(args.auto_init),
|
||||
}, args.owner === undefined ? undefined : String(args.owner));
|
||||
});
|
||||
return { repo };
|
||||
},
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user