diff --git a/01-RESEARCH/043-an-issues-complexity-hints-which-model-solves-it/00-overview.md b/01-RESEARCH/043-an-issues-complexity-hints-which-model-solves-it/00-overview.md new file mode 100644 index 00000000..5edeaf63 --- /dev/null +++ b/01-RESEARCH/043-an-issues-complexity-hints-which-model-solves-it/00-overview.md @@ -0,0 +1,93 @@ +--- +status: active +initiated: 2026-10-10 +touches: + - 03-DESIGN/01-to-be/53-the-issue-tracker.md + - 02-DECISIONS/0275-the-mesh-numbers-and-tracks-its-issues-in-a-module-of-its-own-and-hq-keeps-only-the-narrative-of-a-design-or-governance-issue.md + - 02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md + - 02-DECISIONS/0206-a-node-reports-the-anthropic-grant-it-holds-and-the-licence-manager-adopts-a-licence-by-refreshing-it.md + - 00-META/process/03-issues.md + - 00-META/glossary.md + - the agent module's tools and the licences it spends +became: [] +--- + +# 043 — An issue's complexity, as a hint to which model should solve it + +## What + +Whether the issue tracker can estimate **how hard an issue is to solve**, and use that estimate to +suggest **which model an agent should solve it with**: a smaller, cheaper model for a simple fix, the +strongest available for a change to the core of the mesh. The idea is the operator's, from 2026-10-10, +and he named it a research topic, not a build. + +The tracker already asks a model one question about every new issue: the duplicate check +([ADR 0278](../../02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md)) +reads the new issue's report against the backlog, through the agent module's tool that answers a +structured question with no tools of its own. An estimate of complexity could be one more answer from +the same kind of judgement. Whether it should be, and whether its answer is worth acting on, is what this +effort is for. + +## Why + +Agents solve most issues on the mesh, and every session spends a licence's allowance. Today the model a +session runs is whatever the session was started with: nothing in an issue says what it needs, and +nothing records afterwards which model solved it, how long it took or whether its fix held. A strong +model on a one-line fix spends more than it has to; a weak model on a change to the controller or the +node-engine can produce a fix that looks done, passes a careless review, and breaks a node that updates +unattended. The second cost is far larger than the first, so a hint is only worth having if it is right +where it matters most. + +## The questions + +1. **What "complexity" means here.** Lines changed, repositories touched, whether a core component or + the store's schema is involved, whether the cause is known yet, whether a decision is needed first? + Are these one scale or several? Which of them can be read from a **report**, before anyone has + diagnosed the issue, and which only become known once it is `located`? An estimate made at `open` may + have to be revised at `located`, and the question is whether the first estimate is worth making at all. +2. **Who estimates, and when.** A judgement at `open`, beside the duplicate check; one at `located`, when + the owner is known; the agent that diagnoses the issue, in its own words; or the operator. Can the + owner named in `located-in` alone carry most of the signal (a module's own tool versus the controller)? +3. **Where the hint lives.** A field stored on each issue, set and changed through a verb with who set it + and why, or a value **derived** each time from the issue's fields and a setting that maps classes of + work to models? A stored hint can be overruled per issue; a derived one changes everywhere when the + setting changes, which matters when a newer model arrives. Is the hint a model's name, or a tier that a + setting maps to whatever model holds that tier today? +4. **How the outcome is measured, so the choice rests on evidence.** What the mesh would have to record + per solved issue: the model used, tokens and wall time spent, whether the pull request passed its + independent review and both required checks (`mesh/merge-gate`, `mesh/repo-check`) the first time, + how many rounds of rework it took, and whether the issue was reopened or a follow-up opened against + the same cause. Which of these does the mesh already hold (the tracker's notes and links, the + forge's review history, the delivery's checks), and which are recorded nowhere today? Without this, + any mapping from complexity to model is an assertion. +5. **The risk of a weak model on a core change.** Failure must be loud: how does a session that was + given too weak a model say so, rather than hand over a fix that reports success? Should a hint ever + lower the model below what the session was started with, or only raise it? Should some owners (the + controller, the node-engine, the store, the foundation) be outside the hint entirely, always solved + with the strongest model? Is review by a stronger model than the author's a cheaper safeguard than + choosing the author well? +6. **How it relates to the fields being added to the tracker.** Severity says how much an issue hurts; + what an issue needs says what must happen before it can move. Complexity says how hard the fix is. + Are these three separate fields, or is complexity derivable from the other two and the owner? A + severe issue can be simple, and a minor one can sit in the core. +7. **Who acts on the hint.** Does an agent session read it and start a sub-agent with that model; does + the operator see it when choosing what to hand off; does a future dispatcher assign work by it? A + hint nobody reads changes nothing, and the effort should name its reader before anything is built. +8. **Prior art.** Routing a request to the cheapest model that answers it well is studied for single + questions (model cascades and routers that learn from preference data); estimating a task's + difficulty is studied for software issues in agent benchmarks, and in teams as estimation by size or + points. What carries over to a repair done by an agent across a pull request, review and delivery, + and what does not? + +## Words + +The glossary has no word for an issue's estimated difficulty, nor for a suggested model. This effort +uses **complexity** and **model hint** as plain descriptions until a decision settles them, and +proposes them then through the glossary's own rules. + +## What it touches + +The issue tracker's fields and verbs ([to-be 53](../../03-DESIGN/01-to-be/53-the-issue-tracker.md)), +the judgement the duplicate check already makes (ADR 0278), the agent module and the licences it spends +(ADR 0206), the issues playbook ([03](../../00-META/process/03-issues.md)), and, if outcomes are recorded, +the forge's reviews and the delivery's required checks as sources of evidence.