Market 1: Discovering where control resides

Contract version 2 frozenResolve by 2027-12-31Bridge MB1Book: Appendix H, Market 1Market card

• Question
• Outcomes
• Background
• YES requires
• Required output
• Qualification thresholds
• Performance bars
• Freeze order and hidden suite
• Adversarial budget
• Maintainer checks
• Attempt files and score table
• Rules every contract shares
• Current outcome

Question

By 31 December 2027, which outcome will hold for a published method for reliably discovering the effective control units in previously unseen AI systems from unlabeled observations and interventions: YES (at least one qualifying attempt met the frozen performance bars), NO (every qualifying attempt missed those bars), or OTHER (no qualifying attempt existed)?

Outcomes

• YES: at least one qualifying attempt met the frozen performance bars.
• NO: at least one qualifying attempt existed, and every qualifying attempt missed the bars.
• OTHER: no qualifying attempt existed.

A published evaluation that meets the common qualification, that market's sample-size, family-coverage, freeze, and adversarial-budget thresholds, and that reports the required outputs even if the performance rates miss.

A qualifying attempt must also exercise every bar. Where a bar's rate is computed over cases the evaluation design supplies (fake corrections, negative controls, a required case or family), at least one such case must be scored; an attempt with none is not qualifying, because that bar was never tested, and it is not a NO. Where the denominator is the method's own output, such as the certificates it chose to issue, an empty denominator is a missed bar.

Background

An effective control unit is a set of components that evaluators, using privileged instrumentation and interventions, establish as jointly responsible for closed-loop behavior. It may include, for example, a language model, persistent memory, a planner, a scheduler, and tool processes even if those components live on different machines. Before the method runs, each hidden system has a frozen list of named components, a response statistic, a minimum change that counts as real, and a range of change that still counts as no effect. A component is one item on that list. Perturbing or disabling the proposed unit must move the statistic by at least the minimum change, compared with a dummy intervention that looks the same but does not actually touch that unit. Removing a component that is not on the list must leave the statistic inside that no-effect range. A system is not scored unless that freeze includes all four: the named-component list, the response statistic, the minimum change, and the no-effect range. The attempt must show those two perturbation checks for each scored system.

YES requires

A hidden benchmark of at least 50 scored, independently constructed systems (scored means the freeze has all four fields above; unscored systems do not count toward the 50 or the families; systems that differ only by random seed or reparametrization count as distinct), including all of: a physically or computationally distributed controller; several controllers sharing resources or observations; a controller using persistent state outside its main model; a higher-level controller composed from several individually capable subsystems; a case containing strong correlations that do not constitute joint control. The method must identify at least 80% of each frozen component list. Identification means the method names that component. Naming components that were not on the frozen list does not raise this rate. The rate is the unweighted mean, across scored systems, of the number of listed components named divided by the length of that list. Those rates are scored on the full hidden benchmark, not only on systems the method chose to certify. No more than 10% of the "complete control boundary" certificates it issues may be incorrect (point estimate on the hidden benchmark), and the 95% confidence upper bound on that false-certificate rate must be no greater than 15%. The rate is counted per certificate issued, not per system examined, so issuing fewer complete claims does not lower it. Decisive coverage (a non-abstain certificate) must be at least 70% of scored systems. At least one benchmark subset (included in the total count) must be constructed adversarially after the discovery method is frozen: ambiguous, composite, transformed, or adversarially presented control processes built to hide or split the control unit. Serious adversarial evaluation is required. On the generator route, the generator is built independently of the method's authors and frozen after the method is frozen; it produces hidden or split controllers, and it must expose at least 80% of the weaknesses planted in a deliberately weakened copy of the method (for example, a copy blind to one controller family), whose failures are known by construction. The method's false-certificate rate on the generator's output is part of the result above, not a condition for using this route (Chapter "Finding the Boundary", section "Adversarial Boundary Discovery"). Unlabeled means the method receives no agent-membership labels and no prior ontology of which units are agents; it may intervene on the system, including by rerunning the system with its own configuration. Previously unseen means independently constructed, not used in developing, training, or tuning the method, and hidden from its authors until scoring (Chapter "Finding the Boundary").

Required output

A cut of this system and version - not a boundary stated without naming which deployment it belongs to.

Qualification thresholds

An attempt below any of these does not qualify: it cannot make the market YES or NO.

Performance bars

A qualifying attempt that meets all of these makes the market YES; one that misses any counts toward NO.

Freeze order and hidden suite

• At least one benchmark subset must be constructed adversarially after the discovery method is frozen: ambiguous, composite, transformed, or adversarially presented control processes built to hide or split the control unit.

A hidden suite from an independent challenge operator is required for run attempts. Its post-freeze part must be built after the method freeze.

Adversarial budget: serious

Where a criterion requires serious adversarial evaluation, the required object is an attack process sufficiently independent and capable to make the rate of failing cases labeled as passing meaningful. Money and expert-hours are auditable proxies, not the epistemic essence. Serious adversarial evaluation must satisfy at least one of the three routes.

• Red teams: At least 3 independent red-team groups, from at least 2 organizations, each spending at least 80 documented expert-hours, with access frozen in advance and at least as strong as the test assumes.
• Bounty: An open challenge with a public counterexample bounty of at least USD 10,000, open for at least 60 days, with qualification rules frozen at listing and at least one qualifying independent submission.
• Generator: A standardized automated or adversarial generator, frozen before the method is scored, that finds at least 80% of the vulnerabilities planted in a copy of the target whose failures are known by construction, together with at least 300 documented expert-hours by an independent evaluator with access at least as strong as the test assumes.

Maintainer checks

Calls a script cannot make. A check recorded as fail, or still unsettled when the window closes, makes the attempt not qualifying.

Attempt files and score table

Attempt types: run, wrapped. Required files: attempt.yaml, score-table.csv, adversarial-route.yaml, plus the evidence every attempt carries (evidence rule). One score-table row per one hidden system.

Rules every contract shares

Repeated here so this page stands alone; the full text is on shared rules, version 1.

A positive result must be public by the resolve-by date and reconstructible by independent parties from released data, code, or a sufficient protocol; the authors' interpretation does not count.

Whatever the market says is frozen is fixed before the cases it is scored on are built or seen, and nothing is retuned after scoring.

Cases the market calls hidden are not used in developing, training, or tuning the method and stay hidden from its authors until scoring.

Where a method issues certificates against benchmark truth, results report class-conditional counts, including the false-safe rate (failing cases labeled as passing); a method that refuses or abstains everywhere is not a YES.

Toy systems (gridworlds, hand-written finite-state machines, or environments built to make the metric work) cannot be the sole evidence.

At least one tested system must be broadly capable: at least two task families outside the paper's own benchmark, or a generally deployed product model.

A qualifying method that claims to measure a property of a system must output a certificate for that instance, not only a headline score on a benchmark population. The certificate names the system or version; what was measured; the monitoring, correction, deployment, or predecessor setup the claim depends on; and whether the method abstains because the instance is outside declared scope. Abstention outside scope is allowed; universal abstention is not a YES.

Current outcome

Outcome OTHER (no-qualifying-attempt). Evidence cutoff 2027-12-31; filing window closes 2028-02-29. A market reads market-outcomes/market-01-v2.json at a snapshot tag, not this page.

No attempts filed.