National regulatory agencies — the FDA, FAA, EPA, BaFin, FERC and their peers — translate technical judgment into binding decisions about what may enter a market. The model is a stack:
The stack is stabilized by uncertainty: firms that can't predict exactly how the agency will react act conservatively, often consulting the regulator informally before consequential moves.
Agency review assumes the regulated product decomposes into inspectable evidence; AI systems may not. A drug approval works because the artifact holds still: the molecule is fixed, trials measure its effects, and the evidence formats (protocols, endpoints, statistical plans) are ones reviewers have judged a thousand times. An AI system that is updated continuously, behaves differently across contexts, and can't be reduced to a mechanism a reviewer can interrogate doesn't fit those formats. The agency can still demand evidence, but it no longer knows what evidence would settle the question its statute makes it responsible for.
Agent-generated filings can arrive faster than review capacity grows. Gatekeeping presumed that producing a serious application was expensive — expensive enough that volume self-limited and the queue stayed reviewable. When applications, comment letters, incident reports, and petitions can be generated at near-zero marginal cost, review attention becomes the scarce resource, and the queue becomes a strategic instrument: a firm can bury a controversial filing among routine ones, or exhaust an opposing party's capacity to respond in a comment period.
Firms with capable agents can model the regulator precisely enough to walk up to the enforcement line. Conservative compliance was never just obedience; it was the product of uncertainty about how the agency would react, priced in lawyer hours. An agent trained on the agency's rules, guidance, enforcement history, and published decisions can estimate reactions well enough to optimize against them — staying technically inside every rule while draining the buffer that uncertainty used to enforce. The informal-consultation channel inverts too: instead of asking the regulator, firms ask a simulation of the regulator, and the agency loses the early visibility those consultations used to give it.
Scenario. A mid-size airline's safety agent flags a subtle drift in one of its diagnostic pipelines — a pattern it believes the FAA would care about, but that no current rule explicitly forbids. Under the old regime, the safety officer, Renee, would have written a letter, heard back in four months, and flown the planes in the meantime. Instead she queries the agent the FAA now operates for exactly this purpose: here is our situation, how would the regulator view it? The answer comes back in seconds and she grounds one aircraft for six days. Then the questions start circulating in the general counsel's office: what exactly did the FAA just tell us, what is it worth if the agency later disagrees, and did the airline two gates over get the same answer?
Challenge: Design the regime governing AI-issued regulator-anticipation opinions: their legal status, the consistency the agency commits to, and the protection a firm gets for relying on them — fast enough to be useful at agent speed without the opinions hardening into de facto law that no rulemaking ever produced.
Evaluation. Strong proposals survive both failure directions: opinions so binding that the model's outputs become uncontestable regulation, and opinions so weightless that no firm can prudently act on them.
Scenario. A developer submits a hospital triage model for authorization. The model outperforms clinicians on every retrospective benchmark, but it is updated monthly, its behavior shifts with the patient mix, and no reviewer can trace why it deprioritizes a given case. The agency's evidence playbook — fixed artifact, frozen behavior, pre-specified endpoints — doesn't apply. Rejecting the system means keeping a worse triage process in place; approving it means certifying something the agency cannot inspect and whose behavior next quarter is not the behavior it reviewed.
Challenge: Design the evidence regime by which a regulator authorizes a continuously updated AI system: what the applicant must produce in place of the traditional safety case, what ongoing obligations replace one-time approval, and what triggers re-review.
Evaluation. Strong proposals name evidence the agency can actually evaluate with the staff it has and keep authorization meaningful after the tenth model update; weak ones either re-demand mechanistic transparency the artifact can't provide or collapse into trusting the developer's own evaluations.
Scenario. A newly appointed regulator at a federal financial agency, Arjun, sits down for his orientation and realizes something odd. Every part of his formal training is about rules — which statutes, which enforcement tools, which precedents. Nothing in it asks him to engage with what the agency is actually for — whose savings it protects, what "fair dealing" was meant to cover, why the statute he is learning to enforce was passed in the first place. The senior colleagues who carry that understanding are due to retire in the next decade. Arjun begins to understand that if the training pipeline doesn't change, the agency will still enforce rules but will lose its ability to recognize when the rules and the purpose have diverged.
Challenge: Design a professional-training regime that builds mandate literacy — the capacity to recognize when an institution's rules have drifted from its purpose — into the pipeline that produces a national-scale institution's practitioners, and specify which programs carry it, how it is taught, and how the state's role relates to profession-set content.
Evaluation. A strong design makes mandate literacy something practitioners can be tested and credentialed on rather than a forgettable box-check, and gives the system a way to surface and replace training programs that have themselves drifted.