An AFF research experiment · September 2026
Everyone Can Still Win
If powerful AI can't be stopped, what can we realistically build instead?
The argument in one paragraph
We may not be able to stop the capability. We can still decide how people are held accountable for it. The realistic goal is a system where finding a danger is always safe, ignoring one is always costly, and abusing one is hard to hide, and where every country and company has more to gain inside that system than outside it.
The lens · Pause
The tension this project looks through
Every investigation needs a lens. This one emerged from a disagreement inside the research itself, and we want to show it rather than hide it.
“Being naive about the capabilities of AI is a sure shot to destruction.”
Capabilities are advancing quickly. Underestimating them is itself a risk.
Speed itself may not be the fundamental problem. The more important problem may be people with malicious intent, and the absence of effective accountability.
If dangerous capabilities are coming regardless, the question shifts from how fast to who is responsible.
Neither statement is established fact. Both contain something true. The first warns against complacency about what these systems can do. The second suggests our attention may be better spent on the humans around the systems: who builds them, who deploys them, who misuses them, and who answers for it.
If dangerous capabilities are inevitable, shouldn't our focus increasingly move toward identifying malicious behaviour, creating accountability, and building institutions capable of responding to it?
This project does not claim to have answered that question. It uses it as a lens, and we note its limit: many documented AI harms so far come from accidents, unintended behaviour or structural effects rather than malice. We keep those in view, and we list this as an open question at the end.
Why this is a wicked problem
There is no regulatory switch. Here's why.
In 1973, the planners Horst Rittel and Melvin Webber distinguished “tame” problems from “wicked” ones.1 A tame problem can be defined and solved: build a bridge, cure a known infection. A wicked problem can't. People disagree about what the problem is. Every attempted solution changes it. There is no point at which you're finished, and solutions aren't right or wrong, only better or worse.
Frontier AI fits almost every part of that description. Governments want both innovation and protection. Companies compete, and caution can cost market share. The same model that helps design medicines can be misused. The tools needed to find dangers are the tools that can create them. And the technology moves faster than the institutions trying to govern it. Compute has been proposed as one useful governance lever, though not the only one.17
The clearest way to see it is as a set of tensions. None can be “solved”. Each can only be balanced, and pulling too far either way creates a new failure.
Too far toward progressRisks are discovered by the public, not in testing.
Too far toward safetyDevelopment moves to places with fewer safeguards.
Too far toward opennessDangerous capabilities spread with no way to recall them.
Too far toward controlPower concentrates in a few companies and governments.
Too far toward security researchTesting becomes cover for extracting harmful capabilities.
Too far toward liabilityCompanies stop wanting to know what their systems can do.
Too far toward national sovereigntyEvery country races alone.
Too far toward global riskNo major power signs up.
Too far toward innovationHarms become someone else's problem.
Too far toward accountabilityOnly the largest firms can afford to comply.
Too far toward speedRules arrive after the damage.
Too far toward governanceRules freeze yesterday's technology in place.
Too far toward private companiesProfits are private, risks are public.
Too far toward public consequencesThe expertise sits outside government's reach.
Too far toward legitimate experimentationBad actors hide among researchers.
Too far toward malicious useEvery experiment looks like a crime.
Too far toward centralisationOne failure point, one point of capture.
Too far toward decentralisationNobody is responsible for anything.
This is why “just regulate it” and “just let it run” are both non-answers. The real question is how to build a system that keeps rebalancing these tensions as the technology changes.
Why solve this before a crisis forces us to
The risk isn't only a dramatic failure. It's a quiet one.
If increasingly powerful AI is coming whether we like it or not, governance stops being an optional brake and becomes a precondition for living with it safely. The same was true of electricity, aviation and finance: none became trustworthy by accident.
The failure to worry about most isn't always dramatic. It's the slow erosion of accountability until nobody can say who was responsible for what. That kind of failure compounds.
- 01
Researchers are discouraged. Finding a flaw carries legal risk, so fewer people look.
- 02
Companies externalise risk. What isn't found can't be fixed, and costs fall on the public.
- 03
Safety becomes a competitive disadvantage. Careful firms ship slower and lose ground.
- 04
Governments race independently. Each assumes the others won't hold back.
- 05
Countries distrust one another. Without shared evidence, every claim looks like strategy.
- 06
Dangerous capabilities spread. Nobody has the visibility to notice early.
- 07
Responsibility becomes impossible to assign. When something goes wrong, every actor points at another.
And back to 01: with no one able to answer for failures, researchers are discouraged further.
Some of this is already visible. In 2026, standard commercial liability insurance began excluding AI-related claims across the US market, while a few specialist insurers started offering dedicated AI cover.2 When mainstream insurers can't price a risk, it doesn't disappear. It sits, unpriced, on company balance sheets and on the public.
Rules written in the aftermath of a crisis tend to be blunt, designed to show that something was done rather than to be well calibrated. Aviation's safety culture and modern financial supervision were both shaped by disasters. The opportunity with AI is to design accountability deliberately, while there's still room to think.
The question isn't whether AI will be governed. It's whether the governance will be designed or improvised.
The Flight Recorder Compact
A proposal, not a prediction.
The short version
Aviation didn't become safe by stopping flight. It became safe by recording what happened, reporting near-misses without fear, investigating failures independently, separating honest error from recklessness, and letting certified aircraft fly anywhere.
The Flight Recorder Compact applies that logic to AI through five mechanisms. Record what's being built and decided. Probe: protect anyone who looks for danger in good faith. Respond: hold developers accountable for what they knew, when, and what they did about it. Underwrite: make risk carry a price. Intercept: defend the points where digital capability becomes physical harm.
Countries join in rings, from full members to a universal floor that even rivals can accept. The rules adapt on a fixed cycle as the technology changes.
The research behind this page began with a proposal built on control: special chips that regulators could switch off, and cryptographic proofs of how every model was trained. It also borrowed the nuclear industry's Price-Anderson model, which caps liability rather than strengthening it.16 When we stress-tested it, the problems were clear. It asked the world's major powers to hand an off-switch to someone else, and it depended most on the technologies it was least sure about.
The Flight Recorder Compact starts somewhere else. Instead of trying to control the technology, it focuses on the people and institutions around it: what they knew, what they decided, and what they did next. Control mechanisms still play a role, as one layer among several, and one we expect to weaken over time.
Who takes part
How it works in practice
- 1
Plan
A lab registers a large training run. The cloud provider confirms the customer's identity.
- 2
Test
The model is evaluated by the lab, by the national institute and by independent researchers. Results go in the decision log.
- 3
Find
An independent researcher discovers the model can give meaningful help with a dangerous task. She reports it through the coordination function.
- 4
Respond
The clock starts. The lab mitigates, the fix is re-tested, and the accountable executive signs off.
- 5
Deploy
The model ships with a certification recognised across member countries. Its record feeds its risk contribution.
- 6
Learn
If an incident happens later, investigators ask the three questions. Findings feed next year's standards.
Rings of membership
The Compact
Countries hosting frontier development or key supply-chain steps.
- Gives
- All five mechanisms, incident sharing.
- Gets
- Mutual recognition of certifications and evaluations, a shared market, joint influence over standards.
Associates
Countries adopting the standards without hosting frontier labs.
- Gives
- The rules, incident sharing.
- Gets
- Recognised certification for their own AI firms, shared evaluation tools, technical support, priority access to safety-relevant capabilities.
The universal floor
Commitments any country can make, even rivals.
- Gives
- Human control over nuclear-use decisions; no self-spreading cyber agents against civilian infrastructure; DNA synthesis screening; crisis hotlines; notification of cross-border incidents.
- Gets
- Fewer accidents and escalations nobody wants.
The nuclear non-proliferation regime worked partly because it offered a bargain: accept verification, get access to peaceful technology. The middle ring is that bargain for AI. Joining should be the easiest way to benefit from the technology, not a tax on using it.
What if a major AI power refuses to join? The system is designed not to depend on universal membership. A non-member can still sit on the universal floor. Its companies can still sell into member markets by passing the same published tests as everyone else. And the Intercept layer protects members regardless of where a threat comes from. This is weaker than universal agreement, and using certification as a condition of market access is a trade measure that will need care. It is also more realistic.
Consequences that escalate
- 1Missed obligationLate report, incomplete log.→ Correction notice, public registry entry.
- 2NegligenceReasonable care not taken; required tests not run.→ Civil liability, higher contributions, conditional certification.
- 3RecklessnessA known risk consciously ignored.→ Loss of safe harbor, significant fines, deployment restrictions, personal liability for the accountable executive.
- 4ConcealmentFalsified records, buried findings, retaliation against researchers or whistleblowers.→ Criminal exposure for individuals, loss of certification across all member markets.
- 5Deliberate malicious use→ Criminal law and international law-enforcement cooperation.
For countries: loss of mutual recognition and associate benefits, coordinated export controls, and for red-line violations the same diplomatic and strategic consequences that apply to other weapons concerns.
How it's verified
No single technology carries the weight. Verification combines independently audited decision logs; behavioural testing by national institutes, shared between members;15 chip location checks and cloud identity checks for very large computing purchases; strong whistleblower protection, historically one of the most effective tools in regulated industries; shared incident data; and satellite and energy observation for very large undeclared facilities. Each method can fail. Together they make concealment expensive.
How it adapts
Principles go into law: the three questions, researcher protection, the accountable executive, financial responsibility. Technical details, such as computing thresholds and test suites, go into annexes that technical bodies update every year, the way international aviation standards are maintained. Every threshold expires unless re-justified.
- TriggerIf openly available models reach agreed danger thresholds, release decisions face heavier review.
- TriggerIf capability stops depending on large amounts of computing power, compute reporting becomes less central and testing duties expand.
- TriggerIf AI agents increasingly act on behalf of other agents, rules requiring every agent at scale to be traceable to an accountable organisation switch on.
What changed from the original research
| Original | Version 2.0 |
|---|---|
| Regulators can remotely halt chips | Chips confirm location only; no off-switch |
| Cryptographic proofs of training as the core audit | Testing the model's actual behaviour as the core audit |
| Nuclear-style capped liability with a flat industry pool | Phased, risk-weighted financial responsibility; no cap for negligence or concealment |
| Safe harbor for compliance paperwork | Safe harbor earned by how you respond to what you know |
| Protection only for accredited testers | Protection for any good-faith researcher |
| Identity tracing of all users | Identity checks only for high-risk access |
| Embargo countries that defect | Make membership valuable; defend against harm regardless |
| One new global institution | Existing institutions plus a small secretariat |
Where we expect disagreement
The Compact does not settle which AI risks matter most, whether speed is dangerous in itself, or how to put a price on risks nobody can measure. It tries instead to make disagreement productive: testing methods are published, institutes record dissenting views, and thresholds are reviewed in public.
Finding a danger is not the failure
The difference between discovering a risk and causing one.
Consider two headlines. "A researcher found a dangerous capability in a new AI model." And: "A company knowingly released a model with a dangerous capability."
Today these can blur together. If a company's legal exposure grows every time a flaw is found, the rational response is to find fewer flaws: restrict outside researchers, bind testers with non-disclosure agreements, stop looking. The research behind this project called this the accountability paradox.
The opposite failure is just as real. If responsibility is spread so thin across developers, cloud providers, app builders and users that nobody is ultimately accountable, nothing stops reckless development.
The way out is to change what the law cares about. Not whether a danger exists, since in powerful general systems some almost always will, but how people behaved once they knew, and whether they took reasonable steps to find out.
- 1What did you know?
- 2When did you know it?
- 3What did you do about it?
These questions aren't new. Criminal law has long distinguished acting on purpose, knowingly, recklessly and negligently.4 Financial regulation asks named executives to certify what they report. Aviation separates honest error, which should be reported and learned from, from recklessness, which should be punished. The Compact applies the same distinctions to AI.
- Responsible disclosureSomeone finds a danger and reports it through the proper channel.→ Protected and encouraged.(selected case position)
- Good-faith security researchTesting that stays proportionate, keeps demonstrations minimal and follows disclosure rules.→ Protected.
- NegligenceA developer failed to take reasonable care: skipped required tests, ignored an obvious risk.→ Civil liability, higher costs.
- Reckless deploymentA developer knew about a serious risk and shipped anyway without verified mitigation.→ Loss of safe harbor, personal liability for the accountable executive.
- Deliberate malicious useSomeone intended harm.→ Criminal law.
Case explorer · choose a scenario
A. The jailbreak report
An independent researcher finds a prompt that makes a model give detailed help with a cyberattack. She reports it via the coordination function with a minimal proof of concept.
- What did you know?
- Now, yes.
- When?
- On receipt.
- What did you do?
- Patched within the deadline, re-tested, documented.
Responsible disclosure. Researcher protected; developer keeps its safe harbor; the finding improves standards for everyone.
Real cases are messier. The point isn't that every judgement becomes easy; it's that everyone knows in advance which questions will be asked.
Why this protects researchers
Because discovery is never the violation, a company gains nothing by avoiding outside testers. It gains a great deal by hearing about problems first: every report it receives and handles well strengthens its safe harbor.
There is precedent. In May 2022 the US Department of Justice adopted a policy of not prosecuting good-faith security research under its main computer-crime law, though that policy covers criminal charges only and offers no shield against civil suits.5 Software security has used coordinated disclosure, typically with a 90-day window, for years. The Compact extends protection to civil law, which is where most of the remaining chill lives.
Why this still holds someone responsible
Responsibility can't disappear into the supply chain, because every frontier developer names one accountable executive, with real authority to stop a release, who personally signs its safety reports. Similar regimes exist in UK banking and in US corporate reporting rules.6 Whoever signs is answerable for what they knew.
Some of this is already law in one place. California's Transparency in Frontier AI Act, in effect since January 2026, requires frontier developers to publish safety frameworks, report critical incidents to the state within 15 days, and protects whistleblowers who raise safety concerns.7 The Compact builds on that shape and adds the response duty, verification and financial responsibility.
What nuclear governance actually teaches us
Not "AI needs an IAEA". Something more useful.
It's tempting to say AI needs its own International Atomic Energy Agency. The more useful question is why nuclear governance partly worked, and which of those reasons survive the move to software.
What transfers
- Verification. Trust wasn't assumed; it was checked. The IAEA (1957) tracks nuclear material and inspects declared facilities.8Lesson: Credible verification is the price of cooperation.
- Transparency and incident reporting. Sharing information about accidents improved safety across countries.Lesson: Shared data is a public good.
- Shared standards. Common technical baselines made cooperation practical.Lesson: Harmonise the boring details.
- Reciprocal obligations. The Non-Proliferation Treaty (1968) offered a bargain: accept verification, get access to peaceful nuclear technology.9Lesson: Give non-leaders a reason to join.
- Communication. Crisis hotlines reduced the risk of accidental escalation.Lesson: Build channels before you need them.
- Red lines between rivals. In November 2024 the US and China affirmed that humans, not AI, should control decisions to use nuclear weapons; China repeated the principle in a 2025 white paper.10Lesson: Narrow red lines can be agreed even between competitors.
- Legitimacy. Rules seen as broadly fair were more durable.Lesson: A club of the powerful alone won't last.
Where it breaks
- AI is software. Enriched uranium needs vast industrial facilities. A trained model is a file.
- AI can be copied. Once released, copies can't be recalled.
- AI can be distributed. Models cross borders instantly.
- AI can be developed privately. Nuclear programmes were run by states. Frontier AI is led by companies.
- Capabilities diffuse. What needed the frontier two years ago may run on a laptop later.
- There is no clean dual-use line. Reactor fuel and weapons material differ by enrichment level. The same model can help design a medicine or a toxin.
A direct copy of nuclear governance would fail. The deeper mechanisms — verify, share, standardise, bargain, communicate — are still worth borrowing.
Other regimes worth learning from
Chemical weapons
The Chemical Weapons Convention (1997) tracks precursor chemicals and allows any member to demand a challenge inspection. None has ever been requested, which says something about both deterrence and political reluctance.11
Aviation
Confidential near-miss reporting since 1976 and independent accident investigation built a culture that learns from failure.12
Finance
After 2008, systemically important banks were subjected to stress tests, resolution plans and named-executive responsibility.
Cybercrime
The Budapest Convention (2001) and the UN Convention against Cybercrime (adopted 2024, signing opened 2025) support cross-border cooperation, though the UN treaty has drawn criticism over rights safeguards.13
Two versions of 2032
Illustrative scenarios, not forecasts.
A security researcher in Nairobi finds a way to make a widely used model help plan an attack on a water utility. She reports it through the coordination function on a Tuesday. By Friday the developer has a fix, re-tested and signed off by its accountable executive. She's credited publicly. The finding goes into next year's test suite for every certified model.
Risk now carries a price the way it does in other industries: companies with clean records and fast, verified responses pay noticeably less into the shared pool, and a few insurers have returned to the market because the incident data finally exists. Hospitals and banks buy only certified systems, so certification is the fastest route to the biggest customers. A mid-sized lab in São Paulo gets its model certified once and sells in thirty countries.
Not everything works. A powerful open model with weak safeguards circulates widely. But DNA synthesis companies screen orders, critical infrastructure has been hardened, and when an attempted misuse happens, investigators trace it, prosecute across borders and publish what they learned. Two major powers still disagree on much, but their crisis channel works, and neither has put AI in charge of nuclear decisions.
It isn't safe. It's governable.
Scenarios are illustrations to make institutional design concrete. They are not predictions.
How this research was made
The method is part of the experiment.
This page was produced by passing one problem through several AI systems, each with a different role, then editing the result by hand. We think the process is worth showing, including its limits: AI systems sound confident when they're wrong, and several claims in the original research didn't survive challenge or source-checking.18
- 01
ChatGPT · Prompt design
Used to formulate the initial research question and build a deep-research framework around the wicked problem.
- 02
Gemini · Research
Used to conduct broad research across AI governance, nuclear governance, international institutions, accountability and related fields. Output: a report proposing a hardware- and cryptography-based governance architecture.
- 03
Claude · Challenge and improvement
Used in two rounds. First, to stress-test the research from four perspectives — policy expert, futurist, adversarial researcher and business strategist — and produce an improved proposal. Second, to attack that proposal in turn and verify every source used on this page. Result: the Flight Recorder Compact, version 2.0.
- 04
Lovable · Deployment
Used to turn the research into this web experience.
NotebookLM — used to turn the research into an audio conversation. See the podcast below.
- 05Not yet built
Future distribution · Potential next step
Possible future experiments with AI-driven distribution and discovery, which could include Grok-based bots or agents where appropriate. These do not exist yet.
Listen to the research
An audio conversation generated with NotebookLM.
NotebookLM was used to turn this research into a voice-only conversation. It's another experiment in the same spirit: can an AI-generated discussion make a complex governance question easier to follow? Listen critically.
Everyone Can Still Win: Accountability in the Age of Unstoppable AI
An AI-generated conversation walking through the Flight Recorder Compact: holding AI developers personally liable for what they knew and how they acted.
What we haven't solved
Challenge any of these. That's the point.
Who should be accountable when AI causes harm and several companies built parts of the system?
Challenge this →Who gets to define a "dangerous capability", and how do we stop that definition being captured by incumbents?
Challenge this →Can international institutions keep up with AI development, or only with last year's version of it?
Challenge this →Can safety become a genuine competitive advantage, or will it always cost market share?
Challenge this →Does a "what you knew" standard tempt companies not to look, and is a duty to look enough to close that gap?
Challenge this →How much AI harm comes from malicious intent, and how much from accidents, unintended behaviour or concentration of power?
Challenge this →We'd especially like to hear from policymakers, security researchers, AI developers, insurers, lawyers, founders, academics and anyone working on these problems outside the US and Europe.
The invitation
This is not a finished answer. It is an attempt to put a better question on the table.
The people who can build a globally workable system for AI governance have expertise, authority and resources this project doesn't. Our hope is to give them a sharper starting point.
Underneath it is a simple belief. There may be a future in which AI progress continues, companies can build, researchers can test, governments can protect their citizens, and society can remain meaningfully accountable for what happens.
Everyone can still win.
Not because the risks disappear. But because incentives, institutions and accountability can be redesigned so that progress and responsibility stop being treated as opposites.
This project is an attempt to find that path.
Sources and notes
Sources are listed for transparency. This page was produced with AI assistance and its sources were checked in September 2026, but readers should consult primary sources before relying on any claim.
- 1.Rittel, H. & Webber, M. (1973). "Dilemmas in a General Theory of Planning." Policy Sciences 4(2). ↩
- 2.ISO/Verisk generative-AI exclusion endorsements CG 40 47, CG 40 48 and CG 35 08 (effective 1 January 2026); reporting on carrier AI exclusions and new affirmative AI liability products (2026). ↩
- 3.Peigné, P., Nguyen, K. & Wang, P. (2026). "Zero knowledge verification for frontier AI training is possible." arXiv:2606.05433 (preprint). ↩
- 4.American Law Institute, Model Penal Code §2.02 (culpability levels). ↩
- 5.US Department of Justice (19 May 2022), revised policy on charging under the Computer Fraud and Abuse Act. ↩
- 6.UK Senior Managers and Certification Regime (2016–); US Sarbanes-Oxley Act (2002), §302 and §906 certifications. ↩
- 7.California Senate Bill 53, Transparency in Frontier Artificial Intelligence Act (signed 29 September 2025; effective 1 January 2026). ↩
- 8.Statute of the International Atomic Energy Agency (1957). ↩
- 9.Treaty on the Non-Proliferation of Nuclear Weapons (opened for signature 1968; in force 1970). ↩
- 10.White House readout of the Biden–Xi meeting, Lima, 16 November 2024; China's white paper on arms control, disarmament and non-proliferation (November 2025). ↩
- 11.Chemical Weapons Convention (in force 1997), Article IX; OPCW statements on challenge-inspection readiness exercises. ↩
- 12.NASA Aviation Safety Reporting System (1976–); ICAO Annex 13. ↩
- 13.Council of Europe Convention on Cybercrime (Budapest, 2001); UN Convention against Cybercrime (GA res. 79/243, 24 December 2024; opened for signature Hanoi, 25 October 2025). ↩
- 14.US OSTP Framework for Nucleic Acid Synthesis Screening (April 2024; under revision per May 2025 executive order); International Gene Synthesis Consortium harmonized screening protocol. ↩
- 15.International Network for Advanced AI Measurement, Evaluation and Science (formerly International Network of AI Safety Institutes; launched Seoul 2024, renamed December 2025). ↩
- 16.US Congressional Research Service, "Price-Anderson Act: Nuclear Power Industry Liability Limits" (IF10821, 2025 update). ↩
- 17.Sastry, G., Heim, L. et al. (2024). "Computing Power and the Governance of Artificial Intelligence." ↩
- 18.Original research report: "Global AI Safety Governance and Executive Accountability Architecture" (multi-model research, 2026). ↩