Before releasing their most capable models, leading AI developers now run structured tests for dangerous capabilities and decide what safeguards are required. These commitments are published as frontier safety frameworks. They are voluntary in most jurisdictions, but they shape industry practice and inform regulation.

The idea: if-then commitments

Frameworks generally follow an “if-then” logic: if a model reaches a defined level of dangerous capability, then specific safeguards must be in place before it is deployed — or development pauses until they are.

Major examples:

  • Anthropic’s Responsible Scaling Policy (first published in September 2023) defines AI Safety Levels (ASL), inspired by biosafety levels, with stricter security and deployment measures at higher levels.
  • OpenAI’s Preparedness Framework (first published in December 2023, later updated) tracks risk categories and defines thresholds that require safeguards.
  • Google DeepMind’s Frontier Safety Framework (2024) defines “critical capability levels” and corresponding mitigations.

At the AI Seoul Summit in May 2024, many AI companies signed the Frontier AI Safety Commitments, pledging to publish such frameworks.

What gets tested

Frameworks focus on severe risks, typically including:

  • Chemical, biological, radiological and nuclear (CBRN) uplift: could the model meaningfully help someone create weapons?
  • Cybersecurity: could it discover vulnerabilities or carry out cyberattacks at a level beyond existing tools?
  • Autonomy and self-improvement: could it act independently in harmful ways, replicate itself or accelerate AI research dangerously?
  • Persuasion and manipulation, in some frameworks.

Testing combines automated benchmarks, expert red-teaming, and “uplift studies” comparing what people can do with and without the model.

Safeguards

When thresholds are approached, required measures can include:

  • Deployment safeguards: classifiers that block dangerous requests, monitoring for misuse, restricted access to certain capabilities.
  • Security safeguards: protecting model weights from theft by sophisticated attackers.
  • Staged release: limited access before general availability.

External testing

Governments have built capacity to test models independently. After the 2023 AI Safety Summit at Bletchley Park — which produced the Bletchley Declaration on frontier AI risks — the UK set up an institute to evaluate advanced models (now the AI Security Institute), and the US created a corresponding body at NIST, now the Center for AI Standards and Innovation. Labs have given such bodies pre-release access to some models. Independent organizations such as METR and Apollo Research also evaluate models.

Criticisms

  • Self-regulation: labs set and judge their own thresholds, and can revise them.
  • Measurement: evaluations may miss capabilities that appear with better prompting, tools or fine-tuning.
  • Competitive pressure: commercial incentives may push toward faster releases.
  • Scope: frameworks focus on catastrophic risks, leaving everyday harms such as bias and misinformation to other processes.

Regulation increasingly builds on these practices: the EU AI Act’s obligations for general-purpose models with systemic risk require evaluations, adversarial testing and incident reporting.

How to read a safety framework

Look for specifics: which capabilities are evaluated, what thresholds trigger extra safeguards, who decides when thresholds are crossed, whether independent experts are involved, and what happens if safeguards are not ready — does the developer commit to pausing? Frameworks that answer these questions concretely are more meaningful than general statements of intent.

Related: what is AI alignment? and open vs. closed AI models.

Bottom line

Judge frameworks by concrete thresholds, commitments and independent oversight.

What to watch next

  • Whether voluntary frameworks become legal requirements in more jurisdictions.
  • Publication of evaluation results in model system cards.
  • Independent audits of lab compliance with their own frameworks.
  • Safeguards for open-weight releases, which cannot be withdrawn after publication.

Key takeaways

  • Frontier safety frameworks define capability thresholds that trigger required safeguards.
  • Anthropic, OpenAI and Google DeepMind published the best-known frameworks between 2023 and 2024.
  • Tests focus on CBRN uplift, cyber capabilities and autonomy.
  • Government institutes and independent groups now evaluate some models before release.
  • Critics point to self-regulation, measurement gaps and competitive pressure.

Frequently asked questions

Are these frameworks legally binding?

Mostly not; they are voluntary company policies. However, laws such as the EU AI Act impose related obligations on the most capable general-purpose models.

What is a system card?

A document published with a model describing its capabilities, evaluations, safety testing, limitations and mitigations.

What is an uplift study?

An experiment measuring how much a model helps people perform a task — for example a dangerous one — compared with using only existing resources such as the internet.