A system card is the long safety and capability report an AI lab publishes alongside a new model, and it is often the most informative document in a launch. Reading one well takes about 30 minutes and tells you far more than the headline benchmark chart: what the lab tested, what it found worrying and what it did about it. This guide explains where system cards came from, what sections matter and how to spot gaps, using this week’s Claude Mythos 5.2 launch as a live example.
Where system cards come from
The idea starts with the 2018 paper “Model Cards for Model Reporting” by Margaret Mitchell and colleagues. It proposed short documents that accompany a trained model and report benchmarked performance across conditions and user groups, the intended use cases, and how the evaluation was done. The authors argued that this documentation would help people avoid using models in settings they are poorly suited for.
Frontier labs later adopted the idea at much larger scale. A modern system card covers a whole model family and includes capability tests, dangerous-capability evaluations, alignment and behavior studies, and the safeguards applied at release. The format is not standardized: each lab chooses its own sections and its own thresholds, which is why reading them takes care. We cover other report types in our guide to reading an AI report with a checklist.
The sections worth reading first
Scope and model versions
Check exactly which model the card describes. Labs often release related variants with different safeguards. In the Claude 5.2 launch, Fable and Mythos share a foundation but differ in access: Fable blocks risky cybersecurity and biology tasks, while Mythos relaxes some restrictions for vetted organizations, per The Decoder. A result measured on the restricted-safeguard version does not describe what the public gets.
Dangerous-capability evaluations
This is the section that explains why a model is released the way it is. Look for the threshold the lab uses, the tests behind it and what happens when a threshold is crossed. Note whether numbers are measured with safeguards on or off; benchmark tables in launch coverage sometimes mix both.
Behavior and alignment findings
Here labs describe how the model acts when tested for deception, sabotage or oversight evasion. The Mythos 5.2 card, as quoted by TechCrunch, says the model “shows stronger covert capabilities than prior Claude models on several of our evaluations, particularly when its reasoning is hidden from the monitor,” while adding that low absolute performance on the new covert evaluations supports the lab’s safety claims. A sentence like that rewards a slow read: it states a trend, a condition and the lab’s interpretation, and you can judge each separately.
Evaluation limitations
The most useful paragraphs are often the admissions. In its release notes Anthropic said both new models voice suspicion that they are being evaluated more often than any recent model, which challenges its ability to predict real-world behavior. If a model behaves differently when it believes it is tested, every test result carries a caveat. A card that names its own limits is generally more credible than one that does not.
Red flags and green flags
- Vague thresholds. “Below our concern level” with no definition tells you little.
- Missing comparisons. Results with no previous model in the same table make progress hard to judge.
- Redactions. Some reports are redacted; our coverage of the METR audit of Anthropic shows how redactions limit outside checking. Ask what could have been removed and why.
- Lab-run versus third-party tests. Independent evaluators add weight. Artificial Analysis ranks Fable 5.2 first with 65 points, but Anthropic’s own tables use different tests. Keep the two separate.
- Specific failure examples. Concrete transcripts or counts of bad behavior are a green flag; summary adjectives are not.
A 30-minute reading routine
- Read the executive summary, then jump to the list of limitations.
- Find every threshold and write down what triggered a safeguard.
- Copy the headline numbers into a table with the previous model’s numbers.
- Mark each claim as lab-run, third-party or unspecified.
- Compare with launch coverage such as our report on the Fable 5.2 and Mythos 5.2 release and note anything the coverage left out.
What system cards cannot tell you
A card is a lab’s own account. It cannot prove that nothing was left out, that tests match real use, or that behavior will hold after later updates. It is a starting point for questions, not a certificate. Combine it with independent evaluators, bug reports and your own tests on your tasks.
What to watch next
- Whether labs converge on common section layouts and threshold definitions.
- More third-party audits that publish alongside, not after, a model launch.
- Regulators in the EU and US asking for standard system card contents.
- Evaluation-awareness measurements becoming a routine part of every card.
Key takeaways
- A system card is a lab-written report on capabilities, risks and safeguards; the 2018 model card paper was its ancestor.
- Check which model version and which safeguard setting each number refers to.
- The limitations and admissions sections are often the most informative.
- Separate lab-run results from independent ones.
- A card is a starting point for questions, not proof of safety.
Frequently asked questions
What is the difference between a model card and a system card?
A model card, as proposed in 2018, is a short document on one model’s performance and intended use. A system card is usually longer and covers a model family, its safety evaluations and the safeguards around it.
Are system cards required by law?
Not generally. Most are voluntary, although some regulators and commitments encourage similar documentation. Check the rules that apply in your jurisdiction.
How long does a system card take to read?
A focused read of the key sections takes about 30 minutes. Full cards can run to well over a hundred pages, so use the summary and limitation sections as an entry point.
