AI model evaluation and red teaming have become distinct career tracks: people who measure how well a model performs and people who try to make it fail before attackers do. On November 12, 2026, Anthropic’s public job board listed a Red Team Engineer role at $320,000 to $405,000 a year and a model-evaluation research role at $500,000 to $850,000, while contractor platforms such as Outlier, DataAnnotation and Mercor recruit experts by the hour to rate, rank and test model outputs. This guide explains what the work involves, which skills employers actually ask for, and how to read the pay figures without being misled by them.
What evaluation and red-teaming work involves
The two jobs overlap but start from different questions. Evaluation asks “how good is this model, and is that number trustworthy?” Red teaming asks “how can this system be misused or broken?”
Anthropic’s posting for a Research Engineer in post-training model evaluations describes the evaluation side well. The role involves choosing which measurements say something real about a model during training, noticing when a metric stops being meaningful, monitoring the “eval fleet” while training runs are live, investigating regressions, and building dashboards and alerts that researchers and leadership use to track quality. If you want to see the basic mechanics at a smaller scale, our guide on how to build an evaluation set for LLM apps walks through the same ideas for a single product.
The company’s Red Team Engineer posting describes the adversarial side. The job is to attack Anthropic’s own products the way a sophisticated threat actor would: chaining techniques into “full kill chain” scenarios, probing agent systems and tool use for prompt injection, and looking at abuse such as coordinated account manipulation and payment fraud, not only model jailbreaks. Findings go to product, engineering and policy teams; see our glossary entry on red teaming for the term’s background.
Between those two sits a growing middle tier. Anthropic’s Cyber Evaluations Engineer role, for example, designs evaluations that measure cyber-relevant capabilities in new models, runs safeguard-robustness tests before each major release and builds detection probes for misuse in production.
At the entry level, much of the human evaluation work is done by contractors. Outlier, a platform operated by Scale AI, describes its tasks as writing difficult prompts with correct answers, creating grading rubrics, and rating and ranking AI responses. DataAnnotation lists similar work: reviewing AI answers for accuracy, ranking outputs, refining prompts, fact-checking specialist content and, for coding roles, debugging and evaluating code.
Who hires: three kinds of employers
AI labs. Frontier developers employ full-time evaluation researchers, red-team engineers and the operations staff who manage human data. Anthropic’s Data Operations Manager, Human Data posting, for instance, covers vendor partnerships, quality standards and evaluation frameworks for data used in RLHF, safety and agentic work, and asks for 3+ years in operations, consulting, product or program management rather than a research degree.
Government institutes. The UK AI Security Institute, created after the 2023 Bletchley Park summit, now sits within the Department for Science, Innovation and Technology and hires researchers, engineers and policy analysts, mainly in London. When we checked its careers page it listed no open technical roles, but it describes technical pay packages as a base salary plus a “technical talent allowance”, with an employer pension contribution of 28.97% of base salary.
Contract platforms and expert networks. Outlier says it has paid more than $500 million to experts and that minimum qualifications are typically undergraduate-level expertise, with graduate students and PhD candidates preferred. Mercor’s homepage cites an average contracted rate of $90 an hour and more than $4 million in daily payouts across a network of physicians, lawyers, engineers and other professionals. Labs also run invite-based programs: when OpenAI launched its Red Teaming Network in 2023, SiliconANGLE reported that members would sign nondisclosure agreements, be paid for projects OpenAI commissions, and come from fields such as psychology, healthcare, law and education.
Comparison: pay and requirements by role type
The table below uses only figures published by the employer or platform on the day we checked. Contractor rates are advertised maximums or averages, not guaranteed earnings, and full-time ranges are base salary only.
| Role type | Example source | Published pay | What they ask for |
|---|---|---|---|
| Model-evaluation research engineer | Anthropic, Research Engineer, Post-Training Model Evaluations | $500,000–$850,000 a year | Designing and analyzing ML evaluations at scale, strong Python, statistics and experimental design a plus |
| Red-team engineer | Anthropic, Red Team Engineer, Safeguards | $320,000–$405,000 a year | Penetration testing or red teaming, model jailbreaking, prompt-injection testing, a public body of work such as CVEs or bug bounty reports |
| Cyber evaluations engineer | Anthropic, Cyber Evaluations Engineer | $300,000–$405,000 a year | Building evals or test suites, hands-on security work such as CTFs or vulnerability research, Python |
| Human data operations | Anthropic, Data Operations Manager, Human Data | $270,000–$365,000 a year | 3+ years in operations, consulting, product or program management, familiarity with how LLMs work |
| Expert contractor (coding, ML) | DataAnnotation | Listed as “$150+” an hour | Subject expertise, passing a qualification process |
| Expert contractor (law, medicine, STEM) | DataAnnotation | Listed as “$125+” an hour | Subject expertise |
| Generalist / language contractor | DataAnnotation | $50 and $40 an hour | Reasoning, attention to detail; no coding needed for general projects |
Two cautions apply. First, these Anthropic ranges come from one San Francisco–based company at the top of the market; they are not typical industry pay. Second, the U.S. Bureau of Labor Statistics has no occupation called “AI evaluator” or “red teamer”. The closest benchmarks are computer and information research scientists, whose 2025 median pay was $140,300 a year with 22% projected growth from 2025 to 2035, and data scientists, with a median of $120,230 and 35% projected growth over the same period.
Worked example: a six-month skills plan
Suppose you are a software developer with some security experience who wants to move into evaluation or red teaming. Based on the qualifications that appear repeatedly in the postings above, a realistic plan might look like this:
- Months 1–2: measurement basics. Build a small evaluation set for an open model on a task you know, score it with both automatic metrics and a rubric, and write up where the numbers disagreed with your own judgment. The evaluation research posting explicitly values “thinking twice before trusting a number”.
- Months 2–3: statistics. Learn enough experimental design to report confidence intervals and to spot when a benchmark change is noise. Anthropic lists statistics and experimental design as a strong-candidate signal.
- Months 3–4: adversarial practice. Take part in capture-the-flag events and public AI red-teaming challenges, and document prompt-injection tests against an agent you build yourself. The red-team posting asks for a public body of work, so publish write-ups.
- Months 4–5: tooling. Automate your tests in Python so they can run on every model version. Both the red-team and cyber-evaluation postings ask for custom automation or eval tooling.
- Months 5–6: portfolio and paid practice. Package two or three projects using the approach in our guide to building an AI portfolio that shows real skills, and consider contract evaluation work to gain exposure to real rubrics while you apply.
What to watch next
- Whether more AI developers publish salary ranges for evaluation and safety roles, which would make better market comparisons possible.
- Whether the UK AI Security Institute and similar government bodies reopen technical hiring.
- How contractor platforms change pay and qualification rules as labs demand more specialist experts.
- Whether BLS or other statistical agencies begin to track AI evaluation work as a separate occupation.
Key takeaways
- Evaluation measures how good a model is; red teaming tries to break or misuse it, and the two roles increasingly overlap.
- Anthropic’s posted ranges for these roles ran from $270,000 to $850,000 a year on November 12, 2026, but they reflect one top-paying employer.
- BLS medians of $140,300 for computer and information research scientists and $120,230 for data scientists are better anchors for the broader market.
- Contract platforms offer an entry route, with advertised rates from $40 an hour for language work to “$150+” for coding and ML experts.
- Employers repeatedly ask for Python, statistical rigor, security experience and a public record of work.
Frequently asked questions
Do you need a PhD to work in AI model evaluation?
Not necessarily. Anthropic’s evaluation and red-team postings list a bachelor’s degree or an equivalent combination of education and experience as the minimum, and contract platforms such as Outlier typically require undergraduate-level expertise, although graduate degrees are preferred for some tasks.
How much do AI red teamers earn?
Pay varies widely. One frontier lab listed a red-team engineer role at $320,000 to $405,000 a year, while red-teaming programs for outside experts pay per project and do not publish standard rates. There is no official BLS median for the job title.
Is AI data annotation work a good way into AI evaluation careers?
It can give practical experience with rubrics, ranking and fact-checking model outputs, which are the same skills evaluation teams use. It is task-based contract work without long-term commitment, so treat it as a stepping stone and document what you learn in a portfolio.
