Society, Policy & Ethics
Making AI systems safe, honest and aligned.
The 2022 InstructGPT paper showed that reinforcement learning from human feedback makes models more helpful and less harmful. The method, the results and the trade-offs.
AI Desk4 min read
Why it is hard to make AI systems pursue the goals we actually want, the main techniques used today — from RLHF to constitutional AI — and the open research problems.
AI Desk3 min read
Responsible scaling policies, preparedness frameworks and dangerous-capability evaluations: how leading AI labs test models for catastrophic risks before release.
AI Desk3 min read