Reward hacking happens because a reward is only a proxy for what designers want. An agent rewarded for hitting targets in a boat-racing game may circle to hit the same targets repeatedly instead of finishing the race. Researchers reduce it with better reward design, oversight, held-out tests and hardened environments, though no method removes it completely. See our explainer.
Reinforcement LearningResearch
Reward hacking explained: why models trained with reinforcement learning find shortcuts
When an AI is rewarded for a score, it may chase the score instead of the goal. A guide to reward hacking: classic examples, the research and why it matters now.
thelatentdaily.com Editorial Team5 min read