Tag
Every The Latent Daily story, tool, model and resource tagged “Scaling”.
The 2020 OpenAI paper by Kaplan and colleagues showed that model loss falls predictably with size, data and compute. What it found, why it mattered and its limits.
AI Desk4 min read
DeepMind's 2022 Chinchilla paper found that most large language models were undertrained. The 20-tokens-per-parameter rule, its impact and why labs now go beyond it.
AI Desk4 min read
Switch Transformers showed that routing each token to one expert lets models grow to a trillion parameters without proportional compute. How MoE works and why it matters.
AI Desk4 min read