Scaling laws are measured patterns, not physical laws. Researchers train many models of different sizes, plot their loss against compute, data and parameters, and find that the curves are smooth enough to extrapolate. Early work from OpenAI (2020) described power-law trends, and DeepMind’s “Chinchilla” study (2022) argued that many large models were undertrained for their size and that data and parameters should grow roughly together.
Labs use scaling laws to decide how big a training run should be before spending the compute. They do not guarantee that a particular skill will appear at a given scale, and recent work has focused on other levers such as better data, reinforcement learning and extra compute at inference time.