AI Glossary
The examples a model learns from, such as text, images, code or labelled records; its quality and coverage largely determine what the model can do.
Large language models are trained on filtered web pages, books, code and licensed or synthetic data, amounting to trillions of tokens. Choices about what to include, filter and deduplicate shape a model’s knowledge, biases and languages.
Who owns training data and whether it may be used without permission is the subject of major copyright lawsuits.
News archives, forums and media libraries are being licensed to AI developers. What these deals include, why they happen and how they relate to copyright fights.
thelatentdaily.com Editorial Team3 min read
DeepMind's 2022 Chinchilla paper found that most large language models were undertrained. The 20-tokens-per-parameter rule, its impact and why labs now go beyond it.
thelatentdaily.com Editorial Team4 min read
EleutherAI's 825 GB Pile combined 22 diverse text sources into an open training set. What it contained, why diversity mattered and the copyright questions it raised.
thelatentdaily.com Editorial Team3 min read