AI Glossary
The neural network architecture, based on attention, that underlies nearly all modern large language models and many vision and audio models.
Introduced in the 2017 Google paper Attention Is All You Need, the Transformer processes all tokens of a sequence in parallel and uses attention to relate them, which made training on enormous datasets practical. GPT stands for Generative Pre-trained Transformer.
Read how Transformers work.
The transformer architecture behind ChatGPT, Claude and Gemini, explained step by step: tokens, embeddings, attention and why it changed AI in 2017.
thelatentdaily.com Editorial Team3 min read
In 2020, Google showed that a plain transformer applied to image patches can rival convolutional networks. How ViT works, why it needs lots of data and what it enabled.
thelatentdaily.com Editorial Team4 min read