GPUs were built for video games, but the matrix multiplications at the heart of neural networks map perfectly onto their parallel design. Data center GPUs add large high-bandwidth memory and fast interconnects so thousands can work together.
How much GPU memory you need depends on model size and precision. See GPU memory sizing for LLMs.