More parameters generally allow more knowledge and capability but need more memory and compute. In mixture-of-experts models, the total parameter count differs from the “active” parameters used for each token.
A rough rule: at 16-bit precision, each billion parameters needs about 2 GB of memory for the weights alone.