Unlike weights, hyperparameters are not learned from data. Choosing them well can make the difference between a model that trains smoothly and one that diverges. Teams tune them with grid search, random search or automated methods, using a validation set.
Generation settings such as temperature are sometimes also called hyperparameters, though they are applied at inference time.