AI & Machine Learning · reviewed in July 2026
Hyperparameter
A hyperparameter is a machine learning model setting fixed BEFORE training it — like the learning rate or a decision tree's maximum depth — as opposed to the model's parameters (for example a neural network's weights), which are learned automatically during training.
Frequently asked questions
How do you pick the right hyperparameter value?
By trying several combinations and measuring each one's performance on a validation set, using techniques like grid search (trying every combination in a grid) or random search (trying random combinations).
Can you give examples of common hyperparameters?
The learning rate, the number of trees in a random forest, the number of layers in a neural network, and the regularization parameter that controls how much model complexity is penalized.
Why can't hyperparameters be learned the same way parameters are?
Because they define the learning process or structure itself — changing the learning rate mid-training doesn't have the same mathematical meaning as adjusting a weight, so they're chosen upfront or through a separate search process.