What is a learning rate?
The learning rate controls how large each parameter update step is during training.
- Too high can overshoot
- Too low learns slowly
- Often tuned experimentally
What is a learning rate?
Tagged with training
3 cards
What is gradient descent?
Gradient descent updates model parameters in the direction that reduces the loss function.
- Uses gradients of loss
- Step size is learning rate
- Variants include batch and mini-batch
What is gradient descent?
What is transfer learning?
Transfer learning starts from a pretrained model and adapts it to a new but related task.
- Saves training time
- Useful with limited labeled data
- Common in vision and NLP
What is transfer learning?