Gradient Descent

During machine-learning training, gradient descent uses information about how the loss changes with respect to model parameters and moves those parameters in a direction expected to improve performance. Many modern neural networks are trained with gradient-based variants such as stochastic gradient descent or Adam.