Most neural networks are trained using gradient descent. As such, the gradients of a neural network being good is extremely important. This quiz walks you through reasoning about the goodness of gradients, and the different ways things can go wrong.
Part of course:
Quiz: The Importance of "Good" Gradients