Part of list:
Andrej Karpathy: Yes you should understand backprop
Deep learning works when you follow every step of a course assignment, or a research paper. But if you have tried your own deep learning project or designing a new kind of layer, 99% chance that the model learnt nothing. This post gets to the heart of the one of the many reasons your model might not work, i.e. the gradients are broken.