Part of list:
Recurrent Neural Network Regularization
- Improved algorithm
This article is a summary of this paper, which proposes a method for applying dropout to LSTMs and how it could reduce overfitting in tasks like language modeling, speech recognition, image caption generation and machine translation.
Dropout is a regularization method that drops out (or temporarily removes) units in a neural network (along with all its incoming and outgoing connections). Conventional dropout does not work well with RNNs as the recurrence amplifies the noise and hurts learning.
In the context of language modeling, image caption generation, speech recognition and machine translation, dropout enables training larger networks and reduces the testing error in terms of perplexity and frame accuracy.