"fast weights" can be used to store temporary memories of the recent past and they provide a neurally plausible way of implementing the type of attention to the past that has recently proved very helpful in sequence-to-sequence models.

Here is Geoffrey Hinton's talk on the topic. You can also read Hugo Larochelle's summary of the paper on ShortScience.