How to Contribute
Sign up · Log in
7 months ago
R-NET - Machine Reading Comprehension with Self-matching Networks
[ Edit ]
R-NET is an end-to-end trained neural network model for machine comprehension.
It starts by matching the question and the given passage (using gated attention based RNN) to obtain question-aware passage representation.
Next, it uses a self-matching attention mechanism to refine the passage representation by matching the passage against itself.
Lastly, it uses pointer networks to determine the position of the answer in the passage.
Link to the paper
Question / Passage Encoder
Concatenate the word level and character level embeddings for each word and feed into a bidirectional GRU to obtain question and passage representation.
Gated Attention based RNN
Given question and passage representation, sentence pair representation is generated via soft-alignment of the words in the question and in the passage.
The newly added gate captures the relation between the question and the current passage word as only some parts of the passage are relevant for answering the given question.
Self Matching Attention
The passage representation obtained so far would not capture most of the context.
So the current representation is matched against itself so as to collect evidence from the entire passage and encode the evidence relevant to the current passage word and question.
Use pointer network (initialized using attention pooling over answer representation) to predict the position of the answer.
Loss function is the sum of negative log probabilities of start and end positions.
R-NET is ranked second on
as of 7th August, 2017 and achieves best-published results on MS-MARCO dataset.
Using ideas like sentence ranking, using syntax information performing multihop inference and augmenting question dataset (using seqToseq network) do not help in improving the performance.
Read more…(278 words)
About the contributor:
Analytics and Data Science team @ Adobe Systems
Join the discussion. Add a reply…
Table of contents
Ready to join our community?
Sign up below to automatically get notified of new courses, get
to finish ones you subscribe to, and
lessons to read later.
Continue with Facebook
— OR —
Your Full Name
I have an account. Log in instead
By signing up, you agree to our
Natural Language Processing
UX & UI Design
How to Contribute
Upgrade to Pro
Help and FAQ
Privacy, Terms & Refunds
Get in touch
Copyright 2016-18, Compose Labs Inc. All rights reserved.