Selected main references for each topics

[TOC]

Caveat.

This is an on-going list of essential paper based on coarse-grained topics. Note that, the following papers are not about a complete collection of all the related papers in the following main conferences (ACL, NIPS, ICML, ICLR, EMNLP, EACL, COLING), the criterion with which I select papers is to sieve less motivative and experimentally-weak papers and recommend those that are seminal, of high-quality, or at least well-written-for-understanding ones.

However, in terms of a general review of current trend, I list all the papers from 2017 conferences and mark the good ones with a start symbol (*). For papers before 2017, I just select according to the above standard.

Neural Networks and Deep Learning

1. Word2vec and Semantic Unit Representation

• (*)Efficient Estimation of Word Representations in Vectors Space, 2013.
• (*)Distributed Representations of Words and Phrases and their Compositionality, NIPS 2013.
  • Original paper to efficiently learn word embeddings based on neural network language models.
    • Continuous Bag-of-words or skip-gram with negative sampling or hierarchical softmax loss.
• Word2vec Parameter Learning Explained, 2014.
  • Tutorial paper, well-written to explain negative sampling objective and hierarchical softmax objective.
• (*)GloVe: Global Vectors for Word Representation, EMNLP 2014.
  • Different from word2vec learning, which use SGD to learn local bias, but to learn global bias across corpus, so matrix decomposition method strikes back.
• Neural Word Embedding as Implicit Matrix Factorization, NIPS 2014.
  • Explaining learning mechanism behind Word2vec.

2. Attention Mechanism

• (*)Neural Machine Translationi by Jointly Learning to Align and Translate, ICLR 2014.
  • Original paper that propose attention mechanism for NMT.
• (*)Effective Approaches to Attention-based Neural Machine Translation, EMNLP 2015.
  • Solid work on trying different attention mechanisms.
• A Structured Self-Attentive Sentence Embedding, ICLR 2017.
  • Self-attention which has been proposed before, but this paper explicitly uses self-attention to do representation learning.
• Some blogs for better understanding attention.

3. Memory Networks

• (*)Memory Networks, 2014. ICLR 2015. Facebook AI Research.
  • Original paper which motivates many memory-based deep learning architectures and paradigms, such as neural turing machine, one-shot learning etc.
• (*)End-to-End Memory Networks, NIPS 2015. Facebook AI Research.
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, ICML 2016. Salesforce.
• (*)Key-Value Memory Networks for Directly Reading Documents, EMNLP 2016. Facebook AI Research.
• (*)Neural Turing Machines, 2014. Google DeepMind.
  • Original paper which motivates:
    • many addressing mechanisms (kind of attention) for memory utilizing;
    • many learning-to-program ideas in literature.
• (*)Hybrid Computing using a Neural Network with Dynamic External Memory, Nature Oct. 2016. DeepMind.
  • Nature paper on using Memory and Neural Turing Machine alike techniques.
• Pointer Networks, NIPS 2015. Google Research.
• Hierarchical Memory Networks, rejected by NIPS 2016. MILA Bengio’s group.
  • Hashable memory.
• Neural Random-Access Machine, ICLR 2016. Google.
• Neural Programmer-Interpreters, ICLR 2016. DeepMind.

4. Gating Mechanism

[TO-DO]

Topic models

[TO-DO]

1. Latent Semantic Analysis

2. Latent Dirichlet Allocation

3. Sampling-based Learning/Inference

4. Variational Methods

Sentiment Analysis

• (*)Linguistic Regularized LSTM for Sentiment Classification, ACL 2017.
• Active Sentiment Domain Adaptation, ACL 2017.
• Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation, ACL 2017.
• Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis, EACL 2017.
• Attention Modeling for Targeted Sentiment, EACL 2017.
• Structural Attention Neural Networks for Improved Sentiment Analysis, EACL 2017.
• Recurrent Attention Network on Memory for Aspect Sentiment Analysis, EMNLP 2017.
• A Cognition Based Attention Model for Sentiment Analysis, EMNLP 2017.
• Towards a Universal Sentiment Classifier in Multiple Languages, EMNLP 2017.
• (*)Tensor Fusion Network for Multimodal Sentiment Analysis, EMNLP 2017.
• (*)Document-level Multi-Aspect Sentiment Classification as Machine Comprehension, EMNLP 2017.
• Capturing User and Product Information for Document Level Sentiment Analysis with Deep Memory Network, EMNLP 2017.
• Sentiment Lexicon Construction with Representation Learning Based on Hierarchical Sentiment Supervision, EMNLP 2017.
• Refining Word Embeddings for Sentiment Analysis, EMNLP 2017.

Knowledge Graph, Relation Extraction

[TO-DO]

Text Generation, Neural Machine Translation

[TO-DO]

Representation Learning and Domain Adaptation

[TO-DO]

QA-based Dialogue/Dialogue-based QA

• Key-Value Retrieval Networks for Task-Oriented Dialogue, SIGDIAL 2017.
• (*)Reading Wikipedia to Answer Open-Domain Questions, ACL 2017.
• (*)Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access, ACL 2017.
• (*)Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders, ACL 2017.
• (*)Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses, ACL 2017.
  • Evaluation of dialogue system is notoriously hard. How about learning an evaluation metric? However the same question remains, what information bias can the model learn?
• Neural Belief Tracker: Data-Driven Dialogue State Tracking, ACL 2017. Cambridge Univ.
• Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots, ACL 2017. MSRA.
• Adversarial Learning for Neural Dialogue Generation, EMNLP 2017.
• Towards Implicit Content-Introducing for Generative Short-Text Conversation Systems, EMNLP 2017.
  • Hierarchical gated fusion unit. (Architecture design)
• Affordable On-line Dialogue Policy Learning, EMNLP 2017.
  • Human-in-the-loop.
• Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models, EMNLP 2017.
  • Self-attention, single turn.
• Composite Task-completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning, MSR.
  • Hierarchical RL, task-oriented dialogue.
• Agent-aware Dropout DQN for Safe and Efficient On-line Dialogue Policy Learning, EMNLP 2017.
  • Human-in-the-loop.
• Bootstrapping incremental dialogue systems from minimal data: the generalization power of dialogue grammar, EMNLP 2017.
  • Experimental paper, good point to use compositionality of grammar to better generalize.
• Teaching machines to converse, Jiwei Li’s PhD thesis from Stanford Univ. 10/13/2017.

---