Table of lectures

1. Introduction
2. Word Embeddings
3. Contextualized word embeddings
4. Pre-trainging and fine-tuning
5. Semantic role labeling
6. Coreference resolution
7. Semantic parsing
8. Reading comprehension
9. Open-domain QA
10. Relation extraction
11. Summarization
12. Dialogue
13. Task-oriented dialogue
14. Bias in language
15. Annotation artifacts in NLP
16. Adversarial examples
17. Interpretability
18. General linguistic intelligence

Introduction

Topic: Computational Linguistics and Deep Learning

Knowledge to keep in mind

• Deep Learning for NLP
• Mostly problem-driven
• Focused on English NLP

Textbook

• Introduction to Natural Language Processing
• Speech and Language Processing

How to read papers

• Read the papers in context
  • All the papaers are built on top of other papers
• Grasp the key ideas
  • What is the biggest contribution of this paper
  • Why is this paper important?
• Pay attention to the details (both methodology and experiments)

Topics of Interest

NLP problems at different levels:

• Linguistic levels: (speech), words, syntax, semantics
• Intermediate tasks/tools: parts-of-speech, entities, parsing, coreference
• Full applications: sentiment analysis, question answering, dialogue, text summarization, machine translation

Prominent problems in exitsting NLP systems:

• Bias in language data
• Annotation artifacts
• Interpretability
• Adversarial example
• General linguistic intelligence