Learn-Natural-Language-Processing-Curriculum
This is the curriculum for "Learn Natural Language Processing" by Siraj Raval on Youtube
Course Objective
This is the Curriculum for this video on Learn Natural Language Processing by Siraj Raval on Youtube. After completing this course, start your own startup, do consulting work, or find a full-time job related to NLP. Remember to believe in your ability to learn. You can learn NLP , you will learn NLP, and if you stick to it, eventually you will master it.
Find a study buddy
Join the #NLP_curriculum channel in our Slack channel to find one http://wizards.herokuapp.com
Components each week
- Video Lectures
- Reading Assignments
- Project(s)
Course Length
- 8 weeks
- 2-3 Hours of Study per Day
Tools Used
- Python, PyTorch, NLTK
Prerequisites
- Learn Python https://www.edx.org/course/introduction-python-data-science-2
- Statistics http://web.mit.edu/~csvoss/Public/usabo/stats_handout.pdf
- Probability https://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf
- Calculus http://tutorial.math.lamar.edu/pdf/Calculus_Cheat_Sheet_All.pdf
- Linear Algebra https://www.souravsengupta.com/cds2016/lectures/Savov_Notes.pdf
Week 1 - Language Terminology + preprocessing techniques
Description:
- Overview of NLP (Pragmatics, Semantics, Syntax, Morphology)
- Text preprocessing (stemmings, lemmatization, tokenization, stopword removal)
Video Lectures
- https://web.stanford.edu/~jurafsky/slp3/ videos 1-2.5
- https://www.youtube.com/watch?v=hyT-BzLyVdU&list=PLDcmCgguL9rxTEz1Rsy6x5NhlBjI8z3Gz
Reading Assignments:
- Ch 1-2 of Speech and Language Processing 3rd ed, slides
Project:
- Look at 1-1 to 3-4 to learn NLTK https://github.com/hb20007/hands-on-nltk-tutorial
- Then use NLTK to perform stemming, lemmatiziation, tokenization, stopword removal on a dataset of your choice
Week 2 - Language Models & Lexicons (pre-deep learning)
Description:
- Lexicons
- Pre-deep learning Statistical Language model pre-deep learning ( HMM, Topic Modeling w LDA)
Video Lectures:
Reading Assignments:
- 4,6,7,8,9,10 from the UWash course
Extra
Project
- https://github.com/TreB1eN/HiddenMarkovModel_Pytorch Build Hidden Markov Model for Weather Prediction in PyTorch
Week 3 - Word Embeddings (Word, sentence, and document)
Video lectures:
Reading Assignments
- Suggested readings from course
Project
- 3 Assignments Visualize and Implement Word2Vec, Create dependency parser all in PyTorch (they are assigments from the stanford course)
Week 4-5 - Deep Sequence Modeling
Description:
- Sequence to Sequence Models (translation, summarization, question answering)
- Attention based models
- Deep Semantic Similarity
Video Lectures
Reading Assignments
- Read this on Deep Semantic Similarity Models https://kishorepv.github.io/DSSM/
- Ch 10 Deep Learning Book on Sequence Modeling http://www.deeplearningbook.org/contents/rnn.html
Project
- 3 Assignments, create a translator and a summarizer. All seq2seq models. In pytorch.
Week 6 - Dialogue Systems
Description
- Speech Recognition
- Dialog Managers, NLU
Video Lectures
Reading Assignments
- Ch 24 of this book https://web.stanford.edu/~jurafsky/slp3/24.pdf
Project
- Create a dialogue system using Pytorch https://github.com/ywk991112/pytorch-chatbot and a task oriented dialogue system using DialogFlow to order food
Week 7 - Transfer Learning
Video Lectures
- My videos on BERT and GPT-2, how to build biomedical startup:
- https://www.youtube.com/watch?v=bDxFvr1gpSU
- https://www.youtube.com/watch?v=J9kbZ5I8gdM
- https://www.youtube.com/watch?v=0n95f-eqZdw
- Transfer learning with BERT/GPT-2/ELMO
Reading Assignments
- http://ruder.io/nlp-imagenet/
- https://lilianweng.github.io/lil-log/2019/01/31/generalized-language-models.html
- http://jalammar.github.io/illustrated-bert/
Project
- Play with this https://github.com/huggingface/pytorch-pretrained-BERT#examples pick 2 models, use it for one of 9 downstream tasks, compare their results.
Week 8 - Future NLP
Description
- Visual Semantics
- Deep Reinforcement Learning
Video Lectures
- CMU Video https://www.youtube.com/watch?v=isxzsAelQX0
- Module 5-6 of this https://www.edx.org/course/natural-language-processing-nlp-3
Reading assignments
- https://cs.stanford.edu/people/karpathy/cvpr2015.pdf
- Hilarious https://medium.com/@yoav.goldberg/an-adversarial-review-of-adversarial-generation-of-natural-language-409ac3378bd7
Project:
- Policy gradient text summarization https://github.com/yaserkl/RLSeq2Seq#policy-gradient-w-self-critic-learning-and-temporal-attention-and-intra-decoder-attention reimplement in pytorch