text-analytics
Unstructured Data Analysis (Graduate) @Korea University
Notice
- Syllabus (download)
- Term Project Presentations and Youtube summary (2021 Spring) (link)
- Term Project Presentations and Youtube summary (2020 Spring) (link)
- Term Project ์ ์๋ฐํ
- Instruction [download]
- Deadline: 2021-03-31 23:59
- Term Project ์ค๊ฐ๋ฐํ
- Instruction [download]
- Deadline: 2021-05-16 23:59
- Term Project ์ต์ข
๋ฐํ
- Instruction [download]
- Deadline: 2021-06-20 23:59
Schedule
- 2021-03-09 (Tue): Term Project Meeting (Round 1)
- 1์กฐ: ๊ฐํ์, ๊น์ง๋, ๊นํ์, ๊น์๋น
- 2์กฐ: ์๊ท์ฑ, ์ด์ค์น, ์ ์์
- 3์กฐ: ๊น์ ์ญ, ํ์ฌํ, ์ต์ ์ฐ, ์คํ์
- 4์กฐ: ์ ์์, ์คํ์ฑ ์กฐ๋ฏผ์
- 2021-03-11 (Thu): Term Project Meeting (Round 1)
- 5์กฐ: ๊น๋๊ท , ์ด์ฐฌํธ, ์ฐจํ์ฃผ
- 6์กฐ: ๊นํ์ฃผ, ์ฒ์ฃผ์, ์ ๋ํ
- 7์กฐ: ํ์ข ๊ตญ, ์์๋ฆฐ, ๊ณ ์์ง, ํฉ์์ฒ
- 8์กฐ: ๊น์๋ฆ, ์ต๋ณ๋ก, ์๋ฏผ์, ์์์ฐ
- 2021-03-16 (Tue): Topic Discussion and QA (Topic 1, 2)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-03-09
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-03-14
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-03-18
- 2021-03-18 (Thu): No class
- 2021-03-23 (Tue): Term Project Meeting (Round 1)
- 9์กฐ: ์ ์ด๊ฒฝ, ๊ณ ์์ฑ, ์กฐ๊ฒฝ์ , ๊น์ง์
- 10์กฐ: ๊นํ์ฐ, ์์ํ, ์คํ๋ น, ์กฐํ์
- 11์กฐ: ์์ธ๋ฒ, ๊น์ ์, ํฉ์ฑ์ง, ๊น์๋ฏผ
- 12์กฐ: ์ ํ์ฐฌ, ์ ์ฐ์, ์์์ฐ, ๋ ธ์๊ท , ๊น์ง์ญ, ๊น์์ญ
- 2021-03-25 (Thu): No class
- 2021-03-30 (Tue): Topic Discussion and QA (Topic 4, 5)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-03-23
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-03-28
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-04-01
- 2021-04-01 (Thu): Term Project Meeting (Round 2)
- 1์กฐ, 2์กฐ, 3์กฐ
- 2021-04-06 (Tue): Topic Discussion and QA (Topic 6)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-03-30
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-04-04
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-04-08
- Term Project Meeting (Round 2)
- 4์กฐ, 5์กฐ, 6์กฐ
- 2021-04-13 (Tue): Topic Discussion and QA (Topic 7)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-04-06
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-04-11
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-04-15
- 2021-04-15 (Thu): Term Project Meeting (Round 2)
- 7์กฐ, 8์กฐ, 9์กฐ
- 2021-04-20 (Tue): Topic Discussion and QA (Topic 8 - Seq2Seq Learning & Transformer)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-04-13
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-04-18
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-04-22
- 2021-04-22 (Thu): Term Project Meeting (Round 2)
- 10์กฐ, 11์กฐ, 12์กฐ
- 2021-04-27 (Tue): Topic Discussion and QA (Topic 8 - ELMo, GPT, BERT)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-04-20
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-04-25
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-04-29
- 2021-04-29 (Thu): Term Project Meeting (Round 3)
- 1์กฐ, 2์กฐ, 3์กฐ, 4์กฐ
- 2021-05-04 (Tue): Topic Discussion and QA (Topic 8 - GPT-2, Transformer to T5, GPT-3)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-04-27
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-05-01
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-05-06
- Term Project Meeting (Round 3)
- 5์กฐ, 6์กฐ, 7์กฐ, 8์กฐ
- 2021-05-06 (Thu): Term Project Meeting (Round 3)
- 9์กฐ, 10์กฐ, 11์กฐ, 12์กฐ
- 2021-05-11 (Tue): Topic Discussion and QA (Topic 9 - Document Classification)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-05-04
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-05-09
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-05-13
- 2021-05-13 (Thu): Term Project ์ค๊ฐ๋ฐํ ์ค๋น
- 2021-05-18 (Tue): Topic Discussion and QA (Topic 10 - Sentiment Analysis)
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-05-11
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-05-16
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-05-20
- 2021-05-20 (Thu): Term Project Meeting (Round 4)
- 1์กฐ, 2์กฐ, 3์กฐ, 4์กฐ
- 2021-05-25 (Tue): Topic Discussion and QA (Topic 11 - Text Summairzation (Extractive Summarization))
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-05-18
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-05-23
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-05-27
- 2021-05-27 (Thu): Term Project Meeting (Round 4)
- 5์กฐ, 6์กฐ, 7์กฐ, 8์กฐ
- 2021-06-01 (Tue) & 2021-06-03 (Thu): ์ฐ์ ๊ณตํํ ์ถ๊ณ๊ณต๋ํ์ ๋ํ
- 2021-06-08 (Tue): Topic Discussion and QA (Topic 11 - Text Summairzation (Abstractive Summarization))
- ๋์์ ์์ฒญ ์๋ฃ ๋ฐ ์ง๋ฌธ ๋ฑ๋ก: 2021-06-01
- ์ง๋ฌธ์ ๋ํ ์๊ฐ์ ๋ต๋ณ ๋ฑ๋ก: 2021-06-06
- ์ ํ๋ธ ์์ฝ ์์ ์ ๋ก๋: 2021-06-10
- 2021-06-10 (Thu): Term Project Meeting (Round 4)
- 9์กฐ, 10์กฐ, 11์กฐ, 12์กฐ
- 2021-06-17 (Thu): Term Project ์ต์ข ๋ฐํ
Recommended courses
- CS224d @Stanford: Deep Learning for Natural Language Processing
- Course Homepage: http://cs224d.stanford.edu/
- YouTube Video: https://www.youtube.com/playlist?list=PLlJy-eBtNFt4CSVWYqscHDdP58M3zFHIG
- DSBA Lab Seminar (2017) Playlist: https://youtube.com/playlist?list=PLetSlH8YjIfUf3gBv1JpCLa9WVaL8yE0-
- CS224n @Stanford: Natural Language Processing Deep Learning
- Course Homepage: http://web.stanford.edu/class/cs224n/
- Youtube Video: https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6
- DSBA Lab Seminar (2019) Playlist: https://youtube.com/playlist?list=PLetSlH8YjIfVdobI2IkAQnNTb1Bt5Ji9U
- Deep Natural Lanugage Processing @Oxford
- Course Homepage: https://github.com/oxford-cs-deepnlp-2017/lectures
- DSBA Lab Seminar (2017) Playlist: https://youtube.com/playlist?list=PLetSlH8YjIfVK2_VTNNHnei0PluGv4Vfn
Contents
Topic 1: Introduction to Text Analytics
- Text Analytics: Backgrounds, Applications, & Challanges, and Process [Slide], [Video] (2021-03-09)
- Text Analytics Process [Slide], [Video] (2021-03-09)
- Q & A Session [Slide]
- Reading materials
- NLP Year in Review - 2019 (Medium Post)
- Natural Language Processing in 2020: The Year In Review (Linkedin Post)
Topic 2: Text Preprocessing
- Introduction to Natural Language Processing (NLP) [Slide], [Video] (2021-03-09)
- Lexical analysis [Slide], [Video] (2021-03-09)
- Syntax analysis & Other topics in NLP [Slide], [Video] (2021-03-09)
- Reading materials
- Cambria, E., & White, B. (2014). Jumping NLP curves: A review of natural language processing research. IEEE Computational intelligence magazine, 9(2), 48-57. (PDF)
- Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug), 2493-2537. (PDF)
- Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017). Recent trends in deep learning based natural language processing. arXiv preprint arXiv:1708.02709. (PDF)
Topic 3: Neural Networks Basics (Optional, No Video Lectures)
- Perception, Multi-layered Perceptron
- Convolutional Neural Networks (CNN)
- Recurrent Neural Networks (RNN)
- Practical Techniques
Topic 4: Text Representation I: Classic Methods
Topic 5: Text Representation II: Distributed Representation
- Neural Network Language Model (NNLM) [Slide], [Video] (2021-03-23)
- Word2Vec [Slide], [Video], [Optional Video (๋ฐํ์: ๊น์ง๋)] (2021-03-23)
- GloVe [Slide], [Video], [Optional Video (๋ฐํ์ ์กฐ๊ท์)] (2021-03-23)
- FastText, Doc2Vec, and Other Embeddings [Slide], [Video] [Optional Video (๋ฐํ์: ๊น์๋น] (2021-03-23)
- Q & A Session [Slide]
- (Optional) Analogies Explained: Towards Understanding Word Embeddings [Video (๋ฐํ์: ๊น๋ช ์ญ)]
- Reading materials
- Bengio, Y., Ducharme, R., Vincent, P., & Jauvin, C. (2003). A neural probabilistic language model. Journal of machine learning research, 3(Feb), 1137-1155. (PDF)
- Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. (PDF)
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111-3119). (PDF)
- Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543). (PDF)
- Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2016). Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. (PDF)
Topic 6: Dimensionality Reduction
- Dimensionality Reduction Overview, Supervised Feature Selection [Slide], [Video] (2021-03-30)
- Unsupervised Feature Extraction [Slide], [Video] (2021-03-30)
- Reading materials
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American society for information science, 41(6), 391-407. (PDF)
- Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse processes, 25(2-3), 259-284. (PDF)
- Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of machine learning research, 9(Nov), 2579-2605. (PDF) (Homepage)
Topic 7: Topic Modeling as a Distributed Reprentation
- Topic modeling overview & Latent Semantic Analysis (LSA), Probabilistic Latent Semantic Analysis: pLSA [Slide], [Video] (2021-04-06)
- LDA: Document Generation Process [Slide], [Video] (2021-04-06)
- LDA Inference: Collapsed Gibbs Sampling, LDA Evaluation [Slide], [Video] (2021-04-06)
- Q & A Session [Slide]
- Reading Materials
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American society for information science, 41(6), 391. (PDF)
- Dumais, S. T. (2004). Latent semantic analysis. Annual review of information science and technology, 38(1), 188-230.
- Hofmann, T. (1999, July). Probabilistic latent semantic analysis. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence (pp. 289-296). Morgan Kaufmann Publishers Inc. (PDF)
- Hofmann, T. (2017, August). Probabilistic latent semantic indexing. In ACM SIGIR Forum (Vol. 51, No. 2, pp. 211-218). ACM.
- Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84. (PDF)
- Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022. (PDF) [Optional Video (๋ฐํ์: ์คํ์)]
- Recommended video lectures
- LDA by D. Blei (Lecture Video)
- Variational Inference for LDA by D. Blei (Lecture Video)
Topic 8: Language Modeling & Pre-trained Models
- Sequence-to-Sequence Learning [Slide], [Video] (2021-04-13)
- Transformer [Slide], [Video], [Optional Video (๋ฐํ์: ๊น๋ํ)], [Optional Video (๋ฐํ์: ์๊ท์ฑ)] (2021-04-13)
- (Optional) Reformer: The Efficient Transformer [Video (๋ฐํ์: ๊น์ ํฌ)]
- (Optional) Multimodal Transformer for Unaligned Multimodal Language Sequences [Video (๋ฐํ์: ๋ ธ์๋น)]
- ELMo: Embeddings from Language Models [Slide], [Video] (2021-04-20)
- GPT: Generative Pre-Training of a Language Model [Slide], [Video] (2021-04-20)
- BERT: Bidirectional Encoder Representations from Transformer [Slide], [Video] (2021-04-20)
- (Optional) Semantics-aware BERT for Language Understanding [Video (๋ฐํ์: ๊น๋ช ์ญ)]
- GPT-2: Language Models are Unsupervised Multitask Learners [Slide], [Video] (2021-04-27)
- Transformer to T5 (XLNet, RoBERTa, MASS, BART, MT-DNN,T5) [Video (๋ฐํ์: ์ด์ ๊ฒฝ)] (2021-04-27)
- GPT-3: Language Models are Few Shot Learners [Slide], [Video] (2021-04-27)
- (Optional) How Contextual are Contextualized Word Representations? Comparing the Geoetry of BERT, ELMo, and GPT-2 Embeddings [Slide], [Video (๋ฐํ์: ์ด์ ๊ฒฝ)]
- (Optional) Syntax and Semantics in Language Model Representation [Video (๋ฐํ์: ๊น๋ช ์ญ)]
- Reading Materials
- Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112). (PDF)
- Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. (PDF)
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008). (PDF)
- Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365. (PDF)
- Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. (PDF)
- Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. (PDF)
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 9. (PDF)
- Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237. (PDF)
- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. (PDF)
- Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. Y. (2019). Mass: Masked sequence to sequence pre-training for language generation. arXiv preprint arXiv:1905.02450. (PDF)
- Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., ... & Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. (PDF)
- Liu, X., He, P., Chen, W., & Gao, J. (2019). Multi-task deep neural networks for natural language understanding. arXiv preprint arXiv:1901.11504. (PDF)
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683. (PDF)
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165. (PDF)
Topic 9: Document Classification
- Document classification overview, Vector Space Models (Naive Bayesian Classifier, k-Nearese Neighbor Classifier) [Slide], [Video] (2021-05-04)
- (Optional) Other VSM-based classsification (Lecture videos are taken from IMEN415 (Multivariate Data Analysis for Undergraudate Students @Korea University))
- Logistic Regression: [Formulation], [Learning], [Interpretation]
- Decision Tree: [Recursive Partitioning and Pruning]
- Artificial Neural Network: [Perceptron], [Multi-layer Perceptron]
- Ensemble Models" [Overview], [Bagging], [Random Forest], [AdaBoost], [Gradient Boosting Machine (GBM)]
- CNN-based document classification [Slide], [Video], [(Optional) ๋ฐํ์ ์ด๊ธฐ์ฐฝ] (2021-05-04)
- RNN-based document classification [Slide], [Video] (2021-05-04)
- Reading materials
- Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. (PDF)
- Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. In Advances in neural information processing systems (pp. 649-657) (PDF)
- Lee, G., Jeong, J., Seo, S., Kim, C, & Kang, P. (2018). Sentiment classification with word localization based on weakly supervised learning with a convolutional neural network. Knowledge-Based Systems, 152, 70-82. (PDF)
- Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1480-1489). (PDF)
- Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. (PDF)
- Luong, M. T., Pham, H., & Manning, C. D. (2015). Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. (PDF)
Topic 10: Sentiment Analysis
- Architecture of sentiment analysis [Slide], [Video] (2021-05-11)
- Lexicon-based approach [Slide], [Video] (2021-05-11)
- Machine learning-based approach [Slide], [Video] (2021-05-11)
- Reading materials
- Hamilton, W. L., Clark, K., Leskovec, J., & Jurafsky, D. (2016, November). Inducing domain-specific sentiment lexicons from unlabeled corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing (Vol. 2016, p. 595). NIH Public Access. (PDF)
- Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1253. (PDF)
Topic 11: Text Summarization
- Text summarization Overview [Slide], [Video] (2021-05-18)
- Extractive Summarization 1: Graph-based Summarization (TextRank, LexRank, DivRank) [Slide], [Video] (2021-05-18)
- Extractive Summarization 2: Neural Network-based Summarization (SummaRuNNer, NeuSum) [Slide], [Video] (2021-05-18)
- (Optional) Single Document Summarization as Tree Induction [Video (๋ฐํ์: ์ต์ข ํ)]
- (Optional) Extractive summarization as Text Matching [Video (๋ฐํ์: ์ด์ ๊ฒฝ)]
- Abstractive Summarization [Slide], [Video] (2021-06-01)
- Extractive/Abstractive Summarization (2021-06-01)
- Text Summarization with Pretrained Encoder (BERTSum) [Video (๋ฐํ์: ์ด์ ํ)]
- Reading materials
- Lin, C. Y. (2004, July). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out (pp. 74-81). (PDF)
- Ganesan, K. (2018). Rouge 2.0: Updated and improved measures for evaluation of summarization tasks. arXiv preprint arXiv:1803.01937. (PDF)
- Mihalcea, R., & Tarau, P. (2004, July). Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing (pp. 404-411). (PDF)
- Erkan, G., & Radev, D. R. (2004). Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research, 22, 457-479. (PDF)
- Mei, Q., Guo, J., & Radev, D. (2010, July). Divrank: the interplay of prestige and diversity in information networks. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 1009-1018). (PDF)
- Nallapati, R., Zhai, F., & Zhou, B. (2017, February). Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 31, No. 1). (PDF)
- Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M., & Zhao, T. (2018). Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305. (PDF)
- Nallapati, R., Zhou, B., Gulcehre, C., & Xiang, B. (2016). Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023. (PDF)
- See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. (PDF)
Topic 12: Question and Answering (Optional)
- Question and Answering (CS224N) [Video (๋ฐํ์: ์กฐ๊ท์)]
- SQuAD: 100,000+ questions for machine comprehension of text [Video (๋ฐํ์: ๊นํ์)]
- Know what you don't know: Unanswerable questions for SQuAD [Video (๋ฐํ์: ๊นํ์)]
- End-To-End Memory Networks [Video (๋ฐํ์: ํ์ฌํ)]
- Ask Me Anything: Dynamic Memory Networks for Natural Language Processing [Video (๋ฐํ์: ๊น์ง๋)]
- (Optional) A Regurrent BERT-based Model for Question Generation & Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds [Video (๋ฐํ์: ์กฐ๊ท์)]
- (Optional) VQA: Visual Question Answering [Video (๋ฐํ์: ์ด์ค์น)]
- Reading Materials
- Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250. (PDF)
- Rajpurkar, P., Jia, R., & Liang, P. (2018). Know what you don't know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822. (PDF)
- Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015). End-to-end memory networks. arXiv preprint arXiv:1503.08895. (PDF)
- Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., ... & Socher, R. (2016, June). Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning (pp. 1378-1387). PMLR. (PDF)
Topic 13: (Open) Information Extraction (Optional)
- Open Information Extraction [Video (๋ฐํ์: ๋ ธ์๋น)]