CS224u: Natural Language Understanding
Code for the Stanford course.
Spring 2023
Core components
setup.ipynb
Details on how to get set up to work with this code.
hw_*.ipynb
The set of homeworks for the current run of the course.
tutorial_* notebooks
Introductions to Juypter notebooks, scientific computing with NumPy and friends, and PyTorch.
torch_*.py modules
A generic optimization class (torch_model_base.py) and subclasses for GloVe, Autoencoders, shallow neural classifiers, RNN classifiers, tree-structured networks, and grounded natural language generation.
tutorial_pytorch_models.ipynb shows how to use these modules as a general framework for creating original systems.
evaluation_*.ipynb and projects.md
Notebooks covering key experimental methods and practical considerations, and tips on writing up and presenting work in the field.
iit* and feature_attribution.ipynb
Part of our unit on explainability and model analysis.
np_*.py modules
This is now considered background material for the course.
Reference implementations for the torch_*.py models, designed to reveal more about how the optimization process works.
vsm_*
This is now considered background material for the course.
A unit on vector space models of meaning, covering traditional methods like PMI and LSA as well as newer methods like Autoencoders and GloVe. vsm.py provides a lot of the core functionality, and torch_glove.py and torch_autoencoder.py are the learned models that we cover. vsm_03_contextualreps.ipynb explores methods for deriving static representations from contextual models.
sst_*
This is now considered background material for the course.
A unit on sentiment analysis with the English Stanford Sentiment Treebank. The core code is sst.py, which includes a flexible experimental framework. All the PyTorch classifiers are put to use as well: torch_shallow_neural_network.py, torch_rnn_classifier.py, and torch_tree_nn.py.
finetuning.ipynb
This is now considered background material for the course.
Using pretrained parameters from Hugging Face for featurization and fine-tuning.
utils.py
Miscellaneous core functions used throughout the code.
test/
To run these tests, use
py.test -vv test/*
or, for just the tests in test_shallow_neural_classifiers.py,
py.test -vv test/test_shallow_neural_classifiers.py
If the above commands don't work, try
python3 -m pytest -vv test/test_shallow_neural_classifiers.py
License
The materials in this repo are licensed under the Apache 2.0 license and a Creative Commons Attribution-ShareAlike 4.0 International license.