ML Papers
This repo contains notes and short summaries of some ML related papers I come across, organized by subjects and the summaries are in the form of PDFs.
Self-Supervised & Contrastive Learning
- Self-Supervised Relational Reasoning for Representation Learning (2020): [Paper] [Notes]
- Big Self-Supervised Models are Strong Semi-Supervised Learners (2020) [Paper] [Notes]
- Debiased Contrastive Learning (2020) [Paper] [Notes]
- Selfie: Self-supervised Pretraining for Image Embedding (2019): [Paper] [Notes]
- Self-Supervised Representation Learning by Rotation Feature Decoupling (2019): [Paper] [Notes]
- Revisiting Self-Supervised Visual Representation Learning (2019): [Paper] [Notes]
- AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations (2019): [Paper] [Notes]
- Boosting Self-Supervised Learning via Knowledge Transfer (2018): [Paper] [Notes]
- Self-Supervised Feature Learning by Learning to Spot Artifacts (2018): [Paper] [Notes]
- Unsupervised Representation Learning by Predicting Image Rotations (2018): [Paper] [Notes]
- Cross Pixel Optical-Flow Similarity for Self-Supervised Learning (2018): [Paper] [Notes]
- Multi-task Self-Supervised Visual Learning (2017): [Paper] [Notes]
- Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction (2017): [Paper] [Notes]
- Colorization as a Proxy Task for Visual Understanding (2017): [Paper] [Notes]
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2017): [Paper] [Notes]
- Unsupervised Visual Representation Learning by Context Prediction (2016): [Paper] [Notes]
- Colorful image colorization (2016): [Paper] [Notes]
- Learning visual groups from co-occurrences in space and time (2015): [Paper] [Notes]
- Discriminative unsupervised feature learning with exemplar convolutional neural networks (2015): [Paper] [Notes]
Semi-Supervised Learning
- Negative sampling in semi-supervised learning (2020): [Paper] [Notes]
- Time-Consistent Self-Supervision for Semi-Supervised Learning (2020): [Paper] [Notes]
- Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learning (2019): [Paper] [Notes]
- S4L: Self-Supervised Semi-Supervised Learning (2019): [Paper] [Notes]
- Semi-Supervised Learning by Augmented Distribution Alignment (2019): [Paper] [Notes]
- MixMatch: A Holistic Approach toSemi-Supervised Learning (2019): [Paper] [Notes]
- Unsupervised Data Augmentation (2019): [Paper] [Notes]
- Interpolation Consistency Training for Semi-Supervised Learning (2019): [Paper] [Notes]
- Deep Co-Training for Semi-Supervised Image Recognition (2018): [Paper] [Notes]
- Unifying semi-supervised and robust learning by mixup (2019): [Paper] [Notes]
- Realistic Evaluation of Deep Semi-Supervised Learning Algorithms (2018): [Paper] [Notes]
- Semi-Supervised Sequence Modeling with Cross-View Training (2018): [Paper] [Notes]
- Virtual Adversarial Training (2017): [Paper] [Notes]
- Mean teachers are better role models (2017): [Paper] [Notes]
- Temporal Ensembling for Semi-Supervised Learning (2017): [Paper] [Notes]
- Semi-Supervised Learning with Ladder Networks (2015): [Paper] [Notes]
Video Understanding
- Multiscale Vision Transformers (2021): [Paper] [Notes]
- ViViT A Video Vision Transformer (2021): [Paper] [Notes]
- Space-time Mixing Attention for Video Transformer (2021): [Paper] [Notes]
- Is Space-Time Attention All You Need for Video Understanding (2021): [Paper] [Notes]
- An Image is Worth 16x16 Words What is a Video Worth (2021): [Paper] [Notes]
- Temporal Query Networks for Fine-grained Video Understanding (2021): [Paper] [Notes]
- X3D Expanding Architectures for Efficient Video Recognition (2020): [Paper] [Notes]
- Temporal Pyramid Network for Action Recognition (2020): [Paper] [Notes]
- STM SpatioTemporal and Motion Encoding for Action Recognition (2019): [Paper] [Notes]
- Video Classification with Channel-Separated Convolutional Networks (2019): [Paper] [Notes]
- Video Modeling with Correlation Networks (2019): [Paper] [Notes]
- Videos as Space-Time Region Graphs (2018): [Paper] [Notes]
- SlowFast Networks for Video Recognition (2018): [Paper] [Notes]
- TSM Temporal Shift Module for Efficient Video Understanding (2018): [Paper] [Notes]
- Timeception for Complex Action Recognition (2018): [Paper] [Notes]
- Non-local Neural Networks (2017): [Paper] [Notes]
- Temporal Segment Networks for Action Recognition in Videos. (2017): [Paper] [Notes]
- Quo Vadis Action Recognition A New Model and the Kinetics Dataset (2017): [Paper] [Notes]
- A Closer Look at Spatiotemporal Convolutions for Action Recognition (2017): [Paper] [Notes]
- ActionVLAD Learning spatio-temporal aggregation for action classification (2017): [Paper] [Notes]
- Spatiotemporal Residual Networks for Video Action Recognition (2016): [Paper] [Notes]
- Deep Temporal Linear Encoding Networks (2016): [Paper] [Notes]
- Temporal Convolutional Networks for Action Segmentation and Detection (2016): [Paper] [Notes]
- Learning Spatiotemporal Features with 3D Convolutional Network (2014): [Paper] [Notes]
Domain Adaptation, Domain & Out-of-Distribution Generalization
- Rethinking Distributional Matching Based Domain Adaptation (2020): [Paper] [Notes]
- Transferability vs. Discriminability: Batch Spectral Penalization (2019): [Paper] [Notes]
- On Learning Invariant Representations for Domain Adaptation (2019): [Paper] [Notes]
- Universal Domain Adaptation (2019): [Paper] [Notes]
- Transferable Adversarial Training (2019): [Paper] [Notes]
- Multi-Adversarial Domain Adaptation (2018): [Paper] [Notes]
- Conditional Adversarial Domain Adaptation (2018): [Paper] [Notes]
- Learning Adversarially Fair and Transferable Representations (2018): [Paper] [Notes]
- What is the Effect of Importance Weighting in Deep Learning? (2018): [Paper] [Notes]
Explainability
- Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models (2021): [Paper] [Notes]
- Transformer Interpretability Beyond Attention Visualization (2020): [Paper] [Notes]
- What shapes feature representations Exploring datasets architectures and training (2020): [Paper] [Notes]
- Attention-based Dropout Layer for Weakly Supervised Object Localization (2019): [Paper] [Notes]
- Attention is not Explanation (2019): [Paper] [Notes]
- SmoothGrad removing noise by adding noise (2017): [Paper] [Notes]
- Axiomatic Attribution for Deep Networks (2017): [Paper] [Notes]
- Attention Branch Network: Learning of Attention Mechanism for Visual Explanation (2019): [Paper] [Notes]
- Paying More Attention to Attention: Improving the Performance of CNNs via Attention Transfer (2016): [Paper] [Notes]
Natural Language Processing (NLP)
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021): [Paper] [Notes]
- Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data (2020): [Paper] [Notes]
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning (2021): [Paper] [Notes]
- BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning (2020): [Paper] [Notes]
- FreeLB: Enhanced Adversarial Training for Natural Language Understanding (2020): [Paper] [Notes]
- MixText: Linguistically-Informed Interpolation for Semi-Supervised Text Classification (2020): [Paper] [Notes]
Generative Modeling
- Generative Pretraining from Pixels (2020): [Paper] [Notes]
- Consistency Regularization for Generative Adversarial Networks (2020): [Paper] [Notes]
Unsupervised Learning
- Invariant Information Clustering for Unsupervised Image Classification and Segmentation (2019): [Paper] [Notes]
- Deep Clustering for Unsupervised Learning of Visual Feature (2018): [Paper] [Notes]
Semantic Segmentation
- DeepLabv3+: Encoder-Decoder with Atrous Separable Convolution (2018): [Paper] [Notes]
- Large Kernel Matter, Improve Semantic Segmentation by Global Convolutional Network (2017): [Paper] [Notes]
- Understanding Convolution for Semantic Segmentation (2018): [Paper] [Notes]
- Rethinking Atrous Convolution for Semantic Image Segmentation (2017): [Paper] [Notes]
- RefineNet: Multi-path refinement networks for high-resolution semantic segmentation (2017): [Paper] [Notes]
- Pyramid Scene Parsing Network (2017): [Paper] [Notes]
- SegNet: A Deep ConvolutionalEncoder-Decoder Architecture for ImageSegmentation (2016): [Paper] [Notes]
- ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation (2016): [Paper] [Notes]
- Attention to Scale: Scale-aware Semantic Image Segmentation (2016): [Paper] [Notes]
- Deeplab: semantic image segmentation with DCNN, atrous convs and CRFs (2016): [Paper] [Notes]
- U-Net: Convolutional Networks for Biomedical Image Segmentation (2015): [Paper] [Notes]
- Fully Convolutional Networks for Semantic Segmentation (2015): [Paper] [Notes]
- Hypercolumns for object segmentation and fine-grained localization (2015): [Paper] [Notes]
Weakly- and Semi-supervised Semantic segmentation
- Box-driven Class-wise Region Masking and Filling Rate Guided Loss (2019): [Paper] [Notes]
- FickleNet: Weakly and Semi-supervised Semantic Segmentation using Stochastic Inference (2019): [Paper] [Notes]
- Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (2018): [Paper] [Notes]
- Learning Pixel-level Semantic Affinity with Image-level Supervision (2018): [Paper] [Notes]
- Object Region Mining with Adversarial Erasing (2018): [Paper] [Notes]
- Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Segmentation (2018): [Paper] [Notes]
- Tell Me Where to Look: Guided Attention Inference Network (2018): [Paper] [Notes]
- Semi Supervised Semantic Segmentation Using Generative Adversarial Network (2017): [Paper] [Notes]
- Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation (2015): [Paper] [Notes]
- Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation (2015): [Paper] [Notes]
Information Retrieval
Graph Neural Network
- Pixels to Graphs by Associative Embedding (2017): [Paper] [Notes]
- Associative Embedding: End-to-End Learning forJoint Detection and Grouping (2017): [Paper] [Notes]
- Interaction Networks for Learning about Objects , Relations and Physics (2016): [Paper] [Notes]
- DeepWalk: Online Learning of Social Representation (2014): [Paper] [Notes]
- The graph neural network model (2009): [Paper] [Notes]
Regularization
Deep learning Methods & Models
Document analysis and segmentation
- dhSegment: A generic deep-learning approach for document segmentation (2018): [Paper] [Notes]
- Learning to extract semantic structure from documents using multimodal fully convolutional neural networks (2017): [Paper] [Notes]
- Page Segmentation for Historical Handwritten Document Images Using Conditional Random Fields (2016): [Paper] [Notes]
- ICDAR 2015 competition on text line detection in historical documents (2015): [Paper] [Notes]
- Handwritten text line segmentation using Fully Convolutional Network (2017): [Paper] [Notes]
- Deep Neural Networks for Large Vocabulary Handwritten Text Recognition (2015): [Paper] [Notes]
- Page Segmentation of Historical Document Images with Convolutional Autoencoders (2015): [Paper] [Notes]