Great-Deep-Learning-Tutorials
A Great Collection of Deep Learning Tutorials and Repositories
General Deep Learning Tutorials:
- Browse state-of-the-art Deep Learning based Papers with their associated codes [Extremely Fantastic]
- Deep-Learning-Roadmap
- DeepLizard [Good Tutorials for Deep Learning]
- Sebastian Ruder - Blog [Great NLP & Deep Learning Posts]
- Jeremy Jordan - Blog
- Excellent Blog
- Torchvision Release Notes [Important]
- The 6 most useful Machine Learning projects of the past year (2018)
- ResNet Review
- Receptive Field Estimation [Great]
- An overview of gradient descent optimization algorithms [Useful]
- How to decide on learning rate
- Overview of State-of-the-art Machine Learning Algorithms per Discipline per Task
- Practical Machine Learning
- Awesome Machine Learning and AI Courses
- UVA Deep Learning II Course
- PyTorch Book
- Fast.ai Course: Practical Deep Learning for Coders [Great]
- Neuromatch Deep Learning Course [Great]
- labmlai: 59 Implementations/tutorials of deep learning papers with side-by-side notes [Great]
- labml.ai
- FightingCV-Paper-Reading: understand the most advanced research work in an easier way
- Learn PyTorch for Deep Learning: Zero to Mastery Course [Excellent]
- ML Papers Explained [Excellent]
- Alpha Signal: Latest Research in Machine Learning
- Harvard CS197: AI Research Experiences - The Course Book [Excellent]
- A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
Deep Learning Useful Resources for Computer Vision:
Deep Learning Useful Resources for Natural Language Processing (NLP):
- Great Deep Learning Resources for NLP Tasks [Excellent]
Deep Learning Useful Resources for Spoken Language Processing (Speech Processing):
Deep Learning & Machine Learning Useful Resources for General Data Science Tasks:
General Notes about Generative AI:
Quantization & Distillation of Deep Learning Models:
- Quantization
- Neural Network Distiller
- Introduction to Quantization on PyTorch [Excellent]
- Dynamic Quantization in PyTorch
- Static Quantization in PyTorch
- Intel(R) Math Kernel Library - Intel MKL-DNN
- Intel MKL-Dnn
- ONNX Float32 to Float16
- Neural Network Quantization Introduction [Tutorial]
- Quantization in Deep Learning [Tutorial]
- Speeding up Deep Learning with Quantization [Tutorial]
- Knowledge Distillation in Deep Learning
- Model Distillation Techniques for Deep Learning
- MMRazor: model compression toolkit [Great]
- FP8 Quantization: The Power of the Exponent
AutoML:
Diffusion Models:
- Diffusion Models via lilianweng
- Diffusion Models Papers Survey Taxonomy
- Phenaki: a text-to-video model
Multimodal Deep Learning:
Deep Reasoning:
Deep Reinforcement Learning (Great Courses & Tutorials):
- A Free course in Deep Reinforcement Learning from beginner to expert [Great]
- Deep Reinforcement Learning Algorithms with PyTorch
- Deep Reinforcement Learning - CS 285 Berkeley Course
- solutions to UC Berkeley CS 285
- Reinforcement Learning: An Introduction - main book in this field
- CS234: Reinforcement Learning Course
- Introduction to Reinforcement Learning Course - by DeepMind
Graph Neural Networks:
- An Introduction to Graph Neural Networks
- How to Train Graph Convolutional Network Models in a Graph Database
- A comprehensive survey on graph neural networks
- Graph Neural Networks: A Review of Methods and Applications
Graph Neural Networks Frameworks:
- Spektral
- Deep Graph Library - DGL
- PyTorch Geometric - PyG
- ptgnn: A PyTorch GNN Library
- Graph Data Augmentation Papers
- Neo4j: Graph Data Platform
Best Practices for Training Deep Models:
General Notes for Training Deep Models:
PyTorch Lightening Notes & Accumulate Gradients:
Loss Functions:
Imbalanced Dataset Handling:
- deal with an imbalanced dataset using weightedrandomsampler
- imbalanced-dataset-sampler [Great]
- demystifying pytorchs weightedrandomsampler
- weighted random sampler oversample or undersample
Weight Initialization:
Batch Normalization:
- Batch Normalization in Neural Networks
- Batch Normalization and Dropout in Neural Networks
- Difference between Local Response Normalization and Batch Normalization
Learning Rate Scheduling & Initialization:
- Automated Learning Rate Suggester
- Learning Rate Finder - fastai
- Cyclical Learning Rates for Training Neural Networks
- ignite - Example of FastaiLRFinder
- Find Learning Rate - a gist code
- Learning rate finder - PyTorch Lightning
- RAdam - On the Variance of the Adaptive Learning Rate and Beyond
Early Stopping:
- Early Stopping in PyTorch - Bjarten
- Catalyst - Early Stopping
- ignite - Early Stopping
- PyTorch High-Level Training Sample
- PyTorch Discussion about Early Stopping
Tuning Guide Recipes:
- PyTorch Tuning Guide Tutorial
- PyTorch memory leak with dynamic size tensor input
- Karpathy: A Recipe for Training Neural Networks
Training Optimizer:
PyTorch running & training on TPU (colab):
Evaluation Metrics:
Validating ML Models:
Conferences News:
- Latest Computer Vision Trends from CVPR 2019
- Interesting 2019 CVPR papers
- Summaries of CVPR papers on ShortScience.org
- Summaries of ICCV papers on ShortScience.org
- Summaries of ECCV papers on ShortScience.org
Deep Learning Frameworks and Infrustructures:
- set-up a Paperspace GPU Server
- Distributed ML with OpenMPI
- Tensorflow 2.0 vs Mxnet
- TensorFlow is dead, long live TensorFlow!
Great Libraries:
- The Unified Machine Learning Framework
- Skorch - A scikit-learn compatible neural network library that wraps PyTorch
- Hummingbird - traditional ML models into tensor computations via PyTorch
- BoTorch - Bayesian Optimization in PyTorch
- torchvision 0.3: segmentation, detection models, new datasets and more
- TorchAudio: an audio library for PyTorch
- AudTorch
- TorchAudio-Contrib
- fastText - Facebook AI Research (FAIR)
- Fairseq - Facebook AI Research (FAIR)
- ParlAI - dialogue models - Facebook AI Research (FAIR)
- DALI - highly optimized engine for data pre-processing
- Netron - GitHub [Visualizer for deep learning Models (Excellent)]
- Netron - Web Site
- JupyterLab GPU Dashboards [Good]
- PyTorch Hub
- Neural Structured Learning (NSL) in TensorFlow
- Pywick - High-Level Training framework for Pytorch
- torchbearer: A model fitting library for PyTorch
- torchlayers - Shape inference for PyTorch (like in Keras)
- torchtext - GitHub
- torchtext - Doc
- Optuna - hyperparameter optimization framework
- PyTorchLightning
- Nvidia - runx - An experiment management tool
- MLogger: a Machine Learning logger
- ClearML - ML/DL development and production suite
- Lime: Explaining the predictions of any ML classifier
- Microsoft UniLM AI [Great]
- mlnotify: No need to keep checking your training
- NVIDIA NeMo - toolkit for creating Conversational AI (ASR, TTS, and NLP)
- Microsoft DeepSpeed
- Mojo: a new programming language for AI developers
Great Models:
- ResNext WSL [Great Pretrained Model]
- Semi-Weakly Supervised (SWSL) ImageNet Models [Great Pretrained Model]
- Deep High-Resolution Representation Learning (HRNet)
Deep Model Conversion:
Great Deep Learning Repositories (for learning DL-based programming):
- deeplearning-models - PyTorch & TensorFlow Learning [Very Excellent Repository]
- PyTorch Image Models [Great]
- 5 Advanced PyTorch Tools to Level up Your Workflow [Interesting]
PyTorch High-Level Libraries:
- Catalyst - PyTorch framework for Deep Learning research and development [Great]
- PyTorch Lightning - GitHub [Great]
- PyTorch Lightning - Web Page
- Ignite - GitHub [Great]
- Ignite - Web Page
- TorchMetrics
- Ludwig AI: Data-centric declarative deep learning framework [Great]
- PyTorch Kineto: CPU+GPU Profiling library
- PyTorch Profiler
- PyTorch Benchmarks
Annotation Tools:
Other:
- Clova AI Research - NAVER & LINE
- Exploring Weight Agnostic Neural Networks
- Weight Agnostic Neural Networks
- Weight Agnostic Neural Networks - GitHub
- SAM: Sharpness-Aware Minimization for Efficiently Improving Generalization
- Qualcomm Discusses Secret Dataset Generation Data
- State of AI Report 2021
- Project Blink: AI-powered video editing on the web
- PyTorch Incremental Learning
- Google Research, 2022 & Beyond: Language, Vision and Generative Models
- Elicit: Ask a research question [Interesting]
- Google People + AI Research (PAIR) [Interesting business based AI topics]