D-X-Y/Awesome-AutoDL D-X-Y/Awesome-AutoDL

  • Stars
    star
    2,229
  • Rank 20,681 (Top 0.5 %)
  • Language
    Python
  • License
    MIT License
  • Created about 6 years ago
  • Updated about 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
D-X-Y/Awesome-AutoDL
github

D-X-Y

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Automated Deep Learning: Neural Architecture Search Is Not the End (a curated list of AutoDL resources and an in-depth analysis)

Awesome AutoDL Awesome

A curated list of automated deep learning related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

Awesome Blogs

Awesome AutoDL Libraies

Awesome Benchmarks

Title Venue Code
NAS-Bench-101: Towards Reproducible Neural Architecture Search ICML 2019 GitHub
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search ICLR 2020 Github
NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search arXiv 2020 GitHub
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search ICLR 2020 GitHub
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size TPAMI 2021 GitHub
NAS-Bench-ASR: Reproducible Neural Architecture Search for Speech Recognition ICLR 2021 GitHub
HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark ICLR 2021 GitHub
NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing arXiv 2020 GitHub
NAS-Bench-x11 and the Power of Learning Curves NeurIPS 2021 GitHub

Deep Learning-based NAS and HPO

Type G RL EA PD Other
Explanation gradient-based reinforcement learning evolutionary algorithm performance prediction other types

2021 Venues

Title Venue Type Code
CATE: Computation-aware Neural Architecture Encoding with Transformers ICML O GitHub
Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator CVPR G Github
Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition ICCV EA Github
AutoFormer: Searching Transformers for Visual Recognition ICCV EA GitHub
LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search CVPR EA GitHub
One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking CVPR EA GitHub
DARTS-: Robustly Stepping out of Performance Collapse Without Indicators ICLR G GitHub
Zero-Cost Proxies for Lightweight NAS ICLR O GitHub
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective ICLR - GitHub
DrNAS: Dirichlet Neural Architecture Search ICLR G GitHub
Rethinking Architecture Selection in Differentiable NAS ICLR O GitHub
Evolving Reinforcement Learning Algorithms ICLR EA GitHub
AutoHAS: Differentiable Hyper-parameter and Architecture Search ICLR-W G -
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function CVPR PD github

2020 Venues

Title Venue Type Code
Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search NeurIPS - GitHub
PyGlove: Symbolic Programming for Automated Machine Learning NeurIPS library -
Does Unsupervised Architecture Representation Learning Help Neural Architecture Search NeurIPS PD GitHub
RandAugment: Practical Automated Data Augmentation with a Reduced Search Space NeurIPS GitHub
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians NeurIPS G GitHub
A Study on Encodings for Neural Architecture Search NeurIPS GitHub
AutoBSS: An Efficient Algorithm for Block Stacking Style Search NeurIPS
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS NeurIPS G GitHub
Interstellar: Searching Recurrent Architecture for Knowledge Graph Embedding NeurIPS
Revisiting Parameter Sharing for Automatic Neural Channel Number Search NeurIPS
Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search NeurIPS MCTS GitHub
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search AAAI MCTS GitHub
Representation Sharing for Fast Object Detector Search and Beyond ECCV G GitHub
Are Labels Necessary for Neural Architecture Search? ECCV G -
Single Path One-Shot Neural Architecture Search with Uniform Sampling ECCV EA -
Neural Predictor for Neural Architecture Search ECCV O -
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models ECCV G -
BATS: Binary ArchitecTure Search ECCV - -
AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification ECCV - -
Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization ECCV - -
Angle-based Search Space Shrinking for Neural Architecture Search ECCV - -
Anti-Bandit Neural Architecture Search for Model Defense ECCV - -
TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search ECCV G GitHub
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search ECCV G GitHub
Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search ECCV RL -
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search ECCV G -
Optimizing Millions of Hyperparameters by Implicit Differentiation AISTATS G -
Evolving Machine Learning Algorithms From Scratch ICML EA -
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization ICML G GitHub
NADS: Neural Architecture Distribution Search for Uncertainty Awareness ICML - -
Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data ICML - -
Neural Architecture Search in a Proxy Validation Loss Landscape ICML - -
Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection CVPR - GitHub
Designing Network Design Spaces CVPR - GitHub
UNAS: Differentiable Architecture Search Meets Reinforcement Learning CVPR G/RL GitHub
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation CVPR G GitHub
A Semi-Supervised Assessor of Neural Architectures CVPR PD -
Binarizing MobileNet via Evolution-based Searching CVPR EA -
Rethinking Performance Estimation in Neural Architecture Search CVPR - GitHub
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy CVPR G GitHub
SGAS: Sequential Greedy Architecture Search CVPR G Github
Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS CVPR RL -
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions CVPR G Github
AdversarialNAS: Adversarial Neural Architecture Search for GANs CVPR G Github
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks CVPR G Github
Block-wisely Supervised Neural Architecture Search with Knowledge Distillation CVPR G Github
Overcoming Multi-Model Forgetting in One-Shot NAS with Diversity Maximization CVPR G Github
Densely Connected Search Space for More Flexible Neural Architecture Search CVPR G Github
EfficientDet: Scalable and Efficient Object Detection CVPR RL -
NAS-BENCH-201: Extending the Scope of Reproducible Neural Architecture Search ICLR - Github
Understanding Architectures Learnt by Cell-based Neural Architecture Search ICLR G GitHub
Evaluating The Search Phase of Neural Architecture Search ICLR -
AtomNAS: Fine-Grained End-to-End Neural Architecture Search ICLR GitHub
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search ICLR - GitHub
Once for All: Train One Network and Specialize it for Efficient Deployment ICLR G GitHub
Efficient Transformer for Mobile Applications ICLR - -
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search ICLR G GitHub
Adversarial AutoAugment ICLR - -
NAS evaluation is frustratingly hard ICLR - GitHub
FasterSeg: Searching for Faster Real-time Semantic Segmentation ICLR G GitHub
Computation Reallocation for Object Detection ICLR - -
Towards Fast Adaptation of Neural Architectures with Meta Learning ICLR - GitHub
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video Architectures ICLR EA -
How to Own the NAS in Your Spare Time ICLR - -
Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search ICPR G github

2019 Venues

Title Venue Type Code
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions ICLR - -
DATA: Differentiable ArchiTecture Approximation NeurIPS - -
Random Search and Reproducibility for Neural Architecture Search UAI G GitHub
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition EMNLP G -
Continual and Multi-Task Architecture Search ACL RL -
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation ICCV G GitHub
Multinomial Distribution Learning for Effective Neural Architecture Search ICCV - GitHub
Searching for MobileNetV3 ICCV EA -
Multinomial Distribution Learning for Effective Neural Architecture Search ICCV - GitHub
Fast and Practical Neural Architecture Search ICCV
Teacher Guided Architecture Search ICCV -
AutoDispNet: Improving Disparity Estimation With AutoML ICCV G -
Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help? ICCV EA -
One-Shot Neural Architecture Search via Self-Evaluated Template Network ICCV G Github
Evolving Space-Time Neural Architectures for Videos ICCV EA GitHub
AutoGAN: Neural Architecture Search for Generative Adversarial Networks ICCV RL github
Discovering Neural Wirings NeurIPS G Github
Towards modular and programmable architecture search NeurIPS Other Github
Network Pruning via Transformable Architecture Search NeurIPS G Github
Deep Active Learning with a NeuralArchitecture Search NeurIPS - -
DetNAS: Backbone Search for Object Detection NeurIPS EA GitHub
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers NeurIPS - -
Efficient Forward Architecture Search NeurIPS G Github
Efficient Neural ArchitectureTransformation Search in Channel-Level for Object Detection NeurIPS G -
XNAS: Neural Architecture Search with Expert Advice NeurIPS G GitHub
DARTS: Differentiable Architecture Search ICLR G github
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware ICLR RL/G github
Graph HyperNetworks for Neural Architecture Search ICLR G -
Learnable Embedding Space for Efficient Neural Architecture Compression ICLR Other github
Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution ICLR EA -
SNAS: stochastic neural architecture search ICLR G -
NetTailor: Tuning the Architecture, Not Just the Weights CVPR G Github
Searching for A Robust Neural Architecture in Four GPU Hours CVPR G Github
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation CVPR - -
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search CVPR EA github
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search CVPR G -
RENAS: Reinforced Evolutionary Neural Architecture Search CVPR G -
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation CVPR G GitHub
MnasNet: Platform-Aware Neural Architecture Search for Mobile CVPR RL Github
MFAS: Multimodal Fusion Architecture Search CVPR EA -
A Neurobiological Evaluation Metric for Neural Network Model Search CVPR Other -
Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells CVPR RL -
Customizable Architecture Search for Semantic Segmentation CVPR - -
Regularized Evolution for Image Classifier Architecture Search AAAI EA -
AutoAugment: Learning Augmentation Policies from Data CVPR RL -
Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules ICML EA -
The Evolved Transformer ICML EA Github
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks ICML RL -
NAS-Bench-101: Towards Reproducible Neural Architecture Search ICML Other Github
On Network Design Spaces for Visual Recognition ICCV G Github

2018 Venues

Title Venue Type Code
Towards Automatically-Tuned Deep Neural Networks BOOK - GitHub
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications ECCV - github
Efficient Architecture Search by Network Transformation AAAI RL github
Learning Transferable Architectures for Scalable Image Recognition CVPR RL github
N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning ICLR RL -
A Flexible Approach to Automated RNN Architecture Generation ICLR RL/PD -
Practical Block-wise Neural Network Architecture Generation CVPR RL -
Path-Level Network Transformation for Efficient Architecture Search ICML RL github
Hierarchical Representations for Efficient Architecture Search ICLR EA -
Understanding and Simplifying One-Shot Architecture Search ICML G -
SMASH: One-Shot Model Architecture Search through HyperNetworks ICLR G github
Neural Architecture Optimization NeurIPS G github
Searching for efficient multi-scale architectures for dense image prediction NeurIPS Other -
Progressive Neural Architecture Search ECCV PD github
Neural Architecture Search with Bayesian Optimisation and Optimal Transport NeurIPS Other github
Differentiable Neural Network Architecture Search ICLR-W G -
Accelerating Neural Architecture Search using Performance Prediction ICLR-W PD -

2017 Venues

Title Venue Type Code
Neural Architecture Search with Reinforcement Learning ICLR RL -
Designing Neural Network Architectures using Reinforcement Learning ICLR RL -
Neural Optimizer Search with Reinforcement Learning ICML RL -
Learning Curve Prediction with Bayesian Neural Networks ICLR PD -
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization ICLR PD -
Hyperparameter Optimization: A Spectral Approach NeurIPS-W Other github
Learning to Compose Domain-Specific Transformations for Data Augmentation NeurIPS - -

Previous Venues

2012-2016

Title Venue Type Code
Speeding up Automatic Hyperparameter Optimization of Deep Neural Networksby Extrapolation of Learning Curves IJCAI PD github

arXiv

Title Date Type Code
NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search 2018.10 EA -
Training Frankenstein’s Creature to Stack: HyperTree Architecture Search 2018.10 G -
Population Based Training of Neural Networks 2017.11 EA GitHub
EmotionNAS: Two-stream Architecture Search for Speech Emotion Recognition 2022.3 G -
U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search 2022.3 G Github

Awesome Surveys

Title Venue Year Code
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions ACM Computing Surveys 2021 -
Automated Machine Learning on Graphs: A Survey ICLR-W 2021 GitHub
On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice Neurocomputing 2020 github
AutonoML: Towards an Integrated Framework for Autonomous Machine Learning arXiv 2020 -
Automated Machine Learning Springer Book 2019 -
Neural architecture search: A survey JMLR 2019 -
AutoML: A Survey of the State-of-the-Art arXiv 2019 GitHub
A Survey on Neural Architecture Search arXiv 2019 -
Taking human out of learning applications: A survey on automated machine learning arXiv 2018 -
IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective Engineering Applications of Artificial Intelligence 2022 github

More Repositories

1

AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
Python
1,562
star
2

landmark-detection

Four landmark detection algorithms, implemented in PyTorch.
Python
920
star
3

NAS-Bench-201

NAS-Bench-201 API and Instruction
Python
624
star
4

NATS-Bench

TPAMI 2021: NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
Python
170
star
5

DXY-Projects

Some projects from Xuanyi Dong.
C++
90
star
6

D-X-Y

3
star