Differentiable Optimization-Based Modeling for Machine Learning

• This repository is by Brandon Amos and contains the full source code and data to produce my thesis document.
• The slides are available in pdf and pptx format.

Unpublished work in this thesis

• Chapter 2 provides some preliminaries and background information on differentiable convex optimization layers, including derivations for the optimization (or variational) viewpoints of the ReLU, sigmoid, and softmax.
• Chapter 7 presents an early version of differentiable CVXPY layers, which is now available here. As a bibliographic note, the cone program differentiation derivation in section 7.3 here remains unpublished in this thesis and was done concurrent to and independent of Differentiating Through a Cone Program.

Publications behind this thesis

Some of the content here is behind these publications:

The experimental source code and libraries produced for this thesis are freely available as open source software and are available in the following repositories.

• [cvxgrp/cvxpylayers] Differentiable convex optimization layers in CVXPY.
• [locuslab/mpc.pytorch] A stand-alone PyTorch library for the differentiable model predictive control approach.
• [locuslab/differentiable-mpc] PyTorch experiments for the differentiable MPC work.
• [locuslab/qpth]: A stand-alone PyTorch library for the OptNet QP layers.
• [locuslab/optnet] PyTorch experiments for OptNet.
• [locuslab/icnn] TensorFlow experiments for input-convex neural networks.
• [cmusatyalab/openface] Face recognition with deep neural networks.
• [bamos/block] An intelligent block matrix library for numpy, PyTorch, and beyond.
• [bamos/dcgan-completion.tensorflow] Image Completion with Deep Learning in TensorFlow.
• [bamos/densenet.pytorch] A PyTorch implementation of DenseNet.

• This repository started from Cyrus Omar's thesis code, which is based on a CMU thesis template by David Koes and others before.
• Of standalone interest, refs.sort.sh uses biber to alphabetize and standardize my bibliography in refs.bib so it doesn't get too messy. This uses the configuration in refs.conf.
• I use update-pdf.sh to keep the latest PDF only in HEAD, although Git LFS or a related project may be a better solution.

The BibTeX for this document is:

@phdthesis{amos2019differentiable,
  author       = {Brandon Amos},
  title        = {{Differentiable Optimization-Based Modeling for Machine Learning}},
  school       = {Carnegie Mellon University},
  year         = 2019,
  month        = May,
}