Hypernetworks for PyTorch

This package contains utilities that make it easy to work with hypernetworks in PyTorch.

Installation

You can either install the latest package version via

python3 -m pip install hypnettorch

or, you directly install the current sources

python3 -m pip install git+https://github.com/chrhenning/hypnettorch

Installation for developers

If you actively develop the package, it is easiest to install it in development mode, such that all changes that are done to source files are directly visible when you use the package.

Clone the repository to a location of your choice

git clone https://github.com/chrhenning/hypnettorch.git

and move inside the cloned repo

cd ./hypnettorch

Now, you can simply install the package in editable mode, which will ensure that you can easily update the package sources (cf. development mode)

pip3 install --editable . --user

Since the package was installed in editable mode, you can always update the sources simply by pulling the most recent code

git pull

You can uninstall the package at any point by running python3 setup.py develop -u.

Usage

The basic functionalities of the package are quite intuitive and easy to use, e.g.,

import torch
from hypnettorch.mnets import MLP
from hypnettorch.hnets import HMLP
mnet = MLP(n_in=8, n_out=1, no_weights=True) # Create MLP with 8 inputs and 1 output.
hnet = HMLP(mnet.param_shapes) # Create hypernetwork for the above MLP.
weights = hnet.forward(cond_id=0) # Generate the weights of network `mnet`.
inputs = torch.rand(32, 8) # Random inputs.
mnet.forward(inputs, weights=weights) # Compute predictions using the generated weights.

There are several tutorials. Check out the getting started tutorial when working with hypnettorch for the first time.

You may also checkout the examples. For instance, how to use hypernetworks for continual learning.

Documentation

The documentation can be found here.

Note for developers

The documentation can be build using

python3 setup.py build_sphinx

and opened via the file index.html.

Citation

When using this package in your research project, please consider citing one of our papers for which this package has been developed.

@inproceedings{posterior:replay:2021:henning:cervera,
title={Posterior Meta-Replay for Continual Learning}, 
      author={Christian Henning and Maria R. Cervera and Francesco D'Angelo and Johannes von Oswald and Regina Traber and Benjamin Ehret and Seijin Kobayashi and Benjamin F. Grewe and João Sacramento},
booktitle={Conference on Neural Information Processing Systems},
year={2021},
url={https://arxiv.org/abs/2103.01133}
}

@inproceedings{ehret2020recurrenthypercl,
  title={Continual Learning in Recurrent Neural Networks},
  author={Benjamin Ehret and Christian Henning and Maria R. Cervera and Alexander Meulemans and Johannes von Oswald and Benjamin F. Grewe},
  booktitle={International Conference on Learning Representations},
  year={2021},
  url={https://arxiv.org/abs/2006.12109}
}

@inproceedings{oshg2019hypercl,
title={Continual learning with hypernetworks},
author={Johannes von Oswald and Christian Henning and Benjamin F. Grewe and Jo{\~a}o Sacramento},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://arxiv.org/abs/1906.00695}
}