• Stars
    star
    300
  • Rank 138,870 (Top 3 %)
  • Language
    Python
  • License
    Other
  • Created over 4 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Code for the Proceedings of the National Academy of Sciences 2020 article, "Understanding the Role of Individual Units in a Deep Neural Network"

What is the Role of a Neuron?

When a deep network is trained on a high-level task such as classifying a place or synthesizing a scene, individual neural units within the network will often emerge that match specific human-interpretable concepts, like "trees", "windows", or "human faces."

What role do such individual units serve within a deep network?

We examine this question in two types of networks that contain interpretable units: networks trained to classify images of scenes (supervised image classifiers), and networks trained to synthesize images of scenes (generative adversarial networks).

Understanding the Role of Individual Units in a Deep Network.
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba.
Proceedings of the National Academy of Sciences, September 2020.
MIT, MIT-IBM Watson AI Lab, IBM Research, The Chinese University of Hong Kong, Adobe Research


PNAS Paper

Supplemental

Website

arXiv

Dissecting Units in Classifiers and Generators

Network dissection compares individual network units to the predictions of a semantic segmentation network that can label pixels with a broad set of object, part, material, and color classes. This technique gives us a standard and scalable way to identify any units within the networks that match those same semantic classes.

It works both in classification settings where the image is the input, and in generative settings where the image is the output.

Dissection

We find that both state-of-the-art GANs and classifiers contain object-matching units that correspond to a variety of object and part concepts, with semantics emerging in different layers.

Comparing a Classifier to a Generator

To investigate the role of such units within classifiers, we measure the impact on the accuracy of the network when we turn off units individually or in groups. We find that removing as few as 20 units can destroy the network's ability to detect a class, but retaining only those 20 units and removing 492 other units in the same layer can keep the network's accuracy on that same class mostly intact. Furthermore, we find that those units that are important for the largest number of output classes are also the emergent units that match human-interpretable concepts best.

Classifier Intervention Experiments

In a generative network, we can understand the causal effects of neurons by observing changes to output images when sets of units are turned on and off. We find causal effects are strong enough to enable users to paint images out of object classes by activating neurons; we also find that some units reveal interactions between objects and specific contexts within a model.

Genereator Intervention Experiments

Citation

Bau, David, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences (2020).

Bibtex

@article{bau2020role,
  author = {Bau, David and Zhu, Jun-Yan and Strobelt, Hendrik and Lapedriza, Agata and Zhou, Bolei and Torralba, Antonio},
  title = {Understanding the role of individual units in a deep neural network},
  elocation-id = {201907375},
  year = {2020},
  doi = {10.1073/pnas.1907375117},
  publisher = {National Academy of Sciences},
  issn = {0027-8424},
  URL = {https://www.pnas.org/content/early/2020/08/31/1907375117},
  journal = {Proceedings of the National Academy of Sciences}
}

More Repositories

1

seedrandom

seeded random number generator for Javascript
JavaScript
1,955
star
2

rewriting

Rewriting a Deep Generative Model, ECCV 2020 (oral). Interactive tool to directly edit the rules of a GAN to synthesize scenes with objects added, removed, or altered. Change StyleGANv2 to make extravagant eyebrows, or horses wearing hats.
Python
537
star
3

how-to-read-pytorch

Quick, visual, principled introduction to pytorch code through five colab notebooks.
Jupyter Notebook
316
star
4

ganseeing

Seeing what a GAN cannot generate. Visualizes and quantifies object classes within scenes that are outside the range of a GAN.
Python
182
star
5

baukit

Python
37
star
6

covid-19-chart

Chart of current COVID-19 time series data. Enables a variety of county- state- and nation-level comparisons and data exploration.
HTML
18
star
7

see

see.js interactive eval panel that allows debugging of nested javascript scope.
JavaScript
16
star
8

sidn-handbook

The Structure and Interpretation of Deep Networks Handbook
Jupyter Notebook
12
star
9

xsrand

Several fast 32-bit xor-shift random number generators implemented in Javascript.
JavaScript
12
star
10

conformal

An interactive HTML conformal map viewer.
HTML
7
star
11

gpwidget

Jupyter Notebook
7
star
12

envs

Conda environments for pytorch-oriented deep learning work.
Shell
5
star
13

quick-netdissect

Lightweight reimplementation of netdissect for pytorch (under development)
Python
4
star
14

webcrush

A utility for minifying and LZW compressing an HTML file.
JavaScript
4
star
15

colabsecrets

A repo for keeping track of colab tricks.
Python
4
star
16

splaylist

Splay tree in Javascript, with fast indexing and order statistics. Better than a linked list.
JavaScript
4
star
17

make-full-linux-history

Yoann Padioleau's scripts to graft together a nearly complete history of linux commits as a git repo.
OCaml
3
star
18

miniplaces

baseline setup for alexnet on miniplaces using pytorch
Python
2
star
19

mandelbrot

Updated code for http://davidbau.com/mandelbrot/, an early HTML5 demo of the canvas element. See http://davidbau.com/archives/2009/09/27/mandelbrot.html
HTML
2
star
20

ganclass

Python
1
star
21

heidisudoku

JavaScript
1
star
22

panoptic-segmentation

Python
1
star
23

thisxsite

"This X does not exist" website
Python
1
star
24

pytorch-setup

Just to share a conda setup that I use
Shell
1
star
25

glowjs

A package containing the glowscript javascript libaries as a single javascript file.
JavaScript
1
star
26

sudokui

A simplified sudoku UI for experimenting with A/B testing: MIT 6.831
JavaScript
1
star
27

net-intent

Simple experiment to explore a jacobian method for determining hidden unit intent.
Python
1
star