batchnorm-transferlearning
Partial transfusion: on the expressive influence of trainable batch norm parameters for transfer learning. TL;DR: Fine-tuning only the batch norm affine parameters leads to similar performance as to fine-tuning all of the model parameters