Angular Penalty Softmax Losses Pytorch
Concise Pytorch implementation of the Angular Penalty Softmax Losses presented in:
- ArcFace: https://arxiv.org/abs/1801.07698 [1]
- SphereFace: https://arxiv.org/abs/1704.08063 [2]
- CosFace/Additive Margin: https://arxiv.org/abs/1801.09414 [3] / https://arxiv.org/abs/1801.05599 [4]
(Note: the SphereFace implementation is not exactly as described in their paper but instead uses the 'trick' presented in the ArcFace paper to use arccosine instead of the double angle formula)
from loss_functions import AngularPenaltySMLoss
in_features = 512
out_features = 10 # Number of classes
criterion = AngularPenaltySMLoss(in_features, out_features, loss_type='arcface') # loss_type in ['arcface', 'sphereface', 'cosface']
# Forward method works similarly to nn.CrossEntropyLoss
# x of shape (batch_size, in_features), labels of shape (batch_size,)
# labels should indicate class of each sample, and should be an int, l satisying 0 <= l < out_dim
loss = criterion(x, labels)
loss.backward()Experiments/Demo
There are a simple set of experiments on Fashion-MNIST [2] included in train_fMNIST.py which compares the use of ordinary Softmax and Additive Margin Softmax loss functions by projecting embedding features onto a 3D sphere.
The experiments can be run like so:
python train_fMNIST.py --num-epochs 40 --seed 1234 --use-cuda
Which produces the following results:
Baseline (softmax)
Additive Margin Softmax/CosFace
ArcFace
TODO: fix sphereface results
[1] Deng, J. et al. (2018) ‘ArcFace: Additive Angular Margin Loss for Deep Face Recognition’. Available at: http://arxiv.org/abs/1801.07698.
[2] Liu, W. et al. (2017) ‘SphereFace: Deep hypersphere embedding for face recognition’, in Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 6738–6746. doi: 10.1109/CVPR.2017.713.
[3] Wang, H. et al. (2018) ‘CosFace: Large Margin Cosine Loss for Deep Face Recognition’. Available at: http://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_CosFace_Large_Margin_CVPR_2018_paper.pdf (Accessed: 12 August 2019).
[4] “Additive Margin Softmax for Face Verification.” Wang, Feng, Jian Cheng, Weiyang Liu and Haijun Liu. IEEE Signal Processing Letters 25 (2018): 926-930.
[5] "Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms." Han Xiao, Kashif Rasul, Roland Vollgraf. arXiv:1708.07747