Awesome Temporal Action Localization:

A curated list of temporal action localization/detection and related area (e.g. temporal action proposal) resources.

Contents

• Temporal Action Localization
  • Paper
    • 2021 - 2020 - 2019 - 2018 - 2017 - 2016
  • Dataset
  • Benchmark Results
    • THUMOS14 - ActivityNet v1.3
• Weakly Supervised Temporal Action Localization
  • Paper
    • 2021 - 2020 - 2019 - 2018 - 2017
  • Dataset
  • Benchmark Results
    • THUMOS14 - ActivityNet v1.3 - ActivityNet v1.2

Contributors: SCUT: Runhao Zeng, Zeng You, Xinyu Sun NPU: Le Yang

Temporal Action Localization

Papers

2022

• [TALLFormer] TALLFormer: Temporal Action Localization with Long-memory Transformer - Feng Cheng et al, ECCV 2022.[code]
• [ActionFormer] ActionFormer: Localizing Moments of Actions with Transformers - Chenlin Zhang et al, ECCV 2022. [code]
• [RCL] RCL: Recurrent Continuous Localization for Temporal Action Detection - Qiang Wang et al, CVPR 2022.
• [DCAN] DCAN: Improving Temporal Action Detection via Dual Context Aggregation - Guo Chen et al, AAAI 2022.
• [TadTR] End-to-end Temporal Action Detection with Transformer - Xiaolong Liu et al, TIP 2022. [code]

2021

• [RTD-Net] Relaxed Transformer Decoders for Direct Action Proposal Generation - Jing Tan et al, ICCV 2021. [code]
• [LoFi] Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization - Mengmeng Xu et al, NIPS 2021.
• [ATAG] Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation - Shuning Chang et al, arXiv 2021.
• [AEI] AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation - Khoa Vo et al, BMVC 2021.
• [GCM] Graph Convolutional Module for Temporal Action Localization in Videos - Runhao Zeng et al, TPAMI 2021. [code]
• [AVFusion] Hear Me Out: Fusional Approaches for AudioAugmented Temporal Action Localization - Bagchi et al, arXiv 2021. [code]
• [ContextLoc] Enriching Local and Global Contexts for Temporal Action Localization - Zixin Zhu et al, ICCV 2021.
• [CSA] Class Semantics-based Attention for Action Detection - Deepak Sridhar et al, ICCV 2021.
• [TCANet] Temporal Context Aggregation Network for Temporal Action Proposal Refinement - Zhiwu Qing et al, CVPR 2021.
• [Multi-Task TAD] Three Birds with One Stone: Multi-Task Temporal Action Detection via Recycling Temporal Annotations - Zhihui Li et al, CVPR 2021.
• [Coarse-Fine Networks] Coarse-Fine Networks for Temporal Activity Detection in Videos - Kahatapitiya et al, CVPR 2021.
• [AFSD] Learning Salient Boundary Feature for Anchor-free Temporal Action Localization - Chuming Lin et al, CVPR 2021. [code]
• [MUSEs] Multi-shot temporal event localization: A Benchmark - Xiaolong Liu et al, CVPR 2021
• [SALAD] SALAD: Self-Assessment Learning for Action Detection - Guillaume Vaudaux-Ruth et al, WACV 2021
• [RTD-Net] Relaxed Transformer Decoders for Direct Action Proposal Generation - Jing Tan et al, arxiv 2021. [code]
• [AGT] Activity Graph Transformer for Temporal Action Localization - Megha Nawhal et al, arxiv 2021

2020

• [VSGN] Video Self-Stitching Graph Network for Temporal Action Localization - Chen Zhao et al, ICCV 2021
• [UFA] Temporal Action Detection with Multi-level Supervision - Baifeng Shi et al, arxiv 2020
• [TSP] TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks - Humam Alwassel et al, arxiv 2020
• [BSP] Boundary-sensitive Pre-training for Temporal Localization in Videos - Mengmeng Xu et al, arxiv 2020
• [VAN] Temporal Action Localization with Variance-Aware Networks - Ting-Ting Xie et al, arxiv 2020
• [TSI] TSI: Temporal Scale Invariant Network for Action Proposal Generation - Shuming Liu et al, ACCV 2020. [code]
• [BU-TAL] Bottom-Up Temporal Action Localization with Mutual Regularization - Peisen Zhao et al, ECCV 2020.
• [DBG] Fast Learning of Temporal Action Proposal via Dense Boundary Generator - Chuming Lin et al, AAAI 2020. [code]
• [G-TAD] G-TAD: Sub-Graph Localization for Temporal Action Detection - Mengmeng Xu et al, CVPR 2020. [code]
• [PBRNet] Progressive Boundary Refinement Network for Temporal Action Detection - Qinying Liu et al, AAAI 2020.
• [AGCN] Graph Attention based Proposal 3D ConvNets for Action Detection - Jun Li et al, AAAI 2020.

2019

• [PGCN] Graph Convolutional Networks for Temporal Action Localization - Runhao Zeng et al, ICCV 2019. [code]
• [RAM] Relation Attention for Temporal Action Localization - Peihao Chen et al, TMM 2019.
• [BMN] BMN: Boundary-Matching Network for Temporal Action Proposal Generation - Tianwei Lin et al, ICCV 2019.
• [GTAN] Gaussian Temporal Awareness Networks for Action Localization - Fuchen Long et al, CVPR 2019.
• [DBS] Video Imprint Segmentation for Temporal Action Detection in Untrimmed Videos - Zhanning Gao et al, AAAI 2019.
• [C-TCN] Deep Concept-wise Temporal Convolutional Networks for Action Localization - Xin Li et al, arXiv 2019.

2018

• [TAL-Net] Rethinking the Faster R-CNN Architecture for Temporal Action Localization - Yuwei Chao et al, CVPR 2018.
• [BSN] BSN: Boundary Sensitive Network for Temporal Action Proposal Generation - Tianwei Lin et al, ECCV 2018. [code]
• [Action-Search] Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization - Humam Alwassel et al, ECCV 2018. [code]
• [TPC] Exploring Temporal Preservation Networks for Precise Temporal Action Localization - Ke Yang et al, AAAI 2018.
• [Self-Ad] A Self-Adaptive Proposal Model for Temporal Action Detection based on Reinforcement Learning - Jingjia Huang et al, AAAI 2018.

2017

• [SSN] Temporal Action Detection with Structured Segment Networks - Yue Zhao et al, ICCV 2017. [code]
• [R-C3D] R-C3D: Region Convolutional 3D Network for Temporal Activity Detection - Huijuan Xu et al, ICCV 2017. [code]
• [TCN] Temporal Context Network for Activity Localization in Videos - Xiyang Dai et al, ICCV 2017.
• [TURN] TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals - Jiyang Gao et al, ICCV 2017. [code]
• [SST] SST: Single-Stream Temporal Action Proposals - Shyamal Buch et al, ICCV 2017.
• [CDC] CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos - Zheng Shou et al, CVPR 2017. [code]
• [SCC] SCC: Semantic Context Cascade for Efficient Action Detection - Fabian Caba Heilbron et al, CVPR 2017.
• [SMS] Temporal Action Localization by Structured Maximal Sums - Zehuan Yuan et al, CVPR 2017.

2016

• [S-CNN] Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs - Zheng Shou et al, CVPR 2016. [code]
• [PSDF] Temporal Action Localization with Pyramid of Score Distribution Features - Jun Yuan et al, CVPR 2016.
• [FG] End-to-end Learning of Action Detection from Frame Glimpses in Videos - Serena Yeung et al, CVPR 2016.
• [SLM] Temporal Action Detection Using a Statistical Language Model - Alexander Richard et al, CVPR 2016.
• [DAPs] DAPs: Deep Action Proposals for Action Understanding - Victor Escorcia et al, ECCV 2016.

Dataset

• THUMOS14
• ActivityNet v1.3

Benchmark Results

THUMOS14

ActivityNet v1.3

Weakly Supervised Temporal Action Localization

Paper

2021

• [BackTAL] Background-Click Supervision for Temporal Action Localization - Le Yang et al, TPAMI 2021. [code]
• [ACSNet] ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization - Ziyi Liu et al, AAAI 2021.
• [AMS] Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization - Chen Ju et al, arXiv 2021.
• [AUMN] Action Unit Memory Network for Weakly Supervised Temporal Action Localization - Wang Luo et al, CVPR 2021.
• [CSCL] Weakly-Supervised Temporal Action Localization via Cross-Stream Collaborative Learning - Yuan Ji et al, ACM MM 2021.
• [RefineLoc] RefineLoc: Iterative Refinement for Weakly-Supervised Action Localization - Alejandro Pardo et al, WACV 2021. [code]
• [UM-Net] Weakly-supervised Temporal Action Localization by Uncertainty Modeling - Pilhyeon Lee et al, AAAI 2021.
• [CoLA] CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning - Can Zhang et al, CVPR 2021.
• [ActShufNet] Action Shuffling for Weakly Supervised Temporal Localization - Xiao-Yu Zhang et al, arXiv 2021.
• [$\mathrm{CO_2-Net}$] Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization - Fa-Ting Hong et al, ACM MM 2021.
• [HAM-Net] A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization - Ashraful Islam et al, AAAI 2021. [code]

2020

• [ECM] Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization - Tao Zhao et al, arxiv 2020
• [TCA] Learning Temporal Co-Attention Models for Unsupervised Video Action Localization - Guoqiang Gong et al, CVPR 2020
• [EM-MIL] Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance - Zhekun Luo et al, ECCV 2020.
• [SF-Net] SF-Net: Single-Frame Supervision for Temporal Action Localization - Fan Ma et al, ECCV 2020. [code]
• [A2CL-PT] Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization - Kyle Min et al, ECCV 2020.
• [TSCN] Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization - Yuanhao Zhai et al, ECCV 2020.
• [ActionBytes] ActionBytes: Learning from Trimmed Videos to Localize Actions - Mihir Jain et al, CVPR 2020.
• [DGAM] Weakly-Supervised Action Localization by Generative Attention Modeling - Baifeng Shi et al, CVPR 2020.
• [RPN] Relational Prototypical Network for Weakly Supervised Temporal Action Localization - Linjiang Huang et al, AAAI 2020.
• [BaSNet] Background Suppression Network for Weakly-supervised Temporal Action Localization - Pilhyeon Lee et al, AAAI 2020.
• [DML] Weakly Supervised Temporal Action Localization Using Deep Metric Learning - Ashraful Islam et al, WACV 2020.
• [MCASL] Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks - Maheen Rashid et al, WACV 2020.
• [WSGN] Weakly Supervised Gaussian Networks for Action Detection - Basura Fernando et al, WACV 2020.

2019

• [MAAN] Marginalized Average Attentional Network for Weakly Supervised Learning - Yuan Yuan et al, ICLR 2019.
• [IWO-Net] Breaking Winner-Takes-All: Iterative-Winners-Out Networks for Weakly Supervised Temporal Action Localization - Runhao Zeng et al, TIP 2019.
• [3C-Net] 3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization - Sanath Narayan et al, TIP 2019. [code]
• [BM] Weakly-supervised Action Localization with Background Modeling - Phuc Xuan Nguyen et al, ICCV 2019.
• [TSM] Temporal Structure Mining for Weakly Supervised Action Detection - Tan Yu et al, ICCV 2019.
• [CleanNet] Weakly Supervised Temporal Action Localization through Contrast based Evaluation Networks - Ziyi Liu et al, ICCV 2019.
• [CMCS] Completeness Modeling and Context Separation for Weakly Supervised Temporal Action Localization - Daochang Liu et al, CVPR 2019.
• [STAR] Segregated Temporal Assembly Recurrent Networks for Weakly Supervised Multiple Action Detection - Yunlu Xu et al, AAAI 2019.

2018

• [W-TALC] W-TALC: Weakly-supervised Temporal Activity Localization and Classification - Sujoy Paul et al, ECCV 2018. [code]
• [AutoLoc] AutoLoc: Weakly-supervised Temporal Action Localization in Untrimmed Videos - Zheng Shou et al, ECCV 2018. [code]
• [STPN] Weakly Supervised Action Localization by Sparse Temporal Pooling Network - Phuc Nguyen et al, CVPR 2018.
• [One-Shot] One-Shot Action Localization by Learning Sequence Matching Network - Hongtao Yang et al, CVPR 2018.

2017

• [UNet] UntrimmedNets for Weakly Supervised Action Recognition and Detection - Limin Wang et al, CVPR 2017. [code]
• [H&S] Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization - Krishna Kumar Singh et al, CVPR 2017.

Dataset

• THUMOS14
• ActivityNet v1.3

Benchmark Results

THUMOS14

ActivityNet v1.3

ActivityNet v1.2