Awesome-Referring-Image-Segmentation

A collection of referring image segmentation papers and datasets.

Feel free to create a PR or an issue.

Outline

Awesome-Referring-Image-Segmentation

1. Datasets

Short name	Paper	Source	Code/Project Link
ReferIt	Referit game: Referring to objects in photographs of natural scenes	EMNLP 2014	[project]
Google-Ref	Generation and comprehension of unambiguous object descriptions	CVPR 2016	[dataset]
UNC	Modeling context in referring expressions	ECCV 2016	[dataset]
UNC+	Modeling context in referring expressions	ECCV 2016	[dataset]
CLEVR-Ref+	CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions	CVPR 2019	[project]
VGPhraseCut	PhraseCut: Language-based Image Segmentation in the Wild	CVPR 2020	[project]
ScanRefer	ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language	ECCV 2020	[project]
ClevrTex	ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation	NeurIPS Datasets and Benchmarks 2021	[project]
gRefCOCO	GRES: Generalized Referring Expression Segmentation	CVPR 2023	[dataset] [project]
MeViS	MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions	ICCV 2023	[dataset] [project]

2. Traditional Referring Image Segmentation

Short name	Paper	Source	Code/Project Link
LISA	LISA: Reasoning Segmentation via Large Language Model	arXiv 23.08	[code]
ETRIS	Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation	ICCV 2023	[code]
SEEM	Segment Everything Everywhere All at Once	arXiv 23.04	[code]
SLViT	SLViT: Scale-Wise Language-Guided Vision Transformer for Referring Image Segmentation	IJCAI 2023	[code]
WiCo	WiCo: Win-win Cooperation of Bottom-up and Top-down Referring Image Segmentation	IJCAI 2023
M3Att	Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation	TIP 2023
X-Decoder	X-Decoder: Generalized Decoding for Pixel, Image and Language	CVPR 2023	[code] [project]
Partial-RES	Learning to Segment Every Referring Object Point by Point	CVPR 2023	[code]
MCRES	Meta Compositional Referring Expression Segmentation	CVPR 2023
Global-Local CLIP	Zero-shot Referring Image Segmentation with Global-Local Context Features	CVPR 2023	[code]
PolyFormer	PolyFormer: Referring Image Segmentation as Sequential Polygon Generation	CVPR 2023	[code] [project]
GRES	GRES: Generalized Referring Expression Segmentation	CVPR 2023	[code] [dataset] [project]
CGFormer	Contrastive Grouping with Transformer for Referring Image Segmentation	CVPR 2023	[code]
SADLR	Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation	AAAI 2023
R-RIS	Towards Robust Referring Image Segmentation	arXiv 22.09	[code] [project]
-	Learning From Box Annotations for Referring Image Segmentation	TNNLS 2022	[code]
-	Instance-Specific Feature Propagation for Referring Segmentation	TMM 2022
LAVT	LAVT: Language-Aware Vision Transformer for Referring Image Segmentation	CVPR 2022	[code]
CRIS	CRIS: CLIP-Driven Referring Image Segmentation	CVPR 2022	[code]
ReSTR	ReSTR: Convolution-free Referring Image Segmentation Using Transformers	CVPR 2022	[project]
TV-Net	Two-stage Visual Cues Enhancement Network for Referring Image Segmentation	ACM MM 2021	[code]
VLT	Vision-Language Transformer and Query Generation for Referring Segmentation	ICCV 2021	[code]
MDETR	MDETR - Modulated Detection for End-to-End Multi-Modal Understanding	ICCV 2021	[code] [project]
CEFNet	Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentation	CVPR 2021	[code]
BUSNet	Bottom-Up Shift and Reasoning for Referring Image Segmentation	CVPR 2021	[code]
LTS	Locate then Segment: A Strong Pipeline for Referring Image Segmentation	CVPR 2021
CGAN	Cascade Grouped Attention Network for Referring Expression Segmentation	ACM MM 2020
LSCM	Linguistic Structure Guided Context Modeling for Referring Image Segmentation	ECCV 2020	[code]
CMPC-Refseg	Referring Image Segmentation via Cross-Modal Progressive Comprehension	CVPR 2020	[code]
BRINet	Bi-directional Relationship Inferring Network for Referring Image Segmentation	CVPR 2020	[code]
PhraseCut	PhraseCut: Language-based Image Segmentation in the Wild	CVPR 2020	[code] [project]
MCN	Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation	CVPR 2020	[code]
-	Dual Convolutional LSTM Network for Referring Image Segmentation	TMM 2020
STEP	See-Through-Text Grouping for Referring Image Segmentation	ICCV 2019
lang2seg	Referring Expression Object Segmentation with Caption-Aware Consistency	BMVC 2019	[code]
CMSA	Cross-Modal Self-Attention Network for Referring Image Segmentation	CVPR 2019	[code]
KWA	Key-Word-Aware Network for Referring Expression Image Segmentation	ECCV 2018	[code]
DMN	Dynamic Multimodal Instance Segmentation Guided by Natural Language Queries	ECCV 2018	[code]
RRN	Referring Image Segmentation via Recurrent Refinement Networks	CVPR 2018	[code]
MAttNet	MAttNet: Modular Attention Network for Referring Expression Comprehension	CVPR 2018	[code] [Demo]
RMI	Recurrent Multimodal Interaction for Referring Image Segmentation	ICCV 2017	[code]
LSTM-CNN	Segmentation from natural language expressions	ECCV 2016	[code] [project]

3. Interactive Referring Image Segmentation

Short name	Paper	Source	Code/Project Link
PhraseClick	PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click	ECCV 2020

4. Referring Video Object Segmentation

Short name	Paper	Source	Code/Project Link
LMPM	MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions	ICCV 2023	[code] [project]
OnlineRefer	OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation	ICCV 2023	[code]
SgMg	Spectrum-guided Multi-granularity Referring Video Object Segmentation	ICCV 2023	[code]
R2VOS	Towards Robust Referring Video Object Segmentation with Cyclic Relational Consistency	ICCV 2023	[code]
MANet	Multi-Attention Network for Compressed Video Referring Object Segmentation	ACM MM 2022	[code]
MTTR	End-to-End Referring Video Object Segmentation with Multimodal Transformers	CVPR 2022	[code]
ReferFormer	Language as Queries for Referring Video Object Segmentation	CVPR 2022	[code]
LBDT	Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation	CVPR 2022	[code]
-	Multi-Level Representation Learning with Semantic Alignment for Referring Video Object Segmentation	CVPR 2022
YOFO	You Only Infer Once: Cross-Modal Meta-Transfer for Referring Video Object Segmentation	AAAI 2022
RefVOS	RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation	arXiv 20.10
URVOS	URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark	ECCV 2020	[code]
	Video Object Segmentation with Language Referring Expressions	ACCV 2018

5. Referring 3D Instance Segmentation

Short name	Paper	Source	Code/Project Link
TGNN	Text-Guided Graph Neural Networks for Referring 3D Instance Segmentation	AAAI 2021
InstanceRefer	InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring	ICCV 2021	[code]

MarkMoHR/Awesome-Referring-Image-Segmentation

MarkMoHR

Reviews

Repository Details

Awesome-Referring-Image-Segmentation

1. Datasets

2. Traditional Referring Image Segmentation

3. Interactive Referring Image Segmentation

4. Referring Video Object Segmentation

5. Referring 3D Instance Segmentation

More Repositories