-
Stars70
-
Rank 445,097 (Top 9 %)
-
LanguagePython
-
Created 4 months ago
-
Updated 3 months ago
Reviews
There are no reviews yet. Be the first to send feedback to the community and the maintainers!
Repository Details
This is the official implementation of the paper "Needle In A Multimodal Haystack"
More Repositories
1
LLaMA-Adapter
[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M ParametersPython
5,618
2
InternVL
[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型Python
5,059
3
DragGAN
Unofficial Implementation of DragGAN - "Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold" （DragGAN 全功能实现，在线Demo，本地部署试用，代码、模型已全部开源，支持Windows, macOS, Linux）Python
4,996
4
InternGPT
InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)Python
3,178
5
Ask-Anything
[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.Python
2,943
6
InternImage
[CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable ConvolutionsPython
2,474
7
InternVideo
[ECCV2024] Video Foundation Models & Data for Multimodal UnderstandingPython
1,258
8
VisionLLM
VisionLLM SeriesPython
801
9
VideoMamba
[ECCV2024] VideoMamba: State Space Model for Efficient Video UnderstandingPython
755
10
OmniQuant
[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.Python
643
11
VideoMAEv2
[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual MaskingPython
470
12
DCNv4
[CVPR 2024] Deformable Convolution v4Python
463
13
GITM
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
445
14
all-seeing
[ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"Python
436
15
Multi-Modality-Arena
Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!Python
428
16
CaFo
[CVPR 2023] Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot LearnersPython
335
17
PonderV2
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training ParadigmPython
311
18
Vision-RWKV
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like ArchitecturesPython
297
19
LAMM
[NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI AgentsPython
286
20
UniFormerV2
[ICCV2023] UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormerPython
280
21
unmasked_teacher
[ICCV2023 Oral] Unmasked Teacher: Towards Training-Efficient Video Foundation ModelsPython
276
22
Instruct2Act
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language ModelPython
223
23
HumanBench
This repo is official implementation of HumanBench (CVPR2023)Python
218
24
OmniCorpus
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with TextHTML
210
25
gv-benchmark
General Vision Benchmark, GV-B, a project from OpenGVLabPython
188
26
ControlLLM
ControlLLM: Augment Language Models with Tools by Searching on GraphsPython
181
27
InternVideo2
152
28
UniHCP
Official PyTorch implementation of UniHCPPython
145
29
efficient-video-recognition
Python
114
30
SAM-Med2D
Official implementation of SAM-Med2DJupyter Notebook
114
31
EgoVideo
[CVPR 2024 Champions] Solutions for EgoVis Chanllenges in CVPR 2024Jupyter Notebook
96
32
DiffRate
[ICCV 23]An approach to enhance the efficiency of Vision Transformer (ViT) by concurrently employing token pruning and token merging techniques, while incorporating a differentiable compression rate.Jupyter Notebook
78
33
Awesome-DragGAN
Awesome-DragGAN: A curated list of papers, tutorials, repositories related to DragGAN
75
34
M3I-Pretraining
69
35
STM-Evaluation
Python
69
36
MMT-Bench
ICML'2024 | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIPython
62
37
MUTR
[AAAI 2024] Referred by Multi-Modality: A Unified Temporal Transformers for Video Object SegmentationPython
60
38
ChartAst
ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.Python
60
39
LCL
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression LearningPython
57
40
DDPS
Official Implementation of "Denoising Diffusion Semantic Segmentation with Mask Prior Modeling"Python
53
41
LORIS
Long-Term Rhythmic Video Soundtracker, ICML2023Python
52
42
Awesome-LLM4Tool
A curated list of the papers, repositories, tutorials, and anythings related to the large language models for tools
52
43
GUI-Odyssey
GUI Odyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUI Odyssey consists of 7,735 episodes from 6 mobile devices, spanning 6 types of cross-app tasks, 201 apps, and 1.4K app combos.Python
47
44
PIIP
Parameter-Inverted Image Pyramid Networks (PIIP)Python
44
45
InternVL-MMDetSeg
Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeedJupyter Notebook
40
46
Siamese-Image-Modeling
[CVPR 2023]Implementation of Siamese Image Modeling for Self-Supervised Vision Representation LearningPython
32
47
Multitask-Model-Selector
Implementation of Foundation Model is Efficient Multimodal Multitask Model SelectorPython
27
48
De-focus-Attention-Networks
Learning 1D Causal Visual Representation with De-focus Attention NetworksPython
27
49
Official-ConvMAE-Det
Python
13
50
opengvlab.github.io
12
51
MovieMind
9
52
perception_test_iccv2023
Champion Solutions repository for Perception Test challenges in ICCV2023 workshop.Python
9
53
EmbodiedGPT
5
54
DriveMLM
3
55
.github
2