• Stars
    star
    141
  • Rank 259,971 (Top 6 %)
  • Language Cuda
  • License
    MIT License
  • Created almost 5 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

GGNN: State of the Art Graph-based GPU Nearest Neighbor Search

GGNN: Graph-based GPU Nearest Neighbor Search

*Fabian Groh, Lukas Ruppert, Patrick Wieschollek, Hendrik P.A. Lensch

Approximate nearest neighbor (ANN) search in high dimensions is an integral part of several computer vision systems and gains importance in deep learning with explicit memory representations. Since PQT and FAISS started to leverage the massive parallelism offered by GPUs, GPU-based implementations are a crucial resource for today’s state-of-the-art ANN methods. While most of these methods allow for faster queries, less emphasis is devoted to accelerate the construction of the underlying index structures. In this paper, we propose a novel search structure based on nearest neighbor graphs and information propagation on graphs. Our method is designed to take advantage of GPU architectures to accelerate the hierarchical building of the index structure and for performing the query. Empirical evaluation shows that GGNN significantly surpasses the state-of-the-art GPU- and CPU-based systems in terms of build-time, accuracy and search speed.


Update: 03/14/21

This work is currently under review. We've updated the code. Some new results are shown below. GGNN Plots

How to run the code?

# Get the repository and dependencies
git clone --recursive https://github.com/cgtuebingen/ggnn.git
cd ggnn

# get the SIFT1M data
cd data
./get_sift1m
cd ..

# Build the demo
mkdir build_local
cd build_local
cmake ..
make

# Example for SIFT1M on GPU 0:
./sift1m_multi  --base_filename ../data/sift/sift_base.fvecs 
                --query_filename ../data/sift/sift_query.fvecs 
                --groundtruth_filename ../data/sift/sift_groundtruth.ivecs 
                --gpu_ids="0"

# Example usage with 2 GPUs and 4 shards in total (4x250k = 1M):
./sift1m_multi  --base_filename ../data/sift/sift_base.fvecs 
                --query_filename ../data/sift/sift_query.fvecs 
                --groundtruth_filename ../data/sift/sift_groundtruth.ivecs 
                --gpu_ids="0 1" 
                --factor 10000 
                --base 100 
                --shard 25

Native build

Requirements:

  • CUDA (>10.2)
  • libgflags-dev (sudo apt install libgflags-dev)

Docker build

An alternative to the native build is to use nvidia-docker. Follow instruction on https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#installing-docker-ce

Prepare the docker image by

cd docker
make
cd ../

Make sure you can run

sudo docker run --gpus all cgtuebingen/ggnn:v1 nvidia-smi

Now build the code via

user@host $ sudo docker run --rm -it --user "$(id -u):$(id -g)" -v ${PWD}:/ggnn:rw --gpus all cgtuebingen/ggnn:v1 bash
user@container $ ./build.sh

cd build_docker
make

More Resources

More Repositories

1

NeRD-Neural-Reflectance-Decomposition

NeRD: Neural Reflectance Decomposition from Image Collections - ICCV 2021
Python
246
star
2

Flex-Convolution

Source code for: Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds), accepted at ACCV 2018
C++
115
star
3

SIGNeRF

SIGNeRF: Scene Integrated Generation for Neural Radiance Fields
Python
112
star
4

learning-blind-motion-deblurring

Multiframe Image Deconvolution (ICCV17)
Python
106
star
5

Neural-PIL

Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition - NeurIPS2021
Python
99
star
6

Product-Quantization-Tree

GPU-based large scale Approx. Nearest Neighbor Search, accepted at CVPR 2016
Cuda
93
star
7

pointcloud-viewer

Efficient Large-Scale Point-Cloud Viewer based on OpenGL
C++
79
star
8

will-people-like-your-image

Image Aesthetics Estimation (WACV18)
Python
61
star
9

SpatialDETR

Official implementation of SpatialDETR. The paper will be presented at ECCV 2022
Python
55
star
10

tf_custom_op

Boilerplate template for adding custom operations to TensorFlow
C++
14
star
11

emca

EMCA: Explorer of Monte Carlo based Algorithms
Python
12
star
12

low-poly-painter

Python
11
star
13

LearningToSynchronizeVideos

Implementation of our Video Synchronization paper in TensorFlow (ICMLA 2017)
C
7
star
14

infomark-ui

Front-end for the Infomark online platform
Elm
6
star
15

MedicalAnnotationFramework

Python
4
star
16

shinobi

SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
TypeScript
3
star
17

DualQueryMIL

Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification
Python
3
star
18

hyperrealistic_indoor_streetview

🗺️🏠 Software Project SS24 | Hyperrealistic Indoor Street-View
TypeScript
3
star
19

infomark-backend

scalable, modern and open-source online course management system with auto testing of programming assignments (RESTful API Server)
Go
2
star
20

praktikum-ws16-funfair

praktikum-ws16-funfair created by GitHub Classroom
JavaScript
2
star
21

InfoMark-deprecated

A online platform for distributing exercise sheets and testing exercise submission within Docker
Ruby
2
star
22

mitsuba-emca

Integration of EMCA https://github.com/cgtuebingen/emca into Mitsuba
C++
2
star
23

jannik-hofmann-master-thesis

DNN Visualization in the Unreal Engine for Interactive Fly-through Exploration, an extensible framework for visualizing feed-forward TensorFlow/Keras DNNs. It calculates a force-based layout and displays kernels, kernel activations, saliency maps, and integrated gradients. Published under GNU GPLv3.
C
2
star
24

fastsync

C++
1
star
25

hybrid_zoom

This is the simulation of the hybrid zoom based on the SLM-camera.
Python
1
star
26

snapshot_multispectral_imaging

MATLAB
1
star
27

spatial_xr

🥽🌐 Software Project SS24 | SpatialXR
C#
1
star