• Stars
    star
    586
  • Rank 76,279 (Top 2 %)
  • Language Cuda
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

Β RAFT: Reusable Accelerated Functions and Tools

Navigating the canyons of accelerated possibilities

Resources

Overview

RAFT contains fundamental widely-used algorithms and primitives for data science and machine learning. The algorithms are CUDA-accelerated and form building blocks for rapidly composing analytics.

By taking a primitives-based approach to algorithm development, RAFT

  • accelerates algorithm construction time
  • reduces the maintenance burden by maximizing reuse across projects, and
  • centralizes core reusable computations, allowing future optimizations to benefit all algorithms that use them.

While not exhaustive, the following general categories help summarize the accelerated functions in RAFT:

Category Examples
Data Formats sparse & dense, conversions, data generation
Dense Operations linear algebra, matrix and vector operations, reductions, slicing, norms, factorization, least squares, svd & eigenvalue problems
Sparse Operations linear algebra, eigenvalue problems, slicing, norms, reductions, factorization, symmetrization, components & labeling
Spatial pairwise distances, nearest neighbors, neighborhood graph construction
Basic Clustering spectral clustering, hierarchical clustering, k-means
Solvers combinatorial optimization, iterative solvers
Statistics sampling, moments and summary statistics, metrics
Tools & Utilities common utilities for developing CUDA applications, multi-node multi-gpu infrastructure

RAFT is a C++ header-only template library with an optional shared library that

  1. can speed up compile times for common template types, and
  2. provides host-accessible "runtime" APIs, which don't require a CUDA compiler to use

In addition being a C++ library, RAFT also provides 2 Python libraries:

  • pylibraft - lightweight Python wrappers around RAFT's host-accessible "runtime" APIs.
  • raft-dask - multi-node multi-GPU communicator infrastructure for building distributed algorithms on the GPU with Dask.

RAFT is a C++ header-only template library with optional shared library and lightweight Python wrappers

Getting started

RAPIDS Memory Manager (RMM)

RAFT relies heavily on RMM which eases the burden of configuring different allocation strategies globally across the libraries that use it.

Multi-dimensional Arrays

The APIs in RAFT accept the mdspan multi-dimensional array view for representing data in higher dimensions similar to the ndarray in the Numpy Python library. RAFT also contains the corresponding owning mdarray structure, which simplifies the allocation and management of multi-dimensional data in both host and device (GPU) memory.

The mdarray forms a convenience layer over RMM and can be constructed in RAFT using a number of different helper functions:

#include <raft/core/device_mdarray.hpp>

int n_rows = 10;
int n_cols = 10;

auto scalar = raft::make_device_scalar<float>(handle, 1.0);
auto vector = raft::make_device_vector<float>(handle, n_cols);
auto matrix = raft::make_device_matrix<float>(handle, n_rows, n_cols);

C++ Example

Most of the primitives in RAFT accept a raft::device_resources object for the management of resources which are expensive to create, such CUDA streams, stream pools, and handles to other CUDA libraries like cublas and cusolver.

The example below demonstrates creating a RAFT handle and using it with device_matrix and device_vector to allocate memory, generating random clusters, and computing pairwise Euclidean distances:

#include <raft/core/device_resources.hpp>
#include <raft/core/device_mdarray.hpp>
#include <raft/random/make_blobs.cuh>
#include <raft/distance/distance.cuh>

raft::device_resources handle;

int n_samples = 5000;
int n_features = 50;

auto input = raft::make_device_matrix<float, int>(handle, n_samples, n_features);
auto labels = raft::make_device_vector<int, int>(handle, n_samples);
auto output = raft::make_device_matrix<float, int>(handle, n_samples, n_samples);

raft::random::make_blobs(handle, input.view(), labels.view());

auto metric = raft::distance::DistanceType::L2SqrtExpanded;
raft::distance::pairwise_distance(handle, input.view(), input.view(), output.view(), metric);

It's also possible to create raft::device_mdspan views to invoke the same API with raw pointers and shape information:

#include <raft/core/device_resources.hpp>
#include <raft/core/device_mdspan.hpp>
#include <raft/random/make_blobs.cuh>
#include <raft/distance/distance.cuh>

raft::device_resources handle;

int n_samples = 5000;
int n_features = 50;

float *input;
int *labels;
float *output;

...
// Allocate input, labels, and output pointers
...

auto input_view = raft::make_device_matrix_view(input, n_samples, n_features);
auto labels_view = raft::make_device_vector_view(labels, n_samples);
auto output_view = raft::make_device_matrix_view(output, n_samples, n_samples);

raft::random::make_blobs(handle, input_view, labels_view);

auto metric = raft::distance::DistanceType::L2SqrtExpanded;
raft::distance::pairwise_distance(handle, input_view, input_view, output_view, metric);

Python Example

The pylibraft package contains a Python API for RAFT algorithms and primitives. pylibraft integrates nicely into other libraries by being very lightweight with minimal dependencies and accepting any object that supports the __cuda_array_interface__, such as CuPy's ndarray. The number of RAFT algorithms exposed in this package is continuing to grow from release to release.

The example below demonstrates computing the pairwise Euclidean distances between CuPy arrays. Note that CuPy is not a required dependency for pylibraft.

import cupy as cp

from pylibraft.distance import pairwise_distance

n_samples = 5000
n_features = 50

in1 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)
in2 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)

output = pairwise_distance(in1, in2, metric="euclidean")

The output array in the above example is of type raft.common.device_ndarray, which supports cuda_array_interface making it interoperable with other libraries like CuPy, Numba, PyTorch and RAPIDS cuDF that also support it. CuPy supports DLPack, which also enables zero-copy conversion from raft.common.device_ndarray to JAX and Tensorflow.

Below is an example of converting the output pylibraft.device_ndarray to a CuPy array:

cupy_array = cp.asarray(output)

And converting to a PyTorch tensor:

import torch

torch_tensor = torch.as_tensor(output, device='cuda')

Or converting to a RAPIDS cuDF dataframe:

cudf_dataframe = cudf.DataFrame(output)

When the corresponding library has been installed and available in your environment, this conversion can also be done automatically by all RAFT compute APIs by setting a global configuration option:

import pylibraft.config
pylibraft.config.set_output_as("cupy")  # All compute APIs will return cupy arrays
pylibraft.config.set_output_as("torch") # All compute APIs will return torch tensors

You can also specify a callable that accepts a pylibraft.common.device_ndarray and performs a custom conversion. The following example converts all output to numpy arrays:

pylibraft.config.set_output_as(lambda device_ndarray: return device_ndarray.copy_to_host())

pylibraft also supports writing to a pre-allocated output array so any __cuda_array_interface__ supported array can be written to in-place:

import cupy as cp

from pylibraft.distance import pairwise_distance

n_samples = 5000
n_features = 50

in1 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)
in2 = cp.random.random_sample((n_samples, n_features), dtype=cp.float32)
output = cp.empty((n_samples, n_samples), dtype=cp.float32)

pairwise_distance(in1, in2, out=output, metric="euclidean")

Installing

RAFT itself can be installed through conda, CMake Package Manager (CPM), pip, or by building the repository from source. Please refer to the build instructions for more a comprehensive guide on installing and building RAFT and using it in downstream projects.

Conda

The easiest way to install RAFT is through conda and several packages are provided.

  • libraft-headers RAFT headers
  • libraft (optional) shared library of pre-compiled template instantiations and runtime APIs.
  • pylibraft (optional) Python wrappers around RAFT algorithms and primitives.
  • raft-dask (optional) enables deployment of multi-node multi-GPU algorithms that use RAFT raft::comms in Dask clusters.

Use the following command to install all of the RAFT packages with conda (replace rapidsai with rapidsai-nightly to install more up-to-date but less stable nightly packages). mamba is preferred over the conda command.

mamba install -c rapidsai -c conda-forge -c nvidia raft-dask pylibraft

You can also install the conda packages individually using the mamba command above.

After installing RAFT, find_package(raft COMPONENTS compiled distributed) can be used in your CUDA/C++ cmake build to compile and/or link against needed dependencies in your raft target. COMPONENTS are optional and will depend on the packages installed.

Pip

pylibraft and raft-dask both have experimental packages that can be installed through pip:

pip install pylibraft-cu11 --extra-index-url=https://pypi.nvidia.com
pip install raft-dask-cu11 --extra-index-url=https://pypi.nvidia.com

CMake & CPM

RAFT uses the RAPIDS-CMake library, which makes it easy to include in downstream cmake projects. RAPIDS-CMake provides a convenience layer around CPM. Please refer to these instructions to install and use rapids-cmake in your project.

Example Template Project

You can find an example RAFT project template in the cpp/template directory, which demonstrates how to build a new application with RAFT or incorporate RAFT into an existing cmake project.

CMake Targets

Additional CMake targets can be made available by adding components in the table below to the RAFT_COMPONENTS list above, separated by spaces. The raft::raft target will always be available. RAFT headers require, at a minimum, the CUDA toolkit libraries and RMM dependencies.

Component Target Description Base Dependencies
n/a raft::raft Full RAFT header library CUDA toolkit, RMM, NVTX, CCCL, CUTLASS
compiled raft::compiled Pre-compiled template instantiations and runtime library raft::raft
distributed raft::distributed Dependencies for raft::comms APIs raft::raft, UCX, NCCL

Source

The easiest way to build RAFT from source is to use the build.sh script at the root of the repository:

  1. Create an environment with the needed dependencies:
mamba env create --name raft_dev_env -f conda/environments/all_cuda-118_arch-x86_64.yaml
mamba activate raft_dev_env
./build.sh raft-dask pylibraft libraft tests bench --compile-lib

The build instructions contain more details on building RAFT from source and including it in downstream projects. You can also find a more comprehensive version of the above CPM code snippet the Building RAFT C++ from source section of the build instructions.

Folder Structure and Contents

The folder structure mirrors other RAPIDS repos, with the following folders:

  • ci: Scripts for running CI in PRs
  • conda: Conda recipes and development conda environments
  • cpp: Source code for C++ libraries.
    • bench: Benchmarks source code
    • cmake: CMake modules and templates
    • doxygen: Doxygen configuration
    • include: The C++ API headers are fully-contained here (deprecated directories are excluded from the listing below)
      • cluster: Basic clustering primitives and algorithms.
      • comms: A multi-node multi-GPU communications abstraction layer for NCCL+UCX and MPI+NCCL, which can be deployed in Dask clusters using the raft-dask Python package.
      • core: Core API headers which require minimal dependencies aside from RMM and Cudatoolkit. These are safe to expose on public APIs and do not require nvcc to build. This is the same for any headers in RAFT which have the suffix *_types.hpp.
      • distance: Distance primitives
      • linalg: Dense linear algebra
      • matrix: Dense matrix operations
      • neighbors: Nearest neighbors and knn graph construction
      • random: Random number generation, sampling, and data generation primitives
      • solver: Iterative and combinatorial solvers for optimization and approximation
      • sparse: Sparse matrix operations
        • convert: Sparse conversion functions
        • distance: Sparse distance computations
        • linalg: Sparse linear algebra
        • neighbors: Sparse nearest neighbors and knn graph construction
        • op: Various sparse operations such as slicing and filtering (Note: this will soon be renamed to sparse/matrix)
        • solver: Sparse solvers for optimization and approximation
      • stats: Moments, summary statistics, model performance measures
      • util: Various reusable tools and utilities for accelerated algorithm development
    • internal: A private header-only component that hosts the code shared between benchmarks and tests.
    • scripts: Helpful scripts for development
    • src: Compiled APIs and template instantiations for the shared libraries
    • template: A skeleton template containing the bare-bones file structure and cmake configuration for writing applications with RAFT.
    • test: Googletests source code
  • docs: Source code and scripts for building library documentation (Uses breath, doxygen, & pydocs)
  • python: Source code for Python libraries.
    • pylibraft: Python build and source code for pylibraft library
    • raft-dask: Python build and source code for raft-dask library
  • thirdparty: Third-party licenses

Contributing

If you are interested in contributing to the RAFT project, please read our Contributing guidelines. Refer to the Developer Guide for details on the developer guidelines, workflows, and principals.

References

When citing RAFT generally, please consider referencing this Github project.

@misc{rapidsai,
  title={Rapidsai/raft: RAFT contains fundamental widely-used algorithms and primitives for data science, Graph and machine learning.},
  url={https://github.com/rapidsai/raft},
  journal={GitHub},
  publisher={Nvidia RAPIDS},
  author={Rapidsai},
  year={2022}
}

If citing the sparse pairwise distances API, please consider using the following bibtex:

@article{nolet2021semiring,
  title={Semiring primitives for sparse neighborhood methods on the gpu},
  author={Nolet, Corey J and Gala, Divye and Raff, Edward and Eaton, Joe and Rees, Brad and Zedlewski, John and Oates, Tim},
  journal={arXiv preprint arXiv:2104.06357},
  year={2021}
}

More Repositories

1

cudf

cuDF - GPU DataFrame Library
C++
8,319
star
2

cuml

cuML - RAPIDS Machine Learning Library
C++
3,864
star
3

cugraph

cuGraph - RAPIDS Graph Analytics Library
Cuda
1,668
star
4

cusignal

cuSignal - RAPIDS Signal Processing Library
Python
703
star
5

jupyterlab-nvdashboard

A JupyterLab extension for displaying dashboards of GPU usage.
TypeScript
582
star
6

notebooks

RAPIDS Sample Notebooks
Shell
577
star
7

cuspatial

CUDA-accelerated GIS and spatiotemporal algorithms
Jupyter Notebook
543
star
8

rmm

RAPIDS Memory Manager
C++
420
star
9

deeplearning

Jupyter Notebook
336
star
10

cucim

cuCIM - RAPIDS GPU-accelerated image processing library
Jupyter Notebook
333
star
11

dask-cuda

Utilities for Dask and CUDA interactions
Python
266
star
12

cuxfilter

GPU accelerated cross filtering with cuDF.
Python
261
star
13

node

GPU-accelerated data science and visualization in node
TypeScript
170
star
14

clx

A collection of RAPIDS examples for security analysts, data scientists, and engineers to quickly get started applying RAPIDS and GPU acceleration to real-world cybersecurity use cases.
Jupyter Notebook
167
star
15

libgdf

[ARCHIVED] C GPU DataFrame Library
Cuda
138
star
16

dask-cudf

[ARCHIVED] Dask support for distributed GDF object --> Moved to cudf
Python
135
star
17

cloud-ml-examples

A collection of Machine Learning examples to get started with deploying RAPIDS in the Cloud
Jupyter Notebook
134
star
18

ucx-py

Python bindings for UCX
Python
118
star
19

gpu-bdb

RAPIDS GPU-BDB
Python
103
star
20

kvikio

KvikIO - High Performance File IO
Python
100
star
21

plotly-dash-rapids-census-demo

Jupyter Notebook
92
star
22

gputreeshap

C++
83
star
23

frigate

Frigate is a tool for automatically generating documentation for your Helm charts
Python
76
star
24

wholegraph

WholeGraph - large scale Graph Neural Networks
Cuda
75
star
25

spark-examples

[ARCHIVED] Moved to github.com/NVIDIA/spark-xgboost-examples
Jupyter Notebook
70
star
26

docker

Dockerfile templates for creating RAPIDS Docker Images
Shell
69
star
27

cuvs

cuVS - a library for vector search and clustering on the GPU
Jupyter Notebook
57
star
28

custrings

[ARCHIVED] GPU String Manipulation --> Moved to cudf
Cuda
46
star
29

docs

RAPIDS Documentation Site
HTML
34
star
30

cudf-alpha

[ARCHIVED] cuDF [alpha] - RAPIDS Merge of GoAi into cuDF
34
star
31

rapids-examples

Jupyter Notebook
31
star
32

nvgraph

C++
26
star
33

rapids-cmake

CMake
24
star
34

cuhornet

Cuda
24
star
35

cuDataShader

Jupyter Notebook
22
star
36

gpuci-build-environment

Common build environment used by gpuCI for building RAPIDS
Dockerfile
19
star
37

distributed-join

C++
19
star
38

devcontainers

Shell
18
star
39

dask-cuml

[ARCHIVED] Dask support for multi-GPU machine learning algorithms --> Moved to cuml
Python
16
star
40

integration

RAPIDS - combined conda package & integration tests for all of RAPIDS libraries
Shell
15
star
41

xgboost-conda

Conda recipes for xgboost
Jupyter Notebook
12
star
42

benchmark

Python
11
star
43

ucxx

C++
11
star
44

dependency-file-generator

Python
10
star
45

asvdb

Python
9
star
46

helm-chart

Shell
9
star
47

deployment

RAPIDS Deployment Documentation
Jupyter Notebook
9
star
48

miniforge-cuda

Dockerfile
9
star
49

ci-imgs

Dockerfile
7
star
50

dask-cugraph

Python
7
star
51

rapids.ai

rapids.ai web site
HTML
7
star
52

ptxcompiler

Python
6
star
53

GaaS

Python
5
star
54

rvc

Go
4
star
55

scikit-learn-nv

Python
4
star
56

ops-bot

A Probot application used by the Ops team for automation.
TypeScript
4
star
57

workflows

Shell
4
star
58

rapids-triton

C++
4
star
59

dask-build-environment

Build environments for various dask related projects on gpuCI
Dockerfile
3
star
60

roc

GitHub utilities for the RAPIDS Ops team
Go
3
star
61

multi-gpu-tools

Shell
3
star
62

detect-weak-linking

Python
3
star
63

dask-cuda-benchmarks

Python
2
star
64

shared-workflows

Reusable GitHub Actions workflows for RAPIDS CI
Shell
2
star
65

rapids_triton_pca_example

C++
2
star
66

cugunrock

Cuda
2
star
67

dgl-cugraph-build-environment

Dockerfile
2
star
68

projects

Jupyter Notebook
2
star
69

crossfit

Metric calculation library
Python
2
star
70

gpuci-mgmt

Mangement scripts for gpuCI
Shell
1
star
71

ansible-roles

1
star
72

code-share

C++
1
star
73

build-metrics-reporter

Python
1
star
74

cibuildwheel-imgs

Dockerfile
1
star
75

gpuci-tools

User tools for use within the gpuCI environment
Shell
1
star
76

pynvjitlink

Python
1
star
77

rapids-dask-dependency

Shell
1
star
78

sphinx-theme

This repository contains a Sphinx theme used for RAPIDS documentation
CSS
1
star