NVIDIA/cuda-samples NVIDIA/cuda-samples

  • Stars
    star
    5,203
  • Rank 7,571 (Top 0.2 %)
  • Language
    C
  • License
    Other
  • Created about 6 years ago
  • Updated about 2 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
NVIDIA/cuda-samples
github

NVIDIA

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

CUDA Samples

Samples for CUDA Developers which demonstrates features in CUDA Toolkit. This version supports CUDA Toolkit 12.2.

Release Notes

This section describes the release notes for the CUDA Samples on GitHub only.

CUDA 12.2

older versions...

Getting Started

Prerequisites

Download and install the CUDA Toolkit 12.2 for your corresponding platform. For system requirements and installation instructions of cuda toolkit, please refer to the Linux Installation Guide, and the Windows Installation Guide.

Getting the CUDA Samples

Using git clone the repository of CUDA Samples using the command below.

git clone https://github.com/NVIDIA/cuda-samples.git

Without using git the easiest way to use these samples is to download the zip file containing the current version by clicking the "Download ZIP" button on the repo page. You can then unzip the entire archive and use the samples.

Building CUDA Samples

Windows

The Windows samples are built using the Visual Studio IDE. Solution files (.sln) are provided for each supported version of Visual Studio, using the format:

*_vs<version>.sln - for Visual Studio <version>

Complete samples solution files exist at parent directory of the repo:

Each individual sample has its own set of solution files at: <CUDA_SAMPLES_REPO>\Samples\<sample_dir>\

To build/examine all the samples at once, the complete solution files should be used. To build/examine a single sample, the individual sample solution files should be used.

Linux

The Linux samples are built using makefiles. To use the makefiles, change the current directory to the sample directory you wish to build, and run make:

$ cd <sample_dir>
$ make

The samples makefiles can take advantage of certain options:

  • TARGET_ARCH= - cross-compile targeting a specific architecture. Allowed architectures are x86_64, ppc64le, armv7l, aarch64. By default, TARGET_ARCH is set to HOST_ARCH. On a x86_64 machine, not setting TARGET_ARCH is the equivalent of setting TARGET_ARCH=x86_64.
    $ make TARGET_ARCH=x86_64
    $ make TARGET_ARCH=ppc64le
    $ make TARGET_ARCH=armv7l
    $ make TARGET_ARCH=aarch64
    See here for more details on cross platform compilation of cuda samples.

  • dbg=1 - build with debug symbols

    $ make dbg=1

  • SMS="A B ..." - override the SM architectures for which the sample will be built, where "A B ..." is a space-delimited list of SM architectures. For example, to generate SASS for SM 50 and SM 60, use SMS="50 60".

    $ make SMS="50 60"

  • HOST_COMPILER=<host_compiler> - override the default g++ host compiler. See the Linux Installation Guide for a list of supported host compilers.

    $ make HOST_COMPILER=g++

Samples list

0. Introduction

Basic CUDA samples for beginners that illustrate key concepts with using CUDA and CUDA runtime APIs.

1. Utilities

Utility samples that demonstrate how to query device capabilities and measure GPU/CPU bandwidth.

2. Concepts and Techniques

Samples that demonstrate CUDA related concepts and common problem solving techniques.

3. CUDA Features

Samples that demonstrate CUDA Features (Cooperative Groups, CUDA Dynamic Parallelism, CUDA Graphs etc).

4. CUDA Libraries

Samples that demonstrate how to use CUDA platform libraries (NPP, NVJPEG, NVGRAPH cuBLAS, cuFFT, cuSPARSE, cuSOLVER and cuRAND).

5. Domain Specific

Samples that are specific to domain (Graphics, Finance, Image Processing).

6. Performance

Samples that demonstrate performance optimization.

7. libNVVM

Samples that demonstrate the use of libNVVVM and NVVM IR.

Dependencies

Some CUDA Samples rely on third-party applications and/or libraries, or features provided by the CUDA Toolkit and Driver, to either build or execute. These dependencies are listed below.

If a sample has a third-party dependency that is available on the system, but is not installed, the sample will waive itself at build time.

Each sample's dependencies are listed in its README's Dependencies section.

Third-Party Dependencies

These third-party dependencies are required by some CUDA samples. If available, these dependencies are either installed on your system automatically, or are installable via your system's package manager (Linux) or a third-party website.

FreeImage

FreeImage is an open source imaging library. FreeImage can usually be installed on Linux using your distribution's package manager system. FreeImage can also be downloaded from the FreeImage website.

To set up FreeImage on a Windows system, extract the FreeImage DLL distribution into the folder ../../../Common/FreeImage/Dist/x64 such that it contains the .h and .lib files. Copy the .dll file to root level bin/win64/Debug and bin/win64/Release folder.

Message Passing Interface

MPI (Message Passing Interface) is an API for communicating data between distributed processes. A MPI compiler can be installed using your Linux distribution's package manager system. It is also available on some online resources, such as Open MPI. On Windows, to build and run MPI-CUDA applications one can install MS-MPI SDK.

Only 64-Bit

Some samples can only be run on a 64-bit operating system.

DirectX

DirectX is a collection of APIs designed to allow development of multimedia applications on Microsoft platforms. For Microsoft platforms, NVIDIA's CUDA Driver supports DirectX. Several CUDA Samples for Windows demonstrates CUDA-DirectX Interoperability, for building such samples one needs to install Microsoft Visual Studio 2012 or higher which provides Microsoft Windows SDK for Windows 8.

DirectX12

DirectX 12 is a collection of advanced low-level programming APIs which can reduce driver overhead, designed to allow development of multimedia applications on Microsoft platforms starting with Windows 10 OS onwards. For Microsoft platforms, NVIDIA's CUDA Driver supports DirectX. Few CUDA Samples for Windows demonstrates CUDA-DirectX12 Interoperability, for building such samples one needs to install Windows 10 SDK or higher, with VS 2015 or VS 2017.

OpenGL

OpenGL is a graphics library used for 2D and 3D rendering. On systems which support OpenGL, NVIDIA's OpenGL implementation is provided with the CUDA Driver.

OpenGL ES

OpenGL ES is an embedded systems graphics library used for 2D and 3D rendering. On systems which support OpenGL ES, NVIDIA's OpenGL ES implementation is provided with the CUDA Driver.

Vulkan

Vulkan is a low-overhead, cross-platform 3D graphics and compute API. Vulkan targets high-performance realtime 3D graphics applications such as video games and interactive media across all platforms. On systems which support Vulkan, NVIDIA's Vulkan implementation is provided with the CUDA Driver. For building and running Vulkan applications one needs to install the Vulkan SDK.

OpenMP

OpenMP is an API for multiprocessing programming. OpenMP can be installed using your Linux distribution's package manager system. It usually comes preinstalled with GCC. It can also be found at the OpenMP website.

Screen

Screen is a windowing system found on the QNX operating system. Screen is usually found as part of the root filesystem.

X11

X11 is a windowing system commonly found on *-nix style operating systems. X11 can be installed using your Linux distribution's package manager, and comes preinstalled on Mac OS X systems.

EGL

EGL is an interface between Khronos rendering APIs (such as OpenGL, OpenGL ES or OpenVG) and the underlying native platform windowing system.

EGLOutput

EGLOutput is a set of EGL extensions which allow EGL to render directly to the display.

EGLSync

EGLSync is a set of EGL extensions which provides sync objects that are synchronization primitive, representing events whose completion can be tested or waited upon.

NVSCI

NvSci is a set of communication interface libraries out of which CUDA interops with NvSciBuf and NvSciSync. NvSciBuf allows applications to allocate and exchange buffers in memory. NvSciSync allows applications to manage synchronization objects which coordinate when sequences of operations begin and end.

NvMedia

NvMedia provides powerful processing of multimedia data for true hardware acceleration across NVIDIA Tegra devices. Applications leverage the NvMedia Application Programming Interface (API) to process the image and video data.

CUDA Features

These CUDA features are needed by some CUDA samples. They are provided by either the CUDA Toolkit or CUDA Driver. Some features may not be available on your system.

CUFFT Callback Routines

CUFFT Callback Routines are user-supplied kernel routines that CUFFT will call when loading or storing data. These callback routines are only available on Linux x86_64 and ppc64le systems.

CUDA Dynamic Parallellism

CDP (CUDA Dynamic Parallellism) allows kernels to be launched from threads running on the GPU. CDP is only available on GPUs with SM architecture of 3.5 or above.

Multi-block Cooperative Groups

Multi Block Cooperative Groups(MBCG) extends Cooperative Groups and the CUDA programming model to express inter-thread-block synchronization. MBCG is available on GPUs with Pascal and higher architecture.

Multi-Device Cooperative Groups

Multi Device Cooperative Groups extends Cooperative Groups and the CUDA programming model enabling thread blocks executing on multiple GPUs to cooperate and synchronize as they execute. This feature is available on GPUs with Pascal and higher architecture.

CUBLAS

CUBLAS (CUDA Basic Linear Algebra Subroutines) is a GPU-accelerated version of the BLAS library.

CUDA Interprocess Communication

IPC (Interprocess Communication) allows processes to share device pointers.

CUFFT

CUFFT (CUDA Fast Fourier Transform) is a GPU-accelerated FFT library.

CURAND

CURAND (CUDA Random Number Generation) is a GPU-accelerated RNG library.

CUSPARSE

CUSPARSE (CUDA Sparse Matrix) provides linear algebra subroutines used for sparse matrix calculations.

CUSOLVER

CUSOLVER library is a high-level package based on the CUBLAS and CUSPARSE libraries. It combines three separate libraries under a single umbrella, each of which can be used independently or in concert with other toolkit libraries. The intent ofCUSOLVER is to provide useful LAPACK-like features, such as common matrix factorization and triangular solve routines for dense matrices, a sparse least-squares solver and an eigenvalue solver. In addition cuSolver provides a new refactorization library useful for solving sequences of matrices with a shared sparsity pattern.

NPP

NPP (NVIDIA Performance Primitives) provides GPU-accelerated image, video, and signal processing functions.

NVGRAPH

NVGRAPH is a GPU-accelerated graph analytics library.

NVJPEG

NVJPEG library provides high-performance, GPU accelerated JPEG decoding functionality for image formats commonly used in deep learning and hyperscale multimedia applications.

NVRTC

NVRTC (CUDA RunTime Compilation) is a runtime compilation library for CUDA C++.

Stream Priorities

Stream Priorities allows the creation of streams with specified priorities. Stream Priorities is only available on GPUs with SM architecture of 3.5 or above.

Unified Virtual Memory

UVM (Unified Virtual Memory) enables memory that can be accessed by both the CPU and GPU without explicit copying between the two. UVM is only available on Linux and Windows systems.

16-bit Floating Point

FP16 is a 16-bit floating-point format. One bit is used for the sign, five bits for the exponent, and ten bits for the mantissa.

C++11 CUDA

NVCC support of C++11 features.

CMake

The libNVVM samples are built using CMake 3.10 or later.

Contributors Guide

We welcome your input on issues and suggestions for samples. At this time we are not accepting contributions from the public, check back here as we evolve our contribution model.

We use Google C++ Style Guide for all the sources https://google.github.io/styleguide/cppguide.html

Frequently Asked Questions

Answers to frequently asked questions about CUDA can be found at http://developer.nvidia.com/cuda-faq and in the CUDA Toolkit Release Notes.

References

Attributions

More Repositories

1

nvidia-docker

Build and run Docker containers leveraging NVIDIA GPUs
16,896
star
2

open-gpu-kernel-modules

NVIDIA Linux open GPU kernel module source
C
13,784
star
3

DeepLearningExamples

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.
Jupyter Notebook
12,579
star
4

FastPhotoStyle

Style transfer, deep learning, feature transform
Python
11,020
star
5

NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
Python
10,077
star
6

TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
C++
9,059
star
7

vid2vid

Pytorch implementation of our method for high-resolution (e.g. 2048x1024) photorealistic video-to-video translation.
Python
8,482
star
8

Megatron-LM

Ongoing research training transformer models at scale
Python
8,169
star
9

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
Python
7,915
star
10

pix2pixHD

Synthesizing and manipulating 2048x1024 images with conditional GANs
Python
6,488
star
11

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
C++
6,429
star
12

FasterTransformer

Transformer related optimization, including BERT, GPT
C++
5,313
star
13

thrust

[ARCHIVED] The C++ parallel algorithms library. See https://github.com/NVIDIA/cccl
C++
4,845
star
14

DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
C++
4,839
star
15

tacotron2

Tacotron 2 - PyTorch implementation with faster-than-realtime inference
Jupyter Notebook
4,562
star
16

cutlass

CUDA Templates for Linear Algebra Subroutines
C++
4,278
star
17

DIGITS

Deep Learning GPU Training System
HTML
4,105
star
18

NeMo-Guardrails

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
Python
3,338
star
19

flownet2-pytorch

Pytorch implementation of FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Python
2,938
star
20

nccl

Optimized primitives for collective multi-GPU communication
C++
2,786
star
21

libcudacxx

[ARCHIVED] The C++ Standard Library for your entire system. See https://github.com/NVIDIA/cccl
C++
2,286
star
22

k8s-device-plugin

NVIDIA device plugin for Kubernetes
Go
2,269
star
23

waveglow

A Flow-based Generative Network for Speech Synthesis
Python
2,133
star
24

trt-llm-rag-windows

A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM
Python
2,011
star
25

MinkowskiEngine

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors
Python
2,007
star
26

semantic-segmentation

Nvidia Semantic Segmentation monorepo
Python
1,746
star
27

DeepRecommender

Deep learning for recommender systems
Python
1,662
star
28

Stable-Diffusion-WebUI-TensorRT

TensorRT Extension for Stable Diffusion Web UI
Python
1,660
star
29

cub

[ARCHIVED] Cooperative primitives for CUDA C++. See https://github.com/NVIDIA/cccl
Cuda
1,648
star
30

warp

A Python framework for high performance GPU simulation and graphics
Python
1,573
star
31

OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
Python
1,511
star
32

GenerativeAIExamples

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Python
1,450
star
33

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.
Python
1,400
star
34

VideoProcessingFramework

Set of Python bindings to C++ libraries which provides full HW acceleration for video decoding, encoding and GPU-accelerated color space and pixel format conversions
C++
1,253
star
35

nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs
Go
1,239
star
36

trt-samples-for-hackathon-cn

Simple samples for TensorRT programming
Python
1,211
star
37

Q2RTX

NVIDIA’s implementation of RTX ray-tracing in Quake II
C
1,201
star
38

open-gpu-doc

Documentation of NVIDIA chip/hardware interfaces
C
1,193
star
39

stdexec

`std::execution`, the proposed C++ framework for asynchronous and parallel programming.
C++
1,182
star
40

deepops

Tools for building GPU clusters
Shell
1,165
star
41

partialconv

A New Padding Scheme: Partial Convolution based Padding
Python
1,145
star
42

CUDALibrarySamples

CUDA Library Samples
Cuda
1,122
star
43

gpu-operator

NVIDIA GPU Operator creates/configures/manages GPUs atop Kubernetes
Go
1,117
star
44

MatX

An efficient C++17 GPU numerical computing library with Python-like syntax
C++
1,104
star
45

aistore

AIStore: scalable storage for AI applications
Go
1,074
star
46

sentiment-discovery

Unsupervised Language Modeling at scale for robust sentiment classification
Python
1,055
star
47

nvidia-container-runtime

NVIDIA container runtime
Makefile
1,035
star
48

gpu-monitoring-tools

Tools for monitoring NVIDIA GPUs on Linux
C
974
star
49

retinanet-examples

Fast and accurate object detection with end-to-end GPU optimization
Python
876
star
50

flowtron

Flowtron is an auto-regressive flow-based generative network for text to speech synthesis with control over speech variation and style transfer
Jupyter Notebook
867
star
51

mellotron

Mellotron: a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data
Jupyter Notebook
842
star
52

jetson-gpio

A Python library that enables the use of Jetson's GPIOs
Python
834
star
53

gdrcopy

A fast GPU memory copy library based on NVIDIA GPUDirect RDMA technology
C++
766
star
54

nv-wavenet

Reference implementation of real-time autoregressive wavenet inference
Cuda
728
star
55

libnvidia-container

NVIDIA container runtime library
C
722
star
56

tensorflow

An Open Source Machine Learning Framework for Everyone
C++
719
star
57

spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
Scala
717
star
58

cuda-python

CUDA Python Low-level Bindings
Python
695
star
59

cccl

CUDA C++ Core Libraries
C++
676
star
60

MAXINE-AR-SDK

NVIDIA AR SDK - API headers and sample applications
C
671
star
61

nvvl

A library that uses hardware acceleration to load sequences of video frames to facilitate machine learning training
C++
665
star
62

gvdb-voxels

Sparse volume compute and rendering on NVIDIA GPUs
C
656
star
63

nccl-tests

NCCL Tests
Cuda
648
star
64

modulus

Open-source deep-learning framework for building, training, and fine-tuning deep learning models using state-of-the-art Physics-ML methods
Python
636
star
65

BigVGAN

Official PyTorch implementation of BigVGAN (ICLR 2023)
Python
633
star
66

runx

Deep Learning Experiment Management
Python
630
star
67

DLSS

NVIDIA DLSS is a new and improved deep learning neural network that boosts frame rates and generates beautiful, sharp images for your games
C
588
star
68

dcgm-exporter

NVIDIA GPU metrics exporter for Prometheus leveraging DCGM
Go
551
star
69

Dataset_Synthesizer

NVIDIA Deep learning Dataset Synthesizer (NDDS)
C++
530
star
70

NVFlare

NVIDIA Federated Learning Application Runtime Environment
Python
528
star
71

nvcomp

Repository for nvCOMP docs and examples. nvCOMP is a library for fast lossless compression/decompression on the GPU that can be downloaded from https://developer.nvidia.com/nvcomp.
C++
510
star
72

jitify

A single-header C++ library for simplifying the use of CUDA Runtime Compilation (NVRTC).
C++
495
star
73

libglvnd

The GL Vendor-Neutral Dispatch library
C
462
star
74

enroot

A simple yet powerful tool to turn traditional container/OS images into unprivileged sandboxes.
Shell
459
star
75

multi-gpu-programming-models

Examples demonstrating available options to program multiple GPUs in a single node or a cluster
Cuda
438
star
76

MDL-SDK

NVIDIA Material Definition Language SDK
C++
438
star
77

PyProf

A GPU performance profiling tool for PyTorch models
Python
437
star
78

AMGX

Distributed multigrid linear solver library on GPU
Cuda
434
star
79

gpu-rest-engine

A REST API for Caffe using Docker and Go
C++
421
star
80

nvbench

CUDA Kernel Benchmarking Library
Cuda
413
star
81

framework-reproducibility

Providing reproducibility in deep learning frameworks
Python
412
star
82

cuCollections

C++
410
star
83

hpc-container-maker

HPC Container Maker
Python
404
star
84

NeMo-Framework-Launcher

NeMo Megatron launcher and tools
Python
391
star
85

NvPipe

NVIDIA-accelerated zero latency video compression library for interactive remoting applications
Cuda
384
star
86

cuda-quantum

C++ and Python support for the CUDA Quantum programming model for heterogeneous quantum-classical workflows
C++
363
star
87

data-science-stack

NVIDIA Data Science stack tools
Shell
317
star
88

cuQuantum

Home for cuQuantum Python & NVIDIA cuQuantum SDK C++ samples
Jupyter Notebook
305
star
89

ai-assisted-annotation-client

Client side integration example source code and libraries for AI-Assisted Annotation SDK
C++
302
star
90

video-sdk-samples

Samples demonstrating how to use various APIs of NVIDIA Video Codec SDK
C++
301
star
91

nvidia-settings

NVIDIA driver control panel
C
284
star
92

DCGM

NVIDIA Data Center GPU Manager (DCGM) is a project for gathering telemetry and measuring the health of NVIDIA GPUs
C++
282
star
93

cnmem

A simple memory manager for CUDA designed to help Deep Learning frameworks manage memory
C++
280
star
94

radtts

Provides training, inference and voice conversion recipes for RADTTS and RADTTS++: Flow-based TTS models with Robust Alignment Learning, Diverse Synthesis, and Generative Modeling and Fine-Grained Control over of Low Dimensional (F0 and Energy) Speech Attributes.
Roff
269
star
95

fsi-samples

A collection of open-source GPU accelerated Python tools and examples for quantitative analyst tasks and leverages RAPIDS AI project, Numba, cuDF, and Dask.
Jupyter Notebook
265
star
96

tensorrt-laboratory

Explore the Capabilities of the TensorRT Platform
C++
259
star
97

CleanUNet

Official PyTorch Implementation of CleanUNet (ICASSP 2022)
Python
258
star
98

gpu-feature-discovery

GPU plugin to the node feature discovery for Kubernetes
Go
255
star
99

torch-harmonics

Differentiable spherical harmonic transforms and spherical convolutions in PyTorch
Jupyter Notebook
246
star
100

egl-wayland

The EGLStream-based Wayland external platform
C
243
star