Discover ROCm/MIVisionX Open Source project

MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX delivers highly optimized conformant open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions along with Convolution Neural Net Model Compiler & Optimizer supporting ONNX, and Khronos NNEF™ exchange formats. The toolkit allows for rapid prototyping and deployment of optimized computer vision and machine learning inference workloads on a wide range of computer hardware, including small embedded x86 CPUs, APUs, discrete GPUs, and heterogeneous servers.

Latest Release

Table of Contents
Documentation
AMD OpenVX™
AMD OpenVX™ Extensions
Applications
Neural Net Model Compiler & Optimizer
rocAL
Toolkit
Utilities
Prerequisites
Build & Install MIVisionX
Verify the Installation
- Verifying on Linux / macOS
- Verifying on Windows
Docker
Technical Support
Release Notes
MIVisionX Dependency Map
- HIP Backend

Documentation

Run the steps below to build documentation locally.

sphinx documentation

cd docs
pip3 install -r .sphinx/requirements.txt
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html

Doxygen

doxygen .Doxyfile

AMD OpenVX™

AMD OpenVX™ is a highly optimized conformant open source implementation of the Khronos OpenVX™ 1.3 computer vision specification. It allows for rapid prototyping as well as fast execution on a wide range of computer hardware, including small embedded x86 CPUs and large workstation discrete GPUs.

Khronos OpenVX™ 1.0.1 conformant implementation is available in MIVisionX Lite

AMD OpenVX™ Extensions

The OpenVX framework provides a mechanism to add new vision functionality to OpenVX by vendors. This project has below mentioned OpenVX modules and utilities to extend amd_openvx, which contains the AMD OpenVX™ Core Engine.

amd_loomsl: AMD Radeon Loom stitching library for live 360 degree video applications
amd_media: vx_amd_media is an OpenVX AMD media extension module for encode and decode
amd_migraphx: amd_migraphx extension integrates the AMD's MIGraphx into an OpenVX graph. This extension allows developers to combine the vision funcions in OpenVX with the MIGraphX and build an end-to-end application for inference.
amd_nn: OpenVX neural network module
amd_opencv: OpenVX module that implements a mechanism to access OpenCV functionality as OpenVX kernels
amd_rpp: OpenVX extension providing an interface to some of the RPP's (ROCm Performance Primitives) functions. This extension is used to enable rocAL to perform image augmentation.
amd_winml: WinML extension will allow developers to import a pre-trained ONNX model into an OpenVX graph and add hundreds of different pre & post processing vision / generic / user-defined functions, available in OpenVX and OpenCV interop, to the input and output of the neural net model. This will allow developers to build an end to end application for inference.

Applications

MIVisionX has several applications built on top of OpenVX modules, it uses AMD optimized libraries to build applications that can be used to prototype or use as a model to develop products.

Bubble Pop: This sample application creates bubbles and donuts to pop using OpenVX & OpenCV functionality.
Cloud Inference Application: This sample application does inference using a client-server system.
Digit Test: This sample application is used to recognize hand written digits.
Image Augmentation: This sample application demonstrates the basic usage of rocAL's C API to load JPEG images from the disk and modify them in different possible ways and displays the output images.
MIVisionX Inference Analyzer: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze and summarize images.
MIVisionX OpenVX Classification: This sample application shows how to run supported pre-trained caffe models with MIVisionX RunTime.
MIVisionX Validation Tool: This sample application uses pre-trained ONNX / NNEF / Caffe models to analyze, summarize and validate models.
MIVisionX WinML Classification: This sample application shows how to run supported ONNX models with MIVisionX RunTime on Windows.
MIVisionX WinML YoloV2: This sample application shows how to run tiny yolov2(20 classes) with MIVisionX RunTime on Windows.
Optical Flow: This sample application creates an OpenVX graph to run Optical Flow on a video/live.
External Applications

Neural Net Model Compiler & Optimizer

Neural Net Model Compiler & Optimizer converts pre-trained neural net models to MIVisionX runtime code for optimized inference.

rocAL

The ROCm Augmentation Library - rocAL is designed to efficiently decode and process images and videos from a variety of storage formats and modify them through a processing graph programmable by the user.

Toolkit

MIVisionX Toolkit, is a comprehensive set of helpful tools for neural net creation, development, training, and deployment. The Toolkit provides you with helpful tools to design, develop, quantize, prune, retrain, and infer your neural network work in any framework. The Toolkit is designed to help you deploy your work to any AMD or 3rd party hardware, from embedded to servers.

MIVisionX provides you with tools for accomplishing your tasks throughout the whole neural net life-cycle, from creating a model to deploying them for your target platforms.

Utilities

inference_generator: generate inference library from pre-trained CAFFE models
loom_shell: an interpreter to prototype 360 degree video stitching applications using a script
RunVX: command-line utility to execute OpenVX graph described in GDF text file
RunCL: command-line utility to build, execute, and debug OpenCL programs

Prerequisites

Hardware

CPU: AMD64
GPU: AMD Radeon™ Graphics [optional]
APU: AMD Radeon™ Mobile/Embedded [optional]

Note: Some modules in MIVisionX can be built for CPU ONLY. To take advantage of Advanced Features And Modules we recommend using AMD GPUs or AMD APUs.

Operating System

Windows

Windows 10 / 11
Windows SDK
Visual Studio 2019 or later
Install the latest AMD drivers
Install OpenCL SDK
Install OpenCV 4.6.0
- Set OpenCV_DIR environment variable to OpenCV/build folder
- Add %OpenCV_DIR%\x64\vc14\bin or %OpenCV_DIR%\x64\vc15\bin to your PATH

macOS

macOS - Ventura 13.4
Install Homebrew
Install CMake
Install OpenCV 3/4

Note: macOS build instructions

Linux

Linux distribution
- Ubuntu - 20.04 / 22.04
- CentOS - 7 / 8
- RedHat - 8 / 9
- SLES - 15-SP4
Install ROCm
CMake 3.5 or later
MIOpen for vx_nn extension
MIGraphX for vx_migraphx extension
Protobuf
OpenCV 4.6.0
FFMPEG n4.4.2
rocAL Prerequisites

Prerequisites setup script for Linux

For the convenience of the developer, we provide the setup script MIVisionX-setup.py which will install all the dependencies required by this project.

NOTE: This script only needs to be executed once.

Prerequisites for running the script

Linux distribution
- Ubuntu - 20.04 / 22.04
- CentOS - 7 / 8
- RedHat - 8 / 9
- SLES - 15-SP4
ROCm supported hardware

ROCm

usage:

python MIVisionX-setup.py --directory [setup directory - optional (default:~/)]
                          --opencv    [OpenCV Version - optional (default:4.6.0)]
                          --protobuf  [ProtoBuf Version - optional (default:3.12.4)]
                          --rpp       [RPP Version - optional (default:1.0.0)]
                          --pybind11  [PyBind11 Version - optional (default:v2.10.4)]
                          --ffmpeg    [FFMPEG V4.4.2 Installation - optional (default:ON) [options:ON/OFF]]
                          --rocal     [MIVisionX rocAL Dependency Install - optional (default:ON) [options:ON/OFF]]
                          --neural_net[MIVisionX Neural Net Dependency Install - optional (default:ON) [options:ON/OFF]]
                          --inference [MIVisionX Neural Net Inference Dependency Install - optional (default:ON) [options:ON/OFF]]
                          --developer [Setup Developer Options - optional (default:ON) [options:ON/OFF]]
                          --reinstall [Remove previous setup and reinstall (default:OFF)[options:ON/OFF]]
                          --backend   [MIVisionX Dependency Backend - optional (default:HIP) [options:HIP/OCL/CPU]]
                          --rocm_path [ROCm Installation Path - optional (default:/opt/rocm) - ROCm Installation Required]

Note:

ROCm upgrade with sudo apt upgrade requires the setup script rerun.
use X Window / X11 for remote GUI app control

Build & Install MIVisionX

Building on Windows

Using `Visual Studio`

Install Windows Prerequisites
Use MIVisionX.sln to build for x64 platform

NOTE: vx_nn is not supported on Windows in this release

Building on macOS

macOS build instructions

Building on Linux

Using `apt-get` / `yum` / `zypper`

ROCm supported hardware
Install ROCm
On Ubuntu
```
sudo apt-get install mivisionx
```
On CentOS/RedHat
```
sudo yum install mivisionx
```
On SLES
```
sudo zypper install mivisionxF
```
Note:
- vx_winml is not supported on Linux
- source code will not available with apt-get / yum / zypper install
- the installer will copy
  - Executables into /opt/rocm/bin
  - Libraries into /opt/rocm/lib
  - OpenVX and module header files into /opt/rocm/include/mivisionx
  - Model compiler, & toolkit folders into /opt/rocm/libexec/mivisionx
  - Apps, & samples folder into /opt/rocm/share/mivisionx
  - Docs folder into /opt/rocm/share/doc/mivisionx
- Package (.deb & .rpm) install requires OpenCV v4.6 to execute AMD OpenCV extensions

Using MIVisionX-setup.py

Install ROCm
Use the below commands to set up and build MIVisionX
```
git clone https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX.git
cd MIVisionX
```
Note: MIVisionX has support for two GPU backends: OPENCL and HIP:
- Instructions for building MIVisionX with the HIP GPU backend (i.e., default GPU backend):
  - run the setup script to install all the dependencies required by the HIP GPU backend:
```
python MIVisionX-setup.py
```
  - run the below commands to build MIVisionX with the HIP GPU backend:
```
mkdir build-hip
cd build-hip
cmake ../
make -j8
sudo cmake --build . --target PyPackageInstall
sudo make install
```
  Note:
  - PyPackageInstall used for rocal_pybind installation
  - rocal_pybind not supported on windows.
  - sudo required for pybind installation
- Instructions for building MIVisionX with OPENCL GPU backend

Verify the Installation

Verifying on Linux / macOS

The installer will copy
- Executables into /opt/rocm/bin
- Libraries into /opt/rocm/lib
- OpenVX and OpenVX module header files into /opt/rocm/include/mivisionx
- Apps, & Samples folder into /opt/rocm/share/mivisionx
- Documents folder into /opt/rocm/share/doc/mivisionx
- Model Compiler, and Toolkit folder into /opt/rocm/libexec/mivisionx
Run the below sample to verify the installation

Canny Edge Detection
```
export PATH=$PATH:/opt/rocm/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib
runvx /opt/rocm/share/mivisionx/samples/gdf/canny.gdf
```
Note: More samples are available here

Note: For macOS use export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/opt/rocm/lib

Verifying on Windows

MIVisionX.sln builds the libraries & executables in the folder MIVisionX/x64

Use RunVX to test the build

./runvx.exe PATH_TO/MIVisionX/samples/gdf/skintonedetect.gdf

Docker

MIVisionX provides developers with docker images for Ubuntu 20.04 / 22.04. Using docker images developers can quickly prototype and build applications without having to be locked into a single system setup or lose valuable time figuring out the dependencies of the underlying software.

Docker files to build MIVisionX containers are available

MIVisionX Docker

Docker Workflow on Ubuntu `20.04`/`22.04`

Prerequisites

Ubuntu 20.04/22.04
ROCm supported hardware
ROCm
Docker

Workflow

Step 1 - Get latest docker image
```
sudo docker pull mivisionx/ubuntu-20.04:latest
```
- NOTE: Use the above command to bring in latest changes from upstream
Step 2 - Run docker image

Run docker image: Local Machine

sudo docker run -it --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mem --cap-add=SYS_RAWIO  --group-add video --shm-size=4g --ipc="host" --network=host mivisionx/ubuntu-20.04:latest

Test - Computer Vision Workflow

python3 /workspace/MIVisionX/tests/vision_tests/runVisionTests.py --num_frames 1

Test - Neural Network Workflow

python3 /workspace/MIVisionX/tests/neural_network_tests/runNeuralNetworkTests.py --profiler_level 1

Test - Khronos OpenVX 1.3.0 Conformance Test

python3 /workspace/MIVisionX/tests/conformance_tests/runConformanceTests.py --backend_type HOST

Option 1: Map localhost directory on the docker image

option to map the localhost directory with data to be accessed on the docker image

usage: -v {LOCAL_HOST_DIRECTORY_PATH}:{DOCKER_DIRECTORY_PATH}

sudo docker run -it -v /home/:/root/hostDrive/ -privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mem --cap-add=SYS_RAWIO  --group-add video --shm-size=4g --ipc="host" --network=host mivisionx/ubuntu-20.04:latest

Option 2: Display with docker

Using host display for docker

xhost +local:root
sudo docker run -it --privileged --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host --env DISPLAY=$DISPLAY --volume="$HOME/.Xauthority:/root/.Xauthority:rw" --volume /tmp/.X11-unix/:/tmp/.X11-unix mivisionx/ubuntu-20.04:latest

Test display with MIVisionX sample

runvx -v /opt/rocm/share/mivisionx/samples/gdf/canny.gdf

Run docker image with display: Remote Server Machine

sudo docker run -it --privileged --device=/dev/kfd --device=/dev/dri --cap-add=SYS_RAWIO --device=/dev/mem --group-add video --network host --env DISPLAY=$DISPLAY --volume="$HOME/.Xauthority:/root/.Xauthority:rw" --volume /tmp/.X11-unix/:/tmp/.X11-unix mivisionx/ubuntu-20.04:latest

Test display with MIVisionX sample

runvx -v /opt/rocm/share/mivisionx/samples/gdf/canny.gdf

Technical Support

Please email [email protected] for questions, and feedback on MIVisionX.

Please submit your feature requests, and bug reports on the GitHub issues page.

Release Notes

Latest Release Version

Changelog

Review all notable changes with the latest release

Tested configurations

Windows 10 / 11
Linux distribution
- Ubuntu - 20.04 / 22.04
- CentOS - 7 / 8
- RHEL - 8 / 9
- SLES - 15-SP4
ROCm: rocm-core - 5.4.3.50403-121
miopen-hip - 2.19.0.50403-121
miopen-opencl - 2.18.0.50300-63
migraphx - 2.4.0.50403-121
Protobuf - V3.12.4
OpenCV - 4.6.0
RPP - 1.2.0
FFMPEG - n4.4.2
Dependencies for all the above packages
MIVisionX Setup Script - V2.5.5

Known issues

Package install requires OpenCV V-4.6.0 to execute AMD OpenCV extensions

MIVisionX Dependency Map

HIP Backend

Docker Image: sudo docker build -f docker/ubuntu20/{DOCKER_LEVEL_FILE_NAME}.dockerfile -t {mivisionx-level-NUMBER} .

new component added to the level
existing component from the previous level

Build Level	MIVisionX Dependencies	Modules	Libraries and Executables
`Level_1`	cmake gcc g++	amd_openvx utilities	`libopenvx.so` - OpenVX™ Lib - CPU `libvxu.so` - OpenVX™ immediate node Lib - CPU `runvx` - OpenVX™ Graph Executor - CPU with Display OFF
`Level_2`	ROCm HIP +Level 1	amd_openvx amd_openvx_extensions utilities	`libopenvx.so` - OpenVX™ Lib - CPU/GPU `libvxu.so` - OpenVX™ immediate node Lib - CPU/GPU `runvx` - OpenVX™ Graph Executor - Display OFF
`Level_3`	OpenCV FFMPEG +Level 2	amd_openvx amd_openvx_extensions utilities	`libopenvx.so` - OpenVX™ Lib `libvxu.so` - OpenVX™ immediate node Lib `libvx_amd_media.so` - OpenVX™ Media Extension `libvx_opencv.so` - OpenVX™ OpenCV InterOp Extension `mv_compile` - Neural Net Model Compile `runvx` - OpenVX™ Graph Executor - Display ON
`Level_4`	MIOpenGEMM MIOpen ProtoBuf +Level 3	amd_openvx amd_openvx_extensions apps utilities	`libopenvx.so` - OpenVX™ Lib `libvxu.so` - OpenVX™ immediate node Lib `libvx_amd_media.so` - OpenVX™ Media Extension `libvx_opencv.so` - OpenVX™ OpenCV InterOp Extension `mv_compile` - Neural Net Model Compile `runvx` - OpenVX™ Graph Executor - Display ON `libvx_nn.so` - OpenVX™ Neural Net Extension
`Level_5`	AMD_RPP rocAL deps +Level 4	amd_openvx amd_openvx_extensions apps rocAL utilities	`libopenvx.so` - OpenVX™ Lib `libvxu.so` - OpenVX™ immediate node Lib `libvx_amd_media.so` - OpenVX™ Media Extension `libvx_opencv.so` - OpenVX™ OpenCV InterOp Extension `mv_compile` - Neural Net Model Compile `runvx` - OpenVX™ Graph Executor - Display ON `libvx_nn.so` - OpenVX™ Neural Net Extension `libvx_rpp.so` - OpenVX™ RPP Extension `librocal.so` - Radeon Augmentation Library `rocal_pybind.so` - rocAL Pybind Lib

NOTE: OpenVX and the OpenVX logo are trademarks of the Khronos Group Inc.

ROCm/MIVisionX

ROCm

Reviews

Repository Details

Latest Release

Table of Contents

Documentation

AMD OpenVX™

AMD OpenVX™ Extensions

Applications

Neural Net Model Compiler & Optimizer

rocAL

Toolkit

Utilities

Prerequisites

Hardware

Operating System

Windows

macOS

Linux

Prerequisites setup script for Linux

Prerequisites for running the script

Build & Install MIVisionX

Building on Windows

Using Visual Studio

Building on macOS

Building on Linux

Using apt-get / yum / zypper

Using MIVisionX-setup.py

Verify the Installation

Verifying on Linux / macOS

Verifying on Windows

Docker

MIVisionX Docker

Docker Workflow on Ubuntu 20.04/22.04

Prerequisites

Workflow

Run docker image: Local Machine

Option 1: Map localhost directory on the docker image

Option 2: Display with docker

Run docker image with display: Remote Server Machine

Technical Support

Release Notes

Latest Release Version

Changelog

Tested configurations

Known issues

MIVisionX Dependency Map

HIP Backend

More Repositories

Using `Visual Studio`

Using `apt-get` / `yum` / `zypper`

Docker Workflow on Ubuntu `20.04`/`22.04`