Anakin2.0

Welcome to the Anakin GitHub.

Anakin is a cross-platform, high-performance inference engine, which is originally developed by Baidu engineers and is a large-scale application of industrial products.

Please refer to our release announcement to track the latest feature of Anakin.

Features

• Flexibility

  Anakin is a cross-platform, high-performance inference engine, supports a wide range of neural network architectures and different hardware platforms. It is easy to run Anakin on GPU / x86 / ARM platform.

  Anakin has integrated with NVIDIA TensorRT and open source this part of integrated API to provide services, developers can call the API directly or modify it as needed, which will be more flexible for development requirements.

• High performance

  In order to give full play to the performance of hardware, we optimized the forward prediction at different levels.

  • Automatic graph fusion. The goal of all performance optimizations under a given algorithm is to make the ALU as busy as possible. Operator fusion can effectively reduce memory access and keep the ALU busy.

  • Memory reuse. Forward prediction is a one-way calculation. We reuse the memory between the input and output of different operators, thus reducing the overall memory overhead.

  • Assembly level optimization. Saber is a underlying DNN library for Anakin, which is deeply optimized at assembly level.

NV GPU Benchmark

Machine And Enviornment

CPU: Intel(R) Xeon(R) CPU 5117 @ 2.0GHz
GPU: Tesla P4
cuda: CUDA8
cuDNN: v7

• Time：warmup 10，running 1000 times to get average time
• Latency (ms) and Memory(MB) of different batch

The counterpart of Anakin is the acknowledged high performance inference engine NVIDIA TensorRT 5 , The models which TensorRT 5 doesn't support we use the custom plugins to support.

VGG16

Resnet50

Resnet101

X86 CPU Benchmark

Machine And Enviornment

CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz with HT, for FP32 test
CPU: Intel(R) Xeon(R) Gold 6271 CPU @ 2.60GHz with HT, for INT8 test
System: CentOS 6.3 with GCC 4.8.2, for benchmark between Anakin and Intel Caffe

• All test enable 8 thread parallel
• Time：warmup 10，running 200 times to get average time

The counterpart of Anakin is Intel Cafe(1.1.6) with mklml.

ARM CPU Benchmark

Machine And Enviornment

CPU: Kirin 980
CPU: Snapdragon 652
CPU: Snapdragon 855
CPU: RK3399

• Compile circumstance: Android ndk cross compile，gcc 4.9，enable neon
• Time：warmup 10，running 10 times to get average time
• Note: 1、shufflenetv2 int8 model add swish operator

The counterpart of Anakin is ncnn(20190320). This benchmark we test ARMv7 ARMv8 splitly

ARMv8 TEST

• ABI： arm64-v8a

• Latency (ms) of one batch

ARMv7 TEST

• ABI： armveabi-v7a with neon

• Latency (ms) of one batch

Documentation

All you need is in Doc Index

We also provide English and Chinese tutorial documentation.

• User guide

  You can get the working principle of the project, C++ interface description and code examples from here. You can also learn about the model converter here.

• Developer guide

  You might want to know more details of Anakin and make it better. Please refer to how to add custom devices and how to add custom device operators.

• How to Contribute

  We appreciate your contributions!

Ask Questions

You are welcome to submit questions and bug reports as Github Issues.

Copyright and License

Anakin is provided under the Apache-2.0 license.

Acknowledgement

Anakin refers to the following projects:

• Caffe
• ComputeLibrary
• gemmlowp
• ncnn