• Stars
    star
    332
  • Rank 122,089 (Top 3 %)
  • Language
    C++
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated 10 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Amazon Kinesis Video Streams Producer SDK for C++ is for developers to install and customize for their connected camera and other devices to securely stream video, audio, and time-encoded data to Kinesis Video Streams.

Amazon Kinesis Video Streams CPP Producer, GStreamer Plugin and JNI

Amazon Kinesis Video Streams | Secure Video Ingestion for Analysis & Storage

Coverage Status

Key Features • Build • Run • Documentation • Related • License

Key Features

  • C++ SDK
  • GStreamer Plugin (kvssink)
  • JNI

Amazon Kinesis Video Streams Producer SDK for C/C++ makes it easy to build an on-device application that securely connects to a video stream, and reliably publishes video and other media data to Kinesis Video Streams. It takes care of all the underlying tasks required to package the frames and fragments generated by the device's media pipeline. The SDK also handles stream creation, token rotation for secure and uninterrupted streaming, processing acknowledgements returned by Kinesis Video Streams, and other tasks.

Build

Download

To download run the following command:

git clone https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp.git

Note: You will also need to install pkg-config, CMake, m4 and a build enviroment. If you are building the GStreamer plugin you will also need GStreamer and GStreamer (Development Libraries).

Refer to the FAQ for platform specific instructions.

Configure

Prepare a build directory in the newly checked out repository:

mkdir -p amazon-kinesis-video-streams-producer-sdk-cpp/build
cd amazon-kinesis-video-streams-producer-sdk-cpp/build

If you are building on Windows you need to generate NMake Makefiles, you should run cmake .. -G "NMake Makefiles"

GStreamer and JNI is NOT built by default, if you wish to build both you MUST execute cmake .. -DBUILD_GSTREAMER_PLUGIN=ON -DBUILD_JNI=TRUE

By default we download all the libraries from GitHub and build them locally, so should require nothing to be installed ahead of time.

If you do wish to link to existing libraries you can do cmake .. -DBUILD_DEPENDENCIES=OFF Libraries needed to build producer are: Curl, Openssl and Log4cplus. If you want to build the gstreamer plugin you will need to have gstreamer in your system. On Mac OS you can get the libraries using homebrew

$ brew install pkg-config openssl cmake gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad gst-plugins-ugly log4cplus gst-libav

On Ubuntu and Raspberry Pi OS you can get the libraries by running

$ sudo apt-get install libssl-dev libcurl4-openssl-dev liblog4cplus-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base-apps gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools

Setup desired log level:

Set up the desired log level. The log levels currently available with log4cplus are:

  1. TRACE
  2. DEBUG
  3. INFO
  4. WARN
  5. ERROR
  6. FATAL

To set a log level, update the log level value here

Note: The default log level is DEBUG

Cross-Compilation

If you wish to cross-compile CC and CXX are respected when building the library and all its dependencies. See our ci.yml for an example of this. Every commit is cross compiled to ensure that it continues to work. Please note that GStreamer is not cross-compiled as a part of the cross-compilation of the KVS-SDK, customers will have to cross-compile it separately.

CMake Arguments

You can pass the following options to cmake ...

  • -DBUILD_GSTREAMER_PLUGIN -- Build kvssink GStreamer plugin
  • -DBUILD_JNI -- Build C++ wrapper for JNI to expose the functionality to Java/Android
  • -DBUILD_DEPENDENCIES -- Build depending libraries from source
  • -DBUILD_TEST=TRUE -- Build unit/integration tests, may be useful for confirm support for your device. ./tst/producerTest
  • -DCODE_COVERAGE -- Enable coverage reporting
  • -DCOMPILER_WARNINGS -- Enable all compiler warnings
  • -DADDRESS_SANITIZER -- Build with AddressSanitizer
  • -DMEMORY_SANITIZER -- Build with MemorySanitizer
  • -DTHREAD_SANITIZER -- Build with ThreadSanitizer
  • -DUNDEFINED_BEHAVIOR_SANITIZER Build with UndefinedBehaviorSanitizer
  • -DALIGNED_MEMORY_MODEL Build for aligned memory model only devices. Default is OFF.
  • -DBUILD_LOG4CPLUS_HOST Specify host-name for log4cplus for cross-compilation. Default is OFF.

To Include JNI

JNI examples are NOT built by default. If you wish to build JNI you MUST add -DBUILD_JNI=TRUE when running cmake:

cmake -DBUILD_JNI=TRUE

To Include Building GStreamer Sample Programs

The GStreamer plugin and samples are NOT built by default. If you wish to build them you MUST add -DBUILD_GSTREAMER_PLUGIN=TRUE when running cmake:

cmake -DBUILD_GSTREAMER_PLUGIN=TRUE ..

Compiling

After running cmake, in the same build directory run make:

make

On Windows you should run nmake instead of make

In your build directory you will now have shared objects for all the targets you have selected.

Installing the library

If the library needs to be installed, run make install. This will install in default directory based on system. To install in another directory, run cmake with the -DCMAKE_INSTALL_PREFIX option with the desired directory before running make install

Run

GStreamer Plugin (kvssink)

Loading Element

The GStreamer plugin is located in your build directory.

To load this plugin set the following environment variables. This should be run from the root of the repo, NOT the build directory.

export GST_PLUGIN_PATH=`pwd`/build
export LD_LIBRARY_PATH=`pwd`/open-source/local/lib

The equivalent for Windows is

set GST_PLUGIN_PATH=%CD%\build
set PATH=%PATH%;%CD%\open-source\local\bin;%CD%\open-source\local\lib

Now if you execute gst-inspect-1.0 kvssink you should get information on the plugin like

Factory Details:
  Rank                     primary + 10 (266)
  Long-name                KVS Sink
  Klass                    Sink/Video/Network
  Description              GStreamer AWS KVS plugin
  Author                   AWS KVS <[email protected]>

Plugin Details:
  Name                     kvssink
  Description              GStreamer AWS KVS plugin
  Filename                 /Users/seaduboi/workspaces/amazon-kinesis-video-streams-producer-sdk-cpp/build/libgstkvssink.so
  Version                  1.0
  License                  Proprietary
  Source module            kvssinkpackage
  Binary package           GStreamer
  Origin URL               http://gstreamer.net

If the build failed, or GST_PLUGIN_PATH is not properly set you will get output like

No such element or plugin 'kvssink'

Using Element

The kvssink element has the following required parameters:

  • stream-name -- The name of the destination Kinesis video stream.
  • storage-size -- The storage size of the device in megabytes. For information about configuring device storage, see StorageInfo.
  • access-key -- The AWS access key that is used to access Kinesis Video Streams. You must provide either this parameter or credential-path.
  • secret-key -- The AWS secret key that is used to access Kinesis Video Streams. You must provide either this parameter or credential-path.
  • credential-path -- A path to a file containing your credentials for accessing Kinesis Video Streams. For example credential files, see Sample Static Credential and Sample Rotating Credential. For more information on rotating credentials, see Managing Access Keys for IAM Users. You must provide either this parameter or access-key and secret-key.

For examples of common use cases you can look at Example: Kinesis Video Streams Producer SDK GStreamer Plugin

To Include Images/Events feature

The images feature is available in the sample kvs_gstreamer_audio_video_sample.cpp . To enable it include the argument "-e <event_option>" event option is a string that can be:

notification -- for a notification event image -- for an image event both -- for both

The events will start on the 2nd key frame, and will reoccur every 200 key frames. If you would to change this frequence you can edit the sample.

To run from a file

in the kvs_gstreamer_audio_video_sample.cpp if you would like to upload from a file, include the option flag -f <file_path>

Running in offline mode

By default, the samples run in near realtime mode. To set offline mode, set streamInfo.streamCaps.streamingType to STREAMING_TYPE_OFFLINE, where, streamInfo is of type StreamInfo, streamCaps is of type StreamCaps and streamingType is of type STREAMING_TYPE.

Dockerscripts

DEBUG

  • When building the JNI, if you run into a cmake error Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH), make sure your environment variables are set correctly:
    export JAVA_INCLUDE_PATH2=/Library/Java/JavaVirtualMachines/<YOUR_JDK_VERSION>/Contents/Home/include or export JAVA_INCLUDE_PATH2=$JAVA_HOME/include for Mac OS.
    export JAVA_INCLUDE_PATH2='/usr/java/<JDK_VERSION>/include' for Linux.
  • If you are successfully streaming but run into issue with playback. You can do export KVS_DEBUG_DUMP_DATA_FILE_DIR=/path/to/directory before streaming. Producer will then dump MKV files into that path. The file is exactly what KVS will receive. You can use MKVToolNIX to check that everything looks correct. You can also try to play the MKV file in compatible players.
  • If you would like to visualize the GStreamer pipeline being constructed in a GStreamer application, include the following after the elements have been linked: GST_DEBUG_BIN_TO_DOT_FILE(<gst-bin-object>, GST_DEBUG_GRAPH_SHOW_ALL, <file-name>); For example, if the application created a pipeline object GstPipeline* pipeline = gst_pipeline_new("test-pipeline"), and you would like to see the visualized pipeline with filename pipeline, add: GST_DEBUG_BIN_TO_DOT_FILE((GstBin*) pipeline, GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");. Also ensure to set the path to where you would like the file to be stored. export GST_DEBUG_DUMP_DOT_DIR=.. The file generated would be a .dot format. Convert to PDF to check the visualized pipeline. Also, this requires graphviz to be installed. So make sure to install that.

FAQ

  • Is CPP-SDK and GStreamer supported on Mac/Windows/Linux (Supported Platforms)?
    Yes! We have FAQs and platform specific instructions for Windows, MacOS and Linux

Development

The repository is using develop branch as the aggregation and all of the feature development is done in appropriate feature branches. The PRs (Pull Requests) are cut on a feature branch and once approved with all the checks passed they can be merged by a click of a button on the PR tool. The master branch should always be build-able and all the tests should be passing. We are welcoming any contribution to the code base. The master branch contains our most recent release cycle from develop.

Release

The repository is under active development and even with incremental unit test coverage where some of the tests are actually full integration tests, we require more rigorous internal testing in order to 'cut' release versions. The release is cut against a particular commit that gets approved. The general philosophy is to cut a release when a set of commits contribute to a self-containing feature or when we add major internal functionality improvements.

Versioning

We deploy 3 digit version strings in a form of 'Major.Minor.Revision' scheme.

  • Major version update - Major functionality changes. Might not have direct backward compatibility. For example, multiple public API parameter changes.
  • Minor version update - Additional features. Major bug fixes. Might have some minor backward compatibility issues. For example, an extra parameter on a callback function.
  • Revision version update - Minor features. Bug fixes. Full backward compatibility. For example, an extra fields added to the public structures with version bump.

Related

License

This library is licensed under the Apache 2.0 License.

More Repositories

1

git-secrets

Prevents you from committing secrets and credentials into git repositories
Shell
11,616
star
2

llrt

LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.
JavaScript
7,555
star
3

aws-shell

An integrated shell for working with the AWS CLI.
Python
7,116
star
4

autogluon

AutoGluon: AutoML for Image, Text, and Tabular Data
Python
4,348
star
5

aws-cloudformation-templates

A collection of useful CloudFormation templates
Python
4,302
star
6

mountpoint-s3

A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
Rust
3,986
star
7

gluonts

Probabilistic time series modeling in Python
Python
3,686
star
8

deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Scala
2,871
star
9

aws-lambda-rust-runtime

A Rust runtime for AWS Lambda
Rust
2,829
star
10

aws-sdk-rust

AWS SDK for the Rust Programming Language
2,754
star
11

amazon-redshift-utils

Amazon Redshift Utils contains utilities, scripts and view which are useful in a Redshift environment
Python
2,643
star
12

diagram-maker

A library to display an interactive editor for any graph-like data.
TypeScript
2,359
star
13

amazon-ecr-credential-helper

Automatically gets credentials for Amazon ECR on docker push/docker pull
Go
2,261
star
14

amazon-eks-ami

Packer configuration for building a custom EKS AMI
Shell
2,164
star
15

aws-lambda-powertools-python

A developer toolkit to implement Serverless best practices and increase developer velocity.
Python
2,148
star
16

aws-well-architected-labs

Hands on labs and code to help you learn, measure, and build using architectural best practices.
Python
1,834
star
17

aws-config-rules

[Node, Python, Java] Repository of sample Custom Rules for AWS Config.
Python
1,473
star
18

smithy

Smithy is a protocol-agnostic interface definition language and set of tools for generating clients, servers, and documentation for any programming language.
Java
1,356
star
19

aws-support-tools

Tools and sample code provided by AWS Premium Support.
Python
1,290
star
20

open-data-registry

A registry of publicly available datasets on AWS
Python
1,199
star
21

sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
Python
1,181
star
22

aws-lambda-powertools-typescript

Powertools is a developer toolkit to implement Serverless best practices and increase developer velocity.
TypeScript
1,179
star
23

dgl-ke

High performance, easy-to-use, and scalable package for learning large-scale knowledge graph embeddings.
Python
1,144
star
24

aws-sdk-ios-samples

This repository has samples that demonstrate various aspects of the AWS SDK for iOS, you can get the SDK source on Github https://github.com/aws-amplify/aws-sdk-ios/
Swift
1,038
star
25

aws-sdk-android-samples

This repository has samples that demonstrate various aspects of the AWS SDK for Android, you can get the SDK source on Github https://github.com/aws-amplify/aws-sdk-android/
Java
1,018
star
26

aws-solutions-constructs

The AWS Solutions Constructs Library is an open-source extension of the AWS Cloud Development Kit (AWS CDK) that provides multi-service, well-architected patterns for quickly defining solutions
TypeScript
1,013
star
27

aws-cfn-template-flip

Tool for converting AWS CloudFormation templates between JSON and YAML formats.
Python
981
star
28

amazon-kinesis-video-streams-webrtc-sdk-c

Amazon Kinesis Video Streams Webrtc SDK is for developers to install and customize realtime communication between devices and enable secure streaming of video, audio to Kinesis Video Streams.
C
975
star
29

aws-lambda-go-api-proxy

lambda-go-api-proxy makes it easy to port APIs written with Go frameworks such as Gin (https://gin-gonic.github.io/gin/ ) to AWS Lambda and Amazon API Gateway.
Go
967
star
30

eks-node-viewer

EKS Node Viewer
Go
947
star
31

multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Java
936
star
32

ec2-spot-labs

Collection of tools and code examples to demonstrate best practices in using Amazon EC2 Spot Instances.
Jupyter Notebook
905
star
33

aws-mobile-appsync-sdk-js

JavaScript library files for Offline, Sync, Sigv4. includes support for React Native
TypeScript
902
star
34

aws-saas-boost

AWS SaaS Boost is a ready-to-use toolset that removes the complexity of successfully running SaaS workloads in the AWS cloud.
Java
901
star
35

fargatecli

CLI for AWS Fargate
Go
891
star
36

aws-api-gateway-developer-portal

A Serverless Developer Portal for easily publishing and cataloging APIs
JavaScript
879
star
37

ecs-refarch-continuous-deployment

ECS Reference Architecture for creating a flexible and scalable deployment pipeline to Amazon ECS using AWS CodePipeline
Shell
842
star
38

fortuna

A Library for Uncertainty Quantification.
Python
836
star
39

dynamodb-data-mapper-js

A schema-based data mapper for Amazon DynamoDB.
TypeScript
818
star
40

goformation

GoFormation is a Go library for working with CloudFormation templates.
Go
812
star
41

flowgger

A fast data collector in Rust
Rust
796
star
42

aws-js-s3-explorer

AWS JavaScript S3 Explorer is a JavaScript application that uses AWS's JavaScript SDK and S3 APIs to make the contents of an S3 bucket easy to browse via a web browser.
HTML
771
star
43

aws-icons-for-plantuml

PlantUML sprites, macros, and other includes for Amazon Web Services services and resources
Python
737
star
44

aws-devops-essential

In few hours, quickly learn how to effectively leverage various AWS services to improve developer productivity and reduce the overall time to market for new product capabilities.
Shell
674
star
45

aws-apigateway-lambda-authorizer-blueprints

Blueprints and examples for Lambda-based custom Authorizers for use in API Gateway.
C#
660
star
46

amazon-ecs-nodejs-microservices

Reference architecture that shows how to take a Node.js application, containerize it, and deploy it as microservices on Amazon Elastic Container Service.
Shell
650
star
47

amazon-kinesis-client

Client library for Amazon Kinesis
Java
621
star
48

aws-deployment-framework

The AWS Deployment Framework (ADF) is an extensive and flexible framework to manage and deploy resources across multiple AWS accounts and regions based on AWS Organizations.
Python
617
star
49

aws-lambda-web-adapter

Run web applications on AWS Lambda
Rust
610
star
50

dgl-lifesci

Python package for graph neural networks in chemistry and biology
Python
594
star
51

aws-security-automation

Collection of scripts and resources for DevSecOps and Automated Incident Response Security
Python
585
star
52

aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Python
565
star
53

python-deequ

Python API for Deequ
Python
535
star
54

aws-athena-query-federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.
Java
507
star
55

data-on-eks

DoEKS is a tool to build, deploy and scale Data & ML Platforms on Amazon EKS
HCL
504
star
56

shuttle

Shuttle is a library for testing concurrent Rust code
Rust
465
star
57

ami-builder-packer

An example of an AMI Builder using CI/CD with AWS CodePipeline, AWS CodeBuild, Hashicorp Packer and Ansible.
465
star
58

route53-dynamic-dns-with-lambda

A Dynamic DNS system built with API Gateway, Lambda & Route 53.
Python
461
star
59

aws-servicebroker

AWS Service Broker
Python
461
star
60

amazon-ecs-local-container-endpoints

A container that provides local versions of the ECS Task Metadata Endpoint and ECS Task IAM Roles Endpoint.
Go
456
star
61

datawig

Imputation of missing values in tables.
JavaScript
454
star
62

aws-jwt-verify

JS library for verifying JWTs signed by Amazon Cognito, and any OIDC-compatible IDP that signs JWTs with RS256, RS384, and RS512
TypeScript
452
star
63

amazon-dynamodb-lock-client

The AmazonDynamoDBLockClient is a general purpose distributed locking library built on top of DynamoDB. It supports both coarse-grained and fine-grained locking.
Java
447
star
64

ecs-refarch-service-discovery

An EC2 Container Service Reference Architecture for providing Service Discovery to containers using CloudWatch Events, Lambda and Route 53 private hosted zones.
Go
444
star
65

ssosync

Populate AWS SSO directly with your G Suite users and groups using either a CLI or AWS Lambda
Go
443
star
66

handwritten-text-recognition-for-apache-mxnet

This repository lets you train neural networks models for performing end-to-end full-page handwriting recognition using the Apache MXNet deep learning frameworks on the IAM Dataset.
Jupyter Notebook
442
star
67

awscli-aliases

Repository for AWS CLI aliases.
437
star
68

aws-config-rdk

The AWS Config Rules Development Kit helps developers set up, author and test custom Config rules. It contains scripts to enable AWS Config, create a Config rule and test it with sample ConfigurationItems.
Python
436
star
69

snapchange

Lightweight fuzzing of a memory snapshot using KVM
Rust
427
star
70

aws-security-assessment-solution

An AWS tool to help you create a point in time assessment of your AWS account using Prowler and Scout as well as optional AWS developed ransomware checks.
423
star
71

lambda-refarch-mapreduce

This repo presents a reference architecture for running serverless MapReduce jobs. This has been implemented using AWS Lambda and Amazon S3.
JavaScript
422
star
72

aws-lambda-cpp

C++ implementation of the AWS Lambda runtime
C++
409
star
73

aws-cloudsaga

AWS CloudSaga - Simulate security events in AWS
Python
389
star
74

amazon-kinesis-producer

Amazon Kinesis Producer Library
C++
385
star
75

soci-snapshotter

Go
383
star
76

pgbouncer-fast-switchover

Adds query routing and rewriting extensions to pgbouncer
C
381
star
77

serverless-photo-recognition

A collection of 3 lambda functions that are invoked by Amazon S3 or Amazon API Gateway to analyze uploaded images with Amazon Rekognition and save picture labels to ElasticSearch (written in Kotlin)
Kotlin
378
star
78

amazon-sagemaker-workshop

Amazon SageMaker workshops: Introduction, TensorFlow in SageMaker, and more
Jupyter Notebook
378
star
79

serverless-rules

Compilation of rules to validate infrastructure-as-code templates against recommended practices for serverless applications.
Go
378
star
80

logstash-output-amazon_es

Logstash output plugin to sign and export logstash events to Amazon Elasticsearch Service
Ruby
374
star
81

kinesis-aggregation

AWS libraries/modules for working with Kinesis aggregated record data
Java
370
star
82

smithy-rs

Code generation for the AWS SDK for Rust, as well as server and generic smithy client generation.
Rust
369
star
83

syne-tune

Large scale and asynchronous Hyperparameter and Architecture Optimization at your fingertips.
Python
363
star
84

aws-sdk-kotlin

Multiplatform AWS SDK for Kotlin
Kotlin
359
star
85

dynamodb-transactions

Java
354
star
86

amazon-kinesis-client-python

Amazon Kinesis Client Library for Python
Python
354
star
87

aws-serverless-data-lake-framework

Enterprise-grade, production-hardened, serverless data lake on AWS
Python
349
star
88

threat-composer

A simple threat modeling tool to help humans to reduce time-to-value when threat modeling
TypeScript
346
star
89

amazon-kinesis-agent

Continuously monitors a set of log files and sends new data to the Amazon Kinesis Stream and Amazon Kinesis Firehose in near-real-time.
Java
342
star
90

rds-snapshot-tool

The Snapshot Tool for Amazon RDS automates the task of creating manual snapshots, copying them into a different account and a different region, and deleting them after a specified number of days
Python
337
star
91

amazon-kinesis-scaling-utils

The Kinesis Scaling Utility is designed to give you the ability to scale Amazon Kinesis Streams in the same way that you scale EC2 Auto Scaling groups – up or down by a count or as a percentage of the total fleet. You can also simply scale to an exact number of Shards. There is no requirement for you to manage the allocation of the keyspace to Shards when using this API, as it is done automatically.
Java
333
star
92

landing-zone-accelerator-on-aws

Deploy a multi-account cloud foundation to support highly-regulated workloads and complex compliance requirements.
TypeScript
330
star
93

route53-infima

Library for managing service-level fault isolation using Amazon Route 53.
Java
326
star
94

aws-automated-incident-response-and-forensics

326
star
95

mxboard

Logging MXNet data for visualization in TensorBoard.
Python
326
star
96

aws-sigv4-proxy

This project signs and proxies HTTP requests with Sigv4
Go
325
star
97

statelint

A Ruby gem that provides a command-line validator for Amazon States Language JSON files.
Ruby
324
star
98

graphstorm

Enterprise graph machine learning framework for billion-scale graphs for ML scientists and data scientists.
Python
317
star
99

ecs-nginx-reverse-proxy

Reference architecture for deploying Nginx on ECS, both as a basic static resource server, and as a reverse proxy in front of a dynamic application server.
Nginx
317
star
100

simplebeerservice

Simple Beer Service (SBS) is a cloud-connected kegerator that streams live sensor data to AWS.
JavaScript
316
star