• Stars
    star
    590
  • Rank 73,170 (Top 2 %)
  • Language
    Java
  • License
    BSD 2-Clause "Sim...
  • Created over 4 years ago
  • Updated 26 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Lightweight hardware accelerated video/audio transcoder for Android.

LiTr

Build Status

LiTr (pronounced "lai-tr") is a lightweight video/audio transformation tool which supports transcoding video and audio tracks with optional frame modification.

In its current iteration LiTr supports:

  • changing resolution and/or bitrate of a video track(s)
  • changing sampling rate, channel count and/or bitrate of an audio track(s)
  • overlaying bitmap watermark onto video track(s)
  • applying different effects (brightness/contrast, saturation/hue, blur, etc.) to video pixels
  • including/excluding tracks, which allows muxing/demuxing tracks
  • transforming tracks individually (e.g. apply overlay to one video track, but not the other)
  • positioning source video frame arbitrarily onto target video frame
  • trimming video/audio
  • creating "empty" video, or a video out of single image
  • recording audio
  • creating preview bitmap(s) (with filters applied) at specific timestamp(s) (filmstrip)
  • writing raw audio into WAV container
  • record video using camera2 API

By default, LiTr uses Android MediaCodec stack for hardware accelerated decoding/encoding and OpenGL for rendering. It also uses MediaExtractor and MediaMuxer to read/write media.

Getting Started

Simply grab via Gradle:

 implementation 'com.linkedin.android.litr:litr:1.5.5'

...or Maven:

<dependency>
  <groupId>com.linkedin.android.litr</groupId>
  <artifactId>litr</artifactId>
  <version>1.5.5</version>
</dependency>

How to Transform a Video

First, instantiate MediaTransformer with a Context that can access Uris you will be using for input and output. Most commonly, that will be an application context.

MediaTransformer mediaTransformer = new MediaTransformer(getApplicationContext());

Then simply call transform method to transform your video:

mediaTransformer.transform(requestId,
                           sourceVideoUri,
                           targetVideoFilePath,
                           targetVideoFormat,
                           targetAudioFormat,
                           videoTransformationListener,
                           transformationOptions);

Few notable things related to transformation:

  • make sure to provide a unique requestId, it will be used when calling back on a listener, or needed when cancelling an ongoing transformation
  • target formats will be applied to all tracks of that type, non video or audio tracks will be copied "as is"
  • passing null target format means that you don't want to modify track(s) of that type
  • transformation is performed asynchronously, listener will be called with any transformation progress or state changes
  • by default listener callbacks happen on a UI thread, it is safe to update UI in listener implementation. It is also possible to have them on a non-UI transformation thread, for example, if any "heavy" works needs to be done in listener implementation.
  • if you want to modify video frames, pass in a list of GlFilters in TransformationOptions, which will be applied in order
  • if you want to modify audio frames, pass in a list of BufferFilters in TransformationOptions, which will be applied in order
  • client can call transform multiple times, to queue transformation requests
  • video will be written into MP4 container, we recommend using H.264 ("video/avc" MIME type) for target encoding. If VP8 or VP9 MIME type is used for target video track, audio track will be encoded using Opus codec, and tracks will be written into WebM container.
  • progress update granularity is 100 by default, to match percentage, and can be set in TransformationOptions
  • media can be optionally trimmed by specifying a MediaRange in TransformationOptions

Ongoing transformation can be cancelled by calling cancel with its requestId:

mediaTransformer.cancel(requestId);

When you no longer need MediaTransformer, please release it. Note that MediaTransformer instance becomes unusable after you release it, you will have to instantiate a new one.

mediaTransformer.release();

Handling errors

When transformation fails, exception is not thrown, but rather provided in TransformationListener.onError callback. LiTr defines its own exceptions for different scenarios. For API >= 23, LiTr exception will also contain MediaCodec.CodecException as a cause.

Reporting statistics

When possible, transformation statistics will be provided in listener callbacks. Statistics include source and target track formats, codecs used and transformation result and time for each track.

Beyond Defaults

By default, LiTr uses Android MediaCodec stack to do all media work, and OpenGl for rendering. But this is not set in stone.

At high level, LiTr breaks down transformation into five essential steps:

  • reading encoded frame from source container
  • decoding source frame
  • rendering a source frame onto target frame, optionally modifying it (for example, overlaying a bitmap)
  • encoding target frame
  • writing encoded target frame into target container

Each transformation step is performed by a component. Each component is abstracted as an interface:

  • MediaSource
  • Decoder
  • Renderer
  • Encoder
  • MediaTarget

When using your own component implementations, make sure that output of a component matches the expected input of a next component. For example, if you are using a custom Encoder (AV1?), make sure it accepts whatever frame format Renderer produces (GlSurface, ByteBuffer) and outputs what MediaTarget expects as an input.

Custom components can be used in TrackTransforms in below "low level" transform method:

transform(requestId,
         List<TrackTransform> trackTransforms,
         listener,
         granularity)

This API allows defining components and parameters per media track, thus allowing track based operations, such as muxing/demuxing tracks, transcoding different tracks differently, changing track order, etc.

Using Filters

You can use custom filters to modify video/audio frames. If you are writing a custom video filter, implement GlFilter interface to make extra OpenGL draw operations. If you need to change how source video frame is rendered onto a target video frame, implement GlFrameRender interface. For audio filter, implement BufferFilter.

LiTr now has 40 new GPU accelerated video filters ported from Mp4Composer-android and android-gpuimage projects. You can also create your own filter simply by configuring VideoFrameRenderFilter with your custom shader, with no extra coding!

All video/audio filters live in "filter pack" library, which is available via Gradle:

 implementation 'com.linkedin.android.litr:litr-filters:1.5.5'

...or Maven:

<dependency>
    <groupId>com.linkedin.android.litr</groupId>
    <artifactId>litr-filters</artifactId>
    <version>1.5.5</version>
</dependency>

You can pass in a list of filters when transforming a video or audio track. Keep in mind that filters will be applied in the order they are in the list, so ordering matters.

Using in Tests

MediaTransformer is very intentionally not a singleton, to allow easy mocking of it in unit tests. There is also MockMediaTransformer for UI tests, which can synchronously "play back" a sequence of listener callbacks.

Testing

Core business logic in LiTr is well covered by unit tests. LiTr is designed to use dependency injection pattern, which makes it very easy to write JVM tests with mocked dependencies. We use Mockito framework for mocking.

Demo App

LiTr comes with pretty useful demo app, which lets you transcode video/audio tracks with different parameters, in addition to providing sample code.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

For the versions available, see the tags on this repository.

Snapshots

You can use snapshot builds to test the latest unreleased changes. A new snapshot is published after every merge to the main branch by the Deploy Snapshot Github Action workflow.

Just add the Sonatype snapshot repository to your Gradle scripts:

repositories {
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots/"
    }
}

You can find the latest snapshot version to use in the gradle.properties file.

Authors

  • Izzat Bahadirov - Initial work - LiTr

See also the list of contributors who participated in this project.

License

This project is licensed under the BSD 2-Clause License - see the LICENSE file for details

Acknowledgments

  • A huge thank you to ypresto for his pioneering work on android-transcoder project, which was an inspiration and heavy influence on LiTr
  • A special thank you to MasayukiSuda for his work on Mp4Composer-android project, whose filters now power LiTr, and for his work on ExoPlayerFilter project which was a foundation for filter preview functionality in LiTr.
  • A special thank you to android-gpuimage project for amazing filter collection, which have been ported into LiTr
  • A special thank you to IanDBird for his phenomenal work on making LiTr capable of working as a video/audio recorder, as well as his ongoing contributions to LiTr
  • A thank you to Google's AOSP CTS team for writing Surface to Surface rendering implementation in OpenGL, which became a foundation for GlRenderer in LiTr
  • A thank you to Google Oboe project for high quality audio resampling implementation, which became a foundation of audio processing in LiTr
  • A shout out to my awesome colleagues Amita Sahasrabudhe, Long Peng, Keerthi Korrapati and Vasiliy Kulakov for contributions and code reviews
  • A shout out to my colleague Vidhya Pandurangan for prototyping video trimming, which now became a feature
  • A shout out to our designer Mauroof Ahmed for giving LiTr a visual identity
  • A shout out to PurpleBooth for very useful README.md template
  • A shout out to Ma7moudHatem for his invaluable contributions to LiTr

More Repositories

1

school-of-sre

At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.
HTML
7,649
star
2

css-blocks

High performance, maintainable stylesheets.
TypeScript
6,334
star
3

Burrow

Kafka Consumer Lag Checking
Go
3,644
star
4

databus

Source-agnostic distributed change data capture system
Java
3,587
star
5

qark

Tool to look for several security related Android application vulnerabilities
Python
3,117
star
6

dustjs

Asynchronous Javascript templating for the browser and server
JavaScript
2,917
star
7

cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
Java
2,634
star
8

rest.li

Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs.
Java
2,435
star
9

kafka-monitor

Xinfra Monitor monitors the availability of Kafka clusters by producing synthetic workloads using end-to-end pipelines to obtain derived vital statistics - E2E latency, service produce/consume availability, offsets commit availability & latency, message loss rate and more.
Java
1,991
star
10

dexmaker

A utility for doing compile or runtime code generation targeting Android's Dalvik VM
Java
1,843
star
11

greykite

A flexible, intuitive and fast forecasting library
Python
1,788
star
12

ambry

Distributed object store
Java
1,717
star
13

shiv

shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.
Python
1,693
star
14

swift-style-guide

LinkedIn's Official Swift Style Guide
1,436
star
15

dr-elephant

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark
Java
1,341
star
16

detext

DeText: A Deep Neural Text Understanding Framework for Ranking and Classification Tasks
Python
1,254
star
17

parseq

Asynchronous Java made easier
Java
1,158
star
18

luminol

Anomaly Detection and Correlation library
Python
1,151
star
19

oncall

Oncall is a calendar tool designed for scheduling and managing on-call shifts. It can be used as source of dynamic ownership info for paging systems like http://iris.claims.
Python
1,095
star
20

test-butler

Reliable Android Testing, at your service
Java
1,040
star
21

goavro

Go
948
star
22

PalDB

An embeddable write-once key-value store written in Java
Java
934
star
23

brooklin

An extensible distributed system for reliable nearline data streaming at scale
Java
891
star
24

iris

Iris is a highly configurable and flexible service for paging and messaging.
Python
791
star
25

photon-ml

A scalable machine learning library on Apache Spark
Terra
790
star
26

Hakawai

A powerful, extensible UITextView.
Objective-C
780
star
27

URL-Detector

A Java library to detect and normalize URLs in text
Java
778
star
28

eyeglass

NPM Modules for Sass
TypeScript
741
star
29

opticss

A CSS Optimizer
TypeScript
715
star
30

coral

Coral is a translation, analysis, and query rewrite engine for SQL and other relational languages.
Java
714
star
31

pygradle

Using Gradle to build Python projects
Java
584
star
32

kafka-tools

A collection of tools for working with Apache Kafka.
Python
581
star
33

flashback

mock the internet
Java
578
star
34

LayoutTest-iOS

Write unit tests which test the layout of a view in multiple configurations
Objective-C
565
star
35

FeatureFu

Library and tools for advanced feature engineering
Java
564
star
36

FastTreeSHAP

Fast SHAP value computation for interpreting tree-based models
Python
493
star
37

venice

Venice, Derived Data Platform for Planet-Scale Workloads.
Java
413
star
38

Spyglass

A library for mentions on Android
Java
381
star
39

dagli

Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).
Java
353
star
40

ml-ease

ADMM based large scale logistic regression
Java
333
star
41

cruise-control-ui

Cruise Control Frontend (CCFE): Single Page Web Application to Manage Large Scale of Kafka Clusters
Vue
329
star
42

transport

A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Apache Hive, and Presto.
Java
288
star
43

dph-framework

HTML
285
star
44

spark-tfrecord

Read and write Tensorflow TFRecord data from Apache Spark.
Scala
276
star
45

openhouse

Open Control Plane for Tables in Data Lakehouse
Java
256
star
46

isolation-forest

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.
Scala
217
star
47

LiFT

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.
Scala
167
star
48

shaky-android

Shake to send feedback for Android.
Java
157
star
49

pyexchange

Python wrapper for Microsoft Exchange
Python
151
star
50

asciietch

A graphing library with the goal of making it simple to graphs using ascii characters.
Python
137
star
51

python-avro-json-serializer

Serializes data into a JSON format using AVRO schema.
Python
136
star
52

li-apache-kafka-clients

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
Java
132
star
53

gdmix

A deep ranking personalization framework
Python
131
star
54

dynamometer

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
Java
129
star
55

Avro2TF

Avro2TF is designed to fill the gap of making users' training data ready to be consumed by deep learning training frameworks.
Scala
125
star
56

linkedin-gradle-plugin-for-apache-hadoop

Groovy
117
star
57

datahub-gma

General Metadata Architecture
Java
112
star
58

dex-test-parser

Find all test methods in an Android instrumentation APK
Kotlin
104
star
59

cassette

An efficient, file-based FIFO Queue for iOS and macOS.
Objective-C
95
star
60

spaniel

LinkedIn's JavaScript viewport tracking library and IntersectionObserver polyfill
JavaScript
91
star
61

Hoptimator

Multi-hop declarative data pipelines
Java
77
star
62

migz

Multithreaded, gzip-compatible compression and decompression, available as a platform-independent Java library and command-line utilities.
Java
76
star
63

sysops-api

sysops-api is a framework designed to provide visability from tens of thousands of machines in seconds.
Python
75
star
64

avro-util

Collection of utilities to allow writing java code that operates across a wide range of avro versions.
Java
73
star
65

iceberg

A temporary home for LinkedIn's changes to Apache Iceberg (incubating)
Java
60
star
66

kube2hadoop

Secure HDFS Access from Kubernetes
Java
59
star
67

linkedin.github.com

Listing of all our public GitHub projects.
JavaScript
59
star
68

dynoyarn

DynoYARN is a framework to run simulated YARN clusters and workloads for YARN scale testing.
Java
58
star
69

Tachyon

An Android library that provides a customizable calendar day view UI widget.
Java
57
star
70

DuaLip

DuaLip: Dual Decomposition based Linear Program Solver
Scala
56
star
71

iris-relay

Stateless reverse proxy for thirdparty service integration with Iris API.
Python
49
star
72

Cytodynamics

Classloader isolation library.
Java
48
star
73

concurrentli

Classes for multithreading that expand on java.util.concurrent, adding convenience, efficiency and new tools to multithreaded Java programs
Java
43
star
74

iris-mobile

A mobile interface for linkedin/iris, built for iOS and Android on the Ionic platform
TypeScript
41
star
75

instantsearch-tutorial

Sample code for building an end-to-end instant search solution
JavaScript
39
star
76

lambda-learner

Lambda Learner is a library for iterative incremental training of a class of supervised machine learning models.
Python
37
star
77

self-focused

Helps make a single page application more friendly to screen readers.
JavaScript
35
star
78

tracked-queue

An autotracked implementation of a ring-buffer-backed double-ended queue
TypeScript
35
star
79

PASS-GNN

Python
35
star
80

QueryAnalyzerAgent

Analyze MySQL queries with negligible overhead
Go
35
star
81

TE2Rules

Python library to explain Tree Ensemble models (TE) like XGBoost, using a rule list.
Python
31
star
82

performance-quality-models

Personalizing Performance model repository
Jupyter Notebook
31
star
83

Iris-message-processor

Iris-message-processor is a fully distributed Go application meant to replace the sender functionality of Iris and provide reliable, scalable, and extensible incident and out of band message processing and sending.
Go
26
star
84

smart-arg

Smart Arguments Suite (smart-arg) is a slim and handy python lib that helps one work safely and conveniently with command line arguments.
Python
23
star
85

data-integration-library

The Data Integration Library project provides a library of generic components based on a multi-stage architecture for data ingress and egress.
Java
22
star
86

linkedin-calcite

LinkedIn's version of Apache Calcite
Java
22
star
87

atscppapi

This library provides wrappers around the existing Apache Traffic Server API which will vastly simplify the process of writing Apache Traffic Server plugins.
C++
20
star
88

high-school-trainee

LinkedIn Women in Tech High School Trainee Program
Python
18
star
89

play-parseq

Play-ParSeq is a Play module which seamlessly integrates ParSeq with Play Framework
Scala
17
star
90

forthic

Python
17
star
91

icon-magic

Automated icon build system for iOS, Android and Web
TypeScript
17
star
92

QuantEase

QuantEase, a layer-wise quantization framework, frames the problem as discrete-structured non-convex optimization. Our work leverages Coordinate Descent techniques, offering high-quality solutions without the need for matrix inversion or decomposition.
Python
15
star
93

kafka-remote-storage-azure

Java
13
star
94

play-restli

A library that simplifies building restli services on top of the play server.
Java
12
star
95

spark-inequality-impact

Scala
11
star
96

AlerTiger

Jupyter Notebook
9
star
97

Li-Airflow-Backfill-Plugin

Li-Airflow-Backfill-Plugin is a plugin to work with Apache Airflow to provide data backfill feature, ie. to rerun pipelines for a certain date range.
Python
8
star
98

zookeeper

Mirror of Apache Hadoop ZooKeeper
Java
6
star
99

gobblin-elr

This is a read-only mirror of apache/gobblin
Java
5
star
100

o19-bmc-firmware

OpenBMC is an open software framework to build a complete Linux image for a Board Management Controller (BMC)
C
4
star