Discover linkedin/kafka-remote-storage-azure Open Source project

Kafka Consumer Lag Checking

6,335

Burrow

3,725

databus

Source-agnostic distributed change data capture system

Efficient Triton Kernels for LLM Training

3,636

Liger-Kernel

Tool to look for several security related Android application vulnerabilities

3,312

qark

Asynchronous Javascript templating for the browser and server

3,183

dustjs

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.

2,911

cruise-control

Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs.

2,734

rest.li

Xinfra Monitor monitors the availability of Kafka clusters by producing synthetic workloads using end-to-end pipelines to obtain derived vital statistics - E2E latency, service produce/consume availability, offsets commit availability & latency, message loss rate and more.

2,500

kafka-monitor

A utility for doing compile or runtime code generation targeting Android's Dalvik VM

2,016

dexmaker

A flexible, intuitive and fast forecasting library

1,863

greykite

1,813

ambry

Distributed object store

shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.

1,740

shiv

LinkedIn's Official Swift Style Guide

1,729

swift-style-guide

1,430

dr-elephant

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

DeText: A Deep Neural Text Understanding Framework for Ranking and Classification Tasks

1,353

detext

Anomaly Detection and Correlation library

1,263

luminol

Asynchronous Java made easier

1,182

parseq

Oncall is a calendar tool designed for scheduling and managing on-call shifts. It can be used as source of dynamic ownership info for paging systems like http://iris.claims.

1,165

oncall

Reliable Android Testing, at your service

1,137

test-butler

An embeddable write-once key-value store written in Java

1,046

goavro

972

PalDB

An extensible distributed system for reliable nearline data streaming at scale

937

brooklin

Iris is a highly configurable and flexible service for paging and messaging.

919

iris

A scalable machine learning library on Apache Spark

807

photon-ml

Terra

793

URL-Detector

A Java library to detect and normalize URLs in text

Coral is a translation, analysis, and query rewrite engine for SQL and other relational languages.

782

coral

A powerful, extensible UITextView.

781

Hakawai

eyeglass

opticss

LiTr

Lightweight hardware accelerated video/audio transcoder for Android.

A collection of tools for working with Apache Kafka.

609

kafka-tools

Using Gradle to build Python projects

592

pygradle

flashback

FeatureFu

Library and tools for advanced feature engineering

Write unit tests which test the layout of a view in multiple configurations

568

LayoutTest-iOS

Objective-C

564

FastTreeSHAP

Fast SHAP value computation for interpreting tree-based models

Venice, Derived Data Platform for Planet-Scale Workloads.

509

venice

A library for mentions on Android

487

Spyglass

Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).

386

dagli

Cruise Control Frontend (CCFE): Single Page Web Application to Manage Large Scale of Kafka Clusters

353

cruise-control-ui

Vue

337

ml-ease

ADMM based large scale logistic regression

Open Control Plane for Tables in Data Lakehouse

333

openhouse

dph-framework

transport

A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Apache Hive, and Presto.

Read and write Tensorflow TFRecord data from Apache Spark.

296

spark-tfrecord

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm with support for exporting in ONNX format.

288

isolation-forest

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

224

LiFT

Shake to send feedback for Android.

168

shaky-android

Python wrapper for Microsoft Exchange

160

pyexchange

A graphing library with the goal of making it simple to graphs using ascii characters.

153

asciietch

Serializes data into a JSON format using AVRO schema.

138

python-avro-json-serializer

A deep ranking personalization framework

137

gdmix

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.

131

li-apache-kafka-clients

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

131

dynamometer

Avro2TF is designed to fill the gap of making users' training data ready to be consumed by deep learning training frameworks.

131

Avro2TF

General Metadata Architecture

126

datahub-gma

Find all test methods in an Android instrumentation APK

121

linkedin-gradle-plugin-for-apache-hadoop

Groovy

117

dex-test-parser

Kotlin

106

cassette

An efficient, file-based FIFO Queue for iOS and macOS.

Objective-C

spaniel

LinkedIn's JavaScript viewport tracking library and IntersectionObserver polyfill

Multi-hop declarative data pipelines

Hoptimator

Multithreaded, gzip-compatible compression and decompression, available as a platform-independent Java library and command-line utilities.

migz

Collection of utilities to allow writing java code that operates across a wide range of avro versions.

avro-util

sysops-api is a framework designed to provide visability from tens of thousands of machines in seconds.

sysops-api

A temporary home for LinkedIn's changes to Apache Iceberg (incubating)

iceberg

DuaLip: Dual Decomposition based Linear Program Solver

DuaLip

Secure HDFS Access from Kubernetes

kube2hadoop

DynoYARN is a framework to run simulated YARN clusters and workloads for YARN scale testing.

dynoyarn

Listing of all our public GitHub projects.

linkedin.github.com

An Android library that provides a customizable calendar day view UI widget.

Tachyon

Classloader isolation library.

Cytodynamics

Stateless reverse proxy for thirdparty service integration with Iris API.

iris-relay

Classes for multithreading that expand on java.util.concurrent, adding convenience, efficiency and new tools to multithreaded Java programs

concurrentli

A mobile interface for linkedin/iris, built for iOS and Android on the Ionic platform

iris-mobile

Lambda Learner is a library for iterative incremental training of a class of supervised machine learning models.

lambda-learner

Python library to explain Tree Ensemble models (TE) like XGBoost, using a rule list.

TE2Rules

Sample code for building an end-to-end instant search solution

instantsearch-tutorial

PASS-GNN

Helps make a single page application more friendly to screen readers.

self-focused

An autotracked implementation of a ring-buffer-backed double-ended queue

tracked-queue

Analyze MySQL queries with negligible overhead

QueryAnalyzerAgent

performance-quality-models

Personalizing Performance model repository

Jupyter Notebook

data-integration-library

The Data Integration Library project provides a library of generic components based on a multi-stage architecture for data ingress and egress.

Iris-message-processor is a fully distributed Go application meant to replace the sender functionality of Iris and provide reliable, scalable, and extensible incident and out of band message processing and sending.

Iris-message-processor

smart-arg

Smart Arguments Suite (smart-arg) is a slim and handy python lib that helps one work safely and conveniently with command line arguments.

LinkedIn's version of Apache Calcite

linkedin-calcite

This library provides wrappers around the existing Apache Traffic Server API which will vastly simplify the process of writing Apache Traffic Server plugins.

atscppapi

C++

forthic

LinkedIn Women in Tech High School Trainee Program

high-school-trainee

Play-ParSeq is a Play module which seamlessly integrates ParSeq with Play Framework

play-parseq

Automated icon build system for iOS, Android and Web

icon-magic

QuantEase, a layer-wise quantization framework, frames the problem as discrete-structured non-convex optimization. Our work leverages Coordinate Descent techniques, offering high-quality solutions without the need for matrix inversion or decomposition.

QuantEase

A library that simplifies building restli services on top of the play server.

play-restli

spark-inequality-impact

Li-Airflow-Backfill-Plugin is a plugin to work with Apache Airflow to provide data backfill feature, ie. to rerun pipelines for a certain date range.

Li-Airflow-Backfill-Plugin

A fast and flexible implementation of the xDS protocol

AlerTiger

Jupyter Notebook

diderot

100

gobblin-elr

This is a read-only mirror of apache/gobblin