• Stars
    star
    1,406
  • Rank 33,453 (Top 0.7 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A platform that makes it easy for developers to build realtime, cost-effective, operations-focused applications

Mantis logo

Mantis Documentation

Build Status OSS Lifecycle License

Official Website


Development

Setting up Intellij

Run ./gradlew idea to (re-) generate IntelliJ project and module files from the templates in .baseline. The generated project is pre-configured with Baseline code style settings and support for the CheckStyle-IDEA plugin.

Install Docker

Install and run Docker to support local containers.

Building

$ ./gradlew clean build

Testing

$ ./gradlew clean test

Formatting the code

Run ./gradlew format task which autoformats all Java files using Spotless.

Building deployment into local Maven cache

$ ./gradlew clean publishNebulaPublicationToMavenLocal

Releasing

We release by tagging which kicks off a CI build. The CI build will run tests, integration tests, static analysis, checkstyle, build, and then publish to the public Bintray repo to be synced into Maven Central.

Tag format:

vMajor.Minor.Patch

You can tag via git or through Github's Release UI.

Contributing

Mantis is interested in building the community. We welcome any forms of contributions through discussions on any of our mailing lists or through patches.

For more information on contribution, check out the contributions file here.

Module Structure

This excludes all connectors and examples as they are mostly leaf nodes in the dependency graph.

Module Purpose Example Classes Package Prefixes
mantis-common-serde Support for serializing and deserializing POJOs using Json
  • JsonSerializer
  • io.mantisrx.common.*
  • mantis-discovery-proto todo: need to fill this in
  • JobDiscoveryProto
  • AppJobClustersMap
  • StageWorkers
  • com.netflix.mantis.discovery.proto.*
  • mantis-common Mantis common code shared across modules
  • MantisJobDefinition
  • MachineDefinition
  • MantisJobState
  • io.mantisrx.common.*
  • io.mantisrx.runtime.*
  • io.mantisrx.server.code.*
  • io.reactivx.mantis.operators.*
  • com.mantisrx.common.utils.*
  • mantis-remote-observable connection to other job,stage related code
  • ConnectToObservable
  • ConnectToGroupedObservable
  • RemoteObservable
  • io.reactivex.mantis.remote.observable.*
  • io.reactivex.netty.codec.*
  • mantis-control-plane-core common code between mantis-control-plane-server, mantis-control-plane-client, mantis-server-worker, mantis-server-agent
  • TaskExecutorID
  • ClusterID
  • ExecuteStageRequest
  • JobAssignmentResult
  • Status
  • io.mantisrx.server.core.*
  • io.mantisrx.server.master.resourcecluster.*
  • io.mantisrx.server.worker.*
  • mantis-control-plane-client API to talk to the mantis control plane server
  • MasterClientWrapper
  • MantisMasterGateway
  • MantisMasterClientApi
  • io.mantisrx.server.master.client.*
  • io.mantisrx.server.master.resourcecluster.*
  • mantis-network todo: need to fill this in
    mantis-publish-core todo: need to fill this in
    mantis-server-worker-client API to interact with workers
  • MetricsClient
  • MetricsClientImpl
  • WorkerConnection
  • WorkerMetricsClient
  • io.mantisrx.server.worker.client.*
  • mantis-runtime Runtime that the jobs need to depend upon. Job DSL should go in here along with how to talk to other jobs
  • KeyToKey
  • GroupToGroup
  • Source
  • Sink
  • io.mantisrx.runtime.*
  • mantis-publish-netty todo: need to fill this in
    mantis-client client to interact with mantis control plane
  • MantisClient::submitJob
  • MantisClient::killJob
  • io.mantisrx.client.*
  • mantis-publish-netty-guice todo: need to fill this in
    mantis-control-plane-server Actual server that runs the mantis master code
  • MasterMain
  • SchedulingService
  • ServiceLifecycle
  • io.mantisrx.master.*
  • io.mantisrx.server.master.*
  • mantis-server-agent Contains mantis-runtime agnostic code to start the task executor that runs on the agent
  • TaskExecutor
  • TaskExecutorStarter
  • BlobStore
  • BlobStoreAwareClassLoaderHandle
  • io.mantisrx.server.agent.*
  • mantis-server-worker One implementation of Mantis Worker that depends on the master runtime
  • MantisWorker
  • Task
  • ExecuteStageRequestService
  • JobAutoScaler
  • io.mantisrx.server.worker.config.*
  • io.mantisrx.server.worker.jobmaster.*
  • Dependency Graph

    graph TD;
        A[mantis-common-serde]-->B[mantis-common];
        B-->C[mantis-control-plane-core];
        B-->D[mantis-runtime];
        C-->E[mantis-control-plane-client];
        E-->F[mantis-server-worker-client];
        F-->G[mantis-client];
        E-->G;
        C-->H[mantis-control-plane-server];
        C-->I[mantis-server-worker];
        F-->I;
        D-->I;
        F-->J[mantis-server-agent];
        E-->J;
    

    More Repositories

    1

    Hystrix

    Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.
    Java
    23,594
    star
    2

    chaosmonkey

    Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
    Go
    14,410
    star
    3

    zuul

    Zuul is a gateway service that provides dynamic routing, monitoring, resiliency, security, and more.
    Java
    12,993
    star
    4

    conductor

    Conductor is a microservices orchestration engine.
    Java
    12,842
    star
    5

    eureka

    AWS Service registry for resilient mid-tier load balancing and failover.
    Java
    11,991
    star
    6

    falcor

    A JavaScript library for efficient data fetching
    JavaScript
    10,338
    star
    7

    pollyjs

    Record, Replay, and Stub HTTP Interactions.
    JavaScript
    10,184
    star
    8

    metaflow

    🚀 Build and manage real-life ML, AI, and data science projects with ease!
    Python
    8,012
    star
    9

    SimianArmy

    Tools for keeping your cloud operating in top form. Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
    Java
    7,955
    star
    10

    fast_jsonapi

    No Longer Maintained - A lightning fast JSON:API serializer for Ruby Objects.
    Ruby
    5,078
    star
    11

    vmaf

    Perceptual video quality assessment based on multi-method fusion.
    Python
    4,563
    star
    12

    dispatch

    All of the ad-hoc things you're doing to manage incidents today, done for you, and much more!
    Python
    4,548
    star
    13

    ribbon

    Ribbon is a Inter Process Communication (remote procedure calls) library with built in software load balancers. The primary usage model involves REST calls with various serialization scheme support.
    Java
    4,468
    star
    14

    security_monkey

    Security Monkey monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
    Python
    4,347
    star
    15

    dynomite

    A generic dynamo implementation for different k-v storage engines
    C
    4,104
    star
    16

    vizceral

    WebGL visualization for displaying animated traffic graphs
    JavaScript
    4,047
    star
    17

    vector

    Vector is an on-host performance monitoring framework which exposes hand picked high resolution metrics to every engineer’s browser.
    JavaScript
    3,588
    star
    18

    atlas

    In-memory dimensional time series database.
    Scala
    3,331
    star
    19

    concurrency-limits

    Java
    3,216
    star
    20

    consoleme

    A Central Control Plane for AWS Permissions and Access
    Python
    3,114
    star
    21

    dgs-framework

    GraphQL for Java with Spring Boot made easy.
    Kotlin
    3,044
    star
    22

    flamescope

    FlameScope is a visualization tool for exploring different time ranges as Flame Graphs.
    Python
    2,979
    star
    23

    bless

    Repository for BLESS, an SSH Certificate Authority that runs as a AWS Lambda function
    Python
    2,722
    star
    24

    archaius

    Library for configuration management API
    Java
    2,435
    star
    25

    asgard

    [Asgard is deprecated at Netflix. We use Spinnaker ( www.spinnaker.io ).] Web interface for application deployments and cloud management in Amazon Web Services (AWS). Binary download: http://github.com/Netflix/asgard/releases
    Groovy
    2,235
    star
    26

    curator

    ZooKeeper client wrapper and rich ZooKeeper framework
    Java
    2,138
    star
    27

    EVCache

    A distributed in-memory data store for the cloud
    Java
    2,001
    star
    28

    titus

    1,995
    star
    29

    lemur

    Repository for the Lemur Certificate Manager
    Python
    1,651
    star
    30

    bpftop

    bpftop provides a dynamic real-time view of running eBPF programs. It displays the average runtime, events per second, and estimated total CPU % for each program.
    Rust
    1,647
    star
    31

    genie

    Distributed Big Data Orchestration Service
    Java
    1,635
    star
    32

    metacat

    Java
    1,555
    star
    33

    netflix.github.com

    HTML
    1,419
    star
    34

    servo

    Netflix Application Monitoring Library
    Java
    1,408
    star
    35

    vectorflow

    D
    1,287
    star
    36

    hubcommander

    A Slack bot for GitHub organization management -- and other things too
    Python
    1,262
    star
    37

    rend

    A memcached proxy that manages data chunking and L1 / L2 caches
    Go
    1,174
    star
    38

    hollow

    Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access.
    Java
    1,148
    star
    39

    repokid

    AWS Least Privilege for Distributed, High-Velocity Deployment
    Python
    1,104
    star
    40

    astyanax

    Cassandra Java Client
    Java
    1,034
    star
    41

    Priam

    Co-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra.
    Java
    1,024
    star
    42

    aminator

    A tool for creating EBS AMIs. This tool currently works for CentOS/RedHat Linux images and is intended to run on an EC2 instance.
    Python
    938
    star
    43

    Turbine

    SSE Stream Aggregator
    Java
    831
    star
    44

    governator

    Governator is a library of extensions and utilities that enhance Google Guice to provide: classpath scanning and automatic binding, lifecycle management, configuration to field mapping, field validation and parallelized object warmup.
    Java
    821
    star
    45

    Fido

    C#
    816
    star
    46

    suro

    Netflix's distributed Data Pipeline
    Java
    783
    star
    47

    spectator

    Client library for collecting metrics.
    Java
    743
    star
    48

    security-bulletins

    Security Bulletins that relate to Netflix Open Source
    734
    star
    49

    Fenzo

    Extensible Scheduler for Mesos Frameworks
    Java
    703
    star
    50

    msl

    Message Security Layer
    C++
    687
    star
    51

    unleash

    Professionally publish your JavaScript modules in one keystroke
    JavaScript
    590
    star
    52

    denominator

    Portably control DNS clouds using java or bash
    Java
    573
    star
    53

    blitz4j

    Logging framework for fast asynchronous logging
    Java
    559
    star
    54

    edda

    AWS API Read Cache
    Scala
    554
    star
    55

    PigPen

    Map-Reduce for Clojure
    Clojure
    551
    star
    56

    netflix-graph

    Compact in-memory representation of directed graph data
    Java
    548
    star
    57

    go-env

    a golang library to manage environment variables
    Go
    542
    star
    58

    karyon

    The nucleus or the base container for Applications and Services built using the NetflixOSS ecosystem
    Java
    495
    star
    59

    Prana

    A sidecar for your NetflixOSS based services.
    Java
    492
    star
    60

    iceberg

    Iceberg is a table format for large, slow-moving tabular data
    Java
    465
    star
    61

    Lipstick

    Pig Visualization framework
    JavaScript
    464
    star
    62

    Surus

    Java
    453
    star
    63

    aws-autoscaling

    Tools and Documentation about using Auto Scaling
    Shell
    429
    star
    64

    go-expect

    an expect-like golang library to automate control of terminal or console based programs.
    Go
    422
    star
    65

    nf-data-explorer

    The Data Explorer gives you fast, safe access to data stored in Cassandra, Dynomite, and Redis.
    TypeScript
    420
    star
    66

    Workflowable

    Ruby
    370
    star
    67

    osstracker

    Github organization OSS metrics collector and metrics dashboard
    Scala
    365
    star
    68

    vizceral-example

    Example Vizceral app
    JavaScript
    363
    star
    69

    ndbench

    Netflix Data Store Benchmark
    HTML
    360
    star
    70

    Raigad

    Co-Process for backup/recovery, Auto Deployments and Centralized Configuration management for ElasticSearch
    Java
    346
    star
    71

    recipes-rss

    RSS Reader Recipes that uses several of the Netflix OSS components
    Java
    339
    star
    72

    aegisthus

    A Bulk Data Pipeline out of Cassandra
    Java
    323
    star
    73

    weep

    The ConsoleMe CLI utility
    Go
    322
    star
    74

    metaflow-ui

    🎨 UI for monitoring your Metaflow executions!
    TypeScript
    318
    star
    75

    titus-control-plane

    Titus is the Netflix Container Management Platform that manages containers and provides integrations to the infrastructure ecosystem.
    Java
    316
    star
    76

    dyno-queues

    Dyno Queues is a recipe that provides task queues utilizing Dynomite.
    Java
    264
    star
    77

    image_compression_comparison

    Image Compression Comparison Framework
    Python
    258
    star
    78

    falcor-express-demo

    Demonstration Falcor end point for a Netflix-style Application using express
    HTML
    246
    star
    79

    gradle-template

    Java
    244
    star
    80

    ember-nf-graph

    Composable graphing component library for EmberJS.
    JavaScript
    241
    star
    81

    falcor-router-demo

    A demonstration of how to build a Router for a Netflix-like application
    JavaScript
    236
    star
    82

    titus-executor

    Titus Executor is the container runtime/executor implementation for Titus
    Go
    233
    star
    83

    photon

    Photon is a Java implementation of the Interoperable Master Format (IMF) standard. IMF is a SMPTE standard whose core constraints are defined in the specification st2067-2:2013
    Java
    233
    star
    84

    dial-reference

    C
    228
    star
    85

    s3mper

    s3mper - Consistent Listing for S3
    Java
    218
    star
    86

    ReactiveLab

    Experiments and prototypes with reactive application design.
    Java
    208
    star
    87

    inviso

    JavaScript
    205
    star
    88

    NfWebCrypto

    Web Cryptography API Polyfill
    C++
    205
    star
    89

    staash

    A language-agnostic as well as storage-agnostic web interface for storing data into persistent storage systems, the metadata layer abstracts a lot of storage details and the pattern automation APIs take care of automating common data access patterns.
    Java
    204
    star
    90

    zeno

    Netflix's In-Memory Data Propagation Framework
    Java
    200
    star
    91

    brutal

    A multi-network asynchronous chat bot framework using twisted
    Python
    200
    star
    92

    vizceral-react

    JavaScript
    199
    star
    93

    dispatch-docker

    Shell
    193
    star
    94

    metaflow-service

    🚀 Metadata tracking and UI service for Metaflow!
    Python
    187
    star
    95

    pytheas

    Web Resources and UI Framework
    JavaScript
    187
    star
    96

    dyno

    Java client for Dynomite
    Java
    184
    star
    97

    hal-9001

    Hal-9001 is a Go library that offers a number of facilities for creating a bot and its plugins.
    Go
    178
    star
    98

    Nicobar

    Java
    171
    star
    99

    lemur-docker

    Docker files for the Lemur certificate orchestration tool
    Python
    170
    star
    100

    yetch

    Yet-another-fetch polyfill library. Supports AbortController/AbortSignal
    JavaScript
    168
    star