• This repository has been archived on 29/Nov/2023
  • Stars
    star
    435
  • Rank 100,085 (Top 2 %)
  • Language
    Python
  • License
    GNU Affero Genera...
  • Created about 4 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

ML pipeline orchestration and model deployments on Kubernetes.

Bodywork is a command line tool that deploys machine learning pipelines to Kubernetes. It takes care of everything to do with containers and orchestration, so that you don't have to.

Who is this for?

Bodywork is aimed at teams who want a solution for running ML pipelines and deploying models to Kubernetes. It is a lightweight and simpler alternative to Kubeflow, or to building your own platform based around a workflow orchestration tool like Apache Airflow, Argo Workflows or Dagster.

Pipeline = Jobs + Services

Any stage in a Bodywork pipeline can do one of two things:

  • run a batch job, to prepare features, train models, compute batch predictions, etc.
  • start a long-running process, like a Flask app that serves model predictions via HTTP.

You can use these to compose pipelines for many common ML use-cases, from serving pre-trained models to running continuous training on a schedule.

No Boilerplate Code Required

Defining a stage is as simple as developing an executable Python module or Jupyter notebook that performs the required tasks, and then committing it to your project's Git repository. You are free to structure your codebase as you wish and there are no new APIs to learn.

Git project structure

Easy to Configure

Stages are assembled into DAGs that define your pipeline's workflow. This and other key configuration is contained within a single bodywork.yaml file.

Simplified DevOps for ML

Bodywork removes the need for you to build and manage container images for any stage of your pipeline. It works by running all stages using Bodywork's custom container image. Each stage starts by pulling all the required files directly from your project's Git repository (e.g., from GitHub), pip-installing any required dependencies, and then running the stage's designated Python module (or Jupyter notebook).

Get Started

Bodywork is distributed as a Python package - install it from PyPI:

Add a bodywork.yaml file to your Python project’s Git repo. The contents of this file describe how your project will be deployed:

Bodywork is used from the command-line to deploy projects to Kubernetes clusters. With a single command, you can start Bodywork containers (hosted by us on Docker Hub), that pull Python modules directly from your project’s Git repo, and run them:

You don’t need to build Docker images or understand how to configure Kuberentes resources. Bodywork will fill the gap between executable Python modules and operational jobs and services on Kubernetes.

If you’re new to Kubernetes then check out our guide to Kubernetes for ML - we’ll have you up-and-running with a test cluster on your laptop, in under 10 minutes.

Documentation

The documentation for bodywork-core can be found here. This is the best place to start.

Deployment Templates

To accelerate your project's journey to production, we provide deployment templates for common use-cases:

We want your Feedback

If Bodywork sounds like a useful tool, then please send us a signal with a GitHub ★

Contacting Us

If you:

  • Have a question that these pages haven't answered, or need help getting started with Kubernetes, then please use our discussion board.
  • Have found a bug, then please open an issue.
  • Would like to contribute, then please talk to us first at [email protected].
  • Would like to commission new functionality, then please contact us at [email protected].

Bodywork is brought to you by Bodywork Machine Learning.

More Repositories

1

ml-pipeline-engineering

Best practices for engineering ML pipelines.
Jupyter Notebook
37
star
2

bodywork-ml-pipeline-project

Deployment template for a continuous training pipeline.
Jupyter Notebook
22
star
3

bodywork-pymc3-project

Serving Uncertainty with Bayesian inference, using PyMC3 with Bodywork
Jupyter Notebook
14
star
4

bodywork-scikit-fastapi-project

Deploy Scikit-Learn models on Kubernetes, using FastAPI with Bodywork.
Python
9
star
5

bodywork-mlflow

Deploy MLflow to Kubernetes using Bodywork
Jupyter Notebook
8
star
6

bodywork-serve-model-project

Deployment template for serving a ML model as web-service with a REST API.
Python
7
star
7

bodywork-batch-job-project

Deployment template for batch-scoring a dataset with a pre-trained ML model.
Python
6
star
8

bodywork-pipeline-utils

A package of utilities for engineering ML pipelines.
Python
5
star
9

bodywork-pipeline-with-aporia-monitoring

Integrating Aporia ML model monitoring into a Bodywork serving pipeline.
Jupyter Notebook
4
star
10

bodywork-ml-dashboard-project

Bodywork deployment template for training a ML model and then deploying a Plotly dashboard to visualise model metrics.
Python
3
star
11

bodywork-pipeline-with-cicd

CICD for ML using Bodywork.
Jupyter Notebook
2
star
12

bodywork-jupyter-pipeline-project

Bodywork deployment template demonstrating how to build ML pipelines with Jupyter notebooks and Bodywork.
Jupyter Notebook
2
star
13

bodywork-pipeline-with-mlflow

Bodywork template project for a train-and-serve pipeline that uses MLflow for tracking metrics and managing models.
Python
1
star
14

bodywork-test-project

Test resource for use with Bodywork integration tests.
Python
1
star
15

bodywork-rollback-deployment-test-project

Test resource for use with Bodywork integration tests.
Python
1
star