Metaflow is a human-friendly library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix to boost productivity of data scientists who work on a wide variety of projects from classical statistics to state-of-the-art deep learning.
From prototype to production (and back)
- Rapid local prototyping, support for notebooks, and built-in experiment tracking and versioning.
- Horizontal and vertical scalability to the cloud, utilizing both CPUs and GPUs, and fast data access.
- Managing dependencies and one-click deployments to highly available production orchestrators.
Getting up and running is easy. If you don't know where to start, Metaflow sandbox will have you running and exploring Metaflow in seconds.
Installing Metaflow in your Python environment
To install Metaflow in your local environment, you can install from PyPi:
pip install metaflow
Alternatively, you can also install from conda-forge:
conda install -c conda-forge metaflow
Deploying infrastructure for Metaflow in your cloud
While you can get started with Metaflow easily on your laptop, the main benefits of Metaflow lie in its ability to scale out to external compute clusters and to deploy to production-grade workflow orchestrators. To benefit from these features, follow this guide to configure Metaflow and the infrastructure behind it appropriately.
An active community of thousands of data scientists and ML engineers discussing the ins-and-outs of applied machine learning.
- Introduction to Metaflow
- Natural Language Processing with Metaflow
- Computer Vision with Metaflow
- Recommender Systems with Metaflow
- And more advanced content here
- Infrastructure Stack for Large Language Models
- Parallelizing Stable Diffusion for Production Use Cases
- Whisper with Metaflow on Kubernetes
- Training a Large Language Model With Metaflow, Featuring Dolly
Get in touch
There are several ways to get in touch with us:
We welcome contributions to Metaflow. Please see our contribution guide for more details.