< Python

Share Share Share

Awesome Python Distributed Computing

• dask 11,979

  

  | updated 10 days ago | BSD 3-Clause "New...

  Parallel computing with task scheduling

• faust 6,668

  

  | updated 5 months ago | Other

  Python Stream Processing

• luigi 17,089

  

  | updated 3 months ago | Apache License 2.0

  Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

• mrjob 2,609

  

  | updated about 1 year ago | Other

  Run MapReduce jobs on Hadoop or Amazon Web Services

• ray 30,993

  

  | updated 9 days ago | Apache License 2.0

  Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

• streamparse 1,487

  

  | updated 8 months ago | Apache License 2.0

  Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.