Make-like libraries and tools for Python

This is just a list of tools that I have found and a short overview.

  • Dask: “Dask is a flexible library for parallel computing in Python.”
  • Pyflow: “…tool to manage tasks in the context of a task dependency graph”
  • Celery: “Distributed Task Queue”
  • Luigi: “…helps you build complex pipelines of batch jobs”
  • Taskgraph: “A task graph execution framework for Python”
  • DoIt: “…bringing the power of build-tools to execute any kind of task”