# Make-like libraries and tools for Python

I’ve been searching for a Python library that would allow me to structure build tasks like GNU Make does, however, without much success. My objective is not to write a replacement for Make, but to integrate this process with the rest of my Python-based tools.

Furthermore, I was looking for an object-oriented, extensible and customizable API, that would let me specify targets, requirements and “up to date” definitions arbitrarily.

This is just a list of tools that I have found and a short overview.

• Dask: “Dask is a flexible library for parallel computing in Python.”
• Pyflow: “…tool to manage tasks in the context of a task dependency graph”
• Luigi: “…helps you build complex pipelines of batch jobs”
• DoIt: “…bringing the power of build-tools to execute any kind of task”

The bottom line is that there is no such thing close to what I want. The project that is closest to my wish list is DoIt. DoIt is written in Python an lets you specify target, requirements and “up to date” conditions rather freely, in Python. However, it was designed as a Make-replacement command-line tool. It is intimately tied to command-line arguments, expects to find certain files in the current folder, and so on. My attempts to dig into the source and use it as a library were futile.

This lead me to that painful moment when you realize that you need to write your tool yourself, and that it is not going to be that easy. An open source endeavor begins!

Let us begin with the wish list in the form of an API use case:

builder = Builder(
CmdAction('wc -l b{m}.txt > a{m}.txt'),
FileTarget('a{m}.txt'),
FileDependency('b{m}.txt'),
),

builder['taskA'].run()