bake

The Dutch IRS has a catchy slogan, which goes like this: "Leuker kunnen we 't niet maken, wel makkelijker". Roughly translated this means: "We can't make it more fun, but we can make it easier". Bake adopts a similar philosophy. Building C/C++ code will never be fun, but with bake you'll probably spend a little less time worrying about it.

Here's how bake tries to alleviate some of the pain:

• Minimal, platform independent project configuration (as in 2 lines of JSON minimal)
• Create new projects with a single command
• Builtin emscripten (webasm) support
• Zero-dependencies, calls msvc, gcc, clang and emcc compilers directly
• Refer to dependencies by logical names, no OS/environment dependent paths
• Automatically include headers from dependencies
• Out of the box macro's for exporting symbols
• Automatically discover, order and build projects in directories without additional config
• Recursively build projects & their dependencies for the right config with a single command
• Generate single source/header files from any project
• A test framework
• Clone, build and run projects with their dependencies with a single command
• Make projects warning free on all compilers/compiler versions with strict compilation mode
• Out of the box support for running projects with address sanitizers
• Manage projects & their git repositories with bake bundles

Bake is verified on the following platforms:

• Linux
• MacOS
• Windows

Bake is verified on the following compilers

• gcc (7, 8, 9, 10)
• clang (8, 9, 10)
• msvc

Bake is used as the primary build system for Flecs.

Contents

• Installation
• Getting Started
• FAQ
• Manual
  • Introduction
  • Building projects
  • Running projects
  • Project kinds
  • Project layout
  • Project configuration
  • Project bundles
  • Template functions
  • Templates
  • Configuring Bake
  • Installing Miscellaneous Files
  • Integrating Non-bake Projects
  • The Bake Environment
  • Environment Variables
  • Command line interface
  • Writing drivers
• Authors
• Legal stuff

Installation

Install bake using the following commands:

On Linux/MacOS:

git clone https://github.com/SanderMertens/bake
bake/setup.sh

On Windows (requires C++ CMake tools for Windows individual component included in the Desktop development with C++ workload for Visual Studio):

git clone https://github.com/SanderMertens/bake
cd bake
setup

On MacOS/Linux bake will install a single script in /usr/local/bin which calls the bake binary in ~/bake, which may prompt for your password during installation. This script allows you to call bake from any location without having to make changes to your environment.

If you'd rather not install this script, you can install bake in local mode:

git clone https://github.com/SanderMertens/bake
cd bake
make -C build-$(uname) clean all
./bake setup --local

After the setup has finished, the bake executable will be stored in ~/bake. Make sure to add this directory to your PATH variable before using bake.

Upgrade bake

You can upgrade bake to the latest version by running this command:

bake upgrade

Getting started

The following commands are useful for getting started with bake. Also, check out the bake --help command, which lists all the options and commands available in the bake tool.

Create and run new project

To create and run a new bake application project called my_app, run the following commands:

bake new my_app
bake run my_app

You can also run projects in interactive mode. This will automatically rebuild and restart an application when a project file changes. To run in interactive mode, simply add --interactive to the bake command:

bake run my_app --interactive

Basic configuration with dependency and configuration for C driver

This example shows a simple configuration with a dependency on the foo.bar package and links with pthread.

{
    "id": "my_app",
    "type": "application",
    "value": {
        "use": ["foo.bar"]
    },
    "lang.c": {
        "lib": ["pthread"]
    }
}

Build, rebuild and clean a project

bake
bake rebuild
bake clean

Specify a build configuration:

bake --cfg release

Clone & build a project from git

Build a project and its dependencies directly from a git repository using this command:

bake clone https://github.com/SanderMertens/example

Export an environment variable to the bake environment

Bake can manage environment variables that must be set during the build. To export an environment variable to the bake environment, use this command:

bake export VAR=value

Alternatively, if you want to add a path to an environment variable like PATH or LD_LIBRARY_PATH, use this:

bake export PATH+=/my/path

These variables are stored in a configuration file called bake.json in the root of the bake environment, which by default is $HOME/bake.

To export the bake environment to a terminal, use:

export `bake env`

FAQ

Bake is built under the GPL3.0 license. Does this mean I cannot use it for commercial projects?

No. As long as you do not distribute bake (either as source or binary) as part of your (closed source) deliverable, you can use bake for building your projects. This is no different than when you would use make for your projects, which is also GPL licensed.

I want my customers to use bake. Does the license allow for this?

Yes. As long as your customers use the open source version of bake, and you do not distribute bake binaries or source files with your product, your customers can use bake.

I noticed a premake file in the bake repository. Does bake need premake to be installed?

No. Bake uses premake to generate its makefiles (we would've used bake to build bake- but chicken & egg etc). The generated makefiles are included in the bake repository, so you won't need premake to use bake.

Why yet another build tool?

To put it bluntly, existing tools for C/C++ aren't great.

Is bake a package manager?

No. Bake has package-management like features, like resolving projects by logica name, automatic project discovery & associate projects with git repositories, but it does not pretend to be a full-fledged package manager. The only reason bake has these features is because it makes the build process easier.

Why are bake project configurations so simple?

All C/C++ project build configurations basically boil down to the same set of rules to translate source files into objects, and objects into shared objects or application binaries. Yet with most build systems you'll find yourself copy-pasting the same rules every time you create a new project.

Bake doesn't need to be told how to build C/C++ code. It just needs three pieces of information:

• The project name
• Whether it's an application or a package
• A src and include directory

Why doesn't bake generate makefiles or visual studio files?

Makefiles and Visual Studio files are just elaborate front-ends for how to call compilers. Bake calls the compiler directly. It has builtin drivers for gcc, clang, msvc and emcc, and it automatically detects for which compiler it's building.

How does bake compare to make?

GNU make is a low-level tool that requires you to explicitly list all the rules required to translate code to binaries. Bake is a high-level tool that already knows how to build the code, and can do this with minimal configuration.

How does bake compare to CMake?

CMake is a tool that generates other build configurations. When you use CMake you still need to use another tool like Make or Visual Studio to actually build your project. Bake builds your code directly.

Another big difference is that CMake requires you to specify project configuration in a custom language, whereas Bake configuration is specified in JSON. Here is a CMake configuration and a Bake configuration for the same project:

CMake:

cmake_minimum_required (VERSION 2.6)

include_directories ("bar")
add_subdirectory (bar)
set (EXTRA_LIBS ${EXTRA_LIBS} bar)

project (foo)
add_executable(foo foo.c)

target_link_libraries (foo ${EXTRA_LIBS})

Bake:

{
    "id": "foo",
    "type": "application",
    "value": {
        "use": ["bar"]
    }
}

A difference that jumps out from the examples is that the bake configuration is agnostic to its environment. It knows where to find project bar, whereas in CMake this needs to be explicitly specified.

How does bake compare to premake?

Premake is a lot like CMake, but with a Lua-based project configuration that's slightly less verbose. Premake also generates build configurations, and requires additional tools to actually build your code.

How does bake compare to Bazel?

Out of all build systems, bake and bazel come closest in the way they approach building code. The biggest differences are:

• Bake is a much, much smaller project
• Bake is less controlling when it comes to your environment. If you want absolute control over which version of Python, make, gcc etc. you're using while building, Bazel can do a better job.
• Bake only builds C/C++ code.
• Bazel project identifiers are relative to a workspace. Bake project identifiers are universal.
• Bazel has a custom language for project congiguration

Can I link with non-bake libraries?

Yes. This example shows how to link with libm:

{
    "id": "my_app",
    "type": "application",
    "lang.c": {
        "lib": ["m"]
    }
}

This can be improved by ensuring that libm is only added on Linux (MacOS/Windows don't have libm):

{
    "id": "my_app",
    "type": "application",
    "${os linux}": {
        "lang.c": {
            "lib": ["m"]
        }
    }
}

I want to wrap a C library so I can use it as a bake dependency. How do I do this?

It would be nice if we could wrap libm.so from the previous example in a bake math package, aso we don't have to repeat this configuration for every project. Bake lets us do this with the "dependee" attribute:

{
    "id": "math",
    "type": "package",
    "value": {
        "language": "none"
    },
    "dependee": {
        "${os linux}": {
            "lang.c": {
                "lib": ["m"]
            }
        }
    }
}

This creates a new "math" package that you can now specify as regular bake dependency. The "language": "none" attribute lets bake know that there is no code to build, and this is a configuration-only project. The dependee attribute tells bake to not apply the settings inside the JSON object to the math project, but to the projects that depend on math.

We can now use the math package like this:

{
    "id": "my_app",
    "type": "application",
    "value": {
        "use": ["math"]
    }
}

Where can I find the configuration options for C and C++ projects?

You can find language-specific configuration options in the README of the language driver projects:

For C: https://github.com/SanderMertens/bake/tree/master/drivers/lang/c

For C++: https://github.com/SanderMertens/bake/tree/master/drivers/lang/cpp

What is a driver?

All of the rules and instructions in bake that actually builds code is organized in bake "drivers". Drivers are shared libraries that bake loads when a project needs them. The most common used drivers are "language drivers", which contain all the build instructions for a specific language, like C or C++. Bake automatically loads the language drivers based on the "language" attribute in your project.json, as is specified here:

{
    "id": "my_app",
    "type": "application",
    "value": {
        "language": "c"
    }
}

By default the language is set to "c", so if you do not specify a language, your project will build as a C project.

What does "lang.c" mean? When do I need to specify it?

In some cases you will want to provide configuration options that are specific to a language, like linking with C libraries on your system, or provide additional compiler flags. In that case, you have to tell bake that the configuration you are about to specify is for a specific driver. This is where "lang.c" comes in:

{
    "id": "my_app",
    "type": "application",
    "value": {
        "language": "c"
    },
    "lang.c": {
        "lib": ["m"]
    }
}

The "lang.c" member uniquely identifies the bake driver responsible for building C code, and bake will make all of the attributes inside the object ("lib") available to the driver.

If you want to build a C++ project, instead of using the "lang.c" attribute, you have to use the "lang.cpp" attribute, which identifies the C++ driver:

{
    "id": "my_app",
    "type": "application",
    "value": {
        "language": "c++"
    },
    "lang.cpp": {
        "lib": ["m"]
    }
}

For C++ projects, should I specify cpp or c++ for the language attribute?

You can use either, but for specifying driver-specific configuration you always have to use lang.cpp.

How can I see a list of the available drivers?

The following command will show you a list of the available drivers:

bake list bake.*

Everything except for bake.util is a driver. If you just built bake for the first time, this will only show the "lang.c" and "lang.cpp" drivers.

Can I load more than one driver?

Yes! You can load as many drivers as you want. If you want to add a driver, simply add it to your configuration like this:

{
    "id": "my_app",
    "type": "application",
    "my_custom_driver": { }
}

Are there any example drivers I can use as a template?

Driver documentation is a bit lacking at the moment, but we will eventually address that. In the meantime, you can take a look at the C driver to see what a fully fletched driver looks like:

https://github.com/SanderMertens/bake/tree/master/drivers/lang/c

How do I install bake packages?

Bake relies on git to store packages. To install a package, use the bake clone command with a GitHub repository identifier:

bake clone SanderMertens/example

If your git repository is not hosted on GitHub, simply provide the full git URL:

bake clone https://my_git_server.com/example

Any URL that is accepted by git is accepted by bake.

How does bake find dependencies of cloned projects?

When bake clones a package with dependencies, it will try to also install those dependencies. It does this by taking the git URL specified to bake clone, and replacing the package name with the dependency name. For example, if the https://github.com/SanderMertens/example git repository depends on project foobar, bake would also look for https://github.com/SanderMertens/foobar.

Future versions of bake may provide more intelligent ways to locate packages.

Why use JSON for project configuration?

A number of people have asked me why I used JSON for project configuration. There are two reasons:

• It is a ubiquitous language that everyone understands,
• It has a C parser that can be easily embedded into bake without adding dependencies

A disadvantage of JSON is that while it is fine for trivial configurations, it can get a bit unwieldy once project configurations get more complex. In bake however, you can encapsulate complexity into a configuration-only project, and then include that project as a dependency in your project configuration (example).

Additionally, bake is not like traditional build tools where you specify rules with inputs and outputs in your project configuration. If you want to, for example, add a code generation step to your build, you write a driver for it, and then include the driver in your project configuration.

How can I specify a custom compiler?

The drivers for C & C++ projects by default use gcc/g++ (on Linux) and clang/clang++ (on MacOS). If you want to change the default compiler, you can set the CC (for C) and CXX (for C++) environment variables, as long as the command line options are compatible with gcc. Instead of setting the environment variables manually, you can make them part of a bake environment like this:

bake export CC=clang --env clang_env

To use the environment, and build with clang, you can then invoke bake like this:

bake --env clang_env

To export CC or CXX to the default environment, simply leave out the --env argument.

Where does bake store my binaries?

Bake always stores binaries in the bin/arch-os-config directory of your project. When your project is a public project (this is the default) binaries are also copied to the target bake environment, which by default is $BAKE_HOME/arch-os/config/bin or $BAKE_HOME/bake/arch-os/config/lib. By default, $BAKE_HOME is set to ~/bake.

To prevent a project from being stored in the bake environment, add this to the project.json:

"value": {
    "public": false
}

Usually you do not need to know where binaries are stored, as you can run applications with bake run, and specify dependencies by using their logical name.

How do I do a release build?

By default, binaries are built with the default debug configuration. To build a release configuration, add --cfg release to your bake command. You can add/change configurations in the bake configuration file. See "Configuring Bake" for more details.

How to use different versions of the same package?

Bake does not support having different versions of a package in the same environment. If you want to use different versions of the same package on a machine, you have to use different bake environments. You can do this by setting the BAKE_HOME environment variable. By default, this variable is set to $HOME/bake, but you can override it to any path you want. You can set BAKE_HOME in a new environment called my_env (for example) with this command:

bake export BAKE_HOME=/home/user/my_path --env my_env

To set the variables in this environment, add --env my_env to any bake command, like this:

bake --env my_env

Manual

Introduction

The goal of bake is to bring a level of abstraction to building software that is comparable with npm. Tools like make, cmake and premake abstract away from writing your own compiler commands by hand, but still require users to create their own build system, with proprietary mechanisms for specifying dependencies, build configurations etc.

This makes it difficult to share code between different people and organizations, and is arguably one of the reasons why ecosystems like npm are thriving, while ecosystems for native code are fragmented.

Bake is therefore not just a build tool like make that can automatically generate compiler commands. It is also a build system that specifies how projects are organized and configured. When a project relies on bake, a user does, for example, not need to worry about how to link with it, where to find its include files or whether binaries have been built with incompatible compiler flags.

A secondary goal is to create a zero-dependency build tool that can be easily ported to other platforms. Whereas other build tools exist, like make, premake, rake and gradle, they all rely on their respective ecosystems (unix, lua, ruby, java) which complicates writing platform-independent build configurations. Bake's only dependency is the C runtime.

Creating a new Project

You can create a new bake project with the bake new command. This command has a few options, which lets you create different kinds of projects (see "Project Kinds"). By default, bake creates an "application" project, which is a standard executable. To create a new application project called my_app, run the following command:

bake new my_app

This will create a new directory called my_app with the contents of a basic bake application project. If you want to create a bake package (a shared library), you can simply add --package to the command:

bake new my_pkg --package

When a new project is created, its metadata is also stored in the bake environment. That means that the project is now discoverable by bake, and can be used as dependency of other projects. You can inspect the bake environment with this command:

bake list

Your new project should show up in the list of projects.

Bake lets you create projects with nested identifiers, like foo.bar. This lets you create hierarchies of projects. The . notation is used to denote different elements in the project identifier. To use nested identfiers, simply specify their name with bake new:

bake new foo.bar

This will create a new directory foo-bar. The project will appear as foo.bar when you do bake list.

Building Projects

Bake's primary task is to build the code in your projects, and generate binaries in a reliable and reproducible way. You can simply build a bake project by invoking the bake command:

bake

This will recursively discover and build all bake projects in the current directory. The command is synonymous for running bake with the build action:

bake build

Alternatively you can also specify a directory to build, like so:

bake my_directory
bake build my_directory

Bake has a number of actions, of which the following are related to building your project:

bake build
bake rebuild
bake clean

The build action incrementally builds your project, and will reuse artefacts from previous builds, like object files and binaries. The rebuild action cleans artefacts from previous builds, and is then followed by a regular build. The clean action cleans all build artefacts for the project.

Bake allows you to build for multiple platforms and build configurations from the same source tree, as it stores build artefacts in locations that are platform/configuration specific. When you do a bake rebuild, only the artefacts for the current platform / configuration are cleaned, whereas bake clean cleans artefacts for all platforms / configurations.

Discovery

Bake automatically discovers projects in the provided path, or current directory if no path was specified. It will then order the discovered projects based on their dependencies, so that they are built in the correct order. This removes the need for building makefiles in which you explicitly have to maintain the build order for your projects. Bake uses the information in the use attribute of your project configuration (see Project Configuration).

Bake will not attempt to discover projects in subdirectories of projects if those subdirectories have special meaning. The following directories are skipped, only if they are found inside a bake project directory:

• src
• include
• config
• data
• test
• etc
• lib
• bin
• install
• examples
• .bake_cache

Additionally, bake will skip any directories that start with a ..

Build configurations

Bake lets you build projects with different build configurations, like debug and release. By default, bake has built-in settings for debug and release configurations. You can specify a build configuration with the --cfg flag:

bake my_project --cfg release

The default configuration is debug. The difference between debug and release is that debug disables optimizations and enables debugging code (release adds the -DNDEBUG flag). Furthermore, debug builds add compiler debugging information (like -g in gcc).

Bake never mixes binaries between build configurations. Therefore, if you build a project in release mode, but its dependencies haven't been built in release mode yet, the build will fail.

Recursive builds

To make working across configurations easier, bake lets you do so called "recursive builds". These builds don't just build the current project, but also all dependencies for a project (and dependencies of dependencies, hence recursive builds). When building a project in release mode, but all dependencies have been built in debug mode, you can simply do:

bake my_project --cfg release -r

The -r flag enables recursive building, which will, in addition to the current project, rebuild all dependencies in release mode as well. Recursive builds work for any dependency that is available in the bake environment. Bake keeps track of where the source files of your projects are located on disk, which is how it can start a build for a dependency, even when it is not discoverable from the location where bake was invoked from.

Running Projects

You can run bake projects by using bake run, followed by either a folder or a project id:

bake run foo.bar
bake run my_directory

This only works for application projects (see Project Kinds). Bake will automatically start the executable and monitor its status. Before running the project, bake will first attempt to do a recursive build (see Recursive builds) so that the project and all its dependencies are built and are available for the right configuration. You can specify a configuration just like you would when building:

bake run foo.bar --cfg release

Additionally, bake lets you do interactive builds, which monitor changes from your project, and rebuild the project when a change occurs. To start an interactive build, add the --interactive flag:

bake run foo.bar --interactive

Currently bake does not monitor changes in the source code of dependencies, though it may do so in the future.

Project Kinds

Bake supports different project kinds which are configured in the type property of a project.json file. The project kind determines whether a project is a library or executable, whether a project is installed to a bake environment and whether a project is managed or not. The following table shows an overview of the different project kinds:

Public vs private projects

A public project is a project that is installed to the bake environment. In this environment, bake knows where to find include files, binaries and other project resources. This allows other projects to refer to these resources by the logical project name, and makes specifying dependencies between projects a lot easier.

Private projects are projects that are not installed to a bake environment. Because of this, these projects cannot be located by other projects. Private projects may depend on public projects, but public projects cannot depend on private projects. Binaries of private objects are only stored in the bin folder in the project root.

Project Layout

Each bake project uses the same layout. This makes it very easy to build bake projects, as bake always knows where to find project configuration, include files, source files and so on. A bake project has at least three files:

In addition, a bake project can contain the following optional directories:

Bake will by default build any source file that is in the src directory. If the project is public, files in the include, etc and install folders will be soft-linked to the bake environment on Linux/MacOS, and copied when using Windows.

When bake builds a project, build artefacts are stored in these directories:

Bake stores temporary files in platform- and configuration specific directories, so that you can safely do debug/release builds, and builds for different operating systems from the same source directory.

Project Configuration

A bake project file is located in the root of a project, and must be called project.json. This file contains of an id describing the logical project name, a type describing the kind of project, and a value property which contains properties that customize how the project should be built.

This is a minimal example of a bake project file that builds an shared object. With this configuration, the project will be built with all values set to their defaults.

{
    "id": "my_library",
    "type": "package"
}

This example shows how to specify dependencies and specify additional flags:

{
    "id": "my_application",
    "type": "application",
    "value": {
        "use": ["my_library"]
    },
    "lang.c": {
        "cflags": ["-DHELLO_WORLD"]
    }
}

In this example, if my_library is a project that is discovered by bake, it will be built before my_application.

The following properties are available from the bake configuration and are specified inside the value property:

The cflags attribute is specified inside the lang.c property. This is because cflags is a property specific to the C driver. For documentation on which properties are valid for which drivers, see the driver documentation.

Private dependencies

When projects depend on other projects that require additional library paths or include paths, it may not be desirable to require having these properties propagate to dependees. For example, bar depends on foo, and foo requires adding an include path to the build configuration. Now, helloworld depends on bar, but it does not need to know about foo.

To prevent the foo build settings from propagating to helloworld, bar will need to configure foo as a "private dependency". The following configuration shows how to do this:

{
    "id": "bar",
    "type": "package",
    "value": {
        "use_private": ["foo"]
    }
}

This way, foo is still added as a dependency to bar, but helloworld will not be exposed to foo, nor inherit any of its build settings.

Template Functions

Bake property values may contain calls to template functions, which in many cases allows project configuration files to be more generic or less complex. Additionally, template functions can be used to parameterize bake template projects. Template functions take the following form:

${function_name argument}

They are used like this:

{
    "id": "my_project",
    "type": "package",
    "value": {
        "include": ["${locate include}"]
    }
}

The following functions are currently supported:

The next sections are detailed description of the supported functions:

locate

The locate function allows a project configuration to use any of the project paths in the bake environment. This functionality can also be used programmatically, through the ut_locate function in the bake.util package.

os

The os function can be used to specify platform-specific settings, or use the platform string in a path. The following example demonstrates how it can be used:

{
    "${os linux}": {
        "include": ["includes/linux"]
    }
}

The os function can match both operating system and architecture. The following expressions are all valid:

• x86-linux
• darwin
• x86_64
• x86_64-darwin
• i386

For a full description of the expressions that are supported, see the documentation of ut_os_match.

The os function may be nested:

{
    "${os linux}": {
        "include": ["includes/linux"],
        "${os x86_64}": {
            "lib": ["mylib64"]
        },
        "${os x86}": {
            "lib": ["mylib32"]
        }
    }
}

If no argument is provided to os, it will return the current architecture in the following format:

arch-os

This format is consistent with the platform-specific bin path under which bake stores project binaries (like bin/x86-linux).

language

The language function matches or returns the language of the project.

When an argument is provided, it is matched against the current language:

{
    "${language c}": { 
        "lib": ["my_c_lib"]
    },
    "${language cpp}": {
        "lib": ["my_cpp_lib"]
    }
}

The function accepts both cpp and c++ for C++ projects.

When the argument is ommitted, the current language is returned. This is particularly effective in combination with the dependee attribute, when dependees can be implemented in different languages:

{
    "dependee": {
        "lang.${language}": {

        }
    }
}

id

The id function returns the current project id in various formats. When the function is used without arguments, it returns the project id as it appears in the project.json file:

${id}

To obtain the id in other formats, the following arguments can be passed to the id functions:

Project bundles

Bake bundles let projects specify in which repositories their dependencies are located, and which revision of those dependencies they require. All repositories and their revisions in a bundle are expected to be compatible with each other. This makes it easier to work with projects that have many (open source) dependencies, where the dependencies have their own release cycles and rules for versioning.

If bundles are setup correctly, the following commands can all automatically download and build the correct set of dependencies:

Run directly from a repository:

bake run https://github.com/SanderMertens/example

Clone a bake project and its dependencies:

bake clone https://github.com/SanderMertens/example

Recursively build a project:

bake -r example

This is a simple example of a bundle configuration:

{
    "id": "example",
    "type": "application",
    "value": {
        "use": ["example_library"]
    },
    "bundle": {
        "repositories": {
            "example_library": "https://github.com/SanderMertens/example_library"
        }
    }
}

This example only contains information about where the example_library project can be found. When a user clones, runs or recursively builds the project, bake will automatically clone the example_library project if it is not locally available. Alternatively, this information can be stored in a separate project:

{
    "id": "example_bundle",
    "type": "package",
    "value": {
        "language": "none"
    },
    "bundle": {
        "default-host": "https://github.com/SanderMertens",
        "repositories": {
            "example_library": "SanderMertens/example_library",
            "foo.bar": "SanderMertens/foo-bar",
            "hello.world": "SanderMertens/hello-world"
        }
    }
}

This is a configuration for a project that only contains bundle information. Note how it uses the default-host attribute to shorten the URLs. A bake project can automatically load this bundle when it is built by specifying the bundle in the use-bundle attribute:

{
    "id": "example",
    "type": "application",
    "value": {
        "use-bundle": ["example_bundle"]
    },
    "bundle": {
        "repositories": {
            "example_bundle": "https://github.com/SanderMertens/example_bundle"
        }
    }    
}

Note how the project also adds the bundle project to the repositories list in its bundle section. This is not required, but is recommended so that bake will be able to clone the bundle if example is built in an environment that does not contain example_bundle. If the repository configuration is ommitted and example_bundle cannot be found by bake the project will fail to build.

If a project wants to use a specific branch, commit and/or tag of a repository, it can do so by adding a refs property to the bundle section:

{
    "id": "example_bundle",
    "type": "package",
    "value": {
        "language": "none"
    },
    "bundle": {
        "default-host": "https://github.com/SanderMertens",
        "repositories": {
            "example_library": "SanderMertens/example_library",
            "foo.bar": "SanderMertens/foo-bar",
            "hello.world": "SanderMertens/hello-world"
        },
        "refs": {
            "v1.0": {
                "example_library": {
                    "branch": "master",
                    "tag": "v1.0"
                },
                "foo.bar": {
                    "branch": "master",
                    "tag": "v1.0"
                },
                "hello.world": {
                    "branch": "master",
                    "commit": "52ba2e129a6359f06f3437e7f46b9f466464b495"
                }
            }
        }
    }
}

To use this specific set of revisions, a project should specify its id in the use-bundle property, like so:

{
    "use-bundle": ["example_bundle:v1.0"]
}

The above bundle configuration can be shortened with the default-branch and default-tag properties:

{
    "id": "example_bundle",
    "type": "package",
    "value": {
        "language": "none"
    },
    "bundle": {
        "default-host": "https://github.com/SanderMertens",
        "repositories": {
            "example_library": "SanderMertens/example_library",
            "foo.bar": "SanderMertens/foo-bar",
            "hello.world": "SanderMertens/hello-world"
        },
        "refs": {
            "v1.0": {
                "default-branch": "master",
                "default-tag": "v1.0",
                "hello.world": {
                    "commit": "52ba2e129a6359f06f3437e7f46b9f466464b495"
                }
            }
        }
    }
}

In cases where the revision id is the same as the tag, the tag and default-tag properties can be ommitted, as bake will automatically use this id as the tag when a project has no tag and no commit specified. Additionally, if no branch is specified, master is assumed. Therefore the above configuration is equivalent to this:

{
    "id": "example_bundle",
    "type": "package",
    "value": {
        "language": "none"
    },
    "bundle": {
        "default-host": "https://github.com",
        "repositories": {
            "example_library": "SanderMertens/example_library",
            "foo.bar": "SanderMertens/foo-bar",
            "hello.world": "SanderMertens/hello-world"
        },
        "refs": {
            "v1.0": {
                "hello.world": {
                    "commit": "52ba2e129a6359f06f3437e7f46b9f466464b495"
                }
            }
        }
    }
}

This allows for shorter bundle configurations, where the bundle only needs to capture deviations from what is considered good practice. The exception to this rule is if a set of revisions has the default id. default specifies the set of revisions that are loaded when no bundle id, or default, is specified.

It is possible to add bundles to the bake configuration. This ensures that repositories in the environment are guaranteed to be of the revisions in the configured bundle. If a bundle is configured with bake, and a project is loaded that has a different configuration (repository, branch, tag, commit) for a project, it will fail to build. To add a bundle to bake's configuration, use the bake use command:

bake use example_bundle:v1.0

Attempting to load a project with different settings for any of the projects in the example_bundle bundle will result in an error. For example, if a project specifies project hello.world with tag v1.0, bake will reject it. Additionally, if a project specifies project foo.bar with tag v2.0 bake will also reject the configuration. Even though foo.bar is not explicitly added to v1.0, bake adds it automatically when it loads the bundle configuration.

If a project specifies its repository URL in the repository property, it must match with bundle configurations, or bake will reject the project. For example, trying to load the following project configuration against the previous bundle will fail because the URL does not match:

{
    "id": "foo.bar",
    "type": "library",
    "value": {
        "repository": "https://gitlab.com/foo-bar"
    }
}

Bake will not automatically download dependencies during a build. Dependencies are only automatically downloaded for recursive builds, which are enbled with the -r argument:

bake -r

Templates

Bake lets you create template projects which contain boilerplate code for common types of applications. Template projects look like regular projects, with two exceptions:

• The project type is set to template
• Files may contain template functions that are resolved when instantiating a template

You can create a new template project by specifying the --template flag when using the bake new command:

bake new my_template --template

This creates a new template project which can be instantiated like this:

bake new my_app -t my_template

A template project can be parameterized using bake template functions (see previous chapter), like so:

Using template functions

int main(int argc, char *argv[]) {

    printf("Hello ${id}!");

    return 0;
}

Template functions may occur at any point in your files. Not all files in a project are parsed. Only files with the following extensions are considered by the template parser:

• c, cpp, h, hpp, html, js, css, json, md, sh, bat, lua, python, java, cs, make

Additionally, files with the following filename will be considered:

• Makefile

These lists may be extended with additional extensions and filenames.

Additionally, filenames may also be parameterized with bake template functions. The syntax for doing so is (xxx is a placeholder for parts of the filename):

xxx__<template function>.<file extension>
xxx__<template function>_<template argument>.<file extension>
xxx__<template function>__xxx.<file extension>
xxx__<template function>_<template argument>__xxx.<file extension>

For example, if you want a source file in your template project to have the base name of the instantiated project, you can name it like this:

__id_base.c

This is equivalent to the template function:

${id base}.c

Running templates

If you are developing a new template, you'll often find yourself wanting to instantiate it to test modifications to the template. To make this process easier, bake lets you instantiate templates directly. Simply do:

bake run my_template --template

where my_template is the template name. This will cause bake to automatically instantiate a new temporary project with the template.

Configuring Bake

Bake can be optionally configured with configuration files that specify the environment in which bake should run and the build configuration that should be used. Bake locates a bake configuration file by traveling upwards from the current working directory, and looking for a bake.json file. If multiple files are found, they are applied in reverse order, so that the file that is "closest" to the project takes precedence.

A bake configuration file consists out of an environment and a configuration section. The configuration section contains parameters that are not specific to a project, but influence how code is built. The environment section contains a list of environment variables and their values which are loaded when bake is started.

The bake env command prints the bake environment to the command line in a format that can be used with the export bash command, so that the bake environment can be easily exported to the current shell, like so:

export `bake env`

Bake automatically adds `$BAKE_HOME/bin` to the `PATH` environment variable. This ensures that even when applications (tools) are not installed to a global location, such as `/usr/local/bin`, they can still be directly accessed from a shell when the bake environment is exported.

The following table is a list of the configuration parameters:

It is up to plugins to provide implementations for the above parameters. Not all parameters may be implemented. Refer to the plugin documentation for specifics.

This is an example configuration file:

{
    "configuration":{
        "debug":{
            "symbols":true,
            "debug":true,
            "optimizations":false,
            "coverage":false,
            "strict":false
        },
        "release":{
            "symbols":false,
            "debug":false,
            "optimizations":true,
            "coverage":false,
            "strict":false
        }
    },
    "environment":{
        "default":{
            "PATH": ["/my/path"],
            "FOO": "Some value"
        }
    }
}

Note that environment variables configured as a JSON array (as shown with the PATH variable), are appended to their current value. Elements in the array are separated by a : or ;, depending on the platform.

With the --cfg and --env flags the respective configuration or environment can be selected.

Installing Miscellaneous Files

Files in the install and etc directories are automatically copied to the project-specific locations in the bake environment, so they can be accessed from anywhere (see below). The install folder installs files directly to a location where other projects can also access it, whereas files in etc install to the project-specific location in the bake environment. For example, the following files:

my_app
 |
 |-- etc
 |    | index.html
 |    + style.css
 |
 +-- install/etc
      | image.jpg
      + manual.pdf

would be installed to the following locations:

$BAKE_HOME/platform/config/etc/my_app/index.html
$BAKE_HOME/platform/config/etc/my_app/style.html
$BAKE_HOME/platform/config/etc/image.jpg
$BAKE_HOME/platform/config/etc/manual.pdf

Bake allows projects to differentiate between different platforms when installing files from the etc and install directories. This can be useful when for example distributing binaries for different architectures and operating systems. By default, all files from these directories installed. However, bake will look for subdirectories that match the platform string. Files in those directories will only be installed to that platform. For example, consider the following tree:

my_app
 |
 +-- etc
      |-- Linux/linux_manual.html
      |
      |-- Darwin/darwin_manual.html
      |
      |-- Linux-i686/libmy_binary.so
      |
      |-- Linux-x86_64/libmy_binary.so
      |
      |-- Linux-armv7l/libmy_binary.so
      |
      +-- Darwin-x86_64/libmy_binary.so

Here, only the libmy_binary.so that is in the directory that matches the platform string will be installed.

The platform string is case independent. It allows for a number of different notations. For example, both x86-linux and linux-x86 are allowed. In addition, projects can also just specify the operating system name, in which case the file will be installed to all architectures, as long as the operating system matches the directory name.

To see the exact matching of the platform string, see the implementation of ut_os_match in bake.util.

Integrating Non-Bake Projects

It is not uncommon that a project needs to include or link with a project that itself was not built with bake. Often such projects require that you specify custom include paths, library paths, and link with specific libraries. When you have many projects that depend on such an external project, it can become tedious having to repeat these properties in every project.json.

Wrapping external projects

Bake allows you to create a project that "wraps around" the external project, in which you describe this build configuration once. Once done, projects can simply add this project as a dependency, and the properties will be automatically added.

Consider a project that requires dependees to add /usr/local/include/foobar to their include path, that need to link with libfoobar.so, which is a library located in /usr/local/lib/foobar. For such a project, this is what the project.json could look like:

{
    "id": "foobar",
    "type": "package",
    "value": {
        "language": "none",
    },
    "dependee": {
        "lang.c": {
            "include": ["/usr/local/include/foobar"],
            "libpath": ["/usr/local/lib/foobar"],
            "lib": ["foobar"]
        }
    }
}

Lets go over each property. The first two specify that the project is a bake package with the id foobar. Because bake packages are public by default, we do not have to explicitly add "public": true to the project.json file to ensure that other packages can find this dependency.

The next property is "language": "none". This ensures that when ran, bake does not try to build anything for the package. It also ensures that when specifying this package as a dependency, bake will not try to link with any binaries.

The dependee property is where the properties are specified for projects that depend on foobar. For every dependee project, bake will add the include, libpath and lib properties to those of the dependee configuration. Therefore, a project that depends on foobar, simply can do:

{
    "id": "my_app",
    "type": "application",
    "value": {
        "use": ["foobar"]
    }
}

... and bake will take care of the rest.

Include external files from bake environment

When include files or libraries are not installed to a common system location, you can use bake to make these files available to dependee projects as well. Suppose we have a project called helloworld, which is shipped as a library libhelloworld.so, and two header files called helloworld.h and helloworld_types.h. We could include these files in a bake project, like this:

helloworld
 |
 +-- include
      |-- helloworld.h
      +-- helloworld_types.h

That way, when running bake, they are installed to the bake environment and are available to other projects. However, there is a potential problem with this approach. The helloworld.h file might for example depend on helloworld_types.h, simply by doing:

#include <helloworld_types.h>

Because bake does not automatically add project-specific include paths to the include path when compiling (to prevent name-clashes), that include file will not be found by the compiler. Therefore, an additional property is required that includes the correct path from the bake environment. Instead of hard-coding that path, bake provides a convenient way to do this:

{
    "id": "helloworld",
    "type": "package",
    "value": {
        "language": "none"
    },
    "dependee": {
        "include": ["${locate include}"],
    }
}

The ${locate include} part of the include path will be substituted by the project-specific include folder when the project.json is parsed.

Link external files from global environment

When a project needs to link with an external binary, one option is to install it to a global location. The bake equivalent is to install it to the lib directory in the bake environment. That way, the library will be installed to $BAKE_TARGET/lib.

To install the library to this location, it needs to be added to the project folder. Add the library to this location:

helloworld
 |
 +-- install/lib/libhelloworld.so

The project configuration now needs to be configured so that dependee projects link with the library:

{
    "id": "helloworld",
    "type": "package",
    "value": {
        "language": "none"
    },
    "dependee": {
        "lib": ["helloworld"]
    }
}

Link external files from bake environment

In some cases it may be desirable to link with a library without copying it to a public location, like /usr/local/lib. In that case, the library can also be copied to the bake environment, in the same way we did for the the include file. First, the library needs to be installed to a project specific location. This can be accomplished by storing it in the lib directory in the project:

helloworld
 |
 +-- lib/libhelloworld.so

Now the dependee section in the project.json file needs to be adjusted so that projects depending on helloworld will link with the correct library. To avoid having to rely on LD_LIBRARY_PATH, or having to specify a full path in the project configuration, we can use the link property in combination with a bake template function. The full configuration now looks like this:

{
    "id": "helloworld",
    "type": "package",
    "value": {
        "language": "none"
    },
    "dependee": {
        "link": ["${locate lib}/helloworld"]
    }
}

Bake will automatically expand the expression in the link path so that it contains the lib and .so prefixes. The link property will cause bake to link with the library using a hard-coded path, just like other project dependencies.

Some prebuilt libraries cannot be linked with using a hard-coded path. Typically libraries that have been compiled with "--soname" may cause problems, as the hardcoded path will be overwritten at link-time with the name provided to "--soname", which will cause the runtime linker to fail.

Deploying to multiple operating systems

When deploying binaries, a project likely needs to include versions for multiple operating systems. This can be done by storing the binaries in a directory that matches the target operating system. The following tree shows the helloworld project with two binaries, for Linux and MacOS:

helloworld
 |
 +-- lib
      |-- Linux-i686/libhelloworld.so
      |-- Linux-x86_64/libhelloworld.so
      +-- Darwin-x86_64/libhelloworld.dylib

Bake will automatically install and link with the binary that corresponds with the target platform. Note that bake also automatically tries to find libraries that end in dylib on MacOS.

Putting it all together

The following tree and project file show a non-bake project where the include files and binary file are installed to the bake environment, and the project supports multiple operating systems.

Files:

helloworld
 |-- project.json
 |
 |-- include
 |    |-- helloworld.h
 |    +-- helloworld_types.h
 |    
 +-- lib
      |-- Linux-i686/libhelloworld.so
      |-- Linux-x86_64/libhelloworld.so
      +-- Darwin-x86_64/libhelloworld.dylib

project.json:

{
    "id": "helloworld",
    "type": "package",
    "value": {
        "language": "none"
    },
    "dependee": {
        "include": ["${locate include}"],
        "link": ["${locate lib}/helloworld"]
    }
}

The Bake Environment

Bake installs projects to the "bake environment". The bake environment is located in a location specified by the BAKE_HOME environment variable, and contains all the metadata and binaries for public projects, miscellaneous files and templates. By default, the bake environment is located in ~/bake. A different location can be specified by changing the value of the BAKE_HOME environment variable.

Projects in the bake environment can be automatically discovered and linked with by using their logical name. Here is an example of two public projects, one application and one package, where the application depends on the package:

{
    "id": "my_lib",
    "type": "package"
}

{
    "id": "my_app",
    "type": "application",
    "value": {
        "use": ["my_lib"]
    }
}

Note that neither project configuration specifies where they are built to, or where to find the my_lib project. This is automatically managed by the bake environment.

To get an overview of the projects stored in the bake environment, you can do:

bake list

The bake environment stores platform-specific data (such as binaries) in a location that is specific to a platform and build configuration. For example, if you are doing a debug build on Windows, you will find a directory in $BAKE_HOME called:

x64-Windows/debug

During a build, this directory is accessible through the BAKE_TARGET environment variable. This will contain all binaries (in bin and lib directories) for projects built for this platform and build configuration. The bake list command shows which projects have been built for which build configuration.

Environment Variables

Bake uses the following environment variables:

Command line usage

The following is the output of bake --help

Usage: bake [options] <command> <path>

Options:
  -h,--help                    Display this usage information
  -v,--version                 Display version information

  --cfg <configuration>        Specify configuration id
  --env <environment>          Specify environment id
  --strict                     Manually enable strict compiler options
  --optimize                   Manually enable compiler optimizations

  --package                    Set the project type to package
  --template                   Set the project type to template
  --test                       Create a test project
  --to-env                     Clone projects to the bake environment source path (use with clone)
  --always-clone               Clone dependencies even if found in the bake environment (use with clone)

  --id <project id>            Specify a project id
  --type <package|template>    Specify a project type (default = "application")
  --language <language>        Specify a language for project (default = "c")
  --artefact <binary>          Specify a binary file for project
  -i,--includes <include path> Specify an include path for project
  --private                    Specify a project to be private (not discoverable)

  -a,--args [arguments]        Pass arguments to application (use with run)
  --interactive                Rebuild project when files change (use with run)
  --run-prefix                 Specify prefix command for run
  --test-prefix                Specify prefix command for tests run by test
  -r,--recursive               Recursively build all dependencies of discovered projects
  -t [id]                      Specify template for new project
  -o [path]                    Specify output directory for new projects

  --show-repositories          List loaded repositories (use with list)

  -v,--verbosity <kind>        Set verbosity level (DEBUG, TRACE, OK, INFO, WARNING, ERROR, CRITICAL)
  --trace                      Set verbosity to TRACE
  --debug                      Set verbosity to DEBUG (highest verbosity)

Commands:
  new [path]                   Initialize new bake project
  run [path|project id]        Build & run project
  build [path]                 Build a project (default command)
  rebuild [path]               Clean and build a project
  clean [path]                 Clean a project
  test [path]                  Run tests of project
  coverage [path]              Run coverage analysis for project
  cleanup                      Cleanup bake environment by removing dead or invalid projects
  reset                        Resets bake environment to initial state, save for bake configuration
  publish <patch|minor|major>  Publish new project version
  install <project id>         Install project to bake environment (repository must be known)
  uninstall [project id]       Remove project from bake environment
  clone <git url>              Clone and build git repository and dependencies
  use <project:bundle>         Configure the environment to use specified bundle
  update [project id]          Update an installed package or application
  foreach <cmd>                Run command for each discovered project

  env                          Echo bake environment
  upgrade                      Upgrade to new bake version
  export <NAME>=|+=<VALUE>     Add variable to bake environment

  info <package id>            Display info on a project in the bake environment
  list [filter]                List packages in bake environment

Examples:
  bake                         Build all projects discovered in current directory
  bake my_app                  Build all projects discovered in my_app directory
  bake new                     Create new application project in current directory
  bake new my_app              Create new application project in directory my_app
  bake new my_lib --package    Create new package project in directory my_lib
  bake new my_tmpl --template  Create new template project in directory my_tmpl
  bake new game -t sdl2.basic  Create new project from the sdl2.basic template
  bake run my_app -a hello     Run my_app project, pass 'hello' as argument
  bake publish major           Increase major project version, create git tag
  bake info foo.bar            Show information about package foo.bar
  bake list foo.*              List all packages that start with foo.

Writing Drivers

Bake has a plugin architecture, where a plugin describes how code should be built for a particular language. Bake plugins are essentially parameterized makefiles, with the only difference that they are written in C, and that they use the bake build engine. Plugins allow you to define how projects should be built once, and then reuse it for every project. Plugins can be created for any language.

The bake build engine has a design that is similar to other build engines in that it uses rules that depend on other rules. Rules have rule-actions, which get executed when a rule is outdated. Whether a rule is outdated or not is determined by comparing timestamps of the rule dependencies with the timestamps of the rule output.

Rules are written in their respective language plugins in C. A simple set of rules that builds a binary from a set of source files would look like this:

driver->pattern("SOURCES", "//*.c|*.cpp|*.cxx");
driver->rule("objects", "$SOURCES", driver->target_map(src_to_obj), compile_src);
driver->rule("ARTEFACT", "$objects", driver->target_pattern(NULL), link_binary);

Patterns create a label for a pattern (using the ut_expr syntax). Rules are patterns that have dependencies and actions. The syntax for a rule is:

driver->rule(<id>, <dependencies>, <function to map target to output>, <action>);

Each plugin must have a bakemain entrypoint. This function is called when the plugin is loaded, and must specify the rules and patterns.

Authors

• Sander Mertens - Initial work

Legal stuff

Bake is licensed under the GPL3.0 license. The bake runtime (all code under the util directory) is licensed under the MIT license.