IncludeOS is a minimal unikernel operating system for C++ services running in the cloud and on real hardware. Starting a program with #include <os> will include a tiny operating system into your service during link-time.

IncludeOS is free software, with "no warranties or restrictions of any kind".

Note: IncludeOS is under active development. The public API should not be considered stable.

Key features

• Extreme memory footprint: A tiny bootable C++17 hello world is currently 900 KB and needs only 5 MB RAM on x86_64.
• KVM, VirtualBox and VMWare support with full virtualization, using x86 hardware virtualization, available on most modern x86 CPUs. IncludeOS will run on any x86 hardware platform, even on a physical x86 computer, given appropriate drivers. Officially, we develop for- and test on Linux KVM, and VMWare ESXi/Fusion which means that you can run your IncludeOS service on Linux, Microsoft Windows and macOS, as well as on cloud providers such as Google Compute Engine, OpenStack and VMWare vcloud.
• Instant boot: IncludeOS on Qemu/kvm boots in about 300ms but IBM Research has also integrated IncludeOS with Solo5/uKVM, providing boot times as low as 10 milliseconds.
• Modern C++ support
  • Full C++11/14/17 language support with clang 6 and later.
  • Standard C++ library (STL) libc++ from LLVM.
  • Exceptions and stack unwinding (currently using libgcc).
• Standard C library using musl libc.
• Virtio and vmxnet3 Network drivers with DMA. Virtio provides a highly efficient and widely supported I/O virtualization. vmxnet3 is the VMWare equivalent.
• A highly modular TCP/IP-stack.

A longer list of features and limitations can be found on our documentation site.

Contents

• Getting started
  • Dependencies
  • Hello world
  • Getting started developing packages
• Managing dependencies
• Building with IncludeOS in editable mode
• Contributing to IncludeOS
• C++ Guidelines
• Security contact

Getting started

Dependencies

For building IncludeOS services you will need:

• The conan package manager (1.13.1 or newer)
• cmake, make, nasm (x86/x86_64 only)
• clang, or alternatively gcc on linux. Prebuilt packages are available for clang 6.0 and gcc 7.3.

To boot VMs locally with our tooling you will also need:

• qemu
• python3 packages: psutil, jsonschema

The following command will configure conan to use our build profiles and remote repositories. (Note: this overwrites any existing conan configuration. Set CONAN_USER_HOME to create a separate conan home folder for testing.)

$ conan config install https://github.com/includeos/conan_config.git

Note: If you prefer to set up conan manually the full configuration can be found in the conan_config repository. You can browse our prebuilt conan packages in bintray.com/includeos.

Ubuntu

$ apt-get install python3-pip python3-dev git cmake clang-6.0 gcc nasm make qemu
$ pip3 install setuptools wheel conan psutil jsonschema
$ conan config install https://github.com/includeos/conan_config.git

macOS

If you have homebrew you can use our brew tap to install the dependencies.

$ brew tap includeos/includeos
$ brew install includeos
$ conan config install https://github.com/includeos/conan_config.git

Vagrant

If you want to use a Vagrant box to explore IncludeOS and contribute to IncludeOS development, you can read the getting started with Vagrant. See etc/vagrant.md

Hello World

First select an appropriate conan profile for the target you want to boot on. conan profile list will show the profiles available, including the ones installed in the previous step. When developing for the machine you're currently on, Linux users can typically use clang-6.0-linux-x86_64, and MacOS users can use clang-6.0-macos-x86_64. You can also make your own.

The following steps let you build and boot the IncludeOS hello world example.

$ git clone https://github.com/includeos/hello_world.git
$ mkdir your_build_dir && cd "$_"
$ conan install ../hello_world -pr <your_conan_profile>
$ source activate.sh
$ cmake ../hello_world
$ cmake --build .
$ boot hello

You can use the hello world repo as a starting point for developing your own IncludeOS services. For more advanced examples see the examples repo or the integration tests (under ./IncludeOS/test/*/integration).

Once you're done $ source deactivate.sh will reset the environment to its previous state.

Getting started developing packages

If you are interested in editing/building our dependency packages on your own, you can checkout our repositories at includeos/. Some of our tools and libraries are listed below under Tools and Libraries. You can find the external dependency recipes at includeos/conan. Currently we build with clang-6 and gcc-7.3.0 compiler toolchains. It is expected that these are already installed on your system (see Dependencies for details).

Managing dependencies

Prebuilt conan packages are uploaded to our bintray repository.

We upload to two channels:

• stable: this channel has all the stable packages.
• latest: this channel will have the latest packages in development/test phase (including stable releases).

Note: We only guarantee that the latest 10 packages are kept in the latest channel. All stable packages will be kept in the stable channel unless proven unsafe. One suggested workaround is to copy packages into your own repository.

To set up our remote, we recommend following the steps listed in Dependencies.

Search

If you want to check if a package exists you can search for it with conan search. To list all the available packages on our remote includeos, you can use:

$ conan search -r includeos

This should list all the packages we have uploaded to the includeos repository.

To find all the stable versions uploaded for a particular package:

$ conan search -r includeos '<package_name>/*@includeos/stable'

Prebuilt profiles

The profiles we are using for development can be found in the includeos/conan_config repository under conan_config/profiles/.

The target profiles we have verified are the following:

• clang-6.0-linux-x86
• clang-6.0-linux-x86_64
• gcc-7.3.0-linux-x86_64
• clang-6.0-macos-x86_64

These profiles should have prebuilt packages available and are tested in CI. If you create a custom profile (or use a different profile provided by us) the dependencies may have to be rebuilt locally.

Building with IncludeOS in editable mode

If you are a kernel developer or are simple interested in fiddling with our kernel code, you can use the includeos-package in editable mode. When you rebuild the package will then automatically be updated so it can be used by other packages locally. This is useful when working with several interconnected components and you would like to test your changes across several libraries.

You can read more about how editable mode works in the Conan documentation.

Below we have written down a few steps to get you started with editable packages and IncludeOS.

Note: Currently this is an experimental feature on conan version 1.13 and they have mentioned that for future releases the feature is subject to breaking changes.

Start by cloning the IncludeOS source code and create a build folder. You have to edit etc/layout.txt in the source code to point to the build folder you created, by updating the build_dir variable.

The layout will look similar to the example below. You only have to update build_dir.

  {% set simple=true%}

  {% set build_dir='build' %}

  {% if simple==false %}
  {% set arch=settings.arch|string %}
  {% set platform=options.platform|string %}
  {% set build_dir=build_dir+'/'+arch+'/'+platform %}
  {% endif %}

  [bindirs]
  {#This is so that we can find os.cmake after including conanbuildinfo.cmake #}
  cmake

  [includedirs]
  {#This is to ensure the include directory in conanbuildinfo.cmake includes our API#}
  api

  [libdirs]
  {#This is so that we can find our libraries #}
  {{ build_dir }}/plugins
  {{ build_dir }}/drivers
  {{ build_dir }}/lib
  {{ build_dir }}/platform

  [resdirs]
  {#This is so that we can find ldscript and search for drivers plugins etc#}
  {{ build_dir }}

Note: in the non simple form it is possible to have multiple build folders from the same source which allows multiple architectures and configurations to be tested from the same source however the complexity increases

You should now be able to set the package in editable mode. The following command will add the package as editable based on the specified layout. We inspect the package to get the version, as this has to match exactly.

$ conan editable add . includeos/$(conan inspect -a version . | cut -d " " -f 2)@includeos/latest --layout=etc/layout.txt

The package is now in editable mode and any dependencies of IncludeOS will pick this IncludeOS package from your local cache.

Here is an example on how it looks when its pulled into cache as editable:

$ conan editable list
  includeos/0.14.1-1052@includeos/latest
    Path: ~/git/IncludeOS
    Layout: ~/git/IncludeOS/etc/layout.txt

We are now ready to build the package. Assuming the build-folder is called build under the includeos source directory the following is enough.

$ cd [includeos source root]
$ conan install -if build . -pr <conan_profile> (-o options like platform=nano etc)
$ conan build -bf build .

After making changes to the code you can rebuild the package with

$ cd build && make
   or
$ cmake build --build

Once you have made your changes and the code is finalized you should verify that the conan package still builds. Remove the editable and do a conan create on the package:

$ conan editable remove includeos/<version>@includeos/test
$ conan create <source_path> includeos/latest -pr <conan_profile>

Libraries and tools

We have moved the libraries and tools created by IncludeOS outside the includeos-repository. You can now find them all in their own repositories inside the IncludeOS organization.

To build the libraries and tools, see build instructions in each repository. Typically, the instructions will be in the form:

$ git clone https://github.com/includeos/mana.git
$ cd mana
$ conan create . includeos/latest -pr clang-6.0-linux-x86_64

Below is a list of some of our Libraries/Tools:

• Vmbuild - Vmbuild is an utility for building the IncludeOS virtual machines.

• Vmrunner - Vmrunner is a utility developed for booting IncludeOS binaries.

• Mana - Mana is a web application framework which is used to build a IncludeOS webserver. We have an example named acorn which demonstrates mana's potential.

• Microlb - Microlb is a library written for building the IncludeOS load balancer. We have an example named microlb which demonstrates our load balancer.

• Diskbuilder - Diskbuilder is a tool used for building disks for IncludeOS.

• NaCl - NaCl is the configuration language tool we have tailored for IncludeOS to allow users to configure various network settings such as firewall rules, vlans, ip configurations etc.

Contributing to IncludeOS

IncludeOS is being developed on GitHub. Create your own fork, send us a pull request, and chat with us on Slack. Please read the Guidelines for Contributing to IncludeOS.

C++ Guidelines

We want to adhere as much as possible to the ISO C++ Core Guidelines. When you find code in IncludeOS which doesn't adhere, please let us know in the issue tracker - or even better, fix it in your own fork and send us a pull-request.

Security contact

If you discover a security issue in IncludeOS please avoid the public issue tracker. Instead send an email to [email protected]. For more information and encryption please refer to the documentation.