Toit programming language

This repository contains the Toit language implementation. It consists of the compiler, virtual machine, and standard libraries that together enable Toit programs to run on an ESP32.

Jaguar: Live reloading for the ESP32

You can use Toit to develop, update, and restart your ESP32 applications in less than two seconds through Jaguar. Jaguar is a small Toit program that runs on your ESP32. It uses the capabilities of the Toit virtual machine to let you update and restart your ESP32 applications written in Toit over WiFi whenever your source files change. Once set up, it is as easy as:

jag watch examples/hello.toit

It is also straightforward to install extra drivers and services that can extend the core functionality of your device. Add automatic NTP-based time synchronization without having to write a single line of code:

jag container install ntp examples/ntp/ntp.toit

You can watch a short video that shows how you can experience Jaguar on your ESP32 in less two minutes:

Community

Use this invite to join our Discord server, and follow the development and get help. We're eager to hear of your experience building with Toit. The Discord chat is publicly accessible through our Linen.

We also use GitHub Discussions to discuss and learn.

We follow a code of conduct in all our community interactions.

References

The Toit language is the foundation for the Toit platform that brings robust serviceability to your ESP32-based devices. You can read more about the language and the standard libraries in the platform documentation:

Contributing

We welcome and value your open source contributions to the language implementation and the broader ecosystem. Building or porting drivers to the Toit language is a great place to start. Read about how to get started building I2C-based drivers and get ready to publish your new driver to the package registry.

If you're interested in pitching in, we could use your help with these drivers and more!

Licenses

The Toit compiler, the virtual machine, and all the supporting infrastructure is licensed under the LGPL-2.1 license. The standard libraries contained in the lib/ directory are licensed under the MIT license. The examples contained in the examples/ directory are licensed under the 0BSD license.

Certain subdirectories are under their own open source licenses, detailed in those directories and the files they contain. These subdirectories are:

The subdirectory lib/font/matthew_welch/
Every subdirectory under packages/
Every subdirectory under lib/font/x11_100dpi/
Every subdirectory under src/compiler/third_party/
Every subdirectory under src/third_party/
Every subdirectory under third_party/

Installation

The instructions in this section don't cover the IDE integration. Follow the instructions below to set up Toit support for your editor.

Arch Linux

For Arch Linux (or variants such as Manjaro) use your favorite AUR helper to install the toit or toit-git package.

For example:

yay -S toit

Other platforms

We're actively working on simple installation steps for other platforms. For now, please use the build instructions below or download the .tar.gz files from the release page.

Building

Dependencies

Build system

Linux and Mac

To build Toit and its dependencies the build host requires:

GNU Make
CMake >= 3.13.3
Ninja
GCC
Go >= 1.19
python-is-python3: on Ubuntu machines
glibc-tools: optional and only available on newer Ubuntus

If you are using a Linux distribution with apt capabilities, you can issue the following command to install these:

sudo apt install build-essential cmake ninja-build golang

You can then build Toit by running the following commands in a checkout of this repository:

git submodule update --init --recursive
make

For builds targeting ESP32 hardware additional requirements might be in effect depending on the build host's architecture, see paragraph ESP32 tools.

For builds targeting RISC-V, ARM32, or ARM64 hardware, see the Other platforms README.

Windows

If you are using Windows you can use Chocolatey to install the required dependencies.

After installing Chocolatey, you can install the required dependencies by running the following command in an elevated shell (usually the same you just used to install Chocolatey):

choco install git ninja mingw make golang ccache
choco install cmake.install --installargs '"ADD_CMAKE_TO_PATH=System"'

After that you can use the bash that comes with Git ('git-bash') and compile Toit in a checkout of this repository by running the following commands:

git submodule update --init --recursive
make

ESP-IDF

The Toit VM has a requirement for the Espressif IoT Development Framework, both for Linux and ESP32 builds (for Linux it's for the Mbed TLS implementation).

We recommend you use Toitware's ESP-IDF fork that comes with a few changes:

Custom malloc implementation
Allocation-fixes for UART, etc.
LWIP fixes

The fork's repository has been added as a submodule reference to this repository, so doing a recursive submodule init & update will establish everything nedded:

git submodule update --init --recursive

If the submodule update step fails with:

Submodule path 'esp-idf/components/coap/libcoap': checked out '98954eb30a2e728e172a6cd29430ae5bc999b585'
fatal: remote error: want 7f8c86e501e690301630029fa9bae22424adf618 not valid
Fetched in submodule path 'esp-idf/components/coap/libcoap/ext/tinydtls', but it did not contain 7f8c86e501e690301630029fa9bae22424adf618. Direct fetching of that commit failed.

try following the steps outlined here. It is an issue in the upstream ESP-IDF repository caused by the tinydtls component having changed its remote URL.

To use the offical ESP-IDF, or any other variation, you need to add the Toit specific patches first.

Then make sure it is available in your file system and point IDF_PATH to its path instead before building.

export IDF_PATH=<A_DIFFERENT_ESP_IDF>

ESP32 tools

If you want to build an image for the ESP32, install the ESP32 tools.

On Linux:

$IDF_PATH/install.sh

The default location of $IDF_PATH is under ./third_party/esp-idf

For other platforms, see Espressif's documentation.

Remember to update your environment variables:

source $IDF_PATH/export.sh

Build for host machine

Make sure the required build tools are installed as described in dependency sections ESP-IDF and Build system above.

Then run the following commands at the root of your checkout.

make all

NOTE

These instructions have been tested on Linux and macOS.

Windows support is still preliminary, and the build instructions may differ for Windows. Let us know on the discussions forum how we can improve this README.

This builds the Toit VM, the compiler, the language server and the package manager.

You should then be able to execute a toit file:

build/host/sdk/bin/toit.run examples/hello.toit

The package manager is found at build/host/sdk/bin/toit.pkg:

build/host/sdk/bin/toit.pkg init --project-root=<some-directory>
build/host/sdk/bin/toit.pkg install --project-root=<some-directory> <package-id>

Debugging

See https://github.com/toitlang/toit/wiki/Debugging.

IDE integration

Toit has a VS Code extension. You can either use the published extension or build it yourself from the sources.

In the VS Code extension (version 1.3.7+) set the toitLanguageServer.command setting to ["PATH_TO_SDK/bin/toit.lsp", "--toitc=PATH_TO_SDK/bin/toit.compile"], where PATH_TO_SDK is the path to your build/host/sdk/ folder in the Toit repository.

This makes the extension use the language server that was compiled in the build step.

Other IDEs

The Toit language server is independent of VSCode and can be used with other IDEs. It can be started with:

build/host/sdk/bin/toit.lsp --toitc=build/host/sdk/bin/toit.compile

See the instructions of your IDE on how to integrate the language server.

There are syntax highlighters for VIM and CodeMirror in the ide-tools repository.

Build for ESP32

Make sure the environment variables for the ESP32 tools are set, as described in the dependencies section. Typically this consists of running the following command:

# On Linux and Mac OS X:
third_party/esp-idf/install.sh

# On Windows:
third_party\esp-idf\install.bat

Build firmware that can be flashed onto your ESP32 device. The firmware is generated in build/esp32/firmware.envelope:

make esp32

If you want to flash the generated firmware on your device, you can use the firmware too. Internally, the firmware tool calls out to esptool so you need to install that one first. You can also set the environment variable ESPTOOL_PATH to point to a valid esptool (for example the one in the shipped esp-idf: export ESPTOOL_PATH=$PWD/third_party/esp-idf/components/esptool_py/esptool/esptool.py). Assuming your device is connected through /dev/ttyUSB0 you can then flash a device as follows:

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    flash --port /dev/ttyUSB0 --baud 921600

By default, the image boots up but does not run any application code. You can use your own entry point by installing it into the firmware envelope before flashing:

build/host/sdk/bin/toit.compile -w hello.snapshot examples/hello.toit
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install hello hello.snapshot
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    flash --port /dev/ttyUSB0 --baud 921600

Adding multiple containers

You can add more containers before you flash, so you firmware envelope can have any number of containers. Be aware that adding the NTP example below requires you to configure the WiFi on the ESP32 when you flash.

build/host/sdk/bin/toit.compile -w hello.snapshot examples/hello.toit
build/host/sdk/bin/toit.compile -w ntp.snapshot examples/ntp/ntp.toit

# Typically we set the output envelope the first time we change it.
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    -o custom.envelope \
    container install hello hello.snapshot
build/host/sdk/tools/firmware -e custom.envelope \
    container install ntp ntp.snapshot

You can list the containers in a given firmware envelope:

build/host/sdk/tools/firmware -e custom.envelope container list

The listing shows the containers that are installed.

system:
  Kind: snapshot
  Id: bf14aa94-4b7e-3ddd-94a9-a22f5d1ec92c
  Size: 242104
  Flags:
    - trigger=boot
    - critical
hello:
  Kind: snapshot
  Id: adc2babc-d89a-2301-98a1-3a1dfe34f144
  Size: 141852
  Flags:
    - trigger=boot
ntp:
  Kind: snapshot
  Id: 8d8ff2f0-3c51-13e8-6876-84a70fa359b5
  Size: 187330
  Flags:
    - trigger=boot

You can use the --output-format=json flag to get the output in JSON format.

Adding container assets

Containers have associated assets that they can access at runtime. Add the following code to a file named assets.toit:

import system.assets

main:
  print assets.decode

If you run this on an ESP32, you'll get an empty map printed becase you haven't associated any assets with the container that holds the code.

To associate assets with the container, we first construct an encoded assets file and add this README.md file to it.

build/host/sdk/tools/assets -e encoded.assets create
build/host/sdk/tools/assets -e encoded.assets add readme README.md

Now we can add the encoded.assets to the assets container at install time:

build/host/sdk/bin/toit.compile -w assets.snapshot assets.toit
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install assets assets.snapshot \
    --assets=encoded.assets

If you update the source code in assets.toit slightly, the printed information will be more digestible:

import system.assets

main:
  readme := assets.decode["readme"]
  // Guard against splitting a unicode character by
  // making this non-throwing.
  print readme[0..80].to_string_non_throwing

You'll need to reinstall the container after this by recompiling the assets.toit file to assets.snapshot and re-running:

build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    container install assets assets.snapshot \
    --assets=encoded.assets

Configuring WiFi for the ESP32

You can easily configure the ESP32's builtin WiFi passing it as configuration when you flash:

echo '{ "wifi": { "wifi.ssid": "myssid", "wifi.password": "mypassword" } }' > wifi.json
build/host/sdk/tools/firmware -e build/esp32/firmware.envelope \
    flash --config wifi.json \
    --port /dev/ttyUSB0 --baud 921600

This allows the WiFi to automatically start up when a network interface is opened.