• Stars
    star
    887
  • Rank 49,548 (Top 2 %)
  • Language
    Rust
  • License
    Apache License 2.0
  • Created about 3 years ago
  • Updated 6 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A language binding generator for WebAssembly interface types

wit-bindgen

Guest language bindings generator for WIT and the Component Model

A Bytecode Alliance project

build status supported rustc stable

About

Note: this project is still relatively young and active development with large changes is still happening. If you're considering depending on this at this time it's recommended to reach out to the authors on zulip and get more information first.

This project is a suite of bindings generators for languages that are compiled to WebAssembly and use the component model. Bindings are described with *.wit files which specify imports, exports, and facilitate reuse between bindings definitions.

The wit-bindgen repository is currently focused on guest programs which are those compiled to WebAssembly. Executing a component in a host is not managed in this repository, and some options of how to do so are described below.

WIT as an IDL

The wit-bindgen project extensively uses WIT definitions to describe imports and exports. The items supported by WIT directly map to the component model which allows core WebAssembly binaries produced by native compilers to be transformed into a component. All imports into a WebAssembly binary and all exports must be described with WIT. An example file looks like:

package example:host

world host {
  import print: func(msg: string)

  export run: func()
}

This describes a "world" which describes both imports and exports that the WebAssembly component will have available. In this case the host will provide a print function and the component itself will provide a run function.

Functionality in WIT can also be organized into interfaces:

package example:my-game

interface my-plugin-api {
  record coord {
    x: u32,
    y: u32,
  }

  get-position: func() -> coord
  set-position: func(pos: coord)

  record monster {
    name: string,
    hp: u32,
    pos: coord,
  }

  monsters: func() -> list<monster>
}

world my-game {
  import print: func(msg: string)
  import plugin: self.my-plugin-api

  export run: func()
}

Here the my-plugin-api interface encapsulates a group of functions, types, etc. This can then be imported wholesale into the my-game world via the plugin namespace. The structure of a WIT document and world will affect the generated bindings per-language.

For more information about WIT and its syntax see the upstream description of WIT.

Creating a Component

The end-goal of wit-bindgen is to facilitate creation of a component. Once a component is created it can then be handed off to any one of a number of host runtimes for execution. Creating a component is not supported natively by any language today, however, so wit-bindgen is only one of the pieces in the process of creating a component. The general outline for the build process of a component for a compiled language is:

  1. Using wit-bindgen source code for the language is generated representing bindings to the specified APIs. This source code is then compiled by the native compiler and used by user-written code as well.
  2. The native language toolchain is used to emit a core WebAssembly module. This core wasm module is the "meat" of a component and contains all user-defined code compiled to WebAssembly. The most common native target to use for compilation today is the wasm32-wasi target.
  3. The output core wasm module is transformed into a component using the wasm-tools project, notably the wasm-tools component new subcommand. This will ingest the native core wasm output and wrap the output into the component model binary format.

The precise tooling and commands at each of these steps differs language by language, but this is the general idea. With a component in-hand the binary can then be handed off to a host runtimes for execution.

Creating components: WASI

An important consideration when creating a component today is WASI. All current native toolchains for languages which have WASI support are using the wasi_snapshot_preview1 version of WASI. This definition of WASI was made with historical *.witx files and is not compatible with the component model. There is, however, a means by which to still create components from modules that are using wasi_snapshot_preview1 APIs.

The wasm-tools component new subcommand takes an --adapt argument which acts as a way to polyfill non-component-model APIs, like wasi_snapshot_preview1, with component model APIs. The preview2-prototyping project is the current go-to location to acquire a polyfill from wasi_snapshot_preview1 to an in-development version of "wasi preview2" which is specified with WIT and the component model.

Notably you'll want to download one of the adapter modules and name it wasi_snapshot_preview1.wasm locally to pass as an --adapt argument to wasm-tools component new. Note that there are two modules provided on the downloads page, one is for non-commands which don't have a _start entrypoint in the generated core wasm module (e.g. the cdylib crate type in Rust) and one that is for command modules which has a _start entrypoint (e.g. a src/main.rs in Rust).

Supported Guest Languages

The wit-bindgen project is primarily focused on guest languages which are those compiled to WebAssembly. Each language here already has native support for execution in WebAssembly at the core wasm layer (e.g. targets the current core wasm specification). Brief instructions are listed here for each language of how to use it as well.

Each project below will assume the following *.wit file in the root of your project.

// wit/host.wit
package example:host

world host {
  import print: func(msg: string)

  export run: func()
}

Guest: Rust

The Rust compiler supports a native wasm32-wasi target and can be added to any rustup-based toolchain with:

rustup target add wasm32-wasi

In order to compile a wasi dynamic library, the following must be added to the Cargo.toml file:

[lib]
crate-type = ["cdylib"]

Projects can then depend on wit-bindgen by executing:

cargo add --git https://github.com/bytecodealliance/wit-bindgen wit-bindgen

WIT files are currently added to a wit/ folder adjacent to your Cargo.toml file. Example code using this then looks like:

// src/lib.rs

// Use a procedural macro to generate bindings for the world we specified in
// `host.wit`
wit_bindgen::generate!("host");

// Define a custom type and implement the generated `Host` trait for it which
// represents implementing all the necesssary exported interfaces for this
// component.
struct MyHost;

impl Host for MyHost {
    fn run() {
        print("Hello, world!");
    }
}

export_host!(MyHost);

By using cargo expand or cargo doc you can also explore the generated code.

This project can then be built with:

cargo build --target wasm32-wasi
wasm-tools component new ./target/wasm32-wasi/debug/my-project.wasm \
    -o my-component.wasm --adapt ./wasi_snapshot_preview1.wasm

This creates a my-component.wasm file which is suitable to execute in any component runtime. Using wasm-tools you can inspect the binary as well, for example inferring the WIT world that is the component:

wasm-tools component wit my-component.wasm
# world my-component {
#  import print: func(msg: string)
#  export run: func()
# }

which in this case, as expected, is the same as the input world.

Guest: C/C++

C and C++ code can be compiled for the wasm32-wasi target using the WASI SDK project. The releases on that repository have precompiled clang binaries which are pre-configured to compile for WebAssembly.

To start in C and C++ a *.c and *.h header file is generated for your project to use. These files are generated with the wit-bindgen CLI command in this repository.

wit-bindgen c ./wit
# Generating "host.c"
# Generating "host.h"
# Generating "host_component_type.o"

Some example code using this would then look like

// my-component.c

#include "host.h"

void host_run() {
    host_string_t my_string;
    host_string_set(&my_string, "Hello, world!");

    host_print(&my_string);
}

This can then be compiled with clang from the WASI SDK and assembled into a component with:

clang host.c host_component_type.o my-component.c -o my-core.wasm -mexec-model=reactor
wasm-tools component new ./my-core.wasm -o my-component.wasm

Like with Rust, you can then inspect the output binary:

wasm-tools component wit ./my-component.wasm

Guest: Java

Java bytecode can be compiled to WebAssembly using TeaVM-WASI. With this generator, wit-bindgen will emit *.java files which may be used with any JVM language, e.g. Java, Kotlin, Clojure, Scala, etc.

Guest: TinyGo

Go code can be compiled for the wasm32-wasi target using the TinyGo compiler. For example, the following command compiles main.go to a wasm modules with WASI support:

tinygo build -target=wasi main.go

Note: the current TinyGo bindgen only supports TinyGo version v0.27.0 or later.

To start in Go a *.go and *.h C header file are generated for your project to use. These files are generated with the wit-bindgen CLI command in this repository.

wit-bindgen tiny-go ./wit
# Generating "host.go"
# Generating "host.c"
# Generating "host.h"
# Generating "host_component_type.o"

If your Go code uses result or option type, a second Go file host_types.go will be generated. This file contains the Go types that correspond to the result and option types in the WIT file.

Some example code using this would then look like

// my-component.go
package main

import (
    gen "host/gen"
)

func init() {
    a := HostImpl{}
    gen.SetHost(a)
}

type HostImpl struct {
}

func (e HostImpl) Run() {
  gen.Print("Hello, world!")
}

//go:generate wit-bindgen tiny-go ../wit --out-dir=gen
func main() {}

This can then be compiled with tinygo and assembled into a component with:

go generate # generate bindings for Go
tinygo build -target=wasi -o main.wasm my-component.go # compile
wasm-tools component embed --world host ./wit main.wasm -o main.embed.wasm # create a component
wasm-tools component new main.embed.wasm --adapt wasi_snapshot_preview1.wasm -o main.component.wasm
wasm-tools validate main.component.wasm --features component-model

Guest: Other Languages

Other languages such as JS, Ruby, Python, etc, are hoped to be supported one day with wit-bindgen or with components in general. It's recommended to reach out on zulip if you're intersted in contributing a generator for one of these langauges. It's worth noting, however, that turning an interpreted language into a component is significantly different from how compiled languages currently work (e.g. Rust or C/C++). It's expected that the first interpreted language will require a lot of design work, but once that's implemented the others can ideally relatively quickly follow suit and stay within the confines of the first design.

CLI Installation

To install the CLI for this tool (which isn't the only way it can be used), run the following cargo command. This will let you generate the bindings for any supported language.

cargo install --git https://github.com/bytecodealliance/wit-bindgen wit-bindgen-cli

This CLI IS NOT stable and may change, do not expect it to be or rely on it being stable. Please reach out to us on zulip if you'd like to depend on it, so we can figure out a better alternative for your use case.

Host Runtimes for Components

The wit-bindgen project is intended to facilitate in generating a component, but once a component is in your hands the next thing to do is to actually execute that somewhere. This is not under the purview of wit-bindgen itself but these are some resources and runtimes which can help you work with components:

  • Rust: the wasmtime crate is an implementation of a native component runtime that can run any WIT world. It additionally comes with a bindgen! macro which acts similar to the generate! macro in this repository. This macro takes a WIT package as input and generates trait-based bindings for the runtime to implement and use.

  • JS: the js-component-tools project can be used to execute components in JS either on the web or outside the browser in a runtime such as node. This project generates a polyfill for a single concrete component to execute in a JS environment by extracting the core WebAssembly modules that make up a component and generating JS glue to interact between the host and these modules.

  • Python: the wasmtime project on PyPI has a bindgen mode that works similar to the JS integration. Given a concrete component this will generate Python source code to interact with the component using an embedding of Wasmtime for its core WebAssembly support.

  • Tooling: the wasm-tools project can be used to inspect and modify low-level details of components. For example as previously mentioned you can inspect the WIT-based interface of a component with wasm-tools component wit. You can link two components together with wasm-tools compose as well.

Note that the runtimes above are generally intended to work with arbitrary components, not necessarily only those created by wit-bindgen. This is also not necessarily an exhaustive listing of what can execute a component.

More Repositories

1

wasmtime

A fast and secure runtime for WebAssembly
Rust
14,541
star
2

wasm-micro-runtime

WebAssembly Micro Runtime (WAMR)
C
4,528
star
3

lucet

Lucet, the Sandboxing WebAssembly Compiler.
Rust
4,074
star
4

cranelift

Cranelift code generator
2,482
star
5

javy

JS to WebAssembly toolchain
Rust
1,958
star
6

rustix

Safe Rust bindings to POSIX-ish APIs
Rust
1,286
star
7

wasm-tools

CLI and Rust libraries for low-level manipulation of WebAssembly modules
Rust
1,105
star
8

wizer

The WebAssembly Pre-Initializer
Rust
859
star
9

wasmtime-go

Go WebAssembly runtime powered by Wasmtime
Go
729
star
10

cap-std

Capability-oriented version of the Rust standard library
Rust
614
star
11

cranelift-jit-demo

JIT compiler and runtime for a toy language, using Cranelift
Rust
588
star
12

jco

JavaScript tooling for working with WebAssembly Components
Rust
468
star
13

cargo-wasi

A lightweight Cargo subcommand to build Rust code for the `wasm32-wasi` target
Rust
435
star
14

cargo-component

A Cargo subcommand for creating WebAssembly components based on the component model proposal.
Rust
405
star
15

wasmtime-dotnet

.NET embedding of Wasmtime https://bytecodealliance.github.io/wasmtime-dotnet/
C#
391
star
16

wasmtime-py

Python WebAssembly runtime powered by Wasmtime
Python
352
star
17

wasi

Experimental WASI API bindings for Rust
Rust
210
star
18

wasmparser

A simple event-driven library for parsing WebAssembly binary files
178
star
19

regalloc2

A new register allocator
Rust
178
star
20

wasi.dev

HTML
170
star
21

ComponentizeJS

JS -> WebAssembly Component
Rust
163
star
22

registry

WebAssembly Registry (Warg)
Rust
153
star
23

wasmtime-demos

Historical and dated demos for Wasmtime usage and WASI content
C#
152
star
24

wat

Rust WAT and WAST parser (WebAssembly Text Format)
113
star
25

regalloc.rs

Modular register allocator algorithms
Rust
107
star
26

WASI-Virt

Virtual implementations of WASI APIs
Rust
98
star
27

componentize-py

Rust
89
star
28

spidermonkey-wasm-rs

Rust
86
star
29

preview2-prototyping

Polyfill adapter for preview1-using wasm modules to call preview2 functions.
Rust
78
star
30

wasmtime-rb

Ruby WebAssembly runtime powered by Wasmtime
Rust
73
star
31

wasmtime-cpp

C++
70
star
32

wasm-interface-types

Raw Rust toolchain support for Wasm Interface Types
Rust
70
star
33

wac

WebAssembly Composition (WAC) tooling
Rust
69
star
34

sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
C
64
star
35

rfcs

RFC process for Bytecode Alliance projects
57
star
36

component-docs

Documentation around creating and using WebAssembly Components
Rust
46
star
37

target-lexicon

Target "triple" support
Rust
44
star
38

userfaultfd-rs

Rust bindings for the Linux userfaultfd functionality
Rust
42
star
39

system-interface

Extensions to the Rust standard library
Rust
40
star
40

wasi-nn

High-level bindings for wasi-nn system calls
CSS
36
star
41

wasmprinter

Rust library to print a WebAssembly binary to its textual format
32
star
42

spidermonkey-wasm-build

Utilities to compile SpiderMonkey to wasm32-wasi
JavaScript
22
star
43

wit-deps

WIT dependency manager
Rust
20
star
44

meetings

Python
20
star
45

filecheck

Library for writing tests for utilities that read text files and produce text output
Rust
20
star
46

vscode-wit

Visual Studio Code extension to recognize and highlight the WebAssembly Interface Type (WIT) IDL.
TypeScript
19
star
47

wamr-rust-sdk

Rust
15
star
48

wasm-score

A benchmark for standalone WebAssembly
C
15
star
49

cranelift.vim

Vim editor configuration for working with cranelift IR (clif) files
Vim Script
14
star
50

arf-strings

Encoding and decoding for ARF strings
C
12
star
51

SIG-Registries

11
star
52

bytecodealliance.org

CSS
10
star
53

subscribe-to-label-action

A GitHub action that allows users to subscribe to a label and automatically get @'d when the label is applied
JavaScript
10
star
54

SIG-Guest-Languages

Special Interest Group (SIG) whose goal is to investigate how best to integrate Wasm and components into dynamic programming language ecosystems in a way that feels native to those ecosystems.
10
star
55

wasm-spec-interpreter

Rust bindings for the Wasm spec interpreter.
Rust
8
star
56

governance

7
star
57

wasm-parallel-gzip

Some example scripts for building a parallel compression/decompression tool for WebAssembly
Makefile
6
star
58

wamr-python

Python
6
star
59

fs-set-times

Set filesystem timestamps
Rust
5
star
60

arena-btree

Rust
5
star
61

wasmtime-libfuzzer-corpus

libFuzzer corpus for our wasmtime fuzz targets
Shell
5
star
62

wamr.dev

The WAMR homepage
HTML
5
star
63

wasmtime.dev

The Wasmtime homepage
CSS
4
star
64

cm-go

4
star
65

libc-test

Mirror of git://nsz.repo.hu:49100/repo/libc-test (see https://wiki.musl-libc.org/libc-test.html for more information)
C
3
star
66

wamr-app-framework

WebAssembly Micro Runtime Application Framework
C
2
star
67

wasmtime-wasi-nn

2
star
68

actions

GitHub actions to setup wasm-tools and wasmtime
TypeScript
1
star
69

label-messager-action

Automatically leave a message when an issue or pull request has a certain label
JavaScript
1
star
70

wasm-ml-meetings

Informal working group for machine learning and WebAssembly, especially wasi-nn
1
star