Balena.io layers for Yocto

Description

This repository enables building balenaOS for various devices.

Layers Structure

• meta-balena-common : layer which contains common recipes for all our supported platforms.
• meta-balena-* : layers which contain recipes specific to yocto versions.
• other files : README, COPYING, etc.

Dependencies

• https://docs.yoctoproject.org/brief-yoctoprojectqs/#build-host-packages
• docker
• jq

Versioning

The meta-balena version is kept in the DISTRO_VERSION variable. The balena-<board> version is kept in the file called VERSION located in the root of the balena-<board> repository and read in the build as the variable HOSTOS_VERSION.

• The version of meta-balena is in semver format being 3 numbers separated by a dot. The patch number can have a beta label. e.g. 1.2.3, 1.2.3-beta1, 2.0.0-beta1.
• The first balena-<board> release based on a specific meta-balena release X.Y.Z, will be X.Y.Z, the same as the meta-balena version. Example: the first balena-<board> version based on meta-balena 1.2.3 will be 1.2.3.
• Subsequent balena-<board> releases are constructed by appending to the meta-balena version a rev label. For example a meta-balena 1.2.3 can go through 3 board revisions, being 1.2.3 the initial revision, and 1.2.3+revN the subsequent ones, with the final version being 1.2.3+rev2 .
• When updating meta-balena version in balena-<board>, the version will reset to the meta-balena version. Ex: 1.2.3+rev4 will be updated to 1.2.4 .

We define host OS version as the balena-<board> version and we use this version as HOSTOS_VERSION.

Build flags

Before bitbake-ing with meta-balena support, a few flags can be changed in the conf/local.conf from the build directory. Editing of local.conf is to be done after source-ing. See below for explanation on such build flags.

Configure custom network manager

By default balena uses NetworkManager on host OS to provide connectivity. If you want to change and use other providers, list your packages using NETWORK_MANAGER_PACKAGES. You can add this variable to local.conf. Here is an example:

NETWORK_MANAGER_PACKAGES = "mynetworkmanager mynetworkmanager-client"

Customizing splash

We configure all of our initial images to produce a balena logo at boot, shutdown or reboot. But we encourage any user to go and replace that logo with their own. All you have to do is replace the splash/balena-logo.png file that you will find in the first partition of our images (boot partition) with your own image. NOTE: As it currently stands plymouth expects the image to be named balena-logo.png. In older releases this file was called resin-logo.png.

Docker storage driver

By default the build system will set all the bits needed for the docker to be able to use the aufs storage driver. This can be changed by defining BALENA_STORAGE in your local.conf. It supports aufs and overlay2.

OS development

To configure a development build that disables quiet boot and allows bootloader shell access, edit the build's local.conf adding:

OS_DEVELOPMENT = "1"

This is a development only setting and no OS_DEVELOPMENT configured images are deployed.

The OS

SSH and Avahi services

The OS runs SSH (openSSH) on port 22222. Running this service takes advantage of the socket activation systemd feature so the SSH daemon will only run when there is a SSH connection to the device saving idle resources in this way. In order to connect to a device, one can use it's IP when known or resolve the hostname over mDNS as its hostname is advertised over network using an avahi service. When the latter is used, configuration of the client is needed (see for example https://wiki.archlinux.org/index.php/Avahi#Hostname_resolution).

Time in the OS

We currently have three time sources:

• build time - stored in /etc/timestamp and generated by the build system when the image is generated
• network time - managed by chronyd
• RTC time when available

Early in the boot process, the OS will start three services associated with the sources listed above, which manage the system clock.

The first one is timeinit-rtc. This service, when a RTC is available (/etc/rtc) will update the system clock using the value read from the RTC. If there is no RTC available, the service will not do anything. The second service is timeinit-timestamp which reads the build timestamp and updates the system clock if the timestamp is after the current system clock. The third service is chronyd.service which is responsible of managing the time afterwards over NTP.

The order of the services is as stated above and provides a robust time initialization at boot in both cases where RTC is or not available.

Bootloader

balenaOS relies on the device's bootloader to select the active root filesystem. Several bootloaders are used accross the supported devices line-up:

• U-boot
  • Is used on most of the supported ARM device-types
  • Only block devices can be used with BalenaOS, RAW flash devices are not supported
  • Common functionality is implemented in the u-boot environment, which is provided by the common OS Yocto layer
  • The environment is embedded in the u-boot binary. This allows for the intended configuration to be used with the matching version of BalenaOS and avoids interference from any pre-programmed environment
  • Device-specific functions are provided by the device repository, either in a u-boot script or in the enviroment defined by the board configuration files
  • Specific Jetson modules (TX2, Nano) use an extra extlinux.conf file, which is loaded and parsed by u-boot
  • Three environment files are stored and loaded by u-boot from the BalenaOS boot partition. resinOS_uEnv.txt is used for storing the active root partition index, extra_uEnv.txt stores device-specific configuration elements like optional kernel command-line parameters as well as any custom selected device-tree while bootcount.env stores the number of failed attempted boot retries during an OS update. NOTE: Custom device-tree selection is supported only on specific devices
  • Applies the kernel device-tree overlays specified in uEnv.txt_internal/uEnv.txt on Beaglebone devices
• Grub
  • Is used for the supported x86 device-types
  • Common functionality is implemented by the OS layer in the grub configuration template
• Cboot
  • Is used on Jetson Xavier devices running L4T 32.X
  • Loads device-trees from device-specific A/B partitions
  • Unlike the rest of the bootloaders, it does not support FAT filesystems
  • The current active root filesystem label is defined in the kernel command line provided by the kernel device-tree. The active rootfs is selected at boot time based on the active device-tree
• UEFI L4Tlauncher
  • Is used on the Jetson Orin platforms
  • Obtains the kernel image path and kernel cmdline arguments from extlinux.conf files
  • Rollbacks and active root filesystem selection are implemented in bootloader patches provided bythe balena-jetson-orin device repository
  • Employs the same rollback mechanisms used by balenaOS in u-boot by storing and reading environment variables from the resinOS_uEnv.txt, extra_uEnv.txt and bootcount.env files

Rollback framework

Check docs/rollbacks.md for the rollback documentation

Devices support

WiFi Adapters

We currently tested and provide explicit support for the following WiFi adapters:

• bcm43143 based adapters

Modems

We currently test as part of our release process and provide explicit support for the following modems:

• USB modems (tested on Raspberry Pi 3, Balena Fin, Intel NUC and Nvidia TX2)
  • Huawei MS2131i-8
  • Huawei MS2372
• mPCI modems (tested on Balena Fin and Nvidia TX2 Spacely carrier)
  • Huawei ME909s-120
  • Quectel EC20
  • SIM7600E

Recommended WiFi USB dongle

• Panda N600 Dual-Band (2.4 GHz + 5 GHz) Wireless-N USB Adapter This USB dongle is based on the Ralink RT5572 chipset and is supported by the generic rt2800usb driver. Tests have been done on the PAU09 model of the Panda N600 Dual-Band USB Adapter and having the firmware version 0.36 from firmware file rt2870.bin

How to fix various build errors

• Supervisor fails with a log similar to:

Step 3 : RUN chmod 700 /entry.sh
---> Running in 445fe69866f9
operation not supported

This is probably because of a docker bug where, if you update kernel and don't reboot, docker gets confused. The fix is to reboot your system. More info: http://stackoverflow.com/questions/29546388/getting-an-operation-not-supported-error-when-trying-to-run-something-while-bu

config.json

The behavior of balenaOS can be configured by setting the following keys in the config.json file in the boot partition. This configuration file is also used by the supervisor.

hostname

(string) The configured hostname of the device, otherwise the device UUID is used.

persistentLogging

(boolean) Enable or disable persistent logging on the device - defaults to false. Once persistent journals are enabled, they end up stored as part of the data partition on the device (either on SD card, eMMC, harddisk, etc.). This is located on-device at /var/log/journal/<uuid> where the UUID is variable.

country

(string) Two-letter country code for the country in which the device is operating. This is used for setting the WiFi regulatory domain, and you should check the WiFi device driver for a list of supported country codes.

ntpServers

(string) A space-separated list of NTP servers to use for time synchronization. Defaults to resinio.pool.ntp.org servers:

• 0.resinio.pool.ntp.org
• 1.resinio.pool.ntp.org
• 2.resinio.pool.ntp.org
• 3.resinio.pool.ntp.org

dnsServers

(string) A space-separated list of preferred DNS servers to use for name resolution.

• When dnsServers is not defined, or empty, Google's DNS server (8.8.8.8) is added to the list of DNS servers obtained via DHCP or statically configured in a NetworkManager connection profile.
• When dnsServers is "null" (a string), Google's DNS server (8.8.8.8) will NOT be added as described above.
• When dnsServers is defined and not "null", the listed servers will be added to the list of servers obtained via DHCP or statically configured via a NetworkManager connection profile.

balenaRootCA

(string) A base64-encoded PEM CA certificate that will be installed into the root trust store. This makes the device trust TLS/SSL certificates from this authority. This is useful when the device is running behind a re-encrypting network device, like a transparent proxy or some deep packet inspection devices.

"balenaRootCA": "4oCU4oCTQkVHSU4gQ0VSVElGSUNBVEXigJTi..."

developmentMode

To enable development mode at runtime:

"developmentMode": true

By default development mode enables unauthenticated SSH logins unless custom SSH keys are present, in which case SSH key access is enforced.

Also, development mode provides serial console passwordless login as well as an exposed balena engine socket to use in local mode development.

os

An object containing settings that customize the host OS at runtime.

network

wifi

An object that defines the configuration related to Wi-Fi.

• "randomMacAddressScan" (boolean) Configures MAC address randomization of a Wi-Fi device during scanning

The following example disables MAC address randomization of Wi-Fi device during scanning:

"os": {
 "network" : {
  "wifi": {
    "randomMacAddressScan": false
  }
 }
}

connectivity

An object that defines configuration related to networking connectivity checks. This feature builds on NetworkManager's connectivity check, which is further documented in the connectivity section here.

• "uri" (string) Value of the url to query for connectivity checks. Defaults to $API_ENDPOINT/connectivity-check.
• "interval" (string) Interval between connectivity checks in seconds. Defaults to 3600. To disable the connectivity checks set the interval to "0".
• "response" (string). If set controls what body content is checked for when requesting the URI. If it is an empty value, the HTTP server is expected to answer with status code 204 or send no data.

The following example configures the connectivity check by passing the balenaCloud connectivity endpoint with a 5-minute interval.

"os": {
 "network" : {
  "connectivity": {
    "uri" : "https://api.balena-cloud.com/connectivity-check",
    "interval" : "300"
  }
 }
}

udevRules

An object containing one or more custom udev rules as key:value pairs.

To turn a rule into a format that can be easily added to config.json, use the following command:

cat rulefilename | jq -sR .

For example:

root@resin:/etc/udev/rules.d# cat 64.rules | jq -sR .
"ACTION!=\"add|change\", GOTO=\"modeswitch_rules_end\"\nKERNEL==\"ttyACM*\", ATTRS{idVendor}==\"1546\", ATTRS{idProduct}==\"1146\", TAG+=\"systemd\", ENV{SYSTEMD_WANTS}=\"u-blox-switch@'%E{DEVNAME}'.service\"\nLBEL=\"modeswitch_rules_end\"\n"

The following example contains two custom udev rules that will create /etc/udev/rules.d/56.rules and /etc/udev/rules.d/64.rules. The first time rules are added, or when they are modified, udevd will reload the rules and re-trigger.

"os": {
 "udevRules": {
  "56": "ENV{ID_FS_LABEL_ENC}==\"resin-root*\", IMPORT{program}=\"resin_update_state_probe $devnode\", SYMLINK+=\"disk/by-state/$env{BALENA_UPDATE_STATE}\"",
  "64" : "ACTION!=\"add|change\", GOTO=\"modeswitch_rules_end\"\nKERNEL==\"ttyACM*\", ATTRS{idVendor}==\"1546\", ATTRS{idProduct}==\"1146\", TAG+=\"systemd\", ENV{SYSTEMD_WANTS}=\"u-blox-switch@'%E{DEVNAME}'.service\"\nLBEL=\"modeswitch_rules_end\"\n"
 }
}

sshKeys

(Array) An array of strings containing a list of public SSH keys that will be used by the SSH server for authentication.

"os": {
 "sshKeys": [
  "ssh-rsa AAAAB3Nza...M2JB balena@macbook-pro",
  "ssh-rsa AAAAB3Nza...nFTQ balena@zenbook"
 ]
}

installer

An object that configures the behaviour of the balenaOS installer image.

secureboot

(boolean) Opt-in to installing a secure boot and encrypted disk system for supported device types.

"installer": {
  "secureboot": true
}

migrate

An object that configures the behaviour of the balenaOS installer migration module.

migrate.force

(boolean) Forces the migration to run. By default the migration only runs if the installer is booting in a single disk system.

"installer": {
  "migrate": {
    "force": true
  }
}

Yocto version support

The following Yocto versions are supported:

• Kirkstone (4.0)
• Long Term Support
• Honister (3.4)
• EOL
• Dunfell (3.1)
• Long Term Support
• Warrior (2.7)
• EOL
• Thud (2.6)
• EOL
• Sumo (2.5)
• EOL
• Rocko (2.4)
• EOL
• Pyro (2.3)
• EOL