• Stars
    star
    1,004
  • Rank 45,731 (Top 1.0 %)
  • Language
    Shell
  • License
    MIT License
  • Created over 9 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Command line script to flash SD card images of any kind

flash

CircleCI Build Status Release Stars

Command line script to flash SD card images of any kind.

Note that for some devices (e.g. Raspberry Pi), at the end of the flashing process the tool tries to customize the SD card e.g. it configures a hostname or WiFi. And with a cloud-init enabled image you can do much more like adding users, SSH keys etc.

The typical workflow looks like this:

asciicast

  1. Run flash https://github.com/hypriot/image-builder-rpi/releases/download/v1.12.0/hypriotos-rpi-v1.12.0.img.zip
  2. Insert SD card to your notebook
  3. Press RETURN
  4. Eject SD card and insert it to your Raspberry Pi - done!

This script can

  • download a compressed SD card from the internet or from S3
  • use a local SD card image, either compressed or uncompressed
  • wait until a SD card is plugged in
  • search for a SD card plugged into your Computer
  • show progress bar while flashing (if pv is installed)
  • copy an optional cloud-init user-data and meta-data file into the boot partition of the SD image
  • copy an optional config.txt file into the boot partition of the SD image (eg. to enable onboard WiFi)
  • copy an optional device-init.yaml or occidentalis.txt file into the boot partition of the SD image (for older HypriotOS versions)
  • copy an optional custom file into the boot partition of the SD image
  • optional set the hostname of this SD image
  • optional set the WiFi settings as well
  • play a little sound after flashing
  • unplugs the SD card

At the moment only Mac OS X and Linux is supported.

Installation

Download the appropriate version for Linux or Mac with this command

curl -LO https://github.com/hypriot/flash/releases/download/2.7.2/flash
chmod +x flash
sudo mv flash /usr/local/bin/flash

Install Dependencies

The flash script needs optional tools

  • curl - if you want to flash directly with an HTTP URL
  • aws - if you want to flash directly from an AWS S3 bucket
  • pv - to see a progress bar while flashing with the dd command
  • unzip - to extract zip files.
  • hdparm - to run the program

Mac

brew install pv
brew install awscli

Linux (Debian/Ubuntu)

sudo apt-get install -y pv curl python-pip unzip hdparm
sudo pip install awscli

Usage

$ flash --help
usage: flash [options] [name-of-rpi.img]

Flash a local or remote Raspberry Pi SD card image.

OPTIONS:
   --help|-h      Show this message
   --bootconf|-C  Copy this config file to /boot/config.txt
   --config|-c    Copy this config file to /boot/device-init.yaml (or occidentalis.txt)
   --hostname|-n  Set hostname for this SD image
   --ssid|-s      Set WiFi SSID for this SD image
   --password|-p  Set WiFI password for this SD image
   --clusterlab|-l Start Cluster-Lab on boot: true or false
   --device|-d    Card device to flash to (e.g. /dev/disk2)
   --force|-f     Force flash without security prompt (for automation)
   --userdata|-u  Copy this cloud-init config file to /boot/user-data
   --metadata|-m  Copy this cloud-init config file to /boot/meta-data
   --file|-F      Copy this custom file to /boot

Configuration

The strength of the flash tool is that it can insert some configuration files that gives you the best first boot experience to customize the hostname, WiFi and even user logins and SSH keys automatically.

cloud-init

With HypriotOS v1.7.0 and higher the options --userdata and --metadata can be used to copy both cloud-init config files into the FAT partition.

This is an example how to create our default user with a password.

#cloud-config
# vim: syntax=yaml
#
hostname: black-pearl
manage_etc_hosts: true

users:
  - name: pirate
    gecos: "Hypriot Pirate"
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    groups: users,docker,video
    plain_text_passwd: hypriot
    lock_passwd: false
    ssh_pwauth: true
    chpasswd: { expire: false }

package_upgrade: false

Please have a look at the sample folder, our guest blogpost Bootstrapping a Cloud with Cloud-Init and HypriotOS or at the cloud-init documentation how to do more things like using SSH keys, running additional commands etc.

config.txt

The option --bootconf can be used to copy a config.txt into the SD image before it is unplugged.

With this option it is possible to change some memory, camera, video settings etc. See the config.txt documentation at raspberrypi.org for more details.

The boot config file config.txt has name/value pairs such as:

max_usb_current=1
hdmi_force_hotplug=1

device-init.yaml

For HypriotOS older than v1.7.0 the option --config can be used to copy a device-init.yaml into the SD image before it is unplugged. This YAML file can be read by newer HyperiotOS SD images.

The config file device-init.yaml should look like

hostname: black-pearl
wifi:
  interfaces:
    wlan0:
      ssid: "MyNetwork"
      password: "secret_password"

If you don't want to set any wifi settings, comment out or remove the wlan0, ssid and password.

fake-hwclock.data

HypriotOS 1.12.0 and higher has the /etc/fake-hwclock.data file symlinked to the boot partition. Flash tool updates this timestamp with the current time (UTC timezone). This fixes problems running commands that communicate with the internet with a wrong initial date on first boot.

Use cases

Flash a compressed SD image from the internet

flash https://github.com/hypriot/image-builder-rpi/releases/download/v1.12.0/hypriotos-rpi-v1.12.0.img.zip

Flash and change the hostname

This works only for SD card images that already have occi installed.

flash --hostname mypi hypriot.img

Then unplug the SD card from your computer, plug it into your Pi and boot your Pi. After a while the Pi can be found via Bonjour/avahi and you can log in with

Onboard WiFi

The options --userdata and --bootconf must be used to disable UART and enable onboard WiFi for Raspberry Pi 3 and Pi 0. For external WiFi sticks you do not need to specify the -bootconf option.

flash --userdata sample/wlan-user-data.yaml --bootconf sample/no-uart-config.txt hypriotos-rpi-v1.12.0.img

Automating flash

For non-interactive usage, you can predefine the user input in the flash command with the -d and -f options:

flash -d /dev/mmcblk0 -f hypriotos-rpi-v1.12.0.img

Development

Pull requests and other feedback is always welcome. The flash tool should fit our all needs and environments.

To develop the flash scripts you need either a Linux or macOS machine to test locally. On a Mac you can use Docker to run the Linux tests in a container and if you dare you can run the macOS tests directly. On a Linux machine you can not test the macOS variant directly. But in every case you can send a pull request and push code to GitHub and the CI pipeline with CircleCI (Linux) and TravisCI (macOS) will test your code for both platforms.

Local development

The flash script are checked with the shellcheck static analysis tool.

The integration tests can be run locally on macOS or Linux. We use BATS which is installed with NPM package. So you would need Node.js to setup a local development environment. As the flash script runs dd and some commands with sudo it is recommended to use the isolated test environment with Docker or run this local tests in a macOS / Linux VM.

npm install
npm test

Isolated tests with Docker

If you do not want to install all these development tools (shellcheck, bats, node) and don't trust the flash script enough, you can use Docker instead and run the shellcheck and integration tests in a much safer test environment.

All you need is Docker and make installed to run the following tests.

Shellcheck

The flash script are checked with the shellcheck static analysis tool.

make shellcheck

Integration tests

The flash script also have BATS integration tests. You don't have to install everything on your development machine. It should be enough to test the Linux variant in a Docker container and then run the macOS tests with TravisCI.

make test

Test Linux from Mac

For manual tests of the Linux version on a Mac there is a Vagrant environment. It can be used to investigate Linux problems when you don't have a baremetal Linux machine. With some help I found a way to spin up a VirtualBox Vagrant box with Ubuntu that maps the internal Apple SD card reader into the VM. Thanks to Flexshot for the helper functions I found in NextThingCo/CHIP-SDK#15.

Check the vendor ID and product ID in "About this Mac" -> System Report ... -> Card Reader. I found the vendor ID 0x05ac and product ID 0x8406 can be found in the Vagrantfile.

vagrant up --provider virtualbox
vagrant ssh
cd /vagrant
./flash hypriotos-rpi-v1.12.0.img.zip

More Repositories

1

image-builder-rpi

SD card image for Raspberry Pi with Docker: HypriotOS
Ruby
1,065
star
2

cluster-lab

Hypriot Cluster Lab
Shell
156
star
3

rpi-kernel

Build a Linux kernel for Raspberry Pi 0/1/2/3/3B+
Shell
134
star
4

blog

Repository for the Hypriot Blog
JavaScript
87
star
5

rpi-alpine-scratch

Raspberry Pi compatible Docker Image with a minimal linux system.
Shell
83
star
6

arm-compose

Automated build for docker-compose on ARM
Shell
73
star
7

rpi-node

RPi-compatible Docker image with Node.js
Shell
72
star
8

device-init

Initialize a device on boot with user defined configuration
Go
70
star
9

golang-armbuilds

Pre-compiled GOLANG tarballs for ARM devices.
Shell
69
star
10

os-rootfs

The root filesystem of HypriotOS
Shell
68
star
11

rpi-mysql

RPi-compatible Docker image with Mysql
Shell
67
star
12

rpi-docker-builder

Build steps to build hypriot-docker for Raspberry Pi
Shell
65
star
13

qemu-register

Docker Image to register Qemu interpreters for ARMHF/AARCH64/PPC64LE/Risc-V in Docker Host w/ Linux Kernel 4.8+
Shell
61
star
14

rpi-busybox-httpd

Raspberry Pi compatible Docker Image with a minimal `Busybox httpd` web server.
Makefile
60
star
15

x11-on-HypriotOS

Instructions how to install X11 on a HypriotOS based Raspberry Pi system
Shell
38
star
16

image-builder-odroid-c2

Build SD card image for ODROID C2
Ruby
36
star
17

rpi-node-haproxy-example

Run Node.js webservers behind haproxy with docker-compose on Raspberry Pi
JavaScript
36
star
18

rpi-nano-httpd

A nano sized web server packed into a Docker Nano Container
Assembly
35
star
19

rpi-traefik

Build an ARM docker image for TrΓ¦fΙͺk, a modern reverse proxy
29
star
20

rpi-python

RPi-compatible Docker Image with Python
29
star
21

rpi-node-neopixel-example

Example using Adafruit's NeoPixel with Node.js in Docker on Raspberry Pi
JavaScript
29
star
22

rpi-gogs-raspbian

Raspberry Pi compatible Docker Image with a minimal gogs git server.
Makefile
28
star
23

rpi-alpine

Alpine for ARM
25
star
24

rpi-influxdb

Docker image for InfluxDB on ARM
Shell
25
star
25

rpi-java

RPi-compatible Docker Image with Java
Makefile
22
star
26

image-builder-raw

Raw Image for HypriotOS
Shell
21
star
27

rpi-noip

rpi-noip is a docker image of the official no-ip Dynamic DNS Update client
Dockerfile
20
star
28

rpi-consul

RPi-compatible Docker Image with Consul
Makefile
20
star
29

Pirate-ShieldTV

Hacking the Nvidia ShieldTV to use Ubuntu Linux for ARM64/AARCH64
Shell
19
star
30

rpi-redis

RPi-compatible Docker Image with Redis
Shell
18
star
31

rpi-cluster-lab-demos

Demos for the Hypriot Cluster Lab
18
star
32

rpi-dockerui

Builder for an ARM compatible Docker image of https://github.com/crosbymichael/dockerui
17
star
33

docker-machine-hypriot

Docker Machine driver for Hypriot
Shell
16
star
34

image-builder-odroid-xu4

Build SD card image for ODROID XU4
Ruby
16
star
35

rpi-drone

Raspberry Pi compatible Docker Image with drone CI server
Makefile
15
star
36

rpi-swarm

Raspberry Pi compatible Docker image with Docker Swarm - https://github.com/docker/swarm
15
star
37

rpi-golang

Docker Image packaging for GOLANG, used for Raspberry Pi
Makefile
15
star
38

image-builder-odroid-c1

Build SD card image for ODROID C1 and C1+
Ruby
15
star
39

image-builder-nvidia-shieldtv

Build SD card image for NVIDIA ShieldTV
Ruby
14
star
40

rpi-node-example-hello-world

Node.js example in Docker for Raspberry Pi
HTML
12
star
41

rpi-hugo

Raspberry Pi compatible Docker Image with Hugo - a static webpage builder
Makefile
10
star
42

rpi-whoami

Simple Docker Image that prints its Container ID. For RPi only.
Go
9
star
43

image-builder

Docker image for Hypriot's image-builder-* repos
Dockerfile
9
star
44

rpi-iojs

RPi-compatible Docker Image with io.js
Makefile
8
star
45

rpi-ruby

RPi-compatible Docker image with Ruby
8
star
46

rpi-tor-relay

7
star
47

arm-machine

Automated build for docker-machine on ARM
Shell
6
star
48

swarmlab

Vagrant-based Swarmlab to easily experiment with Docker Swarm Mode.
Shell
6
star
49

rpi-kitematic

Put Kitematic into a Docker Container Image and run it locally on your Raspberry Pi.
Shell
6
star
50

armhf-busybox

busybox ARM Docker container image
Ruby
5
star
51

hypriot.github.io

HTML
5
star
52

rpi-google-coder

Google Coder as ARM compatible Docker image
Makefile
5
star
53

rpi-registrator

Builder for an ARM compatible Docker image of https://github.com/gliderlabs/registrator.
5
star
54

talk

The team at Hypriot simplifies the way you get Docker running on ARM. Let's talk about this in our Gitter chatroom!
3
star
55

image-builder-bananapi

Build SD card image for Banana Pi
Ruby
3
star
56

rpi-crate

RPi-compatible Docker Image with Crate.io
Makefile
3
star
57

rpi-cron-example

RPi-compatible Docker Image - example usage of a cron job in a container
Makefile
2
star
58

rpi-minecraft

Shell
2
star
59

dockerfiles

Repository for our internal dockerfiles
Makefile
2
star
60

hypriot-talks

This is your first repository
HTML
2
star
61

rpi-consul-template

Raspberry Pi compatible Docker Image of Consul Template
Makefile
2
star
62

rpi-java-hello-world

A Hello World in Java with a RPi compatible Docker image
Java
2
star
63

hardware-lists

Contains lists of hardware that is recommended to use in Hypriot projects.
2
star
64

rpi-openvswitch-builder

Builder for Open vSwitch packages compatible to the Raspberry Pi
Shell
2
star
65

image-builder-cubie

This is your first repository
Shell
1
star
66

armhf-hello-world

ARM version of hello-world
1
star
67

rpi-noobs-builder

create NOOBS zip with HypriotOS
Shell
1
star