ARM Assembly Cheat
Almost all the content in this repository has been moved to: https://github.com/cirosantilli/linux-kernel-module-cheat#userland-assembly
Only the core infrastructure of this repo was left here. No major new features are intended to be added here.
Notable advantages of LKMC repository include:
-
a single unified cross arch setup for ARM and x86_64, with cross arch concepts all nicely factored out
-
gem5 support. This is because we have integration of QEMU / gem5 / buildroot setups already done there
-
parallel testing. Mostly because the build system there is Python, which is more flexible.
-
other stuff I can’t remember right now. That setup just has a ton of features, and will continue to get more and more ;-)
There is only one use case left for this repository: since this is more minimal, it is easier to upgrdte to the latest binutils-gdb here without breaking unrelated stuff, in order to get very latest instructions.
The initial motivation for that was [sve] although that specific case has already been moved to LKMC as well!
I might however start taking some risks on LKMC and upgrading binutils-gdb to master when needed anyways, or just build the latest binutils-gdb myself over there for userland only.
1. Old README
Here is the old README with only infrastructure sections left.
ARMv7 and [ARMv8] assembly userland minimal examples tutorial. Runnable asserts on x86 hosts with QEMU user mode or natively on ARM targets. Nice GDB step debug setup. Tested on Ubuntu 18.04 host and Raspberry Pi 2 and 3 targets. Baremetal setup at: https://github.com/cirosantilli/linux-kernel-module-cheat#baremetal-setup x86 cheat at: https://github.com/cirosantilli/x86-assembly-cheat
2. Getting started
On Ubuntu, clone, configure, build QEMU and Binutils from source, run all ARMv7 and ARMv8 examples through QEMU user, and assert that they exit with status 0:
git clone --recursive https://github.com/cirosantilli/arm-assembly-cheat cd arm-assembly-cheat ./download-dependencies make test echo $?
Expected outcome: the exit status is successful:
0
For other operating systems, see: Getting started on non-Ubuntu operating systems.
We compile our own Binutils and QEMU to be able to use the newest ISA features. Those projects build pretty fast (~10 minutes), so it is fine. The cleanest thing would be to also compile GCC with crosstool-NG toolchain.
The armv7 examples are all located under the v7 directory. Run all of them:
cd v7 make test echo $?
Run just one of them:
cd v7 make test-<basename-no-extension> echo $?
E.g.:
make test-add
will run userland/arch/arm/add.S.
This just tests some assertions, but does not output anything. See: Asserts.
Alternatively, to help with tab completion, the following shortcuts all do the same thing as make test-add:
./t add ./t add. ./t add.out
[armv8] examples are all located under the v8 directory. They can be run in the same way as ARMv7 examples:
cd v8 make test-movk
Just build the examples without running:
make
Clean the examples:
make clean
This does not clean QEMU builds themselves. To do that run:
make qemu-clean
2.1. Asserts
Almost all example don’t output anything, they just assert that the computations are as expected and exit 0 is that was the case.
Failures however output clear error messages.
Try messing with the examples to see them fail, e.g. modify userland/arch/arm/add.S to contain:
mov r0, #1 add r1, r0, #2 ASSERT_EQ(r1, 4)
and then watch it fail:
cd v7 make test-add
with:
error 1 at line 12 Makefile:138: recipe for target 'test-add' failed error 1 at line 12
since 1 + 2 tends to equal 3 and not 4.
So look how nice we are: we even gave you the line number 12 of the failing assert!
2.2. Getting started on non-Ubuntu operating systems
If you are not on an Ubuntu host machine, here are some ways in which you can use this repo.
2.2.1. Other Linux distro hosts
For other Linux distros, you can either:
-
have a look at what
download-dependenciesdoes and adapt it to your distro. It should be easy, then proceed normally.Might fail due to some incompatibility, but likely won’t.
-
run this repo with docker. Requires you to know some Docker boilerplate, but cannot (?) fail.
2.2.1.1. Docker host setup
sudo apt install docker sudo docker create -it --name arm-assembly-cheat -w "/host/$(pwd)" -v "/:/host" ubuntu:18.04 sudo docker exec -it arm-assembly-cheat /bin/bash
Then inside Docker just add the --docker flag to ./download-dependencies and proceed otherwise normally:
./download-dependencies --docker make test
The download-dependencies takes a while because build-dep binutils is large.
We share the repository between Docker and host, so you can just edit the files on host with your favorite text editor, and then just run them from inside Docker.
TODO: GDB TUI GUI is broken inside Docker due to terminal quirks. Forwarding the port and connecting from host will likely work, but I’m lazy to try it out now.
2.2.2. Non-Linux host
For non-Linux systems, the easiest thing to do is to use an Ubuntu virtual machine such as VirtualBox: https://askubuntu.com/questions/142549/how-to-install-ubuntu-on-virtualbox.
Porting is not however impossible because we use the C standard library for portability, see: Architecture of this repo. Pull requests are welcome.
2.2.3. Raspberry Pi 2 native
Yay! Let’s see if this actually works on real hardware, or if it is just an emulation pipe dream?
Tested on Raspbian Lite 2018-11-13 with this repo at commit bcddf29c8e00b30afe7b3643558b25f22a64405b.
For now, we will just compile natively, since I’m not in the mood for cross compilation hell today.
According to Wikipedia the Raspberry Pi 2 V 1.1 which I have has a BCM2836 SoC, which has 4 ARM Cortex-A7 cores, which implement ARMv7-A, VFPv4 and [neon].
Therefore we will only be able to run v7 examples on that board.
First connect to your Pi through SSH as explained at: https://stackoverflow.com/revisions/39086537/10
Then inside the Pi:
sudo apt-get update sudo apt-get install git make gcc gdb git clone https://github.com/cirosantilli/arm-assembly-cheat cd arm-assembly-cheat/v7 make NATIVE=y test make NATIVE=y gdb-add
GDB TUI is slightly buggier on the ancient 4.9 toolchain (current line gets different indentation, does not break on the right instruction after asm_main_after_prologue, cannot leave TUI), but it might still be usable
The Pi 0 and 1 however have a BCM2835 SoC, which has an ARM1176JZF-S core, which implements the ARMv6Z ISA, which we don’t support yet on this repo.
2.2.4. Raspberry Pi 3 native
The Raspberry Pi 3 has a BCM2837 SoC, which has 4 Cortex A53 cores, which implement ARMv8-A.
However, as of July 2018, there is no official [armv8] image for the Pi 3, the same ARMv7 image is provided for both: https://raspberrypi.stackexchange.com/questions/43921/raspbian-moving-to-64-bit-mode
Then we look at the following threads:
which lead us to this 64-bit Debian based distro for the Pi: https://github.com/bamarni/pi64
So first we flash pi64’s 2017-07-31 release, and then do exactly the same as for the Raspberry Pi 2, except that you must go into the v8 directory instead of v7.
TODO: can we run the v7 folder in ARMv8? First I can’t even compile it. Related: https://stackoverflow.com/questions/21716800/does-gcc-arm-linux-gnueabi-build-for-a-64-bit-target For runtime: https://stackoverflow.com/questions/22460589/armv8-running-legacy-32-bit-applications-on-64-bit-os
2.3. GDB step debug
Debug one example with GDB:
make gdb-add
Shortcut:
./t -g add
This leaves us right at the end of the prologue of asm_main in GDB TUI mode, which is at the start of the assembly code in the .S file.
Stop on a different symbol instead:
make GDB_BREAK=main gdb-add
Shortcut:
./t -b main -g add
It is not possible to restart the running program from GDB as in gdbserver --multi unfortunately: https://stackoverflow.com/questions/51357124/how-to-restart-qemu-user-mode-programs-from-the-gdb-stub-as-in-gdbserver-multi
Quick GDB tips:
-
print a register:
i r r0
-
print floating point registers:
-
print an array of 4 32-bit integers in hex:
p/x (unsigned[4])my_array_0
-
print the address of a variable:
p &my_array_0
Bibliography: https://stackoverflow.com/questions/20590155/how-to-single-step-arm-assembler-in-gdb-on-qemu/51310791#51310791
2.3.1. Advanced usage
The default setup is opinionated and assumes that your are a newb: it ignores your .gdbinit and puts you in TUI mode.
However, you will sooner or later notice that TUI is crappy print on break Python scripts are the path of light, e.g. GDB dashboard.
In order to prevent our opinionated defaults get in the way of your perfect setup, use:
make GDB_EXPERT=y gdb-add
or the shortcut:
./t -G add
2.4. Disassemble
Even though GDB step debug can already disassemble instructions for us, it is sometimes useful to have the disassembly in a text file for further examination.
Disassemble all examples:
make -j `nproc` objdump
Disassemble one example:
make add.objdump
Examine one disassembly:
less -p asm_main add.objdump
This jumps directly to asm_main, which is what you likely want to see.
Disassembly is still useful even though we are writing assembly because the assembler can do some non-obvious magic that we want to understand.
2.5. crosstool-NG toolchain
Currently we build just Binutils from source, but use the host GCC to save time.
This could lead to incompatibilities, although we haven’t observed any so far.
crosstool-NG is a set of scripts that makes it easy to obtain a cross compiled GCC. Ideally we should track it here as a submodule and automate from there.
You can build the toolchain with crosstool-NG as explained at: https://stackoverflow.com/revisions/51310756/6
Then run this repo with:
make \ CTNG=crosstool-ng/.build/ct_prefix \ PREFIX=arm-cortex_a15-linux-gnueabihf \ test \ ;
2.6. Build the documentation
If you don’t like reading on GitHub, the HTML documentation can be generated from the README with:
make doc xdg-open out/README.html
2.7. Custom build flags
E.g., to pass -static for an emulator that does not support dynamically linked executables like gem5:
make CCFLAGS_CLI=-static
3. Theory
3.1. Architecture of this repo
qemu-arm-static is used for emulation on x86 hosts. It translates ARM to x86, and forwards system calls to the host kernel.
OS portability is achieved with the C standard library which makes system calls for us: this would in theory work in operating systems other than Linux if you port the build system to them.
Using the standard library also allows us to use its convenient functionality such as printf formatting and memcpy to check memory.
Non-OS portable examples will be clearly labeled with their OS.
These examples show how our infrastructure works:
3.1.1. C driver
We link all examples against a C program: main.c. Sample simplified commands:
arm-linux-gnueabihf-gcc -c -o 'main.o' 'main.c' arm-linux-gnueabihf-gcc -c -o 'sub.o' 'sub.S' arm-linux-gnueabihf-gcc -o 'sub.out' 'sub.o' main.o
The C driver then just calls asm_main, which each .S example implements.
This allows us to easily use the C standard library portably: from the point of view of GCC, everything looks like a regular C program, which does the required glibc initialization before main().
4. CONTRIBUTING
4.1. Update QEMU
git -C qemu pull make -B -C v7 qemu make -B -C v8 qemu
If the build fails due to drastic QEMU changes, first do:
make qemu-clean
Then make sure that the tests still pass:
make test
4.2. Bare metal
This tutorial only covers userland concepts.
However, certain instructions can only be used in higher privilege levels from an operating system itself.
Here is a base setup ARM programming without an operating system, also known as "Bare Metal Programming": https://github.com/cirosantilli/linux-kernel-module-cheat/tree/7d6f8c3884a4b4170aa274b986caae55b1bebaaf#baremetal-setup
Features:
-
clean crosstool-NG build for GCC
-
C standard library powevered by Newlib
-
works on both QEMU and gem5
Here are further links:
-
generic:
-
https://stackoverflow.com/questions/38914019/how-to-make-bare-metal-arm-programs-and-run-them-on-qemu/50981397#50981397 generic QEMU question
-
https://github.com/freedomtan/aarch64-bare-metal-qemu/tree/2ae937a2b106b43bfca49eec49359b3e30eac1b1:
-M virtUART bare metal hello world, nothing else, just works -
https://github.com/bravegnu/gnu-eprog Not tested.
-
https://stackoverflow.com/questions/29837892/how-to-run-a-c-program-with-no-os-on-the-raspberry-pi/40063032#40063032 no QEMU restriction
-
https://github.com/cirosantilli/raspberry-pi-bare-metal-blinker minimal, but not very QEMU friendly however because hard to observe LED: https://raspberrypi.stackexchange.com/questions/56373/is-it-possible-to-get-the-state-of-the-leds-and-gpios-in-a-qemu-emulation-like-t
-
-
raspberry PI:
-
https://raspberrypi.stackexchange.com/questions/34733/how-to-do-qemu-emulation-for-bare-metal-raspberry-pi-images/85135#85135 RPI3 specific
-
https://github.com/bztsrc/raspi3-tutorial, getting started: https://raspberrypi.stackexchange.com/questions/34733/how-to-do-qemu-emulation-for-bare-metal-raspberry-pi-images/85135#85135
-
-
gem5:
-
https://github.com/tukl-msd/gem5.bare-metal bare metal UART example. Tested with: https://stackoverflow.com/questions/43682311/uart-communication-in-gem5-with-arm-bare-metal/50983650#50983650
-
-
games:
x86 bare metal tutorial at: https://github.com/cirosantilli/x86-bare-metal-examples