• Stars
    star
    1,345
  • Rank 34,927 (Top 0.7 %)
  • Language
  • License
    GNU General Publi...
  • Created over 3 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A series of posts about QEMU internals:

Introduction

This is a series of posts about QEMU internals. It won't cover everything about QEMU, but should help you understand how it works and foremost how to hack into it for fun and profit.

We won't explain usage and other things that can be found in the official documentation. The following topics will be addressed:

The official code and documentation can be found here:

Terminology

Host and target

The host is the plaform and architecture which QEMU is running on. Usually an x86 machine.

The target is the architecture which is emulated by QEMU. You can choose at build time which one you want:

./configure --target-list=ppc-softmmu ...

As such, in the source code organisation you will find all supported architectures in the target/ directory:

(qemu-git) ll target
drwxrwxr-x 2 xxx xxx 4.0K alpha
drwxrwxr-x 2 xxx xxx 4.0K arm
drwxrwxr-x 2 xxx xxx 4.0K cris
drwxrwxr-x 2 xxx xxx 4.0K hppa
drwxrwxr-x 3 xxx xxx 4.0K i386
drwxrwxr-x 2 xxx xxx 4.0K lm32
drwxrwxr-x 2 xxx xxx 4.0K m68k
drwxrwxr-x 2 xxx xxx 4.0K microblaze
drwxrwxr-x 2 xxx xxx 4.0K mips
drwxrwxr-x 2 xxx xxx 4.0K moxie
drwxrwxr-x 2 xxx xxx 4.0K nios2
drwxrwxr-x 2 xxx xxx 4.0K openrisc
drwxrwxr-x 3 xxx xxx 4.0K ppc
drwxr-xr-x 3 xxx xxx 4.0K riscv
drwxrwxr-x 2 xxx xxx 4.0K s390x
drwxrwxr-x 2 xxx xxx 4.0K sh4
drwxrwxr-x 2 xxx xxx 4.0K sparc
drwxrwxr-x 2 xxx xxx 4.0K tilegx
drwxrwxr-x 2 xxx xxx 4.0K tricore
drwxrwxr-x 2 xxx xxx 4.0K unicore32
drwxrwxr-x 9 xxx xxx 4.0K xtensa

The qemu-system-<target> binaries are built into their respective <target>-softmmu directory:

(qemu-git) ls -ld *-softmmu
drwxr-xr-x  9 xxx xxx 4096 i386-softmmu
drwxrwxr-x 11 xxx xxx 4096 ppc-softmmu
drwxr-xr-x  9 xxx xxx 4096 x86_64-softmmu

System and user modes

QEMU is a system emulator. It offers emulation of a lot of architectures and can be run on a lot of architectures.

It is able to emulate a full system (cpu, devices, kernel and apps) through the qemu-system-<target> command line tool. This is the mode we will dive into.

It also provides a userland emulation mode through the qemu-<target> command line tool.

This allows to directly run <target> architecture Linux binaries on a Linux host. It mainly emulates <target> instructions set and forward system calls to the host Linux kernel. The emulation is only related to user level cpu instructions, not system ones, no device nore low level memory handling.

We won't cover qemu user mode in this blog post series.

Emulation, JIT and virtualization

Initially QEMU was an emulation engine, with a Just-In-Time compiler (TCG). The TCG is here to dynamically translate target instruction set architecture (ISA) to host ISA.

We will later see that in the context of the TCG, the tcg-target becomes the architecture to which the TCG has to generate final assembly code to run on (which is host ISA). Obvious !

There exists scenario where target and host architectures are the same. This is typically the case in classical virtualization environment (VMware, VirtualBox, ...) when a user wants to run Windows on Linux for instance. The terminology is usually Host and Guest (target).

Nowadays, QEMU offers virtualization through different accelerators. Virtualization is considered an accelerator because it prevents unneeded emulation of instructions when host and target share the same architecture. Only system level (aka supervisor/ring0) instructions might be emulated/intercepted.

Of course, the QEMU virtualization capabilities are tied to the host OS and architecture. The x86 architecture offers hardware virtualization extensions (Intel VMX/AMD SVM). But the host operating system must allow QEMU to take benefit of them.

Under an x86-64 Linux host, we found the following accelerators:

$ qemu-system-x86_64 -accel ?
Possible accelerators: kvm, xen, hax, tcg

While on an x86-64 MacOS host:

$ qemu-system-x86_64 -accel ?
Possible accelerators: tcg, hax, hvf

The supported accelerators can be found in qemu_init_vcpu():

void qemu_init_vcpu(CPUState *cpu)
{
...
    if (kvm_enabled()) {
        qemu_kvm_start_vcpu(cpu);
    } else if (hax_enabled()) {
        qemu_hax_start_vcpu(cpu);
    } else if (hvf_enabled()) {
        qemu_hvf_start_vcpu(cpu);
    } else if (tcg_enabled()) {
        qemu_tcg_init_vcpu(cpu);
    } else if (whpx_enabled()) {
        qemu_whpx_start_vcpu(cpu);
    } else {
        qemu_dummy_start_vcpu(cpu);
    }
...
}

To make it short:

  • kvm is the Linux Kernel-based Virtual Machine accelerator;
  • hvf is the MacOS Hypervisor.framework accelerator;
  • hax is the cross-platform Intel HAXM accelerator;
  • whp is the Windows Hypervisor Platform accelerator.

You can take benefit of the speed of x86 hardware virtualization under the three major operating systems. Notice that the TCG is also considered an accelerator. We can enter a long debate about terminology here ...

QEMU APIs

There exists a lot of APIs in QEMU, some are obsolete and not well documented. Reading the source code still remains your best option. There is a good overview available.

The posts series will mainly address QOM, qdev and VMState. The QOM is the more abstract one. While QEMU is developped in C language, the developpers chose to implement the QEMU Object Model to provide a framework for registering user creatable types and instantiating objects from those types: device, machine, cpu, ... People used to OOP concepts will find their mark in the QOM.

We will briefly illustrate how to make use of it, but won't detail its underlying implementation. Stay pragmatic !

The interested reader can have a look at include/qom/object.h.

Disclaimer

It shall be noted that Airbus does not commit itself on the exhaustiveness and completeness regarding this blog post series. The information presented here results from the author knowledge and understandings as of QEMU v4.2.0.

More Repositories

1

bincat

Binary code static analyser, with IDA integration. Performs value and taint analysis, type reconstruction, use-after-free and double-free detection
OCaml
1,662
star
2

cpu_rec

Recognize cpu instructions in an arbitrary binary file
Python
640
star
3

ilo4_toolbox

Toolbox for HPE iLO4 & iLO5 analysis
Python
412
star
4

warbirdvm

An analysis of the Warbird virtual-machine protection for the CI!g_pStore
Ruby
216
star
5

diffware

An extensively configurable tool providing a summary of the changes between two files or directories, ignoring all the fluff you don't care about.
Python
196
star
6

gustave

GUSTAVE is a fuzzing platform for embedded OS kernels. It is based on QEMU and AFL (and all of its forkserver siblings). It allows to fuzz OS kernels like simple applications.
Python
194
star
7

powersap

Powershell SAP assessment tool
PowerShell
187
star
8

crashos

A tool dedicated to the research of vulnerabilities in hypervisors by creating unusual system configurations.
C
182
star
9

c-compiler-security

Security-related flags and options for C compilers
Python
179
star
10

ramooflax

a bare metal (type 1) VMM (hypervisor) with a python remote control API
C
178
star
11

bta

Open source Active Directory security audit framework.
Python
131
star
12

android_emuroot

Android_Emuroot is a Python script that allows granting root privileges on the fly to shells running on Android virtual machines that use google-provided emulator images called Google API Playstore, to help reverse engineers to go deeper into their investigations.
Python
121
star
13

AutoResolv

Python
71
star
14

elfesteem

ELF/PE/Mach-O parsing library
Python
50
star
15

GEA1_break

Implementation of the key recovery attack against GEA-1 keys (Eurocrypt 2021)
C
47
star
16

airbus-seclab.github.io

Conferences, tools, papers, etc.
43
star
17

AFLplusplus-blogpost

Blogpost about optimizing binary-only fuzzing with AFL++
Shell
34
star
18

nbutools

Tools for offensive security of NetBackup infrastructures
Python
30
star
19

rebus

REbus facilitates the coupling of existing tools that perform specific tasks, where one's output will be used as the input of others.
Python
25
star
20

usbq_core

USB man in the middle linux kernel driver
C
19
star
21

AppVsWild

application process protection hypervisor virtualization encryption
9
star
22

gunpack

Generic unpacker (dynamic)
C
8
star
23

usbq_userland

User land program to be used with usbq_core
Python
8
star
24

ramooflax_scripts

ramooflax python scripts
Python
6
star
25

cpu_doc

Curated set of documents about CPU
3
star
26

c2newspeak

C
3
star
27

rebus_demo

REbus demo agents
Python
2
star
28

security-advisories

2
star
29

pwnvasive

semi-automatic discovery and lateralization
Python
1
star
30

pok

forked from pok-kernel/pok
C
1
star
31

afl

Airbus seclab fork of AFL
C
1
star