1# Setup: Linux host, QEMU vm, arm64 kernel
2
3This document will detail the steps involved in setting up a Syzkaller instance fuzzing any ARM64 linux kernel of your choice.
4
5## Create a disk image
6
7We will use buildroot to create the disk image.
8You can obtain buildroot from [here](https://buildroot.uclibc.org/download.html).
9Extract the tarball and perform a `make menuconfig` inside it.
10Choose the following options.
11
12    Target options
13	    Target Architecture - Aarch64 (little endian)
14    Toolchain type
15	    External toolchain - Linaro AArch64
16    System Configuration
17    [*] Enable root login with password
18            ( ) Root password ⇐= set your password using this option
19    [*] Run a getty (login prompt) after boot  --->
20	    TTY port - ttyAMA0
21    Target packages
22	    [*]   Show packages that are also provided by busybox
23	    Networking applications
24	        [*] dhcpcd
25	        [*] iproute2
26	        [*] openssh
27    Filesystem images
28	    [*] ext2/3/4 root filesystem
29	        ext2/3/4 variant - ext3
30	        exact size in blocks - 6000000
31	    [*] tar the root filesystem
32
33Run `make`. After the build, confirm that `output/images/rootfs.ext3` exists.
34
35## Get the ARM64 toolchain from Linaro
36
37You will require an ARM64 kernel with gcc plugin support.
38If not, obtain the ARM64 toolchain from Linaro.
39Get `gcc-linaro-6.1.1-2016.08-x86_64_aarch64-linux-gnu.tar.xz` from [here](https://releases.linaro.org/components/toolchain/binaries/latest/aarch64-linux-gnu/).
40Extract and add its `bin/` to your `PATH`.
41If you have another ARM64 toolchain on your machine, ensure that this newly downloaded toolchain takes precedence.
42
43## Compile the kernel
44
45Once you have obtained the source code for the linux kernel you wish to fuzz, do the following.
46
47    $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make defconfig
48    $ vim .config
49
50Change the following options :
51```
52    CONFIG_KCOV=y
53    CONFIG_KASAN=y
54    CONFIG_DEBUG_INFO=y
55    CONFIG_CMDLINE=”console=ttyAMA0”
56    CONFIG_KCOV_INSTRUMENT_ALL=y
57    CONFIG_DEBUG_FS=y
58    CONFIG_NET_9P=y
59    CONFIG_NET_9P_VIRTIO=y
60    CONFIG_CROSS_COMPILE="aarch64-linux-gnu-"
61```
62```
63    $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j40
64```
65
66If the build was successful, you should have a `arch/arm64/boot/Image` file.
67
68## Obtain qemu for ARM64
69
70Obtain the QEMU source from git or from the latest source release.
71
72    $ ./configure
73    $ make -j40
74
75If the build was successful, you should have a `aarch64-softmmu/qemu-system-aarch64` binary.
76
77## Boot up manually
78
79You should be able to start up the kernel as follows.
80
81    $ /path/to/aarch64-softmmu/qemu-system-aarch64 \
82      -machine virt \
83      -cpu cortex-a57 \
84      -nographic -smp 1 \
85      -hda /path/to/rootfs.ext3 \
86      -kernel /path/to/arch/arm64/boot/Image \
87      -append "console=ttyAMA0 root=/dev/vda oops=panic panic_on_warn=1 panic=-1 ftrace_dump_on_oops=orig_cpu debug earlyprintk=serial slub_debug=UZ" \
88      -m 2048 \
89      -net user,hostfwd=tcp::10023-:22 -net nic
90
91At this point, you should be able to see a login prompt.
92
93## Set up the QEMU disk
94
95Now that we have a shell, let us add a few lines to existing init scripts so that they are executed each time Syzkaller brings up the VM.
96
97At the top of /etc/init.d/S50sshd add the following lines:
98
99    ifconfig eth0 up
100    dhcpcd
101    mount -t debugfs none /sys/kernel/debug
102    chmod 777 /sys/kernel/debug/kcov
103
104Comment out the line
105
106    /usr/bin/ssh-keygen -A
107
108Next we set up ssh. Create an ssh keypair locally and copy the public key to `/authorized_keys` in `/`. Ensure that you do not set a passphrase when creating this key.
109
110Open `/etc/ssh/sshd_config` and modify the following lines as shown below.
111
112    PermitRootLogin yes
113    PubkeyAuthentication yes
114    AuthorizedKeysFile      /authorized_keys
115    PasswordAuthentication yes
116
117Reboot the machine, and ensure that you can ssh from host to guest as.
118
119    $ ssh -i /path/to/id_rsa root@localhost -p 10023
120
121## Build syzkaller
122
123    make TARGETARCH=arm64 [CC=/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++]
124
125## Modify your config file and start off syzkaller
126
127A sample config file that exercises the required options are shown below. Modify according to your needs.
128
129```
130{
131    "name": "QEMU-aarch64",
132    "target": "linux/arm64",
133    "http": ":56700",
134    "workdir": "/path/to/a/dir/to/store/syzkaller/corpus”,
135    "kernel_obj": “/path/to/linux/build/dir",
136    "syzkaller": "/path/to/syzkaller/arm64/",
137    "image": "/path/to/rootfs.ext3",
138    "sshkey": "/path/to/ida_rsa",
139    "procs": 8,
140    "type": "qemu",
141    "vm": {
142        "count": 1,
143        "qemu": "/path/to/qemu-system-aarch64",
144        "cmdline": "console=ttyAMA0 root=/dev/vda",
145        "kernel": “/path/to/Image",
146        "cpu": 2,
147        "mem": 2048
148    }
149}
150```
151
152At this point, you should be able to visit `localhost:56700` and view the results of the fuzzing.
153
154If you get issues after `syz-manager` starts, consider running it with the `-debug` flag.
155Also see [this page](/docs/troubleshooting.md) for troubleshooting tips.
156