1# Debugging protected VMs
2
3AVF is largely about protected VMs. This in turn means that anything that is
4happening inside the VM cannot be observed from outside of the VM. But as a
5developer, you need to be able to look into it when there’s an error in your
6VM. To satisfy such contradictory needs, AVF allows you to start a protected VM
7in a debuggable mode and provides a bunch of debugging mechanisms you can use
8to better understand the behavior of the VM and diagnose issues.
9
10Note: running a protected VM in a debuggable mode introduces many loopholes
11which can be used to nullify the protection provided by the hypervisor.
12Therefore, the debugable mode should never be used in production.
13
14## Enable debugging
15
16The following sections describe the two ways debugging can be enabled.
17
18### Debug level
19
20Debug level is a per-VM property which indicates how debuggable the VM is.
21There currently are two levels defined: NONE and FULL. NONE means that the VM
22is not debuggable at all, and FULL means that [all the debugging
23features](#debugging-features) are supported.
24
25Debug level is by default NONE. You can set it to FULL either via a Java API
26call in your app or via a command line argument `--debug` as follows:
27
28```java
29VirtualMachineConfig.Builder.setDebugLevel(DEBUG_LEVEL_FULL);
30```
31
32or
33
34```shell
35adb shell /apex/com.android.virt/bin/vm run-microdroid --debug full
36```
37
38or
39
40```shell
41m vm_shell
42vm_shell start-microdroid --auto-connect -- --protected --debug full
43```
44
45Note: `--debug full` is the default option when omitted. You need to explicitly
46use `--debug none` to set the debug level to NONE.
47
48### Debug policy
49
50Debug policy is a per-device property which forcibly enables selected debugging
51features, even for the VMs with debug level NONE.
52
53The main purpose of debug policy is in-field debugging by the platform
54developers (device makers, SoC vendors, etc.) To understand it, let’s imagine
55that you have an application of pVM. It’s configured as debug level NONE
56because you finished the development and the team-level testing. However, you
57get a bug report from your QA team or from beta testers. To fix the bug, you
58should be able to look into the pVM but you do not want to change the source
59code to make the VM debuggable and rebuild the entire software, because that
60may hurt the reproducibility of the bug.
61
62Note: Not every devices is guaranteed to support debug policy. It is up to the
63device manufacturer to implement this in their bootloader. Google Pixel
64devices for example support this after Pixel 7 and 7 Pro. Pixel 6 and 6 Pro
65don't support debug policy.
66
67In the Pixel phones supporting debug policy, it is provisioned by installing a
68device tree overlay like below to the Pixel-specific partition `dpm`.
69
70```
71/ {
72    fragment@avf {
73        target-path = "/";
74
75        __overlay__ {
76            avf {
77                guest {
78                    common {
79                        log = <1>; // Enable kernel log and logcat
80                        ramdump = <1>; // Enable ramdump
81                    }
82                    microdroid {
83                        adb = <1>; // Enable ADB connection
84                    }
85                }
86            };
87        };
88    };
89}; /* end of avf */
90```
91
92To not enable a specific debugging feature, set the corresponding property
93value to other than `<1>`, or delete the property.
94
95As a reference, in Pixel phones, debug policy is loaded as below:
96
971. Bootloader loads it from the `dpm` partition and verifies it.
981. Bootloader appends the loaded debug policy as the [configuration
99   data](../../pvmfw/README.md#configuration-data) of the pvmfw.
1001. When a pVM is started, pvmfw [overlays][apply_debug_policy] the debug policy to the baseline
101   device tree from crosvm.
1021. OS payload (e.g. Microdroid) [reads][read_debug_policy] the device tree and enables specific
103   debugging feature accordingly.
104
105**Note**: Bootloader MUST NOT load debug policy when the bootloader is in LOCKED state.
106
107[apply_debug_policy]: https://cs.android.com/android/platform/superproject/main/+/main:packages/modules/Virtualization/pvmfw/src/fdt.rs;drc=0d52747770baa14d44c0779b5505095b4251f2e9;l=790
108[read_debug_policy]: https://cs.android.com/android/platform/superproject/main/+/main:packages/modules/Virtualization/microdroid_manager/src/main.rs;drc=65c9f1f0eee4375535f2025584646a0dbb0ea25c;l=834
109
110## Debugging features
111
112AVF currently supports the following debugging features:
113
114* ADB connection (only for Microdroid)
115* Capturing console output
116* Capturing logcat output (only for Microdroid)
117* [Capturing kernel ramdump](ramdump.md) (only for Microdroid)
118* Capturing userspace crash dump (only for Microdroid)
119* [Attaching GDB to the kernel](gdb_kernel.md)
120* [Attaching GDB to the userspace process](gdb_userspace.md) (only for Microdroid)
121* [Tracing hypervisor events](tracing.md)
122