1# ART Chroot-Based On-Device Testing
2
3This file documents the use of a chroot environment in on-device testing of the
4Android Runtime (ART). Using a chroot allows tests to run a standalone ART from
5a locally built source tree on a device running (almost any) system image and
6does not interfere with the Runtime installed in the device's system partition.
7
8## Introduction
9
10The Android Runtime (ART) supports testing in a chroot-based environment, by
11setting up a chroot directory in a `ART_TEST_CHROOT` directory located under
12`/data/local` (e.g. `ART_TEST_CHROOT=/data/local/art-test-chroot`) on a device,
13installing ART and all other required artifacts there, and having tests use `adb
14shell chroot $ART_TEST_CHROOT <command>` to execute commands on the device
15within this environment.
16
17This way to run tests using a "standalone ART" ("guest system") only affects
18files in the data partition (the system partition and other partitions are left
19untouched) and is as independent as possible from the Android system ("host
20system") running on the device. This has some benefits:
21
22* no need to build and flash a whole device to do ART testing (or "overwriting"
23  an existing ART by syncing the system partition);
24* the possibility to use a smaller AOSP Android manifest
25  ([`master-art`](https://android.googlesource.com/platform/manifest/+/refs/heads/master-art/default.xml))
26  to build ART and the required dependencies for testing;
27* no instability due to updating/replacing ART on the system partition (a
28  functional Android Runtime is necessary to properly boot a device);
29* the possibility to have several standalone ART instances (one per directory,
30  e.g. `/data/local/art-test-chroot1`, `/data/local/art-test-chroot2`, etc.).
31
32Note that using this chroot-based approach requires root access to the device
33(i.e. be able to run `adb root` successfully).
34
35## Quick User Guide
36
370. Unset variables which are not used with the chroot-based approach (if they
38   were set previously):
39   ```bash
40   unset ART_TEST_ANDROID_ROOT
41   unset CUSTOM_TARGET_LINKER
42   unset ART_TEST_ANDROID_ART_ROOT
43   unset ART_TEST_ANDROID_RUNTIME_ROOT
44   unset ART_TEST_ANDROID_I18N_ROOT
45   unset ART_TEST_ANDROID_TZDATA_ROOT
46   ```
471. Set the chroot directory in `ART_TEST_CHROOT`:
48    ```bash
49    export ART_TEST_CHROOT=/data/local/art-test-chroot
50    ```
512. Set lunch target and ADB:
52    * With a minimal `aosp/master-art` tree:
53        1. Initialize the environment:
54            ```bash
55            export SOONG_ALLOW_MISSING_DEPENDENCIES=true
56            . ./build/envsetup.sh
57            ```
58        2. Select a lunch target corresponding to the architecture you want to
59           build and test:
60            * For (32-bit) Arm:
61                ```bash
62                lunch arm_krait-trunk_staging-eng
63                ```
64            * For (64-bit only) Arm64:
65                ```bash
66                lunch armv8-trunk_staging-eng
67                ```
68            * For (32- and 64-bit) Arm64:
69                ```bash
70                lunch arm_v7_v8-trunk_staging-eng
71                ```
72            * For (32-bit) Intel x86:
73                ```bash
74                lunch silvermont-trunk_staging-eng
75                ```
76            * For (64-bit) RISC-V:
77                ```bash
78                lunch aosp_riscv64-trunk_staging-eng
79                ```
80        3. Set up the environment to use a pre-built ADB:
81            ```bash
82            export PATH="$(pwd)/prebuilts/runtime:$PATH"
83            export ADB="$ANDROID_BUILD_TOP/prebuilts/runtime/adb"
84            ```
85    * With a full Android (AOSP) `aosp/main` tree:
86        1. Initialize the environment:
87            ```bash
88            . ./build/envsetup.sh
89            ```
90        2. Select a lunch target corresponding to the architecture you want to
91           build and test:
92            * For (32-bit) Arm:
93                ```bash
94                lunch aosp_arm-trunk_staging-eng
95                ```
96            * For (32- and 64-bit) Arm64:
97                ```bash
98                lunch aosp_arm64-trunk_staging-eng
99                ```
100            * For (32-bit) Intel x86:
101                ```bash
102                lunch aosp_x86-trunk_staging-eng
103                ```
104            * For (32- and 64-bit) Intel x86-64:
105                ```bash
106                lunch aosp_x86_64-trunk_staging-eng
107                ```
108            * For (64-bit) RISC-V:
109                ```bash
110                lunch aosp_riscv64-trunk_staging-eng
111                ```
112        3. Build ADB:
113            ```bash
114            m adb
115            ```
1163. Build ART and required dependencies:
117    ```bash
118    art/tools/buildbot-build.sh --target
119    ```
120    After building it is fine to see it finish with an error like:
121    ```
122    linkerconfig E [...] variableloader.cc:83] Unable to access VNDK APEX at path: <path>: No such file or directory
123    ```
1244. Clean up the device:
125    ```bash
126    art/tools/buildbot-cleanup-device.sh
127    ```
1285. Setup the device (including setting up mount points and files in the chroot
129   directory):
130    ```bash
131    art/tools/buildbot-setup-device.sh
132    ```
1336. Populate the chroot tree on the device (including "activating" APEX packages
134   in the chroot environment):
135    ```bash
136    art/tools/buildbot-sync.sh
137    ```
1387. Run ART gtests:
139    ```bash
140    art/tools/run-gtests.sh -j4
141    ```
142    * Specific tests to run can be passed on the command line, specified by
143      their absolute paths beginning with `/apex/`:
144        ```bash
145        art/tools/run-gtests.sh \
146          /apex/com.android.art/bin/art/arm64/art_cmdline_tests \
147          /apex/com.android.art/bin/art/arm64/art_dexdump_tests
148        ```
149    * Gtest options can be passed to each gtest by passing them after `--`; see
150      the following examples.
151        * To print the list of all test cases of a given gtest, use option
152          `--gtest_list_tests`:
153            ```bash
154            art/tools/run-gtests.sh \
155              /apex/com.android.art/bin/art/arm64/art_cmdline_tests \
156              -- --gtest_list_tests
157            ```
158        * To filter the test cases to execute, use option `--gtest_filter`:
159            ```bash
160            art/tools/run-gtests.sh \
161              /apex/com.android.art/bin/art/arm64/art_cmdline_tests \
162              -- --gtest_filter="*TestJdwp*"
163            ```
164        * To see all the options supported by a gtest, use option `--help`:
165            ```bash
166            art/tools/run-gtests.sh \
167              /apex/com.android.art/bin/art/arm64/art_cmdline_tests \
168              -- --help
169            ```
170    * Note: Some test cases of `art_runtime_tests` defined in
171    `art/runtime/gc/space/image_space_test.cc` may fail when using the full AOSP
172    tree (b/119815008).
173        * Workaround: Run `m clean-oat-host` before the build step
174        (`art/tools/buildbot-build.sh --target`) above.
175    * Note: The `-j` option of script `art/tools/run-gtests.sh` is not honored
176      yet (b/129930445). However, gtests themselves support parallel execution,
177      which can be specified via the gtest option `-j`:
178        ```bash
179        art/tools/run-gtests.sh -- -j4
180        ```
1818. Run ART run-tests:
182    * On a 64-bit target:
183        ```bash
184        art/test/testrunner/testrunner.py --target --64
185        ```
186    * On a 32-bit target:
187        ```bash
188        art/test/testrunner/testrunner.py --target --32
189        ```
1909. Run Libcore tests:
191    * On a 64-bit target:
192        ```bash
193        art/tools/run-libcore-tests.sh --mode=device --variant=X64
194        ```
195    * On a 32-bit target:
196        ```bash
197        art/tools/run-libcore-tests.sh --mode=device --variant=X32
198        ```
19910. Run JDWP tests:
200    * On a 64-bit target:
201        ```bash
202        art/tools/run-libjdwp-tests.sh --mode=device --variant=X64
203        ```
204    * On a 32-bit target:
205        ```bash
206        art/tools/run-libjdwp-tests.sh --mode=device --variant=X32
207        ```
20811. Tear down device setup:
209    ```bash
210    art/tools/buildbot-teardown-device.sh
211    ```
21212. Clean up the device:
213    ```bash
214    art/tools/buildbot-cleanup-device.sh
215    ```
216