1# CrosVM Continuous Integration
2
3Crosvm has a complex set of dependencies and requirements on the host machine to
4successfully build and run test cases. To allow for consistent testing in our
5continuous integration system (kokoro) and reproduction of those tests locally,
6we provide docker containers containing the build toolchain and a VM for
7testing.
8
9## How to run crosvm tests
10
11### Setting up the source
12
13Since crosvm is part of chromiumos, and uses a couple of it's projects as
14dependencies, you need a standard chromiumos checkout as described by the
15[ChromiumOS Developer Guide](https://chromium.googlesource.com/chromiumos/docs/+/master/developer_guide.md#Get-the-Source).
16
17To reduce the number of repositories to download, you can use the `-g crosvm`
18argument on `repo init`. This will be significantly faster:
19
20In summary:
21
22```
23$ repo init -u https://chromium.googlesource.com/chromiumos/manifest.git --repo-url https://chromium.googlesource.com/external/repo.git -g crosvm
24$ repo sync -j4
25$ cd src/platform/crosvm
26```
27
28### Installing Podman (or Docker)
29
30See [Podman Installation](https://podman.io/getting-started/installation) for
31instructions on how to install podman.
32
33For Googlers, see [go/dont-install-docker](http://go/dont-install-docker) for
34special instructions on how to set up podman.
35
36If you already have docker installed, that will do as well. However podman is
37recommended as it will not run containers with root privileges.
38
39### Running all tests
40
41To run all tests, just run:
42
43```
44./test_all
45```
46
47This will run all tests using the x86 and aarch64 builder containers. What does
48this do?
49
501. It will start `./ci/[aarch64_]builder --vm`.
51
52   The builder will build ChromeOS dependencies from your local repo checkout.
53   If you make modifications to these dependencies (e.g. minijail, tpm2, cras)
54   these will be included in tests.
55
56   Then it will start a VM for running tests in the background. The VM is
57   booting while the next step is running.
58
592. Then it will call `./run_tests` inside the builder
60
61   The script will pick which tests to execute and where. Simple tests can be
62   executed directly, other tests require privileged access to devices and will
63   be loaded into the VM to execute.
64
65   Each test will in the end be executed by a call to
66   `cargo test -p crate_name`.
67
68Intermediate build data is stored in a scratch directory at `./target/ci/` to
69allow for faster subsequent calls (Note: If running with docker, these files
70will be owned by root).
71
72### Fast, iterative test runs
73
74For faster iteration time, you can directly invoke some of these steps directly:
75
76To only run x86 tests: `./ci/[aarch64_]builder --vm ./run_tests`.
77
78To run a simple test (e.g. the tempfile crate) that does not need the vm:
79`./ci/[aarch64_]builder cargo test -p tempfile`.
80
81Or run a single test (e.g. kvm_sys) inside the vm:
82`./ci/[aarch64*]builder --vm cargo test -p kvm_sys`.
83
84Since the VM (especially the fully emulated aarch64 VM) can be slow to boot, you
85can start an interactive shell and run commands from there as usual. All cargo
86tests will be executed inside the VM, without the need to restart the VM between
87test calls.
88
89```
90host$ ./ci/aarch64_builder --vm
91crosvm-aarch64$ ./run_tests
92crosvm-aarch64$ cargo test -p kvm_sys
93...
94```
95
96### Reproducing Kokoro locally
97
98Kokoro uses the same builders and the same `run_tests` script to run tests.
99However, to keep the build stable, it syncs the chromiumos checkout to the fixed
100manifest found at `./ci/kokoro/manifest.xml`.
101
102To run tests using the same manifest, as well as the same build process that
103Kokoro uses, you can run: `./ci/kokoro/simulate_all`.
104
105## Implementation Overview
106
107Directories:
108
109- ci/build_environment: Contains tooling for building the dependencies of
110  crosvm.
111- ci/crosvm_aarch64_builder: An x86 docker image to cross-compile for aarch64
112  and test with user-space emulation.
113- ci/crosvm_base: Docker image shared by crosvm_builder and
114  crosvm_aarch64_builder
115- ci/crosvm_builder: A native docker image for building and testing crosvm
116- ci/crosvm_test_vm: Dockerfile to build the VM included in the builder
117  containers.
118- ci/kokoro: Configuration files and build scripts used by Kokoro to run crosvm
119  tests.
120
121Scripts:
122
123- ci/aarch64_builder: Script to start the crosvm_aarch64_builder container
124- ci/builder: Script to start the crosvm_builder container
125- ci/run_container.sh: Implementation behind the above scripts.
126- test_runner.py: Implementation behind the `./test_all` script.
127
128### Building and uploading a new version of builders
129
130The docker images for all builders can be built with `make` and uploaded with
131`make upload`. Of course you need to have docker push permissions for
132`gcr.io/crosvm-packages/` for the upload to work.
133