1# Testing Perfetto 2 3The testing strategy for Perfetto is rather complex due to the wide variety 4of build configurations and embedding targets. 5 6Common test targets (all platforms / checkouts): 7 8`perfetto_unittests`: 9Platform-agnostic unit-tests. 10 11`perfetto_integrationtests`: 12End-to-end tests, involving the protobuf-based IPC transport and ftrace 13integration (Linux/Android only). 14 15`perfetto_benchmarks`: 16Benchmarks tracking the performance of: (i) trace writing, (ii) trace readback 17and (iii) ftrace raw pipe -> protobuf translation. 18 19Running tests on Linux / MacOS 20------------------------------ 21``` 22$ tools/ninja -C out/default perfetto_{unittests,integrationtests,benchmarks} 23$ out/default/perfetto_unittests --gtest_help 24``` 25 26`perfetto_integrationtests` requires that the ftrace debugfs directory is 27is readable/writable by the current user on Linux: 28``` 29sudo chown -R $USER /sys/kernel/debug/tracing 30``` 31 32Running tests on Android 33------------------------ 341A) Connect a device through `adb` 351B) Start the build-in emulator (supported on Linux and MacOS): 36``` 37$ tools/install-build-deps 38$ tools/run_android_emulator & 39``` 40 412) Run the tests (either on the emulator or physical device): 42``` 43$ tools/run_android_test out/default perfetto_unittests 44``` 45 46 47Continuous testing 48------------------ 49Perfetto is tested in a variety of locations: 50 51**Travis CI**: https://perfetto-ci.appspot.com/ 52Builds and runs perfetto_{unittests,integrationtests,benchmarks} from then standalone checkout. Benchmarks are ran in a reduced form for smoke testing. 53 54**Android CI** (see go/apct and go/apct-guide): 55runs only `perfetto_integrationtests` 56 57**Android presubmits (TreeHugger)**: 58Runs before submission of every AOSP CL of `external/perfetto`. 59 60 61**Android CTS** (Android test suite used run to ensure API compatibility): Rolling runs internally. 62 63Note that Travis uses the standalone build system and the others build as 64part of the Android tree. 65 66Unit tests 67---------- 68Unit tests exist for most of the code in Perfetto on the class level. They 69ensure that each class broadly works as expected. 70 71Unit tests are currently ran only on Travis. 72Running unit tests on APCT and Treehugger is WIP. 73 74Integration tests 75----------------- 76Integration tests ensure that subsystems (importantly ftrace and the IPC layer) 77and Perfetto as a whole is working correctly end-to-end. 78 79There are two configurations in which integration tests can be run: 80 81**1. Production mode** (Android-only) 82This mode assumes that both the tracing service (`traced`) and the OS probes 83service (`traced_probes`) are already running. In this mode the test enables 84only the consumer endpoint and tests the interaction with the production 85services. This is the way our Android CTS and APCT tests work. 86 87**2. Standalone mode**: 88Starting up the daemons in the test itself and then testing against them. 89This is how standalone builds are tested. This is the only supported way to 90run integration tests on Linux and MacOS. 91 92Android CTS tests 93----------------- 94CTS tests ensure that any vendors who modify Android remain compliant with the 95platform API. 96 97These tests include a subset of the integration tests above as well as adding 98more complex tests which ensure interaction between platform (e.g. Android apps 99etc.) and Perfetto is not broken. 100 101The relevant targets are `CtsPerfettoProducerApp` and `CtsPerfettoTestCases`. Once these are built, the following commands should be run: 102``` 103adb push $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoTestCases64 /data/local/tmp/ 104adb install -r $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoProducerApp.apk 105``` 106 107Next, the app named `android.perfetto.producer` should be run on the device. 108 109Finally, the following command should be run: 110``` 111adb shell /data/local/tmp/CtsPerfettoTestCases64 112``` 113 114Chromium waterfall 115------------------ 116Coming soon! 117