1# Create and run a suite
2
3A suite allows one to run a group of tests for a given build. Adding or removing
4individual test from a suite can be as simple as a test config file change. One
5use case is that we can easily maintain a collection of stable tests for
6presubmit check. If a test becomes unstable, we can make a single test config
7change to pull it out of presubmit check.
8
9A test can also be easily added to multiple suites by specifying the suite name
10in the test config file. So one doesn't need to maintain a collection of test in
11complicated GCL files, for a test to run in multiple suites.
12
13## Create a suite
14
151.  Create a suite configuration file
16
17Suite configuration file is similar to a TradeFed test configuration file. It
18should always use "TfSuiteRunner" as runner and the only option needed is
19"run-suite-tag", which defines the name of the suite. Any test configuration
20file has option "test-suite-tag" set to that value will be included in that
21suite.
22
23A sample suite configuration file looks like this:
24
25
26```
27<configuration description="Android framework-base-presubmit test suite config.">
28    <test class="com.android.tradefed.testtype.suite.TfSuiteRunner">
29        <option name="run-suite-tag" value="framework-base-presubmit" />
30    </test>
31</configuration>
32```
33
341.  Add test to suite
35
36To include a test in the suite, it must be built to device-tests or
37general-tests artifact. That is, include following in the mk file:
38
39```
40LOCAL_COMPATIBILITY_SUITE := device-tests
41```
42
43The test configuration file should also have option "test-suite-tag" set to the
44value of the suite name, for example:
45
46```
47<option name="test-suite-tag" value="framework-base-presubmit" />
48```
49
50## Run a suite
51
52### Run a suite in local build
53
54After you build target tradefed-all (to include the new test suite
55config), and the test targets (SystemUI and FrameworksNotificationTests in the
56above example), you can run the suite locally. Running a suite job is similar to
57running a test. The TradeFed command line looks like.
58
59```
60ANDROID_TARGET_OUT_TESTCASES=$ANDROID_TARGET_OUT_TESTCASES \
61  tradefed.sh run template/local --template:map \
62  test=suite/framework-base-presubmit --log-level-display VERBOSE
63```
64
65Before running that command line, you should have ran lunch command to set up
66the environment properly.
67
68### Run a suite with remote build
69
70To run a suite with remote build, you need to provide build arguments, e.g.,
71--branch, --build-id etc. A sample command is as follows:
72
73```
74tradefed.sh run google/template/lab-base \
75  --template:map test=suite/framework-base-presubmit --test-tag SUITE \
76  --branch git_main --build-flavor angler-userdebug \
77  --build-os fastbuild3b_linux --build-id $BUILD_ID \
78  --test-zip-file-filter .*device-tests.zip
79```
80
81"test-zip-file-filter" option tells TradeFed to download and extract all device
82tests into a local build provider, so test configs can be located for the given
83suite.
84