1page.title=Trade Federation Overview
2@jd:body
3
4<!--
5    Copyright 2013 The Android Open Source Project
6
7    Licensed under the Apache License, Version 2.0 (the "License");
8    you may not use this file except in compliance with the License.
9    You may obtain a copy of the License at
10
11        http://www.apache.org/licenses/LICENSE-2.0
12
13    Unless required by applicable law or agreed to in writing, software
14    distributed under the License is distributed on an "AS IS" BASIS,
15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16    See the License for the specific language governing permissions and
17    limitations under the License.
18-->
19<p>Trade Federation (tradefed or TF for short) is a continuous test framework designed for running tests
20on Android devices. It's a Java application which runs on a host computer, and communicates to one or
21more Android devices using ddmlib (the library behind DDMS) over adb.</p>
22
23<p>We've listed some of TF's main features below, along with a couple sample usecases.  That said,
24if you want to jump right in and get started, you can head straight for the
25<a href="/devices/tech/test_infra/tradefed/fundamentals/index.html">Start Here</a> page.</p>
26
27<h2>Features</h2>
28<ul>
29<li>modular, flexible, scalable design</li>
30<li>has built in support for running many different types of Android tests:
31  <a href="http://developer.android.com/tools/testing/testing_android.html#Instrumentation">instrumentation</a>,
32  <a href="http://developer.android.com/tools/testing/testing_ui.html">uiautomator</a>,
33  native/gtest, host-based JUnit, etc</li>
34<li>provides reliability and recovery mechanisms on top of adb</li>
35<li>supports scheduling and running tests on multiple devices in parallel</li>
36</ul>
37
38<h2>Sample Trade Federation Usecases</h2>
39<p>Trade Federation's modularity makes it straightforward to slot into environments with existing
40build, test, and reporting infrastructures.  We list below a few demonstrative
41usecases where tradefed could enable efficient, scalable test practices.</p>
42
43<p>First, it is useful to consider the landscape of potential usecases in terms of the question
44"which parts are modifiable, and what parts are static?"  For instance, a Device OEM can modify the
45framework, the system, and the hardware, but has little or no influence over existing applications.
46An application developer, on the other hand, can modify the app, but has little control over most
47aspects of the system or the framework.</p>
48
49<p>As a result, an entity in each usecase will have different testing goals, and will have different
50options in the case of a set of test failures.  Despite these differences, Trade Federation can
51help make each of their test processes efficient, flexible, and scalable.</p>
52
53<h3>Device OEM</h3>
54<p>A Device OEM builds hardware, and will often tweak the Android system and frameworks to run well
55on that hardware.  The OEM might strive to accomplish those goals while retaining stability
56and performance at the hardware and system levels, and making sure the framework changes don't break
57compatibility with existing applications.</p>
58
59<p>The OEM could implement a device flashing module that will execute during the Target Setup stage
60of the <a href="/devices/tech/test_infra/tradefed/fundamentals/lifecycle.html">lifecycle</a>.  That
61module would have complete control over the device during its execution period, which would allow
62it to potentially force the device into the bootloader, flash, and then force the device to reboot
63back into userspace mode.  Combined with a module to tie into a continuous build system, this would
64allow the OEM to run tests on their device as they make changes to the system-level firmware and
65Java-level frameworks.</p>
66
67<p>Once the device is fully booted, the OEM would be able to leverage existing JUnit-based tests,
68or write new ones, to verify the functionality of interest.  Finally, they could write one or more
69result reporting modules to tie into existing test-result repositories, or to report results
70directly (for instance,
71<a href="/reference/com/android/tradefed/result/EmailResultReporter.html">by email</a>).</p>
72
73<h3>App Developer</h3>
74<p>An Application Developer builds an app which needs to run well across a variety of platform
75versions and a variety of devices.  If an issue comes up on a particular platform version and/or
76device, the only remedy is to add a workaround and move on.  For larger developers, the test
77process might be incorporated into a continuous build sequence.  For smaller developers, it
78might be kicked off periodically or by hand.</p>
79
80<p>Most app developers would use the apk test installation modules that already exist in TF.
81There's a version that <a href="/reference/com/android/tradefed/targetprep/InstallApkSetup.html"
82>installs from the local filesystem</a>, as well as a version that can
83<a href="/reference/com/android/tradefed/targetprep/InstallBuildEnvApkSetup.html">install
84apks downloaded from a build service</a>.  It is important to note that the latter version would
85continue to work properly with arbitrarily many TF instances running on the same host machine.</p>
86
87<p>Because of TF's proficiency at dealing with multiple devices, it would be straightforward to
88classify each test result by the type of device that was used for that test.  Thus, TF could
89potentially generate a 2-dimensional (or multi-dimensional) compatibility matrix for every
90build of the application.</p>
91
92<h3>Testing Service</h3>
93<p>A Test Service might, for instance, allow app developers to submit apps and run tests on devices
94instrumented with power-measurement tools to determine power usage for the app.  This differs from
95the prior two usecases in that the service builder does not control the devices or the applications
96that are being run</p>
97
98<p>Because Trade Federation can run any Java class that implements the simple
99<a href="/reference/com/android/tradefed/testtype/IRemoteTest.html">IRemoteTest</a> interface, it's
100trivial to write drivers that can coordinate some external piece of hardware with the test case
101that's being run on the device.  The driver itself can spawn Threads, send requests to other
102servers, or do anything else that it might need.  Moreover, the simplicity and versatility of the
103result reporting interface,
104<a href="/reference/com/android/tradefed/result/ITestInvocationListener.html"
105>ITestInvocationListener</a>, means that it is likewise straightforward to
106represent arbitrary test results (including, for instance, numerical power metrics) into the
107standard result reporting pipeline.</p>
108