1page.title=Running the tests 2@jd:body 3 4<!-- 5 Copyright 2015 The Android Open Source Project 6 7 Licensed under the Apache License, Version 2.0 (the "License"); 8 you may not use this file except in compliance with the License. 9 You may obtain a copy of the License at 10 11 http://www.apache.org/licenses/LICENSE-2.0 12 13 Unless required by applicable law or agreed to in writing, software 14 distributed under the License is distributed on an "AS IS" BASIS, 15 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 16 See the License for the specific language governing permissions and 17 limitations under the License. 18--> 19 20<div id="qv-wrapper"> 21 <div id="qv"> 22 <h2>In this document</h2> 23 <ol id="auto-toc"> 24 </ol> 25 </div> 26</div> 27<p>This page provides instructions for running deqp tests in Linux and Windows 28environments, using command line arguments, and working with the Android 29application package.</p> 30 31<h2 id=linux_and_windows_environments>Linux and Windows environments</h2> 32 33<p>Start by copying the following files and directories to the target.</p> 34 35<table> 36 <tr> 37 <th>Module</th> 38 <th>Directory</th> 39 <th>Target</th> 40 </tr> 41 42 <tr> 43 <td>Execution Server</td> 44 <td><code>build/execserver/execserver</code></td> 45 <td><code><dst>/execserver</code></td> 46 </tr> 47 48 <tr> 49 <td>EGL Module</td> 50 <td><code>build/modules/egl/deqp-egl</code></td> 51 <td><code><dst>/deqp-egl</code></td> 52 </tr> 53 54 <tr> 55 <td rowspan=2 style="vertical-align:middle">GLES2 Module</td> 56 <td><code>build/modules/gles2/deqp-gles2</code></td> 57 <td><code><dst>/deqp-gles2</code></td> 58 </tr> 59 60 61 <tr> 62 <td><code>data/gles2</code></td> 63 <td><code><dst>/gles2</code></td> 64 </tr> 65 66 67 68 <tr> 69 <td rowspan=2 style="vertical-align:middle">GLES3 Module</td> 70 <td><code>build/modules/gles3/deqp-gles3</td> 71 <td><code><dst>/deqp-gles3</code></td> 72 </tr> 73 74 <tr> 75 <td><code>data/gles3</code></td> 76 <td><code><dst>/gles3</code></td> 77 </tr> 78 79 <tr> 80 <td rowspan=2 style="vertical-align:middle">GLES3.1 Module</td> 81 <td><code>build/modules/gles31/deqp-gles31</code></td> 82 <td><code><dst>/deqp-gles31</code></td> 83 </tr> 84 85 <tr> 86 <td><code>data/gles31</code></td> 87 <td><code><dst>/gles31</code></td> 88 </tr> 89 90 91 <tr> 92 <td rowspan=2 style="vertical-align:middle">GLES3.2 Module</td> 93 <td><code>build/modules/gles32/deqp-gles32</code></td> 94 <td><code><dst>/deqp-gles32</code></td> 95 </tr> 96 97 <tr> 98 <td><code>data/gles32</code></td> 99 <td><code><dst>/gles32</code></td> 100 </tr> 101 102</table> 103 104<p>You can deploy the execution service and test binaries anywhere in the target 105file system; however, test binaries expect to find data directories in the 106current working directory. When ready, start the Test Execution Service on the 107target device. For details on starting the service, see 108<a href="{@docRoot}devices/graphics/port-tests.html#test_execution_service">Test 109execution service</a>.</p> 110 111<h2 id=command_line_arguments>Command line arguments</h2> 112 113<p>The following table lists command line arguments that affect execution of all 114test programs.</p> 115 116<table width="100%"> 117<col style="width:50%"> 118<col style="width:50%"> 119 <tr> 120 <th>Argument</th> 121 <th>Description</th> 122 </tr> 123 124 <tr> 125<td><code>--deqp-case=<casename></code></td> 126<td>Run cases that match a given pattern. Wildcard (*) is supported.</td> 127 </tr> 128 129 <tr> 130<td><code>--deqp-log-filename=<filename></code></td> 131<td>Write test results to the file whose name you provide. The test execution 132service will set the filename when starting a test.</td> 133 </tr> 134 135 <tr> 136 <td><code>--deqp-stdin-caselist<br/> 137--deqp-caselist=<caselist><br/> 138--deqp-caselist-file=<filename></code></td> 139<td>Read case list from stdin or from a given argument. The test execution 140service will set the argument according to the execution request received. See 141the next section for a description of the case list format.</td> 142 </tr> 143 <tr> 144<td><code>--deqp-test-iteration-count=<count></code></td> 145<td>Override iteration count for tests that support a variable number of 146iterations.</td> 147 </tr> 148 <tr> 149 <td><code>--deqp-base-seed=<seed></code></td> 150 <td>Base seed for the test cases that use randomization.</td> 151 </tr> 152</table> 153 154<h3 id=gles2_and_gles3-specific_arguments>GLES2 and GLES3-specific arguments</h3> 155 156The following table lists the GLES2- and GLES3-specific arguments. 157 158<table> 159<table width="100%"> 160<col style="width:50%"> 161<col style="width:50%"> 162 <tr> 163 <th>Argument</th> 164 <th>Description</th> 165 </tr> 166 <tr> 167 <td><code>--deqp-gl-context-type=<type></code></td> 168 <td>OpenGL context type. Available context types depend on the platform. On 169 platforms supporting EGL, the value <code>egl</code> can be used to select 170 the EGL context.</td> 171 </tr> 172 <tr> 173 <td><code>--deqp-gl-config-id=<id></code></td> 174 <td>Run tests for the provided GL configuration ID. Interpretation is 175 platform-dependent. On the EGL platform, this is the EGL configuration ID.</td> 176 </tr> 177 <tr> 178 <td><code>--deqp-gl-config-name=<name></code></td> 179 <td><p>Run tests for a named GL configuration. Interpretation is 180 platform-dependent. For EGL, the format is 181 <code>rgb(a)<bits>d<bits>s<bits></code>. For example, a 182 value of <code>rgb888s8</code> will select the first configuration where the 183 color buffer is RGB888 and the stencil buffer has 8 bits.</td> 184 </tr> 185 <tr> 186 <td><code>--deqp-gl-context-flags=<flags></code></td> 187 <td>Creates a context. Specify <code>robust</code> or <code>debug</code>.</td> 188 </tr> 189 <tr> 190 <td><code>--deqp-surface-width=<width><br/> 191 --deqp-surface-height=<height></code></td> 192 <td>Try to create a surface with a given size. Support for this is optional.</td> 193 </tr> 194 <tr> 195 <td><code>--deqp-surface-type=<type></code></td> 196 <td>Use a given surface type as the main test rendering target. Possible 197 types are <code>window</code>, <code>pixmap</code>, <code>pbuffer</code>, 198 and <code>fbo</code>.</td> 199 </tr> 200 <tr> 201 <td><code>--deqp-screen-rotation=<rotation></code></td> 202 <td>Screen orientation in increments of 90 degrees for platforms that 203 support it.</td> 204 </tr> 205</table> 206 207<h3 id=test_case_list_format>Test case list format</h3> 208 209<p>The test case list can be given in two formats. The first option is to list 210the full name of each test on a separate line in a standard ASCII file. As the 211test sets grow, the repetitive prefixes can be cumbersome. To avoid repeating 212the prefixes, use a trie (also known as a prefix tree) syntax shown below.</p> 213 214<pre> 215{nodeName{firstChild{…},…lastChild{…}}} 216</pre> 217 218<p>For example:</p> 219 220<pre> 221{dEQP-EGL{config-list,create_context{rgb565_depth_stencil}}} 222</pre> 223 224<p>Translates into the following two test cases:</p> 225 226<pre> 227dEQP-EGL.config_list 228dEQP-EGL.create_context.rgb565_depth_stencil 229</pre> 230 231<h2 id=android>Android</h2> 232 233<p>The Android application package contains all required components, including 234the test execution service, test binaries, and data files. The test activity is 235a <code>NativeActivity</code> that uses EGL (requires Android 3.2 or higher).</p> 236 237<p>The application package can be installed with the following command (name 238shown is the name of the APK in the Android CTS package; which name depends on 239the build):</p> 240<pre>$ adb –d install –r com.drawelements.deqp.apk</pre> 241 242<p>To launch the test execution service and to setup port forwarding, use the 243following:</p> 244<pre> 245$ adb –d forward tcp:50016 tcp:50016 246$ adb –d shell am start –n com.drawelements.deqp/.execserver.ServiceStarter 247</pre> 248 249<p>Debug prints can be enabled by executing the following before starting the 250tests:</p> 251 252<pre> 253$ adb –d shell setprop log.tag.dEQP DEBUG 254</pre> 255 256<h3 id=executing_tests_on_android_without_android_cts>Executing tests on 257Android without Android CTS</h3> 258 259<p>To manually start the test execution activity, construct an Android intent 260that targets <code>android.app.NativeActivity</code>. The activities can be 261found in the <code>com.drawelements.deqp</code> package. The command line must 262be supplied as an extra string with key <code>"cmdLine"</code> in the Intent.</p> 263 264<p>A test log is written to <code>/sdcard/dEQP-log.qpa</code>. If the test run 265does not start normally, additional debug information is available in the device 266log.</p> 267 268<p>You can launch an activity from the command line using the <code>am</code> 269utility. For example, to run <code>dEQP-GLES2.info</code> tests on a platform 270supporting <code>NativeActivity,</code> use the following commands.</p> 271 272<pre>$ adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e \ 273cmdLine "deqp --deqp-case=dEQP-GLES2.info.* --deqp-log-filename=/sdcard/dEQP-Log.qpa 274</pre> 275 276<h3 id=debugging_on_android>Debugging on Android</h3> 277 278<p>To run the tests under the GDB debugger on Android, first compile and install 279the debug build by running the following two scripts:</p> 280 281<pre> 282$ python android/scripts/build.py --native-build-type=Debug 283$ python android/scripts/install.py 284</pre> 285 286<p>After the debug build is installed on the device, to launch the tests under 287GDB running on the host, run the following command:</p> 288 289<pre>$ python android/scripts/debug.py \ 290--deqp-commandline="--deqp-log-filename=/sdcard/TestLog.qpa --deqp-case=dEQP-GLES2.functional.*" 291</pre> 292 293<p>The deqp command line depends on the test cases to be executed and other 294required parameters. The script adds a default breakpoint at the beginning of 295the deqp execution (<code>tcu::App::App</code>).</p> 296 297<p>The <code>debug.py</code> script accepts multiple command line arguments for 298actions such as setting breakpoints for debugging, gdbserver connection 299parameters, and paths to additional binaries to debug (use <code>debug.py 300--help</code> for all arguments and explanations). The script also copies some 301default libraries from the target device to get symbol listings.</p> 302 303<p>To step through driver code (such as when the GDB needs to know the locations 304of the binaries with full debug information), add more libraries via 305<code>debug.py</code> command line parameters. This script writes out a 306configuration file for the GDB starting from line 132 of the script file. You 307can provide additional paths to binaries, etc., but supplying correct command 308line parameters should be enough.</p> 309 310<p class="note"><strong>Note:</strong> On Windows, the GDB binary requires 311<code>libpython2.7.dll</code>. Before launching <code>debug.py</code>, add 312<code><path-to-ndk>/prebuilt/windows/bin</code> to the PATH variable.</p> 313 314<p class="note"><strong>Note:</strong> Native code debugging does not work on 315stock Android 4.3; for workarounds, refer to 316<a href="https://code.google.com/p/android/issues/detail?id=58373">https://code.google.com/p/android/issues/detail?id=58373</a>. 317Android 4.4 and higher do not contain this bug.</p> 318