1-------------------------------------------------------------------------
2drawElements Quality Program Test Specification
3-----------------------------------------------
4
5Copyright 2014 The Android Open Source Project
6
7Licensed under the Apache License, Version 2.0 (the "License");
8you may not use this file except in compliance with the License.
9You may obtain a copy of the License at
10
11     http://www.apache.org/licenses/LICENSE-2.0
12
13Unless required by applicable law or agreed to in writing, software
14distributed under the License is distributed on an "AS IS" BASIS,
15WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16See the License for the specific language governing permissions and
17limitations under the License.
18-------------------------------------------------------------------------
19    Shader compiler performance tests
20
21Tests:
22 + dEQP-GLES2.performance.compiler.*
23
24Includes:
25 + Shader compile, link and specialization (draw call -time) time measurements
26 + Tests for impact of cache hits on compilation time (note: in release 2012.4)
27 + Front-end time estimation with invalid shaders
28 + Typical use cases
29   - Vertex and fragment lighting
30   - Mandelbrot viewer
31   - Normal mapping, parallax and relief mapping (note: in release 2012.4)
32 + Synthetic cases
33   - A number of texture lookups blended together in fragment shader
34   - Single or nested loops
35   - Single expression consisting of a variable number of vec4 multiplications
36 + Shaders designed to stress compiler (note: in release 2012.4)
37   - High register pressure
38   - Lots of control flow
39
40Excludes:
41 + Shader functional testing
42
43Description:
44
45Shader compiler performance tests measure the time to compile GLSL shader programs.
46The tests will take into account implementations that postpone final compilation
47until execution of a draw call.
48
49Separate cases are made for testing compilation time when avoiding cache hits and
50when allowing them (note that tests that allow cache will be in release 2012.4).
51Cache-avoiding cases use a different shader for different iterations of the same
52test case, whereas cache-allowing cases use the same shader for all iterations.
53Cache hits are however only ever allowed between the iterations of a case and not
54across separate cases.
55
56The method for defeating the shader cache is to append a postfix to all uniform,
57attribute and varying names. The postfix depends on the current time as well as
58the case that is in question. In case of cache-avoiding cases, the postfix also
59depends on the current iteration. This method forces at least re-linking.
60
61Front-end time is estimated by compiling invalid shaders with either an invalid
62character at the end of the shader or an invalid statement at the end of the main()
63block.
64
65The result value is the average number of milliseconds used per test case
66iteration. In cases that allow cache hits the average excludes the first
67compilation.
68
69Before any measurements are done, each test case first draws with a simple dummy
70shader for warm-up.
71
72Overview for valid shader cases:
73 - 1. Create program and shaders; compile and link
74 - 2. Set inputs and draw a small quad
75 - 3. Flush
76 - 4. Repeat without step 1
77 - 5. Compute difference of times measured for 1-3 and 4 and record it as
78      compilation time
79 - Repeat the above steps several times and compute average
80
81Overview for invalid shader cases (front-end time estimation):
82 - 1. Create program and invalid shaders; compile (expect failure)
83 - 2. Record time measured for step 1
84 - Repeat the above steps several times and compute average
85