1-------------------------------------------------------------------------
2drawElements Quality Program Test Specification
3-----------------------------------------------
4
5Copyright 2014 The Android Open Source Project
6
7Licensed under the Apache License, Version 2.0 (the "License");
8you may not use this file except in compliance with the License.
9You may obtain a copy of the License at
10
11     http://www.apache.org/licenses/LICENSE-2.0
12
13Unless required by applicable law or agreed to in writing, software
14distributed under the License is distributed on an "AS IS" BASIS,
15WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16See the License for the specific language governing permissions and
17limitations under the License.
18-------------------------------------------------------------------------
19    Long-running stress tests
20
21Tests:
22 + dEQP-GLES2.stress.long.*
23
24Includes:
25 + Long-running stress testing with various application behaviors
26 + Memory object handling
27   - Buffers and textures
28   - Allocation, uploading
29 + Shader program
30   - Switching between several programs
31   - Re-compile frequently
32 + Draw calls
33   - Client-memory data for vertex indices and attributes
34   - Big draw calls
35
36Excludes:
37 + Functional verification
38
39Description:
40
41The long-running stress tests are meant to be run for a long time to detect
42deterioration (e.g. crash) in the program's behavior. The tests are supposed
43to be run with the command-line tools instead of Candy. The test case uses
44the --deqp-test-iteration-count=<num> command line option to choose the number
45of iterations it runs; this should be set to either a very big number, or
46alternatively a negative number, in which case it runs indefinitely. When
47this option is not given, the number of iterations is only 5; this is only
48useful to check that the test case's basic requisites are fulfilled (e.g.
49that its shader programs compile successfully).
50
51The basic structure of one iteration of the stress cases is as follows.
52
53First, if the case uses more than one program, it randomly picks one of them
54as the program for the current iteration. It then chooses whether or not to
55re-compile the program if it hasn't yet been compiled. When a program is re-
56compiled, attribute, uniform and varying names in the shader sources may
57be mangled with a unique suffix, thus attempting to avoid shader caching.
58
59Next, the textures used by the program are generated and uploaded or, if
60they already exist, they may be re-uploaded with either glTexImage*() or
61glTexSubImage*(). Texture parameters (wrapping etc.) are also set.
62
63Then, buffers are generated and (re-)uploaded in a manner similar to that of
64the textures described above. The buffers that may be used are vertex index
65and attribute buffers. If, however, the case decides to use client-side memory
66for index or attribute data, the respective buffer is not created.
67
68Next, the program's uniforms are set, and a case-dependent amount of primitives
69is drawn (note that this amount also affects the size of the buffers). This is
70repeated for a case-dependent number of times. The draw command used can be
71either glDrawElements() or glDrawArrays().
72
73After drawing, the case may delete some textures and/or buffers.
74
75This concludes an iteration. Between iterations, the usual post-iteration
76operations are done by the framework; this generally contains draw buffer
77swapping.
78
79A stress case's behavior is determined by its parameters:
80 + Program context(s)
81   - Vertex and fragment shader sources
82   - Value ranges for each attribute and uniform
83   - Texture information (size, format etc.)
84 + Approximate GPU memory usage limits that the case shouldn't exceed
85   - Texture memory
86   - Buffer memory
87 + Number of draw calls per iteration
88 + Number of triangles per draw call
89 + A set of probabilities that enable the following per-iteration actions:
90   - Re-build program
91   - Re-upload texture
92   - Re-upload buffer
93   - When re-uploading texture, use glTexImage*() instead of glTexSubImage*()
94   - When re-uploading buffer, use glBufferData() instead of glBufferSubData()
95   - Delete texture at end of iteration
96   - Delete buffer at end of iteration
97   - Don't re-use texture, but delay its deletion until memory limit is hit
98   - Don't re-use buffer, but delay its deletion until memory limit is hit
99   - Use random 'target' argument when uploading buffer
100   - Use random 'usage' argument when uploading buffer
101   - Use glDrawArrays() (and no index buffer) instead of glDrawElements()
102   - Use separate buffers for different attributes (instead of one big buffer)
103   (Note: the texture- or buffer-specific actions above are randomized both
104   per-iteration but also per-object.)
105