README
1=============================================================================
2Notes about performance benchmarks
3=============================================================================
4For each benchmark, here is a brief description and notes about its
5strengths and weaknesses.
6
7-----------------------------------------------------------------------------
8Artificial stress tests
9-----------------------------------------------------------------------------
10bigcode1, bigcode2:
11- Description: Executes a lot of (nonsensical) code.
12- Strengths: Demonstrates the cost of translation which is a large part
13 of runtime, particularly on larger programs.
14- Weaknesses: Highly artificial.
15
16heap:
17- Description: Does a lot of heap allocation and deallocation, and has a lot
18 of heap blocks live while doing so.
19- Strengths: Stress test for an important sub-system; bug #105039 showed
20 that inefficiencies in heap allocation can make a big
21 difference to programs that allocate a lot.
22- Weaknesses: Highly artificial -- allocation pattern is not real, and only
23 a few different size allocations are used.
24
25sarp:
26- Description: Does a lot of stack allocation and deallocation.
27- Strengths: Tests for a specific performance bug that existed in 3.1.0 and
28 all earlier versions.
29- Weaknesses: Highly artificial.
30
31-----------------------------------------------------------------------------
32Real programs
33-----------------------------------------------------------------------------
34bz2:
35- Description: Burrows-Wheeler compression and decompression.
36- Strengths: A real, widely used program, very similar to the 256.bzip2
37 SPEC2000 benchmark. Not dominated by any code, the hottest
38 55 blocks account for only 90% of execution. Has lots of
39 short blocks and stresses the memory system hard.
40- Weaknesses: None, really, it's a good benchmark.
41
42fbench:
43- Description: Does some ray-tracing.
44- Strengths: Moderately realistic program.
45- Weaknesses: Dominated by sin and cos, which are not widely used, and are
46 hardware-supported on x86 but not on other platforms such as
47 PPC.
48
49ffbench:
50- Description: Does a Fast Fourier Transform (FFT).
51- Strengths: Tests common FP ops (mostly adding and multiplying array
52 elements), FFT is a very important operation.
53- Weaknesses: Dominated by the inner loop, which is quite long and flatters
54 Valgrind due to the small dispatcher overhead.
55
56tinycc:
57- Description: A very small and fast C compiler. A munged version of
58 Fabrice Bellard's TinyCC compiling itself multiple times.
59- Strengths: A real program, lots of code (top 100 blocks only account for
60 47% of execution), involves large irregular data structures
61 (presumably, since it's a compiler). Does lots of
62 malloc/free calls and so changes that make a big improvement
63 to perf/heap typically cause a small improvement.
64- Weaknesses None, really, it's a good benchmark.
65
66