1# skpbench
2
3skpbench is a benchmarking tool for replaying skp or mksp files on android devices.
4it achieves a lower variance in framerate by controlling the clock speed and stopping
5all other processes that could cause interference.
6
7## Build
8
9skpbench consists of the skpbench binary which must be built for the phone you intend to run on,
10and skpbench.py which runs on the machine the phone is connected to via ADB and is the entry point.
11
12The to build skia for android are at https://skia.org/user/build#android and reproduced here.
13
14Download the Android NDK
15
16```
17cipd auth-login
18python2 infra/bots/assets/android_ndk_linux/download.py  -t /tmp/ndk
19
20```
21
22After this is set up once, build skpbench for your target cpu (assumed to be arm64 here for a Pixel 3)
23
24```
25bin/gn gen out/arm64 --args='ndk="/tmp/ndk" target_cpu="arm64" is_debug=false'
26ninja -C out/arm64 skpbench
27```
28
29## Benchmark an SKP on a connected device.
30
31First, copy the built skpbench binary and an example skp file to the device.
32(or pull a skp corpus using instructions in the section below)
33
34```
35adb push out/arm64/skpbench /data/local/tmp
36adb push /home/nifong/Downloads/foo.skp /data/local/tmp/skps/
37```
38
39Run skpbench.py
40
41```
42python tools/skpbench/skpbench.py \
43  --adb \
44  --config gles \
45  /data/local/tmp/skpbench \
46  /data/local/tmp/skps/foo.skp
47```
48
49`--adb` specifies that it should use adb to the only connected device and run skpbench there.
50`--force` is necessary because we don't yet have a configuration to monitor vitals on the Pixel 3.
51`--config gles` specifies Open GL ES is the backend GPU config to use.
52
53Additional documentation of arguments is printed by `python tools/skpbench/skpbench.py --help`
54
55Output appears in the following format
56```
57   accum    median       max       min   stddev  samples  sample_ms  clock  metric  config    bench
58  0.1834    0.1832    0.1897    0.1707    1.59%      101         50  cpu    ms      gles      foo.skp
59```
60
61`accum` is the time taken to draw all frames, divided by the number of frames.
62`metric` specifies that the unit is ms (milliseconds per frame)
63
64## MSKP corpus
65
66A manually collected corpus of MSKPs from around 30 top apps (using skia via HWUI) and of about 20
67actions in RenderEngine exists in a google cloud storage folder managed by skia/infra/bots/assets/mskp/upload.py
68
69To download the fileset, first determine the highest current version of the fileset
70
71```
72gsutil ls gs://skia-assets/assets/mskp/
73```
74
75Download the latest version.
76
77```
78gsutil cp gs://skia-assets/assets/mskp/5.zip ~/Downloads
79```
80
81Unzip the archive and adb push it to the device.
82
83To upload a new version of the corpus, use the steps above to download and unzip the last version, change the
84content however you need, then Use the upload tool, passing the directory of the altered archive (not a zip file).
85Note that you must provide it as an absolute path.
86
87```
88python upload.py --target_dir=/home/nifong/scratch/new_mskps
89```
90
91The upload script should print a version number.
92Finally, submit something like https://skia-review.googlesource.com/c/skia/+/304376
93to point jobs at the new version.
94
95## Production
96
97skpbench is run as a tryjob from gerrit, where it uploads the results to perf.skia.org.
98Once such job name is `Perf-Android-Clang-Pixel4XL-GPU-Adreno640-arm64-Release-All-Android_Skpbench`
99
100Perf results are available by querying with this or similar.
101  extra_config = Android_Skpbench
102  sub_result = accum_cpu_ms
103
104Example perf query
105https://perf.skia.org/e/?queries=extra_config%3DAndroid_Skpbench%26sub_result%3Daccum_cpu_ms
106