1# Performance benchmarks
2
3This document lists TensorFlow Lite performance benchmarks when running well
4known models on some Android and iOS devices.
5
6These performance benchmark numbers were generated with the
7[Android TFLite benchmark binary](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark)
8and the [iOS benchmark app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios).
9
10# Android performance benchmarks
11
12For Android benchmarks, the CPU affinity is set to use big cores on the device to
13reduce variance (see [details](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#reducing-variance-between-runs-on-android)).
14
15It assumes that models were download and unzipped to the
16`/data/local/tmp/tflite_models` directory. The benchmark binary is built
17using [these instructions](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark#on-android)
18and assumed in the `/data/local/tmp` directory.
19
20To run the benchmark:
21
22```
23adb shell taskset ${CPU_MASK} /data/local/tmp/benchmark_model \
24  --num_threads=1 \
25  --graph=/data/local/tmp/tflite_models/${GRAPH} \
26  --warmup_runs=1 \
27  --num_runs=50 \
28  --use_nnapi=false
29```
30
31Here, `${GRAPH}` is the name of model and `${CPU_MASK}` is the CPU affinity
32chosen according to the following table:
33
34Device | CPU_MASK |
35-------| ----------
36Pixel 2 | f0 |
37Pixel xl | 0c |
38
39<table>
40  <thead>
41    <tr>
42      <th>Model Name</th>
43      <th>Device </th>
44      <th>Mean inference time</th>
45    </tr>
46  </thead>
47  <tr>
48    <td rowspan = 2>
49      <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a>
50    </td>
51    <td>Pixel 2 </td>
52    <td>123.3 ms</td>
53  </tr>
54   <tr>
55     <td>Pixel XL </td>
56     <td>113.3 ms</td>
57  </tr>
58  <tr>
59    <td rowspan = 2>
60      <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz">Mobilenet_1.0_224 (quant)</a>
61    </td>
62    <td>Pixel 2 </td>
63    <td>65.4 ms</td>
64  </tr>
65   <tr>
66     <td>Pixel XL </td>
67     <td>74.6 ms</td>
68  </tr>
69  <tr>
70    <td rowspan = 2>
71      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a>
72    </td>
73    <td>Pixel 2 </td>
74    <td>273.8 ms</td>
75  </tr>
76   <tr>
77     <td>Pixel XL </td>
78     <td>210.8 ms</td>
79  </tr>
80  <tr>
81    <td rowspan = 2>
82      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a>
83    </td>
84    <td>Pixel 2 </td>
85    <td>234.0 ms</td>
86  </tr>
87   <tr>
88     <td>Pixel XL </td>
89     <td>158.0 ms</td>
90  </tr>
91  <tr>
92    <td rowspan = 2>
93      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a>
94    </td>
95    <td>Pixel 2 </td>
96    <td>2846.0 ms</td>
97  </tr>
98   <tr>
99     <td>Pixel XL </td>
100     <td>1973.0 ms </td>
101  </tr>
102  <tr>
103    <td rowspan = 2>
104      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a>
105    </td>
106    <td>Pixel 2 </td>
107    <td>3180.0 ms</td>
108  </tr>
109   <tr>
110     <td>Pixel XL </td>
111     <td>2262.0 ms</td>
112  </tr>
113
114 </table>
115
116# iOS benchmarks
117
118To run iOS benchmarks, the [benchmark
119app](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark/ios)
120was modified to include the appropriate model and `benchmark_params.json` was
121modified  to set `num_threads` to 1.
122
123<table>
124  <thead>
125    <tr>
126      <th>Model Name</th>
127      <th>Device </th>
128      <th>Mean inference time</th>
129    </tr>
130  </thead>
131  <tr>
132    <td>
133      <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz">Mobilenet_1.0_224(float)</a>
134    </td>
135    <td>iPhone 8 </td>
136    <td>32.2 ms</td>
137  </tr>
138  <tr>
139    <td>
140      <a href="http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz)">Mobilenet_1.0_224 (quant)</a>
141    </td>
142    <td>iPhone 8 </td>
143    <td>24.4 ms</td>
144  </tr>
145  <tr>
146    <td>
147      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/nasnet_mobile_2018_04_27.tgz">NASNet mobile</a>
148    </td>
149    <td>iPhone 8 </td>
150    <td>60.3 ms</td>
151  </tr>
152  <tr>
153    <td>
154      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/squeezenet_2018_04_27.tgz">SqueezeNet</a>
155    </td>
156    <td>iPhone 8 </td>
157    <td>44.3</td>
158  </tr>
159  <tr>
160    <td>
161      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_resnet_v2_2018_04_27.tgz">Inception_ResNet_V2</a>
162    </td>
163    <td>iPhone 8</td>
164    <td>562.4 ms</td>
165  </tr>
166  <tr>
167    <td>
168      <a href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v4_2018_04_27.tgz">Inception_V4</a>
169    </td>
170    <td>iPhone 8 </td>
171    <td>661.0 ms</td>
172  </tr>
173 </table>
174