1# TensorFlow Lite operator versions
2
3This document describes TensorFlow Lite's op versioning schema. Op
4versioning enables developers to add new functionalities and parameters into
5existing ops. In addition, it guarantees the following:
6
7*   Backward compatibility: New TensorFlow Lite implementation should
8    handle an old model file.
9*   Forward compatibility: Old TensorFlow Lite implementation should
10    handle a new model file produced by new version of TOCO, as long as no new
11    features are used.
12*   Forward in-compatibility detection: If an old TensorFlow Lite implementation
13    reads a new model that contains a new version of an op which isn't
14    supported, it should report the error.
15
16## Example: Adding Dilation into Convolution
17
18The remainder of this document explains op versioning in TFLite by showing how
19to add dilation parameters to the convolution operation.
20
21Knowledge of dilation is not required to understand this document. Note that:
22
23*   2 new integer parameters will be added: `dilation_width_factor` and
24    `dilation_height_factor`.
25*   Old convolution kernels that don't support dilation are equivalent to
26    setting the dilation factors to 1.
27
28### Change FlatBuffer Schema
29
30To add new parameters into an op, change the options table in
31`lite/schema/schema.fbs`.
32
33For example, the options table of convolution looks like this:
34
35```
36table Conv2DOptions {
37  padding:Padding;
38  stride_w:int;
39  stride_h:int;
40  fused_activation_function:ActivationFunctionType;
41}
42```
43
44When adding new parameters:
45
46*   Add comments indicating which parameters are supported by which version.
47*   When the new implementation gets the default values for newly added
48    parameters, it should work exactly the same as the old implementation.
49
50The table will be like this after the new parameters are added:
51
52```
53table Conv2DOptions {
54  // Parameters supported by version 1:
55  padding:Padding;
56  stride_w:int;
57  stride_h:int;
58  fused_activation_function:ActivationFunctionType;
59
60  // Parameters supported by version 2:
61  dilation_width_factor:int = 1;
62  dilation_height_factor:int = 1;
63}
64```
65
66### Change C Structures and Kernel Implementation
67
68In TensorFlow Lite, the kernel implementation is decoupled from
69FlatBuffer definition. The kernels read the parameter from C structures defined
70in `lite/builtin_op_data.h`.
71
72The original convolution parameter is as follows:
73
74```
75typedef struct {
76  TfLitePadding padding;
77  int stride_width;
78  int stride_height;
79  TfLiteFusedActivation activation;
80} TfLiteConvParams;
81```
82
83As with the FlatBuffer schema, add comments indicating which parameters are
84supported starting from which version. The result is seen below:
85
86```
87typedef struct {
88  // Parameters supported by version 1: TfLitePadding padding; int
89  stride_width;
90  int stride_height;
91  TfLiteFusedActivation activation;
92
93  // Parameters supported by version 2:
94  int dilation_width_factor;
95  int dilation_height_factor;
96} TfLiteConvParams;
97```
98
99Please also change the kernel implementation to read the newly added parameters
100from the C structures. The details are omitted here.
101
102### Change the FlatBuffer Reading Code
103
104The logic to read FlatBuffer and produce C structure is in `lite/model.cc`.
105
106Update the file to handle the new parameters, as shown below:
107
108```
109case BuiltinOperator_CONV_2D: {
110  TfLiteConvParams* params = MallocPOD<TfLiteConvParams>();
111  if (auto* conv_params = op->builtin_options_as_Conv2DOptions()) {
112    params->padding = parse_padding(conv_params->padding());
113    params->stride_width = conv_params->stride_w();
114    params->stride_height = conv_params->stride_h();
115    params->activation =
116        parse_activation(conv_params->fused_activation_function());
117    params->dilation_width_factor = conv_params->dilation_width_factor();
118    params->dilation_height_factor = conv_params->dilation_height_factor();
119  }
120  *builtin_data = reinterpret_cast<void*>(params);
121  break;
122}
123```
124
125It's not required to check the op version here. When the new implementation
126reads an old model file where dilation factors are missing, it will use 1 as
127the default value, and the new kernel will work consistently with the old
128kernel.
129
130### Change Kernel Registration
131
132The MutableOpResolver (defined in `lite/op_resolver.h`) provides a few functions
133to register op kernels. The minimum and maximum version are 1 by default:
134
135```
136void AddBuiltin(tflite::BuiltinOperator op, TfLiteRegistration* registration,
137                int min_version = 1, int max_version = 1);
138void AddCustom(const char* name, TfLiteRegistration* registration,
139               int min_version = 1, int max_version = 1);
140```
141
142The built-in ops are registered in `lite/kernels/register.cc`. In this example,
143we implemented a new op kernel which can handle `Conv2D` version 1 and 2, so we
144need to change this line:
145
146```
147AddBuiltin(BuiltinOperator_CONV_2D, Register_CONV_2D());
148```
149
150to:
151
152```
153AddBuiltin(BuiltinOperator_CONV_2D, Register_CONV_2D(), 1, 2);
154```
155
156### Change TOCO TFLite exporter
157
158The last step is to make TOCO populate the minimum version that's required to
159execute the op. In this example, it means:
160
161*   Populate version=1 when dilation factors are all 1.
162*   Populate version=2 otherwise.
163
164To do this, you need to override `GetVersion` function for the operator class in
165`lite/toco/tflite/operator.cc`.
166
167For ops with only one version, the `GetVersion` function is defined as:
168
169```
170int GetVersion(const Operator& op) const override { return 1; }
171```
172
173When supporting multiple versions, check the parameters and determine the
174version for the op, as shown in the following example:
175
176```
177int GetVersion(const Operator& op) const override {
178  const auto& conv_op = static_cast<const ConvOperator&>(op);
179  if (conv_op.dilation_width_factor != 1 ||
180      conv_op.dilation_height_factor != 1) {
181    return 2;
182  }
183  return 1;
184}
185```
186
187### Delegation Implementation
188
189TensorFlow Lite provides a delegation API which enables delegating ops to
190hardware backends. In Delegate's `Prepare` function, check if the version
191is supported for every node in Delegation code.
192
193```
194const int kMinVersion = 1;
195TfLiteNode* node;
196TfLiteRegistration;
197context->GetNodeAndRegistration(context, node_index, &node, &registration);
198
199if (registration->version > kMinVersion) {
200  // Reject the node if the version isn't supported.
201}
202```
203
204This is required even if the delegation only supports version 1 ops, so the
205delegation can detect incompatibility when getting a higher version op.
206
207