README.md
1# Tensorflow Lite Core ML Delegate
2
3TensorFlow Lite Core ML Delegate enables running TensorFlow Lite models on
4[Core ML framework](https://developer.apple.com/documentation/coreml),
5which results in faster model inference on iOS devices.
6
7[TOC]
8
9## Supported iOS versions and processors
10
11* iOS 12 and later. In the older iOS versions, Core ML delegate will
12 automatically fallback to CPU.
13* When running on iPhone Xs and later, it will use Neural Engine for faster
14 inference.
15
16## Update code to use Core ML delegate
17
18### Swift
19
20Initialize TensorFlow Lite interpreter with Core ML delegate.
21
22```swift
23let coreMlDelegate = CoreMLDelegate()
24let interpreter = try Interpreter(modelPath: modelPath,
25 delegates: [coreMLDelegate])
26```
27
28### Objective-C++
29
30#### Interpreter initialization
31
32Include `coreml_delegate.h`.
33
34```objectivec++
35#include "tensorflow/lite/experimental/delegates/coreml/coreml_delegate.h"
36```
37
38Modify code following interpreter initialization to apply delegate.
39
40```objectivec++
41// initializer interpreter with model.
42tflite::InterpreterBuilder(*model, resolver)(&interpreter);
43
44// Add following section to use Core ML delegate.
45TfLiteCoreMlDelegateOptions options = {};
46delegate = TfLiteCoreMlDelegateCreate(&options);
47interpreter->ModifyGraphWithDelegate(delegate);
48
49// ...
50```
51
52#### Disposal
53
54Add this code to the section where you dispose of the delegate (e.g. `dealloc`
55of class).
56
57```objectivec++
58TfLiteCoreMlDelegateDelete(delegate);
59```
60
61## Supported ops
62
63Following ops are supported by the Core ML delegate.
64
65* Add
66 * Only certain shapes are broadcastable. In Core ML tensor layout,
67 following tensor shapes are broadcastable. `[B, C, H, W]`, `[B, C, 1,
68 1]`, `[B, 1, H, W]`, `[B, 1, 1, 1]`.
69* AveragePool2D
70* Concat
71* Conv2D
72 * Weights and bias should be constant.
73* DepthwiseConv2D
74 * Weights and bias should be constant.
75* FullyConnected (aka Dense or InnerProduct)
76 * Weights and bias (if present) should be constant.
77 * Only supports single-batch case. Input dimensions should be 1, except
78 the last dimension.
79* Hardswish
80* Logistic (aka Sigmoid)
81* MaxPool2D
82* MirrorPad
83 * Only 4D input with `REFLECT` mode is supported. Padding should be
84 constant, and is only allowed for H and W dimensions.
85* Mul
86 * Only certain shapes are broadcastable. In Core ML tensor layout,
87 following tensor shapes are broadcastable. `[B, C, H, W]`, `[B, C, 1,
88 1]`, `[B, 1, H, W]`, `[B, 1, 1, 1]`.
89* Pad and PadV2
90 * Only 4D input is supported. Padding should be constant, and is only
91 allowed for H and W dimensions.
92* Relu
93* ReluN1To1
94* Relu6
95* Reshape
96 * Only supported when target Core ML version is 2, not supported when
97 targeting Core ML 3.
98* ResizeBilinear
99* SoftMax
100* Tanh
101* TransposeConv
102 * Weights should be constant.
103
104## FAQ
105
106* Does Core ML delegate support fallback to CPU if a graph contains unsupported
107 ops?
108 * Yes.
109* Does Core ML delegate work on iOS Simulator?
110 * Yes. The library includes x86 and x86_64 targets so it can run on
111 a simulator, but you will not see performance boost over CPU.
112* Does TensorFlow Lite and Core ML delegate support macOS?
113 * TensorFlow Lite is only tested on iOS but not macOS.
114* Are custom TF Lite ops supported?
115 * No, CoreML delegate does not support custom ops and they will fallback to
116 CPU.
117
118## Appendix
119
120### Core ML delegate Swift API
121
122```swift
123/// A delegate that uses the `Core ML` framework for performing
124/// TensorFlow Lite graph operations.
125///
126/// - Important: This is an experimental interface that is subject to change.
127public final class CoreMLDelegate: Delegate {
128 /// The configuration options for the `CoreMLDelegate`.
129 public let options: Options
130
131 // Conformance to the `Delegate` protocol.
132 public private(set) var cDelegate: CDelegate
133
134 * /// Creates a new instance configured with the given `options`.
135 ///
136 /// - Parameters:
137 /// - options: Configurations for the delegate. The default is a new instance of
138 /// `CoreMLDelegate.Options` with the default configuration values.
139 public init(options: Options = Options()) {
140 self.options = options
141 var delegateOptions = TfLiteCoreMlDelegateOptions()
142 cDelegate = TfLiteCoreMlDelegateCreate(&delegateOptions)
143 }
144
145 deinit {
146 TfLiteCoreMlDelegateDelete(cDelegate)
147 }
148}
149
150extension CoreMLDelegate {
151 /// Options for configuring the `CoreMLDelegate`.
152 public struct Options: Equatable, Hashable {
153 /// Creates a new instance with the default values.
154 public init() {}
155 }
156}
157```
158
159### Core ML delegate C++ API
160
161```c++
162typedef struct {
163 // We have dummy for now as we can't have empty struct in C.
164 char dummy;
165} TfLiteCoreMlDelegateOptions;
166
167// Return a delegate that uses CoreML for ops execution.
168// Must outlive the interpreter.
169TfLiteDelegate* TfLiteCoreMlDelegateCreate(
170 const TfLiteCoreMlDelegateOptions* options);
171
172// Do any needed cleanup and delete 'delegate'.
173void TfLiteCoreMlDelegateDelete(TfLiteDelegate* delegate);
174```
175