1page.title=Design For Reduced Latency
2@jd:body
3
4<!--
5    Copyright 2013 The Android Open Source Project
6
7    Licensed under the Apache License, Version 2.0 (the "License");
8    you may not use this file except in compliance with the License.
9    You may obtain a copy of the License at
10
11        http://www.apache.org/licenses/LICENSE-2.0
12
13    Unless required by applicable law or agreed to in writing, software
14    distributed under the License is distributed on an "AS IS" BASIS,
15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16    See the License for the specific language governing permissions and
17    limitations under the License.
18-->
19<div id="qv-wrapper">
20  <div id="qv">
21    <h2>In this document</h2>
22    <ol id="auto-toc">
23    </ol>
24  </div>
25</div>
26
27<p>
28The Android 4.1 release introduced internal framework changes for
29a <a href="http://en.wikipedia.org/wiki/Low_latency">lower latency</a>
30audio output path. There were minimal public client API
31or HAL API changes. This document describes the initial design,
32which has continued to evolve over time.
33Having a good understanding of this design should help device OEM and
34SoC vendors implement the design correctly on their particular devices
35and chipsets.  This article is not intended for application developers.
36</p>
37
38<h2 id="trackCreation">Track Creation</h2>
39
40<p>
41The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the
42<code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or
43<code>AudioTrack::set()</code>. Currently the only clients that do so are:
44</p>
45
46<ul>
47<li>Android native audio based on OpenSL ES</li>
48<li><a href="http://developer.android.com/reference/android/media/SoundPool.html">android.media.SoundPool</a></li>
49<li><a href="http://developer.android.com/reference/android/media/ToneGenerator.html">android.media.ToneGenerator</a></li>
50</ul>
51
52<p>
53The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code>
54request and may optionally deny the request at client level. If it
55decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of
56the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method
57<code>IAudioFlinger::createTrack()</code>.
58</p>
59
60<p>
61The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may
62optionally deny the request at server level. It informs the client
63whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the
64shared memory control block.
65</p>
66
67<p>
68The factors that impact the decision include:
69</p>
70
71<ul>
72<li>Presence of a fast mixer thread for this output (see below)</li>
73<li>Track sample rate</li>
74<li>Presence of a client thread to execute callback handlers for this track</li>
75<li>Track buffer size</li>
76<li>Available fast track slots (see below)</li>
77</ul>
78
79<p>
80If the client's request was accepted, it is called a "fast track."
81Otherwise it's called a "normal track."
82</p>
83
84<h2 id="mixerThreads">Mixer Threads</h2>
85
86<p>
87At the time AudioFlinger creates a normal mixer thread, it decides
88whether or not to also create a fast mixer thread. Both the normal
89mixer and fast mixer are not associated with a particular track,
90but rather with a set of tracks. There is always a normal mixer
91thread. The fast mixer thread, if it exists, is subservient to the
92normal mixer thread and acts under its control.
93</p>
94
95<h3 id="fastMixer">Fast mixer</h3>
96
97<h4>Features</h4>
98
99<p>
100The fast mixer thread provides these features:
101</p>
102
103<ul>
104<li>Mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li>
105<li>Per track attenuation</li>
106</ul>
107
108<p>
109Omitted features:
110</p>
111
112<ul>
113<li>Per track sample rate conversion</li>
114<li>Per track effects</li>
115<li>Per mix effects</li>
116</ul>
117
118<h4>Period</h4>
119
120<p>
121The fast mixer runs periodically, with a recommended period of two
122to three milliseconds (ms), or a slightly higher period of five ms if needed for scheduling stability.
123This number was chosen so that, accounting for the complete
124buffer pipeline, the total latency is on the order of 10 ms. Smaller
125values are possible but may result in increased power consumption
126and chance of glitches depending on CPU scheduling predictability.
127Larger values are possible, up to 20 ms, but result in degraded
128total latency and so should be avoided.
129</p>
130
131<h4>Scheduling</h4>
132
133<p>
134The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very
135little CPU time, but must run often and with low scheduling jitter.
136<a href="http://en.wikipedia.org/wiki/Jitter">Jitter</a>
137expresses the variation in cycle time: it is the difference between the
138actual cycle time versus the expected cycle time.
139Running too late will result in glitches due to underrun. Running
140too early will result in glitches due to pulling from a fast track
141before the track has provided data.
142</p>
143
144<h4>Blocking</h4>
145
146<p>
147Ideally the fast mixer thread never blocks, other than at HAL
148<code>write()</code>. Other occurrences of blocking within the fast mixer are
149considered bugs. In particular, mutexes are avoided.
150Instead, <a href="http://en.wikipedia.org/wiki/Non-blocking_algorithm">non-blocking algorithms</a>
151(also known as lock-free algorithms) are used.
152See <a href="avoiding_pi.html">Avoiding Priority Inversion</a> for more on this topic.
153</p>
154
155<h4>Relationship to other components</h4>
156
157<p>
158The fast mixer has little direct interaction with clients. In
159particular, it does not see binder-level operations, but it does
160access the client's shared memory control block.
161</p>
162
163<p>
164The fast mixer receives commands from the normal mixer via a state queue.
165</p>
166
167<p>
168Other than pulling track data, interaction with clients is via the normal mixer.
169</p>
170
171<p>
172The fast mixer's primary sink is the audio HAL.
173</p>
174
175<h3 id="normalMixer">Normal mixer</h3>
176
177<h4>Features</h4>
178
179<p>
180All features are enabled:
181</p>
182
183<ul>
184<li>Up to 32 tracks</li>
185<li>Per track attenuation</li>
186<li>Per track sample rate conversion</li>
187<li>Effects processing</li>
188</ul>
189
190<h4>Period</h4>
191
192<p>
193The period is computed to be the first integral multiple of the
194fast mixer period that is >= 20 ms.
195</p>
196
197<h4>Scheduling</h4>
198
199<p>
200The normal mixer runs at elevated <code>SCHED_OTHER</code> priority.
201</p>
202
203<h4>Blocking</h4>
204
205<p>
206The normal mixer is permitted to block, and often does so at various
207mutexes as well as at a blocking pipe to write its sub-mix.
208</p>
209
210<h4>Relationship to other components</h4>
211
212<p>
213The normal mixer interacts extensively with the outside world,
214including binder threads, audio policy manager, fast mixer thread,
215and client tracks.
216</p>
217
218<p>
219The normal mixer's sink is a blocking pipe to the fast mixer's track 0.
220</p>
221
222<h2 id="flags">Flags</h2>
223
224<p>
225<code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the
226request will be fulfilled.
227</p>
228
229<p>
230<code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear
231in server.
232</p>
233
234<p>
235<code>TRACK_FAST</code> is a client -> server concept.
236</p>
237
238