1page.title=Design For Reduced Latency
2@jd:body
3
4<!--
5    Copyright 2013 The Android Open Source Project
6
7    Licensed under the Apache License, Version 2.0 (the "License");
8    you may not use this file except in compliance with the License.
9    You may obtain a copy of the License at
10
11        http://www.apache.org/licenses/LICENSE-2.0
12
13    Unless required by applicable law or agreed to in writing, software
14    distributed under the License is distributed on an "AS IS" BASIS,
15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16    See the License for the specific language governing permissions and
17    limitations under the License.
18-->
19<div id="qv-wrapper">
20  <div id="qv">
21    <h2>In this document</h2>
22    <ol id="auto-toc">
23    </ol>
24  </div>
25</div>
26
27<p>
28The Android 4.1 release introduced internal framework changes for
29a lower latency audio output path. There were no public client API
30or HAL API changes. This document describes the initial design,
31which is expected to evolve over time.
32Having a good understanding of this design should help device OEM and
33SoC vendors implement the design correctly on their particular devices
34and chipsets.  This article is not intended for application developers.
35</p>
36
37<h2 id="trackCreation">Track Creation</h2>
38
39<p>
40The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the
41<code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or
42<code>AudioTrack::set()</code>. Currently the only clients that do so are:
43</p>
44
45<ul>
46<li>OpenSL ES</li>
47<li>SoundPool</li>
48<li>ToneGenerator</li>
49</ul>
50
51<p>
52The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code>
53request and may optionally deny the request at client level. If it
54decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of
55the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method
56<code>IAudioFlinger::createTrack()</code>.
57</p>
58
59<p>
60The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may
61optionally deny the request at server level. It informs the client
62whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the
63shared memory control block.
64</p>
65
66<p>
67The factors that impact the decision include:
68</p>
69
70<ul>
71<li>Presence of a fast mixer thread for this output (see below)</li>
72<li>Track sample rate</li>
73<li>Presence of a client thread to execute callback handlers for this track</li>
74<li>Track buffer size</li>
75<li>Available fast track slots (see below)</li>
76</ul>
77
78<p>
79If the client's request was accepted, it is called a "fast track."
80Otherwise it's called a "normal track."
81</p>
82
83<h2 id="mixerThreads">Mixer Threads</h2>
84
85<p>
86At the time AudioFlinger creates a normal mixer thread, it decides
87whether or not to also create a fast mixer thread. Both the normal
88mixer and fast mixer are not associated with a particular track,
89but rather with a set of tracks. There is always a normal mixer
90thread. The fast mixer thread, if it exists, is subservient to the
91normal mixer thread and acts under its control.
92</p>
93
94<h3 id="fastMixer">Fast mixer</h3>
95
96<h4>Features</h4>
97
98<p>
99The fast mixer thread provides these features:
100</p>
101
102<ul>
103<li>Mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li>
104<li>Per track attenuation</li>
105</ul>
106
107<p>
108Omitted features:
109</p>
110
111<ul>
112<li>Per track sample rate conversion</li>
113<li>Per track effects</li>
114<li>Per mix effects</li>
115</ul>
116
117<h4>Period</h4>
118
119<p>
120The fast mixer runs periodically, with a recommended period of two
121to three milliseconds (ms), or slightly higher if needed for scheduling stability.
122This number was chosen so that, accounting for the complete
123buffer pipeline, the total latency is on the order of 10 ms. Smaller
124values are possible but may result in increased power consumption
125and chance of glitches depending on CPU scheduling predictability.
126Larger values are possible, up to 20 ms, but result in degraded
127total latency and so should be avoided.
128</p>
129
130<h4>Scheduling</h4>
131
132<p>
133The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very
134little CPU time, but must run often and with low scheduling jitter.
135Running too late will result in glitches due to underrun. Running
136too early will result in glitches due to pulling from a fast track
137before the track has provided data.
138</p>
139
140<h4>Blocking</h4>
141
142<p>
143Ideally the fast mixer thread never blocks, other than at HAL
144<code>write()</code>. Other occurrences of blocking within the fast mixer are
145considered bugs. In particular, mutexes are avoided.
146</p>
147
148<h4>Relationship to other components</h4>
149
150<p>
151The fast mixer has little direct interaction with clients. In
152particular, it does not see binder-level operations, but it does
153access the client's shared memory control block.
154</p>
155
156<p>
157The fast mixer receives commands from the normal mixer via a state queue.
158</p>
159
160<p>
161Other than pulling track data, interaction with clients is via the normal mixer.
162</p>
163
164<p>
165The fast mixer's primary sink is the audio HAL.
166</p>
167
168<h3 id="normalMixer">Normal mixer</h3>
169
170<h4>Features</h4>
171
172<p>
173All features are enabled:
174</p>
175
176<ul>
177<li>Up to 32 tracks</li>
178<li>Per track attenuation</li>
179<li>Per track sample rate conversion</li>
180<li>Effects processing</li>
181</ul>
182
183<h4>Period</h4>
184
185<p>
186The period is computed to be the first integral multiple of the
187fast mixer period that is >= 20 ms.
188</p>
189
190<h4>Scheduling</h4>
191
192<p>
193The normal mixer runs at elevated <code>SCHED_OTHER</code> priority.
194</p>
195
196<h4>Blocking</h4>
197
198<p>
199The normal mixer is permitted to block, and often does so at various
200mutexes as well as at a blocking pipe to write its sub-mix.
201</p>
202
203<h4>Relationship to other components</h4>
204
205<p>
206The normal mixer interacts extensively with the outside world,
207including binder threads, audio policy manager, fast mixer thread,
208and client tracks.
209</p>
210
211<p>
212The normal mixer's sink is a blocking pipe to the fast mixer's track 0.
213</p>
214
215<h2 id="flags">Flags</h2>
216
217<p>
218<code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the
219request will be fulfilled.
220</p>
221
222<p>
223<code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear
224in server.
225</p>
226
227<p>
228<code>TRACK_FAST</code> is a client -> server concept.
229</p>
230
231