1# VirtualDeviceManager Demo Apps
2
3#### Page contents
4
5[Overview](#overview) \
6[Prerequisites](#prerequisites) \
7[Build & Install](#build-and-install) \
8[Run](#run) \
9[Host Options](#host-options) \
10[Client Options](#client-options) \
11[Demos](#demos)
12
13## Overview
14
15The VDM Demo Apps allow for showcasing VDM features, rapid prototyping and
16testing of new features.
17
18The VDM Demo contains 3 apps:
19
20*   **Host**: installed on the host device, creates and manages a virtual
21    device, which represents the client device and communicates with the
22    physical client device by sending audio and frames of the virtual displays,
23    receiving input and sensor data that is injected into the framework. It can
24    launch apps on the virtual device, which are streamed to the client.
25
26*   **Client**: installed on the client device. It receives the audio and frames
27    from the host device, which it renders, and sends back input and sensor
28    data. For best experience with app streaming on multiple displays at the
29    same time, it's best to use a large screen device as a client, like a Pixel
30    Tablet.
31
32*   **Demos**: installed on the host, meant to showcase specific VDM features.
33    The demos can be also run natively on the host to illustrate better the
34    difference in the behavior when they are streamed to a virtual device.
35
36## Prerequisites
37
38*   An Android device running Android 13 or newer to act as a client device.
39
40*   A *rooted* Android device running Android 14 or newer (e.g. a `userdebug` or
41    `eng` build) to act as a host device. Even though VDM is available starting
42    Android 13, the support there is minimal and the Host app is not compatible
43    with Android 13.
44
45*   Both devices need to support
46    [Wi-Fi Aware](https://developer.android.com/develop/connectivity/wifi/wifi-aware)
47
48Note: This example app uses an Android device as a client, but there's no such
49general requirement. The client device, its capabilities, the connectivity layer
50and the communication protocol are entirely up to the virtual device owner.
51
52## Build and Install
53
54### Using the script
55
56Simply connect your devices, navigate to the root of your Android checkout and
57run
58
59```
60./development/samples/VirtualDeviceManager/setup.sh
61```
62
63The interactive script will prompt you which apps to install to which of the
64available devices, build the APKs and install them.
65
66### Manually
67
681.  Source `build/envsetup.sh` and run `lunch` or set
69    `UNBUNDLED_BUILD_SDKS_FROM_SOURCE=true` if there's no local build because
70    the APKs need to be built against a locally built SDK.
71
721.  Build the Host app.
73
74    ```shell
75    m -j VdmHost
76    ```
77
781.  Install the application as a system app on the host device.
79
80    ```shell
81    adb root && adb disable-verity && adb reboot  # one time
82    adb root && adb remount
83    adb push $ANDROID_BUILD_TOP/development/samples/VirtualDeviceManager/host/com.example.android.vdmdemo.host.xml /system/etc/permissions/com.example.android.vdmdemo.host.xml
84    adb shell mkdir /system/priv-app/VdmDemoHost
85    adb push $OUT/system/priv-app/VdmHost/VdmHost.apk /system/priv-app/VdmDemoHost/
86    adb reboot
87    ```
88
89    **Tip:** Subsequent installs without changes to permissions, etc. do not
90    need all the commands above - you can just run \
91    \
92    `adb install -r -d -g $OUT/system/priv-app/VdmHost/VdmHost.apk`
93
941.  Build and install the Demo app on the host device.
95
96    ```shell
97    m -j VdmDemos && adb install -r -d -g $OUT/system/app/VdmDemos/VdmDemos.apk
98    ```
99
1001.  Build and install the Client app on the client device.
101
102    ```shell
103    m -j VdmClient && adb install -r -d -g $OUT/system/app/VdmClient/VdmClient.apk
104    ```
105
106## Run
107
1081.  Start both the Client and the Host apps on each respective device.
109
1101.  They should find each other and connect automatically. On the first launch
111    the Host app will ask to create a CDM association: allow it.
112
113    WARNING: If there are other devices in the vicinity with one of these apps
114    running, they might interfere.
115
1161.  Check out the different [Host Options](#host-options) and
117    [Client Options](#client-options) that allow for changing the behavior of
118    the streamed apps and the virtual device in general.
119
1201.  Check out the [Demo apps](#demos) that are specifically meant to showcase
121    the VDM features.
122
123<!-- LINT.IfChange(host_options) -->
124
125## Host Options
126
127NOTE: Any flag changes require device reboot or "Force stop" of the host app
128because the flag values are cached and evaluated only when the host app is
129starting. Alternatively, run: \
130\
131`adb shell am force-stop com.example.android.vdmdemo.host`
132
133### Launcher
134
135Once the connection with the client device is established, the Host app will
136show a launcher-like list of installed apps on the host device.
137
138-   Clicking an app icon will create a new virtual display, launch the app there
139    and start streaming the display contents to the client. The client will show
140    the surface of that display and render its contents.
141
142-   Long pressing on an app icon will open a dialog to select an existing
143    display to launch the app on instead of creating a new one.
144
145-   The Host app has a **CREATE HOME DISPLAY** button, clicking it will create a
146    new virtual display, launch the secondary home activity there and start
147    streaming the display contents to the client. The display on the Client app
148    will have a home button, clicking it will navigate the streaming experience
149    back to the home activity. Run the commands below to enable this
150    functionality.
151
152    ```shell
153    adb shell device_config put virtual_devices android.companion.virtual.flags.vdm_custom_home true
154    adb shell am force-stop com.example.android.vdmdemo.host
155    ```
156
157-   The Host app has a **CREATE MIRROR DISPLAY** button, clicking it will create
158    a new virtual display, mirror the default host display there and start
159    streaming the display contents to the client. Run the commands below to
160    enable this functionality.
161
162    ```shell
163    adb shell device_config put virtual_devices android.companion.virtual.flags.consistent_display_flags true
164    adb shell device_config put virtual_devices android.companion.virtual.flags.interactive_screen_mirror true
165    adb shell am force-stop com.example.android.vdmdemo.host
166    ```
167
168### Settings
169
170#### Input
171
172The input menu button enables several different mechanisms for injecting input
173from the host device into the focused display on the client device. The focused
174display is indicated by the frame around its header whenever there are more than
175one displays. The display focus is based on user interaction.
176
177Each input screen has a "Back", "Home" and "Forward" buttons.
178
179-   **Touchpad** shows an on-screen touchpad for injecting mouse events into the
180    focused display.
181
182-   **Remote** allows the host device to act as a pointer that controls the
183    mouse movement on the focused display.
184
185-   **Navigation** shows an on-screen D-Pad and touchpad for navigating the
186    activity on the focused display.
187
188-   **Keyboard** shows the host device's on-screen keyboard and sends any key
189    events to the activity on the focused display.
190
191-   **Stylus** allows for injecting simulated stylus events into the focused
192    display. Use together with the stylus demo. Run the commands below to enable
193    this functionality.
194
195    ```shell
196    adb shell device_config put virtual_devices android.companion.virtual.flags.virtual_stylus true
197    adb shell am force-stop com.example.android.vdmdemo.host
198    ```
199
200#### General
201
202-   **Device profile**: Enables device streaming CDM role as opposed to app
203    streaming role, with all differences in policies that this entails. \
204    *Changing this will recreate the virtual device.*
205
206-   **Hide streamed app from recents**: Whether streamed apps should show up in
207    the host device's recent apps. Run the commands below to make this
208    functionality dynamic. \
209    *This can be changed dynamically starting with Android V.*
210
211    ```shell
212    adb shell device_config put virtual_devices android.companion.virtual.flags.dynamic_policy true
213    adb shell am force-stop com.example.android.vdmdemo.host
214    ```
215
216-   **Enable cross-device clipboard**: Whether to share the clipboard between
217    the host and the virtual device. If disabled, both devices will have their
218    own isolated clipboards. Run the commands below to enable this
219    functionality. \
220    *This can be changed dynamically.*
221
222    ```shell
223    adb shell device_config put virtual_devices android.companion.virtual.flags.dynamic_policy true
224    adb shell device_config put virtual_devices android.companion.virtual.flags.cross_device_clipboard true
225    adb shell am force-stop com.example.android.vdmdemo.host
226    ```
227
228#### Client capabilities
229
230-   **Enable client Sensors**: Enables sensor injection from the client device
231    into the host device. Any context that is associated with the virtual device
232    will access the virtual sensors by default. \
233    *Changing this will recreate the virtual device.*
234
235-   **Enable client Camera**: Enables front & back camera injection from the client device
236    into the host device. (WIP: Any context that is associated with the virtual device
237    will the virtual cameras by default). Run the commands below on host device \
238    to enable this functionality.
239    *Changing this will recreate the virtual device.*
240
241    ```shell
242    adb shell device_config put virtual_devices android.companion.virtual.flags.virtual_camera true
243    adb shell device_config put virtual_devices android.companion.virtualdevice.flags.virtual_camera_service_discovery true
244    adb shell am force-stop com.example.android.vdmdemo.host
245    ```
246
247-   **Enable client Audio**: Enables audio output on the client device. Any
248    context that is associated with the virtual device will play audio on the
249    client by default. \
250    *This can be changed dynamically.*
251
252#### Displays
253
254-   **Display rotation**: Whether orientation change requests from streamed apps
255    should trigger orientation change of the relevant display. The client will
256    automatically rotate the relevant display upon such request. Disabling this
257    simulates a fixed orientation display that cannot physically rotate. Then
258    any streamed apps on that display will be letterboxed/pillarboxed if they
259    request orientation change. Run the commands below to enable this
260    functionality. \
261    *This can be changed dynamically but only applies to newly created
262    displays.*
263
264    ```shell
265    adb shell device_config put virtual_devices android.companion.virtual.flags.consistent_display_flags true
266    adb shell am force-stop com.example.android.vdmdemo.host
267    ```
268
269-   **Always unlocked**: Whether the virtual displays should remain unlocked and
270    interactive when the host device is locked. Disabling this will result in a
271    simple lock screen shown on these displays when the host device is locked. \
272    *Changing this will recreate the virtual device.*
273
274-   **Show pointer icon**: Whether pointer icon should be shown for virtual
275    input pointer devices. \
276    *This can be changed dynamically.*
277
278-   **Custom home**: Whether to use a custom activity as home on home displays,
279    or use the device-default secondary home activity. Run the commands below to
280    enable this functionality. \
281    *Changing this will recreate the virtual device.*
282
283    ```shell
284    adb shell device_config put virtual_devices android.companion.virtual.flags.vdm_custom_home true
285    adb shell am force-stop com.example.android.vdmdemo.host
286    ```
287
288#### Input method
289
290Note: The virtual keyboard acts like a physically connected keyboard to the
291host device. If you want the software keyboard to be shown on the virtual
292displays, you likely need to enable this in the host Settings. On a Pixel
293device: System -> Language and input -> Physical keyboard.
294
295-   **Display IME policy**: Choose the IME behavior on remote displays. Run the
296    commands below to enable this functionality. \
297    *This can be changed dynamically.*
298
299    ```shell
300    adb shell device_config put virtual_devices android.companion.virtual.flags.vdm_custom_ime true
301    adb shell am force-stop com.example.android.vdmdemo.host
302    ```
303
304-   **Use the native client IME**: Enables the native client IME instead of
305    streaming the host's IME on the virtual displays. Requires the *Display IME
306    Policy* to be set to *Show IME on the remote display*. Run the commands
307    below to enable this functionality. \
308    *Changing this will recreate the virtual device.*
309
310    ```shell
311    adb shell device_config put virtual_devices android.companion.virtual.flags.vdm_custom_ime true
312    adb shell am force-stop com.example.android.vdmdemo.host
313    ```
314
315#### Debug
316
317-   **Record encoder output**: Enables recording the output of the encoder on
318    the host device to a local file on the device. This can be helpful with
319    debugging Encoding related issues. To download and play the file locally:
320
321    ```shell
322    adb pull /sdcard/Download/vdmdemo_encoder_output_<displayId>.h264
323    ffplay -f h264 vdmdemo_encoder_output_<displayId>.h264
324    ```
325
326<!-- LINT.ThenChange(README.md) -->
327<!-- LINT.IfChange(client_options) -->
328
329## Client Options
330
331### Streamed displays
332
333-   Each display on the Client app has a "Back" and "Close" buttons. When a
334    display becomes empty, it's automatically removed.
335
336-   Each display on the Client app has a "Rotate" button to switch between
337    portrait and landscape orientation. This simulates the physical rotation of
338    the display of the streamed activity. The "Resize" button can be used to
339    change the display dimensions.
340
341-   Each display on the Client app has a "Fullscreen" button which will move the
342    contents of that display to an immersive fullscreen activity. The client's
343    back button/gestures are sent back to the streamed app. Use Volume Down on
344    the client device to exit fullscreen. Volume Up acts as a home key, if the
345    streamed display is a home display.
346
347### Input
348
349The input menu button enables **on-screen D-Pad and touchpad** for navigating
350the activity on the focused display. The focused display is indicated by the
351frame around its header whenever there are more than one displays. The display
352focus is based on user interaction.
353
354In addition, any input events generated from an **externally connected
355keyboard** are forwarded to the activity streamed on the focused display.
356
357**Externally connected mouse** events are also forwarded to the relevant
358display, if the mouse pointer is currently positioned on a streamed display.
359
360<!-- LINT.ThenChange(README.md) -->
361<!-- LINT.IfChange(demos) -->
362
363## Demos
364
365-   **Sensors**: A simple activity balancing a beam on the screen based on the
366    accelerometer events, which allows for selecting which device's sensor to
367    use. By default, will use the sensors of the device it's shown on.
368
369-   **Rotation**: A simple activity that is in landscape by default and can send
370    orientation change requests on demand. Showcases the display rotation on the
371    client, which will rotate the user-visible surface.
372
373-   **Home**: A simple activity with utilities around launching HOME and
374    SECONDARY_HOME Intents.
375
376-   **Secure Window**: A simple activity that declares the Window as secure.
377    This showcases the FLAG_SECURE streaming policies in VDM.
378
379-   **Permissions**: A simple activity with buttons to request and revoke
380    runtime permissions. This can help test the permission streaming and
381    device-aware permission features.
382
383-   **Latency**: Renders a simple counter view that renders a new frame with an
384    incremented counter every second. Can be useful for debugging latency,
385    encoder, decoder issues in the demo application.
386
387-   **Vibration**: A simple activity making vibration requests via different
388    APIs and allows for selecting which device's vibrator to use. By default,
389    will use the vibrator of the device it's shown on. Note that currently there
390    is no vibration support on virtual devices, so vibration requests from
391    streamed activities are ignored.
392
393-   **Stylus**: A simple drawing activity that reacts on stylus input events.
394    Use together with the simulated stylus input feature of the host app.
395
396<!-- LINT.ThenChange(README.md) -->
397
398## SDK Version
399
400### Android 15 / Vanilla Ice Cream / SDK level 35
401
402-   Added support for cross-device clipboard.
403
404-   Added support for custom home activities.
405
406-   Added support for custom IME component.
407
408-   Added support for per-display IME policy.
409
410-   Added support for fixed orientation displays (disable display rotation).
411
412-   Added support for mirroring the default display on the virtual device.
413
414-   Added support for dynamic policy changes, so the device does not need to be
415    recreated.
416
417-   Improved support for displays that support home activities. Removed
418    navigation bar and added support for normal home intents.
419
420-   Improved handling of vibrating requests originating from virtual devices.
421
422-   Improved multi-display mouse support.
423
424-   Fixed bugs with hiding streamed apps from the host's recent apps.
425
426### Android 14 / Upside Down Cake / SDK level 34
427
428-   Added support for virtual sensors.
429
430-   Added device awareness to contexts.
431
432-   Added support for clipboard on the virtual device.
433
434-   Added support for hiding streamed apps from the host's recent apps.
435
436-   Added `COMPANION_DEVICE_NEARBY_DEVICE_STREAMING` device profile.
437
438-   Added support for virtual navigation input: D-Pad and navigation touchpad.
439
440-   Improved support for audio, allowing routing to be based on the origin
441    context.
442
443-   Improved support for creation of virtual displays and input devices.
444
445-   Improved handling of virtual touch events.
446
447### Android 13 / Tiramisu / SDK level 33
448
449-   Added support for virtual audio device.
450
451-   Added support for hiding the mouse pointer icon.
452
453-   Added support for virtual mouse, keyboard, touchscreen.
454
455-   Added support for always unlocked displays.
456
457-   Added `COMPANION_DEVICE_APP_STREAMING` device profile.
458
459-   Added support for virtual device creation.
460