Media on the Android Platform

March 6, 2010

Jason Shah
jshah@jsdfllc.com
http://www.jsdfllc.com
Twitter: jasonshah
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

2
Who am I?

• Current: jsdf, LLC: strategy and technology

consulting

• Past:
• Neuros Technology
• Bain & Company
• Trilogy Software

• MS Computer Science UIUC; MBA University

of Chicago Booth

3
Current Android work

Mediafly Mobile
(confidential)
(confidential)

Chicago Transit
Tracker Pro / Lite
US National Debt
(sample app)

4
 Target audience

• Developers interested in getting started with

building media applications on Android devices

• Android 1.5 = API level 3 (for now…)

5
Source: Google
Goals for today

• Introduce the android.media and related

classes in the API

• Provide an overview of how to use the basic

classes

• Demonstrate android.media

• Introduce additional resources to get help

6
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

7
What Android gives you

Subject Section Description

Codec support General • Internal video, audio and image codec support
MediaPlayer android.media • Control playback of video/audio files and streams
MediaController android.widget • Standard controls for a MediaPlayer
VideoView android.widget • View used to display a video file
SoundPool android.media • Manage and play audio resources for applications
AsyncPlayer android.media • Play a series of audio URIs, but does all the hard
work on another thread

MediaRecorder android.media • Record audio and video

FaceDetector android.media • Identifies the faces of people in a bitmap object
AudioManager android.media • Change various audio system properties (audio
route, volume, ringer, etc.)

AudioTrack android.media • Stream PCM buffers directly to the audio

hardware for playback

Lots of other stuff android.media • Ringtones, audio volume, media scanners, JET
players, audio formats, and more.

Bold = covered today

8
 Audio codec support

Format Enc/Dec Details File types Core/G1

AAC LC/LTP N/Y Mono/Stereo content in any 3GPP (.3gp) and MPEG- Core
combination of standard bit rates up 4 (.mp4, .m4a). No
HE-AACv1 N/Y
to 160 kbps and sampling rates from support for raw AAC
HE-AACv2 N/Y 8 to 48kHz (.aac)
AMR-NB Y/Y 4.75 to 12.2 kbps sampled @ 8kHz 3GPP (.3gp) Core
AMR-WB N/Y 9 rates from 6.60 kbit/s to 23.85 3GPP (.3gp) Core
kbit/s sampled @ 16kHz
MP3 N/Y Mono/Stereo 8-320Kbps constant MP3 (.mp3) Core
(CBR) or variable bit-rate (VBR)
MIDI N/Y MIDI Type 0 and 1. DLS Version 1 Type 0 and 1 (.mid, Core
and 2. XMF and Mobile XMF. Support .xmf, .mxmf). Also
for ringtone formats RTTTL/RTX, RTTTL/RTX (.rtttl, .rtx),
OTA, and iMelody OTA (.ota), and iMelody
(.imy)
Ogg Vorbis N/Y Ogg (.ogg) Core
PCM/WAVE N/Y 8- and 16-bit linear PCM (rates up to WAVE (.wav) Core
limit of hardware)
WMA N/Y Supports WMA standard L1-L3 with Windows Media Audio G1
specific kbps ranges (.wma)
Note: Core/G1 indicates whether codec is guaranteed to work across all devices, or specific to G1 device.
Additional codecs may be supported by future devices. 9
Source: http://developer.android.com/guide/appendix/media-formats.html
 Video codec support

Formats Enc/Dec Details File types Core/G1

H.263 Y/Y 3GPP (.3gp) and MPEG- Core
4 (.mp4)
H.264 N/Y On the G1, this decoder is limited to 3GPP (.3gp) and MPEG- Core
baseline profile up to 480x320, and 4 (.mp4)
600 kbps average bitrate for the
video stream.
MPEG-4 SP N/Y 3GPP (.3gp) Core
WMV N/Y Versions 7, 8 and 9. Simple profile Windows Media Video G1
only (.wmv)

Note: Core/G1 indicates whether codec is guaranteed to work across all devices, or specific to G1 device.
Additional codecs may be supported by future devices. 10
Source: http://developer.android.com/guide/appendix/media-formats.html
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

11
MediaPlayer: Introduction

• MediaPlayer lets you easily control playback of

audio/video files and streams

• Playback source can be:

• Raw resources bundled with the application
• Local files
• http/rtsp streams

• MediaPlayer blocks the UI, so be sure to wrap

your own thread around its functions, or use
AsyncPlayer
12
MediaPlayer: State diagram

Notes:
• Single arrow = synchronous
• Double arrow = asynchronous
• Error and End states can be
reached from any state in the
diagram

13
MediaPlayer: Create and prepare

Option 1:
MediaPlayer mp =
MediaPlayer.create(context,
uri);

Option 2:
MediaPlayer mp;
mp = new MediaPlayer();
mp.setDataSource(uri);
mp.prepare();

Notes:
• setDataSource(): sets the data
source as URI or file descriptor
• prepare(): Synchronously prepares
data for playback by buffering data.
• Asynchronous version
(prepareAsync) also available.
• Callback OnPreparedListener
called if registered on
14
completion
MediaPlayer: Seek to time

mp.seekTo(time);

Notes:
• seekTo(): seeks to a specific time
in the file in milliseconds.
• Call returns immediately,
but seeking may take several
seconds (especially for
streams).
• Callback
OnSeekCompleteListener
called if registered on
completion
• seekTo on streams is broken in
Android 1.6-2.1! See Android bug
#4124

15
MediaPlayer: Start playback

mp.start();

Notes:
• start(): starts playback.

16
MediaPlayer: demo

• Demo basic playback

17
MediaPlayer: Additional control

• mp.pause();
• Stops playback and keeps
current seek position
• mp.stop();
• Stops playback and loses
current seek position
• mp.seekTo();
• Move playhead to new time,
buffer new content, and start
playing. May take several
seconds, especially if
streaming.
• mp.reset();
• Move player back to Idle
state
• mp.release();
• Move player to End state
• Release all internal
resources held by player

18
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

19
MediaController: Introduction

• MediaController gives you standard controls for

a MediaPlayer
• Play, pause, rewind, fast forward, reverse, forward
• Seekable progress bar with overlaid playhead and buffer
status
• Handles control updates and synchronization via
callbacks

• BUT it is peculiar: it is written primarily to be

shown for a period of time then hidden
• Customization (either by extension or rewrite) may be
necessary to achieve your desired behavior

20
 MediaController: Introduction

I’m a
MediaController

21
Image source: http://www.simplehelp.net/2009/07/29/how-to-copy-music-to-your-android-phone-from-windows/
MediaController: Setting it up

• Create and associate your MediaController to an ‘anchor’

view:
MediaController controller = new MediaController(this);
Button showControlsBtn =
(Button)findViewById(R.id.showcontrols);
controller.setAnchorView(showControlsBtn);

• When that ‘anchor’ view is activated, your MediaController

will show for 3 seconds

• To show your MediaController manually, call show:

controller.show();

• MediaController will disappear automatically after 3 seconds

by default
22
MediaController: Handling callbacks

• Register a callback listener to control your MediaPlayer:

controller.setMediaPlayer(new MediaController.MediaPlayer() {
public int getBufferPercentage() {}
public int getCurrentPosition() {}
public int getDuration() {}
public boolean isPlaying() {}
public void pause() {}
public void seekTo(int pos) {}
public void start() {}
});

• Connect the callbacks to your MediaPlayer, e.g.:

public void start() {
mp.start();
}
23
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

24
 VideoView: Introduction

• VideoView attaches to a
MediaPlayer to provide default
video rendering capabilities
for video from various sources
• Computes its own
measurement from the video
• Can be used in any layout
manager
• Provides scaling, tinting, etc.

25
Image source: http://www.saturn.dti.ne.jp/~npaka/android/VideoViewEx/VideoViewEx.gif
VideoView: Setting it up

• Create your VideoView (in XML in this case):

<VideoView android:id="@+id/video”
android:layout_width="fill_parent”
android:layout_height="fill_parent” />

• Load your video file:

File clip=new File("/sdcard/test.mp4");
VideoView video=(VideoView)findViewById(R.id.video);
video.setVideoPath(clip.getAbsolutePath());

• Associate your MediaController to VideoView and vice versa

MediaController controller = new MediaController(this);
controller.setMediaPlayer(video);
video.setMediaController(controller);

• Show your video and its controls

video.requestFocus();
26
VideoView: demo

• Demo VideoView

27
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

28
SoundPool: Introduction

• SoundPool manages and plays audio resources for

audio-heavy applications/games

• SoundPool:
• Loads samples into memory from resources or files and
decompresses them
• Manages the number of actively playing streams by
assigned priority
• Allows playback rate and pitch changes real-time

29
SoundPool: Gaming Use Case

• A typical use case: game has many levels of play,

with unique sounds within each level
• The game would:
• Create a new SoundPool object on level load
• Load each sound using the appropriate SoundPool.load()
object in a thread
• Perform load early in the level load process to allow time
for sound decompression to complete
• Play sounds with SoundPool.play(); pause, resume,
adjust pitch, adjust speed, etc. all as required
• On level complete, clean up with SoundPool.release()

30
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

31
Trouble ahead!

• 640 pixel wide videos do not play

• Downloaded videos have a higher success rate than
streamed videos
• Android reacts arbitrarily to certain videos
• E.g. ABC News 30% success rate
• No ability to push your own bytestream into audio
or video players
• However, if you are recording then playing back audio,
and want to use PCM audio, see AudioTrack and
AudioRecord
• No access to MediaPlayer’s buffers for advanced
buffering algorithms
• seekTo on streams broken on Android 1.6 to 2.1

32
Agenda

• Who am I? / Goals for today

• What Android gives you
• MediaPlayer
• MediaController
• VideoView
• SoundPool
• Trouble ahead!
• References

33
References

Reference Source
Android 1.5 API http://developer.android.com/sd
k/1.5_r3/index.html

CommonsWare Busy Coder series, by http://commonsware.com/

Mark Murphy: “Android Development”
and “Advanced Android Development”

“Mastering the Android Media http://www.youtube.com/watch?

Framework” by David Sparks, Technical v=-0UmSQeWsJc
Lead of Android Media Framework

34
Questions?

Thank you!

Jason Shah
jshah@jsdfllc.com
http://www.jsdfllc.com
Twitter: jasonshah

35