We have discussed
working with the Media Library and the Picture hub with Photo Extras
applications. In this section, we discuss video and audio media. Windows Phone 7 third-party applications cannot
add media to the Media Library but third-party applications can add
content to Isolated Storage in addition to playing video and audio
content in memory.
In this section we cover supported codecs, playing
video, audio, and DRM considerations within the context of the
programming model.
1. Supported Codecs and Containers
Windows Phone 7 has extensive codec and container support as detailed at this link:
http://msdn.microsoft.com/en-us/library/ff462087%28VS.92%29.aspx
Container support for a variety of media includes
WAV, MP3, WMA, 3GP, MP4, 3G2, M4A, WMV, M4V, 3G2, JPG, and PNG. Of note,
GIF, BMP, and TIF are not supported in the Image control in Silverlight.
Decoder support is a long list at the above link, but
of note support is available for MP3, WMA Standard v9, AAC-LC, HE-AAC
v1, HE-AAC v2, AMR-NB, WMV (VC-1, WMV9) simple, main, and advanced. Also
supported are MPEG-4 Part 2, Simple, Advanced Simple, MPEG-4 Part 10
Level 3.0 Baseline, Main, High, and H.263. There are specific
requirements in terms of bitrate, constant bitrate vs. variable bitrate,
as well as methods of playback, so check the above link when
troubleshooting playback issues on a device to see if a capability or
playback method is not correct.
2. Video and Audio in the Emulator
The emulator does not support all of the codecs
supported on a device. Specifically, AAC-LC, HE-AAC v1, HE-AAC v2,
AMR-NB, MPEG-4 Part 2 simple, MPEG-4 Part 2 advanced, MPEG-4 Part 10
level 3 baseline, main or high, and H.263 are all not supported.
The following codecs are supported but with caveats:
WMV (VC-1, WMV9) – Simple Profile – Unsupported above 320 × 176 pixels.
WMV (VC-1, WMV9) - Main Profile – Unsupported above 800 × 488 pixels.
WMV (VC-1, WMV9) - Advanced Profile – Unsupported above 720 × 480 pixels.
When investigating video support for your content, it
is recommended to test on a real device first to ensure that an
accurate assessment is made.
3. Progressive Download Video
Progressive download is generally used for short form
video, clips, music videos, and the like. The file is completely
downloaded over HTP while playback begins immediately.
The MainPage is a menu page to launch additional pages to
test out functionality starting with testing progressive play back in
the MediaPlayerLauncher.
3.1. Progressive Video with MediaPlayerLauncher
Silverlight supports both progressive and streaming video via the MediaElement
control, but a simple way to play progressive non-DRM video is via the
Microsoft.Phone.Tasks.MediaPlayerLauncher task. A page named
MediaPlayeTask.xaml is added to the WorkingWithMedia project that has a
single button titled play video.
private void textBlock1_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
MediaPlayerLauncher mediaPlayerLauncher = new MediaPlayerLauncher();
mediaPlayerLauncher.Controls = MediaPlaybackControls.FastForward |
MediaPlaybackControls.Pause | MediaPlaybackControls.Rewind |
MediaPlaybackControls.Skip | MediaPlaybackControls.Stop;
mediaPlayerLauncher.Location = MediaLocationType.Data;
mediaPlayerLauncher.Media = new
Uri("http://ecn.channel9.msdn.com/o9/ch9/8/9/6/6/3/5/WP7Xbox_ch9.mp4");
mediaPlayerLauncher.Show();
}
As you can see, it is possible to configure what vcr
controls are visible during playback. In the next section, we cover
playing video with the MediaElement, which is pretty simple as
well – but you will need to build up your own media player for best user
experience. This is because the MediaElement is a blank canvas
with a highly programmable API. By default, it does not have any media
controls and the like. Unless you need specific functionality or want
complete control over the playback UI the MediaPlayerLauncher task is easiest.
3.2. Progressive Video with MediaElement
TheMediaElement is a very powerful control with a
rich API to provide fined-grained control during playback with a rich
event model. We create a simple example that hooks into the interesting
events as well as displays values for video related properties.
A new page named MediaElement.xaml is added to the WorkingWithMedia project. A MediaElement
control is added to the page. The application bar is enabled to host a
play and pause button that will programmatically control the MediaElement control. Several TextBlocks are laid out to display values from various MediaElement properties as shown in Figure 1.
There are several interesting events available with the MediaElement control. Listing 1 shows the available media-related events.
Example 1. PictureEditorView .xaml.cs Code File
using System;
using System.Windows;
using System.Windows.Threading;
using Microsoft.Phone.Controls;
namespace WorkingWithMedia.pages
{
public partial class MediaElement : PhoneApplicationPage
{
public MediaElement()
{
InitializeComponent();
DispatcherTimer timer = new DispatcherTimer();
timer.Tick += new EventHandler(timer_Tick);
timer.Interval = new TimeSpan(0, 0, 1);
timer.Start();
}
void timer_Tick(object sender, EventArgs e)
{
CanSeekTextBlock.Text = mediaPlayer.CanSeek.ToString();
CanPauseTextBlock.Text = mediaPlayer.CanPause.ToString();
DroppedFramesTextBlock.Text =
mediaPlayer.DroppedFramesPerSecond.ToString();
}
private void mediaPlayer_MediaOpened(object sender, RoutedEventArgs e)
{
mediaPlayer.Play();
}
private void mediaPlayer_MediaFailed(object sender,
ExceptionRoutedEventArgs e)
{
MessageBox.Show("Media Failed: " + e.ErrorException.Message);
}
private void mediaPlayer_MediaEnded(object sender, RoutedEventArgs e)
{
}
private void PlayAppBarBtn_Click(object sender, EventArgs e)
{
mediaPlayer.Source =
new Uri("http://ecn.channel9.msdn.com/o9/ch9/8/9/6/6/3/5/WP7Xbox_ch9.wmv",
UriKind.Absolute);
}
private void PauseAppBarBtn_Click(object sender, EventArgs e)
{
mediaPlayer.Pause();
}
private void mediaPlayer_CurrentStateChanged(object sender,
RoutedEventArgs e)
{
CurrentStateTextBlock.Text = mediaPlayer.CurrentState.ToString();
}
private void mediaPlayer_BufferingProgressChanged(object sender,
RoutedEventArgs e)
{
BufferingProgressTextBlock.Text = mediaPlayer.BufferingProgress.ToString();
}
private void mediaPlayer_DownloadProgressChanged(object sender,
RoutedEventArgs e)
{
DownloadProgressChangedTextBlock.Text = mediaPlayer.DownloadProgress.ToString();
}
}
}
|
Notice that clicking play doesn't start playback. Instead, it sets the Source property on the MediaElement control. If the video configured on the Source property is successfully opened, the MediaOpened event fires, which is where Play is actually called.
3.3. MediaStreamSource
For custom video delivery, you can use the MediaElement.SetSource() method in your code to specify the media to be played. One overload of SetSource() accepts a System.IO.Stream, which is suited for the scenario where you decide to acquire the media through some other mechanism rather than have the MediaElement handle the download. When you acquire the media file, you can create a Stream around it (using a more concrete type like System.IO.IsolatedStorage.IsolatedStorageFileStream) and pass it to SetSource().
The second overload of SetSource() accepts an instance of the System.Windows.Media.MediaStreamSource type. The MediaStreamSource type is actually a way to plug a video container file format into Silverlight, for which the MediaElement
does not come with a built-in parser. Video container file formats and
related specifications are complex topics, and consequently a treatment
of MediaStreamSource implementations .
When the Source is set by either mechanism, for progressive download scenarios the MediaElement immediately starts to download the media. The MediaElement.DownloadProgressChanged event is raised repeatedly as the download progresses. The MediaElement.DownloadProgress
property reports the download progress as a percentage value (actually a
double between 0 and 1 that you can convert to percentage) that you can
use to track and report the download progress in the DownloadProgressChanged eventhandler.
4. Streaming Video
In addition to progressive download video, streaming
video is another technique used to deliver media to a player. Streaming
does not require downloading the media file locally, and it is well
suited for scenarios involving either live or on-demand broadcasts to a
large population of viewers.
Microsoft provides Smooth Streaming, an IIS Media
Services extension that enables adaptive streaming of media to
Silverlight and other clients over HTTP. Smooth Streaming provides a
high-quality viewing experience that scales massively on content
distribution networks, making true HD 1080p media experiences a reality.
Smooth Streaming is the productized version of
technology first used by Microsoft to deliver on-demand video for the
2008 Summer Olympics at NBCOlympics.com. By dynamically monitoring both
local bandwidth and video rendering performance, Smooth Streaming
optimizes playback of content by switching video quality in real-time to
match current network conditions resulting in a better viewing
experience.
Smooth Streaming uses the simple but powerful concept
of delivering small content fragments (typically two seconds worth of
video) and verifying that each has arrived within the appropriate time
and played back at the expected quality level. If one fragment does not
meet these requirements, the next fragment delivered will be at a
somewhat lower quality level. Conversely, when conditions allow it, the
quality of subsequent fragments will be at a higher level if network
conditions improve.
4.1. SmoothStreamingMediaElement
The Microsoft.Web.Media.SmoothStreaming.SmoothStreamingMediaElement
is a customized version of the MediaElement that works directly with
IIS Smooth Streaming technology. The SmoothStreamingMediaElement
available in Windows Phone 7 is compatible with the IIS Smooth Streaming
Client 1.1. Windows Phone 7 is a subset of the full client support. The
phone icon in the API reference identifies members of classes in the
Microsoft.Web.Media.SmoothStreaming namespace that are available for
Windows Phone 7 development. Review the documentation at the following
link:
http://msdn.microsoft.com/en-us/library/microsoft.web.media.smoothstreaming(v=VS.90).aspx
IIS Smooth Streaming combined with the
SmoothStreamingMediaElement provide an incredibly powerful and flexible
streaming capability with support for multiple video and audio tracks,
multiple languages, subtitles, Timeline Markers, Fast Forward and
Rewind, and metadata in tracks for "pop-up" information during playback
for a rich media experience. Please refer to this link for details on
capabilities:
http://msdn.microsoft.com/en-us/library/ee958035(v=VS.90).aspx
4.2. Silverlight Media Framework
The Silverlight Media Framework (SMF), or "smurf," is
an open source CodePlex project that provides a robust, scalable
customizable media player for IIS Smooth Streaming media delivery. The
SMF builds on the core functionality of the Smooth Streaming Client
(formerly known as the Smooth Streaming Player Development Kit) and adds
a large number of additional features, including an extensibility API
that allows developers to create plug-ins for the framework. The SMF
also now includes full support for Windows Phone 7 so developers can
incorporate high-end video playback experiences in their Windows Phone 7
applications. Please check out the SMF at this link:
http://smf.codeplex.com/
5. DRM
Most long form video, whether it be TV show episodes,
movies, or live TV, has some form of Digital Rights Management (DRM).
As the goal of DRM is to prevent illegal copying of digital content,
securely storing licenses to content is of paramount concern.
Silverlight for Windows Phone 7 supports DRM via
Microsoft PlayReady. Microsoft PlayReady is a platform independent DRM
content access technology that is optimized for broad use across a range
of devices. Microsoft PlayReady supports subscription, purchase,
rental, and pay-per-view business models that can be applied to many
digital content types and to a wide range of audio and video formats.
On Windows Phone 7, there is support for Microsoft
PlayReady built into the platform including a secure key store for
online and offline DRM key storage. After DRM media is accessed the
first time, the license is stored offline enabling offline playback of
DRM'd content.
Here is a link for more information on Microsoft PlayReady technology:
www.microsoft.com/PlayReady/Default.mspx
When it comes to DRM protected content available via
the Zune Service, either purchased or subscription, you can play that
content within Silverlight applications using the MediaElement as well
as the MediaPlayer available in the XNA Framework. License acquisition
occurs behind the scenes with no additional work necessary by the
developer. For content that is not owned by the end-user, you can use
the MarketplaceDetailsTask to show UI that allows the user to purchase
the content.
6. Audio Support
Because audio is critical to video playback, audio
support is covered in pretty good detail in concert with what has been
discussed above regarding video playback. However, in addition to the
MediaElement and SmoothStreamingMediaElement, developers can play audio
using the Microsoft.Xna.Framework.Media.MediaPlayer object.
private void ListBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
object obj = (sender as ListBox).SelectedItem;
MediaLibrary library = new MediaLibrary();
try
{
switch (obj.GetType().ToString())
{
case "Microsoft.Xna.Framework.Media.Album": MediaPlayer.Play(((Album)(obj)).Songs); break;
case "Microsoft.Xna.Framework.Media.Song": MediaPlayer.Play((Song)(obj)); break;
case "Microsoft.Xna.Framework.Media.Playlist": MediaPlayer.Play(((Playlist)(obj)).Songs); break;
case "Microsoft.Xna.Framework.Media.Artist": MediaPlayer.Play(((Artist)(obj)).Songs); break;
}
}
catch (Exception ex)
{
MessageBox.Show("Error: " + ex.Message);
};
}
The MediaPlayer object can play by Album,
Song, Playlist, and Artist, but only one of these items at a time. So,
for example, for a custom play list you must provide one song at a time.
The MediaPlayer object has two events that you can hook, MediaStateChanged and ActiveSongChanged. MediaStateChanged fires when state changes from Paused, Playing, or Stopped to another MediaState value. Here is a code example, not from a specific project but provided as reference:
public MainPage()
{
InitializeComponent();
MediaPlayer.MediaStateChanged += new EventHandler<EventArgs>(MediaPlayer_MediaStateChanged);
MediaPlayer.ActiveSongChanged += new EventHandler<EventArgs>(MediaPlayer_ActiveSongChanged);
}
void MediaPlayer_ActiveSongChanged(object sender, EventArgs e)
{
if (MediaPlayer.Queue.Count == 0)
{
//add a song to Play
MediaPlayer.Play(nextSong);
}
}
void MediaPlayer_MediaStateChanged(object sender, EventArgs e)
{
switch (MediaPlayer.State)
{
case MediaState.Paused:
break;
case MediaState.Playing:
break;
case MediaState.Stopped:
break;
}
}
On a related note, you can prompt the user to purchase additional music using MarketplaceSearchTask and passing in a search string containing the artist, album, and song title. You can also use the WebBrowserTask and Zune Http links for content if you have the ID for the content. Here is an example:
WebBrowserTask task = new WebBrowserTask();
task.URL = HttpUtility.UrlEncode("http://social.zune.net/External/LaunchZuneProtocol.aspx?
pathuri=navigate%3FalbumID%3Dd1935e06-0100-11db-89ca-0019b92a3933");
task.Show();
You can then use the MediaLibrary object
covered above and demonstrated in the WorkingWithMediaLibrary project to
find acquired content and show album art as well as purchase the media.
You might guess that the MarketplaceDetailsTask could also show the purchase UI but it is not supported in this release.
