IT tutorials
 
Mobile
 

Windows Phone 7 Advanced Programming Model : Working with Video and Audio Media

1/11/2013 11:27:16 AM
- How To Install Windows Server 2012 On VirtualBox
- How To Bypass Torrent Connection Blocking By Your ISP
- How To Install Actual Facebook App On Kindle Fire

We have discussed working with the Media Library and the Picture hub with Photo Extras applications. In this section, we discuss video and audio media. Windows Phone 7 third-party applications cannot add media to the Media Library but third-party applications can add content to Isolated Storage in addition to playing video and audio content in memory.

In this section we cover supported codecs, playing video, audio, and DRM considerations within the context of the programming model.

1. Supported Codecs and Containers

Windows Phone 7 has extensive codec and container support as detailed at this link:

http://msdn.microsoft.com/en-us/library/ff462087%28VS.92%29.aspx

Container support for a variety of media includes WAV, MP3, WMA, 3GP, MP4, 3G2, M4A, WMV, M4V, 3G2, JPG, and PNG. Of note, GIF, BMP, and TIF are not supported in the Image control in Silverlight.

Decoder support is a long list at the above link, but of note support is available for MP3, WMA Standard v9, AAC-LC, HE-AAC v1, HE-AAC v2, AMR-NB, WMV (VC-1, WMV9) simple, main, and advanced. Also supported are MPEG-4 Part 2, Simple, Advanced Simple, MPEG-4 Part 10 Level 3.0 Baseline, Main, High, and H.263. There are specific requirements in terms of bitrate, constant bitrate vs. variable bitrate, as well as methods of playback, so check the above link when troubleshooting playback issues on a device to see if a capability or playback method is not correct.

2. Video and Audio in the Emulator

The emulator does not support all of the codecs supported on a device. Specifically, AAC-LC, HE-AAC v1, HE-AAC v2, AMR-NB, MPEG-4 Part 2 simple, MPEG-4 Part 2 advanced, MPEG-4 Part 10 level 3 baseline, main or high, and H.263 are all not supported.

The following codecs are supported but with caveats:

  • WMV (VC-1, WMV9) – Simple Profile – Unsupported above 320 × 176 pixels.

  • WMV (VC-1, WMV9) - Main Profile – Unsupported above 800 × 488 pixels.

  • WMV (VC-1, WMV9) - Advanced Profile – Unsupported above 720 × 480 pixels.

When investigating video support for your content, it is recommended to test on a real device first to ensure that an accurate assessment is made.

3. Progressive Download Video

Progressive download is generally used for short form video, clips, music videos, and the like. The file is completely downloaded over HTP while playback begins immediately.

The MainPage is a menu page to launch additional pages to test out functionality starting with testing progressive play back in the MediaPlayerLauncher.

3.1. Progressive Video with MediaPlayerLauncher

Silverlight supports both progressive and streaming video via the MediaElement control, but a simple way to play progressive non-DRM video is via the Microsoft.Phone.Tasks.MediaPlayerLauncher task. A page named MediaPlayeTask.xaml is added to the WorkingWithMedia project that has a single button titled play video.

private void textBlock1_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
  MediaPlayerLauncher mediaPlayerLauncher = new MediaPlayerLauncher();
  mediaPlayerLauncher.Controls = MediaPlaybackControls.FastForward |
    MediaPlaybackControls.Pause | MediaPlaybackControls.Rewind |
    MediaPlaybackControls.Skip | MediaPlaybackControls.Stop;
  mediaPlayerLauncher.Location = MediaLocationType.Data;
  mediaPlayerLauncher.Media = new
Uri("http://ecn.channel9.msdn.com/o9/ch9/8/9/6/6/3/5/WP7Xbox_ch9.mp4");
  mediaPlayerLauncher.Show();
}

					  

As you can see, it is possible to configure what vcr controls are visible during playback. In the next section, we cover playing video with the MediaElement, which is pretty simple as well – but you will need to build up your own media player for best user experience. This is because the MediaElement is a blank canvas with a highly programmable API. By default, it does not have any media controls and the like. Unless you need specific functionality or want complete control over the playback UI the MediaPlayerLauncher task is easiest.


3.2. Progressive Video with MediaElement

TheMediaElement is a very powerful control with a rich API to provide fined-grained control during playback with a rich event model. We create a simple example that hooks into the interesting events as well as displays values for video related properties.

A new page named MediaElement.xaml is added to the WorkingWithMedia project. A MediaElement control is added to the page. The application bar is enabled to host a play and pause button that will programmatically control the MediaElement control. Several TextBlocks are laid out to display values from various MediaElement properties as shown in Figure 1.

Figure 1. MediaElement.xaml UI

There are several interesting events available with the MediaElement control. Listing 1 shows the available media-related events.

Example 1. PictureEditorView .xaml.cs Code File
using System;
using System.Windows;
using System.Windows.Threading;
using Microsoft.Phone.Controls;

namespace WorkingWithMedia.pages
{
  public partial class MediaElement : PhoneApplicationPage
  {
    public MediaElement()
    {
      InitializeComponent();
      DispatcherTimer timer = new DispatcherTimer();
      timer.Tick += new EventHandler(timer_Tick);
      timer.Interval = new TimeSpan(0, 0, 1);
      timer.Start();

}

    void timer_Tick(object sender, EventArgs e)
    {
      CanSeekTextBlock.Text = mediaPlayer.CanSeek.ToString();
      CanPauseTextBlock.Text = mediaPlayer.CanPause.ToString();
      DroppedFramesTextBlock.Text =
        mediaPlayer.DroppedFramesPerSecond.ToString();
    }

    private void mediaPlayer_MediaOpened(object sender, RoutedEventArgs e)
    {
      mediaPlayer.Play();
    }

    private void mediaPlayer_MediaFailed(object sender,
      ExceptionRoutedEventArgs e)
    {
      MessageBox.Show("Media Failed: " + e.ErrorException.Message);
    }

    private void mediaPlayer_MediaEnded(object sender, RoutedEventArgs e)
    {

    }

    private void PlayAppBarBtn_Click(object sender, EventArgs e)
    {
      mediaPlayer.Source =
        new Uri("http://ecn.channel9.msdn.com/o9/ch9/8/9/6/6/3/5/WP7Xbox_ch9.wmv",
          UriKind.Absolute);
    }

    private void PauseAppBarBtn_Click(object sender, EventArgs e)
    {
      mediaPlayer.Pause();
    }

    private void mediaPlayer_CurrentStateChanged(object sender,
      RoutedEventArgs e)
    {
      CurrentStateTextBlock.Text = mediaPlayer.CurrentState.ToString();
    }

    private void mediaPlayer_BufferingProgressChanged(object sender,
      RoutedEventArgs e)
    {
      BufferingProgressTextBlock.Text = mediaPlayer.BufferingProgress.ToString();
    }

    private void mediaPlayer_DownloadProgressChanged(object sender,
      RoutedEventArgs e)
    {
      DownloadProgressChangedTextBlock.Text = mediaPlayer.DownloadProgress.ToString();

					  

}
  }
}

Notice that clicking play doesn't start playback. Instead, it sets the Source property on the MediaElement control. If the video configured on the Source property is successfully opened, the MediaOpened event fires, which is where Play is actually called.

3.3. MediaStreamSource

For custom video delivery, you can use the MediaElement.SetSource() method in your code to specify the media to be played. One overload of SetSource() accepts a System.IO.Stream, which is suited for the scenario where you decide to acquire the media through some other mechanism rather than have the MediaElement handle the download. When you acquire the media file, you can create a Stream around it (using a more concrete type like System.IO.IsolatedStorage.IsolatedStorageFileStream) and pass it to SetSource().

The second overload of SetSource() accepts an instance of the System.Windows.Media.MediaStreamSource type. The MediaStreamSource type is actually a way to plug a video container file format into Silverlight, for which the MediaElement does not come with a built-in parser. Video container file formats and related specifications are complex topics, and consequently a treatment of MediaStreamSource implementations .

When the Source is set by either mechanism, for progressive download scenarios the MediaElement immediately starts to download the media. The MediaElement.DownloadProgressChanged event is raised repeatedly as the download progresses. The MediaElement.DownloadProgress property reports the download progress as a percentage value (actually a double between 0 and 1 that you can convert to percentage) that you can use to track and report the download progress in the DownloadProgressChanged eventhandler.

4. Streaming Video

In addition to progressive download video, streaming video is another technique used to deliver media to a player. Streaming does not require downloading the media file locally, and it is well suited for scenarios involving either live or on-demand broadcasts to a large population of viewers.

Microsoft provides Smooth Streaming, an IIS Media Services extension that enables adaptive streaming of media to Silverlight and other clients over HTTP. Smooth Streaming provides a high-quality viewing experience that scales massively on content distribution networks, making true HD 1080p media experiences a reality.

Smooth Streaming is the productized version of technology first used by Microsoft to deliver on-demand video for the 2008 Summer Olympics at NBCOlympics.com. By dynamically monitoring both local bandwidth and video rendering performance, Smooth Streaming optimizes playback of content by switching video quality in real-time to match current network conditions resulting in a better viewing experience.

Smooth Streaming uses the simple but powerful concept of delivering small content fragments (typically two seconds worth of video) and verifying that each has arrived within the appropriate time and played back at the expected quality level. If one fragment does not meet these requirements, the next fragment delivered will be at a somewhat lower quality level. Conversely, when conditions allow it, the quality of subsequent fragments will be at a higher level if network conditions improve.

4.1. SmoothStreamingMediaElement

The Microsoft.Web.Media.SmoothStreaming.SmoothStreamingMediaElement is a customized version of the MediaElement that works directly with IIS Smooth Streaming technology. The SmoothStreamingMediaElement available in Windows Phone 7 is compatible with the IIS Smooth Streaming Client 1.1. Windows Phone 7 is a subset of the full client support. The phone icon in the API reference identifies members of classes in the Microsoft.Web.Media.SmoothStreaming namespace that are available for Windows Phone 7 development. Review the documentation at the following link:

http://msdn.microsoft.com/en-us/library/microsoft.web.media.smoothstreaming(v=VS.90).aspx

					  

IIS Smooth Streaming combined with the SmoothStreamingMediaElement provide an incredibly powerful and flexible streaming capability with support for multiple video and audio tracks, multiple languages, subtitles, Timeline Markers, Fast Forward and Rewind, and metadata in tracks for "pop-up" information during playback for a rich media experience. Please refer to this link for details on capabilities:

http://msdn.microsoft.com/en-us/library/ee958035(v=VS.90).aspx

4.2. Silverlight Media Framework

The Silverlight Media Framework (SMF), or "smurf," is an open source CodePlex project that provides a robust, scalable customizable media player for IIS Smooth Streaming media delivery. The SMF builds on the core functionality of the Smooth Streaming Client (formerly known as the Smooth Streaming Player Development Kit) and adds a large number of additional features, including an extensibility API that allows developers to create plug-ins for the framework. The SMF also now includes full support for Windows Phone 7 so developers can incorporate high-end video playback experiences in their Windows Phone 7 applications. Please check out the SMF at this link:

http://smf.codeplex.com/

5. DRM

Most long form video, whether it be TV show episodes, movies, or live TV, has some form of Digital Rights Management (DRM). As the goal of DRM is to prevent illegal copying of digital content, securely storing licenses to content is of paramount concern.

Silverlight for Windows Phone 7 supports DRM via Microsoft PlayReady. Microsoft PlayReady is a platform independent DRM content access technology that is optimized for broad use across a range of devices. Microsoft PlayReady supports subscription, purchase, rental, and pay-per-view business models that can be applied to many digital content types and to a wide range of audio and video formats.

On Windows Phone 7, there is support for Microsoft PlayReady built into the platform including a secure key store for online and offline DRM key storage. After DRM media is accessed the first time, the license is stored offline enabling offline playback of DRM'd content.

Here is a link for more information on Microsoft PlayReady technology:

www.microsoft.com/PlayReady/Default.mspx

When it comes to DRM protected content available via the Zune Service, either purchased or subscription, you can play that content within Silverlight applications using the MediaElement as well as the MediaPlayer available in the XNA Framework. License acquisition occurs behind the scenes with no additional work necessary by the developer. For content that is not owned by the end-user, you can use the MarketplaceDetailsTask to show UI that allows the user to purchase the content.

6. Audio Support

Because audio is critical to video playback, audio support is covered in pretty good detail in concert with what has been discussed above regarding video playback. However, in addition to the MediaElement and SmoothStreamingMediaElement, developers can play audio using the Microsoft.Xna.Framework.Media.MediaPlayer object.

private void ListBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
  object obj = (sender as ListBox).SelectedItem;
  MediaLibrary library = new MediaLibrary();
  try
  {
    switch (obj.GetType().ToString())
    {
      case "Microsoft.Xna.Framework.Media.Album": MediaPlayer.Play(((Album)(obj)).Songs); break;
      case "Microsoft.Xna.Framework.Media.Song": MediaPlayer.Play((Song)(obj)); break;
      case "Microsoft.Xna.Framework.Media.Playlist": MediaPlayer.Play(((Playlist)(obj)).Songs); break;
      case "Microsoft.Xna.Framework.Media.Artist": MediaPlayer.Play(((Artist)(obj)).Songs); break;
    }
  }
  catch (Exception ex)
  {
    MessageBox.Show("Error: " + ex.Message);
  };
}

					  

The MediaPlayer object can play by Album, Song, Playlist, and Artist, but only one of these items at a time. So, for example, for a custom play list you must provide one song at a time. The MediaPlayer object has two events that you can hook, MediaStateChanged and ActiveSongChanged. MediaStateChanged fires when state changes from Paused, Playing, or Stopped to another MediaState value. Here is a code example, not from a specific project but provided as reference:

public MainPage()
{
  InitializeComponent();

MediaPlayer.MediaStateChanged += new EventHandler<EventArgs>(MediaPlayer_MediaStateChanged);
  MediaPlayer.ActiveSongChanged += new EventHandler<EventArgs>(MediaPlayer_ActiveSongChanged);
}

void MediaPlayer_ActiveSongChanged(object sender, EventArgs e)
{
  if (MediaPlayer.Queue.Count == 0)
  {
    //add a song to Play
    MediaPlayer.Play(nextSong);
  }
}

void MediaPlayer_MediaStateChanged(object sender, EventArgs e)
{
  switch (MediaPlayer.State)
  {
    case MediaState.Paused:
      break;
    case MediaState.Playing:
      break;
    case MediaState.Stopped:
      break;
  }
}

					  

On a related note, you can prompt the user to purchase additional music using MarketplaceSearchTask and passing in a search string containing the artist, album, and song title. You can also use the WebBrowserTask and Zune Http links for content if you have the ID for the content. Here is an example:

WebBrowserTask task = new WebBrowserTask();
task.URL = HttpUtility.UrlEncode("http://social.zune.net/External/LaunchZuneProtocol.aspx?
                    pathuri=navigate%3FalbumID%3Dd1935e06-0100-11db-89ca-0019b92a3933");
task.Show();

					  

You can then use the MediaLibrary object covered above and demonstrated in the WorkingWithMediaLibrary project to find acquired content and show album art as well as purchase the media.

You might guess that the MarketplaceDetailsTask could also show the purchase UI but it is not supported in this release.

 
Others
 
- Windows Phone 7 Advanced Programming Model : Building a Photo Extras Application
- IPad : Working with Contacts - Adding Contacts from E-mails, Sending an E-mail Message from Contacts
- IPad : Working with Contacts - Adding a Photo to Contacts, Searching Your Contacts
- Enter Java ME on Symbian OS : Which APIs Are Supported?
- Enter Java ME on Symbian OS : Running a MIDlet on a Symbian Smartphone
- BlackBerry Bold 9700 and 9650 Series : Fixing Problems - Updating the BlackBerry Operating System
- BlackBerry Bold 9700 and 9650 Series : Fixing Problems - Understanding Your Wireless Data Connection
- Android Application Development : The ApiDemos Application (part 2) - Adding Your Own Examples to ApiDemos
- Android Application Development : The ApiDemos Application (part 1) - Application Setup in the Manifest File, Finding the Source to an Interesting Example
- iphone Programming : Adding Missing Features (part 3) - Changing the Display Name, Enabling Rotation
 
 
Top 10
 
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 2) - Wireframes,Legends
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 1) - Swimlanes
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Formatting and sizing lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Adding shapes to lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Sizing containers
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 3) - The Other Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 2) - The Data Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 1) - The Format Properties of a Control
- Microsoft Access 2010 : Form Properties and Why Should You Use Them - Working with the Properties Window
- Microsoft Visio 2013 : Using the Organization Chart Wizard with new data
programming4us programming4us
 
Popular tags
 
Video Tutorail Microsoft Access Microsoft Excel Microsoft OneNote Microsoft PowerPoint Microsoft Project Microsoft Visio Microsoft Word Active Directory Biztalk Exchange Server Microsoft LynC Server Microsoft Dynamic Sharepoint Sql Server Windows Server 2008 Windows Server 2012 Windows 7 Windows 8 Adobe Indesign Adobe Flash Professional Dreamweaver Adobe Illustrator Adobe After Effects Adobe Photoshop Adobe Fireworks Adobe Flash Catalyst Corel Painter X CorelDRAW X5 CorelDraw 10 QuarkXPress 8 windows Phone 7 windows Phone 8 BlackBerry Android Ipad Iphone iOS