Colin Eberhardt's Adventures in .NET

WP7 PhoneGap Backbutton Support Re-visited

January 19th, 2012

About a month ago I published an article which demonstrated how to create a WP7 application using static HTML pages and PhoneGap. Whilst PhoneGap makes the packaging of HTML / JavaScript / CSS and images into a breeze, one thing it doesn’t do is provide correct back-button support. Correct back-button support is a mandatory requirement for marketplace certification. Hitting the back button should navigate back through the various screens of an application. Hitting the back-button on the first screen should terminate the application.

The solution I published previously handles the WP7 back keypress in order to keep track of the back-stack depth. When the back-stack depth is just one, a back-button press exits the application. This works fine if backwards navigation is always controlled via the hardware back-button, however if your application has HTML anchor elements that navigate back to a previous page, or you use code such as javascript:history.go(-1), this back-stack handling code will not detect that a backwards navigation has occurred.

As an aside, WP7 applications really should use the hardware back-button for navigation. If you are writing a cross-platform PhoneGap application consider adapting your UI for each platform. This means removing the back-button you woudl have in your iOS version when ‘skinning’ for WP7.

In order to solve this issue I have updated the code to inspect the URL that the browser control is navigating to in order to detect backwards navigation. The class now maintains a list of URLs, which represent the navigation stack. This allows it to detect a backwards navigation.

/// <summary>
/// Handles the back-button for a PhoneGap application. When the back-button
/// is pressed, the browser history is navigated. If no history is present,
/// the application will exit.
/// </summary>
public class BackButtonHandler
{
  private WebBrowser _browser;
 
  private List<string> _backStack = new List<string>();
 
  public BackButtonHandler(PhoneApplicationPage page, PGView phoneGapView)
  {
    // subscribe to the hardware back-button
    page.BackKeyPress += Page_BackKeyPress;
 
    _browser = phoneGapView.Browser;
 
    // handle navigation events
    _browser.Navigated += Browser_Navigated;
 
  }
 
  /// <summary>
  /// Handle navigation in order to update our back-stack
  /// </summary>
  private void Browser_Navigated(object sender, NavigationEventArgs e)
  {
    string url = _browser.Source.OriginalString;
 
    // ensure all slashes are the same
    // app\www/index.html
    // see: https://issues.apache.org/jira/browse/CB-184
    url = url.Replace("\\", "/");
 
    if (_backStack.Count < 2)
    {
      _backStack.Add(url);
    }
    else
    {
      // check whether the URL represents a backwards navigation
      string previousPage = _backStack[_backStack.Count - 2];
      if (previousPage == url)
      {
        _backStack.RemoveAt(_backStack.Count - 1);
      }
    }
  }
 
  /// <summary>
  /// Handle the hardware back-button
  /// </summary>
  private void Page_BackKeyPress(object sender, CancelEventArgs e)
  {
    // if we have items in the back-stack, route this event
    // to the browser
    if (_backStack.Count > 1)
    {
      _browser.InvokeScript("eval", "history.go(-1)");
      e.Cancel = true;
    }
  }
}

Note: There use of url.Replace("\\", "/"), this is due to a minor issue with the PhoneGap WP7 native code, which I have raised in the Callback JIRA (CB-184).

To use this code simply create an instance of the class, passing the PGView (the PhoneGap user control) into teh constructor:

public MainPage()
{
  InitializeComponent();
 
  new BackButtonHandler(this, PGView);
}

Because back-button handling is a mandatory requirement for WP7 applications, hopefully Nitobi will incorporate this code (or similar) into the PhoneGap Build service (which I tried earlier this week – it is pretty amazing!)

You can download a working example here: HTML5SandwichFlow.zip

(The first three recipe pages have ‘back’ anchor elements, two use the URL, one uses JavaScript)

Regards,
Colin E.

Windows Phone 7 – Browsing your Photos via Bing Maps

January 16th, 2012

The Windows Phone 7 camera gives you the option to record the location where a picture was taken (under Settings => applications => pictures+camera). With this feature turned on, each application has their latitude, longitude and altitude stored as part of the standard EXIF data. I thought it would be fun to combine the previous blog post I wrote on pushpin clustering with the photos on my camera, to allow me to explore them via a Bing Maps control. With not much more than 100 lines of code I came up with an application which I think is a lot of fun to use.

Here are all the photos on my phone, note the way the pushpins are clustered.

Here are a few pictures I took in New York, of the One World Trade Centre and the Stock Exchange.

Here are some pictures around Europe, including one of Gergely Orosz waiting for his turn in the Edinburgh Marathon Relay.

And finally, some pictures I took whilst running around Kielder Water during Kielder marathon.

Accessing the EXIF data

You can access the photos on a WP7 device via the XNA MediaLibrary class. The interface that this class provides gives you access to Picture instances which have properties that allow you to access the width / height and a few other basic attributes. They also have methods that return streams which can be used to read the thumbnail and image data, however, they do not expose the picture location. This is ‘hidden’ within the EXIF data.

Fortunately there is a C# implementation of an EXIF decoder available on codeproject, which, with a few tweaks by Tim Heuer works just fine within Silverlight for Windows Phone 7.

With this library, accessing the EXIF data is a one-liner:

JpegInfo info = ExifReader.ReadJpeg(picture.GetImage(), picture.Name);

The JpegInfo class exposes the raw EXIF geolocation data, which is detailed in the EXIF specification as being expressed as separate components of degrees, minutes and seconds together with a reference direction (North / South, East / West). We can convert from the sexagesimal numeric system used in EXIF, to the decimal system as follows:

private static double DecodeLatitude(JpegInfo info)
{
  double degrees = ToDegrees(info.GpsLatitude);
  return info.GpsLatitudeRef == ExifGpsLatitudeRef.North ? degrees : -degrees;
}
 
private static double DecodeLongitude(JpegInfo info)
{
  double degrees = ToDegrees(info.GpsLongitude);
  return info.GpsLongitudeRef == ExifGpsLongitudeRef.East ? degrees : -degrees;
}
 
public static double ToDegrees(double[] data)
{
  return data[0] + data[1] / 60.0 + data[2] / (60.0 * 60.0);
}

Analysing the images

When the application starts a BackgroundWorker is used to read the EXIF data for all of the pictures in the phone’s media library, with those that have geolocation data available being stored in a separate list:

BackgroundWorker bw = new BackgroundWorker();
bw.WorkerReportsProgress = true;
 
// analyse the pictures that reside in the Media Library in a background thread
bw.DoWork += (s, e) =>
{
  var ml = new MediaLibrary();
 
  using (var pics = ml.Pictures)
  {
    int total = pics.Count;
    int index = 0;
    foreach (Picture picture in pics)
    {
      // read the EXIF data for this image
      JpegInfo info = ExifReader.ReadJpeg(picture.GetImage(), picture.Name);
 
      // check if we have co-ordinates
      if (info.GpsLatitude.First() != 0.0)
      {
        _images.Add(new LocatedImage()
        {
          Picture = picture,
          Lat = DecodeLatitude(info),
          Long = DecodeLongitude(info)
        });
      }
 
      // report progress back to the UI thread
      string progress = string.Format("{0} / {1}", index, total);
      bw.ReportProgress((index * 100 / total), progress);
 
      index++;
    }
  }
};
 
// update progress on the UI thread
bw.ProgressChanged += (s, e) =>
  {
    string title = (string)e.UserState;
    ApplicationTitle.Text = title;
  };
 
bw.RunWorkerAsync();
 
// when analysis is complete, add the pushpins
bw.RunWorkerCompleted += (s, e) =>
  {
    ApplicationTitle.Text = "";
    AddPushpins();
  };

When the pictures have all been analysed, a pushpin is created for each image which is then added to the clusterer described in my previous blog post.

private void AddPushpins()
{
  List<Pushpin> pushPins = new List<Pushpin>();
 
  // create a pushpin for each picture
  foreach (var image in _images)
  {
    Location location = new Location()
    {
      Latitude = image.Lat,
      Longitude = image.Long
    };
 
    Pushpin myPushpin = new Pushpin()
    {
      Location = location,
      DataContext = image,
      Content = image,
      ContentTemplate = this.Resources["MarkerTemplate"] as DataTemplate
    };
 
    pushPins.Add(myPushpin);
  }
 
  // add them to the map via a clusterer
  var clusterer = new PushpinClusterer(map, pushPins, this.Resources["ClusterTemplate"] as DataTemplate);
}

The template used for the pushpins simply renders the image thumbnail:

<DataTemplate x:Key="MarkerTemplate">
  <Border BorderBrush="White" BorderThickness="1">
    <Image Source="{Binding Picture, Converter={StaticResource PictureThumbnailConverter}}"
            Width="80" Height="80"/>
  </Border>
</DataTemplate>

This makes use of a simple value converter which takes a Picture instance and converts it into a BitmapImage which is used as the Source for the image:

public class PictureThumbnailConverter : IValueConverter
{
  public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
  {
    Picture picture = value as Picture;
    BitmapImage src = new BitmapImage();
    src.SetSource(picture.GetThumbnail());
    return src;
  }
 
  public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
  {
    return null;
  }
}

The puhspin clusterer allows you to specify a separate template for clustered pushpins. The DataContext for this template is a list of the DataContexts of the clustered pins that it represents. For this application I created a template which renders what looks like a ‘stack’ of images. The number of pictures in the cluster is rendered as a TextBlock and the last image in the cluster rendered.

<DataTemplate x:Key="ClusterTemplate">
  <Grid Width="75" Height="75">
    <Canvas>
      <Border Style="{StaticResource FakePhoto}"
              Canvas.Left="0" Canvas.Top="0"/>
      <Border Style="{StaticResource FakePhoto}"
              Canvas.Left="5" Canvas.Top="5"/>
      <Border BorderBrush="White" BorderThickness="1"
              Canvas.Left="10" Canvas.Top="10"
              DataContext="{Binding Path=., Converter={StaticResource LastConverter}}">
        <Image Source="{Binding Picture, Converter={StaticResource PictureThumbnailConverter}}"
                Width="60" Height="60"/>
      </Border>
      <TextBlock Text="{Binding Count}"
                  Opacity="0.5"
                  Canvas.Left="25" Canvas.Top="15"
                  FontSize="35"/>
    </Canvas>
  </Grid>      
</DataTemplate>
 
<Style TargetType="Border" x:Key="FakePhoto">
  <Setter Property="Width" Value="60"/>
  <Setter Property="Height" Value="60"/>
  <Setter Property="BorderBrush" Value="White"/>
  <Setter Property="Background" Value="Black"/>
  <Setter Property="BorderThickness" Value="1"/>
</Style>

The code that renders the last image is a bit cunning, it uses a value converter that performs a Linq style ‘last’ operations, extracting the last items from a collection of objects:

public class LastConverter : IValueConverter
{
  public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
  {
    IList enumerable = value as IList;
    return enumerable.Cast<object>().Last();
  }
 
  public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
  {
    return null;
  }
}

This feels quite neat to me :-)

The clustered pins look like the following, which is a cluster of 5 images around Paris, with the stunning La Grande Arche de la Défense as the image at the top of the cluster:

Despite its simplicity, I have had a lot of fun playing with this application. It has certainly encouraged me to take as many photos as possible whenever I go travelling.

You can download the full sourcecode here: PhotoBrowser.zip

Regards, Colin E.

Proud to be a CodeProject MVP 2012

January 9th, 2012

I have just received an email from Chris Maunder, co-founder of CodeProject, informing me that I have been awarded CodeProject MVP status for 2012. I am very pleased to have received this award, which is given to a small handful of individuals each year.

What I like about CodeProject is the fantastic quality of the articles submitted by the thousands of authors that participate in this site, with the only payment being their own please in the writing.

I have not written that many articles throughout the year, just four of them. However, I have put a lot of effort into each one of them – and thankfully they have been very well received. It is great that CodeProject favours quality over quantity. My articles for 2011 were:

I do also circulate some of my blog posts via the CodeProject Associate program..

On that note, it is almost three months since I last wrote an article … fortunately I do have an idea for the next article, so I had better get on with writting it …

Regards, Colin E.

FastClick for WP7 – Improving Browser Responsiveness for PhoneGap Apps

January 5th, 2012

I recently released an update of the HTML5 / PhoneGap application I wrote a few months ago to the marketplace. This update fixed the location detection issue, where the marketplace certification process failed to detect that the application was using geolocation data via the browser (this issue has been fixed in PhoneGap 1.2), plus a few cosmetic improvements.

This application still exhibits a few UI quirks that make it fairly obvious that this is not a native application, the first is the gray box that appears whenever a link is clicked, the second is the overall responsiveness – there is a short delay between clicking on links / buttons and the application responding. It is this responsiveness that I would like to tackle in this blog post. This delay unfortunately gives the impression that your application is less responsive than it actually is!

So where does this delay come from?

Despite the rise in popularity of touch interfaces, much of the web is still designed for interaction with a mouse and keyboard, with the DOM exposing events for mouse-up, mouse-down and click user interactions. The Windows Phone 7 browser control captures touch interactions and translates these into the required mouse events, firing them on the DOM element being interacted with. However, the browser itself also needs to support gestures such as pan, pinch-zoom and double-tap zoom.

If we consider for example the pinch-zoom feature, where the user places two fingers on the screen and moves them apart in order to zoom in. It is highly unlikely they will place both fingers on the screen simultaneously. If, when the first finger was placed, a mouse-down event was fired on the relevant DOM element, there would be no way to ‘cancel’ this event when the user’s second finger made contact with the screen shortly afterwards. For this reason the WP7 mobile browser does not properly support mouse-down / mouse-up events. You will find that mousedown, mouseup then click are always fired immediately in that order regardless of how long you hold your finger on the screen for. Clearly the need to add gesture support to the browser interferes with the way in which native touch events are translated into JavaScript mouse events.

Now consider the double-tap zoom feature, where the user taps twice in quick succession to zoom the web page by some fixed amount. If the DOM click event were fired on the first tap, this would cause a navigation to occur if an anchor element were tapped. This is certainly not the desired behaviour! Again, there is no way to ‘cancel’ an event once it is fired, so the only solution to this problem is, when the user first taps the screen, pause for a few hundred milliseconds to see if they tap a second time. Only after this pause is it safe to pass the interaction onto the DOM, firing a click event on the relevant DOM element.

Stock image courtesy of lovetheson

The need to handle gestures in the native layer causes numerous issues for the browser event handling. Thankfully these issues are relatively minor for most websites, but if your aim is to create a HTML-based application with a native look and feel, these issues become quite major.

Improving responsiveness

The problem I want to tackle in this blog post is that of the click DOM event firing delay to support the double-tap gesture. With HTML5-based applications you will typically want to disable pinch zoom, double-tap zoom and (optionally) scrolling as I described in a previous blog post. As a result your application has no need for the double-tap and pinch gestures and we are safe to short-circuit this behaviour, immediately raising DOM click events when the user taps the screen.

My solution follows a similar approach to the utility class I wrote for suppressing pan / zoom, where event handlers are added to the Border that sits within the WebBrowser control visual tree:

\-WebBrowser
  \-Border
    \-Border
      \-PanZoomContainer
        \-Grid
          \-Border (*)
            \-ContentPresenter
              \-TileHost

In order to forward Tap gestures to the browser, we can add event handlers to the Border element, then use InvokeScript to pass this onto the DOM rendered within the TileHost.

The JavaScript code to achieve this is very simple, the DOM exposes an elementFromPoint function which allows you to hit test the DOM. The IE9 DOM interface also exposes a non-standard click function which invokes any click event handlers, this will cause an anchor element to navigate. Putting it together, if you have an anchor element at (x=100, y=100), you can cause it to navigate using the following simple code:

document.elementFromPoint(100, 100).click()

Calling this JavaScript from our native C# code in response to Tap events is very easy. I have wrapped it up into a simple utility class:

/// <summary>
/// Captures mouse left button up events, triggering an immediate
/// click event on the clicked DOM element.
/// </summary>
public class BrowserFastClick
{
  private WebBrowser _browser;
 
  private Border _border;
 
  public BrowserFastClick(WebBrowser browser)
  {
    _browser = browser;
    browser.Loaded += new RoutedEventHandler(Browser_Loaded);
  }
 
  private void Browser_Loaded(object sender, RoutedEventArgs e)
  {
    // locate the element used to capture mouse events
    _border = _browser.Descendants<Border>().Last() as Border;
    _border.Tap += Border_Tap;
  }
 
  private void Border_Tap(object sender, GestureEventArgs e)
  {
    //determine the click location
    var pos = e.GetPosition(_border);
 
    // forward to the browser
    _browser.InvokeScript("eval",
      string.Format("document.elementFromPoint({0},{1}).click()", pos.X, pos.Y));      
  }
}

To use it within a PhoneGap application, simply create an instance as follows:

new BrowserFastClick(PGView.Browser);

The net result is that your application becomes more responsive, with the browser immediately navigating links when they are first tapped.

I have updated the HTML5 SandwichFlow application I blogged about a few weeks ago to use this FastClick code. You can download the complete example here: HTML5SandwichFlow.zip

Regards, Colin E.