Colin Eberhardt's Technology Adventures

Enhanced Windows Phone 8 Map Gestures

April 15th, 2013

This blog post describes the addition of a two-finger rotation and three-finger pitch gesture to the Windows Phone 8 Map control.

You can see these gesture in action below:

The WP8 release replaced the image-tile based Bing maps with a fully vector-rendered map from Nokia. Being vector-based, this map can be panned, zoomed, rotated and rendered at an angle (i.e. pitched). However, much of this new functionality is not offered to the end user!

The WP8 supports the same gestures that the Bing WP7 map did, i.e. a single-fingered pan gesture and two-fingered pinch to zoom. What about rotation and pitch? Why not allow the user to modify these via gestures?

The key here is to add some new gestures that complement the existing one. I opted for the following:

  • Two-finger rotate – When the user places two fingers on the map this is currently used to zoom the display via a ‘spreading’ motion. However, if the user instead rotates the two touch points around the centre, the map should be rotated.
  • Three-finger pitch – When the user places three fingers on the map, if they drag up or down, the map should adjust its pitch accordingly.

These gestures are enabled simply by creating them with a reference to the map:

new MapRotationGesture(map);
new MapPitchGesture(map);

If you don’t care how this all works, just head over to github and grab the code. If you want to find out more, read on …

Suppressing The Existing Gestures

In order to add these new gestures to the map, there needs to be a mechanism in place to suppress the existing gestures so that they do not interfere.

The technique I used is similar to a technique I demonstrated previously for suppressing pinch and scroll in the Windows Phone Browser control. Both the Map and WebBrowser controls have a visual tree containing a number of user interface elements. The inner structure of the Map is shown below:

Microsoft.Phone.Maps.Controls.Map
  System.Windows.Controls.Border
    System.Windows.Controls.Border
      Microsoft.Phone.Maps.Controls.MapPresentationContainer
        MS.Internal.ExternalInputContainer
          System.Windows.Controls.Grid
            MS.Internal.TileHostV2
            Microsoft.Phone.Maps.Controls.RootMapLayer

The technique for suppressing interactions is quite simple, just add a ManipulationDelta event handler to each one of these elements, setting the event to handled. The complete code is show below:

/// <summary>
/// A base class for map gestures, which allows them to suppress the built-in map gestures.
/// </summary>
public class MapGestureBase
{
  /// <summary>
  /// Gets or sets whether to suppress the existing gestures/
  /// </summary>
  public bool SuppressMapGestures { get; set; }
 
  protected Map Map { get; private set; }
 
  public MapGestureBase(Map map)
  {
    Map = map;
    map.Loaded += (s,e) => CrawlTree(Map);
  }
 
  private void CrawlTree(FrameworkElement el)
  {
    el.ManipulationDelta += MapElement_ManipulationDelta;
    for (int c = 0; c < VisualTreeHelper.GetChildrenCount(el); c++)
    {
      CrawlTree(VisualTreeHelper.GetChild(el, c) as FrameworkElement);
    }
  }
 
  private void MapElement_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
  {
    if (SuppressMapGestures)
      e.Handled = true;
  }
}

If you create an instance of this class and associate it with a map you can turn gestures on and off via the SuppressMapGestures property:

var gestureBase = new MapGestureBase(map);
gestureBase.SuppressMapGestures = true;

NOTE: Unfortunately this doesn’t solve the ‘map in a pivot’ or ‘map in a panorama’ problem that many Windows Phone developers have struggled with – the gestures that are handled are not propagated to a parent control.

A Rotation Gesture

The user can rotate the map by placing two fingers on the screen then rotating them around their central point. Because two fingers are also used for the pinch-to-zoom gesture, a suitable threshold needs to be introduced. I have found that disabling rotation until the user has rotated by 10 degrees feels about right.

RotationGesture

The code that implements the rotation is really quite simple, the MapGestureBase subclass is shown in its entirety below:

/// <summary>
/// Adds a two-finger rotation gesture to a Map control.
/// </summary>
public class MapRotationGesture : MapGestureBase
{
  /// <summary>
  /// Gets or sets the minimuum rotation that the user must apply in order to initiate this gesture.
  /// </summary>
  public double MinimumRotation { get; set; }
 
  private double? _previousAngle;
 
  private bool _isRotating;
 
  public MapRotationGesture(Map map)
    : base(map)
  {
    MinimumRotation = 10.0;
    Touch.FrameReported += Touch_FrameReported;
  }
 
 
  private void Touch_FrameReported(object sender, TouchFrameEventArgs e)
  {
    var touchPoints = e.GetTouchPoints(Map);
 
    if (touchPoints.Count == 2)
    {
      // for the initial touch, record the angle between the fingers
      if (!_previousAngle.HasValue)
      {
        _previousAngle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
      }
 
      // should we rotate?
      if (!_isRotating)
      {
        double angle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
        double delta = angle - _previousAngle.Value;
        if (Math.Abs(delta) > MinimumRotation)
        {
          _isRotating = true;
          SuppressMapGestures = true;
        }
      }
 
      // rotate me
      if (_isRotating)
      {
        double angle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
        double delta = angle - _previousAngle.Value;
        Map.Heading -= delta;
        _previousAngle = angle;
      }
    }
    else
    {
      _previousAngle = null;
      _isRotating = false;
      SuppressMapGestures = false;
    }
  }
 
  private double AngleBetweenPoints(TouchPoint p1, TouchPoint p2)
  {
    return Math.Atan2(p1.Position.Y - p2.Position.Y, p1.Position.X - p2.Position.X)
            *(180 / Math.PI);
  }
}

Touch gestures are detected via the Touch.FrameReported event. When two fingers are placed on the screen the initial rotation angle is recorded. When the minimum rotation is exceeded, the Map.Heading is updated with each ‘delta’ reported. Really simple code, but a fantastic feature for the user!

A Pitch Gesture

You get a real feel for the vector-nature of the maps when you set the ‘pitch’, a style of rendering that is often used on satnavs.

PitchGesture

I initially considered using a two finger pull-down gesture, which is similar to the one which Google Maps on Android uses, but found it very hard to coordinate the three gestures, zoom, rotate, pitch, which all use the same two-fingers! So instead, I opted for a three-finger pull-down gesture to increase the pitch of the map.

The code follows a very similar pattern to the rotate gesture:

/// <summary>
/// Adds a three-finger pitch gesture to a Map control.
/// </summary>
public class MapPitchGesture : MapGestureBase
{
  /// <summary>
  /// Gets or sets the sensitivity of this gesture
  /// </summary>
  public double Sensitivity { get; set; }
 
  private double? _initialPitchYLocation;
 
  public MapPitchGesture(Map map)
    : base(map)
  {
    Sensitivity = 0.5;
    Touch.FrameReported += Touch_FrameReported;
  }
 
  private void Touch_FrameReported(object sender, TouchFrameEventArgs e)
  {
    var touchPoints = e.GetTouchPoints(Map);
 
    SuppressMapGestures = touchPoints.Count == 3;
 
    if (touchPoints.Count == 3)
    {
      if (!_initialPitchYLocation.HasValue)
      {
        _initialPitchYLocation = touchPoints[0].Position.Y;
      }
 
      double delta = touchPoints[0].Position.Y - _initialPitchYLocation.Value;
      double newPitch = Math.Max(0, Math.Min(75, (Map.Pitch + delta * Sensitivity)));
      Map.Pitch = newPitch;
      _initialPitchYLocation = touchPoints[0].Position.Y;
    }
    else
    {
      _initialPitchYLocation = null;
    }
  }
}

As soon as three fingers are placed on the screen, the gesture becomes active. The movement of the first finger is used to determine the delta to apply to the Pitch property. Again, nice and simple!

The sourcecode for these gesture, plus a demo app is available via github. Please let me know if you use this code in any of your apps.

Regards, Colin E.

Tags: ,
Posted in codeproject | 7 Comments »

A gesture-driven Windows Phone to-do application Part Two – drag re-ordering

June 27th, 2012

A couple of weeks ago I blogged about a todo list application which uses gestures to achieve its basic functions, a left swipe deletes an item, while a right-swipe marks it as complete. In this blog post I am adding re-ordering which is initiated via a tap-and-hold gesture and performed via a drag.

As mentioned previously, this application is heavily inspired by the iPhone application Clear, who I give full credit for coming up with such an innovative user-interface design. This blog post is for fun an education, you are strictly prohibited from using this as the basis of a ‘Clear’ clone for WP7.

The video below shows the application so far, with the new re-order functionality:

Initiating a Drag

In the previous blog post we saw how the Toolkit GestureListener can be used to convert the low-level manipulation events into high level gestures. In order to initiate an item drag we could use the Hold event that the GestureListener exposes. The Hold event is fired when the user taps and holds the location of their finger for a few seconds. When I published my previous blogpost one of my readers Simon (Darkside) Jackson kindly pointed out that some of the gesture events are now supported by FrameworkElement directly, Hold is one such event. For this reason I’ll use FrameworkElement.Hold, the equivalent event on GestureListener should be considered deprecated.

The easiest way to allow the user to ‘pick up’ and drag the item is to clone it using a WriteableBitmap, hiding the original. This technique allows us to place the item at a higher Z-index than the list which it comes from, so that it will always be top-most as it is moved up and down the list.

We’ll add the element that is used to render the dragged item to the XAML:

<Grid>
  <ItemsControl ItemsSource="{Binding}" x:Name="todoList">
    ... markup from previous blog post
  </ItemsControl>
 
  <Grid x:Name="dragImageContainer"
        VerticalAlignment="Top"
        Visibility="Collapsed">
    <!-- the image that displays the dragged item -->
    <Image x:Name="dragImage"
          VerticalAlignment="Top">
    </Image>
 
    <!-- lower drop shadow -->
    <Rectangle Height="10"
                VerticalAlignment="Bottom">
      <Rectangle.Fill>
        <LinearGradientBrush EndPoint="0,1" StartPoint="0,0">
          <GradientStop Color="#AA000000"/>
          <GradientStop Color="#00000000" Offset="1"/>
        </LinearGradientBrush>
      </Rectangle.Fill>
      <Rectangle.RenderTransform>
        <TranslateTransform Y="10"/>
      </Rectangle.RenderTransform>
    </Rectangle>
 
    <!-- upper drop shadow -->
    <Rectangle Height="10"
               VerticalAlignment="Top">
      <Rectangle.Fill>
        <LinearGradientBrush EndPoint="0,1" StartPoint="0,0">
          <GradientStop Color="#00000000"/>
          <GradientStop Color="#AA000000" Offset="1"/>
        </LinearGradientBrush>
      </Rectangle.Fill>
      <Rectangle.RenderTransform>
        <TranslateTransform Y="-10"/>
      </Rectangle.RenderTransform>
    </Rectangle>
  </Grid>
</Grid>

The markup contains an image, and a couple of Rectangle elements, each of which is filled with a subtle opacity gradient and offset in order to give the impression of a drop shadow. The Source of the image is set in the Hold event handler as shown:

private void GestureListener_Hold(object sender, Microsoft.Phone.Controls.GestureEventArgs e)
{
  _dragReOrder = true;
 
  // copy the dragged item to our 'dragImage' 
  FrameworkElement draggedItem = sender as FrameworkElement;
  var bitmap = new WriteableBitmap(draggedItem, null);
  dragImage.Source = bitmap;
  dragImageContainer.Visibility = Visibility.Visible;
  dragImageContainer.Opacity = 1.0;\
  dragImageContainer.SetVerticalOffset(draggedItem.GetRelativePosition(todoList).Y);
 
  // hide the real item
  draggedItem.Opacity = 0.0;
 
  // fade out the list
  todoList.Animate(1.0, 0.7, FrameworkElement.OpacityProperty, 300, 0);
 
  _initialDragIndex = _todoItems.IndexOf(((ToDoItem)draggedItem.DataContext));
}

The code above also hides the real object and offsets our ‘fake’ vertically so that it occupies the same position as the original. The list is also faded slightly to give a visual indication that the dragged item is now above the rest of the list:

Dragging the item

In order to allow the user to support dragging of the item we also need to handle ManipulationDelta on the Border element, as shown below:

<ItemsControl.ItemTemplate>
  <DataTemplate>
    <Border Background="{Binding Path=Color, Converter={StaticResource ColorToBrushConverter}}"
            ManipulationDelta="Border_ManipulationDelta"
            ManipulationCompleted="Border_ManipulationCompleted"
            Hold="Border_Hold"
            Canvas.ZIndex="0">
      <!-- gestures that were added in the last blog post to support delete / complete -->
      <toolkit:GestureService.GestureListener>
        <toolkit:GestureListener
                  DragStarted="GestureListener_DragStarted"
                  DragDelta="GestureListener_DragDelta"
                  DragCompleted="GestureListener_DragCompleted"
                  GestureCompleted="GestureListener_GestureCompleted"
                  Flick="GestureListener_Flick"/>
      </toolkit:GestureService.GestureListener>
 
      <Grid>
        <!-- the todo item XAML from the previous blog post -->        
      </Grid>
    </Border>
  </DataTemplate>
</ItemsControl.ItemTemplate>

So why am I handling these events on the Border rather than re-using the DragDelta / GestureComplete on the GestureListener? There are a couple of reasons:

  1. The GestureListener ‘drag’ has a tolerance (as discussed in the previous blog post). The user has to move their finger beyond a certain distance before a drag is started. In the current context we want a drag to occur as soon as the Hold event fires.
  2. When handling the element ‘drag’ we need to suppress the event by setting e.Handled=true, otherwise a drag will bubble up to the ScrollViewer that hosts our elements causing it to scroll. I found (through trial and error) that the GestureListener.DragDelta.Handled property doesn’t appear to have any effect.

The handler for the drag event is pretty simple, moving the copy of our item by the required distance:

private void Border_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
{
  Debug.WriteLine("ManipulationDelta");
 
  if (!_dragReOrder)
    return;
 
  // set the event to handled in order to avoid scrolling the ScrollViewer
  e.Handled = true;
 
  // move our 'drag image'.
  dragImageContainer.SetVerticalOffset(dragImageContainer.GetVerticalOffset().Value + e.DeltaManipulation.Translation.Y);
 
  ShuffleItemsOnDrag();
}

Offsetting the items underneath

The ShuffleItemsOnDrag method is where the fun starts, we’ll get to that shortly. First we’ll take a look at a simple utility method that is used to determine the index that the item being dragged would occupy if it were dropped at the present location. This is achieved by a simple measurement:

// Determines the index that the dragged item would occupy when dropped
private int GetDragIndex()
{
  double dragLocation = dragImageContainer.GetRelativePosition(todoList).Y +
                          VerticalScrollViewer.VerticalOffset +
                          dragImage.ActualHeight / 2;
  int dragIndex = (int)(dragLocation / dragImage.ActualHeight);
  return dragIndex;
}
 
private ScrollViewer _scrollViewer;
 
 
// gets the scrollviewer from the ItemsControl template
private ScrollViewer VerticalScrollViewer
{
  get
  {
    if (_scrollViewer == null)
    {
      _scrollViewer = todoList.Descendants<ScrollViewer>()
                              .Cast<ScrollViewer>()
                              .Single();
    }
    return _scrollViewer;
  }
}

The above code needs to take the current scroll location into consideration, which is why the ScrollViewer property above uses Linq-to-VisualTree to find the ScrollViewer that the ItemsControl generates to hosts our elements.

ShuffleItemsOnDrag is where the fun begins, we want to create an effect where the dragged item ‘pushes’ the other items out of the way as it hovers over them, giving the impression that the list is re-ordering as we drag.

The method below iterates over all of the items in the list to determine whether they need to be offset. An item needs to be offset if it is between the current dragged item index and the items original location.

private void ShuffleItemsOnDrag()
{
  // find its current index
  int dragIndex = GetDragIndex();
 
  // iterate over the items in the list and offset as required
  double offset = dragImage.ActualHeight;
  for (int i = 0; i < _todoItems.Count; i++)
  {
    FrameworkElement item = todoList.ItemContainerGenerator.ContainerFromIndex(i) as FrameworkElement;
 
    // determine which direction to offset this item by
    if (i <= dragIndex && i > _initialDragIndex)
    {
      OffsetItem(-offset, item);
    }
    else if (i >= dragIndex && i < _initialDragIndex)
    {
      OffsetItem(offset, item);
    }
    else
    {
      OffsetItem(0, item);
    }
  }
}

The OffsetItem method performs the actual offset by animating the Y position of each item. The target location is stored in the elements Tag property so that we don’t repeatedly fire the same animation on an element.

private void OffsetItem(double offset, FrameworkElement item)
{
  double targetLocation = item.Tag != null ? (double)item.Tag : 0;
  if (targetLocation != offset)
  {
    var trans = item.GetVerticalOffset().Transform;
    trans.Animate(null, offset, TranslateTransform.YProperty, 500, 0);
    item.Tag = offset;
    _moveSound.Play();
  }
}

Completing the drag

When the user stops dragging the item, the ManipulationCompleted event is fired. Here we perform a number of tasks:

  1. Fade the list back to full opacity
  2. Animate the dragged item so that it ‘snaps’ into location
  3. When the above is complete, we need to re-order the underlying collection of model items, then re-populate the ObservableCollection exposed to the view. This causes all the items to be re-rendered, removing all of the TranslateTransforms that have been applied.
  4. Finally, remove the image which is our copy of the dragged item.

This sounds like a lot of work, but our Animate utility method makes it quite simple:

private void Border_ManipulationCompleted(object sender, ManipulationCompletedEventArgs e)
{
  if (!_dragReOrder)
    return;
 
  _dragReOrder = false;
  _autoScrollTimer.Stop();
 
  int dragIndex = GetDragIndex();
 
  // fade in the list
  todoList.Animate(null, 1.0, FrameworkElement.OpacityProperty, 200, 0);
 
  // animated the dragged item into location
  double targetLocation = dragIndex * dragImage.ActualHeight - VerticalScrollViewer.VerticalOffset;
  var trans = dragImageContainer.GetVerticalOffset().Transform;
  trans.Animate(null, targetLocation, TranslateTransform.YProperty, 200, 0, null,
    () =>
    {
      // clone the list and move the dragged item
      var items = _todoItems.ToList();
      var draggedItem = items[_initialDragIndex];
      items.Remove(draggedItem);
      items.Insert(dragIndex, draggedItem);
 
      // re-populate our ObservableCollection
      _todoItems.Clear();
      _todoItems.AddRange(items);
      UpdateToDoColors();
 
      // fade out the dragged image and collapse on completion
      dragImageContainer.Animate(null, 0.0, FrameworkElement.OpacityProperty, 1000, 0, null, ()
        => dragImageContainer.Visibility = Visibility.Collapsed);
    });
 
}

Scrolling the list

The current implementation only allows the user to drag the item within the bounds of the current screen. What if the list is larger than the screen and the users want to drag right from the bottom to the top?

A common solution to this problem is to auto-scroll the list if the item is dragged near to the top. The following method is invoked periodically by a timer to see whether the item has been dragged within the top or bottom ‘scroll zones’. The velocity of the scroll is proportional to just how far within these zones the item has been dragged. Scrolling is simply a matter of setting the scroll location on the ScrollViewer we located earlier:

// checks the current location of the item being dragged, and scrolls if it is
// close to the top or the bottom
private void AutoScrollList()
{
  // where is the dragged item relative to the list bounds?
  double draglocation = dragImage.GetRelativePosition(todoList).Y + dragImage.ActualHeight / 2;
 
  if (draglocation < AutoScrollHitRegionSize)
  {
    // if close to the top, scroll up
    double velocity = (AutoScrollHitRegionSize - draglocation);
    VerticalScrollViewer.ScrollToVerticalOffset(VerticalScrollViewer.VerticalOffset - velocity);
  }
  else if (draglocation > todoList.ActualHeight - AutoScrollHitRegionSize)
  {
    // if close to the bottom, scroll down
    double velocity = (AutoScrollHitRegionSize - (todoList.ActualHeight - draglocation));
    VerticalScrollViewer.ScrollToVerticalOffset(VerticalScrollViewer.VerticalOffset + velocity);
  }
}

You can see the scroll zones illustrated below:

And finally, we are all done! With our todo application you can use flick and drag gestures to mark items as complete or delete them, and now use hold and drag to re-order. I think it’s about time I made it so that you can add or edit items. We’ll get to that next time!

You can download the sourcecode here: ClearStyle.zip

Regards, Colin E.

Tags: ,
Posted in Uncategorized | 8 Comments »

A gesture-driven Windows Phone to-do application

June 6th, 2012

This blog post describes the implementation of a gesture-based todo-list application. The simple interface is controlled entirely by drag, flick and swipe:

So far the application supports deletion and completion of tasks, but not the addition of new ones. I’ll get to this in a later blog post!

Introduction – gestures, why don’t we use them more?

I think it is fair to say that most Windows Phone applications (and mobile applications in general) have user-interfaces that are a close reflection of how we interact with a desktop computer. Mobile applications have the same buttons, checkboxes and input controls as their desktop equivalent, with the user interacting with the majority of these controls via simple clicks / taps.

The mobile multi-touch interface allows for much more control and expression than a simple mouse pointer device. Standard gestures have been developed such as pinch/stretch, flick, pan, tap-and-hold, however these are quite rarely used; one notable exception being pinch/stretch which is the standard mechanism for manipulating images.
When an application comes along that makes great use of gestures, it really stands out from the crowds. One such application is the iPhone ‘Clear’ application, a simple todo-list with not one button or checkbox in sight. You can see the app in action below:

Interestingly, its use of pinch to navigate the three levels of menu is similar to the Windows 8 concept of ‘semantic zoom’.
When I first saw Clear – the clean, clutter-free interface immediately spoke ‘Metro’ to me! This blog post looks at how to recreate some of the features of Clear using Silverlight for Windows Phone, in order to create a gesture-driven todo application.

NOTE: All of the work on my blog is under a Creative Commons Share-alike licence. For this blog post I just want to add that I do not want someone to take this code in order to release a ‘Clear’ clone on the Windows Phone marketplace. This blog post is for fun and education, to make people think more about the possibilities of gestures.

Rendering the to-do items

I haven’t followed the MVVM pattern for this simple application (I don’t need to unit test or collaborate with a designer!), but have followed the standard approach of using databinding. Each item is represented by an instance of the ToDoItem class which has properties of Text, Completed and Color and implemented INotifyPropertyChanged. A collection of these items is bound to an ItemsControl in order to render the list:

<ItemsControl ItemsSource="{Binding}" x:Name="todoList">
  <ItemsControl.ItemTemplate>
    <DataTemplate>
      <Border Background="{Binding Path=Color, Converter={StaticResource ColorToBrushConverter}}">
        <Grid>
          <Grid.Background>
            <!-- create a subtle gradient that overlays the background -->
            <LinearGradientBrush EndPoint="0,1" StartPoint="0,0">
              <GradientStop Color="#22FFFFFF"/>
              <GradientStop Color="#00000000" Offset="0.1"/>
              <GradientStop Color="#00000000" Offset="0.7"/>
              <GradientStop Color="#22000000" Offset="1"/>
            </LinearGradientBrush>
          </Grid.Background>
 
          <!-- task text -->
          <TextBlock Text="{Binding Text}" Margin="15,15,0,15" FontSize="30"/>
        </Grid>
      </Border>
    </DataTemplate>
  </ItemsControl.ItemTemplate>
 
  <ItemsControl.Template>
    <ControlTemplate TargetType="ItemsControl">
      <ScrollViewer>
        <ItemsPresenter/>
      </ScrollViewer>
    </ControlTemplate>
  </ItemsControl.Template>
</ItemsControl>

The items control is template to host the ItemsPanel within a ScrollViewer, the default panel, a vertically-oriented StackPanel, is used here. The template to render each item is a simple Border, Grid and TextBlock. The code-behind iterates over the ToDoItem instances setting their Color property to produce a pretty looking gradient from red to orange, representing the priority of each item:

Handling gestures

Silverlight for Windows Phone provides manipulation events which you can handle in order to track when a user places one or more fingers on the screen and moves them around. Turning low-level manipulation events into high-level gestures is actually quite tricky. Touch devices give a much greater control when dragging objects, or flicking them, but have a much lower accuracy for the more commonplace task of trying to hit a specific spot on the screen. For this reason, gestures have a built in tolerance. As an example, a drag manipulation gesture is not initiated if the user’s finger moves by a single pixel.

Fortunately, the Silverlight Toolkit contains a GestureListener which handles manipulation events and turns them into the standard gesture events for you. Unless you need a quite fancy gesture (two-finger-swipe for example), the GestureListener probably gives you all you need. To use this class, simply attach it to the element that you want to handle gestures on, then add handlers to the events that the GestureListener provides.

We’ll add a GestureListener to the template that is used to render each ToDoItem:

<ItemsControl ItemsSource="{Binding}" x:Name="todoList">
  <ItemsControl.ItemTemplate>
    <DataTemplate>
      <Border Background="{Binding Path=Color, Converter={StaticResource ColorToBrushConverter}}">
        <!-- handle gestures for the ToDoItem element -->
        <toolkit:GestureService.GestureListener>
          <toolkit:GestureListener
                    DragStarted="GestureListener_DragStarted"
                    DragDelta="GestureListener_DragDelta"
                    DragCompleted="GestureListener_DragCompleted"/>
        </toolkit:GestureService.GestureListener>
 
        <Grid>
          ...
 
          <!-- task text -->
          <TextBlock Text="{Binding Text}" Margin="15,15,0,15" FontSize="30"/>
 
        </Grid>
      </Border>
    </DataTemplate>
  </ItemsControl.ItemTemplate>
 
  ...
</ItemsControl>

The events produced by the GestureListener are used to indicate that a user has dragged their finger across the screen, but they do not actually move the element that the listener is associated with. We need to handle these events and move the element ourselves:

private void GestureListener_DragStarted(object sender, DragStartedGestureEventArgs e)
{
  // initialize the drag
  FrameworkElement fe = sender as FrameworkElement;
  fe.SetHorizontalOffset(0);  
}
 
private void GestureListener_DragDelta(object sender, DragDeltaGestureEventArgs e)
{
  // handle the drag to offset the element
  FrameworkElement fe = sender as FrameworkElement;
  double offset = fe.GetHorizontalOffset().Value + e.HorizontalChange;
  fe.SetHorizontalOffset(offset);
}
 
private void GestureListener_DragCompleted(object sender, DragCompletedGestureEventArgs e)
{
  ToDoItemBounceBack(fe);
}

But what are these mysterious methods, SetHorizontalOffset and GetHorizontalOffset in the above code? They are not found on FrameworkElement. The task of offsetting an element is reasonably straightforward via a TranslateTransform. However, once you set the RenderTransform of an element, it is converted to a MatrixTransform, so you cannot easily retrieve the value of the offset that has just been applied.

In order to hide this slightly messy implementation, I have created a couple of extension methods that use FrameworkElement.Tag to store the current offset:

public static void SetHorizontalOffset(this FrameworkElement fe, double offset)
{
  var trans = new TranslateTransform()
  {
    X = offset
  };
  fe.RenderTransform = trans;
 
  fe.Tag = new Offset()
    {
      Value = offset,
      Transform  = trans
    };
}
 
public static Offset GetHorizontalOffset(this FrameworkElement fe)
{
  return fe.Tag == null ? new Offset() : (Offset)fe.Tag;
}
 
public struct Offset
{
  public double Value { get; set; }
  public TranslateTransform Transform { get; set; }
}

Yes, I know, Tag is never an elegant solution, but at least the above code is hidden behind extension methods, so feel free to replace with a more elegant implementation using attached properties if you wish!

When the drag stops we want the item to bounce back into place:

private void ToDoItemBounceBack(FrameworkElement fe)
{
  var trans = fe.GetHorizontalOffset().Transform;
 
  trans.Animate(trans.X, 0, TranslateTransform.XProperty, 300, 0, new BounceEase()
  {
    Bounciness = 5,
    Bounces = 2
  });
}

Animate is another extension method which I created in order to quickly create DoubleAnimations for the properties of an element:

public static void Animate(this DependencyObject target, double from, double to,
                          object propertyPath, int duration, int startTime,
                          IEasingFunction easing = null, Action completed = null)
{
  if (easing == null)
  {
    easing = new SineEase();
  }
 
  var db = new DoubleAnimation();
  db.To = to;
  db.From = from;
  db.EasingFunction = easing;
  db.Duration = TimeSpan.FromMilliseconds(duration);
  Storyboard.SetTarget(db, target);
  Storyboard.SetTargetProperty(db, new PropertyPath(propertyPath));
 
  var sb = new Storyboard();
  sb.BeginTime = TimeSpan.FromMilliseconds(startTime);
 
  if (completed != null)
  {
    sb.Completed += (s, e) => completed();
  }
 
  sb.Children.Add(db);
  sb.Begin();
}

With the above code in place, we can drag items to one side or the other, then release and watch them bounce back into place.

Mark as complete

When an item is dragged sufficiently far to the right, we’d like to have it marked as complete. To implement this, we’ll check how far an item was dragged when the user releases their finger. If it was dragged more than half way, we’ll mark the item as complete:

private void GestureListener_DragCompleted(object sender, DragCompletedGestureEventArgs e)
{  
  FrameworkElement fe = sender as FrameworkElement;
  if (e.HorizontalChange > fe.ActualWidth / 2)
  {
    ToDoItemCompletedAction(fe);
  }
  else
  {
    ToDoItemBounceBack(fe);
  }
}
 
private void ToDoItemCompletedAction(FrameworkElement fe)
{
  // set the ToDoItem to complete
  ToDoItem completedItem = fe.DataContext as ToDoItem;
  completedItem.Completed = true;
  completedItem.Color = Colors.Green;
 
  // bounce back into place
  ToDoItemBounceBack(fe);
}

The bindings take care of updating the UI so that our item is now green. I also added a Line element which has its Visibility bound to the Completed property of the ToDoItem:

Deleting an item

If instead the use slides the item to the left we’d like to delete it. The DragCompleted event handler can easily be extended to check whether the item was dragged more than half-way across the screen in the other direction. The method that performs the deletion is shown below:

private void ToDoItemDeletedAction(FrameworkElement deletedElement)
{
  var trans = deletedElement.GetHorizontalOffset().Transform;
  trans.Animate(trans.X, -(deletedElement.ActualWidth + 50),
                TranslateTransform.XProperty, 300, 0, new SineEase()
  {
    EasingMode = EasingMode.EaseOut
  },
  () =>
  {
    // find the model object that was deleted
    ToDoItem deletedItem = deletedElement.DataContext as ToDoItem;
 
    // determine how much we have to 'shuffle' up by
    double elementOffset = -deletedElement.ActualHeight;
 
    // find the items in view, and the location of the deleted item in this list
    var itemsInView = todoList.GetItemsInView().ToList();
    var lastItem = itemsInView.Last();
    int startTime = 0;
    int deletedItemIndex = itemsInView.Select(i => i.DataContext)
                                      .ToList().IndexOf(deletedItem);
 
    // iterate over each item
    foreach (FrameworkElement element in itemsInView.Skip(deletedItemIndex))
    {
      // for the last item, create an action that deletes the model object
      // and re-renders the list
      Action action = null;
      if (element == lastItem)
      {
        action = () =>
        {
          // clone the list
          var items = _todoItems.ToList();
          items.Remove(deletedItem);
 
          // re-populate our ObservableCollection
          _todoItems.Clear();
          _todoItems.AddRange(items);
          UpdateToDoColors();
        };
      }
 
      // shuffle this item up
      TranslateTransform elementTrans = new TranslateTransform();
      element.RenderTransform = elementTrans;
      elementTrans.Animate(0, elementOffset, TranslateTransform.YProperty, 200, startTime, null, action);
      startTime += 10;
    }
  });
}

There’s actually rather a lot going on in that method. Firstly the deleted item is animated so that it flies off the screen to the left. Once this animation is complete, we’d like to make the items below ‘shuffle’ up to fill the space. In order to do this, we measure the size of the deleted items, then iterate over all the items within the current view, that are below the deleted item, and apply an animation to each one. The code makes use of the GetItemsInView extension method that I wrote for the WP7 JumpList control – it returns a list of items that are currently visible to the user, taking vertical scroll into consideration.

Once all the elements have shuffled up, our UI now contains a number of ToDoItems that have been ‘artificially’ offset. Rather than try to keep track of how each item is offset, at this point we force the ItemsControl to re-render the entire list.

The result looks pretty cool …

Part of the power of manipulations is their ‘organic’ feel, which is why inertia and the feeling of friction are important considerations. With the current to-do application, when the user releases an item after dragging it, the item springs back to its original positions, as if it were tether via a piece of elastic. The user needs to ‘pull’ the item past the half way mark in order to invoke a delete / complete operation. However, in order to make the interaction feel more ‘organic’ the user should be able to give the todo item a short, fast flick, giving the item enough momentum to pass the critical point. In order to support this we can use the GestureListener.Flick event:

We could do a bit of physics, creating a spring-constant, give our items a nominal mass and determine whether they have ben flicked with a ‘critical velocity’. However, for such a simple application, I’m happy to just come up with a velocity constant, that if passed results in the delet or complete action:

private static double FLICK_VELOCITY = 2000.0;
 
private void GestureListener_Flick(object sender, FlickGestureEventArgs e)
{
  FrameworkElement fe = sender as FrameworkElement;
  if (e.HorizontalVelocity < -FLICK_VELOCITY)
  {
    _flickOccured = true;
    ToDoItemDeletedAction(fe);
  }
  else if (e.HorizontalVelocity > FLICK_VELOCITY)
  {
    _flickOccured = true;
    ToDoItemCompletedAction(fe);        
  }
}

Contextual cues

My friend Graham Odds wrote a great post on the use of contextual cues within user-interface design, which are subtle effects that in Graham’s words “can be invaluable in effectively communicating the functionality and behaviour of our increasingly complex user interfaces”.

The todo-list application uses gestures to delete / complete an item, however, these are not common use interactions so it is likely that the user would have to experiment with the application in order to discover this functionality. They would most likely have to first delete a todo-item by mistake before understanding how to perform a deletion, which could be quite frustrating!

In order to help the user understand the slightly novel interface, we’ll add some very simple contextual cues. In the XAML below a Canvas has been added to the item template, this enables us to position a cross and a tick outside of the visible screen, one to the left and one to the right:

<Grid>
  <!-- add a subtle gradient over the background color for this item -->
  ...
 
  <!-- task text -->
  <TextBlock Text="{Binding Text}" Margin="15,15,0,15" FontSize="30"/>
 
  <!-- the strike-through that is shown when a task is complete -->
  <Line Visibility="{Binding Path=Completed, Converter={StaticResource BoolToVisibilityConverter}}"
        X1="0" Y1="0" X2="1" Y2="0" 
        Stretch="UniformToFill"
        Stroke="White" StrokeThickness="2"
        Margin="8,5,8,0"/>
 
  <!-- a tick and a cross, rendered off screen -->
  <Canvas Opacity="0" x:Name="tickAndCross" >
    <TextBlock Text="×" FontWeight="Bold" FontSize="35"
                Canvas.Left="470" Canvas.Top="8"/>
    <TextBlock Text="✔" FontWeight="Bold" FontSize="35"
                Canvas.Left="-50" Canvas.Top="8"/>
  </Canvas>
</Grid>

In the code-behind, we can locate this Canvas element using Linq-to-VisualTree, then set the opacity so that it fades into view, with the tick and cross elements becoming more pronounced the further the user swipes:

private void GestureListener_DragStarted(object sender, DragStartedGestureEventArgs e)
{
  // initialize the drag
  FrameworkElement fe = sender as FrameworkElement;
  fe.SetHorizontalOffset(0);
  _flickOccured = false;
 
  // find the container for the tick and cross graphics
  _tickAndCrossContainer = fe.Descendants()
                              .OfType<Canvas>()
                              .Single(i => i.Name == "tickAndCross");
}
 
private void GestureListener_DragDelta(object sender, DragDeltaGestureEventArgs e)
{
  // handle the drag to offset the element
  FrameworkElement fe = sender as FrameworkElement;
  double offset = fe.GetHorizontalOffset().Value + e.HorizontalChange;
  fe.SetHorizontalOffset(offset);
 
 
  _tickAndCrossContainer.Opacity = TickAndCrossOpacity(offset);
}
 
private double TickAndCrossOpacity(double offset)
{
  offset = Math.Abs(offset);
  if (offset < 50)
    return 0;
 
  offset -= 50;
  double opacity = offset / 100;
 
  opacity = Math.Max(Math.Min(opacity, 1), 0);
  return opacity;
}

With that subtle visual effect, the first iteration of my gesture-driven to-do application is complete. I’ll add new features, such as drag-to-reorder, and the ability to add / edit items in the near future, again, all driven by gestures.

You can download the code here: ClearStyle.zip

Update: Read more about this application in part two, where I add drag re-ordering.

Regards, Colin E.

Tags: ,
Posted in Uncategorized | 16 Comments »