A couple of days ago we hosted a one-day conference, HTML5 – It Just Got Real, at the Royal Society buildings in London. I’m happy to say that the event was a great success, and that the Q&A session that followed our talks was very lively!

As promised, we have made all of our presentations available online. The first three presentations have all been created using HTML5, whilst the last uses PowerPoint.

Attention!

These presentation were designed to work with the Chrome browser, and are navigated using the left and right arrow keys. They are not intended to be a demonstration of cross-browser HTML5, on the contrary! They illustrate the fact that HTML5 has not solved the need to consider specific browsers and browser-versions when writing code.

HTML5 Revolution

– by Colin Eberhardt, you can view the presentation in your browser, and view the sourcecode on github.

HTML5 – It Just Got Real

– by Chris Price, you can view the presentation in your browser, and view the sourcecode on github.

Designing the Future

– by Graham Odds, you can view the presentation in your browser, and view the sourcecode on github.

Cross-Platform Mobile Development

– by Colin Eberhardt, this presentation is hosted on slideshare, with videos made available via YouTube.

This blog post describes the addition of a two-finger rotation and three-finger pitch gesture to the Windows Phone 8 Map control.

You can see these gesture in action below:

The WP8 release replaced the image-tile based Bing maps with a fully vector-rendered map from Nokia. Being vector-based, this map can be panned, zoomed, rotated and rendered at an angle (i.e. pitched). However, much of this new functionality is not offered to the end user!

The WP8 supports the same gestures that the Bing WP7 map did, i.e. a single-fingered pan gesture and two-fingered pinch to zoom. What about rotation and pitch? Why not allow the user to modify these via gestures?

The key here is to add some new gestures that complement the existing one. I opted for the following:

• Two-finger rotate – When the user places two fingers on the map this is currently used to zoom the display via a ‘spreading’ motion. However, if the user instead rotates the two touch points around the centre, the map should be rotated.
• Three-finger pitch – When the user places three fingers on the map, if they drag up or down, the map should adjust its pitch accordingly.

These gestures are enabled simply by creating them with a reference to the map:

new MapRotationGesture(map);
new MapPitchGesture(map);

If you don’t care how this all works, just head over to github and grab the code. If you want to find out more, read on …

Suppressing The Existing Gestures

In order to add these new gestures to the map, there needs to be a mechanism in place to suppress the existing gestures so that they do not interfere.

The technique I used is similar to a technique I demonstrated previously for suppressing pinch and scroll in the Windows Phone Browser control. Both the Map and WebBrowser controls have a visual tree containing a number of user interface elements. The inner structure of the Map is shown below:

Microsoft.Phone.Maps.Controls.Map
  System.Windows.Controls.Border
    System.Windows.Controls.Border
      Microsoft.Phone.Maps.Controls.MapPresentationContainer
        MS.Internal.ExternalInputContainer
          System.Windows.Controls.Grid
            MS.Internal.TileHostV2
            Microsoft.Phone.Maps.Controls.RootMapLayer

The technique for suppressing interactions is quite simple, just add a ManipulationDelta event handler to each one of these elements, setting the event to handled. The complete code is show below:

/// <summary>
/// A base class for map gestures, which allows them to suppress the built-in map gestures.
/// </summary>
public class MapGestureBase
{
  /// <summary>
  /// Gets or sets whether to suppress the existing gestures/
  /// </summary>
  public bool SuppressMapGestures { get; set; }
 
  protected Map Map { get; private set; }
 
  public MapGestureBase(Map map)
  {
    Map = map;
    map.Loaded += (s,e) => CrawlTree(Map);
  }
 
  private void CrawlTree(FrameworkElement el)
  {
    el.ManipulationDelta += MapElement_ManipulationDelta;
    for (int c = 0; c < VisualTreeHelper.GetChildrenCount(el); c++)
    {
      CrawlTree(VisualTreeHelper.GetChild(el, c) as FrameworkElement);
    }
  }
 
  private void MapElement_ManipulationDelta(object sender, ManipulationDeltaEventArgs e)
  {
    if (SuppressMapGestures)
      e.Handled = true;
  }
}

If you create an instance of this class and associate it with a map you can turn gestures on and off via the SuppressMapGestures property:

var gestureBase = new MapGestureBase(map);
gestureBase.SuppressMapGestures = true;

NOTE: Unfortunately this doesn’t solve the ‘map in a pivot’ or ‘map in a panorama’ problem that many Windows Phone developers have struggled with – the gestures that are handled are not propagated to a parent control.

A Rotation Gesture

The user can rotate the map by placing two fingers on the screen then rotating them around their central point. Because two fingers are also used for the pinch-to-zoom gesture, a suitable threshold needs to be introduced. I have found that disabling rotation until the user has rotated by 10 degrees feels about right.

The code that implements the rotation is really quite simple, the MapGestureBase subclass is shown in its entirety below:

/// <summary>
/// Adds a two-finger rotation gesture to a Map control.
/// </summary>
public class MapRotationGesture : MapGestureBase
{
  /// <summary>
  /// Gets or sets the minimuum rotation that the user must apply in order to initiate this gesture.
  /// </summary>
  public double MinimumRotation { get; set; }
 
  private double? _previousAngle;
 
  private bool _isRotating;
 
  public MapRotationGesture(Map map)
    : base(map)
  {
    MinimumRotation = 10.0;
    Touch.FrameReported += Touch_FrameReported;
  }
 
 
  private void Touch_FrameReported(object sender, TouchFrameEventArgs e)
  {
    var touchPoints = e.GetTouchPoints(Map);
 
    if (touchPoints.Count == 2)
    {
      // for the initial touch, record the angle between the fingers
      if (!_previousAngle.HasValue)
      {
        _previousAngle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
      }
 
      // should we rotate?
      if (!_isRotating)
      {
        double angle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
        double delta = angle - _previousAngle.Value;
        if (Math.Abs(delta) > MinimumRotation)
        {
          _isRotating = true;
          SuppressMapGestures = true;
        }
      }
 
      // rotate me
      if (_isRotating)
      {
        double angle = AngleBetweenPoints(touchPoints[0], touchPoints[1]);
        double delta = angle - _previousAngle.Value;
        Map.Heading -= delta;
        _previousAngle = angle;
      }
    }
    else
    {
      _previousAngle = null;
      _isRotating = false;
      SuppressMapGestures = false;
    }
  }
 
  private double AngleBetweenPoints(TouchPoint p1, TouchPoint p2)
  {
    return Math.Atan2(p1.Position.Y - p2.Position.Y, p1.Position.X - p2.Position.X)
            *(180 / Math.PI);
  }
}

Touch gestures are detected via the Touch.FrameReported event. When two fingers are placed on the screen the initial rotation angle is recorded. When the minimum rotation is exceeded, the Map.Heading is updated with each ‘delta’ reported. Really simple code, but a fantastic feature for the user!

A Pitch Gesture

You get a real feel for the vector-nature of the maps when you set the ‘pitch’, a style of rendering that is often used on satnavs.

I initially considered using a two finger pull-down gesture, which is similar to the one which Google Maps on Android uses, but found it very hard to coordinate the three gestures, zoom, rotate, pitch, which all use the same two-fingers! So instead, I opted for a three-finger pull-down gesture to increase the pitch of the map.

The code follows a very similar pattern to the rotate gesture:

/// <summary>
/// Adds a three-finger pitch gesture to a Map control.
/// </summary>
public class MapPitchGesture : MapGestureBase
{
  /// <summary>
  /// Gets or sets the sensitivity of this gesture
  /// </summary>
  public double Sensitivity { get; set; }
 
  private double? _initialPitchYLocation;
 
  public MapPitchGesture(Map map)
    : base(map)
  {
    Sensitivity = 0.5;
    Touch.FrameReported += Touch_FrameReported;
  }
 
  private void Touch_FrameReported(object sender, TouchFrameEventArgs e)
  {
    var touchPoints = e.GetTouchPoints(Map);
 
    SuppressMapGestures = touchPoints.Count == 3;
 
    if (touchPoints.Count == 3)
    {
      if (!_initialPitchYLocation.HasValue)
      {
        _initialPitchYLocation = touchPoints[0].Position.Y;
      }
 
      double delta = touchPoints[0].Position.Y - _initialPitchYLocation.Value;
      double newPitch = Math.Max(0, Math.Min(75, (Map.Pitch + delta * Sensitivity)));
      Map.Pitch = newPitch;
      _initialPitchYLocation = touchPoints[0].Position.Y;
    }
    else
    {
      _initialPitchYLocation = null;
    }
  }
}

As soon as three fingers are placed on the screen, the gesture becomes active. The movement of the first finger is used to determine the delta to apply to the Pitch property. Again, nice and simple!

The sourcecode for these gesture, plus a demo app is available via github. Please let me know if you use this code in any of your apps.

Regards, Colin E.

This blog post compares the same twitter search application written with both Knockout and Kendo in order to highlight the strengths and weaknesses of each framework.

Introduction

I’ve always been a big fan of Knockout, most likely because it reminds me of Silverlight (rest in peace).

For my thoughts on how Silverlight and Knockout compare, head over to codeproject where I wrote quite a lengthy article on the subject.

More recently I have been dabbling with the use of Knockout for writing mobile applications. Because Knockout is an MVVM framework and doesn’t have any UI components of its own, I used a combination of jQuery Mobile (jQM) and Knockout. I must admit, the integration of these two frameworks wasn’t entirely painless!.

It is the integration pain I experienced with jQM and Knockout that has resulted in my interest in KendoUI.

KendoUI (www.kendoui.com) is a commercial JavaScript framework that includes web and mobile UI widgets together with an MVVM framework, which, on initial inspection, closely resembles Knockout. The promise of an MVVM framework and mobile widgets that play nicely together is quite appealing!

In order to compare Kendo with Knockout + jQM I re-implemented the twitter-search application that I blogged about a few months ago. The rest of this blog post is a collection of thoughts, commentary on the differences and personal opinions.

The two version of the code are available on github:

• https://github.com/ColinEberhardt/Knockout-jQueryMobile
• https://github.com/ColinEberhardt/KendoUI-TwitterSearch

And you can see working examples at the following locations:

• Knockout + jQuery Mobile
• KendoUI

It is worth viewing the above on your phone.

From Knockout to KendoUI

• Knockout has magic properties, Kendo has magic objects!
• Knockout is more flexible regarding view model construction
• Kendo requires a little effort to make it work with Intellisense
• Knockout supports view model hierarchies
• Kendo bindings are not JavaScript!
• Kendo lacks a $data binding
• Kendo has its own UI widgets!
• Kendo UI does not mangle the DOM
• jQM has a more extensive set of styles
• Kendo supports multiple mobile themes
• Kendo has a datasource concept
• Knockout is open source, Kendo costs $$$

Knockout has magic properties, Kendo has magic objects!

JavaScript does not support the notion of change-notification when you change the value of an object’s properties. This is a vital feature for supporting binding within an MVVM framework, so both Kendo and Knockout have to provide this missing change notification.

With Knockout change notification is provided via observable properties:

var knockoutViewModel = {
  name : ko.observable("john")
};

When a property has been created via ko.observable you can subscribe to change notifications for that property.

With Kendo change notification is provided via observable objects:

var kendoViewModel = kendo.observable({
  name : "john"
});

And as a result you can subscribe to change notifications on the object..

Neither Kendo or Knockout are able to detect when a property is set directly, instead, you must set the property value via a function. With Knockout, your property is actually a function which you can set as follows:

knockoutViewModel.name(“john smith”):

Whereas with Kendo you set the property via the owning object:

kendoViewModel.set(“name”, “john smith”);

Both Kendo and Knockout support the concept of dependant properties, where you construct a property by composing one or more of the other objects properties, with change notifications happening automatically.

Both Kendo’s object-centic and Knockout’s property-centric approaches seem quite reasonable, but as we shall see later this small difference has an impact on many aspects of each framework.

Knockout is more flexible regarding view model construction

Because Knockout performs its ‘magic’ at the property level, it does not place any restrictions on the way in which you construct your view models. You can use the literal syntax:

var knockoutViewModel = {
  name : ko.observable("john")
};

Or define a constructor:

function KnockoutViewModel() {
  this.name = ko.observable("john");
}
 
var viewModel = new KnockoutViewModel();

Personally I like the second approach because it gives you a view model ‘factory’, i.e. you can create multiple instances of the same view model, and it also allows you to define both public and private functions:

function KnockoutViewModel() {
  this.name = ko.observable("john");
 
  saveState = function() {
    // do something here
  }
 
  this.update = function() {
    saveState();
  }
}

In the code above the saveState function is private – for more information on this pattern I’d recommend reading Private Members in JavaScript by Douglas Crockford.

All of the Kendo examples create view model using a literal syntax:

var kendoViewModel = kendo.observable({
  name : "john"
});

Initially I was under the impression that it wasn’t possible to create Kendo view models using constructor functions, furthermore, one of the nice guys from the Kendo support rather apologetically told me “I am afraid that it is not possible to emulate Knockout style of view model creation via functions. Kendo MVVM supports only the literal style.”.

However, after a little bit of thought, I realised that it is possible if you explicitly return an observable as follows:

function KendoViewModel() {
  this.name = “john”;
 
  return kendo.observable(this);
}
 
var viewModel = new KendoViewModel ();

However, things are not quite so straightforward – if you need to capture the reference to ‘this’ you must ensure that the reference is to the observable object, as follows:

function KendoViewModel() {
  this.name = “john”;
 
  function saveState = function() { 
    console.log(that);
  }
 
  this.update = function() {
    saveState();
  }
 
  var that = kendo.observable(this);
  return that;
}
 
var viewModel = new KendoViewModel ();

In the above code the variable, that, is always going to refer to the observable object regardless of what, this, refers to.

Kendo requires a little effort to make it work with Intellisense

Following on from the previous point, because the Kendo ‘magic’ is at the object level, Visual Studio Intellisense becomes flummoxed. Now I know that Visual Studio is perhaps not the best tool for JavaScript, but I can bet that quite a few Knockout and Kendo developers use it!

Visual Studio uses pseudo-execution of your JavaScript code to provide Intellisense. With Knockout this works well out-of-the-box – see the image below where the update function, together with its ‘summary’ text, are picked up by Intellisense:

Whereas with Kendo, in order for Visual Studio to be able to execute your code it needs to be aware of the Kendo and jQuery JavaScript files. This can be done by adding a ‘reference’ comment, as shown below:

Note that you can also see the other functions that are present on observable objects.

Knockout supports view model hierarchies

One particularly elegant feature of Knockout is that it supports view model hierarchies. When binding collections of objects, Knockout creates a child binding context that refers to the nested view model data. You can also explicitly set the binding-context of an element via the ‘with’ binding. This approach allows you to create complex nested view models.

With Kendo, when binding a collection of objects the ‘context’ for each item is the parent view model, which means that any event bindings are sent to the owning view model. This is clearly due to the way that Kendo uses observable objects rather than observable properties.

Whilst this is not a significant limitation, I do miss the binding-context concept when working with Kendo. The lack of this feature does mean that view models can become bloated.

Kendo bindings are not JavaScript!

With Knockout, bindings are JavaScript, which means you can have bindings which contain logic. For example:

data-bind="enable: searchTerm().length > 0 && isSearching() == false,
           click: search"

Kendo bindings are not JavaScript!

A simple way to achieve equivalent functionality is to move the JavaScript in the binding into a dependant property:

searchButtonDisabled: function () {
  return this.get("searchTerm").length === 0 && this.get("isSearching") === false;
}

Which is bound as follows:

data-bind="click: executeSearch, { disabled: searchButtonDisabled }"

I do not see this as a significant limitation of Kendo. I do use this feature quite often with Knockout, it feels a little like the Silverlight / WPF concept of value-converters. However, I can see how it could be abused, hiding JavaScript logic within bindings isn’t terribly good for testing or maintenance!

Kendo lacks a $data binding

With Knockout, if you create a ‘foreach’ binding to an array of strings, the template that renders each string can make use of the $data binding:

<span data-bind="text: $data"></span>

The above is needed because the binding-context of the above span element is a string – we want to bind to the object itself rather than a property of the object.

I could not find an equivalent feature with Kendo, so had to create an object as a container for the string:

this.recentSearches.unshift({ searchString: this.searchTerm });

And bind it as follows:

<span data-bind="text:searchString"></span>

Again, a pretty minor limitation.

Kendo has its own UI widgets!

So far much of my discussion has focussed on Knockout features which Kendo does not support. However, Kendo is much more than just an MVVM framework, it includes a large number of web and mobile UI widgets.

In the context of this comparison, I integrated jQM with Knockout to create my mobile UI – and as I mentioned earlier, this integration was pretty painful. In contrast, the Kendo UI Mobile framework integrates seamlessly with the Kendo MVVM framework – as you might expected!

Kendo UI does not mangle the DOM

While we are on the subject of the UI, one of the features of jQM that I really disliked is the way that it totally mangles the DOM.

In order to support the jQM styling it requires a large number of ‘supporting’ elements. As an example a button is transformed from this:

<button type="submit" data-bind="enable: isSearchEnabled,
                                 click: executeSearch">Go</button>

To this:

<div class="ui-btn ui-btn-inline ui-btn-corner-all ui-shadow ui-btn-up-c">
  <span class="ui-btn-inner ui-btn-corner-all">
    <span class="ui-btn-text">Go</span>
  </span>
  <input class="ui-btn-hidden" type="button" value="Go"
                  data-bind="enable: isSearchEnabled, 
                                    click: executeSearch"/>
</div>

This causes all kinds of headaches when trying to use jQM with an MVVM or MVC framework.

In contrast, with Kendo when you create a button:

<a data-role="button" 
   data-bind="click: executeSearch, { disabled: searchButtonDisabled }">Go</a>

It adds very little supporting structure:

<a data-role="button" class=" km-button"
      data-bind="click: executeSearch, { disabled: searchButtonDisabled }">
      <span class="km-text">Go</span>
</a>

And in many cases it leaves the DOM structure unaltered.

jQM has a more extensive set of styles

The jQuery Mobile framework is more extensive than Kendo. As a quick example, jQM nicely styles your list items:

With Kendo, you have to put in a bit of extra effort to style the list:

This is just one example of many – jQM is a big framework.

Kendo supports multiple mobile themes

The jQM UI is designed to look at home on iOS devices, which is just fine for the ~30% of people who own an iPhone. However, this UI just looks plain wrong on an Android or Windows Phone.

There are a few open source Android themes for jQM, but I didn’t get a satisfactory result with any of them. Also, Microsoft have contributed a Metro theme to jQM, but it is rather ugly!

In contrast, Kendo supports iOS, Android, BlackBerry and Windows Phone, as shown on their online demo. This is pretty neat!

Kendo has a datasource concept

With the Knockout + jQM application I wrote a very simple TwitterSearchService which acts as a datasource. The Kendo framework has a DataSource component which makes integrating with services such as Twitter much easier, you typically provide configuration to the DataSource and it does the rest. Also, the UI components like the ListView integrate directly with the DataSource, with features like pull-to-load-more available out-of-the-box.

Knockout is open source, Kendo costs $$$

This one is pretty obvious really …

Knockout is open source, and as a result doesn’t cost you anything to use it. For an open source project it is very well documented and there are many blog posts that provide tutorials and solutions to problems. The interactive tutorial is a particularly fantastic idea.

Kendo will cost you a few hundred dollars to buy (http://www.kendoui.com/purchase.aspx). Considering that Kendo is a UI widget framework, MVVM framework and mobile framework, I would say that the price is very reasonable. Also, the documentation is of a similar quantity and quality to that of Knockout.

Conclusions

This article has been a mixed-bag of observations, thoughts and opinions and it is not intended to be an exhaustive comparison. Despite that, I hope it will be useful for others.

I certainly prefer the Knockout implementation of MVVM, I find it more powerful and flexible than the Kendo equivalent. However, Kendo’s MVVM framework is certainly adequate.

When it comes to building mobile applications, the fully integrated Kendo solution has some pretty big advantages over Knockout and jQuery Mobile.

Regards, Colin E.

Announcing a one-day conference at the Royal Society, London UK, on the 14th of May.

Just a few years back I wrote a white paper titled “Flex, Silverlight or HTML5? Time to decide …” with the help of a couple of my colleagues, comparing the strengths and weaknesses of the three competing technologies. This paper was aimed at helping developers make an informed choice when faced with developing an interactive web site or complex web app.

Don’t bother reading that white paper now! It was only written two years ago, but is now so out-of-date that it might as well have been written ten years ago.

Here’s the penultimate paragraph:

“It is very difficult to predict where we will be in a few years’ time and whether there will be some technology casualties along the way. Can all three technologies co-exist? Or will one or more be relegated to the internet history books like Java Applets and ActiveX?”

Yes, there have been some casualties, both Silverlight … and Flex!

The rise of smartphones and tablets has accelerated HTML5 adoption to the point where plugins are considered a legacy technology. Invest in them at your peril!

The problem is that enterprise-scale HTML5 development is far from easy. With Flex and Silverlight you have the comfort of working within a shrink-wrapped development environment, whereas HTML5 is like the Wild West; assembling a workable set of tools, framework and processes is not an easy task.

This is why we decided to stage our own conference – we want to talk about the challenges of enterprise scale HTML5 development, and how to tackle them.

If you are interested, and are thinking of signing up, take a look at the conference website – html5-it-just-got-real.com – there will be talks on a range of subjects, ranging from practical and insightful to radical and fun (we have a BBC presenter and a guitar-wielding JavaScript rockstar).

Regards, Colin E.

I’ve just received an email from those nice folks at CodeProject letting me know that my article A Windows Phone 8 Run Tracking App in 100 Lines of Code has just won the Best Mobile article of January 2013 award.

I must admit, this one was a bit of a surprise. I put this article together very quickly for the Nokia Wiki Competition … which I didn’t win. The article is quite brief, and the app that it described is very simple – this is in contrast to my usual articles on CodeProject, which are often quite lengthy. Judging from the comments, it looks like people really like these short, but focused articles. I might do a few more of these in future! (they are certainly less effort )

Colin E.

In the past few months I have been immersing myself into the world of iOS development. Whilst the Objective-C language takes a little getting used to, with its odd syntax and memory management rules, it is certainly a fun platform to develop for. I do miss a few C# language features, such as events, but for the most part I am a happy iOS developer.

… that is apart from Linq!

A few weeks back I was writing some examples for a future release of the ShinobiControls Grid. The example in question required a datasource of people grouped by the first letter of their surname. Despite the use of predicates and key valued sorting, the code looked like this:

- (NSArray*) createGroupedTestData
{
    NSMutableArray* groupedData = [[NSMutableArray alloc] init];
 
    // create some people
    NSArray* people = [PersonDataSource generatePeople:50];
 
    // create a set of letters - based on the first letter of the surname
    NSMutableSet* groups = [[NSMutableSet alloc] init];
    for(PersonDataObject* person in people)
    {
        [groups addObject:[person.surname substringToIndex:1]];
    }
 
    // create the groups
    for(NSString* letter in groups)
    {
        PersonGroup* group = [[PersonGroup alloc] init];
        group.letter = letter;
 
        // locate the person objects that belong in this group
        NSPredicate* predicate = [NSPredicate predicateWithBlock:^BOOL(id evaluatedObject, NSDictionary *bindings) {
            PersonDataObject* person = (PersonDataObject*)evaluatedObject;
            return [[person.surname substringToIndex:1] isEqualToString:letter];
        }];
        group.items = [people filteredArrayUsingPredicate:predicate];
 
        [groupedData addObject:group];
    }
 
    // sort the groups
    NSSortDescriptor* sortDescriptor = [[NSSortDescriptor alloc] initWithKey:@"letter" ascending:YES];
    NSArray* sortedGroupedData = [groupedData sortedArrayUsingDescriptors:@[sortDescriptor]];
 
    return sortedGroupedData;
}

That’s a pretty unwieldy lump in my opinion. With Linq I am used to performing this sort of logic with a single query!

So, I decided to go ahead and implement a bunch of Linq methods for Objective-C, which results in a much cleaner implementation:

- (NSArray*) createGroupedTestData
{
 
    // create some people
    NSArray* people = [PersonDataSource generatePeople:50];
 
    Selector firstLetterOfSurname = ^id(id person){
        return [[person surname] substringToIndex:1];
    };
 
    Selector groupForLetter = ^id(id letter) {
        PersonGroup* group = [[PersonGroup alloc] init];
        group.letter = letter;
        group.items = [people where:^BOOL(id person) {
            return [[[person surname] substringToIndex:1] isEqualToString:letter];
        }];
        return group;
    };
 
    return [[[[people   select:firstLetterOfSurname]
                        sort]
                        distinct]
                        select:groupForLetter];
}

Not only is the above code more concise, which is nearly always a good thing (unless you are talking regex-concise!), it is also much more readable. The ‘select, distinct, sort, select’ query makes it much easier to determine what the resulting output of this code will be.

In this blog post I’ll explain what Linq is (for those of you who have not heard of it before), and how I implemented Linq-to-Objective-C. If you just want to use the code, you can grab a copy from github.

An introduction to Linq

If you are a seasoned .NET developer, you can skip this section, however, if you have not heard of Linq before, then read on …

Linq was added to the C# language back in 2007. The name ’linq’ stands for Language Integrated Query, and as the name implies, its original aim was to more tightly integrate queries into the C# language. As an example, when querying database developers find themselves writing queries using SQL statements embedded within literal strings:

string query = “SELECT * FROM Person WHERE age < 10”

The problem with the above code is that the SQL statements are not checked by the compiler, and as a result, errors will only surface at runtime. This is not a problem that is peculiar to C# – you will find SQL statements embedded within strings in virtually every language.

With Linq queries are no longer literal strings, they are instead constructed using keywords that are part of the C# language itself:

var query =  from p in PersonCollection
               where p.age < 10
               select p;

This allows the compiler to check your query syntax, resulting in less error prone code.

C# provides a couple of different syntaxes for Linq, the query syntax, shown above – and the fluent syntax, where method invocations are chained together.

The two are functionally equivalent:

var query =  from p in PersonCollection
               where p.age < 10
               select p;
 
var query = PersonCollection.Where(p => p.age < 10);

Whilst the application of Linq for querying database is obvious, Linq can be used to query practically anything. The .NET framework supplies Linq to SQL, XML and Objects – while the community has added Linq to CSV, JSON, DataSets, Facebook, and my own contribution Linq-to-VisualTree.

I’ve found myself using Linq in virtually every .NET application I have ever written – and I miss it!

Fortunately the ~25 different Linq extensions methods are actually quite easy to implement, as the C# guru Jon Skeet demonstrated in an epic series of blog posts. So, why not bring this API to Objective-C?

Linq to Objective-C

In this section I’ll rattle through the implementation of a number of Linq methods, for the full set, head to github.

Where

The Linq ‘where’ method filters an array of objects based on a predicate that returns true for any object that should be included in the output array. Using a block for the predicate, gives the following method signature:

typedef BOOL (^Predicate)(id);
 
/** Filters a sequence of values based on a predicate.
 
 @param The function to test each source element for a condition.
 @return The elements from the input sequence that satisfy the condition.
 */
- (NSArray*) where:(Predicate)predicate;

Before diving straight into the implementation, I’ll demonstrate its usage via a unit test. Given the following array of Person objects:

- (NSArray*) createTestData
{
    return @[[Person personWithName:@"bob" age:@25],
    [Person personWithName:@"frank" age:@45],
    [Person personWithName:@"ian" age:@35],
    [Person personWithName:@"jim" age:@25],
    [Person personWithName:@"joe" age:@55]];
}

Where Person is a simple object with name and age properties. The test I created for the where method is as follows:

- (void)testWhere
{
    NSArray* input = [self createTestData];
 
    NSArray* peopleWhoAre25 = [input where:^BOOL(id person) {
        return [[person age] isEqualToNumber:@25];
    }];
 
    STAssertEquals(peopleWhoAre25.count, 2U, @"There should have been 2 items returned");
    STAssertEquals([peopleWhoAre25[0] name], @"bob", @"Bob is 25!");
    STAssertEquals([peopleWhoAre25[1] name], @"jim", @"Jim is 25!");
}

Which simply tests that the simple where query successfully returns all Person instances with an age of 25.

The implementation of where is really quite trivial:

- (NSArray *)where:(Predicate)predicate
{
    NSMutableArray* result = [[NSMutableArray alloc] init];
    for(id item in self) {
       if (predicate(item)) {
           [result addObject:item];
       }
    }
    return result;
}

We’re off to a good start!

So, how does this compare to the existing Objective-C APIs? Through the use of predicates it is quite easy to perform the same ‘people who are 25’ query. You can use block syntax:

NSPredicate* predicate = [NSPredicate predicateWithBlock:^BOOL(id person, NSDictionary *bindings) {
    return [[person age] isEqualToNumber:@25];
}];
NSArray* peopleWhoAre25 = [input filteredArrayUsingPredicate:predicate];

Or the more concise string-based predicates:

NSArray* peopleWhoAre25 = [input filteredArrayUsingPredicate:[NSPredicate predicateWithFormat:@"age=25"]];

With the Objective-C support for predicates, this Linq-style where method might not be the most useful, but the others are more interesting.

Select

The Linq ‘select’ method allows you to perform transformation, which is known as a projection. Each element is acted upon by a selector function, with the returned result being used to populate the output sequence. Here’s the method signature:

typedef id (^Selector)(id);
 
/** Projects each element of a sequence into a new form.
 
 @param selector Projects each element of a sequence into a new form.
 @return An array whose elements are the result of invoking the transform function on each element of source.
 */
- (NSArray*) select:(Selector)selector;

Again, I’ll demonstrate the using via a unit test. This example uses the select method to extract the name property for each person objects in the source array:

- (void)testSelect
{
    NSArray* input = [self createTestData];
 
    NSArray* names = [input select:^id(id person) {
        return [person name];
    }];
 
    STAssertEquals(names.count, 5U, nil);
    // 'spot' check a few values
    STAssertEquals(names[0], @"bob", nil);
    STAssertEquals(names[4], @"joe", nil);
}

Again, the implementation is really quite simple:

- (NSArray *)select:(Selector)selector
{
    NSMutableArray* result = [[NSMutableArray alloc] init];
    for(id item in self) {
        [result addObject:selector(item)];
    }
    return result;
}

Sort

Linq doesn’t actually have a sort method, instead it provides OrderBy and ThenBy methods. Because I don’t want to get into multi-property sorting, I’ll substitute both of these for a single ‘sort’ function.

/** Sorts the elements of a sequence in ascending order.

@return An array whose elements are sorted in ascending order.
*/
- (NSArray*) sort;

/** Sorts the elements of a sequence in ascending order by using a specified keySelector.

@param keySelector A selector that provides the ‘key’ which the array should by sorted by.
@return An array whose elements are sorted in ascending order.
*/
- (NSArray*) sort:(Selector)keySelector;

The first method simply sorts the array by invoking the compare method on each object. The second uses a selector to extract the key that is then used for the comparison.

The unit tests are as follows:

- (void)testSort
{
    NSArray* input = @[@21, @34, @25];
 
    NSArray* sortedInput = [input sort];
 
    STAssertEquals(sortedInput.count, 3U, nil);
    STAssertEqualObjects(sortedInput[0], @21, nil);
    STAssertEqualObjects(sortedInput[1], @25, nil);
    STAssertEqualObjects(sortedInput[2], @34, nil);
}
 
- (void)testSortWithKeySelector
{
    NSArray* input = [self createTestData];
 
    NSArray* sortedByName = [input sort:^id(id person) {
        return [person name];
    }];
 
    STAssertEquals(sortedByName.count, 5U, nil);
    STAssertEquals([sortedByName[0] name], @"bob", nil);
    STAssertEquals([sortedByName[1] name], @"frank", nil);
    STAssertEquals([sortedByName[2] name], @"ian", nil);
    STAssertEquals([sortedByName[3] name], @"jim", nil);
    STAssertEquals([sortedByName[4] name], @"joe", nil);
}

As you can see, one simply sorts an array of NSNumber instances, whereas the other sorts an array of People instances via the name property using the key selector.

The implementation is as follows:

- (NSArray *)sort:(Selector)keySelector
{
    return [self sortedArrayUsingComparator:^NSComparisonResult(id obj1, id obj2) {
        id valueOne = keySelector(obj1);
        id valueTwo = keySelector(obj2);
        NSComparisonResult result = [valueOne compare:valueTwo];
        return result;
    }];
}
 
- (NSArray *)sort
{
    return [self sort:^id(id item) { return item;} ];
}

The sort-with-keySelector method makes use of the NSArray sort method that sorts via the result returned by a block. The ‘natural’ sort simply uses an identity selector in order to re-use the same implementation.

OfType

The Linq of-type method is use to filter a list of objects to remove all of those which are not of a specific type. Here’s the method signature:

/** Filters the elements of an an array based on a specified type.
 
 @param type The type to filter the elements of the sequence on.
 @return An array whose elements are all of the given type.
 */
- (NSArray*) ofType:(Class)type;

And here is the associated test:

- (void)testOfType
{
    NSArray* mixed = @[@"foo", @25, @"bar", @33];
 
    NSArray* strings = [mixed ofType:[NSString class]];
 
    STAssertEquals(strings.count, 2U, nil);
    STAssertEqualObjects(strings[0], @"foo", nil);
    STAssertEqualObjects(strings[1], @"bar", nil);
}

Which filters out the NSString instances from a mixed array.

The implementation is quite trivial, making use of the where method which was detailed above:

- (NSArray *)ofType:(Class)type
{
    return [self where:^BOOL(id item) {
        return [[item class] isSubclassOfClass:type];
    }];
}

The ofType method is quite useful for tasks such as finding all the subviews of a UIView that are of a specific type.

Distinct

The distinct method returns the distinct elements from an array, in other words, it removes duplicates based on an equality comparison. The signature is as follows:

/** Returns distinct elements from a sequence.
 
 @return An array of distinct elements.
 */
- (NSArray*) distinct;

The unit test is as follows:

- (void)testDistinct
{
    NSArray* names = @[@"bill", @"bob", @"bob", @"brian", @"bob"];
 
    NSArray* distinctNames = [names distinct];
 
    STAssertEquals(distinctNames.count, 3U, nil);
    STAssertEqualObjects(distinctNames[0], @"bill", nil);
    STAssertEqualObjects(distinctNames[1], @"brian", nil);
    STAssertEqualObjects(distinctNames[2], @"bob", nil);
}

The implementation uses an NSMutableArray to construct an array of unique objects:

- (NSArray *)distinct
{
    NSMutableArray* distinctSet = [[NSMutableArray alloc] init];
    for (id item in self) {
        if (![distinctSet containsObject:item]) {
            [distinctSet addObject:item];
        }
    }
    return distinctSet;
}

Note that the use of containsObject on an NSMutableArray is a potential performance issue for large arrays of data. A better method might make use of an NSSet which uses the object’s hash in order to more quickly determine whether an item is already contained within a set.

Select Many

The Linq select-many method is projection where the selector returns an array of objects. The select-many function flattens the returned arrays into a single array. For a good visual explanation of select-many check out this blog post.

Here is the method signature:

/** Projects each element of a sequence to an NSArray and flattens the resulting sequences into one sequence.
 
 @param transform A transform function to apply to each element, this should return an NSArray.
 @return An array whose elements are the result of invoking the one-to-many transform function on each element of the input sequence.
 */
- (NSArray*) selectMany:(Selector)transform;

The unit test takes an array of string, projecting each element to an array via the componentsSeparatedByString method:

- (void)testSelectMany
{
    NSArray* data = @[@"foo, bar", @"fubar"];
 
    NSArray* components = [data selectMany:^id(id string) {
        return [string componentsSeparatedByString:@", "];
    }];
 
    STAssertEquals(components.count, 3U, nil);
    STAssertEqualObjects(components[0], @"foo", nil);
    STAssertEqualObjects(components[1], @"bar", nil);
    STAssertEqualObjects(components[2], @"fubar", nil);
}

A more realistic example might take an array of orders, projecting each order to the order-items that it contains, with the result being all of the order-items over all orders … this could of course be followed by a ‘distinct’ query which would find all the distinct (i.e. unique) items ordered for a collection of orders.

The implementation involves a couple of nested for-in loops:

- (NSArray *)selectMany:(Selector)transform
{
    NSMutableArray* result = [[NSMutableArray alloc] init];
    for(id item in self) {
        for(id child in transform(item)){
            [result addObject:child];
        }
    }
    return result;
}

Aggregate

The next Linq method, aggregate, is an interesting one because it doesn’t return an array, instead it applies an accumulator function to the elements of an array, returning a single result. The signature is as follows:

/** Applies an accumulator function over a sequence.
 
 @param accumulator An accumulator function to be invoked on each element.
 @return The final accumulator value.
 */
- (id) aggregate:(Accumulator)accumulator;

The unit test uses this method to convert an array of strings into a CSV:

- (void)testAggregate
{
    NSArray* names = @[@"bill", @"bob", @"brian"];
 
    id aggregate = [names aggregate:^id(id item, id aggregate) {
        return [NSString stringWithFormat:@"%@, %@", aggregate, item];
    }];
 
    STAssertEqualObjects(aggregate, @"bill, bob, brian", nil);
}

Although you can use aggregate to do all sorts of things, such as find the maximum value for an array of integers.

The implementation is pretty simple, seeding the aggregate with the initial value and using the accumulator for subsequent items:

- (id)aggregate:(Accumulator)accumulator
{
    id aggregate = nil;
    for (id item in self) {
        if (aggregate == nil) {
            aggregate = item;
        } else {
            aggregate = accumulator(item, aggregate);
        }
    }
    return aggregate;
}

A brief note on deferred execution

In C# Linq methods do not act on arrays, instead they act upon the IEnumerable interface. This allows them to query anything that can be enumerated, which not only includes arrays, but can also include infinite sources of data (such as a stream of random numbers). The implementation of Linq relies on the ‘yield’ keyword, which results in the C# compiler creating a state machine to implement the IEnumerable interface.

The practical implication of all this very clever stuff is that Linq queries are not executed immediately. As an example, when you create a query as follows:

var query = PersonCollection.Where(p => p.Age < 10)
                            .OrderBy(p => p.Name);

Nothing will actually happen until you start to enumerate the ‘query’ variable. In other words, you have to ‘pull’ data from the query.

This results in a feature which is often called ‘deferred execution’, where queries are executed on demand. This can yield tremendous performance benefits, allowing queries to terminate early.

Objective-C does not have ‘yield’ keyword, making it hard to implement Linq in such a way as to allow deferred execution. I did actually manage to create an implementation of a few query methods where each were presented as a state machine. Also, the Obj-C for-in iterator uses the NSFastEnumeration, which I was able to implement giving deferred execution and early termination. However, I decided to stick with the much simpler NSArray implementation here. If anyone is interested in the more complex version, give me a shout – I’d be happy to share.

Conclusions

Creating a Linq-style API for Objective-C has proven to be relatively simple (and fun!). I have provided implementations for some of the most common and useful Linq methods, however the .NET Linq APIs for querying collections have a great many more methods than the ones I have covered.

If you are interested, why not pop over to github, fork and add a few more? I’ll probably add to this project myself over time.

Regards, Colin E.

Earlier today I received an email informing me that I have been awarded a CodeProject MVP for the year 2013. I am very grateful to CodeProject for this award – it is a very popular website for developers, with around 10 million members, and thousands of authors. It is an honour to be among the company of the ~30 other MVPs.

This year I have contributed five articles to Codeproject:

• KnockoutJS vs. Silverlight – An article that helps people transition from Silverlight to Knockout and JavaScript.
• Plotting Circular Relationship Graphs with Silverlight – An article that I wrote on my travels to Denmark, a bit of fun with data visualisation.
• A Gesture-Driven Windows Phone To-Do List – which describes how to create a to-do list application which is controlled via multi-touch gestures. This article won the “Best Mobile article of July 2012″ award.
• Property Finder – a Cross-Platform HTML5 Mobile App – which was an article about the use of HTML5 for cross-platform development, this was the ‘seed’ for the PropertyCross project
• Clearer – A Gesture-Driven Windows 8 To-Do Application – which was basically my attempt to win an Ultrabook by re-hashing my earlier Windows Phone article … which worked

That’s one more article than I contributed the previous year!

Over the past year my development interests have diversified, in 2011 I was mainly focused on WPF, Silverlight and Windows Phone. I now find myself spending just as much time on JavaScript and iOS – neither of which are really the core focus for CodeProject. I will probably be writing for Ray Wenderlich’s iOS site just as much as CodeProject throughout 2013, however, I doubt I will ever give up on CodeProject. It has such a lively community, attracts some of the best technical authors, and provides them with a great environment to work within.

Regards, Colin E.

Yesterday I published a new article on CodePrioject:

• Property Finder – a Cross-Platform Xamarin MonoTouch Mobile App

This article looks at how you can write a cross-platform mobile application with a shared C# code base using Xamarin MonoTouch. The application that is described in the article, a property finder, is exactly the same one that I wrote using HTML5 a few months earlier. This most recent article provides an interesting comparison of the two different techniques.

I actually wrote most of this article about two months ago, it was at this point I decided to embark on PropertyCross, which provides a comparison of a wide range of cross-platform frameworks.

After finishing PropertyCross I decided that the original article was still worth publishing, it provides a lot of technical detail on how to code share with Xamarin specifically.

So, why not pop over to CodeProject and check it out?

Regards, Colin E.