I have recently published an article on codeproject which describes a technique for generating Linq API for querying tree-like structures. This blog post makes use of a generated API for WPF / Silverlight. If you are interested in the more generic approach, and how this API was constructed, (and how it is influenced by XPath) head on over to codeproject …

What I will provide here is a brief overview of the Linq to Visual Tree API. The full sourcecode for this API is at the end of this article.

The Linq to Visual Tree API defines a number of extension methods on DependencyObject that provide mechanisms for navigating to other DependencyObject instances. I will provide a few examples that query the following simple markup:

<Grid x:Name="GridOne" Background="White">
    <Grid.RowDefinitions>
        <RowDefinition Height="Auto" />
        <RowDefinition Height="Auto" />
    </Grid.RowDefinitions>
 
    <TextBox x:Name="TextBoxOne" Text="One" Grid.Row="0" />
 
    <StackPanel x:Name="StackPanelOne" Grid.Row="1">
        <TextBox x:Name="TextBoxTwo" Text="Two" />
 
        <Grid x:Name="GridTwo">
            <Grid.RowDefinitions>
                <RowDefinition Height="Auto" />
                <RowDefinition Height="Auto" />
            </Grid.RowDefinitions>
 
            <TextBox x:Name="TextBoxThree" Text="Three" Grid.Row="0" />
 
            <StackPanel x:Name="StackPanelTwo" Grid.Row="1">
                <TextBox x:Name="TextBoxFour" Text="Four"/>                    
            </StackPanel>
        </Grid>
    </StackPanel>
</Grid>

We’ll start with a simple example. Include the Linq to Visual Tree namespace, then use the Descendants method to obtain all the descendants (i.e. children and children’s children, etc …) of an object within the
visual tree.

Descendants

using LinqToVisualTree;
 
// all items within the visual tree
IEnumerable<DependencyObject> allDescendants = this.Descendants();
/*
gives ...
0 {Grid} 	[GridOne]
1 {TextBox} 	[TextBoxOne]
2 {StackPanel} 	[StackPanelOne]
3 {TextBox} 	[TextBoxTwo]
4 {Grid} 	[GridTwo]
5 {TextBox} 	[TextBoxThree]
6 {StackPanel} 	[StackPanelTwo]
7 {TextBox} 	[TextBoxFour]
*/
 
// all items within the visual tree of 'GridTwo'
var descendantsOfGridTwo = GridTwo.Descendants();
/*
gives ...
0 {TextBox} 	[TextBoxThree]
1 {StackPanel} 	[StackPanelTwo]
2 {TextBox} 	[TextBoxFour]
*/

Each of the extension methods also has a corresponding method with a generic type parameter that filters the collection to find elements of a specific type:

// all items within the visual tree of 'GridTwo' that are textboxes
var textBoxDescendantsOfGridTwo = GridTwo.Descendants()
                                         .Where(i => i is TextBox);
/*
0 {TextBox} 	[TextBoxThree]
1 {TextBox} 	[TextBoxFour]
*/
 
// a shorthand using the generic version of Descendants
var textBoxDescendantsOfGridTwo2 = GridTwo.Descendants<TextBox>();
/*
0 {TextBox} 	[TextBoxThree]
1 {TextBox} 	[TextBoxFour]
*/

Elements

The elements extension method obtains all the direct children of an item in the visual tree:

// find all direct children of this user control 
var userControlChildren = this.Elements();
/*
0 {Grid} 	[GridOne]
*/
 
// find all direct children of the grid 'GridTwo'
var gridChildren = GridTwo.Elements();
/*
0 {TextBox} 	[TextBoxThree]
1 {StackPanel} 	[StackPanelTwo]
*/
 
// find all direct children of the grid 'GridTwo' that are StackPanels
var gridChildren2 = GridTwo.Elements<StackPanel>();
/*
0 {StackPanel} 	[StackPanelTwo]
*/

There are also, ElementsBeforeSelf and ElementsAfterSelf methods that return the elements before and after the item which the method is being invoked upon.

Ancestors

The ancestors methods traverse the tree towards the root, finding all the ancestors:

// the ancestors for 'TextBoxFour'
var ancestors = TextBoxFour.Ancestors();
/*
0 {StackPanel} 	[StackPanelTwo]
1 {Grid} 	[GridTwo]
2 {StackPanel} 	[StackPanelOne]
3 {Grid} 	[GridOne]
4 {MainPage} 	[]
*/
 
// the ancestors for 'TextBoxFour' that are StackPanels
var stackPanelAncestors = TextBoxFour.Ancestors<StackPanel>();
/*
0 {StackPanel} 	[StackPanelTwo]
1 {StackPanel} 	[StackPanelOne]
*/

Putting it all together

The Linq to Tree API not only defines extension methods on DependencyObject, but also the same extension methods on IEnumerable<DependencyObject>. Unless you have previous experience of Linq to XML, I would strongly suggest reading my codeproject article to understand how this works!

This allows you to form much more complex queries. For example, you can find all TextBoxs that have a Grid as a direct parent:

var itemsFluent = this.Descendants<TextBox>()
                      .Where(i => i.Ancestors().FirstOrDefault() is Grid);
 
var itemsQuery = from v in this.Descendants<TextBox>()
                 where v.Ancestors().FirstOrDefault() is Grid
                 select v;
/*
0 {TextBox} 	[TextBoxOne]
1 {TextBox} 	[TextBoxThree]
*/

Here, you can also see we are mixing the fluent and query syntax for Linq. Both give the same result.

The next example finds all StackPanels that are within another StackPanels visual tree:

var items2Fluent = this.Descendants<StackPanel>()
                              .Descendants<StackPanel>();
 
var items2Query = from i in
                      (from v in this.Descendants<StackPanel>()
                       select v).Descendants<StackPanel>()
                  select i;
/*
0 {StackPanel} 	[StackPanelTwo]
*/

Finally, this one-liner, outputs the entire visual tree in ASCII! It makes use of the DescendantsAndSelf, Ancestors and ElementsBeforeSelf methods, plus the funky Linq Aggregate method.

string tree = this.DescendantsAndSelf().Aggregate("",
    (bc, n) => bc + n.Ancestors().Aggregate("", (ac, m) => (m.ElementsAfterSelf().Any() ? "| " : "  ") + ac,
    ac => ac + (n.ElementsAfterSelf().Any() ? "+-" : "\\-")) + n.GetType().Name + "\n");

\-MainPage
  \-Grid
    +-TextBox
    | \-Grid
    |   +-Border
    |   | \-Grid
    |   |   +-Border
    |   |   \-Border
    |   |     \-ScrollViewer
    |   |       \-Border
    |   |         \-Grid
    |   |           +-ScrollContentPresenter
    |   |           | \-TextBoxView
    |   |           +-Rectangle
    |   |           +-ScrollBar
    |   |           \-ScrollBar
    |   +-Border
    |   +-Border
    |   \-Border
    |     \-Grid
    |       +-Path
    |       \-Path
    \-StackPanel
      +-TextBox
      | \-Grid
      |   +-Border
      |   | \-Grid
      |   |   +-Border
      |   |   \-Border
      |   |     \-ScrollViewer
      |   |       \-Border
      |   |         \-Grid
      |   |           +-ScrollContentPresenter
      |   |           | \-TextBoxView
      |   |           +-Rectangle
      |   |           +-ScrollBar
      |   |           \-ScrollBar
      |   +-Border
      |   +-Border
      |   \-Border
      |     \-Grid
      |       +-Path
      |       \-Path
      \-Grid
        +-TextBox
        | \-Grid
        |   +-Border
        |   | \-Grid
        |   |   +-Border
        |   |   \-Border
        |   |     \-ScrollViewer
        |   |       \-Border
        |   |         \-Grid
        |   |           +-ScrollContentPresenter
        |   |           | \-TextBoxView
        |   |           +-Rectangle
        |   |           +-ScrollBar
        |   |           \-ScrollBar
        |   +-Border
        |   +-Border
        |   \-Border
        |     \-Grid
        |       +-Path
        |       \-Path
        \-StackPanel
          \-TextBox
            \-Grid
              +-Border
              | \-Grid
              |   +-Border
              |   \-Border
              |     \-ScrollViewer
              |       \-Border
              |         \-Grid
              |           +-ScrollContentPresenter
              |           | \-TextBoxView
              |           +-Rectangle
              |           +-ScrollBar
              |           \-ScrollBar
              +-Border
              +-Border
              \-Border
                \-Grid
                  +-Path
                  \-Path

Note: this was invoked after the LayoutUpdated event so that we not only see the elements from our XAML, but also the elements created from their templates, giving us our full run-time visual tree.

You can download a simple Silverlight application that demonstrated all the examples given above:
LinqToTree.zip.

Or, if you just want the Linq to VisualTree code, you can copy-n-paste from the windows below which has the entire API, which includes the methods illustrated above,, plus their IEnumerable equivalents, and a few others I have not illustrated.

using System;
using System.Linq;
using System.Collections.Generic;
using System.Windows;
using System.Windows.Media;

namespace LinqToVisualTree
{
    /// 

    /// Adapts a DependencyObject to provide methods required for generate
    /// a Linq To Tree API
    /// 
    public class VisualTreeAdapter : ILinqTree
    {
        private DependencyObject _item;

        public VisualTreeAdapter(DependencyObject item)
        {
            _item = item;
        }

        public IEnumerable Children()
        {
            int childrenCount = VisualTreeHelper.GetChildrenCount(_item);
            for (int i = 0; i < childrenCount; i++)
            {
                yield return VisualTreeHelper.GetChild(_item, i);
            }
        }

        public DependencyObject Parent
        {
            get
            {
                return VisualTreeHelper.GetParent(_item);
            }
        }
    }
}

namespace LinqToVisualTree
{
    /// 

    /// Defines an interface that must be implemented to generate the LinqToTree methods
    /// 
    /// 
    public interface ILinqTree
    {
        IEnumerable Children();

        T Parent { get; }
    }

    public static class TreeExtensions
    {
        /// 

        /// Returns a collection of descendant elements.
        /// 
        public static IEnumerable Descendants(this DependencyObject item)
        {
            ILinqTree adapter = new VisualTreeAdapter(item);
            foreach (var child in adapter.Children())
            {
                yield return child;

                foreach (var grandChild in child.Descendants())
                {
                    yield return grandChild;
                }
            }
        }

        /// 

        /// Returns a collection containing this element and all descendant elements.
        /// 
        public static IEnumerable DescendantsAndSelf(this DependencyObject item)
        {
            yield return item;

            foreach (var child in item.Descendants())
            {
                yield return child;
            }
        }

        /// 

        /// Returns a collection of ancestor elements.
        /// 
        public static IEnumerable Ancestors(this DependencyObject item)
        {
            ILinqTree adapter = new VisualTreeAdapter(item);

            var parent = adapter.Parent;
            while (parent != null)
            {
                yield return parent;
                adapter = new VisualTreeAdapter(parent);
                parent = adapter.Parent;
            }
        }

        /// 

        /// Returns a collection containing this element and all ancestor elements.
        /// 
        public static IEnumerable AncestorsAndSelf(this DependencyObject item)
        {
            yield return item;

            foreach (var ancestor in item.Ancestors())
            {
                yield return ancestor;
            }
        }

        /// 

        /// Returns a collection of child elements.
        /// 
        public static IEnumerable Elements(this DependencyObject item)
        {
            ILinqTree adapter = new VisualTreeAdapter(item);
            foreach (var child in adapter.Children())
            {
                yield return child;
            }
        }

        /// 

        /// Returns a collection of the sibling elements before this node, in document order.
        /// 
        public static IEnumerable ElementsBeforeSelf(this DependencyObject item)
        {
            if (item.Ancestors().FirstOrDefault() == null)
                yield break;
            foreach (var child in item.Ancestors().First().Elements())
            {
                if (child.Equals(item))
                    break;
                yield return child;
            }
        }

        /// 

        /// Returns a collection of the after elements after this node, in document order.
        /// 
        public static IEnumerable ElementsAfterSelf(this DependencyObject item)
        {
            if (item.Ancestors().FirstOrDefault() == null)
                yield break;
            bool afterSelf = false;
            foreach (var child in item.Ancestors().First().Elements())
            {
                if (afterSelf)
                    yield return child;

                if (child.Equals(item))
                    afterSelf = true;
            }
        }

        /// 

        /// Returns a collection containing this element and all child elements.
        /// 
        public static IEnumerable ElementsAndSelf(this DependencyObject item)
        {
            yield return item;

            foreach (var child in item.Elements())
            {
                yield return child;
            }
        }

        /// 

        /// Returns a collection of descendant elements which match the given type.
        /// 
        public static IEnumerable Descendants(this DependencyObject item)
        {
            return item.Descendants().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection of the sibling elements before this node, in document order
        /// which match the given type.
        /// 
        public static IEnumerable ElementsBeforeSelf(this DependencyObject item)
        {
            return item.ElementsBeforeSelf().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection of the after elements after this node, in document order
        /// which match the given type.
        /// 
        public static IEnumerable ElementsAfterSelf(this DependencyObject item)
        {
            return item.ElementsAfterSelf().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection containing this element and all descendant elements
        /// which match the given type.
        /// 
        public static IEnumerable DescendantsAndSelf(this DependencyObject item)
        {
            return item.DescendantsAndSelf().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection of ancestor elements which match the given type.
        /// 
        public static IEnumerable Ancestors(this DependencyObject item)
        {
            return item.Ancestors().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection containing this element and all ancestor elements
        /// which match the given type.
        /// 
        public static IEnumerable AncestorsAndSelf(this DependencyObject item)
        {
            return item.AncestorsAndSelf().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection of child elements which match the given type.
        /// 
        public static IEnumerable Elements(this DependencyObject item)
        {
            return item.Elements().Where(i => i is T).Cast();
        }

        /// 

        /// Returns a collection containing this element and all child elements.
        /// which match the given type.
        /// 
        public static IEnumerable ElementsAndSelf(this DependencyObject item)
        {
            return item.ElementsAndSelf().Where(i => i is T).Cast();
        }

    }

    public static class EnumerableTreeExtensions
    {
        /// 

        /// Applies the given function to each of the items in the supplied
        /// IEnumerable.
        /// 
        private static IEnumerable DrillDown(this IEnumerable items,
            Func> function)
        {
            foreach (var item in items)
            {
                foreach (var itemChild in function(item))
                {
                    yield return itemChild;
                }
            }
        }

        /// 

        /// Applies the given function to each of the items in the supplied
        /// IEnumerable, which match the given type.
        /// 
        public static IEnumerable DrillDown(this IEnumerable items,
            Func> function)
            where T : DependencyObject
        {
            foreach (var item in items)
            {
                foreach (var itemChild in function(item))
                {
                    if (itemChild is T)
                    {
                        yield return (T)itemChild;
                    }
                }
            }
        }

        /// 

        /// Returns a collection of descendant elements.
        /// 
        public static IEnumerable Descendants(this IEnumerable items)
        {
            return items.DrillDown(i => i.Descendants());
        }

        /// 

        /// Returns a collection containing this element and all descendant elements.
        /// 
        public static IEnumerable DescendantsAndSelf(this IEnumerable items)
        {
            return items.DrillDown(i => i.DescendantsAndSelf());
        }

        /// 

        /// Returns a collection of ancestor elements.
        /// 
        public static IEnumerable Ancestors(this IEnumerable items)
        {
            return items.DrillDown(i => i.Ancestors());
        }

        /// 

        /// Returns a collection containing this element and all ancestor elements.
        /// 
        public static IEnumerable AncestorsAndSelf(this IEnumerable items)
        {
            return items.DrillDown(i => i.AncestorsAndSelf());
        }

        /// 

        /// Returns a collection of child elements.
        /// 
        public static IEnumerable Elements(this IEnumerable items)
        {
            return items.DrillDown(i => i.Elements());
        }

        /// 

        /// Returns a collection containing this element and all child elements.
        /// 
        public static IEnumerable ElementsAndSelf(this IEnumerable items)
        {
            return items.DrillDown(i => i.ElementsAndSelf());
        }

        /// 

        /// Returns a collection of descendant elements which match the given type.
        /// 
        public static IEnumerable Descendants(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.Descendants());
        }

        /// 

        /// Returns a collection containing this element and all descendant elements.
        /// which match the given type.
        /// 
        public static IEnumerable DescendantsAndSelf(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.DescendantsAndSelf());
        }

        /// 

        /// Returns a collection of ancestor elements which match the given type.
        /// 
        public static IEnumerable Ancestors(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.Ancestors());
        }

        /// 

        /// Returns a collection containing this element and all ancestor elements.
        /// which match the given type.
        /// 
        public static IEnumerable AncestorsAndSelf(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.AncestorsAndSelf());
        }

        /// 

        /// Returns a collection of child elements which match the given type.
        /// 
        public static IEnumerable Elements(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.Elements());
        }

        /// 

        /// Returns a collection containing this element and all child elements.
        /// which match the given type.
        /// 
        public static IEnumerable ElementsAndSelf(this IEnumerable items)
            where T : DependencyObject
        {
            return items.DrillDown(i => i.ElementsAndSelf());
        }
    }
}

Regards, Colin E.

I think the concept of managed memory, where the cleanup of unused objects from the heap is performed by a garbage collector, is a fantastic idea. It means that developers working with Java or C# (or other CLR languages) can often forget all about memory allocation, concentrating on more interesting tasks. However, whereas in the Java language the concept of memory leaks has almost completely vanished, they unfortunately rear their ugly head all to often when developing applications for the Microsoft Common Language Runtime (CLR). This is almost entirely down to one thing, events.

The problem with events is that they form a strong reference (i.e. a link between two object instances that prohibits garbage collection) in a manner that is not immediately obvious due to the syntax that event subscription uses. When subscribing to an event, the following syntax is used:

// subscribe to an event
textBox.TextChanged += new EventHandler(TextBox_TextChanged);
// or
textBox.TextChanged += TextBox_TextChanged;
 
// remove the subscription
textBox.TextChanged -= new EventHandler(TextBox_TextChanged);
// or
textBox.TextChanged -= TextBox_TextChanged;

Note the second examples use the shortened syntax where the delegate instance, EventHandler, is not explicitly constructed but added by the compiler, there is no difference semantically.

By subscribing to an event the following relationships are constructed, with the direction of reference as indicated:

As you can see from the above, adding an handler to an event creates a strong reference from source to listener. This does not cause significant problems if the listener has a shorter or the same lifecycle that the source, as is the case where you add event handlers for controls in a Windows Form for example. However, if the source has a longer lifecycle than the listener, and the listener fails to remove its event subscription, a memory leak will occur.

In practice this kind of problem often occurs in modular applications where there exists some sort of Shell or Environment that raises events or acts as a mediator. Within this hosting environment their exists numerous loosely coupled modules which by necessity have a shorter lifecycle than their container. If a module subscribes to events from the Shell but fails to unsubscribe at the end of its life a memory leak occurs. In complex systems this happens surprisingly often!

A common solution to this problem is to use Weak Events. These are events where either the reference between source and listener is weak, meaning that listener can be garbage collected if it is only referenced via this event handler. An early implementation of this pattern was developed for WPF and is provided by Greg Schechter on his blog. There is also an excellent codeproject article by Daniel Grunwald which details many different approaches to creating weak event listeners and sources.

However, whilst liberally sprinkling weak events about your code will solve potential memory leaks, it should not be used as a replacement for proper event subscription clean-up. What if a module’s event handler performs non-trivial logic? Do you really want this to occur after the module is supposed to have been destroyed? So, how do you ensure that events are being unsubscribed? This can be done by code-review, or by making use of heap profiling tools such as dotTrace. With these tools you can run your application, create and destroy modules, garbage collect then inspect your heap to see that the modules and their related objects have been removed. If not, you can trace the object references to find the offending event handler. However, this is a time consuming process.

An alternative is to make use of the fact that events are simply delegates with a highly restrictive interface applied. However, within the class where the event is defined you can treat it as an delegate, allowing you to inspect its invocation list to find all the event handlers. Therefore, if you have a long-lived service component that raises events which are handled by shorter lived modules, you can check the event invocation list after all the modules have been destroyed at application shutdown to ensure that all modules have correctly unsubscribed. This can be achieved as follows:

/// <summary>
/// An event which indicates that some stock price has changed
/// </summary>
public event EventHandler<StockPriceUpdateEventArgs> StockPriceUpdate;
 
public void Dispose(object sender, EventArgs e)
{
    // check that when this service is destroyed, that there are no event
    // subscriptions
    CheckEventHasNoSubscribers(StockPriceUpdate);
}
 
/// <summary>
/// Detects whether an event has any subscribers
/// </summary>
[Conditional("DEBUG")]
private void CheckEventHasNoSubscribers(Delegate eventDelegate)
{
    if (eventDelegate != null)
    {
        // if the event has any subscribers, create an informative error message.
        if (eventDelegate.GetInvocationList().Length != 0)
        {
            int subscriberCount = eventDelegate.GetInvocationList().Length;
 
            // determine the consumers of this event
            StringBuilder subscribers = new StringBuilder();
            foreach (Delegate del in eventDelegate.GetInvocationList())
            {
                subscribers.Append((subscribers.Length != 0 ? ", " : "") + del.Target.ToString());
            }
 
            // name and shame them!
            Debug.WriteLine(string.Format("Event: {0} still has {1} subscribers, with the following targets [{2}]",
                eventDelegate.Method.Name, subscriberCount, subscribers.ToString()));
        }
    }
}

Here we have an event which our service raises, on disposal we check that the event has no subscribers. If any modules have failed to cleanup properly we see the following message:

Event: PriceService_StockPriceUpdate still has 2 subscribers, with the following targets:
[MemoryLeakExamples.StockPriceViewer, Text: StockPriceViewer,
MemoryLeakExamples.StockPriceViewer, Text: StockPriceViewer]

This message tells us how many classes still have referenced event handlers, and their type. This should make it very easy to pinpoint the memory leak.

The sourcecode for this blog post includes a very simple WinForms application where a Stock Price service raises events that are handled by simple UI modules that can be created and destroyed by the user. Run it in DEBUG mode to see the memory leaks! Look in StockPriceViewer to see how to fix the problem.

Download the sourcecode: MemoryLeakExample.zip

Regards, Colin E.

The event was titled, “An Introduction to Agile Methodology – Get a Head Start in 2010″, which was suitably broad for my liking! I have experienced heavyweight, formal methodology in the shape of the Rational Unified Process; At Scott Logic, we prefer the Agile way, keeping it light. However, we have a wide range of customers, each with their own preferred development process, so we must exercise further agility by adapting to their way of working. For this reason, I did not want to highlight a single Agile methodology, rather, the common themes held by all Agile methods. My talk was titled “With Agile Development – Communication is the Key”, which pretty much sums up my own opinion on what makes the Agile approach so successful.

To summarise my presentation in a single paragraph; requirements and estimation are probably two of the greatest challenges to any software project. Requirements are hard to capture and as a result most people would agree that it is better to allow them to evolve. Estimation is something that most software engineers would rather avoid! the most common approach is to turn to the most senior member of the team and go with what they suggest. Agile, iterative methodologies promote communication between team members, and between the team and the stakeholders. In this way, the stakeholders are engaged on a regular basis, hastening the evolution of requirements. As the team learns from their previous iterations, their estimations improve. Furthermore, estimation becomes a team exercise, with the team communicating / discussing the tasks. This provides a far superior estimate of effort and team buy-in. In short, the iterative cycles of any iterative methodology promote communication, and it is this which makes them successful. Or, even-shorter, my last slide pretty much captures what I am trying to say:

Following the talks, which included mine, an introduction to Agile Development from Andrew Rumfitt (Perfect Image), and a discussion of the application of Agile techniques to non software projects by Colin Ashurst (Durham Business School), everyone played the “Ball Point Game”. This is a simple training exercise where small teams are formed, the goal being to pass as many balls between the team members as possible in two minutes. However, the added twist is that before each 2 minutes “iteration” the team has to estimate how many balls they will pass. Interestingly, most teams got better with each iteration, passing more balls, but more importantly as they found their natural velocity, their estimates improved.

The Q&A session followed the game, with myself and Andrew fielding questions. This was certainly an interesting challenge! A few of my favourite questions were as follows:

“If you allow requirements to change, what about the project budget? I cannot see how this would be workable?”

Great question! and not an easy one to answer. The problem with projects where the price and requirements are determined up-front is that one of the two parties involved is probably going to be unhappy with the end result. If the requirements are set-in-stone, forming an immovable contract, their natural evolution is stifled. The customer may get exactly what the described at the start of the project, however, it is probably not what they want now after they have used the software and had a chance to think more deeply about the problem. With an Agile, iterative approach, the requirements change and improve as do the estimates. And, as a result, the original budget (or scope) may prove to be unworkable. However, this extra information empowers the customer, allowing them to make a decision about the financial impact of this change in their own requirements. Although, this will only work if their is good communication and trust. To quote the Agile Manifesto, “Customer collaboration over contract negotiation”.

“I am sure that you getter better at estimating after a few projects, however, if you start a project with an Agile approach, and a few days into an iteration you find that your estimates are totally wrong, do you stop the iteration and start out again?”

This question certainly reveals the fear that a lot of software engineers have of estimation! One important point to note is that the person asking the question indicates that they think you need to run a number of agile projects before you get good at estimating. I think this is a flawed assumption. Every software project is different, and there is a certain amount of learning involved in every project. You learn about the problem being solved and about the capabilities, or velocity, of your team. The “Ball Point Game” we played earlier illustrates this point quite nicely, every team, after a few iterations, started to give very accurate estimate. With short iterations the feedback is very rapid, as a result you do not need a lot of experience from previous projects to start giving accurate estimates.

Regarding the question itself, should you abort the sprint? I don’t think so. If everyone (developers, and stakeholders) has bought into the agile approach, what some might see as a failure should be seen as a valuable lesson. Continue with the sprint, maximise the learning. Do not abort the sprint and try to sweep it under the carpet! This does not promote openness. Besides, can you be sure that your revised estimate after just a few days is better? Have you learnt enough?

“Do you need the right tools and the right people for Agile development?”

Personally I would always value people over tools, regardless of the process or project. Agile development should be quite lightweight. You can probably get away with just a simple spreadsheet, or a few post-it notes!

One problem with an approach that favours communication is that software engineers are not always the most communicative. A recent codeproject survey confirmed the commonly held belief that most software engineers are introverts (the INTJ personality type was the most common). Whilst it is certainly not necessary that all team members are extroverts, I think it does help if individuals occupying certain key roles, where they facilitate communicati0n (such as the ScrumMaster in the SRUM methodology), are comfortable communicators.

A few questions followed about how you find the right kind of people. At Scott logic, our interviews concentrate far more on people’s problem solving abilities and how they tackle problems than on just writing code.

After these, and other challenging questions, we have a well earned beer and some food …

An interesting and enjoyable events.

Regards, Colin E.

The mix10k challenge, where you are given 10k to create a Silverlight / HTML5 application, has been on my mind for a while … the Christmas vacation was the perfect excuse to put together my entry. Inspired by the demo’s of the Amiga era, here is my mix10k old-skool demo entry:

You can view my entry (and more importantly vote for it!) at the mix10k gallery.

For me the greatest challenge was the music, not least for the fact that I am no good at writing the stuff myself! The mix10k rules are fairly relaxed, you can download content, such as images and music, allowing you to circumnavigate the 10k limit. However, I thought it would be a more interesting challenge to pack everything into 10k.

Size-limited demos typically use synthesis rather than sampling for the musics. Unfortunately Silverlight isn’t really geared towards sound synthesis, however, a few people have managed to create synthesized media sources. I decided to go the sampling route and use an MP3 track for my music, the idea being to use a short, highly compressed MP3 track and loop it. I posted a message on the 8bitcollective forums and within a day had a funky, and highly compressed, 4 second MP3 clip for my demo. The MP3 filesize of 4,248 bytes left me 5,992 to play with.

However, the next challenge was looping the audio, which is unfortunately something that is not supported out-of-the-box. For longer media clips it is acceptable to catch the MediaEnded event, re-wind and re-start. However, there is a delay of about half a second, which with a 4 second looped audio clip is pretty disruptive! Fortunately Silverlight does expose a lower level API for media via the abstract class MediaStreamSource, which has virtual methods which you can override to supply audio direct from a stream. The codeplex project ManagedMediaHelpers is a great place to start for information on how to load MP3 audio from a file. Typically the code to read and stream an MP3 file is over 10k, however, I was able to trim this down to roughly 1k by removing all the code for parsing the MP3 headers, replacing it with the hard-coded values for my specific MP3 file.

Unfortunately, when the audio was looped, it just didn’t sound right. The trouble is, within MP3 files, the audio is split into frames. Using a useful utility from codeproject, I found that my MP3 file consisted of 56 frames of ~67ms in length. However, the audio loop was not a multiple of 67ms. To rectify this, I downloaded a copy of the free audio editor Audacity, and ’stretched’ the track to fit into an integer number of frames, and re-encoded in MP3 format via lame. Finally, I had looping MP3 audio! (and 4,958 bytes left)

After the battle I had with the audio, constructing the visuals, the scrolling text, interference patterns, plasma and starfield was a piece of cake!

Here are a few quick tips for keeping your filesize under 10k.

• Use the var keyword wherever possible
   
• Use the using for aliasing frequently used types

  For example:

  using K = System.Windows.Media.MediaSourceAttributesKeys;
  using J= System.Windows.Media.MediaStreamAttributeKeys;
  using M = System.Math;

  With type names as long as MediaSourceAttributesKeys, you only need to use them three times for a using alias to make sense!

• chain initialisation of variables

  The following code:

  int x1 = sin1(8) + 200
  int y1 = sin1(3) * 10 + 100
  int g = (tx - x1) * (tx - x1);

  Can be shortened to:

  int x1 = sin1(8) + 200, y1 = sin1(3) * 10 + 100, g = (tx - x1) * (tx - x1);

• Tabs not spaces

  If you are not to bothered about squeezing the most out of those 10k bytes and want your code to look pretty, at least save yourself a bit of room by using Tabs!

• Choose wisely between XAML and C#

  One big advantage that XAML has over C# is its use of value converters, compare the following XAML:

  <Ellipse Fill="White"/>

  to its equivalent in C#:

  var el = new Ellipse()
  {
     Fill = new SolidColorBrush(Color.FromArgb(255,255,255,255))
  };

  In this case, XAML is a clear winner.

• Use lambda expressions for event handlers

  You can achieve great savings by including your event handling logic within anonymous delegates (via lambda expressions), it also means that you can move variables from class scope (i.e. files) to method scope and use the var keyword. Compare:

  WriteableBitmap bitmap = new WriteableBitmap(200,200);
   
  MyPage()
  {
  	var t = new DispatcherTimer();
  	t.Tick +=new EventHandler(t_Tick);
  }
   
  void  t_Tick(object sender, EventArgs e)
  {
  	// do stuff with the bitmap
  	bitmap.Invalidate();
  }

  to:

  MyPage()
  {
  	var bitmap = new WriteableBitmap(200,200);
  	var t = new DispatcherTimer();
  	t.Tick += (s,e) =>
  		{
  			// do stuff with the bitmap
  			bitmap.Invalidate();
  		};
  }

• Ditch that App!

  The App.xaml and App.xaml.cs full of template code that you do not need. Simply ditch these files entirely and replace with a minimal application that indicates the page to display:

  public class A : Application
  {
      public A()
      {
          Startup += (i, j) => RootVisual = new MainPage();
      }
  }

• Minify your code

  Perhaps the greatest savings can be achieved by ‘minifying’ your code, i.e. removing any extra whitespace and reducing variable, method, types names to single characters. The problem is that in doing so, your code will become unreadable!

  In order to minify my code yet maintain readability I created a very simple console application that read my application source and performed a bunch of string replacements in order to reduce its size …

  static Dictionary<string, string> reps = new Dictionary<string, string>()
  {
    {System.Environment.NewLine, ""}, 
    {"  ", " "},
    {" = ", "="},
    {" == ", "=="},
    {" * ", "*"},
    {" > ", ">"},
    {" < ", "<"},
    ...
   
    // methods 
    {"Timer_Tick", "T"},
    {"CreateLine", "g"},
    {"DrawLines", "ln"},
    ...
   
    // xaml
    {"grid", "q"},
    {"image", "o"},
    {"canvas", "c"},
    {"MainPage", "Y"},
    ...
  };

  The output of this console application is sent to another project which builds my final entry. The result of this was to make my reasonably compact code 47% smaller!

You can download my thoroughly messy and unreadable code: mix10ksrc.zip

And vote for it on the mix10k site (Did I say that already?)

Regards, Colin E.

I was sad to hear the news earlier this year that Yahoo! was pulling the plug on GeoCities. Somewhere buried deep within GeoCities is the first web page I ever created, complete with “Under Construction” banner, animated GIFs, guestbooks, and nasty background music. Unfortunately, I have no idea what the URL for that page was, and this is long before Google ran my life!

This blog post is a tribute to one of the many dynamic effects that were popular in the 90’s, animated reflections. These Java applets were quite popular for a while, but have gone the same way as GeoCities. Perhaps it is time for a revival?

The Silverlight application shown above contains a UserControl which renders an animated reflection of a referenced FrameworkElement.

The code which produces the ripple is very simple, A DispatcherTimer increments _time and redraws the reflection. The reflection itself is achieved by constructing a WriteableBitmap from the referenced element, allowing us to grab its pixel values. Another WriteableBitmap is constructed for the reflection image, and rows of pixels are copied across with a suitable Y offset to produce the ripple effect:

private double _time = 0.0;
 
private void Timer_Tick(object sender, EventArgs e)
{
    // increment phi and update the reflection
    _time += 0.4;
    UpdateReflection();
}
 
/// <summary>
/// Copies an inverted image of the referenced FrameworkElement
/// with a 'ripple' effect
/// </summary>
private void UpdateReflection()
{
    FrameworkElement reflectedFE = ReflectedElement as FrameworkElement;
 
    if (reflectedFE == null)
        return;
 
    // synchronize the element width
    Width = reflectedFE.ActualWidth;
 
    // copy the source into a writeable bitmap
    WriteableBitmap sourceBitmap = new WriteableBitmap(reflectedFE, null);
 
    // create a target which is the same width / height as the reflection element
    WriteableBitmap targetBitmap = new WriteableBitmap((int)ActualWidth, (int)ActualHeight);
 
    // copy the reflection
    for (int y = 0; y < targetBitmap.PixelHeight; y++)
    {
        double amplitude = ComputeAmplitude(y, targetBitmap.PixelHeight);
        double sinusoid = ComputeRipple(y, targetBitmap.PixelHeight, _time);
 
        // the offset to the y value index caused by the ripple
        int yOffset = (int)(sinusoid * amplitude);
 
        // compute the Y position of the line to copy from the source image
        int sourceYLocation = sourceBitmap.PixelHeight - 1 -
            ((y + yOffset) * sourceBitmap.PixelHeight) / targetBitmap.PixelHeight;
 
        // check that this value is in range
        sourceYLocation = Math.Min(sourceBitmap.PixelHeight - 1, Math.Max(0, sourceYLocation));
 
        // copy the required row
        int sourceIndex = sourceYLocation * sourceBitmap.PixelWidth;
        int targetIndex = y * targetBitmap.PixelWidth;
        for (int i = 0; i < targetBitmap.PixelWidth; i++)
        {
            targetBitmap.Pixels[targetIndex++] = sourceBitmap.Pixels[sourceIndex++];
        }                
    }
 
    targetBitmap.Invalidate();
 
    LayoutRoot.Source = targetBitmap;
}
 
/// <summary>
/// Compute the amplitude of the oscillations at a given Y position
/// </summary>
private double ComputeAmplitude(int y, int height)
{
    // our amplitude range is 1 to 3
    return ((double)y * 2) / (double)height + 1.0;
}
 
/// <summary>
/// Compute the sinusoid applied to teh image at the given location
/// </summary>
private double ComputeRipple(int y, int height, double time)
{
    // provide a ripple that is the combination of two out of phase sine waves
    double phaseFactor = (double)y / (double)height;
    return Math.Sin(time + phaseFactor * 16) + Math.Sin(time + phaseFactor * 30)
        + Math.Sin(time + phaseFactor * 62);
}

The XAML for this user control is simply an image with an opacity gradient:

<UserControl x:Class="SilverlightShimmer.ReflectionControl"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    Width="300" Height="300">
    <Image x:Name="LayoutRoot">
        <Image.OpacityMask>
            <LinearGradientBrush EndPoint="0.5,1" StartPoint="0.5,0">
                <GradientStop Color="#FF000000" Offset="0"/>
                <GradientStop Color="#00000000" Offset="1"/>
            </LinearGradientBrush>
        </Image.OpacityMask>
    </Image>
</UserControl>

This control is associated with our Christmas-ey image as follows:

<UserControl x:Class="SilverlightShimmer.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:SilverlightShimmer" Width="320" Height="260">
 
    <Grid Background="Black">
 
        <StackPanel Orientation="Vertical" Margin="10">            
            <Border  x:Name="controlToReflect" BorderThickness="5" BorderBrush="LightGray"
                     CornerRadius="3" HorizontalAlignment="Center">
                <Image  Source="christmas.jpg" Margin="3"
                        Stretch="None"/>
            </Border>
            <local:ReflectionControl x:Name="shimmer" Height="80" Margin="3"
                       ReflectedElement="{Binding ElementName=controlToReflect}"/>
        </StackPanel>
    </Grid>
</UserControl>

One interesting point here is the way in which the Border and Image are associated with the ReflectionControl. The ReflectedElement property is bound to the Border via an ElementName binding, however this binding has no Path. Therefore, rather than binding to a property of the referenced element, the ReflectedElement is bound to the element itself. Hence, no need for any code behind to associated the ReflectionControl with the element(s) to render.

This control can be used to render a reflection of anything (even a reflection of a reflection if you so wish). Here is a more complex example:

You can download the full sourcecode here: SilverlightShimmer.zip

Regards, Colin E.