email: ceberhardt@scottlogic.co.uk
on Google+
This blog post presents a thorough analysis of the performance of various WPF Charting components. The results show that a new class of charting solutions, which use raster-based graphics as opposed to retained mode vector graphics, provide a considerable performance advantage.
Introduction
Readers of my blog will know that charting and performance are two things that interest me greatly. One of the first things I did back in 2009 when I started working with WPF and Silverlight was have a go with the Microsoft ‘toolkit’ charts. Unfortunately I found these charts, and the various third party competitors, to be painfully slow when rendering just a few hundred datapoints. As a result of these findings I set about creating a high performance chart for Silverlight and WPF, which eventually became Visiblox charts.
In December 2010 I published a performance comparison that demonstrated that Visiblox was approximately 50-100 times faster than the Toolkit charts and 10 times faster than Visifire charts. At the time the only charts which came close were those developed as part of Microsoft’s now abandoned Dynamic Data Display project. Since publishing these figures, a number of the WPF and Silverlight charting vendors have started to take performance much more seriously, with Visifire more than doubling performance amid claims of being the fastest chart, amCharts have also publishing some very good performance figures as have Telerik.
Much has changed in the year and a half since my original performance blog post and I thought it would be a good idea to repeat the tests. However, this time I wanted to be more thorough, including a few different types of tests and also using WPF instead of Silverlight, due to the number of email requests I have had for a WPF charting comparison.
The Test Suite
In order to allow me to collect performance data for a range of WPF charts across a range of tests, I have written an automated performance test suite. The application uses MEF in order to allow me to easily slot in new tests and new charting providers. If you want a copy of the code, please get in touch.
The suite executes the following tests:
- Line Series Frame Rate – this test initially renders a ‘streaming’ line series which a fixed number of datapoints. New data is added to the chart as quickly as possible, and each time a new point is added to the front and old point is removed in order to keep a fixed number of points in view. The chart axes are auto-scaled (API permitting). The test initially starts with 50 datapoints, with each test run doubling this number.
- Scatter Series Frame Rate – a number of points are added to the chart. The location of these points is then animated in a manner that emulates Brownian motion. The test initially starts with 50 datapoints, with each test run doubling this number.
- Scatter Series Memory Usage – the above test is re-run, but this time rather than measuring render frame rate, the memory consumed as a result of adding the datapoints is recorded.
Initially I used a timer to measure framerate, followed by an attempt to measure via CompositionTarget.Rendering. However, I found that neither method gave an accurate frame rate measure, indeed the WPF team report that the CompositionTarget.Rendering can fire multiple times per frame. Instead, I opted for a more manual measurement using the WpfPerf tool, which makes use of the ETW events emitted by WPF.
Line Series Results
The results of the line series framerate tests are shown below:
NOTE: I am using a log-log scale for the above chart in order to make it easier to see the shape of the various curves.
Regarding which charting vendors I used for comparison, I tried to include all established and well known vendors in my comparison. For those which have already published performance figures or comments relating to Visiblox, I have included their names. For the others I have simply given them a token name ‘Vendor X’, in order to comply with the trial licencing terms which typically prohibit the publishing of performance figures.
One thing worth noting is that some charting vendors use sampling in order to improve framerates. I found that Vendor A (not included above) uses sampling without providing a mechanism for turning it off. Whereas most vendors that do support sampling allow you to turn it on or off and configure the sampling algorithm used. In order to provide a fair comparison, sampling was turned off on all the charts.
The chart above reveals some interesting patterns, due to the two very different classes of chart under test.
Vector based charts – WPF UIs are rendered via retained mode vector graphics. Whilst this approach is what has given rise to the very rich UI framework that WPF applications enjoy, it has been found that this approach just doesn’t perform as well as the bitmap-based approach used by Windows Forms when faced with complex UIs. This has led some people to use WinForms charts within their WPF applications! We can see that the faster vector charts from Microsoft Toolkit, Vendor B, the updated Visifire and Visiblox charts are all close to the limit of what is possible with this style of rendering.
“ … the major ‘partners’ cannot create a WPF chart that renders 2000 floating points of data in less that 5 seconds. I have tried them all and they are slow so we have had to regress to a WinForms version that renders the same data in milliseconds.” – from the channel 9 forums.
Raster based charts – Because of the limits of the WPF rendering system the two fastest charts in this test, Visiblox (via the new raster series type as part of the Ultimate Editition) and SciChart (a newcomer that exclusively uses this approach) both use a raster / bitmap graphics approach, which the WPF and Silverlight frameworks both have support for via the WriteableBitmap class. This approach yields results which are more than 100 times faster than what is possible with vector graphics.
One of the other common complaints made about WPF charts is the initial render time, with charts containing just a few thousands datapoints taking seconds to show on screen. Whilst this test focusses in framerates, it is also gives a good indication of the initial render times, with raster based charts appearing almost instantly when loaded with tens of thousands of points.
Scatter Series Results
The results of the Scatter series tests are shown below:
Again, the same pattern emerges, with the raster based approach yielding a significant increase in frame rate over the vector based.
Scatter Series Memory Usage
The final test looks at the memory consumption of the various charting providers:
The above shows that the memory consumption is directly related to the number of datapoints present in the chart, however, the scale of this relationship is very different for the charts under test. I have a feeling that the Toolkit charts have a memory leak which contributes to the poor performance in this test.
Conclusions
The overall performance of the charting solutions available to WPF (and Silverlight) developers is improving. However, the performance of the WPF rendering system appears to create an upper limit which constrains what can be achieved. The raster-based approach is not subject to these same constraints and as a result provides a considerable improvement in performance.
If you want to have access to the code I used to make these tests, please get in touch. Also, if you are a charting vendor and think you might be Vendor A, B or C in these tests, and wish to be named, let me know.
Really interesting results how the “heavier” charts behave so similarly. Perhaps they are programmed in the same fashion. I’m clad I came across with this site. I’m looking for a fast chart for my project and this helped me a lot with my decision.
I noticed the first post about Lightning Chart. Are those results for real? Sounds pretty interesting if they are. Are you planning to update your results with any new contestants in the near future?
Yes, the LighningCharts are very fast. I think they are predominantly WinForms charts that have been wrapped up to work for WPF – but the speed is for real.
Updating this article with the latest results is on my long list of things to do!
You guys should know Lightning chart uses DirectX and tests are performed on a pc with high-end graphics card, so requires specific hardware to run at its best. The software-charts (e.g. visiblox, scichart) perform pretty well on a laptop with integrated graphics and also work over remote desktop so end-user hardware configuration is less of an issue.
You are partially true, Anton.
LightningChart offloads a lot of computation to the GPU and fast GPU allows the app run very fast. I just tested the line streaming test with a Intel i5 laptop, with low-end “Intel HD Graphics”, it performed 13.1 M points over 30 FPS. So it’s very efficient also with poor GPUs, compare the result to 25k points of Visiblox test made with probably high-end GPU, it’s still over 500 times faster.
LightningChart really needs a true DirectX hardware to run with. Luckily, almost all computers has that. In the past years, Virtual Operating systems didn’t provide support for DirectX or OpenGL access, but that has changed a lot. VMWare, Parallels and Citrix can run DirectX almost without any performance degradation.
Remote desktops seem to use pretty much some kind an image stream from server to client, and the image stream is the bottleneck. v.5.1 of LC WPF supports Win Vista-8 Remote desktop just fine. So it certainly isn’t slower in RD use than other WPF charts, and it GPU/CPU load settles to very low level.
DirectX is the hard way to do graphics. I think most people haven’t got no idea how much testing we have to do with different GPUs and Virtual operating systems, power saving modes, Windows lock states and so on. And the programming of optimal graphics routines, that’s very laborious.
Yeah I bet it was a lot of work, I’ve worked on OpenGL, DirectX, Managed DirectX, WinForms, WPF etc… so know about the pitfalls of DirectX/OpenGL coding vs. other frameworks.
We did actually test out LightningChart on a notebook with Intel Atom and low-end graphics and in our app only got 6FPS, vs SciChart’s 3-4FPS. The slow speed is because the hardware was really slow (unfortunately this is what the client wanted to roll the app out on) and we were displaying a lot of new data per-frame. Just saying that the lack of a decent GPU does narrow the gap in speed between LightningChart and the software-based charts quite a lot.
With Atom, this maybe exactly true. They are typically using emulated GPUs according to Intel specs, and the graphics actually run on the CPU, which itself is very slow this case.
Colin, thanks for verifying the results.
LightningChart Ultimate is available for WinForms too, but the assemblies are totally different. The property set and object model are pretty much similar, result of many years of development, so migrating from WinForms to WPF is easy. The rendering routines are 80% same. We are not using WindowsFormsHost or anything like that to wrap the WinForms chart inside, the WPF version is build totally free of any Winforms depencies.
Hi Colin,
I’ve sent you the benchmark project with LightningChart Ultimate for WPF fitted. Please update it soon, and document the facts how the are.
In the streaming line benchmark, LightningChart can keep over 30 FPS rate with over 20,000,000 points.
In the scatter chart test, LightningChart can render over 30 FPS with over 200,000 points.
Hi Colin,
could you please send me the source code, and we’ll fit our chart there?
Best regards,
Pasi Tuomainen, Lead software architect at Arction Ltd
Hi Pasi – I’ve just sent you the code.
LightningChart Ultimate for WPF beats all of these. Please include it in your benchmark. Infragistics markets their products with terms they shouldn’t be using. They definitely don’t have the fastest rendering WPF charts. Our primary benchmark vs. Infragistics xam chart shows that where Infragistics chart renders a line chart of 1M points in about 700ms, LightningChartUltimate renders 115 M points in same time.
LightningChartUltimate is based on our revolutionary low-level DirectX rendering technology, not System.Windows.Media graphics. We had to program all graphics routines ourselves and practically calculating everything in triangles before sending them over to the GPU.
I’ll gladly be your primary point of contact here at Arction, let’s get this benchmark updated soon!
Hi Pasi – sounds interesting! I have sent you the code. Let me know how it goes.
Thanks for the article.
Can you please provide the source code.
Did you check SoftwareFX or component one charts also?
Hi Ganesh,
Component One is one of the vendors in the charts for this article. I have not tested SoftwareX.
The sourcecode is on its way to you!
Colin E.
Hey,
I also need to evaluate the charting frameworks for our project. Any chance to get the test code?
Sure – it is on its way!
Hi Colin,
nice article. I’m currently working on a project where i have to display at least 2 charts at once (data points are added every 20ms via a dispatcher timers tick event), and i am facing several performance issues (I am working with visifire charts v5). I wonder if you experienced performance issues, too, when using data binding instead of adding data points to a charts series explicitly?
Can you provide me your code so i can check out other possibilities?
These raster based charts sound very interesting.
Matthias
Hi Matthias,
Yes, there is a considerable overhead to data-binding. Dependency properties alone result in a big performance hit:
http://www.scottlogic.co.uk/blog/colin/2009/06/dependency-property-performance-and-lissajous-figures/
If you want simplicity, and do not have much data, databinding is fine. But for performance scenarios – don’t use them!
I’ll email you my test code.
Colin E.
Hi, thanks for sharing this – really useful.
Don’t suppose one of the anonymous vendors is Infragistics? Their claims are something else: “Render millions of data points within milliseconds with the world’s fastest WPF Data Chart”. (I’m looking specifically at the netadvantage for WPF).
If you haven’t tested their library, would you consider adding it and/or sending me the test code please?
Thank you!
Hi Andy,
I have tested Infragistics in the past – but not recently. It looks like they have updated their charts since I last tried them. At the time of writing this article, only SciChart and Visiblox used the faster ‘raster’ based approach. Vector graphics are never going to match this performance.
I have sent you the test code.
Colin E.
Hi Colin,
I’m evaluating VisiBlox charting for our project.
I will be very grateful if you can share source code of your tests.
Thank you for your work!
Hi Vladimir – I’ve just sent you the code.
Hi Colin,
I am currently evaluating different charting solutions for .net.
May I also have your test suite to run some performance comparisons?
Thanks!
You’ve got mail!
Thanks for this great analysis. I am myself evaluating alternatives to the WPF toolkit chart control and I’d be interested by your source code for this test.
Thanks!
Hi Mathieu, I have just sent you the code.
Great article. Could you please supply me the test suite, as I’m trying to find the best WPF charting solution to handle massive amounts of data.
Thanks…
Hi Ron – no problem, I’ve sent you an email.
Hello!
I am currently evaluating different WPF charting solutions, Visifire, SciChart, DynamicDataDisplay, and Visiblox.
May I also have your test suite to run some performance comparisons?
Thanks!
I’ve just sent you the code. Hope it helps.
Hey Matthew, when are you publishing your evaluation?
Hello Collin,
I am really impressed of your performance results. I currently evaluate charting controls for example from Steema. An advantage for us is that they have in addition a 3D charting control.
Did you already test their 2D charting control?
Please could I also get the source code of your test suite.
Thanks in advance
Stefan
You’ve got mail
I’ve also used Steema to great effect – their WPF charts use the drawing API in System.Windows.Imaging.* and I’ve found them to be one of the best performance-wise so it’d be good to see that in your tests too.
Could you pls let me have the source for these tests too ?
Thanks!
Oops – I didn’t see your message before. I have just sent you the code.
Interesting BMT!
Could you please provide me that source code.
I have just sent you an email
Hi Colin, thank you for sending the test suite code. Unfortunately I lost it – can you please send it again? Thanks, Wolf.
Sure – I’ve just sent it over to you.
Hi Colin,
I am evaluating WPF charting controls for building a dashboard application.
Could you please provide me that source code.
Thanks a lot,
Adrian
Sure – I’ll send it over.
Hi Colin,
I am evaluating Visifire, Visiblox and Scichart.
I am interessting by your sample code.
Can you send me this please ?
thx
Sure – I’ll send you an email with the code.
Colin,
I am evaluating few WPF charting solutions. I require the code for the above test suite in order to do some performance comparisons. Could you please provide the code for the same?
Regards,
Jordan
Hi Jordan, no problem – I’ll send you an email.
Colin,
I am waiting for the test suite. Also, I would like to ask how are you calculating the frame rate by adding one point and removing one point thereby keeping the fixed number of points in view. I tried this in Visiblox, it gave the same frame rate always.
Hi Colin,
I have written my own WPF charts and would be very interested in comparing them to the ones you have tested. How can I do this? Can you send me the code of the test suite?
Thanks,
Wolf
Sure – I’ve sent you an email.
Collin,
Thanks for this very useful post. I am surverying charting packages for Silverlight that have to render linear and scatter plots containing tens of thousands of points. I also need to be able to have fairly granular control of the chart’s look and feel, be able to format tickmarks, labels, precision, etc.
Would you say that the results of your benchmark also apply to Silverlight? Also, does Visiblox address the granularity concerns I mentioned above?
Thanks in advance for the reply!
Cheers,
Victor
Hi Victor,
The Silverlight and WPF rendering systems are very different, due to WPF running on being run on Windows OS, whereas Silverlight runs ‘everywhere’. Jeremiah Morrill provides an excellent summary of these differences in one of his blog posts. As a result, you cannot expect the same code to have the same performance characteristics when run on Silverlight vs. WPF.
However, Silverlight does also support the raster-style graphics that Visiblox Ultimate and SciChart uses, and as a result you will find that the performance is much better than the other altrenatives.
Regarding granular control of the chart’s look and feel, Visiblox is very flexible, allowing you to style/template practically every aspect of the UI.
Colin E.
[...] WPF Charting Performance Comparisons (the Battle Continues) – Colin Eberhardt takes a look at the performance of various WPF based charting solutions, looking at their performance refreshing and animating with different sized data sets. [...]
thanks,
I search how display 50,000 to 100,000 points (with a lot of lineserie and a OHLC serie).
Do you test OHLC or Candelstick serie ?
I am interested by your code source to test this.
thanks
Hi Sylvain,
No – I have not tested OHLC / Candlestick because these two are rather specialised types of chart. However, I would anticipate similar performance to the XY scatters shown in this article. Visiblox has a ‘raster’ OHLC and Candlestick series types so should provide very good performance.
Colin E.
[...] WPF Charting Performance Comparisons (the Battle Continues) (Colin Eberhardt) [...]