By Stephen Hampshire, Client Manager, TLF Research
Management Dashboards are a standard feature of business information systems. The metaphor is that a good dashboard will allow managers to "drive" the business forward safely, able to see key metrics at a glance without taking their eyes off the road ahead for long. Sounds great, but how does this metaphor hold up in practice, and how do we go about designing a dashboard that is up to the job?
Measure the right things
What data should form the basis of the dashboard? If you have adopted a model such as the Balanced Scorecard then the answer flows from that—and many organisations use the term "scorecard" instead of dashboard as a result. Actually that's probably a slightly better metaphor…but more on that later. For other businesses picking the measures is more difficult.
One potential danger is that KPIs end up being the things that are easy to measure, rather than the things that really matter. To return to the car analogy, none of the "KPIs" on the dashboard would warn us that we're about to drive over the top of a cliff. In this case the metaphor can be all too close.
It is also important to try to capture a sense of cause and effect in the reporting, where possible. If customer attrition is rising, why is it rising? One of the vital advantages of survey-based measures such as loyalty and satisfaction is that they can be forward-looking and predictive of change, while most KPIs are backward- looking.
As Peter Drucker has commented, the point of these dashboards is to help us think about things. So ask yourself "what are the questions the dashboard is supposed to be helping us answer?" Finally, bear in mind that the things that really matter may not be the same in six months or a year—constantly review the measures you need, and be prepared to drop the ones that no longer matter so much.
Control over measurement
Once you have selected which numbers you want, you need to think about how to get the information. The quality of underlying measurement is vital. It's also essential that you understand the amount of expected variability in measurement for any measures based on a sample (whether surveys or quality-control processes). Only if you understand this can you distinguish a trend or genuine movement from random fluctuations. Failure to discriminate results in knee-jerk reactions to phantom movement.
A much less important issue, but one which is often seen as paramount, is the timing of measurement and the speed of reporting. Which brings me onto my least favourite buzzword—real time measurement. Okay for certain KPIs, but no sample- based measure can be "real time". If you're prepared to spend the money surveys can be very frequent, and very rapidly reported, but they're never going to be real time. Far better to acknowledge this and focus your resources on getting reliable measures that you can trust, even if that doesn't sound quite so 21st century and exciting.
It is good practice to observe some of the data capture for each measure in person. You'll learn a lot about the strengths and weaknesses of the measure, and the quality and type of information it's able to give you. Remember that analysis, however clever, is only as good as the data it is based on:
"THE GOVERNMENT (IS) EXTREMELY FOND OF AMASSING GREAT QUANTITIES OF STATISTICS. THESE ARE RAISED TO THE nth DEGREE, THE CUBE ROOTS ARE EXTRACTED, AND THE RESULTS ARE ARRANGED INTO ELABORATE AND IMPRESSIVE DISPLAYS. WHAT MUST BE KEPT EVER IN MIND, HOWEVER, IS THAT IN EVERY CASE, THE FIGURES ARE FIRST PUT DOWN BY A VILLAGE WATCHMAN, AND HE PUTS DOWN ANYTHING HE DAMN WELL PLEASES."
Sir Josiah Stamp
Why is so much data poorly collected? Because, to the person doing the collecting, it seems like a bureaucratic waste of time. Which it often is. Collecting too much data, particularly if doing so absorbs resources that could be spent on actually achieving something, is at least as dangerous as not collecting enough. Think about ways to minimise time spent on recording information, streamline the data you collect, and make it as easy as possible to record. The quality of data will improve, and productivity probably will too.
Good practice in design
Keep it simple—the design of any report should be "transparent". That is, any design choices should improve the clarity of the data, not get in the way. Heavyhanded graphic design elements should be avoided (don't let them run with the dashboard metaphor!).
It is normally good practice to avoid showing a single number in isolation. As I'll discuss in the next section, this is something that flows from the "dials" part of the dashboard metaphor. A good dashboard should be capable of showing context, trends, and potential relationships between key measures.
Think about the impact design decisions may have on management decisions with real implications. What is the rationale for targets? How have the boundaries between acceptable and unacceptable performance been arrived at?
We've mentioned looking at the context for a single measure and the relationship between two or more measures. This is vital, but you cannot use the same tool for both jobs. The line chart is an excellent way to show historical context, even on the tiny scale of a Sparkline: 84.2. But a line chart cannot effectively allow you to see the relationship between two variables.
The correct, and only, tool for the job is a scatter plot. Have a look at these two charts (shown below). Both show the same data, but the true nature of the relationship between the two variables (none!) is only apparent on the scatter plot. More worryingly, the human tendency to see patterns where none exist means that many people will look at the line chart and think they can see a relationship.
Careful with the metaphor
It is important not to let the base metaphor run the reporting. In a car, dashboard instruments tend to show very simple, one-dimensional, unrelated measurements. Trigger points for warnings are easily interpretable—we're about to run out of petrol. The minimum point on a dial tends to be 0.
Dials are great for car dashboards, but tend to fail in the case of business measures. All metaphors can be taken too far, and this is one of the times when the metaphor has been allowed to override good sense. There are a few tricky issues, like questions over what to make the minimum and maximum points on the dial. But the killer is that dials instantly strip a measure of its ability to show a trend.
The time context can be vital in making decisions, revealing the history of performance that a single figure cannot give. An example makes this clear. Let's say we have three KPIs, all of which stand at 70 at the moment—what does that mean? The answer is that it depends on the context, so let's show that in the reporting. A line chart does the trick (seen above).
Getting the right information to management so that they are in a position to make quick decisions and react to problems is vital. But letting the metaphor or the process take over can be detrimental to the quality of information being presented, the wisdom of any decisions based on that, and to the resources devoted to data collection in the first place.
Lean data collection coupled with clean reporting of analysis focused on trending, differences and relationships will deliver the data you need to make good decisions. Investing time in getting good measurement of the right things rather than over-designed whiz-bang gimmicks shows commitment to improvement rather than a concern with ticking boxes for the sake of it.