There are only 6 graphs you’ll ever see on a performance report and they’re all rubbish. Here they are.

This is Performance Cat, and she doesn’t like rubbish performance reporting.

And she definitely doesn’t like dumbass explanations for meaningless shapes in graphs…

One of the more adorable features of performance reporting in normal ordinary command and control organisations is the total inability to see signal through noise.
EVERYTHING is packed with meaning, every tiny twitch of a line chart, every minor bulge in a pie-chart, every spurt of a bar chart. Like a spurned lover obsessing over their exes Facebook page, every thing means something.

Take three numbers. Go on. I’ll do it too.

10, 20, 30

There are 6 permutations of these three numbers ( ignoring a number appearing more than once). Here they are in every order they can be in a line chart.

These shapes appear all the time on performance reports, but they are accompanied by something weird, an EXPLANATION. An explanation why these meaningless shapes, made out of pure math, are shaped as they are. They’re only actually shaped by random permutations, not anything to do with life’n’that. These shapes are the only shapes that these numbers can make when you’ve got 3 of them! This is it!

The explanations accompanying these squiggles are often exactly the same.

These are [drumroll] The Performance Monitoring Archetypes!

Name: THE UPWARD TREND
What it looks like:

Commonly Interpreted: “Performance continues to be SKY HIGH! We’ve built on our successes and the trend shows we will improve further into next quarter and beyond”


Name: THE DOWNWARD TREND
What it looks like:

Commonly Interpreted: “Performance has been disappointing. A downward trend of 3 months in a row have lead to a new management team who are currently addressing the root causes of performance and will be rolling out a robust improvement process to ensure a turnaround in the next quarter”


Name: THE TURNAROUND
What it looks like:

Commonly Interpreted: “The dip in performance in month 2 has been turned around by the strong leadership of the new management team, ensure we are going in to next month with a strong trajectory”


Name: THE DOWNTURN
What it looks like:

Commonly Interpreted: “Factors have combined to produce an unfortunate downturn in performance. A peak however in month 2 show we are maintaining our strong performance over the quarter.


Name: THE REBOUND
What it looks like:

 

Commonly Interpreted: “An inexplicable decrease in performance in month 2 has been balanced with a rebound in performance in month 3 to ensure we are entering the next quarter with all our ducks lined up.”


Name: THE SETBACK
What it looks like:

Commonly Interpreted: “There’s a small setback in performance at the end of the quarter in month 3, following the stellar success of month 2, but this remains higher than month one’s outturn.”


So there we are, a description of three dots relations to each other and a made-up explanation as to WHY they are in that order…hey presto, a performance report!

For the true performance connoisseur, what tops off the appearance of these is a trend line. A trendline, provided by Excel, sprinkles these turds with the glitter of a statistical validity that they do not deserve.

eg….

Take the last performance archetype, The Setback. With a simple linear trendline added from excel, you can now interpret these 3 numbers arranged in this particular order as showing that performance is improving, that it IS only a setback, and that next month you can continue to see higher numbers. Cos that line is going up, look.

This is of course total rubbish. However it is total rubbish that is being produced daily, weekly, monthly, yearly, with no end in sight. No special IT will make this better. This is not a technical problem requiring a technical solution. It won’t end with a more expensive and colourful IT application producing these shapes and words.

I don’t think it will end when there is a better understanding of numbers and statistics. Saying “these are only three numbers from which it is impossible to draw conclusions of future performance” is true, but pointless. There is a reason why people provide reasons and justifications for random noise and continue to be “fooled by randomness”, and it isn’t  mathematical ignorance. That is not the cause.

Typical hierarchical command and control organisations runs on a binary view of the world where performance is “good” or “bad”, this cause a binary view of the numbers themselves where numbers are “good” or “bad” rather than numbers that just ARE.

For example, the last known control chart in this organisation was several years ago, I should know cos I did it. It didn’t stand a chance because the environment is not conducive to it’s survival. Like putting a lemur 50 feet below the ocean, not going to work, it’s in the wrong place. For a control chart or similar approach to understanding performance to survive, never mind thrive, requires a curiosity and a wish to find out. What thrives in normal typical command and control organisations are the performance archetypes above, and Performance Cat will remain very annoyed indeed.

Please note the 3-numbers and the names i used for the “performance archetypes” are totally the creation and property of Davis Balistacci who is my new favourite, he came up with the arrangement of 3 dots and given them names, and all I’m doing is adding swearing and knob jokes and cat gifs. Go here for the real deal cos you’ll learn loads, I am. Hopefully he won’t get mad if he sees my cheap popularising. I’m referencing my steals!

Advertisements
This entry was posted in data, information, leadership, questions, setting a numerical target is like..., statistics, systems thinking and tagged , , , , . Bookmark the permalink.

5 Responses to There are only 6 graphs you’ll ever see on a performance report and they’re all rubbish. Here they are.

  1. posiwid4me says:

    I love the powerful description in your phrase “Like putting a lemur 50 feet below the ocean, not going to work, it’s in the wrong place”. I will use that!

    Like

  2. Robert says:

    This is brilliant, thanks for writing and sharing! Lots of people seem to be talking about explanation now and at the risk of crude self-promotion here’s my tuppence: https://robertgrantstats.wordpress.com/2017/03/10/false-hope-of-methodological-improvement/

    Like

    • ThinkPurpose says:

      That was really interesing, the stuff i could understand. me being self-taught in stats and that. I recognised Gerd Gingerenzer! And replication crises! And Bayes!
      I’m coming along, anyway.

      Like

  3. Dave Gresty, MCP says:

    Great thoughts – is creative stats the same as creative accounting?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s