gud at maffs

The Civil Service Reform Plan.

There, said it.
Come back! That’s the boring bit over with, those dull words in that boring order. As a reward let’s laugh at it, specifically the bit where Senior Civil Servants get maths all wrong.

This tediously titled but important document is a plan to reform the UK civil service, according to the first of the three forewords (this is a Civil Service document after all) it is supposed to be about:

“Harnessing the world-beating talents of those who work in our Civil Service and making sure they aren’t held back by a system that can be sclerotic and slow. That means learning from the best in the private sector.”

Turns out “learning from the best in the private sector” means “copying from the worst”, by which they mean….staff performance appraisals. There is a masterful take-down of elements of this at Helpgov, I’m not going to re-hash them but am going to kick some other bits until they stop moving.

This bit here. They will:

“identify the top 25% and the bottom 10%. The bottom 10% will need to undertake performance monitoring and improvement planning….
For all staff that remain bottom performers without improvement and are still not meeting the required standards, a decision will quickly be taken over whether they should be exited from the organization”

Isn’t it awful that there is a bottom 10%? Who could disagree that the bottom 10% had better pull their civil service socks up?

If you’ve found your way to this blog, you’ll at least have some misgivings about staff performance appraisals and ranking. As a reminder, and for people coming to this blog by googling “why you shouldn’t touch someone” or “picture of sick”, here’s why:

  • staff arent responsible for most of the work that comes out a system. It’s the thinking of management that shapes the design of the work, this is responsible for 95% of the performance of that system compared with the 5% supplied by individuals working within it. The system never gets a performance appraisal, you can’t get it in a room to fill out it’s self-evaluation, instead the 5% do and get the blame for the 95% outside the room.
  • you can’t score people on something as multifaceted as “work”. You could time them running 100 metres, but that’s not their job. Most people do more than one thing, and more than one thing at a time. Try and score a social worker, nurse or fireman. Score my cat, may as well.
  • appraisals are divisive, nobody likes doing them. Managers hate it, the endless paperwork and forms and awkward meetings. Staff dislike being appraised as if they were cattle at an auction. They’re just rubbish, if there was some merit in them, we would have noticed by now.

As Deming said, just stop doing them. But what do you replace them with asks a foolish man and he replies, “What?! You get rid of one stupid idea and think it needs to be replaced with an even more idiotic one?”

But imagine you know none of this. You want to improve your service, a noble aim. Because you don’t know that it is management thinking that is at the root of all major performance problems, you need to take action somewhere. Pull some sort of lever. You look around at where the work magically appears from. It appears from staff. They do it. So pull their lever. See what happens when you try and improve them. Only they can do it, you can help them but they are the basic unit of work production so that’s where the effort is concentrated. You want the worst to improve. How to find the worst? Score them! Somehow assign a score to each member of staff, now line them up in order, and hey presto you now know who is the worst 10% and who is the top 25%.

Imagine you have ’em all lined up on an expensively purchased performance management IT system. You then take action, in the plan this is called “performance management and improvement planning”. You’re not a barbarian, you won’t just fire them. You’ve identified a training opportunity. Then they apply themselves, and what do you think will happen the next time they are scored at the next annual appraisal?

I bet you £5 that the vast majority of the bottom 10% performing staff will get better, and will not be in the bottom 10% any more.

“Wait, what?“, I hear you cry. “I thought you said performance appraisals and scoring didnt work? If they get a higher score, it must have worked?

It is a phenomon called…

Regression to the mean

These are probably the most important 4 words you will read today. If you don’t like these 4 words, and they arequite off putting, it is also called…

The Sports Illustrated Cover Jinx

or as I remember it…

The Robbie Williams Mistake

We’ll start here. Robbie Williams is a singer-songwriter. He did very well in a band, then went solo and released albums. They did very well too and after several multi-platinum selling albums he was amongst the top selling British singers of all time. He was hot and as his contract was running out it was renewed by EMI who offered him £80 million. And every album since then has sold less and less. He was at his peak, EMI bought his future recordings at his peak, at a peak price. But he had peaked and the only way was down, and down he went.

What’s this got to do with the Sports Illustrated curse? An American sport magazine that features on the cover the latest hot new basketball player or American footballer. It is thought that being put in the cover is a jinx, and that you will suffer an injury or subsequent poor performance. There is an extensive list of sportspeople who have had something dire happen to them here.

The answer is that you only get to be on the cover if you have performed exceptionally well recently. If you are formally a mediocre performer and you hit a winning streak, making lots of attention, you’re much more likely to be on the cover. You get put on the cover most likely when you are performing very well, not as usual. This means outlying performance at the extreme end of your ability will put you there. Then what? Your lucky streak ends. You return to your average level of performance or worse. In sport luck is a huge part in performance. Luck affects you and your opponents. Not having an injury for an extended period is lucky too, and that luck also ends.

As with Robbie Williams and his luck, the luck of being in the right time and place with the right song writers, at the right age. Once any of those perfect variables changes the luck begins to turn.

But there’s no such thing as luck. We name good luck and bad luck according to random and unlikely events like finding a £20 note in the street. Or losing £20 in the street. All it is is the noticeable and remarkable events that are getting noticed. The return to normal life doesn’t, but they are the same features of the same system.

That’s what regression to the mean is. Any outlying performance is more likely than not to be followed by performance of a different nature.
The performance reverts to the average, the position of central tendency.

Back to those bottom 10% civil servants. Given what we know about systems at work, that a lot is not within the control of the individual, how much is luck? How much will be different next month? What pieces of work could be passed “dirty” to a bottom 10 percenter that were not passed to a top 10 percenter? We don’t know.

What we do know is that the unknown quantities combine, and when they do even more variation is introduced. Variation being seen as good or bad “luck” depending on your perspective. If they combined in an extreme way that was fortunate for you, the chances are they won’t next time. If they combined in an extreme way that was bad for you , chances are they won’t next time.
In a system with variation (luck) in it, there is always regression to the mean.

I will win £5 from you, if you accept my bet that this performance management system will “work” in the main, because it identifies outliers. These outliers then will, in the main, revert to the mean. Their performance will improve. Don’t take my word for it, read this by Daniel Kahneman. He has a Nobel prize and I don’t even have my 10 metre swimming badge.

“I had the most satisfying Eureka experience of my career while attempting to teach flight instructors that praise is more effective than punishment for promoting skill-learning. When I had finished my enthusiastic speech, one of the most seasoned instructors in the audience raised his hand and made his own short speech, which began by conceding that positive reinforcement might be good for the birds, but went on to deny that it was optimal for flight cadets. He said, “On many occasions I have praised flight cadets for clean execution of some aerobatic maneuver, and in general when they try it again, they do worse. On the other hand, I have often screamed at cadets for bad execution, and in general they do better the next time. So please don’t tell us that reinforcement works and punishment does not, because the opposite is the case.”

This was a joyous moment, in which I understood an important truth about the world: because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them.

I immediately arranged a demonstration in which each participant tossed two coins at a target behind his back, without any feedback. We measured the distances from the target and could see that those who had done best the first time had mostly deteriorated on their second try, and vice versa.

But I knew that this demonstration would not undo the effects of lifelong exposure to a perverse contingency.”

Start looking for the bigger picture and the longer run.  Look for not one number but a series of numbers. Observe the behaviour of a wider system over time. Don’t confuse single events for the actual history.

Above all, ditch the appraisal system.

This entry was posted in deming, statistics, systems thinking and tagged , , , . Bookmark the permalink.

3 Responses to gud at maffs

  1. Linda says:

    Civil Service appraisals in my area are a complete waste of time, no one ever looks at them they don’t count towards any promotions, or anything else for that matter, and in fact this year in my area every one of us were marked with the same marking, even though we don’t all do the same job. To get a top marking you need to be all singing all dancing, obviously none of us were (around 600 staff) as we all got middle markings. The synical amongst us think it may be something to do with the fact that we normally get a yearly bonus based on our end of year performance appraisal


  2. Anonymous says:

    I found appraisals to be a tool of oppression and bullying. As a matter of principle the appraisal was always bad and was really my boss taking out his dislike of me. I think they are totally pointless. A good firm will recruit good staff at the point of recruitment and such staff should not need to be routinely and formally appraised. Like so much of organisational life, appraisals are just bollocks.


  3. harry stephens says:

    Hi Harry says ” I have carried out appraisals and in the organisation its was always easier not to have high flyers or failures but middle of the roaders because it caused more work either way, questions were asked,so everyone was + or – meaners,good eh”


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s