The Tyranny of Measurement – Bad Targets

Cheating isn’t bad, targets are bad!

You are probably cheating, somewhere in your life you are cheating. Some of this cheating is the good kind, and some isn’t. What’s interesting is it’s mostly driven by one thing. Targets.

Bear with me, this does have relevance to Agile, eventually.

Targets are Bad

Throughout all walks of life we’re beset with targets, and it never stops. What are examples of targets?

  • Pass marks in school and university exams. (Don’t start me off on the evils of batching children by age instead of ability).
  • Commission targets for sales people.
  • Quarterly financial targets for companies.
  • Rigid deadlines for IT projects

 

All these are targets, and all these are subject to cheating. Chances are you know someone that has tried to cheat an exam? Sales people frequently close a sale knowing it’ll get cancelled to hit commission targets, because it’s the ‘sale’ that gets counted. Creative accounting to report not entirely accurate financial statuses of companies. IT projects that release poor quality software that doesn’t really provide the service the users need.

Sound familiar?

Why? We cheat because of the target. You could argue that this is because of a cognitive bias towards self deception, and you’d be partly right. We have that bias, because of the target.

Self deception bias is defined a process of denying or rationalizing away the relevance, significance, or importance of opposing evidence and logical argument (thank you Wikipedia). In our examples this might be “Everyone cheats on their exams, so it’s OK”, or “We are not counting the expenditure on xyz this quarter because…” or “Our project was a complete success, our users are happy and got what they asked for, we delivered on time”. This bias is only there because there is a target. Often they can be perceived as (or actually are) arbitrary and therefore not really ‘relevant, important or significant’.

As a culture we’re obsessed with targets and measuring. NHS waiting lists, speed limits, service level agreements, you name it there is probably a target.

In Agile software development, the most often debated target is ‘velocity’. I was contact by a friend last week that had been urged the get the team from 24 story points to 25. There was no real rationale for it, just the desire to beat the drum a little harder, whip the slaves and get up to ramming speed.

I won’t divulge his answer, suffice to say he gave the right one. Velocity isn’t something that should be used as a target. If someone in your project or company is using velocity as a target then the team will develop the cognitive bias for self deception. They’ll subvert the working practices to achieve the target and make the individual happy. This could take the form of simply inflating the estimations to satisfy the target, much worse would be a less diligent approach to the quality of the product. You get more software, faster, but at much reduced quality. Systems thinking tells us that poor quality causes rework and production costs increase. Still that guy that wanted to go faster is happy right?

 

 

 

 

 

 

Tags: , , , , , , , ,

3 Responses to “The Tyranny of Measurement – Bad Targets”

  1. Glenn Smith July 12, 2012 at 07:47 #

    Targets or any other metric where people are scored need to be used so carefully. It’s very true that you get what you measure, even worse if you reward good and bad results. I’m not necessarily against metrics or targets, I do believe though they need to be used very carefully.

    I think part of this our reaction to targets and metrics is the way the brain works. We’re wired not to want lose and we prefer rewards; the brain is very quick to pump dopamine around the brain to let us know we need to win more when we’ve made progress against something, be that winning a round of cards or moving closer to a target compared to last month. Loss aversion is very powerful and we should all be wary of having triggered it.

    We should also be mindful though to that to a lot of people, metrics and targets are the safety blanket they’ve carried for many years. As change agents in an organisation, either as scrum master or coach, we need to be mindful of this and not just dismiss them because they want them, but work with them to help them understand the potential negative impacts and better ways to get the same result.

  2. Tom Howlett July 11, 2012 at 19:13 #

    Great post and I definitely agree that targets that don’t match the overall purpose of the organisation encourage cheating and waste (it’s a form of sub-optimisation) . So what about a burndown chart, is reaching 0 points on the last day of the sprint a target that encourages cheating?

    • Scrimmers July 12, 2012 at 09:20 #

      Burndowns are interesting.. if we treat them as a target then cheating will likely occur, however if we treat them as a ‘compass’ to see where we are rather than the 0 point target I beleive they are very useful.

Leave a Reply

%d bloggers like this: