0
42
Guest

When Metrics Lie

By: Guest | April 25, 2011 | 
2

Steve McKee is a BusinessWeek.com columnist and chief blogger at StalledStuckorStale.com.

The other night I had the most amazing bite of peanut butter chocolate cheesecake. I say “bite” because, while I did my best to clean my plate, by the time the slice was halfway done I was all done. It was too rich to finish, and I went from shoveling it in to shoving it away in a matter of minutes. You might say that the cheesecake functioned well in terms of “peak performance,” but it certainly wouldn’t have been a good choice for a meal.

In a recent article entitled “Measurements that Mislead,” The Wall Street Journal’s Jonah Lehrer pulled together a wide variety of studies that demonstrated the danger of making decisions based on peak performance evaluations. If you’re someone who hires and fires or buys and sells, it should send a chill down your spine. It did mine.

Think about this:

  • A University of Minnesota study, for example, demonstrated that supermarket cashiers who performed best in a short-term speed test weren’t necessarily the most productive over time.
  • The University of Michigan found that scores on the Law School Admission Test had little relationship with career success, whether measured by income or satisfaction.
  • Economists at the University of Louisville say there’s no statistical relationship between player performances at the annual NFL Scouting Combine and how their pro careers pan out.

That makes sense. Who hasn’t known (or been) a bad test-taker who is highly intelligent in other avenues of life? Or someone who rocked the SAT but couldn’t find their way around town? If we measure by peak performance alone, we’re likely to get an incorrect impression of how people will operate in the real world.

Those of us in the agency business know this all too well. The standard agency review process isn’t just flawed; it’s dangerous. Clients that hire agencies based on their performance in a “pitch” (the word itself is troubling) often learn that who you date isn’t always who you want to marry.

But we’re just as guilty on this side of the table, expecting to land the client or secure the coverage based on a single swing for the fences rather than by slowly and deliberately building trust. Even when my firm is the buyer we’ve been burned by hurriedly linking up with vendors that seem to have the right stuff, only to find the last bite is nowhere near as good as the first. And I know I’m not the only person to make a hiring mistake based on an incorrect initial impression.

Lehrer says we use peak performance evaluations because they don’t take very long, they give us a way to quantify and compare, and they make assessment seem relatively straightforward, reducing our uncertainty of making a decision. But what they can’t do is measure character. Or determination. Or perseverance. Or devotion. Or empathy. Or emotional intelligence. Or any one of a hundred other things that mean the difference between success and failure, between productive relationships and disastrous experiments, and between long-term partnerships and short-lived mistakes.

When time is at a premium it’s easy to rely on shortcuts, especially when they seem so scientific. But if a shortcut might take you down a dangerous alley, it’s prudent to take the long way home. The great John Wooden counseled his UCLA players, “be quick, but don’t hurry.” In business, as in basketball, it’s performance over time that counts.

No one these days has the luxury of moving slowly, but we should be careful about drawing conclusions based on peak performance metrics alone. In some cases that may  amount to just another form of spin. And we know how we feel about that.

Steve McKee is a BusinessWeek.com columnist, chief blogger at StalledStuckorStale.com, president of McKee Wallwork Cleveland and author of When Growth Stalls: How it Happens, Why You’re Stuck and What to Do About It. Find him on Twitter and LinkedIn.

 

2 comments
Jack @ TheJackB
Jack @ TheJackB

Metrics make me crazy because far too often people mistake the trees for the forest. In the push to be more profitable I see a lot of pennywise pound foolish situations.

For example, years ago I worked in construction on the manufacturing side. We offered products for cutting concrete.

Sometimes I would speak with prospective customers about purchasing one of our diamond blades. Some of them would say no because it cost $500 and they could purchase 10 blades of a lesser material for the same price.

But they ignored productivity issues that this created. Every time the user had to stop the saw to change the blade they would also take a smoke break.

Over time those five to ten minute long breaks add up and could easily make the job take longer than it should have.

When they took time to look at the bigger picture it became clear that sometimes spending more on a purchase meant saving more money over all.