If you’re setting up a system to track metrics, a big question is, “Will the people who are responsible for collecting the numbers do the job honestly and competently?” It’s a good question. I think in almost any situation, especially if you set up some sort of checks and balances, it will be done honestly. If you can’t depend on your staff to give you real data, then you have a bigger problem to deal with. It’s the second part of the question that really matters. What are the instructions and incentives you’ve provided? Once you start getting reports, will you be able to check the data to be sure that it’s right? Let’s take a look at how the incentives we set up today affect the way our numbers will look tomorrow.    

The documentary Freakonomics, is a collection of short stories that take about incentives and how to make them work. An ongoing thread is how we often confuse correlation (the things we see) and causality (why the things we see happen).  I thought that the most interesting story was about Sumo wrestlers. Americans think Sumo is a bunch of really big guys who are not wearing a lot, slamming and slapping each other until one is thrown out of a circle. But if you’re Japanese, you see an essential piece of your culture, the world’s oldest martial art, a 1,500 year old sport, and the core ritual of the Shinto religion. Cheating at Sumo is a combination of crime, blasphemy, and just really bad taste that would get you tarred and feathered in the US. Sumo cheating is viewed so negatively that it’s not a subject that is even open for public discussion… especially when there are, AH, certain discrepancies.

Apparently, the game of Sumo sets up some very peculiar incentives. Wrestlers are divided into ranks, with most of the rewards going to the top ranks. Wrestlers who win all of their early season matches are guaranteed to maintain their rank, even if they lose late season games. A top ranked wrestler could throw the end of season games… which have the greatest attendance… and they would still retain their champion status. Then again, a champion who performed flawlessly in early rounds might be tired, injured, or just burned out by the end of the competition.  We have incentive and correlation, but we’re missing a theory of causality to tell us which games have been thrown. For that, Freakonomics looked at who won these bouts. Overwhelmingly, wrestlers who would otherwise be demoted to a lower rank won. So, the big winners, once their rankings are safe, are in a position to strategically lose matches, to help friends in need. Oh, there could be additional incentives (bribes, orders from your manager, etc.), but this alone is enough to explain the excessive number of lost games.

Why is this important in managing big operations in a Fortune 500 firm? Well, if you allow ranges for time (rather than a set time for an activity), the number that will help a shift meet its goals is likely to be selected. If time can be renegotiated (and a record is not kept of renegotiations), then the number that meets required metrics tends to be kept. If time is manually entered, the most beneficial number will generally be used (rounding to a later time, choosing different clocks for different work, etc.). Of course, there are some others that will “cheat” in the opposite direction, and move the numbers against your staff… because they are adjusting for some real world event, such as giving less time to more experienced staff; the problem is that they may not understand that you have already allowed for these factors, and they are double counting.  It goes on and on, but these “cheats” are very hard to find and in the mind of the data gatherers, are not real cheats.  Your data guys are usually just trying to help make you (and your managers) look good.  It’s tricky when the data problems are coming from the people who want to help you the most, and are the last people you would think are the problem. More on this in my next posting… but that’s my Niccolls worth for today!