Managers that have moved away from management by instinct to management by metrics are often frustrated by the strange quirks and illogical results they get from their management reports. This can be terribly frustrating, because it takes a lot of energy and a lot of meetings to get a management team onto the path of metrics. So, what’s going on?  Generally, you’re dealing with one of two problems. The first problem is that your staff is lying to you. When you and your team have invested a lot of time into a project, and you have a lot of pressure on you, there is always going to be a voice in the back of your mind that says, “SOMEBODY is responsible for these wrong numbers, and it can’t be an accident!”   PLEASE, Please, please resist the urge to listen to this voice. Sure, you will have some people who have torn loyalties, and some who have so much pride in their work that they have a bias, and there may even be the very, very rare individual that is trying to hide something embarrassing. But overall, everyone is trying to do what they THINK you want them to do.  Which leads us to the second problem, the instructions on how to collect metrics is less than clear.

First generation metrics are rarely as clear as they need to be. But there’s nothing wrong with that.  You don’t go from 0% right  to 100% right in one step. You probably left in some latitude in interpretation.  For example, in telling a research group that they can take 15 to 30 minutes for a certain type of report. Then you find that one person interprets this one way and other interprets it a different way. If you have multiple shifts, then you will probably find that different shifts have ENTIRELY different interpretations. When you look at this you’ll probably think, “This is hopeless! How do I get past this?” Actually it’s pretty easy. Just remember that everyone that came before you went through this, and they got out of it the same way you will.. by leveraging experience  or you can try trial and error. Either way, you’ll get there. It’s just a matter of how much time you want to devote to this project.

Let’s try the experience route. Bookstores are filled with management books that can tell you how to do this… in about 400 pages. Or you can hire a consultant, which can be quicker but is a lot more expensive, or you can follow these steps:

1. Apply what you’ve learned: After about 30 days, you will probably have some idea of what needs to be tightened up in your original metrics and instructions.
2. Reduce interpretation: Focus on any numbers you’re collecting that allows for interpretation. As in the example above if a report could take 15 or 30 minutes, either redefine this as 2 (or 3 or 4?) types of reports, each with a specific amount of time (10, 15, 25 minutes, etc.) or just split the difference and make it 22.5 minutes for all work (or a better number if you have one).
3. Spread the message: Go back to everyone who collects data and let them know you want honest numbers, and you expect that some numbers will look bad… but it’s OK; until you find the problems you can’t be successful. Make sure that they understand that finding a problem is not a problem in itself. Being able to identify problems is a step towards being successful.
4. Calibrate: Once you’ve made your adjustments, have everyone who collects numbers tested on the same work to see that they come out with the same numbers,  give or take 5%. Then collect  data for another 30 days.
5. Repeat: And then repeat the process. Keep repeating until the numbers start making sense.

Taking this one step at a time, this can be pretty easy. Try it, you’ll see that it works. And that’s my Niccolls worth for today!