When you read about the different quality systems out there, you quickly hit the wall when they start throwing statistics at you. Once you move into the heavy math, the interesting idea start to look like a very complicated and hard to accomplish project. However, it doesn’t have to be that way. If you have all the powerful tools, and the experience in how to use them if take you further, but for most corporate services just a tiny amount of math and some common sense can get you pretty far. If your operation produces millions of products a month, and your service level targets that look like “99.999”, you may need the math. If you produce between hundreds and tens of thousands or products a month, and have service level targets more like “95%”, then we can use a simple process. However, if you produce just 5 or 10 products a month, then there won’t be enough samples for this process to work easily for you. But for the vast majority, this will work and it will show you something very interesting about how your services really work.       

Let’s start with where you are today. I will assume that you have at least minimum metrics in your organization. By this I mean: you are capturing some metrics, including the “the big four” (ex.: 1. On time delivery; 2. Units produced or hours worked; 3. Production level or utilization or how much capacity is available vs. actually used; 4. Level of quality or number of errors or number of complaints). For now let’s assume that you have targeted the right metrics, the data is being collected correctly, and you’ve done enough testing to be sure that it’s all working. Now, here’s the problem. You look at your monthly management report. You’ve set a specific target (let’s say 93%) for your most important service level (let’s say  on time delivery) and after months of striving, your report shows that you have at last reached your 93% goal. You speak to your team and congratulate them on this achievement. You then go to your clients to update them on your progress, but they are mysteriously unimpressed. Some even say, “But there hasn’t been any improvement! In fact, things may be worse!” This isn’t what you expected, and you’re not sure how there can be such a discrepancy between your metrics and the clients experience. What’s going on?

In this case, the culprit is variability. The average, at the end of the month, may be 93% on time, but as human beings we do not “experience” anything in this type of timeframe. We experienced things “now” in a single instance. Although we might look at something in an hour time frame; the daily “rush hour” is a good example. Traffic on a road may be backed up for 2 hours during the rush immediately after 5PM when everyone is headed home. That makes it a terribly crowded road for the rush hour traveler. However, someone else who only uses that road late at night on a Saturday may be confronted by a completely empty road. And a city planner may look at utilization for the road on a monthly basis and say that the road is only 35% utilized.. busy, but not crowded… because he is averaging out the times the road is empty and the rush hours. Which view is correct? The answer is that each view is  right and wrong. It depends on how you’re trying to use the information. In your center, there may not be a single day in a month where the on time statistic was exactly 93%. To answer our questions, we need to look at look at a subset of data. We might look at the month’s information by day (what was the on-time number for January1st, 2^nd, 3^rd, etc.;)  or we might look at the hours of the day (the month’s data for 8am, 9am, 10am, etc.) or we might look at each product (the March on time percentage for: contracts, memo’s, sales presentations, etc.). Each of these analyses might yield different areas for examination and different targets for improvement. But let’s start with the variation by day.  When you have the daily on time data, you will see one (or more) of the following: 

1. Variation, but very limited: If 93% is your goal, you may see that in a given month you never have a day where you perform below 92% and never a day where you perform above 94%. In that case, you would need to look elsewhere for the problem. If this is the first time that you are looking at daily variability it is REALLY unusual to see very tightly grouped numbers like this.    
2. Variation, but explainable: Much more likely, when you look at your numbers you will see days that are WAY off. You might have a month with 93% on time delivery and still have one or two days that drop to 50%, or 40% or even lower. However, in a given month there could be one or two “one-shot” problems: There was a network crash; the firm was hit by a virus; a broken water main delayed the arrival of your staff that day; the city had a transit strike; there was a snow storm, etc. A network crash or a problem with a virus might show that a department (other than yours) needs to improve their emergency or security processes. A broken water pipe is outside of the control of your firm. But a strike or a snow storm usually provide some advance notice and can happen again at any time. These last two may identify days with very poor on time delivery and highlight the need for an improvement program (planning for emergencies).      
3. Variation, but unexplainable:Here you could see almost anything, but might typically see days when on time delivery drops to 70% (it could be any number)… but where no one has any theory as to why this is happening. Well, if 27 times a month you can reach 93% and on just 3 days a month you can’t, this is probably something you can fix. Digging down a little deeper, you are going to find one of two things:
   1. There is a regular pattern: Something like… every 3^rd Tuesday has a low on time delivery number.  This is very convenient, because you just assemble your team and have them sniff around on Tuesday and see what’s different compared to any other day.
   2. There is an irregular pattern: The problem happens 2 or 3 times a month, but there doesn’t seem to be any pattern to predict the next occurance. Here, tell your managers to immediately call Quality Control when the problem arises, but also have your Quality Control staff check numbers daily (or as soon as they are available), since the production staff may not realize when the problem occurs. If QC only finds out after the fact, that’s OK. Just get to managers and ask them what happened while the details are still fresh in everyone’s mind.  

Now, you know why understanding (and identifying) variation is the key to making significant improvements in your services. However, you don’t yet have a  tool or a specific set of steps to consistently identify variation. Well, it’s a good thing that you read this blog, because that’s exactly what we’re going to cover tomorrow. But for today, that’s my Niccolls worth!