Almost every workforce management team reports forecast accuracy, and most report it as a single percentage built on MAPE. It is the easiest metric to explain and the easiest to calculate. It is also capable of ranking a worse forecast above a better one, which makes it a risky thing to manage by.
This post shows how, with a simulation you can reason about, and then lays out which metrics to report instead.
Why MAPE misleads
MAPE averages the percentage error of each interval, so every interval counts equally. A 50 percent miss on a 10-contact overnight interval moves the headline number exactly as much as a 50 percent miss on a 400-contact midday interval, even though the first affects almost no customers and the second decides your service level. MAPE also breaks when actual volume is zero, and it says nothing about direction. A 7 percent MAPE can hide a forecast that runs 5 percent low every single day.
Here are the metrics that fix those gaps, where F is the forecast, A is the actual, and n is the number of intervals.
| Metric | Formula | What it tells you | ||
|---|---|---|---|---|
| MAPE | Average of \ | F minus A\ | divided by A | Average percentage error, unstable at low volume |
| WAPE | Sum of \ | F minus A\ | divided by sum of A | Volume-weighted error, closest to staffing risk |
| Bias | Sum of (F minus A) divided by sum of A | Direction, shows systematic over or under forecasting | ||
| MAE | Average of \ | F minus A\ | Error in raw contacts | |
| sMAPE | Average of \ | F minus A\ | divided by the mean of F and A | Bounded, handles very low volume |
A tiny example makes the weighting difference obvious. Take one interval with 10 contacts actual and 15 forecast, and another with 1,000 actual and 1,020 forecast. MAPE comes out at about 26 percent. WAPE comes out at about 2.5 percent. The second number is the one that reflects where the staffing risk actually sits.
The simulation: the better MAPE lost on service level
To test this properly, We built a 48-interval day for a 24-hour voice queue with a realistic shape, quiet overnight and a midday peak near 400 contacts per interval, with an average handle time of 300 seconds. Actual volume was drawn with natural random variation. Then we compared two forecasts, each used to staff every interval with Erlang C at an 80/20 target, and measured the service level that resulted against actual volume. We repeated the exercise across 200 simulated days.
Forecast A was unbiased. It was very accurate in the busy daytime intervals but noisy in the tiny overnight ones, which is typical of real forecasts.
Forecast B was perfectly clean overnight but ran about 4.5 percent low in every daytime interval, a small, steady under-forecast.
| Result, average of 200 days | Forecast A | Forecast B |
|---|---|---|
| MAPE | about 13% | about 9.7% |
| WAPE | about 6.4% | about 6.6% |
| Bias | about 0% | about minus 4.4% |
| Volume-weighted service level | about 74% | about 60% |
Forecast B won on MAPE in 95 percent of the simulated days. A team ranking forecasts by MAPE would have picked it every time. Yet it delivered service level roughly 14 points lower, because its error all pointed the same way, at the intervals that matter most. Forecast A won on service level in all 200 days.
WAPE was close to a tie, which is the second lesson. WAPE is far better than MAPE for staffing, but it still cannot tell a symmetric error from a one-directional one. Only bias separates them. Random errors partly cancel across a day and cost you some service level. Systematic error never cancels. It compounds.
Benchmarks worth using
One widely cited planning guide for stable voice queues with at least a year of history suggests these ranges, offered as typical rather than as a standard:
| Measure | Typical range for a stable voice queue |
|---|---|
| Daily WAPE | 5 to 10% |
| Weekly WAPE | 3 to 7% |
| Bias over any rolling 13 weeks | Within plus or minus 2% |
Small queues, multi-skill operations, and digital channels usually run higher. Some sources express accuracy as 100 percent minus MAPE and quote 80 to 90 percent as good at interval level. Whichever variant you choose, use the same one across the organization, or comparisons between teams and vendors mean nothing.
What a better accuracy report looks like
Lead with WAPE, not MAPE. It weights error by volume, which is how staffing risk is actually distributed.
Put bias on the same line. Show signed bias next to WAPE every week. Sustained bias beyond about 2 percent over a quarter means the method is wrong, not unlucky.
Report by horizon. Accuracy at interval, day, week, and month tells you where the model is weakest. Most operations degrade quickly beyond about four weeks.
Show rolling 4-week and 13-week figures. One noisy week should not decide whether a method changes.
Split by queue, channel, and skill. A blended number hides the queues that are driving the misses.
Judge a forecast by the service level it would have produced. The final test of accuracy is the outcome. Replay last month's actuals against the staffing the forecast would have created and compare.
The bottom line
A forecast is not good because its average percentage error is small. It is good when its errors are small where volume is high and balanced in direction. Report WAPE and bias together, treat MAPE as a supporting number, and your accuracy report will start rewarding the forecasts that actually protect service level.
Test how a volume miss changes your staffing with the free Erlang C Calculator, and build forecast-driven rosters with the Capacity Planning Calculator.