Demand Forecast Accuracy

Goal of the analysis:

The goal is to quantify how close forecasts are to actual demand, identify where and why errors occur, and prioritize improvements that enhance service levels while reducing inventory, expedite costs, and waste. For executives, forecast accuracy is a core driver of OTIF performance, working capital, production stability, and logistics cost. High accuracy enables confident S&OP/IBP decisions, supports capacity planning, and reduces firefighting; poor accuracy signals process, data, or portfolio issues that erode margin and customer satisfaction.

Data required:

• Actual demand and fulfillment:
  • Shipped quantities (by SKU, customer, channel, region, period).
  • Sell-out/point-of-sale (POS) data where relevant; retailer inventory/on-hand and out-of-stock flags.
  • Backorders, lost sales estimates, returns, cancellations.
• Forecast data (with versioning):
  • Baseline statistical forecast, sales/planner overrides, final consensus forecast.
  • Forecast version timestamps and horizons (e.g., 1, 4, 8, 13 weeks ahead).
  • Promotion uplift assumptions and event tags.
• Master data and hierarchies:
  • Product master and hierarchy (SKU, family, category), life-cycle phase (NPI, mature, EOL), ABC/XYZ classification.
  • Customer/channel hierarchy and geography.
  • Units of measure and conversion factors.
• Calendar and causal drivers:
  • Trading calendars (ISO weeks, fiscal periods), holidays, seasonal events.
  • Promotions, price changes, marketing activity, competitive events, weather (if relevant).
• Supply and inventory context:
  • On-hand inventory, safety stock targets, lead times, service level targets.
  • Stockout events to distinguish constrained vs unconstrained demand.
• Reference forecasts and benchmarks:
  • Naive forecasts (last period actual, last year same period, moving average) for baseline comparison.
• Typical sources: ERP (SAP ECC/S/4HANA, Oracle), planning systems (SAP IBP/APO, Oracle Demantra, Kinaxis, Anaplan), retailer portals/EDI, syndicated POS (Nielsen/IRI), data lake/warehouse.

Detailed step-by-step instruction on how to conduct the analysis:

1. Define scope and metrics.
   • Select horizon(s): e.g., next week, next month, 3 months.
   • Select aggregation: SKU-location, SKU-customer, product family, region.
   • Choose metrics: WAPE (weighted absolute percentage error), MAPE, MAE/RMSE, Bias (%), Tracking Signal; use WAPE for robustness on small-volume SKUs.
   • Decide treatment for zeros/intermittent demand, NPIs, and periods with stockouts (consider excluding or adjusting actuals).
2. Extract forecasts and actuals.
   • Pull forecast versions with their creation dates and horizons from the planning system.
   • Pull actuals and POS; include backorders/lost sales if using unconstrained demand basis.
   • Ensure consistent time buckets and units across sources.
3. Clean and align data.
   • Harmonize calendars, handle returns/cancellations, deduplicate EDI lines.
   • Map SKUs/customers to standard hierarchies; convert units.
   • Tag promo periods and stockout-constrained periods; exclude extreme data errors and NPIs below a minimum history threshold (e.g., first 8–12 weeks).
4. Set the “actual” basis.
   • For service and inventory decisions, use shipped/sell-out actuals; if measuring unconstrained demand accuracy, adjust with lost sales and OOS periods.
   • Document the chosen basis for consistency.
5. Compute item-period errors.
   • Error = Forecast − Actual; Absolute Error = |Forecast − Actual|.
   • Pct Error = (Forecast − Actual) / Actual; use guards for small Actual (cap denominator or use WAPE).
6. Aggregate metrics.
   • WAPE = sum(|F − A|) / sum(A). MAPE = mean(|(F − A)/A|). MAE and RMSE as absolute measures.
   • Bias (%) = [sum(F − A) / sum(A)] × 100. Tracking Signal = Cumulative Forecast Error / MAD.
   • Calculate accuracy% if needed (e.g., Accuracy = 1 − WAPE).
7. Segment the results.
   • By product: SKU, family, ABC/XYZ (volume vs variability).
   • By market: customer, channel, region.
   • By context: promo vs non-promo, NPI vs mature, constrained vs unconstrained periods.
   • By horizon: 1, 4, 13 weeks/months ahead.
8. Time-series and benchmark comparisons.
   • Trend WAPE and Bias over 12–24 months to detect seasonality, structural breaks.
   • Compare to naive forecasts (last period, last year) and compute Forecast Value Add (FVA = accuracy of current process − accuracy of naive).
   • Compare planners/overrides to pure statistical models to see if interventions help or hurt.
9. Diagnose drivers.
   • Correlate accuracy with demand variability (coefficient of variation), lead times, promo intensity, price changes, life-cycle stage.
   • Identify the top 20 SKUs/customers contributing to total absolute error (Pareto).
10. Translate to business impact.
    • Quantify safety stock impact using forecast error (safety stock ~ Z × forecast error × sqrt(lead time)).
    • Estimate inventory and service improvements from accuracy gains (e.g., 5-point WAPE improvement → X days of inventory reduction).

Format of the output of analysis:

• Executive summary with WAPE, Bias, FVA by major business segments.
• Scorecard tables by product family, customer/channel, region, and horizon.
• Heatmaps of accuracy at SKU-location to surface hotspots.
• Line charts of WAPE and Bias over time; horizon accuracy curves.
• Pareto chart of cumulative absolute error contribution by SKU/customer.
• Waterfall for Forecast Value Add (naive → statistical → planner override → consensus).
• Scatter of Forecast vs Actual with 45-degree reference; Bias gauges by segment.
• Exception list of items with low accuracy, high bias, or deteriorating trend.

How to interpret results:

• High accuracy (low WAPE): Desirable; expect stable service and lower stock buffers. Validate that this is not due only to aggregation or naive easy items.
• Low accuracy: Indicates volatility, poor modeling, or data/process issues; expect higher expediting and inventory swings.
• Bias: Positive bias (over-forecast) ties to excess inventory and write-offs; negative bias (under-forecast) ties to stockouts and lost sales. Aim for bias within ±5% at planning family level.
• By segment: ABC/XYZ helps set realistic expectations—A/X items should outperform C/Z items. Channels with frequent promos typically show worse accuracy during promo periods.
• By horizon and level: Accuracy deteriorates with longer horizons and improves with aggregation (family/region vs SKU-location).
• Adjustments: Exclude or adjust stockout periods; for intermittent demand, prefer WAPE/MASE over MAPE.
• Trend: Improving trend suggests process maturity; volatility or step-changes require root-cause review (portfolio, pricing, macro events).
• Benchmarks: Use external ranges as guideposts and rely on internal top quartile and naive-baseline comparisons to set targets.

Steps a company can take to improve on this measure:

• Process and governance (IBP/S&OP):
  • Clarify ownership and RACI for demand planning; institute monthly S&OP with weekly exceptions.
  • Set override policies and freeze windows; require FVA justification for manual changes.
  • Establish exception-based workflows and target bias corridors by segment.
  • Define NPI/phase-out processes with reference class forecasting and guardrails.
• Data and systems:
  • Integrate POS, retailer inventory, and digital signals; enrich with promo, pricing, and weather where relevant.
  • Ensure versioned forecast storage and auditability; automate data pipelines and calendar alignment.
  • Upgrade models: hierarchical reconciliation, probabilistic forecasts, intermittent demand methods (e.g., Croston-like), parameter optimization.
• Capabilities and incentives:
  • Train planners on variability, bias management, and event planning; deploy playbooks by segment (A/X vs C/Z).
  • Align incentives to service, inventory, and accuracy (balanced to avoid gaming).
  • Collaborate with Sales/Marketing/key accounts on promo plans; adopt CPFR where feasible.
• Portfolio and demand shaping:
  • Rationalize long-tail SKUs; standardize pack sizes and reduce optionality that adds noise.
  • Improve promo design and adherence; stabilize price/promo cadence where possible.
  • Reduce replenishment lead times to shorten forecast horizon exposure; enable agile supply options.
• Execution levers:
  • Set safety stocks using forecast error by segment; dynamic reorder points.
  • Deploy VMI/consignment for volatile customers; leverage postponement to absorb variability.
  • If WAPE is high but Bias near zero, focus on variability reduction and flexible capacity; if Bias is strongly positive, tighten override governance and recalibrate models.

Benchmark comparisons:

General benchmarks:

• At aggregated monthly product-family level, many organizations see WAPE in the 15–30% range; top performers can achieve low teens.
• At SKU-location weekly level, WAPE commonly ranges 30–60% depending on volatility; intermittent demand typically sits at the higher end.
• Bias corridors of ±5% at family level are considered good practice; tighter for stable portfolios.
• For NPIs and promo periods, expect materially worse accuracy; use separate targets and methods.

Segment- or industry-specific benchmarks:

• CPG retail, monthly brand/category level: WAPE ~15–25%; item-store weekly: 35–55%.
• Industrial spares, item-customer weekly: 40–70% WAPE; rely on WAPE/MASE due to intermittency.
• Apparel/seasonal, monthly family: 20–40% WAPE; heavy promo periods degrade performance.
• Pharma mature brands, monthly: low-teens WAPE at brand level; SKU-level higher.
• If external benchmarks are limited, construct internal ones: compare to naive forecasts, top quartile SKUs/planners, and cross-BU performance; set tiered targets by ABC/XYZ and horizon.