Measuring Forecast Performance

Objective

After completing this lesson, you will be able to measure forecast performance

Forecast Performance Measurement

The figure describes the General Remarks for Key Figures Related to Forecast Error Calculation.

Take note of the items given in the figure to be aware of the prerequisites for a default SAP6 usage.

Several key figures are related to forecast error calculation in both statistical forecasting and demand planning.

The figure describes the Key Figure Description for Key Figures Related to Forecast Error Calculation.

Several key figures are related to forecast error calculation in the area of sensed demand.

Forecast error calculations help you gain insight into the forecasting accuracy of the planning areas that you are responsible for.

As a demand planner, you can use this information to monitor your company's forecasting performance, as follows:

  • Compare different forecast key figures to see which key figure provides the best value for your forecast.

  • Find areas where forecasting accuracy needs to be improved.

  • Calculate forecast accuracy at any aggregate planning level.

Define the appropriate forecast accuracy goals for various types of products, for example, products with least volatile demand, or highest sales.

The figure describes the differences between the Error Measure Calculations.

Difference Between the Error Measure Calculations

The error measures for which you set up calculations in the Manage Forecast Error Calculations app are also calculated during forecasting, therefore the following question may arise. What is the difference between the error measures calculated in forecasting and the error measures calculated in the Manage Forecast Error Calculations app?

The formulas that the system uses for calculating the error measures are the same, but because the business purpose for which you use the calculated error measures is different, the set of data that serves as the basis for the calculations is also different, and so on.

Basic Concepts in Forecast Error Calculations

  • Planning Level

    A planning level enables you to analyze data and plan at a specific aggregation level.

    For example, the PRODCUST planning level is the level for product-customer, and allows for data analysis that is tied to both the product and the customer and their attributes, or attributes that are specific to product and customer combinations.

    Key figures (either stored or calculated) are defined for the base planning level, which can be different from the level that you usually view the key figure at. Forecasting requires that historical data is available in a key figure whose base planning level defines the lowest granularity on which the historical data is available. Forecasting can be done at more aggregated levels as well. If you usually forecast at a certain level, such as product, group, and country/region, you will want to measure forecasting performance at this level. You can do so by creating various forecast accuracy metrics as calculated key figures defined for your preferred planning level.

  • Lag

    Lags represent the number of periods between the current period (the period in which the forecast is created) and the forecasted period (the period for which the forecast is calculated).

    Lag is defined as follows:

    • A lag 0 forecast means the forecast calculated in week 1 for week 1.

    • A lag 1 forecast means the forecast calculated in week 1 for week 2.

    • A lag 2 forecast means the forecast calculated in week 1 for week 3.

    • A lag 3 forecast means the forecast calculated in week 1 for week 4.

    • A lag 4 forecast means the forecast calculated in week 1 for week 5.

    Once a demand forecast is created, the lag x version of the forecast for a certain period refers to the forecast that was generated for that period, x periods ago.

    You can use this information to monitor the evolution of the forecast. You can compare the forecasting results of various lags with the actual numbers to see how accurate your forecast was for the specific lags.

    The forecast is updated over time. Therefore, it is necessary that you save the forecasting data for the lags that you want to compare. You can do this using the appropriate Copy operator.

Time Settings

Forecast error measures are calculated from the sales history and the forecasting data of previous or future periods or both. When you set up profiles for forecast error calculation, you can define how many historical or future periods should be considered in the calculation.

The point of reference for all-time settings is "Now", which is the current period. If the periodicity is set to Monthly, and the current month is October, then "Now" is equal to October.

Note

"Now" is considered to be part of the future so it is not included in the calculation horizon when the time scope is the past.

Measure Forecast Performance

Steps

  1. Create a forecast error calculation profile in the Manage Forecast Error Calculations Demand Planning. Choose the forecast Base Planning Level with the time dimension and the output planning level without the time dimension as well as additional parameters.

    1. Log on to the SAP Fiori Web UI.

    2. In the Demand Planner section, choose the Manage Forecast Error Calculations Demand Planning tile, and choose New.

    3. Enter Forecast_Error_## as both the profile name and the description.

    4. Choose ZSAP6 as a planning area and LOCPRODCUSTMONTHLY as the planning level.

    5. In the Groups of Forecast Error Measures section, choose Add.

    6. Enter Forecast_Error_## as the name.

    7. Choose Actuals Qty as the Sales History Key Figure and Statistical Forecast Qty as the Forecast Key Figure. Choose KG as the Target UOM.

    8. Choose Past as a Time Scope and enter a Calculation Horizon of 12 Monthly periods.

    9. In the Output settings section, choose LOCPRODCUST in the Planning Level of Output Key Figures.

    10. As the necessary Measures to Calculate, choose Forecast Error Measures - Mean Absolute Percentage Error (MAPE) and assign the relevant key figure Model Fit Error: MAPE.

    11. Choose Add and then choose Save.

  2. Calculate the forecast error.

    1. Log on to the SAP Fiori Web UI.

    2. In the General Planner section, choose the Application Jobs tile, then choose Create.

    3. Select Forecast Error Calculation Operator as the job template.

    4. Proceed through Step 2 of the wizard.

    5. In Step 3 of the wizard, select planning area ZSAP6 and your profile Forecast_Error_##.

    6. Select Version Baseline and use your filter.

    7. Select Schedule.

    8. Monitor the progress of your job.

  3. Check the results by creating a chart to the forecast error with the Advanced Analytics app or in Manage Analytics Story with a respective chart.

    1. In the General Planner group, choose the Analytics Advanced tile.

    2. Choose CreateAnalytics Chart.

    3. Enter Forecast error ## as the name and description.

    4. Choose the ZSAP6 planning area.

    5. Choose Model Fit Error: MAPE as the key figure.

    6. Group by Product ID, Location ID, and Customer ID.

    7. Choose KG as the Target UoM value.

    8. In the filter at the top, choose all your products that end with ###.

    9. Enable Auto-Refresh.

    10. Observe the results and choose Save.