Analyzing the Results of a Time Series Model in SAP Analytics Cloud Smart Predict

You check the quality of your predictive model performance with the Expected MAPE. The expected MAPE is the evaluation of the 'error' made when using the predictive model to estimate the future values of the signal, whatever the horizon.

• For each actual observed value, the predictive model calculates as many forecasted values as requested by the analyst. This is called the horizon of forecasts.
• Each of those forecasted values is compared to the corresponding actual ones. Then, for each possible horizon, a per-horizon MAPE can be calculated. This value is the mean of the absolute differences between actual and forecasted values and is expressed as a percentage of actual values.
• The expected MAPE is the mean of all per-horizon MAPE values that have been calculated.
  • An expected MAPE of zero indicates a perfect predictive model. The lower the Horizon-Wide MAPE, the better your predictive model performance.
  • An expected MAPE of 12% indicates that the error made when using a forecasted value is approximately 12%.

The forecast vs. actual graph displays curves for the predicted values (forecast) and actual values (target) for the time series data source. It shows the accuracy of the predictive model.

The predictions are displayed at the end of the graph. For each forecasted value, the predictive model shows an estimation of the minimum and maximum error.

• The area between this upper and lower limit of the possible errors in the predictive forecasts produced by your predictive model is called the confidence interval. This area is only displayed for predictive forecasts.
• Outliers are values marked with a red circle on the graph.
• The forecasting error indicator is the absolute difference between the actual and predicted values. This indicator is also called the residue. The residue abnormal threshold is set to three times the standard deviation of the residue values on an estimation (or validation) data source. The forecasted value and error limit values are listed in the table for each predictive forecast, as shown in the forecast table.

The forecast table displays the following information to help you analyze the performance of your time series predictive model:

1. Forecast: Predicted values for the predictive model over a set of known data called the validation partition.
2. Error min and max: Minimum and maximum deviation measures of the values around the predictive forecasts. If you choose to segment your time series forecasting predictive model using an entity, you will have this information for each segment.

The output table displays details of the outliers. An actual signal value is qualified as an outlier once its corresponding forecasting error is considered to be abnormal relative to the forecasting error mean, on the estimation data set.

The forecasting error indicator is the absolute difference between the actual and predicted values and is also called the residue. The residue abnormal threshold is set to three times the standard deviation of the residue values on a training data set.

Anomalies are signal values that are outside the zone of possible error for the predictive forecast, which is defined by its upper and lower limit. The signal is compared to all the predictive forecasts.

The textual explanation in the time series breakdown report describes the modeling technique that is used to calculate the forecast. Time series breakdown can be either:

1. A time series breakdown modeling technique that you can apply when working with a time series with limited disruptions.
2. A time series breakdown when a smoothing technique is applied when working with a disrupted time series that doesn't follow a regular trend or cycle.

Both breakdowns include the following:

• The Actual is the observed historical data.
• The Forecast is the result of the prediction in the future.
• Influencers represent the part of the time series impacted by the influencers, specified in the Influencers field of the predictive model settings.
• Fluctuations represent the part of the time series detected by the predictive model that depends on past values of the time series. The report shows the influence of the last observations before the predictive forecast. Fluctuations reflect changes that are not detected at the trend and cycle level. For example, the predictive model can detect that the previous 37 values have an impact on the actual values.
• Residuals refer to what is left when the trends, cycles, and fluctuations have been extracted from the initial time series. Residuals are not systematic or predictable and reflect the part of the time series that Smart Predict cannot explain or model. The smaller the residuals, the better the predictive model. A good predictive model produces residual data that contains no pattern.

Depending on the type of times series breakdowns used, the Trend and Cycles are interpreted differently.

An Impact of Cycles graph is displayed when some cycles are detected in the forecasted time series. It provides details about how the target is impacted by cycles, for the seasonal cycles and for the fixed-length cycles.

The cycles are named after their recurrence (Yearly Cycle, Six Days Cycle, and so on) and each bar represents the impact of the cycle for a given period: that is, how much the cycle increases or decreases the value predicted for this period.

When a smoothing technique is used, Smart Predict always displays the values of the last three occurrences for each period.

Some cycles have a constant amplitude. This means that for a given period within the cycle the impact of this period is the same for any occurrence of the cycle.

In such cases, only one occurrence of the cycle is displayed (there is only one series displayed) as the impact for each period is identical for any occurrence of the cycle.

In the example below, the impact on the prediction for Saturday (Sat) is -85027.87 and this impact is the same every week.

Some cycles repeat over time with an amplitude that can change. For a specific period of the cycle, the impact is different for each occurrence of the cycle. To illustrate this evolution, the impact of the last three occurrences of the cycle is displayed (the chart has three series).

In the following example, the amplitude of the cycle increases over time.

The Past Target Value Contributions identify the past observations that most influence the forecast.

• At the step of identifying the model components, Smart Predict found that previous values of the time series have an impact on the actual values.
• The graph shows how the recent past, or distant past for an autoregressive component, influences the target.
• The lags are numbered with negative integers representing their distance in the past from the predictive forecast. Lag -1 is the point in the past just before the forecast. Lag -5 is five points in the past.
• In the following example, a predictive model is developed to forecast the ozone rate for the next 12 months.
  • The graph shows that the ozone rate is influenced by observed values in the recent or distant past.
  • It also shows the more important dates.
  • The lags are numbered with negative integers that represent how far back in the past they are from the predictive forecasts. Smart Predict found that the 10 previous values have an impact on the subsequent values. Therefore, the graph stops at 10.

Using these lags, you can analyze how the previous values influenced the subsequent ones. Here, the lags -1 and -6 are influential.

The Relative impact of componentschart represents the weight of that component in the absolute value of the actuals across the time series. A relative impact of n% means that the considered component alone represents n% of the actuals. The relative impact for the components adds up to 100%, including the final residuals.

The component final residuals represent the part of the actuals that is not explained by the time series forecasting model.

In the following example:

• In this example, the trend alone weighs for 91% of the actuals.
• The yearly cycle weight represents only small variations around the trend and weighs only for 7% of the actuals.

These signal statistics (minimum, maximum, mean, and standard deviation) are provided for both the training and validation data sets. In this example, the Validation data set information is displayed. To change it, use the dropdown menu.

When you use an entity to segment a time series, the reports for each entity are available.

• If there are fewer than 20 segments, then these reports are available automatically following training.

  Select the column values that appear together forming a segment. For example, Product X, Store Y, from the top-left dropdown list in both tabs to view its report.

• If a predictive model contains more than 20 segments, the reports for each segment are not available automatically following training, instead they are accessed on demand. This is to ensure that time is not lost creating reports for predictive models with a high number of segments, when not all of those reports are required at once.
  • Select the segment, and after a slight delay, the reports are created and made available.
  • Once a report is available, you can then access it immediately at any time.