Explaining predictive modeling concepts

A predictive scenario is a pre-configured workspace that you can use to create predictive models and reports to address a business question requiring the prediction of future events or trends.

You can create one or several predictive models within a predictive scenario. Each predictive model produces intuitive visualizations of the results making it easy to interpret its findings.

In the interaction below, you will further investigate the three types of predictive scenarios.

Choose the predictive scenario that provides the best answers to your business question.

In the learning phase, the model is trained on historical data with a known target outcome.

The model is trained to identify patterns in the data in the past to predict the target in the following, or later, months. In this example, the Target data time frame (April) occurs AFTER the historical data time frame (January to March).

A Reference date of April 1st is set and the model is trained on 3 months of historical data to predict what happens in the following month (April).

In the applying phase, the model is applied on current data where the outcome is unknown.

The model predicts the outcome probability for each client ID. In this example, the model is applied on the latest 3 months of data (April – June) providing the prediction of the probability of churn in the future month (July).

A Reference date of July 1st is set and the model is trained on 3 months of current data to predict what happens in the following month (July).

• In the case of a classification or regression predictive model, the Data Source input dataset can either be a training or an application dataset.
• In the case of a time series forecasting model, only one dataset is used for training and application.

The training dataset contains the past observations that will be used to generate the predictive model.

• In this training dataset, the values of the target variable, which is the variable corresponding to the business issue, are known.
• By analyzing the training dataset, Smart Predict generates a predictive model that explains and predicts the target variable, based on the variables identified as influencers.

Once the predictive model is built (trained), it is then applied on an application dataset.

This application dataset must contain the same information structure as the corresponding training dataset as follows:

• The same number of variables (additional columns will be ignored)
• The same variable names as the corresponding training dataset
• The same order of presentation of these variables

Once the predictive model is applied, the predicted values of the target are calculated in the output dataset.

An output dataset contains the result of applying the predictive model to the application dataset and any additional information requested.

Once the predictive model is applied, the predicted values of the target are created in the output dataset.

The output datasets are saved by default in the folder: Main Menu/Browse/Files, but you can choose another directory if required, or save the output in SAP HANA (if you are connecting to SAP HANA).

To build a predictive model, the following variable roles are defined:

1. Target
2. Influencers

The target is the variable that is to be explained or predicted. For example:

1. A bank wants to predict if a customer will answer a marketing communication or not. The training dataset includes the customer information and contains the target variable <responded to mailings> <responded to mailing>. This target variable may take the values <Yes> or <No> 
2. A company wants to predict the number of complaints that a customer support will receive this week. The target variable is <Number of customer complaints>and it will take<numerical> values.

IMPORTANT: If the value <Yes>is the least frequent value, the application automatically considers that value to be the target category of the target variable.

The influencers are variables that describe the data and explain a target variable.

• An influencer variable corresponds to a column in the dataset.
• The observations relating to each influencer correspond to the rows in the dataset.

During the predictive model creation (learning phase), influencer variables can be excluded from the training process. These exclusions are not taken into account to compute the predictive model, not included in the statistics for the predictive model, not retrieved from the data source, and not needed when the predictive model is applied on an application dataset.

Exclude variables that are directly related to the target, especially variables that contain the target variable indirectly. They are known as the leakers or leak variables:

• These variables are in some way causally related to the target variable. However, instead of being a cause for whatever your target variable represents, they are the result.
• These leakers produce an incorrect model, often with a very high predictive power indicator (because of their high correlation with the target variable).
• To prevent data leakage, any variable created or updated after the target value reference date should be excluded because when you use the model to make new predictions, that data won't be available.

During the training, Smart Predict chooses an optimized number of influencers to include in the predictive model so the toggle is turned off by default.

If there is a reason for overriding the Smart Predict default setup, for example, when it is necessary to focus only on a few influencers that have the most influence on the target, then switch on the toggle and set the maximum number of influencers to be kept in the model.

When you are building your model in SAP Analytics Cloud, a hold out sample is automatically created. This is a sample of observations withheld from the model learning, so that the model's ability to predict future probabilities can be estimated by its ability to predict the data in the hold out sample.

Data is partitioned and split into:

1. a training subset to train the models
2. a validation subset hold out sample to test the model’s performance and choose the best performing model from a range of candidate models

1. The analytical data set feeds into the partition data where it is split into training and validation subsets
2. The training subset is used to train candidate models
3. The validation subset is used to evaluate candidate models to choose the best one and evaluate model performance.

For classification and regression models, the data is automatically randomly split:

• 75% into the training subset
• 25% into the validation subset

In time series models, it is important to preserve the data sequence relative to date/time, the historical data is automatically split sequentially:

• 75% into the training
• 25% into the validation subsets

In the example below, you are building a classification model to predict if a customer will buy a product or not.

To do this, you must prepare a training dataset containing the historical data on customers that have previously bought a similar product. In the training dataset, the actual values of the target variable (Bought) are known.

Once the predictive model is built, it is applied to an application dataset to predict if other customers will buy the product. This dataset contains the same information about the customers as in the training dataset, however, the target variable (Will buy?) is an unknown value (empty) because this is to be predicted.

Smart Predict will use the predictive model to calculate the probability that each customer in the application dataset will buy the product. The target variable column is now added to the output dataset with the predicted outcome (will they buy yes/no).

In the example below, you are building a regression model to predict the number of complaints that customer support will receive next week.

To do this, you must prepare a training dataset containing historical values for several previous weeks. In this dataset, the historical values, or actual values, of the target variable (how many complaints per week) are known.

Once the regression predictive model is built it is applied to a new application dataset which contains the same influencers. The values of the target variable for next week (number of complaints next week) are unknown as they are in the future.

The output dataset contains the prediction of the number of complaints that can be expected next week.

In the example below, you are building a time series model to forecast the product sales for the next 3 months.

The forecasting model is trained on the training dataset and automatically creates forecasts of the signal for N periods into the future.  So, in this sense, the training dataset and application dataset are the same dataset. In this example, product sales forecasts for the next 3 months are generated in an output dataset.

You can optionally include additional variables in the data model to refine and improve the forecasts (e.g. a flag that indicates when a sales promotion occurred Super Promo). The future values for these additional variables are required inputs to generate the forecasts.

In the example below, you are building a time series model using training/application dataset to forecast product sales for multiple products.

To do this, you must add the product-related information to the dataset. This is called the entity. In this example, you specify that the entity variable Product Name will be used to segment the forecast model.

In the output dataset, the observations are divided into Product Name and forecasts are created per product entity.