Building Training Surveys

The final step for any training program is a way to measure how effectively the training program accomplished its stated goals. This step in the training process is frequently overlooked due to time and financial constraints. However, as senior executives demand more accountability, there is a growing interest in measuring and reporting training results. SAP SuccessFactors Learning offers a training evaluation feature that can simplify this process with user-friendly surveys and quick reporting.

To assess training effectiveness, a four-level model like one developed by Donald Kirkpatrick is often used. According to Kirkpatrick’s model, evaluation should start with level one and then, if resources allow, move through the remaining levels in sequence. Information from each previous level is used as a starting point for the next level's evaluation. Each successive level provides a more detailed measure of the training program's effectiveness, but also requires a more thorough and time-consuming analysis.

The four levels of the evaluation model measure:

• Level 1: Reaction – What learners thought and felt about the training.
• Level 2: Learning – How learners’ knowledge or capability has increased.
• Level 3: Behavior – If learners are applying what they have learned.
• Level 4: Results – How learners’ performance impacts the business or environment. This could also be referred to as Return on Investment, comparing the cost of training to the measurable financial benefits.

With the training evaluation tools in SAP SuccessFactors Learning the first three levels of evaluation are supported. Level 1 and Level 3 evaluations are facilitated through surveys, whereas Level 2 evaluations are supported by assessments.

A survey is a configurable entity that may be attached to an item and triggered by specific completion statuses. There are two types of surveys: Course Feedback and Follow-up Surveys.

Before creating a survey, configure the rating scales you will use in the survey questions. It is recommended to use the same rating scale for most questions in course feedback and follow-up evaluation surveys to obtain consistent metrics. The course feedback reports will provide averages for each survey page and individual averages for instructors and items. If needed, different rating scales can be used for different types of surveys. For example, a five-point scale can be used for item evaluation surveys, while a frequency scale can be used for follow-up evaluation surveys.

Rating scales can be customized by navigating to Learning Administration → References → System Administration → Rating Scales.

When creating rating scales, be sure to put the most desirable or preferred outcome with the highest number in the scale and the least desirable choice with the number 1. Do not include N/A in your rating scales as the system will automatically include N/A for each rating scale question when the user is selecting their choice.

Once the survey entity is created, go to the Questions Tab to create the questions.

Questions can be organized in multiple pages or you may choose a simple one page format. Multiple pages are useful to segregate evaluations for different resources.

The figure below shows a one-page survey. Add questions to this page by selecting the Add Question icon.

After selecting the Add Question icon, you will be presented with four question types:

• Rating Scale: Use the question type of rating scale when you want to get quantitative results.
• One Choice: Use this question type when you want the user to choose one correct answer from a group. Additional possible answer choices can be added by selecting the green plus icon.
• Multiple Choice: Use this question type when you want the user to be able to choose multiple correct answers from a group. Additional possible answer choices can be added by selecting the green plus icon.
• Open Ended: Use this question type when you want the user to type an answer (up to 3,990 characters).

Depending on the chosen question type, you will then be prompted to enter the question stem and possible answer choices.

When you are done creating questions on Page 1, select Save Draft. Repeat this process for all pages of the survey.

Pages help to organize your survey questions into topics and make it possible to get mean scores per page if using rating scale questions. You can add a title and instructions for each page. Pages can be used to evaluate multiple resources that requires training evaluation. For example, there may be multiple instructors assigned to the same training. In this case, multiple pages may be used to evaluate each Instructor separately.

To add pages, select the Page icon.

The Item Usage tab in the survey entity allows you to associate your survey with specific item(s). Here's what you can do in this tab:

Search and Select Items: You can search for and select the item(s) you want to associate your survey with.

View Metrics: Once an item is associated, you can view the following metrics:

• Users Assigned: The number of users who have completed the item and been assigned to the survey.
• Completed Surveys: The total number of completed surveys.
• Completion Percentage: The percentage of assigned surveys that have been completed.
• Mean Score: The average rating across all completed surveys for that item. This is calculated only if all rating scale questions in the completed surveys use the same rating scale specified in the Rating Scale field.

For additional survey configuration options, select the Options tab. The available options in this tab will vary depending on the type of survey selected. Different sets of options will be presented based on the survey type chosen. Options will define how the survey behaves from the end user’s perspective.

Follow-up surveys should ideally be conducted three to six months after the training program. These surveys can be assigned to both the learner and their manager within a specific number of days after the completion of the training. The process is automatically triggered by an Automatic Process Manager (APM).

In the options tab, you can define the following:

• Time Periods: Specify the duration after which the survey will be sent.
• Participants: Choose who will receive the survey (the learner, their manager, or both).

Additionally, you can include a comments field for each survey question to gather more detailed feedback.

You may change the standard notification emails sent when a survey is assigned by selecting the Email Notifications tab.

There are two email notification templates associated with survey functionality:

• Questionnaire Survey Assignment Notification Email Template
• Questionnaire Survey Assignment Notification Rollup Email Template