Questionnaires are powerful tools, allowing valuable insights to be gained, that help the change manager to plan targeted and effective interventions. Designing questionnaires with psychological considerations in mind fosters the reliability and validity of the questionnaire. Furthermore, it improves the quality of the collected data and the accuracy of follow-up activities.
Hint
The quality of a questionnaire can be determined by two key criteria:
ReliabilityReliability means that a questionnaire produces stable and repeatable results under the same conditions, over time.
ValidityA questionnaire has a high validity if it accurately measures what it is intended to measure.
A practical exampleImagine you step on a bathroom scale several times in a row to measure your weight. As your weight remains stable during this period of time, the displayed result should always be the same, and the scale is considered reliable.
If the bathroom scale always exactly indicates the time you stand on it, this would be a reliable measurement, but not valid information, as the scale is supposed to measure your weight, not the time.
Different types of survey participants
Survey participants often don’t behave rationally. The general attitude towards the survey topic, individual interests, the current mood, but also personality traits influence the way a questionnaire is answered. Accordingly, different types of survey participants can be distinguished. Some typical examples are depicted in the chart below.
Of course, this kind of differentiation is a simplification of the complex reality. Yet, it helps to create awareness that psychological aspects play an important role in survey design and should not be neglected.
Influencing factors on survey completion
When using questionnaires for collecting data, there are several psychological phenomena that can influence the participants' responses.
Respondent burden: Participating in a survey is always accompanied by a certain amount of effort. This so-called respondent burden should be kept as low as possible. Otherwise, participants might become demotivated, which can result in lower quality responses. In addition, there is a risk of an increased dropout rate before completion, or a refusal to take the survey at all, especially for very long and complex questionnaires.
Satisficing: Satisficing is the tendency of respondents not to strive for the best and most appropriate answer to a question, but to choose an easily available answer without investing too much time or energy. This can lead to a considerable loss of valuable input and feedback, especially on open questions asking the participants for their comments and ideas.
Social desirability: The wish of survey participants to present themselves in a favorable light might lead them to give answers they assume are socially acceptable, or desirable. Instead of stating their individual opinion, they strive to conform their survey responses to existing or assumed norms and expectations. This behavior can cause significant biases, particularly on sensitive topics.
Response tendencies: Response tendencies subsume different preferences of respondents to complete questionnaire, following a specific pattern. Common examples are acquiescence, the tendency to agree regardless of their own point of view, and extreme responses, choosing either very positive or very negative values on a scale.
Order Effects: The sequence in which questions are asked can influence how participants respond. For example, survey participants often pay specific attention to the first and the last questions of a survey. This primacy and recency effect can lead to higher effort in answering these questions, while questions in the middle of the survey receive less attention. In addition, emotionally charged questions at the beginning of a questionnaire can set the stage regarding the perception of the subsequent content and may have an impact on the mood of the participants.