Analyzing Quantitative and Qualitative Data

In the BriscoBikes case study, Sandra already talked about the key characteristics of quantitative and qualitative data.

Go to the following lesson for a short overview of the characteristics and the added value of quantitative data and qualitative data,: Getting Started with Organizational Change Management. Then scroll down to the video and listen to Sandra’s input, starting at 2:17.

Before focusing on the data analysis, let’s have a look at the definition of the two key data types:

• Quantitative data refers to information that can be measured and expressed in numerical values or quantities, e.g., counts, percentages, height, temperature, weight, or age. This type of data enables statistical analyses to identify patterns, relationships, and trends.

• Qualitative data encompasses non-numeric information. It captures the qualities, characteristics, and descriptions of a subject or topic. Thus, it allows a deeper understanding of underlying motivations, attitudes and concepts. Typical examples are interview transcripts, open-ended survey responses, observational notes, or audio and video recordings.

Besides the distinction between quantitative and qualitative data, there are three important ways to classify change survey data:

• Categorical data can be divided into groups or categories, but is not associated with a specific numerical value. Examples are gender or roles.

• Ordinal data have a meaningful order or ranking. While the order matters, the difference between the categories might not be consistent. An example is a rating scale for the perceived benefits of a new cloud solution, providing the answering options "no benefits", "few benefits", "some benefits" and "many benefits".

• Interval data is an important type of quantitative data where the intervals between values are meaningful, but there is no true zero point. Likert scales are a typical example for this data type facilitating an interval level of measurement. An example is the level of satisfaction with training courses, differentiating from 1 to 5, with "1" being "very dissatisfied" and "5" being "very satisfied".

Of course, the different data types have implications for the subsequent data analysis. Therefore, specific attention should be paid on the data type and the respective analysis methods to avoid misleading interpretations and wrong conclusions. This especially applies for advanced analysis methods.

An important step after the end of the data collection period is the data cleansing. Quantitative data should be reviewed to detect irregularities and errors and to remove the data of the respective survey participant. Here are some typical data cleansing activities:

• Review the data for incorrect or inadmissible answers (e.g. numbers beyond the rating scale) and delete them. This might especially occur in paper-based surveys.

• Verify that values conform to expected relationships and constraints (e.g., ensuring that the sum of parts equals the whole) and make sure to identify and remove inconsistent datasets. An example: If a survey participant answers the question "How many hours per week do you work with the SAP cloud solution?" with "100", this is theoretically possible, but very unlikely; so the value should be deleted.

• Watch out for incomplete feedback: If a respondent has missed out entire sections of the survey, you need to consider whether the rest of their answers should be included or not. It is recommended to define a threshold of maximum permissible missing values (e.g., 20 percent) and to exclude the data of survey participants that does not meet this criterion.

• Look for pattern in the data indicating random responses, for example straight-lining (all responses in a straight line) or an alternating clicking of extreme scale poles.

• Check the metadata to find the fastest respondents. Their answers should be verified thoroughly for any inconsistencies.

Leveraging sophisticated statistical methods and artificial intelligence can significantly facilitate the data cleansing process, for example by automated error detection (e.g., scanning the data for incorrect data and inconsistencies) or pattern recognition, supporting the identification of straight-lining and other distorting systematic data entries.

To get as many insights as possible out of the collected data, the data is usually structured into different evaluation units. There are many different options to "slice and dice" data. See the following chart for some typical examples in change surveys and practical advice:

The relevant information for breaking down the data in different categories is usually collected in a "General questions" section of the questionnaire. Invest sufficient time to think about useful sub-group analyses and ask for the respective data. If this opportunity is missed out, these analyses will not be possible. It is tempting to list as many aspects as possible to have a broad range of options for a detailed data analysis. However, the more personal information is requested from the survey participants, the higher is the danger that they fear their anonymity might not be guaranteed, which can lead to insufficient answers or even dropouts.

Additionally, if the number of sub-groups is too high, the complexity of the resulting data is very difficult to manage, and the practical added value is usually limited. Therefore, it is recommended to limit the number of general questions and only ask for information that most probably will have a practical added value for either evaluating previous change management activities, or planning future change management support.

There are many ways to look for insights in quantitative data. The easiest analysis is to sum up the frequency distribution of different answering options. Another very common analysis step is the calculation of mean values.

The mean or arithmetical average requires interval level of measurement. It is determined by adding up all values of the respondents’ answers related to an item and dividing the resulting sum by the count of numbers of your dataset (i.e. number of respondents). The mean is easily calculated and frequently used to analyze change survey data. However, it does not contain information regarding the data distribution. Therefore, it can be biased through statistical outliers. The chart below shows an example for different data equaling the same mean: