Causality: why people do what they do
When gathering requirements, creating the user interface designs and measuring the impact, some things are more important than others and you need to get to the fundamentals that drive performance.
We use a range of techniques to uncover the fundamentals, and in particular causal surveys.
Causal surveying: Design
Unlike traditional surveys, our approach to survey analysis identifies the key drivers behind people’s preferences, satisfaction and intention to re-use and refer to others the service or product they are using. We adopt the scientist-practitioner commonly used in psychology. This means we do the following:
- Specify the null hypothesis (i.e. there is no change in outcome variables from one year to the next) and attempt to prove it through the appropriate use of statistical techniques. Therefore, when the results disprove this hypothesis, greater confidence can be placed on the different being real
- Conduct requirements workshops to understand the nature of the drivers of the key outcomes of interest to ensure these items are included in the survey
- Use specific questions so that when the analysis occurs, there can be a definitive statement of:
- What to do more of
- What to do less of
- What to do differently
- We specify the statistical analysis techniques in advance. Most market researchers either use conventional descriptive statistics (e.g. averages, standard deviations or frequencies) to report on data. They then work out after the even what other techniques to use, if at all. From a statistical rigour point of view, this results in serous problems with the data analysis, as this approach affects the error of statistical analyses and the confidence in the results
We primarily use multiple regression analysis as it provides easily interpretable and actionable data y providing clarity of what to do in response to the obtained results.
Causal surveying: Analysis
Our techniques look beyond the surface features of the data to identify the drivers of the outcomes. Often it is a variable (or constellation of variables) that are hidden within the data that are the key drivers.
We use advanced statistical techniques to maximise the decision making strength of the analysis. As a result, it will be clear exactly what to change to improve the customer experience, service and value of the application.
The primary technique we use is multiple regression analysis (a variation on structural equations modelling) to identify the causal agents and their strength in predicting outcome variables of interest.
The range of univariate and multivariate statistics we use to analyse the resulting data, include:
- Multiple regression analysis to identify the most important predictors of desired outcomes for the department
- Factor analysis to determine if a range of variables can be collapsed into a set of fewer underlying themes and concepts
- Cluster analysis to assign respondents to certain categories (e.g. market segments) so different activities would be performed with them
- Discriminant analysis to describe binary category assignment and why this occurred for use in predicting the membership of others
- T-Tests, ANOVA and other tests of differences between groups to determine if a significant difference exists and can be attributed to variables of interest
- General descriptive statistics (e.g. means, standard deviations, frequency counts, cross tabs, percentage (dis)agreement, etc.)
The total effect of these approaches is to greatly simplify the survey process by conducting it within a rigorous research framework so that confidence can be placed in the results. Rather than having to work through pages and pages of means, standard deviations and cross tabs for all items, these techniques serve to reduce the complexity of the analysis.
That is, these multivariate techniques either collapse the complexity into fewer high-level summary variables, or identify the subset of variables of that act as key drivers.
Our approach is not just about reporting on what customers or staff think now, but rather why they think that way. Therefore, effort can be spent on the areas known with confidence to have the greatest impact, rather than trying to do everything at once, or making a guess at what matters most.
