|||
((This is an abstract I just submitted to the 2019 DGeval conference in Bonn.))
We evaluators often ask questions about what causes what — because we want to know how things work, and because we want to know how stakeholders think things work, because this will influence their behaviour. As a species, our urgent task is to work out how to shift our civilisation onto a sustainable course, so we need good tools for understanding people’s behaviour and their reasons for it.
We human beings are good at learning and communicating causal connections. Of course our causal beliefs are just as vulnerable to bias and illusion as the rest of our beliefs. But we humans are still the best source of causal information available especially in those complex settings which involve other humans. We usually know how to calm a frightened child, or open a stuck door, and we each use this kind of information thousands of times a day. We even have opinions about what causes climate change, or what needs to be done to make more people better at recycling.
But as evaluators, we often prefer to ask stakeholders either quantitative questions (the opinions of the very same unreliable humans, but stripped of causal information and squeezed into a numerical format) or qualitative questions, which we hope to code into themes and categories and narrative summaries, with no special role for causal information.
This presentation looks at our options as evaluators to take a third way to gather information, namely to directly ask our stakeholders “the why question”: what causes what?
We will look at the tools available to code, synthesise and analyse multiple fragments of causal information, each of which is likely to be unreliable and inaccurate in indifferent ways, and to present the synthesised findings, perhaps in the form of a network diagram or “theory of change”.
In particular, this paper presents a new free and open-source online app designed specifically for this kind of task: to import multiple fragments of causal information, to code these fragments as links between different factors, to aggregate and analyse these multiple sets of links (do women mention this link more than men?) and to display the results.
Having tools like this which enable us to work more easily with these kinds of sets of multiple fragments of causal information will enable us as evaluators and researchers to quickly design and execute a new kind of research: one in which we simply ask respondents one or more “why questions” from a relevant domain.