Answering the why” question: piecing together multiple pieces of causal information

((This is an abstract I just submitted to the 2019 DGeval conference in Bonn.))

We evaluators often ask questions about what causes what — because we want to know how things work, and because we want to know how stakeholders think things work, because this will influence their behaviour. As a species, our urgent task is to work out how to shift our civilisation onto a sustainable course, so we need good tools for understanding people’s behaviour and their reasons for it.

We human beings are good at learning and communicating causal connections. Of course our causal beliefs are just as vulnerable to bias and illusion as the rest of our beliefs. But we humans are still the best source of causal information available especially in those complex settings which involve other humans. We usually know how to calm a frightened child, or open a stuck door, and we each use this kind of information thousands of times a day. We even have opinions about what causes climate change, or what needs to be done to make more people better at recycling.

But as evaluators, we often prefer to ask stakeholders either quantitative questions (the opinions of the very same unreliable humans, but stripped of causal information and squeezed into a numerical format) or qualitative questions, which we hope to code into themes and categories and narrative summaries, with no special role for causal information.

This presentation looks at our options as evaluators to take a third way to gather information, namely to directly ask our stakeholders the why question”: what causes what?

We will look at the tools available to code, synthesise and analyse multiple fragments of causal information, each of which is likely to be unreliable and inaccurate in indifferent ways, and to present the synthesised findings, perhaps in the form of a network diagram or theory of change”.

In particular, this paper presents a new free and open-source online app designed specifically for this kind of task: to import multiple fragments of causal information, to code these fragments as links between different factors, to aggregate and analyse these multiple sets of links (do women mention this link more than men?) and to display the results.

Having tools like this which enable us to work more easily with these kinds of sets of multiple fragments of causal information will enable us as evaluators and researchers to quickly design and execute a new kind of research: one in which we simply ask respondents one or more why questions” from a relevant domain.

Up next rbind.fill for 1-dimensional tables in r I have two one-dimensional frequency tables resulting from say the baseline and endline of a survey, recording say membership of 5 age groups. The Inventory & analysis of small conservation grants, C&W Africa - Powell & Mesbach! Lots of charts! Here it is at last This was an interesting job. We visited three countries, did a lot of interesting interviews, and a lot of data analysis. This
Latest posts Causal Mapping - an earlier guide The walk to school in Sarajevo Glitches Draft blog post for AEA365 Theory Maker! Inventory & analysis of small conservation grants, C&W Africa - Powell & Mesbach! Lots of charts! Answering the “why” question: piecing together multiple pieces of causal information rbind.fill for 1-dimensional tables in r yED graph editor Examples of trivial graph format Using attr labels for ggplot An evaluation puzzle: “Talent show” An evaluation puzzle: “Mobile first” An evaluation puzzle: “Many hands” An evaluation puzzle: Loaves and fishes An evaluation puzzle: “Freak weather” An evaluation puzzle: “Billionaire” Publications Using Dropbox for syncing Shiny app data on Amazon EC2 Progress on the Causal Map app Articles and presentations related to Causal Maps and Theorymaker Better ways to present country-level data on a world map: equal-area cartograms A starter kit for reproducible research with R A reproducible workflow for evaluation reports Welcome to the Wiggle Room Realtime comments on a Theory of Change Responses to open questions shown as tooltips in a chart A panel on visualising Theories of Change for EES 2018? Peer mentoring for evaluators How do you explain reproducible research to clients? Links for my AEA eval2017 presentation, Washington DC