March 21, 2019

Using Theorymaker4

((This post is just for me, because Theorymaker4 isn’t even online yet.))

This app is in an alpha state. That means you can use it and it should work, you can even do real coding in it, but it may crash at any point and the app itself will change in the future so you might not be able to continue any projects which you started.

To fix: in principle, everything will happen live” without needing to press save”, update” etc. But during development there are a still couple of Update” buttons, especially the one at top-left. You will sometimes need to press this, and maybe also visit the Display” tab, to make the diagram update itself.

Pressing the Interrupt” button will crash the app!

Tables, e.g. in the Import tab.

These are featureful but a bit flakey.

Sometimes a table doesn’t load properly; just click in it. They extend themselves when you paste in a lot of information, but they can seem to freeze.

Get started by looking at each tab in the interface.

1) Import

Here you paste in statements from several sources or respondents from which you want to extract causal claims. You can paste straight in from Excel, one statement or source per line (more rows will magically appear).

To fix: if you paste in more than about 20 rows, the interface seems to freeze. For larger datasets, add upload excel” option.

There are up to 7 additional columns for data like gender, age or role of the source. These extra categories make your Theorymaker diagrams much more interesting.

Change the names of these columns if required by clicking on edit column names”

You can add additional sources at any time by pasting them in starting in the empty row.

To fix: change column names by clicking on them.

To decide:

  • at the moment, if you delete or paste over the data from this tab, it is lost. Perhaps it is better to have a more conventional Import” function in which new data is appended to existing data.
  • whether to have an option to break up larger passages into smaller ones.

You could make diagrams in Theorymaker4 without looking at this tab, treating everything as coming from one source. But you’d be missing the point.

2) Code

Here, you view each statement one by one, looking for causal links e.g. X leads to Y”. In some statements you might find more than one such link; many will have none.

The bar at the top shows a pager with buttons from 1 to the total number of statements. If you have pasted some statements in on the previous tab, you can now click through and view the statements. The button first uncoded” is one you will be using all the time as it takes you to the first statement which you have not yet coded; you will often use it as another Next” button.

Add arrows: this panel is the beating heart of Theorymaker4. It is designed to make coding quick, easy and powerful.

In the upper box you type the names of one or more variables at the beginning of the arrow(s) and in the lower box you type the names of one or more variables at the end of the arrow(s). Separate the names with a comma or by pressing your tab key. What’s cool is that the names of existing variables are suggested as you type. You can accept the suggestion which has a grey highlight just by pressing Enter.

When you have at least one variable name in both boxes, add the corresponding arrow(s) by clicking Add this arrow”, or just by pressing Tab until this button is highlighted and pressing Enter.

Tip: if you have a statement which implies a chain, e.g. A leads to B which leads to C, you can speed up coding by pressing Add & continue the chain”; the arrow(s) are added but also the variables from the bottom box appear in the top box, to make life easier.

To decide: at the moment, you can’t put several variables in the upper box and several in the lower box.

View and edit arrows: this panel lets you look at the arrows you have created and edit them. Remember to press Update” when you’ve finished. This table, like most others, is sortable (click on column headers) and you can paste into it, right-click for additional options, etc.

Here you can edit:

  • the quote which goes with the statement, by default it is the whole statement.
  • the trust which you as a coder have in the arrow (you can give different levels of trust to different arrows mentioned the same source)
  • the strength from -1 to +1 which this statement claims the arrow has
  • the label which appears by default on the arrow

To decide: Perhaps make it easier to add this information while coding in the Add arrows panel.

View and edit variables: here you can:

  • rename variables by changing the label”
  • don’t touch the id column unless you know what you’re doing
  • you can add a group which is useful for drill-down scenarios

Combine variables: at some point while coding you will decide you want to permanently combined two variables e.g. Safer cycling” and Cycling is safer”. This tab makes this easy:

Advanced options: this is just for development.

Coding by drag-and-drop

You can also edit the diagram by drag-and-drop; just click the edit” button on the diagram. However at the moment, your edits will not be saved. Not sure how useful this is for larger networks.

3) Aggregate

By default, all sister” arrows which share from” and to” variables are combined. If you unclick the checkbox, they will be treated separately. Otherwise, here you can see various aggregated parameters, like frequency (number of arrows which have been combined) and title which is the tooltip shown above the arrows in the diagram. You cannot edit this table.

To decide: make this table editable?

4) Filter

Here you can see the additional statistics which have been calculated from the (possibly aggregated) arrows, for example the number of times the variable has featured as either end of an arrow.

To decide: we will want to have more options like auto-suggestion of grouping, simplification, etc.

5) Display

This is a powerful tab because we can do things like

  • change global appearance e.g. change the border colour of the variables (click the coloured area next to color.border)
  • change specific variables or arrows e.g. hide all variables containing the word village” (under variable, click the control next to hidden and change it to if..., and then type village)
  • making the width of an arrow dependent on the number of mentions (under arrow, click the control next to width and change it to conditional on..., and then select frequency)
  • making the colour of a variable depend on the percentage of sources who were women (assuming you have say gender” or female” as one of your columns in the Import tab, under variable, click the control next to colour background and change it to conditional on..., and then select gender)
  • there are lots of other possibilities!

Diagram (bottom section)

  • node spacing and level separation are important to make the text visible without crowding the diagram. Try them.

To fix and decide: getting the diagram to look right” is quite a big deal as it is about opinion, taste & user-group as well as technology.

To fix: many things … and bugs

6) Diagram

On the right is the diagram. You may need to press Update in the top-left corner to get the latest version.

  • move the diagram around by dragging an empty area
  • zoom in and out by doing the two-finger gesture or scrolling with the mouse
  • move things around and edit the diagram (but at the moment changes won’t stick”)
  • if you have used conditional formatting, there is a primitive legend which sort-of explains the meaning of the colours etc.

At the top-right is a button to show the whole diagram in a larger window above.

Saving your work

You can save a permanent link to your diagram. Type a name for it in the Title box, and press save. A link appears which you can click on, and/or copy and send to someone or bookmark for yourself. If you make further changes to your diagram, remember to save them again. If you want you can save them under the same title, or you can use a new title like my-diagram-version-2”. The title cannot have spaces or unusual characters.


You can right-click on the diagram and paste the picture into a document.

To fix: Download image and Download data; other image formats, etc.

7) Report

Here you will be able to see an auto-generated report - most interesting results per variable, bar charts of mentions, whatever.

To decide: what is most useful?

Diagrams you have saved could appear here.

March 14, 2019

Answering the “why” question: piecing together multiple pieces of causal information

((This is an abstract I just submitted to the 2019 DGeval conference in Bonn.))

We evaluators often ask questions about what causes what — because we want to know how things work, and because we want to know how stakeholders think things work, because this will influence their behaviour. As a species, our urgent task is to work out how to shift our civilisation onto a sustainable course, so we need good tools for understanding people’s behaviour and their reasons for it.

We human beings are good at learning and communicating causal connections. Of course our causal beliefs are just as vulnerable to bias and illusion as the rest of our beliefs. But we humans are still the best source of causal information available especially in those complex settings which involve other humans. We usually know how to calm a frightened child, or open a stuck door, and we each use this kind of information thousands of times a day. We even have opinions about what causes climate change, or what needs to be done to make more people better at recycling.

But as evaluators, we often prefer to ask stakeholders either quantitative questions (the opinions of the very same unreliable humans, but stripped of causal information and squeezed into a numerical format) or qualitative questions, which we hope to code into themes and categories and narrative summaries, with no special role for causal information.

This presentation looks at our options as evaluators to take a third way to gather information, namely to directly ask our stakeholders the why question”: what causes what?

We will look at the tools available to code, synthesise and analyse multiple fragments of causal information, each of which is likely to be unreliable and inaccurate in indifferent ways, and to present the synthesised findings, perhaps in the form of a network diagram or theory of change”.

In particular, this paper presents a new free and open-source online app designed specifically for this kind of task: to import multiple fragments of causal information, to code these fragments as links between different factors, to aggregate and analyse these multiple sets of links (do women mention this link more than men?) and to display the results.

Having tools like this which enable us to work more easily with these kinds of sets of multiple fragments of causal information will enable us as evaluators and researchers to quickly design and execute a new kind of research: one in which we simply ask respondents one or more why questions” from a relevant domain.

February 1, 2019

Better ways to present country-level data on a world map: equal-area cartograms

I’m just working on a report for the IFRC. There is lots of data from 190 different Red Cross Red Crescent National Societies around the world1.

Cutting to the chase:

.. that is our improved map-like solution, and I’ll show the code further down. What it is really missing is a nice stylised outline of continents in the background.

Why do I claim this is a better viz? Sometimes we’ve been using ordinary choropleths:

(In this case we are looking at the percentage of women on the governing board.) These maps look nice but they are really poor at presenting actual information IMHO. You can’t even tell which continent is doing better, and did you spot that Ghana has a pretty poor percentage? Did you spot the high scores for Georgia and Latvia and Moldova?

Equal-area aka equal-size cartograms are better, but I only found examples with ISO3 codes, which are really user-unfriendly. So I got my son to retype rbrath’s map into a spreadsheet and R does the rest.

You need this csv file which just gives you the iso and country names in terms of x and y coordinates.

Then you need a data.frame called say xx which has a column called iso3 and your data. Merge them together and there you go.

xx %>% dplyr::merge(maps,by = "iso3",all.y=F) %>% 
  ggplot(aes(x,y,fill=fem_GovBoard,label=name %>% str_wrap(8)))+geom_tile()+geom_text(hjust=.51,lineheight=.7,color="black")+
  scale_fill_gradient(low = "red",high="white")+
  theme(panel.background = element_rect(fill="#445566"),panel.grid = element_line(color="#445566"))

  1. FDRS, Federation-wide Databank and Reporting System

r nerdvana

This blog by Steve Powell is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License, syndicated on r-bloggers and powered by Blot.
Privacy Policy