Scandal or yawn? Simpson’s Paradox hits the IFRC

A short description of the post.

Steve Powell
2019-01-14

Introduction

I’m working with colleagues from the IFRC on the 2019 edition of the Everyone Counts Report, which is a really interesting annual publication from the IFRC database which includes the data from 190(!) Red Cross and Red Crescent National Societies across the world.

We are interested in whether Red Cross and Red Crescent volunteers are insured (by the National Society) or not. This chart shows that there is a really crass age effect: older volunteers are more likely to be insured.

What a scandal!

In fact, this is a really good example of Simpson’s Paradox. Individual National Societies in general don’t have this bias towards insuring older volunteers. The effect happens because those National Societies which are more likely to have older volunteers (e.g. Japan) are also more likely to insure their volunteers – regardless of their age. Here for example is the same graphic, just for Mexico, a National Society with, in contrast, many young volunteers but also no age bias.

What a yawn!

But the point is, how do we know as analysts which angle to take? Should we control for National Society membership or not?

After decades of struggling with this “paradox”, we now have a solution in the form of Judea Pearl’s “backdoor criterion”. To apply the criterion, we draw this diagram. We are trying to estimate the red arrow.

The diagram tells us that National Society membership influences both variables and so we have to control for it: that means, we have to calculate the connection between age and insurance status at the level of individual National Societies and average these connections out.

When we do this, we find that the corrected influence of age on insurance status is small. The policy relevance is low.

But what Pearl keeps telling us is that we have to be able to draw the causal diagram before knowing what corrections to do. Because the same numbers could have been produced by a different causal situation. Imagine if the volunteers came from another planet and were assigned to National Societies. In this case it would be a serious error to control for National Society membership because it is a mediator which helps deliver the effect of age. Pearl’s backdoor criterion tells us that.

In this (fictional) case there would be policy relevance, because maybe we could influence this (fictional) age-biassed assignment to National Societies.

Practical take-home point

Global databases are full of country-level effects like these. In order to know how to analyse and interpret them (real or spurious??) we have to think causally.

If you’re interested to find out more about Judea Pearl’s work, you might like to try my review in JMDE.

I’m still not sure that what I said above quite gets to the bottom of the situation, …

References

Citation

For attribution, please cite this work as

Powell (2019, Jan. 14). Causality and more - ideas by Steve Powell: Scandal or yawn? Simpson's Paradox hits the IFRC. Retrieved from http://www.pogol.net/ideas/posts/2019-01-14-scandal-or-yawn-simpsons-paradox-hits-the-ifrc/

BibTeX citation

@misc{powell2019scandal,
  author = {Powell, Steve},
  title = {Causality and more - ideas by Steve Powell: Scandal or yawn? Simpson's Paradox hits the IFRC},
  url = {http://www.pogol.net/ideas/posts/2019-01-14-scandal-or-yawn-simpsons-paradox-hits-the-ifrc/},
  year = {2019}
}