|||
This article is work in progress! Feel free to add a comment. There should be a pdf at this link.
After the terrible earthquake in Haiti in 2010, billions of dollars of aid money was launched at the country, much of it aimed at rebuilding homes destroyed. Yet by far the largest number of permanent homes were restored by local people on their own [@un_habitat_haiti_2013] without any significant direct help from outside. People were resilient - even after a terrible disaster, they were remarkably successful in rebuilding their communities. Not surprisingly, relief agencies are starting to say to themselves, and quite rightly “this is what we want! How can we work with the natural and spontaneous ability of people and communities to recover? What can we do to help those who don’t recover well learn from those who do? How can we, even before disasters, increase this natural resilience?” This kind of thinking mirrors what psychotherapists and social workers have been saying at least since the 1970s [@werner_children_1971]: rather than looking only at what makes some people become ill after psychological or social stress, let’s look at how some people survive or even thrive during and after hardship.
Resilience seems to be an idea whose time has come.
As a broad generalisation, one can say that across a wide range of disciplines, from ecology to psychology to development, a focus on damage in previous decades has been replaced by a focus on the other side of the coin - how damage is sometimes naturally avoided. This positive-sounding and very welcome shift in focus across many different disciplines seems to be part of a wider Zeitgeist.
One very powerful argument which explains this interest in resilience across quite different fields is the cost-effectiveness argument. These arguments have recently become quite strong in the development arena [@venton_economics_2012]: if we invest a little in resilience now, we will save a lot - in terms of lives and investments lost - when disaster strikes. As a relief or development agency we can actually save money because we won’t have to spend so much later because there won’t be so much damage to fix.
Many agencies - from the Red Cross movement to national governments or the UN - want to help individuals or communities to increase their resilience to disaster or underlying vulnerabilities. These agencies are searching, with increasing urgency, for ways to “measure resilience”. There are (at least) three different kinds of tasks facing them which would seem to require the measurement of resilience:
The very concept of resilience is a challenging one. One of the sharpest ways to expose and perhaps solve difficulties with a concept is to ask “hmm, resilience, … interesting idea … help me understand …. how would you measure that?” Being clear about the concept is pretty crucial if we want to discuss, plan, encourage, monitor and evaluate our work1. This should help with each of the three different kinds of resilience-measurement tasks outlined above.
Though nearly everyone I believe agrees that resilience as a concept is a bit slippery and that we might benefit from a bit of debate about it, the trouble is that this debate tends to focus around the best definition of resilience. So we end up with something like …
Resilience is the capacity of individuals/communities to recover from ….
… which is fine as far as it goes. But even armed with such a definition, we will find we are still not sure how we are going to measure resilience. Trying to get clearer about measurement can highlight, and also answer, further problems with the concept of resilience.
From this point on this article will be concerned only with resilience in the context of international disaster relief and development, as conducted by aid agencies. But I do rather suspect that some of what I am saying is also relevant to fields like individual development and social work.
Many agencies say their primary focus is community resilience. But what does that mean? Does it mean that we are going to primarily, or even solely, focus on community-level measures when considering resilience as an outcome? Or does it mean we are going to focus primarily, or even solely, on community-level responses to threats and hazards?
We know in order to explain how people survive, to explain the resilience of individuals as an outcome, from a systems perspective, we can’t draw our boundary around just the individual and leave it at that. There are many important factors which affect whether the individual survives and thrives which come from outside the individual - as simple as say the existence of a mobile phone network or a disaster committee (which in some sense are features of a community) or just the efforts of the person’s family (which are in a different sense the efforts of a group of people and so could in some sense perhaps be termed a community response).
Claim 0) In terms of protective factors, phenomena at only the individual level are not enough to explain the survival, the bouncing-back, indeed the resilience of individuals.
In terms of outcomes, we may also want to talk not only about measures at the level of individuals such as individual health (whether or not we aggregate them to the level of community or district) but also, say, the maintenance or destruction of community-level phenomena like infrastructure or community networks. So we might be literally interested not only in knowing how individuals survive but also in how communities or features of communities survive.
Still, to respond to the above claim merely by saying we are concerned with “community resilience” is a bit of a fudge. For one thing, it glosses over our feeling that there are different resilience mechanisms in not one but many concentric and overlapping systems like family, household, street and village work to protect well-being in different ways, and it is a bit of a stretch to call them all “community”.
In fact when we talk about “community resilience” this is probably shorthand for “let’s try to talk about resilience from a systems-level perspective in which we acknowledge a) that there are relevant protective features or resilience mechanisms at different systems levels beyond the individual and also b) that we may be concerned not only with the outcomes of individuals but also of larger systems such as households or villages in their own right.
In this paper I will try to keep things simple: I acknowledge the truth of claim 0, but will in general just write about “resilience” rather than specifically about “individual resilience” or “community resilience” - whatever that means.
After all, even those such as psychologists who are specifically interested in individual resilience (in the sense of outcomes) will have to take into account what kind of contribution is made by features of larger systems such as family and neighbourhood.
In this essay I will try to avoid addressing head-on the tricky issues about how different overlapping systems interact.
Definitions of resilience usually describe it as …
… usually picking on one of these three as a primary focus.
A very simple model of resilience would just consider two levels, capacity and outcome.
So resilience could be model with a function from 1) a set of preconditions and 2) hazard (actual occurence of a particular form of hazard) to outcomes.
The cases are individual entities exposed to some particular hazard.
outcome = f(preconditions,hazard)
You could say “there really is such a thing as (statistical) resilience” if we know enough about the function f to regularly identify values of the precondition variables which produce better outcomes for most levels of the hazard - “to shift the likelihood curve (from hazard to losses) to the right”.
This is equivalent to saying “some of the plates being sold at this shop are more resilient than others - look, the hardened ones from this factory mostly don’t shatter when dropped from this height”.
Nearly all models nowadays agree that there is more to it than this: some entities are more resilient than others because they respond in a different way to a hazard.
response = g(preconditions,hazard)
outcome = h(response,hazard)
Note that active resilience implies statistical resilience (because if g and h exist we can construct an f, like this outcome=h(g(preconditions,hazard),hazard)).
Masten [@masten_ordinary_2001] points out that in practice, resilience capacity does not bring about just any old type of response but a specifically adaptive one - it is assumed that there is a system which takes into account differences between the actual state or trajectory and a desired state or trajectory and makes compensatory responses. So the study of resilience becomes in part the study of adaptive systems.
If we really knew the function f, or the functions g and h, we could point to an entity with certain preconditions and say:
Look, this entity is resilient - we know that faced with certain hazards (…) it is likely to respond in this specific way (…) way which will probably lead to these improved outcomes (…).
Later on, we could say
Look, this entity is responding resiliently - right now it is being faced with certain hazards (…) and it is responding in this specific way (…) way, probably because it had specific pre-existing characteristics (…) which will probably lead to these improved outcomes (…).
Even later, we could say
Look, this entity has been resilient - it has these improved outcomes (…) because when it was faced with certain hazards (…) it probably responded in this specific way (…) way, probably because it had specific pre-existing characteristics (…).
But of course with humans and human communities, although we have some great data and some great research and a lot of good model-building, we don’t yet have that level of certainty.
It is possible to make arguments that any of these three is really primary.
Level 1: Capacity /preparedness: Resilience has to be understood as a preparedness, a pre-existing capacity, the intrinsic ability to respond differently even in different situations. Resilience is more than just the observation that some communities might have better outcomes in specific situations. This on its own is not resilience. You can drop a hundred identical plates on the floor and ten of them might not break but this is not resilience. And there is another issue behind this issue - are we right in thinking that resilience is an active capability which really needs “explaining” or is it just a statistical fact that if a lot of people are put under stress, some fare better and some not so well?
The idea that we all have (to a greater or lesser extent) an innate system, a kind of guardian angel, which may awake in times of our greatest need to guide us to recovery is a beautiful (and archaic) one. Still we don’t need to postulate specific resilience capacities - it might be enough to assume we, within our communities, have more general capacities to look after ourselves and one another which are enough to explain in particular “bouncing back from stresses and shocks” when it happens.
Level 2: Process /response / coping: Resilience can also be understood as a process, an actual response aimed at protecting wellbeing. What did those individuals, households or communities with better outcomes actually do to achieve those better outcomes? If they didn’t do something different - use different behavioural or cognitive strategies, for example - we would say the better outcomes were a fluke. Active coping can happen not just during and immediately after a hazard or crisis but also as a longer-term effort to return to (or even exceed) previous levels of well-being.
The crux of what I am saying is:
Claim 1) all of these levels do indeed constitute and define resilience, and yet none of them are enough without the others.
Each of these three levels - capacity, response and mitigated losses - form different stages in a kind of generic theory of change for resilience. Providing we can be reasonably sure about our theory of change, assessing any one of them is assessing resilience.
These levels form the outline of a causal theory: adequate capacity leads to good responses which help avoid losses, though we acknowledge that there are likely to be feedback loops as well.
But if we aren’t sure about our theory of change, then none of them will suffice. Each of them can answer the question “is this community resilient” independently, if we are sure enough that all three levels are working the way they are supposed to.
A metaphor:
Suppose we have water flowing through a canal. We can measure the water flow at the beginning, the middle or the end. As long as we are sure about our theory of change - that we know water isn’t being added or lost along the way3 - then these three measurements are equivalent. But if we aren’t sure about the water flows - about the theory of change - then we don’t have a question to answer. If we aren’t sure, then it certainly wouldn’t help to add up the three measurements into a “combined index” as is sometimes suggested for resilience. We would be better advised to do at least some informal research to find out what is going on.
These three kinds of measure complement one another. They should not be “added up” but interpreted side-by-side. Sometimes, it is good to resist the urge to aggregate. For example, if our relief agency was intervening to try to improve resilience, we would be encouraged by an improvement in capacity-level measures but then be concerned if losses were not mitigated; or conversely if losses seemed to reduce without any improvement in response or coping, we might want to rethink our theory of change or the need for our project.
Each level validates the others. For example:
Level 1: Suppose a community had seemingly good emergency response plans, say for floods, but when a flood actually came they didn’t use them but just panicked and many people lost their lives. We would say the plans (preparedness & capacity) on their own were not a very valid measure of resilience because the response and consequences in things we really care about, developmental losses, were disappointing.
Also … “capacity” is in turn a bit of a difficult concept when we think about how to measure it. It suggests a bit more than concrete preparedness. It is something latent which is behind behaviour and explains it, and yet is more than just behaviour…
Why not call capacity or preparedness resilience? but this is my point:
You can, if by capacity you mean something clearly in terms of some measurable markers, providing the markers you choose keep getting validated by the other levels. If they don’t, you would stop believing they are good resilience markers. So, to take a silly example, if you thought the capacity to dissociate was part of resilience capacity but you found that dissociation produces poor outcomes, you would change your mind wouldn’t you? Or you could try to dodge out by saying, ah, capacity for resilience is just whatever turns out to produce resilience outcomes. But then you aren’t really telling us anything.
Level 2: On the other hand, suppose a community actually put its flood response plan into action very efficiently when a flood came but it turned out there was still much loss of life and now the people are debating whether the plan was wrong4 and they were not doing the right things. We would say even the seemingly good response was not an adequate measure of resilience because the consequences in things we really care about, our goal-level indicators, was poor.
Level 3: Suppose six communities were exposed to the same flood and three came off quite well and the other three suffered badly - yet the quality of their preparations was quite similar and they all seem to have responded in roughly the same way. We probably wouldn’t say the first three communities were more resilient, we would probably just say they were lucky. So in certain circumstances, avoidance of loss alone would not be enough, we would want to be sure it happened because there was a loss-avoiding mechanism, i.e. resilience responses, which actually got used successfully. This idea that we aren’t satisfied in saying, “yes this is an example of a village behaving resiliently” until we have evidence of a mechanism is very strong in Realistic Evaluation [@pawson_realistic_1997].
Most often though, our information about how communities actually respond (the second level) is foggy at best, and our ability to assess which community fared better or worse in terms of goals for international development (the third level) is extremely limited - so in order to assess probable future resilience we rely on what we can see, our assessment of capacity (the first level): well-prepared and published disaster plans, a high level of trust and social networking, resolute and approachable leaders and so on. In this case, we might say “the level of preparedness & capacity is only a proxy, it is the response and/or the goal level which really counts” and we would be half right. But what I am arguing here is that resilience is a story with three sides to it and the quality of no one level considered on its own is enough for us to be sure that a community is, or has been, “really resilient”.
Or we could say: resilience is like a stool with three legs - with only one or two it will fall over.
Now, somebody might say “aha, but behind those three routes to assessing resilience there is actually a real but hidden toughness in people and communities which actually governs and explains good scores in each of these three phases of the resilience process”. That might be true and it is certainly an interesting research hypothesis. But at this stage of our ability to “measure resilience” we are better advised to stick more closely to what we can observe on these three explicit levels.
Claim 2) approaches for measuring resilience usually need adapting according to the type of hazard, in particular whether the hazard is:
I won’t say much here about the fourth column, “generic resilience” aka “community backbone”; it really deserves a chapter all on its own.
We can now combine Claim 1 and Claim 2 to get a grid where we can place possible aspects of resilience and their corresponding measures.
Scheme for organising resilience measures, with some basic examples. See below for explanation of the X symbol
Please note the first level of resilience is shown at the bottom and the table can be read from the bottom up to better reflect the structure of most planning frameworks (see below). So the lower levels support the higher levels.
So a candidate measure of resilience - whether it is knowledge of malaria risk or healthy eating - can be assigned to one of the cells5.
However, there is an additional complication which is revealed by our proposed schema.
Seen over time, a goal-level measure like mortality from floods can fall not only because communities have become more resilient but also because the incidence of floods has dropped.
Interpreting goal- and response-level measures for shocks and variable stresses - marked with an X in the schema - (and in a different way unforeseen or generic hazards) does depend also on the presence/magnitude of the hazard.
Only when the level of a shock or variable stress is kept constant can we say directly that a community with a good goal-level measurement is more resilient than one with worse scores. This is another way of saying that resilience at this level is a double difference.
Responses to shocks and variable stresses are particularly difficult to measure because it can be difficult to start measuring how the community is responsing when a hazard is actually happening, e.g. do people actually use the disaster plan, or do people succeed in selling livestock for cash? Only in the case of constant hazards - only in the right-hand column of our table - can we more easily plan our measures in advance.
(Finally, judging the contribution of a project to improving resilience to shocks and variable stresses is particularly tricky because it involves the kinds of counterfactual mentioned above as well as the counterfactual challenges familiar to program evaluators: what would have happened without the program? This is the double difference. Only a particularly sophisticated research design would be able to answer these questions with any degree of rigour.)
So our 3x4 table can provide a kind of measurement or research framework for resilience. Each of the cells in the table needs a different approach to measurement - not just measuring different things but measuring them in different ways. Agencies would, very understandably, really like to have just a single tool, ideally a nice simple questionnaire, which they could apply to “measure the overall resilience of the communities”. We are starting to understand why they will be disappointed.
The three reasons for resilience measurement we identified earlier can each be addressed in different ways -
((to be completed))
The majority of projects and programmes rely on quite simple planning schema called logical frameworks or results frameworks which will be very familiar to development and relief professionals but are likely to be less familiar to other readers. They have different levels very reminiscent of the three levels in our table. The names of these levels differ from agency to agency but to cut a long story short we could call them outputs (things which a project can control quite well and can directly act on, such as building a bridge or improving a community’s emergency plans), outcomes (roughly, changes to behaviours which happen as a result of the outputs) and goals (the most valuable variables like survival/mortality which are valuable in themselves). These levels fit quite well with our table. By improving preparedness and capacity (outputs) we hope to improve coping/responding behaviour in a crisis (outcomes) which we hope will lead to reaching our goals of e.g. reduced mortality.
| Resilience levels | Logframe levels |
|---|---|
| 3) Losses avoided (in development outcomes) | Goals |
| 2) Response | Outcomes |
| 1) Capacity/preparedness | Outputs |
I have written much more about this in my work for IFRC in East Africa (add reference when released).
This is the beginning of seeing how the views I have been setting about above resonate or not with what we find in the current literature on measurement of resilience…
FSIN, the Food Security Information Network, recently responded to the pressing need to “measure resilience” with a new Working Group, the Resilience Measurement Technical Working Group, was set up. An initial report was recently released: @usaid_resilience_2014. The report sets out ten principles, all of which are modern and relevant contributions to the debate but none of which seem to address the philosophical headeaches surrounding the problem which I have tried to look at above.
The report adds to the welter of definitions of resilience: “Resilience is the capacity that ensures adverse stressors and shocks do not have long-lasting adverse development consequences” and also, quite neatly, as “a set of ex ante attributes and supports that should positively shift the likelihood function that describes the relationship between shocks and development outcomes, such as food security [@barrett_toward_2013].
The introduction sets the tone by quoting other experts who stress how difficult it is to “measure resilience”, citing Vaitla et al. (2012, p. 5) “academics and practitioners have yet to achieve a consensus on how to measure resilience” and [@frankenberger_ethiopia:_2007]. (2012, p.26):“[t]he continuous, complex and dynamic process of building resilience makes it inherently difficult to measure”.
I believe that the Claims 1 and 2 I present above go some way to opposing implicit assumptions in the majority of these works.
((to be completed))
Most of the major attempts to “measure resilience” e.g. [@sherrieb_measuring_2010], [@alinovi_livelihoods_2010]; [@fao_wfp_unicef_resilience_2012]; [@fao_integrated_2008]; [@sibrian_deriving_2008] suggest calculating a composite resilience index on the basis of adding up a number of subscales on education, say, or trust. Mostly they come from what we have called Level 1 (preparedness/capacity)
Hopefully my analysis above has shown that it will always be a mistake to construct any kind of composite index which mix up the different resilience levels. But even constructing an index on Level 1 makes two assumptions:
Assumption 3a) There are several different dimensions (e.g. strength of social networks, financial resources, etc) which are relevant to resilience and which can be given numerical scores.
Assumption 3b) These scores can then be added up into a single score expressing overall or total resilience.
Well, at some point in the future we might have enough evidence to say that different aspects of resilience are all in a sense part of the same thing; and then we might want to add them up. This is another occasion where it is good to resist the urge to aggregate. But it is also possible that the evidence would indicate that there are two or more quite separate domains. For example evidence might show that social capital and economic capital are both good predictors of how a community recovers from a crisis but they are not related to one another at all and they involve different mechanisms. It might be that, say, economic capital is good for recovering from some kinds of crisis and social capital is good for other kinds of crisis. It might even be that they work against one another to some extent. (And if we had made the mistake of prematurely combining them into a single score, we might never have discovered these two different connections.) In any case, until we know more it, it makes most sense, when we have different scores on dimensions which purport to be related to resilience, to keep them separate from one another, and to analyse them separately too.
We have seem some attempts to build relatively generic measurements - generic in any of these senses:
These kinds of measures will tend to be weak for some of the following reasons:
Households or individuals may be good sources for some of this information (do they have access to improved water sources) but not all of it (e.g. level of trust).
Measuring resilience to different types of hazard will often require different kinds of measurement at different sets of time points.
Measuring the different levels of resilience will also often require different kinds of measurement at different sets of time points. Trying to measure response variables in advance by asking how people would behave is always going to be problematic e.g. asking households or individual community members direct questions like “Do you think your community has adequate infrastructure to respond to crises”. These may be based around theoretical frameworks such as the different forms of capital, e.g. [@norris_community_2007].
Overall, these kinds of questions are quite general and will not be as sensitive to change as questions which are more specifically related to project outputs and outcomes. This means that even if your project has had a positive impact, these kinds of questions might not be sensitive enough to measure it.
So this sounds as if I am saying you can’t even get close to resilience measurement without some horrendous PhD-level research programme. But no…
But projects don’t need to try to solve the “problem of measuring resilience” by introducing generic measures which are likely to have poor validity, reliability and sensitivity. We saw already that the levels of resilience fit quite nicely into traditional project planning frameworks. Most projects addressing hazards will already be trying to influence preparedness, and through that response, and through that to mitigate losses if a hazard occurs. Most projects will have a range of good indicators at preparedness/capacity level which are well fitted to the actual context.
Improvements in any or all of the measures can be taken as an indication of improved resilience, as follows (taking into account also the issues with counterfactuals addressed below).
Levels 2 and 3 are harder to measure, especially level 3 because of the double difference.
As we saw above, the different answers to these questions should not be added up but compared with one another. But we do not expect small projects to be too concerned with providing formal evidence that their theory of change is accurate and therefore to try to measure and compare results on all three levels. Small projects will focus on level 1. Validating it against levels 2 and 3 means:
This kind of validation is difficult and firm evidence is best provided by interagency, focussed and dedicated research projects. But small projects should at least informally always be interested in, and looking for informal evidence of, improvements at levels 2 and 3, and they should gather and share evidence for levels 2 and 3 where they can and where it is cost-effective to do so. The data they do find for levels 2 and 3 will be very valuable because it is likely to be better fitted to the actual context they are working in than anything a researcher could dream up remotely.
((to be completed))
Big thanks to Maria Hagl for very useful comments on an previous version of this.
I am not saying at all that “if you can’t measure it, it isn’t real.” But I do say “let’s see if we can agree how to measure resilience to see if that helps us get a clearer idea of what we are talking about”↩
Here I am using the word “outcome” in its everyday sense which might or might not coincide with the specialised use of the word in the terminology of logic models or logframes.↩
or if we have a good estimate of gains or losses which we can allow for↩
and don’t forget, in real life we never reach certainty about what is actually the right course of action in a disaster. We are only more or less certain, and even experts often disagree.↩
in fact, some measures might appear in more than one place. So for example we might use the overall health of a community as a way to measure a good outcome after a shock, but at the same time if people are healthy one might argue they are also better prepared for a shock before it happens. So the same measure might in principle appear in the bottom row as well as the top row.↩
