|||
There is an interesting discussion on the xceval mailing list right now about value and measurability in development.
Seems to me there are four different claims here. Two are reasonable, two are shocking.
At the start there is what we would now see as an extremist positivist stance in the philosophy of science: the extreme pro-measurement claim that nothing which is not measurable exists or has value. Nowadays I don’t think there are many people who after reflection would agree with that stance. But nevertheless too many years of working with a measurement technology, whether in the hard sciences or development, can lead one to passively acquiesce to it (hence, “you become what you measure”). Sometimes the World Bank might seem to be doing this.
Frustration with this stance might lead one to the equally absurd opposite claim that “those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable”. This is the claim that started the discussion and which was attributed to Andrew Natsios. You could call this the extreme anti-measurement claim, that measurable is the opposite of valuable, at least in development. One black-and-white claim begets another. Isn’t this absurd? Frustration with the original extremist stance is understandable but to want to deny it by turning it on its head is just adolescent. After all, there are libraries full of fairly well-established interventions which are both transformational and measurable, like intelligent investment in teacher in-service training. Unfortunately I fear there are legions of people working in development who out of frustration with the first claim give up and blindly opt for the second.
You could call Einstein’s “Not everything that counts can be counted and not everything that can be counted, counts.” the moderate claim. (This was brought into the discussion by Kita Louisa from Nairobi). This is surely a more mature response to the first extremist claim without falling into the trap of the second: measurable does not equal valuable, but nor are they opposites.
But in the end I agree with Kita Louisa who makes a more modest but more useful fourth claim: ease of measurement bears no relation to value. (This is closer to what Natsios actually said, http://www.cgdev.org/content/publications/detail/1424271, though his more moderate claims get quoted in a more extreme form, which is interesting.) Our attempts to measure value inevitably end up focusing on whatever is easier to measure, and those things tend to be at what we in development often call a “lower level”, which means we often give up on the things at a higher level which are also valuable.
I wouldn’t even say that higher level things tend to be more valuable. A vaccination is quite low level, pretty easy to measure. Its ultimate value depends on the rest of the system, but that is true for higher-level things too. But higher-level things do tend to be harder to measure.
In other words, we need to invest more time and courage into trying to conceptualise and report higher-level outcomes. (This is what most mainstream social scientists have been involved with in one way or another for at least the last few decades, fallibly, but not without some success).
Whereas both extremist positions advocate not even trying.