|||
This is a blog post I’m working on for the AEA365 blog, we’re going to do a series on the Theories of Evaluation TIG.
Hi, I’m Steve Powell, a freelance evaluator. I’m really interested in theories of, about, and within evaluation, and the conceptual headaches they can bring with them. Here I’m posting a couple of evaluation puzzles which may or may not give you a headache, make you smile, or even perhaps have a light-bulb moment.
Sometimes as evaluators we are so busy with practical challenges that we don’t have much time to worry if there is sufficient agreement about the meaning of the words we use, even when we use difficult words like “attribution”, “impact” or “intention”.
Puzzles and paradoxes have often been used in philosophy, from Zeno and Zen to the Sufi mystics, to help us question and sharpen up our concepts. I’ve written a few evaluation puzzles (http://www.pogol.net/tagged/puzzles) which I have found useful in workshops to get people realising that some of the ideas we use when doing evaluation stand on quite shaky ground.
Below are two of them. If you know of any similar evaluation puzzles or paradoxes, I’d be really interested in hearing about them.
Two NGOs prepare ships to help people stranded after a tsunami on a remote island.
NGO A is well-prepared and sets sail in time.
NGO B has barely enough equipment and leaves too late to do any good.
A freak hurricane sends ship A back to base but slings ship B miles ahead.
Ship B reaches the island and just manages to save many lives.
Ship A still manages to arrive on time but its help is no longer needed.
Which NGO was most effective? Which had the most impact? Which should be praised the most? Did NGO A fail?
A billionaire left 10 million EUR in his will to establish a trust, with instructions that it should be used to “just do good”.
During the ten years since then, support for same-sex marriage has shifted from 10% to 80% public approval, and almost all liberals are now in favour.
In the ninth year, the trust gives 1 million to a campaign for a law on marriage equality, which substantially contributes to the passing of the law in the tenth year.
We don’t know for sure, but most likely the billionaire, like most of his peers and friends, did not support marriage equality when he was alive. But most of his peers and friends now support it.
Now, in the 11th year, you are asked to evaluate whether the trust was used effectively and whether the activities were relevant to the intentions of the billionaire.