|||
Another interesting discussion on the M&ENews mailing list - does it make sense to try to measure progress on the highest levels of a logframe?
A couple of opinions -
True, it is often hard to attribute changes in “higher-level” items to our project. But we will often still want to monitor changes if we can. So if a school has introduced a program to reduce truancy, the staff will surely want to monitor truancy level, even if they know it is only to a certain extent within their power to influence. In other words, indicators can be for actual monitoring (in the ordinary sense of the word) as well as for attribution. I guess this is was the original idea behind the M of M&E, which the pseudoscience of logframes sometimes obscures - the idea of keeping an eye on progress towards an important outcome. This “keeping an eye on progress towards the outcome” is about a lot more than just comparing baseline and endline scores and is, I think, one of the marks of real, effective management. It is something we do instinctively when we really care about reaching the outcome. And we do it just the same whether we have 100% control or only 5% control over the outcome, i.e. regardless of whether success or failure can be totally attributed to our efforts. Ordinary life outside the world of projects and logframes are full of this natural monitoring - just think of what we have to keep an eye on when we are bringing up children, or running a school or a business, or managing a national bank. So if a program plan or logframe is a living, day-to-day tool to help our project succeed, it will help us visualise what we are trying to reach and give us a handful of key, usually low-tech ways to regularly test what progress we are making and get warning signs if we are getting off track. Anything else in our plan is dead weight or “logframe bloat”.
What does a parent keep an eye on to tell if a child is getting too tired to finish a task? Now those are indicators and that is monitoring. Who said you need a PhD for M&E? Parents don’t usually even have program plans and yet are experts at using complex, sensory data to monitor and manage progress towards outcomes.
So - I would say that the primary function of a logframe or program plan should be to help us with real-life, daily monitoring of progress, - and the attribution function is secondary and is derived from the primary function.
In practice, we try to judge performance even where attribution is difficult because control over the outcome is less than 100%. We are always judging the performance of politicians, school directors, managers and so on, though we are quite well aware that their successes and failures are only partly to be attributed to their efforts. We take note of the baseline and endline data but we hopefully don’t take too much notice of it. And interestingly, we rarely complain that we only have a sample of one or that the counterfactuals are insufficient.
One more thing - it is true that changes in “higher levels” sometimes involve changes in attitudes, beliefs etc (which are supposed to be hard to measure objectively, giving us a reason not to monitor progress on them). But it is a myth that they always do. To repeat the same example, reducing truancy is an important behavioural outcome which might have several layers below it in a school action plan. But it is not hard to measure. The MDGs are, I guess, “high-level outcomes” (to continue with this not very helpful language of hierarchical levels) but they are very concrete and not particularly hard to measure.