Seven (Eight) Deadly Sins of Impact Evaluation
February 27, 2012 § 1 Comment
Recently, Matthew Forti published his “Seven Deadly Sins of Impact Evaluation” on the Stanford Social Innovation Review blog. I greatly appreciated Forti’s clear and concise rendering of some very apparent but oft ignored mishaps that occur when organizations attempt to assess the difference (or damage) their programs create in the lives of others. To Forti’s list I might also add, “Superiority.”1
There is a tendency among NGOs and impact evaluators to feel that their own expertise is superior to or omniscient of the opinions and desires expressed by their study’s community. Community-based participatory research (CBPR) can overcome this undemocratic fault by involving the community in various aspects of the research process (from question formation, to data gathering and analysis, and even to the interpretation and application of findings). In other words, CBPR is the egalitarian engagement of a researcher with all those who will be affected by their research findings.
Holman assessed the need for such egalitarian methods in his 1987 book “Research from the Underside,” in which he noted a logical injustice:
Social work is about helping the poor
Being poor is about lacking power
Social research is a form of power
And yet social research rarely involves the opinions of the poor2
Now, there certainly instances in which CBPR will not be feasible (e.g. for funding, time or safety constraints, or where the population is unaccessible). However, when CBPR is possible, it can lead to interventions that are better understood, more effective and more accepted by the communities they are seeking to help.
(For further elucidation of some of the benefits and detriments to CBPR, see Kreiger 2002….or look up one of the many books on CBPR).
1This could perhaps be tagged onto Forti’s “Isolation”
2For the sake of this post, “social work” equate to “social intervention” and “poor” equates to “those in need of some social good”
* Thanks to Brett Keller for posting Forti’s “Seven Deadly Sins of Impact Evaluation”