The Use of Indicators – “Audit Culture”
April 18, 2012 § 1 Comment
For those active or interested in the fields of international development and human rights, the uprising of indicators in recent years has surely been noted. We now have indicators to measure Multidimensional Poverty, human trafficking, economically active children, time required to start a business and hundreds of other phenomena. “Indicators are a political technology that can be used for many different purpose, including advocacy, reform, control, and management,” says Sally Engle Merry, an anthropologist from New York University in her new paper, Measuring the World : Indicators, Human Rights, and Global Governance. They are the quantification and standardization of complex data and events across many individuals and communities. They help us to formulate policy and garner decisions in a way that is (intuitively) less subjective than politics or personal values. For example, many institutions decide to invest, to sanction, or to remove support from a given community depending on that community’s level of X as determined by Y indicator. If a nation ranks too low on the US Trafficking in Persons report, for example, they can be subjected to harsh sanctions and public chastisement.
However, “as the world becomes ever more measured and tracked through indicators, it becomes increasingly important to sort out the technical and political dimensions of this new technology.” Engle elucidates this point by noting the fact that the power of indicators to standardize and thereby compare information across different communities is also their achilles heel; the process of standardizing data to fit a pre-specified indicator definition can dissolve many of the indiosyncracies that are inherent within a community. For example, an attempt to measure “child marriage” will rely wholly upon the definition of marriage; but when cultures practice marriage differently (i.e. some include cohabitation, and some do not) then standardization become a process of political decision making.1 Another example of this is the indicator for electrification: if a village has electrical poles and a single drop to light one bulb, does it qualify as being electrified? And what if the electricity is only there for an hour a day? This may sound trivial, but there are NGOs and other organizations that won’t work in a village if it already has access to electricity. Deciding how and when to submerge a community’s idiosyncrasies is, ultimately, a subjective but very powerful decision.
Engle’s paper gives greater detail into the history of indicators and the hesitancy with which we should approach them. She is not condemnatory–but rather, cautious. “How indicators are named and who decides what they represent are fundamental to the way an indicator produces knowledge” says Engle, and because indicators are a technology not only of knowledge production, but also of political decision making (e.g. where foreign aid should go, which countries are most promising for business investment, what nations ought to be sanctioned), we must be careful and aware of the subjectivity inherent in them.2
In other words–don’t swallow stats and indicators hook-line-and-sinker simply because they are published by well-known organizations. Be critical of what you read and expect transparency. Indicators may be a very helpful tool for policy construction and evaluation, but that doesn’t mean they are perfect…or in many cases, useful.
1 Read Engles further discussion of this topic of marriage on page 4 of her report. “One UN staffer sighed and noted that marriage is very complicated. Despite these complexities, they settled on cohabitation. I have since pondered this choice, thinking about the difference it would have made were another criterion chosen and wondering how the decision was made and by whom. What were the criteria? Was it the availability of data? To what extent was this decision based on a theory of early marriage and particular health or social problems?”
2Akin to this message of subjectivity within the construction of indicators is the message of subjectivity within their application. The US State Department Trafficking in Persons (TIP) report, for example, has constantly been critiqued for its tendency to place US political allies on higher “tiers” than political foes (even those with potentially better records).