There’s a Name for the Big Flaw in Our Obsession With Assessment and Metrics

by Pacific Standard Staff
from Pacific Standard

We live in an age obsessed with assessment. Tough-minded CEOs and numbers-driven wonks are forever looking for the perfect metric: the most objective way to define, measure, and reward success in a given arena. But there’s a tragic flaw in this hardheaded approach, and it’s known as Campbell’s Law.

Here’s the gist: The more a given metric—say, a national college ranking—is used to evaluate performance in some domain, the less reliable it becomes as a measure of overall success. Why? The people whose performance is being measured will neglect other parts of their job just to focus on boosting the relevant numbers, sometimes to the point of cheating. The chosen metric will inevitably “distort and corrupt the social processes it is intended to monitor,” suggested the social psychologist Donald Campbell back in 1974.

These days, the folks who cite Campbell’s Law most loudly and often are opponents of high-stakes testing in schools. (The phrase “teaching to the test” is basically a gloss on Campbell’s Law.) But other examples abound.

The focus on “body counts” in the Vietnam War, for instance, shifted attention away from other important metrics, like control of territory. Police departments in the 1960s that rewarded high arrest rates managed to book more suspects, but often failed to improve public safety. And one current theory holds that skyrocketing health care costs result from over-reliance on one metric—malpractice suits—to gauge the reputation of physicians, thus encouraging them to prescribe extra tests and treatments.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s