by Sasha Alyson
You may have heard about the researcher who found that dart-throwing chimpanzees are better at predicting future events than human experts.
Memorable as it is, the statement isn’t true. The researcher, Philip Tetlock, has gone to great pains to squelch this soundbite, but once you’ve compared experts to dart-throwing chimpanzees, you can’t be surprised if the media grab that ball and run wherever they can with it. But if we can stop thinking about dart-throwing chimpanzees – and that won’t be easy, will it? – the study offers a lot of insights. In particular, it helps us understand one reason that development aid doesn’t work.
Over a twenty-year period in the 1980s and 1990s, Tetlock asked a broad assortment of experts, pundits, and forecasters to estimate the probability of certain events that might happen on the national or world stage, ranging from economic growth to leadership changes, then compared more than 30,000 forecasts against the actual outcomes. Tetlock found that as a group, experts did outscore the chimps (barely) but experts did no better than a group he describes as “attentive readers of the New York Times.” Breaking down the data, he found several noteworthy distinctions.(1)
First, the more famous the expert was, the worse that expert performed. When the media wants an expert, they don’t care about that person’s accuracy record. They go after someone who’s well-known and readily available to the press. That means somebody who has spent time honing their brand, rather than honing their wisdom. The media also prefers an expert who will produce good soundbites – grand statements — and deliver them with conviction. But those who are most eager to perform well for the camera are most likely to be wrong because they’re focused on their audience, not on the facts.
Second, Tetlock emphasizes that expertise does exist – in certain realms. Experts do best if they “work in learning-friendly environments in which they make short-term predictions about easy-to-quantify outcomes and get the prompt, clear feedback essential for improvement.” Veteran firefighters, poker players, and neonatal nurses all get fast, clear feedback if they screw up, and they develop strong skills at predicting what will come next. Their survival depends on it.
“Development “experts,” however, operate in precisely the opposite environment. Planning staff at the U.N., the World Bank, and other agencies assume they can predict the outcome of their various development projects. Again and again, reality has shown that they cannot, but the pseudo-experts do their best to hide this shortcoming. Besides being wrong a lot, they share another trait: They’ll never admit having been wrong. It’s difficult for a firefighter to cover up a mistake that leads to human deaths. In the aid industry, it’s second nature to hide the problems.
Two high-profile development projects illustrate this. Time magazine published a 2005 “End of Poverty” cover story about the world-famous economist Jeffrey Sachs; then U.N. gave him money and credentials for his Millennium Villages Project. When it turned out that his theories simply didn’t work in the real world, neither Sachs nor Time nor the U.N. asked what went wrong and tried to learn from it. Instead, they all changed the subject. Sachs produced a U.N. report about Global Happiness. Time turned over an entire issue to Bill Gates, to explain why the world was better than ever.(2)
A year later, the United Nations Development Programme gave credibility and a spotlight to Nicholas Negroponte for his “One Laptop Per Child” idea. When it turned out that this costly project did not achieve the educational gains that Negroponte had confidently promised — it achieved little or nothing, but at great expense — neither he nor the UNDP said so. They fudged as long as they could, then they let it fade from sight.(3)
Tetlock writes: “The three principals—authoritative-sounding experts, the ratings-conscious media, and the attentive public—may thus be locked in a symbiotic triangle. It is tempting to say they need each other too much to terminate a relationship merely because it is based on an illusion.”
These are just a couple of the most blatant and well-documented examples. What is clear is that we cannot rely on media coverage to help us explore the best ways to “lift people in developing countries out of poverty.”(4) Are we out of luck, then? Will we never figure out how to do all that lifting?
That’s asking the wrong question. Countries that call themselves “developed” have achieved that status not because outsiders told them the answers, but because those within each country were busy solving their own problems. Foreign aid consistently keeps control in the hands of distant foreign interests. If we in the West truly want to help poorer countries thrive, the first thing we should do is understand how often we get in the way, and stop doing so.
Notes and Sources
1. No actual chimps were used in this research. Tetlock asked various groups of people to make forecasts. For each possible forecast, he asked them to choose which of three outcomes was most likely: Change in one defined direction, change in the opposite direction, or things largely staying the same. The chimps were just a memorable way to represent a hypothetical group that was choosing at random. And the winners are…
► No group did particularly well.
► The experts outperformed the chimps “but their margins of victory were narrow.”
► “Attentive readers of the New York Times” did just as well – or better.
► So did a computer algorithm which merely predicted, for every item, “things will largely stay the same.”
► Experts with more confidence did worse than those with less confidence.
► Undergraduates at Berkeley did worse than the chimps.
2. Gates was the guest editor of Time‘s first issue of 2018; he invited his wife Melinda, his buddy Warren Buffett, and others to join him discussing how poverty was rapidly declining. And for the three of them, it was true. Jeffrey Sachs, however, wasn’t invited.
3. I’ve told the details in Whatever Happened to One Laptop Per Child?
4. Claiming to “lift people out of poverty” is a favorite self-important cliche of aid institutions. “More than 1 billion people have been lifted out of extreme poverty since 1990,” asserted the U.N., in declaring success for its Millennium Development Goals.
Except as noted, other quotes are from Expert Political Judgment: How Good Is It? How Can We Know? by Philip E. Tetlock, new edition, Princeton University Press, 2017. I also recommend:
The Wisdom of Crowds, by James Surowiecki. Groupthink gets a bad rap, and for good reason. But James Surowiecki shows that when people form opinions independently, based on diverse experience and information, and then those opinions coalesce, the group will routinely outperform the experts.
Thinking, Fast and Slow, by Daniel Kahneman. This widely-hailed book shows how easily our intuition – including the instinct to trust people labeled as experts – will lead us wrong. The author received a Nobel Prize in Economics for his research.
The Tyranny of Experts, by William Easterly. The former World Bank economist, now a leading aid critic, shows just how badly experts from the West have undermined the Rest.
Related stories
Left: Francophonie. By forcing children in its former African colonies to study in French, France thought it was spreading the glory of the French language. But students ended up learning neither French, nor much of anything else.
Right: “Bill Gates predicted the pandemic.” Bill Gates is eager to present himself as a visionary. But he didn’t predict the pandemic at all; he just knows how to work the media to hone his image. Here’s why that’s a problem.
Left: Groupthink often results when people follow pseudo-experts. It has led to many disasters — from burning down ancient peat forests, to compelling children in the South to attend schools where they learn nothing.
Right: Garbage in… UNESCO takes garbage data, runs it through a fancy formula, and claims expertise on the state of education around the globe. But UNESCO really has no idea where things stand.
Left: Millennium Villages Project. Superstar professor Jeffrey Sachs said the MVP would prove that Western aid could end extreme poverty in five years. It didn’t. But its failures offer insight how very little the Western “experts” understand world poverty.
Right: Willful blindness. As it tries to control school policies in the global South, the aid industry publishes piles of data about every subject except one: Are students learning anything? It doesn’t want to know.
Other stories about karma colonialism
Left: UNICEF preaches diversity. But for 77 years, UNICEF has ALWAYS had a white USA citizen in its top spot. That’s hypocritical. Racist, too? You decide. And it leads UNICEF to push Western interests.
Right: Why won’t they hire local talent? World Vision, UNICEF, Save the Children, and others issue frequent reports on how they’re helping people in distant lands. Why do they prefer to hire Western photographers for these reports?
Left: Stop Patronizing Us! A Kenyan woman working in the women’s rights sector tells of her disappointment at inequalities and hierarchical behavior she has found in that space.
Right: What would make a better future? There are ways that wealthier countries can genuinely help others, if they want to. It all starts with ending what one disillusioned World Bank economist calls “The Tyranny of Experts.”