2012 December 01 Saturday
Why We Do Not Demand Prediction Accuracy
Lately I've been reading articles and books on why predictions are hard (e.g Dan Gardner's Future Babble), the incredible inaccuracy of the vast bulk of predictions, even and especially by experts, and how people are unaware or disinterested in all this inaccuracy. Just came across a 2011 article by economist Robin Hanson on Cato Unbound where he responds to Dan Gardner and Philip Tetlock about how we can get better expert predictions. Hanson points out all the other things that people want from listening to famous commentators aside from accuracy.
Consider first the many possible functions and roles of media pundits. Media consumers can be educated and entertained by clever, witty, but accessible commentary, and can coordinate to signal that they are smart and well-read by quoting and discussing the words of the same few focal pundits. Also, impressive pundits with prestigious credentials and clear “philosophical” positions can let readers and viewers gain by affiliation with such impressiveness, credentials, and positions. Being easier to understand and classify helps “hedgehogs” to serve many of these functions.
Affiliate yourself with noted academics as a way to do status signaling and demonstrate in-group membership.
Second, consider the many functions and roles of academics. Academics are primarily selected and rewarded for their impressive mastery and application of difficult academic tools and methods. Students, patrons, and media contacts can gain by affiliation with credentialed academic impressiveness. In forecasts, academic are rewarded much more for showing mastery of impressive tools than for accuracy.
Finally, consider next the many functions and roles of managers, both public and private. By being personally impressive, and by being identified with attractive philosophical positions, leaders can inspire people to work for and affiliate with their organizations. Such support can be threatened by clear tracking of leader forecasts, if that questions leader impressiveness.
Measurement of accuracy gets in the way of people getting the benefit they expect from listening to famous opinionators. The truth is just not that popular.
What's the problem with all this: Our need for prediction accuracy is increasing. We have technology, capital, and living standards that enable us to make much larger mistakes than in the past (e.g. factory fishing ships sweeping oceans of fish, massive burning of coal, large scale tropical deforestation, large population increases, nuclear weapons, genetic engineering of organisms). We now make global mistakes rather than just local or national mistakes. The consequences of our mistakes are therefore larger. At the same time, the global scale of the mistakes makes the incentives for continued mistakes driven by a Tragedy of the Commons.
We've also lost a number of benefits we had going for us as parts of the human race began escaping the Malthusian trap as industrialization took off in the 19th century. Notably, we are experiencing a reversal of important benefits that natural selection used to deliver to some segments of the human race. As we rip thru large amounts of natural resources and automate the lower IQ folks out of jobs the competition for remaining natural resources is going to become more fierce. I worry that the future world economy will become dual-tiered. Resource owners may do most of their trading with robot factory owners and the smart people who design new robots and products. Everyone else will be either figuratively or literally on the outside looking in.
By Randall Parker at 2012 December 01 11:06 AM
We are living so well right now, relative to historical standards, that we can afford to make lots of mistakes. Our GNP could be cut in half, and while we might feel poor, we would still have plenty to eat (only 2% of us are needed to grow food for everyone). I'm much more worried about our vulnerability to EMP, which could send us back to the Stone Age. There is no way we could support 317 million people without our electrical infrastructure.
"people are unaware or disinterested in all this inaccuracy"
You mean "uninterested." A disinterested person is aloof from the fray, having no stake in the outcome.
Robin Hanson's theory is very useful for contemplating the controversy surrounding global warming.
Much of mainstream/consensus/establishment climate science is of relatively poor quality. Many papers contain errors of commission and omission that would not be seen as acceptable in other high-stakes physical science fields. On the other hand, most "skeptical" arguments are uninformed or incomplete. Hard-core "skeptics" give credence to theories that defy textbook physics; google "sky dragon climate" for a prominent example.
Razib Khan and others have discussed a Puritan/Cavalier divide in American society, based on the settlement patterns of European groups into North America. In this respect, "mainstream climate science" has a Puritan moral core -- we will be punished unless we turn aside from our sinful, profligate ways and return to the proper worship of God/Gaia. "Skeptics" exalt the Free Market; as Cavaliers, they object to the Puritans' demands for redemption via communal action and thus for restriction on individuals' autonomy.
Unsurprisingly: the use of these aesthetic positions to frame analysis of weather records, computer modeling, heat transfer, and paleoclimate reconstructions leads to some fairly strange discussions of technical issues.
Even if the climate researchers are doing very high quality work they are still working with systems that are so complex that their predictions are bound to have big errors. The unfortunate thing about this is that we really need to know how we are impacting the environment and what will happen differently as a result. But the only way we are going to find out for sure is to live into the future decades when this will all play out.
We can know what CO2 does in terms of CO2 absorbing IR. That should give us cause for concern. It is certainly one of the things I worry about for the future. But how severe will be the effects of CO2 emissions? Don't know. How much will we emit? Hard to tell given unknowns about such things as future production costs in oil shales and natural gas shales, price declines in photovoltaics, costs of other raw materials, changes in fertility, and huge numbers of other factors.
We cannot know how much we'll emit because that's a decision we have yet to make. We cannot know the exact consequences of any given atmospheric CO2 level because there are many feedback loops, some positive, some negative, and we don't know, to sufficient precision, how strong they are.
What we do know is that there is a real chance of getting into some positive feedback loops that turn out to be strong enough to do us serious harm. For instance, reduced arctic sea ice driving melting of the Greenland ice sheet, and open water at the poles during summer making the water warmer thus making for more open water.
Price declines in photovoltaics, if they can achieve another factor of 4 or so, will make this whole debate moot because we'll transition from coal to solar for simple economic reasons. That's one of the reasons we cannot know how much CO2 we'll produce: it depends in part on the pace of our solar R&D.
My recommendation: push the pace of said R&D. And likewise with wind. Success here will take us off the hook. We won't have to make scary decisions with big economic downsides on one hand and big climate downsides on the other.