There’s been a bit of a blogstorm over the impending release of the sequel to Freakonomics, the obviously-titled SuperFreakonomics. The authors are being taken to task for allegedly questionable science and statistics work, accused of oversimplifying or distorting their results for the sake of contrariness. There’s good discussion and links on Language Log’s post on the controversy:
Overall, the promotion of interesting stories in preference to accurate ones is always in the immediate economic self-interest of the promoter. It’s interesting stories, not accurate ones, that pump up ratings for Beck and Limbaugh. But it’s also interesting stories that bring readers to The Huffington Post and to Maureen Dowd’s column, and it’s interesting stories that sell copies of Freakonomics and Super Freakonomics. In this respect, Levitt and Dubner are exactly like Beck and Limbaugh.
We might call this the Pundit’s Dilemma — a game, like the Prisoner’s Dilemma, in which the player’s best move always seems to be to take the low road, and in which the aggregate welfare of the community always seems fated to fall. And this isn’t just a game for pundits. Scientists face similar choices every day, in deciding whether to over-sell their results, or for that matter to manufacture results for optimal appeal.
In the end, scientists usually over-interpret only a little, and rarely cheat, because the penalties for being caught are extreme. As a result, in an iterated version of the game, it’s generally better to play it fairly straight. Pundits (and regular journalists) also play an iterated version of this game — but empirical observation suggests that the penalties for many forms of bad behavior are too small and uncertain to have much effect. Certainly, the reputational effects of mere sensationalism and exaggeration seem to be negligible.