It is perhaps still too soon to say — since the New Yorker came out on Monday and this is just Thursday — but I think that the zeitgeist is finally turning to serious consideration of the decline effect. Which is great news for me, and for people like Jonathan Schooler.
First, let me explain for those who haven’t read Jonah Lehrer’s article, what the discussion has so far been about. Lehrer has in effect merged two distinct intellectual streams. The first is a tradition going back to Joseph Banks Rhine, the eminent ESP researcher of the 1930’s. He coined the term ‘decline effect’ to describe the tendency for science experiments to wind up being impossible to replicate. Rhine’s test subjects tended to start off showing some degree of psychic talent, guessing card values well above the expected average. Then gradually that talent would fade away.
Skeptics merely rolled their eyes, saying the talent had been an illusion to begin with, or else poor controls on the experiment allowing the subjects to cheat. But decline is a much more robust phenomenon than this. Jonathan Schooler, who is quoted at some length in Lehrer’s article, became interested in this issue when his own path-breaking 1990 study in memory proved impossible to replicate. Worse, each successive attempt only brought his subjects’ scores lower. The same thing happened when colleagues tried to test his findings. The more Schooler talked with other scientists, the more stories emerged of the same thing happening to them, in a half-dozen unrelated fields Science is awash with experimental results that cannot be replicated, that just “wear out”.
So Schooler and a colleague replicated Rhine’s psychic studies using 2,000 undergraduates, only this time hoping to replicate Rhine’s failure. They did, on a massive scale. Many subjects seemed to have genuine psychic ability, or anyway uncanny luck in guessing. Then their luck “wore out”.
The second intellectual stream has been a quiet but growing anxiety among pharmaceutical companies about the failure of promising new drugs to maintain their effectiveness in large-scale trials. Over and over, the pattern would repeat: A drug that had massive promise when administered to 50 test subjects lost much of its advantage over placebos when administered to another 500. Then in clinical use, with thousands of patients, it proved to be not quite as good as the existing drugs in that category. One can find quotes from senior industry people saying that they are worried about the statistical framework they are using — worried, in other words, about the validity of the scientific method as it presently stands. When a billion-dollar company is prepared to question the scientific method, even obliquely, you know the results must be pretty weird.
What Jonah Lehrer has done is very exciting and path-breaking. He has brought the story of Rhine and the story of the angst of Big Pharma together for the first time. He has posed the question in a really new and sweeping way. Now, of course, we need some kind of answer.
Good thing I got started on this book a decade ago . . . seems like I’m just barely in time.