Some Thoughts on Psi Phenomena or Supernatural Events (and Politics)

Steven Novella at the NeuroLogica blog posted a great discussion about what is going on when science finds what appears to be solid evidence of Psi phenomena or something supernatural. Novella writes:

“In 2011 Daryl Bem published a series of ten studies which he claimed demonstrated psi phenomena – that people could “feel the future”. He took standard psychological study methods and simply reversed the order of events, so that the effect was measured prior to the stimulus. Bem claimed to find significant results – therefore psi is real. Skeptics and psychologists were not impressed, for various reasons. At the time, I wrote this:

Perhaps the best thing to come out of Bem’s research is an editorial to be printed with the studies – Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi by Eric Jan Wagenmakers, Ruud Wetzels, Denny Borsboom, & Han van der Maas from the University of Amsterdam. .... They hit the nail absolutely on the head with their analysis.

Their primary point is this – when research finds positive results for an apparently impossible phenomenon, this is probably not telling us something new about the universe, but rather is probably telling us something very important about the limitations of our research methods.

.... Bem had previously authored a chapter in a textbook on research methodology in which he essentially advocated for p-hacking. This refers to a set of bad research methods that gives the researchers enough wiggle room to fudge the results, enough to make negative data seem statistically significant. This could be as seemingly innocent as deciding when to stop collecting data after you have already peeked at some of the results.

Richard Wiseman, who was one of the first psychologists to try to replicate Bem’s research and came up with negative results, recently published a paper discussing this very issue. In his blog post about the article he credits Bem’s research with being a significant motivator for improving research rigor in psychology:

Several researchers noted that the criticisms aimed at Bem’s work also applied to many studies from mainstream psychology. Many of the problems surrounded researchers changing their statistics and hypotheses after they had looked at their data, and so commentators urged researchers to submit a detailed description of their plans prior to running their studies. In 2013, psychologist Chris Chambers played a key role in getting the academic journal Cortex to adopt the procedure (known as a Registered Report), and many other journals quickly followed suit.”

Novella goes on to note that Bem actually participated in a large scale replication of his experiment using preregistration of his protocol to prevent p-hacking. Bem truly believed that his data proved ESP (extrasensory perception) was real and this major replication would confirm it. Unfortunately for Bem, the preregistration of the protocol did prevent p-hacking. It showed that the existence of ESP was not supported by the data as analyzed by the method specified in preregistered protocol. In short, ESP did not exist based on the research and data analysis protocols that were used in the replication experiment.

Bem had spent most of his research life trying to show that ESP was real. He refused to accept the results. Instead, he broke the analysis protocol and used a different statistical analysis on the data. That exercise resulted in a finding that the evidence showing the existence of ESP as a real phenomenon was ‘highly significant’. Bem had reverted back to p-hacking to get the result he desperately wanted.

This is a clear example of a trained scientist like Dr. Bem who is aware of the subtle pitfalls of doing science and coming to the wrong conclusion. He could not escape the trap his mind created. He wanted very much wanted ESP to be real. So, what chance does the average American non-scientist have in escaping the trap that relentless political dark free speech can so effectively create in their minds?

The point here is to try to exemplify how the human mind can effectively but subtly lead people to believe things that are false, even in the face of powerful contradictory evidence. It happens to scientists. It also happens to everyone else, me included.

The best defense? Adopting and applying critical thinking skills is probably about best one can do. It imposes a high cognitive load and I suspect that most people are unwilling to engage in it. That appears to be the case despite the fact that most people already do believe they are critical thinkers. That mostly false belief is yet another persistent illusion that dark free speech creates and maintains in the minds of many people, probably including me at least sometimes.