Academic predators

Today's topic, which comes to me via a long-time loyal reader of Skeptophilia, has a funny side and a not-so-funny side.

The link my friend sent me was to a paper called "The Psychometric Measurement of God," by one George Hammond, M.S. Physics.  In it, he claims to have used the methods of physics to prove that God exists, which would be a pretty good feat.  So I eagerly read the paper, which turned out to be an enormous mélange of sciency-sounding terms, evidently using a template something like this: "(big word) (big word) (big word) God (big word) (big word) (big word) (big word) matrix (big word) (big word) scientific measurement (big word) (big word) (big word) God exists q.e.d."

Don't believe me? Here's a representative passage:

I had already published in 1994 a peer-reviewed paper in a prominent journal pointing out that there was a decussation in the Papez Loop in Jeffrey Gray’s fornical septo-hippocampal system indicating that it regulated not only Anxiety as he said it did, but in a diagonal mode of operational so regulated his Impulsivity dimension.  In the brain the septum is located dead center in the “X” formed by the fornix thus regulating information to and from all 8 cubic lobes of the brain via the fornical Papez Loop.  Since then the septal area is also dead center in Thurstone’s Box in the brain I eventually realized that Gray’s septo-hippocampal system controls all 13 personality dimensions of the Structural Model of Personality!...  Meanwhile, factorization of this 4 x 4 matrix yields one, single, final top 4th order eigenvector of Psychology.  What could this factor be?...  [T]he final top factor in Psychology is in fact the God of the Bible.  Since this is a scientific measurement, God can actually be measured to 2 decimal point accuracy.

Please note that I didn't select this passage because it sounds ridiculous; it all sounds like this.

Or maybe, with my mere B.S. in Physics, I'm just not smart enough to understand it.

The fact that this is a wee bit on the spurious side is accentuated by the various self-congratulatory statements scattered through it, like "this is nothing less than awesome!" and "if you think discovering the gods is an amazing scientific turn of events, brace yourself!" and "my personal scientific opinion as a graduate physicist is that the possibility [of my being correct] is better than 1 in 3."  Also, the inadvertently hilarious statement that "evolutionary biology discovered the 'airbag theory' millions of years before General Motors did" might clue you in to the possibility that this paper may not have been peer reviewed.

But so far, this is just some loony guy writing some loony stuff, which should come as no big surprise, because after all, that's what loony guys do.  And there's not much to be gained by simply poking fun at what, honestly, are low-hanging fruit.  But that brings us to the less-than-amusing part.

The site where this "paper" was published is academia.edu.  My general thought has been that most .edu sites are pretty reliable, but that may have to be revised.  "Academia" is not only not peer reviewed -- it's barely even moderated.  Literally anyone can publish almost anything.

Basically, it's not a site for valid scientific research; it's purely a money-making operation.  If you poke around on the site a little, you'll find you're quickly asked to sign up and give them your email, and badgered to subscribe (for a monthly fee, of course).  I probably don't need to say this, but do not give these people your email.  It turns out there's a page on Quora devoted to the topic of academia.edu, and the comments left by people who have actually interacted with them are nothing short of scathing.  Here's a sampler:

• If you sign up, the people who upload the pdf files will give you exactly what it seemed like they would give you, a paper pdf that makes you sign up using another link, which is also fake!  If you ask to contact the person who wrote it, they will either ignore you or block you. Don’t sign up for Academia, because when you do they just take you to another link, which is ridiculous.  Academia is a public research company, they don’t review anything or enforce rules.
• I found it very unsettling that the ONLY permission they ask for is to….VIEW AND DOWNLOAD YOUR CONTACTS!  That was a SERIOUS tip-off to me that something wasn’t right.
• It’s a scam, they try every trick in the book to get you to sign up; according to them I must be one of the most famous people on the planet.
• I hate this site.  Looks like scammy trash.  I tried to sign up and after receiving my e-mail (use an account you don’t care about), then it looks like I can only proceed if I sign up for a bulk download account, and that costs money.  Fuck 'em.
• They are scammers trying to get your money.  They told me I was cited in more than 2k papers.  My name is not common and I don't participate in the academic world.
• Be careful with this.  Academia.edu was flagged by gmail and seems to have full access to my Google Account, not even partial access.  Given some of the other privacy and IP considerations with sharing your content on this site I would steer clear of it in future regardless - it’s basically a LinkedIn with similar commercial ambitions to make VCs a ton of money so there are the common concerns of “you’re the product” and “your content is now their content”.  Regardless this level of access to gmail is unwarranted and an invasion of privacy and was not clearly disclosed when I signed up (quick sign up to download a document).

So, the sad truth is that just because a site has .edu in its address, it's not necessarily reliable.  I always say "check sources, then check them again," but this is becoming harder and harder with pay-to-play sites (often called "predatory journals") that will publish any damn thing people submit.  From what I found, it seems like academia.edu isn't exactly pay-to-play; there's apparently not a fee for uploading your paper, and the money they make is from people naïve enough to sign up for a subscription.  (Of course, I couldn't dig into their actual rules and policies, because then I would have had to sign up, and I'm damned if I'm letting them get anywhere near my email address, much less my money.)  Even so, what this means is that the papers you find there, like the one by the estimable Mr. Hammond (M.S. Physics) have not passed any kind of gatekeeper.  There may be legitimate papers on the site; it's possible some younger researchers, trying to establish their names in their fields, are lured in by the possibility of getting their work in print somewhere.  Those papers are probably okay.

But as Hammond's "(big word) (big word) (big word) I proved that God exists!  I'm awesome!" paper illustrates, it would be decidedly unwise to trust everything on their site.

So once again: check your sources.  Don't just do a search to find out if what you're looking into has been published somewhere; find out where it's been published, and by whom, and then see if you can find out whether the author and the publication are legitimate.

It may seem like a lot of work, but if you want to stem the rising tide of false claims circulating madly about -- and I hope we all do -- it's well worth the time.

***************************************

People made fun of Donald Rumsfeld for his statement that there are "known unknowns" -- things we know we don't know -- but a far larger number of "unknown unknowns," which are all the things we aren't even aware that we don't know.

While he certainly could have phrased it a little more clearly, and understand that I'm not in any way defending Donald Rumsfeld's other actions and statements, he certainly was right in this case.  It's profoundly humbling to find out how much we don't know, even about subjects about which we consider ourselves experts.  One of the most important things we need to do is to keep in mind not only that we might have things wrong, and that additional evidence may completely overturn what we thought we knew -- and more, that there are some things so far out of our ken that we may not even know they exist.

These ideas -- the perimeter of human knowledge, and the importance of being able to learn, relearn, change directions, and accept new information -- are the topic of psychologist Adam Grant's book Think Again: The Power of Knowing What You Don't Know.  In it, he explores not only how we are all riding around with blinders on, but how to take steps toward removing them, starting with not surrounding yourself with an echo chamber of like-minded people who might not even recognize that they have things wrong.  We should hold our own beliefs up to the light of scrutiny.  As Grant puts it, we should approach issues like scientists looking for the truth, not like a campaigning politician trying to convince an audience.

It's a book that challenges us to move past our stance of "clearly I'm right about this" to the more reasoned approach of "let me see if the evidence supports this."  In this era of media spin, fake news, and propaganda, it's a critical message -- and Think Again should be on everyone's to-read list.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

Scooting past the reviewers

If there's one piece of advice I have for anyone trying to stay informed, it is: check your sources.

Unfortunately, these days, that takes more than just a quick look, or a recommendation from someone with authority.  After all, just two days ago Donald Trump tweeted that Fox News wasn't right-wing enough for him, that all of his faithful MAGA followers should troop on over to OANN (One America News Network),  that it was the only news source that was "fair and balanced."  Of course, this was transparent enough; in Trump-speak, "fair and balanced" means "willing to kiss Trump's ass on a daily basis."  OANN is a far-right outlet allied to sites like Breitbart -- and let's face it, anything to the right of Fox News isn't even within hailing distance of unbiased.

So "sounds like a reliable source" is itself unreliable.  As an example, take the paper that appeared last week, authored by Mathieu Edouard Rebeaud (University of Lausanne), Valentin Ruggeri (University of Grenoble), Michaël Rochoy (University of Lille). and Florian Cova (University of Geneva).  I won't tell you the title, but leap right in with an excerpt:

As the number of push-scooters has been rising in France, so has the number of push-scooters accidents.  Some of these accidents have proven to be deadly and previous YouTube™ and Dropbox© studies have warned against the deadly potential of push-scooters [1].  For a comparison, only three Chinese people had died from the novel coronavirus SARS-CoV-2 at the end of 2019 [2].  It is therefore important to reflect on the use of push-scooters through an accurate and ethical cost-benefit analysis.

Use and promotion of push-scooters have been advocated on the basis that they would contribute to the reduction and slowing of global warming.  In fact, the French scientific elite has been working on the subject and has recently argued that there was no proof of global warming, as he could not see the ice cap melt on his computer [3].  So, even if global warming was real, there are serious reasons to think that France is not affected, as global warming clearly stopped at the closed border [4].  Unfortunately, the debate is being polluted by bots, trolls and so-called experts funded by Big Trottinette to spread misinformation. Indeed, an independent study (in press on the third author’s Google Drive®) found a positive correlation between experts’ positive advocacy of push-scooters and the amount of money they received from Decathlon® (r = 3.14).  The fact that push-scooters are now a ‘generic’ means of locomotion that can be produced by anyone for a cheap price might lead people to the conclusion that no private interest is involved, but we’re not fooled, we know the truth [5].  So, it is important to diminish the increasing number of push-scooter drivers who are sacrificed on a daily basis.

The authors then go on to show that the way to combat the deadly push-scooter accident surge is through doses of hydroxychloroquine, which also shows promise in dealing with the Israeli-Palestinian conflict.  (Well, most of their research supported this.  They didn't have so much luck with Study 2.  "Study 2 was excluded from analysis and from this paper," the authors write, "as it did not provide informative results (i.e. the results we wanted)."

[Image licensed under the Creative Commons Alex Genz, Female rider on Egret One eScooter, CC BY-SA 4.0]

Also notable is that besides the four actual authors, there are also additional co-authors listed as belonging to places like "The Institute of Quick and Dirty Science of Neuneuchâtel, Switzerland" and the "Institute of Chiropteran Studies of East Timor," and one is called a "General Practitioner and Independent Seeker of Science" from Ankh-Morpork, France.

You may be thinking that this must have appeared in some kind of science spoof site like the brilliant Journal of Irreproducible Results.

You may be wrong.

This paper, titled, "SARS-CoV-2 Was Unexpectedly Deadlier than Push-scooters: Could Hydroxychloroquine be the Unique Solution?", was published in the Asian Journal of Medicine and Health.

(If you want to read it -- which I highly recommend -- you should do it soon.  My guess is that it'll be taken down before long.)

Sounds like a legitimate source, doesn't it?  You might be clued in that something was wrong if you noticed that the paper was submitted on July 24, accepted on August 11, and published on August 15 -- I say that notwithstanding the obviously goofy content from the title on, because most of the papers in the AJMH aren't blatantly off.  But if you look at stuff like this -- dates that make it clear that there was zero peer review involved -- there's no doubt left that this is one of those predatory pay-to-play journals, that will publish damn near anything if you give 'em some money.

Which, of course, was the point of the Rebeaud et al. paper.  It wasn't just to give us all a good laugh -- although it did that as well -- it was to shed some light on the way that predatory journals muddy the waters for everyone.

So back to where we started: CHECK.  YOUR.  SOURCES.  Which doesn't just mean a cursory "okay, it's a 'journal of medicine and health,' it must be reliable."  Take five minutes to do a quick search to see if there are any reviews or commentary on the journal itself.  The best thing is to find good sources that you know you can always rely on -- top-flight research journals like Science, Nature, Cell, Proceedings of the National Academy of Science, and PLOS-One, to name five -- as well as research-for-the-layperson journals like Scientific American and Discover.

If you get outside of those realms, though, caveat lector.  You never know what kind of lunacy you'll find, up to and including recommendations for taking hydroxychloroquine to prevent push-scooter accidents.

***************************

Fan of true crime stories?  This week's Skeptophilia book recommendation of the week is for you.

In The Poisoner's Handbook:Murder and the Birth of Forensic Medicine in Jazz Age New York, by Deborah Blum, you'll find out about how forensic science got off the ground -- through the efforts of two scientists, Charles Norris and Alexander Gettler, who took on the corruption-ridden law enforcement offices of Tammany Hall in order to stop people from literally getting away with murder.

In a book that reads more like a crime thriller than it does history, Blum takes us along with Norris and Gettler as they turned crime detection into a true science, resulting in hundreds of people being brought to justice for what would otherwise have been unsolved murders.  In Blum's hands, it's a fast, brilliant read -- if you're a fan of CSI, Forensics Files, and Bones, get a copy of The Poisoner's Handbook, you won't be able to put it down.

[Note: if you purchase this book using the image/link below, part of the proceeds goes to support Skeptophilia!]

Statistical fudging

The last thing we need right now is for people to have another reason to lose their trust in scientists.

It's a crucial moment.  On the one hand, we have the Intergovernmental Panel on Climate Change, which just a week and a half ago released a study that we have only twenty or so years left in which we can take action to limit the warming to an average of 1.5-2.0 C by 2050 -- and even that will almost certainly increase the number of major storms, shift patterns of rainfall, cause a drastic rise in sea level, and increase the number of deadly heat waves.  And it bears mention that a lot of climate scientists think that even this is underselling the point, giving politicians the sense that we can wait to take any action at all.  "It’s always five minutes to midnight, and that is highly problematic," said Oliver Geden, social scientist and visiting fellow at the Max Planck Institute for Meteorology in Hamburg, Germany.  "Policymakers get used to it, and they think there’s always a way out."

Then on the other hand we have our resident Stable Genius, Donald Trump, who claimed two days ago that he understands everything he needs to know about climate because he has "a natural instinct for science."  To bolster this claim, he made a statement that apparently sums up the grand total of his expertise in climatology, which is that "climate goes back and forth, back and forth."  He then added, "You have scientists on both sides of it.  My uncle was a great professor at MIT for many years, Dr. John Trump.  And I didn’t talk to him about this particular subject, but... I will say that you have scientists on both sides of the picture."

It bears mention that Dr. John Trump was an electrical engineer, not a climatologist.  And Donald Trump didn't even ask him for an opinion.

So we have scientists trying like hell to get the public to see that scientific results are reliable, and people like Trump and his cronies trying to portray them as engaging in no better than guesswork and speculation (and of having an agenda).  That's why I did a serious facepalm when I read the article sent to me a few days ago by a friend and frequent contributor to Skeptophilia, Andrew Butters, author and blogger over at Potato Chip Math (which you should all check out because it's awesome).

This article, which appeared over at CBC, comes from a different realm of science -- medical research.  It references a paper authored by Min Qi Wang, Alice F. Yan, and Ralph V. Katz that appeared in Annals of Internal Medicine, titled, "Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians."

If the title isn't alarming enough by itself, take a look at what Wang et al. found:

Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers...  [Among] 522 consulting biostatisticians... (t)he 4 most frequently reported inappropriate requests rated as “most severe” by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative.  These requests were reported most often by younger biostatisticians.

The good news is that a lot of the biostatisticians reported refusing the requests to alter the data.  (Of course, given that this is self-reporting, you have to wonder how many would voluntarily say, "Yeah, I do that all the time.")

"I feel like I've been asked to do quite a few of these at least once," said Andrew Althouse, biostatistician at the University of Pittsburgh.  "I do my best to stand my ground and I've never falsified data....  I was once pressured by a surgeon to provide data on 10-year survival rates after a particular surgical intervention.  The problem — the 10-year data didn't exist because the hospital hadn't been using the procedure long enough...  The surgeon argued with me that it was really important and pleaded with me to find some way to do this.  He eventually relented, but it was one of the most jarring examples I've experienced."

[Image is in the Public Domain]

McGill University bioethicist Jonathan Kimmelman is among those who are appalled by this finding.  "If statisticians are saying no, that's great," he said.  "But to me this is still a major concern...  Everyone has had papers that are turned down by journals because your results were not statistically significant.  Getting tenure, getting pay raises, all sorts of things depend on getting into those journals so there is really strong incentives for people to fudge or shape their findings in a way that it makes it more palatable for those journals.  And what that shows is that there are lots of instances where there is threat of adulteration of the evidence that we use."

It's not surprising that, being human, scientists are prone to the same foibles and pitfalls as the rest of us.  However, you'd think that if you go into science, it's because you have a powerful commitment to the truth.  As Kimmelman says, the stakes are high -- not only prestige, but grant money.  Still, one would hope ethics would win over expediency.

And this is a particularly pivotal moment, when we have an administration that is deeply in the pockets of the corporations, and has shown a complete disregard for scientific findings and the opinions of experts.  The last thing we need is to give them more ammunition for claiming that science is unreliable.

But it's still a good thing, really, that Wang et al. have done this study.  You can't fix a problem when you don't know anything about it.  (Which is a truism Trump could learn from.  "Climate goes back and forth, back and forth," my ass.)  It's to be hoped that this will lead to better oversight of statistical analysis and a more stringent criterion during peer review.  Re-establishing the public trust in scientists is absolutely critical.  Our lives, and the long-term habitability of the Earth, could depend on it.

 ***********************************

This week's Skeptophilia book recommendation is something everyone should read.  Jonathan Haidt is an ethicist who has been studying the connections between morality and politics for twenty-five years, and whose contribution to our understanding of our own motives is second to none.  In The Righteous Mind: Why Good People are Divided by Politics, he looks at what motivates liberals and conservatives -- and how good, moral people can look at the same issues and come to opposite conclusions.

His extraordinarily deft touch for asking us to reconsider our own ethical foundations, without either being overtly partisan or accepting truly immoral stances and behaviors, is a needed breath of fresh air in these fractious times.  He is somehow able to walk that line of evaluating our own behavior clearly and dispassionately, and holding a mirror up to some of our most deep-seated drives.

[If you purchase the book from Amazon using the image/link below, part of the proceeds goes to supporting Skeptophilia!]

What we've got here is a failure to replicate

I frequently post about new scientific discoveries, and having a fascination for neuroscience and psychology, a good many of them have to do with how the human brain works.  Connecting behavior to the underlying brain structure is not easy -- but with the advent of the fMRI, we've begun to make some forays into trying to elucidate how the brain's architecture is connected to neural function, and how neural function is connected to higher-order phenomena like memory, learning, instinct, language, and socialization.

Whenever I post about science I try my hardest to use sources that are from reputable journals such as Science and Nature -- and flag the ones that aren't as speculative.  The reason those gold-standard journals are considered so reliable is because of a rigorous process of peer review, wherein scientists in the field sift through papers with a fine-toothed comb, demanding revisions on anything questionable -- or sometimes rejecting the paper out of hand if it doesn't meet the benchmark.

[Image is in the Public Domain]

That's why a paper published in -- you guessed it -- Nature had me picking my jaw up off the floor.  A team of psychologists and social scientists, led by Colin Camerer of Caltech, took 21 psychological studies that had been published either in Nature or in Science and didn't just review them carefully, but tried to replicate their results.

Only 13 of them turned out to be replicable.

This is a serious problem.  I know that scientists are fallible just like the rest of us, but this to me doesn't sound like ordinary fallibility, it sounds like outright sloppiness, both on the part of the researchers and on the part of the reviewers.  I mean, if you can't trust Nature and Science, who can you trust?

Anna Dreber, of the Stockholm School of Economics, who co-authored the study, was unequivocal about its import.  "A false positive result can make other researchers, and the original researcher, spend lots of time and energy and money on results that turn out not to hold," she said.  "And that's kind of wasteful for resources and inefficient, so the sooner we find out that a result doesn't hold, the better."

Brian Nosek, of the University of Virginia, was also part of the team that did the study, and he thought that the pattern they found went beyond the "publish-or-perish" attitude that a lot of institutions have.  "Some people have hypothesized that, because they're the most prominent outlets they'd have the highest rigor," Nosek said.  "Others have hypothesized that the most prestigious outlets are also the ones that are most likely to select for very 'sexy' findings, and so may be actually less reproducible."

One heartening thing is that as part of the study, the researchers asked four hundred scientists in the field who were not involved with the study to take a look at the 21 papers in question, and make their best assessment as to whether it would pass replication or not.  And the scientists' guesses were usually correct.

So why, then, did eight flawed, non-replicable studies get past the review boards of the two most prestigious science journals in the world?  "The likelihood that a finding will replicate or not is one part of what a reviewer would consider," Nosek said.  "But other things might influence the decision to publish.  It may be that this finding isn't likely to be true, but if it is true, it is super important, so we do want to publish it because we want to get it into the conversation."

Well, okay, but how often are these questionably-correct but "super important" findings labeled as such?  It's rare to find a paper where there's any degree of doubt expressed for the main gist (although many of them do have sections on the limitations of the research, or questions that are still unanswered).  And it's understandable why.  If I were on a review board, I'd definitely look askance at a paper that made a claim and then admitted the results of the research might well be a fluke.

So this is kind of troubling.  It's encouraging that at least the inquiry is being made; identifying that a process is flawed is the first step toward fixing it.  As for me, I'm going to have to be a little more careful with my immediate trust of psychological research just because it was published in Nature or Science.

"The way to get ahead and get a job and get tenure is to publish lots and lots of papers," said Will Gervais of the University of Kentucky, who was one of the researchers whose study failed replication.  "And it's hard to do that if you are able run fewer studies, but in the end I think that's the way to go — to slow down our science and be more rigorous up front."

******************************************

This week's Skeptophilia book recommendation is from one of my favorite thinkers -- Irish science historian James Burke.  Burke has made several documentaries, including Connections, The Day the Universe Changed, and After the Warming -- the last-mentioned an absolutely prescient investigation into climate change that came out in 1991 and predicted damn near everything that would happen, climate-wise, in the twenty-seven years since then.

I'm going to go back to Burke's first really popular book, the one that was the genesis of the TV series of the same name -- Connections.  In this book, he looks at how one invention, one happenstance occurrence, one accidental discovery, leads to another, and finally results in something earthshattering.  (One of my favorites is how the technology of hand-weaving led to the invention of the computer.)  It's simply great fun to watch how Burke's mind works -- each of his little filigrees is only a few pages long, but you'll learn some fascinating ins and outs of history as he takes you on these journeys.  It's an absolutely delightful read.

[If you purchase the book from Amazon using the image/link below, part of the proceeds goes to supporting Skeptophilia!]

Contradicting the narrative

In her marvelous TED Talk "On Being Wrong," writer and journalist Kathryn Schulz describes a "series of unfortunate assumptions" that we tend to make when we find out that there are people who disagree with us.

First, we tend to assume that the dissenters are simply ignorant -- that they don't have access to the same facts as we do, and that if we graciously enlighten them, they'll say, "Oh, of course!" and join our side.  If that doesn't work, if the people who disagree with us turn out to have access to (and understand) the same facts as we have, then we turn to a second assumption -- that they're stupid.  They're taking all of the evidence, putting it together, and are too dumb to do it right.

If that doesn't work -- if our intellectual opponents have the same facts as we do, and turn out to be smart enough, but they still disagree with us -- we move on to a third, and worse, assumption; that they're actually malevolent.  They have the facts, know how to put them together, and are suppressing the right conclusion (i.e. ours) for their own evil purposes.

This, Schulz says, is a catastrophe.  "This attachment to our own sense of rightness keeps us from preventing mistakes when we absolutely need to," she says, "and it causes us to treat each other terribly."

I got a nice, and scientific, object lesson in support of Schulz's claim yesterday, when I stumbled across a paper in PLoS One by Clinton Sanchez, Brian Sundermeier, Kenneth Gray, and Robert J. Calin-Jagemann called "Direct Replication of Gervais & Norenzayan (2012): No Evidence That Analytic Thinking Decreases Religious Belief."  Apparently five years ago, a pair of psychological researchers, Will Gervais of the University of Kentucky and Ara Norenzayan of the University of British Columbia, had published a study showing that there was an inverse correlation between religious belief and analytical thinking, and further, stimulating analytical thinking in the religious has the effect of weakening their beliefs.

[image courtesy of Lucien leGray and the Wikimedia Commons]

Well, all of that fits nicely into the narrative we atheists would like to believe, doesn't it?  Oh, those religious folks -- if we could just teach 'em how to think, the scales would fall from their eyes, and (in Schulz's words) they'd "come on over to our team."  The problem is, when Sanchez et al. tried to replicate Gervais and Norenzayan's findings, they were unable to do so -- despite (or perhaps because of) the fact that Sanchez et al. used a much larger sample size and tighter controls.  The authors write:

What might explain the notable difference between our results and those reported by G&N?  We can rule out substantive differences in materials and procedures, as these were essentially identical.  We can also rule out idiosyncrasies in participant pools, as we collected diverse samples and used extensive quality controls.  Finally, we can also rule out researcher incompetence, as we were able to detect an expected effect of similar size using a positive control. 

One possibility is that Study 2 of G&N substantially over-estimated the effect of the manipulation on religious belief.  This seems likely, not only because of the data presented here but also because evidence published while this project was in progress suggests that the experimental manipulation may not actually influence analytic thinking...

Based on our results and the notable issues of construct validity that have emerged we conclude that the experiments reported by G&N do not provide strong evidence that analytic thinking causes a reduction in religious belief.  This conclusion is further supported by results from an independent set of conceptual replications that was recently published which also found little to no effect of analytic thinking manipulations on religious belief.

To their credit, Gervais and Norenzayan not only cooperated with the research of Sanchez et al., they admitted afterwards that their original experiments had led to a faulty conclusion.  In fact, in his blog, Gervais gives a wryly humorous take on their comeuppance, by presenting the criticisms of Sanchez et al. and only at the end revealing that it was his own paper that had been, more or less, cut to ribbons.  He says of Sanchez's team, "I congratulate them on their fine work."

He also included the following in his postscript:

FFFFFFFFFFUUUUUUUUUUUUUCCCCCCCCCKKKKKKKKKKKK!!!!

Understandably.  As Schulz points out, it's often devastating and embarrassing to find out that we screwed up.  Doubly so when you're a scientist, since your reputation and your livelihood depend on getting things right.  So kudos to Gervais and Norenzayan for admitting their paper hadn't shown what they said it did.

So not only is this a great example of science done right, in the larger analysis, it tells us atheists that we can't get away with dismissing religious folks as simply not being as smart and analytical as we are.  Which, honestly, is just as well, because it would leave me trying to explain friends of mine who are honest, smart, well-read, logical... and highly religious.  It supports the kinder (and more accurate) conclusion that we're all trying to figure things out as best we can with what information we have at hand, and the fact that we come to radically different answers is testimony to the difficulty of understanding a complex and fascinatingly weird universe with our limited perceptions and fallible minds.

Or, as Schulz concludes, "I want to convince you that it is possible to step outside [the feeling of being right about everything], and that if you do so it is the greatest moral, intellectual, and creative leap you can take...  If you really want to rediscover wonder, you need to step out of that tiny, terrified space of rightness, and look around at each other, and look out at the vastness and complexity and mystery of the universe, and be able to say, 'Wow.  I don't know.  Maybe I'm wrong.'"

I smell a rat

I think I've made my position on GMOs plain enough, but let me just be up front about it right out of the starting gate.

There is nothing intrinsically dangerous about genetic modification.  Since each GMO involves messing with a different genetic substructure, the results will be different each time -- and therefore will require separate testing for safety.  The vast majority of GMOs have been extensively tested for deleterious human health effects, and almost all of those have proven safe (the ones that weren't never reached market).

So GMOs are, overall, as safe as any other agricultural practice -- i.e. not 100% foolproof, but with appropriate study, not something that deserves the automatic stigma the term has accrued.

There are a great many people who don't see it that way.  One of the most vocal is Gilles-Éric Séralini, who made headlines back in 2007 with a study that alleged that rats fed genetically modified corn showed blood and liver abnormalities.  When the study was published and other scientists attempted to replicate it (and failed), the results of Séralini's study were attributed to "normal biological variation (for the species in question)."

[image courtesy of the Wikimedia Commons]

Undeterred, Séralini went on in 2012 to publish a paper in Food and Chemical Toxicology about long-term toxicity of glyphosate (RoundUp) that is still the go-to research for the anti-Monsanto crowd.  He claimed that rats dosed with glyphosate developed large tumors and other abnormalities.  But that study, too, failed in attempts to replicate it, and it was withdrawn from FCT, with the editor-in-chief stating that the results were "inconclusive."

So if you smell a rat with respect to Séralini and his alleged studies, you're not alone.

But there's no damage to your reputation that can't be made worse, and Séralini took that dubious path last week -- with a "study" that claims that a homeopathic remedy can protect you from the negative effects of RoundUp.

So, to put it bluntly: a sugar pill can help you fight off the health problems caused by something that probably doesn't cause health problems, at least in the dosages that most of us would ordinarily be exposed to.

Being that such research -- if I can dignify it by that name -- would never pass peer review, Séralini went right to a pay-to-play open-source alt-med journal called BMC Complementary and Alternative Medicine.  Steven Savage, a plant pathologist, had the following to say about the study:

The dose is absurd.  They gave the animals the equivalent of what could be in the spray tank including the surfactants and the a.i. (active ingredients).  If glyphosate or its AMPA metabolite ever end up in a food it is at extremely low concentrations and never with the surfactant.  Unless you were a farmer or gardener who routinely drinks from the spray tank over eight days, this study is meaningless.

Furthermore, Andrew Porterfield, who wrote the scathing critique of Séralini I linked above, pointed out an additional problem:

Scientists have been sharply critical of the study’s methodology and conclusions... the paper has no discussion on the natural variability in locomotion or physiological parameters, making it impossible to tell if anything was truly wrong with any of the animals.

And if that weren't bad enough, Séralini proposes to counteract these most-likely-nonexistent health effects with pills that have been diluted past Avogadro's Limit -- i.e., the point where there is even a single molecule of the original substance left.  There have been dozens of controlled studies of the efficacy of homeopathy, and none of them -- not one -- have shown that it has any effect at all except as a placebo.

So we have doubtful health problems in animals that were not evaluated beforehand for health problems being treated by worthless "remedies" that have been shown to have zero effect in controlled studies.

Of course, considering how powerful confirmation bias is, I'm not expecting this to convince anyone who wasn't already convinced.  I will say, however, that we'd be in a lot better shape as a species if we relied more on reason, logic, and evidence -- and less on our preconceived notions of how we'd like the world to be.

Robustness, data, and the EmDrive

I get asked rather frequently why I put such trust in the scientific process.  Often, that question is framed even more pointedly; doesn't my belief in the reliability of science amount to religious faith -- something that I believe simply because I believe it?

It's an interesting subject, and it will probably come as no real surprise to my readers to hear that I think the answer to the latter question is "no."  When science is working as it should (and it doesn't always; as a human pursuit, it's subject to human fallibility, like anything else), we amass evidence, craft theories to explain said evidence, rule out ideas that aren't supported -- and in so doing, we get closer and closer to a theory that is self-consistent and explains all of the data we currently have.  Scientists call such a theory "robust."

And once we're there, the idea that something could come along and shatter the whole thing falls into the category of "possible, but extremely unlikely."  People like to cite Einstein and relativity as having destroyed the Newtonian view of the world, but this isn't really so; Newtonian physics works perfectly well at the speeds and masses we usually experience.  It is only at velocities near light speed, or masses with a big enough gravitational pull to warp spacetime significantly more than the Earth (or even the Sun) does, that we see a big enough relativistic effect to measure.  So Einstein was more of a refinement of Newton than an invalidation.

The reason all of this comes up is because of the recent news about the EmDrive, a propellant-less propulsion system that its creators claim uses quantum and relativistic effects to generate thrust from a closed system.  The drive, which uses microwave or radio wave generation inside a shielded shell, allegedly has been measured to produce a "few ounces" of thrust, and a paper describing the physics behind it has allegedly passed peer review and will be published this fall.

The problem is, the vast majority of the physicists -- i.e., the people who actually understand the intricacies of how the thing works, not to mention theories of propulsion, thermodynamics, quantum mechanics, and the rest -- are, in general, doubtful.  Physicist Sean Carroll, who writes passionately about the cutting-edge nature of modern physics research, and is up front about how much we don't know, was unequivocal:

There is no such thing as a ‘quantum vacuum virtual plasma,’ so that should be a tip-off right there.  There is a quantum vacuum, but it is nothing like a plasma.  In particular, it does not have a rest frame, so there is nothing to push against, so you can’t use it for propulsion.  The whole thing is just nonsense.  They claim to measure an incredibly tiny effect that could very easily be just noise.

Others have called the explanations given by EmDrive creator, engineer Roger Shawyer, as "Star Trek technobabble," or more simply, "bullshit."  (The latter from physicist John Carlos Baez of the University of California - Riverside.)  In order for a true exhaustless drive to work, it would have to overturn nearly everything we know about mechanics (quantum, Newtonian, relativistic, and otherwise).  In other words, it would point to a great big hole in our understanding of physics.

Once again: possible, but extremely unlikely.

Don't get me wrong -- no one would be more thrilled to have a potential interstellar drive system more than me.  If there was a spaceship that could get us to another star system in weeks or months (instead of tens of thousands of years) I would be elbowing people out of the way to get to the head of the line.  But the likelihood that the EmDrive is it seems slim to me, and the chance that what effect they've seen is simple experimental error or (as Carroll says) just noise nearly 100%.

Like any good skeptic, though, I'm perfectly willing to see the thing tested -- and I hope it does work. For one thing, if it did it would mean that there is a lot of physics out there that we don't understand, which is tremendously exciting.  For another, it would mean that we have a completely new way to power spacecraft, which is somewhere beyond tremendously exciting.  If the alleged peer-reviewed paper turns out to hold water, and tests of the thing -- data from which we should get in six months or so --  show that there is something to it, I will happily eat my words, publicly.  Look for a retraction right here in Skeptophilia -- should such an eventuality come to pass.

Go to the source

One of the many things I harp on with my students is "check your sources."

And "check your sources" doesn't mean the same thing as "cite your sources."  A bullshit citation written up in perfect MLA format, every comma, parenthesis, and colon in place, is still a bullshit citation.  There is no substitute for doing the legwork of making certain that the information you're using comes from a reputable source.

Which brings us, predictably enough, to Natural News.

Natural News has hidden for years under the façade of being a healthy-lifestyle site.  The 10% of their articles that are about better diet and regular exercise as a way of increasing vitality and longevity, however, are drowned beneath piles of nonsense of the worst sort, including anti-vaxx rhetoric, homeopathy, "toxin cleanses," and conspiracy theories about how the scientific world is an evil empire bent on ruining human health permanently.  Here's a sampler of their most-viewed articles as of today:

• 10 shocking reasons why Zika virus fear is another fraudulent medical hoax and vaccine industry funding scam
• How antidepressants ruin your natural serotonin so you can never be happy again ... without your pills
• Why double-blind drug trials are a science FRAUD: The more toxic the side effects, the more patients believe the drugs are 'working'
• Mother beats cancer with JUICING after told she only had two weeks to live

So it's not surprising that you won't find anything from Natural News in a peer-reviewed science journal, and it's not because there's a big conspiracy to keep their discoveries from becoming known.

[image courtesy of the Wikimedia Commons]

But if there's one thing that Mike "The Health Ranger" Adams and his cadre of loons over at Natural News excel at, it's marketing.  They know how to push their ideas into social media, giving it all a nice glossy veneer of respectability, duping the desperate, gullible, and scientifically illiterate into buying what they're selling.  However, they haven't been able to make any inroads into the world of actual scientific research, which is why...

... they are starting their own science journal.

Called the Natural Science Journal, the idea is to make an end run around all of the legitimate checks-and-balances that keep outright bullshit from making its way into print.  Here's a bit from their press release:

In a world where nearly all so-called "science" is actually little more than corporate fraud and government malfeasance, nearly all mainstream science journals have been taken over by pharmaceutical and biotech interests.  As a result, they destroy and suppress human knowledge rather than expanding it. 

All the big science journals -- Nature, The British Medical Journal, The Lancet and so on -- function almost entirely as science prostitutes for corporate interests, spewing out a vomitous cascade of fraudulent, industry ghostwritten "doctored" studies that the industry pretends represent real science.  This sad, filthy corruption of science harms the reputation of science itself and detracts from the valuable expansion of knowledge that can be achieved when science is practiced in the interests of humanity rather than corporate profits.

Which is mighty convenient, given that most of what peer-reviewed science has shown directly contradicts everything that Mike Adams and his crew believe.

And I feel obliged to mention that he really needs to lay off the "bold" typeface.

Adams is adamant that he is fostering actual research:

Please note that this journal is a hard sciences journal, meaning we seek scientific papers based on hard analytics in chemistry, physics, botany and so on. This is not a journal for philosophy or thought experiments that cannot be proven through hard experimental data.

So he's asking for submissions about the following topics:

• Geoengineering and weather modification
• Climate change / carbon dioxide
• Vaccine composition, toxicity and adverse events
• Genetically modified organisms
• Agrochemicals (pesticides / herbicides)
• Epigenetics and chemically-induced genetic expression
• Biosludge and biosolids
• Botany, permaculture and chemical-free agriculture

But don't worry, they're not biased at all.

Look, I know science isn't perfect.  Scientists (like all of us) have their biases, peer review sometimes misses mistakes (and occasionally outright fraud), the money motive drives research to an unfortunate degree, and so on.  But it is still by far the best tool we have for understanding the universe, including ourselves and how our own bodies work.  The idea that Adams has created his own "journal" simply because he doesn't like the fact that scientific research doesn't support what he believes isn't an indication of a failure of science.

It simply means that Adams is wrong.

My fear, though, is that given Natural Science Journal's neutral-sounding name, and its cursory nod in the direction of peer review, it will be taken seriously.  And then it'll be even harder to deal with the unscientific hogwash Adams and his ilk put out.

Ketchum study redux, and why peer review isn't a conspiracy

Well, the peer review is complete on Melba Ketchum's paper claiming she had isolated Bigfoot DNA, and the verdict is:

Fail.  [Source]

No details were released on why the paper failed to pass peer review, but almost certainly it is for the same reason that all failed papers are rejected; flaws in the methodology, data, or inferences, or all three.  The peer review process is there to keep scientists honest; all papers have to be evaluated line-by-line by other scientists in the same field, to make certain that everything is what it seems to be.  Now, to be sure there have been times that flawed papers have slipped through.  Scientists, after all, are only human, and can miss things, make assumptions, make outright mistakes.  But as a process, peer review works pretty damn well at winnowing the grain from the chaff.

Of course, that's not how a lot of Bigfoot enthusiasts see it.  The first to respond was Ketchum's ally, Russian cryptozoologist Igor Burtsev (this is long, but worth reading):

We waited a couple of years the scientific publication by Dr. Melba Ketchum. But scientific magazines refuse to publish her manuscript which deserves to be published. And I want to remind some facts of the destiny of scholars in our field.

Before the First World War our zoologist Vitaly Khahlov described the creature, named it Primihomo asiaticus. He send his scientific report very circumstantial, thorough to the Russian Academy of Sciences. And what? The report was put into the box, and had stayed there till 1959, about half of century. Until Dr. Porshnev found it and published…

I don’t want the new discovery (not the first one, but the next one) to wait for another half a century to be recognised by haughty official scientific establishment!

That is why I broke the tradition, did not let this achievement to wait for near half a century to be recognised. No matter of the publication in the scientific magazine, people should know NOW, what bigfoot/sasquatch is...

Yes, the paper of Dr Ketchum is under reviewing. And it is worth to be published. Just the situation now remindes [sic] me the war between North and South in the beginning of USA history… There are a lot of her supporters as well as a lot of her opponents and even some enemies…

The problem is that some people absolutize the science. Unfortunately science now is too conservative. One third of the population of the USA believes in BFs existing, but academic science even does not want to recognize the problem of their existing or not, just rejecting to discuss this question. In such a condition this subject is under discussion of the broad public. We can’t wait decades when scientists start to study this problem, forest people need to be protect now, not after half a century, when science wakes up.

Re the paper: the reviewed journals in the US refused to publish the paper. That is why Dr Ketchum has sent it to me to arrange publishing in any Russian reviewed journal. And I showed to our geneticists and understood that it was a serious work. I gave it up to the journal, now it’s under reviewing.

Anyway, I informed public about the results of the study. The public waited for this info for more than a year, a lot of rumors were spreading around. And the public has the right to know it nevertheless “science” says about it.

And this was mild when compared with the reaction from the cryptozoological wing of the blogosphere.  Science, it's claimed, is one vast conspiracy, where the scientists who are in the Inner Circles suppress good science that is outside of the current paradigm.  "Smash the heretics!" is, apparently, the battle cry of scientists in general, and peer review boards in particular.  Nothing must be allowed to run against the current model -- and the existence of Sasquatch would overturn everything.  So, at all costs, scientists must squelch any paper that tries to claim that Bigfoot exists.

Reading all of this, my reaction is: do you people actually know any working scientists?  Because it sure as hell sounds like you've never met one.

First of all, the claim that the scientists themselves would squash a legitimate claim solely because it runs counter to the current paradigm is absurd.  In fact, the opposite is true -- the scientists I know are actively looking for new, undiscovered features of our universe to explain.  That's how careers are made!  No working scientist I've ever run into does his/her work with the goal of simply reinforcing the preexisting edifice.  As Neil DeGrasse Tyson put it, "If you're not at the drawing board, you're not doing science.  You're doing something else."  Can you imagine how many papers, grants, and projects would come out of studying a newly-discovered proto-hominid, especially one that in all likelihood would be the nearest living relative of Homo sapiens?  Do you seriously think that the world's evolutionary biologists and primatologists would try to suppress such a discovery just because they're so happy with what they're already doing?

Second, remember where the supposed "scientific edifice" came from.  Virtually all of the pieces of the main scientific model in any field you choose came from someone overturning the previous model.  Consider why names like Darwin, Mendel, Einstein, and Newton are household words.  In each case, it's because they did the scientific version of tearing the house down -- and then rebuilding it from the ground up.  Significantly, though, none of the giants of scientific discovery did so by playing outside the rules.  They used data and the process of scientific induction to show us that the way we were looking at things was wrong (or at least incomplete).  Scientists are not ignorant about their own history -- and the vast majority of them would be delighted to be the next Einstein of their field.

Third, and more specific to the case of Bigfoot; even if there was some sort of grand conspiracy amongst scientists to Protect the Dominant Paradigm, why would Bigfoot represent such a threat?  As I have said more than once in this blog: new species are discovered daily.  There is nothing particularly earthshattering about the idea that one of those as-yet undiscovered species is a near relative of ours.  If such a creature were proven to exist, it would be cool; as I mentioned earlier, my guess is that the zoologists would be elbowing each other out of the way to get dibs on studying it first, not running the other way shouting "la-la-la-la-la, not listening."  But as woo-woo claims go, Bigfoot is the one that would cause the least revision to our current scientific view of the world.  It would add a new branch to the primate tree; it would require some revisiting of humanity's evolutionary descent.  And that's all.  Proof of just about any other claim of this sort -- UFOs, ghosts, telepathy, even the Loch Ness Monster -- would force a far greater revision of our current understanding of natural processes.

Anyhow, I'd like to think that this is the last we'll hear from Ketchum et al.  Whether Bigfoot is out there remains to be seen, but apparently the Ketchum study isn't going to be the one to prove it, so it's to be hoped that they'll just bow out gracefully.  I know, unfortunately, that the conspiracy theorists won't do likewise.  They never run out of energy, more's the pity.