Two New Studies Demonstrate How Not to Do Science

Two New Studies Demonstrate How Not to Do Science
AP Photo/Nick Ut, File
Two New Studies Demonstrate How Not to Do Science
AP Photo/Nick Ut, File
Story Stream
recent articles

Two studies published earlier this week produced results that almost certainly can be filed under "spurious correlations."

Spurious correlations describe variables which seem linked when statistically examined but in reality do not affect each other at all – any association is coincidence. You know, like how per capita cheese consumption correlates with the number of people who died by becoming tangled in their bedsheets or how the age of Miss America is linked to murders by steam, hot vapors, and hot objects.

The studies in question were published in legitimate, well-known scientific journals: Nature's Scientific Reports and PLoS ONE.

The first, published to Scientific Reports, found that "relative poverty (income inequality) predicts frequency of kissing across romantic relationships." The researchers reported this result after surveying 3,109 participants online about their kissing behaviors. Despite the fact that the data is all self-report and includes subjects from just 13 countries, the scientists seem fairly certain of their study's veracity.

"Individuals kiss their partner more in countries where resource competition is likely to be more intense," they explain, "which may play an important role in maintaining long-term stable pair bonds in certain types of harsh environment."

A simpler and more likely explanation is that their finding is a statistical fluke. The researchers probed a number of hypotheses, including that national health would correlate with kissing frequency, but neglected to report on those in their results section, presumably because they didn't turn up positive results. Fishing for findings with a plethora of hypotheses in data sets is frowned upon, as it increases the chance of stumbling upon a statistically significant, but spurious result purely by chance.

The second study, published in PLoS ONE, found air pollution to be associated with lying frequency. Here, some participants were presented with pictures of Beijing on a "polluted" day while others were shown Beijing on a "clean" day. The two groups were then given surveys to determine their propensity to lie. The researchers found that subjects presented with the clean air prompt were actually more likely to lie, but inexplicably ignored their own findings, deferring to prior research which showed air pollution to increase lying.

"Although these results seemed to be inconsistent with previous results showing that air pollution increased lying behaviors, they actually support the same explanation of the underlying mechanisms. As mentioned before, air pollution can influence lying behaviors through two mechanisms—decreasing the effect of moral rules and increasing the desire for resources," they wrote.

While the researchers' experimental design was laughable, it seems pointless to conduct research if you're just going to rationalize away findings that you don't like.

In actuality, when poorly-conducted studies show contradicting results, it's more likely that there is no effect at all. Air pollution is probably not associated with lying, nor does income inequality predict how much couples will kiss.

Show comments Hide Comments

Related Articles