Editor’s Note: This article is a follow-up to one previously written by Tom Hartsfield titled, "NSF Should Stop Funding Social 'Science'".
The term “science” carries a centuries-long aura of legitimacy and respectability. But not every field of research can rightly call itself scientific.
Traditionally, fields such as biology, chemistry, physics and their spinoffs constitute the “hard sciences” while social sciences are called the “soft sciences.” A very good reason exists for this distinction, and it has nothing to do with how difficult, useful or interesting the field is. Instead, it has to do with how scientifically rigorous its research methods are.
What do we mean by scientifically rigorous? Let’s start by discussing what we don’t mean.
Using statistics doesn’t make a field scientifically rigorous. Baseball players and gamblers use statistics everyday. They are not scientists. Even using extremely complicated math and statistics doesn't make a field scientific.
The mathematically intensive field of economics is largely preoccupied with determining correlation and causation. In order to do so, economists employ a statistical technique, multiple regression analysis, which is every bit as complicated as it sounds. But, as the authors of Freakonomics write, “[R]egression analysis is more art than science.”
So, if mind-bending statistical analysis doesn’t make a field scientifically rigorous, what does? Five concepts characterize scientifically rigorous studies:
Clearly defined terminology. Science should not use ambiguous terminology or words with arbitrary definitions. Microbiologists all agree on what constitutes a cell, and chemists all agree on what constitutes a molecule. But this is not always the case in other fields. How does one precisely define a particular political ideology? Or life satisfaction? Or sexism? These ideas, though commonly studied in other fields, have vague definitions that can change over time, across geography, or even between different cultures.
Quantifiability. Rigorous science is quantifiable. Planets are measured in density and orbital velocity. Toxicity is measured in lethal dosages. But how do you measure happiness? Can a person put a reliable number on how happy he is feeling today? Lord Kelvin expressed the importance of measurability when he said:
I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.
Highly controlled conditions. This is probably the most important characteristic, and it is precisely here where many fields fall short. A scientifically rigorous study maintains direct control over as many of the factors that influence the outcome as possible. The experiment is then performed with such precision that any other person in the world, using identical materials and methods, should achieve the exact same result. A scientist testing bacterial growth in France should get the same result as a microbiologist in Australia.
The ability to create highly controlled conditions is simply nonexistent for many soft sciences. Instead, they rely on observational studies in uncontrolled, often chaotic environments. To tease apart correlation from causation, they apply fancy math – like the regression analysis mentioned above – but this isn’t a sufficient substitute for a highly controlled environment.
Let us turn once again to the authors of Freakonomics, who succinctly summarize why economics is not a scientific field:
In a perfect world, an economist could run a controlled experiment just as a physicist or a biologist does: setting up two samples, randomly manipulating one of them, and measuring the effect. But an economist rarely has the luxury of such pure experimentation.
Reproducibility. Having control over conditions allows experiments to be carefully repeated. A rigorous science is able to reproduce the same result over and over again. Multiple researchers on different continents, cities, or even planets should find the exact same results if they precisely duplicated the experimental conditions. Remember the controversy over faster-than-light neutrinos? Reproducible conditions allowed subsequent experiments to disprove this finding. Inaccurate results can be decisively and quickly removed from the canon of scientific truth.
Predictability and testability. A rigorous science is able to make testable predictions. One of the most beautiful examples of this is the periodic table of elements. Dmitri Mendeleev, a Russian chemist, successfully predicted the properties of missing elements on the table – that is, elements that had not been discovered yet. While fields like economics and psychology might be able to explain existing behavior, they do not often do well in predicting future outcomes – if they dare to make predictions at all.
Admittedly, this is a tough list. But, it’s supposed to be. The standard for rigorous science should be very high. Even some fields widely accepted as scientifically rigorous don’t always measure up. Particle physics – most notably, string theory – sometimes makes predictions that are not testable with modern technology. Epidemiology often cannot perform controlled experiments, both for reasons of ethics and practicality. Epidemiologists can’t lock 20,000 people in a room for 20 years to determine if force-feeding them hot dogs will cause cancer. Instead, they rely on observational studies.
But, clearly, some social science fields hardly meet any of the above criteria.
So, returning to the question posed in the title of this article, “What separates science from non-science?” It’s hard to say. There isn’t a crystal clear dividing line between the two. But, what can be definitively said is this: A scientifically rigorous study will meet all or most of the above requirements, and a less rigorous study will meet few if any of those requirements.
As useful and interesting as the social sciences are, they usually fall into the latter category.