American millennials are rejecting the suburbs and returning to the city. That was a prevailing idea in 2019, when I started as a social science reporter at science news. But when I started digging into a possible story about the phenomenon, I found myself in an incoherent mess. Some research showed that suburbs were growing, others that suburbs were shrinking, and others showed growth in both suburbs and cities.
Unable to make sense of this labyrinth of findings, I put aside the idea of history. Then, several months later, I came across a white paper from Harvard University that explained that disagreement in the field stems from competing definitions of what distinguishes a city from a suburb. Some researchers define suburbs as areas outside of census-designated cities. Others only look for markers of suburbanism, such as a high number of single-family homes and car commutes, the researchers wrote.
I have encountered this kind of confusion around the definitions of all sorts of terms and concepts in the years that I have covered the social sciences. Sometimes researchers simply assume that their definition of a key concept is the definition. Or they briefly assent to other definitions and then move on to the one they choose, without much explanation of why. Other times, researchers in one subfield choose one definition, and researchers in another subfield choose a different one, each without even knowing of the other’s existence. It’s enough to make any reporter tear their hair out.
“If you look … you’ll find this morass of definitions and measures” in the social sciences, says quantitative psychologist Jessica Flake of McGill University in Montreal. My experience was common, she assured me.
Definition swamps also exist in other scientific fields. Biologists often disagree about which is the best to define the word “species” (Serial number: 11/1/17). virologists fight on what counts as “alive” when it comes to viruses (Serial Number: 11/1/21). And not all astronomers are happy with the decision to define the word “planet” in a way that leaves Pluto in the cold as a mere dwarf planet (Serial number: 8/24/21).
But the social sciences have some special challenges, says Flake. The field is young compared to a discipline like astronomy, so it has had less time to work out its definitions. And social science concepts are often inherently subjective. Describing abstract ideas like motivation or feelings can be more complicated than describing, say, a meteorite.
It’s tempting to assume, as I did until I started researching this column, that a single, imperfect definition for individual concepts is preferable to this cacophony of definitions. And some researchers encourage this approach. “While no suburban definition will be perfect, Standardization it would increase understanding of how suburban studies relate to one another,” the Harvard researchers wrote in that suburban paper.
But a recent study looking at how we define the middle class showed me how alternative definitions can lead to a shift in perspective.
While most researchers use income as a proxy for class, these researchers used people’s purchasing patterns. That revealed that a fraction of people who appear to be middle class by income struggle to pay for basic necessities such as housing, childcare and groceries, the team reported in July in Social Indicators Research. that is, they live as if they were working class.
Also, that vulnerable group skews black and Hispanic, a disparity that arises, in part, because these families of color often lack the generational wealth of white families, says Melissa Haller, a geographer at Binghamton University in New York. So when calamity strikes, families without that financial cushion can have a hard time getting back on their feet. However, a government or non-profit organization seeking to target aid to families in greatest need, and relying solely on income-based metrics, would miss this vulnerable group.
“Depending on what definition you start with, you will see different facts,” says Anna Alexandrova, a philosopher of science at the University of Cambridge. A standardized definition of middle class, for example, might obscure some of those key facts.
In the social sciences, what is needed instead of conceptual unity, says Alexandrova, is conceptual clarity.
Although social scientists disagree on how to solve this clarity problem, Flake says that not addressing the problem endangers the field as much as it does. other crises shaking the discipline (Serial number: 8/27/18). That’s because how a topic is defined determines the scales, surveys, and other instruments used to study that concept. And that, in turn, shapes how researchers crunch numbers and reach conclusions.
Defining key terms and then selecting the right tool is easy when relying on large external data sets. For example, instead of using national income databases, as is common in studying the middle class, Haller and her team turned to the federal government’s Consumer Expenditure Surveys to understand everyday and emergency purchases. of people.
But often social scientists, particularly psychologists, develop their own scales and surveys to quantify subjective concepts, such as self-esteem, mood, or well-being. The definitions of those terms, and the tools used to study them, can take on a life of their own, says Flake.
She and her team recently showed how this process unfolds in the May-June issue American psychologist. They reviewed the 100 original studies and the 100 replications included in a massive reproducibility project in psychology. The researchers focused on 97 multi-item scales, which measure concepts such as gratitude, motivation and self-esteem, used in the original studies, and found that 54 of those scales had no citations to show where the scales originated. That suggests that the original authors defined their idea and the tool used to measure that idea, on the progressFlake says. The research teams then attempted to replicate 29 of those studies without delving into the sources of the scales, questioning the meaning of their results.
For Flake, the way to achieve conceptual clarity is simple, if unlikely. Researchers must hit the brakes to generate new ideas or replicate old ideas and instead interrogate the swamp of old ones.
He points to a promising, albeit laborious, effort: the Psychological Sciences Accelerator, a collaboration of more than 1,300 researchers in 84 countries. The project aims to identify big ideas in psychology, such as face perception and gender bias, and amass all the resulting instruments and data used to make sense of those ideas in order to discard, refine, or blend definitions. and existing tools.
“Instead of running replications, why don’t we use [this] massive team of researchers representing many perspectives around the world and reviewing concepts first,” says Flake. “We have to stop replicating garbage.”
I could not agree more.