There are hundreds of ways to design scientific studies and analyze results. Often statistical measures help scientists discover relationships between various aspects of the data. (Some statistical perspectives are presented in Table 1.7.)

Every research design, method, and statistic has strengths as well as weaknesses. Now we describe three basic research designs—observation, the experiment, and the survey—and then three ways developmentalists study change over time.

35

Scientific observation requires researchers to record behavior systematically and objectively. Observations often occur in a naturalistic setting (such as a home, school, or public park), where people behave as they usually do and where the observer is ignored or even unnoticed. Observation can also occur in a laboratory, where scientists record human reactions in various situations, often with wall-mounted video cameras and the scientist in another room.

A crucial endeavor in development is to determine the cause and sequence of behavior: Observation does not pin that down. An experiment is needed. In the social sciences, experimenters typically impose a particular treatment on a group of participants (formerly called subjects) or expose them to a specific condition and then note whether their behavior changes.

36

In technical terms, the experimenters manipulate an independent variable, the imposed treatment or special condition (also called the experimental variable). (A variable is anything that can vary.) They note whether this independent variable affects whatever they are studying, called the dependent variable, which depends on the independent variable.

Thus, the independent variable is the new, special treatment; any change in the dependent variable is the result. The purpose of an experiment is to find out whether an independent variable affects the dependent variable. In a typical experiment (as diagrammed in Figure 1.12), two groups of participants are studied. One group, the experimental group, gets a particular treatment (the independent variable). The other group, the comparison group (also called the control group), does not.

For example, many mayors believe that summer jobs prevent juvenile delinquency, a hypothesis that springs from ideology, not science. One scientist (Heller, 2014) reviewed several studies that found no effects of summer work for youth who had already quit school and been arrested for serious crimes. This raises two possibilities: (1) serious delinquents need more than summer work, or (2) summer work itself makes no difference.

To find out, an experiment was needed. Consequently, 1,634 high school students (average age, 17) from high-crime Chicago neighborhoods were divided into a control group (904 students, no job offered) and an experimental group (730 students who were paid minimum wage for 25 hours a week for 8 weeks). As Vygotsky’s idea of guided participation would recommend, the students in the experimental group had mentors, each assigned to about ten students to discuss job issues, such as getting along with supervisors and coworkers.

For 16 months after their summer jobs ended, students’ police and school records were examined. The students in the experimental group had only half as many arrests for violent crime (assaults, rapes, and so on—usually the result of impulsive, uncontrolled anger).

However, the intervention did not reduce arrests for nonviolent crimes (over 16 months, about 1 in 5 were arrested for drugs, theft, and so on) or improve school attendance (students in both groups were absent from school almost 30 days a year).

37

That experiment led to an important conclusion: Summer work helps adolescents to be less impulsive, but it does not transform them into conscientious students or law-abiding citizens. This lack of comprehensive transformation is not surprising, given the importance of context and culture. However, “this study provides causal evidence . . . The results echo a common conclusion in education and health research: that public programs might do more with less by shifting from remediation to prevention” (Heller, 2014, p. 1222).

Note how crucial the scientific method and the experiment can be. Liberals and conservatives have argued for decades about the value of public programs to help the poor. This study provided some evidence about what might do some good for whom—and what was unlikely to change after a summer’s employment. The research came to a sound conclusion, on which people of all political perspectives can agree.

In addition to observation and experimentation, a third research method is the survey. Information is collected from a large number of people, usually by simply asking them questions. This is a quick, direct way to obtain data. It is better than assuming that the experiences and attitudes of people we happen to talk with are valid for everyone we do not know.

For example, if you know a 16-year-old who is pregnant, or an adult who hates his job, or an old person who watches television all day, surveys will keep you from jumping to false conclusions. As you will read later, teenage pregnancy is no longer common, most people appreciate their jobs, and older people watch less television than children do.

Unfortunately, although surveys are quick and direct, they are not always accurate. People sometimes lie, and answers are influenced by the wording and the sequence of the questions. For instance, “climate change” and “global warming” are two ways to describe the same phenomenon, according to many scientists, yet many people believe in climate change but not in global warming (McCright & Dunlap, 2011). For that reason, surveys that seem to be about the same issue may reach opposite conclusions.

Survey respondents may even lie to themselves. For instance, every two years since 1991, high school students in the United States have been surveyed confidentially. The most recent survey included 13,633 students from all 50 states and from schools large and small, public and private (MMWR, June 13, 2014).

Students are asked whether they had sexual intercourse before age 13. Every year, more ninth-grade than twelfth-grade boys say they had sex before age 13, yet those twelfth-graders were ninth-graders a few years before (see Figure 1.13). Do twelfth-graders forget or do ninth-graders lie? Or do some 14-year-olds brag about the same thing that later makes them ashamed? The survey cannot tell us.

38

In addition to conducting observations, experiments, and surveys, developmentalists must measure how people change or remain the same over time, as our definition stresses. Remember that systems are dynamic, ever-changing. To capture that dynamism, developmental researchers use one of three basic research designs: cross-sectional, longitudinal, or cross-sequential.

CROSS-SECTIONAL VERSUS LONGITUDINAL RESEARCH The quickest and least expensive way to study development over time is with cross-sectional research, in which groups of people of one age are compared with people of another age. You saw that at the beginning of the chapter: With every decade of age, the proportion of obese people increases.

Cross-sectional design seems simple. However, it is difficult to ensure that the various groups being compared are similar in every way except age. Because most women now in their 50s gained an average of a pound every year throughout their adulthood, does this mean that women now aged 20 who weigh 140 pounds will, on average, weigh 170 pounds at age 50? Not necessarily.

To help discover whether age itself rather than cohort causes a developmental change, scientists undertake longitudinal research. This requires collecting data repeatedly on the same individuals as they age. It is only through longitudinal research that we learned that a third of overweight children become normal weight adults.

However, longitudinal research has several drawbacks. Over time, participants may withdraw, move to an unknown address, or die. These losses can skew the final results if those who disappear are unlike those who stay, as is often the case. Another problem is that participants become increasingly aware of the questions or the goals of the study—knowledge that could affect their behavior over time.

For example, you saw in Figure 1.2 that most overweight children who became normal-weight adults were actually healthier than adults who had never been overweight. How could that be? Perhaps the fact that they knew they had been heavy and that they were now repeatedly measured caused them to eat more fruits and vegetables than they otherwise would have. That is a wonderful result, but it is also a flaw of longitudinal research.

Probably the biggest problem comes from the historical context. Science, popular culture, and politics alter life experiences, and those changes limit the current relevance of data collected on people born decades ago. Results from longitudinal studies of people born in the early twentieth century, as they made their way through childhood, adulthood, and old age, may not be relevant to people born in the twenty-first century.

39

Many recent substances are thought to be harmful by some people but advocated as beneficial by others, among them phthalates and bisphenol A (BPA) (chemicals used in manufacturing) in plastic baby bottles, hydrofracking (a process used to get gas for fuel from rocks), e-waste (from old computers and cell phones), and more. Some nations and states ban or regulate each of these; others do not, because verified, longitudinal data are not yet possible.

One example that is directly developmental is e-cigarettes, which are less toxic (how much less?) to the heart and lungs than combustible cigarettes. Some (how many?) adult smokers reduce their risk of cancer and heart disease by switching to e-cigs (Bhatnagar et al., 2014). But for some teenagers (how many?) vaping introduces them to using more damaging substances that they otherwise would never use.

Until we know rates of addiction and death for all those e-cig smokers, 10 or 20 years from now, we cannot be sure whether the harm outweighs the benefits (Ramo et al., 2015; Hajek et al., 2014; Dutra & Glantz, 2014). Forty U.S. states have restricted e-cig sales. A spokesman for the Utah Department of Health said, “while we wait for the science on long-term effects . . . thousands of teens in Utah are starting a nicotine addiction via e-cigarettes . . . it’s imperative that we get one finger in the dam until we know more” (Bramwell, quoted in McGill, 2015, p. 12).

CROSS-SEQUENTIAL RESEARCH Scientists have discovered a third strategy, combining cross-sectional and longitudinal research. This combination is called cross-sequential research (also referred to as cohort-sequential or time-sequential research). With this design, researchers study several groups of people of different ages (a cross-sectional approach), follow them over the years (a longitudinal approach), and then combine the results.

40

A cross-sequential design lets researchers compare findings for a group of, say, 16-year-olds with findings for the same individuals at age 1, as well as with findings for groups who were 16 long ago, and who are now ages 31, 46, and 61(see Figure 1.14). Cross-sequential research is complicated, in recruitment and analysis, but it lets scientists disentangle age from history.

One well-known cross-sequential study (the Seattle Longitudinal Study) found that some intellectual abilities (vocabulary) increase even after age 60, whereas others (speed) start to decline at age 30 (Schaie, 2005/2013), confirming that development is multi-directional. This study also discovered that declines in adult math ability are more closely related to education than to age, something neither cross-sectional nor longitudinal research could reveal.

A more recent cross-sequential study looked at self-esteem in late adulthood. The results were surprising: Self-esteem varied markedly from one person to another, but was quite stable over the decades. Elders with high self-esteem were social and self-sufficient, characteristics that often continued from age 70 to 105 (Wagner et al., 2015).

41

Cross-sequential research is useful for young adults as well. For example, drug addiction (called substance use disorder, SUD) is most common in the early 20s and decreases by the late 20s. But one cross-sequential study found that the origins of SUD are much earlier, in adolescent behaviors and in genetic predispositions (McGue et al., 2014).

The scientific method illuminates and illustrates human development as nothing else does. Facts, consequences, and possibilities have all emerged that would not be known without science—and people of all ages are healthier, happier, and more capable than people of previous generations because of it.

For example, infectious diseases in children, illiteracy in adults, depression in late adulthood, and racism and sexism at every age are much less prevalent today than a century ago. Science deserves credit for all these advances. Even violent death is less likely, with scientific discoveries and education likely reasons (Pinker, 2011).

Developmental scientists have also discovered unexpected sources of harm. Video games, cigarettes, television, shift work, asbestos, and even artificial respiration are all less benign than people first thought.

As these examples attest, the benefits of science are many. However, there are also serious pitfalls. We now discuss three potential hazards: misinterpreting correlation, depending too heavily on numbers, and ignoring ethics.

CORRELATION AND CAUSATION Probably the most common mistake in interpreting research is confusing correlation with causation. A correlation exists between two variables if one variable is more (or less) likely to occur when the other does. A correlation is positive if both variables tend to increase together or decrease together, negative if one variable tends to increase while the other decreases, and zero if no connection is evident.

To illustrate: From birth to age 9, there is a positive correlation between age and height (children grow taller as they grow older), a negative correlation between age and amount of sleep (children sleep less as they grow older), and zero correlation between age and number of toes (children do not have more or fewer toes as they grow older).

Expressed in numerical terms, correlations vary from +1.0 (the most positive) to –1.0 (the most negative). Correlations are almost never that extreme; a correlation of +0.3 or –0.3 is noteworthy; a correlation of +0.8 or –0.8 is astonishing.

Many correlations are unexpected. For instance, first-born children are more likely to develop asthma than are later-born children, teenage girls have higher rates of mental health problems than do teenage boys, and counties in the United States with more dentists have fewer obese residents. That later study controlled for the number of medical doctors and the poverty of the community. The authors suggest that dentists provide information about nutrition that improves health (Holzer et al., 2014).

At this point, remember that correlation is not causation. Just because two variables are correlated does not mean that one causes the other—even if it seems logical that it does. It proves only that the variables are connected somehow. Many mistaken and even dangerous conclusions are drawn because people misunderstand correlation.

42

QUANTITY AND QUALITY A second caution concerns how heavily scientists should rely on data produced by quantitative research (from the word quantity). Quantitative research data can be categorized, ranked, or numbered and thus can be easily translated across cultures and for diverse populations. One example of quantitative research is the use of children’s school achievement scores to compare the effectiveness of education within a school or nation.

Since quantities can be easily summarized, compared, charted, and replicated, many scientists prefer quantitative research. Statistics require numbers. Quantitative data are easier to replicate and less open to bias, although researchers who choose this method have some implicit beliefs about evidence and verification (Creswell, 2009).

However, when data are presented in categories and numbers, some nuances and individual distinctions are lost. Many developmental researchers thus turn to qualitative research (from quality)—asking open-ended questions, reporting answers in narrative (not numerical) form.

Qualitative researchers are “interested in understanding how people interpret their experiences, how they construct their worlds . . .” (Merriam, 2009, p. 5). Qualitative research reflects cultural and contextual diversity, but it is also more vulnerable to bias and harder to replicate. Both types of research are needed.

ETHICS The most important caution for all scientists, especially for those studying humans, is to uphold ethical standards. Each academic discipline and professional society involved in the study of human development has a code of ethics (a set of moral principles) and specific practices within a scientific culture to protect the integrity of research.

Ethical standards and codes are increasingly stringent. Most educational and medical institutions have an Institutional Review Board (IRB), a group that permits only research that follows certain guidelines.

Although IRBs often slow down scientific study, some research conducted before they were established was clearly unethical, especially when the participants were children, members of minority groups, prisoners, or animals. Some argue that serious ethical dilemmas remain (Leiter & Herman, 2015).

Researchers must ensure that participation is voluntary, confidential, and harmless. In Western nations, this entails the informed consent of the participants—that is, the participants must understand and agree to the research procedures and know what risks are involved. A dilemma occurs when severe consequences might follow either participation or non-participation.

43

Many ethical dilemmas arose in the Ebola epidemic (Rothstein, 2015; Gillon, 2015). Among them: Is it fair to use vaccines that have not been proven safe, when such proof would take months and the death rate from Ebola would increase? What kind of informed consent is needed to avoid both false hope and false fears? Is it justified to keep relatives away from sick people who might have Ebola, even though social isolation might increase the death rate? Should drugs that researchers are uncertain about be given to Ebola patients?

More broadly, is justice served by a health care system that is inadequate in some countries and high-tech in others? Medicine has tended to focus on individuals, ignoring the customs and systems that make some people more vulnerable. One observer noted:

When people from the United States and Europe working in West Africa have developed Ebola, time and again the first thing they wanted to take was not an experimental drug. It was an airplane that would cart them home.

[Cohen, 2014, p. 911]

Is that ethical?

IMPLICATIONS OF RESEARCH RESULTS Once a study has been completed, additional issues arise. Scientists are obligated to “promote accuracy, honesty, and truthfulness” (American Psychological Association, 2010).

Deliberate falsification leads to ostracism from the scientific community, dismissal from a teaching or research position, and, sometimes, criminal prosecution. Another obvious breach of ethics is to “cook” the data, or distort one’s findings, in order to make a particular conclusion seem to be the only reasonable one.

Some of the benefits (promotion, acclaim) of publishing remarkable, unreplicated findings encourage unethical research, such as slanting conclusions. Further, there is “ferocious . . . pressure from commercial funders to ignore good scientific practice” (Bateson, 2005, p. 645). Pressures from politicians and corporations are part of the problem, but nonprofit research groups and academic institutions also want particular results.

As stressed in the beginning of this chapter, researchers, like all other humans, have strong opinions, which they expect research to confirm. Therefore, they might try (sometimes without even realizing it) to achieve the results they want. As one team explains:

Our job as scientists is to discover truths about the world. We generate hypotheses, collect data, and examine whether or not the data are consistent with those hypotheses . . . . [but we] often lose sight of this goal, yielding to pressure to do whatever is justifiable to compile a set of studies we can publish. This is not driven by a willingness to deceive but by the self-serving interpretation of ambiguity . . .

[Simmons et al., 2011, pp. 1359, 1365]

Obviously, collaboration, replication, and transparency are essential ethical safeguards for all scientists.

UNKNOWNS AND UNKNOWN UNKNOWNS Hundreds of crucial questions regarding human development need answers, and researchers have yet to find them. For instance:

The answer to all these questions is a resounding NO. The reasons are many. Scientists and funders tend to avoid questions that might lead to answers they do not want. People have strong opinions about drugs, income, families, and death that may conflict with scientific findings and conclusions. Religion, politics, and ethics shape scientific research, sometimes stopping investigation before it begins.

44

An even greater question is about the “unknown unknowns,” the topics that we assume we understand but do not, hypotheses that have not yet occurred to anyone because our thinking is limited by our cultures and contexts.

The next cohort of developmental scientists will build on what is known, mindful of what needs to be explored, and will raise questions that no one has thought of before. Remember that the goal is to help all 7 billion people on Earth fulfill their potential. Much more needs to be learned. The next 14 chapters are only a beginning.

WHAT HAVE YOU LEARNED?

Question 1.34

Observation is crucial in developing hypotheses for the causes and sequences of behavior, but experiments are needed to determine cause-and-effect relationships.

Question 1.35

The purpose of an experiment is to find out whether an independent variable (the imposed treatment or special condition) affects the dependent variable (whatever they are studying); therefore, one needs to compare the impact of the independent variable on one group that receives the independent variable (the experimental group) and one group that does not (the control group).

Question 1.36

The biggest strengths of the survey method are that it is quick and direct. Its biggest weakness is that answers may not be accurate because people may lie, want to come across favorably, or be influenced by the wording of the questions.

Question 1.37

It is the quickest and least expensive way to study development over time.

Question 1.38

The biggest advantage of longitudinal research is that it is useful in tracing development over many years. Disadvantages include dropout of participants, participants becoming increasingly aware of the questions or the goals of the study, and the influence of the historical context.

Question 1.39

This research allows researchers to study several groups of people of different ages and then follow those groups over the years. This type of research is the most time-consuming and complex, but it yields the best information.

Question 1.40

Just because two variables are correlated does not mean that one causes the other—even if it seems logical that it does. It proves only that the variables are connected somehow.

Question 1.41

Quantitative research data can be categorized, ranked, or numbered and thus can be easily translated across cultures and for diverse populations. However, when data are presented in categories and numbers, some nuances and individual distinctions are lost.

Question 1.42

Qualitative research reflects cultural and contextual diversity, but it is also more vulnerable to bias and harder to replicate.

Question 1.43

An IRB, or Institutional Review Board, is a group that permits only research that follows certain guidelines. Most medical and educational institutions have an IRB to maintain strict ethical codes and standards.

Question 1.44

Participants must be kept confidential and give informed consent to prove they understand and agree to the research procedures and know what risks are involved. A dilemma occurs when severe consequences might follow either participation or non-participation.

Question 1.45

A political leader may not want to fund this type of research because of potential unethical research. Some of the benefits (promotion, acclaim) of publishing remarkable, unreplicated findings encourage unethical research, such as slanting conclusions.

Question 1.46

Possible examples may include but are not limited to the following: Do we know enough about prenatal drug use to protect every fetus? Do we know enough about dying to enable everyone to die with dignity? Do we know enough about poverty to enable everyone to be healthy?