Using the Scientific Method

There are hundreds of ways to design scientific studies and analyze results. Scientists are aware that no design is absolutely perfect, which is one reason researchers hope for replication of their studies and confirmation from other researchers who use a different design. In addition, statistics and calculations help scientists interpret their data. (Some statistical perspectives are presented in Table 1.2.)

23

Every research design, method, and statistical measure has strengths as well as weaknesses. It is impossible to describe them all here; there are so many that colleges offer year-long courses in research design and other courses in statistics. Here we describe three basic research strategies—observation, the experiment, and the survey—and then three ways developmentalists study change over time.

Observation


Video Activity: What’s Wrong with This Study explores some of the major pitfalls of the process of designing a research study.

Scientific observation requires researchers to record behavior systematically and objectively. Observations often occur in a naturalistic setting such as a home, where people behave normally. Scientific observation can also occur in a laboratory where scientists record human reactions in various situations, often with wall-mounted video cameras and the scientist in another room.

Observation is crucial to the development of hypotheses. However, observation provides issues to explore, not proof.

For example, one study of how long parents lingered when dropping off their children in preschool found that children were slower to engage with toys or friends when their parents stayed longer (Grady et al., 2012). This could mean that parental hesitancy to leave made children less social with friends. However, the opposite interpretation is also possible. Perhaps shy children cause parents to linger. Thus this study led to two alternative hypotheses: More research is needed to ascertain which one is more accurate.

The Experiment

The experiment establishes what causes what. In the social sciences, experimenters typically impose a particular treatment on a group of volunteer participants or expose them to a specific condition and then note whether their behavior changes.

24

In technical terms, the experimenters manipulate an independent variable, which is the imposed treatment or special condition (also called the experimental variable; a variable is anything that can vary). They note whether this independent variable affects whatever they are studying, called the dependent variable (which depends on the independent variable).

Thus, the independent variable is the new, special treatment; any change in the dependent variable is the result. The purpose of an experiment is to find out whether an independent variable affects the dependent variable. In a typical experiment (as diagrammed in Figure 1.7), two groups of participants are studied. One group, the experimental group, is subjected to the particular treatment or condition (the independent variable); the other group, the comparison group (also called the control group), is not.

FIGURE 1.7
How to Conduct an Experiment The basic sequence diagrammed here applies to all experiments. Many additional features, especially the statistical measures listed in Table 1.2 and various ways of reducing experimenter bias, affect whether publication occurs. (Scientific journals reject reports of experiments that were not rigorous in method and analysis.)

The Survey

A third research method is the survey, in which information is collected from a large number of people by interview, questionnaire, or some other means. This is a quick, direct way to obtain data.

Especially for Nurses In the field of medicine, why are experiments conducted to test new drugs and treatments?

Experiments are the only way to determine cause-and-effect relationships. If we want to be sure that a new drug or treatment is safe and effective, an experiment must be conducted to establish that the drug or treatment improves health.

Unfortunately, although surveys may be quick and direct, they are not necessarily accurate. When pollsters try to predict elections, they survey thousands of potential voters. They hope that the people they survey will vote as they say they will, that undecided people will follow the trends, and that people who refuse to give their opinion, or who are not included, will be similar to those surveyed. None of this is certain. Some people lie, some change their minds, some (especially those who don’t have phones or who never talk to strangers) are never counted.

Furthermore, survey answers are influenced by the wording and the sequence of the questions. For instance, according to many scientists, “climate change” and “global warming” are two ways to describe the same phenomenon, yet many people believe in climate change but not in global warming (McCright & Dunlap, 2011). For that reason, surveys that seem to be about the same issue may reach opposite conclusions.

FIGURE 1.8
I Forgot? If this were the only data available, you might conclude that ninth-graders have suddenly become more sexually active than twelfth-graders. But we have 20 years of data—ninth-graders always answer differently by twelfth grade.

Additionally, survey respondents present themselves as they would like to be perceived. For instance, every two years since 1991, high school students in the United States have been surveyed. The participants are carefully chosen to be representative of all students in the nation. The most recent survey included 13,633 students from all 50 states and from schools large and small, public and private (MMWR, June 13, 2014).

Students are asked whether they had sexual intercourse before age 13. Every year, compared to the twelfth-grade boys, far more ninth-grade boys say they had sex before age 13 (see Figure 1.8). Do seniors forget or do ninth-graders lie? Or are some 13-year-olds proud of early sexual experience but ashamed by age 17? Or have all the sexually experienced boys left school by twelfth grade? These are all possibilities; surveys cannot tell us.

To understand responses in more depth, another method can be used—the case study, which is an in-depth study of one person. Case studies usually require personal interviews, background information, test or questionnaire results, and more. Although case studies seem more accurate than more superficial measures, the assumptions and interpretations of the researcher may bias the results.

25

Even if accurate, the case study applies only to one person, who may be quite unlike other people. For instance, my report on my nephew David is a case study, but David is unique: Other embryos exposed to rubella may have quite different lives than David’s.

Studying Development over the Life Span

In addition to conducting observations, experiments, and surveys, developmentalists must be mindful of the last part of our definition of the science of human development—they measure how people change (or remain the same) over time. To do this they use one of three basic research designs: cross-sectional, longitudinal, or cross-sequential (see Figure 1.9).

FIGURE 1.9
Which Approach Is Best? Cross-sequential research is the most time-consuming and complex, but it yields the best information. One reason that hundreds of scientists conduct research on the same topics, replicating one another’s work, is to gain some advantages of cross-sequential research without waiting for decades.

26

Question 1.19

OBSERVATION QUIZ Longitudinal research best illustrates continuity and discontinuity. For Sarah-Maria, what changed over 30 years and what didn’t?

Of course, much changed and much did not change, but evident in the photos is continuity in Sarah–Maria's happy smile and the discontinuity in her hairstyle (which shows dramatic age and cohort changes).

Six Stages of Life These photos show Sarah-Maria, born in 1980 in Switzerland, at six stages of her life: infancy (age 1), early childhood (age 3), middle childhood (age 8), adolescence (age 15), emerging adulthood (age 19), and adulthood (age 30).

Cross-Sectional Research

The quickest and least expensive way to study development over time is with cross-sectional research, in which groups of people of one age are compared with groups of people of another age. Such research has found, for instance, that in the United States in 2012, 74 percent of men aged 25 to 29 were in the labor force, but only 52 percent of those aged 60 to 64 were (U.S. Bureau of Labor Statistics, 2011). It seems that about one-third of all men stop working between age 30 and age 60. Younger adults might imagine them golfing in the sun, happy with their pensions and free time.

Cross-sectional design seems simple. However, it is difficult to ensure that the various groups being compared are similar in every way except age. In this example, the younger U.S. men, on average, had more education than the older ones. Thus, what seems to be the result of age might actually have to do with schooling: Perhaps education, not age, accounted for the higher employment rates of the younger adults. Or perhaps age discrimination was the problem: The older adults may have wanted jobs but been unable to get them.

Longitudinal Research

To help discover whether age itself rather than cohort causes a developmental change, scientists undertake longitudinal research. This research requires collecting data repeatedly on the same individuals as they age. Longitudinal research is particularly useful in tracing development over many years.

Longitudinal research has several drawbacks, however. Over time, participants may withdraw, move to an unknown address, or die. Another problem is that participants become increasingly aware of the goals of the study—knowledge that makes them less typical, and thus the results become less valid.

Finally, the historical context changes, which limits the current relevance of data on people born decades ago. Results from longitudinal studies of people born in 1910 may not be applicable to people born in 2010.

27

Especially for Future Researchers What is the best method for collecting data?

There is no best method for collecting data. The method used depends on many factors, such as the age of participants (infants can't complete questionnaires), the question being researched, and the time frame.

Longitudinal research requires years of data. For example, alarm about possible future harm caused by ingesting phthalates and bisphenol A (BPA) (chemicals used in manufacturing) from plastic baby bottles and infant toys leads many parents to use glass baby bottles. But perhaps the risk of occasional shattered glass causes more harm than the chemicals in plastic, or perhaps the mother’s use of cosmetics, which puts phthalates in breast milk, is a much greater source of the chemicals than any bottles (Wittassek et al., 2011).

Could breast-feeding harm infants? Virtually every scientist and pediatrician is convinced that the benefits of breast milk outweigh any dangers. However, we need accurate predictions about the future for infants who are breast- or bottle-fed now.

Another issue is whether e-cigarettes will entice more teenagers to become addicted to nicotine. Those who profit from vaping say no; public health doctors fear yes. Only longitudinal research can answer for certain, but then it might be too late (Hajek et al., 2014; Bhatnagar et al., 2014; Dutra & Glantz, 2014).

Cross-Sequential Research

Scientists have found a third strategy, called cross-sequential research, which combines cross-sectional and longitudinal research. (It is also referred to as cohort-sequential or time-sequential research.) With this design, researchers study several groups of people of different ages (a cross-sectional approach) and follow them over the years (a longitudinal approach).

With cross-sequential design, researchers compare a group of, say, 6-year-olds, with data on the same individuals at birth and age 3 (a longitudinal approach); they also compare this data with data from another group at birth and at age 3 and with data on a group of newborns (a cross-sectional approach) (see Figure 1.9).

If those 6-year-olds were similar at birth to the new babies and the 3-year-olds, then the newborns will probably develop as the 6-year-old did. But if historical changes have made the new babies healthier, or heavier, or different in any other notable way, then some data on the 6-year-olds may not apply.

Here is one example. Cross-sequential research discovered that when a father is absent or unemployed, his children suffer more in middle childhood than infancy (Sanson et al., 2011). That conclusion could not have been reached by cross-sectional or longitudinal research alone.

Compare These With Those These children seem ideal for cross-sectional research—they are school children of both sexes and many ethnicities. Their only difference seems to be age, so a study might conclude that 6-year-olds raise their hands but 16-year-olds do not. But any two groups in cross-sectional research may differ in ways that are not obvious—perhaps income, national origin, or culture—and that may be the underlying reason for any observed age differences.

28

SUMMING UP   Scientists use many methods and research designs to investigate development. Ideally, conclusions from one type of study are confirmed by other types. For example, careful and systematic observation can discover phenomena that were previously unnoticed, and then experiments can uncover causes. Surveys are quick but vulnerable to bias in the questions asked, the answers given, and the interpretation of those answers. Case studies are detailed, but it is folly to draw general conclusions from the details of any one individual.

To study change over time, cross-sectional, longitudinal, and cross-sequential designs are used, each with advantages and disadvantages. Cross-sectional is quickest, longitudinal may be more accurate, and cross-sequential combines the two, reaching conclusions that neither cross-sectional nor longitudinal research could find in isolation.

WHAT HAVE YOU LEARNED?

  1. Question 1.20

    t8MPA7aFw3oWbMsEaw0O2PJBD0UUY759WFc2ESdSBdkLLpvSz7ZZpvl9/8imfBMxyxpiSCmYnKZX2F7RZGW5oA==
    Scientific observation is a method of testing a hypothesis by unobtrusively watching and recording participants' behavior in a systematic and objective manner—in a natural setting, in a laboratory, or in searches of archival data. An experiment is designed to determine cause and effect. In contrast to scientific observation, researchers in an experiment control the participants and the interventions.
  2. Question 1.21

    C8G8VQjq0qh0w8zP6Ukg/6lngWsK4D6Mhb8wnrLwgem2ktIK4UFLnaLdYwZkFRAjkjd+ZEVwChHOY/JoHt+QyKCX1+LYQJRjCgHIg5KIp42lTwiCxoA6bd37ILedEtpq
    The purpose of an experiment is to find out whether an independent variable affects the dependent variable. In a typical experiment, two groups of participants are studied. One group, the experimental group, is subjected to the particular treatment or condition (the independent variable) while the other group, the comparison group (also called the control group), is not.
  3. Question 1.22

    5a5SlKHNyyu+wnWpKXwrB0Jq6VznXB9i4eB2QYxmqJDOufYjEw3ZdDPBc13dg/v9AiVB3UgtLJxH/Q5oTwoUog==
    The biggest strengths of the survey method are that it is quick and direct. Its biggest weakness is that answers may not be accurate because people may lie, want to come across favorably, or be influenced by the wording of the questions.
  4. Question 1.23

    OCAjRcDrLNcuGcdo504YtFAzb9bbx8E7VIjdNUblXYQqXRau/IZBa6aw7Qg67hvwmyM/6hpeu0ujsKceMU7IG9fWi8lYp78BPgcNjFDArgwkLDuGUau5i0UUZef+SPsj7H9mpbtkxHzuEtjHT3xVs9o9RtTA8RDquFgKPQ==
    The cross–sectional approach is the quickest and least expensive way to study development over time. In this type of research, groups of people of one age are compared with groups of people of another age. However, a weakness is that it is difficult to ensure that the various groups being compared are similar in every way except age.
  5. Question 1.24

    sfT6mZ7g++FkET+h14buf5HddPGtg+o3RzDV/9Y2rcPrbZTzh4yA4il7yP3wgTcyWKN5f3wkYGPhwtkTNFpowSJcrnUPuV/s
    The biggest advantage of longitudinal research is that it is useful in tracing development over many years. Disadvantages include dropout of participants, participants becoming increasingly aware of the questions or the goals of the study, and the influence of the historical context.
  6. Question 1.25

    PwYKBcrzff99pggg52nErArdW1c8d1LjsvVO3bkoZiazYMLuenVp5Zlth6szwnNQIp2VfvJY+Tljaegk1X1b8GlmYXRYOy/XmBWurCQ9sGgBij9Be0wrvxMNhBPB1PHPdRkigpAU16RUdy/wYeiCL0YfF2WIx/E07EjpUQ==
    A cross–sequential design lets researchers study several groups at different ages (like a cross–sectional study does) and follow them over time (like a longitudinal study does). However, cross–sectional research can be affected by historical changes, which may skew the results.