1.4 Using the Scientific Method

There are hundreds of ways to design scientific studies and analyze their results, and researchers continually try to make sure that their data are valid and convincing. Often statistical measures help scientists discover relationships between various aspects of the data. (Some statistical perspectives are presented in TABLE 1.6.) Every research design, method, and statistical measure has strengths as well as weaknesses. Now we describe three basic types of research designs—observation, the experiment, and the survey—and then three ways developmentalists study change over time.

Can They See Her? No. This scientist is observing three boys who are deaf through a window that is a mirror on the other side. Her observations will help them learn to communicate.

ALAMY

Who Participates? For all these measures, the characteristics of the people who participate in the study (formerly called subjects, now called participants) are important, as is the number of people who are studied.

Observation

Scientific observation requires researchers to record behaviour systematically and objectively. Observations often occur in a naturalistic setting (such as a home, school, or public park), where people behave as they usually do and ideally where the observer can be ignored or go unnoticed. Scientific observation can also occur in a laboratory, where scientists record human reactions in various situations, often with wall-mounted video cameras and the scientist in another room.

Observation is crucial in developing hypotheses, as Beal did when she wanted to understand what caused sudden infant death syndrome. However, observation provides issues to explore, not proof. For SIDS, observed differences between Chinese and Australian infants included prenatal care, maternal diet, parental age, breastfeeding, facial features, baby blanket fabrics, and more. Beal’s hypothesis that the crucial difference was sleeping position needed an experiment.

The Experiment

The experiment is the usual research method used to establish what causes what. In the social sciences, experimenters typically impose a particular treatment on a group of volunteer participants (formerly referred to as subjects) or expose them to a specific condition and then note whether their behaviour changes.

In technical terms, the experimenters manipulate an independent variable, which is the imposed treatment or special condition (also called the experimental variable; a variable is anything that can vary). They note whether this independent variable affects whatever they are studying, called the dependent variable (which depends on the independent variable).

Thus, the independent variable is the new, special treatment; any change in the dependent variable is the result. In Beal’s research, convincing some European-Australian parents to put their infants to sleep on their backs (the independent variable) was crucial to connect cause (sleeping position) and effect (survival).

The purpose of an experiment is to find out whether an independent variable affects the dependent variable. In a typical experiment (as diagrammed in Figure 1.12), two groups of participants are studied. One group is called the experimental group, which gets a particular treatment (the independent variable). The other group is the comparison group (also called the control group), which does not. When the scores, or results, of the experimental group and control group are significantly different, based on statistical analysis, then we know that the particular treatment was effective, for better or worse.

FIGURE 1.12 How to Conduct an Experiment The basic sequence diagrammed here applies to all experiments. Many additional features, especially the statistical measures listed in TABLE 1.6 and various ways of reducing experimenter bias, affect whether publication occurs. (Scientific journals reject reports of experiments that were not rigorous in method and analysis.)

Another example of an experiment in this chapter was the study involving 11-year-old African-American boys and their parents. As you remember, half of them (the experimental group) attended special seminars, while the other half (the comparison group) did not. Having that comparison was crucial in showing that the seminars made a difference for the boys with the short version of the 5-HTTLPR gene.

The Survey

A third research method is the survey. Information is collected from a large number of people by interview, questionnaire, or some other means. This is a quick, direct way to obtain data.

Unfortunately, although surveys may be quick and direct, they are not necessarily accurate. When pollsters try to predict elections, they survey thousands of potential voters. They hope that the people they survey will vote as they say they will, that undecided people will follow the trends, and that people who refuse to tell their opinion, or who are not included, will be similar to those surveyed. None of this is certain. Some people lie, some change their minds, some (especially those who don’t have phones or who never talk to strangers) are never counted.

Also, survey answers are influenced by the wording and the sequence of the questions. For example, class evaluations of professors and the courses they teach are used at many universities and colleges. One could imagine that if students found a certain professor charismatic and funny, the positive ratings for that professor would be higher if students were asked “Was the professor able to engage with the students?” instead of “Was the professor well organized?” Course ratings might also be higher if the questions about the professor came first, followed by those about the course content and structure, since the students’ good feelings for the professor would carry over to the course itself.

Another factor that affects survey results is that respondents present themselves as they would like to be perceived. For example, a survey done for CBC News in 2011 asked Canadians if they believed they were overweight. About 44 percent said they were overweight, 7 percent admitted to being obese, and 43 percent stated that they were at a healthy weight. According to Dr. Arya Sharma, obesity management professor at the University of Alberta, two-thirds of Canadians are overweight and 24 percent are clinically obese (CBC News, 2011a)!

To understand responses in more depth, another method can be used—the case study, which is an in-depth study of one person. Case studies usually require personal interviews, background information, test or questionnaire results, and more. Although in some ways case studies seem more accurate than more superficial measures, in other ways they are not: The assumptions and interpretations of the researcher are more likely to bias the results than would a survey that has been validated on hundreds of participants.

Even if accurate, the case study applies only to one person, who may be quite unlike other people. For instance, the report on David is a case study, but David is unique: Other embryos exposed to rubella may have quite different lives than David’s.

Studying Development over the Life Span

In addition to conducting observations, experiments, and surveys, developmentalists must measure how people change or remain the same over time, as our definition of the science of human development stressed. Remember that systems are dynamic, ever-changing. To capture that dynamism, developmental researchers use one of three basic research designs: cross-sectional, longitudinal, or cross-sequential (see Figure 1.13).

FIGURE 1.13 Which Approach Is Best? Cross-sequential research is the most time-consuming and complex, but it yields the best information. One reason that hundreds of scientists conduct research on the same topics, replicating one another’s work, is to gain some advantages of cross-sequential research without waiting for decades.

Cross-Sectional ResearchThe quickest and least expensive way to study development over time is with cross-sectional research, in which groups of people of one age are compared with people of another age. For instance, in Canada in 2012, an estimated 92 percent of men aged 25 to 44 were in the labour force, but only 69 percent of those aged 55 to 64 were (Statistics Canada, 2012b). It seems that one-quarter of all men stop working between ages 45 and 55.

Cross-sectional design seems simple. However it is difficult to ensure that the various groups being compared are similar in every way except age. In this example, the younger men, on average, may have had more education than the older ones. Thus, what seems to be the result of age might actually have to do with schooling: Perhaps education, not age, accounted for the higher employment rates of the younger adults. Or perhaps age discrimination was the problem. The older adults may have wanted jobs but been unable to get them.

Compare Groups These diverse groups seem ideal for cross-sectional research. The younger ones (left) have their hands all over each other and express open-mouth joy—but with age, even smiling classmates, like these high school friends (right), are more restrained. However, with cross-sectional research, it is not certain whether such contrasts are the direct result of chronological age or the result of other variables—perhaps income, cohort, or culture.

CORBIS FLIRT/ALAMY
CATHY WALSH

Five Stages of Life These photos show Susan in infancy (age 7 months), middle childhood (age 9 years), adolescence (age 12), emerging adulthood (age 23), and adulthood (age 30). Continuity (her smile) and discontinuity (her hair) are both evident in longitudinal research.

JAMES CHUANG

Longitudinal ResearchTo help discover whether age itself rather than cohort or economic differences causes a developmental change, scientists undertake longitudinal research. This requires collecting data repeatedly on the same individuals as they age. Longitudinal research is particularly useful in tracing development over many years (Elder & Shanahan, 2006).

For example, in 1994 two agencies of the Canadian government began collaborating on the National Longitudinal Survey of Children and Youth (NLSCY). About 25 000 children were questioned every two years until they reached adulthood. The objective of the NLSCY was to gather information about the critical factors affecting child development in Canada, and then to use this information for policy analysis and program development.

The data collected by the NLSCY has also been very useful for developmental researchers in Canadian universities. As an example, with the wealth of information gathered over time, one group of researchers was able to explore the effects of maternal depression on emotional disorders in adolescents (Naicker et al., 2012). Based on survey responses of more than 900 children, their findings revealed that adolescents who were initially exposed to their mothers’ depression between the ages of 2 and 5 were twice as likely to experience some form of emotional disorder than children who were not exposed to maternal depression at this age. However, those exposed to maternal depression in the first postpartum year or after the age of 5 were not significantly affected. This discovery allowed the research team to conclude that there is a sensitive period for initial exposure to maternal depression that may negatively affect children’s mental health many years later.

It is important to note, however, that longitudinal research has several drawbacks. Over time, participants may withdraw, move to an unknown address, or die. These losses can skew the final results if those who disappear are unlike those who stay, as they usually are. Another problem is that participants become increasingly aware of the questions or the goals of the study and that awareness may cause them to change in ways that differ from most other people.

Probably the biggest problem comes from the historical context. Science, popular culture, and politics alter life experiences, and those changes limit the current relevance of data collected on people born decades ago. Results from longitudinal studies of people born in 1900, as they made their way through childhood, adulthood, and old age, may not be relevant to people born in 2000. Regarding male employment, voluntary retirement before age 60 has been less common over the past decade not because people have changed but because the exosystem has.

Furthermore, longitudinal research requires years of data. For example, alarm about possible future harm caused by ingesting phthalates (chemicals used in manufacturing) in plastic baby bottles and infant toys leads many parents to use glass baby bottles. But perhaps the risk of occasional shattered glass causes more harm than the chemicals in plastic, or perhaps the mother’s use of cosmetics, which puts phthalates in breast milk, is a much greater source of the chemicals than any bottles (Wittassek et al., 2011). Could breastfeeding harm infants? The benefits of breast milk probably outweigh the dangers, but we want answers now, not in decades.

Cross-Sequential ResearchScientists have discovered a third strategy, combining cross-sectional and longitudinal research. This combination is called cross-sequential research (also referred to as cohort-sequential or time-sequential research). With this design, researchers study several groups of people of different ages (a cross-sectional approach) and follow them over the years (a longitudinal approach).

A cross-sequential design lets researchers compare findings for a group of, say, 18-year-olds with findings for the same individuals at age 10, as well as with findings for groups who were 18 a decade or two earlier and with findings for groups who are currently 10 years old (see Figure 1.13). Cross-sequential research is the most complicated, in recruitment and analysis, but it lets scientists disentangle age from history.

One well-known cross-sequential study (the Seattle Longitudinal Study) found that some intellectual abilities (vocabulary) increase even after age 60, whereas others (speed) start to decline at age 30 (Schaie, 2005). This confirms the multidirectional nature of development. This study also discovered that declines in math ability are more closely related to education than to age, a finding that neither cross-sectional nor longitudinal research alone could reveal.

Some more recent cross-sequential research focuses on the mental health of children. Cross-sequential studies find that many factors are influential throughout childhood, but that some harm schoolchildren more than babies. Father absence or unemployment is one of them (Sanson et al., 2011).