Chapter 1: Where Do Data Come From?

Sample surveys

You just don’t under-stand A sample survey of journalists and scientists found quite a communications gap. Journalists think that scientists are arrogant, while scientists think that journalists are ignorant. We won’t take sides, but here is one interesting result from the survey: 82% of the scientists agree that the “media do not understand statistics well enough to explain new findings” in medicine and other fields.

You don’t have to eat the entire pot of soup to know it needs more salt. That is the idea of sampling: to gain information about the whole by examining only a part. Sample surveys are an important kind of observational study. They survey some group of individuals by studying only some of its members, selected not because they are of special interest but because they represent the larger group. Here is the vocabulary we use to discuss sampling.

Populations and samples

The population in a statistical study is the entire group of individuals about which we want information.

A sample is the part of the population from which we actually collect information and is used to draw conclusions about the whole.

Notice that the population is the group we want to study. If we want information about all U.S. college students, that is our population—even if students at only one college are available for sampling. To make sense of any sample result, you must know what population the sample represents. Did a preelection poll, for example, ask the opinions of all adults? Or citizens only? Registered voters only? Democrats only? The sample consists of the people we actually have information about. If the poll can’t contact some of the people it selected, those people aren’t in the sample.

The distinction between population and sample is basic to statistics. The following examples illustrate this distinction and also introduce some major uses of sampling. These brief descriptions also indicate the variables measured for each individual in the sample.

EXAMPLE 4 Public opinion polls

Polls such as those conducted by Gallup and many news organizations ask people’s opinions on a variety of issues. The variables measured are responses to questions about public issues. Though most noticed at election time, these polls are conducted on a regular basis throughout the year. For a typical opinion poll:

Population: U.S. residents 18 years of age and over. Noncitizens and even illegal immigrants are included.

Sample: Between 1000 and 1500 people interviewed by telephone.

EXAMPLE 5 The Current Population Survey

Government economic and social data come from large sample surveys of a nation’s individuals, households, or businesses. The monthly Current Population Survey (CPS) is the most important government sample survey in the United States. Many of the variables recorded by the CPS concern the employment or unemployment of everyone over 16 years old in a household. The government’s monthly unemployment rate comes from the CPS. The CPS also records many other economic and social variables. For the CPS:

Population: The more than 123 million U.S. households. Notice that the individuals are households rather than people or families. A household consists of all people who share the same living quarters, regardless of how they are related to each other.

Sample: About 60,000 households interviewed each month.

EXAMPLE 6 TV ratings

Market research is designed to discover what consumers want and what products they use. One example of market research is the television-rating service of Nielsen Media Research. The Nielsen ratings influence how much advertisers will pay to sponsor a program and whether or not the program stays on the air. For the Nielsen national TV ratings:

Population: The more than 116 million U.S. households that have a television set.

Sample: About 25,000 households that agree to use a “people meter” to record the TV viewing of all people in the household.

The variables recorded include the number of people in the household and their age and sex, whether the TV set is in use at each time period, and, if so, what program is being watched and who is watching it.

EXAMPLE 7 The General Social Survey

Social science research makes heavy use of sampling. The General Social Survey (GSS), carried out every second year by the National Opinion Research Center at the University of Chicago, is the most important social science sample survey. The variables cover the subject’s personal and family background, experiences and habits, and attitudes and opinions on subjects from abortion to war.

Population: Adults (aged 18 and over) living in households in the United States. The population does not include adults in institutions such as prisons and college dormitories. It also does not include persons who cannot be interviewed in English.

Sample: About 3000 adults interviewed in person in their homes.

NOW IT’S YOUR TURN

Question 1.1

1.1 Legalizing marijuana. The Pew Research Center conducted a poll on March 25–29, 2015. They asked:

Do you think the use of marijuana should be made legal or not?

The Pew Research Center reported that the poll consisted of telephone interviews with 1500 randomly selected adult Americans. What do you think the population is? What is the sample?

Most statistical studies use samples in the broad sense. For example, the 638 children with leukemia in Example 3 are supposed to represent all children with leukemia. We usually reserve the dignified term “sample survey” for studies that use an organized plan to choose a sample that represents some specific population. The children with leukemia were patients at centers that specialize in treating children’s cancer. Expert judgment says they are typical of all leukemia patients, even though they come only from special types of hospitals. A sample survey doesn’t rely on judgment: it starts with an entire population and uses specific, quantifiable methods to choose a sample to represent the population. Chapters 2, 3, and 4 discuss the art and science of sample surveys.