20

1 Random Sampling

We can use the information gathered from a sample to generalize about the population when it is impractical or impossible to take a census of the entire population. However, if we get a “bad” sample, the information gleaned from the sample will be misleading, with potentially catastrophic consequences. This section introduces a method of sampling that minimizes many potential biases, which could lead to incorrect generalizations about the population. This sampling method is called random sampling. Everyday examples of random sampling include:

Because random samples are not always practical or desirable, this section also discusses some of the many alternative sampling methods available, including stratified sampling and cluster sampling.

What Is a Random Sample, and Why Do We Need It?

Survey sampling, or polling, has now become so widespread that hardly a day goes by without the results of some new poll or survey making the headlines. Polls are a good example of statistical sampling at work. The pollsters canvass about 1000 or so respondents, analyze the sample results, and then report their statistical inference that, for example, “32% of Americans have used a cell phone to access the Internet.”

Today, many polls are conducted quite scientifically, and their results are usually very accurate. However, such was not always the case. In 1936, the Literary Digest had correctly predicted the past three presidential elections and went to work to predict the winner of the contest between Republican Alf Landon and Democrat Franklin Roosevelt. The magazine sent ballots to 10 million citizens. The results ran strongly in favor of Landon, leading the Literary Digest to predict Landon to win the election. About 25% of the ballots were returned, giving the newsweekly a sample size of 2.5 million. George Gallup, on the other hand, was working with a sample size that was about 1000 times smaller than the Literary Digest's. However, Gallup predicted a victory for Roosevelt. Clearly, with more data, the Literary Digest should have been able to give a more accurate prediction, right? Not necessarily. Roosevelt won in a landslide, and the embarrassed Literary Digest later declared bankruptcy.

21

The problem stemmed from the way that the Literary Digest identified its sample. It used lists of people who owned cars and had telephones, which in the 1930s excluded millions of poor and underprivileged people, who overwhelmingly supported Roosevelt. Its sample of 2.5 million, therefore, was highly biased toward the richer folks, who were less likely to have any great fondness for Roosevelt and his New Deal policies. Gallup, on the other hand, chose his sample more scientifically, and even though his sample size was smaller, it was more representative of the population as a whole.

One inexpensive way of eliminating many types of bias is to make sure your sample is a random sample.

For example, today, the Gallup polling organization uses random digit dialing, a computer program that generates random four-digit numbers, which are then appended to the telephone exchanges and area codes. Thus, each household phone number in America has an equal chance of being included in the sample, regardless of whether it is listed or unlisted.

Random samples may be generated using technology, using the Simple Random Sample applet, or using the random number table provided in Table A in the Appendix (page T-2). At the end of this section, we demonstrate how to generate random samples using the TI-83/84 graphing calculator, Excel, Minitab, SPSS, JMP, and CrunchIt!. The Simple Random Sample applet allows you to produce a random sample of up to 100 elements, in the form of a lotto.

2 More Sampling Methods

In certain circumstances, simple random sampling can have shortcomings. A simple random sample may not provide sufficient information about subgroups within the population. For example, suppose you are interested in knowing the proportion of 19- to 22-year-olds in Walnut, California, who consume caffeine. A random sample size of 100 of all the residents in Walnut may yield only a dozen 19- to 22-year-olds, which may be too small a sample to be useful for statistical inference. Therefore, the researcher needs other methods for obtaining samples, depending on the situation and the research question.

Systematic Sampling

Perhaps the easiest method of sampling is systematic sampling, which is used when a random sample is unobtainable. In systematic sampling, each element of the population is numbered, and the sample is obtained by selecting every th element, where is some whole number. The first element selected corresponds to a random whole number between 1 and . The ancient Romans understood well how to use systematic sampling. When a Roman legion mutinied or showed cowardice in battle, every 10th member was selected and summarily executed before his comrades. Literally, the legion was decimated, from the Latin decem, meaning “ten.”

Stratified Sampling

Often, researchers are interested in investigating characteristics of a certain subgroup of a population, such as 19- to 22-year-olds in Walnut, California. In cases like this, the researcher divides the population into subgroups, or strata, according to some characteristic, such as race or gender. Then a random sample is taken from each stratum. In this way, the researcher knows that a sample will be obtained from each stratum and that it will be large enough to provide reliable statistical inference for each stratum.

Cluster Sampling

Cluster sampling is used when the population is widely scattered geographically or poses other logistical difficulties. For example, if we were interested in estimating the mean income of Manhattan residents, it would be time-consuming and expensive to visit 1000 different locations in Manhattan to elicit sample information. In cluster sampling, the population is divided into clusters, such as precincts or city blocks. Then several clusters are chosen at random, and all of the elements within the chosen clusters are selected for the sample. One disadvantage of cluster sampling is that the respondents from within a certain cluster will tend to be more similar to each other than the elements of a random sample would be. For example, if one of the clusters in the Manhattan income survey was a Fifth Avenue block, the mean income of residents there would be at the higher end of the income scale.

25

Convenience Sampling

In convenience sampling, subjects are chosen based on what is convenient for the survey personnel. If you were to estimate the true proportion of females taking an introductory statistics course using only the people in your class, this would be considered a convenience sample. As we shall see in Example 16, convenience sampling usually does not result in a representative sample.

3 Selection Bias and Questionnaire Design

Here, we learn about some common pitfalls in the design and implementation of a survey, including selection bias and the wording of a questionnaire.

Five Factors for Good Questionnaire Design

You may have heard of the aphorism “Be careful what you ask for; you may get it.” This warning is certainly relevant to the issue of questionnaire design. The wording of questions can greatly affect the responses. Here are several factors to consider when designing a questionnaire.

4 Experimental Studies and Observational Studies

Two major types of statistical studies are experimental studies and observational studies. We have seen that researchers can gather data by consulting existing sources, by distributing a questionnaire, or by taking a sample. However, you may not be able to obtain the information you require by using survey or sampling methods. In this case, you may prefer to conduct an experimental study.

We illustrate experimental studies using the following example.

Two additional factors should be considered when designing an experimental study: randomization and replication.

Randomization. Many biases can be introduced into an experiment. For example, a well-meaning doctor may want to place underweight high-risk babies in the group with the heartbeat, in the hope that such babies will flourish. To eliminate biases like these, the placement of the subjects into the treatment and control groups should be done randomly.

Replication. One major theme of statistical investigation is that larger samples are usually better, because they allow more precise inference. In a statistical study, the treatment and the control groups each must contain a large enough number of subjects to allow detection of meaningful differences between the treatment and control. For example, if a researcher examined only three babies with the heartbeat sound and three babies without the heartbeat sound, this would not be a sufficient number of replications. In Chapter 8, “Confidence Intervals,” we will learn how large a sample size is sufficient for the needs of a particular study.

Observational Studies

30

Circumstances exist where it could be impossible, impractical, or unethical for the researcher to place subjects into treatment and control groups. For example, suppose we are interested in whether women who work outside the home suffer less depression than women who remain at home with the children. The explanatory variable here is whether or not a woman works outside the home. However, it is not possible for the researcher to take women and randomly separate them into groups that either work outside the home or do not work outside the home.

Sometimes an experimental study is not possible for ethical reasons. Suppose you are interested in whether babies born to chemically dependent mothers display differences in cognitive skills from babies born to mothers who are not chemically dependent. It is clearly not ethical to randomly assign half of the mothers in the study to become chemically dependent during their pregnancy. Therefore, researchers need another type of statistical study: the observational study. In an observational study, the researcher observes whether the subjects' differences in the predictor variable are associated with differences in the response variable. No attempt is made to create differences in the predictor variable.

A sample survey is an example of an observational study. Data about a response variable may be obtained through the survey, along with information about possible predictor variables. No attempt is made to manipulate the variables. The researcher analyzes the information to determine whether differences in the predictor variable are associated with differences in the response variable.