7.3 7.2 Bad Sampling Methods
How can we choose a sample that is truly representative of the population? The easiest way, but not the best way, to select a sample is to choose individuals close at hand. If we are interested in finding out how many people have jobs, for example, we could go to a shopping mall and ask people passing by if they are employed.
Convenience Sample DEFINITION
A convenience sample is a sample of individuals who are selected because they are members of a population who are easy (i.e., convenient) to reach. Usually, such a sample cannot be trusted to be representative of the population.
EXAMPLE 2
The Inconvenient Truth About Convenience Samples
Going to the mall, standing by a particular entrance, and surveying as many of the people walking through that entrance as you can seems like a fast, convenient way of finding out Americans’ opinions. But people at malls tend to be more prosperous than typical Americans. They are also more likely to be teenagers or retired. The kinds of stores that are near the particular entrance you are standing by could affect the type of people you might more readily encounter. Also, when we decide which people to approach, we may tend (even unconsciously) to avoid poorly dressed or tough-looking individuals. in short, our shopping mall interviews will result in a sample that is not representative of the entire population because we underrepresent certain types of people. For that matter, we also are underrepresenting those Americans who rarely go to malls in the first place.
In your classroom, your professor may try to “sample” the understanding the class has about a topic by simply calling on the nearest two students in the front row. If students who sit near the front have higher levels of preparation, interest, and engagement, the professor will overestimate how well the class as a whole understands the material.
In both scenarios in Example 2, the inaccuracies obtained cannot simply be explained as a sample’s “bad luck.” They are likely to happen every time, with the same pattern, because unscientific sampling methods have bias. In this context, bias refers matter-of-factly to the built-in systematic error of the procedure itself and not to any political or personal prejudice that the person conducting the poll may have.
Bias DEFINITION
The design of a statistical study has bias (i.e., is biased) if it systematically favors certain outcomes.
EXAMPLE 3 Are Online Polls in Line?
The American Family Association (AFA) is a conservative group that claims to stand for “traditional family values.” it has often posted online poll questions on its website; just click on a response to take part. Because the respondents are people who visit this site, the poll results always support AFA’s positions. Well, almost always; a recent AFA online poll asked about allowing same-sex marriage, and before long, email lists and social-network sites favored mostly by young liberals pointed to the AFA poll. Almost 850,000 people responded, and 60% of them favored the legalization of same-sex marriage. This example shows that the results of an online poll can be skewed one way or the other by particular characteristics of the people who choose to go to that website and participate in the poll.
A related example is the website www.ratemyprofessors.com, where students have chosen to evaluate and post comments on more than 1 million college and university instructors worldwide. However, focus group research indicates the ratings may not be representative because the students most motivated to post assessments are those who believe the teacher is either extremely bad or extremely good. A different kind of problem is that this website may have no way of keeping out multiple responses from the same student—or even from the instructors themselves! it is usually better to rely on the official end-of-course evaluation data compiled by the university.
Online polls are now everywhere; some sites such as SurveyMonkey will even provide help in conducting your own online poll. As Example 3 illustrates, however, the results can’t be trusted. People who take the trouble to write in, call in, or visit a website to respond to an open invitation are not representative of the general population. Polls like these are examples of voluntary response sampling.
Voluntary Response Sample DEFINITION
A voluntary response sample consists of people who choose themselves by responding to a general appeal. Voluntary response samples are biased because people with strong opinions are more likely to respond and will therefore be overrepresented in the sample.
Self Check 2
A hospital sends out surveys to patients who visited there for routine medical tests. The survey asks questions related to the patients’ satisfaction with the service provided by the hospital. The hospital analyzes the data from all responses.
- What is the sample? Is this an example of a voluntary response sample or a convenience sample?
Entering Table 7.1 on line 110, here is a list of two-digit numbers:
38 44 84 87 89 18 33 82 46 97 39 36 44 20 06 76 68 80 87 08 Since we have not reached a sample of size 5, we move to line 111 and continue the process:
81 48 66 94 87 60 51 30 92 97 00 41 27 Our random sample of labels consists of 18, 20, 06, 08, and 27. So, the randomly selected songs will be “All You Need Is Love,” “Lady Madonna,” “A Hard Day’s Night,” “Eight Days a Week,” and “The Long and Winding Road.”
- In what population is the hospital interested?
- Although a few students might refuse to complete the questionnaire, the major problem affecting this survey is undercoverage. The sample will not include students who never eat breakfast or students who were out late Thursday night and didn’t show for Friday breakfast. So, the views of large segments of the student population will not be represented in this survey.
- Do you expect these survey results to accurately portray patients’ satisfaction? Explain.
- Both questions have been designed to elicit particular responses: the first to elicit support for smoking in a hotel room, and the second to obtain support for banning smoking in the workplace. The use of the word freedom in Q1 makes smoking appear as a right and therefore makes it more likely that the respondent will answer “yes.” The introductory phrase in Q2 focuses respondents’ attention on the dangers of smoking and makes it more likely that the respondent will answer “no.”