Sample design in the real world
The basic idea of sampling is straightforward: take an SRS from the population and use a statistic from your sample to estimate a parameter of the population. We now know that the sample statistic is altered behind the scenes to partly correct for nonresponse. The statisticians also have their hands on our beloved SRS. In the real world, most sample surveys use more complex designs.
73
EXAMPLE 7 The Current Population Survey
The population that the Current Population Survey (CPS) is interested in consists of all households in the United States (including Alaska and Hawaii). The sample is chosen in stages. The Census Bureau divides the nation into 2007 geographic areas called primary sampling units (PSUs). These are generally groups of neighboring counties. At the first stage, 754 PSUs are chosen. This isn’t an SRS. If all PSUs had the same chance to be chosen, the sample might miss Chicago and Los Angeles. So 428 highly populated PSUs are automatically in the sample. The other 1579 are divided into 326 groups, called strata, by combining PSUs that are similar in various ways. One PSU is chosen at random to represent each stratum.
Each of the 754 PSUs in the first-stage sample is divided into census blocks (smaller geographic areas). The blocks are also grouped into strata, based on such things as housing types and minority population. The households in each block are arranged in order of their location and divided into groups, called clusters, of about four households each. The final sample consists of samples of clusters (not of individual households) from each stratum of blocks. Interviewers go to all households in the chosen clusters. The samples of clusters within each stratum of blocks are also not SRSs. To be sure that the clusters spread out geographically, the sample starts at a random cluster and then takes, for example, every 10th cluster in the list.
The design of the CPS illustrates several ideas that are common in real-world samples that use face-to-face interviews. Taking the sample in several stages with clusters at the final stage saves travel time for interviewers by grouping the sample households first in PSUs and then in clusters. Note that clustering is not an aspect of all sampling strategies, but it can be quite helpful in situations like the CPS.
The most important refinement mentioned in Example 7 is stratified sampling.
Stratified sample
To choose a stratified random sample:
Step 1. Divide the sampling frame into distinct groups of individuals, called strata. Choose the strata according to any special interest you have in certain groups within the population or because the individuals in each stratum resemble each other.
Step 2. Take a separate SRS in each stratum and combine these to make up the complete sample.
74
We must, of course, choose the strata using facts about the population that are known before we take the sample. You might group a university’s students into undergraduate and graduate students or into those who live on campus and those who commute. Stratified samples have some advantages over an SRS. First, by taking a separate SRS in each stratum, we can set sample sizes to allow separate conclusions about each stratum. Second, a stratified sample usually has a smaller margin of error than an SRS of the same size. The reason is that the individuals in each stratum are more alike than the population as a whole, so working stratum by stratum eliminates some variability in the sample.
It may surprise you that stratified samples can violate one of the most appealing properties of the SRS—stratified samples need not give all individuals in the population the same chance to be chosen. Some strata may be deliberately overrepresented in the sample.
EXAMPLE 8 Stratifying a sample of students
A large university has 30,000 students, of whom 3000 are graduate students. An SRS of 500 students gives every student the same chance to be in the sample. That chance is
We expect an SRS of 500 to contain only about 50 grad students— because grad students make up 10% of the population, we expect them to make up about 10% of an SRS. A sample of size 50 isn’t large enough to estimate grad student opinion with reasonable accuracy. We might prefer a stratified random sample of 200 grad students and 300 undergraduates.
You know how to select such a stratified sample. Label the graduate students 0001 to 3000 and use Table A to select an SRS of 200. Then label the undergraduates 00001 to 27000 and use Table A a second time to select an SRS of 300 of them. These two SRSs together form the stratified sample.
In the stratified sample, each grad student has chance
to be chosen. Each of the undergraduates has a smaller chance,
Because we have two SRSs, it is easy to estimate opinions in the two groups separately. The quick and approximate method (page 46) tells us that the margin of error for a sample proportion will be about
75
for grad students and about
for undergraduates.
Because the sample in Example 8 deliberately overrepresents graduate students, the final analysis must adjust for this to get unbiased estimates of overall student opinion. Remember that our quick method works only for an SRS. In fact, a professional analysis would also take account of the fact that the population contains “only” 30,000 individuals—more job opportunities for statisticians.
NOW IT’S YOUR TURN
Question 4.2
4.2 A stratified sample. The statistics department at Cal Poly, San Luis Obispo, had 18 faculty members and 80 undergraduate majors in 2015. Use the Simple Random Sample applet, other software, or Table A, starting at line 111, to choose a stratified sample of one faculty member and one student to attend a reception being held by the university president.
EXAMPLE 9 The woes of telephone samples
In principle, it would seem that a telephone survey that dials numbers at random could be based on an SRS. Telephone surveys have little need for clustering. Stratifying can still reduce variability, however, and so telephone surveys often take samples in two stages: a stratified sample of telephone number prefixes (area code plus first three digits) followed by individual numbers (last four digits) dialed at random in each prefix.
New York, New York New York City, they say, is bigger, richer, faster, ruder. Maybe there’s something to that. The sample survey firm Zogby International says that as a national average, it takes 5 telephone calls to reach a live person. When calling to New York, it takes 12 calls. Survey firms assign their best interviewers to make calls to New York and often pay them bonuses to cope with the stress.
The real problem with an SRS of telephone numbers is that too few numbers lead to households. Blame technology. Fax machines, modems, and cell phones demand new phone numbers. Between 1988 and 2008, the number of households in the United States grew by 29%, but the number of possible residential phone numbers grew by more than 120%. Some analysts believe that, in the near future, we may have to increase the number of digits for telephone numbers from 10 (including the area code) to 12. This will further exacerbate this problem. Telephone surveys now use “list-assisted samples” that check electronic telephone directories to eliminate prefixes that have no listed numbers before random sampling begins. Fewer calls are wasted, but anyone living where all numbers are unlisted is missed. Prefixes with no listed numbers are, therefore, separately sampled (stratification again), perhaps with a smaller sample size than if included in the list-assisted sample, to fill the gap.
76
The proliferation of cell phones has created additional problems for telephone samples. As of December 2013, about 41% of households had cell phones only. Random digit dialing using a machine is not allowed for cell phone numbers. Phone numbers assigned to cell phones are determined by the location of the cell phone company providing the service and need not coincide with the actual residence of the user. This makes it difficult to implement sophisticated methods of sampling such as stratified sampling by geographic location.
It may be that the woes of telephone sampling prompted the Gallup Organization, in recent years, to drop the phrase “random sampling” from the description of their survey methods at the end of most of their polls. This presumably prevents misinterpreting the results as coming from simple random samples. In the organization’s detailed description of survey methods used for the Gallup World Poll and the Gallup Well-Being Index (available online at the Gallup website), the samples are described as involving random sampling.