Experiments
Our goal in choosing a sample is a picture of the population, disturbed as little as possible by the act of gathering information. All observational studies share the principle “observe but don’t disturb.” When Jane Goodall first began observing chimpanzees in Tanzania, she set up a feeding station where the chimps could eat bananas. She later said that was a mistake because it might have changed the apes’ behavior.
In experiments, on the other hand, we want to change behavior. In doing an experiment, we don’t just observe individuals or ask them questions. We actively impose some treatment in order to observe the response. Experiments can answer questions such as “Does aspirin reduce the chance of a heart attack?” and “Do a majority of college students prefer Pepsi to Coke when they taste both without knowing which they are drinking?”
Experiments
An experiment deliberately imposes some treatment on individuals in order to observe their responses. The purpose of an experiment is to study whether the treatment causes a change in the response.
13
Example 8 Helping welfare mothers find jobs
The Urban Institute in Washington DC reports that most adult welfare recipients are single mothers in their 20s and 30s with one or two children. Observational studies of welfare mothers show that many are able to increase their earnings and leave the welfare system. Some take advantage of voluntary job-training programs to improve their skills. Should participation in job-training and job-search programs be required of all able-bodied welfare mothers? Observational studies of the current system cannot tell us what the effects of such a policy would be. Even if the mothers studied are a properly chosen sample of all welfare recipients, those who seek out training and find jobs may differ in many ways from those who do not. They are observed to have more education, for example, but they may also differ in values and motivation, things that cannot be observed.
To see if a required jobs program will help mothers escape welfare, such a program must actually be tried. Choose two similar groups of mothers when they apply for welfare. Require one group to participate in a job-training program, but do not offer the program to the other group. This is an experiment. Comparing the income and work record of the two groups after several years will show whether requiring training has the desired effect.
[Design Element Start: Heading Level-2 Inserted by Jouve]
Quantitative variables: Stemplots and histograms
A stemplot (also called a stem-and-leaf plot) gives a quick picture of the shape of a distribution while including the actual numerical values in the graph. Stemplots work best for small numbers of observations that are all greater than 0.
[Design Element End]
NOW IT’S YOUR TURN
NOW IT’S YOUR TURN
Question 1.2
1.2 Posting lectures on the class website.
To determine what students found most helpful, an educational researcher examined student comments from several classes he taught. Mentioned most often was the fact that copies of lectures were posted on the class website for students to download. The researcher recommended that instructors post lecture notes on their course websites. Was this an observational study or an experiment?
The welfare example illustrates the big advantage of experiments over observational studies: in principle, experiments can give good evidence for cause and effect. If we design the experiment properly, we start with two very similar groups of welfare mothers. The individual women of course differ from each other in age, education, number of children, and other respects. But the two groups resemble each other when we look at the ages, years of education, and number of children for all women in each group. During the experiment, the women’s lives differ, but there is only one systematic difference between the two groups: whether or not they are in the jobs program. All live through the same good or bad economic times, the same changes in public attitudes, and so on. If the training group does much better than the untrained group in holding jobs and earning money, we can say that the training program actually causes this happy outcome.
[Design Element Start: Inserted From IPS9e]
Example 1.24
Mean time to start a business.
TTS24
The mean time to start a business is
The mean time to start a business for the 24 countries in our data set is 16.3 days. Note that we have rounded the answer. Our goal in using the mean to describe the center of a distribution is not to demonstrate that we can compute with great accuracy. The additional digits do not provide any additional useful information. In fact, they distract our attention from the important digits that are meaningful. Do you think it would be better to report the mean as 16 days?
[Design Element End]
14
One of the big ideas of statistics is that experiments can give good evidence that a treatment causes a response. A big idea needs a big caution: statistical conclusions hold “on the average” for groups of individuals. They don’t tell us much about one individual. If on the average the women in the training program earned more than those who were left out, that says that the program achieved its goal. It doesn’t say that every woman in such a program will be helped. And a big idea may also raise big questions: if we hope the training will raise earnings, is it ethical to offer it to some women and not to others? Chapters 5 and 6 explain how to design good experiments, and Chapter 7 looks at ethical issues.
[Design Element Start: Heading Level-3 Inserted by Jouve]
Data analysis in action: Don’t hang up on me
Many businesses operate call centers to serve customers who want to place an order or make an inquiry. Customers want their requests handled thoroughly. Businesses want to treat customers well, but they also want to avoid wasted time on the phone. They therefore monitor the length of calls and encourage their representatives to keep calls short.
[Design Element End]