Correlation is a marvelous tool because it allows us to know the direction and strength of a relation between two variables. We can also use a correlation coefficient to develop a prediction tool—an equation to predict a person’s score on a scale dependent variable from his or her score on a scale independent variable. For instance, the MSU research team used the amount of time a student spends on Facebook to predict a high score on a measure of social capital.

Being able to predict the future is powerful stuff, but it’s not magic—it’s statistics. Of course, statistical prediction requires building in some margin of error. For example, many universities use variables such as high school grade point average (GPA) and Scholastic Aptitude Test (SAT) score to predict the success of prospective students. They aren’t perfect predictions, but they are much better than gazing into a crystal ball. Similarly, insurance companies input demographic data into an equation to predict the likelihood of a class of people (such as young male drivers) to submit a claim. Mark Zuckerberg, the founder of Facebook, is even alleged to have used data from Facebook users to predict breakups of romantic relationships! He used independent variables, such as the amount of time looking at others’ Facebook profiles, changes in postings to others’ Facebook walls, and photo-tagging patterns, to predict the dependent variable: the end of a relationship as evidenced by the user’s Facebook relationship status. He was right about one-third of the time (“Can Facebook Predict Your Breakup?,” 2010).

395

The name for the prediction tool that we’ve been discussing is regression, a statistical technique that can provide specific quantitative information that predicts relations between variables. More specifically, simple linear regression is a statistical tool that lets us predict a person’s score on a dependent variable from his or her score on one independent variable.

Simple linear regression allows us to calculate the equation for a straight line that describes the data. Once we can graph that line, we can look at any point on the x-axis and find its corresponding point on the y-axis. That corresponding point is what we predict for y. (Note: As with the Pearson correlation coefficient, we are not able to use simple linear regression if the data do not form the pattern of a straight line.) Let’s consider an example of research that uses regression techniques, and then walk through the steps to develop a regression equation.

Christopher Ruhm, an economist, often uses regression in his research. In one study, he wanted to explore the reasons for his finding (Ruhm, 2000) that the death rate decreases when unemployment goes up—a surprising negative relation between the death rate and an economic indicator. He took this relation a step further, into the realm of prediction: He found that an increase of 1% in unemployment predicted a decrease in the death rate of 0.5%, on average. In other words, a poorer economy predicted better health!

To explore the reasons for this surprising finding, Ruhm (2006) conducted regression analyses for independent variables related to health (smoking, obesity, and physical activity) and dependent variables related to the economy (income, unemployment, and the length of the workweek). He analyzed data from a sample of nearly 1.5 million participants collected from telephone surveys between 1987 and 2000. Among other things, Ruhm found that a decrease in working hours predicted decreases in smoking, obesity, and physical inactivity.

Regression can take us a step beyond correlation. Regression can provide specific quantitative predictions that more precisely explain relations among variables. For example, Ruhm reported that a decrease in the workweek of just 1 hour predicted a 1% decrease in physical inactivity. Ruhm suggested that shorter working hours free up time for physical activity—something he might not have thought of without the more specific quantitative information provided by regression. Let’s now conduct a simple linear regression analysis using information that we’re already familiar with: z scores.

In Chapter 13, we calculated a Pearson correlation coefficient to quantify the relation between students’ numbers of absences from statistics class and their statistics final exam grades; the data for the 10 students in the sample are shown in Table 14-1. Remember, the mean number of absences was 3.400; the standard deviation for these data is 2.375. The mean exam grade was 76.000; the standard deviation for these data is 15.040. The Pearson correlation coefficient that we calculated in Chapter 13 was –0.85, but simple linear regression can take us a step further. We can develop an equation to predict students’ final exam grades from their numbers of absences.

396

Let’s say that a student (let’s call him Skip) announces on the first day of class that he intends to skip five classes during the semester. We can refer to the size and direction of the correlation (–0.85) as a benchmark to predict his final exam grade. To predict his grade, we unite regression with a statistic we are more familiar with: z scores. If we know Skip’s z score on one variable, we can multiply by the correlation coefficient to calculate his predicted z score on the second variable. Remember that z scores indicate how far a participant falls from the mean in terms of standard deviations. The formula, called the standardized regression equation because it uses z scores, is:

z_Ŷ = (r_XY)(z_X)

The subscripts in the formula indicate that the first z score is for the dependent variable, Y, and that the second z score is for the independent variable, X. The ˆ symbol over the subscript Y, called a “hat” by statisticians, refers to the fact that this variable is predicted. This is the z score for “Y hat”—the z score for the predicted score on the dependent variable, not the actual score. We cannot, of course, predict the actual score, and the “hat” reminds us of this. When we refer to this score, we can either say “the predicted score for Y” (with no hat, because we have specified with words that it is predicted) or we can use the hat, Ŷ, to indicate that it is predicted. (We would not use both expressions because that would be redundant.) The subscripts X and Y for the Pearson correlation coefficient, r, indicate that this is the correlation between variables X and Y.

398

You may remember the equation for a line that you learned in geometry class. The version you likely learned was: y = m(x) + b. (In this equation, b is the intercept and m is the slope.) In statistics, we use a slightly different version of this formula:

Ŷ = a + b(X)

In the regression formula, a is the intercept, the predicted value for Y when X is equal to 0, which is the point at which the line crosses, or intercepts, the y-axis. In Figure 14-1, the intercept is 5. b is the slope, the amount that Y is predicted to increase for an increase of 1 in X. In Figure 14-1, the slope is 2. As X increases from 3 to 4, for example, we see an increase in what we predict for a Y of 2: from 11 to 13. The equation, therefore, is: Ŷ = 5 + 2(X). If the score on X is 6, for example, the predicted score for Y is: Ŷ = 5 + 2(6) = 5 + 12 = 17. We can verify this on the line in Figure 14-1. Here, we were given the regression equation and regression line, but usually we have to determine these from the data. In this section, we learn the process of calculating a regression equation from data.

399

Once we have the equation for a line, it’s easy to input any value for X to determine the predicted value for Y. Let’s imagine that one of Skip’s classmates, Allie, anticipates two absences this semester. If we had a regression equation, then we could input Allie’s score of 2 on X and find her predicted score on Y. But first we have to develop the regression equation. Using the z score regression equation to find the intercept and slope enables us to “see” where these numbers come from in a way that makes sense (Aron & Aron, 2002). For this, we use the z score regression equation: z_Ŷ = (r_XY)(z_X).

400

The steepness of the slope tells us the amount that the dependent variable changes as the independent variable increases by 1. So, for the skipping class and exam grades example, the slope of –5.39 tells us that for each additional class skipped, we can predict that the exam grade will be 5.39 points lower. Let’s say that another professor uses skipped classes to predict the class grade on a GPA scale of 0–4. And let’s say that we found a slope of –0.23 with these data. For each additional skipped class, we would predict that the grade, in terms of the 0–4 scale, would decrease by 0.23. The problem here is that we can’t directly compare one professor’s findings with another professor’s findings. A decrease of 5.39 is larger than a decrease of 0.23, but they’re not comparable because they’re on different scales.

This problem might remind you of the problems we faced in comparing scores on different scales. To appropriately compare scores, we standardized them using the z statistic. We can standardize slopes in a similar way by calculating the standardized regression coefficient. The standardized regression coefficient, a standardized version of the slope in a regression equation, is the predicted change in the dependent variable in terms of standard deviations for an increase of 1 standard deviation in the independent variable. It is symbolized by β and is often called a beta weight because of its symbol (pronounced “beta”). It is calculated using the formula:

We calculated the slope, –5.39, earlier in this chapter. We calculated the sums of squares in Chapter 13. Table 14-4 repeats part of the calculations for the denominator of the correlation coefficient equation. At the bottom of the table, we can see that the sum of squares for the independent variable of classes skipped is 56.4 and the sum of squares for the dependent variable of exam grade is 2262. By inputting these numbers into the formula, we calculate:

403

Notice that this result is the same as the Pearson correlation coefficient of –0.85. In fact, for simple linear regression, it is always exactly the same. Any difference would be due to rounding decisions for both calculations. Both the standardized regression coefficient and the correlation coefficient indicate the change in standard deviation that we expect when the independent variable increases by 1 standard deviation. Note that the correlation coefficient is not the same as the standardized regression coefficient when an equation includes more than one independent variable, a situation we’ll encounter later in the section “Multiple Regression.”

Because the standardized regression coefficient is the same as the correlation coefficient with simple linear regression, the outcome of hypothesis testing is also identical. The hypothesis-testing process that we used to test whether the correlation coefficient is statistically significantly different from 0 can also be used to test whether the standardized regression coefficient is statistically significantly different from 0. As you’ll remember from Chapter 13, the Pearson correlation coefficient, r = –0.85, was larger in magnitude than the critical value of –0.632 (determined based on 8 degrees of freedom and a p level of 0.05). We rejected the null hypothesis and concluded that number of absences and exam grade seemed to be negatively correlated.

404