Chapter 7. Which Professor Should You Take?

Introduction

In this activity, you will create a survey to measure students’ attitudes about different professors. Just like a researcher, you will determine the best questions to ask, the type of responses to offer, and the best sample to survey.

Dr. Natalie J. Ciarocco, Monmouth University

Dr. David B. Strohmetz, Monmouth University

Dr. Gary W. Lewandowski, Jr., Monmouth University

Something to Think About…

Scenario: It seems to happen every semester. As you register for classes, you find there are several professors teaching the same course. Which professor should you take? Should you register for the section taught by Professor Sandman or the one taught by Professor DeBestie? Sure, you have heard stories about these 2 professors from your friends, but their opinions are mixed. Besides, as a budding scientist, you recognize that anecdotes are not necessarily the best evidence. You could search the Internet to see how past students rated each professor, but, again, this could be problematic. The ratings on these websites could have been generated by disgruntled students or professor “groupies.” There must be a better way to find out how students rate the quality of each professor.

Something to Think About…

We often turn to others for guidance when we are unsure of what to do. The problem is that the information provided might be biased or limited, leading us to make poor decisions. How can you best collect information on students’ opinions about a professor? Science, of course, can help you with this task.

Our Research Question

Picking the Best Research Approach

Now that you have a research question (“How do I best measure students’ attitudes about different professors?”), you must decide which research approach to use to answer this question. There are 2 research methods you can choose from:

• Qualitative Research

• Quantitative Research

Developing the Best Measure

Now that you have decided to use the quantitative method, your next task is to develop a strategy for measuring students’ attitudes toward a professor. There are 2 types of measures that you can use:

• Self-report Measure

• Behavioral Measure

Developing the Best Measure

Since you have decided to use a self-report measure to solicit various students’ attitudes toward Professor Sandman and Professor DeBestie, the next step is to decide what type of questions to ask. You have 2 options:

• Open-ended Questions

• Closed-ended Questions

Developing Your Rate-a-Prof Scale

You now know that you are going to use a self-report measure with closed-ended questions to measure students’ attitudes toward Professor Sandman and Professor DeBestie. To accomplish this task, you can develop your own Likert scale (also known as a summated rating scale). Likert scales are frequently used to measure a person’s attitude toward someone or something. In a Likert scale, you ask a person to evaluate a series of statements using a predetermined set of response options. You then sum the responses to these statements to represent the person’s overall attitude toward what is being evaluated.

Developing Your Rate-a-Prof Scale

You have decided to see to what degree students agree or disagree with various statements about a professor. The next question is, how many response alternatives should you provide them with? 2? 5? 50? 100? Your answer depends on the desired sensitivity of your attitude scale.

• Sensitivity

Developing Your Rate-a-Prof Scale

Developing Your Rate-a-Prof Scale

You now know how students will evaluate each statement in your Rate-a-Prof Scale. You will use 5 response alternatives: Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, and Strongly Disagree. Each response alternative will be assigned a numerical value that you will use to determine the overall scale score for the students’ rating of the professor. Higher values will have more positive response alternatives, as follows:

Developing Your Rate-a-Prof Scale

The next question to consider is, how many statements should you include in your scale to minimize overall error in your measurement of students’ attitudes toward a professor?

• Error

Developing Your Rate-a-Prof Scale

For your Rate-a-Prof Scale, you will ask students to indicate how much they agree or disagree with 5 statements about a specific professor. Your next step is to develop these statements. The key is to pinpoint 5 high-quality items that influence students’ attitudes toward a professor. Based on a quick review of the literature on effective teaching, you decide to ask about 5 things in your scale:

1. The professor’s teaching strategies

2. The professor’s expectations for students in the course

3. The professor’s enthusiasm for teaching

4. The professor’s ability to generate student interest in the course

5. The professor’s concern for student problems

You are now ready to write the actual statements for your Rate-a-Prof Scale.

Developing Your Rate-a-Prof Scale

Developing Your Rate-a-Prof Scale

Developing Your Rate-a-Prof Scale

Developing Your Rate-a-Prof Scale

You want to have an item asking about how interesting a professor makes the course for students. Because you are asking students to evaluate their professors, there is 1 problem of which you should be aware—namely, how do you ensure that the students fully consider each item rather than simply agreeing (or disagreeing) with all of them? In cases where a student has strong feelings about a particular professor, the student may agree with all of the statements in an effort to give the professor the highest score (or vice versa). This reflects a response bias known as the acquiescent response set.

• Acquiescent Response Set

One way to control for the acquiescent response set is to have at least 1 item phrased in the opposite direction compared to the other items. A high rating on this item would have a different meaning than a high rating on the other items. This way, if a student positively evaluates a professor on 1 item, he or she will disagree with the corresponding negative statement. If the student agrees (or disagrees) with this particular statement, then it becomes uncertain whether the student is actually responding to each item or blindly agreeing (or disagreeing) with all of them.

Developing Your Rate-a-Prof Scale

Developing Your Rate-a-Prof Scale

Finding a Sample

Now that you have developed your scale, your next step is to determine who you would like to complete your scale.

Finalizing Your Plan

You now have your scale to measure student attitudes toward Professor Sandman and Professor DeBestie, and you have identified what students you are going to ask to complete this scale. But there is 1 potential problem with your strategy for deciding which professor would be best to take a course with next semester—that is, something unrelated to the quality of the professor could influence students’ attitudes toward their professors. You decide to explore this possibility by asking 1 additional question of students after they have completed your scale.

Collecting Your Data

You are able to recruit 30 students who are currently enrolled in Professor Sandman’s class and 30 students who are taking a class with Professor DeBestie to complete your Rate-a-Prof Scale. You ask the students to provide the grade they expect to receive in that professor’s class. You convert the reported letter grade into the corresponding numerical value used in the calculation of GPAs (i.e., A = 4.0; A- = 3.67; B+ = 3.33, etc.).

Computing Your Rate-a-Prof Scale Attitude Score

This is an example of what your data set would look like. The top row shows the variable names, and the other rows display the data for the first 5 respondents’ ratings of Professor Sandman and the first 5 respondents’ ratings of Professor DeBestie.

Computing Your Rate-a-Prof Scale Attitude Score

You now have student ratings for both professors on the 5 individual items. Your next step is to calculate each student’s overall evaluation of the professor based on individual ratings. Before you can do this, you need to remember that one of your items, “I frequently fall asleep in this professor’s class,” was negatively worded. This means that a low rating on this item is equivalent to a high rating on another item. Students who consider someone to be a high-quality professor will tend to agree with the other items, but will disagree with this particular statement. You will need to adjust the responses for the “frequently fall asleep” item so that agreement reflects a positive evaluation of the professor. To do this, you will use a process called reverse-coding.

• Reverse-coding

Computing Your Rate-a-Prof Scale Attitude Score

In reverse-coding, you substitute a respondent’s original answer with the opposite score on the scale. That is, if a person gave a “1” to the item, the response is now changed to a “5.”

	Original response alternatives values
	1	2	3	4	5
	Strongly Disagree	Disagree	Neither Agree nor Disagree	Agree	Strongly Agree
	New response alternatives values after reverse-coding
	5	4	3	2	1
	Strongly Disagree	Disagree	Neither Agree nor Disagree	Agree	Strongly Agree

Computing Your Rate-a-Prof Scale Attitude Score

For each of the ratings below, provide the new value for the “fall asleep” variable after reverse-coding.

Computing Your Rate-a-Prof Scale Attitude Score

Now that you have reverse-coded the responses to the “falling asleep” item, you are ready to calculate a student’s overall rating of the professor. Because higher scores represent more agreement with each item, you can sum all the student responses to your scale items. Your overall score can range from “5” (that is, the student strongly disagreed with every item regarding the professor’s quality) to “25” (the student strongly agreed with every item regarding the professor’s quality).

Computing Your Rate-a-Prof Scale Attitude Score

Calculate the Rate-a-Prof Attitude Score for each set of ratings.

Computing Your Rate-a-Prof Scale Attitude Score

You now have the total Rate-a-Prof Attitude Score as well as the student’s expected grade in that professor’s class for the first 10 respondents in your study. The completed reverse-coding and calculated total scores for your Rate-a-Prof Scale are provided for you in this second data set.

You are now ready to answer the question, Who is the better professor for you to take a course with: Professor Sandman or Professor DeBestie?

Computing Your Rate-a-Prof Scale Attitude Score

This is an example of what your data set would look like. The top row shows the variable names; the other rows display the data for the first 10 participants.

Summarizing Student Evaluations of Each Professor

To help you make your decision, you should look at the mean Rate-a-Prof Attitude Score for each professor. This will provide you with a sense of which professor may be of high quality in the opinion of students who are currently enrolled in those professors’ classes. In addition to looking at the mean for each professor, you should also look at the variability in the overall attitude scores for each professors. If the variability is markedly lower from 1 professor to the other, then we know there is more consistency in student attitudes toward 1 professor when compared to the other professor. While you could look at the highest and lowest overall evaluation scores for each professor, the standard deviation is the best way of summarizing the variability of your scores.

• Variability

• Standard Deviation

Your Turn: Drawing Conclusions

Below are the mean and standard deviation for each professor:

Your Turn: Drawing Conclusions

Before you decide to enroll in Professor DeBestie’s course next semester, you want to see to what extent overall attitude toward a professor based on the Rate-a-Prof Scale relates to the grade a student expects to receive in that professor’s class. If these are strongly related, then your scale might actually be measuring satisfaction with the course grade rather than opinion of the professor’s quality. To measure the degree to which 2 variables are related, you can use a correlation.

• Correlation

Tutorial: Evaluating Output

The following is an example of output comparing the correlation between 2 variables. The researcher wanted to examine the relationship between number of hours spent studying and a student’s grade on a final exam.

Click on the output to learn more about each of its elements.

This is the correlation between the 2 variables. The closer this value is to 1.0, the stronger the 2 variables are related.

Because this value is positive, the 2 variables are positively related. Higher scores on 1 variable are associated with higher scores on the other variable and vice versa. If this value had been negative, then higher scores on 1 variable would have been associated with lower scores on the other variable.

This is the p level or the significance level. It represents the probability or likelihood that the results happened by chance if the 2 variables were truly unrelated. The lower the level, the less likely the result happened by chance. If this number is lower than .05, then you can conclude that you have a statistically significant correlation, or that the 2 variables are related. In this example, .000 is very low and thus significant. Typically, this value would be expressed in an article as p.001 since p levels can never truly be 0.

Your Turn: Drawing Conclusions

Below is the output examining the correlation between overall attitude toward a professor and expected grade in the class.

Your Turn: Results

Take Home Message

RapidEye/iStockphoto/Getty Images

Congratulations! You have successfully completed this activity.

KMGS Photography/Moment/Getty Images