2.1 The Challenge of Research Methods
When it comes to research, one critical thinking skill is paramount: detecting flaws. Few research plans are perfect. Many contain defects that compromise their value. Identifying and eliminating flaws in research is a critical skill for the psychologist—
Learning About People: A Flawed Strategy
Preview Question
Question
What are some flaws that can reduce the value of a study in psychology?
Here is a research plan with flaws. Spot them.
A FLAWED STRATEGY. A student who recently has moved into a college dorm wonders whether the other dorm residents like him. Being scientifically minded, he decides to conduct research and executes the following method.
37
The student walks down the hall, knocks on the first door he sees, and asks, “Do you like me, or what?” The person answers, “Uh, yeah, sure.”
He continues to the next door. Nobody answers. He tries a few more, finds someone who is in, and asks, “Hey, do you like me?”
“Not really” is the reply.
He finds three more people:
“Do you like me?” / “Well, I don’t know, I guess.”
“You like me, don’t you?” / “I don’t, no.”
“You like me, right?” / “Hey, come in, we’re having a party! Sure, we like you!”
Confident in his plan, the student continues, eventually getting 20 responses. He then returns to his room, tries to remember what everybody said, and writes a summary of his results.
THE FLAWS. As a plan for research, the student’s efforts were far from perfect. But what, precisely, were the problems? Here are seven:
When was the last time you engaged in some dishonesty to spare someone’s feelings?
People might not have answered honestly. Residents who didn’t like the student may not have wanted to admit that to his face, and thus may not have answered honestly.
The research procedure might have altered people’s feelings. Some residents may have liked the student before he did something as ridiculous as knocking on the door and asking, “Do you like me, or what?” Because the student was trying to find out if people liked him, not alter people’s feelings about him, this, too, is a research flaw.
People might have answered differently at a different time. For example, the fifth person was enthusiastic, but maybe that was because a party was going on when the student knocked and asked his question.
Other people’s opinions might have differed. The residents who weren’t in their rooms might have had different opinions than those who were at home, but the student never talked to them.
The varied phrasing of the question may have affected the answers. The student phrased his question differently from one person to the next, and different phrasings may have prompted different responses.
Some responses were difficult to interpret. What, for example, was the real opinion of the person who said, “Well, I don’t know, I guess”? The difficulty of interpreting the responses adds ambiguity to the research results.
The summary of people’s responses may have been inaccurate. When the student sat down to summarize what people said, he may have forgotten some of their statements. His summary thus may not have been accurate.
That’s a lot of flaws. And they add up, with each contributing to one big overall problem: The research may have produced a picture of the world that is distorted. Like a camera with a damaged lens, research with flaws produces a “picture”—but one that portrays the world inaccurately. The student might conclude that most residents like him even if they don’t, or that most don’t like him even if they really do.
THINK ABOUT IT
Fix the flaws in the dorm study. Figure out ways to overcome the problems in its research methods.
38
WHAT DO YOU KNOW?…
Question 1
rIXMayO8McwjZe08pIhCL5b4TCMb3IpEqd/96Vu3oUoycF8E6/brujqN1GJlSd5CrjRk41kX6swhYLnUsjy3FxYRm2ax54cLnVNFDX94yz/tQ3j5p+lIBoKDIhUM5Q+BFX50YoqbLKoJF4DUMpGdTx/ojI5E2yy6xS0lD+OmzBgFQNpeq5tcZg==Fixing the Flaws
Preview Question
Question
Who is capable of thinking critically about problems in psychological research?
Even now, at the start of this chapter, you can figure out how to fix many, if not all, of these seven research flaws. Consider a couple.
How to fix #5: “The varied phrasing of the question may have affected the answers.” It’s easy. Have the student write down and memorize one phrasing of the question and ask it in exactly the same way to all people.
How to fix #6: “Some responses were difficult to interpret.” You could ask people to pick from among various multiple-
choice answers. Their response could then be interpreted unambiguously.
Among the other flaws, some are easy to fix and others hard. But either way, you can imagine ways to fix them. You have the ability to think critically about problems in psychological research. You can detect and repair research flaws by applying critical thinking skills you use in everyday life to questions about research methods. And the research flaws you were just thinking about were realistic, even though our example was made up. The dorm study’s flaws exemplify the sorts of problems psychologists must overcome when conducting research.
Let’s now turn from our hypothetical example to the real thing: the research strategies of psychological science. Psychologists reduce research flaws by employing carefully planned research designs.