Two-Group Design
This activity will ask you to make decisions like a researcher as you design a two-group experiment to address whether or not multitasking is beneficial.
Dr. Natalie J. Ciarocco, Monmouth University
Dr. David B. Strohmetz, Monmouth University
Dr. Gary W. Lewandowski, Jr., Monmouth University
Scenario: As a busy college student, you have a lot to do. You have multiple classes that each come with a combination of assignments, papers, and tests. (And inevitably, several of these assignments will be due on the same day.) You probably also have a job, family obligations, and clubs or organizations that you’re involved in, and you need to find time for exercise, sleep, and—if you’re lucky—a social life. To fit all your responsibilities into the limited amount of hours you have in a day, you might consider multitasking in order to accomplish several things at once.
We often hold our multitasking abilities in high regard, and might even boast about all of the things we can do at once. Can you run on the treadmill while reading a book for class, listening to your iPod, checking Facebook, and posting a selfie to Instagram? Ha! That’s a slow day. Of course, the truth is that you aren’t really focusing on all of these activities at once. Rather, you are quickly switching your attention from task to task—what psychologists call time-sharing—in order to get everything done. Yet you have to wonder how many of these individual tasks you are doing well, and whether switching your attention between them really saves you time. In fact, it’s possible that multitasking causes all of these activities to take longer.
Based on your experience with multitasking, you can develop a research study that examines its effectiveness. However, you will need something to help you make decisions about how to set up your exploration of this topic. Research studies all start with a question, so here is your chance to ask yours.
Which of the research questions below would be the best to ask given the goal of your study?
A. |
B. |
C. |
Now that you have developed a research question (“Does multitasking lead to better task performance for college students?”), it is time to decide what type of research design will best answer your question. To narrow things down, consider the following:
Nonexperimental Design
Experimental Design
Does your research question require a nonexperimental design or an experimental design?
A. |
B. |
Now that you know your research question requires a comparison between groups, it is time to determine how best to compare them.
Given the research question (“Does multitasking lead to better task performance for college students?”), which comparison would be best to use?
A. |
B. |
C. |
Now that you have an experimental design that compares multitasking vs. non-multitasking groups, you can identify your independent and dependent variables.
Independent Variable (IV)
Dependent Variable (DV)
Given the research question, what is your independent variable?
A. |
B. |
C. |
Given the research question, what is your dependent variable?
A. |
B. |
C. |
Because you have an experiment with one independent variable with two levels (multitasking vs. completing one task at a time), you have a two-group design or simple experiment. Next we should identify the experimental and control groups.
Experimental Group
Control Group
Based on your research question, which group is the experimental group?
A. |
B. |
Next, we need to operationally define the independent variable (IV) of multitasking by determining exactly how we will manipulate it. As we define multitasking, we’ll want to be sure our study has a high level of experimental realism.
Experimental Realism
Which of the following options presents the highest level of experimental realism?
A. |
B. |
C. |
Now that we have confirmed experimental realism, we will want to be sure our study has a high level of mundane realism.
Mundane Realism
Which of the following options presents the highest level of mundane realism?
A. |
B. |
C. |
It looks like the task that is highest in experimental and mundane realism involves students in the experimental group switching between a vocabulary list, an IM conversation, and proofreading. We know from before that our control group will not multitask. Our next decision involves picking the best possible control group, which will be the one with the most experimental control.
Experimental Control
Considering your chosen tasks, which of the following groups has the highest level of experimental control?
A. |
B. |
C. |
You now have the key comparison set up between multitasking and a “one at a time” approach. Next, we need to specify the exact nature of our dependent variable of “task performance.” To eliminate some options, first consider the following:
Is it better to measure the dependent variable based on what people think or what people do?
A. |
B. |
We know we want a behavioral measure that measures task performance. You will have to determine which type of behavioral measure to use from the options below:
Behavioral Choice
Behavioral Trace
Behavioral Observation
Which of the following behaviors would be best to measure? Choose as many behaviors as is appropriate.
A. Behavioral observation of the time necessary to complete all of the tasks. | |
B. Behavioral trace of smiling during the task. | |
C. Behavioral observation of their performance on a vocabulary quiz. | |
D. Behavioral choice of which task participants choose to do the most. |
Now that you have determined what you will manipulate and measure, it is time to think about how the study might turn out and formulate an experimental hypothesis.
Experimental Hypothesis
Which of the following hypotheses is best, given the nature of your study?
A. |
B. |
C. |
Before you can conduct your experiment, you need to determine exactly who you want to study and where you can find your target sample.
Which of the following samples would be best to use in your experiment?
A. |
B. |
C. |
Now that you have determined how you will collect your data and chosen your intended sample, you must submit your research procedure to the Institutional Review Board (IRB) for ethical approval. The IRB or ethics board will determine whether or not your study meets all ethical guidelines.
IRB
Each IRB has its own protocol which conforms to the national standard when a researcher submits an application for proposed research to be reviewed. In addition to the appropriate paperwork and other information submitted to the IRB, the board would consider the following description during their evaluation of your proposed experiment:
The purpose of this research is to determine if multitasking leads to better task performance than completing tasks sequentially does. To study this topic, participants will perform three tasks: study 10 vocabulary words, chat using an instant messaging program, and proofread an English paper. Researchers will randomly assign participants to switch back and forth between these tasks (multitasking group) or complete tasks “one at a time” (sequential group). Researchers will then measure how long it took participants to complete all of the tasks, as well as participants’ performance on a 10-question vocabulary quiz.
The IRB reviewed your submission and has one concern. Specifically, they believe that having students perform three tasks is asking participants to do too much, presenting a high degree of cost, or effort, for students.
You must now determine how to respond to the IRB in order to address their concern and uphold the quality of your study. Your options include changing your design or justifying to the IRB why your current methodology is ethically sound.
Which of the following is the best response to the IRB’s concern?
A. |
B. |
C. |
Now that you have secured the IRB’s approval, we should confirm what the entire study will look like. Below are the steps of the study; can you place them in the proper order?
Now that you have a sense of how to conduct this study, it is time to see what data from this study might look like.
If you were to run a full version of this study, you would want to have at least 30 participants in each of your two groups, for a total of 60 participants. Because you have a between-subjects design, each participant will only be in one group.
This is an example of what your data set would look like. The top row shows the variable names; the other rows display the data for the first 10 participants.
In the “Group” column, 1 = Multitasking Group and 2 = Sequential Group. Vocab test scores were graded from a low score of 1 to a high score of 10, and task time was recorded in seconds.
Participant Number | Group | VocabTestScore | TaskTime |
101 | 1 | 4 | 624 |
102 | 1 | 6 | 671 |
103 | 2 | 10 | 575 |
104 | 1 | 3 | 648 |
105 | 2 | 5 | 550 |
106 | 1 | 3 | 712 |
107 | 2 | 5 | 556 |
108 | 1 | 7 | 465 |
109 | 2 | 2 | 556 |
110 | 2 | 10 | 485 |
Now that you have collected your data, you must decide the best way to summarize your findings. The decisions you made about how to collect your data dictate the statistics you can use with your data now. First, you need to consider if your study is descriptive or inferential.
Descriptive
Inferential
Given the nature of your experiment, which of the following is the better statistical method?
A. |
B. |
The following is an example of output for another two-group design. This study was about how hours slept at night (less than 7 hours vs. more than 7 hours) influence self-reported happiness.
To report these numbers in a results section, put the numbers in as follows:
Click on the table below to learn more about each element of the output.
Levene's Test for Equality of Variances | t-test for Equality of Means | ||||||||||
F | Sig. | t | df | Sig. (2-tailed) | Mean Difference | Std. Error Difference | 95% Confidence Interval of the Difference | ||||
Lower | Upper | ||||||||||
happiness |
Equal variances assumed |
.056 |
.814 |
2.076 |
82 |
.041 |
.36470 |
.17566 |
.01526 |
-.71413 |
|
Equal variances not assumed |
2.076 | 81.974 | .041 | .364708 | .17566 | .01526 |
.71414 |
Condition | N | Mean | Std. Deviation | Std. Error Mean | |
happiness | More than 7 hours sleep |
42 |
4.0867 | .81209 | .12531 |
Less than 7 hours sleep | 42 | 3.7220 | .79776 | .12310 |
Below is the output from your study:
condition | N | Mean | Std. Deviation | Std. Error Mean | |
VocabTestScore |
Multi-tasking Group Sequential Group |
30 30 |
4.83 6.11 |
2.156 2.391 |
.394 .437 |
TaskTime |
Multi-tasking Group Sequential Group |
30 30 |
660.91 571.84 |
102.278 112.070 |
18.673 20.461 |
Levene's Test for Equality of Variances | t-test for Equality of Means | ||||||||||
F | Sig. | t | df | Sig. (2-tailed) | Mean Difference | Std. Error Difference | 95% Confidence Interval of the Difference | ||||
Lower | Upper | ||||||||||
VocabtestScore |
Equal variances assumed |
.828 |
.367 |
-2.173 |
58 |
0.34 |
-1.278 |
.588 |
-2.455 |
-.101 |
|
Equal variances not assumed |
-2.173 | 57.389 | 0.34 | -1.278 | .588 | -2.455 |
-.101 |
||||
TaskTime |
Equal variances assumed |
.016 | .901 |
3.215 |
58 |
.002 |
89.071 |
27.701 |
33.621 |
144.520 |
|
Equal variances not assumed |
3.215 |
57.522 | .002 | 89.071 | 27.701 | 33.611 |
144.530 |
Based on the results of your statistical analyses, match the correct number in the “Answer” column to the term requested under “Prompt”:
Prompt | Answer |
t for the vocabulary test DV | |
df for the vocabulary test DV | |
p for the vocabulary test DV | |
t for the task time DV | |
df for the task time DV | |
p for the task time DV |
condition | N | Mean | Std. Deviation | Std. Error Mean | |
VocabTestScore |
Multi-tasking Group Sequential Group |
30 30 |
4.83 6.11 |
2.156 2.391 |
.394 .437 |
TaskTime |
Multi-tasking Group Sequential Group |
30 30 |
660.91 571.84 |
102.278 112.070 |
18.673 20.461 |
Drag the bars of each graph to the correct Mean value.
Mean
Now that you have worked with your data, you must determine the best way to express your findings in written form. You must be sure that how you describe your findings accurately represents the data.
Based on the statistical analysis, which of the following results sections best fits the data and analyses from your study?
A. |
B. |
C. |
D. |
Now that you have determined how to express your findings in a scientifically responsible way, you need to be able to talk about what your findings mean in everyday terms so that the world can benefit from your science.
How would you explain what you found about multitasking to a friend or family member? Select the best option.
A. |
B. |
C. |
D. |
Congratulations! You have successfully completed this activity.