Mixed Design
This activity will ask you to create a design to test the impact of gender and time on perceived attractiveness. Within your design, you will investigate the combined effect of a between-subjects variable and a within-subjects variable on another variable.
Dr. Melanie Maggard
Dr. Natalie J. Ciarocco, Monmouth University
Dr. David B. Strohmetz, Monmouth University
Dr. Gary W. Lewandowski, Jr., Monmouth University
Scenario: Dating has changed drastically over the years, from chaperoned visits with family members to computerized matchmaking on dating Web sites. Speed dating, in particular, has exploded in popularity since its creation in 1998, when a rabbi started this event in a Los Angeles coffee shop as a way for busy adults to meet many potential partners in a short period of time. Though the effectiveness of speed dating for finding long-term mates is questionable, it does present us with an interesting opportunity in today’s fast-paced world.
We might wonder how exposure to so many potential partners, whether good or bad, might change how we perceive individuals as the night progresses. Similar to the “beer goggle effect,” where, the more alcohol we drink, the more likely we are to view people as attractive, is it possible that we become less picky or rate individuals as more attractive as the night goes on? Could fear of not finding a match at the end of the night make us judge those we meet as more attractive than we normally would?
These questions prompt us to develop a research study that examines the relationship between gender, time, and attractiveness. But first, you will need a framework to help you explore this topic. Research studies all start with a question, so here is your chance to ask one of your own.
Which of the following research questions would be best to ask given the goal of our study?
A. |
B. |
C. |
Now that you have a research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), you must decide which type of research design will best answer your research question. To narrow things down, consider the following:
Nonexperimental Design
Experimental Design
Does your research question require a nonexperimental design or an experimental design?
A. |
B. |
Having decided that your research question requires a comparison between groups, you must determine the best comparison to make. However, keep in mind that there are multiple comparisons to be made: between genders and times of rating.
Given the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), which of the following would be best to use for the gender comparison?
A. |
B. |
C. |
Given the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), which of the following would be best for the time of rating comparison?
A. |
B. |
C. |
Now that you have selected an experimental design that compares males’ and females’ ratings of perceived attractiveness at early, middle, and late points in the speed-dating event, you can identify your independent and dependent variables.
Independent Variable (IV)
Dependent Variable (DV)
Given the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), what are your independent variables?
A. |
B. |
C. |
Given the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), what is your dependent variable?
A. |
B. |
C. |
In this study, participants can only be in 1 of the 2 groups for gender; they are either male or female. Thus, gender is a between-subjects variable.
Between-subjects Design
However, time of rating is being measured as how the participants’ ratings change over time. This means each participant gives 3 ratings, early, midway, and late in the speed-dating event. Time of rating is a within-subjects variable.
Within-subjects Design
With the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), comparisons, and types of variables in mind, consider the following designs:
Repeated-measures Design
Factorial Design
Mixed Design
Would your research question require a repeated-measures, factorial, or mixed design?
A. |
B. |
C. |
You have developed an experiment with 2 independent variables (gender and time of rating) and 1 dependent variable (perceived attractiveness). Because we have 1 between-subjects variable (gender) and 1 within-subjects variable (time of rating), we have a mixed design.
Mixed Design
Based on your research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”) and the variables in this study, what is the anatomy of this mixed design?
A. |
B. |
C. |
Next, we need to operationally define our independent variables. The first independent variable (IV), gender, is simply the gender of the participant, with the 2 groups being males and females. We will operationally define the second independent variable (IV), time of rating, by determining exactly how we will manipulate it. As we do, we’ll want to be sure our study has a high level of experimental and mundane realism.
Experimental Realism
Mundane Realism
Which of the following study options has the highest level of experimental and mundane realism?
A. |
B. |
C. |
The task that is highest in experimental and mundane realism is participants rating the level of attractiveness of individuals they have been exposed to up to certain points early, midway, and late in the speed-dating event. Since we have 1 between-subjects variable (gender) and 1 within-subjects variable (time of rating), we will be measuring males at each of the 3 times and females at each of the 3 times.
Which of the following demonstrates the combinations of independent variables that are being used in this mixed design?
A. |
B. |
C. |
We have decided to compare individuals early, midway, and late in a speed-dating event. Since these events typically last about 2 hours, we will use this length for our study. Each participant will meet with a potential date for 5 minutes before rotating to the next person. At the times of the ratings, we will ask participants to rate the perceived attractiveness of the dates they have met thus far. However, before proceeding, we need to clarify exactly when the ratings will occur.
Which of the following options would be the most reasonable time for the ratings to occur?
A. |
B. |
C. |
The following table illustrates the 6 combinations in our 2 x 3 mixed design, where each person will be assigned to 1 of the gender conditions and all 3 of the time of rating conditions:
A: Males + Early
B: Males + Middle
C: Males + Late
D: Females + Early
E: Females + Middle
F: Females + Late
Summary of Our Factorial Study | ||||
Time of Rating (Within-subjects) | ||||
Early | Middle | Late | ||
Gender (Between-subjects) |
Males | A | B | С |
Females | D | E | F |
You have now established the key comparisons between the various conditions created by the 2 independent variables. Next, we need to specify the exact nature of our dependent variable, perceived attractiveness. First, consider the following:
Which type of measure is better for assessing perceived attractiveness?
A. |
B. |
We know we want to use a self-report measure to measure perceived attractiveness. Now it is time to determine which type of self-report measure to use. Keep in mind how many and what types of questions, reliability, validity, and sensitivity would be appropriate for our study.
Reliability
Validity
Sensitivity
Which of the following would be the best self-report measure of perceived attractiveness in this study?
A. |
B. |
C. |
Now that you have determined what you will manipulate and measure, you must formulate an experimental hypothesis. Because we have multiple independent variables, we will also need multiple hypotheses: main effect hypotheses and interaction effect hypotheses.
Main Effect Hypothesis
Interaction Effect Hypothesis
Given the nature of your study, which of the following is the best experimental hypothesis for the main effect of gender?
A. |
B. |
C. |
Given the nature of your study, which of the following is the best experimental hypothesis for the main effect of time of rating?
A. |
B. |
C. |
Now that we have our main effect hypotheses, we must create the interaction effect hypothesis.
Which of the following is the best experimental hypothesis for the interaction effect of gender and time of rating?
A. |
B. |
C. |
Before you can conduct your experiment, you need to determine exactly whom you want to study and where you can find this target sample.
Which of the following samples would be best for your experiment?
A. |
B. |
C. |
Now that you have determined how you will collect your data and your intended sample, you must submit your research procedure to the Institutional Review Board (IRB) for ethical approval. The IRB or ethics board will determine whether or not your study meets all ethical guidelines.
IRB
Each IRB has its own protocol which conforms to the national standard when a researcher submits an application for proposed research to be reviewed. In addition to the appropriate paperwork and other information submitted to the IRB, the board would consider the following description during their evaluation of your proposed experiment:
The purpose of this research is to determine whether there is an interaction effect of gender (males or females) and time of rating (early, middle, or late) on perceived attractiveness. To study this topic, participants who are registered for a local speed-dating event will be told upon arrival that a study is being conducted on the attractiveness of individuals who participate in speed-dating events. Participants will be asked to rate the attractiveness of the daters they have met thus far 15 minutes after the event starts (early), 1 hour after (middle), and at the end of the 2-hour event (late). Participants will be debriefed at the end of the event regarding the true nature of the study.
The IRB reviewed your submission and has 1 concern. Although the study appears to present less than minimal risk to participants, there is no mention of informed consent and voluntary participation.
You must now determine how to respond to the IRB, keeping in mind the ethics of respect for persons and autonomy.
Which of the following is the best response to the IRB’s concern?
A. |
B. |
C. |
Now that we have secured the IRB’s approval, we should determine what the entire study will look like. Below are the steps of the study; can you place them in the proper order?
Now that you have a sense of how to conduct this study, it is time to see what data from this study might look like.
If you were to run a full version of this study, you would want to have at least 30 participants in each between-subjects group (males and females). Because you have a within-subjects design, each participant will be exposed to all levels of the time of rating independent variable. Thus, we need 60 participants for our study.
This is an example of what your data set would look like. The top row shows the variable names; the other rows display the data for 10 participants.
In the “Gender” column, a 1 = Male, and a 2 = Female. The ratings of attractiveness recorded after 15 minutes into the speed-dating event are located under “Early,” after 1 hour under “Middle,” and after 2 hours under “Late.” Ratings range from 1 (very unattractive) to 10 (highly attractive).
Participant Number | Gender | Early | Middle | Late |
101 | 1 | 6 | 5 | 7 |
102 | 1 | 5 | 5 | 8 |
103 | 1 | 6 | 6 | 9 |
104 | 1 | 4 | 6 | 9 |
105 | 1 | 6 | 7 | 7 |
131 | 2 | 5 | 5 | 7 |
132 | 2 | 5 | 6 | 7 |
133 | 2 | 5 | 5 | 7 |
134 | 2 | 5 | 6 | 6 |
135 | 2 | 5 | 6 | 7 |
Now that you have collected your data, you must decide the best way to summarize your findings. The decisions you made about how to collect your data dictate the statistics you can use with your data now. First, you need to consider if your study is descriptive or inferential.
Descriptive
Inferential
Given the nature of your experiment, which of the following is the best statistical method?
A. |
B. |
The following is an example of output for another mixed design ANOVA where participants experienced only multiple conditions. This study was about how hours slept at night (6 hours, 8 hours, and 10 hours) and gender (male or female) influence self-reported happiness. In this study, we recorded happiness after each night of sleep for each participant; thus, hours slept is a within-subjects variable and gender is a between-subjects variable.
Click on the table below to learn more about each element of the output.
Tests of Within-Subjects Effects | |||||||
Measure: MEASURE_1 | |||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared | |
sleep | Sphericity Assumed | 41.344 | 2 | 20.672 | 96.090 | .000 | .624 |
Greenhouse-Geisser | 41.344 | 1.855 | 22.284 | 96.090 | .000 | .624 | |
Huynh-Feldt | 41.344 | 1.947 | 21.233 | 96.090 | .000 | .624 | |
Lower-bound | 41.344 | 1.000 | 41.344 | 96.090 | .000 | .624 | |
sleep * Gender | Sphericity Assumed | 1.033 | 2 | .517 | 2.402 | .095 | .040 |
Greenhouse-Geisser | 1.033 | 1.855 | .557 | 2.402 | .099 | .040 | |
Huynh-Feldt | 1.033 | 1.947 | .531 | 2.402 | .097 | .040 | |
Lower-bound | 1.033 | 1.000 | 1.033 | 2.402 | .127 | .040 | |
Error(sleep) | Sphericity Assumed | 24.956 | 116 | .215 | . | ||
Greenhouse-Geisser | 24.956 | 107.611 | .232 | ||||
Huynh-Feldt | 24.956 | 112.934 | .221 | ||||
Lower-bound | 24.956 | 58.000 | .430 |
Tests of Between-Subjects Effects | ||||||
Measure: MEASURE_1 | ||||||
Transformed Variable: Average | ||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared |
Intercept | 2006.672 | 1 | 2006.672 | 6143.595 | .000 | .991 |
Gender | 48.050 | 1 | 48.050 | 147.109 | .000 | .717 |
Error | 18.944 | 58 | .327 |
To report these numbers in a results section, put the numbers in as follows:
Click on the table below to learn more about each element of the output.
F (#,#) = #.##, p = .##, eta2 = .##.
This should be done for each main effect and the interaction.
Pairwise Comparisons | ||||||
Measure: MEASURE_1 | ||||||
(I) sleep | (J) sleep | Mean Difference (I-J) | Std. Error | Sig.b | 95% Confidence Interval for Differenceb | |
Lower Bound | Upper Bound | |||||
1 | 2 | -1.017* | .076 | .000 | -1.204 | -.829 |
3 | .000 | .095 | 1.000 | -.235 | .235 | |
2 | 1 | 1.017* | .076 | .000 | .829 | 1.204 |
3 | 1.017* | .081 | .000 | .816 | 1.217 | |
3 | 1 | .000 | .095 | 1.000 | -.235 | .235 |
2 | -1.017* | .081 | .000 | -1.217 | -.816 | |
Based on estimated marginal means | ||||||
*. The mean difference is significant at the .05 level. | ||||||
b. Adjustment for multiple comparisons: Bonferroni. |
Click on the table below to learn more about each element of the output.
Estimates | ||||
Measure: MEASURE_1 | ||||
Gender | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | |||
Male | 2.822 | .060 | 2.702 | 2.943 |
Female | 3.856 | .060 | 3.735 | 3.976 |
Descriptive Statistics | ||||
Measure: MEASURE_1 | ||||
Source | Gender | Mean | Std. Deviation | N |
Six | Male | 2.50 | .509 | 30 |
Female | 3.50 | .509 | 30 | |
Total | 3.00 | .713 | 60 | |
Eight | Male | 3.40 | .498 | 30 |
Female | 4.63 | .490 | 30 | |
Total | 4.02 | .792 | 60 | |
Ten | Male | 2.57 | .504 | 30 |
Female | 3.43 | .504 | 30 | |
Total | 3.00 | .664 | 60 |
Happiness
Hours of Sleep
Below is the output from your study:
Tests of Within-Subjects Effects | |||||||
Measure: MEASURE_1 | |||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared | |
time | Sphericity Assumed | 196.078 | 2 | 98.039 | 199.130 | .000 | .774 |
Greenhouse-Geisser | 196.078 | 2.000 | 98.042 | 199.130 | .000 | .774 | |
Huynh-Feldt | 196.078 | 2.000 | 98.039 | 199.130 | .000 | .774 | |
Lower-bound | 196.078 | 1.000 | 196.078 | 199.130 | .000 | .744 | |
time * Gender | Sphericity Assumed | 8.811 | 2 | 4.406 | 8.948 | .000 | .134 |
Greenhouse-Geisser | 8.811 | 2.000 | 4.406 | 8.948 | .000 | .134 | |
Huynh-Feldt | 8.811 | 2.000 | 4.406 | 8.948 | .000 | .134 | |
Lower-bound | 8.811 | 1.000 | 8.811 | 8.948 | .004 | .134 | |
Error(time) | Sphericity Assumed | 57.111 | 116 | .492 | |||
Greenhouse-Geisser | 57.111 | 115.997 | .492 | ||||
Huynh-Feldt | 57.111 | 116.000 | .492 | ||||
Lower-bound | 57.111 | 58.000 | .985 |
Below is the output from your study:
Tests of Between-Subjects Effects | |||||||
Measure: MEASURE_1 | |||||||
Transformed Variable: Average | |||||||
Source | Type III Sum of Squares | df | Mean Square | F | Sig. | Partial Eta Squared | |
Intercept | 6528.089 | 1 | 6528.089 | 16936.692 | .000 | .997 | |
Gender | 35.556 | 1 | 35.556 | 92.247 | .000 | .614 | |
Error | 22.356 | 58 | .385 |
Below is the output from your study:
Pairwise Comparisons | ||||||
Measure: MEASURE_1 | ||||||
(I) time | (J) time | Mean Difference (I-J) | Std. Error | Sig.b | 95% Confidence Interval for Differenceb | |
Lower Bound | Upper Bound | |||||
1 | 2 | -1.117* | .128 | .000 | -1.433 | -.800 |
3 | -2.550* | .128 | .000 | -2.865 | -2.235 | |
2 | 1 | 1.117* | .128 | .000 | .800 | 1.433 |
3 | -1.433* | .128 | .000 | -1.749 | -1.117 | |
3 | 1 | 2.550* | .128 | .000 | 2.235 | 2.865 |
2 | 1.433* | .128 | .000 | 1.117 | 1.749 | |
Based on estimated marginal means | ||||||
*. The mean difference is significant at the .05 level. | ||||||
b. Adjustment for multiple comparisons: Bonferroni. |
Below is the output from your study:
Descriptive Statistics | ||||
Measure: MEASURE_1 | ||||
Source | Gender | Mean | Std. Deviation | N |
Early | Male | 5.03 | .850 | 30 |
Female | 4.57 | .504 | 30 | |
Total | 4.80 | .732 | 60 | |
Middle | Male | 6.27 | .691 | 30 |
Female | 5.57 | .504 | 30 | |
Total | 5.92 | .696 | 60 | |
Late | Male | 8.10 | .885 | 30 |
Female | 6.60 | .498 | 30 | |
Total | 7.35 | 1.039 | 60 |
Estimates | ||||
Measure: MEASURE_1 | ||||
Gender | Mean | Std. Error | 95% Confidence Interval | |
Lower Bound | Upper Bound | |||
Male | 6.467 | .065 | 6.336 | 6.598 |
Female | 5.578 | .065 | 5.447 | 5.709 |
Based on the results of your statistical analyses, match the correct number in the “Answer” column to the term requested under “Prompt”:
Please move the correct answer to the left.
Move the bars to represent the data in the table.
Descriptive Statistics | ||||
Measure: MEASURE_1 | ||||
Source | Gender | Mean | Std. Deviation | N |
Early | Male | 5.03 | .850 | 30 |
Female | 4.57 | .504 | 30 | |
Total | 4.80 | .732 | 60 | |
Middle | Male | 6.27 | .691 | 30 |
Female | 5.57 | .504 | 30 | |
Total | 5.92 | .696 | 60 | |
Late | Male | 8.10 | .885 | 30 |
Female | 6.60 | .498 | 30 | |
Total | 7.35 | 1.039 | 60 |
Gender, Time of Rating, and Attractiveness
Mean Attractiveness Raiting
Time of Rating
Now that you have worked with your data, you must determine the best way to express your findings in written form. You must be sure that how you describe your findings accurately represents the data.
Based on the statistical analysis, which of the following results sections best fits the data and analyses from your study?
A. |
B. |
C. |
Now that you have determined how to express your findings in a scientifically responsible way, you also need to be able to talk about what your findings mean in everyday terms so that the world can benefit from your science.
How would you explain what you found about perceived attractiveness over the course of a speed-dating event based on gender and time of rating to a friend? Select the best option.
A. |
B. |
C. |
D. |
Congratulations! You have successfully completed this activity.