Chapter
1. Do Speed Daters Become Pickier the Later It Gets?
Introduction
Chapter 12
true
true
You must read each slide, and complete any questions on the slide, in sequence.
Nonexperimental Design
A design in which there is no control or manipulation of the variables. This design does not seek to establish cause and effect and instead focuses on describing or summarizing what takes place.
Experimental Design
A design in which the experimenter controls and manipulates the independent variable and makes comparisons between the different levels, allowing the establishment of cause-and-effect relationships between the independent and dependent variables.
Independent Variable (IV)
The variable that influences the dependent variable. In experiments the researcher manipulates or controls this variable.
Dependent Variable (DV)
The variable measured in association with changes in the independent variable; the outcome or effect.
Between-subjects Design
A data collection method in which each participant or subject is only assessed on the dependent variable once.
Within-subjects Design
A data collection method in which each participant or subject is assessed on the dependent variable more than once.
Repeated-measures Design
A design where participants are exposed to each level of the independent variable and are measured on the dependent variable after each level.
Factorial Design
An experimental design that has more than one independent variable, which are all between-subjects.
Mixed Design
An experimental design that combines within-subjects and between-subjects methods of data collection.
Experimental Realism
The degree to which a study participant becomes engrossed in the manipulation and truly influenced by it.
Mundane Realism
The degree to which a study parallels everyday situations in the real world.
Reliability
The stability or consistency of a measure.
Validity
The degree to which a tool measures what it claims to measure.
Sensitivity
The range of data a researcher can gather from a particular instrument.
Main Effect Hypothesis
A prediction that focuses on one independent variable at a time, ignoring all other independent variables.
Interaction Effect Hypothesis
A prediction about how the levels of one independent variable will combine with another independent variable to impact the dependent variable in a way that extends beyond the sum of the two separate main effects.
IRB
A board that reviews the ethical merit of all the human research conducted within an institution.
Descriptive
Describes what is happening.
Inferential
Tests a specific prediction about why something occurs.
Mixed Design
This activity will ask you to create a design to test the impact of gender and time on perceived attractiveness. Within your design, you will investigate the combined effect of a between-subjects variable and a within-subjects variable on another variable.
Dr. Melanie Maggard
Dr. Natalie J. Ciarocco, Monmouth University
Dr. David B. Strohmetz, Monmouth University
Dr. Gary W. Lewandowski, Jr., Monmouth University
Something to Think About…
Scenario: Dating has changed drastically over the years, from chaperoned visits with family members to computerized matchmaking on dating Web sites. Speed dating, in particular, has exploded in popularity since its creation in 1998, when a rabbi started this event in a Los Angeles coffee shop as a way for busy adults to meet many potential partners in a short period of time. Though the effectiveness of speed dating for finding long-term mates is questionable, it does present us with an interesting opportunity in today’s fast-paced world.
We might wonder how exposure to so many potential partners, whether good or bad, might change how we perceive individuals as the night progresses. Similar to the “beer goggle effect,” where, the more alcohol we drink, the more likely we are to view people as attractive, is it possible that we become less picky or rate individuals as more attractive as the night goes on? Could fear of not finding a match at the end of the night make us judge those we meet as more attractive than we normally would?
Our Research Question
Jesse Kunerth/iStock/Getty Images
These questions prompt us to develop a research study that examines the relationship between gender, time, and attractiveness. But first, you will need a framework to help you explore this topic. Research studies all start with a question, so here is your chance to ask one of your own.
Now that you have a research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), you must decide which type of research design will best answer your research question. To narrow things down, consider the following:
Having decided that your research question requires a comparison between groups, you must determine the best comparison to make. However, keep in mind that there are multiple comparisons to be made: between genders and times of rating.
Now that you have selected an experimental design that compares males’ and females’ ratings of perceived attractiveness at early, middle, and late points in the speed-dating event, you can identify your independent and dependent variables.
In this study, participants can only be in 1 of the 2 groups for gender; they are either male or female. Thus, gender is a between-subjects variable.
Between-subjects Design
However, time of rating is being measured as how the participants’ ratings change over time. This means each participant gives 3 ratings, early, midway, and late in the speed-dating event. Time of rating is a within-subjects variable.
Within-subjects Design
Picking the Best Design
Lev Mei/iStock/Getty Images
With the research question (“How do gender and time of rating during a speed-dating event influence perceived attractiveness?”), comparisons, and types of variables in mind, consider the following designs:
You have developed an experiment with 2 independent variables (gender and time of rating) and 1 dependent variable (perceived attractiveness). Because we have 1 between-subjects variable (gender) and 1 within-subjects variable (time of rating), we have a mixed design.
Next, we need to operationally define our independent variables. The first independent variable (IV), gender, is simply the gender of the participant, with the 2 groups being males and females. We will operationally define the second independent variable (IV), time of rating, by determining exactly how we will manipulate it. As we do, we’ll want to be sure our study has a high level of experimental and mundane realism.
The task that is highest in experimental and mundane realism is participants rating the level of attractiveness of individuals they have been exposed to up to certain points early, midway, and late in the speed-dating event. Since we have 1 between-subjects variable (gender) and 1 within-subjects variable (time of rating), we will be measuring males at each of the 3 times and females at each of the 3 times.
We have decided to compare individuals early, midway, and late in a speed-dating event. Since these events typically last about 2 hours, we will use this length for our study. Each participant will meet with a potential date for 5 minutes before rotating to the next person. At the times of the ratings, we will ask participants to rate the perceived attractiveness of the dates they have met thus far. However, before proceeding, we need to clarify exactly when the ratings will occur.
The following table illustrates the 6 combinations in our 2 x 3 mixed design, where each person will be assigned to 1 of the gender conditions and all 3 of the time of rating conditions:
A: Males + Early
B: Males + Middle
C: Males + Late
D: Females + Early
E: Females + Middle
F: Females + Late
Summary of Our Factorial Study
Time of Rating (Within-subjects)
Early
Middle
Late
Gender (Between-subjects)
Males
A
B
С
Females
D
E
F
Operationally Defining the Dependent Variable
You have now established the key comparisons between the various conditions created by the 2 independent variables. Next, we need to specify the exact nature of our dependent variable, perceived attractiveness. First, consider the following:
We know we want to use a self-report measure to measure perceived attractiveness. Now it is time to determine which type of self-report measure to use. Keep in mind how many and what types of questions, reliability, validity, and sensitivity would be appropriate for our study.
Now that you have determined what you will manipulate and measure, you must formulate an experimental hypothesis. Because we have multiple independent variables, we will also need multiple hypotheses: main effect hypotheses and interaction effect hypotheses.
Now that you have determined how you will collect your data and your intended sample, you must submit your research procedure to the Institutional Review Board (IRB) for ethical approval. The IRB or ethics board will determine whether or not your study meets all ethical guidelines.
IRB
Each IRB has its own protocol which conforms to the national standard when a researcher submits an application for proposed research to be reviewed. In addition to the appropriate paperwork and other information submitted to the IRB, the board would consider the following description during their evaluation of your proposed experiment:
The purpose of this research is to determine whether there is an interaction effect of gender (males or females) and time of rating (early, middle, or late) on perceived attractiveness. To study this topic, participants who are registered for a local speed-dating event will be told upon arrival that a study is being conducted on the attractiveness of individuals who participate in speed-dating events. Participants will be asked to rate the attractiveness of the daters they have met thus far 15 minutes after the event starts (early), 1 hour after (middle), and at the end of the 2-hour event (late). Participants will be debriefed at the end of the event regarding the true nature of the study.
Responding to the IRB
wk1003mike/Shutterstock
The IRB reviewed your submission and has 1 concern. Although the study appears to present less than minimal risk to participants, there is no mention of informed consent and voluntary participation.
You must now determine how to respond to the IRB, keeping in mind the ethics of respect for persons and autonomy.
Now that we have secured the IRB’s approval, we should determine what the entire study will look like. Below are the steps of the study; can you place them in the proper order?
A.
B.
C.
D.
E.
F.
Participants rate the attractiveness of others 15 minutes after the event starts.
Obtain informed consent.
Participants rate the attractiveness of others 2 hours after the event starts
Participants’ genders are recorded, and the speed-dating event starts.
Participants rate the attractiveness of others 1 hour after the event starts.
Debrief the participants.
Collecting Data
Image Source/Photodisc/Getty Images
Now that you have a sense of how to conduct this study, it is time to see what data from this study might look like.
If you were to run a full version of this study, you would want to have at least 30 participants in each between-subjects group (males and females). Because you have a within-subjects design, each participant will be exposed to all levels of the time of rating independent variable. Thus, we need 60 participants for our study.
Example Data Set
This is an example of what your data set would look like. The top row shows the variable names; the other rows display the data for 10 participants.
In the “Gender” column, a 1 = Male, and a 2 = Female. The ratings of attractiveness recorded after 15 minutes into the speed-dating event are located under “Early,” after 1 hour under “Middle,” and after 2 hours under “Late.” Ratings range from 1 (very unattractive) to 10 (highly attractive).
Participant Number
Gender
Early
Middle
Late
101
1
6
5
7
102
1
5
5
8
103
1
6
6
9
104
1
4
6
9
105
1
6
7
7
131
2
5
5
7
132
2
5
6
7
133
2
5
5
7
134
2
5
6
6
135
2
5
6
7
Selecting the Proper Tool
Ines Koleva/E+/Getty Images
Now that you have collected your data, you must decide the best way to summarize your findings. The decisions you made about how to collect your data dictate the statistics you can use with your data now. First, you need to consider if your study is descriptive or inferential.
The following is an example of output for another mixed design ANOVA where participants experienced only multiple conditions. This study was about how hours slept at night (6 hours, 8 hours, and 10 hours) and gender (male or female) influence self-reported happiness. In this study, we recorded happiness after each night of sleep for each participant; thus, hours slept is a within-subjects variable and gender is a between-subjects variable.
Click on the table below to learn more about each element of the output.
Tests of Within-Subjects Effects
Measure: MEASURE_1
Source
Type III Sum of Squares
df
Mean Square
F
Sig.
Partial Eta Squared
sleep
Sphericity Assumed
41.344
2
20.672
96.090
.000
.624
Greenhouse-Geisser
41.344
1.855
22.284
96.090
.000
.624
Huynh-Feldt
41.344
1.947
21.233
96.090
.000
.624
Lower-bound
41.344
1.000
41.344
96.090
.000
.624
sleep * Gender
Sphericity Assumed
1.033
2
.517
2.402
.095
.040
Greenhouse-Geisser
1.033
1.855
.557
2.402
.099
.040
Huynh-Feldt
1.033
1.947
.531
2.402
.097
.040
Lower-bound
1.033
1.000
1.033
2.402
.127
.040
Error(sleep)
Sphericity Assumed
24.956
116
.215
.
Greenhouse-Geisser
24.956
107.611
.232
Huynh-Feldt
24.956
112.934
.221
Lower-bound
24.956
58.000
.430
Tests of Between-Subjects Effects
Measure: MEASURE_1
Transformed Variable: Average
Source
Type III Sum of Squares
df
Mean Square
F
Sig.
Partial Eta Squared
Intercept
2006.672
1
2006.672
6143.595
.000
.991
Gender
48.050
1
48.050
147.109
.000
.717
Error
18.944
58
.327
This is the df or degrees of freedom. An ANOVA has 3 dfs, 1 for the within-subjects main effect, 1 for the between-subjects main effect, and 1 for the interaction main effect.
These are the F F statistics, 1 for each main effect and 1 for the interaction. It represents the size of the difference between condition means compared to the size of the residual error.
This is the p level, or the significance level. It represents the probability or likelihood that the results happened by chance. The lower the p level, the less likely the results happened by chance.
The F and p level will only tell you whether there is a significant difference. To determine which means are different, and the nature or direction of those differences, you need to look at the means via a post-hoc test when you are comparing more than 2 means.
Since there are only 2 levels for gender, we do not need to conduct a post-hoc test for this effect.
The eta squared (eta2 ) is the effect size. It tells us the proportion of change in the dependent variable that is associated with being in the different groups of the independent variable or the interaction of the independent variables.
Tutorial: Evaluating Output
chapter_12_table_activity_1
Alex Brylov/Shutterstock
To report these numbers in a results section, put the numbers in as follows:
Click on the table below to learn more about each element of the output.
F (#,#) = #.##, p = .##, eta2 = .##.
This should be done for each main effect and the interaction.
Pairwise Comparisons
Measure: MEASURE_1
(I) sleep
(J) sleep
Mean Difference (I-J)
Std. Error
Sig.b
95% Confidence Interval for Differenceb
Lower Bound
Upper Bound
1
2
-1.017*
.076
.000
-1.204
-.829
3
.000
.095
1.000
-.235
.235
2
1
1.017*
.076
.000
.829
1.204
3
1.017*
.081
.000
.816
1.217
3
1
.000
.095
1.000
-.235
.235
2
-1.017*
.081
.000
-1.217
-.816
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Bonferroni.
The results presented here are from the post-hoc test, which compares each of the groups’ means to all of the other groups’ means.
This is the difference between the mean happiness rating for the 6 hours of sleep and 8 hours of sleep conditions.
This is the difference between the mean happiness rating for the 6 hours of sleep and 10 hours of sleep conditions.
This is the difference between the mean happiness rating for the 8 hours of sleep and 10 hours of sleep conditions.
Happiness was different in the 8 hours sleep condition than in the 6 hours and 10 hours sleep conditions, which had similar ratings.
The post-hoc test tells us which comparisons between the means were significant. The p level tells us the significance level of that comparison.
Tutorial: Evaluating Output
chapter_12_table_graph
Alex Brylov/Shutterstock
Click on the table below to learn more about each element of the output.
Estimates
Measure: MEASURE_1
Gender
Mean
Std. Error
95% Confidence Interval
Lower Bound
Upper Bound
Male
2.822
.060
2.702
2.943
Female
3.856
.060
3.735
3.976
Descriptive Statistics
Measure: MEASURE_1
Source
Gender
Mean
Std. Deviation
N
Six
Male
2.50
.509
30
Female
3.50
.509
30
Total
3.00
.713
60
Eight
Male
3.40
.498
30
Female
4.63
.490
30
Total
4.02
.792
60
Ten
Male
2.57
.504
30
Female
3.43
.504
30
Total
3.00
.664
60
This is the average or mean (M) happiness rating for males.
This is the standard error (SE) of happiness rating for males.
This is the average or mean (M) happiness rating for females.
This is the standard error (SE) of happiness rating for females.
In this case, the means tell us that happiness ratings were higher for females than males. The results from the ANOVA support this finding that females had higher happiness ratings overall.
This is the average or mean (M) happiness rating after 6 hours of sleep.
This is the standard deviation (SD) of happiness rating after 6 hours of sleep.
This is the average or mean (M) happiness rating after 8 hours of sleep.
This is the standard deviation (SD) of happiness rating after 8 hours of sleep.
This is the average or mean (M) happiness rating after 10 hours of sleep.
This is the standard deviation (SD) of happiness rating after 10 hours of sleep.
In this case, the means tell us that happiness ratings were highest after 8 hours of sleep. The results from the post-hoc test support the finding that happiness ratings were similar for 6 and 10 hours of sleep but increased significantly for 8 hours of sleep.
Happiness
Hours of Sleep
This graph shows us that females had higher happiness ratings, regardless of the amount of sleep, than males. Also, the same pattern was observed in higher happiness after 8 hours of sleep and lower happiness after both 6 hours and 10 hours of sleep, regardless of gender. However, there was no interaction between gender and hours of sleep, as the pattern was consistent across levels of each variable.
Your Turn: Evaluating Output
Alex Brylov/Shutterstock
Below is the output from your study:
Tests of Within-Subjects Effects
Measure: MEASURE_1
Source
Type III Sum of Squares
df
Mean Square
F
Sig.
Partial Eta Squared
time
Sphericity Assumed
196.078
2
98.039
199.130
.000
.774
Greenhouse-Geisser
196.078
2.000
98.042
199.130
.000
.774
Huynh-Feldt
196.078
2.000
98.039
199.130
.000
.774
Lower-bound
196.078
1.000
196.078
199.130
.000
.744
time * Gender
Sphericity Assumed
8.811
2
4.406
8.948
.000
.134
Greenhouse-Geisser
8.811
2.000
4.406
8.948
.000
.134
Huynh-Feldt
8.811
2.000
4.406
8.948
.000
.134
Lower-bound
8.811
1.000
8.811
8.948
.004
.134
Error(time)
Sphericity Assumed
57.111
116
.492
Greenhouse-Geisser
57.111
115.997
.492
Huynh-Feldt
57.111
116.000
.492
Lower-bound
57.111
58.000
.985
Your Turn: Evaluating Output
Alex Brylov/Shutterstock
Below is the output from your study:
Tests of Between-Subjects Effects
Measure: MEASURE_1
Transformed Variable: Average
Source
Type III Sum of Squares
df
Mean Square
F
Sig.
Partial Eta Squared
Intercept
6528.089
1
6528.089
16936.692
.000
.997
Gender
35.556
1
35.556
92.247
.000
.614
Error
22.356
58
.385
Your Turn: Evaluating Output
Alex Brylov/Shutterstock
Below is the output from your study:
Pairwise Comparisons
Measure: MEASURE_1
(I) time
(J) time
Mean Difference (I-J)
Std. Error
Sig.b
95% Confidence Interval for Differenceb
Lower Bound
Upper Bound
1
2
-1.117*
.128
.000
-1.433
-.800
3
-2.550*
.128
.000
-2.865
-2.235
2
1
1.117*
.128
.000
.800
1.433
3
-1.433*
.128
.000
-1.749
-1.117
3
1
2.550*
.128
.000
2.235
2.865
2
1.433*
.128
.000
1.117
1.749
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Bonferroni.
Your Turn: Evaluating Output
Alex Brylov/Shutterstock
Below is the output from your study:
Descriptive Statistics
Measure: MEASURE_1
Source
Gender
Mean
Std. Deviation
N
Early
Male
5.03
.850
30
Female
4.57
.504
30
Total
4.80
.732
60
Middle
Male
6.27
.691
30
Female
5.57
.504
30
Total
5.92
.696
60
Late
Male
8.10
.885
30
Female
6.60
.498
30
Total
7.35
1.039
60
Estimates
Measure: MEASURE_1
Gender
Mean
Std. Error
95% Confidence Interval
Lower Bound
Upper Bound
Male
6.467
.065
6.336
6.598
Female
5.578
.065
5.447
5.709
Your Turn: Evaluating Output
chapter_12_multiple_choice_2
Alex Brylov/Shutterstock
Based on the results of your statistical analyses, match the correct number in the “Answer” column to the term requested under “Prompt”:
F for the ANOVA test - main effect of gender
p for the ANOVA test - main effect of gender
F for the ANOVA test - main effect of time of rating
p for the ANOVA test - main effect of time of rating
F for the ANOVA test - interaction effect
p for the ANOVA test – interaction effect
eta2 - main effect of gender
eta2 - main effect of time of rating
eta2 - interaction effect
Please move the correct answer to the left.
92.247
0.000
199.13
0.000
8.948
0.000
0.614
0.774
0.134
Activity: Graphing Results
chapter_12_graph_activity_1
Move the bars to represent the data in the table.
Descriptive Statistics
Measure: MEASURE_1
Source
Gender
Mean
Std. Deviation
N
Early
Male
5.03
.850
30
Female
4.57
.504
30
Total
4.80
.732
60
Middle
Male
6.27
.691
30
Female
5.57
.504
30
Total
5.92
.696
60
Late
Male
8.10
.885
30
Female
6.60
.498
30
Total
7.35
1.039
60
Gender, Time of Rating, and Attractiveness
Mean Attractiveness Raiting
Time of Rating
Your Turn: Results
gguy/Shutterstock
Now that you have worked with your data, you must determine the best way to express your findings in written form. You must be sure that how you describe your findings accurately represents the data.
Now that you have determined how to express your findings in a scientifically responsible way, you also need to be able to talk about what your findings mean in everyday terms so that the world can benefit from your science.