Left Out and Feeling Low

Introduction

In this activity, you will explore the impact of inclusion and exclusion on self-esteem by creating a design to measure change within individuals.

Dr. Melanie Maggard

Dr. Natalie J. Ciarocco, Monmouth University

Dr. David B. Strohmetz, University of West Florida

Dr. Gary W. Lewandowski, Jr., Monmouth University

Something to Think About…

Scenario: Imagine that you are a child again. You are on the playground surrounded by your peers and it’s time for teams to be chosen for a game of kickball. Your hands start to sweat and your heart races as you think, Please let me get picked. Please let me get the chance to play today. You stand there attentively as the team leaders choose their players, but before you can be picked, they reach the number of players they want. You are devastated! As the teams run off to play, you think, Why didn’t I get picked? Do they not like me? What’s wrong with me?

John Birdsall/AGE Fotostock

Something to Think About…

Being excluded from social groups can cause us to reconsider how we feel about ourselves, even if these doubts are temporary. As social beings, we tend to feel better when we are part of a group, not excluded from one. Now, we are going to investigate this concept by exploring how several experiences of being included and excluded can impact self-esteem. Self-esteem affects the way we think, behave, and interact with others, so it is important to determine what factors might influence it.

PhotoAlto/Odilon Dimier/Getty Images

Our Research Question

Based on experiences you may have with being included in or excluded from groups, you can develop a research study that examines the impact of social exclusion on self-esteem. First, you will need a framework to help you explore this topic. Research studies all start with a question, so here is your chance to ask one of your own.

Question 1.1

Which of the following research questions would be best to ask given the goal of our study?

A.Do we have higher self-esteem when we play with others or when we play alone?

B.Does being included in or excluded from playing a game with others impact young adults’ self-esteem?

C.When I am allowed to participate in a game with others, do I feel better about myself?

2

Try again.

Correct.

Incorrect.

Jesse Kunerth/iStock/Getty Images

Picking the Best Design

Now that you have a research question (“Does being included in or excluded from playing a game with others impact young adults’ self-esteem?”), you must decide which type of research design will best answer your research question. To narrow things down, consider the following:

• Nonexperimental Design
• Experimental Design

Question 1.2

Does your research question require a nonexperimental design or an experimental design?

A.Nonexperimental

B.Experimental

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Since comparisons must be made in order to answer your research question (“Does being included in or excluded from playing a game with others impact young adults’ self-esteem?”), consider the following types of experimental designs:

• Two-group Design
• Pretest-posttest Design
• Repeated-measures Design

Question 1.3

Would your research question require a two-group, pretest-posttest, or repeated-measures design?

A.Two-group design

B.Pretest-posttest design

C.Repeated-measures design

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Having decided that your research question requires multiple measurements to determine impact, you must choose the best comparisons to make.

Question 1.4

Given the research question (“Does being included in or excluded from playing a game with others impact young adults’ self-esteem?”), which comparison is best?

A.Inclusion in game vs. Exclusion from game

B.Pretest vs. Inclusion in game vs. Exclusion from game

C.Pretest vs. Inclusion in game vs. Exclusion from game vs. Posttest

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Now that you know you have an experimental design that compares the pretest to inclusion in a game to exclusion from a game, you can identify your independent and dependent variables.

• Independent Variable (IV)
• Dependent Variable (DV)

Question 1.5

Given the research question (“Does being included in or excluded from playing a game with others impact young adults’ self-esteem?”), what is your independent variable?

A.Self-esteem

B.Type of game

C.Game condition (Pretest vs. Inclusion in game vs. Exclusion from game)

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Question 1.6

Given the research question (“Does being included in or excluded from playing a game with others impact young adults’ self-esteem?”), what is your dependent variable?

A.Type of game

B.Self-esteem

C.Pretest

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Question 1.7

Because you have an experiment with 1 independent variable and 3 levels (Pretest vs. Inclusion in game vs. Exclusion from game) that each participant is exposed to, you will use a type of within-subjects design called a repeated-measures design. Why have we selected a within-subjects design to address our research question instead of a between-subjects design?

A.It allows us to draw conclusions about changes within an individual over time due to certain conditions.

B.It allows us to draw conclusions about differences between groups of individuals who are exposed to different conditions or treatments.

C.It requires more participants since each participant is exposed to each condition.

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Picking the Best Design

Your experiment includes 1 independent variable with 3 levels (Pretest vs. Inclusion in game vs. Exclusion from game). The pretest level serves as a baseline measurement to which the other measurements can be compared.

• Baseline Measurement

Question 1.8

Which of the following is an advantage of including a baseline measurement as the pretest in this study?

A.It reduces the number of measurements required to draw conclusions about the impact of inclusion in or exclusion from playing a game on self-esteem.

B.It gives us a benchmark against which to compare the results from the various levels (inclusion in game and exclusion from game) to the self-esteem recorded at the onset of the study.

C.It helps the researcher pretest any measurements they intend to use before collecting “real” data.

2

Try again.

Correct.

Incorrect.

Lev Mei/iStock/Getty Images

Operationally Defining the Independent Variable

Next, we need to operationally define the independent variable (IV) of game condition by determining exactly how we will manipulate it. We will want to be sure that our study has a high level of experimental and mundane realism.

• Experimental Realism
• Mundane Realism

Operationally Defining the Independent Variable

It looks like the task that is highest in experimental and mundane realism involves young adults playing an electronic game of catch called “cyberball.” We know that all participants will be measured at 3 points in time: pretest (before the study begins), after being included in the game, and after being excluded from the game. Therefore, we will have the following design:

Operationally Defining the Dependent Variable

You have now established the key comparison between the Pretest vs. Inclusion in game vs. Exclusion from game. Next, we need to specify the exact nature of our dependent variable, self-esteem. First, consider the following:

Choosing the Best Measure

We know we want to use a self-report measure to measure self-esteem. Now it is time to determine which type of self-report measure to use. Keep in mind how many and what types of questions, and what assessment of reliability, validity, and sensitivity, would be ideal for young adults.

• Reliability
• Validity
• Sensitivity

Choosing the Best Measure

We can update the chart we made earlier to reflect how we will be measuring the dependent variable, which is shown in the last row in the table below

Weighing Our Options

One potential problem with repeated-measures designs is the possibility of order effects.

• Order Effect

The following are 4 types of order effects we could encounter in our study:

• Practice Effect
• Fatigue Effect
• Carryover Effect
• Sensitization Effect

Weighing Our Options

Since we have the potential for multiple order effects in this study, we must consider how to minimize their impact. Fortunately, we do not think that the potential for practice and sensitization effects will drastically impact the results, so we decide to keep the same measure of self-esteem, SSES, constant throughout the study. However, we do think it would be worthwhile to reduce the impact of the carryover effect by using counterbalancing.

• Counterbalancing

Weighing Our Options

Let’s update the chart we made earlier to reflect the counterbalancing method we have chosen for this study. Notice how we now have a second sequence that allows us to measure self-esteem after being excluded from a game and prior to exposure to the inclusion level, thus covering all possible sequences in our study.

Determining Your Hypothesis

Now that you have determined what you will manipulate and measure, you must formulate an experimental hypothesis.

• Experimental Hypothesis

Question 1.14

Which of the following is the best experimental hypothesis given the nature of your study?

A.Young adults’ self-esteem after being included in a game will be higher than it was before playing the game. Their self-esteem after being excluded from a game will be lower than it was before playing the game.

B.There will be no difference in young adults’ self-esteem before playing the game, after being included in the game, and after being excluded from the game.

C.Young adults’ self-esteem after being excluded from playing the game will be lower than it would be after being included in the game.

2

Try again.

Correct.

Incorrect.

Jesse Kunerth/iStock/Getty Images

Finding a Sample

Before you can conduct your experiment, you need to determine exactly whom you want to study and where you can find this target sample.

Question 1.15

Which of the following samples would be best for your experiment?

A.A national random sample of the general population

B.A random sample of college students from a research participant pool

C.A convenience sample of students in the classes you are currently taking

2

Try again.

Correct.

Incorrect.

Caiaimage/Robert Daly/OJO+/Getty Images

Submitting to the IRB

Now that you have determined how you will collect your data and your intended sample, you must submit your research procedure to the Institutional Review Board (IRB) for ethical approval. The IRB will determine whether or not your study meets all ethical guidelines.

• Institutional Review Board

  (IRB)

Each IRB has its own protocol that conforms to the national standard when a researcher submits an application for proposed research. In addition to the appropriate paperwork and other information submitted to the IRB, the board would consider the following description during their evaluation of your proposed experiment:

• The purpose of this research is to determine whether being included in or excluded from playing a virtual game of “cyberball” will result in a change to self-esteem. To study this topic, 30 participants will be randomly selected from the research participant pool at the University. Researchers will measure all participants’ self-esteem via the State Self-Esteem Scale (SSES) at the beginning of the study, after being included in a virtual game of “cyberball” for 5 minutes, and after being excluded from a virtual game of “cyberball” for 5 minutes. Counterbalancing will be used such that half of the participants will receive the inclusion-exclusion sequence and half will receive the exclusion-inclusion sequence. Participants will be debriefed at the end of the study.

wk1003mike/Shutterstock

Responding to the IRB

The IRB reviewed your submission and has one concern. Although the study appears to present less than minimal risk to participants, there is no mention of informed consent and voluntary participation.

You must now determine how to respond to the IRB, keeping in mind the ethics of respect for persons and autonomy.

Question 1.16

Which of the following is the best response to the IRB’s concern?

A.You respond by stating that participants have already consented by being part of the research participation pool; thus, no more confirmation of informed consent is required.

B.You respond by telling the IRB that you will have all participants sign an informed consent form before the study.

C.You respond by stating that verbal assent will be obtained from all participants before the study.

2

Try again.

Correct.

Incorrect.

wk1003mike/Shutterstock

Running the Study

chapter_10_multiple_choice

Now that we have secured the IRB’s approval, we should determine what the entire study will look like. Below are the steps of the study; can you place them in the proper order? (Note: The State Self-Esteem Scale is referred to as SSES.)

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Participants take the SSES for the pretest measure.

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Obtain informed consent.

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Participants receive instructions for the game and play their first game of “cyberball,” which is programmed to include them in or exclude them from the game (depending on their sequence).

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Participants take the SSES after the second game.

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Debrief the participants.

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Participants play their second game of “cyberball,” which is programmed to include them in or exclude them from the game (depending on their sequence).

-

• 1
• 2
• 3
• 4
• 5
• 6
• 7

Participants take the SSES after the first game.

Feedback

ResetSubmit

Wichy/Shutterstock

Collecting Data

Now that you have a sense of how to conduct this study, it is time to see what data from this study might look like.

If you were to run a full version of this study, you would want to have at least 30 participants. Because you have a within-subjects design, each participant will be exposed to all levels of the independent variable.

Image Source/Photodisc/Getty Images

Example Data Set

This is an example of what your data set would look like. The top row shows the variable names; the other rows display the data for the first 5 participants in each sequence.

In the “Sequence” column, a 1 = Inclusion-Exclusion sequence, and a 2 = Exclusion-Inclusion sequence. The Baseline, Inclusion, and Exclusion columns represent a participant’s score measured via the SSES prior to the study, after inclusion in the game, and after exclusion from the game.

Selecting the Proper Tool

Now that you have collected your data, you must decide the best way to summarize your findings. The decisions you made about how to collect your data dictate the statistics you can use with your data now. First, you need to consider if your study is descriptive or inferential.

• Descriptive
• Inferential

Question 1.17

Given the nature of your experiment, which of the following statistical methods is more appropriate?

A.Descriptive statistics

B.Inferential statistics

2

Try again.

Correct.

Incorrect.

Ines Koleva/E+/Getty Images

Tutorial: Evaluating Output

The following is an example of output for another three-level design where participants experienced all 3 conditions in the study. This study was about how hours slept at night (6 hours, 8 hours, and 10 hours) influence self-reported happiness.

Before continuing, go through the tables to learn more about each of its elements.

Tests of Within-Subjects Effects

Measure: MEASURE_1
Source		Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
Hours	Sphericity Assumed	91.289	2	45.644	67.230	.000	.699
	Greenhouse-Geisser	91.289	1.694	53.903	67.230	.000	.699
	Huynh-Feldt	91.289	1.787	51.074	67.230	.000	.699
	Lower-bound	91.289	1.000	91.289	67.230	.000	.699
Error(Hours)	Sphericity Assumed	39.378	58	.679
	Greenhouse-Geisser	39.378	49.113	.802
	Huynh-Feldt	39.378	51.834	.760
	Lower-bound	39.378	29.000	1.358

This is the df or degrees of freedom. An ANOVA has 2 dfs, one for the main effect (within-groups) and one for the error (residual).

This is the F statistic. It represents the size of the difference between condition means compared to the size of the residual error.

This is the p level or the significance level. It represents the probability or likelihood that the results happened by chance. The lower the p level, the less likely the result happened by chance. This would be reported as p < .001 in the results.

The F score and p level will only tell you whether there is a significant difference. To determine which means are different, and the nature or direction of those differences, you need to look at the means via a post-hoc test.

The eta squared (eta² ) is the effect size. It tells us the proportion of change in the dependent variable that is associated with being in the different groups of the independent variable.

Alex Brylov/Shutterstock

Tutorial: Evaluating Output

To report these numbers in a results section, enter the numbers as follows:

F (#,#) = #.##, p = .##, eta² = .##.

F from some variable equals a number; p equals a number; eta squared equals a number.

Pairwise Comparisons

Measure: MEASURE_1
(I) Hours	(J) Hours	Mean Difference (I-J)	Sth. Error	Sig. superscript b	95% Confidence Interval for Difference superscript b
					Lower Bound	Upper Bound
1	2	-2.200*	.222	.000	-2.762	-1.638
	3	-.133	.164	.808	-.549	.283
2	1	2.200*	.222	.000	1.638	2.762
	3	2.067*	.244	.000	1.448	2.685
3	1	.133	.164	.808	-.283	.549
	2	-2.067*	.244	.000	-2.685	-1.448
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Sidak.

The results presented here are from the post-hoc test, which compares each of the groups’ means to all of the other groups’ means.

1 = 6 hours, 2 = 8 hours, and 3 = 10 hours

This is the difference between the mean happiness rating for the 6 hours of sleep and 8 hours of sleep conditions.

This is the difference between the mean happiness rating for the 6 hours of sleep and 10 hours of sleep conditions.

This is the difference between the mean happiness rating for the 8 hours of sleep and 10 hours of sleep conditions.

Happiness was different in the 8 hours sleep condition from the 6 hours and 10 hours sleep conditions, which had similar ratings.

The post-hoc test tells us which comparisons between the means were significant. The p level tells us the significance level of that comparison.

Alex Brylov/Shutterstock

Tutorial: Evaluating Output

Descriptive Statistics

	Mean	Std. Deviation	N
Six	1.77	.728	30
Eight	3.97	.809	30
Ten	1.77	.712	30

This is the average or mean (M) happiness rating after 6 hours of sleep.

This is the standard deviation (SD) of happiness rating after 6 hours of sleep.

This is the average or mean (M) happiness rating after 8 hours of sleep.

This is the standard deviation (SD) of happiness rating after 8 hours of sleep.

This is the average or mean (M) happiness rating after 10 hours of sleep.

This is the standard deviation (SD) of happiness rating after 10 hours of sleep.

In this case, the means tell us that happiness ratings were highest after 8 hours of sleep. The results from the post-hoc test support the finding that happiness ratings were similar for 6 and 10 hours of sleep, but increased significantly with 8 hours of sleep.

Alex Brylov/Shutterstock

Your Turn: Evaluating Output

Below is the output from your study:

Tests of Within-Subjects Effects

Measure: MEASURE_1
Source		Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
GameCondition	Sphericity Assumed	2081.156	2	1040.578	101.918	.000	.778
	Greenhouse-Geisser	2081.156	1.816	1146.056	101.918	.000	.778
	Huynh-Feldt	2081.156	1.930	2081.156	101.918	.000	.778
	Lower-bound	2081.156	1.000	91.289	101.918	.000	.778
Error(GameCondition)	Sphericity Assumed	592.178	58	10.210
	Greenhouse-Geisser	592.178	52.662	11.245
	Huynh-Feldt	592.178	55.983	10.578
	Lower-bound	592.178	29.000	20.420

Alex Brylov/Shutterstock

Your Turn: Evaluating Output

Below is the output from your study:

Pairwise Comparisons

Measure: MEASURE_1
(I) GameCondition	(J) GameCondition	Mean Difference (I-J)	Std. Error	Sig. superscript b	95% Confidence Interval for Difference superscript b
					Lower Bound	Upper Bound
1	2	-3.133*	.728	.001	-4.979	-1.288
	3	8.267*	.788	.000	6.271	10.262
2	1	3.133*	.728	.001	1.288	4.979
	3	11.400*	.944	.000	9.008	13.792
3	1	-8.267*	.788	.000	-10.262	-6.271
	2	-11.400*	.944	.000	-13.792	-9.008
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
b. Adjustment for multiple comparisons: Sidak.

Descriptive Statistics

	Mean	Sth. Deviation	N
Baseline	61.10	25.694	30
Inclusion	64.23	24.694	30
Exclusion	52.83	26.478	30

Alex Brylov/Shutterstock

Your Turn: Evaluating Output

chapter_10_multiple_choice_2

Based on the results of your statistical analyses, match the correct number in the “Answer” column to the term requested under “Prompt”:

F for the ANOVA testF for the ANOVA test

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

df for the main effect of condition (within-groups)df for the main effect of condition (within-groups)

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

df for error (residual)df for error (residual)

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

p for the ANOVA testp for the ANOVA test

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

eta squaredeta²

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

Mean difference between Baseline and InclusionMean difference between Baseline and Inclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

p for the difference between Baseline and Inclusionp for the difference between Baseline and Inclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

Mean difference between Baseline and ExclusionMean difference between Baseline and Exclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

p for the difference between Baseline and Exclusionp for the difference between Baseline and Exclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

Mean difference between Inclusion and ExclusionMean difference between Inclusion and Exclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

p for the difference between Inclusion and Exclusionp for the difference between Inclusion and Exclusion

-

• 0.000
• 11.400
• 3.133
• 8.267
• 58
• 0.778
• 2
• 101.918
• 0.001

Feedback

ResetSubmit

Alex Brylov/Shutterstock

Activity: Graphing Results

In order to visualize your data, use the values on the previous screens to input the mean that corresponds to each condition listed in the output. Then, check out the graphic representation of your data, below.

Your Turn: Results

Now that you have worked with your data, you must determine the best way to express your findings in written form. You must be sure that how you describe your findings accurately represents the data.

Question 1.18

Based on the statistical analysis, which of the following results sections best fits the data and analyses from your study?

A.Contrary to the hypothesis, self-esteem was lower at baseline (M = 61.10, SD = 25.218) than after inclusion in a game (M = 64.23, SD = 24.694).

B.As hypothesized, self-esteem was lower after inclusion in a game (M = 61.10, SD = 25.218) than exclusion from a game (M = 64.23, SD = 24.694).

C.Contrary to the hypothesis, self-esteem was lower after exclusion from a game (M = 52.83, SD = 26.478) than at baseline (M = 61.10, SD = 25.218), which was, in turn, lower than after inclusion in a game (M = 64.23, SD = 24.694).

D.As hypothesized, self-esteem was lower after exclusion from a game (M = 52.83, SD = 26.478) than at baseline (M = 61.10, SD = 25.218), which was, in turn, lower than after inclusion in a game (M = 64.23, SD = 24.694).

2

Try again.

Correct.

Incorrect.

gguy/Shutterstock

Take Home Message

RapidEye/iStockphoto/Getty Images

You have determined how to express your findings in a scientifically responsible way. Now, you need to be able to talk about what your findings mean in everyday terms so that the world can benefit from your science.

Congratulations! You have successfully completed this activity.

KMGS Photography/Moment/Getty Images