Chapter 1. Is the Swagger-Meter 1.0 a Quality Measure?

Introduction

In this activity, you will evaluate whether a device called the Swagger-Meter is a quality measure and will consider the best way to administer this measure.

Dr. Natalie J. Ciarocco, Monmouth University

Dr. David B. Strohmetz, University of West Florida

Dr. Gary W. Lewandowski, Jr., Monmouth University

Something to Think About…

Scenario: The mall is a popular place for the 18- to 25-year-old demographic. Companies know this and continually seek to develop new products that target this key segment of the market. In fact, a new company named YOLO Incorporated has designed a product that, according to their website and press release, “provides user-specific, science-based insights into a key aspect of young adults’ personalities: swagger.” In addition, “the Swagger-Meter uniquely capitalizes on people’s desire for self-knowledge and data-based social comparison information.” Specifically, YOLO Inc. wants to place their new product, the Swagger-Meter 1.0, in mall food courts across the United States.

Your job is to determine whether you should invest your money in this company.

Something to Think About…

If you are thinking like a scientist, you are probably feeling skeptical about the Swagger-Meter 1.0. To decide whether this product will be successful and worth your investment, you need to determine whether the Swagger-Meter 1.0 is a quality measure.

John Giustina/Iconica/Getty Images

Something to Think About…

Before you can know whether something is a quality measure, you need to clearly define the variable that it aims to measure.

Something to Think About…

According to the YOLO Inc. press release, “Swagger is a mental state of being in which a person exudes a high level of confidence and composure. Swagger can border on arrogance, but is more about a sense of positive self-assuredness that makes the person charismatic and admired.”

Now that you know how YOLO Inc. defines “swagger,” it is time to find out how the company attempts to measure “swagger.” First, YOLO Inc. asks individuals to place their right hand on a metal plate. The company then measures their skin conductance, which corresponds to “degree of swagger.” This is what the Swagger-Meter 1.0 looks like. . .

The Swagger-Meter 1.0

Our Research Question

Jesse Kunerth/iStock/Getty Images

Now that you know what the Swagger-Meter 1.0 looks like, and what it claims to measure, you can begin to answer two questions:

1. Is the Swagger-Meter 1.0 a quality measure?
2. Should you invest your money in this product and YOLO Inc.?

Picking the Best Measurement

Let’s start by evaluating the nature of the Swagger-Meter 1.0 measurements.

• Self-report

• Behavioral Measure

Lev Mei/iStock/Getty Images

Identifying the Design

Lev Mei/iStock/Getty Images

Picking the Best Measurement

Now, let’s evaluate the different types of behavioral measures.

• Behavioral Trace

• Behavioral Observation

• Behavioral Choice

Lev Mei/iStock/Getty Images

Evaluating the Measure

Keep in mind that all measurements are approximate. In fact, every measurement consists of three parts: raw score, true score, and error.

• Raw Score

• True Score

• Error

Evaluating the Measure

Since all measures contain some form of error, the Swagger-Meter 1.0 will be no exception. We must identify where error can creep in and determine how problematic it might prove to be. First, let’s review the two types of measurement error: random error and bias (also known as systematic error).

• Random Error

• Bias or Systematic Error

Evaluating the Measure

Now it’s your turn. Indicate whether the following potential sources of measurement error represent random error or bias.

Evaluating the Measure

To decide whether or not the Swagger-Meter 1.0 is a high-quality measure, you need to determine if YOLO Inc. has developed a consistent process for measuring people’s swagger. In other words, you need to know if there is standardization in the administration of the Swagger-Meter 1.0.

• Standardization

Evaluating the Measure

Evaluating the Measure: A Checklist

Evaluating the Measure

If the Swagger-Meter 1.0 is truly a high-quality measure, it will have adequate sensitivity, or a good range of possible measurement levels.

• Sensitivity

Evaluating the Measure: A Checklist

Evaluating the Measure

If the Swagger-Meter 1.0 is a high-quality measure, it must be able to consistently and reliably assess users’ swagger.

• Reliability

Evaluating the Measure

Based on what you know about reliability, you decide that you need to test the Swagger-Meter 1.0 prototype during an investor visit to YOLO Inc.

Evaluating the Measure: A Checklist

Evaluating the Measure

So far, you have determined that the Swagger-Meter 1.0 has adequate standardization, sensitivity, and reliability. Next, you need to evaluate its validity. That is, does the Swagger-Meter 1.0 actually measure “swagger”?

• Validity

Evaluating the Measure

One way to check a measure’s validity is to compare it to other measures of swagger. The YOLO Inc. press release states, “Our Research and Development Department placed a beta version of the Swagger-Meter in a nearby mall where patrons of all ages could choose to test their swagger. Field tests included 490 patrons with equal numbers of males and females ranging in age from 18–87. . . . All test participants reported the Swagger-Meter was uncanny in its accuracy.”

• Simple Random Sampling

• Convenience Sampling

Evaluating the Measure

Anytime someone uses data to support a claim being made on TV, in an advertisement, or, in this case, in a business press release, you need to use your psychological literacy skills to critically evaluate their claim. After all, you don’t want to make the wrong choice based on faulty information. A good place to start your evaluation is the source of the data, namely the sample used. Common issues that can compromise data include the nonresponse bias, volunteer subject problem, and college sophomore problem.

• Nonresponse Bias

• Volunteer Subject Problem

• College Sophomore Problem

Evaluating the Measure

YOLO Inc.’s press release states, “Based on their data and feedback, the R & D Department’s analysis demonstrates that the Swagger-Meter 1.0 accurately measures a person’s ‘hotness’ and ‘electricity of personality’ using a set of strategically placed titanium sensors that assess ‘tactile temperature’ and ‘skin conductance/galvanic skin response’ on the person’s hand. All test participants reported the Swagger-Meter was uncanny in its accuracy.”

For each of the following findings, indicate whether the finding fails to support the validity of the Swagger-Meter 1.0 as a true measure of swagger.

Evaluating the Measure

For each of the following findings, indicate whether the finding fails to support the validity of the Swagger-Meter 1.0 as a true measure of swagger.

• Demand Characteristic

• Social Desirability

Evaluating the Measure: A Checklist

The Decision

As successful investor Warren Buffett advised, “Never invest in a business you cannot understand.” Now that you have spent time studying the Swagger-Meter 1.0, you must make a final determination about whether the Swagger-Meter 1.0 is a quality measure, and, if it is, whether you should invest your money in this company.

The Decision

Because you contacted YOLO Inc. for a copy of its press release, someone from the Investor Relations Department gives you a call as a follow-up to answer any questions you may have. During that call, you explain your evaluation of the measure and concerns about its lack of validity. The company representative freely admits that the Swagger-Meter 1.0 may not actually measure swagger. But, he goes on to explain, “Users are confident that it measures swagger. Besides, if people believe it is true, who cares whether it is? People are free to believe what they want and believe all sorts of things that aren’t true all the time.”

Take Home Message

RapidEye/iStockphoto/Getty Images

Now that you have decided not to invest in YOLO Inc.’s new product, it would be useful to explain what your findings mean in everyday terms so that others can benefit from your evaluation.

Congratulations

Congratulations! You have successfully completed this activity.

KMGS Photography/Moment/Getty Images