Chapter 4. The Psychologist’s Building Blocks for Better Design

Introduction

Something to Think About

Scenario: The mall is a popular place for the 16-25 year-old demographic. Companies know this and continually seek to develop new products that target this key segment of the market. In fact, a new company named YOLO Incorporated has designed a product that, according to their website and press release, “provides user-specific, science-based insights into a key aspect of young adults’ personality.” In addition, “the Swagger-Meter uniquely capitalizes on people’s desire for self-knowledge and data-based social comparison information.” Specifically, YOLO Inc. wants to place their new product the “Swagger-Meter 1.0™” in mall food courts throughout the United States.

Your job is to determine whether you should invest your money in this company.

Something to Think About (continued)

If you are thinking like a scientist, you are probably a bit skeptical about the “Swagger-Meter 1.0™.” To decide whether this product will be successful and ultimately worth investing your money in, you need to determine whether the “Swagger Meter 1.0™” is a quality measure.

Before you can know whether something is a good measure, you first need to know exactly what it aims to measure. That is, you need to clearly define key concepts with an operational definition that describes exactly what a variable means.

Something to Think About (continued)

According to the YOLO Inc. press release, “Swagger is a mental state of being in which a person exudes a high level of confidence and composure. Swagger can border on arrogance, but is more about a sense of positive self-assuredness that makes the person charismatic and admired.”

Now that you know how YOLO Inc. is defining what is meant by “swagger,” the next question is to understand how they are actually measuring “swagger.” Yolo Inc. is doing this by having individuals place their right hand on a mental plate. This plate measures their skin conductance and indicates the corresponding degree of swagger. This is what the “Swagger Meter 1.0™” looks like:

Swagger Meter 1.0™

Our Question

Now that you know what the “Swagger Meter 1.0™” looks like, and what it claims to measure, you can begin to answer two questions:

1. Is the “Swagger Meter 1.0™” a quality measure?
2. Should you invest your money in this company and product?

Picking the Best Measurement

Before you can know whether something is a good measure, you first need to know exactly what it aims to measure. That is, you need to clearly define key concepts with an operational definition that describes exactly what a variable means.

Let’s start by evaluating the nature of the “Swagger Meter 1.0™” measurements.

• Self Report Measure
• Behavioral Measure

Picking the Best Measurement

Picking the Best Measurement

Using one of the types of behavioral measure was a good idea.

• Behavioral Trace
• Behavioral Observation
• Behavioral Choice

Evaluating the Measure

Measurement is inherently imperfect. In fact, every measurement consists of three parts:

• Raw Score
• True Score
• Error

Evaluating the Measure

If all measures contains some type of error, the “Swagger Meter 1.0™” will be no exception. The key will be to identify where error can creep in and determine how much of problem it may be. There are two types of measurement error.

• Random Error
• Bias or Systematic Error

Evaluating the Measure

For each of the following potential sources of measurement error, please indicate whether it represents random error or systematic bias.

Evaluating the Measure

To determine if the “Swagger Meter 1.0™” is a high quality measure, you need to determine if YOLO Inc. has developed a consistent process for measuring a person’s swagger. In other words, you need to know if there is standardization in the administration of the “Swagger Meter 1.0™.”

• Standardization

Evaluating the Measure

Evaluating the Measure: A Checklist

Evaluating the Measure

If the “Swagger Meter 1.0™” is a high quality measure, it will have adequate sensitivity or a good range of possible measurement levels.

• Sensitivity

Evaluating the Measure: A Checklist

Evaluating the Measure

If the “Swagger Meter 1.0™” is a high quality measure, it must be able to consistently or reliably assess users’ swagger.

• Reliability

Evaluating the Measure

Based on what you know about reliability, you decide that you need to test the “Swagger Meter 1.0™” prototype during an investor visit to YOLO Inc.

Evaluating the Measure: A Checklist

Evaluating the Measure

So far you have determined that the “Swagger Meter 1.0™” has adequate standardization, good sensitivity, and good reliability. Next you need to evaluate whether it actually measures what it claims to measure. That is, does it actually measure “swagger?”

• Validity

Evaluating the Measure

One way to check a measure’s validity is to compare it to other measures of swagger. The YOLO Inc.’s press release states, “Our Research and Development Department placed a beta version of the Swagger Meter in the nearby mall where mall patrons of all ages could choose to test their swagger. Our test included 490 patrons with equal numbers of males and females ranging in age from 18 – 87… All test participants reported the Swagger Meter was uncanny in its accuracy.”

• Random Sampling
• Convenience Sampling

Evaluating the Measure

Anytime someone uses data to support a claim being made on TV, in a magazine, an advertisement, or in this case on a business press release, you need to use your psychological literacy skills to critically evaluate that claim. After all, you don’t want to make the wrong choice based on potentially faulty information. A good place to start is to look at the source of the data, namely the sample used. Some common problems with samples that can bias the data include:

• Nonresponse Bias
• Volunteer Subject Problem
• College Sophomore Problem

Evaluating the Measure

YOLO Inc.’s press release states, “Based on field test data, the R & D department’s analysis demonstrates that the “Swagger Meter 1.0™” capably measures a user’s swagger based on user’s ‘hotness’ and the ‘electricity of their personality.’ The Swagger Meter accurately measures these constructs using a set of strategically placed titanium sensors that assess ‘tactile temperature’ and ‘skin conductance/galvanic skin response’ on the person’s palm. All test participants reported the Swagger Meter was uncanny in its accuracy.”

For each of the following, please indicate whether this finding undermines the validity of the “Swagger Meter 1.0™ as a measure of swagger.

Evaluating the Measure

• Demand Characteristics
• Social Desirability Concerns

Evaluating the Measure: A Checklist

The Decision

As Warren Buffet advised, “Never invest in a business you cannot understand.” Now that you have spent time studying the “Swagger Meter 1.0™,” you now must make a final determination about whether the “Swagger Meter 1.0™” is a quality measure, and if so, whether you should invest your money in this company.

The Decision

Because you contacted YOLO Inc. for a copy of its press release, someone from the Investor Relations Department gives you a call as a follow-up to answer any questions you may have. During that call you explain your evaluation of the measure and concerns about its lack of validity. The company representative freely admits that the “Swagger Meter 1.0™” may not actually measure swagger. But, he goes on to explain “Users are confident that it measures swagger. Besides, if people believe it is true, who cares whether it is. People are free to believe what they want and believe all sorts of things that aren’t true all the time.”

Take Home Message

Now that you have decided to not invest in YOLO Inc.’s new product, it is useful to explain what your findings mean in everyday terms so that others can benefit from your evaluation.

Next Step

Your Turn

Before you can know whether something is a good measure, you first need to know exactly what it aims to measure. That is, you need to clearly define key concepts with an operational definition that describes exactly what a variable means.

Your Turn

Activity Completed!

Activity results are being submitted.