5.5 Review of Concepts

Samples and Their Populations

The gold standard of sample selection is random sampling, a procedure in which every member of the population has an equal chance of being chosen for study participation. A random numbers table or a computer-based random numbers generator is used to ensure randomness. For practical reasons, random selection is uncommon in social science research. Many behavioral scientists use a convenience sample, a sample that is readily available to them. One kind of convenience sample is the volunteer sample (also called a self-selected sample), in which participants themselves actively choose to participate in the study. In random assignment, every participant in a study has an equal chance of being assigned to any of the experimental conditions. In conjunction with random assignment, replication—the duplication of scientific results—can go a long way toward increasing generalizability, our ability to generalize our findings beyond our samples.

Probability

Calculating probabilities is essential because human thinking is dangerously biased. Because of a confirmation bias—the tendency to see patterns that we expect to see—we often see meaning in mere coincidence. A confirmation bias often results from an illusory correlation, a relation that appears to be present but does not exist. When we think of probability, many of us think of personal probability, a person’s own judgment about the likelihood that an event will occur. Statisticians, however, are referring to expected relative-frequency probability, or the long-run expected outcome if an experiment or trial were repeated many, many times. A trial refers to each occasion that a procedure is carried out, and an outcome is the result of a trial. A success refers to the outcome for which we’re trying to determine the probability. Probability is a basic building block of inferential statistics. When we draw a conclusion about a population based on a sample, we can only say that it is probable that our conclusion is accurate, not that it is certain.

Inferential Statistics

Inferential statistics, based on probability, start with a hypothesis. The null hypothesis is a statement that usually postulates that there is no average difference between populations. The research, or alternative, hypothesis is a statement that postulates that there is an average difference between populations. After conducting a hypothesis test, we have only two possible conclusions. We can either reject or fail to reject the null hypothesis. When we conduct inferential statistics, we are often comparing an experimental group, the group subjected to an intervention, with a control group, the group that is the same as the experimental group in every way except the intervention. We use probability to draw conclusions about a population by estimating the probability that we would find a given difference between sample means if there is no underlying difference between population means.

Type I and Type II Errors

Statisticians must always be aware that their conclusions may be wrong. If a researcher rejects the null hypothesis, but the null hypothesis is correct, the researcher is making a Type I error. If a researcher fails to reject the null hypothesis, but the null hypothesis is false, the researcher is making a Type II error. Scientific and medical journals tend to publish, and the media tend to report on, the most exciting and surprising findings. As such, Type I errors are often overrepresented among reported findings.