The probability histograms and density curves that picture the probability distributions of random variables resemble our earlier pictures of distributions of data. In describing data, we moved from graphs to numerical measures such as means and standard deviations. Now we make the same move to expand our descriptions of the distributions of random variables. We can speak of the mean winnings in a game of chance or the standard deviation of the randomly varying number of calls a travel agency receives in an hour. In this section, we learn more about how to compute these descriptive measures and about the laws they obey.

The mean of a random variable

In Chapter 1 (page 24), we learned that the mean is the average of the observations in a sample. Recall that a random variable is a numerical outcome of a random process. Think about repeating the random process many times and recording the resulting values of the random variable. In general, you can think of the mean of a random variable as the average of a very large sample. In the case of discrete random variables, the relative frequencies of the values in the very large sample are the same as their probabilities.

Here is an example for a discrete random variable.

220

If you play Tri-State Pick 3 several times, we would, as usual, call the mean of the actual amounts you win . The mean in Example 4.28 is a different quantity—it is the long-run average winnings you expect if you play a very large number of times.

Just as probabilities are an idealized description of long-run proportions, the mean of a probability distribution describes the long-run average outcome. We can't call this mean , so we need a different symbol. The common symbol for the mean of a probability distribution is , the Greek letter mu. We used in Chapter 1 for the mean of a Normal distribution, so this is not a new notation. We will often be interested in several random variables, each having a different probability distribution with a different mean.

To remind ourselves that we are talking about the mean of , we often write rather than simply . In Example 4.28, . Notice that, as often happens, the mean is not a possible value of . You will often find the mean of a random variable called the expected value of . This term can be misleading because we don't necessarily expect an observation on X to equal its expected value.

The mean of any discrete random variable is found just as in Example 4.28. It is not simply an average of the possible outcomes, but a weighted average in which each outcome is weighted by its probability. Because the probabilities add to 1, we have total weight 1 to distribute among the outcomes. An outcome that occurs half the time has probability one-half and gets one-half the weight in calculating the mean. Here is the general definition.

221

Suppose that the random digits in Example 4.29 had a different probability distribution. In Case 4.1 (pages 184–185), we described Benford's law as a probability distribution that describes first digits of numbers in many real situations. Let's calculate the mean for Benford's law.

Figure 4.16 locates the means of and on the two probability histograms. Because the discrete uniform distribution of Figure 4.16(a) is symmetric, the mean lies at the center of symmetry. We can't locate the mean of the right-skewed distribution of Figure 4.16(b) by eye—calculation is needed.

222

What about continuous random variables? The probability distribution of a continuous random variable is described by a density curve. Chapter 1 showed how to find the mean of the distribution: it is the point at which the area under the density curve would balance if it were made out of solid material. The mean lies at the center of symmetric density curves such as the Normal curves. Exact calculation of the mean of a distribution with a skewed density curve requires advanced mathematics.²⁴ The idea that the mean is the balance point of the distribution applies to discrete random variables as well, but in the discrete case, we have a formula that gives us this point.

Mean and the law of large numbers

With probabilities in hand, we have shown that, for discrete random variables, the mean of the distribution can be determined by computing a weighted average in which each possible value of the random variable is weighted by its probability. For example, in Example 4.30, we found the mean of the first digit of numbers obeying Benford's law is 3.441.

Suppose, however, we are unaware of the probabilities of Benford's law but we still want to determine the mean of the distribution. To do so, we choose an SRS of financial statements and record the first digits of entries known to follow Benford's law. We then calculate the sample mean to estimate the unknown population mean . In the vocabulary of statistics, is referred to as a parameter and is called a statistic. These terms and their definitions are more formally described in Section 5.3 when we introduce the ideas of statistical inference.

It seems reasonable to use to estimate . An SRS should fairly represent the population, so the mean of the sample should be somewhere near the mean of the population. Of course, we don't expect to be exactly equal to , and we realize that if we choose another SRS, the luck of the draw will probably produce a different . How can we control the variability of the sample means? The answer is to increase the sample size. If we keep on adding observations to our random sample, the statistic is guaranteed to get as close as we wish to the parameter and then stay that close. We have the comfort of knowing that if we gather up more financial statements and keep recording more first digits, eventually we will estimate the mean value of the first digit very accurately. This remarkable fact is called the law of large numbers. It is remarkable because it holds for any population, not just for some special class such as Normal distributions.

The behavior of is similar to the idea of probability. In the long run, the proportion of outcomes taking any value gets close to the probability of that value, and the average outcome gets close to the distribution mean. Figure 4.1 (page 174) shows how proportions approach probability in one example. Here is an example of how sample means approach the distribution mean.

The mean of a random variable is the average value of the variable in two senses. By its definition, is the average of the possible values, weighted by their probability of occurring. The law of large numbers says that is also the long-run average of many independent observations on the variable. The law of large numbers can be proved mathematically starting from the basic laws of probability.

Thinking about the law of large numbers

The law of large numbers says broadly that the average results of many independent observations are stable and predictable. The gamblers in a casino may win or lose, but the casino will win in the long run because the law of large numbers says what the average outcome of many thousands of bets will be. An insurance company deciding how much to charge for life insurance and a fast-food restaurant deciding how many beef patties to prepare also rely on the fact that averaging over many individuals produces a stable result. It is worth the effort to think a bit more closely about so important a fact.

The “law of small numbers”

224

Both the rules of probability and the law of large numbers describe the regular behavior of chance phenomena in the long run. Psychologists have discovered that our intuitive understanding of randomness is quite different from the true laws of chance.²⁵ For example, most people believe in an incorrect “law of small numbers.” That is, we expect even short sequences of random events to show the kind of average behavior that, in fact, appears only in the long run.

Some teachers of statistics begin a course by asking students to toss a coin 50 times and bring the sequence of heads and tails to the next class. The teacher then announces which students just wrote down a random-looking sequence rather than actually tossing a coin. The faked tosses don't have enough “runs” of consecutive heads or consecutive tails. Runs of the same outcome don't look random to us but are, in fact, common. For example, the probability of a run of three or more consecutive heads or tails in just 10 tosses is greater than 0.8.²⁶ The runs of consecutive heads or consecutive tails that appear in real coin tossing (and that are predicted by the mathematics of probability) seem surprising to us. Because we don't expect to see long runs, we may conclude that the coin tosses are not independent or that some influence is disturbing the random behavior of the coin.

Our intuition doesn't do a good job of distinguishing random behavior from systematic influences. This is also true when we look at data. We need statistical inference to supplement exploratory analysis of data because probability calculations can help verify that what we see in the data is more than a random pattern.

How large is a large number?

The law of large numbers says that the actual mean outcome of many trials gets close to the distribution mean as more trials are made. It doesn't say how many trials are needed to guarantee a mean outcome close to . That depends on the variability of the random outcomes. The more variable the outcomes, the more trials are needed to ensure that the mean outcome is close to the distribution mean . Casinos understand this: the outcomes of games of chance are variable enough to hold the interest of gamblers. Only the casino plays often enough to rely on the law of large numbers. Gamblers get entertainment; the casino has a business.

Rules for means

Imagine yourself as a financial adviser who must provide advice to clients regarding how to distribute their assets among different investments such as individual stocks, mutual funds, bonds, and real estate. With data available on all these financial instruments, you are able to gather a variety of insights, such as the proportion of the time a particular stock outperformed the market index, the average performance of the different investments, the consistency or inconsistency of performance of the different investments, and relationships among the investments. In other words, you are seeking measures of probability, mean, standard deviation, and correlation. In general, the discipline of finance relies heavily on a solid understanding of probability and statistics. In the next case, we explore how the concepts of this chapter play a fundamental role in constructing an investment portfolio.

225

226

Think first not about investments but about making refrigerators. You are studying flaws in the painted finish of refrigerators made by your firm. Dimples and paint sags are two kinds of surface flaw. Not all refrigerators have the same number of dimples: many have none, some have one, some two, and so on. You ask for the average number of imperfections on a refrigerator. The inspectors report finding an average of 0.7 dimple and 1.4 sags per refrigerator. How many total imperfections of both kinds (on the average) are there on a refrigerator? That's easy: if the average number of dimples is 0.7 and the average number of sags is 1.4, then counting both gives an average of flaws.

In more formal language, the number of dimples on a refrigerator is a random variable that varies as we inspect one refrigerator after another. We know only that the mean number of dimples is . The number of paint sags is a second random variable having mean . (As usual, the subscripts keep straight which variable we are talking about.) The total number of both dimples and sags is another random variable, the sum . Its mean is the average number of dimples and sags together. It is just the sum of the individual means and . That's an important rule for how means of random variables behave.

Here's another rule. A large lot of plastic coffee-can lids has a mean diameter of 4.2 inches. What is the mean in centimeters? There are 2.54 centimeters in an inch, so the diameter in centimeters of any lid is 2.54 times its diameter in inches. If we multiply every observation by 2.54, we also multiply their average by 2.54. The mean in centimeters must be , or about 10.7 centimeters. More formally, the diameter in inches of a lid chosen at random from the lot is a random variable with mean . The diameter in centimeters is , and this new random variable has mean .

The point of these examples is that means behave like averages. Here are the rules we need.

At this stage, we only have part of the picture on the aggregated demand random variable—namely, its mean value. In Example 4.39 (pages 232–233), we continue our study of aggregated demand to include the variability dimension that, in turn, will reveal operational benefits from the proposed strategy of centralizing. Let's now consider the portfolio scenario of Case 4.3 (page 225) to demonstrate the use of a combination of the mean rules.

229

The variance of a random variable

The mean is a measure of the center of a distribution. Another important characteristic of a distribution is its spread. The variance and the standard deviation are the standard measures of spread that accompany the choice of the mean to measure center. Just as for the mean, we need a distinct symbol to distinguish the variance of a random variable from the variance of a data set. We write the variance of a random variable as . Once again, the subscript reminds us which variable we have in mind. The definition of the variance of a random variable is similar to the definition of the sample variance given in Chapter 1. That is, the variance is an average value of the squared deviation of the variable from its mean .

As for the mean of a discrete random variable, we use a weighted average of these squared deviations based on the probability of each outcome. Calculating this weighted average is straightforward for discrete random variables but requires advanced mathematics in the continuous case. Here is the definition.

Rules for variances and standard deviations

What are the facts for variances that parallel Rules 1, 2, and 3 for means? The mean of a sum of random variables is always the sum of their means, but this addition rule is true for variances only in special situations. To understand why, take to be the percent of a family's after-tax income that is spent, and take to be the percent that is saved. When increases, decreases by the same amount. Though and may vary widely from year to year, their sum is always 100% and does not vary at all. It is the association between the variables and that prevents their variances from adding.

If random variables are independent, this kind of association between their values is ruled out and their variances do add. As defined earlier for general events and (page 205), two random variables and are independent if knowing that any event involving alone did or did not occur tells us nothing about the occurrence of any event involving alone.

Probability models often assume independence when the random variable outcomes appear unrelated to each other. You should ask in each instance whether the assumption of independence seems reasonable.

When random variables are not independent, the variance of their sum depends on the correlation between them as well as on their individual variances. In Chapter 2, we met the correlation between two observed variables measured on the same individuals. We defined the correlation (page 75) as an average of the products of the standardized and observations. The correlation between two random variables is defined in the same way, once again using a weighted average with probabilities as weights in the case of discrete random variables. We won't give the details—it is enough to know that the correlation between two random variables has the same basic properties as the correlation calculated from data. We use , the Greek letter rho, for the correlation between two random variables. The correlation is a number between −1 and 1 that measures the direction and strength of the linear relationship between two variables. The correlation between two independent random variables is zero.

231

Returning to family finances, if is the percent of a family's after-tax income that is spent and is the percent that is saved, then . This is a perfect linear relationship with a negative slope, so the correlation between and is . With the correlation at hand, we can state the rules for manipulating variances.

Because a variance is the average of squared deviations from the mean, multiplying by constant multiplies by the square of the constant. Adding a constant to a random variable changes its mean but does not change its variability. The variance of is, therefore, the same as the variance of . Because the square of 21 is 1, the addition rule says that the variance of a difference between independent random variables is the sum of the variances. For independent random variables, the difference is more variable than either or alone because variations in both and contribute to variation in their difference.

As with data, we prefer the standard deviation to the variance as a measure of the variability of a random variable. Rule 2 for variances implies that standard deviations of independent random variables do not add. To work with standard deviations, use the rules for variances rather than trying to remember separate rules for standard deviations. For example, the standard deviations of and are both equal to because this is the square root of the variance .

If you buy a Pick 3 ticket, your winnings are because the dollar you paid for the ticket must be subtracted from the payoff. Let's find the mean and variance for this random variable.

Suppose now that you buy a $1 ticket on each of two different days. The payoffs and on the two tickets are independent because separate drawings are held each day. Your total payoff is . Let's find the mean and standard deviation for this payoff.

When we add random variables that are correlated, we need to use the correlation for the calculation of the variance, but not for the calculation of the mean. Here are two examples.

This example illustrates the important supply chain concept known as risk pooling. Many companies such as Walmart and e-commerce retailer Amazon take advantage of the benefits of risk pooling as illustrated by this example.

Example 4.40 illustrates the first step in modern finance, using the mean and standard deviation to describe the behavior of a portfolio. We illustrated a particular mix (70/30), but what is needed is an exploration of different combinations to seek the best construction of the portfolio.

235