Markets do very well at dealing with diversifiable risk and with risk due to uncertainty: situations in which nobody knows what is going to happen, whose house will be flooded, or who will get sick. However, markets have much more trouble with situations in which some people know things that other people don’t know—situations of private information. As we will see, private information can distort economic decisions and sometimes prevent mutually beneficial economic transactions from taking place. (Sometimes economists use the term asymmetric information rather than private information, but they are equivalent.)

Why is some information private? The most important reason is that people generally know more about themselves than other people do. For example, you know whether or not you are a careful driver; but unless you have already been in several accidents, your auto insurance company does not. You are more likely to have a better estimate than your insurance company of whether or not you will need an expensive medical procedure. And if you are selling me your used car, you are more likely to be aware of any problems with it than I am.

But why should such differences in who knows what be a problem? It turns out that there are two distinct sources of trouble: adverse selection, which arises from having private information about the way things are, and moral hazard, which arises from having private information about what people do.

Suppose that someone offers to sell you an almost brand-new car—purchased just three months ago, with only 2,000 miles on the odometer and no dents or scratches. Will you be willing to pay almost the same for it as for a car direct from the dealer?

Probably not, for one main reason: you cannot help but wonder why this car is being sold. Is it because the owner has discovered that something is wrong with it—that it is a “lemon”? Having driven the car for a while, the owner knows more about it than you do—and people are more likely to sell cars that give them trouble.

You might think that the fact that sellers of used cars know more about them than the buyers do represents an advantage to the sellers. But potential buyers know that potential sellers are likely to offer them lemons—they just don’t know exactly which car is a lemon. For this reason, buyers will offer a lower price than they would if they had a guarantee of the car’s quality. And this poor opinion of used cars tends to be self-reinforcing, precisely because it depresses the prices that buyers offer. Used cars sell at a significant discount because buyers expect a disproportionate share of those cars to be lemons.

Even a used car that is not a lemon would sell only at a large discount because buyers don’t know whether it’s a lemon or not. But potential sellers who have good cars are unwilling to sell them at a deep discount, except under exceptional circumstances. So good used cars are rarely offered for sale, and used cars that are offered for sale have a strong tendency to be lemons. (This is why people who have a compelling reason to sell a car, such as moving overseas, make a point of revealing that information to potential buyers—as if to say “This car is not a lemon!”)

The end result, then, is not only that used cars sell for low prices and that there are a large number of used cars with hidden problems. Equally important, many potentially beneficial transactions—sales of good cars by people who would like to get rid of them to people who would like to buy them—end up being frustrated by the inability of potential sellers to convince potential buyers that their cars are actually worth the higher price being asked. So some mutually beneficial trades between those who want to sell used cars and those who want to buy them go unexploited.

Although economists sometimes refer to situations like this as the “lemons problem,” the more formal name of the problem is adverse selection. The reason for the name is obvious: because the potential sellers know more about the quality of what they are selling than the potential buyers, they have an incentive to select the worst things to sell.

Adverse selection does not apply only to used cars. It is a problem for many parts of the economy—notably for insurance companies, and most notably for health insurance companies.

Suppose that a health insurance company were to offer a standard policy to everyone with the same premium. The premium would reflect the average risk of incurring a medical expense. But that would make the policy look very expensive to healthy people, who know that they are less likely than the average person to incur medical expenses. So healthy people would be less likely than less healthy people to buy the policy, leaving the health insurance company with exactly the customers it doesn’t want: people with a higher-than-average risk of needing medical care, who would find the premium to be a good deal.

In order to cover its expected losses from this sicker customer pool, the health insurance company is compelled to raise premiums, driving away more of the remaining healthier customers, and so on. Because the insurance company can’t determine who is healthy and who is not, it must charge everyone the same premium, thereby discouraging healthy people from purchasing policies and encouraging unhealthy people to buy policies.

As we discussed in Chapter 18, before the passage of the Affordable Care Act, adverse selection could lead to a phenomenon called an adverse selection death spiral as the market for health insurance collapsed: insurance companies refused to offer policies because there was no premium at which the company could cover its losses. Because of the severe adverse selection problems, governments in many advanced countries have assumed the role of providing health insurance to their citizens. In the United States, adverse selection in health insurance is avoided in two ways. First, U.S. government insurance programs, which provided almost half of the total payments for medical care in the United States in 2014, are financed by dedicated taxes which people cannot opt out of. Second, the ACA requires that everyone have health insurance, so healthy people cannot opt out of paying premiums.

However, adverse selection still exists in other insurance markets such as auto insurance. In general, people or firms faced with the problem of adverse selection follow one of several well-established strategies for dealing with it. One strategy is screening: using observable information to make inferences about private information. If you apply to purchase auto insurance, you’ll find that the insurance company will ask about your driving record in an attempt to “screen out” unsafe drivers—people they will refuse to insure or will insure only at very high premiums.

Auto insurance companies provide a very good example of the use of statistics in screening to reduce adverse selection. They may not know whether you are a careful driver, but they have statistical data on the accident rates of people who resemble your profile—and use those data in setting premiums. A 19-year-old male who drives a sports car and has already had a fender-bender is likely to pay a much higher premium than a 40-year-old female who drives an SUV and has never had an accident.

In some cases, this may be unfair: some adolescent males are very careful drivers, and some women drive SUVs as if they were F-16’s. But nobody can deny that the insurance companies are right on average.

Another strategy to counter the problems caused by adverse selection is for people who are good prospects to do something signaling their private information—taking some action that wouldn’t be worth taking unless they were indeed good prospects. For example, reputable used-car dealers often offer warranties—promises to repair any problems with the cars they sell that arise within a given amount of time. This isn’t just a way of insuring their customers against possible expenses; it’s a way of credibly showing that they are not selling lemons. As a result, more sales occur and dealers can command higher prices for their used cars.

Finally, in the face of adverse selection, it can be very valuable to establish a good reputation: a used-car dealership will often advertise how long it has been in business to show that it has continued to satisfy its customers. As a result, new customers will be willing to purchase cars and to pay more for that dealer’s cars.

In the late 1970s, New York and other major cities experienced an epidemic of suspicious fires that appeared to be deliberately set. Investigators eventually became aware of patterns in a number of the fires. Particular landlords who owned several buildings seemed to have an unusually large number of their buildings burn down. Although it was difficult to prove, police suspected that most of these fire-prone landlords were hiring professional arsonists to torch their own properties.

Why burn your own building? These buildings were typically in declining neighborhoods, where rising crime and middle-class flight had led to a decline in property values. But the insurance policies on the buildings were written to compensate owners based on historical property values, and so would pay the owner of a destroyed building more than the building was worth in the current market. For an unscrupulous landlord who knew the right people, this presented a profitable opportunity.

The arson epidemic became less severe during the 1980s, partly because insurance companies began making it difficult to overinsure properties, and partly because a boom in real estate values made many previously arson-threatened buildings worth more unburned.

The arson episodes make it clear that it is a bad idea for insurance companies to let customers insure buildings for more than their value—it gives the customers some destructive incentives. You might think, however, that the incentive problem would go away as long as the insurance is no more than 100% of the value of what is being insured.

But, unfortunately, anything close to 100% insurance still distorts incentives—it induces policyholders to behave differently than they would in the absence of insurance. The reason is that preventing fires requires effort and cost on the part of a building’s owner. Fire alarms and sprinkler systems have to be kept in good repair, and fire safety rules have to be strictly enforced. All of this takes time and money that the owner may not find worth spending if the insurance policy will provide close to full compensation for any losses.

Of course, the insurance company could specify in the policy that it won’t pay if basic safety precautions have not been taken. But it isn’t always easy to tell how careful a building’s owner has been—the owner knows, but the insurance company does not.

The point is that the building’s owner has private information about his or her own actions, about whether he or she has really taken all appropriate precautions. As a result, the insurance company is likely to face more claims than if it were able to determine exactly how much effort a building owner exerts to prevent a loss. The problem of distorted incentives arises when an individual has private information about his or her own actions but someone else bears the costs of a lack of care or effort. This is known as moral hazard.

To deal with moral hazard, it is necessary to give individuals with private information some personal stake in what happens so they have a reason to exert effort even if others cannot verify that they have done so. Moral hazard is the reason salespeople in many stores receive a commission on sales: it’s hard for managers to be sure how hard the salespeople are really working, and if they were paid only a straight salary, they would not have an incentive to exert effort to make those sales.

Insurance companies deal with moral hazard by requiring a deductible: they compensate for losses only above a certain amount, so that coverage is always less than 100%. The insurance on your car, for example, may pay for repairs only after the first $500 in loss. This means that a careless driver who gets into a fender-bender will end up paying $500 for repairs even if he is insured, which provides at least some incentive to be careful and reduces moral hazard.

In addition to reducing moral hazard, deductibles provide a partial solution to the problem of adverse selection. Your insurance premium often drops substantially if you are willing to accept a large deductible. This is an attractive option to people who know they are low-risk customers; it is less attractive to people who know they are high-risk—and so are likely to have an accident and end up paying the deductible. By offering a menu of policies with different premiums and deductibles, insurance companies can screen their customers, inducing them to sort themselves out on the basis of their private information.

As the example of deductibles suggests, moral hazard limits the ability of the economy to allocate risks efficiently. You generally can’t get full (100%) insurance on your home or car, even though you would like to buy it, and you bear the risk of large deductibles, even though you would prefer not to. The following Economics in Action illustrates how in some cases moral hazard limits the ability of investors to diversify their investments.