Richard Epstein is right about how to think about antidiscrimination law. The general principle governing transactions between private parties should be freedom of association, for reasons of both liberty and efficiency. Any departure from that rule, such as a prohibition of discrimination, has the burden of proof.
Epstein, however, can’t let go of that general rule when he should. He is strangely myopic about the cultural forces that prevent free markets from delivering the goods.
Antidiscrimination law has important economic and noneconomic goals. The economic one is simplest: It remedies the chronic subordination of some groups, thereby creating a society with broader opportunities. Beyond the economic, however, it can dismantle longstanding structures of dominance and subordination that prevent the state from guaranteeing even those rights that Epstein so admirably wants to protect.
The most basic of antidiscrimination law’s purposes is the amelioration of persistent economic inequality. A central purpose of the Civil Rights Act of 1964 was to reduce black poverty, by making well-paying positions available to black workers.
It succeeded. In 1964, the median income of nonwhite males was 57 percent of median white male income. By 1985, that proportion had risen to 66 percent. The proportion of black men working as professionals or managers relative to whites rose from 32 percent to 64 percent. The most dramatic progress came in the first 10 years after the Act.
Some proposed extensions seem likely to have similar effects. For example, discrimination against gay people causes similar economic harm.
The general rule that governs business transactions, both public accommodation and employment, is contract at will. In most states, most businesses have the privilege of refusing service to anyone for any reason or no reason. They need not justify these actions to any official. Antidiscrimination laws, such as the Civil Rights Act, are exceptions. So long as an economic actor does not engage in the enumerated types of discrimination, she has the privilege of being as arbitrary as she likes. I can, for example, absolutely refuse to hire or do business with anyone whose eyebrows are not at least three inches long.
It is important to understand the reasons for the rule of contract at will, which Epstein lays out so well, so that we can understand what we are doing when we depart from that rule. One traditional justification is rights-based: People have a right, it is sometimes said, to do what they like with their private property. The bankruptcy of this justification became clear during the debate over the Civil Rights Act of 1964, which then-presidential candidate Barry Goldwater opposed on libertarian grounds.
The Civil Rights Act is not an invasion of our precious liberties. On the contrary, it diminishes the amount of oppression in the world. The idea of private property is not as sacrosanct as it once was, because it is understood that the uses of that property can have public effects that are legitimate objects of legislative concern. Even Goldwater eventually abandoned the libertarian argument and supported antidiscrimination protection for gay people.
The more persuasive justification for the rule of employment at will is efficiency-based. It would be a crushing burden on the economy for government officials to have to approve every refusal of a contract that takes place in the private sector. Moreover, there is little reason to think that most types of arbitrary refusal can have much effect on anyone’s opportunities.
Although I may refuse to hire anyone whose eyebrows are less than three inches long, other employers will compete for the services of the short-eyebrowed, and so will bid their wages up to pretty much the same level that they would have been if I had been willing to hire them. And the market will also punish me for my foolishly discriminatory hiring practices, since competent short-eyebrowed workers will go to work for my competitors. My taste for discrimination means that I am turning away better workers and hiring worse ones. I will be punished in the same way if I arbitrarily turn away customers. The overall tendency is for people like me to be driven out of the market.
Considerations of this sort led Epstein to argue that the Civil Rights Act ought to be repealed, because it interfered with freedom of contract for no good reason. In a free market, he argued, we can expect that blacks’ wages (for instance) will be about as high as they would be if there were no discrimination.
Epstein did not persuade many people. The point most commonly made by his critics was that he had left culture out of his model. Some groups are subject to pervasive discrimination. At least when the Civil Rights Act was enacted, his critics argued, racism was sufficiently ubiquitous to withstand the egalitarian tendencies of a well-functioning free market. Economic equality can be achieved even if there is discrimination, indeed even if there is a lot of discrimination, so long as the discriminators are a minor part of the market as a whole. (That’s why it’s a mistake, as I have pointed out elsewhere, to think of the tort of discrimination as an individualized injury rather than an artifact of social engineering.) Sometimes, however, they dominate the market, and then legal intervention is justified. Economic models should take account of this reality.
He resists another, deeper purpose of antidiscrimination law: It helps reshape culture in order to eliminate patterns of stigma and prejudice that constitute some classes of persons as inferior members of society. Habits of discrimination are hard to break, and legal intervention can help to break them. It also stigmatizes racism. That stigma has had powerful cultural effects. Racial attitudes in America are not nearly as bad as they were in 1964. The breaking of the color barrier in so many professions has helped to bring that about.
Epstein has written that “the best way to take into account the full range of symbols, good and bad, noble and vain, is for the legal system to ignore them all—mine and yours alike.” Ignoring racism and sexism, however, will prevent the realization of even the basic rights that Epstein is most eager to protect.
One of the attractions of economic analysis is that it assumes the equal worth of everyone. The happiness of Lord Grantham isn’t given any more weight than that of his servant. It is, happily, an idea that is often unreflectively taken for granted. But it is a contingent cultural formation, the product of a long and bitter struggle that isn’t over yet. Economic analysts should be willing to incur some costs in order to promote the hegemony of economic analysis!
Prejudice, if it is sufficiently deeply ingrained in a society, will prevent—it sometimes has prevented—the state from even providing police protection on an equitable basis. That is bad news for individual liberty. Epstein thinks that the racism of Southern law-enforcement authorities explains the crushing effect of discrimination before passage of the Civil Rights Act. But unless culture changes, what hope is there of improving the behavior of the police? And of course the police are not the only agents of the state whose actions are likely to be contaminated by racism.
Once these legitimate purposes are taken into account, it is not possible to govern our complex world with simple rules. Judgments must be made about when the unregulated market is failing—when antidiscrimination law is justified. Those judgments can go wrong, of course, and Epstein can point to (what he regards as) horror stories and say that we should never have gone down this road to begin with. But anything worth doing will sometimes be done badly. The fact that it is sometimes done badly does not prove that it isn’t worth doing.
 James J. Heckman and J. Hoult Verkerke, “Racial Disparity and Employment Discrimination Law: An Economic Perspective,” Yale Law and Policy Review 8 (1990), 276, 281.
The economic consequences of antigay discrimination are documented, with citations to the scholarship on the topic, in the report accompanying the Employment Non-Discrimination Act of 2013, Senate Report 113-105 (2013), at 14-19.
 Nan D. Hunter, “Accommodating the Public Sphere: Beyond the Market Model,” Minnesota Law Review 85 (2001), 1591, 1617-18. This may be a departure from preexisting common law rules. See Joseph William Singer, “No Right to Exclude: Public Accommodations and Private Property,” Northwestern University Law Review 90 (1996), 1283 (1996). It is now the law in most places, however. See, for example, Feldt v. Marriott Corp., 322 A.2d 913, 915 (D.C. 1974).
 For arguments of this sort, see Ayn Rand, The Virtue of Selfishness: A New Concept of Egoism (New American Library, 1964), pp. 126-34, and also Michael Levin, “Negative Liberty,” Social Philosophy and Policy 2 (1984), 84, 98-100.
 See Rick Perlstein, Before the Storm: Barry Goldwater and the Unmaking of the American Consensus (Hill and Wang, 2001), pp. 363-64. On p. 462 Perlstein quotes from a Goldwater speech, coauthored by William Rehnquist, declaring that “the freedom to associate means the same thing as the freedom not to associate.”
 More generally, property rights are created by the law for reasons and should not be defined in a way inconsistent with those reasons. See Liam Murphy and Thomas Nagel, The Myth of Ownership: Taxes and Justice (Oxford University Press, 2002) and Jeremy Waldron, The Right to Private Property (Clarendon Press, 1988).
 See Barry M. Goldwater, “The Gay Ban: Just Plain Un-American,” Washington Post, June 10, 1993.
 See generally Richard Epstein, Forbidden Grounds: The Case Against Employment Discrimination Laws (Harvard University Press, 1992). For the point about wages, see pp. 28-58.
 See Samuel Issacharoff, “Contractual Liberties in Discriminatory Markets,” Texas Law Review 70 (1992), 1219, 1242-43 (book review). See also a symposium on Richard Epstein’s Forbidden Grounds: The Case Against Employment Discrimination Laws, with critiques by Ian Ayres, Drew Days, Evan Tsen Lee, Jerry L. Mashaw, and Richard H. McAdams, San Diego Law Review 31 (1994), 1-277. John J. Donohue III similarly responds to neoclassical economic attacks on antidiscrimination law in “Is Title VII Efficient?”, University of Pennsylvania Law Review 134 (1986), 1411, 1415-19. I offer a similar defense of sex discrimination law, against Epstein’s criticisms, in “Feminism and Libertarianism: A Response to Richard Epstein,” University of Chicago Legal Forum, Volume 1999, p. 115.
 Forbidden Grounds, pp. 498-99.
 I elaborate on this purpose of antidiscrimination law in Antidiscrimination Law and Social Equality (Yale University Press, 1996).