“Moneyball” is on the outs. In 2002, the Oakland A’s used statistical analysis to assemble an unconventional but successful baseball team. Two decades later, my mother complains every time I see her that analytics (in general) and Minnesota Twins manager Rocco Baldelli (in particular) have ruined baseball. She’s hardly alone. D.K. Thompson writes in The Atlantic about data’s pernicious grip on the national pastime:

Smarties approached baseball like an equation, optimized for Y, solved for X, and proved in the process that a solved sport is a worse one…. 

There are two kinds of games in life: finite and infinite. A finite game is played to win; there are clear victors and losers. An infinite game is played to keep playing; the goal is to maximize winning across all participants. Debate is a finite game. Marriage is an infinite game. …[B]aseball’s finite game was solved so completely that the infinite game was lost.

Thompson suggests that moneyball has infected American culture more broadly, leading to dull, digestible art. We suffer, he argues, from “moneyball-for-everything,” a “living creature that consumes data and spits out homogeneity.”

Thompson’s argument is viscerally appealing. America has loved man-beats-machine stories since John Henry defeated the steam drill. We want to believe that humans are special, that we can see things computers just can’t, whether it’s a 60-year-old baseball scout who just knows a guy can’t run or Peggy Noonan sensing Mitt Romney’s victory in the air. Moneyball is the antithesis of all that. Moneyball says we aren’t special, that the determinism of math has annexed another region of once-human territory, and that our hard-earned experience and intuition actually hinder progress. Thompson’s opposition to moneyball is attractive because it is humane.

Thompson’s position is also attractive because it is obviously correct. Forget baseball’s plight; just sit through Doctor Strange. It’s less a movie and more a filmic attempt at Nozick’s Pleasure Machine. A good film critic often provides more entertainment than the billion-dollar China-optimized monsters churned out by actual studios.

Nowhere is moneyball’s influence plainer than in elections. Moneyball had to work to reduce art and baseball to numbers, but elections were already mostly numbers. Politicos now obsess over how best to manipulate us with Big Data, openly pursuing the “finite game” of winning the next election at the expense of the “infinite game” of cultivating a healthy electorate. From rigorous A/B testing of fundraising emails to polls tactically deployed as propaganda, moneyball is right at home in politics. A few years after left-leaning Nate Silver left the baseball predictions racket to found the modern election predictions racket, the right wing figured out how to combine computers with voter files to impose brutally effective gerrymanders on the 2010s. To us, the citizens supposedly served by our political class, it all feels cold, ugly, and hostile. Even some pollsters seem fed up. When talking heads mention “election models” or “polling aggregates,” it’s easy to imagine Americans making the same awful, guttural noise that my mother makes whenever someone says “Rocco Baldelli.”

We can’t stop moneyball, though.

There’s a Star Trek episode where the Enterprise is being pulled toward destruction in a bizarre “zone of darkness,” where the laws of physics don’t apply. The zone has already killed the logical Vulcan crew of another starship. 

Captain Kirk, desperate for options, asks Mr. Spock what the Vulcans might have tried. Spock shakes his head. The zone is too bizarre. Even as the Vulcans’ instruments showed them dying, Spock says, “Their logic would not have permitted them to believe they were being killed.”

Kirk asks, “And when they died, what do you think they felt?”

Spock raises one rueful eyebrow. “Astonishment.”

That’s what I imagine Republicans felt on Election Night 2022.

A red wave is coming, pundits said, and everyone believed. Republicans salivated, Democrats despaired, and pre-mortems went up on all the horse-race sites. Logically, it should have been a red wave. Midterms are good for the out-party. Inflation was up, presidential approval down, and voters were angry. Besides, didn’t you see those polls from Trafalgar, or the averages at RealClearPolitics? Whatever your party, it was impossible not to feel the vibes: A red wave is coming. Logic permitted nothing else.

A close look at our instruments, though, showed plenty of flashing warning lights. Polls had Republicans struggling throughout the summer. Republicans enjoyed a modest recovery in mid-October, but that recovery stopped just a week later. Although polls-based models agreed that the GOP would probably win the House, evidence was mixed. It was more likely than not that Republicans would land between 218 seats (a bare majority) and 234 (still short of a wave). For all the talk of “waves,” or even “tsunamis,” the models always considered a Blue Congress about as likely as any “red wave.” (My own election night preview said as much.)

The election moneyballers went on to have a great night. Republicans won 222 seats. In an average election, the polls “miss” by an average of 2.28 points in favor of one party. Both the exact size of the miss and the party it will boost are unforeseeable, a key source of election uncertainty. (The “miss” in 2020 was large, almost 5 points.) We don’t yet know the exact polling “miss” for 2022, but, at this writing, the “miss” appears to have been under 1 point, comparable to polling’s strong performance in 2018. Republicans comfortably won the popular vote, but that didn’t translate to many seats, thanks to weak candidates in competitive elections, effective gerrymandering by Democrats, and substantial Republican gains among Hispanic and Black voters who reside in safe Democrat districts. Anyone looking closely could have seen these complex, sometimes counterintuitive currents and concluded that a wave was unlikely. The computer models did exactly that.

The polling was clear, but our logic did not permit us to believe it. We blithely cherry-picked our way to seeing a red wave. Then, as Election Night unfolded, we were astonished.

If this were a one-off defeat for human intuition, that would be one thing… but it wasn’t. Conventional wisdom believed Secretary Clinton would win the 2016 presidential election in a landslide, even though the evidence showed her in a close race that Trump could easily win. Result: astonishment. Conventional wisdom believed the U.K. would reject Brexit, even though the evidence showed a tight race in which Brexit had momentum. Result: astonishment. When conventional wisdom thinks the polls are wrong, bet that polls are indeed wrong… in the opposite direction.

I’m no different. Donald Trump won his first primary poll in 2011. I could not even conceive of a Trump supporter in my experience of conservatism. I confidently informed friends that Republican voters were playing a prank on left-leaning pollsters. After Trump announced his 2016 run, he won pluralities in virtually every poll. Then, to my astonishment, he won the primary. The media does a poor job authentically identifying and profiling Trump fans, so, to this day, my main contact with Trump’s earliest supporters comes through polls. 

This is true of many slices of America. I’m from Minnesota; I don’t know as many Black people as I would like. Thanks to polls, I know they’re one of the more pro-life blocs in the Democratic coalition, which is not what I would guess from watching Black spokesmen and characters on television. Polls tell me that the vast majority of my fellow practicing Catholics still trust the bishops, which floored me. Polls tell me that many people who say masks work don’t believe it, and that America’s most popular scripted broadcast television show—a matter of no small cultural importance!—is (as usual) a show I’ve never seen, that nobody I know watches, and that critical buzz never even discusses. The Dick Wolf Cinematic Universe is as invisible to me as the Dark Matter Universe, except in polls. 

Polls, and the forecasting models we build with them, have to be interpreted cautiously, but they are an essential technology for breaking our bubbles. Polls are sometimes our only line of communication with the vast swaths of America not well-represented in the kinds of intelligent, sophisticated magazines you and I read. I often think of the so-called “chattering class” as sitting atop the tiny peak of a vast, deep iceberg, loudly debating where the iceberg is going, when we don’t even know what our iceberg looks like below the waterline. We often forget anything below the waterline exists. Polls are tiny, uncertain, vital soundings from miles below. We must especially listen to those soundings when they clash with our intuitions. A more data-driven assessment could have saved many of my friends from heartburn on Election Night ’22.

The most frustrating thing about the moneyball narrative, then, is that it’s basically correct: quantitative analysis and modeling trounce human intuition nine times out of ten. People use it in baseball because it provides information that wins games. People use it in politics because it provides information that wins elections. Some of them can’t figure out how to use these tools effectively (alas, Rocco), but enough succeed that we will never escape moneyball.

Perhaps we don’t need to. Americans love a man-beats-machine story, but what Americans love even more is a man-invents-machine-and-beats-the-tar-out-of-his-dinosaur-competitors-with-it story. Hence our long fascination with Henry Ford, Steve Jobs, and, yes, Billy Beane. I don’t think my mother really minds that a bunch of nerds vacuumed up every number in baseball. She minds that the games are boring! It’s important to remember, then, that the analysis itself didn’t make the games boring. Analysis doesn’t change anything.

Baseball changed because the people doing the analysis had the wrong goal: they wanted to win games. Their metrics helped them achieve that goal. However, their goal should have been to make baseball fun to watch. Doing that would have required all the same metrics. Indeed, they would have had to invent even more metrics! With a different objective, though, the metrics would have led to very different decisions. I would bet that “the shift” would have solved itself much faster, and that managers would be more willing to allow their pitchers to throw complete games than they are today. 

Baseball should have valued its “infinite game” above its “finite game.” It’s too easy to blame the computers for successfully refining the finite game, rather than the computers’ operators for prioritizing the wrong game. The incentives pushing teams to prioritize their own success over the success of Major League Baseball may be difficult to unravel, but that is a problem for Conquest’s Third Law, not front-office statisticians.

So, too, with polls. I believe we distrust polls, not because we resent opportunities to better understand our country, but because many of the people who present polls have a short-term agenda, a “finite game.” The aim of greater understanding is often subordinate to those agendas. Poll questions are routinely manipulated to generate specific results, even from theoretically non-partisan firms. Survey-based social science research so often violates basic rules of sampling that you must regard the entire field with suspicion. Horse-race polls are selectively misinterpreted by campaigns. Election modelers may harbor unconscious biases that favor their “team.” The journalists who report on all this are often functionally innumerate. The metrics are sound. Too often, however, they are being put to counterproductive uses—and then blamed for the sins of their abusers.

The average reader can do little about the sins of irresponsible journalists. There are only a handful of analysts I trust to report the straight dope on polls, and most of them are named Nate. They can help us use polls and models effectively. Then you, too, can not only predict election results, but understand the deep national dynamics that bring those results about.

By subordinating the finite game of winning the next vote, we can turn these powerful tools toward the infinite game of forming a more perfect union. Despite our deep and real divisions, it is easy to imagine that an American public that really understood itself, with all its invisible complexity and unexpected contradictions, would be less acrimonious, more capable of compromise, and maybe – just maybe – wise enough to move the pitcher’s mound back two feet.