Unfortunately, exit polling is becoming less reliable each election. Its history in this decade has been ignominious.
In the 2002 midterm elections, the exit polls weren’t published because of a software foul-up. (In 2003, I purchased the raw data and crunched the 2002 numbers so they wouldn’t be lost to history.)
In 2004, the exit polls predicted a narrow Kerry victory. In addition, they initially reported that Bush had garnered 44 percent of the Hispanic vote. After I pointed out how unlikely that was, the polling company announced weeks later that the number should have been about 40 percent. (And keep in mind that Bush only got to 40 percent via his Housing Bubble, which poured hundreds of billions of dollars into the pockets of Hispanic homebuyers and construction workers.)
In 2008, the lone exit poll predicted an Obama landslide. Karl Rove complained right after this election:
“We can’t be precise, because for the third election in a row the exit polls were trash. The raw numbers forecast an 18-point Obama win, news organizations who underwrote the poll arbitrarily dialed it down to a 10-point Obama edge, and the actual margin was six. [Actually, closer to seven than to six, it looks now.]“[How the President-Elect Did It , by Karl Rove, WSJ, November 6, 2008]
Why are exit polls so bad in this decade?
One problem is that there is more early voting and more mail-in voting each election. In 2008, there was also likely to be a large Bradley Effect in which intimidated Republican voters offer politically correct answers to the young, Democratic-looking pollsters who accost them after voting.
Nevertheless, the most fundamental problem is one that’s common in the marketing research industry, where I worked for many years: it has become a monopoly.
There’s an old saying in the marketing research business that in viable industry segment, there’s only room for 1.5 firms. You’ll notice, for example, that Nielsen doesn’t have any competition for TV ratings and Arbitron doesn’t have any competition for radio ratings. They could enter each other’s field, but then they’d both lose money in both fields. Why ruin nice little monopolies? In contrast, in the supermarket sales data field, there have long been two competitors, with rapid technological advancements resulting. That little industry, however, was long notorious among investors for generating terrible profit margins due to a decade-long price war between the two rivals.
Back in 2000, there were three national exit polls, one sponsored by a group of media outlets (which I’ll call the CNN poll for the convenience of its website), one by the New York Times, and one by the Los Angeles Times. They came up with different figures for the GOP share of the Hispanic vote: 31 percent according to the NYT, 35 percent according to CNN and its colleagues, and 38 percent according to the LAT.
This fuzzy math had the dual benefits of keeping you from being too stridently confident about the results (“Well, all we can say is the real number was likely somewhere in the 30s”) while letting you triple-check your numbers (“Yes, although we can’t be sure, 35 percent sounds like a reasonable estimate.”)
Over the course of the decade, unfortunately, the individual newspapers dropped out of the business. The cartel’s poll has wound up as a monopoly, with the usual results in terms of quality and reliability. Without competition to spur them on, they usually do a bad job.
It’s particularly important to understand that exit polls are not a very good way to determine an ethnic group’s share of the vote. There are all sorts of articles exulting over the huge turnout of Hispanics last Tuesday, but they all seem to reference the exit poll rather than real world results. A huge chunk of Hispanic voters are in California and Texas, both states in which there was little campaigning, advertising, or canvassing because they were all wrapped up.
The CNN exit poll has a long history of exaggerating the Hispanic share of the vote in contrast to the gold standard Census Bureau phone survey of 50,000 households that is conducted immediately after each election but not released until the following year. In 2000, the CNN and friends exit poll reported Hispanics made up 7 percent. The Census Bureau said 5.4 percent. In 2004, CNN said 8 percent, the Census 6.0 percent. In 2008, CNN said 9 percent, while, I’m guessing, based on trends going back to the 1970s, that the Census Bureau will eventually report the 2008 Hispanic fraction as a little under 7.0 percent.
It’s worth noting that this year’s much-publicized 9 percent figure for Hispanic’s share of the vote is from the exit poll’s smaller “national sample.” The blogger Audacious Epigone toted up the figures from the exit poll’s much larger “state sample” and came up with 7.54 percent, which sounds more plausible.
In general, exit polls aren’t very good at figuring out turnout shares. If you stop and think about what’s involved in running a national exit poll, you can grasp why.
Only a tiny fraction of all the polling places in the country are covered, so the polling company has to decide ahead of time where to send their pollsters. That isn’t a big problem for calculating, say, the female share of the vote, because males and females generally live in the same neighborhoods. However, racial groups frequently don’t live in the same neighborhoods. Thus, the polling firm has to choose carefully which neighborhoods to survey in order to get the “right” number of voters from a particular group.
Therefore, long before the election, the polling company must come up with an estimate of each group’s expected share in order to decide which polling stations to cover. This prediction tends to become a self-fulfilling prophecy. The firm’s thinking may go something like this: “Okay, we said the Hispanic share last time was 8 percent, and everybody knows they are growing, so we’d better report Hispanics as 9 percent this time, or we’ll look bad. So, let’s figure out which neighborhoods to send pollsters to in order that 9 percent of the voters they interview are Hispanic.”