So, was there a ‘Bradley effect’? (Answer: No)

The U.S. presidential election is over. Was there a ‘Bradley effect’? To find out, I’ll compare the final day’s polling data against the election outcomes for some key states.

In a previous post I looked at the ‘Bradley effect’, a systematic polling error where opinion polls overestimate the vote share of black candidates in U.S. elections. A recent comparison of polling data against election outcomes concluded that the Bradley effect doesn’t exist anymore ^[1]. With the U.S. presidential election results coming in, let’s see if there was any sign of it here.

State of Play On November 4:  Final Polling Data

Table 1.  Final poll aggregates

Poll	Margin
Real Clear Politics	Obama +7.3%
Princeton E.C.	Obama +7.0%
FiveThirtyEight	Obama +6.0%
Poll data on the morning of Nov. 4, 2008 for the U.S. presidential election.

It’s U.S. election day, November 4, and I’m writing at 11:50 am GMT. I’ll freeze the final aggregated poll results here, to avoid any temptation to be wise after the event. Table 1 shows averaged polls as they stand on the morning of the election, taken from Real Clear Politics ^[2], the Princeton Election Consortium ^[3] and FiveThirtyEight ^[4]. The Princeton Election Consortium quotes a  +7.0 ± 0.8% point lead for Obama. The error bar is far smaller than the lead. The opinion polls are certainly making a very clear prediction.

State of Play On November 5:  The Election Results

Right, one day later, I’m writing on November 5, the election’s over, and Barack Obama is the United States’ first African-American president.

The final opinion poll averages were predicting an Obama victory by a margin of 6%–7%. At 3 pm GMT, with nearly all votes counted, the election result stands at Obama 52.4%, McCain 46.3%, Others 1.1% ^[5]. Obama’s margin is  6.1% . Given the typical ±1% error on the aggregated polls, the polls were spot on. There was no Bradley effect.

Digging one layer deeper into the detail, here are the state-by-state polling data and results for the key “battleground states”. The polling data are the final day’s averages from Real Clear Politics ^[6].

The “poll error” shown in Table 2 is Obama’s predicted margin minus his actual margin in the election. That is, a negative poll error indicates that the polls underestimated Obama’s vote margin.

Table 2.  Final polling data and election results for the key battleground states

	Final poll averages from Real Clear Politics			Election result
State	Obama	McCain	RCP Average	Obama	McCain	Margin	Poll error
Florida	49.0%	47.2%	Obama +2%	51%	49%	Obama +2%	0%
Pennsylvania	51.0%	43.7%	Obama +7%	55%	44%	Obama +11%	−4%
Ohio	48.8%	46.3%	Obama +3%	51%	47%	Obama +4%	−1%
North Carolina	48.0%	48.4%	McCain +0.4%	50%	49%	Obama +1%	−1%
Virginia	50.2%	45.8%	Obama +4%	52%	47%	Obama +5%	−1%
Indiana	46.4%	47.8%	McCain +1%	50%	49%	Obama +1%	−2%
Minnesota	51.6%	41.8%	Obama +10%	54%	44%	Obama +10%	0%
Colorado	50.8%	45.3%	Obama +6%	53%	46%	Obama +7%	−1%
Iowa	54.0%	38.7%	Obama +15%	54%	45%	Obama +9%	+6%
Nevada	50.3%	43.8%	Obama +7%	55%	43%	Obama +12%	−5%
New Mexico	50.3%	43.0%	Obama +7%	57%	42%	Obama +15%	−8%
New Hampshire	52.8%	42.2%	Obama +11%	55%	44%	Obama +11%	0%
Polling data are from Real Clear Politics [6]. The “poll error” is Obama’s poll margin minus his election margin.

The averaged polls were mostly right to within about 1 percentage point. Looking at the largest errors, Iowa’s polling overestimated Obama’s margin by 6 percentage points, but on the other hand New Mexico’s polling underestimated his election margin by 8 points. If anything, there’s a slight underestimate of Obama’s margin in most of the polls.

The opinion pollsters got it right. There was no Bradley effect. Is this because voters have stopped lying to pollsters about their voting intentions, or is it because the pollsters have worked out how to correct for this systematic error? It’s impossible to say from this information alone, but either way, the pollsters came out of it pretty well. Aggregated polling data looks reliable.

Update:  Nate Silver At TED

Nate Silver of the poll compilations website fivethirtyeight.com gave a talk at TED about “Picking apart the puzzle of racism in elections”:

Nate Silver: Picking apart the puzzle of racism in elections

February 2009

References

1. No More Wilder Effect, Never a Whitman Effect:  When and Why Polls Mislead about Black and Female Candidates, Daniel J. Hopkins, Department of Government, Harvard University, August 4, 2008  (preprint)  (WebCite cache)
2. General Election: McCain vs. Obama, Real Clear Politics, November 4, 2008  (WebCite cache)
3. Final predictions for 2008, Sam Wang, Princeton Election Consortium, November 4, 2008  (WebCite cache)
4. Today’s Polls, 11/3 (PM Edition), Nate Silver, FiveThirtyEight.com, November 3, 2008  (WebCite cache)
5. US election results map, BBC News, November 5, 2008
6. Latest Presidential Polls, Real Clear Politics, Guardian campaign, November 5, 2008