Polls And Their Samples: Verify The Party ID
If you want a good example of why you should take a close look at polls before accepting what they reflect, I point you to the much-ballyhooed Newsweek polls of the last couple of weeks. You’ll remember that last week the Newsweek and Time magazine polls got a lot of post-convention coverage because they both showed the Bush/Cheney ticket getting a huge double-digit bounce from the convention, jumping out to an 11-point lead over Kerry/Edwards (54%-43%). Several days after the media damage had been done to the political viability of the Kerry/Edwards ticket, various commenters noted that in both polls, especially the Newsweek poll, the sample used was weighed disproportionately towards GOP respondents (374 Republicans, only 303 Democrats, and 300 independents). When weighed to reflect the actual makeup of our current electorate, the results in both polls would have shown a tie or Bush lead within the margin of error, thereby indicating little if any bounce, something that Gallup confirmed earlier last week.
So imagine my surprise when the latest Newsweek poll came out yesterday, using the same pollster, and showing that in a two-way race with registered voters Bush/Cheney now have only a 50%-45% lead over Kerry/Edwards. If you were to believe this poll and the last one, an 11-point Bush margin collapsed to only a five-point Bush margin in a week. What changed in the last week to merit such a drop, aside from the flap over Bush’s National Guard service, (where it has been clearly established that Bush did not fulfill his guard commitments but somehow managed to get an honorable discharge)? And should you believe this poll any more than the last Newsweek Poll?
No. Why? Take a look at the sample size of this latest Newsweek poll and remember this poll showed a five-point Bush/Cheney lead amongst registered voters over Kerry/Edwards:
What? So we’re supposed to believe that if you pulled a qualified sample of 1003 American registered voters that 39% of them would be identified as Republicans and only 30% would be identified as Democrats? And this would be considered a qualified sample? This would come as news to polling experts like Harris, who as late as this February had reported a five-point lead in Democratic self-identification over GOP identification. Yet the Time and Newsweek pollsters use a sample weighted significantly to the GOP’s benefit as somehow representative of the voting populace?
John Zogby himself commented on the problems with Newsweek’s weighting:
Two new polls came out immediately after mine (as of this writing) by the nation's leading weekly news magazines. Both Time's 52% to 41% lead among likely voters and Newsweek's 54% to 43% lead among registered voters give the President a healthy 11 point lead. I have not yet been able to get the details of Time's methodology but I have checked out Newsweek's poll. Their sample of registered voters includes 38% Republican, 31% Democrat and 31% Independent voters. If we look at the three last Presidential elections, the spread was 34% Democrats, 34% Republicans and 33% Independents (in 1992 with Ross Perot in the race); 39% Democrats, 34% Republicans, and 27% Independents in 1996; and 39% Democrats, 35% Republicans and 26% Independents in 2000. While party identification can indeed change within the electorate, there is no evidence anywhere to suggest that Democrats will only represent 31% of the total vote this year.
This is no small consideration. Given the fact that each candidate receives anywhere between eight in ten and nine in ten support from voters in his own party, any change in party identification trades point for point in the candidate's total support. My polls use a party weight of 39% Democrat, 35% Republican and 26% Independent.
And as I said in an earlier post, Zogby with his more realistic weighting has this race as a four-point Bush lead in a poll of likely voters taken through Thursday night (and only two points when “leaners” are included). Given the likelihood of increased Democratic turnout this year, it is very possible that “likely voter” poll results may underreport the true state of voter preference on Election Day. Yet Time came out with its own follow-up poll at the same time as Zogby that now shows Bush maintaining an eleven-point Bush lead amongst likely voters (52%-41%), and a 51%-39% result amongst registered voters. And the breakdown for this sample of registered voters in the Time poll?
Given that the Democratic party ID in the last two presidential elections was 39%, how likely is it that the party ID of those turning out to vote this year will be only 32% Democratic at a time of heightened energy amongst the party’s base to get rid of Bush? Not likely.
The problem with these polls and their questionable sampling methodologies is that they leave a false impression of voter preference with the lazy media that never looks beyond the first paragraph of a press release to verify the reliability of the poll’s fundamentals. Then this lazy media reports something that may very well not be true, that one candidate or another isn’t really as strong as they appear. Before too long, such misreporting does psychological damage by discouraging one side or the other from turning out because they feel they cannot win, when in fact the race was closer than the media reported.