Friday :: Mar 25, 2005

How the Liberal Media Myth is Created - Part 3

by eriposte

In Part 1 of this series, I covered myths created about "liberal bias" using "tone" of media coverage. In Part 2, the creation of "liberal media" myths by the Right using "catch-phrases" was covered. In this part, I highlight a third approach used to create the "liberal media" myth - "newspaper headlines".

The offending pair on the Right in this case are John Lott and Kevin Hassett (of the American Enterprise Institute - AEI), in their 2004 paper "Is Newspaper Coverage of Economic Events Politically Biased?". (Yes, this John Lott). I've covered this paper more extensively at ICM - here, but I'm going to reproduce a few key portions that illustrate how myth-creation works in this case.

The abstract of this paper says the following (bold text is my emphasis):

Accusations of political bias in the media are often made by members of both political parties, yet there have been few systematic studies of such bias to date. This paper develops an econometric technique to test for political bias in news reports that controls for the underlying character of the news reported. Our results suggest that American newspapers tend to give more positive news coverage to the same economic news when Democrats are in the Presidency than for Republicans. When all types of news are pooled into a single analysis, our results are highly significant. However, the results vary greatly depending upon which economic numbers are being reported. When GDP growth is reported, Republicans received between 16 and 24 percentage point fewer positive stories for the same economic numbers than Democrats. For durable goods for all newspapers, Republicans received between 15 and 25 percentage points fewer positive news stories than Democrats. For unemployment, the difference was between zero and 21 percentage points. Retail sales showed no difference. Among the Associated Press and the top 10 papers, the Washington Post, Chicago Tribune, Associated Press, and New York Times tend to be the least likely to report positive news during Republican administrations, while the Houston Chronicle slightly favors Republicans. Only one newspaper treated one Republican administration significantly more positively than the Clinton administration: the Los Angeles Times’ headlines were most favorable to the Reagan administration, but it still favored Clinton over either Bush administration. We also find that the media coverage affects people’s perceptions of the economy. Contrary to the typical impression that bad news sells, we find that good economic news generates more news coverage and that it is usually covered more prominently. We also present some evidence that media treats parties differently when they control both the presidency and the congress.

Why have I highlighted specific words? Well, when you read about the methodology they use, you'll understand (bold text is my emphasis):

In this paper, we attempt to overcome these problems by objectively categorizing newspaper headlines as either positive, negative, neutral or mixed and then comparing those headlines to the actual economic numbers that generated those news articles.

They study newspaper headlines and not the actual content of the articles (wow) - and I would bet that anyone who reads the abstract of the paper could easily miss this point, namely that their "study" is based on headlines - not "news reports", "news coverage", "stories", "news stories", etc. I imagine it must be particularly busy out there at AEI. 

Now, they do acknowledge this silliness (not in so many words of course), among other things (bold text is my emphasis):

We chose headlines because they create the strongest image of the news in readers’ minds, and because headlines are easier to objectively classify, though the headlines we examine may differ systematically from the stories they are associated with. While newspapers write other news stories on the economy that do not coincide with the specific release of economic data, one benefit of limiting ourselves to these announcement dates is that we can more directly link a specific set of economic data to how the media covers that data. It is possible that these other news stories are biased in ways that are different from stories released on announcement dates, and thus announcement date coverage might not give the complete picture of any partisan biases. The values for the different economic variables were those released at the time of the news reports.

So, let's recap.

  • They only look at headlines; they don't look at the actual content of articles

  • They only consider headlines associated with articles that coincide with the release of the economic data and not at any other articles that may be published about the same data subsequently

  • They acknowledge that "It is possible that these other news stories are biased in ways that are different from stories released on announcement dates, and thus announcement date coverage might not give the complete picture of any partisan biases"

  • And predictably, they make firm conclusions from their data anyway

Let's just say I didn't have it this easy in graduate school. And they actually get paid big bucks to write this stuff up, while I have to do this on my own dime. That said, these flaws are the least of the paper's problems.

Tim Lambert of Deltoid, who has done tremendous public service by exposing Lott's repeated shoddy work and lying on the topic of guns, also covered this paper. The following extract from one of his posts shows, in a nutshell, what is wrong with this whole paper:

Now, here’s what Lott and Hassett say:

“In the case of unemployment, 44 percent of the headlines under the Clinton administration were positive while that same number was only 23 percent under Bush II. By comparison, the average unemployment rates were fairly similar, 5.2 percent under Clinton s eight years and 5.5 percent under Bush during the sample. There is also a great deal of overlap (3.9 to 7.1 percent under Clinton to 4.2 to 6.4 percent under Bush II).”
What they fail to mention and what is obvious from the graph is that under Clinton the unemployment rate decreased from 7.1% to 3.9%, while under Bush it increased from 4.2% to 6.4%. Maybe, just maybe, that’s why the headlines were more positive under Clinton. In fact, there seems to be evidence of bias against Clinton—why were only 44% of the headlines about unemployment positive when it just kept going down and down to the lowest levels in decades? Oh, and don’t expect to see a graph of the unemployment rate anywhere in their paper or presentation.

Now, they claim to have controlled for level and trends in unemployment in their analysis, but of course they have not. The only control they have for trend is the change since the previous quarter and it is obvious that changes over longer terms will affect the reporting. Do Lott and Hassett believe that no-one ever compares the unemployment rate with what it was a year or two before?

Through a Google search I came across this post at Dead Parrots Society that explored the unemployment comparison further (not specifically in the context of the Lott/Hassett paper, but in the context of a similarly nonsensical "media bias" post by another blogger, using the employment figures):

Via Glenn Reynolds, I and many others have been reading this Tim Blair post about media framing of unemployment figures. The gist is that CNN described a 5.6% unemployment rate as "low" in 1996, when Clinton was in office, but describes a similar rate as a sign of problems for Bush. Tim's post is being widely cited as yet more proof of media bias; in Glenn's link, he encourages us to "Go figure." So I did.

The graphic is courtesy of the BLS, and shows the unemployment rate charted over the past 15 years. Perhaps it offers a little insight into why 5.6% was considered "low" in early 1996, but not in 2001. Actually, the context was right there in the excerpts Blair chose from the 1996 CNN story:

Economists didn't expect June's unemployment rate to be much different from May's, which was an already-low 5.6 percent. But in fact, it did fall -- to 5.3 percent. The unemployment rate hasn't been that low since June 1990.

And from the 2001 CNN story:

The U.S. unemployment rate jumped to 5.7 percent in November - the highest in six years - as employers cut hundreds of thousands more jobs in response to the first recession in a decade in the world's largest economy.


Still, there is some context that might be helpful. The first place we can look is right there in Tim Blair's 1996 story, to see how the Clinton administration and economic analysts felt about the numbers:

White House: But the Clinton administration was tickled about the increase in jobs, and took credit for the upturn. The president said the figures showed "the most solid American economy in a generation."
Analysts: In January, analysts were concerned that growth was so anemic that the nation was in danger of a recession. But five straight months of strong job gains now have analysts worried more about inflation. ... The Federal Reserve is almost guaranteed to push interest rates up to stave off inflation.

The second place we can look for context is in Tim Blair's 2001 story, to see how the Bush administration and economic analysts felt about that very similar unemployment figure:

White House: President Bush and his Labor Secretary, Elaine Chao, separately expressed alarm at the data and called for Congress to approve a package of economic stimulus. "Today's numbers are not good news, and I think it's a clear reflection that the attacks of Sept. 11 are still reverberating around our economy," Chao told CNNfn's Market Call program.
Analysts: To keep consumers spending despite mounting unemployment, the Federal Reserve has cut its target for short-term interest rates 10 times this year and is expected to do so again after its policy makers meet Tuesday. "Despite some better-than-expected data over the past two weeks, this report is sufficiently gloomy to force the Fed to ease next Tuesday and retain their bias toward further economic weakness," said Steven Wood, economist with FinancialOxygen.

Really, my point here doesn't have anything to do with whether a 5.6% unemployment rate is too hot, too cold or just right. Frankly, I don't have any idea. What I do know is that journalists weren't the only ones who looked at the unemployment figures in a different light between 1996 and 2001. The reality is, the media saw the data the same way as the White House, economic analysts and the Fed.

Thus, even at a fundamental, conceptual level, the Lott/Hassett paper is a bunch of garbage and proves absolutely nothing about bias in news reporting.

This goes back to the point I have been making in each of my previous posts. Accuracy. You cannot assess bias without understanding how accurate the report is, and you certainly can't figure out the accuracy by either looking at headlines alone or headlines compared to out-of-context data points. There is also a lot more detail within those numbers which could influence the news reporting, as one of the commenters (Barry Ritholtz) to the Dead Parrots Society post noted - such as quality of jobs created v. lost, spread between wage growth and CPI, the underemployment rate, those who have dropped out of the workforce, etc.

eriposte :: 10:17 AM :: Comments (15) :: Digg It!