Falls right into place after Mary's recent post. You've got a great chain (for lack of a better word) going here. Re Groseclose and Milyo: I just love it when people use quantitative analyses for qualitative data. It flaws everything they speak about to a point where it can simply be ignored as worthless and inaccurate. I hope your final summation and conclusions are as strong as the arguments you've been making to this point.

I believe a novel is in order. Your posts are extremely insightful about this liberal media BS. You need more than just a website to make a dent into this Borg collective.

You got a link from David Neiwart for this series. Damn.

Thank you.

Great and interesting series.
Thanks for all the work you put into this.

This part here just really hit a nerve with me..

Their confident conclusion that they have proven "liberal media" bias is simply wrong because the study does not examine whether the media's news reporting is accurate.

And therein lies a major problem with not only the study, but the media and "news" reporting... It's the accuracy stupid! Fake news reporters, fake news, fake facts...who cares but oh the humanity the real facts might be considered "liberally biased" and therefore can be disregarded if they are used and pulled from a "liberal" organization such as the ACLU.

Oh yeah, Have you ever contacted the authors of this study at all and have they responded to your concerns or questions you have raised? Or perhaps they blew off your accurate concerns for being connected and cited by a liberal website?(just kidding on the last sentence here)

I have not read the entire post yet, but I focused on the paper cited early on in it. The authors, to whom I shall now refer as Grotesque&Mildew, have a very powerful--at least, rhetorically--rebuttal to Nunberg. It is easy to dismiss Nunberg as a ranting windbag for stogy academics because Nunberg tends to present his arguments at common-man level (likely honed by years of involvement with NPR). Having said that, I also think that Nunberg is right and Grotesque&Mildew are wrong. The study itself is little more than exersise in bias. Consider, for example, a quotation from G&M, page 4:

However, a strong form of the view that reporters offset or blunt their own ideological biases leads to a counterfactual implication. Suppose it is true that all reporters report objectively and their ideological views do not color their reporting. If so, then all news would have the same slant. Yet, few would disagree that Fox News or the Washington Times has a more conservative slant the New York Times.

The statement is accurate, in so far as the counterfactual goes: there is a political spectrum of coverage, so someone must be biased. However, the underlying assumption is that this bias is symmetric. That is, Grotesque&Mildew assume that those at one end of the spectrum (Fox News) are as biased as those at the other end (New York Times and CBS News). Only their ideological slants are different but the degree of bias is the same.

This assumption has nothing to do with the factual statement made above. Fox News clearly wants you to assume that its coverage is objective and everyone else is biased. While those on other end do not make these assumptions as explicit as Fox, they clearly hold them as well, although in the case of the New York Times, it does pride itself on being the liberal newspaper of record, despite repeated evidence to the contrary.

But let us look at another quotation from G&M, page 7:

The tables shed some light on some much debated topics about the ideological position of various think tanks. First, the table reveals that the position of the Brookings Institution clearly leans left.

This is soon followed by another seat-of-the-pants analysis:

Second, contrary to conventional wisdom, the RAND Corporation is fairly liberal. The adjusted ADA score of the average legislator citing it is 53.6, using sentences as the level of observation, and 52.6, using citations as the level of observation. This is significantly to the left of the center of Congress, although not as far left as, say, the Center on Budget and Policy Priorities, the Children’s Defense Fund, or the Economic Policy Institute. We mentioned this finding to some members of RAND, who told us they were not surprised. While RAND strives to be middle-of-the-road ideologically, the more conservative scholars at RAND tend to work on military studies, while the more liberal scholars tend to work on domestic studies. Because the military studies are sometimes classified and often more mundane than the domestic studies, the media and members of Congress tend to cite the domestic studies disproportionately. As a consequence, RAND appears liberal when judged by these citations. It is important to note that this fact—that the scholars at RAND are more conservative than the numbers in Table 1 suggest—will not cause a bias to our results. To see this, think of RAND as two think tanks: RAND I, the left-leaning think tank which produces the research that the media and members of Congress like to cite, and RAND II, the conservative think tank which produces the research that they do not like to cite. Our results exclude RAND II from the analysis. This causes no more bias than excluding any other think tank that is rarely cited in Congress or the media.

Extra credit to anyone who can find evidence in this paragraph. I am quite confident that no one will get a perfect score on this test. Let me translate the bit: Grotesque&Mildew found some data that did not mesh with their theory, so they shopped for an explanation why their theory might be right, then simply assumed that their theory was confirmed because someone had offered an opinion that explained a major discrepancy between their predictions and results. In other words, they got their evidence out of their asses.

There is a fundamental flaw in the assumptions that is clearly revealed in the analysis of RAND data. The assumption is that ideological bias is uniform across issues. In other words, every black box that is a compendium of opinions on pressing issues by one individual can be neatly quantized and taged with a single number that describes the degree of bias and places the black box at a specific point in a right-to-left continuum. In software circles this is called garbage-in-garbage-out. In logical circles, it's simply known as a falacy--if the assumption is false, any conclusion will appear to be true (symbolically, the statement can be written as, "A implies B" is TRUE for all values of A that are FALSE).

While accusing Nunberg of being too ignorant to understand their argument because he ignored their statistical methodology, Grotesque&Mildew are trying to pull a fast one by totally ignoring the fact that statistics is meaningless if the underlying assumptions are false. It seems Nunberg was far too clever for them to understand his criticism.

On page 19, G&M give a list of various think tanks with their respective scores. It would be easy to succumb to the notion that the spectrum seems to represent at least the ordering correctly. In fact, I can easily come up with labels that say, for example, that scores below 15 are the right wing lunatic fringe, 30-50 are middle-of-the-road or moderate or perhaps even unbiased, and 70+ represents the loony left. But, once again, this would be a statistical falacy. Remember, we are judging a collection of statements by individuals on diverse issues, which means that quantifying this information with a single number is meaningless. Consider the think tanks that ended up stuck between 15 and 30. One can consider it either substantive or ironic that every one of them is libertarian! In fact, one can derive some satisfaction, even, from observing that it should not be surprising that ACLU ends up closest to libertarian institutions! Protecting individual liberty is a libertarian value. DUH!

OK, then. How do Grotesque&Mildew explain this discrepancy? (page 8, right after the RAND explanation):

Perhaps the biggest surprise of Table 1 is the average score for the ACLU. Weighted by citations, the average score, is 42.66, which is near the center of congressional scores. Weighted by sentences, the average score is 34.99, which is to the right of the average member of Congress. The primary reason that the ACLU appears so conservative is that it opposed the McCain-Feingold Campaign Finance bill. Consequently, conservatives tended to cite this fact often. Indeed, slightly more than half of the ACLU sentences cited in Congress were due to one person, Mitch McConnell (R.-Kt.), who strongly opposed the McCain-Feingold bill. If we omit ACLU citations that are due to McConnell, then the average score, weighted by sentences, increases to 70.12. Because of this anomaly, in the Appendix we report the results when repeat all of our analyses but omit the ACLU data. This causes the average score of the media outlets to become approximately one ?? point more liberal.

Say what? An opinion on oneissue skewed an organizations score nearly all the way to the opposite end of the spectrum? This alone should have been enough to doubt, if not throw out the entire study. It seems that my two explanations of their results are much more consistent with their own data than their own analysis. First, expressing opinions on a variety of issues is more likely to land someone closer to the "left" end of the spectrum (score of 50 or so), but still smack in the middle of moderate contingent. Yet, someone who expresses a disproportionate number of opinions on a narrow range of subjects may end up being mislabeled.

Let me digress for a moment. Consider another statistical model that attempts to assign a single number to a diverse range of characteristic of a disparate group. I am talking about the power rating in college sports. Although the details I am offering are not exact, the basic design goes approximately like this. Each conference is assigned historical ratings based on past performance--the exact nature of this performance may take a variety of factors into account, but, it is essentially based on past performance. Using this reference point, each team within a conference is then assigned some rating for seeding. Following this, as teams play each other, a fairly complex formula adjusts these ratings based on how well a team does againt opponents who are higher, lower, or at about the same level in the ratings. However, in the end, the ratings always appear skewed toward the teams from the conferences that have traditionally been seeded high. In other words, the initial bias shows through in the final results. Why? Is it because the power-conference teams perform better than other teams? Not necessarily. Even the weakest opponents within the conference--teams that may not win a single game in the conference--are often ranked much higher than well-respected teams from non-power conferences in final rankings. This comes from the seeding, not from the adjustments. The adjustments cannot eliminate initial biases even if they can mitigate some of them.

In chess, the situation is quite different. Every unranked opponent is initially assumed to have the same flat rating. Based on this fact, the ratings are adjusted after every match. The more one plays, the more accurate his rating.

The scheme that Grotesque&Mildew propose is more like the college team rating than the chess rating. It starts out with a bias and some indefensible assumptions (e.g., the possibility of linear ranking on agglomerations of issues) and these biases show through in the end, quite unsurprisingly. Where there are surprises, the authors dismiss them based on opinions, not facts.

In other words, the study is more fiction than fact.

Thanks to all readers for the comments...

emal, I sent an emails to one of the authors today.

Buck, wrt your comment that "Remember, we are judging a collection of statements by individuals on diverse issues, which means that quantifying this information with a single number is meaningless", I see where you are coming from but this in itself in not the most problematic part of the paper. It is certainly a debatable assumption on their part, but studies of this nature require some degree of quantification of the aspects being studied. So, I personally don't blame them for trying to quantify ideology. This is not to say that the adjusted ADA scores they used are the right approach - maybe, maybe not. My point is that regardless of what score they chose, the underlying assumptions and methodology are wrong - so it doesn't make any difference to the results.

Your other comment "An opinion on oneissue skewed an organizations score nearly all the way to the opposite end of the spectrum? This alone should have been enough to doubt, if not throw out the entire study." is quite on point. G-M took away the wrong message from the ACLU example - as I've also explained.

Thought some of you may be interested in this National Media Conference being held in St. Louis.

The National Conference for Media Reform is just six weeks away--and it's bound to be one of the most exciting and inspiring events of the year.

Your presence couldn't be more crucial. Over the next couple of years, policymakers will make decisions that will determine the future of our media. Industry lobbyists are dedicating millions of dollars to lobby, litigate, and lie their way toward rules that will bring big media corporations more power and profits. We need your help to educate and mobilize millions of citizens to take on these powerful corporations and defend the public interest.

Join me and hundreds of others in St. Louis on May 13-15 to gather momentum for this fight.

Register now--the early bird registration discount ends this Thursday, March 31:

http://www.moveon.org/r?r=673

At the conference, you'll meet hundreds of activists, academics, media makers and concerned citizens from around the country. You'll hear speeches by renowned guests like Amy Goodman, Jim Hightower, Naomi Klein and Al Franken, as well as FCC commissioners and members of Congress.

You'll choose from dozens of panels and workshops on media consolidation, grassroots activism, globalization, independent media, commercialism, community Internet, and much more. And you'll get the information, inspiration, and contacts you need to take your media activism to the next level.

More on the conference program, registration, lodging, and everything else is available on Free Press' website at:

http://www.moveon.org/r?r=673

This is a timely argument as I heard a presentation by one of the authors just yesterday at the University of Missouri in regards to their most recent findings.

Dear Buck Turgidson, please keep in mind two things, 1. Dr. Milyo made several comments to suggest his own liberal leanings, 2. Name calling only works for Rush, who wishes to distract the gp from discussions about media. Read the primary documents before resorting to violence.

I was going to bring up the following point in the Q&A, but the time was mostly dominated by sociologists complaining about things outside of the researchers control, such as the education level of think tank members.

While there were some methodological concerns raised by the respondents of the panel, the comparative nature of their most recent results are quite interesting. What the authors seem to miss is all news outlets covered in their study (With the exception of the WSJ) are more conservative than median liberal member of Congress and more liberal than the median conservative member.

This would seem to vindicate both sides. The average liberal (adjusted ADA score of 84) can argue the media is too conservative (the highest score was CBS news at 73). The average conservative (16) can also argue the media is too liberal (Fox news at 35). for all the methodological concerns about whether the center is 50 or 54 or whether NBC news is 57 but really slightly consevative, the data compares news outlets relative to politicians, which is really more important to me.