The 3NR

Several readers provided thoughtful commentary about my recent essay about evidence analysis, “Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate.” This post is an attempt to further develop the arguments advanced in the initial article while addressing the concerns of critics.

1. Normalizing the verbal citation of author qualifications will “nudge” the debate process in the direction of the development of new “metrics of scholarly authority.”

Much of the feedback regarding the article has centered around the competitive outcome of this change in the norm about evidence citation: how will debates about qualifications be resolved?, what qualifications will be preferred?, etc. This largely misses the point: the function of this change in norm is to emphasize the importance of these discussions and encourage debaters and judges to address them explicitly.

Nick Bubb highlighted many of the issues in a thoughtful comment:

[M]any people perceive authors’ opinions to be a politically motivated response to a given issue, rather than an independent evaluation of the truth. … For example, do we minimize Howard Dean’s opinion on health care reform because he’s a democrat and advocates for health care reform? Or do we prefer his analysis because he knows the policy? Or minimize his opinion because he stands to gain politically from the enactment of health care reform? Or do we prefer his opinion because he’s a doctor? What about his opinion on the political implications of health care policy? There are fair arguments to be made on all of these questions, but the structure for interpreting who is qualified to speak to the truth of a given issue is difficult. Certainly some individuals are more qualified than others, but how can we answer that question? If you are to believe some aspects of a hermeneutical process, authors’ qualifications are really their biases and we as the listener have biases for/against their experiences. We can be jaded and dismiss them or we can listen to their reasoning. But which action corresponds with finding the truth? The answer can’t be as simple as to listen to everything, because that degrades back into the problems you’re attempting to address: the prevalence of questionable evidence quality.

There’s also something odd about needing qualifications to speak to an issue. You don’t need a degree from Harvard to talk about poverty. A narrative from a poor person may be equally as powerful. I suppose the “qualifications” can change depending on the context, but then what do qualifications mean?

[A]s a judge, I wouldn’t know how to handle comparative claims. Do I prefer evidence from an economics professor about poverty policy or is it more important to listen to the people that the policy affects?

This is exactly my point: these issues are difficult, but they are also important. In a world where students are exposed to ever-expanding volumes of information, learning to intelligently separate the good from the bad is essential to informed citizenship.

I do not pretend to know the answers to the questions that Nick has posed. I can offer no mechanism for cleanly separating the intellectual wheat from the chaff. But the current model we have adopted in debate is certainly subject to criticism: “if it’s published, it’s evidence” has absolved us of our responsibility to take these issues of scholarly credibility seriously and of teaching students to intelligently navigate through the maze of information at their fingertips.

Effectively determining whom to believe—and more importantly, why to believe them—is arguably the most essential life skill that debate can teach. Perhaps better than any other activity, debate can effectively train students to think critically—to question others’ arguments and to evaluate their claims with skepticism. Working through the complicated business of analyzing sources and comparing qualifications is part and parcel of this facet of debate pedagogy.

The current norm—evidence should be verbally cited only by author’s last name and date of publication—hamstrings our ability to emphasize this aspect of critical thinking and in fact actively undermines it by framing the issue of qualification as separate from instead of intrinsic to the evidence itself.

As I argued in the article, this effect occurs at two levels:

1. Excluding qualifications from verbal presentation implicitly de-values their importance when considering the quality of a piece of evidence. If the author(s)’ qualifications are not important enough to read aloud, after all, how important can they really be? …

2. Requiring students to locate the qualifications of a given piece of evidence “privately”—during speech or prep time—prevents the judge from considering qualifications as part of their initial understanding of the evidence as it is being presented.

Shifting the norm to require verbal citation of author qualifications uniquely addresses these concerns.

2. Debate participants overestimate their ability to present, discern, and evaluate the “warrants” of most arguments.

Troy Bolton questions the need for a shift from “competing warrants” to “competing author qualifications”:

Why exactly is it so imperative to shift debate away from a discussion of competing warrants (which is more or less the norm now, at least in the decent rounds) to a discussion of competing author qualifications? Or put another way, if debate is just a game, and the way to pick the winner is to decide who did the best job arguing a partciular case within a particular set of parameters, why should we shift the parameters away from what they are now?

This was addressed, at least in part, in the previous discussion of information consumption. More importantly, though, there is no forced choice between an evaluation of warrants and an evaluation of qualifications. Indeed, the attempt to separate the two harms the hermeneutical process through which arguments are understood in contest rounds, something that will be discussed in more detail below.

As I mentioned in a comment responding to Roy Levkovitz’s recent article “Revisiting the Toulmin Model in debate,” there is more to an argument than simply a claim and a warrant.

Without grounding, a warrant is just another claim. Many times in debates, the only grounding (or backing) that a piece of evidence offers is its appeal to authority. Even when a card has “warrants,” it rarely has any data to support them (that portion of authors’ arguments tends not to get cut/included in debate evidence).

In reality, the “warrant” for a given argument cannot (and should not) be separated from the larger context from which it is derived. The notion that debaters and judges can intelligently resolve incredibly complex debaters over issues of public policy controversy by simply “comparing the warrants” is naive to the level of absurdity.

We do our best, for sure, and our training in argumentation often helps us make much better decisions than would be made by a layperson. But we are not experts in the fields that are discussed in our debates, and we rely on cited experts for that very reason.

Before discussing this further, a detour is necessary.

In a comment made to the original article, Alex Gulakov argues that the verbal citation of an author’s qualifications is indistinguishable from other, undesirable demands on debaters:

I think your argument taken to its logical endpoint would say debaters should not underline evidence and they should read the entire article or book chapter. Debate is a game, so no one’s going to voluntarily read a qualification like “oil lobbyist” that obviously disadvantages them by pointing this out verbally to the other team. Just like a debater is rewarded for catching some key non-underlined sentence qualifying the opponent’s tag and pointing this out, the same is true if a debater catches their opponent’s card having an unread “biased” qualification. … [T]he underlying logic is the same: that a debater should sacrifice strategic benefit in favor of making debates overall more quality, expert–reliant, or otherwise “better.”

The justification for verbal citation of author qualifications is not the search for truth—if it were, then this comparison would make more sense.

First, the quotation of excerpts from published works is an accepted scholarly practice. When debaters cite excerpted paragraphs from a book or article, they are engaging in a legitimate practice: there is nothing wrong with “highlighting down” an author’s work so that its content may be presented within the time constraints of a contest round. If the underlining/highlighting process changes the content of the author’s work, then that is another matter. If this occurs in a debate, my hope is that it is being done unintentionally. But the issue of whether an author’s qualifications should be cited verbally is distinct from concerns about accepted practices for excerpting published work.

Second, the “oil lobbyist” concern is a benefit to the verbal citation of author qualifications. If students would be embarrassed to read a piece of evidence if the qualifications of its author were cited aloud, they should not read that piece of evidence—this seems self-evident. If, on the other hand, a debater feels that a given piece of evidence is strong despite coming from a source that could be perceived as biased, they should read that piece of evidence and vigorously defend it.

In either case, however, we should not pretend that the lack of verbal citation of the author’s qualifications does not impact the hermeneutical process through which that argument is understood. One of the primary effects of our current norm is that arguments are understood as separate from the authors cited to support them. When an argument about an author’s qualifications is made, it is the first time the judge has heard that qualification—s/he does not have a context for incorporating this new dimension of the argument into what s/he initially understood.

Returning to an example used in the initial article, there is a dramatic difference between a judge’s understanding of a piece of evidence describing the feasibility of clean coal technology from “Lucas ‘9” and their understanding of the same piece of evidence from “Lucas, spokesman for the American Coalition for Clean Coal Electricity, ‘9”. If this piece of evidence is presented in the first instance without the author’s qualification, the opposing team’s assertion that “their evidence is from a coal industry spokesman—prefer our evidence because it is from independent scientists without a financial stake in clean coal” will be isolated in the judge’s mind from the initial argument about clean coal’s feasibility.

This is largely responsible, I think, for the difficulty that judges have in resolving arguments about evidence quality. A similar refrain is heard from a broad swath of judges when questioned about these arguments: “debater’s don’t impact their author qualification arguments enough,” “I just think the argument in the card still makes sense,” “I don’t know what to do with this argument—I don’t want to throw out the card entirely,” etc. This is a predictable response given the aversion most judges have to “intervention.” When author qualification arguments are perceived as distinct from instead of intrinsic to the argument being advanced by a piece of evidence, it will be difficult for judges to avoid feeling “interventionist” about them.

Finally, the incorporation of an author’s qualifications into the understanding of an argument does not mean that certain authors need to be preferred. In the above example, a debater could persuasively defend the authoritativeness of evidence from the American Coalition for Clean Coal Electricity in a variety of ways: it cites scientific studies, it cites the consensus of experts, the author has many incentives not to overstate his claims, the industry’s opponents are unqualified/relying on bad science, etc. A rational decision-maker might decide to listen to evidence from a lobbyist or industry spokesman despite their conflict of interest because he or she made a good argument.

Indeed, Congress routinely solicits testimony from “biased” experts on both sides of an issue like clean coal. Members of Congress and their staff listen to their arguments, read their prepared testimony and supporting materials, ask them questions, and then come to a conclusion about the issue at hand. But the first thing that every expert does when testifying before Congress is introduce themselves and describe their affiliations/positions—in debate-speak, their qualifications. This provides Congress with a context within which to understand their testimony. Debate judges should demand the same context.

Gulakov continues:

The norms you talk about in the introduction are not arbitrary. They did evolve logically: from debate being a competitive game. It is nice to quote an emphatic, totalizing phrase in cross–x (like “after examining every possibility, i conclude there is no other possible challenger to us heg”) to leave an opponent with no possible “but what about x” objection, or to say something like “our author clearly refutes your arg here: ‘While some may consider x to be the root cause of y, such logic is flawed for three reasons.’”

The dominant features of “good evidence” were not selected at random, certainly. But neither are they objectively correct.

Totalizing rhetoric, for example, can be viewed not as helpful but as harmful to an author’s credibility. Academic authors rarely write with the degree of certainty expressed by these hypotheticals (“after examining every possibility”), and for good reason: a degree of modesty about one’s conclusions reveals a thoughtful mind.

Tim Alderete has explicitly discussed this in his judging philosophy:

I am changing how I approach hyperbolic evidence. Particularly with all of the sources available on The Internets, way too many un-peer reviewed cards make ridiculous claims and use Extremely hyperbolic rhetoric. This isn’t only an issue of warrants or qualifications – some well qualified authors make ridiculous but warranted claims using hyperbole. My concern is that debate rewards this because power wording and extreme arguments are what make for “Good Cards” in debate. I am increasingly skeptical of this – I find it very hard to give the same weight to a screeching Weekly Standard Neo-Con that I give to a more reasonable author. Bottom line, if your author is hyperventilating about the blood of terrorists, or claims that his elixir can cure cancer, AIDS, racism and poverty, I think that the hyperbolic rhetoric is an Indict of that author, not just “power wording.” I don’t know exactly where to draw the line, I rarely will just ignore evidence, I don’t know how much of that needs to be made in the round – I don’t Know where I will end up on this. But for practical purposes, I think that I will reward teams that point this out, and I will make every attempt to apply the same standards to hyperbolic Kritik evidence, which I probably haven’t done in the past as well as I can.

The latter of Gulakov’s examples (“x = y is wrong for three reasons”) is not really relevant to this discussion—obviously, authors that explicitly cite and refute another’s argument are fruitful sources of evidence for debates. But this is really external to the question of whether an author’s qualifications should be read aloud.

Gulakov’s final objection is that the verbal citation of author qualifications forces debaters to jeopardize their strategic position:

If debate rounds were structured more like a philosophy or lit dept conference, we’d obviously look for different things in writing. … Can you give me some examples of other times that debate norms changed towards something that is “unstrategic”?

I disagree with the premise that the verbal citation of author qualifications is “unstrategic.” As explained above, I do not think the outcome of a discussion of evidence quality is pre-determined: just because you’ve read a piece of evidence from a lobbyist, for example, does not mean that you will lose the given point. To the extent that the verbal citation of qualifications discourages debaters from reading evidence which, if questioned, they would be unable to defend, however, this demand is desirable.

To separate the source of the piece of evidence being cited from the argument that the piece of evidence is supporting undermines the hermeneutical process through which that argument is understood. If improving our understanding of arguments is “unstrategic”, then we need to recalibrate the norms of our game.

Gulakov proposes a “counterplan”: debaters should textually cite the qualifications of the authors they are citing and judges should consider evidence without such a cited qualification as equivalent to an analytical argument.

Just having the quals in the cite is a better idea for a norm because it is more feasible to implement and doesn’t require the judge to already be convinced. This is not the status quo: the major change would be having judges regard evidence without qualifications (or a debater’s ability to provide one) as on the level of analytics. It would be quite easy for debaters to start putting quals in all their cards especially given this deterrence.

I think this is the status quo, at least for some subsection of judges. For the rest—those willing to evaluate evidence without qualifications and those who prioritize “having a card” over an analytical argument—this transition is undoubtedly an uphill battle, something I am more than willing to admit.

Compared with the initial proposal to normalize the verbal citation of qualifications, however, this counterplan falls short because it retains the hermeneutical wall between the argument advanced by a piece of evidence and the qualifications of the author the piece of evidence is citing. The only unique disadvantage to verbal citation is “time loss”—but this is a red herring. If establishing a norm in favor of orally citing an author’s qualifications results in better debates, then the time investment is justified.

Conclusions

This brings us full circle. If it is true that developing more sophisticated ways of assessing the qualifications of evidence presented in policy debate rounds is desirable, then it is important to seek ways to actualize this outcome. Given the authoritative role played by judges in determining the norms that debaters follow, then it is important that judges take the initial step to “nudge” debaters in this direction. Enforcing an expectation that students verbalize the qualifications of the authors they are citing will bring the issue of author qualification into the hermeneutical process through which arguments are understood in a contest round, thereby encouraging students to develop innovative ways to assess scholarly authority. The resulting proliferation of arguments regarding source quality and author qualifications will improve debates both competitively—by spurring better, deeper argumentation about evidence—and educationally—by training students to behave as intelligent consumers of information.

Additional comments—either here or on the original article—are appreciated.