<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>The 3NR &#187; qualifications</title>
	<atom:link href="http://www.the3nr.com/tag/qualifications/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.the3nr.com</link>
	<description>a collaborative blog about high school policy debate</description>
	<lastBuildDate>Sat, 07 Apr 2012 18:15:43 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Reading Author Qualifications Aloud: A Response To The Critics</title>
		<link>http://www.the3nr.com/2009/06/11/reading-author-qualifications-aloud-a-response-to-the-critics/</link>
		<comments>http://www.the3nr.com/2009/06/11/reading-author-qualifications-aloud-a-response-to-the-critics/#comments</comments>
		<pubDate>Fri, 12 Jun 2009 04:12:29 +0000</pubDate>
		<dc:creator>Bill Batterman</dc:creator>
				<category><![CDATA[Evidence/Research]]></category>
		<category><![CDATA[Judging]]></category>
		<category><![CDATA[evidence]]></category>
		<category><![CDATA[qualifications]]></category>
		<category><![CDATA[research]]></category>

		<guid isPermaLink="false">http://www.the3nr.com/?p=156</guid>
		<description><![CDATA[Several readers provided thoughtful commentary about my recent essay about evidence analysis, &#8220;Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate.&#8221; This post is an attempt to further develop the arguments advanced in the initial article while addressing the concerns of critics. 1. Normalizing the [...]]]></description>
			<content:encoded><![CDATA[<p>Several readers provided thoughtful commentary about my recent essay about evidence analysis, &#8220;<a href="http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/" title="Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate -- The 3NR">Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate</a>.&#8221;  This post is an attempt to further develop the arguments advanced in the initial article while addressing the concerns of critics.</p>
<p><strong>1. Normalizing the verbal citation of author qualifications will &#8220;nudge&#8221; the debate process in the direction of the development of new &#8220;metrics of scholarly authority.&#8221;</strong></p>
<p>Much of the feedback regarding the article has centered around the <em>competitive outcome</em> of this change in the norm about evidence citation: how will debates about qualifications be resolved?, what qualifications will be preferred?, etc.  This largely misses the point: the function of this change in norm is to <em>emphasize the importance of these discussions</em> and <em>encourage debaters and judges to address them explicitly</em>.  </p>
<p>Nick Bubb highlighted many of the issues in <a href="http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/#comment-233" title="Nick Bubb's Comment re: Nudging Evidence Analysis In The Right Direction">a thoughtful comment</a>:</p>
<blockquote>
<p>[M]any people perceive authors’ opinions to be a politically motivated response to a given issue, rather than an independent evaluation of the truth. &#8230; For example, do we minimize Howard Dean’s opinion on health care reform because he’s a democrat and advocates for health care reform? Or do we prefer his analysis because he knows the policy? Or minimize his opinion because he stands to gain politically from the enactment of health care reform? Or do we prefer his opinion because he’s a doctor? What about his opinion on the political implications of health care policy? <strong>There are fair arguments to be made on all of these questions, but the structure for interpreting who is qualified to speak to the truth of a given issue is difficult.</strong> Certainly some individuals are more qualified than others, but how can we answer that question? If you are to believe some aspects of a hermeneutical process, authors’ qualifications are really their biases and we as the listener have biases for/against their experiences. We can be jaded and dismiss them or we can listen to their reasoning. But which action corresponds with finding the truth? The answer can’t be as simple as to listen to everything, because that degrades back into the problems you’re attempting to address: the prevalence of questionable evidence quality.</p>
<p>There’s also something odd about <em>needing</em> qualifications to speak to an issue. You don’t need a degree from Harvard to talk about poverty. A narrative from a poor person may be equally as powerful. I suppose the “qualifications” can change depending on the context, but then what do qualifications mean?</p>
<p>[A]s a judge, I wouldn’t know how to handle comparative claims. Do I prefer evidence from an economics professor about poverty policy or is it more important to listen to the people that the policy affects?</p>
</blockquote>
<p>This is exactly my point: these issues <em>are</em> difficult, but they are also <em>important</em>.  In a world where students are exposed to ever-expanding volumes of information, learning to intelligently separate the good from the bad is essential to informed citizenship.  </p>
<p>I do not pretend to know the answers to the questions that Nick has posed.  I can offer no mechanism for cleanly separating the intellectual wheat from the chaff.  But the current model we have adopted in debate is certainly subject to criticism: &#8220;if it&#8217;s published, it&#8217;s evidence&#8221; has absolved us of our responsibility to take these issues of scholarly credibility seriously and of teaching students to intelligently navigate through the maze of information at their fingertips.</p>
<p>Effectively determining whom to believe&#8212;and more importantly, <em>why</em> to believe them&#8212;is arguably the most essential life skill that debate can teach.  Perhaps better than any other activity, debate can effectively train students to <em>think critically</em>&#8212;to question others&#8217; arguments and to evaluate their claims with skepticism.  Working through the complicated business of analyzing sources and comparing qualifications is part and parcel of this facet of debate pedagogy.  </p>
<p>The current norm&#8212;evidence should be verbally cited only by author&#8217;s last name and date of publication&#8212;hamstrings our ability to emphasize this aspect of critical thinking and in fact <em>actively undermines it</em> by framing the issue of qualification as <em>separate from</em> instead of <em>intrinsic to</em> the evidence itself.  </p>
<p>As I argued in the article, this effect occurs at two levels:</p>
<blockquote>
<ol>
<li>
<p>Excluding qualifications from verbal presentation implicitly de-values their importance when considering the quality of a piece of evidence. If the author(s)’ qualifications are not important enough to read aloud, after all, how important can they really be? &#8230;</p>
</li>
<li>
<p>Requiring students to locate the qualifications of a given piece of evidence “privately”—during speech or prep time—prevents the judge from considering qualifications as part of their initial understanding of the evidence as it is being presented. </p>
</li>
</ol>
</blockquote>
<p>Shifting the norm to require verbal citation of author qualifications uniquely addresses these concerns.</p>
<p><span id="more-156"></span></p>
<p><strong>2. Debate participants overestimate their ability to present, discern, and evaluate the &#8220;warrants&#8221; of most arguments.</strong></p>
<p>Troy Bolton <a href="http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/#comment-234" title="Troy Bolton's comment re: Nudging Evidence Analysis In The Right Direction">questions the need</a> for a shift from &#8220;competing warrants&#8221; to &#8220;competing author qualifications&#8221;:</p>
<blockquote>
<p>Why exactly is it so imperative to shift debate away from a discussion of competing warrants (which is more or less the norm now, at least in the decent rounds) to a discussion of competing author qualifications? Or put another way, if debate is just a game, and the way to pick the winner is to decide who did the best job arguing a partciular case within a particular set of parameters, why should we shift the parameters away from what they are now?</p>
</blockquote>
<p>This was addressed, at least in part, in the previous discussion of information consumption.  More importantly, though, there is <em>no forced choice</em> between an evaluation of warrants and an evaluation of qualifications.  Indeed, the attempt to separate the two harms the hermeneutical process through which arguments are understood in contest rounds, something that will be discussed in more detail below. </p>
<p>As I mentioned in <a href="http://www.the3nr.com/2009/05/16/revisiting-the-toulmin-model-in-debate/#comment-112" title="Bill Batterman's comment re: Revisiting the Toulmin Model in debate">a comment</a> responding to Roy Levkovitz&#8217;s recent article &#8220;<a href="http://www.the3nr.com/2009/05/16/revisiting-the-toulmin-model-in-debate/" title="Revisiting the Toulmin Model in debate - The 3NR">Revisiting the Toulmin Model in debate</a>,&#8221; there is more to an argument than simply a claim and a warrant.</p>
<blockquote>
<p>Without grounding, a warrant is just another claim. Many times in debates, the only grounding (or backing) that a piece of evidence offers is its appeal to authority. Even when a card has “warrants,” it rarely has any data to support them (that portion of authors’ arguments tends not to get cut/included in debate evidence).</p>
</blockquote>
<p>In reality, the &#8220;warrant&#8221; for a given argument cannot (and should not) be separated from the larger context from which it is derived.  The notion that debaters and judges can intelligently resolve incredibly complex debaters over issues of public policy controversy by simply &#8220;comparing the warrants&#8221; is naive to the level of absurdity.  </p>
<p>We do our best, for sure, and our training in argumentation often helps us make much better decisions than would be made by a layperson.  But we are not experts in the fields that are discussed in our debates, and we rely on cited experts for that very reason.  </p>
<p>Before discussing this further, a detour is necessary.</p>
<p>In a comment made to the original article, <a href="http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/#comment-227" title="Alex Gulakov's comment re: Nudging Evidence Analysis In The Right Direction">Alex Gulakov argues</a> that the verbal citation of an author&#8217;s qualifications is indistinguishable from other, undesirable demands on debaters:</p>
<blockquote>
<p>I think your argument taken to its logical endpoint would say debaters should not underline evidence and they should read the entire article or book chapter. Debate is a game, so no one’s going to voluntarily read a qualification like “oil lobbyist” that obviously disadvantages them by pointing this out verbally to the other team. Just like a debater is rewarded for catching some key non-underlined sentence qualifying the opponent’s tag and pointing this out, the same is true if a debater catches their opponent’s card having an unread “biased” qualification. &#8230; [T]he underlying logic is the same: that a debater should sacrifice strategic benefit in favor of making debates overall more quality, expert–reliant, or otherwise “better.”</p>
</blockquote>
<p>The justification for verbal citation of author qualifications is not the search for truth&#8212;if it were, then this comparison would make more sense.  </p>
<p>First, the quotation of excerpts from published works is an accepted scholarly practice.  When debaters cite excerpted paragraphs from a book or article, they are engaging in a legitimate practice: there is nothing wrong with &#8220;highlighting down&#8221; an author&#8217;s work so that its content may be presented within the time constraints of a contest round.  If the underlining/highlighting process <em>changes</em> the content of the author&#8217;s work, then that is another matter.  If this occurs in a debate, my hope is that it is being done unintentionally.  But the issue of whether an author&#8217;s qualifications should be cited verbally is distinct from concerns about accepted practices for excerpting published work.</p>
<p>Second, the &#8220;oil lobbyist&#8221; concern is a <em>benefit</em> to the verbal citation of author qualifications.  If students would be embarrassed to read a piece of evidence if the qualifications of its author were cited aloud, they should not read that piece of evidence&#8212;this seems self-evident.  If, on the other hand, a debater feels that a given piece of evidence is strong despite coming from a source that could be perceived as biased, they should read that piece of evidence and vigorously defend it.  </p>
<p>In either case, however, we should not pretend that the lack of verbal citation of the author&#8217;s qualifications does not impact the hermeneutical process through which that argument is understood.  One of the primary effects of our current norm is that arguments are understood as <em>separate from</em> the authors cited to support them.  When an argument about an author&#8217;s qualifications <em>is</em> made, it is the first time the judge has heard that qualification&#8212;s/he does not have a context for incorporating this new dimension of  the argument into what s/he initially understood.</p>
<p>Returning to an example used in the initial article, there is a dramatic difference between a judge&#8217;s understanding of a piece of evidence describing the feasibility of clean coal technology from &#8220;Lucas &#8216;9&#8221; and their understanding of the same piece of evidence from &#8220;Lucas, spokesman for the American Coalition for Clean Coal Electricity, ‘9&#8221;.  If this piece of evidence is presented in the first instance without the author&#8217;s qualification, the opposing team&#8217;s assertion that &#8220;their evidence is from a coal industry spokesman&#8212;prefer our evidence because it is from independent scientists without a financial stake in clean coal&#8221; will be isolated in the judge&#8217;s mind from the initial argument about clean coal&#8217;s feasibility.</p>
<p>This is largely responsible, I think, for the difficulty that judges have in resolving arguments about evidence quality.  A similar refrain is heard from a broad swath of judges when questioned about these arguments: &#8220;debater&#8217;s don&#8217;t impact their author qualification arguments enough,&#8221; &#8220;I just think the argument in the card still makes sense,&#8221; &#8220;I don&#8217;t know what to do with this argument&#8212;I don&#8217;t want to throw out the card entirely,&#8221; etc.  This is a predictable response given the aversion most judges have to &#8220;intervention.&#8221;  When author qualification arguments are perceived as <em>distinct from</em> instead of <em>intrinsic to</em> the argument being advanced by a piece of evidence, it will be difficult for judges to avoid feeling &#8220;interventionist&#8221; about them.</p>
<p>Finally, the incorporation of an author&#8217;s qualifications into the understanding of an argument does not mean that certain authors need to be preferred.  In the above example, a debater could persuasively defend the authoritativeness of evidence from the American Coalition for Clean Coal Electricity in a variety of ways: it cites scientific studies, it cites the consensus of experts, the author has many incentives not to overstate his claims, the industry&#8217;s opponents are unqualified/relying on bad science, etc.  A rational decision-maker might decide to listen to evidence from a lobbyist or industry spokesman <em>despite their conflict of interest</em> because he or she made a good argument.</p>
<p>Indeed, Congress routinely solicits testimony from &#8220;biased&#8221; experts on both sides of an issue like clean coal.  Members of Congress and their staff listen to their arguments, read their prepared testimony and supporting materials, ask them questions, and then come to a conclusion about the issue at hand.  But the first thing that every expert does when testifying before Congress is introduce themselves and describe their affiliations/positions&#8212;in debate-speak, their qualifications.  This provides Congress with a context within which to understand their testimony.  Debate judges should demand the same context.</p>
<p>Gulakov continues:</p>
<blockquote>
<p>The norms you talk about in the introduction are not arbitrary. They did evolve logically: from debate being a competitive game. It is nice to quote an emphatic, totalizing phrase in cross–x (like “after examining every possibility, i conclude there is no other possible challenger to us heg”) to leave an opponent with no possible “but what about x” objection, or to say something like “our author clearly refutes your arg here: ‘While some may consider x to be the root cause of y, such logic is flawed for three reasons.’” </p>
</blockquote>
<p>The dominant features of &#8220;good evidence&#8221; were not selected at random, certainly.  But neither are they objectively correct.  </p>
<p>Totalizing rhetoric, for example, can be viewed not as <em>helpful</em> but as <em>harmful</em> to an author&#8217;s credibility.  Academic authors rarely write with the degree of certainty expressed by these hypotheticals (&#8220;after examining <em>every</em> possibility&#8221;), and for good reason: a degree of modesty about one&#8217;s conclusions reveals a thoughtful mind.   </p>
<p>Tim Alderete has explicitly discussed this in <a href="http://judgephilosophies.wikispaces.com/Alderete%2C+Tim" title="Tim Alderete's Judging Philosophy">his judging philosophy</a>:</p>
<blockquote>
<p>I am changing how I approach hyperbolic evidence. Particularly with all of the sources available on The Internets, way too many un-peer reviewed cards make ridiculous claims and use Extremely hyperbolic rhetoric. This isn’t only an issue of warrants or qualifications – some well qualified authors make ridiculous but warranted claims using hyperbole. My concern is that debate rewards this because power wording and extreme arguments are what make for &#8220;Good Cards&#8221; in debate. I am increasingly skeptical of this – I find it very hard to give the same weight to a screeching Weekly Standard Neo-Con that I give to a more reasonable author. Bottom line, if your author is hyperventilating about the blood of terrorists, or claims that his elixir can cure cancer, AIDS, racism and poverty, I think that the hyperbolic rhetoric is an Indict of that author, not just &#8220;power wording.&#8221; I don’t know exactly where to draw the line, I rarely will just ignore evidence, I don’t know how much of that needs to be made in the round – I don’t Know where I will end up on this. But for practical purposes, I think that I will reward teams that point this out, and I will make every attempt to apply the same standards to hyperbolic Kritik evidence, which I probably haven’t done in the past as well as I can. </p>
</blockquote>
<p>The latter of Gulakov&#8217;s examples (&#8220;x = y is wrong for three reasons&#8221;) is not really relevant to this discussion&#8212;obviously, authors that explicitly cite and refute another&#8217;s argument are fruitful sources of evidence for debates.  But this is really external to the question of whether an author&#8217;s qualifications should be read aloud.</p>
<p>Gulakov&#8217;s final objection is that the verbal citation of author qualifications forces debaters to jeopardize their strategic position:</p>
<blockquote>
<p>If debate rounds were structured more like a philosophy or lit dept conference, we’d obviously look for different things in writing. &#8230; Can you give me some examples of other times that debate norms changed towards something that is “unstrategic”?</p>
</blockquote>
<p>I disagree with the premise that the verbal citation of author qualifications is &#8220;unstrategic.&#8221;  As explained above, I do not think the outcome of a discussion of evidence quality is pre-determined: just because you&#8217;ve read a piece of evidence from a lobbyist, for example, does not mean that you will lose the given point.  To the extent that the verbal citation of qualifications discourages debaters from reading evidence which, if questioned, they would be unable to defend, however, this demand is desirable.  </p>
<p>To separate the source of the piece of evidence being cited from the argument that the piece of evidence is supporting undermines the hermeneutical process through which that argument is understood.  If improving our understanding of arguments is &#8220;unstrategic&#8221;, then we need to recalibrate the norms of our game.  </p>
<p>Gulakov proposes a &#8220;counterplan&#8221;: debaters should textually cite the qualifications of the authors they are citing and judges should consider evidence without such a cited qualification as equivalent to an analytical argument.</p>
<blockquote>
<p>Just having the quals in the cite is a better idea for a norm because it is more feasible to implement and doesn’t require the judge to already be convinced. This is not the status quo: the major change would be having judges regard evidence without qualifications (or a debater’s ability to provide one) as on the level of analytics. It would be quite easy for debaters to start putting quals in all their cards especially given this deterrence. </p>
</blockquote>
<p>I think this <em>is</em> the status quo, at least for some subsection of judges.  For the rest&#8212;those willing to evaluate evidence without qualifications and those who prioritize &#8220;having a card&#8221; over an analytical argument&#8212;this transition is undoubtedly an uphill battle, something I am more than willing to admit.</p>
<p>Compared with the initial proposal to normalize the verbal citation of qualifications, however, this counterplan falls short because it retains the hermeneutical wall between the argument advanced by a piece of evidence and the qualifications of the author the piece of evidence is citing.  The only unique disadvantage to verbal citation is &#8220;time loss&#8221;&#8212;but this is a red herring.  If establishing a norm in favor of orally citing an author&#8217;s qualifications results in better debates, then the time investment is justified.</p>
<p><strong>Conclusions</strong></p>
<p>This brings us full circle.  If it is true that developing more sophisticated ways of assessing the qualifications of evidence presented in policy debate rounds is desirable, then it is important to seek ways to actualize this outcome.  Given the authoritative role played by judges in determining the norms that debaters follow, then it is important that judges take the initial step to &#8220;nudge&#8221; debaters in this direction.  Enforcing an expectation that students verbalize the qualifications of the authors they are citing will bring the issue of author qualification into the hermeneutical process through which arguments are understood in a contest round, thereby encouraging students to develop innovative ways to assess scholarly authority.  The resulting proliferation of arguments regarding source quality and author qualifications will improve debates both <em>competitively</em>&#8212;by spurring better, deeper argumentation about evidence&#8212;and <em>educationally</em>&#8212;by training students to behave as intelligent consumers of information.</p>
<p>Additional comments&#8212;either here or <a href="http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/" title="Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate -- The 3NR">on the original article</a>&#8212;are appreciated.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.the3nr.com/2009/06/11/reading-author-qualifications-aloud-a-response-to-the-critics/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate</title>
		<link>http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/</link>
		<comments>http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/#comments</comments>
		<pubDate>Wed, 03 Jun 2009 17:26:44 +0000</pubDate>
		<dc:creator>Bill Batterman</dc:creator>
				<category><![CDATA[Columns]]></category>
		<category><![CDATA[Evidence/Research]]></category>
		<category><![CDATA[Judging]]></category>
		<category><![CDATA[evidence]]></category>
		<category><![CDATA[qualifications]]></category>
		<category><![CDATA[research]]></category>
		<category><![CDATA[sources]]></category>

		<guid isPermaLink="false">http://www.the3nr.com/?p=141</guid>
		<description><![CDATA[The State of Evidence Evaluation In Debate The recent discussions of evidence quality in high school policy debate have highlighted the need for debaters, coaches, and judges to revisit the prevailing assumptions about the proper role of cited material in our activity. While a drastic shift in the community&#8217;s approach to the evaluation of evidence [...]]]></description>
			<content:encoded><![CDATA[<p><strong>The State of Evidence Evaluation In Debate</strong></p>
<p>The <a href="http://www.the3nr.com/2009/05/08/new-affirmatives-and-source-credibility/" title="New Affirmatives and Source Credibility -- The 3NR">recent</a> <a href="http://www.the3nr.com/2009/05/11/sps-article-controversy/" title="SPS Article Controversy -- The 3NR">discussions</a> <a href="http://www.the3nr.com/2009/05/11/the-cult-of-evidence-and-the-importance-of-source-quality/" title="The Cult of Evidence and the Importance of Source Quality -- The 3NR">of</a> <a href="http://www.cross-x.com/vb/showthread.php?p=1725570" title="Standards for Evidence -- Cross-X.com">evidence quality</a> in high school policy debate have highlighted the need for debaters, coaches, and judges to revisit the prevailing assumptions about the proper role of cited material in our activity.  While a drastic shift in the community&#8217;s approach to the evaluation of evidence remains exceedingly unlikely, there is an emerging consensus among debate educators that improving this facet of our pedagogy is both possible and necessary.</p>
<p>What is the problem?  In short, the explosion of content enabled by new media has shattered traditional constraints on what constitutes &#8220;published&#8221; scholarship.  While debaters in past decades were limited in their research to published books, journals/magazines, and newspapers, the debaters of today have access to a nearly limitless stream of information&#8212;all at their fingertips, and searchable in ways never before thought possible.  As Gordon Mitchell describes in &#8220;<a href="http://www.the3nr.com/2009/05/20/debate-and-authority-30/" title="Debate and authority 3.0 -- The 3NR">Debate and authority 3.0</a>,&#8221; the resulting information abundance has created a need for new ways of separating the good from the bad.  </p>
<blockquote>
<p>Publication, previously a one-to-many transaction, has become a many-to-many enterprise unfolding across a complex latticework of internetworked digital nodes. Now weblogs, e-books, online journals, and print-on-demand book production and delivery systems make it possible for a whole new population of prospective authors to publish material in what Michael Jensen (2008), National Academy of Sciences Director of Strategic Web Communications, calls an “era of content democracy and abundance.” </p>
<p>In content abundance, the key challenge for readers and referees has less to do with finding scarce information, and more to do with sorting wheat from the proverbial chaff (the ever-burgeoning surplus of digital material available online). The pressing nature of this information-overload challenge has spurred invention of what Jensen (2007) calls “new metrics of scholarly authority” &#8211; essentially, new ways of measuring the credibility and gravitas of knowledge producers in a digital world of content abundance.</p>
</blockquote>
<p>Policy debate&#8217;s &#8220;metrics of scholarly authority&#8221; have developed slowly&#8212;changes in dominant assumptions about what constitutes &#8220;good evidence&#8221; have occurred over decades based on the organic back-and-forth of the contest round.  At the high school level, the influence of summer debate institutes and the trickle-down from intercollegiate competition have played a major part in this evolution.  While regional differences remain, the vast majority of those that participate in policy debate on the &#8220;national circuit&#8221; hold remarkably similar views about what makes a piece of evidence &#8220;good&#8221;.  Indeed, the dominant conception of &#8220;good evidence&#8221; has become so normalized that it is often framed as self-evident: good evidence &#8220;speaks for itself&#8221;.</p>
<p><span id="more-141"></span></p>
<p>But the evidence <em>can&#8217;t</em> &#8220;speak for itself&#8221;&#8212;judges must inevitably make sense of the evidence cited by debaters in order to render a decision in the face of competing arguments.  In a thought-provoking 2007 thread on the e-Debate listserv, Michael Antonucci revealed <a href="http://www.ndtceda.com/pipermail/edebate/2007-March/070098.html" title="[eDebate] Totally Tangential Reply To Branson -- Michael Antonucci">the impossibility of a neutral evaluation of evidence</a>.</p>
<blockquote>
<p>Judges enforce &#8220;evidence quality-control&#8221; <em>all the time.</em></p>
<p>It&#8217;s called reading evidence?</p>
<p>The standards that we apply when reading evidence aren&#8217;t neutral, natural valuations. The community&#8217;s evolved a fairly homogenized set of techniques, but this set isn&#8217;t logically inevitable. &#8230; Judges could apply a radically different set of  standards to evidence, if they so chose. &#8230; Tweaking the standards for good evidence would really be no more or less interventionist than current practice. Ultimately, you have to decide what a &#8220;good&#8221; card is, and it&#8217;s difficult for a debater to discuss every decision criterion you might employ in the course of [five] minutes. At some point, when you&#8217;re reading that card, you&#8217;re on your own.</p>
</blockquote>
<p>Antonucci argues that most judges <a href="http://www.ndtceda.com/pipermail/edebate/2007-March/070125.html" title="[eDebate] Tangential Thread: Reply to Hall, etc. -- Michael Antonucci">approach the evaluation of evidence</a> in more-or-less the same way.</p>
<blockquote>
<p>When reading evidence, judges generally apply a series of tests. How current is the evidence? Does it make a strong claim? Does it have a warrant? Does it appear to respond to the opponent&#8217;s claim? Does it have totally sweet violent imagery? Does it have supersweet 24 point font? Is that font Copperplate, which is clearly the best font ever? </p>
<p>The reading techniques that we currently consider &#8220;neutral&#8221; are anything but. They are artificial, stylized, and frankly a little bit weird. We value claims, or predictions, over warrants. We favor specific phrases and tropes, even &#8212; clean binaries, simple spatial analogies, and some fairly violent imagery generate a warm reception, for pretty subjective reasons.</p>
</blockquote>
<p>This generalized agreement provides an important benefit: because most judges evaluate evidence in the same way, students can enter a contest round with a predictable understanding of what kinds of research will be rewarded.  If the basic benchmarks for separating good evidence from bad were to change from round-to-round, students would be justifiably frustrated by inconsistent standards.  </p>
<p>At the same time, the dominant assumptions about evidence in debate should be subjected to steadfast scrutiny.  If the qualities that currently characterize &#8220;good evidence&#8221; are antiquated or simply <em>wrong</em>, they should be changed&#8212;slowly, perhaps&#8212;through an ongoing process that acknowledges the authoritative role played by judges in the creation and maintenance of community norms.  In other words, judges need to accept that their assessment of evidence is not&#8212;<em>and can never be</em>&#8212;neutral.  This acknowledgment is both <em>constraining</em> and <em>emancipating</em>: while it removes the facade of objectivity protecting judges&#8217; decisions, it also encourages students to experiment with different metrics for the evaluation of evidence.</p>
<p>The danger, of course, is that deviating from existing community norms will result in frustratingly inconsistent decisions.  But improvements in the way that high school policy debate approaches evidence do not require a descent into judicial anarchy.  Minor changes prompted by small shifts in community norms can have immensely positive results <em>without</em> jeopardizing the predictability upon which the legitimacy of contest round decisions relies.</p>
<p><strong>The Debate Judge As &#8220;Change Architect&#8221;</strong></p>
<p>In their 2008 book <em><a href="http://books.google.com/books?id=dSJQn8egXvUC" title="Nudge (2008) -- Google Books">Nudge</a></em>, noted University of Chicago professors Richard Thaler and Cass Sunstein argue that human beings&#8212;when left to our own devices&#8212;tend to make bad choices.  In order to encourage better choices, public policymakers should use &#8220;choice architecture&#8221; to encourage good choices and discourage bad ones.  In the context of high school debate, judges can be appropriately conceived of as &#8220;choice architects&#8221;: they have the responsibility &#8220;for organizing the context in which people [in this case, debaters] make decisions (Thaler and Sunstein, p. 3).&#8221;  While performing as choice architects, judges have enormous influence over the behavior of the students who are competing for their approval.  As the only decision-maker tasked with assigning a win or a loss, even seemingly insignificant changes in the way the judge approaches the debate can have a profound impact on the choices of individual debaters.  </p>
<blockquote>
<p>[S]mall and apparently insignificant details can have minor impacts on people&#8217;s behavior.  A good rule of thumb is to assume that &#8220;everything matters.&#8221;  In many cases, the power of these small details comes from focusing the attention of users in a particular direction. (Thaler and Sunstein, p. 3)</p>
</blockquote>
<p>In this article, I will argue that judges should exercise their discretion as &#8220;change architects&#8221; to encourage students to read the qualifications of their evidence aloud in debates.  This change, while minor, will have a profoundly positive effect on the quality of evidentiary analysis in high school policy debate.  Implementing this norm is simple: judges should insert the following text (or something like it) into their published judging philosophies.</p>
<blockquote>
<p>Debaters are expected to verbally cite the last names of the author(s), their qualifications, and the year/date of publication for each piece of evidence presented in the round.  Evidence that is presented without citation of the author(s)&#8217; qualifications will be assumed to be unqualified (&#8220;no quals available&#8221;).  Debaters are encouraged to advance arguments about the qualifications of the authors/sources cited in the debate.</p>
</blockquote>
<p>These expectations are not mandates: judges need not levy draconian punishments against debaters that do not follow them.  But debaters are easily influenced by the expectations of their judges, and change will occur quickly if teams begin to lose debates because their opponents exploit this opportunity to align themselves with a judge&#8217;s expectations.  The best&#8212;and perhaps only&#8212;way to improve the quality of the evidence read in debates and the discussion of that evidence is for judges to nudge debaters in the desired direction.  If judges are unwilling to alter their expectations&#8212;whether out of fear or an ideological commitment to &#8220;let the debaters be&#8221;&#8212;no change will occur.  If we care about improving the quality of evidentiary analysis in high school policy debate, however, this is an unacceptable outcome.</p>
<p><strong>The Case For Verbal Presentation Of Author Qualifications</strong></p>
<p>Debates about the qualifications of a given piece of evidence are <em>important</em> and <em>desirable</em>.  By verbalizing these qualifications during a contest round, their importance is amplified&#8212;<em>even if</em> students choose not to explicitly discuss them during a given round.  The current norm&#8212;in which only the last name of the author and the year of publication are read aloud&#8212;serves to de-emphasize the importance of qualifications in two important ways:</p>
<p>1. Excluding qualifications from verbal presentation implicitly de-values their importance when considering the quality of a piece of evidence.  If the author(s)&#8217; qualifications are not important enough to read aloud, after all, how important can they really be?  The current community norm emphasizes consideration of only the recency and content of a piece of evidence&#8212;the author&#8217;s last name is just a reference point.  Qualifications are just one of many things that are not deemed important enough to be read aloud, lumped in with the title of the book or article being cited and the page number or URL from which the cited material is taken.  </p>
<p>This does not mean that the debate community values the qualifications of an author equally with the page number from which a card was cut, of course.  But the message sent to debaters is clear: as long as it has an author and a date, a piece of evidence is worthy of consideration.  And not just consideration: <em>equal</em> consideration, at least until the other team challenges the quality of the source/author being cited.  Given this norm, debaters rightly prioritize evidence with outlandish claims and strong language over evidence written by qualified experts. </p>
<p>2. Requiring students to locate the qualifications of a given piece of evidence &#8220;privately&#8221;&#8212;during speech or prep time&#8212;prevents the judge from considering qualifications as part of their initial understanding of the evidence as it is being presented.  The judge is not merely an information processing machine: s/he experiences the presentation of arguments and evidence by debaters and actively participates in the process through which this rhetoric becomes meaningful.  When only a last name and date are verbally cited, the judge <em>cannot</em> consider the qualifications of a piece of evidence as they initially make sense of it&#8212;while judges may ask themselves questions like &#8220;does this evidence contain a strong conclusion?&#8221; or &#8220;is this evidence recent enough to account for relevant changes regarding this issue?&#8221; while they listen to the initial presentation of a piece of evidence, they are prevented from considering questions like &#8220;is this evidence from a qualified author?&#8221; or &#8220;how might this author&#8217;s qualifications affect the argument they are advancing?&#8221;.</p>
<p>Some critics of &#8220;interventionism&#8221; will undoubtedly contend that this exclusion is <em>good</em>: judges should not think about these things, they will argue, because it is the job of the <em>debaters</em> to make these arguments.  But this relies on the naive assumption that the judge <em>can</em> remove themselves from the process through which meaning is created from the back-and-forth of a contest round.  In his seminal defense of the &#8220;critic of argument&#8221; approach to debate judging, V. William Balthrop explained the crucial role played by the judge in the hermeneutical process.</p>
<blockquote>
<p>[A]t some point in time&#8212;either at the end of the debate or, more likely, during the presentation of arguments themselves&#8212;the judge is involved in interpretation, placing these positions into a scheme such that the importance of <em>some</em> evidence and <em>some</em> arguments can be identified relative to others.  It is here, at the point of interpretation, that the judge engages in the hermeneutical process.  It is here, that an awareness of the &#8220;rules&#8221; of debate <em>as that judge understands them</em>, of the content area of the debate <em>as that judge understands it</em>, of various judging paradigms <em>as that judge understands them</em>, all combine to influence the interpretation <em>by that judge</em> as to which team did the better job of debating.  The judge cannot withdraw from participating with the debaters in the creation of meaning, though it is a different kind of participation; and rather than admitting one&#8217;s biases and seeking to keep them from affecting one&#8217;s decision, a better perspective seems to be that which tests one&#8217;s biases against the debate itself and seeks to understand how they affect the decision. (&#8220;The Debate Judge As &#8216;Critic of Argument&#8217;,&#8221; originally published in the <em>Journal of the American Forensic Association</em> (1983), republished in <em>Advanced Debate: Readings In Theory Practice &amp; Teaching</em>, 3rd Edition, 1987, p. 170-171.)</p>
</blockquote>
<p>When author qualifications are not read aloud, they do not enter the hermeneutical process of the contest round unless/until an argument is advanced about them.  But this leaves these arguments weaker and less meaningful than arguments about the <em>content</em> of a given piece of evidence because its <em>content</em> and not its <em>qualifications</em> were factored in to the judge&#8217;s initial understanding of it.  Even if the judge enters the contest round intending to emphasize the importance of author qualifications in their assessment of the evidence presented by the debaters, they will be unable to do so <em>during the course of the debate itself</em> unless these qualifications are read aloud.  While the post-round review process provides an opportunity for the judge to unearth the qualifications of the evidence that has been read, the consideration of these qualifications is at this point artificial and removed from the debate itself.</p>
<p>This is not to say that the verbalization of author qualifications will give judges a green light to consider them in their decisions regardless of the arguments made by debaters during the round.  The onus remains on the student participants to discuss the importance of qualifications within their speeches.  But by enforcing an expectation that qualifications be verbalized as part of the initial presentation of a piece of evidence, judges can &#8220;nudge&#8221; students toward doing so more frequently and more productively while ensuring that the hermeneutical process through which a given debate is rendered meaningful <em>includes</em> author qualifications.</p>
<p><strong>Answering Objections</strong></p>
<p>Critics of my position will undoubtedly advance a variety of objections.  In what follows, I will attempt to respond to some of these objections while further developing the larger case for a change in the community&#8217;s evidence citation norms.  </p>
<p><em>Debaters can make arguments about author qualifications regardless of whether they have been read aloud.</em></p>
<p>This is true, of course: even when only a last name and date are cited aloud, debaters can advance arguments about the qualifications of a given piece of evidence.  But these arguments are necessarily separated from the evidence itself <em>as it was initially understood</em>.  Verbalization <em>at the time of initial presentation of a piece of evidence</em> emphasizes the importance of qualifications while inviting judges and debaters to take them into consideration as they first make sense of the arguments being advanced.  </p>
<p>In addition, time pressures should not be discounted.  Reviewing each piece of evidence read by one&#8217;s opponents in order to discern the authors&#8217; qualifications requires debaters to invest valuable&#8212;and scarce&#8212;time that could be spent on other forms of preparation.  The dearth of qualifications debates in the status quo seems to support that students will most often ignore qualifications in order to preserve valuable preparation time.  </p>
<p>A piece of evidence bestowing the benefits of clean coal from &#8220;Lucas &#8216;9&#8221; will be understood differently than the same piece of evidence from &#8220;Lucas, spokesman for the American Coalition for Clean Coal Electricity, &#8216;9&#8221;&#8212;and for good reason.  By normalizing the verbalization of author qualifications as part of the presentation of evidence in a contest round, judges can encourage students to take qualifications more seriously.  At the same time, this norm will ensure that an author&#8217;s qualifications are understood <em>as part of the evidence itself</em>.</p>
<p><em>Reading qualifications aloud requires scarce speech time: it will tradeoff with the reading of additional evidence.</em></p>
<p>This, too, is true.  But it relies on an unhealthy obsession with quantity over quality and with hyper-technicality over intelligence.  If the verbal presentation of author qualifications results in three or four fewer cards being read in each constructive speech, that seems like an acceptable cost when compared with the benefits of improving the quality of evidentiary analysis in debate.</p>
<p><em>Because discussions of author qualifications are difficult, students will inevitably choose not to engage in them.</em></p>
<p>So long as arguments about author qualifications are not perceived as &#8220;winners&#8221;, debaters will choose not to make them.  This is obvious.  But in their role as &#8220;choice architects,&#8221; judges have immense power to alter the behavior of students.  If judges expect students to read qualifications aloud, they will.  And if judges take qualifications arguments seriously when evaluating debates, students will make them more frequently.  Even in debates where they were not discussed, judges can play a positive role in their post-round commentaries by highlighting opportunities that students missed for advancing arguments about author qualifications.</p>
<p><em>The norm in favor of reading qualifications will be exploited by debaters&#8212;they will &#8220;highlight down&#8221; quals and &#8220;massage&#8221; them to make their authors seem more qualified than they really are.</em></p>
<p>This is a legitimate concern, for sure, but it is not damning to my position.  First, any manipulation of the norm in favor of reading qualifications aloud will still take place within a context in which more emphasis is placed on qualifications.  This provides a natural remedy for manipulative practices: because qualifications will be considered more important, those that attempt to disguise the shortcomings of their evidence will be subjected to heightened scrutiny.  Even in a worst case scenario, this is an improvement over the status quo in which qualifications are largely ignored.</p>
<p>Second, the ethical norms that guide debate research will remain a strong firewall against abusive manipulation.  The fabrication of an author&#8217;s qualifications should be treated in the same way as the fabrication of the text of a piece of evidence: as an unacceptable breach of the community&#8217;s trust.  In short, there is no unique risk of unethical behavior associated with the verbalization of author qualifications.</p>
<p><strong>Conclusion</strong></p>
<p>High school policy debate is a research-driven activity: we rely on the citation of published material to support our arguments about issues of public policy controversy.  In order to make sense of this &#8220;evidence&#8221; as it is presented in contest rounds, debaters and judges have developed a generalized agreement about the &#8220;metrics of scholarly authority&#8221; that determine what separates good evidence from bad.  But recent developments in new media have called into question the veracity of these metrics.  With a functionally limitless stream of information available at our fingertips, separating the good from the bad has become even more important than ever&#8212;and, unfortunately, more difficult.</p>
<p>In order to improve the quality of evidentiary analysis in debate, this article has advocated a shift in the community norm about the verbal citation of evidence.  Instead of expecting students to read aloud only the author&#8217;s last name and the date of publication for each piece of evidence, judges should exercise their discretion as &#8220;change architects&#8221; to enforce an expectation that the author&#8217;s qualifications be verbalized as well.  By adopting this minor change in expectations, judges can emphasize the importance of author qualifications to students while encouraging them to make arguments about qualifications more frequently and more seriously in contest rounds.</p>
<p>This &#8220;nudge&#8221; is no panacea.  Many issues will remain regarding the community&#8217;s approach to evidence even if reading qualifications aloud becomes normalized.  But this small step can have a profoundly positive impact.  By elevating the importance of qualifications and forcing debaters and coaches to take them more seriously before and during debates, this move by judges will contribute to the much-needed development of new &#8220;metrics of scholarly authority&#8221; in high school policy debate.</p>
<p><em>This is the second in a series of full-length essays published by The3NR.com.  If you are interested in syndicating or republishing these essays, please contact the author.</em></p>
]]></content:encoded>
			<wfw:commentRss>http://www.the3nr.com/2009/06/03/nudging-evidence-analysis-in-the-right-direction/feed/</wfw:commentRss>
		<slash:comments>15</slash:comments>
		</item>
	</channel>
</rss>

