<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><span style="font-size: x-large; letter-spacing: 0px;">...</span><span style="font-size: x-large;">knowledge is a source of bias</span><font size="5">…(Who knew??)</font><div><font size="5"><br></font><div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><p style="margin: 0px 0px 16.1px;"><span style="letter-spacing: 0px;"><b><font size="4"><a href="http://blog.coreknowledge.org/">The Core Knowledge Blog</a></font></b></span></p><p style="margin: 0px 0px 14.9px;"><span style="letter-spacing: 0.0px"><a href="http://blog.coreknowledge.org/2014/10/01/reading-test-developers-call-knowledge-a-source-of-bias/"><b><font size="4">Reading Test Developers Call Knowledge a Source of Bias</font></b></a></span></p><div style="margin: 0px;"><span style="letter-spacing: 0.0px"><font size="4">Lisa Hansel</font></span></div><div style="margin: 0px;"><span style="letter-spacing: 0.0px"><i><font size="4">October 1st, 2014</font></i></span></div><p style="margin: 0px 0px 12px; min-height: 16px;"><font size="4"><span style="letter-spacing: 0.0px"></span><br></font></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">You might expect to see a headline like this in the <i>Onion</i>, but you won’t. The <i>Onion</i> can’t run it because it isn’t just ironic—it’s 100% true.</font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4"><b>A few years ago, a researcher at one of the big testing companies told me that when developing a reading comprehension test, knowledge is a source of bias. </b>He did not mean the obvious stuff like knowledge of a yacht’s anemometer. <b>He meant typical K–12 subject matter.</b></font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4"><b>Since reading comprehension depends chiefly on knowledge of the topic </b>(including the vocabulary) in the passage, the student with that knowledge has a large advantage over the student without it. And since there have always been great educational inequities in the United States, <b>students’ knowledge—acquired both at home and at school—is very strongly correlated with socioeconomic status.</b></font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">A logical solution would be to test reading comprehension using only those topics that students have been taught. Teachers can do this, but testing companies can’t—how would they have any idea what topics have been taught in each grade? It’s rare for districts, much less states, to indicate what or when specific books, people, ideas, and events should be taught.</font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">Without a curriculum on which to base their assessments, testing companies have devised their own logic—which is sound given the bind they’re in. They distinguish between common and specialized knowledge, and <b>then they select or write test passages that only have common knowledge</b>. In essence, they’ve defined “reading comprehension skill” as including broad common knowledge. This is perfectly reasonable. When educators, parents, etc. think about reading comprehension ability, they do not think of the ability to read about trains or dolphins or lightning. They expect the ability to read about pretty much anything one encounters in daily life (including the news).</font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">I already had this basic understanding, but still I found the “<a href="http://www.parcconline.org/sites/parcc/files/FairnessReviewGuidelines.pdf"><span style="letter-spacing: 0px; color: rgb(2, 30, 170);"><b>ETS Guidelines for Fairness Review of Assessments</b></span></a>” eye opening. Guideline 1 is to <b>“avoid cognitive sources of construct-irrelevant variance…. If construct-irrelevant knowledge or skill is required to answer an item and the knowledge or skill is not equally distributed across groups, then the fairness of the item is diminished” (p. 8). It continues, growing murkier:</b></font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4"><b>“</b>Avoid unnecessarily difficult language. Use the most accessible level of language that is consistent with valid measurement…. Difficult words and language structures may be used if they are important for validity. For example, difficult words may be appropriate if the purpose of the test is to measure depth of general vocabulary or specialized terminology within a subject-matter area. It may be appropriate to use a difficult word if the word is defined in the test or its meaning is made clear by context. Complicated language structures may be appropriate if the purpose of the test is to measure the ability to read challenging material.</font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">“<b>Avoid unnecessarily specialized vocabulary</b> unless such vocabulary is important to the construct being assessed. What is considered unnecessarily specialized requires judgment. Take into account the maturity and educational level of the test takers in deciding which words are too specialized.”</font></span></p><p style="margin: 0px 0px 12px;"><span style="letter-spacing: 0.0px"><font size="4">On page 10, it offers a handy table that “provides examples of common words that are generally acceptable and examples of specialized words that should be avoided…. The words are within several content areas known to be likely sources of <b>construct-irrelevant knowledge.</b>”</font></span></p><div style="margin: 0px 0px 12px; font-family: Helvetica;"><font size="4"></font><br class="webkit-block-placeholder"></div></div></div></div></div></div></div></div></div></body></html>