Welcome back, gentle readers. In the previous installment of this two-part blog series, I described how the bipartisan No Child Left Behind Act (NCLB), with its focus on standardized test results, propelled American education into an era of testing mania. I cited examples of NCLB’s unfortunate consequences, such as a dramatic decrease in classroom instructional time due to excessive testing and test preparation (Robelen, 2016), as well as the ill effects on students and teachers from overemphasizing test scores (Simpson, 2016; Ryan et al., 2017).
Part 1 also included the story of the ebb and flow of compulsory local testing in the school district I retired from two years ago. I recounted how a few high-level administrators—thinking that preparing students to take standardized tests is tantamount to training marathon runners—required students to take unreasonably long tests throughout the school year, and how other central office staff poo-pooed the science of psychometrics and decided, even in the face of hard data, that their local assessments needed no changes.
After writing the first part of K-12 Testing: To Infinity and Beyond Reason, I remembered another anecdote that demonstrates the harm done to some students when test results are unduly emphasized. Several years ago, a reading specialist emailed me about a practice that had become commonplace at her elementary school. Through the magic of online testing, classroom teachers were able to monitor, in real time, their students’ responses to questions on quarterly tests. When a teacher noticed on her computer screen that a student answered a question incorrectly, she would cue the student until the right answer appeared.
The source of the problem, according to the reading specialist, was the principal’s data wall, which showed each teacher’s quarterly test results. Given the choice of being humiliated by the Wall of Shame and making questionable decisions in their classrooms, many teachers picked the latter. This practice was harmful to children who needed help from the reading specialist because she could no longer use quarterly reading tests to identify the school’s struggling readers. I wonder if the teachers who “assisted” their students during tests or the principal who called out teachers for imperfect test scores ever considered the short- and long-term consequences of their actions.
Bad judgment related to testing is rampant in schools and districts across the US. In her comprehensive report for the Center for American Progress, Testing Overload in America’s Schools, Melissa Lazarín (2014) stated, “there is a culture of testing and test preparation in schools that does not put students first” (p. 3). Experts maintain that testing separates children into winners and losers (Jablon, 2021; Kirylo, 2017; Wieman, 2021). One study indicated that elementary school students’ experiences with high-stakes tests affects “their continued engagement in school” (Dutro & Selland, 2012, p. 340), while a systematic review of research concluded that “motivation for learning … can be discouraged unwittingly by assessment and testing practices” (Harlen & Crick, 2003, p. 204). The adverse effects of America’s testing culture on economically disadvantaged students are more severe because “failure, as defined by test scores, is concentrated in low-income children of color” (Au, 2015, p. 52).
Despite problems that arise from their misguided focus on state-mandated testing, countless educators and policymakers remain preoccupied with standardized test scores. Many of these same people are now worried about children’s “learning loss” during the pandemic. The idea of COVID-19 learning loss gained a foothold just weeks after school closures were announced, when a research brief forecast the impact of shutdowns on student learning (Kuhfeld & Tarasawa, 2020). The brief was published by NWEA—formerly known as the Northwest Evaluation Association—and the authors were NWEA employees. It comes as no surprise that NWEA is an organization that “supports students and educators worldwide by creating assessment solutions” - in other words, a testing company.
The media continues to hype learning loss (e.g., CNBC, 2021; Wall Street Journal, 2021). In our nation’s capital, FOX 5 DC reported, without citing a source, that “one-third of students need to repeat a school grade” (2021). It was inevitable that private testing and test prep companies would come up with products to measure and fix children’s learning loss. Responding to the studies that initiated this craze, the president of Math for America stated that “learning loss is a calculation masquerading as a concept—a rather shallow, naïve, ridiculous concept” (Ewing, 2020, para. 9). The ridiculousness of the entire episode is illustrated in the satirical fable, Henny Penny Discovers Learning Loss (Walsh, 2021).
Recently, a number of educational leaders have advocated for increased testing as the answer to learning loss. Not everyone can afford commercial products, so some districts are using locally developed assessments to gauge how much learning was lost and predict students’ performance on end-of-year state tests. Herein lies the problem: Most locally developed tests are neither accurate nor reliable measures of learning, nor do they serve as trustworthy predictors of future performance on state-mandated assessments (Brown & Coughlin, 2007). Consequently, the results of these local tests are not valid for any of their intended purposes.
Why are local assessments lacking? To put it succinctly, “varying technical capacity at the local level severely limits the potential quality of local assessments” (Barone, 2021, para. 7). What is meant by “varying technical capacity at the local level”? It means that the vast majority of district personnel charged with creating local assessments have not had much, if any, formal training or practical experience in test design and development. As a result, local assessments tend to be built piecemeal instead of with carefully considered blueprints. Frequently, they have not been scrutinized by expert reviewers, and typically, they have not been piloted, field tested, or revised before being rolled out to the masses.
In lieu of local assessments, many districts administer older versions of state-mandated assessments, known as released tests. The idea is that students’ scores on released tests can predict students’ scores on current state tests. Some districts crunch numbers to determine the relationship between the scores of students on released test and the same students’ scores on corresponding state tests, in the mistaken belief that a positive correlation is sufficient proof of predictive validity. In Virginia and other states, the main reason for releasing old tests and test questions to the public is “to assist in understanding the format of the tests and questions” (Virginia Department of Education, 2021, para. 1), not to predict a student's performance on the end-of-year test.
By now you might be wondering, what can I—as a teacher or a parent or a school administrator—do when children under my purview spend too much time in class doing test prep, taking district tests, or both? For answers, let’s turn to scholar and prolific writer Alfie Kohn. Even though his comprehensive article, Fighting the Tests: A Practical Guide to Rescuing Our Schools, was published over 20 years ago, the advice is still relevant.
Here is Kohn’s general counsel to stakeholders: “We must do our best in the short term to protect students from the worst effects of a given policy, but we must also work [long term] to change or eliminate that policy” (2001, para. 9). His short-term recommendations for educators who work in state-test-obsessed schools or districts can be summarized as follows:
“Do what is necessary to prepare students for the tests – and then get back to the real learning.”
“Do no more test preparation than is absolutely necessary.”
“Whatever time is spent on test preparation should be as creative and worthwhile as possible.”
“Never brag about high (or rising) scores…. it serves to legitimate the tests.”
“Whatever your position on the food chain of American education, one of your primary obligations is to be a buffer – to absorb as much pressure as possible from those above you without passing it on to those below.”
I have another suggestion for teachers and parents who believe that local testing or test prep has gotten out of hand at their school. From my own teaching and parenting experience, I’ve found that most principals are reasonable people. Most high-level administrators at the central office were once principals. If your principal or superintendent has an open mind, don’t be afraid to discuss with them the hypocrisy of advocating for ideals such as educational excellence and student well-being while the actual priority is test scores. And don’t forget your local school board, which (if you’re lucky) includes several reasonable, open-minded people.
Our closing quote is from an article titled Too Much Focus on ‘Learning Loss’ Will Be a Historic Mistake. The author’s rhetorical question, asked in the wake of the inequities brought to light during the pandemic, is this:
Can we—should we, in the aftermath of the clarifying events of the last year—find the will to challenge the testing regime, return some agency to both our teachers and our students, bring the science of learning into our classrooms, and honor all children with challenging, engaging work that ushers in a new, better, fairer era in education? (Merrill, 2021, para. 20)
NOTE: Readers may incorrectly assume that I’m against testing. I believe that both commercial and locally developed assessments have a place in schools, but they must be psychometrically sound and appropriately used measures of student learning. Please feel free to leave comments.
