In Part 1, I explained that the purpose of this blog series is to provide updates to my June 2021 journal article. In the article, I analyzed the criteria for identifying gifted students at a large school district and the procedures used to admit children to the district’s exclusive gifted school. These criteria and procedures hinder the equitable representation of Black, Latine, and economically disadvantaged students in the district’s gifted programs.
The underrepresentation of certain identity groups at the gifted school is best illustrated with statistics from the Virginia Department of Education (VDOE) on its Fall Membership Build-A-Table page. The percent of Black students in the district is just over 23% for the current school year, yet the percent of Black students at the gifted school is under 6%. The percentages of Hispanic* students enrolled in the district and the gifted school are approximately 13% and 6%, respectively, while the gap between the district’s economically disadvantaged population and the school’s economically disadvantaged population is nearly 30% (43.6% to 14.4%).
Many people in the community continue to ask the question, “When will our school district live up to the promise of its stated core values with respect to these imbalances?” The district’s core values are:
Put Students First.
Seek Growth.
Be Open to Change.
Do Great Work Together.
Value Differences.
Are the district leaders merely paying lip service to their core values? The racial/ethnic and economic disparities at the gifted school make it clear that economically advantaged White and Asian students are put first. Before this problem can be fixed, decision makers at the central office need to grow (not just seek growth), change (as opposed to being open to change), and prove that they value differences by doing what is right (“great” work will naturally follow).
Universal Screening: Not Cheap and Not Culturally Neutral
Part 2 of this blog series examines universal screening for gifted services in the district. Universal screening is defined as “the process used to provide an assessment for students of one or more grade levels at the same time” (Davis, 2018). The practice is intended to identify potentially gifted students in an entire grade level. Although universal screening has been shown to increase the representation of economically disadvantaged and culturally and linguistically diverse students in gifted programs (Card & Giuliani, 2015), it is costly in terms of time and money (Lakin, 2016).
Every fall, itinerant gifted assessment specialists from the district of interest’s central office kick off the universal screening process by going from school to school to administer standardized tests to every first- and fifth-grade student. With 55 elementary schools in the district, the number of students who participated in universal screening during the 2021-2022 school year exceeded 9,000. A conservative estimate of the annual cost of universal screening, including specialists’ salaries, is well over $200,000. (I was unable to find the actual figure and invite readers with more accurate information to provide it in the comment box.)
The Naglieri Nonverbal Ability Test (NNAT) is the district’s universal screening tool. According to its author, Jack Naglieri, the NNAT is “a nonverbal measure of general ability comprised of progressive matrix items that utilize shapes and geometric designs interrelated through spatial or logical organization” (Naglieri & Ford, 2003). Below is a sample test item from the NNAT Level B assessment designed for first graders. I encourage readers to attempt the item and then see if your answer is correct by scrolling to the bottom of this post.
The NNAT—now in its third edition—is marketed by the multi-billion-dollar conglomerate Pearson. The NNAT brochure states that the test is a “culturally neutral assessment of general ability …. particularly useful for assessing minority [sic] students and those who might be English language learners or who may have limited academic skills” (Pearson, 2018). Not all measurement experts agree with these claims.
While so-called culturally neutral ability tests like the NNAT do an adequate job of controlling for the effects of language, Rhodes, Ochoa, and Ortiz (2005) maintain that they do not control for the effects of culture. In other words, children who have experienced American schools and standardized tests do better on the NNAT than children who do not have these experiences, regardless of the language they speak. In addition, studies have shown that children who qualify for free or reduced-price lunches score significantly lower on the NNAT than children who are not eligible for free/reduced-price lunches (Lohman & Gambrell, 2012).
Speaking of Inexperienced Examinees
For better or worse, most American fifth graders are used to taking standardized tests. This is not the case with students in first grade, the other grade level that the district of interest screens for giftedness. During my first-grade teaching days in Georgia, there were problems every year when I administered the Iowa Test of Basic Skills to my classes. The problems included test anxiety and other issues that affected the reliability and validity of my students’ scores.
Compared with their older peers, first graders are remarkably unreliable test takers (Gao & Grisham-Brown, 2011). The social-emotional, cognitive, and physical developmental needs of first graders do not align with the demands of taking standardized tests. Young children tend to develop rapidly and unevenly, and they are used to sitting and focusing on a single activity for only short periods of time. Because standardized test taking is a new experience for most 6- and 7-year olds, they have yet to master test-taking skills. All of this means that a first grader’s test performance can vary from day to day, which also means that their test scores do not necessarily represent their actual abilities.
Examiner effects is another problem that can arise when standardized tests (e.g., the NNAT) are administered to first-grade students. Research indicates that African American, Latine, and economically disadvantaged children perform better on tests when they know the examiner than when the examiner is an unfamiliar person – for example, an itinerant assessment specialist (Fuchs & Fuchs, 1986; Fuchs & Fuchs, 1989).
Practice Makes Better
The goal of many parents in the district is to get their children identified for gifted services ASAP. A lot of these parents hope that their child will also be accepted to the gifted school. These parents realize that their child must score at the 90th percentile** or higher on the NNAT so their school’s “gifted resource teacher (GRT) will contact [them] … via phone, email or letter to discuss the next step in applying for gifted programs,” according to the district’s website.
These parents also know that practicing for the NNAT will increase their child’s score on the test and that free NNAT sample questions and practice tests are readily available at various sites on the internet (e.g., TestingMom, tests.com, TestPrep-Online, MercerPublishing). To further boost their kids’ test scores, economically advantaged families purchase additional NNAT practice materials, while economically disadvantaged families have to prioritize basic needs over preparing their kids for a standardized test. Concerning the availability of test prep materials and the magnitude of practice effects on tests such as the NNAT, Lohman and Gambrell observed that “parents can easily give their child an advantage on the screening test, thereby further disadvantaging the disadvantaged” (2012).
“Presenting items similar to those on the test” used to be considered unethical (Haladyna, Nolen, & Haas, 1991); however, that’s no longer true. In a book published last year, Marzano et al. stated that “familiarization with the features of test items constitutes what testing experts regard as ethical test preparation” (2021). I argue that it is blatantly unethical when some students are given sample NNAT tests in advance and others are not. Why does the district turn a blind eye to the fact that some students’ NNAT scores are inflated because they practiced for the test?
Advance preparation for the NNAT would not be an issue if there were no educational consequences attached to the results of the test. But a student’s universal screening score becomes part of their portfolio when they apply for admission to the gifted school, which means that a bump in a child’s NNAT score due to test prep could be the difference that gets the child into the school. Based on the predominance of middle- and upper-class White and Asian students at the gifted school, it is likely that more than a few were accepted due in part to high NNAT scores attained because these students had access to the best test prep money can buy.
Is the Juice Worth the Squeeze?
Research suggests that NNAT scores may not reflect the true abilities of children of color and economically disadvantaged students, not to mention young children in general. The district needs to determine whether the NNAT is helping or hindering efforts to diversify the district’s gifted programs. This can be done by examining the percent of students whose NNAT scores are at or above the 90th percentile by ethnicity and socioeconomic status, and then looking at the percent of students in each group who qualify for gifted services, as well as the percent who are accepted for admission to the gifted school.
If the district decides to continue using the NNAT for universal screening, all elementary school students and their caregivers should be given information about accessing practice tests for the NNAT. It doesn’t matter if students practice for the test; it does matter if some kids perform better than others on the test due to inequitable access to test prep materials.
As I stated in my 2021 journal article, universal screening with the NNAT can proceed provided the test is supplemented with another screening method. The additional screening method that would work best for increasing diversity, equity, and inclusion (DEI) in the district’s gifted programs is teacher observation and referral, but only after teachers receive comprehensive DEI training and professional learning in areas such as implicit bias, cultural competency, and anti-racism.
Effective DEI training allows open-minded educators to recognize their own biases, and because studies demonstrate that most teachers identify the exceptional talents of White and Asian children, targeted training will improve their ability to observe gifted characteristics in children from Black, Latine, and Indigenous cultures (American University School of Education, 2021). Although DEI training is conducted regularly by the district, there is currently no measure of its effectiveness. Again, I invite readers who can address this to comment in the text box below.
First- and fifth-grade teachers should also be trained to administer the NNAT to their students. Administering standardized tests is nothing new to experienced teachers and—given the research on examiner effects—it’s better for students to be tested by the teachers they know than the central office staff they don’t know.
Turning over universal screening test responsibilities to teachers will free up the district’s gifted assessment specialists for other work. Knowledgeable and well-trained specialists could visit elementary schools to assist GRTs and teachers “identify students from underrepresented, underserved, and underresourced populations.” The quote in the preceding sentence came from the district’s Local Plan for the Education of the Gifted 2020-2025, a lengthy document that school districts must submit to the VDOE in compliance with Virginia law. I will return to the 5-year plan, as it is commonly called, in future installments of this blog series.
The last recommendation comes directly from my journal article: “Communicate to all parents and guardians at the time they receive [NNAT] screening results that, regardless of their child’s score, they can still refer their child for further assessment for gifted services.” Communication with all caregivers at all schools about all aspects of the district’s gifted services is essential and should not be solely the GRT’s responsibility. To achieve equity in the gifted programs, all educators in the district must be ambassadors for these programs.
Part 3 of “Put Students First. Seek Growth. Be Open to Change” addresses the next steps after a student’s gifted application has been submitted. If you want to comment, ask questions, or provide relevant information as this blog series progresses, please use the “Leave A Comment” box below or email me at doug@edjacent.com.
E is the right answer to the sample NNAT item. How long did it take you to figure out which choice was correct? How long would it have taken you if you had never seen a standardized test before? Do you think that first graders who have had practice with items like this do better on the timed NNAT than first graders who have never seen such items? Feel free to reply in the comment box.
* “Hispanic” is the term used on the VDOE’s Fall Membership Build-A-Table page in the drop-down menu for “Race.” I will continue to use the more inclusive, gender-neutral term “Latine” throughout this blog series.
** Percentile is not the same as the percent answered correctly on a test. Percentiles indicate each test taker’s relative position within a group of people who have taken the same test. If a student scores at the 90th percentile, it means that the student’s performance on the test was better than 90 percent of the testing group. More information here.
