A researcher using state data from the mCLASS reading assessment tool says it is a “surprisingly valid instrument,” particularly in reducing the third-grade racial achievement gap.
Researcher Gary Henry’s findings, presented Tuesday at the State Board of Education’s fall planning meeting, come as the state plans to switch to the Istation assessment tool this year.
Henry, who was contracted by the board to study early literacy data in low-performing schools, said his study found that racial achievement disparities on third-grade reading exams were reduced by more than 20% using mCLASS assessments. As part of the research, Henry’s team also asked the question: Is universal screening through reading assessment an intervention for students?
The researchers compared racial and ethnic reading gaps within each school before early reading assessments began and after. They said their findings, which compared results within the same school to control for outside factors, suggest that the mCLASS assessment closed the gap by about 22%.
“Importantly, these racial achievement gaps between white and nonwhite students, this gap was reduced not by white students scoring more poorly on the exams, but by an increase in the scores of black and Hispanic students,” said Jade Jenkins, a researcher on Henry’s team.
As part of the state’s Read to Achieve legislation, which aims to increase third-grade reading proficiency, teachers in kindergarten to third grade have used mCLASS to measure reading proficiency since 2013.
Asked whether the researchers found that mCLASS was a superior tool for assessing reading proficiency, Henry endorsed the product while acknowledging that others also might perform well.
“We try to use very careful language as researchers, but I would even strengthen that,” he said, responding to a question from State Board member Alan Duncan about whether mCLASS was a preferred tool. “This is a surprisingly valid instrument to predict later reading skills of these students. I did not expect anywhere close to this percentage of accuracy.”
Henry said the study did not compare mCLASS data against that of any other reading assessment tool. He said other reading tools might be reliable, but that reliability is different from validity. Based on his analysis of the data produced by mCLASS, he called mCLASS a valid, evidence-based assessment tool. He added that its ability to predict future reading proficiency is not common.
Supporters of Istation have pointed out that the state has employed mCLASS for several years, during which time third-grade reading scores have not improved. However, Henry said the job of the assessment is to provide accurate and valid data. At that point, teachers can target interventions that would improve reading proficiency. He said he thinks mCLASS does provide the data accurately.
He also said that if the state did switch tools, it could expect to lose some of the interventional benefits from using a reading assessment — because it would take time for every teacher to understand how to use the new tool properly.
“The instrument that we’ve seen here is a valid and reliable instrument,” Henry said. “I would say that any time these things are changed, there’s learning curves. To be fair, if the assessment instrument changes, you can probably expect that there’s going to be a lag in terms of producing [these results]. Even if it’s a valid instrument, there’s going to be a lag.”
At the time Henry and his team began the study, and throughout its five-year analysis, the state was using the mCLASS product for reading assessments.
In June, State Superintendent of Public Instruction Mark Johnson announced a move to the Istation assessment tool. Johnson was not in attendance during the presentation by Henry and his team.
After the switch to Istation was announced, Amplify, which sells the mCLASS product, protested that decision. Johnson and the Department of Public Instruction (DPI) have remained firm in their choice of Istation.
In August, the Department of Information Technology issued a stay in mandated use of Istation, putting the contract on hold until the decision and the process that led to that decision are reviewed. The parties appeared in court last month to argue whether the contract should have been put on hold and are scheduled to appear again next month.
After the stay was granted, Istation agreed to continue providing its product and training services to the state for free.
More than 400 K-3 teachers responded to an EdNC.org survey on the switch, raising concerns that Istation is not developmentally appropriate for young children, lacks appropriate dyslexia screening measures, and would not provide reliable data to inform reading instruction. Questions were raised around the procurement process as DPI released documents showing Amplify was ranked as the better vendor multiple times.
That survey ran on our Reach NC Voices platform from June 25 to June 27. Click here to see the full results.
