SALEM, Ore. - State Schools Superintendent Susan Castillo announced this week the results of a comparability study of Oregon’s computer-based (Technology Enhanced Student Assessment, also called TESA) and conventional paper-and-pencil tests. The study showed that not only do groups of students receive similar scores on TESA and paper-and-pencil tests, but individual students receive similar scores regardless of the type of test given.
“This is good news for Oregon students and schools,” Castillo said. “As you know, Oregon had to drop its computer-based test because of a dispute with the vendor, and all schools are taking paper-and-pencil tests this spring. The results of this study show us that the scores students get on the paper-and-pencil test are not statistically different from the scores they get on TESA. That news should help everyone breathe a little easier as we wait for student achievement scores from this year.”
Some schools and districts have expressed concerns about whether the one-time changes in the testing system this year may create differences in the results on accountability reports this year as compared to last year. Superintendent Castillo expressed this concern to the U.S. Department of Education and Secretary of Education Margaret Spellings in meetings in Washington, DC in April.
“We are committed to implementing a fair and reliable accountability system, and we will work with our partners and stakeholders to identify any issues resulting from this one-time change in the assessment system,” Castillo said. “Secretary Spellings was very sympathetic to our situation, and she offered to see what assistance the US Department could provide the state when we see our test results.”
The comparability study was required as part of the Oregon’s peer review process. The U.S. Department of Education uses peer review, involving experts in the fields of standards and assessments, to determine whether states have met the testing requirements under the No Child Left Behind law. In 2006, Oregon’s peer review status was listed as “Final Review Pending,” and the state was given the opportunity to provide additional evidence to the US Department of Education, including the state’s answer to the comparability question.
Since the beginning of Oregon’s online testing system in 2001, the Oregon Department of Education has offered assessments using both TESA and conventional paper-and-pencil tests. Peer reviewers wanted to know if student scores on the two tests were comparable, since both types of tests were used to determine student achievement scores and to measure whether a school was making “Adequate Yearly Progress.”
The Department has conducted such studies in the past, both before implementing TESA and during its early years. For this year's study, more than 4,700 students from 95 schools across the state participated in a study designed to reconfirm the comparability of computer-based and paper tests. Each student took both the TESA exam and a paper-and-pencil form.
Based on the study, ODE found that the average scores and standard errors of measure were similar across TESA and paper-and-pencil tests with no statistically significant differences. In addition, the TESA and paper-and-pencil scores were highly correlated, indicating that not only do groups of students receive similar scores between TESA and paper tests, but individual students receive similar scores regardless of the mode of test administration. ODE has communicated the results of the comparability study to the US Department of Education as part of the required review of the assessment system.