A test can cause a student to lose sleep, but Dave Matt had a nightmare of his own recently during an online assessment at Orange County Public Schools in rural Virginia.
A faulty local area network switch inside a middle school computer lab shut down the test for all the students in the room for 90 minutes while Matt, the county school system’s director of technology, and his staff repaired it.
Still, he’s enthusiastic about online testing. “It’s the way kids interact with the world these days. It’s definitely the way to go,” he says. “The thing is: nothing can go wrong. The network has to work, the computers have to work, and the Internet connections have to work.”
Testing 1, 2, 3
With federal funding riding on test scores, student performance is crucial. So is getting the technology right—a challenge for a small school district like Orange County, which has 4,800 students and a modest IT budget.
Matt’s motto is, “Do whatever you have to to improve reliability and redundancy.” Orange County recently upgraded its network storage capacity to RAID [redundant array of independent disks; a system which uses multiple hard drives to share data to improve fault tolerance and increase performance] and Matt keeps switches and other components handy in case there’s a problem. “On testing days I’m glued to my network monitoring,” he says. “I have all my vendors alert and ready to go.”
He also keeps on hand about 10 laptops that are preconfigured with the test in case a computer crashes, but one of his biggest hurdles is simply having enough of the latest PCs. The county is buying new computers, but rather than increase the total number of available machines, they are replacing older Windows 98 models that lack sufficient memory and aren’t compatible with some tests.
Bandwidth is not much of a concern, because all tests are downloaded onto PCs before the students show up. “The only thing that goes back and forth is the kids’ answers,” Matt says. Test traffic gets the highest priority on the network, but the total amount for 100 students is less than 10 kilobytes. “It’s very low. It’s spooky low. I would expect it to be way higher.”
If the Internet connection should fail, there’s not much he can do. Redundant T1 lines exist, “but it takes time to change routers to direct traffic from one T1 line to another.”
Performance Enhancers
Precisely because schools can’t control bandwidth, assessments provided by the Northwest Evaluation Association do not depend on the Web. Instead, the nonprofit group, which works with roughly 2,000 school districts around the country, uses a server application that downloads the test and then delivers it to individual student computers.
“Our standard is a child never waits for more than one second for a question to load,” says Allan Olson, president and CEO of the NWEA, which has designed computerized adaptive tests that adjust in difficulty according to student performance.
Olson believes that student information systems are an important ingredient of assessments that some schools may overlook, failing to keep theirs as up-to-date as they should. With turnover rates as high as 50 percent in some schools— but more likely 10 to 20 percent, says Olson—keeping accurate tabs on enrollment, class data, student identification, and other personal information can be difficult.
“If a teacher receives a new student and wants to test that student, but the information system doesn’t know that student is assigned to the teacher, the system wouldn’t be able to test that student,” Olson says.
NWEA asks its schools to update their systems at least once a quarter, but Olson believes they do it at least weekly.
What about Training?
Staff development is also key with any assessment. Olson, however, is less concerned with administering the test than he is with responding to the results, something he says is lacking. “All those people in management roles that should be attending to the student learning experience should be knowledgeable about the data, how to interpret it, use it—how to change what they do,” he says.
But not enough schools are asking assessment vendors to provide specific data that would allow them to pinpoint, for example, beginning readers in 5th grade or algebra-ready students in 6th and assign special staff accordingly.
“Directors of finance should have achievement data to help understand how to assign resources to classrooms in schools based on need,” he says. “Directors of personnel should know achievement characteristics of students in each school to know which school needs particular [instructor] skill sets.”
The focus on online assessments has led to increased teacher training in Orange County, Virginia, not just for test days. The state government there has mandated new positions for “technology integration specialists” to show teachers how to use things like multimedia projectors in their classrooms and Microsoft Outlook to communicate with parents.
“Teachers by and large aren’t that involved in online testing. They might be proctoring the test, but the minute they see something go wrong, they call us,” Matt says.
In the end, the work he does to improve the reliability of online testing pays off the rest of the year, too. “Maybe five years ago, my whole network would go down and I’d be the only one who noticed,” he says. “Now, it’s a major national emergency.”
Chris Heun is a New York–based freelancer who also writes for InformationWeek and InternetWeek.
