Skip to Navigation
University of Pittsburgh
Print This Page Print this pages

April 16, 2009

Making Pitt work: Finals put Scantron, OMET to the test

Pitt’s senior administration grabs most of the headlines. The faculty here get noticed when they bring in research dollars, win teaching awards or publish in their fields.

But behind the scenes, University staff, some 7,000 strong across five campuses, often toil in jobs ranging from the mundane to the esoteric. From mailroom workers to data entry specialists, costume designers to biosafety officers, photographers to accountants, staff at Pitt perform tasks great and small, year-in and year-out, for the greater good of the University.

Like the proverbial purloined letter, some staff, such as secretaries, receptionists and maintenance workers, go unnoticed even though daily they plug away at their jobs in plain view.

This is one in an occasional series profiling University staff, providing a glimpse of some of the less recognized employees whose primary business is making Pitt work.

***

As students sharpen their No. 2 pencils and prepare for final exams, staff in the Office of Measurement and Evaluation of Teaching (OMET) are readying their Scantron machine for the busiest time of the term.

With the exception of the first week of the term, demand for machine-graded test scoring remains relatively steady, ranging from about 90 to about 140 requests per week, according to OMET statistics. However, during finals, requests pour in. The office may receive as many as 80 in just a day. In fall 2007, a typical term, OMET scored 428 different final exams for Pitt professors.

Many of the tests come from the sciences: Psychology, biology, physics, chemistry, pharmacy and the medical school are among the areas that frequently bring in Scantron sheets for grading. While some subject areas are more conducive to the multiple-choice format, OMET staffer Liz Wyman said class size is more of a factor: Professors with big classes are more likely to give multiple-choice tests.

Processing a typical week’s workload of scanning jobs may take about an hour a day, estimated Elaine Rubinstein, who sets up the jobs and feeds stacks of answer sheets into Pitt’s OpScan 10 machine. “Most of the time it’s manageable,” she said.

If answer sheets are brought to the OMET office by 5 p.m., professors can count on picking up the graded papers by 10 the next morning. Although there is a mail slot for latecomers who can’t arrive before the office closes at 5 p.m., Wyman said sometimes OMET staff can hear the late-day jobs coming as the sound of racing footsteps echo down the corridor outside OMET’s office on the Cathedral of Learning’s ground floor.

The actual scanning process is simple. Professors receive a blue claim ticket from OMET when they leave answer sheets to be scanned. That card or an ID is required in order to collect the graded tests.

Rubinstein sets up the scanning job on a computer attached to the machine, puts in the answer key and students’ forms and watches as they are processed.

Ordinarily, the OMET staff allow the jobs to accumulate throughout the day and run them as a batch late in the afternoon. The high-speed scanner — a noisy machine that is housed in a closet-sized room to keep the grinding and clanging noises it emits from disturbing others in the OMET office — can process a job in a matter of minutes. During busy times, two batches a day are run, Rubinstein said.

That all changes at the end of the term. “During finals week we scan all the time,” said Wyman.

OMET, which has five full-time staff, six part-timers (three of them graduate student assistants) and a contracted Computing Services and Systems Development employee, does more than test scoring. OMET administers student evaluations of professors. The office also provides testing services for admissions and certification exams such as the SAT, GRE and MCAT and offers consulting services to aid in research and survey design and analysis.

A 20-year veteran of the OMET office, Rubinstein officially is a research specialist. Her main job is to consult with faculty and graduate students about research projects and to help develop testing instruments and assist with data analysis.

Following the departure last May of the staffer who handled Scantron grading, “We’ve all taken turns doing it,” said Rubinstein, who since fall has taken the lead in processing the tests.

With master’s and doctoral degrees from Pitt’s educational research methodology program, she’s more than qualified merely to feed the Scantron machine’s hopper. She’s able to explain in detail the additional analysis that can accompany a job.

Faculty can request just the basics — student names and scores — but also can receive statistics, including standard deviation, the grading distribution curve and how consistently the students answered items across the entire test.

The analysis can reveal more than just data about the students; it can provide statistics on the test itself. Professors can learn which questions were easiest and which tripped up students most frequently. It also gives the frequency of keyed responses — how often each option A, B, C, D or E was the correct answer, for instance. Experts who teach how to create multiple-choice tests advise professors to keep the choices equal, Rubinstein said.

The University has just one optical grading machine to handle the workload. Should it fail, the entire process could grind to a halt. But OMET staff are proactive: Late in the term, they call in a service technician for a little preventive maintenance to ensure all goes smoothly once finals week puts the machine to its biggest test.

“It went out last fall,” Rubinstein said. OMET staff had to call all the faculty members who had orders pending to let them know their scores would be delayed.

“Fortunately, that doesn’t happen that often,” she said.

—Kimberly K. Barlow


Leave a Reply