University Times

Knowing one’s place in the pecking order isn’t just for chickens. Who doesn’t like to be able to shout, “We’re No. 1,” or at least to know where he or she stands compared with the rest of the flock?

In academic circles, U.S. News & World Report is the annual arbiter of who stands where, and ranking near the top of the U.S. News listings is familiar territory for many Pitt programs.

In business schools, however, U.S. News isn’t the only player when it comes to rankings. Unlike other disciplines, business schools — and MBA programs in particular — are ranked by numerous publications, each using different criteria. And while the chance to shine is tantalizing, the pursuit of top rankings forces schools like Pitt’s Katz Graduate School of Business into a nearly constant —and costly — grind of compiling the data requested by surveyors.

In all, Pitt’s business programs receive eight to 10 rankings a year, said Katz Dean John T. Delaney. Each business school survey has its own focus. The Wall Street Journal focuses on recruiters’ assessment of students; the U.S. News survey is based more on static data and reputational assessment by deans; BusinessWeek is broader based, factoring in student evaluations and an intellectual capital component that assesses faculty; Financial Times adds an international component.

The upside to that many rankings: “If you don’t like what you see, wait a month,” Delaney said. The downside: the expense in both time and dollars of providing the data. “The cost of the ratings enterprise has gotten large enough for schools to weigh the value of the rating with the cost,” he said.

“We’re a relatively small business school, but we have to invest the same as the bigger schools,” Delaney said, noting the price tag runs into the six figures.

Because rankings draw attention — from students, parents, recruiters, alumni and donors — only schools with tip-top reputations such as Harvard and Wharton can afford to ignore them.

“It’s a prisoners’ dilemma,” said Delaney. “It’s critical to be there. If you drop out of the rankings, it creates negatives in people’s minds.”

Conversely, a good showing in the rankings can be a gold mine. “If you’re one of BusinessWeek’s top schools, it’s the equivalent of millions of dollars of publicity,” he said.

Since Delaney arrived at Pitt in August, multiple business school rankings have come out.

MBA program rankings placed the Katz School at No. 46 in a regional Wall Street Journal ranking and No. 73 in The Economist’s ranking.

The Katz executive MBA (EMBA) program was ranked No. 35 globally in Financial Times rankings released in late October.

But sometimes the effort expended in compiling the data doesn’t pay off in the form of a number. For the second time in a row, Katz has been shut out of the BusinessWeek MBA rankings.

The survey, conducted every two years, placed Pitt on its “also considered” list, meaning that the school wasn’t assessed due to inadequate response, this time on the student portion of the survey, said BusinessWeek associate editor Louis Lavelle. Lavelle said 20 student responses were received; 30 were needed in order to rank the school.

Pitt suffered the same fate in BusinessWeek’s 2004 ranking, although it had appeared on the magazine’s “second tier” listing in the prior four rankings. (Second-tier schools are those that didn’t place among the top 30 although they had sufficient response rates to qualify for the ranking.)

Delaney said the lack of student response in the most recent BusinessWeek survey likely is immaterial. “Even if the students had responded, I don’t know if we would have changed where we were,” somewhere beyond the top 30, he said.

But he added, “I will be very reluctant to take any grief about the rankings if the students won’t fill out the forms to get us ranked.”

However, the dean understands why students sometimes don’t respond. Delaney said it’s not unusual for an MBA student to be contacted for two or three media surveys in addition to internal benchmarking surveys. Confusion when some students receive multiple surveys may lead them to just say “forget it,” Delaney said.

“Everyone who wants to do a rating now wants to survey students,” he said, adding that the business school does not respond to ranking requests from minor publications in hopes of avoiding student burnout.

The dilemma has caught the attention of the Association to Advance Collegiate Schools of Business.

A 2005 AACSB task force report critical of the status quo labeled the numerous rankings and varied criteria “often arbitrary, selected based on convenience, and definitely controversial.

Characteristics that are of little importance are often included, while important characteristics are excluded because they are more difficult to measure. Even when the measures do correlate with quality, media attempts to draw significant differences among similar programs are inappropriate. Indeed, weights that are applied to different characteristics to determine ranks are subjective and generally not justified,” the report stated.

And, because MBA program rankings often are touted as “best-of” lists, the public’s view of what’s important in business schools has narrowed, the report stated.

“This diminishes the importance of faculty research, undergraduate programs and doctoral education and compels schools to invest more heavily in highly visible MBA programs.

“Many schools have re-allocated resources to activities that can enhance its ranking, such as marketing campaigns, luxurious facilities for a small number of MBA students, and concierge services for recruiters; but these gestures have little to do with quality. The result is an increase in the cost of delivering an MBA program, which generally translates to higher tuition for students,” the report stated.

“Rankings that rely on student or recruiter satisfaction can favor surface-level changes over substantive improvements,” the report cautioned.

In its recommendations, the task force proposed that AACSB serve as a warehouse for standardized data to take pressure off individual schools that now must provide different data to satisfy multiple publications’ requests. It also called upon AACSB to commission its own studies to address “the methods, validity and impacts of media rankings; the link between research, teaching and practice, and clusters of schools that are similar in quality.”

Delaney agreed that there is a need to standardize the data to make it useful. “People are trying to find simple ways to compare,” he said, adding that the “rankings” format is popular because society desires winners and likes the simplicity. “People want this product,” he said, admitting that the rankings dilemma will not be easily resolved.

Pitt’s recent rankings have varied: Its place in the WSJ poll has dropped from No. 22 in 2004 to No. 27 in 2005 to its current No. 46. At the same time, The Economist ranking has held steady. And, while Pitt’s MBA program hasn’t broken the Financial Times top 100, its EMBA program has risen four places from last year to No. 35.

“Every business school has a slightly different mission,” Delaney said, acknowledging that there is a demand for a means to compare programs, but recognizing that the current system is inadequate. “Simplifications can create a situation where the comparisons aren’t appropriate,” he said. More helpful, Delaney said, is to compare where a school ranks against the average in several polls to get the picture. “If you’re positioned well in each, it says something,” he said, adding that Pitt tends to be in the top 50.

“We’ve got some work to do,” he said, noting that rankings are a rearview mirror. No single number tells the story, but trends can. “If the trend is negative, it says we have to be concerned,” he said.

Rather than focus on boosting the rankings, Delaney said improvements in program fundamentals ultimately will affect Pitt’s position. “If you build a strong program, students will want to take that program,” he said, reasoning that graduates of a good program who have strong skills will make recruiters notice and the rankings will follow.

Currently, the surveys show that Katz students give an above-average assessment of the teaching, Delaney said. However, recruiters’ comments hint that the students need some more applicable skills.

Delaney said Pitt’s programs are strong in analytic skills such as valuation and accounting. But “soft” skills such as teamwork, leadership, conflict resolution and vision to see the big picture also are in demand. Such skills are especially important because they can’t easily be exported or replaced by technology. “I’m not sure we’ve been effective at providing these skill sets,” he said, adding that the curriculum is being examined to assess possible changes.

“Faculty are going to be concerned about the issues,” he said. “I think the faculty are going to pick this up.”

Some marketing research also is on the way. Delaney said he will ask faculty to compare the Katz curriculum against other local schools and top business schools. He also plans to seek feedback from faculty, students, the community, alumni and local business people to see what gaps may need filling and how well the students are being served.

“The focus is on the fundamentals: the teaching, the courses and satisfying students with the services they receive,” he said.

—Kimberly K. Barlow