Senate plenary session:
Teaching excellence & tenure
All discussions of tenure and promotion for faculty at Pitt must derive from the premise that teaching excellence is a fundamental component of the University’s mission.
That was the common theme for speakers and panelists April 14 at the University Senate’s spring plenary session, “Teaching Excellence as a Criterion for Promotion and Tenure.”
Although the three panelists came to the discussion from widely varying perspectives — representing Arts and Sciences, the regional campuses and the School of Medicine — certain parallels emerged regarding the definition and importance of tenure to faculty members’ careers and to Pitt’s reputation as a community of scholars.
The plenary session was hosted by the Senate tenure and academic freedom committee and moderated by committee chair Carey Balaban, who is a professor of otolaryngology, neurobiology, communication science and disorders, and bioengineering.
In introducing the panelists, Balaban said Pitt should be proud of its culture of teaching excellence.
“So, this discussion is central to the University, but it is intended to help raise our consciousness rather than to be an end-all,” Balaban said.
The University’s mission is the best starting point, Balaban said.
“Effective education really is a central value implicit in every part of the mission statement, and furthermore it is part of an institutional culture. Knowledge is acquired through research, though synthesis, through practice and through teaching. In fact it’s not just knowledge, but know-how, how to solve problems. We have a faculty-based culture that holds teaching excellence to be a fundamental component. Inspired teaching keeps the flame of scholarship alive,” Balaban said.
Arts and Sciences: Measuring & evaluating teaching excellence
Panelist James Knapp, senior associate dean in the School of Arts and Sciences who has served on many tenure evaluation committees, began by reiterating the section on tenure in the school’s bylaws, which were developed in the 1970s.
Within Arts and Sciences, Knapp said, “Tenure is awarded for demonstrated excellence — together with the promise of continued excellence — in scholarship, in whatever form that scholarship takes. Teaching and research, or creative activity, the two principal functions of the University, are also the two principal forms of scholarship. The relationship between the two is complex, and no single formula could serve as an adequate guide in every tenure case.
“Excellence in research should not exclude the competence of teaching, and teaching that is not founded in scholarship can make no claim to excellence.”
The school’s bylaws are not just an old graying document filed away, he said. In fact, the tenure procedures in the bylaws are reviewed at all evaluation committee meetings, “to underline that those are the boundaries we’re interested in when we make those crucial decisions,” Knapp said.
“That’s important first of all because it does establish that excellent teaching is a long-standing value of Arts and Sciences. But there’s another interesting thing and that is the notion that teaching is complex; it’s not a simple thing,” he said.
Part of its complexity is due to the abundant diversity in the modes of education delivery, including the wide range of class sizes and formats. “But teaching also takes place in science labs, in arts studios, in architecture studios, on stage in the theatre. It takes place in writing workshops, where people critique each other’s work. It takes place in undergraduate research,” Knapp said.
Teaching also includes supervising dissertations, advising and student and peer mentoring, he added.
“Some of these things are pretty hard to measure, but the point I would make is that important teaching is taking place in all of them and because teaching is a central value of the University, we need to learn to evaluate it very well,” Knapp said.
However, postmortem evaluations of teaching are only part of the big picture, Knapp said. “We need to begin the day faculty arrive on campus and we need to expect them to be, or more realistically to develop into, excellent teachers and we need to make it possible for that to happen. So we’re looking at a process that should begin as soon as they arrive and continue right on through the years,” he said.
How does Arts and Sciences take on such a tall order?
“There are several ways. One way, shared across the University, is we evaluate our faculty on a regular basis,” including in annual reviews by department chairs and in contract renewal discussions, Knapp said. “What we want to do in Arts and Sciences is to consider those evaluations within a process that reinforces the importance of teaching well and builds a culture in which excellent teaching is something the faculty will perpetuate.”
Knapp leads two workshops a year on the tenure process and requirements for tenure-stream faculty. At the fall workshop for new hires, he tells the new faculty to start a teaching portfolio. “It’s very important to be thinking about how am I going to be able to show somebody a few years from now that I am an excellent teacher? How am I going to document that?”
As part of a faculty member’s dossier, Arts and Sciences requires a teaching statement. “That statement is an opportunity to inform the evaluators about the candidate’s teaching. In one sense it’s a chance to make your case and say: This is what I tried to do in my teaching and this how I think about it,” Knapp said.
Perhaps more important, he said, this statement requires people to reflect seriously on their teaching and to think about what their difficulties are, to begin to develop their own philosophy of teaching.
In addition to documenting teaching, Knapp said, faculty dossiers must include a curriculum vitae, a research statement and Office of Measurement and Evaluation of Teaching student evaluations.
“But we also encourage people to put in other supplementary materials,” he said, such as course syllabi, course goals and teaching strategies, feedback from students, sample examinations, lecture notes, sample readings and writing assignments, including resulting student papers, and student-generated discussion questions.
“As far as the workshop goes, I’m trying to give as clear as possible a sense of what we think good teaching is,” he said, returning to the school’s bylaws for reference:
“In judging excellence,” Knapp quoted, “the indispensable ingredient to promotion to tenured rank should be creative intellectual vitality as reflected in the candidate’s teaching and in the candidate’s contribution to the advancement of knowledge or in his or her artistic activity. … These should be assessed with the evidence they reveal of intellectual power and originality. Tenure is not a reward for past services, but a kind of contract, a lifetime of security in exchange for a lifetime of continued creative scholarship.”
Knapp said, “So, intellectual vitality, creative scholarship — those are the key words. We’re interested in what you did the last five years, but we’re more concerned with the next 30 years, to see the kind of person who really cares about teaching, who’s going to keep doing it.”
Experience has taught Knapp that tenure candidate evaluations rarely are cut-and-dried.
“If we’re looking at teaching evaluations, for instance, and we have one that shows consistently good evaluations, versus the one whose evaluations are up and down and all over the place, that second person might be more interesting” because the fluctuation might reflect the candidate’s desire to try different things, even if they are sometimes unsuccessful, he said. “Maybe that’s the more interesting person 30 years from now, and that’s what we’re trying to figure out. It’s very hard to figure that out, but that’s what we’re trying to do.”
Arts and Sciences also has several built-in mechanisms to aid in a faculty member’s development as a teacher, he noted, including a requirement that every new tenure-stream hire be provided with appropriate mentoring.
“We also require peer review of teaching, so that your colleagues in the department have to come to your class from time to time. We make it clear that that doesn’t mean it’s like a movie review: ‘Well I liked that great chase scene.’ But rather it’s someone who learns about the class, talks to the faculty member beforehand, looks at the materials, talks with you after and really understands the much more holistic context of the course. That’s something that puts you in touch with your colleagues around the issue of teaching,” he said.
The workshop Knapp leads in April is for those candidates who are up for tenure and facing an evaluation. In that workshop, Knapp also invites the department chairs. The goal is to make clear to the candidate and the chair what will be reviewed, so that everyone is on the same page and knows what to expect, he said.
Pitt-Greensburg: A liberal arts college perspective
UPG President Sharon Smith drew on her experience as the associate director of the Project on Faculty Retirement, a study commissioned in the 1980s by the Higher Education Association to examine the effects of the elimination of the federal mandatory retirement age for educators.
“That position … really colored my thinking on the nature of institutions of higher education, what the meaning of tenure is and the role of faculty in those institutions,” Smith said.
“In performing the study it was important to understand the nature of the different institutions that we believed might be seriously affected: research universities, where teaching loads tend to be lighter and the value of the connection to a particular institution tends to be much higher; and liberal arts colleges, where faculty sizes are small, teaching is the primary mission and a single faculty member may be the only expert in a particular discipline. So if that faculty member becomes ineffectual, the entire discipline disappears from the institution,” Smith said.
“Yes, I’m a president of a regional campus within the University of Pittsburgh system, but I go back to the institutional identity of that campus, which is a liberal arts college.”
Nonetheless, Pitt-Greensburg holds certain values in common with all units of the University, she noted. “The mission of every part of the University is for the advancement of learning, and we seek to discover the knowledge and to transmit it to our students but at the same time to help them find careers.”
In addition, Pitt’s three-part charge to its faculty members — to demonstrate proficiency in teaching; research or scholarship, and service or citizenship — is enforced across the board at all campuses, she noted.
“As we have each said, the mission of the University is critical to begin to decide what is tenure and how we come about thinking how teaching fits into that,” Smith said.
“Tenure for the individual who is tenured clearly is an enormous value. It certainly accords to the members of the faculty who have demonstrated that high ability and achievement the recognition they have merited for their dedication to the growth of human knowledge,” she said.
“But the purpose of tenure transcends the individual value. It’s also intended to assure the University that there will be continuity in these experienced faculty, so that the function for which faculty are responsible will be carried forth. The University is recognizing the priceless worth for independence of mind and the freedom to inquire. That’s the real value for the individual of tenure, that they are free to investigate and express what they believe and what they discover.”
The process of awarding tenure, however, carries a kind of paradox: On the one hand, tenure is awarded based on evidence of an individual’s high ability and actual achievement, accomplishments that are reviewed retrospectively, Smith noted.
“But the decision itself is prospective. It is a decision that pertains to a future that may be longer than 40 years. We must therefore be clear in our minds when we assign such a future role to a faculty member that we think about what the University of Pittsburgh is to become, not simply what it is now. We who make the decision have to take into serious and equal or even greater consideration: How does this individual fit into what we want the University to become over what will be decades of connection?”
As a liberal arts college enrolling undergraduates exclusively, Pitt-Greensburg’s primary commitment is teaching effectiveness, Smith said.
“Nevertheless, we do not ignore those other criteria, because, as has been said, the relationship is complex. Actual accomplishment must be evident in each category. Extraordinary achievement in one category does not offset insufficiency in another,” she said.
She cited a quote from John Slaughter, president of Occidental College: “Research is to teaching as sin is to confession. If you don’t participate in the former, you have very little to say in the latter.”
Smith noted: “There is a special connection between teaching and advising, and research scholarship at both undergraduate and graduate levels. As it turns out, the nature of that research and the relationship and interaction with the student is going to vary substantially,” making teaching evaluation a particular challenge.
Educational data show that teaching graduate students typically advances the faculty member’s current research agenda, but teaching undergraduates makes faculty members ask fundamental new questions. “Undergraduates do not buy into assumptions. So it’s a different relationship, a different partnership,” between student and teacher, she pointed out.
Another study of graduate programs in the sciences at the top universities demonstrated that a graduate student enrolled in a program immediately after a shift in the discipline’s paradigm — for example, in the field of geology after plate theory emerged — holds a long-term career advantage over a grad student who completes the program prior to the paradigm shift.
“If faculty are actively engaged in research, then the paradigm shift is immediately reflected in their teaching. It’s critically important to the quality of what’s going on in the classroom,” Smith explained.
Teaching and service also are intertwined. “Teaching often inspires service on the campus and in the outside community,” Smith said.
She pointed to Pitt-Greensburg’s Academic Villages program, in which residence halls are organized as intellectual communities around a particular subject, such as sciences and technology, the arts, politics, history and foreign cultures.
“The faculty get involved in making presentations at the villages, in partnering with students in learning experiences. Faculty also may sponsor a student club that might be discipline-related or another where they share a common interest such as sustainability,” Smith said.
“We added some team-taught interdisciplinary academic village seminars that are just a real winner with the students and the faculty members. The faculty are finding they’re learning from one another new ways of teaching. Faculty would not have any opportunity to interact in this kind of shared learning experience on any other occasion,” she said.
So how does Pitt-Greensburg gauge teaching effectiveness?
A successful classroom performance as measured by student evaluations is an important start, Smith said.
“It is best if your system has every class the faculty member teaches evaluated and obviously the evaluations have to be done before grading. The cynic might say that’s just an evaluation of performance, rating whoever is the ‘best star,’” Smith said. “It isn’t. It turns out that when you look at student evaluations of teaching effectiveness versus peer reviews of the same teaching exercise, the student evaluations are very robust measures of teaching effectiveness, even at the undergraduate level where the student may never have been exposed to the discipline before. It is true that the comments that are added by the students, supplemented by serious peer reviews, are also part of that.”
Equally important is how faculty respond to the feedback of their reviews, she noted. “A group of students may say that that faculty member is disorganized. If the faculty member sees that as constructive feedback and acts on that, that’s a wonderful learning experience and a clear sign of a commitment to continuous improvement,” Smith maintained.
Promotion and tenure evaluators also must be aware of certain trends in student evaluations to avoid attaching undue weight to criticisms, she said.
“Data show the humanities are graded the most generously; the natural sciences the most harshly, and social sciences in the middle. Students grade required courses more harshly than elective courses,” Smith said.
“At Pitt-Greensburg, knowing these influences on how the classes are evaluated, we also look for the signs of a commitment to teaching and learning, that faculty members are actively engaged in developing new courses that enrich the curriculum and the majors and that demonstrate the faculty creativity and the connection with the emerging ideas.”
In the past four years, UPG faculty have developed new majors in chemistry, early childhood education, secondary education and Spanish, as well as a new track in the psychology major, she noted.
“We also look for our faculty to be engaged in assessment and revision. That demonstrates a commitment to continuous improvement to the individual courses and the overall program,” Smith said.
Part of effective teaching at UPG includes academic advising.
“We have professional academic advisers as students go through their freshman and sophomore year, but as they go deeper into their discipline they can find faculty to advise them,” she said.
“There’s also special advising for particular areas, whether it’s pre-med, pre-law, K-12 teaching or our new Ben Franklin Society to mentor our students to seek scholarships and fellowships.”
In addition, student engagement includes mentoring students in independent study and capstone projects, advising clubs and honorary societies, career counseling and community service.
“What’s the long-term impact of effective teaching? It’s well-taught, high-achieving students that are the outcome we seek now and into the future. Those students become successful alumni. The alumni owe their success to you, the faculty. They are the school’s true endowment and they are our gift to the state of Pennsylvania,” Smith concluded.
School of Medicine: The evolution of evaluating medical education
Panelist Thomas Smitherman, a professor in the School of Medicine and a long-time member of the school’s tenure and promotion committee, offered a short history lesson on the evolution of medical education as a way of comparing old and new philosophies of the role of teaching.
“Medical education in America was fairly limited until the turn of the 20th century. Two key events were vital to change that,” Smitherman said.
In 1889, Sir William Osler, a Canadian physician and scholar who is sometimes called the father of modern medicine, and his colleagues founded Johns Hopkins Hospital in Baltimore and four years later founded its School of Medicine.
“One of Osler’s key contributions was the importance of early exposure of medical students to hospital patients, and he later established medical residences, where medical school graduates spent several years living and sleeping in the hospital to increase their knowledge and skills. He reduced the time spent in didactic lectures and established teaching rounds on hospitalized patients,” Smitherman said.
The second pivotal event took place in 1904 when the American Medical Association created the Council on Medical Education to restructure medical education, he said.
“They developed the standards of pre-medical school training, and the structure of medical school education, including requiring two years of basic sciences and two years at a teaching hospital.”
At that time there were 155 U.S. medical schools. “Many were proprietary and for-profit with no affiliation with a university,” Smitherman said.
Then, in 1908 the Carnegie Foundation began a survey of medical education headed by a professional educator, Abraham Flexner.
Flexner visited all the medical schools in the United States and Canada and wrote an unflattering report in 1910, citing students being subjected to “interminable lectures and recitations … delivered in methodical fashion by part-time teachers. If fortunate enough to gain admission to a hospital, they observed more than participated.”
Pitt’s School of Medicine traces its roots to 1886. It started as a proprietary school founded by several local physicians and in 1892 it became loosely affiliated with the Western University of Pennsylvania (Pitt’s former name), and became fully integrated with the University in 1908.
“Flexner visited in 1910. He commended many aspects of the medical school and particularly cited it as an example of what could be accomplished with sound university management,” Smitherman said.
“Pitt med survived the Flexner report, but most schools did not. By 1935 only 66 of 155 medical schools survived and all but nine were fully integrated into a university. A new era had begun.”
He cited two other factors that have influenced the direction of medical education: “After World War II, large and complex medical centers as we know them evolved for medical schools and hospitals affiliated with them. The second factor, of course, was the burgeoning of biomedical research in America, driven largely by the development of the National Institutes of Health in 1937.”
In Osler’s era, physicians trained mostly in a single specialty after graduating from medical school. Now physicians often train in a second and even a third subspecialty, Smitherman noted.
“With the explosion of biomedical knowledge and biomedical research, we need to teach and train more graduate students and post-graduate trainees in the basic biomedical sciences,” he said.
“This explosion has also meant we have to keep educating ourselves. So we have a lot of people to teach. We teach medical students, graduate students, postdoctoral clinical and research trainees and we teach ourselves.”
In the current academic year, Pitt’s medical school faculty teach and train more than 3,000 individuals.
Large complex academic medical centers like Pitt’s require an ever-larger and more complex, varied and collaborative medical faculty, he said. “As of March, there were 2,115 faculty members, up 9 percent from academic year 2007. The size of the faculty has more than doubled in the last 25 years and it has increased 200-fold since [the school’s beginning] in 1886,” Smitherman said.
“Our teaching has become more complex, with fewer large lectures, more small-group sessions. Almost all of our teaching now is part of a team effort. Much of it is performed in small groups at the hospital bedside or in outpatient clinics, operating rooms, procedural laboratories or the bench of basic laboratory or clinical research, often closely allied to the faculty’s clinical service and research interests,” Smitherman said.
From the post-Flexner era into the 1950s, medical school faculties were much smaller relative to today.
“Most of the faculty members were clinician/investigators or basic scientists, principally working as investigator-educators. Together they sorted out much of the basic details of human biology, the basic mechanisms of many diseases and the basic treatment approaches to those diseases,” Smitherman said.
“They sat on a three-legged stool with nearly equal legs of service, teaching and research. Evaluations of those three activities for promotion and tenure decisions were reasonably straightforward in that era.”
But today’s medical faculty members have to learn to sit comfortably on a stool with three legs of unequal length, he maintained. “The older and simpler methods of evaluating faculty have become less effective.”
Some medical schools in the current era have limited conferral of tenure to those faculty who function primarily as investigator-educators. “That can obviously be detrimental to morale, recruitment and retention, and at Pitt med we set out on a different course. We recognize the need to be flexible and to identify and reward excellence of all sorts with promotion and tenure,” Smitherman explained.
To facilitate faculty career development and planning for reviews for promotion and referral of tenure, the medical school developed five somewhat nuanced pathways to divide its faculty.
The five pathways for medical faculty are: clinician, clinician-educator, clinician-investigator, investigator-educator and researcher.
• The clinician pathway, which is limited to the non-tenure-stream, is for faculty members who devote most of their time — up to 80 percent — to the direct delivery of health care services, but also are active teachers and may participate, usually as collaborators, in research activities. Their scholarship is measured principally by their scholarship in teaching.
• The clinician-educator pathway is for faculty members who are clinicians, but who also devote substantial time to educational efforts, often playing major roles in curriculum and program development and revision. Their scholarship is primarily measured by their scholarship in teaching, but that includes curriculum and program development.
“This pathway is largely limited to the non-tenure stream, but in a very innovative approach, each year a small number, usually one or two, based on exceptional teaching accomplishments, are placed in the tenure-stream,” Smitherman noted.
• “The clinician-investigator pathway is one that we have been innovating here by substantially resurrecting it. It is for busy clinicians who are also actively involved in research and teaching. The tripartite activities of these faculty members are often closely related. The large majority are in the non-tenure stream, but the busy clinician who is an active teacher and is considered to be an exceptional investigator in the top tier of clinician-investigators at Pitt med may be placed in the tenure stream and now represent about 20-30 percent of all of the candidates in the tenure-stream, which is rather innovative if you look across America,” Smitherman explained.
• The investigator-educator pathway is the traditional pathway for the researcher who also contributes substantially to service and teaching and whose teaching activities typically account for about 10 percent of their contributions. The majority of these faculty members are in the tenure-stream.
• The researcher pathway is rarely used and it’s restricted to the non-tenure-stream. It is strictly for investigators who choose to contribute to the academic mission primarily by collaboration, rather than as principal investigators.
“In all the pathways, we’ve tried to place our emphasis on assessing the candidate for promotion or the conferral of tenure on our sense of the candidate’s overall impact in teaching, service and scholarly activities. We believe we’re making some innovative and fruitful progress in the use of teaching excellence as a criterion for tenure and promotions,” he said.
“We think we’re pretty good at measuring the quantity of teaching for Pitt med faculty members, but how do we measure the quality?”
Evaluators look at two levels of quality: measuring the overall quality of the MD degree, as well as the quality of courses, programs, departments and divisions; and the quality of each faculty member.
“I polled a number of Pitt med faculty members who are heavily invested in teaching. Using a 5-point score of poor, fair, good, excellent or outstanding, there was remarkable consensus that we do an excellent job of evaluating the overall quality of our teaching, but at the individual faculty member level there also was remarkable consensus that we only do a fair-to-good job,” Smitherman said.
“What do we use to measure teaching quality overall for our students? First, we have immediate and extensive feedback from students on each course and further feedback at a later time. These are obviously helpful, but it’s noteworthy that the later feedback is almost always less positive than the immediate feedback, something we need to understand better,” he said.
In measuring overall quality, evaluators look at student grades on the three parts of the U.S. medical licensing exam; the quality rankings of the internships into which Pitt students are placed; the performance on standardized examinations and specialty board certification examinations of postdoctoral clinical training program residents and clinical fellows, and the positions into which nonclinical postdoctoral fellows are recruited upon completion of their training.
“Our students’ scores are high and correlate to some degree with their performance in medical school, but we recruit top students. That leaves open the question of who’s responsible: bright students or our teaching abilities?” Smitherman noted. “Students may choose an internship for reasons other than the program quality, which also confounds the data, and fellows may accept a job offer for reasons other than the perceived quality and prestige of the institution.”
In assessing the teaching quality of the individual faculty member, “we have feedback from the students and trainees soon after the teaching activity for the individual participants as well as for the overall course or rotation. This provides useful information, but the grades may be influenced by the popularity factor versus course content or course relevance,” Smitherman said.
Similar to other Pitt units, department chairs and division chiefs carry out annual performance appraisals for all activities, including teaching.
“But my poll provided consensus that the objective of those teaching appraisals could be improved and that such improvement would also lead to an improvement in conviction of our faculty that evaluation of teaching excellence is viewed to be very important,” he said.
“In my poll again there was remarkable consensus that an in-depth peer review of each course or program and each participant soon after the course or program was completed would be very valuable and more effective than anything in widespread use now. So why don’t we do it all the time? Such activities are time-consuming, they’re difficult to organize and they could be uncomfortable, but the enthusiasm for such activities from so many leads me to conclude that this idea deserves strong consideration for the future in evaluating teaching at Pitt med,” Smitherman said.
“In conclusion, I think we can say that recent and current experience show that teaching excellence plays an important role in promotion and tenure considerations in the School of Medicine. I think it’s very unfortunate that many of our faculty still may not fully recognize that,” he said.
“I further conclude that we’re doing an excellent job on evaluating the educational quality of our courses, programs and degrees and that the results of these annual evaluations confirm their excellence. Finally, however, I conclude that while we’re doing an adequate job of evaluating the quality of individual Pitt med faculty members, there appears to be an opportunity for improvement and maybe that is something that can be talked about further.”
Following the panel presentations and a brief question-and-answer session, event moderator Carey Balaban said, “I think we’ve all heard the issues very well. But the most astounding part of this to me, in this experiment bringing together representatives of what we think are three very diverse academic units of the University, is how we really did have a shared view of the role of teaching and being educators in our career as scholars.
“Teaching is an integral part of our mission at the University; it is a shared culture to have effective teaching across all of our units; we encompass many different audiences, from undergraduate education through continuing education through graduate education, and everything in between. We act as educators of the public. We present material and concepts for policy decisions in front of congressional committees, for example. And in fact we have a tremendous amount of serious thought and dedicated effort going into this, how we develop this and how we evaluate this to better develop it within our faculty.”
Balaban concluded with a quote attributed to Aristotle: “In general it is a sign of the human who knows and the human who does not know that the former can teach, and therefore we think art more truly knowledge than experience is, for the artist can teach and humans of mere experience cannot.”
Balaban said: “I think that captures very well in our intellectual history what we’re all about as a university. Teaching is a measure of your overall scholarship and grasp, what an integrated person you are. We’re individuals developing knowledge, but it’s not our knowledge, it’s the collective knowledge of a community of scholars within our departments, within our schools and beyond that project out into society both in the United States and the world.”
Chancellor Mark Nordenberg, Provost Patricia Beeson and University Senate President Michael Pinsky also offered brief remarks. The full plenary session is available for viewing online at http://mediasite.cidde.pitt.edu/mediasite/Viewer/?peid=667cd9b0327c41c2a7f4e0e106730d41.