For professors, it’s up or out when it comes to tenure and promotion: After five years on the job as assistant professors, junior professors’ promise is evaluated by their more senior colleagues.
These senior colleagues judge whether a junior colleague has developed a reputation for their research, and contributed to teaching, service, and sometimes to other priorities such as equity or community engagement. Colleagues within the same university are not usually experts on a junior researcher’s work, so universities have, for over 80 years, solicited external review letters from experts within the junior colleague’s field to inform tenure review panels’ decision-making.
These external review letters are often cited as the most important factor determining whether a junior professor can stay at their university, gain access to security of employment and continue their research. Often, when external review letters are not favorable, they lead to the faculty member having to pack up their research, lives and families and start over elsewhere or switch careers entirely.
A new paper in press at Research Policy calls into question whether the reliance on external review letters is justified. A team led by UC Merced psychology Professor Christiane Spitzmueller and Juan Madera, professor of hospitality management at the University of Houston, examined 950 external review letters for almost 200 professors who were seeking promotion. The findings are likely to disrupt how universities think of external review letters.
If these letters were as good and objective indicators of a scholar's promise as they are deemed to be almost universally, the letters’ favorability should be closely linked to the quality of the faculty members’ research, including how widely cited their works are and whether they are published at a solid rate in good journals, the researchers said.
“Our analyses of the external review letters and their links with promotion outcomes for faculty shows that unfortunately, there is a stronger link with who the external review letter writers are than there is a link with the junior professor’s research quality” said Madera, the paper’s first author. For example, factors such as the institutional prestige of the letter writers’ institution, and the letter writer’s gender are more impactful in determining letter favorability than a scholar’s quality of research, the study shows.
In practice, this might play out in a way where Julia, a highly productive junior faculty member, seeks promotion. Her department chair, in consultation with the promotion and tenure committee, determines who is invited to serve as external review letter writers. The chair and committee select a list of writers who are nationally recognized experts in the promotion candidates’ area of work, but all the letter writers work at liberal arts colleges that are R2 institutions rather than at prestigious and highly ranked R1 universities. When the committee convenes to review the portfolio and read the letters, its members interpret the lack of prestige for the letter writers’ institutions as evidence of Julia’s limited luster and reputation at top institutions, not knowing that she had no input into who was asked to provide external review letters. Julia receives unfavorable votes and is denied tenure.
The research shows similar scenarios are likely to play out frequently, impacting careers in inequitable and unpredictable ways.
In most cases, junior professors have little to no control over who the external review letter writers are, creating a situation where much is left to chance.
“Ultimately, that means the current system and its reliance on external review letters does not always lead to the promotion of the best and brightest with the highest likelihood to advance science. In fact, the current system needs to be revised substantially,” said Spitzmueller, the paper’s senior author.
“Altogether, it’s about time for a very close look at validity and fairness in decision-making processes that make or break careers and lives of faculty members,” she added.
That might, for some universities, entail eliminating external review letters entirely or include adding structure to external review letter solicitations. Adding structure has, in research on employment interviews, been shown to lead to closer relationships between interview outcomes and candidates’ subsequent job performance.
And asking specific questions about a candidate’s performance dimensions and standardizing the structure of the letter responses may improve validity and limit the impact of extraneous factors such as letter writer gender or institutional prestige.
For now, the research team recommends institutions examine in their own data whether letter writer characteristics are linked more strongly to promotion outcomes than measures of candidate productivity. If that’s the case, it’s time to change how academics are promoted.
The current research was supported with funding from the National Science Foundation and the Alfred P. Sloan Foundation.
