Criteria for assessing research quality in the humanities: a Delphi study among scholars of English literature, German literature and art history

Abstract

It is a complex issue how to evaluate the quality of humanities research. The fact that
humanities researchers continue to strongly oppose attempts to quantify and evaluate
research quality suggests that existing evaluation schemes and methods are not well suited to
the humanities fields, even though several projects have attempted this in recent years. Using
a multi-round Delphi survey, we gathered quality criteria from researchers in the fields of art
history, German literary studies, and English literature studies from researchers in Switzerland
and the League of European Research Universities (LERU). Using 70 different criteria, the first
Delphi round produced a thorough set of 19 standards for high-quality research. While ten of
these criteria are widely utilised in different types of evaluations, nine of them are either not
used at all or are used very seldom. Consensual criteria and elements were discovered in
each field in the second Delphi round. These were things that were obviously endorsed by the
majority and condemned by very few experts. Eleven criteria were agreed upon by all three
disciplines (shared criteria), six by one or two disciplines (discipline-specific criteria), and two
by none ('production' and'relation to and influence on society'). Let me break it down for you.
This study's findings support earlier ones showing that researchers have not implemented
evaluation criteria linked to Mode 2. We are now discussing the implications for evaluating
research. In particular, we will be looking at the problem of evaluators' and academics' criteria
not matching up and how to fix it.