Reforming Research Assessment: How to implement responsible procedures and move beyond impact factors and h-index?

• March 20, 2023

LMU Open Science Center held a symposium titled “Reforming Research Assessment: how to implement responsible procedures and move beyond impact factors and h-index?” on March 13, 2023, which aimed to address the concerns regarding the existing metrics that are used to assess research productivity. The use of journal impact factors and h-index to assess individual researchers has been heavily criticized for promoting a “publish or perish” culture and lacking validity.

The symposium discussed multiple initiatives that called for alternatives to the existing metrics that better reflected the quality of research. The themes of the recommendations of initiatives such as The Declaration on Research Assessment (DORA) and the Coalition for Advancing Research Assessment (CoARA) included the elimination of journal-based metrics in funding, appointment, and promotion considerations, the assessment of research on its own merits, and the recognition of diverse outputs, practices, and activities that maximize the quality and impact of research.

As of January 12, 2023, 441 organizations, including the European Commission, the League of European Research Universities, and the European University Association, had signed the Agreement on Reforming Research Assessment, agreeing to define an action plan within the year.

The symposium aimed to address how to practically implement and evaluate a reformed research assessment, what the consequences would be for researchers from signing institutions or applying for signatory funding agencies, how to ensure a smooth transition for early-career researchers, and how to reconcile diverse needs across disciplines. The event included presentations from CoARA Steering Board Member Prof Dr Toma Susi, Vice President Research & Development at Ernst-Abbe-Hochschule Jena Prof Dr Kristin Mitte, Head of Division Life Sciences II at the German Research Foundation (DFG) head office Dr Tobias Grimm, Scientific Strategic Advisor in the Scientific Director’s Office at the Max Delbrück Center in Berlin Dr Jess Rohmann, and Managing Director of the LMU Open Science Center Prof Dr Felix Schönbrodt. The event was chaired by Open and Reproducible Data Science Consultant Dr Heidi Seibold, an Associate Member of the LMU Open Science Center.

The symposium shed light on the need for profound changes in how academic achievements are evaluated in both hiring and funding decisions. The practical implementation of the reformed research assessment and the transition for early-career researchers are essential aspects that need to be considered while moving beyond the existing impact factors and h-index.

You can find the slides and the recording of the presentations on the LMU Open Science Center OSF repository: https://doi.org/10.17605/OSF.IO/EQKJZ

To join the LMU Open Science Center community and be notified of our workshops and meetups (see event list here: https://www.osc.uni-muenchen.de/events/index.html), you can subscribe to their mailing list: https://lists.lrz.de/mailman/listinfo/lmu-osc. 

Source: https://www.osc.uni-muenchen.de/members/join/index.html.