A New Era in Research Assessment

• December 3, 2024

The research assessment landscape is shifting to prioritise open scholarship and preprints, driven by funding bodies and academic institutions aiming to enhance transparency, accessibility, and equity. Zen Faulkes, Program Director, The Declaration on Research Assessment (DORA), and Haley Hazlett, Program Manager, DORA, here explore how emerging policies are transforming evaluation, the challenges of moving beyond traditional metrics, and the importance of building trust in open scholarship.

Embracing Open Scholarship: A New Era in Research Assessment

In recent years, the landscape of research assessment has undergone a significant transformation, driven by a growing recognition of open scholarship and the use of preprints. This shift, championed by funding organisations and academic institutions, aims to enhance transparency, accessibility, and equity in research.

Emerging Policies and the Role of Preprints
Research funders and universities play a pivotal role in shaping research culture by defining what is valued in assessments. Increasingly, organisations such as Wellcome, the Chan Zuckerberg Initiative, and EMBO are incorporating open scholarship into their policies. Preprints, which allow researchers to share their work openly before journal publication, are now recognised as valuable research outputs by funders like NIH and UKRI Councils. This approach reduces reliance on traditional journal prestige and supports the dissemination of diverse research outcomes, including null results and studies with limited scope.

A landmark policy shift came from the Bill and Melinda Gates Foundation, which will mandate preprint sharing and cease paying article processing charges (APCs) from 2025. By prioritising accessibility and cost reduction, this policy exemplifies the potential of open scholarship to democratise research dissemination.

Addressing Challenges and Building Trust
Transitioning away from entrenched systems, such as reliance on journal impact factors, poses challenges. Concerns include how preprints will influence research culture and peer review processes. Preprints are lightly screened before publication, offering a platform for community feedback, which can enhance transparency and rigor. Initiatives like VeriXiv and Review Commons are pioneering structured peer review for preprints, fostering trust in these outputs.

Building trust also requires a shared understanding of preprint reviews, which should evaluate rigor, declare competing interests, and verify reviewer identities. Addressing concerns, especially those of early-career researchers about potential retaliation, is essential for fostering an inclusive and supportive research environment.

Intersections of Open Scholarship and Equity
Open scholarship, responsible research assessment, and equity are deeply interconnected. By decoupling research evaluation from journal prestige, institutions can mitigate biases and broaden the recognition of diverse scholarly contributions, such as datasets and software. This approach aligns with the goals of transparency and inclusivity, fostering trust among academics and the public alike.

A Call to Action for Sustainable Change
Reforming research assessment demands policies that reward diverse outputs, emphasise transparency and rigor, and reduce the barriers posed by APCs. Efforts to index preprints and link reviews are key steps toward fostering trust. Open dialogue and iterative policy refinements will ensure these changes are effective and sustainable.

As institutions and funders continue to innovate, the opportunity to reshape research culture for greater equity and accessibility is within reach. Through collaboration and a commitment to responsible research assessment, the academic ecosystem can move closer to a future that values openness and inclusivity.

Source: https://www.templetonworldcharity.org/blog/encouraging-innovation-in-open-scholarship-while-fostering-trust