Open Science Monitoring Initiative

• May 23, 2024

Earlier this month, the Open Science Monitoring Initiative shared a draft of Open Science monitoring principles, launching a worldwide consultation.

We at OPUS are thrilled to see this initiative take shape and fully support the efforts of PLOS and others involved, even though we are not officially participating. Drawing from our experience developing Open Science Indicators, Pilots and Incentives, we understand the importance of building upon collective knowledge and efforts. We are encouraged by the broad engagement of stakeholders from the scholarly community in this important endeavor.

Organizations that promote Open Science, like PLOS and OPUS, need effective mechanisms to monitor the adoption of Open Science practices. Context is crucial. Research is a global enterprise supported by a vast network of academic institutions, service and infrastructure providers, funders, and policy-making groups. The solutions we develop must address the priorities and answer the questions of each of these groups. So far, this has led to various monitoring solutions that are not comparable, limiting the utility of collected data and risking misalignment.

In our efforts to advance Open Science, OPUS has encountered first-hand the barriers researchers face when governing bodies lack alignment. This is why the work of the Open Science Monitoring Initiative is so vital: it will take a multi-stakeholder collaboration to create context-specific but comparable monitoring solutions that support pathways to Open Science adoption for diverse communities.

A shared foundation accelerates progress for all

UNESCO’s comprehensive recommendation on Open Science is a significant milestone towards a shared understanding and vision for Open Science. Implementing this recommendation will require the ability to monitor its adoption in ways that honor its principles.

The Open Science monitoring initiative focuses on collaboration and consultation with a wide range of stakeholders. Despite the disparate needs or motivations an institution in Kenya, a funder in the UK, and a policy-making body in France may have for monitoring Open Science, adopting common principles will help provide a global understanding of progress.

The more we share and learn from each other and study new aspects of the prevalence and effects of Open Science, the better equipped we will be to identify significant barriers for researchers, systemic challenges, structural inequities, and potential biases. With a common set of principles for monitoring, we are one step closer to establishing equitable paths to Open Science.

Open Science monitoring for better Open Science practice

When PLOS envisioned Open Science Indicators in 2022, it was with the recognition that advancing Open Science adoption requires understanding our starting point and evaluating the effectiveness of our solutions.

It was important to establish underlying principles to guide future development, communicate our goals transparently, and help others understand and use our tools and data responsibly. This same reasoning informed our support for the Open Science Monitoring initiative.

The data gathered through OSIs provides a better understanding of the current landscape of Open Science practices, helping PLOS see where interventions and solutions could be effective—or not. Since implementing OSIs, they’ve gained insights into regional differences in Open Science behaviors, differences by discipline, and the impact of policies and solutions on these behaviors.

For example, at PLOS Computational Biology, code-sharing rates rose from 53% to 87% in the first year after implementing a mandatory code-sharing policy, indicating a significant move towards this Open Science practice. We are also tracking the impact of facilitating preprint sharing and incentives that could promote better data-sharing practices.

However, journal policy is just one mechanism for changing research-sharing norms. By the time researchers are ready to share their work, many choices have already been influenced by their context and circumstances: Does their national research body support sharing more components of research? Will their institution recognize efforts towards transparency? Is there funding and infrastructure to make it easy? Do the solutions available fit the research aims?

PLOS shares their dataset and findings publicly, hoping they may benefit others, and they are always open to feedback and additional context-specific interpretations.

Working together to build a path forward

There are numerous routes researchers can take to make their work more open. At OPUS, part of our mission is to support multiple pathways to Open Science by understanding the academic landscape and researchers’ motivations.

We know from our research that Open Science monitoring is a shared need among many organizations. We have been collaborating to understand how monitoring needs align and differ between funders, institutions, and publishers.

PLOS’ collaborative work with funders and institutions, particularly with the UK Reproducibility Network (UKRN) to develop open research indicator pilots for institutions, illustrates that measuring the prevalence of Open Science practices is not enough. PLOS also needs to measure the effects or qualities of those practices to achieve the aims of greater transparency, integrity, and inclusion that are at the core of Open Science. We believe that Open Science is better science, and effective monitoring—and rigorous meta-research—can provide more evidence for this.

A systemic challenge requires a systemic solution. This must be a multistakeholder endeavor, creating a feedback loop between funders, policy-makers, institutions, infrastructure providers, researchers, and publishers. Importantly, the diversity of perspectives is critical, not only across sectors but also across disciplines, regional, and economic contexts.

At OPUS, we are committed to supporting these efforts and working collaboratively to build a path forward for Open Science. Together, we can create a more transparent, inclusive, and effective research ecosystem.

Original article

Photo via Middleware.io