OPUS Workshop in Brussels: How do we transform open science principles into practical researcher assessment frameworks that institutions can actually implement? 

• February 8, 2023

On 6-7 February 2023, OPUS project consortium members gathered in Brussels for an intensive two-day workshop centred on one fundamental question: how can open science principles be transformed into practical researcher assessment frameworks that institutions can genuinely implement?

The workshop brought together pilot organisations from across Europe, including research performing organisations (RPOs) and research funding organisations (RFOs), to move beyond theoretical discussions and tackle the real-world challenges of integrating open science into researcher evaluation processes.

Setting the Stage to Establish a Researcher Assessment Framework

The OPUS Brussels workshop wasn’t just another project meeting. It represented a critical juncture where high-level policy ambitions meet institutional reality. As coordinator’s opened the session, three key objectives were laid out:

1. Assess the current status of open science practices across pilot partners through comprehensive questionnaire results
2. Develop practical frameworks for interventions and indicators that institutions could realistically implement
3. Create actionable templates for Action Plans that would guide pilot organisations through their open science transformation

The ultimate goal was ambitious yet practical: to establish a researcher assessment framework that could bridge the gap between open science policy and everyday academic practice.

The Questionnaire Revelations

The morning session revealed fascinating insights from the pilot organisations, each representing different national contexts and institutional approaches to open science.

Universidade Nova de Lisboa (UNL) showcased Portugal’s emerging approach: while research data management remained largely at the recommendation stage, there was clear intention to formalise policies through the OPUS project. The institution had promising citizen science initiatives but lacked comprehensive policy frameworks. Perhaps most intriguingly, they were planning internal monetary awards for open science practices and had established a social science faculty group to drive institution-wide engagement.

University of Cyprus (UCY) presented a more developed landscape with budget provisions for open access and data curation already in place. Their commitment to implementing CoARA principles (Coalition for Advancing Research Assessment) demonstrated serious institutional buy-in, though they acknowledged resistance from senior researchers and the need for carefully targeted communication strategies.

University of Rijeka (UNIRI) brought the Croatian perspective, benefiting from strong national open science infrastructure. As early signatories of both DORA and CoARA, they had already established an open science policy adopted by the university senate. Their diamond access model for nationally funded journals and dedicated Centre for Open Science within the library showed institutional commitment in action.

Research Council of Lithuania (RCL) highlighted the challenges facing research funders, particularly around predatory journal identification and the practical difficulties of implementing guidelines. Their involvement in multiple EU projects (RESTORE, FOSTER, OpenAIRE) demonstrated extensive experience, though resource constraints remained a significant barrier.

Romanian Executive Agency for Higher Education, Research, Development and Innovation Funding (UEFISCDI) represented national-level coordination, with a comprehensive R&I strategy mandating open access for publicly funded publications and FAIR data principles. Their involvement in Coalition S and multiple international initiatives showed commitment, though implementation challenges persisted.

Framework Development: From Theory to Practice

The afternoon session tackled the complex task of developing practical frameworks. The consortium agreed on a three-step approach:

1. Create a researcher assessment framework with concrete indicators
2. Develop targeted interventions based on these indicators
3. Establish metrics to measure intervention effectiveness

A key insight emerged: rather than reinventing the wheel, OPUS would build upon existing frameworks like OS-CAM (Open Science Career Assessment Matrix), incorporating principles from DORA, the Leiden Manifesto, and CoARA to create something both comprehensive and implementable.

Connecting Concept to Reality

The second day focused on the crucial challenge of connecting framework concepts to institutional reality. A pragmatic approach emerged: start with existing activities within institutions, monitor what’s already happening, then strategically add new elements.

Gordon’s insight from the GRRIP project (Gordon Dalton, GRRIP Project Coordinator) proved invaluable: institutions should focus on “low-hanging fruit” rather than attempting overly ambitious transformations. The most successful institutional changes came from selecting achievable interventions and building momentum through early wins.

Cultural Change as the Foundation

A critical discussion emerged around the role of cultural change in open science adoption. John Crowley’s contribution highlighted the theory of change: awareness → engagement → institutional and individual transformation. This insight shaped OPUS’s focus on the first two stages, recognising that lasting change requires more than policy mandates.

The workshop identified trust-building as essential, recommending dedicated working groups involving representatives across seniority levels and departments, researchers, administrators, librarians, and management.

Action Plan Development

The final session tackled the practical matter of the OPUS Action Plan templates. Key elements emerged:

• Institutional context descriptions, including roles in researcher assessment
• Logic explanations for intervention choices
• Resource allocation (often limited to staff time rather than additional budget)
• Risk assessment and mitigation strategies
• Industry engagement components for sustainability

A crucial decision was made to keep Action Plans concise and focused, building on existing institutional structures rather than requiring wholesale system overhauls.

Key Insights and Takeaways

The Implementation Reality Check

Perhaps the workshop’s most valuable contribution was its honest assessment of implementation challenges. Participants acknowledged that:

• Many institutions already have open science interventions but haven’t integrated them into researcher assessment
• Senior researcher resistance remains a significant barrier requiring strategic communication
• Resource constraints limit what pilot organisations can realistically achieve
• Cultural change takes time and requires carefully planned, trust-building approaches

Building on Existing Foundations

Rather than starting from scratch, the workshop emphasised building on existing initiatives. This pragmatic approach recognised that sustainable change comes from enhancing current practices rather than replacing entire systems.

The Power of Mutual Learning

The mutual learning approach emerged as a key success factor. Institutions facing similar challenges could learn from each other’s experiences, share practical solutions, and build collective momentum for change.

From Brussels to Implementation

The Brussels workshop established the foundation for OPUS’s practical phase. The frameworks developed would undergo monthly iterations with pilot feedback, ensuring they remained grounded in institutional reality while advancing open science goals.

Most importantly, the workshop demonstrated that meaningful change is possible when policy ambitions are matched with practical wisdom, institutional commitment, and realistic timelines. The pilot organisations left Brussels not just with frameworks and templates, but with a shared understanding of how to make open science researcher assessment a reality.

The journey from policy to practice had begun in earnest, with Brussels serving as the crucial waypoint where ambitious visions met implementable realities.

---

The OPUS Brussels workshop exemplified collaborative policy development at its best, bringing together diverse stakeholders, acknowledging real constraints, and focusing on achievable solutions that could create lasting change in how we assess and reward researchers in the open science era.