Research

In the quest to foster innovation and excellence in research, the evaluation process for funding proposals plays a pivotal role. Recognizing the need for a more comprehensive approach, DORA (Declaration on Research Assessment) has introduced “Balanced, Broad, Responsible: A Practical Guide for Research Evaluators.” This initiative comprises a short, informative video and a one-page brief designed to serve as invaluable resources for both public and private research funders.

The Need for a New Approach

The guide underscores the importance of research assessment in shaping the research culture. Traditionally, there has been an overreliance on quantitative metrics, which can limit evaluators’ ability to fully appreciate the breadth and impact of an applicant’s work. Just as research itself is dynamic and evolving, the methods used to assess research should also be subject to continuous improvement and innovation. This shift is essential to ensure that evaluations are holistic and reflective of the diverse contributions researchers make to their fields.

Practical Suggestions for Holistic Evaluation

To aid funders in adopting more responsible assessment practices, the guide provides a practical checklist with six key suggestions:

1. Align Decision-Making: Ensure that decisions align with the strategic objectives and specific criteria of the funding institution or instrument.
2. Balance Metrics: Clearly communicate the context and limitations of any quantitative metrics used and balance them with qualitative assessments of the funding proposal.
3. Broaden the Scope: Look beyond traditional metrics such as publications and grants to capture the full range of a researcher’s contributions, including activities like mentoring, community engagement, and collaborative projects.
4. Address Biases: Be mindful of biases that can arise from scientific and cultural stereotypes, striving to create a fair and inclusive evaluation process.
5. Promote Accountability: Encourage personal and group accountability for responsible research assessment throughout the evaluation process.
6. Manage Conflicts of Interest: If there is any uncertainty about potential conflicts of interest, seek guidance from the funding institution to ensure transparency and impartiality.

Implementing the Guide

The one-page brief offers practical suggestions for various contexts, including sharing the video and brief with grant evaluators, institutions, and researchers. By disseminating these resources widely, funders can promote a more balanced and responsible approach to research evaluation.

Collaborative Development

These resources are the result of a partnership between DORA and the Luxembourg National Research Fund (FNR). The development process included iterative feedback from the DORA Steering Committee, the DORA Funder Discussion Group, active researchers, and other funding agencies. This collaborative effort has ensured that the checklist is both practical and grounded in the real-world experiences of those involved in research funding and assessment.

To sum up, “Balanced, Broad, Responsible: A Practical Guide for Research Evaluators” represents a significant step towards improving how research proposals are evaluated. By embracing a more holistic approach, funders can better recognize and support the full spectrum of contributions that researchers make to their fields and society at large.

More at DORA

LIBER’s Open Access Working Group invites you to a pivotal webinar aimed at exploring the financial dimensions of transitioning to open access. This session will delve into findings from two recent studies and engage in a roundtable discussion on the implications for libraries’ open access strategies moving forward. The webinar is scheduled for 03 September 2024, at 14:00 CEST, and will be conducted online.

Webinar Facilitator

Dr. Birgit Schmidt, the Head of Knowledge Commons at the Göttingen State and University Library, will facilitate this insightful webinar. Dr. Schmidt brings extensive expertise in open access and knowledge management, promising an engaging and informative session.

Registration

To participate in this webinar, please register here. Your insights and perspectives will be valuable additions to the discussions.

Background

Research libraries worldwide are reevaluating their open access strategies, including moving away from traditional big deals with publishers. Empirical data on current investments in open access and its impact on the publishing landscape is crucial for libraries as they negotiate with publishers and support alternative models like diamond open access.

The studies to be discussed are based on open data, showcasing the potential for transparent analysis in scholarly publishing. These analyses highlight gaps in monitoring the open access transition, offering valuable information for library decision-making.

Speakers and Topics

Bianca Kramer, Independent Research Analyst at Sesame Open Science

Bianca Kramer specializes in open science, open metadata, and open infrastructure. She will present findings from her report on:

Scientific Publishing in Europe: Development, Diversity, and Transparency of Costs

Commissioned by the European Commission, this report investigates the financial aspects of open access, focusing on the transparency of transformative agreements with publishers. It includes an analysis of publication patterns in 27 EU countries (plus the UK, Norway, and Switzerland), with particular attention to diamond open access and journals not included in the Directory of Open Access Journals (DOAJ). The report offers targeted recommendations for member states, institutions, and libraries to enhance transparency and encourage publishers to do the same.

Najko Jahn, Data Analyst at the State and University Library Göttingen, University of Göttingen

Najko Jahn’s work centers on scholarly communication analytics. He will discuss his preprint on:

How Open Are Hybrid Journals Included in Transformative Agreements?

This preprint analyzes the development and limitations of open access in hybrid journals. Based on data from transformative agreements covering 12,000 hybrid journals, it reveals the varying uptake of open access across different publishers and countries. While some European countries have significantly increased their open access share through these agreements, the overall implementation levels do not yet indicate a large-scale transition from subscription-based to full open access publishing.

Final remarks

This webinar presents an invaluable opportunity for librarians, researchers, and open access advocates to gain insights into the financial intricacies of open access and engage in meaningful discussions on shaping future strategies. Join us to learn from the experts and contribute to the dialogue on advancing open access in scholarly publishing.

Facilitator Contact:
Dr. Birgit Schmidt
Head of Knowledge Commons, Göttingen State and University Library

Don’t miss this chance to be part of a significant conversation on the future of open access. Register now and bring your perspectives to the table.

European repositories play an essential role in the open science ecosystem by acquiring, preserving, and providing open access to vast amounts of valuable research outputs. In January 2023, four leading organizations—OpenAIRE, LIBER, SPARC Europe, and COAR—launched a collaborative strategy to bolster the European repository network. This initiative seeks to highlight the importance of repositories, disseminate best practices, and support the creation of robust national networks.

Survey of European Repositories

The first phase of this strategy involved a comprehensive survey conducted from February to March 2023, aimed at mapping the current European repository landscape. The findings underscored the crucial role repositories play as non-profit infrastructure in the open science framework, managing tens or even hundreds of millions of research outputs. However, the survey also highlighted three significant challenges that repositories face:

1. Software Platform Maintenance: Repositories need to maintain up-to-date, highly functional software platforms to ensure seamless operations.
2. Best Practices Implementation: There is a need for consistent and comprehensive best practices in metadata, preservation, and usage statistics across repositories.
3. Visibility in the Scholarly Ecosystem: Repositories must achieve better visibility and integration within the broader scholarly community.

Developing a Vision for the Future

In response to these challenges, OpenAIRE, LIBER, SPARC Europe, and COAR have decided to draft a position paper outlining a collective vision for the future of European repositories. To ensure this vision is well-informed and comprehensive, the organizations are seeking input from a wide range of stakeholders through an open consultation process.

Participate in the Consultation

The consultation aims to gather valuable insights and feedback from various communities, which will inform the initial draft of the position paper. This draft will subsequently be refined through focus groups. The consultation is open until the end of August 2024, and your participation is crucial.

We invite all stakeholders to share their opinions and contribute to shaping the future of European repositories. Your input will play a vital role in enhancing the repository infrastructure, ultimately supporting the advancement of open science.

Contribute with your opinion.

Thank you for your time and valuable contributions.

Jul 23, 2024, 14:00 CEST: Bibliographic citation, the practice of referring from a citing entity to the cited one, is a foundational activity for authors in creating any scholarly work. This acknowledgment of sources is essential to the scholarly enterprise, forming the backbone of academic integrity and intellectual development. By weaving together a network of citations from academic articles, books, conference proceedings, and more, researchers can navigate the vast landscape of knowledge and build upon existing work.

The Role of Citations in Research
For various scholars, citations serve as a critical tool:

• PhD students rely on citations to survey the literature for their theses, finding relevant articles to support their research.
• Senior researchers use citations to continually discover new material and deepen their understanding of specific topics.
• Reviewers examine citations to ensure that the cited works are current and well-integrated with other research.
• Professors crafting project proposals utilize citations to identify recent advancements and establish valuable connections.

These examples illustrate the diverse ways citations support the academic community, enhancing the credibility and depth of scholarly work.

Understanding the Reasons Behind Citations
The motives for citing previous works are varied. Authors typically cite sources to acknowledge assistance received in the form of background information, ideas, methodologies, or data. However, citations can also serve other purposes, such as reviewing, critiquing, or refuting earlier works. Understanding these citation intents is crucial for appreciating the full context of scholarly communication.

Advancing Citation Analysis with GraspOS
In an upcoming seminar, existing data models for classifying citation intents, or functions, will be introduced. These models are being utilized as a foundation within the GraspOS project. The seminar will also showcase a tool under development designed to extract citation semantics from scholarly articles in PDF format. This tool aims to enhance our ability to analyze and understand the intricate web of academic citations, thereby supporting more nuanced and informed research practices.

By delving into the reasons behind citations and leveraging advanced tools to analyze them, the GraspOS project aspires to enrich the scholarly community’s understanding of citation dynamics and improve the overall quality of academic research.

Register here: Zoom Meeting

By Heidi Seibold

Overview

Reproducible research is crucial for advancing science, allowing others to verify results and build upon previous work. This guide outlines six impactful steps to make your research reproducible and open. It is structured to help you implement these steps gradually, depending on your current practices and goals.

1. Get Your Files and Folders in Order

The Importance of Organization

A well-organized file and folder structure is fundamental for managing complex research projects. It helps you and your collaborators find files quickly, reduces redundancy, and prevents errors.

Suggested Folder Structure

Here is a basic template for organizing your research project:

.
└── project-name
    ├── analysis       # Data analysis files
    ├── src            # Source files and functions
    ├── comm           # Communication documents
    │   ├── internal-comm
    │   └── journal-comm
    ├── data
    │   ├── data_clean
    │   └── data_raw   # Untouched raw data
    ├── dissemination  # Manuscripts, posters, presentations
    ├── documentation  # Documentation files
    └── misc           # Miscellaneous files

This structure can be tailored to fit the specific needs of your project.

Your Tasks

• Review and reorganize your current projects using a similar structure.
• Discuss with collaborators and adjust the structure based on feedback.

2. Use Good Names

The Power of Naming

Good naming conventions for files, folders, and functions can significantly improve the clarity and usability of your project.

Examples of Bad and Good Names

Bad:

• Myabstract.docx
• figure 1.png

Good:

• 2024-07-08_abstract-for-conference.docx
• Fig01_scatterplot-talk-length-vs-interest.png

Naming Guidelines

• Machine readable: Avoid spaces and special characters.
• Human readable: Use descriptive names.
• Consistent: Stick to a naming convention throughout your project.
• Default ordering: Use dates in YYYY-MM-DD format.

Your Tasks

• Audit and improve naming conventions in your current project.
• Establish and adhere to a naming convention.

3. Document with Care

Importance of Documentation

Proper documentation ensures that others (and your future self) can understand and reproduce your work.

Key Documentation Practices

• README: Include a README file in each project outlining its purpose, involved parties, and key information.
• Code Documentation: Use literate programming or clear comments to explain your code.
• Metadata: Document information about your data, including its origin, content, and licensing.

Your Tasks

• Create or update the README file in your current project.
• Start or improve code commenting practices.
• Ensure metadata is documented.

4. Version Control

Why Version Control?

Version control is essential for tracking changes, collaborating with others, and maintaining a history of your work.

Using Git

Git is the most popular version control system, allowing you to track changes, collaborate, and revert to previous versions easily.

Steps to Get Started

1. Install Git.
2. Create an account on GitHub or GitLab.
3. Initialize a repository and start committing your changes.

Your Tasks

• Set up Git for your current project.
• Learn Git through tutorials or workshops.

5. Stabilize Your Computing Environment and Software

Avoiding Environment Issues

Stabilizing your computing environment ensures that your code runs consistently over time, regardless of updates to software or operating systems.

Strategies

1. Record your computing environment: Document software versions using tools like sessionInfo() in R.
2. Virtual Machines: Use virtual machines to encapsulate your environment.
3. Containers: Utilize Docker or Apptainer for portable and shareable environments.

Your Tasks

• Choose a method for stabilizing your environment.
• Implement this method in your current project.

6. Publish Your Research Outputs

Sharing Your Work

Publishing your research outputs ensures that others can access and build on your work.

Options for Publishing

• General purpose repositories: Zenodo, Open Science Framework.
• Institutional repositories: University-specific services.
• Field-specific repositories: Discipline-focused platforms.

If Data Cannot Be Shared Openly

Consider publishing metadata, synthetic data, or sharing data with specific researchers if your data is sensitive.

Your Tasks

• Identify suitable repositories for your field.
• Start publishing your data and code in these repositories.

Conclusion

Reproducible research enhances the credibility and impact of your work. By implementing these six steps, you can make your research more transparent and accessible. Remember, improving reproducibility is a gradual process—take it one step at a time and continuously seek to enhance your practices.

Seibold, H. (2024). 6 Steps Towards Reproducible Research. Zenodo. https://zenodo.org/records/12744715

Implementing for Impact: Measuring Open Science for the SDGs

Register Now

In the quest to achieve the Sustainable Development Goals (SDGs), open science and open scholarship emerge as crucial catalysts. These frameworks foster a collaborative and transparent environment, essential for driving progress across all SDGs. Yet, the path to embracing and assessing open science is marred by diverse methodologies and a global disparity in research and development. UNESCO’s Open Science Outlook 1 cautions that the “cultural shift to open science will only be possible with adequate monitoring of its impacts, including its possible unintended consequences for science and/or society.”

Against this backdrop, the United Nations’ Dag Hammarskjöld Library is set to host a pivotal side event to the High-Level Political Forum. The biennial UN Open Science Conference, organized by the Dag Hammarskjöld Library in collaboration with its partners, frequently underscores the need for academic institutions to transcend conventional metrics. Instead, these institutions are encouraged to reclaim their transformative role in society, aligning research agendas with global significance rather than mere journal visibility. Key questions arise: How do global policies aimed at promoting open science and the SDGs influence local research evaluation frameworks? What is their impact on individual researchers and their contributions? Do these policies facilitate or obstruct the realization of the SDGs?

To explore these critical issues, a distinguished panel of speakers has been assembled:

• Dr. Suchiradipta Bhattacharjee, International Water Management Institute
• Dr. Yensi Flores-Bueso, Global Young Academy
• Dr. Elizabeth (Lizzie) Gadd, Loughborough University, Coalition on Advancing Research Assessment (CoARA)
• Dr. Geeta Swamy, Duke University, Higher Education Leadership Initiative for Open Scholarship (HELIOS)
• Dr. Giannis Tsakonas, University of Patras, LIBER Europe

Moderator: Mx. Meg Wacha, Dag Hammarskjöld Library

This virtual panel discussion will take place on 17 July 2024, from 8:30 to 9:45 a.m. (EDT – New York Time). It promises to be a vital conversation, bringing together diverse perspectives on the intersection of open science and sustainable development, and the ways in which policy can influence and accelerate progress towards achieving the SDGs.

Join for this essential event to understand how the global shift towards open science can be measured and managed to ensure positive outcomes for science and society alike. Register now to secure your place in this important dialogue.

Introduction

The rise of Open Access (OA) publishing has transformed the landscape of academic research dissemination. While the OA model offers significant benefits in terms of accessibility and visibility, it also imposes financial burdens on researchers and funding agencies due to Article Processing Charges (APCs). This study investigates the extent of APC expenditures in Spanish publicly funded research, specifically focusing on projects funded under the Competitive Knowledge Generation (CKG) call.

Methodology

We conducted a comprehensive analysis of APC expenditures associated with CKG projects awarded between 2013 and 2019. Our dataset, which covers 94% of Gold and Hybrid journals, offers a substantial but not complete view of APC-related costs. The analysis also examines the relationship between APC expenses and project budgets across different research fields.

Findings

Quantifying APC Expenditures

The first research question (RQ1) aimed to quantify the amount of money spent on APCs in CKG projects. Our findings reveal that at least €45.87 million were transferred to various publishers for APCs. This figure, though significant, is likely an underestimate given the dataset’s incomplete coverage of journals. Notably, the APC expenditure is almost equivalent to the total budget allocated for CKG psychology projects during the same period (€45.83 million). This raises critical questions about alternative uses for these funds, such as personnel hiring, equipment procurement, or the development of alternative publishing models.

APC Expenditures Relative to Project Budgets

To contextualize APC expenses, we compared them with project budgets, addressing our second research question (RQ2). The analysis shows that APC costs constitute an average of 3 to 8% of the total project funding. However, this percentage varies significantly across disciplines, with some projects spending over half of their budget on APCs. This variability underscores the need for strategic discussions on project funding allocation and the compatibility of meeting OA mandates while fulfilling project objectives. Moreover, researchers often rely on personal funds or transformative agreements to cover APCs, highlighting potential inequities in the APC model, particularly for researchers in less affluent regions or early-career stages.

Field-Specific APC Expenditures

Our third research question (RQ3) explored the correlation between publication intensity and APC expenditures. Results indicate that fields with higher publication rates tend to spend more on APCs. However, this relationship is complex. Some high-publication fields have developed cost-effective OA strategies, such as utilizing embargo periods or preprint servers, which mitigate APC costs. Conversely, fields with lower publication rates, like Humanities and Social Sciences, often publish in journals that do not charge APCs, further affecting expenditure patterns.

Project Budget Size and APC Expenditures

The fourth research question (RQ4) examined whether projects with larger budgets spend more on APCs. Our analysis confirmed this trend, suggesting that ample funding reduces the pressure to minimize APC costs. However, this situation disadvantages researchers with smaller budgets, potentially exacerbating funding inequalities. Institutional and regional disparities further complicate this issue, with institutions specializing in well-funded fields being more likely to secure larger projects and, consequently, spend more on APCs.

Availability of OA Journals and APC Expenditures

The final research question (RQ5) investigated the correlation between the availability of OA journals and APC expenditures. We found a negative correlation: higher availability of OA journals, especially those without APCs (Diamond journals), correlates with lower APC expenditures. This suggests that increasing the number of OA journals can alleviate some financial pressures associated with APCs. However, the relationship is multifaceted and may involve market dynamics that influence APC pricing strategies.

Discussion and Conclusion

The study highlights significant financial implications of the OA publishing model on Spanish publicly funded research. As OA mandates become more prevalent and the Gold OA model expands, APC expenditures are likely to rise. The development of alternative publishing models, such as Diamond journals, presents a potential solution to mitigate these costs. National case studies, like this one, are essential for understanding the broader impact of APCs and informing strategies to ensure equitable access to OA publishing.

Future research should focus on obtaining a more accurate estimate of APC expenditures by including publications in Spanish journals. Additionally, exploring cross-country comparisons of APC expenses and funding allocations could provide valuable insights into the global landscape of OA publishing. Understanding researchers’ behaviors and funding strategies through qualitative methods would also contribute to a more nuanced view of APC-related challenges.

In conclusion, while APCs facilitate OA publishing, they also pose significant financial challenges. Addressing these issues requires a multifaceted approach involving policy adjustments, funding reallocations, and the promotion of alternative publishing models. This study contributes to the ongoing discourse on sustainable and equitable OA publishing practices.

Alonso-Álvarez, P., Sastrón-Toledo, P. & Mañana-Rodriguez, J. The cost of open access: comparing public projects’ budgets and article processing charges expenditure. Scientometrics (2024). https://doi.org/10.1007/s11192-024-04988-3

In June, the Japanese government announced a significant move toward achieving its goal of making publicly funded research papers freely accessible by April 2025. As reported by Dalmeet Singh Chawla for Nature News, this initiative positions Japan as one of the pioneering countries to implement a nationwide open access (OA) plan.

Major Investment in Infrastructure

To facilitate the transition to OA, the Japanese government has committed ¥10 billion (approximately £50 million) to standardize data and publication repositories across universities. Each institution will maintain its own repository to host research produced by its academics. However, these repositories will be integrated into a single national server. This integration will create a unified record of all research produced by Japanese academics, ensuring that articles published in Japanese are not overlooked.

Adopting a Green OA Strategy

Japan’s approach to open science will be based on the green OA model, which the government believes is more feasible for universities compared to the gold OA model. Green OA involves self-archiving where researchers publish their work in repositories, making it freely available. This strategy has received positive feedback from experts in the field of open science and OA.

Johan Rooryck, Executive Director of cOAlition S, endorsed the use of green OA, particularly for content currently behind paywalls. He emphasized that this model would help democratize access to research. Similarly, Kathleen Shearer, Executive Director of the Confederation of Open Access Repositories, praised the Japanese government’s plans for their equitable nature, highlighting how they would ensure broader and more inclusive access to scientific knowledge.

With this strategic investment and commitment to green OA, Japan is set to lead the way in making academic research more accessible, fostering a more inclusive and collaborative scientific community on a national scale.

Original article via The Publication Plan

The current system of academic publishing serves only a select few and needs to be overhauled to better align with academic ideals and provide better value for money, writes Tim Glawion.

It’s widely acknowledged that academic publishing is problematic. Unlike other complex socio-economic phenomena that become less intimidating as you learn more about them, the deeper you delve into publishing, the more concerning it becomes. One bright spot is the Open Science movement, which holds great promise and attracts dedicated individuals, many in Africa. However, it still lags behind the private sector in usability, accessibility, and reach.

Despite understanding the limitations of impact metrics, scholars continue to publish in high-impact journals because these metrics serve as recognizable benchmarks in the competitive academic job market. Academics tolerate exorbitant article processing charges (APCs) of up to £8,000, which have no relation to the actual cost of publishing, because research institutions cover these fees. This practice reinforces the dominance of these institutions and marginalizes those who cannot afford the “open access” fees, resulting in their work being cited less.

Journal editors often remain with large publishers because these publishers simplify their tasks by negotiating impact factors with Clarivate, submitting articles to databases, and providing services like copyediting, typesetting, and archiving. Starting a new independent journal quickly reveals the challenges of handling these responsibilities without such support.

The capitalist nature of scholarly publishing undermines its quality. When profits are tied to APCs, there is an incentive to increase submissions at the expense of quality assurance. This might explain the proliferation of in-house journals and series by large publishers and the disproportionate increase in the number of published articles compared to doctorates issued.

Non-capitalist publishing alternatives align more closely with academic principles of free scholarly exchange and public outreach but are often too complicated. For instance, hosting a journal using the Open Journal System requires web development and online database skills, which can be daunting for most users. More publicly funded institutions are needed to offer the comprehensive services that large, profit-driven publishers currently provide.

Such institutions are rare but do exist. For example, Science for Africa funds and connects Open Science projects, the Public Knowledge Project enables journal hosting, and Open Research Europe publishes through open peer review. Additionally, university librarians in North America or Europe can offer a surprising range of in-house not-for-profit publishing services tailored to academic needs.

So why aren’t these alternatives more widely known and used? European science funders have created parallel tracks, doubling the financial burden on taxpayers instead of reducing it. For example, many institutions fund the overpriced APCs of large publishers through public science funds—the same funds meant to support independent Open Access journals. This approach supports both David and Goliath, without any clear rationale.

A New System

To move away from the current flawed system, we must make it harder to profit excessively and simplify Open Science. Additionally, journals and articles should be evaluated separately.

First, we need a clear alternative to the Journal Impact Factor. Current alternatives are merely variations of quantified metrics. Journals should be ranked based on the quality of service they offer authors, including meaningful review processes and professional editing. This would encourage a race to the top in scholarly services.

Second, we need to reassess the role of articles in academic careers. The trend of measuring an academic’s worth through journal rank, personal citation scores, and publication counts needs a straightforward alternative that conveys a scholar’s standing. A relational score, potentially utilizing Artificial Intelligence to analyze an author’s work for its methodological, theoretical, and empirical depth, could be beneficial. Hiring committees should focus on finding the right fit for the team, not just the most high-profile candidate.

Third, we must stop paying APCs. Instead, long-term funding should be directed to Open Access journals hosted by academic institutions globally, especially in Africa where research and publishing funds are scarce. This funding should collaborate with local publishers to professionalize the OA movement, preventing it from becoming as US-Eurocentric as the broader higher education sector. Localizing scholarship can help integrate indigenous knowledge and decolonizing epistemologies into global academia.

Shifting from APC funding to journal and infrastructure funding is a political decision that could save taxpayers money. This ongoing debate about how we fund academic research and publishing should aim to establish a simpler, more equitable system.

About the Author

Tim Glawion is an interim professor of political science at the University of Freiburg and a senior researcher at the Arnold-Bergstraesser-Institute. He was co-Editor in Chief of Africa Spectrum from 2022 to 2024 and is the author of “The Security Arena in Africa” (Cambridge University Press).

Original article at LSE

Photo via Royal Society of Chemistry