multiobs

Our Recent Productions

Datasets, working papers, and peer-reviewed articles produced by MultiObs researchers.

Showing 1–19 of 19

  1. Working Paper2026Open Access
    From interdependent movements to shared action: Connecting publishing and assessment reforms(opens in a new tab)

    Waltman, Ludo; de Moura Rocha Lima, Giovanna; Corker, Katherine S.; Dijkstal, Felix; Lawrence, Rebecca; Barbour, Virginia; Boulton, Geoffrey; Brasil, André; Dube, Nosisa; Gadd, Elizabeth; Hatch, Anna; Hill, Mandy; Joseph, Heather; Kiermer, Veronique; Koley, Moumita; Lynott, Dermot; Murray, Susan; Naidorf, Judith; Pölönen, Janne; Pattinson, Damian; Puebla, Iratxe; Ratan, Kristen; Rieck, Katharina; Shawrav, Mostafa Moonir

    Research assessment and scholarly publishing play interconnected roles in shaping research culture, incentives, and norms that ultimately determine what knowledge is produced, communicated, trusted, and rewarded. These two interdependent components of the research system are undergoing change, fueled by initiatives that share core principles, goals, and challenges, but act largely in parallel. Coordinated action in assessment and publishing reform holds the opportunity for more effective, fairer, and more durable change. The problems in research assessment and scholarly publishing are well documented, from overreliance on journal prestige and the dominance of selective bibliographic databases to the underrecognition of peer review and the limited visibility of research outputs beyond journal articles. Various reform initiatives have emerged to deal with these problems, both in the research assessment domain and in the scholarly publishing domain. However, these problems cannot be effectively addressed by a reform in either of these domains alone. Reform needs to be coordinated across domains. Fragmented reform efforts may not only duplicate work, but may also send inconsistent signals to researchers and institutions, which may undermine the adoption of more responsible practices. Stronger coordination creates tangible benefits across the research system. For reform initiatives, it increases collective reach, legitimacy, and influence while reducing duplication of effort and strengthening shared messages. For research performing and funding organizations, and policymakers, it provides a more coherent foundation for policy development and implementation, enabling organizations to draw on shared experience while adapting approaches to their own missions and contexts. For researchers, coordination helps ensure that emerging practices in scholarly publishing are recognized and rewarded consistently, reducing uncertainty and creating more pathways for diverse contributions to be valued. In this paper, we give an overview of how we are considering these issues and propose ways forward for coordinating the reform movements in research assessment and scholarly publishing. Importantly, this coordination is not intended to replace existing initiatives or create new layers of governance. Rather, it is a light-touch and inclusive effort to coordinate agendas, amplify existing work, and promote shared learning.

  2. Working Paper2026Open Access
    Collective action for Open Research Information: Introducing ORION-DBs and a new FORCE11 Working Group(opens in a new tab)

    Mazoni, Alysson; Kramer, Bianca; Neylon, Cameron; Costas, Rodrigo; van Eck, Nees Jan

    With ORION-DBs, a growing set of actors are hosting large research information datasets in shared spaces, notably in Google BigQuery. To coordinate amongst current and future contributors and work towards community standards for discovery, processing, preservation and documentation, we are starting a FORCE11 working group. The working group will also seek to build a community around future alternatives to proprietary cloud systems for data sharing at scale and identify future paths to community-led training and development resources. Our goal is to build a community with a focus on supporting the users of Open Research Information resources to make more sophisticated and complex use of these data sources. We will do this by consolidating and expanding the resources themselves through supporting providers, gathering user stories and developing training materials. Finally we will scope pathways to technological independence from proprietary systems consistent with the needs of users. This webinar will introduce the ORION-DB initiative, present a number of use cases and introduce the FORCE11 Working Group. If you are a data provider, metadata user or infrastructure developer and want to learn more about ORION-DB and the planned activities of the working group, you are invited to attend the webinar. The version of this slide deck on Google was originally: https://docs.google.com/presentation/d/1ua88tr2gP2HTurQPkM4-A7eWjFiDLUI_ht3MJdHxOxc/edit?usp=sharing

  3. Working Paper2026Open Access
    Report: Researchers just love high impact journals, but are there career-proof alternatives?(opens in a new tab)

    Brasil, André; van Himbergen, Thomas; Nooren, Irene M.A.; Constantin, Maria Ermioni

    At the SURF research day 2026, we asked the attendees of our workshop if they agree or not with four statements. This report summarizes the views of the participants. Slides are available at Presentation: Unspoken truth: Everyone wants to publish in high impact factor journals. But how to adopt the alternatives | Zenodo

  4. Working Paper2026Open Access
    MEECSE: Monitoring the implementation of Enabling Environments for Citizen Science & Societal Engagement practices(opens in a new tab)

    Gold, Margaret; Costas, Rodrigo; Wehn, Uta

    The MEECSE project furthers the NPOS Ambition2030 and rolling agenda 'towards societal engagement and participation' by supporting the implementation of effective actions to establish enabling environments for Citizen Science & Societal Engagement (CS&SE) practices. The core activities in the project are to: identify ways that enabling environments for CS&SE are being implemented, identify barriers to implementation, develop and test indicators for monitoring progress and effectiveness of enabling actions identify related CS&SE impact indicators, and seek integration of both indicator-sets in OS monitoring frameworks, thereby strengthening the evidence base for CS&SE practices within Open Science.

  5. Working Paper2026Open Access
    Are University Journals Truly Local? National and Institutional Anchoring in Seven Research Systems(opens in a new tab)

    Nazarovets, Maryna; Beigel, Fernanda; Van Schalkwyk, François B.; Taşkın, Zehra; Meng, Ping

    University journals (UJs) are often described as local or inward-looking, although national and institutional embeddedness are rarely distinguished. This study examines the internationally indexed segment of university journals, analysing Web of Science-indexed titles from seven countries with contrasting publishing traditions and evaluation regimes (Argentina, China, Germany, South Africa, Türkiye, Ukraine, and the United Kingdom), based on publications from 2022-2024. National anchoring (N-Anch) measures the share of papers with at least one author from the journal’s home country, while institutional anchoring (I-Anch) captures participation by authors from the publishing university. The results reveal a clear divergence between these dimensions. National anchoring is high in Türkiye, China, Ukraine, South Africa, and Argentina, but considerably lower in Germany and UK university-press journals. Institutional anchoring remains low across all countries. Thus, strong integration into a national research community does not imply institutional anchoring. The findings show that patterns of national and institutional anchoring vary substantially across countries with different research, evaluation, and publishing systems and cannot be explained by university ownership alone.

  6. Working Paper2026Open Access
    English dominance in scholarly communication: Historical roots and bibliometric evidence(opens in a new tab)

    Brasil, André

    English functions as the principal shared language of international scholarly communication, but its dominance also creates inequalities in the production, dissemination, visibility, and assessment of research. Developed as a complementary output of the CoARA Working Group on Multilingualism and Language Biases in Research Assessment, this report examines the historical development of English dominance and the contemporary mechanisms through which it is reproduced. It combines evidence from the literature with descriptive bibliometric analyses based on Web of Science, Scopus, OpenAlex, and CWTS Journal Indicators. The findings show that English is especially dominant in journals with high field normalised citation impact and in the selective segments of scholarly communication most frequently used in research evaluation. However, the extent of this dominance varies considerably across disciplines and countries, and publication in national languages remains important in the social sciences, humanities, and fields directed toward local professional and societal audiences. The report also demonstrates how database selection, metadata quality, and analytical choices affect the visibility of multilingual scholarship. Web of Science and Scopus predominantly represent English language journal literature, whereas OpenAlex reveals a much broader body of multilingual research, although incomplete language, authorship, and institutional metadata remain important limitations. The report further examines the financial, linguistic, and time burdens associated with English language publication and considers how publishing infrastructures, bibliometric indicators, rankings, and assessment practices influence which research becomes visible and valued. This report contributes to the landscape analysis conducted by the Coalition for Advancing Research Assessment (CoARA) Working Group (WG) on Multilingualism and language biases in research assessment and it complements the Implementation proposal for language-aware assessments (Annex II.3).

  7. Working Paper2026Open Access
    Appendix A. Definitions of technological information(opens in a new tab)

    Damian, Ieda Pelógia Martins; Araujo de Rezende Alvares, Lillian Maria; Segundo, Washington; Garcia de Souza, Marcel

    Introduction: technological information plays a strategic role in scientific, technological, and industrial development, especially in contexts marked by innovation and competitiveness. Objective: this study aims to present an updated concept of technological information that reflects current demands in academic and productive sectors. Methodology: a Systematic Literature Review (SLR) was conducted using national and international databases in the field of Information Science. The search resulted in 1,108 documents, from which 68 relevant definitions of technological information were selected and analyzed. Results: the analysis enabled the identification of recurring themes and common elements in existing definitions, allowing for a deeper understanding of the concept. The study highlights the main characteristics and functions of technological information, such as its association with innovation, competitiveness, and strategic decision-making. It emphasizes its role in improving products, processes, and services, and its relevance in patents and research outputs. Conclusion: the study proposes a new definition of technological information as knowledge related to technologies that generate strategic value through their application. This updated definition contributes to theoretical advances in Information Science and offers practical support to organizations aiming to use technological information as a strategic asset. It also broadens the understanding of the term’s scope, reinforcing its importance for sustainable development and innovation.

  8. Working Paper2026Open Access
    Different operationalizations of local research and their associated citation patterns: A study of African publications(opens in a new tab)

    Alonso-Álvarez, Patricia; Boshoff, Nelius

    Local research has gained increasing attention in recent years, with a growing body of literature emphasizing its significance—especially for peripheral communities underrepresented in mainstream scientific discourse. However, despite this growing interest, there remains no clear consensus on how to define local research. Conceptual clarity is essential for conducting consistent and replicable analyses that capture the specific characteristics, contexts, and societal relevance of local research. This need is particularly acute in the Global South, where structural inequalities in visibility and recognition often marginalize regionally focused scholarship. In Africa, for instance, a substantial share of research is published in local or regional journals that are poorly represented in global indexing databases. As bibliometric analyses typically rely on these databases, this exclusion diminishes their perceived relevance. To address this gap, the present study incorporates data from regional and open-access sources to examine citation patterns associated with African local research. It uses multiple conceptualizations of local research to classify publications as either local or non-local depending on the definition used. These classifications are correlated with three citation-based indicators: average citations per publication, time to first citation, and the proportion of uncited publications. Additionally, the study explores the characteristics of citing publications, distinguishing between local and non-local sources as well as domestic and international citations. The findings reveal that citation patterns differ significantly across locality definitions, underscoring the importance of adopting nuanced, context-sensitive approaches to assess and support local research.

  9. Working Paper2026Open Access
    Operationalizing FAIR in National Research Information Systems: The BrCris Example and Its Role in Open Repositories(opens in a new tab)

    Segundo, Washington; Dias, Thiago Magela; Souza, Marcel Garcia; do Canto, Fábio Lorensi; Sena, Priscila Machado

    The adoption of the FAIR principles (Findable, Accessible, Interoperable, Reusable) has become a cornerstone of Open Science, guiding practices aimed at improving the quality, transparency, and reuse of scientific data. In the context of CRIS (Current Research Information Systems), these principles play a strategic role in supporting the integration, standardization, and governance of scientific information on a national scale. This article analyzes the application of the FAIR principles within the Brazilian Scientific Research Information Ecosystem (BrCris), developed by the Brazilian Institute of Information in Science and Technology (Ibict), with the aim of discussing how its technical and organizational practices contribute to building a more reliable, interoperable, and reusable scientific data infrastructure. The methodology adopted is based on a literature review of the FAIR principles and a document analysis of technical reports, scientific articles, and institutional materials related to BrCris. The results indicate that BrCris shows significant progress in adopting the FAIR principles, especially in the use of persistent identifiers, semantic data integration, interoperability with international infrastructures, and the availability of data in open and reusable formats. It is concluded that BrCris constitutes a strategic infrastructure for the implementation of the FAIR principles in Brazil.

  10. Working Paper2026Open Access
    Opening the Black Box of Grant Peer Review: Exploratory Analysis of Linguistic and Sentiment Signals in Funding Outcomes(opens in a new tab)

    Bin, Adriana; Pinto, Daniela Maciel; Pereira, Cesar A. Guimarães; Cristofoletti, Evandro Coggo; Mena-Chalco, Jesus Paschoal; Salles Filho, Sérgio; Sugimoto, Cassidy R.

    Peer review plays a central role in research funding decisions, yet the evaluative language used in review reports remains insufficiently explored, particularly in funding agencies. This study examines how linguistic and sentiment characteristics of peer-review reports relate to funding outcomes at the São Paulo Research Foundation (FAPESP), Brazil, focusing on the Regular Research Grant (RRG) and Young Investigator Grant (YIG) schemes between 2000 and 2017. Drawing on 19,333 proposals and their corresponding review reports, we analyze three aggregated corpora — overall reports, proposal-related reports, and applicant track-record reports — using natural language processing and sentiment analysis. We compare approved and rejected proposals across fields of knowledge, reviewer gender, applicant gender, and peer-review report sections. Results show that reviewers use distinct textual and sentiment patterns when justifying approval or rejection. These patterns vary according to the section evaluated — applicant track record or technical proposal — and also differ by field of knowledge and, more locally, by reviewer–applicant gender configurations. The pattern is more regular and stable in RRG than in YIG. Overall, the findings show how evaluative language illuminates fairness, consistency, and transparency in grant peer review.

  11. Working Paper2026Open Access
    Global research funding flows according to available grant data in Dimensions(opens in a new tab)

    Dudek, Jonathan; Ni, Rong; van Honk, Jeroen; Ding, Gabrielle; Bovenzi, Nicandro; Durán-Silva, Nicolau; Ràfols, Ismael; Costas, Rodrigo

    While most research funding is national, international funding plays a relatively important role, particularly for some regions of the Global South. However, the extent of funding flows between world regions remains unclear. This study represents a first attempt to map global research funding flows based on a set of grant metadata from Dimensions, augmented with data from the Gates Foundation and the European Commission (CORDIS). We find major regional gaps in the data outside of the leading scientific regions. With the data currently available, we find that Europe and Sub-Saharan Africa are the biggest recipients of inter-regional funding, with North America as main funder, and that over the last 25 years, total annual research funding has grown to above USD 60 billion, while inter-regional funding flows have stagnated at around USD 1.6 billion per year. These insights are contingent on data availability, raising the urgency for improving funding metadata infrastructures.

  12. Working Paper2026Open Access
    Exploring the use of funding acknowledgments for mapping global research funding flows(opens in a new tab)

    Ni, Rong; van Honk, Jeroen; Dudek, Jonathan; Ding, Gabrielle; Ràfols, Ismael; Costas, Rodrigo

    Funding acknowledgments (FAs) provide a potential but underexplored source for mapping global research funding flows. This study develops a framework for constructing global funding flow measures using FA data from Dimensions and assesses both the opportunities and limitations of this approach. We compare total and inter-regional funding flows based on funding grant information available in FAs of scientific publications. Our analysis identifies a key limitation arising from collaborative publications, where funding is attributed at the publication level rather than to individual authors. This obscures the interpretation of cross-border funding flows. We show that this attribution structure introduces systematic ambiguity and limits the validity of FA-based analyses. We argue that disaggregated, author-level funding attribution is essential for accurately tracing funding flows. This requires coordinated action among funders, publishers, and researchers, as well as the adoption of more standardized, machine-readable FAs formats.

  13. Working Paper2026Open Access
    The benefits and challenges of understanding funding flows in the Global South(opens in a new tab)

    Ràfols, Ismael

    This is a position paper (still as a work-in-progress) with the main insights of the project "Tracking Research Funding Flows in the Global South" funded by IDRC and presented in the Wellcome Trust in the workshop 'Mapping research funding flows for global alignment and impact' on 23rd of April 2026. The full list of contributors to the project is included in the document. A long and a short presentation of this position paper are also attached as powerpoint files. A recording of the long presentation in the centre CENIT of the Universidad San Martín de Buenos Aires, in Spanish, is available here.

  14. Working Paper2026Open Access
    CWTS Engagement & Inclusion focal area strategic plan 2026-2028(opens in a new tab)

    area, CWTS Engagement & Inclusion focal; de Bordes, Carole; Costas, Rodrigo; Holtrop, Tjitske; Traag, Vincent

    This document outlines the strategic plan for the Engagement & Inclusion focal area at the Centre for Science and Technology Studies (CWTS) at Leiden University for the period 2026–2028, presenting the focal area’s vision, long-term strategic impacts, and operational objectives developed through collaborative discussions among members and coordinators.

  15. Working Paper2025Open Access
    The multiversatory: fostering diversity and inclusion in research information by means of a multiple-perspective observatory(opens in a new tab)

    Rafols, Ismael; Costas, Rodrigo; Bezuidenhout, Louise; Brasil, André

    For science and technology to contribute to social justice, scientometrics analyses need to be able to produce descriptions that are appropriate to specific contexts and values. Given that the participation of stakeholders in the analyses is crucial for these perspectives to be realized, this requires research information analyses that are “open”. In this paper, we propose that this “openness” in research information concerns several dimensions. In the first place, research information should be open in the sense of being transparent and accessible. Second, research information should be open in the sense of being inclusive and diverse, which includes two dimensions: that stakeholders can actually use it, and that the contents include different types of knowledge. Third, the research information should be provided in forms that empower interrogation by stakeholders, so that they can retrieve and construct the descriptions of science and technology that are more appropriate for their context. This last step implies efforts to make visible many scientific contributions from the global south which are currently invisible. We call this vision “the multiversatory”: an approach to the observation of science that facilitates pluralistic analyses of the knowledge created across a variety of places and contexts, the pluriverse.