2023 on Crossref

Discovering relationships between preprints and journal articles

Dominika Tkaczyk — Thu, 07 Dec 2023 00:00:00 +0000

In the scholarly communications environment, the evolution of a journal article can be traced by the relationships it has with its preprints. Those preprint–journal article relationships are an important component of the research nexus. Some of those relationships are provided by Crossref members (including publishers, universities, research groups, funders, etc.) when they deposit metadata with Crossref, but we know that a significant number of them are missing. To fill this gap, we developed a new automated strategy for discovering relationships between preprints and journal articles and applied it to all the preprints in the Crossref database. We made the resulting dataset, containing both publisher-asserted and automatically discovered relationships, publicly available for anyone to analyse.

TL;DR

We have developed a new, heuristic-based strategy for matching journal articles to their preprints. It achieved the following results on the evaluation dataset: precision 0.99, recall 0.95, F0.5 0.98. The code is available here.
We applied the strategy to all the preprints in the Crossref database. It discovered 627K preprint–journal article relationships.
We gathered all preprint–journal article relationships deposited by Crossref members, merged them with those discovered by the new strategy, and made everything available as a dataset. There are 642K relationships in the dataset, including:
- 296K provided by the publisher and discovered by the strategy,
- 331K new relationships discovered by the strategy only,
- 15K provided by the publisher only.
In the future, we plan to replace our current matching strategy with the new one and make all discovered relationships available through the Crossref REST API.

Introduction

Relationships between preprints and journal articles link different versions of research outputs and allow one to follow the evolution of a publication over time. The Crossref deposit schema allows Crossref members to provide these relationships for new publications, either as a has-preprint relationship deposited with a journal article, or an is-preprint-of relationship deposited with a preprint.

To assist members who deposit preprints, we also try to connect deposited journal articles with preprints. The current method looks for an exact match between the title and first authors. We send possible matches as suggestions to the preprint server, which decides whether to update the metadata with the relationship.

At the time of writing, 137,837 journal articles in the Crossref database have a has-preprint relationship¹, and 562,225 works of type posted-content (preprints belong to this type) have an is-preprint-of relationship².

We suspected that many preprint–journal article relationships are missing, as some members inevitably fail to deposit them, even after suggestions from the current matching strategy. Another factor is that the current strategy is fairly conservative, and probably misses a significant number of relationships. For these reasons, we decided to investigate whether we could improve on the current process. Doing so would allow us to infer missing relationships on a large scale, similar to how we automatically match bibliographic references to DOIs.

This preprint matching task can be defined in two directions:

We start with a journal article and we want to find all its preprints.
We start with a preprint and we want to find a subsequently published journal article.

On the one hand, matching from journal articles to preprints would allow us to enrich the database continually with new relationships, either periodically or every time new content is added. Since journal articles tend to appear in the database later than their preprints, it makes sense for a new journal article to trigger the matching and not the other way round. This way we can expect the potential matches to be already in the database at the time of matching.

On the other hand, matching from preprints to journal articles can be useful in a situation where we want to add relationships in an existing database retrospectively. In our case, the database contains many more journal articles than preprints, so for performance reasons it is better to start with preprints.

In both cases we are dealing with structured matching, meaning that we match a metadata record of a work (preprint or journal article), rather than unstructured text.

As a result of matching a single preprint or a single journal article, we should expect zero or more matched journal articles/preprints. Multiple matches occur when:

there are multiple versions of the matched preprint and/or
matched works have duplicates.

The image shows the result of matching a journal article to two versions of a preprint:

Matching strategy

Our matching strategy uses the following workflow:

Gathering a short list of candidates using the Crossref REST API.
Scoring the similarity between the input item and each candidate.
A final decision about which candidates, if any, should be returned as matches.

Gathering candidates is done using the Crossref REST API’s query.bibliographic parameter. The query is a concatenation of the title and authors’ last names of the input item. We filter the candidates based on their type, to leave only preprints or only journal articles, depending on the direction of the matching. In the future, instead of getting the candidates from the REST API, we will be using a dedicated search engine, optimised for preprint matching.

Scoring candidates is heuristic-based. Similarities between titles, authors, and years are scored independently, and the final score is their average. Titles are compared in a fuzzy way using the rapidfuzz library. Authors are compared pairwise using the ORCID ID, or first/last names if ORCID ID is not available. The similarity score between issued years is 1 if the article was published no earlier than one year before the preprint and no later than three years after the preprint, or 0 otherwise.

The final decision is made based on two parameters: minimum score and maximum score difference, both chosen based on a validation dataset. The following diagram depicts the results of applying these two parameters in all possible scenarios. First, any candidate scoring below the minimum score is rejected (grey area in the diagram). Second, the scores of the remaining candidates are compared with the score of the top candidate. If the score of a candidate is close enough to the score of the top candidate, it is returned as a match (blue area).

This process can result in the following scenarios:

Scenario A: there is no candidate above the minimum score. This means nothing matches sufficiently, so nothing is returned.
Scenario B: there is only one candidate above the minimum score. This means it is the best match and we don’t have much of a choice, so it is returned.
Scenario C: there are multiple candidates above the minimum score, and they all have similar scores. This means they all are similarly good matches, so all are returned.
Scenario D: there are multiple candidates above the minimum score, but their scores differ a lot. In this case, we don’t want to return all of them, but only those that are close to the top match. Intuitively, we don’t want to return less-than-great matches if we have really great ones. This is when the maximum score difference comes into play: we return the candidates with the “score distance” to the top candidate lower than the maximum score difference.

We evaluated this strategy on a test set sampled from the Crossref metadata records. The test set contains 3,000 pairs (journal article, set of corresponding preprints). Half of the journal articles have known preprints and the other half don’t. The test set can be accessed here.

We used precision, recall, and F0.5 as evaluation metrics:

Precision measures the fraction of the matched relationships that are correct.
Recall measures the fraction of the true relationships that were matched.
F0.5 combines precision and recall in a way that favours precision.

The strategy achieved the following results: precision 0.9921, recall 0.9474, F0.5 0.9828. The average processing time was 0.96s.

We have made this strategy (journal article -> preprints) available through the (experimental) API: https://marple-research-crossref-org.pluma.sjfc.edu/match?task=preprint-matching&strategy=preprint-sbmv&input=10.1109/access.2022.3213707. The input is the DOI of a journal article we want to match to preprints, and the output is a list of matches found, along with the score for each.

We have investigated other approaches to making decisions about which candidates to return as matches (step 3 above), including using machine learning. At present none have outperformed the heuristic approach described above. The heuristic method is also preferred because of its fast performance.

Preprint–journal article relationship dataset

We applied the strategy to the entire Crossref database:

We selected all preprints published until the end of August 2023. This included only works with type posted-content and subtype preprint, as reported by the REST API. There were 1,050,247 of them.
We ran the matching strategy (preprint -> journal article) on them. This resulted in 627,011 preprint–journal article relationships.
The resulting relationships were combined with the relationships deposited by the Crossref members. We included relationships of types has-preprint or is-preprint-of, where both sides of the relationship exist in our database, were published until the end of August 2023, and are of proper types and subtypes (type=journal-article for the journal article and type=posted-content, subtype=preprint for the preprint).

The resulting dataset is a single CSV file with the following fields:

preprint DOI (string)
journal article DOI (string)
whether the publisher of the journal article deposited this relationship (boolean)
whether the publisher of the preprint deposited this relationship (boolean)
the confidence score returned by the strategy (float, empty if the strategy did not discover this relationship)

The dataset contains:

641,950 relationships in total, including 580,532 preprints and 565,129 journal articles,
14,939 of them were deposited by the Crossref members, but not discovered by the strategy,
330,826 of them were discovered by the strategy, but not provided by any Crossref member,
296,185 of them were both deposited by a Crossref member and discovered by the strategy.

The dataset can be downloaded here.

Conclusions and what’s next

Overall, based on the number of existing and newly discovered preprint–journal article relationships, it seems that employing automated matching strategies would approximately double the number of these relationships in the Crossref database. In the future, we would like to match new journal articles on an ongoing basis. We also plan to make all discovered relationships available through the REST API.

In the meantime, we will be publishing the discovered relationships in the form of datasets, and we invite anyone interested to further analyse this data. And if you find out something interesting about preprints and their relationships, do let us know!

Perspectives: Madhura Amdekar on meeting the community and pursuing passion for research integrity

Madhura Amdekar — Tue, 05 Dec 2023 00:00:00 +0000

The second half of 2023 brought with itself a couple of big life changes for me: not only did I move to the Netherlands from India, I also started a new and exciting job at Crossref as the newest Community Engagement Manager. In this role, I am a part of the Community Engagement and Communications team, and my key responsibility is to engage with the global community of scholarly editors, publishers, and editorial organisations to develop sustained programs that help editors to leverage rich metadata.

This represents an exciting phase in my professional journey, as I now have the chance to learn and develop new skills, broaden my understanding of the publishing landscape, and at the same time be able to leverage the experience I gained so far. I originally trained as an ecologist, obtaining a PhD studying colour change in a tropical agamid lizard in India at the Indian Institute of Science (Bengaluru, India). Having immensely enjoyed the process of writing manuscripts based on the data that resulted from my PhD thesis, I was drawn to working in the scholarly publishing industry. I worked for 3.5 years as a Senior Associate at Wiley, overseeing an editor support service by devising strategic scale-up planning and process improvement initiatives.

I then moved countries as well as jobs and joined Crossref. The world of scholarly communications is a rapidly changing ecosystem, that is ably supported by scholarly infrastructure - the sets of tools and services that support this industry. Being a part of Crossref, a global organisation that provides open scholarly infrastructure, allows me to work with and make an impact on the broad scholarly community that ranges from publishers of all shapes and sizes, funders, to academic institutions, and researchers.

So far, the integrity of the scholarly record (ISR) has been the focus of my work. Now more than ever, the community is cognizant of the need to uphold the integrity of the scholarly output. Metadata and relationships between research outputs can support this endeavour in a substantial manner because information such as who contributed to a research output, who funded it, who cites it, whether it was updated after publication, aids provenance and provides signals about whether the output is trustworthy.

Most of Crossref’s tools and services play a key role here: be it reference linking to allow researchers to increase discoverability of their work, tracking post-publication updates to research outputs via Crossmark, or detecting text plagiarism via Similarity Check. We noticed that not all editors and editorial teams associate metadata as signals of integrity, and might be unaware of the benefits of rich metadata. Therefore, my priority is to utilise opportunities to engage with editors about how metadata can provide trust indicators about a research output. I aim to empower editors to collect and leverage rich metadata.

While I am no stranger to the world of scholarly communications, engaging with the broader Crossref community has been a new experience for me. In my day to day work, I employ a range of different skills such as program design and management, content planning and outreach, networking, and meeting facilitation. I have also been participating in trainings to enhance my skill set – I recently completed a training course on Community Engagement Fundamentals, which has equipped me with a better understanding of the concepts and strategies that I will need as a community manager. Additionally, I also underwent the Group Facilitation Methods training course led by the Institute of Cultural Affairs (ICA) where I learnt a couple of effective methods for group facilitation and leading workshops.

Equipped with these skills, I have moderated a few community events already – most prominently the community call about Crossref and Retraction Watch to discuss Crossref’s acquisition and opening up of the Retraction Watch database. It was a valuable experience to contribute to the planning of an online event and host a panel of distinguished guests.

I was also fortunate to be able to meet our community members in-person: I supported the organisation of the Frankfurt roundtable event that was held as part of Crossref’s Integrity of the Scholarly Record (ISR) program, where we engaged with community members to get their perspectives on how to work together towards preserving the integrity of the scholarly record (keep watching this space for a forthcoming blog summarising the outcomes from this event!). Additionally, I attended the Frankfurt Book Fair – the experience of getting to meet our members and to hear from them first-hand about all things Crossref, was unparalleled! I used this opportunity to meet several of our publisher members and discuss their view points about engaging with editors on ISR. The idea was received positively: we heard specific suggestions of metadata that would be of interest to readers of scientific manuscripts, and our members also expressed interest in finding out more about how metadata can act as markers of trust for a research output. I plan to use the insights from these meetings for the development of the ISR editor engagement program.

As I reflect on the past three months, there are a few things that have stood out to me. In terms of work, no two days are the same. My work plan for the day can range from making presentations for outreach activities, creating content such as this blogpost, working on an engagement strategy, to planning events, attending online or offline community meetings, facilitating or moderating some of those events, and networking with community members. This variety in work keeps me motivated to give my best each day. I am also grateful that I have the ability to make an impact with my work in an area that I am passionate about. In my previous job, I had developed a good understanding of research integrity and publication ethics. As a community manager now, I’m looking to work with editorial teams on the integrity of the scholarly record. This role gives me an opportunity to further nurture this interest of mine.

At times, working from home remotely has been a challenge. However, I have enjoyed attending in-person events as they are not just a chance to meet our community members, but also a chance to meet my colleagues and connect with them.

I feel privileged to be able to connect with research communities all over the world and make a meaningful contribution towards supporting the discoverability and impact of their work. I am particularly excited to work at the forefront of shaping the future of preserving the integrity of the scholarly record, in tandem with our community. If this is a topic that excites you as well, I am keen to hear from you. It has been a wonderful three months at Crossref so far and I look forward to future collaborations with our community to develop effective ways of supporting and empowering editors to make the most of metadata for their publications.

Joint statement on research data

Hylke Koers — Tue, 28 Nov 2023 00:00:00 +0000

STM, DataCite, and Crossref are pleased to announce an updated joint statement on research data.

In 2012, DataCite and STM drafted an initial joint statement on the linkability and citability of research data. With nearly 10 million data citations tracked, thousands of repositories adopting data citation best practices, thousands of journals adopting data policies, data availability statements and establishing persistent links between articles and datasets, and the introduction of data policies by an increasing number of funders, there has been significant progress since. It now seems appropriate to focus on providing updated recommendations for the various stakeholders involved in research data sharing.

The premise of the original joint statement still stands: most stakeholders across the spectrum of researchers, funders, librarians and publishers agree about the benefits of making research data available and findable for reuse by others. This improves utility and rigor of the scholarly record. Still, research data sharing is not yet a self-evident step in the research lifecycle. We now have sufficient scholarly communication infrastructure in place to bring about widespread change and believe momentum is building for collective action.

It is in this context that DataCite, a global membership community working with over 2800 repositories around the world, and STM, whose membership consists of over 140 scientific, technical, and medical publishing organisations, are issuing this joint statement. Crossref, a nonprofit open infrastructure with over 18,000 institutional members from 150 countries, joins this call, recognising the need for an amplified focus on data citation. The aim of this statement is to accelerate adoption of best practices and policies, and encourage further development of critical policies in collaboration with a wide group of stakeholders.

Signatories of this statement recommend the following as best practice in research data sharing:

When publishing their results, researchers deposit related research data and outputs in a trustworthy data repository that assigns persistent identifiers (DOIs where available). Researchers link to research data using persistent identifiers.
When using research data created by others, researchers provide attribution by citing the datasets in the reference section using persistent identifiers.
Data repositories enable sharing of research outputs in a FAIR way, including support for metadata quality and completeness.
Publishers set appropriate journal data policies, describing the way in which data is to be shared alongside the published article.
Publishers set instructions for authors to include Data Citations with persistent identifiers in the references section of articles.
Publishers include Data Citations and links to data in Data Availability Statements with persistent identifiers (DOIs where available) in the article metadata registered with Crossref.
In addition to Data Citations, Data Availability Statements (human- and machine-readable) are included in published articles where appropriate.
Repositories and publishers connect articles and datasets through persistent identifier connections in the metadata and reference lists.
Funders and research organisations provide researchers with guidance on open science practices, track compliance with open science policies where possible, and promote and incentivize researchers to openly share, cite and link research data.
Funders, policymaking institutions, publishers and research organisations collaborate towards aligning FAIR research data policies and guidelines.
All stakeholders collaborate in the development of tools, processes, and incentives throughout the research cycle to enable sharing of high-quality research data, making all steps in the process clear, easy and efficient for researchers by providing support and guidance.
Stakeholders responsible for research assessment take into account data sharing and data citation in their reward and recognition system structures.

We, the following signatories shall adopt and promote the relevant best practices laid out above. We hope that our action inspires the community, including researchers, research funders, research institutions, data repositories and publishers, to join us in making it easy for researchers to share, link and cite research data.

Endorse the statement here.

What was the talk of #Crossref2023?

Kornelia Korzec — Tue, 21 Nov 2023 00:00:00 +0000

Have you attended any of our annual meeting sessions this year? Ah, yes – there were many in this conference-style event. I, as many of my colleagues, attended them all because it is so great to connect with our global community, and hear your thoughts on the developments at Crossref, and the stories you share.

Let me offer some highlights from the event and a reflection on some emergent themes of the day. You can browse the recordings and slides archived on our Annual Meeting page.

Ginny Hendricks opened the meeting by reminding everyone about the research nexus vision, and the work that’s underway to bring us closer to it. Ginny went on to highlight progress in metadata and relationships being registered by our members, and mentioned members that have particularly rich metadata records – with the special joint recognition for learned societies of South Korea. Participation statistics can be reviewed in our Labs Member Metadata Metrics Tables.

Since 2018 we’ve seen a 512% increase in the number of abstracts included in the metadata; with Wiley’s recent addition of millions of abstracts to their records largely contributing to this change. On the relationships side, in the same period, we’ve noted a staggering 3004% growth in preprint-to-article links, and we’re pleased to report a growing number of funding relationships being made available thanks to more and more funders registering Crossref DOIs for grants.

For those who couldn’t join us at such an early hour, Ed Penz included some of these highlights in his own strategic update later in the day. However, he focused on our activity and plans towards fulfilling our four strategic goals:

To contribute to an environment where the community identifies and co-creates solutions for broad benefit
To be a sustainable source of complete, open, and global scholarly metadata and relationships
To be publicly accountable to the Principles of Open Scholarly Infrastructure (POSI) practices of sustainability, insurance, and governance
To foster a strong team—because reliable infrastructure needs committed people who contribute to and realise the vision, and thrive doing it

Speakers from across our global community shared their initiatives too. Most of these talks have been accompanied by posters or abstracts shared on our Community Forum and still available for preview and discussion:

Making data citations available at scale: The Global Open Data Citation Corpus by Iratxe Puebla;
“Who Cares?” Defining Citation Style in Scholarly Journals by Vincas Grigas and Pavla Vizváry;
DOI registration for scholarly blogs by Martin Fenner;
Enhancing Research Connections through Metadata: A Case Study with AGU and CHORUS by Tara Packer, Kristina Vrouwenvelder, Shelley Stall;
Index Crossref, Integrity, Professional And Institutional Development by Engjellushe Zenelaj;
Brazilian retractions in the Retraction Watch Database - RWDB by Edilson Damasio; and
Now that you’ve published, what do you do with Metadata? - by Joann Fogleson.

In addition to these updates, we’ve heard from:

Izabela Szyprowska (OP, European Commission), Nikolaos Mitrakis (RTD, European Commission), and Paola Mazzucchi (mEDRA) talked about the process and rationale of implementing Crossref DOIs for grants at the European Commission; and
Amanda French from ROR/Crossref about the new ‘ROR / Open Funder Registry overlap’ tool.

We also assembled a diverse panel and invited the community to discuss “What we still need to build a robust Research Nexus?” The discussion ranged from how different parts of our community currently use existing metadata, to how we can come together to make improvements, especially in the area of standards and equitability, and touched on metadata priorities. I’ll highlight some of the threads below, but it’s certainly worth engaging with the full recording of the discussion, and offering your own perspective on the Community Forum, commenting below.

Having participated in the whole day of talks, I found that a few themes emerged as popular in the community: data citations, making it easier to register metadata, making better use of metadata, retractions, and equity of participation in the research nexus.

Data citations

With the advances in the Crossref API relationships endpoint, Martyn Rittman demonstrated how we’re now providing more comprehensive support for data citations. You can follow his demonstration in the Collab Notebook he used for the demo and shared for your perusal. He also mentioned that the developments in this feature of our API will soon replace the current service provided via the Events API. Feel free to connect with Martin on the community forum and comment with questions and suggestions.

As mentioned above, DataCite’s Iratxe Puebla mentioned the Make Data Count initiative and the leaky pipeline of data citations we’ve got at the moment in the scholarly literature, obscuring the true picture of data reuse. This prevents the community from recognising and incentivising data creation and reuse appropriately. One way of addressing this is the Global Open Data Citation Corpus. Crossref and DataCite collaborate closely in connecting and making that data available.

Linking datasets, as well as software, was reported as part of the AGU and CHORUS initiative in Enhancing Research Connections through Metadata.

Data sharing and citing is as much a culture as a technology problem. As Iratxe Puebla admitted, there are many norms and processes for capturing and sharing that information,and DataCite is interested to hear about different use cases. As highlighting data’s relationship with works is a growing interest for our community, hopefully more understanding and perhaps even commonality can be built soon.

Making it easier to register metadata

As part of the Demonstrations session, we’ve seen two developments to support members with registering their metadata more easily.

Crossref’s Lena Stoll shared plans for the new version of the Crossref Registration Form, the helper tool for manual registration of metadata, which translates the submission into XML, for inclusion in the Crossref database. At the moment, the form only accepts grant registrations, but it will be bolstered before the end of the year to include journal articles then other record types in time.

Erik Hanson from PKP demonstrated the latest OJS version, commenting on specific changes made in the new version in response to the key pain points reported by users of the previous release.

In addition, we’ve heard of two independent projects by Martin Fenner and Esha Data to enable metadata registration and Crossref DOIs for scholarly blogs.

Making better use of metadata

Supported by the beginner’s demo of our REST API by Luis Montilla, there were many voices about opportunities for making good use of Crossref’s open metadata. Nikolaos Mitrakis of the European Commission talked about the implementation of Crossref IDs for grants as a step towards tracing and connecting the grants with not just academic but also societal outcomes of the awards, and the plans for using those in the evaluation and steering of their funding programmes.

Joann Fogleson of the American Society of Civil Engineers gave a buzzy metaphor of publishers’ role in their work with metadata being comparable with that of a pollinator – collecting the metadata at one end, then registering, displaying and making it available to different services, in order to enable a reacher scholarly environment for discovery.

Many of the major themes have found their way to the discussion of what is still needed to build a robust network of connections between scholarly objects, institutions and individuals. One of the ways Ludo Waltman of CWTS, Leiden University, intends to use our open metadata is as part of the upcoming open-source version of the Laiden rankings and he invited the community to contribute and help optimise this project to provide an alternative to closed and selective databases.

Panellists also spoke of new opportunities in the light of data mining and machine learning. Ran Dang, Atlantis Press, as a publisher shared a concern about the standard of metadata across cultures and disciplines, and the need to digitise past publications – which can then help better leverage multi-lingual scholarship. Matt Buys of DataCite, pointed out to the Global Data Citation Corpus they are developing, which leverages a SciBERT model to pull out data citations, which is brought together with Crossref/DataCite citation metadata.

Opening the data is essential to enabling its wider use, and here Ludo gave the example of the fantastic outcome for references metadata, which has been made open by default for the entire corpus of Crossref-registred works. He hopes that this can inspire us to make similar progress in other areas.

A little on a tangent with regards to metadata use, yet speaking of excellent examples of the community making progress together, Ginny pointed out ROR, how this is becoming a new standard for solving a longstanding problem of standardising affiliations metadata.

Retractions

Perhaps not entirely surprising, given the recent acquisition of the Retraction Watch database by Crossref and making the data openly available, retractions featured in a few different talks at the meeting. First, Lena Stoll and Martin Eve from Crossref, shared how that data can be accessed – that is as the csv file from https://api-labs-crossref-org.pluma.sjfc.edu/data/retractionwatch?[your-email@here](add your email as indicated), and the Crossref Labs API also displays information about retractions in the /works/ route when metadata is available. There are plans for incorporating this information with our REST API in the future.

Ed and Ginny have shown stats for increases in retraction metadata registered in Crossmark but commented on limited participation in Crossmark overall. Recording retraction information in this way is still important, alongside the Retraction Watch data, this allows for multiple assertions of that information, and increases confidence in its accuracy. We’re preparing to consult with the community at large about the future direction of the Crossmark service, to make it easier to implement and more useful for the readers.

Finally, Edilson Damasio from State University of Maringá-UEM, Brazil, and a long-time Crossref Ambassador, presented the analysis of Brazilian records in the Retraction Watch data, and he promises further analysis to come, comparing the situation across geographies.

Equity of participation in the research nexus

Amanda Bartell opened the research nexus discussion with a reminder of what that vision entails and pointing out commonality of goals in the community – “Like others, Crossref has a vision of a rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society. We call this interconnected network the Research Nexus, but others in the community have different names for it, such as knowledge graph or PID graph.”

The richness of this network depends upon the participation of all those who produce and publish scholarship, so naturally the topic of equality emerged in that discussion. In addition to Ran Dang’s concern for multilingualism and digitisation of past publications from all parts of the world, Mercury Shitindo of St Paul’s University, Kenya talked of the need for more education, training and accessible resources for her community, to be able to participate more effectively in this ecosystem. She can see that affiliations and citations are of priority there, as these enable transparency and facilitate collaborations. Matt Buys of DataCite echoed her point, talking about the importance of the role of contributors “It’s important not to lose sight of people and places – to recognise the importance of contributor roles in the PID-graph”.

Earlier in the day, we mentioned the launch of our Global Equitable Membership, or GEM programme. Since January, 110 new organisations from eligible countries have joined Crossref fee-free. Ginny was quick to admit that the need for a fee-waiver programme like this stems from the regular fees schedule not being in tune with our global membership, and she mentioned the upcoming fees review.

Financial barriers are often what get attention, yet reducing barriers to participation with technology is equally important for building a robust research nexus. With the planned changes to our registration form, we’ll make it easier to register works for those who don’t regularly use XML.

Johanssen Obanda took time to show the examples of community activity and events organised by our global network of Ambassadors, and to thank all our advocates and partners for their tireless work. They are also helping tackle barriers, supporting our members to actively participate in the research nexus with their metadata, and help enable the community to make good use of the network of relationships that data denotes.

Showcasing our “One member one vote” truth, the Board election was the focal point of the annual meeting, as always. We closed the ballot and announced the results, with seven members selected to join the Board in 2024.

The event went very smoothly overall. Talks were delivered efficiently, the panellists shared diverse perspectives and we elected our new Board members. Huge thanks to Rosa Clark, our Communications and Events Manager, who orchestrated the event and has been a constant behind-the-scenes presence supervising the entire show. I’m grateful to all colleagues at Crossref, who helped make it an enjoyable experience and an informative event for our community. Finally – it wouldn’t be a real meeting without the active participation of the speakers and panellists, who shared their metadata stories, and even joined us for some relaxed unplugged chats.

Perspectives: Luis Montilla on making science fiction concepts a reality in the scholarly ecosystem

Luis Montilla — Mon, 20 Nov 2023 00:00:00 +0000

Hello, readers! My name is Luis, and I’ve recently started a new role as the Technical Community Manager at Crossref, where I aim to bridge the gap between some of our services and our community awareness to enhance the Research Nexus. I’m excited to share my thoughts with you.

My journey from research to science communications infrastructure has been a gradual transition. As a Masters student in Biological Sciences, I often felt curious about the behind-the-scenes after a paper is submitted and published. For example, the fate of data being stored in the drawer or copied and forgotten in the hard drive after the paper is online. I come from a university that shares its name with at least three completely different universities in Latin America, and that also is pretty similar to another one with multiple offices across the region, which made me wonder if there was a standard way of identifying our affiliations. And then we have the topic of our names in hispanoamerica. We use two family names, and more often than not, we have a middle name (and then I could tell you stories about multiple-word middle names), which inevitably leads to authors having many combinations of full names and hyphenations.

This curiosity led me to volunteer in the Journal of the Venezuelan Society of Ecology. This role has been a transformative experience because my goal was to learn more about the publishing aspect of science. Still, today I realize that this is a fraction of what the scholarly ecosystem represents. The experience allowed me to grasp the importance of having a community with a sense of belonging, the relevance of multilingualism, and the importance of having access to an open infrastructure that allows smaller communities to be participants in the global dynamics. Moreover, it seemed to me that a research paper is more than the capstone of a building that we place and then move on to the next project or the next experiment; instead, it is a node in the vast network of human knowledge, connected to other papers through references, but also to all the other elements that are produced as part of the research, namely datasets, protocols, code, presentations, posters, preprints, peer-review reports and more. In short, the research metadata extends the life of the research output and makes it visible to the rest of the community.

This brings us to my onboarding to the Crossref team. At Crossref, I became part of a team and a driving force whose idea of the Research Nexus ¹ aligns perfectly with my aspirations. And to explain myself better, I’ll draw an analogy using one of my favorite authors. In Isaac Asimov’s Second Foundation, a character shows to another a wall covered to the last millimeter with equations and writings. He describes his contribution to “The Plan” as follows: “…Every red mark you see on the wall is the contribution of a man among us who lived since Seldon”.² This idea sounded fascinating to me and only possible in a sci-fi book; a massive integrated research ecosystem where scientists focused more on how their contributions fit in the big picture. Today I have come to think that metadata helps materialize this idea by interconnecting all knowledge, and more importantly, in stark contrast to Asimov’s plan developed and guarded by a secret society, Crossref’s research nexus is a “reusable open network,” “a scholarly record that the global community can build on forever.” In a world with undeniably unequal access to resources, providing open access and fostering community efforts to contribute to this growing collective effort is a fundamental condition to empower and visualize underrepresented voices.

We make available a series of tools to access and probe this data, including our REST API, but we know its potential is far from being realized. As Technical Community Manager at Crossref, my primary responsibility is to understand the needs of our community members who interact with our REST API. I aim to build and maintain relationships with new and existing metadata users to promote the effective usage of our API. I will also be working closely with organisations such as hosting platforms, manuscript submission systems, and general publisher services. In essence, I want to ensure that our community across the globe is aware of the vast possibilities that imply using and contributing to the Research Nexus.

I am committed to fostering an engaged and collaborative technical community. As we move forward, I look forward to sharing insights, experiences, and knowledge with all of you. Stay tuned for more updates, and let’s explore the world of APIs, metadata, and scholarly communities together!

Crossref (2021) The research nexus. Accessed on 20 October 2023. ↩︎
Asimov, I. (1953) Second Foundation. Gnome Press. ↩︎

Similarity check update: A new similarity report and AI writing detection tool soon to be available to iThenticate v2 users

Fabienne Michaud — Wed, 01 Nov 2023 00:00:00 +0000

In May, we updated you on the latest changes and improvements to the new version of iThenticate and let you know that a new similarity report and AI writing detection tool were on the horizon.

On Wednesday 1 November 2023, Turnitin (who produce iThenticate) will be releasing a brand new similarity report and a free preview to their AI writing detection tool in iThenticate v2. The AI writing detection tool will be enabled by default and account administrators will be able to switch it off/on.

Turnitin will be running a webinar on their new similarity report and AI writing detection tool on ~~Tuesday 28 November~~ (EDIT 23/11/16: Monday 11 December 2023). More information on the webinar and how to register will be communicated by Turnitin in the coming weeks.

New similarity report

On Wednesday, all iThenticate v2 users will have access to the new version of the similarity report which will include:

a word count and the number of text blocks for each matched source
the ability to include or exclude overlapping sources from the overall similarity score
a clearer colour differentiation between the different sources
improved accessibility features

Enabling the new similarity report

The new similarity report will be enabled as a default for all your journals. Account administrators wishing to switch off the new similarity report can do so by going to Settings and selecting from the General tab, under the New Similarity Report Experience heading, the Disable option.

Classic view / new view

As this will be a significant change to your current experience, Turnitin have provided access for a period of time to the ‘classic view’ and you will be able to toggle between the original interface and the new one by clicking on ‘Switch to the classic view’ or ‘Switch to the new view’ buttons at the top of your report.

The similarity score will continue to be available at the top right-hand corner of the similarity report.

Exclusions

By clicking on the Filters button you’ll be able to check and/or adjust your report’s section and repository exclusions.

Please note that the exclusions previously set up by account administrators should be unchanged by this release.

Sources / Match Groups view

The Sources view will be the default view and will list all sources. By using the on/off button next to ‘Show overlapping sources’, you’ll be able to include or exclude overlapping sources. This will be ‘off’ as a default.

The Match Groups view is completely new and may not suit everyone’s needs. It is divided into four categories ‘Not Cited or Quoted’, ‘Missing Quotations’, ‘Missing Citation’ and ‘Cited and Quoted’ and will highlight matches found in your text.

PDF report

You’ll also now find the PDF report in the top right-hand corner of the similarity report, by clicking on the ‘download’ icon.

Submission details

‘Submission Details’ is located now under the ‘i’ icon in the top right-hand corner of your report. This is where you will find the oid (or unique number) for your manuscript which Turnitin will ask you to provide when you are reporting a technical issue.

Turnitin’s documentation for the new similarity report

AI writing detection tool

Many of you have been concerned about the use of AI writing in the research papers you’ve received since the launch of ChatGPT last November and have been in touch to enquire about the availability of an AI writing detection tool for Crossref members.

You will also have read that Turnitin have developed an AI writing detector tool and have made it available to their education sector customers since April. Turnitin have published an update in May, a helpful video and further information on the false positive rates in June based on the feedback they’ve received from the education community.

I am pleased to announce that Turnitin’s AI writing detection tool will be available as a free preview to iThenticate v2 users, via the new version of the similarity report, from Wednesday 1 November until the end of December 2023.

Enabling AI writing detection

Our preference was to have the new AI writing detection tool turned ‘off’ as a default, however this hasn’t been possible. Account administrators can turn this feature off by going to Settings and selecting the Crossref Web tab and scrolling down to the AI Writing section at the very bottom of the page. The feature is applied to all submissions when it is enabled.

Please note that AI Writing detection is only available in the new similarity report.

Integrations

There is currently no integration between manuscript tracking systems and the AI writing detection tool. However the AI score will be available via the similarity report. If the AI writing detection tool has been set as ‘off’ by the account administrator, there will be no score and the ‘AI Writing’ heading will not be visible on the similarity report:

File requirements

Turnitin have made some important file requirements available for the tool to run a report:

Must be written in English
A minimum 300 words
A maximum of 15,000 words
The file size must be less than 100 MB
Accepted file types are .docx, .pdf, .rtf and .txt

If your file does not meet the above requirements, iThenticate v2 will display the following message:

Turnitin’s AI writing detection tool has been developed to detect GPT 3, 3.5, 4 and other variants. More information on this is available on their FAQs page.

Turnitin have provided the following guidance regarding the AI scores:

“Blue with a percentage between 0 and 100: The submission has processed successfully. The displayed percentage indicates the amount of qualifying text within the submission that Turnitin’s AI writing detection model determines was generated by AI. As noted previously, this percentage is not necessarily the percentage of the entire submission. If text within the submission was not considered long-form prose text, it will not be included.

Our testing has found that there is a higher incidence of false positives when the percentage is between 1 and 20. In order to reduce the likelihood of misinterpretation, the AI indicator will display an asterisk (*) for percentages between 1 and 20 to call attention to the fact that the score is less reliable.

To explore the results of the AI writing detection capabilities, select the indicator to open the AI writing report. The AI writing report opens in a new tab of the window used to launch the Similarity Report. If you have a pop-up blocker installed, ensure it allows Turnitin pop-ups.”

Please note that unlike the similarity report, the AI writing report will only provide a score and highlight the blocks of texts likely to have been written by an AI tool and will not list source matches.

We encourage you to test the writing detection tool as much as possible during the free preview period (1 November-31 December 2023).

Paraphrase detection

~~Turnitin are planning to release a beta version of their new paraphrase detection tool at the end of this year/Q1, 2024. It will be initially available as a free preview for a short period of time.~~ (EDIT 23/11/16: There is currently no timeline available for Turnitin’s paraphrase detection tool which is having a knock-on effect on the availiblity of the AI writing and paraphrase detection bundle and associated fees previously mentioned in this post)

AI and paraphrase detection bundle (EDIT 23/11/16: AI writing detection tool)

Once the free preview period ends, Turnitin ~~would like to offer Crossref members an AI and paraphrase detection bundle~~ (EDIT 23/11/16: are planning to make their AI writing detection tool available) from 2024 - this means that if you choose to subscribe to this new service, you will be charged an additional fee each time you upload a manuscript.

Fixes

Many of you have been waiting for fixes to the aggregation of URLs issues in the matched sources of the similarity report and to the doc-to-doc PDF report in iThenticate v2. Turnitin are planning to release fixes for these before the end of 2023.

✏️ Do get in touch via support@crossref.org if you have any questions about iThenticate v1 or v2 or start a discussion by commenting on this post below or in our Community Forum.

Perspectives: Audrey Kenni-Nemaleu on scholarly communications in Cameroon

Audrey Kenni-Nemaleu — Thu, 05 Oct 2023 00:00:00 +0000

Our Perspectives blog series highlights different members of our diverse, global community at Crossref. We learn more about their lives and how they came to know and work with us, and we hear insights about the scholarly research landscape in their country, the challenges they face, and their plans for the future.

Notre série de blogs Perspectives met en lumière différents membres de la communauté internationale de Crossref. Nous en apprenons davantage sur leur vie et sur la manière dont ils ont appris à nous connaître et à travailler avec nous, et nous entendons parler du paysage de la recherche universitaire dans leur pays, des défis auxquels ils sont confrontés et de leurs projets pour l’avenir.

Today, we meet Audrey Kenni-Nemaleu, Crossref Ambassador in Cameroon and Assistant Editor of the Pan-African Medical Journal (PAMJ). Audrey is excited about engaging Crossref’s community in French West Africa. Please take a moment to read and listen to Audrey’s perspective.

Aujourd’hui, nous rencontrons Audrey Kenni-Nemaleu, ambassadrice Crossref au Cameroun et rédactrice adjointe du Pan-African Medical Journal (PAMJ). Audrey est enthousiaste à l’idée d’impliquer la communauté Crossref en Afrique occidentale française. Veuillez prendre un moment pour lire et écouter le point de vue d’Audrey.

English

Français

Tell us a bit about your organisation, your objectives, and your role
Pouvez-vous nous parler de votre organisation, vos objectifs et votre rôle ?

My name is Audrey Kenni Nganmeni-Nemaleu, assistant editor for the Pan-African Medical Journal. I am specifically responsible for editing the articles in terms of form, ensuring that they meet the journal’s standards. Furthermore, I am the focal point of my journal for Crossref, that is to say I am responsible for managing all the problems that all publishers may encounter with DOIs and the various Crossref services to which our journal has subscribed. My role is also to manage all the conflicts that we may encounter with the DOIs submitted to Crossref. I train our journal staff in using Crossref services. I am also the focal point of my journal for COPE (Committee of Publications Ethics) which is an organisation that helps to regulate ethical publishing practices. It is in this capacity that I participate COPE’s webinars on behalf of our journal.

Je m’appelle Audrey Kenni Nganmeni Nemaleu, éditrice assistante pour le Pan African Medical Journal. Je m’occupe précisément de traiter les articles sur le plan de la forme en m’assurant qu’ils respectent les normes du journal. Par ailleurs je suis point focal de mon journal pour Crossref c’est-à-dire je suis chargée de gérer tous les problèmes que l’ensemble des éditeurs peuvent rencontrer avec les DOIs et les différents services de Crossref auxquels notre journal a souscrit. Mon rôle également c’est de gérer tous les conflits qu’on peut rencontrer avec les DOIs soumis à Crossref. Je forme également le personnel de notre journal à l’utilisation des services de Crossref. Je suis aussi point focal de mon journal pour COPE (Committee of Publications ethics) qui est un organisme qui aide dans la régulation des pratiques éthiques en matière de publication. C’est dans ce cadre que je participe à tous les webinaires de cette organisation afin qu’il y ait toujours au moins une personne qui participe à ces webinaires pour le compte de notre journal.

What is one thing that others should know about your country and its research activity?
Que doivent savoir les autres sur les activités de recherche dans votre pays ?

In my country, Cameroon, the research activity is still young. There are few scientific journals and we are actually the most influential journal in our country and subregion. There are also few schools or institutions that focus especially on research. For the time being, research activities in my country mainly revolve around congresses and conferences where researchers can exhibit their works. There is very little support for scientific research in my country.

Dans mon pays, le Cameroun, la recherche scientifique est encore jeune. Il existe peu de revues scientifiques et nous sommes en fait le journal le plus influent de notre pays et de notre sous-région. Il existe également peu d’écoles ou d’nstitutions qui spécialisées sur la recherche. Pour l’instant, les activités de recherche dans mon pays s’articulent principalement autour de congrès et de conférences où les chercheurs peuvent exposer leurs travaux. Il y a très peu de soutien à la recherche scientifique dans mon pays.

Are there trends in scholarly communications that are unique to your part of the world?
Existe-t-il des tendances particulières en matière de recherche scientifique dans votre région ?

In this part of the world, we do our best to follow the code of ethics of the various organisations in which we are a member: Committee of publication ethics (COPE), World Association of Medical Editors (WAME), Open Access Scholarly Publishing Association (OASPA). What we have seen emerging recently is the organisation, by professional scientific societies, of small conferences, workshops and meetings to exchange information. These small events are less costly to organize, hence their gain in popularity. We support these activities through sponsorship, and use them as opportunities to strengthen young researchers’ capacities in areas such as scientific writing, publication ethics. We also use those opportunities to introduce to young researchers concepts such as Open Access, Open Science, DOIs and other modern publishing services.

Dans notre pays, nous nous efforçons de suivre le code de déontologie des différentes organisations dont nous sommes membre : Committee of publication ethics (COPE), World Association of Medical Editors (WAME), Open Access Scholarly Publishing Association (OASPA). Ce que l’on a vu émerger récemment, c’est l’organisation, par des sociétés scientifiques professionnelles, de petits colloques, ateliers et réunions d’échange d’informations. Ces petits événements sont moins coûteux à organiser, d’où leur gain en popularité. Nous soutenons ces activités par le sponsoring et les utilisons comme des opportunités pour renforcer les capacités des jeunes chercheurs dans des domaines tels que l’écriture scientifique, l’éthique de la publication. Nous utilisons également ces opportunités pour leur présenter des concepts tels que le libre accès, la science ouverte, les DOIs et d’autres services d’édition modernes.

What about any political policies, challenges, or mandates that you have to consider in your work?
Quels sont les politiques, défis ou mandats auxquels vous faites face dans votre travail ?

Operating a journal in our context is challenging. The critical challenges are as basic as constant availability of electricity or stable and fast internet connectivity. How to maintain a stable stream revenue to support the journal is also a critical challenge. Most of our authors are young, self-funded and with limited resources. Most cannot afford the amount we charge for article publishing fees, which in comparison, is very limited. So we have to be extremely creative to operate.

Faire fonctionner une revue dans notre contexte est difficile. Les défis critiques sont aussi fondamentaux que la disponibilité constante de l’électricité ou une connexion Internet stable et rapide. Comment maintenir un flux stable des revenus pour soutenir la revue constitue également un défi crucial. La plupart de nos auteurs sont jeunes, autofinancés, avec des ressources limitées et par conséquent n’arrivent pas à payer les frais de publication d’articles pourtant très bas. Nous devons donc être extrêmement créatifs pour gérer nos charges.

How would you describe the value of being part of the Crossref community; what impact has your participation had on your goals?
Comment décririez-vous la valeur de faire partie de la communauté Crossref ? Quel est l’impact de votre participation sur vos objectifs ?

As a Crossref ambassador, I talk about Crossref around me, among my colleagues whether they are in Kenya or Cameroon. I shared the links to participate in Crossref webinars with my colleagues. I invited them to become ambassadors by sharing with them the links to join the community. I participated in several ambassador training webinars on different themes including: how to submit DOI to Crossref, ORCID. I participated in a Crossref event in Nairobi, Kenya. It was a memorable moment where I was able to meet other ambassadors. We were able to have a small meeting on the difficulties we encountered in growing the Crossref community in Africa. We produced a document to this effect which we submitted to Crossref in 2022. For the moment, I have not yet been able to organize an event as an ambassador, but I would like to with the help of Crossref. But being an ambassador is not the easiest thing because sometimes in our context people do not understand the use of Crossref’s services because we are in an environment where the DOI is not yet very well known, and where even publishers know nothing about this. A question I am often asked is whether this work is paid and are discouraged when they learn that it is voluntary work.

Comme ambassadrice de Crossref, je parle autour de moi de Crossref, parmi mes collègues qu’ils soient au Kenya ou au Cameroun. J’ai partagé les liens pour participer à des webinaires de Crossref à mes collègues. Je les ai invités à devenir des ambassadeurs en partageant avec eux les liens pour rejoindre la communauté. J’ai participé à plusieurs webinaires de formation des ambassadeurs sur différents thèmes notamment ORCID. J’ai également participe à un évènement de Crossref à Nairobi au Kenya. C’était un moment mémorable ou j’ai pu rencontrer d’autres ambassadeurs. Nous avons pu faire une petite réunion sur les difficultés que nous rencontrons pour faire grandir la communauté Crossref en Afrique. Nous avons d’ailleurs produit un document à cet effet que nous avons soumis à Crossref en 2022. Pour l’instant, je n’ai pas encore pu organiser d’évènement dans le cadre d’ambassadeur, mais j’aimerais avec l’aide de Crossref voir comment le faire. Etre ambassadrice n’est pas la chose la plus facile car parfois dans notre contexte les gens ne comprennent pas le bien-fondé des services de Crossref car on est dans un environnement ou le DOI n’est pas encore très connu, et où beaucoup de journaux et même d’editeurs ne savent rien de cela. Une question qu’on me pose souvent est celle savoir si ce travail est remunere et se découragent quand ils apprennent que c’est du bénévolat.

For you, what would be the most important thing Crossref could change (do more of/do better in)?
Pour vous, quelle serait la chose la plus importante que Crossref pourrait changer (faire plus/faire mieux) ?

Crossref could invest in more capacity building, events, and communications in this part of the world. Why not localize Crossref in the francophone part of Africa? Crossref could offer continuing educational activities to professionals in order to improve their skills or acquire new knowledge in metadata and correlative disciplines. Crossref could also sponsor/support journal publishing and scholarship in Africa.

Crossref pourrait investir dans davantage de renforcement des capacités, d’événements et de communications dans cette partie du monde. Pourquoi ne pas localiser Crossref dans la partie francophone de l’Afrique ? Crossref pourrait proposer des activités de formation continue aux professionnels afin d’améliorer leurs compétences ou d’acquérir de nouvelles connaissances dans les métadonnées et les disciplines corrélatives. Crossref pourrait également sponsoriser/soutenir la publication de revues et les bourses d’études en Afrique.

Which other organisations do you collaborate with or are pivotal to your work in open science?
Avec quelles autres organisations collaborez-vous ou alors quelles sont les organismes pivot au cœur de votre travail en science ouverte ?

I collaborate with various institutions such as COPE (Committee on Publication Ethics), AJOL African Journals Online, and OASPA (Open Access Scholarly Publishing Association). I attend webinars of these organisations on behalf of my journal.

Je collabore avec diverses institutions telles que COPE (Committee on Publication Ethics), AJOL African Journals Online, et OASPA (Open Access Scholarly Publishing Association). J’assiste à des webinaires de ceux-ci organisations au nom de ma revue.

What are your plans for the future?
Quels sont vos plans pour l’avenir ?

My plan for the future is to continue working in science communication with different other organisations, and more within my community.

Mon plan pour l’avenir est de continuer à travailler dans le domaine de la communication scientifique avec différentes autres organisations, et davantage au sein de ma communauté.

Thank you, Audrey!

Merci, Audrey !

Feedback on automatic digital preservation and self-healing DOIs

Martin Eve — Thu, 28 Sep 2023 00:00:00 +0000

Thank you to everyone who responded with feedback on the Op Cit proposal. This post clarifies, defends, and amends the original proposal in light of the responses that have been sent. We have endeavoured to respond to every point that was raised, either here or in the document comments themselves.

We strongly prefer for this to be developed in collaboration with CLOCKSS, LOCKSS, and/or Portico, i.e. through established preservation services that already have existing arrangements in place, are properly funded, and understand the problem space. There is low level of trust in the Internet Archive, also given a number of ongoing court cases and erratic behavior in the past. People are questioning the sustainability and stability of IA, and given it is not funded by publishers or other major STM stakeholders there is low confidence in IA setting their priorities in a way that is aligned with that of the publishing industry.

We acknowledge that some of our members have a low level of trust in The Internet Archive, but many of our (primarily open access members) work very closely with the IA and our research has shown that, without the IA, the majority our smaller open access members would have almost no preservation at all. We have already had conversations with CLOCKSS and Portico about involvement in the pilot and thinking through what a scale-to-production would look like. That said, for a proof-of-concept, the Internet Archive presents a very easy way to get off the ground, with a stable system that has been running for almost 30 years.

This seems to be a service for OA content only, but people wonder for how long. Someone already spotted an internal CrossRef comment on the working doc that suggested “why not just make it default for everything & everyone”, and that raises concern.

The primary audience for this service is small OA publishers that are, at present, poorly preserved. These publishers present a problem for the whole scholarly environment because linking to their works can prove non-persistent if preservation is not well handled. Enhancing preservation for this sector therefore benefits the entire publishing industry by creating a persistent linking environment. We have no plans to make this the “default for everything and everyone” because the licensing challenges alone are massive, but also because it isn’t necessary. Large publishers like Elsevier are doing a good job of digitally preserving their content. We want this service to target the areas that are currently weaker.

Crossref will always respect the content rights of our members. We never force our members to release their content through Crossref that they don’t ask us to release.

The purpose of the Op Cit project is to make it easier for our members to fulfil commitments they already made when they joined Crossref.

Crossref is fundamentally an infrastructure for preserving citations and links in the scholarly record. We cannot do that if the content being cited or linked to disappears.

When signing the Crossref membership agreement, members agree to employ their best efforts to preserve their content with archiving services so that Crossref can continue to link citations to it even in extremis. For example- if they have ceased operations.

Some of our members already do this well. They have already made arrangements with the major archiving providers. They do not need the Op Cit service to help them with archiving. However, the Op Cit service will still help them ensure that the DOIs that they cite continue work. So it will still benefit them even if they don’t use it directly.

However, our research shows that many of our members are not fulfilling the commitments they made when joining Crossref. Over the next few years, we will be trying to fix this. Primarily through outreach- encouraging members to set up and record with Crossref archiving arrangements with the archives of their choice.

But we know some members will find this too technically challenging and/or costly. [And frankly, given what we’ve learned of the archiving landscape, we can see their point.] The proposed Op Cit service is for these members. The vast majority of these members are Open Access publishers, so the “rights” questions are far more straightforward- making the implementation of such a service much more tractable.

Someone asked what this means for the publisher-specific DOI prefix for this content? Will this be lost?

No.

There is concern about the interstitial page that Crossref would build that gives the user access options. The value of Crossref to publishers is adding services that are invisible and beneficial to users, not adding a visible step that requires user action.

There is nothing in Crossref’s terms that says that we have to be invisible. The basic truth is that detecting content drift is really hard and several efforts to do so before have failed. Without a reliable way of knowing whether we should display the interstitial page, which may become possible in future, we have to display something for now, or the preservation function will not work.

Crossref has, also, supported user-facing interstitial services for over a decade, including:

Multiple Resolution
Coaccess
CrossMark
Crossref Metdata Search
REST API

So we have a long track record of non-B2B service provision.

There is confusion about why Crossref seems to want to build the capacity to “lock” records in absence of flexibility. People feel no need for Crossref to get involved here.

This is a misunderstanding of the terminology. The Internet Archive allows the domain owner to request content to be removed. This would mean that, in future, if a new domain owner wanted, they could remove previously preserved material from the archive, thereby breaking the preservation function. When we say we want to “lock” a record, we mean that a future domain owner cannot remove content from the preservation archive. This also prevents domain hijackers from compromising the digital preservation.

There is concern about the possibility to hack this system to give uncontrolled access to all full-text content by attacking publishing systems and making them unavailable. This is an unhappy path scenario but something on people’s minds.

The system only works on content that is provided with an explicitly stated open license (see response above).

I think this project would be improved by better addressing the people doing the preservation maintenance work that this requires. Digital preservation is primarily a labor problem, as the technical challenges are usually easier than the challenge of consistently paying people to keep everything maintained over time. Through that lens, this is primarily a technical solution to offload labor resources from small repositories to (for now) the Internet Archive, where you can get benefits from the economies of scale. There are definitely cases where that could be useful! But I think making this more explicit will further a shared understanding of advantages and disadvantages and help you all see future roadblocks and opportunities for this approach.

This consultation phase was designed, precisely, to ensure that those working in the space could have their say. While this is a technical project, we recognize that any solution must value and understand labor. That means that any scaling to production must and will also include a funding solution to address the social labor challenge.

Is there any sense in polling either the IA Wayback Machine or the LANL Memento Aggregator first to determine if snapshot(s) already exist?

We could do this, but it would add an additional hop/lookup on deposit. Plus, we want to store the specific version deposited at the specific time it is done, including re-deposits.

I would encourage looking at a distributed file system like IPFS (https://en.wikipedia.org/wiki/InterPlanetary_File_System). This would allow easy duplication, switching and peering of preservation providers. Correctly leveraged with IPNS; resolution, version tracking and version immutability also become benefits. Later after beta the IPNS metadata could be included as DOI metadata.

We had considered IPFS for other projects, but really, for this, we want to go with recognised archives, not end up running our own infrastructure for preservation.

It might be useful to look into the 10320/loc option for the Handle server: the https://www.handle.net/overviews/handle_type_10320_loc.html. I can imagine a use case where a machine agent might want to access an archive directly without needing to go to an interstitial page.

It is good to see reference to the HANDLE system and alternative ways that we might use it. We will consult internally on the technical viability of this.

In general, though, we prefer to use web-native mechanisms when they are available. We already support direct machine access via HTTP redirects and by exposing resource URLs in the metadata that can be retreivd via content negotiation. In this case, we would be looking at supporting the 300 (multiple choice) semantics.

I’m curious to see how this will work for DOI versioning mechanisms like in Zenodo, where you have one DOI to reference all versions as well as version specific DOIs. If your record contains metadata + many files and a new version just versions one of the several files my assumption is that within the proposed system an entire new set (so all files) is archived. In theory this could also be a logical package, where simply the delta is stored, but I guess in a distributed preservation framework like the one proposed here, this would be hard to achieve.

This is a good point and it could lead to many more, frustrating, hops before the user reaches the content. We will conduct further research into this scenario, but we also note that Zenodo’s DOIs do not come from Crossref, but from DataCite.

There’s a decent body of research at this point on automated content drift detection. This recent paper: https://ceur-ws.org/Vol-3246/10_Paper3.pdf likely has links to other relevant articles.

We have no illusions about the difficulty of detecting semantic drift but this is helpful and interesting. We will read this material and related articles to appraise the current state of content drift detection.

Out of curiosity, will we be using one type of archive (i.e., IA or CLOCKSS or LOCKSS or whatever) or will it possibly be a combination of a few archives? Reading the comments, it looks like some of them charge a fee, so I see why we’d use open source solutions first. Also, eventually could it be something that the member chooses? i.e. which archive they might want to use. Again, the latter question isn’t something for the prototype, but I’m curious about this use case. Also, I wonder about the implementation details if it is more than one archive. The question is totally moot of course, if we’re sticking with one archive for now.

The design will allow for deposit in multiple archives – and we will have to design a sustainability model that will cover those archives that need funding. As above, this is an important part of the move to production.

Will be good for future interoperability to make sure at least one of the hashes is a SoftWare Hash IDentifier (see swhid.org). The ID is not really software specific and will interoperate with the Software Heritage Archive and git repositories.

We will certainly ensure best practices for checksums.

Comments on the Interstitial Page

I’d keep the interstitial page without planning its eradication. (See why in the last paragraph) I’d even advocate for it to be a beautiful and useful reminder to users that “This content is preserved”. I’d go further and recommend that publishers deposit alternate urls of other preservation agents like PMC etc, that would also be displayed. This page could even be merged with multi-resolution system.

The why: I’m concerned of hackers and of predatory publishers exploiting the spider heuristics by highjacking small journals and keeping just enough metadata as in them as to fool the resolver and then adding links to whatever products, scams and whatnots…

Technical. Scraping landing pages is hard. We’ve had a lot of projects to do this over the years. You can mitigate the risk by tiering / heuristics. Maybe even feedback loop to publishers to encourage them to put the right metadata on the landing page.

This is the only part of this proposal that I don’t like. People are used to DOIs resolving directly to content, and I don’t think that should be changed unless absolutely necessary. I would prefer that the DOI resolves to the publisher’s copy if it exists, and the IA copy otherwise.

We will continue the discussion about the interstitial page. The basic technical fact, as above, is that detecting content drift is hard and so we may need, at least, to start with the page. However, some commentators presented reasons for keeping it.

We also have already supported interstitial pages for multiple resolution and co-access for over a decade.

It is member’s choice whether they wish to deposit alternative URLs and we already have a mechanism for this.

2023 board election slate

Lucy Ofiesh — Wed, 27 Sep 2023 00:00:00 +0000

I’m pleased to share the 2023 board election slate. Crossref’s Nominating Committee received 87 submissions from members worldwide to fill seven open board seats.

We maintain a balance of eight large member seats and eight small member seats. A member’s size is determined based on the membership fee tier they pay. We look at how our total revenue is generated across the membership tiers and split it down the middle. Like last year, about half of our revenue came from members in the tiers $0 - $1,650, and the other half came from members in tiers $3,900 - $50,000. We have two large member seats and five small member seats open for election in 2023.

The Nominating Committee presents the following slate.

The 2023 slate

Tier 1 candidates (electing five seats):

Beilstein-Institut, Wendy Patterson
Korean Council of Science Editors, Kihong Kim
Lujosh Ventures Limited, Olu Joshua
NISC Ltd, Mike Schramm
OpenEdition, Marin Dacos
Universidad Autónoma de Chile, Dr. Ivan Suazo
Vilnius University, Vincas Grigas

Tier 2 candidates (electing two seats):

Association for Computing Machinery (ACM), Scott Delman
Oxford University Press, James Phillpotts
Public Library of Science (PLOS), Dan Shanahan
University of Chicago Press, Ashley Towne

Here are the candidates’ organisational and personal statements

You can be part of this important process by voting in the election

If your organisation is a voting member in good standing of Crossref as of September 10th, 2023, you are eligible to vote when voting opens on September 27th, 2023.

How can you vote?

Your organisation’s designated voting contact will receive an email from eBallot the week of September 25th with the Formal Notice of Meeting and Proxy Form with concise instructions on how to vote. The email will include a username and password with a link to our voting platform.

The election results will be announced at the LIVE23 online meeting on October 31st, 2023. Save the date! Incoming members will take their seats at the March 2024 board meeting.

News: Crossref and Retraction Watch

Ginny Hendricks — Tue, 12 Sep 2023 00:00:00 +0000

Crossref acquires Retraction Watch data and opens it for the scientific community

Agreement to combine and publicly distribute data about tens of thousands of retracted research papers, and grow the service together

12th September 2023 —– The Center for Scientific Integrity, the organisation behind the Retraction Watch blog and database, and Crossref, the global infrastructure underpinning research communications, both not-for-profits, announced today that the Retraction Watch database has been acquired by Crossref and made a public resource. An agreement between the two organisations will allow Retraction Watch to keep the data populated on an ongoing basis and always open, alongside publishers registering their retraction notices directly with Crossref.

Both organisations have a shared mission to make it easier to assess the trustworthiness of scholarly outputs. Retractions are an important part of science and scholarship regulating themselves and are a sign that academic publishing is doing its job. But there are more journals and papers than ever, so identifying and tracking retracted papers has become much harder for publishers and readers. That, in turn, makes it difficult for readers and authors to know whether they are reading or citing work that has been retracted. Combining efforts to create the largest single open-source database of retractions reduces duplication, making it more efficient, transparent, and accessible for all.

Product Director Rachael Lammey says, “Crossref is focused on documenting and clarifying the scholarly record in an open and scalable form. For a decade, our members have been recording corrections and retractions through our infrastructure, and incorporating the Crossmark button to alert readers. Collaborating with Retraction Watch augments publisher efforts by filling in critical gaps in our coverage, helps the downstream services that rely on high-quality, open data about retractions, and ultimately directly benefits the research community.”

The Center for Scientific Integrity and the Retraction Watch blog will remain separate from Crossref and will continue their journalistic work investigating retractions and related issues; the agreement with Crossref is confined to the database only and Crossref itself remains a neutral facilitator in efforts to assess the quality of scientific works. Both organisations consider publishers to be the primary stewards of the scholarly record and they are encouraged to continue to add retractions to their Crossref metadata as a priority.

“Retraction Watch has always worked to make our highly comprehensive and accurate retraction data available to as many people as possible. We are deeply grateful to the foundations, individuals, and members of the publishing services industry who have supported our efforts and laid the groundwork for this development,” said Ivan Oransky, executive director of the Center for Scientific Integrity and co-founder of Retraction Watch. “This agreement means that the Retraction Watch Database has sustainable funding to allow its work to continue and improve.”

Please join Crossref and Retraction Watch leadership, among other special guests, for a community call on 27th September at 1 p.m. UTC to discuss this new development in the pursuit of research integrity.

Supporting details

Crossref retractions number 14k, and the Retraction Watch database currently numbers 43k. There is some overlap, making a total of around 50k retractions.
The full dataset has been released through Crossref’s Labs API, initially as a .csv file to download directly: https://api-labs-crossref-org.pluma.sjfc.edu/data/retractionwatch?name@email.org (add your ‘mailto’). Edit: 2024-10-10: The full dataset is available in a git repository at https://gitlab.com/crossref/retraction-watch-data.
The Crossref Labs API also displays information about retractions in the /works/ route when metadata is available, such as https://api-labs-crossref-org.pluma.sjfc.edu/works/10.2147/CMAR.S324920?name@email.org (add your ‘mailto’). If you don’t have a .json viewer, please see below for screenshot.
Crossref is paying an initial acquisition fee of USD $175,000 and will pay Retraction Watch USD $120,000 each year, increasing by 5% each year.
The initial term of the contract is five years. ~~The full text of the contract will be made public in the coming fortnight.~~ EDIT 2023-09-26: Here is the signed agreement.
There will be a community call on 27th September at 1 p.m. UTC (your time zone here). Please register.
An open FAQ document is available to collect questions to be answered at the webinar.
This announcement will always be accessible via Crossref DOI https://doi-org.pluma.sjfc.edu/10.13003/c23rw1d9; please use this persistent link for sharing.

About Retraction Watch and The Center for Scientific Integrity

The Center for Scientific Integrity is a U.S. 501(c)3 non-profit whose mission is to promote transparency and integrity in science and scientific publishing, and to disseminate best practices and increase efficiency in science. In addition to maintaining and curating the Retraction Watch Database, the Center is the home of Retraction Watch, a blog founded in 2010 that reports on scholarly retractions and related issues in research integrity.

About Crossref

Crossref is a global community infrastructure that makes all kinds of research objects easy to find, assess, and reuse through a number of services critical to research communications, including an open metadata API that sees over 2 billion queries every month. Crossref’s >19,000 members come from 151 countries and are predominantly university-based. Their ~150 million DOI records contribute to the collective vision of a rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society.

Enquiries

For Retraction Watch/Center for Scientific Integrity: Ivan Oransky, ivan@retractionwatch.com
For Crossref: Ginny Hendricks, ginny@crossref.org

A screenshot of an example Labs API metadata record with a Retraction Watch-asserted retraction

Open Funder Registry to transition into Research Organization Registry (ROR)

Amanda French — Thu, 07 Sep 2023 00:00:00 +0000

There is some overlap between the Open Funder Registry and the Research Organization Registry (ROR), and funders and publishers have been asking us whether they should use Open Funder Registry IDs or ROR IDs to identify funders when they appear in both registries. We aim to merge the two registries over time. We will ensure Crossref members can use ROR to simplify persistent identifier integrations, to register better metadata, and to help connect research outputs to research funders.

Just yesterday, we published a summary of a recent workshop between funders and publishers on funding metadata workflows that we convened with the Dutch Research Council (NWO) and Sesame Open Science. As the report notes, “open funding metadata is arguably the next big thing” [in Open Science]. That being the case, we think this is the ideal time to strengthen our support of open funding metadata by beginning the transition to ROR.

Comparing the features of ROR and the Open Funder Registry

Let’s look at some of the similarities and differences between the two registries, including their history, features, scope, and usage, since there are important nuances and distinctions that are helpful to understand.

Overview

ROR	Open Funder Registry
Launched in 2019	Launched in 2013
Primary use case is contributor affiliation	Primary use case is funding acknowledgement
105k+ records	35k+ records
CC0 data	CC0 data
REST API	REST API
Free to use	Free to use
Entire registry downloadable as JSON and CSV	Entire registry downloadable as RDF; funder names and IDs downloadable as CSV
Records contain mappings to other IDs	Records do not contain mappings to other IDs
few organisation relationships and hierarchy	multiple organisation relationships and hierarchy
organisation level with no funding programs/schemes	organisation level with some funding programs/schemes
8 organisation types	2 funder types, 8 funder subtypes
Open source code and multiple open-source tools available	Open source code
Web-based registry search	Web-based search for works in Crossref associated with each Open Funder Registry ID
Web-based landing pages for each ROR record	JSON landing pages for each Open Funder Registry record
Updated monthly	Updated bimonthly
Public curation process	Private curation process
Anyone can request changes and additions	Anyone can request changes and additions
Stable financial support (Crossref, DataCite, CDL)	Stable financial support (Crossref, Elsevier)
Beginning to be supported in funding and publishing workflows	Well supported in most funding and publishing workflows
Currently used by 260+ Crossref members ¹	Currently used by 2100+ Crossref members ²

History

The Open Funder Registry was launched as FundRef over a decade ago to enable the community to cite research funding and support and assert it within the scholarly record, acknowledging the organisations granting their support. Elsevier generously donated the seed data for the Open Funder Registry and has managed its curation for the last ten years, while we have maintained the technical operations and promoted community adoption of the Open Funder Registry.

The Research Organization Registry (ROR) was introduced in 2019 by the California Digital Library, DataCite, and Crossref to enable the community to cite contributor affiliations and assert them within the scholarly record, acknowledging the organisations that housed or performed the research. Digital Science generously donated the seed data for the Research Organization Registry from its Global Research Identifier Database (GRID) initiative, and Crossref, DataCite, and the California Digital Library have contributed labour and resources to turn ROR into a mature, independent, freely available service.

Scope

One key difference between the registries is that ROR has always included funding organisations, and ROR records have always included mappings to Funder IDs where available, while the reverse is not true: the Open Funder Registry includes only funding organisations, not other kinds of organisations, and Open Funder Registry records do not currently include mappings to ROR IDs or other identifiers. It therefore makes sense to expand our initial contributor affiliation use case for ROR to include the identification of organisations that fund and support research.

Usage

More Crossref members use Funder IDs than use ROR IDs, to be sure. You can see from the table above that the number of Crossref members using Funder IDs in Crossref records is higher by almost a factor of 10 than the number of Crossref members using ROR IDs in Crossref records. But note too that the current rate of adoption is far higher for ROR than it is for the Open Funder Registry. Since January of 2022, we’ve seen a gratifying number of publishers and service providers beginning to use ROR identifiers for contributor affiliations in Crossref. In the last year, the number of Crossref members depositing ROR IDs has increased by 356%, while the number depositing Funder IDs has increased only by 12%. As evidenced by its ballooning API traffic, too, with more than 20 million requests last month,³ ROR is clearly being used by many scholarly research systems for many purposes. The more systems that use an identifier, the more valuable that identifier becomes as a vehicle for exchanging information.

ROR’s primary use case is to identify contributor affiliations and is already being used by funders. Nineteen funding organisations are depositing ROR IDs in their grant records with Crossref to denote principal investigator affiliations,⁴ and, following a meeting of the our Funder Advisory Group last month, all eighty funder members are primed to start using ROR IDs to identify themselves in grant records.

Tools and services

Both the Open Funder Registry and ROR have open data and open source code, but we think that our suite of free and open source utilities for ROR gives it an advantage. We know that publishers and their service providers have ongoing challenges in collecting and matching funding information from authors and in validating Funder IDs. With our extensive ROR toolkit, publishers and their technology providers who adopt ROR will be in a better position to improve the accuracy of funding acknowledgements in metadata, which can in turn enable the development of reliable analytics, tools, and services for funders, regulators, research facilities, and the public.

Crossref has built tools based on OpenRefine for both the Open Funder Registry and ROR: the Open Funder Registry Reconciliation Service and the ROR Reconciler are both useful ways to clean messy data. ROR, however, also offers a much-used API endpoint that helps match organisation names to ROR IDs, and several third parties have also developed and shared open source matching tools and services for ROR. Crossref is also collaborating on new strategies for affiliation matching that will improve connections for funding acknowledgements.

Community engagement models

The Open Funder Registry has been curated for over a decade through time and expertise generously donated by Elsevier and is community-governed by Crossref and it’s membership and board. ROR offers more transparent community involvement and is jointly governed by Crossref, DataCite, and the California Digital Library. ROR is openly curated and is aided by a global Curation Advisory Board of volunteers.

What will this mean for you?

The many organisations whose tools, services, and workflows have been architected to use Open Funder Registry (OFR) IDs will find this transition a challenge, and we don’t want to make light of that issue. Over the last ten years, we have encouraged the community to adopt Funder IDs, and the community has demonstrably recognized the benefits of doing so. Publishers have put a great deal of time, thought, and effort into collecting funder data and including it in Crossref metadata, and they have built internal reports and workflows around the Open Funder Registry. Crossref is committed to making the transition from the Open Funder Registry to the Research Organization Registry as simple as possible for the community.

If you are not already using the Open Funder Registry and are planning to begin standardizing funding data, we recommend that you use ROR to identify funders. If you are currently using the Open Funder Registry in your systems and workflows, don’t worry! In the medium term, Open Funder Registry IDs aren’t going away. Eventually, however, the Open Funder Registry may cease to be updated. Funder IDs and their mapping to ROR IDs will be maintained, so if Crossref members submit a Funder ID, it will get mapped to a ROR ID automatically. Note, too, that Crossref is committed to maintaining the current funder API endpoints.

In short, if you are already using Funder IDs, you can and should continue to do so. However, we do recommend that you begin looking at what it will take to integrate ROR into your systems and workflows for identifying funders as well as affiliations.

We face challenges in this transition, too. Of these, we think the largest will be (1) completing the reconciliation work involved in mapping Funder IDs to ROR IDs, and (2) updating Crossref’s schemas, APIs, and deposit tools to support ROR IDs in many the ways we currently support Funder IDs. We’ll discuss both of these challenges in future blog posts.

Tell us what you need?

We want to hear from you. You can use our Community Forum talk to us about the Crossref Open Funder Registry or contact ROR staff at Crossref via our request form. You can attend online Crossref events, including ROR-specific webinars to get updates from us and ask us your questions.

One of the major messages we’re already hearing from funders and publishers is expressed in yesterday’s post on open funding metadata: “While many concluded that there was still a long way to go to solve the many technical challenges related to funding metadata, attendees were unanimous on its importance.” We look forward to beginning this important work together.

Open funding metadata through Crossref; a workshop to discuss challenges and improving workflows

Hans de Jonge — Wed, 06 Sep 2023 00:00:00 +0000

Ten years on from the launch of the Open Funder Registry (OFR, formerly FundRef), there is renewed interest in the potential of openly available funding metadata through Crossref. And with that: calls to improve the quality and completeness of that data. Currently, about 25% of Crossref records contain some kind of funding information. Over the years, this figure has grown steadily. A number of recent publications have shown, however, that there is considerable variation in the extent to which publishers deposit these data to Crossref. Technical but also business issues seem to lie at the root of this. Crossref - in close collaboration with the Dutch Research Council NWO and Sesame Open Science - brought together a group of 26 organisations from across the ecosystem to discuss the barriers and possible solutions. This blog presents some anonymized lessons learned.

There is no Open Science without open metadata

The interest in the potential of this open-source funding metadata seems to be entering a new stage. When registering (or updating) a DOI record for a publication, publishers can include information about the funding of the research. The Open Funder Registry grew out of recommendations in the report from the US Scholarly Publishing Roundtable in 2010. During the Annual Meeting of Crossref that year, Frederick Dylla, CEO of the American Institute of Physics, argued that in order to make research funding information in publications accessible, it needed to be presented in a standard way and stored in a central location.

The benefits of having open funding metadata available, listed by Dylla in his presentation 13 years ago, are still very valid:

Researchers benefit because it increases transparency of their funding sources and supports the requirements they already have from their funders.
For funders, having this data available is essential because it allows them to identify the published outcomes of publicly funded research. Essential to monitor compliance with open access policies, but also important given the pressures funders face to account for their spending of public money.
For publishers, funding metadata provides a valuable service, as it provides insight into how the research they publish is funded.

Although Crossref has been collating funding metadata for many years, there seems to be a renewed interest in this service. Publishers have long expressed a desire to solve the challenges, meta-researchers need this information in order to analyze research on research, editors are concerned with research integrity, including funding trends, and funders themselves need to track the reach and return of their support.

Open Science seems to be an important driver: As we move to an ecosystem built on Open Science principles, not only publications, data, and software need to be openly available, but also the metadata associated with those scholarly outputs. Indeed, in an Open Science world, all meta information should be open, and academia should not be dependent anymore on data from proprietary bibliographic databases. Indicators for research assessment and policy development should be open indicators, derived from open metadata. Much has been done in this area already, in the context of Open Citations and Open Abstracts. While many in the community have focused on the bigger picture of advocating for all open metadata, e.g. Metadata 20/20, open funding metadata is arguably the next big thing. Open Research Information, including open metadata, must be a strategic priority for science and society.

Room for improvement

After ten years of collecting funding metadata, 25% of records in Crossref contain some kind of funding information, and this figure was reached by a steady growth over that time. A number of recent studies have shown, however, that there is room for improvement. A case study published by two of the present authors has shown that the extent to which publishers deposit funding information to Crossref varies considerably. Some larger society presses - American Chemical Society (ACS), American Physical Society (APS), and Royal Society of Chemistry (RSC) - perform exceptionally well, with almost 100% of publications containing funding information. But there is still a large number of publishers - among them large legacy publishers - that attain substantially lower figures or do not seem to deposit funding metadata at all. Our case study has shown that often this cannot be explained by the fact that authors have not provided any funding information, as often this information is available in the acknowledgement sections of the papers. Somehow, however, this data does not find its way to Crossref.

Workflows and challenges: collect, retain, validate, deposit

In order to chart the challenges that publishers face when collecting this information, we organized a roundtable session. 26 organisations were invited from across the ecosystem. These included: major publishers (American Chemical Society, British Medical Journal, Elsevier, IOP Publishing, PLOS, Royal Society of Chemistry, Sage, Springer Nature, Taylor & Francis, and Wiley), funders (European Research Council, Austrian Research Council, Dutch Research Council, OSTI-DOE, UKRI, and Michael J Fox Foundation) as well as service providers (Aries Editorial Manager, PKP / OJS, Scholastica, and eJournal Press).

In order to map the potential barriers and challenges publishers face, participants were presented with a workflow scheme representing a hypothetical production process.

This workflow outlined the steps in the production process at which funder information would potentially be handled, as well as some of the considerations that might be at play at each step.

collecting funder information (upon submission or acceptance)
extracting funder information from full text
retaining funder information through the production workflow
including funder information in article metadata
making metadata and/or full text available for indexing

Participants were invited to comment on this workflow and place digital dots in the scheme to identify challenges in the collection, retention, and deposit of funding information. These pain points were afterwards fleshed out in break-out groups.

Lessons learned

1. Still a lack of awareness among editors and authors

For many journals and publishers, collecting funding information starts when papers are submitted through submission systems. Many publishers use the same systems: ScholarOne and Editorial Manager, though many have multiple systems in place for different portfolios of journals. Around 25,000 journals use PKP’s Open Journal System, and Scholastica and eJournal Press are growing in popularity and importance. All of them provide the possibility for authors to enter funder information but this does not by all means mean that all journals make use of it. Submission systems are highly customizable, and publishers tend to tailor systems to the needs and wishes of their journals. Editors who do not see much value in collecting funding metadata therefore present a first ‘weak link’. Publishers and tech providers agreed that more outreach is needed about the importance of funding metadata among editors and authors.

2. Improvements are needed in submission systems

Where journals and publishers agree on asking authors to register funding information through the submission systems, many express a tension between collecting structured metadata and making it as easy as possible for authors. Many are hesitant to use mandatory input fields. Instead, funding metadata is often collected as free text, giving rise to a plethora of ambiguities. Most systems provide suggestions based on the input of the author based on the Open Funder Registry. A lot seems to go wrong at this stage. Authors often persist in the wrong spelling of their funder and do not choose predefined suggestions, making it very difficult to match input to Funder IDs. Publishers estimated the number of non-matches up to 50%. Trivial issues like “Bill & Melinda” versus “Bill and Melinda” or “Netherlands organisation” versus “Netherlands Organisation” result in errors. Here, autocomplete techniques seem to be in dire need of improvement. Based on a preliminary analysis of funder name variants used in Crossref, adding up to 3 of the most frequently used name variants to the list of ‘alternative funder names’ in the Funder registry could solve around 60% of missed matches.

3. A lot can be learned from how some publishers have changed and organized their workflows

Faced with these issues, the Royal Society of Chemistry has invested in innovative workflows to enhance the availability of funding metadata. Instead of relying solely on the free text input of the authors, RSC presented to the group the details of how they have tackled the issue. In addition to author-provided acknowledgements, they work with third-party production vendors to programmatically extract information from the acknowledgement section of papers. Data from the two sources are compared, and when differences or conflicts are being noted, the data is fixed, completed, and reformatted. The next step is crucial - the newly-cleansed funding data is fed back to the author for validation, and retained during the production phase of the paper. Implementation of this validation stage has increased the availability of funding metadata by 30%. In 2023 80% of papers published by RSC have some kind of structured funding metadata. An additional benefit of this feedback loop was its educational effect by alerting authors to the importance of correct funding information. But even RSC continues to struggle with issues of funder name ambiguity, use of acronyms, authors reporting grant or award names instead of funder names, issues with phraseology of funding acknowledgements, and frustrations with the user experience of the service provider integrations with the OFR.

Many publishers agreed that collecting funding information from full-text papers is the preferred option. Not only because it lowers the burden for authors, but also because this potentially renders better data as this is where authors are expected to include this information as part of their funder’s commitments.

4. Retaining information and submitting: no big deal

At the beginning of the workshop, it was expected that maybe the retention of funding information and the propagation through various interlinked systems might pose problems for publishers. However, this was not identified as a problem by participants. Nor was there mention of any challenges in depositing information to Crossref, nor of downstream databases having difficulties retrieving the metadata.

5. There is a genuine interest across the ecosystem to improve funding information in Crossref

While many concluded that there was still a long way to go to solve the many technical challenges related to funding metadata, attendees were unanimous on its importance. Participants agreed that these improvements would require investments from publishers. A willingness to do those was expressed, but also a sense that publishers who do should be incentivised for it, maybe as part of the agreements they have with library consortia. JISC’s recent contract with Taylor & Francis (page 164, Section 7a (iii)) is a good example of how consortia can successfully negotiate the supply of high quality metadata, including funding metadata. It was agreed that another solution could be to allow the additional deposit of the free-text acknowledgement section as a metadata field in Crossref. Instead of educating authors to enter their data correctly or relying on publishers and tech providers to improve their systems to turn free text funder acknowledgement text to structured data, text mining and machine learning could facilitate the improvement of this data.

Next steps

For this workshop, we concentrated on the collection and registration of funding metadata by publishers and did not go into the important, related, issue of the Crossref Grant Linking System (Grant IDs) nor of the plans to further align funder IDs with ROR IDs, both projects that help the community to better record funding information.

Next steps resulting from this community workshop, as

Funders are encouraged to join and register their grants with Crossref DOIs so that registered grants can in future be linked directly to publications and other outputs. About 50 funders have already created around 90,000 grant records. The more grant DOIs that are created by funders, the more likely publishers will be able to prioritize collecting them in their own publication metadata.
Publishers are encouraged to work with their service providers to prioritize the quality of the open funding metadata through Crossref, which is a source for downstream analyses and inclusion by many thousands of tools and services.
Other stakeholders are also offering opportunities to focus on funding metadata, showing a growing interest in the completeness of funder metadata. For example, OA Switchboard’s funder pilot, which also looks at the potential to feed enriched metadata back to Crossref to make them publicly available, and the Open Research Funder Group’s work to promote the improvement of tracking research output, including funding metadata, which includes an active working group in this area.
Crossref will continue to work with publishers and service providers to encourage and make it easier to include funder information in article metadata, including the use of grant identifiers and funder identifiers. Work is underway to bring the Open Funder Registry closer to ROR (Research Organization Registry), and is planning, at some point in the future, to merge the OFR into ROR, as ROR has a much wider scope and is more broadly community-governed. Crossref has also begun some work on collecting ROR IDs where we currently collect Funder IDs. More technical information is available in this ticket).

We would like to thank all the participants of the workshop for their openness and commitment to working through these issues together. It was a rare opportunity to share insights from publishers, service providers, funders, and researchers - and a useful first step in co-creating a shared understanding of the challenges and charting a path forward.

Perspectives: My thoughts on starting my new role at Crossref

Johanssen Obanda — Thu, 06 Jul 2023 00:00:00 +0000

My name is Johanssen Obanda. I joined Crossref in February 2023 as a Community Engagement Manager to look after the Ambassadors program and help with other outreach activities. I work remotely from Kenya, where there is an increasing interest in improving the exposure of scholarship by Kenyan researchers and ultimately by the wider community of African researchers. In this blog, I’m sharing the experience and insights of my first 4 months in this role.

Right before joining Crossref, I was working as Stakeholder Manager with AfricArXiv, a community-led digital archive for African research communication. I transitioned to working with Crossref to take up a more challenging role, so I can apply the community-building and social innovation skills I gained over the last five years in my profession.

What surprised me the most here is realising that such a robust infrastructure is being administered by a relatively small team. I wondered how the team keeps the services running and builds new solutions for the community. However, I am impressed by the collaborative culture, positive and healthy work environment, and great systems.

I work within the Community Engagement and Communications team, where we collaboratively address members’ questions and challenges, plan events, create helpful content for our community and keep in touch with them. We help grow our community and create a better experience using our products and services.

My main focus has been the Ambassador programme, which started in 2018 and currently comprises 48 Ambassadors globally. The Ambassadors are our trusted contacts who support and engage our communities locally to make scholarly communications better. Through one-on-one virtual interaction with most of them, I noted that there was little interaction among the Ambassadors. Most of our Ambassadors want to connect more, both face-to-face and online. In the coming months, we aim to design our meetings together with the Ambassadors to encourage better exchange and relationships.

I value Crossref’s insistence on diversity, equity and inclusion, and I enjoy contributing to those activities. Working with my colleagues in the outreach team to organise webinars and activities for the Global Equitable Membership (GEM) programme has been an exciting experience. I particularly enjoyed engaging with our Ambassadors Shaharima Parvin and Jahangir Alam from Bangladesh, and Binayak Pandey from Nepal, in organising the initial webinars for the GEM program in their countries. I feel it is one of the ways of creating more in-depth connections between our communities and our Ambassadors while making it possible for more institutions to be part of Crossref and contribute to scholarly communication.

I have made a few webinar presentations online and recently did one in-person poster presentation in South Africa at the Sustainability, Research and Innovation conference. I gained more confidence interacting with the wider Crossref community and a deeper understanding of Crossref’s services. I look forward to more opportunities to discuss Crossref’s mission with the community and to collaborate with like-minded organisations, contributing to joint initiatives, such as the upcoming Better Together webinar series with ORCID and DataCite, and the Forum for Open Research in MENA events.

I experienced the challenges of working remotely in many ways. A couple of days, there was no power, other days the internet connection was painfully slow, and hopping from one restaurant to another was something I had to deal with from time to time, with the hopes of finding quiet most times to have a good meeting with my colleagues, until I had more dependable work station. On the positive side, coordinating meeting times with colleagues, taking on tasks asynchronously and collaborating in real-time across different tools are making me more agile, patient and empathetic with myself and my colleagues.

I am driven by the impact I want to contribute in my career working with Crossref, which is to build an inclusive research ecosystem where researchers across the globe can easily access scientific knowledge and make meaningful connections. And I feel confident about my colleagues, our systems and infrastructure and my capabilities to be part of a thriving community and organisation.

A Request for Comment - Automatic Digital Preservation and Self-Healing DOIs

Martin Eve — Thu, 29 Jun 2023 00:00:00 +0000

Digital preservation is crucial to the “persistence” of persistent identifiers. Without a reliable archival solution, if a Crossref member ceases operations or there is a technical disaster, the identifier will no longer resolve. This is why the Crossref member terms insist that publishers make best efforts to ensure deposit in a reputable archive service. This means that, if there is a system failure, the DOI will continue to resolve and the content will remain accessible. This is how we protect the integrity of the scholarly record.

I will write another post, soon, on the reality of preservation of items with a Crossref DOI, but recent work in the Labs team has determined that we have a situation of drastic under-preservation of much scholarly material that has been assigned a persistent identifier. In particular, content from our smaller Crossref members, with limited financial resources, is often precariously preserved. Further, DOI URLs are not always updated, even when, for instance, the underlying domain has been registered by a different third party. This results in DOIs pointing to new, hijacked, and elapsed content that does not reflect the metadata that we hold.

We (Geoffrey) have (has) long-harboured ambitions to build a system that would allow for automatic deposit into an archive and then to present access options to the resolving user. This would ensure that all Crossref content had at least one archival solution backing it and greatly contribute to the improved persistent resolvability of our DOIs. We refer to this, internally, as “Project Op Cit”. And we’re now in a position to begin building it.

However, we need to get this right from the design phase out. We need input from librarians working in the digital preservation space. We need input from members on whether they would use such a service. We are not digital preservation experts and we are acutely aware that we need the expertise of those who are, particularly where we’ve had to take some shortcuts. For instance: we are aware that the Internet Archive is perhaps not the first choice of many digital preservation librarians and specialists, who opt for specific scholarly-communications solutions. However, it is easy, open, and free. Hence, we propose for the prototype to use IA, on the assumption that this will be a proof-of-concept only, which we will expand to other archives if there is demand and once it works.

So: please do read the below and add your comments and questions to this thread in the community forum (link below), or send me queries/concerns by email. It would be excellent if we could receive comments by mid-August 2023. If you would rather comment on a Google doc, that’s also possible.

If enough people are interested, we could also host a community call to discuss this design and its prototyping. Do please, when emailing, let me know if this is of interest.

Project Op Cit (Self-Healing DOIs)

Request for Comment

This document sets out the problem statement, a proposed prototype solution, and a transition path to production if successful.

Proposed Prototype Solution

For members who opt-in to the service, We have a special class of DOI (only for open-access content) where, when the DOI is registered:

We immediately make an archive of the item with any archiving services that care to participate in the project (minimally, the Internet Archive, which is the easiest for us to begin with, but a modular/pluggable archival system). The Internet Archive Python Library should let us submit to them. We could pursue other arrangements with CLOCKSS, LOCKSS, and Portico.
We update the XML to reflect the archives to which it has been submitted.
The DOI landing page is redirected to an interstitial page that we control. This page gives the user access options.
We develop processes to determine whether the original URL “works”. The heuristics that define whether a resource has changed substantially or works need long-term consideration and real-world testing. Using the interstitial page approach will allow us to refine this, with a long-term goal of eradicating it.

Figure 1: The Deposit Process

Figure 2: The Resolution Process

Potential Challenges

Content drift. It would be extremely difficult to detect content change vs. (eg) page structure change, except in the case of binary fulltext. However, we can poll for the DOI at an HTML endpoint and detect when binary fulltext items, such as a PDF, change.
Latency on resolver if lookup is real-time. For this reason, we need a periodic crawler so that resolvers do not wait for real-time detection on access.
If using Internet Archive, the domain owner (at the present moment) can request the removal of content. We would need the capacity to “lock” records that are being used as Op Cit redirection archival copies. This requires a further conversation with the Internet Archive.

Prototype Components/Architecture

Registration Proxy and Database (“Fleming”)

The registration proxy implements a pass-through to the deposit API and hosts a relational database of self-healing DOIs (Postgres). It will be hosted at api.labs.crossref.org/deposit/opcit and clients will have to use this endpoint to deposit. Simultaneously, the proxy will:

Determine the license status of the incoming item.
If the license is open and fulltext is provided, deposit a copy in selected digital preservation archives. Store proof of licensing attestation.
In the case of binary files (fulltext PDF), store a hash of the content.
Store the DOI, binary hash, and all URLs in a relational database under “pending” state.
Pass through the request to Crossref’s content registration system.
Monitor the result of this request and remove stored data if registration fails.
Re-registration through Fleming will update existing entries and re-fix their data against content drift at this time.

Spider (“Shelob”)

A series of components that:

Check that “pending” DOIs have been successfully registered. Remove those that have not and move those that have to “active” state.
Dereference “active” DOIs and ensure that we have the most current URL in case updates have gone directly to the live resolver.
Periodically crawl URLs in the self-healing database.
On HTTP 301 code, update database entry to point to new permanent URL.
On HTTP 302 code, follow the temporary redirect expecting the original content.
On HTTP 4xx codes, mark the entry as dead.
On HTTP 200 code of HTML landing page, parse the page for the presence of the DOI. If the DOI is not present, mark the entry as dead.

Resolver Proxy (“Hippocrates”)

Display an interstitial landing page with archival versions and an explanation.
At some future point, for active entries, resolve to the stored URL (faster but could be de-synced) or pass the request to the live resolver (requires an extra hop but will always be in-sync with deposit).

Observability and statistics

Metrics we will collect:

Count of DOIs using Op Cit
Count of visitors arriving on Op Cit landing pages
Usage count of each outgoing link/access option

A daily report will present:

Newly “failed” entries that we believe have died
These will be checked extensively, particularly at first, to ascertain whether our failure heuristics are valid
Entries that have recovered

Errors will be logged and monitored via Grafana.

Documentation and Automated Tests

Core assumptions and new behaviours of the platform will be documented as part of the prototype.
Automated tests will be written, especially for the spider (“Shelob”), which must handle a diverse variety of real-world situations.

Prototype Architecture Requirements

Postgres RDS for resolution/self-healing DOI data (AWS).
FastAPI hosting for passthrough proxy (fly.io).
EC2 hosting for the spider (AWS).
FastAPI hosting for resolver proxy (fly.io).

Transition to Production

If this prototype garners popular appeal, a transition to production would need to keep some prototype components and rewrite others.

“Fleming” would need to be rewritten as a deposit module / integrated with Manifold’s (the next-generation system at Crossref) deposit. If this would create too much overhead, it need not be a blocking process in the deposit.
“Shelob” would continue to need to run continuously and to scale with the adoption of self-healing DOIs unless one of the other options were used.
Prototype architecture will be written so that spidering can be distributed between several servers, if required.
“Hippocrates” would need to be integrated into the live link resolver. Depending on how a field for a self-healing DOI is embedded in Manifold, this may not need any additional database hits.

Back Content

We also have a database of back content stored by the Internet Archive, mapped to DOIs where they have been able to do so. This data source could be used to enable self-healing DOIs on all content in this archive.

Crossref Research and Development: Releasing our Tools from the Ground Up

Martin Eve — Wed, 21 Jun 2023 00:00:00 +0000

This is the first post in a series designed to showcase what we do in the Crossref R&D group, also known as Crossref Labs, which over the last few years has been strengthened, first with Dominika Tkaczyk and Esha Datta, last year with part of Paul Davis’s time, and more recently, yours truly. Research and development are, obviously, crucial for any organisation that doesn’t want to stand still. The R&D group builds prototypes, experimental solutions, and data-mining applications that can help us to understand our member base, in the service of future evolution of the organisation. One of the strategic pillars of Crossref is that we want to contribute to an environment in which the scholarly research community identifies shared problems and co-creates solutions for broad benefit. We do this in all teams through research and engagement with our expanding community.

For example, if the metadata team wants to implement a new field in our schema, it helps to have a prototype to show to members. The Labs team would implement such a prototype. If we want to know the answer to a question about the 150m or so metadata records we have – e.g. how many DOIs are duplicates? – it’s the Labs team that will work on this.

When building such prototypes, which can often seem esoteric and one-off, though, it can be easy to believe that there is no way anybody else would re-use our components. At the same time, we find ourselves consistently working with the same infrastructures, re-using different code blocks across many applications. One of the tasks I have been working on is to extract these duplicated functions and to get them into external code libraries.

Why is this important? As many readers doubtless know, Crossref is committed to The Principles of Open Scholarly Infrastructure. For reasons of insurance, everything we do and newly develop is open source and we want our members to be able to re-use the software that we create. It’s also important because, if we centralize these low-level building blocks, we make it much easier to fix bugs when they occur, which would otherwise be distributed across all of our projects.

As a result, Crossref Labs has a series of small code libraries that we have released for various service interactions. We often find ourselves needing to interact with AWS services. Indeed, Crossref’s live systems are in the process of transitioning to running in the cloud, rather than our own data centre. It makes sense, therefore, for prototype Labs systems to run on this infrastructure, too. However, the boto3 library is not terribly Pythonic. As a result, many of our low-level tools interact with AWS. These include:

CLAWS: the Crossref Labs Amazon Web Services toolkit. The CLAWS library gives speedy and Pythonic access to functions that we use again and again. This includes downloading files from and pushing data to S3 buckets (often in parallel/asynchronously), fetching secrets from AWS Secrets Manager, generating pre-signed URLs, and more.
Longsight: A range of common logging functions for the observability of Python AWS cloud applications. Less mature than CLAWS, this is the starting point for observability across Labs applications. It supports running in AWS Lambda function contexts or pushing your logs to AWS Cloudwatch from anywhere else. It also supports logging metrics in structured forms. Crucially, the logs are all converted into machine-readable JSON format. This allows us to export the metrics into Grafana dashboards to visualize failure and performance.
Distrunner: decentralized data processing on AWS services. Easily the least mature and experimental of these libraries, distrunner is one of the ways that we distribute the workloads of our recurrent data processing. A number of the Labs projects require us to run recurrent data-processing tasks. For instance, my colleague Dominika Tkaczyk has developed the sampling framework that is regenerated once per week. We use Apache Airflow (and, specifically, Amazon Managed Workflows for Apache Airflow) to host these periodic tasks. This is useful because it gives us quick, visual oversight if tasks fail. However, the Airflow worker instances on AWS are quite severely underpowered and unsuitable for large in-memory activities. Hence, the sampling framework fires up a Spark instance for its processing. Often, though, we do not need the parallelization of Spark and just want to be able to run a generic Python script in a more powerful environment. That’s what distrunner is designed to do. The current version uses Coiled but this may change in the future.

While these tools will be useful to nobody except programmers – and this has been quite a technical post – there is a broader philosophical point to be made about this approach, in which everything is available for re-use, “from the ground up”. The point is: we also try, in Labs and in the process of “R&Ding”, to work without privileged access. That is: I don’t get “inside” access to a database that isn’t accessible to external users. I have to work with the same APIs and systems as would an end-user of our services. This means that, when we develop internal libraries, it’s worth releasing them. Because they use systems that are accessible to any of our users.

I should also say that our openness is more than unidirectional. While we are putting a lot of effort into ensuring that everything new we put out is openly accessible, we are also open to contributions coming in. If we’ve built something and you make changes or improve it, please do get in touch or submit a pull request. Openness has to work both ways if projects are truly to be used by the community.

Future posts – coming soon! – will introduce some of the technologies and projects that we have been building atop this infrastructure. This includes a Labs API system; new functionality to retrieve unpaginated datasets of whole API routes; a study of the preservation status of DOI-assigned content; and a mechanism for modeling new metadata fields.

Our annual call for board nominations

Lucy Ofiesh — Tue, 30 May 2023 00:00:00 +0000

The Crossref Nominating Committee invites expressions of interest to join the Board of Directors of Crossref for the term starting in March 2024. The committee will gather responses from those interested and create the slate of candidates that our members will vote on in an election in September.

Expressions of interest will be due Monday, June 26th, 2023.

About the board elections

The board is elected through the “one member, one vote” policy wherein every member organisation of Crossref has a single vote to elect representatives to the Crossref board. Board terms are for three years; this year, seven seats are open for election.

The board maintains a balance of seats, with eight seats for smaller members and eight seats for larger members (based on total revenue to Crossref). This is to ensure that the diversity of experiences and perspectives of the scholarly community are represented in decisions made at Crossref.

This year we will elect two of the larger member seats (membership tiers $3,900 and above) and five of the smaller member seats (membership tiers $1,650 and below). You don’t need to specify which seat you are applying for. We will provide that information to the nominating committee.

The election takes place online, and voting will open in September. Election results will be shared at the annual meeting on October 31st. New members will commence their term in March 2024.

About the Nominating Committee

The Nominating Committee reviews the expressions of interest and selects a slate of candidates for election. The slate put forward will exceed the total number of open seats. The committee considers the statements of interest, organisational size, geography, and experience.

2023 Nominating Committee:

Aaron Wood, American Psychological Association, chair*
Oscar Donde, Pan Africa Science Journal*
David Haber, American Society for Microbiology
Rose L’Huillier, Elsevier*
Marie Souliere, Frontiers

(*) indicates Crossref board member

What does the committee look for

The committee looks for skills and experience that will complement the rest of the board. Candidates from countries and regions that are not currently reflected on the board are strongly encouraged to apply. Successful candidates often have some or all of these characteristics:

demonstrate a commitment to or understanding of our strategic agenda or the Principles of Open Scholarly Infrastructure;
have expertise that may be underrepresented on the board currently;
hold senior/director-level positions in their organisations;
have experience with governance or community involvement;
represent member organisations that are active in the scholarly communications ecosystem;
demonstrate metadata best practices as shown in the member’s participation report

Board roles and responsibilities

Crossref’s services provide a central infrastructure to scholarly communications. Crossref’s board helps shape the future of our services and, by extension, impacts the broader scholarly ecosystem. We are looking for board members to contribute their experience and perspective.

The role of the board at Crossref is to provide strategic and financial oversight of the organisation, as well as guidance to the Executive Director and the staff leadership team, with the key responsibilities being:

Setting the strategic direction for the organisation;
Providing financial oversight; and
Approving new policies and services.

The board is representative of our membership base and guides the staff leadership team on trends affecting scholarly communications. The board sets strategic directions for the organisation while also providing oversight into policy changes and implementation. Board members have a fiduciary responsibility to ensure sound operations. Board members do this by attending board meetings, as well as joining more specific board committees.

Who can apply to join the board?

Any active member of Crossref can apply to join the board. Crossref membership is open to organisations that produce content, such as academic presses, commercial publishers, standards organisations, and research funders.

What is expected of board members?

Board members attend three meetings each year that typically take place in March, July, and November. Meetings have taken place in various international locations, and travel support is provided when needed. March and November board meetings are held virtually, and all committee meetings take place virtually. Each board member should sit on at least one Crossref committee. Care is taken to accommodate the wide range of timezones in which our board members live.

While the expressions of interest are specific to an individual, the seat that is elected to the board belongs to the member organisation. The primary board member also names an alternate who may attend meetings if the primary board member cannot. There is no personal financial obligation to sit on the board. The member organisation must remain in good standing.

Board members are expected to be comfortable assuming the responsibilities listed above and to prepare and participate in board meeting discussions.

How to apply

Please click here to submit your expression of interest. We ask for a brief statement about how your organisation could enhance the Crossref board and a brief personal statement about your interest and experience with Crossref.

Please contact me with any questions at lofiesh@crossref.org

Metadata connects the global community – summary of our Community update 2023

Kornelia Korzec — Fri, 12 May 2023 00:00:00 +0000

We were delighted to engage with over 200 community members in our latest Community update calls. We aimed to present a diverse selection of highlights on our progress and discuss your questions about participating in the Research Nexus. For those who didn’t get a chance to join us, I’ll briefly summarise the content of the sessions here and I invite you to join the conversations on the Community Forum.

You can take a look at the slides here and the recordings of the calls are available here.

TL;DR

The membership is growing, including that in the GEM programme countries, and we focus on adding new Sponsors in areas where we have insufficient coverage to support prospective members
The grant registration form is available for funders who don’t use XML, and we’re working to expand to other record types
The preview of the Relationship API endpoint is available – start exploring relationships between different records and record types, from citations to funding, and more
Usefulness of metadata records for inferring integrity of the content or publisher relies on all members of the community contributing to this effort. Crossref will continue to enrich our schema to capture new types of relevant information and to promote the best metadata practices.
Cited-by is now open for everyone to use 🎉 – no need for additional authorisation steps – Registering your references will have even greater impact now!
The Labs participation report is available and it’s been a hit. Please note that this tool is still underdevelopment – new functionalities can be added but there might also be bugs that we are yet to resolve, so don’t hold off with feedback.
We’ve received close to 1,000 responses in our first ever Metadata Priorities Survey. It’s still open until 18th of May and we encourage all members to take it. So far we’ve learnt that majority of our respondents are keen to deposit as much metadata as possible – and some would like to register more than we currently enable.

Metadata completeness and integrity

A key theme of the call was encouraging greater participation in the Research Nexus and the importance of complete metadata. One particular benefit of a rich and transparent metadata network is the opportunity to infer judgments on the integrity of the scholarly record (ISR). Amanda Bartell, Head of Member Experience, highlighted that the community agrees that availability of information about relationships between research outputs, institutions and other elements of the scholarly ecosystem together provide essential context for deciding about trustworthiness of organisations and their published content. Conversely, it can make it harder for parties to pass off information as trustworthy when that context is missing. Amanda summarised community feedback related to Crossref’s role in the integrity of the scholarly record in her recent blog post.

Our members can contribute to that rich network of relationships by curating their metadata and providing contextual information – especially the highly sought for elements highlighted in the presentation.

Our community

Since LIVE22, we have had 1,130 new members join us. That includes 51 organisations from countries included in our Global Equitable Membership (GEM) programme. You can find out more in the latest news about the programme on our Community Forum from Susan Collins, Community Engagement Manager.

We see great opportunities with enriching our metadata corpus with works carried out in some of the least economically-advantaged regions of the world. Registering their content with us will increase its discoverability for the global scholarship, while adding important relationships into the Research Nexus. We’re glad at the new members joining us under the auspices of the Global Equitable Membership (GEM) programme and we’re reaching out to existing and new communities with our Ambassadors, to encourage more metadata registrations.

Our Sponsors and Ambassadors, alongside our Outreach and Membership Team, support members to participate as effectively as possible in the Research Nexus. We’re delighted to see both programmes growing, with eight new Sponsors and seven new Ambassadors having joined us since October.

Simultaneously, we’re working with like-minded organisations to provide useful resources for the growing and changing scholarly communications community. The recent launch of the online forum for new publishers seeking to learn about best practices in the industry, The PLACE, is another way in which we hope to support wider participation in the Research Nexus, and promote open and sustainable practices.

With our growing community, there’s always interest in We have planned a webinar later this month to provide an overview of Crossref – including the members benefits and obligations, and how to use our services.

Service news

References metadata is essential for connecting works with one another. It enables provision of citation information, aids discoverability for researchers, as well as assessment and evaluation for institutions and funders. It’s almost a year since all the references metadata deposited with Crossref has been made openly available. At the moment, 52.0% of journal articles, and 44.5% of all works have references. Martyn Rittman, Product Manager for the Cited-by service says “It’s not bad, but we can do better!”

With three different mechanisms for doing it available to our members, we hope that all have a suitable tool to fit with their needs. You can register references with XML via HTTPS POST (structured or unstructured), with the dedicated OJS Plugin if you’re an OJS user, or with our Simple Text Query (unstructured text) – this is especially relevant to the Web Deposit Form users. We find that journal articles with deposited references seem to be cited more than those without, and by a lot: 21.8 vs. 6.1 incoming citations on average!

We have now made our Cited-by service open to all. To realise its full benefit, it is essential to register your references.

There were concerns in the community about references ‘lost’ as part of supplementary material that may not be registered in its own right. Colleagues advised that if the data has an identifier, such as a DataCite DOI, you can add a relationship to say that it’s supplementary material (see https://www-crossref-org.pluma.sjfc.edu/documentation/schema-library/markup-guide-metadata-segments/relationships/) or add them as a reference. Martyn is curious to hear from others in the community on this topic. There is an increasing focus on data citations and we’d like to see how we can better support them.

Many members have questions related to plans for replacing Metadata Manager. Rachael Lammey, Director of Product, explained that we’re working on broadening our new Grant Registration Form to include more record types over the course of 2023. It has a few advantages over the current Web Deposit Form. It allows you to save a local copy once you first register a piece of content. It makes updating your records easier, as you can drop that file onto the form to add the metadata so that you can update it and redeposit rather than having to fill out the information all over again, and we have started adding automatic lookup fields to help users populate information on affiliations using ROR IDs more accurately. We will keep you posted on the progress with new developments and ask for beta testers for new record types as they are added.

Metadata information about individual work is not as useful as the opportunity to interrogate the relationships between works and within the global scholarly output. [The preview of the Relationship API endpoint](https://community-crossref-org.pluma.sjfc.edu/t/relationships-are-here/3523, modest as it is at this stage – with only 1% of our relationship metadata included (or 10 mln relationships) – offers a powerful demonstration of the way in which metadata contextualises research outputs within the entangled network of ever-progressing scholarship.

We’ve also mentioned the recent transition of our website to GitLab, which allows everyone to contribute by creating merge requests and issues. Through this open collaboration, which supports our commitment to meet the Principles of Open Scholarly Infrastructure, we aim to cultivate a sense of ownership among contributors and make our information and documentation more useful and efficient for everyone.

Labs participation report

For organisations who wish to keep a close eye on their metadata – to understand what they deposit, how that compares with other members, and what could be improved, can start using our Lab participation reports. We encourage you to test this not-yet-finished tool and let us know your feedback. Participants at our updates found it very informative, with the opportunity to preview contents of recent deposits, see the participation breakdowns by a prefix, and improved data visualisation. We had questions about how data citation counts are generated in the report. Martyn Rittman explained that: “This is a prototype and that’s one of the issues we need to tidy up! We know via Event Data and our Scholix endpoint what is a dataset, but that hasn’t yet been incorporated to the Labs Reports”. There was also a suggestion of enabling export of simple lists of all member’s DOIs with respective URLs from the report and the team might look into that. Yet, lists of DOIs missing specific metadata types are already downloadable.

To learn more about the reports, try them out, and to provide feedback, please take a look at the information shared recently by Paul Davis, Tech Support Specialist & R&D Support Analyst.

Metadata priorities

Patricia Feeney, Head of Metadata, shared some updates about the current metadata corpus registered with Crossref, and some recent trends.

She then went on to summarise some preliminary results of our ongoing metadata priorities survey, which all members are encouraged to take part in by 18th of May. So far, we’ve received close to 1,000 responses. We’ve learnt that majority of our respondents are keen to deposit as much metadata as possible – and some would like to register more than we currently enable. Close to a half of the respondents who did not express an interest in sharing all metadata are still interested to learn more about the value of their metadata.

She then went on to summarise some preliminary results of our ongoing metadata priorities survey, which all members are encouraged to take part in by 18th of May. So far, We’ve received close to 1,000 responses. We’ve learnt that majority of our respondents are keen to deposit as much metadata as possible – and some would like to register more than we currently enable. However, close to a half of the respondents are interested to learn more about the value of their metadata.

The survey consults our members about their preferences for developing any of the potential projects under consideration:

Contributor IDs
Contributor roles/ CRediT
Alternate names
Multilingual metadata
Expand abstract support
Citation types (content)
Conference event IDs

It appears that support for citation types is the strongest among our respondents, while very polarised views have been shared about multilingual metadata and expanding support for abstracts. Among other suggestions, we received a lot of comments related to keywords. Overall, support for all projects was strong.

The verdicts are not in yet – still time to respond to the survey and make your metadata priorities known!

Thank you and keep in touch

With much of the content shared ahead of the time through our Community Forum, the sessions were bubbling with questions and valuable comments from the community. We look forward to continuing the conversations asynchronously on the Community Forum. Please don’t hesitate to share your thoughts and ask further questions. We’d also love to hear suggestions for topics of the most interest for our future updates.

The more complete the metadata we collect together, the more connections in the ecosystem become transparent. This creates opportunities for discovery and collaborations, and greater insights about the scholarly process. Our community is growing in numbers, diversity, and technical capacity for building the Research Nexus together. We welcome your questions and suggestions of initiatives that support the fullest participation possible.

2023 public data file now available with new and improved retrieval options

Patrick Polischuk — Tue, 02 May 2023 00:00:00 +0000

We have some exciting news for fans of big batches of metadata: this year’s public data file is now available. Like in years past, we’ve wrapped up all of our metadata records into a single download for those who want to get started using all Crossref metadata records.

We’ve once again made this year’s public data file available via Academic Torrents, and in response to some feedback we’ve received from public data file users, we’ve taken a few additional steps to make accessing this 185 gb file a little easier.

First, we’re proactively hosting seeds in a few locations around the world to improve torrent download performance in terms of both speed and reliability.

And second, we’ve added an option to download this year’s public data file directly from Amazon S3 for a small transaction fee paid by the recipient, bypassing the need to use the torrent altogether. The fee just covers the AWS cost of the download. Instructions for downloading the public data file via the “Requester Pays” method are available on the “Tips for working with Crossref public data files and Plus snapshots” page.

The 2023 public data file features over 140 million metadata records deposited with Crossref through the end of March 2023, including over 76,000 grant records. Because Crossref metadata is always openly available, you can use our API to keep your local copy of our metadata corpus up to date with new and updated records.

In previous years, closed and limited references were removed from the public data file. Since we updated our membership terms to make all deposited references open in 2022, the 2023 public data file for the first time includes all references deposited with us.

We hope you find this public data file useful. Should you have any questions about how to access or use the file, please see the tips below, or bring your questions to our community forum.

Tips for using the torrent and retrieving incremental updates

Use the public data file if you want all Crossref metadata records. Everyone is welcome to the metadata, but it will be much faster for you and much easier on our APIs to get so many records in one file. Here are some tips on how to work with the file.
Use the REST API to incrementally add new and updated records once you have the initial file. Here is how to get started (and avoid getting blocked in your enthusiasm to use all this great metadata!).
While bibliographic metadata is generally required, because lots of metadata is optional, records will vary in quality and completeness.

Questions, comments, and feedback are welcome at support@crossref.org.

Similarity Check: look out for a refreshed interface and improvements for iThenticate v2 account administrators

Fabienne Michaud — Mon, 01 May 2023 00:00:00 +0000

In 2022, we flagged up some changes to Similarity Check, which were taking place in v2 of Turnitin’s iThenticate tool used by members participating in the service. We noted that further enhancements were planned, and want to highlight some changes that are coming very soon. These changes will affect functionality that is used by account administrators, and doesn’t affect the Similarity Reports themselves.

From Wednesday 3 May 2023, administrators of iThenticate v2 accounts will notice some changes to the interface and improvements to the Users, Groups, Integrations, Statistics and Paper Lookup sections.

Logging in

iThenticate v2 account administrators and browser users will see a new login page when logging in to iThenticate v2:

A refreshed interface

Once logged in to iThenticate v2, account administrators will see an updated design, with improved notifications to let them know whether a task/action has been successfully completed or not.

Users

There will be improvements to the user management system for account administrators, including a much clearer navigation menu for managing active, pending and deactivated users.

There will also be a filtering option on the Users page to search for active, pending and deactivated users by first name, last name, email address, group and date added. In addition coloured labels will be introduced to easily identify the level of access (or ‘Role’) for each user.

An improved bulk user import process will be available, with clearer guidance on any issues that may arise during the upload. This new development will also include new screens for adding and editing users with more notifications to help prevent mistakes.

Integrations

For account administrators managing peer review management system integrations and needing to generate API keys, the Integrations page will be improved to make copying API keys simpler.

Statistics

iThenticate v2 administrators will also notice some improvements to the Statistics page. Usage data should load faster and will be sortable by user group. They will also be able to generate large usage reports of over 100k submissions.

Paper lookup

The Paper lookup will allow iThenticate v2 account administrators to find submissions that have been made from any integration connected to their iThenticate v2 account. They can be found by searching the paper ID (or oid number) of the submission.

Please note: the ability to search for submissions by the user’s name is available for manuscripts submitted via the iThenticate v2 website only and not for papers submitted via an integration.

New password requirements

To improve the security of users’ accounts, new password requirements will be introduced, including a minimum of 8 symbols, 1 special symbol, 1 upper case letter, and 1 number.

Next in iThenticate v2

Turnitin, who produce iThenticate, are currently working on a number of new features and developments including an improved similarity report, paraphrase and AI writing detection. A detailed timeline is not yet available but we’ll be updating you on these new developments in the coming months.

✏️ Do get in touch via support@crossref.org if you have any questions about iThenticate v1 or v2 or start a discussion by commenting on this post below.

ISR part four: Working together as a community to preserve the integrity of the scholarly record

Amanda Bartell — Wed, 26 Apr 2023 00:00:00 +0000

We’ve been spending some time speaking to the community about our role in research integrity, and particularly the integrity of the scholarly record. In this blog, we’ll be sharing what we’ve discovered, and what we’ve been up to in this area.

We’ve discussed in our previous posts in the “Integrity of the Scholarly Record (ISR)” series that the infrastructure Crossref builds and operates (together with our partners and integrators) captures and preserves the scholarly record, making it openly available for humans and machines through metadata and relationships about all research activity. This Research Nexus makes it easier and faster for everyone involved in research performance, management, and communications to understand information in context and make decisions about the trustworthiness of organisations and their published research outputs. Conversely, it can make it harder for parties to pass off information as trustworthy when the information doesn’t include that context.

The community needs open scholarly infrastructure that can adapt to the changes in scholarly research and communications, and we’ve been changing and adapting already by building on the concept of the scholarly record with our vision:

Like others, we envision a rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society.

We don’t assess the quality of the work that our members register, and we keep the barriers to membership deliberately low to ensure that we are capturing as much of the scholarly record as possible and encouraging best practice. We are careful to talk about Crossref’s specific role being with the Integrity of the Scholarly Record (ISR), and not the broader area of ‘research integrity’ (i.e. the integrity of the research process or content itself).

But there are many challenges and threats to research integrity and the integrity of the scholarly record, and there are tradeoffs with keeping the barriers to membership low. With that in mind, we have been dedicating more time to speaking with the community to explore what part we are and should in future play to help the community assess and improve trustworthiness in the scholarly record. We also want to work out where we can make use of our neutral, central role to convene different groups in scholarly communications to work together on these challenges.

A revealing afternoon in Frankfurt

Our starting point was a roundtable discussion in Frankfurt in October 2022. We organized it to coincide with the Frankfurt Book Fair, but the invited participants were from a wider spectrum than just publishers. The 40 invited participants represented editors, funders, research integrity professionals at publishers, representatives of ministries of science, and other partner organisations such as OASPA, COPE, STEM and DOAJ.

This half-day session enabled us to sense-check our thinking with the community and get input into whether our position is the best one for their needs.

Ed Pentz introduced the session by reminding participants that integrity is key to Crossref’s mission and is the basis of the shared Research Nexus vision. Amanda (that’s me) talked through our current membership processes, recent membership trends, and why wider participation is key and also the sort of questions the community comes to Crossref to solve (eg title ownership disputes). And finally, Ginny Hendricks talked through the specific services and metadata that Crossref has already developed to support the community as signals of trustworthiness, and introduced some new activities and ideas.

You can check out the slide deck and for more background, read our previous posts in the ISR series.

Participants then split into small groups representing a mix of communities, and we asked them to discuss three key questions:

Is Crossref’s role what you expected? What surprised you? What are we missing?
Are you aware of Crossref services? What are the barriers to more uptake? What are the challenges and opportunities?
What more could Crossref or its members do?

After discussion, each small group fed back to the room, and we followed up with a whole group discussion, before ending the day with a post-it note exercise for what Crossref should start doing, stop doing, and continue doing.

Here’s what we learned.

The importance of whole community involvement in research integrity and ISR

The need for all parts of the community to come together to solve the problems of research integrity came through loud and clear - there is no single group that can solve this problem on its own.

Publishers expressed frustration that responsibility for research integrity has been placed seemingly solely in their hands when institutions and funders can “unwittingly incentivise bad behaviour”. But it was clear that funders are just as concerned with research integrity issues, with many having made a dedicated trip for the roundtable. There were comments that bringing publishers and funders together around these issues was a rare but important opportunity, and there were calls for this to be an annual event. Both funders and publishers called for more involvement from and inclusion of research institutions in the discussion.

The group agreed that Crossref’s main focus should continue to be capturing and sharing the scholarly record, and that metadata and relationships are key for attribution, evidence, and provenance. One participant commented that “you can’t make open science work unless the metadata is complete” and that this would only happen with efforts throughout the community. Accurate and complete metadata needs to be:

pushed for by funders and institutions (through advocacy and policy)
provided by the authors and other contributors
collated, curated, and registered by the publishers and repositories
collected, matched, (sometimes cleansed), and distributed by Crossref.
(and we would add “prioritised by all who want to support open infrastructure over commercial alternatives”)

Interestingly, this echoes the ‘metadata personas’ output of the Metadata 20/20 initiative which defined roles in the community’s collective metadata effort:

Metadata Creators: providing descriptive information (metadata) about research and scholarly objects.
Metadata Curators: classifying, normalising, and standardising this descriptive information to increase its value as a resource.
Metadata Custodians: storing and maintaining this descriptive information and making it available for consumers.
Metadata Consumers: knowingly or unknowingly using the descriptive information to find, discover, connect, cite, and assess research objects.

Importance of whole-publisher involvement

A few participants, particularly those in editorial or integrity roles at publishing organisations, had not previously made the connection that metadata could be important signals of integrity. This highlighted a key problem - working with Crossref is seen by publishers as a technical/production workflow issue, and so knowledge of the benefits of metadata can be siloed within those teams. Crossref needs to reach out to editorial and research integrity teams to explain that good metadata isn’t just an end in itself and reinforce the impact it has on research integrity. This buy-in from across publisher organisations is vital.

We’re currently recruiting a Community Engagement Manager with editorial or research integrity experience to dedicate time to this area, to advocate for richer metadata within the editorial community, and progress this important conversation.

Agreement of the importance of metadata but an acknowledgment that this brings extra cost

Most participants agreed that rich metadata and relationships provide a core tool in establishing and protecting integrity. But they also acknowledged that collecting and registering more metadata often comes with an extra cost - whether that’s from system changes or just extra staff time. This is particularly true where publishers are working with third-party platforms and suppliers where there may be additional costs for adding fields and functionality to collect more metadata and register it with Crossref. Where knowledge of metadata is siloed in technical and production teams, and the wider benefits aren’t acknowledged, it can be hard to get internal buy-in for these extra costs and efforts.

The Frankfurt group also pointed out that the benefits of more comprehensive metadata (and what this means for ISR) are spread across the research ecosystem, but it is the publisher that usually bears the costs.

Need to define which metadata elements are trust signals and make it easier for the community to provide and access them

Through the course of the discussion, various elements were determined to be important to capture as “trust signals” and to identify relationships such as for retractions, conferences, reviewers, data, and when Crossref membership has been revoked for cause. We need to spend time identifying and prioritising these so that our members can do the same.

We need to make it easier for smaller, less technically-resourced members to provide this metadata, both through our tools and our documentation, as “doing this work can be very geeky and the documentation isn’t easy to understand as a layperson”.

There was also a discussion about where the metadata comes from - should community members be able to contribute metadata and assertions to other members’ records? If the provenance is captured then yes.

Once the metadata is captured, there remain challenges for users in where to start with the 145 million Crossref records. The groups asked Crossref to make it easier for community members to understand and use these records to make informed decisions, including by creating and sharing sample queries, libraries, and case studies.

We’re currently recruiting a Technical Community Manager to help improve the support we provide in this area to API users, service providers, and other metadata integrators .

The importance of retractions/corrections information

There was a lot of discussion about retractions and their importance as trust indicators. The group was surprised by how few retractions are currently registered with Crossref through Crossmark (12k). There was a lot of discussion around why Crossmark isn’t currently being adopted, and interest in taking this forward.

This needs to be a focus for Crossref, to encourage members to register retractions, corrections, and updates, and to make it easier for smaller publishers. There are new and emerging publishers who really want tools to help them demonstrate the legitimacy of their research, and an easy way for them to record corrections and retractions is key.

In their paper Towards a connected and dynamic scholarly record of updates, corrections, and retractions (September 17th, 2022), Ginny Hendricks, Rachael Lammey, and Martyn Rittman discuss how retraction information could be more effectively used - for example, letting a preprint reader know that the resulting article has been retracted, or letting the author of an article know the data that they’ve based their work on has been withdrawn.

Collecting the information is just the start - cascading retraction information throughout the research ecosystem is the main goal, and Crossref plays a central role here. As noted in the Information Quality Lab’s project Reducing the inadvertent spread of retracted science: Shaping a research and implementation agenda, “Many retracted papers are not marked as retracted on publisher and aggregator sites, and retracted articles may still be found in readers’ PDF libraries, including in reference management systems such as Zotero, EndNote, and Mendeley”.

It’s particularly important that this information is fed back to funders and institutions, and the group discussed having push notifications to these audiences for retractions. Some funders even employ staff members whose main purpose is to identify retractions.

It was pointed out that there may be good sources of retraction information (such as Retraction Watch) that Crossref could incorporate and match in our metadata.

Gaps in ‘ownership’, and Crossref’s role

The group discussed the many gaps in ownership for elements of research integrity, and some groups wondered if Crossref should actually change our approach and take on more responsibility for vetting content. However, after discussion, the group mostly agreed that this would mean a change of mission (and more staff) for Crossref and potentially limit global participation, thus making the metadata corpus less useful. Crossref should provide the widest possible metadata in an easy-to-consume format, and “other organisations can provide the verification layer”.

It was acknowledged that it would be easy for Crossref to get overwhelmed, so we ended the day by discussing not only what we should start doing, but also what we should stop doing. Unsurprisingly, there was a lot more to continue or start doing than stop doing!

However, the fact remains that there are gaps in ownership - for example, there is no central arbiter of who ‘owns’ a journal. Also, where do you go if you have a problem with a journal? Often the Committee on Publication Ethics (COPE) is seen as a solution, but they can’t solve this problem alone - it needs a coordinated effort from funders, institutions, publishers, and other partner organisations such as the Open Access Scholarly Publishing Association (OASPA), the Directory of Opena Access Journals (DOAJ), and like-minded organisations.

Many noted that Crossref is well-positioned to convene horizontal multi-stakeholder discussions to start to find solutions.

We also know that there are other industry initiatives aimed at supporting this work. The STM Association’s work on an Integrity Hub is gathering pace and aims to provide, among other things ‘a cloud-based environment for publishers to check submitted articles for research integrity issues’.

What happened next? Turns out, it really is all about relationships…

Since this meeting in Frankfurt last October, we’ve been focusing on relationships - thinking about how we capture them in our metadata, and working in partnership with other organisations to bolster our support for ISR.

The rest of this blog post highlights some of the activities underway:

Increasing participation in Crossref

In January 2023, we launched our new GEM Program, which offers relief from fees for members in the least economically-advantaged countries in the world. By opening up participation even further, we aim to extend the corpus of open metadata, giving opportunities for more connections, more context, and more relationships.

Supporting members in meeting best practices

ISR blog 2 explained more about how we help new members become “good Crossref citizens” with automated onboarding emails, extensive documentation, events and webinars, and help from our support team, Ambassadors, and other members in our Community Forum.

We’ve recently joined forces with COPE, DOAJ, and OASPA to create a new online public forum for organisations interested in adopting best practices in scholarly publishing. At the Publishers Learning And Community Exchange or The PLACE, new scholarly publishers can access information from multiple agencies in one place, ask questions of the experts, and join conversations with each other. Do take a look!

Being clearer on the impact of better metadata

As discussed earlier, better metadata can sometimes bring extra costs, and it’s helpful to understand the impact of this investment. We know from our ongoing outreach work that it’s difficult for our members to keep hearing that Crossref needs more and better metadata. They ask us for resources and increasingly want to see hard evidence of benefits to them. We recently showcased the journey of the American Society for Microbiology which went from ‘zero to hero’ in terms of metadata participation and completeness in Crossref. They describe their efforts to increase their registered metadata over the last few years, and note a significant increase in their average monthly successful DOI resolutions from ~390,000 in 2015 to an average of ~3.7 million in 2022. They found that “the more metadata we push out into the ecosystem, the more it appears to be used… Remembering that your publishing program benefits as much as everyone else’s when you deposit more metadata can help refine your short-term and long-term priorities.”

We know we sound like a broken record sometimes, but now other members can take it from ASM!

Encouraging better metadata and more relationships and identifying ’trust signals'

We’re trying to make it easier for members to accurately register key metadata fields, with the launch of our new grants registration form which will be extended to journals and other record types soon. This includes a ROR lookup - adding this unique identifier for research organisations gives even better context for the metadata.

We are also working to make it possible for anyone to contribute to metadata records, and have the provenance of these contributions clearly asserted.

Metadata adoption is still a key goal for our staff; indeed our new 2023-2025 strategic roadmap specifies…

“We want to be a sustainable source of complete, open, and global scholarly metadata and relationships. We are working towards this vision of a ‘Research Nexus’ by demonstrating the value of richer and connected open metadata, incentivising people to meet best practices, while making it easier to do so.”

… with item number one under projects ‘in focus’, being: “Adoption activities to focus on top metadata adoption priorities, which are:

We’re continuing to talk with the community to work out which metadata elements are most useful as trust signals, and we’re trying to prioritise some of the schema changes required to capture new elements. If you haven’t already, please respond to Patricia Feeney’s metadata priorities survey.

Thinking about retractions and corrections

We’ve been closely involved with the NISO CREC working group, and they should be making the initial draft recommendations public soon - watch this space!

Making it easier to view and compare metadata and expand the relationships

Our Participation Reports provide a visualisation of the metadata that’s available via our free REST API. There’s a separate Participation Report for each member, and it shows what percentage of that member’s content includes nine key metadata elements. It’s an important tool to help those in the community understand our metadata more easily.

We have been working on a new version of Participation Reports, allowing more comparison between members, and extra metadata elements to communicate trustworthiness, including whether each member has thought about the long-term preservation of their content, and whether it has been added to a repository. There is a test version to look at in our Labs sandbox. Do take a look and provide feedback.

We’ve also made public our list of members whose membership was revoked for contravention of the membership terms.

Continuing to work with funders

We’re continuing to work with funders through our growing funder membership, the Funder Advisory Group and other groups, including the Open Research Funders Group, the HRA, Altum, Europe PMC, and the ORCID Funder Interest Group. And we’re continuing to build the important relationships between funding and outputs (see Dominika Tkaczyk’s recent report) and engage with this key audience for research integrity.

Discussions with the community

We’ll be talking about ISR at our next community update on May 3rd - there are two versions of the meeting depending on your timezone - do sign up if you haven’t already. And if you’re attending the SSP conference in June, do come along to our panel “Working together to preserve the integrity of the scholarly record in a transparent and trustworthy way”.

We’re hiring! New technical, community, and membership roles at Crossref

Michelle Cancel — Fri, 21 Apr 2023 00:00:00 +0000

Do you want to help make research communications better in all corners of the globe? Come and join the world of nonprofit open infrastructure and be part of improving the creation and sharing of knowledge.

We are recruiting for three new staff positions, all new roles and all fully remote and flexible. See below for more about our ethos and what it’s like working at Crossref.

🚀 Technical Community Manager, working with our ‘integrators’ so all repository/publishing platforms and plugins, all API users incl. managing contracts with subscribers, and generally helping a very nice bunch of RESTful API dabblers, both novice and intermediate. The goal is to offer more interactive engagement such as sprints, and more technical consultation to help the community with things like query efficiency, public data dump ingestion, etc. Thousands of users exist, from individual researchers and small academic tools to giant technology companies. Researching and analysing usage and building tools to meet their needs is key, so this role works closely with Product and R&D colleagues and likely needs a developer or developer-advocacy background.

🎯 Member Experience Manager, ramping up to handle the mammoth operation that is… membership, currently 18,000 members from 150 countries, and onboarding the ~180 new joiners we welcome monthly, mostly from Africa and Asia. This role involves lots of education and relationship management, but because of the scale, we also need someone with a real business process/analysis approach, improving how our systems function so that the operation flows seamlessly and isn’t a pain for people (both members and staff). This role manages two full-time Member Support Specialists (UK and Indonesia) and three part-time contractors (USA, France, and one other as yet unknown).

🎈 Community Engagement Manager, working with the global community of scholarly editors at a time when research integrity is top of mind for our entire ecosystem. This is a classic community role for someone keen to cross over from managing or editing journals or books and perhaps make your volunteer work official. Activities will include program and project management, event and working group facilitation, communications and content creation. You’d be interacting with groups like the Asian Council of Science Editors, the European Association of Science Editors, and the Council of Science Editors, plus many more that you’d identify. It’s all about helping editors, who work hand-in-hand with authors, to think about metadata as signals of trust and better use available services, such as those for retraction management or plagiarism checking, and helping to define needs for emerging activity too, such as machine-generated content.

Working at Crossref

We’re a not-for-profit membership organisation that exists to make scholarly communications better. We rally the community; tag and share metadata; run an open infrastructure; play with technology; and make tools and services—all to help put research in context.

Crossref sits at the heart of the global exchange of research information, and our job is to make it possible—and easier—to find, cite, link, assess, and reuse research, from journals and books, to preprints, data, and grants. Through partnerships and collaborations we engage with members in 150 countries (and counting) and it’s very important to us to nurture that community.

We’re about 45 staff and remote-first. This means that we support our teams working asynchronously and with flexible hours. We are dedicated to an open and fair research ecosystem and that’s reflected in our ethos and staff culture. We like to work hard but we have fun too! We take a creative, iterative approach to our projects, and believe that all team members can enrich the culture and performance of our whole organisation. Check out the organisation chart.

We are active supporters of ongoing professional development opportunities and promote self-learning at every opportunity. Crossref has a healthy financial situation and we only continue to grow. While we won’t have a clear hierarchical path for staff to follow, there are always evolving opportunities to progress and be challenged.

We especially encourage applications from people with backgrounds historically under-represented in research and scholarly communications.

Bookmark our jobs page to watch for future opportunities!

The PLACE for new publishers – a one-stop-shop for information and a friendly community

Kornelia Korzec — Mon, 17 Apr 2023 00:00:00 +0000

The Publishers Learning And Community Exchange (PLACE) at theplace.discourse.group is a new online public forum created for organisations interested in adopting best practices in scholarly publishing. New scholarly publishers can access information from multiple agencies in one place, ask questions of the experts and join conversations with each other.

Scholarly publishing is an interesting niche of an industry – it appears at the same time ancillary and necessary to the practice and development of scholarship itself. The sooner and more easily a piece of academic work is shared, the greater the chance that others will find and build upon it. Many practices of the publishing industry have been developed to support discovery and integrity of the scholarship that produces shareable works, and as the landscape of scholarly communications constantly evolves, a number of agencies arose to promote and continuously update the standards and best practices within it.

We realise that the sheer number of agencies involved in regulating and preserving scholarly content is in itself a challenge and can be confusing. Newer publishers may find it difficult to know where to go to find the right information, what policies they need to follow or international criteria they need to meet and how to go about doing so. When time or finances are tight, it’s not easy to try to reinvent the wheel.

Following the long-established practice of signposting organisations between us, we’ve worked together with the Committee on Publication Ethics (COPE), the Directory of Open Access Journals (DOAJ), and the Open Access Scholarly Publishers Association (OASPA) to establish the PLACE. We share values and goals to work more effectively to better support the needs of our communities. Each organisation is taking actions to lower barriers to participation and provide greater support for the organisations that publish scholarly and professional content that we work with.

Hence, we envisaged the PLACE as a ‘one stop shop’ for access to more consolidated and plainly put information, to support publishers in adopting best practices the industry developed. We also hope that by setting the information service as a forum, we will encourage open exchange with publishers who aspire to do things right, as well as between them.

Renewed persistence

Joe Wass — Sat, 01 Apr 2023 00:00:00 +0000

We believe in Persistent Identifiers. We believe in defence in depth. Today we’re excited to announce an upgrade to our data resilience strategy.

Defence in depth means layers of security and resilience, and that means layers of backups. For some years now, our last line of defence has been a reliable, tried-and-tested technology. One that’s been around for a while. Yes, I’m talking about the humble 5¼ inch floppy disk.

This may come as surprise to some. When things go well, you’re probably never aware of them. In day to day use, the only time a typical Crossref user sees a floppy disk is when they click ‘save’ (yes, some journals still require submissions in Microsoft Word).

History

But why?

Let me take you back to the early days of Crossref. The technology scene was different. This data was too important to trust to new and unproven technologies like Zip disks, CD-Rs or USB Thumb Drives. So we started with punched cards.

IBM 5081-style punched card.

Punched cards are reliable and durable as long as you don’t fold, spindle or mutilate them. But even in 2001 we knew that punched cards’ days were numbered. The capacity of 80 characters kept DOIs short. Translating DOIs into EBCDIC made ASCII a challenge, let alone SICIs. We kept a close eye on the nascent Unicode.

Breathing Room

In 2017 the change of DOI display guidelines from http://dx.doi.org.pluma.sjfc.edu to https://doi-org.pluma.sjfc.edu shortened each DOI by 2 characters, buying us some time. But eventually we knew we had to upgrade to something more modern.

So we migrated to 5¼ inch floppy disks.

5¼ Floppy disk in drive

At 640 KB per disk these were a huge improvement. We could fit around 20,000 DOIs on one floppy. Today we only need around 10,000 floppy disks to store all of our DOIs (not the metadata, just the DOIs). Surprisingly this only takes about 20 metres of shelf space to store.

Typical work from home setup. Getting ready to backup some DOIs!

The move to working-from-home brought an unexpected benefit. Staff mail floppy disks to each other and keep them in constant rotation, which produces a distributed fault tolerant system.

Persistence Means Change

But it can’t last forever. DOIs registration shows no sign of slowing down. It’s clear we need a new, compact storage medium. So, after months of research, we’ve invested in new equipment.

Today we announce our migration to 3½ inch floppies.

If it goes to plan you won’t even notice the change.

Image credits

Punched card: IBM 5081-style punched card. Derived from public domain by Gwern.

Start citing data now. Not later

Geoffrey Bilder — Thu, 23 Mar 2023 00:00:00 +0000

Recording data citations supports data reuse and aids research integrity and reproducibility. Crossref makes it easy for our members to submit data citations to support the scholarly record.

TL;DR

Citations are essential/core metadata that all members should submit for all articles, conference proceedings, preprints, and books. Submitting data citations to Crossref has long been possible. And it’s easy, you just need to:

Include data citations in the references section as you would for any other citation
Include a DOI or other persistent identifier for the data if it is available - just as you would for any other citation
Submit the references to Crossref through the content registration process as you would for any other record

And your data citations will flow through all the normal processes that Crossref applies to citations. And it will be distributed openly to the community (including DataCite!) via Crossref’s services and APIs. All data citations deposited with Crossref will be exposed in the (soon-to-be launched) Data Citation Corpus.

And then, you can sit back and congratulate yourself for making your publication more useful to researchers who want to be able to reuse the data underlying your publications.

Background

You might ask, “So if submitting Data Citations to Crossref has long been possible, why do you have to write this?”

Historically, authors did not cite data in the way they cited publications. Instead, they would often refer to the data in the main text of the article. This has made it hard to determine what data lay behind the research and/or access the data.

But the research community has increasingly recognized that data is a first-class research output and that we should treat it as such. In short, we should formally cite data.

But because citing data is a comparatively new practice, it has been subject to a lot of new analysis. And unsurprisingly, people analyzing data citation have discovered that there is a lot of nuance to citation of any kind.

There are lots of reasons for citing something. There are lots of internalized conventions for citing things. And there are different conventions for citation for different research objects. And SSH citation practice differs from STEM. And legal citation practices are different from scholarly citation practices. And citation practices even vary by subdiscipline and by journal.

Those who have been looking at what it means to “cite data” have naturally stumbled into a thicket of divergent practices - some of which are historical holdovers, some of which are stylistic preferences, and some of which are clearly adaptations to deal with the specific needs of certain research objects/containers or different disciplines.

The temptation has been to try and rationalize this before extending the practice of citation to data.

“Maybe because data is a distinct record type, we should include the fact that it is a data citation in the citation itself?”

“Maybe because people cite data for different reasons, we should include a typology of citation types in all data citations?”

And so you may hear some people say, “hold off on data citation - we don’t have an optimal way to do it yet, and it can be very complicated.”

But guess what?

We currently don’t label citations to monographs as “citation to monograph.”

And we don’t currently include the reason for citation when we are citing a journal article.

It would be very cool if we did. And it would likely make citations even more useful if we did.

But citations are already useful even without these features. And so, to delay citing data indefinitely because we have an opportunity to improve the act of citation is just perverse. Our community has always opted for progress over perfection.

For one thing - the efforts are not mutually exclusive. We can start citing data with the current limitations of citation practices and simultaneously propose mechanisms for making citation more useful in the future, including new guidelines to deal with the unique issues that citing data poses.

But in the meantime, we will be doing researchers a giant favour if we at least include our imperfect and ambiguous, and unconventional references to data in the references section of an article so that they can be accessed and processed along with all the other imperfect, ambiguous and variant citations that we find so useful.

Some of our members are already doing this. They have been for a long time. And they haven’t found it any more complicated than managing non-data references in the past.

Join them and make your metadata more useful.

Cite data now. Don’t put it off.

And Crossref will continue to work with DataCite and the rest of the community to make the distribution even easier and more useful.

So who is already citing data?

Top 10 members depositing data citations from November-May 2022

(broken down by DOI prefix, which is why you see some publishers listed twice):

Prefix	Member name	Data citations deposited
10.1038	Springer Science and Business Media LLC	7174
10.1016	Elsevier BV	6527
10.1007	Springer Science and Business Media LLC	4748
10.5194	Copernicus GmbH	3017
10.1080	Informa UK Limited	2346
10.1177	SAGE Publications	2082
10.1002	Wiley	2048
10.1111	Wiley	1888
10.1108	Emerald	1876
10.3390	MDPI AG	1827

Top 10 data citations per deposited work

(again, broken down by prefix)

Member name	Prefix	Data citations deposited	Data citations per work
Consortium Erudit	10.7202	580	1.149
SLACK, Inc.	10.3928	462	0.646
S. Karger AG	10.1159	1653	0.532
Proceedings of the National Academy of Sciences	10.1073	973	0.502
American Academy of Pediatrics (AAP)	10.1542	486	0.397
F1000 Research Ltd	10.12688	552	0.341
American Association for the Advancement of Science (AAAS)	10.1126	952	0.317
Springer Science and Business Media LLC	10.1038	7174	0.231
JMIR Publications Inc.	10.2196	864	0.187
American Geophysical Union (AGU)	10.1029	692	0.166

These are for the prefixes with the most data citations deposited (>500 in 6 months) so there might be smaller members doing better than this.

Summaries are great, but I want to see some actual examples!

Here are some examples showing how data is cited by our members:

This eLife article: https://doi-org.pluma.sjfc.edu/10.7554/eLife.26410 cites this dataset in Dryad https://doi-org.pluma.sjfc.edu/10.5061/dryad.854j2.
This Copernicus article: https://doi-org.pluma.sjfc.edu/10.5194/acp-22-7105-2022 cite to this dataset https://doi-org.pluma.sjfc.edu/10.24381/cds.bd0915c6
This Sciendo article: https://doi-org.pluma.sjfc.edu/10.2478/plc-2021-0008 cites this APA-hosted language competence test https://doi-org.pluma.sjfc.edu/10.1037/t15159-000
This De Gruyter article: https://doi-org.pluma.sjfc.edu/10.1515/opth-2020-0160 cites this bibliography at Oxford Bibliographies: https://doi-org.pluma.sjfc.edu/10.1093/OBO/9780195396584-0012

And here are some example API requests for discovering more metadata citations. You can use these API requests as examples and adapt to your own needs.

Shooting for the stars – ASM’s journey towards complete metadata

Kornelia Korzec — Tue, 14 Mar 2023 00:00:00 +0000

At Crossref, we care a lot about the completeness and quality of metadata. Gathering robust metadata from across the global network of scholarly communication is essential for effective co-creation of the research nexus and making the inner workings of academia traceable and transparent. We invest time in community initiatives such as Metadata 20/20 and Better Together webinars. We encourage members to take time to look up their participation reports, and our team can support you if you’re looking to understand and improve any aspects of metadata coverage of your content.

In 2022, we have observed with delight the growth of one of our members from basic coverage of their publications to over 90% in most areas, and no less than 70% of the corpus is covered by all key types of metadata Crossref enables (see their own participation report for details). Here, Deborah Plavin and David Haber share the story of ASM’s success and lessons learnt along the way.

Could you introduce your organisation?

The American Society for Microbiology publishes 16 peer-reviewed journals advancing the microbial sciences, from food microbiology, to genomics and the microbiome, comprising 14% of all microbiology articles. Six of those are open-access journals, and 56% of ASM’s published papers are open access. Together, our journals contribute 25% of all microbiology citations.

Would you tell us a little more about yourselves?

DH: David Haber, Publishing Operations Director at the American Society for Microbiology. I live in a century-old house that is in a perpetual state of renovation due to my inability to stop starting new projects before I complete old ones.

DP: Deborah Plavin, Digital Publishing Manager at the American Society for Microbiology. Following David’s example, my apartment in Washington D.C. is just up the block from one of the homes Duke Ellington lived in https://www.hmdb.org/m.asp?m=142334.

What value do society publishers in general see in metadata in your view?

DP: In my view, robust metadata allows publishers to look at changes over time, do comparative analysis within and across research areas, more easily identify trends, and plan for future analysis (e.g., if we deposit data citation information and we change our processes to make it more straightforward, do we see any change in the percentage of articles that include that information, etc.).

DH: To echo Deborah’s point, to be able to name something distinctly and clearly identify its specific attributes is vital to understanding past research and planning for future possibilities. One of our fundamental roles as a publisher for a non-profit society is to properly lay this metadata foundation so that we can provide services and new venues for our members, authors, and readers that match their needs and track with the trends in research. Without good and robust metadata, it is impossible to truly understand the direction in which our community is pointing us.

Metadata for your own research outputs in the last year has grown rapidly. Why such focus on metadata in 2022?

DP: This is something that ASM has been chipping away at over time. Years ago we found that it wasn’t always easy to take advantage of deposits that included new kinds of metadata. That was either because we needed to work out how and where to capture it in the process or because platform providers weren’t always ready — coming up with ways to process the XML that publishers supply in many different ways takes time. These back-end processes that feed the infrastructure aren’t usually of great interest to stakeholders, and so it allowed us to play around, flounder, fail, refine, and try again.

We looked at having 3rd parties deposit metadata for us, and while that helped expand the kind of metadata we were delivering, it created workflow challenges of its own. What turned out to be most effective was budgeting for content cleanup projects and depositing updated and more robust metadata to Crossref.

We also benefited from a platform migration, which allowed us to take advantage of additional resources during that process.

DH: Coming from a production background, I have always been fascinated with the when and how of capturing key metadata during the publishing process. When are those data good and valuable, and when should they be tossed or cleaned up for downstream deliveries? Because Deborah and ASM directors saw a more complete Crossref metadata set for our corpus as a truly valuable target, we were able to really think hard about what kind of data we were capturing and when, how those requirements may have influenced our various policies and copyediting requirements over the years, and how best to re-engineer our processes with the goal of good metadata capture throughout our publishing workflows. From our perspective, Crossref gave us a target, a “this-is-cool-bit-of-info" that Crossref can collect in a deposit; therefore, how can we capture that during our processes while driving further efficiencies? ASM journals had been so driven by legacy print workflows that such a change in perspective (toward metadata as a publishing object) really allowed us to re-imagine almost everything we do as a publisher.

Has the OSTP memo influenced your effort?

DP: I think that the Nelson memo hasn’t changed our focus; instead, I think it’s been another data point supporting our efforts and work in this area.

DH: Deborah is exactly right. The release of this memo only re-affirmed our commitment to creating complete and rich metadata. The Nelson memo points to many possible paths forward, in terms of both Open Access and Open Science, but we feel our work on improving our metadata outputs positions us well to pick a path that best suits our goals as a non-profit society publisher.

How big was this effort? Could you draw us a picture of how many colleagues or parts of the organisation were involved? Did you involve any external stakeholders, such as authors, editors, or others?

DH: It was simple. Took five minutes… In all seriousness, the key is having the support of the organisation as a whole. To do this properly, it is vitally important to know the end from the beginning, so to speak. It is one thing to say let’s start capturing ORCID IDs and deliver them to Crossref, but it is completely another to create a cohesive process in which those IDs are authenticated and validated throughout the workflow. So something as simple as a statement “ORCID IDs seem cool, let’s try to capture them” could affect how researchers submit files, how reviewers log into various systems (i.e., ORCID as SSO), how data are passed to production vendors, what copyeditors and XML QC people need to be focused on, and what integrations authors may expect at the time of publication. Being part of an organisation that embraced such change allowed us to proceed with care with each improvement to the metadata we made.

But that is more about incremental improvement. The beginning of this process started when we were making upgrades to our online publishing platform, and we were trying to figure out how best to get DOIs registered for our older content. When we started looking at this, we soon realized that, sure, we could do the bare minimum and just assign DOIs to this older content outside the source XML/SGML, but did that make sense? Wouldn’t it make more sense, especially since we were updating the corpus to a new DTD, to populate the source content with these newly assigned DOIs? Once we decided that we were going to revise the older content with DOIs, it made sense for us to create a custom XSL transform routine to generate Crossref deposits that would capture as much metadata as possible. So, working with a vendor to clean and update our content for one project (an online platform update) allowed us also to make massive improvements to our Crossref metadata as a side benefit.

Of course, I do have to apologize to the STM community for the Crossref outages in late 2019. That was just me depositing thousands of records in batches one sleepless night.

What were the key challenges you encountered in this project, and how did you overcome them?

DH: Resources and time are always an issue. Much of the work was done in-house in spare moments captured here and there. But there are great resources in github and at Crossref to help focus on defining what is important and what is possible in such a project. And, honestly, defining what was important and weighing that against the effort to find said important bit in the corpus of articles we have was the most challenging part of this process. In other words, limiting the focus. Once one decides to start looking at the inconsistencies in older content, it is hard not to say: “Oh, look. That semi-important footnote was treated as a generic author note rather than a conflict-of-interest statement; let’s fix that.” Once you start down that path, you can spend years fiddling with stuff. For me, a key mantra was: “We now have access to the content. We can always do another Crossref metadata update if things change or shift over time.”

Have there been any important milestones along the way you were able to celebrate? Or any set-backs you had to resolve in the process?

DP: For as long as I can remember, the importance of good metadata has been among the loudest messages of best practice in the industry. I don’t think that I have been able to really quantify/ demonstrate the value of that work. Looking at the consistent increases in the Crossref monthly resolution reports that we saw between 2015 and 2022 and looking at our participation reports has helped provide some measure of progress. For example, the number of average monthly successful resolutions in that Crossref report in 2015 was ~390,000. The last time I checked, the 2022 numbers were ~ 3.7 million. In 2023, I hope that we will be able to leverage Event Data for this as well.

The setbacks have fallen into two categories: timing and process. Our internal resourcing to get this done within our preferred time frame, to have the content loaded and delivered, and triage problems—it’s a battle between the calendar and competing priorities.

DH: When Deborah first shared those stats with me, I was floored. I don’t think either of us suspected such an increase was possible. For me, the biggest setback was mistakenly sending about ~50,000 DOI records to queue and watching them all fail because I grabbed the wrong batch. Ooops. I never made that mistake again, though.

Was any specific type of metadata or any part of the schema particularly easy or particularly difficult to get right in ASM’s production process?

DH: For us, the most difficult piece of metadata revolves around data availability and how we capture linked data resources (outside of data citation resources). Because of our current editorial style (which had been print-centric for years), we did not do a good job of identifying whether there are data associated with published content in a consistent machine-readable way. We did some experiments with one of our journals to capture this outside of our normal Crossref deposit routine, but that was not as accurate or sustainable as we would have liked. But, in that experiment, we learned a few things about how we treat these data throughout our publishing process and we have plans to create a sustainable integrated workflow for this to capture resource/data linkages in our Crossref deposits.

What were your thoughts on last year’s move to open references metadata? Has that impacted on your project in any way?

DP: We were really excited about this; based on the rather limited approach to sorting out impact at the moment, the more metadata we push out into the ecosystem, the more it appears to be used. In my view, that is at the core of what society publishers want to do—ensure that research is accessible and discoverable wherever our users expect to find it.

DH: 100% agree.

How did you keep motivated and on-course throughout?

DP: These kinds of things are never done; for example, we have placeholders for CRediT roles, and getting ready for that work as part of a DTD migration will be the next big thing. The motivation for that is really meeting our commitment to the community, seeing the impact of the author metadata versus article metadata, and seeing what we can learn.

DH: Metadata at its core is one of the pillars of our service as a publisher. To provide the best service, we need to provide the best metadata possible. Just remembering that this can be incremental, allows us to celebrate the large moments and the small. And whether one is partying with a massive 7 layer cake or a smaller cake pop, both are sweet and motivating.

Now that the project is completed, are you seeing the benefits you were hoping to achieve?

DP: This is a hard one to answer as we are using limited measurements at this time. At a high level, I am pleased. While I am eager to leverage event data in the coming year, it would be really helpful to get feedback from the community on how we can improve as well as other ways to evaluate impact.

DH: I want to take up this idea of metadata as a service once more. I don’t mean in terms of discoverability or searchability, either. Let’s take ORCID deposited into Crossref as an example. When done properly (with the proper authentication and validation occurring in the background), we are able to integrate citation data directly to an author’s ORCID profile. We have found that this small service is really appreciated.

Is there any metadata that you’d like to be able to include with your publishing records in the future that isn’t possible currently? What would it be and why?

DP: CRediT roles would be great because it could give greater insight into collaboration within and across disciplines, it could allow for some automation and integration opportunities in the peer review process, and maybe it would visualize aspects of authors’ careers.

DH: I second capturing CRediT roles. What would be really interesting is also creating a standard that quantifies the accessibility conformance/rating of content and passing that into Crossref.

What was the key lesson you learned from this project?

DP: Incremental change can be just as challenging as a massive overhaul, and so it’s important to reevaluate your goals along the way—things always change. There have been cases where we were able to do things that we hadn’t initially thought were feasible.

DH: Always keep the larger goal in mind and remember that any project can birth a new project. Everything does not happen at once.

What’s your next big challenge for 2023?

DP: There is a lot to contend with in the industry right now, and in addition to that we are going through some serious infrastructure changes in our program. With all that madness comes many opportunities. For that reason, when I take a step back from the tactical implications of all that and what we are interested in doing, I think our biggest challenge in 2023 will be identifying what has made an impact and why.

DH: In the short-term, it is making sure that none of our production process changes has negatively affected the past metadata work we spent so much time honing. Once that settles down, it will be determining the best way forward from a publishing perspective in handling true versioning and capturing accurate event data.

Based on your experience, what would be your advice for colleagues from other scholarly publishing organisations?

DP: It can seem daunting, but the small wins can create momentum and do not have to be expensive. Remembering that your publishing program benefits as much as everyone else’s when you deposit more metadata can help refine your short-term and long-term priorities.

DH: Don’t be afraid of making a mess of things. Messes are okay. They aren’t risky. They just reveal the clutter. And clutter gives one reason to clean things up.

THANK YOU for the interview!

About the American Society for Microbiology

The American Society for Microbiology is one of the largest professional societies dedicated to the life sciences and is composed of 30,000 scientists and health practitioners. ASM’s mission is to promote and advance the microbial sciences.

ASM advances the microbial sciences through conferences, publications, certifications and educational opportunities. It enhances laboratory capacity around the globe through training and resources. It provides a network for scientists in academia, industry and clinical settings. Additionally, ASM promotes a deeper understanding of the microbial sciences to diverse audiences. For more information about ASM visit asm.org.

In the know on workflows: The metadata user working group

Jennifer Kemp — Tue, 28 Feb 2023 00:00:00 +0000

What’s in the metadata matters because it is So.Heavily.Used.

You might be tired of hearing me say it but that doesn’t make it any less true. Our open APIs now see over 1 billion queries per month. The metadata is ingested, displayed and redistributed by a vast, global array of systems and services that in whole or in part are often designed to point users to relevant content. It’s also heavily used by researchers, who author the content that is described in the metadata they analyze. It’s an interconnected supply chain of users large and small, occasional and entirely reliant on regular querying.

Tl;dr

Crossref recently wrapped up our first Working Group for users of the metadata, a group that plays a key role in discoverability and the metadata supply chain. You can jump directly to the stakeholder-specific recommendations or take a moment to share your use case or feedback.

Why a metadata user group? Why now?

A majority of Crossref metadata users rely on our free, open APIs and many are anonymous. A small but growing group of users pay for a guaranteed service level option and while their individual needs and feedback have long been integrated into Crossref’s work, as a group they provide a window into the workflows and use cases for the metadata of the scholarly record. As this use grows in strategic importance, to both Crossref and the wider community, it was clear that we might be overdue for a deeper dive into user workflows.

In 2021, we surveyed these subscribers for their feedback and brought together a few volunteers over a series of 5 calls to dig into a number of topics specific to regular users of metadata. This group, the first primarily non-member working group at Crossref, wrapped up in December 2022, and we are grateful for their time:

Achraf Azhar, Centre pour la Communication Scientifique Directe (CCSD)
Satam Choudhury, HighWire Press
Nees Jan van Eck, CWTS-Leiden University
Bethany Harris, Jisc
Ajay Kumar, Nova Techset
David Levy, Pubmill
Bruno Ohana, biologit
Michael Parkin, European Bioinformatics Institute (EMBL-EBI)
Axton Pitt, Litmaps
Dave Schott, Copyright Clearance Center (CCC)
Stephan Stahlschmidt, German Centre for Higher Education Research and Science Studies (DZHW)

This post is intended to summarize the work we did, to highlight the role of metadata users in research communications, to provide a few ideas for future efforts and, crucially, to get your feedback on the findings and recommendations. Though this particular group set out to meet for a limited time, we hope this report helps facilitate ongoing conversations with the user community.

Survey Highlights

If you’re looking for an easy overview of users and use cases, here’s a great starting point.

If you interpret this graphic to mean that there is a lot of variety centered on a few high level use cases, the survey and our experiences with users certainly supports that. A few key takeaways from the 2021 survey may be useful context:

Frequency of use: At least 60% of respondents query metadata on a daily basis
Use cases
- Finding and enhancing metadata as well as using it for general discovery are all common use cases
- For most users, matching DOIs and citations is a common need but for a significant group, it is their primary use case
- Analyzing the corpus for research was a consistent use case for 13% of respondents
Metadata of particular interest
- Abstracts are the most desirable non-bibliographic metadata, followed by affiliation information, including RORs
  - Some other elements (beyond citation information) that respondents find useful are:
    - Corrections and retractions
    - Relationship metadata
    - Book chapters
    - Grant information

NB: The survey did not ask about references but we are frequently asked why they’re not included more often.

It’s also worth noting that about a third of respondents said that correct metadata is more important to them than any particular element.

There is more to this survey that isn’t covered here but it was kept fairly short to help with the response rate. Knowing we would have some focused time to discuss issues too numerous or nuanced to reasonably address in a survey, we compiled a long list of questions and topics for the Working Group then followed up with a second, more detailed survey to kick off the meeting series.

What we set out to address

We had three primary goals for this Working Group:

Highlight the efforts of metadata users in enabling discovery and discoverability
Determine direction(s) for improved engagement
Inform the Crossref product development roadmap for metadata retrieval services

Of course, everyone involved had some questions and topics of interest to cover, including (but not limited to):

Understanding publisher workflows
How best to introduce changes, e.g. for a high volume of updated records
Understanding the Crossref schema
Query efficiencies, i.e. ‘tips and tricks’ (here for the REST API)
Which scripts, tools and/or programs are used in workflows
What other metadata sources are used
What kind of normalization or processing is done on ingest
How metadata errors are handled

What did we learn?

Workflows
I started with the admittedly ambitious goal of collecting a library of workflows. After a few years of working with users, I learned never to assume what a user was doing with the metadata, why or how. For example, some subscribers use Plus snapshots (a monthly set of all records), regularly or occasionally and some don’t use them at all. Understanding why users make the choices they do is always helpful.

In my experience, workflows are frequently characterized as “set it and forget it.” It’s hard to know how often and how easily they might be adapted when, for example, a new record type like peer review reports becomes available. So, it’s worth exploring when and how to highlight to users changes that might be of interest.

As it turned out, half the group had their workflows mostly or fully documented. The rest are partially documented, not documented at all or the availability of documentation was unknown. Helping users document their workflows, to the extent possible, should be a mutually beneficial effort to explore going forward. We’re doing similar work with the aim of making ours more transparent and replicable.

Feedback on subscriber services
User feedback might be the most obvious and directly consequential work of this group, at least for Crossref - understanding how well the services used meet their needs and what might be improved.

One frequent suggestion for improvement is faster response time on queries. This is an area we’ve focused on for some time, because refining queries to be more efficient is often the most straightforward way to improve response times and one reason for the emphasis on workflows.

We also discussed the possibility of whether or how to notify users of changes of interest. Just defining “change” is complex since they are so frequent and may often be considered very minor. We’ve been experimenting a bit over the past few years with notifying these users in cases where we’re aware of upcoming large volumes of changes, which is sometimes the case when landing page URLs are updated due to a platform change, for example. It was incredibly useful to discuss with the group what volume of records would be a useful threshold to trigger a notification (100K if you’re curious).

But perhaps the most common feedback we get from all users is on the metadata itself and the myriad quality issues involved. The group spent a fair amount of time discussing how this affects their work and shared a few examples of notable concerns:

Author name issues, e.g. ‘Anonymous’ is an option for authors but that or things like ‘n/a’ are sometimes used in surname fields
Invalid DOIs are sometimes found in reference lists
Garbled characters from text not rendering properly
Affiliation information is often not included or incomplete (e.g. doesn’t include RORs)
Inconsistencies in commonly included information, e.g. ISSNs

It’s worth noting that a common misunderstanding - not just among users - is what is required in the metadata. Users nearly always expect more metadata and more consistency than is actually available. The introduction of Participation Reports a few years ago was a very useful start to what is an ongoing discussion about the variable nature of metadata quality and completeness.

Users in the metadata supply chain
A few years ago, our colleague Joe Wass used Event Data to put together this chart of referrals from non-publisher sources in 2015.

The role of metadata users in discoverability of content is key in my view and one that often doesn’t get enough attention, especially given that the systems and services that use this information often use it to point their own users to relevant resources. And because they work so closely with the metadata, users frequently report errors and so serve as a sort of de facto quality control. So, unfortunately, the effects of incomplete or incorrect metadata on these users might be the most powerful way to highlight the need for more and better metadata.

What are the recommendations?

In discussions with the Working Group, a few themes emerged, largely around best practices, which, by their nature, tend to be aspirational.

If you’re not already familiar with the personas and Best Practices and Principles of Metadata 2020, that is a useful starting point (I am admittedly biased here!) and many are echoed in the following recommendations:

For users:

Document and periodically review workflows
Report errors to members or to Crossref support and reflect corrections when they’re made (metadata and content)
Understand what is and isn’t in the metadata
Follow best practices for using APIs

For Crossref:

Define a set of metadata changes, e.g. to affiliations, to further the discussion around thresholds for notifying users of ‘high volumes’ of changes
Provide an output schema.
Continue refining the input schema to include information like preprint server name, journal article sub types (research article, review article, letter, editorial, etc.), corresponding author flags, raw funding statement texts, provenance information, etc.
Collaborate on improving processes for reporting metadata errors and making corrections and enhancements

For metadata providers (publishers, funders and their service providers):

Follow Metadata 2020 Metadata Principles and Practices
Consistency is important, e.g. using the same, correct relationship for preprint to VoR links for all records
- Workarounds such as putting information into a field that is ‘close’ but not meant for it can be considered a kind of error
Understand the roles and needs of users in amplifying your outputs
Respond promptly to reports of metadata errors
Whenever possible, provide PIDs (ORCID IDs, ROR IDs, etc.) in addition to (not as a substitute for) textual metadata

What is still unclear or unfinished?

Honestly, a lot. We knew from the outset that the group would conclude with much more work to be done, in part because there is so much variety under the umbrella of metadata users and many answers lead to more questions and in part because the metadata and the user community will continue to evolve. Even without a standing group that meets regularly, it’s very much an ongoing conversation and we invite you to join it.

Now it’s your turn–can you help fill in the blanks?

Does any or all of this resonate with you? Do you take exception to any of it? Do you have suggestions for continuing the conversation?

Specifically, can you help fill in any of the literal blanks? We’ve prepared a short survey that we hope can serve as a template for collecting (anonymous) workflows. Please take just a few minutes to answer a few short questions such as how often you query for metadata.

If you are willing to share examples of your queries or have questions or further comments, please get in touch.

Perspectives: Mohamad Mostafa on scholarly communications in UAE

Mohamad Mostafa — Mon, 27 Feb 2023 00:00:00 +0000

تسلط سلسلة مدونة توقعات - وجهات نظر الخاصة بنا الضوء على أعضاء مختلفين من مجتمعنا العالمي المتنوع في كروس رف .نتعلم المزيد عن حياتهم وكيف تعرفوا وعملوا معنا، ونسمع رؤى حول مشهد البحث العلمي في بلدهم، والتحديات التي يواجهونها، وخططهم للمستقبل.

As we continue with our Perspectives blog series, today, we meet Mohamad Mostafa, Crossref Ambassador in the UAE and Production Manager at Knowledge E. Mohamad is passionate about helping improve the discoverability of research through rich metadata. We invite you to read and listen to what Mohamad has to say!

بينما نواصل سلسلة مدونة توقعات - وجهات نظر الخاصة بنا، نلتقي اليوم مع محمد مصطفى، سفير كروس رف في الإمارات العربية المتحدة ومدير الإنتاج في نوليدج اي . محمد متحمس للمساعدة في تحسين إمكانية اكتشاف البحث من خلال البيانات الوصفية الغنية. ندعوكم لقراءة ما يقوله محمد والاستماع إليه!

English عربي

Tell us a bit about your organisation, your objectives, and your role

أخبرنا قليلاً عن مؤسستك وأهدافك ودورك

My name is Mohamad Mostafa, and I am the Production Manager at Knowledge E. Within our publishing program, we publish around 2000 articles across 13 titles that are fully Open Access, which is something that I really value.

اسمي محمد مصطفى، وأنا مدير الإنتاج في نولدج إي. ضمن برنامج النشر الخاص بنا، ننشر حوالي 2000 مقالة عبر 13 عنوانًا مفتوح الوصول بالكامل، وهو أمر أقدره حقًا.

In a world that’s moving faster than ever, the availability, quality, and pursuit of knowledge are fundamental for advancement. Knowledge E, in line with its vision of developing a more knowledgeable world, helps institutions advance the quality of their research; move towards teaching excellence; upgrade library technology, services, and practices; and advance scholarship through journal publication, management, and training. In other words, it works with higher education institutions, research centres, ministries, publishers, and scholars to solve our society’s most significant challenges.

في عالم يتحرك بشكل أسرع من أي وقت مضى، يعد توافر المعرفة وجودتها والسعي وراءها أمورًا أساسية للتقدم. إن نوليدج إي، تماشياً مع رؤيتها لتطوير عالم أكثر معرفة ودراية، تساعد المؤسسات على تحسين جودة أبحاثها؛ التحرك نحو التميز في التدريس؛ ترقية مكتباتها الرقمية والخدمات والممارسات المتعلقة بها؛ ودعم المنح الدراسية المتقدمة من خلال نشر المجلات وإدارتها والتدريب. بمعنى آخر، تعمل شركة نولدج إي مع مؤسسات التعليم العالي ومراكز البحث والوزارات والناشرين والعلماء لحل أهم التحديات التي تواجه مجتمعنا.

I am also a Crossref Ambassador. As part of the ambassador program, we aim to raise awareness about Crossref services among librarians, publishers, editors, and authors in the Middle East and North Africa region. As part of this, we run workshops in English and Arabic, emphasizing the importance of comprehensive metadata and persistent identifiers. We also help research communities improve their understanding of how to use Crossref services. The importance of making regional research objects easy to find, cite and reuse encouraged me to join the ambassador program.

أنا أيضًا سفير كروس رف. كجزء من برنامج السفراء، نهدف إلى زيادة الوعي حول خدمات Crossref بين أمناء المكتبات والناشرين والمحررين والمؤلفين في منطقة الشرق الأوسط وشمال إفريقيا. وكجزء من هذا، فإننا ندير ورش عمل باللغتين الإنجليزية والعربية، للتأكيد على أهمية البيانات الوصفية الشاملة والمعرفات المستمرة. نحن أيضًا نساعد مجتمعات البحث على تحسين فهمهم لكيفية استخدام خدمات .Crossref شجعتني أهمية تسهيل العثور على عناصر البحث الإقليمية والاستشهاد بها وإعادة استخدامها على الانضمام إلى برنامج سفراء كروس رف.

What is one thing that others should know about your country and its research activity?

ما هو الشيء الذي يجب أن يعرفه الآخرون عن بلدك ونشاطه البحثي؟

A lot of regional research is being produced (in Arabic) and even without proper infrastructure (the lack of language support within the international publishing ecosystems such as peer review systems, indexes, citations databases, submissions systems, etc.) and the inadequate awareness about the various services (such as Crossref solutions) that can help with the discoverability and visibility of this research, the Arab region is increasingly recognised as a global leader in research outputs. Generally, these are some of the challenges and frustrations associated with the MENA (Middle East/North Africa) region.

يتم إنتاج الكثير من الأبحاث الإقليمية (باللغة العربية) وحتى بدون بنية تحتية مناسبة (نقص الدعم اللغوي داخل أنظمة النشر الدولية مثل أنظمة مراجعة الأقران، والفهارس، وقواعد بيانات الاستشهادات، وأنظمة التقديم، وما إلى ذلك) وعدم كفاية الوعي حول الخدمات المختلفة (مثل حلول(Crossref التي يمكن أن تساعد في اكتشاف هذه البحوث وإبرازها، يتم الاعتراف بالمنطقة العربية بشكل متزايد كرائد عالمي في مخرجات البحث. بشكل عام، هذه بعض التحديات والإحباطات المرتبطة بمنطقة الشرق الأوسط وشمال إفريقيا.

Are there trends in scholarly communications that are unique to your part of the world?

هل توجد اتجاهات في الاتصالات العلمية فريدة من نوعها في الجزء الذي تعيش فيه من العالم؟

In general, Open Access and Open Research are getting more and more attention in our region currently. We have recently launched the Forum for Open Research in MENA to raise awareness about all the new scholarly communications trends and support the Middle East and North Africa movement towards Open Science.

بشكل عام، يحظى الوصول الحر والبحث المفتوح باهتمام متزايد في منطقتنا حاليًا. لقد أطلقنا مؤخرًا منتدى الأبحاث المفتوحة في منطقة الشرق الأوسط وشمال إفريقيا لزيادة الوعي حول الاتصالات العلمية الجديدة ودعم حركة الشرق الأوسط وشمال إفريقيا نحو العلوم المفتوحة.

The Forum for Open Research in MENA (FORM) is a non-profit membership organisation supporting the advancement of open science policies and practices in research communities and institutions across the Arab world.

منتدى البحوث المفتوحة في الشرق الأوسط وشمال إفريقيا (FORM) هو منظمة غير ربحية ذات عضوية تدعم النهوض بسياسات وممارسات العلوم المفتوحة في المجتمعات والمؤسسات البحثية في جميع أنحاء العالم العربي.

We believe the Arab world has the resources and capability to play a pivotal role in the global transition towards more accessible, sustainable, and inclusive research and education models. And we want to support all our research communities and stakeholder groups in the journey towards a more ‘open’ world. Our vision is to help unlock research for and in the Arab world. Our mission is to support the advancement of open science practices in research libraries and universities across the Arab world by facilitating the exchange of actionable insights and developing practical policies.

نعتقد أن العالم العربي لديه الموارد والقدرة على لعب دور محوري في التحول العالمي نحو نماذج بحث وتعليم أكثر سهولة واستدامة وشمولية. ونريد دعم جميع مجتمعاتنا البحثية ومجموعات أصحاب المصلحة في رحلتنا نحو عالم أكثر "انفتاحًا". رؤيتنا هي دعم الوصول الحر والبحوث المفتوحة في العالم العربي. ومهمتنا هي دعم تقدم ممارسات العلوم المفتوحة في مكتبات البحث والجامعات في جميع أنحاء العالم العربي من خلال تسهيل تبادل الأفكار القابلة للتنفيذ وتطوير السياسات العملية.

Our first Annual Forum was held in Cairo in October 2022 (as part of the global Open Access Week initiative). The event was a huge success, with over 1,100 delegates from over 48 countries across the globe. The next Annual Forum will be hosted in the UAE in October 2023, and details will be available shortly on our website.

عقد المنتدى السنوي الأول في القاهرة في أكتوبر 2022 (كجزء من مبادرة أسبوع الوصول الحر العالمي). حقق الحدث نجاحًا كبيرًا، حيث حضره أكثر من 1100 مندوب من أكثر من 48 دولة حول العالم. سيتم استضافة المنتدى السنوي القادم في دولة الإمارات العربية المتحدة في أكتوبر 2023، وستتوفر التفاصيل قريبًا على موقعنا.

How would you describe the value of being part of the Crossref community; what impact has your participation had on your goals?

كيف تصف قيمة أن تكون جزءًا من مجتمعCrossref ؟ ما هو تأثير مشاركتك على أهدافك؟

I have been a Crossref ambassador for more than 5 years now, and I can really say that it has been a great experience being part of such an amazing and collaborative community. We got the chance to interact with different publishers and service providers and participate in different Crossref annual events. It’s also perfectly aligned with our vision of supporting Open Research.

لقد كنت سفيرًا لـ Crossref لأكثر من 5 سنوات حتى الآن، ويمكنني حقًا أن أقول إنها كانت تجربة رائعة أن أكون جزءًا من هذا المجتمع المذهل والتعاوني. لقد أتيحت لنا الفرصة للتفاعل مع مختلف الناشرين ومقدمي الخدمات والمشاركة في الأحداث السنوية المختلفة لـ .Crossref كما أنه يتماشى تمامًا مع رؤيتنا لدعم البحث المفتوح.

Recently, we have delivered a series of three Arabic webinars that offered basic metadata information and advanced insights about the role of metadata and how Crossref services can help an institution. These webinars have been well received by the community of regional publishers, university presses, and librarians. Dozens of questions have been answered, and technical enquires have been resolved. It was a great experience, and it was good to see that kind of interest in our community. Also, more educational webinars are yet to come!

قدمنا مؤخرًا سلسلة من ثلاث ندوات عربية عبر الإنترنت تمحورت حول معلومات البيانات الوصفية الأساسية ورؤى متقدمة حول دور البيانات الوصفية وكيف يمكن لخدمات Crossref أن تساعد المؤسسات البحثية. لقيت هذه الندوات عبر الإنترنت استحسان مجتمع الناشرين الإقليميين دور النشر الجامعية وأمناء المكتبات. تمت الإجابة على عشرات الأسئلة، وتم الرد على الاستفسارات الفنية. لقد كانت تجربة رائعة، وكان من المفرح أن نرى هذا النوع من الاهتمام في مجتمعنا. بالإضافة إلى ذلك، سيتم تقديم المزيد من الندوات التعليمية على الإنترنت في المستقبل.

For you, what would be the most important thing Crossref could change (do more of/do better in)?

بالنسبة لك، ما هو الشيء الأكثر أهمية الذي يمكن لـ Crossref تغييره (القيام بالمزيد / القيام بعمل أفضل في)؟

Language is still a barrier in some parts of the Arab region, so producing more educational content in different formats (webinars, flyers, videos with subtitles, etc.) would be highly appreciated here. 

لا تزال اللغة تشكل حاجزًا في بعض المناطق العربية، لذا سيكون إنتاج المزيد من المحتوى التعليمي بتنسيقات مختلفة (ندوات عبر الإنترنت، ونشرات، ومقاطع فيديو مع ترجمة، وما إلى ذلك) موضع تقدير كبير هنا.

Which other organisations do you collaborate with or are pivotal to your work in open scholarship?

ما هي المنظمات الأخرى التي تتعاون معها أو التي تلعب دورًا محوريًا في عملك في مجال الابحاث المفتوحة؟

We work closely with ORCiD and invite them to our events, support DOAJ via our charitable Foundation, and rely heavily on PKP products mainly the Open Journal Systems (OJS) with plans to expand and start using Open Monograph Press (OMP).

إننا نعمل عن كثب مع ORCiD ونقدر دعمهم لفاعلياتنا، كما ندعم DOAJ عبر موقعنا ومؤسستنا الخيرية، ونعتمد بشكل كبير على منتجات مشروع المعرفة العامة وخاصة المجلة المفتوحة أنظمة (OJS) كما أننا نود التوسع والبدء في استخدام Open Monograph Press (OMP).

What are the post-pandemic challenges/hopes you are facing and how are you adapting to them/what you’re looking forward to?

ما هي التحديات / الآمال التي تواجهها في فترة ما بعد الجائحة وكيف تتكيف معها / ما الذي تتطلع إليه؟

We aim for more face-to-face meetings and onsite workshops/conferences as the world opens up again. In addition, we have launched the Forum for Open Research in MENA (FORM) (a non-profit membership organisation supporting the advancement of Open Science policies and practices in research communities and institutions across the Arab region.)

نحن نهدف إلى المزيد من الاجتماعات وجهًا لوجه وورش العمل / المؤتمرات. بالإضافة إلى ذلك، أطلقنا منتدى البحث المفتوحMENA (FORM) ، وهي منظمة غير ربحية ذات عضوية تدعم النهوض بسياسات وممارسات العلوم المفتوحة في مجتمعات ومؤسسات البحث في جميع أنحاء المنطقة العربية.

A catalyst for positive action, we work with key stakeholders to develop and implement a pragmatic programme to facilitate the transition toward more accessible, inclusive, and sustainable research and education models in the Arab region. Our driving focus is on building the resources, the membership, the organisational structures, and the broader community to support the advancement of Open Science in research communities and research institutions across the Arab world.

كمحفز للعمل الإيجابي، نحن نعمل مع أصحاب المصلحة الرئيسيين للتطوير وتنفيذ برنامج عملي لتسهيل الانتقال نحو المزيد من نماذج البحث والتعليم الشاملة والمستدامة والتي يسهل الوصول إليها في المنطقة العربية. ينصب تركيزنا الدافع على بناء الموارد، والعضوية، والهياكل التنظيمية، والمجتمع الأوسع لدعم تقدم العلوم المفتوحة في المجتمعات البحثية والمؤسسات البحثية عبر العالم العربي.

Following the huge success of our 2022 Annual Forum (held in Cairo with the support and endorsement of UNESCO and the Egyptian Knowledge Bank), which attracted over 1100 delegates from 48 countries, our 2023 Annual Forum will be held in Abu Dhabi in the UAE. For more details about the event and the call for papers, see our website: https://forumforopenresearch.com

بعد النجاح الكبير لمنتدى 2022 السنوي (الذي عقد في القاهرة مع دعم وتأييد اليونسكو وبنك المعرفة المصري)، التي اجتذبت أكثر من 1100 مندوب من 48 دولة، المنتدى السنوي لعام 2023 سيعقد في أبو ظبي في دولة الإمارات العربية المتحدة. لمزيد من التفاصيل حول الحدث والدعوة للمشاركة، راجع موقعنا على الإنترنت:https://forumforopenresearch.com

What are your plans for the future?

ما هي خططك المستقبلية؟

Keep working with different global and regional stakeholders to help the transition of our region towards Open Science.

استمر في العمل مع مختلف الشركاء العالميين والإقليميين للمساعدة في انتقال منطقتنا العربية نحو العلوم المفتوحة.

Thank you, Mohamad!

شكرا لك يا محمد!

The more the merrier, or how more registered grants means more relationships with outputs

Dominika Tkaczyk — Wed, 22 Feb 2023 00:00:00 +0000

One of the main motivators for funders registering grants with Crossref is to simplify the process of research reporting with more automatic matching of research outputs to specific awards. In March 2022, we developed a simple approach for linking grants to research outputs and analysed how many such relationships could be established. In January 2023, we repeated this analysis to see how the situation changed within ten months. Interested? Read on!

TL;DR

The overall numbers changed a lot between March 2022 and January 2023:
- the total number of registered grants doubled (from ~38k to ~76k)
- the total numbers of relationships established between grants and research outputs quadrupled (from 21k to 92k)
- the percentage of linked grants increased substantially (from 10% to 23%)
Most of this growth can be attributed to one funder, the European Union. They started registering grants with us in December 2022, and:
- their grants constitute 47% of all grants registered by January 2023 and 95% of grants registered between March 2022 and January 2023
- 72% of all established relationships involve their grants
We have further work planned both internally and with the community to consolidate and build out important relationships between funding and research outputs.

Introduction

When we started to develop, think and talk about grant registration at Crossref back in 2017, one of the key things we expected this to support was easier, more efficient, accurate analysis of research outputs funded by specific awards.

This is backed up by conversations with funders who are keen to fill in gaps in the map of the research landscape with new data points and better quality information, search for grants, investigators, projects or organisations associated with awards and simplify the process of research reporting and with automatic matching of outputs to grants.

This is in keeping with and informed our recent recommendations about how funding agencies can meet open science guidance using existing open infrastructure, which included input from ORCID and DataCite. It’s also in keeping with recent studies on how important funding and grant metadata is to help the community use this information in their own research.

To meet these expectations, we need not only identifiers and metadata of grants, but also relationships between them and research outputs supported by them. Unfortunately, our schema does not make it easy to directly deposit such relationships, and so there are only a handful of them available. But we wouldn’t let such a minor obstacle stop us! In March 2022 we analysed the metadata of registered grants and developed a simple matching approach to automatically link grants to research outputs supported by them. Back then, we were able to find 20,834 relationships, involving 17,082 research outputs and 3,858 grants (which was 10% of all registered grants).

Now that we are seeing the accumulation of grant metadata being registered with Crossref, we have a bigger dataset to test these expectations against than we did a year ago. So we decided to do the analysis again. And the results are in, they’re open, and they’re positive. We’ll explain below.

The methodology

To spare you from having to read the old analysis in detail, here is a very brief summary of the matching methodology. To find relationships between grants and research outputs, we iterated over all registered grants, and for each grant we searched for research outputs that looked like they might have been supported by this grant. We established a relationship between a grant and a research output if one of the following three scenarios was true:

The research output contained the DOI of the grant (deposited as the award number).
The award number in the grant was the same as the award number in the research output, the research output contained the funder ID, and one of the following was true:
a. Funder ID in the grant was the same as the funder ID in the research output
b. Funder ID in the grant replaced or was replaced by the funder ID in the research output
c. Funder ID in the grant was an ancestor or the descendant of the funder ID in the research output
The award number in the grant was the same as the award number in the research output, the research output did not contain the funder ID, and one of the following was true:
a. Funder name in the research output was the same as the funder name in the grant
b. Funder name in the research output was the same as the name of a funder that replaced or was replaced by the funder in the grant
c. Funder name in the research output was the same as the name of an ancestor or a descendant of the funder in the grant

Note that the replaced/replaced-by relationships and ancestor/descendant hierarchy are taken from the Funder Registry.

Current results

Since March 2022, six additional funders have started registering grants with us. As a result, the total number of grants doubled, and the total number of established relationships between grants and research outputs, linked grants, and linked research outputs quadrupled. Here is the comparison of the total numbers of grants, established relationships, linked grants, and linked research outputs in March 2022 and in January 2023:

95% of grants registered within ten months between March 2022 and January 2023 were registered by one funder: the European Union. This suggests that this funder contributed a lot to this rapid increase in the number of established relationships. It looks like this funder’s grant metadata is of high quality and matches well the funding information given in the research outputs supported by this funder’s grants.

Let’s also compare the breakdowns of all established relationships by the matching method:

The distributions are a bit different. Currently, the percentage of relationships established based on the replaced/replaced-by relationship is much smaller than before, suggesting that newer data uses correct funder IDs instead of deprecated ones. Also, the percentage of the relationships matched by the funder ID increased from 40% to 48%, which is great, because this is the most reliable way of matching.

And here we have the statistics broken down by grant registrants. Only funders with at least 100 registered grants are included. The table shows the number of relationships, grants, linked grants, and linked research outputs, and is sorted by the percentage of linked grants.

funder	relationships	linked research outputs	grants	linked grants
European Union	66,562	60,630	35,530	12,688 (36%)
Gordon and Betty Moore Foundation	93	92	113	33 (29%)
Japan Science and Technology Agency (JST)	15,584	13,464	9,923	2,323 (23%)
James S. McDonnell Foundation	519	513	577	121 (21%)
Melanoma Research Alliance	188	185	425	82 (19%)
Muscular Dystrophy Association	50	50	178	25 (14%)
Parkinson’s Foundation	30	29	107	15 (14%)
Asia-Pacific Network for Global Change Research	127	127	560	70 (13%)
The ALS Association	96	90	477	58 (12%)
Wellcome	8,868	6,436	17,537	1,735 (10%)
American Cancer Society	19	19	266	15 (6%)
Templeton World Charity organisation	2	2	281	2 (0.7%)
Office of Scientific and Technical Information (OSTI)	73	69	8,723	62 (0.7%)
Children’s Tumor Foundation	1	1	662	1 (0.1%)

There are substantial differences between the percentages of linked grants from different funders. One of the newest registrants, the European Union, is at the top of the table with 36% of their grants linked to research outputs. This further confirms the high quality of the metadata registered by this member. It is worth noticing that this member is responsible for the majority of the growth reported here as they cover Horizon Europe, the European Research Council, and many other funding bodies and schemes.

Why are these percentages so low for some funders? It could be caused by systematic discrepancies between the award numbers attached to the grants and those reported in research outputs. It could also be the case that most grants registered by a given funder are new grants, and the research outputs supported by them simply have not been published yet. Time will tell!

What’s next

We’re dedicating lots of time in 2023 to examine, evolve, and expose the matching we do and can do at Crossref across different metadata fields. We then plan to incorporate matching improvements into our services so that everyone can benefit.

This isn’t a standalone piece of work. As you can see, the more award metadata we have connected to grants by funders and connected to outputs by those who post or publish research, the better we’ll be able to do this. To make it easier for more funders to participate, and based on funder feedback, we’ve built a simple tool for members to register their grants. We will also work to help incorporate grant identifiers into publishing and funder workflows, and further our discussions with the funders in our Funder Advisory Group and the wider community, including working together with the Open Research Funders Group, the HRA, Altum, Europe PMC, the OSTP, and the ORCID Funder Interest Group. And there will be more to come as we work together to consolidate and build out important relationships between funding and outputs - for everyone.

Follow-up

Every new thing takes time to get off the ground and to show evidence of its value. We’ve seen a significant step forward recently with funders joining and contributing to the research nexus. Publishers have been contributing funding data for years, and it’s now becoming much clearer to see how these two communities and these two sets of metadata are coming together to make research smoother and easier to manage and evaluate. If you are ready to register grants, talk about linking up your outputs, or just want to learn more about this work, we’d love to hear from you.

Don't take it from us: Funder metadata matters

Jennifer Kemp — Thu, 16 Feb 2023 00:00:00 +0000

Why the focus on funding information?

We are often asked who uses Crossref metadata and for what. One common use case is researchers in bibliometrics and scientometrics (among other fields) doing meta analyses on the entire corpus of records. As we pass the 10 year mark for the Funder Registry and 5 years of funders joining Crossref as members to register their grants, it’s worth a look at some recent research that focuses specifically on funding information. After all, there is funding behind so much scholarly work it seems obvious that it would be routinely documented in the scholarly record. But it often isn’t and that’s a problem. These sources make clear the need for accurate funding information and the problems that the lack of it creates.

First, a few notes for context on these sources and the issues they discuss :

The percent of records with funding information reached about 25% as of 2021. Not all items registered are the result of funding but surely it is much higher than 25% so there is considerable room for improvement. The authors cite publishers that omit funding information as well as those that include it routinely. Overall, society publishers are at the top of the list of those that do it well.
Three of the four sources found problems in some cases confirming funding information from the metadata in the original sources. This initially surprised me though less so once I thought about the strange nature of metadata workflows.
The complexity of fully and correctly acknowledging multiple sources of funding in any given publication is a recurring theme.
All of the sources mention the need for manual work in analyzing funding and publication information.

The first two papers are from the same 2022 issue of Quantitative Science Studies and are complementary.

Alexis-Michel Mugabushaka, Nees Jan van Eck, Ludo Waltman; Funding COVID-19 research: Insights from an exploratory analysis using open data infrastructures. Quantitative Science Studies 2022; 3 (3): 560–582. doi: https://doi-org.pluma.sjfc.edu/10.1162/qss_a_00212

This first paper tackles the timely question of determining which funders have supported publications of COVID-19 research and compares coverage of funding data in Crossref to that in Scopus and Web of Science. Even with so much urgent attention focused on the pandemic, the authors found that only 17% of publications in the COVID-focused CORD-19 database have funding identified in their Crossref records. We’re often asked about differences in the metadata (and citation counts) between Crossref and other sources such as Scopus. In this case, both proprietary sources studied have more funder coverage. If you are disappointed in these results or want to learn more, I encourage you to read the authors’ recommendations for improving funding data in Crossref or get in touch with us.

Bianca Kramer, Hans de Jonge; The availability and completeness of open funder metadata: Case study for publications funded by the Dutch Research Council. Quantitative Science Studies 2022; 3 (3): 583–599. doi: https://doi-org.pluma.sjfc.edu/10.1162/qss_a_00210

This next paper focuses on a set of outputs funded by the NWO (the Dutch Research Council). Since the funder is already known, the authors could look at multiple sources (Crossref and others) to see whether or where the NWO is correctly identified as the funder. This study also found better coverage than Crossref in proprietary sources like Web of Science. Knowing that not all outputs are the result of funded research, this paper provides a new and useful baseline for comparing percentages of coverage. Discussions of research funding so often focus on the physical and life sciences so it’s very good to see that 37% of works in this study are in the humanities and social sciences.

Borst, T., Mielck, J., Nannt, M., Riese, W. (2022). Extracting Funder Information from Scientific Papers - Experiences with Question Answering. In: , et al. Linking Theory and Practice of Digital Libraries. TPDL 2022. Lecture Notes in Computer Science, vol 13541. Springer, Cham. https://doi-org.pluma.sjfc.edu/10.1007/978-3-031-16802-4_24

Given the considerable effort required to conduct these analyses, it’s only logical to consider automating as much of the work as possible. This next paper focuses on automatic recognition of funders in economics papers in digital libraries. An interesting complication described here is the inclusion of funding for open access fees in acknowledgments and while the authors conclude that automated text mining of funder information performs better than manual curation, they also state that manual indexing is still necessary “for a gold standard of reliable metadata.”

Habermann, T. (2022). Funder Metadata: Identifiers and Award Numbers. https://metadatagamechangers.com/blog/2022/2/2/funder-metadata-identifiers-and-award-numbers

Finally, this concise blog post looks at RORs as well as funder names and acronyms. The author shows how acronyms contribute to the need for manual analysis. He also spends some time on award numbers, which is one of the three funding elements publishers can (and, as we’ve seen, should) include in their metadata. Award numbers are also a focus of this work and, unfortunately, another frequent reason for additional manual work.

A common theme: More metadata needed

Though collectively, this research paints a fairly dim picture of the current availability, completeness and accuracy of existing funding information in publication metadata, all is not lost. This is a good opportunity to point out the value and availability of grant records since unique, persistent identifiers for grants (yes, DOIs for grants) paired with more and better funding metadata from publishers go a very long way to realizing the vision of the Research Nexus. And it certainly would make things a whole lot easier for the researchers who use this open metadata to analyze the scholarly record for the rest of us.

Refocusing our Sponsors Program; a call for new Sponsors in specific countries

Susan Collins — Mon, 06 Feb 2023 00:00:00 +0000

Some small organisations who want to register metadata for their research and participate in Crossref are not able to do so due to financial, technical, or language barriers. To attempt to reduce these barriers we have developed several programs to help facilitate membership. One of the most significant—and successful—has been our Sponsor program.

Sponsors are organisations that are generally not producing scholarly content themselves but work with or publish on behalf of groups of smaller organisations that wish to join Crossref but face barriers to do so independently. Sponsors work directly with Crossref in order to provide billing, technical, and, if applicable, language support to Members.

Because Sponsors are important partners in facilitating membership there is a high bar to meet to be accepted as a Sponsor. To ensure that an organisation can accurately represent Crossref and has the resources to be successful we created a set of criteria that must be met to be considered.

Our Sponsors program has grown considerably over the last decade and has now become the primary route to membership for emerging markets and small or academic-adjacent publishing operations.

The program began in 2012 with four Sponsors, based primarily in South Korea and Turkey, representing fewer than 100 members. In the next stage of development, the program covered Brazil, India, and Ukraine, and nearly 1300 members. At the end of 2022, the program had grown to over 100 sponsors from 45 countries representing over 11,000 of our members.

Though the program continues to expand, there are still regions where we lack Sponsors, while having an abundance in others. We are working with members, ambassadors, and the community to help identify organisations that may be a fit with the Sponsor program and based in those regions where coverage is lacking.

This January we announced our Global Equitable Membership (GEM) Program which offers relief from membership and content registration fees for members in the least economically-advantaged countries in the world. Eligibility for the program is based on a member’s country on our curated list.

Though the GEM program reduces financial barriers to becoming a member, many organisations still require technical assistance and local language support. Working with a Sponsor would help organisations overcome these burdens. However, there is little or no Sponsor coverage for organisations located in most GEM-eligible countries. That means that in places like Bangladesh, Nepal, and Senegal, where we’ve seen a lot of growth, more organisations could join us if a suitable local Sponsor could support them.

We have made the decision to pause accepting new Sponsors from regions where Sponsor numbers are already very high or not based in a GEM region. By doing so we can focus on growing the program in areas where there is the greatest need.

We are also going to focus on how best to support our current 100+ Sponsors and work with them to evaluate ways to improve the program. We will bolster the training and resources, outreach activities, and solicit feedback on additional ways we can help.

We would love to hear from organisations based in GEM countries who might consider becoming a Sponsor. But our invitation for Sponsors is not limited to the support for the GEM program. There are countries where the GEM program won’t apply, but where growth is high and no Sponsor is present. In particular, we seek support in the following countries where member numbers are growing but could be better supported.

Country/state	Region	No. Crossref members
Nigeria	Sub-Saharan Africa (Western)	99
Philippines	South-eastern Asia	81
Kenya	Sub-Saharan Africa (Eastern)	40
Egypt	Northern Africa	26
Sri Lanka	Southern Asia	13

If your organisation is based in one of these regions and supports or provides services to scholarly publishers in one of the above countries —please take a look at the criteria set out on our website and do get in touch to start the conversation if you think you can meet them. We’re excited to hear from you!

Measuring Metadata Impacts: Books Discoverability in Google Scholar

Lettie Conrad — Wed, 25 Jan 2023 00:00:00 +0000

This blog post is from Lettie Conrad and Michelle Urberg, cross-posted from the The Scholarly Kitchen.
As sponsors of this project, we at Crossref are excited to see this work shared out.

The scholarly publishing community talks a LOT about metadata and the need for high-quality, interoperable, and machine-readable descriptors of the content we disseminate. However, as we’ve reflected on previously in the Kitchen, despite well-established information standards (e.g., persistent identifiers), our industry lacks a shared framework to measure the value and impact of the metadata we produce.

In 2021, we embarked on a Crossref-sponsored study designed to measure how metadata impacts end-user experiences and contributes to the successful discovery of academic and research literature via the mainstream web. Specifically, we set out to learn if scholarly books with DOIs (and associated metadata) were more easily found in Google Scholar than those without DOIs.

Initial results indicated that DOIs have an indirect influence on the discoverability of scholarly books in Google Scholar – however, we found no direct linkage between book DOIs and the quality of Google Scholar indexing or users’ ability to access the full text via search-result links. Although Google Scholar claims to not use DOI metadata in its search index, the results of our mixed-methods study of 100+ books (from 20 publishers) demonstrate that books with DOIs are generally more discoverable than those without DOIs.

As we finalize our analysis, we are sharing some early results and inviting input from our community. What relevant lessons can we glean from this exercise? What changes might book publishers consider based on the outcomes of this study?

Background on the study

This study was designed to evaluate metadata impacts & benefits to users. Given its popularity with a range of stakeholders in our industry, we set out to measure metadata impacts on discoverability in the mainstream web – namely, Google Scholar.

Our test method and analysis rubric was developed based on our own information-user research, in particular how readers search and retrieve scholarly ebooks, as well as published studies about academic information experiences and research practices. We rated the search performance of more than 100 scholarly books using preset test queries (two for each title). The books tested in this study came from publishers of all sorts and sizes, and represent both monographs and edited volumes from a range of fields; some were open access and others were published under traditional licensing models.

We developed and executed known-item test searches that were designed to simulate common researcher practices. Heuristic analysis of the search results was used to rate the search performance on a 5-point scoring rubric, which was designed to measure the degree of friction in locating the book in question. This method allowed us to assess specific book and metadata attributes by their search performance scores to assess the impact of book metadata on content discoverability in Google Scholar.

Results and findings

In this study, we learned that high-value fields include the primary title paired with subtitles, author/editor surnames and/or field of study. Queries using full book titles performed the best across the board. Those using publication dates and/or author/editor surnames and/or publisher names, but without the book title, were the lowest performers.

Surprisingly, our discoverability scores show no significant variation in performance by the type of book, whether edited or authored. Open-access titles performed somewhat better than traditional ones. Books covering humanities and social science fields performed a bit better than STM books, but only by a slim difference (that is not statistically significant).

We primarily tested the discoverability of book titles, from equal numbers of books with and without chapter-level DOIs. We ran similar tests for chapter-title discoverability but found the majority of test queries for chapters lead users to the full book itself. While books without title-level DOIs were found to be less discoverable, we did not find a measurable difference between books with or without chapter-level DOIs. (Note: All books in this study with chapter-level DOIs assigned also carried a title-level DOI, which was found to be fairly common.)

Based on these results, we are developing a theory that books with DOIs perform better in Google Scholar because they benefit from the structured, open metadata associated with those DOIs – which are used by hundreds of platforms and services, and therefore are “seeded” throughout the mainstream web, which Scholar may draw on for indexing, linking, etc. That said, however, these results also suggest that publishers are best served by a metadata strategy that is well attuned to the protocols expected of each channel for book search and discovery. In a recent conversation about our findings, Anurag Acharya himself noted that these results underscore the need for publishers to invest in the robust construction and broad distribution of book metadata.

In this study, we have observed that the metadata protocols surrounding Google Scholar are not fully integrated into our industry’s established scholarly information standards bodies, like NISO, or infrastructure organisations, like Crossref. While some mainstream data standards prevail in the Scholar index, like the use of schema.org and HTTP, some key metadata attributes seem to be lacking. For example, an indicator of the type of scholarly book (monograph, handbook, etc.) would improve Google Scholar’s search index and could be used to filter search results, thereby improving users’ experiences discovering scholarly books. One clear challenge for book publishers today is the fact that Google Scholar operates outside of our community-governed scholarly information infrastructure.

What comes next

While this study focused on Google Scholar, the results and lessons learned are applicable to other mainstream channels of information seeking/discovery. Our report, due out spring 2023, will contribute to the literature intended to support user-centric information systems design and content architecture by scholarly publishers and service providers.

As we write up our findings, we intend to develop a framework that can help publishers and others measure the impact of their work to enrich and distribute scholarly metadata. We hope this first systematic review of the impacts of metadata on the discoverability of books in Google Scholar will provide valuable insights for this community. In the meantime, please share your thoughts and questions in the comments below – or reach out to us directly (see Lettie’s profile here and Michelle’s profile here).

Acknowledgments: The authors would like to thank Jennifer Kemp at Crossref for the inspiration to take this dive into the metadata literature and reflect on its impact on research information experiences. Special thanks to Anurag Acharya at Google Scholar for his consultation during this study.