DOIs on Crossref

Collaboration with Knowledge Futures to build support for high-volume DOI registration

Travis Rich — Mon, 08 Jun 2026 00:00:00 +0000

Cross-posted from the Knowledge Futures blog.

For many years, PubPub has made it possible for communities to assign DOIs to a range of outputs and component Pubs. Knowledge Futures and Crossref are building together to test the limits of what’s possible for high-volume, high-granularity DOI management. That means fast prototypes, real building, and learning through the process.

What this looks like

We’re starting by building. The goal is to get working prototypes in front of real use cases as quickly as we can, and let the technical, UX, operational, and infrastructure questions get answered through that process. What does it take to register and manage DOIs at a level of volume and granularity that goes beyond what most existing tools support? We’ll find out by trying.

A broader orientation for KF

This is also an example of where Knowledge Futures is headed more generally. We’re taking what we’ve learned from building publishing infrastructure and applying it across different parts of the scholarly communication ecosystem. Not siloed within PubPub development, but open to building more broadly and collaborating across organizational lines.

We’ve spent close to a decade learning what it takes to build and maintain reliable infrastructure for knowledge communities. That experience doesn’t have to live inside one product. We think working this way puts us in a stronger position as stewards of the things we maintain, and it opens the door to more collaboration across the ecosystem.

Aligned direction with Crossref

As Crossref adoption has skyrocketed, enabling DOIs for a vast range of research objects and organizations, they are looking to support these objects at scale and further upstream than traditional outputs. Alongside its fee remodelling effort begun in 2023, Crossref is backing this work with a $258k investment, partnering with Knowledge Futures to explore new models for the future of open research infrastructure.

Get involved

We’d love to hear your thoughts about high-volume, high-granularity DOIs. What’s your use case? What would it unlock for your community? Want to be involved in the design process? This collaboration with Crossref is just one piece of where we’re headed. If you’re curious about what we’re up to, or have something you’d like to share with us, get in touch. We’d love to tell you what we’re working on and hear what excites you too.

Mission Accessible: building better user interfaces for everyone

Lena Stoll — Thu, 21 May 2026 00:00:00 +0000

Today is Global Accessibility Awareness Day, and accessibility has been on our minds lately. We’ve recently completed an internal audit of all our user interfaces, and have added a new accessibility page to our website, where you can find the accessibility documentation that we put together as part of the audit.

Why accessibility matters

Of course we want to keep the barriers to participation in Crossref as low as possible for users with various disabilities. But also, more accessible tools work better for everyone. A person’s access needs can change really quickly: even if you consider yourself to be relatively able-bodied, you are only one minor inconvenience away from at least a temporary disability. All it takes is some dazzling sunlight hitting your eye or your phone screen, or perhaps your dog going after a rabbit in an awkward direction while you are holding the lead (ask me how I know!) - and before you know it, you will be relying on accessibility features to navigate the digital and/or physical world for a while.

An accessible user interface is one that you can navigate and interact with by various methods, including a mouse or touchpad, keyboard, screen reader, voice control, and other assistive technologies. It can be used on various screen sizes and supports zooming in or out without losing any content or functionality. It has sufficient colour contrast, doesn’t flash fast-moving images at you, and has a clear structure that can be understood by both humans and machines.

Where we are today

It is worth mentioning that we didn’t only start thinking about accessibility when we started tackling the full audit of our user interfaces in March 2026. For example, Patrick Vale has previously written in this blog about a browser extension he has created to improve the accessibility of DOI links anywhere on the Internet. And we have known for a long time that there were accessibility gaps in many of our tools, but we didn’t have this centrally documented anywhere.

When we did begin testing all our interfaces for compliance with level AA of the Web Content Accessibility Guidelines (WCAG 2.2) as part of the audit, we knew that some of what we would find was not going to be pretty. In the 26+ years of working with and for the scholarly community, Crossref has built countless tools and reports to offer to members and users, many of which we still maintain today. These are often decades old and have been built in a way that makes it virtually impossible to make them more accessible without rebuilding them entirely. So we know that we will continue to have accessibility gaps for the foreseeable future, but at least now we have a better idea of the scale of the challenge.

It’s also not all doom and gloom: more recently created user interfaces, such our new Metadata Manager, performed much better in the audit than legacy alternatives such as the web deposit form. We found a similar trend when looking at our report interfaces. To illustrate this, compare what happens when running the axe DevTools extension for Google Chrome on a member’s participation report - this is a user interface that was completely re-implemented in 2025. Doing this brings up 26 issues:

Meanwhile, the browsable title list, which has completed a few more trips around the sun, has 254 issues listed:

Beyond WCAG

If you’ve read this far, I hope you are convinced that accessibility is more than just ticking boxes on a conformance report. But especially for a global community like ours, there are other, less technical barriers to participation that we have to consider. For example, language is a major accessibility factor: much of what we as Crossref staff write and say is in English. When we host community events, we enable captions, and we try to leave space for these captions at the bottom of our slides.

We have also started experimenting with simultaneous interpretation during our online events, such as our recent project showcase event for the 2026 metadata sprint in São Paulo. You can find recordings of this event in Spanish, Portuguese, and English on our YouTube channel to see the promising results of these efforts.

What we are working on next

We are currently addressing the accessibility issues identified in our audit of the Crossmark service. Many Crossref members have implemented the Crossmark button and pop-up on their own platforms and websites, so we thought this was a great place to start the remediation efforts following our audit.

We are also in the process of redesigning our main website, www.crossref.org, following an information architecture review completed in 2025. Making changes to the design and navigation of our website will be the perfect opportunity to make our content not just more discoverable and more understandable, but also more accessible.

Clearly there is even more to be done, so watch this space for more updates on our accessibility roadmap and improvements. And if you have first-hand experience of using Crossref services and interfaces with assistive technologies, or you have other input or feedback you’d like to share, leave a comment below or start a discussion in our community forum.

References

Vale, P. (2025). Enhancing DOI Accessibility for All Users. Crossref. https://doi-org.pluma.sjfc.edu/10.64000/pp4rw-mtv44
World Wide Web Consortium (2024). Web Content Accessibility Guidelines (WCAG) 2.2. Retrieved May 8, 2026, from https://www.w3.org/TR/WCAG22/
Montilla, L. & Mahomed, R. (2026). Voices from Crossref Metadata Sprint in São Paulo. Crossref. https://doi-org.pluma.sjfc.edu/10.64000/a5qzf-k1738
Stoll, L. & Korzec, K. (2025). Request for proposals: Crossref website information architecture review. Crossref. https://doi-org.pluma.sjfc.edu/10.64000/058mr-k3s56

DOI resolution and deposit outage on 17 March 2026

Dominika Tkaczyk — Mon, 30 Mar 2026 00:00:00 +0000

On 17 March 2026, we experienced an outage that affected DOI resolution for Crossref DOIs and the deposit of metadata records by Crossref members. In this summary, we outline what happened, the impact on our community, and the steps we are taking to strengthen our systems and processes as a result.

Starting around 06:30 UTC, a surge of traffic to the Handle System, governed by the DOI Foundation and operated by CNRI, caused instability across several handle servers. As a result, users couldn’t resolve Crossref DOIs. Because the Crossref system relies on the Handle System to register new DOIs and update resolution information for existing ones, metadata deposits by Crossref members were also failing.

Although the handle service was restored by 09:30 UTC, the Crossref system held onto stale authentication sessions and did not automatically recover. Once we restarted the affected servers, deposit processing returned to normal around 10:18 UTC. We later reprocessed the failed submissions to ensure that members did not need to take any action.

While incidents like this are disruptive for our community, several aspects of our response worked well. Our support and technology teams reacted quickly, and we were able to restore the deposit workflow promptly. We were also able to reach CNRI through established communication channels, and they responded out of hours to begin mitigation.

The incident also highlighted areas where we can improve our processes and resilience. In addition to the changes made by CNRI (more backend handle servers and tighter rate limiting), Crossref is taking the following actions:

Knowledge sharing and documentation: Crossref, like all Registration Agencies, relies on the Handle System. We are reviewing the technical documentation describing how the Crossref system integrates with the Handle System, and we will continue working with CNRI to stay informed as the handle infrastructure evolves.
Improving handle session management: We are reviewing how our system manages handle sessions to explore how recovery could occur automatically after connection disruptions.
Submission rerun process improvements: We will review, clarify, and document the workflow for identifying and rerunning affected member submissions after disruptions to deposit processing.

Reliable DOI resolution and deposit processing are central to the research ecosystem. We’re committed to continuously improving the resilience of our systems and we will continue to learn from incidents like this one to strengthen them further.

If you have any questions, we are always happy to hear from you.

You can stay informed about the status of our services by subscribing to updates at status.crossref.org

On metadata enrichment

Dominika Tkaczyk — Thu, 19 Mar 2026 00:00:00 +0000

Metadata is communication; it can tell a story about research and paint a picture for others to respond to and learn from, across the world and throughout the forthcoming generations. Metadata can feel technical with words like ‘infrastructure’ and ‘schema’, and sometimes, like tech in general, it comes with hyperbole. But metadata really is part art (storytelling and pictures) and part science (structured models and standards) with both aspects being equally important, and requiring people as well as systems. That necessary combination of human and machine involvement also makes metadata challenging.

Crossref, as the earliest adopter of DOIs specialising in scholarly research, became synonymous with DOIs in this community. However, not everyone realises that DOIs can be registered with any one of nine different agencies, which are all separate organisations with entirely separate systems that do not at present integrate or connect. And what’s more – there isn’t a central or shared “DOI schema” – each agency develops the metadata for the purposes of their organisation or community. In Crossref’s case, with our vision to create the research nexus as a complete and robust network of relationships between objects, people, and institutions of scholarship – that community encompasses the whole of the research enterprise.

The immense 180 million records of research outputs in Crossref are maintained in a system that 24,000 member organisations have already invested in. Those records benefit from rich and format-appropriate metadata schema, developed in close collaboration with the community, which makes it possible for our members to offer contextual information about each object they register. We have a long history of working with our members on recording that context, creating tools, and providing support to adopt standard metadata, enriching the context for the benefit of the scholarly community, and society at large.

Of course, those metadata records are not perfect, both in terms of quality and completeness, and the frustration around gaps in metadata is particularly strong. We are working to improve the quality and completeness of the metadata from many angles: by working with the community to understand their needs and obstacles, by identifying and analysing potential sources for additional metadata, by maintaining and adopting the existing system to changing environment, and by planning a new flexible system that will allow third-party assertions and automated enrichment workflows.

In 2020, we published a paper for the inaugural issue of Quantitative Science Studies on Crossref: The Sustainable Source of Community-Owned Scholarly Metadata and blogged an introduction to it under Crossref Metadata for Bibliometrics. One of the things our analyses in 2019 showed was that over 80% of records between 2013-2016 had been updated. Reviewing the numbers recently, we continue to see this stewardship and maintenance of metadata, amounting to almost 70% of records from the past decade being updated at least once. On the dawn of reaching 2 billion citation links, we’d like to share our experience, plans, and views on this ubiquitous activity of updating and connecting metadata – by our members and by automations built into the system by us. Altogether, these constitute the enrichment process to improve the usability of the information for the community.

Metadata available through Crossref

Crossref collects, processes, stores, and shares metadata records for a wide range of research outputs. While each record describes an individual research output, it also mentions other entities and their attributes - and, most importantly, the relationships between them. Two works identified by DOIs, for example, may be linked by a citation relationship. A person identified by an ORCID may be connected to an institution identified by a ROR ID through an affiliation relationship. A preprint and its corresponding journal article, each with its own DOI, can be linked by an “is preprint of” relationship. A research output may be associated with a grant through a “financed by” relationship. Together, these entities and relationships form the foundational building blocks of the research nexus.

As of March 14, 2026, the Crossref database contains 180,034,490 metadata records describing research outputs. You can download all the records and examine them yourself in the latest public data file. The plot below illustrates how the number of works has changed over time, showing that the rate of growth is accelerating.

The metadata records describe research outputs of various types, including:

journal articles
books and book chapters
conference proceedings
peer reviews
reports
datasets
preprints
dissertations
grants
and more

The majority of works in the Crossref database (67%) are journal articles. However, the distribution of record types has changed considerably over time. Newer types, such as components, datasets, and posted content, are growing more quickly than more traditional ways of communicating research:

Research outputs in the Crossref database are represented by rich metadata records, which may include:

basic bibliographic metadata (title, publication dates, contributors, journal title, conference name, volume and issue numbers)
authors’ affiliations and ORCID identifiers
abstracts and links to full text
funding metadata, including funders and grant details
license metadata
bibliographic reference lists
clinical trial numbers
updates such as corrections or retractions
relationships between works and other entities, such as “is translation of”, “is review of”, “is preprint of”, or “is version of”
components associated with the work, such as figures, tables, and supplemental materials

All metadata is freely available through the Crossref REST API, and additional services, such as Crossref Search, are also provided.

A natural question is: where does all this metadata come from? This is important for two main reasons. First, it helps address the question of trust, as understanding the origin of the metadata allows users to better assess its reliability. Second, it points us to the right place when investigating or addressing issues or gaps in the data.

At first glance, the answer might seem straightforward: from Crossref members. Crossref members, such as publishers, research institutions, universities, funders, museums, libraries, data and subject repositories, and conference providers, register metadata for the outputs they publish. Crossref stores this metadata and makes it available to the community.

In reality, however, the story is more complicated.

Metadata enrichment layers

The initial metadata deposit is only the beginning of what can become a long and rather fascinating journey. What users can see in our REST API is often the result of a series of updates and additions that occur over time, sometimes coming from multiple sources and happening in different ways. We can think of these ways as enrichment layers.

Each enrichment layer offers opportunities to improve the metadata while also introducing its own considerations and challenges. Rather than forming a sequence of clearly separated stages, these layers intertwine, overlap, and affect one another, collectively shaping how a research output is represented within the research nexus.

Enrichment layers are essential for completeness of the research nexus. If we relied solely on the original, one-off deposits from members, the metadata would be full of gaps, limiting the usefulness of any analysis or assessment based on it. While the scholarly metadata will never be perfectly complete, applying these enrichment layers is how we gradually and collectively build a fuller, more accurate picture of the research nexus.

One important caveat is that more metadata doesn’t magically equal better metadata. In fact, there’s often a delicate tradeoff between completeness and quality: the harder one pushes to fill every gap, the greater the chance of introducing errors. At Crossref, we believe quality comes first. We recognise that no dataset will ever be perfect, but we’re equally unwilling to apply enrichment processes without quality control. Any enrichment we introduce must meet a high bar for accuracy — no exceptions, no shortcuts.

The order of the enrichment layers discussed here loosely reflects how established they are within the scholarly ecosystem. There also might be a correlation, or at least a perceived one, between this ordering and the reliability of the underlying processes. That said, one must tread carefully when making such interpretations: perceived reliability is not the same as actual reliability.

Layer 1: Member updates

Crossref members not only deposit metadata, but also update it over time. This is an essential part of the system for several reasons. There may be errors in the originally deposited metadata that need to be corrected. Also, the initial record may contain gaps that can be filled later as more information becomes available. In addition, many changes naturally occur: landing page URLs may change, works may be archived in new locations, or identifiers for affiliated organisations may become available. Those situations also ideally result in an update.

This update process is well established. Over 24,000 Crossref members form a large global community that operates under shared membership terms. As part of these terms, members are responsible for maintaining and updating their metadata records. In this governance framework it is clearly defined who owns and stewards the metadata associated with each record, and who is responsible for the quality level and issues.

Member updates are very common. As an example, over 80% of works deposited between 2013 and 2020 were updated at least once. This demonstrates the community’s commitment to improving completeness and quality of the scholarly record. The plot below shows the percentage of works created in a given month that were updated at least once.

However, this layer also comes with challenges. It relies on members actively meeting their obligations to maintain and improve their metadata. As a result, gaps and inconsistencies can remain, and overall metadata quality is never perfect.

Our plans for the future in this area largely build on what is already happening. This includes developing and maintaining effective user interfaces for updating metadata, evolving the input metadata schema to keep pace with changes in the scholarly landscape, offering regular workshops on metadata improvements, and collaboratively establishing best practices while educating members on how to apply them.

Layer 2: Community feedback loop

Crossref metadata is widely used and examined by a large community of consumers. As a result, issues with metadata are sometimes identified by community members and reported back to us. When this happens, Crossref does not directly correct the metadata records. Instead, we contact the relevant member responsible for the record and able to deposit an update.

In this layer, the stewardship of metadata remains with the member, while responsibility for metadata quality broadens to include other actors in the community. This creates significant potential for scaling by involving a large community in identifying and reporting metadata issues.

At present, however, this process is not automated. Crossref staff effectively act as intermediaries between those reporting issues and the responsible member. As a result, the process has limited scalability. It also depends on the willingness of members to act on the reports they receive, as they are not obligated to respond to such reports.

In the future, we may explore automating portions of this workflow to handle community feedback more efficiently and lighten the load on everyone involved.

Layer 3: Metadata matching

Metadata matching is the task of finding an identifier for an item based on a structured or unstructured description of it. Matching strategies run as fully automated processes that analyse information deposited and updated by members and add identifiers, filling gaps in the metadata.

There are many instances of metadata matching problems, for example:

bibliographic reference matching: finding a DOI for a cited paper based on a bibliographic reference,
funder matching: finding the ROR ID for a funder based on its name,
affiliation matching: finding the ROR ID for an organisation based on an affiliation string,
preprint matching: finding the DOI for a preprint that precedes a given journal article,
grant matching: finding the grant DOI based on an award number and a funder name.

This layer is unique, as it focuses on a crucial type of gap in the scholarly record: the missing relationships between entities. Indeed, adding an identifier for an entity mentioned within a metadata record of a research output is typically an equivalent of asserting a relationship between that output and the matched entity. For example, bibliographic reference matching inserts citation relationships, and funder name matching - funding relationships between a research output and a funding organisation. These relationships form the foundation of the research nexus.

Currently, at Crossref, we perform two types of matching. We match bibliographic references to the DOIs of cited outputs, and funder names to Funder IDs. Both processes rely on fuzzy comparisons and other heuristic approaches to identify likely matches.

In the case of bibliographic reference matching, as it turns out, more than half of the cited DOIs (1 billion) available in the Crossref database originate from automated metadata matching:

In the case of funder name matching, the distribution is very different, but the matching strategy was still able to fill in some of the gap:

Metadata matching is a particularly valuable form of enrichment for several reasons. Matching strategies can often achieve high levels of accuracy while working in a fully automated way. This makes them highly scalable and drastically reduces the need for human oversight. Their focus on relationships also strengthens the foundations of the research nexus.

At the same time, this enrichment layer presents a number of challenges.

Its most fundamental limitation to remember is that metadata matching can only fill gaps when there is at least some useful information to work with. For example, it can identify a cited document only using structured or unstructured citation data, and the funding organisation can only be identified if some funding information is available. But if citation information, or funding information, is completely absent, as is the case for 101M (56%) records and 166M (92%) records respectively, then matching simply isn’t possible.

Matching strategies can also be complex and time-consuming to research, develop, and maintain. They require additional considerations of issues such as openness, explainability, complexity, flexibility, and cost.

Perhaps most importantly, in the case of matching, it becomes less clear who is responsible for the information introduced through the matching process. This is particularly important because matching results are never perfect, meaning there is always a risk of introducing errors. The risk is further amplified by the fact that matching strategies typically operate in a fully automated, unsupervised manner. As a result, careful evaluation of matching performance, as well as maintaining accurate provenance records, becomes increasingly important.

At Crossref, we have ambitious plans in this area. We intend to rebuild Crossref’s metadata matching workflows using modern software development and data science practices. The goal is to create a dedicated, consolidated matching service that will eventually replace all existing production matching processes, with results made available through the REST API. This project will cover six matching tasks: bibliographic reference matching, funder name matching, preprint matching, affiliation matching, grant matching, and title matching. You can learn more about metadata matching at Crossref at a dedicated project page.

Layer 4: Third-party datasets

There are many databases containing scholarly data, and one way to fill gaps in Crossref member-provided metadata is to incorporate additional metadata from those external sources.

We already have one example of this. Crossref ingests data from the Retraction Watch database to supplement information about retractions and other updates to records:

This layer has several advantages. It draws on subject-specific and metadata-specific expertise, avoids reinventing work that has already been done elsewhere, and reflects a collaborative community-driven approach to improving the scholarly record.

However, there are also important challenges to consider. Integrating external data often involves multiple data licenses or acquisition arrangements, and there may be less control over data quality compared to metadata that comes directly from members. There is also a risk that relying too heavily on external sources could shift responsibility away from the member stewards of the metadata. Finally, it can be difficult to determine which external datasets provide sufficient value and longevity to justify long-term integration.

Looking ahead, we plan to explore further opportunities to incorporate third-party datasets, carefully considering the value they bring, as well as issues of licensing, sustainability, and data quality.

Layer 5: Unstructured content scraping

A significant amount of scholarly information still exists in fully unstructured forms, such as full-text PDF documents and web pages. In principle, extracting information from these sources could help fill many gaps in existing metadata.

In a lighter-touch approach, analysing full-text documents can also help verify existing metadata elements. If such a check fails, the unverified element may be removed from the record — which, perhaps counterintuitively, can also count as enrichment, since improving accuracy is every bit as important as adding new information.

There are also important challenges to consider. Extracting metadata directly from unstructured sources could substantially shift responsibility away from the original data stewards or owners, weakening the current stewardship model. The results of automated extraction may also be inconsistent or of relatively low quality. In addition, there are potential legal and rights-related concerns, particularly when processing full-text materials. Finally, developing reliable extraction methods would require substantial research and engineering effort.

For all these reasons, the practical usefulness of this approach remains uncertain, and Crossref currently has no plans to run such processes in production. We will, however, keep a close eye on emerging extraction technologies and may consider adopting them in some form if future evaluations show clear value.

Summary

Metadata is far more than a technical afterthought of the publishing process. It is the connective tissue of the scholarly ecosystem, linking research objects, people, and institutions into a coherent, navigable network. At Crossref, this takes the form of a vast and continually evolving corpus of more than 180 million metadata records, all contributing to the emerging research nexus, being built through collective community effort to help the global research community discover, interpret, and reuse knowledge effectively.

The initial metadata record deposited by members is only the beginning. Its quality and completeness can improve over time through multiple enrichment layers: member-driven updates, community feedback, automated metadata matching, and the incorporation of third-party datasets. These processes help fill gaps and strengthen the reliability of the scholarly record, all while upholding a firm commitment to accuracy and stewardship.

Taken together, these layers reflect a long-term, collaborative effort across technology developments, community participation, and responsible automation, to ensure that scholarly metadata becomes richer, more interconnected, and more useful for everyone who relies on it.

Innovation in scientific publishing and its implications for Crossref DOI registration practices - MetaROR’s approach

Ludo Waltman — Tue, 03 Feb 2026 00:00:00 +0000

A couple of months ago, Ludo Waltman and André Brasil raised some questions about good practices for Crossref DOI registration, asking for input from the scholarly communication community. In this post, Ludo and André reflect on the input received and discuss the approach to DOI registration that the MetaROR publish-review-curate platform is going to take.

Practices for assigning DOIs and structuring the associated metadata are not merely technical details. They shape how scholarly outputs are discovered, cited, evaluated, indexed, and preserved over time. As new models of publishing emerge, especially those that decouple dissemination from evaluation, these infrastructural choices increasingly influence what counts as a scholarly object, as well as how credit and accountability mechanisms are organized.

As editors of MetaROR (MetaResearch Open Review), a platform launched in 2024 and operating under the publish-review-curate model, we are interested in good practices for Crossref DOI registration in the context of innovative new approaches to scientific publishing. In the earlier blog post, we invited members of the broader scholarly communication community to share their perspective on the following two questions:

For each article on the MetaROR platform, there is a corresponding article on a preprint server. Is it acceptable to have two Crossref DOIs, one registered by the preprint server and one registered by the MetaROR platform, for essentially the same article?
If Crossref DOIs are registered for articles on the MetaROR platform, should the articles be assigned the type ‘journal-article’ or the type ‘preprint’ in their Crossref metadata, or something else entirely?

We were pleasantly surprised by the level of interest in these two questions. We received about 15 responses from colleagues in the scholarly communication community. Some colleagues posted a reply at the bottom of our blog post. Others responded on social media (Bluesky, LinkedIn) or shared their perspective by email.

Below we reflect on the responses received and we outline the approach to Crossref DOI registration that MetaROR is going to take.

DOI registration for articles on the MetaROR platform

Colleagues offered mixed opinions on the question of whether articles on the MetaROR platform should have their own DOI, in addition to the DOI these articles have on the preprint server on which they were originally published. Some colleagues argued there is no good reason for registering DOIs for articles on the MetaROR platform and suggested this may cause confusion. One colleague reasoned that “if we want peer review to be something more ongoing and evolve beyond a single point in time judgment”, our approach should be to “better map the connections between events” rather than registering a new DOI each time an article has been peer-reviewed.

However, other colleagues expressed support for registering DOIs for articles on the MetaROR platform. One colleague pointed out that this “allows the user to reference the exact artefact they have consulted”. This colleague also reminded us that in the past “people were worried about having a different DOI for a preprint and another for a VoR (version of record)”, while nowadays this is a generally accepted practice. Another colleague emphasized the value of decentralization and suggested to “let a thousand DOIs bloom”. Authors of an article peer-reviewed by MetaROR argued in favor of “an overarching DOI for the full package (preprint, reviews, author response and link to updated preprint)”, which in their view would make MetaROR’s “process more coherent”.

Having considered the various arguments in favor of or against registering DOIs for articles on the MetaROR platform, we feel the arguments in favor are more compelling. Our perspective is that an article on the MetaROR platform differs in a meaningful way from the corresponding article on a preprint server, since the article on the MetaROR platform has been enriched with an evaluation by peer reviewers and editors. MetaROR provides a carefully curated package that includes not only the article itself, but also review reports and an editorial assessment. In our view, this justifies registering DOIs for articles on the MetaROR platform. We also see DOI registration for articles on the MetaROR platform as a way to promote appropriate recognition for authors of articles peer-reviewed by MetaROR, similar to the way authors get recognition for articles published in traditional peer-reviewed journals.

Of course, when an article has multiple versions, each with their own DOI, it is important to establish a link between the different DOIs, indicating that the DOIs are associated with the same work. This is important for articles published first on a preprint server and then on a platform such as MetaROR just like it is important for articles published first on a preprint server and then in a peer-reviewed journal. In practice, we establish these links by registering relationships between DOIs in the associated metadata. In this way, we ensure that indexing services, discovery systems, and research analytics tools are able to recognize that the DOIs refer to different manifestations of the same work rather than independent outputs.

Record type for articles on the MetaROR platform

Our second question is about the record type to be used when registering a Crossref DOI for an article on the MetaROR platform. Many colleagues who provided input on this question argued there is a need for a new Crossref record type for ‘reviewed preprints’.

We feel the idea of such a new record type is interesting and its pros and cons deserve further consideration. However, any solution that requires changes in Crossref’s metadata schema will take time to realize, while for MetaROR we need a solution in the short term. At the moment, the most obvious options for MetaROR therefore seem to be to use either the record type ‘journal-article’ or the record type ‘preprint’ (which is in fact a subtype of the record type ‘posted-content’).

The use of the record type ‘preprint’ seems somewhat problematic to us, because preprints are typically understood to be articles that have not yet been formally peer-reviewed. In a way, articles on the MetaROR platform are the opposite of this, since these articles have undergone formal peer review. An article on the MetaROR platform is part of a package that also includes review reports and an editorial assessment. Such a package provides readers with a more informed understanding of an article than what they get from reading only the article itself. For this reason, we do not consider the record type ‘preprint’ to be suitable for articles on the MetaROR platform.

Instead of the record type ‘preprint’, we have decided to use the record type ‘journal-article’ for articles on the MetaROR platform. The record type ‘journal-article’ is intended for articles published in journals. To be clear, MetaROR considers itself a ‘platform’, not a ‘journal’. However, the distinction between ‘platforms’ and ‘journals’ is not very well defined and the choice of terminology therefore involves a certain degree of arbitrariness. Moreover, articles on the MetaROR platform have been formally evaluated, and in that sense they resemble articles in traditional peer-reviewed journals. Although the nature of the evaluation is different (i.e., MetaROR provides a narrative assessment, while traditional journals provide a ‘stamp of approval’), we feel the resemblance justifies the use of the record type ‘journal-article’. We also hope that the use of this record type will help to ensure that articles evaluated by publish-review-curate (PRC) platforms are treated similarly to articles evaluated by traditional journals, advancing beyond more conservative ways of dealing with articles on PRC platforms.

There is a precedent for using the Crossref record type ‘journal-article’ for articles evaluated by PRC platforms. For over a decade, this approach has been used by platforms operated by F1000, such as F1000Research, Gates Open Research, Open Research Europe, and Wellcome Open Research. The approach we are taking at MetaROR is similar to the approach taken by these platforms. At the same time, our approach is different from the approach of eLife, another prominent PRC platform. eLife uses the record type ‘preprint’ for all versions of an article on its platform except for the version that the authors consider to be final and that they choose to designate as the ‘version of record’. This version has the record type ‘journal-article’.

Summary of MetaROR’s approach to Crossref DOI registration

Figure 1 summarizes MetaROR’s approach to Crossref DOI registration. The figure considers the situation in which an article went through two rounds of peer review by MetaROR. Both rounds of peer review involved two reviewers. After two rounds of peer review by MetaROR, the article was published in a journal. We emphasize that journal publication is optional in MetaROR’s PRC approach. It is included in Figure 1 for the sake of completeness.

Figure 1: MetaROR’s approach to Crossref DOI registration

Each element in Figure 1 represents an item that has its own Crossref DOI. The shape of an element indicates the Crossref record type of an item (‘preprint’, ‘journal-article’, ‘peer-review’). MetaROR is responsible for the blue elements in the figure. The gray elements are the responsibility of other actors, either a preprint server or a journal. Arrows represent relationships between items. These relationships are captured in the Crossref metadata of the various items.

Figure 1 shows how MetaROR treats articles, review reports, editorial assessments, and author responses as first-class research objects. Each object has its own DOI, while the objects are linked through structured metadata. Assigning DOIs to review reports, editorial assessments, and author responses is central to our commitment to transparency, recognition, and reuse of evaluative contributions.

We note that Figure 1 assumes each version of an article on a preprint server has its own DOI. This is indeed how DOI registration is handled by many preprint servers, such as the OSF servers (e.g., MetaArXiv, PsyArXiv, SocArXiv), ChemRxiv, Research Square, and Preprints.org. However, some preprint servers use a single DOI for all versions of an article. This is the case for bioRxiv and medRxiv and also for arXiv, which registers DOIs with DataCite rather than Crossref. In the future, we hope these preprint servers will also adopt versioned DOIs.

Outlook

Over the past 25 years, practices for registering DOIs and associated metadata have evolved along with broader developments in the scholarly communication landscape. Inevitably, DOI registration practices will always be lagging behind the most recent developments in scholarly communication. From this point of view, the lack of agreement on good practices for DOI registration in the context of PRC platforms is not surprising. This lack of agreement can in fact be seen as part of a larger discussion about the pros and cons of different infrastructural approaches for handling ‘preprint review metadata’, including for instance the COAR Notify approach and the DocMaps approach.

MetaROR’s approach to DOI registration demonstrates both the power and richness of Crossref’s metadata schema and its limitations. As discussed above, several colleagues who responded to our earlier blog post consider the lack of a record type for ‘reviewed preprints’ to be a significant limitation. With the growing interest in PRC models for scientific publishing, there appears to be a need to systematically evaluate possible improvements that can be made to Crossref’s metadata schema to offer better support for new approaches to scientific publishing.

We see this not only as a technical challenge but also as an issue of infrastructure governance. We therefore invite further dialogue between DOI registration agencies, other metadata infrastructures, preprint servers, PRC platforms, and indexing services to explore pathways for improving metadata standards, whether through new record types, extended relationship vocabularies, or shared best practices. We hope our experiences with MetaROR will contribute to the collective effort needed to ensure that emerging models of scholarly communication are represented accurately, transparently, and responsibly in the scholarly record.

Crossref note: This discussion chimes with related plans for extending our schemas: more granular vocabulary for items within journal articles, preprints, reviews, and others; clearer relationship types; and support for the forthcoming NISO JAV recommendations. Our Preprint Advisory Group will discuss the topic this year, and our Metadata Advisory Group has both ‘journal article type vocab’ and ‘relationships’ on its radar for 2026. We look forward to engaging further on this topic as we work towards more flexible schemas in support of the Research Nexus.

Innovation in scientific publishing and its implications for Crossref DOI registration practices - Request for input

Ludo Waltman — Thu, 25 Sep 2025 00:00:00 +0000

Lots of exciting innovations are being made in scientific publishing, often raising fundamental questions about established publishing practices. In this guest post, Ludo Waltman and André Brasil discuss the recently launched MetaROR publish-review-curate platform and the questions it raises about good practices for Crossref DOI registration in this emerging landscape.

Digital Object Identifiers (DOIs) are unique identifiers commonly assigned to research outputs such as journal articles, preprints, peer review reports, and datasets. The DOI of a research output allows the output to be identified online in a persistent way, even when the underlying publishing infrastructure changes (e.g., a journal moving from one publisher to another).

There are several DOI registration agencies. Most of the larger scientific publishers work with Crossref, and so do many preprint servers, and therefore our focus in this post is on Crossref. Crossref also keeps track of metadata associated with research outputs, such as the title, authors, and publication date of an output, and it makes this metadata openly available via APIs for all kinds of services to ingest and reuse. Because indexing, discovery, and evaluation tools rely heavily on this metadata, content registration practices and metadata design choices can have major effects on the visibility and findability of research outputs and on analytics used to monitor and assess research outputs and their contributors.

For the most common types of research outputs, such as journal articles and preprints, a broad consensus has emerged over the past decades on good practices for DOI registration. Such consensus means that articles are assigned the record type ‘article’ in their Crossref metadata. Likewise, many preprint servers register DOIs for preprints at Crossref, with the record type ‘preprint’ in the metadata. (The arXiv preprint server is an exception; it registers DOIs for preprints with DataCite rather than Crossref.)

For innovative new publication platforms, however, good practices for DOI registration are less clear. The approaches to scientific publishing offered by these platforms often do not fit neatly into established ways of working. For instance, for some of these platforms, the traditional distinction between peer-reviewed articles published in scientific journals and non-peer-reviewed articles posted on preprint servers is no longer applicable. This raises fundamental questions about suitable DOI registration practices for new approaches to scientific publishing.

MetaROR

The MetaROR (MetaResearch Open Review) platform, launched in November 2024 by the Research on Research Institute (RoRI) and the Association for Interdisciplinary Meta-Research and Open Science (AIMOS), offers an example of the challenge of developing appropriate DOI registration practices for new publishing models.

Inspired by similar initiatives such as eLife and others, MetaROR adopts the so-called publish-review-curate model. Authors first publish their article on a preprint server and then submit it to MetaROR. MetaROR then organizes an open peer review process for the article. Review reports are published on the MetaROR platform, along with a copy of the preprinted article and an editorial assessment. Rather than a simple binary decision (accept vs. reject), an editorial assessment is a short one-paragraph statement summarizing the strengths and weaknesses of an article. Each review report and each editorial assessment has its own DOI registered at Crossref. In this way, review reports are treated as first-class research outputs that can, for instance, be indexed in scientific literature databases and can be cited in other research outputs.

For an article submitted to MetaROR, the publication of the review reports, the editorial assessment, and a copy of the article itself concludes MetaROR’s publish-review-curate process. The authors of the article may revise their work in light of the feedback received, and MetaROR may review the revised article. However, there is no requirement that revisions must be made. The primary aim of the review reports and the editorial assessment published on the MetaROR platform is to offer context for readers of the article, helping readers understand the strengths and weaknesses of the article.

Crossref DOI registration

Registration of DOIs for open peer review reports is increasingly common. By registering Crossref DOIs for review reports and editorial assessments, MetaROR enables reviewers and editors to be recognized for their contributions. But what about recognition for authors?

A crucial element in MetaROR’s philosophy is that authors of articles peer-reviewed by MetaROR deserve to be recognized in a similar way as authors of articles published in traditional peer-reviewed journals. One way to promote appropriate recognition for authors of articles peer-reviewed by MetaROR is to ensure that articles on the MetaROR platform, just like articles in peer-reviewed journals, have their own DOI. While this may seem straightforward to arrange, it actually raises two non-trivial questions about good practices for Crossref DOI registration:

For each article on the MetaROR platform, there is a corresponding article on a preprint server. Is it acceptable to have two Crossref DOIs, one registered by the preprint server and one registered by the MetaROR platform, for essentially the same article?
If Crossref DOIs are registered for articles on the MetaROR platform, should the articles be assigned the type ‘article’ or the type ‘preprint’ in their Crossref metadata, or something else entirely?

On the first question, it could be argued that having two Crossref DOIs for the same article is problematic and that MetaROR, therefore, should not register DOIs for articles on its platform. Alternatively, one could argue that an article on the MetaROR platform differs in a meaningful way from the corresponding article on a preprint server, since the article on the MetaROR platform has been enriched with peer review reports and an editorial assessment, similar to the way an article in a peer-reviewed journal may be seen as an enriched version of the corresponding article on a preprint server. This line of reasoning would justify registering DOIs for articles on the MetaROR platform.

On the second question, the argument could be made that articles on the MetaROR platform should be assigned the type ‘preprint’ in their Crossref metadata, since the type ‘article’ is intended for articles in journals and MetaROR does not consider itself to be a journal (in fact, MetaROR works with partner journals to enable articles peer-reviewed by MetaROR to be published in journals) and does not certify articles in the way journals do (i.e., MetaROR does not make accept/reject decisions). On the other hand, one could argue that articles on the MetaROR platform should be assigned the type ‘article’, since the peer-reviewed nature of articles in journals is typically seen as the key factor distinguishing these articles from articles on preprint servers. Articles on the MetaROR platform have been peer-reviewed, and in that sense, they resemble articles in journals. A third line of reasoning could be that neither the ‘preprint’ nor the ‘article’ type is fully appropriate for articles on the MetaROR platform and, consequently, that there is a need for a new Crossref record type.

What is your take?

The MetaROR team, in consultation with Crossref, will need to decide how to deal with the two questions discussed in this blog post. After some preliminary conversations between the MetaROR team and Crossref, we decided to share these questions more widely to solicit input from the broader community. We invite you to share your thoughts on the two questions, either by posting a comment on this blog post or by reaching out to us on social media or by email. Community perspectives will help shape good practices not only for MetaROR but also for other publish-review-curate initiatives facing similar questions. We look forward to hearing from you!

Ludo Waltman and André Brasil are members of the editorial team of MetaROR. Ludo and André are grateful to Ginny Hendricks at Crossref for valuable discussions about the issues raised in this blog post.

What do we know about DOIs

Martin Eve — Thu, 29 Feb 2024 00:00:00 +0000

Crossref holds metadata for approximately 150 million scholarly artifacts. These range from peer reviewed journal articles through to scholarly books through to scientific blog posts. In fact, amid such heterogeneity, the only singular factor that unites such items is that they have been assigned a document object identifier (DOI); a unique identification string that can be used to resolve to a resource pertaining to said metadata (often, but not always, a copy of the work identified by the metadata).

What, though, do we actually know about the state of persistence of these links? How many DOIs resolve correctly? How many landing pages, at the other end of the DOI resolution, contain the information that is supposed to be there, including the title and the DOI itself? How can we find out?

The first and seemingly most obvious way that we can obtain some of these data is by working through the most recent sample of DOIs and attempting to fetch metadata from each of them using a standard python script. This involves using the httpx library to attempt to resolve each of the DOIs to a resource, visiting that resource and seeing what the landing page yields.

Even this is not straightforward. Landing pages can be HTML resources or they can be PDF files, among other things. In the case of PDF files, to detect a run of text is not simple as a single line break can be enough to foil our search. Nonetheless, when using this strategy we find the following statistics:

Total DOI count in sample: 5000
Number of HTTP 200 response: 3301*
Percentage of HTTP 200 responses: 66.02%
Number of titles found on landing page: 1580
Percentage of titles found on landing page: 31.60%
Number of DOIs in recommended format found on landing page: 1410
Percentage of DOIs in recommended format found on landing page: 28.20%
Number of titles and DOIs found on landing page: 929
Percentage of titles and DOIs found on landing page: 18.58%
Number of PDFs found on landing page: 1469
Percentage of PDFs found on landing page: 29.38%
Percent of PDFs found on landing pages that loaded: 44.50%

* an HTTP 200 response means that the web page loaded correctly

While these numbers look quite low, the problem here is that a large number of scholarly publishers use Digital Rights Management techniques on their sites that block a crawl of this type. We can use systems like Playwright to remote control browsers to do the crawling, so that the request looks as much like a genuine user as possible and to evade such detection systems. However, lots of these sites detect headless browsers (where the browser is invisible and running on a server) and block them with a 403 Permission Denied error.

There’s a great Github javascript suite that aims to help evade headless detection. The tests it uses are:

User Agent: in a browser running with puppeteer in headless mode, user agent includes Headless.
App Version: same as User Agent above.
Plugins: headless browsers don’t have any plugins. So we can say that if it has plugin it’s headful, but not otherwise since some browsers, like Firefox, don’t have default plugins.
Plugins Prototype: check if the Plugin and PluginsArray prototype are correct.
Mime Type: similar to Plugins test, where headless browsers don’t have any mime type
Mime Type Prototype: check if the MimeType and MimeTypeArrayprototype are correct.
Languages: all headful browser has at least one language. So we can say that if it has no language it’s headless.
Webdriver: this property is true when running in a headless browser.
Time elapse: it pops an alert() on page and if it’s closed too fast, means that it’s headless.
Chrome element: it’s specific for chrome browser that has an element window.chrome.
Permission: in headless mode Notification.permission and navigator.permissions.query report contradictory values.
Devtool: puppeteer works on devtools protocol, this test checks if devtool is present or not.
Broken Image: all browser has a default nonzero broken image size, and this may not happen on a headless browser.
Outer Dimension: the attributes outerHeight and outerWidth have value 0 on headless browser.
Connection Rtt: The attribute navigator.connection.rtt,if present, has value 0 on headless browser.
Mouse Move: The attributes movementX and movementY on every MouseEvent have value 0 on headless browser.

Using the stealth plugin for Playwright also allows us to evade most of these checks. This just leaves Mouse Move and Broken Image detection, which I thought would not outweigh all the other factors. We can also jitter the connection with arbitrary delays so that it should appear to be coming at random intervals, rather than a robotic crawl.

Yet the basic fact is that we are still blocked from crawling many sites. This does not happen when we put the browser into headful mode, so current detection techniques have clearly evolved in the past half decade (since Detect Headless) was designed.

If, however, we run the browser in a headful mode, the results are somewhat stunningly different:

Total DOI count in sample: 5000
Number of HTTP 200 response: 4852
Percent of HTTP 200 responses: 97.04%
Number of titles found on landing page: 2547
Percentage of titles found on landing page: 50.94%
Number of DOIs in recommended format found on landing page: 2424
Percentage of DOIs in recommended format found on landing page: 48.48%
Number of titles and DOIs found on landing page: 1574
Percentage of titles and DOIs found on landing page: 31.48%
Number of PDFs found on landing page: 2085
Percentage of PDFs found on landing page: 41.70%
Percentage of PDFs found on landing pages that loaded: 42.97%

Let’s talk about the resolution statistics. Other studies, looking at general links on the web, have found a link-rot rate of about 60%-70% over a ten-year period (Lessig, Zittrain, and Albert 2014; Stox 2022). The DOI resolution rate that we have, with 97% of links resolving (or a 3% link-rot rate), is far better and more robust than a web link in general.

Is 3% a good or a bad number? It’s more robust than the web in general, but it still means that for every 100 DOIs, just under 3 will fail to resolve. We also cannot tell whether these DOIs are resolving to the correct target, except by using the metadata detection metrics (are the title and DOI on the landing page, which we could only detect at a far lower rate). It is entirely possible for a website to resolve with an HTTP 200 (OK) response, but for the page in question to be something very different to what the user expected, a phenomenon dubbed content drift. A good example is domain hijacking, where a domain name expires and spam companies buy them up. These still resolve to a web page, but instead of an article on RNA, for a hypothetical example, the user gets adverts for rubber welding hose. That said, other studies are also prone to this and there is no guarantee that content drift doesn’t affect a huge proportion of supposedly good links in the other studies, too.

Of course, one of the most frustrating elements of this exercise is having to work around publisher blocks on content when visiting using a server-only robot script. It’s important for us periodically to monitor the uptime rate of the DOI system. We also recognise, though, that publishers want to block malicious traffic. However, we can’t perform our monitoring in an easy, automatic way if headless scripts are blocked from resolving DOIs and visiting their respective landing pages. This is not even a call for open access; it’s just saying that current anti-bot techniques, sometimes implemented for legitimate reasons, stifle our ability to know the landscape. Even if the bot resolved a DOI to just a paywall, it would be easier for us to monitor this than it is now. Similarly, CAPTCHA systems such as Cloudflare that would seem to offer an easy way to distinguish between humans (good) and robots (bad) can make life very difficult at the monitoring end. We would certainly be grateful for any proposed solution that could help us to work around these mechanisms.

Conclusion

The context in which I wanted to know this information was so that we can take a snapshot of a page and then, at a later stage, determine whether it is down or has changed substantially. To do this, we are developing Shelob, an experimental content drift spider system; that’s what we’ve used so far to conduct this analysis. Over time, Shelob will evolve, we hope, to give us a way to detect when content has drifted or gone offline. If, however, we can’t detect whether an endpoint is good in the first place, then we likewise cannot detect when things have gone wrong. On the other hand, if, when we first visit, we find the DOI and title on the landing page, but at some future point this degrades, we might be able to say with some confidence that the original has died. I, personally, would encourage publishers not to block automated crawlers, because it’s good when we can determine these types of figures.

Works Cited

Lessig, Lawrence, Jonathan Zittrain, and Kendra Albert. 2014. ‘Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations’. Harvard Law Review 127 (4). https://harvardlawreview.org/forum/vol-127/perma-scoping-and-addressing-the-problem-of-link-and-reference-rot-in-legal-citations/.(https://www.zotero.org/google-docs/?970bfS

Stox, Patrick. 2022. ‘Ahrefs Study on Link Rot’. SEO Blog by Ahrefs. 29 April 2022. https://ahrefs.com/blog/link-rot-study/.

Forming new relationships: contributing to open source

Patrick Vale — Wed, 19 Oct 2022 00:00:00 +0000

TL;DR

One of the things that makes me glad to work at Crossref is the principles to which we hold ourselves, and the most public and measurable of those must be the Principles of Open Scholarly Infrastructure, or POSI, for short. These ambitions lay out how we want to operate - to be open in our governance, in our membership and also in our source code and data. And it’s that openness of source code that’s the reason for my post today - on 26th September 2022, our first collaboration with the JSON Forms open-source project was released into the wild.

Like most organisations, we depend heavily on open-source software for our operations - the software is universally available, generally high quality and ‘free’. And it’s easy to take that dependency, and the associated dependency on free time and effort on the part of the maintainers, for granted - but that’s not very sustainable. In fact, we believe relying on open-source software without helping to sustain it is an anti-pattern, and this project marks the start of our efforts to make funding open-source software a standard part of our technology budget.

This isn’t the first time we’ve supported or released open-source software. Indeed for the past few years, all our new software is open source, and we’re in the process of replacing old closed code with new, so that eventually all our code will be open source. But this is the first time we’ve contributed extensively to something that isn’t focussed primarily on us, and our services. This is a project that we will find very useful, but it is a general purpose tool, and it’s already gaining traction in the community.

Background and motivations

A while back, I was tasked to do a quick spike of work on testing the theory that we could use automated form generation tools to bring new interfaces to our users more quickly, and make them easier for “people who aren’t devs” to adapt and manage. We wanted to build a new user interface for registering content, and especially we wanted to make it easier for funders to register the grants they were awarding. As well as being more approachable by a less-technical audience, we also wanted these forms to be accessible (in terms of a11y and users of assistive technology) and localisable - we wanted a solution that would cater to the needs of our rapidly diversifying membership.

Enter JSON Schema

We were clear about one side of the puzzle - we knew that we had to look beyond the XML ecosystem upon which much of our existing system is built - and landed on JSON Schema. JSON Schema is a ‘vocabulary that allows you to annotate and validate JSON documents’. This means you can describe the shape you expect your data to take, and apply constraints-based validation to that. Which means, in terms of a form library, that you can infer the structure of the form and test that the data entered into it matches what you expect. More than that, you can use that built-in validation to provide error messages to help people get the data right, first time.

Working backwards from the outcome, the argument for adopting JSON Schema is compelling. It provides a mechanism for checking that data you are handling (for example, receiving input from a form) conforms to the constraints that you declare, but also allows you to tell people up-front, in a human and machine-readable way, what structure and format you will accept. This closed-loop of data annotation and validation gets more appealing when you look at the wide adoption of JSON Schema across languages and libraries. You can pretty much guarantee that for whatever client or server -side technology you are using, there will be a JSON Schema validator for it. Being able to share schemas across your systems (and equally importantly, with third parties) moves JSON schema from ‘just’ being about data validation, to a key supportive technology.

Building a form derived from a JSON Schema is an equally attractive prospect. JSON Schema was conceived during the AjaxWorld conference in 2007 as a ‘JSON-based format for defining the structure of JSON data’, and its use as a form-generation tool is relatively new, but there is growing community interest. There is even a discussion about how to best create a JSON Schema vocabulary, specifically geared towards addressing some of the needs of form generation users. However, even in its current form, a JSON Schema can be passed to a library, and a very serviceable user interface appears. The devil is always in the detail, and the client-side libraries differ in their abilities to customise areas such as layout (you may not always want your form fields to appear in exactly the same order as they do in your JSON Schema), custom elements (you might want something that wasn’t a form input, or that changes based on user input) and localisation. The ability to flexibly customise the appearance and behaviour of the interface was a key factor in our selection of a client-side form generation library.

Choosing a library

The other side of the puzzle was less clear - choosing a UI library that would take this JSON Schema, and turn it into a useful, and usable, form. I made the prototype using the venerable React JSON Schema form. This worked well as a proof of concept, but veered dramatically off our chosen Frontend stack of VueJS and Vuetify, and had some architectural constraints that would limit the scope of customisations we could make to our forms. So I went off looking for libraries that would work with our stack and came up with Vuetify JSON Schema Form, and JSON Forms.

Vuetify JSON Schema Form matched our stack perfectly, but made some interesting decisions about the layout of data within the form, and that wouldn’t suit our purposes without dramatic modification.

JSON Forms was an abstracted library, with a core handling the JSON Schema transformation and validation, and separate rendering libraries to handle the form generation. This was great - they had renderers for Angular, React, and even some support for VueJS. But not Vuetify.

Clearly, we were going to have to make something.

We made contact with the maintainers of both short-listed libraries to see how we could collaborate in creating a tool that would meet all of our (and hopefully, much of the wider community’s) requirements. Both maintainers were very helpful, and we had constructive discussions in both cases. In the end, we decided that the abstracted nature of the JSON Forms project was a better fit for our needs, providing a flexible platform on which we - and others - could extend. We were fortunate to receive funding from the Gordon and Betty Moore Foundation (Grant Agreement #10485) in order to accelerate this work, so we could provide a Grant Registration UI more quickly. We paid a large portion of that funding to the library maintainers, and Crossref contributed a portion of my time on the project. This allowed us to enter into an agreement with EclipseSource, the maintainers of JSON Forms, to collaboratively develop the new VueJS and Vuetify renderer library. Stefan Dirix, the lead maintainer, worked with me to build it.

We didn’t forget about Vuetify JSON Schema Form though, and by way of appreciation for their help in the early stages, Crossref made a contribution towards the continued development of that library.

JSON Forms - now with Vuetify

Work started on the JSON Forms Vuetify renderer set in September 2021 - Stefan quickly created the first early prototypes of the new form renderers - but then we had a stroke of luck. Our repository received more input from the community. The one that made us sit up and take real notice was the news that someone else had already ported the JSON Forms React renderer set to Vue/Vuetify - and was offering this as a contribution. Krasimir Chobantonov’s fantastic first contribution got merged in at the end of the month. This propelled the project forward massively, and was an early validation of the value of working in the open. Needless to say, we were very grateful. Another example of the open source value chain was that Stefan - as the maintainer - could take the time to carefully review and tidy up the incoming code, so what was merged was the product of two great developers.

Having this great head start meant we could turn our attention to one of the other big areas we wanted to get right - localisation. Traditionally, JSON Schema -generated forms have handled localisation (translation of text and adjustment of date and numerical formats) by wholesale duplication and translation of the schema. This is cumbersome, and doesn’t integrate very well with custom error messages, nor external sources of interface messages (think form labels, descriptions, placeholders). So Stefan came up with a proposal, which we accepted, to add complete i18n support to the library. We now have a mechanism by which you can hook up a translation engine of your choice, and JSON forms will use that to lookup messages, before falling back to the validator (also localised!) and finally, the JSON Schema’s defaults. This gives much stronger integration and allows the community to plug in their existing localisation methods - no wasted effort.

Since the localisation addition, we’ve been working on fine-tuning the layout engine, making bug fixes, and integrating more closely with the underlying Vuetify library. This allows developers to more easily use the existing Vuetify parameters to change the style and behaviour of their form widgets. Again, no wasted effort.

We’re lucky to have an active community - @kchobantonov continues to make great contributions and push the library forward in unexpected ways - and the library is gaining popularity, with an average of a few hundred downloads per day.

Some of our funder members have already seen this work in action, and given their feedback on early iterations of the user interface that supports registering grant records. We’ll be releasing this publicly very soon to get feedback from members - and then using that feedback to iterate on the grants registration form, and look towards extending it to other record types.

Open source POSItivity

A continuous theme throughout this project has been the willingness of people working on these open source projects to be generous with their time and experience. Whether it has been form generation libraries, the JSON Schema project or maintainers of localisation plug-ins - help, advice and encouragement have never been far away. And that’s appreciated. But it’s not something that we, or any other organisation who relies on the software they produce, should take for granted. Open source software helps everyone who uses it, and there’s a real opportunity within our community to make meaningful steps towards supporting its sustainability. Ironically, it’s often the most-used general purpose tools that get the least attention. We can change that.

Look out for more

Look out for more posts from the engineering team, coming soon!

References

JSON Binpack: A space-efficient schema-driven and schema-less binary serialization specification based on JSON Schema (Chapter 3.2.1 History and Relevance)

https://web.archive.org/web/20071026190426/http://www.json.com/2007/09/27/json-schema-proposal-collaboration/

Accessibility for Crossref DOI Links: Call for comments on proposed new guidelines

Jennifer Kemp — Tue, 06 Sep 2022 00:00:00 +0000

Our entire community – members, metadata users, service providers, community organisations and researchers – create and/or use DOIs in some way so making them more accessible is a worthy and overdue effort.

For the first time in five years and only the second time ever, we are recommending some changes to our DOI display guidelines (the changes aren’t really for display but more on that below). We don’t take such changes lightly, because we know it means updating established workflows. We appreciate the questions that prompted us to make this recommendation and we know it’s critical that we get community input on the proposed updates.

TL;DR

Here is a quick overview:

DOIs and URLs themselves don’t really tell readers much. People with visual impairments rely on screen readers to read out loud the contents of a page. We’re asking for the title of each DOI to be added, in an ARIA (Accessible Rich Internet Applications) attribute, so these users understand what these links are for.
Accessible text, as this kind of description is known, should be included for all links, but at this time, we’re specifically recommending it for landing pages of newly registered records.
It’s not required, yet. We’re proposing a 2 year recommendation period and we want your feedback on the particulars, including timing and how we can help. Please take a short survey and/or get in touch and share your thoughts.
We’ll finalize these recommendations after assessing the feedback. Please check back for updates.

What is changing, when and why

The proposed updates are meant to improve overall usability, particularly for people with visual impairments, by aligning our guidelines with modern accessibility requirements such as the new W3C recommendations and the European Accessibility Act. This means that assistive technologies such as screen readers can interpret DOI links.

Why are changes being recommended?

DOIs are unique and persistent links to items in the scholarly record so it makes sense that they link to the full URLs for the associated content –for example, a journal article. The issue for people who rely on screen readers is that a DOI link doesn’t provide title or other information to give that link context. Users of screen readers need to know what the destination of a link is.

These users often lack the context that other users have; in fact, they may be presented with links in a document as a list. That’s why all links, not just DOI links, need what is called “accessible text.” Providing additional information for links requires ARIA (Accessible Rich Internet Applications) techniques. This speaks to the Web Content Accessibility Guidelines (WCAG), the standard guidelines for accessibility across the web, specifically success criterion 2.4.4 - Link Purpose (In Context), which aims to ‘help users understand the purpose of each link so they can decide whether they want to follow the link.’

For your feedback: recommended draft changes

We recommend the addition of an aria-label attribute for DOI links, containing as its value the descriptive title of the content represented by the DOI, so that screen readers can interpret DOI links. This means that, while the DOI display itself doesn’t actually change, the link is enhanced with additional, contextual information for the user of assistive technology, in one of two ways, either:

an aria-label attribute, described as ‘a way to place a descriptive text label on an object,’ identifying the destination, or
an aria-describedby attribute pointing to where the destination is identified in the surrounding text.

The updated HTML for a journal article*, for example, would be:

<a href="https://doi-org.pluma.sjfc.edu/10.5555/12345678" aria-label="DOI for Toward a Unified Theory of High-Energy Metaphysics: Silly String Theory">https://doi-org.pluma.sjfc.edu/10.5555/12345678</a>

Here the aria-label has been set to the value of the ‘title’ property as retrieved from the Crossref REST API at https://api-crossref-org.pluma.sjfc.edu/v1/works/10.5555/12345678.

*Note that fields may vary slightly for different record types.

This proposed solution allows screen readers to read aloud to users the value of the aria-label attribute, instead of the full DOI in the link text.

At this time, we are recommending the change for landing pages in particular, but it can and should be applied to wherever DOI links appear, whenever feasible (more on this below).

Our guidelines will continue to state that the DOI should always be displayed as a full URL link–that will not change. Neither will content registration–we are not asking for additional information in your deposits.

It’s not perfect, but it’s very worthwhile

This recommendation has some limitations worth noting but it must be said that there is no perfect solution.

DOI links appear in lots of places - PDFs for one notable example. We reviewed and tested the recommendation with Bill Kasdorf, Principal, Kasdorf & Associates, LLC, Richard Orme, CEO, DAISY Consortium, and George Kerscher, Chief Innovations Officer, DAISY Consortium-Senior Officer, Global Literacy, Benetech, who graciously provided their time and expertise. EPUBs and websites proved to be easy to update; other formats, notably PDFs, less so. Widespread adoption of accessible DOIs is so important and we don’t want confusion or frustration to get in the way of making progress. We support and welcome efforts to include an ARIA attribute wherever DOI links appear, but we recommend focusing on landing pages, for now.

Patrick Vale, Crossref Senior Front End Developer, explains that:

”DOI links serve a very specific purpose: to provide the persistent link to an item in the scholarly record. And as such, they present an unusual set of requirements when balancing accurately presenting the information they encode - the persistent link - and making that link accessible, and understandable. With these proposed changes, we hope to strike this balance.“

We know it will be a challenge (more on that below) but we think it’s absolutely a worthwhile effort. Indeed, we are undertaking a project to update our own website to meet these recommendations and to review overall accessibility.

As Bill Kasdorf notes:

“Most people have no idea how many people with visual impairments there are. Not only is it unfair to those people not to provide accessible text for links, the authors and publishers of the linked resource are missing a lot of readers. This update is a great move by Crossref, and every bit aligned with its mission to make scholarly content discoverable and consumable.”

We propose the following timeline, also for your feedback

Once finalized, following community feedback, the updated guidelines will be issued as a recommendation for a suggested period of two years starting next year, 2023. Beginning in 2025, the changes will be required for landing pages of newly registered content (and strongly recommended for existing registered content). Feedback on this approach and timeline is also encouraged.

Help us help you

We are conscious that adding descriptive information to DOI links places a significant responsibility on the members and Service Providers creating and hosting these links. Therefore, we are also considering the creation of a tool to help with implementation. Initial discussions suggest this could be a JavaScript helper tool, which could be included on member websites. We also welcome feedback as to how such a tool might be implemented, and how it would best integrate with existing sites and workflows.

Call for comments - by 1st November

We hope that this proposal is a welcome one and that the timing is good for moving forward together toward greater accessibility of the scholarly record. We welcome questions, feedback and suggestions through 1st November via the survey below or by email to feedback@crossref.org

Small changes, big impact

We’re excited to make changes that improve accessibility and we look forward to the community’s response to our proposal. We will share aggregated feedback in an updated post later this year.

A note on language

Multiple sources were consulted to find the most appropriate and inclusive term(s) for users of screen readers in this context. “Print disabled,” for example, seemed to be a good candidate but was ultimately deemed likely to be confusing to a very global publishing audience, who often don’t physically print anything. Sources differ slightly, for example between the US and UK and of course, this English text may well be translated into other languages. Feedback on the terms used here is also very welcome.

Additional resources

The Inclusive Publishing Hub (DAISY Consortium)
National Center on Disability and Journalism (Arizona State University, US)
Inclusive Language guidance (UK government)
The American Psychological Association (APA) Bias-Free Language Disability Guide
The Open Access Books Network (OABN)

Stepping up our deposit processing game

Isaac Farley — Mon, 08 Mar 2021 00:00:00 +0000

Some of you who have submitted content to us during the first two months of 2021 may have experienced content registration delays. We noticed; you did, too.

The time between us receiving XML from members, to the content being registered with us and the DOI resolving to the correct resolution URL, is usually a matter of minutes. Some submissions take longer - for example, book registrations with large reference lists, or very large files from larger publishers can take up to 24 to 48 hours to process.

However, in January and February 2021 we saw content registration delays of several days for all record types and all file sizes.

Tell me more

Januaries and Februaries are usually busy at Crossref. Journal ownership changes hands. Members migrate from one platform to another (and can need to update tens of thousands of their resolution URLs). And, many of you are registering your first issues, books, or conferences of the year. Others of you have heard the calls of The Initiative for Open Citations (I4OC) and The Initiative for Open Abstracts (I4OA) and are enriching your metadata accordingly (thank you!). Tickets into our support and membership colleagues peak for the year. But did we see significantly more submissions this year?

As you can see, we did see larger-than-normal numbers of submissions in the first two months of the year. For the entire month of January 2021, we received nearly 1 million more submissions into our admin tool deposit queue than we did in January 2020 (2,757,781 in 2021 versus 1,848,261 in 2020). Under normal circumstances, this would lead to an increase in our processing times, so there’s that to consider. But there was also something else at play this year. We desperately needed to upgrade our load balancer, and so we did. Unfortunately, unforeseen at the time, these upgrades caused hiccups in our deposit processing and slowed down submissions even further, building up the number of unprocessed submissions in the queue.

When we saw the impact this was having we suspended the load balancer work until things were stable again. We also increased the resources serving our queue to bring it back down to normal. To make sure we don’t face the same problem again, we have put in better tools to detect trends in queue usage- tools which, in turn, will allow us to anticipate problems in the queue instead of reacting to them after they’ve already occurred. And as a longer-term project, we are addressing two decades of technical debt and rearchitecting our system so that our entire system is much more efficient.

Gory technical details

As part of our effort to resolve our technical debt, we’re looking to transition more of our services to the cloud. To accomplish this, we first needed to upgrade our internal traffic handling capabilities to route things to their new locations better. This upgrade caused some unforeseen and hard to notice problems, like the queue being stalled. Since the queue still showed things in process, it wasn’t immediately apparent that things were not processing (normally the processing on the queue will clear a thread if a significant problem occurs).

We initially noticed a problem on 5 February and thought we had a fix in place on the 10th. But, we again realized on 16 February that the underlying problem had recurred, and we needed a closer investigation.

For many reasons it took us too much time to realize the connection, until people started complaining.

While our technical team worked on those load balancer upgrades, some of your submissions lingered for days in the deposit queue. In a few examples, larger submissions took over a week to complete processing. Total pending submissions began to push nearly 100,000, an unusually large backlog. We called an emergency meeting, paused all related work, and dedicated additional time and resources to processing all pending submissions. On 22 February, we completed working through the backlog of pending submissions and new submissions were being processed at normal levels. As we finish up this blog on 2 March, there are less than 3,000 pending submissions in the queue, the oldest of which has been there for less than three hours.

This brings us back to the entire rationale for what we are doing with the load balancer - which, ironically, was to move some services out of the data centre so that we could free-up resources and scale things more dynamically to match the ebbs and flows of your content registration.

But before we proceed, we’ll be looking at what happened. The bumps associated with upgrading ancient software were expected, so we were looking for side effects. We just didn’t look in the right place. And we should have detected that the queues had stalled well before people started to report it to us. A lot of our queue management is still manual. This means we are not adjusting it 24x7. So if something does come in when we are not around, it can exacerbate problems quickly.

What are we going to do about it?

In a word: much. We know that timely deposit processing is critical. We can and will do better.

First off, we have increased the number of concurrently processing threads dedicated to metadata uploads in our deposit queue from 20 to 25. That’s a permanent increase. A million more submissions in a month necessitates additional resources, but that’s only a short-term patch. And we were only able to make this change recently due to some index optimizations we implemented late last year.

One of the other things that we’ve immediately put into place is a better system for measuring trends in our queue usage so that we can, in turn, anticipate rather than react to surges in the queue. And, of course, the next step will be to automate this queue management.

All this is part of an overall, multi-year effort to address a boat-load of technical debt that we’ve accumulated over two decades. Our system was designed to handle a few million DOIs. It has been incrementally poked and prodded to deal with well over a hundred million. But it is suffering.

Anybody who is even semi-technically-aware might be wondering what all the fuss is about? Why can’t we fix this relatively easily? After all, 130 million records—though a significant milestone for Crossref—does not in any way qualify as “big data.” All our DOI records fit onto an average sized micro-SD card. There are open source toolchains that can manage data many, many times this size. We’ve occasionally used these tools to load and analyse all our DOI records on a desktop computer. And it has taken in just a few minutes (admittedly using a beefier-than-usual desktop computer). So how can a queue with just 100,000 items in it take so long to process?

Our scale problem isn’t so much about the number of records we process. It is about the 20 years of accumulated processing rules and services that we have in place. Much of it undocumented and the rationale for which has been lost over the decades. It is this complexity that slows us down.

And one of the challenges we face as we move to a new architecture is deciding which of these rules and services are “essential complexity” and which are not. For example, we have very complex rules for verifying that submissions contain a correct journal title. These rules involve a lot of text matching and, until they are successfully completed, they block the rest of the registration process.

But the workflow these rules are designed for is one that was developed before ISSNs were widely deposited and before we had our own, internal title identifiers for items that do not have an ISSN. And so a lot of this process is probably anachronistic. It is not clear which (if any) parts of it are still essential.

We have layers upon layers of these kinds of processing rules, many of which are mutually dependent and which are therefore not easily amenable to the kind of horizontal scaling that is the basis for modern, scalable data processing toolchains. All this means that, as part of moving to a new architecture, we also have to understand which rules and services we need to move over and which ones have outlived their usefulness. And we need to understand which remaining rules can be decoupled so that they can be run in parallel instead of in sequence.

Pro tip: Due to the current checks performed in our admin tool, for those of you submitting XML, the most efficient way to do so is by packaging the equivalent of a journal issue’s worth of content in each submission (i.e., ten to twelve content items - a 1 MB submission is our suggested file size when striving for efficient processing)

Which brings us conveniently back to queues. We did not react soon enough to the queue backing up. We can do much better at monitoring and managing our existing registration pipeline infrastructure. But we are not fooling ourselves into thinking this will deal with the systemic issue.

We recognize that, with current technology and tools, it is absurd that a queue of 100,000 items should take so long to process. It is also important that people know that we are addressing the root of the issues as well. And that we’re not succumbing to the now-legendary anti-pattern of trying to rewrite our system from scratch. Instead we are building a framework that will allow us to incrementally extract the essential complexity of our existing system and discard some of the anachronistic jetsam that has accumulated over the years.

Content Registration should typically take seconds. We wanted to let you know, that we know, and we are working on it.

A wrap up of the Crossref blog series for SciELO

Christine Cormack Wood — Fri, 05 Oct 2018 00:00:00 +0000

Crossref member SciELO (Scientific Electronic Library Online), based in Brazil, celebrated two decades of operation last week with a three-day event The SciELO 20 Years Conference.

The celebration constituted an important landmark in SciELO’s evolution, and an exceptional moment for them to promote the advancement of an inclusive, global approach to scholarly communication and to the open access movement.

As part of the anniversary activities SciELO asked us to write a series of five blogs that would help the organisations of Brazil to better understand the following:

Why all articles should have a DOI
The critical role of the DOI
The basics of record types, translations, preprints, Crossmark, and more
The basics of Crossref sponsorship, and
How to make the most of your Crossref membership

Below you’ll find an abstract of each of these blog posts as well as a link to the published posts in Brazilian Portuguese, Spanish and English.

Why all articles should have a DOI
In today’s world, an author’s work needs a Digital Object Identifier (DOI) for it to become discoverable, citable, and linkable. This unique alphanumeric string identifies the content of a research work, and remains associated with it irrespective of changes to its web location. Discover the origins of the DOI, how Crossref was founded, and why they continue to exist and persist.

Read the full blog in Brazilian Portuguese, Spanish, or English

The critical role of the DOI
Find out why URL links to research articles are fragile, and how DOIs are essential in building stable, persistent links between research objects. This is achieved through the metadata that members deposit with Crossref, as part of their obligations. Learn how we can all contribute to creating a global, robust research record.

Read the full blog in Spanish or English

The basics of record types: Preprints, Crossmark, translations, and more
What’s the difference between preprints and ahead of print? When should you use each; and, what are the DOI requirements? This article answers those questions and provides a basic overview of how to connect the metadata records of related record types, like translations.

Read the full blog in Brazilian Portuguese, Spanish, or English

The basics of Crossref sponsorship
There are many organisations that want to register content and benefit from the services Crossref provides, but may not be able to do so alone. These organisations use sponsors. Sponsors are organisations who publish on behalf of groups of smaller organisations. Nearly 650 of our 800 Brazilian members are represented by such a sponsor.

Read the full blog in Brazilian Portuguese, Spanish, or English

How to make the most of your Crossref membership
Since Crossref was founded in 2000, its member organisations have registered metadata and persistent identifiers (DOIs) for over 100 million content items. This information is used extensively by the research community—individuals and organisations—who need to find, cite, link and assess research outputs. As a SciELO member, the metadata you provide to Crossref when you register content is key to the discoverability of your journal content.

Read the full blog in Brazilian Portuguese, Spanish, or English

Revised Crossref DOI display guidelines are now active

Ed Pentz — Wed, 15 Mar 2017 00:00:00 +0000

We have updated our DOI display guidelines as of March 2017, this month! I described the what and the why in my previous blog post New Crossref DOI display guidelines are on the way and in an email I wrote to all our members in September 2016. I’m pleased to say that the updated Crossref DOI display guidelines are available via this fantastic new website and are now active. Here is the URL of the full set of guidelines in case you want to bookmark it (https://doi-org.pluma.sjfc.edu/10.13003/5jchdy) and a shareable image to spread the word on social media.

This blog is a quick reminder that all Crossref members should now be displaying DOIs in the recommended new format from this month, on any new content you publish online. Please note these guidelines are for Crossref DOIs only, we have nearly 90 million registered but there are others, and not all DOIs are made equal.

The main changes are to display the DOI as a full, linked URL using HTTPS:

https://doi-org.pluma.sjfc.edu/10.xxxx/xxxxx

For background on the HTTPS issue please read Geoffrey Bilder’s blog post, Linking DOIs using HTTPS.

What will happen if you don’t update your Crossref DOI display?

We tell members that they should be working towards making the change even if they can’t do it until later - we recognize that it is not always an easy change to make.

However, if members don’t make the change, nothing immediate will happen (Crossref won’t fine you!) although as more members make the change your display will look odd and out of place compared with other members’ content.

If you have any questions please do not hesitate to contact us.

Taking the "con" out of conferences

Geoffrey Bilder — Wed, 15 Feb 2017 00:00:00 +0000

TL;DR

Crossref and DataCite are forming a working group to explore conference identifiers and project identifiers. If you are interested in joining this working group and in doing some actual work for it, please contact us at community@crossref.org and include the text conference identifiers WG in the subject heading.

All the times I could have gone to Walt Disney World…

Back around 2010 I added a filter to my email settings that automatically flagged and binned any email that contained the word “Orlando.” Back then this was a remarkably effective way of detecting and ignoring spam from the numerous fake technology conferences that all seemed to advertise the city of Orlando, Florida as the location for their non-events. I suspected they all chose Orlando as it would provide the punter that little bit of extra motivation to pay and register for the conference as they simultaneously plotted how they could tag-on some holiday time at Walt Disney World. I finally had to remove the filter last year when I realised that the scammers had moved on to advertising more realistically gritty cities in their calls for submissions and that meanwhile I had managed to miss all the mail informing me of the ALA’s summer 2016 meeting held in, you guessed it… Orlando.

Clearly we need better mechanisms to flag dubious conferences.

Late last year Crossref’s Strategic initiatives group was approached by “CounterMock,” a group of Crossref members (including major proceedings publishers like Springer Nature, Elsevier, IEEE, ACM, IET, etc) who were actively exploring the establishment of an identifier system and registry for scholarly conferences.

The long term goal of the group is to make it easier for publishers, researchers and other stakeholders to identify fraudulent and/or low-quality conferences. There has recently been a proliferation of conferences that seem to have been developed specifically to dupe international and early-career researchers into paying substantial conference and publication fees. Sometimes these conferences are intentionally named after long-standing and well-respected conferences. At worst these conferences are entirely fake - no meetings are held and no publications are issued. At best they produce subpar publications of questionable academic integrity. Members of the group are concerned that these “mock conferences” (Hence “COUNTERMOCK”) will:

Waste researcher time.
Waste publisher time.
Undermine academic trust in conferences and conference proceedings as a trustworthy means of scholarly communication.

The group understands that the “evaluation of a conference quality” and the “unambiguous identification of conferences” are separate concerns (as they are with publications, contributors, etc). But they also realise that it will be hard to address the quality issue without an infrastructure for unambiguously identifying conferences and providing meaningful provenance metadata about those conferences. Moreover, having unique identifiers for conference series would enable a number of other applications. Examples include conference-level metrics, better and more structured info about forthcoming conferences on a certain topic, and more visibility of conferences in research evaluation.

Springer Nature has built a POC prototype of a conference identifier system and shown it to a number of other parties. The feedback has been that there is interest in the project, but that the consensus is that it should be managed a run by a neutral industry group. They have approached us to form a working group and explore how this project can be advanced.

This is all good. Crossref itself doesn’t make value judgements on the quality of content registered with us. Crossref DOIs are not quality marks. But we do believe that unambiguous identification of research artifacts is a perquisite to building effective trust and reputation tools.

It is possible that the issue of conference identifiers can be folded into the work we are doing with DataCite and ORCID on organisation identifiers. For example, some have argued that organisation identifiers should include identifiers for projects or other less formal and more ephemeral corporate entities that are often included in affiliation and/or bibliographic data. It is possible to make the similar arguments in the case of conferences.

On the other hand we have also been interested in the issue of “project identifiers.” Martin Fenner and Tom Demeranville have made a strong argument that ‘projects’ can be thought of as containers for collections of project outputs, project members and project funders. Again, it seems plausible that one could make the same case for conferences.

At the very least it is important to coordinate any work that is done on conference, project and organisation identifiers. This why we have decided to form a joint Crossref/DataCite working group to specifically explore conference and project identifiers and determine how they relate both to each other and to our already ongoing work with ORCID on organisation identifiers.

Additionally, it is likely that the working group will discuss and explore how conference/project identifiers might be used for increasing the transparency of peer review at conferences, better attribution for programme chairs and program committee members, and how they might be incorporated into other services like Crossref Metadata Search, DataCite search, CrossMark, etc.

If you are interested in doing some work on this- then please indicate your interest in joining a working group by sending email to community@crossref.org and include the text conference identifiers WG in the subject heading.

We will update this blog as the group convenes and makes progress.

Linking DOIs using HTTPs: the background to our new guidelines

Geoffrey Bilder — Tue, 17 Jan 2017 00:00:00 +0000

Recently we announced that we were making some new recommendations in our DOI display guidelines. One of them was to use the secure HTTPS protocol to link Crossref DOIs, instead of the insecure HTTP.

Some people asked whether the move to HTTPS might affect their ability to measure referrals (i.e. where the people who visit your site come from).

TL;DR: Yes

Yes. If you do not move your DOI links to HTTPS, Crossref, its members and the members of other DOI registration agencies (e.g. DataCite, JLC, CNKI) will find it increasingly difficult to accurately measure referrals. You should link DOIs using HTTPS.

In fact, if you do not support HTTPS on your site now, it is likely that your ability to measure referrals is already impaired. If you do not already have a plan to move your site to HTTPS, you should develop one.

If you have already transitioned your site to HTTPS, you should follow the new guidelines and link DOIs via HTTPS as soon as possible. As it stands, you are not sending any referrer information when DOIs are clicked on and followed from your site. You should also make sure that the URLs you have registered with Crossref are HTTPS URLs, otherwise you will not get referrer information on your site when they are followed.

Read on if you want some grody details. We’ll try to keep it as non-technical as possible.

Two protocols, one web

To start with your web browser supports two closely related protocols, HTTP and HTTPS.

The first, HTTP, is the protocol that the web started out with. It is an unencrypted protocol and it is also easy to intercept and modify. It is also very easy and inexpensive to implement.

The second protocol, HTTPS, is a secure version of the first protocol. It is very difficult to intercept and modify. It has historically been more complex and expensive to implement.

Here you might say - “Great, but HTTPS has been around for a long time. We’ve used it for sensitive transactions like authentication and credit card transactions. Why do we want to use DOI links with HTTPS?” Why are you suggesting that we should even consider moving our entire site to HTTPS?

The pressure to move to HTTPS

The insecure HTTP protocol has become a major vector for a lot of security issues on the web. It allows user web pages to be intercepted and modified between the server and the browser. This flaw is being abused for everything from spying, to inserting unwanted advertisements into web pages, to distributing viruses, ransomware and botnets.

As such, there has been a steady drumbeat of industry encouragement to move to the more secure HTTPS protocol for all website functions.

We are not going to argue all the points here. Instead we will mention the major constituencies that are advocating for a move to HTTPS and provide you with some pointers. We apologise that these are all so US-centric, but a lot of the web’s global direction does seem to be presaged by US adoption trends.

Google

It is probably easiest to start with Google, since its practices tend to focus the attention of those managing websites.

Back in 2014 Google announced that they would slowly move toward including the use of HTTPS as a ranking signal. In 2015 they upped the ante by announcing that they would start indexing HTTPS versions of pages by default. It looks like in early 2017 they will really start to take the gloves off as they modify their Chrome browser to flag sites that do not use HTTPS as being insecure.

Every top website, evah

It looks like Google's plan is working too. Their 2016 transparency report shows that most top websites have already transitioned to HTTPS and that this translates to approximately 25% of all web traffic worldwide taking place using HTTPS. Indeed, over 50% of all web pages viewed by desktop users are delivered via HTTPS.

Government agencies

The USA’s Whitehouse issued [a directive instructing all Federal websites to adopt HTTPS]. As of December 2016 64% of federal websites have made the transition.

Libraries

Much of the pressure to move to HTTPS is coming from the library community who have a historical tradition of protecting patron privacy and resisting efforts to censor content. The third principle of the American Library Association's code of ethics reads:

We protect each library user’s right to privacy and confidentiality with respect to information sought or received and resources consulted, borrowed, acquired or transmitted.

Recently there has been a major push by the Electronic Frontier Foundation to get libraries to adopt a number of security and privacy practices, including the use of HTTPS by all library systems as well as those used by library vendors.

What are Crossref members doing about HTTPS?

How big an issue is this? How many of our members have moved to HTTPS? How many plan to? Well, we looked at the URLs that are registered with Crossref and we tested them with both protocols. Eventually we will write a blog post detailing our findings - but the highlights are:

Slightly fewer than half of the member domains tested only support HTTP.
Slightly fewer than half of the member domains tested support both HTTP and HTTPS.
About 370 of the member domains tested only support HTTPS.

The transition to HTTPS and the issue of DOI referrals

The HTTP referrer is a piece of information passed on by a browser that indicates the site from which the user navigated.

So, for example, if a user visiting site A clicks on a link which takes them to site B, site B will then record in its logs that a user visited them from site A. Obviously, this is important information for understanding where your web site traffic comes from.

The default rules for referrals are¹:

If you link between two sites with the same level of security, all referral information is retained.
When you follow a link from an insecure (HTTP) web site to a secure (HTTPS) site, referral data is passed on to the secure web site.
If you follow a link from a secure (HTTPS) web site to an insecure (HTTP) site, referral data is not passed on to the insecure web site.

So let's see what the situation would look like with normal links. If we had two sites, `A` & `B`, the following table maps the possible combinations of protocols that can be used to link from `A` to `B`. So, for example, row #2 reads:

A user browses site A using HTTP and clicks on a HTTPS link to publisher B who hosts their site using HTTPS.

The last column indicates if the referrer information is passed along by the browser. In the case of row #2, the answer is “yes”. The user has navigated from a less secure site to a more secure site.

User views site A using	Site A links to site B using	Browser reports referrer to site B
HTTP	HTTP	Yes
HTTP	HTTPS	Yes
HTTPS	HTTP	No
HTTPS	HTTPS	Yes

But this gets a little more complicated with DOIs. In this case publisher `A` links to publisher `B` through the DOI system. This means there are two parts to the link. The first `(A->doi.org)` results in a redirect (A->B). Again we use the last columns to indicate when referrer information is passed along to site B. Again, let’s look at row #2. It reads:

A user browses the site of member A using HTTP and clicks on a HTTP DOI link. The DOI system redirects the browser to member B using an HTTPS link registered with Crossref by member B. The middle column and the last column records whether Crossref and the publisher were able to see referrer information. The answer in both cases is “yes”. In the first case (A->DOI) because the link was from a less secure site (HTTP on A) to a more secure site (HTTPS at DOI). The second case because the link is between two sites at the same security level (HTTP).

	User views site A using	Site A links DOI using	Browser reports referrer to Crossref²	Crossref redirects to site B using³	Browser reports referrer to site B
1	HTTP	HTTP	Yes	HTTP	Yes
2	HTTP	HTTP	Yes	HTTPS	Yes
3	HTTP	HTTPS	Yes	HTTP	Yes
4	HTTP	HTTPS	Yes	HTTPS	Yes
5	HTTPS	HTTP	No	HTTP	No
6	HTTPS	HTTP	No	HTTPS	No
7	HTTPS	HTTPS	Yes	HTTP	No
8	HTTPS	HTTPS	Yes	HTTPS	Yes

So what does this mean?

Our old display guidelines recommended linking DOIs using HTTP. Rows #1, #2, #5, #6 represent the status quo.

About half of our members support HTTPS. A few support it exclusively and it seems, given the industry pressures mentioned above, those who support both protocols are likely doing so as a transition stage to HTTPS-only sites.

This means that the scenarios represented in row #5 & #6 are already happening. The referral information for any user viewing one of our member sites using HTTPS is being lost when they click on DOIs that use the HTTP protocol. Crossref doesn’t get the referral data and neither does the member whose DOI has been clicked on.

Of course this applies to non-member sites that link to DOIs as well. Wikipedia is the largest referrer of DOIs from outside the industry. In 2015 The Wikimedia Foundation made a highly publicised transition to HTTPS on all of their sites. This means that any of our members who are running HTTP sites have already lost the ability to see any referral information from Wikipedia on their own sites. However, Crossref worked closely with Wikimedia to ensure that, at the very least, Crossref was still able to record Wikimedia referral data on behalf of our members.

A solution

It is largely this work with Wikimedia that has helped us to understand just how important it is for Crossref to get ahead of the curve in helping our community to transition to HTTPS.

As long as our members are running a combination of HTTP and HTTPS sites, there is no way for our community to avoid some disruption in the flow of referral data. And we certainly would never entertain the notion of asking our members to keep using HTTP.The best we can do is recommend a practice that will help smooth the transition to HTTPS. That is what we are doing.Our new recommendation is to move to linking DOIs using HTTPS. This is represented in rows #3, #4, #7 and #8 in the table above.

This is a particularly important step for our members who have already moved to hosting their sites on HTTPS. As long as they are using HTTP DOIs on their site, they will be sending no referral traffic to Crossref, other Crossref members or other users of the DOI infrastructure. This is captured in scenarios #5 and #6.

If our linking guidelines are followed during the industry’s transition to HTTPS, then scenario #5 and #6 will eventually be replaced with scenario #7. It is still not perfect, but at least it means that, during the transition, publishers who are still running HTTP sites will be able to get some DOI referral data via Crossref. And of course, once our members have widely transitioned to HTTPS, everything will go back to normal and they will be able to see referral data on their own sites as well (i.e.they will have moved from the state represented in row #1 to state represented in row #8.)

In summary, please change your sites to use HTTPS to link DOIs. They should look like this:

https://doi-org.pluma.sjfc.edu/10.7554/eLife.20320

FAQ

Q: If I have moved my site to HTTPS, do I need to redeposit my URLs to that they use the HTTPS protocol instead?

A: Yes. If you want to be able to still collect referrer information on your site (scenario #8) as opposed to via Crossref (scenario #7).

Q: But can’t I avoid redepositing my URLs and get referrer data again if I simply redirect HTTP URLs to HTTPS on my own site?

A: No. The browser will strip referrer information if there is any HTTP step in the redirects. Even if the redirect is done on your own site.

Q: Can I avoid having to redeposit all my URLs? Can’t Crossref just update the protocol on our existing DOIs for us?

A: Contact support@crossref.org. We’ll see what we can do.

Q: What about all the old PDFs that are are there? They link to DOIs using HTTP.

A: That is true. But links followed from PDFs don’t send referrer information anyway.

Q: And what about my new PDFs? Should I start linking DOIs from them using HTTPS.

A: Probably. But not because of the DOI referrer problem. Simply because HTTPS is a more secure, private, and future-proof protocol.

Q: Don’t some countries block HTTPS?

A: Typically countries block specific sites and/or services. We do not know of any countries that have a blanket block on the HTTPS protocol.

Q: I use a link resolver that uses OpenURL + a cookie pusher to redirect my users to local resources. What do I need to do?

A: You need to change your cookie pusher script to enable the Secure attribute for cookies for HTTPS-linked DOIs.

Q: Can I use protocol-relative URLs (e.g. //doi.org/10.7554/eLife.20320)?

A: Protocol-relative URLs can be used in HTML HREFs to help ease the transition from HTTP to HTTPS, but use the full protocol in the text of the DOI link itself. So, for example, the following is fine:

https://doi-org.pluma.sjfc.edu/10.7554/eLife.20320

Q: I hear that HTTP and HTTPS versions of URI identifiers are considered to be different identifiers. Doesn’t this mean that by moving to HTTPS we are essentially doubling the number of DOI-based identifiers out there?

A: Yes. It isn’t a problem that is only being faced by DOIs. Basically all HTTP-URI based identifiers face the same issue. We will put in place appropriate same-as assertions in our metadata and HTTP headers to allow people to understand that the HTTP and HTTPS representations of the DOI point to the same thing.

On a personal note (@gbilder speaking- don’t blame @CrossrefOrg) - it breaks my brain that the official line is that the protocol difference means they are different identifiers. As a practical matter (a concept the W3C seems to be increasingly alienated from), it would be insane for anybody to follow this policy to the letter. You can probably be pretty safe swapping the protocols on DOIs and being sure you will get the same thing.

Q: I see that the Crossref site isn’t running on HTTPS. Are you just a bunch of hypocrites?

A: ~~Yes. The site will be moving to HTTPS-only very soon. Then we won’t be.~~ We do now.

References

These rules can be tweaked using meta referrer tags (https://www.w3.org/TR/referrer-policy/), but not in any way that both avoids the fundamental problems outlined here and that preserves the security/privacy characteristics that are the very reason to implement HTTPS in the first place.
To be pedantic- it actually passes referrer information to the DOI proxy (https://doi-org.pluma.sjfc.edu/), which in turn is reported to Crossref.
To continue with the pedantry- the DOI proxy does the redirect based on the URL member B has deposited with Crossref.

New Crossref DOI display guidelines are on the way

Ed Pentz — Tue, 27 Sep 2016 00:00:00 +0000

TL;DR

Crossref will be updating its DOI Display Guidelines within the next couple of weeks. This is a big deal. We last made a change in 2011 so it’s not something that happens often or that we take lightly. In short, the changes are to drop “dx” from DOI links and to use “https:” rather than “http:”. An example of the new best practice in displaying a Crossref DOI link is: https://doi-org.pluma.sjfc.edu/10.1629/22161

Hey Ho, “doi:” and “dx” have got to go

The updated Crossref DOI Display guidelines recommend that https://doi-org.pluma.sjfc.edu/ be used and not http://dx.doi.org.pluma.sjfc.edu/ in DOI links. Originally the “dx” separated the DOI resolver from the International DOI Foundation (IDF) website but this has changed and the IDF has already updated its recommendations so we are bringing ours in line with theirs.

We are also recommending the use of HTTPS because it makes for more secure browsing. When you use an HTTPS link, the connection between the person who clicks the DOI and the DOI resolver is secure. This means it can’t be tampered with or eavesdropped on. The DOI resolver will redirect to both HTTP and HTTPS URLs.

Timing and backwards compatibility

We are requesting all Crossref member publishers and anyone using Crossref DOIs to start following the updated guidelines as soon as possible. But realistically we are setting a goal of six months for implementation; we realize that updating systems and websites can take time. We at Crossref will also be updating our systems within six months - we already use HTTPS for some of our services and our new website (coming very soon!) will use HTTPS.

An important point about backwards compatibility is that “http://dx.doi.org.pluma.sjfc.edu/” and “http://doi.org.pluma.sjfc.edu/” are valid and will continue to work forever-or as long as Crossref DOIs continue to work-and we plan to be around a long time.

We need to do better

Reflecting on the 2011 update to the display guidelines it’s fair to say that we have been disappointed. It is still much too common to see unlinked DOIs in the form doi:10.1063/1.3599050 or DOI: 10.1629/22161 or even unlinked in this form: http://dx.doi.org.pluma.sjfc.edu/10.1002/poc.3551

What’s so wrong with this approach? To demonstrate, please click on this DOI doi:10.1063/1.3599050 - oh, you can’t click on it? How about I send you to a real example of a publisher page. What I’d like you to do is click the following link and then copy the DOI you find there and come back - http://dx.doi.org.pluma.sjfc.edu/10.1002/poc.3551.

Are you back? I expect you had to carefully highlight the “10.1063/1.3599050” and then do “edit”, “copy”. That wasn’t too bad but the next step is to put the DOI into an email and send it to someone. But wait - what are they going to do with “10.1063/1.3599050”? It’s useless. If you want it to be useful you’ll have to add “http://doi.org.pluma.sjfc.edu” or https://doi-org.pluma.sjfc.edu/ in the front.

When publishers follow the guidelines it makes things easier - if you go to https://doi-org.pluma.sjfc.edu/10.1063/1.3599050 you’ll note that you can just right click on the full DOI link on the page and get a full menu of options of what to do with it. One of which is to copy the link and then you can easily paste into an email or anywhere else.

However-putting a positive spin on the spotty adherence to the 2011 update to the DOI display guidelines-everyone has another chance with the latest set of updates to make all the changes at once!

More on HTTPS (future-proofing scholarly linking)

We take providing the central linking infrastructure for scholarly publishing seriously. Because we form the link between publisher sites all over the web, it’s important that we do our bit to enable secure browsing from start to finish. In addition, HTTPS is now a ranking signal for Google who gives sites using HTTPS a small ranking boost.

The process of enabling HTTPS on publisher sites will be a long one and, given the number of members we have, it may a while before everyone’s made the transition. But by using HTTPS we are future-proofing scholarly linking on the web.

Some years ago we started the process of making our new services available exclusively over HTTPS. The Crossref Metadata API is HTTPS enabled, and Crossmark and our Assets CDN use HTTPS exclusively. Last year we collaborated with Wikipedia to make all of their DOI links HTTPS. We hope that we’ll start to see more of the scholarly publishing industry doing the same.

So-it’s simple-always make the DOI a full link - https://doi-org.pluma.sjfc.edu/10.1006/jmbi.1995.0238 - even when it’s on the abstract or full text page of the content that the DOI identifies - and use “https://doi-org.pluma.sjfc.edu/”.

Using the Crossref Metadata API. Part 1 (with Authorea)

Rachael Lammey — Thu, 18 Aug 2016 00:00:00 +0000

Did you know that we have a shiny, not so new, API kicking around? If you missed Geoffrey’s post in 2014 (or don’t want a Cyndi Lauper song stuck in your head all day), the short explanation is that the Crossref Metadata API exposes the information that publishers provide Crossref when they register their content with us. And it’s not just the bibliographic metadata either-funding and licensing information, full-text links (useful for text-mining), ORCID iDs and update information (via Crossmark)-are all available, if included in the publishers’ metadata.

Interested? This is the kickoff a series of case studies on the innovative and interesting things people are doing with the Metadata API. Welcome to Part 1.

What can you do with the Metadata API?

Build search interfaces. We’ve built some ourselves. Check out Crossref Metadata Search to search the metadata of over 80 million journal articles, books, standards, datasets & more. Or Crossref Funder Search to search nearly 15,000 funders and the 982,162 records we have that contain funding data.

Provide cross-publisher support for text and data mining applications.

Get really interesting top-level reports on the metadata Crossref holds - or look at subsets of the information you’re interested in.

Third parties are free to build their own products and tools that build off of the Metadata API (below are some of the many examples that we will highlight in this series).

Importantly, there’s no sign-up required to use the Metadata API - the data are facts from members, therefore not subject to copyright and free to use for whatever purpose anyone chooses.

To help, Scott Chamberlain of rOpenSci has built a set of robust libraries for accessing the Metadata API. These libraries are now available in the R, Python and Ruby languages. Scott’s blog post has some great information on those. For those using the libraries, there have been a few updates since Scott’s post - to serrano, and support for field queries has been added to habanero (coming to serrano and rCrossref soon). Any feedback/bug reports can be submitted via the GitHub repos serrano or habanero. There’s also a javascript library, authored by Robin Berjon.

Who’s using the Crossref Metadata API?

We get around 30 million requests a month. We’d like to share a few case studies to showcase what they’re doing and how they’re using it. Look out for a series of posts over the next few months where we’ll open the floor to those using the API and let them explain how and why.

We’ll let Authorea kick things off…

Alberto Pepe, co-founder of Authorea explains:

Authorea is a word processor for researchers and scholars. It is a collaboration platform to write, share and openly research in real-time: write manuscripts and include rich media, such as data sets, software, source code and videos. The media-rich, data-driven capabilities of Authorea make it the perfect platform to create and disseminate a new generation of research articles, which are natively web-based, open, and reproducible. Authorea is free to use.

How is the Crossref Metadata API used within Authorea?

Authorea is specifically made for scholarly documents such as research articles, conference papers, grey literature, class notes, student papers, and problem sets. What makes scholarly documents so peculiar are their citations and references, mathematical notation, tables, and data. For citations and references, we built a citation tool which allows authors to search and cite scholarly papers with ease, without having to leave the editor. While in the middle of writing a sentence, authors can click the “cite” button and a citation tool opens up:

We currently use two engines for searching scholarly literature via their APIs: Crossref and Pubmed. Our authors love being able to search (by author name, paper title, topic, etc) and add references to their papers on the fly, in one click.

What are the future plans for Authorea?

Among the many plans we have for the future, there is one which is also tied to Crossref: we are going to let authors assign DOIs to Authorea articles such as blog posts, preprints, “data papers”, “software papers” and other kinds of grey literature which does not fit in the traditional scholarly journals.

What else would you like to see in our metadata?

Well, since you ask: we would love to see unique BibTex IDs being served by the Metadata API (right now, you create the ID automatically using author name and year). Also, in some cases, some important metadata fields are missing (even author or title). I think it is actually more important to fix existing metadata rather than add new fields!

Keen to share what you’re doing with the Crossref Metadata API? Contact feedback@crossref.org and share your story.

Using AWS S3 as a large key-value store for Chronograph

Joe Wass — Tue, 02 Aug 2016 00:00:00 +0000

One of the cool things about working in Crossref Labs is that interesting experiments come up from time to time. One experiment, entitled “what happens if you plot DOI referral domains on a chart?” turned into the Chronograph project. In case you missed it, Chronograph analyses our DOI resolution logs and shows how many times each DOI link was resolved per month, and also how many times a given domain referred traffic to DOI links per day.

We’ve released a new version of Chronograph. This post explains how it was put together. One for the programmers out there.

Big enough to be annoying

Chronograph sits on the boundary between normal-sized data and large-enough-to-be-annoying-size data. It doesn’t store data for all DOIs (it includes only those that are used on average once a day), but it has information on up to 1 million DOIs per month over about 5 years, and about 500 million data points in total.

Storing 500 million data points is within the capabilities of a well-configured database. In the first iteration of Chronograph a MySQL database was used. But that kind of data starts to get tricky to back up, move around and index.

Every month or two new data comes in for processing, and it needs to be uploaded and merged into the database. Indexes need to be updated. Disk space needs to be monitored. This can be tedious.

Key values

Because the data for a DOI is all retrieved at once, it can be stored together. So instead of a table that looks like

<td>
<span >2010-01-01</span>
</td>
<td>
<span >5</span>
</td>

<td>
<span >2010-02-01</span>
</td>
<td>
<span >7</span>
</td>

<td>
<span >2010-03-01</span>
</td>
<td>
<span >3</span>
</td>

10.5555/12345678

Instead we can store

<td>
{&#8220;2010-01-01&#8221;: 5, &#8220;2010-02-01&#8221;: 7, &#8220;2010-03-01&#8221;: 3}
</td>

10.5555/12345678

This is much lighter on the indexes and takes much less space to store. However, it means that adding new data is expensive. Every time there’s new data for a month, the structure must be parsed, merged with the new data, serialised and stored again millions of times over.

After trials with MySql, MongoDB and MapDB, this approach was taken with MySQL in the original Chronograph.

Keep it Simple Storage Service Stupid

In the original version of Chronograph the data was processed using Apache Spark. There are various solutions for storing this kind of data, including Cassandra, time-series databases and so on.

The flip side of being able to do interesting experiments is wanting them to stick around without having to bother a sysadmin. The data is important to us, but we’d rather not have to worry about running another server and database if possible.

Chronograph fits into the category of ‘interesting’ rather than ‘mission-critical’ projects, so we’d rather not have to maintain expensive infrastructure if possible.

I decided to look into using Amazon Web Services Simple Storage Service (AWS S3) to store the data. AWS itself is a key-value store, so it seems like a good fit. S3 is a great service because, as the name suggests, it’s a simple service for storing a large number of files. It’s cheap and its capabilities and cost scale well.

However, storing and updating up to 80 million very small keys (one per DOI) isn’t very clever, and certainly isn’t practical. I looked at DynamoDB, but we still face the overhead of making a large number of small updates.

Is it weird?

In these days of plentiful databases with cheap indexes (and by ‘these days’ I mean the 1970s onward) it seems somehow wrong to use plain old text files. However, the whole Hadoop “Big Data” movement was predicated on a return to batch processing files. Commoditisation of services like S3 and the shift to do more in the browser have precipitated a bit of a rethink. The movement to abandon LAMP stacks and use static site generators is picking up pace. The term ‘serverless architecture’ is hard to avoid if you read certain news sites.

Using Apache Spark (with its brilliant RDD concept) was useful for bootstrapping the data processing for Chronograph, but the new code has an entirely flat-file workflow. The simplicity of not having to unnecessarily maintain a Hadoop HDFS instance seems to be the right choice in this case.

Repurposing the Wheel

The solution was to use S3 as a big hash table to store the final data that’s served to users.

The processing pipeline uses flat files all the way through from input log files to projections to aggregations. At the penultimate stage of the pipeline blocks of CSV per DOI are produced that represent date-value pairs.

<td>
2010-01
</td>
<td>
2010-01-01,05<br /> 2010-02-01,02<br /> 2010-01-03,08<br /> &#8230;
</td>

<td>
2010-02
</td>
<td>
2010-02-1,10<br /> 2010-02-01,7<br /> 2010-02-03,22<br /> &#8230;
</td>

10.5555/12345678

At the last stage, these are combined into blocks of all dates for a DOI

<td>
2010-01
</td>
<td>
2010-01-01,05<br /> 2010-02-01,02<br /> 2010-01-03,08<br /> &#8230;<br /> 2010-02-1,10<br /> 2010-02-01,7<br /> 2010-02-03,22<br /> &#8230;
</td>

10.5555/12345678

The DOIs are then hashed into 12 bits and stored as chunks of CSV

day-doi.csv-chunks_8841:

10.1038/ng.3020
2014-06-24,4
2014-06-25,4
2014-06-26,3
...
10.1007/978-94-007-2869-1_7
2012-06-01,12
2012-06-02,8
...
10.1371/journal.pone.0145509
2016-02-01,13
2016-02-02,75
2016-02-03,30
...

There are 65,536 (0x000 to 0xFFFF) possible files, each with about a thousand DOIs worth of data in each.

When the browser requests data for a DOI, it is hashed and then the request for the appropriate file in S3 is made. The browser then has to perform a linear scan of the file to find the DOI it is looking for.

This is the simplest possible form of hash table: simple addressing with separate linear chaining. The hash function is a 16-bit mask of MD5, chosen because of availability in the browser. It does a great job of evenly distributing the DOIs over all 65,536 possible files.

Striking the balance

In any data structure implementation, there are balances to be struck. Traditionally these concern memory layout, the shape of the data, practicalities of disk access and CPU cost.

In this instance, the factors in play included the number of buckets that need to be uploaded and the cost of the browser downloading an over-large bucket. The size of the bucket doesn’t matter much for CPU (as far as the user is concerned it takes about the same time to scan 10 entries as it does 10,000), but it does make a difference asking user to download a 10kb bucket or a 10MB one.

I struck the balance at 4096 buckets, resulting in files of around 100k, which is the size of a medium sized image.

It works

The result is a simple system that allows people to look up data for millions of DOIs, without having to look after another server. It’s also portable to any other file storage service.

The approach isn’t groundbreaking, but it works.

DOI-like strings and fake DOIs

Geoffrey Bilder — Wed, 29 Jun 2016 00:00:00 +0000

TL;DR

Crossref discourages our members from using DOI-like strings or fake DOIs.

Details

Recently we have seen quite a bit of debate around the use of so-called “fake-DOIs.” We have also been quoted as saying that we discourage the use of “fake DOIs” or “DOI-like strings”. This post outlines some of the cases in which we’ve seen fake DOIs used and why we recommend against doing so.

Using DOI-like strings as internal identifiers

Some of our members use DOI-like strings as internal identifiers for their manuscript tracking systems. These only get registered as real DOIs with Crossref once an article is published. This seems relatively harmless, except that, frequently, the unregistered DOI-like strings for unpublished (e.g. under review or rejected manuscripts) content ‘escape’ into the public as well. People attempting to use these DOI-like strings get understandably confused and angry when they don’t resolve or otherwise work as DOIs. After years of experiencing the frustration that these DOI-like things cause, we have taken to recommending that our members not use DOI-like strings as their internal identifiers.

Using DOI-like strings in access control compliance applications

We’ve also had members use DOI-like strings as the basis for systems that they use to detect and block tools designed to bypass the member’s access control system and bulk-download content. The methods employed by our members have fallen into two broad categories:

Spider (or robot) traps.
Proxy bait.

Spider traps

A “spider trap” is essentially a tripwire that allows a site owner to detect when a spider/robot is crawling their site to download content. The technique involves embedding a special trigger URL in a public page on a web site. The URL is embedded such that a normal user should not be able see it or follow it, but an automated bot (aka “spider”) will detect it and follow it. The theory is that when one of these trap URLs is followed, the website owner can then conclude that the ip address from which it was followed harbours a bot and take action. Usually the action is to inform the organisation from which the bot is connecting and to ask them to block it. But sometimes triggering a spider trap has resulted in the IP address associated with it being instantly cut off. This, in turn, can affect an entire university’s access to said member’s content.

When a spider/bot trap includes a DOI-like string, then we have seen some particularly pernicious problems as they can trip-up legitimate tools and activities as well. For example, a bibliographic management browser plugin might automatically extract DOIs and retrieve metadata on pages visited by a researcher. If the plugin were to pick up one of these spider traps DOI-like strings, it might inadvertently trigger the researcher being blocked- or worse- the researcher’s entire university being blocked. In the past, this has even been a problem for Crossref itself. We periodically run tools to test DOI resolution and to ensure that our members are properly displaying DOIs, Crossmarks, and metadata as per their member obligations. We’ve occasionally been blocked when we ran across the spider traps as well.

Proxy bait

Using proxy bait is similar to using a spider trap, but it has an important difference. It does not involve embedding specially crafted DOI like strings on the member’s website itself. The DOI-like strings are instead fed directly to tools designed to subvert the member’s access control systems. These tools, in turn, use proxies on a subscriber’s network to retrieve the “bait” DOI-like string. When the member sees one of these special DOI-like strings being requested from a particular institution, they then know that said institution’s network harbours a proxy. In theory this technique never exposes the DOI-like strings to the public and automated tools should not be able to stumble upon them. However, recently one of our members had some of these DOI-like strings “escape” into the public and at least one of them was indexed by Google. The problem was compounded because people clicking on these DOI-like strings sometimes ended having their university’s IP address banned from the member’s web site. As you can imagine, there has been a lot of gnashing of teeth. We are convinced, in this case, that the member was doing their best to make sure the DOI-like strings never entered the public. But they did nonetheless. We think this just underscores how hard it is to ensure DOI-like strings remain private and why we recommend our members not use them.

Pedantry and terminology

Notice that we have not used the phrase “fake DOI” yet. This is because, internally, at least, we have distinguished between “DOI-like strings” and “fake DOIs.” The terminology might be daft, but it is what we’ve used in the past and some of our members at least will be familiar with it. We don’t expect anybody outside of Crossref to know this.

To us, the following is not a DOI:

10.5454/JPSv1i220161014

It is simply a string of alphanumeric characters that copy the DOI syntax. We call them “DOI-like strings.” It is not registered with any DOI registration agency and one cannot lookup metadata for it. If you try to “resolve” it, you will simply get an error. Here, you can try it. Don’t worry- clicking on it will not disable access for your university.

http://doi.org.pluma.sjfc.edu/10.5454/JPSv1i220161014

The following is what we have sometimes called a “fake DOI”

10.5555/12345678

It is registered with Crossref, resolves to a fake article in a fake journal called The Journal of Psychoceramics (the study of Cracked Pots) run by a fictitious author (Josiah Carberry) who has a fake ORCID (http://orcid.org/0000-0002-1825-0097) but who is affiliated with a real university (Brown University).

Again, you can try it.

http://doi.org.pluma.sjfc.edu/10.5555/12345678

And you can even look up metadata for it.

https://api-crossref-org.pluma.sjfc.edu/v1/works/10.5555/12345678

Our dirty little secret is that this “fake DOI” was registered and is controlled by Crossref.

Why does this exist? Aren’t we subverting the scholarly record? Isn’t this awful? Aren’t we at the very least hypocrites? And how does a real university feel about having this fake author and journal associated with them?

Well- the DOI is using a prefix that we use for testing. It follows a long tradition of test identifiers starting with “5”. Fake phone numbers in the US start with “555”. Many credit card companies reserve fake numbers starting with “5”. For example, Mastercard’s are “5555555555554444” and “5105105105105100.”

We have created this fake DOI, the fake journal and the fake ORCID so that we can test our systems and demonstrate interoperable features and tools. The fake author, Josiah Carberry, is a long-running joke at Brown University. He even has a Wikipedia entry. There are also a lot of other DOIs under the test prefix “5555.”

We acknowledge that the term “fake DOI” might not be the best in this case- but it is a term we’ve used internally at least and it is worth distinguishing it from the case of DOI-like strings mentioned above.

But back to the important stuff….

As far as we know, none of our members has ever registered a “fake DOI” (as defined above) in order to detect and prevent the circumvention of their access control systems. If they had, we would consider it much more serious than the mere creation of DOI-like strings. The information associated with registered DOIs becomes part of the persistent scholarly citation record. Many, many third party systems and tools make use of our API and metadata including bibliographic management tools, TDM tools, CRIS systems, altmetrics services, etc. It would be a very bad thing if people started to worry that the legitimate use of registered DOIs could inadvertently block them from accessing content. Crossref DOIs are designed to encourage discovery and access- not block it.

And again, we have absolutely no evidence that any of our members has registered fake DOIs.

But just in case, we will continue to discourage our members from using DOI-like strings and/or registering fake DOIs.

This has been a public service announcement from the identifier dweebs at Crossref.

Image Credits

Unless otherwise noted, included images purchased from The Noun Project

Beyond the DOI to richer metadata

April Ondis — Wed, 15 Jun 2016 00:00:00 +0000

The act of registering a DOI (Digital Object Identifier) for scholarly content is sometimes conflated with the notion of conferring a seal of approval or other mark of good quality upon an item of content. This is a fundamental misunderstanding.

A DOI is a tool, not a badge of honor.

The presence of a Crossref DOI on content sends a signal that:

The owner of the content would like to be formally cited if the content is used in a scholarly context.
The owner of the content considers that it is worthy of being made persistent.

Beyond the DOI

For Crossref, a DOI is just one of several types of metadata we register, albeit an important one.

Metadata about scholarly works extends beyond the DOI. In addition to bibliographic details, layers of information accompanying published works may now extend to data that describes the research, such as the source of research funding. It may also include non-descriptive information that facilitates usage, such as copyright and access permissions.

In fact, this “richer” metadata can tell you more about the context of the content deposited for a published work than you might realize.

For example:

Author data - Crossref metadata may include information specifying the author’s unique ORCID, allowing you to find other works by the same person.

Copyright and access indicators - You can view the license terms under which the full content may be available, which is very helpful for scholars who want to access the full content for research and teaching or for text and data mining.

Funding data - Metadata may also include the identity of the grant-making institution that funded the research, so that the funder and, in the case of publicly funded research, the general public and other researchers, have visibility on the resulting research outputs.

Clinical Trials data - Similarly, when research involves a clinical trial, (testing of medicines and treatments on human beings), Crossref metadata can enhance output visibility by displaying the clinical trial number and the related clinical trial registry.

Like the full content they describe, these metadata have become research resources in their own right. Unfortunately, too much metadata is entered into Crossref with missing, incomplete, or duplicated fields. This “bad” metadata slows the pace of discovery, confounding attempts to find and understand scholarly content and its context.

As a community, we really need to do something about that.

“The Map is not the Territory”

And the metadata is not the content. In Metadata (MIT Press), Jeffrey Pomerantz quotes Alfred Korzybski’s insight that a map is a simplified representation of a territory, a tool of abstraction that allows us to find our way. Jennifer Lin contributed the concept of the scholarly road map as a useful metaphor for the way we use metadata about scholarly works to find our way between and among them in the digital world.

Metadata deposited with Crossref amounts to pieces of information-structured, descriptive, administrative, contextual-about published works that humans can read and machines can use to automate linking and retrieval. The systematic development of such metadata allows us to make sense of such complex information by finding, linking, citing, and assessing scholarly content.

If you want to understand how Crossref acts as a map of scholarly metadata, try searching for content on search.crossref.org (our human API interface). Or simply talk with us @CrossrefOrg and via member@crossref.org.

HTTPS and Wikipedia

Joe Wass — Tue, 31 May 2016 00:00:00 +0000

This is a joint blog post with Dario Taraborelli, coming from WikiCite 2016.

In 2014 we were taking our first steps along the path that would lead us to Crossref Event Data. At this time I started looking into the DOI resolution logs to see if we could get any interesting information out of them. This project, which became Chronograph, showed which domains were driving traffic to Crossref DOIs.

You can read about the latest results from this analysis in the “Where do DOI Clicks Come From” blog post.

Having this data tells us, amongst other things:

where people are using DOIs in unexpected places
where people are using DOIs in unexpected ways
where we knew people were using DOIs but the links are more popular than we realised

By the time the ALM Workshop 2014 rolled around there was some preliminary data and we realised that Wikipedia came into the third category. There are lots of DOIs in Wikipedia and people click them!

I met with Dario Taraborelli, head of research at the Wikimedia Foundation, and shared the data. Dario — who co-authored in 2010 the Altmetrics Manifesto — has been interested in understanding how scholarly citations are used in Wikipedia. Over the years, Wikipedia contributors have made extensive use of references to the scientific literature using DOIs, and by doing so they have created a resource that represents today in many ways the “front matter to all research”. There is growing interest in the community in understanding how DOIs are being used in Wikipedia and in non traditional scholarship.

During our discussions the subject of Wikipedia’s gradual transition to HTTPS was raised: we anticipated that this change would affect our data gathering.

Changes

When you’re reading webpage and click on a link to another page, your web browser will usually tell the server of that second page the last page you were on. This forms the basis of trackers like Google Analytics.

In the days before HTTPS, the next site would know the full URL that you were previously on. With the change to HTTPS, this was reduced to just sending the domain name and not the full URL, or no data at all if you click from an HTTPS page to HTTP.

DOI hyperlinks are just like any other hyperlink, and are mostly HTTP not HTTPS.

Up until 2015, Wikipedia was served over HTTP, only switching to HTTPS when users were logged in or if they requested it. The Wikimedia Foundation started planning to move to HTTPS and we knew that if they did that, and continued to use HTTP DOIs then we would lose valuable research data.

A Plan

We decided that the best course of action was to try and change the DOIs in Wikipedia to use HTTPS. Simple, right?

After some further research, Dario posted a proposal on how to mitigate the impact of the HTTPS rollout, to make sure that Wikipedia can still signal its importance as a traffic source, while preserving the privacy of its users. Discussion followed and the conclusion was to change the format of every single DOI on Wikipedia, which fortunately could be done without having to edit millions of pages. You can read the full story in this post from a year ago.

The result of this effort was that well in advance of the HTTPS switchover, the DOI links were ready to continue reporting referral data.

The Switch

In June 2015 the Wikimedia foundation made the announcement that they were finalising the switch, and that within a few weeks all traffic would be HTTPS.

We held our breath. Would it work? Would we lose all referral data from Wikipedia sites? In February 2016 the last piece of the puzzle fell into place as Wikipedia gained a ‘meta referrer’ tag to explicitly specify how they would like referrers to be sent: a detailed report on the effect of this change is coming up on the Wikimedia Foundation’s blog.

The results

As detailed in the last blog post the traffic that we measured coming from Wikipedia doesn’t seem to have slowed down during 2015:

I’d call that a success! Over the period covered in the graph, Wikipedia remained prominent as a non-publisher referral of traffic to DOIs.

Looking at the balance of HTTP vs HTTPS traffic coming from wikipedia.org, the switchover was dramatic:

Thank you to Dario Taraborelli, Nemo (Federico Leva), Aaron Halfaker, Alex Stinson and everyone who put in this effort.

I’ll leave the last word to Dario:

It’s great to see this data. It shows that the switchover happened successfully, which better protects the privacy of our users whilst still reporting the fact that Wikipedia is a prominent source of traffic. This is important validation of the increasing role that Wikipedia plays in the education and scientific community.

Where do DOI clicks come from?

Joe Wass — Thu, 19 May 2016 00:00:00 +0000

As part of our Event Data work we’ve been investigating where DOI resolutions come from. A resolution could be someone clicking a DOI hyperlink, or a search engine spider gathering data or a publisher’s system performing its duties. Our server logs tell us every time a DOI was resolved and, if it was by someone using a web browser, which website they were on when they clicked the DOI. This is called a referral.

This information is interesting because it shows not only where DOI hyperlinks are found across the web, but also when they are actually followed. This data allows us a glimpse into scholarly citation beyond references in traditional literature.

Last year Crossref Labs announced Chronograph, an experimental system for browsing some of this data. We’re working toward a new version, but in the meantime I’d like to share the results for 2015 and some of 2016. We have filtered out domains that belong to Crossref member publishers to highlight citations beyond traditional publications.

Top 10 DOI referrals from websites in 2015

This chart shows the top 10 referring non-primary-publisher domains of DOIs per month. Note that if browsers don’t send the referrer (e.g. from an HTTPS page), we don’t get to find out. Because the top 10 can be different month to month, the total number of domains mentioned can be more than 10. Subdomains are combined, which means that, for example, the wikipedia.org entry covers all Wikipedia languages. This chart covers all of 2015 and the first two months of 2016.

The top 10 referring domains for the period:

webofknowledge.com
baidu.com
serialssolutions.com
scopus.com
exlibrisgroup.com
wikipedia.org
google.com
uni-trier.de
ebsco.com
google.co.uk

It’s not surprising to see some of these domains here: for example serialssolutions.com and exlibrisgroup.com are effectively proxies for link resolvers, Baidu and Google are incredibly popular search engines which would show up anywhere. But it is exciting to see Wikipedia ranked amongst these. For more detail look out for the new Chronograph.

HTTP vs HTTPS in 2015

We’ve also seen a steady increase in HTTPS referral traffic, i.e. people clicking on DOIs from sites that are using HTTPS. While it is still dwarfed by HTTP, there was a steady uptick throughout 2015.

This chart shows HTTP vs HTTPS referrals per day, which shows up the weekly spikes. It doesn’t include resolutions where we don’t know the referrer.

Increasing numbers of people are moving to HTTPS for reasons of security, privacy and protection from tampering. Google has announced plans to take HTTPS into account when ranking search results. Wikipedia has moved exclusively to HTTPS, and I’ll be telling the story of how Crossref and Wikipedia collaborated in an upcoming blog post.

Chronograph

Another version of Chronograph will be available soon. It will contain full data for all non-primary-publisher referring domains. Stay tuned!

Getting Started with Crossref DOIs, courtesy of Scholastica

Anna Tolwinska — Mon, 25 Apr 2016 00:00:00 +0000

I had a great chat with Danielle Padula of Scholastica, a journals platform with an integrated peer-review process that was founded in 2011. We talked about how journals get started with Crossref, and she turned our conversation into a blog post that describes the steps to begin registering content and depositing metadata with us. Since the result is a really useful description of our new member on-boarding process, I want to share it with you here as well. As always, comments and questions are welcome here, at member@Crossref.org, and @CrossrefOrg. - Anna_

The internet is in a constant state of change, with new content being added to the web by the minute and old content sometimes getting moved around. While the benefit of publishing scholarly outputs online is that it’s possible to update them at any moment, moving or modifying content can also …

The Wikipedia Library: A Partnership of Wikipedia and Publishers to Enhance Research and Discovery

Rachael Lammey — Mon, 04 Apr 2016 00:00:00 +0000

Back in 2014, Geoffrey Bilder blogged about the kick-off of an initiative between Crossref and Wikimedia to better integrate scholarly literature into the world’s largest knowledge space, Wikipedia. Since then, Crossref has been working to coordinate activities with Wikimedia: Joe Wass has worked with them to create a live stream of content being cited in Wikipedia; and we’re including Wikipedia in Event Data, a new service to launch later this year. In that time, we’ve also seen Wikipedia importance grow in terms of the volume of DOI referrals.

Alex Stinson, Project Manager for the Wikipedia Library, and our guest blogger! This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license (Source: Myleen Hollero Photography)

Alex Stinson, Project Manager for the Wikipedia Library, and guest blogger! This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license (Source: Myleen Hollero Photography)

How can we keep this momentum going and continue to improve the way we link Wikipedia articles with the formal literature? We invited Alex Stinson, a project manager at The Wikipedia Library (and one of our first guest bloggers) to explain more:

Wikipedia provides the most public gateway to academic and scholarly research. With millions of citations to academic as well as non-academic but reliable sources, like those produced by newspapers, its ecosystem of 5 million English Wikipedia articles and 35 million articles in hundreds of languages provides the first stop for researchers in both scholarly and informal research situations. The practice of “checking Wikipedia” has become ubiquitous in a number of fields; for example, Wikipedia is the most visited source of medical information online, even providing the first stop for many medical students and medical practitioners when looking for medical literature.

The Wikipedia Library program helps Wikipedia’s volunteer editors access and use the best sources in their research and citations. Through partnerships with over fifty leading publishers and aggregators, like JSTOR, Project Muse, Elsevier, Newspapers.com, Highbeam, Oxford University Press and others, we have been able to give over 3000 of our most prolific volunteers access to over 5500 accounts. These are clear, win-win relationships where Wikipedia editors get to use these databases to improve Wikipedia, while in turn linking to authoritative resources and enhancing their discovery.

JSTOR has been working with us since 2012, providing over 500 accounts to our editors. Kristen Garlock at JSTOR writes:

“We’re very happy to collaborate with the Wikipedia Library to provide JSTOR access to Wikipedia editors. Supporting the initiative to increase editor access to scholarly resources and improve the quality of information and sources on Wikipedia has the potential to help all Wikipedia readers. In addition to providing more discoverability for our institutional subscribers, introducing new audiences to the scholarship on JSTOR them discover access opportunities like our Register & Read program.”

There are strong signals that Wikipedia’s role in the citation ecosystem helps ensure the best materials reach the public through its over 400 million monthly readers:

The latest estimates by Crossref show that Wikipedia has risen from the 8th most prolific referrer to DOIs to the 5th.

Two of our access partners have found that around half of the referrals arriving from Wikipedia were able to authenticate into their subscription resources, suggesting that a large portion of our readers can take advantage of subscriptions provided by scholarly institutions.

Wikipedia is highly influential in the open access ecosystem as well, with a recent study showing higher citation rates for OA materials than those behind a paywall.

Altmetrics tools (such as Altmetric.com, ImpactStory or Plum Analytics) are recognizing Wikipedia’s importance by including Wikipedia citations in their impact metrics.

Despite these advances, we think this is only the beginning of Wikipedia’s impact on the landscape of scholarly research and discovery. Wikipedia can become a highly integrated research platform within the broader research ecosystem, where the best scholarship is summarized and discoverable-where Wikipedia effectively becomes the front matter to all research.

However, there are some clear barriers to fulfilling this vision. Currently, most citations on Wikipedia are stored in free-text and not readily available in machine-readable formats; our community is working to fix this. Wikipedia also has major systematic gaps in topics where either we lack volunteer interest or Wikipedia reflects larger systemic biases within society or scholarship.We need the help of volunteers, experts, industry partners, and information technologists to grow Wikipedia’s collection of citations, especially around key missing areas, and to transform existing citations into structured formats.

WikiData, Wikipedia’s sister project which crowdsources structured metadata, offers an excellent opportunity for improving the impact of Wikipedia in research. Having Wikipedia citations stored in this structured ecosystem, connecting metadata with semantic meaning, would allow the citations in Wikipedia to become the backbone for discovery tools which emphasize the hand-curated interrelationships between authoritative sources and the knowledge collected by Wikipedia and Wikidata editors.

We need more collaborators to realize the full vision of Wikipedia supporting research in the most effective ways:

We need help from publishers with subscription databases, to help us give our editors access to the databases through The Wikipedia Library’s access partnership program. These high-quality source materials allow our editors to expose that research in a number of languages and for millions of readers.

We need help from the open access community, to figure out how to better support increased citation and strategic use of open access materials within Wikipedia and other Wikimedia projects. Our community has some ideas, but we need your input and collaboration.

We need your expertise to build our structured metadata ecosystem, by helping Wikidata map and collect citation data.

We need the larger research community to promote Wikipedia as a scholarly communications tool and make contributing to Wikipedia an important part of the social responsibility of experts. Wider citation of sources in Wikipedia ensures widespread discovery and dissemination of that research.

If you think you can help, we invite you to contact us at wikipedialibrary@wikimedia.org or via Twitter @WikiLibrary.

Community responses to our proposal for early content registration

Geoffrey Bilder — Tue, 01 Mar 2016 00:00:00 +0000

TL;DR:

We will proceed with implementing the proposed support for registering content before online availability. Adopting the workflow will be optional and will involve no extra fees.

Background

At the end of January, Crossref issued a “request for community comment” on a proposed new process to support the registration of content including DOIs before online availability. We promised that we would summarize the results of the survey once we had received and analyzed all the responses.

Support for Crossref implementing the proposed new workflow was overwhelming. Of the 104 responses, 90 were positive, 7 were neutral and 7 were negative. As such we will proceed to make the necessary changes to better support registering content before online availability. We aim to enable this functionality in the second half of 2016.

We received survey responses varying in length from one or two sentences to multiple pages. A lot of the responses also interspersed questions and observations about entirely different issues that were of interest to respondents. As such, it has taken a while for us to analyze the results. We also found it was pretty much impossible for us to tabulate a summary of the responses to the direct questions. Instead we’ll summarize the responses at a high level and then drill down into some of the nuances in the answers and issues that were raised from the responses.

The positive responses

By “positive” we mean the respondent understood the issues we were trying to address and thought what we were proposing was a reasonable way to address the problems. Here are a few (anonymized) excerpts from the responses:

“[This] is very timely as we have been made aware of changes to manuscript deposit requirements for UK authors. Authors who partake in the REF system will have to deposit articles at their manuscript stage before publication. We need to set an embargo on the articles so that they only become discoverable at some point after the publication date. Ideally we would like this to happen with all articles regardless of where they are from as authors will put their own work up on open access sites.”

“Your proposal and the associated workflow look good to us and will help with our media embargo timelines, as well as our authors’ institutional requirements.”

“The workflows and solutions seem reasonable … The temporary landing page seems like a sustainable technical solution. Hosting by Crossref is key to this – there is no way that all publishers would otherwise take on maintaining temporary pages. And having a standard display for metadata consistency is crucial too.”

“Early assignment and recording of DOIs from the point of acceptance forms a key step in [the university’s] proposed ‘Submit-accept-deposit’ workflow. We welcome the proposal by Crossref to enable early assignment of DOIs for publications.”

Note that a positive response did not mean the respondents thought the problems necessarily applied to them or that they would necessarily be implementing the changes - just that what we were proposing seemed sound for those who needed to address the issues.

“While not directly relevant to our business the proposal seems aimed to protect the integrity of DOIs and Crossref’s role and that is not a bad thing.”

“I would consider it an irresponsible use of the system on the part of a publisher to circulate dois that don’t (yet) work. This is bound to lead to frustration with users encountering errors. However I appreciate that this situation may arise in some workflows and therefore your proposals to implement temporary landing pages make sense.”

“I was not aware of these issues, but think that your solutions seem feasible. We are a small journal and generally don’t add doi’s or publish until the article is complete (i.e., we don’t post anything that’s just accepted - only finalized). So we would be unlikely to update our workflow.”

Also, though respondents might have been generally positive about the proposal - that didn’t always mean they were also sanguine about it. For example, several shared concerns about the potential costs of changing their workflows.

“[we] would consider implementing this change into our workflow. Limiting factors would include the effort and additional cost to enable our paper management system vendor…”

“My only comment is that the process needs to be streamlined as much as possible so small publishers without great technical capacity will not be burdened with twice the work or with additional expense. After reading through Crossref’s proposal, I believe you have taken such things into account and will implement an efficient and worthwhile system.”

“The workflow makes sense as a solution to the problem you describe […] but will require extensive workflow changes on our end in order to implement. Speaking for a small publishing house I’m not sure it’s reasonable to expect this from us on any short term.”

Several of the positive respondents also wondered about how we would handle particular edge cases (e.g. rescinding acceptances) and/or offered suggestions to improve the proposal. We will discuss these further at the end of this post.

The neutral responses

The responses we categorised as “neutral” were generally too short to conclude much about. They consisted of one or two sentences that said something like “this doesn’t apply to me.” It wasn’t clear whether it didn’t apply to them because they didn’t have the problems we described or because they’d already solved the problems we described (e.g. by providing their own interim landing pages). They also didn’t comment on the applicability to other members or whether they thought the issues might eventually affect them.

The negative responses

We categorised responses as “negative” when the member rejected that the issues we outlined were actual problems or they rejected the mechanisms we were proposing to address the problems.

“…a formal letter of acceptance on a letter in PDF will be OK for authors. Why a DOI is better?”

“…I am aware of funder and institutional requirements for authors to take action on acceptance of manuscripts for publication in journals but don’t think the time pressure is so high that it has to happen in short time between acceptance and published ahead of print online…”

“Of all the accepted-but-not-yet-published papers in existence at any time, the number whose existence must be demonstrated to promotion and tenure review boards must be awfully small.”

There are a few common themes here. The first is that, historically, the industry has been content with acceptance letters as proof of publication and that it was relatively rare for authors to have to produce such proof.

The problem that has led us to propose support for a modified workflow is that now we have situations where all the researchers in a country require such letters on a regular basis - not just when they are up for promotion or tenure. This is the new reality faced by researchers and institutions who are subject to regular national evaluation schemes like the REF and ERA.

One of the negative respondents added:

“This is very familiar territory. It’s definitely coming out of STEM.”

Indeed, the initial pressure to support the earlier registration of DOIs is certainly falling on our members who focus on STEM publishing. Researchers in the STEM fields are generally under more pressure to publish articles frequently and they are primarily affected by emerging funder mandates. The relatively high research output in STEM fields combined with the need for regular compliance checks and regular evaluation schemes is creating an environment that requires more automated mechanisms to keep track of publications. Asking for and processing letters of acceptance in these situations just doesn’t scale.

Some of the negative responses also questioned our assertions about the hazards of promulgating unregistered DOI-like strings and/or the problems associated with the delay between when content is made available online and when the content is registered.

“I don’t buy the argument that people lose trust in DOIs in general because they once tried to resolve one and it didn’t lead to an article. By the same argument, URLs in general are similarly undermined.”

“where authors ask me for their DOIs so they can accurately cite the paper in another publication or use it for grants and applications. I explain that it won’t work until the issue as a whole posts and I have never heard back about confusion or distrust of the system.”

To this all we can say is, we have the data. Next to typos, unregistered DOIs account for the second greatest category of failed resolutions on the Crossref system. Our help desk has to explain them to researchers constantly. We have promoted DOIs as being more robust, persistent identifiers than ordinary URLs. People are not surprised when URLs don’t work. They are surprised when DOIs don’t work. We’d like to keep it that way.

What seems to be at the the root of the few negative responses - is that most assumed that Crossref was mandating that publishers change their workflow - even if they didn’t face any of the issues outlined in the proposal. There is very little that Crossref mandates to participate. This is by design. Our membership is just too diverse for us to have mandates that can be sensibly applied to all. Still - we should have made it clearer in the proposal that the proposed changes would not be mandated. We will certainly need to make this clearer when we roll out support for the new processes.

Oh yeah - one respondent called us out for using the phrase “advanced publication” instead of “advance publication”. For this we are truly sorry. The employee who made this mistake has been dragged out and shotted.

Issues raised and questions asked

Both the positive and negative respondents raised issues, asked questions and provided suggestions regarding the proposal. We will make sure that, when the proposal is implemented, we address all of these issues more clearly, but in the meantime, we thought it would be helpful if we answered some of them briefly here.

Q: Would Crossref charge extra for the new workflow?

A: No. We should have made this clear in the proposal. We should have also mentioned that, in the “Crossref-facilitated Early Registration” scenario members will only be charged once they have replaced the “registered_content” metadata with metadata for the published item using one of the existing content schemas (e.g. article, book, confproc).

Q: Would Crossref require that publishers adopt the new proposed workflows?

A: No. But we will recommend them to members who need to address the issues outlined in the proposal. And in general, we will recommend that our members register DOIs as early in the process as practicable.

Q: What does “acceptance” mean? It was pointed out that there were lots of variations of “acceptance” including “acceptance pending revisions”, etc.

A: We would expect that “contingent acceptance” does not constitute final acceptance and that in this case “acceptance” should mean that the publisher has a copy of the manuscript in which the author has made all of the changes asked of them.

Q: Doesn’t “acceptance” works both ways? A researcher has to grant permission to publish to the publisher.

A: This is a vital point - the publisher should only register content for which they have already secured the rights to publish.

Q: Collecting and verifying the metadata associated with a paper is expensive and time consuming. As such, some publishers only produce complete and robust metadata after a paper has been accepted. We face a Catch-22. if we deposit metadata immediately after acceptance, it will be sparse and unreliable. If we wait to collect and verify the metadata, then we risk violating some of the emerging mandates. How do we resolve this dilemma?

A: This is clearly beyond our control, but we expect that those issuing the mandates will have to make some reasonable accommodations if they expect publishers to register content both early and with reasonably useful metadata.

Q: How would publishers handle rescinded acceptances?

A: Publishers can handle this the same way they handle retractions or withdrawals. Additionally, the registered record type and the “intent to publish” landing page will both support Crossmark for those members who use Crossmark to promulgate corrections to the literature. We will explore adding a new “acceptance rescinded” update type to the Crossmark schema.

Q: The Crossref DOIs we generate contain embedded publication information such as volume and issue. We don’t know these details at acceptance so how can we register DOIs early?

A: Many of our members generate Crossref DOIs with embedded semantic information in them such as volume/issue, publication date or even author initials and title. After 16 years of experience, we have found that this tends to be a bad idea. Publication schedules slip. Metadata changes. We will soon be revising our guidelines on DOI best practice in Crossref DOI generation to recommend against embedding such information into the DOI itself. Clearly, if you decide to assign Crossref DOIs at acceptance, you will need to adopt a DOI structure that accommodates this.

Q: Our hosting provider manages DOI registrations for us. If we have to register DOIs earlier in the process, can we have one party (e.g. a manuscript tracking system vendor) register the initial “registered_content” metadata and then have different party (e.g. hosting provider or typesetter) replace that record with the final metadata?

A: Yes.

Q: Will you be working with industry vendors to help them support this new workflow?

A: Yes.

Q: Will we support the pre-registration of DOIs in the the deposit forms on the Crossref site?

A: Yes.

Q: If Crossref hosts the “intent to publish” landing page, how will publishers be able to account for visits to the page and incorporate that information into their metrics?

A: While visitors to the Crossref hosted page will not show up in the publisher’s own hosting platform logs, publishers will be able to easily see how many times their “intent to publish” landing page was accessed by looking at their standard Crossref DOI resolution logs.

Q: Could the Crossref-hosted landing page also include the URL that the DOI will eventually be associated with?

A: This is an interesting idea and was suggested by two separate respondents. The challenge will be in explaining to the user that the URL might or might not work. We are also concerned that this would reduce the incentive for publishers to replace the holding page in a timely manner. We’ll explore this option as we continue to work on implementation.

Q: Would the Crossref-hosted landing page be open to indexing by Google and others? If so, wouldn’t this undermine articles under press embargoes?

A: The idea behind the limited metadata required for registering content is that it allows the publisher to control the balance between discovery (needed to meet funder requirements) and discretion (needed to manage publicity). So yes, the Crossref-hosted landing pages would be open to indexing, but publishers can still control what gets indexed by withholding metadata as needed.

Q: The table of required metadata elements for the “registered content” type does not include the author. How are such records supposed to be used as proof of acceptance if they do not include the author name?

A: ~~We made a mistake. The table should have included the contributor in the required element column.~~ Update: We retract our retraction! We are trying to accommodate several different use cases for ’early content registration’ and these different use cases often have contradictory metadata implications. So, for example, including the author is certainly important for monitoring mandate compliance. However, including the author might be problematic when the publisher is trying to manage publicity around an upcoming publication. Again, Crossref is not in a position to resolve this dilemma and we expect that those issuing the mandates will make some reasonable accommodations with publishers who need to manage publicity around publications. In short, “authors” will remain optional metadata.

Summary and conclusions

We were delighted with the response rate on the proposal. It is clear to us that a lot of the respondents really appreciated both being alerted to a set of issues that they were not yet aware of and that they valued the chance to comment on our proposed mechanisms for addressing said issues. We also learned some lessons on how to better structure any such future surveys in order to make them easier for us to summarise and respond to. The wide variety of responses and detailed descriptions of different workflows reconfirmed our sense that Crossref members vary widely in their working practices. We need to continue to work directly with members and understand these different working practices so that we can provide appropriately flexible services to our membership and to the scholarly community in general.

Finally, the feedback we received will be used by our product team and our communications & outreach teams to refine our rollout plans for registering content before online availability. We expect that we will rollout this functionality in the second half of 2016.

Thanks to those who responded to our RFC. Some of those responses included questions about other matters relating to Crossref. We have attempted to extract these and answer them directly- but if we have not yet answered one of your questions, please follow-up with us at feedback@crossref.org

Request for Community Comment: registering content before online availability

Ginny Hendricks — Thu, 21 Jan 2016 00:00:00 +0000

Crossref is proposing a process to support the registration of content—including DOIs and other metadata—prior to that content being made available, or published, online. We’ve drafted a paper providing background on the reasons we want to support this and highlighting the use cases. One of the main needs is in journal publishing to support registration of Accepted Manuscripts immediately on or shortly after acceptance, and dealing with press embargoes.

Proposal doc for community comment

We request community comment on the __proposed approach as outlined in this report.

Some examples of what we’d like to know:

Are you aware of the issues outlined in this proposal?
Are you aware of the funder and institutional requirements for authors to take action on acceptance of manuscripts for publication in journals?
Do you think the proposed solution and workflows are reasonable?
Are you likely to update your workflow to register content early?
If you are likely to update your workflow, how long do you estimate it will take?
Any other general comments, questions or feedback on anything raised in this document.

Please send comments, feedback and questions to me, Ginny, at feedback@crossref.org. The deadline for comments is February 4th. Thanks!

January 2015 DOI Outage: Followup Report

Geoffrey Bilder — Tue, 17 Mar 2015 00:00:00 +0000

Background

On January 20th, 2015 the main DOI HTTP proxy at doi.org experienced a partial, rolling global outage. The system was never completely down, but for at least part of the subsequent 48 hours, up to 50% of DOI resolution traffic was effectively broken. This was true for almost all DOI registration agencies, including Crossref, DataCite and mEDRA.

At the time we kept people updated on what we knew via Twitter, mailing lists and our technical blog at CrossTech. We also promised that, once we’d done a thorough investigation, we’d report back. Well, we haven’t finished investigating all implications of the outage. There are both substantial technical and governance issues to investigate. But last week we provided a preliminary report to the Crossref board on the basic technical issues, and we thought we’d share that publicly now.

The Gory Details

First, the outage of January 20th was not caused by a software or hardware failure, but was instead due to an administrative error at the Corporation for National Research Initiatives (CNRI). The domain name “doi.org” is managed by CNRI on behalf of the International DOI Foundation (IDF). The domain name was not on “auto-renew” and CNRI staff simply forgot to manually renew the domain. Once the domain name was renewed, it took about 48 hours for the fix to propagate through the DNS system and for the DOI resolution service to return to normal. Working with CNRI we analysed traffic through the Handle HTTP proxy and here’s the graph:

The above graph shows traffic over a 24 hour period on each day from January 12, 2015 through February 10th, 2015. The heavy blue line for January 20th and the heavy red line for January 21st show how referrals declined as the doi.org domain was first deleted, and then added back to DNS.

It could have been much worse. The domain registrar (GoDaddy) at least had a “renewal grace and registry redemption period” which meant that even though CNRI forgot to pay its bill to renew the domain, the domain was simply “parked” and could easily be renewed by them. This is the standard setting for GoDaddy. Cheaper domain registrars might not include this kind of protection by default. Had there been no grace period, then it would have been possible for somebody other than CNRI to quickly buy the domain name as soon as it expired. There are many automated processes which search for and register recently expired domain names. Had this happened, at the very least it would have been expensive for CNRI to buy the domain back. The interruption to DOI resolutions during this period would have also been almost complete.

So we got off relatively easy. The domain name is now on auto-renew. The outage was not as bad as it could have been. It was addressed quickly and we can be reasonably confident that the same administrative error will not happen again. Crossref even managed to garner some public praise for the way in which we handled the outage. It is tempting to heave a sigh of relief and move on.

We also know that everybody involved at CNRI, the IDF and Crossref have felt truly dreadful about what happened. So it is also tempting to not re-open old wounds.

But it would be a mistake if we did not examine a fundamental strategic issue that this partial outage has raised: How can Crossref claim that its DOIs are ‘persistent’ if Crossref does not control some of the key infrastructure on which it depends? What can we do to address these dependencies?

What do we mean by “persistent?”

@kaythaney tweets on definition of “persistent”

To start with, we should probably explore what we mean by ‘persistent’. We use the word “persistent” or “persistence” about 470 times on the Crossref web site. The word “persistent” appears central to our image of ourselves and of the services that we provide. We describe our core, mandatory service as the “Crossref Persistent Citation Infrastructure.”

The primary sense of the word “persistent” in the New Oxford American Dictionary is:

Continuing firmly or obstinately in a course of action in spite of difficulty or opposition.

We play on this sense of the word as a synonym for “stubborn” when we half-jokingly say that, “Crossref DOIs are as persistent as Crossref staff.” Underlying this joke is a truth, which is that persistence is primarily a social issue, not a technical issue.

Yet presumably we once chose to use the word “persistent” instead of “perpetual” or “permanent” for other reasons. “Persistence” implies longevity, without committing to “forever.” Scholarly publishers, perhaps more than most industries, understand the long term. After all, the scholarly record dates back to at least 1665 and we know that the scholarly community values even our oldest journal backfiles. By using the word “persistent” as opposed to the more emphatic “permanent” we are essentially acknowledging that we, as an industry, understand the complexity and expense of stewarding the content for even a few hundred years to say nothing of “forever.” Only the chronologically naïve would recklessly coin terms like “permalink” for standard HTTP links which have a documented half-life of well under a decade.

So “persistent” implies longevity- without committing to forever- but this still begs questions. What time span is long enough to qualify as “persistent?” What, in particular, do we mean by “persistent” when we talk about Crossref’s “Persistent Citation Infrastructure?” or of Crossref DOIs being “persistent identifiers?”

What do we mean by “persistent identifiers?”

]5 @violetailik tweets on outage and implication for term “persistent identifier”

First, we often make the mistake of talking about “persistent identifiers” as if there is some technical magic that makes them continue working when things like HTTP URIs break. The very term “persistent identifier” encourages this kind of magical thinking and, ideally, we would instead talk about “persist-able” identifiers. That is, those that have some form of indirection built into them. There are many technologies that do this- Handles, DOIs, Purls, ARKs and every URL shortener in existence. Each of them simply introduces a pointer mapping between an identifier and location where a resource or content resides. This mapping can be updated when the content moves, thus preserving the link. Of course, just because an identifier is persist-able doesn’t mean it is persistent. If Purls or DOIs are not updated when content moves, then they are no more persistent than normal URLs.

Andrew Treloar points out that when we talk about “persistent identifiers,” we tend to conflate several things:

The persistence of the identifier- that is the token or string itself.
The persistence of the thing being pointed at by the identifier. For example, the content.
The persistence of the mapping of the identifier to the thing being identified.
The persistence of the resolver that allows one to follow the mapping of the identifier to the thing being identified.
The persistence of a mechanism for updating the mapping of the identifier to the thing being identified.

If any of the above fails, then “persistence” fails. This is probably why we tend to conflate them in the first place.

Each of these aspects of “persistence” is worthy of much closer scrutiny, however, in the most recent case of the January outage of “doi.org,” the problem specifically occurred with item “D”- the persistence of the resolver. When CNRI failed to renew the domain name for “doi.org” on time, the DOI resolver was rendered unavailable to a large percentage of people over a period of about 48 hours as global DNS servers first removed, and then added back the “doi.org” domain.

Turtles all the way down*

The initial public reaction to the outage was, almost unanimous in one respect- people assumed that the problem originated with Crossref.

@iainh_z tweets to Crossref enquiring about failed DOI resoluton

@LibSkrat tweets at Crossref about DOI outage

This is both surprising and unsurprising. It is surprising because we have fairly recent data indicating that lots of people recognise the DOI brand, but not the Crossref brand. Chances are, that this relatively superficial “brand” awareness does not correlate with understanding how the system works or how it relates to persistence. It is likely plenty of people clicked on DOIs at the time of the outage and, when they didn’t work, simply shrugged or cursed under their breath. They were aware of the term ‘DOI’ but not of the promise of “persistence”. Hence, they did not take to twitter to complain about it, and if they did, they probably wouldn’t have known who to complain to or even how to complain to them (neither CNRI or the IDF has a Twitter account).

But the focus on Crossref is also unsurprising. Crossref is by far the largest and most visible DOI Registration Agency. Many otherwise knowledgeable people in the industry simply don’t know that there are even other RAs.

They also generally didn’t know of the strategic dependencies that exist in the Crossref system. By “strategic dependencies” we are not talking about the vendors, equipment and services that virtually every online enterprise depends on. These kinds of services are largely fungible. Their failures may be inconvenient and even dramatic, but they are rarely existential.

Instead we are talking about dependencies that underpin Crossref’s ability to deliver on its mission. Dependencies that not only affect Crossref’s operations, but also its ability to self-govern and meet the needs of its membership. In this case there are three major dependencies: Two of which are specific to Crossref and other DOI registration agencies and one which is shared by virtually all online enterprises today. The organisations are: The International DOI Foundation (IDF), Corporation for National Research Initiatives (CNRI) and the Internet Corporation for Assigned Names and Numbers (ICANN).

Dependency of RAs on IDF, CNRI and ICANN. Turtles all the way down.

Each of these agencies has technology, governance and policy impacts on Crossref and the other DOI registration agencies, but here we will focus on the technological dependencies.

At the top of the diagram are a subset of the various DOI Registration Agencies. Each RA uses the DOI for a particular constituency (e.g. scholarly publishers) and application (e.g. citation). Sometimes these constituencies/applications overlap (as with mEDRA, Crossref and DataCite), but sometimes they are orthogonal to the other RAs, as is the case with EIDR. All, however, are members of the IDF.

The IDF sets technical policies and development agendas for the DOI infrastructure. This includes recommendations about how RAs should display and link DOIs. Of course all of these decisions have an impact on the RAs. However, the IDF provides little technical infrastructure of its own as it has no full-time staff. Instead it outsources the operation of the system to CNRI, this includes the management of the doi.org domain which the IDF owns.

The actual DOI infrastructure is hosted on a platform called the Handle System which was developed by and is currently run by CNRI. The Handle System is part of a quite complex and sophisticated platform for managing digital objects that was originally developed for DARPA. A subset of the Handle system is designated for use by DOIs and is identified by the “10” prefix (e.g. 10.5555/12345678). The Handle system itself is not based on HTTP (the web protocol). Indeed, one of the much touted features of the Handle System is that it isn’t based on any specific resolution technology. This was seen as a great virtue in the late 1990s when the DOI system was developed and the internet had just witnessed an explosion of seemingly transient, competing protocols (e.g. Gopher, WAIS, Archie, HyperWave/Hyper-G, HTTP, etc.). But what looked like a wild-west of protocols quickly settled into an HTTP hegemony. In practice, virtually all DOI interactions with the Handle system are via HTTP and so, in order to interact with the web, the Handle System employs a “Handle proxy” which translates back and forth between HTTP, and the native Handle system. This all may sound complicated, and the backend of the Handle system is really very sophisticated, but it turns out that the DOI really uses only a fraction of the Handle system’s features. In fact, the vast majority of DOI interactions merely use the Handle system as a giant lookup table which allows one to translate an identifier into a web location. For example, it will take a DOI Handle like this:

10.5555/12345678

and redirect it to (as of this writing) the following URL:

http://psychoceramics.labs.crossref.org.pluma.sjfc.edu/10.5555-12345678.html

This whole transformation is normally never seen by a user. It is handled transparently by the web browser, which does the lookup and redirection in the background using HTTP and talking to the Handle Proxy. In the late 1990s, even doing this simple translation quickly, at scale with a robust distributed infrastructure, was not easy. These days however we see dozens if not hundreds of “URL Shorteners” doing exactly the same thing at far greater scale than the Handle System.

It may seem a shame that more of the Handle Systems features are not used, but the truth is the much touted platform independence of the Handle System rapidly became more of a liability and impediment to persistence than an aid. To be blunt, if in X years a new technology comes out that supersedes the web, what do we think the societal priority is going to be?

To provide a robust and transparent transition from the squillions of existing HTTP URI identifiers that the entire world depends on?
To provide a robust and transparent transition from the tiny subset of Handle-based identifiers that are used by about a hundred million specialist resources?

Quite simply, the more the Handle/DOI systems diverge from common web protocols and practice, then the more we will jeopardise the longevity of our so-called persistent identifiers.

So, in the end, DOI registration agencies really only use the Handle system for translating web addresses. All of the other services and features one might associate with DOIs (reference resolution, metadata lookup, content negotiation, OAI-PMH, REST APIs, Crossmark, CrossCheck, TDM Services, FundRef etc) are all provided at the RA level.

But this address resolution is still critical. And it is exactly what failed for many users on January 20th 2015. And to be clear, it wasn’t the robust and scaleable Handle System that failed. It wasn’t the Handle Proxy that failed. And it certainly wasn’t any RA-controlled technology that failed. These systems were all up and running. What happened was that the standard handle proxy that the IDF recommends RAs use, “dx.doi.org”, was effectively rendered invisible to wide portions the internet because the “doi.org” domain was not renewed. This underscores two important points.

The first is that it doesn’t much matter what precisely caused the outage. In this case it was an administrative error. But the effect would have been similar if the Handle proxies had failed of if the Handle system itself had somehow collapsed. In the end, Crossref and all DOI registration agencies are existentially dependent on the Handle system running and being accessible.

The second is that the entire chain of dependencies from the RAs down through CNRI are also dependent on the DNS system which, in turn, is governed by ICANN. We should really not be making too much of the purported technology independence of the DOI and Handle systems. To be fair, this limitation is inherent to all persistent identifier schemes that aim to work with the web. It really is “turtles all the way down.”

What didn’t fail on January 19th/20th and why?

You may have noticed a lot of hedging in our description of the outage of January 19th/20th. For one thing, we use the term “rolling outage.” Access to the Handle Proxy via “dx.doi.org” was never completely unavailable during the period. As we’ve explained, this is because the error was discovered very quickly and the domain was renewed hours after it expired. The nature of DNS propagation meant that even as some DNS servers were deleting the “doi.org” entry, others were adding it back to their tables. In some ways this was really confusing because it meant it was difficult to predict where the system was working and where it wasn’t. Ultimately it all stabilised after the standard 48-hour DNS propagation cycle.

But there were also some Handle-based services that simply were not affected at all by the outage. During the outage, a few people asked us if there was an alternative way to resolve DOIs. The answer was “yes,” there were several. It turns out that “doi.org” is not the only DNS name that points to the Handle Proxy. People could easily substitute “dx.doi.org” with “dx.crossref.org” or “dx.medra.org” or “hdl.handle.net” and “resolve” any DOI. Many of Crossref’s internal services use these internal names and so the services continued to work. This is partly why we only discovered the “doi.org” was down from people reporting it on Twitter.

And, of course, there were other services that were not affected by the outage. Crossmark, the REST API, and Crossref Metadata Search all continued to work during the outage.

Protecting ourselves

So what can we do to reduce our dependencies and/or the risks intrinsic to those dependencies?

Obviously, the simplest way to have avoided the outage would have been to ensure that the “doi.org” domain was set to automatically renew. That’s been done. Is there anything else we should do? A few ideas have been floated that might allow us to provide even more resilience. They range greatly in complexity and involvement.

Provide well-publicised public status dashboards that show what systems are up and which clearly map dependencies so that people could, for instance, see that the doi.org server was not visible to systems that depended on it. Of course, if such a dashboard had been hosted at doi.org, nobody would have been able to connect to it. Stoopid turtles.
Encourage DOI RAs to have the members point to Handle proxies using domain names under the RA’s control. Simply put, if Crossref members had been using “dx.crossref.org” instead of “dx.doi.org”, then Crossref DOIs would have continued to work throughout the outage of “doi.org”. The same with mEDRA, and the other RAs. This way each RA would have control over another critical piece of their infrastructure. It would also mean that if any single RA made a similar domain name renewal mistake, the impact would be isolated to a particular constituency. Finally, using RA-specific domains for resolving DOIs might also make it clear that different DOIs are managed by different RAs and might have different services associated with them. Perhaps Crossref would spend less time supporting non-Crossref DOIs?
Provide a parallel, backup resolution technology that could be pointed to in the event of a catastrophic Handle System failure. For example we could run a parallel system based on PURLs, ARKs or another persist-able identifier infrastructure.
Explore working with ICANN to get the handle resolvers moved under the special “.arpa” top level domain (TLD). This TLD (RFC 3172) is reserved for services that are considered to be “critical to the operation of the internet.” This is an option that was first discussed at a meeting of persistent identifier providers in 2011.

These are all tactical approaches to addressing the specific technical problem of the Handle System becoming unavailable, but they do not address deeper issues relating to our strategic dependence on several third parties. Even though the IDF and CNRI provide us with pretty simple and limited functionality, that functionality is critical to our operations and our claim to be providing persistent identifiers. Yet these technologies are not in our direct control. We had to scramble to get hold of people to fix the problem. For a while, we were not able to tell our users or members what was happening because we did not know ourselves.

The irony is that Crossref was held to account, and we were in the firing line the entire time. Again, this was almost unavoidable. In addition to being the largest DOI RA, we are also the only RA that has any significant social media presence and support resources. Still, it meant that we were the public face of the outage while the IDF and CNRI remained in the background.

And this is partly why our board has encouraged us to investigate another option:

Explore what it would take to remove Crossref dependencies on the IDF and CNRI.

Crossref is just part of a chain of dependencies the goes from our members down through the IDF, CNRI and, ultimately, ICANN. Our claim to providing a persistent identifier structure depends entirely on the IDF and CNRI. Here we have explored some of the technical dependencies. But there are also complex governance and policy implications of these dependencies. Each organisation has membership rules, guidelines and governance structures which can impact Crossref members. Indeed, the IDF and CNRI are themselves members of groups (ISO and DONA, respectively) which might ultimately have policy or governance impact for DOI registration agencies. We will need to understand the strategic implications of these non technical dependencies as well.

Note that the Crossref board has merely asked us to “explore” what it would take to remove dependencies. They have not asked us to actually take any action. Crossref has been massively supportive of the IDF and CNRI, and they have been massively supportive of us. Still, over the years we have all grown and our respective circumstances have changed. It is important that occasionally we question what we might have once considered to be axioms. As we discussed above, we use the term “persistent” which, in turn, is a synonym for “stubborn.” At the very least we need to document the inter-dependencies that we have so that we can understand just how stubborn we can reasonably expect our identifiers to be.

The outage of January 20th was a humbling experience. But in a way we were lucky: Forgetting to renew the domain name was a silly and prosaic way to partially bring down a persistent identifier infrastructure, but it was also relatively easy to fix. Inevitably, there was a little snark and some pointed barbs directed at us during the outage, but we were truly overwhelmed by the support and constructive criticism we received as well. We have also been left with a clear message that, in order for this good-will to continue, we need to follow-up with a public, detailed and candid analysis of our infrastructure and its dependencies. Consider this to be the first section of a multi-part report.

@kevingashley tweets asking for followup analysis

@WilliamKilbride tweets asking for followup and lessons learned

Image Credits

Turtle image CC-BY “Unrecognised MJ” from the Noun Project

DOIs for PHD Comics’ Valentine’s Day Reading List

Geoffrey Bilder — Tue, 14 Feb 2012 00:00:00 +0000

PHD Comics has posted its Valentine’s Day Reading list.

Without DOIs!

So in order to preserve the scholarly citation record, we’ve resolved those that have DOIs….

Title: The St. Valentine’s Day Frontal Passage

Citation: Sassen, K, 1980, ‘The St. Valentine’s Day Frontal Passage’, Bulletin of the American Meteorological Society, vol. 61, no. 2, p. 122.

Crossref DOI: http://dx.doi.org.pluma.sjfc.edu/10.1175/1520-0477(1980)061<0122:TSVDFP>2.0.CO;2

Title: SUICIDE AND HOMICIDE ON ST. VALENTINE’S DAY

Citation: LESTER, D, 1990, ‘SUICIDE AND HOMICIDE ON ST. VALENTINE’S DAY’, Perceptual and Motor Skills, vol. 71, no. 7, p. 994.

Crossref DOI: http://dx.doi.org.pluma.sjfc.edu/10.2466/PMS.71.7.994-994

Title: The St. Valentineʼs Day Massacre

Citation: Eckert, W, 1980, ‘The St. Valentineʼs Day Massacre’, The American Journal of Forensic Medicine and Pathology, vol. 1, no. 1, pp. 67-70.

Crossref DOI: http://dx.doi.org.pluma.sjfc.edu/10.1097/00000433-198003000-00011

Title: For Valentine’s Day

Citation: Kutzner, H, 2001, ‘For Valentine’s Day’, Cancer, vol. 91, no. 4, pp. 804-805.

Crossref DOI: http://dx.doi.org.pluma.sjfc.edu/10.1002/1097-0142(20010215)91:4<804::AID-CNCR1067>3.3.CO;2-K

Turning DOIs into formatted citations

Karl Ward — Mon, 28 Nov 2011 00:00:00 +0000

Today two new record types were added to dx.doi.org resolution for Crossref DOIs. These allow anyone to retrieve DOI bibliographic metadata as formatted bibliographic entries. To perform the formatting we’re using the citation style language processor, citeproc-js which supports a shed load of citation styles and locales.

In fact, all the styles and locales found in the CSL repositories, including many common styles such as bibtex, apa, ieee, harvard, vancouver and chicago are supported. First off, if you’d like to try citation formatting without using content negotiation, there’s a simple web UI that allows input of a DOI, style and locale selection. If you’re more into accessing the web via your favorite programming language, have a look at these content negotiation curl examples. To make a request for the new “text/bibliography” record type: $ curl -LH “Accept: text/bibliography; style=bibtex” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 @article{Atkins_Gershell_2002, title={From the analyst’s couch: Selective anticancer drugs}, volume={1}, DOI={10.1038/nrd842}, number={7}, journal={Nature Reviews Drug Discovery}, author={Atkins, Joshua H. and Gershell, Leland J.}, year={2002}, month={Jul}, pages={491-492}} A locale can be specified with the “locale” record type parameter, like this: $ curl -LH “Accept: text/bibliography; style=mla; locale=fr-FR” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 Atkins, Joshua H., et Leland J. Gershell. « From the analyst’s couch: Selective anticancer drugs ». Nature Reviews Drug Discovery 1.7 (2002): 491-492. You may want to process metadata through CSL yourself. For this use case, there’s another new record type, “application/citeproc+json” that returns metadata in a citeproc-friendly JSON form: $ curl -LH “Accept: application/citeproc+json” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 {“volume”:“1”,“issue”:“7”,“DOI”:“10.1038/nrd842”,“title”:“From the analyst’s couch: Selective anticancer drugs”,“container-title”:“Nature Reviews Drug Discovery”,“issued”:{“date-parts”:[[2002,7]]},“author”:[{“family”:“Atkins”,“given”:“Joshua H.”},{“family”:“Gershell”,“given”:“Leland J.”}],“page”:“491-492”,“type”:“article-journal”} Finally, to retrieve lists of supported styles and locales, see:

* https://crosscite.org

style and locale repositories. There’s one big caveat to all this. The CSL processor will do its best with Crossref metadata which can unfortunately be quite patchy at times. There may be pieces of metadata missing, inaccurate metadata or even metadata items stored under the wrong field, all resulting in odd-looking formatted citations. Most of the time, though, it works.

DOIs and PubMed Central - why no links?

Ed Pentz — Wed, 14 May 2008 00:00:00 +0000

Further to my previous post “NIH Mandate and PMCIDs” we’ve been looking into linking to articles on publishers’ sites from PubMed Central (PMC). There are a couple of ways this happens currently (see details below) but these are complicated and will lead to broken links and more difficulty for PMC and publishers in managing the links. Crossref is going to be putting together a briefing note for its members on this soon.

The main issue we are raising with PMC, and that we will encourage publishers to raise too, is why doesn’t PMC just automatically link DOIs? Most of the articles in PMC have DOIs so this would require very little effort from PMC and no effort from publishers and would give readers a perisistent link to the publisher’s version of an article.

Current PMC linking methods. 1) Links on Author Manuscripts in PMC are pulled in from PubMed’s LinkOut service which requires the publisher to register with PubMed and provide linking files. The DOI can be specified as the linking mechanism via LinkOut.

For final version of articles in PMC the journal image at the top of the page can be linked to the journal homepage or can have a “this article” link to the publisher’s site. The publisher has to sign up with PMC for specifying the header graphic and the links. https://web.archive.org/web/20080916065531/http://www.pubmedcentral.nih.gov.pluma.sjfc.edu/pmcdoc/pubsetup.doc (word doc) say “The static base (http://www.biomedcentral.com/) of the URLs for this link comes from the HTML template. PMC then dynamically completes the URL by adding an issn/vol/page. ” and then says that any item in the XML (such as the DOI) can be used.

Both of the approaches outlined above require extra work and will be difficult for smaller publishers. In addition, the links will be fragile by not being based on DOIs. Publishers can specify that DOIs can be used but it isn’t easy. We’d like to leverage the resources that publishers have already put into the DOI system but automatically making the DOIs active links - it would be very easy.

Digital Objects

Tony Hammond — Tue, 30 Jan 2007 00:00:00 +0000

A couple weeks back there was a meeting of the Open Archive Initiative‘s Object Reuse and Exchange (OAI-ORE) Technical Committee hosted in the Butler Library at Columbia University, New York.

Lorcan Dempsey of OCLC blogs here on the report (PDF format) that was generated from that meeting. As does Pete Johnston of Eduserv here.

Background:

OAI-ORE is being positioned as a companion activity to the more familiar OAI-PMH protocol for metadata harvesting. OAI-ORE relates to the expression and exchange of digital objects across repositories rather than just the exchange of metadata about those objects.

The basic problem is that scholarly communication deals in units which are compound resulting from a complex of documents and/or datasets expressed in multiple formats, versions, relationships, etc. The underlying web architecture provides a fairly simple model of resources (identified with URIs) which are interconnected and can be interacted with by retrieving representations of those resources. In practice, this usually results in unique URIs (and thus resources) for each representation - think of one URI for an HTML document, another for a PDF document of the same work, and yet new URIs for those same document formats for a new version of the work. Clearly, all these representations (or documents) are related, and more importantly relate to a single underlying “work”. Web architecture as generally practiced does not provide ready mechanisms to aggregate (and compartmentalize) related documents and datasets.

My fairly simple mental picture is that the web landscape is rather like the early universe in which energy (and matter) is distributed uniformly and there is little local “intelligence” which is gradually built up through time by matter formation and aggregations of this matter leading to the more familiar “clumpy” universe with its recognizable galaxies, stars and other objects. This “clumpiness” is precisely what we are missing in the scholarly web.

Jon Udell and DOIs

Crossref — Mon, 29 Jan 2007 00:00:00 +0000

Not to get too self-referential here, but it was very cool to see that Tony Hammond has managed to get Not to get too self-referential here, but it was very cool to see that Tony Hammond has managed to get This based on a podcast interview with Tony posted on January 26th.