Linked Data on Crossref

On metadata enrichment

Dominika Tkaczyk — Thu, 19 Mar 2026 00:00:00 +0000

Metadata is communication; it can tell a story about research and paint a picture for others to respond to and learn from, across the world and throughout the forthcoming generations. Metadata can feel technical with words like ‘infrastructure’ and ‘schema’, and sometimes, like tech in general, it comes with hyperbole. But metadata really is part art (storytelling and pictures) and part science (structured models and standards) with both aspects being equally important, and requiring people as well as systems. That necessary combination of human and machine involvement also makes metadata challenging.

Crossref, as the earliest adopter of DOIs specialising in scholarly research, became synonymous with DOIs in this community. However, not everyone realises that DOIs can be registered with any one of nine different agencies, which are all separate organisations with entirely separate systems that do not at present integrate or connect. And what’s more – there isn’t a central or shared “DOI schema” – each agency develops the metadata for the purposes of their organisation or community. In Crossref’s case, with our vision to create the research nexus as a complete and robust network of relationships between objects, people, and institutions of scholarship – that community encompasses the whole of the research enterprise.

The immense 180 million records of research outputs in Crossref are maintained in a system that 24,000 member organisations have already invested in. Those records benefit from rich and format-appropriate metadata schema, developed in close collaboration with the community, which makes it possible for our members to offer contextual information about each object they register. We have a long history of working with our members on recording that context, creating tools, and providing support to adopt standard metadata, enriching the context for the benefit of the scholarly community, and society at large.

Of course, those metadata records are not perfect, both in terms of quality and completeness, and the frustration around gaps in metadata is particularly strong. We are working to improve the quality and completeness of the metadata from many angles: by working with the community to understand their needs and obstacles, by identifying and analysing potential sources for additional metadata, by maintaining and adopting the existing system to changing environment, and by planning a new flexible system that will allow third-party assertions and automated enrichment workflows.

In 2020, we published a paper for the inaugural issue of Quantitative Science Studies on Crossref: The Sustainable Source of Community-Owned Scholarly Metadata and blogged an introduction to it under Crossref Metadata for Bibliometrics. One of the things our analyses in 2019 showed was that over 80% of records between 2013-2016 had been updated. Reviewing the numbers recently, we continue to see this stewardship and maintenance of metadata, amounting to almost 70% of records from the past decade being updated at least once. On the dawn of reaching 2 billion citation links, we’d like to share our experience, plans, and views on this ubiquitous activity of updating and connecting metadata – by our members and by automations built into the system by us. Altogether, these constitute the enrichment process to improve the usability of the information for the community.

Metadata available through Crossref

Crossref collects, processes, stores, and shares metadata records for a wide range of research outputs. While each record describes an individual research output, it also mentions other entities and their attributes - and, most importantly, the relationships between them. Two works identified by DOIs, for example, may be linked by a citation relationship. A person identified by an ORCID may be connected to an institution identified by a ROR ID through an affiliation relationship. A preprint and its corresponding journal article, each with its own DOI, can be linked by an “is preprint of” relationship. A research output may be associated with a grant through a “financed by” relationship. Together, these entities and relationships form the foundational building blocks of the research nexus.

As of March 14, 2026, the Crossref database contains 180,034,490 metadata records describing research outputs. You can download all the records and examine them yourself in the latest public data file. The plot below illustrates how the number of works has changed over time, showing that the rate of growth is accelerating.

The metadata records describe research outputs of various types, including:

journal articles
books and book chapters
conference proceedings
peer reviews
reports
datasets
preprints
dissertations
grants
and more

The majority of works in the Crossref database (67%) are journal articles. However, the distribution of record types has changed considerably over time. Newer types, such as components, datasets, and posted content, are growing more quickly than more traditional ways of communicating research:

Research outputs in the Crossref database are represented by rich metadata records, which may include:

basic bibliographic metadata (title, publication dates, contributors, journal title, conference name, volume and issue numbers)
authors’ affiliations and ORCID identifiers
abstracts and links to full text
funding metadata, including funders and grant details
license metadata
bibliographic reference lists
clinical trial numbers
updates such as corrections or retractions
relationships between works and other entities, such as “is translation of”, “is review of”, “is preprint of”, or “is version of”
components associated with the work, such as figures, tables, and supplemental materials

All metadata is freely available through the Crossref REST API, and additional services, such as Crossref Search, are also provided.

A natural question is: where does all this metadata come from? This is important for two main reasons. First, it helps address the question of trust, as understanding the origin of the metadata allows users to better assess its reliability. Second, it points us to the right place when investigating or addressing issues or gaps in the data.

At first glance, the answer might seem straightforward: from Crossref members. Crossref members, such as publishers, research institutions, universities, funders, museums, libraries, data and subject repositories, and conference providers, register metadata for the outputs they publish. Crossref stores this metadata and makes it available to the community.

In reality, however, the story is more complicated.

Metadata enrichment layers

The initial metadata deposit is only the beginning of what can become a long and rather fascinating journey. What users can see in our REST API is often the result of a series of updates and additions that occur over time, sometimes coming from multiple sources and happening in different ways. We can think of these ways as enrichment layers.

Each enrichment layer offers opportunities to improve the metadata while also introducing its own considerations and challenges. Rather than forming a sequence of clearly separated stages, these layers intertwine, overlap, and affect one another, collectively shaping how a research output is represented within the research nexus.

Enrichment layers are essential for completeness of the research nexus. If we relied solely on the original, one-off deposits from members, the metadata would be full of gaps, limiting the usefulness of any analysis or assessment based on it. While the scholarly metadata will never be perfectly complete, applying these enrichment layers is how we gradually and collectively build a fuller, more accurate picture of the research nexus.

One important caveat is that more metadata doesn’t magically equal better metadata. In fact, there’s often a delicate tradeoff between completeness and quality: the harder one pushes to fill every gap, the greater the chance of introducing errors. At Crossref, we believe quality comes first. We recognise that no dataset will ever be perfect, but we’re equally unwilling to apply enrichment processes without quality control. Any enrichment we introduce must meet a high bar for accuracy — no exceptions, no shortcuts.

The order of the enrichment layers discussed here loosely reflects how established they are within the scholarly ecosystem. There also might be a correlation, or at least a perceived one, between this ordering and the reliability of the underlying processes. That said, one must tread carefully when making such interpretations: perceived reliability is not the same as actual reliability.

Layer 1: Member updates

Crossref members not only deposit metadata, but also update it over time. This is an essential part of the system for several reasons. There may be errors in the originally deposited metadata that need to be corrected. Also, the initial record may contain gaps that can be filled later as more information becomes available. In addition, many changes naturally occur: landing page URLs may change, works may be archived in new locations, or identifiers for affiliated organisations may become available. Those situations also ideally result in an update.

This update process is well established. Over 24,000 Crossref members form a large global community that operates under shared membership terms. As part of these terms, members are responsible for maintaining and updating their metadata records. In this governance framework it is clearly defined who owns and stewards the metadata associated with each record, and who is responsible for the quality level and issues.

Member updates are very common. As an example, over 80% of works deposited between 2013 and 2020 were updated at least once. This demonstrates the community’s commitment to improving completeness and quality of the scholarly record. The plot below shows the percentage of works created in a given month that were updated at least once.

However, this layer also comes with challenges. It relies on members actively meeting their obligations to maintain and improve their metadata. As a result, gaps and inconsistencies can remain, and overall metadata quality is never perfect.

Our plans for the future in this area largely build on what is already happening. This includes developing and maintaining effective user interfaces for updating metadata, evolving the input metadata schema to keep pace with changes in the scholarly landscape, offering regular workshops on metadata improvements, and collaboratively establishing best practices while educating members on how to apply them.

Layer 2: Community feedback loop

Crossref metadata is widely used and examined by a large community of consumers. As a result, issues with metadata are sometimes identified by community members and reported back to us. When this happens, Crossref does not directly correct the metadata records. Instead, we contact the relevant member responsible for the record and able to deposit an update.

In this layer, the stewardship of metadata remains with the member, while responsibility for metadata quality broadens to include other actors in the community. This creates significant potential for scaling by involving a large community in identifying and reporting metadata issues.

At present, however, this process is not automated. Crossref staff effectively act as intermediaries between those reporting issues and the responsible member. As a result, the process has limited scalability. It also depends on the willingness of members to act on the reports they receive, as they are not obligated to respond to such reports.

In the future, we may explore automating portions of this workflow to handle community feedback more efficiently and lighten the load on everyone involved.

Layer 3: Metadata matching

Metadata matching is the task of finding an identifier for an item based on a structured or unstructured description of it. Matching strategies run as fully automated processes that analyse information deposited and updated by members and add identifiers, filling gaps in the metadata.

There are many instances of metadata matching problems, for example:

bibliographic reference matching: finding a DOI for a cited paper based on a bibliographic reference,
funder matching: finding the ROR ID for a funder based on its name,
affiliation matching: finding the ROR ID for an organisation based on an affiliation string,
preprint matching: finding the DOI for a preprint that precedes a given journal article,
grant matching: finding the grant DOI based on an award number and a funder name.

This layer is unique, as it focuses on a crucial type of gap in the scholarly record: the missing relationships between entities. Indeed, adding an identifier for an entity mentioned within a metadata record of a research output is typically an equivalent of asserting a relationship between that output and the matched entity. For example, bibliographic reference matching inserts citation relationships, and funder name matching - funding relationships between a research output and a funding organisation. These relationships form the foundation of the research nexus.

Currently, at Crossref, we perform two types of matching. We match bibliographic references to the DOIs of cited outputs, and funder names to Funder IDs. Both processes rely on fuzzy comparisons and other heuristic approaches to identify likely matches.

In the case of bibliographic reference matching, as it turns out, more than half of the cited DOIs (1 billion) available in the Crossref database originate from automated metadata matching:

In the case of funder name matching, the distribution is very different, but the matching strategy was still able to fill in some of the gap:

Metadata matching is a particularly valuable form of enrichment for several reasons. Matching strategies can often achieve high levels of accuracy while working in a fully automated way. This makes them highly scalable and drastically reduces the need for human oversight. Their focus on relationships also strengthens the foundations of the research nexus.

At the same time, this enrichment layer presents a number of challenges.

Its most fundamental limitation to remember is that metadata matching can only fill gaps when there is at least some useful information to work with. For example, it can identify a cited document only using structured or unstructured citation data, and the funding organisation can only be identified if some funding information is available. But if citation information, or funding information, is completely absent, as is the case for 101M (56%) records and 166M (92%) records respectively, then matching simply isn’t possible.

Matching strategies can also be complex and time-consuming to research, develop, and maintain. They require additional considerations of issues such as openness, explainability, complexity, flexibility, and cost.

Perhaps most importantly, in the case of matching, it becomes less clear who is responsible for the information introduced through the matching process. This is particularly important because matching results are never perfect, meaning there is always a risk of introducing errors. The risk is further amplified by the fact that matching strategies typically operate in a fully automated, unsupervised manner. As a result, careful evaluation of matching performance, as well as maintaining accurate provenance records, becomes increasingly important.

At Crossref, we have ambitious plans in this area. We intend to rebuild Crossref’s metadata matching workflows using modern software development and data science practices. The goal is to create a dedicated, consolidated matching service that will eventually replace all existing production matching processes, with results made available through the REST API. This project will cover six matching tasks: bibliographic reference matching, funder name matching, preprint matching, affiliation matching, grant matching, and title matching. You can learn more about metadata matching at Crossref at a dedicated project page.

Layer 4: Third-party datasets

There are many databases containing scholarly data, and one way to fill gaps in Crossref member-provided metadata is to incorporate additional metadata from those external sources.

We already have one example of this. Crossref ingests data from the Retraction Watch database to supplement information about retractions and other updates to records:

This layer has several advantages. It draws on subject-specific and metadata-specific expertise, avoids reinventing work that has already been done elsewhere, and reflects a collaborative community-driven approach to improving the scholarly record.

However, there are also important challenges to consider. Integrating external data often involves multiple data licenses or acquisition arrangements, and there may be less control over data quality compared to metadata that comes directly from members. There is also a risk that relying too heavily on external sources could shift responsibility away from the member stewards of the metadata. Finally, it can be difficult to determine which external datasets provide sufficient value and longevity to justify long-term integration.

Looking ahead, we plan to explore further opportunities to incorporate third-party datasets, carefully considering the value they bring, as well as issues of licensing, sustainability, and data quality.

Layer 5: Unstructured content scraping

A significant amount of scholarly information still exists in fully unstructured forms, such as full-text PDF documents and web pages. In principle, extracting information from these sources could help fill many gaps in existing metadata.

In a lighter-touch approach, analysing full-text documents can also help verify existing metadata elements. If such a check fails, the unverified element may be removed from the record — which, perhaps counterintuitively, can also count as enrichment, since improving accuracy is every bit as important as adding new information.

There are also important challenges to consider. Extracting metadata directly from unstructured sources could substantially shift responsibility away from the original data stewards or owners, weakening the current stewardship model. The results of automated extraction may also be inconsistent or of relatively low quality. In addition, there are potential legal and rights-related concerns, particularly when processing full-text materials. Finally, developing reliable extraction methods would require substantial research and engineering effort.

For all these reasons, the practical usefulness of this approach remains uncertain, and Crossref currently has no plans to run such processes in production. We will, however, keep a close eye on emerging extraction technologies and may consider adopting them in some form if future evaluations show clear value.

Summary

Metadata is far more than a technical afterthought of the publishing process. It is the connective tissue of the scholarly ecosystem, linking research objects, people, and institutions into a coherent, navigable network. At Crossref, this takes the form of a vast and continually evolving corpus of more than 180 million metadata records, all contributing to the emerging research nexus, being built through collective community effort to help the global research community discover, interpret, and reuse knowledge effectively.

The initial metadata record deposited by members is only the beginning. Its quality and completeness can improve over time through multiple enrichment layers: member-driven updates, community feedback, automated metadata matching, and the incorporation of third-party datasets. These processes help fill gaps and strengthen the reliability of the scholarly record, all while upholding a firm commitment to accuracy and stewardship.

Taken together, these layers reflect a long-term, collaborative effort across technology developments, community participation, and responsible automation, to ensure that scholarly metadata becomes richer, more interconnected, and more useful for everyone who relies on it.

Joint statement on research data

Hylke Koers — Tue, 28 Nov 2023 00:00:00 +0000

STM, DataCite, and Crossref are pleased to announce an updated joint statement on research data.

In 2012, DataCite and STM drafted an initial joint statement on the linkability and citability of research data. With nearly 10 million data citations tracked, thousands of repositories adopting data citation best practices, thousands of journals adopting data policies, data availability statements and establishing persistent links between articles and datasets, and the introduction of data policies by an increasing number of funders, there has been significant progress since. It now seems appropriate to focus on providing updated recommendations for the various stakeholders involved in research data sharing.

The premise of the original joint statement still stands: most stakeholders across the spectrum of researchers, funders, librarians and publishers agree about the benefits of making research data available and findable for reuse by others. This improves utility and rigor of the scholarly record. Still, research data sharing is not yet a self-evident step in the research lifecycle. We now have sufficient scholarly communication infrastructure in place to bring about widespread change and believe momentum is building for collective action.

It is in this context that DataCite, a global membership community working with over 2800 repositories around the world, and STM, whose membership consists of over 140 scientific, technical, and medical publishing organisations, are issuing this joint statement. Crossref, a nonprofit open infrastructure with over 18,000 institutional members from 150 countries, joins this call, recognising the need for an amplified focus on data citation. The aim of this statement is to accelerate adoption of best practices and policies, and encourage further development of critical policies in collaboration with a wide group of stakeholders.

Signatories of this statement recommend the following as best practice in research data sharing:

When publishing their results, researchers deposit related research data and outputs in a trustworthy data repository that assigns persistent identifiers (DOIs where available). Researchers link to research data using persistent identifiers.
When using research data created by others, researchers provide attribution by citing the datasets in the reference section using persistent identifiers.
Data repositories enable sharing of research outputs in a FAIR way, including support for metadata quality and completeness.
Publishers set appropriate journal data policies, describing the way in which data is to be shared alongside the published article.
Publishers set instructions for authors to include Data Citations with persistent identifiers in the references section of articles.
Publishers include Data Citations and links to data in Data Availability Statements with persistent identifiers (DOIs where available) in the article metadata registered with Crossref.
In addition to Data Citations, Data Availability Statements (human- and machine-readable) are included in published articles where appropriate.
Repositories and publishers connect articles and datasets through persistent identifier connections in the metadata and reference lists.
Funders and research organisations provide researchers with guidance on open science practices, track compliance with open science policies where possible, and promote and incentivize researchers to openly share, cite and link research data.
Funders, policymaking institutions, publishers and research organisations collaborate towards aligning FAIR research data policies and guidelines.
All stakeholders collaborate in the development of tools, processes, and incentives throughout the research cycle to enable sharing of high-quality research data, making all steps in the process clear, easy and efficient for researchers by providing support and guidance.
Stakeholders responsible for research assessment take into account data sharing and data citation in their reward and recognition system structures.

We, the following signatories shall adopt and promote the relevant best practices laid out above. We hope that our action inspires the community, including researchers, research funders, research institutions, data repositories and publishers, to join us in making it easy for researchers to share, link and cite research data.

Endorse the statement here.

Redirecting redirection

Geoffrey Bilder — Tue, 24 Apr 2018 00:00:00 +0000

Crossref has decided to change the HTTP redirect code used by our DOIs from 303 back to the more commonly used 302. Our implementation of 303 redirects back in 2010 was based on recommended best practice for supporting linked data identifiers. Unfortunately, very few other parties have adopted this practice.

What’s more, because using a 303 redirect is still unusual, it tends to throw SEO tools into a tizzy- and we spend a lot of time fielding SEO questions from our members about our use of 303s.

TL;DR

At this point, we need to emphasise that we have never seen our use of 303s actually affect page rankings. But at the same time, use of 303 redirects has not had wider uptake. Maintaining this quixotic behaviour just isn’t worth the effort. We hope that, in the future, we can use other techniques (e.g. signposting & cite-as) to achieve some of the things that 303 was supposed to do.

Note that these changes will not affect users or machines using DOIs. The change should be entirely transparent.

Below we provide some background to our decision and after that we provide some detailed technical notes from Jonathan Rees and Henry Thompson who have been very kind in helping to provide Crossref technical guidance on how we can help DOIs best support linked open data and adhere to HTTP best practice.

Background

Back in 2010, Crossref, DataCite (and later, several other RAs) responded to concerns that DOIs were not “linked-data friendly.” There were three problems with DOIs at that time:

It was not clear that DOIs could be used and expressed as HTTP URIs.
There was no standard way to ask a DOI to return a machine-readable representation of the data.
It wasn’t always clear if the DOI resolved to “the thing” (e.g. an article) or “something about the thing” (e.g. a landing page).

On the advice of several people in the linked data community, we proposed some options for fixing this. And we finally settled on:

Recommending that Crossref DOIs be expressed and displayed as HTTP (now HTTPS) URIs. This made it clear that DOIs could be used with HTTP applications.
Enabling DOI registration agencies to support content negotiation. This allowed RAs to support providing machine-readable representations of the data associated with a DOI.
Changing the underlying redirect code from the normal 302 to 303. This was designed to clarify what, at the time, was true- that most DOIs resolved to a landing page, not the article itself.

By any practical measure, machine use of DOIs has exploded since we made these decisions back in 2010. Crossref’s APIs and content negotiation handle over 800 million requests for machine readable data a month. Our sibling organisation, DataCite, has also seen a huge growth in machine use of DOIs. Many applications, from bibliographic management tools, to authoring systems and CRIS systems, make use of machine actionable DOIs all the time. So clearly our work to promote DOIs as machine actionable identifiers is working, but we are certain that our current use of 303 redirects has nothing to do with this growth.

First of all, as we said, very few parties have actually subscribed to the notion of using 303s to help distinguish “the thing” from “something about the thing”.

Secondly, even if they did try to rely on 303s to make this distinction, they would quickly get confused because the DOI is so often just the first in a chain of redirects which do not implement the same semantic distinction. At this point we should be clear - Crossref thinks these kinds of long redirect chains are a bad idea for two main reasons:

They slow down resolution.
They increase the number of potential failure points between the DOI and the item it resolves to.

But we also cannot legislate them away. They exist. And in the real world you will find plenty of DOIs that do a 303 redirect to a system that, in turn, does a 302 redirect to a system that does a 301 redirect and…eventually ends up someplace returning a 200. You get the picture. How on earth is a machine supposed to interpret a 303->302->301->302 redirect chain?

Furthermore - nowadays, after following this chain of redirects, you will often find yourself on a “page” that is both a landing page and the article itself. Dynamic, one-page applications can simply morph the one into the other without the use of additional HTTP requests.

In other words, using 303s is not helping machines interpret what the DOI is pointing at. And yet, people seem to be making good use of machine actionable DOIs and they are not complaining much about it.

Personally, I’d might have just been happy to switch back to using 302s simply so that I could cut down on my conversations with SEO hacks. But that wouldn’t be a principled approach. In 2010 we spent a lot of time considering the initial switch to 303s- we needed to consult with the LOD community on a potential switch back to 302s. At the January 2018 PIDapalooza I had a chance to talk to Henry Thomson about the 302/303 dilemma we faced, and he along with Jonathan Rees very generously provided the following feedback.

Best practices for HTTP redirection by persistent identifier resolvers: 302 vs. 303

Jonathan Rees (MIT CSAIL, https://orcid.org/0000-0001-7694-8250)
Henry Thompson (University of Edinburgh, School of Informatics, https://orcid.org/0000-0001-5490-1347)

If one goes to the trouble to organize an identifier system, then the desire that such a system should last as long as possible leads one to aspirationally say it’s a persistent identifier (PID) system. The unwillingness of the major browser suppliers to implement new URI schemes for PIDs initially hindered their use on the Web and this in turn inhibited widespread adoption. More recently a number of PID approaches have enjoyed very rapid growth as a result of a compromise: these PIDs participate in the World Wide Web by defining simple conversion rules mapping identifiers to actionable (‘http:’ and/or ‘https:’) forms and providing resolution servers that redirect requests for such forms to the appropriate destination.This approach has been widely adopted and is very successful, because it is so useful. An identifier’s actionable form leads, via the HTTP protocol and one or more redirections, to a web page that bears on the ground identity of the associated entity – or perhaps even directly to the entity itself, if the system is one for document entities that are naturally provided as web pages. The nature of the retrieved web page varies from one system to the next.

A confusion arose, however, over claims in various technical specifications (URIs, HTTP, Web Architecture) that the normal case is for the protocol to yield a “representation” of the “resource” “identified” by the URI. None of these terms is adequately defined by the specifications, and initially the language was not taken as normative. Those deploying identifier systems took the HTTP “resource” to be the entity associated with an identifier, and understood the “resource” as being “identified” by the URI, but it was never clear what was, or wasn’t, a “representation” of a given entity/resource: a description of the resource, the resource itself, a version of the resource, instructions on how to find the resource, etc. Sixteen years ago, in an attempt to clarify the intent of this part of the theory of URIs, and to allow applications to usefully and uniformly exploit the idea that an HTTP 200 response must deliver a “representation” of the “resource”, Tim Berners-Lee asked the W3C Technical Architecture Group to consider what came to be known as the httpRange-14 issue. It’s now 13 years after the TAG gave advice which almost no one was happy with, and 5 years after work on issue httpRedirections-57 (which superseded httpRange-14) ground to a halt. There’s still no consensus on whether it’s OK to return landing pages with a 200 status in response to requests for pictures or publications, but the Web seems to be working nonetheless, and no one seems to be bothered much anymore.

The provision of HTTP-based resolution services has stimulated widespread support for the use of identifier systems with Web resolution, particularly in the scholarly journal publication context. Those setting up HTTP resolvers responsible for identifier systems must decide which HTTP response code should be used. The TAG’s advice sows doubt on the use of the 200 response code when the response would have been a landing page, and many resolvers avoid 200 regardless and use redirection for administrative purposes, for example

‘https://dx-doi-org.pluma.sjfc.edu/10.1109/5.771073’ to

‘http://ieeexplore.ieee.org/document/771073/?reload=true’ for the DOI

‘10.1109/5.771073’, or ‘https://identifiers.org/uniprot/A0A022YWF9’ to

‘http://www.uniprot.org/uniprot/A0A022YWF9’ for the Uniprot identifier

‘A0A022YWF9’.

So the response should be a redirection, but what kind, 301, 302, or 303? (Or 307, which is almost the same as 302.) A 301 redirect seems to say that the URI is not persistent (since its target is deemed “more persistent”). A 302 redirect seems to say that the response could have come via a 200, and so suffers the same fate as 200. That leaves 303, as hinted at in the TAG’s advice. This idea got some traction: Ten years ago a Semantic Web interest group promoted the TAG’s advice in a published note, and seven years ago one of us wrote a blog post giving the same advice for resolvers for PIDs in publishing.

However, not only is there neither consensus nor general utility around this strict understanding of the use of the various response codes – that is, that resolution to a landing page is inconsistent with a 200 (and a posteriori therefore with a 302) – but also the range of usage patterns for redirection of HTTP requests has grown and ramified over time as the Web has grown and become more complex. It’s on the face of it unlikely that a mere three response codes can capture all the resulting complexity or cover the space of outcomes (in terms of e.g. what ends up in the browser address bar or what search engines index a page under) that a page owner might like to signal.

We find in practice that some PID redirections are ending up (usually after further publisher-local redirects) at the “identified” document, some at landing pages, and some at one or the other depending on the requesting site, for example in the case of paywalled material.

In the absence of a rethinking of the whole 3xx space, it seems to us that only the 301 vs. 302 distinct ion (roughly, 301 = permanent = please fix the link, and 302 = temporary = don’t change the link) is well understood and more or less consistently treated, whereas for 303, web servers are not very consistent and both search engine and citation crawler behaviours are at best inconsistent and at worst downright unhelpful.

So, we believe it is in both users’ and publishers’ interests for resolvers of actionable-form PIDs to use 302 redirects, not 303.

If we want to help machines better understand the resource that a DOI points at, we have to explore using more nuanced mechanisms.

Just using 302 for the first redirect doesn’t do everything necessary to effectively support the emerging PID+redirection architecture. It’s at the end of the redirect chains that we need more: a standardised way to find the PID back at the start of the chain. The ‘cite-as’ proposal does exactly this, and we hope it’s quickly approved and widely adopted. Once that happens a proposal for augmenting browser (and API) behaviour to prefer, or at least offer, the ‘cite-as’ link for bookmarking and copying will be needed.

DOI-like strings and fake DOIs

Geoffrey Bilder — Wed, 29 Jun 2016 00:00:00 +0000

TL;DR

Crossref discourages our members from using DOI-like strings or fake DOIs.

Details

Recently we have seen quite a bit of debate around the use of so-called “fake-DOIs.” We have also been quoted as saying that we discourage the use of “fake DOIs” or “DOI-like strings”. This post outlines some of the cases in which we’ve seen fake DOIs used and why we recommend against doing so.

Using DOI-like strings as internal identifiers

Some of our members use DOI-like strings as internal identifiers for their manuscript tracking systems. These only get registered as real DOIs with Crossref once an article is published. This seems relatively harmless, except that, frequently, the unregistered DOI-like strings for unpublished (e.g. under review or rejected manuscripts) content ‘escape’ into the public as well. People attempting to use these DOI-like strings get understandably confused and angry when they don’t resolve or otherwise work as DOIs. After years of experiencing the frustration that these DOI-like things cause, we have taken to recommending that our members not use DOI-like strings as their internal identifiers.

Using DOI-like strings in access control compliance applications

We’ve also had members use DOI-like strings as the basis for systems that they use to detect and block tools designed to bypass the member’s access control system and bulk-download content. The methods employed by our members have fallen into two broad categories:

Spider (or robot) traps.
Proxy bait.

Spider traps

A “spider trap” is essentially a tripwire that allows a site owner to detect when a spider/robot is crawling their site to download content. The technique involves embedding a special trigger URL in a public page on a web site. The URL is embedded such that a normal user should not be able see it or follow it, but an automated bot (aka “spider”) will detect it and follow it. The theory is that when one of these trap URLs is followed, the website owner can then conclude that the ip address from which it was followed harbours a bot and take action. Usually the action is to inform the organisation from which the bot is connecting and to ask them to block it. But sometimes triggering a spider trap has resulted in the IP address associated with it being instantly cut off. This, in turn, can affect an entire university’s access to said member’s content.

When a spider/bot trap includes a DOI-like string, then we have seen some particularly pernicious problems as they can trip-up legitimate tools and activities as well. For example, a bibliographic management browser plugin might automatically extract DOIs and retrieve metadata on pages visited by a researcher. If the plugin were to pick up one of these spider traps DOI-like strings, it might inadvertently trigger the researcher being blocked- or worse- the researcher’s entire university being blocked. In the past, this has even been a problem for Crossref itself. We periodically run tools to test DOI resolution and to ensure that our members are properly displaying DOIs, Crossmarks, and metadata as per their member obligations. We’ve occasionally been blocked when we ran across the spider traps as well.

Proxy bait

Using proxy bait is similar to using a spider trap, but it has an important difference. It does not involve embedding specially crafted DOI like strings on the member’s website itself. The DOI-like strings are instead fed directly to tools designed to subvert the member’s access control systems. These tools, in turn, use proxies on a subscriber’s network to retrieve the “bait” DOI-like string. When the member sees one of these special DOI-like strings being requested from a particular institution, they then know that said institution’s network harbours a proxy. In theory this technique never exposes the DOI-like strings to the public and automated tools should not be able to stumble upon them. However, recently one of our members had some of these DOI-like strings “escape” into the public and at least one of them was indexed by Google. The problem was compounded because people clicking on these DOI-like strings sometimes ended having their university’s IP address banned from the member’s web site. As you can imagine, there has been a lot of gnashing of teeth. We are convinced, in this case, that the member was doing their best to make sure the DOI-like strings never entered the public. But they did nonetheless. We think this just underscores how hard it is to ensure DOI-like strings remain private and why we recommend our members not use them.

Pedantry and terminology

Notice that we have not used the phrase “fake DOI” yet. This is because, internally, at least, we have distinguished between “DOI-like strings” and “fake DOIs.” The terminology might be daft, but it is what we’ve used in the past and some of our members at least will be familiar with it. We don’t expect anybody outside of Crossref to know this.

To us, the following is not a DOI:

10.5454/JPSv1i220161014

It is simply a string of alphanumeric characters that copy the DOI syntax. We call them “DOI-like strings.” It is not registered with any DOI registration agency and one cannot lookup metadata for it. If you try to “resolve” it, you will simply get an error. Here, you can try it. Don’t worry- clicking on it will not disable access for your university.

http://doi.org.pluma.sjfc.edu/10.5454/JPSv1i220161014

The following is what we have sometimes called a “fake DOI”

10.5555/12345678

It is registered with Crossref, resolves to a fake article in a fake journal called The Journal of Psychoceramics (the study of Cracked Pots) run by a fictitious author (Josiah Carberry) who has a fake ORCID (http://orcid.org/0000-0002-1825-0097) but who is affiliated with a real university (Brown University).

Again, you can try it.

http://doi.org.pluma.sjfc.edu/10.5555/12345678

And you can even look up metadata for it.

https://api-crossref-org.pluma.sjfc.edu/v1/works/10.5555/12345678

Our dirty little secret is that this “fake DOI” was registered and is controlled by Crossref.

Why does this exist? Aren’t we subverting the scholarly record? Isn’t this awful? Aren’t we at the very least hypocrites? And how does a real university feel about having this fake author and journal associated with them?

Well- the DOI is using a prefix that we use for testing. It follows a long tradition of test identifiers starting with “5”. Fake phone numbers in the US start with “555”. Many credit card companies reserve fake numbers starting with “5”. For example, Mastercard’s are “5555555555554444” and “5105105105105100.”

We have created this fake DOI, the fake journal and the fake ORCID so that we can test our systems and demonstrate interoperable features and tools. The fake author, Josiah Carberry, is a long-running joke at Brown University. He even has a Wikipedia entry. There are also a lot of other DOIs under the test prefix “5555.”

We acknowledge that the term “fake DOI” might not be the best in this case- but it is a term we’ve used internally at least and it is worth distinguishing it from the case of DOI-like strings mentioned above.

But back to the important stuff….

As far as we know, none of our members has ever registered a “fake DOI” (as defined above) in order to detect and prevent the circumvention of their access control systems. If they had, we would consider it much more serious than the mere creation of DOI-like strings. The information associated with registered DOIs becomes part of the persistent scholarly citation record. Many, many third party systems and tools make use of our API and metadata including bibliographic management tools, TDM tools, CRIS systems, altmetrics services, etc. It would be a very bad thing if people started to worry that the legitimate use of registered DOIs could inadvertently block them from accessing content. Crossref DOIs are designed to encourage discovery and access- not block it.

And again, we have absolutely no evidence that any of our members has registered fake DOIs.

But just in case, we will continue to discourage our members from using DOI-like strings and/or registering fake DOIs.

This has been a public service announcement from the identifier dweebs at Crossref.

Image Credits

Unless otherwise noted, included images purchased from The Noun Project

Clinical trial data and articles linked for the first time

Daniel Shanahan — Tue, 17 May 2016 00:00:00 +0000

It’s here. After years of hard work and with a huge cast of characters involved, I am delighted to announce that you will now be able to instantly link to all published articles related to an individual clinical trial through the Crossmark dialogue box. Linked Clinical Trials are here!

In practice, this means that anyone reading an article will be able to pull a list of both clinical trials relating to that article and all other articles related to those clinical trials – be it the protocol, statistical analysis plan, results articles or others – all at the click of a button.

Linked Clinical Trials interface

Now I’m sure you’ll agree that this sounds nifty. It’s definitely a ‘nice-to-have’. But why was it worth all the effort? Well, simply put: “to move a mountain, you begin by carrying away the small stones”.

Science communication in its current form is an anachronism, or at the very least somewhat redundant.

You may have read about the ‘crisis in reproducibility’. Good science, at its heart, should be testable, falsifiable and reproducible, but an historical over-emphasis on results has led to a huge number of problems that seriously undermine the integrity of the scientific literature.

Issues such as publication bias, selective reporting of outcome and analyses, hypothesising after the results are known (HARKing) and p-hacking are widespread, and can seriously distort the literature base (unless anyone seriously considers Nicholas Cage to be causally related to people drowning in swimming pools).

This is, of course, nothing new. Calls for prospective registration of clinical trials date back to the 1980s and it is now becoming increasingly commonplace, recognising that the quality of research lies in the questions it asks and the methods it uses, not the results observed.

Uptake of trial registration year-on-year since 2000

Building on this, a number of journals and funders – starting with BioMed Central’s Trials over 10 years ago – have also pushed for the prospective publication of a study’s protocol and, more recently, statistical analysis plan. The idea that null and non-confirmatory results have value and should be published has also gained increasing support.

Over the last ten years, there has been a general trend towards increasing transparency. So what is the problem? Well, to borrow an analogy from Jeremy Grimshaw, co-Editor-in-Chief of Trials – we’ve gone from Miró to Pollock.

Although a results paper may reference a published study protocol, there is nothing to link that report to subsequent published articles; and no link from the protocol itself to the results article.

A single clinical trial can result in multiple publications: the study protocol and traditional results paper or papers, as well as commentaries, secondary analyses and, eventually, systematic reviews, among others, many published in different journals, years apart. This situation is further complicated by an ever-growing body of literature.

Researchers need access to all of these articles if they are to reliably evaluate bias or selective reporting in a research object, but – as any systematic reviewer can tell you – actually finding them all is like looking for a needle in a haystack. When you don’t know how many needles there are. With the haystack still growing.

That’s where we come in. The advent of trial registration means that there is a unique identifier associated with every clinical trial, at the study-level, rather than the article level. Building on this, the Linked Clinical Trials project set out to connect all articles relating to an individual trial together using its trial registration number (TRN).

By adapting the existing Crossmark standard, we have captured additional metadata about an article, namely the TRN and the trial registry, with this information then associated with the article’s DOI on publication. This means that you will be able to pull all articles related to an individual clinical trial from the Crossmark dialogue box on any relevant article.

This obviously has huge implications for the way science is reported and used. By quickly and easily linking to related published articles, it will enable editors, reviewers and researchers to evaluate any selective reporting in the study, and help to provide far greater context for the results.

As all the metadata will be open access (CC0), with no copyright, it will also be possible to access this article ‘thread’ through the Crossref Metadata Search, or independently through an application programming interface (API). This provides a platform for others to build on, with many already looking to take the next step, such as Ben Goldacre’s new Open Trials initiative.

However, in order for this to work, we must capture as many articles and trials as possible to create a truly comprehensive thread of publications. We currently have data from the NIHR Libraries, PLoS and, of course, BioMed Central, but need more publishers and journals to join us in depositing clinical trial metadata. After all, without metadata, this is all merely wishful thinking.

Let’s hope we’re the pebble that starts the landslide.

Crossref & the Art of Cartography: an Open Map for Scholarly Communications

Jennifer Lin — Fri, 08 Jan 2016 00:00:00 +0000

In the 2015 Crossref Annual Meeting, I introduced a metaphor for the work that we do at Crossref. I re-present it here for broader discussion as this narrative continues to play a guiding role in the development of products and services this year.

Metadata enable connections

At Crossref, we make research outputs easy to find, cite, link, and assess through DOIs. Publishers register their publications and deposit metadata through a variety of channels (XML, CSV, PDF, manual entry), which we process and transform into Crossref XML for inclusion into our corpus. This data infrastructure which makes possible scholarly communications without restrictions on publisher, subject area, geography, etc. is far more than a reference list, index or directory.

If research builds on what came before, one could claim that the process of knowledge production is partly the story of the very relationships between results disseminated (i.e., publications). So let’s consider each publication as a node in a graph where each has a coordinate and is connected by its citations to other publications (as well those that cite it). Additionally, each is associated with a set of people and places, along with a whole host of elements involved in the research and dissemination process.

But take a wider berth, and we begin to capture relationships between all such contributing agents and objects involved in the research process. Here we find an array of entities belonging to the scholarly graph, including different types of research artifacts, publisher and journal, funders, ORCIDs, peer reviews, publication status updates (corrections, retractions, etc.), citations, license information, additional URLs (machine destinations, hosting platforms, etc.), underlying data, software and protocols, materials, discussions and blog posts, recommendations, reference work mentions, etc. The entities on the graph multiply at an even higher rate as researchers share more outputs across more channels. And over time, the graph expands exponentially, producing a webbing that is far more dense and far more vast than we can currently imagine. Perhaps even to the point we realize Borges’ story where a cartographer builds a map so large it replicates the territory itself (On Exactitude in Science)!

From graph to cartography

At the heart of Borges’s poignant story is the map. Crossref’s graph of scholarly communications could be seen in the same light. It has a representational aspect, which is not purely abstract and can be visualized. Here, a map becomes an incredibly potent metaphor. Each link enabled by publisher-deposited metadata is a new street, bridge, or highway that takes us to a particular place (i.e., entity) of interest. These roads lead to articles, researchers, funders, institutions, etc., and in doing so, make them discoverable. They tell a story about the roles of each in the broader research in the landscape dotted with a plethora of places.

The scholarly web has a growing corpus of more than 78 million publications at this very moment registered with Crossref. On average ten to fifteen thousand new objects appear every day. Maps are all the more essential for getting around in a bewildering environment of new and unfamiliar places, even for known ones in areas of exploding growth. They are critical for orienteering, discovering relationships, identifying sets of associated objects, naming new neighborhoods that emerge (i.e., new research specialties), etc. And if each connection on the map is seen as an event, maps can also represent micro-narratives about the research process and the agents involved. A multi-dimensional map containing all these entities, which serves as an evolving representation of spacetime that is constantly updated and always available, would finally begin to depict the process of scholarly activity as a dynamic, evolving, almost living system.

An open map for scholarly communication

Crossref builds such a scholarly map of the research enterprise and makes it openly available for the entire research ecosystem. Call this a meta map or, more recently, call it metastructure. No matter what name it goes by we call it infrastructure at Crossref.

Crossref’s open map for scholarly communications is a core part of the open information infrastructure for scholarly research. Crossref map data are open, portable, as well as licensed and provisioned for maximum reuse to serve the whole community. This open resource has two entrances: one for humans, another for machines. The Crossref REST API enables machines to traverse this environment and mine it in equal measure to the humans behind them. It is configured so that a robot can learn, a phone can access, and platforms can be built.

OpenStreetMap and Google Maps, both widely used and mature infrastructure maps, are instructive examples when we consider a map of this kind for scholarly communications. Map data can be represented in unlimited ways, depending on any variety of needs and users. Third parties can add content via interactive layers that tell different stories such as health expenditure by country based on GDP and coral reefs at risk. They have a broad base of users across business models from philanthropic services aimed at disaster relief (Refugeemaps.eu) to commercial entities providing drivers with locations on open parking spaces (AppyParking on Google Map, PocketParker on OpenStreetMap). They power platforms and services that build maps for others (MapQuest, MapBox). They have applications far beyond the business of maps. For example, Place picker is a Google Maps widget that supports easy auto-complete the entry of any place or location on a mobile app where typing is a chore. And as far use cases close to home, the two have served as raw data for academic research (ex: workflow for generating multi-agent traffic simulation scenarios, automatic classification of GPS trajectories for transportation modes, etc.).

In kind, the Crossref infrastructure map also supports: the development of any variety of new maps which re-present the data, the makers of map platforms that power the research enterprise, tools that use map data, as well as academic research (bibliometrics). We extract slices of data of common interest from the map and add them as additional layers by which anyone can access and create applications on or across these bands of data:

Contributors (authors, editors, reviewers)
Funding information (funding body, grant number)
Trial & study information (clinical trials registry number, registered report, replication study)
Publication history (versions, updates, revisions, corrections, retractions, dates received/accepted/published)
Peer review (status, type, reviews)
Access indicators (publication license for text & data mining, machine mining URLs)
Resources & associated research artifacts (preprints, figures & tables, datasets, software, protocols, research resource IDs)
Activity surrounding the publication (peer reviews, comments & discussions, bookmarks, social shares, recommendations).

Today, the map powers a host of public and commercial organisations alike for a wide range of scholarly and non-scholarly purposes:

<td style="border: 1px solid #ffffff;">
<ul>
<li>
<span >Indexing services</span>
</li>
<li>
<span >Publishing vendors</span>
</li>
<li>
<span >Peer review systems</span>
</li>
<li>
<span >Reference manager systems</span>
</li>
<li>
<span >Lab & diagnostics suppliers</span>
</li>
<li>
<span >Info management systems</span>
</li>
<li>
<span >Educational tools</span>
</li>
<li>
<span >Data analytics systems</span>
</li>
<li>
<span >Literature discovery services</span>
</li>
</ul>
</td>

Publishers
Funders
Research institutions
Archives & repositories
Research councils
Data centres
Professional networks
Patent offices
Registration Agencies

We will follow up this post to highlight a cross-section of these consumers in the Crossref map ecosystem and elaborate on what & how they have built from our data. An infrastructure map offers endless potential to third parties across publishers, funders, research institutions, and vendors working to serve the scholarly research enterprise.

The art of cartography

In the Crossref Product Management team, we have ambitious plans for map enhancements this year. They focus on expanding information density and ease of access to the data. In the former case, we will introduce a new class of locations where activity surrounding the publications are occurring when we launch the DOI Event Tracker. We will also initiate an extensive publisher campaign to achieve full metadata deposit completeness across our membership. No one can keep pace with the sheer volume of research activity happening online nor wander the Lonely Web of research alone. The more metadata publishers provide for a publication, the more roads lead to its map location. After all, discoverability is closely associated with connectedness on a map. And finally, in the latter case, we will refresh and enhance the user interface to make it more powerful for humans to traverse the ever-changing landscape (as easily as the REST API enables machines!).

I gratefully acknowledge the feedback received from the following who served as generous and insightful sounding boards: Virginia Barbour, Theo Bloom, Martin Eve, Daniel S. Katz, Amye Kenall, Catriona MacCullum, Cameron Neylon, Mark Patterson, Kristen Ratan, Carly Strasser, and Kaitlin Thaney.

DOI Event Tracker (DET): Pilot progresses and is poised for launch

Jennifer Lin — Tue, 15 Sep 2015 00:00:00 +0000

Publishers, researchers, funders, institutions and technology providers are all interested in better understanding how scholarly research is used. Scholarly content has always been discussed by scholars outside the formal literature and by others beyond the academic community. We need a way to monitor and distribute this valuable information.

The Crossref DOI Event Tracker (DET)

To meet this need, Crossref will be introducing a new service that tracks activity surrounding a research work from potentially any web source where an event is associated with a DOI. Following a successful pilot run started Spring 2014, the service has been approved to move toward production and is expected to launch in 2016. Any party wishing to join this phase is welcome to contact Jennifer Lin. The DOI Event Tracker (DET) registers a wide variety of events such as bookmarks, comments, social shares, citations, and links to other research entities, from a growing list of online sources. DET aggregates them, and stores and delivers the data in many ways.

Open, portable, and licensed for maximum reuse

Crossref has long served as the citation linking and metadata infrastructure provider for scholarly communication; the new DOI Event Tracker is a natural next step, providing a practical solution as a resource for the whole community. The tracker offers the following features:

Data on event activity across a common pool of online channels.
Near real-time alerting for select sources with push notifications to the system.
Cross-publisher monitoring to enable benchmarking and provide context to the data.
Common format for normalizing data results across the diverse set of sources via modern REST API.
Secure and regularly refreshed backups of critical data for long term data preservation.
Transparency of data collection so as to ensure auditable, replicable, and trustworthy results.
Query-initiated retrieval or real-time alerts when an event of interest occurs.
CC-0 license for open and flexible propagation of data.

A number of platforms are already confirmed and more parties are welcomed at any stage. So far we have confirmation to track DOI events on the following platforms:

[table id=1 /]

This set of sources reflects our initial focus on parties willing to allow their data to be redistributed in the common pool. Efforts are underway to expand the source list to include Twitter and MyScienceWork, among others. Publishers can also act as sources by publishing and distributing DOI event data via the DET when an event occurs on its platform (for example, when a PDF is downloaded, or when a comment mentions a DOI in a locally hosted discussion forum, etc.). This would make local DOI activity globally available to funders, researchers, institutions, etc.

DET provides benefits of scale and ease of access as a central point for collecting and propagating data to the community. As a single point of access, it overcomes the business and technical hurdles that are a part of managing multiple online sources where scholarly activity occurs, in a rapidly changing landscape of online channels. This resource covers content across publishers and serves as a strong foundation to support the development of tools and services by any party. DET users will always be able to combine the DET data with those individually collected via negotiated or paid access. DET remains a utility separate from any value-added amenities, such as analytics, presentation, and reporting.

DET Service-Level Agreement

For those who seek the highest level of service and a more flexible range of access options, Crossref will provide a Service-Level Agreement (SLA) service for the DOI Event Tracker. The DET SLA includes the following additional features on top of the common data offering:

Access to the complete suite of sources, which includes restricted and/or paid sources in addition to common data, providing the fullest picture of DOI usage activity possible.
Guaranteed uptime and response time to the latest raw data on the aggregate activity surrounding a DOI.
Guaranteed support response time to questions and issues surrounding data and data delivery.
Flexible data access options: on-demand real time data access and scheduled bulk downloads for processing batch analytics.
Optimum retrieval rates and accelerated delivery speeds with the dedicated SLA API.
Access to a webhook API for events of interest as an alternative to polling DET.
Standardized and enhanced linkback service for the difficult-to-track, grey literature.

The DET SLA service has a simple, value-based pricing model based on subscriber size. Register your interest in Crossref’s DOI Event Tracker and the DET SLA service if you would like stay informed of the upcoming launch. Please contact Jennifer Lin for more information.

Image modified from “Radar” icon by Karsten Barnett from the Noun Project.

♫ Researchers just wanna have funds ♫

Geoffrey Bilder — Thu, 10 Apr 2014 00:00:00 +0000

photo credit

Summary

You can use a new Crossref API to query all sorts of interesting things about who funded the research behind the content Crossref members publish.

Background

Back in May 2013 we launched Crossref’s FundRef service. It can be summarized like this:

Crossref keeps and manages a canonical list of Funder Names (ephemeral) and associated identifiers (persistent).
We encourage our members (or anybody, really- the list is available under A CC-Zero license waiver) to use this list for collecting information on who funded the research behind the content that our members publish.
We then ask that our members deposit this data in their normal Crossref metadata deposits.

And that was cool.

But then people started asking us awkward questions. Questions like “what can I do with the funder data?” and “how do I query it?”.

Stoopit people.

Can’t you just let us bask for a few minutes in the sunny glow of actually conceiving of and launching a project within a year?

But seriously, funders, were interested to see how they could use the funder metadata being collected in Crossref. In particular, some funding agencies were interested in being able to measure Key Performance Indicators (“KPIs” to management wonks) related to recent mandates such as the February 22nd 2013 OSTP memo, Public Access to the Results of Federally Funded Research. Two groups also approached us, CHORUS and SHARE. Both are interested in exploring how to build reporting tools for funders, institutions and researchers and each brought us a gigantic hairball of use-cases they were hoping we would be able to meet.

Conveniently, we were in the process of creating a revised, modern Crossref API that is entirely buzzword-compliant, and so we set to work…

We thought people might be interested in seeing what you can do with the Crossref REST API in relation to funding information and the expectations that are increasingly being attached to them. CHORUS is already using the Crossref REST API heavily and we expect that SHARE will soon start making use of it as well. The feedback from both groups has been very useful, but we are looking for broader feedback as well. The API is still in development, so now is your chance to help us shape it.

Brief Examples

Please note, the following are APIs calls, although you can copy and paste the URIs into your browser, the data is returned in a machine readable representation called JSON. If you want the results to look a little more presentable, we advise you install the JSONView plugin:

Firefox Users: JSONView
Chrome Users: JSONView

Also note that publishers have only just started to deposit the metadata needed for these APIs to work, so the data is currently sparse. We know that many of our members are working feverishly to populate more of the needed metadata, but this requires updates to the their manuscript tracking systems, production systems and hosting systems. It takes time.

But for now you can paste the relevant URIs below into your browser and see the results that we do have. Expect these numbers to increase sharply over the next few months

To start with, you might want to know how many articles in Crossref have FundRef metadata:

https://api-crossref-org.pluma.sjfc.edu/v1/works?filter=has-funder:true&rows=0

You could then be interested in knowing how many works in Crossref use FundRef to credit the United States’ National Science Foundation (NSF) for funding their research? First you need to find out what the FundRef identifier is for the NSF:

https://api-crossref-org.pluma.sjfc.edu/v1/funders?query=NSF

You can see that there are several entries that match “NSF”, and that the one we are looking for has the identifier http://dx.doi.org.pluma.sjfc.edu/10.13039/100000001. Remember, funding agency names can change frequently, the ID provides a persistent link to the funder even if their name changes.

If you are curious, you can see the details for the NSF entry, including its location, parent and child organisations:

https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001

Notice that the results also lists the work-count. This is the number of works in the Crossref metadata that list the US NSF as having funded the research.

So perhaps you would like to see the list of works. The following will list the first twenty:

https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001/works

You can page through the results with the offset argument:

https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001/works?offset=20
https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001/works?offset=40
...

How many works that have listed the NSF as a funder have license information:

https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001/works?filter=has-license:true&rows=0

Lets see the first batch that have license information:

https://api-crossref-org.pluma.sjfc.edu/v1/funders/10.13039/100000001/works?filter=has-license:true

Lets look at the metadata for one of the DOIs returned:

https://api-crossref-org.pluma.sjfc.edu/v1/works/10.1063/1.3593378

Interesting, the metadata shows an article published by AIP. It includes license information (CC-BY 3.0) as well as a link to the full text. If you follow the link to the full text, you can retrieve it:

http://link.aip.org.pluma.sjfc.edu/link/applab/v98/i21/p216101/pdf/CHORUS

Wow- A pretty short article. But you can see that it does credit the NSF and that the award number recorded in the text is the same as the award number recorded in the FundRef section of the Crossref metadata. Yay.

You can see in the brief examples above that there is a lot of other metadata you may want to query on and explore. It can include ORCIDS, information about archiving arrangements- even abstracts. It all depends on what the Crossref member has decided to provide.

You can get a simple overview of what a Crossref member has provided by looking at a member summary. Here is an example for Hindawi:

https://api-crossref-org.pluma.sjfc.edu/v1/members?query=hindawi

Note again that names are fickle, so the above query can also be accomplished using the member identifier like this:

https://api-crossref-org.pluma.sjfc.edu/v1/members/98

Groovy init?

If you want more pointers on where you can learn how to use the API, read on…

The Future of the Crossref REST API.

Our aim is for the Crossref REST API to go into production this Summer (2014). As with most of our newer APIs, there will be a free API for public use and a paid for API for professional use. The only difference between the two will be that the professional version will come with a service level agreement (SLA) covering uptime, response time and support. Naturally, this also means that the professional one will be on dedicated hosting equipment so that we can meet these SLAs, whereas the performance of the free version will be subject to the vicissitudes inherent in using a shared, constrained resource (i.e. the server and network it is running on).

Again, the basics of the API are in place. It should be fairly stable, but we do reserve the right to make changes to it over the next few months. Please send us feedback.

— The Weasel

Turning DOIs into formatted citations

Karl Ward — Mon, 28 Nov 2011 00:00:00 +0000

Today two new record types were added to dx.doi.org resolution for Crossref DOIs. These allow anyone to retrieve DOI bibliographic metadata as formatted bibliographic entries. To perform the formatting we’re using the citation style language processor, citeproc-js which supports a shed load of citation styles and locales.

In fact, all the styles and locales found in the CSL repositories, including many common styles such as bibtex, apa, ieee, harvard, vancouver and chicago are supported. First off, if you’d like to try citation formatting without using content negotiation, there’s a simple web UI that allows input of a DOI, style and locale selection. If you’re more into accessing the web via your favorite programming language, have a look at these content negotiation curl examples. To make a request for the new “text/bibliography” record type: $ curl -LH “Accept: text/bibliography; style=bibtex” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 @article{Atkins_Gershell_2002, title={From the analyst’s couch: Selective anticancer drugs}, volume={1}, DOI={10.1038/nrd842}, number={7}, journal={Nature Reviews Drug Discovery}, author={Atkins, Joshua H. and Gershell, Leland J.}, year={2002}, month={Jul}, pages={491-492}} A locale can be specified with the “locale” record type parameter, like this: $ curl -LH “Accept: text/bibliography; style=mla; locale=fr-FR” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 Atkins, Joshua H., et Leland J. Gershell. « From the analyst’s couch: Selective anticancer drugs ». Nature Reviews Drug Discovery 1.7 (2002): 491-492. You may want to process metadata through CSL yourself. For this use case, there’s another new record type, “application/citeproc+json” that returns metadata in a citeproc-friendly JSON form: $ curl -LH “Accept: application/citeproc+json” http://dx.doi.org.pluma.sjfc.edu/10.1038/nrd842 {“volume”:“1”,“issue”:“7”,“DOI”:“10.1038/nrd842”,“title”:“From the analyst’s couch: Selective anticancer drugs”,“container-title”:“Nature Reviews Drug Discovery”,“issued”:{“date-parts”:[[2002,7]]},“author”:[{“family”:“Atkins”,“given”:“Joshua H.”},{“family”:“Gershell”,“given”:“Leland J.”}],“page”:“491-492”,“type”:“article-journal”} Finally, to retrieve lists of supported styles and locales, see:

* https://crosscite.org

style and locale repositories. There’s one big caveat to all this. The CSL processor will do its best with Crossref metadata which can unfortunately be quite patchy at times. There may be pieces of metadata missing, inaccurate metadata or even metadata items stored under the wrong field, all resulting in odd-looking formatted citations. Most of the time, though, it works.

DataCite supporting content negotiation

Geoffrey Bilder — Mon, 10 Oct 2011 00:00:00 +0000

In April In April for its DOIs. At the time I cheekily called-out DataCite to start supporting content negotiation as well.

Edward Zukowski (DataCite’s resident propellor-head) took up the challenge with gusto and, as of September 22nd DataCite has also been supporting content negotiation for its DOIs. This means that one million more DOIs are now linked-data friendly. Congratulations to Ed and the rest of the team at DataCite.

We hope this is a trend. Back in June Knowledge Exchange organized a seminar on Persistent Object Identifiers. One of the outcomes of the meeting was “Den Haag Manifesto” a document outlining five relatively simple steps that different persistent identifier systems could take in order to increase interoperability. Most of these steps involved adopting linked data principles including support for content negotiation. We look forward to hearing about other persistent identifiers adopting these principles over the next year.

Having said that, this time I will refrain from calling-out anybody specifically…

Content Negotiation for Crossref DOIs

Geoffrey Bilder — Tue, 19 Apr 2011 00:00:00 +0000

So does anybody remember the posting DOIs and Linked Data: Some Concrete Proposals?

Well, we went with option “D.”

From now on, DOIs, expressed as HTTP URIs, can be used with content-negotiation.

Let’s get straight to the point. If you have curl installed, you can start playing with content-negotiation and Crossref DOIs right away:

curl -D - -L -H   “Accept: application/rdf+xml” “http://dx.doi.org.pluma.sjfc.edu/10.1126/science.1157784”

curl -D - -L -H   “Accept: text/turtle” “http://dx.doi.org.pluma.sjfc.edu/10.1126/science.1157784”

curl -D - -L -H   “Accept: application/atom+xml” “http://dx.doi.org.pluma.sjfc.edu/10.1126/science.1157784”

Or if you are already using Crossref’s “unixref” format:

curl -D - -L -H “Accept: application/unixref+xml” “http://dx.doi.org.pluma.sjfc.edu/10.1126/science.1157784&#8221;

This will work with over 46 million Crossref DOIs as of today, but the beauty of the setup is that from now on, any DOI registration agency can enable content negotiation for their constituencies as well. DataCite- we’re looking at you 😉 .

It also means that, as registration agency members (Crossref publishers, for instance) start providing more complete and richer representations of their content, we can simply redirect content-negotiated requests directly to them.

We expect that that this development will round-out Crossref’s efforts to support standard APIs including OpenURL and OAI_PMH and we look forward to seeing DOIs increasingly used in linked data applications.

Finally, Crossref would just like to thank the IDF and CNRI for their hard work on this as well as Tony Hammond and Leigh Dodds for their valuable advice and persistent goading.

DOIs and Linked Data: Some Concrete Proposals

Geoffrey Bilder — Thu, 25 Mar 2010 00:00:00 +0000

Since last month’s threads (here, here, here and here) talking about the issues involved in making the DOI a first-class identifier for linked data applications, I’ve had the chance to actually sit down with some of the thread’s participants (Tony Hammond, Leigh Dodds, Norman Paskin) and we’ve been able sketch-out some possible scenarios for migrating the DOI into a linked data world.

I think that several of us were struck by how little actually needs to be done in order to fully address virtually all of the concerns that the linked data community has expressed about DOIs. Not only that- but in some of these scenarios we would put ourselves in a position to be able to semantically-enable over 40 million DOIs with what amounts to the flick of a switch.

Given the huge interest in linked data on the part of researchers and Crossref members- it seems like it would be a fantastic boon to both the IDF (International DOI Foundation) and Crossref if we were able to do something quickly here.

Anyway- The following are notes outlining several concrete proposals for addressing the limitations of DOIs as identifiers in linked data applications. They range in complexity/effort involved- with the simplest scenario providing minimal (yet functional) LD capabilities for just one RA’s members (Crossref’s) and the most complex providing per-RA and per-RA-member configurability on how DOIs would behave for LD applications.

We’d appreciate comments, questions, suggestions, corrections, etc.

A: Simplest Scenario

What would need to be done?

Crossref implements a linked data service. For example, hosted at rdf.crossref.org.
Crossref recommends that any member publisher who wants to add rudimentary linked data capabilities to their site could simply insert some simple link elements into their landing Pages. So, for instance, for the article with the DOI 10.5555/1234567 in the Journal of Psychoceramics, the publisher would put the following in the landing page for the article:

<link rel="primarytopic" href="http://doi.crossref.org.pluma.sjfc.edu/10.5555/1234567" /> <link rel="alternate" type="application/rdf+xml" href="http://rdf.crossref.org.pluma.sjfc.edu/metadata/10.5555/1234567.rdf" title="RDF/XML version of this document"/> <link rel="alternate" type="text/html" href="http://www.journalofpsychoceramics.org/10.5555/1234567.html" title="HTML version of this document"/> <link rel="alternate" type="application/json" href="http://rdf.crossref.org.pluma.sjfc.edu/metadata/10.5555/1234567.json" title="RDF/JSON version of this document"/> <link rel="alternate" type="text/turtle" href="http://rdf.crossref.org.pluma.sjfc.edu/metadata/10.5555/1234567.ttl" title="Turtle version of this document"/>

In the above snippet the HTML version of the document is the publisher’s existing landing page.

How it would work

A sem-web-enabled browser would query dx.doi.org/10.5555/1234567 and get a normal 302 redirect to the publisher’s landing page.
The sem-web-enabled browser would sniff the page for the link elements and retrieve the representations it wanted from rdf.crossref.org
The returned document would contain an appropriate representation of the metadata that the publisher has deposited with Crossref. It would also assert that:

doi.crossref.org/10.5555/12334567 owl:sameAs dx.doi.org/10.5555/1234567 . dx.doi.org/10.5555/12334567 owl:sameAs info:doi/10.5555/12334567</p> info:doi/10.5555/12334567 owl:sameAs doi:10.5555/1234567

Alternatively, the publisher could implement their own linked data support on their own domain using whatever appropriate method they want. So, for instance, a larger publisher could support content negotiation at their site and return different/enhanced metadata, etc.

Pros

Doesn’t require changes at DOI/Handle levels
Is easy for publisher to opt-in or opt-out
Requires minimal development on the part of Crossref.

Cons

Only applies to Crossref DOIs.
It depends on publishers taking action. Might be a long time before publishers add the needed links to their landing pages or support content negotiation.
DOI system is still not strictly LD compliant (e.g. it is returning 302 redirects. Naive sem-web browsers might ‘stop’ after getting a 302. Should ideally use 303s, content negotiation, etc.)
Doesn’t work for DOIs that currently bypass landing pages and which go directly to content.

B: Simple + IDF Global Semantic Compliance

What would need to be done?

Same as “Simplest Scenario”
IDF globally changes dx.doi.org to return 303 redirect

How would it work?

Same as Simplest Scenario, except that, because sem-web-enabled browser had been told it was being redirected to a NIR (via the 303), it would presumably be more likely to continue.

Pros

All DOIs conform to expectations for LD identifiers
Easy for publisher to opt-in or opt-out
Requires minimal development on part of Crossref
Requires minimal work (?) on part of IDF

Cons

Requires global change on part of IDF. Global change might conflict with requirements of other RAs.
It depends on publishers taking action. Might be a long time before publishers add needed links to their landing pages or support content negotiation.
Doesn’t work for DOIs that currently bypass landing pages (e.g. OECD spreadhseets, UICR datasets, etc.)

C: Simple + IDF Global Semantic Compliance + RA CN Intercept

What would need to be done?

Same as “B: Simple + IDF Global Semantic Compliance” Scenario
IDF changes dx.doi.org to redirect content-negotiated dx.doi.org queries to RA-controlled resolver depending on the preferences of the RA.
RA implements DOI resolver (e.g. dx.crossref.org) that supports content negotiation. RA allows its members to specify to the RA that they want either: <ol type=a>
RA to forward all requests to the member’s site.
RA to “intercept” content-negotiations for non-HTML representations and direct them appropriately (e.g. return appropriate representation from rdf.crossref.org)

How would it work?

Pros

All DOIs conform to expectations for LD identifiers
Allows RA to potentially LD-enable its members very quickly.
Easy for ra-members to opt-in or opt-out
Requires minimal development on part of Crossref
Would even work for DOIs that bypass landing pages

Cons

Requires global change on part of IDF. Global change might conflict with requirements of other RAs.
Requires change to add decision logic implementation on part of IDF.
Requires development of RA resolvers that implement per-member resolution logic (note- this would probably actually be done at DOI level)

D: Simple + IDF Selective Semantic Compliance + RA CN Intercept

What would need to be done?

Same as Simplest Scenario
IDF changes dx.doi.org to return either 302 or 303 redirect depending on the preferences of the RA.
IDF changes dx.doi.org to redirect content-negotiated dx.doi.org queries to RA-controlled resolver depending on the preferences of the RA.
RA implements DOI resolver (e.g. dx.crossref.org) that supports content negotiation. RA allows its members to specify to the RA that they want either:
1. RA to forward all requests to the member’s site.
2. RA to “intercept” content-negotiations for non-HTML representations and direct them appropriately (e.g. return appropriate representation from rdf.crossref.org)

How would it work?

Pros

Allows RA to potentially LD-enable its members very quickly.
Easy for ra-members to opt-in or opt-out
Requires minimal development on part of Crossref
Would even work for DOIs that bypass landing pages

Cons

Only some DOIs conform to expectations for LD identifiers
Requires change to add decision logic implementation on part of IDF.
Requires development of RA resolvers that implement per-member resolution logic (note- this would probably actually be done at DOI level)

Is FRBR the OSI for Web Architecture?

Tony Hammond — Sat, 13 Feb 2010 00:00:00 +0000

(This post is just a repost of a comment to Geoff’s last entry made because it’s already rather long, because it contains one original thought - FRBR as OSI - and because, well, it didn’t really want to wait for moderation.)

Hi Geoff:

First off, there is no question but that Crossref was established to take on the reference linking challenge for scholarly literature. (Hell, it’s there, as you point out, in the organisation name - PILA - as well as in the application name - Crossref.)

But one should also remember that DOI as it was sold at the time was promising so much more. I disagree with you that the participants back then were as wholly innocent of the FRBR terms as you might suggest. Certainly there were ample presentations on DOI that sought to elucidate those relationships.

No matter. FRBR is a useful reference model to clarify some of these concepts. But not one that we are overly concerned with at this time. Nor even whether DOI maps one to one onto a given FRBR layer. What we are more concerned with on a pragmatic level is how DOI maps onto the Web architecture and especially how it plays along with Linked Data concepts.

(Aside: A propos FRBR we might be in danger of repeating the OSI mistake for standardizing the network layer model. Ultimately that was maintained as a reference model but dropped as a concrete model in favour of the TCP/IP stack. Could be that FRBR is our OSI and Linked Data is our TCP/IP stack? That is, we might have to settle on the coarser data model in order to get a coherent story out the door where all can agree.)

You say:

“we need a mechanism to distinguish between when we are getting the thing pointed to by the Crossref DOI (the PDF , HTML, etc.) as opposed to “something about the thing” (e.g. the landing page, metadata record, etc.)”

But that is exactly what we were chasing up in the earlier posts (both my DOI: What Do We Got? and John Erickson’s DOIs, URIs and Cool Resolution). You want to distinguish between a thing and a description about a thing. And Web architecture does just that: it distinguishes between Information Resources (i.e. the things) and Non-Information Resources (i.e. descriptions of the things).

Now is this something that Crossref can truly distinguish and make apparent in its service architecture? If we retain the notion of landing page we are already essentially saying that a Crossref HTTP URI identifies a decsription of the resource, i.e. a Non-Information Resource, or Other Resource, and that is properly indicated within the architecture by returning a “303 See Other status” code.

I think that’s all we’re saying at the moment as a first step.

Web architecture wants to know if the DOI HTTP URI is a thing or description of a thing. I say the latter. You seem to suggest in your comment the latter too. I wonder if we could get a vote on that.

And btw, I am not suggesting that Crossref needs to dive into the business of “tracking compoend documents in their entirety”. Far from it. Lets just get a common resource architecture agreed publicly and then we can build on that.

This observation I received in a private email is something I fully support:

“The real problem is what doi http uri identify on the web. Everything flows from the answer to that Q.”

Tony

Does a Crossref DOI identify a "work?"

Geoffrey Bilder — Thu, 11 Feb 2010 00:00:00 +0000

Tony’s recent thread on making DOIs play nicely in a linked data world has raised an issue I’ve meant to discuss here for some time- a lot of the thread is predicated on the idea that Crossref DOIs are applied at the abstract “work” level. Indeed, that it what it currently says in our guidelines. Unfortunately, this is a case where theory, practice and documentation all diverge.

When the Crossref linking system was developed it was focused primarily on facilitating persistent linking amongst journals and conference proceedings. The system was quickly adapted to handle books and more recently to handle working papers, technical reports, standards and “components”- a catchall term used to refer to everything from individual article images to database records.

In practice the content outside of the core journals and conference proceedings has accounted for relatively low volume. However, we expect that over the next few years this will change and that books and databases will increasingly drive the future growth in Crossref’s citation linking services. Interestingly, these record types all share characteristics that make them substantially different from the journals and conference proceedings that we have hitherto focused on.

Both books and databases introduce new challenges to technology and policies of our citation linking service. The challenges revolved around two areas:

Structure: Both books and databases can have complex structures and the publishers of this content are likely to require granular identification of these content substructures along with a mechanism for documenting the relationship between these substructures (e.g. this section is part of this chapter which is part of this monograph which is part of this series)
Versioning: Unlike typical journals and conference proceedings, books and database records sometimes change over time.

When confronted with the issues of structure and versioning publishers are often tempted to take shortcuts and decide to simply assign DOIs at the highest level structure and to the “work” instead of a particular “manifestation” or version of that work. Indeed, section 5.5 of Crossref’s [DOI Name Information and Guidelines][2] recommends this. But this approach could have a negative impact on the integrity of the scholarly citation record that Crossref is attempting to maintain.

Fundamentally, Crossref DOIs are aimed at providing a persistent online citation infrastructure for scholarly and professional publishers. Consequently, decisions about where to apply Crossref DOIs should be guided by common expectations about the way in which citations work. Citations are typically used to credit ideas or provide evidence. A reader follows a citation in order to obtain more detail or to verify that an author is accurately representing the item cited. A rule of thumb is that a reader has a reasonable expectation that when they follow a citation, they will be taken to what the author saw when creating the citation. Any divergent behavior could result in the reader concluding that the author was misrepresenting the item cited. A further implication of this is that any changes to content that are likely to effect the crediting or interpretation of the content should result in that changed content getting a new Crossref DOI.

Typically, this means that Crossref DOIs should be probably assigned at the expression level and different expressions should be assigned different Crossref DOIs. This is because assigning a Crossref DOI at the higher “work” level is generally not granular enough to guarantee that a reader following the citation will see what the author saw when creating the citation. For example, one translation of a work might be substantially different from another translation of the same work. Similarly a draft version of a work might be substantially different from the final published version of the work. In each case, resolving a citation to a different expression of the work than the expression that was originally cited might result in the reader interpreting the content differently than the citing author.

In general, different “equivalent manifestations” of the same work can safely be assigned the same Crossref DOI. So, for instance, the HTML formatted version an article and the PDF formatted version of an article can almost always be assigned the same Crossref DOI. Any differences between the two are unlikely to affect the crediting of, or reader’s interpretation of, the work. But sometimes it is even possible that different manifestations of an expression will differ enough to merit different Crossref DOIs. For instance, a semantically enhanced version of an article might require new crediting (e.g. the parties responsible for adding the semantic information) and the resulting semantic enhancement may conceivably alter the reader’s interpretation of the article.

Unfortunately, there is no hard and fast rule about where and when to assign new Crossref DOIs. Instead there is only a guideline, namely:

“Assign new Crossref DOIs to content in a way that will ensure that a reader following the citation will see something as close to what the original author cited as is possible.”

The implications of this to publishers are important, especially when they are assigning DOIs to protean records types. For instance, it may mean that:

Book publishers should be expected to keep old editions of books available for link resolution purposes.
Publishers of content that can change rapidly (e.g. by the second) should provide facilities for creating frozen, archived snapshots of content for citation purposes.
All publishers of protean content should issue guidelines instructing researchers on when it is appropriate to cite a work, manifestation or version.

Crossref needs to actively consider these issues as publishers start assigning Crossref DOIs to more dynamic types of content. Minimally, we should be able to provide publishers with recommendations on how to make dynamic content citable. We may even want to consider enshrining certain types of behavior in our terms and conditions so as to ensure the future integrity of the scholarly citation record.

In short, we need to update our guidelines.

[2]: Crossref DOI display guidelines

The Response Page

Tony Hammond — Wed, 10 Feb 2010 00:00:00 +0000

(Update - 2010.02.10: I just saw that I posted here on this same topic over a year ago. Oh well, I guess this is a perennial.)

I am opening a new entry to pick up one point that John Erickson made in his last comment to the previous entry:

“I am suggesting that one “baby step” might be to introduce (e.g.) RDFa coding standards for embedding the doi:D syntax.”

Yea!

It might be worth consulting the latest Crossref DOI Name Information and Guidelines to see what that has to say about this. Section 6.3 - The response page has these two specific requirements for publishers:

When metadata and DOIs are deposited with Crossref, the publisher must have active response pages in place so that they can resolve incoming links.
- A minimal response page must contain a full bibliographic citation displayed to the user. A response page without bibliographic information should never be presented to a user.

What is truly shocking about these requirements is that this are purely user focussed. There is no mention whatsoever of machines. One might have thought that with the Linked Data gospel in full swing there would at least be a nod to machine-readable metadata. But there’s none. I’m not saying that there should be any requirement, or even any recommendation. But a mention might have been useful to chivvy us all along.

I agree with John that publishers could be encouraged (or even just reminded) that machine-readable metadata could be made available through various mechanisms: HTML META tags (such as we currently provide at Nature - and as blogged here earlier), COinS objects, RDF/XML comments, or best of all RDFa markup as John mentions.

The Web is getting semantic. It’s about time that Crossref members joined the wave. And would be helpful if Crossref were there to help us with some new guidelines too!

DOI: What Do We Got?

Tony Hammond — Tue, 09 Feb 2010 00:00:00 +0000

(Click image for full size graphic.)

Following the JISC seminar last week on persistent identifiers (#jiscpid on Twitter) there was some discussion about DOI and its role within a Linked Data context. John Erickson has responded with a very thoughtful post DOIs, URIs and Cool Resolution, which ably summarizes how the current problem with DOI in that the way the DOI is is implemented by the handle HTTP proxy may not have kept pace with actual HTTP developments. (For example, John notes that the proxy is not capable of dealing with ‘Accept’ headers.) He has proposed a solution, and the post has attracted several comments.

I just wanted to offer here the above diagram in an attempt to corral some of the various facets relating to DOI that I am aware of. I realize that this may seem like an open invitation to flame on - and this is a very preliminary draft - but … be kind!

So, this may be totally off the wall but it represents my best understanding of DOI as used by Crossref.

I have distinguished three main contexts:

Generic Data - A generalized information context where the an object is identified with a DOI, an identifier system that is currently being ratified through the ISO process. This is the raw DOI number. (This definitely is not a first class object on the Web as it has no URI.)
- Web Data - An online information context (here I use the term ‘Web’ in its widest sense) where resources are identified by URI (not necessarily an HTTP URI). Here DOI is represented under two URI schemes: ‘doi:’ (unregistered but preferred by Crossref), and ‘info:’ (registered and available for general URI use). Also it has a presence on the Web via an HTTP proxy (dx.doi.org) URL where it is used as a slug to create a permalink (as listed at ‘A’). A simple HTTP redirect is used (with status code 302) to turn this permalink into the publisher response page http://example/1. (Note that typically a second redirect will occur on the publisher platform, here shown by the redirect to http://example/2.)
  - Linked Data - An online information context where resources are identified by HTTP URI and conform to Linked Data principles. Now this is where there is a tension arises between the common publisher perspective and the strict semantic viewpoint. Implicit in the general Web context given above was the notion that the permalink (‘A’) was somehow related to the abstract object and the redirection service applied to it associated the abstract resource with concrete representations of the object.

So how do we relate the DOI HTTP URI with the abstract (‘work’) identifier listed at ‘D’ in the diagram?

Well the Architecture of the World Wide Web recognizes two distinct classes of resources: Information Resources (IR) and Non-Information Resources (NR). (Note: Only the term ‘information resource’ is used in AWWW.) IR are those that can be directly retrieved using HTTP, whereas NR are not directly retrievable but have an associated description which is retrievable and is itself a proxy for the real world object.

So either the HTTP URI denotes an IR (as listed at ‘B’) and is resolved (through HTTP status code ‘302 Found’) to a default representation, which is the view that the Linked Data community would currently have of DOI. But this is at odds with what the Crossref position which regards DOI as identifying the abstract work. Alternately to fit better the Crossref model of DOI the HTTP URI would denote an NR (as listed at ‘A’) which would be resolved (through HTTP status code ‘303 See Other’) to an associated description - a publisher response page.

There will be those self-appointed URI czars who will bemoan the fact of there being multiple URIs. But frankly there is nothing inherently wrong with that. Just as in the real world there are many languages so in the online world there are multiple contexts and histories. We can attempt to make some sense of this by making use of the well-known semantic properties owl:sameAs and ore:similarTo and declare (as also shown in the diagram) the following assertions:

``

(Click image for full size graphic.)

So, this may be totally off the wall but it represents my best understanding of DOI as used by Crossref.

I have distinguished three main contexts:

Generic Data - A generalized information context where the an object is identified with a DOI, an identifier system that is currently being ratified through the ISO process. This is the raw DOI number. (This definitely is not a first class object on the Web as it has no URI.)
- Web Data - An online information context (here I use the term ‘Web’ in its widest sense) where resources are identified by URI (not necessarily an HTTP URI). Here DOI is represented under two URI schemes: ‘doi:’ (unregistered but preferred by Crossref), and ‘info:’ (registered and available for general URI use). Also it has a presence on the Web via an HTTP proxy (dx.doi.org) URL where it is used as a slug to create a permalink (as listed at ‘A’). A simple HTTP redirect is used (with status code 302) to turn this permalink into the publisher response page http://example/1. (Note that typically a second redirect will occur on the publisher platform, here shown by the redirect to http://example/2.)
  - Linked Data - An online information context where resources are identified by HTTP URI and conform to Linked Data principles. Now this is where there is a tension arises between the common publisher perspective and the strict semantic viewpoint. Implicit in the general Web context given above was the notion that the permalink (‘A’) was somehow related to the abstract object and the redirection service applied to it associated the abstract resource with concrete representations of the object.

So how do we relate the DOI HTTP URI with the abstract (‘work’) identifier listed at ‘D’ in the diagram?

``

Note that ore:similarTo (stemming from the OAI-ORE work) is a weaker kind of relationship than owl:sameAs (which comes from OWL) and may be appropriate in this usage.

In sum, scenario ‘A’ is what we have currently implemented, scenario ‘B’ is what might be commonly perceived as being implemented, and scenario ‘C’ may be a more correct semantic position.

Your comments (and not unkind comments, please;) are more than welcome.

Linked Data on Crossref

On metadata enrichment

Metadata available through Crossref

Metadata enrichment layers

Layer 1: Member updates

Layer 2: Community feedback loop

Layer 3: Metadata matching

Layer 4: Third-party datasets

Layer 5: Unstructured content scraping

Summary

Joint statement on research data

Redirecting redirection

TL;DR

Background

Best practices for HTTP redirection by persistent identifier resolvers: 302 vs. 303

DOI-like strings and fake DOIs

TL;DR

Details

Using DOI-like strings as internal identifiers

Using DOI-like strings in access control compliance applications

Spider traps

Proxy bait

Pedantry and terminology

Image Credits

Clinical trial data and articles linked for the first time

Crossref & the Art of Cartography: an Open Map for Scholarly Communications

Metadata enable connections

From graph to cartography

An open map for scholarly communication

The art of cartography

DOI Event Tracker (DET): Pilot progresses and is poised for launch

The Crossref DOI Event Tracker (DET)

DET Service-Level Agreement

♫ Researchers just wanna have funds ♫

Summary

Background

Brief Examples

More examples and documentation.

The Future of the Crossref REST API.

Turning DOIs into formatted citations

DataCite supporting content negotiation

Content Negotiation for Crossref DOIs

DOIs and Linked Data: Some Concrete Proposals

A: Simplest Scenario

What would need to be done?

How it would work

Pros

Cons

B: Simple + IDF Global Semantic Compliance

What would need to be done?

How would it work?

Pros

Cons

C: Simple + IDF Global Semantic Compliance + RA CN Intercept

What would need to be done?

How would it work?

Pros

Cons

D: Simple + IDF Selective Semantic Compliance + RA CN Intercept

What would need to be done?

How would it work?

Pros

Cons

Is FRBR the OSI for Web Architecture?

Does a Crossref DOI identify a "work?"

The Response Page

DOI: What Do We Got?