Programs on Crossref

The programs approach: our experiences during the first quarter of 2025

Helena Cousijn — Tue, 08 Apr 2025 00:00:00 +0000

At the end of last year, we were excited to announce our renewed commitment to community and the launch of three cross-functional programs to guide and accelerate our work. We introduced this new approach to work towards better cross-team alignment, shared responsibility, improved communication and learning, and make more progress on the things members need.

In line with the Crossref strategic agenda, the three programs focus on:

Co-creation and Community Trends (CCT): This program is responsible for interfaces such as reports/dashboards, record registration interfaces, connections and collaborations such as Open Funder Registry, ROR, ORCID auto-update, as well as OJS and other partner integrations. This program also includes the Crossref website and any front-end interfaces to support other programs. It includes initiatives aimed at upholding the integrity of the scholarly record and our tools in this area, such as Crossmark and retraction/correction tooling, and Similarity Check for text comparisons.
Contributing to the Research Nexus (CRN): This program manages and oversees all activities relating to contributing to the Research Nexus. A lot of the work in this program revolves around our REST API, but also includes our other APIs, incorporating external data sources like Retraction Watch and Event Data, building out metadata matching services with the new data science team, supporting the community of metadata users with API sprints and more modern options for retrieving metadata based on usage and need.
Open and Sustainable Operations (OSO): This program manages and oversees all activities related to making our operations more open, transparent, and sustainable. This program focuses on supporting and strengthening the core functions our members rely on and enabling future growth. It includes metadata deposit and processing, most apps for e.g. managing titles, authentication, and architectural and infrastructural projects like moving from the data centre to the AWS cloud service. This program also includes modernising our operations in general, which is not just technology but also finance and human resources, so projects like membership process automation, financial analyses, and business system integrations.

The approach we are taking is to support the work within the programs through (internal) cross-functional steering groups. Led by three program leads (who share updates on their programs below), three program steering groups meet regularly to discuss the topics and work that fall within the scope of each program. The steering groups consist of representatives from all teams within Crossref, which means every steering group has people from the community team, membership team, technical team, data science team, and operations and finance team, bringing all the perspectives and expertise needed to prioritise the next steps for Crossref and fostering broad knowledge sharing and shared responsibility.

Although the whole organisation contributes to these programs, they are coordinated by the Programs and Services team. The team was formed towards the end of 2024, and on the 1st of February, Helena Cousijn joined Crossref in the new role of Director of Programs and Services. Helena has a background in both product management and community engagement and is very excited to help Crossref shape the programs approach and work with all teams across the organisation to drive the strategic agenda forward!

If you’d like to keep an eye on the work that is happening within each program, you can find more information on the Crossref productboard.

Co-creation and Community Trends (CCT)

The mission of the CCT program is to build and foster relationships with our community and other services and organisations within it, so that Crossref can meet and anticipate community needs. Curiosity and listening are at the core of how we co-create to tackle emerging challenges, develop best practices, and explore new ideas for building the Research Nexus. We want our work to benefit all of Crossref’s diverse stakeholders - from our own colleagues and members to underrepresented communities in the wider scholarly ecosystem.

In the first quarter of 2025, our focus areas have been:

Improvements to our new record registration form for journal articles, which already supports grants, and was launched in beta for articles in 2024. For example, the form now has a built-in reference deposit feature. Join the conversation on the community forum for updates and feedback on this new helper tool.
Running a series of multilingual metadata health check webinars. There are more of these coming up throughout Q2, so it’s not too late to sign up for one if you are interested.
Integrating with Rogue Scholar to automate the assignment of DOIs to, and the archiving of, posts on this very blog.
Planning for the inaugural Crossref Metadata Awards - join our upcoming community call on 7 May to find out what this is all about.

In the coming months, we are hoping to tackle the following:

Kick off a project to review the information architecture of this website and look into how we can make our documentation and related information more helpful and easier to navigate.
Expand the record registration form for journal articles to allow easy editing of previously submitted records. This will allow us to sunset the long-deprecated Metadata Manager tool, as was first announced in 2021.
Begin building new record registration forms for more work types. Watch this space.
Explore options for supporting the integration of additional software systems with Crossref, building on our existing approach with OJS plugins, with a focus on open-source tools relied upon by our members for registering metadata.
Restore faceted search on Crossref Metadata Search. This feature was disabled in 2022 following intermittent performance issues. We believe recent improvements to Metadata Search will allow us to bring some filters back, although we will need to start small so as not to overload our systems with these more complex queries.

Contributing to the Research Nexus (CRN)

The research nexus is a rich and reusable open network that represents scholarly activity. It consists of connections between research organisations, people, things, and actions; it’s an evolving model of the scholarly record that the global community can build on forever for the benefit of society.

Our metadata is already a contribution to the research nexus, however, there is much more we would like to do. Our next steps will be to consolidate our existing data and services, and build the technical capacity, partnerships, and knowledge to enhance our contribution with new relationships. Some parts of our data storage and workflows don’t yet have the flexibility to fully capture all types of research objects and how they are connected.

To support this process, the main priorities in the program are:

Collaborate with our community. We want to get to know users of our metadata better and work more collaboratively alongside them. Also, we seek partners to contribute new data sources that will enhance our metadata with additional relationships.
Share the research nexus vision. We know that we aren’t alone in developing the research nexus, so we will reach out to others with a similar vision and identify where we have common goals.
Maintain our technology. We have already identified technical improvements we can make to our REST API, and we need to keep on top of monitoring and fixing bugs. We also need to build capacity for new types of data and relationships. Our other endpoints, such as the XML API and forwardlinks (for citations), need maintenance and are likely to be affected by a planned redesign of our core architecture.
Building a new matching service. Identifying relationships between metadata records is a key part of the research nexus. We have already improved reference matching over the years, and we’re looking to implement funding, affiliations, and version matching next. We’ve carried out research on several types of matching and are looking at building a new service to handle it in production.

In the first quarter of 2025, we’ve been working on:

Schema changes, making the first significant updates to our schema for several years, including adding the capacity for depositing ROR IDs for funding organisations in funding metadata.
Delivering Retraction Watch retractions via the REST API, integrated with member-supplied retraction/correction data.
Getting the community involved and understanding needs, planning a sprint and various workshops.
Plenty of under-the-hood updates to the REST API, and more significant upgrades to come later this year.

Next up, we will:

Plan and build out the new matching service.
Improve representation of some metadata in the REST API, including Crossref members, journals, and typed citations such as data citations.
Update the grants schema to extend the award types and respond to new funder member requests
Add contributor roles to the schema, including CRediT.
Ask our community about metadata retrieval, including the various APIs and the Metadata Plus subscription service.
Upgrade elements of the REST API and optimise the underlying technical infrastructure.

Open and Sustainable Operations (OSO)

The OSO program is centered on transparency and sustainability of our technical systems and our business and people operations. We focus on maintaining critical systems and operations and ensuring their security, addressing technical and operational debt, and controlling or reducing costs - to Crossref, our community, or the environment. We’re always keen to tackle projects to automate repetitive and manual tasks – of which we have many – and pay down technical debt, being as open and transparent as possible along the way.

Our most recently completed work includes:

Moving from Oracle to an open-source database, PostgreSQL. This work aligns with the POSI principles and sets us up for a more robust, reliable, and modern infrastructure.
Implementing metadata schema changes for deposit submission and processing, so we can now accept ROR IDs in funding metadata, as well as the changes in latest schema version (5.4.0) which includes the new ability to label references with a type (such as dataset, software, blog post, article, etc.).
Automating parts of the process to keep Sponsor information on our website up to date and make it easier to search, so our community can find relevant and accurate information about our Sponsors and how to work with them, and our membership team spends less time keeping the website current.

Ongoing work in our program includes:

Moving from a physical data center into the cloud (AWS). The PostgreSQL migration was the first step needed to enable our move to the cloud, which will allow us to operate more sustainably and efficiently.
Automating new member setup in our systems, which is largely a manual process now.

And coming up are:

Making changes in our core system to accept the upcoming 5.5 metadata schema version.
Extracting billing code from our main codebase, to set up as its own service. This will allow us to simplify our code and make it easier to maintain. We’ll also be implementing the changes to billing enacted as part of the Resourcing Crossref for Future Sustainability program (TBD!).
Holding a “systems workshop” in April, to understand how our current system(s) are and aren’t meeting staff, member, and community needs, and how we might go about building the open, sustainable Crossref system of the future.

What have we learned so far?

Internal communication

One of the reasons to implement a programs approach was to improve internal communication across the organisation. With all teams being represented on all steering groups, everyone is in the loop when decisions are taken. We see that this way, people feel more connected to the strategic agenda and, importantly, the ‘why’ is clearer to people. It is easier to get perspectives from across the organisation because contributing to these conversations is now part of people’s day jobs and so it’s easier to ask for their time. We are still looking to improve how we facilitate group discussions and decision-making to ensure we make the most of the program steering groups.

Planning and delivery

Working closely with people from across the organisation has helped with more effective planning. A closer collaboration between program leads and developers makes the delivery of new features and functionality more accurate and predictable. With the community and support teams also being part of the conversation, they can plan related comms and support/documentation efforts in a timely manner. So far, it has also been easier to get more things delivered. We have some big projects coming up this year that will be a good test for the programs approach!

Cross-cutting topics

The implementation of a cross-functional approach facilitates discussions around cross-cutting topics, but also leads to the question of how cross-cutting topics fit within a specific program! Maybe you already noticed that work on metadata schema 5.4 and the planned work on 5.5 is included under both Contributing to the Research Nexus and Open and Sustainable Operations in the update above. Because metadata development impacts many of our systems, work was needed within all programs to enable these changes - the input, the output and the interfaces. Later this year, we’re planning to share some visuals that better explain which projects sit with which program and how we deal with cross-cutting topics.

Alignment

One of the most important things for the approach to be successful is that people are bought in and willing to participate and communicate. For cross-organisational alignment, a culture needs to be in place (or developed) where people are willing to collaborate and be open and transparent about their work. In a practical sense, we are still looking at how we can better align our code bases with the current programs so that it is easier to develop the relevant expertise within the programs.

We hope to see many of you at our upcoming community call on 7 May. Please register to join as we discuss some of the work included in this update.

A progress update and a renewed commitment to community

Ginny Hendricks — Thu, 12 Dec 2024 00:00:00 +0000

Looking back over 2024, we wanted to reflect on where we are in meeting our goals, and report on the progress and plans that affect you - our community of 21,000 organisational members as well as the vast number of research initiatives and scientific bodies that rely on Crossref metadata.

In this post, we will give an update on our roadmap, including what is completed, underway, and up next, and a bit about what’s paused and why. We’ll describe how we have been making resourcing and prioritisation decisions, including a revised management structure, and introduce new cross-functional program groups to collectively take the work forward more effectively.

It’s important to acknowledge that Crossref has evolved significantly from just five years ago - our member count has more than doubled from 10,000 to 21,000 organisations since 2019 and they include all kinds of organisations such as funders, universities, government bodies, NGOs, and of course scholar- and library-led publishers. The smaller organisations now collectively contribute the majority of Crossref funding. We’ve gone from 100 million records to 160 million in five years, and our metadata is retrieved more than 2 billion times monthly, quadrupling what it was five years ago.

It’s within this context that we’ve spent quite a lot of time thinking about scalability, how we collect and process feedback and contributions from many organisations, how to automate our operations, and refining the plans for the next few years.

Our strategic agenda remains the same

A few times a year we update the strategy page where there is a quadrant of projects showing what’s completed, in progress, up next, and in planning/ideas - for each strategic theme. We also link from there to our live public roadmap which shows more specifics about individual projects, including projected timelines, and is updated more frequently.

If you’ve been watching the strategy page, checking in on the public roadmap or this blog, or joining webinars and annual meetings, you’ll know that we’ve had some longstanding plans to—among other things—reduce technical debt, rebuild our metadata management system, move to the cloud, modernise our schema, support multiple languages, and partner with multiple data sources to build the Research Nexus.

You’ve heard us talk about these initiatives a lot, but you’ve not seen particularly swift action.

Moving the work forward more effectively

Earlier this year, it became clear that our almost three-year project to build a new relationships API had not worked out. The project, dubbed ‘manifold’, was to initially deliver data citations, and eventually replace our central metadata system, but what was prototyped didn’t scale, even with a subset of our metadata. We weren’t confident enough about the project’s timeline or costs to justifiably continue investing further time and resources.

Meanwhile, we’d barely scratched the surface of our aim to pay down technical and operational debt, and we’d also been neglecting to keep the live system up to date with the numerous metadata changes that have been queued up, waiting to be implemented.

We knew the manifold project was ambitious – our system has grown in complexity over the years. We were trying to rebuild the car while driving it (our system needed to continue to operate and be maintained by our team) while trying to design a new approach to manage the many relationships between 160+ million database records. In the years we worked on this project, we learned a lot that will inform future plans for a large system redesign.

In March this year, we decided to pause the manifold project. We apologised to our community partners for not delivering the promised data<->literature matches they hoped to use. They were frustrated but thankfully understanding.

We then resolved to focus on backend infrastructural changes, conduct cross-training so that all of our staff would become familiar with current in-use systems instead of greenfield tech (for now), and start to make a dent in the backlog of bugs and long-promised schema updates in our mainstream services.

We’re happy to report some movement on these things and some milestones that have been achieved in these areas in recent months.

Fostering a happy and dedicated team

Any kind of work can only happen when our staff are in a good place, feeling supported and comfortable to question things, and well-equipped with information, purpose, and clear priorities. In June, when the whole staff met up in person, we had some really good conversations about culture, communication, and about sharing responsibilities. Some people ran birds-of-a-feather sessions to explore the issues that had been keeping them up at night, such as authentication/security, and rebuilding the Crossref System (CS), and the team also co-created a set of prioritisation drivers that are now in use within our roadmap and planning processes.

Taking on feedback from the all-staff meeting and then the July board meeting, we thought strategically about the organisational structure Crossref would need over the next few years to reflect the growth in scope and size, and fulfil its longer term goals. We have long had an ambitious agenda but realised we didn’t yet have the capacity to do it all. So we came to the conclusion that we needed an updated team and management structure to take us through the next phase of our development.

The structural changes were concluded at the end of November. They included:

Moving Technology under Operations, since Technology—though a vital enabler—still works in service to our mission and in support of our community, just like other operational things like board governance and finance.
Reframing product development as Programs and Services, and reducing our workstreams from five product portfolios to three programs. We formed cross-team steering groups around clearly articulated program areas (more on those below).
Broadening the leadership to include an Executive team and an extended Director team, and forming a Senior Management Team (SMT). These changes ensure that the collective responsibility for Crossref now rests on a wider group of experts who can back each other up and share the risk and the knowledge, rather than on just a few individuals.
We started recruiting for directors for two new leadership positions. We’ll welcome a new Director of Programs and Services and a new Director of Technology in the new year.
Evolving the strategic initiatives team into a data science team, integrating research & development functions throughout all teams and with the SMT taking collective responsibility for strategic initiatives.

Unfortunately, with the shift in approach for product development and by sharing responsibility for strategic initiatives and research among the wider team, we made the difficult decision that four positions would no longer work within the new structure.

A new approach: joined-up initiatives and cross-functional programs

Research has always been an important role for Crossref, but as this function had been annexed from our regular work, it became hard to coordinate strategic initiatives across the wider organisation. In recent years we inadvertently created more technical debt for ourselves, i.e., built multiple prototype tools without plans for adoption or moving them into production. Strategic initiatives, by their nature, need thorough research and high-level alignment, so we made such initiatives—things like Resourcing Crossref for Future Sustainability (RCFS) and improving the Integrity of the Scholarly record (ISR)—the responsibility of the whole senior management team.

Some useful research had been conducted, but we were never in a position to act on any of it. Particularly promising work has been in the field of metadata matching, and with the growth in the community reliance on our metadata, and attention on data quality rightly increasing, we decided to create a new data science team to be dedicated to this work, led by Dominika Tkaczyk.

We had also struggled with a traditional product management approach since all our tools and activities are interconnected, and we found we were trying to do too many things at once but not all of them very effectively. We also acknowledged that product management comes from the commercial e.g. retail world and therefore is designed to help companies sell/upsell, which is not our goal. So we looked to other approaches more suitable to mission-based nonprofits.

Introducing three programs

We have introduced cross-functional program management in order to work towards the following:

better cross-team alignment
shared responsibility
improve communication and learning
make more progress on the things members need.

Supporting the strategic theme of co-creation, a new program, facilitated by Program Lead Lena Stoll, now manages and oversees all activities around co-creation and community trends. A cross-team steering group just began meeting regularly and will be responsible for interfaces such as reports/dashboards, record registration interfaces, connections and collaborations such as Open Funder Registry, ROR, ORCID auto-update, as well as OJS and other partner integrations. This program also includes the Crossref website and any front-end things to support other programs. And it includes ISR (the integrity of the scholarly record) and our tools in this area such as Crossmark and retraction/correction tooling, and Similarity Check for text comparisons.

Supporting the strategic theme of complete and global metadata and relationships, a new program, facilitated by Program Lead Martyn Rittman, now manages and oversees all activities relating to contributing to the Research Nexus. Working particularly closely with the metadata team, led by Patricia Feeney, this program addresses how metadata is modelled, used, enriched, and extended. Work includes our APIs, incorporating external data sources like Retraction Watch and Event Data, building out metadata matching services with the new data science team, supporting the community of metadata users with API sprints and more modern options for retrieving metadata based on usage and need.

Supporting the strategic theme of open and sustainable operations and keeping to the POSI framework, a new program, facilitated by Program Lead Sara Bowman, now manages and oversees all activities relating to making our operations more open, transparent, and sustainable. This program focuses on supporting and strengthening the core functions our members rely on and enabling future growth. It includes metadata deposit and processing, most apps for e.g. managing titles, authentication, and architectural and infrastructural projects like moving from the data centre to the AWS cloud service. This program also includes modernising our operations in general, which is not just technology but also finance and human resources, so projects like membership process automation, fee modelling and financial analyses, and business system integrations.

The Programs will start to be reflected across our website and in our communications from next year.

What are Crossref’s new prioritisation drivers?

These are the drivers that our ~40 staff co-created in June that are guiding decisions about the priorities on our roadmap. New ideas will be evaluated in the following areas:

Encourage participation from new or under-represented communities
Respond to and lead trends in scholarly communications
Benefit the greatest number of members and users
Reflect on how the community works with each other and allow members to self-serve
Expand to support and connect relevant resource types and metadata fields
Make it easier to create and update metadata
Enhance metadata for completeness and accuracy
Make it easier to retrieve and use metadata
Automate repetitive/manual tasks
Address technical and operational debt
Maintain critical systems and operations and ensure their security
Control or reduce costs - to Crossref, our community, or the environment

We’re happy to report that the changes made this year have resulted in a productive last few months of the year. As reported in our annual meeting, here is the progress update.

What’s paused

A relationships API endpoint and, therefore, a specific data citation feed
Manifold, the three-year effort to modernise our tech stack
Most of the strategic initiatives prototypes that can’t yet be scaled, such as Labs API and Labs reports

What’s recently completed

We succeeded in moving the entire Crossref corpus to an open-source database, PostgreSQL
Fixed numerous REST API data quality issues and lots of troublesome bugs
Schema development - support for ROR as a Funder identifier is live and currently in testing
We automated some very manual membership and billing processes, saving hundreds of staff hours a year
Released a new form for journal article record registration, building on the grant registration form
Upgraded Participation Reports to include Affiliations and ROR IDs
Launched a new API Learning Hub

Since the rest of the community stops for no Crossref product roadmap issue, we also progressed a number of community and governance initiatives:

The Grant Linking System (GLS) reached 5 years with over 40 funders joining Crossref and registering over 130,000 grants and awards, including use of facilities and projects
Our research for Resourcing Crossref for Future Sustainability (RCFS) with the Membership & Fees Committee is going well, and we’ll have new fee proposals for review in 2025
The integrity of the Scholarly Record (ISR) conversations have deepened, and we’ve formed strong relationships with editorial experts and research integrity sleuths, who are getting up to speed on our metadata, and we’re working with some sleuthing consultants to change our processes to handle deceptive member behaviour such as paper mills, cloned journals, and citation manipulation. The new data science team plays a role here, along with membership and governance.

What’s currently in focus

In our efforts to do less but do it more effectively, we have two current priorities:

Get out of the physical data centre and into the cloud.
Develop Schema 5.4.

These two projects are underway, involving lots of communication and learning. Since we haven’t released any schema updates in many years, all our staff are learning for the first time how a metadata schema model is interpreted in a systemic way, learning about the structure of research objects, and honing the process as they go. We’ve high hopes we’ll be in a position to release continuous metadata schema versions and catch up on the backlog over the coming years.

What’s next

Continuous metadata development, with contributor roles up next
Retraction Watch data integrated into the REST API so users have a single source of retraction/correction data
Upgraded preprint matching and notifications
Modelling more equitable fees through the RCFS projects
Piloting a non-voting membership category

Once we’re fully in the cloud and in the groove of metadata updates, and with the support of newly-hired technology and program directors joining in the new year, we’ll turn our attention to rebuilding the central metadata system that we call the Crossref System, or “CS” and report more on this next year.

So that was our summary of 2024 and an indication of what’s coming in 2025 and beyond; sorry it’s so long, and thanks for reading this far! Next year we’ll get back to more regular updates as the strategic agenda and the programs progress.