Attributing and Referencing (Research) Software: Best Practices and Outlook from Inria

Pierre Alliez, Roberto Di Cosmo, Benjamin Guedj, Alain Girault, Mohand-Said Hacid, Arnaud Legrand, Nicolas P. Rougier

Abstract

Software is a fundamental pillar of modern scientific research across all fields and disciplines. However, adequate means to cite and reference software remain elusive due to complexities in authorship, varying development roles, and the dynamic nature of code. This paper contributes to the development of proper guidelines for software citation, drawing on the extensive experience of Inria, the French research institute for digital sciences. We propose a richer taxonomy for software contributions, advocate for human-centered evaluation, and emphasize the critical distinction between citation and reference.

Introduction

In the modern scientific landscape, software is as fundamental as the papers that describe it. However, the academic "credit economy" has been slow to adapt. Researchers often struggle to receive proper recognition for software development because current systems treat code like static papers. This paper addresses the gap between how we use software and how we attribute and archive it, proposing a framework that prioritizes both academic credit and technical reproducibility.

The Complexity of Software Attribution

Attributing credit for software is significantly more complex than for traditional publications. Authorship can range from a single developer to a large, evolving team over decades. Furthermore, software development involves diverse roles—architects, coders, debuggers, testers, and managers—all of whom contribute to the intellectual weight of the project.

Automated metrics, such as line counts or commit frequency, often fail to capture the true value of these contributions. A ten-line bug fix may represent more intellectual effort than a thousand lines of boilerplate code. Consequently, the authors argue that only a "human-in-the-loop" moderation process can accurately assess the quality and impact of research software, moving away from algorithmic evaluation toward expert review.

Infrastructure for Archival and Preservation

To address these challenges, Inria has developed a robust internal infrastructure. Central to this is the BIL Database (Base d’Information des Logiciels), which tracks hundreds of projects, their maturity, and contributor roles.

A key innovation is the integration between HAL (the French national open access portal) and Software Heritage. This workflow allows researchers to deposit code into HAL, which then triggers a moderated archival process into Software Heritage, ensuring long-term preservation and accessibility.

The moderated archival workflow connecting HAL and Software Heritage ensures that research software is preserved in a dedicated repository.

Citation vs. Reference: Solving the Reproducibility Gap

A core contribution of this work is the formal distinction between citation and reference:

  • Citation is a human-readable pointer intended to provide academic credit to authors (e.g., "I used the Coq proof assistant").
  • Reference is a machine-actionable pointer to a specific version or even a specific line of code (using intrinsic identifiers like SWHIDs) to ensure an experiment can be perfectly replicated.

This distinction is vital because of "dependency hell." As illustrated by the dependency graph of common libraries like matplotlib, a single package can rely on a massive, hidden web of OS-level dependencies. Referencing the exact environment is a monumental task that requires specific, versioned identifiers rather than general citations.

The complex dependency graph of matplotlib illustrates the challenge of full software environment referencing and the necessity of machine-actionable identifiers.

Conclusion and Recommendations

Inria’s model demonstrates that software self-assessment can be successfully integrated into career promotion and team evaluation cycles. By adopting a qualitative taxonomy for contributions and distinguishing between credit-based citation and technical reference, the research community can better support the software that powers modern science. While challenges remain—including the manual effort required for moderation and the need for global standards like CodeMeta—these practices provide a blueprint for institutional and national research agencies to ensure their code is both recognized and preserved.

Summary updated

Abstract

Software is a fundamental pillar of modern scientiic research, not only in computer science, but actually across all elds and disciplines. However, there is a lack of adequate means to cite and reference software, for many reasons. An obvious rst reason is software authorship, which can range from a single developer to a whole team, and can even vary in time. The panorama is even more complex than that, because many roles can be involved in software development: software architect, coder, debugger, tester, team manager, and so on. Arguably, the researchers who have invented the key algorithms underlying the software can also claim a part of the authorship. And there are many other reasons that make this issue complex. We provide in this paper a contribution to the ongoing eeorts to develop proper guidelines and recommendations for software citation, building upon the internal experience of Inria, the French research institute for digital sciences. As a central contribution, we make three key recommendations. (1) We propose a richer taxonomy for software contributions with a qualitative scale. (2) We claim that it is essential to put the human at the heart of the evaluation. And (3) we propose to distinguish citation from reference.
BibTeX
@inproceedings{alliez2020attributing,
  author = {Pierre Alliez and Roberto Di Cosmo and Benjamin Guedj and Alain Girault and Mohand-Said Hacid and Arnaud Legrand and Nicolas P. Rougier},
  title = {{Attributing and Referencing (Research) Software: Best Practices and Outlook from Inria}},
  booktitle = {Comput. Sci. Eng},
  year = {2020},
  url = {https://openreview.net/forum?id=A_-w9Y6NjN},
}
Figure