TL;DR
CratesBump is the first large-scale, public dataset designed to study and predict version bumps in the Rust ecosystem. By analyzing over 2 million release pairs from 220,000 crates, the authors provide a rich feature set for machine learning and reveal that nearly 15% of "minor" Rust releases actually contain breaking API changes.
What Problem Does This Paper Address?
In modern software development, we rely heavily on Semantic Versioning (SemVer) to decide if a dependency update is "safe." A major version bump implies breaking changes, while minor and patch bumps should be backward compatible. However, maintainers are human; they make mistakes, or they might not realize a change is breaking.
While researchers have built datasets to study this in Java (Maven) and JavaScript (npm), the Rust ecosystem—with its unique "Cargo" versioning rules—lacked a comprehensive dataset. Specifically, Cargo treats pre-1.0 versions (e.g., 0.4.5) differently than the standard SemVer spec, considering a change in the leftmost non-zero digit (e.g., 0.4.5 to 0.5.0) as a major, breaking bump. This paper fills that gap, providing the data needed to build "semantic version calculators" that can automatically flag potentially mislabeled releases.
Methods
The authors developed an open-source pipeline called SEMVER-EXTRACT to build the dataset. The process involves several sophisticated steps:
- Data Collection: It pulls metadata from the official
crates.iodump and downloads source code tarballs for every release. - Feature Extraction: For every consecutive pair of releases, the tool computes 41 features. This includes AST diffs (using the
syncrate to count changes in functions, structs, and blocks), dependency changes inCargo.toml, and code complexity metrics. - Social Signal Integration: It clones the crate’s repository and uses the GitHub API to pull commit history, issue activity, and PR data between the two release timestamps.
- Labeling: Each pair is given two labels: a "literal" SemVer reading and a "Cargo" label that follows Rust's specific breaking-change rules for
0.y.zversions.

Datasets
The primary contribution is CratesBump, which contains:
- 2,079,790 release pairs from 219,955 crates.
- 41 feature columns categorized into code changes, dependencies, complexity, timing, and GitHub activity.
- A snapshot of the ecosystem as of September 2026.
The authors also provide the semver-extract tool, allowing researchers to regenerate or update the dataset as the ecosystem grows.
Key Insights and Ideas
- The Pre-1.0 Dominance: A staggering 84% of crates in the dataset have never reached version 1.0. Because Cargo treats
0.y.zbumps as breaking changes, the "literal" versioning and "Cargo" versioning disagree on 82.3% of all pairs. - AST Diffs are King: When training models to predict version bumps, features derived from Abstract Syntax Tree (AST) diffs provided the strongest signal. GitHub activity (like the number of open issues) was the weakest predictor.
- Cargo’s Shielding Effect: By default, Cargo uses "caret" requirements that prevent users from automatically pulling in
0.y.zbreaking updates. The authors found that even though many breaking releases exist in the0.xrange, Cargo's rules successfully shield 98.5% of dependents from accidentally adopting them. - The "Liar" Releases: An audit using
cargo-semver-checksfound that many releases are mislabeled. Specifically, 14.9% of minor releases and 8.7% of patch releases actually broke the public API.
Key Results
The authors benchmarked several machine learning models to see if they could predict the "Cargo" bump label based on the 41 features.
- Predictability: Tree-based ensembles (like Random Forest and XGBoost) performed best, achieving a Macro-F1 score of ~0.46. While this is significantly better than the random baseline (0.24), it suggests that predicting version bumps is a non-trivial task that is far from "solved."
- Hardest Class: Patch releases were the most difficult to identify correctly (F1 score of ~0.17), likely because the line between a "bug fix" and a "minor feature" is often blurry in the code itself.
Limitations and Future Work
- Declared vs. Actual: The dataset labels are based on what maintainers declared, not necessarily what they did. While the authors audited a sample, the full dataset doesn't yet have a "ground truth" label for every single API break.
- GitHub Dependency: About 16% of pairs lack GitHub features because the crates are hosted elsewhere (like GitLab) or the repositories were deleted/private.
- Future Directions: The authors suggest that future versions of the dataset could include labels from automated tools like
cargo-semver-checksfor every buildable pair, as well as signals from test suites and changelogs to capture behavioral changes that AST diffs might miss.
Summary updated