Age or Vintage? Ensembling Over What Pipe Failure Data Cannot Tell Apart

Utilities decide which buried pipes to replace using models trained on decades of break records. A long-running dispute sits underneath those models. Is a pipe risky because it is old (wear-out), or because of when it was laid, with that era's metallurgy, standards and installation crews (its "vintage cohort")? In Syracuse, ranking water mains by age alone barely beat random. In Korean district heating, burial year was the dominant predictor. The two readings are hard to separate for a structural reason, not a modeling one. Age equals calendar year minus installation year, so a linear trend can be moved between "age" and "vintage" without changing the fit to the data. Within one utility the error is invisible, because it leaves next year's ranking of pipes unchanged. It becomes visible when a model is carried to another utility whose cohorts differ, or when it is used to forecast how many failures a budget must cover.

We propose an ensemble whose members do not differ by random seed or architecture. Each member is a censoring-aware failure-rate model that fits the same data equally well but splits the shared trend between age and vintage in a different way. The members are weighted using cross-utility validation, because validation inside one utility cannot tell them apart by construction. The claim is that this ensemble transfers to a new utility and forecasts failure counts better than a single tuned gradient-boosted tree, and that its spread reports a kind of uncertainty that ordinary bagging never shows.

Core Claim

Write a pipe's log failure rate as a sum of an age effect, a calendar-year effect and a vintage effect:

$$\log \lambda(A, P, C) = f(A) + g(P) + h(C), \qquad A = P - C,$$

where $A$ is the pipe's age, $P$ the calendar year, $C$ the installation year, and $f, g, h$ arbitrary functions. For any slope $t$, replacing $f(A)$ with $f(A)+tA$, $g(P)$ with $g(P)-tP$, and $h(C)$ with $h(C)+tC$ leaves every predicted rate unchanged. This is the classical age–period–cohort identification problem. At a fixed forecast year the $-tP$ term is the same for every pipe, so within-utility rankings are unaffected. Absolute forecasts and cross-utility transfer are not. If wear-out physics $f$ is shared across utilities with the same pipe material, and vintage effects $h$ are local, then a model that has absorbed vintage trend into its age curve will transfer badly. Sharp age "thresholds" at 20, 30 or 35 years (Lee et al.; Lee et al.; Kim) and the "age myth" result (Kumar et al.) can both be true descriptions of one utility's data under different, equally consistent choices of $t$. The nonlinear curvature of each effect is identified; only the linear split is not.

The new ingredient is to treat $t$, and more generally the identifying constraint, as the ensemble's axis of diversity, and to fit the weights over that axis on held-out utilities rather than held-out rows.

Relation to Prior Work

The infrastructure papers supply the motivation and the data. Kumar et al. built block-level gradient-boosted decision tree (GBDT) rankings for Syracuse water mains from digitized work orders and old field notebooks, showing that vintage, diameter and past breaks dominate age. Weeraddana et al. fused pipe records with soil and weather for an Australian utility using a random forest, then extrapolated 20-year failure curves with a linear model. Their pre-1985 asbestos-cement step change is exactly the kind of effect that could be vintage or age. Two Korean district-heating studies on the same 326,788 records differ in how they manufacture positive labels. Lee et al. relabel the 10% of intact pipes most similar to failures as "high-risk." Lee et al. label every pipe older than 20–30 years as damaged.

These studies constrain our design in two ways. First, both labeling tricks hard-code an answer to the age-versus-vintage question into the targets, so their headline scores cannot adjudicate it. Second, their reported F2 and AUC are computed against those manufactured labels, so they are not comparable to scores on real failures. Every model we build is therefore trained and scored on observed failures with censoring handled explicitly. None of these papers models calendar year, and none tests transfer across utilities, which their own discussions list as an open problem.

The ensemble literature supplies the techniques. Survival forests and implicit survival-distribution models handle right-censored failure times without relabeling: Consistency of survival tree and forest models analyzes splitting bias, Learning Survival Distribution with Implicit Survival Function learns the distribution directly, and a battery-life application uses survival forests on equipment. We reuse them as members and baselines. Stacking, in the sense of meta-models fit on held-out base predictions (Fei et al.), supplies the combiner. The departure is that our held-out unit is a utility, not a fold.

Dynamic, locally weighted fusion such as the expert fusion model EFM+ for wind power (Adaptive expert fusion model) is the closest "adaptive ensemble" idea in the file. It reweights experts by local error among nearest neighbours. That cannot help here, because members that differ only in $t$ have identical local error inside a utility.

Research Program

We first build a dated, censored panel for each utility. Each pipe or block contributes one row per calendar year until it fails, is replaced, or the observation window ends. The attributes are material, diameter, installation year, soil where available, and break history. The Syracuse work orders span 2004–2015 and Western Water distinguishes bursts from fitting failures, so both support a genuine calendar-year axis. The Korean district-heating data is used only if repair dates are recoverable. If it is a single cross-section, age and vintage are exactly collinear there, and we use it only for within-utility comparisons.

The base learner is a discrete-time failure-rate model, a GBDT predicting the yearly probability of failure from attributes plus age, year and vintage terms. It is trained without relabeling, using the class-weighting and recalibration practices catalogued in the imbalance survey (Comprehensive Survey on Imbalanced Data Learning). Members are produced by imposing different identifying constraints on the linear part of the age, year and vintage terms. These include zero linear calendar trend, zero linear vintage trend within a material, an age slope fixed to a physics prior drawn from the bathtub wear-out picture (Ben Nengjun et al.), and a grid of intermediate values of $t$. Each member is implemented by fitting the linear components in a constrained additive layer and letting the trees model the nonlinear residual and interactions. Interactions between age and material, and between vintage and material, are allowed; the analysis checks how far they break the exact invariance above.

The combiner weights members to minimize forecast error on utilities held out from training. With three or four utilities this is leave-one-utility-out. To understand behavior under known ground truth, we also build a semi-synthetic benchmark. It plants a shared bathtub-shaped age curve and utility-specific vintage curves into the real Syracuse and Western Water asset inventories, then simulates failures, so it is known which member is correct.

Baselines

The naive reference points are those reported by Kumar et al. on Syracuse: random ranking (precision at the top 1% of blocks, P@1, of 0.08), ranking by age (0.10), and ranking by past breaks (0.48). We recompute all three on our panels and splits so that they are directly comparable.

The strongest existing method is the tuned GBDT of the same paper, which reports P@1 of 0.62 on Syracuse under temporal cross-validation. We re-run it on identical features, the same forward-in-time splits and the same hyperparameter-search budget as our members. It is the main competitor because trees can already split on installation year and age, and within a utility that may be all that matters.

For district heating, Lee et al. report F2 0.921 and AUC 0.993 for XGBoost with relabeling, and Lee et al. report F2 0.804 and AUC 0.837 for LightGBM with age-threshold labels. Both are measured against manufactured labels. We re-run both pipelines and score them only on real held-out failures, and we do not compare against their published numbers. For Western Water, the reference is "the top 10% of pipes capture over 30% of bursts" from Weeraddana et al.. We report recall at 10% in the same form.

Further baselines are a random survival forest and a seed-bagged GBDT ensemble with the same number of members. Two ablations isolate the key ingredient. The first averages the constraint members without weights. The second weights them by in-utility validation instead of cross-utility validation. If in-utility weighting matches cross-utility weighting, the central argument is wrong.

Beating the baselines has a specific meaning. Within a utility, we expect a tie with the tuned GBDT on P@1 and recall at 10%, and that tie is the predicted outcome. The win has to come on the held-out utility: higher P@1 and recall at 10%, lower error on annual failure counts, and better calibration. Each is reported with bootstrap intervals over pipes.

Why not just…?

Why not just tune a single GBDT with age and installation year as features? Within one utility this is a real threat, and our own algebra predicts it will match us on rankings. The difference appears only on transfer and on count forecasts. A tree's split between age and vintage is set by arbitrary details of tie-breaking and regularization, not by the data. The leave-one-utility-out experiment settles this directly. If the single GBDT transfers as well, the proposal fails.

Why not just bag GBDTs over seeds and use their spread as uncertainty? Seed-to-seed variance measures sampling noise. It says nothing about the unidentified slope $t$, because every seed sees the same collinearity and resolves it in similar ways. We test this by comparing seed spread with spread across constraint members, on forecasts for cohorts about to cross ages that have not yet been observed. On the semi-synthetic benchmark we also check which spread actually covers the true error.

Why not just pick the physically motivated constraint and skip the ensemble? If physics gave a reliable age slope for each material, this would be the right answer. That single member is included as a baseline. Its weakness is that the wear-out priors in the file are qualitative bathtub shapes, not slopes per material. The decisive comparison is the physics-only member against the cross-utility-weighted ensemble on held-out utilities. If the physics member wins, we report that and drop the ensemble.

Why not just route experts by vintage cohort, for example with gated tabular networks? Gated feature-masking networks such as GANDALF (Joseph & Raj) or cohort-specific experts change how vintage is represented. They do not change which linear split the data permits, so they inherit the same ambiguity. We expect a cohort-gated model to behave like the single GBDT on transfer, and we include one only if time allows.

Proposed Experiments

The first check is on the semi-synthetic benchmark, where the true age curve is known. It asks whether in-utility validation is flat across the constraint members while cross-utility validation recovers the planted slope, and whether the weighted ensemble's count forecasts beat the single GBDT's under a shift in cohort mix. If interactions between material and age or vintage in realistic inventories already make $t$ identifiable in practice, members will differ in in-utility likelihood. That would make the problem less serious than claimed, and we would report it.

The second check is on real data. We transfer between Syracuse and Western Water in both directions, using material-matched subsets, and evaluate P@1, recall at 10%, and mean absolute error of annual failure counts over a three-year forward window. Within each utility we also confirm the predicted tie with the tuned GBDT.

The third check addresses the age-myth dispute directly. We report the age curve implied by the cross-utility-weighted ensemble for asbestos-cement and cast-iron pipe, with the spread across members. The test is whether the 30-year "inflection" survives once vintage is allowed to absorb trend, or whether it is better explained as a pre-1985 cohort effect.

Data access is the main practical risk. None of these utility datasets is public in the file, and the proposal depends on obtaining at least two dated failure panels for the same material.