Using Unlabeled Patients to Stabilize the Shape Functions of Additive KANs

Clinicians trust a risk model more when they can see, for each variable, a curve showing how risk rises or falls with it: how mortality changes with lactate, age, or creatinine. Models built this way are called generalized additive models (GAMs). A Kolmogorov-Arnold Network (KAN), a neural network whose connections carry small learnable one-dimensional curves instead of single numbers, can be restricted to exactly this additive form while remaining trainable by gradient descent on a GPU. The difficulty is that clinical labels such as adjudicated outcomes or long-term follow-up are scarce. With few labels, a flexible curve fits noise. Accuracy may hold up, but the curves a clinician reads become wiggly, shift from one training sample to the next, and are least reliable in the sparse tails where the sickest patients sit.

This proposal treats unlabeled patient records as a way to make the explanation itself more trustworthy, and only secondarily as a way to gain accuracy. Unlabeled data shows where patients actually lie along each variable, and an averaged "teacher" copy of the model can supply soft targets on those patients. Both can be applied directly to the KAN's edge curves. The model's explanation is then regularized without the costly gradients-of-gradients that earlier explanation-consistency methods required. The headline question is whether, at 0.5–10% label fractions on intensive care unit (ICU) mortality data, a semi-supervised additive KAN recovers per-variable risk curves closer to those of a fully labeled model, and less variable across label draws, than strong supervised and distillation-based alternatives.

Core Claim

In an additive KAN, each variable's contribution $\phi_i(x_i)$ is both the model and the explanation. This allows three uses of unlabeled data that act on the explanation directly. First, the spline knots of each $\phi_i$ are placed at quantiles of the unlabeled marginal of $x_i$, so the curve has resolution where patients are and stays coarse where they are not. Second, an exponential-moving-average teacher produces class-balanced, confidence-filtered pseudo-labels on unlabeled patients. Third, a shape-consistency term asks the student's curves to agree with the teacher's on the points where unlabeled patients actually lie:

$$\mathcal{L}_{\text{shape}} = \sum_{i=1}^{p} \mathbb{E}_{x \sim \mathcal{U}}\big[(\phi_i^{s}(x_i) - \phi_i^{t}(x_i))^2\big]$$

Here $\mathcal{U}$ is the unlabeled pool, $\phi_i^{s}$ is the student's curve for variable $i$, and $\phi_i^{t}$ is the teacher's curve, computed with edge dropout applied to the student. This is a first-order loss on quantities the model already computes.

The specific claim is that this combination mainly reduces explanation variance under label scarcity. It should beat both labeled-only smoothing penalties and black-box-teacher distillation on curve fidelity, while matching them on AUROC.

Grounding in Prior Work

Four lines of work set up the problem. Two motivate it, one constrains it, and one supplies parts of the method.

The motivation comes from two KAN analyses. On the expressiveness and spectral bias of KANs shows that KANs lack the low-frequency bias of ordinary networks. As a result they fit high-frequency noise quickly, and refining the spline grid in undersampled regions makes training loss oscillate. That is precisely the low-label failure mode for clinical curves. Generalization Bounds and Model Complexity for Kolmogorov–Arnold Networks bounds generalization error by a complexity term $\tilde{\alpha}$, built from the size and wiggliness of the edge-function coefficients, scaling roughly as $\tilde{\alpha}^3/n$ in the number of labeled samples $n$. It also shows that dropout on edges lowers $\tilde{\alpha}$. We use this only as motivation for regularizing edge curves directly and for using edge dropout as the teacher's perturbation, not as a guarantee for the semi-supervised setting.

The closest prior work falls into three groups, each covering two of the three directions. Interpreting Kolmogorov-Arnold Networks in Neuroimaging: A Path-Based Attribution Framework traces attribution through KAN edges for Alzheimer's MRI. It is fully supervised and uses images, whereas we study tabular records with scarce labels. NODE-GAM builds a differentiable GAM from soft decision trees and uses masked-feature pretraining, reporting gains of "up to 10%" at 0.5% labels. Its unlabeled data enters only before fine-tuning and never acts on the shape curves during training. Explanation Consistency Training asks explanations, not just predictions, to stay consistent across perturbations. It does so through saliency maps or attention, which requires second-order gradients, and it evaluates only on vision and text. Our contribution is to apply its idea to a model whose explanation is native and whose consistency loss is therefore cheap.

Two papers constrain the design. VIME perturbs a tabular row by swapping some feature values with values drawn from other rows' marginals, then trains with consistency between the original and perturbed rows. In an additive model, corrupting feature $j$ leaves $\phi_i(x_i)$ unchanged for every $i \neq j$, so input-side consistency gives almost no signal about the curves. This is why our consistency is model-side, between teacher and student. Revisiting Clinical Outcome Prediction for MIMIC-IV documents severe class imbalance, under which pseudo-labeling tends to amplify the majority class. This is why our pseudo-labels are class-balanced and evaluated by PR-AUC. Finally, MIMIC-IF finds that attention weights fail remove-and-retrain (ROAR) faithfulness tests on ICU mortality. That finding argues for glass-box curves over post-hoc attention, and it supplies the ROAR protocol we reuse.

Research Program

We will build an additive KAN with one B-spline edge per variable plus a small, sparsity-penalized set of pairwise edges, keeping the model at most a GA²M (main effects plus pairwise interactions) so every component can be plotted. For speed we will use warpKAN's GPU kernels or PowerMLP-style restructuring where available, though tabular widths are small enough that vanilla KANs suffice.

The training loss has four parts: supervised cross-entropy on labeled patients; class-balanced pseudo-label cross-entropy on confident unlabeled patients; $\mathcal{L}_{\text{shape}}$; and a curvature penalty weighted by inverse unlabeled density, so that curves are stiff where few patients exist. Each ingredient can be switched off independently.

The analysis has two parts. On synthetic tabular data with known ground-truth shape functions, we will vary the label fraction, the unlabeled pool size, and the tail sparsity, and measure how error in the recovered curves decomposes into bias and variance. On real ICU data we will treat the curves from a fully labeled model as a reference and measure how far low-label curves fall from it.

Baselines

The naive reference point is the same additive KAN trained only on the labeled subset, with standard L1 and smoothness penalties tuned on a labeled validation split. It isolates whether unlabeled data contributes anything at all.

The strongest interpretable supervised competitor is a tree-based Explainable Boosting Machine (EBM), the tree-based GAM discussed in NODE-GAM, trained on labeled data only. It is the default clinical glass-box model and is known to be robust with few samples. We will also run NODE-GA²M with its masked pretraining on the same unlabeled pool, since it is the existing method that most directly uses unlabeled data for an interpretable tabular model. On low-label gains, NODE-GAM reports improvements of "up to 10%" at 0.5% labeled data. For its MIMIC-II, MIMIC-III, and Support2 AUROC, we found no reported result on Lacuna in the file. For semi-supervised black-box accuracy, VIME reports an AUROC of "0.8602" on prostate cancer treatment prediction, against "0.8423" for XGBoost. We will rerun VIME on our splits rather than borrow that number.

A further competitor is distillation. A black-box teacher (XGBoost, or VIME's network) is trained on labeled data, labels the unlabeled pool, and an additive KAN or EBM is fit to its soft outputs.

Ablations remove, one at a time, quantile knots, pseudo-labels, the shape-consistency loss, and density-weighted curvature. The most important ablation keeps pseudo-labels but drops shape consistency, which isolates whether regularizing the curves adds anything beyond extra targets.

For a fair comparison, all methods see identical labeled and unlabeled splits, drawn with ten random seeds per label fraction. Each gets the same hyperparameter search budget of 50 trials, chosen by labeled-validation PR-AUC, and comparable GPU hours.

Beating the baselines means two things. First, AUROC must be statistically indistinguishable from or better than the best competitor. Second, curve error against the fully labeled reference, and across-seed curve variance, must be significantly lower than both EBM and distillation at 1% labels or fewer.

Why not just…?

Why not just fit an EBM on the labeled data? This is a real threat. Boosted GAMs are hard to beat at low sample sizes, and if EBM's curves are already as stable as ours, the proposal fails. The only argument against it is structural: EBM never sees where unlabeled patients lie, so its step-function curves in sparse tails rest on a handful of labeled points. The across-seed curve-variance metric, broken down by density quantile, settles this directly. We expect any advantage to appear in the tails and not in the bulk of the data.

Why not distill a strong black-box teacher into a GAM using unlabeled data? This also uses the unlabeled pool, and with a better teacher it may well match us on AUROC. Its weakness is that the GAM inherits whatever the teacher learned, including non-additive artifacts. Those artifacts get pushed onto the curves in ways that may not reflect the variable's actual effect. We will measure curve error on synthetic data where the true shapes are known. If distillation ties on curve error as well, the honest conclusion is that the source of pseudo-labels matters less than having them. In that case we will report shape consistency and density-aware knots as add-ons to distillation rather than as a replacement.

Why not just apply ECT-style explanation consistency or VIME to the KAN? Input-corruption consistency carries almost no information about the curves of an additive model, as argued above, and saliency-based ECT pays a second-order gradient cost for a quantity the KAN exposes directly. The VIME-perturbation variant of our model, included as an ablation, should show near-zero gain on curve metrics. That outcome would confirm why the consistency must be model-side.

Proposed Experiments

The benchmarks are MIMIC-II, MIMIC-III mortality, and Support2, as used by NODE-GAM, plus MIMIC-IV in-hospital mortality following the MIMIC-IV outcome study. We use label fractions of 0.5%, 1%, 5%, and 10%, with the remaining training patients treated as unlabeled. Synthetic additive-plus-pairwise tasks with heavy-tailed marginals supply ground-truth curves.

The metrics fall into three groups. For prediction we report AUROC, PR-AUC, and expected calibration error, overall and on the demographic subgroups that MIMIC-IF flags. For explanation we report the integrated squared distance, weighted by unlabeled density, between each low-label curve and the full-label or true curve; across-seed variance of each curve; and ROAR degradation. We also run a small blinded review in which clinicians judge whether curves from different methods for lactate, age, and creatinine are clinically plausible.

The decisive check works as follows. If, at 1% labels or fewer, the full method lowers tail-region curve error and variance by a clear margin relative to EBM and distillation, with no AUROC loss, and the ablation without shape consistency loses most of that gain, then unlabeled data is improving explanations through the mechanism proposed here. If the gains appear only in AUROC, or vanish against distillation, the core claim is false, and we will report that result.