Patients on breathing machines often fall out of step with the machine. The most common and hardest-to-see mismatch is the ineffective effort: the patient tries to take a breath and the ventilator never responds. Detectors trained on bedside signals catch most mismatches. When they are limited to the two signals every ventilator records, mask pressure and airflow, they miss many ineffective efforts. Adding chest and abdominal effort belts helps, but belts are uncomfortable, slip, and are rarely worn outside a sleep laboratory. A clinician choosing a two-signal monitor today learns only that its ineffective-effort accuracy is "lower on average." She has no way to know when to stop trusting it.
We propose to turn that average into a map. We train the two-channel detector, collect its mistakes on patients it has never seen, and search for a short set of human-readable rules that describe where those mistakes concentrate. The rules are written over sliding-window context features, meaning summaries of the preceding thirty seconds to ten minutes of breathing, such as rising leak, unstable breath timing, or a recent burst of machine-initiated breaths. They are not written over the waveform of the single breath being classified. Our hypothesis is that two-channel misses of ineffective efforts are confident, clustered in time, and predictable from context. If so, a compact rule set found on one group of patients will mark high-risk periods in new patients better than the detector's own confidence does. That rule set is a global explanation in the useful sense: it describes the model's blind spots across a whole population, and a clinician can use it to decide when to add a belt or review the waveform.
Core Claim
Existing interpretable asynchrony work explains individual breaths. SHIP classifies breaths as double triggering, autocycling, ineffective effort, or normal, using shapelets (short discriminative waveform snippets) plus statistics, and explains a prediction by its nearest snippet. GMGC-NL summarizes each breath with path signatures, scores it with a density model to get calibrated probabilities and out-of-distribution flags, and corrects for noisy expert labels. Both motivate this proposal. SHIP reports that ineffective efforts are its weakest class: it reaches an ineffective-effort F1 of 0.7050 even with four channels, and performance drops further with two channels, although the file gives no two-channel ineffective-effort number. Neither paper says which breaths or periods account for the drop.
Our claim is specific and falsifiable. Rules mined on training-patient errors will have three properties. First, they will cover a small fraction of held-out breaths that contains a large share of held-out ineffective-effort misses. Second, they will beat the detector's own uncertainty at the same coverage. Third, they will lose most of this advantage if restricted to within-breath features. That third property is what makes the multi-breath window, not the rule miner, the key ingredient.
Research Program
The data come from the Austin Health non-invasive ventilation titration recordings used by SHIP and GMGC-NL: 59 patients with motor neurone disease, 104 overnight studies, and more than 300,000 annotated breaths with pressure, flow, and both effort belts. Recent study protocol for a randomised controlled trial work describes the titration setting. All splits are by patient.
The audited model is a two-channel detector. We train SHIP's architecture on pressure and flow, and in parallel a LightGBM classifier on per-breath features, since the GMGC-NL authors report that a well-tuned LightGBM matches their model at an AUROC of approximately 0.98. Out-of-fold predictions across patient folds give every breath an honest prediction from a model that never saw that patient.
The description language is a set of multi-scale window summaries computed at each breath over lookbacks of roughly 30 seconds, 2 minutes, and 10 minutes. It follows the temporal-aggregation practice of Kurasawa et al., who used nested 3-, 6-, and 12-month windows. It also follows the observation in Amrollahi et al. that means hide volatility, so percentiles and rates of change are needed. Our features include percentiles and slopes of breath period, tidal-flow amplitude, and an estimated leak (baseline flow offset). They also include variability of expiratory time and the detector's own recent predicted-class densities, for example how many autocycles it flagged in the last two minutes. We keep the language deliberately at hundreds to a few thousand binarized features. This is well inside the tens of thousands that the pattern miner is reported to handle, and it avoids the 149,699-feature sprawl Kurasawa et al. needed.
The audit method is PREMISE. PREMISE searches for a small set of AND/XOR rules that, under the minimum description length principle (prefer the explanation that compresses the error labels most), best describes which inputs a classifier gets wrong. It filters spurious rules with Fisher's exact test. PREMISE was built and tested on language and visual question answering tokens. We reuse it unchanged on breath-level context features and treat "missed ineffective effort" as its label. Because MDL charges for every rule, it also gives a principled answer to the usually arbitrary choice of window length: windows that do not help describe errors simply do not appear in the rules.
Two clinical controls are built in. First, GMGC-NL shows that expert labels on these data are noisy, so some "errors" are annotation mistakes. For a stratified sample of breaths inside and outside rule regions, a respiratory clinician will re-score each breath blind to model output and to rule membership, using all four channels. Second, inside rule regions the effort belts serve as a physiological reference: a true missed ineffective effort should show a belt deflection with no ventilator response.
Baselines
The naive reference point is uniform suspicion. A monitor that flags a random fraction of breaths recovers misses only in proportion to that fraction, so any audit must beat the diagonal on a curve of "share of misses captured" against "share of breaths flagged."
The strongest competitor is the detector's own uncertainty. This is the maximum-softmax confidence of the two-channel SHIP and LightGBM models, and, more strongly, the GMGC-NL density and out-of-distribution score. GMGC-NL was designed to flag atypical breaths and is reported to give better calibration and smaller between-patient variability. The file gives no reported calibration number for it, only the AUROC of approximately 0.98. We will run each uncertainty score on the same out-of-fold predictions and compare it to the rules at matched coverage, the fraction of held-out breaths flagged. Beating it means capturing more held-out ineffective-effort misses at, say, 5% and 10% coverage, with a patient-level bootstrap interval that excludes zero.
The simplest global alternative is a shallow decision tree, the CART algorithm, fit to the same error labels over the same context features. We cap it at the same number of leaves as PREMISE has rules and tune depth with the same patient-fold budget. This tests whether the MDL search adds anything over standard slice discovery.
The key ablation reruns PREMISE using only within-breath features: per-breath statistics, SHIP shapelet distances, and breath duration. If within-breath rules describe held-out errors as well as context rules, then the sliding window is not the source of the gain and the central hypothesis fails.
The ceiling reference is the four-channel detector. Its ineffective-effort F1 of 0.7050 (SHIP) sets the target that "two channels plus belts only where the rules fire" tries to approach.
Why not just…?
Why not just use the detector's confidence to decide when to distrust it? This is the real threat. If ineffective-effort misses happen on breaths the model already finds ambiguous, then calibrated uncertainty such as GMGC-NL's does everything the rules do, and the rules only add a description. Our hypothesis is the opposite: many two-channel misses are confident, because an effort too weak to move flow produces a waveform that looks like an ordinary expiration. The matched-coverage comparison settles this directly. We also report the joint distribution of rule membership and confidence. If rules capture no misses that uncertainty misses, we will say the contribution is descriptive only.
Why not just add the window features to the detector and retrain? That is a sensible engineering fix, and we will run it. If context features raise two-channel ineffective-effort F1, that is useful, and the audit is what tells us which contexts to add. Retraining does not remove the need for a blind-spot map, though. Whatever errors remain still need describing, and a single aggregate F1 still cannot tell a clinician when to add a belt. We will audit both the original and the context-augmented detector and report how much error mass remains rule-describable after augmentation.
Why not just fit a decision tree on the errors? A tree may work nearly as well. The CART baseline exists to find out. The specific reasons to expect PREMISE to help are its significance filtering, which matters with only 59 patients, and its XOR clauses, which matter where mutually exclusive contexts lead to the same failure. If CART matches it on held-out lift at equal size, we adopt the simpler tool and the claim shrinks to "context-based audits work," which is still the main point.
Why not just recommend ventilator setting changes, as counterfactual explanation trees would? Counterfactual Explanation Trees build one tree that assigns consistent actions to subgroups of a tabular dataset. Applied here, its "actions" would change the classifier's inputs, not the patient's physiology. Without randomized or carefully modeled setting changes, a recommendation such as "raise trigger sensitivity" would be a statement about the model, not about the lung. We therefore deliberately stop at an audit of where the detector should not be trusted, which needs no causal assumption.
Proposed Experiments
The primary experiment uses patient-grouped five-fold cross-validation, nested so that rules are mined only on training-fold errors and scored only on held-out patients. The outcome is the miss-capture curve described above, summarized at 5% and 10% coverage, for context rules, within-breath rules, CART, the two uncertainty scores, and random flagging.
The decisive check has three parts that must all hold. Context rules must beat both uncertainty baselines at matched coverage. They must beat the within-breath ablation. Their lift must persist on held-out patients, not just on training folds. If the first part fails, the work reduces to a description of known uncertainty. If the second fails, the sliding-window hypothesis is wrong.
Secondary checks address whether the rules mean something. Blind clinician re-scoring estimates what fraction of in-rule "errors" are true misses rather than label noise; if most are label noise, the rules are auditing the annotators, which is worth reporting but changes the conclusion. Belt deflections give a physiological check on the true misses. A simulated deployment combines the two-channel detector with four-channel input only inside rule regions. We report its ineffective-effort F1 against the 0.7050 four-channel reference and the fraction of time belts would be needed.
As a stretch test of transfer, we will apply the same audit to the Colorado esophageal-balloon cohort from the deformed lung ventilator paper. That cohort is invasive ventilation with flow-limited labels, so we will only test whether context rules again outperform within-breath rules for that detector's errors, not whether the specific rules carry over.
References
- SHIP: A Shapelet-Based Approach for Interpretable Patient-Ventilator Asynchrony Detection
- Uncertainty-aware non-invasive patient-ventilator asynchrony detection using latent Gaussian mixture generative classifier with noisy label correction
- Polysomnographic titration of non-invasive ventilation in motor neurone disease (3TLA): study protocol for a randomised controlled trial
- Treatment Discontinuation Prediction in Patients With Diabetes Using a Ranking Model: Machine Learning Model Development
- Predicting Hospital Readmission among Patients with Sepsis Using Clinical and Wearable Data
- Label-Descriptive Patterns and Their Application to Characterizing Classification Errors
- Counterfactual Explanation Trees: Transparent and Consistent Actionable Recourse with Decision Trees
- Quantifiable identification of flow-limited ventilator dyssynchrony with the deformed lung ventilator model