TL;DR
Peter McCullagh’s 1980 seminal paper provides a robust statistical framework for regression when the outcome is ordinal—data that has a natural order (like "mild," "moderate," "severe") but no fixed numerical distance between categories. By introducing the proportional odds and proportional hazards models, McCullagh enables researchers to analyze these datasets without the pitfalls of treating categories as arbitrary numbers or ignoring their inherent sequence.
What Problem Does This Paper Address?
In fields ranging from sociology to medicine, researchers frequently encounter ordinal data. For example, a patient's recovery might be rated as "poor," "fair," or "good." Historically, statisticians handled this in two suboptimal ways:
- Treating it as nominal: Ignoring the order entirely, which throws away valuable information.
- Assigning arbitrary scores: Treating "poor" as 1, "fair" as 2, and "good" as 3. This assumes the "distance" between poor and fair is identical to the distance between fair and good—an assumption that is rarely justifiable.
McCullagh tackles the challenge of building a regression model that respects the stochastic ordering of the data—the idea that one group is generally "higher" on the scale than another—without assuming cardinality.
Methods
The paper proposes a general class of models based on cumulative probabilities. Instead of modeling the probability of being in a specific category, McCullagh models the probability of being in category $j$ or lower.
The core equation takes the form: $$link\{\gamma_j(x)\} = \theta_j - \beta^T x$$ Where:
- $\gamma_j(x)$ is the cumulative probability for category $j$.
- $\theta_j$ are "cut points" or thresholds that represent the boundaries between categories on an underlying latent continuous scale.
- $\beta$ represents the effect of the covariates (explanatory variables).
The paper highlights two primary models:
- The Proportional Odds Model: Uses the logit link function. It assumes that the odds of being below any given threshold are multiplied by a constant factor for each unit change in a covariate.
- The Proportional Hazards Model: Uses the complementary log-log link function. This is particularly useful for asymmetric data or when the categories represent intervals of a survival-like process.
Estimation is performed using Maximum Likelihood (ML), and McCullagh demonstrates that the method of iteratively reweighted least squares converges reliably to the ML estimate.
Datasets
McCullagh illustrates his models using several classic datasets:
- Tonsil Size: Data from 1398 children classified by tonsil size (not enlarged, enlarged, greatly enlarged) and whether they were carriers of Streptococcus pyogenes.
- Family Income: Distribution of family income in the Northeast U.S. in 1960 vs. 1970 across seven categories.
- Disturbed Dreams: A study of the severity of disturbed dreams in boys aged 5–15.
- Vision Quality: A comparison of right-eye vision quality between men and women.
- Taste-Testing: A five-treatment experiment where judges rated products from "Terrible" to "Excellent."
Key Insights and Ideas
- Invariance Under Grouping: A major "aha moment" is that these models are invariant to how categories are grouped. If you merge "enlarged" and "greatly enlarged" tonsils into a single "enlarged" category, the estimate for the treatment effect ($\beta$) remains essentially the same. This makes the model robust to subjective categorization.
- Latent Variable Interpretation: The models can be viewed as partitioning a hidden, continuous variable (like "severity" or "tolerance") into intervals. This provides a physical intuition for the "cut points" ($\theta_j$).
- Stochastic Ordering: The models provide a formal way to say that one population is "stochastically larger" than another, meaning its cumulative distribution function is shifted.
Key Results
In the tonsil size example, the model clearly quantifies the impact of being a carrier.
By analyzing the logits (as seen in the analysis below), McCullagh shows that the odds of having "greatly enlarged" tonsils are approximately 1.8 times higher for carriers than for non-carriers.
In the income distribution example, the proportional hazards model reveals a consistent shift toward higher income brackets from 1960 to 1970.
The differences across categories in Table 5 are remarkably stable (ranging from 0.38 to 0.67), suggesting that the proportional hazards assumption is a good fit for describing how the entire income distribution shifted over the decade.
Limitations and Future Work
- Sparse Data: The author notes that maximum likelihood estimates may fail to exist or become infinite if the data is too sparse (e.g., many categories with zero observations).
- Link Selection: While logit and log-log links are preferred for their interpretability, choosing between them requires careful consideration of the data's symmetry.
- Non-Linearity: The paper introduces extensions for non-linear models (where the "scale" or variance might also change between groups), but acknowledges these are computationally more demanding and harder to interpret.
- Multivariate Responses: The framework primarily focuses on a single ordinal response; extending this to bivariate or multivariate ordinal responses is mentioned as an area for further research.
Summary updated