TL;DR
This paper introduces a novel statistical framework for "density-on-scalar regression" that models entire probability distributions as responses to scalar predictors. This approach, grounded in Bayes Hilbert spaces and estimated using gradient boosting, allows for flexible modeling of complex density shapes, including mixed discrete-continuous distributions, and offers interpretable results through odds ratios. Applied to German gender economics data, it reveals nuanced insights into female income shares, highlighting persistent East-West differences and the impact of children.
What Problem Does This Paper Address?
Traditional regression models often focus on scalar summaries of data, like means or medians. However, when analyzing distributions, such as income shares, these summaries can obscure crucial information about the entire shape, multimodality, or the presence of discrete point masses. Existing methods for modeling densities as responses face challenges in preserving the fundamental properties of probability distributions (non-negativity and integration to one) and often struggle with mixed discrete-continuous data, which arises naturally in contexts like income shares where zero or full income for one partner creates point masses. This paper addresses the need for a flexible, theoretically sound, and interpretable statistical framework to model entire probability densities as a function of scalar covariates, particularly for mixed-type densities.
Methods
The core of the paper is the introduction of an additive density-on-scalar regression model within Bayes Hilbert spaces. This choice is motivated by the need to handle probability densities as elements of a linear space while respecting their inherent properties. The authors leverage the centered log-ratio (clr) transformation to map densities into a standard L² space, enabling the use of powerful functional data analysis tools.
Estimation is performed using a gradient boosting algorithm, adapted for Bayes Hilbert spaces. This approach offers several advantages:
- Flexibility: It allows for a wide range of covariate effects, including linear, nonlinear, and interaction terms.
- Model Selection: The boosting process inherently performs variable and effect selection through early stopping.
- Regularization: It naturally incorporates regularization to prevent overfitting.
A key methodological contribution is the development of properties of Bayes Hilbert spaces related to subcompositional coherence. This allows for interpretable odds-ratio interpretations of effect functions and simplifies the estimation of mixed densities through orthogonal decomposition.
Datasets
The primary dataset used in this paper is the German Socio-Economic Panel (SOEP). This dataset contains longitudinal information on households and individuals in Germany. For their analysis, the authors utilize data from 1984 to 2016 (with East German data available from 1991 onwards), focusing on couples and their income shares.
Key Insights and Ideas
- Densities as Richer Responses: Modeling entire probability densities, rather than just scalar summaries, provides a much deeper understanding of the underlying phenomena, revealing nuances like multimodality and shifts in probability mass.
- Bayes Hilbert Spaces for Densities: This mathematical framework elegantly handles the properties of probability densities (non-negativity, integration to one) and provides a linear space for statistical modeling, crucial for additive regression.
- Gradient Boosting for Flexible Density Regression: The proposed gradient boosting algorithm offers a powerful and flexible way to estimate complex density-on-scalar models, performing both estimation and model selection simultaneously.
- Interpretable Effects via Odds Ratios: The framework allows for intuitive odds-ratio interpretations of covariate effects, making the results more accessible and meaningful, especially in the context of gender economics.
- Mixed Densities are Crucial: The paper highlights that real-world data, like income shares, often involve mixed discrete-continuous distributions (point masses at 0 and 1 for single-earner couples), and the proposed method effectively handles this complexity.
- Persistent East-West Differences: The application to German data reveals that while West-East differences in female income shares and the childhood penalty have diminished since reunification, they remain persistent over time.
Key Results
- The proposed additive density-on-scalar regression model successfully captures complex density shapes and relationships with covariates.
- The gradient boosting estimation procedure demonstrates good performance in simulation studies, with high explained variance (e.g., 47% for continuous components, 69% for discrete components in the application).
- Analysis of the SOEP data reveals:
- More symmetric income share distributions in East Germany compared to West Germany after reunification.
- A smaller "child penalty" (lower female income share in the presence of children) in East Germany compared to West Germany.
- These West-East differences in the childhood penalty have decreased over time but remain persistent.
- Trends show an increase in part-time employment for women over time, affecting density shapes.
Limitations and Future Work
- Inference: The current gradient boosting approach does not directly provide formal statistical inference (p-values, confidence intervals). The authors suggest that bootstrap-based approaches or selective inference could be developed in the future.
- Finite Reference Measures: The current definition of Bayes Hilbert spaces is restricted to finite reference measures. Extending this to σ-finite measures, particularly for the real line with Lebesgue measure, would be beneficial.
- Positive Densities: Bayes Hilbert spaces only contain positive densities. While density estimation methods can often avoid zero values, alternative transformations like the square root velocity transformation might be considered for cases where zero densities are problematic.
- Individual-Level Densities: The current approach models estimated densities. Future work could extend the framework to directly model conditional densities for individual observations, accounting for estimation uncertainty at the individual level.
- Functional Covariates: While the paper focuses on scalar covariates, extensions to functional covariates are suggested as a possible future direction, building on existing work in functional regression.
Summary updated