TL;DR
This paper introduces a novel statistical framework, the Penalized Multiple-Inflated Negative Binomial (MINB) model, designed to analyze self-reported count data where respondents tend to "heap" or round their answers (e.g., reporting 10 or 20 instead of 9 or 21). By combining Negative Binomial regression with adaptive LASSO regularization, the model automatically identifies which specific values are inflated and which underlying factors actually drive behavior, significantly outperforming standard models that only account for zero-inflation.
What Problem Does This Paper Address?
In social science and market research, we often rely on self-reported counts—like "How many times did you use a designated driver in the last six months?" This data suffers from two major issues:
- Zero-Inflation: A huge chunk of the population never uses the service.
- Multiple Inflation (Heaping): Humans are bad at precise recall. Instead of saying "11 times," someone might say "10." This creates artificial spikes at "round" numbers (5, 10, 20), which distorts standard statistical distributions like Poisson or Negative Binomial.
If a researcher ignores these spikes, their regression estimates become biased. While "Zero-Inflated" models exist, they usually ignore the other spikes. Existing "Multiple-Inflated" models often require the researcher to manually guess which values are inflated by looking at a histogram—a subjective and error-prone process.
Methods
The authors propose the MINB model with penalization. Here’s the breakdown of their approach:
- The Base Model: They use a Negative Binomial (NB) distribution because it handles "overdispersion" (when data is more spread out than a simple Poisson model expects).
- The "Multiple Inflation" Component: They treat the observed data as a mixture. A data point either comes from the "standard" NB process or from one of several "inflated" points (like 0, 10, or 20).
- Adaptive LASSO Regularization: This is the key innovation. Instead of the researcher picking which values are inflated, the model uses a penalty function (adaptive LASSO) to shrink unimportant mixing proportions and irrelevant factors to zero.
- Simultaneous Selection: The algorithm (an Expectation-Maximization or EM algorithm) simultaneously decides:
- Which integers are actually "inflated" (data-driven selection).
- Which external variables (covariates) are statistically significant.
Datasets
The model was validated using two primary sources:
- Online Chauffeured Service Demand (OCSD) Data: A real-world dataset from a leading Chinese mobility platform (likely Didi). It contains 3,996 survey responses regarding the use of designated driver services, including demographics, drinking habits, and car status.
- Extensive Simulations: The authors generated synthetic datasets with varying levels of "noise," sparsity, and inflation to prove their model could recover the "ground truth" better than existing methods.
Key Insights and Ideas
- Beyond the "Round Number" Assumption: While we expect inflation at 10 or 20, the model identified inflation at 3, 6, and 12. The authors suggest these represent "once every two months" or "once/twice a month" recall patterns, which a human researcher might have overlooked.
- Correcting Nonsensical Results: Standard models applied to the OCSD data suggested that stricter drunk driving punishments decreased demand for designated drivers—a result that defies logic. The MINB model corrected this, showing that awareness of punishment actually increases demand, as expected.
- The "Non-Demanding" Group: By accurately modeling the 0-inflation (64% of the data), the model distinguishes between people who can't use the service and those who choose not to based on specific factors like age or car ownership.
Key Results
- Superior Fit: Using the Vuong test (a standard for comparing non-nested models), the MINB model significantly outperformed six other common approaches, including standard Poisson, NB, and even two-stage inflated models.
- Factor Identification: The model identified 10 critical factors for service demand. For example:
- Age: Users under 40 are significantly more likely to use the service, likely due to higher tech-literacy with app-based platforms.
- Psychology: Car owners who are protective of their vehicles (don't like strangers touching them) are less likely to hire a chauffeur.
- Drinking Frequency: As expected, higher drinking frequency is a primary driver of demand.
- Mathematical Consistency: The authors proved that as the sample size grows, the model’s estimates converge to the true values (root-n consistency) and it correctly identifies the "true" factors (selection consistency).
Limitations and Future Work
- Constant Mixing Proportions: Currently, the model assumes the probability of "rounding" to 10 is the same for everyone. In reality, certain demographics (e.g., older vs. younger) might have different rounding behaviors. Future work could make these mixing proportions "covariate-dependent."
- Distributional Assumptions: The model still relies on the Negative Binomial as a baseline. If the underlying data follows a wildly different distribution, the model might still struggle. The authors suggest exploring "distribution-free" versions using Generalized Estimating Equations (GEE) in the future.
Summary updated