Abstract
Predicting citizen happiness is a complex task that has traditionally relied on interpretable but limited linear models. While deep learning offers superior accuracy, its "black box" nature hinders its utility in social science. This paper presents a framework using Neural Factorization Machines (NFM) combined with Shapley values to provide mathematically rigorous explanations for happiness predictions. Our results show that Shapley values align more closely with sociological theory than standard attention mechanisms, identifying health and social equity as primary drivers of happiness.
Introduction
Happiness is a multi-faceted emotional state influenced by a wide array of personal, economic, and social factors. For decades, sociologists have utilized Multivariate Linear Regression (MLR) to understand these drivers. However, MLR often fails to capture the non-linear interactions between variables—such as how income's impact on happiness might vary significantly based on an individual's health status.
Deep Learning (DL) models can capture these complexities but lack transparency. For policy-makers, understanding the why behind a prediction is as critical as the prediction itself. This study addresses the accuracy-interpretability trade-off by applying coalitional game theory to explain neural network outputs.
Methodology
The proposed framework utilizes Deep Neural Networks (DNNs), specifically the Neural Factorization Machine (NFM), which is adept at capturing second-order feature interactions. To interpret these models, the authors employ Shapley values.
Borrowed from coalitional game theory, a Shapley value determines a feature's "fair share" of a prediction by calculating its average marginal contribution across all possible feature combinations. To handle the computational complexity of 140+ features, a Monte-Carlo sampling algorithm was implemented to approximate these values efficiently. The approach was compared against the popular Attention mechanism to evaluate the quality of explanations.
Experimental Setup
The study used the Chinese General Social Survey (CGSS) 2015 dataset, comprising 8,000 samples. Each sample includes 140 features categorized into Personal Information (e.g., health, job), Economics (e.g., family status, insurance), and Life-style (e.g., media usage, socializing). The target variable is a 5-level happiness scale.
Results and Discussion
The NFM model achieved a Micro-F1 score of 0.4841, outperforming traditional Logistic Regression (0.4637). This confirms that non-linear feature interactions are vital for accurate happiness modeling.
A key contribution of this work is the comparison between Shapley values and the Attention mechanism for interpretability. As shown in Figure 1, the Attention mechanism often assigns high, static weights to features like "nationality," which is less informative in a single-country survey context.
In contrast, Shapley values (Figure 2) provide granular insights that vary across happiness levels. For instance, "health" (labeled as 'heath' in the data) is correctly identified as a dominant factor for higher happiness levels, aligning with established sociological literature. Unlike attention weights, Shapley values satisfy mathematical properties like symmetry and additivity, ensuring a fairer distribution of feature importance.
Across all categories, the top five influential factors identified were Equity, Depression, Family Status, Class, and Health.
Conclusion
By bridging the gap between high-accuracy deep learning and transparent social science, this research demonstrates that Shapley values offer a superior method for interpreting complex happiness models. While computational complexity remains a challenge, future work will focus on optimizing sampling methods to enable the analysis of even larger datasets for real-world policy decision-making.
Summary updated