Towards the Quantitative Interpretability Analysis of Citizens Happiness Prediction

Lin Li, Xiaohua Wu, Miao Kong, Dong Zhou, Xiaohui Tao

Abstract

Predicting citizen happiness is a complex task that has traditionally relied on interpretable but limited linear models. While deep learning offers superior accuracy, its "black box" nature hinders its utility in social science. This paper presents a framework using Neural Factorization Machines (NFM) combined with Shapley values to provide mathematically rigorous explanations for happiness predictions. Our results show that Shapley values align more closely with sociological theory than standard attention mechanisms, identifying health and social equity as primary drivers of happiness.

Introduction

Happiness is a multi-faceted emotional state influenced by a wide array of personal, economic, and social factors. For decades, sociologists have utilized Multivariate Linear Regression (MLR) to understand these drivers. However, MLR often fails to capture the non-linear interactions between variables—such as how income's impact on happiness might vary significantly based on an individual's health status.

Deep Learning (DL) models can capture these complexities but lack transparency. For policy-makers, understanding the why behind a prediction is as critical as the prediction itself. This study addresses the accuracy-interpretability trade-off by applying coalitional game theory to explain neural network outputs.

Methodology

The proposed framework utilizes Deep Neural Networks (DNNs), specifically the Neural Factorization Machine (NFM), which is adept at capturing second-order feature interactions. To interpret these models, the authors employ Shapley values.

Borrowed from coalitional game theory, a Shapley value determines a feature's "fair share" of a prediction by calculating its average marginal contribution across all possible feature combinations. To handle the computational complexity of 140+ features, a Monte-Carlo sampling algorithm was implemented to approximate these values efficiently. The approach was compared against the popular Attention mechanism to evaluate the quality of explanations.

Experimental Setup

The study used the Chinese General Social Survey (CGSS) 2015 dataset, comprising 8,000 samples. Each sample includes 140 features categorized into Personal Information (e.g., health, job), Economics (e.g., family status, insurance), and Life-style (e.g., media usage, socializing). The target variable is a 5-level happiness scale.

Results and Discussion

The NFM model achieved a Micro-F1 score of 0.4841, outperforming traditional Logistic Regression (0.4637). This confirms that non-linear feature interactions are vital for accurate happiness modeling.

A key contribution of this work is the comparison between Shapley values and the Attention mechanism for interpretability. As shown in Figure 1, the Attention mechanism often assigns high, static weights to features like "nationality," which is less informative in a single-country survey context.

Attention mechanism weights for personal information features.

In contrast, Shapley values (Figure 2) provide granular insights that vary across happiness levels. For instance, "health" (labeled as 'heath' in the data) is correctly identified as a dominant factor for higher happiness levels, aligning with established sociological literature. Unlike attention weights, Shapley values satisfy mathematical properties like symmetry and additivity, ensuring a fairer distribution of feature importance.

Shapley values for personal information features across happiness levels.

Across all categories, the top five influential factors identified were Equity, Depression, Family Status, Class, and Health.

Conclusion

By bridging the gap between high-accuracy deep learning and transparent social science, this research demonstrates that Shapley values offer a superior method for interpreting complex happiness models. While computational complexity remains a challenge, future work will focus on optimizing sampling methods to enable the analysis of even larger datasets for real-world policy decision-making.

Summary updated

Abstract

Evaluating the high-effect factors of citizens' happiness is beneficial to a wide range of policy-making for economics and politics in most countries. Benefiting from the high-efficiency of regression models, previous efforts by sociology scholars have analyzed the effect of happiness factors with high interpretability. However, restricted to their research concerns, they are specifically interested in some subset of factors modeled as linear functions. Recently, deep learning shows promising prediction accuracy while addressing challenges in interpretability. To this end, we introduce Shapley value that is inherent in solid theory for factor contribution interpretability to work with deep learning models by taking into account interactions between multiple factors. The proposed solution computes the Shapley value of a factor, i.e., its average contribution to the prediction in different coalitions based on coalitional game theory. Aiming to evaluate the interpretability quality of our solution, experiments are conducted on a Chinese General Social Survey (CGSS) questionnaire dataset. Through systematic reviews, the experimental results of Shapley value are highly consistent with academic studies in social science, which implies our solution for citizens' happiness prediction has 2-fold implications, theoretically and practically.
BibTeX
@inproceedings{li2022towards,
  author = {Lin Li and Xiaohua Wu and Miao Kong and Dong Zhou and Xiaohui Tao},
  title = {{Towards the Quantitative Interpretability Analysis of Citizens Happiness Prediction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/707},
}
Figure