TL;DR
This paper introduces a novel multidisciplinary framework that bridges the gap between social science and machine learning to investigate complex socio-technical issues. By explicitly mapping where human and algorithmic biases enter the research cycle, the authors demonstrate how to use "anchored" topic modeling and explainable AI (XAI) to uncover digital vulnerabilities within minoritised ethnic communities in the UK.
What Problem Does This Paper Address?
As essential services like social housing, health, and energy move online, there is a growing risk that "digital by default" policies will exacerbate existing inequalities. Investigating these "online harms" is difficult because social scientists (who understand human context) and computer scientists (who can scale analysis) often work in silos.
Furthermore, "big data" is frequently biased, and machine learning models are often "black boxes." This paper tackles the challenge of creating a transparent, reproducible pipeline where social science rigor (like inductive coding) informs ML models, and ML models, in turn, provide the scale and "explainability" needed to influence policy.
Methods
The authors propose a three-part framework: Data Collection, Data Analysis, and Verification.
- Inductive Coding (IC): Instead of starting with a theory, researchers manually labeled interview transcripts to let themes emerge naturally—a standard social science practice.
- Anchored Topic Modeling (CorEx): Unlike standard unsupervised models (like LDA or BERTopic), the authors used CorEx, which allows researchers to provide "seed words" (e.g., "housing," "health"). This ensures the machine-generated topics align with the research goals while still discovering new patterns.
- Sentiment Analysis with XAI: They fine-tuned a GPT-2 Large model to classify sentiment and then applied SHAP (SHapley Additive exPlanations). This allowed them to see exactly which words (like "difficult" or "speaking") were driving negative sentiment scores, making the model's "reasoning" transparent.
- Latent Class Analysis (LCA): A person-centered statistical method used to identify subgroups (latent classes) within larger populations based on their survey responses.
Datasets
The study utilized a mix of primary qualitative data and large-scale quantitative surveys:
- PRIME Interviews: 100 semi-structured interviews with individuals from minoritised ethnic (ME) communities in four UK cities (79 usable transcripts).
- PRIME Survey: A multilingual online survey with 878 respondents.
- EVENS Dataset: The "Evidence for Equality National Survey," a large-scale public dataset (4,601 data points) used to verify the findings from the smaller interview sample.
Key Insights and Ideas
- The "Bias Loop": The paper visualizes bias as a cyclic process. They use "lightening bolt" icons in their framework to show where personal bias (e.g., during interview translation) or algorithmic bias (e.g., biased training data) can creep in, and "inverted triangles" to show where researchers can mitigate it through reflexivity and triangulation.
- CorEx as the "Middle Ground": While BERTopic is popular in CS, the authors found CorEx more useful for multidisciplinary work. Its ability to integrate domain knowledge (via anchoring) made the results more interpretable for social scientists without sacrificing the power of ML.
- Ethnic Groups are Not a Monolith: The results from the Latent Class Analysis (LCA) proved that different ethnic groups face vastly different hurdles. For instance, Black African communities showed the highest relevance to social housing issues, while Bangladeshi groups presented unique clustering patterns in survey data that didn't match any other group.
Key Results
- Vulnerability Identification: The framework successfully identified four key vulnerabilities: discrimination, digital poverty, lack of digital literacy, and lack of English proficiency.
- Quantitative Accuracy: The fine-tuned sentiment analysis model achieved an overall accuracy of 72.88%, with a higher accuracy of 85.7% specifically within the language domain.
- The "Difficult" Factor: XAI word clouds revealed that the word "difficult" was the strongest predictor of negative sentiment across both digital literacy and language domains, highlighting a universal barrier to accessing digitalized services.
- Disproportionate Impact: The analysis showed that Black African communities were more likely to experience these vulnerabilities when trying to access digitalized social housing.
Limitations and Future Work
The authors acknowledge that their interview sample was gathered via convenience sampling, which introduces selection bias. Additionally, the sub-sample used for the deep-dive sentiment analysis (21 transcripts) was relatively small.
Future work involves taking these findings into co-design workshops with stakeholders—including service providers and policymakers—to create "harm-reducing tools" specifically designed for the needs of minoritised ethnic communities.
Summary updated