A Unifying Bias-aware Multidisciplinary Framework for Investigating Socio-Technical Issues

Sacha Hasan, Mehdi Rizvi, Yingfang Yuan, Chen Kefan, Lynne Baillie, Wei Pang

TL;DR

This paper introduces a novel multidisciplinary framework that bridges the gap between social science and machine learning to investigate complex socio-technical issues. By explicitly mapping where human and algorithmic biases enter the research cycle, the authors demonstrate how to use "anchored" topic modeling and explainable AI (XAI) to uncover digital vulnerabilities within minoritised ethnic communities in the UK.

What Problem Does This Paper Address?

As essential services like social housing, health, and energy move online, there is a growing risk that "digital by default" policies will exacerbate existing inequalities. Investigating these "online harms" is difficult because social scientists (who understand human context) and computer scientists (who can scale analysis) often work in silos.

Furthermore, "big data" is frequently biased, and machine learning models are often "black boxes." This paper tackles the challenge of creating a transparent, reproducible pipeline where social science rigor (like inductive coding) informs ML models, and ML models, in turn, provide the scale and "explainability" needed to influence policy.

Methods

The authors propose a three-part framework: Data Collection, Data Analysis, and Verification.

  • Inductive Coding (IC): Instead of starting with a theory, researchers manually labeled interview transcripts to let themes emerge naturally—a standard social science practice.
  • Anchored Topic Modeling (CorEx): Unlike standard unsupervised models (like LDA or BERTopic), the authors used CorEx, which allows researchers to provide "seed words" (e.g., "housing," "health"). This ensures the machine-generated topics align with the research goals while still discovering new patterns.
  • Sentiment Analysis with XAI: They fine-tuned a GPT-2 Large model to classify sentiment and then applied SHAP (SHapley Additive exPlanations). This allowed them to see exactly which words (like "difficult" or "speaking") were driving negative sentiment scores, making the model's "reasoning" transparent.
  • Latent Class Analysis (LCA): A person-centered statistical method used to identify subgroups (latent classes) within larger populations based on their survey responses.

Datasets

The study utilized a mix of primary qualitative data and large-scale quantitative surveys:

  1. PRIME Interviews: 100 semi-structured interviews with individuals from minoritised ethnic (ME) communities in four UK cities (79 usable transcripts).
  2. PRIME Survey: A multilingual online survey with 878 respondents.
  3. EVENS Dataset: The "Evidence for Equality National Survey," a large-scale public dataset (4,601 data points) used to verify the findings from the smaller interview sample.

Key Insights and Ideas

  • The "Bias Loop": The paper visualizes bias as a cyclic process. They use "lightening bolt" icons in their framework to show where personal bias (e.g., during interview translation) or algorithmic bias (e.g., biased training data) can creep in, and "inverted triangles" to show where researchers can mitigate it through reflexivity and triangulation.
  • CorEx as the "Middle Ground": While BERTopic is popular in CS, the authors found CorEx more useful for multidisciplinary work. Its ability to integrate domain knowledge (via anchoring) made the results more interpretable for social scientists without sacrificing the power of ML.
  • Ethnic Groups are Not a Monolith: The results from the Latent Class Analysis (LCA) proved that different ethnic groups face vastly different hurdles. For instance, Black African communities showed the highest relevance to social housing issues, while Bangladeshi groups presented unique clustering patterns in survey data that didn't match any other group.

Key Results

  • Vulnerability Identification: The framework successfully identified four key vulnerabilities: discrimination, digital poverty, lack of digital literacy, and lack of English proficiency.
  • Quantitative Accuracy: The fine-tuned sentiment analysis model achieved an overall accuracy of 72.88%, with a higher accuracy of 85.7% specifically within the language domain.
  • The "Difficult" Factor: XAI word clouds revealed that the word "difficult" was the strongest predictor of negative sentiment across both digital literacy and language domains, highlighting a universal barrier to accessing digitalized services.
  • Disproportionate Impact: The analysis showed that Black African communities were more likely to experience these vulnerabilities when trying to access digitalized social housing.

Limitations and Future Work

The authors acknowledge that their interview sample was gathered via convenience sampling, which introduces selection bias. Additionally, the sub-sample used for the deep-dive sentiment analysis (21 transcripts) was relatively small.

Future work involves taking these findings into co-design workshops with stakeholders—including service providers and policymakers—to create "harm-reducing tools" specifically designed for the needs of minoritised ethnic communities.

Summary updated

Abstract

This paper aims to bring together the disciplines of social science (SS) and computer science (CS) in the design and implementation of a novel multidisciplinary framework for systematic, transparent, ethically-informed, and bias-aware investigation of socio-technical issues. For this, various analysis approaches from social science and machine learning (ML) were applied in a structured sequence to arrive at an original methodology of identifying and quantifying objects of inquiry. A core feature of this framework is that it highlights where bias occurs and suggests possible steps to mitigate it. This is to improve the robustness, reliability, and explainability of the framework and its results. Such an approach also ensures that the investigation of socio-technical issues is transparent about its own limitations and potential sources of bias. To test our framework, we utilised it in the multidisciplinary investigation of the online harms encountered by minoritised ethnic (ME) communities when accessing and using digitalised social housing services in the UK. We draw our findings from 100 interviews with ME individuals in four cities across the UK to understand ME vulnerabilities when accessing and using digitalised social housing services. In our framework, a sub-sample of interviews focusing on ME individuals residing in social housing units were inductively coded. This resulted in the identification of the topics of discrimination, digital poverty, lack of digital literacy, and lack of English proficiency as key vulnerabilities of ME communities. Further ML techniques such as Topic Modelling and Sentiment Analysis were used within our framework where we found that Black African communities are more likely to experience these vulnerabilities in the access, use and outcome of digitalised social housing services.
BibTeX
@inproceedings{hasan2025unifying,
  author = {Sacha Hasan and Mehdi Rizvi and Yingfang Yuan and Chen Kefan and Lynne Baillie and Wei Pang},
  title = {{A Unifying Bias-aware Multidisciplinary Framework for Investigating Socio-Technical Issues}},
  booktitle = {Heriot-Watt University},
  year = {2025},
  url = {https://openreview.net/forum?id=pgFNO8vaqz},
}

Loading related content...

Figure