Probing Social Bias via Controlled Synthetic Generation

21 concepts17 papers1 proposal

Evaluating social bias in machine learning models has traditionally relied on static, human-annotated benchmarks. While valuable, these datasets suffer from fundamental limitations: they are expensive to produce, frequently risk violating user privacy when dealing with sensitive attributes, and quickly become obsolete as models evolve. More importantly, static datasets lack the granularity required to isolate the exact causes of biased behavior. If a model treats two demographic groups differently, is it due to historical prejudice, or is it a reflection of confounding variables like text sentiment or context?

To overcome these limitations, a powerful paradigm has emerged: controlled synthetic generation. Instead of collecting real-world data, researchers leverage generative models—such as Large Language Models (LLMs) and text-to-image diffusion models—as highly controllable laboratories. By systematically varying parameters, generating counterfactual pairs, or forcing models to reveal their internal default assumptions, this approach allows for a fine-grained, scalable, and causal analysis of algorithmic unfairness.

Causal Isolation via Counterfactual and Parameter-Varied Generation

The core intuition behind controlled synthetic generation is the ability to achieve causal isolation. In traditional datasets, demographic attributes are often entangled with other contextual factors. Synthetic generation allows researchers to hold the entire context completely constant while varying only the specific attribute under investigation.

Standard Prompt: "Write a story about a doctor." 
   └─► Model output often defaults to a specific gender/race.

Counterfactual Pairing:
   ├─► Prompt A: "Image of a female doctor in a hospital."
   └─► Prompt B: "Image of a male doctor in a hospital."
        └─► Directly isolates model behavior based on gender.

This counterfactual approach is particularly effective for uncovering intersectional biases, where multiple sensitive attributes (such as race and gender) interact. Howard et al. demonstrated this by using text-to-image diffusion models to generate large-scale, synthetic counterfactual image-text pairs. Because the visual context remains identical across variations, any discrepancy in how a downstream vision-language model evaluates these pairs can be directly attributed to intersectional bias.

Beyond binary counterfactuals, researchers can systematically vary continuous or categorical parameters to map out a model's behavioral landscape. For instance, Luo et al. prompted LLMs by systematically varying parameters such as restaurant category, price point, and sentiment. By comparing the resulting synthetic text to human-authored datasets, the authors isolated how models internalize status-based stereotypes and linguistic "othering" across different socioeconomic tiers. To formalize this level of control, Baumann 0002 et al. introduced a framework that utilizes structural equations to define explicit relationships between sensitive attributes and target outcomes. By adjusting specific parameters within these equations, researchers can "dial in" precise types of algorithmic bias (e.g., historical or measurement bias) to observe exactly how they propagate through a model.

Revealing Internal Defaults through Unconstrained Generation

While some frameworks feed synthetic data into a target model to audit its outputs, another facet of this research direction analyzes the synthetic data generated by a model to diagnose its internal worldview. When a generative model is given a neutral or unconstrained prompt, it must rely on its internal training distribution to fill in the blanks. The choices it makes reveal its implicit stereotypes.

A stark example of this was documented by Ducel et al., who prompted medical LLMs with gender-neutral instructions to generate clinical cases for various disorders. The authors found that for heart attacks, the models generated eight times more cases for male patients than female patients. By analyzing these generated distributions against real-world epidemiological data, this comparative generation strategy exposes how a model's internal associations override neutral instructions, reinforcing dangerous real-world stereotypes.

Scaling Naturalness: From Templates to LLM-Driven Synthesis

Early behavioral testing frameworks, such as CheckList, relied on rigid, human-designed templates to swap out target words (e.g., "I hate [identity_group]"). While useful for establishing baseline fairness metrics, templates lack the linguistic complexity and natural variation of real-world text. Marchiori Manerba et al. extended this behavioral testing concept by integrating custom synthetic datasets, demonstrating that realistic and diverse synthetic examples can reveal fine-grained biases in abusive language detection that aggregate metrics completely miss.

To scale this process without sacrificing naturalness, researchers now use high-capacity LLMs to automatically synthesize open-ended test cases.

  • AutoBiasTest: Kocielnik et al. and the authors developed a framework that leverages LLMs to automatically generate diverse, natural-sounding sentences for open-ended social bias testing. This eliminates the cost of human annotation while preserving linguistic fluidity.
  • BTC-SAM: Similarly, Kardkovács et al. applied LLM-driven generation to sentiment analysis models, demonstrating that automated test case generation uncovers subtle, contextual biases that traditional template-based methods fail to trigger.
  • SynthBias: To push models to their reasoning limits, Wang et al. introduced an automated benchmark synthesis method that generates tens of thousands of semantically complex test cases. This allows researchers to evaluate not just whether a model makes a biased decision, but whether it is "confidently wrong" due to underlying stereotypes.

Furthermore, this automated synthesis provides a critical privacy-preserving mechanism. In highly sensitive domains like online radical content detection, collecting real-world data poses severe ethical and legal challenges. Riabi et al. bypassed these constraints by using generative models to synthesize realistic user profiles and posts, allowing for controlled demographic bias testing without exposing or utilizing real user data.

Inverting the Probe: Conditional Generation and Canonical Sets

Most bias probing methods operate in a forward direction: change the input and observe the output. However, conditional generative architectures allow researchers to invert this process.

Algaba et al. introduced this inverse approach with LUCID-GAN. Instead of generating inputs to see what a model predicts, their framework generates synthetic data samples conditioned on a desired model prediction and specific feature values.

Forward Probing:  Input (Demographic) ──► Target Model ──► Output (Prediction)
Inverse Probing:  Desired Prediction  ──► LUCID-GAN    ──► Generated "Canonical Set"
                                                                │
                                                      (Reveal feature distributions)

By analyzing the distributions of protected features within these generated "canonical sets," researchers can visually and statistically audit what a black-box model "expects" an input to look like for a given outcome, providing a transparent look into the model's internal preferences.

Multi-Modal and Cross-Linguistic Expansion

The utility of controlled synthetic generation extends far beyond English text. It has become vital for auditing vision-language models (VLMs) and morphologically rich languages, where bias manifests in complex structural ways.

Vision-Language Auditing

In the visual domain, Saravanan et al. utilized synthetic images generated by text-to-image foundation models to represent intersectional social identities across downstream classification and editing tasks, revealing that optimizing for model performance often amplifies underlying stereotypes. To pinpoint specific behavioral flaws, Abdollahi et al. engineered specialized datasets using DALL-E 3 to isolate "Gender-Activity Binding" bias—the tendency of models to incorrectly associate specific actions with a gender when multiple individuals are present in an image.

Morphological and Reference-Free Evaluation

In natural language processing, languages with rich grammatical gender systems present unique evaluation challenges. Buleshnyi et al. addressed this by generating a synthetic Ukrainian dataset that explicitly varies masculine and feminine forms, pronouns, and professional experiences, providing a rigorous benchmark for hiring scenarios.

Finally, controlled synthesis is changing how we evaluate model outputs. Standard metrics like BLEU rely on rigid human reference strings, which fail to capture valid, inclusive language variations. To solve this, Piergentili et al. used synthetic datasets to train reference-free classification models. These models learn to evaluate whether a machine translation system successfully generates gender-neutral text based on structural properties rather than matching a specific human reference string.

By turning generative AI against itself, controlled synthetic generation provides the scalability, flexibility, and mathematical control necessary to transform bias probing from a reactive, aggregate-based exercise into a proactive, causal science.

Key Foundational Work

Fine-Grained Fairness Analysis of Abusive Language Detection Systems with CheckList (2021) — contributed 2 concept elements to this direction.
Analyzing and Reducing the Damage of Dataset Bias to Face Recognition With Synthetic Data. (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_controlled_bias_probing_8cfe9310,
  title = {controlled bias probing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/controlled-bias-probing/rdr_8cfe93104fc646fa9bc66bb7e67ac59f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_controlled_bias_probing_8cfe9310_v1,
  title = {controlled bias probing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/controlled-bias-probing/rdr_8cfe93104fc646fa9bc66bb7e67ac59f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake