Abstract
In data engineering, the common assumption is that transparency—providing users with insights into how data is processed—automatically builds trust. This paper challenges that assumption by introducing a formal socio-technical trust model and a systematic design framework for data pipelines. Through a credit-scoring use case involving record linkage, the researchers demonstrate that transparent explanations do not always foster trust and can, in some cases, undermine technological optimism by exposing system limitations. The work advocates for a shift from "ad-hoc" trust to intentional, evidence-based trust design.
Introduction: The Transparency Myth
The data engineering community often treats trust as a byproduct of performance metrics like accuracy or precision. There is a prevailing belief that if a user understands the cleaning, integration, or transformation steps within a pipeline, they will naturally trust the output. However, this paper argues that trust is a complex human-centric relationship that is currently poorly defined and rarely measured systematically in technical contexts. As data pipelines increasingly drive critical decisions in finance and healthcare, relying on "trust by gut feeling" is no longer sufficient.
A Socio-Technical Model of Trust
To address this gap, the authors adapt the ABI model (Ability, Benevolence, Integrity) from philosophy and psychology to the data engineering domain. They define trust as a relationship where a human Truster accepts vulnerability to a Trustee. Crucially, the Trustee is not just a piece of software; it is a composite of the Data Engineering Technology Artifacts (DETAs), the developers and designers who created them, and the organization that deployed them.
In this model, trust is distinct from mere reliance. While one might rely on a tool because it functions consistently, one trusts a system when they are aware of the risks and potential for failure but choose to proceed anyway. This distinction is vital for designing systems that handle sensitive or high-stakes data.
Designing for Trustworthiness
The researchers propose a structured framework to integrate trust into the standard data engineering lifecycle. This involves three core phases:
- Identify Trust Scenarios and Breakpoints: Determining who the user is, what is at stake, and which specific pipeline stages (e.g., a complex data join) are most likely to trigger doubt.
- Devise a Trust Strategy: Using a six-dimensional inquiry (What, For what, Timeframe, Type, Presentation, and Requirements) to decide what evidence should be presented to the user.
- Evidence Management: Systematically collecting and distributing evidence of trustworthiness throughout the data lifecycle.
The Transparency Paradox: Key Results
The framework was evaluated using a record linkage use case for credit scoring. In a study with 19 participants, the researchers compared trust levels between a group provided with explanations for data matches and a control group without explanations.
The results revealed a "Transparency Paradox":
- No Significant Trust Boost: There was no statistically significant difference in perceived trustworthiness between the two groups.
- Decreased Technological Optimism: Participants shown explanations exhibited a statistically significant decrease (p=0.009) in technological optimism.
These findings suggest that providing more information can expose the "seams" of a system. If an explanation reveals that a match was made based on weak or questionable logic, the user’s trust may plummet rather than increase. Transparency is not a magic bullet; it can make users more critical of a system's inherent flaws.
Conclusion
This research highlights that trust in data engineering must be designed intentionally rather than assumed. Context is paramount; a data scientist may require raw SQL as evidence, while a bank clerk may need high-level semantic indicators. Future work will focus on refining these trust strategies to balance transparency, ensuring it provides enough evidence to be helpful without overwhelming the user or unnecessarily discouraging system use.
Summary updated