On the Fairness of Quality-based Data Markets

Dan Zhang, Hongzhi Wang, Xiaoou Ding, Yice Zhang, Jianzhong LiarXiv 2018 · arXiv

Abstract

In emerging online data markets, the value of a dataset is intrinsically linked to its quality. However, raw data often suffers from inconsistencies, missing values, and inaccuracies, necessitating expensive cleaning processes for the buyer. This paper proposes a novel pricing mechanism that adjusts costs based on multi-dimensional quality metrics. To prevent "savvy" buyers from exploiting these quality-based adjustments to obtain unfair discounts, the framework incorporates a cryptographic Fairness Assurance Mechanism. Experimental results on real-world datasets demonstrate that the system is both computationally efficient and resilient against market manipulation.

Introduction

As data becomes a primary commodity, the mechanisms for pricing it must evolve beyond simple volume-based models. In practice, data quality is highly variable. A buyer purchasing a list of university locations, for instance, may find missing entries or conflicting country codes. Intuitively, the price should reflect these deficiencies. However, integrating quality into pricing creates a transparency paradox: if a market reveals exactly how quality affects price, a rational buyer can probe the system to deduce the dataset's flaws and manipulate their stated preferences to minimize costs. This paper addresses the dual challenge of quality-driven pricing and market fairness.

The Challenge of Quality-Based Pricing

Data quality is rarely uniform. When datasets are aggregated from multiple sources, they often contain "dirty" records that require significant effort to clean.

Example of a dataset with various quality issues including inconsistent formats, missing values, and conflicting data.

As shown in the example above, a single table can exhibit multiple quality failures:

  • Inconsistency: The Apply_Deadline follows different date formats, and t1 and t3 list different Country_Code values for the same location.
  • Incompleteness: The deadline for t3 is missing entirely.
  • Accuracy Concerns: The Min_Score for t3 (3.5) is on a completely different scale than t1 and t2 (90 and 85), suggesting potential data entry errors.

If a buyer indicates they care deeply about completeness but less about consistency, a savvy buyer might realize that by claiming the opposite, they can receive the same data at a lower price if the dataset happens to be more consistent than it is complete.

A Multi-Dimensional Quality Framework

The proposed system evaluates data across four primary dimensions:

  1. Accuracy: The degree to which data correctly describes the real-world objects it represents.
  2. Completeness: The extent to which all required data is present.
  3. Timeliness: The degree to which data is up-to-date for the task at hand.
  4. Consistency: The absence of contradictions within the dataset (e.g., adhering to functional dependencies).

The Market Management System (MMS) calculates a "violation rate" for these metrics and adjusts a base price using a hybrid additive-multiplicative formula. This ensures that low-quality data acts as a financial offset, compensating the buyer for the cleaning labor they must perform.

Ensuring Fairness: The Cheat-Free Market

To neutralize the "savvy buyer" problem, the system employs cryptographic obfuscation. Instead of returning a single, transparent price, the MMS provides a price range and an encrypted final price.

Transactions are conducted using encrypted account balances. To ensure the market remains honest, a Trusted Third Party (TTP) is utilized. Through the mathematical properties of discrete logarithms and the Extended Euclidean algorithm, the buyer can verify that the amount deducted from their balance was correct and fell within the promised range, without ever learning the exact price or the underlying quality distribution that produced it. This "blind" transaction prevents buyers from reverse-engineering the pricing model.

Evaluation and Results

The framework was tested using five real-world datasets, including UK university locations and historical weather data. The evaluation focused on three key areas:

  • Efficiency: Preprocessing the entire database for quality took less than 5 seconds for all datasets, while individual query processing remained under 0.1 seconds, making it suitable for real-time markets.
  • Price Sensitivity: The system successfully lowered prices proportionally when errors were manually injected into the datasets.
  • Cheat Prevention: In controlled experiments, buyers attempting to game the system by tweaking their preference weights gained no statistical advantage over random weight selection, proving the effectiveness of the cryptographic hiding.

Conclusion

By treating data quality as a financial liability and protecting pricing logic through cryptography, this framework establishes a more equitable data economy. Future work will focus on moving from instance-based quality evaluation to dynamic, query-based evaluation and utilizing machine learning to automate the elicitation of buyer preferences.

Summary updated

Abstract

For data pricing, data quality is a factor that must be considered. To keep the fairness of data market from the aspect of data quality, we proposed a fair data market that considers data quality while pricing. To ensure fairness, we first design a quality-driven data pricing strategy. Then based on the strategy, a fairness assurance mechanism for quality-driven data marketplace is proposed. In this mechanism, we ensure that savvy consumers cannot cheat the system and users can verify each consumption with Trusted Third Party (TTP) that they are charged properly. Based on this mechanism, we develop a fair quality-driven data market system. Extensive experiments are performed to verify the effectiveness of proposed techniques. Experimental results show that our quality-driven data pricing strategy could assign a reasonable price to the data according to data quality and the fairness assurance mechanism could effectively protect quality-driven data pricing from potential cheating.
BibTeX
@misc{zhang2018fairness,
  author = {Dan Zhang and Hongzhi Wang and Xiaoou Ding and Yice Zhang and Jianzhong Li and Hong Gao},
  title = {{On the Fairness of Quality-based Data Markets}},
  year = {2018},
  eprint = {1808.01624},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1808.01624},
}
Figure